Compare commits

..
Author SHA1 Message Date
AIOS Governed Continuation c8c25fa672 fix(discovery): make P0 contract hermetic
autonomy-smoke-tests phase1-p0 PASS evidence_sha256=13aff2dfe664be536470ca26a6ac3f20ba15fdca37cd4f8e799c0c8c36974be9
autonomy-unit-tests phase1-p0 PASS evidence_sha256=118708029df05d861585179a31e4f4eaf0d7953563f5347b76732f2055785ba0
campaign-approval-gate phase1-p0 PASS evidence_sha256=4e39d586b32ad2d17da8fc38f5d359ace8a239af8687b4f894a338fe9f0bf83c
git-diff-check phase1-p0 PASS evidence_sha256=2638b909fa281012b6a43aaf34e1d611b0dba1fc78bf47f54941739973ee6281
py-compile phase1-p0 PASS evidence_sha256=440cf54c4225f809a575eff9b24238acd361497fd18bffbb359087e88310dcd6
reconciliation-contract phase1-p0 PASS evidence_sha256=131fb1042ff1ceb565dbf1ec7aed1cd3879d728f912745c0589a02b96381baaf
release-state-machine phase1-p0 PASS evidence_sha256=a74b5192b248b6e342afeea01c85ecbb8a02d71891bcf334df2e89d96cc1577d
secret-scan phase1-p0 PASS evidence_sha256=d4789e5c45ba681c651c613427d70d911a4cdd70fbead0772c4e1e376bef41cd
signer-validator-contract phase1-p0 PASS evidence_sha256=a29c7664de7aa9e30e966f8c4a14c0eaf8191fbb92ba726a72ba3c7ec2659eec
2026-09-07 20:33:11 +08:00
AIOS Governed Continuation 1735371e06 feat(discovery): integrate P0 discovery with task admission bootstrap
autonomy-smoke-tests phase1-p0 PASS evidence_sha256=a96e50527a39e969b53df42ce799526faa8e3bcd618b6f5e2ec2e2e13c574f02
autonomy-unit-tests phase1-p0 FAIL evidence_sha256=d61da9d108d6ace7d845f78b1010773b4b1ce315a538209d7e60eb870c8eb953
campaign-approval-gate phase1-p0 PASS evidence_sha256=29d249ebfb389c3137e411c1d87fb88c7f755cdba4fea5118be187259c707295
git-diff-check phase1-p0 FAIL evidence_sha256=a0adbb54c3b220b9ffc82656470f1d67904823e461fb8ffedd713846814acc78
py-compile phase1-p0 PASS evidence_sha256=f504e34aa0a28ad009ba3b3fd3a75448297be43c452f4dc5ba445d3a4d0f7f07
reconciliation-contract phase1-p0 PASS evidence_sha256=8965ee765da46e32e79d046eeb57db023a2e9bd5105f057a27546b42eed470e3
release-state-machine phase1-p0 PASS evidence_sha256=e5a4c127d4b5b6eb27918fd9c54c06ee23d36d613bb816be475006c81b8ff479
secret-scan phase1-p0 PASS evidence_sha256=fc82389ca65008f045aa9357e5f276eeb0ce040ac5511197d65f1027975bc238
signer-validator-contract phase1-p0 PASS evidence_sha256=79a1d0246c73f0485816f77345fbc769c23f659cd177fefba90bb27e842214f3
2026-09-07 20:14:22 +08:00
AIOS Governed Continuation 447163fe82 test(security): allow exact historical idempotency fixture
autonomy-smoke-tests phase1-p0 PASS evidence_sha256=fdb08fc46698d7d38b3981df0e27bc5b5bb1da1c57f63fc756a4f08b649a132c
autonomy-unit-tests phase1-p0 PASS evidence_sha256=a7d74280ec71babbb7ff2824c0a8f1ec29db647478f395457979e02164360f9d
campaign-approval-gate phase1-p0 PASS evidence_sha256=d0d77e65dee6f2cf1303a1030473331754ab91982b830684284a562fe5320f0d
git-diff-check phase1-p0 PASS evidence_sha256=5e9628231a64e6faa04570e4817ff4109d791cea2838541994698847987983fb
py-compile phase1-p0 PASS evidence_sha256=47df225f475589b8d9c843ba6d49908c39ce7920890e8b2f355d24818d0558e0
reconciliation-contract phase1-p0 PASS evidence_sha256=24840ca69508034e002e4b12a5525822042bb01b5a6f73a697ca5fdbea30f8ec
release-state-machine phase1-p0 PASS evidence_sha256=805e5a17e6cfb8a2070a34c36e52c89808b89fd4055800dc1fa9b3d58c6c656b
secret-scan phase1-p0 PASS evidence_sha256=eb3dc75806b3af6c1963f1d284cb8b90971694356fdac63d2a09cd08972d8c74
signer-validator-contract phase1-p0 PASS evidence_sha256=da317f9fc7c8ce90b99417c4ec129ed5565c058ac96bb89e56a1a72ce5ab97b3
2026-09-07 18:00:05 +08:00
AIOS Governed Continuation 35f8d189e6 test(governed): normalize bootstrap contract line endings
autonomy-smoke-tests phase1-p0 PASS evidence_sha256=e174976180ecd717a6d24749c91cc53e35a573273de650004fb4a5b7983ccc23
autonomy-unit-tests phase1-p0 PASS evidence_sha256=45c6e4bd0d53ee3f0c64186f16cadbe13adf07b8d4fa24d4624ed90fc08468b2
campaign-approval-gate phase1-p0 PASS evidence_sha256=b182a5af7b5605da5bda1856d6f69774ec19305666fb563b6b4254b85eea167f
git-diff-check phase1-p0 PASS evidence_sha256=6b50f3b275db390e111297b3780a66a97269aa102317dae7584f9987fbd72981
py-compile phase1-p0 PASS evidence_sha256=c82d4aedfb4c752b67467a25ea9f8b9f75f0bea008553abcbcaa78f03751f164
reconciliation-contract phase1-p0 PASS evidence_sha256=cd0071eb74bc0606654eea7159ab835a6ea087a24ce594657a0d7b286206cb59
release-state-machine phase1-p0 PASS evidence_sha256=d4b8b0d8838fc523eaab64dbb4618971b341148f068c1da2395c3955d6fd6e23
secret-scan phase1-p0 FAIL evidence_sha256=47d13c1b7d3c967277c56ce2c62c12bd2b9f221e1e42cbb38461c44574f0919f
signer-validator-contract phase1-p0 PASS evidence_sha256=cdb8009765558b5acd2c3de15bdf5681814e64b2c1587676a5528be5bbca1e95
2026-09-07 17:40:07 +08:00
AIOS Governed Continuation 6d4f79a8c2 fix(governed): make P0 task admission self-contained 2026-09-07 15:22:20 +08:00
AIOS Governed Continuation 1efa7724c1 feat(discovery): governed immutable repository discovery P0 2026-09-04 00:16:46 +08:00
AIOS Governed Continuation a7e835b9d3 feat(governed): close P0 task admission bootstrap 2026-09-03 21:40:32 +08:00
AIOS Governed Continuation 91f104ef62 fix(gitops): repair trusted-builder receipt replay framing
autonomy-smoke-tests focused receipt/replay and Task lifecycle smoke: PASS
autonomy-unit-tests full Linux regression: 834 passed, 3 skipped, 154 subtests passed
campaign-approval-gate standing Owner V15 exact Repair Task authorization and formal Task verification: PASS
git-diff-check exact two-file governed diff and whitespace check: PASS
py-compile active-runtime and tests compileall: PASS
reconciliation-contract WriterLease completed replay and governed branch convergence: PASS
release-state-machine full Linux regression includes release state-machine contract: PASS; no release mutation
secret-scan exact changed-file high-signal credential/private-key scan: PASS
signer-validator-contract receipt framing/replay/adoption contract: 18 focused tests PASS; private key non-exportable
2026-08-30 00:05:00 +08:00
owner f8ad8889a4 Merge pull request 'fix(gitops): bind trusted-builder provisioning to final common main' (#71) from agent/v15-post-pr70-builder-release-readiness-20260829 into main
Reviewed-on: #71
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 21:39:20 +08:00
AIOS Governed Continuation 36ffb00ae9 fix(gitops): bind trusted-builder provisioning to final common main
autonomy-smoke-tests authorized task-writer smoke: PASS
autonomy-unit-tests isolated unit suite: 28 passed; 2 opt-in skipped
campaign-approval-gate standing Owner V15 authorization bound and verified
git-diff-check exact four-file diff and whitespace check: PASS
py-compile trusted-builder provisioner compile: PASS
reconciliation-contract authority/Gitea Git/API durable receipt convergence: PASS
release-state-machine release workflow state unit contract: PASS
secret-scan exact changeset credential and private-key scan: PASS
signer-validator-contract canonical signer compatibility/self-test contract: PASS
2026-08-29 20:53:18 +08:00
owner c2db4ade3f Merge pull request 'feat(release): add purpose-bound trusted-builder signer provisioner' (#70) from agent/v15-trusted-builder-signer-provisioning-20260829 into main
Reviewed-on: #70
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 17:23:29 +08:00
AIOS Governed Continuation d58227f637 feat(release): add purpose-bound trusted-builder signer provisioner
py-compile Python compileall PASS on exact PR head
git-diff-check git diff-tree --check PASS on exact PR head
secret-scan Exact changed-file secret scan PASS; no credential material
signer-validator-contract Trusted-builder signer provisioner contract suite PASS; live signer remains intentionally unprovisioned
autonomy-unit-tests 821 passed, 3 skipped, 154 subtests passed
autonomy-smoke-tests Full repository source regression and smoke contract suite PASS
reconciliation-contract Full repository regression includes reconciliation/replay contract PASS
release-state-machine Full repository regression includes release controller/state-machine contract PASS; no production mutation
campaign-approval-gate Source governance and campaign gate contract PASS; candidate creation remains post-merge
2026-08-29 17:02:00 +08:00
owner c391d5baf7 Merge pull request 'fix(release): bind trusted builder to canonical purpose' (#69) from agent/v15-trusted-builder-purpose-repair-r3-20260829 into main
Reviewed-on: #69
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 15:54:53 +08:00
AIOS Governed Continuation 98bdeae6b4 fix(release): bind trusted builder to canonical purpose
git-diff-check git diff --check PASS on exact PR head
py-compile Python compileall PASS on active-runtime/app
reconciliation-contract Full source regression includes completed-replay/reconciliation contract PASS
secret-scan Diff-added lines secret scan PASS; no credential material
signer-validator-contract Signer contract suite PASS; live signer remains opt-in and no key was provisioned
campaign-approval-gate V2R2C2 trusted-controller source binding gate PASS; durable candidate creation remains post-merge
release-state-machine V2R2C2 release-controller source contract PASS; no production mutation enabled
autonomy-smoke-tests Source snapshot smoke/contract suite passed; production mutations disabled
autonomy-unit-tests 816 passed, 3 skipped, 154 subtests passed
2026-08-29 15:25:31 +08:00
owner 8e90f317b3 Merge pull request 'feat(v2r2c2): activate durable trusted release controller' (#68) from agent/v12-trusted-release-controller-activation-20260829 into main
Reviewed-on: #68
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 13:11:38 +08:00
AIOS Governed Continuation 98fc05d761 style(v2r2c2): normalize source eof
autonomy-smoke-tests Source snapshot smoke/contract suite passed; production mutations disabled
autonomy-unit-tests 816 passed, 3 skipped, 154 subtests passed
git-diff-check git diff --check PASS on exact PR head
py-compile Python compileall PASS on active-runtime/app
reconciliation-contract Full source regression includes completed-replay/reconciliation contract PASS
secret-scan Diff-added lines secret scan PASS; no credential material
signer-validator-contract Signer contract suite PASS; live signer remains opt-in and no key was provisioned
campaign-approval-gate V2R2C2 trusted-controller source binding gate PASS; durable candidate creation remains post-merge
release-state-machine V2R2C2 release-controller source contract PASS; no production mutation enabled
2026-08-29 13:06:09 +08:00
AIOS Governed Continuation c82fa9dad8 test(v2r2c2): update canonical release route assertions 2026-08-29 13:03:33 +08:00
AIOS Governed Continuation 6499b61a86 test(v2r2c2): align release compatibility contracts 2026-08-29 13:01:43 +08:00
AIOS Governed Continuation 54c8e47b2e fix(v2r2c2): preserve release query compatibility 2026-08-29 12:58:43 +08:00
AIOS Governed Continuation 9718109977 feat(v2r2c2): activate trusted candidate builder 2026-08-29 12:52:24 +08:00
owner d2b5a8339a Merge pull request 'fix(v2r2c2): bind trusted production runtime payload' (#67) from agent/v2r2c2-release-runtime-binding-20260829 into main
Reviewed-on: #67
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 10:16:33 +08:00
AIOS Governed Continuation 0d9d0479bf feat(v2r2c2): bind trusted release runtime and deterministic artifact
autonomy-smoke-tests V2R2C2 release-runtime binding smoke and offline package preflight PASS
autonomy-unit-tests 782 passed, 3 skipped, 154 subtests passed on source branch
git-diff-check governed source diff-check PASS against merged main f1695ce
py-compile all tracked Python source compile PASS
reconciliation-contract completed-replay reconciliation contract preserved; 11 focused tests PASS
secret-scan narrow source and deployable payload secret scan PASS; no credentials included
signer-validator-contract trusted release plan/adapter contract 48 tests PASS; formal Owner signature not asserted here
campaign-approval-gate V2R2C2 candidate binding contract PASS; production approval remains formal-envelope gated
release-state-machine V2R2C2 plan-driven release state contract PASS; no production mutation performed
2026-08-29 09:37:46 +08:00
owner f1695cebf6 Merge pull request 'feat(v2r2c2): integrate completed-replay reconciliation' (#66) from agent/v2r2c2-release-source-integration-20260829 into main
Reviewed-on: #66
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 09:13:44 +08:00
AIOS Governed Continuation 8427c7bf22 test(release): cover trusted plan dispatch and adapter binding
autonomy-smoke-tests V2R2C2 trusted-plan/runtime contract smoke passed; 67 focused tests
autonomy-unit-tests governed source full Linux suite: 781 passed, 3 skipped, 154 subtests
git-diff-check final PR head diff-check passed against protected main
py-compile all tracked Python sources compile in memory without errors
reconciliation-contract completed-replay reconciliation contract: 11 passed
secret-scan final PR diff high-signal secret scan found no credentials
signer-validator-contract signer/validator contract: 5 passed, 2 explicit environment skips
campaign-approval-gate trusted V2R2C2 release-plan dispatch and exact binding contract passed
release-state-machine plan-driven release handler and generic adapter contract passed; legacy path isolated
2026-08-29 09:10:05 +08:00
AIOS Governed Continuation cdacdfc1b7 feat(release): bind new release execution to trusted plan 2026-08-29 09:09:02 +08:00
AIOS Governed Continuation e5b466ed2e test(signer): skip root-only live self-test when unprivileged 2026-08-29 08:54:44 +08:00
AIOS Governed Continuation 4a1433c306 fix(release): enforce candidate identity in trusted plan binding 2026-08-29 08:51:50 +08:00
AIOS Governed Continuation 5cd992e211 test(release): bind candidate identity in trusted plan fixture 2026-08-29 08:50:37 +08:00
AIOS Governed Continuation 036c72b553 test(release): cover trusted plan tamper and binding denial 2026-08-29 08:49:23 +08:00
AIOS Governed Continuation c8796a47dc feat(release): bind campaign acceptance and executor to trusted plan 2026-08-29 08:48:08 +08:00
AIOS Governed Continuation e13a3e4a3e feat(release): add canonical trusted release execution plan 2026-08-29 08:45:08 +08:00
AIOS Governed Continuation 44d5f7a3c1 fix(v2r2c2): normalize staging adapter file ending
autonomy-smoke-tests V2R2C2 candidate staging smoke and health projection passed on frozen candidate
autonomy-unit-tests governed source full Linux suite: 775 passed, 2 skipped, 154 subtests
git-diff-check PR head diff-check passed after governed whitespace normalization
py-compile full source tree compileall passed with repository PYTHONPATH
reconciliation-contract completed-replay reconciliation focused tests passed: 18 passed
secret-scan immutable candidate/package secret preflight passed for governed source payload
signer-validator-contract signer/validator contract included in full suite and passed
campaign-approval-gate blocked: required V2R2C2 release binding and final release evidence are not present
release-state-machine blocked: current release handler/executor is static V2R2C1; no legal V2R2C2 binding
2026-08-29 05:59:23 +08:00
AIOS Governed Continuation 7bd493d449 fix(v2r2c2): restore release version validation block indentation 2026-08-29 05:58:34 +08:00
AIOS Governed Continuation 1fae628eab feat(v2r2c2): publish completed-replay reconciliation source
aios/v2r2c2-source-regression V2R2C2 governed source integration exact-head checks passed: 1fae628eab849cf7cccf9711ca5a014560efccf2
2026-08-29 05:17:51 +08:00
AIOS Governed Continuation 1d35a7c0cd fix: restore signed deterministic recovery and bridge sandbox contract 2026-08-28 13:44:37 +08:00
AIOS governed merge broker ece82ad789 Merge commit '348c6db9a9c73464d5d39b65e4c4291ec4e99ba6' into HEAD 2026-08-26 17:53:33 +08:00
aios-autonomy-agent 348c6db9a9 feat(openbao): add autonomy runtime v1 candidate 2026-08-26 16:51:07 +08:00
AIOS Governed Continuation 2afc2cf41f docs: add unattended V2R2C2 convergence execution contract 2026-08-25 19:04:51 +08:00
AIOS Governed Continuation 7526fef619 test: cover convergence acceptance ownership and continuation 2026-08-25 18:57:05 +08:00
AIOS Governed Continuation c121527194 feat: add minimal AIOS convergence contracts 2026-08-25 18:54:47 +08:00
AIOS Owner Reconciliation 174e64dd02 fix(action-surface): bind governed reads to canonical release bridge 2026-08-25 18:17:42 +08:00
AIOS Owner Reconciliation d86e900332 docs: record restart persistence evidence 2026-08-25 16:30:54 +08:00
AIOS Owner Reconciliation 353f524a53 docs: record action contract staging repair evidence 2026-08-25 16:29:11 +08:00
AIOS Owner Reconciliation 2ec49005bd fix: wire staging release projections to runtime inputs 2026-08-25 16:17:23 +08:00
AIOS Owner Reconciliation 304bf174ae fix: bind release checks to staging baseline authority 2026-08-25 16:13:06 +08:00
AIOS Owner Reconciliation 92706e387e fix: reconcile authority-only isolated branch 2026-08-25 16:01:46 +08:00
AIOS Owner Reconciliation dfc37560e6 fix: reconcile absent remote branch creation 2026-08-25 15:59:30 +08:00
AIOS Owner Reconciliation 108aca1264 fix: preserve remote-create mode during durable reconcile 2026-08-25 15:57:14 +08:00
AIOS Owner Reconciliation 20462e58bb fix: recover initial remote branch push during reconcile 2026-08-25 15:56:36 +08:00
AIOS Owner Reconciliation 73e02c2324 fix: permit governed staging branch push 2026-08-25 15:54:27 +08:00
AIOS Owner Reconciliation 27f4be40e4 fix: make governed create-new lease replay idempotent 2026-08-25 15:45:46 +08:00
AIOS Owner Reconciliation c6d9ed4873 fix: normalize task branch refs in authority CAS 2026-08-25 15:42:10 +08:00
AIOS Owner Reconciliation c9aaa19873 fix: initialize governed create-new branch through authority CAS 2026-08-25 15:38:18 +08:00
AIOS Owner Reconciliation 1d073f8400 fix: include canonical V2R2C1 staging registry projection 2026-08-25 15:22:21 +08:00
AIOS Owner Reconciliation 79a1ad1f59 fix: close staging baseline and empty-state contracts 2026-08-25 15:18:28 +08:00
AIOS Owner Reconciliation 1d9b2ffef1 fix: repair staging action contracts and release overlay 2026-08-25 15:02:10 +08:00
AIOS Governed Continuation 35eec9afe1 runbook: add unattended agent execution contract for Action repair 2026-08-25 14:24:41 +08:00
AIOS Governed Continuation efcae0a58c docs: capture Action regression remediation and acceptance gates 2026-08-25 14:19:00 +08:00
151 changed files with 18810 additions and 12735 deletions
+10
View File
@@ -0,0 +1,10 @@
title = "AIOS repository secret scan policy"
[extend]
useDefault = true
[allowlist]
description = "Exact non-secret Task Admission test idempotency fixture in the historical PR commit"
condition = "AND"
paths = ['''tests/test_task_admission_bootstrap_http\.py$''']
regexes = ['''idempotency-1''']
+7 -14
View File
@@ -13,7 +13,6 @@ from hashlib import sha256
from pathlib import Path
import json
from typing import Any
from message_handoff import stable_event_id
def utc_now() -> str:
@@ -56,19 +55,13 @@ class AgentEventBus:
payload: dict[str, Any],
correlation_id: str | None = None,
) -> dict[str, Any]:
event_id = stable_event_id(
source=source, target=target, message_type=kind, payload=payload,
tenant_id=(payload.get('tenant_id') if isinstance(payload, dict) else None),
user_id=(payload.get('user_id') if isinstance(payload, dict) else None),
project_id=(payload.get('project_id') if isinstance(payload, dict) else None),
conversation_id=(payload.get('conversation_id') if isinstance(payload, dict) else None),
task_id=(payload.get('task_id') if isinstance(payload, dict) else None),
run_id=(payload.get('run_id') if isinstance(payload, dict) else None),
correlation_id=(payload.get('correlation_id') if isinstance(payload, dict) else None),
causation_id=(payload.get('causation_id') if isinstance(payload, dict) else None),
sequence=(payload.get('sequence') if isinstance(payload, dict) else None),
idempotency_key=(payload.get('idempotency_key') if isinstance(payload, dict) else None),
)
event_id = "evt_" + sha256(
json.dumps(
{"kind": kind, "source": source, "summary": summary, "payload": payload},
sort_keys=True,
ensure_ascii=False,
).encode("utf-8")
).hexdigest()[:16]
event = AgentEvent(event_id, kind, source, target, summary, payload, correlation_id=correlation_id)
self._write("pending", event.to_dict())
self._audit("created", event.to_dict())
@@ -7,7 +7,7 @@ class AgentEventObservabilityBridge:
def __init__(self, hooks: AgentObservabilityHooks | None = None):
self.hooks = hooks or AgentObservabilityHooks()
def consume(self, event: dict, *, actor=None, trace=None):
def consume(self, event: dict):
kind = event.get("kind")
payload = event.get("payload", {})
if kind == "agent_started":
@@ -16,8 +16,6 @@ class AgentEventObservabilityBridge:
visible_scope=payload.get("scope", "self"),
entry_agent=payload.get("agent", "unknown"),
execution_id=event.get("id"),
actor=actor,
trace=trace,
)
if kind == "agent_delegated":
return self.hooks.delegate(event["execution_id"], payload.get("agent", "unknown"))
-12
View File
@@ -31,18 +31,6 @@ class AgentExecutionRecord:
status: str = "completed"
started_at: str = field(default_factory=utc_now)
ended_at: str = field(default_factory=utc_now)
trace_id: str | None = None
span_id: str | None = None
parent_span_id: str | None = None
correlation_id: str | None = None
causation_id: str | None = None
tenant_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
task_id: str | None = None
claim_id: str | None = None
run_id: str | None = None
message_id: str | None = None
def to_dict(self):
return asdict(self)
+6 -30
View File
@@ -9,49 +9,25 @@ try:
except Exception:
APIRouter = None
from .agent_observability import AgentExecutionRecord, AgentObservability
from .agent_observability import AgentExecutionRecord
from .agent_observability_dashboard import AgentObservabilityDashboard
from .agent_observability_store import AgentObservabilityStore
from .trace_tool_credential_governance import GovernanceActor, TraceContext, authorize_scope
store = AgentObservabilityStore()
dashboard = AgentObservabilityDashboard(store.load())
def record_agent_execution(payload: dict[str, Any], *, actor: GovernanceActor | dict[str, Any] | None = None, trace: TraceContext | None = None):
scoped = any(payload.get(name) is not None for name in (
"trace_id", "span_id", "tenant_id", "project_id", "conversation_id", "task_id", "run_id"
))
values = dict(payload)
if scoped:
if actor is None or trace is None:
raise RuntimeError("TRACE_ACTOR_CONTEXT_REQUIRED")
trusted = GovernanceActor.from_value(actor)
authorize_scope(trusted, trace.scope)
bound = {
"trace_id": trace.trace_id, "span_id": trace.span_id,
"parent_span_id": trace.parent_span_id, "correlation_id": trace.correlation_id,
"causation_id": trace.causation_id, "tenant_id": trace.scope.tenant_id,
"user_id": trace.scope.user_id, "project_id": trace.scope.project_id,
"conversation_id": trace.scope.conversation_id, "task_id": trace.task_id,
"claim_id": trace.claim_id, "run_id": trace.scope.run_id, "message_id": trace.message_id,
}
for name, authoritative in bound.items():
if values.get(name) not in (None, authoritative):
raise RuntimeError("TRACE_SCOPE_MISMATCH_DENY:" + name)
values[name] = authoritative
record = AgentExecutionRecord(**values)
def record_agent_execution(payload: dict[str, Any]):
record = AgentExecutionRecord(**payload)
return {"status": "ok", "record": store.append(record)}
def get_agent_observability(user_id: str | None = None, group_id: str | None = None, role: str = "user", *, actor: GovernanceActor | dict[str, Any] | None = None):
records = store.query(user_id, group_id, role, actor=actor)
visible = AgentObservability()
visible.records = [AgentExecutionRecord(**record) for record in records]
def get_agent_observability(user_id: str | None = None, group_id: str | None = None, role: str = "user"):
records = store.query(user_id, group_id, role)
return {
"status": "ok",
"records": records,
"summary": visible.summary(None),
"summary": store.load().summary(user_id if role != "owner" else None),
}
@@ -14,11 +14,6 @@ from uuid import uuid4
from .agent_observability import AgentExecutionRecord
from .agent_observability_store import AgentObservabilityStore
from .trace_tool_credential_governance import (
GovernanceActor,
TraceContext,
authorize_scope,
)
def utc_now() -> str:
@@ -41,7 +36,6 @@ class AgentRunContext:
input_tokens: int = 0
output_tokens: int = 0
status: str = "running"
trace: TraceContext | None = None
def delegate(self, agent_name: str) -> None:
if agent_name not in self.path:
@@ -65,19 +59,7 @@ class AgentObservabilityHooks:
visible_scope: str,
entry_agent: str,
execution_id: str | None = None,
*,
actor: GovernanceActor | dict[str, Any] | None = None,
trace: TraceContext | None = None,
) -> dict[str, Any]:
if execution_id is not None and execution_id in self.active:
raise RuntimeError("OBSERVABILITY_EXECUTION_CONFLICT")
if trace is not None:
if actor is None:
raise RuntimeError("TRACE_ACTOR_CONTEXT_REQUIRED")
trusted_actor = GovernanceActor.from_value(actor)
authorize_scope(trusted_actor, trace.scope)
if trace.scope.user_id is not None and trace.scope.user_id != user_id:
raise RuntimeError("TRACE_SCOPE_MISMATCH_DENY:user_id")
run = AgentRunContext(
execution_id=execution_id or f"exec_{uuid4().hex[:12]}",
user_id=user_id,
@@ -85,7 +67,6 @@ class AgentObservabilityHooks:
entry_agent=entry_agent,
path=[entry_agent],
used_hermes=entry_agent.lower() == "hermes",
trace=trace,
)
self.active[run.execution_id] = run
return {"status": "started", "execution_id": run.execution_id, "run": run.__dict__}
@@ -125,18 +106,6 @@ class AgentObservabilityHooks:
status=status,
started_at=run.started_at,
ended_at=ended_at,
trace_id=run.trace.trace_id if run.trace else None,
span_id=run.trace.span_id if run.trace else None,
parent_span_id=run.trace.parent_span_id if run.trace else None,
correlation_id=run.trace.correlation_id if run.trace else None,
causation_id=run.trace.causation_id if run.trace else None,
tenant_id=run.trace.scope.tenant_id if run.trace else None,
project_id=run.trace.scope.project_id if run.trace else None,
conversation_id=run.trace.scope.conversation_id if run.trace else None,
task_id=run.trace.task_id if run.trace else None,
claim_id=run.trace.claim_id if run.trace else None,
run_id=run.trace.scope.run_id if run.trace else None,
message_id=run.trace.message_id if run.trace else None,
)
return {"status": "stored", "record": self.store.append(record)}
@@ -12,12 +12,6 @@ from pathlib import Path
from typing import Any
from .agent_observability import AgentExecutionRecord, AgentObservability
from .trace_tool_credential_governance import (
GovernanceActor,
GovernanceError,
GovernanceScope,
authorize_scope,
)
class AgentObservabilityStore:
@@ -29,11 +23,6 @@ class AgentObservabilityStore:
def append(self, record: AgentExecutionRecord) -> dict[str, Any]:
path = self.records_dir / f"{record.execution_id}.json"
data = record.to_dict()
if path.exists():
existing = json.loads(path.read_text(encoding="utf-8"))
if existing == data:
return existing
raise RuntimeError("OBSERVABILITY_EXECUTION_CONFLICT")
path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")
return data
@@ -44,36 +33,8 @@ class AgentObservabilityStore:
observability.records.append(AgentExecutionRecord(**raw))
return observability
def query(self, user_id: str | None = None, group_id: str | None = None, role: str = "user", *, actor: GovernanceActor | dict[str, Any] | None = None) -> list[dict[str, Any]]:
def query(self, user_id: str | None = None, group_id: str | None = None, role: str = "user") -> list[dict[str, Any]]:
records = [record.to_dict() for record in self.load().records]
scoped = [record for record in records if record.get("tenant_id") is not None]
legacy = [record for record in records if record.get("tenant_id") is None]
if actor is not None:
trusted = GovernanceActor.from_value(actor)
visible: list[dict[str, Any]] = []
for record in scoped:
try:
authorize_scope(
trusted,
GovernanceScope(
"tenant",
tenant_id=record.get("tenant_id"),
user_id=record.get("user_id"),
project_id=record.get("project_id"),
conversation_id=record.get("conversation_id"),
run_id=record.get("run_id"),
),
)
except GovernanceError:
continue
visible.append(record)
if trusted.privileged:
visible.extend(legacy)
else:
visible.extend(record for record in legacy if record.get("user_id") == trusted.scope.user_id)
return visible
# Caller-supplied role remains a legacy compatibility path only.
records = legacy
if role == "owner":
return records
if role == "group_admin" and group_id:
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,811 @@
"""Canonical Task projection intent and same-identity recovery primitives.
This module is intentionally Task-Plane-owned. It does not open a network
listener and it does not authenticate callers; existing control-plane services
must enforce their normal service-side ``agent-task:control`` boundary before
calling it.
The recovery primitive has one purpose: converge a durable, complete
ProjectionIntent to exactly one canonical Task row. It never fabricates task
semantics from a Bridge binding and it never generates a replacement task id.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass
from datetime import datetime, timezone
from enum import Enum
import hashlib
import json
from pathlib import Path
import re
import sqlite3
from typing import Any, Mapping, Sequence
TASK_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{2,179}$")
JOB_ID_RE = TASK_ID_RE
SHA40_RE = re.compile(r"^[0-9a-f]{40}$")
SHA64_RE = re.compile(r"^[0-9a-f]{64}$")
BRANCH_RE = re.compile(r"^agent/[A-Za-z0-9._/-]{1,220}$")
SCHEMA = "aios.canonical-task-projection-intent.v1"
RECEIPT_SCHEMA = "aios.canonical-task-projection-receipt.v1"
class ProjectionRecoveryError(RuntimeError):
def __init__(self, code: str, status: int = 409) -> None:
super().__init__(code)
self.code = code
self.status = status
class ProjectionState(str, Enum):
PENDING = "PENDING"
CONVERGED = "CONVERGED"
DRIFT = "DRIFT"
FAILED = "FAILED"
BLOCKED = "BLOCKED"
UNKNOWN = "UNKNOWN"
BACKFILL_REQUIRED = "BACKFILL_REQUIRED"
def _now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _canonical(value: object) -> bytes:
return json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
def sha256_json(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _string(value: object, code: str, pattern: re.Pattern[str] | None = None) -> str:
if not isinstance(value, str) or not value.strip():
raise ProjectionRecoveryError(code, 400)
result = value.strip()
if pattern is not None and pattern.fullmatch(result) is None:
raise ProjectionRecoveryError(code, 400)
return result
def _integer(value: object, code: str, *, minimum: int = 0) -> int:
if isinstance(value, bool) or not isinstance(value, int) or value < minimum:
raise ProjectionRecoveryError(code, 400)
return value
def _plain_mapping(value: object, code: str) -> dict[str, Any]:
if not isinstance(value, Mapping):
raise ProjectionRecoveryError(code, 400)
return {str(k): v for k, v in value.items()}
def _plain_list(value: object, code: str) -> list[Any]:
if not isinstance(value, list):
raise ProjectionRecoveryError(code, 400)
return list(value)
@dataclass(frozen=True, slots=True)
class CanonicalTaskPayload:
task_id: str
goal: str
instructions: str
success_criteria: tuple[Mapping[str, Any], ...]
constraints: Mapping[str, Any]
required_capabilities: tuple[str, ...]
metadata: Mapping[str, Any]
priority: int = 100
max_attempts: int = 20
deadline: str | None = None
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "CanonicalTaskPayload":
allowed = {
"task_id",
"goal",
"instructions",
"success_criteria",
"constraints",
"required_capabilities",
"metadata",
"priority",
"max_attempts",
"deadline",
}
if set(value) - allowed:
raise ProjectionRecoveryError("TASK_PAYLOAD_UNKNOWN_FIELDS", 400)
task_id = _string(value.get("task_id"), "TASK_ID_INVALID", TASK_ID_RE)
goal = _string(value.get("goal"), "TASK_GOAL_INVALID")
instructions = _string(value.get("instructions"), "TASK_INSTRUCTIONS_INVALID")
if len(goal) > 8192 or len(instructions) > 65536:
raise ProjectionRecoveryError("TASK_TEXT_TOO_LARGE", 413)
criteria_raw = _plain_list(value.get("success_criteria"), "TASK_SUCCESS_CRITERIA_INVALID")
criteria: list[Mapping[str, Any]] = []
for item in criteria_raw:
if not isinstance(item, Mapping):
raise ProjectionRecoveryError("TASK_SUCCESS_CRITERIA_INVALID", 400)
criteria.append(dict(item))
if not criteria:
raise ProjectionRecoveryError("TASK_SUCCESS_CRITERIA_REQUIRED", 400)
constraints = _plain_mapping(value.get("constraints"), "TASK_CONSTRAINTS_INVALID")
capabilities_raw = _plain_list(
value.get("required_capabilities", []),
"TASK_CAPABILITIES_INVALID",
)
capabilities: list[str] = []
for item in capabilities_raw:
capabilities.append(_string(item, "TASK_CAPABILITY_INVALID"))
# Recovery recreates the pristine canonical row. Initial execution
# projection remains the sole owner of runtime capabilities.
if capabilities:
raise ProjectionRecoveryError(
"TASK_RECOVERY_REQUIRES_PRISTINE_CAPABILITIES",
409,
)
metadata = _plain_mapping(value.get("metadata", {}), "TASK_METADATA_INVALID")
if "execution" in metadata or "execution_projection" in metadata:
raise ProjectionRecoveryError(
"TASK_RECOVERY_REQUIRES_PRISTINE_METADATA",
409,
)
priority = _integer(value.get("priority", 100), "TASK_PRIORITY_INVALID", minimum=0)
max_attempts = _integer(
value.get("max_attempts", 20),
"TASK_MAX_ATTEMPTS_INVALID",
minimum=1,
)
deadline = value.get("deadline")
if deadline is not None:
deadline = _string(deadline, "TASK_DEADLINE_INVALID")
return cls(
task_id=task_id,
goal=goal,
instructions=instructions,
success_criteria=tuple(criteria),
constraints=constraints,
required_capabilities=tuple(capabilities),
metadata=metadata,
priority=priority,
max_attempts=max_attempts,
deadline=deadline,
)
def public(self) -> dict[str, Any]:
return {
"task_id": self.task_id,
"goal": self.goal,
"instructions": self.instructions,
"success_criteria": [dict(x) for x in self.success_criteria],
"constraints": dict(self.constraints),
"required_capabilities": list(self.required_capabilities),
"metadata": dict(self.metadata),
"priority": self.priority,
"max_attempts": self.max_attempts,
"deadline": self.deadline,
}
@property
def payload_sha256(self) -> str:
return sha256_json(self.public())
@dataclass(frozen=True, slots=True)
class GovernedProjectionBinding:
governed_task_id: str
bridge_job_id: str
repository: str
branch: str
generation: int
base_oid: str
expected_branch_oid: str
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "GovernedProjectionBinding":
allowed = {
"governed_task_id",
"bridge_job_id",
"repository",
"branch",
"generation",
"base_oid",
"expected_branch_oid",
}
if set(value) != allowed:
raise ProjectionRecoveryError("GOVERNED_BINDING_FIELDS_INVALID", 400)
repository = _string(value["repository"], "REPOSITORY_INVALID")
if repository != "aios/aios":
raise ProjectionRecoveryError("REPOSITORY_MISMATCH", 409)
return cls(
governed_task_id=_string(
value["governed_task_id"],
"GOVERNED_TASK_ID_INVALID",
TASK_ID_RE,
),
bridge_job_id=_string(
value["bridge_job_id"],
"BRIDGE_JOB_ID_INVALID",
JOB_ID_RE,
),
repository=repository,
branch=_string(value["branch"], "BRANCH_INVALID", BRANCH_RE),
generation=_integer(value["generation"], "GENERATION_INVALID", minimum=1),
base_oid=_string(value["base_oid"], "BASE_OID_INVALID", SHA40_RE),
expected_branch_oid=_string(
value["expected_branch_oid"],
"EXPECTED_OID_INVALID",
SHA40_RE,
),
)
def public(self) -> dict[str, Any]:
return asdict(self)
@dataclass(frozen=True, slots=True)
class ProjectionSourceArtifact:
ref: str
sha256: str
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "ProjectionSourceArtifact":
if set(value) != {"ref", "sha256"}:
raise ProjectionRecoveryError("SOURCE_ARTIFACT_FIELDS_INVALID", 400)
return cls(
ref=_string(value["ref"], "SOURCE_ARTIFACT_REF_INVALID"),
sha256=_string(value["sha256"], "SOURCE_ARTIFACT_SHA_INVALID", SHA64_RE),
)
@dataclass(frozen=True, slots=True)
class CanonicalTaskProjectionIntent:
projection_id: str
projection_generation: int
idempotency_key: str
task: CanonicalTaskPayload
binding: GovernedProjectionBinding
source_artifact: ProjectionSourceArtifact
schema: str = SCHEMA
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "CanonicalTaskProjectionIntent":
allowed = {
"schema",
"projection_id",
"projection_generation",
"idempotency_key",
"task",
"governed_binding",
"source_artifact",
"task_payload_sha256",
}
if set(value) != allowed:
raise ProjectionRecoveryError("PROJECTION_INTENT_FIELDS_INVALID", 400)
if value["schema"] != SCHEMA:
raise ProjectionRecoveryError("PROJECTION_INTENT_SCHEMA_INVALID", 400)
task = CanonicalTaskPayload.from_mapping(
_plain_mapping(value["task"], "TASK_PAYLOAD_INVALID")
)
binding = GovernedProjectionBinding.from_mapping(
_plain_mapping(value["governed_binding"], "GOVERNED_BINDING_INVALID")
)
source = ProjectionSourceArtifact.from_mapping(
_plain_mapping(value["source_artifact"], "SOURCE_ARTIFACT_INVALID")
)
if binding.governed_task_id != task.task_id:
raise ProjectionRecoveryError("PROJECTION_TASK_IDENTITY_MISMATCH", 409)
expected_hash = _string(
value["task_payload_sha256"],
"TASK_PAYLOAD_SHA_INVALID",
SHA64_RE,
)
if task.payload_sha256 != expected_hash:
raise ProjectionRecoveryError("TASK_PAYLOAD_SHA_MISMATCH", 409)
projection_id = _string(
value["projection_id"],
"PROJECTION_ID_INVALID",
TASK_ID_RE,
)
idempotency_key = _string(
value["idempotency_key"],
"IDEMPOTENCY_KEY_INVALID",
TASK_ID_RE,
)
generation = _integer(
value["projection_generation"],
"PROJECTION_GENERATION_INVALID",
minimum=1,
)
return cls(
projection_id=projection_id,
projection_generation=generation,
idempotency_key=idempotency_key,
task=task,
binding=binding,
source_artifact=source,
)
def public(self) -> dict[str, Any]:
return {
"schema": self.schema,
"projection_id": self.projection_id,
"projection_generation": self.projection_generation,
"idempotency_key": self.idempotency_key,
"task": self.task.public(),
"governed_binding": self.binding.public(),
"source_artifact": asdict(self.source_artifact),
"task_payload_sha256": self.task.payload_sha256,
}
@property
def request_sha256(self) -> str:
return sha256_json(self.public())
class CanonicalTaskProjectionStore:
"""Task-Plane-owned intent/recovery store.
The caller must already be inside the Task Plane authorization boundary.
"""
def __init__(self, db_path: Path | str) -> None:
self.db_path = Path(db_path)
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.db_path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@staticmethod
def _columns(connection: sqlite3.Connection, table: str) -> dict[str, sqlite3.Row]:
return {
str(row["name"]): row
for row in connection.execute(f'PRAGMA table_info("{table}")')
}
@staticmethod
def _table_names(connection: sqlite3.Connection) -> set[str]:
return {
str(row[0])
for row in connection.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
)
}
@classmethod
def _event_table(
cls,
connection: sqlite3.Connection,
) -> tuple[str, dict[str, sqlite3.Row]]:
tables = cls._table_names(connection)
for name in ("task_events", "events"):
if name in tables:
columns = cls._columns(connection, name)
if "task_id" in columns and "event_type" in columns:
return name, columns
raise ProjectionRecoveryError("TASK_EVENT_TABLE_UNSUPPORTED", 503)
@staticmethod
def _ensure_control_tables(connection: sqlite3.Connection) -> None:
connection.execute(
"""
CREATE TABLE IF NOT EXISTS task_projection_intents (
projection_id TEXT PRIMARY KEY,
task_id TEXT NOT NULL,
bridge_job_id TEXT NOT NULL,
projection_generation INTEGER NOT NULL,
idempotency_key TEXT NOT NULL,
request_sha256 TEXT NOT NULL,
task_payload_sha256 TEXT NOT NULL,
intent_json TEXT NOT NULL,
state TEXT NOT NULL,
last_error TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
UNIQUE(task_id, projection_generation),
UNIQUE(task_id, idempotency_key)
)
"""
)
connection.execute(
"""
CREATE TABLE IF NOT EXISTS task_projection_receipts (
projection_id TEXT PRIMARY KEY,
task_id TEXT NOT NULL,
bridge_job_id TEXT NOT NULL,
projection_generation INTEGER NOT NULL,
task_payload_sha256 TEXT NOT NULL,
canonical_revision INTEGER NOT NULL,
state TEXT NOT NULL,
created_new_row INTEGER NOT NULL,
observed_at TEXT NOT NULL,
receipt_json TEXT NOT NULL
)
"""
)
def persist_intent(
self,
intent: CanonicalTaskProjectionIntent,
) -> dict[str, Any]:
now = _now()
request_sha = intent.request_sha256
intent_text = _canonical(intent.public()).decode("utf-8")
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
self._ensure_control_tables(connection)
existing = connection.execute(
"""
SELECT * FROM task_projection_intents
WHERE task_id=? AND projection_generation=?
""",
(intent.task.task_id, intent.projection_generation),
).fetchone()
if existing is not None:
if (
str(existing["projection_id"]) != intent.projection_id
or str(existing["idempotency_key"]) != intent.idempotency_key
or str(existing["request_sha256"]) != request_sha
):
raise ProjectionRecoveryError(
"PROJECTION_INTENT_IDEMPOTENCY_CONFLICT",
409,
)
connection.commit()
return {
"projection_id": intent.projection_id,
"task_id": intent.task.task_id,
"state": str(existing["state"]),
"idempotent": True,
}
connection.execute(
"""
INSERT INTO task_projection_intents(
projection_id,task_id,bridge_job_id,projection_generation,
idempotency_key,request_sha256,task_payload_sha256,intent_json,
state,last_error,created_at,updated_at
) VALUES (?,?,?,?,?,?,?,?,?,?,?,?)
""",
(
intent.projection_id,
intent.task.task_id,
intent.binding.bridge_job_id,
intent.projection_generation,
intent.idempotency_key,
request_sha,
intent.task.payload_sha256,
intent_text,
ProjectionState.PENDING.value,
None,
now,
now,
),
)
connection.commit()
return {
"projection_id": intent.projection_id,
"task_id": intent.task.task_id,
"state": ProjectionState.PENDING.value,
"idempotent": False,
}
def _task_row_payload(
self,
row: sqlite3.Row,
) -> dict[str, Any]:
try:
success = json.loads(row["success_criteria_json"])
constraints = json.loads(row["constraints_json"])
capabilities = json.loads(row["required_capabilities_json"])
metadata = json.loads(row["metadata_json"])
except (TypeError, json.JSONDecodeError) as exc:
raise ProjectionRecoveryError("CANONICAL_TASK_JSON_INVALID", 409) from exc
if (
not isinstance(success, list)
or not isinstance(constraints, Mapping)
or not isinstance(capabilities, list)
or not isinstance(metadata, Mapping)
):
raise ProjectionRecoveryError("CANONICAL_TASK_SHAPE_INVALID", 409)
return {
"task_id": str(row["task_id"]),
"goal": str(row["goal"]),
"instructions": str(row["instructions"]),
"success_criteria": success,
"constraints": dict(constraints),
"required_capabilities": capabilities,
"metadata": dict(metadata),
"priority": int(row["priority"]),
"max_attempts": int(row["max_attempts"]),
"deadline": row["deadline"],
}
def _verify_existing_task(
self,
row: sqlite3.Row,
intent: CanonicalTaskProjectionIntent,
) -> int:
observed = self._task_row_payload(row)
observed_hash = sha256_json(observed)
if observed_hash != intent.task.payload_sha256:
raise ProjectionRecoveryError(
"CANONICAL_TASK_PROJECTION_IDENTITY_CONFLICT",
409,
)
return int(row["revision"])
@staticmethod
def _required_task_columns() -> set[str]:
return {
"task_id",
"goal",
"instructions",
"success_criteria_json",
"required_capabilities_json",
"constraints_json",
"metadata_json",
"priority",
"max_attempts",
"attempt",
"status",
"claimed_by",
"lease_token_hash",
"lease_expires_at",
"result_json",
"verification_json",
"revision",
"created_at",
"updated_at",
}
def _insert_task(
self,
connection: sqlite3.Connection,
intent: CanonicalTaskProjectionIntent,
) -> int:
columns = self._columns(connection, "tasks")
if not self._required_task_columns().issubset(columns):
raise ProjectionRecoveryError("TASK_SCHEMA_RECOVERY_UNSUPPORTED", 503)
now = _now()
values: dict[str, Any] = {
"task_id": intent.task.task_id,
"goal": intent.task.goal,
"instructions": intent.task.instructions,
"success_criteria_json": _canonical(
[dict(x) for x in intent.task.success_criteria]
).decode("utf-8"),
"required_capabilities_json": "[]",
"constraints_json": _canonical(dict(intent.task.constraints)).decode("utf-8"),
"metadata_json": _canonical(dict(intent.task.metadata)).decode("utf-8"),
"deadline": intent.task.deadline,
"priority": intent.task.priority,
"max_attempts": intent.task.max_attempts,
"attempt": 0,
"status": "QUEUED",
"claimed_by": None,
"lease_token_hash": None,
"lease_expires_at": None,
"result_json": None,
"verification_json": None,
"revision": 1,
"created_at": now,
"updated_at": now,
}
for optional in (
"result_digest",
"result_idempotency_key",
):
if optional in columns:
values[optional] = None
names = [name for name in values if name in columns]
placeholders = ",".join("?" for _ in names)
connection.execute(
f'INSERT INTO tasks ({",".join(chr(34)+n+chr(34) for n in names)}) '
f"VALUES ({placeholders})",
tuple(values[name] for name in names),
)
self._insert_task_created_event(connection, intent)
return 1
def _insert_task_created_event(
self,
connection: sqlite3.Connection,
intent: CanonicalTaskProjectionIntent,
) -> None:
table, columns = self._event_table(connection)
event = {
"schema": "aios.task-projection-recovery.event.v1",
"task_id": intent.task.task_id,
"bridge_job_id": intent.binding.bridge_job_id,
"projection_id": intent.projection_id,
"projection_generation": intent.projection_generation,
"task_payload_sha256": intent.task.payload_sha256,
"source_artifact_ref": intent.source_artifact.ref,
"source_artifact_sha256": intent.source_artifact.sha256,
"recovery": True,
}
values: dict[str, Any] = {
"task_id": intent.task.task_id,
"event_type": "TASK_CREATED",
}
payload_text = _canonical(event).decode("utf-8")
if "payload_json" in columns:
values["payload_json"] = payload_text
elif "payload" in columns:
values["payload"] = payload_text
else:
raise ProjectionRecoveryError("TASK_EVENT_PAYLOAD_UNSUPPORTED", 503)
if "created_at" in columns:
values["created_at"] = _now()
if "agent_id" in columns:
values["agent_id"] = "control-plane-projection-recovery"
if "actor_id" in columns:
values["actor_id"] = "control-plane-projection-recovery"
names = list(values)
connection.execute(
f'INSERT INTO "{table}" ({",".join(chr(34)+n+chr(34) for n in names)}) '
f'VALUES ({",".join("?" for _ in names)})',
tuple(values[name] for name in names),
)
def reconcile(
self,
intent: CanonicalTaskProjectionIntent,
) -> dict[str, Any]:
# Persist first so a crash before Task creation leaves a durable PENDING
# intent for startup/supervisor reconciliation.
self.persist_intent(intent)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
self._ensure_control_tables(connection)
existing_receipt = connection.execute(
"SELECT * FROM task_projection_receipts WHERE projection_id=?",
(intent.projection_id,),
).fetchone()
if existing_receipt is not None:
if (
str(existing_receipt["task_id"]) != intent.task.task_id
or str(existing_receipt["bridge_job_id"]) != intent.binding.bridge_job_id
or str(existing_receipt["task_payload_sha256"])
!= intent.task.payload_sha256
):
raise ProjectionRecoveryError(
"PROJECTION_RECEIPT_IDENTITY_CONFLICT",
409,
)
connection.commit()
receipt = json.loads(existing_receipt["receipt_json"])
receipt["idempotent"] = True
return receipt
row = connection.execute(
"SELECT * FROM tasks WHERE task_id=?",
(intent.task.task_id,),
).fetchone()
created_new = False
if row is None:
revision = self._insert_task(connection, intent)
created_new = True
row = connection.execute(
"SELECT * FROM tasks WHERE task_id=?",
(intent.task.task_id,),
).fetchone()
if row is None:
raise ProjectionRecoveryError(
"TASK_RECOVERY_OBSERVATION_FAILED",
503,
)
else:
revision = self._verify_existing_task(row, intent)
# Observation after mutation is mandatory.
revision = self._verify_existing_task(row, intent)
observed_at = _now()
receipt = {
"schema": RECEIPT_SCHEMA,
"projection_id": intent.projection_id,
"governed_task_id": intent.binding.governed_task_id,
"canonical_task_id": intent.task.task_id,
"bridge_job_id": intent.binding.bridge_job_id,
"projection_generation": intent.projection_generation,
"task_payload_sha256": intent.task.payload_sha256,
"canonical_revision": revision,
"state": ProjectionState.CONVERGED.value,
"created_new_row": created_new,
"source_artifact": asdict(intent.source_artifact),
"observed_at": observed_at,
"idempotent": False,
}
connection.execute(
"""
INSERT INTO task_projection_receipts(
projection_id,task_id,bridge_job_id,projection_generation,
task_payload_sha256,canonical_revision,state,created_new_row,
observed_at,receipt_json
) VALUES (?,?,?,?,?,?,?,?,?,?)
""",
(
intent.projection_id,
intent.task.task_id,
intent.binding.bridge_job_id,
intent.projection_generation,
intent.task.payload_sha256,
revision,
ProjectionState.CONVERGED.value,
1 if created_new else 0,
observed_at,
_canonical(receipt).decode("utf-8"),
),
)
connection.execute(
"""
UPDATE task_projection_intents
SET state=?, last_error=NULL, updated_at=?
WHERE projection_id=?
""",
(
ProjectionState.CONVERGED.value,
observed_at,
intent.projection_id,
),
)
connection.commit()
return receipt
def get_projection(
self,
task_id: str,
) -> dict[str, Any] | None:
task_id = _string(task_id, "TASK_ID_INVALID", TASK_ID_RE)
with self._connect() as connection:
self._ensure_control_tables(connection)
intent = connection.execute(
"""
SELECT * FROM task_projection_intents
WHERE task_id=?
ORDER BY projection_generation DESC
LIMIT 1
""",
(task_id,),
).fetchone()
if intent is None:
return None
receipt = connection.execute(
"SELECT receipt_json FROM task_projection_receipts WHERE projection_id=?",
(intent["projection_id"],),
).fetchone()
return {
"projection_id": intent["projection_id"],
"task_id": task_id,
"bridge_job_id": intent["bridge_job_id"],
"projection_generation": int(intent["projection_generation"]),
"state": intent["state"],
"last_error": intent["last_error"],
"task_payload_sha256": intent["task_payload_sha256"],
"receipt": json.loads(receipt[0]) if receipt else None,
}
@@ -0,0 +1,44 @@
"""HTTP integration helpers for canonical Task projection recovery.
This module intentionally contains no HTTP server. Existing Task Plane /
projection-owner handlers should call ``recover_from_json`` only after their
normal service-side ``agent-task:control`` authorization succeeds.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any, Mapping
from aios_agent_task_projection_recovery import (
CanonicalTaskProjectionIntent,
CanonicalTaskProjectionStore,
ProjectionRecoveryError,
)
def recover_from_json(
db_path: Path | str,
task_id: str,
body: Mapping[str, Any],
) -> tuple[int, dict[str, Any]]:
try:
intent = CanonicalTaskProjectionIntent.from_mapping(body)
if intent.task.task_id != task_id:
raise ProjectionRecoveryError("PROJECTION_TASK_IDENTITY_MISMATCH", 409)
result = CanonicalTaskProjectionStore(db_path).reconcile(intent)
return (200 if result.get("idempotent") else 201), result
except ProjectionRecoveryError as exc:
return exc.status, {"error": exc.code}
def get_projection(
db_path: Path | str,
task_id: str,
) -> tuple[int, dict[str, Any]]:
try:
result = CanonicalTaskProjectionStore(db_path).get_projection(task_id)
except ProjectionRecoveryError as exc:
return exc.status, {"error": exc.code}
if result is None:
return 404, {"error": "TASK_PROJECTION_NOT_FOUND"}
return 200, result
@@ -1,427 +0,0 @@
"""Owner-controlled, CAS-bound retry extension for exhausted Online Agent Tasks."""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import hmac
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
import os
from pathlib import Path
import re
import sqlite3
from typing import Any, Mapping
TASK_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{2,179}$")
KEY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{7,159}$")
MAX_BODY = 64 * 1024
AUTHORIZE_EXTENSION_ID_ENV = "AIOS_AGENT_TASK_AUTHORIZATION_ID"
class RetryExtensionError(RuntimeError):
def __init__(self, code: str, status: int = 409) -> None:
super().__init__(code)
self.code = code
self.status = status
def _canonical(value: object) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def _sha(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _required_authorization_id() -> str:
value = os.environ.get(AUTHORIZE_EXTENSION_ID_ENV, "").strip()
if not value:
raise RuntimeError("TASK_RETRY_AUTHORIZATION_ID_MISSING")
if KEY_RE.fullmatch(value) is None:
raise RuntimeError("TASK_RETRY_AUTHORIZATION_ID_INVALID")
return value
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _require_string(value: object, code: str, pattern: re.Pattern[str] | None = None) -> str:
if not isinstance(value, str) or not value.strip():
raise RetryExtensionError(code, 400)
value = value.strip()
if pattern is not None and pattern.fullmatch(value) is None:
raise RetryExtensionError(code, 400)
return value
@dataclass(frozen=True)
class RetryExtensionRequest:
expected_status: str
expected_revision: int
expected_attempt: int
expected_max_attempts: int
next_max_attempts: int
previous_verification_sha256: str
authorization_id: str
idempotency_key: str
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "RetryExtensionRequest":
allowed = {
"expected_status", "expected_revision", "expected_attempt", "expected_max_attempts",
"next_max_attempts", "previous_verification_sha256", "authorization_id", "idempotency_key",
}
if set(value) != allowed:
raise RetryExtensionError("RETRY_EXTENSION_FIELDS_INVALID", 400)
status = _require_string(value["expected_status"], "EXPECTED_STATUS_INVALID")
if status != "FAILED":
raise RetryExtensionError("EXPECTED_STATUS_INVALID", 400)
ints: dict[str, int] = {}
for field in ("expected_revision", "expected_attempt", "expected_max_attempts", "next_max_attempts"):
item = value[field]
if not isinstance(item, int) or isinstance(item, bool) or item < 0:
raise RetryExtensionError("RETRY_EXTENSION_INTEGER_INVALID", 400)
ints[field] = item
if ints["expected_attempt"] != ints["expected_max_attempts"]:
raise RetryExtensionError("RETRY_EXTENSION_NOT_EXHAUSTED", 409)
if ints["next_max_attempts"] != ints["expected_max_attempts"] + 1:
raise RetryExtensionError("RETRY_EXTENSION_MUST_BE_BOUNDED_ONE_STEP", 400)
if ints["next_max_attempts"] > 50:
raise RetryExtensionError("RETRY_EXTENSION_LIMIT_EXCEEDED", 400)
verification = _require_string(value["previous_verification_sha256"], "VERIFICATION_SHA256_INVALID")
if re.fullmatch(r"[0-9a-f]{64}", verification) is None:
raise RetryExtensionError("VERIFICATION_SHA256_INVALID", 400)
authorization = _require_string(value["authorization_id"], "AUTHORIZATION_ID_INVALID", KEY_RE)
key = _require_string(value["idempotency_key"], "IDEMPOTENCY_KEY_INVALID", KEY_RE)
return cls(
status,
ints["expected_revision"],
ints["expected_attempt"],
ints["expected_max_attempts"],
ints["next_max_attempts"],
verification,
authorization,
key,
)
def public(self) -> dict[str, Any]:
return {
"expected_status": self.expected_status,
"expected_revision": self.expected_revision,
"expected_attempt": self.expected_attempt,
"expected_max_attempts": self.expected_max_attempts,
"next_max_attempts": self.next_max_attempts,
"previous_verification_sha256": self.previous_verification_sha256,
"authorization_id": self.authorization_id,
"idempotency_key": self.idempotency_key,
}
class RetryExtensionStore:
"""Owns the official mutation. Direct/manual DB edits remain forbidden."""
def __init__(self, db_path: Path) -> None:
self.db_path = Path(db_path)
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.db_path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@staticmethod
def _columns(connection: sqlite3.Connection, table: str) -> dict[str, sqlite3.Row]:
return {str(row["name"]): row for row in connection.execute(f'PRAGMA table_info("{table}")')}
@staticmethod
def _event_table(connection: sqlite3.Connection) -> tuple[str, dict[str, sqlite3.Row]]:
tables = {
row["name"]
for row in connection.execute("SELECT name FROM sqlite_master WHERE type='table'")
}
for name in ("task_events", "events"):
if name not in tables:
continue
columns = RetryExtensionStore._columns(connection, name)
if "task_id" in columns and ("event_type" in columns or "type" in columns):
return name, columns
raise RetryExtensionError("TASK_EVENT_TABLE_UNSUPPORTED", 503)
@staticmethod
def _verification_hash(raw: object) -> str:
if raw in (None, ""):
value: object = {}
elif isinstance(raw, str):
try:
value = json.loads(raw)
except json.JSONDecodeError as exc:
raise RetryExtensionError("TASK_VERIFICATION_INVALID", 409) from exc
else:
value = raw
return _sha(value)
@staticmethod
def _ensure_extension_table(connection: sqlite3.Connection) -> None:
connection.execute(
"""
CREATE TABLE IF NOT EXISTS task_retry_extensions (
task_id TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
request_sha256 TEXT NOT NULL,
authorization_id TEXT NOT NULL,
previous_verification_sha256 TEXT NOT NULL,
previous_max_attempts INTEGER NOT NULL,
next_max_attempts INTEGER NOT NULL,
expected_revision INTEGER NOT NULL,
applied_revision INTEGER NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY(task_id, idempotency_key)
)
"""
)
@staticmethod
def _insert_event(
connection: sqlite3.Connection,
task_id: str,
payload: Mapping[str, Any],
) -> None:
table, columns = RetryExtensionStore._event_table(connection)
values: dict[str, Any] = {"task_id": task_id}
if "event_type" in columns:
values["event_type"] = "TASK_RETRY_EXTENSION_AUTHORIZED"
else:
values["type"] = "TASK_RETRY_EXTENSION_AUTHORIZED"
payload_text = json.dumps(payload, sort_keys=True, separators=(",", ":"))
if "payload_json" in columns:
values["payload_json"] = payload_text
elif "payload" in columns:
values["payload"] = payload_text
else:
raise RetryExtensionError("TASK_EVENT_PAYLOAD_UNSUPPORTED", 503)
if "created_at" in columns:
values["created_at"] = _now()
if "agent_id" in columns:
values["agent_id"] = "owner-control"
if "actor_id" in columns:
values["actor_id"] = "owner-control"
names = list(values)
sql = f'INSERT INTO "{table}" ({",".join(f"""\"{name}\"""" for name in names)}) VALUES ({",".join("?" for _ in names)})'
connection.execute(sql, tuple(values[name] for name in names))
def extend(self, task_id: str, request: RetryExtensionRequest) -> dict[str, Any]:
task_id = _require_string(task_id, "TASK_ID_INVALID", TASK_RE)
required_authorization_id = _required_authorization_id()
if request.authorization_id != required_authorization_id:
raise RetryExtensionError("RETRY_EXTENSION_AUTHORIZATION_ID_MISMATCH", 403)
request_payload = request.public()
request_sha = _sha(request_payload)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
self._ensure_extension_table(connection)
replay = connection.execute(
"SELECT * FROM task_retry_extensions WHERE task_id=? AND idempotency_key=?",
(task_id, request.idempotency_key),
).fetchone()
if replay is not None:
if replay["request_sha256"] != request_sha:
raise RetryExtensionError("RETRY_EXTENSION_IDEMPOTENCY_CONFLICT", 409)
connection.commit()
return {
"task_id": task_id,
"status": "QUEUED",
"attempt": request.expected_attempt,
"max_attempts": int(replay["next_max_attempts"]),
"revision": int(replay["applied_revision"]),
"idempotent": True,
"authorization_id": replay["authorization_id"],
"previous_verification_sha256": replay["previous_verification_sha256"],
}
columns = self._columns(connection, "tasks")
required = {
"task_id", "status", "attempt", "max_attempts", "revision", "claimed_by",
"lease_token_hash", "lease_expires_at", "verification_json",
}
if not required.issubset(columns):
raise RetryExtensionError("TASK_SCHEMA_RETRY_EXTENSION_UNSUPPORTED", 503)
task = connection.execute("SELECT * FROM tasks WHERE task_id=?", (task_id,)).fetchone()
if task is None:
raise RetryExtensionError("TASK_NOT_FOUND", 404)
actual_verification_sha = self._verification_hash(task["verification_json"])
if actual_verification_sha != request.previous_verification_sha256:
raise RetryExtensionError("RETRY_EXTENSION_VERIFICATION_DRIFT", 409)
if (
task["status"] != request.expected_status
or int(task["revision"]) != request.expected_revision
or int(task["attempt"]) != request.expected_attempt
or int(task["max_attempts"]) != request.expected_max_attempts
):
raise RetryExtensionError("RETRY_EXTENSION_CAS_MISMATCH", 409)
for lease_field in ("lease_token_hash", "lease_expires_at"):
if lease_field in columns and task[lease_field] not in (None, ""):
raise RetryExtensionError("RETRY_EXTENSION_ACTIVE_LEASE_DENY", 409)
previous_claimed_by = task["claimed_by"]
applied_revision = request.expected_revision + 1
cursor = connection.execute(
"""
UPDATE tasks
SET status='QUEUED', max_attempts=?, revision=?, claimed_by=NULL,
lease_token_hash=NULL, lease_expires_at=NULL
WHERE task_id=? AND status=? AND revision=? AND attempt=? AND max_attempts=?
AND claimed_by IS ?
AND lease_token_hash IS NULL AND lease_expires_at IS NULL
""",
(
request.next_max_attempts,
applied_revision,
task_id,
request.expected_status,
request.expected_revision,
request.expected_attempt,
request.expected_max_attempts,
previous_claimed_by,
),
)
if cursor.rowcount != 1:
raise RetryExtensionError("RETRY_EXTENSION_CAS_MISMATCH", 409)
event_payload = {
"schema": "aios.task-retry-extension.event.v1",
"task_id": task_id,
"authorization_id": request.authorization_id,
"idempotency_key": request.idempotency_key,
"previous_verification_sha256": request.previous_verification_sha256,
"previous_max_attempts": request.expected_max_attempts,
"next_max_attempts": request.next_max_attempts,
"previous_revision": request.expected_revision,
"applied_revision": applied_revision,
"request_sha256": request_sha,
"previous_claimed_by": previous_claimed_by,
"stale_terminal_claim_normalized": previous_claimed_by not in (None, ""),
}
self._insert_event(connection, task_id, event_payload)
connection.execute(
"""
INSERT INTO task_retry_extensions(
task_id,idempotency_key,request_sha256,authorization_id,previous_verification_sha256,
previous_max_attempts,next_max_attempts,expected_revision,applied_revision,created_at
) VALUES (?,?,?,?,?,?,?,?,?,?)
""",
(
task_id,
request.idempotency_key,
request_sha,
request.authorization_id,
request.previous_verification_sha256,
request.expected_max_attempts,
request.next_max_attempts,
request.expected_revision,
applied_revision,
_now(),
),
)
connection.commit()
return {
"task_id": task_id,
"status": "QUEUED",
"attempt": request.expected_attempt,
"max_attempts": request.next_max_attempts,
"revision": applied_revision,
"idempotent": False,
"authorization_id": request.authorization_id,
"previous_verification_sha256": request.previous_verification_sha256,
}
class RetryExtensionHandler(BaseHTTPRequestHandler):
store: RetryExtensionStore | None = None
control_token: str = ""
def log_message(self, *_args: Any) -> None:
return
def _json(self, value: Mapping[str, Any], status: int) -> None:
data = json.dumps(dict(value), sort_keys=True, separators=(",", ":")).encode("utf-8")
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def _authorized(self) -> bool:
header = self.headers.get("Authorization", "")
return (
header.startswith("Bearer ")
and bool(self.control_token)
and hmac.compare_digest(header[7:], self.control_token)
)
def do_GET(self) -> None:
if self.path == "/health":
self._json({"status": "ok", "service": "aios-agent-task-retry-extension"}, 200)
else:
self._json({"error": "NOT_FOUND"}, 404)
def do_POST(self) -> None:
match = re.fullmatch(r"/v1/tasks/([^/]+)/retry-extension", self.path)
if match is None:
self._json({"error": "NOT_FOUND"}, 404)
return
if not self._authorized():
self._json({"error": "UNAUTHORIZED"}, 401)
return
try:
length = int(self.headers.get("Content-Length", "0"))
except ValueError:
self._json({"error": "CONTENT_LENGTH_INVALID"}, 400)
return
if length < 1 or length > MAX_BODY:
self._json({"error": "REQUEST_SIZE_INVALID"}, 413)
return
try:
value = json.loads(self.rfile.read(length).decode("utf-8"))
if not isinstance(value, Mapping):
raise ValueError
request = RetryExtensionRequest.from_mapping(value)
if self.store is None:
raise RetryExtensionError("RUNTIME_UNAVAILABLE", 503)
result = self.store.extend(match.group(1), request)
self._json(result, 200 if result["idempotent"] else 201)
except RetryExtensionError as exc:
self._json({"error": exc.code}, exc.status)
except (UnicodeDecodeError, json.JSONDecodeError, ValueError):
self._json({"error": "REQUEST_INVALID"}, 400)
def load_control_token(path: Path) -> str:
if path.is_symlink() or not path.is_file():
raise RuntimeError("TASK_RETRY_CONTROL_CREDENTIAL_UNAVAILABLE")
value = path.read_text(encoding="utf-8").strip()
if len(value) < 32:
raise RuntimeError("TASK_RETRY_CONTROL_CREDENTIAL_INVALID")
return value
def serve() -> None:
db = Path(os.environ.get("AIOS_AGENT_TASK_DB", "/var/lib/aios-agent-task/state.sqlite3"))
_required_authorization_id()
token_path = Path(
os.environ.get(
"AIOS_AGENT_TASK_CONTROL_TOKEN_FILE",
str(Path(os.environ.get("CREDENTIALS_DIRECTORY", "/run/credentials")) / "agent_task_control_token"),
)
)
host = os.environ.get("AIOS_TASK_RETRY_HOST", "127.0.0.1")
port = int(os.environ.get("AIOS_TASK_RETRY_PORT", "18941"))
RetryExtensionHandler.store = RetryExtensionStore(db)
RetryExtensionHandler.control_token = load_control_token(token_path)
ThreadingHTTPServer((host, port), RetryExtensionHandler).serve_forever()
if __name__ == "__main__":
serve()
@@ -94,89 +94,23 @@ class AuthorityGitExecutor:
self._git(["worktree", "add", "--detach", str(path), base_oid])
return path
def recover_failed_detached_worktree_exact(
self,
*,
path: Path,
expected_oid: str,
stale_oid: str,
target_ref: str,
) -> str:
"""Recreate one exact clean, detached, single-attempt worktree at its durable head."""
_oid(expected_oid, "expected_oid")
_oid(stale_oid, "stale_oid")
if not TASK_BRANCH_RE.fullmatch(target_ref):
def create_task_branch(self, branch: str, base_oid: str) -> str:
"""Create an isolated agent branch once, using an empty-ref CAS."""
ref = branch if branch.startswith("refs/heads/") else "refs/heads/" + branch
if not TASK_BRANCH_RE.fullmatch(ref):
raise AuthorityGitError("TASK_BRANCH_SCOPE_DENIED")
if self.read_ref(target_ref) is not None:
raise AuthorityGitError("AUTHORITY_TARGET_BRANCH_ALREADY_EXISTS")
if (
not path.is_absolute()
or path.parent != self.worktree_root
or path.name in {"", ".", ".."}
or path.is_symlink()
or not path.is_dir()
):
raise AuthorityGitError("WORKTREE_RECOVERY_PATH_INVALID")
head = subprocess.run(
["git", "-C", str(path), "rev-parse", "HEAD"],
text=True, capture_output=True, check=False,
)
if head.returncode != 0 or head.stdout.strip() != stale_oid or stale_oid == expected_oid:
raise AuthorityGitError("WORKTREE_RECOVERY_HEAD_INVALID")
symbolic = subprocess.run(
["git", "-C", str(path), "symbolic-ref", "-q", "HEAD"],
text=True, capture_output=True, check=False,
)
if symbolic.returncode != 1:
raise AuthorityGitError("WORKTREE_RECOVERY_NOT_DETACHED")
status = subprocess.run(
["git", "-C", str(path), "status", "--porcelain=v1", "--untracked-files=all"],
text=True, capture_output=True, check=False,
)
if status.returncode != 0 or status.stdout:
raise AuthorityGitError("WORKTREE_RECOVERY_NOT_CLEAN")
parents = self._git(["rev-list", "--parents", "-n", "1", stale_oid]).split()
if parents != [stale_oid, expected_oid]:
raise AuthorityGitError("WORKTREE_RECOVERY_LINEAGE_INVALID")
registered = [
line.removeprefix("worktree ")
for line in self._git(["worktree", "list", "--porcelain"]).splitlines()
if line.startswith("worktree ")
]
if registered.count(str(path)) != 1:
raise AuthorityGitError("WORKTREE_RECOVERY_REGISTRATION_INVALID")
self._git(["worktree", "remove", "--force", str(path)])
if path.exists() or path.is_symlink():
raise AuthorityGitError("WORKTREE_RECOVERY_REMOVE_FAILED")
recreated = self.create_detached_worktree(expected_oid, path.name, path.parent)
readback = subprocess.run(
["git", "-C", str(recreated), "rev-parse", "HEAD"],
text=True, capture_output=True, check=False,
)
detached = subprocess.run(
["git", "-C", str(recreated), "symbolic-ref", "-q", "HEAD"],
text=True, capture_output=True, check=False,
)
if readback.returncode != 0 or readback.stdout.strip() != expected_oid or detached.returncode != 1:
raise AuthorityGitError("WORKTREE_RECOVERY_READBACK_MISMATCH")
return stale_oid
def create_task_branch_CAS(self, branch: str, expected_base_oid: str, new_oid: str) -> str:
if not TASK_BRANCH_RE.fullmatch(branch):
raise AuthorityGitError("TASK_BRANCH_SCOPE_DENIED")
_oid(expected_base_oid, "expected_base_oid")
_oid(new_oid, "new_oid")
current = self.read_ref(branch)
_oid(base_oid, "base_oid")
if not self.commit_exists(base_oid):
raise AuthorityGitError("BASE_COMMIT_NOT_FOUND")
current = self.read_ref(ref)
if current is not None:
if current == new_oid:
return new_oid
raise AuthorityGitError("AUTHORITY_TARGET_BRANCH_ALREADY_EXISTS")
if not self.ancestry_check(expected_base_oid, new_oid):
raise AuthorityGitError("NEW_OID_NOT_DESCENDANT")
self._git(["update-ref", branch, new_oid, "0" * 40])
readback = self.read_ref(branch)
if readback != new_oid:
raise AuthorityGitError("AUTHORITY_CREATE_READBACK_MISMATCH")
if current == base_oid:
return current
raise AuthorityGitError("AUTHORITY_BRANCH_EXPECTATION_MISMATCH")
self._git(["update-ref", ref, base_oid, "0" * 40])
readback = self.read_ref(ref)
if readback != base_oid:
raise AuthorityGitError("AUTHORITY_CAS_READBACK_MISMATCH")
return readback
def advance_task_branch_CAS(self, branch: str, expected_oid: str, new_oid: str) -> str:
+125 -9
View File
@@ -22,8 +22,15 @@ from gitea_tag_publication import (
TagPublisher,
redact,
validate_version,
validate_gitea_api_url,
)
from orchestrator.action_monitor import ActionMonitor
try:
from orchestrator.action_monitor import ActionMonitor
except ModuleNotFoundError:
# The deployed service normally supplies runtime/agent-os on sys.path.
# Keep direct, isolated contract execution bound to the repository copy.
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "runtime" / "agent-os"))
from orchestrator.action_monitor import ActionMonitor
from silent_release_hardening import (
ApprovalKeyRing,
ApprovalRevocationStore,
@@ -63,6 +70,11 @@ except ModuleNotFoundError:
repository_snapshot,
run_authoritative_git,
)
try:
from aios_repository_discovery import RepositoryDiscovery, RepositoryDiscoveryError
except ModuleNotFoundError:
sys.path.insert(0, str(Path(__file__).resolve().parent))
from aios_repository_discovery import RepositoryDiscovery, RepositoryDiscoveryError
ROOT = Path(os.getenv('AIOS_GITOPS_ROOT', '/data/aios-gitops'))
WS = Path(os.getenv('AIOS_GITOPS_WORKSPACE', str(ROOT / 'workspace')))
@@ -108,6 +120,8 @@ READ_ONLY_GET_PATHS = frozenset({
'/admin/ai-evolution/api/data',
'/api/release-consistency', '/api/tag-publication-status',
'/api/release-acceptance',
'/api/repository/tree', '/api/repository/search',
'/api/repository/diff-index', '/api/repository/file',
'/getReleaseCandidate', '/getReleasePlan', '/getReleaseAcceptance',
'/getApprovalEnvelope', '/getReleaseSaga', '/getCapabilityStatus',
'/getRecoveryStatus',
@@ -308,8 +322,7 @@ def resolve_commit_oid(value: str | None, field: str = 'target_oid') -> str:
raise ValueError(f'{field} does not resolve to the exact approved commit') from exc
def existing_tag_target(tag: str) -> str | None:
if not re.fullmatch(r'V\d+R\d+C\d+(?:-[A-Za-z0-9_.-]+)?$', str(tag or '')):
raise ValueError('invalid release version')
validate_version(tag)
try:
result = run_authoritative_git(
['rev-parse', '--verify', f'refs/tags/{tag}^{{commit}}'],
@@ -342,9 +355,80 @@ def required_authoritative_git_dir() -> Path:
return Path(str(snapshot["authoritative_git_dir"]))
_DISCOVERY_ACTIONS = {
'/api/repository/tree': (
'list_repository_tree',
{'repository', 'ref', 'limit'},
{'prefix', 'depth', 'cursor'},
),
'/api/repository/search': (
'search_repository_source',
{'repository', 'ref', 'query', 'max_matches', 'max_bytes'},
{'path_globs', 'cursor'},
),
'/api/repository/diff-index': (
'get_diff_index',
{'repository', 'from', 'to', 'limit'},
{'path_prefix', 'cursor'},
),
'/api/repository/file': (
'read_repository_file',
{'repository', 'ref', 'path'},
{'max_bytes'},
),
}
def repository_discovery_request(
path: str, query: dict[str, list[str]]
) -> dict[str, Any]:
operation, required, optional = _DISCOVERY_ACTIONS[path]
if set(query) - required - optional:
raise ValueError('unknown repository discovery query field')
request: dict[str, Any] = {}
for key, values in query.items():
if key == 'path_globs':
if not values:
raise ValueError('empty repository discovery path glob')
request[key] = list(values)
continue
if len(values) != 1:
raise ValueError('duplicate repository discovery query field')
value: Any = values[0]
if key in {'depth', 'limit', 'max_matches', 'max_bytes'}:
try:
value = int(value, 10)
except (TypeError, ValueError) as exc:
raise ValueError('invalid repository discovery integer') from exc
request[key] = value
if required - set(request):
raise ValueError('missing repository discovery query field')
if not REPOSITORY_IDENTITY:
raise RepositoryDiscoveryError('REPOSITORY_DISCOVERY_UNAVAILABLE')
try:
discovery = RepositoryDiscovery(
allowed_repositories=(REPOSITORY_IDENTITY,),
git_dir=required_authoritative_git_dir(),
cursor_secret=os.getenv('AIOS_REPOSITORY_DISCOVERY_CURSOR_SECRET'),
)
except RuntimeError as exc:
if isinstance(exc, RepositoryDiscoveryError):
raise
raise RepositoryDiscoveryError('REPOSITORY_DISCOVERY_UNAVAILABLE') from exc
public_to_internal = {
'ref': 'revision',
'path': 'file_path',
'path_prefix': 'prefix',
}
internal_request = {
public_to_internal.get(key, key): value for key, value in request.items()
}
return dict(getattr(discovery, operation)(internal_request))
def publication_publisher() -> TagPublisher:
authoritative_repo = required_authoritative_git_dir()
return TagPublisher(
publisher = TagPublisher(
WS,
GitTagBackend(WS, authoritative_repo=authoritative_repo),
GiteaTagClient(
@@ -355,6 +439,10 @@ def publication_publisher() -> TagPublisher:
),
os.getenv('AIOS_GITOPS_GITEA_REMOTE','gitea'),
)
publisher.baseline_path = Path(os.getenv(
'AIOS_GITOPS_BASELINE_AUTHORITY', str(WS / 'baseline' / 'current.json')
))
return publisher
def commit(paths, msg):
ensure_git(); paths = [relpath(p) for p in paths] if paths else ['.']; sh(['add'] + paths, True)
@@ -437,6 +525,8 @@ class H(BaseHTTPRequestHandler):
return self.sendj({'error': 'GOVERNED_BRIDGE_UNAVAILABLE', 'detail': type(exc).__name__}, 503)
def do_POST(self):
path = urllib.parse.urlparse(self.path).path
if path in _DISCOVERY_ACTIONS:
return self.sendj({'error': 'method_not_allowed'},405)
if path in {
'/createTagApprovalRequest', '/requestOrCreateTag', '/getTagPublicationStatus',
'/buildProductionArtifact', '/executeApprovedProductionRelease',
@@ -452,12 +542,21 @@ class H(BaseHTTPRequestHandler):
def do_GET(self):
u=urllib.parse.urlparse(self.path); path=u.path.rstrip('/') if u.path != '/' else u.path
if path not in READ_ONLY_GET_PATHS: return self.sendj({'error':'not_found','path':path},404)
q=urllib.parse.parse_qs(u.query)
if len(self.path) > 8192:
return self.sendj({'error':'query_too_large'},413)
try:
q=urllib.parse.parse_qs(
u.query, keep_blank_values=True, max_num_fields=64
)
except ValueError:
return self.sendj({'error':'query_invalid'},400)
try:
if path in ('/health','/api/health'): return self.sendj({'status':'ok','service':'chatgpt-gitops-adapter','workspace':str(WS)})
if path in ('/api/openapi.yaml','/api/openapi-chatgpt-actions.yaml','/api/schema'):
return self.sendt(OPENAPI.read_text(encoding='utf-8'), 'application/yaml; charset=utf-8') if OPENAPI.exists() else self.sendj({'error':'schema_missing'},404)
if path in ('/api','/api/status'): return self.sendj(api_spec())
if path in _DISCOVERY_ACTIONS:
return self.sendj(repository_discovery_request(path, q))
if path in ('/api/base','/api/baseline','/base'):
data=jload(WS/'baseline'/'current.json', {})
data.setdefault('authoritative_repository', repository_snapshot(
@@ -510,7 +609,15 @@ class H(BaseHTTPRequestHandler):
return self.sendj({'error':redact(exc)},413)
except (AuthoritativeGitError, RuntimeError) as exc:
return self.sendj({'error':redact(exc)},503)
return self.sendj({'from':frm,'to':to,'diff':diff})
payload = {'from': frm, 'to': to, 'diff': diff}
payload.update({
key: diff[key]
for key in (
'from_oid', 'to_oid', 'changed_paths', 'stats',
'total_size_bytes', 'truncated', 'patch_limit_bytes',
)
})
return self.sendj(payload)
if path in ('/api/patch','/api/patches','/api/branches','/patch'):
snapshot = repository_snapshot(
git_dir=AUTHORITATIVE_GIT_DIR,
@@ -622,7 +729,8 @@ class H(BaseHTTPRequestHandler):
return self.sendj(
service.get_capability_status(candidate_id)
)
return self.sendj(service.get_release_saga(candidate_id))
result = service.get_release_saga(candidate_id)
return self.sendj(result, 404 if result.get('status') == 'NOT_FOUND' else 200)
if path == '/getRecoveryStatus':
return self.sendj(
silent_release_service().get_recovery_status()
@@ -635,6 +743,8 @@ class H(BaseHTTPRequestHandler):
return self.sendj({'error':'not_found','path':path},404)
except FileNotFoundError:
return self.sendj({'error':'release candidate not found'},404)
except RepositoryDiscoveryError as e:
return self.sendj({'error': redact(e)}, getattr(e, 'status_code', 400))
except ValueError as e:
error=redact(e)
monitor.record_failure('do_GET', error, {'path':path})
@@ -651,6 +761,8 @@ class H(BaseHTTPRequestHandler):
return self.sendj({'error':'not_found','path':path},404)
def do_POST(self):
path=urllib.parse.urlparse(self.path).path.rstrip('/')
if path in _DISCOVERY_ACTIONS:
return self.sendj({'error':'method_not_allowed'},405)
if reject_non_read_http(self, 'POST', path): return
data=self.body()
try:
@@ -773,7 +885,10 @@ class H(BaseHTTPRequestHandler):
a=self.need('tag:create');
if not a: return
ver=data.get('version') or data.get('tag')
if not re.match(r'^V\d+R\d+C\d+(-[A-Za-z0-9_.-]+)?$', ver or ''): return self.sendj({'error':'version must match VxRxCx'},400)
try:
validate_version(ver)
except ValueError:
return self.sendj({'error':'version must match VxRxCx'},400)
if not data.get('target_oid') or not data.get('evidence_closure_oid') or not data.get('approval_plan_id'):
return self.sendj({
'error':'production_tag_evidence_required',
@@ -944,9 +1059,10 @@ return{vs,cmds,ar,cmd,cdet,tot,sok,sfail,runs,baseline,refreshMsg,loadCmd,selVer
def api_spec():
return {'version':'V1R1C0-chatgpt-actions','openapi':'/api/openapi.yaml','read':['GET /api/base','GET /api/ver','GET /api/diff','GET /api/patch','GET /api/model','GET /api/run','GET /getReleaseCandidate','GET /getApprovalEnvelope','GET /getReleaseSaga','GET /admin/ai-evolution'],'write':['POST /api/branch','POST /api/file','POST /api/commit','POST /api/patch','POST /api/tag','POST /api/rollback','POST /api/model','PUT /api/model-priority','POST /api/nl','POST /createReleaseCandidate','POST /submitOwnerApproval','POST /startDeployment'],'policy':{'bearer_required_for_write':True,'direct_main_push':False,'delete_tag':False,'owner_approval_required_for_v_level':True,'api_keys_in_git':False,'git_secret_field':'secret_ref','online_git_sync':True,'silent_release_execution_enabled':SILENT_RELEASE_EXECUTION_ENABLED,'silent_release_shadow_default':True}}
return {'version':'V1R1C0-chatgpt-actions','openapi':'/api/openapi.yaml','read':['GET /api/base','GET /api/ver','GET /api/diff','GET /api/repository/tree','GET /api/repository/search','GET /api/repository/diff-index','GET /api/repository/file','GET /api/patch','GET /api/model','GET /api/run','GET /getReleaseCandidate','GET /getApprovalEnvelope','GET /getReleaseSaga','GET /admin/ai-evolution'],'write':['POST /api/branch','POST /api/file','POST /api/commit','POST /api/patch','POST /api/tag','POST /api/rollback','POST /api/model','PUT /api/model-priority','POST /api/nl','POST /createReleaseCandidate','POST /submitOwnerApproval','POST /startDeployment'],'policy':{'bearer_required_for_write':True,'direct_main_push':False,'delete_tag':False,'owner_approval_required_for_v_level':True,'api_keys_in_git':False,'git_secret_field':'secret_ref','online_git_sync':True,'silent_release_execution_enabled':SILENT_RELEASE_EXECUTION_ENABLED,'silent_release_shadow_default':True}}
def main():
validate_gitea_api_url(os.getenv('AIOS_GITOPS_GITEA_API_URL', ''))
print(f'AIOS ChatGPT GitOps adapter listening on {HOST}:{PORT}')
ThreadingHTTPServer((HOST,PORT), H).serve_forever()
if __name__ == '__main__': main()
+192 -67
View File
@@ -6,15 +6,20 @@ import hashlib
from pathlib import Path
from typing import Any, Callable, Mapping
from aios_task_admission_bridge_contract import (
TASK_ADMISSION_OPERATION_OPTIONAL_FIELDS,
TASK_ADMISSION_OPERATION_REQUIRED_FIELDS,
TASK_ADMISSION_ROUTES,
)
BRIDGE_SCHEMA_VERSION = "aios.chatgpt-gitops-bridge.v2"
BRIDGE_OPERATIONS = (
"getRuntimeStatus",
"getGovernedWriteCapabilities",
"createGovernedTask",
"projectOnlineAgentExecution",
"acquireWriterLease",
"bootstrapWriterLease",
"renewWriterLease",
"commitFileGoverned",
"commitFilesGoverned",
"getGitOpsJob",
@@ -28,18 +33,23 @@ BRIDGE_OPERATIONS = (
"getReleaseConsistency",
"runReleaseAcceptance",
"getControlPlaneSnapshot",
"listRepositoryTree",
"searchRepositorySource",
"getDiffIndex",
"readRepositoryFile",
)
TASK_ADMISSION_OPERATIONS = tuple(TASK_ADMISSION_OPERATION_REQUIRED_FIELDS)
OPERATION_REQUIRED_FIELDS = {
"createGovernedTask": ("task_id", "repository", "base_oid", "target_branch", "risk", "description", "idempotency_key"),
"acquireWriterLease": ("schema", "task_id", "repository", "branch_mode", "base_oid", "expected_branch_oid", "target_branch", "generation", "lease_seconds"),
"bootstrapWriterLease": ("schema", "task_id", "repository", "branch_mode", "base_oid", "expected_branch_oid", "target_branch", "generation", "lease_seconds"),
"renewWriterLease": ("schema", "task_id", "repository", "branch_mode", "base_oid", "expected_branch_oid", "target_branch", "generation", "lease_seconds"),
"commitFileGoverned": ("task_id", "repository", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "path", "content_base64", "message"),
"commitFilesGoverned": ("task_id", "repository", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "files", "message"),
"getGitOpsJob": ("job_id",),
"resumeGitOpsJob": ("job_id", "expected_revision"),
"reconcileGitOpsJob": ("job_id", "expected_revision"),
"projectOnlineAgentExecution": ("task_id", "required_capabilities"),
"acquireWriterLease": ("schema", "task_id", "job_id", "repository", "branch_mode", "base_oid", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "lease_seconds"),
"commitFileGoverned": ("task_id", "job_id", "repository", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "path", "content_base64", "message"),
"commitFilesGoverned": ("task_id", "job_id", "repository", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "files", "message"),
"getGitOpsJob": ("task_id", "job_id"),
"resumeGitOpsJob": ("task_id", "job_id", "expected_revision"),
"reconcileGitOpsJob": ("task_id", "job_id", "expected_revision"),
"createTagApprovalRequest": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id", "owner_approval", "execute"),
"requestOrCreateTag": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id", "execute"),
"getTagPublicationStatus": ("version",),
@@ -48,51 +58,145 @@ OPERATION_REQUIRED_FIELDS = {
"getReleaseConsistency": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id"),
"runReleaseAcceptance": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id"),
"getControlPlaneSnapshot": (),
**TASK_ADMISSION_OPERATION_REQUIRED_FIELDS,
"listRepositoryTree": ("repository", "ref", "limit"),
"searchRepositorySource": ("repository", "ref", "query", "max_matches", "max_bytes"),
"getDiffIndex": ("repository", "from", "to", "limit"),
"readRepositoryFile": ("repository", "ref", "path"),
}
BRIDGE_OPERATION_CONTRACTS = {
"acquireWriterLease": {
"method": "POST",
"path": "/v2/writer-lease",
"required_fields": OPERATION_REQUIRED_FIELDS["acquireWriterLease"],
"response_fields": (
"task_id", "job_id", "generation", "lease_seconds", "lease_expires_at",
"create_target_branch", "worktree_mode", "cas_ref_update_required",
),
},
"bootstrapWriterLease": {
"method": "POST",
"path": "/v2/writer-lease/bootstrap",
"required_fields": OPERATION_REQUIRED_FIELDS["bootstrapWriterLease"],
"response_fields": (
"task_id", "job_id", "generation", "lease_seconds", "lease_expires_at",
"create_target_branch", "worktree_mode", "cas_ref_update_required",
"recovered", "bootstrapped", "runtime_rehydrated",
"durable_branch_lease_present", "idempotent",
),
},
"renewWriterLease": {
"method": "POST",
"path": "/v2/writer-lease/renew",
"required_fields": OPERATION_REQUIRED_FIELDS["renewWriterLease"],
"response_fields": (
"task_id", "job_id", "generation", "lease_seconds", "lease_expires_at",
"recovered", "bootstrapped", "runtime_rehydrated",
"durable_branch_lease_present", "idempotent",
),
},
"commitFileGoverned": {
"method": "POST", "path": "/v2/commit-file",
"required_fields": OPERATION_REQUIRED_FIELDS["commitFileGoverned"],
"response_fields": ("job_id", "commit_oid", "commit_visible", "branch_mode", "target_branch", "generation"),
},
"commitFilesGoverned": {
"method": "POST", "path": "/v2/commit-files",
"required_fields": OPERATION_REQUIRED_FIELDS["commitFilesGoverned"],
"response_fields": ("job_id", "commit_oid", "commit_visible", "branch_mode", "target_branch", "generation"),
},
OPERATION_OPTIONAL_FIELDS = {
"listRepositoryTree": ("prefix", "depth", "cursor"),
"searchRepositorySource": ("path_globs", "cursor"),
"getDiffIndex": ("path_prefix", "cursor"),
"readRepositoryFile": ("max_bytes",),
}
READ_ONLY_OPERATIONS = frozenset(
{
"getRuntimeStatus",
"getGovernedWriteCapabilities",
"getControlPlaneSnapshot",
"listRepositoryTree",
"searchRepositorySource",
"getDiffIndex",
"readRepositoryFile",
}
)
_OPERATION_PATHS = {
"createGovernedTask": "/v2/governed-task",
"projectOnlineAgentExecution": "/v2/execution-envelope",
"acquireWriterLease": "/v2/writer-lease",
"commitFileGoverned": "/v2/commit-file",
"commitFilesGoverned": "/v2/commit-files",
"getGitOpsJob": "/v2/job",
"resumeGitOpsJob": "/v2/job/resume",
"reconcileGitOpsJob": "/v2/job/reconcile",
"getTagPublicationStatus": "/v2/tag/status",
"getReleaseConsistency": "/v2/release/consistency",
"runReleaseAcceptance": "/v2/release/acceptance",
**{operation: path for path, operation in TASK_ADMISSION_ROUTES.items()},
"listRepositoryTree": "/v2/repository/tree",
"searchRepositorySource": "/v2/repository/search",
"getDiffIndex": "/v2/repository/diff-index",
"readRepositoryFile": "/v2/repository/file",
}
def _has_complete_task_admission_handlers(
handlers: Mapping[str, Callable[[Mapping[str, Any]], Any]] | None,
) -> bool:
return handlers is not None and all(
callable(handlers.get(operation)) for operation in TASK_ADMISSION_OPERATIONS
)
def operation_contracts(
handlers: Mapping[str, Callable[[Mapping[str, Any]], Any]] | None = None,
) -> dict[str, dict[str, object]]:
"""Return the runtime contract consumed by adapters and schema tooling."""
contracts = {
operation: {
"method": "GET" if operation in OPERATION_OPTIONAL_FIELDS else "POST",
"path": _OPERATION_PATHS.get(operation),
"required_fields": list(fields),
"optional_fields": list(OPERATION_OPTIONAL_FIELDS.get(operation, ())),
"read_only": operation in READ_ONLY_OPERATIONS,
}
for operation, fields in OPERATION_REQUIRED_FIELDS.items()
if operation not in TASK_ADMISSION_OPERATIONS
}
if _has_complete_task_admission_handlers(handlers):
contracts.update(
{
operation: {
"method": "GET" if operation == "getTaskAdmissionStatus" else "POST",
"path": _OPERATION_PATHS[operation],
"required_fields": list(
TASK_ADMISSION_OPERATION_REQUIRED_FIELDS[operation]
),
"optional_fields": list(
TASK_ADMISSION_OPERATION_OPTIONAL_FIELDS[operation]
),
}
for operation in TASK_ADMISSION_OPERATIONS
}
)
return contracts
def render_openapi_contract_fragment() -> str:
required = "\n".join(
f" - {field}"
for field in OPERATION_REQUIRED_FIELDS["commitFilesGoverned"]
)
return (
" /v2/commit-files:\n"
" post:\n"
" operationId: commitFilesGoverned\n"
" summary: Submit one atomic governed commit containing multiple files\n"
" security: [{taskPrincipalAuth: []}]\n"
" x-aios-task-principal: Bridge verifies a fresh aios.task-principal-assertion.v1 and exact task/job/repository/branch/generation/revision/attempt/execution-digest/capability binding; caller JSON cannot provide a principal and missing verification is denied.\n"
" requestBody:\n"
" required: true\n"
" content:\n"
" application/json:\n"
" schema:\n"
" type: object\n"
" additionalProperties: false\n"
" required:\n"
f"{required}\n"
" properties:\n"
" task_id: {type: string}\n"
" job_id: {type: string, description: External durable Job identity for exact completed-replay reconciliation}\n"
" source_lease_epoch: {type: integer, minimum: 1}\n"
" repository: {type: string, example: aios/aios}\n"
" expected_branch_oid: {type: string, pattern: '^[0-9a-f]{40}$'}\n"
" target_branch: {type: string, pattern: '^agent/[A-Za-z0-9._/-]+$'}\n"
" generation: {type: integer, minimum: 1}\n"
" idempotency_key: {type: string}\n"
" files:\n"
" type: array\n"
" minItems: 1\n"
" items:\n"
" type: object\n"
" additionalProperties: false\n"
" required: [path, content_base64]\n"
" properties:\n"
" path: {type: string, minLength: 1, maxLength: 512}\n"
" content_base64: {type: string}\n"
" message: {type: string, minLength: 1, maxLength: 200}\n"
" responses:\n"
" '200': {description: Governed commit job and verified branch result}\n"
" '202': {description: Existing external Job alias requires authoritative reconciliation; no replacement Job is created}\n"
" '401': {description: No verified Task principal is available}\n"
" '403': {description: Verified Task principal lacks exact task, external Job, or capability scope}\n"
" '400': {description: Contract or policy rejection}\n"
" '409': {description: Branch CAS or idempotency conflict}\n"
" '423': {description: Narrow P0 Task-principal mutation gate is frozen; assertion is not consumed}\n"
)
class RuntimeDriftError(RuntimeError):
pass
@@ -156,27 +260,38 @@ class ControlPlaneReleaseManifest:
class BridgeV2Adapter:
"""Thin operation adapter; all mutating handlers are injected seams."""
def __init__(self, *, runtime_status: Mapping[str, Any], handlers: Mapping[str, Callable[[Mapping[str, Any]], Any]] | None = None) -> None:
self.runtime_status = dict(runtime_status)
def __init__(self, *, runtime_status: Mapping[str, Any] | Callable[[], Mapping[str, Any]], handlers: Mapping[str, Callable[[Mapping[str, Any]], Any]] | None = None) -> None:
self._runtime_status_source = runtime_status if callable(runtime_status) else dict(runtime_status)
self.handlers = dict(handlers or {})
def _runtime_status(self) -> dict[str, Any]:
value = self._runtime_status_source() if callable(self._runtime_status_source) else self._runtime_status_source
if not isinstance(value, Mapping):
raise BridgeV2Error("RUNTIME_STATUS_INVALID")
return dict(value)
def getRuntimeStatus(self) -> dict[str, Any]:
return {"bridge_schema_version": BRIDGE_SCHEMA_VERSION, **self.runtime_status}
return {"bridge_schema_version": BRIDGE_SCHEMA_VERSION, **self._runtime_status()}
def getGovernedWriteCapabilities(self) -> dict[str, Any]:
runtime_status = self._runtime_status()
governed_writes_enabled = runtime_status.get("governed_writes_enabled") is True
return {
"bridge_schema_version": BRIDGE_SCHEMA_VERSION,
"legacy_writes_enabled": False,
"governed_writes_enabled": True,
"operations": list(BRIDGE_OPERATIONS),
"operation_contracts": {
operation: {
**contract,
"required_fields": list(contract["required_fields"]),
"response_fields": list(contract["response_fields"]),
}
for operation, contract in BRIDGE_OPERATION_CONTRACTS.items()
},
"governed_writes_enabled": governed_writes_enabled,
"governed_write_blockers": list(
runtime_status.get("governed_write_blockers") or []
),
"operations": [
*BRIDGE_OPERATIONS,
*(
TASK_ADMISSION_OPERATIONS
if _has_complete_task_admission_handlers(self.handlers)
else ()
),
],
"operation_contracts": operation_contracts(self.handlers),
"force_push": False,
"main_write": False,
"tag_write": False,
@@ -193,21 +308,24 @@ class BridgeV2Adapter:
@staticmethod
def legacy_write_disabled() -> dict[str, object]:
return {"error": "LEGACY_WRITE_DISABLED", "replacement": "createGovernedTask", "governed_writes_available": True}
return {"error": "LEGACY_WRITE_DISABLED", "replacement": "createGovernedTask", "governed_writes_available": False}
def _call(self, operation: str, request: Mapping[str, Any]) -> Any:
required = OPERATION_REQUIRED_FIELDS[operation]
if any(key not in request for key in required):
raise BridgeV2Error("BRIDGE_REQUEST_FIELDS_MISSING")
if operation in OPERATION_OPTIONAL_FIELDS:
allowed = set(required) | set(OPERATION_OPTIONAL_FIELDS[operation])
if set(request) - allowed:
raise BridgeV2Error("BRIDGE_REQUEST_FIELDS_UNKNOWN")
handler = self.handlers.get(operation)
if handler is None:
raise BridgeV2Error("GOVERNED_HANDLER_UNAVAILABLE")
return handler(dict(request))
def createGovernedTask(self, request: Mapping[str, Any]) -> Any: return self._call("createGovernedTask", request)
def projectOnlineAgentExecution(self, request: Mapping[str, Any]) -> Any: return self._call("projectOnlineAgentExecution", request)
def acquireWriterLease(self, request: Mapping[str, Any]) -> Any: return self._call("acquireWriterLease", request)
def bootstrapWriterLease(self, request: Mapping[str, Any]) -> Any: return self._call("bootstrapWriterLease", request)
def renewWriterLease(self, request: Mapping[str, Any]) -> Any: return self._call("renewWriterLease", request)
def commitFileGoverned(self, request: Mapping[str, Any]) -> Any: return self._call("commitFileGoverned", request)
def commitFilesGoverned(self, request: Mapping[str, Any]) -> Any: return self._call("commitFilesGoverned", request)
def getGitOpsJob(self, request: Mapping[str, Any]) -> Any: return self._call("getGitOpsJob", request)
@@ -220,5 +338,12 @@ class BridgeV2Adapter:
def executeApprovedProductionRelease(self, request: Mapping[str, Any]) -> Any: return self._call("executeApprovedProductionRelease", request)
def getReleaseConsistency(self, request: Mapping[str, Any]) -> Any: return self._call("getReleaseConsistency", request)
def runReleaseAcceptance(self, request: Mapping[str, Any]) -> Any: return self._call("runReleaseAcceptance", request)
def getTaskAdmissionEnvelope(self, request: Mapping[str, Any]) -> Any: return self._call("getTaskAdmissionEnvelope", request)
def submitTaskAdmissionAuthorization(self, request: Mapping[str, Any]) -> Any: return self._call("submitTaskAdmissionAuthorization", request)
def getTaskAdmissionStatus(self, request: Mapping[str, Any]) -> Any: return self._call("getTaskAdmissionStatus", request)
def listRepositoryTree(self, request: Mapping[str, Any]) -> Any: return self._call("listRepositoryTree", request)
def searchRepositorySource(self, request: Mapping[str, Any]) -> Any: return self._call("searchRepositorySource", request)
def getDiffIndex(self, request: Mapping[str, Any]) -> Any: return self._call("getDiffIndex", request)
def readRepositoryFile(self, request: Mapping[str, Any]) -> Any: return self._call("readRepositoryFile", request)
def getControlPlaneSnapshot(self, request: Mapping[str, Any] | None = None) -> Any: return self._call("getControlPlaneSnapshot", request or {})
File diff suppressed because it is too large Load Diff
+60 -137
View File
@@ -51,14 +51,12 @@ class AuthorityPort(Protocol):
def branch_exists(self, branch: str) -> bool: ...
def branch_head(self, branch: str) -> str: ...
def is_ancestor(self, older: str, newer: str) -> bool: ...
def create_task_branch_cas(self, branch: str, expected_base: str, new: str) -> str: ...
def compare_and_swap_branch(self, branch: str, expected: str, new: str) -> str: ...
class RemotePort(Protocol):
def repository_identity(self) -> str: ...
def push_new_exact(self, branch: str, new: str) -> None: ...
def push_exact(self, branch: str, expected: str, new: str) -> None: ...
def push_exact(self, branch: str, expected: str, new: str, *, allow_absent: bool = False) -> None: ...
def read_git_ref(self, branch: str) -> str | None: ...
def read_api_branch(self, branch: str) -> str | None: ...
def commit_visible(self, oid: str) -> bool: ...
@@ -105,9 +103,6 @@ class AuthorityGitAdapter:
def is_ancestor(self, older: str, newer: str) -> bool:
return self.executor.ancestry_check(older, newer)
def create_task_branch_cas(self, branch: str, expected_base: str, new: str) -> str:
return self.executor.create_task_branch_CAS(self._ref(branch), expected_base, new)
def compare_and_swap_branch(self, branch: str, expected: str, new: str) -> str:
return self.executor.advance_task_branch_CAS(self._ref(branch), expected, new)
@@ -263,19 +258,11 @@ class GiteaContinuationRemote:
raise ContinuationExecutionError("GITEA_GIT_READBACK_INVALID")
return oid
def push_new_exact(self, branch: str, new: str) -> None:
def push_exact(self, branch: str, expected: str, new: str, *, allow_absent: bool = False) -> None:
self._require_projected()
self.repository_identity()
if self.read_git_ref(branch) is not None:
raise ContinuationExecutionError("GITEA_TARGET_BRANCH_ALREADY_EXISTS")
_run(["git", "push", self.remote_name, f"{new}:refs/heads/{branch}"], cwd=self.workspace)
if self.read_git_ref(branch) != new:
raise ContinuationExecutionError("GITEA_CREATE_READBACK_MISMATCH")
def push_exact(self, branch: str, expected: str, new: str) -> None:
self._require_projected()
self.repository_identity()
if self.read_git_ref(branch) != expected:
current = self.read_git_ref(branch)
if current != expected and not (allow_absent and current is None):
raise ContinuationExecutionError("GITEA_BRANCH_EXPECTATION_MISMATCH")
_run(["git", "push", self.remote_name, f"{new}:refs/heads/{branch}"], cwd=self.workspace)
@@ -343,13 +330,16 @@ class ContinuationPermissions:
@dataclass(frozen=True, slots=True)
class ContinuationRuntimeBinding:
request: WriterLeaseRequestV2
job_id: str
workspace_path: Path
lease_not_before: datetime
lease_expires_at: datetime
original_receipt_valid: bool
original_grant_consumed: bool
permissions: ContinuationPermissions
remote_branch_create_allowed: bool = False
external_job_id: str | None = None
worker_principal: str = "aios-autonomy-agent"
worker_lineage: str = "governed-continuation"
@dataclass(frozen=True, slots=True)
@@ -404,20 +394,13 @@ class ContinuationExecutor:
def observe_generation(self, branch: str, generation: int) -> None:
self._observed_generations[branch] = max(generation, self._observed_generations.get(branch, 0))
def _max_generation(self, branch: str, *, exclude_job_id: str | None = None) -> int:
def _max_generation(self, branch: str) -> int:
observed = self._observed_generations.get(branch, 0)
with self.controller.store._connect() as connection:
if exclude_job_id is None:
row = connection.execute(
"SELECT MAX(generation) AS generation FROM branch_leases WHERE branch=?",
(branch,),
).fetchone()
else:
row = connection.execute(
"SELECT MAX(generation) AS generation FROM branch_leases "
"WHERE branch=? AND job_id!=?",
(branch, exclude_job_id),
).fetchone()
row = connection.execute(
"SELECT MAX(generation) AS generation FROM branch_leases WHERE branch=?",
(branch,),
).fetchone()
durable = int(row["generation"] or 0)
return max(observed, durable)
@@ -426,7 +409,6 @@ class ContinuationExecutor:
return {
"schema": WriterLeaseRequestV2.schema,
"task_id": request.task_id,
"job_id": request.job_id,
"repository": request.repository,
"branch_mode": request.branch_mode,
"base_oid": request.base_oid,
@@ -436,21 +418,10 @@ class ContinuationExecutor:
"lease_seconds": request.lease_seconds,
}
def _validate_preflight(self, binding: ContinuationRuntimeBinding, *, job_id: str | None = None) -> None:
def _validate_preflight(self, binding: ContinuationRuntimeBinding) -> None:
request = binding.request
request_mapping = self._request_mapping(request)
normalized = validate_writer_lease_request_v2(request_mapping, self.authority)
expected_normalized = {
**request_mapping,
"create_target_branch": request.branch_mode == "CREATE_NEW",
"worktree_mode": (
"detached_at_base_oid"
if request.branch_mode == "CREATE_NEW"
else "detached_at_expected_head"
),
"cas_ref_update_required": True,
}
if normalized != expected_normalized:
normalized = validate_writer_lease_request_v2(self._request_mapping(request), self.authority)
if normalized != {**self._request_mapping(request), "create_target_branch": False, "worktree_mode": "detached_at_expected_head", "cas_ref_update_required": True}:
raise ContinuationExecutionError("CONTINUATION_NORMALIZATION_INVALID")
if not binding.original_receipt_valid or not binding.original_grant_consumed:
raise ContinuationExecutionError("ORIGINAL_GRANT_LINEAGE_INVALID")
@@ -460,7 +431,7 @@ class ContinuationExecutor:
raise ContinuationExecutionError("LEASE_NOT_ACTIVE")
if now >= binding.lease_expires_at:
raise ContinuationExecutionError("LEASE_EXPIRED")
if request.generation <= self._max_generation(request.target_branch, exclude_job_id=job_id):
if request.generation <= self._max_generation(request.target_branch):
raise ContinuationExecutionError("GENERATION_NOT_MONOTONIC")
if not self.workspace.is_detached():
raise ContinuationExecutionError("WORKTREE_NOT_DETACHED")
@@ -484,104 +455,70 @@ class ContinuationExecutor:
"payload_digest": hashlib.sha256(payload).hexdigest(),
"lineage_anchor_oid": request.base_oid,
"generation": request.generation,
"remote_branch_create_allowed": binding.remote_branch_create_allowed,
"external_job_id": binding.external_job_id,
"files_digest": hashlib.sha256(
json.dumps(
{path: hashlib.sha256(content).hexdigest() for path, content in sorted(commit.files.items())},
sort_keys=True, separators=(",", ":"),
).encode("utf-8")
).hexdigest(),
"message_digest": hashlib.sha256(commit.message.encode("utf-8")).hexdigest(),
"worker_principal": binding.worker_principal,
"worker_lineage": binding.worker_lineage,
}
def execute(
self,
binding: ContinuationRuntimeBinding,
commit: ContinuationCommit,
*,
before_complete: Callable[[str], None] | None = None,
) -> dict[str, object]:
commit.payload()
if not binding.job_id or binding.request.job_id != binding.job_id:
raise ContinuationExecutionError("RUNTIME_LINEAGE_MISSING_JOB_ID")
current = self.controller.store.read(binding.job_id)
current_state = JobState(current["state"])
if current_state == JobState.COMPLETED:
def execute(self, binding: ContinuationRuntimeBinding, commit: ContinuationCommit) -> dict[str, object]:
payload = commit.payload()
submitted = self.controller.submit(self._job_request(binding, commit, payload), payload=payload)
current = self.controller.store.read(submitted.job_id)
if JobState(current["state"]) == JobState.COMPLETED:
return self.controller.store._receipt(current, duplicate=True).as_dict()
if current_state == JobState.REMOTE_VERIFIED:
receipt = json.loads(str(current["receipt_json"] or "{}"))
commit_oid = str(receipt.get("authority_oid") or "")
if not OID_RE.fullmatch(commit_oid):
raise ContinuationExecutionError("GOVERNED_COMMIT_OID_INVALID")
if before_complete is not None:
before_complete(commit_oid)
return self.controller.advance(
binding.job_id,
JobState.COMPLETED,
side_effect_state="REMOTE_VERIFIED",
receipt=receipt,
).as_dict()
self._validate_preflight(binding, job_id=binding.job_id)
self._validate_preflight(binding)
request = binding.request
projected = False
try:
self.credential.project()
projected = True
row = self.controller.store.read(binding.job_id)
state = JobState(row["state"])
if state == JobState.REQUESTED:
row = self.controller.store.read(submitted.job_id)
if JobState(row["state"]) == JobState.REQUESTED:
self.controller.advance(
binding.job_id,
submitted.job_id,
JobState.AUTHORIZED,
side_effect_state="AUTHORIZED",
receipt={"task_id": request.task_id, "original_grant_consumed": True},
expected_revision=int(row["revision"]),
)
row = self.controller.store.read(binding.job_id)
state = JobState(row["state"])
if state == JobState.AUTHORIZED:
row = self.controller.store.read(submitted.job_id)
if JobState(row["state"]) == JobState.AUTHORIZED:
self.controller.lease_job(
binding.job_id,
lease_id=f"lease-{binding.job_id}",
submitted.job_id,
lease_id=f"lease-{submitted.job_id}",
lineage_anchor_oid=request.base_oid,
current_head_oid=request.expected_branch_oid,
generation=request.generation,
lease_expires_at=binding.lease_expires_at.astimezone(timezone.utc).isoformat(),
expected_revision=int(row["revision"]),
)
row = self.controller.store.read(binding.job_id)
state = JobState(row["state"])
if state == JobState.LEASED:
self.controller.advance(binding.job_id, JobState.WORKTREE_READY,
side_effect_state="WORKTREE_READY", expected_revision=int(row["revision"]))
row = self.controller.store.read(binding.job_id)
state = JobState(row["state"])
if state == JobState.WORKTREE_READY:
self.controller.advance(binding.job_id, JobState.EXECUTING,
side_effect_state="IN_PROGRESS", expected_revision=int(row["revision"]))
elif state != JobState.EXECUTING:
raise ContinuationExecutionError("JOB_EXECUTION_STATE_INVALID")
self.controller.advance(submitted.job_id, JobState.WORKTREE_READY, side_effect_state="WORKTREE_READY")
self.controller.advance(submitted.job_id, JobState.EXECUTING, side_effect_state="IN_PROGRESS")
new_oid = self.workspace.commit_files(commit.files, commit.message)
if not OID_RE.fullmatch(new_oid) or not self.authority.is_ancestor(request.expected_branch_oid, new_oid):
raise ContinuationExecutionError("NON_FAST_FORWARD_COMMIT")
if request.branch_mode == "CREATE_NEW":
authority_oid = self.authority.create_task_branch_cas(request.target_branch, request.base_oid, new_oid)
else:
authority_oid = self.authority.compare_and_swap_branch(request.target_branch, request.expected_branch_oid, new_oid)
authority_oid = self.authority.compare_and_swap_branch(request.target_branch, request.expected_branch_oid, new_oid)
if authority_oid != new_oid:
raise ContinuationExecutionError("AUTHORITY_CAS_READBACK_MISMATCH")
self.controller.advance(
binding.job_id,
submitted.job_id,
JobState.LOCAL_VERIFIED,
side_effect_state="LOCAL_VERIFIED",
receipt={"authority_oid": authority_oid, "force_push": False},
)
try:
self.controller.advance(binding.job_id, JobState.REMOTE_SYNCING, side_effect_state="IN_PROGRESS")
if request.branch_mode == "CREATE_NEW":
self.remote.push_new_exact(request.target_branch, new_oid)
else:
self.remote.push_exact(request.target_branch, request.expected_branch_oid, new_oid)
self.controller.advance(submitted.job_id, JobState.REMOTE_SYNCING, side_effect_state="IN_PROGRESS")
self.remote.push_exact(request.target_branch, request.expected_branch_oid, new_oid, allow_absent=binding.remote_branch_create_allowed)
except Exception:
return self._remote_pending(binding.job_id, authority_oid, new_oid)
return self._finish_remote_verification(
binding.job_id,
authority_oid,
new_oid,
before_complete=before_complete,
)
return self._remote_pending(submitted.job_id, authority_oid, new_oid)
return self._finish_remote_verification(submitted.job_id, authority_oid, new_oid)
finally:
if projected:
self.credential.revoke()
@@ -592,6 +529,7 @@ class ContinuationExecutor:
def _remote_pending(self, job_id: str, authority_oid: str, new_oid: str) -> dict[str, object]:
receipt = {
"authority_oid": authority_oid,
"commit_oid": new_oid,
"desired_remote_oid": new_oid,
"remote_verified": False,
"force_push": False,
@@ -604,14 +542,7 @@ class ContinuationExecutor:
)
return result.as_dict()
def _finish_remote_verification(
self,
job_id: str,
authority_oid: str,
new_oid: str,
*,
before_complete: Callable[[str], None] | None = None,
) -> dict[str, object]:
def _finish_remote_verification(self, job_id: str, authority_oid: str, new_oid: str) -> dict[str, object]:
row = self.controller.store.read(job_id)
branch = row["branch"]
request = json.loads(row["request_json"])["binding"]
@@ -621,6 +552,7 @@ class ContinuationExecutor:
return self._remote_pending(job_id, authority_oid, new_oid)
receipt = {
"authority_oid": authority_oid,
"commit_oid": new_oid,
"git_remote_oid": git_oid,
"api_branch_oid": api_oid,
"commit_visible": True,
@@ -628,20 +560,8 @@ class ContinuationExecutor:
"reconciliation_decision": decision.action.value,
"force_push": False,
}
self.controller.advance(
job_id,
JobState.REMOTE_VERIFIED,
side_effect_state="REMOTE_VERIFIED",
receipt=receipt,
)
if before_complete is not None:
before_complete(new_oid)
return self.controller.advance(
job_id,
JobState.COMPLETED,
side_effect_state="REMOTE_VERIFIED",
receipt=receipt,
).as_dict()
self.controller.advance(job_id, JobState.REMOTE_VERIFIED, side_effect_state="REMOTE_VERIFIED", receipt=receipt)
return self.controller.advance(job_id, JobState.COMPLETED, side_effect_state="REMOTE_VERIFIED", receipt=receipt).as_dict()
def reconcile(self, job_id: str) -> dict[str, object]:
row = self.controller.store.read(job_id)
@@ -681,11 +601,14 @@ class ContinuationExecutor:
and self.authority.is_ancestor(git_oid, new_oid)
):
self.controller.advance(job_id, JobState.RECONCILING, side_effect_state="OBSERVATION_FIRST_CHAIN")
self.remote.push_exact(row["branch"], git_oid, new_oid)
self.remote.push_exact(row["branch"], git_oid, new_oid, allow_absent=bool(request.get("remote_branch_create_allowed")) or request.get("branch_mode") == "CREATE_NEW")
return self._finish_remote_verification(job_id, new_oid, new_oid)
if authority_oid == new_oid and git_oid == api_oid == expected_oid:
allow_remote_create = bool(request.get("remote_branch_create_allowed")) or request.get("branch_mode") == "CREATE_NEW" or (git_oid is None and api_oid is None)
remote_matches_expected = git_oid == api_oid == expected_oid
remote_absent_for_create = git_oid is None and api_oid is None
if authority_oid == new_oid and (remote_matches_expected or remote_absent_for_create):
self.controller.advance(job_id, JobState.RECONCILING, side_effect_state="OBSERVATION_FIRST_RETRY")
self.remote.push_exact(row["branch"], expected_oid, new_oid)
self.remote.push_exact(row["branch"], expected_oid, new_oid, allow_absent=allow_remote_create)
return self._finish_remote_verification(job_id, new_oid, new_oid)
return self.controller.store._receipt(row, duplicate=True).as_dict()
finally:
@@ -49,7 +49,6 @@ class WriterLeaseRequestV2:
target_branch: str
generation: int
lease_seconds: int = 900
job_id: str = ""
schema = "aios.task-writer-lease.v2"
accepted_schemas = frozenset({"aios.task-writer-lease.v2", "aios.task-writer-lease-request.v2"})
@@ -57,8 +56,6 @@ class WriterLeaseRequestV2:
def validate(self, probe: BranchProbe) -> None:
if not self.task_id or self.repository != "aios/aios":
raise ContinuationError("request_identity_invalid")
if not self.job_id or not self.job_id.startswith("job-"):
raise ContinuationError("job_id_invalid")
if self.branch_mode not in {"CREATE_NEW", "CONTINUE_EXISTING"}:
raise ContinuationError("branch_mode_invalid")
_oid(self.base_oid, "base_oid")
@@ -10,12 +10,16 @@ def _copy(value: Mapping[str, Any] | None, keys: tuple[str, ...]) -> dict[str, A
def build_control_plane_snapshot(*, target_version: str, runtime: Mapping[str, Any] | None = None, repository: Mapping[str, Any] | None = None, production: Mapping[str, Any] | None = None, task: Mapping[str, Any] | None = None, capabilities: Mapping[str, Any] | None = None, credential_broker: Mapping[str, Any] | None = None) -> dict[str, Any]:
task_snapshot = _copy(task, ("task_id", "branch", "authority_branch_oid", "gitea_git_oid", "gitea_api_oid", "commit_visible", "generation", "reconciliation_state"))
for key in ("external_job_id", "commit_oid", "current_head_oid", "completed_replay_reconciliation_id", "receipt_authority_oid"):
if task is not None and key in task:
task_snapshot[key] = task[key]
return {
"version": {"target_release": target_version},
"runtime": _copy(runtime, ("bridge_schema_version", "control_plane_release_version", "runtime_release_commit", "runtime_path", "artifact_sha256", "drift_status")),
"repository": _copy(repository, ("identity", "authority_git_dir", "authority_main_oid")),
"production": _copy(production, ("baseline_version", "target_oid", "deployment_state")),
"task": _copy(task, ("task_id", "branch", "authority_branch_oid", "gitea_git_oid", "gitea_api_oid", "commit_visible", "generation", "reconciliation_state")),
"task": task_snapshot,
"capabilities": _copy(capabilities, ("legacy_writes_enabled", "governed_writes_enabled", "force_push", "main_write", "tag_write", "baseline_write", "production_write")),
"credential_broker": _copy(credential_broker, ("status", "provider")),
"secret_material_exposed": False,
@@ -0,0 +1,478 @@
"""Minimal convergence contracts for AIOS V2R2C2.
This module deliberately contains no external side effects. It provides the shared
data contracts and pure evaluation rules used by domain controllers. Domain
controllers remain authoritative for their own final states.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
from enum import Enum
import hashlib
import json
from typing import Any, Iterable, Mapping, Sequence
class ConvergenceContractError(ValueError):
"""Raised when an invalid convergence contract is constructed."""
class EvidenceState(str, Enum):
PASS = "PASS"
FAIL = "FAIL"
UNKNOWN = "UNKNOWN"
BLOCKED = "BLOCKED"
NOT_APPLICABLE = "NOT_APPLICABLE"
class AcceptanceDecision(str, Enum):
ACCEPT = "ACCEPT"
REJECT = "REJECT"
BLOCKED = "BLOCKED"
UNKNOWN = "UNKNOWN"
class ObservedHealth(str, Enum):
HEALTHY = "HEALTHY"
DEGRADED = "DEGRADED"
FAILED = "FAILED"
UNKNOWN = "UNKNOWN"
class ReconciliationState(str, Enum):
PENDING = "PENDING"
OBSERVING = "OBSERVING"
DIFF_FOUND = "DIFF_FOUND"
APPLYING = "APPLYING"
VERIFYING = "VERIFYING"
CONVERGED = "CONVERGED"
DEGRADED = "DEGRADED"
BLOCKED = "BLOCKED"
class ContinuationStopReason(str, Enum):
CONTEXT_BOUNDARY = "context_boundary"
WORKER_RESTART = "worker_restart"
HANDOFF = "handoff"
AUTHORITY_GATE = "authority_gate"
WAIT_EXTERNAL = "wait_external"
def _utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _canonical_json(value: Mapping[str, Any]) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def _sha256(value: Mapping[str, Any]) -> str:
return "sha256:" + hashlib.sha256(_canonical_json(value)).hexdigest()
def _nonempty(value: object, field_name: str) -> str:
text = str(value or "").strip()
if not text:
raise ConvergenceContractError(f"{field_name}_required")
return text
def _generation(value: object, field_name: str = "generation") -> int:
try:
parsed = int(value)
except (TypeError, ValueError) as exc:
raise ConvergenceContractError(f"{field_name}_invalid") from exc
if parsed < 1:
raise ConvergenceContractError(f"{field_name}_invalid")
return parsed
@dataclass(frozen=True, slots=True)
class StateOwnership:
object_type: str
authoritative_service: str
authoritative_store: str
final_writer: str | None = None
projections: tuple[str, ...] = ()
schema_version: str = "aios.state-ownership.v1"
def __post_init__(self) -> None:
_nonempty(self.object_type, "object_type")
_nonempty(self.authoritative_service, "authoritative_service")
_nonempty(self.authoritative_store, "authoritative_store")
if self.final_writer is not None:
_nonempty(self.final_writer, "final_writer")
def as_dict(self) -> dict[str, Any]:
return asdict(self)
DEFAULT_STATE_OWNERSHIP: tuple[StateOwnership, ...] = (
StateOwnership(
object_type="goal",
authoritative_service="goal-kernel",
authoritative_store="goal-store",
final_writer="goal-kernel",
projections=("admin-web", "context-pack"),
),
StateOwnership(
object_type="task_run",
authoritative_service="task-plane-18940",
authoritative_store="durable-task-store",
final_writer="task-plane-18940",
projections=("admin-web", "context-pack"),
),
StateOwnership(
object_type="release",
authoritative_service="release-controller",
authoritative_store="release-registry",
final_writer="release-controller",
projections=("baseline-authority", "admin-web"),
),
StateOwnership(
object_type="git_branch",
authoritative_service="authority-git-executor",
authoritative_store="authoritative-git",
final_writer="authority-git-executor",
projections=("gitea",),
),
StateOwnership(
object_type="external_projection",
authoritative_service="projection-controller",
authoritative_store="projection-store",
final_writer="projection-controller",
projections=("openapi", "gpt-action", "nginx"),
),
)
def ownership_index(
ownership: Iterable[StateOwnership] = DEFAULT_STATE_OWNERSHIP,
) -> dict[str, StateOwnership]:
result: dict[str, StateOwnership] = {}
for row in ownership:
if row.object_type in result:
raise ConvergenceContractError(f"duplicate_state_owner:{row.object_type}")
result[row.object_type] = row
return result
@dataclass(frozen=True, slots=True)
class AcceptanceCriterion:
criterion_id: str
required: bool
state: EvidenceState
verifier_id: str
evidence_refs: tuple[str, ...] = ()
def __post_init__(self) -> None:
_nonempty(self.criterion_id, "criterion_id")
_nonempty(self.verifier_id, "verifier_id")
@dataclass(frozen=True, slots=True)
class AcceptanceInvariant:
invariant_id: str
state: EvidenceState
evidence_refs: tuple[str, ...] = ()
def __post_init__(self) -> None:
_nonempty(self.invariant_id, "invariant_id")
@dataclass(frozen=True, slots=True)
class AcceptanceVector:
subject_type: str
subject_id: str
subject_revision: int
criteria: tuple[AcceptanceCriterion, ...]
invariants: tuple[AcceptanceInvariant, ...] = ()
unresolved_side_effects: tuple[str, ...] = ()
unresolved_blockers: tuple[str, ...] = ()
runtime_proofs: tuple[str, ...] = ()
schema: str = "aios.acceptance-vector.v1"
def __post_init__(self) -> None:
_nonempty(self.subject_type, "subject_type")
_nonempty(self.subject_id, "subject_id")
_generation(self.subject_revision, "subject_revision")
ids = [row.criterion_id for row in self.criteria]
if len(ids) != len(set(ids)):
raise ConvergenceContractError("duplicate_acceptance_criterion")
invariant_ids = [row.invariant_id for row in self.invariants]
if len(invariant_ids) != len(set(invariant_ids)):
raise ConvergenceContractError("duplicate_acceptance_invariant")
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
for row in value["criteria"]:
row["state"] = row["state"].value
for row in value["invariants"]:
row["state"] = row["state"].value
return value
@dataclass(frozen=True, slots=True)
class AcceptanceResult:
decision: AcceptanceDecision
reasons: tuple[str, ...]
vector_hash: str
def as_dict(self) -> dict[str, Any]:
return {
"decision": self.decision.value,
"reasons": list(self.reasons),
"vector_hash": self.vector_hash,
}
def evaluate_acceptance(vector: AcceptanceVector) -> AcceptanceResult:
"""Pure fail-closed acceptance evaluator.
The evaluator never writes a domain final state. The domain owner consumes this
result and remains the only final writer for Goal/Release/Product state.
"""
reasons: list[str] = []
required = [row for row in vector.criteria if row.required]
for row in required:
if row.state == EvidenceState.FAIL:
reasons.append(f"criterion_failed:{row.criterion_id}")
elif row.state == EvidenceState.BLOCKED:
reasons.append(f"criterion_blocked:{row.criterion_id}")
elif row.state == EvidenceState.UNKNOWN:
reasons.append(f"criterion_unknown:{row.criterion_id}")
for row in vector.invariants:
if row.state == EvidenceState.FAIL:
reasons.append(f"invariant_failed:{row.invariant_id}")
elif row.state in {EvidenceState.UNKNOWN, EvidenceState.BLOCKED}:
reasons.append(f"invariant_unknown:{row.invariant_id}")
if vector.unresolved_side_effects:
reasons.append("unresolved_side_effects")
if vector.unresolved_blockers:
reasons.append("unresolved_blockers")
digest = _sha256(vector.as_dict())
if any(reason.startswith(("criterion_failed:", "invariant_failed:")) for reason in reasons):
decision = AcceptanceDecision.REJECT
elif any(reason.startswith("criterion_blocked:") for reason in reasons) or vector.unresolved_blockers:
decision = AcceptanceDecision.BLOCKED
elif reasons:
decision = AcceptanceDecision.UNKNOWN
else:
decision = AcceptanceDecision.ACCEPT
return AcceptanceResult(decision=decision, reasons=tuple(reasons), vector_hash=digest)
@dataclass(frozen=True, slots=True)
class DesiredState:
resource_id: str
resource_type: str
desired_generation: int
desired_spec: Mapping[str, Any]
owner_goal_id: str | None = None
lifecycle: str = "active"
schema: str = "aios.desired-state.v1"
def __post_init__(self) -> None:
_nonempty(self.resource_id, "resource_id")
_nonempty(self.resource_type, "resource_type")
_generation(self.desired_generation, "desired_generation")
if self.lifecycle not in {"active", "suspended", "deleted"}:
raise ConvergenceContractError("desired_lifecycle_invalid")
def identity(self) -> tuple[str, int]:
return self.resource_id, self.desired_generation
@dataclass(frozen=True, slots=True)
class ObservedState:
resource_id: str
observed_generation: int | None
observed_spec: Mapping[str, Any]
health: ObservedHealth
source_identity: str
observed_at: str
evidence_refs: tuple[str, ...] = ()
schema: str = "aios.observed-state.v1"
def __post_init__(self) -> None:
_nonempty(self.resource_id, "resource_id")
_nonempty(self.source_identity, "source_identity")
_nonempty(self.observed_at, "observed_at")
if self.observed_generation is not None:
_generation(self.observed_generation, "observed_generation")
@dataclass(frozen=True, slots=True)
class ReconciliationPlan:
resource_id: str
desired_generation: int
state: ReconciliationState
diff: Mapping[str, Any]
observation_required: bool
idempotency_key: str
reason: str
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
value["state"] = self.state.value
return value
def plan_reconciliation(
desired: DesiredState,
observed: ObservedState | None,
) -> ReconciliationPlan:
"""Return a side-effect-free reconciliation decision.
UNKNOWN/missing observed state always requires observation and never converges.
"""
identity = {
"resource_id": desired.resource_id,
"desired_generation": desired.desired_generation,
"desired_spec": desired.desired_spec,
}
key = _sha256(identity)
if observed is None:
return ReconciliationPlan(
desired.resource_id,
desired.desired_generation,
ReconciliationState.OBSERVING,
{"observed": None},
True,
key,
"observed_state_missing",
)
if observed.resource_id != desired.resource_id:
raise ConvergenceContractError("resource_identity_mismatch")
if observed.health == ObservedHealth.UNKNOWN:
return ReconciliationPlan(
desired.resource_id,
desired.desired_generation,
ReconciliationState.OBSERVING,
{"health": observed.health.value},
True,
key,
"observed_state_unknown",
)
same_generation = observed.observed_generation == desired.desired_generation
same_spec = dict(observed.observed_spec) == dict(desired.desired_spec)
if same_generation and same_spec and observed.health == ObservedHealth.HEALTHY:
return ReconciliationPlan(
desired.resource_id,
desired.desired_generation,
ReconciliationState.CONVERGED,
{},
False,
key,
"desired_equals_observed",
)
return ReconciliationPlan(
desired.resource_id,
desired.desired_generation,
ReconciliationState.DIFF_FOUND,
{
"desired_generation": desired.desired_generation,
"observed_generation": observed.observed_generation,
"desired_spec": dict(desired.desired_spec),
"observed_spec": dict(observed.observed_spec),
"observed_health": observed.health.value,
},
False,
key,
"desired_observed_drift",
)
@dataclass(frozen=True, slots=True)
class SideEffectReceipt:
mutation_id: str
external_system: str
state: str
observed_at: str
external_request_id: str | None = None
external_object_id: str | None = None
observed_result_hash: str | None = None
evidence_refs: tuple[str, ...] = ()
schema: str = "aios.side-effect-receipt.v1"
def __post_init__(self) -> None:
_nonempty(self.mutation_id, "mutation_id")
_nonempty(self.external_system, "external_system")
_nonempty(self.observed_at, "observed_at")
if self.state not in {
"confirmed_applied",
"confirmed_not_applied",
"unknown",
}:
raise ConvergenceContractError("side_effect_receipt_state_invalid")
@dataclass(frozen=True, slots=True)
class ContinuationManifest:
trace_id: str
goal_id: str
goal_revision: int
task_id: str
run_id: str
current_node_id: str
acceptance_vector_ref: str
next_action: str
stop_reason: ContinuationStopReason
checkpoint_ref: str | None = None
unresolved_blocker_refs: tuple[str, ...] = ()
authoritative_decision_refs: tuple[str, ...] = ()
resource_refs: tuple[str, ...] = ()
artifact_refs: tuple[str, ...] = ()
evidence_refs: tuple[str, ...] = ()
created_at: str = field(default_factory=_utc_now)
schema: str = "aios.continuation-manifest.v1"
def __post_init__(self) -> None:
for name in (
"trace_id",
"goal_id",
"task_id",
"run_id",
"current_node_id",
"acceptance_vector_ref",
"next_action",
"created_at",
):
_nonempty(getattr(self, name), name)
_generation(self.goal_revision, "goal_revision")
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
value["stop_reason"] = self.stop_reason.value
return value
@property
def content_hash(self) -> str:
return _sha256(self.as_dict())
def validate_state_ownership(
object_type: str,
writer: str,
ownership: Iterable[StateOwnership] = DEFAULT_STATE_OWNERSHIP,
*,
final_state: bool = False,
) -> None:
row = ownership_index(ownership).get(object_type)
if row is None:
raise ConvergenceContractError(f"state_owner_missing:{object_type}")
expected = row.final_writer if final_state and row.final_writer else row.authoritative_service
if writer != expected:
raise ConvergenceContractError(
f"STATE_OWNERSHIP_VIOLATION:{object_type}:{writer}:{expected}"
)
+4 -50
View File
@@ -6,7 +6,6 @@ from datetime import datetime, timedelta, timezone
import os
from pathlib import Path
import secrets
import re
from typing import Any, Callable, Mapping
@@ -22,14 +21,9 @@ class CredentialLease:
scopes: tuple[str, ...]
provider: str
state: str = "ACTIVE"
scope_type: str = "system"
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
run_id: str | None = None
def as_dict(self) -> dict[str, Any]:
return {"lease_id": self.lease_id, "task_id": self.task_id, "repository": self.repository, "target_branch": self.target_branch, "generation": self.generation, "issued_at": self.issued_at, "expires_at": self.expires_at, "scopes": list(self.scopes), "provider": self.provider, "state": self.state, "scope_type": self.scope_type, "tenant_id": self.tenant_id, "user_id": self.user_id, "project_id": self.project_id, "run_id": self.run_id}
return {"lease_id": self.lease_id, "task_id": self.task_id, "repository": self.repository, "target_branch": self.target_branch, "generation": self.generation, "issued_at": self.issued_at, "expires_at": self.expires_at, "scopes": list(self.scopes), "provider": self.provider, "state": self.state}
class FileCredentialProvider:
@@ -49,19 +43,11 @@ class CredentialBroker:
self.now = now or (lambda: datetime.now(timezone.utc))
self._leases: dict[str, CredentialLease] = {}
def issue(self, *, task_id: str, repository: str, target_branch: str, generation: int, scopes: tuple[str, ...] = ("git:push", "gitea:read"), lease_seconds: int = 900, scope_type: str = "system", tenant_id: str | None = None, user_id: str | None = None, project_id: str | None = None, run_id: str | None = None) -> dict[str, Any]:
def issue(self, *, task_id: str, repository: str, target_branch: str, generation: int, scopes: tuple[str, ...] = ("git:push", "gitea:read"), lease_seconds: int = 900) -> dict[str, Any]:
if repository != "aios/aios" or not target_branch.startswith("agent/") or generation < 1 or not 1 <= lease_seconds <= 86400:
raise ValueError("CREDENTIAL_LEASE_BINDING_INVALID")
if scope_type == "tenant":
if not tenant_id:
raise ValueError("CREDENTIAL_SCOPE_DENY")
elif scope_type in {"system", "global"}:
if any((tenant_id, user_id, project_id, run_id)):
raise ValueError("CREDENTIAL_SCOPE_DENY")
else:
raise ValueError("CREDENTIAL_SCOPE_DENY")
now = self.now().astimezone(timezone.utc)
lease = CredentialLease("lease-" + secrets.token_hex(16), task_id, repository, target_branch, generation, now.isoformat(), (now + timedelta(seconds=lease_seconds)).isoformat(), tuple(scopes), type(self.provider).__name__, "ACTIVE", scope_type, tenant_id, user_id, project_id, run_id)
lease = CredentialLease("lease-" + secrets.token_hex(16), task_id, repository, target_branch, generation, now.isoformat(), (now + timedelta(seconds=lease_seconds)).isoformat(), tuple(scopes), type(self.provider).__name__)
self._leases[lease.lease_id] = lease
return lease.as_dict()
@@ -78,39 +64,7 @@ class CredentialBroker:
lease = self._get(lease_id)
if lease.state != "ACTIVE":
raise RuntimeError("CREDENTIAL_LEASE_NOT_ACTIVE")
environment = self.provider.environment(lease)
for key, value in environment.items():
upper = str(key).upper()
if not isinstance(value, str):
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
sensitive = any(marker in upper for marker in ("CREDENTIAL", "TOKEN", "SECRET", "PASSWORD", "PRIVATE_KEY", "API_KEY"))
if re.search(r"(?i)bearer\s+\S+|(?:token|secret|password|api[_-]?key)\s*[:=]", value):
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
if sensitive and upper.endswith("_FILE"):
candidate = Path(value)
credential_root = Path("/run/credentials")
try:
resolved = candidate.resolve(strict=False)
except OSError as exc:
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY") from exc
if not candidate.is_absolute() or not (resolved == credential_root or credential_root in resolved.parents):
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
elif sensitive and upper.endswith("_REFERENCE"):
if re.fullmatch(r"(?:systemd-credential|vault|secret-manager):[A-Za-z0-9][A-Za-z0-9._:/#-]{0,255}", value) is None:
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
elif sensitive:
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
return environment
def authorize(self, lease_id: str, *, task_id: str, operation: str, scope_type: str = "system", tenant_id: str | None = None, user_id: str | None = None, project_id: str | None = None, run_id: str | None = None) -> Mapping[str, str]:
lease = self._get(lease_id)
if lease.state != "ACTIVE" or lease.task_id != task_id or operation not in lease.scopes:
raise RuntimeError("CREDENTIAL_SCOPE_DENY")
requested = (scope_type, tenant_id, user_id, project_id, run_id)
actual = (lease.scope_type, lease.tenant_id, lease.user_id, lease.project_id, lease.run_id)
if requested != actual:
raise RuntimeError("CREDENTIAL_SCOPE_DENY")
return self.environment(lease_id)
return self.provider.environment(lease)
def status(self, lease_id: str) -> dict[str, Any]:
return self._get(lease_id).as_dict()
+327 -115
View File
@@ -27,19 +27,6 @@ class ServiceUnavailable(JobError):
pass
@dataclass(frozen=True, slots=True)
class LeaseRenewalResult:
"""Durable outcome of one writer-lease recovery attempt."""
renewed: bool
idempotent: bool
durable: bool
lease_expires_at: str | None
def __bool__(self) -> bool:
return self.renewed or self.idempotent
class BindingConflict(JobError):
pass
@@ -54,6 +41,8 @@ class JobState(str, Enum):
REMOTE_SYNCING = "REMOTE_SYNCING"
REMOTE_VERIFIED = "REMOTE_VERIFIED"
COMPLETED = "COMPLETED"
FAILED = "FAILED"
CANCELLED = "CANCELLED"
RECOVERABLE_FAILURE = "RECOVERABLE_FAILURE"
RECONCILING = "RECONCILING"
RECONCILIATION_REQUIRED = "RECONCILIATION_REQUIRED"
@@ -72,21 +61,18 @@ _ORDER = {
JobState.COMPLETED: 8,
}
_RECOVERABLE = {JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}
_TERMINAL = {JobState.COMPLETED, JobState.MANUAL_OWNER_REVIEW_REQUIRED}
_TERMINAL = {
JobState.COMPLETED,
JobState.FAILED,
JobState.CANCELLED,
JobState.MANUAL_OWNER_REVIEW_REQUIRED,
}
def utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def parse_utc(value: str) -> datetime:
normalized = value[:-1] + "+00:00" if value.endswith("Z") else value
parsed = datetime.fromisoformat(normalized)
if parsed.tzinfo is None:
raise ValueError("LEASE_EXPIRY_INVALID")
return parsed.astimezone(timezone.utc)
def canonical_json(value: Any) -> bytes:
return (
json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + "\n"
@@ -192,6 +178,63 @@ class JobStore:
connection.execute("PRAGMA busy_timeout=30000")
return connection
@classmethod
def for_existing_append(cls, root: str | os.PathLike[str]) -> "JobStore":
"""Open an existing job database without initialization side effects."""
path = Path(root)
if (
not path.is_absolute()
or path.is_symlink()
or not path.is_dir()
or not (path / "state.sqlite3").is_file()
or (path / "state.sqlite3").is_symlink()
):
raise JobError("JOB_ROOT_NOT_FOUND")
instance = cls.__new__(cls)
instance.root = path
instance.db_path = path / "state.sqlite3"
return instance
@staticmethod
def _ensure_completed_replay_schema(connection: sqlite3.Connection) -> None:
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS completed_replay_reconciliations (
reconciliation_id TEXT PRIMARY KEY, job_id TEXT NOT NULL,
identity_digest TEXT NOT NULL UNIQUE,
source_lease_epoch INTEGER NOT NULL,
task_id TEXT NOT NULL, external_job_id TEXT NOT NULL,
internal_job_id TEXT NOT NULL, repository TEXT NOT NULL,
target_branch TEXT NOT NULL, generation INTEGER NOT NULL,
branch_mode TEXT NOT NULL, payload_digest TEXT NOT NULL,
files_digest TEXT NOT NULL, message_digest TEXT NOT NULL,
commit_oid TEXT NOT NULL, authority_head_oid TEXT NOT NULL,
current_head_oid TEXT NOT NULL, historical_lease_head_oid TEXT NOT NULL,
worker_principal TEXT NOT NULL, worker_lineage TEXT NOT NULL,
durable_revision INTEGER NOT NULL, completion_identity TEXT NOT NULL,
reason TEXT NOT NULL, provenance_json TEXT NOT NULL,
created_at TEXT NOT NULL, FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE INDEX IF NOT EXISTS completed_replay_job_idx
ON completed_replay_reconciliations(job_id);
CREATE TRIGGER IF NOT EXISTS completed_replay_no_update
BEFORE UPDATE ON completed_replay_reconciliations
BEGIN SELECT RAISE(ABORT, 'COMPLETED_REPLAY_APPEND_ONLY'); END;
CREATE TRIGGER IF NOT EXISTS completed_replay_no_delete
BEFORE DELETE ON completed_replay_reconciliations
BEGIN SELECT RAISE(ABORT, 'COMPLETED_REPLAY_APPEND_ONLY'); END;
"""
)
columns = {
str(item[1]) for item in connection.execute(
"PRAGMA table_info(completed_replay_reconciliations)"
).fetchall()
}
if columns and "historical_lease_head_oid" not in columns:
connection.execute(
"ALTER TABLE completed_replay_reconciliations ADD COLUMN historical_lease_head_oid TEXT"
)
def _initialize(self) -> None:
try:
with self._connect() as connection:
@@ -230,9 +273,43 @@ class JobStore:
kind TEXT NOT NULL, digest TEXT NOT NULL, payload_json TEXT NOT NULL,
created_at TEXT NOT NULL, FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE TABLE IF NOT EXISTS completed_replay_reconciliations (
reconciliation_id TEXT PRIMARY KEY, job_id TEXT NOT NULL,
identity_digest TEXT NOT NULL UNIQUE,
source_lease_epoch INTEGER NOT NULL,
task_id TEXT NOT NULL, external_job_id TEXT NOT NULL,
internal_job_id TEXT NOT NULL, repository TEXT NOT NULL,
target_branch TEXT NOT NULL, generation INTEGER NOT NULL,
branch_mode TEXT NOT NULL, payload_digest TEXT NOT NULL,
files_digest TEXT NOT NULL, message_digest TEXT NOT NULL,
commit_oid TEXT NOT NULL, authority_head_oid TEXT NOT NULL,
current_head_oid TEXT NOT NULL, historical_lease_head_oid TEXT NOT NULL,
worker_principal TEXT NOT NULL, worker_lineage TEXT NOT NULL,
durable_revision INTEGER NOT NULL, completion_identity TEXT NOT NULL,
reason TEXT NOT NULL, provenance_json TEXT NOT NULL,
created_at TEXT NOT NULL, FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE INDEX IF NOT EXISTS completed_replay_job_idx
ON completed_replay_reconciliations(job_id);
CREATE TRIGGER IF NOT EXISTS completed_replay_no_update
BEFORE UPDATE ON completed_replay_reconciliations
BEGIN SELECT RAISE(ABORT, 'COMPLETED_REPLAY_APPEND_ONLY'); END;
CREATE TRIGGER IF NOT EXISTS completed_replay_no_delete
BEFORE DELETE ON completed_replay_reconciliations
BEGIN SELECT RAISE(ABORT, 'COMPLETED_REPLAY_APPEND_ONLY'); END;
INSERT OR REPLACE INTO metadata(key, value) VALUES('schema_version', '1');
"""
)
columns = {
str(item[1]) for item in connection.execute(
"PRAGMA table_info(completed_replay_reconciliations)"
).fetchall()
}
if columns and "historical_lease_head_oid" not in columns:
connection.execute(
"ALTER TABLE completed_replay_reconciliations "
"ADD COLUMN historical_lease_head_oid TEXT"
)
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
@@ -265,6 +342,227 @@ class JobStore:
raise JobError("JOB_NOT_FOUND")
return dict(row)
@staticmethod
def _completed_replay_identity(envelope: Mapping[str, Any]) -> dict[str, Any]:
required = (
"task_id", "external_job_id", "internal_job_id", "repository",
"target_branch", "generation", "branch_mode", "payload_digest",
"files_digest", "message_digest", "commit_oid", "authority_head_oid",
"historical_lease_head_oid", "worker_principal", "worker_lineage", "source_lease_epoch",
"durable_revision", "completion_identity", "reason", "provenance",
)
if any(key not in envelope for key in required):
raise BindingConflict("COMPLETED_REPLAY_BINDING_INCOMPLETE")
text_fields = (
"task_id", "external_job_id", "internal_job_id", "repository",
"target_branch", "branch_mode", "payload_digest", "files_digest",
"message_digest", "commit_oid", "authority_head_oid", "worker_principal",
"worker_lineage", "historical_lease_head_oid", "completion_identity", "reason",
)
result: dict[str, Any] = {}
for key in text_fields:
value = envelope[key]
if not isinstance(value, str) or not value.strip():
raise BindingConflict("COMPLETED_REPLAY_BINDING_INVALID")
result[key] = value
for key in ("generation", "source_lease_epoch", "durable_revision"):
value = envelope[key]
if isinstance(value, bool) or not isinstance(value, int) or value < 1:
raise BindingConflict("COMPLETED_REPLAY_BINDING_INVALID")
result[key] = value
for key in ("commit_oid", "authority_head_oid", "historical_lease_head_oid"):
if len(result[key]) != 40 or any(ch not in "0123456789abcdef" for ch in result[key]):
raise BindingConflict("COMPLETED_REPLAY_BINDING_INVALID")
for key in ("payload_digest", "files_digest", "message_digest"):
if len(result[key]) != 64 or any(ch not in "0123456789abcdef" for ch in result[key]):
raise BindingConflict("COMPLETED_REPLAY_BINDING_INVALID")
if result["repository"] != "aios/aios" or result["branch_mode"] not in {"CREATE_NEW", "CONTINUE_EXISTING"}:
raise BindingConflict("COMPLETED_REPLAY_BINDING_INVALID")
if not result["target_branch"].startswith("agent/") or result["target_branch"] in {"main", "master"}:
raise BindingConflict("COMPLETED_REPLAY_BINDING_INVALID")
provenance = envelope["provenance"]
if not isinstance(provenance, Mapping) or not provenance:
raise BindingConflict("COMPLETED_REPLAY_PROVENANCE_INVALID")
result["provenance"] = dict(provenance)
if result["commit_oid"] != result["authority_head_oid"]:
raise BindingConflict("COMPLETED_REPLAY_AUTHORITY_CONFLICT")
return result
def append_completed_replay_reconciliation(
self,
job_id: str,
envelope: Mapping[str, Any],
*,
authority_head_oid: str,
) -> dict[str, Any]:
"""Append exact completed-replay evidence without rewriting history."""
identity = self._completed_replay_identity(envelope)
if authority_head_oid != identity["authority_head_oid"]:
raise BindingConflict("COMPLETED_REPLAY_AUTHORITY_CONFLICT")
if identity["internal_job_id"] != job_id:
raise BindingConflict("COMPLETED_REPLAY_INTERNAL_JOB_CONFLICT")
identity_digest = sha256_json(identity)
reconciliation_id = "reconcile-" + identity_digest[:32]
now = utc_now()
try:
with self._connect() as connection:
self._ensure_completed_replay_schema(connection)
connection.execute("BEGIN IMMEDIATE")
row = connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone()
if row is None:
raise JobError("JOB_NOT_FOUND")
if JobState(row["state"]) != JobState.COMPLETED:
raise BindingConflict("COMPLETED_REPLAY_JOB_NOT_COMPLETED")
durable_request = json.loads(row["request_json"])
binding = durable_request.get("binding", durable_request)
exact_binding = {
"task_id": binding.get("task_id") or binding.get("campaign_id"),
"external_job_id": binding.get("external_job_id"),
"internal_job_id": job_id,
"repository": binding.get("repository") or row["repository"],
"target_branch": binding.get("target_branch") or binding.get("branch"),
"generation": binding.get("generation"),
"branch_mode": binding.get("branch_mode"),
"payload_digest": binding.get("payload_digest") or row["payload_digest"],
"files_digest": binding.get("files_digest"),
"message_digest": binding.get("message_digest"),
"worker_principal": binding.get("worker_principal"),
"worker_lineage": binding.get("worker_lineage"),
}
for key, value in exact_binding.items():
if str(value) != str(identity[key]):
raise BindingConflict("COMPLETED_REPLAY_BINDING_CONFLICT")
receipt = json.loads(row["receipt_json"]) if row["receipt_json"] else None
if not isinstance(receipt, Mapping):
raise BindingConflict("COMPLETED_REPLAY_RECEIPT_MISSING")
if receipt.get("commit_oid") != identity["commit_oid"] or receipt.get("authority_oid") != identity["authority_head_oid"]:
raise BindingConflict("COMPLETED_REPLAY_RECEIPT_AUTHORITY_CONFLICT")
completion_identity = sha256_json({
"job_id": row["job_id"],
"revision": int(row["revision"]),
"receipt": receipt,
})
if identity["completion_identity"] != completion_identity:
raise BindingConflict("COMPLETED_REPLAY_COMPLETION_IDENTITY_CONFLICT")
lease_columns = {
str(item[1]) for item in connection.execute("PRAGMA table_info(branch_leases)").fetchall()
}
if "epoch" in lease_columns:
lease = connection.execute(
"SELECT lease_id, state, current_head_oid, generation FROM branch_leases WHERE job_id=? AND epoch=? ORDER BY rowid DESC LIMIT 1",
(job_id, identity["source_lease_epoch"]),
).fetchone()
else:
if identity["source_lease_epoch"] != 1:
raise BindingConflict("COMPLETED_REPLAY_LEASE_EPOCH_UNAVAILABLE")
lease = connection.execute(
"SELECT lease_id, state, current_head_oid, generation FROM branch_leases WHERE job_id=? ORDER BY rowid DESC LIMIT 1",
(job_id,),
).fetchone()
if lease is None:
raise BindingConflict("COMPLETED_REPLAY_LEASE_MISSING")
lease_state = str(lease["state"])
lease_head = str(lease["current_head_oid"])
lease_generation = int(lease["generation"])
event_tables = {
str(item[0]) for item in connection.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name='branch_lease_events'"
).fetchall()
}
if "branch_lease_events" in event_tables:
event = connection.execute(
"SELECT state, current_head_oid, generation FROM branch_lease_events WHERE lease_id=? ORDER BY rowid DESC LIMIT 1",
(lease["lease_id"],),
).fetchone()
if event is not None:
lease_state = str(event["state"])
lease_head = str(event["current_head_oid"])
lease_generation = int(event["generation"])
if lease_state not in {"ACTIVE", "USED", "EXPIRED"}:
raise BindingConflict("COMPLETED_REPLAY_LEASE_STATE_INVALID")
if lease_generation != identity["generation"]:
raise BindingConflict("COMPLETED_REPLAY_LEASE_GENERATION_CONFLICT")
if len(lease_head) != 40 or any(ch not in "0123456789abcdef" for ch in lease_head):
raise BindingConflict("COMPLETED_REPLAY_LEASE_HEAD_INVALID")
if lease_head != identity["historical_lease_head_oid"]:
raise BindingConflict("COMPLETED_REPLAY_HISTORICAL_LEASE_CONFLICT")
if identity["commit_oid"] == identity["historical_lease_head_oid"]:
raise BindingConflict("COMPLETED_REPLAY_EVIDENCE_INCOHERENT")
existing = connection.execute(
"SELECT * FROM completed_replay_reconciliations WHERE job_id=? ORDER BY created_at ASC",
(job_id,),
).fetchall()
for prior in existing:
if prior["identity_digest"] != identity_digest:
raise BindingConflict("COMPLETED_REPLAY_AMBIGUOUS")
if existing:
result = dict(existing[0])
connection.commit()
return result
provenance_json = json.dumps(identity["provenance"], sort_keys=True, separators=(",", ":"))
connection.execute(
"INSERT INTO completed_replay_reconciliations(reconciliation_id,job_id,identity_digest,source_lease_epoch,task_id,external_job_id,internal_job_id,repository,target_branch,generation,branch_mode,payload_digest,files_digest,message_digest,commit_oid,authority_head_oid,worker_principal,worker_lineage,current_head_oid,historical_lease_head_oid,durable_revision,completion_identity,reason,provenance_json,created_at) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
(
reconciliation_id, job_id, identity["identity_digest"] if "identity_digest" in identity else identity_digest, identity["source_lease_epoch"],
identity["task_id"], identity["external_job_id"], identity["internal_job_id"],
identity["repository"], identity["target_branch"], identity["generation"],
identity["branch_mode"], identity["payload_digest"], identity["files_digest"],
identity["message_digest"], identity["commit_oid"], identity["authority_head_oid"],
identity["worker_principal"], identity["worker_lineage"], identity["authority_head_oid"], identity["historical_lease_head_oid"], identity["durable_revision"],
identity["completion_identity"], identity["reason"], provenance_json, now,
),
)
result = dict(connection.execute(
"SELECT * FROM completed_replay_reconciliations WHERE reconciliation_id=?",
(reconciliation_id,),
).fetchone())
connection.commit()
return result
except (JobError, BindingConflict):
raise
except sqlite3.IntegrityError as exc:
try:
with self._connect() as connection:
existing = connection.execute(
"SELECT * FROM completed_replay_reconciliations WHERE identity_digest=?",
(identity_digest,),
).fetchone()
if existing is not None:
return dict(existing)
except sqlite3.Error:
pass
raise BindingConflict("COMPLETED_REPLAY_IDEMPOTENCY_CONFLICT") from exc
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
def completed_replay_evidence(self, job_id: str) -> dict[str, Any] | None:
try:
with self._connect() as connection:
rows = connection.execute(
"SELECT * FROM completed_replay_reconciliations WHERE job_id=? ORDER BY created_at ASC",
(job_id,),
).fetchall()
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
if not rows:
return None
if len({str(row["identity_digest"]) for row in rows}) != 1:
raise BindingConflict("COMPLETED_REPLAY_AMBIGUOUS")
first = dict(rows[0])
immutable_fields = tuple(
key for key in first
if key != "created_at"
)
for row in rows[1:]:
if any(row[key] != first[key] for key in immutable_fields):
raise BindingConflict("COMPLETED_REPLAY_AMBIGUOUS")
if not (
first["commit_oid"] == first["authority_head_oid"] == first["current_head_oid"]
and first["commit_oid"] != first["historical_lease_head_oid"]
):
raise BindingConflict("COMPLETED_REPLAY_EVIDENCE_INCOHERENT")
return first
def _receipt(self, row: Mapping[str, Any], *, duplicate: bool) -> JobReceipt:
state = JobState(row["state"])
if state == JobState.COMPLETED:
@@ -316,6 +614,11 @@ class GitOpsJobController:
"check_context",
"check_state",
"check_description",
"external_job_id",
"files_digest",
"message_digest",
"worker_principal",
"worker_lineage",
)
for field in optional:
value = request.get(field)
@@ -425,16 +728,13 @@ class GitOpsJobController:
self.store._mirror(result)
return self.store._receipt(result, duplicate=True)
def update_lease(self, lease_id: str, *, expected_revision: int, current_head_oid: str, generation: int, state: str = "ACTIVE", lease_expires_at: str | None = None) -> None:
def update_lease(self, lease_id: str, *, expected_revision: int, current_head_oid: str, generation: int, state: str = "ACTIVE") -> None:
if len(current_head_oid) != 40 or generation < 1:
raise JobError("LEASE_BINDING_INVALID")
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
if lease_expires_at is None:
updated = connection.execute("UPDATE branch_leases SET current_head_oid=?,generation=?,state=?,revision=revision+1 WHERE lease_id=? AND revision=?", (current_head_oid, generation, state, lease_id, expected_revision))
else:
updated = connection.execute("UPDATE branch_leases SET current_head_oid=?,generation=?,lease_expires_at=?,state=?,revision=revision+1 WHERE lease_id=? AND revision=?", (current_head_oid, generation, lease_expires_at, state, lease_id, expected_revision))
updated = connection.execute("UPDATE branch_leases SET current_head_oid=?,generation=?,state=?,revision=revision+1 WHERE lease_id=? AND revision=?", (current_head_oid, generation, state, lease_id, expected_revision))
if updated.rowcount != 1:
raise JobError("LEASE_REVISION_CAS_CONFLICT")
connection.commit()
@@ -443,94 +743,6 @@ class GitOpsJobController:
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
def renew_lease(
self,
*,
branch: str,
lineage_anchor_oid: str,
current_head_oid: str,
generation: int,
lease_expires_at: str,
) -> LeaseRenewalResult:
if (
not branch
or len(lineage_anchor_oid) != 40
or len(current_head_oid) != 40
or generation < 1
or not lease_expires_at.strip()
):
raise JobError("LEASE_BINDING_INVALID")
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
rows = connection.execute(
"SELECT lease_id, revision, generation, state, lease_expires_at "
"FROM branch_leases "
"WHERE branch=? AND lineage_anchor_oid=? AND current_head_oid=? "
"AND generation=? AND state='ACTIVE' ORDER BY revision DESC",
(branch, lineage_anchor_oid, current_head_oid, generation),
).fetchall()
if not rows:
connection.commit()
return LeaseRenewalResult(False, False, False, None)
if len(rows) != 1:
raise JobError("LEASE_BINDING_CONFLICT")
row = rows[0]
try:
current_expires_at = parse_utc(str(row["lease_expires_at"]))
requested_expires_at = parse_utc(lease_expires_at)
except (TypeError, ValueError) as exc:
raise JobError("LEASE_EXPIRY_INVALID") from exc
now = datetime.now(timezone.utc)
if requested_expires_at <= now:
raise JobError("LEASE_EXPIRY_INVALID")
if current_expires_at > now:
connection.commit()
return LeaseRenewalResult(
False,
True,
True,
str(row["lease_expires_at"]),
)
updated = connection.execute(
"UPDATE branch_leases SET lease_expires_at=?,revision=revision+1 "
"WHERE lease_id=? AND revision=? AND generation=? "
"AND state='ACTIVE' AND lease_expires_at=?",
(
lease_expires_at,
row["lease_id"],
int(row["revision"]),
generation,
row["lease_expires_at"],
),
)
if updated.rowcount != 1:
current = connection.execute(
"SELECT lease_expires_at, generation, state FROM branch_leases "
"WHERE lease_id=?",
(row["lease_id"],),
).fetchone()
if (
current is not None
and int(current["generation"]) == generation
and current["state"] == "ACTIVE"
and parse_utc(str(current["lease_expires_at"])) > now
):
connection.commit()
return LeaseRenewalResult(
False,
True,
True,
str(current["lease_expires_at"]),
)
raise JobError("LEASE_REVISION_CAS_CONFLICT")
connection.commit()
return LeaseRenewalResult(True, False, True, lease_expires_at)
except JobError:
raise
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
def advance(self, job_id: str, state: JobState, *, side_effect_state: str, receipt: Mapping[str, Any] | None = None, expected_revision: int | None = None) -> JobReceipt:
return self._advance(
job_id,
+7 -1
View File
@@ -5,6 +5,8 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from typing import Any
from release_version import validate_release_version
ROOT = Path(os.getenv('AIOS_GITOPS_ROOT', '/data/aios-gitops'))
WORKSPACE = Path(os.getenv('AIOS_GITOPS_WORKSPACE', str(ROOT / 'workspace')))
DB_PATH = Path(os.getenv('AIOS_GITOPS_DB', str(ROOT / 'runtime' / 'gitops_model.sqlite3')))
@@ -218,7 +220,11 @@ class Handler(BaseHTTPRequestHandler):
a = self.require_write(['owner']);
if not a: return
ver=data.get('version') or data.get('tag')
if not re.match(r'^V\d+R\d+C\d+(-[a-zA-Z0-9_.-]+)?$', ver or ''): self.send_json({'error':'version must match VxRxCx'},400); return
try:
validate_release_version(ver)
except ValueError:
self.send_json({'error':'version must match VxRxCx'},400)
return
msg=data.get('message') or f'release {ver}'; out=git_cmd(['tag','-a',ver,'-m',msg]);
if data.get('push',False): git_cmd(['push','origin',ver])
level='V' if re.match(r'^V[1-9]\d*R0C0', ver) else ('R' if re.match(r'^V\d+R[1-9]\d*C0', ver) else 'C'); exec_sql('INSERT OR REPLACE INTO gitops_versions(version,tag,level,summary,status,created_by,created_at) VALUES(?,?,?,?,?,?,?)',(ver,ver,level,msg,'ready',a['username'],now())); exec_sql('UPDATE gitops_baseline SET active_version=?,active_tag=?,updated_at=? WHERE id=?',(ver,ver,now(),'current')); audit(a['username'],'git_tag',ver,{'level':level}); self.send_json({'status':'tagged','version':ver,'git':out}); return
@@ -9,6 +9,7 @@ import re
from typing import Any, Mapping
from aios_gitops_job_controller import GitOpsJobController, JobStore
from aios_online_agent_execution import GovernedExecutionBinding, project_execution_envelope
OID_RE = re.compile(r"^[0-9a-f]{40}$")
BRANCH_RE = re.compile(r"^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$")
@@ -53,7 +54,46 @@ class GovernedTaskBootstrap:
def _read_existing(self, task_id: str) -> GovernedTaskBinding | None:
path = self._metadata(task_id)
if not path.is_file() or path.is_symlink(): return None
return GovernedTaskBinding(**json.loads(path.read_text(encoding="utf-8")))
def get_binding(self, task_id: str) -> GovernedTaskBinding:
"""Read the exact durable binding; never synthesize missing lineage."""
binding = self._read_existing(str(task_id))
if binding is None or not binding.job_id:
raise BootstrapError("TASK_EXECUTION_PROJECTION_MISSING")
return binding
def execution_binding(self, binding: GovernedTaskBinding) -> GovernedExecutionBinding:
return GovernedExecutionBinding.from_mapping(
{
"task_id": binding.task_id,
"job_id": binding.job_id,
"repository": binding.repository,
"base_oid": binding.base_oid,
"expected_branch_oid": binding.expected_branch_oid,
"target_branch": binding.target_branch,
"branch_mode": binding.branch_mode,
"generation": binding.generation,
"idempotency_key": binding.idempotency_key,
}
)
def execution_envelope(
self,
task_id: str,
execution_request: Mapping[str, Any] | None = None,
*,
required_capabilities: tuple[str, ...] = (),
expected_job_id: str | None = None,
) -> dict[str, Any]:
binding = self.get_binding(task_id)
if expected_job_id and binding.job_id != expected_job_id:
raise BootstrapError("JOB_BINDING_CONFLICT")
return project_execution_envelope(
self.execution_binding(binding),
execution_request,
required_capabilities=required_capabilities,
)
def _write(self, binding: GovernedTaskBinding) -> None:
path = self._metadata(binding.task_id); temporary = path.with_suffix(".tmp"); temporary.write_text(json.dumps(asdict(binding), sort_keys=True), encoding="utf-8"); temporary.replace(path)
@@ -1,306 +0,0 @@
"""Uncredentialed deterministic executor for governed Online Agent changes.
This process never receives Agent/Owner/Bridge credentials and never writes Git refs.
It operates only on an exchange bundle and returns hash-bound changed files + test evidence.
"""
from __future__ import annotations
import base64
from dataclasses import dataclass
import errno
import hashlib
import io
import json
import os
from pathlib import Path
import re
import shutil
import socket
import subprocess
import tarfile
import tempfile
import time
from typing import Any, Mapping, Sequence
ALLOWED_PROGRAMS = {"/usr/bin/python3", "/usr/bin/git", "/bin/true", "/usr/bin/true"}
MAX_ARCHIVE_BYTES = 64 * 1024 * 1024
MAX_FILE_BYTES = 4 * 1024 * 1024
SECRET_ENV_TOKENS = ("TOKEN", "SECRET", "PASSWORD", "CREDENTIAL", "AUTHORIZATION", "PRIVATE_KEY")
REQUEST_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{15,254}$")
SENSITIVE_VALUE_RE = re.compile(
r"(?i)(authorization\s*[:=]\s*bearer\s+\S+|authorization\s+bearer\s+\S+|"
r"(?:password|passwd|secret|token|api[_-]?key)\s*[:=]\s*\S+)"
)
class IsolatedExecutorError(RuntimeError):
pass
def sha256(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def canonical(value: object) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def safe_relative(value: object) -> str:
if not isinstance(value, str) or not value or value.startswith("/"):
raise IsolatedExecutorError("PATH_INVALID")
path = Path(value)
if ".." in path.parts or "." in path.parts or any(part in {"", ".git"} for part in path.parts):
raise IsolatedExecutorError("PATH_INVALID")
return path.as_posix()
def resolve_under(root: Path, relative: str) -> Path:
root = root.resolve()
path = (root / safe_relative(relative)).resolve(strict=False)
if path == root or root not in path.parents:
raise IsolatedExecutorError("PATH_ESCAPE")
return path
def _credential_exposure(directory: Path) -> str | None:
"""Return an exposure code; EACCES is the intended credential isolation proof."""
try:
if not directory.exists():
return None
if directory.is_symlink():
return "CREDENTIAL_MOUNT_SYMLINK"
if os.access(directory, os.R_OK):
return "CREDENTIAL_MOUNT_ACCESSIBLE"
if directory.is_dir():
for child in directory.iterdir():
if child.is_file() and os.access(child, os.R_OK):
return "CREDENTIAL_FILE_ACCESSIBLE"
except PermissionError:
return None
return None
def assert_uncredentialed() -> None:
credentials = Path("/run/credentials")
candidates = {credentials}
credentials_directory = Path(os.environ.get("CREDENTIALS_DIRECTORY", "/run/credentials"))
candidates.add(credentials_directory)
leaked = sorted(
key for key, value in os.environ.items()
if value
and (any(token in key.upper() for token in SECRET_ENV_TOKENS) or SENSITIVE_VALUE_RE.search(str(value)))
)
if leaked:
raise IsolatedExecutorError("CREDENTIAL_ENV_EXPOSED:" + ",".join(leaked))
for directory in candidates:
exposure = _credential_exposure(directory)
if exposure is not None:
raise IsolatedExecutorError(exposure)
# In the intended PrivateNetwork namespace host loopback services are unreachable.
try:
probe = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except OSError as exc:
if exc.errno in {errno.EACCES, errno.EPERM, errno.EAFNOSUPPORT}:
return
raise IsolatedExecutorError("NETWORK_ISOLATION_PROBE_FAILED") from exc
probe.settimeout(0.15)
try:
if probe.connect_ex(("127.0.0.1", 18892)) == 0:
raise IsolatedExecutorError("GOVERNED_BRIDGE_REACHABLE")
finally:
probe.close()
def extract_source(archive: Path, root: Path) -> None:
if archive.is_symlink() or not archive.is_file() or archive.stat().st_size > MAX_ARCHIVE_BYTES:
raise IsolatedExecutorError("SOURCE_ARCHIVE_INVALID")
with tarfile.open(archive, "r:gz") as tf:
for member in tf.getmembers():
path = Path(member.name)
if path.is_absolute() or ".." in path.parts or ".git" in path.parts:
raise IsolatedExecutorError("SOURCE_ARCHIVE_PATH_INVALID")
if member.issym() or member.islnk():
raise IsolatedExecutorError("SOURCE_ARCHIVE_LINK_DENY")
tf.extractall(root, filter="data")
def run_command(item: Mapping[str, Any], root: Path) -> dict[str, Any]:
argv = item.get("argv")
if not isinstance(argv, Sequence) or isinstance(argv, (str, bytes)) or not argv:
raise IsolatedExecutorError("COMMAND_INVALID")
argv = [str(part) for part in argv]
if argv[0] not in ALLOWED_PROGRAMS:
raise IsolatedExecutorError("COMMAND_PROGRAM_DENY")
if argv[0] == "/usr/bin/python3" and "-m" in argv:
index = argv.index("-m")
if index + 1 >= len(argv) or argv[index + 1] not in {"pytest", "py_compile"}:
raise IsolatedExecutorError("PYTHON_MODULE_DENY")
if argv[0] == "/usr/bin/git" and any(
token in {"push", "update-ref", "commit", "merge", "rebase", "reset", "clean", "branch"}
for token in argv[1:]
):
raise IsolatedExecutorError("GIT_MUTATION_DENY")
timeout = int(item.get("timeout_seconds") or 300)
if not 1 <= timeout <= 1800:
raise IsolatedExecutorError("COMMAND_TIMEOUT_INVALID")
cwd = root if item.get("cwd", ".") == "." else resolve_under(root, str(item.get("cwd")))
started = time.monotonic()
completed = subprocess.run(
argv,
cwd=cwd,
capture_output=True,
text=True,
check=False,
timeout=timeout,
env={"PATH": "/usr/bin:/bin", "LC_ALL": "C.UTF-8"},
)
result = {
"label": str(item.get("label") or "command"),
"exit_code": completed.returncode,
"stdout_sha256": sha256(completed.stdout.encode("utf-8", "replace")),
"stderr_sha256": sha256(completed.stderr.encode("utf-8", "replace")),
"duration_ms": int((time.monotonic() - started) * 1000),
}
if completed.returncode != 0:
raise IsolatedExecutorError(
"COMMAND_FAILED:" + result["label"] + ":" + str(completed.returncode)
)
return result
def apply_changes(request: Mapping[str, Any], root: Path) -> list[str]:
changes = request.get("change_set")
if not isinstance(changes, list) or not changes:
raise IsolatedExecutorError("CHANGE_SET_INVALID")
changed: list[str] = []
for change in changes:
if not isinstance(change, Mapping):
raise IsolatedExecutorError("CHANGE_INVALID")
relative = safe_relative(change.get("path"))
path = resolve_under(root, relative)
current = path.read_bytes() if path.is_file() else None
expected = change.get("expected_sha256")
if expected is not None and (current is None or sha256(current) != expected):
raise IsolatedExecutorError("SOURCE_DRIFT:" + relative)
operation = change.get("operation")
if operation == "write_file":
content = change.get("content")
if not isinstance(content, str):
raise IsolatedExecutorError("WRITE_CONTENT_INVALID")
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8")
elif operation == "replace_text":
if current is None:
raise IsolatedExecutorError("REPLACE_SOURCE_MISSING")
old = str(change.get("old"))
new = str(change.get("new"))
count = int(change.get("count", 1))
text = current.decode("utf-8")
if text.count(old) != count:
raise IsolatedExecutorError("REPLACE_COUNT_INVALID:" + relative)
path.write_text(text.replace(old, new), encoding="utf-8")
elif operation == "delete_file":
if path.exists():
path.unlink()
else:
raise IsolatedExecutorError("CHANGE_OPERATION_INVALID")
changed.append(relative)
return sorted(set(changed))
def changed_bundle(root: Path, changed: list[str]) -> list[dict[str, Any]]:
files = []
for relative in changed:
path = resolve_under(root, relative)
if path.exists():
data = path.read_bytes()
if len(data) > MAX_FILE_BYTES:
raise IsolatedExecutorError("CHANGED_FILE_TOO_LARGE:" + relative)
files.append({
"path": relative,
"deleted": False,
"sha256": sha256(data),
"content_b64": base64.b64encode(data).decode("ascii"),
})
else:
files.append({"path": relative, "deleted": True, "sha256": None, "content_b64": None})
return files
def execute_request(request_path: Path, result_path: Path) -> None:
assert_uncredentialed()
request = json.loads(request_path.read_text(encoding="utf-8"))
if request.get("schema") != "aios.isolated-execution.request.v1":
raise IsolatedExecutorError("REQUEST_SCHEMA_INVALID")
request_id = request.get("request_id")
if not isinstance(request_id, str) or not REQUEST_ID_RE.fullmatch(request_id):
raise IsolatedExecutorError("REQUEST_ID_INVALID")
source_archive = Path(str(request.get("source_archive") or ""))
if not source_archive.is_absolute():
raise IsolatedExecutorError("SOURCE_ARCHIVE_PATH_INVALID")
expected_archive_sha = str(request.get("source_archive_sha256") or "")
if sha256(source_archive.read_bytes()) != expected_archive_sha:
raise IsolatedExecutorError("SOURCE_ARCHIVE_SHA_MISMATCH")
with tempfile.TemporaryDirectory(prefix="aios-isolated-executor-") as temporary:
root = Path(temporary) / "workspace"
root.mkdir()
extract_source(source_archive, root)
changed = apply_changes(request, root)
commands = [
run_command(item, root)
for item in request.get("verification", [])
if isinstance(item, Mapping)
]
result = {
"schema": "aios.isolated-execution.result.v1",
"request_id": request_id,
"request_sha256": sha256(canonical(request)),
"changed_files": changed_bundle(root, changed),
"verification": commands,
"credential_isolation": {
"credential_env_absent": True,
"credential_mount_inaccessible": True,
"governed_bridge_unreachable": True,
},
}
result["result_sha256"] = sha256(canonical(result))
temporary_result = result_path.with_suffix(".tmp")
temporary_result.write_text(json.dumps(result, sort_keys=True, separators=(",", ":")), encoding="utf-8")
os.replace(temporary_result, result_path)
def serve(root: Path) -> None:
assert_uncredentialed()
requests = root / "requests"
results = root / "results"
failed = root / "failed"
for path in (requests, results, failed):
path.mkdir(parents=True, exist_ok=True)
while True:
for request in sorted(requests.glob("*.json")):
if request.is_symlink():
continue
result = results / request.name
try:
execute_request(request, result)
request.unlink()
except Exception as exc:
(failed / request.name).write_text(
json.dumps({"error": str(exc) or type(exc).__name__}, sort_keys=True),
encoding="utf-8",
)
request.unlink(missing_ok=True)
time.sleep(0.5)
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--root", default="/var/lib/aios-online-agent-executor")
parser.add_argument("--request")
parser.add_argument("--result")
args = parser.parse_args()
if args.request and args.result:
execute_request(Path(args.request), Path(args.result))
else:
serve(Path(args.root))
+216 -6
View File
@@ -4,6 +4,7 @@ import hashlib, hmac, json, os, re, subprocess, sys, time, urllib.parse, urllib.
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from baseline_authority import BaselineAuthority, BaselineAuthorityError
from silent_release_controller import ReleaseCandidate
from typing import Any
try:
@@ -28,11 +29,18 @@ from orchestrator.action_monitor import ActionMonitor
from silent_release_hardening import (
ApprovalKeyRing,
ApprovalRevocationStore,
BuilderAttestation,
BuilderAttestationKeyRing,
BuilderAttestationVerifier,
CapabilityAuthorizationError,
ReleasePlan,
SagaConflict,
SignedApprovalVerifier,
SignedReleasePlanApproval,
SilentReleaseActionService,
TrustedCandidateBuilderUnavailable,
TrustedCandidateBundle,
TrustedReleasePolicy,
)
try:
from aios_gitops_job_controller import GitOpsJobController, JobError, ServiceUnavailable, JobStore
@@ -82,13 +90,34 @@ SILENT_RELEASE_STATE_ROOT = Path(os.getenv(
'AIOS_SILENT_RELEASE_STATE_ROOT',
'/var/lib/aios-gitops/silent-release',
))
SILENT_RELEASE_EXECUTION_ENABLED = str(os.getenv(
'AIOS_SILENT_RELEASE_EXECUTION_ENABLED', 'false'
)).lower() in {'1', 'true', 'yes', 'on'}
# This immutable staging API can prepare only a shadow candidate. An
# environment value must not turn its HTTP process into a release executor.
SILENT_RELEASE_EXECUTION_ENABLED = False
SILENT_RELEASE_PUBLIC_KEYS = Path(os.getenv(
'AIOS_SILENT_RELEASE_PUBLIC_KEYS',
'/etc/aios-gitops/silent-release-owner-public-keys.json',
))
TRUSTED_BUILDER_ENABLED = str(os.getenv(
'AIOS_TRUSTED_CANDIDATE_BUILDER_ENABLED', 'false'
)).lower() in {'1', 'true', 'yes', 'on'}
TRUSTED_BUILDER_POLICY = Path(os.getenv(
'AIOS_TRUSTED_CANDIDATE_BUILDER_POLICY', ''
))
TRUSTED_BUILDER_CREDENTIALS_DIRECTORY = Path(os.getenv(
'AIOS_TRUSTED_CANDIDATE_BUILDER_CREDENTIALS_DIRECTORY',
os.getenv('CREDENTIALS_DIRECTORY', ''),
))
TRUSTED_BUILDER_SOCKET = Path(os.getenv(
'AIOS_TRUSTED_CANDIDATE_BUILDER_SOCKET',
'/run/aios-trusted-candidate-builder.sock',
))
TRUSTED_BUILDER_PUBLIC_KEYRING = Path(os.getenv(
'AIOS_TRUSTED_CANDIDATE_BUILDER_PUBLIC_KEYRING', ''
))
TRUSTED_RELEASE_POLICY = Path(os.getenv(
'AIOS_SILENT_RELEASE_TRUSTED_POLICY',
str(WS / 'config' / 'gitops' / 'silent-release-policy.json'),
))
monitor = ActionMonitor()
BASE_MODELS = [
@@ -166,6 +195,86 @@ def _file_sha256(path: Path) -> str:
digest.update(chunk)
return digest.hexdigest()
def _trusted_release_policy_from_file(path: Path) -> TrustedReleasePolicy:
payload = json.loads(path.read_text(encoding='utf-8'))
required = {
'services', 'release_roots', 'cutover_links', 'dropin_paths',
'authoritative_git_dirs', 'repository_identities',
}
if not isinstance(payload, dict) or set(payload) != required:
raise ValueError('trusted release policy schema is invalid')
values = {}
for name in required:
raw = payload[name]
if not isinstance(raw, list) or not raw or not all(
isinstance(item, str) and item for item in raw
):
raise ValueError('trusted release policy values are invalid')
values[name] = frozenset(raw)
return TrustedReleasePolicy(**values)
class _HostTrustedCandidateBuilderAdapter:
"""Maps only a complete host-owned silent-release bundle response.
The API never manufactures candidate evidence or a builder signature. A
host builder protocol that returns a different response is therefore
unavailable, rather than silently downgraded to caller-owned evidence.
"""
def __init__(self, client):
self._client = client
def build(self, intent):
import secrets
from aios_trusted_candidate_builder import BuilderError
try:
if not isinstance(intent, dict):
raise TypeError('intent must be a mapping')
version = str(intent.get('version') or 'V2R2C2')
# A createReleaseIntent request is intentionally identity-light.
# Candidate identity is resolved by the host builder policy, not by
# the caller. Explicit release-worker requests may still carry it.
candidate_id = str(intent.get('candidate_id') or '')
source_commit = str(intent.get('source_commit') or '')
if not version or not source_commit:
raise KeyError('version/source_commit')
except (KeyError, TypeError) as exc:
raise CapabilityAuthorizationError(
'trusted builder requires a release intent or worker request'
) from exc
try:
response = self._client.build(
version,
candidate_id,
source_commit,
nonce=secrets.token_hex(32),
)
except BuilderError as exc:
raise TrustedCandidateBuilderUnavailable(
'trusted builder request failed'
) from exc
raw_bundle = response.get('silent_release_bundle')
if not isinstance(raw_bundle, dict):
raise CapabilityAuthorizationError(
'trusted builder response is not a silent release bundle'
)
try:
return TrustedCandidateBundle(
candidate=ReleaseCandidate.from_dict(raw_bundle['candidate']),
release_plan=ReleasePlan.from_dict(raw_bundle['release_plan']),
checks=raw_bundle['checks'],
attestation=BuilderAttestation.from_dict(
raw_bundle['attestation'], require_fingerprint=True
),
)
except (KeyError, TypeError, ValueError) as exc:
raise CapabilityAuthorizationError(
'trusted builder response is invalid'
) from exc
def _build_silent_release_service() -> SilentReleaseActionService:
verifier=None
if SILENT_RELEASE_PUBLIC_KEYS.is_file():
@@ -175,12 +284,88 @@ def _build_silent_release_service() -> SilentReleaseActionService:
SILENT_RELEASE_STATE_ROOT / 'revocations'
),
)
policy_path=WS / 'config' / 'gitops' / 'silent-release-policy.json'
policy_path=TRUSTED_RELEASE_POLICY
policy_sha=_file_sha256(policy_path) if policy_path.is_file() else ''
candidate_builder = None
builder_attestation_verifier = None
trusted_policy = None
builder_readiness_probe = None
if TRUSTED_BUILDER_ENABLED:
# Every path is opt-in. In particular, this API receives its own
# client HMAC credential, never the builder's signing private key.
if not (
TRUSTED_BUILDER_POLICY.is_file()
and TRUSTED_BUILDER_CREDENTIALS_DIRECTORY.is_dir()
and TRUSTED_BUILDER_PUBLIC_KEYRING.is_file()
and TRUSTED_RELEASE_POLICY.is_file()
):
raise CapabilityAuthorizationError(
'trusted builder configuration is incomplete'
)
try:
from aios_trusted_candidate_builder import (
BuilderKeyring,
TrustedCandidateBuilderConfig,
UnixTrustedCandidateBuilderClient,
load_builder_policy,
readiness_probe,
)
builder_policy = load_builder_policy(TRUSTED_BUILDER_POLICY)
builder_config = TrustedCandidateBuilderConfig.from_policy(
builder_policy
)
client_hmac = (
TRUSTED_BUILDER_CREDENTIALS_DIRECTORY / 'builder_hmac'
).read_bytes().strip()
client_keyring = BuilderKeyring(builder_hmac=client_hmac)
client = UnixTrustedCandidateBuilderClient(
builder_config,
client_keyring,
socket_path=TRUSTED_BUILDER_SOCKET,
)
candidate_builder = _HostTrustedCandidateBuilderAdapter(client)
builder_attestation_verifier = BuilderAttestationVerifier(
BuilderAttestationKeyRing.from_file(
TRUSTED_BUILDER_PUBLIC_KEYRING
)
)
trusted_policy = _trusted_release_policy_from_file(
TRUSTED_RELEASE_POLICY
)
def builder_readiness_probe():
if not TRUSTED_BUILDER_SOCKET.exists():
return {'status': 'UNAVAILABLE'}
remote = dict(readiness_probe(client))
if remote.get('policy_sha256') != builder_config.policy_sha256:
return {'status': 'UNAVAILABLE'}
builder_attestation_verifier.keyring.require_active_identity(
key_id=str(remote.get('builder_key_id') or ''),
generation=remote.get('builder_key_generation'),
algorithm=str(remote.get('signature_algorithm') or ''),
purpose=str(remote.get('signature_purpose') or ''),
)
return {
**remote,
'hmac_credential_loaded': True,
'peer_verification_configured': True,
'public_key_trusted': True,
'policy_loaded': True,
}
except Exception as exc:
raise CapabilityAuthorizationError(
'trusted builder configuration is invalid'
) from exc
return SilentReleaseActionService(
SILENT_RELEASE_STATE_ROOT,
execution_enabled=SILENT_RELEASE_EXECUTION_ENABLED,
approval_verifier=verifier,
candidate_builder=candidate_builder,
builder_attestation_verifier=builder_attestation_verifier,
trusted_policy=trusted_policy,
builder_readiness_probe=builder_readiness_probe,
allow_legacy_candidate_input=False,
expected_repository=REPOSITORY_IDENTITY,
policy_sha256=policy_sha,
)
@@ -343,6 +528,15 @@ def publication_request_from_query(q: dict[str, list[str]]) -> PublicationReques
(q['approval_plan_id'])[0],
)
def governed_release_query_payload(q: dict[str, list[str]]) -> dict[str, str]:
request = publication_request_from_query(q)
return {
'version': request.version,
'target_oid': request.target_oid,
'evidence_closure_oid': request.evidence_closure_oid,
'approval_plan_id': request.approval_plan_id,
}
def required_authoritative_git_dir() -> Path:
snapshot = repository_snapshot(
git_dir=AUTHORITATIVE_GIT_DIR,
@@ -466,8 +660,11 @@ class H(BaseHTTPRequestHandler):
q=urllib.parse.parse_qs(u.query)
try:
if path in ('/health','/api/health'):
builder_readiness = silent_release_service().get_builder_readiness()
return self.sendj({
'status': 'ok',
'process_liveness': 'PASS',
'builder_readiness': builder_readiness,
'service': os.getenv('AIOS_GITOPS_MODEL_API_SERVICE', 'aios-gitops-model-api-v2r2c1-staging.service'),
'workspace': str(WS),
'version': 'V2R2C1',
@@ -706,8 +903,15 @@ class H(BaseHTTPRequestHandler):
if path == '/createReleaseCandidate':
a=self.need('release:candidate')
if not a: return
service = silent_release_service()
readiness = service.get_builder_readiness()
if readiness.get('status') != 'READY':
return self.sendj({
'error': 'TRUSTED_BUILDER_UNAVAILABLE',
'builder_readiness': readiness,
}, 503)
return self.sendj(
silent_release_service().create_release_candidate(data)
service.create_release_candidate(data)
)
if path == '/createReleaseIntent':
a=self.need('release:candidate')
@@ -747,6 +951,13 @@ class H(BaseHTTPRequestHandler):
error=redact(e)
monitor.record_failure('do_POST', error, {'path':path})
return self.sendj({'error':error},403)
except TrustedCandidateBuilderUnavailable:
monitor.record_failure(
'do_POST', 'TRUSTED_BUILDER_UNAVAILABLE', {'path': path}
)
return self.sendj(
{'error': 'TRUSTED_BUILDER_UNAVAILABLE'}, 503
)
except SagaConflict as e:
error=redact(e)
monitor.record_failure('do_POST', error, {'path':path})
@@ -1019,4 +1230,3 @@ def main():
print(f'AIOS ChatGPT GitOps adapter listening on {HOST}:{PORT}')
ThreadingHTTPServer((HOST,PORT), H).serve_forever()
if __name__ == '__main__': main()
@@ -1,301 +0,0 @@
"""Durable checkpoint journal for the generic Online Agent worker.
The JSON task state remains a fast cache. This SQLite journal is the durable,
append-only evidence used for restart/resume/replay decisions.
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import json
from pathlib import Path
import sqlite3
from typing import Any, Mapping
PHASE_ORDER = {
"claimed": 10,
"workspace_prepared": 20,
"verification_passed": 30,
"source_capture_done": 40,
"commit_done": 40,
"reconciliation_required": 45,
"result_pending": 50,
"result_submitted": 60,
}
class CheckpointError(RuntimeError):
pass
def canonical(value: object) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False, default=str).encode("utf-8")
def sha256_json(value: object) -> str:
return hashlib.sha256(canonical(value)).hexdigest()
def now() -> str:
return datetime.now(timezone.utc).isoformat()
@dataclass(frozen=True)
class CheckpointBinding:
task_id: str
attempt: int
execution_digest: str
job_id: str
repository: str
base_oid: str
expected_branch_oid: str
target_branch: str
generation: int
def material(self) -> dict[str, Any]:
return {
"task_id": self.task_id,
"attempt": self.attempt,
"execution_digest": self.execution_digest,
"job_id": self.job_id,
"repository": self.repository,
"base_oid": self.base_oid,
"expected_branch_oid": self.expected_branch_oid,
"target_branch": self.target_branch,
"generation": self.generation,
}
@property
def binding_sha256(self) -> str:
return sha256_json(self.material())
class CheckpointStore:
def __init__(self, path: Path) -> None:
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
with self._connect() as connection:
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS checkpoints(
task_id TEXT NOT NULL,
attempt INTEGER NOT NULL,
sequence INTEGER NOT NULL,
phase TEXT NOT NULL,
phase_order INTEGER NOT NULL,
execution_digest TEXT NOT NULL,
job_id TEXT NOT NULL,
repository TEXT NOT NULL,
base_oid TEXT NOT NULL,
expected_branch_oid TEXT NOT NULL,
target_branch TEXT NOT NULL,
generation INTEGER NOT NULL,
binding_sha256 TEXT NOT NULL,
payload_json TEXT NOT NULL,
payload_sha256 TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY(task_id, attempt, sequence)
);
CREATE INDEX IF NOT EXISTS idx_checkpoints_task_latest
ON checkpoints(task_id, attempt DESC, sequence DESC);
CREATE UNIQUE INDEX IF NOT EXISTS idx_checkpoints_exact_replay
ON checkpoints(task_id, attempt, phase, payload_sha256);
CREATE TABLE IF NOT EXISTS checkpoint_heads(
task_id TEXT PRIMARY KEY,
attempt INTEGER NOT NULL,
sequence INTEGER NOT NULL,
phase TEXT NOT NULL,
phase_order INTEGER NOT NULL,
execution_digest TEXT NOT NULL,
binding_sha256 TEXT NOT NULL,
payload_sha256 TEXT NOT NULL,
updated_at TEXT NOT NULL
);
"""
)
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA synchronous=FULL")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@staticmethod
def _phase_order(phase: str) -> int:
try:
return PHASE_ORDER[phase]
except KeyError as exc:
raise CheckpointError("CHECKPOINT_PHASE_INVALID") from exc
def append(
self,
binding: CheckpointBinding,
phase: str,
payload: Mapping[str, Any],
) -> dict[str, Any]:
order = self._phase_order(phase)
payload_value = dict(payload)
payload_sha = sha256_json(payload_value)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
replay = connection.execute(
"""
SELECT * FROM checkpoints
WHERE task_id=? AND attempt=? AND phase=? AND payload_sha256=?
""",
(binding.task_id, binding.attempt, phase, payload_sha),
).fetchone()
if replay is not None:
connection.commit()
return dict(replay)
head = connection.execute(
"SELECT * FROM checkpoint_heads WHERE task_id=?",
(binding.task_id,),
).fetchone()
if head is not None:
head_attempt = int(head["attempt"])
head_order = int(head["phase_order"])
if binding.attempt < head_attempt:
raise CheckpointError("CHECKPOINT_ATTEMPT_REGRESSION")
if binding.attempt == head_attempt:
if head["execution_digest"] != binding.execution_digest:
raise CheckpointError("CHECKPOINT_EXECUTION_DIGEST_CONFLICT")
if head["binding_sha256"] != binding.binding_sha256:
raise CheckpointError("CHECKPOINT_SCOPE_MISMATCH_DENY")
if order < head_order and phase != "reconciliation_required":
raise CheckpointError("CHECKPOINT_PHASE_REGRESSION")
sequence = int(head["sequence"]) + 1
else:
sequence = 1
else:
sequence = 1
created_at = now()
connection.execute(
"""
INSERT INTO checkpoints(
task_id,attempt,sequence,phase,phase_order,execution_digest,job_id,
repository,base_oid,expected_branch_oid,target_branch,generation,
binding_sha256,payload_json,payload_sha256,created_at
) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
""",
(
binding.task_id,
binding.attempt,
sequence,
phase,
order,
binding.execution_digest,
binding.job_id,
binding.repository,
binding.base_oid,
binding.expected_branch_oid,
binding.target_branch,
binding.generation,
binding.binding_sha256,
canonical(payload_value).decode("utf-8"),
payload_sha,
created_at,
),
)
connection.execute(
"""
INSERT INTO checkpoint_heads(
task_id,attempt,sequence,phase,phase_order,execution_digest,
binding_sha256,payload_sha256,updated_at
) VALUES (?,?,?,?,?,?,?,?,?)
ON CONFLICT(task_id) DO UPDATE SET
attempt=excluded.attempt,
sequence=excluded.sequence,
phase=excluded.phase,
phase_order=excluded.phase_order,
execution_digest=excluded.execution_digest,
binding_sha256=excluded.binding_sha256,
payload_sha256=excluded.payload_sha256,
updated_at=excluded.updated_at
""",
(
binding.task_id,
binding.attempt,
sequence,
phase,
order,
binding.execution_digest,
binding.binding_sha256,
payload_sha,
created_at,
),
)
connection.commit()
row = connection.execute(
"SELECT * FROM checkpoints WHERE task_id=? AND attempt=? AND sequence=?",
(binding.task_id, binding.attempt, sequence),
).fetchone()
return dict(row)
def latest(self, task_id: str) -> dict[str, Any] | None:
with self._connect() as connection:
row = connection.execute(
"""
SELECT * FROM checkpoints
WHERE task_id=?
ORDER BY attempt DESC, sequence DESC
LIMIT 1
""",
(task_id,),
).fetchone()
if row is None:
return None
value = dict(row)
value["payload"] = json.loads(value.pop("payload_json"))
return value
def latest_for_attempt(self, task_id: str, attempt: int) -> dict[str, Any] | None:
with self._connect() as connection:
row = connection.execute(
"""
SELECT * FROM checkpoints
WHERE task_id=? AND attempt=?
ORDER BY sequence DESC
LIMIT 1
""",
(task_id, attempt),
).fetchone()
if row is None:
return None
value = dict(row)
value["payload"] = json.loads(value.pop("payload_json"))
return value
def verify_latest_binding(self, binding: CheckpointBinding) -> dict[str, Any] | None:
value = self.latest(binding.task_id)
if value is None:
return None
if int(value["attempt"]) > binding.attempt:
raise CheckpointError("CHECKPOINT_FUTURE_ATTEMPT")
# A previous attempt may be reused only when the immutable execution/scope binding matches.
if value["execution_digest"] != binding.execution_digest:
return None
expected = CheckpointBinding(
task_id=value["task_id"],
attempt=int(value["attempt"]),
execution_digest=value["execution_digest"],
job_id=value["job_id"],
repository=value["repository"],
base_oid=value["base_oid"],
expected_branch_oid=value["expected_branch_oid"],
target_branch=value["target_branch"],
generation=int(value["generation"]),
)
current_scope = binding.material()
historical_scope = expected.material()
current_scope.pop("attempt", None)
historical_scope.pop("attempt", None)
if sha256_json(current_scope) != sha256_json(historical_scope):
raise CheckpointError("CHECKPOINT_SCOPE_MISMATCH_DENY")
return value
@@ -1,463 +0,0 @@
"""Bounded read-only repository interface map for governed Online Agent analysis.
The compact map intentionally returns symbols and short source excerpts rather than a
source archive. It is designed for control-plane consumers that cannot safely ingest a
multi-megabyte repository snapshot.
"""
from __future__ import annotations
import ast
import hashlib
import json
from pathlib import Path
import re
from typing import Any, Iterable, Mapping, Sequence
SCHEMA = "aios.engineering.interface-map.v1"
MAX_RESULT_BYTES = 120 * 1024
TARGET_RESULT_BYTES = 108 * 1024
MAX_MODULES = 40
MAX_SYMBOLS = 80
MAX_TESTS = 40
MAX_EXCERPTS = 25
MAX_EXCERPT_CHARS = 1200
MAX_FILE_BYTES = 2 * 1024 * 1024
ROOTS = (
"active-runtime/app",
"runtime/agent-os",
"tests",
"db/migrations",
"migrations",
"deploy/systemd",
"docs/task-plane",
"docs/superpowers/specs",
)
SUFFIXES = {".py", ".sql", ".service", ".md", ".json", ".yaml", ".yml"}
DEFAULT_TERMS = {
"agent", "worker", "fleet", "scheduler", "scheduling", "task", "claim", "lease",
"heartbeat", "capacity", "concurrency", "priority", "fairness", "queue", "retry",
"backoff", "circuit", "health", "liveness", "scope", "tenant", "project", "trace",
"message", "checkpoint", "recovery", "idempotency", "dispatch", "assignment",
}
FLEET_CONTRACT_TERMS: dict[str, tuple[str, ...]] = {
"FLEET_AGENT_REGISTRATION": ("agent", "register", "worker", "capability", "version"),
"FLEET_CAPACITY_SCHEDULING": ("capacity", "slot", "concurrency", "scheduler", "queue"),
"FLEET_FAIRNESS": ("fair", "priority", "queue", "tenant", "project", "starvation"),
"FLEET_FAULT_ISOLATION": ("failure", "health", "liveness", "timeout", "circuit", "poison"),
"FLEET_WORKER_REASSIGNMENT": ("reassign", "assignment", "claim", "lease", "retry", "worker"),
"FLEET_SCOPE_ISOLATION": ("tenant", "project", "scope", "conversation", "task", "run"),
"FLEET_IDEMPOTENT_DISPATCH": ("idempot", "dispatch", "claim", "lease", "result", "attempt"),
"FLEET_FAILURE_RECOVERY": ("checkpoint", "resume", "recover", "retry", "cleanup", "receipt"),
}
MODEL_GOVERNANCE_CONTRACT_TERMS: dict[str, tuple[str, ...]] = {
"MODEL_PROVIDER_REGISTRY": ("model", "provider", "registry", "endpoint", "capability"),
"MODEL_SELECTION_POLICY": ("model", "select", "route", "policy", "capability"),
"MODEL_VERSION_PINNING": ("model", "version", "pin", "revision", "snapshot"),
"MODEL_BUDGET_ACCOUNTING": ("budget", "token", "cost", "quota", "usage"),
"MODEL_FALLBACK_POLICY": ("fallback", "retry", "route", "provider", "model"),
"MODEL_SCOPE_ISOLATION": ("tenant", "project", "scope", "policy", "model"),
"MODEL_CREDENTIAL_BOUNDARY": ("credential", "secret", "provider", "tool", "token"),
"MODEL_TRACE_AUDIT": ("trace", "audit", "model", "provider", "request"),
"MODEL_EVALUATION_GATE": ("eval", "evaluation", "approval", "quality", "gate"),
"MODEL_ROUTING_RESILIENCE": ("route", "retry", "fallback", "timeout", "health"),
}
PROFILE_CLASSIFICATION = {
"fleet": "P2_INTERFACE_MAP_CAPTURED",
"model_governance": "MODEL_GOVERNANCE_INTERFACE_MAP_CAPTURED",
}
MODEL_PROFILE_MARKERS = {
"model", "provider", "llm", "inference", "routing", "router", "budget",
"token", "cost", "quota", "fallback", "evaluation", "eval", "approval",
"version", "pinning",
}
SENSITIVE_RE = re.compile(
r"(?i)(authorization\s*:\s*bearer\s+\S+|"
r"(?:password|passwd|secret|token|api[_-]?key)\s*[:=]\s*['\"]?[^'\"\s,;]+)"
)
SQL_TABLE_RE = re.compile(
r"(?is)\bcreate\s+table\s+(?:if\s+not\s+exists\s+)?[\"`[]?([A-Za-z_][A-Za-z0-9_.-]*)"
)
class CompactInterfaceMapError(RuntimeError):
pass
def _sha(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def _bounded_excerpt(text: str, start_line: int, end_line: int) -> str:
lines = text.splitlines()
start = max(0, start_line - 2)
end = min(len(lines), max(end_line + 1, start + 1))
excerpt = "\n".join(lines[start:end])
excerpt = SENSITIVE_RE.sub("<redacted-sensitive-source-pattern>", excerpt)
if len(excerpt) > MAX_EXCERPT_CHARS:
excerpt = excerpt[: MAX_EXCERPT_CHARS - 1] + "…"
return excerpt
def _signal_terms(signals: Sequence[str]) -> set[str]:
terms = set(DEFAULT_TERMS)
for signal in signals:
if not isinstance(signal, str):
continue
terms.update(
token.lower()
for token in re.findall(r"[A-Za-z][A-Za-z0-9_.-]{2,63}", signal)
)
return set(sorted(terms)[:96])
def _profile(source_signals: Sequence[str]) -> str:
tokens: set[str] = set()
for signal in source_signals:
if not isinstance(signal, str):
continue
tokens.update(
token.lower()
for token in re.findall(r"[A-Za-z][A-Za-z0-9_.-]{2,63}", signal)
)
model_score = len(tokens & MODEL_PROFILE_MARKERS)
return "model_governance" if model_score >= 2 else "fleet"
def _contract_terms(profile: str) -> Mapping[str, tuple[str, ...]]:
if profile == "model_governance":
return MODEL_GOVERNANCE_CONTRACT_TERMS
if profile == "fleet":
return FLEET_CONTRACT_TERMS
raise CompactInterfaceMapError("INTERFACE_MAP_PROFILE_INVALID")
def _candidate_files(root: Path, terms: set[str]) -> list[dict[str, Any]]:
root = root.resolve()
candidates: list[dict[str, Any]] = []
for relative_root in ROOTS:
directory = (root / relative_root).resolve(strict=False)
if directory == root or root not in directory.parents:
raise CompactInterfaceMapError("INTERFACE_MAP_SCOPE_INVALID")
if not directory.is_dir() or directory.is_symlink():
continue
for path in sorted(directory.rglob("*")):
if path.is_symlink() or not path.is_file() or path.suffix.lower() not in SUFFIXES:
continue
resolved = path.resolve(strict=True)
if root not in resolved.parents:
raise CompactInterfaceMapError("INTERFACE_MAP_PATH_ESCAPE")
data = path.read_bytes()
if len(data) > MAX_FILE_BYTES:
continue
relative = path.relative_to(root).as_posix()
text = data.decode("utf-8", "ignore")
searchable = (relative + "\n" + text[:384000]).lower()
matched = sorted(term for term in terms if term in searchable)
if not matched:
continue
path_bonus = sum(3 for term in terms if term in relative.lower())
test_bonus = 3 if relative.startswith("tests/") else 0
score = min(250, len(matched) + path_bonus + test_bonus)
candidates.append(
{
"path": relative,
"sha256": _sha(data),
"bytes": len(data),
"score": score,
"text": text,
"suffix": path.suffix.lower(),
}
)
candidates.sort(key=lambda item: (-item["score"], item["path"]))
return candidates
def _python_symbols(item: Mapping[str, Any], terms: set[str]) -> list[dict[str, Any]]:
try:
tree = ast.parse(str(item["text"]), filename=str(item["path"]))
except SyntaxError:
return []
result: list[dict[str, Any]] = []
class Visitor(ast.NodeVisitor):
def __init__(self) -> None:
self.stack: list[str] = []
def _record(self, node: ast.AST, kind: str, name: str) -> None:
qualified = ".".join([*self.stack, name]) if self.stack else name
haystack = f"{item['path']} {qualified}".lower()
score = sum(3 if term in haystack else 0 for term in terms)
if name.startswith("test_"):
score += 12
result.append(
{
"path": item["path"],
"symbol": qualified,
"kind": kind,
"line": int(getattr(node, "lineno", 1)),
"end_line": int(getattr(node, "end_lineno", getattr(node, "lineno", 1))),
"score": score + int(item["score"]),
}
)
def visit_ClassDef(self, node: ast.ClassDef) -> Any:
self._record(node, "class", node.name)
self.stack.append(node.name)
self.generic_visit(node)
self.stack.pop()
def visit_FunctionDef(self, node: ast.FunctionDef) -> Any:
self._record(node, "function", node.name)
self.stack.append(node.name)
self.generic_visit(node)
self.stack.pop()
def visit_AsyncFunctionDef(self, node: ast.AsyncFunctionDef) -> Any:
self._record(node, "async_function", node.name)
self.stack.append(node.name)
self.generic_visit(node)
self.stack.pop()
Visitor().visit(tree)
return result
def _surface_ref(symbol: Mapping[str, Any]) -> str:
return f"{symbol['path']}::{symbol['symbol']}"
def _contract_map(
symbols: Sequence[Mapping[str, Any]],
modules: Sequence[Mapping[str, Any]],
contract_terms: Mapping[str, tuple[str, ...]],
) -> dict[str, Any]:
result: dict[str, Any] = {}
module_paths = [str(item["path"]) for item in modules]
for contract, terms in contract_terms.items():
scored: list[tuple[int, str]] = []
for symbol in symbols:
haystack = _surface_ref(symbol).lower()
score = sum(1 for term in terms if term in haystack)
if score:
scored.append((score, _surface_ref(symbol)))
scored.sort(key=lambda item: (-item[0], item[1]))
surfaces = [value for _, value in scored[:6]]
if not surfaces:
for path in module_paths:
score = sum(1 for term in terms if term in path.lower())
if score:
surfaces.append(path)
if len(surfaces) == 6:
break
result[contract] = {
"status": "PARTIAL" if surfaces else "GAP",
"surfaces": surfaces,
}
return result
def _gap_evidence(
symbols: Sequence[Mapping[str, Any]],
profile: str,
) -> tuple[list[str], list[str], list[str]]:
haystack = "\n".join(_surface_ref(item).lower() for item in symbols)
if profile == "model_governance":
invariants = [
"Online Task claim/heartbeat remains the authoritative worker ownership boundary.",
"Provider/model credentials remain outside uncredentialed planner/executor processes.",
"P0-A tenant/project scope and P0-C trace/audit lineage must constrain model routing.",
"P1 retry/checkpoint semantics remain authoritative for model-call execution recovery.",
]
failure_windows: list[str] = []
if "provider" not in haystack and "model" not in haystack:
failure_windows.append("No first-class model/provider registry surface detected.")
if "budget" not in haystack and "cost" not in haystack and "quota" not in haystack:
failure_windows.append("No explicit durable model budget/cost/quota accounting surface detected.")
if "fallback" not in haystack and "route" not in haystack:
failure_windows.append("No explicit model fallback/routing policy surface detected.")
if "eval" not in haystack and "approval" not in haystack:
failure_windows.append("No explicit evaluation/approval gate surface detected for model changes.")
if "version" not in haystack and "pin" not in haystack:
failure_windows.append("No explicit model-version pinning surface detected.")
desired = (
("trace", "P0-C trace/audit lineage"),
("scope", "P0-A tenant/project scope isolation"),
("credential", "P0-C credential governance"),
("checkpoint", "P1 durable execution recovery"),
("message", "P0-B MessageEnvelope/handoff"),
("retry", "P1 retry/idempotency"),
)
else:
invariants = [
"Online Task claim/heartbeat remains the authoritative worker ownership boundary.",
"Governed mutation remains separated from read-only analysis and requires WriterLease.",
"P1 attempt-aware result idempotency and durable checkpoint recovery are reusable fleet primitives.",
"Scope/trace/message lineage must remain attached when work is scheduled or reassigned.",
]
failure_windows = []
if "scheduler" not in haystack:
failure_windows.append("No first-class durable fleet scheduler symbol detected; worker-local polling may still be the scheduling mechanism.")
if "capacity" not in haystack and "slot" not in haystack:
failure_windows.append("No explicit durable worker-capacity/slot primitive detected.")
if "fair" not in haystack and "starvation" not in haystack:
failure_windows.append("No explicit fairness/starvation policy symbol detected.")
if "circuit" not in haystack:
failure_windows.append("No explicit circuit-breaker primitive detected for repeated worker/task failures.")
if "reassign" not in haystack:
failure_windows.append("No explicit worker-reassignment primitive detected beyond lease expiry/retry semantics.")
desired = (
("checkpoint", "P1 durable checkpoint/resume"),
("message", "P0-B MessageEnvelope/handoff"),
("trace", "P0-C trace correlation"),
("scope", "P0-A scope isolation"),
("lease", "claim/WriterLease ownership"),
("heartbeat", "worker liveness"),
("idempot", "retry-safe dispatch/result"),
)
reuse_points: list[str] = []
for token, label in desired:
match = next((item for item in symbols if token in _surface_ref(item).lower()), None)
if match is not None:
reuse_points.append(f"{label}: {_surface_ref(match)}")
return invariants, failure_windows, reuse_points
def _trim_to_budget(value: dict[str, Any]) -> dict[str, Any]:
def size() -> int:
return len(json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":")).encode("utf-8"))
while size() > TARGET_RESULT_BYTES and value["bounded_excerpts"]:
value["bounded_excerpts"].pop()
while size() > TARGET_RESULT_BYTES and len(value["implementation_map"]["functions"]) > 20:
value["implementation_map"]["functions"].pop()
while size() > TARGET_RESULT_BYTES and len(value["implementation_map"]["classes"]) > 15:
value["implementation_map"]["classes"].pop()
while size() > TARGET_RESULT_BYTES and len(value["implementation_map"]["tests"]) > 15:
value["implementation_map"]["tests"].pop()
while size() > TARGET_RESULT_BYTES and len(value["implementation_map"]["modules"]) > 20:
value["implementation_map"]["modules"].pop()
final_size = size()
if final_size > MAX_RESULT_BYTES:
raise CompactInterfaceMapError("INTERFACE_MAP_RESULT_TOO_LARGE")
value["result_bytes"] = final_size
return value
def build_compact_interface_map(
worktree: Path,
source_signals: Sequence[str],
*,
source_commit: str,
) -> dict[str, Any]:
root = Path(worktree)
if not root.is_dir() or root.is_symlink():
raise CompactInterfaceMapError("INTERFACE_MAP_WORKTREE_INVALID")
if re.fullmatch(r"[0-9a-f]{40}", source_commit) is None:
raise CompactInterfaceMapError("INTERFACE_MAP_SOURCE_COMMIT_INVALID")
profile = _profile(source_signals)
terms = _signal_terms(source_signals)
candidates = _candidate_files(root, terms)
if not candidates:
raise CompactInterfaceMapError("INTERFACE_MAP_EMPTY")
module_candidates = candidates[:MAX_MODULES]
modules = [
{
"path": item["path"],
"sha256": item["sha256"],
"bytes": item["bytes"],
"score": item["score"],
}
for item in module_candidates
]
symbols: list[dict[str, Any]] = []
tables: list[dict[str, Any]] = []
systemd_units: list[dict[str, Any]] = []
for item in candidates:
if item["suffix"] == ".py":
symbols.extend(_python_symbols(item, terms))
elif item["suffix"] == ".sql":
for table in SQL_TABLE_RE.findall(str(item["text"])):
tables.append({"path": item["path"], "table": table, "sha256": item["sha256"]})
elif item["suffix"] == ".service":
systemd_units.append({"path": item["path"], "unit": Path(str(item["path"])).name, "sha256": item["sha256"]})
symbols.sort(key=lambda item: (-int(item["score"]), str(item["path"]), str(item["symbol"])))
all_symbols = list(symbols)
test_symbols = [
item for item in all_symbols
if str(item["path"]).startswith("tests/") or str(item["symbol"]).split(".")[-1].startswith("test_")
][: min(MAX_TESTS, 24)]
non_test_symbols = [
item for item in all_symbols
if item not in test_symbols
][: max(0, MAX_SYMBOLS - len(test_symbols))]
symbols = [*test_symbols, *non_test_symbols]
symbols.sort(key=lambda item: (-int(item["score"]), str(item["path"]), str(item["symbol"])))
classes = [
{k: item[k] for k in ("path", "symbol", "line", "end_line")}
for item in symbols
if item["kind"] == "class"
]
functions = [
{k: item[k] for k in ("path", "symbol", "line", "end_line")}
for item in symbols
if item["kind"] in {"function", "async_function"} and not str(item["symbol"]).split(".")[-1].startswith("test_")
]
tests = [
{k: item[k] for k in ("path", "symbol", "line", "end_line")}
for item in symbols
if str(item["path"]).startswith("tests/") or str(item["symbol"]).split(".")[-1].startswith("test_")
][:MAX_TESTS]
excerpts: list[dict[str, Any]] = []
by_path = {str(item["path"]): item for item in candidates}
for symbol in symbols:
if len(excerpts) == MAX_EXCERPTS:
break
item = by_path.get(str(symbol["path"]))
if item is None:
continue
excerpt = _bounded_excerpt(str(item["text"]), int(symbol["line"]), int(symbol["end_line"]))
if not excerpt.strip():
continue
excerpts.append(
{
"path": symbol["path"],
"symbol": symbol["symbol"],
"sha256": item["sha256"],
"excerpt": excerpt,
}
)
invariants, failure_windows, reuse_points = _gap_evidence(all_symbols, profile)
result = {
"schema": SCHEMA,
"profile": profile,
"classification": PROFILE_CLASSIFICATION[profile],
"source_commit": source_commit,
"implementation_map": {
"modules": modules,
"classes": classes[:MAX_SYMBOLS],
"functions": functions[:MAX_SYMBOLS],
"tables": tables[:40],
"systemd_units": systemd_units[:20],
"tests": tests,
"current_invariants": invariants,
"failure_windows": failure_windows,
"reuse_points": reuse_points,
},
"bounded_excerpts": excerpts,
"contracts": _contract_map(all_symbols, modules, _contract_terms(profile)),
}
return _trim_to_budget(result)
@@ -24,7 +24,10 @@ BRANCH_RE = re.compile(r"^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$")
KEY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{7,159}$")
PROTECTED_PREFIXES = ("main", "master", "refs/", "tags/", "tag/", "release/", "baseline/", "production/")
MUTATION_ACTIONS = frozenset({"deterministic"})
SUPPORTED_ACTIONS = frozenset({"deterministic", "source_capture"})
READ_ONLY_ACTIONS = frozenset({"source_capture", "agent_runtime"})
SUPPORTED_ACTIONS = frozenset({"deterministic", "source_capture", "agent_runtime"})
AGENT_RUNTIME_ADAPTERS = frozenset({"hermes", "openclaw"})
MAX_AGENT_RUNTIME_INPUT_BYTES = 64 * 1024
class ExecutionProjectionError(RuntimeError):
@@ -172,6 +175,76 @@ def _source_signals(value: object) -> tuple[str, ...]:
return tuple(result)
def _agent_runtime_input(value: object) -> dict[str, Any]:
if not isinstance(value, Mapping):
_error("AGENT_RUNTIME_INPUT_REQUIRED")
raw = dict(value)
allowed = {
"goal", "instructions", "success_criteria", "constraints",
"scope", "trace_id",
}
if set(raw) - allowed:
_error("AGENT_RUNTIME_INPUT_FIELDS_INVALID")
goal = _string(raw.get("goal"), "AGENT_RUNTIME_GOAL_REQUIRED")
instructions = raw.get("instructions", "")
if not isinstance(instructions, str):
_error("AGENT_RUNTIME_INSTRUCTIONS_INVALID")
criteria = raw.get("success_criteria", [])
if not isinstance(criteria, list) or len(criteria) > 32:
_error("AGENT_RUNTIME_SUCCESS_CRITERIA_INVALID")
normalized_criteria: list[dict[str, str]] = []
for item in criteria:
if not isinstance(item, Mapping):
_error("AGENT_RUNTIME_SUCCESS_CRITERIA_INVALID")
criterion_id = _string(item.get("id"), "AGENT_RUNTIME_SUCCESS_CRITERIA_INVALID")
description = _string(
item.get("description"), "AGENT_RUNTIME_SUCCESS_CRITERIA_INVALID"
)
normalized_criteria.append(
{"id": criterion_id[:120], "description": description[:1000]}
)
constraints = raw.get("constraints", {})
if not isinstance(constraints, Mapping):
_error("AGENT_RUNTIME_CONSTRAINTS_INVALID")
sensitive_markers = ("token", "secret", "password", "credential", "authorization", "api_key", "apikey")
for key in constraints:
lowered = str(key).lower()
if any(marker in lowered for marker in sensitive_markers):
_error("AGENT_RUNTIME_SECRET_FIELD_DENY")
scope = raw.get("scope", {})
if not isinstance(scope, Mapping):
_error("AGENT_RUNTIME_SCOPE_INVALID")
scope = {
"scope_type": str(scope.get("scope_type") or "legacy"),
"tenant_id": scope.get("tenant_id"),
"project_id": scope.get("project_id"),
}
if scope["scope_type"] not in {"legacy", "system", "tenant"}:
_error("AGENT_RUNTIME_SCOPE_INVALID")
if scope["scope_type"] == "tenant":
if not isinstance(scope["tenant_id"], str) or not scope["tenant_id"].strip():
_error("AGENT_RUNTIME_SCOPE_INVALID")
elif scope["tenant_id"] not in (None, "") or scope["project_id"] not in (None, ""):
_error("AGENT_RUNTIME_SCOPE_INVALID")
trace_id = str(raw.get("trace_id") or "").strip()
if not trace_id or len(trace_id) > 255:
_error("AGENT_RUNTIME_TRACE_ID_INVALID")
normalized = {
"goal": goal[:12000],
"instructions": instructions[:32000],
"success_criteria": normalized_criteria,
"constraints": dict(constraints),
"scope": scope,
"trace_id": trace_id,
}
if len(_canonical(normalized).encode("utf-8")) > MAX_AGENT_RUNTIME_INPUT_BYTES:
_error("AGENT_RUNTIME_INPUT_TOO_LARGE")
return normalized
def _capabilities(value: object) -> tuple[str, ...]:
if value is None:
return ()
@@ -288,6 +361,22 @@ def validate_execution_envelope(
signals = list(_source_signals(envelope.get("source_signals") or []))
if envelope.get("source_signals") != signals:
_error("SOURCE_CAPTURE_SIGNALS_INVALID")
if envelope.get("runtime_input") not in (None, {}):
_error("SOURCE_CAPTURE_RUNTIME_INPUT_DENY")
elif action == "agent_runtime":
if changes or verification:
_error("AGENT_RUNTIME_MUTATION_DENY")
adapter = str(envelope.get("adapter") or "")
if adapter not in AGENT_RUNTIME_ADAPTERS:
_error("AGENT_RUNTIME_ADAPTER_INVALID")
runtime_input = _agent_runtime_input(envelope.get("runtime_input"))
if envelope.get("runtime_input") != runtime_input:
_error("AGENT_RUNTIME_INPUT_NOT_CANONICAL")
required_runtime_capability = f"agent.runtime.{adapter}"
if required_runtime_capability not in capability_scope:
_error("AGENT_RUNTIME_CAPABILITY_REQUIRED")
if envelope.get("source_signals") not in (None, []):
_error("AGENT_RUNTIME_SOURCE_SIGNALS_DENY")
elif not changes:
_error("CHANGE_SET_INVALID")
declared = str(envelope.get("execution_digest") or "")
@@ -329,11 +418,15 @@ def project_execution_envelope(
"schema": EXECUTION_SCHEMA,
**binding.public(),
"action": action,
"adapter": str(request.get("adapter") or ("deterministic" if action == "deterministic" else "source_capture")),
"adapter": str(
request.get("adapter")
or ("deterministic" if action == "deterministic" else "source_capture")
),
"capability_scope": list(capabilities),
"change_set": list(request.get("change_set") or []),
"verification": list(request.get("verification") or []),
"source_signals": list(_source_signals(request.get("source_signals") or [])) if action == "source_capture" else [],
"runtime_input": _agent_runtime_input(request.get("runtime_input")) if action == "agent_runtime" else {},
"governance": governance,
"workspace": {
"authority_git_dir": "/data/aios-gitops/repo.git",
@@ -352,7 +445,45 @@ def project_online_task(
) -> dict[str, Any]:
request = dict(task_request)
required = _capabilities(request.get("required_capabilities"))
execution = project_execution_envelope(binding, execution_request, required_capabilities=required)
execution_value = dict(execution_request or {})
if str(execution_value.get("action") or "source_capture") == "agent_runtime":
adapter = str(execution_value.get("adapter") or "")
if adapter not in AGENT_RUNTIME_ADAPTERS:
_error("AGENT_RUNTIME_ADAPTER_INVALID")
runtime_input = execution_value.get("runtime_input")
tenant_id = request.get("tenant_id")
project_id = request.get("project_id")
declared_scope = str(
request.get("scope_type")
or ("tenant" if isinstance(tenant_id, str) and tenant_id.strip() else "legacy")
)
if runtime_input is None:
runtime_input = {
"goal": request.get("goal"),
"instructions": request.get("instructions") or "",
"success_criteria": request.get("success_criteria") or [],
"constraints": request.get("constraints") or {},
}
elif not isinstance(runtime_input, Mapping):
_error("AGENT_RUNTIME_INPUT_REQUIRED")
runtime_input = dict(runtime_input)
# Scope/trace are authoritative Task identity, never caller-selected runtime hints.
runtime_input["scope"] = {
"scope_type": declared_scope,
"tenant_id": tenant_id,
"project_id": project_id,
}
runtime_input["trace_id"] = str(
request.get("trace_id")
or request.get("run_id")
or binding.task_id
)
execution_value["runtime_input"] = runtime_input
execution = project_execution_envelope(
binding,
execution_value,
required_capabilities=required,
)
metadata = dict(request.get("metadata") or {})
if "execution" in metadata:
validate_execution_envelope(_object(metadata["execution"], "EXECUTION_ENVELOPE_INVALID"), binding=binding)
@@ -1,187 +0,0 @@
"""Durable, authority-bound deterministic execution plans for Online Agent Tasks."""
from __future__ import annotations
from contextlib import contextmanager
from dataclasses import dataclass
import fcntl
import hashlib
import json
from pathlib import Path
from typing import Any, Mapping
from aios_online_agent_execution import (
GovernedExecutionBinding,
project_execution_envelope,
)
class ExecutionPlanError(RuntimeError):
pass
def _canonical(value: object) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def _digest(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _task_key(task_id: str) -> str:
readable = "".join(ch if ch.isalnum() or ch in "._-" else "-" for ch in task_id).strip("-")[:72] or "task"
return readable + "-" + hashlib.sha256(task_id.encode("utf-8")).hexdigest()[:24]
@dataclass(frozen=True)
class ExecutionPlanRecord:
task_id: str
job_id: str
source_head_oid: str
generation: int
idempotency_key: str
plan_digest: str
execution: dict[str, Any]
def public(self) -> dict[str, Any]:
return {
"schema": "aios.online-agent.plan-record.v1",
"task_id": self.task_id,
"job_id": self.job_id,
"source_head_oid": self.source_head_oid,
"generation": self.generation,
"idempotency_key": self.idempotency_key,
"plan_digest": self.plan_digest,
"execution": dict(self.execution),
}
class ExecutionPlanStore:
def __init__(self, root: Path) -> None:
self.root = Path(root)
self.root.mkdir(parents=True, exist_ok=True)
def _path(self, task_id: str) -> Path:
return self.root / (_task_key(task_id) + ".json")
@contextmanager
def _lock(self, task_id: str):
path = self.root / (_task_key(task_id) + ".lock")
with path.open("a+b") as handle:
fcntl.flock(handle.fileno(), fcntl.LOCK_EX)
try:
yield
finally:
fcntl.flock(handle.fileno(), fcntl.LOCK_UN)
@staticmethod
def _source_head(binding: GovernedExecutionBinding) -> str:
return binding.expected_branch_oid if binding.branch_mode == "CONTINUE_EXISTING" else binding.base_oid
def _read(self, task_id: str) -> ExecutionPlanRecord | None:
path = self._path(task_id)
if not path.is_file() or path.is_symlink():
return None
value = json.loads(path.read_text(encoding="utf-8"))
if not isinstance(value, Mapping) or value.get("schema") != "aios.online-agent.plan-record.v1":
raise ExecutionPlanError("EXECUTION_PLAN_RECORD_INVALID")
execution = value.get("execution")
if not isinstance(execution, Mapping):
raise ExecutionPlanError("EXECUTION_PLAN_RECORD_INVALID")
return ExecutionPlanRecord(
task_id=str(value.get("task_id") or ""),
job_id=str(value.get("job_id") or ""),
source_head_oid=str(value.get("source_head_oid") or ""),
generation=int(value.get("generation") or 0),
idempotency_key=str(value.get("idempotency_key") or ""),
plan_digest=str(value.get("plan_digest") or ""),
execution=dict(execution),
)
def register(
self,
binding: GovernedExecutionBinding,
request: Mapping[str, Any],
) -> dict[str, Any]:
with self._lock(binding.task_id):
return self._register_locked(binding, request)
def _register_locked(
self,
binding: GovernedExecutionBinding,
request: Mapping[str, Any],
) -> dict[str, Any]:
if request.get("schema") != "aios.online-agent.plan.v1":
raise ExecutionPlanError("EXECUTION_PLAN_SCHEMA_INVALID")
if request.get("task_id") != binding.task_id or request.get("job_id") != binding.job_id:
raise ExecutionPlanError("EXECUTION_PLAN_BINDING_CONFLICT")
if int(request.get("generation") or 0) != binding.generation:
raise ExecutionPlanError("EXECUTION_PLAN_GENERATION_CONFLICT")
source_head = str(request.get("source_head_oid") or "")
if source_head != self._source_head(binding):
raise ExecutionPlanError("EXECUTION_PLAN_SOURCE_HEAD_CONFLICT")
key = str(request.get("idempotency_key") or "")
if len(key) < 8:
raise ExecutionPlanError("EXECUTION_PLAN_IDEMPOTENCY_INVALID")
execution_request = request.get("execution")
if not isinstance(execution_request, Mapping) or execution_request.get("action") != "deterministic":
raise ExecutionPlanError("EXECUTION_PLAN_DETERMINISTIC_REQUIRED")
envelope = project_execution_envelope(binding, execution_request)
material = {
"task_id": binding.task_id,
"job_id": binding.job_id,
"source_head_oid": source_head,
"generation": binding.generation,
"idempotency_key": key,
"execution": dict(execution_request),
}
plan_digest = _digest(material)
existing = self._read(binding.task_id)
expected_previous = request.get("expected_previous_plan_digest")
if existing is not None:
if existing.idempotency_key == key:
if existing.plan_digest != plan_digest:
raise ExecutionPlanError("EXECUTION_PLAN_IDEMPOTENCY_CONFLICT")
return {**existing.public(), "idempotent": True, "execution_digest": envelope["execution_digest"]}
if expected_previous != existing.plan_digest:
raise ExecutionPlanError("EXECUTION_PLAN_REPLACEMENT_CAS_MISMATCH")
if existing.source_head_oid != source_head:
raise ExecutionPlanError("EXECUTION_PLAN_STALE_REPLACEMENT_DENY")
elif expected_previous not in (None, ""):
raise ExecutionPlanError("EXECUTION_PLAN_REPLACEMENT_CAS_MISMATCH")
record = ExecutionPlanRecord(
task_id=binding.task_id,
job_id=binding.job_id,
source_head_oid=source_head,
generation=binding.generation,
idempotency_key=key,
plan_digest=plan_digest,
execution=dict(execution_request),
)
path = self._path(binding.task_id)
temporary = path.with_suffix(".tmp")
temporary.write_text(
json.dumps(record.public(), sort_keys=True, separators=(",", ":")),
encoding="utf-8",
)
temporary.replace(path)
return {**record.public(), "idempotent": False, "execution_digest": envelope["execution_digest"]}
def resolve(self, binding: GovernedExecutionBinding) -> dict[str, Any] | None:
record = self._read(binding.task_id)
if record is None:
return None
if (
record.task_id != binding.task_id
or record.job_id != binding.job_id
or record.generation != binding.generation
):
raise ExecutionPlanError("EXECUTION_PLAN_BINDING_CONFLICT")
if record.source_head_oid != self._source_head(binding):
raise ExecutionPlanError("EXECUTION_PLAN_SOURCE_HEAD_DRIFT")
envelope = project_execution_envelope(binding, record.execution)
return {
"execution": dict(record.execution),
"plan_digest": record.plan_digest,
"execution_digest": envelope["execution_digest"],
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,967 @@
"""Bounded, read-only discovery over a verified immutable Git repository.
The module is deliberately independent from Task, lease, credential, worktree,
and write code. Every public operation verifies the exact repository identity,
resolves a floating ref once to a full commit OID, and then reads only that
immutable snapshot.
"""
from __future__ import annotations
import base64
from collections.abc import Iterable, Iterator, Mapping
import fnmatch
import hashlib
import hmac
import json
import os
from pathlib import Path
import re
import subprocess
import time
from authoritative_git import AuthoritativeGitError, verify_repository
_OID_RE = re.compile(r"^[0-9a-f]{40}$")
_REF_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._/-]{0,239}$")
_IDENTITY_RE = re.compile(
r"^[A-Za-z0-9][A-Za-z0-9._-]{0,127}/[A-Za-z0-9][A-Za-z0-9._-]{0,127}$"
)
_READ_ONLY_COMMANDS = frozenset({"cat-file", "diff", "ls-tree", "rev-parse"})
_MAX_COMMAND_OUTPUT = 1_048_576
_MAX_RECORD_BYTES = 1_048_576
_MAX_LIMIT = 1_000
_MAX_DEPTH = 32
_MAX_BYTES = 1_048_576
_MAX_GLOBS = 16
_MAX_QUERY_LENGTH = 512
_MAX_CURSOR_LENGTH = 4_096
_GIT_TIMEOUT_SECONDS = 8.0
class RepositoryDiscoveryError(RuntimeError):
"""A fail-closed repository discovery request error."""
def __init__(self, code: str, *, status_code: int | None = None) -> None:
self.code = str(code)
self.status_code = (
status_code if status_code is not None else self._status_for(self.code)
)
super().__init__(self.code)
@staticmethod
def _status_for(code: str) -> int:
if "OUTPUT_LIMIT" in code:
return 413
if code.endswith("_NOT_FOUND"):
return 404
if code in {"REPOSITORY_NOT_ALLOWED", "REPOSITORY_IDENTITY_MISMATCH"}:
return 403
if code in {
"REPOSITORY_DISCOVERY_UNAVAILABLE",
"REPOSITORY_DISCOVERY_GIT_FAILED",
"REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID",
"REPOSITORY_DISCOVERY_CURSOR_SECRET_REQUIRED",
}:
return 503
return 400
class RepositoryDiscoveryNotFound(RepositoryDiscoveryError):
"""A validated immutable object or path was not found."""
class RepositoryDiscoveryOutputTooLarge(RepositoryDiscoveryError):
"""A fixed read-only Git query exceeded its collection boundary."""
def _error(code: str) -> RepositoryDiscoveryError:
return RepositoryDiscoveryError(code)
def _request_mapping(request: Mapping[str, object]) -> Mapping[str, object]:
if not isinstance(request, Mapping):
raise _error("REPOSITORY_DISCOVERY_REQUEST_INVALID")
return request
def _required_request_value(request: Mapping[str, object], field: str) -> object:
value = request.get(field)
if value is None:
raise _error("REPOSITORY_DISCOVERY_REQUEST_INVALID")
return value
def _canonical(value: Mapping[str, object]) -> bytes:
return json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=True
).encode("ascii")
def _positive_int(value: object, field: str, maximum: int) -> int:
if isinstance(value, bool) or not isinstance(value, int) or not 1 <= value <= maximum:
raise _error(f"REPOSITORY_DISCOVERY_INVALID_{field.upper()}")
return value
def _safe_relative_path(
value: object,
field: str,
*,
allow_empty: bool,
allow_glob: bool = False,
) -> str:
if not isinstance(value, str) or value != value.strip() or "\x00" in value:
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
if "\\" in value:
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
if value in ("", "."):
if allow_empty:
return ""
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
if value.startswith(("/", "-")) or "//" in value or ":" in value:
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
if not allow_glob and any(character in value for character in "*?["):
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
parts = value.split("/")
if any(part in ("", ".", "..") for part in parts):
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
if len(value) > 512:
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
return value
def _safe_glob(value: object) -> str:
return _safe_relative_path(value, "path_glob", allow_empty=False, allow_glob=True)
def _safe_ref(value: str) -> bool:
return bool(
_REF_RE.fullmatch(value)
and ".." not in value
and "//" not in value
and "@{" not in value
and not value.endswith(("/", ".", ".lock"))
and not any(part in ("", ".", "..") for part in value.split("/"))
)
def _parse_tree_record(record: bytes) -> dict[str, str] | None:
try:
header, raw_path = record.split(b"\t", 1)
mode, kind, oid = header.decode("ascii").split(" ", 2)
path = raw_path.decode("utf-8")
except (UnicodeDecodeError, ValueError) as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID") from exc
if kind not in {"blob", "tree", "commit"} or not _OID_RE.fullmatch(oid.lower()):
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
try:
_safe_relative_path(path, "tree_path", allow_empty=False)
except RepositoryDiscoveryError:
# Git permits filenames outside the public POSIX-relative contract.
# They are not exposed as usable discovery paths.
return None
return {"mode": mode, "type": kind, "object_oid": oid.lower(), "path": path}
def _decode_output_path(raw: bytes) -> str:
try:
value = raw.decode("utf-8")
except UnicodeDecodeError as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID") from exc
return _safe_relative_path(value, "git_output_path", allow_empty=False)
class RepositoryDiscovery:
"""Repository-scoped immutable Git discovery with signed cursors."""
def __init__(
self,
*,
allowed_repositories: Iterable[str] | None = None,
git_dir: str | Path | None = None,
cursor_secret: str | bytes | None = None,
) -> None:
configured = allowed_repositories
if configured is None:
configured = (os.getenv("AIOS_GITOPS_REPOSITORY_IDENTITY", ""),)
identities = tuple(configured)
if not identities or any(
not isinstance(item, str) or not _IDENTITY_RE.fullmatch(item)
for item in identities
):
raise _error("REPOSITORY_DISCOVERY_ALLOWLIST_INVALID")
self._allowed_repositories = frozenset(identities)
self._git_dir = git_dir
secret = (
cursor_secret
if cursor_secret is not None
else os.getenv("AIOS_REPOSITORY_DISCOVERY_CURSOR_SECRET", "")
)
if isinstance(secret, str):
secret = secret.encode("utf-8")
if not isinstance(secret, bytes) or len(secret) < 16:
raise _error("REPOSITORY_DISCOVERY_CURSOR_SECRET_REQUIRED")
self._cursor_secret = secret
def _repository_path(self, repository: object) -> Path:
if not isinstance(repository, str) or repository not in self._allowed_repositories:
raise _error("REPOSITORY_NOT_ALLOWED")
try:
path, actual_identity = verify_repository(
git_dir=self._git_dir,
repository_identity=repository,
)
except AuthoritativeGitError as exc:
code = (
"REPOSITORY_IDENTITY_MISMATCH"
if "REPOSITORY_IDENTITY_MISMATCH" in str(exc)
else "REPOSITORY_DISCOVERY_UNAVAILABLE"
)
raise _error(code) from exc
if actual_identity != repository:
raise _error("REPOSITORY_IDENTITY_MISMATCH")
return path
@staticmethod
def _git_env() -> dict[str, str]:
return {
**os.environ,
"GIT_OPTIONAL_LOCKS": "0",
"GIT_CONFIG_NOSYSTEM": "1",
"GIT_TERMINAL_PROMPT": "0",
}
@staticmethod
def _run(
path: Path,
args: list[str],
*,
max_output: int = _MAX_COMMAND_OUTPUT,
) -> bytes:
if not args or args[0] not in _READ_ONLY_COMMANDS:
raise _error("REPOSITORY_DISCOVERY_GIT_COMMAND_REJECTED")
output_limit = min(max_output, _MAX_COMMAND_OUTPUT)
process: subprocess.Popen[bytes] | None = None
try:
process = subprocess.Popen(
["git", "--git-dir", str(path), *args],
shell=False,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
env=RepositoryDiscovery._git_env(),
)
assert process.stdout is not None
output = process.stdout.read(output_limit + 1)
if len(output) > output_limit:
process.kill()
process.wait(timeout=1)
raise RepositoryDiscoveryOutputTooLarge(
"REPOSITORY_DISCOVERY_OUTPUT_LIMIT"
)
returncode = process.wait(timeout=_GIT_TIMEOUT_SECONDS)
except RepositoryDiscoveryError:
raise
except (OSError, subprocess.TimeoutExpired) as exc:
if process is not None:
try:
process.kill()
process.wait(timeout=1)
except (OSError, subprocess.TimeoutExpired):
pass
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED") from exc
if returncode != 0:
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED")
return output
@staticmethod
def _read_blob_prefix(path: Path, oid: str, maximum: int) -> tuple[bytes, bool]:
"""Read at most maximum bytes plus one sentinel byte."""
process: subprocess.Popen[bytes] | None = None
try:
process = subprocess.Popen(
["git", "--git-dir", str(path), "cat-file", "blob", oid],
shell=False,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
env=RepositoryDiscovery._git_env(),
)
assert process.stdout is not None
content = process.stdout.read(maximum + 1)
truncated = len(content) > maximum
if truncated:
process.kill()
process.wait(timeout=1)
return content[:maximum], True
if process.wait(timeout=_GIT_TIMEOUT_SECONDS) != 0:
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED")
return content, False
except RepositoryDiscoveryError:
raise
except (OSError, subprocess.TimeoutExpired) as exc:
if process is not None:
try:
process.kill()
process.wait(timeout=1)
except (OSError, subprocess.TimeoutExpired):
pass
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED") from exc
def _iter_nul_records(self, path: Path, args: list[str]) -> Iterator[bytes]:
if not args or args[0] not in _READ_ONLY_COMMANDS:
raise _error("REPOSITORY_DISCOVERY_GIT_COMMAND_REJECTED")
process: subprocess.Popen[bytes] | None = None
buffer = b""
started = time.monotonic()
try:
process = subprocess.Popen(
["git", "--git-dir", str(path), *args],
shell=False,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
env=self._git_env(),
)
assert process.stdout is not None
while True:
if time.monotonic() - started > _GIT_TIMEOUT_SECONDS:
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED")
chunk = process.stdout.read(64 * 1024)
if not chunk:
break
buffer += chunk
if len(buffer) > _MAX_RECORD_BYTES and b"\0" not in buffer:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
while b"\0" in buffer:
record, buffer = buffer.split(b"\0", 1)
if len(record) > _MAX_RECORD_BYTES:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
yield record
if buffer:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
if process.wait(
timeout=max(0.1, _GIT_TIMEOUT_SECONDS - (time.monotonic() - started))
) != 0:
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED")
except RepositoryDiscoveryError:
raise
except (OSError, subprocess.TimeoutExpired) as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED") from exc
finally:
if process is not None and process.poll() is None:
try:
process.kill()
process.wait(timeout=1)
except (OSError, subprocess.TimeoutExpired):
pass
def _resolve_at(self, path: Path, revision: object, field: str) -> str:
if (
not isinstance(revision, str)
or revision != revision.strip()
or not revision
or revision.startswith("-")
):
raise _error("REPOSITORY_DISCOVERY_INVALID_REVISION")
if _OID_RE.fullmatch(revision.lower()):
candidates = [revision.lower()]
elif revision == "HEAD":
candidates = ["HEAD"]
elif _safe_ref(revision) and not any(
token in revision for token in ("^", "~", ":", "\\")
):
if revision.startswith("refs/"):
if not revision.startswith(("refs/heads/", "refs/tags/")):
raise _error("REPOSITORY_DISCOVERY_INVALID_REVISION")
candidates = [revision]
else:
candidates = [f"refs/heads/{revision}", f"refs/tags/{revision}"]
else:
raise _error("REPOSITORY_DISCOVERY_INVALID_REVISION")
resolved: list[str] = []
for candidate in candidates:
try:
output = self._run(
path,
["rev-parse", "--verify", f"{candidate}^{{commit}}"],
max_output=256,
)
except RepositoryDiscoveryError:
continue
oid = output.decode("ascii", errors="ignore").strip().lower()
if _OID_RE.fullmatch(oid):
resolved.append(oid)
if not resolved:
raise RepositoryDiscoveryNotFound(
f"REPOSITORY_DISCOVERY_{field.upper()}_NOT_FOUND"
)
if len(set(resolved)) != 1:
raise _error("REPOSITORY_DISCOVERY_AMBIGUOUS_REVISION")
return resolved[0]
def resolve_revision(self, repository: str, revision: str) -> str:
return self._resolve_at(self._repository_path(repository), revision, "revision")
def _cursor_offset(
self,
cursor: str | None,
*,
operation: str,
repository: str,
resolved_oid: str,
query: Mapping[str, object],
) -> int:
if cursor is None:
return 0
if (
not isinstance(cursor, str)
or not cursor
or len(cursor) > _MAX_CURSOR_LENGTH
or "." not in cursor
):
raise _error("REPOSITORY_DISCOVERY_CURSOR_INVALID")
encoded, supplied_signature = cursor.split(".", 1)
try:
padded = encoded + "=" * (-len(encoded) % 4)
payload = json.loads(
base64.urlsafe_b64decode(padded.encode("ascii")).decode("ascii")
)
except (UnicodeError, ValueError, json.JSONDecodeError) as exc:
raise _error("REPOSITORY_DISCOVERY_CURSOR_INVALID") from exc
expected_signature = hmac.new(
self._cursor_secret, encoded.encode("ascii"), hashlib.sha256
).hexdigest()
if not hmac.compare_digest(supplied_signature, expected_signature):
raise _error("REPOSITORY_DISCOVERY_CURSOR_INVALID")
expected = {
"operation": operation,
"repository": repository,
"resolved_oid": resolved_oid,
"query": dict(query),
}
if (
not isinstance(payload, dict)
or payload.get("v") != 1
or any(payload.get(key) != value for key, value in expected.items())
):
raise _error("REPOSITORY_DISCOVERY_CURSOR_INVALID")
offset = payload.get("offset")
if isinstance(offset, bool) or not isinstance(offset, int) or offset < 0:
raise _error("REPOSITORY_DISCOVERY_CURSOR_INVALID")
return offset
def _next_cursor(
self,
*,
operation: str,
repository: str,
resolved_oid: str,
query: Mapping[str, object],
offset: int,
) -> str:
payload = {
"v": 1,
"operation": operation,
"repository": repository,
"resolved_oid": resolved_oid,
"query": dict(query),
"offset": offset,
}
encoded = base64.urlsafe_b64encode(_canonical(payload)).decode("ascii").rstrip("=")
signature = hmac.new(
self._cursor_secret, encoded.encode("ascii"), hashlib.sha256
).hexdigest()
return f"{encoded}.{signature}"
def _iter_tree_entries(
self,
path: Path,
oid: str,
prefix: str,
*,
include_trees: bool,
) -> Iterator[dict[str, str]]:
args = ["ls-tree", "-z", "-r", "--full-tree"]
if include_trees:
args.append("-t")
args.append(oid)
if prefix:
args.extend(["--", prefix])
for record in self._iter_nul_records(path, args):
entry = _parse_tree_record(record)
if entry is None:
continue
if not prefix or entry["path"] == prefix or entry["path"].startswith(prefix + "/"):
yield entry
def list_tree(
self,
*,
repository: str,
revision: str,
prefix: str = "",
depth: int = 8,
limit: int = 100,
cursor: str | None = None,
) -> dict[str, object]:
prefix = _safe_relative_path(prefix, "prefix", allow_empty=True)
depth = _positive_int(depth, "depth", _MAX_DEPTH)
limit = _positive_int(limit, "limit", _MAX_LIMIT)
path = self._repository_path(repository)
resolved_oid = self._resolve_at(path, revision, "revision")
query = {"prefix": prefix, "depth": depth, "limit": limit}
offset = self._cursor_offset(
cursor,
operation="list_tree",
repository=repository,
resolved_oid=resolved_oid,
query=query,
)
page: list[dict[str, str]] = []
matched = 0
for entry in self._iter_tree_entries(path, resolved_oid, prefix, include_trees=True):
relative = entry["path"][len(prefix):].lstrip("/") if prefix else entry["path"]
if relative.count("/") + 1 > depth:
continue
if matched >= offset and len(page) < limit:
page.append(entry)
matched += 1
has_more = offset + len(page) < matched
return {
"repository": repository,
"resolved_oid": resolved_oid,
"entries": page,
"truncated": has_more,
"next_cursor": (
self._next_cursor(
operation="list_tree",
repository=repository,
resolved_oid=resolved_oid,
query=query,
offset=offset + len(page),
)
if has_more
else None
),
}
def list_repository_tree(self, request: Mapping[str, object]) -> dict[str, object]:
request = _request_mapping(request)
return self.list_tree(
repository=_required_request_value(request, "repository"), # type: ignore[arg-type]
revision=request.get("revision", request.get("ref", "HEAD")), # type: ignore[arg-type]
prefix=request.get("prefix", ""), # type: ignore[arg-type]
depth=request.get("depth", 8), # type: ignore[arg-type]
limit=request.get("limit", 100), # type: ignore[arg-type]
cursor=request.get("cursor"), # type: ignore[arg-type]
)
def _blob_size(self, path: Path, oid: str) -> int:
output = self._run(path, ["cat-file", "-s", oid], max_output=128)
try:
size = int(output.decode("ascii").strip())
except (UnicodeDecodeError, ValueError) as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID") from exc
if size < 0:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
return size
def search_source(
self,
*,
repository: str,
revision: str,
query: str,
path_globs: Iterable[str] | None = None,
max_matches: int = 100,
max_bytes: int = 262_144,
cursor: str | None = None,
) -> dict[str, object]:
if (
not isinstance(query, str)
or not query
or "\x00" in query
or len(query) > _MAX_QUERY_LENGTH
):
raise _error("REPOSITORY_DISCOVERY_QUERY_INVALID")
max_matches = _positive_int(max_matches, "max_matches", _MAX_LIMIT)
max_bytes = _positive_int(max_bytes, "max_bytes", _MAX_BYTES)
if path_globs is None:
globs = ("*",)
else:
if isinstance(path_globs, (str, bytes)) or not isinstance(path_globs, Iterable):
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
globs = tuple(_safe_glob(item) for item in path_globs)
if not globs or len(globs) > _MAX_GLOBS:
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
path = self._repository_path(repository)
resolved_oid = self._resolve_at(path, revision, "revision")
query_binding = {
"query": query,
"path_globs": list(globs),
"max_matches": max_matches,
"max_bytes": max_bytes,
}
offset = self._cursor_offset(
cursor,
operation="search_source",
repository=repository,
resolved_oid=resolved_oid,
query=query_binding,
)
scanned_bytes = 0
skipped_for_bound = False
binary_blobs = 0
matched = 0
matches: list[dict[str, object]] = []
has_more = False
for entry in self._iter_tree_entries(path, resolved_oid, "", include_trees=False):
if entry["type"] != "blob" or not any(
fnmatch.fnmatchcase(entry["path"], pattern) for pattern in globs
):
continue
blob_size = self._blob_size(path, entry["object_oid"])
remaining = max_bytes - scanned_bytes
if blob_size > remaining:
skipped_for_bound = True
continue
raw, clipped = self._read_blob_prefix(path, entry["object_oid"], blob_size)
if clipped:
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED")
scanned_bytes += len(raw)
if b"\0" in raw:
binary_blobs += 1
continue
text = raw.decode("utf-8", errors="replace")
for number, line in enumerate(text.splitlines(), start=1):
if query not in line:
continue
if matched < offset:
matched += 1
continue
if len(matches) >= max_matches:
has_more = True
break
matches.append(
{
"path": entry["path"],
"line_start": number,
"line_end": number,
"blob_oid": entry["object_oid"],
"snippet": line[:512],
}
)
matched += 1
if has_more:
break
return {
"repository": repository,
"resolved_oid": resolved_oid,
"matches": matches,
"scanned_bytes": scanned_bytes,
"max_bytes": max_bytes,
"binary_blobs_skipped": binary_blobs,
"truncated": bool(has_more or skipped_for_bound),
"next_cursor": (
self._next_cursor(
operation="search_source",
repository=repository,
resolved_oid=resolved_oid,
query=query_binding,
offset=offset + len(matches),
)
if has_more
else None
),
}
def search_repository_source(self, request: Mapping[str, object]) -> dict[str, object]:
request = _request_mapping(request)
path_globs = request.get("path_globs")
if path_globs is not None and (
isinstance(path_globs, (str, bytes)) or not isinstance(path_globs, Iterable)
):
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
return self.search_source(
repository=_required_request_value(request, "repository"), # type: ignore[arg-type]
revision=request.get("revision", request.get("ref", "HEAD")), # type: ignore[arg-type]
query=_required_request_value(request, "query"), # type: ignore[arg-type]
path_globs=path_globs, # type: ignore[arg-type]
max_matches=request.get("max_matches", 100), # type: ignore[arg-type]
max_bytes=request.get("max_bytes", 262_144), # type: ignore[arg-type]
cursor=request.get("cursor"), # type: ignore[arg-type]
)
def _iter_diff_name_entries(
self,
path: Path,
from_oid: str,
to_oid: str,
prefix: str,
) -> Iterator[dict[str, object]]:
suffix = ["--", prefix] if prefix else []
records = self._iter_nul_records(
path,
[
"diff",
"--no-ext-diff",
"--no-textconv",
"--find-renames=50%",
"--name-status",
"-z",
from_oid,
to_oid,
*suffix,
],
)
iterator = iter(records)
while True:
try:
status_raw = next(iterator)
except StopIteration:
return
try:
status = status_raw.decode("ascii")
except UnicodeDecodeError as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID") from exc
if not status or status[0] not in {"A", "C", "D", "M", "R", "T", "U", "X"}:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
try:
if status[0] in {"R", "C"}:
old_path = _decode_output_path(next(iterator))
new_path = _decode_output_path(next(iterator))
yield {
"status": status,
"path": new_path,
"old_path": old_path,
}
else:
yield {
"status": status,
"path": _decode_output_path(next(iterator)),
"old_path": None,
}
except StopIteration as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID") from exc
@staticmethod
def _parse_numstat_count(value: bytes) -> int | None:
if value == b"-":
return None
try:
parsed = int(value.decode("ascii"), 10)
except (UnicodeDecodeError, ValueError) as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID") from exc
if parsed < 0:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
return parsed
def _diff_stats(
self,
path: Path,
from_oid: str,
to_oid: str,
prefix: str,
wanted_paths: set[str],
) -> dict[str, tuple[int | None, int | None]]:
suffix = ["--", prefix] if prefix else []
records = self._iter_nul_records(
path,
[
"diff",
"--no-ext-diff",
"--no-textconv",
"--find-renames=50%",
"--numstat",
"-z",
from_oid,
to_oid,
*suffix,
],
)
iterator = iter(records)
stats: dict[str, tuple[int | None, int | None]] = {}
for row in iterator:
if not row:
continue
fields = row.split(b"\t", 2)
if len(fields) != 3:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
additions = self._parse_numstat_count(fields[0])
deletions = self._parse_numstat_count(fields[1])
if fields[2]:
stat_path = _decode_output_path(fields[2])
else:
try:
next(iterator) # old path
stat_path = _decode_output_path(next(iterator))
except StopIteration as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID") from exc
if stat_path in wanted_paths:
stats[stat_path] = (additions, deletions)
if wanted_paths <= set(stats):
break
return stats
def diff_index(
self,
*,
repository: str,
from_revision: str,
to_revision: str,
prefix: str = "",
limit: int = 100,
cursor: str | None = None,
) -> dict[str, object]:
prefix = _safe_relative_path(prefix, "prefix", allow_empty=True)
limit = _positive_int(limit, "limit", _MAX_LIMIT)
path = self._repository_path(repository)
from_oid = self._resolve_at(path, from_revision, "from")
to_oid = self._resolve_at(path, to_revision, "to")
query = {
"from_oid": from_oid,
"to_oid": to_oid,
"prefix": prefix,
"limit": limit,
}
offset = self._cursor_offset(
cursor,
operation="diff_index",
repository=repository,
resolved_oid=to_oid,
query=query,
)
page: list[dict[str, object]] = []
total = 0
for entry in self._iter_diff_name_entries(path, from_oid, to_oid, prefix):
if total >= offset and len(page) < limit:
page.append(entry)
total += 1
wanted_paths = {str(entry["path"]) for entry in page}
stat_by_path = (
self._diff_stats(path, from_oid, to_oid, prefix, wanted_paths)
if page
else {}
)
for entry in page:
additions, deletions = stat_by_path.get(
str(entry["path"]), (None, None)
)
entry["additions"] = additions
entry["deletions"] = deletions
has_more = offset + len(page) < total
return {
"repository": repository,
"resolved_from_oid": from_oid,
"resolved_to_oid": to_oid,
"entries": page,
"truncated": has_more,
"next_cursor": (
self._next_cursor(
operation="diff_index",
repository=repository,
resolved_oid=to_oid,
query=query,
offset=offset + len(page),
)
if has_more
else None
),
}
def get_diff_index(self, request: Mapping[str, object]) -> dict[str, object]:
request = _request_mapping(request)
from_revision = request.get("from_revision", request.get("from"))
to_revision = request.get("to_revision", request.get("to"))
prefix = request.get("prefix", request.get("path_prefix", ""))
if from_revision is None or to_revision is None:
raise _error("REPOSITORY_DISCOVERY_REQUEST_INVALID")
return self.diff_index(
repository=_required_request_value(request, "repository"), # type: ignore[arg-type]
from_revision=from_revision, # type: ignore[arg-type]
to_revision=to_revision, # type: ignore[arg-type]
prefix=prefix, # type: ignore[arg-type]
limit=request.get("limit", 100), # type: ignore[arg-type]
cursor=request.get("cursor"), # type: ignore[arg-type]
)
def read_file(
self,
*,
repository: str,
revision: str,
file_path: str,
max_bytes: int = 262_144,
) -> dict[str, object]:
file_path = _safe_relative_path(file_path, "file_path", allow_empty=False)
max_bytes = _positive_int(max_bytes, "max_bytes", _MAX_BYTES)
path = self._repository_path(repository)
resolved_oid = self._resolve_at(path, revision, "revision")
exact: list[dict[str, str]] = []
for entry in self._iter_tree_entries(
path, resolved_oid, file_path, include_trees=False
):
if entry["path"] == file_path and entry["type"] == "blob":
exact.append(entry)
if len(exact) != 1:
raise RepositoryDiscoveryNotFound("REPOSITORY_DISCOVERY_FILE_NOT_FOUND")
blob_oid = exact[0]["object_oid"]
content, truncated = self._read_blob_prefix(path, blob_oid, max_bytes)
return {
"repository": repository,
"resolved_oid": resolved_oid,
"blob_oid": blob_oid,
"path": file_path,
"content": content.decode("utf-8", errors="replace"),
"content_base64": base64.b64encode(content).decode("ascii"),
"content_bytes": len(content),
"truncated": truncated,
}
def read_repository_file(self, request: Mapping[str, object]) -> dict[str, object]:
request = _request_mapping(request)
file_path = request.get("file_path", request.get("path"))
if file_path is None:
raise _error("REPOSITORY_DISCOVERY_REQUEST_INVALID")
return self.read_file(
repository=_required_request_value(request, "repository"), # type: ignore[arg-type]
revision=request.get("revision", request.get("ref", "HEAD")), # type: ignore[arg-type]
file_path=file_path, # type: ignore[arg-type]
max_bytes=request.get("max_bytes", 262_144), # type: ignore[arg-type]
)
def create_repository_discovery(
*,
allowed_repositories: Iterable[str] | None = None,
git_dir: str | Path | None = None,
cursor_secret: str | bytes | None = None,
) -> RepositoryDiscovery:
return RepositoryDiscovery(
allowed_repositories=allowed_repositories,
git_dir=git_dir,
cursor_secret=cursor_secret,
)
def list_repository_tree(
request: Mapping[str, object], *, discovery: RepositoryDiscovery | None = None
) -> dict[str, object]:
return (discovery or create_repository_discovery()).list_repository_tree(request)
def search_repository_source(
request: Mapping[str, object], *, discovery: RepositoryDiscovery | None = None
) -> dict[str, object]:
return (discovery or create_repository_discovery()).search_repository_source(request)
def get_diff_index(
request: Mapping[str, object], *, discovery: RepositoryDiscovery | None = None
) -> dict[str, object]:
return (discovery or create_repository_discovery()).get_diff_index(request)
def read_repository_file(
request: Mapping[str, object], *, discovery: RepositoryDiscovery | None = None
) -> dict[str, object]:
return (discovery or create_repository_discovery()).read_repository_file(request)
@@ -0,0 +1,945 @@
"""Fail-closed bootstrap for narrow Owner-delegated Agent Task admission.
This module solves the authorization bootstrap problem without turning the
runtime anchor into a writer and without weakening TaskPrincipal semantics.
Security model:
* A task-admission envelope is created from a server-normalized request.
* The envelope is authorized either by an already-valid OwnerDelegationEpoch
or by an Owner signature verified by an injected trust backend.
* The derived TaskAdmissionGrant can only narrow the parent authorization.
* The grant is handed to the existing Agent Task control plane through an
injected TaskAdmissionPort. This module never writes the Task database.
* TaskPrincipal issuance remains downstream of a real Task claim/TaskLease.
* Exact idempotent replay returns the same admitted Task; conflicting reuse is
rejected.
"""
from __future__ import annotations
import base64
from contextlib import closing, contextmanager
from dataclasses import asdict, dataclass
from datetime import datetime, timezone
import hashlib
import json
from pathlib import Path, PurePosixPath
import re
import secrets
import sqlite3
import threading
from typing import Any, Callable, Mapping, Protocol, Sequence
SCHEMA_ENVELOPE = "aios.task-admission-envelope.v1"
SCHEMA_GRANT = "aios.task-admission-grant.v1"
SCHEMA_EPOCH = "aios.owner-delegation-epoch.v1"
SCHEMA_RECEIPT = "aios.task-admission-receipt.v1"
OID_RE = re.compile(r"^[0-9a-f]{40}$")
TASK_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{2,179}$")
BRANCH_RE = re.compile(r"^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$")
IDEMPOTENCY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:@/-]{7,255}$")
CAPABILITY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{1,127}$")
KEY_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:@+-]{2,255}$")
PROHIBITED_CAPABILITIES = frozenset(
{
"git.global_write",
"git.main_write",
"git.force_push",
"tag.publish",
"baseline.finalize",
"production.write",
"production.switch",
"runtime_anchor.write",
"shell.arbitrary",
"credential.export",
}
)
class TaskAdmissionError(RuntimeError):
"""Stable fail-closed task-admission error."""
def __init__(
self,
code: str,
status: int = 400,
*,
details: Mapping[str, Any] | None = None,
) -> None:
super().__init__(code)
self.code = code
self.status = status
self.details = dict(details or {})
@dataclass(frozen=True, slots=True)
class TaskAdmissionPolicy:
repository: str = "aios/aios"
trust_domain: str = "aios.owner"
branch_prefix: str = "agent/"
allowed_capabilities: frozenset[str] = frozenset(
{
"governed.task.create",
"agent-task:control",
"task.principal.issue",
"writer.lease",
"governed.write.files",
"git.read",
"git.test",
"governed.write.branch",
"governed.write.file",
"testing",
"verification",
}
)
max_file_count: int = 64
max_task_ttl_seconds: int = 3600
max_envelope_ttl_seconds: int = 900
max_risk: str = "medium"
policy_digest: str = ""
def normalized_policy_digest(self) -> str:
if self.policy_digest:
return self.policy_digest
return sha256_json(
{
"repository": self.repository,
"trust_domain": self.trust_domain,
"branch_prefix": self.branch_prefix,
"allowed_capabilities": sorted(self.allowed_capabilities),
"max_file_count": self.max_file_count,
"max_task_ttl_seconds": self.max_task_ttl_seconds,
"max_envelope_ttl_seconds": self.max_envelope_ttl_seconds,
"max_risk": self.max_risk,
"prohibited_capabilities": sorted(PROHIBITED_CAPABILITIES),
}
)
@dataclass(frozen=True, slots=True)
class OwnerDelegationEpoch:
"""Already-verified durable Owner authorization.
The verifier/backend is responsible for authenticating the epoch before
returning it to this service.
"""
epoch_id: str
owner_identity: str
trust_domain: str
repository: str
allowed_branch_prefixes: tuple[str, ...]
allowed_capabilities: frozenset[str]
allowed_file_prefixes: tuple[str, ...]
risk_ceiling: str
issued_at: int
not_before: int
expires_at: int
revocation_epoch: int
policy_digest: str
schema: str = SCHEMA_EPOCH
def as_public_dict(self) -> dict[str, Any]:
value = asdict(self)
value["allowed_capabilities"] = sorted(self.allowed_capabilities)
value["allowed_branch_prefixes"] = list(self.allowed_branch_prefixes)
value["allowed_file_prefixes"] = list(self.allowed_file_prefixes)
return value
@dataclass(frozen=True, slots=True)
class TaskAdmissionEnvelope:
envelope_id: str
idempotency_key: str
task_id: str
repository: str
authority_base_oid: str
target_branch: str
generation: int
execution_digest: str
requested_capabilities: tuple[str, ...]
file_allowlist: tuple[str, ...]
risk: str
trust_domain: str
policy_digest: str
issued_at: int
not_before: int
expires_at: int
nonce: str
request_digest: str
schema: str = SCHEMA_ENVELOPE
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
value["requested_capabilities"] = list(self.requested_capabilities)
value["file_allowlist"] = list(self.file_allowlist)
return value
@dataclass(frozen=True, slots=True)
class TaskAdmissionGrant:
grant_id: str
envelope_id: str
task_id: str
repository: str
authority_base_oid: str
target_branch: str
generation: int
execution_digest: str
capabilities: tuple[str, ...]
file_allowlist: tuple[str, ...]
risk: str
trust_domain: str
policy_digest: str
authorization_kind: str
owner_identity: str
parent_epoch_id: str | None
issued_at: int
expires_at: int
one_time_use: bool = True
schema: str = SCHEMA_GRANT
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
value["capabilities"] = list(self.capabilities)
value["file_allowlist"] = list(self.file_allowlist)
return value
class OwnerAuthorizationBackend(Protocol):
"""Bridge to existing trusted Owner authorization.
No permissive default implementation is provided.
"""
def verify_delegation_epoch(self, epoch_id: str) -> OwnerDelegationEpoch:
...
def verify_owner_signature(
self,
*,
key_id: str,
canonical_payload: bytes,
signature_b64: str,
) -> str:
"""Return the verified Owner identity or raise."""
...
class TaskAdmissionPort(Protocol):
"""Existing Agent Task control-plane port."""
def admit(self, grant: TaskAdmissionGrant) -> Mapping[str, Any]:
...
def utc_seconds() -> int:
return int(datetime.now(timezone.utc).timestamp())
def canonical_json_bytes(value: Any) -> bytes:
return json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
def sha256_json(value: Any) -> str:
return hashlib.sha256(canonical_json_bytes(value)).hexdigest()
def _normalize_files(raw: Sequence[object], *, maximum: int) -> tuple[str, ...]:
if not isinstance(raw, (list, tuple)):
raise TaskAdmissionError("TASK_ADMISSION_FILE_ALLOWLIST_REJECTED", 403)
if not raw or len(raw) > maximum:
raise TaskAdmissionError("TASK_ADMISSION_FILE_ALLOWLIST_REJECTED", 403)
normalized: list[str] = []
seen: set[str] = set()
for item in raw:
if (
not isinstance(item, str)
or not item
or "\\" in item
or any(ord(char) < 32 or ord(char) == 127 for char in item)
):
raise TaskAdmissionError("TASK_ADMISSION_FILE_PATH_INVALID", 403)
path = PurePosixPath(item)
if path.is_absolute() or ".." in path.parts or "." in path.parts:
raise TaskAdmissionError("TASK_ADMISSION_FILE_PATH_INVALID", 403)
text = path.as_posix()
# Do not normalize a caller supplied path. A signature must bind the
# exact spelling, not a convenient equivalent such as ``a//b``.
if text != item:
raise TaskAdmissionError("TASK_ADMISSION_FILE_PATH_INVALID", 403)
if text.startswith(".git/") or text == ".git":
raise TaskAdmissionError("TASK_ADMISSION_FILE_PATH_INVALID", 403)
if text not in seen:
normalized.append(text)
seen.add(text)
if not normalized:
raise TaskAdmissionError("TASK_ADMISSION_FILE_ALLOWLIST_REJECTED", 403)
return tuple(sorted(normalized))
def _normalize_capabilities(
raw: Sequence[object],
*,
policy: TaskAdmissionPolicy,
) -> tuple[str, ...]:
if not raw:
raise TaskAdmissionError("TASK_ADMISSION_CAPABILITIES_REQUIRED", 403)
values: set[str] = set()
for item in raw:
if not isinstance(item, str) or not CAPABILITY_RE.fullmatch(item):
raise TaskAdmissionError("TASK_ADMISSION_CAPABILITY_INVALID", 403)
if "*" in item:
raise TaskAdmissionError("TASK_ADMISSION_WILDCARD_CAPABILITY_REJECTED", 403)
values.add(item)
if values & PROHIBITED_CAPABILITIES:
raise TaskAdmissionError("TASK_ADMISSION_PROHIBITED_CAPABILITY", 403)
if not values.issubset(policy.allowed_capabilities):
raise TaskAdmissionError("TASK_ADMISSION_CAPABILITY_OUT_OF_POLICY", 403)
return tuple(sorted(values))
def _risk_rank(value: str) -> int:
ranks = {"low": 0, "medium": 1, "high": 2}
try:
return ranks[value]
except KeyError:
raise TaskAdmissionError("TASK_ADMISSION_RISK_INVALID", 403) from None
def _request_for_digest(
*,
task_id: str,
repository: str,
authority_base_oid: str,
target_branch: str,
generation: int,
execution_digest: str,
requested_capabilities: tuple[str, ...],
file_allowlist: tuple[str, ...],
risk: str,
trust_domain: str,
policy_digest: str,
) -> dict[str, Any]:
return {
"task_id": task_id,
"repository": repository,
"authority_base_oid": authority_base_oid,
"target_branch": target_branch,
"generation": generation,
"execution_digest": execution_digest,
"requested_capabilities": list(requested_capabilities),
"file_allowlist": list(file_allowlist),
"risk": risk,
"trust_domain": trust_domain,
"policy_digest": policy_digest,
}
class TaskAdmissionStore:
"""Portable durable state; all replay decisions are transactionally stored."""
def __init__(self, path: Path) -> None:
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
self.lock_root = Path(str(self.path) + ".locks")
self.lock_root.mkdir(parents=True, exist_ok=True)
self._lock = threading.RLock()
with self._connection() as con:
con.executescript(
"""
PRAGMA journal_mode=WAL;
CREATE TABLE IF NOT EXISTS task_admission_envelopes(
idempotency_key TEXT PRIMARY KEY,
envelope_id TEXT NOT NULL UNIQUE,
request_digest TEXT NOT NULL,
envelope_json TEXT NOT NULL,
status TEXT NOT NULL,
created_at INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS task_admission_grants(
envelope_id TEXT PRIMARY KEY,
grant_id TEXT NOT NULL UNIQUE,
grant_digest TEXT NOT NULL,
grant_json TEXT NOT NULL,
authorization_digest TEXT NOT NULL DEFAULT '',
created_at INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS task_admission_receipts(
envelope_id TEXT PRIMARY KEY,
task_id TEXT NOT NULL UNIQUE,
receipt_json TEXT NOT NULL,
created_at INTEGER NOT NULL
);
"""
)
columns = {
row["name"] for row in con.execute("PRAGMA table_info(task_admission_grants)")
}
if "authorization_digest" not in columns:
con.execute(
"ALTER TABLE task_admission_grants ADD COLUMN authorization_digest TEXT NOT NULL DEFAULT ''"
)
def _connect(self) -> sqlite3.Connection:
con = sqlite3.connect(self.path, timeout=30, isolation_level=None)
con.row_factory = sqlite3.Row
return con
@contextmanager
def _connection(self):
"""Yield an owned main-store connection and close it deterministically."""
with closing(self._connect()) as con:
try:
yield con
finally:
if con.in_transaction:
con.rollback()
@contextmanager
def admission_lock(self, envelope_id: str, *, timeout_seconds: float = 5.0):
"""Hold a crash-releasing lock for one envelope, outside the main DB."""
lock_name = hashlib.sha256(envelope_id.encode("utf-8")).hexdigest() + ".sqlite3"
lock_path = self.lock_root / lock_name
with closing(sqlite3.connect(lock_path, timeout=timeout_seconds, isolation_level=None)) as con:
try:
con.execute("BEGIN EXCLUSIVE")
except sqlite3.OperationalError as exc:
raise TaskAdmissionError("TASK_ADMISSION_IN_PROGRESS", 503) from exc
try:
yield
con.execute("COMMIT")
finally:
if con.in_transaction:
con.rollback()
def get_by_idempotency(self, key: str) -> dict[str, Any] | None:
with self._connection() as con:
row = con.execute(
"SELECT * FROM task_admission_envelopes WHERE idempotency_key=?",
(key,),
).fetchone()
return dict(row) if row else None
def get_envelope(self, envelope_id: str) -> dict[str, Any] | None:
with self._connection() as con:
row = con.execute(
"SELECT * FROM task_admission_envelopes WHERE envelope_id=?",
(envelope_id,),
).fetchone()
return dict(row) if row else None
def save_envelope(
self,
*,
idempotency_key: str,
envelope_id: str,
request_digest: str,
envelope_json: str,
created_at: int,
) -> dict[str, Any]:
with self._lock, self._connection() as con:
con.execute("BEGIN IMMEDIATE")
row = con.execute(
"SELECT request_digest,envelope_id FROM task_admission_envelopes WHERE idempotency_key=?",
(idempotency_key,),
).fetchone()
if row is not None:
con.execute("ROLLBACK")
if row["request_digest"] != request_digest:
raise TaskAdmissionError("TASK_ADMISSION_IDEMPOTENCY_CONFLICT", 409)
return json.loads(
con.execute(
"SELECT envelope_json FROM task_admission_envelopes WHERE idempotency_key=?",
(idempotency_key,),
).fetchone()["envelope_json"]
)
con.execute(
"""
INSERT INTO task_admission_envelopes(
idempotency_key,envelope_id,request_digest,envelope_json,status,created_at
) VALUES(?,?,?,?,?,?)
""",
(
idempotency_key,
envelope_id,
request_digest,
envelope_json,
"PENDING_AUTHORIZATION",
created_at,
),
)
con.execute("COMMIT")
return json.loads(envelope_json)
def claim_grant(
self,
*,
envelope_id: str,
grant: TaskAdmissionGrant,
authorization_digest: str,
created_at: int,
) -> tuple[TaskAdmissionGrant, bool]:
"""Persist the immutable grant before the external Task side effect.
``BEGIN IMMEDIATE`` serializes only this SQLite writer transaction;
it does not serialize unrelated envelopes. A second proof cannot
replace the winning proof/grant for the same envelope.
"""
grant_json = canonical_json_bytes(grant.as_dict()).decode("utf-8")
grant_digest = hashlib.sha256(grant_json.encode("utf-8")).hexdigest()
with self._lock, self._connection() as con:
con.execute("BEGIN IMMEDIATE")
row = con.execute(
"SELECT grant_json,authorization_digest FROM task_admission_grants WHERE envelope_id=?",
(envelope_id,),
).fetchone()
if row is not None:
con.execute("ROLLBACK")
if row["authorization_digest"] != authorization_digest:
raise TaskAdmissionError("TASK_ADMISSION_AUTHORIZATION_CONFLICT", 409)
raw = json.loads(row["grant_json"])
return (
TaskAdmissionGrant(
**{
**raw,
"capabilities": tuple(raw["capabilities"]),
"file_allowlist": tuple(raw["file_allowlist"]),
}
),
False,
)
con.execute(
"INSERT INTO task_admission_grants(envelope_id,grant_id,grant_digest,grant_json,authorization_digest,created_at) VALUES(?,?,?,?,?,?)",
(envelope_id, grant.grant_id, grant_digest, grant_json, authorization_digest, created_at),
)
con.execute(
"UPDATE task_admission_envelopes SET status='ADMITTING' WHERE envelope_id=?",
(envelope_id,),
)
con.execute("COMMIT")
return grant, True
def receipt(self, envelope_id: str) -> dict[str, Any] | None:
with self._connection() as con:
row = con.execute(
"SELECT receipt_json FROM task_admission_receipts WHERE envelope_id=?",
(envelope_id,),
).fetchone()
return json.loads(row["receipt_json"]) if row else None
def save_grant_and_receipt(
self,
*,
envelope_id: str,
grant: TaskAdmissionGrant,
receipt: Mapping[str, Any],
created_at: int,
) -> dict[str, Any]:
grant_json = canonical_json_bytes(grant.as_dict()).decode("utf-8")
grant_digest = hashlib.sha256(grant_json.encode("utf-8")).hexdigest()
receipt_json = canonical_json_bytes(dict(receipt)).decode("utf-8")
with self._lock, self._connection() as con:
con.execute("BEGIN IMMEDIATE")
existing = con.execute(
"SELECT receipt_json FROM task_admission_receipts WHERE envelope_id=?",
(envelope_id,),
).fetchone()
if existing is not None:
con.execute("ROLLBACK")
return json.loads(existing["receipt_json"])
stored = con.execute(
"SELECT grant_id,grant_digest FROM task_admission_grants WHERE envelope_id=?",
(envelope_id,),
).fetchone()
if stored is None:
con.execute(
"INSERT INTO task_admission_grants(envelope_id,grant_id,grant_digest,grant_json,authorization_digest,created_at) VALUES(?,?,?,?,?,?)",
(envelope_id, grant.grant_id, grant_digest, grant_json, "", created_at),
)
elif stored["grant_id"] != grant.grant_id or stored["grant_digest"] != grant_digest:
con.execute("ROLLBACK")
raise TaskAdmissionError("TASK_ADMISSION_GRANT_CONFLICT", 409)
con.execute(
"""
INSERT INTO task_admission_receipts(
envelope_id,task_id,receipt_json,created_at
) VALUES(?,?,?,?)
""",
(
envelope_id,
grant.task_id,
receipt_json,
created_at,
),
)
con.execute(
"UPDATE task_admission_envelopes SET status='ADMITTED' WHERE envelope_id=?",
(envelope_id,),
)
con.execute("COMMIT")
return dict(receipt)
def status(self, envelope_id: str) -> dict[str, Any]:
row = self.get_envelope(envelope_id)
if row is None:
raise TaskAdmissionError("TASK_ADMISSION_ENVELOPE_NOT_FOUND", 404)
receipt = self.receipt(envelope_id)
return {
"envelope_id": envelope_id,
"status": row["status"],
"task": receipt,
}
class TaskAdmissionBootstrapService:
def __init__(
self,
*,
store: TaskAdmissionStore,
authorization: OwnerAuthorizationBackend,
task_port: TaskAdmissionPort,
policy: TaskAdmissionPolicy | None = None,
clock=utc_seconds,
post_admission: Callable[[TaskAdmissionGrant, Mapping[str, Any]], Any] | None = None,
) -> None:
self.store = store
self.authorization = authorization
self.task_port = task_port
self.policy = policy or TaskAdmissionPolicy()
self.clock = clock
# This is a narrow server-side projection boundary. The official
# Agent Task port remains the admission authority; a projection may
# only establish local execution state after its exact Task readback.
self.post_admission = post_admission
self._admit_lock = threading.RLock()
def create_envelope(self, raw: Mapping[str, Any]) -> dict[str, Any]:
required = {
"idempotency_key",
"task_id",
"repository",
"authority_base_oid",
"target_branch",
"generation",
"execution_digest",
"requested_capabilities",
"file_allowlist",
"risk",
}
optional = {"expires_in_seconds"}
if set(raw) - (required | optional) or not required.issubset(raw):
raise TaskAdmissionError("TASK_ADMISSION_REQUEST_FIELDS_INVALID")
idempotency_key = str(raw["idempotency_key"])
task_id = str(raw["task_id"])
repository = str(raw["repository"])
authority_base_oid = str(raw["authority_base_oid"])
target_branch = str(raw["target_branch"])
execution_digest = str(raw["execution_digest"])
if not IDEMPOTENCY_RE.fullmatch(idempotency_key):
raise TaskAdmissionError("TASK_ADMISSION_IDEMPOTENCY_KEY_INVALID")
if not TASK_RE.fullmatch(task_id):
raise TaskAdmissionError("TASK_ADMISSION_TASK_ID_INVALID")
if repository != self.policy.repository:
raise TaskAdmissionError("TASK_ADMISSION_REPOSITORY_REJECTED", 403)
if not OID_RE.fullmatch(authority_base_oid):
raise TaskAdmissionError("TASK_ADMISSION_BASE_OID_INVALID")
if (
not BRANCH_RE.fullmatch(target_branch)
or not target_branch.startswith(self.policy.branch_prefix)
or "//" in target_branch
or ".." in target_branch
or "@{" in target_branch
or target_branch.endswith(("/", "."))
or any(
not component
or component.startswith(".")
or component.endswith(".lock")
for component in target_branch.split("/")
)
):
raise TaskAdmissionError("TASK_ADMISSION_BRANCH_REJECTED", 403)
generation = raw["generation"]
if not isinstance(generation, int) or isinstance(generation, bool):
raise TaskAdmissionError("TASK_ADMISSION_GENERATION_INVALID")
if generation < 1:
raise TaskAdmissionError("TASK_ADMISSION_GENERATION_INVALID")
if not re.fullmatch(r"^[0-9a-f]{64}$", execution_digest):
raise TaskAdmissionError("TASK_ADMISSION_EXECUTION_DIGEST_INVALID")
requested_capabilities = _normalize_capabilities(
raw["requested_capabilities"],
policy=self.policy,
)
file_allowlist = _normalize_files(
raw["file_allowlist"],
maximum=self.policy.max_file_count,
)
risk = str(raw["risk"])
if _risk_rank(risk) > _risk_rank(self.policy.max_risk):
raise TaskAdmissionError("TASK_ADMISSION_RISK_OUT_OF_POLICY", 403)
ttl = raw.get("expires_in_seconds", self.policy.max_envelope_ttl_seconds)
if (
not isinstance(ttl, int)
or isinstance(ttl, bool)
or ttl < 60
or ttl > self.policy.max_envelope_ttl_seconds
):
raise TaskAdmissionError("TASK_ADMISSION_ENVELOPE_TTL_INVALID")
now = int(self.clock())
policy_digest = self.policy.normalized_policy_digest()
request_value = _request_for_digest(
task_id=task_id,
repository=repository,
authority_base_oid=authority_base_oid,
target_branch=target_branch,
generation=generation,
execution_digest=execution_digest,
requested_capabilities=requested_capabilities,
file_allowlist=file_allowlist,
risk=risk,
trust_domain=self.policy.trust_domain,
policy_digest=policy_digest,
)
request_digest = sha256_json(request_value)
nonce = secrets.token_urlsafe(24)
envelope_id = "task-admission-" + hashlib.sha256(
(idempotency_key + "\0" + request_digest + "\0" + nonce).encode("utf-8")
).hexdigest()[:32]
envelope = TaskAdmissionEnvelope(
envelope_id=envelope_id,
idempotency_key=idempotency_key,
task_id=task_id,
repository=repository,
authority_base_oid=authority_base_oid,
target_branch=target_branch,
generation=generation,
execution_digest=execution_digest,
requested_capabilities=requested_capabilities,
file_allowlist=file_allowlist,
risk=risk,
trust_domain=self.policy.trust_domain,
policy_digest=policy_digest,
issued_at=now,
not_before=now,
expires_at=now + ttl,
nonce=nonce,
request_digest=request_digest,
)
envelope_dict = envelope.as_dict()
persisted = self.store.save_envelope(
idempotency_key=idempotency_key,
envelope_id=envelope_id,
request_digest=request_digest,
envelope_json=canonical_json_bytes(envelope_dict).decode("utf-8"),
created_at=now,
)
won = persisted["envelope_id"] == envelope_id
return {
"schema": SCHEMA_ENVELOPE,
"envelope": persisted,
"canonical_payload_base64": base64.b64encode(
canonical_json_bytes(persisted)
).decode("ascii"),
"idempotent": not won,
}
def _load_envelope(self, envelope_id: str) -> TaskAdmissionEnvelope:
row = self.store.get_envelope(envelope_id)
if row is None:
raise TaskAdmissionError("TASK_ADMISSION_ENVELOPE_NOT_FOUND", 404)
raw = json.loads(row["envelope_json"])
return TaskAdmissionEnvelope(
**{
**raw,
"requested_capabilities": tuple(raw["requested_capabilities"]),
"file_allowlist": tuple(raw["file_allowlist"]),
}
)
def _verify_epoch(
self,
*,
epoch_id: str,
envelope: TaskAdmissionEnvelope,
now: int,
) -> OwnerDelegationEpoch:
epoch = self.authorization.verify_delegation_epoch(epoch_id)
if epoch.schema != SCHEMA_EPOCH:
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_SCHEMA_INVALID", 403)
if epoch.epoch_id != epoch_id:
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_ID_MISMATCH", 403)
if (
epoch.trust_domain != envelope.trust_domain
or epoch.repository != envelope.repository
or epoch.policy_digest != envelope.policy_digest
):
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_BINDING_MISMATCH", 403)
if epoch.revocation_epoch != 0:
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_REVOKED", 403)
if not epoch.not_before <= now < epoch.expires_at:
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_EXPIRED", 403)
if not any(
envelope.target_branch.startswith(prefix)
for prefix in epoch.allowed_branch_prefixes
):
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_BRANCH_REJECTED", 403)
caps = set(envelope.requested_capabilities)
if not caps.issubset(epoch.allowed_capabilities):
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_CAPABILITY_WIDENING", 403)
if _risk_rank(envelope.risk) > _risk_rank(epoch.risk_ceiling):
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_RISK_WIDENING", 403)
for path in envelope.file_allowlist:
if not any(
path == prefix.rstrip("/")
or path.startswith(prefix.rstrip("/") + "/")
for prefix in epoch.allowed_file_prefixes
):
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_FILE_WIDENING", 403)
return epoch
def authorize_and_admit(self, raw: Mapping[str, Any]) -> dict[str, Any]:
required = {"envelope_id", "authorization_kind"}
allowed = required | {
"delegation_epoch_id",
"owner_key_id",
"signature_base64",
}
if set(raw) - allowed or not required.issubset(raw):
raise TaskAdmissionError("TASK_ADMISSION_AUTH_FIELDS_INVALID")
envelope_id = str(raw["envelope_id"])
with self.store.admission_lock(envelope_id):
# Every replay revalidates policy, temporal bounds, revocation and
# proof after the crash-releasing lock is owned.
envelope = self._load_envelope(envelope_id)
now = int(self.clock())
if envelope.policy_digest != self.policy.normalized_policy_digest():
raise TaskAdmissionError("TASK_ADMISSION_POLICY_DIGEST_DRIFT", 403)
if now < envelope.not_before or now >= envelope.expires_at:
raise TaskAdmissionError("TASK_ADMISSION_ENVELOPE_EXPIRED", 403)
kind = str(raw["authorization_kind"])
parent_epoch: OwnerDelegationEpoch | None = None
if kind == "delegation_epoch":
epoch_id = str(raw.get("delegation_epoch_id") or "")
if not epoch_id:
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_REQUIRED")
parent_epoch = self._verify_epoch(epoch_id=epoch_id, envelope=envelope, now=now)
owner_identity = parent_epoch.owner_identity
parent_epoch_id: str | None = parent_epoch.epoch_id
elif kind == "owner_signature":
key_id = str(raw.get("owner_key_id") or "")
signature = str(raw.get("signature_base64") or "")
if not KEY_ID_RE.fullmatch(key_id) or not signature:
raise TaskAdmissionError("TASK_ADMISSION_OWNER_SIGNATURE_REQUIRED")
owner_identity = self.authorization.verify_owner_signature(
key_id=key_id,
canonical_payload=canonical_json_bytes(envelope.as_dict()),
signature_b64=signature,
)
if not isinstance(owner_identity, str) or not owner_identity:
raise TaskAdmissionError("TASK_ADMISSION_OWNER_SIGNATURE_INVALID", 403)
parent_epoch_id = None
else:
raise TaskAdmissionError("TASK_ADMISSION_AUTHORIZATION_KIND_REJECTED", 403)
authorization_digest = sha256_json(dict(raw))
grant_id = "task-grant-" + hashlib.sha256(
(envelope.envelope_id + "\0" + kind + "\0" + owner_identity + "\0" + (parent_epoch_id or "")).encode("utf-8")
).hexdigest()[:32]
grant = TaskAdmissionGrant(
grant_id=grant_id, envelope_id=envelope.envelope_id, task_id=envelope.task_id,
repository=envelope.repository, authority_base_oid=envelope.authority_base_oid,
target_branch=envelope.target_branch, generation=envelope.generation,
execution_digest=envelope.execution_digest, capabilities=envelope.requested_capabilities,
file_allowlist=envelope.file_allowlist, risk=envelope.risk,
trust_domain=envelope.trust_domain, policy_digest=envelope.policy_digest,
authorization_kind=kind, owner_identity=owner_identity,
parent_epoch_id=parent_epoch_id, issued_at=now,
expires_at=min(
envelope.expires_at,
now + self.policy.max_task_ttl_seconds,
parent_epoch.expires_at if parent_epoch is not None else envelope.expires_at,
),
)
persisted_grant, _ = self.store.claim_grant(
envelope_id=envelope_id, grant=grant,
authorization_digest=authorization_digest, created_at=now,
)
if now >= persisted_grant.expires_at:
raise TaskAdmissionError("TASK_ADMISSION_GRANT_EXPIRED", 403)
if parent_epoch is not None and persisted_grant.expires_at > parent_epoch.expires_at:
raise TaskAdmissionError("TASK_ADMISSION_GRANT_PARENT_EXPIRY_CONFLICT", 409)
existing_receipt = self.store.receipt(envelope_id)
if existing_receipt is not None:
return {**existing_receipt, "idempotent": True}
try:
result = self.task_port.admit(persisted_grant)
except TaskAdmissionError:
raise
except Exception as exc:
raise TaskAdmissionError("TASK_ADMISSION_TASK_STATE_UNKNOWN", 503) from exc
if not isinstance(result, Mapping):
raise TaskAdmissionError("TASK_ADMISSION_PORT_RESPONSE_INVALID", 502)
state = str(result.get("state") or result.get("status") or "")
if str(result.get("task_id") or "") != persisted_grant.task_id:
raise TaskAdmissionError("TASK_ADMISSION_PORT_TASK_MISMATCH", 502)
if state not in {"QUEUED", "CLAIMED", "RUNNING"}:
raise TaskAdmissionError("TASK_ADMISSION_PORT_STATE_INVALID", 502, details={"state": state})
if self.post_admission is not None:
projection_result: Mapping[str, Any] = result
projection_reader = getattr(self.task_port, "projection_result", None)
if callable(projection_reader):
try:
candidate = projection_reader(persisted_grant, result)
except TaskAdmissionError:
raise
except Exception as exc:
raise TaskAdmissionError(
"TASK_ADMISSION_EXECUTION_PROJECTION_FAILED", 503
) from exc
if not isinstance(candidate, Mapping):
raise TaskAdmissionError(
"TASK_ADMISSION_EXECUTION_PROJECTION_INVALID", 503
)
projection_result = candidate
try:
projection = self.post_admission(
persisted_grant, projection_result
)
except TaskAdmissionError:
raise
except Exception as exc:
raise TaskAdmissionError(
"TASK_ADMISSION_EXECUTION_PROJECTION_FAILED", 503
) from exc
if projection is not None and not isinstance(projection, Mapping):
raise TaskAdmissionError(
"TASK_ADMISSION_EXECUTION_PROJECTION_INVALID", 503
)
receipt = {
"schema": SCHEMA_RECEIPT, "envelope_id": persisted_grant.envelope_id,
"grant_id": persisted_grant.grant_id, "task_id": persisted_grant.task_id,
"task_state": state, "authorization_kind": persisted_grant.authorization_kind,
"owner_identity": persisted_grant.owner_identity, "parent_epoch_id": persisted_grant.parent_epoch_id,
"authority_base_oid": persisted_grant.authority_base_oid,
"target_branch": persisted_grant.target_branch, "generation": persisted_grant.generation,
"execution_digest": persisted_grant.execution_digest,
"grant_digest_sha256": sha256_json(persisted_grant.as_dict()),
"admitted_at": now, "idempotent": False,
}
return self.store.save_grant_and_receipt(
envelope_id=envelope_id, grant=persisted_grant, receipt=receipt, created_at=now,
)
def status(self, envelope_id: str) -> dict[str, Any]:
return self.store.status(envelope_id)
@@ -0,0 +1,87 @@
"""Bridge contract surface for Task Admission Bootstrap.
This module is intentionally transport-agnostic. The live Bridge may merge
these routes/operations into its existing authenticated/HTTPS surface, but the
authorization proof for the authorize operation is the verified delegation
epoch or Owner signature carried by the request itself.
No handler here can issue TaskPrincipal assertions or WriterLeases. Those remain
strictly downstream of a real admitted/claimed Agent Task.
"""
from __future__ import annotations
from typing import Any, Mapping
from aios_task_admission_bootstrap import TaskAdmissionBootstrapService
TASK_ADMISSION_ROUTES = {
"/v2/task-admission/envelope": "getTaskAdmissionEnvelope",
"/v2/task-admission/authorize": "submitTaskAdmissionAuthorization",
"/v2/task-admission/status": "getTaskAdmissionStatus",
}
TASK_ADMISSION_OPERATION_REQUIRED_FIELDS = {
"getTaskAdmissionEnvelope": (
"idempotency_key",
"task_id",
"repository",
"authority_base_oid",
"target_branch",
"generation",
"execution_digest",
"requested_capabilities",
"file_allowlist",
"risk",
),
"submitTaskAdmissionAuthorization": (
"envelope_id",
"authorization_kind",
),
"getTaskAdmissionStatus": ("envelope_id",),
}
TASK_ADMISSION_OPERATION_OPTIONAL_FIELDS = {
"getTaskAdmissionEnvelope": ("expires_in_seconds",),
"submitTaskAdmissionAuthorization": (
"delegation_epoch_id",
"owner_key_id",
"signature_base64",
),
"getTaskAdmissionStatus": (),
}
class TaskAdmissionBridgeContract:
def __init__(self, service: TaskAdmissionBootstrapService) -> None:
self.service = service
@staticmethod
def _validate_fields(operation: str, request: Mapping[str, Any]) -> None:
required = set(TASK_ADMISSION_OPERATION_REQUIRED_FIELDS[operation])
allowed = required | set(TASK_ADMISSION_OPERATION_OPTIONAL_FIELDS[operation])
if not required.issubset(request):
raise ValueError("TASK_ADMISSION_BRIDGE_REQUIRED_FIELDS_MISSING")
if set(request) - allowed:
raise ValueError("TASK_ADMISSION_BRIDGE_UNKNOWN_FIELDS")
def dispatch(self, operation: str, request: Mapping[str, Any]) -> dict[str, Any]:
if operation not in TASK_ADMISSION_OPERATION_REQUIRED_FIELDS:
raise ValueError("TASK_ADMISSION_BRIDGE_OPERATION_NOT_FOUND")
self._validate_fields(operation, request)
if operation == "getTaskAdmissionEnvelope":
return self.service.create_envelope(request)
if operation == "submitTaskAdmissionAuthorization":
return self.service.authorize_and_admit(request)
if operation == "getTaskAdmissionStatus":
return self.service.status(str(request["envelope_id"]))
raise ValueError("TASK_ADMISSION_BRIDGE_OPERATION_NOT_FOUND")
def handlers(self) -> dict[str, Any]:
return {
operation: (
lambda request, operation=operation: self.dispatch(operation, request)
)
for operation in TASK_ADMISSION_OPERATION_REQUIRED_FIELDS
}
@@ -0,0 +1,841 @@
"""Narrow adapters for the already-authorized Owner and Agent Task seams.
This file intentionally contains no credential lookup, HTTP client construction,
or fallback identity. The service host injects its authenticated durable-epoch
provider and its service-managed Agent Task transport; a request body can never
provide either authority.
"""
from __future__ import annotations
import base64
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import importlib.util
import json
import os
from pathlib import Path
import stat
import time
from typing import Any, Callable, Mapping, Protocol
import urllib.error
import urllib.request
from aios_task_admission_bootstrap import (
OwnerDelegationEpoch,
TaskAdmissionError,
TaskAdmissionGrant,
canonical_json_bytes,
sha256_json,
)
OWNER_NAMESPACE = "aios-owner-approval-v1"
OWNER_SIGNER_IDENTITY = "aios-task-grant-signer"
class OfficialTaskTransport(Protocol):
"""Service-managed transport; it owns the Task API control credential."""
def request(self, method: str, path: str, body: Mapping[str, Any] | None = None) -> Mapping[str, Any]:
...
class BoundOwnerAuthorizationBackend:
"""Bind epoch lookup to trusted transport identity, never caller JSON."""
def __init__(
self,
*,
authenticated_caller: Mapping[str, Any],
verified_epoch_provider: Callable[..., OwnerDelegationEpoch],
signature_verifier: Any,
) -> None:
if not isinstance(authenticated_caller, Mapping) or not authenticated_caller:
raise ValueError("AUTHENTICATED_CALLER_REQUIRED")
self._caller = dict(authenticated_caller)
self._epoch_provider = verified_epoch_provider
self._signature_verifier = signature_verifier
def verify_delegation_epoch(self, epoch_id: str) -> OwnerDelegationEpoch:
try:
epoch = self._epoch_provider(epoch_id=epoch_id, caller=dict(self._caller))
except TypeError as exc:
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_BACKEND_INVALID", 503) from exc
if not isinstance(epoch, OwnerDelegationEpoch):
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_BACKEND_INVALID", 503)
return epoch
def verify_owner_signature(
self, *, key_id: str, canonical_payload: bytes, signature_b64: str
) -> str:
try:
armored = base64.b64decode(signature_b64, validate=True).decode("utf-8")
except (ValueError, UnicodeDecodeError) as exc:
raise TaskAdmissionError("TASK_ADMISSION_OWNER_SIGNATURE_INVALID", 403) from exc
verify = getattr(self._signature_verifier, "verify", None)
if not callable(verify) or verify(
namespace=OWNER_NAMESPACE,
payload=canonical_payload,
signature=armored,
key_id=key_id,
) is not True:
raise TaskAdmissionError("TASK_ADMISSION_OWNER_SIGNATURE_INVALID", 403)
return OWNER_SIGNER_IDENTITY
class AgentTaskAdmissionPort:
"""Official Task API adapter with exact identity reconciliation.
It never exposes or returns the API control bearer. A timeout or 5xx is
reconciled by exact GET; if that cannot prove the state, the durable caller
receives UNKNOWN instead of a second blind POST.
"""
def __init__(
self,
transport: OfficialTaskTransport,
*,
execution_plan_provider: Callable[[TaskAdmissionGrant], Mapping[str, Any]] | None = None,
) -> None:
if not callable(getattr(transport, "request", None)):
raise ValueError("OFFICIAL_TASK_TRANSPORT_REQUIRED")
self._transport = transport
self._execution_plan_provider = execution_plan_provider
def _execution_plan(self, grant: TaskAdmissionGrant) -> dict[str, Any]:
if self._execution_plan_provider is None:
raise TaskAdmissionError("TASK_ADMISSION_EXECUTION_PLAN_REQUIRED", 503)
try:
raw = self._execution_plan_provider(grant)
except TaskAdmissionError:
raise
except Exception as exc:
raise TaskAdmissionError("TASK_ADMISSION_EXECUTION_PLAN_UNAVAILABLE", 503) from exc
if not isinstance(raw, Mapping):
raise TaskAdmissionError("TASK_ADMISSION_EXECUTION_PLAN_INVALID", 403)
expected = {
"repository": grant.repository,
"target_branch": grant.target_branch,
"generation": grant.generation,
"execution_digest": grant.execution_digest,
}
if any(canonical_json_bytes(raw.get(key)) != canonical_json_bytes(value) for key, value in expected.items()):
raise TaskAdmissionError("TASK_ADMISSION_EXECUTION_PLAN_CONFLICT", 409)
external_job_id = raw.get("external_job_id")
if not isinstance(external_job_id, str) or not external_job_id.strip():
raise TaskAdmissionError("TASK_ADMISSION_EXECUTION_PLAN_INVALID", 403)
# The trusted provider resolves the immutable execution payload by its
# signed digest. Preserve that payload: returning just coordinates
# would discard the executable plan needed by the real worker.
return dict(raw)
@staticmethod
def _metadata(grant: TaskAdmissionGrant, execution: Mapping[str, Any]) -> dict[str, Any]:
return {
"task_admission_schema": "aios.task-admission-grant.v1",
"grant_id": grant.grant_id,
"grant_digest_sha256": sha256_json(grant.as_dict()),
"repository": grant.repository,
"authority_base_oid": grant.authority_base_oid,
"target_branch": grant.target_branch,
"generation": grant.generation,
"execution_digest": grant.execution_digest,
"file_allowlist": list(grant.file_allowlist),
"capabilities": list(grant.capabilities),
"risk": grant.risk,
"execution": dict(execution),
}
@staticmethod
def _deadline(epoch: int) -> str:
# TaskStore.iso preserves the explicit UTC offset; exact reconciliation
# compares against that stored representation rather than a Z variant.
return datetime.fromtimestamp(epoch, timezone.utc).isoformat()
def _body(self, grant: TaskAdmissionGrant, execution: Mapping[str, Any]) -> dict[str, Any]:
metadata = self._metadata(grant, execution)
immutable = canonical_json_bytes(metadata).decode("utf-8")
return {
"task_id": grant.task_id,
"goal": "Execute the immutable governed task admission grant.",
"instructions": "Use only the admitted immutable metadata; execution plan digest: " + grant.execution_digest,
"success_criteria": ["Task processing preserves the admitted immutable identity."],
"required_capabilities": sorted(grant.capabilities),
"constraints": {
"repository": grant.repository,
"authority_base_oid": grant.authority_base_oid,
"target_branch": grant.target_branch,
"generation": grant.generation,
"file_allowlist": list(grant.file_allowlist),
"immutable_metadata": immutable,
},
"metadata": metadata,
"deadline": self._deadline(grant.expires_at),
"priority": 0,
"max_attempts": 1,
"idempotency_key": "task-admission/" + grant.grant_id,
}
def _exact(
self, task: Mapping[str, Any], grant: TaskAdmissionGrant, execution: Mapping[str, Any]
) -> dict[str, Any]:
expected = self._body(grant, execution)
immutable_fields = (
"task_id", "goal", "instructions", "success_criteria",
"required_capabilities", "constraints", "metadata", "deadline",
"priority", "max_attempts",
)
def stable(field: str, value: Any) -> Any:
"""Exclude only re-admission provenance, never execution scope."""
if field == "metadata" and isinstance(value, Mapping):
return {
key: item
for key, item in value.items()
if key not in {"grant_id", "grant_digest_sha256"}
}
if field == "constraints" and isinstance(value, Mapping):
return {
key: item
for key, item in value.items()
if key != "immutable_metadata"
}
return value
if any(
canonical_json_bytes(stable(field, task.get(field)))
!= canonical_json_bytes(stable(field, expected[field]))
for field in immutable_fields
):
raise TaskAdmissionError("TASK_ADMISSION_TASK_IDENTITY_CONFLICT", 409)
state = str(task.get("state") or task.get("status") or "")
if state not in {"QUEUED", "CLAIMED", "RUNNING"}:
raise TaskAdmissionError("TASK_ADMISSION_PORT_STATE_INVALID", 502, details={"state": state})
# The public admission port reports only authoritative Task identity
# and state. The execution-plan identity is an internal capability
# used later by a trusted post-admission projection, never a caller
# visible part of this HTTP read model.
return {"task_id": grant.task_id, "state": state}
def _read(self, grant: TaskAdmissionGrant, execution: Mapping[str, Any]) -> dict[str, Any] | None:
try:
task = self._transport.request("GET", "/v1/tasks/" + grant.task_id)
except Exception as exc:
# A transport is expected to expose a status for normal 404s. A
# generic failure is ambiguous and must not be treated as absent.
statuses = [
getattr(exc, attribute, None)
for attribute in ("status", "http_status", "code")
if getattr(exc, attribute, None) is not None
]
if statuses and all(status == 404 for status in statuses):
return None
raise TaskAdmissionError("TASK_ADMISSION_TASK_STATE_UNKNOWN", 503) from exc
if not isinstance(task, Mapping):
raise TaskAdmissionError("TASK_ADMISSION_PORT_RESPONSE_INVALID", 502)
return self._exact(task, grant, execution)
def admit(self, grant: TaskAdmissionGrant) -> Mapping[str, Any]:
execution = self._execution_plan(grant)
existing = self._read(grant, execution)
if existing is not None:
return existing
try:
created = self._transport.request("POST", "/v1/tasks", self._body(grant, execution))
except Exception as exc:
recovered = self._read(grant, execution)
if recovered is not None:
return recovered
raise TaskAdmissionError("TASK_ADMISSION_TASK_STATE_UNKNOWN", 503) from exc
if not isinstance(created, Mapping):
raise TaskAdmissionError("TASK_ADMISSION_PORT_RESPONSE_INVALID", 502)
# Authoritative GET prevents a POST response from becoming the source
# of task identity/state and handles a task claimed before readback.
recovered = self._read(grant, execution)
if recovered is None:
raise TaskAdmissionError("TASK_ADMISSION_TASK_STATE_UNKNOWN", 503)
return recovered
def projection_result(
self,
grant: TaskAdmissionGrant,
admitted: Mapping[str, Any],
) -> Mapping[str, Any]:
"""Return the server-side execution identity for a validated readback.
`admit()` intentionally keeps this identity out of its public result.
The bootstrap service calls this only for its local post-admission
projection; the value comes from the content-addressed execution-plan
provider, not from the HTTP caller or the Task API response.
"""
if str(admitted.get("task_id") or "") != grant.task_id:
raise TaskAdmissionError("TASK_ADMISSION_PORT_TASK_MISMATCH", 502)
state = str(admitted.get("state") or admitted.get("status") or "")
if state not in {"QUEUED", "CLAIMED", "RUNNING"}:
raise TaskAdmissionError(
"TASK_ADMISSION_PORT_STATE_INVALID", 502, details={"state": state}
)
execution = self._execution_plan(grant)
external_job_id = execution.get("external_job_id")
if not isinstance(external_job_id, str) or not external_job_id.strip():
raise TaskAdmissionError("TASK_ADMISSION_EXECUTION_PLAN_INVALID", 403)
return {
"task_id": grant.task_id,
"state": state,
"external_job_id": external_job_id,
}
class TaskAdmissionHostError(RuntimeError):
"""Stable fail-closed host-composition error."""
class OfficialTaskHTTPError(RuntimeError):
def __init__(self, status: int, body: object | None = None) -> None:
super().__init__(f"AGENT_TASK_HTTP_{status}")
self.status = status
self.http_status = status
self.code = status
self.body = body
def _host_strict_object(pairs: list[tuple[str, Any]]) -> dict[str, Any]:
value: dict[str, Any] = {}
for key, item in pairs:
if key in value:
raise TaskAdmissionHostError("DUPLICATE_JSON_FIELD")
value[key] = item
return value
def _host_read_json_object(path: Path) -> dict[str, Any]:
try:
value = json.loads(
path.read_text(encoding="utf-8"),
object_pairs_hook=_host_strict_object,
)
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
raise TaskAdmissionHostError("JSON_INPUT_INVALID") from exc
if not isinstance(value, dict):
raise TaskAdmissionHostError("JSON_OBJECT_REQUIRED")
return value
def _host_posix_metadata_enforced() -> bool:
"""Return whether POSIX ownership/mode bits are security-significant here.
The live AIOS authority/runtime is Linux, where mode/uid checks are a real
trust boundary. Windows development/test files expose synthesized POSIX
mode bits that do not represent ACL authority, so interpreting ``022``
there would reject every secure temporary directory without improving
live security.
"""
return os.name == "posix"
def _host_require_regular(path: Path, *, root_owned: bool) -> None:
try:
info = path.lstat()
except OSError as exc:
raise TaskAdmissionHostError("SECURE_FILE_MISSING") from exc
if path.is_symlink() or not stat.S_ISREG(info.st_mode):
raise TaskAdmissionHostError("SECURE_FILE_INVALID")
if _host_posix_metadata_enforced():
if stat.S_IMODE(info.st_mode) & 0o022:
raise TaskAdmissionHostError("SECURE_FILE_WRITABLE_BY_GROUP_OR_OTHER")
if root_owned and info.st_uid != 0:
raise TaskAdmissionHostError("SECURE_FILE_NOT_ROOT_OWNED")
def _host_require_directory(path: Path, *, root_owned: bool) -> None:
try:
info = path.lstat()
except OSError as exc:
raise TaskAdmissionHostError("SECURE_DIRECTORY_MISSING") from exc
if path.is_symlink() or not stat.S_ISDIR(info.st_mode):
raise TaskAdmissionHostError("SECURE_DIRECTORY_INVALID")
if _host_posix_metadata_enforced():
if stat.S_IMODE(info.st_mode) & 0o022:
raise TaskAdmissionHostError("SECURE_DIRECTORY_WRITABLE_BY_GROUP_OR_OTHER")
if root_owned and info.st_uid != 0:
raise TaskAdmissionHostError("SECURE_DIRECTORY_NOT_ROOT_OWNED")
def load_owner_approval_verifier(module_path: Path) -> Any:
"""Load the already-installed trusted public Owner verifier."""
_host_require_regular(module_path, root_owned=True)
spec = importlib.util.spec_from_file_location(
"aios_live_owner_approval_verifier",
module_path,
)
if spec is None or spec.loader is None:
raise TaskAdmissionHostError("OWNER_VERIFIER_IMPORT_INVALID")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
cls = getattr(module, "GrantRootOwnerApprovalVerifier", None)
if cls is None:
raise TaskAdmissionHostError("OWNER_VERIFIER_CLASS_MISSING")
verifier = cls()
if not callable(getattr(verifier, "verify", None)):
raise TaskAdmissionHostError("OWNER_VERIFIER_INVALID")
return verifier
@dataclass(frozen=True, slots=True)
class SignedEpochDirectoryProvider:
"""Re-verify the exact signed Owner epoch on every admission use."""
root: Path
verifier: Any
revocation_file: Path
expected_caller: Mapping[str, Any]
require_root_owner: bool = True
clock: Any = time.time
def _revoked_ids(self) -> set[str]:
_host_require_regular(
self.revocation_file,
root_owned=self.require_root_owner,
)
value = _host_read_json_object(self.revocation_file)
if (
set(value) != {"schema", "revoked_epoch_ids"}
or value.get("schema") != "aios.owner-delegation-revocations.v1"
or not isinstance(value.get("revoked_epoch_ids"), list)
or not all(
isinstance(item, str) and item
for item in value["revoked_epoch_ids"]
)
):
raise TaskAdmissionHostError(
"OWNER_EPOCH_REVOCATION_REGISTRY_INVALID"
)
return set(value["revoked_epoch_ids"])
def __call__(
self,
*,
epoch_id: str,
caller: Mapping[str, Any],
) -> OwnerDelegationEpoch:
if dict(caller) != dict(self.expected_caller):
raise TaskAdmissionError(
"TASK_ADMISSION_CALLER_BINDING_MISMATCH",
403,
)
_host_require_directory(
self.root,
root_owned=self.require_root_owner,
)
canonical = self.root / "owner-delegation-epoch.canonical.json"
signature = self.root / "owner-delegation-epoch.sig"
key_id_file = self.root / "owner-delegation-epoch.key-id"
for path in (canonical, signature, key_id_file):
_host_require_regular(
path,
root_owned=self.require_root_owner,
)
payload = canonical.read_bytes()
value = _host_read_json_object(canonical)
if canonical_json_bytes(value) != payload:
raise TaskAdmissionError(
"TASK_ADMISSION_EPOCH_CANONICAL_MISMATCH",
403,
)
if value.get("schema") != "aios.owner-delegation-epoch.v1":
raise TaskAdmissionError(
"TASK_ADMISSION_EPOCH_SCHEMA_INVALID",
403,
)
if value.get("epoch_id") != epoch_id:
raise TaskAdmissionError(
"TASK_ADMISSION_EPOCH_ID_MISMATCH",
403,
)
if value.get("owner_identity") != OWNER_SIGNER_IDENTITY:
raise TaskAdmissionError(
"TASK_ADMISSION_EPOCH_OWNER_IDENTITY_INVALID",
403,
)
if value.get("revocation_epoch") != 0:
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_REVOKED", 403)
if epoch_id in self._revoked_ids():
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_REVOKED", 403)
key_id = key_id_file.read_text(encoding="utf-8").strip()
armored = signature.read_text(encoding="utf-8")
if self.verifier.verify(
namespace=OWNER_NAMESPACE,
payload=payload,
signature=armored,
key_id=key_id,
) is not True:
raise TaskAdmissionError(
"TASK_ADMISSION_EPOCH_SIGNATURE_INVALID",
403,
)
now = int(self.clock())
not_before = value.get("not_before")
expires_at = value.get("expires_at")
if (
not isinstance(not_before, int)
or isinstance(not_before, bool)
or not isinstance(expires_at, int)
or isinstance(expires_at, bool)
or not not_before <= now < expires_at
):
raise TaskAdmissionError("TASK_ADMISSION_EPOCH_EXPIRED", 403)
branches = value.get("allowed_branch_prefixes")
capabilities = value.get("allowed_capabilities")
files = value.get("allowed_file_prefixes")
if (
not isinstance(branches, list)
or not all(isinstance(item, str) and item for item in branches)
or not isinstance(capabilities, list)
or not all(
isinstance(item, str) and item
for item in capabilities
)
or not isinstance(files, list)
or not all(isinstance(item, str) and item for item in files)
):
raise TaskAdmissionError(
"TASK_ADMISSION_EPOCH_SCOPE_INVALID",
403,
)
return OwnerDelegationEpoch(
epoch_id=str(value["epoch_id"]),
owner_identity=str(value["owner_identity"]),
trust_domain=str(value["trust_domain"]),
repository=str(value["repository"]),
allowed_branch_prefixes=tuple(branches),
allowed_capabilities=frozenset(capabilities),
allowed_file_prefixes=tuple(files),
risk_ceiling=str(value["risk_ceiling"]),
issued_at=int(value["issued_at"]),
not_before=not_before,
expires_at=expires_at,
revocation_epoch=int(value["revocation_epoch"]),
policy_digest=str(value["policy_digest"]),
schema=str(value["schema"]),
)
@dataclass(frozen=True, slots=True)
class ExecutionPlanDirectoryProvider:
"""Resolve a trusted execution plan by its canonical SHA256 digest."""
root: Path
require_root_owner: bool = True
def __call__(self, grant: TaskAdmissionGrant) -> Mapping[str, Any]:
digest = str(grant.execution_digest)
if (
len(digest) != 64
or any(ch not in "0123456789abcdef" for ch in digest)
):
raise TaskAdmissionError(
"TASK_ADMISSION_EXECUTION_PLAN_DIGEST_INVALID",
403,
)
_host_require_directory(
self.root,
root_owned=self.require_root_owner,
)
path = self.root / (digest + ".json")
_host_require_regular(
path,
root_owned=self.require_root_owner,
)
value = _host_read_json_object(path)
if hashlib.sha256(canonical_json_bytes(value)).hexdigest() != digest:
raise TaskAdmissionError(
"TASK_ADMISSION_EXECUTION_PLAN_DIGEST_MISMATCH",
403,
)
expected = {
"task_id": grant.task_id,
"repository": grant.repository,
"target_branch": grant.target_branch,
"generation": grant.generation,
}
if any(value.get(key) != item for key, item in expected.items()):
raise TaskAdmissionError(
"TASK_ADMISSION_EXECUTION_PLAN_CONFLICT",
409,
)
external_job_id = value.get("external_job_id")
if not isinstance(external_job_id, str) or not external_job_id.strip():
raise TaskAdmissionError(
"TASK_ADMISSION_EXECUTION_PLAN_INVALID",
403,
)
return {**value, "execution_digest": digest}
@dataclass(frozen=True, slots=True)
class LocalAgentTaskHTTPTransport:
"""Server-side localhost Task API transport; token never enters caller JSON."""
base_url: str
token_file: Path
timeout_seconds: float = 5.0
require_root_owner: bool = True
def _token(self) -> str:
_host_require_regular(
self.token_file,
root_owned=self.require_root_owner,
)
token = self.token_file.read_text(encoding="utf-8").strip()
if len(token) < 16 or any(ch.isspace() for ch in token):
raise TaskAdmissionHostError("AGENT_TASK_CONTROL_TOKEN_INVALID")
return token
def request(
self,
method: str,
path: str,
body: Mapping[str, Any] | None = None,
) -> Mapping[str, Any]:
if method not in {"GET", "POST"}:
raise TaskAdmissionHostError("AGENT_TASK_HTTP_METHOD_INVALID")
if not path.startswith("/v1/tasks"):
raise TaskAdmissionHostError("AGENT_TASK_HTTP_PATH_INVALID")
raw = None if body is None else canonical_json_bytes(dict(body))
request = urllib.request.Request(
self.base_url.rstrip("/") + path,
data=raw,
method=method,
headers={
"Accept": "application/json",
"Authorization": "Bearer " + self._token(),
**(
{"Content-Type": "application/json"}
if raw is not None
else {}
),
},
)
try:
with urllib.request.urlopen(
request,
timeout=self.timeout_seconds,
) as response:
value = json.loads(
response.read().decode("utf-8"),
object_pairs_hook=_host_strict_object,
)
except urllib.error.HTTPError as exc:
body_value: object | None = None
try:
body_value = json.loads(exc.read().decode("utf-8"))
except Exception:
body_value = None
raise OfficialTaskHTTPError(
exc.code,
body_value,
) from exc
if not isinstance(value, Mapping):
raise TaskAdmissionHostError("AGENT_TASK_HTTP_RESPONSE_INVALID")
return dict(value)
def task_admission_policy_from_epoch(
epoch: OwnerDelegationEpoch,
) -> Any:
"""Build a local ceiling exactly narrowed to the signed epoch."""
from aios_task_admission_bootstrap import TaskAdmissionPolicy
return TaskAdmissionPolicy(
repository=epoch.repository,
trust_domain=epoch.trust_domain,
branch_prefix="agent/",
allowed_capabilities=frozenset(epoch.allowed_capabilities),
max_file_count=min(
64,
max(1, len(epoch.allowed_file_prefixes)),
),
max_task_ttl_seconds=min(
3600,
max(60, epoch.expires_at - epoch.not_before),
),
max_envelope_ttl_seconds=900,
max_risk=epoch.risk_ceiling,
policy_digest=epoch.policy_digest,
)
def _build_task_admission_handlers(
*,
caller_service: str,
epoch_root: Path,
epoch_id: str,
revocation_file: Path,
execution_plan_dir: Path,
state_db: Path,
task_api_url: str,
task_control_token_file: Path,
owner_verifier_module: Path,
post_admission: Callable[[TaskAdmissionGrant, Mapping[str, Any]], Any] | None = None,
) -> Mapping[str, Any]:
"""Compose handlers from already-trusted server-side coordinates."""
from aios_task_admission_bootstrap import (
TaskAdmissionBootstrapService,
TaskAdmissionStore,
)
from aios_task_admission_bridge_contract import TaskAdmissionBridgeContract
verifier = load_owner_approval_verifier(owner_verifier_module)
caller = {
"service": caller_service,
"trust_domain": "aios.owner",
}
provider = SignedEpochDirectoryProvider(
root=epoch_root,
verifier=verifier,
revocation_file=revocation_file,
expected_caller=caller,
require_root_owner=True,
)
epoch = provider(epoch_id=epoch_id, caller=caller)
authorization = BoundOwnerAuthorizationBackend(
authenticated_caller=caller,
verified_epoch_provider=provider,
signature_verifier=verifier,
)
task_port = AgentTaskAdmissionPort(
LocalAgentTaskHTTPTransport(
base_url=task_api_url,
token_file=task_control_token_file,
require_root_owner=True,
),
execution_plan_provider=ExecutionPlanDirectoryProvider(
root=execution_plan_dir,
require_root_owner=True,
),
)
service = TaskAdmissionBootstrapService(
store=TaskAdmissionStore(state_db),
authorization=authorization,
task_port=task_port,
policy=task_admission_policy_from_epoch(epoch),
post_admission=post_admission,
)
return TaskAdmissionBridgeContract(service).handlers()
def build_task_admission_handlers_from_activation_file(
activation_file: Path,
*,
caller_service: str = "aios-chatgpt-bridge-v2-staging.service",
post_admission: Callable[[TaskAdmissionGrant, Mapping[str, Any]], Any] | None = None,
) -> Mapping[str, Any]:
"""Load one root-owned, fail-closed staging activation descriptor."""
_host_require_regular(activation_file, root_owned=True)
value = _host_read_json_object(activation_file)
required = {
"schema",
"epoch_root",
"epoch_id",
"revocation_file",
"execution_plan_dir",
"state_db",
"task_api_url",
"task_control_token_file",
"owner_verifier_module",
}
if set(value) != required or value.get("schema") != "aios.task-admission-host-activation.v1":
raise TaskAdmissionHostError("TASK_ADMISSION_ACTIVATION_FILE_INVALID")
path_fields = (
"epoch_root",
"revocation_file",
"execution_plan_dir",
"state_db",
"task_control_token_file",
"owner_verifier_module",
)
paths: dict[str, Path] = {}
for key in path_fields:
raw = value.get(key)
if not isinstance(raw, str) or not raw.startswith("/"):
raise TaskAdmissionHostError("TASK_ADMISSION_ACTIVATION_PATH_INVALID")
path = Path(raw)
if ".." in path.parts:
raise TaskAdmissionHostError("TASK_ADMISSION_ACTIVATION_PATH_INVALID")
paths[key] = path
epoch_id = value.get("epoch_id")
task_api_url = value.get("task_api_url")
if not isinstance(epoch_id, str) or not epoch_id:
raise TaskAdmissionHostError("TASK_ADMISSION_ACTIVATION_EPOCH_INVALID")
if task_api_url != "http://127.0.0.1:18940":
raise TaskAdmissionHostError("TASK_ADMISSION_ACTIVATION_TASK_API_INVALID")
return _build_task_admission_handlers(
caller_service=caller_service,
epoch_root=paths["epoch_root"],
epoch_id=epoch_id,
revocation_file=paths["revocation_file"],
execution_plan_dir=paths["execution_plan_dir"],
state_db=paths["state_db"],
task_api_url=task_api_url,
task_control_token_file=paths["task_control_token_file"],
owner_verifier_module=paths["owner_verifier_module"],
post_admission=post_admission,
)
def build_task_admission_handlers_from_environment(
*,
caller_service: str = "aios-chatgpt-bridge-v2-staging.service",
post_admission: Callable[[TaskAdmissionGrant, Mapping[str, Any]], Any] | None = None,
) -> Mapping[str, Any]:
"""Compose all three handlers from server-side trusted environment only."""
required = (
"AIOS_TASK_ADMISSION_EPOCH_ROOT",
"AIOS_TASK_ADMISSION_EPOCH_ID",
"AIOS_TASK_ADMISSION_REVOCATION_FILE",
"AIOS_TASK_ADMISSION_EXECUTION_PLAN_DIR",
)
missing = [
key
for key in required
if not os.environ.get(key, "").strip()
]
if missing:
raise TaskAdmissionHostError(
"TASK_ADMISSION_HOST_ENV_MISSING:"
+ ",".join(sorted(missing))
)
return _build_task_admission_handlers(
caller_service=caller_service,
epoch_root=Path(os.environ["AIOS_TASK_ADMISSION_EPOCH_ROOT"]),
epoch_id=os.environ["AIOS_TASK_ADMISSION_EPOCH_ID"],
revocation_file=Path(os.environ["AIOS_TASK_ADMISSION_REVOCATION_FILE"]),
execution_plan_dir=Path(os.environ["AIOS_TASK_ADMISSION_EXECUTION_PLAN_DIR"]),
state_db=Path(os.environ.get(
"AIOS_TASK_ADMISSION_STATE_DB",
"/var/lib/aios-gitops/bridge-v2-staging/task-admission.sqlite3",
)),
task_api_url=os.environ.get(
"AIOS_AGENT_TASK_API_URL",
"http://127.0.0.1:18940",
),
task_control_token_file=Path(os.environ.get(
"AIOS_AGENT_TASK_CONTROL_TOKEN_FILE",
"/etc/aios-gitops/agent-task-control.token",
)),
owner_verifier_module=Path(os.environ.get(
"AIOS_OWNER_APPROVAL_VERIFIER_MODULE",
"/opt/aios/agent-task-api/"
"914eb324f657a695e2635f80cf53202eba636eb350630b12883e2741ce67e535"
"-p0-verification/runtime/app/aios_owner_approval_verifier.py",
)),
post_admission=post_admission,
)
@@ -0,0 +1,148 @@
"""Fail-closed signed deterministic execution amendment for R7."""
from datetime import datetime, timezone
import hashlib, json, sqlite3
from pathlib import Path
from typing import Mapping
from aios_task_terminal_recovery import DeterministicPlan, RecoveryError, canonical_json
AMENDMENT_SCHEMA="aios.deterministic-execution-amendment.v1"
APPLY_SCHEMA="aios.deterministic-execution-amendment-apply.v1"
NAMESPACE="aios-owner-approval-v1"
EVENT="TASK_RECOVERY_EXECUTION_AMENDMENT_APPLIED"
APPLY_STATES=frozenset({"FAILED", "QUEUED"})
def _err(code): raise RecoveryError(code)
def _sha(v): return hashlib.sha256(canonical_json(v).encode()).hexdigest()
def amendment_body(x):
return {k:v for k,v in x.items() if k not in {"signature","amendment_id"}}
def amendment_id(x): return "sha256:"+_sha(amendment_body(x))
def signed_payload(x):
return canonical_json({"amendment_id":x["amendment_id"],**amendment_body(x)}).encode()
def parse_amendment(raw):
x=dict(raw)
required={"schema","amendment_id","nonce","task_id","job_id","generation",
"repository","branch","expected_branch_oid","recovery_epoch","decision_id",
"plan_id","plan_sha256","expected_apply_status","expected_apply_revision",
"expected_apply_attempt","expected_current_capabilities","envelope",
"owner_identity","key_id","issued_at","not_before","expires_at",
"one_time_use","signature"}
if set(x)<required or x.get("schema")!=AMENDMENT_SCHEMA: _err("AMENDMENT_SCHEMA_INVALID")
if x["expected_apply_status"] not in APPLY_STATES: _err("AMENDMENT_APPLY_STATUS_INVALID")
if x.get("one_time_use") is not True: _err("AMENDMENT_ONE_TIME_REQUIRED")
if not isinstance(x["envelope"],Mapping): _err("AMENDMENT_ENVELOPE_INVALID")
if not isinstance(x["expected_current_capabilities"],list): _err("AMENDMENT_CAPABILITIES_INVALID")
for k in ("generation","expected_apply_revision","expected_apply_attempt"):
if isinstance(x[k],bool) or not isinstance(x[k],int) or x[k]<0: _err("AMENDMENT_CAS_INVALID")
for k in ("issued_at","not_before","expires_at"):
try:d=datetime.fromisoformat(str(x[k]).replace("Z","+00:00"))
except ValueError as e: raise RecoveryError("AMENDMENT_TIME_INVALID") from e
if d.tzinfo is None:_err("AMENDMENT_TIME_INVALID")
x[k]=d.astimezone(timezone.utc).isoformat().replace("+00:00","Z")
x["expected_current_capabilities"]=sorted(set(x["expected_current_capabilities"]))
x["envelope"]=dict(x["envelope"])
return x
def _validate_envelope(env,plan,x):
try:
from aios_online_agent_execution import GovernedExecutionBinding,validate_execution_envelope
binding=GovernedExecutionBinding.from_mapping(env)
e=dict(validate_execution_envelope(env,binding=binding,worker_capabilities=plan.required_capabilities))
except Exception as z: raise RecoveryError("AMENDMENT_EXECUTION_ENVELOPE_INVALID") from z
if e.get("action")!="deterministic" or e.get("adapter")!="deterministic":_err("AMENDMENT_EXECUTION_ACTION_INVALID")
if tuple(sorted(e.get("capability_scope") or ()))!=plan.required_capabilities:_err("AMENDMENT_EXECUTION_CAPABILITIES_MISMATCH")
if not e.get("change_set") or not e.get("verification"):_err("AMENDMENT_EXECUTION_OPERATIONS_REQUIRED")
if e.get("runtime_input") not in (None,{}) or e.get("source_signals") not in (None,[]):_err("AMENDMENT_STALE_RUNTIME_FIELDS_FORBIDDEN")
for k,w in (("task_id",x["task_id"]),("job_id",x["job_id"]),("repository",x["repository"]),
("generation",x["generation"]),("target_branch",x["branch"]),
("expected_branch_oid",x["expected_branch_oid"]),("base_oid",x["expected_branch_oid"]),
("plan_id",x["plan_id"]),("plan_sha256",x["plan_sha256"]),("recovery_epoch",x["recovery_epoch"])):
if e.get(k)!=w:_err("AMENDMENT_EXECUTION_BINDING_MISMATCH")
g=e.get("governance") or {}
if g.get("writer_lease") is not True:_err("AMENDMENT_WRITER_LEASE_REQUIRED")
if any(g.get(k,False) for k in ("main_write","production_write","tag_write","baseline_write","force_push","protected_ref_bypass")):_err("AMENDMENT_PROTECTED_WRITE_FORBIDDEN")
return e
class DeterministicExecutionAmendmentController:
def __init__(self,db_path,verifier,*,clock=None,envelope_validator=None):
self.db_path=Path(db_path); self.verifier=verifier
self.clock=clock or (lambda:datetime.now(timezone.utc)); self._init()
def _db(self):
c=sqlite3.connect(self.db_path,timeout=10,isolation_level=None); c.row_factory=sqlite3.Row
c.execute("PRAGMA busy_timeout=10000"); return c
def _now(self): return self.clock().astimezone(timezone.utc).isoformat().replace("+00:00","Z")
def _init(self):
with self._db() as c:
c.execute("DROP TRIGGER IF EXISTS aios_r7_recovery_capability_projection")
c.executescript("""
CREATE TABLE IF NOT EXISTS deterministic_execution_amendments(
amendment_id TEXT PRIMARY KEY,nonce TEXT UNIQUE NOT NULL,task_id TEXT NOT NULL,
decision_id TEXT NOT NULL,plan_id TEXT NOT NULL,plan_sha256 TEXT NOT NULL,
envelope_digest TEXT NOT NULL,envelope_json TEXT NOT NULL,payload_json TEXT NOT NULL,
created_at TEXT NOT NULL,UNIQUE(task_id,decision_id));
DROP TRIGGER IF EXISTS aios_r7_recovery_requires_executable_amendment;
CREATE TRIGGER aios_r7_recovery_requires_executable_amendment
BEFORE UPDATE OF status ON tasks
WHEN OLD.status='FAILED' AND NEW.status='QUEUED' BEGIN
SELECT CASE WHEN NOT EXISTS(
SELECT 1 FROM deterministic_execution_amendments a
WHERE a.task_id=OLD.task_id
AND a.decision_id=json_extract(NEW.metadata_json,'$.recovery.decision_id')
AND a.amendment_id=json_extract(OLD.metadata_json,'$.recovery.execution_amendment_id')
AND a.envelope_digest=json_extract(OLD.metadata_json,'$.execution.execution_digest')
) THEN RAISE(ABORT,'SIGNED_EXECUTION_AMENDMENT_REQUIRED') END;
END;""")
def _plan(self,c,x):
r=c.execute("SELECT * FROM deterministic_plans WHERE plan_id=?",(x["plan_id"],)).fetchone()
if r is None or r["plan_sha256"]!=x["plan_sha256"]:_err("AMENDMENT_PLAN_NOT_FOUND_OR_MISMATCH")
p=DeterministicPlan.from_dict(json.loads(r["payload_json"]))
if (p.task_id,p.job_id,p.generation,p.repository,p.branch,p.expected_branch_oid,p.recovery_epoch)!=(x["task_id"],x["job_id"],x["generation"],x["repository"],x["branch"],x["expected_branch_oid"],x["recovery_epoch"]):_err("AMENDMENT_PLAN_BINDING_MISMATCH")
return p
def persist(self,raw):
x=parse_amendment(raw)
if x["amendment_id"]!=amendment_id(x):_err("AMENDMENT_ID_DIGEST_MISMATCH")
now=self._now()
if now<x["not_before"] or now>=x["expires_at"]:_err("AMENDMENT_EXPIRED_OR_NOT_YET_VALID")
if not self.verifier.verify(namespace=NAMESPACE,payload=signed_payload(x),signature=x["signature"],key_id=x["key_id"]):_err("AMENDMENT_SIGNATURE_INVALID")
with self._db() as c:
c.execute("BEGIN IMMEDIATE"); e=_validate_envelope(x["envelope"],self._plan(c,x),x)
ej=canonical_json(e); ed=e["execution_digest"]; stored=dict(x); stored["signature"]="<present>"
old=c.execute("SELECT * FROM deterministic_execution_amendments WHERE amendment_id=?",(x["amendment_id"],)).fetchone()
if old:
if old["envelope_json"]!=ej:_err("AMENDMENT_ID_REUSE_MISMATCH")
c.commit(); return {"status":"PRESENT","amendment_id":x["amendment_id"],"task_id":x["task_id"],"decision_id":x["decision_id"],"plan_id":x["plan_id"],"execution_digest":ed}
if c.execute("SELECT 1 FROM deterministic_execution_amendments WHERE nonce=? OR (task_id=? AND decision_id=?)",(x["nonce"],x["task_id"],x["decision_id"])).fetchone():_err("AMENDMENT_REPLAY_OR_DECISION_REUSE")
c.execute("INSERT INTO deterministic_execution_amendments VALUES(?,?,?,?,?,?,?,?,?,?)",(x["amendment_id"],x["nonce"],x["task_id"],x["decision_id"],x["plan_id"],x["plan_sha256"],ed,ej,canonical_json(stored),now)); c.commit()
return {"status":"PERSISTED","amendment_id":x["amendment_id"],"task_id":x["task_id"],"decision_id":x["decision_id"],"plan_id":x["plan_id"],"execution_digest":ed}
def apply(self,raw):
if raw.get("schema")!=APPLY_SCHEMA:_err("AMENDMENT_APPLY_SCHEMA_INVALID")
aid=raw.get("amendment_id"); tid=raw.get("task_id")
with self._db() as c:
c.execute("BEGIN IMMEDIATE"); a=c.execute("SELECT * FROM deterministic_execution_amendments WHERE amendment_id=? AND task_id=?",(aid,tid)).fetchone()
if a is None:_err("AMENDMENT_NOT_FOUND")
x=json.loads(a["payload_json"]); d=json.loads(a["envelope_json"])
t=c.execute("SELECT * FROM tasks WHERE task_id=?",(tid,)).fetchone()
if t is None:_err("TASK_NOT_FOUND")
m=json.loads(t["metadata_json"] or "{}"); cp=tuple(sorted(json.loads(t["required_capabilities_json"] or "[]"))); dc=tuple(sorted(d.get("capability_scope") or ()))
rec=m.get("recovery"); rec=dict(rec) if isinstance(rec,Mapping) else {}
old=m.get("execution")
if isinstance(old,Mapping) and old.get("execution_digest")==a["envelope_digest"] and cp==dc and rec.get("execution_amendment_id")==aid:
c.commit(); return {"status":"PRESENT","task_id":tid,"amendment_id":aid,"execution_digest":a["envelope_digest"],"revision":int(t["revision"]),"attempt":int(t["attempt"])}
if t["status"]!=x["expected_apply_status"] or int(t["revision"])!=x["expected_apply_revision"] or int(t["attempt"])!=x["expected_apply_attempt"] or cp!=tuple(x["expected_current_capabilities"]) or t["claimed_by"] is not None or t["lease_token_hash"] is not None:_err("AMENDMENT_APPLY_TASK_CAS_MISMATCH")
if t["status"]=="QUEUED" and rec.get("decision_id")!=x["decision_id"]:_err("AMENDMENT_DECISION_BINDING_MISMATCH")
m["execution"]=d; m["execution_projection"]={"schema":"aios.online-agent.task-projection.v1","task_id":tid,"job_id":d.get("job_id"),"execution_digest":a["envelope_digest"]}
rec["execution_amendment_id"]=aid; m["recovery"]=rec; now=self._now()
q=c.execute("""UPDATE tasks SET required_capabilities_json=?,metadata_json=?,revision=revision+1,updated_at=? WHERE task_id=? AND status=? AND revision=? AND attempt=? AND claimed_by IS NULL AND lease_token_hash IS NULL""",(canonical_json(list(dc)),canonical_json(m),now,tid,x["expected_apply_status"],x["expected_apply_revision"],x["expected_apply_attempt"]))
if q.rowcount!=1:_err("AMENDMENT_APPLY_TASK_CAS_FAILED")
c.execute("INSERT INTO events(task_id,agent_id,event_type,payload_json,created_at) VALUES(?,?,?,?,?)",(tid,None,EVENT,canonical_json({"amendment_id":aid,"decision_id":x["decision_id"],"plan_id":x["plan_id"],"execution_digest":a["envelope_digest"],"required_capabilities":list(dc)}),now)); c.commit()
return {"status":"APPLIED","task_id":tid,"amendment_id":aid,"decision_id":x["decision_id"],"plan_id":x["plan_id"],"execution_digest":a["envelope_digest"],"required_capabilities":list(dc),"revision":x["expected_apply_revision"]+1,"attempt":x["expected_apply_attempt"]}
@@ -0,0 +1,798 @@
#!/usr/bin/env python3
"""Fail-closed AF_UNIX builder for server-owned trusted candidate evidence."""
from __future__ import annotations
import argparse
import base64
import hashlib
import hmac
import json
import os
import secrets
import socket
import stat
import struct
import subprocess
import time
from datetime import datetime, timezone
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Callable, Mapping, Protocol
class BuilderError(RuntimeError):
"""A stable, non-sensitive error code returned by the builder boundary."""
def _canonical_json(value: Mapping[str, Any]) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=True).encode("utf-8")
def _sha256(*parts: str) -> str:
return hashlib.sha256("\0".join(parts).encode("utf-8")).hexdigest()
@dataclass(frozen=True)
class PeerIdentity:
pid: int
uid: int
gid: int
class DomainPurposeSigner(Protocol):
"""A signer deliberately limited to the builder's one domain purpose."""
@property
def key_id(self) -> str:
...
@property
def key_generation(self) -> int:
...
def sign_for_purpose(self, purpose: str, payload: bytes) -> bytes:
...
class CredentialPathEd25519Signer:
"""Reads a systemd-mounted credential path; never exports or logs key bytes."""
def __init__(self, credential_path: Path, *, key_id: str, key_generation: int, allowed_purpose: str):
self._credential_path = Path(credential_path)
self._key_id = str(key_id)
self._key_generation = key_generation
self._allowed_purpose = str(allowed_purpose)
if not self._key_id or not isinstance(key_generation, int) or key_generation < 1:
raise BuilderError("SIGNING_IDENTITY_INVALID")
if not self._credential_path.is_file():
raise BuilderError("SIGNING_CREDENTIAL_UNAVAILABLE")
try:
from cryptography.hazmat.primitives.serialization import load_pem_private_key
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
self._private_key = load_pem_private_key(self._credential_path.read_bytes(), password=None)
if not isinstance(self._private_key, Ed25519PrivateKey):
raise TypeError("signing key is not Ed25519")
except Exception as exc: # do not expose parse/key material details
raise BuilderError("SIGNING_CREDENTIAL_INVALID") from exc
@property
def key_id(self) -> str:
return self._key_id
@property
def key_generation(self) -> int:
return self._key_generation
def sign_for_purpose(self, purpose: str, payload: bytes) -> bytes:
if purpose != self._allowed_purpose:
raise BuilderError("SIGNING_PURPOSE_DENIED")
return self._private_key.sign(payload)
@dataclass(frozen=True)
class BuilderKeyring:
builder_hmac: bytes
signing_private_key_path: Path | None = None
def __post_init__(self) -> None:
if len(self.builder_hmac) < 32:
raise BuilderError("BUILDER_HMAC_INVALID")
@classmethod
def from_credentials_directory(cls, credentials_directory: Path) -> "BuilderKeyring":
root = Path(credentials_directory)
hmac_path = root / "builder_hmac"
signing_path = root / "signing_private_key"
try:
secret = hmac_path.read_bytes().strip()
except OSError as exc:
raise BuilderError("BUILDER_HMAC_UNAVAILABLE") from exc
if not signing_path.is_file():
raise BuilderError("SIGNING_CREDENTIAL_UNAVAILABLE")
return cls(builder_hmac=secret, signing_private_key_path=signing_path)
@dataclass(frozen=True)
class TrustedCandidateBuilderConfig:
policy_schema: str
repository: str
request_schema: str
response_schema: str
operation: str
readiness_operation: str
maximum_request_age_seconds: int
request_timeout_seconds: int
allowed_peer_uid: int
allowed_peer_cgroups: tuple[str, ...]
allowed_peer_executable_prefix: str
allowed_peer_runtime_prefix: str
allowed_peer_runtime_suffix: str
required_checks: tuple[str, ...]
evidence_layout: Mapping[str, str]
recipes: Mapping[str, Mapping[str, str]]
governed_recipes: Mapping[str, Mapping[str, str]]
policy_sha256: str
@classmethod
def from_policy(cls, policy: Mapping[str, Any]) -> "TrustedCandidateBuilderConfig":
required = {
"schema", "repository", "request_schema", "response_schema", "operation",
"readiness_operation", "maximum_request_age_seconds", "request_timeout_seconds",
"allowed_peer_uid", "allowed_peer_cgroups", "allowed_peer_executable_prefix",
"allowed_peer_runtime_prefix", "allowed_peer_runtime_suffix",
"required_checks", "evidence_layout", "recipes", "governed_recipes",
}
if set(policy) != required or policy.get("schema") != "aios.trusted-candidate-builder-policy.v1":
raise BuilderError("POLICY_SCHEMA_INVALID")
recipes = policy.get("recipes")
if not isinstance(recipes, Mapping) or set(recipes) != {"V2R2C2", "V2R2C3"}:
raise BuilderError("POLICY_RECIPES_INVALID")
c2, c3 = recipes["V2R2C2"], recipes["V2R2C3"]
if not isinstance(c2, Mapping) or set(c2) != {"mode", "task_id", "candidate_id", "base_oid", "source_commit", "version", "purpose"} or c2.get("mode") != "fixed":
raise BuilderError("POLICY_V2R2C2_INVALID")
if not isinstance(c3, Mapping) or set(c3) != {"mode", "derives_from", "version", "purpose"} or c3.get("mode") != "governed-derived-only" or c3.get("derives_from") != "V2R2C2":
raise BuilderError("POLICY_V2R2C3_INVALID")
if (
policy["repository"] != "aios/aios"
or policy["operation"] != "BuildTrustedCandidateV1"
or policy["readiness_operation"] != "ProbeTrustedCandidateBuilderV1"
):
raise BuilderError("POLICY_SCOPE_INVALID")
if not isinstance(policy["maximum_request_age_seconds"], int) or not 1 <= policy["maximum_request_age_seconds"] <= 300:
raise BuilderError("POLICY_FRESHNESS_INVALID")
if not isinstance(policy["request_timeout_seconds"], int) or not 1 <= policy["request_timeout_seconds"] <= 120:
raise BuilderError("POLICY_TIMEOUT_INVALID")
if not isinstance(policy["allowed_peer_uid"], int) or policy["allowed_peer_uid"] != 1000:
raise BuilderError("POLICY_PEER_UID_INVALID")
cgroups, checks = policy["allowed_peer_cgroups"], policy["required_checks"]
if not isinstance(cgroups, list) or not cgroups or not all(isinstance(value, str) and value.startswith("/") for value in cgroups):
raise BuilderError("POLICY_CGROUPS_INVALID")
for field in (
"allowed_peer_executable_prefix",
"allowed_peer_runtime_prefix",
"allowed_peer_runtime_suffix",
):
if not isinstance(policy[field], str) or not policy[field].startswith("/"):
raise BuilderError("POLICY_PROCESS_IDENTITY_INVALID")
if not isinstance(checks, list) or not checks or len(set(checks)) != len(checks) or not all(isinstance(value, str) for value in checks):
raise BuilderError("POLICY_CHECKS_INVALID")
layout = policy["evidence_layout"]
evidence_names = {
"artifact", "manifest", "package_closure", "tests", "canary",
"gitea", "rollback", "policy", "secret_scan",
}
if (
not isinstance(layout, Mapping)
or set(layout) != evidence_names
or not all(
isinstance(value, str)
and value.startswith("{version}/")
and ".." not in Path(value).parts
and not Path(value).is_absolute()
for value in layout.values()
)
):
raise BuilderError("POLICY_EVIDENCE_LAYOUT_INVALID")
return cls(
policy_schema=str(policy["schema"]), repository=str(policy["repository"]),
request_schema=str(policy["request_schema"]), response_schema=str(policy["response_schema"]),
operation=str(policy["operation"]), readiness_operation=str(policy["readiness_operation"]),
maximum_request_age_seconds=policy["maximum_request_age_seconds"],
request_timeout_seconds=policy["request_timeout_seconds"],
allowed_peer_uid=policy["allowed_peer_uid"], allowed_peer_cgroups=tuple(cgroups),
allowed_peer_executable_prefix=policy["allowed_peer_executable_prefix"],
allowed_peer_runtime_prefix=policy["allowed_peer_runtime_prefix"],
allowed_peer_runtime_suffix=policy["allowed_peer_runtime_suffix"],
required_checks=tuple(checks), evidence_layout=dict(layout),
recipes={name: dict(value) for name, value in recipes.items()},
governed_recipes={name: dict(value) for name, value in policy["governed_recipes"].items()} if isinstance(policy["governed_recipes"], Mapping) else _invalid_governed_recipes(),
policy_sha256=hashlib.sha256(_canonical_json(dict(policy))).hexdigest(),
)
def fixed_recipe(self, name: str) -> Mapping[str, str]:
recipe = self.recipes.get(name)
if recipe is None:
raise BuilderError("RECIPE_UNKNOWN")
if recipe["mode"] == "governed-derived-only":
derived = self.governed_recipes.get(name)
required = {"mode", "task_id", "candidate_id", "base_oid", "source_commit", "version", "purpose", "derived_from_v2r2c2_attestation", "builder_engineering_oid"}
if not isinstance(derived, Mapping) or set(derived) != required or derived.get("mode") != "governed-derived" or derived.get("version") != name or derived.get("purpose") != recipe.get("purpose"):
raise BuilderError("GOVERNED_V2R2C3_DERIVATION_REQUIRED")
for name in ("base_oid", "source_commit", "builder_engineering_oid"):
value = derived.get(name)
if not isinstance(value, str) or len(value) != 40 or any(char not in "0123456789abcdef" for char in value.lower()):
raise BuilderError("GOVERNED_V2R2C3_DERIVATION_REQUIRED")
return dict(derived)
return recipe
def resolve_recipe(self, version: str, candidate_id: str | None, source_commit: str) -> Mapping[str, str]:
recipe = self.fixed_recipe(version)
# Intent-only callers do not choose a candidate identity. The
# policy-owned recipe supplies it; explicit worker requests are
# still checked against the same immutable recipe.
effective_candidate_id = str(candidate_id or recipe.get("candidate_id") or "")
if {"version": recipe.get("version"), "candidate_id": recipe.get("candidate_id"), "source_commit": recipe.get("source_commit")} != {"version": version, "candidate_id": effective_candidate_id, "source_commit": source_commit}:
raise BuilderError("REQUEST_RELEASE_BINDING_INVALID")
return recipe
def _invalid_governed_recipes() -> Mapping[str, Mapping[str, str]]:
raise BuilderError("POLICY_GOVERNED_RECIPES_INVALID")
def load_builder_policy(path: Path) -> Mapping[str, Any]:
try:
decoded = json.loads(Path(path).read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
raise BuilderError("POLICY_UNAVAILABLE") from exc
if not isinstance(decoded, Mapping):
raise BuilderError("POLICY_SCHEMA_INVALID")
TrustedCandidateBuilderConfig.from_policy(decoded)
return decoded
class ServerOwnedEvidenceSource:
"""Recomputes a fixed on-host evidence closure without caller-owned paths."""
def __init__(
self,
config: TrustedCandidateBuilderConfig,
authority_git_dir: Path,
evidence_root: Path,
*,
expected_owner_uid: int | None = 0,
git_runner: Callable[..., Any] = subprocess.run,
):
self._config = config
self._authority_git_dir = Path(authority_git_dir)
self._evidence_root = Path(evidence_root)
self._expected_owner_uid = expected_owner_uid
self._git_runner = git_runner
def _regular_path(self, relative: str) -> Path:
root = self._evidence_root.resolve(strict=True)
candidate = self._evidence_root / relative
current = self._evidence_root
for part in Path(relative).parts:
current = current / part
if current.is_symlink():
raise BuilderError("COLLECTOR_SYMLINK_DENIED")
try:
resolved = candidate.resolve(strict=True)
resolved.relative_to(root)
metadata = resolved.stat(follow_symlinks=False)
except (OSError, ValueError) as exc:
raise BuilderError("COLLECTOR_EVIDENCE_UNAVAILABLE") from exc
if not stat.S_ISREG(metadata.st_mode):
raise BuilderError("COLLECTOR_SPECIAL_FILE_DENIED")
if os.name != "nt" and metadata.st_mode & 0o022:
raise BuilderError("COLLECTOR_EVIDENCE_MODE_DENIED")
if (
os.name != "nt"
and
self._expected_owner_uid is not None
and metadata.st_uid != self._expected_owner_uid
):
raise BuilderError("COLLECTOR_EVIDENCE_OWNER_DENIED")
return resolved
def _digest(self, path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def _json(self, path: Path) -> Mapping[str, Any]:
try:
if path.stat().st_size > 2 * 1024 * 1024:
raise BuilderError("COLLECTOR_EVIDENCE_SIZE_INVALID")
value = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise BuilderError("COLLECTOR_EVIDENCE_INVALID") from exc
if not isinstance(value, Mapping):
raise BuilderError("COLLECTOR_EVIDENCE_INVALID")
return value
def __call__(self, recipe_name: str, recipe: Mapping[str, str]) -> Mapping[str, Any]:
try:
tree_result = self._git_runner(
["git", f"--git-dir={self._authority_git_dir}", "rev-parse", f"{recipe['source_commit']}^{{tree}}"],
check=True, capture_output=True, text=True, timeout=15,
env={"PATH": os.environ.get("PATH", "")},
)
ancestry_result = self._git_runner(
["git", f"--git-dir={self._authority_git_dir}", "merge-base", "--is-ancestor", recipe["base_oid"], recipe["source_commit"]],
check=False, capture_output=True, text=True, timeout=15,
env={"PATH": os.environ.get("PATH", "")},
)
tree_oid = tree_result.stdout.strip().lower()
except (OSError, subprocess.SubprocessError) as exc:
raise BuilderError("COLLECTOR_EVIDENCE_UNAVAILABLE") from exc
if getattr(ancestry_result, "returncode", 1) != 0:
raise BuilderError("COLLECTOR_GIT_MISMATCH")
receipt = self._json(self._regular_path(f"{recipe_name}/receipt.json"))
if set(receipt) != {"schema", "version", "candidate", "release_plan"}:
raise BuilderError("COLLECTOR_EVIDENCE_INVALID")
if (
receipt.get("schema") != "aios.trusted-candidate-builder.evidence.v1"
or receipt.get("version") != recipe_name
):
raise BuilderError("COLLECTOR_EVIDENCE_INVALID")
candidate = receipt.get("candidate")
release_plan = receipt.get("release_plan")
if not isinstance(candidate, Mapping) or not isinstance(release_plan, Mapping):
raise BuilderError("COLLECTOR_EVIDENCE_INVALID")
if (
candidate.get("candidate_id") != recipe["candidate_id"]
or candidate.get("source_commit") != recipe["source_commit"]
or candidate.get("tree_oid") != tree_oid
):
raise BuilderError("COLLECTOR_GIT_MISMATCH")
git_consistency = candidate.get("git_consistency")
if (
not isinstance(git_consistency, Mapping)
or git_consistency.get("consistent") is not True
or git_consistency.get("source_commit") != recipe["source_commit"]
or git_consistency.get("tree_oid") != tree_oid
or git_consistency.get("repository") != self._config.repository
):
raise BuilderError("COLLECTOR_GIT_MISMATCH")
paths = {
name: self._regular_path(template.format(version=recipe_name))
for name, template in self._config.evidence_layout.items()
}
deployment = release_plan.get("deployment")
if not isinstance(deployment, Mapping):
raise BuilderError("COLLECTOR_EVIDENCE_INVALID")
digests = {name: self._digest(path) for name, path in paths.items()}
if (
digests["artifact"] != candidate.get("artifact_sha256")
or digests["manifest"] != candidate.get("manifest_sha256")
or digests["package_closure"] != deployment.get("package_closure_sha256")
or digests["policy"] != release_plan.get("policy_sha256")
or digests["policy"] != deployment.get("policy_sha256")
):
raise BuilderError("COLLECTOR_DIGEST_MISMATCH")
tests = self._json(paths["tests"])
canary = self._json(paths["canary"])
gitea = self._json(paths["gitea"])
rollback = self._json(paths["rollback"])
secret_scan = self._json(paths["secret_scan"])
if tests != candidate.get("tests") or tests.get("passed") is not True:
raise BuilderError("COLLECTOR_TESTS_MISMATCH")
if canary != candidate.get("canary") or canary.get("passed") is not True:
raise BuilderError("COLLECTOR_CANARY_MISMATCH")
if (
digests["tests"] != hashlib.sha256(_canonical_json(dict(tests))).hexdigest()
or digests["canary"] != hashlib.sha256(_canonical_json(dict(canary))).hexdigest()
):
raise BuilderError("COLLECTOR_EVIDENCE_CANONICALIZATION_MISMATCH")
if (
gitea != candidate.get("gitea_consistency")
or gitea.get("consistent") is not True
or gitea.get("repository") != self._config.repository
or gitea.get("source_commit") != recipe["source_commit"]
or gitea.get("tree_oid") != tree_oid
or gitea.get("remote_ref_oid") != recipe["source_commit"]
):
raise BuilderError("COLLECTOR_GITEA_MISMATCH")
if (
candidate.get("rollback_ready") is not True
or rollback.get("ready") is not True
or rollback.get("source_commit") != recipe["source_commit"]
or rollback.get("rollback_sha256") != deployment.get("rollback_sha256")
):
raise BuilderError("COLLECTOR_ROLLBACK_MISMATCH")
if (
digests["gitea"] != hashlib.sha256(_canonical_json(dict(gitea))).hexdigest()
or digests["rollback"] != hashlib.sha256(_canonical_json(dict(rollback))).hexdigest()
):
raise BuilderError("COLLECTOR_EVIDENCE_CANONICALIZATION_MISMATCH")
if (
secret_scan.get("passed") is not True
or secret_scan.get("artifact_sha256") != candidate.get("artifact_sha256")
):
raise BuilderError("COLLECTOR_SECRET_SCAN_MISMATCH")
checks = {name: True for name in self._config.required_checks}
return {
"candidate": dict(candidate),
"release_plan": dict(release_plan),
"checks": checks,
"evidence_digests": digests,
}
class DeterministicEvidenceCollector:
"""Validates a server-owned source; request fields never influence collection."""
_BUNDLE_FIELDS = {"candidate", "release_plan", "checks", "evidence_digests"}
def __init__(self, config: TrustedCandidateBuilderConfig, *, evidence_source: Callable[[str, Mapping[str, str]], Mapping[str, Any]] | None = None):
self._config = config
self._evidence_source = evidence_source
def collect(self, recipe_name: str, recipe: Mapping[str, str]) -> Mapping[str, Any]:
if self._evidence_source is None:
raise BuilderError("COLLECTOR_UNCONFIGURED")
bundle = self._evidence_source(recipe_name, recipe)
if not isinstance(bundle, Mapping) or set(bundle) != self._BUNDLE_FIELDS:
raise BuilderError("COLLECTOR_EVIDENCE_INVALID")
raw_checks = bundle.get("checks")
if (
not isinstance(raw_checks, Mapping)
or set(raw_checks) != set(self._config.required_checks)
or any(raw_checks.get(name) is not True for name in self._config.required_checks)
):
raise BuilderError("COLLECTOR_CHECKS_INCOMPLETE")
evidence_digests = bundle.get("evidence_digests")
if (
not isinstance(evidence_digests, Mapping)
or set(evidence_digests) != set(self._config.evidence_layout)
or any(
not isinstance(value, str)
or len(value) != 64
or any(char not in "0123456789abcdef" for char in value.lower())
for value in evidence_digests.values()
)
):
raise BuilderError("COLLECTOR_EVIDENCE_DIGESTS_INVALID")
try:
from silent_release_controller import ReleaseCandidate
from silent_release_hardening import ReleasePlan, TrustedCandidateBundle
candidate = ReleaseCandidate.from_dict(dict(bundle["candidate"]))
release_plan = ReleasePlan.from_dict(dict(bundle["release_plan"]))
checks = dict(raw_checks)
TrustedCandidateBundle(candidate, release_plan, checks).validate()
except (ImportError, KeyError, TypeError, ValueError, RuntimeError) as exc:
raise BuilderError("COLLECTOR_EVIDENCE_INVALID") from exc
if candidate.source_commit != recipe["source_commit"] or candidate.candidate_id != recipe["candidate_id"]:
raise BuilderError("COLLECTOR_SOURCE_MISMATCH")
return {
"candidate": candidate.to_dict(),
"release_plan": release_plan.to_dict(),
"checks": checks,
"evidence_digests": dict(evidence_digests),
}
def _extract_peer_identity(connection: socket.socket) -> PeerIdentity:
if not hasattr(socket, "SO_PEERCRED"):
raise BuilderError("SO_PEERCRED_UNAVAILABLE")
raw = connection.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, struct.calcsize("3i"))
if len(raw) != struct.calcsize("3i"):
raise BuilderError("SO_PEERCRED_INVALID")
return PeerIdentity(*struct.unpack("3i", raw))
def _resolve_cgroups(pid: int) -> tuple[str, ...]:
try:
return tuple(
line.rsplit(":", 1)[1].strip()
for line in Path(f"/proc/{pid}/cgroup").read_text(encoding="utf-8").splitlines()
if ":" in line and line.rsplit(":", 1)[1].strip().startswith("/")
)
except OSError as exc:
raise BuilderError("PEER_CGROUP_UNAVAILABLE") from exc
def _resolve_process_identity(pid: int) -> tuple[str, tuple[str, ...]]:
try:
executable = os.path.realpath(f"/proc/{pid}/exe")
argv = tuple(
item.decode("utf-8", "strict")
for item in Path(f"/proc/{pid}/cmdline").read_bytes().split(b"\0")
if item
)
except (OSError, UnicodeDecodeError) as exc:
raise BuilderError("PEER_PROCESS_UNAVAILABLE") from exc
if not executable or not argv:
raise BuilderError("PEER_PROCESS_UNAVAILABLE")
return executable, argv
class TrustedCandidateBuilder:
def __init__(
self, config: TrustedCandidateBuilderConfig, keyring: BuilderKeyring, *,
collector: DeterministicEvidenceCollector, signer: DomainPurposeSigner,
peer_identity_resolver: Callable[[socket.socket], PeerIdentity] = _extract_peer_identity,
cgroup_resolver: Callable[[int], tuple[str, ...] | str] = _resolve_cgroups,
process_identity_resolver: Callable[[int], tuple[str, tuple[str, ...]]] = _resolve_process_identity,
clock: Callable[[], float] = time.time, nonce_ledger_path: Path | None = None,
) -> None:
self.config, self.keyring, self.collector, self.signer = config, keyring, collector, signer
self.peer_identity_resolver, self.cgroup_resolver, self.clock = peer_identity_resolver, cgroup_resolver, clock
self.process_identity_resolver = process_identity_resolver
self._nonce_ledger_path = Path(nonce_ledger_path) if nonce_ledger_path else None
self._nonce_ledger: dict[str, tuple[bytes, dict[str, Any]]] = self._load_nonce_ledger()
def _load_nonce_ledger(self) -> dict[str, tuple[bytes, dict[str, Any]]]:
if self._nonce_ledger_path is None or not self._nonce_ledger_path.exists():
return {}
if self._nonce_ledger_path.is_symlink():
raise BuilderError("NONCE_LEDGER_INVALID")
try:
raw = json.loads(self._nonce_ledger_path.read_text(encoding="utf-8"))
return {nonce: (bytes.fromhex(value["request_sha256"]), dict(value["result"])) for nonce, value in raw.items()}
except (OSError, ValueError, TypeError, KeyError, json.JSONDecodeError) as exc:
raise BuilderError("NONCE_LEDGER_INVALID") from exc
def _persist_nonce_ledger(self) -> None:
if self._nonce_ledger_path is None:
return
try:
self._nonce_ledger_path.parent.mkdir(mode=0o700, parents=True, exist_ok=True)
temporary = self._nonce_ledger_path.with_suffix(".tmp")
payload = {nonce: {"request_sha256": digest.hex(), "result": result} for nonce, (digest, result) in self._nonce_ledger.items()}
with temporary.open("wb") as handle:
handle.write(_canonical_json(payload))
handle.flush()
os.fsync(handle.fileno())
os.chmod(temporary, 0o600)
os.replace(temporary, self._nonce_ledger_path)
if os.name != "nt":
descriptor = os.open(self._nonce_ledger_path.parent, os.O_RDONLY)
try:
os.fsync(descriptor)
finally:
os.close(descriptor)
except OSError as exc:
raise BuilderError("NONCE_LEDGER_PERSIST_FAILED") from exc
def _verify_peer(self, connection: socket.socket) -> None:
peer = self.peer_identity_resolver(connection)
if peer.uid != self.config.allowed_peer_uid or peer.pid <= 0:
raise BuilderError("PEER_IDENTITY_DENIED")
resolved = self.cgroup_resolver(peer.pid)
cgroups = (resolved,) if isinstance(resolved, str) else tuple(resolved)
if not cgroups or not any(item in self.config.allowed_peer_cgroups for item in cgroups):
raise BuilderError("PEER_CGROUP_DENIED")
executable, argv = self.process_identity_resolver(peer.pid)
if not executable.startswith(self.config.allowed_peer_executable_prefix):
raise BuilderError("PEER_PROCESS_DENIED")
runtime_arguments = [
value
for value in argv
if value.startswith(self.config.allowed_peer_runtime_prefix)
and value.endswith(self.config.allowed_peer_runtime_suffix)
]
if len(runtime_arguments) != 1:
raise BuilderError("PEER_PROCESS_DENIED")
def handle_request(self, request: Mapping[str, Any], *, connection: socket.socket) -> dict[str, Any]:
if not isinstance(request, Mapping):
raise BuilderError("REQUEST_SCHEMA_INVALID")
expected_keys = {"schema", "operation", "recipe", "repository", "task_id", "candidate_id", "base_oid", "source_commit", "version", "purpose", "issued_at", "nonce", "hmac"}
if (
set(request) != expected_keys
or request.get("schema") != self.config.request_schema
or request.get("operation") not in {
self.config.operation,
self.config.readiness_operation,
}
):
raise BuilderError("REQUEST_SCHEMA_INVALID")
supplied_mac = request.get("hmac")
unsigned = {key: value for key, value in request.items() if key != "hmac"}
if not isinstance(supplied_mac, str) or not hmac.compare_digest(hmac.new(self.keyring.builder_hmac, _canonical_json(unsigned), hashlib.sha256).hexdigest(), supplied_mac):
raise BuilderError("REQUEST_HMAC_INVALID")
issued_at, nonce = request.get("issued_at"), request.get("nonce")
if not isinstance(issued_at, int) or not isinstance(nonce, str) or len(nonce) < 16 or len(nonce) > 256:
raise BuilderError("REQUEST_SCHEMA_INVALID")
self._verify_peer(connection)
request_bytes = _canonical_json(dict(request))
replay = self._nonce_ledger.get(nonce)
if replay is not None:
if hmac.compare_digest(replay[0], hashlib.sha256(request_bytes).digest()):
return dict(replay[1])
raise BuilderError("REPLAY_CONFLICT")
if abs(int(self.clock()) - issued_at) > self.config.maximum_request_age_seconds:
raise BuilderError("REQUEST_STALE")
recipe_name = request.get("recipe")
if not isinstance(recipe_name, str):
raise BuilderError("REQUEST_SCHEMA_INVALID")
recipe = self.config.resolve_recipe(str(request.get("version") or ""), str(request.get("candidate_id") or ""), str(request.get("source_commit") or ""))
if recipe_name != recipe.get("version"):
raise BuilderError("REQUEST_RELEASE_BINDING_INVALID")
bound = {"repository": self.config.repository, "task_id": recipe["task_id"], "candidate_id": recipe["candidate_id"], "base_oid": recipe["base_oid"], "source_commit": recipe["source_commit"], "version": recipe["version"], "purpose": recipe["purpose"]}
if {key: request.get(key) for key in bound} != bound:
raise BuilderError("REQUEST_SCOPE_INVALID")
bundle = dict(self.collector.collect(recipe_name, recipe))
candidate = bundle["candidate"]
release_plan = bundle["release_plan"]
checks = bundle.get("checks")
evidence_digests = bundle["evidence_digests"]
if not isinstance(checks, Mapping) or set(checks) != set(self.config.required_checks) or not all(value is True for value in checks.values()):
raise BuilderError("COLLECTOR_CHECKS_INCOMPLETE")
if request["operation"] == self.config.readiness_operation:
result = {
"schema": self.config.response_schema,
"status": "READY",
"repository": self.config.repository,
"recipe": recipe_name,
"candidate_id": recipe["candidate_id"],
"source_commit": recipe["source_commit"],
"builder_key_id": self.signer.key_id,
"builder_key_generation": self.signer.key_generation,
"signature_algorithm": "Ed25519",
"signature_purpose": "AIOS_TRUSTED_RELEASE_BUILDER",
"policy_sha256": self.config.policy_sha256,
"collector_ready": True,
"signer_ready": True,
}
if len(self._nonce_ledger) >= 2048:
self._nonce_ledger.pop(next(iter(self._nonce_ledger)))
self._nonce_ledger[nonce] = (
hashlib.sha256(request_bytes).digest(),
dict(result),
)
self._persist_nonce_ledger()
return result
issued = datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
attestation = {
"builder_attestation_id": "builder-attestation-" + _sha256(recipe_name, recipe["task_id"], candidate["artifact_sha256"])[:24],
"builder_key_id": self.signer.key_id,
"builder_key_generation": self.signer.key_generation,
"signature_algorithm": "Ed25519",
"signature_purpose": "AIOS_TRUSTED_RELEASE_BUILDER",
"source_commit": candidate["source_commit"], "tree_oid": candidate["tree_oid"],
"artifact_sha256": candidate["artifact_sha256"], "manifest_sha256": candidate["manifest_sha256"],
"package_closure_sha256": release_plan["deployment"]["package_closure_sha256"],
"test_evidence_sha256": evidence_digests["tests"],
"canary_evidence_sha256": evidence_digests["canary"],
"gitea_evidence_sha256": evidence_digests["gitea"],
"rollback_evidence_sha256": evidence_digests["rollback"],
"policy_evidence_sha256": evidence_digests["policy"],
"secret_scan_evidence_sha256": evidence_digests["secret_scan"],
"issued_at": issued, "trust_domain": "aios.trusted-builder.attestation.v1",
}
canonical_attestation = _canonical_json(attestation)
fingerprint = hashlib.sha256(canonical_attestation).hexdigest()
signature = base64.b64encode(self.signer.sign_for_purpose("AIOS_TRUSTED_RELEASE_BUILDER", canonical_attestation)).decode("ascii")
result = {
"schema": self.config.response_schema, "status": "BUILT", "recipe": recipe_name,
"repository": self.config.repository, "task_id": recipe["task_id"], "base_oid": recipe["base_oid"],
"purpose": recipe["purpose"], "checks": dict(checks),
"evidence": {"candidate_id": candidate["candidate_id"], "source_commit": candidate["source_commit"], "tree_oid": candidate["tree_oid"], "artifact_sha256": candidate["artifact_sha256"], "manifest_sha256": candidate["manifest_sha256"], "package_closure_sha256": release_plan["deployment"]["package_closure_sha256"], "test_evidence_sha256": attestation["test_evidence_sha256"], "canary_evidence_sha256": attestation["canary_evidence_sha256"]},
"attestation": {**attestation, "fingerprint": fingerprint, "signature": signature},
}
result["silent_release_bundle"] = {"candidate": candidate, "release_plan": release_plan, "checks": dict(checks), "attestation": result["attestation"]}
if len(self._nonce_ledger) >= 2048:
self._nonce_ledger.pop(next(iter(self._nonce_ledger)))
self._nonce_ledger[nonce] = (hashlib.sha256(request_bytes).digest(), dict(result))
self._persist_nonce_ledger()
return result
def serve_forever(self, listener: socket.socket) -> None:
while True:
connection, _ = listener.accept()
with connection:
try:
raw = connection.makefile("rb").readline(65537)
if not raw or len(raw) > 65536:
raise BuilderError("REQUEST_SIZE_INVALID")
result = self.handle_request(json.loads(raw), connection=connection)
except (BuilderError, UnicodeDecodeError, json.JSONDecodeError):
result = {"schema": self.config.response_schema, "status": "REJECTED", "error_code": "REQUEST_REJECTED"}
connection.sendall(_canonical_json(result) + b"\n")
class UnixTrustedCandidateBuilderClient:
def __init__(self, config: TrustedCandidateBuilderConfig, keyring: BuilderKeyring, *, socket_path: Path | None = None, clock: Callable[[], float] = time.time) -> None:
self.config, self.keyring, self.socket_path, self.clock = config, keyring, socket_path, clock
def create_request(self, version: str, candidate_id: str | None, source_commit: str, *, nonce: str, operation: str | None = None) -> dict[str, Any]:
selected = self.config.resolve_recipe(version, candidate_id, source_commit)
selected_operation = operation or self.config.operation
if selected_operation not in {self.config.operation, self.config.readiness_operation}:
raise BuilderError("REQUEST_OPERATION_INVALID")
request = {"schema": self.config.request_schema, "operation": selected_operation, "recipe": version, "repository": self.config.repository, "task_id": selected["task_id"], "candidate_id": selected["candidate_id"], "base_oid": selected["base_oid"], "source_commit": selected["source_commit"], "version": selected["version"], "purpose": selected["purpose"], "issued_at": int(self.clock()), "nonce": nonce}
request["hmac"] = hmac.new(self.keyring.builder_hmac, _canonical_json(request), hashlib.sha256).hexdigest()
return request
def _exchange(self, request: Mapping[str, Any]) -> Mapping[str, Any]:
if self.socket_path is None:
raise BuilderError("BUILDER_SOCKET_UNCONFIGURED")
try:
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as connection:
connection.settimeout(self.config.request_timeout_seconds)
connection.connect(os.fspath(self.socket_path))
connection.sendall(_canonical_json(dict(request)) + b"\n")
raw = connection.makefile("rb").readline(65537)
except (OSError, TimeoutError) as exc:
raise BuilderError("BUILDER_TRANSPORT_UNAVAILABLE") from exc
try:
result = json.loads(raw)
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise BuilderError("BUILDER_RESPONSE_INVALID") from exc
if not isinstance(result, Mapping) or result.get("schema") != self.config.response_schema:
raise BuilderError("BUILDER_RESPONSE_INVALID")
return result
def build(self, version: str, candidate_id: str | None, source_commit: str, *, nonce: str) -> Mapping[str, Any]:
result = self._exchange(
self.create_request(version, candidate_id, source_commit, nonce=nonce)
)
if result.get("status") != "BUILT":
raise BuilderError("BUILDER_REQUEST_REJECTED")
return result
def probe(self, *, nonce: str | None = None) -> Mapping[str, Any]:
recipe = self.config.fixed_recipe("V2R2C2")
result = self._exchange(
self.create_request(
"V2R2C2",
recipe["candidate_id"],
recipe["source_commit"],
nonce=nonce or secrets.token_hex(32),
operation=self.config.readiness_operation,
)
)
if result.get("status") != "READY":
raise BuilderError("BUILDER_NOT_READY")
return result
def readiness_probe(client: UnixTrustedCandidateBuilderClient) -> Mapping[str, Any]:
result = dict(client.probe())
required_true = {"collector_ready", "signer_ready"}
if any(result.get(name) is not True for name in required_true):
raise BuilderError("BUILDER_NOT_READY")
return result
def _main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--socket-fd", required=True, type=int)
parser.add_argument("--policy", required=True, type=Path)
arguments = parser.parse_args()
config = TrustedCandidateBuilderConfig.from_policy(load_builder_policy(arguments.policy))
credentials = os.environ.get("CREDENTIALS_DIRECTORY")
if not credentials:
raise SystemExit("CREDENTIALS_DIRECTORY is required")
keyring = BuilderKeyring.from_credentials_directory(Path(credentials))
signer = CredentialPathEd25519Signer(
keyring.signing_private_key_path,
key_id=os.environ["AIOS_TRUSTED_BUILDER_KEY_ID"],
key_generation=int(os.environ["AIOS_TRUSTED_BUILDER_KEY_GENERATION"]),
allowed_purpose="AIOS_TRUSTED_RELEASE_BUILDER",
)
listener = socket.fromfd(arguments.socket_fd, socket.AF_UNIX, socket.SOCK_STREAM)
collector = DeterministicEvidenceCollector(
config,
evidence_source=ServerOwnedEvidenceSource(
config,
Path(os.environ["AIOS_TRUSTED_BUILDER_GIT_DIR"]),
Path(os.environ["AIOS_TRUSTED_BUILDER_EVIDENCE_ROOT"]),
),
)
TrustedCandidateBuilder(config, keyring, collector=collector, signer=signer, nonce_ledger_path=Path(os.environ["AIOS_TRUSTED_BUILDER_NONCE_LEDGER"])).serve_forever(listener)
if __name__ == "__main__":
_main()
+56 -21
View File
@@ -31,6 +31,8 @@ import time
from pathlib import Path
from typing import Any, Mapping, Sequence
from trusted_release_execution_plan import PlanError, TrustedReleasePlan
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
PLAN_RE = re.compile(r"^sha256:[0-9a-f]{64}$")
@@ -73,6 +75,24 @@ class Binding:
production_port: int
repository_identity: str = "aios/aios"
candidate_id: str | None = None
release_plan_sha256: str | None = None
@classmethod
def from_trusted_plan(cls, plan: TrustedReleasePlan) -> "Binding":
value = plan.value
adapter = value["adapter_policy"]
binding = cls(
version=str(value["version"]), target_oid=str(value["source_commit"]),
evidence_closure_oid=str(value["evidence_closure_oid"]), artifact_sha256=str(value["artifact_sha256"]),
approval_plan_id=str(value["approval_plan_id"]), artifact_path=str(value["artifact"]["path"]),
artifact_size=int(value["artifact"]["size"]), production_service=str(adapter["service"]),
production_port=int(adapter["port"]), candidate_id=str(value["candidate_id"]),
release_plan_sha256=plan.release_plan_sha256,
)
plan.assert_binding(dataclasses.asdict(binding))
return binding
def __post_init__(self) -> None:
if not self.version or not OID_RE.fullmatch(self.target_oid):
raise ValueError("invalid release target binding")
@@ -107,6 +127,8 @@ class Binding:
"production_service": self.production_service,
"production_port": self.production_port,
"repository_identity": self.repository_identity,
"candidate_id": self.candidate_id,
"release_plan_sha256": self.release_plan_sha256,
}
return hashlib.sha256(canonical_json(payload)).hexdigest()
@@ -126,9 +148,12 @@ class Binding:
"production_service": self.production_service,
"production_port": self.production_port,
"repository_identity": self.repository_identity,
"candidate_id": self.candidate_id,
"release_plan_sha256": self.release_plan_sha256,
}
def canonical_json(value: object) -> bytes:
return (json.dumps(value, sort_keys=True, separators=(",", ":")) + "\n").encode("utf-8")
@@ -282,6 +307,12 @@ def verify_authorization(receipt: Mapping[str, Any], binding: Binding) -> dict[s
"artifact_sha256": binding.artifact_sha256,
"approval_plan_id": binding.approval_plan_id,
}
if binding.candidate_id is not None:
expected["candidate_id"] = binding.candidate_id
if binding.release_plan_sha256 is not None:
expected["release_plan_sha256"] = binding.release_plan_sha256
if receipt.get("candidate_binding_sha256") is None:
raise ManualOwnerReviewRequired("Owner authorization missing candidate binding")
for key, value in expected.items():
if receipt.get(key) != value:
raise ManualOwnerReviewRequired(f"Owner authorization mismatch: {key}")
@@ -498,7 +529,14 @@ def execute(
deploy_command: Sequence[str],
registry_command: Sequence[str],
baseline_command: Sequence[str],
release_plan_path: Path | None = None,
) -> dict[str, Any]:
plan = TrustedReleasePlan.load(release_plan_path) if release_plan_path is not None else None
if plan is not None:
try:
plan.assert_binding(dataclasses.asdict(binding))
except PlanError as exc:
raise ReleaseBlocked(f"trusted release plan mismatch: {exc}") from exc
authorization = verify_authorization(secure_json(authorization_path), binding)
lineage = verify_git_lineage(repository, binding)
artifact = verify_artifact(binding)
@@ -591,17 +629,18 @@ def execute(
def parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser()
p.add_argument("--authorization", required=True, type=Path)
p.add_argument("--release-plan", type=Path)
p.add_argument("--repository", required=True, type=Path)
p.add_argument("--state-root", required=True, type=Path)
p.add_argument("--artifact", required=True, type=Path)
p.add_argument("--artifact-size", required=True, type=int)
p.add_argument("--version", required=True)
p.add_argument("--target-oid", required=True)
p.add_argument("--evidence-closure-oid", required=True)
p.add_argument("--artifact-sha256", required=True)
p.add_argument("--approval-plan-id", required=True)
p.add_argument("--production-service", required=True)
p.add_argument("--production-port", required=True, type=int)
p.add_argument("--artifact", type=Path)
p.add_argument("--artifact-size", type=int)
p.add_argument("--version")
p.add_argument("--target-oid")
p.add_argument("--evidence-closure-oid")
p.add_argument("--artifact-sha256")
p.add_argument("--approval-plan-id")
p.add_argument("--production-service")
p.add_argument("--production-port", type=int)
p.add_argument("--repository-identity", default="aios/aios")
p.add_argument("--tag-adapter", required=True, nargs="+")
p.add_argument("--deploy-adapter", required=True, nargs="+")
@@ -612,18 +651,13 @@ def parser() -> argparse.ArgumentParser:
def main() -> int:
args = parser().parse_args()
binding = Binding(
version=args.version,
target_oid=args.target_oid,
evidence_closure_oid=args.evidence_closure_oid,
artifact_sha256=args.artifact_sha256,
approval_plan_id=args.approval_plan_id,
artifact_path=str(args.artifact),
artifact_size=args.artifact_size,
production_service=args.production_service,
production_port=args.production_port,
repository_identity=args.repository_identity,
)
plan = TrustedReleasePlan.load(args.release_plan) if args.release_plan else None
if plan is not None:
binding = Binding.from_trusted_plan(plan)
else:
legacy=(args.artifact,args.artifact_size,args.version,args.target_oid,args.evidence_closure_oid,args.artifact_sha256,args.approval_plan_id,args.production_service,args.production_port)
if any(value is None for value in legacy): raise SystemExit("legacy binding is incomplete")
binding = Binding(version=args.version,target_oid=args.target_oid,evidence_closure_oid=args.evidence_closure_oid,artifact_sha256=args.artifact_sha256,approval_plan_id=args.approval_plan_id,artifact_path=str(args.artifact),artifact_size=args.artifact_size,production_service=args.production_service,production_port=args.production_port,repository_identity=args.repository_identity)
try:
result = execute(
binding=binding,
@@ -634,6 +668,7 @@ def main() -> int:
deploy_command=parse_command(args.deploy_adapter),
registry_command=parse_command(args.registry_adapter),
baseline_command=parse_command(args.baseline_adapter),
release_plan_path=args.release_plan,
)
print(json.dumps(result, sort_keys=True))
return 0
+65 -2
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import os
import re
import subprocess
import time
import urllib.parse
from pathlib import Path
from typing import Iterable
@@ -306,14 +307,76 @@ def authoritative_diff(
from_oid = validate_git_revision(from_revision, "from", git_dir=path)
to_oid = validate_git_revision(to_revision, "to", git_dir=path)
args = ["diff", "--no-ext-diff", "--no-textconv", from_oid, to_oid]
result = run_authoritative_git(args, git_dir=path)
patch_limit = max(1, min(_MAX_OUTPUT, 256 * 1024))
process = subprocess.Popen(
["git", "--git-dir", str(path), *args],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
env={**os.environ, "GIT_OPTIONAL_LOCKS": "0"},
)
chunks: list[bytes] = []
stored_bytes = 0
total_bytes = 0
started = time.monotonic()
try:
while process.stdout is not None:
chunk = process.stdout.read(64 * 1024)
if not chunk:
break
total_bytes += len(chunk)
if stored_bytes < patch_limit:
remaining = patch_limit - stored_bytes
selected = chunk[:remaining]
chunks.append(selected)
stored_bytes += len(selected)
if time.monotonic() - started > _TIMEOUT:
process.kill()
raise AuthoritativeGitError("GIT_REF_QUERY_TIMEOUT")
stderr = (
process.stderr.read(8192).decode("utf-8", errors="replace")
if process.stderr is not None
else ""
)
returncode = process.wait(timeout=max(0.1, _TIMEOUT - (time.monotonic() - started)))
except (OSError, subprocess.TimeoutExpired) as exc:
process.kill()
process.wait()
raise AuthoritativeGitError("GIT_REF_QUERY_FAILED") from exc
if returncode != 0:
raise AuthoritativeGitError(_redact(stderr.strip() or "GIT_REF_QUERY_FAILED")[:200])
try:
names_result = run_authoritative_git(
["diff", "--no-ext-diff", "--no-textconv", "--name-status", from_oid, to_oid],
git_dir=path,
)
changed_paths = [
line.split("\t", 1)[-1]
for line in names_result.stdout.splitlines()
if line.strip()
]
except AuthoritativeGitOutputTooLarge:
changed_paths = []
try:
stats_result = run_authoritative_git(
["diff", "--no-ext-diff", "--no-textconv", "--stat", from_oid, to_oid],
git_dir=path,
)
stats = stats_result.stdout.strip()
except AuthoritativeGitOutputTooLarge:
stats = "GIT_STAT_OUTPUT_TRUNCATED"
patch = b"".join(chunks).decode("utf-8", errors="replace")
return {
"code": 0,
"stdout": result.stdout,
"stdout": patch,
"stderr": "",
"cmd": ["git", f"--git-dir={path}", *args],
"from_oid": from_oid,
"to_oid": to_oid,
"changed_paths": changed_paths,
"stats": stats,
"total_size_bytes": total_bytes,
"truncated": total_bytes > patch_limit,
"patch_limit_bytes": patch_limit,
}
@@ -964,7 +964,6 @@ def validate_writer_lease_request_v2(raw: Mapping[str, Any], probe: Any) -> dict
required = {
"schema",
"task_id",
"job_id",
"repository",
"branch_mode",
"base_oid",
@@ -977,7 +976,6 @@ def validate_writer_lease_request_v2(raw: Mapping[str, Any], probe: Any) -> dict
raise ProvisioningError("LEASE_REQUEST_V2_FIELDS_REJECTED", 409)
request = WriterLeaseRequestV2(
task_id=str(raw["task_id"]),
job_id=str(raw["job_id"]),
repository=str(raw["repository"]),
branch_mode=str(raw["branch_mode"]),
base_oid=str(raw["base_oid"]),
@@ -990,7 +988,6 @@ def validate_writer_lease_request_v2(raw: Mapping[str, Any], probe: Any) -> dict
return {
"schema": WriterLeaseRequestV2.schema,
"task_id": request.task_id,
"job_id": request.job_id,
"repository": request.repository,
"branch_mode": request.branch_mode,
"base_oid": request.base_oid,
@@ -12,6 +12,7 @@ import tempfile
from typing import Any, Mapping
from release_campaign_authorization import CampaignAuthorization, CampaignError
from trusted_release_execution_plan import PlanError, TrustedReleasePlan
class AcceptanceError(RuntimeError):
@@ -61,6 +62,7 @@ def evaluate(
staging: Mapping[str, Any],
main: Mapping[str, Any],
tag: Mapping[str, Any],
release_plan: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
required_binding = {
"version",
@@ -98,8 +100,27 @@ def evaluate(
raise AcceptanceError("rollback_readiness_failed")
if tag.get("conflict") is not False:
raise AcceptanceError("tag_conflict")
trusted_plan: TrustedReleasePlan | None = None
if release_plan is not None:
try:
trusted_plan = TrustedReleasePlan.from_mapping(release_plan)
trusted_plan.assert_binding(binding)
except PlanError as exc:
raise AcceptanceError(f"trusted_release_plan_invalid:{exc}") from exc
if trusted_plan.value["campaign_authorization_sha256"] != campaign.digest:
raise AcceptanceError("campaign_authorization_digest_mismatch")
if int(trusted_plan.value["campaign_generation"]) != int(campaign.data["generation"]):
raise AcceptanceError("campaign_generation_mismatch")
if main.get("tree_oid") not in (None, trusted_plan.value["tree_oid"]):
raise AcceptanceError("tree_oid_mismatch")
if evidence.get("release_plan_sha256") not in (None, trusted_plan.release_plan_sha256):
raise AcceptanceError("evidence_release_plan_mismatch")
if artifact.get("manifest_sha256") not in (None, trusted_plan.value["manifest_sha256"]):
raise AcceptanceError("manifest_digest_mismatch")
if artifact.get("package_closure_sha256") not in (None, trusted_plan.value["package_closure_sha256"]):
raise AcceptanceError("package_closure_digest_mismatch")
campaign.assert_operation("derive_campaign_acceptance")
return {
result = {
"schema": "aios.campaign-release-acceptance.v1",
"accepted": True,
"binding_replaced": False,
@@ -108,6 +129,22 @@ def evaluate(
**dict(binding),
"owner_interaction_required": False,
}
if trusted_plan is not None:
result.update({
"candidate_id": trusted_plan.value["candidate_id"],
"release_plan_sha256": trusted_plan.release_plan_sha256,
"candidate_binding_sha256": trusted_plan.candidate_binding_sha256,
"campaign_authorization_sha256": trusted_plan.value["campaign_authorization_sha256"],
"campaign_generation": trusted_plan.value["campaign_generation"],
"tree_oid": trusted_plan.value["tree_oid"],
"manifest_sha256": trusted_plan.value["manifest_sha256"],
"package_closure_sha256": trusted_plan.value["package_closure_sha256"],
"deployment_plan_sha256": trusted_plan.value["deployment_plan_sha256"],
"policy_sha256": trusted_plan.value["policy_sha256"],
"rollback": trusted_plan.value["rollback"],
"baseline": trusted_plan.value["baseline"],
})
return result
def main() -> int:
@@ -120,6 +157,7 @@ def main() -> int:
p.add_argument("--staging", type=Path, required=True)
p.add_argument("--main-facts", type=Path, required=True)
p.add_argument("--tag-facts", type=Path, required=True)
p.add_argument("--release-plan", type=Path)
p.add_argument("--output", type=Path, required=True)
a = p.parse_args()
@@ -136,6 +174,7 @@ def main() -> int:
secure_json(a.staging),
secure_json(a.main_facts),
secure_json(a.tag_facts),
secure_json(a.release_plan) if a.release_plan else None,
)
atomic_new(a.output, receipt)
print(json.dumps(receipt, sort_keys=True))
+16 -7
View File
@@ -8,6 +8,7 @@ import json
import os
import re
import subprocess
import sys
import tempfile
import time
import urllib.error
@@ -23,10 +24,15 @@ try:
from authoritative_git import AuthoritativeGitError, run_authoritative_git
except ModuleNotFoundError:
from .authoritative_git import AuthoritativeGitError, run_authoritative_git
try:
from release_version import RELEASE_VERSION_RE, validate_release_version
except ModuleNotFoundError:
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
from release_version import RELEASE_VERSION_RE, validate_release_version
OID_RE = re.compile(r"^[0-9a-f]{40}$")
VERSION_RE = re.compile(r"^V\d+R\d+C\d+(?:-[A-Za-z0-9_.-]+)?$")
VERSION_RE = RELEASE_VERSION_RE
DEPLOYMENT_EVIDENCE_RE = re.compile(r"^(?:sha256:[0-9a-f]{64}|git:[0-9a-f]{40})$")
FINGERPRINT_FIELDS = (
"version",
@@ -83,9 +89,14 @@ class PublicationManualOwnerReviewRequired(
def validate_version(version: str) -> str:
value = str(version or "")
if not VERSION_RE.fullmatch(value):
raise ValueError("invalid release version")
return validate_release_version(version)
def validate_gitea_api_url(api_url: str) -> str:
value = str(api_url or "").strip().rstrip("/")
parsed = urllib.parse.urlsplit(value)
if parsed.scheme not in {"http", "https"} or not parsed.netloc:
raise ValueError("Gitea API URL must be HTTP(S)")
return value
@@ -411,13 +422,11 @@ class GiteaTagClient:
timeout: float = 15,
token_line: int = 1,
):
self.api_url = api_url.rstrip("/")
self.api_url = validate_gitea_api_url(api_url)
self.repository = repository.strip("/")
self.token_file = token_file
self.timeout = timeout
self.token_line = token_line
if not self.api_url.startswith(("https://", "http://")):
raise ValueError("Gitea API URL must be HTTP(S)")
if self.repository.count("/") != 1:
raise ValueError("Gitea repository must be owner/name")
@@ -1,4 +1,7 @@
"""Governed V2R2C1 production release orchestration.
"""Governed release orchestration with a trusted-plan V2R2C2 path.
The legacy V2R2C1 implementation remains available only for compatibility;
new release requests are dispatched to the plan-driven executor below.
This module is deliberately scoped to the Owner-approved V2R2C1 binding. It
uses the existing TagPublisher and metadata adapters; it does not introduce a
@@ -20,6 +23,8 @@ from dataclasses import dataclass
from pathlib import Path
from typing import Any, Mapping
from trusted_release_execution_plan import PlanError, TrustedReleasePlan
from gitea_tag_publication import (
GiteaTagClient,
GitTagBackend,
@@ -406,6 +411,14 @@ class GovernedProductionReleaseExecutor:
config.remote_name,
state_root=self.root / "tag-publication",
)
# Release checks must use the same explicitly configured staging baseline
# as the Action/model runtime; do not fall back to the publication workspace.
self.publisher.baseline_path = Path(
os.environ.get("AIOS_GITOPS_BASELINE_AUTHORITY", str(config.baseline_path))
)
# The immutable runtime package is the source of the version projection;
# the mutable state workspace only stores publication state.
self.publisher.versions_root = Path(__file__).resolve().parents[1] / "versions"
def _request(self, request: Mapping[str, Any]) -> PublicationRequest:
binding = self.binding
@@ -546,7 +559,139 @@ def _health(url: str, binding: ReleaseBinding, artifact: Mapping[str, Any]) -> d
return {"status": "ACTIVE_VERIFIED" if exact else "CONFLICT", "http_status": status, **body}
def build_executor_config(config: Any) -> GovernedProductionReleaseExecutor:
class TrustedPlanReleaseExecutor:
"""Release handler whose identity comes only from an immutable trusted plan.
The bridge may prepare/read evidence and request tag publication, but it does
not perform production side effects. The privileged approved-release
service consumes the same plan and exact Owner approval envelope.
"""
def __init__(self, config: Any):
self.config = config
configured = getattr(config, "release_plan_path", None)
self.plan_path = Path(configured) if configured else None
self.state_root = Path(config.state_root) / "trusted-release"
self.state_root.mkdir(parents=True, exist_ok=True)
self._approval: dict[str, Any] | None = None
self._tag: dict[str, Any] | None = None
def _plan(self) -> TrustedReleasePlan:
path = self.plan_path
if path is None:
env_path = os.environ.get("AIOS_RELEASE_PLAN_PATH", "")
path = Path(env_path) if env_path else None
if path is None:
raise ReleaseBlocked("TRUSTED_RELEASE_PLAN_REQUIRED")
try:
return TrustedReleasePlan.load(path)
except PlanError as exc:
raise ReleaseBlocked(f"TRUSTED_RELEASE_PLAN_INVALID:{exc}") from exc
@staticmethod
def _binding(plan: TrustedReleasePlan) -> dict[str, Any]:
value = plan.value
return {
"version": value["version"],
"target_oid": value["source_commit"],
"evidence_closure_oid": value["evidence_closure_oid"],
"artifact_sha256": value["artifact_sha256"],
"artifact_size": value["artifact"]["size"],
"approval_plan_id": value["approval_plan_id"],
"candidate_id": value["candidate_id"],
"release_plan_sha256": plan.release_plan_sha256,
}
def _request(self, request: Mapping[str, Any]) -> tuple[TrustedReleasePlan, dict[str, Any]]:
plan = self._plan()
binding = self._binding(plan)
for key, expected in binding.items():
if key in request and request[key] != expected:
raise ManualOwnerReviewRequired("trusted release request binding mismatch:" + key)
return plan, binding
def approval_request(self, request: Mapping[str, Any]) -> dict[str, Any]:
if request.get("execute") is not False:
raise ReleaseBlocked("TAG_APPROVAL_REQUEST_MUST_BE_DRY_RUN")
plan, binding = self._request(request)
approval = request.get("owner_approval")
if not isinstance(approval, Mapping) or approval.get("approved") is not True:
raise ReleaseBlocked("OWNER_RELEASE_APPROVAL_REQUIRED")
for key in ("candidate_id", "release_plan_sha256", "approval_plan_id"):
if approval.get(key) != binding[key]:
raise ManualOwnerReviewRequired("OWNER_APPROVAL_BINDING_MISMATCH:" + key)
record = {"plan": plan.as_dict(), "owner_approval": dict(approval)}
digest = hashlib.sha256(canonical(record)).hexdigest()
if self._approval is not None and self._approval != record:
raise ManualOwnerReviewRequired("OWNER_APPROVAL_REPLAY_CONFLICT")
self._approval = record
return {"status": "RECORDED", "execute": False, "release_plan_sha256": plan.release_plan_sha256, "approval_digest_sha256": digest}
def _require_approval(self, plan: TrustedReleasePlan) -> None:
if self._approval is None or self._approval.get("plan") != plan.as_dict():
raise ReleaseBlocked("OWNER_RELEASE_APPROVAL_NOT_RECORDED")
def build_artifact(self, request: Mapping[str, Any]) -> dict[str, Any]:
plan, _ = self._request(request)
artifact = dict(plan.value["artifact"])
path = Path(str(artifact["path"]))
if path.is_symlink() or not path.is_file() or path.stat().st_size != artifact["size"]:
raise ReleaseBlocked("TRUSTED_ARTIFACT_NOT_PRESENT")
actual = hashlib.sha256(path.read_bytes()).hexdigest()
if actual != plan.value["artifact_sha256"]:
raise ReleaseBlocked("TRUSTED_ARTIFACT_DIGEST_MISMATCH")
return {"status": "VERIFIED", "version": plan.value["version"], "source_commit": plan.value["source_commit"], "tree_oid": plan.value["tree_oid"], "artifact": str(path), "artifact_size": artifact["size"], "artifact_sha256": actual, "manifest_sha256": plan.value["manifest_sha256"], "package_closure_sha256": plan.value["package_closure_sha256"], "release_plan_sha256": plan.release_plan_sha256}
def _publisher(self, plan: TrustedReleasePlan) -> TagPublisher:
root = self.state_root / plan.value["candidate_id"]
root.mkdir(parents=True, exist_ok=True)
publisher = TagPublisher(root / "publication-workspace", GitTagBackend(root / "publication-workspace", authoritative_repo=Path(self.config.authoritative_git_dir)), GiteaTagClient(self.config.api_base, "aios/aios", Path(self.config.token_path)), self.config.remote_name, state_root=root / "tag-publication")
publisher.baseline_path = Path(os.environ.get("AIOS_GITOPS_BASELINE_AUTHORITY", str(self.config.baseline_path)))
publisher.versions_root = Path(__file__).resolve().parents[1] / "versions"
return publisher
def publish_tag(self, request: Mapping[str, Any]) -> dict[str, Any]:
if request.get("execute") is not True:
raise ReleaseBlocked("TAG_PUBLICATION_REQUIRES_EXECUTE_TRUE")
plan, _ = self._request(request)
self._require_approval(plan)
publication = PublicationRequest(plan.value["version"], plan.value["source_commit"], plan.value["evidence_closure_oid"], plan.value["approval_plan_id"])
try:
result = self._publisher(plan)._verify_publication(publication, allow_push=True, previously_deployed=False)
except (PublicationManualOwnerReviewRequired, PublicationPending, PublicationReconciliationRequired) as exc:
raise ReconciliationRequired(str(exc)) from exc
self._tag = dict(result)
return {"status": "REMOTE_VERIFIED", **result, "release_plan_sha256": plan.release_plan_sha256}
def tag_status(self, request: Mapping[str, Any]) -> dict[str, Any]:
plan, _ = self._request(request)
return self._publisher(plan).get_tag_publication_status(plan.value["version"])
def execute_request(self, request: Mapping[str, Any]) -> dict[str, Any]:
plan, _ = self._request(request)
self._require_approval(plan)
raise ReleaseBlocked("PRODUCTION_EXECUTION_REQUIRES_APPROVED_RELEASE_EXECUTOR")
def consistency(self, request: Mapping[str, Any]) -> dict[str, Any]:
plan, binding = self._request(request)
result = {"status": "PASS", "release_binding": binding, "trusted_plan": plan.as_dict(), "production_mutation": False}
result["baseline"] = _read_json(Path(self.config.baseline_path))
result["registry"] = _read_json(Path(self.config.registry_root) / f"{plan.value['version']}.json")
return result
def acceptance(self, request: Mapping[str, Any]) -> dict[str, Any]:
result = self.consistency(request)
result["accepted"] = result["status"] == "PASS"
return result
def build_executor_config(config: Any) -> Any:
"""Select legacy compatibility only when no trusted new-release plan exists."""
plan_path = getattr(config, "release_plan_path", None) or os.environ.get("AIOS_RELEASE_PLAN_PATH", "")
target_version = getattr(config, "target_version", "")
if plan_path or target_version != VERSION:
return TrustedPlanReleaseExecutor(config)
return GovernedProductionReleaseExecutor(config)
+6 -2
View File
@@ -34,6 +34,8 @@ from gitea_tag_publication import (
TagPublisher,
)
from release_version import validate_release_version
OID_RE = re.compile(r"^[0-9a-f]{40}$")
PLAN_RE = re.compile(r"^sha256:[0-9a-f]{64}$")
@@ -44,8 +46,10 @@ def load_request() -> dict[str, Any]:
raise ValueError("request_object_required")
if value.get("schema") != "aios.production-executor-request.v1":
raise ValueError("request_schema_invalid")
if not re.fullmatch(r"V\d+R\d+C\d+(?:-[A-Za-z0-9_.-]+)?", str(value.get("version", ""))):
raise ValueError("version_invalid")
try:
validate_release_version(value.get("version", ""))
except ValueError as exc:
raise ValueError("version_invalid") from exc
for name in ("target_oid", "evidence_closure_oid"):
if not OID_RE.fullmatch(str(value.get(name, ""))):
raise ValueError(f"{name}_invalid")
-456
View File
@@ -1,456 +0,0 @@
from __future__ import annotations
import dataclasses
import datetime as dt
import hashlib
import json
import sqlite3
import re
from pathlib import Path
from typing import Any, Iterable, Mapping
UTC = dt.timezone.utc
_ALLOWED_CONFIDENTIALITY = frozenset({"public", "tenant", "project", "private", "run"})
_PRIVILEGED = frozenset({"*", "admin", "system.admin", "message.admin", "task.admin"})
class HandoffError(RuntimeError):
code = "HANDOFF_ERROR"
class MessageScopeMismatch(HandoffError):
code = "MESSAGE_SCOPE_MISMATCH_DENY"
class ConfidentialityScopeDenied(HandoffError):
code = "MESSAGE_CONFIDENTIALITY_SCOPE_ENFORCE"
class MessageExpired(HandoffError):
code = "MESSAGE_EXPIRED"
class SequenceGap(HandoffError):
code = "MESSAGE_SEQUENCE_GAP_DETECT"
class SequenceConflict(HandoffError):
code = "MESSAGE_SEQUENCE_CONFLICT"
class IdempotencyConflict(HandoffError):
code = "MESSAGE_IDEMPOTENCY_CONFLICT"
def _utcnow() -> dt.datetime:
return dt.datetime.now(UTC)
def _iso(value: dt.datetime | str | None) -> str | None:
if value is None:
return None
parsed = dt.datetime.fromisoformat(value.replace("Z", "+00:00")) if isinstance(value, str) else value
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=UTC)
return parsed.astimezone(UTC).isoformat()
def _canonical(value: Any) -> str:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False, default=str)
def _sha(value: Any) -> str:
return hashlib.sha256(_canonical(value).encode("utf-8")).hexdigest()
def _clean(value: Any) -> str | None:
if value is None:
return None
text = str(value).strip()
return text or None
def _tuple_scope(value: Iterable[str] | str | None) -> tuple[str, ...]:
if value is None:
return ()
if isinstance(value, str):
value = value.split(",")
return tuple(sorted({str(item).strip() for item in value if str(item).strip()}))
_SECRET_KEY = re.compile(r"(?i)(?:token|secret|password|private[_-]?key|api[_-]?key)")
_SECRET_VALUE = re.compile(r"(?i)bearer\s+[A-Za-z0-9._~+/-]{8,}|(?:token|secret|password|api[_-]?key)\s*[:=]\s*\S+")
_SAFE_REFERENCE = re.compile(r"^(?:systemd-credential|vault|secret-manager):[A-Za-z0-9][A-Za-z0-9._:/#-]{0,255}$")
def assert_payload_secret_safe(value: Any, *, key: str = "") -> None:
if isinstance(value, Mapping):
for child_key, child in value.items():
assert_payload_secret_safe(child, key=str(child_key))
return
if isinstance(value, (list, tuple)):
for child in value:
assert_payload_secret_safe(child, key=key)
return
if isinstance(value, (bytes, bytearray)):
raise ValueError("CREDENTIAL_SECRET_EXPOSURE_DENY")
if isinstance(value, str):
if _SECRET_VALUE.search(value):
raise ValueError("CREDENTIAL_SECRET_EXPOSURE_DENY")
if _SECRET_KEY.search(key) and value and _SAFE_REFERENCE.fullmatch(value) is None:
raise ValueError("CREDENTIAL_SECRET_EXPOSURE_DENY")
@dataclasses.dataclass(frozen=True)
class DeliveryContext:
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
run_id: str | None = None
capabilities: tuple[str, ...] = ()
@classmethod
def from_actor(cls, actor: Any) -> "DeliveryContext":
def get(name: str) -> Any:
if isinstance(actor, Mapping):
return actor.get(name)
return getattr(actor, name, None)
return cls(
tenant_id=_clean(get("tenant_id")),
user_id=_clean(get("user_id")),
project_id=_clean(get("project_id")),
conversation_id=_clean(get("conversation_id")),
run_id=_clean(get("run_id")),
capabilities=_tuple_scope(get("capabilities") or get("capability_scope")),
)
@property
def privileged(self) -> bool:
return bool(set(self.capabilities) & _PRIVILEGED)
@dataclasses.dataclass(frozen=True)
class MessageEnvelope:
source: str
target: str
message_type: str
payload: Mapping[str, Any]
sequence: int
idempotency_key: str
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
task_id: str | None = None
run_id: str | None = None
correlation_id: str | None = None
causation_id: str | None = None
capability_scope: tuple[str, ...] = ()
confidentiality_scope: str = "private"
expires_at: str | None = None
created_at: str = dataclasses.field(default_factory=lambda: _iso(_utcnow()) or "")
envelope_id: str = ""
def __post_init__(self) -> None:
assert_payload_secret_safe(self.payload)
for field in ("source", "target", "message_type", "idempotency_key", "tenant_id", "user_id",
"project_id", "conversation_id", "task_id", "run_id", "correlation_id", "causation_id"):
object.__setattr__(self, field, _clean(getattr(self, field)) if field not in {
"source", "target", "message_type", "idempotency_key"
} else (_clean(getattr(self, field)) or ""))
object.__setattr__(self, "capability_scope", _tuple_scope(self.capability_scope))
object.__setattr__(self, "confidentiality_scope", (_clean(self.confidentiality_scope) or "private").lower())
object.__setattr__(self, "expires_at", _iso(self.expires_at))
object.__setattr__(self, "created_at", _iso(self.created_at) or "")
if not self.source or not self.target or not self.message_type or not self.idempotency_key:
raise ValueError("MESSAGE_ENVELOPE_IDENTITY_REQUIRED")
if not isinstance(self.sequence, int) or self.sequence < 1:
raise ValueError("MESSAGE_SEQUENCE_MUST_BE_POSITIVE")
if self.confidentiality_scope not in _ALLOWED_CONFIDENTIALITY:
raise ValueError("MESSAGE_CONFIDENTIALITY_SCOPE_INVALID")
if self.confidentiality_scope != "public" and self.tenant_id is None:
raise ValueError("MESSAGE_TENANT_REQUIRED")
if self.confidentiality_scope in {"project", "private", "run"} and self.project_id is None:
raise ValueError("MESSAGE_PROJECT_REQUIRED")
if self.confidentiality_scope in {"private", "run"} and self.user_id is None:
raise ValueError("MESSAGE_USER_REQUIRED")
if self.confidentiality_scope == "run" and self.run_id is None:
raise ValueError("MESSAGE_RUN_REQUIRED")
expected = stable_event_id(
source=self.source, target=self.target, message_type=self.message_type, payload=self.payload,
tenant_id=self.tenant_id, user_id=self.user_id, project_id=self.project_id,
conversation_id=self.conversation_id, task_id=self.task_id, run_id=self.run_id,
correlation_id=self.correlation_id, causation_id=self.causation_id,
sequence=self.sequence, idempotency_key=self.idempotency_key,
)
if self.envelope_id and self.envelope_id != expected:
raise ValueError("MESSAGE_ENVELOPE_ID_MISMATCH")
object.__setattr__(self, "envelope_id", expected)
@property
def payload_sha256(self) -> str:
return _sha(self.payload)
@property
def stream_key(self) -> str:
return _sha({
"tenant_id": self.tenant_id, "source": self.source, "target": self.target,
"conversation_id": self.conversation_id, "task_id": self.task_id, "run_id": self.run_id,
})
def is_expired(self, *, now: dt.datetime | None = None) -> bool:
if self.expires_at is None:
return False
return dt.datetime.fromisoformat(self.expires_at) <= (now or _utcnow()).astimezone(UTC)
def as_record(self) -> dict[str, Any]:
return {
"envelope_id": self.envelope_id, "stream_key": self.stream_key, "source": self.source,
"target": self.target, "message_type": self.message_type, "payload_json": _canonical(self.payload),
"payload_sha256": self.payload_sha256, "tenant_id": self.tenant_id, "user_id": self.user_id,
"project_id": self.project_id, "conversation_id": self.conversation_id, "task_id": self.task_id,
"run_id": self.run_id, "correlation_id": self.correlation_id,
"causation_id": self.causation_id, "sequence": self.sequence,
"capability_scope_json": _canonical(self.capability_scope),
"confidentiality_scope": self.confidentiality_scope, "idempotency_key": self.idempotency_key,
"expires_at": self.expires_at, "created_at": self.created_at,
}
def stable_event_id(*, source: Any, target: Any, message_type: Any, payload: Any,
tenant_id: Any = None, user_id: Any = None, project_id: Any = None,
conversation_id: Any = None, task_id: Any = None, run_id: Any = None,
correlation_id: Any = None, causation_id: Any = None,
sequence: Any = None, idempotency_key: Any = None) -> str:
assert_payload_secret_safe(payload)
material = {
"source": _clean(source), "target": _clean(target), "message_type": _clean(message_type),
"payload_sha256": _sha(payload), "tenant_id": _clean(tenant_id), "user_id": _clean(user_id),
"project_id": _clean(project_id), "conversation_id": _clean(conversation_id),
"task_id": _clean(task_id), "run_id": _clean(run_id), "causation_id": _clean(causation_id),
"sequence": sequence, "idempotency_key": _clean(idempotency_key),
}
# Preserve historical envelope IDs for legacy rows that predate correlation_id.
if _clean(correlation_id) is not None:
material["correlation_id"] = _clean(correlation_id)
return _sha(material)
def bind_envelope_scope(envelope: MessageEnvelope, actor: Any) -> MessageEnvelope:
ctx = DeliveryContext.from_actor(actor)
if ctx.privileged:
return envelope
if ctx.tenant_id is None:
if envelope.tenant_id is not None:
raise MessageScopeMismatch(MessageScopeMismatch.code)
return envelope
updates: dict[str, Any] = {}
for field in ("tenant_id", "user_id", "project_id", "conversation_id", "run_id"):
trusted = getattr(ctx, field)
requested = getattr(envelope, field)
if requested is not None and requested != trusted:
raise MessageScopeMismatch(f"{MessageScopeMismatch.code}:{field}")
if requested is None and trusted is not None:
updates[field] = trusted
if not updates:
return envelope
data = dataclasses.asdict(envelope)
data.update(updates)
data["envelope_id"] = ""
return MessageEnvelope(**data)
def authorize_delivery(actor: Any, envelope: MessageEnvelope) -> None:
ctx = DeliveryContext.from_actor(actor)
if ctx.privileged:
return
if not set(envelope.capability_scope).issubset(set(ctx.capabilities)):
raise ConfidentialityScopeDenied(f"{ConfidentialityScopeDenied.code}:capability")
if envelope.confidentiality_scope == "public":
return
if ctx.tenant_id != envelope.tenant_id:
raise MessageScopeMismatch(f"{MessageScopeMismatch.code}:tenant_id")
if envelope.confidentiality_scope in {"project", "private", "run"} and ctx.project_id != envelope.project_id:
raise ConfidentialityScopeDenied(f"{ConfidentialityScopeDenied.code}:project_id")
if envelope.confidentiality_scope in {"private", "run"} and ctx.user_id != envelope.user_id:
raise ConfidentialityScopeDenied(f"{ConfidentialityScopeDenied.code}:user_id")
if envelope.confidentiality_scope == "run" and ctx.run_id != envelope.run_id:
raise ConfidentialityScopeDenied(f"{ConfidentialityScopeDenied.code}:run_id")
_SCHEMA = """
CREATE TABLE IF NOT EXISTS message_handoff (
envelope_id TEXT PRIMARY KEY,
stream_key TEXT NOT NULL,
source TEXT NOT NULL,
target TEXT NOT NULL,
message_type TEXT NOT NULL,
payload_json TEXT NOT NULL,
payload_sha256 TEXT NOT NULL,
tenant_id TEXT,
user_id TEXT,
project_id TEXT,
conversation_id TEXT,
task_id TEXT,
run_id TEXT,
correlation_id TEXT,
causation_id TEXT,
sequence INTEGER NOT NULL CHECK(sequence >= 1),
capability_scope_json TEXT NOT NULL,
confidentiality_scope TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
expires_at TEXT,
created_at TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'PENDING',
delivered_at TEXT,
acked_at TEXT,
UNIQUE(stream_key, idempotency_key),
UNIQUE(stream_key, sequence)
);
CREATE INDEX IF NOT EXISTS idx_message_handoff_target_status
ON message_handoff(target, status, created_at);
CREATE INDEX IF NOT EXISTS idx_message_handoff_scope
ON message_handoff(tenant_id, project_id, user_id, conversation_id, run_id);
"""
class DurableHandoffStore:
def __init__(self, path: str | Path) -> None:
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
with self._connect() as conn:
conn.executescript(_SCHEMA)
columns = {str(row[1]) for row in conn.execute("PRAGMA table_info(message_handoff)")}
if "correlation_id" not in columns:
conn.execute("ALTER TABLE message_handoff ADD COLUMN correlation_id TEXT")
conn.execute(
"CREATE INDEX IF NOT EXISTS idx_message_handoff_lineage "
"ON message_handoff(correlation_id, causation_id)"
)
def _connect(self) -> sqlite3.Connection:
conn = sqlite3.connect(self.path, timeout=30.0)
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA foreign_keys=ON")
conn.execute("PRAGMA busy_timeout=30000")
return conn
@staticmethod
def _from_row(row: sqlite3.Row) -> MessageEnvelope:
return MessageEnvelope(
source=row["source"], target=row["target"], message_type=row["message_type"],
payload=json.loads(row["payload_json"]), tenant_id=row["tenant_id"], user_id=row["user_id"],
project_id=row["project_id"], conversation_id=row["conversation_id"], task_id=row["task_id"],
run_id=row["run_id"], correlation_id=row["correlation_id"],
causation_id=row["causation_id"], sequence=int(row["sequence"]),
capability_scope=tuple(json.loads(row["capability_scope_json"])),
confidentiality_scope=row["confidentiality_scope"], idempotency_key=row["idempotency_key"],
expires_at=row["expires_at"], created_at=row["created_at"], envelope_id=row["envelope_id"],
)
def put(self, envelope: MessageEnvelope) -> tuple[str, MessageEnvelope]:
if envelope.is_expired():
raise MessageExpired(MessageExpired.code)
record = envelope.as_record()
with self._connect() as conn:
conn.execute("BEGIN IMMEDIATE")
replay = conn.execute(
"SELECT * FROM message_handoff WHERE stream_key=? AND idempotency_key=?",
(envelope.stream_key, envelope.idempotency_key),
).fetchone()
if replay is not None:
existing = self._from_row(replay)
if existing.envelope_id != envelope.envelope_id or existing.payload_sha256 != envelope.payload_sha256:
raise IdempotencyConflict(IdempotencyConflict.code)
conn.commit()
return "REPLAY", existing
seq = conn.execute(
"SELECT * FROM message_handoff WHERE stream_key=? AND sequence=?",
(envelope.stream_key, envelope.sequence),
).fetchone()
if seq is not None:
existing = self._from_row(seq)
if existing.envelope_id == envelope.envelope_id:
conn.commit()
return "REPLAY", existing
raise SequenceConflict(SequenceConflict.code)
maximum = int(conn.execute(
"SELECT COALESCE(MAX(sequence),0) FROM message_handoff WHERE stream_key=?",
(envelope.stream_key,),
).fetchone()[0])
expected = maximum + 1
if envelope.sequence != expected:
raise SequenceGap(f"{SequenceGap.code}:expected={expected}:actual={envelope.sequence}")
columns = tuple(record)
placeholders = ",".join("?" for _ in columns)
conn.execute(
f"INSERT INTO message_handoff ({','.join(columns)}) VALUES ({placeholders})",
tuple(record[column] for column in columns),
)
conn.commit()
return "CREATED", envelope
def get(self, envelope_id: str, *, actor: Any) -> MessageEnvelope:
with self._connect() as conn:
row = conn.execute("SELECT * FROM message_handoff WHERE envelope_id=?", (envelope_id,)).fetchone()
if row is None:
raise KeyError(envelope_id)
envelope = self._from_row(row)
if envelope.is_expired():
conn.execute("UPDATE message_handoff SET status='EXPIRED' WHERE envelope_id=?", (envelope_id,))
conn.commit()
raise MessageExpired(MessageExpired.code)
authorize_delivery(actor, envelope)
return envelope
def pending(self, *, target: str, actor: Any, limit: int = 100) -> list[MessageEnvelope]:
if not 1 <= limit <= 1000:
raise ValueError("MESSAGE_PENDING_LIMIT_INVALID")
visible: list[MessageEnvelope] = []
with self._connect() as conn:
rows = conn.execute(
"SELECT * FROM message_handoff WHERE target=? AND status='PENDING' ORDER BY created_at, sequence LIMIT ?",
(target, limit),
).fetchall()
for row in rows:
envelope = self._from_row(row)
if envelope.is_expired():
conn.execute("UPDATE message_handoff SET status='EXPIRED' WHERE envelope_id=?", (envelope.envelope_id,))
continue
try:
authorize_delivery(actor, envelope)
except HandoffError:
continue
visible.append(envelope)
conn.commit()
return visible
def mark_delivered(self, envelope_id: str, *, actor: Any) -> MessageEnvelope:
envelope = self.get(envelope_id, actor=actor)
with self._connect() as conn:
conn.execute(
"UPDATE message_handoff SET status='DELIVERED', delivered_at=? "
"WHERE envelope_id=? AND status='PENDING'",
(_iso(_utcnow()), envelope_id),
)
conn.commit()
return envelope
def acknowledge(self, envelope_id: str, *, actor: Any) -> MessageEnvelope:
envelope = self.get(envelope_id, actor=actor)
with self._connect() as conn:
conn.execute(
"UPDATE message_handoff SET status='ACKED', acked_at=? WHERE envelope_id=?",
(_iso(_utcnow()), envelope_id),
)
conn.commit()
return envelope
class HandoffFabric:
def __init__(self, store: DurableHandoffStore) -> None:
self.store = store
def publish(self, envelope: MessageEnvelope, *, actor: Any | None = None) -> tuple[str, MessageEnvelope]:
return self.store.put(bind_envelope_scope(envelope, actor) if actor is not None else envelope)
@@ -1,68 +0,0 @@
from __future__ import annotations
from typing import Any, Mapping
from message_handoff import MessageEnvelope
def _get(value: Any, *names: str, default: Any = None) -> Any:
for name in names:
if isinstance(value, Mapping) and name in value:
return value[name]
if hasattr(value, name):
return getattr(value, name)
return default
def agent_event_to_envelope(event: Any, *, target: str | None = None, sequence: int = 1,
idempotency_key: str | None = None, **scope: Any) -> MessageEnvelope:
payload = _get(event, "payload", "data", default={}) or {}
target = target or _get(event, "target", "recipient")
if not target:
raise ValueError("MESSAGE_TARGET_REQUIRED")
return MessageEnvelope(
source=str(_get(event, "source", "producer", "agent_id", default="agent-event")),
target=str(target),
message_type=str(_get(event, "event_type", "type", "message_type", default="agent.event")),
payload=dict(payload),
sequence=sequence,
idempotency_key=idempotency_key or str(_get(event, "id", "event_id", default="")) or f"agent-event:{sequence}",
causation_id=_get(event, "causation_id"),
**scope,
)
def taskplane_event_to_envelope(event: Any, *, target: str, sequence: int = 1,
idempotency_key: str | None = None, **scope: Any) -> MessageEnvelope:
data = _get(event, "data", "payload", default={}) or {}
event_id = str(_get(event, "id", "event_id", default=""))
return MessageEnvelope(
source=str(_get(event, "source", default="task-plane")),
target=target,
message_type=str(_get(event, "type", "event_type", default="task.event")),
payload=dict(data),
sequence=sequence,
idempotency_key=idempotency_key or event_id or f"task-event:{sequence}",
causation_id=event_id or None,
**scope,
)
def legacy_message_to_envelope(message: Any, *, target: str | None = None, sequence: int = 1,
idempotency_key: str | None = None, **scope: Any) -> MessageEnvelope:
payload = _get(message, "payload", "data", "body", "content", default={})
if not isinstance(payload, Mapping):
payload = {"value": payload}
target = target or _get(message, "target", "recipient", "receiver")
if not target:
raise ValueError("MESSAGE_TARGET_REQUIRED")
source = _get(message, "source", "sender", "agent_id", default="legacy-message-bus")
kind = _get(message, "message_type", "type", "kind", default="legacy.message")
identifier = _get(message, "id", "message_id", "event_id")
return MessageEnvelope(
source=str(source), target=str(target), message_type=str(kind), payload=dict(payload),
sequence=sequence,
idempotency_key=idempotency_key or (str(identifier) if identifier else f"legacy:{sequence}"),
causation_id=str(identifier) if identifier else None,
**scope,
)
@@ -27,6 +27,8 @@ import urllib.request
from pathlib import Path
from typing import Any, Mapping
from release_version import validate_release_version
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
PLAN_RE = re.compile(r"^sha256:[0-9a-f]{64}$")
@@ -86,8 +88,10 @@ def load_request() -> dict[str, Any]:
raise ValueError("request_schema_invalid")
if value.get("repository_identity") != "aios/aios":
raise ValueError("repository_identity_invalid")
if not re.fullmatch(r"V\d+R\d+C\d+(?:-[A-Za-z0-9_.-]+)?", str(value.get("version", ""))):
raise ValueError("version_invalid")
try:
validate_release_version(value.get("version", ""))
except ValueError as exc:
raise ValueError("version_invalid") from exc
for name in ("target_oid", "evidence_closure_oid"):
if not OID_RE.fullmatch(str(value.get(name, ""))):
raise ValueError(f"{name}_invalid")
+1 -9
View File
@@ -11,7 +11,6 @@ from dataclasses import asdict, dataclass, field
from typing import Any
from .search_gateway_orchestrator import SearchGatewayOrchestrator
from .trace_tool_credential_governance import GovernanceActor, TraceContext
@dataclass(frozen=True)
@@ -44,9 +43,6 @@ class SearchGatewayAgentAdapter:
domain_profiles: list[str] | None = None,
user_scope: str = "public",
correlation_id: str | None = None,
*,
actor: GovernanceActor | dict[str, Any] | None = None,
trace: TraceContext | None = None,
) -> dict[str, Any]:
payload = {
"query": query,
@@ -63,11 +59,7 @@ class SearchGatewayAgentAdapter:
"session_profile": context.session_profile,
"correlation_id": correlation_id,
}
if actor is None and trace is None:
# Preserve adapter compatibility; the governed orchestrator itself
# still fails closed when trusted context is absent.
return self.orchestrator.execute(payload)
return self.orchestrator.execute(payload, actor=actor, trace=trace)
return self.orchestrator.execute(payload)
class OpenClawSearchClient(SearchGatewayAgentAdapter):
+4 -70
View File
@@ -13,18 +13,10 @@ from __future__ import annotations
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
import hashlib
import json
from typing import Any, Protocol
from uuid import uuid4
from .search_profile_resolver import DomainQueryProfileRegistry, SearchProfile, SearchProfileResolver
from .trace_tool_credential_governance import (
GovernanceActor,
GovernanceError,
GovernanceStore,
TraceContext,
)
def utc_now() -> str:
@@ -164,25 +156,13 @@ class SearchPolicyGuard:
class SearchGateway:
def __init__(self, providers: list[SearchProvider] | None = None, *, governance_store: GovernanceStore | None = None):
def __init__(self, providers: list[SearchProvider] | None = None):
self.providers = providers or []
self.governance_store = governance_store
self.profile_resolver = SearchProfileResolver()
self.planner = QueryPlanner()
self.guard = SearchPolicyGuard()
def search(self, request: SearchRequest, *, actor: GovernanceActor | dict[str, Any] | None = None, trace: TraceContext | None = None) -> SearchResponse:
trusted_actor: GovernanceActor | None = None
if self.governance_store is not None:
if actor is None or trace is None:
raise GovernanceError("TOOL_MCP_AUTHZ_DENY")
trusted_actor = GovernanceActor.from_value(actor)
if request.requester != trusted_actor.identity:
raise GovernanceError("TOOL_MCP_AUTHZ_DENY", field="requester")
if request.correlation_id is not None and request.correlation_id not in {
trace.trace_id, trace.correlation_id
}:
raise GovernanceError("TOOL_SCOPE_PROPAGATION", field="correlation_id")
def search(self, request: SearchRequest) -> SearchResponse:
request_id = request.correlation_id or f"search_{uuid4().hex[:12]}"
profile = self.profile_resolver.resolve(request.group_profile, request.user_profile, request.session_profile)
expanded_queries = self.planner.expand(request, profile)
@@ -204,60 +184,14 @@ class SearchGateway:
attempts: list[SearchProviderAttempt] = []
providers = self._ordered_providers(profile)
for provider in providers:
if request.authorized_sources and provider.name not in request.authorized_sources:
attempts.append(SearchProviderAttempt(provider=provider.name, status="denied", reason="SOURCE_AUTHZ_DENY"))
continue
if request.user_scope != "public" and provider.name not in request.authorized_sources:
attempts.append(SearchProviderAttempt(provider=provider.name, status="denied", reason="SOURCE_AUTHZ_REQUIRED"))
continue
try:
if self.governance_store is not None:
assert trusted_actor is not None and trace is not None
input_digest = hashlib.sha256(
json.dumps(
{"query": request.query, "intent": request.intent, "provider": provider.name},
sort_keys=True, separators=(",", ":"), ensure_ascii=False,
).encode("utf-8")
).hexdigest()
invocation_key = hashlib.sha256(
f"{trace.trace_id}:{trace.span_id}:{provider.name}:{input_digest}".encode("utf-8")
).hexdigest()
authorization = self.governance_store.authorize_tool(
actor=trusted_actor,
tool_name=f"search.{provider.name}",
trace=trace,
requested_scope=trace.scope,
input_digest=input_digest,
invocation_key=invocation_key,
)
if authorization.get("idempotent"):
raise GovernanceError("TOOL_INVOCATION_REPLAY")
provider_evidence, attempt = provider.search(request, expanded_queries)
if self.governance_store is not None:
result_digest = hashlib.sha256(
json.dumps(
{
"evidence": [item.to_dict() for item in provider_evidence],
"attempt": attempt.to_dict(),
},
sort_keys=True, separators=(",", ":"), ensure_ascii=False,
).encode("utf-8")
).hexdigest()
self.governance_store.record_tool_receipt(
invocation_key=invocation_key,
result_digest=result_digest,
actor=trusted_actor,
trace=trace,
)
evidence.extend(provider_evidence)
attempts.append(attempt)
if provider_evidence:
break
except GovernanceError:
raise
except Exception:
# Provider exceptions may contain URLs, tokens, or local paths.
attempts.append(SearchProviderAttempt(provider=getattr(provider, "name", "unknown"), status="failed", reason="PROVIDER_FAILURE"))
except Exception as exc:
attempts.append(SearchProviderAttempt(provider=getattr(provider, "name", "unknown"), status="failed", reason=str(exc)))
if evidence:
return SearchResponse(
+6 -8
View File
@@ -7,29 +7,27 @@ try:
except Exception:
APIRouter = None
from .search_gateway import SearchRequest
from .search_gateway_orchestrator import SearchGatewayOrchestrator
from .trace_tool_credential_governance import GovernanceActor, TraceContext
from .search_gateway import SearchGateway, SearchRequest
from .search_gateway_dashboard import SearchGatewayDashboard
orchestrator = SearchGatewayOrchestrator()
gateway = SearchGateway()
dashboard = SearchGatewayDashboard()
def search_gateway_request(payload: dict[str, Any], *, actor: GovernanceActor | dict[str, Any] | None = None, trace: TraceContext | None = None) -> dict[str, Any]:
return orchestrator.execute(payload, actor=actor, trace=trace)
def search_gateway_request(payload: dict[str, Any]) -> dict[str, Any]:
request = SearchRequest(**payload)
return gateway.search(request).to_dict()
def get_search_gateway_status() -> dict[str, Any]:
return {
"module": "search-gateway",
"status": "ready",
"providers_configured": len(orchestrator.gateway.providers),
"providers_configured": len(gateway.providers),
"policy_guard": "enabled",
"profile_customization": "enabled",
"fallback_required_when_provider_unavailable": True,
"governance": "required",
}
@@ -3,49 +3,24 @@
from __future__ import annotations
from typing import Any
import os
from .search_audit_store import SearchAuditStore
from .search_event_bridge import SearchEventBridge
from .search_gateway import SearchGateway, SearchRequest
from .search_provider_registry import SearchProviderRegistry
from .trace_tool_credential_governance import (
GovernanceActor,
GovernanceScope,
GovernanceStore,
ToolDescriptor,
TraceContext,
)
class SearchGatewayOrchestrator:
def __init__(self, *, governance_store: GovernanceStore | None = None):
def __init__(self):
self.registry = SearchProviderRegistry()
self.audit = SearchAuditStore()
self.events = SearchEventBridge()
self.governance = governance_store or GovernanceStore(
os.environ.get("AIOS_SEARCH_GOVERNANCE_DB", "runtime/search-governance.sqlite3")
)
registrar = GovernanceActor(
"search-runtime-bootstrap", GovernanceScope("system"), ("system.admin",)
)
for provider in self.registry.ordered():
self.governance.register_tool(
ToolDescriptor(
f"search.{provider.name}", "tool", GovernanceScope("global"), ("tool.search",)
),
registrar,
)
self.gateway = SearchGateway(self.registry.ordered(), governance_store=self.governance)
self.gateway = SearchGateway(self.registry.ordered())
def execute(self, payload: dict[str, Any], *, actor: GovernanceActor | dict[str, Any] | None = None, trace: TraceContext | None = None) -> dict[str, Any]:
if actor is None or trace is None:
from .trace_tool_credential_governance import GovernanceError
raise GovernanceError("TOOL_MCP_AUTHZ_DENY")
self.governance.record_trace(trace, actor)
def execute(self, payload: dict[str, Any]) -> dict[str, Any]:
request = SearchRequest(**payload)
self.events.search_started(request.to_dict())
response = self.gateway.search(request, actor=actor, trace=trace)
response = self.gateway.search(request)
result = response.to_dict()
for attempt in result.get("attempts", []):
self.events.provider_attempted(result["request_id"], attempt)
@@ -60,7 +35,6 @@ class SearchGatewayOrchestrator:
"registry": self.registry.status(),
"audit": "enabled",
"events": "enabled",
"governance": "required",
}
@@ -22,11 +22,12 @@ from enum import Enum
from pathlib import Path
from typing import Any, Protocol
from release_version import validate_release_version
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SHA256_RE = re.compile(r"^[0-9a-f]{64}$")
CANDIDATE_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.-]{2,127}$")
VERSION_RE = re.compile(r"^V\d+R\d+C\d+(?:-[A-Za-z0-9_.-]+)?$")
SECRET_PATTERNS = (
re.compile(r"(?i)(authorization\s*[:=]\s*)(?:bearer|token)\s+\S+"),
re.compile(r"(?i)((?:access_)?token\s*[:=]\s*)\S+"),
@@ -769,8 +770,10 @@ class TagApprovalEnvelope:
def validate(self) -> None:
if self.action != "production_tag":
raise ApprovalError("approval does not authorize production tag")
if not VERSION_RE.fullmatch(self.version):
raise ApprovalError("invalid release version")
try:
validate_release_version(self.version)
except ValueError as exc:
raise ApprovalError("invalid release version") from exc
_validate_oid("tag_oid", self.tag_oid)
_validate_oid("target_oid", self.target_oid)
if _parse_utc(self.expires_at) <= datetime.now(timezone.utc):
+603 -32
View File
@@ -9,6 +9,7 @@ from __future__ import annotations
import base64
import hashlib
import hmac
import json
import os
import re
@@ -40,6 +41,8 @@ from silent_release_controller import (
IDENTIFIER_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.:-]{2,255}$")
NONCE_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.:-]{15,255}$")
BUILDER_ATTESTATION_TRUST_DOMAIN = "aios.trusted-builder.attestation.v1"
BUILDER_ATTESTATION_PURPOSE = "AIOS_TRUSTED_RELEASE_BUILDER"
ALLOWED_CAPABILITIES = frozenset(
{
"runtime_switch",
@@ -62,6 +65,10 @@ class CapabilityAuthorizationError(RuntimeError):
"""The signed plan cannot authorize the requested operation."""
class TrustedCandidateBuilderUnavailable(RuntimeError):
"""The purpose-bound builder dependency cannot currently serve a build."""
class StaleApprovalError(CapabilityAuthorizationError):
"""Live release evidence no longer matches the signed approval."""
@@ -222,6 +229,10 @@ JOURNAL_GENESIS_HASH = "0" * 64
def _fsync_directory(path: Path) -> None:
if os.name == "nt":
# Windows does not permit opening a directory with os.open. Durable
# Linux host execution retains the directory fsync below.
return
descriptor = os.open(Path(path), os.O_RDONLY)
try:
os.fsync(descriptor)
@@ -715,6 +726,14 @@ class BuilderAttestation:
canary_evidence_sha256: str
issued_at: str
signature: str
trust_domain: str = BUILDER_ATTESTATION_TRUST_DOMAIN
builder_key_generation: int | None = None
signature_algorithm: str | None = None
signature_purpose: str | None = None
gitea_evidence_sha256: str = ""
rollback_evidence_sha256: str = ""
policy_evidence_sha256: str = ""
secret_scan_evidence_sha256: str = ""
def __post_init__(self) -> None:
_validate_identifier(
@@ -731,7 +750,35 @@ class BuilderAttestation:
"canary_evidence_sha256",
):
_validate_sha(name, getattr(self, name))
for name in (
"gitea_evidence_sha256",
"rollback_evidence_sha256",
"policy_evidence_sha256",
"secret_scan_evidence_sha256",
):
if getattr(self, name):
_validate_sha(name, getattr(self, name))
_parse_utc(self.issued_at)
if self.trust_domain != BUILDER_ATTESTATION_TRUST_DOMAIN:
raise CapabilityAuthorizationError(
"builder attestation trust domain is invalid"
)
metadata = (
self.builder_key_generation,
self.signature_algorithm,
self.signature_purpose,
)
if any(value is not None for value in metadata):
if (
not isinstance(self.builder_key_generation, int)
or isinstance(self.builder_key_generation, bool)
or self.builder_key_generation < 0
or self.signature_algorithm != "Ed25519"
or self.signature_purpose != BUILDER_ATTESTATION_PURPOSE
):
raise CapabilityAuthorizationError(
"builder attestation signing metadata is invalid"
)
@property
def canonical_payload(self) -> dict[str, Any]:
@@ -754,20 +801,290 @@ class BuilderAttestation:
}
@classmethod
def from_dict(cls, payload: Mapping[str, Any]) -> "BuilderAttestation":
def from_dict(
cls,
payload: Mapping[str, Any],
*,
require_fingerprint: bool = False,
) -> "BuilderAttestation":
values = dict(payload)
values.pop("fingerprint", None)
return cls(**values)
supplied_fingerprint = values.pop("fingerprint", None)
# Older local records did not spell out the domain. They are only
# usable when re-signed by the current builder; their original
# signatures cannot validate against this domain-separated payload.
values.setdefault("trust_domain", BUILDER_ATTESTATION_TRUST_DOMAIN)
value = cls(**values)
if supplied_fingerprint is None:
if require_fingerprint:
raise CapabilityAuthorizationError(
"builder attestation digest is required"
)
return value
expected = _validate_sha(
"builder attestation digest", str(supplied_fingerprint)
)
if not hmac.compare_digest(expected, value.fingerprint):
raise CapabilityAuthorizationError(
"builder attestation digest is invalid"
)
return value
@dataclass(frozen=True)
class BuilderAttestationKeyRecord:
"""One purpose-bound public verification key from the host keyring."""
key_id: str
public_key: bytes
algorithm: str
purpose: str
generation: int
not_before: str
expires_at: str
status: str
def __post_init__(self) -> None:
_validate_identifier("builder_key_id", self.key_id)
if self.algorithm != "Ed25519":
raise ValueError("unsupported builder signature algorithm")
if self.purpose != BUILDER_ATTESTATION_PURPOSE:
raise ValueError("builder key purpose is invalid")
if not isinstance(self.generation, int) or isinstance(
self.generation, bool
) or self.generation < 0:
raise ValueError("builder key generation is invalid")
if self.status not in {"ACTIVE", "REVOKED"}:
raise ValueError("builder key status is invalid")
if len(self.public_key) != 32:
raise ValueError("Ed25519 public key must contain 32 bytes")
Ed25519PublicKey.from_public_bytes(self.public_key)
if _parse_utc(self.not_before) >= _parse_utc(self.expires_at):
raise ValueError("builder key validity window is invalid")
@property
def public_key_sha256(self) -> str:
return hashlib.sha256(self.public_key).hexdigest()
def validate_at(self, instant: datetime) -> None:
if self.status != "ACTIVE":
raise CapabilityAuthorizationError(
"builder attestation key is unavailable or revoked"
)
if instant.tzinfo is None:
raise CapabilityAuthorizationError(
"builder attestation verification time is invalid"
)
if not _parse_utc(self.not_before) <= instant <= _parse_utc(
self.expires_at
):
raise CapabilityAuthorizationError(
"builder attestation key is outside its validity window"
)
@classmethod
def from_dict(
cls, payload: Mapping[str, Any]
) -> "BuilderAttestationKeyRecord":
values = dict(payload)
encoded = values.pop("public_key", values.pop("public_key_b64", None))
if not isinstance(encoded, str):
raise ValueError("builder public key is missing")
try:
public_key = base64.b64decode(encoded, validate=True)
except (TypeError, ValueError) as exc:
raise ValueError("builder public key encoding is invalid") from exc
supplied_digest = values.pop(
"public_key_sha256",
values.pop("key_sha256", values.pop("digest", None)),
)
if not isinstance(supplied_digest, str):
raise ValueError("builder public key digest is missing")
digest = _validate_sha("builder public key digest", supplied_digest)
if not hmac.compare_digest(
digest, hashlib.sha256(public_key).hexdigest()
):
raise ValueError("builder public key digest is invalid")
allowed = {
"algorithm",
"purpose",
"key_id",
"generation",
"not_before",
"expires_at",
"status",
}
unknown = set(values).difference(allowed)
if unknown:
raise ValueError("builder key record contains unknown fields")
return cls(public_key=public_key, **values)
class BuilderAttestationKeyRing:
"""Validated, public-only keyring for builder attestations.
The file format deliberately cannot carry Owner approval keys: both the
keyring and every record must declare the builder-attestation purpose.
"""
def __init__(
self,
records: Iterable[BuilderAttestationKeyRecord],
*,
require_attestation_metadata: bool = False,
):
self._records: dict[str, BuilderAttestationKeyRecord] = {}
self._require_attestation_metadata = require_attestation_metadata
for record in records:
if record.key_id in self._records:
raise ValueError("duplicate builder key id")
self._records[record.key_id] = record
@classmethod
def from_public_keys(
cls, public_keys: Mapping[str, bytes]
) -> "BuilderAttestationKeyRing":
"""Compatibility adapter for explicit in-process builder key maps.
This is intentionally not used for host configuration; host keyrings
must use :meth:`from_file` so purpose, digest, validity and status are
checked before the API accepts traffic.
"""
return cls(
BuilderAttestationKeyRecord(
key_id=_validate_identifier("builder_key_id", key_id),
public_key=bytes(public_key),
algorithm="Ed25519",
purpose=BUILDER_ATTESTATION_PURPOSE,
generation=0,
not_before="1970-01-01T00:00:00Z",
expires_at="9999-12-31T23:59:59Z",
status="ACTIVE",
)
for key_id, public_key in public_keys.items()
)
@classmethod
def from_file(cls, path: Path) -> "BuilderAttestationKeyRing":
payload = json.loads(Path(path).read_text(encoding="utf-8"))
if payload.get("algorithm") != "Ed25519":
raise ValueError("unsupported builder keyring algorithm")
if payload.get("purpose") != BUILDER_ATTESTATION_PURPOSE:
raise ValueError("builder keyring purpose is invalid")
raw_records = payload.get("keys")
if not isinstance(raw_records, list) or not raw_records:
raise ValueError("builder keyring keys are invalid")
allowed = {"algorithm", "purpose", "keys", "keyring_sha256"}
if set(payload).difference(allowed):
raise ValueError("builder keyring contains unknown fields")
if "keyring_sha256" not in payload:
raise ValueError("builder keyring digest is missing")
expected = _validate_sha(
"builder keyring digest", str(payload["keyring_sha256"])
)
canonical = _canonical_json(
{
"algorithm": payload["algorithm"],
"purpose": payload["purpose"],
"keys": raw_records,
}
)
if not hmac.compare_digest(
expected, hashlib.sha256(canonical).hexdigest()
):
raise ValueError("builder keyring digest is invalid")
if not all(isinstance(record, Mapping) for record in raw_records):
raise ValueError("builder keyring record is invalid")
return cls(
(
BuilderAttestationKeyRecord.from_dict(record)
for record in raw_records
),
require_attestation_metadata=True,
)
def verify(
self,
key_id: str,
signature: bytes,
message: bytes,
*,
at: datetime,
generation: int | None,
algorithm: str | None,
purpose: str | None,
) -> BuilderAttestationKeyRecord:
record = self._records.get(key_id)
if record is None:
raise CapabilityAuthorizationError(
"builder attestation key is not trusted"
)
record.validate_at(at)
metadata = (generation, algorithm, purpose)
if self._require_attestation_metadata and any(
value is None for value in metadata
):
raise CapabilityAuthorizationError(
"builder attestation signing metadata is required"
)
if any(value is not None for value in metadata) and metadata != (
record.generation,
record.algorithm,
record.purpose,
):
raise CapabilityAuthorizationError(
"builder attestation signing metadata does not match key"
)
try:
Ed25519PublicKey.from_public_bytes(record.public_key).verify(
signature, message
)
except InvalidSignature as exc:
raise CapabilityAuthorizationError(
"builder attestation signature is invalid"
) from exc
return record
def require_active_identity(
self,
*,
key_id: str,
generation: int,
algorithm: str,
purpose: str,
at: datetime | None = None,
) -> BuilderAttestationKeyRecord:
record = self._records.get(key_id)
if record is None:
raise CapabilityAuthorizationError(
"builder attestation key is not trusted"
)
if (
record.generation != generation
or record.algorithm != algorithm
or record.purpose != purpose
):
raise CapabilityAuthorizationError(
"builder attestation key metadata is invalid"
)
record.validate_at(at or datetime.now(timezone.utc))
return record
class BuilderAttestationVerifier:
"""Public-key-only verifier; no builder private material is accepted."""
def __init__(self, public_keys: Mapping[str, bytes]):
self.public_keys = {
_validate_identifier("builder_key_id", key_id): bytes(value)
for key_id, value in public_keys.items()
}
def __init__(
self,
public_keys: BuilderAttestationKeyRing | Mapping[str, bytes],
):
if isinstance(public_keys, BuilderAttestationKeyRing):
self.keyring = public_keys
elif isinstance(public_keys, Mapping):
self.keyring = BuilderAttestationKeyRing.from_public_keys(
public_keys
)
else:
raise TypeError("builder verifier requires a builder keyring")
def verify(
self,
@@ -793,22 +1110,54 @@ class BuilderAttestationVerifier:
raise CapabilityAuthorizationError(
"builder attestation does not bind candidate evidence"
)
key = self.public_keys.get(attestation.builder_key_id)
if key is None:
raise CapabilityAuthorizationError(
"builder attestation key is not trusted"
)
try:
signature = base64.b64decode(
attestation.signature, validate=True
)
Ed25519PublicKey.from_public_bytes(key).verify(
signature, attestation.canonical_bytes()
key_record = self.keyring.verify(
attestation.builder_key_id,
signature,
attestation.canonical_bytes(),
at=_parse_utc(attestation.issued_at),
generation=attestation.builder_key_generation,
algorithm=attestation.signature_algorithm,
purpose=attestation.signature_purpose,
)
except (InvalidSignature, ValueError, TypeError) as exc:
raise CapabilityAuthorizationError(
"builder attestation signature is invalid"
) from exc
if key_record.generation >= 1:
expected_gitea = hashlib.sha256(
_canonical_json(candidate.gitea_consistency)
).hexdigest()
expected_rollback = hashlib.sha256(
_canonical_json(
{
"ready": candidate.rollback_ready,
"rollback_sha256": release_plan.deployment.rollback_sha256,
"source_commit": candidate.source_commit,
}
)
).hexdigest()
production_evidence = (
attestation.gitea_evidence_sha256,
attestation.rollback_evidence_sha256,
attestation.policy_evidence_sha256,
attestation.secret_scan_evidence_sha256,
)
if any(not value for value in production_evidence):
raise CapabilityAuthorizationError(
"builder attestation production evidence is incomplete"
)
if (
attestation.gitea_evidence_sha256 != expected_gitea
or attestation.rollback_evidence_sha256 != expected_rollback
or attestation.policy_evidence_sha256 != release_plan.policy_sha256
):
raise CapabilityAuthorizationError(
"builder attestation production evidence is invalid"
)
return attestation
@@ -3181,9 +3530,12 @@ class SilentReleaseActionService:
builder_attestation_verifier: BuilderAttestationVerifier | None = None,
trusted_policy: TrustedReleasePolicy | None = None,
orchestrator: ReleaseOrchestrator | None = None,
builder_readiness_probe: Callable[[], Mapping[str, Any]] | None = None,
allow_legacy_candidate_input: bool = False,
expected_repository: str = "",
policy_sha256: str = "",
max_builder_attempts: int = 3,
builder_attempt_timeout_seconds: int = 180,
):
self.root = Path(root)
self.execution_enabled = bool(execution_enabled)
@@ -3194,11 +3546,28 @@ class SilentReleaseActionService:
self.builder_attestation_verifier = builder_attestation_verifier
self.trusted_policy = trusted_policy
self.orchestrator = orchestrator
self.builder_readiness_probe = builder_readiness_probe
self.allow_legacy_candidate_input = bool(
allow_legacy_candidate_input
)
self.expected_repository = expected_repository
self.policy_sha256 = policy_sha256
if (
not isinstance(max_builder_attempts, int)
or isinstance(max_builder_attempts, bool)
or not 1 <= max_builder_attempts <= 10
):
raise ValueError("max_builder_attempts must be between 1 and 10")
self.max_builder_attempts = max_builder_attempts
if (
not isinstance(builder_attempt_timeout_seconds, int)
or isinstance(builder_attempt_timeout_seconds, bool)
or not 1 <= builder_attempt_timeout_seconds <= 3600
):
raise ValueError(
"builder_attempt_timeout_seconds must be between 1 and 3600"
)
self.builder_attempt_timeout_seconds = builder_attempt_timeout_seconds
self.candidates = ReleaseCandidateStore(
self.root / "candidates"
)
@@ -3235,10 +3604,26 @@ class SilentReleaseActionService:
+ ", ".join(missing)
)
def get_builder_readiness(self) -> dict[str, Any]:
"""Report dependency readiness without conflating it with API liveness."""
if self.candidate_builder is None:
return {"status": "UNAVAILABLE"}
if self.builder_readiness_probe is None:
# Explicit in-process test/adaptor construction remains usable;
# production wiring provides a probe backed by its credentials.
return {"status": "READY"}
try:
value = dict(self.builder_readiness_probe())
except Exception:
return {"status": "UNAVAILABLE"}
if value.get("status") != "READY":
return {"status": "UNAVAILABLE"}
return _sanitize(value)
def create_release_intent(
self, payload: Mapping[str, Any]
) -> dict[str, Any]:
allowed = {"event_id", "source_commit"}
allowed = {"event_id", "source_commit", "version"}
unknown = set(payload).difference(allowed)
if unknown:
raise CapabilityAuthorizationError(
@@ -3250,12 +3635,17 @@ class SilentReleaseActionService:
source_commit = _validate_oid(
"source_commit", str(payload.get("source_commit") or "")
)
version = str(payload.get("version") or "")
if version:
_validate_identifier("version", version)
intent_id = "intent-" + hashlib.sha256(
f"{event_id}\0{source_commit}".encode("utf-8")
).hexdigest()[:24]
path = _safe_child(self.intents_root, intent_id)
was_existing = path.exists()
record = {
"created_at": utc_now(),
"attempt_count": 0,
"event_id": event_id,
"intent_id": intent_id,
"source_commit": source_commit,
@@ -3265,42 +3655,153 @@ class SilentReleaseActionService:
else "BUILDER_UNAVAILABLE"
),
}
if path.exists():
if version:
record["version"] = version
if was_existing:
existing = json.loads(path.read_text(encoding="utf-8"))
if (
existing.get("event_id") != event_id
or existing.get("source_commit") != source_commit
or str(existing.get("version") or "") != version
):
raise SagaConflict(
"release intent id is bound to different evidence"
)
return {
**_sanitize(existing),
"idempotent": True,
"execution_enabled": self.execution_enabled,
"shadow_mode": not self.execution_enabled,
"side_effects_executed": False,
if existing.get("status") == "BUILDING":
try:
started_at = _parse_utc(
str(existing.get("attempt_started_at") or "")
)
elapsed = (
datetime.now(timezone.utc) - started_at
).total_seconds()
except (TypeError, ValueError):
elapsed = self.builder_attempt_timeout_seconds + 1
if elapsed <= self.builder_attempt_timeout_seconds:
return self._intent_response(existing, idempotent=True)
existing.update(
{
"failure_code": "TRUSTED_BUILDER_CRASH_RECOVERED",
"last_recovery_code": "TRUSTED_BUILDER_CRASH_RECOVERED",
"recovery_count": int(existing.get("recovery_count") or 0) + 1,
"retryable": int(existing.get("attempt_count") or 0)
< self.max_builder_attempts,
"status": "BUILDER_RETRYABLE_FAILURE",
"updated_at": utc_now(),
}
)
_durable_json(path, existing)
if (
existing.get("status") != "BUILDER_RETRYABLE_FAILURE"
or int(existing.get("attempt_count") or 0)
>= self.max_builder_attempts
):
return self._intent_response(existing, idempotent=True)
record = dict(existing)
if self.candidate_builder is None:
record.update(
{
"retryable": False,
"status": "BUILDER_UNAVAILABLE",
}
)
_durable_json(path, record)
return self._intent_response(record, idempotent=was_existing)
record.update(
{
"attempt_count": int(record.get("attempt_count") or 0) + 1,
"attempt_started_at": utc_now(),
"retryable": False,
"status": "BUILDING",
"updated_at": utc_now(),
}
)
_durable_json(path, record)
result = dict(record)
if self.candidate_builder is not None:
try:
candidate = self.create_release_candidate(
{
"event_id": event_id,
"intent_id": intent_id,
"source_commit": source_commit,
**({"version": version} if version else {}),
}
)
result.update(
except TrustedCandidateBuilderUnavailable:
record.update(
{
"candidate_id": candidate["candidate_id"],
"status": candidate["saga_state"],
"failure_code": "TRUSTED_BUILDER_UNAVAILABLE",
"retryable": record["attempt_count"]
< self.max_builder_attempts,
"status": (
"BUILDER_RETRYABLE_FAILURE"
if record["attempt_count"] < self.max_builder_attempts
else "BUILDER_TERMINAL_FAILURE"
),
}
)
_durable_json(path, result)
_durable_json(path, record)
return self._intent_response(record, idempotent=False)
except CapabilityAuthorizationError:
record.update(
{
"failure_code": "TRUSTED_BUILDER_REJECTED",
"retryable": False,
"status": "BUILDER_TERMINAL_FAILURE",
}
)
_durable_json(path, record)
return self._intent_response(record, idempotent=False)
except (ConnectionError, OSError, TimeoutError):
record.update(
{
"failure_code": "TRUSTED_BUILDER_UNAVAILABLE",
"retryable": record["attempt_count"]
< self.max_builder_attempts,
"status": (
"BUILDER_RETRYABLE_FAILURE"
if record["attempt_count"] < self.max_builder_attempts
else "BUILDER_TERMINAL_FAILURE"
),
}
)
_durable_json(path, record)
return self._intent_response(record, idempotent=False)
except Exception:
# The durable intent is recoverable without serializing an
# arbitrary exception (which may contain a credential or host
# path). Repeated requests consume the small fixed retry budget.
record.update(
{
"failure_code": "TRUSTED_BUILDER_FAILURE",
"retryable": record["attempt_count"]
< self.max_builder_attempts,
"status": (
"BUILDER_RETRYABLE_FAILURE"
if record["attempt_count"] < self.max_builder_attempts
else "BUILDER_TERMINAL_FAILURE"
),
}
)
_durable_json(path, record)
return self._intent_response(record, idempotent=False)
result = dict(record)
result.update(
{
"candidate_id": candidate["candidate_id"],
"retryable": False,
"status": candidate["saga_state"],
}
)
_durable_json(path, result)
return self._intent_response(result, idempotent=False)
def _intent_response(
self, record: Mapping[str, Any], *, idempotent: bool
) -> dict[str, Any]:
return {
**_sanitize(result),
"idempotent": False,
**_sanitize(dict(record)),
"idempotent": idempotent,
"execution_enabled": self.execution_enabled,
"shadow_mode": not self.execution_enabled,
"side_effects_executed": False,
@@ -3321,6 +3822,7 @@ class SilentReleaseActionService:
self, payload: Mapping[str, Any]
) -> dict[str, Any]:
if self.candidate_builder is not None:
values = dict(payload)
forbidden_evidence = {
"artifact_sha256",
"canary",
@@ -3333,13 +3835,69 @@ class SilentReleaseActionService:
"tests",
"tree_oid",
}
if forbidden_evidence.intersection(payload):
if forbidden_evidence.intersection(values):
raise CapabilityAuthorizationError(
"candidate evidence must come from the trusted builder"
)
worker_fields = {"candidate_id", "source_commit", "version"}
intent_fields = {"event_id", "intent_id", "source_commit", "version"}
supplied = set(values)
is_worker_request = worker_fields.issubset(supplied)
if is_worker_request:
if supplied != worker_fields:
raise ValueError("release worker request contains unknown fields")
_validate_identifier(
"candidate_id", str(values.get("candidate_id") or "")
)
_validate_identifier(
"version", str(values.get("version") or "")
)
else:
if not supplied.issubset(intent_fields):
raise ValueError("trusted builder intent contains unknown fields")
# Preserve the established internal intent contract
# (source_commit + version) while allowing newer callers to
# add an event/intent identifier. Neither form can supply
# acceptance evidence; the host builder owns all collectors.
if values.get("event_id"):
_validate_identifier(
"event_id", str(values["event_id"])
)
if values.get("intent_id"):
_validate_identifier(
"intent_id", str(values["intent_id"])
)
if values.get("version"):
_validate_identifier(
"version", str(values["version"])
)
_validate_oid(
"source_commit", str(values.get("source_commit") or "")
)
readiness = self.get_builder_readiness()
if readiness.get("status") != "READY":
raise TrustedCandidateBuilderUnavailable(
"trusted builder dependency is unavailable"
)
bundle = self.candidate_builder.build(dict(payload)).validate(
self.builder_attestation_verifier
)
requested_source = str(payload.get("source_commit") or "")
if (
requested_source
and bundle.candidate.source_commit != requested_source
):
raise CapabilityAuthorizationError(
"trusted builder source commit does not match intent"
)
if (
is_worker_request
and bundle.candidate.candidate_id
!= str(values["candidate_id"])
):
raise CapabilityAuthorizationError(
"trusted builder candidate id does not match request"
)
if self.trusted_policy is None:
raise CapabilityAuthorizationError(
"trusted release policy is not configured"
@@ -3457,6 +4015,19 @@ class SilentReleaseActionService:
}
def get_release_saga(self, candidate_id: str) -> dict[str, Any]:
# A read of a never-created candidate is a normal empty-state. Do
# not turn a missing journal into a durable manual-review snapshot.
saga_path = self.sagas._path(candidate_id)
journal_path = self.sagas._journal(candidate_id)
if not saga_path.exists() and not journal_path.exists():
return {
"candidate_id": candidate_id,
"status": "NOT_FOUND",
"state": "NOT_FOUND",
"execution_enabled": self.execution_enabled,
"shadow_mode": not self.execution_enabled,
"side_effects_executed": False,
}
return {
**self.sagas.load(candidate_id).to_dict(),
"execution_enabled": self.execution_enabled,
+26 -56
View File
@@ -34,7 +34,6 @@ from .models import (
)
from .policy import PolicyEngine
from .store import TaskPlaneStore
from .scope import bind_task_scope
from .worker_registry import WorkerRegistry
@@ -69,54 +68,28 @@ class TaskPlaneService:
self.delegation = DelegationEngine(store)
self.store.probe_attestation_verifier = self.policy.probe_attestation_verifier
def _authorized_task(self, task_id: str, actor: str, action: str) -> TaskRecord:
task = self.store.get_task(task_id, self.policy.actor_context(actor))
self.policy.assert_allowed(action, actor, task=task)
return task
def create_task(
self,
body: dict[str, Any],
idempotency_key: str | None = None,
*,
actor_identity: str | None = None,
) -> dict[str, Any]:
def create_task(self, body: dict[str, Any], idempotency_key: str | None = None) -> dict[str, Any]:
if body.get("parent_task_id"):
raise ForbiddenError("child tasks must use the delegate endpoint")
issuer = body.get("issuer", {})
declared_actor = issuer.get("id", "anon") if isinstance(issuer, dict) else "anon"
actor = actor_identity or declared_actor
actor = issuer.get("id", "anon") if isinstance(issuer, dict) else "anon"
self.policy.assert_allowed("create", actor)
try:
plan = TaskPlan.from_dict(body)
except (TypeError, ValueError) as exc:
raise ValidationError("invalid task plan") from exc
if actor_identity is not None:
plan = bind_task_scope(
plan, self.policy.actor_context(actor_identity), public_create=True
)
task = self.store.create_task_with_event(plan, idempotency_key=idempotency_key)
return task.to_dict()
def list_tasks(self, actor: str | None = None) -> list[dict[str, Any]]:
context = None
if actor is not None:
self.policy.assert_allowed("read", actor)
context = self.policy.actor_context(actor)
return [
task.to_dict(include_task=False)
for task in self.store.list_tasks(context)
]
return [task.to_dict(include_task=False) for task in self.store.list_tasks()]
def get_task(self, task_id: str, actor: str | None = None) -> dict[str, Any]:
context = None
if actor is not None:
self.policy.assert_allowed("read", actor)
context = self.policy.actor_context(actor)
task = self.store.get_task(task_id, context)
if actor is not None:
self.policy.assert_allowed("read", actor, task=task)
return task.to_dict()
self.policy.assert_allowed("read", actor, task=self.store.get_task(task_id))
return self.store.get_task(task_id).to_dict()
def get_worker(self, worker_id: str, actor: str | None = None) -> dict[str, Any]:
if actor is not None:
@@ -190,8 +163,9 @@ class TaskPlaneService:
if not worker_id:
raise ValidationError("worker_id required")
task = self._authorized_task(task_id, actor, "claim")
task = self.store.get_task(task_id)
self.policy.assert_readonly_worker_binding(actor, str(worker_id))
self.policy.assert_allowed("claim", actor, task=task)
lease_seconds = _bounded_int(
body.get("lease_seconds", 30), "lease_seconds", 1, 3600
)
@@ -210,9 +184,10 @@ class TaskPlaneService:
if not claim_id:
raise ValidationError("claim_id required")
actor = body.get("actor_identity") or body.get("worker_identity") or "worker"
task = self._authorized_task(task_id, actor, "heartbeat")
task = self.store.get_task(task_id)
claim = self.store.get_claim(claim_id)
self.policy.assert_readonly_worker_binding(actor, claim.worker_id)
self.policy.assert_allowed("heartbeat", actor, task=task)
if claim.worker_identity != actor:
raise ForbiddenError("claim identity mismatch")
lease_seconds = _bounded_int(
@@ -231,9 +206,10 @@ class TaskPlaneService:
message = body.get("message")
if progress is None:
raise ValidationError("progress required")
task = self._authorized_task(task_id, actor, "progress")
task = self.store.get_task(task_id)
claim = self.store.get_claim(claim_id)
self.policy.assert_readonly_worker_binding(actor, claim.worker_id)
self.policy.assert_allowed("progress", actor, task=task)
if task.task_plan.data_classification == "credential-secret":
if message not in {None, ""} or contains_secret_text(stage):
raise ValidationError(
@@ -258,7 +234,7 @@ class TaskPlaneService:
if not claim_id:
raise ValidationError("claim_id required")
actor = body.get("actor_identity") or body.get("worker_identity") or "worker"
task = self._authorized_task(task_id, actor, "result")
task = self.store.get_task(task_id)
status = body.get("status", "COMPLETED")
outputs = body.get("outputs", [])
@@ -271,6 +247,7 @@ class TaskPlaneService:
raise ValidationError("evidence items must be objects")
claim = self.store.get_claim(claim_id)
self.policy.assert_readonly_worker_binding(actor, claim.worker_id)
self.policy.assert_allowed("result", actor, task=task)
if claim.worker_identity != actor:
raise ForbiddenError("claim identity mismatch")
if task.task_plan.data_classification == "credential-secret":
@@ -349,7 +326,8 @@ class TaskPlaneService:
reason = body.get("reason", "failed")
actor = body.get("actor_identity") or body.get("worker_identity") or "worker"
self.policy.assert_allowed("fail", actor)
task = self._authorized_task(task_id, actor, "fail")
task = self.store.get_task(task_id)
self.policy.assert_allowed("fail", actor, task=task)
claim = self.store.get_claim(claim_id)
if claim.worker_identity != actor:
raise ForbiddenError("claim identity mismatch")
@@ -361,7 +339,8 @@ class TaskPlaneService:
actor = body.get("actor_identity") or body.get("worker_identity") or "owner"
self.policy.assert_allowed("cancel", actor)
reason = body.get("reason", "cancelled")
task = self._authorized_task(task_id, actor, "cancel")
task = self.store.get_task(task_id)
self.policy.assert_allowed("cancel", actor, task=task)
updated = self.store.cancel_task_with_event(
task_id, task.revision, reason=reason
)
@@ -370,7 +349,8 @@ class TaskPlaneService:
def delegate(self, task_id: str, body: dict[str, Any]) -> dict[str, Any]:
actor = body.get("actor_identity") or "owner"
self.policy.assert_allowed("delegate", actor)
parent = self._authorized_task(task_id, actor, "delegate")
parent = self.store.get_task(task_id)
self.policy.assert_allowed("delegate", actor, task=parent)
if "task" in body:
try:
child_plan = TaskPlan.from_dict(body["task"])
@@ -384,31 +364,25 @@ class TaskPlaneService:
except (TypeError, ValueError) as exc:
raise ValidationError("invalid child task plan") from exc
child_plan.parent_task_id = task_id
child_plan = bind_task_scope(
child_plan, self.policy.actor_context(actor), public_create=False
)
child = self.delegation.delegate(task_id, child_plan, caller=actor)
return child.to_dict()
def get_events(self, task_id: str, actor: str | None = None) -> list[dict[str, Any]]:
if actor is not None:
self._authorized_task(task_id, actor, "audit")
else:
self.store.get_task(task_id)
self.policy.assert_allowed("audit", actor, task=self.store.get_task(task_id))
self.store.get_task(task_id)
return self.store.list_events(task_id)
def get_evidence(self, task_id: str, actor: str | None = None) -> list[dict[str, Any]]:
if actor is not None:
self._authorized_task(task_id, actor, "audit")
else:
self.store.get_task(task_id)
self.policy.assert_allowed("audit", actor, task=self.store.get_task(task_id))
self.store.get_task(task_id)
return self.evidence.list(task_id)
def get_children(self, task_id: str, actor: str | None = None) -> list[str]:
if actor is not None:
self._authorized_task(task_id, actor, "read")
else:
self.store.get_task(task_id)
self.policy.assert_allowed("read", actor, task=self.store.get_task(task_id))
self.store.get_task(task_id)
return self.store.list_children(task_id)
@@ -532,11 +506,7 @@ class TaskPlaneRequestHandler(BaseHTTPRequestHandler):
):
raise ForbiddenError("issuer identity mismatch")
body["issuer"] = {"type": "authenticated", "id": identity}
return 201, self.service.create_task(
body or {},
headers.get("idempotency-key"),
actor_identity=identity,
)
return 201, self.service.create_task(body or {}, headers.get("idempotency-key"))
if body.get("actor_identity") not in {None, identity}:
raise ForbiddenError("actor identity mismatch")
body["actor_identity"] = identity
-5
View File
@@ -106,8 +106,3 @@ class DurableOutboxDispatcher:
receipt=str(receipt) if receipt else None,
error=None if receipt else f"{adapter.name} receipt unavailable",
)
# R4.2 P0-B compatibility adapter.
def to_handoff_envelope(event, *, target, sequence=1, idempotency_key=None, **scope):
from message_handoff_adapters import taskplane_event_to_envelope
return taskplane_event_to_envelope(event, target=target, sequence=sequence, idempotency_key=idempotency_key, **scope)
-911
View File
@@ -1,911 +0,0 @@
"""Durable multi-agent fleet scheduling for TaskPlane.
The fleet layer decides *which* eligible worker should receive *which* queued task.
It never replaces TaskPlane claim/lease ownership: callers must still perform the
authoritative TaskPlane claim CAS after a reservation is returned.
"""
from __future__ import annotations
from contextlib import contextmanager
from dataclasses import dataclass
import hashlib
import json
import math
import sqlite3
import threading
import time
from typing import Any, Iterable, Mapping, Sequence
DEFAULT_RESERVATION_SECONDS = 30
DEFAULT_CIRCUIT_THRESHOLD = 3
DEFAULT_CIRCUIT_SECONDS = 30
MAX_CIRCUIT_SECONDS = 15 * 60
MAX_SLOTS = 128
class FleetError(RuntimeError):
pass
class FleetConflict(FleetError):
pass
def _canonical(value: object) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def _sha(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _text(value: object, default: str = "") -> str:
if value is None:
return default
return str(value).strip() or default
def _int(value: object, default: int, minimum: int, maximum: int) -> int:
if value is None:
return default
if isinstance(value, bool):
raise FleetError("FLEET_INTEGER_INVALID")
try:
result = int(value)
except (TypeError, ValueError) as exc:
raise FleetError("FLEET_INTEGER_INVALID") from exc
if result < minimum or result > maximum:
raise FleetError("FLEET_INTEGER_OUT_OF_RANGE")
return result
def _float(value: object, default: float, minimum: float, maximum: float) -> float:
if value is None:
return default
try:
result = float(value)
except (TypeError, ValueError) as exc:
raise FleetError("FLEET_NUMBER_INVALID") from exc
if not math.isfinite(result) or result < minimum or result > maximum:
raise FleetError("FLEET_NUMBER_OUT_OF_RANGE")
return result
def _mapping(value: object) -> Mapping[str, Any]:
if isinstance(value, Mapping):
return value
if hasattr(value, "to_dict"):
data = value.to_dict()
if isinstance(data, Mapping):
return data
if hasattr(value, "__dict__"):
return vars(value)
return {}
def _attr(value: object, name: str, default: Any = None) -> Any:
if isinstance(value, Mapping):
return value.get(name, default)
return getattr(value, name, default)
def _task_plan(task: object) -> object:
return _attr(task, "task_plan", _attr(task, "plan", task))
def _task_id(task: object) -> str:
return _text(_attr(task, "task_id", _attr(_task_plan(task), "task_id", "")))
def _task_status(task: object) -> str:
return _text(_attr(task, "status", "")).upper()
def _task_revision(task: object) -> int:
return _int(_attr(task, "revision", 0), 0, 0, 2**31 - 1)
def _task_created_at(task: object) -> float:
value = _attr(task, "created_at", _attr(task, "updated_at", 0.0))
try:
return float(value or 0.0)
except (TypeError, ValueError):
return 0.0
def _task_required_capabilities(task: object) -> frozenset[str]:
plan = _task_plan(task)
value = _attr(plan, "required_capabilities", ())
if not isinstance(value, Sequence) or isinstance(value, (str, bytes)):
return frozenset()
return frozenset(_text(item) for item in value if _text(item))
def _task_metadata(task: object) -> Mapping[str, Any]:
plan = _task_plan(task)
value = _attr(plan, "metadata", {})
return value if isinstance(value, Mapping) else {}
def _task_scope_key(task: object) -> str:
plan = _task_plan(task)
scope_type = _text(_attr(plan, "scope_type", "legacy"), "legacy")
tenant = _text(_attr(plan, "tenant_id", ""))
project = _text(_attr(plan, "project_id", ""))
user = _text(_attr(plan, "user_id", ""))
if scope_type == "tenant":
return "tenant:" + tenant + "|project:" + (project or "*")
if scope_type == "system":
return "system"
return "legacy:" + (user or "*")
def _worker_id(worker: object) -> str:
return _text(_attr(worker, "worker_id", ""))
def _worker_capabilities(worker: object) -> frozenset[str]:
value = _attr(worker, "capabilities", ())
if isinstance(value, Mapping):
value = value.keys()
if not isinstance(value, Iterable) or isinstance(value, (str, bytes)):
return frozenset()
return frozenset(_text(item) for item in value if _text(item))
@dataclass(frozen=True)
class FleetWorkerPolicy:
worker_id: str
max_slots: int = 1
failure_domain: str = "default"
weight: float = 1.0
labels: Mapping[str, str] | None = None
revision: int = 1
@dataclass(frozen=True)
class FleetDispatchDecision:
reservation_id: str
task_id: str
worker_id: str
task_revision: int
scope_key: str
failure_domain: str
reserved_until: float
score: tuple[float, float, float, str]
def to_dict(self) -> dict[str, Any]:
return {
"reservation_id": self.reservation_id,
"task_id": self.task_id,
"worker_id": self.worker_id,
"task_revision": self.task_revision,
"scope_key": self.scope_key,
"failure_domain": self.failure_domain,
"reserved_until": self.reserved_until,
"score": list(self.score),
}
class FleetStateStore:
"""Small durable fleet journal colocated in the existing TaskPlane SQLite database."""
def __init__(self, connection: sqlite3.Connection, lock: threading.RLock | None = None) -> None:
self.conn = connection
self.lock = lock or threading.RLock()
self._savepoint_counter = 0
self._ensure_schema()
@contextmanager
def _transaction(self):
with self.lock:
self._savepoint_counter += 1
name = f"fleet_{threading.get_ident()}_{self._savepoint_counter}"
self.conn.execute(f"SAVEPOINT {name}")
try:
yield
except Exception:
self.conn.execute(f"ROLLBACK TO {name}")
self.conn.execute(f"RELEASE {name}")
raise
else:
self.conn.execute(f"RELEASE {name}")
def _ensure_schema(self) -> None:
with self.lock:
self.conn.executescript(
"""
CREATE TABLE IF NOT EXISTS fleet_worker_capacity (
worker_id TEXT PRIMARY KEY,
max_slots INTEGER NOT NULL,
failure_domain TEXT NOT NULL,
weight REAL NOT NULL,
labels_json TEXT NOT NULL,
revision INTEGER NOT NULL,
updated_at REAL NOT NULL
);
CREATE TABLE IF NOT EXISTS fleet_scope_state (
scope_key TEXT PRIMARY KEY,
virtual_finish REAL NOT NULL,
dispatch_count INTEGER NOT NULL,
last_dispatched_at REAL,
revision INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS fleet_dispatches (
reservation_id TEXT PRIMARY KEY,
task_id TEXT NOT NULL UNIQUE,
worker_id TEXT NOT NULL,
task_revision INTEGER NOT NULL,
scope_key TEXT NOT NULL,
failure_domain TEXT NOT NULL,
state TEXT NOT NULL,
reserved_until REAL NOT NULL,
claim_id TEXT,
created_at REAL NOT NULL,
updated_at REAL NOT NULL
);
CREATE INDEX IF NOT EXISTS ix_fleet_dispatch_worker_state
ON fleet_dispatches(worker_id, state, reserved_until);
CREATE TABLE IF NOT EXISTS fleet_worker_failures (
worker_id TEXT PRIMARY KEY,
consecutive_failures INTEGER NOT NULL,
circuit_open_until REAL,
last_failure_class TEXT,
revision INTEGER NOT NULL,
updated_at REAL NOT NULL
);
CREATE TABLE IF NOT EXISTS fleet_task_worker_failures (
task_id TEXT NOT NULL,
worker_id TEXT NOT NULL,
failure_count INTEGER NOT NULL,
blocked_until REAL,
last_failure_class TEXT,
revision INTEGER NOT NULL,
updated_at REAL NOT NULL,
PRIMARY KEY(task_id, worker_id)
);
"""
)
def configure_worker(
self,
worker_id: str,
*,
max_slots: int = 1,
failure_domain: str = "default",
weight: float = 1.0,
labels: Mapping[str, str] | None = None,
expected_revision: int | None = None,
now: float | None = None,
) -> FleetWorkerPolicy:
worker_id = _text(worker_id)
if not worker_id:
raise FleetError("FLEET_WORKER_ID_REQUIRED")
max_slots = _int(max_slots, 1, 1, MAX_SLOTS)
weight = _float(weight, 1.0, 0.05, 100.0)
failure_domain = _text(failure_domain, "default")
normalized_labels = {
_text(k): _text(v)
for k, v in dict(labels or {}).items()
if _text(k) and _text(v)
}
now = float(time.time() if now is None else now)
with self._transaction():
row = self.conn.execute(
"SELECT revision FROM fleet_worker_capacity WHERE worker_id=?",
(worker_id,),
).fetchone()
current = int(row[0]) if row else 0
if expected_revision is not None and current != int(expected_revision):
raise FleetConflict("FLEET_WORKER_POLICY_CAS_MISMATCH")
revision = current + 1
self.conn.execute(
"""
INSERT INTO fleet_worker_capacity(
worker_id,max_slots,failure_domain,weight,labels_json,revision,updated_at
) VALUES (?,?,?,?,?,?,?)
ON CONFLICT(worker_id) DO UPDATE SET
max_slots=excluded.max_slots,
failure_domain=excluded.failure_domain,
weight=excluded.weight,
labels_json=excluded.labels_json,
revision=excluded.revision,
updated_at=excluded.updated_at
""",
(
worker_id,
max_slots,
failure_domain,
weight,
json.dumps(normalized_labels, sort_keys=True, separators=(",", ":")),
revision,
now,
),
)
return FleetWorkerPolicy(
worker_id, max_slots, failure_domain, weight, normalized_labels, revision
)
def worker_policy(self, worker_id: str) -> FleetWorkerPolicy:
worker_id = _text(worker_id)
with self.lock:
row = self.conn.execute(
"""
SELECT worker_id,max_slots,failure_domain,weight,labels_json,revision
FROM fleet_worker_capacity WHERE worker_id=?
""",
(worker_id,),
).fetchone()
if row is None:
return FleetWorkerPolicy(worker_id=worker_id)
return FleetWorkerPolicy(
worker_id=str(row[0]),
max_slots=int(row[1]),
failure_domain=str(row[2]),
weight=float(row[3]),
labels=json.loads(row[4] or "{}"),
revision=int(row[5]),
)
def _cleanup_expired(self, now: float) -> None:
self.conn.execute(
"""
UPDATE fleet_dispatches
SET state='EXPIRED', updated_at=?
WHERE state='RESERVED' AND reserved_until<=?
""",
(now, now),
)
def reserved_slots(self, worker_id: str, now: float) -> int:
with self.lock:
self._cleanup_expired(now)
row = self.conn.execute(
"""
SELECT COUNT(*) FROM fleet_dispatches
WHERE worker_id=? AND state='RESERVED' AND reserved_until>?
""",
(worker_id, now),
).fetchone()
return int(row[0] if row else 0)
def scope_virtual_finish(self, scope_key: str) -> float:
with self.lock:
row = self.conn.execute(
"SELECT virtual_finish FROM fleet_scope_state WHERE scope_key=?",
(scope_key,),
).fetchone()
return float(row[0]) if row else 0.0
def reserve(
self,
*,
task_id: str,
worker_id: str,
task_revision: int,
scope_key: str,
failure_domain: str,
score: tuple[float, float, float, str],
reservation_seconds: int,
now: float,
scope_weight: float,
) -> FleetDispatchDecision:
reservation_seconds = _int(
reservation_seconds, DEFAULT_RESERVATION_SECONDS, 1, 300
)
reservation_id = "fleet-" + _sha(
{
"task_id": task_id,
"worker_id": worker_id,
"task_revision": task_revision,
"scope_key": scope_key,
}
)[:32]
reserved_until = now + reservation_seconds
with self._transaction():
self._cleanup_expired(now)
existing = self.conn.execute(
"""
SELECT reservation_id,worker_id,task_revision,scope_key,failure_domain,
reserved_until,state
FROM fleet_dispatches WHERE task_id=?
""",
(task_id,),
).fetchone()
# Capacity is rechecked inside the same durable reservation transaction.
# This closes the cross-thread/process race between eligibility scoring and reserve.
policy_row = self.conn.execute(
"SELECT max_slots FROM fleet_worker_capacity WHERE worker_id=?",
(worker_id,),
).fetchone()
max_slots = int(policy_row[0]) if policy_row else 1
if existing and str(existing[6]) in {"RESERVED", "CLAIMED"} and float(existing[5]) > now:
if str(existing[1]) != worker_id or int(existing[2]) != task_revision:
raise FleetConflict("FLEET_TASK_ALREADY_RESERVED")
return FleetDispatchDecision(
str(existing[0]), task_id, worker_id, task_revision, str(existing[3]),
str(existing[4]), float(existing[5]), score
)
claim_columns = {
str(row[1])
for row in self.conn.execute("PRAGMA table_info(claims)").fetchall()
}
active_claims = 0
if {"worker_id", "status"}.issubset(claim_columns):
sql = "SELECT COUNT(*) FROM claims WHERE worker_id=? AND status='ACTIVE'"
params: list[Any] = [worker_id]
if "lease_expires_at" in claim_columns:
sql += " AND lease_expires_at>?"
params.append(now)
row = self.conn.execute(sql, tuple(params)).fetchone()
active_claims = int(row[0] if row else 0)
row = self.conn.execute(
"""
SELECT COUNT(*) FROM fleet_dispatches
WHERE worker_id=? AND task_id<>?
AND state='RESERVED' AND reserved_until>?
""",
(worker_id, task_id, now),
).fetchone()
active_reservations = int(row[0] if row else 0)
if active_claims + active_reservations >= max_slots:
raise FleetConflict("FLEET_WORKER_CAPACITY_EXHAUSTED")
self.conn.execute(
"""
INSERT INTO fleet_dispatches(
reservation_id,task_id,worker_id,task_revision,scope_key,failure_domain,
state,reserved_until,claim_id,created_at,updated_at
) VALUES (?,?,?,?,?,?,'RESERVED',?,NULL,?,?)
ON CONFLICT(task_id) DO UPDATE SET
reservation_id=excluded.reservation_id,
worker_id=excluded.worker_id,
task_revision=excluded.task_revision,
scope_key=excluded.scope_key,
failure_domain=excluded.failure_domain,
state='RESERVED',
reserved_until=excluded.reserved_until,
claim_id=NULL,
updated_at=excluded.updated_at
""",
(
reservation_id, task_id, worker_id, task_revision, scope_key,
failure_domain, reserved_until, now, now,
),
)
row = self.conn.execute(
"SELECT virtual_finish,dispatch_count,revision FROM fleet_scope_state WHERE scope_key=?",
(scope_key,),
).fetchone()
current_finish = float(row[0]) if row else 0.0
dispatch_count = int(row[1]) if row else 0
revision = int(row[2]) if row else 0
next_finish = max(current_finish, 0.0) + 1.0 / max(scope_weight, 0.05)
self.conn.execute(
"""
INSERT INTO fleet_scope_state(
scope_key,virtual_finish,dispatch_count,last_dispatched_at,revision
) VALUES (?,?,?,?,?)
ON CONFLICT(scope_key) DO UPDATE SET
virtual_finish=excluded.virtual_finish,
dispatch_count=excluded.dispatch_count,
last_dispatched_at=excluded.last_dispatched_at,
revision=excluded.revision
""",
(scope_key, next_finish, dispatch_count + 1, now, revision + 1),
)
return FleetDispatchDecision(
reservation_id, task_id, worker_id, task_revision, scope_key,
failure_domain, reserved_until, score
)
def bind_claim(self, reservation_id: str, claim_id: str, *, now: float | None = None) -> None:
now = float(time.time() if now is None else now)
with self._transaction():
row = self.conn.execute(
"SELECT state FROM fleet_dispatches WHERE reservation_id=?",
(reservation_id,),
).fetchone()
if row is None:
raise FleetError("FLEET_RESERVATION_NOT_FOUND")
if str(row[0]) not in {"RESERVED", "CLAIMED"}:
raise FleetConflict("FLEET_RESERVATION_NOT_ACTIVE")
self.conn.execute(
"""
UPDATE fleet_dispatches
SET state='CLAIMED', claim_id=?, updated_at=?
WHERE reservation_id=?
""",
(claim_id, now, reservation_id),
)
def release(
self,
task_id: str,
*,
state: str = "RELEASED",
now: float | None = None,
) -> None:
if state not in {"RELEASED", "COMPLETED", "FAILED", "EXPIRED"}:
raise FleetError("FLEET_RESERVATION_STATE_INVALID")
now = float(time.time() if now is None else now)
with self._transaction():
self.conn.execute(
"UPDATE fleet_dispatches SET state=?,updated_at=? WHERE task_id=?",
(state, now, task_id),
)
def worker_circuit_open(self, worker_id: str, now: float) -> bool:
with self.lock:
row = self.conn.execute(
"SELECT circuit_open_until FROM fleet_worker_failures WHERE worker_id=?",
(worker_id,),
).fetchone()
return bool(row and row[0] is not None and float(row[0]) > now)
def task_worker_blocked(self, task_id: str, worker_id: str, now: float) -> bool:
with self.lock:
row = self.conn.execute(
"""
SELECT blocked_until FROM fleet_task_worker_failures
WHERE task_id=? AND worker_id=?
""",
(task_id, worker_id),
).fetchone()
return bool(row and row[0] is not None and float(row[0]) > now)
def record_worker_failure(
self,
worker_id: str,
failure_class: str,
*,
threshold: int = DEFAULT_CIRCUIT_THRESHOLD,
base_circuit_seconds: int = DEFAULT_CIRCUIT_SECONDS,
now: float | None = None,
) -> dict[str, Any]:
now = float(time.time() if now is None else now)
threshold = _int(threshold, DEFAULT_CIRCUIT_THRESHOLD, 1, 20)
base_circuit_seconds = _int(base_circuit_seconds, DEFAULT_CIRCUIT_SECONDS, 1, 3600)
with self._transaction():
row = self.conn.execute(
"""
SELECT consecutive_failures,revision FROM fleet_worker_failures
WHERE worker_id=?
""",
(worker_id,),
).fetchone()
failures = (int(row[0]) if row else 0) + 1
revision = (int(row[1]) if row else 0) + 1
open_until = None
if failures >= threshold:
exponent = min(failures - threshold, 8)
open_until = now + min(
MAX_CIRCUIT_SECONDS, base_circuit_seconds * (2**exponent)
)
self.conn.execute(
"""
INSERT INTO fleet_worker_failures(
worker_id,consecutive_failures,circuit_open_until,last_failure_class,
revision,updated_at
) VALUES (?,?,?,?,?,?)
ON CONFLICT(worker_id) DO UPDATE SET
consecutive_failures=excluded.consecutive_failures,
circuit_open_until=excluded.circuit_open_until,
last_failure_class=excluded.last_failure_class,
revision=excluded.revision,
updated_at=excluded.updated_at
""",
(worker_id, failures, open_until, _text(failure_class, "unknown"), revision, now),
)
return {
"worker_id": worker_id,
"consecutive_failures": failures,
"circuit_open_until": open_until,
"revision": revision,
}
def record_worker_success(self, worker_id: str, *, now: float | None = None) -> None:
now = float(time.time() if now is None else now)
with self._transaction():
self.conn.execute(
"""
INSERT INTO fleet_worker_failures(
worker_id,consecutive_failures,circuit_open_until,last_failure_class,
revision,updated_at
) VALUES (?,0,NULL,NULL,1,?)
ON CONFLICT(worker_id) DO UPDATE SET
consecutive_failures=0,
circuit_open_until=NULL,
last_failure_class=NULL,
revision=revision+1,
updated_at=excluded.updated_at
""",
(worker_id, now),
)
def record_task_worker_failure(
self,
task_id: str,
worker_id: str,
failure_class: str,
*,
quarantine_seconds: int = 60,
now: float | None = None,
) -> dict[str, Any]:
now = float(time.time() if now is None else now)
quarantine_seconds = _int(quarantine_seconds, 60, 1, 3600)
with self._transaction():
row = self.conn.execute(
"""
SELECT failure_count,revision FROM fleet_task_worker_failures
WHERE task_id=? AND worker_id=?
""",
(task_id, worker_id),
).fetchone()
count = (int(row[0]) if row else 0) + 1
revision = (int(row[1]) if row else 0) + 1
blocked_until = now + quarantine_seconds
self.conn.execute(
"""
INSERT INTO fleet_task_worker_failures(
task_id,worker_id,failure_count,blocked_until,last_failure_class,
revision,updated_at
) VALUES (?,?,?,?,?,?,?)
ON CONFLICT(task_id,worker_id) DO UPDATE SET
failure_count=excluded.failure_count,
blocked_until=excluded.blocked_until,
last_failure_class=excluded.last_failure_class,
revision=excluded.revision,
updated_at=excluded.updated_at
""",
(
task_id, worker_id, count, blocked_until,
_text(failure_class, "unknown"), revision, now,
),
)
return {
"task_id": task_id,
"worker_id": worker_id,
"failure_count": count,
"blocked_until": blocked_until,
"revision": revision,
}
class FleetScheduler:
"""Deterministic capacity/fairness scheduler over existing TaskPlane records."""
def __init__(
self,
store: Any,
workers: Any,
*,
now_fn: Any = time.time,
reservation_seconds: int = DEFAULT_RESERVATION_SECONDS,
) -> None:
self.store = store
self.workers = workers
self.now_fn = now_fn
self.reservation_seconds = _int(
reservation_seconds, DEFAULT_RESERVATION_SECONDS, 1, 300
)
connection = getattr(store, "conn", None)
if not isinstance(connection, sqlite3.Connection):
raise FleetError("FLEET_TASKPLANE_SQLITE_CONNECTION_REQUIRED")
lock = getattr(store, "_lock", None)
self.state = FleetStateStore(connection, lock)
def configure_worker(self, worker_id: str, **kwargs: Any) -> FleetWorkerPolicy:
return self.state.configure_worker(worker_id, **kwargs)
def _online(self, worker_id: str, now: float) -> bool:
method = getattr(self.workers, "is_online", None)
if not callable(method):
raise FleetError("FLEET_WORKER_REGISTRY_ONLINE_CHECK_REQUIRED")
try:
return bool(method(worker_id, now=now))
except TypeError:
return bool(method(worker_id))
def _worker_record(self, worker_id: str) -> object:
getter = getattr(self.store, "get_worker", None)
if not callable(getter):
raise FleetError("FLEET_WORKER_LOOKUP_REQUIRED")
return getter(worker_id)
def _active_claims(self, worker_id: str, now: float) -> int:
# The TaskPlane store owns one SQLite connection in the current runtime.
# Fleet reads must share its lock just like fleet writes do, otherwise two
# concurrent pullers can issue overlapping operations on the same handle.
with self.state.lock:
columns = {
str(row[1])
for row in self.store.conn.execute("PRAGMA table_info(claims)").fetchall()
}
required = {"worker_id", "status"}
if not required.issubset(columns):
raise FleetError("FLEET_CLAIMS_SCHEMA_UNSUPPORTED")
sql = "SELECT COUNT(*) FROM claims WHERE worker_id=? AND status='ACTIVE'"
params: list[Any] = [worker_id]
if "lease_expires_at" in columns:
sql += " AND lease_expires_at>?"
params.append(now)
row = self.store.conn.execute(sql, tuple(params)).fetchone()
return int(row[0] if row else 0)
@staticmethod
def _labels_match(policy: FleetWorkerPolicy, task: object) -> bool:
metadata = _task_metadata(task)
affinity = metadata.get("fleet_affinity_labels") or {}
if affinity and not isinstance(affinity, Mapping):
return False
labels = dict(policy.labels or {})
for key, value in dict(affinity).items():
if labels.get(_text(key)) != _text(value):
return False
avoid = metadata.get("fleet_avoid_failure_domains") or []
if isinstance(avoid, Sequence) and not isinstance(avoid, (str, bytes)):
if policy.failure_domain in {_text(item) for item in avoid}:
return False
return True
def _eligible_worker(self, worker_id: str, task: object, now: float) -> tuple[bool, int, FleetWorkerPolicy]:
if not self._online(worker_id, now):
return False, 0, self.state.worker_policy(worker_id)
worker = self._worker_record(worker_id)
policy = self.state.worker_policy(worker_id)
if self.state.worker_circuit_open(worker_id, now):
return False, 0, policy
if self.state.task_worker_blocked(_task_id(task), worker_id, now):
return False, 0, policy
if not self._labels_match(policy, task):
return False, 0, policy
required = _task_required_capabilities(task)
capabilities = _worker_capabilities(worker)
if required and not required.issubset(capabilities):
return False, 0, policy
active = self._active_claims(worker_id, now)
reserved = self.state.reserved_slots(worker_id, now)
load = active + reserved
if load >= policy.max_slots:
return False, load, policy
return True, load, policy
@staticmethod
def _task_priority(task: object, now: float) -> float:
metadata = _task_metadata(task)
base = _int(metadata.get("fleet_priority"), 0, -100, 100)
created = _task_created_at(task)
waited = max(0.0, now - created) if created > 0 else 0.0
aging = min(100.0, waited / 60.0)
return float(base) + aging
def _task_score(self, task: object, now: float) -> tuple[float, float, float, str]:
scope = _task_scope_key(task)
virtual_finish = self.state.scope_virtual_finish(scope)
priority = self._task_priority(task, now)
return (virtual_finish, -priority, _task_created_at(task), _task_id(task))
def _tasks(self, actor_context: Any = None) -> list[object]:
lister = getattr(self.store, "list_tasks", None)
if not callable(lister):
raise FleetError("FLEET_TASK_LIST_REQUIRED")
try:
tasks = lister(actor_context)
except TypeError:
tasks = lister()
return [
task for task in tasks
if _task_status(task) in {"CREATED", "QUEUED"}
and _task_id(task)
]
def rank_for_worker(
self,
worker_id: str,
*,
actor_context: Any = None,
now: float | None = None,
) -> list[object]:
"""Return fleet-ranked tasks without reserving or claiming anything."""
now = float(self.now_fn() if now is None else now)
worker_id = _text(worker_id)
if not worker_id:
return []
ranked: list[object] = []
for task in self._tasks(actor_context):
eligible, _, _ = self._eligible_worker(worker_id, task, now)
if eligible:
ranked.append(task)
return sorted(ranked, key=lambda task: self._task_score(task, now))
def reserve_for_worker(
self,
worker_id: str,
task: object,
*,
now: float | None = None,
) -> FleetDispatchDecision:
"""Reserve exactly one already-ranked task, rechecking eligibility and capacity."""
now = float(self.now_fn() if now is None else now)
worker_id = _text(worker_id)
eligible, _, policy = self._eligible_worker(worker_id, task, now)
if not eligible:
raise FleetConflict("FLEET_WORKER_NOT_ELIGIBLE")
metadata = _task_metadata(task)
scope_weight = _float(metadata.get("fleet_scope_weight"), 1.0, 0.05, 100.0)
return self.state.reserve(
task_id=_task_id(task),
worker_id=worker_id,
task_revision=_task_revision(task),
scope_key=_task_scope_key(task),
failure_domain=policy.failure_domain,
score=self._task_score(task, now),
reservation_seconds=self.reservation_seconds,
now=now,
scope_weight=scope_weight,
)
def select(
self,
worker_ids: Sequence[str],
*,
actor_context: Any = None,
limit: int = 1,
now: float | None = None,
) -> list[FleetDispatchDecision]:
now = float(self.now_fn() if now is None else now)
limit = _int(limit, 1, 1, 100)
worker_ids = tuple(dict.fromkeys(_text(item) for item in worker_ids if _text(item)))
tasks = sorted(self._tasks(actor_context), key=lambda task: self._task_score(task, now))
decisions: list[FleetDispatchDecision] = []
local_load: dict[str, int] = {}
for task in tasks:
if len(decisions) >= limit:
break
candidates: list[tuple[int, float, str, FleetWorkerPolicy]] = []
for worker_id in worker_ids:
eligible, load, policy = self._eligible_worker(worker_id, task, now)
if not eligible:
continue
effective_load = local_load.get(worker_id, load)
if effective_load >= policy.max_slots:
continue
candidates.append(
(
effective_load,
-policy.weight,
worker_id,
policy,
)
)
if not candidates:
continue
candidates.sort(key=lambda item: (item[0], item[1], item[2]))
_, _, worker_id, policy = candidates[0]
score = self._task_score(task, now)
metadata = _task_metadata(task)
scope_weight = _float(metadata.get("fleet_scope_weight"), 1.0, 0.05, 100.0)
try:
decision = self.state.reserve(
task_id=_task_id(task),
worker_id=worker_id,
task_revision=_task_revision(task),
scope_key=_task_scope_key(task),
failure_domain=policy.failure_domain,
score=score,
reservation_seconds=self.reservation_seconds,
now=now,
scope_weight=scope_weight,
)
except FleetConflict:
continue
local_load[worker_id] = local_load.get(
worker_id,
self._active_claims(worker_id, now)
+ self.state.reserved_slots(worker_id, now)
- 1,
) + 1
decisions.append(decision)
return decisions
-27
View File
@@ -13,8 +13,6 @@ from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Any
from .scope import normalize_task_scope
SENSITIVE_KEYS = {
"secret",
@@ -68,12 +66,6 @@ TASK_PLAN_KEYS = {
"allowed_input_paths",
"allowed_output_paths",
"output_policy",
"scope_type",
"tenant_id",
"user_id",
"project_id",
"conversation_id",
"run_id",
}
@@ -245,12 +237,6 @@ class TaskPlan:
schema_version: int = 1
task_id: str | None = None
task_type: str = "code.change"
scope_type: str = "legacy"
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
run_id: str | None = None
issuer: dict[str, Any] = field(default_factory=dict)
intent: str | None = None
required_capabilities: list[str] = field(default_factory=list)
@@ -325,17 +311,10 @@ class TaskPlan:
raise ValueError("credential-secret requires metadata-only logging")
if output_policy != "hash-count-status-only":
raise ValueError("credential-secret requires restricted output")
scope = normalize_task_scope(data)
return cls(
schema_version=schema_version,
task_id=validate_identifier(data["task_id"], "task_id"),
task_type=validate_type(data.get("task_type", "code.change"), "task_type"),
scope_type=scope.scope_type,
tenant_id=scope.tenant_id,
user_id=scope.user_id,
project_id=scope.project_id,
conversation_id=scope.conversation_id,
run_id=scope.run_id,
issuer=issuer,
intent=data.get("intent"),
required_capabilities=_normalize_list(data.get("required_capabilities", [])),
@@ -369,12 +348,6 @@ class TaskPlan:
"schema_version": self.schema_version,
"task_id": self.task_id,
"task_type": self.task_type,
"scope_type": self.scope_type,
"tenant_id": self.tenant_id,
"user_id": self.user_id,
"project_id": self.project_id,
"conversation_id": self.conversation_id,
"run_id": self.run_id,
"issuer": self.issuer,
"intent": self.intent,
"required_capabilities": list(self.required_capabilities),
+1 -20
View File
@@ -3,11 +3,10 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Callable, Mapping
from typing import Callable
from .errors import ForbiddenError, LockedError
from .models import ProbeEvidence, ResultEnvelope, TaskRecord
from .scope import TaskActorContext, actor_context as build_actor_context
RISK_RANK = {"low": 1, "medium": 2, "high": 3}
@@ -94,10 +93,6 @@ class PolicyEngine:
result_signature_verifier: Callable[[ResultEnvelope], bool] | None = None,
probe_attestation_verifier: Callable[[ProbeEvidence], bool] | None = None,
identity_capabilities: dict[str, set[str]] | None = None,
identity_actor_contexts: Mapping[
str, TaskActorContext | Mapping[str, Any]
]
| None = None,
admin_identities: set[str] | None = None,
readonly_worker_execution_enabled: bool = False,
readonly_worker_profiles: list[ReadonlyWorkerProfile] | None = None,
@@ -114,26 +109,12 @@ class PolicyEngine:
identity: set(capabilities)
for identity, capabilities in (identity_capabilities or {}).items()
}
self.identity_actor_contexts = dict(identity_actor_contexts or {})
self.admin_identities = set(admin_identities or ())
self.readonly_worker_execution_enabled = readonly_worker_execution_enabled
self.readonly_worker_profiles = {
profile.identity: profile for profile in (readonly_worker_profiles or [])
}
def actor_context(self, identity: str) -> TaskActorContext:
configured = self.identity_actor_contexts.get(
identity, self.identity_actor_contexts.get("*")
)
capabilities = set(
self.identity_capabilities.get(
identity, self.identity_capabilities.get("*", set())
)
)
if identity in self.admin_identities:
capabilities.add("task.admin")
return build_actor_context(identity, configured, capabilities)
def _readonly_worker_allows(
self, action: str, actor: str, task: TaskRecord | None
) -> bool:
-200
View File
@@ -74,203 +74,3 @@ class TaskScheduler:
operation_digest=task.task_plan.operation_digest(),
)
return claim
# R4.2 P2 durable multi-agent fleet scheduling public surface.
from .fleet import (
FleetConflict,
FleetDispatchDecision,
FleetError,
FleetScheduler,
FleetStateStore,
FleetWorkerPolicy,
)
# R4.2 P2-B fleet-aware pull scheduling.
#
# The original TaskScheduler remains the security oracle and claim-CAS implementation.
# Fleet ordering only controls the order presented to that oracle.
import inspect as _fleet_inspect
import threading as _fleet_threading
from collections.abc import Mapping as _FleetMapping
from .worker_registry import WorkerRegistry as _FleetWorkerRegistry
from .errors import NotFoundError as _FleetNotFoundError
_LegacyTaskScheduler = TaskScheduler
_LegacyScheduleParameters = _fleet_inspect.signature(
_LegacyTaskScheduler.schedule_for_worker
).parameters
_LegacyNextParameters = _fleet_inspect.signature(
_LegacyTaskScheduler.next_queued_task
).parameters
class _FleetOrderedStoreProxy:
def __init__(self, delegate, tasks):
self._delegate = delegate
self._tasks = tuple(tasks)
def list_tasks(self, *args, **kwargs):
return list(self._tasks)
def __getattr__(self, name):
return getattr(self._delegate, name)
def _fleet_claim_field(claim, name, default=None):
if isinstance(claim, _FleetMapping):
return claim.get(name, default)
return getattr(claim, name, default)
class TaskScheduler(_LegacyTaskScheduler):
"""Fleet-integrated pull scheduler preserving legacy security and claim semantics."""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
store = getattr(self, "store", args[0] if args else None)
if store is None:
raise FleetError("FLEET_SCHEDULER_STORE_REQUIRED")
registry = getattr(self, "workers", None)
if registry is None:
registry = _FleetWorkerRegistry(store)
self.fleet_scheduler = FleetScheduler(store, registry)
self._fleet_local = _fleet_threading.local()
def _fleet_legacy_next(self, worker_id, tasks, *args, **kwargs):
proxy = _FleetOrderedStoreProxy(self.store, tasks)
legacy = _LegacyTaskScheduler(proxy)
legacy_kwargs = dict(kwargs)
if "actor_context" not in _LegacyNextParameters:
legacy_kwargs.pop("actor_context", None)
return legacy.next_queued_task(worker_id, *args, **legacy_kwargs)
def next_queued_task(self, worker_id, *args, **kwargs):
actor_context = kwargs.get(
"actor_context", getattr(self._fleet_local, "actor_context", None)
)
# Preserve legacy fail-closed worker checks before fleet ordering.
probe_kwargs = dict(kwargs)
if "actor_context" not in _LegacyNextParameters:
probe_kwargs.pop("actor_context", None)
legacy_probe = _LegacyTaskScheduler(self.store).next_queued_task(
worker_id, *args, **probe_kwargs
)
if legacy_probe is None:
return None
ranked = self.fleet_scheduler.rank_for_worker(
worker_id, actor_context=actor_context
)
# Preserve FleetScheduler fairness order. The legacy scheduler is used
# only as a fail-closed security oracle: present one fleet-ranked task
# at a time so its internal sort cannot reorder across scopes.
# Keep the legacy priority dimension ahead of Fleet fairness for
# compatibility with existing TaskPlane scheduling contracts; the
# Fleet order remains authoritative among equal-priority tasks.
def _legacy_priority(task):
plan = getattr(task, "task_plan", task)
metadata = getattr(plan, "metadata", {}) or {}
if not isinstance(metadata, _FleetMapping):
return 0
try:
return int(metadata.get("priority", 0))
except (TypeError, ValueError):
return 0
ranked = sorted(ranked, key=lambda task: -_legacy_priority(task))
for ranked_candidate in ranked:
try:
candidate = self._fleet_legacy_next(
worker_id, [ranked_candidate], *args, **kwargs
)
except _FleetNotFoundError:
continue
if candidate is None:
continue
ranked_id = getattr(
ranked_candidate,
"task_id",
getattr(getattr(ranked_candidate, "task_plan", None), "task_id", None),
)
candidate_id = getattr(
candidate,
"task_id",
getattr(getattr(candidate, "task_plan", None), "task_id", None),
)
if candidate_id != ranked_id:
raise FleetConflict("FLEET_LEGACY_ORACLE_ORDER_VIOLATION")
try:
decision = self.fleet_scheduler.reserve_for_worker(
worker_id, candidate
)
except FleetConflict:
continue
self._fleet_local.pending = decision
return candidate
return None
def schedule_for_worker(self, worker_id, *args, **kwargs):
actor_context = kwargs.get("actor_context")
legacy_kwargs = dict(kwargs)
if "actor_context" not in _LegacyScheduleParameters:
legacy_kwargs.pop("actor_context", None)
self._fleet_local.actor_context = actor_context
self._fleet_local.pending = None
try:
claim = _LegacyTaskScheduler.schedule_for_worker(
self, worker_id, *args, **legacy_kwargs
)
decision = getattr(self._fleet_local, "pending", None)
if claim is None:
if decision is not None:
self.fleet_scheduler.state.release(
decision.task_id, state="RELEASED"
)
return None
if decision is not None:
claim_id = _fleet_claim_field(claim, "claim_id")
claim_task_id = _fleet_claim_field(claim, "task_id", decision.task_id)
if not claim_id:
self.fleet_scheduler.state.release(
decision.task_id, state="RELEASED"
)
raise FleetError("FLEET_CLAIM_ID_REQUIRED")
if claim_task_id != decision.task_id:
self.fleet_scheduler.state.release(
decision.task_id, state="RELEASED"
)
raise FleetConflict("FLEET_CLAIM_TASK_MISMATCH")
self.fleet_scheduler.state.bind_claim(
decision.reservation_id, str(claim_id)
)
return claim
except Exception:
decision = getattr(self._fleet_local, "pending", None)
if decision is not None:
self.fleet_scheduler.state.release(
decision.task_id, state="RELEASED"
)
raise
finally:
self._fleet_local.pending = None
self._fleet_local.actor_context = None
def record_worker_failure(self, worker_id, failure_class, **kwargs):
return self.fleet_scheduler.state.record_worker_failure(
worker_id, failure_class, **kwargs
)
def record_worker_success(self, worker_id, **kwargs):
return self.fleet_scheduler.state.record_worker_success(
worker_id, **kwargs
)
def quarantine_task_worker(
self, task_id, worker_id, failure_class, **kwargs
):
return self.fleet_scheduler.state.record_task_worker_failure(
task_id, worker_id, failure_class, **kwargs
)
-181
View File
@@ -1,181 +0,0 @@
"""Trusted TaskPlane scope model and SQL authorization helpers."""
from __future__ import annotations
from dataclasses import dataclass, replace
from typing import Any, Mapping
from .errors import ForbiddenError, ValidationError
SCOPE_FIELDS = ("tenant_id", "user_id", "project_id", "conversation_id", "run_id")
PRIVILEGED_CAPABILITIES = frozenset(
{"*", "admin", "system.admin", "task.admin", "task.read.all"}
)
def _value(value: Any, field: str) -> Any:
if hasattr(value, "task_plan"):
value = value.task_plan
if isinstance(value, Mapping):
return value.get(field)
return getattr(value, field, None)
def _text(value: Any) -> str | None:
if value is None:
return None
result = str(value).strip()
return result or None
@dataclass(frozen=True)
class TaskScope:
scope_type: str
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
run_id: str | None = None
@dataclass(frozen=True)
class TaskActorContext:
identity: str
scope_type: str = "legacy"
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
run_id: str | None = None
capabilities: frozenset[str] = frozenset()
def normalize_task_scope(value: Any) -> TaskScope:
fields = {field: _text(_value(value, field)) for field in SCOPE_FIELDS}
declared = _text(_value(value, "scope_type"))
if declared is None:
declared = "tenant" if fields["tenant_id"] else "legacy"
if declared == "tenant":
if fields["tenant_id"] is None:
raise ValidationError(
"tenant scope requires tenant_id", code="TASK_SCOPE_TENANT_ID_REQUIRED"
)
elif declared == "legacy":
if any(fields.values()):
raise ValidationError(
"legacy scope cannot contain scoped ownership",
code="TASK_SCOPE_LEGACY_WITH_SCOPED_FIELDS",
)
elif declared == "system":
if any(fields[field] for field in ("tenant_id", "user_id", "project_id", "conversation_id")):
raise ValidationError(
"system scope cannot contain tenant ownership",
code="TASK_SCOPE_SYSTEM_WITH_TENANT_FIELDS",
)
else:
raise ValidationError("invalid task scope type", code="TASK_SCOPE_TYPE_INVALID")
return TaskScope(declared, **fields)
def actor_is_privileged(actor: TaskActorContext) -> bool:
return bool(actor.capabilities & PRIVILEGED_CAPABILITIES)
def actor_context(
identity: str,
configured: TaskActorContext | Mapping[str, Any] | None,
capabilities: set[str] | frozenset[str],
) -> TaskActorContext:
if isinstance(configured, TaskActorContext):
context = configured
else:
data = dict(configured or {})
data.pop("identity", None)
data.pop("capabilities", None)
context = TaskActorContext(identity=str(identity), **data)
scope = normalize_task_scope(context)
return replace(
context,
identity=str(identity),
scope_type=scope.scope_type,
tenant_id=scope.tenant_id,
user_id=scope.user_id,
project_id=scope.project_id,
conversation_id=scope.conversation_id,
run_id=scope.run_id,
capabilities=frozenset(str(item) for item in capabilities),
)
def bind_task_scope(plan: Any, actor: TaskActorContext, *, public_create: bool) -> Any:
requested = normalize_task_scope(plan)
if actor_is_privileged(actor):
trusted = normalize_task_scope(actor)
if requested.scope_type == "legacy" and trusted.scope_type != "legacy":
plan.scope_type = trusted.scope_type
for field in SCOPE_FIELDS:
setattr(plan, field, getattr(actor, field))
return plan
trusted = normalize_task_scope(actor)
if public_create and trusted.scope_type == "legacy":
raise ForbiddenError(
"authenticated identity has no tenant scope",
code="TASK_ACTOR_SCOPE_UNAVAILABLE",
)
if trusted.scope_type != "tenant":
if requested.scope_type == "legacy":
plan.scope_type = trusted.scope_type
for field in SCOPE_FIELDS:
setattr(plan, field, getattr(actor, field))
return plan
if requested.scope_type != trusted.scope_type:
raise ForbiddenError("task scope mismatch", code="TASK_SCOPE_MISMATCH_DENY")
return plan
if requested.scope_type not in {"legacy", "tenant"}:
raise ForbiddenError("task scope mismatch", code="TASK_SCOPE_MISMATCH_DENY")
updates: dict[str, Any] = {"scope_type": "tenant"}
for field in SCOPE_FIELDS:
declared = getattr(requested, field)
authoritative = getattr(actor, field)
if declared is not None and declared != authoritative:
raise ForbiddenError(
"task scope mismatch",
code="TASK_SCOPE_MISMATCH_DENY",
details={"field": field},
)
updates[field] = authoritative
for field, item in updates.items():
setattr(plan, field, item)
return plan
def actor_scope_sql(actor: TaskActorContext | None) -> tuple[str, tuple[Any, ...]]:
if actor is None or actor_is_privileged(actor):
return "", ()
scope = normalize_task_scope(actor)
if scope.scope_type == "legacy":
return "scope_type = 'legacy'", ()
if scope.scope_type == "system":
return "scope_type = 'system'", ()
clauses = ["scope_type = 'tenant'", "tenant_id = ?"]
params: list[Any] = [scope.tenant_id]
for field in ("user_id", "project_id", "conversation_id", "run_id"):
value = getattr(scope, field)
if value is None:
clauses.append(f"{field} IS NULL")
else:
clauses.append(f"({field} IS NULL OR {field} = ?)")
params.append(value)
return " AND ".join(clauses), tuple(params)
def task_scope_values(value: Any) -> tuple[str | None, ...]:
scope = normalize_task_scope(value)
return (
scope.scope_type,
scope.tenant_id,
scope.user_id,
scope.project_id,
scope.conversation_id,
scope.run_id,
)
+11 -95
View File
@@ -32,7 +32,6 @@ from .models import (
redact_value,
validate_identifier,
)
from .scope import TaskActorContext, actor_scope_sql, task_scope_values
EventEnvelope = tuple[str, str, str, str, str, float]
@@ -135,12 +134,6 @@ class TaskPlaneStore:
status TEXT NOT NULL,
task_json TEXT NOT NULL,
task_type TEXT NOT NULL,
scope_type TEXT NOT NULL DEFAULT 'legacy',
tenant_id TEXT,
user_id TEXT,
project_id TEXT,
conversation_id TEXT,
run_id TEXT,
root_task_id TEXT NOT NULL,
parent_task_id TEXT,
correlation_id TEXT,
@@ -279,21 +272,6 @@ class TaskPlaneStore:
)
"""
)
self._ensure_columns(
"tasks",
{
"scope_type": "TEXT NOT NULL DEFAULT 'legacy'",
"tenant_id": "TEXT",
"user_id": "TEXT",
"project_id": "TEXT",
"conversation_id": "TEXT",
"run_id": "TEXT",
},
)
self.conn.execute(
"UPDATE tasks SET scope_type='legacy' "
"WHERE scope_type IS NULL OR TRIM(scope_type)=''"
)
self._ensure_columns(
"workers",
{
@@ -321,14 +299,6 @@ class TaskPlaneStore:
)
self.conn.execute("CREATE INDEX IF NOT EXISTS ix_tasks_status ON tasks(status)")
self.conn.execute("CREATE INDEX IF NOT EXISTS ix_tasks_parent ON tasks(parent_task_id)")
self.conn.execute(
"CREATE INDEX IF NOT EXISTS ix_tasks_scope_tenant_project "
"ON tasks(scope_type, tenant_id, project_id)"
)
self.conn.execute(
"CREATE INDEX IF NOT EXISTS ix_tasks_scope_owner "
"ON tasks(scope_type, tenant_id, user_id, conversation_id, run_id)"
)
self.conn.execute("CREATE INDEX IF NOT EXISTS ix_workers_status ON workers(status)")
self.conn.execute("CREATE INDEX IF NOT EXISTS ix_claims_task ON claims(task_id)")
self.conn.execute(
@@ -377,42 +347,6 @@ class TaskPlaneStore:
raise NotFoundError("task")
task_json = json.loads(row["task_json"])
task_plan = TaskPlan.from_dict(task_json)
stored_scope = tuple(
row[field]
for field in (
"scope_type",
"tenant_id",
"user_id",
"project_id",
"conversation_id",
"run_id",
)
)
json_has_scope = any(
field in task_json
for field in (
"scope_type",
"tenant_id",
"user_id",
"project_id",
"conversation_id",
"run_id",
)
)
if json_has_scope and stored_scope != task_scope_values(task_plan):
raise ConflictError(
"task scope persistence mismatch",
code="TASK_SCOPE_PERSISTENCE_MISMATCH",
)
if not json_has_scope:
(
task_plan.scope_type,
task_plan.tenant_id,
task_plan.user_id,
task_plan.project_id,
task_plan.conversation_id,
task_plan.run_id,
) = stored_scope
task_plan.parent_task_id = row["parent_task_id"]
task_plan.correlation_id = row["correlation_id"]
return TaskRecord(
@@ -455,7 +389,6 @@ class TaskPlaneStore:
task_id = task_plan.task_id or f"task-{uuid.uuid4()}"
task_plan.task_id = task_id
task_json = redact_value(task_plan.to_dict(redact=False))
scope_values = task_scope_values(task_plan)
now = _now()
with self._transaction():
if idempotency_key:
@@ -464,11 +397,7 @@ class TaskPlaneStore:
(idempotency_key,),
).fetchone()
if existing:
existing_plan = TaskPlan.from_dict(
json.loads(existing["task_json"])
)
existing_json = redact_value(existing_plan.to_dict(redact=False))
if _canonical(existing_json) != _canonical(task_json):
if _canonical(json.loads(existing["task_json"])) != _canonical(task_json):
raise ConflictError("idempotency key payload mismatch")
return self.get_task(existing["task_id"])
root_task_id = task_id
@@ -483,18 +412,14 @@ class TaskPlaneStore:
self.conn.execute(
"""
INSERT INTO tasks
(task_id, revision, status, task_json, task_type,
scope_type, tenant_id, user_id, project_id, conversation_id, run_id,
root_task_id, parent_task_id, correlation_id, idempotency_key,
created_at, updated_at, failure_reason)
VALUES (?, 1, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, NULL)
(task_id, revision, status, task_json, task_type, root_task_id, parent_task_id, correlation_id, idempotency_key, created_at, updated_at, failure_reason)
VALUES (?, 1, ?, ?, ?, ?, ?, ?, ?, ?, ?, NULL)
""",
(
task_id,
"CREATED",
json.dumps(task_json, ensure_ascii=False),
task_plan.task_type,
*scope_values,
root_task_id,
task_plan.parent_task_id,
task_plan.correlation_id,
@@ -526,30 +451,21 @@ class TaskPlaneStore:
)
return self.get_task(task.task_id)
def get_task(
self, task_id: str, actor_context: TaskActorContext | None = None
) -> TaskRecord:
def get_task(self, task_id: str) -> TaskRecord:
task_id = _validated_id(task_id, "task_id")
predicate, scope_params = actor_scope_sql(actor_context)
sql = "SELECT * FROM tasks WHERE task_id=?"
if predicate:
sql += f" AND ({predicate})"
with self._lock:
row = self.conn.execute(sql, (task_id, *scope_params)).fetchone()
row = self.conn.execute(
"SELECT * FROM tasks WHERE task_id=?", (task_id,)
).fetchone()
if row is None:
raise NotFoundError(f"task {task_id} not found")
return self._row_to_task(row)
def list_tasks(
self, actor_context: TaskActorContext | None = None
) -> list[TaskRecord]:
predicate, scope_params = actor_scope_sql(actor_context)
sql = "SELECT * FROM tasks"
if predicate:
sql += f" WHERE ({predicate})"
sql += " ORDER BY created_at DESC"
def list_tasks(self) -> list[TaskRecord]:
with self._lock:
rows = self.conn.execute(sql, scope_params).fetchall()
rows = self.conn.execute(
"SELECT * FROM tasks ORDER BY created_at DESC"
).fetchall()
return [self._row_to_task(row) for row in rows]
def get_task_budget(self, task_id: str) -> Budget:
@@ -1,759 +0,0 @@
"""Scope-bound governance for traces, Tool/MCP calls, and credentials.
This module deliberately stores references and audit metadata only. Secret
material remains in the configured credential provider boundary.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
from contextlib import contextmanager
import hashlib
import json
import re
import sqlite3
from pathlib import Path
from typing import Any, Iterable, Mapping
SCOPE_FIELDS = ("tenant_id", "user_id", "project_id", "conversation_id", "run_id")
PRIVILEGED_CAPABILITIES = frozenset({"*", "admin", "system.admin", "governance.admin"})
REFERENCE_RE = re.compile(r"^(systemd-credential|vault|secret-manager):[A-Za-z0-9][A-Za-z0-9._:/#-]{0,255}$")
IDENTIFIER_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{0,179}$")
class GovernanceError(RuntimeError):
def __init__(self, code: str, *, field: str | None = None) -> None:
super().__init__(code if field is None else f"{code}:{field}")
self.code = code
self.field = field
def _clean(value: Any) -> str | None:
if value is None:
return None
text = str(value).strip()
return text or None
def _scope_tuple(value: Iterable[str] | str | None) -> tuple[str, ...]:
if value is None:
return ()
if isinstance(value, str):
value = value.split(",")
return tuple(sorted({_clean(item) for item in value if _clean(item)}))
def _canonical(value: Any) -> str:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False, default=str)
def _sha(value: Any) -> str:
return hashlib.sha256(_canonical(value).encode("utf-8")).hexdigest()
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _identifier(value: Any, code: str) -> str:
text = _clean(value)
if text is None or IDENTIFIER_RE.fullmatch(text) is None:
raise GovernanceError(code)
return text
@dataclass(frozen=True)
class GovernanceScope:
"""Explicit resource scope; system/global resources never get fake tenants."""
scope_type: str
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
run_id: str | None = None
def __post_init__(self) -> None:
object.__setattr__(self, "scope_type", (_clean(self.scope_type) or "").lower())
for name in SCOPE_FIELDS:
object.__setattr__(self, name, _clean(getattr(self, name)))
if self.scope_type == "tenant":
if self.tenant_id is None:
raise GovernanceError("GOVERNANCE_TENANT_ID_REQUIRED")
elif self.scope_type in {"system", "global"}:
if any(getattr(self, name) is not None for name in SCOPE_FIELDS):
raise GovernanceError("GOVERNANCE_NON_TENANT_SCOPE_INVALID")
elif self.scope_type == "legacy":
if any(getattr(self, name) is not None for name in SCOPE_FIELDS):
raise GovernanceError("GOVERNANCE_LEGACY_SCOPE_INVALID")
else:
raise GovernanceError("GOVERNANCE_SCOPE_TYPE_INVALID")
@classmethod
def from_value(cls, value: Any) -> "GovernanceScope":
def get(name: str) -> Any:
return value.get(name) if isinstance(value, Mapping) else getattr(value, name, None)
declared = _clean(get("scope_type"))
tenant_id = _clean(get("tenant_id"))
return cls(
scope_type=declared or ("tenant" if tenant_id else "legacy"),
tenant_id=tenant_id,
user_id=get("user_id"),
project_id=get("project_id"),
conversation_id=get("conversation_id"),
run_id=get("run_id"),
)
def as_dict(self) -> dict[str, Any]:
return asdict(self)
@dataclass(frozen=True)
class GovernanceActor:
identity: str
scope: GovernanceScope
capabilities: tuple[str, ...] = ()
def __post_init__(self) -> None:
object.__setattr__(self, "identity", _identifier(self.identity, "GOVERNANCE_ACTOR_INVALID"))
object.__setattr__(self, "capabilities", _scope_tuple(self.capabilities))
@classmethod
def from_value(cls, value: Any) -> "GovernanceActor":
if isinstance(value, cls):
return value
if not isinstance(value, Mapping):
raise GovernanceError("GOVERNANCE_ACTOR_INVALID")
return cls(
identity=value.get("identity") or value.get("actor_id") or value.get("user_id"),
scope=GovernanceScope.from_value(value),
capabilities=_scope_tuple(value.get("capabilities") or value.get("capability_scope")),
)
@property
def privileged(self) -> bool:
return bool(set(self.capabilities) & PRIVILEGED_CAPABILITIES)
def authorize_scope(actor: GovernanceActor | Mapping[str, Any], resource: GovernanceScope | Mapping[str, Any]) -> None:
actor = GovernanceActor.from_value(actor)
resource = resource if isinstance(resource, GovernanceScope) else GovernanceScope.from_value(resource)
if actor.privileged:
return
if resource.scope_type in {"system", "global"}:
raise GovernanceError("GOVERNANCE_SYSTEM_SCOPE_AUTHZ_DENY")
if resource.scope_type == "legacy":
if actor.scope.scope_type != "legacy":
raise GovernanceError("GOVERNANCE_LEGACY_SCOPE_AUTHZ_DENY")
return
if actor.scope.scope_type != "tenant":
raise GovernanceError("GOVERNANCE_SCOPE_MISMATCH_DENY", field="scope_type")
for name in SCOPE_FIELDS:
expected = getattr(resource, name)
actual = getattr(actor.scope, name)
if expected is not None and expected != actual:
raise GovernanceError("GOVERNANCE_SCOPE_MISMATCH_DENY", field=name)
@dataclass(frozen=True)
class TraceContext:
trace_id: str
span_id: str
scope: GovernanceScope
parent_span_id: str | None = None
correlation_id: str | None = None
causation_id: str | None = None
task_id: str | None = None
claim_id: str | None = None
message_id: str | None = None
def __post_init__(self) -> None:
object.__setattr__(self, "trace_id", _identifier(self.trace_id, "TRACE_ID_INVALID"))
object.__setattr__(self, "span_id", _identifier(self.span_id, "TRACE_SPAN_ID_INVALID"))
for name in ("parent_span_id", "correlation_id", "causation_id", "task_id", "claim_id", "message_id"):
value = _clean(getattr(self, name))
if value is not None and IDENTIFIER_RE.fullmatch(value) is None:
raise GovernanceError("TRACE_LINEAGE_IDENTIFIER_INVALID", field=name)
object.__setattr__(self, name, value)
def child(self, *, span_id: str, actor: GovernanceActor | Mapping[str, Any], **lineage: Any) -> "TraceContext":
authorize_scope(actor, self.scope)
for name in ("trace_id", "task_id", "claim_id", "message_id", "correlation_id"):
supplied = _clean(lineage.get(name))
expected = getattr(self, name)
if supplied is not None and expected is not None and supplied != expected:
raise GovernanceError("TRACE_LINEAGE_CONTINUITY", field=name)
requested_scope = lineage.pop("scope", self.scope)
requested_scope = requested_scope if isinstance(requested_scope, GovernanceScope) else GovernanceScope.from_value(requested_scope)
if requested_scope != self.scope:
raise GovernanceError("TRACE_SCOPE_MISMATCH_DENY")
return TraceContext(
trace_id=self.trace_id,
span_id=span_id,
parent_span_id=self.span_id,
correlation_id=self.correlation_id,
causation_id=_clean(lineage.get("causation_id")) or self.span_id,
task_id=self.task_id,
claim_id=self.claim_id,
message_id=self.message_id,
scope=self.scope,
)
def as_dict(self) -> dict[str, Any]:
return {
"trace_id": self.trace_id,
"span_id": self.span_id,
"parent_span_id": self.parent_span_id,
"correlation_id": self.correlation_id,
"causation_id": self.causation_id,
"task_id": self.task_id,
"claim_id": self.claim_id,
"message_id": self.message_id,
**self.scope.as_dict(),
}
def bind_trace_lineage(
*,
actor: GovernanceActor | Mapping[str, Any],
task: Any,
message: Any,
trace_id: str,
span_id: str,
parent: TraceContext | None = None,
) -> TraceContext:
actor = GovernanceActor.from_value(actor)
task_scope = GovernanceScope.from_value(task)
message_scope = GovernanceScope.from_value(message)
if task_scope != message_scope:
raise GovernanceError("TRACE_SCOPE_MISMATCH_DENY")
authorize_scope(actor, task_scope)
def get(value: Any, name: str) -> Any:
return value.get(name) if isinstance(value, Mapping) else getattr(value, name, None)
task_id = _clean(get(task, "task_id"))
message_task = _clean(get(message, "task_id"))
if task_id and message_task and task_id != message_task:
raise GovernanceError("TRACE_LINEAGE_CONTINUITY", field="task_id")
context = TraceContext(
trace_id=trace_id,
span_id=span_id,
parent_span_id=parent.span_id if parent else None,
correlation_id=_clean(get(task, "correlation_id")) or _clean(get(message, "correlation_id")),
causation_id=_clean(get(message, "causation_id")) or (parent.span_id if parent else None),
task_id=task_id or message_task,
claim_id=_clean(get(task, "claim_id")) or _clean(get(message, "claim_id")),
message_id=_clean(get(message, "envelope_id")) or _clean(get(message, "id")),
scope=task_scope,
)
if parent is not None:
if parent.trace_id != context.trace_id or parent.scope != context.scope:
raise GovernanceError("TRACE_LINEAGE_CONTINUITY")
if parent.task_id and context.task_id and parent.task_id != context.task_id:
raise GovernanceError("TRACE_LINEAGE_CONTINUITY", field="task_id")
return context
@dataclass(frozen=True)
class ToolDescriptor:
name: str
kind: str
scope: GovernanceScope
required_capabilities: tuple[str, ...] = ()
credential_ref: str | None = None
def __post_init__(self) -> None:
object.__setattr__(self, "name", _identifier(self.name, "TOOL_NAME_INVALID"))
object.__setattr__(self, "kind", (_clean(self.kind) or "").lower())
if self.kind not in {"tool", "mcp"}:
raise GovernanceError("TOOL_KIND_INVALID")
object.__setattr__(self, "required_capabilities", _scope_tuple(self.required_capabilities))
if self.credential_ref is not None and REFERENCE_RE.fullmatch(self.credential_ref) is None:
raise GovernanceError("CREDENTIAL_SECRET_EXPOSURE_DENY")
@dataclass(frozen=True)
class CredentialGrant:
grant_id: str
credential_ref: str
scope: GovernanceScope
operations: tuple[str, ...]
task_id: str | None = None
tool_name: str | None = None
expires_at: str | None = None
def __post_init__(self) -> None:
object.__setattr__(self, "grant_id", _identifier(self.grant_id, "CREDENTIAL_GRANT_ID_INVALID"))
if REFERENCE_RE.fullmatch(self.credential_ref or "") is None:
raise GovernanceError("CREDENTIAL_SECRET_EXPOSURE_DENY")
object.__setattr__(self, "operations", _scope_tuple(self.operations))
if not self.operations:
raise GovernanceError("CREDENTIAL_OPERATIONS_REQUIRED")
if self.expires_at is not None:
parsed = datetime.fromisoformat(self.expires_at.replace("Z", "+00:00"))
if parsed.tzinfo is None:
raise GovernanceError("CREDENTIAL_EXPIRY_INVALID")
def authorize(
self,
actor: GovernanceActor | Mapping[str, Any],
*,
operation: str,
task_id: str | None = None,
tool_name: str | None = None,
now: datetime | None = None,
) -> str:
try:
authorize_scope(actor, self.scope)
except GovernanceError as exc:
raise GovernanceError("CREDENTIAL_SCOPE_DENY", field=exc.field) from exc
if operation not in self.operations:
raise GovernanceError("CREDENTIAL_SCOPE_DENY", field="operation")
if self.task_id is not None and self.task_id != _clean(task_id):
raise GovernanceError("CREDENTIAL_SCOPE_DENY", field="task_id")
if self.tool_name is not None and self.tool_name != _clean(tool_name):
raise GovernanceError("CREDENTIAL_SCOPE_DENY", field="tool_name")
if self.expires_at is not None:
current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
expiry = datetime.fromisoformat(self.expires_at.replace("Z", "+00:00")).astimezone(timezone.utc)
if current >= expiry:
raise GovernanceError("CREDENTIAL_SCOPE_DENY", field="expires_at")
return self.credential_ref
def public(self) -> dict[str, Any]:
return {
"grant_id": self.grant_id,
"credential_ref": self.credential_ref,
"operations": list(self.operations),
"task_id": self.task_id,
"tool_name": self.tool_name,
"expires_at": self.expires_at,
**self.scope.as_dict(),
}
_SCHEMA = """
CREATE TABLE IF NOT EXISTS governance_schema_metadata (
singleton INTEGER PRIMARY KEY CHECK(singleton=1),
schema_version INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS governed_tools (
name TEXT PRIMARY KEY,
kind TEXT NOT NULL,
scope_json TEXT NOT NULL,
required_capabilities_json TEXT NOT NULL,
credential_ref TEXT,
descriptor_sha256 TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS governed_trace_spans (
trace_id TEXT NOT NULL,
span_id TEXT NOT NULL,
parent_span_id TEXT,
context_json TEXT NOT NULL,
context_sha256 TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY(trace_id, span_id)
);
CREATE TABLE IF NOT EXISTS governed_credential_grants (
grant_id TEXT PRIMARY KEY,
grant_json TEXT NOT NULL,
grant_sha256 TEXT NOT NULL,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS governed_tool_invocations (
invocation_key TEXT PRIMARY KEY,
request_sha256 TEXT NOT NULL,
tool_name TEXT NOT NULL,
task_id TEXT NOT NULL,
claim_id TEXT NOT NULL,
trace_id TEXT NOT NULL,
span_id TEXT NOT NULL,
status TEXT NOT NULL,
receipt_sha256 TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS governance_audit_evidence (
sequence INTEGER PRIMARY KEY AUTOINCREMENT,
evidence_id TEXT NOT NULL UNIQUE,
action TEXT NOT NULL,
actor_identity TEXT NOT NULL,
trace_id TEXT,
span_id TEXT,
resource_id TEXT NOT NULL,
outcome TEXT NOT NULL,
details_json TEXT NOT NULL,
previous_sha256 TEXT NOT NULL,
evidence_sha256 TEXT NOT NULL,
created_at TEXT NOT NULL
);
"""
class GovernanceStore:
def __init__(self, path: str | Path) -> None:
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
with self._connection() as connection:
has_metadata = connection.execute(
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='governance_schema_metadata'"
).fetchone()
if has_metadata:
row = connection.execute(
"SELECT schema_version FROM governance_schema_metadata WHERE singleton=1"
).fetchone()
if row is not None and int(row[0]) != 1:
raise GovernanceError("GOVERNANCE_SCHEMA_VERSION_UNSUPPORTED")
connection.executescript(_SCHEMA)
row = connection.execute(
"SELECT schema_version FROM governance_schema_metadata WHERE singleton=1"
).fetchone()
if row is None:
connection.execute(
"INSERT INTO governance_schema_metadata(singleton,schema_version) VALUES(1,1)"
)
connection.commit()
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@contextmanager
def _connection(self):
connection = self._connect()
try:
yield connection
connection.commit()
except Exception:
connection.rollback()
raise
finally:
connection.close()
def _audit(
self,
connection: sqlite3.Connection,
*,
action: str,
actor: GovernanceActor,
resource_id: str,
outcome: str,
trace: TraceContext | None,
details: Mapping[str, Any],
) -> dict[str, Any]:
previous = connection.execute(
"SELECT evidence_sha256 FROM governance_audit_evidence ORDER BY sequence DESC LIMIT 1"
).fetchone()
previous_sha = str(previous[0]) if previous else ""
created_at = _now()
material = {
"action": action,
"actor_identity": actor.identity,
"trace_id": trace.trace_id if trace else None,
"span_id": trace.span_id if trace else None,
"resource_id": resource_id,
"outcome": outcome,
"details": dict(details),
"previous_sha256": previous_sha,
"created_at": created_at,
}
digest = _sha(material)
evidence_id = "gev-" + digest[:32]
connection.execute(
"""INSERT INTO governance_audit_evidence
(evidence_id,action,actor_identity,trace_id,span_id,resource_id,outcome,
details_json,previous_sha256,evidence_sha256,created_at)
VALUES (?,?,?,?,?,?,?,?,?,?,?)""",
(
evidence_id, action, actor.identity, material["trace_id"], material["span_id"],
resource_id, outcome, _canonical(material["details"]), previous_sha, digest, created_at,
),
)
return {"evidence_id": evidence_id, "evidence_sha256": digest, **material}
def record_trace(self, trace: TraceContext, actor: GovernanceActor | Mapping[str, Any]) -> dict[str, Any]:
actor = GovernanceActor.from_value(actor)
authorize_scope(actor, trace.scope)
context = trace.as_dict()
digest = _sha(context)
with self._connection() as connection:
existing = connection.execute(
"SELECT context_sha256 FROM governed_trace_spans WHERE trace_id=? AND span_id=?",
(trace.trace_id, trace.span_id),
).fetchone()
if existing is not None and existing[0] != digest:
raise GovernanceError("TRACE_LINEAGE_CONFLICT")
if existing is not None:
return {"context": context, "context_sha256": digest, "idempotent": True}
if trace.parent_span_id:
parent = connection.execute(
"SELECT context_json FROM governed_trace_spans WHERE trace_id=? AND span_id=?",
(trace.trace_id, trace.parent_span_id),
).fetchone()
if parent is None:
raise GovernanceError("TRACE_LINEAGE_CONTINUITY", field="parent_span_id")
parent_context = json.loads(parent[0])
if GovernanceScope.from_value(parent_context) != trace.scope:
raise GovernanceError("TRACE_SCOPE_MISMATCH_DENY")
connection.execute(
"INSERT OR IGNORE INTO governed_trace_spans VALUES (?,?,?,?,?,?)",
(trace.trace_id, trace.span_id, trace.parent_span_id, _canonical(context), digest, _now()),
)
evidence = self._audit(
connection, action="TRACE_RECORDED", actor=actor,
resource_id=f"{trace.trace_id}:{trace.span_id}", outcome="ALLOW", trace=trace,
details={"context_sha256": digest},
)
return {"context": context, "context_sha256": digest, "evidence": evidence}
def register_tool(self, descriptor: ToolDescriptor, actor: GovernanceActor | Mapping[str, Any]) -> dict[str, Any]:
actor = GovernanceActor.from_value(actor)
if not actor.privileged:
raise GovernanceError("TOOL_MCP_AUTHZ_DENY")
public = {
"name": descriptor.name,
"kind": descriptor.kind,
"required_capabilities": list(descriptor.required_capabilities),
"credential_ref": descriptor.credential_ref,
**descriptor.scope.as_dict(),
}
digest = _sha(public)
with self._connection() as connection:
existing = connection.execute(
"SELECT descriptor_sha256 FROM governed_tools WHERE name=?", (descriptor.name,)
).fetchone()
if existing is not None and existing[0] != digest:
raise GovernanceError("TOOL_DESCRIPTOR_CONFLICT")
if existing is not None:
return {"descriptor": public, "descriptor_sha256": digest, "idempotent": True}
connection.execute(
"INSERT OR IGNORE INTO governed_tools VALUES (?,?,?,?,?,?)",
(descriptor.name, descriptor.kind, _canonical(descriptor.scope.as_dict()),
_canonical(descriptor.required_capabilities), descriptor.credential_ref, digest),
)
evidence = self._audit(
connection, action="TOOL_REGISTERED", actor=actor, resource_id=descriptor.name,
outcome="ALLOW", trace=None, details={"descriptor_sha256": digest},
)
return {"descriptor": public, "descriptor_sha256": digest, "evidence": evidence}
def authorize_tool(
self,
*,
actor: GovernanceActor | Mapping[str, Any],
tool_name: str,
trace: TraceContext,
requested_scope: GovernanceScope | Mapping[str, Any],
input_digest: str,
invocation_key: str,
) -> dict[str, Any]:
actor = GovernanceActor.from_value(actor)
invocation_key = _identifier(invocation_key, "TOOL_INVOCATION_KEY_INVALID")
if not re.fullmatch(r"[0-9a-f]{64}", input_digest):
raise GovernanceError("TOOL_INPUT_DIGEST_INVALID")
if trace.task_id is None or trace.claim_id is None:
raise GovernanceError("TOOL_MCP_AUTHZ_DENY", field="active_claim")
requested_scope = requested_scope if isinstance(requested_scope, GovernanceScope) else GovernanceScope.from_value(requested_scope)
if requested_scope != trace.scope:
raise GovernanceError("TOOL_SCOPE_PROPAGATION")
authorize_scope(actor, requested_scope)
with self._connection() as connection:
persisted_trace = connection.execute(
"SELECT context_sha256 FROM governed_trace_spans WHERE trace_id=? AND span_id=?",
(trace.trace_id, trace.span_id),
).fetchone()
if persisted_trace is None or persisted_trace[0] != _sha(trace.as_dict()):
raise GovernanceError("TOOL_MCP_AUTHZ_DENY", field="trace")
row = connection.execute("SELECT * FROM governed_tools WHERE name=?", (tool_name,)).fetchone()
if row is None:
raise GovernanceError("TOOL_MCP_AUTHZ_DENY")
descriptor_scope = GovernanceScope.from_value(json.loads(row["scope_json"]))
if descriptor_scope.scope_type == "tenant" and descriptor_scope != requested_scope:
raise GovernanceError("TOOL_MCP_AUTHZ_DENY")
required = set(json.loads(row["required_capabilities_json"]))
if not required.issubset(set(actor.capabilities)):
raise GovernanceError("TOOL_MCP_AUTHZ_DENY")
request_sha = _sha({
"tool_name": tool_name, "input_digest": input_digest,
"task_id": trace.task_id, "claim_id": trace.claim_id,
"trace_id": trace.trace_id, "span_id": trace.span_id,
"scope": requested_scope.as_dict(),
})
replay = connection.execute(
"SELECT * FROM governed_tool_invocations WHERE invocation_key=?",
(invocation_key,),
).fetchone()
if replay is not None:
if replay["request_sha256"] != request_sha:
raise GovernanceError("TOOL_INVOCATION_IDEMPOTENCY_CONFLICT")
if replay["status"] != "VERIFIED":
raise GovernanceError("TOOL_INVOCATION_RECONCILIATION_REQUIRED")
return {
"tool_name": tool_name, "trace_id": trace.trace_id,
"span_id": trace.span_id, "invocation_key": invocation_key,
"status": "VERIFIED", "receipt_sha256": replay["receipt_sha256"],
"idempotent": True,
}
created_at = _now()
connection.execute(
"""INSERT INTO governed_tool_invocations
(invocation_key,request_sha256,tool_name,task_id,claim_id,trace_id,span_id,
status,receipt_sha256,created_at,updated_at)
VALUES(?,?,?,?,?,?,?,?,?,?,?)""",
(invocation_key, request_sha, tool_name, trace.task_id, trace.claim_id,
trace.trace_id, trace.span_id, "AUTHORIZED", None, created_at, created_at),
)
evidence = self._audit(
connection, action="TOOL_AUTHORIZED", actor=actor, resource_id=tool_name,
outcome="ALLOW", trace=trace,
details={"input_sha256": input_digest, "scope_sha256": _sha(requested_scope.as_dict()),
"invocation_key": invocation_key},
)
return {
"tool_name": tool_name,
"trace_id": trace.trace_id,
"span_id": trace.span_id,
"invocation_key": invocation_key,
"status": "AUTHORIZED",
"credential_required": row["credential_ref"] is not None,
"evidence": evidence,
}
def record_tool_receipt(
self,
*,
invocation_key: str,
result_digest: str,
actor: GovernanceActor | Mapping[str, Any],
trace: TraceContext,
) -> dict[str, Any]:
actor = GovernanceActor.from_value(actor)
invocation_key = _identifier(invocation_key, "TOOL_INVOCATION_KEY_INVALID")
if not re.fullmatch(r"[0-9a-f]{64}", result_digest):
raise GovernanceError("TOOL_RECEIPT_DIGEST_INVALID")
authorize_scope(actor, trace.scope)
with self._connection() as connection:
row = connection.execute(
"SELECT * FROM governed_tool_invocations WHERE invocation_key=?",
(invocation_key,),
).fetchone()
if row is None or row["trace_id"] != trace.trace_id or row["span_id"] != trace.span_id:
raise GovernanceError("TOOL_RECEIPT_BINDING_INVALID")
if row["status"] == "VERIFIED":
if row["receipt_sha256"] != result_digest:
raise GovernanceError("TOOL_RECEIPT_CONFLICT")
return {"invocation_key": invocation_key, "status": "VERIFIED", "idempotent": True}
if row["status"] != "AUTHORIZED":
raise GovernanceError("TOOL_INVOCATION_RECONCILIATION_REQUIRED")
connection.execute(
"UPDATE governed_tool_invocations SET status='VERIFIED',receipt_sha256=?,updated_at=? "
"WHERE invocation_key=? AND status='AUTHORIZED'",
(result_digest, _now(), invocation_key),
)
evidence = self._audit(
connection, action="TOOL_RESULT_VERIFIED", actor=actor,
resource_id=row["tool_name"], outcome="ALLOW", trace=trace,
details={"invocation_key": invocation_key, "result_sha256": result_digest},
)
return {"invocation_key": invocation_key, "status": "VERIFIED", "evidence": evidence}
def store_grant(self, grant: CredentialGrant, actor: GovernanceActor | Mapping[str, Any]) -> dict[str, Any]:
actor = GovernanceActor.from_value(actor)
if not actor.privileged:
raise GovernanceError("CREDENTIAL_SCOPE_DENY")
public = grant.public()
digest = _sha(public)
with self._connection() as connection:
existing = connection.execute(
"SELECT grant_sha256 FROM governed_credential_grants WHERE grant_id=?", (grant.grant_id,)
).fetchone()
if existing is not None and existing[0] != digest:
raise GovernanceError("CREDENTIAL_GRANT_CONFLICT")
connection.execute(
"INSERT OR IGNORE INTO governed_credential_grants VALUES (?,?,?,?)",
(grant.grant_id, _canonical(public), digest, _now()),
)
evidence = self._audit(
connection, action="CREDENTIAL_GRANT_STORED", actor=actor,
resource_id=grant.grant_id, outcome="ALLOW", trace=None,
details={"grant_sha256": digest},
)
return {"grant": public, "grant_sha256": digest, "evidence": evidence}
def authorize_credential(
self,
*,
grant_id: str,
actor: GovernanceActor | Mapping[str, Any],
trace: TraceContext,
operation: str,
task_id: str | None,
tool_name: str | None,
now: datetime | None = None,
) -> dict[str, Any]:
actor = GovernanceActor.from_value(actor)
with self._connection() as connection:
row = connection.execute(
"SELECT grant_json FROM governed_credential_grants WHERE grant_id=?", (grant_id,)
).fetchone()
if row is None:
raise GovernanceError("CREDENTIAL_SCOPE_DENY")
raw = json.loads(row[0])
grant = CredentialGrant(
grant_id=raw["grant_id"], credential_ref=raw["credential_ref"],
scope=GovernanceScope.from_value(raw), operations=tuple(raw["operations"]),
task_id=raw.get("task_id"), tool_name=raw.get("tool_name"),
expires_at=raw.get("expires_at"),
)
if trace.scope != grant.scope:
raise GovernanceError("CREDENTIAL_SCOPE_DENY", field="trace_scope")
reference = grant.authorize(
actor, operation=operation, task_id=task_id, tool_name=tool_name, now=now
)
evidence = self._audit(
connection, action="CREDENTIAL_AUTHORIZED", actor=actor,
resource_id=grant_id, outcome="ALLOW", trace=trace,
details={"operation": operation, "task_id": task_id, "tool_name": tool_name,
"credential_ref_sha256": _sha(reference)},
)
return {"credential_ref": reference, "evidence": evidence}
def verify_audit_chain(self, required_actions: Iterable[str] = ()) -> bool:
ordered_actions: list[str] = []
previous = ""
with self._connection() as connection:
rows = connection.execute("SELECT * FROM governance_audit_evidence ORDER BY sequence").fetchall()
for row in rows:
details = json.loads(row["details_json"])
material = {
"action": row["action"], "actor_identity": row["actor_identity"],
"trace_id": row["trace_id"], "span_id": row["span_id"],
"resource_id": row["resource_id"], "outcome": row["outcome"],
"details": details, "previous_sha256": row["previous_sha256"],
"created_at": row["created_at"],
}
if row["previous_sha256"] != previous or row["evidence_sha256"] != _sha(material):
return False
previous = row["evidence_sha256"]
ordered_actions.append(row["action"])
position = 0
for required in required_actions:
try:
position = ordered_actions.index(required, position) + 1
except ValueError:
return False
return True
__all__ = [
"CredentialGrant", "GovernanceActor", "GovernanceError", "GovernanceScope",
"GovernanceStore", "ToolDescriptor", "TraceContext", "authorize_scope",
"bind_trace_lineage",
]
@@ -0,0 +1,905 @@
#!/usr/bin/env python3
"""Root-only, purpose-bound trusted-builder signer provisioner.
The provisioner consumes one exact Owner-signed request after the normal
engineering review has prepared it. It creates the Ed25519 private material
and builder HMAC on the host, encrypts both for systemd, publishes only the
public verification key, and appends one replay-protected receipt. It never
accepts or writes main, tag, baseline, production, or Owner-approval keys.
"""
from __future__ import annotations
import argparse
import base64
import datetime as dt
try:
import fcntl
except ImportError: # pragma: no cover - Windows only affects local payload tooling.
class _NoopFcntl:
LOCK_EX = 0
@staticmethod
def flock(_file, _mode):
return None
fcntl = _NoopFcntl()
import hashlib
import json
import os
from pathlib import Path
import secrets
import shutil
import stat
import subprocess
import tempfile
from typing import Any, Mapping
PURPOSE = "AIOS_TRUSTED_RELEASE_BUILDER"
TRUST_DOMAIN = "aios.trusted-builder.attestation.v1"
OWNER_NAMESPACE = "aios-owner-approval-v1"
OWNER_PRINCIPAL = "aios-task-grant-signer"
SCHEMA = "aios.trusted-builder-signer-provisioning.v2"
ALLOWED_CAPABILITIES = frozenset(
{
"builder_receipt_signing",
"artifact_provenance_signing",
"manifest_provenance_signing",
"builder_owned_release_evidence_signing",
}
)
DENIED_CAPABILITIES = frozenset(
{
"owner_release_approval",
"protected_main_merge",
"arbitrary_ref_write",
"tag_publication",
"baseline_finalize",
"production_runtime_switch",
}
)
OID_FIELDS = {
"gitea_main_oid",
"authority_main_oid_at_capture",
"governed_main_oid",
"gitea_main_tree_oid",
"pr_head_oid",
"protected_merge_commit",
}
SHA_FIELDS = {"source_evidence_sha256"}
PACKAGE_MANIFEST_PATH = "deploy/trusted-builder/aios-trusted-builder-package-manifest.json"
PACKAGE_MANIFEST_SCHEMA = "aios.trusted-builder-package.v1"
SOURCE_EVIDENCE_PATHS = (
"active-runtime/app/trusted_builder_signer_provisioning.py",
"active-runtime/app/aios_trusted_candidate_builder.py",
"config/gitops/trusted-candidate-builder-policy.json",
"deploy/systemd/aios-trusted-candidate-builder.service",
"deploy/systemd/aios-trusted-candidate-builder.socket",
"deploy/systemd/aios-gitops-model-api-trusted-builder.conf.example",
"deploy/libexec/aios-trusted-builder-signer-provisioner",
PACKAGE_MANIFEST_PATH,
)
SOURCE_EVIDENCE_MODES = {path: frozenset({0o600, 0o640, 0o644, 0o664}) for path in SOURCE_EVIDENCE_PATHS}
def package_contract_document() -> dict[str, Any]:
"""The non-secret package/activation contract for a fresh B2 payload."""
files = []
for path in SOURCE_EVIDENCE_PATHS:
install_mode = "0755" if path == "deploy/libexec/aios-trusted-builder-signer-provisioner" else "0644"
files.append({"source_path": path, "source_mode": "0644", "install_mode": install_mode})
return {
"schema": PACKAGE_MANIFEST_SCHEMA,
"version": 1,
"package_name": "aios-trusted-candidate-builder",
"source_root": "/opt/aios/trusted-candidate-builder/current",
"files": files,
"activation": {
"daemon_reload": True,
"enable_units": ["aios-trusted-candidate-builder.socket"],
"service_start": "socket-activation-only",
"credential_installation": False,
"signer_invocation": False,
},
}
def _source_evidence(source_root: Path, source_files: list[str] | tuple[str, ...]) -> dict[str, str]:
if set(source_files) != set(SOURCE_EVIDENCE_PATHS) or len(source_files) != len(SOURCE_EVIDENCE_PATHS):
raise ProvisioningError("SOURCE_EVIDENCE_SET_INVALID")
evidence: dict[str, str] = {}
for path in SOURCE_EVIDENCE_PATHS:
candidate = source_root / path
if candidate.is_symlink() or not candidate.is_file():
raise ProvisioningError("SOURCE_EVIDENCE_MISSING")
if stat.S_IMODE(candidate.stat().st_mode) not in SOURCE_EVIDENCE_MODES[path]:
raise ProvisioningError("SOURCE_EVIDENCE_MODE_INVALID")
evidence[path] = _digest_file(candidate)
return evidence
def verify_package_contract(source_root: Path) -> None:
manifest = source_root / PACKAGE_MANIFEST_PATH
raw, value = _read_json(manifest)
if value != package_contract_document() or raw != pretty_json(package_contract_document()):
raise ProvisioningError("PACKAGE_CONTRACT_INVALID")
_source_evidence(source_root, list(SOURCE_EVIDENCE_PATHS))
class ProvisioningError(RuntimeError):
"""Stable fail-closed error code; never includes secret material."""
def __init__(self, code: str) -> None:
super().__init__(code)
self.code = code
def canonical_json(value: Any) -> bytes:
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":")).encode("utf-8")
def pretty_json(value: Any) -> bytes:
return (json.dumps(value, ensure_ascii=False, sort_keys=True, indent=2) + "\n").encode("utf-8")
def sha256_bytes(value: bytes) -> str:
return hashlib.sha256(value).hexdigest()
def utc_now() -> dt.datetime:
return dt.datetime.now(dt.timezone.utc)
def iso(value: dt.datetime) -> str:
return value.astimezone(dt.timezone.utc).isoformat().replace("+00:00", "Z")
def _require_oid(value: Any, code: str) -> str:
if not isinstance(value, str) or len(value) != 40 or any(c not in "0123456789abcdef" for c in value.lower()):
raise ProvisioningError(code)
return value.lower()
def _require_sha(value: Any, code: str) -> str:
if not isinstance(value, str) or len(value) != 64 or any(c not in "0123456789abcdef" for c in value.lower()):
raise ProvisioningError(code)
return value.lower()
def _read_json(path: Path) -> tuple[bytes, dict[str, Any]]:
if path.is_symlink() or not path.is_file():
raise ProvisioningError("PROVISIONING_INPUT_MISSING")
try:
raw = path.read_bytes()
value = json.loads(raw.decode("utf-8"))
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ProvisioningError("PROVISIONING_INPUT_INVALID") from exc
if not isinstance(value, dict):
raise ProvisioningError("PROVISIONING_OBJECT_REQUIRED")
return raw, value
def _digest_file(path: Path) -> str:
if path.is_symlink() or not path.is_file():
raise ProvisioningError("SOURCE_EVIDENCE_MISSING")
digest = hashlib.sha256()
with path.open("rb") as stream:
for block in iter(lambda: stream.read(1024 * 1024), b""):
digest.update(block)
return digest.hexdigest()
def _validate_payload(payload: Mapping[str, Any]) -> None:
expected = {
"schema", "status", "operation", "repository", "source",
"signing_identity", "allowed_capabilities", "denied_capabilities",
"runtime_binding", "provisioning_constraints", "required_preconditions",
"provisioner_contract", "issued_at", "expires_in_seconds", "nonce",
}
if set(payload) != expected or payload.get("schema") != SCHEMA or payload.get("status") != "OWNER_SIGNED_PROVISIONING_REQUEST":
raise ProvisioningError("PROVISIONING_SCHEMA_INVALID")
if payload.get("operation") != "PROVISION_PURPOSE_BOUND_TRUSTED_RELEASE_BUILDER_SIGNER" or payload.get("repository") != "aios/aios":
raise ProvisioningError("PROVISIONING_SCOPE_INVALID")
source = payload.get("source")
identity = payload.get("signing_identity")
binding = payload.get("runtime_binding")
constraints = payload.get("provisioning_constraints")
contract = payload.get("provisioner_contract")
if not all(isinstance(item, Mapping) for item in (source, identity, binding, constraints, contract)):
raise ProvisioningError("PROVISIONING_BINDING_INVALID")
required_source = OID_FIELDS | {"authority_reconciliation_completed", "pr_number", "merge_parent_oids", "source_repository_identity", "source_evidence_sha256"}
if set(source) != required_source:
raise ProvisioningError("SOURCE_BINDING_INVALID")
for name in OID_FIELDS:
_require_oid(source.get(name), "SOURCE_OID_INVALID")
if not (
source["gitea_main_oid"] == source["authority_main_oid_at_capture"] == source["governed_main_oid"] == source["protected_merge_commit"]
) or source.get("authority_reconciliation_completed") is not True:
raise ProvisioningError("SOURCE_MAIN_NOT_CONVERGED")
if not isinstance(source.get("pr_number"), int) or isinstance(source["pr_number"], bool) or source["pr_number"] < 1 or source.get("source_repository_identity") != "aios/aios":
raise ProvisioningError("SOURCE_PR_BINDING_INVALID")
parents = source.get("merge_parent_oids")
if not isinstance(parents, list) or len(parents) != 2:
raise ProvisioningError("SOURCE_ANCESTRY_INVALID")
for item in parents:
_require_oid(item, "SOURCE_ANCESTRY_INVALID")
if parents[1] != source["pr_head_oid"] or parents[0] == parents[1]:
raise ProvisioningError("SOURCE_PR_HEAD_SECOND_PARENT_INVALID")
evidence = source.get("source_evidence_sha256")
if not isinstance(evidence, Mapping) or set(evidence) != set(SOURCE_EVIDENCE_PATHS):
raise ProvisioningError("SOURCE_EVIDENCE_SET_INVALID")
for path, digest in evidence.items():
if path not in SOURCE_EVIDENCE_MODES:
raise ProvisioningError("SOURCE_EVIDENCE_PATH_INVALID")
_require_sha(digest, "SOURCE_EVIDENCE_DIGEST_INVALID")
required_identity = {
"key_id", "key_generation", "algorithm", "purpose", "trust_domain",
"private_key_export", "private_key_delivery", "public_key_registration",
"service_principal_bound", "service_principal",
}
if set(identity) != required_identity or identity.get("algorithm") != "Ed25519" or identity.get("purpose") != PURPOSE or identity.get("trust_domain") != TRUST_DOMAIN:
raise ProvisioningError("SIGNING_IDENTITY_INVALID")
if identity.get("private_key_export") is not False or identity.get("service_principal_bound") is not True:
raise ProvisioningError("SIGNING_CAPABILITY_INVALID")
if not isinstance(identity.get("key_id"), str) or not identity["key_id"] or not isinstance(identity.get("key_generation"), int) or isinstance(identity["key_generation"], bool) or identity["key_generation"] < 1:
raise ProvisioningError("SIGNING_IDENTITY_INVALID")
if not isinstance(identity.get("service_principal"), str) or identity["service_principal"] != "aios-trusted-candidate-builder.service":
raise ProvisioningError("SERVICE_PRINCIPAL_INVALID")
if set(payload["allowed_capabilities"]) != ALLOWED_CAPABILITIES or set(payload["denied_capabilities"]) != DENIED_CAPABILITIES:
raise ProvisioningError("CAPABILITY_SCOPE_INVALID")
required_binding = {
"builder_service", "builder_socket", "credential_directory", "builder_policy",
"builder_public_keyring", "builder_hmac_credential", "signing_private_key_credential",
"hmac_credential_required", "peer_authentication_required", "execution_user",
}
if set(binding) != required_binding or binding.get("builder_service") != identity["service_principal"] or binding.get("hmac_credential_required") is not True or binding.get("peer_authentication_required") is not True or binding.get("execution_user") != "root":
raise ProvisioningError("RUNTIME_BINDING_INVALID")
required_constraints = {
"one_shot", "replay_protected", "append_only_receipt", "owner_approval_scope",
"production_mutation", "protected_ref_mutation", "tag_mutation", "baseline_mutation", "credential_material_in_payload",
}
if set(constraints) != required_constraints or any(constraints.get(name) is not False for name in ("production_mutation", "protected_ref_mutation", "tag_mutation", "baseline_mutation", "credential_material_in_payload")) or constraints.get("one_shot") is not True or constraints.get("replay_protected") is not True or constraints.get("append_only_receipt") is not True or constraints.get("owner_approval_scope") != "trusted-builder-signer-provisioning-only":
raise ProvisioningError("PROVISIONING_CONSTRAINTS_INVALID")
if not isinstance(contract.get("command"), str) or not contract["command"].startswith("/") or contract.get("credential_store") != "/etc/credstore.encrypted/aios-trusted-candidate-builder" or contract.get("receipt_log") != "/var/lib/aios-gitops/trusted-builder/provisioning-receipts.jsonl":
raise ProvisioningError("PROVISIONER_CONTRACT_INVALID")
if not isinstance(payload.get("required_preconditions"), list) or not payload["required_preconditions"]:
raise ProvisioningError("PROVISIONING_PRECONDITIONS_INVALID")
try:
issued = dt.datetime.fromisoformat(str(payload["issued_at"]).replace("Z", "+00:00"))
except ValueError as exc:
raise ProvisioningError("PROVISIONING_TIME_INVALID") from exc
if issued.tzinfo is None or not isinstance(payload.get("expires_in_seconds"), int) or isinstance(payload["expires_in_seconds"], bool) or not 60 <= payload["expires_in_seconds"] <= 900:
raise ProvisioningError("PROVISIONING_TIME_INVALID")
if not isinstance(payload.get("nonce"), str) or len(payload["nonce"]) < 16:
raise ProvisioningError("PROVISIONING_NONCE_INVALID")
def build_payload(
source_root: Path,
output: Path,
*,
main_oid: str,
tree_oid: str,
pr_number: int,
pr_head_oid: str,
merge_parent_oids: list[str],
source_files: list[str],
nonce: str | None = None,
) -> dict[str, Any]:
main_oid = _require_oid(main_oid, "MAIN_OID_INVALID")
tree_oid = _require_oid(tree_oid, "TREE_OID_INVALID")
pr_head_oid = _require_oid(pr_head_oid, "PR_HEAD_OID_INVALID")
if not isinstance(pr_number, int) or isinstance(pr_number, bool) or pr_number < 1:
raise ProvisioningError("PR_NUMBER_INVALID")
if len(merge_parent_oids) != 2:
raise ProvisioningError("MERGE_PARENTS_INVALID")
parents = [_require_oid(value, "MERGE_PARENT_INVALID") for value in merge_parent_oids]
if parents[1] != pr_head_oid or parents[0] == parents[1]:
raise ProvisioningError("MERGE_PARENT_ORDER_INVALID")
evidence = _source_evidence(source_root, source_files)
payload: dict[str, Any] = {
"schema": SCHEMA,
"status": "OWNER_SIGNED_PROVISIONING_REQUEST",
"operation": "PROVISION_PURPOSE_BOUND_TRUSTED_RELEASE_BUILDER_SIGNER",
"repository": "aios/aios",
"source": {
"gitea_main_oid": main_oid,
"gitea_main_tree_oid": tree_oid,
"authority_main_oid_at_capture": main_oid,
"governed_main_oid": main_oid,
"authority_reconciliation_completed": True,
"pr_number": pr_number,
"pr_head_oid": pr_head_oid,
"protected_merge_commit": main_oid,
"merge_parent_oids": parents,
"source_repository_identity": "aios/aios",
"source_evidence_sha256": evidence,
},
"signing_identity": {
"key_id": "builder-attestation-ed25519-g1",
"key_generation": 1,
"algorithm": "Ed25519",
"purpose": PURPOSE,
"trust_domain": TRUST_DOMAIN,
"private_key_export": False,
"private_key_delivery": "server_side_systemd_credential_only",
"public_key_registration": "/etc/aios-gitops/trusted-builder/public-keyring.json",
"service_principal_bound": True,
"service_principal": "aios-trusted-candidate-builder.service",
},
"allowed_capabilities": sorted(ALLOWED_CAPABILITIES),
"denied_capabilities": sorted(DENIED_CAPABILITIES),
"runtime_binding": {
"builder_service": "aios-trusted-candidate-builder.service",
"builder_socket": "/run/aios-trusted-candidate-builder.sock",
"credential_directory": "/run/credentials/aios-trusted-candidate-builder.service",
"builder_policy": "config/gitops/trusted-candidate-builder-policy.json",
"builder_public_keyring": "/etc/aios-gitops/trusted-builder/public-keyring.json",
"builder_hmac_credential": "builder_hmac",
"signing_private_key_credential": "signing_private_key",
"hmac_credential_required": True,
"peer_authentication_required": True,
"execution_user": "root",
},
"provisioning_constraints": {
"one_shot": True,
"replay_protected": True,
"append_only_receipt": True,
"owner_approval_scope": "trusted-builder-signer-provisioning-only",
"production_mutation": False,
"protected_ref_mutation": False,
"tag_mutation": False,
"baseline_mutation": False,
"credential_material_in_payload": False,
},
"required_preconditions": [
"FINAL_COMMON_MAIN_PROTECTED_MERGE_VERIFIED",
"AUTHORITY_MAIN_RECONCILIATION_COMPLETED",
"FINAL_MAIN_MERGE_PARENT_ORDER_VERIFIED",
"PR_HEAD_SECOND_PARENT_VERIFIED",
"TRUSTED_BUILDER_IMMUTABLE_PACKAGE_CONTRACT_VERIFIED",
"PROVISIONER_GENERATES_PRIVATE_KEY_SERVER_SIDE",
"PUBLIC_KEYRING_PURPOSE_AND_KEY_ID_MATCH",
"BUILDER_HMAC_PROVISIONED_SERVER_SIDE",
"BUILDER_SERVICE_PEER_POLICY_VERIFIED",
"APPROVED_SIGNER_RECEIPT_APPEND_ONLY",
],
"provisioner_contract": {
"command": "/usr/local/libexec/aios-trusted-builder-signer-provisioner",
"credential_store": "/etc/credstore.encrypted/aios-trusted-candidate-builder",
"receipt_log": "/var/lib/aios-gitops/trusted-builder/provisioning-receipts.jsonl",
"signature_namespace": OWNER_NAMESPACE,
"signature_principal": OWNER_PRINCIPAL,
"private_key_export": False,
},
"issued_at": iso(utc_now()),
"expires_in_seconds": 900,
"nonce": nonce or "trusted-builder-provisioning-" + secrets.token_hex(16),
}
_validate_payload(payload)
output.parent.mkdir(parents=True, exist_ok=True)
if output.exists():
raise ProvisioningError("PROVISIONING_OUTPUT_EXISTS")
output.write_bytes(pretty_json(payload))
os.chmod(output, 0o600)
return payload
class ProvisionerConfig:
def __init__(self) -> None:
self.credential_store = Path(os.getenv("AIOS_TRUSTED_BUILDER_CREDENTIAL_STORE", "/etc/credstore.encrypted/aios-trusted-candidate-builder"))
self.public_keyring = Path(os.getenv("AIOS_TRUSTED_BUILDER_PUBLIC_KEYRING", "/etc/aios-gitops/trusted-builder/public-keyring.json"))
self.binding_file = Path(os.getenv("AIOS_TRUSTED_BUILDER_BINDING_FILE", "/etc/aios-gitops/trusted-builder/service-principal-binding.json"))
self.receipt_log = Path(os.getenv("AIOS_TRUSTED_BUILDER_RECEIPT_LOG", "/var/lib/aios-gitops/trusted-builder/provisioning-receipts.jsonl"))
self.allowed_signers = Path(os.getenv("AIOS_TRUSTED_BUILDER_ALLOWED_SIGNERS", "/etc/aios/trust/aios-task-grant-allowed-signers"))
self.source_root = Path(os.getenv("AIOS_TRUSTED_BUILDER_SOURCE_ROOT", "/opt/aios/trusted-candidate-builder/current"))
self.authority_git_dir = Path(os.getenv("AIOS_TRUSTED_BUILDER_AUTHORITY_GIT_DIR", "/data/aios-gitops/repo.git"))
self.service_unit = Path(os.getenv("AIOS_TRUSTED_BUILDER_SERVICE_UNIT", "/etc/systemd/system/aios-trusted-candidate-builder.service"))
self.systemd_creds = Path(os.getenv("AIOS_SYSTEMD_CREDS", "/usr/bin/systemd-creds"))
self.ssh_keygen = Path(os.getenv("AIOS_SSH_KEYGEN", "/usr/bin/ssh-keygen"))
def _safe_parent(path: Path) -> None:
if not path.is_absolute() or path.is_symlink():
raise ProvisioningError("PROVISIONING_PATH_INVALID")
path.parent.mkdir(parents=True, exist_ok=True)
if path.parent.is_symlink():
raise ProvisioningError("PROVISIONING_PATH_SYMLINK")
def _atomic_write(path: Path, data: bytes, mode: int = 0o600) -> None:
_safe_parent(path)
fd, temporary = tempfile.mkstemp(prefix=".trusted-builder-", dir=path.parent)
try:
os.fchmod(fd, mode)
with os.fdopen(fd, "wb", closefd=True) as stream:
stream.write(data)
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary, path)
os.chmod(path, mode)
finally:
if os.path.exists(temporary):
os.unlink(temporary)
def _append(path: Path, row: Mapping[str, Any]) -> None:
_safe_parent(path)
fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_APPEND, 0o600)
try:
fcntl.flock(fd, fcntl.LOCK_EX)
os.write(fd, canonical_json(dict(row)) + b"\n")
os.fsync(fd)
fcntl.flock(fd, fcntl.LOCK_UN)
finally:
os.close(fd)
os.chmod(path, 0o600)
def _receipts(path: Path) -> list[dict[str, Any]]:
if not path.exists() and not path.is_symlink():
return []
if path.is_symlink() or not path.is_file():
raise ProvisioningError("RECEIPT_LOG_INVALID")
try:
text = path.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError) as exc:
raise ProvisioningError("RECEIPT_LOG_INVALID") from exc
if not text:
return []
# New receipts are canonical JSONL: exactly one object per physical line.
# If that framing does not parse, accept exactly one complete legacy object
# so the original pretty-printed receipt remains immutable and replayable.
lines = text.splitlines()
result: list[dict[str, Any]] = []
jsonl_valid = bool(lines)
for line in lines:
if not line:
jsonl_valid = False
break
try:
value = json.loads(line)
except json.JSONDecodeError:
jsonl_valid = False
break
if not isinstance(value, dict):
jsonl_valid = False
break
result.append(value)
if jsonl_valid:
return result
try:
legacy = json.loads(text)
except json.JSONDecodeError as exc:
raise ProvisioningError("RECEIPT_LOG_INVALID") from exc
if not isinstance(legacy, dict):
raise ProvisioningError("RECEIPT_LOG_INVALID")
return [legacy]
def _run_signature_verify(config: ProvisionerConfig, payload: bytes, signature: Path) -> None:
for path in (config.allowed_signers, signature, config.ssh_keygen):
if path.is_symlink() or not path.is_file():
raise ProvisioningError("OWNER_SIGNATURE_INPUT_INVALID")
result = subprocess.run(
[str(config.ssh_keygen), "-Y", "verify", "-f", str(config.allowed_signers), "-I", OWNER_PRINCIPAL, "-n", OWNER_NAMESPACE, "-s", str(signature)],
input=payload,
capture_output=True,
check=False,
timeout=30,
)
if result.returncode != 0:
raise ProvisioningError("OWNER_SIGNATURE_VERIFY_FAILED")
def _verify_service_policy(config: ProvisionerConfig) -> None:
if config.service_unit.is_symlink() or not config.service_unit.is_file():
raise ProvisioningError("BUILDER_SERVICE_UNIT_MISSING")
text = config.service_unit.read_text(encoding="utf-8")
required = (
"User=root", "LoadCredentialEncrypted=signing_private_key:",
"LoadCredentialEncrypted=builder_hmac:", "NoNewPrivileges=true",
"PrivateNetwork=true", "RestrictAddressFamilies=AF_UNIX",
)
if any(item not in text for item in required):
raise ProvisioningError("BUILDER_SERVICE_POLICY_INVALID")
def _git(config: ProvisionerConfig, *args: str) -> subprocess.CompletedProcess:
return subprocess.run(
["git", "--git-dir", str(config.authority_git_dir), *args],
capture_output=True, check=False, timeout=30,
)
def _verify_source(config: ProvisionerConfig, source: Mapping[str, Any]) -> None:
live_main = subprocess.run(
["git", "--git-dir", str(config.authority_git_dir), "rev-parse", "refs/heads/main"],
capture_output=True, text=True, check=False, timeout=30,
)
live_tree = subprocess.run(
["git", "--git-dir", str(config.authority_git_dir), "rev-parse", f"{source['protected_merge_commit']}^{{tree}}"],
capture_output=True, text=True, check=False, timeout=30,
)
parents = subprocess.run(
["git", "--git-dir", str(config.authority_git_dir), "rev-list", "--parents", "-n", "1", source["protected_merge_commit"]],
capture_output=True, text=True, check=False, timeout=30,
)
ancestry = subprocess.run(
["git", "--git-dir", str(config.authority_git_dir), "merge-base", "--is-ancestor", source["pr_head_oid"], source["protected_merge_commit"]],
capture_output=True, text=True, check=False, timeout=30,
)
expected_parents = [source["protected_merge_commit"], *source["merge_parent_oids"]]
if (live_main.returncode != 0 or live_main.stdout.strip().lower() != source["gitea_main_oid"]
or live_tree.returncode != 0 or live_tree.stdout.strip().lower() != source["gitea_main_tree_oid"]
or parents.returncode != 0 or parents.stdout.strip().split() != expected_parents
or ancestry.returncode != 0):
raise ProvisioningError("AUTHORITY_MAIN_SOURCE_DRIFT")
if source["merge_parent_oids"][1] != source["pr_head_oid"]:
raise ProvisioningError("SOURCE_PR_HEAD_SECOND_PARENT_INVALID")
verify_package_contract(config.source_root)
for relative, expected in source["source_evidence_sha256"].items():
actual = _digest_file(config.source_root / relative)
if actual != expected:
raise ProvisioningError("SOURCE_EVIDENCE_DIGEST_MISMATCH")
_verify_service_policy(config)
def _verify_replay_source(config: ProvisionerConfig, source: Mapping[str, Any]) -> None:
"""Verify the historical f8-bound source after authority main advances."""
commit = source["protected_merge_commit"]
live_main = _git(config, "rev-parse", "refs/heads/main")
tree = _git(config, "rev-parse", f"{commit}^{{tree}}")
parents = _git(config, "rev-list", "--parents", "-n", "1", commit)
source_ancestry = _git(config, "merge-base", "--is-ancestor", commit, live_main.stdout.strip().decode("ascii", "ignore"))
pr_ancestry = _git(config, "merge-base", "--is-ancestor", source["pr_head_oid"], commit)
expected_parents = [commit, *source["merge_parent_oids"]]
if (
live_main.returncode != 0
or tree.returncode != 0
or tree.stdout.strip().decode("ascii", "ignore").lower() != source["gitea_main_tree_oid"]
or parents.returncode != 0
or parents.stdout.decode("utf-8", "replace").strip().split() != expected_parents
or source_ancestry.returncode != 0
or pr_ancestry.returncode != 0
):
raise ProvisioningError("AUTHORITY_HISTORICAL_SOURCE_DRIFT")
if source["merge_parent_oids"][1] != source["pr_head_oid"]:
raise ProvisioningError("SOURCE_PR_HEAD_SECOND_PARENT_INVALID")
for relative, expected in source["source_evidence_sha256"].items():
historical = _git(config, "show", f"{commit}:{relative}")
if historical.returncode != 0 or sha256_bytes(historical.stdout) != expected:
raise ProvisioningError("SOURCE_EVIDENCE_DIGEST_MISMATCH")
# The installed repair package must still satisfy the governed package and
# service policy, but its bytes are expected to differ from historical f8.
verify_package_contract(config.source_root)
_verify_service_policy(config)
def _keyring(identity: Mapping[str, Any], public: bytes, now: dt.datetime) -> dict[str, Any]:
record = {
"algorithm": "Ed25519",
"purpose": PURPOSE,
"key_id": identity["key_id"],
"generation": identity["key_generation"],
"not_before": iso(now - dt.timedelta(minutes=1)),
"expires_at": iso(now + dt.timedelta(days=365)),
"status": "ACTIVE",
"public_key": base64.b64encode(public).decode("ascii"),
"public_key_sha256": sha256_bytes(public),
}
body = {"algorithm": "Ed25519", "purpose": PURPOSE, "keys": [record]}
return {**body, "keyring_sha256": sha256_bytes(canonical_json(body))}
def _encrypt(config: ProvisionerConfig, plain: Path, target: Path, name: str) -> None:
if config.systemd_creds.is_symlink() or not config.systemd_creds.is_file():
raise ProvisioningError("SYSTEMD_CREDS_UNAVAILABLE")
result = subprocess.run(
[str(config.systemd_creds), "--with-key=host", "--name=" + name, "encrypt", str(plain), str(target)],
capture_output=True,
check=False,
timeout=60,
)
if result.returncode != 0 or target.is_symlink() or not target.is_file() or target.stat().st_size <= 0:
raise ProvisioningError("CREDENTIAL_ENCRYPTION_FAILED")
os.chmod(target, 0o600)
def _require_private_regular_file(path: Path, code: str) -> None:
if path.is_symlink() or not path.is_file():
raise ProvisioningError(code)
metadata = path.stat()
if stat.S_IMODE(metadata.st_mode) != 0o600 or metadata.st_size <= 0:
raise ProvisioningError(code)
def _read_state_json(path: Path, code: str) -> dict[str, Any]:
_require_private_regular_file(path, code)
try:
value = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ProvisioningError(code) from exc
if not isinstance(value, dict):
raise ProvisioningError(code)
return value
def _parse_active_window(record: Mapping[str, Any]) -> None:
try:
not_before = dt.datetime.fromisoformat(str(record["not_before"]).replace("Z", "+00:00"))
expires_at = dt.datetime.fromisoformat(str(record["expires_at"]).replace("Z", "+00:00"))
except (KeyError, ValueError) as exc:
raise ProvisioningError("KEYRING_BINDING_INVALID") from exc
now = utc_now()
if not_before.tzinfo is None or expires_at.tzinfo is None or not (not_before <= now < expires_at):
raise ProvisioningError("KEYRING_BINDING_INVALID")
def _verify_existing_state(
config: ProvisionerConfig,
payload: Mapping[str, Any],
raw: bytes,
signature_bytes: bytes,
row: Mapping[str, Any],
) -> None:
receipt_keys = {
"schema", "status", "created_at", "operation_key", "payload_sha256",
"signature_sha256", "source_commit", "source_tree", "key_id",
"generation", "algorithm", "purpose", "service_principal",
"service_principal_bound", "public_key_sha256",
"private_key_exportable", "credential_projection",
"negative_capability_tests", "production_mutation",
"protected_ref_mutation", "tag_mutation", "baseline_mutation",
}
if set(row) != receipt_keys:
raise ProvisioningError("RECEIPT_CONFLICT")
try:
created_at = dt.datetime.fromisoformat(str(row["created_at"]).replace("Z", "+00:00"))
except ValueError as exc:
raise ProvisioningError("RECEIPT_CONFLICT") from exc
identity = payload["signing_identity"]
source = payload["source"]
expected = {
"schema": "aios.trusted-builder-signer-provisioning-receipt.v1",
"status": "COMPLETED",
"operation_key": "payload:" + sha256_bytes(raw) + ":signature:" + sha256_bytes(signature_bytes),
"payload_sha256": sha256_bytes(raw),
"signature_sha256": sha256_bytes(signature_bytes),
"source_commit": source["gitea_main_oid"],
"source_tree": source["gitea_main_tree_oid"],
"key_id": identity["key_id"],
"generation": identity["key_generation"],
"algorithm": "Ed25519",
"purpose": PURPOSE,
"service_principal": identity["service_principal"],
"service_principal_bound": True,
"private_key_exportable": False,
"credential_projection": "systemd-encrypted-host-credential",
"negative_capability_tests": {name: True for name in sorted(DENIED_CAPABILITIES)},
"production_mutation": False,
"protected_ref_mutation": False,
"tag_mutation": False,
"baseline_mutation": False,
}
if created_at.tzinfo is None or any(row.get(key) != value for key, value in expected.items()):
raise ProvisioningError("RECEIPT_CONFLICT")
public_keyring = _read_state_json(config.public_keyring, "KEYRING_BINDING_INVALID")
if set(public_keyring) != {"algorithm", "purpose", "keys", "keyring_sha256"}:
raise ProvisioningError("KEYRING_BINDING_INVALID")
keys = public_keyring.get("keys")
if not isinstance(keys, list) or len(keys) != 1 or not isinstance(keys[0], Mapping):
raise ProvisioningError("KEYRING_BINDING_INVALID")
record = dict(keys[0])
if set(record) != {
"algorithm", "purpose", "key_id", "generation", "not_before",
"expires_at", "status", "public_key", "public_key_sha256",
}:
raise ProvisioningError("KEYRING_BINDING_INVALID")
try:
public_key = base64.b64decode(str(record["public_key"]), validate=True)
except (ValueError, TypeError) as exc:
raise ProvisioningError("KEYRING_BINDING_INVALID") from exc
public_sha = sha256_bytes(public_key)
keyring_body = {key: public_keyring[key] for key in ("algorithm", "purpose", "keys")}
if (
len(public_key) != 32
or public_keyring["algorithm"] != "Ed25519"
or public_keyring["purpose"] != PURPOSE
or record.get("algorithm") != "Ed25519"
or record.get("purpose") != PURPOSE
or record.get("key_id") != identity["key_id"]
or record.get("generation") != identity["key_generation"]
or record.get("status") != "ACTIVE"
or record.get("public_key_sha256") != public_sha
or row.get("public_key_sha256") != public_sha
or public_keyring.get("keyring_sha256") != sha256_bytes(canonical_json(keyring_body))
):
raise ProvisioningError("KEYRING_BINDING_INVALID")
_parse_active_window(record)
binding = _read_state_json(config.binding_file, "SERVICE_PRINCIPAL_BINDING_INVALID")
expected_binding = {
"schema": "aios.trusted-builder.service-principal-binding.v1",
"purpose": PURPOSE,
"service_principal": identity["service_principal"],
"key_id": identity["key_id"],
"generation": identity["key_generation"],
"algorithm": "Ed25519",
"private_key_export": False,
"public_key_sha256": public_sha,
"source_commit": source["gitea_main_oid"],
"source_tree": source["gitea_main_tree_oid"],
"payload_sha256": sha256_bytes(raw),
}
if binding != expected_binding:
raise ProvisioningError("SERVICE_PRINCIPAL_BINDING_INVALID")
if (
config.credential_store.is_symlink()
or not config.credential_store.is_dir()
or stat.S_IMODE(config.credential_store.stat().st_mode) != 0o700
):
raise ProvisioningError("CREDENTIAL_STORE_INVALID")
_require_private_regular_file(
config.credential_store / "signing_private_key", "RECEIPT_OUTPUT_MISSING"
)
_require_private_regular_file(
config.credential_store / "builder_hmac", "RECEIPT_OUTPUT_MISSING"
)
def provision(config: ProvisionerConfig, payload_path: Path, signature_path: Path) -> dict[str, Any]:
if os.name != "nt" and os.geteuid() != 0:
raise ProvisioningError("ROOT_REQUIRED")
raw, payload = _read_json(payload_path)
_validate_payload(payload)
_run_signature_verify(config, raw, signature_path)
signature_bytes = signature_path.read_bytes()
operation_key = "payload:" + sha256_bytes(raw) + ":signature:" + sha256_bytes(signature_bytes)
lock_path = config.receipt_log.with_suffix(".lock")
_safe_parent(lock_path)
with lock_path.open("a+b") as lock:
fcntl.flock(lock.fileno(), fcntl.LOCK_EX)
rows = _receipts(config.receipt_log)
previous = [row for row in rows if row.get("operation_key") == operation_key]
if previous:
row = previous[-1]
if any(canonical_json(item) != canonical_json(row) for item in previous):
raise ProvisioningError("RECEIPT_CONFLICT")
_verify_replay_source(config, payload["source"])
_verify_existing_state(config, payload, raw, signature_bytes, row)
return {
"status": "IDEMPOTENT_REPLAY_PASS",
"idempotent": True,
"mutation_executed": False,
"existing_signer_state_adopted": True,
**{key: row[key] for key in (
"operation_key", "payload_sha256", "signature_sha256",
"source_commit", "source_tree", "key_id", "generation",
"public_key_sha256", "purpose", "service_principal_bound",
"private_key_exportable",
)},
"negative_capability_tests": row["negative_capability_tests"],
}
_verify_source(config, payload["source"])
outputs = [config.public_keyring, config.binding_file, config.credential_store / "signing_private_key", config.credential_store / "builder_hmac"]
if any(path.exists() or path.is_symlink() for path in outputs):
raise ProvisioningError("PREEXISTING_STATE_CONFLICT")
config.credential_store.mkdir(parents=True, exist_ok=True, mode=0o700)
os.chmod(config.credential_store, 0o700)
temp_root = Path(tempfile.mkdtemp(prefix="trusted-builder-provision-"))
try:
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
now = utc_now()
private = Ed25519PrivateKey.generate()
public = private.public_key().public_bytes(serialization.Encoding.Raw, serialization.PublicFormat.Raw)
private_path = temp_root / "signing_private_key.pem"
private_path.write_bytes(private.private_bytes(serialization.Encoding.PEM, serialization.PrivateFormat.PKCS8, serialization.NoEncryption()))
os.chmod(private_path, 0o600)
hmac_path = temp_root / "builder_hmac"
hmac_path.write_bytes(secrets.token_bytes(32))
os.chmod(hmac_path, 0o600)
enc_private = temp_root / "signing_private_key.cred"
enc_hmac = temp_root / "builder_hmac.cred"
_encrypt(config, private_path, enc_private, "signing_private_key")
_encrypt(config, hmac_path, enc_hmac, "builder_hmac")
keyring = _keyring(payload["signing_identity"], public, now)
binding = {
"schema": "aios.trusted-builder.service-principal-binding.v1",
"purpose": PURPOSE,
"service_principal": payload["signing_identity"]["service_principal"],
"key_id": payload["signing_identity"]["key_id"],
"generation": payload["signing_identity"]["key_generation"],
"algorithm": "Ed25519",
"private_key_export": False,
"public_key_sha256": sha256_bytes(public),
"source_commit": payload["source"]["gitea_main_oid"],
"source_tree": payload["source"]["gitea_main_tree_oid"],
"payload_sha256": sha256_bytes(raw),
}
_atomic_write(temp_root / "public-keyring.json", pretty_json(keyring))
_atomic_write(temp_root / "service-principal-binding.json", pretty_json(binding))
config.public_keyring.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
config.binding_file.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
os.replace(temp_root / "signing_private_key.cred", config.credential_store / "signing_private_key")
os.replace(temp_root / "builder_hmac.cred", config.credential_store / "builder_hmac")
os.replace(temp_root / "public-keyring.json", config.public_keyring)
os.replace(temp_root / "service-principal-binding.json", config.binding_file)
for path in outputs:
os.chmod(path, 0o600)
receipt = {
"schema": "aios.trusted-builder-signer-provisioning-receipt.v1",
"status": "COMPLETED",
"created_at": iso(now),
"operation_key": operation_key,
"payload_sha256": sha256_bytes(raw),
"signature_sha256": sha256_bytes(signature_bytes),
"source_commit": payload["source"]["gitea_main_oid"],
"source_tree": payload["source"]["gitea_main_tree_oid"],
"key_id": payload["signing_identity"]["key_id"],
"generation": payload["signing_identity"]["key_generation"],
"algorithm": "Ed25519",
"purpose": PURPOSE,
"service_principal": payload["signing_identity"]["service_principal"],
"service_principal_bound": True,
"public_key_sha256": sha256_bytes(public),
"private_key_exportable": False,
"credential_projection": "systemd-encrypted-host-credential",
"negative_capability_tests": {name: True for name in sorted(DENIED_CAPABILITIES)},
"production_mutation": False,
"protected_ref_mutation": False,
"tag_mutation": False,
"baseline_mutation": False,
}
_append(config.receipt_log, receipt)
return {"status": "PROVISIONED", "idempotent": False, "mutation_executed": True, "existing_signer_state_adopted": False, **{key: receipt[key] for key in ("operation_key", "payload_sha256", "signature_sha256", "source_commit", "source_tree", "key_id", "generation", "purpose", "service_principal_bound", "public_key_sha256", "private_key_exportable")}, "negative_capability_tests": receipt["negative_capability_tests"]}
finally:
shutil.rmtree(temp_root, ignore_errors=True)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="Provision the purpose-bound trusted-builder signer")
parser.add_argument("--payload", type=Path)
parser.add_argument("--signature", type=Path)
parser.add_argument("--result-json", type=Path)
parser.add_argument("--self-test", action="store_true")
parser.add_argument("--verify-package", action="store_true")
parser.add_argument("--source-root", type=Path)
args = parser.parse_args(argv)
if args.self_test:
print("PASS")
return 0
if args.verify_package:
if args.source_root is None:
parser.error("--verify-package requires --source-root")
try:
verify_package_contract(args.source_root)
print("PACKAGE_CONTRACT_VALID=PASS")
return 0
except ProvisioningError as exc:
print(json.dumps({"status": "BLOCKED", "error_code": exc.code}, sort_keys=True))
return 42
if not args.payload or not args.signature or not args.result_json:
parser.error("--payload, --signature and --result-json are required")
try:
result = provision(ProvisionerConfig(), args.payload, args.signature)
_atomic_write(args.result_json, pretty_json(result))
print(json.dumps(result, sort_keys=True))
return 0
except ProvisioningError as exc:
result = {"schema": "aios.trusted-builder-signer-provisioning-result.v1", "status": "BLOCKED", "error_code": exc.code}
try:
_atomic_write(args.result_json, pretty_json(result))
except Exception:
pass
print(json.dumps(result, sort_keys=True))
return 42
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,117 @@
#!/usr/bin/env python3
"""Plan-bound production adapter for releases newer than the legacy baseline.
All identity comes from the already verified executor request. The adapter has
no Git/ref/tag authority and refuses a missing candidate/release-plan binding.
"""
from __future__ import annotations
import hashlib, json, os, shutil, subprocess, sys, tempfile, urllib.error, urllib.request
from pathlib import Path
from typing import Any, Mapping
class AdapterBlocked(RuntimeError): pass
ALLOWED_SERVICE_PORTS = {"aios-v2r2c2-production.service": 18928}
def _safe(value: str) -> str:
if not value or any(ch not in "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_.-" for ch in value):
raise AdapterBlocked("invalid_release_identity")
return value
def _run(*args: str, check: bool = True) -> subprocess.CompletedProcess[str]:
result = subprocess.run(args, check=False, capture_output=True, text=True, timeout=60, env={"PATH": os.environ.get("PATH", "")})
if check and result.returncode != 0: raise AdapterBlocked(result.stderr.strip() or "system_command_failed")
return result
def _sha(path: Path) -> str:
if path.is_symlink() or not path.is_file(): raise AdapterBlocked("artifact_must_be_regular_file")
digest=hashlib.sha256()
with path.open("rb") as stream:
for block in iter(lambda: stream.read(1024*1024), b""): digest.update(block)
return digest.hexdigest()
def _binding(value: Mapping[str, Any]) -> dict[str, Any]:
required=("version","target_oid","artifact_sha256","artifact_path","artifact_size","production_service","production_port","candidate_id","release_plan_sha256")
if any(key not in value for key in required): raise AdapterBlocked("trusted_binding_incomplete")
if str(value["version"]).startswith("V2R1") or not value["candidate_id"] or not value["release_plan_sha256"]: raise AdapterBlocked("legacy_or_unbound_release_rejected")
service = str(value["production_service"])
port = int(value["production_port"])
if ALLOWED_SERVICE_PORTS.get(service) != port:
raise AdapterBlocked("production_adapter_policy_rejected")
return {key:value[key] for key in required}
def _paths(binding: Mapping[str, Any]) -> dict[str, Path]:
version=_safe(str(binding["version"])); service=_safe(str(binding["production_service"]))
root=Path(os.environ.get("AIOS_RELEASE_ROOT", "/opt/aios/releases"))
slug=version.lower()
return {"release_dir":root/(version+"-"+str(binding["target_oid"])[:12]+"-"+str(binding["artifact_sha256"])[:12]),"current_link":Path("/opt/aios")/slug,"env_file":Path("/etc/aios")/(slug+".env"),"state_file":Path("/var/lib/aios")/slug/"deployment-state.json","unit_file":Path("/etc/systemd/system")/(service+".service" if not service.endswith(".service") else service),"health_url":f"http://127.0.0.1:{int(binding['production_port'])}/health"}
def _health(url: str, binding: Mapping[str, Any]) -> dict[str, Any]:
try:
with urllib.request.urlopen(url, timeout=5) as response: body=json.loads(response.read().decode()); status=response.status
except (OSError, ValueError, urllib.error.URLError): return {"status":"UNAVAILABLE"}
exact=(status==200 and body.get("version")==binding["version"] and body.get("commit_oid")==binding["target_oid"] and body.get("artifact_sha256")==binding["artifact_sha256"] and body.get("deployment_state")=="ACTIVE" and body.get("production_deployed") is True)
return {"status":"ACTIVE_VERIFIED" if exact else "CONFLICT", "http_status":status, **body}
def _extract(artifact: Path, destination: Path) -> None:
import tarfile
with tarfile.open(artifact, "r:*") as archive:
for member in archive.getmembers():
name=Path(member.name)
if name.is_absolute() or ".." in name.parts or member.issym() or member.islnk() or not member.isfile() or not member.name.startswith("payload/"):
raise AdapterBlocked("unsafe_artifact_member")
rel=name.relative_to("payload"); target=destination/rel; target.parent.mkdir(parents=True,exist_ok=True)
source=archive.extractfile(member)
if source is None: raise AdapterBlocked("artifact_member_unreadable")
target.write_bytes(source.read()); os.chmod(target,0o755 if rel.parts and rel.parts[0]=="scripts" else 0o644)
def probe(value: Mapping[str, Any]) -> dict[str, Any]:
binding=_binding(value); p=_paths(binding); active=_run("systemctl","is-active","--quiet",str(binding["production_service"]),check=False).returncode==0
return _health(str(p["health_url"]),binding) if active else {"status":"NOT_APPLIED","version":binding["version"],"target_oid":binding["target_oid"],"artifact_sha256":binding["artifact_sha256"],"production_deployed":False,"deployment_state":"ABSENT"}
def apply(value: Mapping[str, Any]) -> dict[str, Any]:
binding=_binding(value); p=_paths(binding); p["release_dir"].parent.mkdir(parents=True, exist_ok=True); artifact=Path(str(binding["artifact_path"]))
if artifact.stat().st_size != int(binding["artifact_size"]) or _sha(artifact)!=binding["artifact_sha256"]: raise AdapterBlocked("artifact_binding_mismatch")
marker={"schema":"aios.trusted-runtime-release-binding.v1",**binding}
marker_path=p["release_dir"]/"DEPLOYMENT-BINDING.json"
if p["release_dir"].exists():
if not marker_path.is_file() or json.loads(marker_path.read_text())!=marker: raise AdapterBlocked("existing_release_binding_conflict")
else:
temp=Path(tempfile.mkdtemp(prefix=p["release_dir"].name+".",dir=str(p["release_dir"].parent)))
try:
_extract(artifact,temp); (temp/"DEPLOYMENT-BINDING.json").write_text(json.dumps(marker,sort_keys=True,separators=(",",":"))+"\n"); os.rename(temp,p["release_dir"])
except Exception:
shutil.rmtree(temp,ignore_errors=True); raise
version_file=p["release_dir"] / "VERSION"
if not version_file.is_file() or version_file.read_text().strip()!=binding["version"]: raise AdapterBlocked("release_version_mismatch")
unit_source=p["release_dir"] / "deploy" / "aios-v2" / "systemd" / str(binding["production_service"])
if not unit_source.is_file(): raise AdapterBlocked("release_unit_missing")
p["unit_file"].parent.mkdir(parents=True,exist_ok=True); shutil.copyfile(unit_source,p["unit_file"])
p["current_link"].parent.mkdir(parents=True,exist_ok=True)
if p["current_link"].exists() and not p["current_link"].is_symlink(): raise AdapterBlocked("current_release_path_conflict")
if not p["current_link"].is_symlink(): p["current_link"].symlink_to(p["release_dir"])
p["state_file"].parent.mkdir(parents=True,exist_ok=True); p["state_file"].write_text(json.dumps({"version":binding["version"],"commit_oid":binding["target_oid"],"artifact_sha256":binding["artifact_sha256"],"candidate_id":binding["candidate_id"],"release_plan_sha256":binding["release_plan_sha256"],"deployment_state":"ACTIVE","production_deployed":True},sort_keys=True)+"\n")
env=(f"RELEASE_VERSION={binding['version']}\n" f"TARGET_OID={binding['target_oid']}\n"
f"ARTIFACT_SHA256={binding['artifact_sha256']}\n" f"CANDIDATE_ID={binding['candidate_id']}\n"
f"RELEASE_PLAN_SHA256={binding['release_plan_sha256']}\n" f"DEPLOYMENT_STATE=ACTIVE\n"
f"DEPLOYMENT_STATE_FILE={p['state_file']}\n" f"AIOS_V2R2C2_PORT={binding['production_port']}\n"
"AIOS_GITOPS_WRITES_ENABLED=false\n")
p["env_file"].parent.mkdir(parents=True,exist_ok=True); p["env_file"].write_text(env)
_run("systemctl","daemon-reload"); _run("systemctl","enable",str(binding["production_service"])); _run("systemctl","restart",str(binding["production_service"]))
result=_health(str(p["health_url"]),binding)
if result["status"]!="ACTIVE_VERIFIED": raise AdapterBlocked("production_health_binding_mismatch")
return result
def rollback(value: Mapping[str, Any]) -> dict[str, Any]:
binding=_binding(value); _run("systemctl","stop",str(binding["production_service"]),check=False); _run("systemctl","disable",str(binding["production_service"]),check=False)
p=_paths(binding); p["state_file"].parent.mkdir(parents=True,exist_ok=True); p["state_file"].write_text(json.dumps({"version":binding["version"],"commit_oid":binding["target_oid"],"artifact_sha256":binding["artifact_sha256"],"deployment_state":"ROLLED_BACK","production_deployed":False},sort_keys=True)+"\n")
return {"status":"ROLLED_BACK","version":binding["version"],"target_oid":binding["target_oid"],"artifact_sha256":binding["artifact_sha256"],"production_deployed":False,"deployment_state":"ROLLED_BACK"}
def main() -> int:
value=json.loads(sys.stdin.read()); operation=value.get("operation")
result=probe(value) if operation=="probe" else apply(value) if operation=="apply" else rollback(value) if operation=="rollback" else (_ for _ in ()).throw(AdapterBlocked("unsupported_operation"))
print(json.dumps(result,sort_keys=True)); return 0
if __name__=="__main__":
try: raise SystemExit(main())
except Exception as exc: print(json.dumps({"status":"FAIL_CLOSED","error":type(exc).__name__,"reason":str(exc)},sort_keys=True)); raise SystemExit(2)
@@ -0,0 +1,101 @@
#!/usr/bin/env python3
"""Canonical, fail-closed projection for trusted release execution plans.
A release plan is the only authority for candidate, source, artifact, policy,
rollback, and release identity. Callers may not provide executable paths or
replace individual fields after the plan has been accepted.
"""
from __future__ import annotations
import hashlib, json, re, stat
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Mapping
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
PLAN_ID_RE = re.compile(r"^sha256:[0-9a-f]{64}$")
IDENTIFIER_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.:-]{2,255}$")
TOP_LEVEL_KEYS = frozenset({"schema","version","candidate_id","source_commit","tree_oid","artifact_sha256","manifest_sha256","package_closure_sha256","deployment_plan_sha256","policy_sha256","release_plan_id","approval_plan_id","evidence_closure_oid","campaign_authorization_sha256","campaign_generation","artifact","rollback","baseline","adapter_policy","release_plan_sha256"})
FORBIDDEN_KEYS = frozenset({"command","commands","executable","exec","shell","argv","adapter_command","tag_command","deploy_command","registry_command","baseline_command","sudo","force_push","direct_main_push"})
class PlanError(ValueError):
"""A release plan is malformed, inconsistent, or unsafe."""
def canonical_json(value: object) -> bytes:
return (json.dumps(value, sort_keys=True, separators=(",", ":")) + "\n").encode("utf-8")
def _sha(value: Any, label: str) -> str:
if not isinstance(value, str) or not SHA_RE.fullmatch(value): raise PlanError(f"{label}_invalid")
return value
def _oid(value: Any, label: str) -> str:
if not isinstance(value, str) or not OID_RE.fullmatch(value): raise PlanError(f"{label}_invalid")
return value
def _walk_forbidden(value: Any) -> None:
if isinstance(value, Mapping):
for key, item in value.items():
if str(key).lower() in FORBIDDEN_KEYS: raise PlanError(f"forbidden_plan_key:{key}")
_walk_forbidden(item)
elif isinstance(value, list):
for item in value: _walk_forbidden(item)
def _exact_mapping(value: Any, required: set[str], label: str) -> Mapping[str, Any]:
if not isinstance(value, Mapping) or set(value) != required: raise PlanError(f"{label}_fields_invalid")
return value
def _secure_json(path: Path) -> dict[str, Any]:
try: st = path.lstat()
except OSError as exc: raise PlanError(f"plan_unreadable:{path}") from exc
if path.is_symlink() or not stat.S_ISREG(st.st_mode): raise PlanError("plan_must_be_regular_file")
if st.st_uid != 0 or st.st_mode & 0o022: raise PlanError("plan_ownership_or_mode_invalid")
try: value = json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError) as exc: raise PlanError("plan_json_invalid") from exc
if not isinstance(value, dict): raise PlanError("plan_object_required")
return value
@dataclass(frozen=True)
class TrustedReleasePlan:
value: Mapping[str, Any]
release_plan_sha256: str
@classmethod
def from_mapping(cls, raw: Mapping[str, Any]) -> "TrustedReleasePlan":
if not isinstance(raw, Mapping) or set(raw) - TOP_LEVEL_KEYS: raise PlanError("release_plan_fields_invalid")
required = TOP_LEVEL_KEYS - {"release_plan_sha256"}
if set(raw) not in (required, TOP_LEVEL_KEYS): raise PlanError("release_plan_fields_incomplete")
_walk_forbidden(raw)
if raw.get("schema") != "aios.trusted-release-execution-plan.v1": raise PlanError("release_plan_schema_invalid")
version = raw.get("version")
if not isinstance(version, str) or not re.fullmatch(r"V[0-9]+R[0-9]+C[0-9]+", version): raise PlanError("version_invalid")
if not isinstance(raw.get("candidate_id"), str) or not IDENTIFIER_RE.fullmatch(raw["candidate_id"]): raise PlanError("candidate_id_invalid")
for key in ("source_commit","tree_oid","evidence_closure_oid"): _oid(raw.get(key), key)
for key in ("artifact_sha256","manifest_sha256","package_closure_sha256","deployment_plan_sha256","policy_sha256","campaign_authorization_sha256"): _sha(raw.get(key), key)
for key in ("release_plan_id","approval_plan_id"):
if not isinstance(raw.get(key), str) or not PLAN_ID_RE.fullmatch(raw[key]): raise PlanError(f"{key}_invalid")
if not isinstance(raw.get("campaign_generation"), int) or raw["campaign_generation"] < 1: raise PlanError("campaign_generation_invalid")
artifact = _exact_mapping(raw.get("artifact"), {"path","size"}, "artifact")
if not isinstance(artifact["path"], str) or not artifact["path"].startswith("/") or "\\" in artifact["path"]: raise PlanError("artifact_path_invalid")
if not isinstance(artifact["size"], int) or artifact["size"] <= 0: raise PlanError("artifact_size_invalid")
rollback = _exact_mapping(raw.get("rollback"), {"version","target_oid","artifact_sha256"}, "rollback")
if not isinstance(rollback["version"], str) or not rollback["version"] or rollback["version"] == version: raise PlanError("rollback_version_invalid")
_oid(rollback["target_oid"], "rollback_target_oid"); _sha(rollback["artifact_sha256"], "rollback_artifact_sha256")
baseline = _exact_mapping(raw.get("baseline"), {"version","target_oid"}, "baseline")
if baseline["version"] != rollback["version"]: raise PlanError("baseline_rollback_version_mismatch")
_oid(baseline["target_oid"], "baseline_target_oid")
adapter = _exact_mapping(raw.get("adapter_policy"), {"policy_id","service","port"}, "adapter_policy")
if adapter["policy_id"] != "aios.production-adapter.allowlisted.v1": raise PlanError("adapter_policy_invalid")
if not isinstance(adapter["service"], str) or not IDENTIFIER_RE.fullmatch(adapter["service"]): raise PlanError("adapter_service_invalid")
if not isinstance(adapter["port"], int) or not 1024 <= adapter["port"] <= 65535: raise PlanError("adapter_port_invalid")
payload = {key: raw[key] for key in TOP_LEVEL_KEYS if key != "release_plan_sha256"}
digest = hashlib.sha256(canonical_json(payload)).hexdigest()
if raw.get("release_plan_sha256") is not None and raw["release_plan_sha256"] != digest: raise PlanError("release_plan_sha256_mismatch")
return cls(value=dict(payload), release_plan_sha256=digest)
@classmethod
def load(cls, path: Path) -> "TrustedReleasePlan": return cls.from_mapping(_secure_json(path))
@property
def candidate_binding_sha256(self) -> str:
value = {key: self.value[key] for key in ("candidate_id","source_commit","tree_oid","artifact_sha256","manifest_sha256")}
return hashlib.sha256(canonical_json(value)).hexdigest()
def as_dict(self) -> dict[str, Any]: return {**dict(self.value),"release_plan_sha256":self.release_plan_sha256,"candidate_binding_sha256":self.candidate_binding_sha256}
def assert_binding(self, binding: Mapping[str, Any]) -> None:
expected={"version":self.value["version"],"target_oid":self.value["source_commit"],"evidence_closure_oid":self.value["evidence_closure_oid"],"artifact_sha256":self.value["artifact_sha256"],"artifact_size":self.value["artifact"]["size"],"approval_plan_id":self.value["approval_plan_id"]}
for key,value in expected.items():
if binding.get(key) != value: raise PlanError(f"binding_plan_mismatch:{key}")
if binding.get("candidate_id") is not None and binding.get("candidate_id") != self.value["candidate_id"]:
raise PlanError("binding_plan_mismatch:candidate_id")
def assert_artifact(self, artifact: Mapping[str, Any]) -> None:
expected=self.value["artifact"]
if artifact.get("sha256") != self.value["artifact_sha256"] or artifact.get("size") != expected["size"]: raise PlanError("artifact_plan_mismatch")
if artifact.get("manifest_sha256") != self.value["manifest_sha256"]: raise PlanError("manifest_plan_mismatch")
if artifact.get("package_closure_sha256") != self.value["package_closure_sha256"]: raise PlanError("package_closure_plan_mismatch")
@@ -0,0 +1,5 @@
"""V2R2C2 production health entry point; all identity is plan supplied."""
from v2r1c6_privacy_tools import main
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,57 @@
{
"allowed_peer_cgroups": [
"/system.slice/aios-gitops-model-api-v2r2c1-staging.service"
],
"allowed_peer_executable_prefix": "/usr/bin/python3",
"allowed_peer_runtime_prefix": "/opt/aios/gitops-model-api/",
"allowed_peer_runtime_suffix": "/runtime/app/aios_model_api_staging.py",
"allowed_peer_uid": 1000,
"evidence_layout": {
"artifact": "{version}/artifact.tar.gz",
"canary": "{version}/canary.json",
"gitea": "{version}/gitea.json",
"manifest": "{version}/manifest.json",
"package_closure": "{version}/checksums.sha256",
"policy": "{version}/trusted-release-policy.json",
"rollback": "{version}/rollback.json",
"secret_scan": "{version}/secret-scan.json",
"tests": "{version}/tests.json"
},
"governed_recipes": {},
"maximum_request_age_seconds": 60,
"operation": "BuildTrustedCandidateV1",
"readiness_operation": "ProbeTrustedCandidateBuilderV1",
"recipes": {
"V2R2C2": {
"base_oid": "d2b5a8339a34a8f958a177f8a3b9e8296a57a7a6",
"candidate_id": "rc-v2r2c2-d2b5a8339a34",
"mode": "fixed",
"purpose": "AIOS_TRUSTED_RELEASE_BUILDER",
"source_commit": "d2b5a8339a34a8f958a177f8a3b9e8296a57a7a6",
"task_id": "v12-durable-trusted-release-controller-activation-20260829",
"version": "V2R2C2"
},
"V2R2C3": {
"derives_from": "V2R2C2",
"mode": "governed-derived-only",
"purpose": "AIOS_TRUSTED_RELEASE_BUILDER",
"version": "V2R2C3"
}
},
"repository": "aios/aios",
"request_schema": "aios.trusted-candidate-builder.request.v1",
"request_timeout_seconds": 30,
"required_checks": [
"authoritative_git",
"artifact",
"canary",
"gitea",
"manifest",
"package_closure",
"rollback",
"secret_scan",
"tests"
],
"response_schema": "aios.trusted-candidate-builder.response.v1",
"schema": "aios.trusted-candidate-builder-policy.v1"
}
+20
View File
@@ -0,0 +1,20 @@
{
"authoritative_git_dirs": [
"/data/aios-gitops/repo.git"
],
"cutover_links": [
"/opt/aios/v2r2c2"
],
"dropin_paths": [
"/var/lib/aios-gitops/release-only/v2r2c2-d2b5a8339a34/payload/deploy/aios-v2/systemd/aios-v2r2c2-production.service"
],
"release_roots": [
"/var/lib/aios-gitops/release-only"
],
"repository_identities": [
"aios/aios"
],
"services": [
"aios-v2r2c2-production.service"
]
}
@@ -1,36 +0,0 @@
-- R4.2 P0-B durable Message/Handoff Fabric.
-- Forward-only. The existing 20260707 message_outbox remains a compatibility source.
CREATE TABLE IF NOT EXISTS message_handoff (
envelope_id TEXT PRIMARY KEY,
stream_key TEXT NOT NULL,
source TEXT NOT NULL,
target TEXT NOT NULL,
message_type TEXT NOT NULL,
payload_json TEXT NOT NULL,
payload_sha256 TEXT NOT NULL,
tenant_id TEXT,
user_id TEXT,
project_id TEXT,
conversation_id TEXT,
task_id TEXT,
run_id TEXT,
causation_id TEXT,
sequence INTEGER NOT NULL CHECK(sequence >= 1),
capability_scope_json TEXT NOT NULL,
confidentiality_scope TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
expires_at TEXT,
created_at TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'PENDING',
delivered_at TEXT,
acked_at TEXT,
UNIQUE(stream_key, idempotency_key),
UNIQUE(stream_key, sequence)
);
CREATE INDEX IF NOT EXISTS idx_message_handoff_target_status
ON message_handoff(target, status, created_at);
CREATE INDEX IF NOT EXISTS idx_message_handoff_scope
ON message_handoff(tenant_id, project_id, user_id, conversation_id, run_id);
@@ -0,0 +1,28 @@
[Unit]
Description=AIOS V2R2C2 governed trusted release health service
After=network.target
[Service]
Type=simple
User=aios-v2r1c7
Group=aios-v2r1c7
WorkingDirectory=/opt/aios/v2r2c2
EnvironmentFile=/etc/aios/v2r2c2.env
ExecStart=/usr/bin/python3 /opt/aios/v2r2c2/active-runtime/app/v2r2c2_health_service.py --serve --host 127.0.0.1 --port ${AIOS_V2R2C2_PORT}
Restart=on-failure
RestartSec=5
NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectControlGroups=true
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
IPAddressDeny=any
IPAddressAllow=localhost
UMask=0077
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,9 @@
#!/usr/bin/python3
"""Installed entry point for the governed trusted-builder provisioner."""
from __future__ import annotations
import runpy
import sys
sys.path.insert(0, "/opt/aios/trusted-candidate-builder/current/app")
runpy.run_module("trusted_builder_signer_provisioning", run_name="__main__")
@@ -1,19 +1,20 @@
[Unit]
Description=AIOS owner-controlled Task retry-extension API
After=aios-agent-task-api.service
Wants=aios-agent-task-api.service
Description=AIOS owner-controlled Task initial execution projection API
After=aios-agent-task-api.service aios-chatgpt-bridge-v2-staging.service
Wants=aios-agent-task-api.service aios-chatgpt-bridge-v2-staging.service
[Service]
Type=simple
User=aios-agent-task
Group=aios-agent-task
LoadCredential=agent_task_control_token:/etc/aios-gitops/agent-task-control.token
EnvironmentFile=/etc/aios-gitops/agent-task-retry-extension.env
Environment=AIOS_AGENT_TASK_DB=/var/lib/aios-agent-task/state.sqlite3
Environment=AIOS_AGENT_TASK_CONTROL_TOKEN_FILE=%d/agent_task_control_token
Environment=AIOS_TASK_RETRY_HOST=127.0.0.1
Environment=AIOS_TASK_RETRY_PORT=18941
ExecStart=/usr/bin/python3 -B /opt/aios/current/active-runtime/app/aios_agent_task_retry_extension.py
Environment=AIOS_GOVERNED_WRITE_API_URL=http://127.0.0.1:18892
Environment=AIOS_AGENT_TASK_INITIAL_EXECUTION_PROJECTION_AUTHORIZATION_ID=owner-initial-execution-projection-recovery-20260820
Environment=AIOS_TASK_INITIAL_EXECUTION_PROJECTION_HOST=127.0.0.1
Environment=AIOS_TASK_INITIAL_EXECUTION_PROJECTION_PORT=18943
ExecStart=/usr/bin/python3 -B /opt/aios/current/active-runtime/app/aios_agent_task_initial_execution_projection.py
Restart=on-failure
RestartSec=2
NoNewPrivileges=true
@@ -15,17 +15,9 @@ ExecStart=/usr/bin/python3 /usr/libexec/aios-approved-release-executor/current/a
--authorization ${AIOS_RELEASE_AUTHORIZATION} \
--repository ${AIOS_GITOPS_AUTHORITATIVE_GIT_DIR} \
--state-root ${AIOS_RELEASE_EXECUTION_STATE_ROOT} \
--artifact ${AIOS_RELEASE_ARTIFACT} \
--artifact-size ${AIOS_RELEASE_ARTIFACT_SIZE} \
--version ${AIOS_RELEASE_VERSION} \
--target-oid ${AIOS_RELEASE_TARGET_OID} \
--evidence-closure-oid ${AIOS_RELEASE_EVIDENCE_CLOSURE_OID} \
--artifact-sha256 ${AIOS_RELEASE_ARTIFACT_SHA256} \
--approval-plan-id ${AIOS_RELEASE_APPROVAL_PLAN_ID} \
--production-service aios-v2r1c7-production.service \
--production-port 18927 \
--release-plan ${AIOS_RELEASE_PLAN} \
--tag-adapter /usr/bin/python3 /usr/libexec/aios-approved-release-executor/current/active-runtime/app/governed_tag_adapter.py \
--deploy-adapter /usr/bin/python3 /usr/libexec/aios-approved-release-executor/current/active-runtime/app/v2r1c7_production_adapter.py \
--deploy-adapter /usr/bin/python3 /usr/libexec/aios-approved-release-executor/current/active-runtime/app/trusted_production_adapter.py \
--registry-adapter /usr/bin/env AIOS_RELEASE_METADATA_MODE=registry /usr/bin/python3 /usr/libexec/aios-approved-release-executor/current/active-runtime/app/release_metadata_adapter.py \
--baseline-adapter /usr/bin/env AIOS_RELEASE_METADATA_MODE=baseline /usr/bin/python3 /usr/libexec/aios-approved-release-executor/current/active-runtime/app/release_metadata_adapter.py
NoNewPrivileges=true
@@ -4,19 +4,32 @@ After=network-online.target
[Service]
Type=simple
User=aios-bridge-staging
Group=aios-bridge-staging
WorkingDirectory=/opt/aios/gitops-control-plane/current
User=aiman
Group=gitops
PermissionsStartOnly=true
WorkingDirectory=@RUNTIME_ROOT@
Environment=AIOS_BRIDGE_SCHEMA_VERSION=aios.chatgpt-gitops-bridge.v2
Environment=AIOS_GITOPS_WRITES_ENABLED=false
Environment=AIOS_GITOPS_DURABLE_JOB_EXECUTION_ENABLED=true
Environment=AIOS_GITOPS_WRITES_ENABLED=false
Environment=AIOS_GITEA_API_BASE=https://g.cnnetmkt.top/git/api/v1
Environment=AIOS_GITEA_TOKEN_FILE=/run/credentials/aios-chatgpt-bridge-v2-staging.service/gitea_token
Environment=AIOS_GITEA_REMOTE=gitea-http
Environment=AIOS_GITOPS_BASELINE_AUTHORITY=/var/lib/aios-gitops/model-api-staging/baseline/current.json
Environment=AIOS_CONTROL_PLANE_TARGET_VERSION=V2R2C1
Environment=AIOS_CONTROL_PLANE_TARGET_OID=@SOURCE_OID@
Environment=AIOS_CONTROL_PLANE_ARTIFACT_SHA256=@ARTIFACT_SHA256@
Environment=PYTHONPATH=@RUNTIME_ROOT@:@RUNTIME_ROOT@/app:@RUNTIME_ROOT@/runtime-agent-os
Environment=AIOS_DEPLOYMENT_STATE=STAGING
ExecStart=/usr/bin/python3 /opt/aios/gitops-control-plane/current/bridge-v2-staging-server.py
LoadCredential=gitea_token:/run/credentials/aios-governance-executor.service/gitea-governance-executor-token
ExecStartPre=/usr/bin/install -d -o aiman -g gitops -m 0750 /var/lib/aios-gitops/bridge-v2-staging
ExecStartPre=/usr/bin/setfacl -m u:aiman:--x /usr/local/libexec
ExecStart=/usr/bin/python3 -B @RUNTIME_ROOT@/app/aios_chatgpt_bridge_v2_server.py
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/aios-gitops/bridge-v2-staging
ReadWritePaths=/var/lib/aios-gitops/bridge-v2-staging /data/aios-gitops/repo.git /data/aios-gitops/repo.git/worktrees
Restart=on-failure
[Install]
@@ -0,0 +1,14 @@
# Non-secret activation template for a verified immutable model-api package.
# Install only through the governed runtime activation procedure.
[Service]
Environment=AIOS_GITOPS_WORKSPACE=/opt/aios/gitops-model-api/<VERIFIED_PACKAGE_OID>/runtime
Environment=AIOS_TRUSTED_CANDIDATE_BUILDER_ENABLED=true
Environment=AIOS_TRUSTED_CANDIDATE_BUILDER_SOCKET=/run/aios-trusted-candidate-builder.sock
Environment=AIOS_TRUSTED_CANDIDATE_BUILDER_POLICY=/opt/aios/gitops-model-api/<VERIFIED_PACKAGE_OID>/runtime/config/gitops/trusted-candidate-builder-policy.json
Environment=AIOS_TRUSTED_CANDIDATE_BUILDER_PUBLIC_KEYRING=/etc/aios-gitops/trusted-builder/public-keyring.json
Environment=AIOS_TRUSTED_CANDIDATE_BUILDER_CREDENTIALS_DIRECTORY=%d
Environment=AIOS_SILENT_RELEASE_TRUSTED_POLICY=/opt/aios/gitops-model-api/<VERIFIED_PACKAGE_OID>/runtime/config/gitops/trusted-release-policy.json
LoadCredential=builder_hmac:/etc/aios-gitops/credential-references/trusted-candidate-builder-hmac
# The credential source above is a reference only; no secret value belongs here.
# Keep execution shadow-disabled and do not add Owner/deploy/tag/baseline switches.
Environment=AIOS_SILENT_RELEASE_EXECUTION_ENABLED=false
@@ -1,33 +0,0 @@
[Unit]
Description=AIOS uncredentialed deterministic Planner/Executor
After=local-fs.target
[Service]
Type=simple
User=aios-engineering-executor
Group=aios-executor-exchange
ExecStart=/usr/bin/python3 -B /opt/aios/current/active-runtime/app/aios_isolated_executor.py --root /var/lib/aios-online-agent-executor
Restart=always
RestartSec=2
NoNewPrivileges=true
PrivateTmp=true
PrivateNetwork=true
ProtectSystem=strict
ProtectHome=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
LockPersonality=true
RestrictSUIDSGID=true
RestrictRealtime=true
RestrictNamespaces=true
MemoryDenyWriteExecute=true
RestrictAddressFamilies=AF_UNIX AF_INET
IPAddressDeny=any
IPAddressAllow=localhost
InaccessiblePaths=/run/credentials
ReadWritePaths=/var/lib/aios-online-agent-executor
UMask=0027
[Install]
WantedBy=multi-user.target
@@ -1,46 +0,0 @@
[Unit]
Description=AIOS governed Online Agent Worker
After=aios-agent-task-api.service aios-chatgpt-bridge-v2-staging.service
Wants=aios-agent-task-api.service aios-chatgpt-bridge-v2-staging.service
[Service]
Type=simple
User=aios-engineering-worker
Group=aios-engineering-worker
SupplementaryGroups=gitops aios-executor-exchange
LoadCredentialEncrypted=agent_token:/etc/credstore.encrypted/aios-online-agent-worker-20260815/agent_token
Environment=PYTHONUNBUFFERED=1
Environment=AIOS_AGENT_TASK_API_URL=http://127.0.0.1:18940
Environment=AIOS_GOVERNED_WRITE_API_URL=http://127.0.0.1:18892
Environment=AIOS_AGENT_ID=online-engineering-worker
Environment=AIOS_AGENT_TYPE=trusted-online-engineering-worker
Environment=AIOS_AGENT_VERSION=2026-08-17.2-durable
Environment=AIOS_WORKER_CAPABILITIES=git.read,git.test,governed.write.branch,governed.write.file
Environment=AIOS_AUTHORITY_GIT_DIR=/data/aios-gitops/repo.git
Environment=AIOS_ISOLATED_EXECUTOR_ROOT=/var/lib/aios-online-agent-executor
Environment=AIOS_WORKER_STATE_ROOT=/var/lib/aios-online-agent-worker
Environment=AIOS_WORKER_CHECKPOINT_DB=/var/lib/aios-online-agent-worker/checkpoints.sqlite3
Environment=AIOS_WORKER_WORKSPACE_ROOT=/var/lib/aios-online-agent-worker/workspaces
ExecStart=/usr/bin/python3 -B /opt/aios/current/active-runtime/app/aios_online_agent_worker.py
Restart=always
RestartSec=5
TimeoutStopSec=20
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
LockPersonality=true
RestrictSUIDSGID=true
RestrictRealtime=true
RestrictNamespaces=true
MemoryDenyWriteExecute=true
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
ReadOnlyPaths=/data/aios-gitops/repo.git
ReadWritePaths=/var/lib/aios-online-agent-worker
UMask=0027
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,44 @@
[Unit]
Description=AIOS trusted candidate builder
Requires=aios-trusted-candidate-builder.socket
After=aios-trusted-candidate-builder.socket
[Service]
Type=simple
User=root
Group=root
UMask=0077
LoadCredentialEncrypted=signing_private_key:/etc/credstore.encrypted/aios-trusted-candidate-builder/signing_private_key
LoadCredentialEncrypted=builder_hmac:/etc/credstore.encrypted/aios-trusted-candidate-builder/builder_hmac
Environment=AIOS_TRUSTED_BUILDER_POLICY=/opt/aios/trusted-candidate-builder/current/config/gitops/trusted-candidate-builder-policy.json
Environment=AIOS_TRUSTED_BUILDER_GIT_DIR=/data/aios-gitops/repo.git
Environment=AIOS_TRUSTED_BUILDER_EVIDENCE_ROOT=/var/lib/aios-trusted-candidate-builder/evidence
Environment=AIOS_TRUSTED_BUILDER_NONCE_LEDGER=/var/lib/aios-trusted-candidate-builder/nonce-ledger.json
Environment=AIOS_TRUSTED_BUILDER_KEY_ID=builder-attestation-ed25519-g1
Environment=AIOS_TRUSTED_BUILDER_KEY_GENERATION=1
Environment=PYTHONDONTWRITEBYTECODE=1
ExecStart=/usr/bin/python3 -B /opt/aios/trusted-candidate-builder/current/app/aios_trusted_candidate_builder.py --socket-fd 3 --policy ${AIOS_TRUSTED_BUILDER_POLICY}
Restart=on-failure
RestartSec=2
StateDirectory=aios-trusted-candidate-builder
StateDirectoryMode=0700
NoNewPrivileges=true
PrivateTmp=true
PrivateNetwork=true
ProtectSystem=strict
ProtectHome=true
ProtectControlGroups=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectKernelLogs=true
RestrictAddressFamilies=AF_UNIX
LockPersonality=true
MemoryDenyWriteExecute=true
RestrictSUIDSGID=true
SystemCallArchitectures=native
CapabilityBoundingSet=
ReadOnlyPaths=/opt/aios/trusted-candidate-builder /data/aios-gitops/repo.git /var/lib/aios-trusted-candidate-builder/evidence
ReadWritePaths=/var/lib/aios-trusted-candidate-builder
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,13 @@
[Unit]
Description=AIOS trusted candidate builder AF_UNIX socket
[Socket]
ListenStream=/run/aios-trusted-candidate-builder.sock
SocketUser=root
SocketGroup=gitops
SocketMode=0660
RemoveOnStop=yes
Accept=no
[Install]
WantedBy=sockets.target
@@ -1,4 +0,0 @@
g aios-executor-exchange -
u aios-engineering-executor - "AIOS isolated engineering executor" /var/lib/aios-online-agent-executor /usr/sbin/nologin
m aios-engineering-executor aios-executor-exchange
m aios-engineering-worker aios-executor-exchange
@@ -1,5 +0,0 @@
d /var/lib/aios-online-agent-executor 2770 aios-engineering-executor aios-executor-exchange -
d /var/lib/aios-online-agent-executor/requests 2770 aios-engineering-worker aios-executor-exchange -
d /var/lib/aios-online-agent-executor/results 2770 aios-engineering-executor aios-executor-exchange -
d /var/lib/aios-online-agent-executor/failed 2770 aios-engineering-executor aios-executor-exchange -
d /var/lib/aios-online-agent-executor/sources 2770 aios-engineering-worker aios-executor-exchange -
@@ -0,0 +1,57 @@
{
"activation": {
"credential_installation": false,
"daemon_reload": true,
"enable_units": [
"aios-trusted-candidate-builder.socket"
],
"service_start": "socket-activation-only",
"signer_invocation": false
},
"files": [
{
"install_mode": "0644",
"source_mode": "0644",
"source_path": "active-runtime/app/trusted_builder_signer_provisioning.py"
},
{
"install_mode": "0644",
"source_mode": "0644",
"source_path": "active-runtime/app/aios_trusted_candidate_builder.py"
},
{
"install_mode": "0644",
"source_mode": "0644",
"source_path": "config/gitops/trusted-candidate-builder-policy.json"
},
{
"install_mode": "0644",
"source_mode": "0644",
"source_path": "deploy/systemd/aios-trusted-candidate-builder.service"
},
{
"install_mode": "0644",
"source_mode": "0644",
"source_path": "deploy/systemd/aios-trusted-candidate-builder.socket"
},
{
"install_mode": "0644",
"source_mode": "0644",
"source_path": "deploy/systemd/aios-gitops-model-api-trusted-builder.conf.example"
},
{
"install_mode": "0755",
"source_mode": "0644",
"source_path": "deploy/libexec/aios-trusted-builder-signer-provisioner"
},
{
"install_mode": "0644",
"source_mode": "0644",
"source_path": "deploy/trusted-builder/aios-trusted-builder-package-manifest.json"
}
],
"package_name": "aios-trusted-candidate-builder",
"schema": "aios.trusted-builder-package.v1",
"source_root": "/opt/aios/trusted-candidate-builder/current",
"version": 1
}
@@ -0,0 +1,79 @@
# ADR: Bounded Immutable Repository Discovery
- Status: Implemented locally; pending governed publication and live runtime
deployment
- Date: 2026-09-02
- Scope: AIOS Repository Discovery P0
## Context
Repository discovery was needed to inspect unknown paths and compare exact
repository snapshots, while the ordinary SSH route was not an authenticated
execution plane. The discovery surface must not turn a readable runtime into a
general-purpose shell, grep, checkout, or Git writer. It also must remain safe
when a branch or tag moves between requests and when a diff is large.
## Decision
Implement a repository-scoped, read-only module behind the formal Bridge and
Actions read routes:
1. Verify the exact `aios/aios` identity before every request.
2. Resolve `HEAD`, branch, tag, or full commit input to one full commit OID;
reject ambiguous short names and revision expressions.
3. Invoke only fixed read-only Git subcommands (`rev-parse`, `ls-tree`,
`cat-file`, and `diff`) with `shell=False`, disabled prompts, and bounded
output. Stream NUL-delimited tree and diff records rather than collecting a
complete tree or patch.
4. Enforce normalized relative paths, repository allowlisting, depth/page/
match/byte limits, binary-source exclusion, and no patch response.
5. Sign pagination cursors with an HMAC over the operation, repository,
resolved OID, complete normalized query, and offset. A cursor for another
snapshot or query is invalid.
6. Return resolved commit identity for every read and blob identity for file
reads. Keep the four operations read-only and non-consequential at the
Bridge contract boundary.
7. Keep the mutating implementation workflow separate: Task and TaskPrincipal
admission, claim/heartbeat, WriterLease, CAS, and `commitFilesGoverned` are
still mandatory for source changes.
The canonical runtime paths are `/v2/repository/*`; the Actions adapter maps
the same contract to `/api/repository/*`. The OpenAPI document records the
mapping without copying a Path Item or duplicating operation IDs.
## Alternatives rejected
- Retrying arbitrary SSH credentials or inventing a new network profile would
bypass the supported execution-plane boundary.
- User-controlled `grep`, shell commands, checkout paths, or direct bare-repo
reads would make the repository and host boundary non-deterministic.
- Reusing the write route would introduce unnecessary Task/lease/credential
authority into a read-only request and make a read endpoint a write probe.
- Returning a full patch or collecting all Git output before pagination would
violate the bounded-output requirement for large repositories.
## Consequences
The API callers must carry the returned OID/cursor and must not assume that a
floating ref remains unchanged. Search may return fewer results when its byte
bound is exhausted, and a diff is an index rather than a patch. Cursor secret
configuration becomes a required runtime dependency for paginated operations.
The isolated implementation is testable against a disposable bare repository.
Windows can compile and run the portable discovery tests, but importing the
full Bridge/Actions runtime requires POSIX `fcntl`; Linux HTTP and locking
integration remains a separate verification item.
At the time of this decision, live HTTPS authority/runtime readback was
available, but the live OpenAPI/capability surface did not advertise these four
discovery operations and no authorized P0 Task envelope was present in the
execution context. Therefore this ADR does not claim a governed commit,
remote publication, or deployed runtime.
## Verification gate
This decision is complete for local implementation only after Python compile,
discovery unit/large-diff tests, and Bridge/OpenAPI/Actions drift tests pass.
It becomes live only after a formal Task, TaskPrincipal, lease, governed
commit, and independent authority/remote/API OID readback prove the exact same
commit, with all frozen write and staging invariants unchanged.
@@ -0,0 +1,102 @@
# ADR: P0-0 Task Admission Bootstrap without weakening TaskPrincipal
Date: 2026-09-03
Status: Candidate for semantic port
## Context
The live control plane can expose governed writer operations while still being
unable to create the Agent Task/TaskLease required by TaskPrincipal. This is a
bootstrap deadlock, not a reason to enable global repository writes.
The safety invariant is:
```text
Owner / durable delegation
-> Task admission grant
-> real Agent Task
-> claim + heartbeat + TaskLease
-> TaskPrincipal
-> WriterLease
-> governed commit
```
`GovernedTaskBootstrap.create()` may create a GitOps binding/Job. It is not, by
itself, evidence of an Agent Task claim or TaskLease.
## Decision
Introduce a narrow Task Admission Bootstrap substrate with three operations:
- `getTaskAdmissionEnvelope`
- `submitTaskAdmissionAuthorization`
- `getTaskAdmissionStatus`
The envelope normalizes and binds repository, source OID, target branch,
generation, execution digest, capabilities, file allowlist, risk, policy digest,
expiry and nonce.
Authorization is accepted only from:
1. an already-verified `OwnerDelegationEpoch`; or
2. an Owner signature over the exact canonical envelope.
There is deliberately no permissive default authorization backend.
The derived `TaskAdmissionGrant` can only narrow the parent authorization.
Production, tag, baseline, main, global-write, force-push, runtime-anchor and
arbitrary-shell capabilities are explicitly rejected.
The grant is passed to the existing Agent Task control plane via an injected
`TaskAdmissionPort`. This component never writes the Task database.
The production adapter accepts only a service-managed official Task transport;
it never returns its control credential. It first reconciles the exact task
identity, then POSTs the official `/v1/tasks` contract only if absent, and
reconciles an ambiguous response before reporting `UNKNOWN`. Task receipts use
actual `QUEUED`, `CLAIMED`, or `RUNNING` state, never a synthetic ready state.
Durable epochs are resolved through an injected provider that receives the
authenticated caller identity from trusted middleware. A body-supplied epoch
identifier is only a lookup key. The existing durable preparation authorization
does not become file-write or TaskPrincipal authority.
TaskPrincipal is not used to bootstrap the Task. It remains downstream of a
real claim/TaskLease.
## Crash / replay semantics
Envelope creation is idempotent by exact request digest.
Authorization/admission is durable and exact replay returns the same Task
receipt. A SQLite `BEGIN IMMEDIATE` claim persists the immutable grant and its
authorization-proof digest before the Task side effect. Cross-instance callers
observe the durable envelope winner; a different proof cannot replace the
stored grant. The live Task API retains the second exact-identity boundary.
A crash after authorization but before Task API admission must resume the same
envelope/grant identity. It must not create a replacement task or widen scope.
## Integration rule for current live authority
The source snapshot used for this candidate is not proven byte-identical to the
2026-09-03 live authority. Therefore the code is a semantic-port candidate.
The live integration must:
- preserve existing TaskPrincipal verification;
- adapt `TaskAdmissionPort` to the actual Agent Task API;
- adapt `OwnerAuthorizationBackend` to the actual trusted Owner/delegation
verifier;
- expose admission routes outside the TaskPrincipal gate, because the
cryptographic/delegation proof is the bootstrap authority;
- never turn the runtime anchor into a writer;
- never use Gitea/SSH/downstream credentials as Owner authority;
- publish OpenAPI/capability metadata only when handlers are actually wired.
## Consequences
The control plane can become unattended after one durable Owner delegation
epoch, while retaining fail-closed boundaries. If no valid delegation exists,
the only legitimate stop is an already-generated canonical signing envelope
awaiting the Owner's cryptographic signature.
+205 -213
View File
@@ -240,115 +240,222 @@ components:
additionalProperties: false
properties:
candidate_id: {type: string}
BridgeWriterLeaseRecoveryRequest:
RepositoryTreeEntry:
type: object
additionalProperties: false
required: [schema, task_id, repository, branch_mode, base_oid, expected_branch_oid, target_branch, generation, lease_seconds]
required: [path, type, object_oid]
properties:
schema:
type: string
enum: [aios.task-writer-lease.v2, aios.task-writer-lease-request.v2]
task_id: {type: string, minLength: 1, maxLength: 256}
job_id: {type: string, pattern: '^job-[A-Za-z0-9][A-Za-z0-9._-]{7,255}$'}
repository: {type: string, const: aios/aios}
branch_mode: {type: string, enum: [CREATE_NEW, CONTINUE_EXISTING]}
base_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
expected_branch_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
target_branch: {type: string, pattern: '^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$'}
generation: {type: integer, minimum: 1}
lease_seconds: {type: integer, minimum: 1, maximum: 86400}
BridgeWriterLeaseResponse:
type: object
additionalProperties: false
required: [schema, task_id, job_id, repository, branch_mode, base_oid, expected_branch_oid, target_branch, generation, lease_seconds, create_target_branch, worktree_mode, cas_ref_update_required]
properties:
schema: {type: string, enum: [aios.task-writer-lease.v2]}
task_id: {type: string}
job_id: {type: string, pattern: '^job-[A-Za-z0-9][A-Za-z0-9._-]{7,255}$'}
repository: {type: string, const: aios/aios}
branch_mode: {type: string, enum: [CREATE_NEW, CONTINUE_EXISTING]}
base_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
expected_branch_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
target_branch: {type: string, pattern: '^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$'}
generation: {type: integer, minimum: 1}
lease_seconds: {type: integer, minimum: 1, maximum: 86400}
create_target_branch: {type: boolean}
worktree_mode: {type: string, enum: [detached_at_base_oid, detached_at_expected_head]}
cas_ref_update_required: {type: boolean, const: true}
lease_expires_at: {type: string, format: date-time}
BridgeWriterLeaseRecoveryResponse:
type: object
additionalProperties: false
required: [schema, task_id, job_id, repository, branch_mode, base_oid, expected_branch_oid, target_branch, generation, lease_seconds, lease_expires_at, create_target_branch, worktree_mode, cas_ref_update_required, recovered, bootstrapped, idempotent, runtime_rehydrated, durable_branch_lease_present]
properties:
schema: {type: string, enum: [aios.task-writer-lease.v2]}
task_id: {type: string}
job_id: {type: string, pattern: '^job-[A-Za-z0-9][A-Za-z0-9._-]{7,255}$'}
repository: {type: string, const: aios/aios}
branch_mode: {type: string, enum: [CREATE_NEW, CONTINUE_EXISTING]}
base_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
expected_branch_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
target_branch: {type: string, pattern: '^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$'}
generation: {type: integer, minimum: 1}
lease_seconds: {type: integer, minimum: 1, maximum: 86400}
lease_expires_at: {type: string, format: date-time}
create_target_branch: {type: boolean, const: false}
worktree_mode: {type: string, enum: [detached_at_base_oid, detached_at_expected_head]}
cas_ref_update_required: {type: boolean, const: true}
recovered: {type: boolean}
bootstrapped: {type: boolean}
idempotent: {type: boolean}
runtime_rehydrated: {type: boolean}
durable_branch_lease_present: {type: boolean}
BridgeCommitFileRequest:
type: object
additionalProperties: false
required: [task_id, repository, expected_branch_oid, target_branch, generation, idempotency_key, path, content_base64, message]
properties:
task_id: {type: string, minLength: 1}
job_id: {type: string, pattern: '^job-[A-Za-z0-9][A-Za-z0-9._-]{7,255}$'}
repository: {type: string, const: aios/aios}
expected_branch_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
target_branch: {type: string, pattern: '^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$'}
generation: {type: integer, minimum: 1}
idempotency_key: {type: string, minLength: 1, maxLength: 256}
path: {type: string, minLength: 1, maxLength: 512}
content_base64: {type: string, format: byte}
message: {type: string, minLength: 1, maxLength: 200}
BridgeCommitFilesRequest:
type: {type: string, enum: [blob, tree, commit]}
object_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
mode: {type: string}
RepositoryTreeResponse:
type: object
additionalProperties: false
required: [task_id, repository, expected_branch_oid, target_branch, generation, idempotency_key, files, message]
required: [repository, resolved_oid, entries, truncated, next_cursor]
properties:
task_id: {type: string, minLength: 1}
job_id: {type: string, pattern: '^job-[A-Za-z0-9][A-Za-z0-9._-]{7,255}$'}
repository: {type: string, const: aios/aios}
expected_branch_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
target_branch: {type: string, pattern: '^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$'}
generation: {type: integer, minimum: 1}
idempotency_key: {type: string, minLength: 1, maxLength: 256}
files: {type: object, additionalProperties: {type: string, format: byte}, minProperties: 1}
message: {type: string, minLength: 1, maxLength: 200}
BridgeCommitResponse:
resolved_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
entries: {type: array, items: {$ref: '#/components/schemas/RepositoryTreeEntry'}}
truncated: {type: boolean}
next_cursor: {type: string, nullable: true, maxLength: 4096}
RepositorySearchMatch:
type: object
additionalProperties: true
required: [job_id, commit_oid, commit_visible, branch_mode, target_branch, generation]
required: [path, line_start, line_end, blob_oid, snippet]
properties:
job_id: {type: string, pattern: '^job-[A-Za-z0-9][A-Za-z0-9._-]{7,255}$'}
commit_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
commit_visible: {type: boolean}
branch_mode: {type: string, enum: [CONTINUE_EXISTING]}
target_branch: {type: string, pattern: '^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$'}
generation: {type: integer, minimum: 1}
BridgeGovernedCapabilities:
path: {type: string, minLength: 1, maxLength: 512}
line_start: {type: integer, minimum: 1}
line_end: {type: integer, minimum: 1}
blob_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
snippet: {type: string, maxLength: 512}
RepositorySearchResponse:
type: object
required: [bridge_schema_version, governed_writes_enabled, operations, operation_contracts]
required: [repository, resolved_oid, matches, scanned_bytes, max_bytes, truncated, next_cursor]
properties:
bridge_schema_version: {type: string, const: aios.chatgpt-gitops-bridge.v2}
governed_writes_enabled: {type: boolean, const: true}
operations: {type: array, items: {type: string}}
operation_contracts: {type: object, additionalProperties: true}
repository: {type: string, const: aios/aios}
resolved_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
matches: {type: array, items: {$ref: '#/components/schemas/RepositorySearchMatch'}}
scanned_bytes: {type: integer, minimum: 0}
max_bytes: {type: integer, minimum: 1, maximum: 1048576}
binary_blobs_skipped: {type: integer, minimum: 0}
truncated: {type: boolean}
next_cursor: {type: string, nullable: true, maxLength: 4096}
RepositoryDiffEntry:
type: object
required: [path, status, additions, deletions, old_path]
properties:
path: {type: string, minLength: 1, maxLength: 512}
status: {type: string, pattern: '^[ACDMRTUX][0-9]*$'}
additions: {type: integer, nullable: true, minimum: 0}
deletions: {type: integer, nullable: true, minimum: 0}
old_path: {type: string, nullable: true, maxLength: 512}
RepositoryDiffIndexResponse:
type: object
required: [repository, resolved_from_oid, resolved_to_oid, entries, truncated, next_cursor]
properties:
repository: {type: string, const: aios/aios}
resolved_from_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
resolved_to_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
entries: {type: array, items: {$ref: '#/components/schemas/RepositoryDiffEntry'}}
truncated: {type: boolean}
next_cursor: {type: string, nullable: true, maxLength: 4096}
RepositoryFileResponse:
type: object
required: [repository, resolved_oid, blob_oid, path, content, content_base64, content_bytes, truncated]
properties:
repository: {type: string, const: aios/aios}
resolved_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
blob_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
path: {type: string, minLength: 1, maxLength: 512}
content: {type: string}
content_base64: {type: string, format: byte}
content_bytes: {type: integer, minimum: 0}
truncated: {type: boolean}
paths:
/v2/repository/tree:
get:
operationId: listRepositoryTree
summary: List one bounded page of an immutable repository tree
security: []
x-openai-isConsequential: false
x-aios-read-only: true
x-aios-bounded: true
x-aios-repository-scoped: true
x-aios-no-shell: true
x-aios-legacy-path: /api/repository/tree
parameters:
- {in: query, name: repository, required: true, schema: {type: string, const: aios/aios}}
- {in: query, name: ref, required: true, schema: {type: string, minLength: 1, maxLength: 240}}
- {in: query, name: prefix, schema: {type: string, maxLength: 512}}
- {in: query, name: depth, schema: {type: integer, minimum: 1, maximum: 32, default: 8}}
- {in: query, name: limit, required: true, schema: {type: integer, minimum: 1, maximum: 1000}}
- {in: query, name: cursor, schema: {type: string, maxLength: 4096}}
responses:
'200': {description: Immutable tree page, content: {application/json: {schema: {$ref: '#/components/schemas/RepositoryTreeResponse'}}}}
'400': {description: Invalid repository, ref, path, bounds, or cursor}
'403': {description: Repository identity is not allowed}
'404': {description: Repository ref or path not found}
'503': {description: Authoritative repository or discovery configuration unavailable}
/v2/repository/search:
get:
operationId: searchRepositorySource
summary: Search bounded source blobs at one immutable repository revision
security: []
x-openai-isConsequential: false
x-aios-read-only: true
x-aios-bounded: true
x-aios-repository-scoped: true
x-aios-no-shell: true
x-aios-legacy-path: /api/repository/search
parameters:
- {in: query, name: repository, required: true, schema: {type: string, const: aios/aios}}
- {in: query, name: ref, required: true, schema: {type: string, minLength: 1, maxLength: 240}}
- {in: query, name: query, required: true, schema: {type: string, minLength: 1, maxLength: 512}}
- {in: query, name: path_globs, style: form, explode: true, schema: {type: array, maxItems: 16, items: {type: string, maxLength: 512}}}
- {in: query, name: max_matches, required: true, schema: {type: integer, minimum: 1, maximum: 1000}}
- {in: query, name: max_bytes, required: true, schema: {type: integer, minimum: 1, maximum: 1048576}}
- {in: query, name: cursor, schema: {type: string, maxLength: 4096}}
responses:
'200': {description: Bounded source matches with blob identity, content: {application/json: {schema: {$ref: '#/components/schemas/RepositorySearchResponse'}}}}
'400': {description: Invalid repository, ref, query, path glob, bounds, or cursor}
'403': {description: Repository identity is not allowed}
'404': {description: Repository ref not found}
'503': {description: Authoritative repository or discovery configuration unavailable}
/v2/repository/diff-index:
get:
operationId: getDiffIndex
summary: List a bounded immutable diff index without returning a patch
security: []
x-openai-isConsequential: false
x-aios-read-only: true
x-aios-bounded: true
x-aios-repository-scoped: true
x-aios-no-shell: true
x-aios-legacy-path: /api/repository/diff-index
parameters:
- {in: query, name: repository, required: true, schema: {type: string, const: aios/aios}}
- {in: query, name: from, required: true, schema: {type: string, minLength: 1, maxLength: 240}}
- {in: query, name: to, required: true, schema: {type: string, minLength: 1, maxLength: 240}}
- {in: query, name: path_prefix, schema: {type: string, maxLength: 512}}
- {in: query, name: limit, required: true, schema: {type: integer, minimum: 1, maximum: 1000}}
- {in: query, name: cursor, schema: {type: string, maxLength: 4096}}
responses:
'200': {description: Bounded diff index with immutable endpoints, content: {application/json: {schema: {$ref: '#/components/schemas/RepositoryDiffIndexResponse'}}}}
'400': {description: Invalid repository, revision, path prefix, bounds, or cursor}
'403': {description: Repository identity is not allowed}
'404': {description: Repository revision not found}
'503': {description: Authoritative repository or discovery configuration unavailable}
/v2/repository/file:
get:
operationId: readRepositoryFile
summary: Read one bounded file from an immutable repository revision
security: []
x-openai-isConsequential: false
x-aios-read-only: true
x-aios-bounded: true
x-aios-repository-scoped: true
x-aios-no-shell: true
x-aios-legacy-path: /api/repository/file
parameters:
- {in: query, name: repository, required: true, schema: {type: string, const: aios/aios}}
- {in: query, name: ref, required: true, schema: {type: string, minLength: 1, maxLength: 240}}
- {in: query, name: path, required: true, schema: {type: string, minLength: 1, maxLength: 512}}
- {in: query, name: max_bytes, schema: {type: integer, minimum: 1, maximum: 1048576, default: 262144}}
responses:
'200': {description: Bounded file content with resolved commit and blob identity, content: {application/json: {schema: {$ref: '#/components/schemas/RepositoryFileResponse'}}}}
'400': {description: Invalid repository, ref, path, or bounds}
'403': {description: Repository identity is not allowed}
'404': {description: Repository ref or file not found}
'503': {description: Authoritative repository or discovery configuration unavailable}
/v2/commit-files:
post:
operationId: commitFilesGoverned
summary: Submit one atomic governed commit containing multiple files
security: [{taskPrincipalAuth: []}]
x-aios-task-principal: Bridge verifies a fresh aios.task-principal-assertion.v1 and exact task/job/repository/branch/generation/revision/attempt/execution-digest/capability binding; caller JSON cannot provide a principal and missing verification is denied.
requestBody:
required: true
content:
application/json:
schema:
type: object
additionalProperties: false
required:
- task_id
- job_id
- repository
- expected_branch_oid
- target_branch
- generation
- idempotency_key
- files
- message
properties:
task_id: {type: string}
job_id: {type: string, description: External durable Job identity for exact completed-replay reconciliation}
source_lease_epoch: {type: integer, minimum: 1}
repository: {type: string, example: aios/aios}
expected_branch_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
target_branch: {type: string, pattern: '^agent/[A-Za-z0-9._/-]+$'}
generation: {type: integer, minimum: 1}
idempotency_key: {type: string}
files:
type: array
minItems: 1
items:
type: object
additionalProperties: false
required: [path, content_base64]
properties:
path: {type: string, minLength: 1, maxLength: 512}
content_base64: {type: string}
message: {type: string, minLength: 1, maxLength: 200}
responses:
'200': {description: Governed commit job and verified branch result}
'202': {description: Existing external Job alias requires authoritative reconciliation; no replacement Job is created}
'401': {description: No verified Task principal is available}
'403': {description: Verified Task principal lacks exact task, external Job, or capability scope}
'400': {description: Contract or policy rejection}
'409': {description: Branch CAS or idempotency conflict}
'423': {description: Narrow P0 Task-principal mutation gate is frozen; assertion is not consumed}
/createReleaseIntent:
post:
operationId: createReleaseIntent
@@ -675,118 +782,3 @@ paths:
'200': {description: Safe GitOps plan}
'401': {description: Unauthorized}
x-openai-isConsequential: true
/v2/capabilities:
get:
operationId: getGovernedWriteCapabilities
summary: Read the governed Bridge v2 capability contract
responses:
'200':
description: Governed operations and their request/response contracts
content:
application/json:
schema: {$ref: '#/components/schemas/BridgeGovernedCapabilities'}
/v2/writer-lease:
post:
operationId: acquireWriterLease
summary: Acquire the exact governed writer lease for a task generation
security: [{bearerAuth: []}]
requestBody:
required: true
content:
application/json:
schema: {$ref: '#/components/schemas/BridgeWriterLeaseRecoveryRequest'}
responses:
'200':
description: Exact detached worktree lease binding
content:
application/json:
schema: {$ref: '#/components/schemas/BridgeWriterLeaseResponse'}
'400': {description: Invalid lease binding}
'409': {description: Branch, base, or generation conflict}
x-openai-isConsequential: true
/v2/writer-lease/bootstrap:
post:
operationId: bootstrapWriterLease
summary: Bootstrap a governed writer runtime without creating a durable lease
description: >
Staging-only runtime recovery. It validates the exact task, repository,
base OID, expected branch OID, target branch, and generation. It must
not create a branch, insert a branch lease, or consume a generation.
security: [{bearerAuth: []}]
requestBody:
required: true
content:
application/json:
schema: {$ref: '#/components/schemas/BridgeWriterLeaseRecoveryRequest'}
responses:
'200':
description: Runtime binding bootstrapped or returned idempotently
content:
application/json:
schema: {$ref: '#/components/schemas/BridgeWriterLeaseRecoveryResponse'}
'400': {description: Invalid lease binding}
'409': {description: A matching durable lease already exists or binding conflicts}
'503': {description: Runtime or durable state unavailable}
x-openai-isConsequential: true
/v2/writer-lease/renew:
post:
operationId: renewWriterLease
summary: Recover an expired governed writer lease without changing its binding
description: >
Renews only the current task generation and exact repository, base OID,
expected branch OID, and target branch. Durable renewal is revision-CAS
protected; runtime rehydration is reported separately.
security: [{bearerAuth: []}]
requestBody:
required: true
content:
application/json:
schema: {$ref: '#/components/schemas/BridgeWriterLeaseRecoveryRequest'}
responses:
'200':
description: Lease recovered or idempotently observed as already active
content:
application/json:
schema: {$ref: '#/components/schemas/BridgeWriterLeaseRecoveryResponse'}
'400': {description: Invalid schema, generation, or lease duration}
'409': {description: Task, base, repository, branch, generation, or durable lease conflict}
'503': {description: Durable job store or runtime unavailable}
x-openai-isConsequential: true
/v2/commit-file:
post:
operationId: commitFileGoverned
summary: Commit one file through the governed writer lease
security: [{bearerAuth: []}]
requestBody:
required: true
content:
application/json:
schema: {$ref: '#/components/schemas/BridgeCommitFileRequest'}
responses:
'200':
description: Authority and remote visibility receipt
content:
application/json:
schema: {$ref: '#/components/schemas/BridgeCommitResponse'}
'409': {description: Lease, expected-head, idempotency, or repository conflict}
'503': {description: Reconciliation required}
x-openai-isConsequential: true
/v2/commit-files:
post:
operationId: commitFilesGoverned
summary: Atomically commit a bounded set of files through the governed writer lease
security: [{bearerAuth: []}]
requestBody:
required: true
content:
application/json:
schema: {$ref: '#/components/schemas/BridgeCommitFilesRequest'}
responses:
'200':
description: Authority and remote visibility receipt
content:
application/json:
schema: {$ref: '#/components/schemas/BridgeCommitResponse'}
'409': {description: Lease, expected-head, idempotency, or repository conflict}
'503': {description: Reconciliation required}
x-openai-isConsequential: true
@@ -0,0 +1,109 @@
# AIOS Task Admission Bootstrap OpenAPI fragment.
# Integrate into the canonical Bridge OpenAPI only when the corresponding live
# handlers and authorization backend are configured. This fragment does not
# enable legacy repository mutation or change TaskPrincipal/WriterLease gates.
components:
schemas:
TaskAdmissionEnvelopeRequest:
type: object
additionalProperties: false
required:
- idempotency_key
- task_id
- repository
- authority_base_oid
- target_branch
- generation
- execution_digest
- requested_capabilities
- file_allowlist
- risk
properties:
idempotency_key: {type: string, minLength: 8, maxLength: 256}
task_id: {type: string, minLength: 3, maxLength: 180}
repository: {type: string, enum: [aios/aios]}
authority_base_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
target_branch: {type: string, pattern: '^agent/.+'}
generation: {type: integer, minimum: 1}
execution_digest: {type: string, pattern: '^[0-9a-f]{64}$'}
requested_capabilities:
type: array
minItems: 1
uniqueItems: true
items: {type: string}
file_allowlist:
type: array
minItems: 1
uniqueItems: true
items: {type: string}
risk: {type: string, enum: [low, medium]}
expires_in_seconds:
type: integer
minimum: 60
maximum: 900
TaskAdmissionAuthorizationRequest:
type: object
additionalProperties: false
required: [envelope_id, authorization_kind]
properties:
envelope_id: {type: string}
authorization_kind:
type: string
enum: [delegation_epoch, owner_signature]
delegation_epoch_id: {type: string}
owner_key_id: {type: string}
signature_base64: {type: string}
paths:
/v2/task-admission/envelope:
post:
operationId: getTaskAdmissionEnvelope
summary: Build and persist a canonical narrow Task admission envelope
description: >
No Git/Task mutation occurs. The server normalizes repository, branch,
capabilities, file allowlist, risk, policy digest, expiry and nonce.
requestBody:
required: true
content:
application/json:
schema: {$ref: '#/components/schemas/TaskAdmissionEnvelopeRequest'}
responses:
'200': {description: Canonical envelope and signing payload}
'403': {description: Requested scope exceeds bootstrap policy}
'409': {description: Idempotency key conflicts with a different request}
/v2/task-admission/authorize:
post:
operationId: submitTaskAdmissionAuthorization
summary: Verify durable delegation or Owner signature and admit one exact Task
description: >
This operation does not accept a general-purpose bearer as authority.
Authorization is proven by a verified OwnerDelegationEpoch or by the
Owner signature over the exact canonical envelope. The resulting grant
can only narrow scope and is handed to the existing Agent Task API.
A delegation identifier in caller JSON is insufficient: the backend
binds the verified parent epoch to the authenticated service caller.
requestBody:
required: true
content:
application/json:
schema: {$ref: '#/components/schemas/TaskAdmissionAuthorizationRequest'}
responses:
'200': {description: Exact QUEUED, CLAIMED, or RUNNING Task receipt or idempotent replay}
'403': {description: Invalid, expired, revoked or widened authorization}
'409': {description: Conflicting idempotent admission}
/v2/task-admission/status:
get:
operationId: getTaskAdmissionStatus
summary: Read durable Task-admission state
parameters:
- in: query
name: envelope_id
required: true
schema: {type: string}
responses:
'200': {description: Pending or admitted receipt}
'404': {description: Envelope not found}
@@ -0,0 +1,117 @@
# Repository Discovery P0 Runbook
This runbook covers the read-only repository discovery surface and the
governed workflow used to publish a change to it. It does not authorize a
production deployment, a protected-ref update, a tag operation, or a runtime
mode change.
## Safety invariants
Before any operation, record the repository identity and the exact authority
OID. The only repository identity in this contract is `aios/aios`. A floating
ref is resolved once; all subsequent reads in the response are bound to the
returned full `resolved_oid` (or `resolved_from_oid` and `resolved_to_oid` for
a diff) and, for files, the returned `blob_oid`.
The following invariants remain frozen for this P0:
```text
AIOS_GITOPS_WRITES_ENABLED=false
legacy_global_writes_enabled=false
18893 runtime_mode=IMMUTABLE_STAGING
direct bare-repository writes/pushes=forbidden
direct Task DB writes=forbidden
```
Discovery is a read-only operation. It does not acquire a WriterLease and it
does not create a Task. Any implementation change still has to use the formal
Task → TaskPrincipal → TaskLease/claim/heartbeat → WriterLease →
`commitFilesGoverned` path. A request to discovery must never be used as a
write-capability probe.
## Execution-plane preflight
1. Prefer the formal HTTPS control plane at the configured public API/Bridge
endpoint. If raw SSH fails before authentication, preserve that evidence
and do not retry arbitrary usernames, keys, ports, or shell commands.
2. Read runtime, capability, snapshot, and OpenAPI facts. Confirm that the
authority identity and OID are the expected ones, and record whether the
discovery routes are actually deployed on that plane.
3. Classify the available plane as A (network-enabled Codex profile), B
(attested online worker), C (formal HTTPS control plane), or D (existing
executor/coordinator). Select only a supported plane; do not invent a new
SSH or credential path.
4. Treat a readable runtime or a candidate branch as evidence of availability,
not as authorization to write.
The durable discovery record is
`execution-plane-discovery.json`. It must not be included in the governed
source change unless the task explicitly allowlists it.
## Read operations
The canonical Bridge routes are:
| Operation | Method | Required query fields | Optional query fields |
| --- | --- | --- | --- |
| `listRepositoryTree` | `GET /v2/repository/tree` | `repository`, `ref`, `limit` | `prefix`, `depth`, `cursor` |
| `searchRepositorySource` | `GET /v2/repository/search` | `repository`, `ref`, `query`, `max_matches`, `max_bytes` | `path_globs`, `cursor` |
| `getDiffIndex` | `GET /v2/repository/diff-index` | `repository`, `from`, `to`, `limit` | `path_prefix`, `cursor` |
| `readRepositoryFile` | `GET /v2/repository/file` | `repository`, `ref`, `path` | `max_bytes` |
The Actions adapter exposes the corresponding read-only legacy paths under
`/api/repository/*`; the OpenAPI document records each mapping with
`x-aios-legacy-path`.
Every request is bounded. Repository identity is allowlisted, refs are
limited to a branch/tag/full commit form, paths are normalized POSIX-relative
paths, and traversal, option-like paths, shell metacharacters, and ambiguous
refs fail closed. Search skips binary blobs and enforces its cumulative byte
bound. Tree, search, and diff responses are paginated. A cursor is an HMAC-
bound record containing the operation, repository, resolved OID, complete
query, and offset; never edit or manually construct one.
The diff endpoint returns an index only: status, path, optional old path, and
bounded additions/deletions. It never returns a patch body. The file endpoint
returns bounded text plus `content_base64`, `resolved_oid`, and `blob_oid`.
## Publishing an implementation change
1. Preserve the existing dirty worktree. Create a new isolated worktree at
the exact authority OID and use a task-scoped branch.
2. Run discovery unit, pagination, cursor, identity, traversal, binary,
max-byte, immutable-head, and large-diff tests. Run the Bridge/OpenAPI/
Actions drift tests and compile checks.
3. Obtain a formal governed Task and keep its exact task, job, generation,
idempotency, branch, and base-OID identities. Claim and heartbeat through
the supported control plane; do not edit the Task DB or copy a credential.
4. Acquire the Task-bound principal and WriterLease. Submit the allowlisted
source files through `commitFilesGoverned`, then reconcile the Job if the
result is uncertain.
5. Independently read back the authority Git ref, the remote Git API ref, and
the control-plane projection. Completion requires three-way equality to the
governed commit OID, plus the required deployment/acceptance/restart/
official-verification evidence for any release claim.
If the discovery routes are absent from the live OpenAPI/runtime surface, stop
at local validation. Report the implementation and tests as local evidence and
mark governed Task, WriterLease, governed commit, and post-commit readback as
`NOT_VERIFIED`.
## Failure and recovery
- `REPOSITORY_NOT_ALLOWED`, identity mismatch, invalid path/ref, invalid
cursor, or ambiguous ref is a client or security rejection; do not broaden
the allowlist.
- An unavailable authority repository or missing cursor secret is a runtime
configuration failure; do not disable cursor signing or fall back to a
workspace checkout.
- A timeout, oversized output, or large diff is bounded failure. Narrow the
prefix/page or repair the existing supported runtime, never return an
unbounded patch.
- On Windows, a missing `fcntl` means POSIX integration is unverified. Keep
the Python compile and portable unit evidence, then rerun the HTTP and file
locking probes on the supported Linux runtime.
No recovery step in this runbook resets, cleans, overwrites, force-pushes, or
auto-commits a user worktree.
@@ -0,0 +1,102 @@
# Task Admission Bootstrap P0 Runbook
## Goal
Repair the Task-admission bootstrap chain without enabling legacy/global writes.
## Preconditions
Re-read live authority, runtime mode, OpenAPI, capabilities and current Task
API. Do not assume the package source paths are current.
Required invariants:
```text
AIOS_GITOPS_WRITES_ENABLED=false
legacy_global_writes_enabled=false
18893=IMMUTABLE_STAGING
main/tag/baseline/production writes=false
```
## Semantic-port sequence
1. Locate the live Agent Task create/admission implementation.
2. Locate the current Owner/delegation signature verifier.
3. Port `aios_task_admission_bootstrap.py` as the common substrate.
4. Implement `TaskAdmissionPort.admit()` with the official Agent Task API.
5. Implement `OwnerAuthorizationBackend` with the current trusted verifier.
6. Wire the three task-admission operations into the existing Bridge.
7. Keep TaskPrincipal issuance downstream of real Task claim/TaskLease.
8. Merge the OpenAPI fragment only after the live handlers are wired.
9. Run impacted Linux/POSIX, TaskPrincipal, Bridge and writer regression.
10. Create the bootstrap-repair governed commit through the existing sanctioned
recovery path.
11. Use the repaired admission path to create the Repository Discovery P0 Task.
12. Claim/heartbeat -> TaskLease -> TaskPrincipal -> WriterLease.
13. Commit only the existing Repository Discovery allowlist through
`commitFilesGoverned`.
14. Reconcile ambiguous outcomes before any retry.
Admission uses a per-envelope, crash-releasing SQLite lock under the
admission state path's `.locks` directory. It persists the immutable
grant before an effect, then the official Task API GET reconciles any
crashed or ambiguous effect before a POST is permitted.
15. Replays revalidate the supplied proof, policy, expiry and revocation before
reading a receipt. A different otherwise-valid proof conflicts with the
persisted authorization digest; it never becomes a harmless receipt replay.
16. The Task API adapter must receive a server-resolved trusted execution plan
for the signed execution digest. It supplies `metadata.execution` with
the exact repository, branch, generation, digest and nonempty external job
identifier. Do not derive a job identity from request JSON or generate one.
17. Require Authority / remote Git / API OID equality and idempotent replay.
18. Publish immutable staging artifact and run Repository Discovery live E2E.
## Never do
- direct Task DB inserts/updates;
- direct bare-repo/ref changes;
- raw Git push as a substitute for governed publication;
- borrow SSH/Gitea/service credentials as Owner authorization;
- disable TaskPrincipal verification;
- issue TaskPrincipal without a live TaskLease;
- set global writes true;
- turn 18893 into a writer;
- include production/tag/baseline/main permissions in a task grant.
## Acceptance
Bootstrap repair:
```text
ENVELOPE_EXACT_REPLAY=PASS
SCOPE_NARROWING=PASS
DELEGATION_REVOKE_EXPIRY=PASS
OWNER_SIGNATURE_VERIFY=PASS
CONCURRENT_ADMISSION_SINGLE_TASK=PASS
TASK_DB_DIRECT_WRITE=false
TASKPRINCIPAL_BYPASS=false
GLOBAL_WRITES=false
CRASH_AFTER_EFFECT_REPLAY=PASS
CRASH_BEFORE_EFFECT_REPLAY=PASS
CHANGED_VALID_PROOF_CONFLICT=PASS
TASK_API_EXACT_IDENTITY=PASS
```
These are implementation and test acceptance conditions only. They are not
evidence of a live claim, governed publication, remote OID equality, or staging
E2E execution; obtain those independently through the sanctioned live runbook.
Repository Discovery publication:
```text
TASK_CREATED=PASS
TASK_CLAIMED=PASS
TASK_HEARTBEAT=PASS
TASK_LEASE=PASS
TASK_PRINCIPAL=PASS
WRITER_LEASE=PASS
COMMIT_FILES_GOVERNED=PASS
AUTHORITY_REMOTE_API_OID_EQUAL=PASS
IDEMPOTENCY_REPLAY=PASS
STAGING_LIVE_E2E=PASS
TASK_VERIFIED=SUCCEEDED
```
@@ -0,0 +1,35 @@
# Trusted-builder signer provisioning
`trusted_builder_signer_provisioning.py` provisions the purpose-bound
`AIOS_TRUSTED_RELEASE_BUILDER` signer only from an exact
`aios.trusted-builder-signer-provisioning.v2` payload and detached SSH signature
for `aios-task-grant-signer` in namespace `aios-owner-approval-v1`.
A fresh payload is bound to the final common Gitea, authority, and governed
`main` OID; the protected merge tree; the ordered two parents; and the selected
PR head as the merge's second parent. The provisioner rechecks those facts in
the authority repository and rejects ancestry or source drift. It does not
reuse PR69, PR70, or a release-candidate recipe binding.
The payload evidence is an exact allowlist: provisioner, candidate builder,
policy, service, socket, model-API config template, launcher, and the versioned
package manifest. Missing, extra, modified, or mode-incompatible evidence is
rejected. The package manifest is non-secret and declares daemon reload plus
socket activation, with no credential installation or signer invocation. It
requires the installed launcher target mode to be `0755`; source checkout mode
is treated separately from installed package mode. A controlled deterministic
worktree may hold ordinary source evidence at `0600`, `0640`, `0644`, or `0664`; the
installed package still enforces its declared target modes.
The provisioner generates the Ed25519 key and builder HMAC only on the host.
Plaintext exists only in its private temporary directory and is encrypted with
`systemd-creds --with-key=host`; the service receives encrypted credentials via
`LoadCredentialEncrypted`. The public keyring and service-principal binding
contain no private key.
Receipts append at
`/var/lib/aios-gitops/trusted-builder/provisioning-receipts.jsonl`. An exact
replay returns `IDEMPOTENT_REPLAY_PASS`; mismatched or unreceipted state fails
closed. Provisioning is configuration only: activation, restart persistence,
public-key verification and negative-capability tests are separate gates before
`TRUSTED_BUILDER_SIGNER_READINESS=PASS`.
@@ -1,54 +0,0 @@
# R4.2 P0-A TaskPlane Scope Isolation Design
Status: approved candidate design. This document is not evidence of a protected-main merge or production deployment.
## Boundary
- Task: `r42-p0a-taskplane-scope-isolation-20260816`
- Governed Job: `job-3fd743b51971c6007923699c1233f84c`
- Repository: `aios/aios`
- Base: `0eb6e1a864edfdaeef37b438991707a110e21558`
- Target branch: `agent/r42-p0a-taskplane-scope-isolation-20260816`
- Generation: `1`
The implementation reuses this exact boundary. It does not create a replacement Task, Job, branch, generation, or idempotency key.
## Model and invariants
`TaskPlan` gains first-class `scope_type`, `tenant_id`, `user_id`, `project_id`, `conversation_id`, and `run_id` fields. The fields participate in accepted-key validation, parsing, serialization, persistence, and authorization.
Historical rows without scope data normalize to `scope_type=legacy`; no tenant is inferred. Tenant-scoped tasks require a non-empty `tenant_id`. System/global resources use an explicit `scope_type=system` invariant and do not receive a fabricated tenant. A task may add user, project, conversation, and run ownership within its tenant.
## Trusted ActorContext
TaskPlane defines a small immutable `TaskActorContext` used only for authorization. The HTTP authenticator returns an identity. `PolicyEngine` resolves that identity through server-owned policy configuration into the effective context and capabilities. Request fields such as `tenant_id`, `user_id`, `group_id`, and `role` are declarations only and never expand authority.
Public create, list, and get paths always use the resolved context. Internal compatibility calls may omit a context and retain explicit legacy behavior, but the HTTP adapter never uses that path.
## Persistence
SQLite remains the current runtime. `_create_schema()` performs an idempotent forward migration that adds the six scope columns and indexes, then backfills only missing `scope_type` values to `legacy`.
New writes persist scope in both the canonical `task_json` representation and first-class columns. Reads accept historical JSON without scope, using legacy columns, but reject a new-row mismatch between JSON and columns. SQL list/get predicates are parameterized and applied before row hydration.
PostgreSQL remains a separate migration target and is not claimed by this change.
## Authorization flow
1. HTTP bearer authentication yields an identity.
2. `PolicyEngine` verifies capability and resolves the identity to `TaskActorContext`.
3. Create binds the requested scope to the trusted context and rejects mismatches.
4. List/get pass the trusted context to `TaskPlaneStore`.
5. Store applies tenant and ownership predicates in SQL; privileged capabilities may explicitly cross scope.
Tenant actors cannot read legacy tasks, another tenant, or mismatched owned resources. Legacy actors can only read legacy tasks. System actors require explicit privileged capability for cross-scope access.
## Failure behavior
Missing tenant scope, forged scope, unknown scope types, JSON/column divergence, and unauthorized list/get operations fail closed with canonical TaskPlane errors. Public get does not reveal whether an out-of-scope task exists.
## Verification
Tests exercise real model round trips, SQLite migration/restart, Service/Policy/Store wiring, tenant A versus tenant B list/get denial, ownership filtering, forged create payload rejection, legacy isolation, privileged access, and SQL parameterization. Existing TaskPlane API, RBAC, recovery/security, and security-contract suites remain mandatory.
The candidate is complete only after real `TASK_CLAIMED`, `TASK_HEARTBEAT`, exact WriterLease, governed commit, OID convergence, result submission, and `TASK_VERIFIED`. Protected main and production remain unchanged.
@@ -1,90 +0,0 @@
# R4.2 P0-C Trace / Tool-MCP / Credential Governance
## Status boundary
This change is a candidate implementation on the governed P0-C branch. It does
not promote protected `main`, deploy Production, or assert that all R4.2 target
architecture is complete.
## Scope model
Governance resources use an explicit `scope_type`:
- `tenant` requires `tenant_id` and may narrow to user, project,
conversation, and run;
- `system` and `global` prohibit tenant ownership fields;
- `legacy` preserves unscoped historical records without inventing ownership.
System and global tools therefore never receive a synthetic tenant merely to
satisfy a tenant-oriented schema.
## Trace lineage
`TaskPlan` and `MessageEnvelope` scope is bound to a `TraceContext`. The trace
records `trace_id`, `span_id`, parent span, correlation, causation, task,
message, and the canonical P0-A scope. Child spans must keep the parent's trace,
task, and scope. Privileged visibility does not permit cross-scope trace
splicing.
`MessageEnvelope.correlation_id` is persisted and identity-bound. Existing
handoff databases are upgraded in place with an idempotent column migration;
legacy rows retain their prior event IDs because a null correlation field is
not added to the historical identity material.
Observability keeps its legacy presentation fields, but new governed records
also carry immutable trace and scope fields. Scoped records require a trusted
server actor for queries. A repeated execution ID is an idempotent replay only
when the complete record matches; conflicting overwrites are denied.
## Tool and MCP governance
`ToolDescriptor` binds a tool or MCP server to:
- an explicit resource scope;
- required actor capabilities;
- an optional credential reference.
Invocation authorization requires exact propagation of the trace scope and a
capability superset. Unknown tools, cross-tenant calls, scope substitution, and
non-privileged access to system/global resources fail closed. Every successful
registration and invocation appends hash-chained audit evidence.
## Credential governance
Credentials are represented only by references whose schemes are
`systemd-credential`, `vault`, or `secret-manager`. Raw bearer tokens, passwords,
and arbitrary values are rejected. Grants are bound to scope, operation, task,
tool, and expiry. Public/audit records contain a reference hash where disclosure
is unnecessary; secret material is never persisted by this module.
The existing runtime credential broker remains backward compatible for system
leases, adds explicit tenant/system/global invariants, and rejects providers
that attempt to return raw token/secret/password environment values instead of
file/reference indirection.
## Durable evidence
The SQLite governance store is restart-safe and idempotently initializes four
tables: governed tools, trace spans, credential grants, and audit evidence.
Audit entries form a SHA-256 chain. Acceptance requires the chain to verify and
contain the required actions; missing evidence is not treated as success.
## Acceptance tests
The candidate must pass:
- `TRACE_LINEAGE_CONTINUITY`;
- `TRACE_SCOPE_MISMATCH_DENY`;
- `TOOL_MCP_AUTHZ_DENY`;
- `TOOL_SCOPE_PROPAGATION`;
- `CREDENTIAL_SCOPE_DENY`;
- `CREDENTIAL_SECRET_EXPOSURE_DENY`;
- `AUDIT_EVIDENCE_COMPLETENESS`;
- P0-A scope and P0-B handoff regression suites;
- Python compilation, `git diff --check`, and a diff-scoped secret scan.
The repository-wide baseline currently has seven unrelated failures in the
isolated host environment (five stale continuation-contract fixtures and two
host-permission fixtures). The same seven failures reproduce at the untouched
`a642b045775b9154a00ae78543037fbef4fb447b` base; they are not suppressed or
modified by P0-C.
@@ -1,75 +0,0 @@
# R4.2 P1 — Durable Online Agent Execution
Status: candidate implementation. This document does not imply protected-main promotion or Production deployment.
## State model
The generic worker uses two local durable layers:
1. `tasks/<task_id>.json`: atomic fast-state cache for compatibility and quick restart.
2. `checkpoints.sqlite3`: append-only attempt/phase journal with `synchronous=FULL`.
Checkpoint identity is bound to:
- task_id
- attempt
- execution_digest
- job_id
- repository
- base_oid
- expected_branch_oid
- target_branch
- generation
A checkpoint from another repository/branch/generation is never reused.
Phases are monotonic within an attempt:
`claimed → workspace_prepared → verification_passed → source_capture_done|commit_done → result_pending → result_submitted`
`reconciliation_required` is an explicit recovery phase.
## Attempt-aware result idempotency
Result submission idempotency keys include the official Task attempt and the execution digest. The same
attempt reuses the same key after a timeout/restart; a later official attempt cannot collide with an
earlier result.
Before submitting a result the worker durably writes `result_pending`, including the result hash and
idempotency key. After a successful or exact-idempotency-reconciled API response it writes
`result_submitted`.
A 409 is accepted only if the Task API already contains the exact same result body and the Task is
VERIFYING/SUCCEEDED. A conflicting result remains a hard failure.
## Resume rules
- `source_capture_done` and `result_pending` may be reused across later official attempts only when the
execution/scope binding is identical. This avoids repeating read-only source capture.
- A deterministic task that crashed before governed commit replays from a new clean detached worktree.
This is safe because there is no external side effect yet.
- `commit_done` is never recommitted. Recovery requires exact Job receipt OID convergence plus exactly
one ACTIVE WriterLease whose `current_head_oid` equals the governed commit.
- delayed Git visibility remains `RECONCILIATION_REQUIRED`, preserving an explicit recovery entry.
## Crash cleanup
On worker startup, stale `task-*` workspaces under the configured workspace root are removed before
polling begins. The path boundary is verified before deletion.
## Contracts
P1 gates include:
- CHECKPOINT_DURABLE_WRITE
- RESUME_FROM_CHECKPOINT
- RETRY_IDEMPOTENT_NO_DUPLICATE_SIDE_EFFECT
- CRASH_RECOVERY_CONVERGENCE
- WORKER_RESTART_RESUME
- CHECKPOINT_SCOPE_ISOLATION
- RESULT_SUBMISSION_REPLAY_SAFE
- WORKSPACE_CLEANUP_RECOVERY
- LEASE_RENEWAL_RECOVERY
WriterLease and governed Job authority remain in the existing Bridge/GitOps durable stores; the worker
checkpoint DB is not a competing authority for Git state.
@@ -1,18 +0,0 @@
# R4.2 P2 Agent Fleet / Multi-Agent Scheduling / Fault Isolation
P2 adds a durable fleet decision layer inside TaskPlane without replacing TaskPlane claim/lease
ownership. Worker registration/heartbeat remains authoritative for liveness. TaskPlane claim CAS
remains authoritative for ownership. Probe-before-reassignment remains mandatory after an expired
lease. P1 checkpoint/result replay remains authoritative after worker restart.
`FleetScheduler` adds durable worker capacity slots, weighted scope fairness, starvation aging,
affinity/anti-affinity labels, worker circuit breakers, per-task/worker quarantine and idempotent
dispatch reservations. Fleet state is colocated in the existing TaskPlane SQLite database and is
created idempotently; no new daemon or database is introduced.
A fleet reservation is only a scheduling decision. It does not claim a Task. Callers must perform
the normal TaskPlane claim and may then bind the returned claim ID to the reservation. This preserves
the existing ownership and recovery invariants.
Failure recovery never bypasses probe evidence. When TaskPlane recovery places work back in a
claimable state, P2 may quarantine the failed worker and prefer another worker/failure domain.
@@ -1,79 +0,0 @@
# R4.2 P2 Compact Interface Map Runtime
## Purpose
The persistent Online Agent already supports the canonical read-only `source_capture` action, but
its `SourceCaptureExecutor` always returns an `aios.engineering.source-snapshot.v2` tar.gz+base64
payload. That is correct for implementation source transfer, but too large for control-plane
consumers that only need a bounded implementation map.
P2 adds a new **adapter**, not a new execution action:
```text
action=source_capture
adapter=compact_interface_map
```
This preserves the existing execution-v2 schema and Bridge policy:
- read-only repository;
- no change set;
- no verification commands;
- `writer_lease=false`;
- no branch or commit;
- existing capability and execution-digest validation.
## Compact map contract
`aios_online_agent_compact_map.py` scans only approved repository roots and source suffixes. It
returns:
- at most 40 relevant modules;
- at most 80 selected Python symbols in total;
- at most 40 test symbols;
- at most 25 excerpts;
- at most 1200 characters per excerpt;
- SQL table names;
- systemd unit names;
- current invariants, failure windows and reuse points;
- P2 fleet contract-to-surface mappings.
The result is hard-bounded to 120 KiB and contains no source archive or `payload_b64`.
Source excerpts apply a conservative sensitive-pattern redaction before result construction.
## P2 contracts
The map supplies evidence surfaces for:
- `FLEET_AGENT_REGISTRATION`
- `FLEET_CAPACITY_SCHEDULING`
- `FLEET_FAIRNESS`
- `FLEET_FAULT_ISOLATION`
- `FLEET_WORKER_REASSIGNMENT`
- `FLEET_SCOPE_ISOLATION`
- `FLEET_IDEMPOTENT_DISPATCH`
- `FLEET_FAILURE_RECOVERY`
Missing first-class scheduling/fleet primitives are reported as gaps rather than invented.
## Result
When the adapter is selected, the worker returns:
```text
status=SUCCEEDED
classification=P2_INTERFACE_MAP_CAPTURED
execution_mode=compact_interface_map
writer_lease=NONE
```
The existing `source_capture_done` / P1 checkpoint/retry path is reused, so the compact analysis
inherits attempt-aware result idempotency and restart-safe result submission.
## Governance
This capability is runtime infrastructure only. The implementation candidate may use a governed
branch/WriterLease for its own code change, but any task executed through `compact_interface_map`
is read-only and must not create a WriterLease, branch, commit, protected-ref write, or Production
side effect.
@@ -1,17 +0,0 @@
# R4.2 P2-B Fleet Order Preservation Repair
P2-C live acceptance exposed a real integration defect in the verified P2-B candidate. The fleet
scheduler correctly produced durable cross-scope fairness order, but the integration passed the
entire ranked list to the legacy scheduler. The legacy scheduler then applied its own priority /
created_at / task_id sort and silently discarded fleet ordering.
The repair preserves the legacy scheduler as the fail-closed security oracle while changing how it
is called: one fleet-ranked candidate is presented at a time. A candidate rejected by legacy
security/policy advances to the next fleet candidate. A candidate returned by the oracle must have
the same task ID as the candidate presented, otherwise execution fails closed.
Task ownership remains unchanged: the original schedule_for_worker/claim_task CAS remains
authoritative, and a fleet reservation is still only bound after the claim succeeds.
This repair does not modify FleetScheduler state semantics, capacity, WriterLease, checkpoint,
protected refs or Production behavior.
@@ -1,23 +0,0 @@
# R4.2 P2-B Fleet Runtime Integration
P2-B connects the verified P2 fleet primitives to the existing TaskPlane pull scheduler without
replacing its security or ownership logic.
The integration uses two stages. `FleetScheduler.rank_for_worker()` performs only read-only
capacity/fairness/fault-isolation ranking. The original `TaskScheduler.next_queued_task()` is then
run over that ordered list and remains the fail-closed security oracle for worker liveness,
capabilities, data classification, execution zone and other existing policy checks. Only a task
accepted by the original scheduler receives a short durable fleet reservation.
The original `TaskScheduler.schedule_for_worker()` remains the sole claim path and continues to call
the existing TaskPlane `claim_task` CAS. After a successful claim, the claim ID is bound to the
fleet reservation. A claim error releases the reservation. A fleet reservation is never treated as
task ownership.
P2-B also makes the reservation capacity check atomic inside the fleet SQLite transaction so two
concurrent pullers cannot oversubscribe the same worker slot. Circuit and task/worker quarantine
state are enforced on the real pull path. Explicit execution-failure signals can be recorded through
the integrated scheduler's failure/success/quarantine methods; policy/claim failures are not
automatically mislabeled as worker crashes.
No new daemon, database or production write path is introduced.
@@ -1,18 +0,0 @@
# Generic Compact Interface-Map Profiles
The compact interface-map runtime is a reusable read-only control-plane capability. It keeps the
canonical `source_capture` action and `compact_interface_map` adapter, but selects a bounded semantic
profile from the task's source signals.
The existing `fleet` profile remains backward compatible and returns
`P2_INTERFACE_MAP_CAPTURED` with the `FLEET_*` contracts.
The new `model_governance` profile is selected when model/provider/routing/budget/evaluation terms
are present. It returns `MODEL_GOVERNANCE_INTERFACE_MAP_CAPTURED` and model-specific contracts for
provider registry, selection policy, version pinning, budget accounting, fallback, scope isolation,
credential boundary, trace/audit, evaluation gates and routing resilience.
Profile and classification are emitted by the compact-map module and revalidated by the worker
before result submission. Unknown or mismatched profile/classification pairs fail closed.
No new Bridge action, daemon, database, WriterLease, mutation path or credential surface is added.
@@ -1,9 +0,0 @@
# R4.2 P0-A TaskPlane Scope Isolation
Status: candidate implementation. This is not proof of protected-main promotion or production deployment.
Task scope is now represented by `scope_type`, `tenant_id`, `user_id`, `project_id`, `conversation_id`, and `run_id` in the model and SQLite persistence. Historical rows become `legacy`; their tenant is never guessed. Tenant tasks require `tenant_id`. System resources use `scope_type=system` without a fabricated tenant.
The HTTP adapter uses the authenticated identity. `PolicyEngine` resolves that identity through server-owned `identity_actor_contexts`; request payload fields do not grant scope. Public create binds the trusted scope, and public list/get apply parameterized scope predicates in `TaskPlaneStore`.
SQLite is the current runtime database. PostgreSQL remains a separate migration and runtime-acceptance gate.
@@ -1,38 +0,0 @@
# R4.2 P0-B — Durable Message / Handoff Fabric
Status: candidate implementation; not evidence of protected-main merge or Production deployment.
`MessageEnvelope` is the single durable handoff identity and carries source/target/type, tenant/user/
project/conversation/task/run scope, causation, sequence, capability scope, confidentiality scope,
idempotency key, expiry and delivery state.
Identity includes **target + scope + sequence + idempotency key + payload hash**, fixing the existing
AgentEventBus target collision. Server ActorContext is authoritative; caller scope may narrow but may
not widen/forge trusted tenant/user/project/conversation/run fields.
`DurableHandoffStore` provides SQLite durability in the current runtime. Each logical stream has
unique `(stream_key, idempotency_key)` and `(stream_key, sequence)`. Exact retry returns `REPLAY`;
changed payload under the same key is rejected; sequence gaps are rejected; expired messages are not
delivered.
Confidentiality scopes:
- public: no tenant requirement
- tenant: same tenant
- project: same tenant + project
- private: same tenant + project + user
- run: same tenant + project + user + run
Required `capability_scope` must be present on the authenticated delivery actor unless the actor is
explicitly administrative.
Compatibility adapters convert AgentEventBus events, TaskPlane CloudEvents and legacy Agent OS
MessageBus messages into the durable envelope. Existing message_outbox remains intact as a migration
source; P0-B does not delete or silently rewrite it.
Required gates:
`MESSAGE_SCOPE_MISMATCH_DENY`, `MESSAGE_TARGET_COLLISION_DENY`, `MESSAGE_IDEMPOTENCY_REPLAY`,
`MESSAGE_SEQUENCE_GAP_DETECT`, `MESSAGE_CONFIDENTIALITY_SCOPE_ENFORCE`,
`HANDOFF_CROSS_TENANT_DENY`, plus existing P0-A TaskPlane scope/security regressions.
SQLite remains current runtime persistence. PostgreSQL remains a separate migration gate.
@@ -0,0 +1,80 @@
#!/usr/bin/env bash
set -Eeuo pipefail
umask 027
ROOT="/data/aios-gitops"
NGINX_SITE="/etc/nginx/sites-enabled/aios-user-portal.conf"
STAMP="$(date -u +%Y%m%dT%H%M%SZ)"
BACKUP_DIR="$ROOT/backups/action-surface-final-$STAMP"
LOG="$ROOT/logs/action-surface-final-$STAMP.log"
mkdir -p "$BACKUP_DIR" "$ROOT/logs"
exec > >(tee -a "$LOG") 2>&1
sudo cp -a "$NGINX_SITE" "$BACKUP_DIR/aios-user-portal.conf"
SITE_BACKUP="$BACKUP_DIR/aios-user-portal.conf"
python3 - "$SITE_BACKUP" "$BACKUP_DIR/aios-user-portal.conf.next" <<'PY'
from pathlib import Path
import sys
source, output = map(Path, sys.argv[1:])
text = source.read_text(encoding="utf-8")
marker = " # AIOS external Action governed-read ownership"
if marker in text:
raise SystemExit("governed-read locations already present in backup")
block = r''' # AIOS external Action governed-read ownership
# These reads are owned by the authenticated 18890 facade, not the 18893 model API.
location = /api/governed-write-capabilities {
proxy_pass http://127.0.0.1:18890/api/governed-write-capabilities;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Authorization $http_authorization;
proxy_set_header X-AIOS-Token $http_x_aios_token;
}
location = /api/control-plane-snapshot {
proxy_pass http://127.0.0.1:18890/api/control-plane-snapshot;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Authorization $http_authorization;
proxy_set_header X-AIOS-Token $http_x_aios_token;
}
location = /api/online-agent-task {
proxy_pass http://127.0.0.1:18890/api/online-agent-task;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Authorization $http_authorization;
proxy_set_header X-AIOS-Token $http_x_aios_token;
}
location = /api/online-agent-task-events {
proxy_pass http://127.0.0.1:18890/api/online-agent-task-events;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Authorization $http_authorization;
proxy_set_header X-AIOS-Token $http_x_aios_token;
}
'''
needle = " location /api/ {"
if text.count(needle) != 1:
raise SystemExit("expected exactly one generic /api/ location")
text = text.replace(needle, block + "\n" + needle, 1)
output.write_text(text, encoding="utf-8")
PY
sudo cp "$BACKUP_DIR/aios-user-portal.conf.next" "$NGINX_SITE"
sudo nginx -t
sudo systemctl reload nginx
printf 'ACTION_SURFACE_FINAL_OK backup=%s log=%s\n' "$BACKUP_DIR" "$LOG"
@@ -0,0 +1,121 @@
{
"schema": "aios.canonical-task-projection-intent.v1",
"projection_id": "projection-v2r2c2-parent-recovery-20260826",
"projection_generation": 1,
"idempotency_key": "projection-v2r2c2-parent-recovery-20260826-v1",
"task": {
"task_id": "TASK-CONV-V2R2C2-RECONCILE-SNAPSHOT-20260826",
"goal": "Complete V2R2C2 staging-only control-plane convergence and repair the control-plane self-recovery defect exposed by the missing canonical Task projection.",
"instructions": "Resume the immutable runbook at runbooks/agent-v2r2c2-control-plane-convergence-20260825.md from branch agent/v2r2c2-convergence-integration-20260825. Preserve the parent task/job/generation/branch/OID. Complete canonical projection, worker claim, execution binding, heartbeat, WriterLease/CAS, required convergence code, tests, immutable staging build/deploy, first acceptance, restart-persistent second acceptance, and governed Evidence Closure. Recovery subgraphs must return to the parent goal. Never mutate main, V2R2C1 protected tag, production baseline, or production services.",
"success_criteria": [
{
"id": "control_plane_self_recovery",
"required": true,
"expected": "PASS"
},
{
"id": "canonical_task_projection",
"required": true,
"expected": "PASS"
},
{
"id": "worker_execution_path",
"required": true,
"expected": "PASS"
},
{
"id": "writer_path",
"required": true,
"expected": "PASS"
},
{
"id": "governed_mutation",
"required": true,
"expected": "PASS"
},
{
"id": "v2r2c2_code",
"required": true,
"expected": "PASS"
},
{
"id": "tests",
"required": true,
"expected": "PASS"
},
{
"id": "staging_deployment",
"required": true,
"expected": "PASS"
},
{
"id": "runtime_component_drift",
"required": true,
"expected": "PASS"
},
{
"id": "release_consistency",
"required": true,
"expected": "PASS"
},
{
"id": "tag_publication",
"required": true,
"expected": "PASS"
},
{
"id": "release_acceptance",
"required": true,
"expected": "PASS"
},
{
"id": "restart_persistence",
"required": true,
"expected": "PASS"
},
{
"id": "evidence_closure",
"required": true,
"expected": "PASS"
}
],
"constraints": {
"staging_only": true,
"production_write": false,
"main_direct_write": false,
"protected_tag_write": false,
"baseline_write": false,
"force_push": false,
"direct_task_db_write_outside_task_plane": false,
"secret_exposure": false,
"return_to_parent_after_recovery": true
},
"required_capabilities": [],
"metadata": {
"parent_goal": "V2R2C2 staging-only control-plane convergence",
"governed_task_id": "TASK-CONV-V2R2C2-RECONCILE-SNAPSHOT-20260826",
"bridge_job_id": "job-9eee1142f2fc8bb7b571d065c01f40e0",
"recovery_reason": "MISSING_DURABLE_CANONICAL_TASK_PROJECTION_RECOVERY",
"runbook_ref": "runbooks/agent-v2r2c2-control-plane-convergence-20260825.md",
"runbook_oid": "2afc2cf41fc9090cf01e3882d1b8db8d54cf1844",
"runbook_sha256": "a500504a76fb99b26541eaf4edd842a942992ae170fc0f3838530189ca672320"
},
"priority": 100,
"max_attempts": 20,
"deadline": null
},
"governed_binding": {
"governed_task_id": "TASK-CONV-V2R2C2-RECONCILE-SNAPSHOT-20260826",
"bridge_job_id": "job-9eee1142f2fc8bb7b571d065c01f40e0",
"repository": "aios/aios",
"branch": "agent/v2r2c2-convergence-integration-20260825",
"generation": 2,
"base_oid": "2afc2cf41fc9090cf01e3882d1b8db8d54cf1844",
"expected_branch_oid": "2afc2cf41fc9090cf01e3882d1b8db8d54cf1844"
},
"source_artifact": {
"ref": "git:2afc2cf41fc9090cf01e3882d1b8db8d54cf1844:runbooks/agent-v2r2c2-control-plane-convergence-20260825.md",
"sha256": "a500504a76fb99b26541eaf4edd842a942992ae170fc0f3838530189ca672320"
},
"task_payload_sha256": "dcc293ffcd9db36ea8dd35b79a15e77cca57ee4ad5d6f976e24ee454d46eb0c9"
}

Some files were not shown because too many files have changed in this diff Show More