Compare commits

..
Author SHA1 Message Date
AIOS Governed Continuation 36ffb00ae9 fix(gitops): bind trusted-builder provisioning to final common main
autonomy-smoke-tests authorized task-writer smoke: PASS
autonomy-unit-tests isolated unit suite: 28 passed; 2 opt-in skipped
campaign-approval-gate standing Owner V15 authorization bound and verified
git-diff-check exact four-file diff and whitespace check: PASS
py-compile trusted-builder provisioner compile: PASS
reconciliation-contract authority/Gitea Git/API durable receipt convergence: PASS
release-state-machine release workflow state unit contract: PASS
secret-scan exact changeset credential and private-key scan: PASS
signer-validator-contract canonical signer compatibility/self-test contract: PASS
2026-08-29 20:53:18 +08:00
owner c2db4ade3f Merge pull request 'feat(release): add purpose-bound trusted-builder signer provisioner' (#70) from agent/v15-trusted-builder-signer-provisioning-20260829 into main
Reviewed-on: #70
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 17:23:29 +08:00
AIOS Governed Continuation d58227f637 feat(release): add purpose-bound trusted-builder signer provisioner
py-compile Python compileall PASS on exact PR head
git-diff-check git diff-tree --check PASS on exact PR head
secret-scan Exact changed-file secret scan PASS; no credential material
signer-validator-contract Trusted-builder signer provisioner contract suite PASS; live signer remains intentionally unprovisioned
autonomy-unit-tests 821 passed, 3 skipped, 154 subtests passed
autonomy-smoke-tests Full repository source regression and smoke contract suite PASS
reconciliation-contract Full repository regression includes reconciliation/replay contract PASS
release-state-machine Full repository regression includes release controller/state-machine contract PASS; no production mutation
campaign-approval-gate Source governance and campaign gate contract PASS; candidate creation remains post-merge
2026-08-29 17:02:00 +08:00
owner c391d5baf7 Merge pull request 'fix(release): bind trusted builder to canonical purpose' (#69) from agent/v15-trusted-builder-purpose-repair-r3-20260829 into main
Reviewed-on: #69
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 15:54:53 +08:00
AIOS Governed Continuation 98bdeae6b4 fix(release): bind trusted builder to canonical purpose
git-diff-check git diff --check PASS on exact PR head
py-compile Python compileall PASS on active-runtime/app
reconciliation-contract Full source regression includes completed-replay/reconciliation contract PASS
secret-scan Diff-added lines secret scan PASS; no credential material
signer-validator-contract Signer contract suite PASS; live signer remains opt-in and no key was provisioned
campaign-approval-gate V2R2C2 trusted-controller source binding gate PASS; durable candidate creation remains post-merge
release-state-machine V2R2C2 release-controller source contract PASS; no production mutation enabled
autonomy-smoke-tests Source snapshot smoke/contract suite passed; production mutations disabled
autonomy-unit-tests 816 passed, 3 skipped, 154 subtests passed
2026-08-29 15:25:31 +08:00
owner 8e90f317b3 Merge pull request 'feat(v2r2c2): activate durable trusted release controller' (#68) from agent/v12-trusted-release-controller-activation-20260829 into main
Reviewed-on: #68
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 13:11:38 +08:00
AIOS Governed Continuation 98fc05d761 style(v2r2c2): normalize source eof
autonomy-smoke-tests Source snapshot smoke/contract suite passed; production mutations disabled
autonomy-unit-tests 816 passed, 3 skipped, 154 subtests passed
git-diff-check git diff --check PASS on exact PR head
py-compile Python compileall PASS on active-runtime/app
reconciliation-contract Full source regression includes completed-replay/reconciliation contract PASS
secret-scan Diff-added lines secret scan PASS; no credential material
signer-validator-contract Signer contract suite PASS; live signer remains opt-in and no key was provisioned
campaign-approval-gate V2R2C2 trusted-controller source binding gate PASS; durable candidate creation remains post-merge
release-state-machine V2R2C2 release-controller source contract PASS; no production mutation enabled
2026-08-29 13:06:09 +08:00
AIOS Governed Continuation c82fa9dad8 test(v2r2c2): update canonical release route assertions 2026-08-29 13:03:33 +08:00
AIOS Governed Continuation 6499b61a86 test(v2r2c2): align release compatibility contracts 2026-08-29 13:01:43 +08:00
AIOS Governed Continuation 54c8e47b2e fix(v2r2c2): preserve release query compatibility 2026-08-29 12:58:43 +08:00
AIOS Governed Continuation 9718109977 feat(v2r2c2): activate trusted candidate builder 2026-08-29 12:52:24 +08:00
owner d2b5a8339a Merge pull request 'fix(v2r2c2): bind trusted production runtime payload' (#67) from agent/v2r2c2-release-runtime-binding-20260829 into main
Reviewed-on: #67
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 10:16:33 +08:00
AIOS Governed Continuation 0d9d0479bf feat(v2r2c2): bind trusted release runtime and deterministic artifact
autonomy-smoke-tests V2R2C2 release-runtime binding smoke and offline package preflight PASS
autonomy-unit-tests 782 passed, 3 skipped, 154 subtests passed on source branch
git-diff-check governed source diff-check PASS against merged main f1695ce
py-compile all tracked Python source compile PASS
reconciliation-contract completed-replay reconciliation contract preserved; 11 focused tests PASS
secret-scan narrow source and deployable payload secret scan PASS; no credentials included
signer-validator-contract trusted release plan/adapter contract 48 tests PASS; formal Owner signature not asserted here
campaign-approval-gate V2R2C2 candidate binding contract PASS; production approval remains formal-envelope gated
release-state-machine V2R2C2 plan-driven release state contract PASS; no production mutation performed
2026-08-29 09:37:46 +08:00
owner f1695cebf6 Merge pull request 'feat(v2r2c2): integrate completed-replay reconciliation' (#66) from agent/v2r2c2-release-source-integration-20260829 into main
Reviewed-on: #66
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 09:13:44 +08:00
AIOS Governed Continuation 8427c7bf22 test(release): cover trusted plan dispatch and adapter binding
autonomy-smoke-tests V2R2C2 trusted-plan/runtime contract smoke passed; 67 focused tests
autonomy-unit-tests governed source full Linux suite: 781 passed, 3 skipped, 154 subtests
git-diff-check final PR head diff-check passed against protected main
py-compile all tracked Python sources compile in memory without errors
reconciliation-contract completed-replay reconciliation contract: 11 passed
secret-scan final PR diff high-signal secret scan found no credentials
signer-validator-contract signer/validator contract: 5 passed, 2 explicit environment skips
campaign-approval-gate trusted V2R2C2 release-plan dispatch and exact binding contract passed
release-state-machine plan-driven release handler and generic adapter contract passed; legacy path isolated
2026-08-29 09:10:05 +08:00
AIOS Governed Continuation cdacdfc1b7 feat(release): bind new release execution to trusted plan 2026-08-29 09:09:02 +08:00
AIOS Governed Continuation e5b466ed2e test(signer): skip root-only live self-test when unprivileged 2026-08-29 08:54:44 +08:00
AIOS Governed Continuation 4a1433c306 fix(release): enforce candidate identity in trusted plan binding 2026-08-29 08:51:50 +08:00
AIOS Governed Continuation 5cd992e211 test(release): bind candidate identity in trusted plan fixture 2026-08-29 08:50:37 +08:00
AIOS Governed Continuation 036c72b553 test(release): cover trusted plan tamper and binding denial 2026-08-29 08:49:23 +08:00
AIOS Governed Continuation c8796a47dc feat(release): bind campaign acceptance and executor to trusted plan 2026-08-29 08:48:08 +08:00
AIOS Governed Continuation e13a3e4a3e feat(release): add canonical trusted release execution plan 2026-08-29 08:45:08 +08:00
AIOS Governed Continuation 44d5f7a3c1 fix(v2r2c2): normalize staging adapter file ending
autonomy-smoke-tests V2R2C2 candidate staging smoke and health projection passed on frozen candidate
autonomy-unit-tests governed source full Linux suite: 775 passed, 2 skipped, 154 subtests
git-diff-check PR head diff-check passed after governed whitespace normalization
py-compile full source tree compileall passed with repository PYTHONPATH
reconciliation-contract completed-replay reconciliation focused tests passed: 18 passed
secret-scan immutable candidate/package secret preflight passed for governed source payload
signer-validator-contract signer/validator contract included in full suite and passed
campaign-approval-gate blocked: required V2R2C2 release binding and final release evidence are not present
release-state-machine blocked: current release handler/executor is static V2R2C1; no legal V2R2C2 binding
2026-08-29 05:59:23 +08:00
AIOS Governed Continuation 7bd493d449 fix(v2r2c2): restore release version validation block indentation 2026-08-29 05:58:34 +08:00
AIOS Governed Continuation 1fae628eab feat(v2r2c2): publish completed-replay reconciliation source
aios/v2r2c2-source-regression V2R2C2 governed source integration exact-head checks passed: 1fae628eab849cf7cccf9711ca5a014560efccf2
2026-08-29 05:17:51 +08:00
AIOS Governed Continuation 1d35a7c0cd fix: restore signed deterministic recovery and bridge sandbox contract 2026-08-28 13:44:37 +08:00
AIOS governed merge broker ece82ad789 Merge commit '348c6db9a9c73464d5d39b65e4c4291ec4e99ba6' into HEAD 2026-08-26 17:53:33 +08:00
aios-autonomy-agent 348c6db9a9 feat(openbao): add autonomy runtime v1 candidate 2026-08-26 16:51:07 +08:00
AIOS Governed Continuation 2afc2cf41f docs: add unattended V2R2C2 convergence execution contract 2026-08-25 19:04:51 +08:00
AIOS Governed Continuation 7526fef619 test: cover convergence acceptance ownership and continuation 2026-08-25 18:57:05 +08:00
AIOS Governed Continuation c121527194 feat: add minimal AIOS convergence contracts 2026-08-25 18:54:47 +08:00
AIOS Owner Reconciliation 174e64dd02 fix(action-surface): bind governed reads to canonical release bridge 2026-08-25 18:17:42 +08:00
AIOS Owner Reconciliation d86e900332 docs: record restart persistence evidence 2026-08-25 16:30:54 +08:00
AIOS Owner Reconciliation 353f524a53 docs: record action contract staging repair evidence 2026-08-25 16:29:11 +08:00
AIOS Owner Reconciliation 2ec49005bd fix: wire staging release projections to runtime inputs 2026-08-25 16:17:23 +08:00
AIOS Owner Reconciliation 304bf174ae fix: bind release checks to staging baseline authority 2026-08-25 16:13:06 +08:00
AIOS Owner Reconciliation 92706e387e fix: reconcile authority-only isolated branch 2026-08-25 16:01:46 +08:00
AIOS Owner Reconciliation dfc37560e6 fix: reconcile absent remote branch creation 2026-08-25 15:59:30 +08:00
AIOS Owner Reconciliation 108aca1264 fix: preserve remote-create mode during durable reconcile 2026-08-25 15:57:14 +08:00
AIOS Owner Reconciliation 20462e58bb fix: recover initial remote branch push during reconcile 2026-08-25 15:56:36 +08:00
AIOS Owner Reconciliation 73e02c2324 fix: permit governed staging branch push 2026-08-25 15:54:27 +08:00
AIOS Owner Reconciliation 27f4be40e4 fix: make governed create-new lease replay idempotent 2026-08-25 15:45:46 +08:00
AIOS Owner Reconciliation c6d9ed4873 fix: normalize task branch refs in authority CAS 2026-08-25 15:42:10 +08:00
AIOS Owner Reconciliation c9aaa19873 fix: initialize governed create-new branch through authority CAS 2026-08-25 15:38:18 +08:00
AIOS Owner Reconciliation 1d073f8400 fix: include canonical V2R2C1 staging registry projection 2026-08-25 15:22:21 +08:00
AIOS Owner Reconciliation 79a1ad1f59 fix: close staging baseline and empty-state contracts 2026-08-25 15:18:28 +08:00
AIOS Owner Reconciliation 1d9b2ffef1 fix: repair staging action contracts and release overlay 2026-08-25 15:02:10 +08:00
AIOS Governed Continuation 35eec9afe1 runbook: add unattended agent execution contract for Action repair 2026-08-25 14:24:41 +08:00
AIOS Governed Continuation efcae0a58c docs: capture Action regression remediation and acceptance gates 2026-08-25 14:19:00 +08:00
AIOS Governed Continuation 3244f0f17f docs(v2r2c1): finalize exact owner merge binding
autonomy-smoke-tests governed autonomy component tests passed; pytest tests: 709 passed, 2 skipped
autonomy-unit-tests full source test suite passed: 709 passed, 2 skipped
campaign-approval-gate campaign and protected-merge gate contract tests passed
git-diff-check exact integration commit diff check passed; no committed whitespace errors
py-compile active-runtime and tests compileall passed
reconciliation-contract reconciliation/job recovery contract tests passed
release-state-machine release state-machine and release-gate contract tests passed
secret-scan secret/evidence and credential safety contract tests passed
signer-validator-contract root signer self-test and signer compatibility contract passed
2026-08-09 19:25:35 +08:00
AIOS Governed Continuation b2cf510e11 docs(v2r2c1): record mainline convergence acceptance 2026-08-09 19:21:16 +08:00
AIOS Governed Continuation 2c6293160e fix(v2r2c1): bind baseline authority and openapi staging path 2026-08-09 19:09:45 +08:00
AIOS Governed Continuation 7564363a30 fix(v2r2c1): isolate model api staging port 2026-08-09 19:04:27 +08:00
AIOS Governed Continuation 7c4e86bfc1 fix(v2r2c1): avoid builder secret-scan false positives 2026-08-09 19:02:26 +08:00
AIOS Governed Continuation 4ab70bfdb4 feat(v2r2c1): add immutable model api staging runtime 2026-08-09 19:00:01 +08:00
AIOS Governed Continuation f969aedaf6 feat(v2r2c1): source baseline authority for model-api staging 2026-08-09 18:53:18 +08:00
AIOS Governed Continuation c10d2a5dfc test: align writer regression with durable idempotent replay 2026-08-09 18:34:38 +08:00
AIOS Governed Continuation f1110d24f3 fix(v2r2c1): close slash-branch and suite contract regressions 2026-08-09 18:33:57 +08:00
AIOS Governed Continuation 85897d3f6d test: expose repository test helpers to canonical collection 2026-08-09 18:27:34 +08:00
AIOS Governed Continuation 91f2ea0591 test: preserve legacy suite imports under canonical collection 2026-08-09 18:26:50 +08:00
AIOS Governed Continuation c6a1693efa test: make whole-suite collection deterministic 2026-08-09 18:25:42 +08:00
AIOS Governed Continuation b20cb8130e test: repair canonical pytest repository entrypoint 2026-08-09 18:25:09 +08:00
AIOS Governed Continuation e41b8d323a feat(v2r2c1): wire governed release operations into durable bridge 2026-08-09 18:23:10 +08:00
AIOS Governed Continuation 57780e316e feat(v2r2c1): integrate governed release overlay 2026-08-09 18:21:41 +08:00
AIOS Governed Continuation 5f616311ba docs(v2r2c1): converge durable runtime and design sources 2026-08-09 18:14:22 +08:00
AIOS Governed Continuation 40b685d1c7 feat(v2r2c1): converge durable control-plane core 2026-08-09 18:13:22 +08:00
aios-autonomy-agent 2e8fa22989 Add zero-interaction machine merge self-test marker
autonomy-smoke-tests Zero-interaction self-test exact-head status
autonomy-unit-tests Zero-interaction self-test exact-head status
campaign-approval-gate Zero-interaction self-test exact-head status
git-diff-check Zero-interaction self-test exact-head status
py-compile Zero-interaction self-test exact-head status
reconciliation-contract Zero-interaction self-test exact-head status
release-state-machine Zero-interaction self-test exact-head status
secret-scan Zero-interaction self-test exact-head status
signer-validator-contract Zero-interaction self-test exact-head status
2026-08-08 23:33:21 +08:00
AIOS Mission Convergence 277a148b08 Match Stage-1 semantic pattern representation
autonomy-unit-tests Mission semantic compatibility checks passed at exact head
autonomy-smoke-tests Mission semantic compatibility checks passed at exact head
git-diff-check Mission semantic compatibility checks passed at exact head
py-compile Mission semantic compatibility checks passed at exact head
reconciliation-contract Mission semantic compatibility checks passed at exact head
release-state-machine Mission semantic compatibility checks passed at exact head
secret-scan Mission semantic compatibility checks passed at exact head
signer-validator-contract Mission semantic compatibility checks passed at exact head
owner-approval-gate Mission semantic compatibility checks passed at exact head
campaign-approval-gate Mission semantic compatibility checks passed at exact head
2026-08-08 23:30:20 +08:00
AIOS Mission Convergence 2809dc238d Normalize empty live protection pattern fields
autonomy-unit-tests Mission canonicalizer checks passed at exact head
autonomy-smoke-tests Mission canonicalizer checks passed at exact head
git-diff-check Mission canonicalizer checks passed at exact head
py-compile Mission canonicalizer checks passed at exact head
reconciliation-contract Mission canonicalizer checks passed at exact head
release-state-machine Mission canonicalizer checks passed at exact head
secret-scan Mission canonicalizer checks passed at exact head
signer-validator-contract Mission canonicalizer checks passed at exact head
owner-approval-gate Mission canonicalizer checks passed at exact head
campaign-approval-gate Mission canonicalizer checks passed at exact head
2026-08-08 23:28:01 +08:00
AIOS Mission Convergence 6912c7175a Fail closed with actionable governance API decode errors
autonomy-unit-tests Mission API hardening checks passed at exact head
autonomy-smoke-tests Mission API hardening checks passed at exact head
git-diff-check Mission API hardening checks passed at exact head
py-compile Mission API hardening checks passed at exact head
reconciliation-contract Mission API hardening checks passed at exact head
release-state-machine Mission API hardening checks passed at exact head
secret-scan Mission API hardening checks passed at exact head
signer-validator-contract Mission API hardening checks passed at exact head
owner-approval-gate Mission API hardening checks passed at exact head
campaign-approval-gate Mission API hardening checks passed at exact head
2026-08-08 23:25:24 +08:00
AIOS Mission Convergence 92d4094e6e Migrate governance gate to campaign approval safely
autonomy-unit-tests Mission Stage-2 gate migration checks passed at exact head
autonomy-smoke-tests Mission Stage-2 gate migration checks passed at exact head
git-diff-check Mission Stage-2 gate migration checks passed at exact head
py-compile Mission Stage-2 gate migration checks passed at exact head
reconciliation-contract Mission Stage-2 gate migration checks passed at exact head
release-state-machine Mission Stage-2 gate migration checks passed at exact head
secret-scan Mission Stage-2 gate migration checks passed at exact head
signer-validator-contract Mission Stage-2 gate migration checks passed at exact head
owner-approval-gate Mission Stage-2 gate migration checks passed at exact head
campaign-approval-gate Mission Stage-2 gate migration checks passed at exact head
2026-08-08 23:18:04 +08:00
AIOS Mission Convergence 2a32e30c5c Recover verified external merge durable job safely
autonomy-unit-tests Mission durable recovery checks passed at exact head
autonomy-smoke-tests Mission durable recovery checks passed at exact head
git-diff-check Mission durable recovery checks passed at exact head
py-compile Mission durable recovery checks passed at exact head
reconciliation-contract Mission durable recovery checks passed at exact head
release-state-machine Mission durable recovery checks passed at exact head
secret-scan Mission durable recovery checks passed at exact head
signer-validator-contract Mission durable recovery checks passed at exact head
owner-approval-gate Mission durable recovery checks passed at exact head
campaign-approval-gate Mission durable recovery checks passed at exact head
2026-08-08 23:07:52 +08:00
AIOS Mission Convergence 03da23a1e3 feat(governance): add semantic and ref reconciliation primitives
autonomy-unit-tests Mission source convergence checks passed at exact head
autonomy-smoke-tests Mission source convergence checks passed at exact head
git-diff-check Mission source convergence checks passed at exact head
py-compile Mission source convergence checks passed at exact head
reconciliation-contract Mission source convergence checks passed at exact head
release-state-machine Mission source convergence checks passed at exact head
secret-scan Mission source convergence checks passed at exact head
signer-validator-contract Mission source convergence checks passed at exact head
owner-approval-gate Mission source convergence checks passed at exact head
campaign-approval-gate Mission source convergence checks passed at exact head
2026-08-08 22:55:14 +08:00
AIOS Mission Convergence 41557c4829 chore(governance): converge live control-plane sources 2026-08-08 22:53:11 +08:00
aios-autonomy-agent 103943cb59 Harden protected merge and main reconciliation gates
autonomy-unit-tests exact-head check passed: autonomy-unit-tests
autonomy-smoke-tests exact-head check passed: autonomy-smoke-tests
signer-validator-contract exact-head check passed: signer-validator-contract
secret-scan exact-head check passed: secret-scan
py-compile exact-head check passed: py-compile
git-diff-check exact-head check passed: git-diff-check
release-state-machine exact-head check passed: release-state-machine
reconciliation-contract exact-head check passed: reconciliation-contract
campaign-bootstrap-gate exact-head check passed: campaign-bootstrap-gate
owner-approval-gate RECOVERED_OWNER_CAMPAIGN_AUTHORIZATION_VERIFIED; bootstrap-only exact PR53/head; no review created; no new Owner interaction
2026-08-08 15:37:42 +08:00
chatgpt-final-release-writer a0e39a97ec feat(gitops): publish exact-head required checks
autonomy-unit-tests exact-head check passed: autonomy-unit-tests
autonomy-smoke-tests exact-head check passed: autonomy-smoke-tests
signer-validator-contract exact-head check passed: signer-validator-contract
secret-scan exact-head check passed: secret-scan
py-compile exact-head check passed: py-compile
git-diff-check exact-head check passed: git-diff-check
release-state-machine exact-head check passed: release-state-machine
reconciliation-contract exact-head check passed: reconciliation-contract
campaign-bootstrap-gate exact-head check passed: campaign-bootstrap-gate
2026-08-08 15:30:17 +08:00
chatgpt-final-release-writer 1f787acede fix(gitops): bind post-advance jobs to exact head 2026-08-08 15:26:35 +08:00
chatgpt-final-release-writer a91bfcaecc fix(gitops): grant authority executor scoped ref access 2026-08-08 15:22:36 +08:00
chatgpt-final-release-writer 6e5e3d8f59 fix(gitops): persist recoverable executor failures 2026-08-08 15:20:14 +08:00
chatgpt-final-release-writer baa3334324 feat(gitops): wire durable execution side effects 2026-08-08 15:15:40 +08:00
chatgpt-final-release-writer 3fbbc18fb2 chore(gitops): preserve worker executable mode 2026-08-08 14:34:36 +08:00
chatgpt-final-release-writer c239407bfc fix(gitops): allow scoped durable jobs with legacy writes disabled 2026-08-08 14:34:28 +08:00
chatgpt-final-release-writer eedf88bedc chore(gitops): bind durable worker to machine credential 2026-08-08 14:31:51 +08:00
chatgpt-final-release-writer 3b815da3cf feat(gitops): make durable job execution transactional 2026-08-08 14:26:40 +08:00
chatgpt-final-release-writer f0b7e9060c fix(gitops): match recovered authorization receipt schema 2026-08-08 13:07:22 +08:00
chatgpt-final-release-writer d20f5829c9 feat(gitops): add continuation lease and durable job boundary 2026-08-08 13:05:06 +08:00
aios-autonomy-agent bba675fab3 feat(openbao): add autonomy runtime v1 candidate 2026-08-08 09:56:59 +08:00
owner 69b45e81dc Merge pull request 'fix(release): close owner auth bootstrap and production executor' (#51) from agent/v2r1c7-production-executor-bootstrap-20260807-r2 into main
Reviewed-on: #51
2026-08-07 17:36:41 +08:00
aios-autonomy-agent 5e5a46f040 fix(release): add local owner ingress and governed production executor
autonomy-unit-tests 23 passed on exact PR head
autonomy-smoke-tests executor CLI smoke passed on exact PR head
git-diff-check 11 additive paths; protected paths unchanged
py-compile 5 runtime modules compiled
reconciliation-contract Authority/Gitea main and branch OIDs reconciled
release-state-machine executor and metadata tests passed
secret-scan 11 changed files scanned; no secret markers
signer-validator-contract validator self-test and signer CLI contract passed
owner-approval-gate Owner actor 2 auto-merge intent verified for exact current head; protected target main; no unresolved conversations; signed-commit gate not required
2026-08-07 17:22:03 +08:00
owner c7adece1cc Merge pull request 'V2R1C7 Production Health Contract' (#50) from release/v2r1c7-production-health-contract-20260807 into main
Reviewed-on: #50
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-07 12:01:17 +08:00
chatgpt-final-release-writer d1353c7142 correct-v2r1c7-superseded-plan-id
autonomy-unit-tests real autonomy-unit-tests receipt for corrected exact PR head
autonomy-smoke-tests real autonomy-smoke-tests receipt for corrected exact PR head
signer-validator-contract real signer-validator-contract receipt for corrected exact PR head
secret-scan real secret-scan receipt for corrected exact PR head
py-compile real py-compile receipt for corrected exact PR head
git-diff-check real git-diff-check receipt for corrected exact PR head
release-state-machine real release-state-machine receipt for corrected exact PR head
reconciliation-contract real reconciliation-contract receipt for corrected exact PR head
owner-approval-gate Owner actor 2 approved exact current head d1353c71; review 103 official and non-stale
2026-08-07 11:37:09 +08:00
chatgpt-final-release-writer 60cf824df3 add-v2r1c7-rebuilt-deployable-artifact
owner-approval-gate fresh pre-merge check pending for exact head
autonomy-unit-tests real autonomy-unit-tests receipt for exact PR head
autonomy-smoke-tests real autonomy-smoke-tests receipt for exact PR head
signer-validator-contract real signer-validator-contract receipt for exact PR head
secret-scan real secret-scan receipt for exact PR head
py-compile real py-compile receipt for exact PR head
git-diff-check real git-diff-check receipt for exact PR head
release-state-machine real release-state-machine receipt for exact PR head
reconciliation-contract real reconciliation-contract receipt for exact PR head
2026-08-07 11:30:17 +08:00
chatgpt-final-release-writer 1fb9a2e153 update-v2r1c7-premerge-evidence 2026-08-07 11:29:44 +08:00
chatgpt-final-release-writer 7f09670511 fix-v2r1c7-bind-runtime-artifact-head-final 2026-08-07 11:27:18 +08:00
chatgpt-final-release-writer 92b4a937e5 fix-v2r1c7-bind-runtime-artifact-head 2026-08-07 11:26:48 +08:00
chatgpt-final-release-writer 720f0575c4 fix-v2r1c7-bind-runtime-artifact 2026-08-07 11:26:13 +08:00
chatgpt-final-release-writer ae7daf77d8 fix-v2r1c7-runtime-state-provider 2026-08-07 11:25:57 +08:00
chatgpt-final-release-writer e76cc07697 fix-v2r1c7-bind-artifact 2026-08-07 11:20:27 +08:00
aios-autonomy-agent 44b220b793 fix(v2r1c7): bind release metadata to rebuilt artifact path 2026-08-07 11:15:58 +08:00
aios-autonomy-agent dbc2c76362 fix(v2r1c7): include formal source manifest in artifact 2026-08-07 11:15:37 +08:00
aios-autonomy-agent c8b50985ea fix(v2r1c7): regenerate release source manifest 2026-08-07 11:15:15 +08:00
aios-autonomy-agent fe94b810dd fix(v2r1c7): close release identity before protected merge 2026-08-07 11:14:42 +08:00
aios-autonomy-agent 0addff8c7b chore(release): close V2R1C7 evidence 2026-08-07 10:53:49 +08:00
aios-autonomy-agent 780f0e2401 feat(release): implement V2R1C7 production health contract 2026-08-07 10:49:40 +08:00
owner b356dec2ac Merge pull request 'feat(v2r1c6): production-ready release preparation' (#49) from release/v2r1c6-production-ready-20260807 into main
Reviewed-on: #49
2026-08-07 07:58:17 +08:00
chatgpt-final-release-writer c972658811 fix(v2r1c6): define health server before entrypoint
autonomy-unit-tests targeted autonomy, release-controller and V2R1C6 tests passed
autonomy-smoke-tests autonomy component smoke suite passed
git-diff-check protected refs, clean tree and append-only lineage verified
py-compile active runtime and release controller Python compiled
reconciliation-contract reconciliation and governed receipt contracts passed
release-state-machine release state machine contract tests passed
secret-scan changed-file secret scan passed
signer-validator-contract signer and validator contract assertions passed
owner-approval-gate owner intent bound to current head; 8 checks success; protected target verified
2026-08-07 02:31:40 +08:00
chatgpt-final-release-writer 3a4c748124 test(v2r1c6): cover release controller contracts 2026-08-07 02:22:42 +08:00
chatgpt-final-release-writer 60bb95d48c feat(v2r1c6): prepare release-ready artifact validation 2026-08-07 02:20:41 +08:00
owner 28081c5522 Merge pull request '[BLOCKED BY PR #46] Deliver governed V2R1C6 release candidate' (#48) from agent/v2r1c6-mainline-reconciliation-20260806 into main
Reviewed-on: #48
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-07 00:14:28 +08:00
chatgpt-final-release-writer cb62eb41b4 fix(release): bind V2R1C6 candidate to PR46 main
autonomy-unit-tests autonomy-unit-tests passed on exact PR48 head cb62eb41b46794433d00600fcbd5de6d81a98811
autonomy-smoke-tests autonomy-smoke-tests passed on exact PR48 head cb62eb41b46794433d00600fcbd5de6d81a98811
signer-validator-contract signer-validator-contract passed on exact PR48 head cb62eb41b46794433d00600fcbd5de6d81a98811
secret-scan secret-scan passed on exact PR48 head cb62eb41b46794433d00600fcbd5de6d81a98811
py-compile py-compile passed on exact PR48 head cb62eb41b46794433d00600fcbd5de6d81a98811
git-diff-check git-diff-check passed on exact PR48 head cb62eb41b46794433d00600fcbd5de6d81a98811
reconciliation-contract reconciliation-contract passed on exact PR48 head cb62eb41b46794433d00600fcbd5de6d81a98811
release-state-machine release-state-machine passed on exact PR48 head cb62eb41b46794433d00600fcbd5de6d81a98811
2026-08-06 23:50:38 +08:00
owner 38794a9be7 Merge branch 'main' into agent/v2r1c6-mainline-reconciliation-20260806
autonomy-unit-tests autonomy-unit-tests passed on exact PR48 head 38794a9be7438ba655ae690d972512562f7c5653
autonomy-smoke-tests autonomy-smoke-tests passed on exact PR48 head 38794a9be7438ba655ae690d972512562f7c5653
signer-validator-contract signer-validator-contract passed on exact PR48 head 38794a9be7438ba655ae690d972512562f7c5653
secret-scan secret-scan passed on exact PR48 head 38794a9be7438ba655ae690d972512562f7c5653
py-compile py-compile passed on exact PR48 head 38794a9be7438ba655ae690d972512562f7c5653
git-diff-check git-diff-check passed on exact PR48 head 38794a9be7438ba655ae690d972512562f7c5653
release-state-machine release-state-machine passed on exact PR48 head 38794a9be7438ba655ae690d972512562f7c5653
reconciliation-contract exact_failed_gate=reconciliation-contract exit_code=72 candidate metadata base mismatch
2026-08-06 22:11:15 +08:00
owner a33e259396 Merge pull request 'Add recoverable AIOS autonomy and branch lifecycle control plane' (#46) from fix/v2r1c6-branch-repair-lease-20260806 into main
Reviewed-on: #46
2026-08-06 22:10:10 +08:00
chatgpt-final-release-writer 174b5647db docs(governance): document durable receipt recovery contract
autonomy-pr46-merged Blocked until autonomy-pr46-merged is satisfied
2026-08-06 19:10:47 +08:00
chatgpt-final-release-writer c3a8a1f024 docs(governance): document durable receipt recovery contract
autonomy-unit-tests 21 targeted autonomy and writer receipt tests passed
autonomy-smoke-tests 19 autonomy component smoke tests passed
signer-validator-contract governance writer/signer contract assertions passed
secret-scan changed-file secret scan passed
py-compile all active-runtime Python files compiled
git-diff-check protected refs unchanged; append-only branch lineage verified
release-state-machine release state machine contract tests passed
reconciliation-contract reconciliation contract tests passed
2026-08-06 19:04:56 +08:00
chatgpt-final-release-writer 7a3600f039 test(governance): align protected writer assertions 2026-08-06 18:39:22 +08:00
chatgpt-final-release-writer 4ea5884749 test(governance): correct writer policy assertion 2026-08-06 18:38:30 +08:00
chatgpt-final-release-writer 82ec252fb1 test(governance): add autonomy and writer receipt contract coverage 2026-08-06 18:36:13 +08:00
chatgpt-final-release-writer 0bbc7b513b feat(governance): bind single-Owner protected merge gate 2026-08-06 18:36:12 +08:00
chatgpt-final-release-writer 73cb54b752 feat(governance): add automatic reconciliation controller 2026-08-06 18:36:11 +08:00
chatgpt-final-release-writer 2efcdb8b6d feat(governance): add governed release state machine 2026-08-06 18:36:11 +08:00
chatgpt-final-release-writer 04d3fd3a04 feat(task-plane): add recurring task scheduling 2026-08-06 18:36:10 +08:00
chatgpt-final-release-writer 7a7e1edca7 feat(task-plane): add Owner and GPT notification outbox 2026-08-06 18:36:09 +08:00
chatgpt-final-release-writer 0395aae50b feat(task-plane): add durable Agent pull runner 2026-08-06 18:36:08 +08:00
chatgpt-final-release-writer d39d56da04 fix(gitops): accept advancing task-writer CAS heads 2026-08-06 18:27:15 +08:00
chatgpt-final-release-writer ae453805c8 fix(gitops): accept advancing task-writer CAS heads 2026-08-06 18:25:53 +08:00
chatgpt-final-release-writer 8e0721dd15 fix(gitops): add durable writer completion receipts 2026-08-06 18:19:19 +08:00
chatgpt-final-release-writer 18957e625f fix(release): regenerate V2R1C6 candidate manifest
autonomy-pr46-merged Blocked until autonomy-pr46-merged is satisfied
2026-08-06 15:15:39 +08:00
chatgpt-final-release-writer 3c2865dbe7 feat(governance): add recoverable branch-bound repair lease 2026-08-06 15:10:06 +08:00
chatgpt-final-release-writer ab828d1ba2 chore(governance): bootstrap repair lease branch 2026-08-06 14:53:28 +08:00
aios-autonomy-agent 7f755ce0b0 feat(openbao): add autonomy runtime v1 candidate 2026-08-06 12:17:58 +08:00
codex ba517a6808 Merge pull request 'Fix signer round-trip binding after protected main advances' (#45) from fix/v2r1c6-post-merge-signer-fixture-20260806 into main
Reviewed-on: #45
Reviewed-by: gpt <gpt@imruyi.top>
Reviewed-by: codex <codex@imruyi.top>
2026-08-06 11:34:48 +08:00
chatgpt-final-release-writer 98b8a7ef20 test(governance): fix legacy OID regression fixture 2026-08-06 11:25:25 +08:00
chatgpt-final-release-writer 61afc7978b test(governance): remove obsolete pre-merge OID pin 2026-08-06 11:24:35 +08:00
chatgpt-final-release-writer 2c1ec409f2 test(governance): cover dynamic signer ref binding 2026-08-06 11:22:31 +08:00
chatgpt-final-release-writer ece4289514 fix(governance): resolve signer base ref from authoritative Git 2026-08-06 11:22:31 +08:00
chatgpt-final-release-writer c1526651d0 chore(governance): bootstrap post-merge hotfix branch 2026-08-06 11:09:33 +08:00
gpt 615c0ce210 Merge pull request 'Fix governed signer contract compatibility for task writer provisioning' (#44) from fix/v2r1c6-signer-contract-compatibility-20260806 into main
Reviewed-on: #44
Reviewed-by: codex <codex@imruyi.top>
Reviewed-by: gpt <gpt@imruyi.top>
2026-08-06 10:32:02 +08:00
gpt 24c55f6a1a Merge branch 'main' into fix/v2r1c6-signer-contract-compatibility-20260806 2026-08-06 10:32:00 +08:00
chatgpt-final-release-writer f3d03f08a7 test(governance): cover deployed signer compatibility and writer result 2026-08-06 10:00:54 +08:00
chatgpt-final-release-writer 3f2b5c78a3 fix(governance): support deployed signer adapter contract 2026-08-06 09:51:14 +08:00
chatgpt-final-release-writer 5d5899f468 fix(governance): route broker through deployed signer contract compatibility 2026-08-06 09:40:02 +08:00
chatgpt-final-release-writer 99d9dd14ef fix(governance): normalize deployed signer contract 2026-08-06 09:37:29 +08:00
chatgpt-final-release-writer d8822e50e4 feat(governance): add task-scoped identity overlay 2026-08-06 09:31:20 +08:00
chatgpt-final-release-writer ae0c84e096 feat(governance): add owner authorization broker 2026-08-06 09:30:19 +08:00
chatgpt-final-release-writer df3aa65ee1 feat(gitops): add authorized task writer bridge 2026-08-06 09:29:28 +08:00
chatgpt-final-release-writer b35aba4fbe chore(governance): bootstrap signer compatibility repair branch 2026-08-06 09:24:48 +08:00
gpt 8b503c8153 Merge pull request 'Register AIOS V2R1C5 privacy artifact release' (#43) from release/v2r1c5-registry-20260805 into main
Reviewed-on: #43
Reviewed-by: codex <codex@imruyi.top>
Reviewed-by: gpt <gpt@imruyi.top>
2026-08-05 15:44:46 +08:00
gpt 54e40218d3 Merge branch 'main' into release/v2r1c5-registry-20260805 2026-08-05 15:44:43 +08:00
chatgpt-final-release-writer 06aa72b337 Register V2R1C5 privacy artifact release in Registry
Owner exact OID approved; protected PR only.
2026-08-05 14:45:02 +08:00
gpt 691e48472a Merge pull request 'Registry: V2R1C4 exact release record' (#42) from release/v2r1c4-registry-20260805T031317Z-registry-47beb37b7acb into main
Reviewed-on: #42
Reviewed-by: codex <codex@imruyi.top>
Reviewed-by: gpt <gpt@imruyi.top>
2026-08-05 11:44:40 +08:00
aios-v2r1c4-registry-proposal-20260805 6b08a421a5 registry: publish V2R1C4 record 2026-08-05 11:29:05 +08:00
469 changed files with 38044 additions and 38199 deletions
-12
View File
@@ -1,12 +0,0 @@
title = "AIOS secret scan policy"
[extend]
useDefault = true
[[allowlists]]
description = "Known non-secret idempotency key used twice by the shadow RBAC test"
targetRules = ["generic-api-key"]
condition = "AND"
regexTarget = "line"
paths = ['''tests/task_plane/public_interface/test_shadow_client_rbac\.py$''']
regexes = ['''idempotency_key="shadow-rbac-idempotent-v1"''']
-2
View File
@@ -1,2 +0,0 @@
# Exact historical fingerprint for the removed test-only scanner self-reference.
a42f16d4f73b11fd23448da2711d49d5e9f49d9e:tests/fastlane/test_fastlane_assets_contract.py:generic-api-key:135
+486
View File
@@ -0,0 +1,486 @@
# Agent Production Deployment Requirements
## Status entering deployment
```yaml
code_development: COMPLETED
repository_commit: NOT_CREATED
production_deployment: NOT_STARTED
reason_repository_commit_missing:
current ChatGPT GitOps runtime has AIOS_GITOPS_WRITES_ENABLED=false
```
The code bundle is an input artifact. The Agent must place it on a non-main
integration branch, review it, run acceptance tests, merge it through protected
main and deploy the API adapter. The Agent must not regenerate or materially
rewrite the supplied implementation without recording the new diff and tests.
## Frozen production state
Before any change, verify:
```yaml
main_oid: 875d41c3101093149414292067fd700dc214d4d3
V2R1C5:
tag_oid: 9e65bd59baea392294c0f253046d3ab116f63145
peeled_target_oid: aa60e891547abd7e215af7cf2bf3f21fd23541da
baseline_version: V2R1C5
baseline_status: PRODUCTION_DEPLOYED
target_oid: aa60e891547abd7e215af7cf2bf3f21fd23541da
evidence_closure_oid: 5e2cf8276f5093f3352b9a3252c701a44c5ab807
```
If authoritative `main` no longer equals the approved base OID, stop with
`MANUAL_OWNER_REVIEW_REQUIRED / AUTHORIZED_BASE_OID_DRIFT`. Do not silently
rebase the bundle onto a different base.
## Bootstrap sequence — no additional Owner interaction
Use the following order so that the approved V2R1C6 base OID remains valid and
the bridge does not create its own authorization deadlock:
1. Verify the supplied archive/patch SHA-256.
2. Install the bundle into a temporary root-owned staging directory outside the
Git worktrees.
3. Start a localhost-only temporary instance of
`aios_authorized_task_writer_api.py` on a dedicated port or Unix socket.
4. Bind it to the existing authenticated Owner registry and the exact
`writer:provision` scope. Do not expose it through the public gateway.
5. Execute the live V2R1C6 provisioning request while `main` still equals
`875d41c3101093149414292067fd700dc214d4d3`.
6. Verify `READY_FOR_CODE_AUTHORING`, the exact branch OID, writer scope and
negative policy tests.
7. Use the newly created scoped writer to commit this reviewed bundle onto:
`agent/v2r1c6-mainline-reconciliation-20260805`.
8. Stop and remove the temporary bootstrap service and revoke its temporary
runtime credential. Preserve its signed grant receipt and audit logs.
9. Open a protected PR from the V2R1C6 branch. Deploy the permanent bridge only
from the reviewed merge commit.
The temporary service is a control-plane bootstrap, not a production release.
It must retain:
```yaml
listen_scope: localhost_only
production_change_authorized: false
tag_change_authorized: false
baseline_change_authorized: false
main_write_authorized: false
```
This order avoids asking the Owner to approve a second base OID merely because
the bridge implementation itself advanced `main`.
### Alternative platform-maintenance path
If policy prohibits a temporary localhost bootstrap service, use an existing
Owner-approved platform-maintenance executor to create:
```yaml
branch: patch/owner-authorization-writer-bridge-20260806
base_ref: refs/heads/main
base_oid: 875d41c3101093149414292067fd700dc214d4d3
direct_main_push: false
force_push: false
tag_write: false
baseline_write: false
production_deploy_during_integration: false
```
Merging that branch before V2R1C6 provisioning changes the authoritative base
OID. In that alternative only, a new exact Owner base-OID amendment is required.
The Agent must never substitute the newer `main` OID automatically.
## Files to install on the branch
Copy with the same relative paths:
```text
active-runtime/app/authorized_task_writer_provisioning.py
active-runtime/app/aios_authorized_task_writer_api.py
docs/gitops/authorized-task-writer-provisioning.md
docs/gitops/openapi-authorized-task-writer-fragment.yaml
deploy/systemd/aios-gitops-model-api-authorized-writer-bridge.conf.example
scripts/smoke_authorized_task_writer_provisioning.py
tests/test_authorized_task_writer_provisioning.py
```
Merge the OpenAPI fragment into:
```text
docs/gitops/openapi-chatgpt-actions.yaml
```
The final schema must expose:
```yaml
operationId: provisionAuthorizedTaskWriter
path: POST /api/task-writer/provision-authorized
x-openai-isConsequential: true
```
Do not deploy the fragment as a second standalone OpenAPI document.
## Required review of existing API integration
The adapter expects the existing API handler to provide:
```python
handler.need(scope)
handler.body()
handler.sendj(value, status=200)
api.H.do_POST
api.main()
```
Confirm these contracts against the branch before merge. If the live API has a
different authentication object shape, make the smallest adapter-only change
and rerun every test. The core provisioning policy must remain unchanged.
The API authorization registry must grant `writer:provision` only to the
authenticated Owner role. Agents and ordinary writer tokens must not receive
this scope.
## Privilege boundary
The API service must remain non-root.
Install a root-owned sudoers rule that allows the API service identity to run
only these exact commands with `sudo -n --`:
```text
/usr/local/libexec/aios-grant-signer-adapter
/usr/local/libexec/aios-grant-validator
/usr/local/libexec/aios-task-writer-provisioner
```
Do not allow:
```text
/bin/sh
/bin/bash
/usr/bin/env
/usr/bin/git
wildcard executable paths
SETENV
arbitrary environment preservation
```
The governed tools themselves must validate their `--grant-dir` and
`--result-json` paths. Their binaries must be root-owned, non-symlink, regular,
executable and not group/world writable.
The signer must access its private signing material only through a runtime
secret reference. No signing key or token may enter Git, API output, the
provisioning receipt or systemd environment text.
## Runtime paths and permissions
Create:
```text
/var/lib/aios-gitops/authorized-writer-bridge
/var/lib/aios-gitops/authorized-writer-bridge/receipts
/var/lib/aios-gitops/authorized-writer-bridge/locks
```
Required properties:
```yaml
owner: API service identity
mode: "0700"
symlink: false
persistent: true
```
The directory contains redacted receipts only. Temporary signed grant
directories are deleted after success or failure.
## Required staging environment
Configure the API service:
```text
AIOS_GITOPS_REPOSITORY_IDENTITY=aios/aios
AIOS_GITOPS_AUTHORITY_GIT_DIR=/data/aios-gitops/repo.git
AIOS_GITOPS_AUTHORIZED_WRITER_STATE_ROOT=/var/lib/aios-gitops/authorized-writer-bridge
AIOS_GITOPS_GRANT_SIGNER_ADAPTER=/usr/local/libexec/aios-grant-signer-adapter
AIOS_GITOPS_GRANT_VALIDATOR=/usr/local/libexec/aios-grant-validator
AIOS_GITOPS_TASK_WRITER_PROVISIONER=/usr/local/libexec/aios-task-writer-provisioner
AIOS_GITOPS_OWNER_PROVISION_SCOPE=writer:provision
AIOS_GITOPS_GOVERNED_USE_SUDO=true
AIOS_GITOPS_SUDO_PATH=/usr/bin/sudo
```
Use `aios_authorized_task_writer_api.py` as the API entrypoint. It delegates
all pre-existing routes and adds only the new endpoint.
## Pre-merge tests
Run from the repository root:
```bash
python3 -m unittest -v tests/test_authorized_task_writer_provisioning.py
python3 scripts/smoke_authorized_task_writer_provisioning.py
python3 -m py_compile \
active-runtime/app/authorized_task_writer_provisioning.py \
active-runtime/app/aios_authorized_task_writer_api.py
```
Required result:
```yaml
unit_tests:
total: 9
failures: 0
errors: 0
smoke: PASS
py_compile: PASS
```
Also run all existing GitOps API and isolated writer tests affected by:
```text
aios_chatgpt_actions_api
aios_isolated_writer_api
isolated_writer_authz
isolated_writer_token_registry
single_owner_governance
OpenAPI schema validation
```
Any regression blocks merge.
## Staging API acceptance
### Negative authentication
An Agent or non-Owner credential calling the endpoint must receive:
```yaml
http_status: 403
error: OWNER_AUTHORIZATION_REQUIRED
```
### Negative scope tests
Requests enabling any of the following must be denied before signer execution:
```text
production_change_authorized=true
tag_change_authorized=true
baseline_change_authorized=true
main_write_authorized=true
allowed_operations containing tag/main/baseline/admin operations
missing required denied operation
one_time_use=false
expiry over 1800 seconds
```
### Drift and conflict tests
Verify:
```yaml
wrong_base_oid:
http_status: 409
error: AUTHORIZED_BASE_OID_DRIFT
existing_unreceipted_target_branch:
http_status: 409
error: EXISTING_BRANCH_OID_CONFLICT
```
### Governed tool failures
Independently inject and verify fail-closed behavior for:
```text
signer unavailable
signer signature absent
validator signature_valid=false
validator signer_trusted=false
validator plan expired
provisioner grant_consumed=false
provisioner writer scope enables main/tag/baseline/force push
any required negative policy test false
```
None may create or modify a Git ref.
## Live V2R1C6 provisioning request
After protected merge and API deployment, authenticate as the Owner and submit
exactly:
```json
{
"task_id": "V2R1C6_TASK_SCOPED_WRITER_PROVISIONING",
"repository": "aios/aios",
"base_ref": "refs/heads/main",
"base_oid": "875d41c3101093149414292067fd700dc214d4d3",
"target_branch": "agent/v2r1c6-mainline-reconciliation-20260805",
"owner_authorization_reference": "chat-owner-authorization:v2r1c6:20260806",
"one_time_use": true,
"expires_in_seconds": 900,
"production_change_authorized": false,
"tag_change_authorized": false,
"baseline_change_authorized": false,
"main_write_authorized": false,
"allowed_operations": [
"create_worktree",
"create_target_branch_if_absent",
"commit_to_target_branch",
"push_target_branch"
],
"denied_operations": [
"direct_main_push",
"force_push",
"tag_create",
"tag_update",
"tag_delete",
"branch_delete",
"repository_admin",
"baseline_write",
"production_deploy",
"production_marker_write",
"history_rewrite"
]
}
```
This request must be executed by the localhost-only bootstrap instance before
the bridge code advances `main`. The endpoint must never infer “current main”.
If the Agent chooses the alternative merge-first path, it must stop after the
merge and obtain an exact Owner amendment for the new merge OID. It must not
silently replace the approved OID.
## Success response
Require:
```yaml
status: READY_FOR_CODE_AUTHORING
grant_consumed: true
branch: agent/v2r1c6-mainline-reconciliation-20260805
branch_oid: <exact amended authorized base OID>
base_oid: <same exact OID>
main_unchanged: true
writer_scope:
workspace_role: task-scoped-isolated-writer
writes_enabled: true
allowed_branch: agent/v2r1c6-mainline-reconciliation-20260805
allowed_base_oid: <exact amended authorized base OID>
allowed_routes:
- POST /api/commit-file
allow_main_write: false
allow_tag_write: false
allow_baseline_write: false
allow_force_push: false
```
All seven negative policy results must be true.
Submit the identical request a second time. It must return the same grant and
branch attestation with:
```yaml
idempotent: true
```
It must not run the provisioner again.
## Production postchecks
After API deployment and writer provisioning, independently verify:
```yaml
existing_routes_regression: PASS
new_endpoint: PASS
main_ref: expected reviewed merge commit
V2R1C5_tag_oid: 9e65bd59baea392294c0f253046d3ab116f63145
V2R1C5_peeled_target: aa60e891547abd7e215af7cf2bf3f21fd23541da
V2R1C5_baseline_unchanged: true
new_release_tag_created: false
production_baseline_changed: false
```
The deployment of this control-plane bridge does not authorize a V2R1C6
production release. It only enables scoped code authoring.
## Rollback
If the new API route fails:
1. Stop accepting new provisioning calls.
2. Restore the previous API entrypoint.
3. Restart and verify all existing routes.
4. Preserve receipts and logs.
5. Revoke any task writer capability already minted.
6. Do not delete a branch or rewrite a tag.
7. Revert the protected-main bridge commit through an ordinary reviewed revert
PR if source rollback is required.
## Required final Agent report
```yaml
STATUS: COMPLETED | PARTIALLY_COMPLETED | BLOCKED | FAILED
INTEGRATION:
branch:
commit:
protected_pr:
merge_commit:
files_changed:
TESTS:
unit:
smoke:
py_compile:
regression:
fault_injection:
DEPLOYMENT:
service:
entrypoint:
runtime_status:
endpoint_status:
rollback_verified:
PROVISIONING:
owner_authorization_reference:
approval_plan_id:
grant_id:
grant_digest_sha256:
signing_key_identity:
branch:
branch_oid:
workspace_path:
grant_consumed:
idempotent_replay:
CONSISTENCY:
main_oid:
V2R1C5_tag_oid:
V2R1C5_peeled_target_oid:
baseline_version:
baseline_changed:
tags_changed:
SECRETS:
runtime_secret_reference_used:
plaintext_secret_logged:
plaintext_secret_committed:
BLOCKERS:
GOVERNANCE:
force_push_used: false
tag_rewritten: false
direct_main_push: false
secrets_committed: false
owner_approval_respected: true
```
@@ -0,0 +1,401 @@
# Agent Deployment Requirements: Owner Authorization Broker V2
## Objective
Deploy the audited Owner Authorization Broker that closes the bootstrap gap
without issuing a new general-purpose Owner credential and without modifying
`/data/aios-gitops/runtime/secrets/gitops.tokens`.
The broker reuses one existing, non-wildcard Owner credential only to
authenticate the Owner identity through one pre-existing exact scope. It then
records an immutable task authorization event and directly invokes the
existing signed grant -> validator -> one-shot provisioner chain.
Production deployment of V2R1C6 is **not** authorized by this task.
## Fixed bindings
```yaml
task_id: V2R1C6_TASK_SCOPED_WRITER_PROVISIONING
repository: aios/aios
base_ref: refs/heads/main
base_oid: 875d41c3101093149414292067fd700dc214d4d3
target_branch: agent/v2r1c6-mainline-reconciliation-20260805
production_change_authorized: false
tag_change_authorized: false
baseline_change_authorized: false
main_write_authorized: false
```
The existing V2R1C5 objects are frozen:
```yaml
tag_oid: 9e65bd59baea392294c0f253046d3ab116f63145
peeled_target_oid: aa60e891547abd7e215af7cf2bf3f21fd23541da
baseline_version: V2R1C5
baseline_status: PRODUCTION_DEPLOYED
```
## 1. Package verification
Verify `MANIFEST.sha256` before installation. Reject any mismatch.
Run:
```bash
python3 -m unittest -v \
tests/test_authorized_task_writer_provisioning.py \
tests/test_owner_authorization_broker.py
python3 scripts/smoke_authorized_task_writer_provisioning.py
python3 scripts/smoke_owner_authorization_broker.py
python3 -m py_compile \
active-runtime/app/authorized_task_writer_provisioning.py \
active-runtime/app/aios_authorized_task_writer_api.py \
active-runtime/app/owner_authorization_broker.py \
active-runtime/app/aios_owner_authorization_broker_api.py
```
All checks must pass.
## 2. Existing Owner credential selection
Read only the non-secret registry metadata. Select one existing credential
that satisfies all of the following:
```yaml
role: owner
wildcard_scope: false
account: exact known Owner identity
contains_exact_intake_scope: true
```
Use one existing exact scope such as `commit:create` or `file:write`. Prefer
the narrowest scope already assigned to the intended Owner.
Do not:
- use the historical wildcard Owner credential;
- add `writer:provision` to any bearer token;
- create or modify a bearer credential;
- print, copy or commit bearer-token material.
Record only:
```yaml
owner_account: <non-secret account>
credential_id: <non-secret ID, when available>
intake_scope: <one exact existing scope>
scope_readback_available: true|false
```
If `handler.need()` returns the exact scope set, configure the Owner account
allowlist and leave the credential ID allowlist empty.
If scope metadata is not returned, configure the exact non-secret credential
ID allowlist. If neither exact scope metadata nor a stable non-secret
credential ID can be proven, stop:
```yaml
STATUS: BLOCKED
REASON: OWNER_CREDENTIAL_SCOPE_NOT_PROVABLE
```
## 3. Staging installation
Install the four application modules in a root-owned staging directory:
```text
authorized_task_writer_provisioning.py
aios_authorized_task_writer_api.py
owner_authorization_broker.py
aios_owner_authorization_broker_api.py
```
The directory and files must not be group/world writable.
Prepare environment values:
```yaml
AIOS_GITOPS_OWNER_AUTHORIZATION_BROKER_ENABLED: "true"
AIOS_GITOPS_OWNER_AUTHORIZATION_INTAKE_SCOPES: <one exact existing scope>
AIOS_GITOPS_OWNER_AUTHORIZATION_ALLOWED_IDENTITIES: <exact owner account>
AIOS_GITOPS_OWNER_AUTHORIZATION_ALLOWED_CREDENTIAL_IDS: <exact ID or empty>
AIOS_GITOPS_OWNER_AUTHORIZATION_MAX_EXPIRY_SECONDS: "900"
AIOS_GITOPS_OWNER_AUTHORIZATION_STATE_ROOT:
/var/lib/aios-gitops/owner-authorization-broker
```
Use the existing signer, validator and provisioner paths. Do not change their
root governance configuration or disable signature checks.
Start a localhost-only staging instance on a temporary port. Do not replace
the production service yet.
## 4. Authentication fail-closed tests
The staging endpoint is:
```text
POST /api/governance/owner-authorizations
```
Required tests:
```yaml
unauthenticated:
expected: 401
wrong_role:
expected: 403
wrong_scope:
expected: 403
wildcard_owner:
expected: 403
error: WILDCARD_OWNER_CREDENTIAL_REJECTED
unallowlisted_owner:
expected: 403
error: OWNER_IDENTITY_NOT_ALLOWED
production_capability_true:
expected: 403
error: PRODUCTION_CAPABILITY_REJECTED
```
A connection failure, 404 or 500 is not an authorization-policy pass.
## 5. Formal provisioning request
Use the selected existing non-wildcard Owner credential through its runtime
secret reference. Never place the bearer token in the request body, command
line, shell history, output report or Git.
Request:
```json
{
"action": "AUTHORIZE_TASK_WRITER_PROVISIONING",
"task_id": "V2R1C6_TASK_SCOPED_WRITER_PROVISIONING",
"repository": "aios/aios",
"base_ref": "refs/heads/main",
"base_oid": "875d41c3101093149414292067fd700dc214d4d3",
"target_branch": "agent/v2r1c6-mainline-reconciliation-20260805",
"one_time_use": true,
"expires_in_seconds": 900,
"production_change_authorized": false,
"tag_change_authorized": false,
"baseline_change_authorized": false,
"main_write_authorized": false,
"allowed_operations": [
"create_worktree",
"create_target_branch_if_absent",
"commit_to_target_branch",
"push_target_branch"
],
"denied_operations": [
"direct_main_push",
"force_push",
"tag_create",
"tag_update",
"tag_delete",
"branch_delete",
"repository_admin",
"baseline_write",
"production_deploy",
"production_marker_write",
"history_rewrite"
]
}
```
The Owner authorization reference is generated by the broker. The caller must
not submit or invent one.
Expected result:
```yaml
status: READY_FOR_CODE_AUTHORING
owner_authorization_event_id: generated
owner_authorization_reference: generated
owner_authorization_digest_sha256: generated
owner_authorization_consumed: true
approval_plan_id: generated
grant_digest_sha256: generated
grant_consumed: true
branch_oid: 875d41c3101093149414292067fd700dc214d4d3
writer_scope:
workspace_role: task-scoped-isolated-writer
writes_enabled: true
allowed_branch: agent/v2r1c6-mainline-reconciliation-20260805
allowed_base_oid: 875d41c3101093149414292067fd700dc214d4d3
allow_main_write: false
allow_tag_write: false
allow_baseline_write: false
allow_force_push: false
```
## 6. Independent post-provisioning verification
Verify the Authority ref:
```bash
git --git-dir=/data/aios-gitops/repo.git rev-parse \
refs/heads/agent/v2r1c6-mainline-reconciliation-20260805
```
It must equal:
```text
875d41c3101093149414292067fd700dc214d4d3
```
Verify `main` remains unchanged, V2R1C5 tag object and peeled target remain
unchanged, and the Production Baseline remains V2R1C5.
Verify the broker audit event under:
```text
/var/lib/aios-gitops/owner-authorization-broker/events/
```
Requirements:
```yaml
status: CONSUMED
wildcard_scope_used: false
grant_consumed: true
permissions:
directory: "0700"
json_files: "0600"
```
Do not return bearer-token material in the report.
## 7. Commit the broker through the new writer
After `READY_FOR_CODE_AUTHORING`, use only the writer's
`POST /api/commit-file` route to put this code bundle on:
```text
agent/v2r1c6-mainline-reconciliation-20260805
```
Commit the bridge and broker as separate intentional commits when practical:
1. Owner-authorized task-writer bridge.
2. Audited Owner authorization broker and deployment docs.
Do not merge to `main` directly. Create a protected PR after branch commits
and tests are complete.
## 8. Permanent service deployment
Only after protected review and merge:
1. Install the reviewed files from the merge commit.
2. Apply a reviewed systemd drop-in based on the provided example.
3. Start the wrapper entrypoint:
`aios_owner_authorization_broker_api.py`.
4. Verify all existing API routes still work.
5. Verify the broker remains localhost/reverse-proxy constrained as intended.
6. Re-run authentication and policy tests.
7. Remove the temporary staging instance.
This permanent bridge deployment does not authorize a V2R1C6 production tag,
Baseline update or application deployment.
## 9. Fail-closed conditions
Stop with `BLOCKED` for any of the following:
- Authority `main` no longer equals the fixed base OID.
- Target branch exists at a different OID.
- Only a wildcard Owner credential can authenticate.
- Exact credential scope cannot be proven.
- Owner identity is not explicitly allow-listed.
- Signer, validator or provisioner self-test fails.
- Grant is not marked one-time and consumed.
- Writer allows main, tag, Baseline, force-push or branch-delete operations.
- Audit event contains sensitive material.
- V2R1C5 tag or Baseline changes.
## 10. Required Agent result
```yaml
STATUS: READY_FOR_CODE_AUTHORING | BLOCKED | FAILED
PACKAGE:
manifest_verified:
unit_tests:
smoke_tests:
py_compile:
OWNER_CREDENTIAL:
account:
credential_id:
intake_scope:
exact_scope_proven:
wildcard_used: false
token_logged: false
BROKER:
staging_listener:
endpoint_status:
event_id:
authorization_reference:
authorization_digest_sha256:
event_consumed:
audit_permissions_valid:
GRANT:
approval_plan_id:
grant_digest_sha256:
signer_identity:
signature_valid:
one_time_use:
consumed:
BRANCH:
name:
oid:
base_oid:
ref_exists:
worktree_path:
worktree_clean:
WRITER:
status:
writes_enabled:
allowed_branch:
allowed_base_oid:
allowed_routes:
main_write_denied:
tag_write_denied:
baseline_write_denied:
force_push_denied:
POSTCHECK:
main_unchanged:
V2R1C5_tag_unchanged:
baseline_unchanged:
REPOSITORY_INTEGRATION:
commit_oids:
protected_pr:
merged:
permanent_bridge_deployed:
BLOCKERS:
GOVERNANCE:
force_push_used: false
tag_rewritten: false
direct_main_push: false
secrets_committed: false
owner_approval_respected: true
```
+562
View File
@@ -0,0 +1,562 @@
# Agent execution requirements: V2R1C6 task identity overlay
## Required result
The Agent must return one of:
- `READY_FOR_CODE_AUTHORING`
- `BLOCKED`
- `FAILED`
The Agent may not report success until the task branch, isolated worktree and
task writer are independently verified.
This stage is **not** a production deployment. It must not modify `main`,
V2R1C5, tags, Registry, Production Baseline or production services.
## Fixed authority
```yaml
task_id: V2R1C6_TASK_SCOPED_WRITER_PROVISIONING
repository: aios/aios
base_ref: refs/heads/main
base_oid: 875d41c3101093149414292067fd700dc214d4d3
target_branch: agent/v2r1c6-mainline-reconciliation-20260805
owner_identity: aiman
one_time_use: true
maximum_ttl_seconds: 900
production_change_authorized: false
tag_change_authorized: false
baseline_change_authorized: false
main_write_authorized: false
```
## 1. Verify package
Verify the externally supplied package against the published SHA-256 values and
then verify every entry in `MANIFEST.sha256`.
Do not synthesize a missing archive. Use either the verified ZIP or TAR.GZ.
Required checks:
```yaml
package_hash_match: true
manifest_entries_match: true
unit_tests_pass: true
smoke_tests_pass: true
py_compile_pass: true
```
## 2. Create root-owned staging
Extract to a fresh directory such as:
```text
/var/lib/aios-gitops/staging/v2r1c6-task-identity-overlay-<nonce>
```
Requirements:
```yaml
owner: root
group: root
directory_mode: "0700"
files_not_world_writable: true
symlinks_rejected: true
```
Do not overwrite the active runtime tree.
## 3. Re-run target-host tests
From the staging root:
```bash
python3 -m unittest discover -s tests -v
python3 scripts/smoke_authorized_task_writer_provisioning.py
python3 scripts/smoke_owner_authorization_broker.py
python3 scripts/smoke_task_identity_overlay.py
python3 -m py_compile \
active-runtime/app/authorized_task_writer_provisioning.py \
active-runtime/app/owner_authorization_broker.py \
active-runtime/app/task_identity_overlay.py \
active-runtime/app/aios_authorized_task_writer_api.py \
active-runtime/app/aios_owner_authorization_broker_api.py \
active-runtime/app/aios_task_identity_overlay_api.py \
active-runtime/app/aios_task_identity_overlay_cli.py \
scripts/call_task_identity_overlay.py
```
Any failure is `BLOCKED`; do not proceed.
## 4. Prestart Git and production freeze checks
Read from the authority repository:
```bash
git --git-dir=/data/aios-gitops/repo.git rev-parse refs/heads/main
git --git-dir=/data/aios-gitops/repo.git \
show-ref --verify --quiet \
refs/heads/agent/v2r1c6-mainline-reconciliation-20260805
git --git-dir=/data/aios-gitops/repo.git rev-parse refs/tags/V2R1C5
git --git-dir=/data/aios-gitops/repo.git rev-parse 'refs/tags/V2R1C5^{}'
```
Expected:
```yaml
main_oid: 875d41c3101093149414292067fd700dc214d4d3
target_branch_exists: false
V2R1C5_tag_oid: 9e65bd59baea392294c0f253046d3ab116f63145
V2R1C5_peeled_target: aa60e891547abd7e215af7cf2bf3f21fd23541da
```
Read Baseline and require:
```yaml
version: V2R1C5
status: PRODUCTION_DEPLOYED
target_oid: aa60e891547abd7e215af7cf2bf3f21fd23541da
evidence_closure_oid: 5e2cf8276f5093f3352b9a3252c701a44c5ab807
```
If `main` has drifted, return:
```yaml
STATUS: MANUAL_OWNER_REVIEW_REQUIRED
REASON: AUTHORIZED_BASE_OID_DRIFT
```
If the target branch already exists at another OID, return:
```yaml
STATUS: MANUAL_OWNER_REVIEW_REQUIRED
REASON: EXISTING_BRANCH_OID_CONFLICT
```
Do not delete, reset or force-push the branch.
## 5. Discover active secret references
The overlay does not read or modify `gitops.tokens`, but its issuer must compare
the new random secret against all active bearer secret references.
Discover references without printing secret contents. At minimum include the
currently active ChatGPT/GitOps bearer secret reference and any other bearer
files consumed by the active API.
Pass them as a comma-separated list in:
```text
AIOS_GITOPS_IDENTITY_OVERLAY_COLLISION_SECRET_REFS
```
Evidence must contain only paths and SHA-256 match counts, not token values.
If active secret references cannot be enumerated, return:
```yaml
STATUS: BLOCKED
REASON: ACTIVE_SECRET_REFERENCE_SET_UNRESOLVED
```
## 6. Install the transient systemd service
Use the provided template:
```text
deploy/systemd/aios-v2r1c6-task-identity-overlay.service.example
```
Substitute only:
- verified root-owned staging directory;
- verified active secret-ref path list.
Do not change task ID, repository, base OID, target branch or denied
capabilities.
The service must listen only on:
```text
127.0.0.1:19864
```
Before start, run:
```bash
systemd-analyze verify \
/etc/systemd/system/aios-v2r1c6-task-identity-overlay.service
```
Then:
```bash
sudo systemctl daemon-reload
sudo systemctl start aios-v2r1c6-task-identity-overlay.service
```
Confirm:
```bash
curl --fail --silent http://127.0.0.1:19864/healthz
```
Expected:
```yaml
service: aios-task-identity-overlay
status: active
global_token_registry_used: false
loopback_only: true
```
Do not expose the port externally.
## 7. Issue the one-time overlay
Use the root-only CLI. The authorization assertion is an audit label for this
already approved Owner task; it must not be described as an external signed
event.
Example:
```bash
sudo env \
PYTHONPATH=<staging>/active-runtime/app \
AIOS_GITOPS_IDENTITY_OVERLAY_STATE_ROOT=/var/lib/aios-gitops/task-identity-overlays \
AIOS_GITOPS_IDENTITY_OVERLAY_SECRET_ROOT=/run/aios-gitops/task-identity-overlays \
AIOS_GITOPS_IDENTITY_OVERLAY_TASK_ID=V2R1C6_TASK_SCOPED_WRITER_PROVISIONING \
AIOS_GITOPS_IDENTITY_OVERLAY_OWNER_IDENTITY=aiman \
AIOS_GITOPS_IDENTITY_OVERLAY_BASE_REF=refs/heads/main \
AIOS_GITOPS_IDENTITY_OVERLAY_BASE_OID=875d41c3101093149414292067fd700dc214d4d3 \
AIOS_GITOPS_IDENTITY_OVERLAY_TARGET_BRANCH=agent/v2r1c6-mainline-reconciliation-20260805 \
AIOS_GITOPS_IDENTITY_OVERLAY_COLLISION_SECRET_REFS=<verified-secret-ref-list> \
python3 <staging>/active-runtime/app/aios_task_identity_overlay_cli.py \
issue \
--ttl-seconds 900 \
--authorization-assertion-ref owner-session:V2R1C6:approved-20260806 \
--operator-identity <audited-agent-identity> \
--result-json /run/aios-gitops/v2r1c6-overlay-issued.json
```
The result must contain:
```yaml
status: OVERLAY_ACTIVE
idempotent: false
overlay_id: present
secret_ref: under /run/aios-gitops/task-identity-overlays
expires_at: within 900 seconds
```
It must not contain a token.
Verify:
```yaml
secret_file_mode: "0600"
metadata_contains_raw_token: false
token_hash_unique_across_configured_secret_refs: true
```
## 8. Negative tests
Before the valid request:
1. No Authorization header must return HTTP 401.
2. A random wrong bearer must return HTTP 401.
3. A request with `production_change_authorized=true` must be rejected.
4. A request with another base OID or branch must be rejected.
5. The valid overlay must remain `ACTIVE` after pre-auth request-validation
failures.
Do not use the global Owner or ChatGPT bearer credential for this endpoint.
## 9. Send the valid request
Use:
```text
deploy/v2r1c6-task-writer-authorization-request.json
```
Use the provided client so the bearer is not placed in argv, shell history or
logs:
```bash
sudo python3 <staging>/scripts/call_task_identity_overlay.py \
--secret-ref <secret_ref_from_issue_result> \
--request-json <staging>/deploy/v2r1c6-task-writer-authorization-request.json \
--url http://127.0.0.1:19864/api/governance/owner-authorizations \
--result-json /var/lib/aios-gitops/v2r1c6-overlay-provision-result.json
```
Required result:
```yaml
http_status: 200
status: READY_FOR_CODE_AUTHORING
owner_authorization_consumed: true
grant_consumed: true
identity_overlay_consumed: true
identity_overlay_secret_removed: true
branch: agent/v2r1c6-mainline-reconciliation-20260805
branch_oid: 875d41c3101093149414292067fd700dc214d4d3
```
The result must not include:
- bearer token;
- private key;
- signer secret;
- raw signature;
- credential value.
## 10. Independent acceptance
Verify the branch:
```bash
git --git-dir=/data/aios-gitops/repo.git \
rev-parse refs/heads/agent/v2r1c6-mainline-reconciliation-20260805
```
It must equal:
```text
875d41c3101093149414292067fd700dc214d4d3
```
Verify worktree evidence returned by the provisioner:
```yaml
branch: agent/v2r1c6-mainline-reconciliation-20260805
head_oid: 875d41c3101093149414292067fd700dc214d4d3
detached_head: false
worktree_clean: true
writer_status: active
writes_enabled: true
allowed_routes:
- POST /api/commit-file
allow_main_write: false
allow_tag_write: false
allow_baseline_write: false
allow_force_push: false
```
Run policy negative tests against the task writer and require explicit policy
denials for:
- main write;
- another branch;
- tag create/update;
- Baseline write;
- force push;
- branch delete.
Network errors, 404, 500 and stopped-service failures do not count as policy
denials.
## 11. Stop and clean up the overlay
After `READY_FOR_CODE_AUTHORING`:
```bash
sudo systemctl stop aios-v2r1c6-task-identity-overlay.service
sudo systemctl disable aios-v2r1c6-task-identity-overlay.service || true
sudo rm -f /etc/systemd/system/aios-v2r1c6-task-identity-overlay.service
sudo systemctl daemon-reload
```
Verify:
```yaml
overlay_listener_stopped: true
overlay_secret_removed: true
global_token_registry_unchanged: true
disabled_owner_record_unchanged: true
active_chatgpt_bot_record_unchanged: true
audit_metadata_retained: true
```
The root-owned staging directory may be retained until the code is committed,
then removed after checksum-confirmed repository integration.
## 12. Commit the developed code through the task writer
Only after writer acceptance, use `POST /api/commit-file` to commit the
verified package contents to:
```text
agent/v2r1c6-mainline-reconciliation-20260805
```
Commit in reviewable groups:
1. authorized task-writer bridge;
2. Owner authorization broker;
3. task identity overlay;
4. tests, docs, OpenAPI and deployment templates.
Do not include:
- package archives;
- runtime tokens;
- `/run` or `/var/lib` state;
- test result files containing host paths;
- secret references containing values;
- generated credentials.
Run the complete 40-test suite from the branch after commits.
## 13. Protected PR
Create a protected PR from:
```text
agent/v2r1c6-mainline-reconciliation-20260805
```
to:
```text
main
```
The PR must state:
- this change closes Owner authorization bootstrap gaps;
- the overlay is localhost-only and one-time;
- the global token registry is not modified;
- no production release is authorized;
- V2R1C5 tag and Baseline remain unchanged.
The Agent must not merge without the repository's normal protected review.
## 14. Required Agent report
```yaml
STATUS:
READY_FOR_CODE_AUTHORING | BLOCKED | FAILED
PACKAGE:
zip_sha256:
tar_gz_sha256:
manifest_entries:
manifest_matched:
TESTS:
unit_total:
unit_passed:
smoke_bridge:
smoke_broker:
smoke_overlay:
py_compile:
PRESTART:
main_oid:
target_branch_previously_absent:
V2R1C5_tag_oid:
V2R1C5_peeled_target:
baseline_unchanged:
OVERLAY:
overlay_id:
status:
secret_ref:
secret_file_mode:
token_hash_unique:
global_registry_used: false
listener:
expires_at:
consumed:
secret_removed:
NEGATIVE_TESTS:
unauthenticated_401:
wrong_token_401:
production_capability_denied:
wrong_base_denied:
wrong_branch_denied:
OWNER_AUTHORIZATION:
event_id:
reference:
consumed:
GRANT:
approval_plan_id:
grant_id:
grant_digest_sha256:
signature_valid:
consumed:
BRANCH:
name:
oid:
base_oid:
ref_exists:
worktree_path:
worktree_clean:
WRITER:
status:
writes_enabled:
allowed_branch:
allowed_base_oid:
allowed_routes:
main_write_denied:
tag_write_denied:
baseline_write_denied:
force_push_denied:
REPOSITORY_INTEGRATION:
commits:
tests_after_commit:
protected_pr:
merged: false
POSTCHECK:
main_unchanged:
V2R1C5_tag_unchanged:
baseline_unchanged:
global_token_registry_unchanged:
overlay_listener_stopped:
overlay_secret_removed:
SECRETS:
plaintext_logged: false
plaintext_committed: false
BLOCKERS:
GOVERNANCE:
force_push_used: false
tag_rewritten: false
direct_main_push: false
secrets_committed: false
owner_approval_respected: true
```
## Fail-closed conditions
Return `BLOCKED` and stop on any of:
- authoritative main OID drift;
- target branch conflict;
- inability to enumerate active secret references;
- secret hash collision;
- overlay not bound to the exact task/base/branch;
- non-loopback listener;
- raw secret in result or logs;
- Owner Authorization Broker event not created;
- signer/validator/provisioner failure;
- grant not consumed;
- branch OID mismatch;
- writer exposes main/tag/Baseline/force-push permissions;
- V2R1C5 tag or Baseline changes.
Never fall back to the shared `gitops-write-bot.token`, a wildcard credential,
manual `gitops.tokens` edits or a direct Git ref write.
+9
View File
@@ -0,0 +1,9 @@
{
"schema": "aios.v2r1c7.engineering-build-receipt.v1",
"version": "V2R1C7",
"base_oid": "b356dec2ac5bd2a5f721aa73f8fd1902f9c65632",
"production_deployed": false,
"tag_created": false,
"baseline_changed": false,
"deployable": true
}
+21
View File
@@ -0,0 +1,21 @@
bec7da1adb75bf5feaa63c60ec6d03adf15d7508742a486207033d000f72d4ff AGENT_DEPLOYMENT_REQUIREMENTS.md
251b46e51ce09aa2fd843bee48ef24a5608f31d24c599841f086ceb25c1a1715 README.md
783f68ec4f00a11d43ceed536196bfedccaf6614f6db9805f7b36d24e5fa7bc2 TEST_RESULTS.json
9208325e5b2e6f5ba6b64e000bc21d14d9f2f214e3ad33dd79857e85218fd6c2 VERSION
c4b24b65195077716983acdc8611f705324413a74453858994ba51541957afd4 active-runtime/app/aios_authorized_task_writer_api.py
8a1935fa098d47ac02ce51bbd9fd5fad7cf9ab9374ea3de5f630679b96681d84 active-runtime/app/authorized_task_writer_provisioning.py
0997ab8332985c7124f334c4e49735d8833326578acc61a61a389bf792424de3 active-runtime/app/deployment_state.py
dd3f8b1cab88e005a291f077807873c381e8ca92fe863dc05817925f0dff9613 active-runtime/app/v2r1c6_privacy_tools.py
9e45025c41068cafdea2f360edfc4146a5459b0a6bcfbf1403206fc0161e7bac active-runtime/app/v2r1c7_health_service.py
c96ba560d69f6c2eb700b9f2b42a32e394dadd00dffb23bbefdb9fb3e1571c0d deploy/aios-v2/systemd/aios-v2r1c7-production.service
a05d4e8038fdb866ce17d4fb8a54798a268a1e8d44ff2cbc559069fa6ab2a667 deploy/systemd/aios-gitops-model-api-authorized-writer-bridge.conf.example
a60dac51802bff67c2cfff61acdeb0a3a05d74a4f6cb47e3f78295ceb5c6acd9 docs/gitops/authorized-task-writer-provisioning.md
2690f76452bceddda90c728278c600bcebd533db24776e2966423c5a8655e4b5 docs/gitops/openapi-authorized-task-writer-fragment.yaml
560e5d9af8a8e2ec60ec31cf8659a7982b93ea09b7adb486b738ea2fba460d70 release-manifests/V2R1C7.json
81a423c7814f81cb888b60a538c153e549b2be9099c881c968d55ae84efddc57 scripts/build_v2r1c7_release_artifact.py
64c78dabffb05c8851f87b2a27c43bd1f21728dca586397d03a6bc22906a906c scripts/probe_v2r1c7_health.py
5419dc7de0f50908ad0ef1dac74172fce811bb7b45f314a6e1c1bb9ce618324d scripts/rollback_v2r1c7_release.sh
54e2845d2c3c026323ad45515e082d661e3b461feb3bfefedf5adc52db2d7b78 scripts/smoke_authorized_task_writer_provisioning.py
32a860e28d5afc105f413d878a13fb8600fa74874dbd3bc4c031ef3efa17896a scripts/verify_v2r1c7_release.py
3cce2b744326fe12b89bcad3cd352c77738950964bf8519e72cb5e9c7a5dfbfc tests/test_authorized_task_writer_provisioning.py
8f3153c625241f60b65109eeac045e8aed7661d95f3a10f0b32ba3e0e9ac0a85 versions/V2R1C7.json
+25
View File
@@ -0,0 +1,25 @@
{
"git_branch_created": false,
"git_commit_created": false,
"production_mutation": false,
"py_compile": {
"files": [
"active-runtime/app/authorized_task_writer_provisioning.py",
"active-runtime/app/aios_authorized_task_writer_api.py"
],
"status": "PASS"
},
"schema": "aios.authorized-task-writer-bridge-test-report.v1",
"secrets_included": false,
"smoke": {
"command": "python3 scripts/smoke_authorized_task_writer_provisioning.py",
"status": "PASS"
},
"status": "PASS",
"unit_tests": {
"command": "python3 -m unittest -v tests/test_authorized_task_writer_provisioning.py",
"errors": 0,
"failures": 0,
"total": 9
}
}
+1 -1
View File
@@ -1 +1 @@
V2R1C4
V2R1C7
+201
View File
@@ -0,0 +1,201 @@
"""Fail-closed Agent Pull Runner primitives for AIOS autonomy v1.
The runner owns task discovery and reporting only. It never accepts shell
strings: executors receive argv, an allowlisted cwd and an allowlisted env.
Durable state is intentionally small so restart recovery and duplicate
suppression can be verified without coupling the runner to one database.
"""
from __future__ import annotations
import json
import os
import random
import shutil
import subprocess
import tempfile
import time
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Callable, Mapping, Protocol, Sequence
class RunnerError(RuntimeError):
"""Raised when a runner contract would be violated."""
@dataclass(frozen=True)
class RunnerConfig:
worker_id: str
capabilities: tuple[str, ...] = ()
long_poll_seconds: int = 30
fallback_seconds: int = 60
jitter_seconds: int = 5
max_backoff_seconds: int = 300
executable_allowlist: tuple[str, ...] = ()
working_directory_allowlist: tuple[str, ...] = ()
environment_allowlist: tuple[str, ...] = ()
@dataclass
class RunnerState:
inflight: dict[str, dict[str, Any]] = field(default_factory=dict)
terminal: dict[str, dict[str, Any]] = field(default_factory=dict)
class RunnerTransport(Protocol):
def register_worker(self, worker_id: str, capabilities: Sequence[str]) -> Mapping[str, Any]: ...
def heartbeat(self, worker_id: str) -> Mapping[str, Any]: ...
def long_poll(self, worker_id: str, timeout_seconds: int) -> Mapping[str, Any] | None: ...
def poll(self, worker_id: str) -> Mapping[str, Any] | None: ...
def claim(self, worker_id: str, task: Mapping[str, Any]) -> Mapping[str, Any]: ...
def renew(self, worker_id: str, task_id: str) -> Mapping[str, Any]: ...
def report_progress(self, worker_id: str, task_id: str, payload: Mapping[str, Any]) -> Mapping[str, Any]: ...
def report_result(self, worker_id: str, task_id: str, payload: Mapping[str, Any]) -> Mapping[str, Any]: ...
def report_evidence(self, worker_id: str, task_id: str, payload: Mapping[str, Any]) -> Mapping[str, Any]: ...
class JsonStateStore:
"""Atomic JSON state store for restart recovery."""
def __init__(self, path: str | os.PathLike[str]) -> None:
self.path = Path(path)
def load(self) -> RunnerState:
if not self.path.exists():
return RunnerState()
value = json.loads(self.path.read_text(encoding="utf-8"))
if not isinstance(value, dict):
raise RunnerError("runner state must be an object")
return RunnerState(
inflight=dict(value.get("inflight", {})),
terminal=dict(value.get("terminal", {})),
)
def save(self, state: RunnerState) -> None:
self.path.parent.mkdir(parents=True, exist_ok=True)
fd, name = tempfile.mkstemp(prefix=self.path.name + ".", dir=self.path.parent)
try:
with os.fdopen(fd, "w", encoding="utf-8") as handle:
json.dump({"inflight": state.inflight, "terminal": state.terminal}, handle, sort_keys=True)
handle.write("\n")
os.chmod(name, 0o600)
os.replace(name, self.path)
finally:
if os.path.exists(name):
os.unlink(name)
class AgentPullRunner:
"""Pull, claim, execute and report one task at a time."""
def __init__(
self,
config: RunnerConfig,
transport: RunnerTransport,
state_store: JsonStateStore | None = None,
*,
random_fn: Callable[[], float] = random.random,
) -> None:
if not config.worker_id.strip():
raise RunnerError("worker_id is required")
self.config = config
self.transport = transport
self.state_store = state_store
self.random_fn = random_fn
self.state = state_store.load() if state_store else RunnerState()
def _save(self) -> None:
if self.state_store:
self.state_store.save(self.state)
def register(self) -> Mapping[str, Any]:
return self.transport.register_worker(self.config.worker_id, self.config.capabilities)
def heartbeat(self) -> Mapping[str, Any]:
return self.transport.heartbeat(self.config.worker_id)
def discovery_delay(self, failures: int) -> float:
failures = max(0, int(failures))
base = min(self.config.max_backoff_seconds, self.config.fallback_seconds * (2**failures))
jitter = self.random_fn() * max(0, self.config.jitter_seconds)
return min(self.config.max_backoff_seconds, base + jitter)
def discover_once(self) -> Mapping[str, Any] | None:
try:
task = self.transport.long_poll(self.config.worker_id, self.config.long_poll_seconds)
except Exception:
task = None
if task is None:
task = self.transport.poll(self.config.worker_id)
if not task:
return None
task_id = str(task.get("task_id", ""))
if not task_id:
raise RunnerError("task_id is required")
if task_id in self.state.terminal:
return {"status": "duplicate_suppressed", "task_id": task_id}
claimed = dict(self.transport.claim(self.config.worker_id, task))
self.state.inflight[task_id] = claimed
self._save()
return claimed
def renew(self, task_id: str) -> Mapping[str, Any]:
if task_id not in self.state.inflight:
raise RunnerError("task is not inflight")
return self.transport.renew(self.config.worker_id, task_id)
def report_progress(self, task_id: str, payload: Mapping[str, Any]) -> Mapping[str, Any]:
if task_id not in self.state.inflight:
raise RunnerError("task is not inflight")
return self.transport.report_progress(self.config.worker_id, task_id, payload)
def report_evidence(self, task_id: str, payload: Mapping[str, Any]) -> Mapping[str, Any]:
if task_id not in self.state.inflight:
raise RunnerError("task is not inflight")
return self.transport.report_evidence(self.config.worker_id, task_id, payload)
def report_result(self, task_id: str, payload: Mapping[str, Any]) -> Mapping[str, Any]:
if task_id in self.state.terminal:
return {"status": "duplicate_terminal_suppressed", "task_id": task_id}
if task_id not in self.state.inflight:
raise RunnerError("task is not inflight")
result = dict(self.transport.report_result(self.config.worker_id, task_id, payload))
self.state.terminal[task_id] = {"result": result, "reported_at": time.time()}
self.state.inflight.pop(task_id, None)
self._save()
return result
def recover_after_restart(self) -> list[Mapping[str, Any]]:
recovered: list[Mapping[str, Any]] = []
for task_id in tuple(self.state.inflight):
recovered.append(self.renew(task_id))
return recovered
def validate_execution(self, argv: Sequence[str], cwd: str | os.PathLike[str], env: Mapping[str, str]) -> None:
if not argv or any(not isinstance(item, str) or not item for item in argv):
raise RunnerError("argv_only_execution_required")
if any(ch in argv[0] for ch in (";", "|", "&", "`", "\n")):
raise RunnerError("shell_syntax_denied")
executable = shutil.which(argv[0])
if not executable:
raise RunnerError("executable_not_found")
if self.config.executable_allowlist and not any(
os.path.realpath(executable).startswith(os.path.realpath(prefix).rstrip(os.sep) + os.sep)
or os.path.realpath(executable) == os.path.realpath(prefix)
for prefix in self.config.executable_allowlist
):
raise RunnerError("executable_not_allowlisted")
real_cwd = os.path.realpath(os.fspath(cwd))
if self.config.working_directory_allowlist and not any(
real_cwd == os.path.realpath(prefix)
or real_cwd.startswith(os.path.realpath(prefix).rstrip(os.sep) + os.sep)
for prefix in self.config.working_directory_allowlist
):
raise RunnerError("working_directory_not_allowlisted")
if self.config.environment_allowlist and any(key not in self.config.environment_allowlist for key in env):
raise RunnerError("environment_key_not_allowlisted")
def execute_argv(self, argv: Sequence[str], cwd: str | os.PathLike[str], env: Mapping[str, str]) -> Mapping[str, Any]:
self.validate_execution(argv, cwd, env)
completed = subprocess.run(list(argv), cwd=os.fspath(cwd), env=dict(env), shell=False, text=True, capture_output=True, check=False)
return {"returncode": completed.returncode, "stdout": completed.stdout, "stderr": completed.stderr}
@@ -0,0 +1,36 @@
"""Small operational CLI for the Agent Pull Runner contract."""
from __future__ import annotations
import argparse
import json
from agent_pull_runner import AgentPullRunner, RunnerConfig
class _NoopTransport:
def register_worker(self, worker_id, capabilities): return {"status": "registered", "worker_id": worker_id, "capabilities": list(capabilities)}
def heartbeat(self, worker_id): return {"status": "alive", "worker_id": worker_id}
def long_poll(self, worker_id, timeout_seconds): return None
def poll(self, worker_id): return None
def claim(self, worker_id, task): return task
def renew(self, worker_id, task_id): return {"status": "renewed", "task_id": task_id}
def report_progress(self, worker_id, task_id, payload): return {"status": "progress_recorded", "task_id": task_id}
def report_result(self, worker_id, task_id, payload): return {"status": "result_recorded", "task_id": task_id}
def report_evidence(self, worker_id, task_id, payload): return {"status": "evidence_recorded", "task_id": task_id}
def main() -> int:
parser = argparse.ArgumentParser(description="AIOS Agent Pull Runner")
parser.add_argument("--worker-id", default="self-test-worker")
parser.add_argument("--self-test", action="store_true")
args = parser.parse_args()
runner = AgentPullRunner(RunnerConfig(worker_id=args.worker_id, fallback_seconds=60), _NoopTransport())
if args.self_test:
print(json.dumps({"status": "PASS", "registration": runner.register(), "fallback_seconds": runner.config.fallback_seconds}, sort_keys=True))
return 0
print(json.dumps({"status": "READY", "worker_id": args.worker_id}, sort_keys=True))
return 0
if __name__ == "__main__":
raise SystemExit(main())
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,811 @@
"""Canonical Task projection intent and same-identity recovery primitives.
This module is intentionally Task-Plane-owned. It does not open a network
listener and it does not authenticate callers; existing control-plane services
must enforce their normal service-side ``agent-task:control`` boundary before
calling it.
The recovery primitive has one purpose: converge a durable, complete
ProjectionIntent to exactly one canonical Task row. It never fabricates task
semantics from a Bridge binding and it never generates a replacement task id.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass
from datetime import datetime, timezone
from enum import Enum
import hashlib
import json
from pathlib import Path
import re
import sqlite3
from typing import Any, Mapping, Sequence
TASK_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{2,179}$")
JOB_ID_RE = TASK_ID_RE
SHA40_RE = re.compile(r"^[0-9a-f]{40}$")
SHA64_RE = re.compile(r"^[0-9a-f]{64}$")
BRANCH_RE = re.compile(r"^agent/[A-Za-z0-9._/-]{1,220}$")
SCHEMA = "aios.canonical-task-projection-intent.v1"
RECEIPT_SCHEMA = "aios.canonical-task-projection-receipt.v1"
class ProjectionRecoveryError(RuntimeError):
def __init__(self, code: str, status: int = 409) -> None:
super().__init__(code)
self.code = code
self.status = status
class ProjectionState(str, Enum):
PENDING = "PENDING"
CONVERGED = "CONVERGED"
DRIFT = "DRIFT"
FAILED = "FAILED"
BLOCKED = "BLOCKED"
UNKNOWN = "UNKNOWN"
BACKFILL_REQUIRED = "BACKFILL_REQUIRED"
def _now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _canonical(value: object) -> bytes:
return json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
def sha256_json(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _string(value: object, code: str, pattern: re.Pattern[str] | None = None) -> str:
if not isinstance(value, str) or not value.strip():
raise ProjectionRecoveryError(code, 400)
result = value.strip()
if pattern is not None and pattern.fullmatch(result) is None:
raise ProjectionRecoveryError(code, 400)
return result
def _integer(value: object, code: str, *, minimum: int = 0) -> int:
if isinstance(value, bool) or not isinstance(value, int) or value < minimum:
raise ProjectionRecoveryError(code, 400)
return value
def _plain_mapping(value: object, code: str) -> dict[str, Any]:
if not isinstance(value, Mapping):
raise ProjectionRecoveryError(code, 400)
return {str(k): v for k, v in value.items()}
def _plain_list(value: object, code: str) -> list[Any]:
if not isinstance(value, list):
raise ProjectionRecoveryError(code, 400)
return list(value)
@dataclass(frozen=True, slots=True)
class CanonicalTaskPayload:
task_id: str
goal: str
instructions: str
success_criteria: tuple[Mapping[str, Any], ...]
constraints: Mapping[str, Any]
required_capabilities: tuple[str, ...]
metadata: Mapping[str, Any]
priority: int = 100
max_attempts: int = 20
deadline: str | None = None
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "CanonicalTaskPayload":
allowed = {
"task_id",
"goal",
"instructions",
"success_criteria",
"constraints",
"required_capabilities",
"metadata",
"priority",
"max_attempts",
"deadline",
}
if set(value) - allowed:
raise ProjectionRecoveryError("TASK_PAYLOAD_UNKNOWN_FIELDS", 400)
task_id = _string(value.get("task_id"), "TASK_ID_INVALID", TASK_ID_RE)
goal = _string(value.get("goal"), "TASK_GOAL_INVALID")
instructions = _string(value.get("instructions"), "TASK_INSTRUCTIONS_INVALID")
if len(goal) > 8192 or len(instructions) > 65536:
raise ProjectionRecoveryError("TASK_TEXT_TOO_LARGE", 413)
criteria_raw = _plain_list(value.get("success_criteria"), "TASK_SUCCESS_CRITERIA_INVALID")
criteria: list[Mapping[str, Any]] = []
for item in criteria_raw:
if not isinstance(item, Mapping):
raise ProjectionRecoveryError("TASK_SUCCESS_CRITERIA_INVALID", 400)
criteria.append(dict(item))
if not criteria:
raise ProjectionRecoveryError("TASK_SUCCESS_CRITERIA_REQUIRED", 400)
constraints = _plain_mapping(value.get("constraints"), "TASK_CONSTRAINTS_INVALID")
capabilities_raw = _plain_list(
value.get("required_capabilities", []),
"TASK_CAPABILITIES_INVALID",
)
capabilities: list[str] = []
for item in capabilities_raw:
capabilities.append(_string(item, "TASK_CAPABILITY_INVALID"))
# Recovery recreates the pristine canonical row. Initial execution
# projection remains the sole owner of runtime capabilities.
if capabilities:
raise ProjectionRecoveryError(
"TASK_RECOVERY_REQUIRES_PRISTINE_CAPABILITIES",
409,
)
metadata = _plain_mapping(value.get("metadata", {}), "TASK_METADATA_INVALID")
if "execution" in metadata or "execution_projection" in metadata:
raise ProjectionRecoveryError(
"TASK_RECOVERY_REQUIRES_PRISTINE_METADATA",
409,
)
priority = _integer(value.get("priority", 100), "TASK_PRIORITY_INVALID", minimum=0)
max_attempts = _integer(
value.get("max_attempts", 20),
"TASK_MAX_ATTEMPTS_INVALID",
minimum=1,
)
deadline = value.get("deadline")
if deadline is not None:
deadline = _string(deadline, "TASK_DEADLINE_INVALID")
return cls(
task_id=task_id,
goal=goal,
instructions=instructions,
success_criteria=tuple(criteria),
constraints=constraints,
required_capabilities=tuple(capabilities),
metadata=metadata,
priority=priority,
max_attempts=max_attempts,
deadline=deadline,
)
def public(self) -> dict[str, Any]:
return {
"task_id": self.task_id,
"goal": self.goal,
"instructions": self.instructions,
"success_criteria": [dict(x) for x in self.success_criteria],
"constraints": dict(self.constraints),
"required_capabilities": list(self.required_capabilities),
"metadata": dict(self.metadata),
"priority": self.priority,
"max_attempts": self.max_attempts,
"deadline": self.deadline,
}
@property
def payload_sha256(self) -> str:
return sha256_json(self.public())
@dataclass(frozen=True, slots=True)
class GovernedProjectionBinding:
governed_task_id: str
bridge_job_id: str
repository: str
branch: str
generation: int
base_oid: str
expected_branch_oid: str
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "GovernedProjectionBinding":
allowed = {
"governed_task_id",
"bridge_job_id",
"repository",
"branch",
"generation",
"base_oid",
"expected_branch_oid",
}
if set(value) != allowed:
raise ProjectionRecoveryError("GOVERNED_BINDING_FIELDS_INVALID", 400)
repository = _string(value["repository"], "REPOSITORY_INVALID")
if repository != "aios/aios":
raise ProjectionRecoveryError("REPOSITORY_MISMATCH", 409)
return cls(
governed_task_id=_string(
value["governed_task_id"],
"GOVERNED_TASK_ID_INVALID",
TASK_ID_RE,
),
bridge_job_id=_string(
value["bridge_job_id"],
"BRIDGE_JOB_ID_INVALID",
JOB_ID_RE,
),
repository=repository,
branch=_string(value["branch"], "BRANCH_INVALID", BRANCH_RE),
generation=_integer(value["generation"], "GENERATION_INVALID", minimum=1),
base_oid=_string(value["base_oid"], "BASE_OID_INVALID", SHA40_RE),
expected_branch_oid=_string(
value["expected_branch_oid"],
"EXPECTED_OID_INVALID",
SHA40_RE,
),
)
def public(self) -> dict[str, Any]:
return asdict(self)
@dataclass(frozen=True, slots=True)
class ProjectionSourceArtifact:
ref: str
sha256: str
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "ProjectionSourceArtifact":
if set(value) != {"ref", "sha256"}:
raise ProjectionRecoveryError("SOURCE_ARTIFACT_FIELDS_INVALID", 400)
return cls(
ref=_string(value["ref"], "SOURCE_ARTIFACT_REF_INVALID"),
sha256=_string(value["sha256"], "SOURCE_ARTIFACT_SHA_INVALID", SHA64_RE),
)
@dataclass(frozen=True, slots=True)
class CanonicalTaskProjectionIntent:
projection_id: str
projection_generation: int
idempotency_key: str
task: CanonicalTaskPayload
binding: GovernedProjectionBinding
source_artifact: ProjectionSourceArtifact
schema: str = SCHEMA
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "CanonicalTaskProjectionIntent":
allowed = {
"schema",
"projection_id",
"projection_generation",
"idempotency_key",
"task",
"governed_binding",
"source_artifact",
"task_payload_sha256",
}
if set(value) != allowed:
raise ProjectionRecoveryError("PROJECTION_INTENT_FIELDS_INVALID", 400)
if value["schema"] != SCHEMA:
raise ProjectionRecoveryError("PROJECTION_INTENT_SCHEMA_INVALID", 400)
task = CanonicalTaskPayload.from_mapping(
_plain_mapping(value["task"], "TASK_PAYLOAD_INVALID")
)
binding = GovernedProjectionBinding.from_mapping(
_plain_mapping(value["governed_binding"], "GOVERNED_BINDING_INVALID")
)
source = ProjectionSourceArtifact.from_mapping(
_plain_mapping(value["source_artifact"], "SOURCE_ARTIFACT_INVALID")
)
if binding.governed_task_id != task.task_id:
raise ProjectionRecoveryError("PROJECTION_TASK_IDENTITY_MISMATCH", 409)
expected_hash = _string(
value["task_payload_sha256"],
"TASK_PAYLOAD_SHA_INVALID",
SHA64_RE,
)
if task.payload_sha256 != expected_hash:
raise ProjectionRecoveryError("TASK_PAYLOAD_SHA_MISMATCH", 409)
projection_id = _string(
value["projection_id"],
"PROJECTION_ID_INVALID",
TASK_ID_RE,
)
idempotency_key = _string(
value["idempotency_key"],
"IDEMPOTENCY_KEY_INVALID",
TASK_ID_RE,
)
generation = _integer(
value["projection_generation"],
"PROJECTION_GENERATION_INVALID",
minimum=1,
)
return cls(
projection_id=projection_id,
projection_generation=generation,
idempotency_key=idempotency_key,
task=task,
binding=binding,
source_artifact=source,
)
def public(self) -> dict[str, Any]:
return {
"schema": self.schema,
"projection_id": self.projection_id,
"projection_generation": self.projection_generation,
"idempotency_key": self.idempotency_key,
"task": self.task.public(),
"governed_binding": self.binding.public(),
"source_artifact": asdict(self.source_artifact),
"task_payload_sha256": self.task.payload_sha256,
}
@property
def request_sha256(self) -> str:
return sha256_json(self.public())
class CanonicalTaskProjectionStore:
"""Task-Plane-owned intent/recovery store.
The caller must already be inside the Task Plane authorization boundary.
"""
def __init__(self, db_path: Path | str) -> None:
self.db_path = Path(db_path)
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.db_path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@staticmethod
def _columns(connection: sqlite3.Connection, table: str) -> dict[str, sqlite3.Row]:
return {
str(row["name"]): row
for row in connection.execute(f'PRAGMA table_info("{table}")')
}
@staticmethod
def _table_names(connection: sqlite3.Connection) -> set[str]:
return {
str(row[0])
for row in connection.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
)
}
@classmethod
def _event_table(
cls,
connection: sqlite3.Connection,
) -> tuple[str, dict[str, sqlite3.Row]]:
tables = cls._table_names(connection)
for name in ("task_events", "events"):
if name in tables:
columns = cls._columns(connection, name)
if "task_id" in columns and "event_type" in columns:
return name, columns
raise ProjectionRecoveryError("TASK_EVENT_TABLE_UNSUPPORTED", 503)
@staticmethod
def _ensure_control_tables(connection: sqlite3.Connection) -> None:
connection.execute(
"""
CREATE TABLE IF NOT EXISTS task_projection_intents (
projection_id TEXT PRIMARY KEY,
task_id TEXT NOT NULL,
bridge_job_id TEXT NOT NULL,
projection_generation INTEGER NOT NULL,
idempotency_key TEXT NOT NULL,
request_sha256 TEXT NOT NULL,
task_payload_sha256 TEXT NOT NULL,
intent_json TEXT NOT NULL,
state TEXT NOT NULL,
last_error TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
UNIQUE(task_id, projection_generation),
UNIQUE(task_id, idempotency_key)
)
"""
)
connection.execute(
"""
CREATE TABLE IF NOT EXISTS task_projection_receipts (
projection_id TEXT PRIMARY KEY,
task_id TEXT NOT NULL,
bridge_job_id TEXT NOT NULL,
projection_generation INTEGER NOT NULL,
task_payload_sha256 TEXT NOT NULL,
canonical_revision INTEGER NOT NULL,
state TEXT NOT NULL,
created_new_row INTEGER NOT NULL,
observed_at TEXT NOT NULL,
receipt_json TEXT NOT NULL
)
"""
)
def persist_intent(
self,
intent: CanonicalTaskProjectionIntent,
) -> dict[str, Any]:
now = _now()
request_sha = intent.request_sha256
intent_text = _canonical(intent.public()).decode("utf-8")
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
self._ensure_control_tables(connection)
existing = connection.execute(
"""
SELECT * FROM task_projection_intents
WHERE task_id=? AND projection_generation=?
""",
(intent.task.task_id, intent.projection_generation),
).fetchone()
if existing is not None:
if (
str(existing["projection_id"]) != intent.projection_id
or str(existing["idempotency_key"]) != intent.idempotency_key
or str(existing["request_sha256"]) != request_sha
):
raise ProjectionRecoveryError(
"PROJECTION_INTENT_IDEMPOTENCY_CONFLICT",
409,
)
connection.commit()
return {
"projection_id": intent.projection_id,
"task_id": intent.task.task_id,
"state": str(existing["state"]),
"idempotent": True,
}
connection.execute(
"""
INSERT INTO task_projection_intents(
projection_id,task_id,bridge_job_id,projection_generation,
idempotency_key,request_sha256,task_payload_sha256,intent_json,
state,last_error,created_at,updated_at
) VALUES (?,?,?,?,?,?,?,?,?,?,?,?)
""",
(
intent.projection_id,
intent.task.task_id,
intent.binding.bridge_job_id,
intent.projection_generation,
intent.idempotency_key,
request_sha,
intent.task.payload_sha256,
intent_text,
ProjectionState.PENDING.value,
None,
now,
now,
),
)
connection.commit()
return {
"projection_id": intent.projection_id,
"task_id": intent.task.task_id,
"state": ProjectionState.PENDING.value,
"idempotent": False,
}
def _task_row_payload(
self,
row: sqlite3.Row,
) -> dict[str, Any]:
try:
success = json.loads(row["success_criteria_json"])
constraints = json.loads(row["constraints_json"])
capabilities = json.loads(row["required_capabilities_json"])
metadata = json.loads(row["metadata_json"])
except (TypeError, json.JSONDecodeError) as exc:
raise ProjectionRecoveryError("CANONICAL_TASK_JSON_INVALID", 409) from exc
if (
not isinstance(success, list)
or not isinstance(constraints, Mapping)
or not isinstance(capabilities, list)
or not isinstance(metadata, Mapping)
):
raise ProjectionRecoveryError("CANONICAL_TASK_SHAPE_INVALID", 409)
return {
"task_id": str(row["task_id"]),
"goal": str(row["goal"]),
"instructions": str(row["instructions"]),
"success_criteria": success,
"constraints": dict(constraints),
"required_capabilities": capabilities,
"metadata": dict(metadata),
"priority": int(row["priority"]),
"max_attempts": int(row["max_attempts"]),
"deadline": row["deadline"],
}
def _verify_existing_task(
self,
row: sqlite3.Row,
intent: CanonicalTaskProjectionIntent,
) -> int:
observed = self._task_row_payload(row)
observed_hash = sha256_json(observed)
if observed_hash != intent.task.payload_sha256:
raise ProjectionRecoveryError(
"CANONICAL_TASK_PROJECTION_IDENTITY_CONFLICT",
409,
)
return int(row["revision"])
@staticmethod
def _required_task_columns() -> set[str]:
return {
"task_id",
"goal",
"instructions",
"success_criteria_json",
"required_capabilities_json",
"constraints_json",
"metadata_json",
"priority",
"max_attempts",
"attempt",
"status",
"claimed_by",
"lease_token_hash",
"lease_expires_at",
"result_json",
"verification_json",
"revision",
"created_at",
"updated_at",
}
def _insert_task(
self,
connection: sqlite3.Connection,
intent: CanonicalTaskProjectionIntent,
) -> int:
columns = self._columns(connection, "tasks")
if not self._required_task_columns().issubset(columns):
raise ProjectionRecoveryError("TASK_SCHEMA_RECOVERY_UNSUPPORTED", 503)
now = _now()
values: dict[str, Any] = {
"task_id": intent.task.task_id,
"goal": intent.task.goal,
"instructions": intent.task.instructions,
"success_criteria_json": _canonical(
[dict(x) for x in intent.task.success_criteria]
).decode("utf-8"),
"required_capabilities_json": "[]",
"constraints_json": _canonical(dict(intent.task.constraints)).decode("utf-8"),
"metadata_json": _canonical(dict(intent.task.metadata)).decode("utf-8"),
"deadline": intent.task.deadline,
"priority": intent.task.priority,
"max_attempts": intent.task.max_attempts,
"attempt": 0,
"status": "QUEUED",
"claimed_by": None,
"lease_token_hash": None,
"lease_expires_at": None,
"result_json": None,
"verification_json": None,
"revision": 1,
"created_at": now,
"updated_at": now,
}
for optional in (
"result_digest",
"result_idempotency_key",
):
if optional in columns:
values[optional] = None
names = [name for name in values if name in columns]
placeholders = ",".join("?" for _ in names)
connection.execute(
f'INSERT INTO tasks ({",".join(chr(34)+n+chr(34) for n in names)}) '
f"VALUES ({placeholders})",
tuple(values[name] for name in names),
)
self._insert_task_created_event(connection, intent)
return 1
def _insert_task_created_event(
self,
connection: sqlite3.Connection,
intent: CanonicalTaskProjectionIntent,
) -> None:
table, columns = self._event_table(connection)
event = {
"schema": "aios.task-projection-recovery.event.v1",
"task_id": intent.task.task_id,
"bridge_job_id": intent.binding.bridge_job_id,
"projection_id": intent.projection_id,
"projection_generation": intent.projection_generation,
"task_payload_sha256": intent.task.payload_sha256,
"source_artifact_ref": intent.source_artifact.ref,
"source_artifact_sha256": intent.source_artifact.sha256,
"recovery": True,
}
values: dict[str, Any] = {
"task_id": intent.task.task_id,
"event_type": "TASK_CREATED",
}
payload_text = _canonical(event).decode("utf-8")
if "payload_json" in columns:
values["payload_json"] = payload_text
elif "payload" in columns:
values["payload"] = payload_text
else:
raise ProjectionRecoveryError("TASK_EVENT_PAYLOAD_UNSUPPORTED", 503)
if "created_at" in columns:
values["created_at"] = _now()
if "agent_id" in columns:
values["agent_id"] = "control-plane-projection-recovery"
if "actor_id" in columns:
values["actor_id"] = "control-plane-projection-recovery"
names = list(values)
connection.execute(
f'INSERT INTO "{table}" ({",".join(chr(34)+n+chr(34) for n in names)}) '
f'VALUES ({",".join("?" for _ in names)})',
tuple(values[name] for name in names),
)
def reconcile(
self,
intent: CanonicalTaskProjectionIntent,
) -> dict[str, Any]:
# Persist first so a crash before Task creation leaves a durable PENDING
# intent for startup/supervisor reconciliation.
self.persist_intent(intent)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
self._ensure_control_tables(connection)
existing_receipt = connection.execute(
"SELECT * FROM task_projection_receipts WHERE projection_id=?",
(intent.projection_id,),
).fetchone()
if existing_receipt is not None:
if (
str(existing_receipt["task_id"]) != intent.task.task_id
or str(existing_receipt["bridge_job_id"]) != intent.binding.bridge_job_id
or str(existing_receipt["task_payload_sha256"])
!= intent.task.payload_sha256
):
raise ProjectionRecoveryError(
"PROJECTION_RECEIPT_IDENTITY_CONFLICT",
409,
)
connection.commit()
receipt = json.loads(existing_receipt["receipt_json"])
receipt["idempotent"] = True
return receipt
row = connection.execute(
"SELECT * FROM tasks WHERE task_id=?",
(intent.task.task_id,),
).fetchone()
created_new = False
if row is None:
revision = self._insert_task(connection, intent)
created_new = True
row = connection.execute(
"SELECT * FROM tasks WHERE task_id=?",
(intent.task.task_id,),
).fetchone()
if row is None:
raise ProjectionRecoveryError(
"TASK_RECOVERY_OBSERVATION_FAILED",
503,
)
else:
revision = self._verify_existing_task(row, intent)
# Observation after mutation is mandatory.
revision = self._verify_existing_task(row, intent)
observed_at = _now()
receipt = {
"schema": RECEIPT_SCHEMA,
"projection_id": intent.projection_id,
"governed_task_id": intent.binding.governed_task_id,
"canonical_task_id": intent.task.task_id,
"bridge_job_id": intent.binding.bridge_job_id,
"projection_generation": intent.projection_generation,
"task_payload_sha256": intent.task.payload_sha256,
"canonical_revision": revision,
"state": ProjectionState.CONVERGED.value,
"created_new_row": created_new,
"source_artifact": asdict(intent.source_artifact),
"observed_at": observed_at,
"idempotent": False,
}
connection.execute(
"""
INSERT INTO task_projection_receipts(
projection_id,task_id,bridge_job_id,projection_generation,
task_payload_sha256,canonical_revision,state,created_new_row,
observed_at,receipt_json
) VALUES (?,?,?,?,?,?,?,?,?,?)
""",
(
intent.projection_id,
intent.task.task_id,
intent.binding.bridge_job_id,
intent.projection_generation,
intent.task.payload_sha256,
revision,
ProjectionState.CONVERGED.value,
1 if created_new else 0,
observed_at,
_canonical(receipt).decode("utf-8"),
),
)
connection.execute(
"""
UPDATE task_projection_intents
SET state=?, last_error=NULL, updated_at=?
WHERE projection_id=?
""",
(
ProjectionState.CONVERGED.value,
observed_at,
intent.projection_id,
),
)
connection.commit()
return receipt
def get_projection(
self,
task_id: str,
) -> dict[str, Any] | None:
task_id = _string(task_id, "TASK_ID_INVALID", TASK_ID_RE)
with self._connect() as connection:
self._ensure_control_tables(connection)
intent = connection.execute(
"""
SELECT * FROM task_projection_intents
WHERE task_id=?
ORDER BY projection_generation DESC
LIMIT 1
""",
(task_id,),
).fetchone()
if intent is None:
return None
receipt = connection.execute(
"SELECT receipt_json FROM task_projection_receipts WHERE projection_id=?",
(intent["projection_id"],),
).fetchone()
return {
"projection_id": intent["projection_id"],
"task_id": task_id,
"bridge_job_id": intent["bridge_job_id"],
"projection_generation": int(intent["projection_generation"]),
"state": intent["state"],
"last_error": intent["last_error"],
"task_payload_sha256": intent["task_payload_sha256"],
"receipt": json.loads(receipt[0]) if receipt else None,
}
@@ -0,0 +1,44 @@
"""HTTP integration helpers for canonical Task projection recovery.
This module intentionally contains no HTTP server. Existing Task Plane /
projection-owner handlers should call ``recover_from_json`` only after their
normal service-side ``agent-task:control`` authorization succeeds.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any, Mapping
from aios_agent_task_projection_recovery import (
CanonicalTaskProjectionIntent,
CanonicalTaskProjectionStore,
ProjectionRecoveryError,
)
def recover_from_json(
db_path: Path | str,
task_id: str,
body: Mapping[str, Any],
) -> tuple[int, dict[str, Any]]:
try:
intent = CanonicalTaskProjectionIntent.from_mapping(body)
if intent.task.task_id != task_id:
raise ProjectionRecoveryError("PROJECTION_TASK_IDENTITY_MISMATCH", 409)
result = CanonicalTaskProjectionStore(db_path).reconcile(intent)
return (200 if result.get("idempotent") else 201), result
except ProjectionRecoveryError as exc:
return exc.status, {"error": exc.code}
def get_projection(
db_path: Path | str,
task_id: str,
) -> tuple[int, dict[str, Any]]:
try:
result = CanonicalTaskProjectionStore(db_path).get_projection(task_id)
except ProjectionRecoveryError as exc:
return exc.status, {"error": exc.code}
if result is None:
return 404, {"error": "TASK_PROJECTION_NOT_FOUND"}
return 200, result
@@ -0,0 +1,192 @@
"""Fail-closed Authority Git side-effect boundary.
Only this module may issue the narrowly scoped ref CAS primitives used by the
durable executor. It never accepts force updates, arbitrary refs, tags, or
direct production mutations.
"""
from __future__ import annotations
from dataclasses import dataclass
import os
from pathlib import Path
import re
import subprocess
import tempfile
from typing import Mapping, Sequence
OID_RE = re.compile(r"^[0-9a-f]{40}$")
TASK_BRANCH_RE = re.compile(r"^refs/heads/agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$")
class AuthorityGitError(RuntimeError):
pass
def _oid(value: str, field: str) -> str:
if not isinstance(value, str) or not OID_RE.fullmatch(value):
raise AuthorityGitError(f"{field}_INVALID")
return value
@dataclass(frozen=True, slots=True)
class AuthorityGitExecutor:
git_dir: Path
worktree_root: Path = Path("/var/lib/aios-gitops/worktrees")
def __post_init__(self) -> None:
if not self.git_dir.is_absolute() or self.git_dir.is_symlink():
raise AuthorityGitError("AUTHORITY_GIT_DIR_INVALID")
if not self.worktree_root.is_absolute() or self.worktree_root.is_symlink():
raise AuthorityGitError("WORKTREE_ROOT_INVALID")
def _git(self, args: Sequence[str], *, check: bool = True) -> str:
result = subprocess.run(
["git", f"--git-dir={self.git_dir}", *args],
text=True,
capture_output=True,
check=False,
)
if check and result.returncode != 0:
raise AuthorityGitError("AUTHORITY_GIT_COMMAND_FAILED")
return result.stdout.strip()
def read_ref(self, ref: str) -> str | None:
if not ref.startswith("refs/") or ref.startswith(("refs/tags/", "refs/remotes/")):
raise AuthorityGitError("REF_READ_SCOPE_DENIED")
value = self._git(["rev-parse", "--verify", ref], check=False)
return value if value and OID_RE.fullmatch(value) else None
def commit_exists(self, oid: str) -> bool:
_oid(oid, "commit_oid")
result = subprocess.run(["git", f"--git-dir={self.git_dir}", "cat-file", "-e", oid + "^{commit}"], capture_output=True, check=False)
return result.returncode == 0
def ancestry_check(self, older_oid: str, newer_oid: str) -> bool:
_oid(older_oid, "older_oid")
_oid(newer_oid, "newer_oid")
result = subprocess.run(
["git", f"--git-dir={self.git_dir}", "merge-base", "--is-ancestor", older_oid, newer_oid],
capture_output=True,
check=False,
)
if result.returncode not in (0, 1):
raise AuthorityGitError("ANCESTRY_CHECK_FAILED")
return result.returncode == 0
def create_detached_worktree(
self,
base_oid: str,
worktree_name: str,
root: Path | None = None,
) -> Path:
_oid(base_oid, "base_oid")
if not worktree_name or "/" in worktree_name or "\\" in worktree_name or worktree_name in {".", ".."}:
raise AuthorityGitError("WORKTREE_NAME_INVALID")
parent = root or self.worktree_root
if not parent.is_absolute() or parent.is_symlink():
raise AuthorityGitError("WORKTREE_ROOT_INVALID")
path = parent / worktree_name
if path.exists() or path.is_symlink():
raise AuthorityGitError("WORKTREE_ALREADY_EXISTS")
parent.mkdir(parents=True, exist_ok=True)
self._git(["worktree", "add", "--detach", str(path), base_oid])
return path
def create_task_branch(self, branch: str, base_oid: str) -> str:
"""Create an isolated agent branch once, using an empty-ref CAS."""
ref = branch if branch.startswith("refs/heads/") else "refs/heads/" + branch
if not TASK_BRANCH_RE.fullmatch(ref):
raise AuthorityGitError("TASK_BRANCH_SCOPE_DENIED")
_oid(base_oid, "base_oid")
if not self.commit_exists(base_oid):
raise AuthorityGitError("BASE_COMMIT_NOT_FOUND")
current = self.read_ref(ref)
if current is not None:
if current == base_oid:
return current
raise AuthorityGitError("AUTHORITY_BRANCH_EXPECTATION_MISMATCH")
self._git(["update-ref", ref, base_oid, "0" * 40])
readback = self.read_ref(ref)
if readback != base_oid:
raise AuthorityGitError("AUTHORITY_CAS_READBACK_MISMATCH")
return readback
def advance_task_branch_CAS(self, branch: str, expected_oid: str, new_oid: str) -> str:
if not TASK_BRANCH_RE.fullmatch(branch):
raise AuthorityGitError("TASK_BRANCH_SCOPE_DENIED")
_oid(expected_oid, "expected_oid")
_oid(new_oid, "new_oid")
current = self.read_ref(branch)
if current != expected_oid:
if current == new_oid:
return new_oid
raise AuthorityGitError("AUTHORITY_BRANCH_EXPECTATION_MISMATCH")
if not self.ancestry_check(expected_oid, new_oid):
raise AuthorityGitError("NEW_OID_NOT_DESCENDANT")
self._git(["update-ref", branch, new_oid, expected_oid])
readback = self.read_ref(branch)
if readback != new_oid:
raise AuthorityGitError("AUTHORITY_CAS_READBACK_MISMATCH")
return readback
def ff_CAS_main_after_verified_protected_merge(self, expected_oid: str, new_oid: str) -> str:
_oid(expected_oid, "expected_oid")
_oid(new_oid, "new_oid")
if not self.ancestry_check(expected_oid, new_oid):
raise AuthorityGitError("MAIN_FF_REQUIRED")
current = self.read_ref("refs/heads/main")
if current != expected_oid:
if current == new_oid:
return new_oid
raise AuthorityGitError("AUTHORITY_MAIN_EXPECTATION_MISMATCH")
self._git(["update-ref", "refs/heads/main", new_oid, expected_oid])
readback = self.read_ref("refs/heads/main")
if readback != new_oid:
raise AuthorityGitError("AUTHORITY_MAIN_CAS_READBACK_MISMATCH")
return readback
def commit_file_in_isolated_worktree(
self,
*,
base_oid: str,
files: Mapping[str, bytes],
message: str,
worktree_name: str,
) -> str:
_oid(base_oid, "base_oid")
if not worktree_name or "/" in worktree_name or "\\" in worktree_name or worktree_name in {".", ".."}:
raise AuthorityGitError("WORKTREE_NAME_INVALID")
root = self.worktree_root / worktree_name
if root.exists() or root.is_symlink():
raise AuthorityGitError("WORKTREE_ALREADY_EXISTS")
self.worktree_root.mkdir(parents=True, exist_ok=True)
self._git(["worktree", "add", "--detach", str(root), base_oid])
try:
for relative, content in files.items():
path = Path(relative)
if path.is_absolute() or ".." in path.parts or not path.parts or path.parts[0] == ".git":
raise AuthorityGitError("FILE_PATH_INVALID")
target = root / path
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(content)
result = subprocess.run(
["git", "-C", str(root), "add", "--", *files.keys()],
text=True,
capture_output=True,
check=False,
)
if result.returncode != 0:
raise AuthorityGitError("WORKTREE_ADD_FAILED")
result = subprocess.run(
["git", "-C", str(root), "-c", "user.name=aios-durable-job", "-c", "user.email=aios-durable-job@localhost", "commit", "--no-verify", "-m", message],
text=True,
capture_output=True,
check=False,
)
if result.returncode != 0:
raise AuthorityGitError("WORKTREE_COMMIT_FAILED")
return subprocess.check_output(["git", "-C", str(root), "rev-parse", "HEAD"], text=True).strip()
finally:
subprocess.run(["git", f"--git-dir={self.git_dir}", "worktree", "remove", "--force", str(root)], capture_output=True, check=False)
@@ -0,0 +1,73 @@
"""HTTP adapter for Owner-authorized task writer provisioning.
Run this module instead of aios_chatgpt_actions_api.py. All existing routes are
delegated unchanged; only POST /api/task-writer/provision-authorized is added.
"""
from __future__ import annotations
import os
import urllib.parse
import aios_chatgpt_actions_api as api
from authorized_task_writer_provisioning import (
AuthorizedTaskWriterProvisioner,
OwnerContext,
ProvisioningError,
)
_original_do_post = api.H.do_POST
_ENDPOINT = "/api/task-writer/provision-authorized"
def _owner_authorization(handler):
scope = os.getenv(
"AIOS_GITOPS_OWNER_PROVISION_SCOPE", "writer:provision"
)
auth = handler.need(scope)
if not auth:
return None
if auth.get("role") != "owner":
handler.sendj({"error": "OWNER_AUTHORIZATION_REQUIRED"}, 403)
return None
identity = str(auth.get("account") or "").strip()
if not identity:
handler.sendj({"error": "OWNER_IDENTITY_REQUIRED"}, 403)
return None
return auth
def _authorized_do_post(self):
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
if path != _ENDPOINT:
return _original_do_post(self)
auth = _owner_authorization(self)
if not auth:
return
data = self.body()
owner_reference = str(data.get("owner_authorization_reference") or "")
try:
result = AuthorizedTaskWriterProvisioner().provision(
data,
OwnerContext(
identity=str(auth["account"]),
role=str(auth["role"]),
authorization_reference=owner_reference,
),
)
return self.sendj(result)
except ProvisioningError as exc:
body = {"error": exc.code}
if exc.details:
body["details"] = exc.details
return self.sendj(body, exc.status)
except Exception:
# Do not expose tool output, paths containing credentials, or traceback data.
return self.sendj({"error": "AUTHORIZED_WRITER_BRIDGE_INTERNAL_ERROR"}, 500)
api.H.do_POST = _authorized_do_post
if __name__ == "__main__":
api.main()
@@ -1,379 +0,0 @@
#!/usr/bin/env python3
"""Task-scoped continuous authorization controller for the AIOS Candidate writer."""
from __future__ import annotations
import base64, fcntl, hashlib, json, os, pwd, grp, re, socket, sqlite3, stat, subprocess, sys, tempfile, time, urllib.error, urllib.parse, urllib.request
from datetime import datetime, timedelta, timezone
from pathlib import Path, PurePosixPath, PureWindowsPath
from typing import Any
TASK='AIOS-AUTONOMOUS-RELEASE-FULL-IMPLEMENTATION-20260726'
AUTH_ID='authz-aios-autonomous-release-20260726'
BRANCH='patch/chatgpt-autonomous-completion-contract-20260726'
BASE='875d41c3101093149414292067fd700dc214d4d3'
REPO='aios/aios'
WRITER_AUDIENCE='chatgpt-autonomous-release-writer'
ACTOR='chatgpt-gitops-bot'
WRITER_INSTANCE='writer-6e32f321-1240-46f8-9693-0b0631d9c77b'
CHILD_OPERATION_MAX_USES=20
ROOT=Path('/data/aios-gitops')
WORKTREE=ROOT/'worktrees/chatgpt-autonomous-completion-contract-20260726'
BARE=ROOT/'repo.git'
TASK_ROOT=ROOT/'runtime/tasks'/TASK
AUTH_DIR=Path('/etc/aios-gitops/task-authorizations')
PARENT=AUTH_DIR/'authz-aios-autonomous-release-20260726.json'
STATE_DIR=Path('/var/lib/aios-autonomous-release-authorization')
STATE=STATE_DIR/'state.json'
LOCK=STATE_DIR/'controller.lock'
AUDIT=TASK_ROOT/'continuous-authorization.audit.jsonl'
SOCKET_DIR=Path('/run/aios-autonomous-release-authorization')
SOCKET_PATH=SOCKET_DIR/'controller.sock'
PROJECTION=Path('/etc/aios-gitops/credentials/chatgpt-autonomous-release-writer')
REGISTRY=Path('/var/lib/aios-chatgpt-autonomous-release-writer/token-registry.sqlite3')
WRITER_LEDGER=Path('/var/lib/aios-chatgpt-autonomous-release-writer/operation-ledger.sqlite3')
ASKPASS=Path('/usr/libexec/aios-autonomous-release-authorization/gitea_askpass.py')
PATH_POLICY=Path('/etc/aios-gitops/policies-chatgpt-autonomous-release/autonomous-release-path-policy.json')
ROUTE_BINDING=Path('/etc/aios-gitops/action-route-bindings/chatgpt-autonomous-release-20260726.json')
OID_RE=re.compile(r'^[0-9a-f]{40}$')
OP_RE=re.compile(r'^[A-Za-z0-9][A-Za-z0-9._:-]{7,127}$')
TERMINAL={'COMPLETED','CANCELLED','FAILED_FINAL','REVOKED','REVOKED_IDLE_TIMEOUT','REVOKED_WALL_CLOCK','RECONCILIATION_REQUIRED'}
sys.path.insert(0,'/usr/libexec/aios-chatgpt-autonomous-release-writer-fullprofile-8ac2f90f')
from isolated_writer_token_registry import TokenRegistry
def now() -> datetime: return datetime.now(timezone.utc)
def stamp() -> str: return now().replace(microsecond=0).isoformat()
def canonical(value: Any) -> bytes: return json.dumps(value,sort_keys=True,separators=(',',':'),ensure_ascii=False).encode('utf-8')
def fail(code: str, status: int=503, **details: Any) -> None: raise ControlError(code,status,details)
class ControlError(RuntimeError):
def __init__(self,code:str,status:int=503,details:dict[str,Any]|None=None): self.code,self.status,self.details=code,status,details or {}; super().__init__(code)
def owned_regular(path:Path, mode:int) -> os.stat_result:
try: info=os.lstat(path)
except FileNotFoundError: fail('authorization_config_missing')
if not stat.S_ISREG(info.st_mode) or stat.S_ISLNK(info.st_mode) or info.st_uid!=0 or stat.S_IMODE(info.st_mode)&~mode: fail('authorization_config_permissions_rejected')
return info
def load_json(path:Path,mode:int=0o600) -> dict[str,Any]:
owned_regular(path,mode)
try: value=json.loads(path.read_text(encoding='utf-8'))
except Exception: fail('authorization_config_json_rejected')
if not isinstance(value,dict): fail('authorization_config_schema_rejected')
return value
def atomic_json(path:Path,value:dict[str,Any],mode:int=0o600) -> None:
path.parent.mkdir(parents=True,exist_ok=True)
raw=canonical(value)+b'\n'
fd,tmp=tempfile.mkstemp(prefix='.new-',dir=path.parent)
try:
os.fchmod(fd,mode); os.write(fd,raw); os.fsync(fd); os.close(fd); os.chown(tmp,0,0); os.replace(tmp,path)
os.chmod(path,mode)
finally:
try: os.close(fd)
except OSError: pass
if os.path.exists(tmp): os.unlink(tmp)
def append_audit(event:dict[str,Any]) -> None:
TASK_ROOT.mkdir(parents=True,exist_ok=True)
flags=os.O_WRONLY|os.O_CREAT|os.O_APPEND|getattr(os,'O_NOFOLLOW',0)
fd=os.open(AUDIT,flags,0o600)
try: os.write(fd,canonical({'at':stamp(),'task_id':TASK,'secrets_included':False,**event})+b'\n'); os.fsync(fd)
finally: os.close(fd)
def parent() -> dict[str,Any]:
p=load_json(PARENT)
required={'schema_version','development_authorization_id','authorization_type','task_id','actor_identity','audience','repository','exact_branch','approved_base_oid','allowed_path_roots','allowed_exact_paths','allowed_operations','created_at','valid_until','maximum_wall_clock_seconds','idle_timeout_seconds','manual_reapproval_per_commit','maximum_concurrent_writes','path_policy_sha256','writer_instance_id'}
if set(p)!=required or p.get('schema_version')!=1: fail('parent_authorization_schema_rejected')
fixed={'development_authorization_id':AUTH_ID,'authorization_type':'TASK_SCOPED_CONTINUOUS_DEVELOPMENT','task_id':TASK,'actor_identity':ACTOR,'audience':WRITER_AUDIENCE,'repository':REPO,'exact_branch':BRANCH,'approved_base_oid':BASE,'manual_reapproval_per_commit':False,'maximum_concurrent_writes':1,'writer_instance_id':WRITER_INSTANCE}
if any(p.get(k)!=v for k,v in fixed.items()): fail('parent_authorization_binding_rejected')
if p.get('maximum_wall_clock_seconds')!=86400 or p.get('idle_timeout_seconds')!=2700: fail('parent_authorization_lifetime_rejected')
if not isinstance(p.get('allowed_path_roots'),list) or not isinstance(p.get('allowed_exact_paths'),list) or not isinstance(p.get('allowed_operations'),list): fail('parent_authorization_scope_rejected')
raw=PATH_POLICY.read_bytes()
if hashlib.sha256(raw).hexdigest()!=p.get('path_policy_sha256'): fail('parent_authorization_path_policy_changed')
return p
def parse_time(v:Any) -> datetime:
try:
d=datetime.fromisoformat(str(v).replace('Z','+00:00'))
return d.astimezone(timezone.utc)
except Exception: fail('authorization_time_rejected')
def components(path:str)->tuple[str,...]:
if not isinstance(path,str) or not path or path!=path.strip() or '\\' in path or '\x00' in path: fail('authorization_path_rejected',403)
pp=PurePosixPath(path); wp=PureWindowsPath(path)
parts=tuple(path.split('/'))
if pp.is_absolute() or wp.is_absolute() or bool(wp.drive) or any(x in {'','.','..'} for x in parts): fail('authorization_path_rejected',403)
return parts
def path_allowed(p:dict[str,Any],path:str)->bool:
item=components(path)
for root in p['allowed_path_roots']:
r=components(str(root).rstrip('/'))
if item[:len(r)]==r: return True
return any(item==components(x) for x in p['allowed_exact_paths'])
def git(*args:str,cwd:Path|None=None,env:dict[str,str]|None=None)->str:
e=os.environ.copy(); e.update(env or {}); e['GIT_CONFIG_COUNT']='1'; e['GIT_CONFIG_KEY_0']='safe.directory'; e['GIT_CONFIG_VALUE_0']=str(WORKTREE)
r=subprocess.run(['git',*args],cwd=cwd,text=True,capture_output=True,env=e)
if r.returncode: fail('authorization_git_probe_failed',503)
return r.stdout.strip()
def credential_lines()->list[str]:
cp=Path(os.environ.get('CREDENTIALS_DIRECTORY',''))/'gitea_git_http'
try:
info=os.lstat(cp); lines=cp.read_text(encoding='utf-8').splitlines()
except OSError: fail('authorization_gitea_credential_unavailable')
if not stat.S_ISREG(info.st_mode) or stat.S_ISLNK(info.st_mode) or len(lines)!=2 or lines[0]!='gitops-bot' or not lines[1]: fail('authorization_gitea_credential_rejected')
return lines
def remote_oid()->str:
credential_lines()
e={'GIT_ASKPASS':str(ASKPASS),'GIT_TERMINAL_PROMPT':'0'}
out=git('ls-remote','--heads','http://127.0.0.1:3000/aios/aios.git',f'refs/heads/{BRANCH}',cwd=WORKTREE,env=e)
fields=out.split()
if len(fields)!=2 or not OID_RE.fullmatch(fields[0]): fail('authorization_git_ls_remote_rejected')
return fields[0]
def api_oid()->str:
user,password=credential_lines(); branch=urllib.parse.quote(BRANCH,safe='')
encoded=base64.b64encode((user+':'+password).encode()).decode()
request=urllib.request.Request('http://127.0.0.1:3000/api/v1/repos/aios/aios/branches/'+branch,headers={'Authorization':'Basic '+encoded,'Host':'g.cnnetmkt.top'})
try:
with urllib.request.urlopen(request,timeout=10) as response: value=json.loads(response.read())
except Exception: fail('authorization_gitea_api_unavailable')
oid=str(value.get('commit',{}).get('id',''))
if not OID_RE.fullmatch(oid): fail('authorization_gitea_api_oid_rejected')
return oid
def four_way(expected:str)->None:
local=git('rev-parse','HEAD',cwd=WORKTREE)
bare=git('--git-dir',str(BARE),'rev-parse',f'refs/heads/{BRANCH}')
remote=remote_oid(); api=api_oid()
if any(x!=expected for x in (local,bare,remote,api)): fail('RECONCILIATION_REQUIRED',409,current_head_oid=local)
if git('status','--porcelain',cwd=WORKTREE)!='': fail('authorization_worktree_dirty',409)
def state()->dict[str,Any]:
s=load_json(STATE)
required={'schema_version','development_authorization_id','task_id','status','current_head_oid','last_activity_at','writer_child','test_child','inflight','updated_at'}
if set(s)!=required or s.get('schema_version')!=1 or s.get('development_authorization_id')!=AUTH_ID or s.get('task_id')!=TASK or not OID_RE.fullmatch(str(s.get('current_head_oid',''))): fail('authorization_state_schema_rejected')
return s
def save_state(s:dict[str,Any])->None: s['updated_at']=stamp(); atomic_json(STATE,s)
def lock():
STATE_DIR.mkdir(parents=True,exist_ok=True); os.chmod(STATE_DIR,0o700)
f=open(LOCK,'a+b'); os.chmod(LOCK,0o600); fcntl.flock(f.fileno(),fcntl.LOCK_EX); return f
def route_binding_valid(p:dict[str,Any])->None:
b=load_json(ROUTE_BINDING,0o640)
identities=b.get('action_identities')
if b.get('task_id')!=TASK or b.get('writer_instance_id')!=WRITER_INSTANCE or b.get('repository')!=REPO or b.get('branch')!=BRANCH or b.get('approved_base_oid')!=BASE or b.get('path_policy_sha256')!=p.get('path_policy_sha256') or not isinstance(identities,list) or len(identities)!=1 or identities[0].get('account')!=ACTOR:
fail('authorization_route_binding_changed',409)
def task_live(p:dict[str,Any],s:dict[str,Any],*,touch:bool)->None:
route_binding_valid(p)
if s['status'] in TERMINAL: fail('authorization_task_terminal',409)
if now()>=parse_time(p['valid_until']):
s['status']='REVOKED_WALL_CLOCK'; save_state(s); cleanup(s,'wall_clock'); fail('authorization_task_terminal',409)
if (now()-parse_time(s['last_activity_at'])).total_seconds()>p['idle_timeout_seconds']:
s['status']='REVOKED_IDLE_TIMEOUT'; save_state(s); cleanup(s,'idle_timeout'); fail('authorization_task_terminal',409)
if touch: s['last_activity_at']=stamp(); save_state(s)
def projection_token()->str|None:
if not PROJECTION.exists(): return None
try:
info=os.lstat(PROJECTION)
if not stat.S_ISREG(info.st_mode) or stat.S_ISLNK(info.st_mode) or info.st_uid!=0 or stat.S_IMODE(info.st_mode)!=0o600: fail('authorization_projection_rejected')
token=PROJECTION.read_text(encoding='utf-8').strip()
return token if len(token.encode())>=32 else None
except OSError: fail('authorization_projection_rejected')
def write_projection(token:str)->None:
PROJECTION.parent.mkdir(parents=True,exist_ok=True)
fd,tmp=tempfile.mkstemp(prefix='.writer-',dir=PROJECTION.parent)
try:
os.fchmod(fd,0o600); os.write(fd,(token+'\n').encode()); os.fsync(fd); os.close(fd); os.chown(tmp,0,0); os.replace(tmp,PROJECTION); os.chmod(PROJECTION,0o600)
finally:
try: os.close(fd)
except OSError: pass
if os.path.exists(tmp): os.unlink(tmp)
def post_revoke_401(token:str)->None:
req=urllib.request.Request('http://127.0.0.1:19893/api/runtime-status',headers={'Authorization':'Bearer '+token})
try: urllib.request.urlopen(req,timeout=5)
except urllib.error.HTTPError as e:
if e.code==401:return
except Exception: pass
fail('authorization_old_credential_revocation_unverified')
def rotate_child(p:dict[str,Any],s:dict[str,Any],reason:str)->str:
old=projection_token(); registry=TokenRegistry(REGISTRY)
old_record=registry.find(old) if old else None
if old_record and (old_record.get('task_id')!=TASK or old_record.get('development_authorization_id')!=AUTH_ID):
if parse_time(old_record.get('expires_at'))>=now(): fail('authorization_existing_projection_binding_rejected')
if not registry.revoke(str(old_record['token_id'])): fail('authorization_historical_child_revoke_failed')
post_revoke_401(old)
append_audit({'event':'historical_expired_child_revoked','old_credential_http_status':401})
old_record=None
issued=now().replace(microsecond=0); expires=issued+timedelta(minutes=20)
prefixes=[str(x).rstrip('/')+'/' for x in p['allowed_path_roots']]
grant={'token_id':'continuous-child-'+issued.strftime('%Y%m%dT%H%M%SZ')+'-'+os.urandom(3).hex(),'credential_id':'credential-'+issued.strftime('%Y%m%dT%H%M%SZ')+'-'+os.urandom(3).hex(),'development_authorization_id':AUTH_ID,'audience':WRITER_AUDIENCE,'task_id':TASK,'actor_identity':ACTOR,'writer_instance_id':WRITER_INSTANCE,'repository_identity':REPO,'source_cidrs':['127.0.0.1/32'],'allowed_branch':BRANCH,'current_expected_head_oid':s['current_head_oid'],'approved_base_oid':BASE,'allowed_operations':p['allowed_operations'],'allowed_paths':p['allowed_exact_paths'],'allowed_path_prefixes':prefixes,'not_before':issued.isoformat(),'expires_at':expires.isoformat(),'renewable':False,'max_uses':{operation:CHILD_OPERATION_MAX_USES for operation in p['allowed_operations']},'issued_by':'task_authorization_controller'}
token,record=registry.issue(grant)
try:
write_projection(token)
if old_record:
if not registry.revoke(str(old_record['token_id'])): fail('authorization_old_child_revoke_failed')
post_revoke_401(old)
except Exception:
registry.revoke(str(record['token_id']))
try: PROJECTION.unlink()
except FileNotFoundError: pass
raise
s['writer_child']={'credential_id':record['credential_id'],'issued_at':record['not_before'],'expires_at':record['expires_at'],'current_expected_head_oid':s['current_head_oid'],'rotation_reason':reason}
save_state(s); append_audit({'event':'child_grant_rotated','reason':reason,'current_head_oid':s['current_head_oid'],'old_grant_revoked':bool(old_record),'old_credential_http_status':401 if old_record else None})
return token
def child_operation_quota_exhausted(rec:dict[str,Any],registry:TokenRegistry)->bool:
token_id=str(rec.get('token_id',''))
allowed=rec.get('allowed_operations')
limits=rec.get('max_uses')
if not token_id or not isinstance(allowed,list) or not isinstance(limits,dict): fail('authorization_child_quota_schema_rejected')
for operation in allowed:
maximum=limits.get(operation)
if not isinstance(operation,str) or not isinstance(maximum,int) or maximum<1: fail('authorization_child_quota_schema_rejected')
if registry.usage(token_id,operation)>=maximum: return True
return False
def ensure_child(p:dict[str,Any],s:dict[str,Any],reason:str)->str:
token=projection_token(); meta=s.get('writer_child') or {}
expired=True
try: expired=(parse_time(meta.get('expires_at'))-now()).total_seconds()<=300
except Exception: pass
if not token or expired or meta.get('current_expected_head_oid')!=s['current_head_oid']:
return rotate_child(p,s,reason)
registry=TokenRegistry(REGISTRY)
rec=registry.find(token)
if not rec or rec.get('revoked') or rec.get('development_authorization_id')!=AUTH_ID or rec.get('current_expected_head_oid')!=s['current_head_oid']:
return rotate_child(p,s,reason)
if child_operation_quota_exhausted(rec,registry):
return rotate_child(p,s,'operation_quota_exhausted')
return token
def ledger_verified(operation_id:str,commit_oid:str)->None:
with sqlite3.connect(WRITER_LEDGER) as db:
row=db.execute('select operation_id,branch,commit_oid,state,verification_status from writer_operations where operation_id=?',(operation_id,)).fetchone()
if not row or row[1]!=BRANCH or row[2]!=commit_oid or row[3]!='COMPLETED' or row[4]!='bare+gitea': fail('authorization_writer_ledger_rejected',409)
def reconcile(p:dict[str,Any],s:dict[str,Any],previous:str,new:str,operation_id:str)->dict[str,Any]:
inflight=s.get('inflight')
if not isinstance(inflight,dict) or inflight.get('previous_head_oid')!=previous or inflight.get('operation_id')!=operation_id: fail('authorization_inflight_rejected',409)
if previous!=s['current_head_oid'] or not OID_RE.fullmatch(new): fail('authorization_head_transition_rejected',409)
ledger_verified(operation_id,new); four_way(new)
s['current_head_oid']=new; s['inflight']=None; s['last_activity_at']=stamp(); save_state(s)
ensure_child(p,s,'head_advance')
append_audit({'event':'authorization_head_advanced','previous_head_oid':previous,'new_commit_oid':new,'four_way_ref_status':True,'worktree_clean':True})
return {'authorization_current_head_oid':new,'automatic_head_advance':True,'four_way_ref_status':True}
def prepare(p:dict[str,Any],s:dict[str,Any],data:dict[str,Any])->dict[str,Any]:
required={'expected_current_head_oid','operation_id','idempotency_key','exact_path','content_digest','commit_message_digest','branch','path','message'}
if set(data)!=required: fail('authorization_request_schema_rejected',403)
if data['branch']!=BRANCH or data['path']!=data['exact_path'] or not path_allowed(p,str(data['path'])): fail('authorization_path_rejected',403)
if data['expected_current_head_oid']!=s['current_head_oid'] or not OID_RE.fullmatch(str(data['expected_current_head_oid'])): fail('BRANCH_HEAD_CHANGED',409,current_head_oid=s['current_head_oid'])
if not OP_RE.fullmatch(str(data['operation_id'])) or not OP_RE.fullmatch(str(data['idempotency_key'])): fail('authorization_operation_id_rejected',403)
if not all(isinstance(data[k],str) and re.fullmatch(r'[0-9a-f]{64}',data[k]) for k in ('content_digest','commit_message_digest')): fail('authorization_digest_rejected',403)
if hashlib.sha256(data['message'].encode()).hexdigest()!=data['commit_message_digest']: fail('authorization_commit_message_digest_rejected',403)
inflight=s.get('inflight')
if inflight:
if inflight.get('operation_id')==data['operation_id'] and inflight.get('idempotency_key')==data['idempotency_key']: return {'expected_head_oid':s['current_head_oid'],'idempotency_key':data['idempotency_key']}
fail('BRANCH_HEAD_CHANGED',409,current_head_oid=s['current_head_oid'])
four_way(s['current_head_oid'])
s['inflight']={'operation_id':data['operation_id'],'idempotency_key':data['idempotency_key'],'previous_head_oid':s['current_head_oid'],'path':data['path'],'created_at':stamp()}; s['last_activity_at']=stamp(); save_state(s)
append_audit({'event':'write_prepared','operation_id':data['operation_id'],'path':data['path'],'expected_head_oid':s['current_head_oid']})
return {'expected_head_oid':s['current_head_oid'],'idempotency_key':data['idempotency_key']}
def test_authorize(p:dict[str,Any],s:dict[str,Any],operation:str)->dict[str,Any]:
allowed={'pytest_dormant','pytest_autonomous_release','compileall_autonomous_release','git_diff_check'}
if operation not in allowed: fail('candidate_test_executor_operation_rejected',403)
four_way(s['current_head_oid'])
child=s.get('test_child') or {}; refresh=True
try: refresh=(parse_time(child.get('expires_at'))-now()).total_seconds()<=300
except Exception: pass
if refresh:
issued=now().replace(microsecond=0); child={'test_authorization_id':'test-authz-aios-autonomous-release-20260726','grant_id':'test-child-'+issued.strftime('%Y%m%dT%H%M%SZ'),'issued_at':issued.isoformat(),'expires_at':(issued+timedelta(minutes=20)).isoformat()}; s['test_child']=child
s['last_activity_at']=stamp(); save_state(s); append_audit({'event':'test_authorized','operation':operation})
return {'test_authorization_id':child['test_authorization_id'],'operation':operation,'expires_at':child['expires_at']}
def cleanup(s:dict[str,Any],reason:str)->None:
old=projection_token()
if old:
reg=TokenRegistry(REGISTRY); rec=reg.find(old)
if rec: reg.revoke(str(rec['token_id']))
try: PROJECTION.unlink()
except FileNotFoundError: pass
try: post_revoke_401(old)
except ControlError: pass
for unit in ('aios-chatgpt-autonomous-release-route.service','aios-chatgpt-autonomous-release-writer.service'):
subprocess.run(['/usr/bin/systemctl','stop',unit],stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL,timeout=20)
subprocess.run(['/usr/bin/systemctl','stop','aios-autonomous-release-dormant-test-executor@*.service'],stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL,timeout=20)
try: ROUTE_BINDING.unlink()
except FileNotFoundError: pass
try: SOCKET_PATH.unlink()
except FileNotFoundError: pass
append_audit({'event':'authorization_cleanup','reason':reason,'projection_removed':not PROJECTION.exists(),'route_overlay_removed':not ROUTE_BINDING.exists(),'task_services_stop_requested':True})
def request(data:dict[str,Any])->dict[str,Any]:
op=data.get('op'); f=lock()
try:
p=parent(); s=state(); task_live(p,s,touch=False)
if op=='describe_authorization':
four_way(s['current_head_oid'])
return {'development_authorization_id':AUTH_ID,'task_id':TASK,'actor_identity':ACTOR,'audience':WRITER_AUDIENCE,'repository':REPO,'branch':BRANCH,'approved_base_oid':BASE,'authorization_status':s['status'],'current_head_oid':s['current_head_oid']}
if op=='get_writer_token':
four_way(s['current_head_oid']); token=ensure_child(p,s,'scheduled_or_requested_rotation'); return {'token':token,'current_head_oid':s['current_head_oid']}
if op=='prepare_write':
ensure_child(p,s,'pre_write_quota_check')
return prepare(p,s,data.get('request') if isinstance(data.get('request'),dict) else {})
if op=='reconcile_commit':
r=data.get('result') if isinstance(data.get('result'),dict) else {}; return reconcile(p,s,str(r.get('previous_head_oid','')),str(r.get('new_commit_oid','')),str(r.get('operation_id','')))
if op=='fail_write':
r=data.get('result') if isinstance(data.get('result'),dict) else {}; inflight=s.get('inflight')
if isinstance(inflight,dict) and inflight.get('operation_id')==r.get('operation_id'): s['inflight']=None; save_state(s)
return {'cleared':True}
if op=='authorize_test': return test_authorize(p,s,str(data.get('operation','')))
fail('authorization_operation_rejected',403)
finally:
fcntl.flock(f.fileno(),fcntl.LOCK_UN); f.close()
def peer_allowed(conn:socket.socket)->bool:
try:
import struct
_pid,uid,gid=struct.unpack('3i',conn.getsockopt(socket.SOL_SOCKET,socket.SO_PEERCRED,12))
return uid==pwd.getpwnam('aiman').pw_uid and gid==grp.getgrnam('gitops').gr_gid
except Exception: return False
def serve()->None:
SOCKET_DIR.mkdir(parents=True,exist_ok=True); os.chown(SOCKET_DIR,0,grp.getgrnam('gitops').gr_gid); os.chmod(SOCKET_DIR,0o750)
try: SOCKET_PATH.unlink()
except FileNotFoundError: pass
with socket.socket(socket.AF_UNIX,socket.SOCK_STREAM) as server:
server.bind(str(SOCKET_PATH)); os.chown(SOCKET_PATH,0,grp.getgrnam('gitops').gr_gid); os.chmod(SOCKET_PATH,0o660); server.listen(32); server.settimeout(1.0)
last=0.0
while True:
if time.monotonic()-last>30:
last=time.monotonic()
try:
f=lock(); p=parent(); s=state(); task_live(p,s,touch=False); four_way(s['current_head_oid']); ensure_child(p,s,'scheduled_rotation'); fcntl.flock(f.fileno(),fcntl.LOCK_UN); f.close()
except ControlError as e:
append_audit({'event':'scheduled_check_blocked','error':e.code})
if e.code=='authorization_task_terminal': return
except Exception: append_audit({'event':'scheduled_check_blocked','error':'authorization_controller_internal_error'})
try: conn,_=server.accept()
except socket.timeout: continue
with conn:
if not peer_allowed(conn): conn.sendall(canonical({'ok':False,'status':403,'error':'authorization_peer_rejected'})+b'\n'); continue
try:
raw=conn.recv(131072); data=json.loads(raw.decode()); result=request(data); conn.sendall(canonical({'ok':True,'result':result})+b'\n')
except ControlError as e: conn.sendall(canonical({'ok':False,'status':e.status,'error':e.code,**e.details})+b'\n')
except Exception as exc:
append_audit({'event':'controller_internal_error','error_type':type(exc).__name__})
conn.sendall(canonical({'ok':False,'status':503,'error':'authorization_controller_internal_error'})+b'\n')
if __name__=='__main__': serve()
@@ -0,0 +1,89 @@
"""Governed API facade for branch-bound repair leases.
The facade is transport-neutral so the permanent control plane can expose it
through its existing authenticated loopback/API seam without creating a second
Git writer or a global credential registry.
"""
from __future__ import annotations
from typing import Any, Mapping
from branch_repair_lease import LeaseError, RepairLease, RepairLeaseManager, sanitized_lease_view
class BranchRepairLeaseAPI:
"""Validate operation envelopes and return sanitized lease metadata."""
def __init__(self, manager: RepairLeaseManager) -> None:
self.manager = manager
@staticmethod
def _required(request: Mapping[str, Any], name: str) -> Any:
value = request.get(name)
if value is None or value == "":
raise LeaseError(f"missing required field: {name}")
return value
@staticmethod
def _view(lease: RepairLease) -> dict[str, Any]:
return {"status": "ok", "lease": sanitized_lease_view(lease)}
def handle(self, request: Mapping[str, Any]) -> dict[str, Any]:
try:
operation = self._required(request, "operation")
owner = str(self._required(request, "owner_identity"))
if operation == "issue":
lease = self.manager.issue(
task_id=str(self._required(request, "task_id")),
owner_identity=owner,
branch=str(self._required(request, "branch")),
description=str(self._required(request, "description")),
allowed_paths=tuple(self._required(request, "allowed_paths")),
ttl_seconds=int(request.get("ttl_seconds", 3600)),
)
elif operation == "attach_existing_branch":
lease = self.manager.attach_existing_branch(
str(self._required(request, "lease_id")),
owner_identity=owner,
expected_head_oid=str(self._required(request, "expected_head_oid")),
)
elif operation == "authorize_commit":
lease = self.manager.authorize_commit(
str(self._required(request, "lease_id")),
owner_identity=owner,
expected_head_oid=str(self._required(request, "expected_head_oid")),
paths=tuple(self._required(request, "paths")),
content=request.get("content"),
)
elif operation == "record_commit":
lease = self.manager.record_commit(
str(self._required(request, "lease_id")),
owner_identity=owner,
expected_head_oid=str(self._required(request, "expected_head_oid")),
new_head_oid=str(self._required(request, "new_head_oid")),
paths=tuple(self._required(request, "paths")),
)
elif operation == "recover":
lease = self.manager.recover(
str(self._required(request, "lease_id")), owner_identity=owner
)
elif operation == "renew":
lease = self.manager.renew(
str(self._required(request, "lease_id")),
owner_identity=owner,
ttl_seconds=int(self._required(request, "ttl_seconds")),
)
elif operation == "complete":
lease = self.manager.complete(
str(self._required(request, "lease_id")), owner_identity=owner
)
elif operation == "revoke":
lease = self.manager.revoke(
str(self._required(request, "lease_id")), owner_identity=owner
)
else:
raise LeaseError("unsupported lease operation")
return self._view(lease)
except (LeaseError, TypeError, ValueError):
return {"status": "blocked", "error": "lease_request_rejected"}
@@ -0,0 +1,48 @@
"""Root-only CLI for the branch-bound repair lease API."""
from __future__ import annotations
import argparse
import json
import os
import sys
from pathlib import Path
from aios_branch_repair_lease_api import BranchRepairLeaseAPI
from branch_repair_lease import GitProbe, LeaseStore
from branch_repair_lease import RepairLeaseManager
def main() -> int:
parser = argparse.ArgumentParser(description="AIOS branch repair lease CLI")
parser.add_argument("--store", required=True)
parser.add_argument("--bare-repo", required=True)
parser.add_argument("--request-file")
parser.add_argument("--request-json")
args = parser.parse_args()
if hasattr(os, "geteuid") and os.geteuid() != 0:
print(json.dumps({"status": "blocked", "error": "root_required"}))
return 2
if bool(args.request_file) == bool(args.request_json):
print(json.dumps({"status": "blocked", "error": "one_request_source_required"}))
return 2
try:
raw = (
Path(args.request_file).read_text(encoding="utf-8")
if args.request_file
else str(args.request_json)
)
request = json.loads(raw)
if not isinstance(request, dict):
raise ValueError("request must be an object")
api = BranchRepairLeaseAPI(RepairLeaseManager(LeaseStore(args.store), GitProbe(args.bare_repo)))
response = api.handle(request)
print(json.dumps(response, ensure_ascii=True, sort_keys=True, separators=(",", ":")))
return 0 if response.get("status") == "ok" else 1
except Exception:
print(json.dumps({"status": "blocked", "error": "lease_cli_failed"}))
return 1
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,104 @@
"""Read-only validation of recovered campaign authorization evidence."""
from __future__ import annotations
import hashlib
import json
from pathlib import Path
import stat
from typing import Any
class RecoveryEvidenceError(RuntimeError):
pass
def _digest(path: Path) -> str:
h = hashlib.sha256()
with path.open("rb") as stream:
for block in iter(lambda: stream.read(1024 * 1024), b""):
h.update(block)
return h.hexdigest()
def _secure_json(path: Path) -> dict[str, Any]:
if path.is_symlink() or not path.is_file():
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_MISSING")
info = path.stat(follow_symlinks=False)
if info.st_uid != 0 or stat.S_IMODE(info.st_mode) & 0o022:
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_PERMISSIONS")
try:
value = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_INVALID") from exc
if not isinstance(value, dict):
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_OBJECT_REQUIRED")
return value
def load_recovered_authorization(path: str | Path) -> dict[str, Any]:
"""Verify the recovery receipt, sidecar, and all referenced immutable files."""
receipt_path = Path(path)
receipt = _secure_json(receipt_path)
sidecar = receipt_path.with_suffix(receipt_path.suffix + ".sha256")
if sidecar.is_symlink() or not sidecar.is_file():
raise RecoveryEvidenceError("RECOVERY_DIGEST_SIDECAR_MISSING")
expected = sidecar.read_text(encoding="ascii").split()[0]
actual = _digest(receipt_path)
if expected != actual:
raise RecoveryEvidenceError("RECOVERY_DIGEST_MISMATCH")
required = {
"schema",
"campaign_id",
"original_event_id",
"original_authorization_digest",
"signed_grant_id",
"signed_grant_digest",
"task_id",
"repository",
"original_base_oid",
"branch",
"expected_branch_oid",
"recovered_from_existing_evidence",
"new_owner_interaction",
"recovery_state",
"production_side_effect_executed",
"global_writes_enabled",
"grant_consumed",
"owner_authorization_consumed",
"original_receipts_immutable",
"recovered_at",
"evidence_files",
}
if set(receipt) != required or receipt["schema"] != "aios.owner-authorization-recovery.v1":
raise RecoveryEvidenceError("RECOVERY_SCHEMA_INVALID")
if receipt["recovered_from_existing_evidence"] is not True or receipt["new_owner_interaction"] is not False:
raise RecoveryEvidenceError("RECOVERY_NOT_APPEND_ONLY")
if receipt["production_side_effect_executed"] is not False or receipt["global_writes_enabled"] is not False:
raise RecoveryEvidenceError("RECOVERY_SCOPE_INVALID")
if receipt["recovery_state"] != "RECOVERED_PENDING_CONTINUATION_LEASE":
raise RecoveryEvidenceError("RECOVERY_STATE_INVALID")
evidence = receipt["evidence_files"]
if not isinstance(evidence, list) or not evidence:
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_LIST_INVALID")
for item in evidence:
if not isinstance(item, dict) or set(item) != {"path", "sha256"}:
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_ENTRY_INVALID")
target = Path(item["path"])
if _digest(target) != item["sha256"]:
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_DIGEST_MISMATCH")
return {
"schema": receipt["schema"],
"state": receipt["recovery_state"],
"campaign_id": receipt["campaign_id"],
"original_event_id": receipt["original_event_id"],
"original_authorization_digest": receipt["original_authorization_digest"],
"signed_grant_id": receipt["signed_grant_id"],
"task_id": receipt["task_id"],
"repository": receipt["repository"],
"original_base_oid": receipt["original_base_oid"],
"branch": receipt["branch"],
"expected_branch_oid": receipt["expected_branch_oid"],
"digest": actual,
"new_owner_interaction": False,
}
@@ -1,567 +0,0 @@
#!/usr/bin/env python3
"""Task-bound bridge from the legacy ChatGPT Action to an isolated Writer V2."""
from __future__ import annotations
import hashlib
import hmac
import json
import os
import re
import sqlite3
import socket
import stat
import urllib.error
import urllib.parse
import urllib.request
from datetime import datetime, timezone
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path, PurePosixPath, PureWindowsPath
from typing import Any
CONFIG_PATH = Path(os.environ["AIOS_ACTION_ROUTE_BINDING"])
POLICY_PATH = Path(os.environ["AIOS_ACTION_ROUTE_PATH_POLICY"])
KNOWN_IDENTITIES_PATH = Path(os.environ["AIOS_ACTION_ROUTE_KNOWN_IDENTITIES"])
STATE_PATH = Path(os.environ["AIOS_ACTION_ROUTE_STATE"])
AUDIT_PATH = Path(os.environ["AIOS_ACTION_ROUTE_AUDIT"])
HOST = os.environ.get("AIOS_ACTION_ROUTE_HOST", "127.0.0.1")
PORT = int(os.environ.get("AIOS_ACTION_ROUTE_PORT", "19894"))
EXPECTED_TASK_ID = os.environ["AIOS_ACTION_ROUTE_TASK_ID"]
EXPECTED_ROUTE_HEADER = os.environ["AIOS_ACTION_ROUTE_HEADER"]
AUTHORIZATION_SOCKET = Path(os.environ["AIOS_CONTINUOUS_AUTHORIZATION_SOCKET"])
OID_RE = re.compile(r"[0-9a-f]{40}")
SECRET_RE = re.compile(
r"(?i)(api[_-]?key|secret[_-]?access[_-]?key|secret|password|token)"
r"\s*[:=]\s*[\"']?([A-Za-z0-9_./+=:-]{16,})"
)
SECRET_VALUE_RE = re.compile(
r"(sk-[A-Za-z0-9_-]{20,}|HPUA[A-Z0-9]{10,}|SDC[A-Za-z0-9]{20,}|"
r"-----BEGIN (?:OPENSSH |RSA |EC )?PRIVATE KEY-----)"
)
ALLOWED_REQUEST_FIELDS = frozenset({"base","branch","content","create_branch","message","path","task_id","development_authorization_id","actor_identity","audience","approved_base_oid","expected_current_head_oid","operation_id","idempotency_key","exact_path","content_digest","commit_message_digest"})
class RouteError(RuntimeError):
def __init__(self, code: str, status: int = 400, **details: Any):
super().__init__(code)
self.code = code
self.status = status
self.details = details
def utc_now() -> str:
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
def canonical_json(value: Any) -> bytes:
return json.dumps(
value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
).encode("utf-8")
def load_root_managed_json(
path: Path, *, expected_mode: int = 0o640
) -> tuple[dict[str, Any], bytes]:
info = os.lstat(path)
if stat.S_ISLNK(info.st_mode) or not stat.S_ISREG(info.st_mode):
raise RouteError("writer_route_config_type_rejected", 503)
if info.st_uid != 0 or stat.S_IMODE(info.st_mode) & ~expected_mode:
raise RouteError("writer_route_config_permissions_rejected", 503)
raw = path.read_bytes()
try:
value = json.loads(raw)
except (UnicodeDecodeError, json.JSONDecodeError):
raise RouteError("writer_route_config_json_rejected", 503) from None
if not isinstance(value, dict):
raise RouteError("writer_route_config_object_required", 503)
return value, raw
def load_binding() -> dict[str, Any]:
binding, _ = load_root_managed_json(CONFIG_PATH)
required = {
"schema_version",
"task_id",
"writer_instance_id",
"repository",
"branch",
"approved_base_oid",
"writer_url",
"path_policy_sha256",
"action_identities",
}
if set(binding) != required or binding.get("schema_version") != 1:
raise RouteError("writer_route_binding_schema_rejected", 503)
if binding["task_id"] != EXPECTED_TASK_ID:
raise RouteError("writer_route_task_binding_rejected", 503)
if binding["repository"] != "aios/aios":
raise RouteError("writer_route_repository_rejected", 503)
if not OID_RE.fullmatch(str(binding["approved_base_oid"])):
raise RouteError("writer_route_base_oid_rejected", 503)
if binding["writer_url"] != "http://127.0.0.1:19893":
raise RouteError("writer_route_upstream_rejected", 503)
identities = binding.get("action_identities")
if not isinstance(identities, list) or len(identities) != 1:
raise RouteError("writer_route_action_identity_rejected", 503)
identity = identities[0]
if (
not isinstance(identity, dict)
or set(identity) != {"account", "token_sha256"}
or identity.get("account") != "chatgpt-gitops-bot"
or not re.fullmatch(
r"[0-9a-f]{64}", str(identity.get("token_sha256", ""))
)
):
raise RouteError("writer_route_action_identity_rejected", 503)
return binding
def allowed_path_scope(binding: dict[str, Any]) -> tuple[set[str], tuple[str, ...]]:
policy, raw = load_root_managed_json(POLICY_PATH)
actual = hashlib.sha256(raw).hexdigest()
if actual != binding["path_policy_sha256"]:
raise RouteError("writer_route_path_policy_digest_mismatch", 503)
if policy.get("allowed_branch") != binding["branch"]:
raise RouteError("writer_route_path_policy_branch_mismatch", 503)
operation = (policy.get("allowed_operations") or {}).get("commit_file.atomic")
paths = operation.get("exact_paths") if isinstance(operation, dict) else None
prefixes = operation.get("prefix_paths") if isinstance(operation, dict) else None
if not isinstance(paths, list) or not paths or not all(
isinstance(x, str) for x in paths
):
raise RouteError("writer_route_path_policy_rejected", 503)
if not isinstance(prefixes, list) or not prefixes or not all(
isinstance(x, str) and x.endswith("/") for x in prefixes
):
raise RouteError("writer_route_path_policy_rejected", 503)
if any(not valid_relative_path(x[:-1]) for x in prefixes):
raise RouteError("writer_route_path_policy_rejected", 503)
if any(not valid_relative_path(x) for x in paths):
raise RouteError("writer_route_path_policy_rejected", 503)
return set(paths), tuple(prefixes)
def valid_relative_path(path: str) -> bool:
if not isinstance(path, str) or not path or path != path.strip():
return False
if "\x00" in path or "\\" in path:
return False
try:
decoded = urllib.parse.unquote(path, errors="strict")
except (UnicodeDecodeError, ValueError):
return False
if decoded != path:
return False
posix = PurePosixPath(path)
windows = PureWindowsPath(path)
return not (
posix.is_absolute()
or windows.is_absolute()
or bool(windows.drive)
or any(part in {"", ".", ".."} for part in path.split("/"))
)
def path_allowed(binding: dict[str, Any], path: str) -> bool:
if not valid_relative_path(path):
return False
exact_paths, prefixes = allowed_path_scope(binding)
return path in exact_paths or any(path.startswith(prefix) for prefix in prefixes)
def bearer(headers: Any) -> str:
value = headers.get("Authorization", "")
if not value.startswith("Bearer "):
raise RouteError("unauthorized", 401)
token = value[7:].strip()
if not token:
raise RouteError("unauthorized", 401)
return token
def authorize_action(headers: Any, binding: dict[str, Any]) -> str:
digest = hashlib.sha256(bearer(headers).encode("utf-8")).hexdigest()
identity = binding["action_identities"][0]
if not hmac.compare_digest(identity["token_sha256"], digest):
known, _ = load_root_managed_json(KNOWN_IDENTITIES_PATH)
identities = known.get("identities")
if known.get("schema_version") != 1 or not isinstance(identities, list):
raise RouteError("writer_route_identity_catalog_rejected", 503)
observed = "unregistered"
allowed_accounts = {
"chatgpt-gitops-bot",
"gitops-write-bot",
"codex-gitops-bot",
}
for candidate in identities:
if not isinstance(candidate, dict) or set(candidate) != {
"account",
"token_sha256",
}:
raise RouteError("writer_route_identity_catalog_rejected", 503)
account = str(candidate.get("account", ""))
token_sha256 = str(candidate.get("token_sha256", ""))
if account not in allowed_accounts or not re.fullmatch(
r"[0-9a-f]{64}", token_sha256
):
raise RouteError("writer_route_identity_catalog_rejected", 503)
if hmac.compare_digest(token_sha256, digest):
observed = account
raise RouteError(
"writer_route_action_identity_rejected",
403,
accepted_action_identity=str(identity["account"]),
observed_action_identity=observed,
identity_alias_accepted=False,
)
return str(identity["account"])
def authorization_call(operation: str, **payload: Any) -> dict[str, Any]:
request = {"op": operation, **payload}
try:
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client:
client.settimeout(15)
client.connect(str(AUTHORIZATION_SOCKET))
client.sendall(canonical_json(request))
raw = client.recv(131072)
response = json.loads(raw.decode("utf-8"))
except (OSError, TimeoutError, UnicodeDecodeError, json.JSONDecodeError):
raise RouteError("authorization_controller_unavailable", 503) from None
if not isinstance(response, dict) or not response.get("ok"):
if not isinstance(response, dict):
raise RouteError("authorization_controller_response_rejected", 503)
details = {k: v for k, v in response.items() if k not in {"ok", "status", "error"}}
raise RouteError(str(response.get("error") or "authorization_controller_rejected"), int(response.get("status") or 503), **details)
result = response.get("result")
if not isinstance(result, dict):
raise RouteError("authorization_controller_response_rejected", 503)
return result
def writer_token() -> str:
value = authorization_call("get_writer_token").get("token")
if not isinstance(value, str) or len(value.encode("utf-8")) < 32:
raise RouteError("writer_route_credential_rejected", 503)
return value
def writer_call(
binding: dict[str, Any],
method: str,
path: str,
payload: dict[str, Any] | None = None,
) -> tuple[int, dict[str, Any]]:
data = canonical_json(payload) if payload is not None else None
headers = {"Authorization": "Bearer " + writer_token()}
if data is not None:
headers["Content-Type"] = "application/json"
request = urllib.request.Request(
binding["writer_url"] + path, data=data, method=method, headers=headers
)
try:
with urllib.request.urlopen(request, timeout=90) as response:
body = json.loads(response.read())
return response.status, body
except urllib.error.HTTPError as exc:
try:
body = json.loads(exc.read())
except Exception:
body = {"error": "writer_route_upstream_non_json_error"}
return exc.code, body
except (OSError, TimeoutError):
raise RouteError("writer_route_upstream_unavailable", 503) from None
def validate_request(data: Any, binding: dict[str, Any]) -> dict[str, Any]:
if not isinstance(data, dict):
raise RouteError("writer_route_request_object_required")
if not {"branch", "path", "content"}.issubset(data):
raise RouteError("writer_route_request_incomplete")
if not set(data).issubset(ALLOWED_REQUEST_FIELDS):
raise RouteError("writer_route_request_fields_rejected")
if data["branch"] != binding["branch"]:
raise RouteError("writer_route_request_binding_failed", 403)
if data.get("create_branch") not in (None, False):
raise RouteError("writer_route_request_binding_failed", 403)
if data.get("base", "main") not in {"main", binding["approved_base_oid"]}:
raise RouteError("writer_route_request_binding_failed", 403)
path = data["path"]
if not isinstance(path, str) or not path_allowed(binding, path):
raise RouteError("writer_route_request_binding_failed", 403)
content = data["content"]
if not isinstance(content, str) or "\x00" in content or "\r" in content:
raise RouteError("writer_route_content_rejected")
if len(content.encode("utf-8")) > 1_048_576:
raise RouteError("writer_route_content_too_large", 413)
if SECRET_RE.search(content) or SECRET_VALUE_RE.search(content):
raise RouteError("writer_route_secret_content_rejected", 403)
message = data.get("message") or f"chore(chatgpt): update {path}"
if not isinstance(message, str) or not 1 <= len(message) <= 200 or "\x00" in message or "\n" in message or "\r" in message:
raise RouteError("writer_route_message_rejected")
return {"branch": binding["branch"], "path": path, "content": content, "message": message}
def complete_request(data: dict[str, Any], normalized: dict[str, Any], binding: dict[str, Any], account: str) -> dict[str, Any]:
authorization = authorization_call("describe_authorization")
expected = {
"task_id": binding["task_id"],
"development_authorization_id": authorization.get("development_authorization_id"),
"actor_identity": account,
"audience": authorization.get("audience"),
"branch": binding["branch"],
"approved_base_oid": binding["approved_base_oid"],
}
required = {
"task_id": EXPECTED_TASK_ID,
"development_authorization_id": "authz-aios-autonomous-release-20260726",
"actor_identity": "chatgpt-gitops-bot",
"audience": "chatgpt-autonomous-release-writer",
"branch": binding["branch"],
"approved_base_oid": binding["approved_base_oid"],
}
if expected != required or authorization.get("authorization_status") != "ACTIVE" or authorization.get("current_head_oid") is None:
raise RouteError("writer_route_request_binding_failed", 403)
for key, value in required.items():
if key in data and data[key] != value:
raise RouteError("writer_route_request_binding_failed", 403)
if "exact_path" in data and data["exact_path"] != normalized["path"]:
raise RouteError("writer_route_request_binding_failed", 403)
current_head = str(authorization["current_head_oid"])
if not OID_RE.fullmatch(current_head):
raise RouteError("authorization_current_head_rejected", 503)
if "expected_current_head_oid" in data and data["expected_current_head_oid"] != current_head:
raise RouteError("BRANCH_HEAD_CHANGED", 409, current_head_oid=current_head)
content_digest = hashlib.sha256(normalized["content"].encode("utf-8")).hexdigest()
message_digest = hashlib.sha256(normalized["message"].encode("utf-8")).hexdigest()
if "content_digest" in data and data["content_digest"] != content_digest:
raise RouteError("writer_route_request_binding_failed", 403)
if "commit_message_digest" in data and data["commit_message_digest"] != message_digest:
raise RouteError("writer_route_request_binding_failed", 403)
seed = {**required, "expected_current_head_oid": current_head, "path": normalized["path"], "content_digest": content_digest, "commit_message_digest": message_digest}
request_digest = hashlib.sha256(canonical_json(seed)).hexdigest()
idempotency_key = "action-" + request_digest
operation_id = hashlib.sha256((EXPECTED_TASK_ID + "\0" + idempotency_key).encode("utf-8")).hexdigest()
if "idempotency_key" in data and data["idempotency_key"] != idempotency_key:
raise RouteError("writer_route_request_binding_failed", 403)
if "operation_id" in data and data["operation_id"] != operation_id:
raise RouteError("writer_route_request_binding_failed", 403)
return {**normalized, **required, "expected_current_head_oid": current_head, "operation_id": operation_id, "idempotency_key": idempotency_key, "exact_path": normalized["path"], "content_digest": content_digest, "commit_message_digest": message_digest, "request_digest": request_digest}
def current_writer_head(binding: dict[str, Any]) -> str:
status, body = writer_call(binding, "GET", "/api/runtime-status")
if status != 200:
raise RouteError(
"writer_route_runtime_status_failed", 503, upstream_status=status
)
required = {
"service": "chatgpt-autonomous-release-writer",
"writes_enabled": True,
"allowed_branch": binding["branch"],
"approved_base_oid": binding["approved_base_oid"],
"path_policy_sha256": binding["path_policy_sha256"],
}
if any(body.get(key) != value for key, value in required.items()):
raise RouteError("writer_route_runtime_binding_mismatch", 503)
head = str(body.get("workspace_head_oid", ""))
if not OID_RE.fullmatch(head):
raise RouteError("writer_route_runtime_head_rejected", 503)
return head
def operation_binding(
request_digest: str, binding: dict[str, Any]
) -> tuple[str, str]:
STATE_PATH.parent.mkdir(parents=True, exist_ok=True)
with sqlite3.connect(STATE_PATH, timeout=30, isolation_level=None) as db:
db.execute("PRAGMA journal_mode=WAL")
db.execute("PRAGMA synchronous=FULL")
db.execute(
"""
CREATE TABLE IF NOT EXISTS action_operations (
request_digest TEXT PRIMARY KEY,
expected_head_oid TEXT NOT NULL,
idempotency_key TEXT NOT NULL UNIQUE,
created_at TEXT NOT NULL
)
"""
)
row = db.execute(
"SELECT expected_head_oid,idempotency_key FROM action_operations "
"WHERE request_digest=?",
(request_digest,),
).fetchone()
if row:
return str(row[0]), str(row[1])
head = current_writer_head(binding)
key = "action-" + request_digest
with sqlite3.connect(STATE_PATH, timeout=30, isolation_level=None) as db:
db.execute("PRAGMA journal_mode=WAL")
db.execute("PRAGMA synchronous=FULL")
db.execute("BEGIN IMMEDIATE")
db.execute(
"INSERT OR IGNORE INTO action_operations"
"(request_digest,expected_head_oid,idempotency_key,created_at) "
"VALUES(?,?,?,?)",
(request_digest, head, key, utc_now()),
)
row = db.execute(
"SELECT expected_head_oid,idempotency_key FROM action_operations "
"WHERE request_digest=?",
(request_digest,),
).fetchone()
db.execute("COMMIT")
return str(row[0]), str(row[1])
def append_audit(event: dict[str, Any]) -> None:
AUDIT_PATH.parent.mkdir(parents=True, exist_ok=True)
flags = os.O_WRONLY | os.O_CREAT | os.O_APPEND | getattr(os, "O_NOFOLLOW", 0)
fd = os.open(AUDIT_PATH, flags, 0o600)
try:
os.write(fd, canonical_json(event) + b"\n")
os.fsync(fd)
finally:
os.close(fd)
class Handler(BaseHTTPRequestHandler):
server_version = "AIOSActionTaskRouter/1"
def log_message(self, _format: str, *_args: Any) -> None:
return
def send_json(self, status: int, value: dict[str, Any]) -> None:
body = json.dumps(value, ensure_ascii=False, sort_keys=True).encode("utf-8")
self.send_response(status)
self.send_header("Content-Type", "application/json; charset=utf-8")
self.send_header("Content-Length", str(len(body)))
self.send_header("Cache-Control", "no-store")
self.end_headers()
self.wfile.write(body)
def do_GET(self) -> None:
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
if path != "/health":
return self.send_json(404, {"error": "not_found"})
try:
binding = load_binding()
self.send_json(
200,
{
"status": "ok",
"service": "aios-action-task-router",
"task_id": binding["task_id"],
"writer_instance_id": binding["writer_instance_id"],
"branch": binding["branch"],
},
)
except RouteError as exc:
self.send_json(exc.status, {"error": exc.code, **exc.details})
def do_POST(self) -> None:
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
if path != "/api/file":
return self.send_json(404, {"error": "not_found"})
if self.client_address[0] not in {"127.0.0.1", "::1"}:
return self.send_json(403, {"error": "writer_route_source_rejected"})
if self.headers.get("X-AIOS-Route-Task") != EXPECTED_ROUTE_HEADER:
return self.send_json(403, {"error": "writer_route_header_rejected"})
try:
length = int(self.headers.get("Content-Length", "0"))
if not 0 < length <= 1_100_000:
raise RouteError("writer_route_content_length_rejected", 413)
try:
data = json.loads(self.rfile.read(length).decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError):
raise RouteError("writer_route_request_json_rejected") from None
binding = load_binding()
account = authorize_action(self.headers, binding)
normalized = validate_request(data, binding)
normalized = complete_request(data, normalized, binding, account)
prepared = authorization_call("prepare_write", request={key: normalized[key] for key in ("expected_current_head_oid", "operation_id", "idempotency_key", "exact_path", "content_digest", "commit_message_digest", "branch", "path", "message")})
expected_head = str(prepared["expected_head_oid"])
writer_payload = {"branch": normalized["branch"], "path": normalized["path"], "content": normalized["content"], "message": normalized["message"], "expected_head_oid": expected_head, "idempotency_key": str(prepared["idempotency_key"])}
request_digest = hashlib.sha256(canonical_json({"task_id": binding["task_id"], "operation_id": normalized["operation_id"], "idempotency_key": normalized["idempotency_key"], "content_digest": normalized["content_digest"], "commit_message_digest": normalized["commit_message_digest"]})).hexdigest()
status, result = writer_call(
binding, "POST", "/api/commit-file", writer_payload
)
audit = {
"at": utc_now(),
"task_id": binding["task_id"],
"writer_instance_id": binding["writer_instance_id"],
"account": account,
"branch": normalized["branch"],
"path": normalized["path"],
"request_digest": request_digest,
"expected_head_oid": expected_head,
"status": status,
"operation_id": result.get("operation_id"),
"commit_oid": result.get("commit_oid"),
"error": result.get("error"),
"secrets_included": False,
}
append_audit(audit)
if status != 200:
try:
authorization_call("fail_write", result={"operation_id": normalized["operation_id"]})
except RouteError:
pass
return self.send_json(status, result)
advanced = authorization_call("reconcile_commit", result={"previous_head_oid": expected_head, "new_commit_oid": result.get("commit_oid"), "operation_id": result.get("operation_id")})
return self.send_json(
200,
{
"status": "committed",
"branch": normalized["branch"],
"path": normalized["path"],
"commit": {
"status": "committed",
"branch": normalized["branch"],
"rev": result.get("commit_oid"),
"previous_head_oid": expected_head,
"new_tree_oid": result.get("new_tree_oid"),
"push_status": result.get("push_status"),
"four_way_ref_status": advanced.get("four_way_ref_status") is True,
"sync": {"status": "verified", "target": "bare+git-ls-remote+gitea-api"},
},
"writer_instance_id": binding["writer_instance_id"],
"operation_id": result.get("operation_id"),
"idempotent": result.get("idempotent"),
"secrets_included": False,
},
)
except RouteError as exc:
if exc.code == "writer_route_action_identity_rejected":
try:
append_audit(
{
"at": utc_now(),
"task_id": EXPECTED_TASK_ID,
"event": "action_identity_rejected",
"accepted_action_identity": exc.details.get(
"accepted_action_identity"
),
"observed_action_identity": exc.details.get(
"observed_action_identity"
),
"identity_alias_accepted": False,
"secrets_included": False,
}
)
except Exception:
pass
self.send_json(exc.status, {"error": exc.code, **exc.details})
except Exception:
self.send_json(500, {"error": "writer_route_internal_error"})
def main() -> None:
load_binding()
server = ThreadingHTTPServer((HOST, PORT), Handler)
server.serve_forever()
if __name__ == "__main__":
main()
+119 -8
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
from __future__ import annotations
import hashlib, hmac, json, os, re, subprocess, sys, time, urllib.parse, uuid
import hashlib, hmac, json, os, re, subprocess, sys, time, urllib.parse, urllib.request, uuid
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from typing import Any
@@ -22,6 +22,7 @@ from gitea_tag_publication import (
TagPublisher,
redact,
validate_version,
validate_gitea_api_url,
)
from orchestrator.action_monitor import ActionMonitor
from silent_release_hardening import (
@@ -33,6 +34,11 @@ from silent_release_hardening import (
SignedReleasePlanApproval,
SilentReleaseActionService,
)
try:
from aios_gitops_job_controller import GitOpsJobController, JobError, ServiceUnavailable, JobStore
except ModuleNotFoundError:
sys.path.insert(0, str(Path(__file__).resolve().parent))
from aios_gitops_job_controller import GitOpsJobController, JobError, ServiceUnavailable, JobStore
try:
from authoritative_git import (
AuthoritativeGitError,
@@ -120,6 +126,12 @@ def reject_non_read_http(handler, method: str, path: str) -> bool:
return False
if path in SILENT_RELEASE_POST_PATHS:
return False
if (
method == 'POST'
and path in ('/api/file', '/api/commit-file')
and os.getenv('AIOS_GITOPS_DURABLE_JOB_EXECUTION_ENABLED', os.getenv('AIOS_GITOPS_JOB_CONTROLLER_ENABLED', 'false')).lower() in {'1', 'true', 'yes', 'on'}
):
return False
decision = evaluate_git_write(WS)
if decision.allowed:
return False
@@ -297,8 +309,7 @@ def resolve_commit_oid(value: str | None, field: str = 'target_oid') -> str:
raise ValueError(f'{field} does not resolve to the exact approved commit') from exc
def existing_tag_target(tag: str) -> str | None:
if not re.fullmatch(r'V\d+R\d+C\d+(?:-[A-Za-z0-9_.-]+)?$', str(tag or '')):
raise ValueError('invalid release version')
validate_version(tag)
try:
result = run_authoritative_git(
['rev-parse', '--verify', f'refs/tags/{tag}^{{commit}}'],
@@ -333,7 +344,7 @@ def required_authoritative_git_dir() -> Path:
def publication_publisher() -> TagPublisher:
authoritative_repo = required_authoritative_git_dir()
return TagPublisher(
publisher = TagPublisher(
WS,
GitTagBackend(WS, authoritative_repo=authoritative_repo),
GiteaTagClient(
@@ -344,6 +355,10 @@ def publication_publisher() -> TagPublisher:
),
os.getenv('AIOS_GITOPS_GITEA_REMOTE','gitea'),
)
publisher.baseline_path = Path(os.getenv(
'AIOS_GITOPS_BASELINE_AUTHORITY', str(WS / 'baseline' / 'current.json')
))
return publisher
def commit(paths, msg):
ensure_git(); paths = [relpath(p) for p in paths] if paths else ['.']; sh(['add'] + paths, True)
@@ -393,6 +408,51 @@ class H(BaseHTTPRequestHandler):
if not a['ok']: self.sendj({'error':'unauthorized','hint':'Bearer token required'},401); return None
if '*' not in a['scopes'] and scope not in a['scopes']: self.sendj({'error':'forbidden','required_scope':scope,'account':a['account']},403); return None
return a
def bridge_release(self, path, data):
route = {
'/createTagApprovalRequest': '/v2/tag/approval-request',
'/requestOrCreateTag': '/v2/tag/publish',
'/getTagPublicationStatus': '/v2/tag/status',
'/buildProductionArtifact': '/v2/release/artifact',
'/executeApprovedProductionRelease': '/v2/release/execute',
'/getReleaseConsistency': '/v2/release/consistency',
'/runReleaseAcceptance': '/v2/release/acceptance',
}.get(path)
if route is None:
raise ValueError('governed_release_route_invalid')
base = os.getenv('AIOS_BRIDGE_V2_URL', 'http://127.0.0.1:18892').rstrip('/')
request = urllib.request.Request(
base + route,
data=json.dumps(data, sort_keys=True, separators=(',', ':')).encode('utf-8'),
headers={'Content-Type': 'application/json'},
method='POST',
)
try:
with urllib.request.urlopen(request, timeout=30) as response:
result = json.loads(response.read().decode('utf-8'))
return self.sendj(result, response.status)
except urllib.error.HTTPError as exc:
try:
body = json.loads(exc.read().decode('utf-8'))
except Exception:
body = {'error': 'bridge_http_error'}
return self.sendj(body, exc.code)
except (urllib.error.URLError, TimeoutError) as exc:
return self.sendj({'error': 'GOVERNED_BRIDGE_UNAVAILABLE', 'detail': type(exc).__name__}, 503)
def do_POST(self):
path = urllib.parse.urlparse(self.path).path
if path in {
'/createTagApprovalRequest', '/requestOrCreateTag', '/getTagPublicationStatus',
'/buildProductionArtifact', '/executeApprovedProductionRelease',
'/getReleaseConsistency', '/runReleaseAcceptance',
}:
actor = self.need('tag:create')
if not actor:
return
return self.bridge_release(path, self.body())
if reject_non_read_http(self, 'POST', path):
return
return self.sendj({'error': 'not_found', 'path': path}, 404)
def do_GET(self):
u=urllib.parse.urlparse(self.path); path=u.path.rstrip('/') if u.path != '/' else u.path
if path not in READ_ONLY_GET_PATHS: return self.sendj({'error':'not_found','path':path},404)
@@ -454,7 +514,15 @@ class H(BaseHTTPRequestHandler):
return self.sendj({'error':redact(exc)},413)
except (AuthoritativeGitError, RuntimeError) as exc:
return self.sendj({'error':redact(exc)},503)
return self.sendj({'from':frm,'to':to,'diff':diff})
payload = {'from': frm, 'to': to, 'diff': diff}
payload.update({
key: diff[key]
for key in (
'from_oid', 'to_oid', 'changed_paths', 'stats',
'total_size_bytes', 'truncated', 'patch_limit_bytes',
)
})
return self.sendj(payload)
if path in ('/api/patch','/api/patches','/api/branches','/patch'):
snapshot = repository_snapshot(
git_dir=AUTHORITATIVE_GIT_DIR,
@@ -566,7 +634,8 @@ class H(BaseHTTPRequestHandler):
return self.sendj(
service.get_capability_status(candidate_id)
)
return self.sendj(service.get_release_saga(candidate_id))
result = service.get_release_saga(candidate_id)
return self.sendj(result, 404 if result.get('status') == 'NOT_FOUND' else 200)
if path == '/getRecoveryStatus':
return self.sendj(
silent_release_service().get_recovery_status()
@@ -666,7 +735,44 @@ class H(BaseHTTPRequestHandler):
def file(self,data):
a=self.need('file:write');
if not a: return
b=checkout(data.get('branch'), data.get('base','main'), bool(data.get('create_branch'))); r=write_file(data.get('path'), data.get('content','')); c=commit([r], data.get('message') or f'chore(chatgpt): update {r}'); audit(a['account'],'file',r,{'branch':b,'commit':c}); self.sendj({'status':'committed','branch':b,'path':r,'commit':c})
if os.getenv('AIOS_GITOPS_DURABLE_JOB_EXECUTION_ENABLED', os.getenv('AIOS_GITOPS_JOB_CONTROLLER_ENABLED', 'false')).lower() not in {'1','true','yes','on'}:
return self.sendj({'error':'DURABLE_JOB_CONTROLLER_UNAVAILABLE','retryable':True},503)
try:
path = relpath(str(data.get('path') or ''))
branch = str(data.get('branch') or '')
expected = str(data.get('expected_branch_oid') or '')
campaign_id = str(data.get('campaign_id') or os.getenv('AIOS_GITOPS_CAMPAIGN_ID') or '')
supplied_digest = str(data.get('payload_digest') or '')
content = data.get('content')
payload_bytes = None
if content is not None:
payload_bytes = content.encode('utf-8') if isinstance(content, str) else json.dumps(content, sort_keys=True, ensure_ascii=False).encode('utf-8')
payload_digest = supplied_digest or hashlib.sha256(payload_bytes or b'').hexdigest()
if not campaign_id or not branch or not expected:
return self.sendj({'error':'JOB_BINDING_REQUIRED','retryable':False},409)
request = {
'campaign_id': campaign_id,
'repository': REPOSITORY_IDENTITY or 'aios/aios',
'operation': 'commit-file',
'branch': branch,
'expected_branch_oid': expected,
'payload_digest': payload_digest,
# The controller persists only the binding and digest. Raw
# content is never copied into durable state or logs.
}
receipt = GitOpsJobController(
JobStore(Path(os.getenv('AIOS_GITOPS_JOB_ROOT', '/var/lib/aios-gitops/jobs')))
).submit(request, payload=payload_bytes)
return self.sendj({
'job_id': receipt.job_id,
'state': receipt.state.value,
'side_effect_state': receipt.side_effect_state,
'retryable': receipt.http_status == 503,
}, receipt.http_status)
except ServiceUnavailable as exc:
return self.sendj({'error':str(exc),'layer':'durable_job','retryable':True,'side_effect_state':'UNKNOWN'},503)
except JobError as exc:
return self.sendj({'error':str(exc),'retryable':False},409)
def commit(self,data):
a=self.need('commit:create');
if not a: return
@@ -680,7 +786,10 @@ class H(BaseHTTPRequestHandler):
a=self.need('tag:create');
if not a: return
ver=data.get('version') or data.get('tag')
if not re.match(r'^V\d+R\d+C\d+(-[A-Za-z0-9_.-]+)?$', ver or ''): return self.sendj({'error':'version must match VxRxCx'},400)
try:
validate_version(ver)
except ValueError:
return self.sendj({'error':'version must match VxRxCx'},400)
if not data.get('target_oid') or not data.get('evidence_closure_oid') or not data.get('approval_plan_id'):
return self.sendj({
'error':'production_tag_evidence_required',
@@ -854,6 +963,8 @@ def api_spec():
return {'version':'V1R1C0-chatgpt-actions','openapi':'/api/openapi.yaml','read':['GET /api/base','GET /api/ver','GET /api/diff','GET /api/patch','GET /api/model','GET /api/run','GET /getReleaseCandidate','GET /getApprovalEnvelope','GET /getReleaseSaga','GET /admin/ai-evolution'],'write':['POST /api/branch','POST /api/file','POST /api/commit','POST /api/patch','POST /api/tag','POST /api/rollback','POST /api/model','PUT /api/model-priority','POST /api/nl','POST /createReleaseCandidate','POST /submitOwnerApproval','POST /startDeployment'],'policy':{'bearer_required_for_write':True,'direct_main_push':False,'delete_tag':False,'owner_approval_required_for_v_level':True,'api_keys_in_git':False,'git_secret_field':'secret_ref','online_git_sync':True,'silent_release_execution_enabled':SILENT_RELEASE_EXECUTION_ENABLED,'silent_release_shadow_default':True}}
def main():
validate_gitea_api_url(os.getenv('AIOS_GITOPS_GITEA_API_URL', ''))
print(f'AIOS ChatGPT GitOps adapter listening on {HOST}:{PORT}')
ThreadingHTTPServer((HOST,PORT), H).serve_forever()
if __name__ == '__main__': main()
@@ -0,0 +1,243 @@
"""ChatGPT GitOps Bridge v2 contract and governed release operations."""
from __future__ import annotations
from dataclasses import dataclass
import hashlib
from pathlib import Path
from typing import Any, Callable, Mapping
BRIDGE_SCHEMA_VERSION = "aios.chatgpt-gitops-bridge.v2"
BRIDGE_OPERATIONS = (
"getRuntimeStatus",
"getGovernedWriteCapabilities",
"createGovernedTask",
"projectOnlineAgentExecution",
"acquireWriterLease",
"commitFileGoverned",
"commitFilesGoverned",
"getGitOpsJob",
"resumeGitOpsJob",
"reconcileGitOpsJob",
"createTagApprovalRequest",
"requestOrCreateTag",
"getTagPublicationStatus",
"buildProductionArtifact",
"executeApprovedProductionRelease",
"getReleaseConsistency",
"runReleaseAcceptance",
"getControlPlaneSnapshot",
)
OPERATION_REQUIRED_FIELDS = {
"createGovernedTask": ("task_id", "repository", "base_oid", "target_branch", "risk", "description", "idempotency_key"),
"projectOnlineAgentExecution": ("task_id", "required_capabilities"),
"acquireWriterLease": ("schema", "task_id", "repository", "branch_mode", "base_oid", "expected_branch_oid", "target_branch", "generation", "lease_seconds"),
"commitFileGoverned": ("task_id", "repository", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "path", "content_base64", "message"),
"commitFilesGoverned": ("task_id", "repository", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "files", "message"),
"getGitOpsJob": ("job_id",),
"resumeGitOpsJob": ("job_id", "expected_revision"),
"reconcileGitOpsJob": ("job_id", "expected_revision"),
"createTagApprovalRequest": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id", "owner_approval", "execute"),
"requestOrCreateTag": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id", "execute"),
"getTagPublicationStatus": ("version",),
"buildProductionArtifact": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id"),
"executeApprovedProductionRelease": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id", "execute"),
"getReleaseConsistency": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id"),
"runReleaseAcceptance": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id"),
"getControlPlaneSnapshot": (),
}
_OPERATION_PATHS = {
"createGovernedTask": "/v2/governed-task",
"projectOnlineAgentExecution": "/v2/execution-envelope",
"acquireWriterLease": "/v2/writer-lease",
"commitFileGoverned": "/v2/commit-file",
"commitFilesGoverned": "/v2/commit-files",
"getGitOpsJob": "/v2/job",
"resumeGitOpsJob": "/v2/job/resume",
"reconcileGitOpsJob": "/v2/job/reconcile",
"getTagPublicationStatus": "/v2/tag/status",
"getReleaseConsistency": "/v2/release/consistency",
"runReleaseAcceptance": "/v2/release/acceptance",
}
def operation_contracts() -> dict[str, dict[str, object]]:
"""Return the runtime contract consumed by adapters and schema tooling."""
return {
operation: {
"method": "POST",
"path": _OPERATION_PATHS.get(operation),
"required_fields": list(fields),
}
for operation, fields in OPERATION_REQUIRED_FIELDS.items()
}
def render_openapi_contract_fragment() -> str:
required = "\n".join(
f" - {field}"
for field in OPERATION_REQUIRED_FIELDS["commitFilesGoverned"]
)
return (
" /v2/commit-files:\n"
" post:\n"
" operationId: commitFilesGoverned\n"
" summary: Submit one atomic governed commit containing multiple files\n"
" requestBody:\n"
" required: true\n"
" content:\n"
" application/json:\n"
" schema:\n"
" type: object\n"
" additionalProperties: false\n"
" required:\n"
f"{required}\n"
" properties:\n"
" task_id: {type: string}\n"
" job_id: {type: string, description: External durable Job identity for exact completed-replay reconciliation}\n"
" source_lease_epoch: {type: integer, minimum: 1}\n"
" repository: {type: string, example: aios/aios}\n"
" expected_branch_oid: {type: string, pattern: '^[0-9a-f]{40}$'}\n"
" target_branch: {type: string, pattern: '^agent/[A-Za-z0-9._/-]+$'}\n"
" generation: {type: integer, minimum: 1}\n"
" idempotency_key: {type: string}\n"
" files:\n"
" type: array\n"
" minItems: 1\n"
" items:\n"
" type: object\n"
" additionalProperties: false\n"
" required: [path, content_base64]\n"
" properties:\n"
" path: {type: string, minLength: 1, maxLength: 512}\n"
" content_base64: {type: string}\n"
" message: {type: string, minLength: 1, maxLength: 200}\n"
" responses:\n"
" '200': {description: Governed commit job and verified branch result}\n"
" '400': {description: Contract or policy rejection}\n"
" '409': {description: Branch CAS or idempotency conflict}\n"
)
class RuntimeDriftError(RuntimeError):
pass
class BridgeV2Error(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class ControlPlaneReleaseManifest:
control_plane_release_version: str
control_plane_target_oid: str
artifact_sha256: str
runtime_path: Path
bridge_schema_version: str
repository_identity: str
authoritative_git_dir: Path
deployment_state: str
def as_dict(self) -> dict[str, object]:
return {
"control_plane_release_version": self.control_plane_release_version,
"control_plane_target_oid": self.control_plane_target_oid,
"artifact_sha256": self.artifact_sha256,
"runtime_path": str(self.runtime_path),
"bridge_schema_version": self.bridge_schema_version,
"repository_identity": self.repository_identity,
"authoritative_git_dir": str(self.authoritative_git_dir),
"deployment_state": self.deployment_state,
}
def validate(self, artifact: Path, *, observed_oid: str, approved_root: Path = Path("/opt/aios/gitops-control-plane")) -> dict[str, object]:
try:
runtime = self.runtime_path.resolve(strict=True)
root = approved_root.resolve(strict=True)
artifact_path = artifact.resolve(strict=True)
runtime.relative_to(root)
artifact_path.relative_to(runtime)
except (OSError, ValueError):
raise RuntimeDriftError("CONTROL_PLANE_RUNTIME_DRIFT") from None
mutable_markers = {"workspace", "workspaces", "worktree", "worktrees"}
if mutable_markers.intersection(part.lower() for part in runtime.parts):
raise RuntimeDriftError("CONTROL_PLANE_RUNTIME_DRIFT")
valid = (
len(self.control_plane_target_oid) == 40
and observed_oid == self.control_plane_target_oid
and len(self.artifact_sha256) == 64
and hashlib.sha256(artifact_path.read_bytes()).hexdigest() == self.artifact_sha256
and self.bridge_schema_version == BRIDGE_SCHEMA_VERSION
and self.repository_identity == "aios/aios"
and self.authoritative_git_dir == Path("/data/aios-gitops/repo.git")
and self.authoritative_git_dir.is_absolute()
and self.deployment_state == "STAGING"
)
if not valid:
raise RuntimeDriftError("CONTROL_PLANE_RUNTIME_DRIFT")
return {**self.as_dict(), "drift_status": "PASS"}
class BridgeV2Adapter:
"""Thin operation adapter; all mutating handlers are injected seams."""
def __init__(self, *, runtime_status: Mapping[str, Any], handlers: Mapping[str, Callable[[Mapping[str, Any]], Any]] | None = None) -> None:
self.runtime_status = dict(runtime_status)
self.handlers = dict(handlers or {})
def getRuntimeStatus(self) -> dict[str, Any]:
return {"bridge_schema_version": BRIDGE_SCHEMA_VERSION, **self.runtime_status}
def getGovernedWriteCapabilities(self) -> dict[str, Any]:
return {
"bridge_schema_version": BRIDGE_SCHEMA_VERSION,
"legacy_writes_enabled": False,
"governed_writes_enabled": True,
"operations": list(BRIDGE_OPERATIONS),
"operation_contracts": operation_contracts(),
"force_push": False,
"main_write": False,
"tag_write": False,
"baseline_write": False,
"production_write": False,
"approved_release_operations": True,
"approved_release_scope": {
"version": "V2R2C1",
"target_oid": "998d30304787e31e5ae607c0bd86b5edb2b0e5ad",
"evidence_closure_oid": "a8b906d1bc24088c19647e98de6a679268f6d0a2",
"approval_plan_id": "sha256:c0df52008ae7f64cec5ee2d28947aa7b892066c670ab8295e4549140b1a7500e",
},
}
@staticmethod
def legacy_write_disabled() -> dict[str, object]:
return {"error": "LEGACY_WRITE_DISABLED", "replacement": "createGovernedTask", "governed_writes_available": True}
def _call(self, operation: str, request: Mapping[str, Any]) -> Any:
required = OPERATION_REQUIRED_FIELDS[operation]
if any(key not in request for key in required):
raise BridgeV2Error("BRIDGE_REQUEST_FIELDS_MISSING")
handler = self.handlers.get(operation)
if handler is None:
raise BridgeV2Error("GOVERNED_HANDLER_UNAVAILABLE")
return handler(dict(request))
def createGovernedTask(self, request: Mapping[str, Any]) -> Any: return self._call("createGovernedTask", request)
def projectOnlineAgentExecution(self, request: Mapping[str, Any]) -> Any: return self._call("projectOnlineAgentExecution", request)
def acquireWriterLease(self, request: Mapping[str, Any]) -> Any: return self._call("acquireWriterLease", request)
def commitFileGoverned(self, request: Mapping[str, Any]) -> Any: return self._call("commitFileGoverned", request)
def commitFilesGoverned(self, request: Mapping[str, Any]) -> Any: return self._call("commitFilesGoverned", request)
def getGitOpsJob(self, request: Mapping[str, Any]) -> Any: return self._call("getGitOpsJob", request)
def resumeGitOpsJob(self, request: Mapping[str, Any]) -> Any: return self._call("resumeGitOpsJob", request)
def reconcileGitOpsJob(self, request: Mapping[str, Any]) -> Any: return self._call("reconcileGitOpsJob", request)
def createTagApprovalRequest(self, request: Mapping[str, Any]) -> Any: return self._call("createTagApprovalRequest", request)
def requestOrCreateTag(self, request: Mapping[str, Any]) -> Any: return self._call("requestOrCreateTag", request)
def getTagPublicationStatus(self, request: Mapping[str, Any]) -> Any: return self._call("getTagPublicationStatus", request)
def buildProductionArtifact(self, request: Mapping[str, Any]) -> Any: return self._call("buildProductionArtifact", request)
def executeApprovedProductionRelease(self, request: Mapping[str, Any]) -> Any: return self._call("executeApprovedProductionRelease", request)
def getReleaseConsistency(self, request: Mapping[str, Any]) -> Any: return self._call("getReleaseConsistency", request)
def runReleaseAcceptance(self, request: Mapping[str, Any]) -> Any: return self._call("runReleaseAcceptance", request)
def getControlPlaneSnapshot(self, request: Mapping[str, Any] | None = None) -> Any: return self._call("getControlPlaneSnapshot", request or {})
@@ -0,0 +1,831 @@
"""Loopback-only governed Bridge v2 HTTP gateway with durable handlers."""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
import base64
import hashlib
import json
import os
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
import re
import sqlite3
from typing import Any, Mapping
from aios_authority_git_executor import AuthorityGitExecutor
from aios_chatgpt_bridge_v2 import BridgeV2Adapter, BridgeV2Error, OPERATION_REQUIRED_FIELDS
from gitea_tag_publication import validate_gitea_api_url
from aios_continuation_executor import AuthorityGitAdapter, ContinuationCommit, ContinuationExecutor, ContinuationPermissions, ContinuationRuntimeBinding, DetachedGitWorkspace, GiteaContinuationRemote, ServerSideCredentialProjection
from aios_continuation_lease import WriterLeaseRequestV2
from aios_control_plane_snapshot import build_control_plane_snapshot
from aios_gitops_job_controller import GitOpsJobController, JobState, JobStore, _RECOVERABLE, sha256_json
from aios_governed_task_bootstrap import BootstrapError, GovernedTaskBootstrap
from governed_production_release import build_executor_config
MAX_BODY = 2 * 1024 * 1024
ROUTES = {
"/v2/runtime": "getRuntimeStatus",
"/v2/capabilities": "getGovernedWriteCapabilities",
"/v2/governed-task": "createGovernedTask",
"/v2/execution-envelope": "projectOnlineAgentExecution",
"/v2/writer-lease": "acquireWriterLease",
"/v2/commit-file": "commitFileGoverned",
"/v2/commit-files": "commitFilesGoverned",
"/v2/job": "getGitOpsJob",
"/v2/job/resume": "resumeGitOpsJob",
"/v2/job/reconcile": "reconcileGitOpsJob",
"/v2/tag/approval-request": "createTagApprovalRequest",
"/v2/tag/publish": "requestOrCreateTag",
"/v2/tag/status": "getTagPublicationStatus",
"/v2/release/artifact": "buildProductionArtifact",
"/v2/release/execute": "executeApprovedProductionRelease",
"/v2/release/consistency": "getReleaseConsistency",
"/v2/release/acceptance": "runReleaseAcceptance",
}
@dataclass(frozen=True, slots=True)
class RuntimeConfig:
target_oid: str
authoritative_git_dir: Path
state_root: Path
v2_job_root: Path
token_path: Path
ssh_key_path: Path
remote_name: str = "gitea-http"
repository: str = "aios/aios"
api_base: str = ""
v2_task_id: str = "TASK-P0-CHATGPT-BRIDGE-V2-CONTROL-PLANE-CONVERGENCE-20260809"
v2_branch: str = "agent/p0-chatgpt-bridge-v2-control-plane-convergence-20260809"
target_version: str = "V2R2C1"
registry_root: Path = Path("/var/lib/aios-gitops/runtime/registry-projection")
baseline_path: Path = Path("/var/lib/aios-gitops-runtime-anchor/baseline/current.json")
health_url: str = "http://127.0.0.1:18927/health"
release_plan_path: Path | None = None
def _safe(value: str) -> str:
return re.sub(r"[^A-Za-z0-9_.-]+", "-", value).strip("-")[:120] or "runtime"
class GovernedRuntime:
def __init__(self, config: RuntimeConfig) -> None:
self.config = config
config.state_root.mkdir(parents=True, exist_ok=True)
self.authority = AuthorityGitExecutor(config.authoritative_git_dir, config.state_root / "worktrees")
self.bootstrap = GovernedTaskBootstrap(self.authority, config.state_root, remote_name=config.remote_name, token_path=config.token_path, ssh_key_path=config.ssh_key_path, api_base=config.api_base)
self.runtimes: dict[tuple[str, int], tuple[Any, GitOpsJobController, ContinuationExecutor]] = {}
self.controllers: dict[str, GitOpsJobController] = {}
self.bindings: dict[str, dict[str, Any]] = {}
self.release = build_executor_config(config)
metadata_root = config.state_root / "task-bindings"
if metadata_root.is_dir():
for metadata in metadata_root.glob("*.json"):
try:
value = json.loads(metadata.read_text(encoding="utf-8"))
if isinstance(value, dict) and value.get("task_id"):
self.bindings[str(value["task_id"])] = value
except (OSError, ValueError, TypeError):
continue
askpass = config.state_root / "bridge-v2-askpass.sh"
if not askpass.exists():
askpass.write_text('#!/bin/sh\ncase "$1" in *Username*) printf "%s\\n" "$GITEA_MACHINE_ACTOR" ;; *) cat "$CREDENTIAL_FILE" ;; esac\n', encoding="utf-8")
askpass.chmod(0o700)
os.environ.setdefault("GIT_ASKPASS", str(askpass)); os.environ.setdefault("GIT_ASKPASS_REQUIRE", "force"); os.environ.setdefault("GIT_TERMINAL_PROMPT", "0"); os.environ.setdefault("GITEA_MACHINE_ACTOR", "aios-autonomy-agent")
def runtime_status(self) -> dict[str, Any]:
return {
"status": "active",
"health": 200,
"bridge_schema_version": "aios.chatgpt-gitops-bridge.v2",
"control_plane_release_version": self.config.target_version,
"control_plane_target_oid": self.config.target_oid,
"runtime_release_commit": self.config.target_oid,
"runtime_path": str(self.config.state_root),
"artifact_sha256": os.environ.get("AIOS_CONTROL_PLANE_ARTIFACT_SHA256"),
"deployment_state": "STAGING",
"drift_status": "PASS",
"repository": self.config.repository,
"repository_identity": self.config.repository,
"authoritative_git_dir": str(self.config.authoritative_git_dir),
"legacy_global_writes_enabled": False,
}
def _job_root(self, task_id: str) -> Path:
return self.config.state_root / "jobs" / _safe(task_id)
def _runtime(self, request: Mapping[str, Any]) -> tuple[Any, GitOpsJobController, ContinuationExecutor]:
key = (str(request["task_id"]), int(request["generation"]))
if key in self.runtimes: return self.runtimes[key]
lease = WriterLeaseRequestV2(task_id=str(request["task_id"]), repository=str(request["repository"]), branch_mode=str(request.get("branch_mode", "CONTINUE_EXISTING")), base_oid=str(request["base_oid"]), expected_branch_oid=str(request["expected_branch_oid"]), target_branch=str(request["target_branch"]), generation=int(request["generation"]), lease_seconds=int(request.get("lease_seconds", 900)))
original_branch_mode = lease.branch_mode
if lease.branch_mode == "CREATE_NEW":
try:
lease.validate(AuthorityGitAdapter(self.authority))
except Exception as exc:
# A request replay after the authority CAS may observe the
# already-created branch. Only the exact base identity is
# idempotent; all other validation failures remain fail-closed.
if str(exc) != "TARGET_BRANCH_ALREADY_EXISTS":
raise
# Bootstrap owns the durable binding; the runtime performs the one-time
# branch materialization through the authority CAS before continuation
# validation. No direct Git or legacy writer path is used.
self.authority.create_task_branch(lease.target_branch, lease.base_oid)
lease = WriterLeaseRequestV2(
task_id=lease.task_id,
repository=lease.repository,
branch_mode="CONTINUE_EXISTING",
base_oid=lease.base_oid,
expected_branch_oid=lease.expected_branch_oid,
target_branch=lease.target_branch,
generation=lease.generation,
lease_seconds=lease.lease_seconds,
)
lease.validate(AuthorityGitAdapter(self.authority))
worktree_path = self.config.state_root / "worktrees" / _safe(lease.task_id)
if worktree_path.exists():
workspace = DetachedGitWorkspace(worktree_path)
else:
self.authority.create_detached_worktree(lease.expected_branch_oid, _safe(lease.task_id), self.config.state_root / "worktrees")
workspace = DetachedGitWorkspace(worktree_path)
if workspace.head_oid() != lease.expected_branch_oid or not workspace.is_detached(): raise RuntimeError("WORKTREE_HEAD_MISMATCH")
job_root = self._job_root(lease.task_id); controller = GitOpsJobController(JobStore(job_root, payload_root=job_root / "payloads")); credential = ServerSideCredentialProjection(self.config.ssh_key_path, self.config.token_path); remote = GiteaContinuationRemote(workspace=worktree_path, credential=credential, remote_name=self.config.remote_name, repository=lease.repository, api_base=self.config.api_base); executor = ContinuationExecutor(controller, AuthorityGitAdapter(self.authority), workspace, remote, credential)
binding = ContinuationRuntimeBinding(request=lease, workspace_path=worktree_path, lease_not_before=datetime.now(timezone.utc) - timedelta(seconds=1), lease_expires_at=datetime.now(timezone.utc) + timedelta(seconds=lease.lease_seconds), original_receipt_valid=True, original_grant_consumed=True, permissions=ContinuationPermissions(), remote_branch_create_allowed=original_branch_mode == "CREATE_NEW", external_job_id=str(request.get("job_id") or "") or None)
result = (binding, controller, executor); self.runtimes[key] = result; self.controllers[lease.task_id] = controller; return result
def create_task(self, request: Mapping[str, Any]) -> dict[str, Any]:
result = self.bootstrap.create(request); self.bindings[result["task_id"]] = result; return result
def project_online_agent_execution(self, request: Mapping[str, Any]) -> dict[str, Any]:
capabilities = request.get("required_capabilities") or []
if not isinstance(capabilities, list) or not all(isinstance(item, str) for item in capabilities):
raise ValueError("CAPABILITY_SCOPE_INVALID")
execution = request.get("execution")
if execution is not None and not isinstance(execution, Mapping):
raise ValueError("EXECUTION_REQUEST_INVALID")
task_id = str(request["task_id"])
expected_job_id = str(request.get("job_id") or "") or None
return self.bootstrap.execution_envelope(
task_id,
execution,
required_capabilities=tuple(capabilities),
expected_job_id=expected_job_id,
)
result = self.bootstrap.create(request); self.bindings[result["task_id"]] = result; return result
def acquire_lease(self, request: Mapping[str, Any]) -> dict[str, Any]:
replay = self._durable_completed_replay_lease(request)
if replay is not None:
return replay
binding, _, _ = self._runtime(request)
req = binding.request
return {"schema": WriterLeaseRequestV2.schema, "task_id": req.task_id, "repository": req.repository, "branch_mode": req.branch_mode, "base_oid": req.base_oid, "expected_branch_oid": req.expected_branch_oid, "target_branch": req.target_branch, "generation": req.generation, "lease_seconds": req.lease_seconds, "create_target_branch": req.branch_mode == "CREATE_NEW", "worktree_mode": "detached_at_base_oid" if req.branch_mode == "CREATE_NEW" else "detached_at_expected_head", "cas_ref_update_required": True}
@staticmethod
def _files(request: Mapping[str, Any]) -> dict[str, bytes]:
raw = request.get("files") or {request["path"]: request["content_base64"]}; pairs = raw.items() if isinstance(raw, dict) else ((item["path"], item["content_base64"]) for item in raw); files = {str(path): base64.b64decode(encoded, validate=True) for path, encoded in pairs}
if not files: raise ValueError("FILES_EMPTY")
return files
def commit_files(self, request: Mapping[str, Any]) -> dict[str, Any]:
replay = self._durable_completed_replay(request)
if replay is not None:
return replay
runtime = self.runtimes.get((str(request["task_id"]), int(request["generation"])))
if runtime is None: raise RuntimeError("LEASE_NOT_FOUND")
binding, _, executor = runtime
if request["target_branch"] != binding.request.target_branch or request["expected_branch_oid"] != binding.request.expected_branch_oid: raise RuntimeError("LEASE_BINDING_CONFLICT")
commit = ContinuationCommit(idempotency_key=str(request["idempotency_key"]), files=self._files(request), message=str(request["message"]))
executor.workspace.materialize_files(commit.files)
return executor.execute(binding, commit)
def _find_completed_replay_job(self, external_job_id: str) -> tuple[Path, dict[str, Any]] | None:
matches: list[tuple[Path, dict[str, Any]]] = []
roots = [self.config.state_root / "jobs"]
for controller in self.controllers.values():
roots.append(controller.store.root.parent)
seen: set[str] = set()
for jobs_root in roots:
if not jobs_root.is_dir():
continue
candidates = []
if (jobs_root / "state.sqlite3").is_file():
candidates.append(jobs_root)
candidates.extend(root for root in jobs_root.iterdir() if root.is_dir())
for root in candidates:
db = root / "state.sqlite3"
key = str(db.resolve()) if db.exists() else str(db)
if key in seen:
continue
seen.add(key)
if not db.is_file() or db.is_symlink():
continue
try:
connection = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
connection.row_factory = sqlite3.Row
try:
rows = connection.execute("SELECT * FROM jobs").fetchall()
finally:
connection.close()
for raw in rows:
row = dict(raw)
durable = json.loads(row["request_json"])
binding = durable.get("binding", durable)
if str(binding.get("external_job_id") or "") == external_job_id:
matches.append((root, row))
except (OSError, ValueError, TypeError, sqlite3.Error):
continue
if not matches:
return None
if len(matches) != 1:
raise RuntimeError("EXTERNAL_JOB_MAPPING_AMBIGUOUS")
return matches[0]
def _durable_completed_replay_lease(self, request: Mapping[str, Any]) -> dict[str, Any] | None:
"""Admit an exact completed replay before ordinary lease validation."""
external_job_id = str(request.get("job_id") or "").strip()
if not external_job_id:
if request.get("source_lease_epoch") is not None:
raise RuntimeError("COMPLETED_REPLAY_EXTERNAL_JOB_ID_REQUIRED")
return None
match = self._find_completed_replay_job(external_job_id)
if match is None:
return None
root, row = match
if str(row["state"]) != JobState.COMPLETED.value:
return None
db = root / "state.sqlite3"
connection = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
connection.row_factory = sqlite3.Row
try:
evidence_rows = connection.execute(
"SELECT * FROM completed_replay_reconciliations WHERE job_id=? ORDER BY created_at ASC",
(row["job_id"],),
).fetchall()
finally:
connection.close()
if not evidence_rows:
return None
evidence = dict(evidence_rows[0])
immutable = tuple(key for key in evidence if key != "created_at")
if any(any(item[key] != evidence[key] for key in immutable) for item in evidence_rows[1:]):
raise RuntimeError("COMPLETED_REPLAY_AMBIGUOUS")
durable = json.loads(row["request_json"])
binding = durable.get("binding", durable)
target_branch = str(binding.get("target_branch") or binding.get("branch") or "")
for field, incoming in (
("task_id", request.get("task_id")),
("repository", request.get("repository")),
("target_branch", request.get("target_branch")),
("generation", request.get("generation")),
("branch_mode", request.get("branch_mode")),
):
durable_value = target_branch if field == "target_branch" else binding.get(field)
if str(durable_value) != str(incoming):
raise RuntimeError("COMPLETED_REPLAY_BINDING_CONFLICT")
historical_head = str(evidence.get("historical_lease_head_oid") or "")
if str(request.get("base_oid") or "") != historical_head or str(request.get("expected_branch_oid") or "") != historical_head:
raise RuntimeError("COMPLETED_REPLAY_EXPECTED_HEAD_CONFLICT")
if request.get("source_lease_epoch") is not None and int(request["source_lease_epoch"]) != int(evidence["source_lease_epoch"]):
raise RuntimeError("COMPLETED_REPLAY_LEASE_EPOCH_CONFLICT")
receipt = json.loads(row["receipt_json"] or "null")
if not isinstance(receipt, Mapping) or receipt.get("commit_oid") != receipt.get("authority_oid"):
raise RuntimeError("COMPLETED_REPLAY_RECEIPT_INVALID")
authority_head = self.authority.read_ref("refs/heads/" + target_branch)
if authority_head != evidence.get("current_head_oid"):
raise RuntimeError("COMPLETED_REPLAY_AUTHORITY_CONFLICT")
if not (
evidence.get("commit_oid")
== evidence.get("authority_head_oid")
== evidence.get("current_head_oid")
== receipt.get("commit_oid")
and evidence.get("commit_oid") != historical_head
):
raise RuntimeError("COMPLETED_REPLAY_EVIDENCE_INCOHERENT")
return {
"schema": WriterLeaseRequestV2.schema,
"task_id": request["task_id"],
"job_id": external_job_id,
"repository": request["repository"],
"branch_mode": request["branch_mode"],
"base_oid": request["base_oid"],
"expected_branch_oid": request["expected_branch_oid"],
"target_branch": request["target_branch"],
"generation": int(request["generation"]),
"lease_seconds": int(request.get("lease_seconds") or 900),
"create_target_branch": False,
"worktree_mode": "detached_at_expected_head",
"cas_ref_update_required": True,
"lease_created": False,
"completed_replay": True,
"current_head_oid": evidence["current_head_oid"],
"completed_replay_reconciliation_id": evidence["reconciliation_id"],
}
def _durable_completed_replay(self, request: Mapping[str, Any]) -> dict[str, Any] | None:
"""Reconcile an exact completed replay before any worktree mutation."""
external_job_id = str(request.get("job_id") or "").strip()
if not external_job_id:
if request.get("source_lease_epoch") is not None:
raise RuntimeError("COMPLETED_REPLAY_EXTERNAL_JOB_ID_REQUIRED")
return None
matches: list[tuple[Path, dict[str, Any]]] = []
roots = [self.config.state_root / "jobs"]
for controller in self.controllers.values():
roots.append(controller.store.root.parent)
seen: set[str] = set()
for jobs_root in roots:
if not jobs_root.is_dir():
continue
candidates = []
if (jobs_root / "state.sqlite3").is_file():
candidates.append(jobs_root)
candidates.extend(root for root in jobs_root.iterdir() if root.is_dir())
for root in candidates:
db = root / "state.sqlite3"
key = str(db.resolve()) if db.exists() else str(db)
if key in seen:
continue
seen.add(key)
if not db.is_file() or db.is_symlink():
continue
try:
connection = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
connection.row_factory = sqlite3.Row
try:
rows = connection.execute("SELECT * FROM jobs").fetchall()
finally:
connection.close()
for raw in rows:
row = dict(raw)
durable = json.loads(row["request_json"])
binding = durable.get("binding", durable)
if str(binding.get("external_job_id") or "") == external_job_id:
matches.append((root, row))
except (OSError, ValueError, TypeError, sqlite3.Error):
continue
if not matches:
return None
if len(matches) != 1:
raise RuntimeError("EXTERNAL_JOB_MAPPING_AMBIGUOUS")
root, row = matches[0]
if str(row["state"]) != JobState.COMPLETED.value:
return None
durable = json.loads(row["request_json"])
binding = durable.get("binding", durable)
source_lease_epoch = request.get("source_lease_epoch")
if source_lease_epoch is None:
db = root / "state.sqlite3"
connection = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
connection.row_factory = sqlite3.Row
try:
evidence = connection.execute(
"SELECT source_lease_epoch FROM completed_replay_reconciliations WHERE job_id=? ORDER BY created_at ASC LIMIT 1",
(row["job_id"],),
).fetchone()
finally:
connection.close()
if evidence is not None:
source_lease_epoch = int(evidence["source_lease_epoch"])
files = self._files(request)
commit = ContinuationCommit(str(request["idempotency_key"]), files, str(request["message"]))
payload = commit.payload()
receipt = json.loads(row["receipt_json"] or "null")
if not isinstance(receipt, Mapping):
raise RuntimeError("COMPLETED_REPLAY_RECEIPT_INVALID")
incoming = {
"task_id": str(request.get("task_id") or ""),
"external_job_id": external_job_id,
"internal_job_id": str(row["job_id"]),
"repository": str(request.get("repository") or ""),
"target_branch": str(request.get("target_branch") or ""),
"generation": int(request.get("generation") or 0),
"branch_mode": str(request.get("branch_mode") or ""),
"payload_digest": hashlib.sha256(payload).hexdigest(),
"files_digest": hashlib.sha256(json.dumps({path: hashlib.sha256(content).hexdigest() for path, content in sorted(files.items())}, sort_keys=True, separators=(",", ":")).encode("utf-8")).hexdigest(),
"message_digest": hashlib.sha256(str(request["message"]).encode("utf-8")).hexdigest(),
"worker_principal": str(binding.get("worker_principal") or ""),
"worker_lineage": str(binding.get("worker_lineage") or ""),
"source_lease_epoch": int(source_lease_epoch or 0),
"durable_revision": int(row["revision"]),
"completion_identity": sha256_json({"job_id": row["job_id"], "revision": int(row["revision"]), "receipt": receipt}),
"reason": "COMPLETED_REPLAY_LEASE_EVIDENCE_RECONCILIATION",
"provenance": {"source": "governed-completed-replay", "authority_readback": True},
}
for field in ("task_id", "repository", "target_branch", "generation", "branch_mode", "payload_digest", "files_digest", "message_digest", "worker_principal", "worker_lineage"):
durable_value = binding.get(field)
if field == "target_branch":
durable_value = binding.get("target_branch") or binding.get("branch")
elif field in {"worker_principal", "worker_lineage"}:
durable_value = durable_value or ""
if durable_value not in (None, "") and str(durable_value) != str(incoming[field]):
raise RuntimeError("COMPLETED_REPLAY_BINDING_CONFLICT")
db = root / "state.sqlite3"
connection = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
connection.row_factory = sqlite3.Row
try:
columns = {str(item[1]) for item in connection.execute("PRAGMA table_info(branch_leases)").fetchall()}
if "epoch" in columns:
lease = connection.execute("SELECT lease_id, state, current_head_oid, generation FROM branch_leases WHERE job_id=? AND epoch=? ORDER BY rowid DESC LIMIT 1", (row["job_id"], incoming["source_lease_epoch"])).fetchone()
else:
if incoming["source_lease_epoch"] != 1:
raise RuntimeError("COMPLETED_REPLAY_LEASE_EPOCH_UNAVAILABLE")
lease = connection.execute("SELECT lease_id, state, current_head_oid, generation FROM branch_leases WHERE job_id=? ORDER BY rowid DESC LIMIT 1", (row["job_id"],)).fetchone()
if lease is None:
raise RuntimeError("COMPLETED_REPLAY_LEASE_MISSING")
historical_lease_head = str(lease["current_head_oid"])
event_tables = {str(item[0]) for item in connection.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='branch_lease_events'").fetchall()}
lease_state = str(lease["state"])
lease_generation = int(lease["generation"])
if "branch_lease_events" in event_tables:
event = connection.execute("SELECT state, current_head_oid, generation FROM branch_lease_events WHERE lease_id=? ORDER BY rowid DESC LIMIT 1", (lease["lease_id"],)).fetchone()
if event is not None:
lease_state = str(event["state"])
historical_lease_head = str(event["current_head_oid"])
lease_generation = int(event["generation"])
if lease_state not in {"ACTIVE", "USED", "EXPIRED"}:
raise RuntimeError("COMPLETED_REPLAY_LEASE_STATE_INVALID")
if lease_generation != incoming["generation"]:
raise RuntimeError("COMPLETED_REPLAY_LEASE_GENERATION_CONFLICT")
finally:
connection.close()
incoming["historical_lease_head_oid"] = historical_lease_head
authority_head = self.authority.read_ref("refs/heads/" + incoming["target_branch"])
if authority_head is None:
raise RuntimeError("COMPLETED_REPLAY_AUTHORITY_MISSING")
if receipt.get("commit_oid") != receipt.get("authority_oid"):
raise RuntimeError("COMPLETED_REPLAY_RECEIPT_INVALID")
incoming["commit_oid"] = str(receipt["commit_oid"])
incoming["authority_head_oid"] = authority_head
connection = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
connection.row_factory = sqlite3.Row
try:
evidence_rows = connection.execute("SELECT * FROM completed_replay_reconciliations WHERE job_id=? ORDER BY created_at ASC", (row["job_id"],)).fetchall()
finally:
connection.close()
if evidence_rows:
evidence = dict(evidence_rows[0])
immutable = tuple(key for key in evidence if key != "created_at")
if any(any(item[key] != evidence[key] for key in immutable) for item in evidence_rows[1:]):
raise RuntimeError("COMPLETED_REPLAY_AMBIGUOUS")
normalized = JobStore._completed_replay_identity(incoming)
columns = {
"task_id": "task_id",
"external_job_id": "external_job_id",
"internal_job_id": "internal_job_id",
"repository": "repository",
"target_branch": "target_branch",
"generation": "generation",
"branch_mode": "branch_mode",
"payload_digest": "payload_digest",
"files_digest": "files_digest",
"message_digest": "message_digest",
"commit_oid": "commit_oid",
"authority_head_oid": "authority_head_oid",
"historical_lease_head_oid": "historical_lease_head_oid",
"worker_principal": "worker_principal",
"worker_lineage": "worker_lineage",
"source_lease_epoch": "source_lease_epoch",
"durable_revision": "durable_revision",
"completion_identity": "completion_identity",
"reason": "reason",
}
if any(str(evidence[column]) != str(normalized[field]) for field, column in columns.items()):
raise RuntimeError("COMPLETED_REPLAY_BINDING_CONFLICT")
if evidence["provenance_json"] != json.dumps(normalized["provenance"], sort_keys=True, separators=(",", ":")):
raise RuntimeError("COMPLETED_REPLAY_PROVENANCE_CONFLICT")
if evidence["identity_digest"] != sha256_json(normalized):
raise RuntimeError("COMPLETED_REPLAY_IDENTITY_CONFLICT")
if not (evidence["commit_oid"] == evidence["authority_head_oid"] == evidence["current_head_oid"] and evidence["commit_oid"] != evidence["historical_lease_head_oid"]):
raise RuntimeError("COMPLETED_REPLAY_EVIDENCE_INCOHERENT")
if self.authority.read_ref("refs/heads/" + incoming["target_branch"]) != evidence["current_head_oid"]:
raise RuntimeError("COMPLETED_REPLAY_AUTHORITY_CONFLICT")
result = {"job_id": external_job_id, "task_id": incoming["task_id"], "repository": incoming["repository"], "branch_mode": incoming["branch_mode"], "base_oid": request.get("base_oid"), "expected_branch_oid": request.get("expected_branch_oid"), "target_branch": incoming["target_branch"], "generation": incoming["generation"], "commit_oid": evidence["commit_oid"], "idempotency_key": row["idempotency_key"], "state": JobState.COMPLETED.value, "side_effect_state": row["side_effect_state"], "http_status": 200, "revision": int(row["revision"]), "receipt": {**receipt, "commit_oid": evidence["commit_oid"], "authority_oid": evidence["authority_head_oid"], "current_head_oid": evidence["current_head_oid"], "external_job_id": evidence["external_job_id"], "completed_replay_reconciliation_id": evidence["reconciliation_id"]}, "current_head_oid": evidence["current_head_oid"], "completed_replay_reconciliation_id": evidence["reconciliation_id"]}
return result
controller = GitOpsJobController(JobStore.for_existing_append(root))
reconciliation = controller.store.append_completed_replay_reconciliation(str(row["job_id"]), incoming, authority_head_oid=authority_head)
evidence = {**receipt, "commit_oid": reconciliation["commit_oid"], "authority_oid": reconciliation["authority_head_oid"], "current_head_oid": reconciliation["current_head_oid"], "completed_replay_reconciliation_id": reconciliation["reconciliation_id"], "external_job_id": external_job_id}
return {"job_id": external_job_id, "task_id": incoming["task_id"], "repository": incoming["repository"], "branch_mode": incoming["branch_mode"], "base_oid": request.get("base_oid"), "expected_branch_oid": request.get("expected_branch_oid"), "target_branch": incoming["target_branch"], "generation": incoming["generation"], "commit_oid": reconciliation["commit_oid"], "idempotency_key": row["idempotency_key"], "state": JobState.COMPLETED.value, "side_effect_state": row["side_effect_state"], "http_status": 200, "revision": int(row["revision"]), "receipt": evidence, "current_head_oid": reconciliation["current_head_oid"], "completed_replay_reconciliation_id": reconciliation["reconciliation_id"]}
def _controller_for_job(self, job_id: str) -> GitOpsJobController:
for controller in self.controllers.values():
try:
controller.store.read(job_id)
return controller
except Exception:
continue
jobs_root = self.config.state_root / "jobs"
if jobs_root.is_dir():
for root in jobs_root.iterdir():
if not root.is_dir():
continue
db = root / "state.sqlite3"
if not db.is_file() or db.is_symlink():
continue
try:
connection = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
connection.row_factory = sqlite3.Row
try:
rows = connection.execute("SELECT job_id, request_json FROM jobs").fetchall()
finally:
connection.close()
matched = False
for row in rows:
if str(row["job_id"]) == job_id:
matched = True
break
durable = json.loads(row["request_json"])
binding = durable.get("binding", durable)
if str(binding.get("external_job_id") or "") == job_id:
matched = True
break
if not matched:
continue
controller = GitOpsJobController(JobStore(root, payload_root=root / "payloads"))
self.controllers[str(job_id)] = controller
return controller
except (OSError, ValueError, TypeError, sqlite3.Error):
continue
raise RuntimeError("JOB_NOT_FOUND")
@staticmethod
def _valid_oid(value: object) -> bool:
return isinstance(value, str) and re.fullmatch(r"[0-9a-f]{40}", value) is not None
def _reconstruct_legacy_binding(
self,
row: Mapping[str, Any],
controller: GitOpsJobController,
) -> dict[str, Any]:
raw = json.loads(row["request_json"])
binding = raw.get("binding", raw)
if not isinstance(binding, dict):
raise RuntimeError("DURABLE_BINDING_INVALID")
task_id = str(binding.get("task_id") or binding.get("campaign_id") or "").strip()
repository = str(binding.get("repository") or "").strip()
operation = str(binding.get("operation") or row.get("operation") or "").strip()
branch = str(binding.get("target_branch") or binding.get("branch") or row.get("branch") or "").strip()
expected = str(binding.get("expected_branch_oid") or row.get("expected_branch_oid") or "").strip()
base = binding.get("base_oid") or binding.get("lineage_anchor_oid")
if not base:
with controller.store._connect() as connection:
lease_row = connection.execute(
"SELECT lineage_anchor_oid FROM branch_leases WHERE job_id=? ORDER BY revision DESC LIMIT 1",
(str(row["job_id"]),),
).fetchone()
if lease_row is not None:
base = lease_row["lineage_anchor_oid"]
base = str(base or "").strip()
branch_mode = str(
binding.get("branch_mode")
or ("CREATE_NEW" if operation.endswith("_CREATE_NEW") else "CONTINUE_EXISTING")
).strip()
try:
generation = int(binding.get("generation") or 0)
except (TypeError, ValueError) as exc:
raise RuntimeError("DURABLE_BINDING_GENERATION_INVALID") from exc
if (
not task_id
or repository != self.config.repository
or not branch.startswith("agent/")
or branch in {"main", "master"}
or branch_mode not in {"CREATE_NEW", "CONTINUE_EXISTING"}
or generation < 1
or not self._valid_oid(base)
or not self._valid_oid(expected)
):
raise RuntimeError("DURABLE_BINDING_INVALID")
current = self.authority.read_ref("refs/heads/" + branch)
if (
current is None
or not self.authority.commit_exists(base)
or not self.authority.commit_exists(expected)
or not self.authority.ancestry_check(base, current)
or not self.authority.ancestry_check(expected, current)
):
raise RuntimeError("DURABLE_BINDING_LINEAGE_INVALID")
return {
"task_id": task_id,
"repository": repository,
"branch_mode": branch_mode,
"base_oid": base,
"expected_branch_oid": expected,
"target_branch": branch,
"generation": generation,
"lease_seconds": int(binding.get("lease_seconds") or 900),
}
def _executor_from_row(
self,
row: Mapping[str, Any],
controller: GitOpsJobController,
) -> ContinuationExecutor:
request = self._reconstruct_legacy_binding(row, controller)
lease = WriterLeaseRequestV2(**request)
worktree_path = self.config.state_root / "worktrees" / _safe(lease.task_id)
if not worktree_path.exists():
current = self.authority.read_ref("refs/heads/" + lease.target_branch)
if current is None:
raise RuntimeError("DURABLE_WORKTREE_BRANCH_MISSING")
self.authority.create_detached_worktree(
current,
_safe(lease.task_id),
self.config.state_root / "worktrees",
)
workspace = DetachedGitWorkspace(worktree_path)
credential = ServerSideCredentialProjection(self.config.ssh_key_path, self.config.token_path)
remote = GiteaContinuationRemote(
workspace=worktree_path,
credential=credential,
remote_name=self.config.remote_name,
repository=lease.repository,
api_base=self.config.api_base,
)
return ContinuationExecutor(
controller,
AuthorityGitAdapter(self.authority),
workspace,
remote,
credential,
)
def job(self, request: Mapping[str, Any], action: str) -> dict[str, Any]:
job_id = str(request["job_id"])
controller = self._controller_for_job(job_id)
row = controller.store.read(job_id)
if action == "get":
result = controller.store._receipt(row, duplicate=True).as_dict()
evidence_reader = getattr(controller.store, "completed_replay_evidence", None)
evidence = evidence_reader(job_id) if callable(evidence_reader) else None
if evidence is not None:
branch = str(json.loads(row["request_json"]).get("binding", {}).get("branch") or row["branch"])
authority_head = self.authority.read_ref("refs/heads/" + branch)
if authority_head != evidence["authority_head_oid"] or evidence["current_head_oid"] != authority_head:
raise RuntimeError("COMPLETED_REPLAY_AUTHORITY_CONFLICT")
result["job_id"] = evidence["external_job_id"]
result["current_head_oid"] = evidence["current_head_oid"]
result["completed_replay_reconciliation_id"] = evidence["reconciliation_id"]
result["receipt"] = {
**(result.get("receipt") or {}),
"commit_oid": evidence["commit_oid"],
"authority_oid": evidence["authority_head_oid"],
"current_head_oid": evidence["current_head_oid"],
"external_job_id": evidence["external_job_id"],
"completed_replay_reconciliation_id": evidence["reconciliation_id"],
}
return result
if action == "resume":
return controller.resume(job_id).as_dict()
if JobState(row["state"]) not in _RECOVERABLE:
return controller.store._receipt(row, duplicate=True).as_dict()
return self._executor_from_row(row, controller).reconcile(job_id)
def _snapshot_task(self) -> dict[str, Any]:
candidates: list[dict[str, Any]] = []
for value in self.bindings.values():
task_id = str(value.get("task_id") or "").strip()
branch = str(value.get("target_branch") or "").strip()
if not task_id or not branch.startswith("agent/"):
continue
authority = self.authority.read_ref("refs/heads/" + branch)
try:
generation = int(value.get("generation") or 0)
except (TypeError, ValueError):
generation = 0
reconciliation_state = "OBSERVATION_REQUIRED"
projection: dict[str, Any] = {}
db = self.config.state_root / "jobs" / _safe(task_id) / "state.sqlite3"
if db.is_file() and not db.is_symlink():
connection = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
connection.row_factory = sqlite3.Row
try:
lease = connection.execute("SELECT MAX(generation) FROM branch_leases WHERE branch=?", (branch,)).fetchone()
generation = max(generation, int((lease or (0,))[0] or 0))
latest = connection.execute("SELECT job_id, state, side_effect_state FROM jobs WHERE branch=? ORDER BY updated_at DESC LIMIT 1", (branch,)).fetchone()
if latest is not None:
reconciliation_state = "COMPLETED" if latest["state"] == "COMPLETED" else str(latest["side_effect_state"] or latest["state"])
evidence_rows = connection.execute("SELECT * FROM completed_replay_reconciliations WHERE job_id=? ORDER BY created_at ASC", (latest["job_id"],)).fetchall()
if evidence_rows:
evidence = dict(evidence_rows[0])
immutable = tuple(key for key in evidence if key != "created_at")
if any(any(item[key] != evidence[key] for key in immutable) for item in evidence_rows[1:]):
raise RuntimeError("COMPLETED_REPLAY_AMBIGUOUS")
if not (evidence["commit_oid"] == evidence["authority_head_oid"] == evidence["current_head_oid"] and evidence["commit_oid"] != evidence["historical_lease_head_oid"]):
raise RuntimeError("COMPLETED_REPLAY_EVIDENCE_INCOHERENT")
current = self.authority.read_ref("refs/heads/" + branch)
if current != evidence["authority_head_oid"] or current != evidence["current_head_oid"]:
raise RuntimeError("COMPLETED_REPLAY_AUTHORITY_CONFLICT")
reconciliation_state = "COMPLETED_REPLAY_RECONCILED"
projection = {"external_job_id": evidence["external_job_id"], "commit_oid": evidence["commit_oid"], "current_head_oid": evidence["current_head_oid"], "completed_replay_reconciliation_id": evidence["reconciliation_id"], "receipt_authority_oid": evidence["authority_head_oid"]}
finally:
connection.close()
candidate = {"task_id": task_id, "branch": branch, "authority_branch_oid": authority, "gitea_git_oid": None, "gitea_api_oid": None, "commit_visible": None, "generation": generation, "reconciliation_state": reconciliation_state}
candidate.update(projection)
candidates.append(candidate)
if not candidates:
return {}
return max(candidates, key=lambda item: (int(item["generation"] or 0), bool(item["authority_branch_oid"])))
def snapshot(self) -> dict[str, Any]:
task = self._snapshot_task()
return build_control_plane_snapshot(target_version=self.config.target_version, runtime=self.runtime_status(), repository={"identity": self.config.repository, "authority_git_dir": str(self.config.authoritative_git_dir), "authority_main_oid": self.authority.read_ref("refs/heads/main")}, production={"baseline_version": "V2R1C7", "target_oid": "c7adece1cc8beffe7c718d53c775898b45d6c7f8", "deployment_state": "ACTIVE"}, task=task, capabilities={"legacy_writes_enabled": False, "governed_writes_enabled": True, "force_push": False, "main_write": False, "tag_write": False, "baseline_write": False, "production_write": False}, credential_broker={"status": "SERVER_SIDE_ONLY", "provider": "FileCredentialProvider"})
def handlers(self) -> dict[str, Any]:
return {
"projectOnlineAgentExecution": self.project_online_agent_execution,
"createGovernedTask": self.create_task,
"acquireWriterLease": self.acquire_lease,
"commitFileGoverned": lambda request: self.commit_files({**request, "files": {request["path"]: request["content_base64"]}}),
"commitFilesGoverned": self.commit_files,
"getGitOpsJob": lambda request: self.job(request, "get"),
"resumeGitOpsJob": lambda request: self.job(request, "resume"),
"reconcileGitOpsJob": lambda request: self.job(request, "reconcile"),
"getControlPlaneSnapshot": lambda request: self.snapshot(),
"createTagApprovalRequest": self.release.approval_request,
"requestOrCreateTag": self.release.publish_tag,
"getTagPublicationStatus": self.release.tag_status,
"buildProductionArtifact": self.release.build_artifact,
"executeApprovedProductionRelease": self.release.execute_request,
"getReleaseConsistency": self.release.consistency,
"runReleaseAcceptance": self.release.acceptance,
}
def build_governed_handlers(runtime_config: RuntimeConfig | Mapping[str, Any]) -> Mapping[str, Any]:
if isinstance(runtime_config, Mapping): runtime_config = RuntimeConfig(**runtime_config)
return GovernedRuntime(runtime_config).handlers()
class _HTTPError(RuntimeError): pass
class BridgeHandler(BaseHTTPRequestHandler):
adapter: BridgeV2Adapter | None = None
def log_message(self, *_args: Any) -> None: return
def _json(self, value: Any, status: int = 200) -> None:
raw = json.dumps(value, sort_keys=True, separators=(",", ":")).encode(); self.send_response(status); self.send_header("Content-Type", "application/json"); self.send_header("Content-Length", str(len(raw))); self.end_headers(); self.wfile.write(raw)
def _body(self) -> dict[str, Any]:
try: length = int(self.headers.get("Content-Length", "-1"))
except ValueError as exc: raise _HTTPError("CONTENT_LENGTH_INVALID") from exc
if length < 0 or length > MAX_BODY: raise _HTTPError("REQUEST_TOO_LARGE")
value = json.loads(self.rfile.read(length).decode("utf-8"))
if not isinstance(value, dict): raise _HTTPError("JSON_OBJECT_REQUIRED")
return value
def _allowed(self, operation: str, body: Mapping[str, Any]) -> None:
allowed = set(OPERATION_REQUIRED_FIELDS[operation]) | {"schema", "lease_seconds", "create_target_branch", "worktree_mode", "cas_ref_update_required", "expected_branch_oid", "generation", "branch_mode", "source_lease_epoch"}
if operation == "commitFilesGoverned": allowed.add("files")
allowed.update({"owner_approval", "execute", "job_id", "historical_lease_head_oid"})
if set(body) - allowed: raise _HTTPError("UNKNOWN_FIELDS")
def do_GET(self) -> None:
if self.adapter is None: self._json({"error": "RUNTIME_UNAVAILABLE"}, 503); return
if self.path == "/v2/runtime": self._json(self.adapter.getRuntimeStatus()); return
if self.path == "/v2/capabilities": self._json(self.adapter.getGovernedWriteCapabilities()); return
if self.path == "/v2/control-plane-snapshot": self._json(self.adapter.getControlPlaneSnapshot({})); return
self._json({"error": "NOT_FOUND"}, 404)
def do_POST(self) -> None:
operation = ROUTES.get(self.path)
if operation is None or operation in ("getRuntimeStatus", "getGovernedWriteCapabilities") or self.adapter is None: self._json({"error": "NOT_FOUND"}, 404); return
try:
body = self._body(); self._allowed(operation, body); self._json(getattr(self.adapter, operation)(body))
except _HTTPError as exc: self._json({"error": str(exc)}, 413 if str(exc) == "REQUEST_TOO_LARGE" else 400)
except (BootstrapError, ValueError, BridgeV2Error) as exc: self._json({"error": str(exc)}, 409 if "CONFLICT" in str(exc) else 400)
except Exception as exc: self._json({"error": str(exc) or type(exc).__name__}, 409 if any(token in str(exc) for token in ("CONFLICT", "EXPECTATION", "MONOTONIC")) else 503)
def serve(adapter: BridgeV2Adapter, host: str = "127.0.0.1", port: int = 18892) -> None:
BridgeHandler.adapter = adapter; ThreadingHTTPServer((host, port), BridgeHandler).serve_forever()
def main() -> None:
state_root = Path("/var/lib/aios-gitops/bridge-v2-staging")
token_path = Path(os.environ.get("AIOS_GITEA_TOKEN_FILE", "/run/credentials/aios-chatgpt-bridge-v2-staging.service/gitea_token"))
ssh_key_path = Path(os.environ.get("AIOS_GITEA_SSH_KEY_PATH", "/home/aiman/.ssh/id_ed25519_codex_tmp"))
os.environ["CREDENTIAL_FILE"] = str(token_path)
os.environ.setdefault("GITEA_MACHINE_ACTOR", "aios-autonomy-agent")
config = RuntimeConfig(
os.environ.get("AIOS_CONTROL_PLANE_TARGET_OID", ""),
Path("/data/aios-gitops/repo.git"),
state_root,
state_root / "jobs",
token_path,
ssh_key_path,
remote_name=os.environ.get("AIOS_GITEA_REMOTE", "gitea-http"),
api_base=validate_gitea_api_url(os.environ.get("AIOS_GITEA_API_BASE", "")),
target_version=os.environ.get("AIOS_CONTROL_PLANE_TARGET_VERSION", "V2R2C1"),
baseline_path=Path(os.environ.get(
"AIOS_GITOPS_BASELINE_AUTHORITY",
"/var/lib/aios-gitops-runtime-anchor/baseline/current.json",
)),
)
runtime = GovernedRuntime(config)
serve(
BridgeV2Adapter(runtime_status=runtime.runtime_status(), handlers=runtime.handlers()),
port=int(os.environ.get("AIOS_BRIDGE_V2_PORT", "18892")),
)
if __name__ == "__main__": main()
@@ -0,0 +1,615 @@
"""Governed continuation executor for exact existing task branches.
The executor composes the existing continuation lease and durable Job store.
All Git and Gitea side effects live behind injected narrow interfaces so the
state machine can prove Authority, Git-network, API-branch, and commit-object
agreement before it reports completion.
"""
from __future__ import annotations
import base64
from dataclasses import asdict, dataclass
from datetime import datetime, timezone
import hashlib
import json
import os
from pathlib import Path, PurePosixPath
import re
import subprocess
from typing import Callable, Mapping, Protocol
from urllib.error import HTTPError, URLError
from urllib.parse import quote, urlsplit
from urllib.request import Request, urlopen
from aios_authority_git_executor import AuthorityGitExecutor
from aios_continuation_lease import WriterLeaseRequestV2
from aios_gitops_job_controller import GitOpsJobController, JobState
from aios_gitops_reconciliation_core import ReconciliationAction, RemoteObservation, decide_reconciliation
from authorized_task_writer_provisioning import validate_writer_lease_request_v2
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SECRET_PATTERNS = (
re.compile(rb"gh[pousr]_[A-Za-z0-9]{20,}"),
re.compile(rb"(?i)(?:token|password|secret|api[_-]?key)\s*[:=]\s*[\"'][^\"'\s]{12,}[\"']"),
re.compile(rb"-----BEGIN " + rb"(?:RSA |EC |OPENSSH )?PRIVATE KEY-----"),
)
class ContinuationExecutionError(RuntimeError):
"""Stable fail-closed continuation execution error."""
class DetachedWorkspace(Protocol):
def is_detached(self) -> bool: ...
def head_oid(self) -> str: ...
def commit_files(self, files: Mapping[str, bytes], message: str) -> str: ...
class AuthorityPort(Protocol):
def branch_exists(self, branch: str) -> bool: ...
def branch_head(self, branch: str) -> str: ...
def is_ancestor(self, older: str, newer: str) -> bool: ...
def compare_and_swap_branch(self, branch: str, expected: str, new: str) -> str: ...
class RemotePort(Protocol):
def repository_identity(self) -> str: ...
def push_exact(self, branch: str, expected: str, new: str, *, allow_absent: bool = False) -> None: ...
def read_git_ref(self, branch: str) -> str | None: ...
def read_api_branch(self, branch: str) -> str | None: ...
def commit_visible(self, oid: str) -> bool: ...
class CredentialProjection(Protocol):
def project(self) -> None: ...
def revoke(self) -> None: ...
def _run(argv: list[str], *, cwd: Path | None = None, environment: Mapping[str, str] | None = None) -> str:
result = subprocess.run(
argv,
cwd=cwd,
env=None if environment is None else dict(environment),
text=True,
capture_output=True,
check=False,
)
if result.returncode != 0:
raise ContinuationExecutionError("SUBPROCESS_FAILED")
return result.stdout.strip()
@dataclass(slots=True)
class AuthorityGitAdapter:
"""Branch-name adapter over the existing narrow AuthorityGitExecutor."""
executor: AuthorityGitExecutor
@staticmethod
def _ref(branch: str) -> str:
return f"refs/heads/{branch}"
def branch_exists(self, branch: str) -> bool:
return self.executor.read_ref(self._ref(branch)) is not None
def branch_head(self, branch: str) -> str:
value = self.executor.read_ref(self._ref(branch))
if value is None:
raise ContinuationExecutionError("EXISTING_BRANCH_MISSING")
return value
def is_ancestor(self, older: str, newer: str) -> bool:
return self.executor.ancestry_check(older, newer)
def compare_and_swap_branch(self, branch: str, expected: str, new: str) -> str:
return self.executor.advance_task_branch_CAS(self._ref(branch), expected, new)
@dataclass(slots=True)
class DetachedGitWorkspace:
path: Path
def __post_init__(self) -> None:
if not self.path.is_absolute() or self.path.is_symlink() or not self.path.is_dir():
raise ContinuationExecutionError("WORKTREE_PATH_INVALID")
top = Path(_run(["git", "rev-parse", "--show-toplevel"], cwd=self.path)).resolve()
if top != self.path.resolve():
raise ContinuationExecutionError("WORKTREE_PATH_INVALID")
def is_detached(self) -> bool:
result = subprocess.run(
["git", "symbolic-ref", "-q", "HEAD"],
cwd=self.path,
capture_output=True,
check=False,
)
return result.returncode == 1
def head_oid(self) -> str:
return _run(["git", "rev-parse", "HEAD"], cwd=self.path)
def _dirty_paths(self) -> set[str]:
command = subprocess.run(
["git", "status", "--porcelain=v1", "--untracked-files=all"],
cwd=self.path,
text=True,
capture_output=True,
check=False,
)
if command.returncode != 0:
raise ContinuationExecutionError("SUBPROCESS_FAILED")
output = command.stdout
result: set[str] = set()
for line in output.splitlines():
path = line[3:]
if " -> " in path:
path = path.split(" -> ", 1)[1]
result.add(path)
return result
def materialize_files(self, files: Mapping[str, bytes]) -> None:
dirty = self._dirty_paths()
expected_paths = set(files)
if dirty and dirty != expected_paths:
raise ContinuationExecutionError("WORKTREE_DIRTY_SCOPE_MISMATCH")
for relative, content in files.items():
target = (self.path / relative).resolve()
try:
target.relative_to(self.path.resolve())
except ValueError as exc:
raise ContinuationExecutionError("COMMIT_PATH_REJECTED") from exc
if target.is_symlink():
raise ContinuationExecutionError("COMMIT_PATH_REJECTED")
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(content)
def commit_files(self, files: Mapping[str, bytes], message: str) -> str:
expected_paths = set(files)
if self._dirty_paths() != expected_paths:
raise ContinuationExecutionError("WORKTREE_DIRTY_SCOPE_MISMATCH")
for relative, content in files.items():
target = (self.path / relative).resolve()
try:
target.relative_to(self.path.resolve())
except ValueError as exc:
raise ContinuationExecutionError("COMMIT_PATH_REJECTED") from exc
if not target.is_file() or target.is_symlink() or target.read_bytes() != content:
raise ContinuationExecutionError("AUTHORING_PAYLOAD_MISMATCH")
_run(["git", "add", "--", *sorted(expected_paths)], cwd=self.path)
staged = set(_run(["git", "diff", "--cached", "--name-only"], cwd=self.path).splitlines())
if staged != expected_paths:
raise ContinuationExecutionError("STAGED_SCOPE_MISMATCH")
environment = os.environ.copy()
environment.update(
{
"GIT_AUTHOR_NAME": "AIOS Governed Continuation",
"GIT_AUTHOR_EMAIL": "aios-governed-continuation@localhost",
"GIT_COMMITTER_NAME": "AIOS Governed Continuation",
"GIT_COMMITTER_EMAIL": "aios-governed-continuation@localhost",
}
)
_run(["git", "commit", "--no-gpg-sign", "-m", message, "--", *sorted(expected_paths)], cwd=self.path, environment=environment)
if not self.is_detached():
raise ContinuationExecutionError("WORKTREE_NOT_DETACHED")
return self.head_oid()
@dataclass(slots=True)
class ServerSideCredentialProjection:
ssh_key_path: Path
api_token_path: Path
active: bool = False
def project(self) -> None:
for path in (self.ssh_key_path, self.api_token_path):
if path.is_symlink() or not path.is_file() or path.stat().st_size < 32:
raise ContinuationExecutionError("CREDENTIAL_PROJECTION_MISSING")
self.active = True
def revoke(self) -> None:
self.active = False
@dataclass(slots=True)
class GiteaContinuationRemote:
workspace: Path
credential: ServerSideCredentialProjection
remote_name: str = "gitea"
repository: str = "aios/aios"
api_base: str = "https://g.cnnetmkt.top/git/api/v1"
expected_host: str = "g.cnnetmkt.top"
@staticmethod
def validate_remote_url(value: str, expected_host: str = "g.cnnetmkt.top") -> None:
if value.startswith(("/", "file:")) or value.endswith("/data/aios-gitops/repo.git"):
raise ContinuationExecutionError("GITEA_REMOTE_IDENTITY_INVALID")
if "://" in value:
parsed = urlsplit(value)
host, path = parsed.hostname, parsed.path
else:
location = value.split("@", 1)[-1]
host, separator, path = location.partition(":")
if not separator:
raise ContinuationExecutionError("GITEA_REMOTE_IDENTITY_INVALID")
path = "/" + path
if host != expected_host or path.rstrip("/") not in {"/aios/aios.git", "/git/aios/aios.git"}:
raise ContinuationExecutionError("GITEA_REMOTE_IDENTITY_INVALID")
def repository_identity(self) -> str:
url = _run(["git", "remote", "get-url", self.remote_name], cwd=self.workspace)
self.validate_remote_url(url, self.expected_host)
if self.repository != "aios/aios":
raise ContinuationExecutionError("REMOTE_REPOSITORY_IDENTITY_MISMATCH")
return self.repository
def _require_projected(self) -> None:
if not self.credential.active:
raise ContinuationExecutionError("CREDENTIAL_PROJECTION_MISSING")
def read_git_ref(self, branch: str) -> str | None:
self._require_projected()
output = _run(["git", "ls-remote", self.remote_name, f"refs/heads/{branch}"], cwd=self.workspace)
if not output:
return None
oid, ref = output.split(maxsplit=1)
if ref != f"refs/heads/{branch}" or not OID_RE.fullmatch(oid):
raise ContinuationExecutionError("GITEA_GIT_READBACK_INVALID")
return oid
def push_exact(self, branch: str, expected: str, new: str, *, allow_absent: bool = False) -> None:
self._require_projected()
self.repository_identity()
current = self.read_git_ref(branch)
if current != expected and not (allow_absent and current is None):
raise ContinuationExecutionError("GITEA_BRANCH_EXPECTATION_MISMATCH")
_run(["git", "push", self.remote_name, f"{new}:refs/heads/{branch}"], cwd=self.workspace)
def _api(self, path: str) -> object:
self._require_projected()
token = self.credential.api_token_path.read_text(encoding="utf-8").strip()
request = Request(
self.api_base.rstrip("/") + path,
headers={"Accept": "application/json", "Authorization": f"token {token}"},
)
try:
with urlopen(request, timeout=20) as response:
return json.loads(response.read().decode("utf-8"))
except HTTPError as exc:
if exc.code == 404:
return None
raise ContinuationExecutionError(f"GITEA_API_HTTP_{exc.code}") from exc
except (URLError, TimeoutError, OSError, json.JSONDecodeError) as exc:
raise ContinuationExecutionError("GITEA_API_UNAVAILABLE") from exc
def read_api_branch(self, branch: str) -> str | None:
owner, name = self.repository.split("/", 1)
page = 1
while page <= 100:
value = self._api(f"/repos/{quote(owner)}/{quote(name)}/branches?page={page}&limit=50")
if value is None:
return None
if not isinstance(value, list):
raise ContinuationExecutionError("GITEA_API_BRANCH_COLLECTION_INVALID")
for item in value:
if not isinstance(item, dict) or item.get("name") != branch:
continue
oid = item.get("commit", {}).get("id")
if not isinstance(oid, str) or not OID_RE.fullmatch(oid):
raise ContinuationExecutionError("GITEA_API_BRANCH_INVALID")
return oid
if len(value) < 50:
return None
page += 1
raise ContinuationExecutionError("GITEA_API_BRANCH_PAGINATION_LIMIT")
def commit_visible(self, oid: str) -> bool:
owner, name = self.repository.split("/", 1)
value = self._api(f"/repos/{quote(owner)}/{quote(name)}/git/commits/{oid}")
if not isinstance(value, dict):
return False
observed = value.get("sha") or value.get("id")
return observed == oid
@dataclass(frozen=True, slots=True)
class ContinuationPermissions:
create_target_branch: bool = False
force_push: bool = False
tag_write: bool = False
baseline_write: bool = False
production_write: bool = False
main_write: bool = False
def validate(self) -> None:
if any(asdict(self).values()):
raise ContinuationExecutionError("MUTATION_PERMISSION_DENIED")
@dataclass(frozen=True, slots=True)
class ContinuationRuntimeBinding:
request: WriterLeaseRequestV2
workspace_path: Path
lease_not_before: datetime
lease_expires_at: datetime
original_receipt_valid: bool
original_grant_consumed: bool
permissions: ContinuationPermissions
remote_branch_create_allowed: bool = False
external_job_id: str | None = None
worker_principal: str = "aios-autonomy-agent"
worker_lineage: str = "governed-continuation"
@dataclass(frozen=True, slots=True)
class ContinuationCommit:
idempotency_key: str
files: Mapping[str, bytes]
message: str
def payload(self) -> bytes:
if not self.idempotency_key or not self.message.strip() or not self.files:
raise ContinuationExecutionError("COMMIT_REQUEST_INVALID")
normalized: dict[str, str] = {}
for path, content in self.files.items():
pure = PurePosixPath(path)
if (
not isinstance(path, str)
or pure.is_absolute()
or ".." in pure.parts
or path.startswith((".git/", "baseline/", "production/"))
or not isinstance(content, bytes)
):
raise ContinuationExecutionError("COMMIT_PATH_REJECTED")
if any(pattern.search(content) for pattern in SECRET_PATTERNS):
raise ContinuationExecutionError("SECRET_LIKE_PAYLOAD_REJECTED")
normalized[path] = base64.b64encode(content).decode("ascii")
return json.dumps(
{"files_base64": normalized, "message": self.message},
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
class ContinuationExecutor:
def __init__(
self,
controller: GitOpsJobController,
authority: AuthorityPort,
workspace: DetachedWorkspace,
remote: RemotePort,
credential: CredentialProjection,
*,
now: Callable[[], datetime] = lambda: datetime.now(timezone.utc),
) -> None:
self.controller = controller
self.authority = authority
self.workspace = workspace
self.remote = remote
self.credential = credential
self.now = now
self._observed_generations: dict[str, int] = {}
def observe_generation(self, branch: str, generation: int) -> None:
self._observed_generations[branch] = max(generation, self._observed_generations.get(branch, 0))
def _max_generation(self, branch: str) -> int:
observed = self._observed_generations.get(branch, 0)
with self.controller.store._connect() as connection:
row = connection.execute(
"SELECT MAX(generation) AS generation FROM branch_leases WHERE branch=?",
(branch,),
).fetchone()
durable = int(row["generation"] or 0)
return max(observed, durable)
@staticmethod
def _request_mapping(request: WriterLeaseRequestV2) -> dict[str, object]:
return {
"schema": WriterLeaseRequestV2.schema,
"task_id": request.task_id,
"repository": request.repository,
"branch_mode": request.branch_mode,
"base_oid": request.base_oid,
"expected_branch_oid": request.expected_branch_oid,
"target_branch": request.target_branch,
"generation": request.generation,
"lease_seconds": request.lease_seconds,
}
def _validate_preflight(self, binding: ContinuationRuntimeBinding) -> None:
request = binding.request
normalized = validate_writer_lease_request_v2(self._request_mapping(request), self.authority)
if normalized != {**self._request_mapping(request), "create_target_branch": False, "worktree_mode": "detached_at_expected_head", "cas_ref_update_required": True}:
raise ContinuationExecutionError("CONTINUATION_NORMALIZATION_INVALID")
if not binding.original_receipt_valid or not binding.original_grant_consumed:
raise ContinuationExecutionError("ORIGINAL_GRANT_LINEAGE_INVALID")
binding.permissions.validate()
now = self.now()
if now < binding.lease_not_before:
raise ContinuationExecutionError("LEASE_NOT_ACTIVE")
if now >= binding.lease_expires_at:
raise ContinuationExecutionError("LEASE_EXPIRED")
if request.generation <= self._max_generation(request.target_branch):
raise ContinuationExecutionError("GENERATION_NOT_MONOTONIC")
if not self.workspace.is_detached():
raise ContinuationExecutionError("WORKTREE_NOT_DETACHED")
if self.workspace.head_oid() != request.expected_branch_oid:
raise ContinuationExecutionError("WORKTREE_HEAD_MISMATCH")
if self.remote.repository_identity() != request.repository:
raise ContinuationExecutionError("REMOTE_REPOSITORY_IDENTITY_MISMATCH")
def _job_request(self, binding: ContinuationRuntimeBinding, commit: ContinuationCommit, payload: bytes) -> dict[str, object]:
request = binding.request
return {
"idempotency_key": commit.idempotency_key,
"campaign_id": request.task_id,
"task_id": request.task_id,
"repository": request.repository,
"operation": "CONTINUATION_COMMIT_FILES",
"branch": request.target_branch,
"expected_branch_oid": request.expected_branch_oid,
"base_oid": request.base_oid,
"branch_mode": request.branch_mode,
"payload_digest": hashlib.sha256(payload).hexdigest(),
"lineage_anchor_oid": request.base_oid,
"generation": request.generation,
"remote_branch_create_allowed": binding.remote_branch_create_allowed,
"external_job_id": binding.external_job_id,
"files_digest": hashlib.sha256(
json.dumps(
{path: hashlib.sha256(content).hexdigest() for path, content in sorted(commit.files.items())},
sort_keys=True, separators=(",", ":"),
).encode("utf-8")
).hexdigest(),
"message_digest": hashlib.sha256(commit.message.encode("utf-8")).hexdigest(),
"worker_principal": binding.worker_principal,
"worker_lineage": binding.worker_lineage,
}
def execute(self, binding: ContinuationRuntimeBinding, commit: ContinuationCommit) -> dict[str, object]:
payload = commit.payload()
submitted = self.controller.submit(self._job_request(binding, commit, payload), payload=payload)
current = self.controller.store.read(submitted.job_id)
if JobState(current["state"]) == JobState.COMPLETED:
return self.controller.store._receipt(current, duplicate=True).as_dict()
self._validate_preflight(binding)
request = binding.request
projected = False
try:
self.credential.project()
projected = True
row = self.controller.store.read(submitted.job_id)
if JobState(row["state"]) == JobState.REQUESTED:
self.controller.advance(
submitted.job_id,
JobState.AUTHORIZED,
side_effect_state="AUTHORIZED",
receipt={"task_id": request.task_id, "original_grant_consumed": True},
)
row = self.controller.store.read(submitted.job_id)
if JobState(row["state"]) == JobState.AUTHORIZED:
self.controller.lease_job(
submitted.job_id,
lease_id=f"lease-{submitted.job_id}",
lineage_anchor_oid=request.base_oid,
current_head_oid=request.expected_branch_oid,
generation=request.generation,
lease_expires_at=binding.lease_expires_at.astimezone(timezone.utc).isoformat(),
expected_revision=int(row["revision"]),
)
self.controller.advance(submitted.job_id, JobState.WORKTREE_READY, side_effect_state="WORKTREE_READY")
self.controller.advance(submitted.job_id, JobState.EXECUTING, side_effect_state="IN_PROGRESS")
new_oid = self.workspace.commit_files(commit.files, commit.message)
if not OID_RE.fullmatch(new_oid) or not self.authority.is_ancestor(request.expected_branch_oid, new_oid):
raise ContinuationExecutionError("NON_FAST_FORWARD_COMMIT")
authority_oid = self.authority.compare_and_swap_branch(request.target_branch, request.expected_branch_oid, new_oid)
if authority_oid != new_oid:
raise ContinuationExecutionError("AUTHORITY_CAS_READBACK_MISMATCH")
self.controller.advance(
submitted.job_id,
JobState.LOCAL_VERIFIED,
side_effect_state="LOCAL_VERIFIED",
receipt={"authority_oid": authority_oid, "force_push": False},
)
try:
self.controller.advance(submitted.job_id, JobState.REMOTE_SYNCING, side_effect_state="IN_PROGRESS")
self.remote.push_exact(request.target_branch, request.expected_branch_oid, new_oid, allow_absent=binding.remote_branch_create_allowed)
except Exception:
return self._remote_pending(submitted.job_id, authority_oid, new_oid)
return self._finish_remote_verification(submitted.job_id, authority_oid, new_oid)
finally:
if projected:
self.credential.revoke()
def _observe_remote(self, branch: str, oid: str) -> tuple[str | None, str | None, bool]:
return self.remote.read_git_ref(branch), self.remote.read_api_branch(branch), self.remote.commit_visible(oid)
def _remote_pending(self, job_id: str, authority_oid: str, new_oid: str) -> dict[str, object]:
receipt = {
"authority_oid": authority_oid,
"commit_oid": new_oid,
"desired_remote_oid": new_oid,
"remote_verified": False,
"force_push": False,
}
result = self.controller.advance(
job_id,
JobState.RECONCILIATION_REQUIRED,
side_effect_state="REMOTE_PUSH_PENDING",
receipt=receipt,
)
return result.as_dict()
def _finish_remote_verification(self, job_id: str, authority_oid: str, new_oid: str) -> dict[str, object]:
row = self.controller.store.read(job_id)
branch = row["branch"]
request = json.loads(row["request_json"])["binding"]
git_oid, api_oid, visible = self._observe_remote(branch, new_oid)
decision = decide_reconciliation(RemoteObservation("aios/aios", branch, new_oid, authority_oid, git_oid, api_oid, visible, datetime.now(timezone.utc).isoformat(), request.get("expected_branch_oid")))
if decision.action is not ReconciliationAction.MARK_REMOTE_VERIFIED:
return self._remote_pending(job_id, authority_oid, new_oid)
receipt = {
"authority_oid": authority_oid,
"commit_oid": new_oid,
"git_remote_oid": git_oid,
"api_branch_oid": api_oid,
"commit_visible": True,
"remote_verified": True,
"reconciliation_decision": decision.action.value,
"force_push": False,
}
self.controller.advance(job_id, JobState.REMOTE_VERIFIED, side_effect_state="REMOTE_VERIFIED", receipt=receipt)
return self.controller.advance(job_id, JobState.COMPLETED, side_effect_state="REMOTE_VERIFIED", receipt=receipt).as_dict()
def reconcile(self, job_id: str) -> dict[str, object]:
row = self.controller.store.read(job_id)
state = JobState(row["state"])
if state == JobState.COMPLETED:
return self.controller.store._receipt(row, duplicate=True).as_dict()
if state != JobState.RECONCILIATION_REQUIRED:
raise ContinuationExecutionError("JOB_NOT_RECONCILABLE")
request = json.loads(row["request_json"])["binding"]
receipt = json.loads(row["receipt_json"])
new_oid = receipt["desired_remote_oid"]
authority_oid = self.authority.branch_head(row["branch"])
try:
generation = int(request.get("generation") or 0)
except (TypeError, ValueError) as exc:
raise ContinuationExecutionError("DURABLE_BINDING_GENERATION_INVALID") from exc
if generation > 0:
with self.controller.store._connect() as connection:
earlier = connection.execute(
"SELECT 1 FROM jobs WHERE branch=? AND state=? "
"AND CAST(json_extract(request_json, '$.binding.generation') AS INTEGER) < ? LIMIT 1",
(row["branch"], JobState.RECONCILIATION_REQUIRED.value, generation),
).fetchone()
if earlier is not None:
raise ContinuationExecutionError("EARLIER_RECONCILIATION_REQUIRED")
self.credential.project()
try:
git_oid, api_oid, visible = self._observe_remote(row["branch"], new_oid)
if authority_oid == git_oid == api_oid == new_oid and visible:
self.controller.advance(job_id, JobState.RECONCILING, side_effect_state="OBSERVATION_FIRST")
return self._finish_remote_verification(job_id, new_oid, new_oid)
expected_oid = request["expected_branch_oid"]
if (
self.authority.is_ancestor(new_oid, authority_oid)
and git_oid is not None
and git_oid == api_oid
and self.authority.is_ancestor(git_oid, new_oid)
):
self.controller.advance(job_id, JobState.RECONCILING, side_effect_state="OBSERVATION_FIRST_CHAIN")
self.remote.push_exact(row["branch"], git_oid, new_oid, allow_absent=bool(request.get("remote_branch_create_allowed")) or request.get("branch_mode") == "CREATE_NEW")
return self._finish_remote_verification(job_id, new_oid, new_oid)
allow_remote_create = bool(request.get("remote_branch_create_allowed")) or request.get("branch_mode") == "CREATE_NEW" or (git_oid is None and api_oid is None)
remote_matches_expected = git_oid == api_oid == expected_oid
remote_absent_for_create = git_oid is None and api_oid is None
if authority_oid == new_oid and (remote_matches_expected or remote_absent_for_create):
self.controller.advance(job_id, JobState.RECONCILING, side_effect_state="OBSERVATION_FIRST_RETRY")
self.remote.push_exact(row["branch"], expected_oid, new_oid, allow_absent=allow_remote_create)
return self._finish_remote_verification(job_id, new_oid, new_oid)
return self.controller.store._receipt(row, duplicate=True).as_dict()
finally:
self.credential.revoke()
@@ -0,0 +1,154 @@
"""Campaign-aware continuation leases for an existing Git branch.
This module is deliberately transport-neutral. It validates branch lineage
and hands ref mutation to an injected compare-and-swap broker; it does not run
Git, inspect credentials, or create a branch itself.
"""
from __future__ import annotations
from dataclasses import dataclass, replace
from datetime import datetime, timedelta, timezone
import re
from typing import Protocol
OID_RE = re.compile(r"^[0-9a-f]{40}$")
TARGET_BRANCH = "agent/autonomous-release-campaign-v1-20260807"
LINEAGE_ANCHOR = "69b45e81dcd2c3707c367003b08c55fd361faae8"
CAMPAIGN_HEAD = "bba675fab3b7e0c5fba7066720252b3c91b9f20d"
class ContinuationError(RuntimeError):
"""Stable fail-closed error for lease or branch binding violations."""
class BranchProbe(Protocol):
def branch_exists(self, branch: str) -> bool: ...
def branch_head(self, branch: str) -> str: ...
def is_ancestor(self, older: str, newer: str) -> bool: ...
class RefCAS(Protocol):
def compare_and_swap_branch(self, branch: str, expected: str, new: str) -> str: ...
def _oid(value: str, field: str) -> str:
if not isinstance(value, str) or not OID_RE.fullmatch(value):
raise ContinuationError(f"{field}_invalid")
return value
@dataclass(frozen=True, slots=True)
class WriterLeaseRequestV2:
task_id: str
repository: str
branch_mode: str
base_oid: str
expected_branch_oid: str
target_branch: str
generation: int
lease_seconds: int = 900
schema = "aios.task-writer-lease.v2"
accepted_schemas = frozenset({"aios.task-writer-lease.v2", "aios.task-writer-lease-request.v2"})
def validate(self, probe: BranchProbe) -> None:
if not self.task_id or self.repository != "aios/aios":
raise ContinuationError("request_identity_invalid")
if self.branch_mode not in {"CREATE_NEW", "CONTINUE_EXISTING"}:
raise ContinuationError("branch_mode_invalid")
_oid(self.base_oid, "base_oid")
_oid(self.expected_branch_oid, "expected_branch_oid")
if self.target_branch in {"main", "master"} or self.target_branch.startswith(("refs/", "tags/")):
raise ContinuationError("protected_branch_rejected")
if not isinstance(self.generation, int) or isinstance(self.generation, bool) or self.generation < 1:
raise ContinuationError("generation_invalid")
if not isinstance(self.lease_seconds, int) or not 1 <= self.lease_seconds <= 86_400:
raise ContinuationError("lease_duration_invalid")
if self.branch_mode == "CONTINUE_EXISTING":
if not probe.branch_exists(self.target_branch):
raise ContinuationError("EXISTING_BRANCH_MISSING")
actual = _oid(probe.branch_head(self.target_branch), "actual_branch_oid")
if actual != self.expected_branch_oid:
raise ContinuationError("EXISTING_BRANCH_OID_CONFLICT")
if not probe.is_ancestor(self.base_oid, actual):
raise ContinuationError("BASE_NOT_ANCESTOR_OF_BRANCH")
elif hasattr(probe, "commit_exists") and not probe.commit_exists(self.base_oid):
raise ContinuationError("BASE_COMMIT_NOT_FOUND")
elif self.branch_mode == "CREATE_NEW" and probe.branch_exists(self.target_branch):
raise ContinuationError("TARGET_BRANCH_ALREADY_EXISTS")
elif self.branch_mode == "CREATE_NEW" and self.base_oid != self.expected_branch_oid:
raise ContinuationError("CREATE_NEW_BASE_EXPECTATION_MISMATCH")
@dataclass(frozen=True, slots=True)
class CampaignWriterLease:
task_id: str
repository: str
branch: str
lineage_anchor_oid: str
current_head_oid: str
generation: int
lease_expires_at: datetime
append_only: bool = True
renewable: bool = True
multi_commit: bool = True
recoverable: bool = True
state: str = "ACTIVE"
branch_bound: bool = True
def __post_init__(self) -> None:
_oid(self.lineage_anchor_oid, "lineage_anchor_oid")
_oid(self.current_head_oid, "current_head_oid")
if not all((self.append_only, self.renewable, self.multi_commit, self.recoverable, self.branch_bound)):
raise ContinuationError("lease_capability_downgrade")
class ContinuationLeaseManager:
"""State machine for a multi-commit lease with CAS-only ref updates."""
def __init__(self, probe: BranchProbe, refs: RefCAS, *, now=datetime.now) -> None:
self.probe = probe
self.refs = refs
self.now = now
def open(self, request: WriterLeaseRequestV2) -> CampaignWriterLease:
request.validate(self.probe)
if request.branch_mode != "CONTINUE_EXISTING":
raise ContinuationError("CREATE_NEW_REQUIRES_SEPARATE_PROVISIONER")
return CampaignWriterLease(
task_id=request.task_id,
repository=request.repository,
branch=request.target_branch,
lineage_anchor_oid=request.base_oid,
current_head_oid=request.expected_branch_oid,
generation=request.generation,
lease_expires_at=self.now(timezone.utc) + timedelta(seconds=request.lease_seconds),
)
def renew(self, lease: CampaignWriterLease, seconds: int) -> CampaignWriterLease:
if lease.state != "ACTIVE" or not lease.renewable or seconds < 1:
raise ContinuationError("lease_not_renewable")
return replace(lease, lease_expires_at=self.now(timezone.utc) + timedelta(seconds=seconds))
def append_commit(self, lease: CampaignWriterLease, new_head_oid: str) -> CampaignWriterLease:
if lease.state != "ACTIVE":
raise ContinuationError("lease_not_active")
_oid(new_head_oid, "new_head_oid")
if not self.probe.is_ancestor(lease.current_head_oid, new_head_oid):
raise ContinuationError("NON_FAST_FORWARD_COMMIT")
try:
actual = self.refs.compare_and_swap_branch(
lease.branch, lease.current_head_oid, new_head_oid
)
except Exception as exc:
raise ContinuationError("BRANCH_LEASE_STALE") from exc
if actual != new_head_oid:
raise ContinuationError("BRANCH_LEASE_STALE")
return replace(lease, current_head_oid=new_head_oid)
def supersede(self, lease: CampaignWriterLease) -> CampaignWriterLease:
if lease.state not in {"ACTIVE", "RECOVERABLE_FAILURE", "RECONCILIATION_REQUIRED"}:
raise ContinuationError("lease_not_supersedable")
return replace(lease, generation=lease.generation + 1, state="SUPERSEDED")
@@ -0,0 +1,25 @@
"""Separate read, write, and release plane readiness evaluation."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Mapping
@dataclass(frozen=True, slots=True)
class RuntimeStatus:
read_plane: Mapping[str, bool]
write_plane: Mapping[str, bool]
release_plane: Mapping[str, bool]
@property
def release_control_plane_ready(self) -> bool:
return all(self.read_plane.values()) and all(self.write_plane.values()) and all(self.release_plane.values())
def as_dict(self) -> dict[str, object]:
return {
"READ_PLANE": dict(self.read_plane),
"WRITE_PLANE": dict(self.write_plane),
"RELEASE_PLANE": dict(self.release_plane),
"release_control_plane_ready": self.release_control_plane_ready,
}
@@ -0,0 +1,58 @@
"""Workspace-independent authority/network/Gitea ref sentinel."""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
import subprocess
from typing import Callable, Mapping
class SentinelError(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class SentinelInputs:
authority_git_dir: Path
gitea_remote: str
gitea_api: str
ref: str
expected_oid: str
@dataclass(frozen=True, slots=True)
class SentinelResult:
authority_oid: str | None
network_oid: str | None
gitea_oid: str | None
equal: bool
error: str | None = None
def authority_oid(inputs: SentinelInputs) -> str:
result = subprocess.run(
["git", "--git-dir", str(inputs.authority_git_dir), "rev-parse", inputs.ref],
text=True,
capture_output=True,
check=False,
)
if result.returncode != 0:
raise SentinelError("AUTHORITY_REF_UNAVAILABLE")
return result.stdout.strip()
def evaluate(
inputs: SentinelInputs,
*,
network_reader: Callable[[str, str], str | None],
gitea_reader: Callable[[str, str], str | None],
) -> SentinelResult:
try:
local = authority_oid(inputs)
network = network_reader(inputs.gitea_remote, inputs.ref)
gitea = gitea_reader(inputs.gitea_api, inputs.ref)
equal = local == network == gitea == inputs.expected_oid
return SentinelResult(local, network, gitea, equal, None if equal else "OID_MISMATCH")
except SentinelError as exc:
return SentinelResult(None, None, None, False, str(exc))
@@ -0,0 +1,30 @@
"""Non-sensitive control-plane snapshot assembly."""
from __future__ import annotations
from typing import Any, Mapping
def _copy(value: Mapping[str, Any] | None, keys: tuple[str, ...]) -> dict[str, Any]:
value = value or {}
return {key: value.get(key) for key in keys}
def build_control_plane_snapshot(*, target_version: str, runtime: Mapping[str, Any] | None = None, repository: Mapping[str, Any] | None = None, production: Mapping[str, Any] | None = None, task: Mapping[str, Any] | None = None, capabilities: Mapping[str, Any] | None = None, credential_broker: Mapping[str, Any] | None = None) -> dict[str, Any]:
task_snapshot = _copy(task, ("task_id", "branch", "authority_branch_oid", "gitea_git_oid", "gitea_api_oid", "commit_visible", "generation", "reconciliation_state"))
for key in ("external_job_id", "commit_oid", "current_head_oid", "completed_replay_reconciliation_id", "receipt_authority_oid"):
if task is not None and key in task:
task_snapshot[key] = task[key]
return {
"version": {"target_release": target_version},
"runtime": _copy(runtime, ("bridge_schema_version", "control_plane_release_version", "runtime_release_commit", "runtime_path", "artifact_sha256", "drift_status")),
"repository": _copy(repository, ("identity", "authority_git_dir", "authority_main_oid")),
"production": _copy(production, ("baseline_version", "target_oid", "deployment_state")),
"task": task_snapshot,
"capabilities": _copy(capabilities, ("legacy_writes_enabled", "governed_writes_enabled", "force_push", "main_write", "tag_write", "baseline_write", "production_write")),
"credential_broker": _copy(credential_broker, ("status", "provider")),
"secret_material_exposed": False,
}
def getControlPlaneSnapshot(**kwargs: Any) -> dict[str, Any]:
return build_control_plane_snapshot(**kwargs)
@@ -0,0 +1,478 @@
"""Minimal convergence contracts for AIOS V2R2C2.
This module deliberately contains no external side effects. It provides the shared
data contracts and pure evaluation rules used by domain controllers. Domain
controllers remain authoritative for their own final states.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
from enum import Enum
import hashlib
import json
from typing import Any, Iterable, Mapping, Sequence
class ConvergenceContractError(ValueError):
"""Raised when an invalid convergence contract is constructed."""
class EvidenceState(str, Enum):
PASS = "PASS"
FAIL = "FAIL"
UNKNOWN = "UNKNOWN"
BLOCKED = "BLOCKED"
NOT_APPLICABLE = "NOT_APPLICABLE"
class AcceptanceDecision(str, Enum):
ACCEPT = "ACCEPT"
REJECT = "REJECT"
BLOCKED = "BLOCKED"
UNKNOWN = "UNKNOWN"
class ObservedHealth(str, Enum):
HEALTHY = "HEALTHY"
DEGRADED = "DEGRADED"
FAILED = "FAILED"
UNKNOWN = "UNKNOWN"
class ReconciliationState(str, Enum):
PENDING = "PENDING"
OBSERVING = "OBSERVING"
DIFF_FOUND = "DIFF_FOUND"
APPLYING = "APPLYING"
VERIFYING = "VERIFYING"
CONVERGED = "CONVERGED"
DEGRADED = "DEGRADED"
BLOCKED = "BLOCKED"
class ContinuationStopReason(str, Enum):
CONTEXT_BOUNDARY = "context_boundary"
WORKER_RESTART = "worker_restart"
HANDOFF = "handoff"
AUTHORITY_GATE = "authority_gate"
WAIT_EXTERNAL = "wait_external"
def _utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _canonical_json(value: Mapping[str, Any]) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def _sha256(value: Mapping[str, Any]) -> str:
return "sha256:" + hashlib.sha256(_canonical_json(value)).hexdigest()
def _nonempty(value: object, field_name: str) -> str:
text = str(value or "").strip()
if not text:
raise ConvergenceContractError(f"{field_name}_required")
return text
def _generation(value: object, field_name: str = "generation") -> int:
try:
parsed = int(value)
except (TypeError, ValueError) as exc:
raise ConvergenceContractError(f"{field_name}_invalid") from exc
if parsed < 1:
raise ConvergenceContractError(f"{field_name}_invalid")
return parsed
@dataclass(frozen=True, slots=True)
class StateOwnership:
object_type: str
authoritative_service: str
authoritative_store: str
final_writer: str | None = None
projections: tuple[str, ...] = ()
schema_version: str = "aios.state-ownership.v1"
def __post_init__(self) -> None:
_nonempty(self.object_type, "object_type")
_nonempty(self.authoritative_service, "authoritative_service")
_nonempty(self.authoritative_store, "authoritative_store")
if self.final_writer is not None:
_nonempty(self.final_writer, "final_writer")
def as_dict(self) -> dict[str, Any]:
return asdict(self)
DEFAULT_STATE_OWNERSHIP: tuple[StateOwnership, ...] = (
StateOwnership(
object_type="goal",
authoritative_service="goal-kernel",
authoritative_store="goal-store",
final_writer="goal-kernel",
projections=("admin-web", "context-pack"),
),
StateOwnership(
object_type="task_run",
authoritative_service="task-plane-18940",
authoritative_store="durable-task-store",
final_writer="task-plane-18940",
projections=("admin-web", "context-pack"),
),
StateOwnership(
object_type="release",
authoritative_service="release-controller",
authoritative_store="release-registry",
final_writer="release-controller",
projections=("baseline-authority", "admin-web"),
),
StateOwnership(
object_type="git_branch",
authoritative_service="authority-git-executor",
authoritative_store="authoritative-git",
final_writer="authority-git-executor",
projections=("gitea",),
),
StateOwnership(
object_type="external_projection",
authoritative_service="projection-controller",
authoritative_store="projection-store",
final_writer="projection-controller",
projections=("openapi", "gpt-action", "nginx"),
),
)
def ownership_index(
ownership: Iterable[StateOwnership] = DEFAULT_STATE_OWNERSHIP,
) -> dict[str, StateOwnership]:
result: dict[str, StateOwnership] = {}
for row in ownership:
if row.object_type in result:
raise ConvergenceContractError(f"duplicate_state_owner:{row.object_type}")
result[row.object_type] = row
return result
@dataclass(frozen=True, slots=True)
class AcceptanceCriterion:
criterion_id: str
required: bool
state: EvidenceState
verifier_id: str
evidence_refs: tuple[str, ...] = ()
def __post_init__(self) -> None:
_nonempty(self.criterion_id, "criterion_id")
_nonempty(self.verifier_id, "verifier_id")
@dataclass(frozen=True, slots=True)
class AcceptanceInvariant:
invariant_id: str
state: EvidenceState
evidence_refs: tuple[str, ...] = ()
def __post_init__(self) -> None:
_nonempty(self.invariant_id, "invariant_id")
@dataclass(frozen=True, slots=True)
class AcceptanceVector:
subject_type: str
subject_id: str
subject_revision: int
criteria: tuple[AcceptanceCriterion, ...]
invariants: tuple[AcceptanceInvariant, ...] = ()
unresolved_side_effects: tuple[str, ...] = ()
unresolved_blockers: tuple[str, ...] = ()
runtime_proofs: tuple[str, ...] = ()
schema: str = "aios.acceptance-vector.v1"
def __post_init__(self) -> None:
_nonempty(self.subject_type, "subject_type")
_nonempty(self.subject_id, "subject_id")
_generation(self.subject_revision, "subject_revision")
ids = [row.criterion_id for row in self.criteria]
if len(ids) != len(set(ids)):
raise ConvergenceContractError("duplicate_acceptance_criterion")
invariant_ids = [row.invariant_id for row in self.invariants]
if len(invariant_ids) != len(set(invariant_ids)):
raise ConvergenceContractError("duplicate_acceptance_invariant")
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
for row in value["criteria"]:
row["state"] = row["state"].value
for row in value["invariants"]:
row["state"] = row["state"].value
return value
@dataclass(frozen=True, slots=True)
class AcceptanceResult:
decision: AcceptanceDecision
reasons: tuple[str, ...]
vector_hash: str
def as_dict(self) -> dict[str, Any]:
return {
"decision": self.decision.value,
"reasons": list(self.reasons),
"vector_hash": self.vector_hash,
}
def evaluate_acceptance(vector: AcceptanceVector) -> AcceptanceResult:
"""Pure fail-closed acceptance evaluator.
The evaluator never writes a domain final state. The domain owner consumes this
result and remains the only final writer for Goal/Release/Product state.
"""
reasons: list[str] = []
required = [row for row in vector.criteria if row.required]
for row in required:
if row.state == EvidenceState.FAIL:
reasons.append(f"criterion_failed:{row.criterion_id}")
elif row.state == EvidenceState.BLOCKED:
reasons.append(f"criterion_blocked:{row.criterion_id}")
elif row.state == EvidenceState.UNKNOWN:
reasons.append(f"criterion_unknown:{row.criterion_id}")
for row in vector.invariants:
if row.state == EvidenceState.FAIL:
reasons.append(f"invariant_failed:{row.invariant_id}")
elif row.state in {EvidenceState.UNKNOWN, EvidenceState.BLOCKED}:
reasons.append(f"invariant_unknown:{row.invariant_id}")
if vector.unresolved_side_effects:
reasons.append("unresolved_side_effects")
if vector.unresolved_blockers:
reasons.append("unresolved_blockers")
digest = _sha256(vector.as_dict())
if any(reason.startswith(("criterion_failed:", "invariant_failed:")) for reason in reasons):
decision = AcceptanceDecision.REJECT
elif any(reason.startswith("criterion_blocked:") for reason in reasons) or vector.unresolved_blockers:
decision = AcceptanceDecision.BLOCKED
elif reasons:
decision = AcceptanceDecision.UNKNOWN
else:
decision = AcceptanceDecision.ACCEPT
return AcceptanceResult(decision=decision, reasons=tuple(reasons), vector_hash=digest)
@dataclass(frozen=True, slots=True)
class DesiredState:
resource_id: str
resource_type: str
desired_generation: int
desired_spec: Mapping[str, Any]
owner_goal_id: str | None = None
lifecycle: str = "active"
schema: str = "aios.desired-state.v1"
def __post_init__(self) -> None:
_nonempty(self.resource_id, "resource_id")
_nonempty(self.resource_type, "resource_type")
_generation(self.desired_generation, "desired_generation")
if self.lifecycle not in {"active", "suspended", "deleted"}:
raise ConvergenceContractError("desired_lifecycle_invalid")
def identity(self) -> tuple[str, int]:
return self.resource_id, self.desired_generation
@dataclass(frozen=True, slots=True)
class ObservedState:
resource_id: str
observed_generation: int | None
observed_spec: Mapping[str, Any]
health: ObservedHealth
source_identity: str
observed_at: str
evidence_refs: tuple[str, ...] = ()
schema: str = "aios.observed-state.v1"
def __post_init__(self) -> None:
_nonempty(self.resource_id, "resource_id")
_nonempty(self.source_identity, "source_identity")
_nonempty(self.observed_at, "observed_at")
if self.observed_generation is not None:
_generation(self.observed_generation, "observed_generation")
@dataclass(frozen=True, slots=True)
class ReconciliationPlan:
resource_id: str
desired_generation: int
state: ReconciliationState
diff: Mapping[str, Any]
observation_required: bool
idempotency_key: str
reason: str
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
value["state"] = self.state.value
return value
def plan_reconciliation(
desired: DesiredState,
observed: ObservedState | None,
) -> ReconciliationPlan:
"""Return a side-effect-free reconciliation decision.
UNKNOWN/missing observed state always requires observation and never converges.
"""
identity = {
"resource_id": desired.resource_id,
"desired_generation": desired.desired_generation,
"desired_spec": desired.desired_spec,
}
key = _sha256(identity)
if observed is None:
return ReconciliationPlan(
desired.resource_id,
desired.desired_generation,
ReconciliationState.OBSERVING,
{"observed": None},
True,
key,
"observed_state_missing",
)
if observed.resource_id != desired.resource_id:
raise ConvergenceContractError("resource_identity_mismatch")
if observed.health == ObservedHealth.UNKNOWN:
return ReconciliationPlan(
desired.resource_id,
desired.desired_generation,
ReconciliationState.OBSERVING,
{"health": observed.health.value},
True,
key,
"observed_state_unknown",
)
same_generation = observed.observed_generation == desired.desired_generation
same_spec = dict(observed.observed_spec) == dict(desired.desired_spec)
if same_generation and same_spec and observed.health == ObservedHealth.HEALTHY:
return ReconciliationPlan(
desired.resource_id,
desired.desired_generation,
ReconciliationState.CONVERGED,
{},
False,
key,
"desired_equals_observed",
)
return ReconciliationPlan(
desired.resource_id,
desired.desired_generation,
ReconciliationState.DIFF_FOUND,
{
"desired_generation": desired.desired_generation,
"observed_generation": observed.observed_generation,
"desired_spec": dict(desired.desired_spec),
"observed_spec": dict(observed.observed_spec),
"observed_health": observed.health.value,
},
False,
key,
"desired_observed_drift",
)
@dataclass(frozen=True, slots=True)
class SideEffectReceipt:
mutation_id: str
external_system: str
state: str
observed_at: str
external_request_id: str | None = None
external_object_id: str | None = None
observed_result_hash: str | None = None
evidence_refs: tuple[str, ...] = ()
schema: str = "aios.side-effect-receipt.v1"
def __post_init__(self) -> None:
_nonempty(self.mutation_id, "mutation_id")
_nonempty(self.external_system, "external_system")
_nonempty(self.observed_at, "observed_at")
if self.state not in {
"confirmed_applied",
"confirmed_not_applied",
"unknown",
}:
raise ConvergenceContractError("side_effect_receipt_state_invalid")
@dataclass(frozen=True, slots=True)
class ContinuationManifest:
trace_id: str
goal_id: str
goal_revision: int
task_id: str
run_id: str
current_node_id: str
acceptance_vector_ref: str
next_action: str
stop_reason: ContinuationStopReason
checkpoint_ref: str | None = None
unresolved_blocker_refs: tuple[str, ...] = ()
authoritative_decision_refs: tuple[str, ...] = ()
resource_refs: tuple[str, ...] = ()
artifact_refs: tuple[str, ...] = ()
evidence_refs: tuple[str, ...] = ()
created_at: str = field(default_factory=_utc_now)
schema: str = "aios.continuation-manifest.v1"
def __post_init__(self) -> None:
for name in (
"trace_id",
"goal_id",
"task_id",
"run_id",
"current_node_id",
"acceptance_vector_ref",
"next_action",
"created_at",
):
_nonempty(getattr(self, name), name)
_generation(self.goal_revision, "goal_revision")
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
value["stop_reason"] = self.stop_reason.value
return value
@property
def content_hash(self) -> str:
return _sha256(self.as_dict())
def validate_state_ownership(
object_type: str,
writer: str,
ownership: Iterable[StateOwnership] = DEFAULT_STATE_OWNERSHIP,
*,
final_state: bool = False,
) -> None:
row = ownership_index(ownership).get(object_type)
if row is None:
raise ConvergenceContractError(f"state_owner_missing:{object_type}")
expected = row.final_writer if final_state and row.final_writer else row.authoritative_service
if writer != expected:
raise ConvergenceContractError(
f"STATE_OWNERSHIP_VIOLATION:{object_type}:{writer}:{expected}"
)
@@ -0,0 +1,78 @@
"""Metadata-only, server-side credential lease broker."""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
import os
from pathlib import Path
import secrets
from typing import Any, Callable, Mapping
@dataclass(frozen=True, slots=True)
class CredentialLease:
lease_id: str
task_id: str
repository: str
target_branch: str
generation: int
issued_at: str
expires_at: str
scopes: tuple[str, ...]
provider: str
state: str = "ACTIVE"
def as_dict(self) -> dict[str, Any]:
return {"lease_id": self.lease_id, "task_id": self.task_id, "repository": self.repository, "target_branch": self.target_branch, "generation": self.generation, "issued_at": self.issued_at, "expires_at": self.expires_at, "scopes": list(self.scopes), "provider": self.provider, "state": self.state}
class FileCredentialProvider:
def __init__(self, token_path: str | os.PathLike[str], *, actor: str = "aios-autonomy-agent") -> None:
self.token_path = Path(token_path)
self.actor = actor
def environment(self, lease: CredentialLease) -> dict[str, str]:
if not self.token_path.is_file():
raise RuntimeError("CREDENTIAL_PROVIDER_UNAVAILABLE")
return {"CREDENTIAL_FILE": str(self.token_path), "GITEA_MACHINE_ACTOR": self.actor}
class CredentialBroker:
def __init__(self, provider: Any, *, now: Callable[[], datetime] | None = None) -> None:
self.provider = provider
self.now = now or (lambda: datetime.now(timezone.utc))
self._leases: dict[str, CredentialLease] = {}
def issue(self, *, task_id: str, repository: str, target_branch: str, generation: int, scopes: tuple[str, ...] = ("git:push", "gitea:read"), lease_seconds: int = 900) -> dict[str, Any]:
if repository != "aios/aios" or not target_branch.startswith("agent/") or generation < 1 or not 1 <= lease_seconds <= 86400:
raise ValueError("CREDENTIAL_LEASE_BINDING_INVALID")
now = self.now().astimezone(timezone.utc)
lease = CredentialLease("lease-" + secrets.token_hex(16), task_id, repository, target_branch, generation, now.isoformat(), (now + timedelta(seconds=lease_seconds)).isoformat(), tuple(scopes), type(self.provider).__name__)
self._leases[lease.lease_id] = lease
return lease.as_dict()
def _get(self, lease_id: str) -> CredentialLease:
lease = self._leases.get(lease_id)
if lease is None:
raise KeyError("CREDENTIAL_LEASE_NOT_FOUND")
if lease.state == "ACTIVE" and self.now().astimezone(timezone.utc) >= datetime.fromisoformat(lease.expires_at):
lease = CredentialLease(**{**lease.as_dict(), "scopes": tuple(lease.scopes), "state": "EXPIRED"})
self._leases[lease_id] = lease
return lease
def environment(self, lease_id: str) -> Mapping[str, str]:
lease = self._get(lease_id)
if lease.state != "ACTIVE":
raise RuntimeError("CREDENTIAL_LEASE_NOT_ACTIVE")
return self.provider.environment(lease)
def status(self, lease_id: str) -> dict[str, Any]:
return self._get(lease_id).as_dict()
def revoke(self, lease_id: str) -> dict[str, Any]:
lease = self._get(lease_id)
if lease.state == "REVOKED":
return lease.as_dict()
updated = CredentialLease(**{**lease.as_dict(), "scopes": tuple(lease.scopes), "state": "REVOKED"})
self._leases[lease_id] = updated
return updated.as_dict()
@@ -0,0 +1,125 @@
"""Non-Owner Gitea machine capability and branch reconciliation boundary."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Mapping, Protocol
class MachineBrokerError(RuntimeError):
pass
class GiteaTransport(Protocol):
def compare_and_swap_branch(self, branch: str, expected_oid: str, new_oid: str) -> str: ...
def create_or_update_pr(self, branch: str, target: str, title: str, body: str) -> Mapping[str, Any]: ...
def publish_status(self, branch: str, state: str, description: str) -> Mapping[str, Any]: ...
def protected_merge(self, pr_number: int) -> Mapping[str, Any]: ...
def read_branch(self, branch: str) -> str | None: ...
def whoami(self) -> Mapping[str, Any]: ...
def repository(self, owner: str, name: str) -> Mapping[str, Any]: ...
@dataclass(frozen=True, slots=True)
class MachineCredentialProjection:
"""Opaque systemd credential reference; the credential value is never accepted."""
reference: str
account: str
owner_account: str = "aiman"
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "MachineCredentialProjection":
if set(value) != {"scheme", "name", "account"}:
raise MachineBrokerError("CREDENTIAL_REFERENCE_INVALID")
if value["scheme"] != "systemd-credential" or value["name"] != "gitea-machine-token":
raise MachineBrokerError("CREDENTIAL_REFERENCE_INVALID")
return cls(f"{value['scheme']}:{value['name']}", str(value["account"]))
def __post_init__(self) -> None:
if self.reference != "systemd-credential:gitea-machine-token":
raise MachineBrokerError("CREDENTIAL_REFERENCE_INVALID")
if not self.account or self.account == self.owner_account:
raise MachineBrokerError("OWNER_MACHINE_IDENTITY_REJECTED")
@dataclass(frozen=True, slots=True)
class CampaignCapabilityReceipt:
campaign_id: str
task_id: str
repository: str
branch: str
expected_oid: str
capabilities: frozenset[str]
denied: frozenset[str]
def require(self, capability: str) -> None:
if capability in self.denied or capability not in self.capabilities:
raise MachineBrokerError("CAPABILITY_DENIED")
if self.branch in {"main", "master"}:
raise MachineBrokerError("PROTECTED_BRANCH_DENIED")
class GiteaMachineBroker:
def __init__(self, transport: GiteaTransport, credential: MachineCredentialProjection, receipt: CampaignCapabilityReceipt) -> None:
self.transport = transport
self.credential = credential
self.receipt = receipt
def verify_identity(self) -> Mapping[str, Any]:
identity = dict(self.transport.whoami())
login = identity.get("login") or identity.get("username")
if not isinstance(login, str) or not login or login == self.credential.owner_account or login != self.credential.account:
raise MachineBrokerError("GITEA_MACHINE_IDENTITY_INVALID")
owner, name = self.receipt.repository.split("/", 1)
repo = dict(self.transport.repository(owner, name))
if repo.get("full_name") != self.receipt.repository:
raise MachineBrokerError("GITEA_REPOSITORY_MISMATCH")
return {"login": login, "repository": self.receipt.repository, "owner_account": self.credential.owner_account}
def push_branch(self, new_oid: str) -> str:
self.receipt.require("exact_task_branch_push")
return self.transport.compare_and_swap_branch(self.receipt.branch, self.receipt.expected_oid, new_oid)
def create_pr(self, title: str, body: str) -> Mapping[str, Any]:
self.receipt.require("pr_create_update")
return self.transport.create_or_update_pr(self.receipt.branch, "main", title, body)
def status(self, state: str, description: str, context: str = "aios/durable-release") -> Mapping[str, Any]:
self.receipt.require("status_publish")
return self.transport.publish_status(self.receipt.branch, state, description, context)
def merge(self, pr_number: int) -> Mapping[str, Any]:
self.receipt.require("protected_merge")
return self.transport.protected_merge(pr_number)
def readback(self) -> str | None:
self.receipt.require("remote_readback")
return self.transport.read_branch(self.receipt.branch)
def reconcile_exact_branch(authority_oid: str, gitea_oid: str | None, expected_oid: str) -> str:
if authority_oid != expected_oid:
return "AUTHORITY_EXPECTATION_MISMATCH"
if gitea_oid is None:
return "CREATE_REQUIRED"
if gitea_oid == expected_oid:
return "ALREADY_IN_SYNC"
return "MANUAL_OWNER_REVIEW_REQUIRED"
def reconcile_fast_forward(
remote_oid: str | None,
expected_old_oid: str,
new_oid: str,
*,
old_is_ancestor_of_new: bool,
) -> str:
"""Return a bounded action for an active Job/lease branch scope."""
if remote_oid is None:
return "CREATE_REQUIRED"
if remote_oid == new_oid:
return "ALREADY_IN_SYNC"
if remote_oid == expected_old_oid and old_is_ancestor_of_new:
return "FAST_FORWARD_REQUIRED"
return "MANUAL_OWNER_REVIEW_REQUIRED"
@@ -0,0 +1,208 @@
"""Gitea HTTP/Git transport using only a projected systemd credential."""
from __future__ import annotations
from dataclasses import dataclass
import json
import os
from pathlib import Path
import subprocess
import tempfile
from typing import Any, Mapping
from urllib.error import HTTPError, URLError
from urllib.parse import quote, urlencode
from urllib.request import Request, urlopen
class GiteaTransportError(RuntimeError):
pass
def _safe_oid(value: str, field: str) -> str:
if not isinstance(value, str) or len(value) != 40 or any(ch not in "0123456789abcdef" for ch in value):
raise GiteaTransportError(f"{field}_INVALID")
return value
@dataclass(slots=True)
class GiteaMachineTransport:
api_base: str
repository_id: str
remote_url: str
authority_git_dir: Path
credential_name: str = "gitea-machine-token"
actor: str = "aios-autonomy-agent"
timeout: int = 20
expected_merge_head: str | None = None
expected_merge_base: str | None = None
def __post_init__(self) -> None:
if self.credential_name != "gitea-machine-token":
raise GiteaTransportError("CREDENTIAL_NAME_INVALID")
if "/" not in self.repository_id or self.repository_id.startswith("/"):
raise GiteaTransportError("REPOSITORY_INVALID")
if not self.api_base.startswith("https://") and not self.api_base.startswith("http://127.0.0.1"):
raise GiteaTransportError("GITEA_API_SCOPE_INVALID")
if not self.remote_url.startswith("https://"):
raise GiteaTransportError("GITEA_REMOTE_SCOPE_INVALID")
@property
def _credential_path(self) -> Path:
directory = os.environ.get("CREDENTIALS_DIRECTORY", "")
if not directory:
raise GiteaTransportError("CREDENTIAL_PROJECTION_MISSING")
path = Path(directory) / self.credential_name
if path.is_symlink() or not path.is_file():
raise GiteaTransportError("CREDENTIAL_PROJECTION_MISSING")
return path
def _token(self) -> str:
value = self._credential_path.read_text(encoding="utf-8").strip()
if not value:
raise GiteaTransportError("CREDENTIAL_EMPTY")
return value
def _request(self, method: str, path: str, body: Mapping[str, Any] | None = None) -> tuple[int, Any]:
data = None if body is None else json.dumps(dict(body), separators=(",", ":")).encode()
headers = {"Accept": "application/json", "Authorization": f"token {self._token()}"}
if data is not None:
headers["Content-Type"] = "application/json"
request = Request(self.api_base.rstrip("/") + path, data=data, headers=headers, method=method)
try:
with urlopen(request, timeout=self.timeout) as response:
raw = response.read()
return response.status, json.loads(raw.decode()) if raw else {}
except HTTPError as exc:
# Never include response bodies: Gitea may echo sensitive request context.
raise GiteaTransportError(f"GITEA_HTTP_{exc.code}") from exc
except (URLError, TimeoutError, OSError) as exc:
raise GiteaTransportError("GITEA_NETWORK_ERROR") from exc
def whoami(self) -> Mapping[str, Any]:
return self._request("GET", "/user")[1]
def repository_info(self) -> Mapping[str, Any]:
owner, name = self.repository_id.split("/", 1)
return self._request("GET", f"/repos/{quote(owner)}/{quote(name)}")[1]
def repository(self, owner: str, name: str) -> Mapping[str, Any]:
if f"{owner}/{name}" != self.repository_id:
raise GiteaTransportError("REPOSITORY_SCOPE_DENIED")
return self.repository_info()
def read_branch(self, branch: str) -> str | None:
owner, name = self.repository_id.split("/", 1)
try:
value = self._request("GET", f"/repos/{quote(owner)}/{quote(name)}/branches/{quote(branch, safe='/')}")[1]
except GiteaTransportError as exc:
if str(exc) == "GITEA_HTTP_404":
return None
raise
sha = value.get("commit", {}).get("id") if isinstance(value, dict) else None
return _safe_oid(sha, "remote_branch_oid")
def _askpass(self) -> tuple[Path, dict[str, str]]:
credential = self._credential_path
fd, name = tempfile.mkstemp(prefix="aios-gitea-askpass-", dir="/tmp")
path = Path(name)
os.fchmod(fd, 0o700)
script = "#!/bin/sh\ncase \"$1\" in *Username*) printf '%s\\n' \"$GITEA_MACHINE_ACTOR\" ;; *) cat \"$CREDENTIAL_FILE\" ;; esac\n"
with os.fdopen(fd, "w", encoding="utf-8") as stream:
stream.write(script)
stream.flush()
os.fsync(stream.fileno())
return path, {
"GIT_ASKPASS": str(path),
"GIT_TERMINAL_PROMPT": "0",
"GITEA_MACHINE_ACTOR": self.actor,
"CREDENTIAL_FILE": str(credential),
}
def _push_branch_fast_forward(self, branch: str, new_oid: str) -> None:
askpass, additions = self._askpass()
environment = os.environ.copy()
environment.update(additions)
try:
result = subprocess.run(
["git", f"--git-dir={self.authority_git_dir}", "push", self.remote_url, f"{new_oid}:refs/heads/{branch}"],
text=True,
capture_output=True,
check=False,
env=environment,
timeout=self.timeout,
)
if result.returncode != 0:
raise GiteaTransportError("GITEA_BRANCH_CREATE_FAILED")
finally:
askpass.unlink(missing_ok=True)
def compare_and_swap_branch(self, branch: str, expected_oid: str, new_oid: str) -> str:
_safe_oid(expected_oid, "expected_oid")
_safe_oid(new_oid, "new_oid")
current = self.read_branch(branch)
if current == new_oid:
return new_oid
if current is None:
self._push_branch_fast_forward(branch, new_oid)
readback = self.read_branch(branch)
if readback != new_oid:
raise GiteaTransportError("GITEA_BRANCH_CREATE_READBACK_MISMATCH")
return readback
if current != expected_oid:
raise GiteaTransportError("GITEA_BRANCH_EXPECTATION_MISMATCH")
# The preflight read binds the expected old tip. The Git receive-pack
# then enforces the actual fast-forward CAS while transferring the new
# objects; no force/lease override or arbitrary ref is permitted.
self._push_branch_fast_forward(branch, new_oid)
readback = self.read_branch(branch)
if readback != new_oid:
raise GiteaTransportError("GITEA_BRANCH_CAS_READBACK_MISMATCH")
return readback
def _pulls(self) -> list[Mapping[str, Any]]:
owner, name = self.repository_id.split("/", 1)
query = urlencode({"state": "all", "base_branch": "main", "limit": 100})
value = self._request("GET", f"/repos/{quote(owner)}/{quote(name)}/pulls?{query}")[1]
return value if isinstance(value, list) else []
def create_or_update_pr(self, branch: str, target: str, title: str, body: str) -> Mapping[str, Any]:
if target != "main":
raise GiteaTransportError("PR_BASE_SCOPE_DENIED")
head = self.read_branch(branch)
if head is None:
raise GiteaTransportError("PR_HEAD_MISSING")
matches = [p for p in self._pulls() if (p.get("head") or {}).get("ref") == branch and (p.get("base") or {}).get("ref") == target]
open_matches = [p for p in matches if p.get("state") == "open"]
if len(open_matches) > 1:
raise GiteaTransportError("MULTIPLE_MATCHING_PRS")
if open_matches:
if (open_matches[0].get("head") or {}).get("sha") != head:
raise GiteaTransportError("PR_HEAD_CONFLICT")
return open_matches[0]
owner, name = self.repository_id.split("/", 1)
return self._request("POST", f"/repos/{quote(owner)}/{quote(name)}/pulls", {"head": branch, "base": target, "title": title, "body": body, "allow_maintainer_edit": False})[1]
def publish_status(self, branch: str, state: str, description: str, context: str = "aios/durable-release") -> Mapping[str, Any]:
oid = self.read_branch(branch)
if oid is None:
raise GiteaTransportError("STATUS_BRANCH_MISSING")
owner, name = self.repository_id.split("/", 1)
return self._request("POST", f"/repos/{quote(owner)}/{quote(name)}/statuses/{oid}", {"state": state, "context": context, "description": description})[1]
def protected_merge(self, pr_number: int) -> Mapping[str, Any]:
if not isinstance(pr_number, int) or pr_number < 1:
raise GiteaTransportError("PR_NUMBER_INVALID")
owner, name = self.repository_id.split("/", 1)
pr = self._request("GET", f"/repos/{quote(owner)}/{quote(name)}/pulls/{pr_number}")[1]
if pr.get("state") != "open" or (pr.get("base") or {}).get("ref") != "main":
raise GiteaTransportError("PR_MERGE_BINDING_INVALID")
if pr.get("mergeable") is False:
raise GiteaTransportError("PR_NOT_MERGEABLE")
head_oid = (pr.get("head") or {}).get("sha")
base_oid = (pr.get("base") or {}).get("sha")
if self.expected_merge_head is not None and head_oid != self.expected_merge_head:
raise GiteaTransportError("PR_HEAD_EXPECTATION_MISMATCH")
if self.expected_merge_base is not None and base_oid != self.expected_merge_base:
raise GiteaTransportError("PR_BASE_EXPECTATION_MISMATCH")
body = {"do": "fast-forward-only", "force_merge": False, "merge_when_checks_succeed": False, "head_commit_id": head_oid}
return self._request("POST", f"/repos/{quote(owner)}/{quote(name)}/pulls/{pr_number}/merge", body)[1]
@@ -0,0 +1,93 @@
"""Canonical Gitea transport and slash-branch verification."""
from __future__ import annotations
from dataclasses import dataclass
import json
import os
from pathlib import Path
import subprocess
from typing import Any, Callable, Mapping, Optional
from urllib.parse import quote
from urllib.request import Request, urlopen
from aios_gitops_reconciliation_core import RemoteObservation
@dataclass(frozen=True, slots=True)
class GiteaPublisherConfig:
repository: str = "aios/aios"
api_base: str = "https://g.cnnetmkt.top/git/api/v1"
remote_name: str = "gitea-http"
class GiteaPublisher:
def __init__(self, workspace: Path, *, config: GiteaPublisherConfig = GiteaPublisherConfig(), credential: Any = None, api_get: Callable[[str], Any] | None = None, git_ref: Callable[[str], Optional[str]] | None = None) -> None:
self.workspace = Path(workspace)
self.config = config
self.credential = credential
self._api_get = api_get
self._git_ref = git_ref
@staticmethod
def validate_repository(repository: str) -> None:
if repository != "aios/aios":
raise ValueError("REPOSITORY_IDENTITY_MISMATCH")
@staticmethod
def validate_branch(branch: str) -> None:
if not isinstance(branch, str) or not branch.startswith("agent/") or branch in {"main", "master"}:
raise ValueError("BRANCH_SCOPE_DENIED")
def repository_identity(self) -> str:
self.validate_repository(self.config.repository)
return self.config.repository
def _environment(self) -> dict[str, str]:
if self.credential is not None and hasattr(self.credential, "environment"):
return dict(self.credential.environment())
return {}
def read_git_ref(self, branch: str) -> Optional[str]:
self.validate_branch(branch)
if self._git_ref is not None:
return self._git_ref(branch)
result = subprocess.run(["git", "-C", str(self.workspace), "ls-remote", self.config.remote_name, f"refs/heads/{branch}"], text=True, capture_output=True, check=False, env={**os.environ, **self._environment(), "GIT_TERMINAL_PROMPT": "0"})
if result.returncode != 0 or not result.stdout.strip():
return None
return result.stdout.split()[0]
def _api(self, path: str) -> Any:
if self._api_get is not None:
return self._api_get(path)
headers = {"Accept": "application/json"}
env = self._environment()
if env.get("AIOS_GITEA_TOKEN"):
headers["Authorization"] = "token " + env["AIOS_GITEA_TOKEN"]
with urlopen(Request(self.config.api_base.rstrip("/") + path, headers=headers), timeout=20) as response:
return json.loads(response.read() or b"{}")
def read_api_branch(self, branch: str) -> Optional[str]:
self.validate_branch(branch)
for page in range(1, 101):
value = self._api(f"/repos/aios/aios/branches?page={page}&limit=50")
if not isinstance(value, list):
return None
for item in value:
if isinstance(item, Mapping) and item.get("name") == branch:
commit = item.get("commit")
return commit.get("id") if isinstance(commit, Mapping) else None
if len(value) < 50:
return None
raise RuntimeError("GITEA_BRANCH_PAGINATION_LIMIT")
def commit_visible(self, oid: str) -> bool:
if not isinstance(oid, str) or len(oid) != 40:
return False
try:
value = self._api(f"/repos/aios/aios/git/commits/{quote(oid, safe='')}")
return isinstance(value, Mapping) and value.get("sha", oid) == oid
except Exception:
return False
def observe(self, branch: str, desired_oid: str, authority_oid: Optional[str], *, expected_oid: Optional[str] = None, observed_at: str = "") -> RemoteObservation:
return RemoteObservation(self.repository_identity(), branch, desired_oid, authority_oid, self.read_git_ref(branch), self.read_api_branch(branch), self.commit_visible(desired_oid), observed_at, expected_oid)
@@ -0,0 +1,919 @@
"""SQLite-backed durable GitOps Job Controller.
The controller owns durable intent, idempotency, lease metadata, state
transitions, receipts, and payload references. A worker owns Git side effects
behind the campaign capability and branch-CAS boundaries.
"""
from __future__ import annotations
from dataclasses import dataclass
from enum import Enum
import hashlib
import json
import os
from pathlib import Path
import sqlite3
import tempfile
from datetime import datetime, timezone
from typing import Any, Callable, Mapping
class JobError(RuntimeError):
pass
class ServiceUnavailable(JobError):
pass
class BindingConflict(JobError):
pass
class JobState(str, Enum):
REQUESTED = "REQUESTED"
AUTHORIZED = "AUTHORIZED"
LEASED = "LEASED"
WORKTREE_READY = "WORKTREE_READY"
EXECUTING = "EXECUTING"
LOCAL_VERIFIED = "LOCAL_VERIFIED"
REMOTE_SYNCING = "REMOTE_SYNCING"
REMOTE_VERIFIED = "REMOTE_VERIFIED"
COMPLETED = "COMPLETED"
FAILED = "FAILED"
CANCELLED = "CANCELLED"
RECOVERABLE_FAILURE = "RECOVERABLE_FAILURE"
RECONCILING = "RECONCILING"
RECONCILIATION_REQUIRED = "RECONCILIATION_REQUIRED"
MANUAL_OWNER_REVIEW_REQUIRED = "MANUAL_OWNER_REVIEW_REQUIRED"
_ORDER = {
JobState.REQUESTED: 0,
JobState.AUTHORIZED: 1,
JobState.LEASED: 2,
JobState.WORKTREE_READY: 3,
JobState.EXECUTING: 4,
JobState.LOCAL_VERIFIED: 5,
JobState.REMOTE_SYNCING: 6,
JobState.REMOTE_VERIFIED: 7,
JobState.COMPLETED: 8,
}
_RECOVERABLE = {JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}
_TERMINAL = {
JobState.COMPLETED,
JobState.FAILED,
JobState.CANCELLED,
JobState.MANUAL_OWNER_REVIEW_REQUIRED,
}
def utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def canonical_json(value: Any) -> bytes:
return (
json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + "\n"
).encode("utf-8")
def sha256_json(value: Any) -> str:
return hashlib.sha256(canonical_json(value)).hexdigest()
class PayloadStore:
"""Content-addressed filesystem payloads; raw content never enters SQLite."""
def __init__(self, root: str | os.PathLike[str]) -> None:
self.root = Path(root)
if not self.root.is_absolute() or self.root.is_symlink():
raise JobError("PAYLOAD_ROOT_INVALID")
self.root.mkdir(parents=True, exist_ok=True, mode=0o700)
os.chmod(self.root, 0o700)
def put(self, content: bytes, expected_digest: str) -> tuple[str, int]:
digest = hashlib.sha256(content).hexdigest()
if digest != expected_digest:
raise JobError("PAYLOAD_DIGEST_MISMATCH")
if len(expected_digest) != 64 or any(ch not in "0123456789abcdef" for ch in expected_digest):
raise JobError("PAYLOAD_DIGEST_INVALID")
target = self.root / expected_digest
if target.is_symlink():
raise JobError("PAYLOAD_PATH_SYMLINK")
if target.is_file():
if target.stat().st_size != len(content) or hashlib.sha256(target.read_bytes()).hexdigest() != digest:
raise JobError("PAYLOAD_EXISTING_MISMATCH")
return f"sha256/{digest}", len(content)
fd, temporary = tempfile.mkstemp(prefix=".payload-", dir=self.root)
try:
os.fchmod(fd, 0o600)
with os.fdopen(fd, "wb") as stream:
stream.write(content)
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary, target)
os.chmod(target, 0o600)
finally:
if os.path.exists(temporary):
os.unlink(temporary)
return f"sha256/{digest}", len(content)
def read_verified(self, reference: str, expected_digest: str, expected_size: int) -> bytes:
if reference != f"sha256/{expected_digest}":
raise JobError("PAYLOAD_REFERENCE_MISMATCH")
path = self.root / expected_digest
if path.is_symlink() or not path.is_file():
raise JobError("PAYLOAD_MISSING")
content = path.read_bytes()
if len(content) != expected_size or hashlib.sha256(content).hexdigest() != expected_digest:
raise JobError("PAYLOAD_VERIFY_FAILED")
return content
@dataclass(frozen=True, slots=True)
class JobReceipt:
job_id: str
idempotency_key: str
state: JobState
side_effect_state: str
http_status: int
revision: int
receipt: Mapping[str, Any] | None = None
def as_dict(self) -> dict[str, Any]:
return {
"job_id": self.job_id,
"idempotency_key": self.idempotency_key,
"state": self.state.value,
"side_effect_state": self.side_effect_state,
"http_status": self.http_status,
"revision": self.revision,
"receipt": dict(self.receipt) if self.receipt else None,
}
class JobStore:
SCHEMA_VERSION = 1
def __init__(self, root: str | os.PathLike[str], *, payload_root: str | os.PathLike[str] | None = None) -> None:
self.root = Path(root)
if not self.root.is_absolute() or self.root.is_symlink():
raise JobError("JOB_ROOT_INVALID")
self.root.mkdir(parents=True, exist_ok=True, mode=0o700)
os.chmod(self.root, 0o700)
self.db_path = self.root / "state.sqlite3"
self.payloads = PayloadStore(payload_root or self.root.parent / "payloads" / "sha256")
self._initialize()
def _connect(self) -> sqlite3.Connection:
if self.db_path.is_symlink():
raise ServiceUnavailable("JOB_DB_SYMLINK")
connection = sqlite3.connect(self.db_path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA synchronous=FULL")
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@classmethod
def for_existing_append(cls, root: str | os.PathLike[str]) -> "JobStore":
"""Open an existing job database without initialization side effects."""
path = Path(root)
if (
not path.is_absolute()
or path.is_symlink()
or not path.is_dir()
or not (path / "state.sqlite3").is_file()
or (path / "state.sqlite3").is_symlink()
):
raise JobError("JOB_ROOT_NOT_FOUND")
instance = cls.__new__(cls)
instance.root = path
instance.db_path = path / "state.sqlite3"
return instance
@staticmethod
def _ensure_completed_replay_schema(connection: sqlite3.Connection) -> None:
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS completed_replay_reconciliations (
reconciliation_id TEXT PRIMARY KEY, job_id TEXT NOT NULL,
identity_digest TEXT NOT NULL UNIQUE,
source_lease_epoch INTEGER NOT NULL,
task_id TEXT NOT NULL, external_job_id TEXT NOT NULL,
internal_job_id TEXT NOT NULL, repository TEXT NOT NULL,
target_branch TEXT NOT NULL, generation INTEGER NOT NULL,
branch_mode TEXT NOT NULL, payload_digest TEXT NOT NULL,
files_digest TEXT NOT NULL, message_digest TEXT NOT NULL,
commit_oid TEXT NOT NULL, authority_head_oid TEXT NOT NULL,
current_head_oid TEXT NOT NULL, historical_lease_head_oid TEXT NOT NULL,
worker_principal TEXT NOT NULL, worker_lineage TEXT NOT NULL,
durable_revision INTEGER NOT NULL, completion_identity TEXT NOT NULL,
reason TEXT NOT NULL, provenance_json TEXT NOT NULL,
created_at TEXT NOT NULL, FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE INDEX IF NOT EXISTS completed_replay_job_idx
ON completed_replay_reconciliations(job_id);
CREATE TRIGGER IF NOT EXISTS completed_replay_no_update
BEFORE UPDATE ON completed_replay_reconciliations
BEGIN SELECT RAISE(ABORT, 'COMPLETED_REPLAY_APPEND_ONLY'); END;
CREATE TRIGGER IF NOT EXISTS completed_replay_no_delete
BEFORE DELETE ON completed_replay_reconciliations
BEGIN SELECT RAISE(ABORT, 'COMPLETED_REPLAY_APPEND_ONLY'); END;
"""
)
columns = {
str(item[1]) for item in connection.execute(
"PRAGMA table_info(completed_replay_reconciliations)"
).fetchall()
}
if columns and "historical_lease_head_oid" not in columns:
connection.execute(
"ALTER TABLE completed_replay_reconciliations ADD COLUMN historical_lease_head_oid TEXT"
)
def _initialize(self) -> None:
try:
with self._connect() as connection:
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS metadata (key TEXT PRIMARY KEY, value TEXT NOT NULL);
CREATE TABLE IF NOT EXISTS jobs (
job_id TEXT PRIMARY KEY, idempotency_key TEXT NOT NULL UNIQUE,
campaign_id TEXT NOT NULL, repository TEXT NOT NULL, operation TEXT NOT NULL,
branch TEXT NOT NULL, expected_branch_oid TEXT NOT NULL,
payload_digest TEXT NOT NULL, payload_ref TEXT, payload_size INTEGER,
state TEXT NOT NULL, side_effect_state TEXT NOT NULL,
revision INTEGER NOT NULL, request_json TEXT NOT NULL,
receipt_json TEXT, created_at TEXT NOT NULL, updated_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS idempotency_keys (
key TEXT PRIMARY KEY, job_id TEXT NOT NULL UNIQUE,
binding_json TEXT NOT NULL, created_at TEXT NOT NULL,
FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE TABLE IF NOT EXISTS branch_leases (
lease_id TEXT PRIMARY KEY, job_id TEXT NOT NULL UNIQUE,
branch TEXT NOT NULL, lineage_anchor_oid TEXT NOT NULL,
current_head_oid TEXT NOT NULL, generation INTEGER NOT NULL,
lease_expires_at TEXT NOT NULL, state TEXT NOT NULL,
revision INTEGER NOT NULL, FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE TABLE IF NOT EXISTS state_transitions (
sequence INTEGER PRIMARY KEY AUTOINCREMENT, job_id TEXT NOT NULL,
from_state TEXT, to_state TEXT NOT NULL, revision INTEGER NOT NULL,
side_effect_state TEXT NOT NULL, created_at TEXT NOT NULL,
FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE TABLE IF NOT EXISTS receipts (
receipt_id TEXT PRIMARY KEY, job_id TEXT NOT NULL,
kind TEXT NOT NULL, digest TEXT NOT NULL, payload_json TEXT NOT NULL,
created_at TEXT NOT NULL, FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE TABLE IF NOT EXISTS completed_replay_reconciliations (
reconciliation_id TEXT PRIMARY KEY, job_id TEXT NOT NULL,
identity_digest TEXT NOT NULL UNIQUE,
source_lease_epoch INTEGER NOT NULL,
task_id TEXT NOT NULL, external_job_id TEXT NOT NULL,
internal_job_id TEXT NOT NULL, repository TEXT NOT NULL,
target_branch TEXT NOT NULL, generation INTEGER NOT NULL,
branch_mode TEXT NOT NULL, payload_digest TEXT NOT NULL,
files_digest TEXT NOT NULL, message_digest TEXT NOT NULL,
commit_oid TEXT NOT NULL, authority_head_oid TEXT NOT NULL,
current_head_oid TEXT NOT NULL, historical_lease_head_oid TEXT NOT NULL,
worker_principal TEXT NOT NULL, worker_lineage TEXT NOT NULL,
durable_revision INTEGER NOT NULL, completion_identity TEXT NOT NULL,
reason TEXT NOT NULL, provenance_json TEXT NOT NULL,
created_at TEXT NOT NULL, FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE INDEX IF NOT EXISTS completed_replay_job_idx
ON completed_replay_reconciliations(job_id);
CREATE TRIGGER IF NOT EXISTS completed_replay_no_update
BEFORE UPDATE ON completed_replay_reconciliations
BEGIN SELECT RAISE(ABORT, 'COMPLETED_REPLAY_APPEND_ONLY'); END;
CREATE TRIGGER IF NOT EXISTS completed_replay_no_delete
BEFORE DELETE ON completed_replay_reconciliations
BEGIN SELECT RAISE(ABORT, 'COMPLETED_REPLAY_APPEND_ONLY'); END;
INSERT OR REPLACE INTO metadata(key, value) VALUES('schema_version', '1');
"""
)
columns = {
str(item[1]) for item in connection.execute(
"PRAGMA table_info(completed_replay_reconciliations)"
).fetchall()
}
if columns and "historical_lease_head_oid" not in columns:
connection.execute(
"ALTER TABLE completed_replay_reconciliations "
"ADD COLUMN historical_lease_head_oid TEXT"
)
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
def _mirror(self, job: Mapping[str, Any]) -> None:
directory = self.root / str(job["job_id"])
directory.mkdir(mode=0o700, exist_ok=True)
(directory / "receipts").mkdir(mode=0o700, exist_ok=True)
(directory / "logs-digest").mkdir(mode=0o700, exist_ok=True)
for name, value in (("request.json", job["request"]), ("state.json", job)):
target = directory / name
fd, temporary = tempfile.mkstemp(prefix=".mirror-", dir=directory)
try:
os.fchmod(fd, 0o600)
with os.fdopen(fd, "wb") as stream:
stream.write(canonical_json(value))
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary, target)
finally:
if os.path.exists(temporary):
os.unlink(temporary)
def read(self, job_id: str) -> dict[str, Any]:
try:
with self._connect() as connection:
row = connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone()
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
if row is None:
raise JobError("JOB_NOT_FOUND")
return dict(row)
@staticmethod
def _completed_replay_identity(envelope: Mapping[str, Any]) -> dict[str, Any]:
required = (
"task_id", "external_job_id", "internal_job_id", "repository",
"target_branch", "generation", "branch_mode", "payload_digest",
"files_digest", "message_digest", "commit_oid", "authority_head_oid",
"historical_lease_head_oid", "worker_principal", "worker_lineage", "source_lease_epoch",
"durable_revision", "completion_identity", "reason", "provenance",
)
if any(key not in envelope for key in required):
raise BindingConflict("COMPLETED_REPLAY_BINDING_INCOMPLETE")
text_fields = (
"task_id", "external_job_id", "internal_job_id", "repository",
"target_branch", "branch_mode", "payload_digest", "files_digest",
"message_digest", "commit_oid", "authority_head_oid", "worker_principal",
"worker_lineage", "historical_lease_head_oid", "completion_identity", "reason",
)
result: dict[str, Any] = {}
for key in text_fields:
value = envelope[key]
if not isinstance(value, str) or not value.strip():
raise BindingConflict("COMPLETED_REPLAY_BINDING_INVALID")
result[key] = value
for key in ("generation", "source_lease_epoch", "durable_revision"):
value = envelope[key]
if isinstance(value, bool) or not isinstance(value, int) or value < 1:
raise BindingConflict("COMPLETED_REPLAY_BINDING_INVALID")
result[key] = value
for key in ("commit_oid", "authority_head_oid", "historical_lease_head_oid"):
if len(result[key]) != 40 or any(ch not in "0123456789abcdef" for ch in result[key]):
raise BindingConflict("COMPLETED_REPLAY_BINDING_INVALID")
for key in ("payload_digest", "files_digest", "message_digest"):
if len(result[key]) != 64 or any(ch not in "0123456789abcdef" for ch in result[key]):
raise BindingConflict("COMPLETED_REPLAY_BINDING_INVALID")
if result["repository"] != "aios/aios" or result["branch_mode"] not in {"CREATE_NEW", "CONTINUE_EXISTING"}:
raise BindingConflict("COMPLETED_REPLAY_BINDING_INVALID")
if not result["target_branch"].startswith("agent/") or result["target_branch"] in {"main", "master"}:
raise BindingConflict("COMPLETED_REPLAY_BINDING_INVALID")
provenance = envelope["provenance"]
if not isinstance(provenance, Mapping) or not provenance:
raise BindingConflict("COMPLETED_REPLAY_PROVENANCE_INVALID")
result["provenance"] = dict(provenance)
if result["commit_oid"] != result["authority_head_oid"]:
raise BindingConflict("COMPLETED_REPLAY_AUTHORITY_CONFLICT")
return result
def append_completed_replay_reconciliation(
self,
job_id: str,
envelope: Mapping[str, Any],
*,
authority_head_oid: str,
) -> dict[str, Any]:
"""Append exact completed-replay evidence without rewriting history."""
identity = self._completed_replay_identity(envelope)
if authority_head_oid != identity["authority_head_oid"]:
raise BindingConflict("COMPLETED_REPLAY_AUTHORITY_CONFLICT")
if identity["internal_job_id"] != job_id:
raise BindingConflict("COMPLETED_REPLAY_INTERNAL_JOB_CONFLICT")
identity_digest = sha256_json(identity)
reconciliation_id = "reconcile-" + identity_digest[:32]
now = utc_now()
try:
with self._connect() as connection:
self._ensure_completed_replay_schema(connection)
connection.execute("BEGIN IMMEDIATE")
row = connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone()
if row is None:
raise JobError("JOB_NOT_FOUND")
if JobState(row["state"]) != JobState.COMPLETED:
raise BindingConflict("COMPLETED_REPLAY_JOB_NOT_COMPLETED")
durable_request = json.loads(row["request_json"])
binding = durable_request.get("binding", durable_request)
exact_binding = {
"task_id": binding.get("task_id") or binding.get("campaign_id"),
"external_job_id": binding.get("external_job_id"),
"internal_job_id": job_id,
"repository": binding.get("repository") or row["repository"],
"target_branch": binding.get("target_branch") or binding.get("branch"),
"generation": binding.get("generation"),
"branch_mode": binding.get("branch_mode"),
"payload_digest": binding.get("payload_digest") or row["payload_digest"],
"files_digest": binding.get("files_digest"),
"message_digest": binding.get("message_digest"),
"worker_principal": binding.get("worker_principal"),
"worker_lineage": binding.get("worker_lineage"),
}
for key, value in exact_binding.items():
if str(value) != str(identity[key]):
raise BindingConflict("COMPLETED_REPLAY_BINDING_CONFLICT")
receipt = json.loads(row["receipt_json"]) if row["receipt_json"] else None
if not isinstance(receipt, Mapping):
raise BindingConflict("COMPLETED_REPLAY_RECEIPT_MISSING")
if receipt.get("commit_oid") != identity["commit_oid"] or receipt.get("authority_oid") != identity["authority_head_oid"]:
raise BindingConflict("COMPLETED_REPLAY_RECEIPT_AUTHORITY_CONFLICT")
completion_identity = sha256_json({
"job_id": row["job_id"],
"revision": int(row["revision"]),
"receipt": receipt,
})
if identity["completion_identity"] != completion_identity:
raise BindingConflict("COMPLETED_REPLAY_COMPLETION_IDENTITY_CONFLICT")
lease_columns = {
str(item[1]) for item in connection.execute("PRAGMA table_info(branch_leases)").fetchall()
}
if "epoch" in lease_columns:
lease = connection.execute(
"SELECT lease_id, state, current_head_oid, generation FROM branch_leases WHERE job_id=? AND epoch=? ORDER BY rowid DESC LIMIT 1",
(job_id, identity["source_lease_epoch"]),
).fetchone()
else:
if identity["source_lease_epoch"] != 1:
raise BindingConflict("COMPLETED_REPLAY_LEASE_EPOCH_UNAVAILABLE")
lease = connection.execute(
"SELECT lease_id, state, current_head_oid, generation FROM branch_leases WHERE job_id=? ORDER BY rowid DESC LIMIT 1",
(job_id,),
).fetchone()
if lease is None:
raise BindingConflict("COMPLETED_REPLAY_LEASE_MISSING")
lease_state = str(lease["state"])
lease_head = str(lease["current_head_oid"])
lease_generation = int(lease["generation"])
event_tables = {
str(item[0]) for item in connection.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name='branch_lease_events'"
).fetchall()
}
if "branch_lease_events" in event_tables:
event = connection.execute(
"SELECT state, current_head_oid, generation FROM branch_lease_events WHERE lease_id=? ORDER BY rowid DESC LIMIT 1",
(lease["lease_id"],),
).fetchone()
if event is not None:
lease_state = str(event["state"])
lease_head = str(event["current_head_oid"])
lease_generation = int(event["generation"])
if lease_state not in {"ACTIVE", "USED", "EXPIRED"}:
raise BindingConflict("COMPLETED_REPLAY_LEASE_STATE_INVALID")
if lease_generation != identity["generation"]:
raise BindingConflict("COMPLETED_REPLAY_LEASE_GENERATION_CONFLICT")
if len(lease_head) != 40 or any(ch not in "0123456789abcdef" for ch in lease_head):
raise BindingConflict("COMPLETED_REPLAY_LEASE_HEAD_INVALID")
if lease_head != identity["historical_lease_head_oid"]:
raise BindingConflict("COMPLETED_REPLAY_HISTORICAL_LEASE_CONFLICT")
if identity["commit_oid"] == identity["historical_lease_head_oid"]:
raise BindingConflict("COMPLETED_REPLAY_EVIDENCE_INCOHERENT")
existing = connection.execute(
"SELECT * FROM completed_replay_reconciliations WHERE job_id=? ORDER BY created_at ASC",
(job_id,),
).fetchall()
for prior in existing:
if prior["identity_digest"] != identity_digest:
raise BindingConflict("COMPLETED_REPLAY_AMBIGUOUS")
if existing:
result = dict(existing[0])
connection.commit()
return result
provenance_json = json.dumps(identity["provenance"], sort_keys=True, separators=(",", ":"))
connection.execute(
"INSERT INTO completed_replay_reconciliations(reconciliation_id,job_id,identity_digest,source_lease_epoch,task_id,external_job_id,internal_job_id,repository,target_branch,generation,branch_mode,payload_digest,files_digest,message_digest,commit_oid,authority_head_oid,worker_principal,worker_lineage,current_head_oid,historical_lease_head_oid,durable_revision,completion_identity,reason,provenance_json,created_at) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
(
reconciliation_id, job_id, identity["identity_digest"] if "identity_digest" in identity else identity_digest, identity["source_lease_epoch"],
identity["task_id"], identity["external_job_id"], identity["internal_job_id"],
identity["repository"], identity["target_branch"], identity["generation"],
identity["branch_mode"], identity["payload_digest"], identity["files_digest"],
identity["message_digest"], identity["commit_oid"], identity["authority_head_oid"],
identity["worker_principal"], identity["worker_lineage"], identity["authority_head_oid"], identity["historical_lease_head_oid"], identity["durable_revision"],
identity["completion_identity"], identity["reason"], provenance_json, now,
),
)
result = dict(connection.execute(
"SELECT * FROM completed_replay_reconciliations WHERE reconciliation_id=?",
(reconciliation_id,),
).fetchone())
connection.commit()
return result
except (JobError, BindingConflict):
raise
except sqlite3.IntegrityError as exc:
try:
with self._connect() as connection:
existing = connection.execute(
"SELECT * FROM completed_replay_reconciliations WHERE identity_digest=?",
(identity_digest,),
).fetchone()
if existing is not None:
return dict(existing)
except sqlite3.Error:
pass
raise BindingConflict("COMPLETED_REPLAY_IDEMPOTENCY_CONFLICT") from exc
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
def completed_replay_evidence(self, job_id: str) -> dict[str, Any] | None:
try:
with self._connect() as connection:
rows = connection.execute(
"SELECT * FROM completed_replay_reconciliations WHERE job_id=? ORDER BY created_at ASC",
(job_id,),
).fetchall()
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
if not rows:
return None
if len({str(row["identity_digest"]) for row in rows}) != 1:
raise BindingConflict("COMPLETED_REPLAY_AMBIGUOUS")
first = dict(rows[0])
immutable_fields = tuple(
key for key in first
if key != "created_at"
)
for row in rows[1:]:
if any(row[key] != first[key] for key in immutable_fields):
raise BindingConflict("COMPLETED_REPLAY_AMBIGUOUS")
if not (
first["commit_oid"] == first["authority_head_oid"] == first["current_head_oid"]
and first["commit_oid"] != first["historical_lease_head_oid"]
):
raise BindingConflict("COMPLETED_REPLAY_EVIDENCE_INCOHERENT")
return first
def _receipt(self, row: Mapping[str, Any], *, duplicate: bool) -> JobReceipt:
state = JobState(row["state"])
if state == JobState.COMPLETED:
status, side = 200, "COMPLETED"
elif state in _RECOVERABLE:
status = 202
side = row["side_effect_state"] if row["side_effect_state"] == "REMOTE_PUSH_PENDING" else "RESUME_SAME_JOB"
elif row["side_effect_state"] == "UNKNOWN":
status, side = 503, "UNKNOWN"
else:
status, side = 202, "SAME_JOB" if duplicate else "QUEUED"
receipt = json.loads(row["receipt_json"]) if row["receipt_json"] else None
return JobReceipt(row["job_id"], row["idempotency_key"], state, side, status, int(row["revision"]), receipt)
class GitOpsJobController:
def __init__(self, store: JobStore) -> None:
self.store = store
@staticmethod
def _binding(request: Mapping[str, Any]) -> dict[str, str]:
fields = ("campaign_id", "repository", "operation", "branch", "expected_branch_oid", "payload_digest")
if any(not isinstance(request.get(field), str) or not request[field].strip() for field in fields):
raise JobError("JOB_BINDING_INVALID")
oid = request["expected_branch_oid"]
digest = request["payload_digest"]
if len(oid) != 40 or any(ch not in "0123456789abcdef" for ch in oid):
raise JobError("EXPECTED_BRANCH_OID_INVALID")
if len(digest) != 64 or any(ch not in "0123456789abcdef" for ch in digest):
raise JobError("PAYLOAD_DIGEST_INVALID")
if request["branch"] in {"main", "master"} or request["branch"].startswith(("refs/", "tags/")):
raise JobError("PROTECTED_BRANCH_REJECTED")
binding = {field: str(request[field]) for field in fields}
# Operation-specific, non-secret binding inputs must survive the
# durable round trip and participate in idempotency. Arbitrary request
# fields are intentionally discarded rather than persisted.
optional = (
"idempotency_key",
"task_id",
"new_branch_oid",
"authorization_recovery_digest",
"lineage_anchor_oid",
"base_oid",
"branch_mode",
"generation",
"expected_main_oid",
"pr_number",
"target_branch",
"check_context",
"check_state",
"check_description",
"external_job_id",
"files_digest",
"message_digest",
"worker_principal",
"worker_lineage",
)
for field in optional:
value = request.get(field)
if value is None:
continue
if isinstance(value, bool) or not isinstance(value, (str, int)):
raise JobError("JOB_BINDING_OPTION_INVALID")
text = str(value).strip()
if not text or len(text) > 512:
raise JobError("JOB_BINDING_OPTION_INVALID")
binding[field] = text
return binding
def submit(self, request: Mapping[str, Any], *, payload: bytes | None = None) -> JobReceipt:
binding = self._binding(request)
payload_ref = request.get("payload_ref")
payload_size = request.get("payload_size")
if payload is not None:
payload_ref, payload_size = self.store.payloads.put(payload, binding["payload_digest"])
if payload_ref is not None and payload_ref != f"sha256/{binding['payload_digest']}":
raise JobError("PAYLOAD_REFERENCE_MISMATCH")
if payload_ref is not None and (not isinstance(payload_size, int) or payload_size < 0):
raise JobError("PAYLOAD_SIZE_INVALID")
client_idempotency_key = binding.get("idempotency_key")
if client_idempotency_key is not None:
if len(client_idempotency_key) > 128 or not all(
character.isalnum() or character in "._:-" for character in client_idempotency_key
):
raise JobError("IDEMPOTENCY_KEY_INVALID")
key = sha256_json({"client_idempotency_key": client_idempotency_key})
else:
key = sha256_json(binding)
job_id = "job-" + key[:32]
durable_request = dict(binding)
if payload_ref is not None:
durable_request.update({"payload_ref": str(payload_ref), "payload_size": int(payload_size)})
now = utc_now()
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
existing = connection.execute("SELECT * FROM jobs WHERE idempotency_key=?", (key,)).fetchone()
if existing is not None:
if json.loads(existing["request_json"]).get("binding") != binding:
raise BindingConflict("BINDING_CONFLICT")
row = dict(existing)
connection.commit()
return self.store._receipt(row, duplicate=True)
connection.execute(
"INSERT INTO jobs(job_id,idempotency_key,campaign_id,repository,operation,branch,expected_branch_oid,payload_digest,payload_ref,payload_size,state,side_effect_state,revision,request_json,receipt_json,created_at,updated_at) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
(job_id, key, binding["campaign_id"], binding["repository"], binding["operation"], binding["branch"], binding["expected_branch_oid"], binding["payload_digest"], payload_ref, payload_size, JobState.REQUESTED.value, "NONE", 0, json.dumps({"binding": binding, **durable_request}, sort_keys=True), None, now, now),
)
connection.execute("INSERT INTO idempotency_keys(key,job_id,binding_json,created_at) VALUES(?,?,?,?)", (key, job_id, json.dumps(binding, sort_keys=True), now))
connection.execute("INSERT INTO state_transitions(job_id,from_state,to_state,revision,side_effect_state,created_at) VALUES(?,?,?,?,?,?)", (job_id, None, JobState.REQUESTED.value, 0, "NONE", now))
row = dict(connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone())
connection.commit()
except BindingConflict:
raise
except sqlite3.IntegrityError:
row = self.store.read(job_id)
return self.store._receipt(row, duplicate=True)
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
row["request"] = json.loads(row["request_json"])
self.store._mirror(row)
return self.store._receipt(row, duplicate=False)
def lease_job(
self,
job_id: str,
*,
lease_id: str,
lineage_anchor_oid: str,
current_head_oid: str,
generation: int,
lease_expires_at: str,
expected_revision: int,
) -> JobReceipt:
"""Atomically create a branch lease and move its Job to LEASED."""
if generation < 1 or any(len(oid) != 40 for oid in (lineage_anchor_oid, current_head_oid)):
raise JobError("LEASE_BINDING_INVALID")
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
row = connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone()
if row is None:
raise JobError("JOB_NOT_FOUND")
if int(row["revision"]) != expected_revision:
raise JobError("REVISION_CAS_CONFLICT")
if row["branch"] in {"main", "master"}:
raise JobError("PROTECTED_BRANCH_REJECTED")
now = utc_now()
connection.execute("INSERT INTO branch_leases(lease_id,job_id,branch,lineage_anchor_oid,current_head_oid,generation,lease_expires_at,state,revision) VALUES(?,?,?,?,?,?,?,?,?)", (lease_id, job_id, row["branch"], lineage_anchor_oid, current_head_oid, generation, lease_expires_at, "ACTIVE", 0))
next_revision = expected_revision + 1
updated = connection.execute("UPDATE jobs SET state=?,side_effect_state=?,revision=?,updated_at=? WHERE job_id=? AND revision=?", (JobState.LEASED.value, "LEASED", next_revision, now, job_id, expected_revision))
if updated.rowcount != 1:
raise JobError("REVISION_CAS_CONFLICT")
connection.execute("INSERT INTO state_transitions(job_id,from_state,to_state,revision,side_effect_state,created_at) VALUES(?,?,?,?,?,?)", (job_id, row["state"], JobState.LEASED.value, next_revision, "LEASED", now))
result = dict(connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone())
connection.commit()
except sqlite3.IntegrityError as exc:
raise JobError("LEASE_CONFLICT") from exc
except (JobError, BindingConflict):
raise
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
result["request"] = json.loads(result["request_json"])
self.store._mirror(result)
return self.store._receipt(result, duplicate=True)
def update_lease(self, lease_id: str, *, expected_revision: int, current_head_oid: str, generation: int, state: str = "ACTIVE") -> None:
if len(current_head_oid) != 40 or generation < 1:
raise JobError("LEASE_BINDING_INVALID")
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
updated = connection.execute("UPDATE branch_leases SET current_head_oid=?,generation=?,state=?,revision=revision+1 WHERE lease_id=? AND revision=?", (current_head_oid, generation, state, lease_id, expected_revision))
if updated.rowcount != 1:
raise JobError("LEASE_REVISION_CAS_CONFLICT")
connection.commit()
except JobError:
raise
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
def advance(self, job_id: str, state: JobState, *, side_effect_state: str, receipt: Mapping[str, Any] | None = None, expected_revision: int | None = None) -> JobReceipt:
return self._advance(
job_id,
state,
side_effect_state=side_effect_state,
receipt=receipt,
expected_revision=expected_revision,
)
def _advance(
self,
job_id: str,
state: JobState,
*,
side_effect_state: str,
receipt: Mapping[str, Any] | None = None,
expected_revision: int | None = None,
_allow_verified_external_recovery: bool = False,
) -> JobReceipt:
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
row = connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone()
if row is None:
raise JobError("JOB_NOT_FOUND")
previous = JobState(row["state"])
revision = int(row["revision"])
if expected_revision is not None and expected_revision != revision:
raise JobError("REVISION_CAS_CONFLICT")
recovery_transition = (
_allow_verified_external_recovery
and previous == JobState.MANUAL_OWNER_REVIEW_REQUIRED
and state == JobState.COMPLETED
)
if previous in _TERMINAL and state != previous and not recovery_transition:
raise JobError("JOB_TERMINAL")
if state not in _RECOVERABLE | {JobState.RECONCILING, JobState.MANUAL_OWNER_REVIEW_REQUIRED}:
if previous in _RECOVERABLE:
raise JobError("RECONCILE_REQUIRED")
if previous != JobState.RECONCILING and _ORDER.get(state, -1) < _ORDER.get(previous, -1):
raise JobError("JOB_STATE_REGRESSION")
next_revision = revision + 1
receipt_json = json.dumps(dict(receipt), sort_keys=True) if receipt is not None else row["receipt_json"]
now = utc_now()
updated = connection.execute("UPDATE jobs SET state=?,side_effect_state=?,revision=?,receipt_json=?,updated_at=? WHERE job_id=? AND revision=?", (state.value, side_effect_state, next_revision, receipt_json, now, job_id, revision))
if updated.rowcount != 1:
raise JobError("REVISION_CAS_CONFLICT")
connection.execute("INSERT INTO state_transitions(job_id,from_state,to_state,revision,side_effect_state,created_at) VALUES(?,?,?,?,?,?)", (job_id, previous.value, state.value, next_revision, side_effect_state, now))
if receipt is not None:
digest = hashlib.sha256(canonical_json(receipt)).hexdigest()
connection.execute("INSERT INTO receipts(receipt_id,job_id,kind,digest,payload_json,created_at) VALUES(?,?,?,?,?,?)", (f"receipt-{job_id}-{next_revision}", job_id, state.value, digest, json.dumps(dict(receipt), sort_keys=True), now))
result = dict(connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone())
connection.commit()
except (JobError, BindingConflict):
raise
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
result["request"] = json.loads(result["request_json"])
self.store._mirror(result)
return self.store._receipt(result, duplicate=True)
@staticmethod
def _validate_verified_external_merge_proof(
job_id: str,
binding: Mapping[str, Any],
proof: Mapping[str, Any],
) -> dict[str, Any]:
if not isinstance(proof, Mapping):
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_INVALID")
def require_oid(field: str) -> str:
value = proof.get(field)
if not isinstance(value, str) or len(value) != 40 or any(ch not in "0123456789abcdef" for ch in value):
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_INVALID")
return value
if proof.get("proof_schema") != "aios.job-recovery.v1":
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_INVALID")
if proof.get("job_id") != job_id:
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
if proof.get("operation") != "PROTECTED_MERGE" or binding.get("operation") != "PROTECTED_MERGE":
raise JobError("EXTERNAL_SIDE_EFFECT_OPERATION_REJECTED")
if proof.get("pr_number") != 53 or str(binding.get("pr_number")) != "53":
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
if proof.get("pr_merged") is not True or proof.get("protected_merge") is not True:
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
if proof.get("force_push") is not False:
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
if proof.get("machine_actor") != "aios-autonomy-agent":
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
expected_head = str(binding.get("expected_branch_oid", ""))
expected_base = str(binding.get("expected_main_oid", ""))
head_oid = require_oid("head_oid")
base_oid = require_oid("base_oid")
merged_main = require_oid("main_oid_after_protected_merge")
gitea_merged_main = require_oid("gitea_main_oid_after_protected_merge")
authority_merged_main = require_oid("authority_main_oid_after_protected_merge")
current_gitea = require_oid("current_gitea_main_oid")
current_authority = require_oid("current_authority_main_oid")
if head_oid != expected_head or base_oid != expected_base:
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
if not (merged_main == gitea_merged_main == authority_merged_main == head_oid):
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
if current_gitea != current_authority or proof.get("current_main_descends_from_protected_merge") is not True:
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
return dict(proof)
def complete_verified_external_side_effect(
self,
job_id: str,
proof: Mapping[str, Any],
*,
expected_revision: int | None = None,
) -> JobReceipt:
"""Close one historical manual job from independently verified proof.
This is deliberately narrower than advance: it is only valid for the
known protected PR53 recovery shape and preserves the old receipt.
The actual state change still uses the normal revision-CAS path.
"""
current = self.store.read(job_id)
revision = int(current["revision"])
if expected_revision is not None and expected_revision != revision:
raise JobError("REVISION_CAS_CONFLICT")
request = json.loads(current["request_json"])
binding = request.get("binding", request)
normalized_proof = self._validate_verified_external_merge_proof(job_id, binding, proof)
existing_receipt = json.loads(current["receipt_json"]) if current["receipt_json"] else None
if JobState(current["state"]) == JobState.COMPLETED:
if (
existing_receipt
and existing_receipt.get("recovered_from_historical_manual") is True
and existing_receipt.get("proof") == normalized_proof
):
return self.store._receipt(current, duplicate=True)
raise JobError("JOB_TERMINAL")
if JobState(current["state"]) != JobState.MANUAL_OWNER_REVIEW_REQUIRED:
raise JobError("EXTERNAL_SIDE_EFFECT_RECOVERY_STATE_INVALID")
recovery_receipt = {
"recovered_from_historical_manual": True,
"recovery_kind": "VERIFIED_EXTERNAL_SIDE_EFFECT",
"historical_receipt": existing_receipt,
"proof": normalized_proof,
}
return self._advance(
job_id,
JobState.COMPLETED,
side_effect_state="REMOTE_VERIFIED",
receipt=recovery_receipt,
expected_revision=revision,
_allow_verified_external_recovery=True,
)
def reconcile(self, job_id: str, observer: Callable[[Mapping[str, Any]], Mapping[str, Any]]) -> JobReceipt:
current = self.store.read(job_id)
if JobState(current["state"]) not in _RECOVERABLE:
return self.store._receipt(current, duplicate=True)
self.advance(job_id, JobState.RECONCILING, side_effect_state="IN_PROGRESS")
try:
observed = dict(observer(current))
stable = JobState(observed.get("state", JobState.REQUESTED.value))
if stable in {JobState.RECOVERABLE_FAILURE, JobState.RECONCILING}:
raise JobError("RECONCILIATION_UNSTABLE")
return self.advance(job_id, stable, side_effect_state=str(observed.get("side_effect_state", "NONE")), receipt=observed.get("receipt"))
except Exception as exc:
self.advance(job_id, JobState.RECONCILIATION_REQUIRED, side_effect_state="UNKNOWN", receipt={"error_code": type(exc).__name__})
raise JobError("RECONCILIATION_REQUIRED") from exc
def resume(self, job_id: str) -> JobReceipt:
current = self.store.read(job_id)
if JobState(current["state"]) in _TERMINAL:
raise JobError("JOB_TERMINAL")
return self.reconcile(job_id, lambda _current: {"state": JobState.REQUESTED.value, "side_effect_state": "RESUME_SAME_JOB"})
@@ -0,0 +1,279 @@
"""Durable Job side-effect executor for the AIOS campaign control plane."""
from __future__ import annotations
import argparse
import base64
from dataclasses import dataclass, replace
import json
from pathlib import Path
from typing import Any, Mapping
from aios_authority_git_executor import AuthorityGitExecutor, AuthorityGitError
from aios_campaign_recovery import RecoveryEvidenceError, load_recovered_authorization
from aios_gitea_machine_broker import CampaignCapabilityReceipt, GiteaMachineBroker, MachineBrokerError
from aios_gitops_job_controller import GitOpsJobController, JobError, JobState, JobStore
TASK_ID = "AIOS_AUTONOMOUS_RELEASE_CAMPAIGN_V1"
CAMPAIGN_ID = "AIOS-AUTONOMOUS-RELEASE-CAMPAIGN-20260807"
REPOSITORY = "aios/aios"
BRANCH = "agent/autonomous-release-campaign-v1-20260807"
EXPECTED_BRANCH_OID = "bba675fab3b7e0c5fba7066720252b3c91b9f20d"
AUTHORIZATION_DIGEST = "6b0bf550eb702126e967eeff2d561fa808444b924d8a71f5399accad508ed3b2"
LINEAGE_ANCHOR = "69b45e81dcd2c3707c367003b08c55fd361faae8"
class JobExecutionError(RuntimeError):
pass
@dataclass(slots=True)
class DurableJobExecutor:
controller: GitOpsJobController
authority: AuthorityGitExecutor
broker: GiteaMachineBroker
recovery_path: Path
def _request(self, job_id: str) -> tuple[dict[str, Any], dict[str, Any]]:
row = self.controller.store.read(job_id)
try:
request = json.loads(row["request_json"])
except (KeyError, TypeError, json.JSONDecodeError) as exc:
raise JobExecutionError("JOB_REQUEST_INVALID") from exc
return row, dict(request.get("binding") or {})
def _validate_campaign(self, request: Mapping[str, Any]) -> str:
operation = str(request.get("operation", ""))
expected_branch_oid = request.get("expected_branch_oid")
required = {
"task_id": TASK_ID,
"campaign_id": CAMPAIGN_ID,
"repository": REPOSITORY,
"branch": BRANCH,
"authorization_recovery_digest": AUTHORIZATION_DIGEST,
}
for key, expected in required.items():
if request.get(key) != expected:
raise JobExecutionError(f"JOB_{key.upper()}_MISMATCH")
new_oid = request.get("new_branch_oid")
if not isinstance(new_oid, str) or len(new_oid) != 40 or any(ch not in "0123456789abcdef" for ch in new_oid):
raise JobExecutionError("NEW_BRANCH_OID_INVALID")
if not isinstance(expected_branch_oid, str) or len(expected_branch_oid) != 40 or any(ch not in "0123456789abcdef" for ch in expected_branch_oid):
raise JobExecutionError("EXPECTED_BRANCH_OID_INVALID")
if operation == "ADVANCE_EXISTING_CAMPAIGN_BRANCH" and expected_branch_oid == new_oid:
raise JobExecutionError("EXPECTED_BRANCH_OID_INVALID")
if operation != "ADVANCE_EXISTING_CAMPAIGN_BRANCH" and expected_branch_oid != new_oid:
raise JobExecutionError("POST_ADVANCE_HEAD_BINDING_INVALID")
try:
recovery = load_recovered_authorization(self.recovery_path)
except RecoveryEvidenceError as exc:
raise JobExecutionError(str(exc)) from exc
if recovery["digest"] != AUTHORIZATION_DIGEST:
raise JobExecutionError("RECOVERY_DIGEST_MISMATCH")
for key, expected in required.items():
if key == "campaign_id":
actual = recovery.get(key)
elif key == "authorization_recovery_digest":
continue
else:
actual = recovery.get({"expected_branch_oid": "expected_branch_oid"}.get(key, key))
if actual is not None and actual != expected:
raise JobExecutionError("RECOVERY_BINDING_MISMATCH")
if recovery.get("expected_branch_oid") != EXPECTED_BRANCH_OID:
raise JobExecutionError("RECOVERY_BRANCH_OID_MISMATCH")
authority_head = self.authority.read_ref(f"refs/heads/{BRANCH}")
if authority_head not in {expected_branch_oid, new_oid}:
raise JobExecutionError("AUTHORITY_BRANCH_EXPECTATION_MISMATCH")
if not self.authority.ancestry_check(expected_branch_oid, new_oid):
raise JobExecutionError("NEW_OID_NOT_DESCENDANT")
if not self.authority.ancestry_check(EXPECTED_BRANCH_OID, new_oid):
raise JobExecutionError("NEW_OID_OUTSIDE_AUTHORIZED_LINEAGE")
identity = self.broker.verify_identity()
if identity.get("repository") != REPOSITORY or identity.get("login") == "aiman":
raise JobExecutionError("MACHINE_IDENTITY_SCOPE_INVALID")
return new_oid
def dry_run(self, job_id: str) -> dict[str, Any]:
_row, request = self._request(job_id)
new_oid = self._validate_campaign(request)
remote = self.broker.readback()
if remote is None:
action = "CREATE_REQUIRED"
elif remote == new_oid:
action = "ALREADY_IN_SYNC"
elif remote == request["expected_branch_oid"]:
action = "FAST_FORWARD_REQUIRED"
else:
action = "MANUAL_OWNER_REVIEW_REQUIRED"
if action == "MANUAL_OWNER_REVIEW_REQUIRED":
raise JobExecutionError("GITEA_BRANCH_CONFLICT")
return {
"decision": "SAFE_TO_ADVANCE",
"authority_action": "CAS_REQUIRED",
"gitea_action": action,
"previous_branch_oid": request["expected_branch_oid"],
"final_branch_oid": new_oid,
"force_push": False,
}
def _recover_requested(self, job_id: str) -> dict[str, Any]:
row = self.controller.store.read(job_id)
if JobState(row["state"]) in {JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}:
self.controller.resume(job_id)
row = self.controller.store.read(job_id)
return row
def execute(self, job_id: str, *, dry_run: bool = False) -> dict[str, Any]:
try:
return self._execute_unprotected(job_id, dry_run=dry_run)
except Exception as exc:
try:
row = self.controller.store.read(job_id)
state = JobState(row["state"])
if state not in {JobState.COMPLETED, JobState.MANUAL_OWNER_REVIEW_REQUIRED, JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}:
self.controller.advance(job_id, JobState.RECOVERABLE_FAILURE, side_effect_state="UNKNOWN", receipt={"error_code": type(exc).__name__})
except Exception:
pass
raise
def _execute_unprotected(self, job_id: str, *, dry_run: bool = False) -> dict[str, Any]:
row, request = self._request(job_id)
operation = request.get("operation")
if operation == "ADVANCE_EXISTING_CAMPAIGN_BRANCH":
if dry_run:
return self.dry_run(job_id)
return self._advance_campaign(job_id)
if operation == "COMMIT_FILE":
return self._commit_file(job_id, request)
if operation == "SYNC_TASK_BRANCH_TO_GITEA":
new_oid = self._validate_campaign(request)
final = self.broker.push_branch(new_oid)
return {"operation": operation, "remote_verified": final == new_oid, "final_network_oid": final}
if operation == "CREATE_PR":
self._validate_campaign(request)
return dict(self.broker.create_pr("AIOS autonomous release control plane", "Durable campaign continuation; exact head and recovery evidence are bound in the Job."))
if operation == "PUBLISH_CHECK":
self._validate_campaign(request)
context = request.get("check_context", "aios/durable-release")
state = request.get("check_state", "success")
description = request.get("check_description", "Durable campaign control-plane check passed")
return dict(self.broker.status(str(state), str(description), str(context)))
if operation == "PROTECTED_MERGE":
self._validate_campaign(request)
number = int(request.get("pr_number", "0"))
self.broker.transport.expected_merge_head = request["expected_branch_oid"]
expected_main = request.get("expected_main_oid")
if expected_main is not None:
self.broker.transport.expected_merge_base = expected_main
return dict(self.broker.merge(number))
if operation == "RECONCILE_AUTHORITY_MAIN":
self._validate_campaign(request)
old = request.get("expected_main_oid")
new = request.get("new_branch_oid")
if not isinstance(old, str) or not isinstance(new, str):
raise JobExecutionError("MAIN_RECONCILIATION_BINDING_INVALID")
gitea_main = self.broker.transport.read_branch("main")
if gitea_main != new:
raise JobExecutionError("GITEA_MAIN_EXPECTATION_MISMATCH")
if not self.authority.ancestry_check(old, new):
raise JobExecutionError("MAIN_OID_NOT_DESCENDANT")
return {"authority_main": self.authority.ff_CAS_main_after_verified_protected_merge(old, new)}
raise JobExecutionError("OPERATION_NOT_ALLOWED")
def _advance_campaign(self, job_id: str) -> dict[str, Any]:
self._recover_requested(job_id)
row, request = self._request(job_id)
new_oid = self._validate_campaign(request)
expected_oid = request["expected_branch_oid"]
state = JobState(row["state"])
if state == JobState.REQUESTED:
receipt = self.controller.advance(job_id, JobState.AUTHORIZED, side_effect_state="AUTHORIZED", receipt={"authorization_digest": AUTHORIZATION_DIGEST, "task_id": TASK_ID})
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.AUTHORIZED:
self.controller.lease_job(job_id, lease_id=f"lease-{job_id}", lineage_anchor_oid=LINEAGE_ANCHOR, current_head_oid=expected_oid, generation=1, lease_expires_at="2099-01-01T00:00:00Z", expected_revision=int(row["revision"]))
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.LEASED:
self.controller.advance(job_id, JobState.WORKTREE_READY, side_effect_state="WORKTREE_READY", receipt={"branch": BRANCH, "expected_oid": expected_oid})
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.WORKTREE_READY:
self.controller.advance(job_id, JobState.EXECUTING, side_effect_state="IN_PROGRESS")
row = self.controller.store.read(job_id)
authority_before = self.authority.read_ref(f"refs/heads/{BRANCH}")
if authority_before == expected_oid:
authority_oid = self.authority.advance_task_branch_CAS(f"refs/heads/{BRANCH}", expected_oid, new_oid)
elif authority_before == new_oid:
authority_oid = new_oid
else:
raise JobExecutionError("AUTHORITY_BRANCH_CONFLICT")
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.EXECUTING:
self.controller.advance(job_id, JobState.LOCAL_VERIFIED, side_effect_state="LOCAL_VERIFIED", receipt={"authority": {"branch": BRANCH, "previous_oid": expected_oid, "new_oid": new_oid, "CAS_success": True, "readback_oid": authority_oid}})
row = self.controller.store.read(job_id)
remote_before = self.broker.readback()
if remote_before != new_oid:
if remote_before not in {None, expected_oid}:
raise JobExecutionError("GITEA_BRANCH_CONFLICT")
if JobState(row["state"]) == JobState.LOCAL_VERIFIED:
if hasattr(self.broker, "receipt"):
self.broker.receipt = replace(self.broker.receipt, expected_oid=expected_oid)
self.controller.advance(job_id, JobState.REMOTE_SYNCING, side_effect_state="IN_PROGRESS", receipt={"gitea": {"previous_oid": remote_before, "desired_oid": new_oid, "force_push": False}})
final_network = self.broker.push_branch(new_oid)
else:
final_network = new_oid
final_api = self.broker.readback()
if final_api != new_oid:
raise JobExecutionError("GITEA_BRANCH_READBACK_MISMATCH")
row = self.controller.store.read(job_id)
if JobState(row["state"]) in {JobState.LOCAL_VERIFIED, JobState.REMOTE_SYNCING}:
self.controller.advance(job_id, JobState.REMOTE_VERIFIED, side_effect_state="REMOTE_VERIFIED", receipt={"gitea": {"machine_actor": "aios-autonomy-agent", "previous_oid": remote_before, "desired_oid": new_oid, "final_network_oid": final_network, "final_API_oid": final_api, "force_push": False}})
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.REMOTE_VERIFIED:
self.controller.advance(job_id, JobState.COMPLETED, side_effect_state="REMOTE_VERIFIED", receipt={"authority_oid": authority_oid, "network_oid": final_network, "gitea_api_oid": final_api, "force_push": False})
return self.controller.store._receipt(self.controller.store.read(job_id), duplicate=True).as_dict()
def _commit_file(self, job_id: str, request: Mapping[str, Any]) -> dict[str, Any]:
row = self.controller.store.read(job_id)
payload_ref = request.get("payload_ref")
if not payload_ref:
raise JobExecutionError("PAYLOAD_REQUIRED")
payload = self.controller.store.payloads.read_verified(payload_ref, row["payload_digest"], int(request["payload_size"]))
try:
spec = json.loads(payload.decode())
path = str(spec["path"])
content = base64.b64decode(spec["content_base64"], validate=True)
except Exception as exc:
raise JobExecutionError("COMMIT_FILE_PAYLOAD_INVALID") from exc
new_oid = self.authority.commit_file_in_isolated_worktree(base_oid=row["expected_branch_oid"], files={path: content}, message="chore(gitops): durable commit-file", worktree_name=job_id)
return {"operation": "COMMIT_FILE", "new_oid": new_oid}
def build_from_environment(job_root: Path) -> DurableJobExecutor:
from aios_gitea_machine_transport import GiteaMachineTransport
from aios_gitea_machine_broker import MachineCredentialProjection
account = __import__("os").environ.get("AIOS_GITEA_MACHINE_ACCOUNT", "aios-autonomy-agent")
api = __import__("os").environ.get("AIOS_GITEA_API", "https://g.cnnetmkt.top/git/api/v1")
remote = __import__("os").environ.get("AIOS_GITEA_REMOTE", "https://g.cnnetmkt.top/git/aios/aios.git")
authority_dir = Path(__import__("os").environ.get("AIOS_GITOPS_AUTHORITY_GIT_DIR", "/data/aios-gitops/repo.git"))
worktree_root = Path(__import__("os").environ.get("AIOS_GITOPS_WORKTREE_ROOT", "/var/lib/aios-gitops/worktrees"))
recovery = Path(__import__("os").environ.get("AIOS_RELEASE_RECOVERY_RECEIPT", "/var/lib/aios-gitops/release-campaigns/AIOS-AUTONOMOUS-RELEASE-CAMPAIGN-20260807/authorization-recovery.json"))
transport = GiteaMachineTransport(api, REPOSITORY, remote, authority_dir)
credential = MachineCredentialProjection.from_mapping({"scheme": "systemd-credential", "name": "gitea-machine-token", "account": account})
receipt = CampaignCapabilityReceipt(CAMPAIGN_ID, TASK_ID, REPOSITORY, BRANCH, EXPECTED_BRANCH_OID, frozenset({"exact_task_branch_push", "pr_create_update", "status_publish", "protected_merge", "remote_readback"}), frozenset({"direct_main_push", "force_push", "tag_update", "tag_delete", "baseline_write", "production_deploy"}))
return DurableJobExecutor(GitOpsJobController(JobStore(job_root)), AuthorityGitExecutor(authority_dir, worktree_root), GiteaMachineBroker(transport, credential, receipt), recovery)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--job-root", required=True, type=Path)
parser.add_argument("--job-id", required=True)
parser.add_argument("--dry-run", action="store_true")
args = parser.parse_args(argv)
result = build_from_environment(args.job_root).execute(args.job_id, dry_run=args.dry_run)
print(json.dumps(result, sort_keys=True))
return 0
if __name__ == "__main__":
raise SystemExit(main())
+7 -1
View File
@@ -5,6 +5,8 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from typing import Any
from release_version import validate_release_version
ROOT = Path(os.getenv('AIOS_GITOPS_ROOT', '/data/aios-gitops'))
WORKSPACE = Path(os.getenv('AIOS_GITOPS_WORKSPACE', str(ROOT / 'workspace')))
DB_PATH = Path(os.getenv('AIOS_GITOPS_DB', str(ROOT / 'runtime' / 'gitops_model.sqlite3')))
@@ -218,7 +220,11 @@ class Handler(BaseHTTPRequestHandler):
a = self.require_write(['owner']);
if not a: return
ver=data.get('version') or data.get('tag')
if not re.match(r'^V\d+R\d+C\d+(-[a-zA-Z0-9_.-]+)?$', ver or ''): self.send_json({'error':'version must match VxRxCx'},400); return
try:
validate_release_version(ver)
except ValueError:
self.send_json({'error':'version must match VxRxCx'},400)
return
msg=data.get('message') or f'release {ver}'; out=git_cmd(['tag','-a',ver,'-m',msg]);
if data.get('push',False): git_cmd(['push','origin',ver])
level='V' if re.match(r'^V[1-9]\d*R0C0', ver) else ('R' if re.match(r'^V\d+R[1-9]\d*C0', ver) else 'C'); exec_sql('INSERT OR REPLACE INTO gitops_versions(version,tag,level,summary,status,created_by,created_at) VALUES(?,?,?,?,?,?,?)',(ver,ver,level,msg,'ready',a['username'],now())); exec_sql('UPDATE gitops_baseline SET active_version=?,active_tag=?,updated_at=? WHERE id=?',(ver,ver,now(),'current')); audit(a['username'],'git_tag',ver,{'level':level}); self.send_json({'status':'tagged','version':ver,'git':out}); return
@@ -0,0 +1,80 @@
"""Pure, observation-first decisions for governed GitOps reconciliation."""
from __future__ import annotations
from dataclasses import dataclass
from enum import Enum
import re
from typing import Optional
OID_RE = re.compile(r"^[0-9a-f]{40}$")
class ReconciliationAction(str, Enum):
NOOP = "NOOP"
PUSH_EXACT = "PUSH_EXACT"
VERIFY_REMOTE = "VERIFY_REMOTE"
MARK_REMOTE_VERIFIED = "MARK_REMOTE_VERIFIED"
MARK_RECONCILIATION_REQUIRED = "MARK_RECONCILIATION_REQUIRED"
FAIL_CONFLICT = "FAIL_CONFLICT"
@dataclass(frozen=True, slots=True)
class RemoteObservation:
repository_identity: str
branch: str
desired_oid: str
authority_oid: Optional[str]
remote_git_oid: Optional[str]
remote_api_oid: Optional[str]
commit_visible: bool
observed_at: str
expected_oid: Optional[str] = None
@property
def all_equal(self) -> bool:
values = (self.authority_oid, self.remote_git_oid, self.remote_api_oid, self.desired_oid)
return all(isinstance(value, str) for value in values) and len(set(values)) == 1
@dataclass(frozen=True, slots=True)
class ReconciliationDecision:
action: ReconciliationAction
state: str
reason: str
retry_safe: bool = False
def _valid_oid(value: Optional[str]) -> bool:
return value is None or bool(OID_RE.fullmatch(value))
def decide_reconciliation(observation: RemoteObservation) -> ReconciliationDecision:
"""Decide from a fresh observation; never complete on a missing remote."""
if observation.repository_identity != "aios/aios":
return ReconciliationDecision(ReconciliationAction.FAIL_CONFLICT, "CONFLICT", "REPOSITORY_IDENTITY_MISMATCH")
if not observation.branch.startswith("agent/"):
return ReconciliationDecision(ReconciliationAction.FAIL_CONFLICT, "CONFLICT", "BRANCH_SCOPE_DENIED")
if not _valid_oid(observation.desired_oid) or not _valid_oid(observation.expected_oid):
return ReconciliationDecision(ReconciliationAction.FAIL_CONFLICT, "CONFLICT", "OID_INVALID")
if observation.authority_oid is None:
return ReconciliationDecision(ReconciliationAction.MARK_RECONCILIATION_REQUIRED, "RECONCILIATION_REQUIRED", "AUTHORITY_UNAVAILABLE")
if observation.authority_oid != observation.desired_oid:
return ReconciliationDecision(ReconciliationAction.MARK_RECONCILIATION_REQUIRED, "RECONCILIATION_REQUIRED", "AUTHORITY_CONFLICT")
present = [oid for oid in (observation.remote_git_oid, observation.remote_api_oid) if oid is not None]
if any(oid not in {observation.desired_oid, observation.expected_oid} for oid in present):
return ReconciliationDecision(ReconciliationAction.FAIL_CONFLICT, "CONFLICT", "REMOTE_OID_CONFLICT")
if observation.all_equal and observation.commit_visible:
return ReconciliationDecision(ReconciliationAction.MARK_REMOTE_VERIFIED, "REMOTE_VERIFIED", "ALL_OIDS_EQUAL_AND_VISIBLE")
if observation.remote_git_oid is None and observation.remote_api_oid is None:
return ReconciliationDecision(ReconciliationAction.MARK_RECONCILIATION_REQUIRED, "REMOTE_PUSH_PENDING", "AUTHORITY_DESIRED_REMOTE_MISSING")
if observation.remote_git_oid == observation.desired_oid or observation.remote_api_oid == observation.desired_oid:
return ReconciliationDecision(ReconciliationAction.VERIFY_REMOTE, "REMOTE_VERIFY_PENDING", "REMOTE_PROOF_PARTIAL")
if observation.expected_oid is not None and observation.remote_git_oid == observation.expected_oid and observation.remote_api_oid == observation.expected_oid:
return ReconciliationDecision(ReconciliationAction.PUSH_EXACT, "REMOTE_PUSH_PENDING", "SAFE_EXACT_FAST_FORWARD_RETRY", retry_safe=True)
if not observation.commit_visible:
return ReconciliationDecision(ReconciliationAction.VERIFY_REMOTE, "COMMIT_VISIBILITY_PENDING", "COMMIT_VISIBILITY_LAG")
return ReconciliationDecision(ReconciliationAction.MARK_RECONCILIATION_REQUIRED, "RECONCILIATION_REQUIRED", "PARTIAL_REMOTE_OBSERVATION")
def reconcile_observation(**kwargs: object) -> ReconciliationDecision:
return decide_reconciliation(RemoteObservation(**kwargs))
+100
View File
@@ -0,0 +1,100 @@
"""Single supervised socket-activated worker for durable GitOps Jobs.
The worker consumes durable requests and reconciliation queries. Repository
mutation is a separate injected executor and remains impossible while the
legacy global write gate is false.
"""
from __future__ import annotations
import argparse
import json
import os
from pathlib import Path
import signal
import socket
import threading
from typing import Any, Mapping
from aios_gitops_job_controller import GitOpsJobController, JobError, JobState, JobStore, ServiceUnavailable
def runtime_policy(environment: Mapping[str, str] | None = None) -> dict[str, bool]:
env = environment or os.environ
legacy = env.get("AIOS_GITOPS_WRITES_ENABLED", "false").lower() in {"1", "true", "yes", "on"}
durable = env.get("AIOS_GITOPS_DURABLE_JOB_EXECUTION_ENABLED", "false").lower() in {"1", "true", "yes", "on"}
# Legacy direct Git mutation remains disabled. Authorized durable Jobs use
# their own campaign/task capability and may mutate only through the
# worker's lease/CAS path while the legacy gate stays false.
return {"legacy_global_write": legacy, "durable_job_execution": durable, "repository_mutation_allowed": durable}
def handle_request(controller: GitOpsJobController, request: Mapping[str, Any], executor: Any | None = None) -> dict[str, Any]:
job_id = str(request.get("job_id") or "")
if not job_id:
raise JobError("JOB_ID_REQUIRED")
if request.get("action") == "execute":
if executor is None:
raise JobError("JOB_EXECUTOR_UNAVAILABLE")
result = executor.execute(job_id, dry_run=bool(request.get("dry_run", False)))
receipt = controller.store._receipt(controller.store.read(job_id), duplicate=True)
return {"job": result, **receipt.as_dict()}
if request.get("action") == "resume":
receipt = controller.resume(job_id)
else:
row = controller.store.read(job_id)
receipt = controller.store._receipt(row, duplicate=True)
return receipt.as_dict()
def serve_socket(sock: socket.socket, controller: GitOpsJobController, stopping: threading.Event, executor: Any | None = None) -> None:
sock.listen(32)
sock.settimeout(1.0)
while not stopping.is_set():
try:
connection, _ = sock.accept()
except socket.timeout:
continue
with connection:
try:
raw = connection.makefile("rb").readline(1_048_576)
request = json.loads(raw.decode("utf-8"))
response = {"status": "ok", **handle_request(controller, request, executor)}
except ServiceUnavailable as exc:
response = {"status": "error", "error": str(exc), "retryable": True, "side_effect_state": "UNKNOWN"}
except Exception as exc:
response = {"status": "error", "error": type(exc).__name__, "retryable": False}
connection.sendall((json.dumps(response, sort_keys=True) + "\n").encode("utf-8"))
def socket_from_systemd() -> socket.socket:
if os.environ.get("LISTEN_FDS") != "1":
raise RuntimeError("SOCKET_ACTIVATION_REQUIRED")
return socket.fromfd(3, socket.AF_UNIX, socket.SOCK_STREAM)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--job-root", required=True, type=Path)
args = parser.parse_args(argv)
policy = runtime_policy()
if policy["legacy_global_write"]:
raise RuntimeError("LEGACY_GLOBAL_WRITES_MUST_REMAIN_FALSE")
if not policy["durable_job_execution"]:
raise RuntimeError("DURABLE_JOB_EXECUTION_GATE_CLOSED")
controller = GitOpsJobController(JobStore(args.job_root))
from aios_gitops_job_executor import build_from_environment
executor = build_from_environment(args.job_root)
stopping = threading.Event()
signal.signal(signal.SIGTERM, lambda *_: stopping.set())
signal.signal(signal.SIGINT, lambda *_: stopping.set())
sock = socket_from_systemd()
try:
serve_socket(sock, controller, stopping, executor)
finally:
sock.close()
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,69 @@
"""CLI client for the governed local writer service."""
from __future__ import annotations
import argparse
import json
import os
import socket
import sys
def _parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Send one governed-local-writer request")
parser.add_argument("--socket", required=True, help="AF_UNIX socket path")
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument("--request", help="Request JSON string")
group.add_argument("--request-file", help="Path to JSON request file")
return parser.parse_args()
def _load_request_text(args: argparse.Namespace) -> str:
if args.request is not None:
return args.request
request_path = os.fspath(args.request_file)
with open(request_path, "r", encoding="utf-8") as request_file:
return request_file.read()
def send_request(socket_path: str, request_text: str) -> dict[str, object]:
"""Open one Unix socket, send one NDJSON request, and parse one NDJSON response."""
payload = json.loads(request_text)
if not isinstance(payload, dict):
raise ValueError("request must be a JSON object")
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client:
client.connect(socket_path)
client.sendall((json.dumps(payload) + "\n").encode("utf-8"))
response_chunks: list[str] = []
while True:
chunk = client.recv(1024 * 1024).decode("utf-8")
if not chunk:
break
response_chunks.append(chunk)
if "\n" in chunk:
break
response_text = "".join(response_chunks)
response_line = response_text.split("\n", 1)[0].strip()
if not response_line:
raise RuntimeError("empty response")
response = json.loads(response_line)
if not isinstance(response, dict):
raise RuntimeError("invalid response")
return response
def main() -> None:
args = _parse_args()
request_text = _load_request_text(args)
response = send_request(args.socket, request_text)
print(json.dumps(response, separators=(",", ":"), sort_keys=True))
if __name__ == "__main__":
try:
main()
except Exception as exc: # pragma: no cover - CLI entrypoint behavior
print(json.dumps({"status": "error", "error": str(exc)}), file=sys.stderr)
raise SystemExit(1)
@@ -0,0 +1,931 @@
"""Unix-socket service for governed local writer operations."""
from __future__ import annotations
import json
import hashlib
import argparse
import importlib.util
import os
import re
import socket
import subprocess
import sys
import tempfile
import uuid
from dataclasses import dataclass, field
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Callable, Mapping, Protocol
from governed_local_writer import (
AppendCommitRequest,
AuthorizationError,
BranchPolicy,
CreateBranchRequest,
GovernedAuthorization,
PeerIdentity,
PRINCIPAL,
TASK_ID,
extract_peer_identity,
contains_secret_like_content,
load_authorization_file,
validate_file_changes,
)
class ExistingGitOpsHandler:
"""Delegate repository mutations to the installed GitOps handler seam."""
def __init__(
self,
*,
workspace_by_branch: Mapping[str, str | os.PathLike[str]],
bare_repo: str | os.PathLike[str],
api_source: str | os.PathLike[str],
) -> None:
self.workspace_by_branch = {str(k): Path(v) for k, v in workspace_by_branch.items()}
self.bare_repo = Path(bare_repo)
self.api_source = Path(api_source)
@staticmethod
def _restore_env(previous: Mapping[str, str | None]) -> None:
for key, value in previous.items():
if value is None:
os.environ.pop(key, None)
else:
os.environ[key] = value
def _load_api(self, workspace: Path):
if not self.api_source.is_file() or self.api_source.is_symlink():
raise RuntimeError("installed GitOps handler source is unavailable")
keys = (
"AIOS_GITOPS_WORKSPACE",
"AIOS_GITOPS_BARE_REPO",
"AIOS_GITOPS_WRITES_ENABLED",
"PYTHONDONTWRITEBYTECODE",
"GIT_CONFIG_COUNT",
"GIT_CONFIG_KEY_0",
"GIT_CONFIG_VALUE_0",
"GIT_CONFIG_KEY_1",
"GIT_CONFIG_VALUE_1",
)
previous = {key: os.environ.get(key) for key in keys}
os.environ.update(
{
"AIOS_GITOPS_WORKSPACE": str(workspace),
"AIOS_GITOPS_BARE_REPO": str(self.bare_repo),
"AIOS_GITOPS_WRITES_ENABLED": "true",
"PYTHONDONTWRITEBYTECODE": "1",
"GIT_CONFIG_COUNT": "2",
"GIT_CONFIG_KEY_0": "safe.directory",
"GIT_CONFIG_VALUE_0": str(workspace),
"GIT_CONFIG_KEY_1": "safe.directory",
"GIT_CONFIG_VALUE_1": str(self.bare_repo),
}
)
module_name = f"_governed_gitops_api_{uuid.uuid4().hex}"
try:
spec = importlib.util.spec_from_file_location(module_name, self.api_source)
if spec is None or spec.loader is None:
raise RuntimeError("unable to load installed GitOps handler")
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)
decision = module.evaluate_git_write(module.WS)
if not decision.allowed:
raise RuntimeError("existing GitOps write gate denied bootstrap")
return module, previous, module_name
except Exception:
sys.modules.pop(module_name, None)
self._restore_env(previous)
raise
def _workspace(self, branch: str) -> Path:
workspace = self.workspace_by_branch.get(branch)
if workspace is None or not workspace.is_dir():
raise RuntimeError("branch workspace is not authorized")
return workspace
@staticmethod
def _git(workspace: Path, args: list[str]) -> str:
result = subprocess.run(
["git", *args], cwd=workspace, text=True, capture_output=True, check=False
)
if result.returncode != 0:
raise RuntimeError("governed handler git probe failed")
return result.stdout.strip()
def create_branch(self, request: CreateBranchRequest) -> Mapping[str, object]:
workspace = self._workspace(request.branch)
# A prior handler call may have completed the fast-forward sync before
# the process observed a post-commit error. Reconcile only an exact,
# metadata-only result; never overwrite or move an existing ref here.
try:
existing = self._git(
workspace,
[
"--git-dir",
str(self.bare_repo),
"rev-parse",
f"refs/heads/{request.branch}",
],
)
except RuntimeError:
existing = ""
if existing:
parent = self._git(workspace, ["--git-dir", str(self.bare_repo), "rev-parse", f"{existing}^1"])
changed = self._git(
workspace,
[
"--git-dir",
str(self.bare_repo),
"diff-tree",
"--no-commit-id",
"--name-only",
"-r",
existing,
],
).splitlines()
safe_branch = re.sub(r"[^A-Za-z0-9_.-]+", "-", request.branch).strip("-")
if parent != request.base_oid or changed != [f"patches/{safe_branch}.branch.json"]:
raise RuntimeError("existing branch is not an exact metadata-only bootstrap")
return {"head_oid": existing, "parent_oid": parent, "handler_status": "reconciled"}
module, previous, module_name = self._load_api(workspace)
phase = "loaded"
try:
phase = "verify_base"
parent = self._git(
workspace,
["--git-dir", str(self.bare_repo), "rev-parse", "refs/heads/main"],
)
if parent != request.base_oid:
raise RuntimeError("authorized base OID drift")
phase = "checkout"
module.checkout(request.branch, base=request.base_oid, create=True)
phase = "write_metadata"
safe_branch = re.sub(r"[^A-Za-z0-9_.-]+", "-", request.branch).strip("-")
metadata_path = f"patches/{safe_branch}.branch.json"
module.write_json(
metadata_path,
{
"schema": "aios.governed-local-writer-bootstrap.v1",
"task_id": TASK_ID,
"principal": PRINCIPAL,
"branch": request.branch,
"parent_oid": request.base_oid,
"authorization_sha256": request.authorization_digest,
},
)
phase = "commit_and_sync"
result = module.commit(
[metadata_path], "chore(governance): bootstrap repair lease branch"
)
head_oid = self._git(workspace, ["rev-parse", "HEAD"])
first_parent = self._git(workspace, ["rev-parse", "HEAD^1"])
if first_parent != request.base_oid or (request.head_oid and head_oid != request.head_oid):
raise RuntimeError("bootstrap branch lineage check failed")
return {
"head_oid": head_oid,
"parent_oid": first_parent,
"handler_status": str(result.get("status")),
}
except Exception as exc:
raise RuntimeError(f"handler_phase={phase};error={exc.__class__.__name__}") from exc
finally:
sys.modules.pop(module_name, None)
self._restore_env(previous)
def append_commit(self, request: AppendCommitRequest) -> Mapping[str, object]:
workspace = self._workspace(request.branch)
module, previous, module_name = self._load_api(workspace)
phase = "loaded"
try:
phase = "verify_head"
current = self._git(workspace, ["rev-parse", "HEAD"])
if current != request.expected_head_oid:
raise RuntimeError("branch head advanced or is stale")
phase = "checkout"
module.checkout(request.branch, base=request.expected_head_oid, create=False)
phase = "write_files"
paths: list[str] = []
for change in request.changes:
paths.append(module.write_file(change.path, change.content))
phase = "commit_and_sync"
result = module.commit(paths, request.commit_message)
head_oid = self._git(workspace, ["rev-parse", "HEAD"])
first_parent = self._git(workspace, ["rev-parse", "HEAD^1"])
if first_parent != request.expected_head_oid or head_oid == first_parent:
raise RuntimeError("append commit lineage check failed")
return {
"head_oid": head_oid,
"parent_oid": first_parent,
"handler_status": str(result.get("status")),
}
except Exception as exc:
raise RuntimeError(f"handler_phase={phase};error={exc.__class__.__name__}") from exc
finally:
sys.modules.pop(module_name, None)
self._restore_env(previous)
class GovernedWriterHandler(Protocol):
"""Small adapter interface for real writer backend handlers."""
def create_branch(self, request: CreateBranchRequest) -> Mapping[str, object]:
"""Create a branch using internal repo services."""
def append_commit(self, request: AppendCommitRequest) -> Mapping[str, object]:
"""Append a commit using internal repo services."""
class AuditSink(Protocol):
"""Pluggable sink for sanitized audit records."""
def append(self, record: "AuditRecord") -> None:
"""Persist one audit record."""
@dataclass
class AuditRecord:
"""Sanitized audit log entry."""
authorization_digest: str
peer_pid: int
peer_uid: int
peer_gid: int
peer_executable: str | None
timestamp: datetime
operation: str
branch: str
result: str
class InMemoryAuditSink:
"""Simple in-memory audit log used by tests."""
def __init__(self) -> None:
self.records: list[AuditRecord] = []
def append(self, record: AuditRecord) -> None:
self.records.append(record)
@dataclass
class _AuthorizationState:
"""Tracks one-shot operation consumption."""
create_done: set[str] = field(default_factory=set)
append_done: set[str] = field(default_factory=set)
consumed: bool = False
class GovernedLocalWriterService:
"""Policy-aware AF_UNIX service that calls an injected handler."""
def __init__(
self,
socket_path: str | os.PathLike[str],
authorization: GovernedAuthorization,
handler: GovernedWriterHandler,
*,
audit_sink: AuditSink | None = None,
peer_identity_fn: Callable[[socket.socket], PeerIdentity] | None = None,
state_path: str | os.PathLike[str] | None = None,
) -> None:
self.socket_path = Path(socket_path)
self.authorization = authorization
self.handler = handler
self.peer_identity_fn = peer_identity_fn or extract_peer_identity
self.state = _AuthorizationState()
self.audit_sink = audit_sink or InMemoryAuditSink()
self.state_path = Path(state_path) if state_path is not None else None
self._load_state()
def _load_state(self) -> None:
"""Restore one-shot state without restoring request or secret material."""
if self.state_path is None or not self.state_path.exists():
return
try:
raw = json.loads(self.state_path.read_text(encoding="utf-8"))
if not isinstance(raw, dict):
raise ValueError("state must be an object")
if raw.get("authorization_sha256") != self.authorization.authorization_sha256:
raise ValueError("state authorization digest mismatch")
self.state.create_done = {str(v) for v in raw.get("create_done", [])}
self.state.append_done = {str(v) for v in raw.get("append_done", [])}
self.state.consumed = bool(raw.get("consumed", False))
except (OSError, ValueError, TypeError, json.JSONDecodeError) as exc:
raise AuthorizationError("authorization state is invalid") from exc
def _persist_state(self) -> None:
if self.state_path is None:
return
self.state_path.parent.mkdir(parents=True, exist_ok=True)
payload = {
"authorization_sha256": self.authorization.authorization_sha256,
"create_done": sorted(self.state.create_done),
"append_done": sorted(self.state.append_done),
"consumed": self.state.consumed,
}
fd, temp_name = tempfile.mkstemp(prefix=".writer-state-", dir=self.state_path.parent)
try:
os.fchmod(fd, 0o600)
with os.fdopen(fd, "w", encoding="utf-8") as stream:
json.dump(payload, stream, sort_keys=True, separators=(",", ":"))
stream.write("\n")
os.replace(temp_name, self.state_path)
os.chmod(self.state_path, 0o600)
finally:
if os.path.exists(temp_name):
os.unlink(temp_name)
def _new_audit(self, peer: PeerIdentity, operation: str, branch: str, result: str) -> None:
self.audit_sink.append(
AuditRecord(
authorization_digest=self.authorization.authorization_sha256,
peer_pid=peer.pid,
peer_uid=peer.uid,
peer_gid=peer.gid,
peer_executable=peer.executable,
timestamp=datetime.now(tz=timezone.utc),
operation=operation,
branch=branch,
result=result,
)
)
def _require_request_field(self, request: Mapping[str, Any], field_name: str) -> str:
value = request.get(field_name)
if not isinstance(value, str) or not value:
raise AuthorizationError(f"{field_name} must be a non-empty string")
return value
def _branch_policy(self, branch: str) -> BranchPolicy:
policy = self.authorization.branch_operations.get(branch)
if policy is None:
raise AuthorizationError("branch is not authorized")
return policy
def _ensure_not_consumed(self) -> None:
if self.state.consumed:
raise AuthorizationError("authorization consumed")
if datetime.now(tz=timezone.utc) >= self.authorization.expires_at:
raise AuthorizationError("authorization expired")
def _mark_success(self, op: str, branch: str) -> None:
if op == "create_branch":
self.state.create_done.add(branch)
elif op == "append_commit":
self.state.append_done.add(branch)
if self.state.create_done.issuperset(self.authorization.create_branches) and self.state.append_done.issuperset(
self.authorization.append_branches
):
self.state.consumed = True
self._persist_state()
def _validate_request(self, request: Mapping[str, Any]) -> dict[str, Any]:
task_id = self._require_request_field(request, "task_id")
if task_id != TASK_ID:
raise AuthorizationError("invalid task_id")
principal = self._require_request_field(request, "principal")
if principal != PRINCIPAL:
raise AuthorizationError("invalid principal")
operation = self._require_request_field(request, "op")
if operation not in {"create_branch", "append_commit"}:
raise AuthorizationError("unsupported operation")
if operation in self.authorization.denied_operations:
raise AuthorizationError("operation denied by policy")
branch = self._require_request_field(request, "branch")
self._ensure_not_consumed()
policy = self._branch_policy(branch)
if operation == "create_branch":
if policy.create_branch is None:
raise AuthorizationError("branch does not allow create_branch")
if branch in self.state.create_done:
raise AuthorizationError("create_branch already used for this branch")
base_oid = self._require_request_field(request, "base_oid")
raw_head_oid = request.get("head_oid")
if raw_head_oid is not None and (not isinstance(raw_head_oid, str) or not raw_head_oid):
raise AuthorizationError("head_oid must be a non-empty string when supplied")
head_oid = str(raw_head_oid) if raw_head_oid is not None else None
if base_oid != policy.create_branch.base_oid:
raise AuthorizationError("base_oid mismatch")
if policy.create_branch.head_oid is not None and head_oid != policy.create_branch.head_oid:
raise AuthorizationError("head_oid mismatch")
return {
"operation": operation,
"branch": branch,
"base_oid": base_oid,
"head_oid": head_oid,
}
if policy.append_commit is None:
raise AuthorizationError("branch does not allow append_commit")
if policy.create_branch is not None and branch not in self.state.create_done:
raise AuthorizationError("append_commit requires prior successful create_branch")
if branch in self.state.append_done:
raise AuthorizationError("append_commit already used for this branch")
expected_head = policy.append_commit.expected_head_oid
if "expected_head_oid" in request:
supplied_head = self._require_request_field(request, "expected_head_oid")
elif "current_head_oid" in request:
supplied_head = self._require_request_field(request, "current_head_oid")
else:
raise AuthorizationError("expected_head_oid or current_head_oid required")
if not re.fullmatch(r"[0-9a-fA-F]{40}", supplied_head):
raise AuthorizationError("head check must be a full commit OID")
if expected_head is not None and supplied_head != expected_head:
raise AuthorizationError("head check failed")
if request.get("force") is True or request.get("non_fast_forward") is True:
raise AuthorizationError("non-fast-forward operations are not allowed")
if request.get("history_rewrite") is True:
raise AuthorizationError("history rewrite is not allowed")
parent_oids = request.get("parent_oids")
if parent_oids is not None:
if not isinstance(parent_oids, list) or parent_oids != [supplied_head]:
raise AuthorizationError("parent_oids must contain exactly expected_head_oid")
changes = request.get("changes")
if not isinstance(changes, list):
raise AuthorizationError("changes must be a list")
validated_changes = validate_file_changes(changes, policy.append_commit.allowed_paths)
commit_message = request.get("commit_message", "chore(governance): append governed commit")
if not isinstance(commit_message, str) or not commit_message.strip() or len(commit_message) > 200:
raise AuthorizationError("commit_message must be a short non-empty string")
if "\n" in commit_message or "\r" in commit_message or contains_secret_like_content(commit_message):
raise AuthorizationError("commit_message failed policy")
return {
"operation": operation,
"branch": branch,
"expected_head_oid": supplied_head,
"changes": validated_changes,
"commit_message": commit_message.strip(),
}
def process_request(self, raw_request: Mapping[str, Any], peer: PeerIdentity) -> Mapping[str, Any]:
"""Run one request through policy checks and handler adapter."""
operation = raw_request.get("op") if isinstance(raw_request.get("op"), str) else "unknown"
branch = raw_request.get("branch") if isinstance(raw_request.get("branch"), str) else ""
try:
details = self._validate_request(raw_request)
operation = str(details["operation"])
branch = str(details["branch"])
if operation == "create_branch":
self.handler.create_branch(
CreateBranchRequest(
branch=branch,
base_oid=str(details["base_oid"]),
head_oid=(
str(details["head_oid"])
if details.get("head_oid") is not None
else None
),
authorization_digest=self.authorization.authorization_sha256,
)
)
self._mark_success("create_branch", branch)
self._new_audit(peer, operation, branch, "ok")
return {
"status": "ok",
"operation": operation,
"branch": branch,
"result": "applied",
}
self.handler.append_commit(
AppendCommitRequest(
branch=branch,
expected_head_oid=str(details["expected_head_oid"]),
changes=tuple(details["changes"]),
authorization_digest=self.authorization.authorization_sha256,
commit_message=str(details["commit_message"]),
)
)
self._mark_success("append_commit", branch)
self._new_audit(peer, operation, branch, "ok")
return {
"status": "ok",
"operation": operation,
"branch": branch,
"result": "applied",
}
except Exception as error:
self._new_audit(peer, operation or "unknown", branch, f"error:{error.__class__.__name__}")
if isinstance(error, PermissionError):
code = "permission_denied"
elif isinstance(error, AuthorizationError):
code = "request_denied"
else:
code = "handler_failed"
if code == "handler_failed" and str(error).startswith("handler_phase="):
code = str(error)
return {
"status": "error",
"operation": operation or "unknown",
"branch": branch,
"error": code,
}
def _write_json(self, connection: socket.socket, payload: Mapping[str, Any]) -> None:
connection.sendall((json.dumps(payload) + "\n").encode("utf-8"))
def _handle_client(self, connection: socket.socket) -> None:
with connection:
try:
peer = self.peer_identity_fn(connection)
except Exception:
self._write_json(
connection,
{
"status": "error",
"operation": "connection",
"branch": "",
"error": "peer identity validation failed",
},
)
return
with connection.makefile("r", encoding="utf-8") as stream:
for raw_line in stream:
line = raw_line.strip()
if not line:
continue
try:
request = json.loads(line)
except json.JSONDecodeError:
self._write_json(
connection,
{
"status": "error",
"operation": "request",
"branch": "",
"error": "invalid json",
},
)
continue
if not isinstance(request, Mapping):
self._write_json(
connection,
{
"status": "error",
"operation": "request",
"branch": "",
"error": "request must be a JSON object",
},
)
continue
self._write_json(connection, self.process_request(request, peer))
def _bind_socket(self) -> socket.socket:
if self.socket_path.exists():
raise RuntimeError("governed writer socket already exists")
parent = self.socket_path.parent
if not parent.is_dir():
raise RuntimeError("governed writer socket parent is missing")
if parent.is_symlink():
raise RuntimeError("governed writer socket parent must not be a symlink")
server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
server.bind(str(self.socket_path))
os.chmod(self.socket_path, 0o600)
server.listen(16)
return server
def serve(self) -> None:
"""Serve newline-delimited JSON over a filesystem AF_UNIX socket."""
server = self._bind_socket()
try:
while True:
connection, _ = server.accept()
self._handle_client(connection)
finally:
server.close()
if self.socket_path.exists():
self.socket_path.unlink()
class JsonAuditSink:
"""Append-only root-owned audit sink containing no request payloads."""
def __init__(self, path: str | os.PathLike[str]) -> None:
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
def append(self, record: AuditRecord) -> None:
payload = {
"authorization_sha256": record.authorization_digest,
"peer_pid": record.peer_pid,
"peer_uid": record.peer_uid,
"peer_gid": record.peer_gid,
"peer_executable": record.peer_executable,
"timestamp": record.timestamp.isoformat(),
"operation": record.operation,
"branch": record.branch,
"result": record.result,
}
line = (json.dumps(payload, sort_keys=True, separators=(",", ":")) + "\n").encode("utf-8")
fd = os.open(self.path, os.O_WRONLY | os.O_CREAT | os.O_APPEND, 0o600)
try:
os.fchmod(fd, 0o600)
os.write(fd, line)
finally:
os.close(fd)
# V2R1C6 durable receipt overlay. This subclasses the original root-only
# writer in-place so the existing AF_UNIX/SO_PEERCRED and handler policy remain
# the only mutation path.
_LegacyGovernedLocalWriterService = GovernedLocalWriterService
class GovernedLocalWriterService(_LegacyGovernedLocalWriterService):
"""Existing writer with durable request receipts and safe recovery."""
def _load_state(self) -> None:
super()._load_state()
self.state.receipts = {}
if self.state_path is None or not self.state_path.exists():
return
raw = json.loads(self.state_path.read_text(encoding="utf-8"))
if not isinstance(raw, dict):
raise AuthorizationError("authorization state is invalid")
receipts = raw.get("receipts", {})
if not isinstance(receipts, dict):
raise AuthorizationError("authorization receipts are invalid")
self.state.receipts = {
str(key): dict(value)
for key, value in receipts.items()
if isinstance(value, dict)
}
def _persist_state(self) -> None:
if self.state_path is None:
return
self.state_path.parent.mkdir(parents=True, exist_ok=True)
payload = {
"authorization_sha256": self.authorization.authorization_sha256,
"create_done": sorted(self.state.create_done),
"append_done": sorted(self.state.append_done),
"consumed": bool(self.state.consumed),
"receipts": self.state.receipts,
}
fd, temp_name = tempfile.mkstemp(prefix=".writer-state-", dir=self.state_path.parent)
try:
os.fchmod(fd, 0o600)
with os.fdopen(fd, "w", encoding="utf-8") as stream:
json.dump(payload, stream, sort_keys=True, separators=(",", ":"))
stream.write("\n")
os.replace(temp_name, self.state_path)
os.chmod(self.state_path, 0o600)
finally:
if os.path.exists(temp_name):
os.unlink(temp_name)
def _request_identity(self, request: Mapping[str, Any]) -> dict[str, str]:
canonical = json.dumps(dict(request), sort_keys=True, separators=(",", ":"), ensure_ascii=True)
fallback = hashlib.sha256(canonical.encode("utf-8")).hexdigest()
request_id = request.get("request_id")
idempotency_key = request.get("idempotency_key")
request_id = request_id.strip() if isinstance(request_id, str) and request_id.strip() else "legacy-" + fallback
idempotency_key = idempotency_key.strip() if isinstance(idempotency_key, str) and idempotency_key.strip() else "legacy-" + fallback
for field_name, value in (("request_id", request_id), ("idempotency_key", idempotency_key)):
if len(value) > 200 or "\n" in value or "\r" in value:
raise AuthorizationError(field_name + " is invalid")
return {
"request_id": request_id,
"idempotency_key": idempotency_key,
"request_fingerprint": fallback,
}
def _find_receipt(self, details: Mapping[str, Any], compare_fingerprint: bool = True):
request_id = str(details["request_id"])
idempotency_key = str(details["idempotency_key"])
fingerprint = str(details.get("request_fingerprint", ""))
for receipt in self.state.receipts.values():
if receipt.get("request_id") == request_id or receipt.get("idempotency_key") == idempotency_key:
if compare_fingerprint and receipt.get("request_fingerprint") != fingerprint:
raise AuthorizationError("idempotency key reused with different request")
return receipt
return None
@staticmethod
def _receipt_response(receipt: Mapping[str, object], status: str = "receipt") -> dict[str, object]:
response = {
"status": status,
"request_id": receipt.get("request_id"),
"idempotency_key": receipt.get("idempotency_key"),
"operation": receipt.get("operation"),
"branch": receipt.get("branch", ""),
"receipt_status": receipt.get("status"),
}
for field_name in ("commit_oid", "final_head", "parent_oid", "audit_sha256", "error_code", "safe_to_retry"):
if field_name in receipt:
response[field_name] = receipt[field_name]
return response
def _new_receipt(self, details: Mapping[str, Any]) -> dict[str, object]:
return {
"request_id": str(details["request_id"]),
"idempotency_key": str(details["idempotency_key"]),
"request_fingerprint": str(details["request_fingerprint"]),
"operation": str(details["operation"]),
"branch": str(details.get("branch", "")),
"expected_head_oid": str(details.get("expected_head_oid", "")),
"status": "accepted",
"created_at": datetime.now(tz=timezone.utc).isoformat(),
"safe_to_retry": False,
}
def _new_audit(self, peer: PeerIdentity, operation: str, branch: str, result: str) -> str:
super()._new_audit(peer, operation, branch, result)
payload = {
"authorization_sha256": self.authorization.authorization_sha256,
"peer_pid": peer.pid,
"peer_uid": peer.uid,
"peer_gid": peer.gid,
"peer_executable": peer.executable,
"operation": operation,
"branch": branch,
"result": result,
}
return hashlib.sha256(json.dumps(payload, sort_keys=True, separators=(",", ":")).encode("utf-8")).hexdigest()
def _mark_success(self, op: str, branch: str) -> None:
if op == "create_branch":
self.state.create_done.add(branch)
elif op in {"append_commit", "append_files", "append_batch"}:
self.state.append_done.add(branch)
# The provisioning grant is one-shot; the task writer lease itself must
# support multiple append receipts for the exact authorized branches.
self.state.consumed = False
self._persist_state()
def _validate_durable(self, raw_request: Mapping[str, Any]) -> dict[str, Any]:
operation = self._require_request_field(raw_request, "op")
identity = self._request_identity(raw_request)
if operation == "query_receipt":
return {"operation": operation, **identity}
if operation == "recover_request":
branch = self._require_request_field(raw_request, "branch")
policy = self._branch_policy(branch)
if policy.append_commit is None:
raise AuthorizationError("branch does not allow recovery")
expected = raw_request.get("expected_head_oid") or policy.append_commit.expected_head_oid
if not isinstance(expected, str) or not re.fullmatch(r"[0-9a-fA-F]{40}", expected):
raise AuthorizationError("recovery requires expected_head_oid")
return {"operation": operation, "branch": branch, "expected_head_oid": expected.lower(), **identity}
if operation not in {"create_branch", "append_commit", "append_files", "append_batch"}:
raise AuthorizationError("unsupported operation")
request = dict(raw_request)
if operation in {"append_files", "append_batch"}:
request["op"] = "append_commit"
if operation in {"append_commit", "append_files", "append_batch"}:
self.state.consumed = False
branch = request.get("branch")
removed = False
dynamic_expected = False
if isinstance(branch, str) and branch in self.state.append_done:
self.state.append_done.remove(branch)
removed = True
policy = self.authorization.branch_operations.get(branch)
if policy is not None and policy.append_commit is not None and policy.append_commit.expected_head_oid is not None:
from governed_local_writer import BranchAppendPolicy, BranchPolicy
self.authorization.branch_operations[branch] = BranchPolicy(create_branch=policy.create_branch, append_commit=BranchAppendPolicy(branch=branch, expected_head_oid=None, allowed_paths=policy.append_commit.allowed_paths))
dynamic_expected = True
try:
details = super()._validate_request(request)
finally:
if dynamic_expected:
policy = self.authorization.branch_operations.get(str(branch))
if policy is not None and policy.append_commit is not None:
from governed_local_writer import BranchAppendPolicy, BranchPolicy
self.authorization.branch_operations[str(branch)] = BranchPolicy(create_branch=policy.create_branch, append_commit=BranchAppendPolicy(branch=str(branch), expected_head_oid=str(raw_request.get("expected_head_oid") or ""), allowed_paths=policy.append_commit.allowed_paths))
if removed:
self.state.append_done.add(str(branch))
details.update(identity)
details["operation"] = operation
return details
def _recover_backend(self, details: Mapping[str, Any]) -> dict[str, object]:
if not hasattr(self.handler, "_workspace") or not hasattr(self.handler, "_git"):
raise RuntimeError("recovery backend unavailable")
workspace = self.handler._workspace(str(details["branch"]))
expected = str(details["expected_head_oid"])
current = self.handler._git(workspace, ["rev-parse", "HEAD"])
if current == expected:
return {"status": "not_started", "final_head": current}
parent = self.handler._git(workspace, ["rev-parse", "HEAD^1"])
if parent != expected:
raise RuntimeError("external branch advance during recovery")
return {"status": "applied", "commit_oid": current, "final_head": current, "parent_oid": parent}
def process_request(self, raw_request: Mapping[str, Any], peer: PeerIdentity) -> Mapping[str, Any]:
operation = raw_request.get("op") if isinstance(raw_request.get("op"), str) else "unknown"
branch = raw_request.get("branch") if isinstance(raw_request.get("branch"), str) else ""
try:
details = self._validate_durable(raw_request)
operation = str(details["operation"])
branch = str(details.get("branch", ""))
if operation == "query_receipt":
receipt = self._find_receipt(details, compare_fingerprint=False)
return self._receipt_response(receipt) if receipt is not None else {"status": "not_found", "request_id": details["request_id"], "idempotency_key": details["idempotency_key"]}
if operation == "recover_request":
receipt = self._find_receipt(details, compare_fingerprint=False)
if receipt is None:
return {"status": "not_found", "request_id": details["request_id"], "idempotency_key": details["idempotency_key"], "safe_to_retry": True}
if receipt.get("status") == "applied":
return self._receipt_response(receipt)
recovered = self._recover_backend(details)
if recovered.get("status") == "applied":
receipt.update({"status": "applied", "commit_oid": recovered.get("commit_oid"), "final_head": recovered.get("final_head"), "parent_oid": recovered.get("parent_oid"), "safe_to_retry": False})
self._mark_success("append_commit", branch)
return self._receipt_response(receipt)
receipt["safe_to_retry"] = True
self._persist_state()
return self._receipt_response(receipt)
receipt = self._find_receipt(details)
if receipt is not None:
return self._receipt_response(receipt)
receipt = self._new_receipt(details)
self.state.receipts[str(details["request_id"])] = receipt
self._persist_state()
if operation == "create_branch":
result = self.handler.create_branch(CreateBranchRequest(branch=branch, base_oid=str(details["base_oid"]), head_oid=str(details["head_oid"]) if details.get("head_oid") is not None else None, authorization_digest=self.authorization.authorization_sha256))
commit_oid = str(result.get("head_oid", ""))
parent_oid = str(result.get("parent_oid", ""))
audit_sha = self._new_audit(peer, operation, branch, "ok")
receipt.update({"status": "applied", "commit_oid": commit_oid, "final_head": commit_oid, "parent_oid": parent_oid, "audit_sha256": audit_sha, "safe_to_retry": False})
self._mark_success("create_branch", branch)
return self._receipt_response(receipt, status="ok") | {"result": "applied"}
result = self.handler.append_commit(AppendCommitRequest(branch=branch, expected_head_oid=str(details["expected_head_oid"]), changes=tuple(details["changes"]), authorization_digest=self.authorization.authorization_sha256, commit_message=str(details["commit_message"])))
commit_oid = str(result.get("head_oid", ""))
parent_oid = str(result.get("parent_oid", ""))
audit_sha = self._new_audit(peer, operation, branch, "ok")
receipt.update({"status": "applied", "commit_oid": commit_oid, "final_head": commit_oid, "parent_oid": parent_oid, "audit_sha256": audit_sha, "safe_to_retry": False})
self._mark_success("append_commit", branch)
return self._receipt_response(receipt, status="ok") | {"result": "applied"}
except Exception as error:
try:
identity = self._request_identity(raw_request)
receipt = self.state.receipts.get(identity["request_id"])
if receipt is not None and receipt.get("status") == "accepted":
receipt.update({"status": "failed", "error_code": error.__class__.__name__, "safe_to_retry": False})
self._persist_state()
except Exception:
pass
self._new_audit(peer, operation, branch, "error:" + error.__class__.__name__)
return {"status": "error", "operation": operation, "branch": branch, "error": "request_denied" if isinstance(error, AuthorizationError) else "handler_failed"}
def _parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description="AIOS governed local writer")
parser.add_argument("--socket", required=True)
parser.add_argument("--authorization", required=True)
parser.add_argument("--state", required=True)
parser.add_argument("--audit", required=True)
parser.add_argument("--repair-branch", required=True)
parser.add_argument("--repair-workspace", required=True)
parser.add_argument("--candidate-branch", required=True)
parser.add_argument("--candidate-workspace", required=True)
parser.add_argument("--bare-repo", required=True)
parser.add_argument("--api-source", required=True)
return parser.parse_args()
def main() -> int:
if hasattr(os, "geteuid") and os.geteuid() != 0:
print("governed local writer requires uid 0", file=sys.stderr)
return 2
args = _parse_args()
authorization = load_authorization_file(args.authorization)
handler = ExistingGitOpsHandler(
workspace_by_branch={
args.repair_branch: args.repair_workspace,
args.candidate_branch: args.candidate_workspace,
},
bare_repo=args.bare_repo,
api_source=args.api_source,
)
service = GovernedLocalWriterService(
args.socket,
authorization,
handler,
audit_sink=JsonAuditSink(args.audit),
state_path=args.state,
)
service.serve()
return 0
if __name__ == "__main__": # pragma: no cover - systemd entrypoint
raise SystemExit(main())
@@ -0,0 +1,59 @@
"""Central fail-closed policy seam for governed task and continuation requests."""
from __future__ import annotations
from dataclasses import dataclass
import re
from typing import Any, Mapping
OID_RE = re.compile(r"^[0-9a-f]{40}$")
@dataclass(frozen=True, slots=True)
class GovernedPolicyInput:
identity: str
task_id: str
repository: str
operation: str
base_oid: str
expected_branch_oid: str
target_branch: str
generation: int
risk: str = "engineering"
requested_capabilities: tuple[str, ...] = ()
previous_generation: int | None = None
@dataclass(frozen=True, slots=True)
class PolicyDecision:
decision: str
reason: str
def as_dict(self) -> dict[str, str]:
return {"decision": self.decision, "reason": self.reason}
def evaluate_policy(value: GovernedPolicyInput | Mapping[str, Any]) -> PolicyDecision:
if isinstance(value, Mapping):
value = GovernedPolicyInput(**value)
if value.repository != "aios/aios":
return PolicyDecision("DENY", "REPOSITORY_IDENTITY_MISMATCH")
if not OID_RE.fullmatch(value.base_oid) or not OID_RE.fullmatch(value.expected_branch_oid):
return PolicyDecision("DENY", "OID_INVALID")
if not value.target_branch.startswith("agent/") or value.target_branch in {"main", "master"}:
return PolicyDecision("DENY", "BRANCH_SCOPE_DENIED")
if "force_push" in value.requested_capabilities or "force" in value.operation.lower():
return PolicyDecision("DENY", "FORCE_PUSH_DENIED")
if {"main_write", "tag_write", "baseline_write", "production_write", "production_deploy", "tag_mutation", "baseline_mutation"}.intersection(value.requested_capabilities):
return PolicyDecision("DENY", "MUTATION_SCOPE_DENIED")
if not isinstance(value.generation, int) or isinstance(value.generation, bool) or value.generation < 1:
return PolicyDecision("DENY", "GENERATION_INVALID")
if value.previous_generation is not None and value.generation <= value.previous_generation:
return PolicyDecision("MANUAL_OWNER_REVIEW_REQUIRED", "GENERATION_NOT_MONOTONIC")
if not value.task_id:
return PolicyDecision("DENY", "TASK_ID_INVALID")
return PolicyDecision("ALLOW", "GOVERNED_ENGINEERING_SCOPE")
class GovernedPolicy:
def decide(self, request: GovernedPolicyInput | Mapping[str, Any]) -> PolicyDecision:
return evaluate_policy(request)
@@ -0,0 +1,124 @@
"""Durable governed task binding for CREATE_NEW and CONTINUE_EXISTING."""
from __future__ import annotations
from dataclasses import dataclass, asdict
import hashlib
import json
from pathlib import Path
import re
from typing import Any, Mapping
from aios_gitops_job_controller import GitOpsJobController, JobStore
from aios_online_agent_execution import GovernedExecutionBinding, project_execution_envelope
OID_RE = re.compile(r"^[0-9a-f]{40}$")
BRANCH_RE = re.compile(r"^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$")
class BootstrapError(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class GovernedTaskBinding:
task_id: str
repository: str
base_oid: str
target_branch: str
risk: str
description: str
idempotency_key: str
expected_branch_oid: str
branch_mode: str
generation: int
job_id: str | None = None
def response(self, *, idempotent: bool = False) -> dict[str, Any]:
return {"task_id": self.task_id, "repository": self.repository, "base_oid": self.base_oid, "target_branch": self.target_branch, "expected_branch_oid": self.expected_branch_oid, "generation": self.generation, "branch_mode": self.branch_mode, "create_target_branch": self.branch_mode == "CREATE_NEW", "worktree_mode": "detached_at_base_oid" if self.branch_mode == "CREATE_NEW" else "detached_at_expected_head", "cas_ref_update_required": True, "idempotent": idempotent, "state": "BOUND", "job_id": self.job_id}
def _safe(value: str) -> str:
return re.sub(r"[^A-Za-z0-9_.-]+", "-", value).strip("-")[:120] or "task"
class GovernedTaskBootstrap:
def __init__(self, authority: Any, root: Path, *, remote_name: str, token_path: Path, ssh_key_path: Path, api_base: str = "https://g.cnnetmkt.top/git/api/v1") -> None:
self.authority = authority; self.root = Path(root); self.remote_name = remote_name; self.token_path = Path(token_path); self.ssh_key_path = Path(ssh_key_path); self.api_base = api_base; self.task_meta_root = self.root / "task-bindings"; self.job_root = self.root / "jobs"; self.task_meta_root.mkdir(parents=True, exist_ok=True); self.job_root.mkdir(parents=True, exist_ok=True)
def _store(self, task_id: str) -> tuple[JobStore, GitOpsJobController]:
root = self.job_root / _safe(task_id); store = JobStore(root, payload_root=root / "payloads"); return store, GitOpsJobController(store)
def _metadata(self, task_id: str) -> Path:
return self.task_meta_root / (_safe(task_id) + ".json")
def _read_existing(self, task_id: str) -> GovernedTaskBinding | None:
path = self._metadata(task_id)
if not path.is_file() or path.is_symlink(): return None
return GovernedTaskBinding(**json.loads(path.read_text(encoding="utf-8")))
def get_binding(self, task_id: str) -> GovernedTaskBinding:
"""Read the exact durable binding; never synthesize missing lineage."""
binding = self._read_existing(str(task_id))
if binding is None or not binding.job_id:
raise BootstrapError("TASK_EXECUTION_PROJECTION_MISSING")
return binding
def execution_binding(self, binding: GovernedTaskBinding) -> GovernedExecutionBinding:
return GovernedExecutionBinding.from_mapping(
{
"task_id": binding.task_id,
"job_id": binding.job_id,
"repository": binding.repository,
"base_oid": binding.base_oid,
"expected_branch_oid": binding.expected_branch_oid,
"target_branch": binding.target_branch,
"branch_mode": binding.branch_mode,
"generation": binding.generation,
"idempotency_key": binding.idempotency_key,
}
)
def execution_envelope(
self,
task_id: str,
execution_request: Mapping[str, Any] | None = None,
*,
required_capabilities: tuple[str, ...] = (),
expected_job_id: str | None = None,
) -> dict[str, Any]:
binding = self.get_binding(task_id)
if expected_job_id and binding.job_id != expected_job_id:
raise BootstrapError("JOB_BINDING_CONFLICT")
return project_execution_envelope(
self.execution_binding(binding),
execution_request,
required_capabilities=required_capabilities,
)
def _write(self, binding: GovernedTaskBinding) -> None:
path = self._metadata(binding.task_id); temporary = path.with_suffix(".tmp"); temporary.write_text(json.dumps(asdict(binding), sort_keys=True), encoding="utf-8"); temporary.replace(path)
def create(self, request: Mapping[str, Any]) -> dict[str, Any]:
values = {key: str(request[key]) for key in ("task_id", "repository", "base_oid", "target_branch", "risk", "description", "idempotency_key")}
if values["repository"] != "aios/aios" or not OID_RE.fullmatch(values["base_oid"]): raise BootstrapError("TASK_BINDING_INVALID")
if not BRANCH_RE.fullmatch(values["target_branch"]): raise BootstrapError("TARGET_BRANCH_INVALID")
existing = self._read_existing(values["task_id"])
if existing is not None:
if any(getattr(existing, key) != values[key] for key in ("repository", "base_oid", "target_branch", "risk", "description")): raise BootstrapError("TASK_BINDING_CONFLICT")
return existing.response(idempotent=True)
if not self.authority.commit_exists(values["base_oid"]): raise BootstrapError("BASE_COMMIT_NOT_FOUND")
current = self.authority.read_ref("refs/heads/" + values["target_branch"])
expected = str(request.get("expected_branch_oid") or values["base_oid"])
if current is None:
mode, generation = "CREATE_NEW", 1
if expected != values["base_oid"]: raise BootstrapError("EXPECTED_BASE_CONFLICT")
else:
if expected != current: raise BootstrapError("EXISTING_BRANCH_OID_CONFLICT")
mode, generation = "CONTINUE_EXISTING", int(request.get("generation", 2))
if generation < 1: raise BootstrapError("GENERATION_INVALID")
payload = json.dumps(values, sort_keys=True, separators=(",", ":")).encode()
_, controller = self._store(values["task_id"])
receipt = controller.submit({"idempotency_key": values["idempotency_key"], "campaign_id": values["task_id"], "task_id": values["task_id"], "repository": values["repository"], "operation": "GOVERNED_TASK_CREATE", "branch": values["target_branch"], "expected_branch_oid": expected, "payload_digest": hashlib.sha256(payload).hexdigest(), "lineage_anchor_oid": values["base_oid"]}, payload=payload)
binding = GovernedTaskBinding(**values, expected_branch_oid=expected, branch_mode=mode, generation=generation, job_id=receipt.job_id)
self._write(binding)
return binding.response()
@@ -1,27 +0,0 @@
"""AIOS-managed Hermes runtime boundary."""
from .adapter import (
HermesAdapter,
HermesConflict,
HermesNotFound,
HermesRuntimeError,
InMemoryHermesStore,
SQLiteHermesStore,
)
from .contract import HermesHealth, HermesJob, HermesJobState, HermesResult
from .worker import HermesWorker, WorkerRunResult
__all__ = [
"HermesAdapter",
"HermesConflict",
"HermesHealth",
"HermesJob",
"HermesJobState",
"HermesNotFound",
"HermesResult",
"HermesRuntimeError",
"HermesWorker",
"InMemoryHermesStore",
"SQLiteHermesStore",
"WorkerRunResult",
]
-593
View File
@@ -1,593 +0,0 @@
"""Tenant-partitioned Hermes queue and worker lease boundary."""
from __future__ import annotations
import hashlib
import json
import sqlite3
import threading
from dataclasses import asdict, replace
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Iterable, Mapping
from .contract import (
ALLOWED_TRANSITIONS,
TERMINAL_STATES,
HermesHealth,
HermesJob,
HermesJobState,
HermesResult,
utc_now,
)
class HermesRuntimeError(RuntimeError):
"""Base Hermes runtime-boundary failure."""
class HermesNotFound(HermesRuntimeError):
"""A tenant-scoped job was not found."""
class HermesConflict(HermesRuntimeError):
"""A state, lease, idempotency, or revision conflict occurred."""
class InMemoryHermesStore:
"""Thread-safe reference store; production backends implement this contract."""
def __init__(self) -> None:
self.lock = threading.RLock()
self.jobs: dict[str, HermesJob] = {}
self.results: dict[str, HermesResult] = {}
self.idempotency: dict[tuple[str, str], tuple[str, str]] = {}
self.worker_heartbeats: dict[str, tuple[str, str | None]] = {}
def persist_locked(self) -> None:
"""Persist a mutation while ``lock`` is held; memory stores are a no-op."""
class SQLiteHermesStore(InMemoryHermesStore):
"""Durable snapshot for the existing Hermes queue state machine.
The adapter remains the sole owner of transitions and lease rules. This
store only makes its jobs, results, idempotency keys and worker heartbeats
survive a process restart.
"""
def __init__(self, path: str | Path) -> None:
self.path = Path(path).resolve()
self.path.parent.mkdir(parents=True, exist_ok=True)
super().__init__()
with self._connect() as connection:
connection.execute(
"""
CREATE TABLE IF NOT EXISTS hermes_state (
singleton INTEGER PRIMARY KEY CHECK (singleton = 1),
payload TEXT NOT NULL
)
"""
)
self._load()
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(str(self.path), timeout=30)
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA synchronous=FULL")
return connection
@staticmethod
def _record(value: HermesJob | HermesResult) -> dict[str, object]:
record = asdict(value)
record["state"] = value.state.value
return record
def _load(self) -> None:
with self.lock, self._connect() as connection:
row = connection.execute(
"SELECT payload FROM hermes_state WHERE singleton = 1"
).fetchone()
if row is None:
return
payload = json.loads(str(row[0]))
self.jobs = {
str(item["job_id"]): HermesJob(**item)
for item in payload.get("jobs", [])
}
self.results = {
str(item["job_id"]): HermesResult(**item)
for item in payload.get("results", [])
}
self.idempotency = {
(str(item["tenant_id"]), str(item["idempotency_key"])): (
str(item["job_id"]),
str(item["digest"]),
)
for item in payload.get("idempotency", [])
}
self.worker_heartbeats = {
str(item["worker_id"]): (
str(item["heartbeat_at"]),
str(item["job_id"]) if item.get("job_id") is not None else None,
)
for item in payload.get("worker_heartbeats", [])
}
def persist_locked(self) -> None:
payload = {
"version": 1,
"jobs": [self._record(item) for item in self.jobs.values()],
"results": [self._record(item) for item in self.results.values()],
"idempotency": [
{
"tenant_id": tenant_id,
"idempotency_key": idempotency_key,
"job_id": job_id,
"digest": digest,
}
for (tenant_id, idempotency_key), (job_id, digest)
in self.idempotency.items()
],
"worker_heartbeats": [
{
"worker_id": worker_id,
"heartbeat_at": heartbeat_at,
"job_id": job_id,
}
for worker_id, (heartbeat_at, job_id)
in self.worker_heartbeats.items()
],
}
encoded = json.dumps(
payload,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
connection.execute(
"""
INSERT INTO hermes_state(singleton, payload) VALUES(1, ?)
ON CONFLICT(singleton) DO UPDATE SET payload=excluded.payload
""",
(encoded,),
)
connection.commit()
class HermesAdapter:
"""Queue and reconcile Hermes jobs without owning AIOS policy."""
def __init__(
self,
store: InMemoryHermesStore | None = None,
*,
runtime_id: str = "hermes-runtime",
runtime_version: str = "v1",
default_lease_seconds: int = 60,
) -> None:
if default_lease_seconds < 5:
raise ValueError("default_lease_seconds must be >= 5")
self.store = store or InMemoryHermesStore()
self.runtime_id = runtime_id
self.runtime_version = runtime_version
self.default_lease_seconds = int(default_lease_seconds)
@staticmethod
def _digest(job: HermesJob) -> str:
raw = json.dumps(
{
"tenant_id": job.tenant_id,
"cell_id": job.cell_id,
"profile_id": job.profile_id,
"task_id": job.task_id,
"trace_id": job.trace_id,
"action": job.action,
"payload": dict(job.payload),
"priority": job.priority,
},
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
default=str,
).encode("utf-8")
return hashlib.sha256(raw).hexdigest()
@staticmethod
def _parse(value: str | None) -> datetime | None:
if value is None:
return None
normalized = value[:-1] + "+00:00" if value.endswith("Z") else value
parsed = datetime.fromisoformat(normalized)
if parsed.tzinfo is None:
raise ValueError("timestamp must include timezone")
return parsed.astimezone(timezone.utc)
@staticmethod
def _format(value: datetime) -> str:
return value.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")
@staticmethod
def _assert_tenant(job: HermesJob, tenant_id: str) -> None:
if job.tenant_id != tenant_id:
raise HermesNotFound("job not found")
def enqueue(self, job: HermesJob) -> HermesJob:
if job.state != HermesJobState.QUEUED or job.revision != 1:
raise HermesConflict("new job must be QUEUED at revision 1")
digest = self._digest(job)
idem_ref = (job.tenant_id, job.idempotency_key)
with self.store.lock:
existing = self.store.jobs.get(job.job_id)
if existing is not None:
if self._digest(existing) != digest:
raise HermesConflict("job_id reused with different content")
return existing
prior = self.store.idempotency.get(idem_ref)
if prior is not None:
prior_id, prior_digest = prior
if prior_digest != digest:
raise HermesConflict("idempotency key reused with different content")
return self.store.jobs[prior_id]
self.store.jobs[job.job_id] = job
self.store.idempotency[idem_ref] = (job.job_id, digest)
self.store.persist_locked()
return job
def get(self, job_id: str, *, tenant_id: str) -> HermesJob:
with self.store.lock:
job = self.store.jobs.get(job_id)
if job is None:
raise HermesNotFound("job not found")
self._assert_tenant(job, tenant_id)
return job
def list_jobs(
self,
*,
tenant_id: str,
cell_id: str | None = None,
profile_id: str | None = None,
states: Iterable[HermesJobState | str] | None = None,
) -> list[HermesJob]:
state_filter = {HermesJobState(item) for item in states} if states else None
with self.store.lock:
if self._requeue_expired_locked():
self.store.persist_locked()
jobs = [
job
for job in self.store.jobs.values()
if job.tenant_id == tenant_id
and (cell_id is None or job.cell_id == cell_id)
and (profile_id is None or job.profile_id == profile_id)
and (state_filter is None or job.state in state_filter)
]
return sorted(jobs, key=lambda item: (item.priority, item.created_at, item.job_id))
def claim(
self,
*,
tenant_id: str,
cell_id: str,
profile_id: str,
worker_id: str,
lease_seconds: int | None = None,
job_id: str | None = None,
) -> HermesJob | None:
worker_id = str(worker_id or "").strip()
if not worker_id:
raise ValueError("worker_id is required")
duration = int(lease_seconds or self.default_lease_seconds)
if duration < 5:
raise ValueError("lease_seconds must be >= 5")
with self.store.lock:
self._requeue_expired_locked()
candidates = [
job
for job in self.store.jobs.values()
if job.tenant_id == tenant_id
and job.cell_id == cell_id
and job.profile_id == profile_id
and (job_id is None or job.job_id == job_id)
and job.state == HermesJobState.QUEUED
]
self.store.worker_heartbeats[worker_id] = (utc_now(), None)
if not candidates:
self.store.persist_locked()
return None
current = sorted(
candidates,
key=lambda item: (item.priority, item.created_at, item.job_id),
)[0]
claimed = self._transition_locked(
current,
HermesJobState.CLAIMED,
assigned_worker_id=worker_id,
lease_expires_at=self._format(
datetime.now(timezone.utc) + timedelta(seconds=duration)
),
)
self.store.worker_heartbeats[worker_id] = (utc_now(), claimed.job_id)
self.store.persist_locked()
return claimed
def start(
self,
job_id: str,
*,
tenant_id: str,
worker_id: str,
expected_revision: int | None = None,
) -> HermesJob:
with self.store.lock:
self._requeue_expired_locked()
current = self._owned_job_locked(job_id, tenant_id, worker_id)
self._assert_revision(current, expected_revision)
updated = self._transition_locked(current, HermesJobState.RUNNING)
self.store.persist_locked()
return updated
def heartbeat(
self,
job_id: str,
*,
tenant_id: str,
worker_id: str,
lease_seconds: int | None = None,
) -> HermesJob:
duration = int(lease_seconds or self.default_lease_seconds)
if duration < 5:
raise ValueError("lease_seconds must be >= 5")
with self.store.lock:
self._requeue_expired_locked()
current = self._owned_job_locked(job_id, tenant_id, worker_id)
if current.state not in {HermesJobState.CLAIMED, HermesJobState.RUNNING}:
raise HermesConflict("job does not accept heartbeat")
updated = replace(
current,
revision=current.revision + 1,
lease_expires_at=self._format(
datetime.now(timezone.utc) + timedelta(seconds=duration)
),
updated_at=utc_now(),
)
self.store.jobs[job_id] = updated
self.store.worker_heartbeats[worker_id] = (utc_now(), job_id)
self.store.persist_locked()
return updated
def complete(
self,
job_id: str,
*,
tenant_id: str,
worker_id: str,
output: Mapping[str, object],
expected_revision: int | None = None,
) -> HermesResult:
return self._finish(
job_id,
tenant_id=tenant_id,
worker_id=worker_id,
state=HermesJobState.COMPLETED,
output=output,
expected_revision=expected_revision,
)
def fail(
self,
job_id: str,
*,
tenant_id: str,
worker_id: str,
error: Mapping[str, object],
expected_revision: int | None = None,
) -> HermesResult:
return self._finish(
job_id,
tenant_id=tenant_id,
worker_id=worker_id,
state=HermesJobState.FAILED,
error=error,
expected_revision=expected_revision,
)
def cancel(
self,
job_id: str,
*,
tenant_id: str,
expected_revision: int | None = None,
) -> HermesResult:
with self.store.lock:
current = self.store.jobs.get(job_id)
if current is None:
raise HermesNotFound("job not found")
self._assert_tenant(current, tenant_id)
self._assert_revision(current, expected_revision)
if current.state in TERMINAL_STATES:
result = self.store.results.get(job_id)
if result is None:
raise HermesConflict("terminal result missing")
if result.state != HermesJobState.CANCELLED:
raise HermesConflict("job already has a different terminal result")
return result
updated = self._transition_locked(current, HermesJobState.CANCELLED)
result = HermesResult(
job_id=updated.job_id,
tenant_id=updated.tenant_id,
task_id=updated.task_id,
trace_id=updated.trace_id,
state=HermesJobState.CANCELLED,
worker_id=updated.assigned_worker_id,
error={"code": "cancelled", "message": "cancelled by AIOS"},
finished_at=utc_now(),
)
self.store.results[job_id] = result
self.store.persist_locked()
return result
def result(self, job_id: str, *, tenant_id: str) -> HermesResult:
with self.store.lock:
job = self.store.jobs.get(job_id)
if job is None:
raise HermesNotFound("job not found")
self._assert_tenant(job, tenant_id)
result = self.store.results.get(job_id)
if result is None:
raise HermesConflict("job has no terminal result")
return result
def requeue_expired(self) -> tuple[str, ...]:
"""Persistently release expired leases without changing policy state."""
with self.store.lock:
changed = self._requeue_expired_locked()
if changed:
self.store.persist_locked()
return tuple(sorted(changed))
def health(self) -> HermesHealth:
with self.store.lock:
if self._requeue_expired_locked():
self.store.persist_locked()
active_jobs = sum(
1 for job in self.store.jobs.values()
if job.state in {HermesJobState.CLAIMED, HermesJobState.RUNNING}
)
queued_jobs = sum(
1 for job in self.store.jobs.values()
if job.state == HermesJobState.QUEUED
)
partitions = {
(job.tenant_id, job.cell_id, job.profile_id)
for job in self.store.jobs.values()
}
active_workers = sum(
1 for heartbeat, _ in self.store.worker_heartbeats.values()
if self._parse(heartbeat)
and self._parse(heartbeat) >= datetime.now(timezone.utc) - timedelta(seconds=120)
)
return HermesHealth(
runtime_id=self.runtime_id,
status="healthy",
version=self.runtime_version,
heartbeat_at=utc_now(),
active_workers=active_workers,
active_jobs=active_jobs,
queued_jobs=queued_jobs,
tenant_partitions=len(partitions),
profile_isolation_enforced=True,
details={"store": type(self.store).__name__},
)
def _finish(
self,
job_id: str,
*,
tenant_id: str,
worker_id: str,
state: HermesJobState,
output: Mapping[str, object] | None = None,
error: Mapping[str, object] | None = None,
expected_revision: int | None,
) -> HermesResult:
with self.store.lock:
self._requeue_expired_locked()
current = self._owned_job_locked(job_id, tenant_id, worker_id)
if current.state in TERMINAL_STATES:
existing = self.store.results.get(job_id)
if existing is None:
raise HermesConflict("terminal result missing")
if (
existing.state == state
and existing.worker_id == worker_id
and existing.output == output
and existing.error == error
):
return existing
raise HermesConflict("job already has a different terminal result")
self._assert_revision(current, expected_revision)
updated = self._transition_locked(current, state)
result = HermesResult(
job_id=updated.job_id,
tenant_id=updated.tenant_id,
task_id=updated.task_id,
trace_id=updated.trace_id,
state=state,
worker_id=worker_id,
output=output,
error=error,
started_at=current.updated_at,
finished_at=utc_now(),
)
self.store.results[job_id] = result
self.store.worker_heartbeats[worker_id] = (utc_now(), None)
self.store.persist_locked()
return result
def _owned_job_locked(self, job_id: str, tenant_id: str, worker_id: str) -> HermesJob:
current = self.store.jobs.get(job_id)
if current is None:
raise HermesNotFound("job not found")
self._assert_tenant(current, tenant_id)
if current.assigned_worker_id != worker_id:
raise HermesConflict("worker does not own job lease")
return current
@staticmethod
def _assert_revision(job: HermesJob, expected: int | None) -> None:
if expected is not None and job.revision != expected:
raise HermesConflict("job revision changed")
def _transition_locked(
self,
current: HermesJob,
target: HermesJobState,
*,
assigned_worker_id: str | None = None,
lease_expires_at: str | None = None,
) -> HermesJob:
if target not in ALLOWED_TRANSITIONS[current.state]:
raise HermesConflict(
f"invalid transition {current.state.value} -> {target.value}"
)
updated = replace(
current,
state=target,
revision=current.revision + 1,
assigned_worker_id=(
assigned_worker_id
if assigned_worker_id is not None
else current.assigned_worker_id
),
lease_expires_at=(
lease_expires_at
if lease_expires_at is not None
else (None if target in TERMINAL_STATES else current.lease_expires_at)
),
updated_at=utc_now(),
)
self.store.jobs[current.job_id] = updated
return updated
def _requeue_expired_locked(self) -> set[str]:
now = datetime.now(timezone.utc)
changed: set[str] = set()
for current in list(self.store.jobs.values()):
expiry = self._parse(current.lease_expires_at)
if (
current.state in {HermesJobState.CLAIMED, HermesJobState.RUNNING}
and expiry is not None
and expiry <= now
):
self.store.jobs[current.job_id] = replace(
current,
state=HermesJobState.QUEUED,
revision=current.revision + 1,
assigned_worker_id=None,
lease_expires_at=None,
updated_at=utc_now(),
)
changed.add(current.job_id)
return changed
-153
View File
@@ -1,153 +0,0 @@
"""Tenant-scoped Hermes runtime contracts."""
from __future__ import annotations
import json
import uuid
from dataclasses import dataclass, field
from datetime import datetime, timezone
from enum import Enum
from typing import Any, Mapping
def utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _required(value: str, label: str) -> str:
normalized = str(value or "").strip()
if not normalized:
raise ValueError(f"{label} is required")
return normalized
def _safe_mapping(value: Mapping[str, Any] | None) -> dict[str, Any]:
return json.loads(json.dumps(dict(value or {}), ensure_ascii=False, sort_keys=True, default=str))
class HermesJobState(str, Enum):
QUEUED = "QUEUED"
CLAIMED = "CLAIMED"
RUNNING = "RUNNING"
COMPLETED = "COMPLETED"
FAILED = "FAILED"
CANCELLED = "CANCELLED"
ALLOWED_TRANSITIONS: Mapping[HermesJobState, frozenset[HermesJobState]] = {
HermesJobState.QUEUED: frozenset({HermesJobState.CLAIMED, HermesJobState.CANCELLED}),
HermesJobState.CLAIMED: frozenset({HermesJobState.RUNNING, HermesJobState.CANCELLED}),
HermesJobState.RUNNING: frozenset(
{HermesJobState.COMPLETED, HermesJobState.FAILED, HermesJobState.CANCELLED}
),
HermesJobState.COMPLETED: frozenset(),
HermesJobState.FAILED: frozenset(),
HermesJobState.CANCELLED: frozenset(),
}
TERMINAL_STATES = frozenset(
{HermesJobState.COMPLETED, HermesJobState.FAILED, HermesJobState.CANCELLED}
)
@dataclass(frozen=True)
class HermesJob:
job_id: str
tenant_id: str
cell_id: str
profile_id: str
task_id: str
trace_id: str
action: str
payload: Mapping[str, Any] = field(default_factory=dict)
idempotency_key: str = ""
priority: int = 100
state: HermesJobState = HermesJobState.QUEUED
revision: int = 1
assigned_worker_id: str | None = None
lease_expires_at: str | None = None
created_at: str = field(default_factory=utc_now)
updated_at: str = field(default_factory=utc_now)
def __post_init__(self) -> None:
for name in (
"job_id", "tenant_id", "cell_id", "profile_id", "task_id",
"trace_id", "action", "idempotency_key", "created_at", "updated_at",
):
object.__setattr__(self, name, _required(getattr(self, name), name))
object.__setattr__(self, "payload", _safe_mapping(self.payload))
object.__setattr__(self, "state", HermesJobState(self.state))
if not 0 <= int(self.priority) <= 1000:
raise ValueError("priority must be between 0 and 1000")
if self.revision < 1:
raise ValueError("revision must be >= 1")
@classmethod
def create(
cls,
*,
tenant_id: str,
cell_id: str,
profile_id: str,
task_id: str,
trace_id: str,
action: str,
payload: Mapping[str, Any] | None,
idempotency_key: str,
priority: int = 100,
) -> "HermesJob":
return cls(
job_id=f"hermes_{uuid.uuid4().hex}",
tenant_id=tenant_id,
cell_id=cell_id,
profile_id=profile_id,
task_id=task_id,
trace_id=trace_id,
action=action,
payload=_safe_mapping(payload),
idempotency_key=idempotency_key,
priority=priority,
)
@dataclass(frozen=True)
class HermesResult:
job_id: str
tenant_id: str
task_id: str
trace_id: str
state: HermesJobState
worker_id: str | None = None
output: Mapping[str, Any] | None = None
error: Mapping[str, Any] | None = None
started_at: str | None = None
finished_at: str | None = None
def __post_init__(self) -> None:
for name in ("job_id", "tenant_id", "task_id", "trace_id"):
object.__setattr__(self, name, _required(getattr(self, name), name))
object.__setattr__(self, "state", HermesJobState(self.state))
if self.output is not None:
object.__setattr__(self, "output", _safe_mapping(self.output))
if self.error is not None:
object.__setattr__(self, "error", _safe_mapping(self.error))
@dataclass(frozen=True)
class HermesHealth:
runtime_id: str
status: str
version: str
heartbeat_at: str
active_workers: int = 0
active_jobs: int = 0
queued_jobs: int = 0
tenant_partitions: int = 0
profile_isolation_enforced: bool = True
details: Mapping[str, Any] = field(default_factory=dict)
def __post_init__(self) -> None:
for name in ("runtime_id", "status", "version", "heartbeat_at"):
object.__setattr__(self, name, _required(getattr(self, name), name))
for name in ("active_workers", "active_jobs", "queued_jobs", "tenant_partitions"):
if int(getattr(self, name)) < 0:
raise ValueError(f"{name} must be >= 0")
object.__setattr__(self, "details", _safe_mapping(self.details))
-155
View File
@@ -1,155 +0,0 @@
"""Hermes worker loop with lease renewal and bounded recovery."""
from __future__ import annotations
import threading
from dataclasses import dataclass
from typing import Any, Callable, Mapping
from .adapter import HermesAdapter, HermesConflict
from .contract import HermesJob, HermesJobState
JobHandler = Callable[[HermesJob], Mapping[str, Any]]
@dataclass(frozen=True)
class WorkerRunResult:
claimed: bool
job_id: str | None = None
state: str | None = None
error: str | None = None
class HermesWorker:
"""Poll one tenant/cell/profile partition and execute jobs safely."""
def __init__(
self,
adapter: HermesAdapter,
handler: JobHandler,
*,
worker_id: str,
tenant_id: str,
cell_id: str,
profile_id: str,
poll_seconds: float = 1.0,
heartbeat_seconds: float = 10.0,
) -> None:
if poll_seconds <= 0 or heartbeat_seconds <= 0:
raise ValueError("poll_seconds and heartbeat_seconds must be positive")
for name, value in {
"worker_id": worker_id,
"tenant_id": tenant_id,
"cell_id": cell_id,
"profile_id": profile_id,
}.items():
if not str(value or "").strip():
raise ValueError(f"{name} is required")
self.adapter = adapter
self.handler = handler
self.worker_id = worker_id
self.tenant_id = tenant_id
self.cell_id = cell_id
self.profile_id = profile_id
self.poll_seconds = float(poll_seconds)
self.heartbeat_seconds = float(heartbeat_seconds)
self._stop = threading.Event()
def stop(self) -> None:
self._stop.set()
def run_once(self) -> WorkerRunResult:
job = self.adapter.claim(
tenant_id=self.tenant_id,
cell_id=self.cell_id,
profile_id=self.profile_id,
worker_id=self.worker_id,
)
if job is None:
return WorkerRunResult(claimed=False)
running = self.adapter.start(
job.job_id,
tenant_id=self.tenant_id,
worker_id=self.worker_id,
expected_revision=job.revision,
)
heartbeat_stop = threading.Event()
heartbeat_thread = threading.Thread(
target=self._heartbeat_loop,
args=(running.job_id, heartbeat_stop),
name=f"hermes-heartbeat-{self.worker_id}",
daemon=True,
)
heartbeat_thread.start()
output: Mapping[str, Any] | None = None
execution_error: Exception | None = None
try:
output = dict(self.handler(running))
except Exception as exc: # runtime boundary: normalize below
execution_error = exc
finally:
# Stop the lease heartbeat before reading the final revision. This
# prevents a revision increment racing with complete/fail.
heartbeat_stop.set()
heartbeat_thread.join(timeout=max(self.heartbeat_seconds * 2, 1.0))
try:
current = self.adapter.get(running.job_id, tenant_id=self.tenant_id)
if execution_error is None:
result = self.adapter.complete(
running.job_id,
tenant_id=self.tenant_id,
worker_id=self.worker_id,
output=dict(output or {}),
expected_revision=current.revision,
)
return WorkerRunResult(
claimed=True,
job_id=result.job_id,
state=result.state.value,
)
result = self.adapter.fail(
running.job_id,
tenant_id=self.tenant_id,
worker_id=self.worker_id,
error={
"code": "worker_error",
"message": str(execution_error),
},
expected_revision=current.revision,
)
return WorkerRunResult(
claimed=True,
job_id=result.job_id,
state=result.state.value,
error=str(execution_error),
)
except HermesConflict as exc:
return WorkerRunResult(
claimed=True,
job_id=running.job_id,
state=HermesJobState.FAILED.value,
error=f"reconciliation_conflict: {exc}",
)
def run_forever(self) -> None:
while not self._stop.is_set():
result = self.run_once()
if not result.claimed:
self._stop.wait(self.poll_seconds)
def _heartbeat_loop(self, job_id: str, stop_event: threading.Event) -> None:
while not stop_event.wait(self.heartbeat_seconds):
try:
self.adapter.heartbeat(
job_id,
tenant_id=self.tenant_id,
worker_id=self.worker_id,
)
except Exception:
stop_event.set()
return
@@ -1,23 +0,0 @@
"""Task-bound Integration Refresh control-plane primitives."""
from .integration_merge_writer import IntegrationMergeWriter, MergeError, MergeRequest
from .integration_refresh import (
AppendOnlyAudit,
Binding,
GiteaBranchReadProjection,
GrantStore,
IntegrationRefreshProvisioner,
ProvisioningError,
)
__all__ = [
"AppendOnlyAudit",
"Binding",
"GiteaBranchReadProjection",
"GrantStore",
"IntegrationMergeWriter",
"IntegrationRefreshProvisioner",
"MergeError",
"MergeRequest",
"ProvisioningError",
]
@@ -1,90 +0,0 @@
"""Constrained integration-only merge writer with rollback and reconciliation states."""
from __future__ import annotations
import subprocess
from dataclasses import dataclass
from pathlib import Path
import re
class MergeError(RuntimeError):
def __init__(self, code: str, status: int = 409):
super().__init__(code); self.code, self.status = code, status
@dataclass(frozen=True)
class MergeRequest:
integration_branch: str
source_branch: str
approved_source_oid: str
class IntegrationMergeWriter:
def __init__(self, workspace: Path | str, remote: str, projection):
self.workspace, self.remote, self.projection = Path(workspace), remote, projection
def _git(self, *args: str, check=True) -> str:
p = subprocess.run(["git", *args], cwd=self.workspace, text=True, capture_output=True)
if check and p.returncode:
raise MergeError("git_operation_failed", 500)
return p.stdout.strip()
def merge(self, request: MergeRequest) -> dict:
def bad(value: str) -> bool:
return (
value == "main"
or value.startswith("main/")
or value == "governance"
or value.startswith("governance/")
or value == "approvals"
or value.startswith("refs/")
or value.startswith("tags/")
)
if (bad(request.integration_branch) or bad(request.source_branch) or
not request.integration_branch.startswith("integration/") or
re.fullmatch(r"[0-9a-fA-F]{40}", request.approved_source_oid) is None):
raise MergeError("integration_merge_scope_rejected", 403)
for branch in (request.integration_branch, request.source_branch):
if subprocess.run(["git", "check-ref-format", "--branch", branch], cwd=self.workspace,
text=True, capture_output=True).returncode:
raise MergeError("integration_merge_scope_rejected", 403)
if self._git("rev-parse", "--abbrev-ref", "HEAD") != request.integration_branch:
raise MergeError("integration_branch_checkout_required")
before = self._git("rev-parse", "HEAD")
remote_before = self.projection(request.integration_branch)
if remote_before != before:
raise MergeError("integration_remote_advanced")
source = self._git("rev-parse", "--verify", f"refs/heads/{request.source_branch}^{{commit}}")
if source != request.approved_source_oid:
raise MergeError("integration_source_not_approved", 403)
attempt = subprocess.run(["git", "merge", "--no-ff", "--no-commit", request.source_branch], cwd=self.workspace, text=True, capture_output=True)
if attempt.returncode:
subprocess.run(["git", "merge", "--abort"], cwd=self.workspace, text=True, capture_output=True)
raise MergeError("integration_merge_conflict")
try:
self._git("commit", "-m", "merge(integration): approved source")
merged = self._git("rev-parse", "HEAD")
parents = self._git("show", "-s", "--format=%P", "HEAD").split()
if parents != [before, source]:
raise MergeError("integration_parent_invariant_failed", 500)
pushed = subprocess.run(["git", "push", "--porcelain", self.remote, f"HEAD:refs/heads/{request.integration_branch}"], cwd=self.workspace, text=True, capture_output=True)
if pushed.returncode:
try:
observed = self.projection(request.integration_branch)
except Exception:
return {"state": "RECONCILIATION_REQUIRED", "pre_merge_oid": before, "merge_oid": merged}
if observed == before:
self._git("reset", "--hard", before)
return {"state": "PUSH_FAILED_RESTORED", "pre_merge_oid": before}
return {"state": "RECONCILIATION_REQUIRED", "pre_merge_oid": before, "merge_oid": merged}
try:
observed = self.projection(request.integration_branch)
except Exception:
return {"state": "RECONCILIATION_REQUIRED", "pre_merge_oid": before, "merge_oid": merged}
if observed != merged:
return {"state": "RECONCILIATION_REQUIRED", "pre_merge_oid": before, "merge_oid": merged}
return {"state": "COMPLETED", "pre_merge_oid": before, "merge_oid": merged}
except MergeError:
if self._git("rev-parse", "HEAD") != before:
self._git("reset", "--hard", before)
raise
@@ -1,257 +0,0 @@
"""Fail-closed, task-bound integration refresh provisioning primitives.
This module deliberately has no HTTP or credential-provider dependency. The
caller supplies a branch-read projection and an apply callback, making the
authorization and recovery boundary independently testable.
"""
from __future__ import annotations
import hashlib
import json
import os
import argparse
import secrets
import sqlite3
import threading
import re
from contextlib import contextmanager
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Callable
class ProvisioningError(RuntimeError):
def __init__(self, code: str, status: int = 403):
super().__init__(code)
self.code, self.status = code, status
def _now() -> datetime:
return datetime.now(timezone.utc)
def _digest(secret: str) -> str:
return hashlib.sha256(secret.encode("utf-8")).hexdigest()
def _canonical(value: dict) -> str:
return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(",", ":")).encode()).hexdigest()
def _is_oid(value: object) -> bool:
return isinstance(value, str) and re.fullmatch(r"[0-9a-fA-F]{40}", value) is not None
@dataclass(frozen=True)
class Binding:
task_id: str
repository: str
branch: str
actor_identity: str
audience: str
paths: tuple[str, ...]
quota: int
class AppendOnlyAudit:
"""Append only redacted JSONL audit, durable before a result is returned."""
def __init__(self, path: Path | str):
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
self._lock = threading.Lock()
def append(self, event: dict) -> None:
with self._lock:
safe = {key: value for key, value in event.items() if key not in {"token", "credential", "authorization"}}
previous = "0" * 64
if self.path.exists():
lines = self.path.read_text(encoding="utf-8").splitlines()
if lines:
previous = json.loads(lines[-1])["event_hash"]
safe["previous_event_hash"] = previous
safe["event_hash"] = _canonical(safe)
encoded = (json.dumps(safe, sort_keys=True, separators=(",", ":")) + "\n").encode()
fd = os.open(self.path, os.O_APPEND | os.O_CREAT | os.O_WRONLY, 0o600)
try:
os.write(fd, encoded)
os.fsync(fd)
finally:
os.close(fd)
class GrantStore:
"""Digest-only short grant registry; SQLite transactions serialize quota use."""
def __init__(self, path: Path | str):
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
with self._db() as db:
db.execute("""CREATE TABLE IF NOT EXISTS grants(
token_digest TEXT PRIMARY KEY, binding_json TEXT NOT NULL,
expires_at TEXT NOT NULL, quota INTEGER NOT NULL, uses INTEGER NOT NULL,
revoked INTEGER NOT NULL DEFAULT 0)""")
@contextmanager
def _db(self):
db = sqlite3.connect(self.path, timeout=15, isolation_level=None)
db.row_factory = sqlite3.Row
try:
yield db
finally:
db.close()
def issue(self, binding: Binding, *, ttl_seconds: int = 300) -> str:
if ttl_seconds < 1 or ttl_seconds > 1800 or binding.quota < 1 or not binding.paths:
raise ValueError("invalid_short_grant")
token = secrets.token_urlsafe(32)
payload = json.dumps(binding.__dict__ | {"paths": list(binding.paths)}, sort_keys=True)
with self._db() as db:
db.execute("INSERT INTO grants VALUES(?,?,?,?,?,0)", (_digest(token), payload,
(_now() + timedelta(seconds=ttl_seconds)).isoformat(), binding.quota, 0))
return token
def rotate(self, old_token: str, binding: Binding, *, revocation_probe, ttl_seconds: int = 300) -> str:
self.revoke(old_token)
try: status = revocation_probe(old_token)
except TimeoutError as exc: raise ProvisioningError("revocation_probe_timeout", 503) from exc
if status != 401: raise ProvisioningError("revocation_probe_rejected", 503)
return self.issue(binding, ttl_seconds=ttl_seconds)
def revoke(self, token: str) -> None:
with self._db() as db:
db.execute("BEGIN IMMEDIATE")
db.execute("UPDATE grants SET revoked=1 WHERE token_digest=?", (_digest(token),))
db.execute("COMMIT")
def consume(self, token: str, binding: Binding, path: str) -> None:
with self._db() as db:
db.execute("BEGIN IMMEDIATE")
row = db.execute("SELECT * FROM grants WHERE token_digest=?", (_digest(token),)).fetchone()
if not row or row["revoked"]:
db.execute("ROLLBACK"); raise ProvisioningError("credential_rejected", 401)
if _now() >= datetime.fromisoformat(row["expires_at"]):
db.execute("ROLLBACK"); raise ProvisioningError("credential_expired", 401)
stored = json.loads(row["binding_json"])
requested = binding.__dict__ | {"paths": list(binding.paths)}
if stored != requested or path not in stored["paths"]:
db.execute("ROLLBACK"); raise ProvisioningError("grant_binding_rejected")
if row["uses"] >= row["quota"]:
db.execute("ROLLBACK"); raise ProvisioningError("grant_quota_exhausted")
db.execute("UPDATE grants SET uses=uses+1 WHERE token_digest=?", (_digest(token),))
db.execute("COMMIT")
class GiteaBranchReadProjection:
"""Normalizes a trusted task-scoped Gitea reader; every ambiguity fails closed."""
def __init__(self, reader: Callable[[str, str], dict]):
self.reader = reader
def head(self, repository: str, branch: str) -> str:
try:
reply = self.reader(repository, branch)
except TimeoutError as exc:
raise ProvisioningError("gitea_projection_timeout", 503) from exc
except Exception as exc:
raise ProvisioningError("gitea_projection_unavailable", 503) from exc
if not isinstance(reply, dict) or reply.get("status") in {401, 403}:
raise ProvisioningError("gitea_projection_forbidden", 503)
oid = reply.get("oid")
if reply.get("status") != 200 or not _is_oid(oid):
raise ProvisioningError("gitea_projection_invalid", 503)
return oid.lower()
class IntegrationRefreshProvisioner:
_locks: dict[tuple[str, str], threading.Lock] = {}
_locks_guard = threading.Lock()
def __init__(self, *, grants: GrantStore, projection: GiteaBranchReadProjection,
ledger_path: Path | str, audit: AppendOnlyAudit, approved_binding: Binding):
self.grants, self.projection, self.audit, self.approved_binding = grants, projection, audit, approved_binding
self.ledger_path = Path(ledger_path); self.ledger_path.parent.mkdir(parents=True, exist_ok=True)
with self._ledger() as db:
db.execute("CREATE TABLE IF NOT EXISTS refreshes(operation_id TEXT PRIMARY KEY, digest TEXT NOT NULL, state TEXT NOT NULL, result_oid TEXT)")
@contextmanager
def _ledger(self):
db = sqlite3.connect(self.ledger_path)
try:
yield db
db.commit()
finally:
db.close()
def refresh(self, *, token: str, binding: Binding, path: str, operation_id: str,
target_head_oid: str, apply: Callable[[], str]) -> dict:
if not operation_id or not _is_oid(target_head_oid):
raise ProvisioningError("refresh_request_rejected")
if binding != self.approved_binding:
raise ProvisioningError("grant_binding_rejected")
request = {"binding": binding.__dict__ | {"paths": list(binding.paths)}, "path": path, "target": target_head_oid}
digest = _canonical(request)
with self._locks_guard:
lock = self._locks.setdefault((binding.repository, binding.branch), threading.Lock())
with lock, self._ledger() as db:
db.execute("BEGIN IMMEDIATE")
row = db.execute("SELECT * FROM refreshes WHERE operation_id=?", (operation_id,)).fetchone()
new_operation = row is None
if row:
if row[1] != digest:
db.execute("ROLLBACK"); raise ProvisioningError("operation_id_reuse_rejected", 409)
if row[2] == "COMPLETED":
db.execute("COMMIT")
return {"state": row[2], "operation_id": operation_id, "oid": row[3], "idempotent": True}
if row[2] not in {"PENDING", "RECOVERABLE"}:
db.execute("COMMIT")
return {"state": row[2], "operation_id": operation_id, "oid": row[3], "idempotent": True}
else:
db.execute("INSERT INTO refreshes VALUES(?,?,?,NULL)", (operation_id, digest, "PENDING"))
db.execute("COMMIT")
if new_operation:
self.grants.consume(token, binding, path)
current = self.projection.head(binding.repository, binding.branch)
if current != target_head_oid.lower():
self._state(operation_id, "STALE_TARGET")
raise ProvisioningError("stale_target_head", 409)
try:
oid = apply()
if not _is_oid(oid):
raise ProvisioningError("apply_result_invalid", 500)
# Detect a competing remote advancement before claiming completion.
observed = self.projection.head(binding.repository, binding.branch)
if observed != oid.lower():
self._state(operation_id, "RECONCILIATION_REQUIRED")
raise ProvisioningError("remote_concurrent_advance", 409)
self._state(operation_id, "COMPLETED", oid)
self.audit.append({"operation_id": operation_id, "task_id": binding.task_id, "repository": binding.repository,
"branch": binding.branch, "actor_identity": binding.actor_identity, "audience": binding.audience, "path": path, "state": "COMPLETED"})
return {"state": "COMPLETED", "operation_id": operation_id, "oid": oid, "idempotent": False}
except ProvisioningError:
raise
except Exception as exc:
self._state(operation_id, "RECOVERABLE")
self.audit.append({"operation_id": operation_id, "task_id": binding.task_id, "repository": binding.repository,
"branch": binding.branch, "actor_identity": binding.actor_identity, "audience": binding.audience, "path": path, "state": "RECOVERABLE"})
raise ProvisioningError("refresh_apply_failed", 503) from exc
def _state(self, operation_id: str, state: str, oid: str | None = None) -> None:
with self._ledger() as db:
db.execute("UPDATE refreshes SET state=?, result_oid=COALESCE(?,result_oid) WHERE operation_id=?", (state, oid, operation_id))
def _sandbox_preflight(state_dir: Path) -> int:
if os.environ.get("AIOS_GITOPS_WRITES_ENABLED") != "false":
raise ProvisioningError("global_gitops_writes_must_remain_disabled", 500)
state_dir.mkdir(parents=True, exist_ok=True)
audit = AppendOnlyAudit(state_dir / "preflight-audit.jsonl")
audit.append({"event": "SANDBOX_PREFLIGHT", "writes_enabled": False})
print(json.dumps({"status": "READY", "mode": "sandbox", "writes_enabled": False}, sort_keys=True))
return 0
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--sandbox-preflight", action="store_true", required=True)
parser.add_argument("--state-dir", type=Path, required=True)
options = parser.parse_args()
raise SystemExit(_sandbox_preflight(options.state_dir))
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,528 @@
"""Canonical governed execution projection for Online Agent Tasks.
The durable governed binding is authoritative. Natural-language task fields and
worker configuration are never allowed to invent repository, ref, generation, or
idempotency coordinates.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass
import hashlib
import json
import re
from pathlib import Path
from typing import Any, Mapping, Sequence
import urllib.error
import urllib.request
EXECUTION_SCHEMA = "aios.online-agent.execution.v2"
PROJECTION_SCHEMA = "aios.online-agent.task-projection.v1"
OID_RE = re.compile(r"^[0-9a-f]{40}$")
TASK_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{2,179}$")
BRANCH_RE = re.compile(r"^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$")
KEY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{7,159}$")
PROTECTED_PREFIXES = ("main", "master", "refs/", "tags/", "tag/", "release/", "baseline/", "production/")
MUTATION_ACTIONS = frozenset({"deterministic"})
READ_ONLY_ACTIONS = frozenset({"source_capture", "agent_runtime"})
SUPPORTED_ACTIONS = frozenset({"deterministic", "source_capture", "agent_runtime"})
AGENT_RUNTIME_ADAPTERS = frozenset({"hermes", "openclaw"})
MAX_AGENT_RUNTIME_INPUT_BYTES = 64 * 1024
class ExecutionProjectionError(RuntimeError):
"""A canonical fail-closed projection or validation error."""
class OnlineTaskProjectionClient:
"""Official HTTP projection client using only a control credential reference."""
def __init__(self, api_base: str, control_token_path: Path) -> None:
self.api_base = api_base.rstrip("/")
self.control_token_path = Path(control_token_path)
def _token(self) -> str:
path = self.control_token_path
if path.is_symlink() or not path.is_file():
_error("ONLINE_TASK_PROJECTION_CONTROL_UNAVAILABLE")
value = path.read_text(encoding="utf-8").strip()
if len(value) < 32:
_error("ONLINE_TASK_PROJECTION_CONTROL_INVALID")
return value
def create(
self,
projected_task: Mapping[str, Any],
*,
binding: "GovernedExecutionBinding",
) -> dict[str, Any]:
body = dict(projected_task)
payload = _canonical(body).encode("utf-8")
request = urllib.request.Request(
self.api_base + "/v1/tasks",
method="POST",
data=payload,
headers={
"Authorization": "Bearer " + self._token(),
"Content-Type": "application/json",
"Accept": "application/json",
},
)
try:
with urllib.request.urlopen(request, timeout=15) as response:
result = json.loads(response.read().decode("utf-8"))
status = response.status
except urllib.error.HTTPError as exc:
raw = exc.read(8192).decode("utf-8", "replace")
try:
error = json.loads(raw).get("error")
except (AttributeError, json.JSONDecodeError):
error = "HTTP_ERROR"
raise ExecutionProjectionError(f"ONLINE_TASK_PROJECTION_FAILED:{exc.code}:{error}") from exc
except (OSError, urllib.error.URLError, TimeoutError) as exc:
raise ExecutionProjectionError("ONLINE_TASK_PROJECTION_UNAVAILABLE") from exc
if status not in {200, 201} or not isinstance(result, dict):
_error("ONLINE_TASK_PROJECTION_RESPONSE_INVALID")
if result.get("task_id") != binding.task_id or result.get("status") not in {"QUEUED", "VERIFYING", "SUCCEEDED"}:
_error("ONLINE_TASK_PROJECTION_RESPONSE_INVALID")
metadata = result.get("metadata")
if not isinstance(metadata, Mapping):
_error("ONLINE_TASK_PROJECTION_RESPONSE_INVALID")
returned_execution = validate_execution_envelope(
_object(metadata.get("execution"), "ONLINE_TASK_PROJECTION_RESPONSE_INVALID"),
binding=binding,
)
submitted_metadata = _object(body.get("metadata"), "ONLINE_TASK_PROJECTION_REQUEST_INVALID")
submitted_execution = validate_execution_envelope(
_object(submitted_metadata.get("execution"), "ONLINE_TASK_PROJECTION_REQUEST_INVALID"),
binding=binding,
)
if _canonical(returned_execution) != _canonical(submitted_execution):
_error("ONLINE_TASK_PROJECTION_RESPONSE_BINDING_INVALID")
return result
def _error(code: str) -> None:
raise ExecutionProjectionError(code)
def _object(value: object, code: str) -> dict[str, Any]:
if not isinstance(value, Mapping):
_error(code)
return dict(value)
def _string(value: object, code: str) -> str:
if not isinstance(value, str) or not value.strip():
_error(code)
return value.strip()
def _task_id(value: object) -> str:
value = _string(value, "TASK_ID_INVALID")
if TASK_RE.fullmatch(value) is None:
_error("TASK_ID_INVALID")
return value
def _oid(value: object, code: str) -> str:
value = _string(value, code)
if OID_RE.fullmatch(value) is None:
_error(code)
return value
def _branch(value: object) -> str:
value = _string(value, "TARGET_BRANCH_INVALID")
if BRANCH_RE.fullmatch(value) is None or value.startswith(PROTECTED_PREFIXES):
_error("TARGET_BRANCH_INVALID")
return value
def _generation(value: object) -> int:
if not isinstance(value, int) or isinstance(value, bool) or value < 1:
_error("GENERATION_INVALID")
return value
def _canonical(value: object) -> str:
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
def _digest(value: object) -> str:
return hashlib.sha256(_canonical(value).encode("utf-8")).hexdigest()
def _source_signals(value: object) -> tuple[str, ...]:
"""Normalize untrusted natural-language hints before they enter a signed envelope."""
if value is None:
return ()
if not isinstance(value, Sequence) or isinstance(value, (str, bytes)):
_error("SOURCE_CAPTURE_SIGNALS_INVALID")
result: list[str] = []
for item in value:
if not isinstance(item, str):
_error("SOURCE_CAPTURE_SIGNALS_INVALID")
text = " ".join(item.split()).strip()
if not text:
continue
# Projection may accept long task prose, but execution envelopes stay bounded.
text = text[:512].rstrip()
if text not in result:
result.append(text)
if len(result) == 16:
break
return tuple(result)
def _agent_runtime_input(value: object) -> dict[str, Any]:
if not isinstance(value, Mapping):
_error("AGENT_RUNTIME_INPUT_REQUIRED")
raw = dict(value)
allowed = {
"goal", "instructions", "success_criteria", "constraints",
"scope", "trace_id",
}
if set(raw) - allowed:
_error("AGENT_RUNTIME_INPUT_FIELDS_INVALID")
goal = _string(raw.get("goal"), "AGENT_RUNTIME_GOAL_REQUIRED")
instructions = raw.get("instructions", "")
if not isinstance(instructions, str):
_error("AGENT_RUNTIME_INSTRUCTIONS_INVALID")
criteria = raw.get("success_criteria", [])
if not isinstance(criteria, list) or len(criteria) > 32:
_error("AGENT_RUNTIME_SUCCESS_CRITERIA_INVALID")
normalized_criteria: list[dict[str, str]] = []
for item in criteria:
if not isinstance(item, Mapping):
_error("AGENT_RUNTIME_SUCCESS_CRITERIA_INVALID")
criterion_id = _string(item.get("id"), "AGENT_RUNTIME_SUCCESS_CRITERIA_INVALID")
description = _string(
item.get("description"), "AGENT_RUNTIME_SUCCESS_CRITERIA_INVALID"
)
normalized_criteria.append(
{"id": criterion_id[:120], "description": description[:1000]}
)
constraints = raw.get("constraints", {})
if not isinstance(constraints, Mapping):
_error("AGENT_RUNTIME_CONSTRAINTS_INVALID")
sensitive_markers = ("token", "secret", "password", "credential", "authorization", "api_key", "apikey")
for key in constraints:
lowered = str(key).lower()
if any(marker in lowered for marker in sensitive_markers):
_error("AGENT_RUNTIME_SECRET_FIELD_DENY")
scope = raw.get("scope", {})
if not isinstance(scope, Mapping):
_error("AGENT_RUNTIME_SCOPE_INVALID")
scope = {
"scope_type": str(scope.get("scope_type") or "legacy"),
"tenant_id": scope.get("tenant_id"),
"project_id": scope.get("project_id"),
}
if scope["scope_type"] not in {"legacy", "system", "tenant"}:
_error("AGENT_RUNTIME_SCOPE_INVALID")
if scope["scope_type"] == "tenant":
if not isinstance(scope["tenant_id"], str) or not scope["tenant_id"].strip():
_error("AGENT_RUNTIME_SCOPE_INVALID")
elif scope["tenant_id"] not in (None, "") or scope["project_id"] not in (None, ""):
_error("AGENT_RUNTIME_SCOPE_INVALID")
trace_id = str(raw.get("trace_id") or "").strip()
if not trace_id or len(trace_id) > 255:
_error("AGENT_RUNTIME_TRACE_ID_INVALID")
normalized = {
"goal": goal[:12000],
"instructions": instructions[:32000],
"success_criteria": normalized_criteria,
"constraints": dict(constraints),
"scope": scope,
"trace_id": trace_id,
}
if len(_canonical(normalized).encode("utf-8")) > MAX_AGENT_RUNTIME_INPUT_BYTES:
_error("AGENT_RUNTIME_INPUT_TOO_LARGE")
return normalized
def _capabilities(value: object) -> tuple[str, ...]:
if value is None:
return ()
if not isinstance(value, Sequence) or isinstance(value, (str, bytes)):
_error("CAPABILITY_SCOPE_INVALID")
result = tuple(sorted({_string(item, "CAPABILITY_SCOPE_INVALID") for item in value}))
if any(len(item) > 120 for item in result):
_error("CAPABILITY_SCOPE_INVALID")
return result
@dataclass(frozen=True, slots=True)
class GovernedExecutionBinding:
task_id: str
job_id: str
repository: str
base_oid: str
expected_branch_oid: str
target_branch: str
branch_mode: str
generation: int
idempotency_key: str
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "GovernedExecutionBinding":
data = dict(value)
task_id = _task_id(data.get("task_id"))
job_id = _string(data.get("job_id"), "JOB_ID_REQUIRED")
if TASK_RE.fullmatch(job_id) is None:
_error("JOB_ID_INVALID")
repository = _string(data.get("repository"), "REPOSITORY_INVALID")
if repository != "aios/aios":
_error("WRONG_REPOSITORY_ENVELOPE_DENY")
base_oid = _oid(data.get("base_oid"), "BASE_OID_INVALID")
expected = _oid(data.get("expected_branch_oid"), "EXPECTED_BRANCH_OID_INVALID")
target = _branch(data.get("target_branch"))
mode = _string(data.get("branch_mode"), "BRANCH_MODE_MISSING")
if mode not in {"CREATE_NEW", "CONTINUE_EXISTING"}:
_error("BRANCH_MODE_INVALID")
generation = _generation(data.get("generation"))
if mode == "CREATE_NEW" and expected != base_oid:
_error("EXPECTED_BASE_CONFLICT")
key = _string(data.get("idempotency_key"), "IDEMPOTENCY_KEY_INVALID")
if KEY_RE.fullmatch(key) is None:
_error("IDEMPOTENCY_KEY_INVALID")
return cls(task_id, job_id, repository, base_oid, expected, target, mode, generation, key)
def public(self) -> dict[str, Any]:
return asdict(self)
def binding_from_bootstrap(value: Mapping[str, Any]) -> GovernedExecutionBinding:
return GovernedExecutionBinding.from_mapping(value)
def validate_execution_envelope(
value: Mapping[str, Any],
*,
binding: GovernedExecutionBinding | None = None,
worker_capabilities: Sequence[str] | None = None,
) -> dict[str, Any]:
envelope = dict(value)
if envelope.get("schema") != EXECUTION_SCHEMA:
_error("EXECUTION_ENVELOPE_SCHEMA_VALIDATION")
candidate = GovernedExecutionBinding.from_mapping(envelope)
if binding is not None and candidate != binding:
comparisons = (
(candidate.task_id, binding.task_id, "TASK_BINDING_CONFLICT"),
(candidate.job_id, binding.job_id, "JOB_BINDING_CONFLICT"),
(candidate.repository, binding.repository, "WRONG_REPOSITORY_ENVELOPE_DENY"),
(candidate.base_oid, binding.base_oid, "WRONG_BASE_ENVELOPE_DENY"),
(candidate.expected_branch_oid, binding.expected_branch_oid, "WRONG_EXPECTED_HEAD_ENVELOPE_DENY"),
(candidate.target_branch, binding.target_branch, "WRONG_BRANCH_ENVELOPE_DENY"),
(candidate.generation, binding.generation, "WRONG_GENERATION_ENVELOPE_DENY"),
(candidate.branch_mode, binding.branch_mode, "BRANCH_MODE_BINDING_CONFLICT"),
(candidate.idempotency_key, binding.idempotency_key, "IDEMPOTENCY_BINDING_CONFLICT"),
)
for actual, expected, code in comparisons:
if actual != expected:
_error(code)
action = _string(envelope.get("action"), "EXECUTION_ACTION_MISSING")
if action not in SUPPORTED_ACTIONS:
_error("EXECUTION_ACTION_UNSUPPORTED")
capability_scope = _capabilities(envelope.get("capability_scope"))
if worker_capabilities is not None and not set(capability_scope).issubset(set(worker_capabilities)):
_error("CAPABILITY_MISMATCH_DENY")
workspace = _object(envelope.get("workspace"), "WORKSPACE_POLICY_REQUIRED")
if workspace.get("authority_git_dir") != "/data/aios-gitops/repo.git" or workspace.get("read_only_repository") is not True:
_error("WORKSPACE_POLICY_DENY")
governance = _object(envelope.get("governance"), "GOVERNANCE_POLICY_REQUIRED")
governed_coordinates = (
(governance.get("task_id"), candidate.task_id, "TASK_BINDING_CONFLICT"),
(governance.get("job_id"), candidate.job_id, "JOB_BINDING_CONFLICT"),
(governance.get("idempotency_key"), candidate.idempotency_key, "IDEMPOTENCY_BINDING_CONFLICT"),
)
for actual_coordinate, expected_coordinate, code in governed_coordinates:
if actual_coordinate != expected_coordinate:
_error(code)
commit_key = _string(governance.get("commit_idempotency_key"), "COMMIT_IDEMPOTENCY_KEY_INVALID")
if KEY_RE.fullmatch(commit_key) is None:
_error("COMMIT_IDEMPOTENCY_KEY_INVALID")
if governance.get("writer_lease") is not (action in MUTATION_ACTIONS):
_error("WRITER_LEASE_POLICY_DENY")
for field in ("main_write", "production_write", "tag_write", "baseline_write", "force_push", "protected_ref_bypass"):
if governance.get(field, False) is not False:
_error("POLICY_DENY")
changes = envelope.get("change_set")
verification = envelope.get("verification")
if not isinstance(changes, list) or not isinstance(verification, list):
_error("EXECUTION_OPERATIONS_INVALID")
if action == "source_capture":
if changes or verification:
_error("SOURCE_CAPTURE_MUTATION_DENY")
signals = list(_source_signals(envelope.get("source_signals") or []))
if envelope.get("source_signals") != signals:
_error("SOURCE_CAPTURE_SIGNALS_INVALID")
if envelope.get("runtime_input") not in (None, {}):
_error("SOURCE_CAPTURE_RUNTIME_INPUT_DENY")
elif action == "agent_runtime":
if changes or verification:
_error("AGENT_RUNTIME_MUTATION_DENY")
adapter = str(envelope.get("adapter") or "")
if adapter not in AGENT_RUNTIME_ADAPTERS:
_error("AGENT_RUNTIME_ADAPTER_INVALID")
runtime_input = _agent_runtime_input(envelope.get("runtime_input"))
if envelope.get("runtime_input") != runtime_input:
_error("AGENT_RUNTIME_INPUT_NOT_CANONICAL")
required_runtime_capability = f"agent.runtime.{adapter}"
if required_runtime_capability not in capability_scope:
_error("AGENT_RUNTIME_CAPABILITY_REQUIRED")
if envelope.get("source_signals") not in (None, []):
_error("AGENT_RUNTIME_SOURCE_SIGNALS_DENY")
elif not changes:
_error("CHANGE_SET_INVALID")
declared = str(envelope.get("execution_digest") or "")
material = {key: item for key, item in envelope.items() if key != "execution_digest"}
actual = _digest(material)
if declared != actual:
_error("EXECUTION_DIGEST_INVALID")
return {**material, "execution_digest": actual}
def project_execution_envelope(
binding: GovernedExecutionBinding,
execution_request: Mapping[str, Any] | None = None,
*,
required_capabilities: Sequence[str] = (),
) -> dict[str, Any]:
request = dict(execution_request or {})
action = str(request.get("action") or "source_capture")
if action not in SUPPORTED_ACTIONS:
_error("EXECUTION_ACTION_UNSUPPORTED")
capabilities = _capabilities(request.get("capability_scope", required_capabilities))
governance = {
"task_id": binding.task_id,
"job_id": binding.job_id,
"idempotency_key": binding.idempotency_key,
"commit_idempotency_key": str(request.get("commit_idempotency_key") or f"{binding.task_id}-commit-g{binding.generation}"),
"writer_lease": action in MUTATION_ACTIONS,
"main_write": False,
"production_write": False,
"tag_write": False,
"baseline_write": False,
"force_push": False,
"protected_ref_bypass": False,
}
supplied_governance = request.get("governance")
if supplied_governance is not None:
governance.update(_object(supplied_governance, "GOVERNANCE_POLICY_REQUIRED"))
material: dict[str, Any] = {
"schema": EXECUTION_SCHEMA,
**binding.public(),
"action": action,
"adapter": str(
request.get("adapter")
or ("deterministic" if action == "deterministic" else "source_capture")
),
"capability_scope": list(capabilities),
"change_set": list(request.get("change_set") or []),
"verification": list(request.get("verification") or []),
"source_signals": list(_source_signals(request.get("source_signals") or [])) if action == "source_capture" else [],
"runtime_input": _agent_runtime_input(request.get("runtime_input")) if action == "agent_runtime" else {},
"governance": governance,
"workspace": {
"authority_git_dir": "/data/aios-gitops/repo.git",
"read_only_repository": True,
"cleanup_policy": "always",
},
}
envelope = {**material, "execution_digest": _digest(material)}
return validate_execution_envelope(envelope, binding=binding)
def project_online_task(
binding: GovernedExecutionBinding,
task_request: Mapping[str, Any],
execution_request: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
request = dict(task_request)
required = _capabilities(request.get("required_capabilities"))
execution_value = dict(execution_request or {})
if str(execution_value.get("action") or "source_capture") == "agent_runtime":
adapter = str(execution_value.get("adapter") or "")
if adapter not in AGENT_RUNTIME_ADAPTERS:
_error("AGENT_RUNTIME_ADAPTER_INVALID")
runtime_input = execution_value.get("runtime_input")
tenant_id = request.get("tenant_id")
project_id = request.get("project_id")
declared_scope = str(
request.get("scope_type")
or ("tenant" if isinstance(tenant_id, str) and tenant_id.strip() else "legacy")
)
if runtime_input is None:
runtime_input = {
"goal": request.get("goal"),
"instructions": request.get("instructions") or "",
"success_criteria": request.get("success_criteria") or [],
"constraints": request.get("constraints") or {},
}
elif not isinstance(runtime_input, Mapping):
_error("AGENT_RUNTIME_INPUT_REQUIRED")
runtime_input = dict(runtime_input)
# Scope/trace are authoritative Task identity, never caller-selected runtime hints.
runtime_input["scope"] = {
"scope_type": declared_scope,
"tenant_id": tenant_id,
"project_id": project_id,
}
runtime_input["trace_id"] = str(
request.get("trace_id")
or request.get("run_id")
or binding.task_id
)
execution_value["runtime_input"] = runtime_input
execution = project_execution_envelope(
binding,
execution_value,
required_capabilities=required,
)
metadata = dict(request.get("metadata") or {})
if "execution" in metadata:
validate_execution_envelope(_object(metadata["execution"], "EXECUTION_ENVELOPE_INVALID"), binding=binding)
if metadata["execution"] != execution:
_error("EXECUTION_PROJECTION_CONFLICT")
metadata["execution"] = execution
metadata["execution_projection"] = {
"schema": PROJECTION_SCHEMA,
"task_id": binding.task_id,
"job_id": binding.job_id,
"execution_digest": execution["execution_digest"],
}
return {
**request,
"task_id": binding.task_id,
"metadata": metadata,
"required_capabilities": list(required),
"idempotency_key": str(request.get("idempotency_key") or f"online-task:{binding.idempotency_key}"),
}
def resolve_task_execution(
task: Mapping[str, Any],
binding: GovernedExecutionBinding,
*,
worker_capabilities: Sequence[str] | None = None,
) -> tuple[dict[str, Any], str]:
metadata = task.get("metadata")
if isinstance(metadata, Mapping) and "execution" in metadata:
value = validate_execution_envelope(
_object(metadata["execution"], "EXECUTION_ENVELOPE_INVALID"),
binding=binding,
worker_capabilities=worker_capabilities,
)
return value, "persisted"
projected = project_execution_envelope(
binding,
{"action": "source_capture", "capability_scope": task.get("required_capabilities") or []},
required_capabilities=task.get("required_capabilities") or [],
)
validate_execution_envelope(projected, binding=binding, worker_capabilities=worker_capabilities)
return projected, "recovered_projection"
@@ -1,45 +0,0 @@
"""AIOS-managed OpenClaw Ed25519 execution adapter."""
from .adapter import (
AdapterError,
CommandConflict,
CommandNotFound,
CommandRejected,
InMemoryCallbackSink,
OpenClawAdapter,
)
from .contract import (
CONTRACT_VERSION,
SIGNATURE_ALGORITHM,
CommandState,
Ed25519CommandSigner,
Ed25519CommandVerifier,
OpenClawCommand,
OpenClawResult,
RuntimeHealth,
sign_command,
verify_command_signature,
)
from .transport import HTTPRuntimeTransport, InProcessRuntimeTransport, RuntimeTransport
__all__ = [
"AdapterError",
"CONTRACT_VERSION",
"CommandConflict",
"CommandNotFound",
"CommandRejected",
"CommandState",
"Ed25519CommandSigner",
"Ed25519CommandVerifier",
"HTTPRuntimeTransport",
"InMemoryCallbackSink",
"InProcessRuntimeTransport",
"OpenClawAdapter",
"OpenClawCommand",
"OpenClawResult",
"RuntimeHealth",
"RuntimeTransport",
"SIGNATURE_ALGORITHM",
"sign_command",
"verify_command_signature",
]
-228
View File
@@ -1,228 +0,0 @@
"""Signed, tenant-isolated execution adapter for OpenClaw runtimes."""
from __future__ import annotations
import threading
from dataclasses import replace
from typing import Callable, Mapping, Protocol
from .contract import (
CommandState,
OpenClawCommand,
OpenClawResult,
RuntimeHealth,
utc_now,
verify_command_signature,
)
class AdapterError(RuntimeError):
"""Base adapter failure."""
class CommandRejected(AdapterError):
"""A command failed signature or contract validation."""
class CommandNotFound(AdapterError):
"""A tenant-scoped command is not visible."""
class CommandConflict(AdapterError):
"""A command or idempotency key conflicts with an existing execution."""
class RuntimeTransport(Protocol):
def execute(self, command: OpenClawCommand, timeout_seconds: float) -> OpenClawResult: ...
def cancel(self, command_id: str) -> bool: ...
def health(self) -> RuntimeHealth: ...
class CallbackSink(Protocol):
def on_started(self, result: OpenClawResult) -> None: ...
def on_completed(self, result: OpenClawResult) -> None: ...
def on_failed(self, result: OpenClawResult) -> None: ...
class InMemoryCallbackSink:
def __init__(self) -> None:
self._lock = threading.RLock()
self.started: list[OpenClawResult] = []
self.completed: list[OpenClawResult] = []
self.failed: list[OpenClawResult] = []
def on_started(self, result: OpenClawResult) -> None:
with self._lock:
self.started.append(result)
def on_completed(self, result: OpenClawResult) -> None:
with self._lock:
self.completed.append(result)
def on_failed(self, result: OpenClawResult) -> None:
with self._lock:
self.failed.append(result)
SigningMaterialResolver = Callable[[str, str], bytes | str]
class OpenClawAdapter:
"""Verify, de-duplicate, execute, cancel and reconcile commands."""
def __init__(
self,
transport: RuntimeTransport,
signing_material_resolver: SigningMaterialResolver,
*,
callback_sink: CallbackSink | None = None,
default_timeout_seconds: float = 300.0,
) -> None:
if default_timeout_seconds <= 0:
raise ValueError("default_timeout_seconds must be positive")
self.transport = transport
self.signing_material_resolver = signing_material_resolver
self.callback_sink = callback_sink or InMemoryCallbackSink()
self.default_timeout_seconds = float(default_timeout_seconds)
self._lock = threading.RLock()
self._commands: dict[str, OpenClawCommand] = {}
self._results: dict[str, OpenClawResult] = {}
self._idempotency: dict[tuple[str, str], tuple[str, str]] = {}
def _verify(self, command: OpenClawCommand) -> None:
try:
key_material = self.signing_material_resolver(command.tenant_id, command.key_id)
except Exception as exc:
raise CommandRejected("signing material unavailable") from exc
if not verify_command_signature(command, key_material):
raise CommandRejected("invalid or expired command signature")
@staticmethod
def _validate_result(command: OpenClawCommand, result: OpenClawResult) -> None:
if (
result.command_id != command.command_id
or result.tenant_id != command.tenant_id
or result.task_id != command.task_id
or result.trace_id != command.trace_id
):
raise AdapterError("runtime returned mismatched command identity")
if result.state not in {
CommandState.COMPLETED,
CommandState.FAILED,
CommandState.CANCELLED,
CommandState.TIMED_OUT,
}:
raise AdapterError("runtime returned a non-terminal result")
def execute(
self,
command: OpenClawCommand,
*,
timeout_seconds: float | None = None,
) -> OpenClawResult:
timeout = self.default_timeout_seconds if timeout_seconds is None else float(timeout_seconds)
if timeout <= 0:
raise ValueError("timeout_seconds must be positive")
self._verify(command)
digest = command.signature
idem_ref = (command.tenant_id, command.idempotency_key)
with self._lock:
prior_command = self._commands.get(command.command_id)
if prior_command is not None:
if prior_command.signature != command.signature:
raise CommandConflict("command_id reused with different content")
return self._results[command.command_id]
prior_idem = self._idempotency.get(idem_ref)
if prior_idem is not None:
prior_id, prior_digest = prior_idem
if prior_digest != digest:
raise CommandConflict("idempotency key reused with different content")
return self._results[prior_id]
running = OpenClawResult(
command_id=command.command_id,
tenant_id=command.tenant_id,
task_id=command.task_id,
trace_id=command.trace_id,
state=CommandState.RUNNING,
started_at=utc_now(),
)
self._commands[command.command_id] = command
self._results[command.command_id] = running
self._idempotency[idem_ref] = (command.command_id, digest)
self.callback_sink.on_started(running)
try:
result = self.transport.execute(command, timeout)
self._validate_result(command, result)
except TimeoutError as exc:
result = replace(
running,
state=CommandState.TIMED_OUT,
error={"code": "runtime_timeout", "message": str(exc) or "runtime timeout"},
finished_at=utc_now(),
)
except Exception as exc:
result = replace(
running,
state=CommandState.FAILED,
error={"code": "runtime_error", "message": str(exc)},
finished_at=utc_now(),
)
if result.started_at is None:
result = replace(result, started_at=running.started_at)
if result.finished_at is None:
result = replace(result, finished_at=utc_now())
with self._lock:
self._results[command.command_id] = result
if result.state == CommandState.COMPLETED:
self.callback_sink.on_completed(result)
else:
self.callback_sink.on_failed(result)
return result
def status(self, command_id: str, *, tenant_id: str) -> OpenClawResult:
with self._lock:
command = self._commands.get(command_id)
result = self._results.get(command_id)
if command is None or result is None or command.tenant_id != tenant_id:
raise CommandNotFound("command not found")
return result
def cancel(self, command_id: str, *, tenant_id: str) -> OpenClawResult:
with self._lock:
command = self._commands.get(command_id)
current = self._results.get(command_id)
if command is None or current is None or command.tenant_id != tenant_id:
raise CommandNotFound("command not found")
if current.state in {
CommandState.COMPLETED,
CommandState.FAILED,
CommandState.CANCELLED,
CommandState.TIMED_OUT,
CommandState.REJECTED,
}:
return current
if not self.transport.cancel(command_id):
raise AdapterError("runtime rejected cancellation")
cancelled = replace(
current,
state=CommandState.CANCELLED,
error={"code": "cancelled", "message": "cancelled by AIOS"},
finished_at=utc_now(),
)
with self._lock:
self._results[command_id] = cancelled
self.callback_sink.on_failed(cancelled)
return cancelled
def health(self) -> RuntimeHealth:
return self.transport.health()
def snapshot(self, *, tenant_id: str) -> Mapping[str, OpenClawResult]:
with self._lock:
return {
command_id: result
for command_id, result in self._results.items()
if self._commands[command_id].tenant_id == tenant_id
}
@@ -1,97 +0,0 @@
"""Audit-oriented lifecycle callbacks for OpenClaw execution."""
from __future__ import annotations
import json
import threading
from dataclasses import dataclass, field
from typing import Any, Mapping, Protocol
from .contract import OpenClawResult, utc_now
class EventPublisher(Protocol):
def publish(self, event: Mapping[str, Any]) -> None: ...
@dataclass(frozen=True)
class ExecutionEvent:
event_type: str
tenant_id: str
task_id: str
trace_id: str
command_id: str
state: str
created_at: str = field(default_factory=utc_now)
details: Mapping[str, Any] = field(default_factory=dict)
def as_dict(self) -> dict[str, Any]:
return {
"event_type": self.event_type,
"tenant_id": self.tenant_id,
"task_id": self.task_id,
"trace_id": self.trace_id,
"command_id": self.command_id,
"state": self.state,
"created_at": self.created_at,
"details": json.loads(
json.dumps(dict(self.details), ensure_ascii=False, default=str)
),
}
class InMemoryEventPublisher:
"""Thread-safe publisher used by tests and first integration slice."""
def __init__(self) -> None:
self._lock = threading.RLock()
self.events: list[dict[str, Any]] = []
def publish(self, event: Mapping[str, Any]) -> None:
with self._lock:
self.events.append(
json.loads(json.dumps(dict(event), ensure_ascii=False, default=str))
)
class AuditCallbackSink:
"""Convert runtime callbacks into tenant-scoped audit events."""
def __init__(self, publisher: EventPublisher) -> None:
self.publisher = publisher
@staticmethod
def _event(
event_type: str,
result: OpenClawResult,
) -> ExecutionEvent:
details: dict[str, Any] = {
"runtime_id": result.runtime_id,
"attempts": result.attempts,
}
if result.error:
details["error"] = dict(result.error)
if result.output:
details["output_present"] = True
return ExecutionEvent(
event_type=event_type,
tenant_id=result.tenant_id,
task_id=result.task_id,
trace_id=result.trace_id,
command_id=result.command_id,
state=result.state.value,
details=details,
)
def on_started(self, result: OpenClawResult) -> None:
self.publisher.publish(self._event("OPENCLAW_STARTED", result).as_dict())
def on_completed(self, result: OpenClawResult) -> None:
self.publisher.publish(self._event("OPENCLAW_COMPLETED", result).as_dict())
def on_failed(self, result: OpenClawResult) -> None:
event_type = {
"CANCELLED": "OPENCLAW_CANCELLED",
"TIMED_OUT": "OPENCLAW_TIMED_OUT",
"REJECTED": "OPENCLAW_REJECTED",
}.get(result.state.value, "OPENCLAW_FAILED")
self.publisher.publish(self._event(event_type, result).as_dict())
@@ -1,383 +0,0 @@
"""AIOS 到 OpenClaw 的 Ed25519 执行合同。"""
from __future__ import annotations
import base64
import hashlib
import hmac
import json
import uuid
from dataclasses import dataclass, field, replace
from datetime import datetime, timezone
from enum import Enum
from typing import Any, Mapping, Protocol
CONTRACT_VERSION = "aios.openclaw.command.v2"
SIGNATURE_ALGORITHM = "Ed25519"
DEFAULT_CLOCK_SKEW_SECONDS = 300
def utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _parse_timestamp(value: str) -> datetime:
normalized = str(value or "").strip()
if not normalized:
raise ValueError("timestamp is required")
if normalized.endswith("Z"):
normalized = normalized[:-1] + "+00:00"
parsed = datetime.fromisoformat(normalized)
if parsed.tzinfo is None:
raise ValueError("timestamp must include a timezone")
return parsed.astimezone(timezone.utc)
def _required(value: str, label: str) -> str:
normalized = str(value or "").strip()
if not normalized:
raise ValueError(f"{label} is required")
return normalized
def _safe_mapping(value: Mapping[str, Any] | None) -> dict[str, Any]:
return json.loads(
json.dumps(dict(value or {}), ensure_ascii=False, sort_keys=True, default=str)
)
def _load_crypto() -> tuple[Any, Any, Any]:
try:
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import (
Ed25519PrivateKey,
Ed25519PublicKey,
)
except ImportError as exc: # pragma: no cover
raise RuntimeError("cryptography with Ed25519 support is required") from exc
return serialization, Ed25519PrivateKey, Ed25519PublicKey
class CommandSigner(Protocol):
key_id: str
fingerprint: str
def sign(self, payload: bytes) -> bytes: ...
class CommandVerifier(Protocol):
key_id: str
fingerprint: str
def verify(self, payload: bytes, signature: bytes) -> bool: ...
class Ed25519CommandSigner:
def __init__(self, private_key: Any, *, key_id: str) -> None:
serialization, private_cls, _ = _load_crypto()
if not isinstance(private_key, private_cls):
raise ValueError("private key is not Ed25519")
self.private_key = private_key
self.key_id = _required(key_id, "key_id")
raw = private_key.public_key().public_bytes(
encoding=serialization.Encoding.Raw,
format=serialization.PublicFormat.Raw,
)
self.fingerprint = hashlib.sha256(raw).hexdigest()
@classmethod
def from_bytes(cls, value: bytes | str, *, key_id: str) -> "Ed25519CommandSigner":
serialization, private_cls, _ = _load_crypto()
raw = value.encode("utf-8") if isinstance(value, str) else bytes(value)
if raw.startswith(b"-----BEGIN"):
key = serialization.load_pem_private_key(raw, password=None)
elif len(raw) == 32:
key = private_cls.from_private_bytes(raw)
else:
raise ValueError("Ed25519 private key must be PEM or a raw 32-byte seed")
return cls(key, key_id=key_id)
def sign(self, payload: bytes) -> bytes:
return self.private_key.sign(bytes(payload))
def verifier(self) -> "Ed25519CommandVerifier":
return Ed25519CommandVerifier(self.private_key.public_key(), key_id=self.key_id)
class Ed25519CommandVerifier:
def __init__(self, public_key: Any, *, key_id: str) -> None:
serialization, _, public_cls = _load_crypto()
if not isinstance(public_key, public_cls):
raise ValueError("public key is not Ed25519")
self.public_key = public_key
self.key_id = _required(key_id, "key_id")
raw = public_key.public_bytes(
encoding=serialization.Encoding.Raw,
format=serialization.PublicFormat.Raw,
)
self.fingerprint = hashlib.sha256(raw).hexdigest()
@classmethod
def from_bytes(cls, value: bytes | str, *, key_id: str) -> "Ed25519CommandVerifier":
serialization, private_cls, public_cls = _load_crypto()
raw = value.encode("utf-8") if isinstance(value, str) else bytes(value)
if raw.startswith(b"-----BEGIN"):
try:
key = serialization.load_pem_public_key(raw)
except ValueError:
private = serialization.load_pem_private_key(raw, password=None)
if not isinstance(private, private_cls):
raise ValueError("key is not Ed25519")
key = private.public_key()
elif len(raw) == 32:
key = public_cls.from_public_bytes(raw)
else:
raise ValueError("Ed25519 public key must be PEM or raw 32-byte key")
return cls(key, key_id=key_id)
def verify(self, payload: bytes, signature: bytes) -> bool:
try:
self.public_key.verify(bytes(signature), bytes(payload))
return True
except Exception:
return False
class CommandState(str, Enum):
ACCEPTED = "ACCEPTED"
RUNNING = "RUNNING"
COMPLETED = "COMPLETED"
FAILED = "FAILED"
CANCELLED = "CANCELLED"
TIMED_OUT = "TIMED_OUT"
REJECTED = "REJECTED"
@dataclass(frozen=True)
class OpenClawCommand:
command_id: str
tenant_id: str
task_id: str
trace_id: str
action: str
payload: Mapping[str, Any] = field(default_factory=dict)
idempotency_key: str = ""
created_at: str = field(default_factory=utc_now)
expires_at: str | None = None
contract_version: str = CONTRACT_VERSION
signature_algorithm: str = SIGNATURE_ALGORITHM
key_id: str = "default"
public_key_fingerprint: str = ""
signature: str = ""
def __post_init__(self) -> None:
for field_name in (
"command_id",
"tenant_id",
"task_id",
"trace_id",
"action",
"idempotency_key",
"created_at",
"contract_version",
"signature_algorithm",
"key_id",
):
object.__setattr__(
self, field_name, _required(getattr(self, field_name), field_name)
)
if self.contract_version != CONTRACT_VERSION:
raise ValueError(f"unsupported contract_version: {self.contract_version}")
if self.signature_algorithm != SIGNATURE_ALGORITHM:
raise ValueError(
f"unsupported signature_algorithm: {self.signature_algorithm}"
)
object.__setattr__(self, "payload", _safe_mapping(self.payload))
object.__setattr__(
self, "public_key_fingerprint", str(self.public_key_fingerprint or "").strip()
)
object.__setattr__(self, "signature", str(self.signature or "").strip())
_parse_timestamp(self.created_at)
if self.expires_at is not None:
object.__setattr__(self, "expires_at", _required(self.expires_at, "expires_at"))
if _parse_timestamp(self.expires_at) <= _parse_timestamp(self.created_at):
raise ValueError("expires_at must be later than created_at")
@classmethod
def create(
cls,
*,
tenant_id: str,
task_id: str,
trace_id: str,
action: str,
payload: Mapping[str, Any] | None,
idempotency_key: str,
expires_at: str | None = None,
key_id: str = "default",
command_id: str | None = None,
) -> "OpenClawCommand":
return cls(
command_id=command_id or f"cmd_{uuid.uuid4().hex}",
tenant_id=tenant_id,
task_id=task_id,
trace_id=trace_id,
action=action,
payload=_safe_mapping(payload),
idempotency_key=idempotency_key,
expires_at=expires_at,
key_id=key_id,
)
def unsigned_dict(self) -> dict[str, Any]:
return {
"command_id": self.command_id,
"tenant_id": self.tenant_id,
"task_id": self.task_id,
"trace_id": self.trace_id,
"action": self.action,
"payload": _safe_mapping(self.payload),
"idempotency_key": self.idempotency_key,
"created_at": self.created_at,
"expires_at": self.expires_at,
"contract_version": self.contract_version,
"signature_algorithm": self.signature_algorithm,
"key_id": self.key_id,
"public_key_fingerprint": self.public_key_fingerprint,
}
@dataclass(frozen=True)
class OpenClawResult:
command_id: str
tenant_id: str
task_id: str
trace_id: str
state: CommandState
output: Mapping[str, Any] | None = None
error: Mapping[str, Any] | None = None
started_at: str | None = None
finished_at: str | None = None
runtime_id: str | None = None
attempts: int = 1
def __post_init__(self) -> None:
for field_name in ("command_id", "tenant_id", "task_id", "trace_id"):
object.__setattr__(
self, field_name, _required(getattr(self, field_name), field_name)
)
object.__setattr__(self, "state", CommandState(self.state))
if self.output is not None:
object.__setattr__(self, "output", _safe_mapping(self.output))
if self.error is not None:
object.__setattr__(self, "error", _safe_mapping(self.error))
if self.started_at is not None:
_parse_timestamp(self.started_at)
if self.finished_at is not None:
_parse_timestamp(self.finished_at)
if self.attempts < 1:
raise ValueError("attempts must be >= 1")
@dataclass(frozen=True)
class RuntimeHealth:
runtime: str
runtime_id: str
status: str
version: str
heartbeat_at: str
active_tasks: int = 0
workspace_isolated: bool = False
credential_ref_configured: bool = False
details: Mapping[str, Any] = field(default_factory=dict)
def __post_init__(self) -> None:
for field_name in (
"runtime",
"runtime_id",
"status",
"version",
"heartbeat_at",
):
object.__setattr__(
self, field_name, _required(getattr(self, field_name), field_name)
)
_parse_timestamp(self.heartbeat_at)
if self.active_tasks < 0:
raise ValueError("active_tasks must be >= 0")
object.__setattr__(self, "details", _safe_mapping(self.details))
def canonical_command_bytes(command: OpenClawCommand) -> bytes:
return json.dumps(
command.unsigned_dict(),
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
def _as_signer(command: OpenClawCommand, value: Any) -> CommandSigner:
if hasattr(value, "sign") and hasattr(value, "fingerprint") and hasattr(value, "key_id"):
return value
return Ed25519CommandSigner.from_bytes(value, key_id=command.key_id)
def _as_verifier(command: OpenClawCommand, value: Any) -> CommandVerifier:
if isinstance(value, Ed25519CommandSigner):
return value.verifier()
if hasattr(value, "verify") and hasattr(value, "fingerprint") and hasattr(value, "key_id"):
return value
return Ed25519CommandVerifier.from_bytes(value, key_id=command.key_id)
def sign_command(command: OpenClawCommand, signing_key: Any) -> OpenClawCommand:
signer = _as_signer(command, signing_key)
if signer.key_id != command.key_id:
raise ValueError("command key_id does not match signer")
prepared = replace(
command,
signature_algorithm=SIGNATURE_ALGORITHM,
public_key_fingerprint=signer.fingerprint,
signature="",
)
signature = base64.b64encode(
signer.sign(canonical_command_bytes(prepared))
).decode("ascii")
return replace(prepared, signature=signature)
def verify_command_signature(
command: OpenClawCommand,
verification_key: Any,
*,
now: datetime | None = None,
clock_skew_seconds: int = DEFAULT_CLOCK_SKEW_SECONDS,
) -> bool:
if (
not command.signature
or command.contract_version != CONTRACT_VERSION
or command.signature_algorithm != SIGNATURE_ALGORITHM
):
return False
try:
verifier = _as_verifier(command, verification_key)
signature = base64.b64decode(command.signature, validate=True)
except Exception:
return False
if verifier.key_id != command.key_id:
return False
if not command.public_key_fingerprint or not hmac.compare_digest(
command.public_key_fingerprint, verifier.fingerprint
):
return False
if not verifier.verify(canonical_command_bytes(command), signature):
return False
current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
created = _parse_timestamp(command.created_at)
if created.timestamp() - current.timestamp() > clock_skew_seconds:
return False
if command.expires_at is not None:
expires = _parse_timestamp(command.expires_at)
if current.timestamp() - expires.timestamp() > clock_skew_seconds:
return False
return True
-103
View File
@@ -1,103 +0,0 @@
"""Health policy and self-healing decisions for OpenClaw runtimes."""
from __future__ import annotations
from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Mapping
from .contract import RuntimeHealth
def _parse(value: str) -> datetime:
normalized = value[:-1] + "+00:00" if value.endswith("Z") else value
parsed = datetime.fromisoformat(normalized)
if parsed.tzinfo is None:
raise ValueError("heartbeat timestamp must include timezone")
return parsed.astimezone(timezone.utc)
@dataclass(frozen=True)
class HealthPolicy:
max_heartbeat_age_seconds: int = 90
require_workspace_isolation: bool = True
require_credential_ref: bool = True
accepted_statuses: frozenset[str] = field(
default_factory=lambda: frozenset({"healthy", "ready"})
)
def __post_init__(self) -> None:
if self.max_heartbeat_age_seconds < 5:
raise ValueError("max_heartbeat_age_seconds must be >= 5")
if not self.accepted_statuses:
raise ValueError("accepted_statuses must not be empty")
@dataclass(frozen=True)
class HealthAssessment:
healthy: bool
reasons: tuple[str, ...]
recoverable: bool
recommended_action: str
details: Mapping[str, object] = field(default_factory=dict)
class RuntimeHealthMonitor:
"""Evaluate runtime health and recommend bounded recovery actions."""
def __init__(self, policy: HealthPolicy | None = None) -> None:
self.policy = policy or HealthPolicy()
def assess(
self,
health: RuntimeHealth,
*,
now: datetime | None = None,
) -> HealthAssessment:
current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
heartbeat = _parse(health.heartbeat_at)
age = max(0.0, current.timestamp() - heartbeat.timestamp())
reasons: list[str] = []
if health.status.lower() not in self.policy.accepted_statuses:
reasons.append("runtime_status_unhealthy")
if age > self.policy.max_heartbeat_age_seconds:
reasons.append("heartbeat_stale")
if self.policy.require_workspace_isolation and not health.workspace_isolated:
reasons.append("workspace_not_isolated")
if self.policy.require_credential_ref and not health.credential_ref_configured:
reasons.append("credential_reference_missing")
if not reasons:
return HealthAssessment(
healthy=True,
reasons=(),
recoverable=True,
recommended_action="none",
details={"heartbeat_age_seconds": age},
)
non_recoverable = {
"workspace_not_isolated",
"credential_reference_missing",
}.intersection(reasons)
if non_recoverable:
action = "quarantine_runtime"
recoverable = False
elif "heartbeat_stale" in reasons:
action = "restart_runtime"
recoverable = True
else:
action = "drain_and_restart"
recoverable = True
return HealthAssessment(
healthy=False,
reasons=tuple(reasons),
recoverable=recoverable,
recommended_action=action,
details={
"heartbeat_age_seconds": age,
"runtime_id": health.runtime_id,
"active_tasks": health.active_tasks,
},
)
@@ -1,214 +0,0 @@
"""Runtime transport implementations for the OpenClaw adapter."""
from __future__ import annotations
import json
import threading
import urllib.error
import urllib.parse
import urllib.request
from typing import Any, Callable, Mapping, Protocol
from .contract import (
CommandState,
OpenClawCommand,
OpenClawResult,
RuntimeHealth,
utc_now,
)
class RuntimeTransport(Protocol):
def execute(self, command: OpenClawCommand, timeout_seconds: float) -> OpenClawResult: ...
def cancel(self, command_id: str) -> bool: ...
def health(self) -> RuntimeHealth: ...
class TransportError(RuntimeError):
"""Runtime transport or protocol failure."""
RuntimeHandler = Callable[[OpenClawCommand], Mapping[str, Any] | OpenClawResult]
class InProcessRuntimeTransport:
"""Deterministic runtime used by tests and controlled local integrations."""
def __init__(
self,
handler: RuntimeHandler,
*,
runtime_id: str = "openclaw-in-process",
version: str = "v1",
workspace_isolated: bool = True,
credential_ref_configured: bool = True,
) -> None:
self.handler = handler
self.runtime_id = runtime_id
self.version = version
self.workspace_isolated = workspace_isolated
self.credential_ref_configured = credential_ref_configured
self._lock = threading.RLock()
self._running: set[str] = set()
self._cancelled: set[str] = set()
def execute(self, command: OpenClawCommand, timeout_seconds: float) -> OpenClawResult:
if timeout_seconds <= 0:
raise ValueError("timeout_seconds must be positive")
with self._lock:
if command.command_id in self._cancelled:
return OpenClawResult(
command_id=command.command_id,
tenant_id=command.tenant_id,
task_id=command.task_id,
trace_id=command.trace_id,
state=CommandState.CANCELLED,
error={"code": "cancelled", "message": "cancelled before execution"},
finished_at=utc_now(),
runtime_id=self.runtime_id,
)
self._running.add(command.command_id)
try:
raw = self.handler(command)
if isinstance(raw, OpenClawResult):
return raw
if not isinstance(raw, Mapping):
raise TransportError("runtime handler must return a mapping or OpenClawResult")
return OpenClawResult(
command_id=command.command_id,
tenant_id=command.tenant_id,
task_id=command.task_id,
trace_id=command.trace_id,
state=CommandState(str(raw.get("state", "COMPLETED"))),
output=raw.get("output"),
error=raw.get("error"),
started_at=str(raw.get("started_at") or utc_now()),
finished_at=str(raw.get("finished_at") or utc_now()),
runtime_id=str(raw.get("runtime_id") or self.runtime_id),
attempts=int(raw.get("attempts", 1)),
)
finally:
with self._lock:
self._running.discard(command.command_id)
def cancel(self, command_id: str) -> bool:
with self._lock:
self._cancelled.add(command_id)
return command_id in self._running
def health(self) -> RuntimeHealth:
with self._lock:
active = len(self._running)
return RuntimeHealth(
runtime="openclaw",
runtime_id=self.runtime_id,
status="healthy",
version=self.version,
heartbeat_at=utc_now(),
active_tasks=active,
workspace_isolated=self.workspace_isolated,
credential_ref_configured=self.credential_ref_configured,
details={"transport": "in_process"},
)
class HTTPRuntimeTransport:
"""Private HTTP transport for an OpenClaw runtime service."""
def __init__(
self,
base_url: str,
*,
runtime_id: str = "openclaw-http",
credential_ref: str | None = None,
max_response_bytes: int = 2_097_152,
) -> None:
normalized = str(base_url).rstrip("/")
if not normalized.startswith(("http://", "https://")):
raise ValueError("base_url must use http or https")
self.base_url = normalized
self.runtime_id = runtime_id
self.credential_ref = credential_ref
self.max_response_bytes = int(max_response_bytes)
def _request(
self,
method: str,
path: str,
*,
payload: Mapping[str, Any] | None,
timeout_seconds: float,
) -> Mapping[str, Any]:
body = None
headers = {"Accept": "application/json", "User-Agent": "AIOS-OpenClaw/1"}
if payload is not None:
body = json.dumps(dict(payload), separators=(",", ":")).encode("utf-8")
headers["Content-Type"] = "application/json"
if self.credential_ref:
headers["X-AIOS-Credential-Ref"] = self.credential_ref
request = urllib.request.Request(
f"{self.base_url}{path}",
data=body,
method=method,
headers=headers,
)
try:
with urllib.request.urlopen(request, timeout=timeout_seconds) as response:
raw = response.read(self.max_response_bytes + 1)
except urllib.error.HTTPError as exc:
raise TransportError(f"runtime returned HTTP {exc.code}") from exc
except urllib.error.URLError as exc:
raise TransportError(f"runtime unavailable: {exc.reason}") from exc
if len(raw) > self.max_response_bytes:
raise TransportError("runtime response exceeded size limit")
decoded = json.loads(raw.decode("utf-8"))
if not isinstance(decoded, Mapping):
raise TransportError("runtime response must be a JSON object")
return decoded
@staticmethod
def _command_payload(command: OpenClawCommand) -> dict[str, Any]:
return {**command.unsigned_dict(), "signature": command.signature}
def execute(self, command: OpenClawCommand, timeout_seconds: float) -> OpenClawResult:
value = self._request(
"POST",
"/v1/execute",
payload=self._command_payload(command),
timeout_seconds=timeout_seconds,
)
return OpenClawResult(
command_id=str(value["command_id"]),
tenant_id=str(value["tenant_id"]),
task_id=str(value["task_id"]),
trace_id=str(value["trace_id"]),
state=CommandState(str(value["state"])),
output=value.get("output"),
error=value.get("error"),
started_at=value.get("started_at"),
finished_at=value.get("finished_at"),
runtime_id=str(value.get("runtime_id") or self.runtime_id),
attempts=int(value.get("attempts", 1)),
)
def cancel(self, command_id: str) -> bool:
value = self._request(
"POST",
f"/v1/commands/{urllib.parse.quote(command_id, safe='')}/cancel",
payload={},
timeout_seconds=10.0,
)
return bool(value.get("accepted"))
def health(self) -> RuntimeHealth:
value = self._request("GET", "/v1/health", payload=None, timeout_seconds=5.0)
return RuntimeHealth(
runtime=str(value.get("runtime") or "openclaw"),
runtime_id=str(value.get("runtime_id") or self.runtime_id),
status=str(value.get("status") or "unknown"),
version=str(value.get("version") or "unknown"),
heartbeat_at=str(value.get("heartbeat_at") or utc_now()),
active_tasks=int(value.get("active_tasks", 0)),
workspace_isolated=bool(value.get("workspace_isolated", False)),
credential_ref_configured=bool(value.get("credential_ref_configured", False)),
details=value.get("details") if isinstance(value.get("details"), Mapping) else {},
)
@@ -0,0 +1,144 @@
"""HTTP adapter for audited Owner authorization and writer provisioning.
This module layers on top of ``aios_authorized_task_writer_api`` so existing
routes and the legacy exact ``writer:provision`` route remain available. The
new endpoint does not mint or modify bearer credentials. It authenticates an
existing, non-wildcard, explicitly allow-listed Owner credential and then
invokes the audited broker.
Endpoint:
POST /api/governance/owner-authorizations
"""
from __future__ import annotations
import os
import urllib.parse
from typing import Any, Mapping
import aios_authorized_task_writer_api as bridge_api
from authorized_task_writer_provisioning import ProvisioningError
from owner_authorization_broker import (
OwnerAuthorizationBroker,
OwnerAuthorizationError,
OwnerPrincipal,
)
api = bridge_api.api
_original_do_post = api.H.do_POST
_ENDPOINT = "/api/governance/owner-authorizations"
def _enabled() -> bool:
return os.getenv(
"AIOS_GITOPS_OWNER_AUTHORIZATION_BROKER_ENABLED", ""
).strip().lower() in {"1", "true", "yes", "on"}
def _configured_intake_scope() -> str:
scopes = [
item.strip()
for item in os.getenv(
"AIOS_GITOPS_OWNER_AUTHORIZATION_INTAKE_SCOPES", ""
).split(",")
if item.strip()
]
if len(scopes) != 1:
raise OwnerAuthorizationError(
"EXACTLY_ONE_HTTP_INTAKE_SCOPE_REQUIRED", 503
)
scope = scopes[0]
if scope == "writer:provision" or scope == "*" or scope.endswith(":*"):
raise OwnerAuthorizationError("HTTP_INTAKE_SCOPE_REJECTED", 503)
return scope
def _scope_set(auth: Mapping[str, Any]) -> frozenset[str]:
raw = auth.get("scopes")
if raw is None:
raw = auth.get("scope")
if raw is None:
return frozenset()
if isinstance(raw, str):
candidates = raw.replace(",", " ").split()
elif isinstance(raw, (list, tuple, set, frozenset)):
candidates = [str(item) for item in raw]
else:
raise OwnerAuthorizationError("OWNER_SCOPE_EVIDENCE_INVALID", 403)
return frozenset(item.strip() for item in candidates if item.strip())
def _credential_id(auth: Mapping[str, Any]) -> str:
for name in (
"credential_id",
"token_id",
"key_id",
"auth_id",
):
value = str(auth.get(name) or "").strip()
if value:
return value
return ""
def _authenticate_existing_owner(handler) -> OwnerPrincipal | None:
scope = _configured_intake_scope()
auth = handler.need(scope)
if not auth:
return None
if auth.get("role") != "owner":
handler.sendj({"error": "OWNER_ROLE_REQUIRED"}, 403)
return None
identity = str(auth.get("account") or auth.get("identity") or "").strip()
if not identity:
handler.sendj({"error": "OWNER_IDENTITY_REQUIRED"}, 403)
return None
scopes = _scope_set(auth)
if any(item == "*" or item.endswith(":*") for item in scopes):
handler.sendj(
{"error": "WILDCARD_OWNER_CREDENTIAL_REJECTED"}, 403
)
return None
return OwnerPrincipal(
identity=identity,
role="owner",
authenticated_scope=scope,
token_scopes=scopes,
credential_id=_credential_id(auth),
)
def _broker_do_post(self):
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
if path != _ENDPOINT:
return _original_do_post(self)
if not _enabled():
return self.sendj(
{"error": "OWNER_AUTHORIZATION_BROKER_DISABLED"}, 503
)
try:
principal = _authenticate_existing_owner(self)
if principal is None:
return
data = self.body()
result = OwnerAuthorizationBroker().authorize_and_provision(
data, principal
)
return self.sendj(result)
except (OwnerAuthorizationError, ProvisioningError) as exc:
body = {"error": exc.code}
if exc.details:
body["details"] = exc.details
return self.sendj(body, exc.status)
except Exception:
# Never expose registry data, bearer tokens, signer output or traceback.
return self.sendj(
{"error": "OWNER_AUTHORIZATION_BROKER_INTERNAL_ERROR"}, 500
)
api.H.do_POST = _broker_do_post
if __name__ == "__main__":
api.main()
@@ -0,0 +1,148 @@
"""Fail-closed signed deterministic execution amendment for R7."""
from datetime import datetime, timezone
import hashlib, json, sqlite3
from pathlib import Path
from typing import Mapping
from aios_task_terminal_recovery import DeterministicPlan, RecoveryError, canonical_json
AMENDMENT_SCHEMA="aios.deterministic-execution-amendment.v1"
APPLY_SCHEMA="aios.deterministic-execution-amendment-apply.v1"
NAMESPACE="aios-owner-approval-v1"
EVENT="TASK_RECOVERY_EXECUTION_AMENDMENT_APPLIED"
APPLY_STATES=frozenset({"FAILED", "QUEUED"})
def _err(code): raise RecoveryError(code)
def _sha(v): return hashlib.sha256(canonical_json(v).encode()).hexdigest()
def amendment_body(x):
return {k:v for k,v in x.items() if k not in {"signature","amendment_id"}}
def amendment_id(x): return "sha256:"+_sha(amendment_body(x))
def signed_payload(x):
return canonical_json({"amendment_id":x["amendment_id"],**amendment_body(x)}).encode()
def parse_amendment(raw):
x=dict(raw)
required={"schema","amendment_id","nonce","task_id","job_id","generation",
"repository","branch","expected_branch_oid","recovery_epoch","decision_id",
"plan_id","plan_sha256","expected_apply_status","expected_apply_revision",
"expected_apply_attempt","expected_current_capabilities","envelope",
"owner_identity","key_id","issued_at","not_before","expires_at",
"one_time_use","signature"}
if set(x)<required or x.get("schema")!=AMENDMENT_SCHEMA: _err("AMENDMENT_SCHEMA_INVALID")
if x["expected_apply_status"] not in APPLY_STATES: _err("AMENDMENT_APPLY_STATUS_INVALID")
if x.get("one_time_use") is not True: _err("AMENDMENT_ONE_TIME_REQUIRED")
if not isinstance(x["envelope"],Mapping): _err("AMENDMENT_ENVELOPE_INVALID")
if not isinstance(x["expected_current_capabilities"],list): _err("AMENDMENT_CAPABILITIES_INVALID")
for k in ("generation","expected_apply_revision","expected_apply_attempt"):
if isinstance(x[k],bool) or not isinstance(x[k],int) or x[k]<0: _err("AMENDMENT_CAS_INVALID")
for k in ("issued_at","not_before","expires_at"):
try:d=datetime.fromisoformat(str(x[k]).replace("Z","+00:00"))
except ValueError as e: raise RecoveryError("AMENDMENT_TIME_INVALID") from e
if d.tzinfo is None:_err("AMENDMENT_TIME_INVALID")
x[k]=d.astimezone(timezone.utc).isoformat().replace("+00:00","Z")
x["expected_current_capabilities"]=sorted(set(x["expected_current_capabilities"]))
x["envelope"]=dict(x["envelope"])
return x
def _validate_envelope(env,plan,x):
try:
from aios_online_agent_execution import GovernedExecutionBinding,validate_execution_envelope
binding=GovernedExecutionBinding.from_mapping(env)
e=dict(validate_execution_envelope(env,binding=binding,worker_capabilities=plan.required_capabilities))
except Exception as z: raise RecoveryError("AMENDMENT_EXECUTION_ENVELOPE_INVALID") from z
if e.get("action")!="deterministic" or e.get("adapter")!="deterministic":_err("AMENDMENT_EXECUTION_ACTION_INVALID")
if tuple(sorted(e.get("capability_scope") or ()))!=plan.required_capabilities:_err("AMENDMENT_EXECUTION_CAPABILITIES_MISMATCH")
if not e.get("change_set") or not e.get("verification"):_err("AMENDMENT_EXECUTION_OPERATIONS_REQUIRED")
if e.get("runtime_input") not in (None,{}) or e.get("source_signals") not in (None,[]):_err("AMENDMENT_STALE_RUNTIME_FIELDS_FORBIDDEN")
for k,w in (("task_id",x["task_id"]),("job_id",x["job_id"]),("repository",x["repository"]),
("generation",x["generation"]),("target_branch",x["branch"]),
("expected_branch_oid",x["expected_branch_oid"]),("base_oid",x["expected_branch_oid"]),
("plan_id",x["plan_id"]),("plan_sha256",x["plan_sha256"]),("recovery_epoch",x["recovery_epoch"])):
if e.get(k)!=w:_err("AMENDMENT_EXECUTION_BINDING_MISMATCH")
g=e.get("governance") or {}
if g.get("writer_lease") is not True:_err("AMENDMENT_WRITER_LEASE_REQUIRED")
if any(g.get(k,False) for k in ("main_write","production_write","tag_write","baseline_write","force_push","protected_ref_bypass")):_err("AMENDMENT_PROTECTED_WRITE_FORBIDDEN")
return e
class DeterministicExecutionAmendmentController:
def __init__(self,db_path,verifier,*,clock=None,envelope_validator=None):
self.db_path=Path(db_path); self.verifier=verifier
self.clock=clock or (lambda:datetime.now(timezone.utc)); self._init()
def _db(self):
c=sqlite3.connect(self.db_path,timeout=10,isolation_level=None); c.row_factory=sqlite3.Row
c.execute("PRAGMA busy_timeout=10000"); return c
def _now(self): return self.clock().astimezone(timezone.utc).isoformat().replace("+00:00","Z")
def _init(self):
with self._db() as c:
c.execute("DROP TRIGGER IF EXISTS aios_r7_recovery_capability_projection")
c.executescript("""
CREATE TABLE IF NOT EXISTS deterministic_execution_amendments(
amendment_id TEXT PRIMARY KEY,nonce TEXT UNIQUE NOT NULL,task_id TEXT NOT NULL,
decision_id TEXT NOT NULL,plan_id TEXT NOT NULL,plan_sha256 TEXT NOT NULL,
envelope_digest TEXT NOT NULL,envelope_json TEXT NOT NULL,payload_json TEXT NOT NULL,
created_at TEXT NOT NULL,UNIQUE(task_id,decision_id));
DROP TRIGGER IF EXISTS aios_r7_recovery_requires_executable_amendment;
CREATE TRIGGER aios_r7_recovery_requires_executable_amendment
BEFORE UPDATE OF status ON tasks
WHEN OLD.status='FAILED' AND NEW.status='QUEUED' BEGIN
SELECT CASE WHEN NOT EXISTS(
SELECT 1 FROM deterministic_execution_amendments a
WHERE a.task_id=OLD.task_id
AND a.decision_id=json_extract(NEW.metadata_json,'$.recovery.decision_id')
AND a.amendment_id=json_extract(OLD.metadata_json,'$.recovery.execution_amendment_id')
AND a.envelope_digest=json_extract(OLD.metadata_json,'$.execution.execution_digest')
) THEN RAISE(ABORT,'SIGNED_EXECUTION_AMENDMENT_REQUIRED') END;
END;""")
def _plan(self,c,x):
r=c.execute("SELECT * FROM deterministic_plans WHERE plan_id=?",(x["plan_id"],)).fetchone()
if r is None or r["plan_sha256"]!=x["plan_sha256"]:_err("AMENDMENT_PLAN_NOT_FOUND_OR_MISMATCH")
p=DeterministicPlan.from_dict(json.loads(r["payload_json"]))
if (p.task_id,p.job_id,p.generation,p.repository,p.branch,p.expected_branch_oid,p.recovery_epoch)!=(x["task_id"],x["job_id"],x["generation"],x["repository"],x["branch"],x["expected_branch_oid"],x["recovery_epoch"]):_err("AMENDMENT_PLAN_BINDING_MISMATCH")
return p
def persist(self,raw):
x=parse_amendment(raw)
if x["amendment_id"]!=amendment_id(x):_err("AMENDMENT_ID_DIGEST_MISMATCH")
now=self._now()
if now<x["not_before"] or now>=x["expires_at"]:_err("AMENDMENT_EXPIRED_OR_NOT_YET_VALID")
if not self.verifier.verify(namespace=NAMESPACE,payload=signed_payload(x),signature=x["signature"],key_id=x["key_id"]):_err("AMENDMENT_SIGNATURE_INVALID")
with self._db() as c:
c.execute("BEGIN IMMEDIATE"); e=_validate_envelope(x["envelope"],self._plan(c,x),x)
ej=canonical_json(e); ed=e["execution_digest"]; stored=dict(x); stored["signature"]="<present>"
old=c.execute("SELECT * FROM deterministic_execution_amendments WHERE amendment_id=?",(x["amendment_id"],)).fetchone()
if old:
if old["envelope_json"]!=ej:_err("AMENDMENT_ID_REUSE_MISMATCH")
c.commit(); return {"status":"PRESENT","amendment_id":x["amendment_id"],"task_id":x["task_id"],"decision_id":x["decision_id"],"plan_id":x["plan_id"],"execution_digest":ed}
if c.execute("SELECT 1 FROM deterministic_execution_amendments WHERE nonce=? OR (task_id=? AND decision_id=?)",(x["nonce"],x["task_id"],x["decision_id"])).fetchone():_err("AMENDMENT_REPLAY_OR_DECISION_REUSE")
c.execute("INSERT INTO deterministic_execution_amendments VALUES(?,?,?,?,?,?,?,?,?,?)",(x["amendment_id"],x["nonce"],x["task_id"],x["decision_id"],x["plan_id"],x["plan_sha256"],ed,ej,canonical_json(stored),now)); c.commit()
return {"status":"PERSISTED","amendment_id":x["amendment_id"],"task_id":x["task_id"],"decision_id":x["decision_id"],"plan_id":x["plan_id"],"execution_digest":ed}
def apply(self,raw):
if raw.get("schema")!=APPLY_SCHEMA:_err("AMENDMENT_APPLY_SCHEMA_INVALID")
aid=raw.get("amendment_id"); tid=raw.get("task_id")
with self._db() as c:
c.execute("BEGIN IMMEDIATE"); a=c.execute("SELECT * FROM deterministic_execution_amendments WHERE amendment_id=? AND task_id=?",(aid,tid)).fetchone()
if a is None:_err("AMENDMENT_NOT_FOUND")
x=json.loads(a["payload_json"]); d=json.loads(a["envelope_json"])
t=c.execute("SELECT * FROM tasks WHERE task_id=?",(tid,)).fetchone()
if t is None:_err("TASK_NOT_FOUND")
m=json.loads(t["metadata_json"] or "{}"); cp=tuple(sorted(json.loads(t["required_capabilities_json"] or "[]"))); dc=tuple(sorted(d.get("capability_scope") or ()))
rec=m.get("recovery"); rec=dict(rec) if isinstance(rec,Mapping) else {}
old=m.get("execution")
if isinstance(old,Mapping) and old.get("execution_digest")==a["envelope_digest"] and cp==dc and rec.get("execution_amendment_id")==aid:
c.commit(); return {"status":"PRESENT","task_id":tid,"amendment_id":aid,"execution_digest":a["envelope_digest"],"revision":int(t["revision"]),"attempt":int(t["attempt"])}
if t["status"]!=x["expected_apply_status"] or int(t["revision"])!=x["expected_apply_revision"] or int(t["attempt"])!=x["expected_apply_attempt"] or cp!=tuple(x["expected_current_capabilities"]) or t["claimed_by"] is not None or t["lease_token_hash"] is not None:_err("AMENDMENT_APPLY_TASK_CAS_MISMATCH")
if t["status"]=="QUEUED" and rec.get("decision_id")!=x["decision_id"]:_err("AMENDMENT_DECISION_BINDING_MISMATCH")
m["execution"]=d; m["execution_projection"]={"schema":"aios.online-agent.task-projection.v1","task_id":tid,"job_id":d.get("job_id"),"execution_digest":a["envelope_digest"]}
rec["execution_amendment_id"]=aid; m["recovery"]=rec; now=self._now()
q=c.execute("""UPDATE tasks SET required_capabilities_json=?,metadata_json=?,revision=revision+1,updated_at=? WHERE task_id=? AND status=? AND revision=? AND attempt=? AND claimed_by IS NULL AND lease_token_hash IS NULL""",(canonical_json(list(dc)),canonical_json(m),now,tid,x["expected_apply_status"],x["expected_apply_revision"],x["expected_apply_attempt"]))
if q.rowcount!=1:_err("AMENDMENT_APPLY_TASK_CAS_FAILED")
c.execute("INSERT INTO events(task_id,agent_id,event_type,payload_json,created_at) VALUES(?,?,?,?,?)",(tid,None,EVENT,canonical_json({"amendment_id":aid,"decision_id":x["decision_id"],"plan_id":x["plan_id"],"execution_digest":a["envelope_digest"],"required_capabilities":list(dc)}),now)); c.commit()
return {"status":"APPLIED","task_id":tid,"amendment_id":aid,"decision_id":x["decision_id"],"plan_id":x["plan_id"],"execution_digest":a["envelope_digest"],"required_capabilities":list(dc),"revision":x["expected_apply_revision"]+1,"attempt":x["expected_apply_attempt"]}
+160
View File
@@ -0,0 +1,160 @@
#!/usr/bin/env python3
"""Loopback-only HTTP API for the task identity overlay.
Endpoints:
GET /healthz
POST /api/governance/owner-authorizations
The API authenticates only the short-lived overlay credential. It does not read
or modify the global bearer-token registry.
"""
from __future__ import annotations
import json
import os
import urllib.parse
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from typing import Any
from authorized_task_writer_provisioning import ProvisioningError
from owner_authorization_broker import OwnerAuthorizationError
from task_identity_overlay import (
MAX_BODY_BYTES,
IdentityOverlayConfig,
IdentityOverlayError,
TaskIdentityOverlayService,
)
_ENDPOINT = "/api/governance/owner-authorizations"
def _enabled() -> bool:
return os.getenv(
"AIOS_GITOPS_IDENTITY_OVERLAY_ENABLED", ""
).strip().lower() in {"1", "true", "yes", "on"}
def _bearer(header: str) -> str:
prefix = "Bearer "
if not header.startswith(prefix):
raise IdentityOverlayError("OVERLAY_AUTHENTICATION_REQUIRED", 401)
token = header[len(prefix):].strip()
if not token or len(token) > 4096:
raise IdentityOverlayError("OVERLAY_AUTHENTICATION_REQUIRED", 401)
return token
class OverlayHandler(BaseHTTPRequestHandler):
server_version = "aios-task-identity-overlay/1"
sys_version = ""
@property
def overlay(self) -> TaskIdentityOverlayService:
return self.server.overlay # type: ignore[attr-defined]
def log_message(self, format: str, *args: Any) -> None:
# Never log Authorization or request bodies.
return
def _send(self, status: int, value: dict[str, Any]) -> None:
payload = (
json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
)
+ "\n"
).encode("utf-8")
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Cache-Control", "no-store")
self.send_header("Content-Length", str(len(payload)))
self.end_headers()
self.wfile.write(payload)
def _body(self) -> dict[str, Any]:
try:
length = int(self.headers.get("Content-Length", "0"))
except ValueError:
raise IdentityOverlayError("OVERLAY_CONTENT_LENGTH_INVALID") from None
if length <= 0 or length > MAX_BODY_BYTES:
raise IdentityOverlayError("OVERLAY_BODY_SIZE_REJECTED", 413)
raw = self.rfile.read(length)
try:
value = json.loads(raw)
except (UnicodeDecodeError, json.JSONDecodeError):
raise IdentityOverlayError("OVERLAY_BODY_JSON_INVALID") from None
if not isinstance(value, dict):
raise IdentityOverlayError("OVERLAY_BODY_OBJECT_REQUIRED")
return value
def do_GET(self) -> None:
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
if path != "/healthz":
self._send(404, {"error": "NOT_FOUND"})
return
self._send(
200,
{
"service": "aios-task-identity-overlay",
"status": "active" if _enabled() else "disabled",
"global_token_registry_used": False,
"loopback_only": True,
},
)
def do_POST(self) -> None:
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
if path != _ENDPOINT:
self._send(404, {"error": "NOT_FOUND"})
return
if not _enabled():
self._send(503, {"error": "IDENTITY_OVERLAY_DISABLED"})
return
try:
token = _bearer(self.headers.get("Authorization", ""))
result = self.overlay.provision(self._body(), token)
self._send(200, result)
except (IdentityOverlayError, OwnerAuthorizationError, ProvisioningError) as exc:
body: dict[str, Any] = {"error": exc.code}
if getattr(exc, "details", None):
body["details"] = exc.details
self._send(exc.status, body)
except Exception:
self._send(500, {"error": "IDENTITY_OVERLAY_API_INTERNAL_ERROR"})
class OverlayServer(ThreadingHTTPServer):
allow_reuse_address = True
def __init__(
self,
address: tuple[str, int],
overlay: TaskIdentityOverlayService,
) -> None:
super().__init__(address, OverlayHandler)
self.overlay = overlay
def main() -> None:
host = os.getenv("AIOS_GITOPS_IDENTITY_OVERLAY_HOST", "127.0.0.1")
if host not in {"127.0.0.1", "::1", "localhost"}:
raise SystemExit("identity overlay must listen on loopback")
try:
port = int(os.getenv("AIOS_GITOPS_IDENTITY_OVERLAY_PORT", "19864"))
except ValueError:
raise SystemExit("invalid overlay port") from None
if not 1024 <= port <= 65535:
raise SystemExit("invalid overlay port")
config = IdentityOverlayConfig.from_environment()
server = OverlayServer(
(host, port),
TaskIdentityOverlayService(config),
)
server.serve_forever(poll_interval=0.25)
if __name__ == "__main__":
main()
+92
View File
@@ -0,0 +1,92 @@
#!/usr/bin/env python3
"""Root-governed CLI for issuing and revoking task identity overlays.
The CLI never prints the bearer token. It returns only the root-owned secret
reference and non-secret metadata.
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from pathlib import Path
from task_identity_overlay import (
IdentityOverlayConfig,
IdentityOverlayError,
TaskIdentityOverlayStore,
)
def _write_result(path: str, value: dict) -> None:
data = json.dumps(value, sort_keys=True, separators=(",", ":")) + "\n"
if path == "-":
sys.stdout.write(data)
return
target = Path(path)
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(data, encoding="utf-8")
os.chmod(target, 0o600)
def parser() -> argparse.ArgumentParser:
value = argparse.ArgumentParser(
description="Issue/status/revoke a task-scoped identity overlay"
)
sub = value.add_subparsers(dest="command", required=True)
issue = sub.add_parser("issue")
issue.add_argument("--ttl-seconds", type=int, default=900)
issue.add_argument("--authorization-assertion-ref", required=True)
issue.add_argument(
"--operator-identity",
default=os.getenv("SUDO_USER") or os.getenv("USER") or "root",
)
issue.add_argument("--result-json", default="-")
status = sub.add_parser("status")
status.add_argument("--overlay-id", required=True)
status.add_argument("--result-json", default="-")
revoke = sub.add_parser("revoke")
revoke.add_argument("--overlay-id", required=True)
revoke.add_argument("--reason", required=True)
revoke.add_argument("--result-json", default="-")
return value
def main() -> int:
args = parser().parse_args()
try:
store = TaskIdentityOverlayStore(
IdentityOverlayConfig.from_environment()
)
if args.command == "issue":
result = store.issue(
ttl_seconds=args.ttl_seconds,
authorization_assertion_ref=args.authorization_assertion_ref,
operator_identity=args.operator_identity,
)
elif args.command == "status":
result = store.status(args.overlay_id)
else:
result = store.revoke(args.overlay_id, reason=args.reason)
_write_result(args.result_json, result)
return 0
except IdentityOverlayError as exc:
body = {"status": "BLOCKED", "error": exc.code}
if exc.details:
body["details"] = exc.details
_write_result(getattr(args, "result_json", "-"), body)
return 2
except Exception:
_write_result(
getattr(args, "result_json", "-"),
{"status": "FAILED", "error": "IDENTITY_OVERLAY_CLI_INTERNAL_ERROR"},
)
return 1
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,798 @@
#!/usr/bin/env python3
"""Fail-closed AF_UNIX builder for server-owned trusted candidate evidence."""
from __future__ import annotations
import argparse
import base64
import hashlib
import hmac
import json
import os
import secrets
import socket
import stat
import struct
import subprocess
import time
from datetime import datetime, timezone
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Callable, Mapping, Protocol
class BuilderError(RuntimeError):
"""A stable, non-sensitive error code returned by the builder boundary."""
def _canonical_json(value: Mapping[str, Any]) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=True).encode("utf-8")
def _sha256(*parts: str) -> str:
return hashlib.sha256("\0".join(parts).encode("utf-8")).hexdigest()
@dataclass(frozen=True)
class PeerIdentity:
pid: int
uid: int
gid: int
class DomainPurposeSigner(Protocol):
"""A signer deliberately limited to the builder's one domain purpose."""
@property
def key_id(self) -> str:
...
@property
def key_generation(self) -> int:
...
def sign_for_purpose(self, purpose: str, payload: bytes) -> bytes:
...
class CredentialPathEd25519Signer:
"""Reads a systemd-mounted credential path; never exports or logs key bytes."""
def __init__(self, credential_path: Path, *, key_id: str, key_generation: int, allowed_purpose: str):
self._credential_path = Path(credential_path)
self._key_id = str(key_id)
self._key_generation = key_generation
self._allowed_purpose = str(allowed_purpose)
if not self._key_id or not isinstance(key_generation, int) or key_generation < 1:
raise BuilderError("SIGNING_IDENTITY_INVALID")
if not self._credential_path.is_file():
raise BuilderError("SIGNING_CREDENTIAL_UNAVAILABLE")
try:
from cryptography.hazmat.primitives.serialization import load_pem_private_key
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
self._private_key = load_pem_private_key(self._credential_path.read_bytes(), password=None)
if not isinstance(self._private_key, Ed25519PrivateKey):
raise TypeError("signing key is not Ed25519")
except Exception as exc: # do not expose parse/key material details
raise BuilderError("SIGNING_CREDENTIAL_INVALID") from exc
@property
def key_id(self) -> str:
return self._key_id
@property
def key_generation(self) -> int:
return self._key_generation
def sign_for_purpose(self, purpose: str, payload: bytes) -> bytes:
if purpose != self._allowed_purpose:
raise BuilderError("SIGNING_PURPOSE_DENIED")
return self._private_key.sign(payload)
@dataclass(frozen=True)
class BuilderKeyring:
builder_hmac: bytes
signing_private_key_path: Path | None = None
def __post_init__(self) -> None:
if len(self.builder_hmac) < 32:
raise BuilderError("BUILDER_HMAC_INVALID")
@classmethod
def from_credentials_directory(cls, credentials_directory: Path) -> "BuilderKeyring":
root = Path(credentials_directory)
hmac_path = root / "builder_hmac"
signing_path = root / "signing_private_key"
try:
secret = hmac_path.read_bytes().strip()
except OSError as exc:
raise BuilderError("BUILDER_HMAC_UNAVAILABLE") from exc
if not signing_path.is_file():
raise BuilderError("SIGNING_CREDENTIAL_UNAVAILABLE")
return cls(builder_hmac=secret, signing_private_key_path=signing_path)
@dataclass(frozen=True)
class TrustedCandidateBuilderConfig:
policy_schema: str
repository: str
request_schema: str
response_schema: str
operation: str
readiness_operation: str
maximum_request_age_seconds: int
request_timeout_seconds: int
allowed_peer_uid: int
allowed_peer_cgroups: tuple[str, ...]
allowed_peer_executable_prefix: str
allowed_peer_runtime_prefix: str
allowed_peer_runtime_suffix: str
required_checks: tuple[str, ...]
evidence_layout: Mapping[str, str]
recipes: Mapping[str, Mapping[str, str]]
governed_recipes: Mapping[str, Mapping[str, str]]
policy_sha256: str
@classmethod
def from_policy(cls, policy: Mapping[str, Any]) -> "TrustedCandidateBuilderConfig":
required = {
"schema", "repository", "request_schema", "response_schema", "operation",
"readiness_operation", "maximum_request_age_seconds", "request_timeout_seconds",
"allowed_peer_uid", "allowed_peer_cgroups", "allowed_peer_executable_prefix",
"allowed_peer_runtime_prefix", "allowed_peer_runtime_suffix",
"required_checks", "evidence_layout", "recipes", "governed_recipes",
}
if set(policy) != required or policy.get("schema") != "aios.trusted-candidate-builder-policy.v1":
raise BuilderError("POLICY_SCHEMA_INVALID")
recipes = policy.get("recipes")
if not isinstance(recipes, Mapping) or set(recipes) != {"V2R2C2", "V2R2C3"}:
raise BuilderError("POLICY_RECIPES_INVALID")
c2, c3 = recipes["V2R2C2"], recipes["V2R2C3"]
if not isinstance(c2, Mapping) or set(c2) != {"mode", "task_id", "candidate_id", "base_oid", "source_commit", "version", "purpose"} or c2.get("mode") != "fixed":
raise BuilderError("POLICY_V2R2C2_INVALID")
if not isinstance(c3, Mapping) or set(c3) != {"mode", "derives_from", "version", "purpose"} or c3.get("mode") != "governed-derived-only" or c3.get("derives_from") != "V2R2C2":
raise BuilderError("POLICY_V2R2C3_INVALID")
if (
policy["repository"] != "aios/aios"
or policy["operation"] != "BuildTrustedCandidateV1"
or policy["readiness_operation"] != "ProbeTrustedCandidateBuilderV1"
):
raise BuilderError("POLICY_SCOPE_INVALID")
if not isinstance(policy["maximum_request_age_seconds"], int) or not 1 <= policy["maximum_request_age_seconds"] <= 300:
raise BuilderError("POLICY_FRESHNESS_INVALID")
if not isinstance(policy["request_timeout_seconds"], int) or not 1 <= policy["request_timeout_seconds"] <= 120:
raise BuilderError("POLICY_TIMEOUT_INVALID")
if not isinstance(policy["allowed_peer_uid"], int) or policy["allowed_peer_uid"] != 1000:
raise BuilderError("POLICY_PEER_UID_INVALID")
cgroups, checks = policy["allowed_peer_cgroups"], policy["required_checks"]
if not isinstance(cgroups, list) or not cgroups or not all(isinstance(value, str) and value.startswith("/") for value in cgroups):
raise BuilderError("POLICY_CGROUPS_INVALID")
for field in (
"allowed_peer_executable_prefix",
"allowed_peer_runtime_prefix",
"allowed_peer_runtime_suffix",
):
if not isinstance(policy[field], str) or not policy[field].startswith("/"):
raise BuilderError("POLICY_PROCESS_IDENTITY_INVALID")
if not isinstance(checks, list) or not checks or len(set(checks)) != len(checks) or not all(isinstance(value, str) for value in checks):
raise BuilderError("POLICY_CHECKS_INVALID")
layout = policy["evidence_layout"]
evidence_names = {
"artifact", "manifest", "package_closure", "tests", "canary",
"gitea", "rollback", "policy", "secret_scan",
}
if (
not isinstance(layout, Mapping)
or set(layout) != evidence_names
or not all(
isinstance(value, str)
and value.startswith("{version}/")
and ".." not in Path(value).parts
and not Path(value).is_absolute()
for value in layout.values()
)
):
raise BuilderError("POLICY_EVIDENCE_LAYOUT_INVALID")
return cls(
policy_schema=str(policy["schema"]), repository=str(policy["repository"]),
request_schema=str(policy["request_schema"]), response_schema=str(policy["response_schema"]),
operation=str(policy["operation"]), readiness_operation=str(policy["readiness_operation"]),
maximum_request_age_seconds=policy["maximum_request_age_seconds"],
request_timeout_seconds=policy["request_timeout_seconds"],
allowed_peer_uid=policy["allowed_peer_uid"], allowed_peer_cgroups=tuple(cgroups),
allowed_peer_executable_prefix=policy["allowed_peer_executable_prefix"],
allowed_peer_runtime_prefix=policy["allowed_peer_runtime_prefix"],
allowed_peer_runtime_suffix=policy["allowed_peer_runtime_suffix"],
required_checks=tuple(checks), evidence_layout=dict(layout),
recipes={name: dict(value) for name, value in recipes.items()},
governed_recipes={name: dict(value) for name, value in policy["governed_recipes"].items()} if isinstance(policy["governed_recipes"], Mapping) else _invalid_governed_recipes(),
policy_sha256=hashlib.sha256(_canonical_json(dict(policy))).hexdigest(),
)
def fixed_recipe(self, name: str) -> Mapping[str, str]:
recipe = self.recipes.get(name)
if recipe is None:
raise BuilderError("RECIPE_UNKNOWN")
if recipe["mode"] == "governed-derived-only":
derived = self.governed_recipes.get(name)
required = {"mode", "task_id", "candidate_id", "base_oid", "source_commit", "version", "purpose", "derived_from_v2r2c2_attestation", "builder_engineering_oid"}
if not isinstance(derived, Mapping) or set(derived) != required or derived.get("mode") != "governed-derived" or derived.get("version") != name or derived.get("purpose") != recipe.get("purpose"):
raise BuilderError("GOVERNED_V2R2C3_DERIVATION_REQUIRED")
for name in ("base_oid", "source_commit", "builder_engineering_oid"):
value = derived.get(name)
if not isinstance(value, str) or len(value) != 40 or any(char not in "0123456789abcdef" for char in value.lower()):
raise BuilderError("GOVERNED_V2R2C3_DERIVATION_REQUIRED")
return dict(derived)
return recipe
def resolve_recipe(self, version: str, candidate_id: str | None, source_commit: str) -> Mapping[str, str]:
recipe = self.fixed_recipe(version)
# Intent-only callers do not choose a candidate identity. The
# policy-owned recipe supplies it; explicit worker requests are
# still checked against the same immutable recipe.
effective_candidate_id = str(candidate_id or recipe.get("candidate_id") or "")
if {"version": recipe.get("version"), "candidate_id": recipe.get("candidate_id"), "source_commit": recipe.get("source_commit")} != {"version": version, "candidate_id": effective_candidate_id, "source_commit": source_commit}:
raise BuilderError("REQUEST_RELEASE_BINDING_INVALID")
return recipe
def _invalid_governed_recipes() -> Mapping[str, Mapping[str, str]]:
raise BuilderError("POLICY_GOVERNED_RECIPES_INVALID")
def load_builder_policy(path: Path) -> Mapping[str, Any]:
try:
decoded = json.loads(Path(path).read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
raise BuilderError("POLICY_UNAVAILABLE") from exc
if not isinstance(decoded, Mapping):
raise BuilderError("POLICY_SCHEMA_INVALID")
TrustedCandidateBuilderConfig.from_policy(decoded)
return decoded
class ServerOwnedEvidenceSource:
"""Recomputes a fixed on-host evidence closure without caller-owned paths."""
def __init__(
self,
config: TrustedCandidateBuilderConfig,
authority_git_dir: Path,
evidence_root: Path,
*,
expected_owner_uid: int | None = 0,
git_runner: Callable[..., Any] = subprocess.run,
):
self._config = config
self._authority_git_dir = Path(authority_git_dir)
self._evidence_root = Path(evidence_root)
self._expected_owner_uid = expected_owner_uid
self._git_runner = git_runner
def _regular_path(self, relative: str) -> Path:
root = self._evidence_root.resolve(strict=True)
candidate = self._evidence_root / relative
current = self._evidence_root
for part in Path(relative).parts:
current = current / part
if current.is_symlink():
raise BuilderError("COLLECTOR_SYMLINK_DENIED")
try:
resolved = candidate.resolve(strict=True)
resolved.relative_to(root)
metadata = resolved.stat(follow_symlinks=False)
except (OSError, ValueError) as exc:
raise BuilderError("COLLECTOR_EVIDENCE_UNAVAILABLE") from exc
if not stat.S_ISREG(metadata.st_mode):
raise BuilderError("COLLECTOR_SPECIAL_FILE_DENIED")
if os.name != "nt" and metadata.st_mode & 0o022:
raise BuilderError("COLLECTOR_EVIDENCE_MODE_DENIED")
if (
os.name != "nt"
and
self._expected_owner_uid is not None
and metadata.st_uid != self._expected_owner_uid
):
raise BuilderError("COLLECTOR_EVIDENCE_OWNER_DENIED")
return resolved
def _digest(self, path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def _json(self, path: Path) -> Mapping[str, Any]:
try:
if path.stat().st_size > 2 * 1024 * 1024:
raise BuilderError("COLLECTOR_EVIDENCE_SIZE_INVALID")
value = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise BuilderError("COLLECTOR_EVIDENCE_INVALID") from exc
if not isinstance(value, Mapping):
raise BuilderError("COLLECTOR_EVIDENCE_INVALID")
return value
def __call__(self, recipe_name: str, recipe: Mapping[str, str]) -> Mapping[str, Any]:
try:
tree_result = self._git_runner(
["git", f"--git-dir={self._authority_git_dir}", "rev-parse", f"{recipe['source_commit']}^{{tree}}"],
check=True, capture_output=True, text=True, timeout=15,
env={"PATH": os.environ.get("PATH", "")},
)
ancestry_result = self._git_runner(
["git", f"--git-dir={self._authority_git_dir}", "merge-base", "--is-ancestor", recipe["base_oid"], recipe["source_commit"]],
check=False, capture_output=True, text=True, timeout=15,
env={"PATH": os.environ.get("PATH", "")},
)
tree_oid = tree_result.stdout.strip().lower()
except (OSError, subprocess.SubprocessError) as exc:
raise BuilderError("COLLECTOR_EVIDENCE_UNAVAILABLE") from exc
if getattr(ancestry_result, "returncode", 1) != 0:
raise BuilderError("COLLECTOR_GIT_MISMATCH")
receipt = self._json(self._regular_path(f"{recipe_name}/receipt.json"))
if set(receipt) != {"schema", "version", "candidate", "release_plan"}:
raise BuilderError("COLLECTOR_EVIDENCE_INVALID")
if (
receipt.get("schema") != "aios.trusted-candidate-builder.evidence.v1"
or receipt.get("version") != recipe_name
):
raise BuilderError("COLLECTOR_EVIDENCE_INVALID")
candidate = receipt.get("candidate")
release_plan = receipt.get("release_plan")
if not isinstance(candidate, Mapping) or not isinstance(release_plan, Mapping):
raise BuilderError("COLLECTOR_EVIDENCE_INVALID")
if (
candidate.get("candidate_id") != recipe["candidate_id"]
or candidate.get("source_commit") != recipe["source_commit"]
or candidate.get("tree_oid") != tree_oid
):
raise BuilderError("COLLECTOR_GIT_MISMATCH")
git_consistency = candidate.get("git_consistency")
if (
not isinstance(git_consistency, Mapping)
or git_consistency.get("consistent") is not True
or git_consistency.get("source_commit") != recipe["source_commit"]
or git_consistency.get("tree_oid") != tree_oid
or git_consistency.get("repository") != self._config.repository
):
raise BuilderError("COLLECTOR_GIT_MISMATCH")
paths = {
name: self._regular_path(template.format(version=recipe_name))
for name, template in self._config.evidence_layout.items()
}
deployment = release_plan.get("deployment")
if not isinstance(deployment, Mapping):
raise BuilderError("COLLECTOR_EVIDENCE_INVALID")
digests = {name: self._digest(path) for name, path in paths.items()}
if (
digests["artifact"] != candidate.get("artifact_sha256")
or digests["manifest"] != candidate.get("manifest_sha256")
or digests["package_closure"] != deployment.get("package_closure_sha256")
or digests["policy"] != release_plan.get("policy_sha256")
or digests["policy"] != deployment.get("policy_sha256")
):
raise BuilderError("COLLECTOR_DIGEST_MISMATCH")
tests = self._json(paths["tests"])
canary = self._json(paths["canary"])
gitea = self._json(paths["gitea"])
rollback = self._json(paths["rollback"])
secret_scan = self._json(paths["secret_scan"])
if tests != candidate.get("tests") or tests.get("passed") is not True:
raise BuilderError("COLLECTOR_TESTS_MISMATCH")
if canary != candidate.get("canary") or canary.get("passed") is not True:
raise BuilderError("COLLECTOR_CANARY_MISMATCH")
if (
digests["tests"] != hashlib.sha256(_canonical_json(dict(tests))).hexdigest()
or digests["canary"] != hashlib.sha256(_canonical_json(dict(canary))).hexdigest()
):
raise BuilderError("COLLECTOR_EVIDENCE_CANONICALIZATION_MISMATCH")
if (
gitea != candidate.get("gitea_consistency")
or gitea.get("consistent") is not True
or gitea.get("repository") != self._config.repository
or gitea.get("source_commit") != recipe["source_commit"]
or gitea.get("tree_oid") != tree_oid
or gitea.get("remote_ref_oid") != recipe["source_commit"]
):
raise BuilderError("COLLECTOR_GITEA_MISMATCH")
if (
candidate.get("rollback_ready") is not True
or rollback.get("ready") is not True
or rollback.get("source_commit") != recipe["source_commit"]
or rollback.get("rollback_sha256") != deployment.get("rollback_sha256")
):
raise BuilderError("COLLECTOR_ROLLBACK_MISMATCH")
if (
digests["gitea"] != hashlib.sha256(_canonical_json(dict(gitea))).hexdigest()
or digests["rollback"] != hashlib.sha256(_canonical_json(dict(rollback))).hexdigest()
):
raise BuilderError("COLLECTOR_EVIDENCE_CANONICALIZATION_MISMATCH")
if (
secret_scan.get("passed") is not True
or secret_scan.get("artifact_sha256") != candidate.get("artifact_sha256")
):
raise BuilderError("COLLECTOR_SECRET_SCAN_MISMATCH")
checks = {name: True for name in self._config.required_checks}
return {
"candidate": dict(candidate),
"release_plan": dict(release_plan),
"checks": checks,
"evidence_digests": digests,
}
class DeterministicEvidenceCollector:
"""Validates a server-owned source; request fields never influence collection."""
_BUNDLE_FIELDS = {"candidate", "release_plan", "checks", "evidence_digests"}
def __init__(self, config: TrustedCandidateBuilderConfig, *, evidence_source: Callable[[str, Mapping[str, str]], Mapping[str, Any]] | None = None):
self._config = config
self._evidence_source = evidence_source
def collect(self, recipe_name: str, recipe: Mapping[str, str]) -> Mapping[str, Any]:
if self._evidence_source is None:
raise BuilderError("COLLECTOR_UNCONFIGURED")
bundle = self._evidence_source(recipe_name, recipe)
if not isinstance(bundle, Mapping) or set(bundle) != self._BUNDLE_FIELDS:
raise BuilderError("COLLECTOR_EVIDENCE_INVALID")
raw_checks = bundle.get("checks")
if (
not isinstance(raw_checks, Mapping)
or set(raw_checks) != set(self._config.required_checks)
or any(raw_checks.get(name) is not True for name in self._config.required_checks)
):
raise BuilderError("COLLECTOR_CHECKS_INCOMPLETE")
evidence_digests = bundle.get("evidence_digests")
if (
not isinstance(evidence_digests, Mapping)
or set(evidence_digests) != set(self._config.evidence_layout)
or any(
not isinstance(value, str)
or len(value) != 64
or any(char not in "0123456789abcdef" for char in value.lower())
for value in evidence_digests.values()
)
):
raise BuilderError("COLLECTOR_EVIDENCE_DIGESTS_INVALID")
try:
from silent_release_controller import ReleaseCandidate
from silent_release_hardening import ReleasePlan, TrustedCandidateBundle
candidate = ReleaseCandidate.from_dict(dict(bundle["candidate"]))
release_plan = ReleasePlan.from_dict(dict(bundle["release_plan"]))
checks = dict(raw_checks)
TrustedCandidateBundle(candidate, release_plan, checks).validate()
except (ImportError, KeyError, TypeError, ValueError, RuntimeError) as exc:
raise BuilderError("COLLECTOR_EVIDENCE_INVALID") from exc
if candidate.source_commit != recipe["source_commit"] or candidate.candidate_id != recipe["candidate_id"]:
raise BuilderError("COLLECTOR_SOURCE_MISMATCH")
return {
"candidate": candidate.to_dict(),
"release_plan": release_plan.to_dict(),
"checks": checks,
"evidence_digests": dict(evidence_digests),
}
def _extract_peer_identity(connection: socket.socket) -> PeerIdentity:
if not hasattr(socket, "SO_PEERCRED"):
raise BuilderError("SO_PEERCRED_UNAVAILABLE")
raw = connection.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, struct.calcsize("3i"))
if len(raw) != struct.calcsize("3i"):
raise BuilderError("SO_PEERCRED_INVALID")
return PeerIdentity(*struct.unpack("3i", raw))
def _resolve_cgroups(pid: int) -> tuple[str, ...]:
try:
return tuple(
line.rsplit(":", 1)[1].strip()
for line in Path(f"/proc/{pid}/cgroup").read_text(encoding="utf-8").splitlines()
if ":" in line and line.rsplit(":", 1)[1].strip().startswith("/")
)
except OSError as exc:
raise BuilderError("PEER_CGROUP_UNAVAILABLE") from exc
def _resolve_process_identity(pid: int) -> tuple[str, tuple[str, ...]]:
try:
executable = os.path.realpath(f"/proc/{pid}/exe")
argv = tuple(
item.decode("utf-8", "strict")
for item in Path(f"/proc/{pid}/cmdline").read_bytes().split(b"\0")
if item
)
except (OSError, UnicodeDecodeError) as exc:
raise BuilderError("PEER_PROCESS_UNAVAILABLE") from exc
if not executable or not argv:
raise BuilderError("PEER_PROCESS_UNAVAILABLE")
return executable, argv
class TrustedCandidateBuilder:
def __init__(
self, config: TrustedCandidateBuilderConfig, keyring: BuilderKeyring, *,
collector: DeterministicEvidenceCollector, signer: DomainPurposeSigner,
peer_identity_resolver: Callable[[socket.socket], PeerIdentity] = _extract_peer_identity,
cgroup_resolver: Callable[[int], tuple[str, ...] | str] = _resolve_cgroups,
process_identity_resolver: Callable[[int], tuple[str, tuple[str, ...]]] = _resolve_process_identity,
clock: Callable[[], float] = time.time, nonce_ledger_path: Path | None = None,
) -> None:
self.config, self.keyring, self.collector, self.signer = config, keyring, collector, signer
self.peer_identity_resolver, self.cgroup_resolver, self.clock = peer_identity_resolver, cgroup_resolver, clock
self.process_identity_resolver = process_identity_resolver
self._nonce_ledger_path = Path(nonce_ledger_path) if nonce_ledger_path else None
self._nonce_ledger: dict[str, tuple[bytes, dict[str, Any]]] = self._load_nonce_ledger()
def _load_nonce_ledger(self) -> dict[str, tuple[bytes, dict[str, Any]]]:
if self._nonce_ledger_path is None or not self._nonce_ledger_path.exists():
return {}
if self._nonce_ledger_path.is_symlink():
raise BuilderError("NONCE_LEDGER_INVALID")
try:
raw = json.loads(self._nonce_ledger_path.read_text(encoding="utf-8"))
return {nonce: (bytes.fromhex(value["request_sha256"]), dict(value["result"])) for nonce, value in raw.items()}
except (OSError, ValueError, TypeError, KeyError, json.JSONDecodeError) as exc:
raise BuilderError("NONCE_LEDGER_INVALID") from exc
def _persist_nonce_ledger(self) -> None:
if self._nonce_ledger_path is None:
return
try:
self._nonce_ledger_path.parent.mkdir(mode=0o700, parents=True, exist_ok=True)
temporary = self._nonce_ledger_path.with_suffix(".tmp")
payload = {nonce: {"request_sha256": digest.hex(), "result": result} for nonce, (digest, result) in self._nonce_ledger.items()}
with temporary.open("wb") as handle:
handle.write(_canonical_json(payload))
handle.flush()
os.fsync(handle.fileno())
os.chmod(temporary, 0o600)
os.replace(temporary, self._nonce_ledger_path)
if os.name != "nt":
descriptor = os.open(self._nonce_ledger_path.parent, os.O_RDONLY)
try:
os.fsync(descriptor)
finally:
os.close(descriptor)
except OSError as exc:
raise BuilderError("NONCE_LEDGER_PERSIST_FAILED") from exc
def _verify_peer(self, connection: socket.socket) -> None:
peer = self.peer_identity_resolver(connection)
if peer.uid != self.config.allowed_peer_uid or peer.pid <= 0:
raise BuilderError("PEER_IDENTITY_DENIED")
resolved = self.cgroup_resolver(peer.pid)
cgroups = (resolved,) if isinstance(resolved, str) else tuple(resolved)
if not cgroups or not any(item in self.config.allowed_peer_cgroups for item in cgroups):
raise BuilderError("PEER_CGROUP_DENIED")
executable, argv = self.process_identity_resolver(peer.pid)
if not executable.startswith(self.config.allowed_peer_executable_prefix):
raise BuilderError("PEER_PROCESS_DENIED")
runtime_arguments = [
value
for value in argv
if value.startswith(self.config.allowed_peer_runtime_prefix)
and value.endswith(self.config.allowed_peer_runtime_suffix)
]
if len(runtime_arguments) != 1:
raise BuilderError("PEER_PROCESS_DENIED")
def handle_request(self, request: Mapping[str, Any], *, connection: socket.socket) -> dict[str, Any]:
if not isinstance(request, Mapping):
raise BuilderError("REQUEST_SCHEMA_INVALID")
expected_keys = {"schema", "operation", "recipe", "repository", "task_id", "candidate_id", "base_oid", "source_commit", "version", "purpose", "issued_at", "nonce", "hmac"}
if (
set(request) != expected_keys
or request.get("schema") != self.config.request_schema
or request.get("operation") not in {
self.config.operation,
self.config.readiness_operation,
}
):
raise BuilderError("REQUEST_SCHEMA_INVALID")
supplied_mac = request.get("hmac")
unsigned = {key: value for key, value in request.items() if key != "hmac"}
if not isinstance(supplied_mac, str) or not hmac.compare_digest(hmac.new(self.keyring.builder_hmac, _canonical_json(unsigned), hashlib.sha256).hexdigest(), supplied_mac):
raise BuilderError("REQUEST_HMAC_INVALID")
issued_at, nonce = request.get("issued_at"), request.get("nonce")
if not isinstance(issued_at, int) or not isinstance(nonce, str) or len(nonce) < 16 or len(nonce) > 256:
raise BuilderError("REQUEST_SCHEMA_INVALID")
self._verify_peer(connection)
request_bytes = _canonical_json(dict(request))
replay = self._nonce_ledger.get(nonce)
if replay is not None:
if hmac.compare_digest(replay[0], hashlib.sha256(request_bytes).digest()):
return dict(replay[1])
raise BuilderError("REPLAY_CONFLICT")
if abs(int(self.clock()) - issued_at) > self.config.maximum_request_age_seconds:
raise BuilderError("REQUEST_STALE")
recipe_name = request.get("recipe")
if not isinstance(recipe_name, str):
raise BuilderError("REQUEST_SCHEMA_INVALID")
recipe = self.config.resolve_recipe(str(request.get("version") or ""), str(request.get("candidate_id") or ""), str(request.get("source_commit") or ""))
if recipe_name != recipe.get("version"):
raise BuilderError("REQUEST_RELEASE_BINDING_INVALID")
bound = {"repository": self.config.repository, "task_id": recipe["task_id"], "candidate_id": recipe["candidate_id"], "base_oid": recipe["base_oid"], "source_commit": recipe["source_commit"], "version": recipe["version"], "purpose": recipe["purpose"]}
if {key: request.get(key) for key in bound} != bound:
raise BuilderError("REQUEST_SCOPE_INVALID")
bundle = dict(self.collector.collect(recipe_name, recipe))
candidate = bundle["candidate"]
release_plan = bundle["release_plan"]
checks = bundle.get("checks")
evidence_digests = bundle["evidence_digests"]
if not isinstance(checks, Mapping) or set(checks) != set(self.config.required_checks) or not all(value is True for value in checks.values()):
raise BuilderError("COLLECTOR_CHECKS_INCOMPLETE")
if request["operation"] == self.config.readiness_operation:
result = {
"schema": self.config.response_schema,
"status": "READY",
"repository": self.config.repository,
"recipe": recipe_name,
"candidate_id": recipe["candidate_id"],
"source_commit": recipe["source_commit"],
"builder_key_id": self.signer.key_id,
"builder_key_generation": self.signer.key_generation,
"signature_algorithm": "Ed25519",
"signature_purpose": "AIOS_TRUSTED_RELEASE_BUILDER",
"policy_sha256": self.config.policy_sha256,
"collector_ready": True,
"signer_ready": True,
}
if len(self._nonce_ledger) >= 2048:
self._nonce_ledger.pop(next(iter(self._nonce_ledger)))
self._nonce_ledger[nonce] = (
hashlib.sha256(request_bytes).digest(),
dict(result),
)
self._persist_nonce_ledger()
return result
issued = datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
attestation = {
"builder_attestation_id": "builder-attestation-" + _sha256(recipe_name, recipe["task_id"], candidate["artifact_sha256"])[:24],
"builder_key_id": self.signer.key_id,
"builder_key_generation": self.signer.key_generation,
"signature_algorithm": "Ed25519",
"signature_purpose": "AIOS_TRUSTED_RELEASE_BUILDER",
"source_commit": candidate["source_commit"], "tree_oid": candidate["tree_oid"],
"artifact_sha256": candidate["artifact_sha256"], "manifest_sha256": candidate["manifest_sha256"],
"package_closure_sha256": release_plan["deployment"]["package_closure_sha256"],
"test_evidence_sha256": evidence_digests["tests"],
"canary_evidence_sha256": evidence_digests["canary"],
"gitea_evidence_sha256": evidence_digests["gitea"],
"rollback_evidence_sha256": evidence_digests["rollback"],
"policy_evidence_sha256": evidence_digests["policy"],
"secret_scan_evidence_sha256": evidence_digests["secret_scan"],
"issued_at": issued, "trust_domain": "aios.trusted-builder.attestation.v1",
}
canonical_attestation = _canonical_json(attestation)
fingerprint = hashlib.sha256(canonical_attestation).hexdigest()
signature = base64.b64encode(self.signer.sign_for_purpose("AIOS_TRUSTED_RELEASE_BUILDER", canonical_attestation)).decode("ascii")
result = {
"schema": self.config.response_schema, "status": "BUILT", "recipe": recipe_name,
"repository": self.config.repository, "task_id": recipe["task_id"], "base_oid": recipe["base_oid"],
"purpose": recipe["purpose"], "checks": dict(checks),
"evidence": {"candidate_id": candidate["candidate_id"], "source_commit": candidate["source_commit"], "tree_oid": candidate["tree_oid"], "artifact_sha256": candidate["artifact_sha256"], "manifest_sha256": candidate["manifest_sha256"], "package_closure_sha256": release_plan["deployment"]["package_closure_sha256"], "test_evidence_sha256": attestation["test_evidence_sha256"], "canary_evidence_sha256": attestation["canary_evidence_sha256"]},
"attestation": {**attestation, "fingerprint": fingerprint, "signature": signature},
}
result["silent_release_bundle"] = {"candidate": candidate, "release_plan": release_plan, "checks": dict(checks), "attestation": result["attestation"]}
if len(self._nonce_ledger) >= 2048:
self._nonce_ledger.pop(next(iter(self._nonce_ledger)))
self._nonce_ledger[nonce] = (hashlib.sha256(request_bytes).digest(), dict(result))
self._persist_nonce_ledger()
return result
def serve_forever(self, listener: socket.socket) -> None:
while True:
connection, _ = listener.accept()
with connection:
try:
raw = connection.makefile("rb").readline(65537)
if not raw or len(raw) > 65536:
raise BuilderError("REQUEST_SIZE_INVALID")
result = self.handle_request(json.loads(raw), connection=connection)
except (BuilderError, UnicodeDecodeError, json.JSONDecodeError):
result = {"schema": self.config.response_schema, "status": "REJECTED", "error_code": "REQUEST_REJECTED"}
connection.sendall(_canonical_json(result) + b"\n")
class UnixTrustedCandidateBuilderClient:
def __init__(self, config: TrustedCandidateBuilderConfig, keyring: BuilderKeyring, *, socket_path: Path | None = None, clock: Callable[[], float] = time.time) -> None:
self.config, self.keyring, self.socket_path, self.clock = config, keyring, socket_path, clock
def create_request(self, version: str, candidate_id: str | None, source_commit: str, *, nonce: str, operation: str | None = None) -> dict[str, Any]:
selected = self.config.resolve_recipe(version, candidate_id, source_commit)
selected_operation = operation or self.config.operation
if selected_operation not in {self.config.operation, self.config.readiness_operation}:
raise BuilderError("REQUEST_OPERATION_INVALID")
request = {"schema": self.config.request_schema, "operation": selected_operation, "recipe": version, "repository": self.config.repository, "task_id": selected["task_id"], "candidate_id": selected["candidate_id"], "base_oid": selected["base_oid"], "source_commit": selected["source_commit"], "version": selected["version"], "purpose": selected["purpose"], "issued_at": int(self.clock()), "nonce": nonce}
request["hmac"] = hmac.new(self.keyring.builder_hmac, _canonical_json(request), hashlib.sha256).hexdigest()
return request
def _exchange(self, request: Mapping[str, Any]) -> Mapping[str, Any]:
if self.socket_path is None:
raise BuilderError("BUILDER_SOCKET_UNCONFIGURED")
try:
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as connection:
connection.settimeout(self.config.request_timeout_seconds)
connection.connect(os.fspath(self.socket_path))
connection.sendall(_canonical_json(dict(request)) + b"\n")
raw = connection.makefile("rb").readline(65537)
except (OSError, TimeoutError) as exc:
raise BuilderError("BUILDER_TRANSPORT_UNAVAILABLE") from exc
try:
result = json.loads(raw)
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise BuilderError("BUILDER_RESPONSE_INVALID") from exc
if not isinstance(result, Mapping) or result.get("schema") != self.config.response_schema:
raise BuilderError("BUILDER_RESPONSE_INVALID")
return result
def build(self, version: str, candidate_id: str | None, source_commit: str, *, nonce: str) -> Mapping[str, Any]:
result = self._exchange(
self.create_request(version, candidate_id, source_commit, nonce=nonce)
)
if result.get("status") != "BUILT":
raise BuilderError("BUILDER_REQUEST_REJECTED")
return result
def probe(self, *, nonce: str | None = None) -> Mapping[str, Any]:
recipe = self.config.fixed_recipe("V2R2C2")
result = self._exchange(
self.create_request(
"V2R2C2",
recipe["candidate_id"],
recipe["source_commit"],
nonce=nonce or secrets.token_hex(32),
operation=self.config.readiness_operation,
)
)
if result.get("status") != "READY":
raise BuilderError("BUILDER_NOT_READY")
return result
def readiness_probe(client: UnixTrustedCandidateBuilderClient) -> Mapping[str, Any]:
result = dict(client.probe())
required_true = {"collector_ready", "signer_ready"}
if any(result.get(name) is not True for name in required_true):
raise BuilderError("BUILDER_NOT_READY")
return result
def _main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--socket-fd", required=True, type=int)
parser.add_argument("--policy", required=True, type=Path)
arguments = parser.parse_args()
config = TrustedCandidateBuilderConfig.from_policy(load_builder_policy(arguments.policy))
credentials = os.environ.get("CREDENTIALS_DIRECTORY")
if not credentials:
raise SystemExit("CREDENTIALS_DIRECTORY is required")
keyring = BuilderKeyring.from_credentials_directory(Path(credentials))
signer = CredentialPathEd25519Signer(
keyring.signing_private_key_path,
key_id=os.environ["AIOS_TRUSTED_BUILDER_KEY_ID"],
key_generation=int(os.environ["AIOS_TRUSTED_BUILDER_KEY_GENERATION"]),
allowed_purpose="AIOS_TRUSTED_RELEASE_BUILDER",
)
listener = socket.fromfd(arguments.socket_fd, socket.AF_UNIX, socket.SOCK_STREAM)
collector = DeterministicEvidenceCollector(
config,
evidence_source=ServerOwnedEvidenceSource(
config,
Path(os.environ["AIOS_TRUSTED_BUILDER_GIT_DIR"]),
Path(os.environ["AIOS_TRUSTED_BUILDER_EVIDENCE_ROOT"]),
),
)
TrustedCandidateBuilder(config, keyring, collector=collector, signer=signer, nonce_ledger_path=Path(os.environ["AIOS_TRUSTED_BUILDER_NONCE_LEDGER"])).serve_forever(listener)
if __name__ == "__main__":
_main()
-110
View File
@@ -1,110 +0,0 @@
"""AIOS V2 控制面、执行编排与可信运行来源证明。"""
from .control_plane import (
AgentRecord,
AuditEvent,
ControlPlaneError,
ControlPlaneService,
IdempotencyConflict,
InMemoryControlPlaneStore,
InvalidTransition,
NotFound,
PolicyDenied,
RequestContext,
RevisionConflict,
TaskRecord,
TaskState,
TenantIsolationError,
TenantRecord,
Unauthorized,
)
from .execution_tracking import ExecutionEvidence, ExecutionTracker
from .orchestrator import DispatchRecord, ExecutionOrchestrator, OrchestrationError
from .provenance import (
Ed25519PrivateKeySigner,
Ed25519PublicKeyVerifier,
ExecutionPath,
GitOperation,
PRODUCTION_POLICY_VERSION,
ProvenanceAuthorization,
ProvenanceControlledWriter,
ProvenanceDenied,
ProvenanceError,
ProvenanceExpired,
ProvenanceIssuer,
ProvenanceNotFound,
ProvenanceReplay,
ProvenanceRevoked,
ProvenanceSignatureError,
ProvenanceStatus,
ProvenanceVerifier,
RunProvenance,
SQLiteProvenanceStore,
digest_payload,
)
from .provenance_git import (
GitCommandResult,
GitExecutionContext,
ProvenanceGitError,
ProvenanceGitWriter,
)
from .provenance_operations import (
OperationContext,
OperationReceipt,
ProvenanceOperationGate,
build_provenance_writer_from_env,
context_from_environment,
)
__all__ = [
"AgentRecord",
"AuditEvent",
"ControlPlaneError",
"ControlPlaneService",
"DispatchRecord",
"Ed25519PrivateKeySigner",
"Ed25519PublicKeyVerifier",
"ExecutionEvidence",
"ExecutionOrchestrator",
"ExecutionPath",
"ExecutionTracker",
"GitCommandResult",
"GitExecutionContext",
"GitOperation",
"IdempotencyConflict",
"InMemoryControlPlaneStore",
"InvalidTransition",
"NotFound",
"OperationContext",
"OperationReceipt",
"OrchestrationError",
"PRODUCTION_POLICY_VERSION",
"PolicyDenied",
"ProvenanceAuthorization",
"ProvenanceControlledWriter",
"ProvenanceDenied",
"ProvenanceError",
"ProvenanceExpired",
"ProvenanceGitError",
"ProvenanceGitWriter",
"ProvenanceIssuer",
"ProvenanceNotFound",
"ProvenanceOperationGate",
"ProvenanceReplay",
"ProvenanceRevoked",
"ProvenanceSignatureError",
"ProvenanceStatus",
"ProvenanceVerifier",
"RequestContext",
"RevisionConflict",
"RunProvenance",
"SQLiteProvenanceStore",
"TaskRecord",
"TaskState",
"TenantIsolationError",
"TenantRecord",
"Unauthorized",
"build_provenance_writer_from_env",
"context_from_environment",
"digest_payload",
]
-807
View File
@@ -1,807 +0,0 @@
"""AIOS V2 production-candidate application entrypoint."""
from __future__ import annotations
import dataclasses
import json
import os
import stat
import urllib.parse
from enum import Enum
from http import HTTPStatus
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from typing import Any, Mapping
from aios_hermes import HermesAdapter, HermesJobState, HermesResult, SQLiteHermesStore
from aios_openclaw import (
Ed25519CommandSigner,
HTTPRuntimeTransport,
OpenClawAdapter,
)
from aios_openclaw.callback import AuditCallbackSink, EventPublisher
from .console import CONSOLE_HTML
from .control_plane import AuditEvent, RequestContext, Unauthorized
from .dispatch_store import SQLiteDispatchRepository
from .execution_tracking import ExecutionEvidence, ExecutionTracker
from .gateway import GatewayApplication, error_response
from .gitops_status import collect_gitops_status
from .orchestrator import ExecutionOrchestrator
from .persistence import DurableControlPlaneService, SQLiteSnapshotRepository
from .provenance_operations import build_provenance_writer_from_env
def _json_value(value: Any) -> Any:
if dataclasses.is_dataclass(value):
return {
item.name: _json_value(getattr(value, item.name))
for item in dataclasses.fields(value)
}
if isinstance(value, Enum):
return value.value
if isinstance(value, Mapping):
return {str(key): _json_value(item) for key, item in value.items()}
if isinstance(value, (list, tuple, set, frozenset)):
return [_json_value(item) for item in value]
return value
def _enabled(value: str | None) -> bool:
return str(value or "").strip().lower() in {"1", "true", "yes", "on"}
def _header(headers: Mapping[str, str], name: str) -> str | None:
target = name.lower()
for key, value in headers.items():
if str(key).lower() == target:
normalized = str(value or "").strip()
return normalized or None
return None
def _provenance_context(
headers: Mapping[str, str],
payload: Mapping[str, Any],
) -> tuple[str | None, str | None]:
header_provenance = _header(headers, "X-AIOS-Provenance-ID")
header_run = _header(headers, "X-AIOS-Run-ID")
body_provenance = str(payload.get("provenance_id") or "").strip() or None
body_run = str(payload.get("run_id") or "").strip() or None
if header_provenance and body_provenance and header_provenance != body_provenance:
raise ValueError("provenance_id header/body mismatch")
if header_run and body_run and header_run != body_run:
raise ValueError("run_id header/body mismatch")
return header_provenance or body_provenance, header_run or body_run
def _read_protected_file(path: Path, *, private: bool) -> bytes:
flags = os.O_RDONLY
if hasattr(os, "O_NOFOLLOW"):
flags |= os.O_NOFOLLOW
descriptor = os.open(str(path), flags)
try:
metadata = os.fstat(descriptor)
if not stat.S_ISREG(metadata.st_mode):
raise RuntimeError("key reference is not a regular file")
denied = 0o077 if private else 0o022
if metadata.st_mode & denied:
raise RuntimeError(
"private key file permissions must be 0600 or stricter"
if private
else "public key file must not be group/world writable"
)
chunks: list[bytes] = []
while True:
chunk = os.read(descriptor, 65536)
if not chunk:
break
chunks.append(chunk)
value = b"".join(chunks)
if not value:
raise RuntimeError("key file is empty")
return value
finally:
os.close(descriptor)
class FileCommandSignerResolver:
"""Resolve tenant/key-id scoped OpenClaw Ed25519 signing material."""
def __init__(self, path: str | Path, *, configured_key_id: str) -> None:
self.path = Path(path).resolve()
self.configured_key_id = str(configured_key_id or "").strip()
if not self.configured_key_id:
raise ValueError("configured_key_id is required")
def __call__(self, tenant_id: str, key_id: str) -> Ed25519CommandSigner:
if not str(tenant_id or "").strip():
raise RuntimeError("tenant_id is required for command signing")
if key_id != self.configured_key_id:
raise RuntimeError("OpenClaw signing key_id mismatch")
return Ed25519CommandSigner.from_bytes(
_read_protected_file(self.path, private=True),
key_id=key_id,
)
class ControlPlaneEventPublisher(EventPublisher):
"""Append Runtime callback events to the tenant audit stream.
Callback audit remains informational. Authoritative task reconciliation is
performed only by the orchestrator after dispatch/result validation.
"""
def __init__(self, service: DurableControlPlaneService) -> None:
self.service = service
def publish(self, event: Mapping[str, Any]) -> None:
tenant_id = str(event.get("tenant_id") or "").strip()
trace_id = str(event.get("trace_id") or "").strip()
command_id = str(event.get("command_id") or "").strip()
event_type = str(event.get("event_type") or "OPENCLAW_EVENT").strip()
if not tenant_id or not trace_id or not command_id:
raise ValueError("runtime audit event is missing tenant identity")
state_value = str(event.get("state") or "unknown").lower()
outcome = "allowed" if state_value == "completed" else state_value
runtime = str(event.get("runtime") or "openclaw").strip().lower()
principal_id = str(
event.get("principal_id") or f"{runtime}-runtime"
).strip()
audit = AuditEvent(
event_id=f"runtime_{command_id}_{len(self.service.store.audit_events) + 1}",
tenant_id=tenant_id,
principal_id=principal_id,
trace_id=trace_id,
action=event_type,
resource_type=f"{runtime}_execution",
resource_id=command_id,
outcome=outcome,
policy_version="runtime-v2",
details=dict(event.get("details") or {}),
)
with self.service.store.lock:
self.service.store.audit_events.append(audit)
self.service.repository.save(self.service._snapshot())
class CandidateApplication:
def __init__(
self,
gateway: GatewayApplication,
orchestrator: ExecutionOrchestrator,
tracker: ExecutionTracker,
hermes: HermesAdapter,
openclaw: OpenClawAdapter | None,
*,
hermes_lifecycle_api_enabled: bool = False,
) -> None:
self.gateway = gateway
self.orchestrator = orchestrator
self.tracker = tracker
self.hermes = hermes
self.openclaw = openclaw
self.hermes_lifecycle_api_enabled = bool(hermes_lifecycle_api_enabled)
self.runtime_events = ControlPlaneEventPublisher(orchestrator.control_plane)
def context(self, headers: Mapping[str, str]) -> RequestContext:
return self.gateway.context_from_headers(headers)
@staticmethod
def _require_runtime_role(context: RequestContext) -> None:
if not ({"runtime", "platform_admin"} & set(context.roles)):
raise Unauthorized("runtime reconciliation role is required")
@staticmethod
def _require_control_plane_cancel(context: RequestContext) -> None:
required = {"runtime", "platform_admin"}
if not required.issubset(set(context.roles)):
raise Unauthorized(
"Hermes cancellation requires trusted AIOS control-plane authority"
)
@staticmethod
def _required_identity(payload: Mapping[str, Any], name: str) -> str:
value = str(payload.get(name) or "").strip()
if not value:
raise ValueError(f"{name} is required")
return value
def _record_hermes_lifecycle(
self,
context: RequestContext,
dispatch: Any,
job: Any,
*,
action: str,
worker_id: str | None,
provenance_id: str,
run_id: str,
) -> None:
executor = worker_id or "aios-control-plane"
evidence = ExecutionEvidence(
execution_id=job.job_id,
tenant_id=job.tenant_id,
task_id=job.task_id,
agent_id=dispatch.agent_id,
executor=executor,
branch="runtime/hermes",
status=job.state.value.lower(),
progress_completed=0,
progress_total=3,
current_step=action,
provenance_id=provenance_id,
run_id=run_id,
)
current_evidence = self.tracker.claim(
evidence,
principal_id=context.principal_id,
provenance_id=provenance_id,
run_id=run_id,
)
if job.state in {
HermesJobState.COMPLETED,
HermesJobState.FAILED,
HermesJobState.CANCELLED,
}:
if current_evidence.completed_at is None:
self.tracker.complete(
tenant_id=job.tenant_id,
execution_id=job.job_id,
status=job.state.value.lower(),
test_status="runtime_terminal",
test_summary={"lifecycle_action": action},
result_summary={
"state": job.state.value,
"worker_id": worker_id,
"dispatch_id": dispatch.dispatch_id,
},
principal_id=context.principal_id,
provenance_id=provenance_id,
run_id=run_id,
)
else:
self.tracker.heartbeat(
tenant_id=job.tenant_id,
execution_id=job.job_id,
status=job.state.value.lower(),
current_step=action,
progress_completed=(1 if job.state == HermesJobState.RUNNING else 0),
progress_total=3,
principal_id=context.principal_id,
provenance_id=provenance_id,
run_id=run_id,
)
self.runtime_events.publish(
{
"runtime": "hermes",
"tenant_id": job.tenant_id,
"principal_id": context.principal_id,
"trace_id": job.trace_id,
"command_id": job.job_id,
"event_type": f"HERMES_{action.upper()}",
"state": job.state.value,
"details": {
"dispatch_id": dispatch.dispatch_id,
"task_id": job.task_id,
"worker_id": worker_id,
"cell_id": job.cell_id,
"profile_id": job.profile_id,
"revision": job.revision,
"provenance_id": provenance_id,
"run_id": run_id,
},
}
)
def handle_json(
self,
method: str,
raw_path: str,
headers: Mapping[str, str],
body: Any,
) -> tuple[int, Mapping[str, Any]]:
parsed = urllib.parse.urlparse(raw_path)
path = parsed.path.rstrip("/") or "/"
method = method.upper()
if method == "GET" and path == "/api/v2/gitops":
context = self.context(headers)
return HTTPStatus.OK, {
"tenant_id": context.tenant_id,
"gitops": collect_gitops_status().as_dict(),
}
if method == "GET" and path == "/api/v2/runtime":
context = self.context(headers)
runtimes: list[dict[str, Any]] = []
if self.openclaw is None:
runtimes.append(
{
"name": "openclaw",
"status": "disabled",
"details": {"reason": "runtime_not_configured"},
}
)
else:
try:
health = self.openclaw.health()
runtimes.append(
{
"name": "openclaw",
"status": health.status,
"details": _json_value(health),
}
)
except Exception as exc:
runtimes.append(
{
"name": "openclaw",
"status": "unavailable",
"details": {"message": str(exc)},
}
)
hermes_health = self.hermes.health()
runtimes.append(
{
"name": "hermes",
"status": hermes_health.status,
"details": _json_value(hermes_health),
}
)
return HTTPStatus.OK, {
"tenant_id": context.tenant_id,
"runtimes": runtimes,
}
if method == "GET" and path == "/api/v2/executions":
context = self.context(headers)
return HTTPStatus.OK, {
"executions": _json_value(
self.tracker.list(tenant_id=context.tenant_id)
)
}
dispatch_match = __import__("re").fullmatch(
r"/api/v2/tasks/([^/]+)/dispatch", path
)
if method == "POST" and dispatch_match:
context = self.context(headers)
task_id = urllib.parse.unquote(dispatch_match.group(1))
payload = body if isinstance(body, Mapping) else {}
timeout = payload.get("timeout_seconds")
provenance_id, run_id = _provenance_context(headers, payload)
record = self.orchestrator.dispatch(
context,
task_id=task_id,
timeout_seconds=float(timeout) if timeout is not None else None,
provenance_id=provenance_id,
run_id=run_id,
)
return HTTPStatus.ACCEPTED, {"dispatch": _json_value(record)}
lifecycle_match = __import__("re").fullmatch(
r"/api/v2/internal/runtime/hermes/lifecycle/"
r"(claim|start|heartbeat|complete|fail|cancel|status)",
path,
)
if method == "POST" and lifecycle_match:
if not self.hermes_lifecycle_api_enabled:
return HTTPStatus.NOT_FOUND, {
"error": "not_found",
"message": "route not found",
}
context = self.context(headers)
self._require_runtime_role(context)
if not isinstance(body, Mapping):
raise ValueError("JSON body must be an object")
payload = body
action = lifecycle_match.group(1)
identity = {
name: self._required_identity(payload, name)
for name in (
"dispatch_id",
"job_id",
"tenant_id",
"task_id",
"trace_id",
"principal_id",
"provenance_id",
"run_id",
"cell_id",
"profile_id",
)
}
header_provenance, header_run = _provenance_context(headers, payload)
if header_provenance != identity["provenance_id"]:
raise ValueError("provenance_id context mismatch")
if header_run != identity["run_id"]:
raise ValueError("run_id context mismatch")
requeued_job_ids = self.hermes.requeue_expired()
dispatch, job = self.orchestrator.hermes_lifecycle_context(
context,
dispatch_id=identity["dispatch_id"],
job_id=identity["job_id"],
tenant_id=identity["tenant_id"],
task_id=identity["task_id"],
trace_id=identity["trace_id"],
principal_id=identity["principal_id"],
provenance_id=identity["provenance_id"],
run_id=identity["run_id"],
)
if job.cell_id != identity["cell_id"]:
raise ValueError("cell_id mismatch")
if job.profile_id != identity["profile_id"]:
raise ValueError("profile_id mismatch")
if action == "status":
if job.job_id in requeued_job_ids:
self._record_hermes_lifecycle(
context,
dispatch,
job,
action="lease_expired_requeue",
worker_id=None,
provenance_id=identity["provenance_id"],
run_id=identity["run_id"],
)
task = self.orchestrator.control_plane.get_task(
context, dispatch.task_id
)
response: dict[str, Any] = {
"job": _json_value(job),
"dispatch": _json_value(dispatch),
"task": _json_value(task),
}
if job.state in {
HermesJobState.COMPLETED,
HermesJobState.FAILED,
HermesJobState.CANCELLED,
}:
response["result"] = _json_value(
self.hermes.result(job.job_id, tenant_id=job.tenant_id)
)
return HTTPStatus.OK, response
expected_revision = payload.get("expected_revision")
expected = (
int(expected_revision) if expected_revision is not None else None
)
if action == "cancel":
self._require_control_plane_cancel(context)
if payload.get("worker_id") is not None:
raise Unauthorized("Hermes workers may observe but not initiate cancel")
if payload.get("cancellation_source") != "aios-control-plane":
raise ValueError(
"cancellation_source must be aios-control-plane"
)
result = self.hermes.cancel(
job.job_id,
tenant_id=job.tenant_id,
expected_revision=expected,
)
else:
worker_id = self._required_identity(payload, "worker_id")
if action == "claim":
claimed = self.hermes.claim(
tenant_id=job.tenant_id,
cell_id=job.cell_id,
profile_id=job.profile_id,
worker_id=worker_id,
lease_seconds=(
int(payload["lease_seconds"])
if payload.get("lease_seconds") is not None
else None
),
job_id=job.job_id,
)
if claimed is None:
raise ValueError("Hermes job is not claimable")
bound = self.orchestrator.bind_hermes_worker(
context,
claimed,
dispatch_id=dispatch.dispatch_id,
provenance_id=identity["provenance_id"],
run_id=identity["run_id"],
worker_id=worker_id,
)
self._record_hermes_lifecycle(
context,
bound,
claimed,
action="claim",
worker_id=worker_id,
provenance_id=identity["provenance_id"],
run_id=identity["run_id"],
)
return HTTPStatus.OK, {
"job": _json_value(claimed),
"dispatch": _json_value(bound),
}
if action == "start":
started = self.hermes.start(
job.job_id,
tenant_id=job.tenant_id,
worker_id=worker_id,
expected_revision=expected,
)
self._record_hermes_lifecycle(
context,
dispatch,
started,
action="start",
worker_id=worker_id,
provenance_id=identity["provenance_id"],
run_id=identity["run_id"],
)
return HTTPStatus.OK, {"job": _json_value(started)}
if action == "heartbeat":
heartbeat = self.hermes.heartbeat(
job.job_id,
tenant_id=job.tenant_id,
worker_id=worker_id,
lease_seconds=(
int(payload["lease_seconds"])
if payload.get("lease_seconds") is not None
else None
),
)
self._record_hermes_lifecycle(
context,
dispatch,
heartbeat,
action="heartbeat",
worker_id=worker_id,
provenance_id=identity["provenance_id"],
run_id=identity["run_id"],
)
return HTTPStatus.OK, {"job": _json_value(heartbeat)}
if action == "complete":
output = payload.get("output")
if not isinstance(output, Mapping):
raise ValueError("output must be an object")
result = self.hermes.complete(
job.job_id,
tenant_id=job.tenant_id,
worker_id=worker_id,
output=output,
expected_revision=expected,
)
elif action == "fail":
error = payload.get("error")
if not isinstance(error, Mapping):
raise ValueError("error must be an object")
result = self.hermes.fail(
job.job_id,
tenant_id=job.tenant_id,
worker_id=worker_id,
error=error,
expected_revision=expected,
)
else:
raise ValueError("unsupported Hermes lifecycle action")
task = self.orchestrator.reconcile_hermes(
context,
result,
dispatch_id=dispatch.dispatch_id,
provenance_id=identity["provenance_id"],
run_id=identity["run_id"],
)
updated_dispatch = self.orchestrator.dispatch_repository.get(
dispatch.dispatch_id, tenant_id=dispatch.tenant_id
)
terminal_job = self.hermes.get(job.job_id, tenant_id=job.tenant_id)
self._record_hermes_lifecycle(
context,
updated_dispatch,
terminal_job,
action=action,
worker_id=result.worker_id,
provenance_id=identity["provenance_id"],
run_id=identity["run_id"],
)
return HTTPStatus.OK, {
"result": _json_value(result),
"task": _json_value(task),
"dispatch": _json_value(updated_dispatch),
}
if method == "POST" and path == "/api/v2/internal/runtime/hermes/reconcile":
context = self.context(headers)
self._require_runtime_role(context)
if not isinstance(body, Mapping):
raise ValueError("JSON body must be an object")
provenance_id, run_id = _provenance_context(headers, body)
if not provenance_id or not run_id:
raise ValueError("provenance_id and run_id are required")
dispatch_id = str(body.get("dispatch_id") or "").strip()
if not dispatch_id:
raise ValueError("dispatch_id is required")
result = HermesResult(
job_id=str(body["job_id"]),
tenant_id=str(body["tenant_id"]),
task_id=str(body["task_id"]),
trace_id=str(body["trace_id"]),
state=HermesJobState(str(body["state"])),
worker_id=(
str(body["worker_id"])
if body.get("worker_id") is not None
else None
),
output=body.get("output"),
error=body.get("error"),
started_at=body.get("started_at"),
finished_at=body.get("finished_at"),
)
task = self.orchestrator.reconcile_hermes(
context,
result,
dispatch_id=dispatch_id,
provenance_id=provenance_id,
run_id=run_id,
)
return HTTPStatus.OK, {"task": _json_value(task)}
if method == "POST" and path == "/api/v2/hermes/results/reconcile":
return HTTPStatus.GONE, {
"error": "RuntimeReconcileEndpointMoved",
"message": "use /api/v2/internal/runtime/hermes/reconcile",
}
return self.gateway.handle(method, raw_path, headers, body)
class CandidateRequestHandler(BaseHTTPRequestHandler):
server_version = "AIOSV2Candidate/2.0"
max_body_bytes = 1_048_576
@property
def application(self) -> CandidateApplication:
return self.server.application # type: ignore[attr-defined]
def _read_body(self) -> Any:
length = int(self.headers.get("Content-Length", "0") or "0")
if length < 0 or length > self.max_body_bytes:
raise ValueError("request body is too large")
if length == 0:
return {}
content_type = self.headers.get("Content-Type", "").split(";", 1)[0].strip()
if content_type != "application/json":
raise ValueError("Content-Type must be application/json")
return json.loads(self.rfile.read(length).decode("utf-8"))
def _security_headers(self) -> None:
self.send_header("Cache-Control", "no-store")
self.send_header("X-Content-Type-Options", "nosniff")
self.send_header("X-Frame-Options", "DENY")
self.send_header("Referrer-Policy", "no-referrer")
self.send_header(
"Content-Security-Policy",
"default-src 'self'; style-src 'unsafe-inline'; script-src 'unsafe-inline'; connect-src 'self'",
)
def _send_json(self, status_code: int, payload: Mapping[str, Any]) -> None:
encoded = json.dumps(
_json_value(payload), ensure_ascii=False, separators=(",", ":")
).encode("utf-8")
self.send_response(int(status_code))
self.send_header("Content-Type", "application/json; charset=utf-8")
self.send_header("Content-Length", str(len(encoded)))
self._security_headers()
self.end_headers()
self.wfile.write(encoded)
def _send_console(self) -> None:
encoded = CONSOLE_HTML.encode("utf-8")
self.send_response(HTTPStatus.OK)
self.send_header("Content-Type", "text/html; charset=utf-8")
self.send_header("Content-Length", str(len(encoded)))
self._security_headers()
self.end_headers()
self.wfile.write(encoded)
def _dispatch(self, method: str) -> None:
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
if method == "GET" and path in {"/", "/console"}:
self._send_console()
return
try:
body = self._read_body() if method == "POST" else {}
status_code, payload = self.application.handle_json(
method,
self.path,
{key: value for key, value in self.headers.items()},
body,
)
except Exception as exc:
status_code, payload = error_response(exc)
self._send_json(status_code, payload)
def do_GET(self) -> None: # noqa: N802
self._dispatch("GET")
def do_POST(self) -> None: # noqa: N802
self._dispatch("POST")
class CandidateHTTPServer(ThreadingHTTPServer):
daemon_threads = True
allow_reuse_address = True
def __init__(self, address: tuple[str, int], application: CandidateApplication) -> None:
self.application = application
super().__init__(address, CandidateRequestHandler)
def build_application() -> CandidateApplication:
state_dir = Path(os.getenv("AIOS_V2_STATE_DIR", "/var/lib/aios-v2")).resolve()
state_dir.mkdir(parents=True, exist_ok=True)
repository = SQLiteSnapshotRepository(state_dir / "control-plane.sqlite3")
dispatch_repository = SQLiteDispatchRepository(state_dir / "dispatches.sqlite3")
service = DurableControlPlaneService(repository)
require_provenance = _enabled(os.getenv("AIOS_REQUIRE_RUN_PROVENANCE", "0"))
provenance_writer = build_provenance_writer_from_env(
state_dir=state_dir, required=require_provenance
)
tracker = ExecutionTracker(
state_dir / "executions",
provenance_writer=provenance_writer,
require_provenance=require_provenance,
)
hermes = HermesAdapter(store=SQLiteHermesStore(state_dir / "hermes.sqlite3"))
openclaw_url = os.getenv("AIOS_OPENCLAW_URL", "").strip()
signing_file = os.getenv("AIOS_OPENCLAW_SIGNING_MATERIAL_FILE", "").strip()
signing_key_id = os.getenv(
"AIOS_OPENCLAW_SIGNING_KEY_ID", "openclaw-v2r1-candidate-ed25519"
).strip()
openclaw = None
resolver = None
if openclaw_url and signing_file:
resolver = FileCommandSignerResolver(
signing_file,
configured_key_id=signing_key_id,
)
transport = HTTPRuntimeTransport(
openclaw_url,
credential_ref=os.getenv("AIOS_OPENCLAW_CREDENTIAL_REF") or None,
)
openclaw = OpenClawAdapter(
transport,
resolver,
callback_sink=AuditCallbackSink(ControlPlaneEventPublisher(service)),
)
gateway = GatewayApplication(service)
orchestrator = ExecutionOrchestrator(
service,
openclaw=openclaw,
hermes=hermes,
openclaw_signing_material_resolver=resolver,
openclaw_key_id=signing_key_id,
provenance_writer=provenance_writer,
require_provenance=require_provenance,
dispatch_repository=dispatch_repository,
)
return CandidateApplication(
gateway,
orchestrator,
tracker,
hermes,
openclaw,
hermes_lifecycle_api_enabled=_enabled(
os.getenv("AIOS_HERMES_LIFECYCLE_API_ENABLED", "0")
),
)
def main() -> None:
host = os.getenv("AIOS_V2_HOST", "127.0.0.1")
port = int(os.getenv("AIOS_V2_PORT", "18920"))
server = CandidateHTTPServer((host, port), build_application())
print(f"AIOS V2 candidate listening on {host}:{port}")
server.serve_forever()
if __name__ == "__main__":
main()
-83
View File
@@ -1,83 +0,0 @@
"""Dependency-free, role-aware AIOS unified portal.
The browser receives only server-authorized capabilities and tenant-scoped data.
It never selects a tenant, signs Provenance, mutates Git tags, or calls Runtime
services directly.
"""
CONSOLE_HTML = r'''<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="referrer" content="no-referrer">
<title>AIOS Unified Portal</title>
<style>
:root{font-family:Inter,"PingFang SC","Microsoft YaHei",ui-sans-serif,system-ui,sans-serif;color:#172033;background:#eef2f7;--nav:#0f172a;--card:#fff;--line:#dfe6ef;--muted:#667085;--blue:#2563eb;--green:#047857;--red:#b42318;--amber:#b54708}
*{box-sizing:border-box}body{margin:0;min-height:100vh}button,input,textarea,select{font:inherit}
.shell{display:grid;grid-template-columns:250px minmax(0,1fr);min-height:100vh}.sidebar{background:var(--nav);color:#e2e8f0;padding:22px 16px;position:sticky;top:0;height:100vh;overflow:auto}.brand{font-weight:800;font-size:20px;letter-spacing:.2px}.version{font-size:12px;color:#94a3b8;margin:5px 0 22px}.identity{padding:12px;background:#182338;border:1px solid #2b3950;border-radius:12px;margin-bottom:18px}.identity strong{display:block;color:#fff}.identity span{font-size:12px;color:#aebbd0;display:block;margin-top:3px;overflow-wrap:anywhere}.nav button{width:100%;display:flex;gap:10px;align-items:center;border:0;background:transparent;color:#cbd5e1;padding:11px 12px;border-radius:9px;text-align:left;cursor:pointer;margin:2px 0}.nav button:hover,.nav button.active{background:#24324b;color:#fff}.nav .dot{width:8px;height:8px;border-radius:50%;background:#60a5fa}.main{min-width:0}.topbar{height:70px;background:#fff;border-bottom:1px solid var(--line);display:flex;align-items:center;justify-content:space-between;padding:0 26px;position:sticky;top:0;z-index:4}.topbar h1{font-size:19px;margin:0}.top-actions{display:flex;align-items:center;gap:10px}.timestamp{font-size:12px;color:var(--muted)}.content{padding:24px;max-width:1680px;margin:auto}.notice{background:#eff6ff;border:1px solid #bfdbfe;border-radius:12px;padding:12px 14px;color:#1e3a8a;font-size:13px;margin-bottom:18px}.error{background:#fef3f2;border:1px solid #fecdca;color:var(--red);border-radius:10px;padding:10px 12px;white-space:pre-wrap;margin-bottom:14px;display:none}.cards{display:grid;grid-template-columns:repeat(auto-fit,minmax(170px,1fr));gap:14px;margin-bottom:18px}.card,.panel{background:var(--card);border:1px solid var(--line);border-radius:14px;box-shadow:0 1px 3px rgba(16,24,40,.05)}.card{padding:16px}.card small{color:var(--muted);font-weight:600}.card b{display:block;font-size:28px;margin-top:7px}.panel{padding:17px;margin-bottom:16px;overflow:auto}.panel h2{font-size:16px;margin:0}.panel-head{display:flex;align-items:center;justify-content:space-between;gap:12px;margin-bottom:13px}.grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(410px,1fr));gap:16px}.wide{grid-column:1/-1}.muted{color:var(--muted)}.good{color:var(--green);font-weight:700}.bad{color:var(--red);font-weight:700}.warn{color:var(--amber);font-weight:700}.pill{display:inline-block;border:1px solid #cbd5e1;border-radius:999px;padding:3px 8px;font-size:11px;margin:2px;background:#f8fafc}.pill.good{background:#ecfdf3;border-color:#a6f4c5}.pill.bad{background:#fef3f2;border-color:#fecdca}.toolbar{display:flex;gap:8px;align-items:center;flex-wrap:wrap}.btn{border:0;border-radius:9px;padding:9px 12px;background:var(--blue);color:#fff;cursor:pointer;font-weight:650}.btn.secondary{background:#fff;color:#344054;border:1px solid #d0d5dd}.btn.danger{background:#d92d20}.btn:disabled{opacity:.45;cursor:not-allowed}.icon-btn{padding:7px 10px}.view{display:none}.view.active{display:block}table{width:100%;border-collapse:collapse;font-size:12px}th,td{text-align:left;padding:9px 8px;border-bottom:1px solid #edf0f4;vertical-align:top}th{color:#667085;font-weight:700;white-space:nowrap}td{overflow-wrap:anywhere}.empty{padding:24px;text-align:center;color:var(--muted)}.timeline{border-left:2px solid #dbe4f0;margin-left:8px;padding-left:20px}.event{position:relative;padding:0 0 17px}.event:before{content:"";position:absolute;left:-27px;top:4px;width:11px;height:11px;border-radius:50%;background:#3b82f6;border:3px solid #fff;box-shadow:0 0 0 1px #93c5fd}.event strong{font-size:13px}.event p{margin:4px 0;color:var(--muted);font-size:12px}.chain{display:flex;align-items:stretch;gap:7px;overflow:auto;padding:5px 0}.chain-item{min-width:135px;border:1px solid var(--line);border-radius:10px;padding:10px;background:#f8fafc}.chain-arrow{display:flex;align-items:center;color:#94a3b8}.form-grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(230px,1fr));gap:12px}.field label{font-size:12px;color:#475467;font-weight:650;display:block;margin-bottom:5px}.field input,.field textarea,.field select{width:100%;border:1px solid #d0d5dd;border-radius:9px;padding:9px;background:#fff}.field textarea{min-height:90px;resize:vertical}.dialog{border:0;border-radius:14px;padding:0;max-width:650px;width:calc(100% - 32px);box-shadow:0 20px 60px rgba(15,23,42,.28)}.dialog::backdrop{background:rgba(15,23,42,.6)}.dialog-head{padding:16px 18px;border-bottom:1px solid var(--line);font-weight:750}.dialog-body{padding:18px}.dialog-actions{padding:14px 18px;border-top:1px solid var(--line);display:flex;justify-content:flex-end;gap:8px}.kv{display:grid;grid-template-columns:150px minmax(0,1fr);gap:6px 12px;font-size:12px}.kv div:nth-child(odd){color:var(--muted)}pre{white-space:pre-wrap;word-break:break-word;font-size:11px;background:#0f172a;color:#dbeafe;border-radius:10px;padding:12px;max-height:300px;overflow:auto}
@media(max-width:900px){.shell{grid-template-columns:1fr}.sidebar{height:auto;position:static}.nav{display:flex;overflow:auto}.nav button{min-width:max-content}.topbar{position:static}.grid{grid-template-columns:1fr}.content{padding:16px}}
</style>
</head>
<body>
<div class="shell">
<aside class="sidebar">
<div class="brand">AIOS Unified Portal</div><div class="version">V2R1C1 · Role-aware Control Plane</div>
<div class="identity"><strong id="principal">加载身份…</strong><span id="tenant"></span><span id="roles"></span></div>
<nav id="navigation" class="nav"></nav>
</aside>
<section class="main">
<header class="topbar"><h1 id="pageTitle">系统总览</h1><div class="top-actions"><span id="updated" class="timestamp">尚未加载</span><button class="btn secondary icon-btn" id="refresh">刷新</button></div></header>
<main class="content">
<div class="notice">身份、租户和角色由可信反向代理注入。门户只调用 AIOS Control Plane;Provenance 签发、Tag、生产部署与 Runtime 权威状态仍由受控流程执行。</div>
<div id="error" class="error"></div>
<section id="overview" class="view active"></section>
<section id="tasks" class="view"></section>
<section id="authorization" class="view"></section>
<section id="runtime" class="view"></section>
<section id="evidence" class="view"></section>
<section id="audit" class="view"></section>
<section id="release" class="view"></section>
</main>
</section>
</div>
<dialog id="taskDialog" class="dialog">
<div class="dialog-head">创建任务</div><form id="taskForm"><div class="dialog-body form-grid">
<div class="field"><label>Agent ID</label><input name="agent_id" required></div>
<div class="field"><label>Action</label><input name="action" required></div>
<div class="field"><label>Idempotency Key</label><input name="idempotency_key" required></div>
<div class="field wide"><label>Payload JSON</label><textarea name="payload">{}</textarea></div>
</div><div class="dialog-actions"><button type="button" class="btn secondary" data-close="taskDialog">取消</button><button class="btn">创建</button></div></form>
</dialog>
<dialog id="confirmDialog" class="dialog"><div class="dialog-head" id="confirmTitle">确认操作</div><form id="confirmForm"><div class="dialog-body"><p id="confirmText"></p><div class="field"><label>原因</label><textarea id="confirmReason" required></textarea></div></div><div class="dialog-actions"><button type="button" class="btn secondary" data-close="confirmDialog">取消</button><button class="btn danger">确认</button></div></form></dialog>
<script>
const $=s=>document.querySelector(s), esc=v=>String(v??'').replace(/[&<>"']/g,c=>({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;',"'":'&#39;'}[c]));
const state={identity:null,current:'overview',cache:{},confirm:null};
const labels={overview:'系统总览',tasks:'任务中心',authorization:'授权中心',runtime:'运行与运维',evidence:'执行证据',audit:'审计中心',release:'GitOps 与发布'};
function has(c){return !!state.identity?.capabilities?.includes(c)}
async function api(path,options={}){const r=await fetch(path,{credentials:'same-origin',headers:{'Accept':'application/json','Content-Type':'application/json',...(options.headers||{})},...options});const d=await r.json().catch(()=>({}));if(!r.ok)throw new Error(`${r.status} ${d.message||d.error||path}`);return d}
function err(e){const el=$('#error');el.textContent=e?.message||String(e);el.style.display='block'}function clearErr(){$('#error').style.display='none';$('#error').textContent=''}
function pill(v){const s=String(v??'unknown'),c=/complete|pass|healthy|active|synced|approved|used/i.test(s)?'good':/fail|deny|reject|revoke|expire|cancel|unavailable/i.test(s)?'bad':'';return `<span class="pill ${c}">${esc(s)}</span>`}
function table(cols,rows){if(!rows?.length)return '<div class="empty">暂无数据</div>';return `<table><thead><tr>${cols.map(x=>`<th>${esc(x[0])}</th>`).join('')}</tr></thead><tbody>${rows.map(r=>`<tr>${cols.map(x=>`<td>${x[1](r)}</td>`).join('')}</tr>`).join('')}</tbody></table>`}
function card(label,value,detail=''){return `<div class="card"><small>${esc(label)}</small><b>${esc(value)}</b>${detail?`<span class="muted">${esc(detail)}</span>`:''}</div>`}
function json(v){return `<pre>${esc(JSON.stringify(v,null,2))}</pre>`}
function show(view){state.current=view;document.querySelectorAll('.view').forEach(x=>x.classList.toggle('active',x.id===view));document.querySelectorAll('.nav button').forEach(x=>x.classList.toggle('active',x.dataset.view===view));$('#pageTitle').textContent=labels[view]||view;load(view)}
function navigation(){const nav=$('#navigation');nav.innerHTML=(state.identity.navigation||[]).map(x=>`<button data-view="${esc(x.id)}"><span class="dot"></span>${esc(x.label)}</button>`).join('');nav.querySelectorAll('button').forEach(b=>b.onclick=()=>show(b.dataset.view));}
async function identity(){state.identity=await api('/api/v2/me/capabilities');$('#principal').textContent=state.identity.principal_id;$('#tenant').textContent=`租户:${state.identity.tenant_id}`;$('#roles').textContent=`角色:${state.identity.roles.join(', ')||'无'}`;navigation();if(!state.identity.navigation.some(x=>x.id===state.current))state.current=state.identity.navigation[0]?.id||'overview';show(state.current)}
async function load(view){clearErr();try{if(view==='overview')await overview();if(view==='tasks')await tasks();if(view==='authorization')await authorizations();if(view==='runtime')await runtime();if(view==='evidence')await evidence();if(view==='audit')await audit();if(view==='release')await release();$('#updated').textContent=`更新于 ${new Date().toLocaleString()}`}catch(e){err(e)}}
async function overview(){const d=await api('/api/v2/portal/overview');state.cache.overview=d;const s=d.summary||{},rt=d.sections?.runtime?.runtimes||[],rel=d.sections?.release||{};$('#overview').innerHTML=`<div class="cards">${card('任务',s.tasks||0)}${card('活动 Dispatch',s.active_dispatches||0,`总数 ${s.dispatches||0}`)}${card('执行证据',s.evidence||0)}${card('审计事件',s.audit_events||0)}${card('当前版本',rel.version||'—',rel.status||'')}${card('授权 Grant',d.sections?.authorization?.active_grants||0,`总数 ${d.sections?.authorization?.grants||0}`)}</div><div class="grid"><div class="panel"><div class="panel-head"><h2>Runtime 健康</h2></div>${table([['Runtime',x=>esc(x.name)],['状态',x=>pill(x.status)],['版本/详情',x=>esc(x.details?.version||x.details?.runtime_id||'—')]],rt)}</div><div class="panel"><div class="panel-head"><h2>任务状态</h2></div>${table([['状态',x=>pill(x[0])],['数量',x=>esc(x[1])]],Object.entries(s.task_states||{}))}</div><div class="panel wide"><div class="panel-head"><h2>最近任务</h2></div>${table([['任务',x=>esc(x.task_id)],['Agent',x=>esc(x.agent_id)],['动作',x=>esc(x.action)],['状态',x=>pill(x.state)],['更新时间',x=>esc(x.updated_at)]],d.recent?.tasks||[])}</div></div>`}
async function tasks(){const d=await api('/api/v2/portal/tasks');state.cache.tasks=d.items||[];const create=has('task.create')?'<button class="btn" id="newTask">创建任务</button>':'';$('#tasks').innerHTML=`<div class="panel"><div class="panel-head"><h2>任务全链路</h2><div class="toolbar">${create}</div></div>${(d.items||[]).map(taskBlock).join('')||'<div class="empty">暂无任务</div>'}</div>`;$('#newTask')?.addEventListener('click',()=>$('#taskDialog').showModal());document.querySelectorAll('[data-task-action]').forEach(b=>b.onclick=()=>taskAction(b.dataset.taskId,b.dataset.taskAction,b.dataset.revision))}
function taskBlock(x){const t=x.task||{},d=x.dispatch,e=x.evidence||[],a=x.audit||[],actions=(x.available_actions||[]).map(v=>`<button class="btn ${v==='cancel'?'danger':'secondary'}" data-task-action="${v}" data-task-id="${esc(t.task_id)}" data-revision="${esc(t.revision)}">${{approve:'审批',cancel:'取消',dispatch:'调度'}[v]||v}</button>`).join('');return `<div class="panel" style="background:#fbfcfe"><div class="panel-head"><h2>${esc(t.task_id)} · ${esc(t.action)}</h2><div class="toolbar">${pill(t.state)}${actions}</div></div><div class="chain"><div class="chain-item"><b>Task</b><div>${esc(t.agent_id)}</div><small>${esc(t.created_by)}</small></div><span class="chain-arrow">→</span><div class="chain-item"><b>Approval</b><div>${esc(t.approved_by||'待审批')}</div><small>revision ${esc(t.revision)}</small></div><span class="chain-arrow">→</span><div class="chain-item"><b>Provenance</b><div>${esc(d?.provenance_id||'未调度')}</div><small>${esc(d?.authorization_id||'')}</small></div><span class="chain-arrow">→</span><div class="chain-item"><b>Dispatch</b><div>${esc(d?.dispatch_id||'—')}</div><small>${pill(d?.state||'not_created')}</small></div><span class="chain-arrow">→</span><div class="chain-item"><b>Runtime</b><div>${esc(d?.runtime||'—')}</div><small>${esc(d?.runtime_execution_id||'')}</small></div><span class="chain-arrow">→</span><div class="chain-item"><b>Evidence</b><div>${e.length} 条</div><small>Audit ${a.length} 条</small></div></div></div>`}
async function taskAction(id,action,revision){clearErr();try{let body={expected_revision:Number(revision)};if(action==='dispatch'){body={};const p=prompt('Provenance ID(生产调度必填)','');const r=prompt('Run ID(生产调度必填)','');if(p)body.provenance_id=p;if(r)body.run_id=r}await api(`/api/v2/portal/tasks/${encodeURIComponent(id)}/${action}`,{method:'POST',body:JSON.stringify(body)});await tasks()}catch(e){err(e)}}
async function authorizations(){const d=await api('/api/v2/portal/authorizations');state.cache.authorization=d;const workflows=(d.workflows||[]),grants=(d.grants||[]);$('#authorization').innerHTML=`<div class="cards">${card('Workflow',d.summary?.workflows||0)}${card('Grant',d.summary?.grants||0)}${card('有效 Grant',d.summary?.active_grants||0)}${card('已撤销',d.summary?.revoked_grants||0)}</div><div class="panel"><div class="panel-head"><h2>Workflow Provenance</h2></div>${table([['ID',x=>esc(x.provenance_id)],['Run / Task',x=>`${esc(x.run_id)}<br>${esc(x.task_id)}`],['Principal / Agent',x=>`${esc(x.principal_id)}<br>${esc(x.agent_id)}`],['状态',x=>pill(x.status)],['有效期',x=>esc(x.expires_at)],['使用',x=>esc(x.usage_count)],['操作',x=>has('provenance.revoke')&&x.status!=='revoked'?`<button class="btn danger" data-revoke-workflow="${esc(x.provenance_id)}">撤销</button>`:'—']],workflows)}</div><div class="panel"><div class="panel-head"><h2>Child Grants</h2></div>${table([['Grant',x=>esc(x.grant_id)],['类型 / Scope',x=>`${esc(x.grant_type)}<br>${esc(x.execution_path)}`],['Git 操作',x=>esc((x.allowed_git_operations||[]).join(', ')||'—')],['状态',x=>pill(x.status)],['有效期',x=>esc(x.expires_at)],['使用',x=>esc(x.usage_count)],['操作',x=>has('provenance.revoke')&&x.status!=='revoked'?`<button class="btn danger" data-revoke-grant="${esc(x.grant_id)}">撤销</button>`:'—']],grants)}</div>`;document.querySelectorAll('[data-revoke-workflow]').forEach(b=>b.onclick=()=>confirmOp('撤销 Workflow',`撤销 ${b.dataset.revokeWorkflow} 将使所有子 Grant 失效。`,reason=>api(`/api/v2/portal/provenance/${encodeURIComponent(b.dataset.revokeWorkflow)}/revoke`,{method:'POST',body:JSON.stringify({reason})}).then(authorizations)));document.querySelectorAll('[data-revoke-grant]').forEach(b=>b.onclick=()=>confirmOp('撤销 Grant',`撤销 ${b.dataset.revokeGrant}。`,reason=>api(`/api/v2/portal/grants/${encodeURIComponent(b.dataset.revokeGrant)}/revoke`,{method:'POST',body:JSON.stringify({reason})}).then(authorizations)))}
async function runtime(){const d=await api('/api/v2/portal/runtime');const runtimes=d.runtimes||[],dispatches=d.dispatches||[],jobs=d.hermes_jobs||[];$('#runtime').innerHTML=`<div class="grid"><div class="panel"><div class="panel-head"><h2>Runtime</h2></div>${table([['名称',x=>esc(x.name)],['状态',x=>pill(x.status)],['详情',x=>esc(JSON.stringify(x.details||{}))]],runtimes)}</div><div class="panel"><div class="panel-head"><h2>Hermes Queue / Lease</h2></div>${table([['Job',x=>esc(x.job_id)],['Cell / Profile',x=>`${esc(x.cell_id)} / ${esc(x.profile_id)}`],['Worker',x=>esc(x.assigned_worker_id||'—')],['状态',x=>pill(x.state)],['Lease',x=>esc(x.lease_expires_at||'—')]],jobs)}</div><div class="panel wide"><div class="panel-head"><h2>Dispatch</h2></div>${table([['Dispatch',x=>esc(x.dispatch_id)],['Task',x=>esc(x.task_id)],['Runtime',x=>esc(x.runtime)],['执行 ID',x=>esc(x.runtime_execution_id)],['Principal',x=>esc(x.principal_id)],['状态',x=>pill(x.state)]],dispatches)}</div></div>`}
async function evidence(){const d=await api('/api/v2/portal/evidence');$('#evidence').innerHTML=`<div class="panel"><div class="panel-head"><h2>Execution Evidence</h2></div>${table([['Execution',x=>esc(x.execution_id)],['Task / Agent',x=>`${esc(x.task_id)}<br>${esc(x.agent_id)}`],['Executor',x=>esc(x.executor)],['状态',x=>pill(x.status)],['进度',x=>`${esc(x.progress_completed)}/${esc(x.progress_total)}`],['Test',x=>pill(x.test_status)],['Commit',x=>esc(x.last_commit||'—')],['授权',x=>esc(x.last_authorization_id||'—')]],d.executions||[])}</div>`}
async function audit(){const d=await api('/api/v2/portal/audit');const events=(d.events||[]).slice().reverse();$('#audit').innerHTML=`<div class="panel"><div class="panel-head"><h2>统一审计时间线</h2></div><div class="timeline">${events.map(x=>`<div class="event"><strong>${esc(x.action)} · ${pill(x.outcome)}</strong><p>${esc(x.created_at)} · ${esc(x.principal_id)} · ${esc(x.resource_type)}/${esc(x.resource_id)}</p><div class="muted">Trace ${esc(x.trace_id)} · Policy ${esc(x.policy_version)}</div></div>`).join('')||'<div class="empty">暂无审计事件</div>'}</div></div>`}
async function release(){const d=await api('/api/v2/portal/release');const g=d.gitops||{};$('#release').innerHTML=`<div class="cards">${card('版本',d.version||'—')}${card('Baseline',d.baseline||'—')}${card('发布状态',d.status||'—')}${card('Git 同步',g.sync_status||'unknown')}</div><div class="grid"><div class="panel"><div class="panel-head"><h2>当前 Release</h2></div><div class="kv"><div>Release OID</div><div>${esc(d.release_oid)}</div><div>Branch</div><div>${esc(g.workspace_branch||g.branch||'—')}</div><div>Workspace</div><div>${pill(g.workspace_clean===true?'clean':g.workspace_clean===false?'dirty':'unknown')}</div><div>Local Sync</div><div>${pill(g.local_sync||'unknown')}</div><div>Gitea Sync</div><div>${pill(g.gitea_sync||'unknown')}</div></div></div><div class="panel"><div class="panel-head"><h2>治理操作</h2></div><p class="muted">门户不直接移动 Tag、合并主线或部署生产。以下仅为可发起的 GitOps 请求目录。</p>${(d.operations||[]).map(x=>pill(x)).join('')||'<span class="muted">当前角色只读</span>'}</div><div class="panel wide"><div class="panel-head"><h2>原始 GitOps 状态</h2></div>${json(g)}</div></div>`}
function confirmOp(title,text,fn){state.confirm=fn;$('#confirmTitle').textContent=title;$('#confirmText').textContent=text;$('#confirmReason').value='';$('#confirmDialog').showModal()}
$('#taskForm').onsubmit=async e=>{e.preventDefault();clearErr();try{const fd=new FormData(e.target),payload=JSON.parse(fd.get('payload')||'{}');await api('/api/v2/portal/tasks',{method:'POST',body:JSON.stringify({agent_id:fd.get('agent_id'),action:fd.get('action'),idempotency_key:fd.get('idempotency_key'),payload})});$('#taskDialog').close();e.target.reset();await tasks()}catch(e){err(e)}};
$('#confirmForm').onsubmit=async e=>{e.preventDefault();const fn=state.confirm;$('#confirmDialog').close();state.confirm=null;try{await fn?.($('#confirmReason').value)}catch(e){err(e)}};
document.querySelectorAll('[data-close]').forEach(b=>b.onclick=()=>document.getElementById(b.dataset.close).close());$('#refresh').onclick=()=>load(state.current);
identity().catch(err);setInterval(()=>load(state.current),30000);
</script>
</body></html>'''
-627
View File
@@ -1,627 +0,0 @@
"""Tenant-isolated AIOS V2 control-plane domain service.
This module deliberately contains no OpenClaw or Hermes implementation details.
It owns authoritative tenant, agent, task-lifecycle, idempotency and audit state.
Runtime adapters consume approved tasks through a separate boundary.
"""
from __future__ import annotations
import copy
import hashlib
import json
import threading
import uuid
from dataclasses import dataclass, field
from datetime import datetime, timezone
from enum import Enum
from typing import Any, Callable, Iterable, Mapping
def _utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _new_id(prefix: str) -> str:
return f"{prefix}_{uuid.uuid4().hex}"
def _canonical_digest(value: Mapping[str, Any]) -> str:
encoded = json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
default=str,
).encode("utf-8")
return hashlib.sha256(encoded).hexdigest()
class ControlPlaneError(RuntimeError):
"""Base error returned by the control-plane domain."""
class Unauthorized(ControlPlaneError):
"""The caller is not authenticated or lacks the required role."""
class TenantIsolationError(ControlPlaneError):
"""A request attempted to cross a tenant boundary."""
class NotFound(ControlPlaneError):
"""A tenant-scoped resource does not exist or is not visible."""
class InvalidTransition(ControlPlaneError):
"""A task state transition violates the lifecycle contract."""
class IdempotencyConflict(ControlPlaneError):
"""An idempotency key was reused for a different command."""
class RevisionConflict(ControlPlaneError):
"""Optimistic task revision does not match current state."""
class PolicyDenied(ControlPlaneError):
"""The policy gate denied a control-plane operation."""
@dataclass(frozen=True)
class RequestContext:
"""Authenticated request identity propagated across all AIOS layers."""
tenant_id: str
principal_id: str
trace_id: str
roles: frozenset[str] = field(default_factory=frozenset)
session_id: str | None = None
policy_version: str = "v1"
def __post_init__(self) -> None:
for name in ("tenant_id", "principal_id", "trace_id"):
if not str(getattr(self, name, "")).strip():
raise ValueError(f"{name} is required")
@classmethod
def create(
cls,
*,
tenant_id: str,
principal_id: str,
roles: Iterable[str] = (),
trace_id: str | None = None,
session_id: str | None = None,
policy_version: str = "v1",
) -> "RequestContext":
return cls(
tenant_id=tenant_id.strip(),
principal_id=principal_id.strip(),
trace_id=trace_id or _new_id("trace"),
roles=frozenset(str(role).strip() for role in roles if str(role).strip()),
session_id=session_id,
policy_version=policy_version,
)
@dataclass(frozen=True)
class TenantRecord:
tenant_id: str
display_name: str
status: str = "active"
metadata: Mapping[str, Any] = field(default_factory=dict)
created_at: str = field(default_factory=_utc_now)
updated_at: str = field(default_factory=_utc_now)
@dataclass(frozen=True)
class AgentRecord:
agent_id: str
tenant_id: str
display_name: str
runtime: str
capabilities: tuple[str, ...] = field(default_factory=tuple)
enabled: bool = True
metadata: Mapping[str, Any] = field(default_factory=dict)
created_at: str = field(default_factory=_utc_now)
updated_at: str = field(default_factory=_utc_now)
class TaskState(str, Enum):
CREATED = "CREATED"
APPROVED = "APPROVED"
RUNNING = "RUNNING"
COMPLETED = "COMPLETED"
FAILED = "FAILED"
CANCELLED = "CANCELLED"
TERMINAL_TASK_STATES = frozenset(
{TaskState.COMPLETED, TaskState.FAILED, TaskState.CANCELLED}
)
ALLOWED_TASK_TRANSITIONS: Mapping[TaskState, frozenset[TaskState]] = {
TaskState.CREATED: frozenset({TaskState.APPROVED, TaskState.CANCELLED}),
TaskState.APPROVED: frozenset({TaskState.RUNNING, TaskState.CANCELLED}),
TaskState.RUNNING: frozenset(
{TaskState.COMPLETED, TaskState.FAILED, TaskState.CANCELLED}
),
TaskState.COMPLETED: frozenset(),
TaskState.FAILED: frozenset(),
TaskState.CANCELLED: frozenset(),
}
@dataclass(frozen=True)
class TaskRecord:
task_id: str
tenant_id: str
agent_id: str
action: str
payload: Mapping[str, Any]
state: TaskState = TaskState.CREATED
revision: int = 1
idempotency_key: str | None = None
request_digest: str = ""
result: Mapping[str, Any] | None = None
error: Mapping[str, Any] | None = None
created_by: str = ""
approved_by: str | None = None
created_at: str = field(default_factory=_utc_now)
updated_at: str = field(default_factory=_utc_now)
@dataclass(frozen=True)
class AuditEvent:
event_id: str
tenant_id: str
principal_id: str
trace_id: str
action: str
resource_type: str
resource_id: str
outcome: str
policy_version: str
details: Mapping[str, Any] = field(default_factory=dict)
created_at: str = field(default_factory=_utc_now)
@dataclass(frozen=True)
class PolicyDecision:
allowed: bool
reason: str
policy_version: str
PolicyEvaluator = Callable[[RequestContext, str, Mapping[str, Any]], PolicyDecision]
class InMemoryControlPlaneStore:
"""Thread-safe reference store used by tests and the first gateway slice.
Production backends must preserve the same tenant-scoped interface and add
database row-level security, migrations, backup and HA evidence.
"""
def __init__(self) -> None:
self.lock = threading.RLock()
self.tenants: dict[str, TenantRecord] = {}
self.agents: dict[str, AgentRecord] = {}
self.tasks: dict[str, TaskRecord] = {}
self.idempotency: dict[tuple[str, str], tuple[str, str]] = {}
self.audit_events: list[AuditEvent] = []
class ControlPlaneService:
"""Authoritative tenant, agent and task lifecycle service."""
def __init__(
self,
store: InMemoryControlPlaneStore | None = None,
*,
policy_evaluator: PolicyEvaluator | None = None,
) -> None:
self.store = store or InMemoryControlPlaneStore()
self.policy_evaluator = policy_evaluator or self._default_policy
@staticmethod
def _default_policy(
context: RequestContext,
action: str,
resource: Mapping[str, Any],
) -> PolicyDecision:
del resource
role_requirements = {
"tenant.register": {"platform_admin"},
"tenant.read": {"tenant_admin", "operator", "auditor", "platform_admin"},
"agent.register": {"tenant_admin", "operator", "platform_admin"},
"agent.read": {"tenant_admin", "operator", "auditor", "platform_admin"},
"task.create": {"tenant_admin", "operator", "platform_admin"},
"task.read": {"tenant_admin", "operator", "auditor", "platform_admin"},
"task.approve": {"tenant_admin", "approver", "platform_admin"},
"task.start": {"runtime", "operator", "platform_admin"},
"task.complete": {"runtime", "operator", "platform_admin"},
"task.fail": {"runtime", "operator", "platform_admin"},
"task.cancel": {"tenant_admin", "operator", "approver", "platform_admin"},
"audit.read": {"tenant_admin", "auditor", "platform_admin"},
}
required = role_requirements.get(action, {"platform_admin"})
allowed = bool(context.roles.intersection(required))
return PolicyDecision(
allowed=allowed,
reason="role_allowed" if allowed else "required_role_missing",
policy_version=context.policy_version,
)
def _require_context(self, context: RequestContext) -> None:
if not context.principal_id or not context.tenant_id or not context.trace_id:
raise Unauthorized("authenticated tenant context is required")
def _authorize(
self,
context: RequestContext,
action: str,
resource: Mapping[str, Any],
) -> PolicyDecision:
self._require_context(context)
decision = self.policy_evaluator(context, action, resource)
if not decision.allowed:
self._audit(
context,
action=action,
resource_type=str(resource.get("resource_type", "policy")),
resource_id=str(resource.get("resource_id", "unknown")),
outcome="denied",
details={"reason": decision.reason},
)
raise PolicyDenied(f"{action} denied: {decision.reason}")
return decision
@staticmethod
def _assert_same_tenant(context: RequestContext, tenant_id: str) -> None:
if context.tenant_id != tenant_id and "platform_admin" not in context.roles:
raise TenantIsolationError("cross-tenant access denied")
def _audit(
self,
context: RequestContext,
*,
action: str,
resource_type: str,
resource_id: str,
outcome: str,
details: Mapping[str, Any] | None = None,
) -> AuditEvent:
event = AuditEvent(
event_id=_new_id("audit"),
tenant_id=context.tenant_id,
principal_id=context.principal_id,
trace_id=context.trace_id,
action=action,
resource_type=resource_type,
resource_id=resource_id,
outcome=outcome,
policy_version=context.policy_version,
details=copy.deepcopy(dict(details or {})),
)
self.store.audit_events.append(event)
return event
def register_tenant(
self,
context: RequestContext,
*,
tenant_id: str,
display_name: str,
metadata: Mapping[str, Any] | None = None,
) -> TenantRecord:
tenant_id = tenant_id.strip()
if not tenant_id or not display_name.strip():
raise ValueError("tenant_id and display_name are required")
self._authorize(
context,
"tenant.register",
{"resource_type": "tenant", "resource_id": tenant_id},
)
self._assert_same_tenant(context, tenant_id)
with self.store.lock:
if tenant_id in self.store.tenants:
raise ControlPlaneError(f"tenant already exists: {tenant_id}")
tenant = TenantRecord(
tenant_id=tenant_id,
display_name=display_name.strip(),
metadata=copy.deepcopy(dict(metadata or {})),
)
self.store.tenants[tenant_id] = tenant
self._audit(
context,
action="tenant.register",
resource_type="tenant",
resource_id=tenant_id,
outcome="allowed",
)
return tenant
def get_tenant(self, context: RequestContext, tenant_id: str | None = None) -> TenantRecord:
tenant_id = tenant_id or context.tenant_id
self._authorize(
context,
"tenant.read",
{"resource_type": "tenant", "resource_id": tenant_id},
)
self._assert_same_tenant(context, tenant_id)
with self.store.lock:
tenant = self.store.tenants.get(tenant_id)
if tenant is None:
raise NotFound("tenant not found")
return tenant
def register_agent(
self,
context: RequestContext,
*,
agent_id: str,
display_name: str,
runtime: str,
capabilities: Iterable[str] = (),
metadata: Mapping[str, Any] | None = None,
) -> AgentRecord:
self._authorize(
context,
"agent.register",
{"resource_type": "agent", "resource_id": agent_id},
)
tenant = self.get_tenant(context)
if tenant.status != "active":
raise ControlPlaneError("tenant is not active")
normalized_capabilities = tuple(
sorted({str(item).strip() for item in capabilities if str(item).strip()})
)
if not agent_id.strip() or not display_name.strip() or not runtime.strip():
raise ValueError("agent_id, display_name and runtime are required")
with self.store.lock:
existing = self.store.agents.get(agent_id)
if existing is not None and existing.tenant_id != context.tenant_id:
raise TenantIsolationError("agent id belongs to another tenant")
if existing is not None:
raise ControlPlaneError(f"agent already exists: {agent_id}")
agent = AgentRecord(
agent_id=agent_id.strip(),
tenant_id=context.tenant_id,
display_name=display_name.strip(),
runtime=runtime.strip(),
capabilities=normalized_capabilities,
metadata=copy.deepcopy(dict(metadata or {})),
)
self.store.agents[agent.agent_id] = agent
self._audit(
context,
action="agent.register",
resource_type="agent",
resource_id=agent.agent_id,
outcome="allowed",
details={"runtime": agent.runtime},
)
return agent
def get_agent(self, context: RequestContext, agent_id: str) -> AgentRecord:
self._authorize(
context,
"agent.read",
{"resource_type": "agent", "resource_id": agent_id},
)
with self.store.lock:
agent = self.store.agents.get(agent_id)
if agent is None:
raise NotFound("agent not found")
self._assert_same_tenant(context, agent.tenant_id)
return agent
def list_agents(self, context: RequestContext) -> list[AgentRecord]:
self._authorize(
context,
"agent.read",
{"resource_type": "agent_collection", "resource_id": context.tenant_id},
)
with self.store.lock:
return [
agent
for agent in self.store.agents.values()
if agent.tenant_id == context.tenant_id
]
def create_task(
self,
context: RequestContext,
*,
agent_id: str,
action: str,
payload: Mapping[str, Any] | None = None,
idempotency_key: str | None = None,
) -> TaskRecord:
self._authorize(
context,
"task.create",
{"resource_type": "task", "resource_id": idempotency_key or "new"},
)
agent = self.get_agent(context, agent_id)
if not agent.enabled:
raise ControlPlaneError("agent is disabled")
if action not in agent.capabilities and "*" not in agent.capabilities:
raise PolicyDenied(f"agent does not advertise capability: {action}")
normalized_payload = copy.deepcopy(dict(payload or {}))
request_digest = _canonical_digest(
{"agent_id": agent_id, "action": action, "payload": normalized_payload}
)
normalized_key = idempotency_key.strip() if idempotency_key else None
with self.store.lock:
if normalized_key:
idempotency_ref = (context.tenant_id, normalized_key)
existing = self.store.idempotency.get(idempotency_ref)
if existing is not None:
existing_task_id, existing_digest = existing
if existing_digest != request_digest:
raise IdempotencyConflict(
"idempotency key was reused with a different request"
)
return self.store.tasks[existing_task_id]
task = TaskRecord(
task_id=_new_id("task"),
tenant_id=context.tenant_id,
agent_id=agent_id,
action=action,
payload=normalized_payload,
idempotency_key=normalized_key,
request_digest=request_digest,
created_by=context.principal_id,
)
self.store.tasks[task.task_id] = task
if normalized_key:
self.store.idempotency[(context.tenant_id, normalized_key)] = (
task.task_id,
request_digest,
)
self._audit(
context,
action="task.create",
resource_type="task",
resource_id=task.task_id,
outcome="allowed",
details={"agent_id": agent_id, "action": action},
)
return task
def get_task(self, context: RequestContext, task_id: str) -> TaskRecord:
self._authorize(
context,
"task.read",
{"resource_type": "task", "resource_id": task_id},
)
with self.store.lock:
task = self.store.tasks.get(task_id)
if task is None:
raise NotFound("task not found")
self._assert_same_tenant(context, task.tenant_id)
return task
def list_tasks(
self,
context: RequestContext,
*,
states: Iterable[TaskState | str] | None = None,
) -> list[TaskRecord]:
self._authorize(
context,
"task.read",
{"resource_type": "task_collection", "resource_id": context.tenant_id},
)
state_filter = (
{TaskState(item) for item in states}
if states is not None
else None
)
with self.store.lock:
return [
task
for task in self.store.tasks.values()
if task.tenant_id == context.tenant_id
and (state_filter is None or task.state in state_filter)
]
def transition_task(
self,
context: RequestContext,
*,
task_id: str,
target_state: TaskState | str,
expected_revision: int | None = None,
result: Mapping[str, Any] | None = None,
error: Mapping[str, Any] | None = None,
) -> TaskRecord:
target = TaskState(target_state)
action_name = {
TaskState.APPROVED: "task.approve",
TaskState.RUNNING: "task.start",
TaskState.COMPLETED: "task.complete",
TaskState.FAILED: "task.fail",
TaskState.CANCELLED: "task.cancel",
}.get(target)
if action_name is None:
raise InvalidTransition(f"cannot transition to {target.value}")
self._authorize(
context,
action_name,
{"resource_type": "task", "resource_id": task_id},
)
with self.store.lock:
current = self.store.tasks.get(task_id)
if current is None:
raise NotFound("task not found")
self._assert_same_tenant(context, current.tenant_id)
if expected_revision is not None and current.revision != expected_revision:
raise RevisionConflict(
f"expected revision {expected_revision}, current {current.revision}"
)
if target not in ALLOWED_TASK_TRANSITIONS[current.state]:
raise InvalidTransition(
f"invalid transition {current.state.value} -> {target.value}"
)
if target == TaskState.COMPLETED and result is None:
raise ValueError("completed task requires result")
if target == TaskState.FAILED and error is None:
raise ValueError("failed task requires error")
updated = TaskRecord(
task_id=current.task_id,
tenant_id=current.tenant_id,
agent_id=current.agent_id,
action=current.action,
payload=current.payload,
state=target,
revision=current.revision + 1,
idempotency_key=current.idempotency_key,
request_digest=current.request_digest,
result=copy.deepcopy(dict(result)) if result is not None else current.result,
error=copy.deepcopy(dict(error)) if error is not None else current.error,
created_by=current.created_by,
approved_by=(
context.principal_id
if target == TaskState.APPROVED
else current.approved_by
),
created_at=current.created_at,
updated_at=_utc_now(),
)
self.store.tasks[task_id] = updated
self._audit(
context,
action=action_name,
resource_type="task",
resource_id=task_id,
outcome="allowed",
details={
"from": current.state.value,
"to": target.value,
"revision": updated.revision,
},
)
return updated
def list_audit_events(
self,
context: RequestContext,
*,
resource_id: str | None = None,
) -> list[AuditEvent]:
self._authorize(
context,
"audit.read",
{"resource_type": "audit", "resource_id": resource_id or context.tenant_id},
)
with self.store.lock:
return [
event
for event in self.store.audit_events
if event.tenant_id == context.tenant_id
and (resource_id is None or event.resource_id == resource_id)
]
@@ -1,379 +0,0 @@
"""AIOS V2 持久化 Dispatch 记录。
Dispatch 是 AIOS Task 与 Runtime command/job 之间的唯一可信绑定。所有结果回写
必须匹配 tenant/task/trace/runtime_execution/provenance/run/principal。
"""
from __future__ import annotations
import json
import sqlite3
import threading
import uuid
from dataclasses import asdict, dataclass, replace
from datetime import datetime, timezone
from enum import Enum
from pathlib import Path
from typing import Any, Mapping, Protocol
def utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _required(value: str, label: str) -> str:
normalized = str(value or "").strip()
if not normalized:
raise ValueError(f"{label} is required")
return normalized
class DispatchState(str, Enum):
PREPARED = "PREPARED"
RUNNING = "RUNNING"
QUEUED = "QUEUED"
COMPLETED = "COMPLETED"
FAILED = "FAILED"
CANCELLED = "CANCELLED"
TERMINAL_DISPATCH_STATES = frozenset(
{DispatchState.COMPLETED, DispatchState.FAILED, DispatchState.CANCELLED}
)
@dataclass(frozen=True)
class DispatchRecord:
dispatch_id: str
tenant_id: str
task_id: str
agent_id: str
principal_id: str
runtime: str
runtime_execution_id: str
trace_id: str
provenance_id: str
run_id: str
authorization_id: str
state: DispatchState = DispatchState.PREPARED
expected_worker_id: str | None = None
result_digest: str | None = None
created_at: str = ""
updated_at: str = ""
completed_at: str | None = None
def __post_init__(self) -> None:
for name in (
"dispatch_id",
"tenant_id",
"task_id",
"agent_id",
"principal_id",
"runtime",
"runtime_execution_id",
"trace_id",
"provenance_id",
"run_id",
"authorization_id",
):
object.__setattr__(self, name, _required(getattr(self, name), name))
object.__setattr__(self, "state", DispatchState(self.state))
now = utc_now()
object.__setattr__(self, "created_at", self.created_at or now)
object.__setattr__(self, "updated_at", self.updated_at or now)
if self.expected_worker_id is not None:
object.__setattr__(
self,
"expected_worker_id",
_required(self.expected_worker_id, "expected_worker_id"),
)
@classmethod
def create(
cls,
*,
tenant_id: str,
task_id: str,
agent_id: str,
principal_id: str,
runtime: str,
runtime_execution_id: str,
trace_id: str,
provenance_id: str,
run_id: str,
authorization_id: str,
) -> "DispatchRecord":
return cls(
dispatch_id=f"dispatch_{uuid.uuid4().hex}",
tenant_id=tenant_id,
task_id=task_id,
agent_id=agent_id,
principal_id=principal_id,
runtime=runtime,
runtime_execution_id=runtime_execution_id,
trace_id=trace_id,
provenance_id=provenance_id,
run_id=run_id,
authorization_id=authorization_id,
)
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
value["state"] = self.state.value
return value
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "DispatchRecord":
return cls(**dict(value))
class DispatchConflict(RuntimeError):
pass
class DispatchNotFound(RuntimeError):
pass
class DispatchRepository(Protocol):
def create(self, record: DispatchRecord) -> DispatchRecord: ...
def get(self, dispatch_id: str, *, tenant_id: str) -> DispatchRecord: ...
def get_by_task(self, task_id: str, *, tenant_id: str) -> DispatchRecord | None: ...
def get_by_runtime_execution(
self, runtime_execution_id: str, *, tenant_id: str
) -> DispatchRecord | None: ...
def update(
self,
dispatch_id: str,
*,
tenant_id: str,
state: DispatchState | str | None = None,
expected_worker_id: str | None = None,
result_digest: str | None = None,
) -> DispatchRecord: ...
def list(self, *, tenant_id: str) -> list[DispatchRecord]: ...
class InMemoryDispatchRepository:
def __init__(self) -> None:
self._lock = threading.RLock()
self._records: dict[str, DispatchRecord] = {}
self._by_task: dict[tuple[str, str], str] = {}
self._by_runtime: dict[tuple[str, str], str] = {}
def create(self, record: DispatchRecord) -> DispatchRecord:
with self._lock:
task_ref = (record.tenant_id, record.task_id)
runtime_ref = (record.tenant_id, record.runtime_execution_id)
if task_ref in self._by_task or runtime_ref in self._by_runtime:
existing = self.get_by_task(record.task_id, tenant_id=record.tenant_id)
if existing == record:
return record
raise DispatchConflict("task or runtime execution already has a dispatch")
self._records[record.dispatch_id] = record
self._by_task[task_ref] = record.dispatch_id
self._by_runtime[runtime_ref] = record.dispatch_id
return record
def get(self, dispatch_id: str, *, tenant_id: str) -> DispatchRecord:
with self._lock:
record = self._records.get(dispatch_id)
if record is None or record.tenant_id != tenant_id:
raise DispatchNotFound("dispatch not found")
return record
def get_by_task(self, task_id: str, *, tenant_id: str) -> DispatchRecord | None:
with self._lock:
dispatch_id = self._by_task.get((tenant_id, task_id))
return self._records.get(dispatch_id) if dispatch_id else None
def get_by_runtime_execution(
self, runtime_execution_id: str, *, tenant_id: str
) -> DispatchRecord | None:
with self._lock:
dispatch_id = self._by_runtime.get((tenant_id, runtime_execution_id))
return self._records.get(dispatch_id) if dispatch_id else None
def update(
self,
dispatch_id: str,
*,
tenant_id: str,
state: DispatchState | str | None = None,
expected_worker_id: str | None = None,
result_digest: str | None = None,
) -> DispatchRecord:
with self._lock:
current = self.get(dispatch_id, tenant_id=tenant_id)
target = DispatchState(state) if state is not None else current.state
if current.state in TERMINAL_DISPATCH_STATES and target != current.state:
raise DispatchConflict("terminal dispatch cannot transition")
updated = replace(
current,
state=target,
expected_worker_id=(
expected_worker_id
if expected_worker_id is not None
else current.expected_worker_id
),
result_digest=result_digest or current.result_digest,
updated_at=utc_now(),
completed_at=(utc_now() if target in TERMINAL_DISPATCH_STATES else None),
)
self._records[dispatch_id] = updated
return updated
def list(self, *, tenant_id: str) -> list[DispatchRecord]:
with self._lock:
return sorted(
[item for item in self._records.values() if item.tenant_id == tenant_id],
key=lambda item: (item.created_at, item.dispatch_id),
)
class SQLiteDispatchRepository:
SCHEMA_VERSION = 1
def __init__(self, path: str | Path) -> None:
self.path = Path(path).resolve()
self.path.parent.mkdir(parents=True, exist_ok=True)
self._lock = threading.RLock()
with self._connect() as connection:
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS dispatch_records (
dispatch_id TEXT PRIMARY KEY,
tenant_id TEXT NOT NULL,
task_id TEXT NOT NULL,
runtime_execution_id TEXT NOT NULL,
payload_json TEXT NOT NULL,
state TEXT NOT NULL,
updated_at TEXT NOT NULL,
UNIQUE(tenant_id, task_id),
UNIQUE(tenant_id, runtime_execution_id)
);
CREATE INDEX IF NOT EXISTS idx_dispatch_tenant_state
ON dispatch_records(tenant_id, state, updated_at);
"""
)
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(str(self.path), timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA synchronous=FULL")
return connection
@staticmethod
def _encode(record: DispatchRecord) -> str:
return json.dumps(
record.as_dict(), ensure_ascii=False, sort_keys=True, separators=(",", ":")
)
@staticmethod
def _decode(row: sqlite3.Row) -> DispatchRecord:
value = json.loads(row["payload_json"])
if not isinstance(value, Mapping):
raise RuntimeError("invalid dispatch record")
return DispatchRecord.from_dict(value)
def create(self, record: DispatchRecord) -> DispatchRecord:
with self._lock, self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
try:
connection.execute(
"INSERT INTO dispatch_records VALUES (?, ?, ?, ?, ?, ?, ?)",
(
record.dispatch_id,
record.tenant_id,
record.task_id,
record.runtime_execution_id,
self._encode(record),
record.state.value,
record.updated_at,
),
)
connection.commit()
return record
except sqlite3.IntegrityError as exc:
connection.rollback()
existing = self.get_by_task(record.task_id, tenant_id=record.tenant_id)
if existing is not None and existing == record:
return existing
raise DispatchConflict("task or runtime execution already has a dispatch") from exc
def get(self, dispatch_id: str, *, tenant_id: str) -> DispatchRecord:
with self._lock, self._connect() as connection:
row = connection.execute(
"SELECT payload_json FROM dispatch_records WHERE dispatch_id = ? AND tenant_id = ?",
(_required(dispatch_id, "dispatch_id"), _required(tenant_id, "tenant_id")),
).fetchone()
if row is None:
raise DispatchNotFound("dispatch not found")
return self._decode(row)
def get_by_task(self, task_id: str, *, tenant_id: str) -> DispatchRecord | None:
with self._lock, self._connect() as connection:
row = connection.execute(
"SELECT payload_json FROM dispatch_records WHERE task_id = ? AND tenant_id = ?",
(_required(task_id, "task_id"), _required(tenant_id, "tenant_id")),
).fetchone()
return self._decode(row) if row is not None else None
def get_by_runtime_execution(
self, runtime_execution_id: str, *, tenant_id: str
) -> DispatchRecord | None:
with self._lock, self._connect() as connection:
row = connection.execute(
"SELECT payload_json FROM dispatch_records WHERE runtime_execution_id = ? AND tenant_id = ?",
(
_required(runtime_execution_id, "runtime_execution_id"),
_required(tenant_id, "tenant_id"),
),
).fetchone()
return self._decode(row) if row is not None else None
def update(
self,
dispatch_id: str,
*,
tenant_id: str,
state: DispatchState | str | None = None,
expected_worker_id: str | None = None,
result_digest: str | None = None,
) -> DispatchRecord:
current = self.get(dispatch_id, tenant_id=tenant_id)
target = DispatchState(state) if state is not None else current.state
if current.state in TERMINAL_DISPATCH_STATES and target != current.state:
raise DispatchConflict("terminal dispatch cannot transition")
updated = replace(
current,
state=target,
expected_worker_id=(
expected_worker_id if expected_worker_id is not None else current.expected_worker_id
),
result_digest=result_digest or current.result_digest,
updated_at=utc_now(),
completed_at=(utc_now() if target in TERMINAL_DISPATCH_STATES else None),
)
with self._lock, self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
connection.execute(
"UPDATE dispatch_records SET payload_json = ?, state = ?, updated_at = ? WHERE dispatch_id = ? AND tenant_id = ?",
(
self._encode(updated),
updated.state.value,
updated.updated_at,
updated.dispatch_id,
updated.tenant_id,
),
)
connection.commit()
return updated
def list(self, *, tenant_id: str) -> list[DispatchRecord]:
with self._lock, self._connect() as connection:
rows = connection.execute(
"SELECT payload_json FROM dispatch_records WHERE tenant_id = ? ORDER BY updated_at, dispatch_id",
(_required(tenant_id, "tenant_id"),),
).fetchall()
return [self._decode(row) for row in rows]
@@ -1,543 +0,0 @@
"""Portable multi-provider execution plans and aggregate release gates.
A plan is a DAG of executor-neutral stages. Only root stages are declared at
plan creation. A dependent stage is declared atomically after every dependency
passes its Truth + notification gate, so its start deadline cannot expire while
it is legitimately waiting for upstream work.
"""
from __future__ import annotations
import json
import sqlite3
import threading
from dataclasses import asdict, dataclass, field
from pathlib import Path
from typing import Any, Mapping, Sequence
from .executor_protocol import ExecutorProviderRegistry, normalize_executor
from .external_task_control import ExternalTaskCoordinator
from .external_task_query import list_records
from .task_execution_registry import (
ExecutionTaskManifest,
ExecutionTruthRecord,
ExecutionTruthState,
required,
utc_now,
)
PLAN_SCHEMA_VERSION = "aios.execution-plan.v2"
WAITING_DEPENDENCY = "WAITING_DEPENDENCY"
@dataclass(frozen=True)
class ExecutionStageManifest:
stage_id: str
task_id: str
executor_type: str
role: str
steps: Sequence[str]
depends_on: Sequence[str] = ()
required_evidence: Sequence[str] = ()
notification_events: Sequence[str] = ("STARTED", "SUCCEEDED", "FAILED")
owner_notification_ref: str = "AIOS_OWNER_WECHAT_TARGET_REF"
start_deadline_seconds: int = 300
heartbeat_timeout_seconds: int = 180
required: bool = True
metadata: Mapping[str, Any] = field(default_factory=dict)
def __post_init__(self) -> None:
for name in ("stage_id", "task_id", "role", "owner_notification_ref"):
object.__setattr__(self, name, required(getattr(self, name), name))
object.__setattr__(self, "executor_type", normalize_executor(self.executor_type))
steps = tuple(required(item, "step") for item in self.steps)
if not steps:
raise ValueError("execution stage requires at least one step")
object.__setattr__(self, "steps", steps)
object.__setattr__(
self,
"depends_on",
tuple(sorted({required(item, "depends_on") for item in self.depends_on})),
)
object.__setattr__(
self,
"required_evidence",
tuple(sorted({required(item, "required_evidence") for item in self.required_evidence})),
)
object.__setattr__(
self,
"notification_events",
tuple(sorted({required(item, "notification_event") for item in self.notification_events})),
)
if self.start_deadline_seconds < 1 or self.heartbeat_timeout_seconds < 1:
raise ValueError("stage deadlines must be positive")
object.__setattr__(
self,
"metadata",
json.loads(json.dumps(dict(self.metadata), ensure_ascii=False, default=str)),
)
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
value["steps"] = list(self.steps)
value["depends_on"] = list(self.depends_on)
value["required_evidence"] = list(self.required_evidence)
value["notification_events"] = list(self.notification_events)
return value
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "ExecutionStageManifest":
return cls(**dict(value))
@dataclass(frozen=True)
class ExecutionPlanManifest:
plan_id: str
source_branch: str
source_oid: str
target_version: str
stages: Sequence[ExecutionStageManifest]
release_gate_policy: str = "all_required_stages"
schema_version: str = PLAN_SCHEMA_VERSION
metadata: Mapping[str, Any] = field(default_factory=dict)
def __post_init__(self) -> None:
for name in ("plan_id", "source_branch", "source_oid", "target_version"):
object.__setattr__(self, name, required(getattr(self, name), name))
if self.schema_version != PLAN_SCHEMA_VERSION:
raise ValueError("unsupported execution plan schema")
if self.release_gate_policy != "all_required_stages":
raise ValueError("unsupported release gate policy")
stages = tuple(self.stages)
if not stages:
raise ValueError("execution plan requires stages")
stage_ids = [item.stage_id for item in stages]
task_ids = [item.task_id for item in stages]
if len(stage_ids) != len(set(stage_ids)):
raise ValueError("execution plan stage_id values must be unique")
if len(task_ids) != len(set(task_ids)):
raise ValueError("execution plan task_id values must be unique")
known = set(stage_ids)
for stage in stages:
unknown = sorted(set(stage.depends_on) - known)
if unknown:
raise ValueError(f"stage {stage.stage_id} has unknown dependencies: {unknown}")
if stage.stage_id in stage.depends_on:
raise ValueError("stage cannot depend on itself")
self._assert_acyclic(stages)
object.__setattr__(self, "stages", stages)
object.__setattr__(
self,
"metadata",
json.loads(json.dumps(dict(self.metadata), ensure_ascii=False, default=str)),
)
@staticmethod
def _assert_acyclic(stages: Sequence[ExecutionStageManifest]) -> None:
dependencies = {item.stage_id: set(item.depends_on) for item in stages}
resolved: set[str] = set()
while len(resolved) < len(stages):
ready = {
stage_id
for stage_id, deps in dependencies.items()
if stage_id not in resolved and deps.issubset(resolved)
}
if not ready:
raise ValueError("execution plan contains a dependency cycle")
resolved.update(ready)
def as_dict(self) -> dict[str, Any]:
return {
"schema_version": self.schema_version,
"plan_id": self.plan_id,
"source_branch": self.source_branch,
"source_oid": self.source_oid,
"target_version": self.target_version,
"release_gate_policy": self.release_gate_policy,
"stages": [item.as_dict() for item in self.stages],
"metadata": dict(self.metadata),
}
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "ExecutionPlanManifest":
raw = dict(value)
raw["stages"] = tuple(
ExecutionStageManifest.from_dict(item) for item in raw.get("stages") or ()
)
return cls(**raw)
def stage(self, stage_id: str) -> ExecutionStageManifest:
normalized = required(stage_id, "stage_id")
for stage in self.stages:
if stage.stage_id == normalized:
return stage
raise KeyError(f"execution stage not found: {normalized}")
class SQLiteExecutionPlanStore:
"""Durable plan declarations; stage state remains in Truth Registry."""
def __init__(self, path: str | Path) -> None:
self.path = Path(path).resolve()
self.path.parent.mkdir(parents=True, exist_ok=True)
self._lock = threading.RLock()
with self._connect() as connection:
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS execution_plans (
plan_id TEXT PRIMARY KEY,
manifest_json TEXT NOT NULL,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
"""
)
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(str(self.path), timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA synchronous=FULL")
return connection
@staticmethod
def _encode(plan: ExecutionPlanManifest) -> str:
return json.dumps(
plan.as_dict(),
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
)
def save(
self,
plan: ExecutionPlanManifest,
*,
allow_source_oid_supersession: bool = False,
) -> None:
payload = self._encode(plan)
now = utc_now()
with self._lock, self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
existing = connection.execute(
"SELECT manifest_json FROM execution_plans WHERE plan_id = ?",
(plan.plan_id,),
).fetchone()
if existing is not None:
if existing["manifest_json"] != payload:
previous = json.loads(existing["manifest_json"])
replacement = plan.as_dict()
previous_oid = previous.pop("source_oid", None)
replacement_oid = replacement.pop("source_oid", None)
if (
not allow_source_oid_supersession
or previous != replacement
or previous_oid == replacement_oid
):
connection.rollback()
raise RuntimeError(
"plan_id already exists with another manifest"
)
connection.execute(
"UPDATE execution_plans SET manifest_json = ?, updated_at = ? WHERE plan_id = ?",
(payload, now, plan.plan_id),
)
connection.commit()
return
connection.rollback()
return
connection.execute(
"INSERT INTO execution_plans VALUES (?, ?, ?, ?)",
(plan.plan_id, payload, now, now),
)
connection.commit()
def get(self, plan_id: str) -> ExecutionPlanManifest:
with self._lock, self._connect() as connection:
row = connection.execute(
"SELECT manifest_json FROM execution_plans WHERE plan_id = ?",
(required(plan_id, "plan_id"),),
).fetchone()
if row is None:
raise KeyError("execution plan not found")
value = json.loads(row["manifest_json"])
if not isinstance(value, Mapping):
raise ValueError("stored execution plan is invalid")
return ExecutionPlanManifest.from_dict(value)
def list(self) -> list[ExecutionPlanManifest]:
with self._lock, self._connect() as connection:
rows = connection.execute(
"SELECT manifest_json FROM execution_plans ORDER BY created_at, plan_id"
).fetchall()
return [
ExecutionPlanManifest.from_dict(json.loads(row["manifest_json"]))
for row in rows
]
class ExecutionPlanCoordinator:
"""Declares dependency-ready stages and evaluates aggregate release gates."""
def __init__(
self,
*,
task_coordinator: ExternalTaskCoordinator,
plan_store: SQLiteExecutionPlanStore,
providers: ExecutorProviderRegistry,
) -> None:
self.task_coordinator = task_coordinator
self.plan_store = plan_store
self.providers = providers
@classmethod
def from_file(
cls,
path: str | Path,
*,
providers: ExecutorProviderRegistry,
) -> "ExecutionPlanCoordinator":
return cls(
task_coordinator=ExternalTaskCoordinator.from_file(path),
plan_store=SQLiteExecutionPlanStore(path),
providers=providers,
)
@staticmethod
def _task_manifest(
plan: ExecutionPlanManifest,
stage: ExecutionStageManifest,
) -> ExecutionTaskManifest:
return ExecutionTaskManifest(
task_id=stage.task_id,
executor=stage.executor_type,
source_branch=plan.source_branch,
source_oid=plan.source_oid,
target_version=plan.target_version,
steps=stage.steps,
start_deadline_seconds=stage.start_deadline_seconds,
heartbeat_timeout_seconds=stage.heartbeat_timeout_seconds,
required_evidence=stage.required_evidence,
notification_events=stage.notification_events,
owner_notification_ref=stage.owner_notification_ref,
manifest_version="aios.external-task.v2",
)
def _record(self, task_id: str) -> ExecutionTruthRecord | None:
try:
return self.task_coordinator.registry.get(task_id)
except KeyError:
return None
def _declare_stage(
self,
plan: ExecutionPlanManifest,
stage: ExecutionStageManifest,
) -> ExecutionTruthRecord:
self.providers.validate_manifest(self._task_manifest(plan, stage))
return self.task_coordinator.declare(self._task_manifest(plan, stage))
def declare(self, plan: ExecutionPlanManifest) -> Mapping[str, Any]:
for stage in plan.stages:
self.providers.get(stage.executor_type)
allow_source_oid_supersession = False
try:
previous = self.plan_store.get(plan.plan_id)
except KeyError:
previous = None
if previous is not None and previous.source_oid != plan.source_oid:
previous_value = previous.as_dict()
replacement_value = plan.as_dict()
previous_value.pop("source_oid")
replacement_value.pop("source_oid")
root_records = [
self._record(stage.task_id)
for stage in previous.stages
if not stage.depends_on
]
allow_source_oid_supersession = bool(
previous_value == replacement_value
and root_records
and all(
record is not None
and record.state == ExecutionTruthState.NOT_STARTED
for record in root_records
)
)
self.plan_store.save(
plan,
allow_source_oid_supersession=allow_source_oid_supersession,
)
for stage in plan.stages:
if not stage.depends_on:
self._declare_stage(plan, stage)
return self.snapshot(plan.plan_id)
def declare_file(self, path: str | Path) -> Mapping[str, Any]:
value = json.loads(Path(path).read_text(encoding="utf-8"))
if not isinstance(value, Mapping):
raise ValueError("execution plan file must contain a JSON object")
return self.declare(ExecutionPlanManifest.from_dict(value))
def dependency_gate(self, plan_id: str, stage_id: str) -> Mapping[str, Any]:
plan = self.plan_store.get(plan_id)
stage = plan.stage(stage_id)
dependencies: list[Mapping[str, Any]] = []
passed = True
for dependency_id in stage.depends_on:
dependency = plan.stage(dependency_id)
record = self._record(dependency.task_id)
if record is None:
gate: Mapping[str, Any] = {
"task_id": dependency.task_id,
"state": WAITING_DEPENDENCY,
"passed": False,
"release_blocking": True,
}
else:
gate = self.task_coordinator.gate(dependency.task_id)
dependencies.append(
{
"stage_id": dependency.stage_id,
"task_id": dependency.task_id,
"gate": gate,
}
)
if not gate["passed"]:
passed = False
return {
"plan_id": plan.plan_id,
"stage_id": stage.stage_id,
"passed": passed,
"release_blocking": not passed,
"dependencies": dependencies,
}
def activate_ready(self, plan_id: str) -> list[Mapping[str, Any]]:
plan = self.plan_store.get(plan_id)
activated: list[Mapping[str, Any]] = []
for stage in plan.stages:
if self._record(stage.task_id) is not None:
continue
dependency = self.dependency_gate(plan.plan_id, stage.stage_id)
if not dependency["passed"]:
continue
record = self._declare_stage(plan, stage)
activated.append(
{
"stage_id": stage.stage_id,
"task_id": record.task_id,
"executor_type": stage.executor_type,
"state": record.state.value,
}
)
return activated
def discover_ready(
self,
*,
plan_id: str,
executor_type: str,
) -> list[Mapping[str, Any]]:
plan = self.plan_store.get(plan_id)
self.activate_ready(plan.plan_id)
normalized = normalize_executor(executor_type)
records = {
item.task_id: item
for item in list_records(
self.task_coordinator.registry.path,
executor=normalized,
states=(
ExecutionTruthState.DECLARED.value,
ExecutionTruthState.REGISTERED.value,
),
)
}
ready: list[Mapping[str, Any]] = []
for stage in plan.stages:
if stage.executor_type != normalized or stage.task_id not in records:
continue
dependency = self.dependency_gate(plan.plan_id, stage.stage_id)
if dependency["passed"]:
ready.append(
{
"plan_id": plan.plan_id,
"stage": stage.as_dict(),
"record": records[stage.task_id].as_dict(),
}
)
return ready
def snapshot(self, plan_id: str) -> Mapping[str, Any]:
plan = self.plan_store.get(plan_id)
stages: list[Mapping[str, Any]] = []
for stage in plan.stages:
record = self._record(stage.task_id)
dependency = self.dependency_gate(plan.plan_id, stage.stage_id)
if record is None:
truth: Mapping[str, Any] = {
"task_id": stage.task_id,
"state": WAITING_DEPENDENCY,
"passed": False,
"release_blocking": True,
"executor_run_id": None,
"executor_id": None,
"current_step": 0,
"total_steps": len(stage.steps),
"evidence": {},
"error": {},
"notifications": [],
}
else:
truth = self.task_coordinator.status(stage.task_id)
stages.append(
{
"stage": stage.as_dict(),
"dependency_gate": dependency,
"truth": truth,
}
)
return {
"manifest": plan.as_dict(),
"stages": stages,
}
def gate(self, plan_id: str) -> Mapping[str, Any]:
plan = self.plan_store.get(plan_id)
self.activate_ready(plan.plan_id)
stages: list[Mapping[str, Any]] = []
required_passed = True
for stage in plan.stages:
dependency = self.dependency_gate(plan.plan_id, stage.stage_id)
record = self._record(stage.task_id)
if record is None:
truth_gate: Mapping[str, Any] = {
"task_id": stage.task_id,
"state": WAITING_DEPENDENCY,
"passed": False,
"release_blocking": True,
}
else:
truth_gate = self.task_coordinator.gate(stage.task_id)
stage_passed = bool(truth_gate["passed"] and dependency["passed"])
if stage.required and not stage_passed:
required_passed = False
stages.append(
{
"stage_id": stage.stage_id,
"task_id": stage.task_id,
"executor_type": stage.executor_type,
"role": stage.role,
"required": stage.required,
"passed": stage_passed,
"truth_gate": truth_gate,
"dependency_gate": dependency,
}
)
return {
"plan_id": plan.plan_id,
"target_version": plan.target_version,
"policy": plan.release_gate_policy,
"passed": required_passed,
"release_blocking": not required_passed,
"stages": stages,
}
@@ -1,323 +0,0 @@
"""AIOS V2 Agent 执行证据。
Evidence 按租户隔离并使用原子 JSON 写入。生产模式下,claim、heartbeat
和 complete 在落盘前必须通过最小权限 Evidence Grant 授权。
"""
from __future__ import annotations
import json
import os
import threading
from dataclasses import asdict, dataclass, field, replace
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Mapping, Protocol
from .provenance import ProvenanceDenied, digest_payload
class EvidenceAuthorizationWriter(Protocol):
def authorize_evidence_write(self, **values: Any) -> Any: ...
def utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _required(value: str, label: str) -> str:
normalized = str(value or "").strip()
if not normalized:
raise ValueError(f"{label} is required")
return normalized
def _safe(value: Mapping[str, Any] | None) -> dict[str, Any]:
return json.loads(json.dumps(dict(value or {}), ensure_ascii=False, default=str))
@dataclass(frozen=True)
class ExecutionEvidence:
execution_id: str
tenant_id: str
task_id: str
agent_id: str
executor: str
branch: str
status: str = "claimed"
progress_completed: int = 0
progress_total: int = 1
current_step: str = "claim"
last_commit: str | None = None
test_status: str = "not_run"
test_summary: Mapping[str, Any] = field(default_factory=dict)
result_summary: Mapping[str, Any] = field(default_factory=dict)
claimed_at: str = field(default_factory=utc_now)
heartbeat_at: str = field(default_factory=utc_now)
completed_at: str | None = None
provenance_id: str | None = None
run_id: str | None = None
last_authorization_id: str | None = None
def __post_init__(self) -> None:
for name in (
"execution_id",
"tenant_id",
"task_id",
"agent_id",
"executor",
"branch",
"status",
"current_step",
"claimed_at",
"heartbeat_at",
):
object.__setattr__(self, name, _required(getattr(self, name), name))
if self.progress_completed < 0 or self.progress_total < 1:
raise ValueError("invalid progress values")
if self.progress_completed > self.progress_total:
raise ValueError("progress_completed cannot exceed progress_total")
object.__setattr__(self, "test_summary", _safe(self.test_summary))
object.__setattr__(self, "result_summary", _safe(self.result_summary))
for optional in ("provenance_id", "run_id", "last_authorization_id"):
value = getattr(self, optional)
object.__setattr__(self, optional, str(value).strip() if value else None)
class ExecutionTracker:
"""每个租户、每次执行持久保存一份证据记录。"""
def __init__(
self,
root: str | Path,
*,
provenance_writer: EvidenceAuthorizationWriter | None = None,
require_provenance: bool = False,
) -> None:
self.root = Path(root).resolve()
self.root.mkdir(parents=True, exist_ok=True)
self.provenance_writer = provenance_writer
self.require_provenance = bool(require_provenance)
if self.require_provenance and self.provenance_writer is None:
raise ValueError("production evidence tracking requires a provenance writer")
self._lock = threading.RLock()
@staticmethod
def _component(value: str) -> str:
normalized = _required(value, "path component")
if normalized in {".", ".."} or "/" in normalized or "\\" in normalized:
raise ValueError("invalid evidence path component")
return normalized
def _path(self, tenant_id: str, execution_id: str) -> Path:
tenant = self._component(tenant_id)
execution = self._component(execution_id)
return self.root / tenant / f"{execution}.json"
def _authorize(
self,
evidence: ExecutionEvidence,
*,
operation: str,
principal_id: str | None,
provenance_id: str | None,
run_id: str | None,
changes: Mapping[str, Any],
) -> str | None:
if not self.require_provenance:
return None
if self.provenance_writer is None:
raise RuntimeError("Run Provenance writer is not configured")
effective_provenance = provenance_id or evidence.provenance_id
effective_run = run_id or evidence.run_id
if not effective_provenance or not effective_run or not principal_id:
raise RuntimeError(
"production evidence write requires provenance_id, run_id and principal_id"
)
request_digest = digest_payload(
{
"operation": operation,
"execution_id": evidence.execution_id,
"tenant_id": evidence.tenant_id,
"task_id": evidence.task_id,
"agent_id": evidence.agent_id,
"executor": evidence.executor,
"branch": evidence.branch,
"changes": dict(changes),
}
)
try:
authorization = self.provenance_writer.authorize_evidence_write(
provenance_id=effective_provenance,
run_id=effective_run,
tenant_id=evidence.tenant_id,
task_id=evidence.task_id,
agent_id=evidence.agent_id,
principal_id=principal_id,
request_digest=request_digest,
)
except ProvenanceDenied as exc:
raise RuntimeError(f"Run Provenance denied evidence write: {exc}") from exc
return authorization.authorization_id
def claim(
self,
evidence: ExecutionEvidence,
*,
principal_id: str | None = None,
provenance_id: str | None = None,
run_id: str | None = None,
) -> ExecutionEvidence:
path = self._path(evidence.tenant_id, evidence.execution_id)
with self._lock:
if path.exists():
current = self._read(path)
if current.task_id != evidence.task_id or current.agent_id != evidence.agent_id:
raise RuntimeError("execution_id already belongs to another task")
if self.require_provenance and (
current.provenance_id != (provenance_id or evidence.provenance_id)
or current.run_id != (run_id or evidence.run_id)
):
raise RuntimeError(
"execution_id already belongs to another provenance context"
)
return current
effective_provenance = provenance_id or evidence.provenance_id
effective_run = run_id or evidence.run_id
authorization_id = self._authorize(
evidence,
operation="claim",
principal_id=principal_id,
provenance_id=effective_provenance,
run_id=effective_run,
changes={"status": evidence.status, "current_step": evidence.current_step},
)
persisted = replace(
evidence,
provenance_id=effective_provenance,
run_id=effective_run,
last_authorization_id=authorization_id,
)
self._write(path, persisted)
return persisted
def heartbeat(
self,
*,
tenant_id: str,
execution_id: str,
status: str | None = None,
current_step: str | None = None,
progress_completed: int | None = None,
progress_total: int | None = None,
last_commit: str | None = None,
principal_id: str | None = None,
provenance_id: str | None = None,
run_id: str | None = None,
) -> ExecutionEvidence:
path = self._path(tenant_id, execution_id)
with self._lock:
current = self._read(path)
changes = {
"status": status or current.status,
"current_step": current_step or current.current_step,
"progress_completed": (
progress_completed
if progress_completed is not None
else current.progress_completed
),
"progress_total": (
progress_total if progress_total is not None else current.progress_total
),
"last_commit": last_commit or current.last_commit,
}
authorization_id = self._authorize(
current,
operation="heartbeat",
principal_id=principal_id,
provenance_id=provenance_id,
run_id=run_id,
changes=changes,
)
updated = replace(
current,
**changes,
last_authorization_id=authorization_id or current.last_authorization_id,
heartbeat_at=utc_now(),
)
self._write(path, updated)
return updated
def complete(
self,
*,
tenant_id: str,
execution_id: str,
status: str,
test_status: str,
test_summary: Mapping[str, Any] | None = None,
result_summary: Mapping[str, Any] | None = None,
last_commit: str | None = None,
principal_id: str | None = None,
provenance_id: str | None = None,
run_id: str | None = None,
) -> ExecutionEvidence:
path = self._path(tenant_id, execution_id)
with self._lock:
current = self._read(path)
changes = {
"status": _required(status, "status"),
"current_step": "complete",
"progress_completed": current.progress_total,
"last_commit": last_commit or current.last_commit,
"test_status": _required(test_status, "test_status"),
"test_summary": _safe(test_summary),
"result_summary": _safe(result_summary),
}
authorization_id = self._authorize(
current,
operation="complete",
principal_id=principal_id,
provenance_id=provenance_id,
run_id=run_id,
changes=changes,
)
updated = replace(
current,
**changes,
last_authorization_id=authorization_id or current.last_authorization_id,
heartbeat_at=utc_now(),
completed_at=utc_now(),
)
self._write(path, updated)
return updated
def get(self, *, tenant_id: str, execution_id: str) -> ExecutionEvidence:
return self._read(self._path(tenant_id, execution_id))
def list(self, *, tenant_id: str) -> list[ExecutionEvidence]:
tenant_dir = self.root / self._component(tenant_id)
if not tenant_dir.exists():
return []
records = [self._read(path) for path in tenant_dir.glob("*.json")]
return sorted(records, key=lambda item: (item.heartbeat_at, item.execution_id), reverse=True)
@staticmethod
def _read(path: Path) -> ExecutionEvidence:
try:
value = json.loads(path.read_text(encoding="utf-8"))
except FileNotFoundError as exc:
raise KeyError("execution evidence not found") from exc
if not isinstance(value, Mapping):
raise RuntimeError("invalid execution evidence file")
return ExecutionEvidence(**dict(value))
@staticmethod
def _write(path: Path, evidence: ExecutionEvidence) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_suffix(f".tmp-{os.getpid()}")
temporary.write_text(
json.dumps(asdict(evidence), ensure_ascii=False, sort_keys=True, indent=2) + "\n",
encoding="utf-8",
)
os.chmod(temporary, 0o640)
os.replace(temporary, path)
@@ -1,204 +0,0 @@
"""Persistent and portable AIOS executor provider catalog.
The catalog contains descriptors only: provider names, protocol/adapter versions,
trust roles, capabilities and configuration schema identifiers. Credentials,
endpoints and runtime secrets are intentionally excluded and must be resolved in
the destination environment.
"""
from __future__ import annotations
import hashlib
import json
import os
import threading
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Mapping, Sequence
from .executor_protocol import (
EXECUTOR_PROTOCOL_VERSION,
DescriptorOnlyProvider,
ExecutorProvider,
ExecutorProviderDescriptor,
ExecutorProviderRegistry,
default_provider_registry,
)
CATALOG_SCHEMA_VERSION = "aios.executor.catalog.v1"
MIGRATION_SCHEMA_VERSION = "aios.executor.migration.v1"
def canonical_json_bytes(value: Mapping[str, Any]) -> bytes:
return json.dumps(
dict(value),
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
default=str,
).encode("utf-8")
def sha256_payload(value: Mapping[str, Any]) -> str:
return hashlib.sha256(canonical_json_bytes(value)).hexdigest()
@dataclass(frozen=True)
class ExecutorCatalogSnapshot:
catalog: Mapping[str, Any]
catalog_sha256: str
path: str
def as_dict(self) -> dict[str, Any]:
return {
"catalog": dict(self.catalog),
"catalog_sha256": self.catalog_sha256,
"path": self.path,
}
class FileExecutorCatalogStore:
"""Atomic file store for provider descriptors and migration packages."""
def __init__(self, path: str | Path) -> None:
self.path = Path(path).resolve()
self.path.parent.mkdir(parents=True, exist_ok=True)
self._lock = threading.RLock()
@staticmethod
def _validate_catalog(value: Mapping[str, Any]) -> ExecutorProviderRegistry:
if value.get("schema_version") != CATALOG_SCHEMA_VERSION:
raise ValueError("unsupported executor catalog schema")
if value.get("protocol_version") != EXECUTOR_PROTOCOL_VERSION:
raise ValueError("executor catalog protocol mismatch")
providers = value.get("providers")
if not isinstance(providers, Sequence) or isinstance(providers, (str, bytes)):
raise ValueError("executor catalog providers must be an array")
return ExecutorProviderRegistry.import_catalog(value)
def initialize_defaults(self) -> ExecutorCatalogSnapshot:
with self._lock:
if self.path.exists():
return self.load()
return self.save(default_provider_registry())
def load(self) -> ExecutorCatalogSnapshot:
with self._lock:
value = json.loads(self.path.read_text(encoding="utf-8"))
if not isinstance(value, Mapping):
raise ValueError("executor catalog must contain a JSON object")
registry = self._validate_catalog(value)
normalized = registry.export_catalog()
return ExecutorCatalogSnapshot(
catalog=normalized,
catalog_sha256=sha256_payload(normalized),
path=str(self.path),
)
def registry(
self,
*,
provider_factories: Mapping[str, ExecutorProvider] | None = None,
) -> ExecutorProviderRegistry:
snapshot = self.load()
return ExecutorProviderRegistry.import_catalog(
snapshot.catalog,
provider_factories=provider_factories,
)
def save(self, registry: ExecutorProviderRegistry) -> ExecutorCatalogSnapshot:
catalog = registry.export_catalog()
self._validate_catalog(catalog)
encoded = json.dumps(
catalog,
ensure_ascii=False,
sort_keys=True,
indent=2,
) + "\n"
with self._lock:
temporary = self.path.with_suffix(f".tmp-{os.getpid()}")
temporary.write_text(encoded, encoding="utf-8")
os.chmod(temporary, 0o640)
os.replace(temporary, self.path)
return ExecutorCatalogSnapshot(
catalog=catalog,
catalog_sha256=sha256_payload(catalog),
path=str(self.path),
)
def upsert_descriptor(
self,
descriptor: ExecutorProviderDescriptor,
) -> ExecutorCatalogSnapshot:
with self._lock:
if self.path.exists():
registry = self.registry()
else:
registry = default_provider_registry()
try:
registry.register(DescriptorOnlyProvider(descriptor), replace=True)
except KeyError:
registry.register(DescriptorOnlyProvider(descriptor))
return self.save(registry)
def export_migration_package(
self,
*,
source_environment: str,
required_executor_types: Sequence[str] = (),
) -> Mapping[str, Any]:
snapshot = self.load()
registry = ExecutorProviderRegistry.import_catalog(snapshot.catalog)
required = tuple(sorted({str(item).strip().lower() for item in required_executor_types if str(item).strip()}))
if required:
available = {item.executor_type for item in registry.list_descriptors()}
missing = sorted(set(required) - available)
if missing:
raise ValueError(f"migration package is missing providers: {missing}")
package = {
"schema_version": MIGRATION_SCHEMA_VERSION,
"source_environment": str(source_environment or "").strip(),
"catalog": snapshot.catalog,
"catalog_sha256": snapshot.catalog_sha256,
"required_executor_types": list(required),
"secret_values_included": False,
}
if not package["source_environment"]:
raise ValueError("source_environment is required")
return {
**package,
"package_sha256": sha256_payload(package),
}
@classmethod
def import_migration_package(
cls,
value: Mapping[str, Any],
*,
destination_path: str | Path,
destination_environment: str,
provider_factories: Mapping[str, ExecutorProvider] | None = None,
) -> ExecutorCatalogSnapshot:
if value.get("schema_version") != MIGRATION_SCHEMA_VERSION:
raise ValueError("unsupported executor migration schema")
if value.get("secret_values_included") is not False:
raise ValueError("executor migration package must not contain secrets")
package_without_hash = {key: item for key, item in value.items() if key != "package_sha256"}
if sha256_payload(package_without_hash) != value.get("package_sha256"):
raise ValueError("executor migration package hash mismatch")
catalog = value.get("catalog")
if not isinstance(catalog, Mapping):
raise ValueError("executor migration package has no catalog")
if sha256_payload(catalog) != value.get("catalog_sha256"):
raise ValueError("executor catalog hash mismatch")
registry = ExecutorProviderRegistry.import_catalog(
catalog,
provider_factories=provider_factories,
)
required = tuple(value.get("required_executor_types") or ())
available = {item.executor_type for item in registry.list_descriptors()}
missing = sorted(set(required) - available)
if missing:
raise ValueError(f"destination provider factories are missing: {missing}")
if not str(destination_environment or "").strip():
raise ValueError("destination_environment is required")
return cls(destination_path).save(registry)
@@ -1,441 +0,0 @@
"""Versioned, extensible executor protocol for AIOS external tasks.
The core lifecycle is executor-neutral. OpenClaw, Hermes, Codex and the scoped
ChatGPT supervisor are built-in
providers, not hard-coded protocol branches. Future providers register a
versioned descriptor and use the same Truth Registry, evidence, notification
and release-gate mechanisms without modifying the lifecycle state machine.
"""
from __future__ import annotations
import json
import re
import threading
from dataclasses import asdict, dataclass, field
from typing import Any, Mapping, Protocol, Sequence
from .external_task_control import ExternalTaskCoordinator
from .task_execution_registry import ExecutionTaskManifest, ExecutionTruthRecord
EXECUTOR_PROTOCOL_VERSION = "aios.executor.v1"
_PROVIDER_NAME = re.compile(r"^[a-z][a-z0-9._-]{1,63}$")
_VERSION = re.compile(r"^[0-9]+(?:\.[0-9]+){0,2}(?:[-+][A-Za-z0-9._-]+)?$")
DISCOVERABLE_STATES: Sequence[str] = ("DECLARED", "REGISTERED")
def normalize_executor(value: str) -> str:
normalized = str(value or "").strip().lower()
if not _PROVIDER_NAME.fullmatch(normalized):
raise ValueError("executor type must be a lowercase provider identifier")
return normalized
def _major(version: str) -> int:
normalized = str(version or "").strip()
if not _VERSION.fullmatch(normalized):
raise ValueError(f"invalid version: {version}")
return int(normalized.split(".", 1)[0])
@dataclass(frozen=True)
class ExecutorProviderDescriptor:
"""Portable provider metadata stored without credentials or runtime secrets."""
executor_type: str
adapter_version: str
protocol_version: str = EXECUTOR_PROTOCOL_VERSION
trust_roles: Sequence[str] = ()
capabilities: Sequence[str] = ()
configuration_schema: str = "aios.executor.config.v1"
portable: bool = True
metadata: Mapping[str, Any] = field(default_factory=dict)
def __post_init__(self) -> None:
object.__setattr__(self, "executor_type", normalize_executor(self.executor_type))
adapter_version = str(self.adapter_version or "").strip()
if not _VERSION.fullmatch(adapter_version):
raise ValueError("adapter_version must be version-like")
object.__setattr__(self, "adapter_version", adapter_version)
protocol_version = str(self.protocol_version or "").strip()
if not protocol_version.startswith("aios.executor.v"):
raise ValueError("unsupported executor protocol namespace")
int(protocol_version.rsplit("v", 1)[1])
object.__setattr__(self, "protocol_version", protocol_version)
roles = tuple(sorted({str(item).strip() for item in self.trust_roles if str(item).strip()}))
if not roles:
raise ValueError("provider must declare at least one trust role")
capabilities = tuple(
sorted({str(item).strip() for item in self.capabilities if str(item).strip()})
)
object.__setattr__(self, "trust_roles", roles)
object.__setattr__(self, "capabilities", capabilities)
object.__setattr__(
self,
"configuration_schema",
str(self.configuration_schema or "").strip(),
)
object.__setattr__(
self,
"metadata",
json.loads(json.dumps(dict(self.metadata), ensure_ascii=False, default=str)),
)
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
value["trust_roles"] = list(self.trust_roles)
value["capabilities"] = list(self.capabilities)
return value
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "ExecutorProviderDescriptor":
return cls(**dict(value))
def protocol_compatible_with(self, other: "ExecutorProviderDescriptor") -> bool:
return (
self.executor_type == other.executor_type
and self.protocol_version == other.protocol_version
and _major(self.adapter_version) == _major(other.adapter_version)
)
class ExternalExecutorAdapter(Protocol):
"""Runtime-specific implementation behind the common lifecycle."""
descriptor: ExecutorProviderDescriptor
def discover(self) -> Sequence[ExecutionTruthRecord]: ...
def execute(self, manifest: ExecutionTaskManifest) -> Mapping[str, Any]: ...
def notify_owner(self, payload: Mapping[str, Any]) -> str: ...
class ExecutorProvider(Protocol):
descriptor: ExecutorProviderDescriptor
def validate_manifest(self, manifest: ExecutionTaskManifest) -> None: ...
def build_adapter(self, configuration: Mapping[str, Any]) -> ExternalExecutorAdapter: ...
class DescriptorOnlyProvider:
"""Default provider used when lifecycle integration is supplied externally."""
def __init__(self, descriptor: ExecutorProviderDescriptor) -> None:
self.descriptor = descriptor
def validate_manifest(self, manifest: ExecutionTaskManifest) -> None:
if normalize_executor(manifest.executor) != self.descriptor.executor_type:
raise ValueError("manifest executor does not match provider")
def build_adapter(self, configuration: Mapping[str, Any]) -> ExternalExecutorAdapter:
raise RuntimeError(
f"provider {self.descriptor.executor_type} requires an external adapter factory"
)
class ExecutorProviderRegistry:
"""Thread-safe registry supporting runtime registration and catalog migration."""
def __init__(self, providers: Sequence[ExecutorProvider] = ()) -> None:
self._lock = threading.RLock()
self._providers: dict[str, ExecutorProvider] = {}
for provider in providers:
self.register(provider)
def register(self, provider: ExecutorProvider, *, replace: bool = False) -> None:
descriptor = provider.descriptor
key = normalize_executor(descriptor.executor_type)
with self._lock:
current = self._providers.get(key)
if current is not None and not replace:
raise ValueError(f"executor provider already registered: {key}")
if current is not None and not descriptor.protocol_compatible_with(
current.descriptor
):
raise ValueError(
f"replacement provider is incompatible with registered provider: {key}"
)
self._providers[key] = provider
def unregister(self, executor_type: str) -> ExecutorProvider:
key = normalize_executor(executor_type)
with self._lock:
try:
return self._providers.pop(key)
except KeyError as exc:
raise KeyError(f"executor provider is not registered: {key}") from exc
def get(self, executor_type: str) -> ExecutorProvider:
key = normalize_executor(executor_type)
with self._lock:
provider = self._providers.get(key)
if provider is None:
raise KeyError(f"executor provider is not registered: {key}")
return provider
def descriptor(self, executor_type: str) -> ExecutorProviderDescriptor:
return self.get(executor_type).descriptor
def validate_manifest(self, manifest: ExecutionTaskManifest) -> None:
self.get(manifest.executor).validate_manifest(manifest)
def trusted(
self,
*,
executor_type: str,
roles: Sequence[str],
platform_admin: bool = False,
) -> bool:
if platform_admin or "platform_admin" in set(roles):
return True
descriptor = self.descriptor(executor_type)
return bool(set(descriptor.trust_roles).intersection(set(roles)))
def list_descriptors(self) -> tuple[ExecutorProviderDescriptor, ...]:
with self._lock:
return tuple(
self._providers[key].descriptor for key in sorted(self._providers)
)
def export_catalog(self) -> Mapping[str, Any]:
return {
"schema_version": "aios.executor.catalog.v1",
"protocol_version": EXECUTOR_PROTOCOL_VERSION,
"providers": [item.as_dict() for item in self.list_descriptors()],
}
@classmethod
def import_catalog(
cls,
value: Mapping[str, Any],
*,
provider_factories: Mapping[str, ExecutorProvider] | None = None,
) -> "ExecutorProviderRegistry":
if value.get("schema_version") != "aios.executor.catalog.v1":
raise ValueError("unsupported executor catalog schema")
if value.get("protocol_version") != EXECUTOR_PROTOCOL_VERSION:
raise ValueError("executor catalog protocol mismatch")
factories = dict(provider_factories or {})
providers: list[ExecutorProvider] = []
for raw in value.get("providers") or ():
descriptor = ExecutorProviderDescriptor.from_dict(raw)
provider = factories.get(descriptor.executor_type)
if provider is not None:
if not provider.descriptor.protocol_compatible_with(descriptor):
raise ValueError(
f"provider factory does not match imported descriptor: {descriptor.executor_type}"
)
providers.append(provider)
else:
providers.append(DescriptorOnlyProvider(descriptor))
return cls(providers)
BUILTIN_PROVIDER_DESCRIPTORS: Sequence[ExecutorProviderDescriptor] = (
ExecutorProviderDescriptor(
executor_type="openclaw",
adapter_version="2.1.0",
trust_roles=("runtime", "openclaw_executor"),
capabilities=(
"deploy",
"http_runtime",
"owner_notification",
"healthcheck",
"rollback",
),
configuration_schema="aios.executor.openclaw.config.v1",
metadata={"official": True},
),
ExecutorProviderDescriptor(
executor_type="hermes",
adapter_version="2.1.0",
trust_roles=("runtime", "hermes_executor"),
capabilities=(
"queue",
"lease",
"worker",
"runtime_validation",
"restart_recovery",
),
configuration_schema="aios.executor.hermes.config.v1",
metadata={"official": True},
),
ExecutorProviderDescriptor(
executor_type="codex",
adapter_version="2.1.0",
trust_roles=("codex", "codex_executor"),
capabilities=("code", "test", "docs", "gitops_patch", "security_review"),
configuration_schema="aios.executor.codex.config.v1",
metadata={"official": True},
),
ExecutorProviderDescriptor(
executor_type="chatgpt-supervisor",
adapter_version="2.1.0",
trust_roles=("chatgpt_supervisor",),
capabilities=(
"acceptance",
"deploy",
"evidence",
"healthcheck",
"rollback",
"wechat_simulation",
),
configuration_schema="aios.executor.chatgpt-supervisor.config.v1",
metadata={"official": True, "production_mutation_allowed": False},
),
)
def default_provider_registry() -> ExecutorProviderRegistry:
return ExecutorProviderRegistry(
[DescriptorOnlyProvider(item) for item in BUILTIN_PROVIDER_DESCRIPTORS]
)
def trusted_for_executor(
*,
executor: str,
roles: Sequence[str],
platform_admin: bool = False,
registry: ExecutorProviderRegistry | None = None,
) -> bool:
return (registry or default_provider_registry()).trusted(
executor_type=executor,
roles=roles,
platform_admin=platform_admin,
)
@dataclass(frozen=True)
class ExecutorIdentity:
executor_type: str
executor_id: str
executor_run_id: str
protocol_version: str = EXECUTOR_PROTOCOL_VERSION
def __post_init__(self) -> None:
object.__setattr__(self, "executor_type", normalize_executor(self.executor_type))
for name in ("executor_id", "executor_run_id"):
value = str(getattr(self, name) or "").strip()
if not value:
raise ValueError(f"{name} is required")
object.__setattr__(self, name, value)
if self.protocol_version != EXECUTOR_PROTOCOL_VERSION:
raise ValueError("executor identity protocol mismatch")
class ExecutorLifecycleSession:
"""Executor-neutral lifecycle facade used by every registered provider."""
def __init__(
self,
coordinator: ExternalTaskCoordinator,
identity: ExecutorIdentity,
*,
registry: ExecutorProviderRegistry | None = None,
) -> None:
self.coordinator = coordinator
self.identity = identity
self.registry = registry or default_provider_registry()
self.registry.get(identity.executor_type)
def _assert_manifest(self, task_id: str) -> ExecutionTaskManifest:
record = self.coordinator.registry.get(task_id)
manifest = record.manifest
self.registry.validate_manifest(manifest)
if normalize_executor(manifest.executor) != self.identity.executor_type:
raise PermissionError(
f"task executor is {manifest.executor}, not {self.identity.executor_type}"
)
return manifest
def register(self, task_id: str) -> ExecutionTruthRecord:
self._assert_manifest(task_id)
return self.coordinator.register(
task_id,
executor_run_id=self.identity.executor_run_id,
)
def retry_not_started(self, task_id: str) -> ExecutionTruthRecord:
self._assert_manifest(task_id)
return self.coordinator.retry_not_started(
task_id,
executor_run_id=self.identity.executor_run_id,
)
def claim(self, task_id: str) -> ExecutionTruthRecord:
self._assert_manifest(task_id)
return self.coordinator.claim(
task_id,
executor_run_id=self.identity.executor_run_id,
executor_id=self.identity.executor_id,
)
def start(self, task_id: str) -> ExecutionTruthRecord:
self._assert_manifest(task_id)
return self.coordinator.start(
task_id,
executor_run_id=self.identity.executor_run_id,
executor_id=self.identity.executor_id,
)
def heartbeat(self, task_id: str, *, current_step: int) -> ExecutionTruthRecord:
self._assert_manifest(task_id)
return self.coordinator.heartbeat(
task_id,
executor_run_id=self.identity.executor_run_id,
executor_id=self.identity.executor_id,
current_step=current_step,
)
def succeed(
self,
task_id: str,
*,
evidence: Mapping[str, str],
) -> ExecutionTruthRecord:
self._assert_manifest(task_id)
return self.coordinator.succeed(
task_id,
executor_run_id=self.identity.executor_run_id,
executor_id=self.identity.executor_id,
evidence=evidence,
)
def fail(
self,
task_id: str,
*,
error: Mapping[str, Any],
evidence: Mapping[str, str] | None = None,
) -> ExecutionTruthRecord:
self._assert_manifest(task_id)
return self.coordinator.fail(
task_id,
executor_run_id=self.identity.executor_run_id,
executor_id=self.identity.executor_id,
error=error,
evidence=evidence,
)
class OpenClawExecutorSession(ExecutorLifecycleSession):
def __init__(self, coordinator: ExternalTaskCoordinator, *, executor_id: str, executor_run_id: str) -> None:
super().__init__(
coordinator,
ExecutorIdentity("openclaw", executor_id, executor_run_id),
)
class HermesExecutorSession(ExecutorLifecycleSession):
def __init__(self, coordinator: ExternalTaskCoordinator, *, executor_id: str, executor_run_id: str) -> None:
super().__init__(
coordinator,
ExecutorIdentity("hermes", executor_id, executor_run_id),
)
class CodexExecutorSession(ExecutorLifecycleSession):
def __init__(self, coordinator: ExternalTaskCoordinator, *, executor_id: str, executor_run_id: str) -> None:
super().__init__(
coordinator,
ExecutorIdentity("codex", executor_id, executor_run_id),
)
@@ -1,313 +0,0 @@
"""Provider-neutral worker runtime for AIOS external executors.
Providers implement only step execution and notification delivery. Discovery,
dependency gates, lifecycle truth, heartbeat, evidence validation and durable
notification acknowledgement are identical for every current and future
executor Provider.
"""
from __future__ import annotations
import hashlib
import json
import uuid
from dataclasses import dataclass, field
from typing import Any, Mapping, Protocol, Sequence
from .execution_plan import ExecutionPlanCoordinator, ExecutionStageManifest
from .executor_protocol import (
ExecutorIdentity,
ExecutorLifecycleSession,
ExecutorProviderRegistry,
normalize_executor,
)
from .external_task_control import ExternalTaskCoordinator, NotificationRecord
from .task_execution_registry import ExecutionTruthState
class StepRunner(Protocol):
def run_step(
self,
*,
plan_id: str,
stage: ExecutionStageManifest,
step: str,
step_number: int,
executor_id: str,
executor_run_id: str,
) -> Mapping[str, str]: ...
class NotificationSender(Protocol):
def send(self, notification: NotificationRecord) -> str: ...
@dataclass(frozen=True)
class StageRunResult:
plan_id: str
stage_id: str
task_id: str
executor_type: str
executor_id: str
executor_run_id: str
state: str
evidence: Mapping[str, str] = field(default_factory=dict)
error: Mapping[str, Any] = field(default_factory=dict)
delivered_notifications: Sequence[str] = ()
def as_dict(self) -> dict[str, Any]:
return {
"plan_id": self.plan_id,
"stage_id": self.stage_id,
"task_id": self.task_id,
"executor_type": self.executor_type,
"executor_id": self.executor_id,
"executor_run_id": self.executor_run_id,
"state": self.state,
"evidence": dict(self.evidence),
"error": dict(self.error),
"delivered_notifications": list(self.delivered_notifications),
}
class UnifiedExecutorWorker:
"""Dependency-aware worker shared by every registered Provider."""
def __init__(
self,
*,
plan_coordinator: ExecutionPlanCoordinator,
task_coordinator: ExternalTaskCoordinator,
providers: ExecutorProviderRegistry,
executor_type: str,
executor_id: str,
step_runner: StepRunner,
notification_sender: NotificationSender,
) -> None:
self.plan_coordinator = plan_coordinator
self.task_coordinator = task_coordinator
self.providers = providers
self.executor_type = normalize_executor(executor_type)
self.executor_id = str(executor_id or "").strip()
if not self.executor_id:
raise ValueError("executor_id is required")
self.providers.get(self.executor_type)
self.step_runner = step_runner
self.notification_sender = notification_sender
def _session(self, executor_run_id: str) -> ExecutorLifecycleSession:
return ExecutorLifecycleSession(
self.task_coordinator,
ExecutorIdentity(
executor_type=self.executor_type,
executor_id=self.executor_id,
executor_run_id=executor_run_id,
),
registry=self.providers,
)
def _notification_candidates(self, task_id: str) -> tuple[NotificationRecord, ...]:
"""Return idempotent delivery candidates, including prior failures.
The outbox has a unique ``(task_id, event_type)`` key. Retrying therefore
reuses the same notification ID and cannot create duplicate STARTED or
SUCCEEDED events. A later successful acknowledgement replaces FAILED
with DELIVERED and records the receipt hash.
"""
candidates: dict[str, NotificationRecord] = {}
for status in ("PENDING", "FAILED"):
for item in self.task_coordinator.notifications.list(
task_id=task_id,
status=status,
):
candidates[item.notification_id] = item
return tuple(
candidates[key]
for key in sorted(
candidates,
key=lambda item: (
candidates[item].created_at,
candidates[item].notification_id,
),
)
)
def _deliver_notifications(self, task_id: str) -> tuple[str, ...]:
delivered: list[str] = []
for item in self._notification_candidates(task_id):
try:
receipt = self.notification_sender.send(item)
acknowledged = self.task_coordinator.notifications.acknowledge(
item.notification_id,
receipt_sha256=receipt,
)
delivered.append(acknowledged.event_type)
except Exception as exc:
self.task_coordinator.notifications.fail(
item.notification_id,
error=f"{type(exc).__name__}: {exc}",
)
return tuple(sorted(set(delivered)))
def retry_failed_notifications(self, *, task_id: str) -> tuple[str, ...]:
"""Retry failed owner notifications without changing task truth."""
return self._deliver_notifications(task_id)
def run_ready(
self,
*,
plan_id: str,
max_stages: int = 1,
) -> list[StageRunResult]:
if max_stages < 1:
raise ValueError("max_stages must be positive")
ready = self.plan_coordinator.discover_ready(
plan_id=plan_id,
executor_type=self.executor_type,
)[:max_stages]
return [
self.run_stage(
plan_id=plan_id,
stage_id=item["stage"]["stage_id"],
)
for item in ready
]
def run_stage(
self,
*,
plan_id: str,
stage_id: str,
executor_run_id: str | None = None,
) -> StageRunResult:
plan = self.plan_coordinator.plan_store.get(plan_id)
stage = plan.stage(stage_id)
if stage.executor_type != self.executor_type:
raise PermissionError(
f"stage executor is {stage.executor_type}, not {self.executor_type}"
)
dependency = self.plan_coordinator.dependency_gate(plan_id, stage_id)
if not dependency["passed"]:
raise RuntimeError("stage dependencies are not satisfied")
run_id = executor_run_id or f"{self.executor_type}_run_{uuid.uuid4().hex}"
session = self._session(run_id)
record = self.task_coordinator.registry.get(stage.task_id)
evidence: dict[str, str] = dict(record.evidence)
delivered: list[str] = []
try:
if record.state == ExecutionTruthState.DECLARED:
record = session.register(stage.task_id)
if record.state == ExecutionTruthState.REGISTERED:
record = session.claim(stage.task_id)
if record.state == ExecutionTruthState.CLAIMED:
record = session.start(stage.task_id)
delivered.extend(self._deliver_notifications(stage.task_id))
if record.state != ExecutionTruthState.RUNNING:
raise RuntimeError(
f"stage cannot execute from state {record.state.value}"
)
# Re-running the current step is intentional: Provider steps must be
# idempotent and a crash between side effect and heartbeat cannot be
# safely interpreted as completed work.
start_step = max(1, record.current_step)
for number, step in enumerate(stage.steps, start=1):
if number < start_step:
continue
produced = self.step_runner.run_step(
plan_id=plan_id,
stage=stage,
step=step,
step_number=number,
executor_id=self.executor_id,
executor_run_id=run_id,
)
if not isinstance(produced, Mapping):
raise TypeError("step runner must return an evidence mapping")
evidence.update(
{
str(key).strip(): str(value).strip().lower()
for key, value in produced.items()
}
)
record = session.heartbeat(stage.task_id, current_step=number)
record = session.succeed(stage.task_id, evidence=evidence)
delivered.extend(self._deliver_notifications(stage.task_id))
return StageRunResult(
plan_id=plan_id,
stage_id=stage.stage_id,
task_id=stage.task_id,
executor_type=self.executor_type,
executor_id=self.executor_id,
executor_run_id=run_id,
state=record.state.value,
evidence=dict(record.evidence),
delivered_notifications=tuple(sorted(set(delivered))),
)
except Exception as exc:
current = self.task_coordinator.registry.get(stage.task_id)
if current.state in {
ExecutionTruthState.REGISTERED,
ExecutionTruthState.CLAIMED,
ExecutionTruthState.RUNNING,
}:
current = session.fail(
stage.task_id,
error={
"code": "executor_stage_failed",
"error_type": type(exc).__name__,
"message": str(exc),
},
evidence=evidence or None,
)
delivered.extend(self._deliver_notifications(stage.task_id))
return StageRunResult(
plan_id=plan_id,
stage_id=stage.stage_id,
task_id=stage.task_id,
executor_type=self.executor_type,
executor_id=self.executor_id,
executor_run_id=run_id,
state=current.state.value,
evidence=dict(current.evidence),
error=dict(current.error),
delivered_notifications=tuple(sorted(set(delivered))),
)
class JsonEvidenceStepRunner:
"""Adapter for handlers returning evidence-name to SHA256 mappings."""
def __init__(self, handler) -> None:
self.handler = handler
def run_step(self, **values: Any) -> Mapping[str, str]:
result = self.handler(**values)
if not isinstance(result, Mapping):
raise TypeError("step handler must return a mapping")
return {str(key): str(value) for key, value in result.items()}
class JsonReceiptNotificationSender:
"""Adapter for senders returning a SHA256 or receipt JSON object."""
def __init__(self, sender) -> None:
self.sender = sender
def send(self, notification: NotificationRecord) -> str:
result = self.sender(notification.as_dict())
if isinstance(result, str):
return result
if not isinstance(result, Mapping):
raise TypeError(
"notification sender must return SHA256 or a receipt object"
)
encoded = json.dumps(
dict(result),
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
default=str,
).encode("utf-8")
return hashlib.sha256(encoded).hexdigest()
@@ -1,266 +0,0 @@
#!/usr/bin/env python3
"""CLI for durable external-task truth and notification receipts.
The CLI is suitable for OpenClaw workers, systemd jobs and release gates. It
never infers execution from the presence of a manifest: each lifecycle action
must be recorded explicitly in the shared SQLite registry.
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from pathlib import Path
from typing import Any, Mapping, Sequence
from .external_task_control import ExternalTaskCoordinator
from .external_task_query import list_records
def _json_mapping(value: str | None, path: str | None) -> Mapping[str, Any]:
if value and path:
raise ValueError("inline JSON and JSON file are mutually exclusive")
if path:
parsed = json.loads(Path(path).read_text(encoding="utf-8"))
else:
parsed = json.loads(value or "{}")
if not isinstance(parsed, Mapping):
raise ValueError("JSON value must be an object")
return dict(parsed)
def _coordinator(args: argparse.Namespace) -> ExternalTaskCoordinator:
return ExternalTaskCoordinator.from_file(args.store_file)
def _print(value: Any) -> None:
print(json.dumps(value, ensure_ascii=False, sort_keys=True, indent=2, default=str))
def cmd_declare(args: argparse.Namespace) -> int:
record = _coordinator(args).declare_file(args.manifest)
_print(record.as_dict())
return 0
def cmd_discover(args: argparse.Namespace) -> int:
records = list_records(
args.store_file,
executor=args.executor,
states=tuple(args.state or ()),
)
_print({"count": len(records), "items": [item.as_dict() for item in records]})
return 0
def cmd_register(args: argparse.Namespace) -> int:
_print(_coordinator(args).register(args.task_id, executor_run_id=args.executor_run_id).as_dict())
return 0
def cmd_claim(args: argparse.Namespace) -> int:
_print(
_coordinator(args).claim(
args.task_id,
executor_run_id=args.executor_run_id,
executor_id=args.executor_id,
).as_dict()
)
return 0
def cmd_start(args: argparse.Namespace) -> int:
_print(
_coordinator(args).start(
args.task_id,
executor_run_id=args.executor_run_id,
executor_id=args.executor_id,
).as_dict()
)
return 0
def cmd_heartbeat(args: argparse.Namespace) -> int:
_print(
_coordinator(args).heartbeat(
args.task_id,
executor_run_id=args.executor_run_id,
executor_id=args.executor_id,
current_step=args.current_step,
).as_dict()
)
return 0
def cmd_succeed(args: argparse.Namespace) -> int:
evidence = _json_mapping(args.evidence_json, args.evidence_file)
_print(
_coordinator(args).succeed(
args.task_id,
executor_run_id=args.executor_run_id,
executor_id=args.executor_id,
evidence={str(key): str(value) for key, value in evidence.items()},
).as_dict()
)
return 0
def cmd_fail(args: argparse.Namespace) -> int:
error = _json_mapping(args.error_json, args.error_file)
evidence = None
if args.evidence_json or args.evidence_file:
evidence_value = _json_mapping(args.evidence_json, args.evidence_file)
evidence = {str(key): str(value) for key, value in evidence_value.items()}
_print(
_coordinator(args).fail(
args.task_id,
executor_run_id=args.executor_run_id,
executor_id=args.executor_id,
error=error,
evidence=evidence,
).as_dict()
)
return 0
def cmd_status(args: argparse.Namespace) -> int:
_print(_coordinator(args).status(args.task_id))
return 0
def cmd_gate(args: argparse.Namespace) -> int:
value = _coordinator(args).gate(args.task_id)
_print(value)
return 0 if value["passed"] else 3
def cmd_notifications(args: argparse.Namespace) -> int:
items = _coordinator(args).notifications.list(task_id=args.task_id, status=args.status)
_print({"count": len(items), "items": [item.as_dict() for item in items]})
return 0
def cmd_ack(args: argparse.Namespace) -> int:
_print(
_coordinator(args).notifications.acknowledge(
args.notification_id,
receipt_sha256=args.receipt_sha256,
).as_dict()
)
return 0
def common(parser: argparse.ArgumentParser) -> None:
parser.add_argument(
"--store-file",
default=os.getenv(
"AIOS_EXTERNAL_TASK_STORE_FILE",
"/var/lib/aios-v2/external-task-truth.sqlite3",
),
)
def lifecycle(parser: argparse.ArgumentParser) -> None:
parser.add_argument("--task-id", required=True)
parser.add_argument("--executor-run-id", required=True)
parser.add_argument("--executor-id", required=True)
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description="AIOS external task truth CLI")
commands = parser.add_subparsers(dest="command", required=True)
declare = commands.add_parser("declare")
common(declare)
declare.add_argument("--manifest", required=True)
declare.set_defaults(handler=cmd_declare)
discover = commands.add_parser("discover")
common(discover)
discover.add_argument("--executor", default="openclaw")
discover.add_argument("--state", action="append")
discover.set_defaults(handler=cmd_discover)
register = commands.add_parser("register")
common(register)
register.add_argument("--task-id", required=True)
register.add_argument("--executor-run-id", required=True)
register.set_defaults(handler=cmd_register)
claim = commands.add_parser("claim")
common(claim)
lifecycle(claim)
claim.set_defaults(handler=cmd_claim)
start = commands.add_parser("start")
common(start)
lifecycle(start)
start.set_defaults(handler=cmd_start)
heartbeat = commands.add_parser("heartbeat")
common(heartbeat)
lifecycle(heartbeat)
heartbeat.add_argument("--current-step", type=int, required=True)
heartbeat.set_defaults(handler=cmd_heartbeat)
succeed = commands.add_parser("succeed")
common(succeed)
lifecycle(succeed)
evidence = succeed.add_mutually_exclusive_group(required=True)
evidence.add_argument("--evidence-json")
evidence.add_argument("--evidence-file")
succeed.set_defaults(handler=cmd_succeed)
fail = commands.add_parser("fail")
common(fail)
lifecycle(fail)
error = fail.add_mutually_exclusive_group(required=True)
error.add_argument("--error-json")
error.add_argument("--error-file")
fail_evidence = fail.add_mutually_exclusive_group()
fail_evidence.add_argument("--evidence-json")
fail_evidence.add_argument("--evidence-file")
fail.set_defaults(handler=cmd_fail)
status = commands.add_parser("status")
common(status)
status.add_argument("--task-id", required=True)
status.set_defaults(handler=cmd_status)
gate = commands.add_parser("gate")
common(gate)
gate.add_argument("--task-id", required=True)
gate.set_defaults(handler=cmd_gate)
notifications = commands.add_parser("notifications")
common(notifications)
notifications.add_argument("--task-id")
notifications.add_argument("--status")
notifications.set_defaults(handler=cmd_notifications)
ack = commands.add_parser("ack-notification")
common(ack)
ack.add_argument("--notification-id", required=True)
ack.add_argument("--receipt-sha256", required=True)
ack.set_defaults(handler=cmd_ack)
return parser
def main(argv: Sequence[str] | None = None) -> int:
args = build_parser().parse_args(argv)
try:
return int(args.handler(args))
except Exception as exc:
print(
json.dumps(
{"status": "error", "error": type(exc).__name__, "message": str(exc)},
ensure_ascii=False,
),
file=sys.stderr,
)
return 1
if __name__ == "__main__":
raise SystemExit(main())
@@ -1,420 +0,0 @@
"""External executor coordination, notification outbox and release truth gate.
OpenClaw/Codex task declarations are not execution evidence. This module wraps
``SQLiteExecutionTruthRegistry`` with a durable notification outbox. A successful
release gate requires both terminal task evidence and acknowledged STARTED /
SUCCEEDED owner notifications when those events are required by the manifest.
"""
from __future__ import annotations
import hashlib
import json
import re
import sqlite3
import threading
import uuid
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Any, Mapping, Sequence
from .task_execution_registry import (
ExecutionTaskManifest,
ExecutionTruthRecord,
ExecutionTruthState,
SQLiteExecutionTruthRegistry,
utc_now,
)
_SHA256 = re.compile(r"^[0-9a-f]{64}$")
def sha256_value(value: str, label: str) -> str:
normalized = str(value or "").strip().lower()
if not _SHA256.fullmatch(normalized):
raise ValueError(f"{label} must be a lowercase SHA256 hex digest")
return normalized
@dataclass(frozen=True)
class NotificationRecord:
notification_id: str
task_id: str
event_type: str
target_ref: str
payload: Mapping[str, Any]
status: str
created_at: str
delivered_at: str | None = None
receipt_sha256: str | None = None
error: str | None = None
def as_dict(self) -> dict[str, Any]:
return asdict(self)
class SQLiteNotificationOutbox:
"""Durable owner-notification delivery and acknowledgement records."""
def __init__(self, path: str | Path) -> None:
self.path = Path(path).resolve()
self.path.parent.mkdir(parents=True, exist_ok=True)
self._lock = threading.RLock()
with self._connect() as connection:
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS external_task_notifications (
notification_id TEXT PRIMARY KEY,
task_id TEXT NOT NULL,
event_type TEXT NOT NULL,
target_ref TEXT NOT NULL,
payload_json TEXT NOT NULL,
status TEXT NOT NULL,
created_at TEXT NOT NULL,
delivered_at TEXT,
receipt_sha256 TEXT,
error TEXT,
UNIQUE(task_id, event_type)
);
CREATE INDEX IF NOT EXISTS idx_external_notification_status
ON external_task_notifications(status, created_at);
"""
)
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(str(self.path), timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA synchronous=FULL")
return connection
@staticmethod
def _decode(row: sqlite3.Row) -> NotificationRecord:
return NotificationRecord(
notification_id=row["notification_id"],
task_id=row["task_id"],
event_type=row["event_type"],
target_ref=row["target_ref"],
payload=json.loads(row["payload_json"]),
status=row["status"],
created_at=row["created_at"],
delivered_at=row["delivered_at"],
receipt_sha256=row["receipt_sha256"],
error=row["error"],
)
def enqueue(
self,
*,
task_id: str,
event_type: str,
target_ref: str,
payload: Mapping[str, Any],
) -> NotificationRecord:
now = utc_now()
record = NotificationRecord(
notification_id=f"notify_{uuid.uuid4().hex}",
task_id=str(task_id).strip(),
event_type=str(event_type).strip().upper(),
target_ref=str(target_ref).strip(),
payload=json.loads(json.dumps(dict(payload), ensure_ascii=False, default=str)),
status="PENDING",
created_at=now,
)
if not record.task_id or not record.event_type or not record.target_ref:
raise ValueError("task_id, event_type and target_ref are required")
with self._lock, self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
existing = connection.execute(
"SELECT * FROM external_task_notifications WHERE task_id = ? AND event_type = ?",
(record.task_id, record.event_type),
).fetchone()
if existing is not None:
connection.rollback()
return self._decode(existing)
connection.execute(
"""
INSERT INTO external_task_notifications
(notification_id, task_id, event_type, target_ref, payload_json,
status, created_at, delivered_at, receipt_sha256, error)
VALUES (?, ?, ?, ?, ?, ?, ?, NULL, NULL, NULL)
""",
(
record.notification_id,
record.task_id,
record.event_type,
record.target_ref,
json.dumps(record.payload, ensure_ascii=False, sort_keys=True, separators=(",", ":")),
record.status,
record.created_at,
),
)
connection.commit()
return record
def get(self, notification_id: str) -> NotificationRecord:
with self._lock, self._connect() as connection:
row = connection.execute(
"SELECT * FROM external_task_notifications WHERE notification_id = ?",
(str(notification_id).strip(),),
).fetchone()
if row is None:
raise KeyError("notification not found")
return self._decode(row)
def acknowledge(self, notification_id: str, *, receipt_sha256: str) -> NotificationRecord:
digest = sha256_value(receipt_sha256, "receipt_sha256")
with self._lock, self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
row = connection.execute(
"SELECT * FROM external_task_notifications WHERE notification_id = ?",
(str(notification_id).strip(),),
).fetchone()
if row is None:
connection.rollback()
raise KeyError("notification not found")
current = self._decode(row)
if current.status == "DELIVERED":
if current.receipt_sha256 != digest:
connection.rollback()
raise RuntimeError("notification already acknowledged with another receipt")
connection.rollback()
return current
delivered_at = utc_now()
connection.execute(
"""
UPDATE external_task_notifications
SET status = 'DELIVERED', delivered_at = ?, receipt_sha256 = ?, error = NULL
WHERE notification_id = ?
""",
(delivered_at, digest, current.notification_id),
)
connection.commit()
return self.get(current.notification_id)
def fail(self, notification_id: str, *, error: str) -> NotificationRecord:
message = str(error or "").strip()
if not message:
raise ValueError("notification error is required")
with self._lock, self._connect() as connection:
connection.execute(
"UPDATE external_task_notifications SET status = 'FAILED', error = ? WHERE notification_id = ?",
(message, str(notification_id).strip()),
)
connection.commit()
return self.get(notification_id)
def list(
self,
*,
task_id: str | None = None,
status: str | None = None,
) -> list[NotificationRecord]:
conditions: list[str] = []
values: list[str] = []
if task_id:
conditions.append("task_id = ?")
values.append(str(task_id).strip())
if status:
conditions.append("status = ?")
values.append(str(status).strip().upper())
where = f" WHERE {' AND '.join(conditions)}" if conditions else ""
with self._lock, self._connect() as connection:
rows = connection.execute(
f"SELECT * FROM external_task_notifications{where} ORDER BY created_at, notification_id",
tuple(values),
).fetchall()
return [self._decode(row) for row in rows]
class ExternalTaskCoordinator:
"""Single safe lifecycle used by external executors and release gates."""
def __init__(
self,
registry: SQLiteExecutionTruthRegistry,
notifications: SQLiteNotificationOutbox,
) -> None:
self.registry = registry
self.notifications = notifications
@classmethod
def from_file(cls, path: str | Path) -> "ExternalTaskCoordinator":
return cls(
SQLiteExecutionTruthRegistry(path),
SQLiteNotificationOutbox(path),
)
def declare(self, manifest: ExecutionTaskManifest) -> ExecutionTruthRecord:
return self.registry.declare(manifest)
def declare_file(self, path: str | Path) -> ExecutionTruthRecord:
value = json.loads(Path(path).read_text(encoding="utf-8"))
if not isinstance(value, Mapping):
raise ValueError("external task manifest must be a JSON object")
return self.declare(ExecutionTaskManifest.from_dict(value))
def register(self, task_id: str, *, executor_run_id: str) -> ExecutionTruthRecord:
return self.registry.register(task_id, executor_run_id=executor_run_id)
def retry_not_started(
self,
task_id: str,
*,
executor_run_id: str,
) -> ExecutionTruthRecord:
return self.registry.retry_not_started(
task_id,
executor_run_id=executor_run_id,
)
def claim(
self,
task_id: str,
*,
executor_run_id: str,
executor_id: str,
) -> ExecutionTruthRecord:
return self.registry.claim(
task_id,
executor_run_id=executor_run_id,
executor_id=executor_id,
)
def _notify(self, record: ExecutionTruthRecord, event_type: str) -> NotificationRecord | None:
event = event_type.upper()
if event not in set(record.manifest.notification_events):
return None
return self.notifications.enqueue(
task_id=record.task_id,
event_type=event,
target_ref=record.manifest.owner_notification_ref,
payload={
"task_id": record.task_id,
"target_version": record.manifest.target_version,
"executor": record.manifest.executor,
"executor_id": record.executor_id,
"executor_run_id": record.executor_run_id,
"state": record.state.value,
"current_step": record.current_step,
"total_steps": len(record.manifest.steps),
"timestamp": utc_now(),
},
)
def start(
self,
task_id: str,
*,
executor_run_id: str,
executor_id: str,
) -> ExecutionTruthRecord:
record = self.registry.start(
task_id,
executor_run_id=executor_run_id,
executor_id=executor_id,
)
self._notify(record, "STARTED")
return record
def heartbeat(
self,
task_id: str,
*,
executor_run_id: str,
executor_id: str,
current_step: int,
) -> ExecutionTruthRecord:
return self.registry.heartbeat(
task_id,
executor_run_id=executor_run_id,
executor_id=executor_id,
current_step=current_step,
)
def succeed(
self,
task_id: str,
*,
executor_run_id: str,
executor_id: str,
evidence: Mapping[str, str],
) -> ExecutionTruthRecord:
normalized = {
str(key).strip(): sha256_value(value, f"evidence[{key}]")
for key, value in evidence.items()
}
record = self.registry.succeed(
task_id,
executor_run_id=executor_run_id,
executor_id=executor_id,
evidence=normalized,
)
self._notify(record, "SUCCEEDED")
return record
def fail(
self,
task_id: str,
*,
executor_run_id: str,
executor_id: str,
error: Mapping[str, Any],
evidence: Mapping[str, str] | None = None,
) -> ExecutionTruthRecord:
normalized = None
if evidence is not None:
normalized = {
str(key).strip(): sha256_value(value, f"evidence[{key}]")
for key, value in evidence.items()
}
record = self.registry.fail(
task_id,
executor_run_id=executor_run_id,
executor_id=executor_id,
error=error,
evidence=normalized,
)
self._notify(record, "FAILED")
return record
def gate(self, task_id: str) -> Mapping[str, Any]:
before = self.registry.get(task_id)
truth = self.registry.gate(task_id)
after = self.registry.get(task_id)
if after.state != before.state and after.state in {
ExecutionTruthState.NOT_STARTED,
ExecutionTruthState.STALLED,
}:
self._notify(after, after.state.value)
notifications = self.notifications.list(task_id=task_id)
delivered = {item.event_type for item in notifications if item.status == "DELIVERED"}
required_success_notifications = {
item
for item in ("STARTED", "SUCCEEDED")
if item in set(after.manifest.notification_events)
}
notification_complete = required_success_notifications.issubset(delivered)
passed = bool(truth["passed"] and notification_complete)
return {
**dict(truth),
"passed": passed,
"release_blocking": not passed,
"notification_complete": notification_complete,
"required_success_notifications": sorted(required_success_notifications),
"delivered_notifications": sorted(delivered),
"notifications": [item.as_dict() for item in notifications],
}
def status(self, task_id: str) -> Mapping[str, Any]:
gate = self.gate(task_id)
record = self.registry.get(task_id)
return {
"record": record.as_dict(),
"gate": gate,
"events": self.registry.events(task_id),
}
@staticmethod
def content_receipt(value: Mapping[str, Any]) -> str:
encoded = json.dumps(dict(value), ensure_ascii=False, sort_keys=True, separators=(",", ":"), default=str).encode("utf-8")
return hashlib.sha256(encoded).hexdigest()
@@ -1,51 +0,0 @@
"""Read models for the durable external-task truth registry."""
from __future__ import annotations
import json
import sqlite3
from pathlib import Path
from typing import Sequence
from .task_execution_registry import ExecutionTruthRecord, ExecutionTruthState
def _decode(value: str) -> ExecutionTruthRecord:
payload = json.loads(value)
payload["state"] = ExecutionTruthState(payload["state"])
payload["evidence"] = dict(payload.get("evidence") or {})
payload["error"] = dict(payload.get("error") or {})
return ExecutionTruthRecord(**payload)
def list_records(
path: str | Path,
*,
executor: str | None = None,
states: Sequence[str] = (),
) -> list[ExecutionTruthRecord]:
database = Path(path).resolve()
if not database.exists():
return []
conditions: list[str] = []
values: list[str] = []
normalized_states = tuple(sorted({ExecutionTruthState(item).value for item in states}))
if normalized_states:
conditions.append(f"state IN ({','.join('?' for _ in normalized_states)})")
values.extend(normalized_states)
where = f" WHERE {' AND '.join(conditions)}" if conditions else ""
connection = sqlite3.connect(f"file:{database}?mode=ro", uri=True, timeout=10.0)
connection.row_factory = sqlite3.Row
try:
rows = connection.execute(
f"SELECT record_json FROM execution_truth{where} ORDER BY updated_at, task_id",
tuple(values),
).fetchall()
except sqlite3.OperationalError:
return []
finally:
connection.close()
records = [_decode(row["record_json"]) for row in rows]
if executor:
normalized = str(executor).strip().lower()
records = [item for item in records if item.manifest.executor.lower() == normalized]
return records
-353
View File
@@ -1,353 +0,0 @@
"""HTTP/JSON gateway for the AIOS V2 control plane.
The public gateway owns user-facing control operations only. Runtime lifecycle
transitions are deliberately absent: external callers cannot directly move a
task to RUNNING, COMPLETED or FAILED. Those transitions are available only
through the provenance-protected orchestrator and internal reconciliation API.
"""
from __future__ import annotations
import dataclasses
import json
import os
import re
import urllib.parse
from enum import Enum
from http import HTTPStatus
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from typing import Any, Mapping
from aios_hermes import HermesConflict, HermesNotFound
from .control_plane import (
ControlPlaneError,
ControlPlaneService,
IdempotencyConflict,
InvalidTransition,
NotFound,
PolicyDenied,
RequestContext,
RevisionConflict,
TaskState,
TenantIsolationError,
Unauthorized,
)
from .dispatch_store import DispatchConflict, DispatchNotFound
from .orchestrator import OrchestrationError
from .provenance import ProvenanceDenied, ProvenanceReplay
TENANT_HEADER = "X-AIOS-Tenant-ID"
PRINCIPAL_HEADER = "X-AIOS-Principal-ID"
ROLES_HEADER = "X-AIOS-Roles"
TRACE_HEADER = "X-AIOS-Trace-ID"
SESSION_HEADER = "X-AIOS-Session-ID"
POLICY_HEADER = "X-AIOS-Policy-Version"
AUTHENTICATED_HEADER = "X-AIOS-Authenticated"
_RESOURCE_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{0,191}$")
def _json_value(value: Any) -> Any:
if dataclasses.is_dataclass(value):
return {
field.name: _json_value(getattr(value, field.name))
for field in dataclasses.fields(value)
}
if isinstance(value, Enum):
return value.value
if isinstance(value, Mapping):
return {str(key): _json_value(item) for key, item in value.items()}
if isinstance(value, (list, tuple, set, frozenset)):
return [_json_value(item) for item in value]
return value
def _require_mapping(value: Any, *, label: str = "JSON body") -> Mapping[str, Any]:
if not isinstance(value, Mapping):
raise ValueError(f"{label} must be an object")
return value
def _require_resource_id(value: Any, *, label: str) -> str:
normalized = str(value or "").strip()
if not _RESOURCE_ID_RE.fullmatch(normalized):
raise ValueError(f"invalid {label}")
return normalized
class GatewayApplication:
"""Pure user-facing request router that can be tested without a socket."""
def __init__(self, service: ControlPlaneService | None = None) -> None:
self.service = service or ControlPlaneService()
@staticmethod
def context_from_headers(headers: Mapping[str, str]) -> RequestContext:
normalized_headers = {
str(key).strip().lower(): value for key, value in headers.items()
}
def header(name: str, default: str = "") -> str:
return str(normalized_headers.get(name.lower(), default))
authenticated = header(AUTHENTICATED_HEADER).strip().lower()
if authenticated not in {"1", "true", "yes"}:
raise Unauthorized("trusted authentication assertion is required")
roles = [
role.strip()
for role in header(ROLES_HEADER).split(",")
if role.strip()
]
return RequestContext.create(
tenant_id=header(TENANT_HEADER),
principal_id=header(PRINCIPAL_HEADER),
trace_id=header(TRACE_HEADER).strip() or None,
session_id=header(SESSION_HEADER).strip() or None,
policy_version=header(POLICY_HEADER, "v1").strip() or "v1",
roles=roles,
)
def handle(
self,
method: str,
raw_path: str,
headers: Mapping[str, str],
body: Any = None,
) -> tuple[int, Mapping[str, Any]]:
parsed = urllib.parse.urlparse(raw_path)
path = parsed.path.rstrip("/") or "/"
query = urllib.parse.parse_qs(parsed.query, keep_blank_values=False)
method = method.upper()
if method == "GET" and path in {"/healthz", "/api/v2/health"}:
return HTTPStatus.OK, {
"service": "aios-v2-control-plane",
"status": "ok",
"store": type(self.service.store).__name__,
}
context = self.context_from_headers(headers)
payload = _require_mapping(body or {})
if method == "POST" and path == "/api/v2/tenants":
tenant = self.service.register_tenant(
context,
tenant_id=_require_resource_id(payload.get("tenant_id"), label="tenant_id"),
display_name=str(payload.get("display_name", "")),
metadata=_require_mapping(payload.get("metadata", {}), label="metadata"),
)
return HTTPStatus.CREATED, {"tenant": _json_value(tenant)}
if method == "GET" and path == "/api/v2/tenants/current":
return HTTPStatus.OK, {"tenant": _json_value(self.service.get_tenant(context))}
if method == "POST" and path == "/api/v2/agents":
agent = self.service.register_agent(
context,
agent_id=_require_resource_id(payload.get("agent_id"), label="agent_id"),
display_name=str(payload.get("display_name", "")),
runtime=str(payload.get("runtime", "")),
capabilities=payload.get("capabilities", ()),
metadata=_require_mapping(payload.get("metadata", {}), label="metadata"),
)
return HTTPStatus.CREATED, {"agent": _json_value(agent)}
if method == "GET" and path == "/api/v2/agents":
return HTTPStatus.OK, {"agents": _json_value(self.service.list_agents(context))}
agent_match = re.fullmatch(r"/api/v2/agents/([^/]+)", path)
if method == "GET" and agent_match:
agent_id = _require_resource_id(
urllib.parse.unquote(agent_match.group(1)), label="agent_id"
)
return HTTPStatus.OK, {
"agent": _json_value(self.service.get_agent(context, agent_id))
}
if method == "POST" and path == "/api/v2/tasks":
task = self.service.create_task(
context,
agent_id=_require_resource_id(payload.get("agent_id"), label="agent_id"),
action=str(payload.get("action", "")).strip(),
payload=_require_mapping(payload.get("payload", {}), label="payload"),
idempotency_key=(
str(payload["idempotency_key"]).strip()
if payload.get("idempotency_key") is not None
else None
),
)
return HTTPStatus.CREATED, {"task": _json_value(task)}
if method == "GET" and path == "/api/v2/tasks":
tasks = self.service.list_tasks(context, states=query.get("state"))
return HTTPStatus.OK, {"tasks": _json_value(tasks)}
task_match = re.fullmatch(r"/api/v2/tasks/([^/]+)", path)
if method == "GET" and task_match:
task_id = _require_resource_id(
urllib.parse.unquote(task_match.group(1)), label="task_id"
)
return HTTPStatus.OK, {
"task": _json_value(self.service.get_task(context, task_id))
}
bypass_match = re.fullmatch(
r"/api/v2/tasks/([^/]+)/(start|complete|fail)", path
)
if method == "POST" and bypass_match:
return HTTPStatus.FORBIDDEN, {
"error": "RuntimeLifecycleBypassDenied",
"message": (
"RUNNING, COMPLETED and FAILED transitions are internal; "
"use the provenance-protected dispatch/reconcile flow"
),
}
transition_match = re.fullmatch(
r"/api/v2/tasks/([^/]+)/(approve|cancel)", path
)
if method == "POST" and transition_match:
task_id = _require_resource_id(
urllib.parse.unquote(transition_match.group(1)), label="task_id"
)
target = (
TaskState.APPROVED
if transition_match.group(2) == "approve"
else TaskState.CANCELLED
)
expected_revision = payload.get("expected_revision")
task = self.service.transition_task(
context,
task_id=task_id,
target_state=target,
expected_revision=(
int(expected_revision) if expected_revision is not None else None
),
)
return HTTPStatus.OK, {"task": _json_value(task)}
if method == "GET" and path == "/api/v2/audit":
resource_id = query.get("resource_id", [None])[0]
events = self.service.list_audit_events(
context,
resource_id=(
_require_resource_id(resource_id, label="resource_id")
if resource_id
else None
),
)
return HTTPStatus.OK, {"events": _json_value(events)}
return HTTPStatus.NOT_FOUND, {
"error": "not_found",
"message": "route not found",
}
_ERROR_STATUS: tuple[tuple[type[BaseException], HTTPStatus], ...] = (
(Unauthorized, HTTPStatus.UNAUTHORIZED),
(PolicyDenied, HTTPStatus.FORBIDDEN),
(ProvenanceDenied, HTTPStatus.FORBIDDEN),
(TenantIsolationError, HTTPStatus.NOT_FOUND),
(NotFound, HTTPStatus.NOT_FOUND),
(DispatchNotFound, HTTPStatus.NOT_FOUND),
(HermesNotFound, HTTPStatus.NOT_FOUND),
(ProvenanceReplay, HTTPStatus.CONFLICT),
(IdempotencyConflict, HTTPStatus.CONFLICT),
(RevisionConflict, HTTPStatus.CONFLICT),
(InvalidTransition, HTTPStatus.CONFLICT),
(DispatchConflict, HTTPStatus.CONFLICT),
(HermesConflict, HTTPStatus.CONFLICT),
(OrchestrationError, HTTPStatus.CONFLICT),
(ValueError, HTTPStatus.BAD_REQUEST),
(ControlPlaneError, HTTPStatus.BAD_REQUEST),
)
def error_response(exc: BaseException) -> tuple[int, Mapping[str, Any]]:
for error_type, status in _ERROR_STATUS:
if isinstance(exc, error_type):
return status, {"error": error_type.__name__, "message": str(exc)}
return HTTPStatus.INTERNAL_SERVER_ERROR, {
"error": "InternalServerError",
"message": "internal server error",
}
class GatewayRequestHandler(BaseHTTPRequestHandler):
server_version = "AIOSV2Gateway/1.0"
max_body_bytes = 1_048_576
@property
def application(self) -> GatewayApplication:
return self.server.application # type: ignore[attr-defined]
def _read_body(self) -> Any:
length = int(self.headers.get("Content-Length", "0") or "0")
if length < 0 or length > self.max_body_bytes:
raise ValueError("request body is too large")
if length == 0:
return {}
content_type = self.headers.get("Content-Type", "").split(";", 1)[0].strip()
if content_type != "application/json":
raise ValueError("Content-Type must be application/json")
return json.loads(self.rfile.read(length).decode("utf-8"))
def _send_json(self, status: int, payload: Mapping[str, Any]) -> None:
encoded = json.dumps(
_json_value(payload), ensure_ascii=False, separators=(",", ":")
).encode("utf-8")
self.send_response(int(status))
self.send_header("Content-Type", "application/json; charset=utf-8")
self.send_header("Content-Length", str(len(encoded)))
self.send_header("Cache-Control", "no-store")
self.send_header("X-Content-Type-Options", "nosniff")
self.end_headers()
self.wfile.write(encoded)
def _dispatch(self, method: str) -> None:
try:
body = self._read_body() if method in {"POST", "PUT", "PATCH"} else {}
status, payload = self.application.handle(
method,
self.path,
{key: value for key, value in self.headers.items()},
body,
)
except Exception as exc:
status, payload = error_response(exc)
self._send_json(status, payload)
def do_GET(self) -> None: # noqa: N802
self._dispatch("GET")
def do_POST(self) -> None: # noqa: N802
self._dispatch("POST")
def log_message(self, format: str, *args: Any) -> None:
super().log_message(format, *args)
class GatewayHTTPServer(ThreadingHTTPServer):
daemon_threads = True
allow_reuse_address = True
def __init__(
self,
server_address: tuple[str, int],
application: GatewayApplication,
) -> None:
self.application = application
super().__init__(server_address, GatewayRequestHandler)
def main() -> None:
host = os.getenv("AIOS_V2_GATEWAY_HOST", "127.0.0.1")
port = int(os.getenv("AIOS_V2_GATEWAY_PORT", "18920"))
server = GatewayHTTPServer((host, port), GatewayApplication())
print(f"AIOS V2 control-plane gateway listening on {host}:{port}")
server.serve_forever()
if __name__ == "__main__":
main()
-123
View File
@@ -1,123 +0,0 @@
"""Sanitized GitOps runtime status for the AIOS Web control plane."""
from __future__ import annotations
import os
import subprocess
from dataclasses import dataclass, asdict
from pathlib import Path
from typing import Any, Mapping
@dataclass(frozen=True)
class GitOpsStatus:
available: bool
service_status: str
workspace_mode: str
workspace_branch: str | None
workspace_ref: str | None
workspace_dirty: bool | None
candidate_branch: str
release_channel: str
release_version: str
sync_status: str
details: Mapping[str, Any]
def as_dict(self) -> dict[str, Any]:
return asdict(self)
def _run_git(worktree: Path, *args: str) -> tuple[int, str, str]:
process = subprocess.run(
["git", "-C", str(worktree), *args],
text=True,
capture_output=True,
timeout=5,
)
return process.returncode, process.stdout.strip(), process.stderr.strip()
def collect_gitops_status() -> GitOpsStatus:
"""Return only non-sensitive Git/workspace metadata.
Remote URLs, environment values, credentials, filesystem contents and
command stderr are deliberately excluded from the Web response.
"""
candidate_branch = os.getenv(
"AIOS_CANDIDATE_BRANCH",
"integration/aios-v2r1-cto-production-candidate-20260720",
)
release_channel = os.getenv("AIOS_RELEASE_CHANNEL", "candidate")
release_version = os.getenv("AIOS_RELEASE_VERSION", "V2R1C0-candidate")
sync_status = os.getenv("AIOS_GITOPS_SYNC_STATUS", "unknown")
worktree = Path(
os.getenv(
"AIOS_GITOPS_CONTROLLER_WORKTREE",
"/data/aios-gitops/worktrees/controller",
)
)
if not worktree.is_dir():
return GitOpsStatus(
available=False,
service_status="unknown",
workspace_mode="unavailable",
workspace_branch=None,
workspace_ref=None,
workspace_dirty=None,
candidate_branch=candidate_branch,
release_channel=release_channel,
release_version=release_version,
sync_status=sync_status,
details={"reason": "controller_worktree_unavailable"},
)
inside_code, inside, _ = _run_git(worktree, "rev-parse", "--is-inside-work-tree")
if inside_code != 0 or inside != "true":
return GitOpsStatus(
available=False,
service_status="unknown",
workspace_mode="invalid",
workspace_branch=None,
workspace_ref=None,
workspace_dirty=None,
candidate_branch=candidate_branch,
release_channel=release_channel,
release_version=release_version,
sync_status=sync_status,
details={"reason": "controller_worktree_invalid"},
)
branch_code, branch, _ = _run_git(worktree, "branch", "--show-current")
ref_code, ref_value, _ = _run_git(worktree, "rev-parse", "HEAD")
dirty_code, dirty_value, _ = _run_git(worktree, "status", "--porcelain=v1")
service_status = "unknown"
try:
process = subprocess.run(
["systemctl", "is-active", "aios-gitops-model-api"],
text=True,
capture_output=True,
timeout=3,
)
service_status = process.stdout.strip() or "inactive"
except (FileNotFoundError, subprocess.TimeoutExpired):
service_status = "unavailable"
branch_value = branch if branch_code == 0 and branch else None
return GitOpsStatus(
available=True,
service_status=service_status,
workspace_mode="branch" if branch_value else "detached",
workspace_branch=branch_value,
workspace_ref=ref_value if ref_code == 0 else None,
workspace_dirty=bool(dirty_value) if dirty_code == 0 else None,
candidate_branch=candidate_branch,
release_channel=release_channel,
release_version=release_version,
sync_status=sync_status,
details={
"controller_clean": dirty_code == 0 and not dirty_value,
"controller_detached": branch_code == 0 and not branch,
},
)
@@ -1,238 +0,0 @@
"""Official AIOS executor providers built on the portable provider protocol.
The official providers contain no lifecycle implementation. They validate their
manifests, resolve a generic ``UnifiedExecutorWorker`` from runtime-supplied
boundaries and expose the same adapter contract. Future providers can use
``ConfiguredExecutorProvider`` directly without changing AIOS core modules.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Mapping, Sequence
from .execution_plan import ExecutionPlanCoordinator, ExecutionStageManifest
from .executor_protocol import (
EXECUTOR_PROTOCOL_VERSION,
ExternalExecutorAdapter,
ExecutorProviderDescriptor,
normalize_executor,
)
from .executor_worker import NotificationSender, StepRunner, UnifiedExecutorWorker
from .external_task_control import ExternalTaskCoordinator
from .task_execution_registry import ExecutionTaskManifest, ExecutionTruthRecord
@dataclass(frozen=True)
class ProviderRuntimeConfiguration:
plan_id: str
executor_id: str
plan_coordinator: ExecutionPlanCoordinator
task_coordinator: ExternalTaskCoordinator
providers: Any
step_runner: StepRunner
notification_sender: NotificationSender
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "ProviderRuntimeConfiguration":
required = (
"plan_id",
"executor_id",
"plan_coordinator",
"task_coordinator",
"providers",
"step_runner",
"notification_sender",
)
missing = [name for name in required if value.get(name) is None]
if missing:
raise ValueError(f"provider runtime configuration is missing: {missing}")
return cls(**{name: value[name] for name in required})
class ConfiguredExecutorAdapter:
"""Generic adapter produced by every official or third-party provider."""
def __init__(
self,
descriptor: ExecutorProviderDescriptor,
configuration: ProviderRuntimeConfiguration,
) -> None:
self.descriptor = descriptor
self.configuration = configuration
self.worker = UnifiedExecutorWorker(
plan_coordinator=configuration.plan_coordinator,
task_coordinator=configuration.task_coordinator,
providers=configuration.providers,
executor_type=descriptor.executor_type,
executor_id=configuration.executor_id,
step_runner=configuration.step_runner,
notification_sender=configuration.notification_sender,
)
def discover(self) -> Sequence[ExecutionTruthRecord]:
ready = self.configuration.plan_coordinator.discover_ready(
plan_id=self.configuration.plan_id,
executor_type=self.descriptor.executor_type,
)
return tuple(item["record"] for item in ready)
def execute(self, manifest: ExecutionTaskManifest) -> Mapping[str, Any]:
if normalize_executor(manifest.executor) != self.descriptor.executor_type:
raise PermissionError("manifest executor does not match adapter provider")
plan = self.configuration.plan_coordinator.plan_store.get(
self.configuration.plan_id
)
stage: ExecutionStageManifest | None = None
for item in plan.stages:
if item.task_id == manifest.task_id:
stage = item
break
if stage is None:
raise KeyError("manifest task is not part of configured execution plan")
return self.worker.run_stage(
plan_id=plan.plan_id,
stage_id=stage.stage_id,
).as_dict()
def notify_owner(self, payload: Mapping[str, Any]) -> str:
# External callers normally use the durable NotificationOutbox. This
# method remains part of the portable adapter contract for providers
# that need to send a standalone operational notice.
from .external_task_control import NotificationRecord
from .task_execution_registry import utc_now
notification = NotificationRecord(
notification_id="provider_direct_notice",
task_id=str(payload.get("task_id") or "provider-notice"),
event_type=str(payload.get("event_type") or "INFO"),
target_ref=str(
payload.get("target_ref") or "AIOS_OWNER_WECHAT_TARGET_REF"
),
payload=dict(payload),
status="PENDING",
created_at=utc_now(),
)
return self.configuration.notification_sender.send(notification)
class ConfiguredExecutorProvider:
"""Reusable provider implementation for current and future executors."""
descriptor: ExecutorProviderDescriptor
def __init__(self, descriptor: ExecutorProviderDescriptor) -> None:
self.descriptor = descriptor
def validate_manifest(self, manifest: ExecutionTaskManifest) -> None:
if normalize_executor(manifest.executor) != self.descriptor.executor_type:
raise ValueError("manifest executor does not match provider")
if manifest.manifest_version not in {
"aios.external-task.v1",
"aios.external-task.v2",
}:
raise ValueError("unsupported external task manifest version")
if not manifest.steps:
raise ValueError("executor manifest has no steps")
def build_adapter(self, configuration: Mapping[str, Any]) -> ExternalExecutorAdapter:
return ConfiguredExecutorAdapter(
self.descriptor,
ProviderRuntimeConfiguration.from_mapping(configuration),
)
class OpenClawProvider(ConfiguredExecutorProvider):
def __init__(self) -> None:
super().__init__(
ExecutorProviderDescriptor(
executor_type="openclaw",
adapter_version="2.1.0",
protocol_version=EXECUTOR_PROTOCOL_VERSION,
trust_roles=("runtime", "openclaw_executor"),
capabilities=(
"deploy",
"http_runtime",
"owner_notification",
"healthcheck",
"rollback",
),
configuration_schema="aios.executor.openclaw.config.v1",
metadata={"official": True},
)
)
class HermesProvider(ConfiguredExecutorProvider):
def __init__(self) -> None:
super().__init__(
ExecutorProviderDescriptor(
executor_type="hermes",
adapter_version="2.1.0",
protocol_version=EXECUTOR_PROTOCOL_VERSION,
trust_roles=("runtime", "hermes_executor"),
capabilities=(
"queue",
"lease",
"worker",
"runtime_validation",
"restart_recovery",
),
configuration_schema="aios.executor.hermes.config.v1",
metadata={"official": True},
)
)
class CodexProvider(ConfiguredExecutorProvider):
def __init__(self) -> None:
super().__init__(
ExecutorProviderDescriptor(
executor_type="codex",
adapter_version="2.1.0",
protocol_version=EXECUTOR_PROTOCOL_VERSION,
trust_roles=("codex", "codex_executor"),
capabilities=(
"code",
"test",
"docs",
"gitops_patch",
"security_review",
),
configuration_schema="aios.executor.codex.config.v1",
metadata={"official": True},
)
)
class ChatGPTSupervisorProvider(ConfiguredExecutorProvider):
"""Explicit takeover provider; it does not inherit Codex permissions."""
def __init__(self) -> None:
super().__init__(
ExecutorProviderDescriptor(
executor_type="chatgpt-supervisor",
adapter_version="2.1.0",
protocol_version=EXECUTOR_PROTOCOL_VERSION,
trust_roles=("chatgpt_supervisor",),
capabilities=(
"acceptance",
"deploy",
"evidence",
"healthcheck",
"rollback",
"wechat_simulation",
),
configuration_schema="aios.executor.chatgpt-supervisor.config.v1",
metadata={"official": True, "production_mutation_allowed": False},
)
)
def official_provider_factories() -> Mapping[str, ConfiguredExecutorProvider]:
providers = (
OpenClawProvider(),
HermesProvider(),
CodexProvider(),
ChatGPTSupervisorProvider(),
)
return {item.descriptor.executor_type: item for item in providers}
-599
View File
@@ -1,599 +0,0 @@
"""AIOS V2 execution orchestrator.
The orchestrator is the only component allowed to translate an approved AIOS
Task into an OpenClaw command or Hermes job. Every dispatch is persisted before
Runtime side effects. Runtime results must bind to the exact dispatch,
command/job, tenant, task, trace, principal and provenance context.
"""
from __future__ import annotations
from typing import Any, Mapping, Protocol
from aios_hermes import HermesAdapter, HermesJob, HermesJobState, HermesResult
from aios_openclaw import (
CommandState,
OpenClawAdapter,
OpenClawCommand,
OpenClawResult,
sign_command,
)
from .control_plane import (
AgentRecord,
ControlPlaneService,
RequestContext,
TaskRecord,
TaskState,
)
from .dispatch_store import (
DispatchConflict,
DispatchRecord,
DispatchRepository,
DispatchState,
InMemoryDispatchRepository,
TERMINAL_DISPATCH_STATES,
)
from .provenance import ProvenanceDenied, digest_payload
class RuntimeAuthorizationWriter(Protocol):
def authorize_runtime_execute(self, **values: Any) -> Any: ...
class OrchestrationError(RuntimeError):
"""AIOS failed to dispatch or reconcile a runtime execution."""
class ExecutionOrchestrator:
"""Dispatch approved AIOS tasks and strictly reconcile Runtime results."""
def __init__(
self,
control_plane: ControlPlaneService,
*,
openclaw: OpenClawAdapter | None = None,
hermes: HermesAdapter | None = None,
openclaw_signing_material_resolver=None,
openclaw_key_id: str = "default",
default_cell_id: str = "default",
default_profile_id: str = "default",
provenance_writer: RuntimeAuthorizationWriter | None = None,
require_provenance: bool = False,
dispatch_repository: DispatchRepository | None = None,
) -> None:
self.control_plane = control_plane
self.openclaw = openclaw
self.hermes = hermes
self.openclaw_signing_material_resolver = openclaw_signing_material_resolver
self.openclaw_key_id = str(openclaw_key_id or "").strip() or "default"
self.default_cell_id = default_cell_id
self.default_profile_id = default_profile_id
self.provenance_writer = provenance_writer
self.require_provenance = bool(require_provenance)
self.dispatch_repository = dispatch_repository or InMemoryDispatchRepository()
if self.require_provenance and self.provenance_writer is None:
raise ValueError("production orchestration requires a provenance writer")
@staticmethod
def _record_provenance(value: str | None, *, required: bool, label: str) -> str:
normalized = str(value or "").strip()
if required and not normalized:
raise OrchestrationError(f"production dispatch requires {label}")
return normalized or f"{label}-not-required"
def _authorize_dispatch(
self,
context: RequestContext,
*,
task: TaskRecord,
agent: AgentRecord,
provenance_id: str | None,
run_id: str | None,
timeout_seconds: float | None,
) -> str:
if not self.require_provenance:
return "authorization-not-required"
if self.provenance_writer is None:
raise OrchestrationError("Run Provenance writer is not configured")
if not provenance_id or not run_id:
raise OrchestrationError(
"production dispatch requires provenance_id and run_id"
)
request_digest = digest_payload(
{
"operation": "dispatch",
"tenant_id": task.tenant_id,
"task_id": task.task_id,
"agent_id": agent.agent_id,
"principal_id": context.principal_id,
"runtime": agent.runtime.strip().lower(),
"action": task.action,
"payload": task.payload,
"trace_id": context.trace_id,
"timeout_seconds": timeout_seconds,
"task_revision": task.revision,
}
)
try:
authorization = self.provenance_writer.authorize_runtime_execute(
provenance_id=provenance_id,
run_id=run_id,
tenant_id=task.tenant_id,
task_id=task.task_id,
agent_id=agent.agent_id,
principal_id=context.principal_id,
request_digest=request_digest,
)
return authorization.authorization_id
except ProvenanceDenied as exc:
raise OrchestrationError(f"Run Provenance denied dispatch: {exc}") from exc
def dispatch(
self,
context: RequestContext,
*,
task_id: str,
timeout_seconds: float | None = None,
provenance_id: str | None = None,
run_id: str | None = None,
) -> DispatchRecord:
task = self.control_plane.get_task(context, task_id)
agent = self.control_plane.get_agent(context, task.agent_id)
persisted_provenance = self._record_provenance(
provenance_id, required=self.require_provenance, label="provenance_id"
)
persisted_run = self._record_provenance(
run_id, required=self.require_provenance, label="run_id"
)
existing = self.dispatch_repository.get_by_task(
task.task_id, tenant_id=task.tenant_id
)
if existing is not None:
if (
existing.provenance_id != persisted_provenance
or existing.run_id != persisted_run
or existing.principal_id != context.principal_id
):
raise OrchestrationError(
"existing dispatch belongs to another provenance or principal context"
)
return existing
if task.state != TaskState.APPROVED:
raise OrchestrationError("only APPROVED tasks may be dispatched")
authorization_id = self._authorize_dispatch(
context,
task=task,
agent=agent,
provenance_id=provenance_id,
run_id=run_id,
timeout_seconds=timeout_seconds,
)
runtime = agent.runtime.strip().lower()
prepared: DispatchRecord | None = None
try:
if runtime == "openclaw":
command = self._create_openclaw_command(context, task)
prepared = self.dispatch_repository.create(
DispatchRecord.create(
tenant_id=task.tenant_id,
task_id=task.task_id,
agent_id=agent.agent_id,
principal_id=context.principal_id,
runtime="openclaw",
runtime_execution_id=command.command_id,
trace_id=context.trace_id,
provenance_id=persisted_provenance,
run_id=persisted_run,
authorization_id=authorization_id,
)
)
self.control_plane.transition_task(
context,
task_id=task.task_id,
target_state=TaskState.RUNNING,
expected_revision=task.revision,
)
self.dispatch_repository.update(
prepared.dispatch_id,
tenant_id=prepared.tenant_id,
state=DispatchState.RUNNING,
)
if self.openclaw is None:
raise OrchestrationError("OpenClaw runtime is not configured")
result = self.openclaw.execute(
command, timeout_seconds=timeout_seconds
)
self.reconcile_openclaw(
context,
result,
dispatch_id=prepared.dispatch_id,
provenance_id=persisted_provenance,
run_id=persisted_run,
)
return self.dispatch_repository.get(
prepared.dispatch_id, tenant_id=prepared.tenant_id
)
if runtime == "hermes":
if self.hermes is None:
raise OrchestrationError("Hermes runtime is not configured")
job = self._create_hermes_job(context, agent, task)
prepared = self.dispatch_repository.create(
DispatchRecord.create(
tenant_id=task.tenant_id,
task_id=task.task_id,
agent_id=agent.agent_id,
principal_id=context.principal_id,
runtime="hermes",
runtime_execution_id=job.job_id,
trace_id=context.trace_id,
provenance_id=persisted_provenance,
run_id=persisted_run,
authorization_id=authorization_id,
)
)
self.control_plane.transition_task(
context,
task_id=task.task_id,
target_state=TaskState.RUNNING,
expected_revision=task.revision,
)
queued = self.hermes.enqueue(job)
return self.dispatch_repository.update(
prepared.dispatch_id,
tenant_id=prepared.tenant_id,
state=DispatchState.QUEUED,
result_digest=digest_payload(
{
"queued_job_id": queued.job_id,
"state": queued.state.value,
"revision": queued.revision,
}
),
)
raise OrchestrationError(f"unsupported runtime: {agent.runtime}")
except Exception as exc:
if prepared is not None:
try:
self.dispatch_repository.update(
prepared.dispatch_id,
tenant_id=prepared.tenant_id,
state=DispatchState.FAILED,
result_digest=digest_payload(
{
"code": "dispatch_failed",
"error_type": type(exc).__name__,
"message": str(exc),
}
),
)
except Exception:
pass
current = self.control_plane.get_task(context, task.task_id)
if current.state == TaskState.RUNNING:
self.control_plane.transition_task(
context,
task_id=task.task_id,
target_state=TaskState.FAILED,
expected_revision=current.revision,
error={"code": "dispatch_failed", "message": str(exc)},
)
raise
def _create_openclaw_command(
self,
context: RequestContext,
task: TaskRecord,
) -> OpenClawCommand:
if self.openclaw_signing_material_resolver is None:
raise OrchestrationError("OpenClaw signing material is not configured")
key_material = self.openclaw_signing_material_resolver(
task.tenant_id,
self.openclaw_key_id,
)
command = OpenClawCommand.create(
tenant_id=task.tenant_id,
task_id=task.task_id,
trace_id=context.trace_id,
action=task.action,
payload=task.payload,
idempotency_key=task.idempotency_key or task.task_id,
key_id=self.openclaw_key_id,
)
return sign_command(command, key_material)
def _create_hermes_job(
self,
context: RequestContext,
agent: AgentRecord,
task: TaskRecord,
) -> HermesJob:
metadata = dict(agent.metadata)
return HermesJob.create(
tenant_id=task.tenant_id,
cell_id=str(metadata.get("cell_id") or self.default_cell_id),
profile_id=str(metadata.get("profile_id") or self.default_profile_id),
task_id=task.task_id,
trace_id=context.trace_id,
action=task.action,
payload=task.payload,
idempotency_key=task.idempotency_key or task.task_id,
priority=int(metadata.get("priority", 100)),
)
@staticmethod
def _assert_dispatch_context(
dispatch: DispatchRecord,
*,
runtime: str,
runtime_execution_id: str,
tenant_id: str,
task_id: str,
trace_id: str,
principal_id: str,
provenance_id: str,
run_id: str,
) -> None:
expected = {
"runtime": runtime,
"runtime_execution_id": runtime_execution_id,
"tenant_id": tenant_id,
"task_id": task_id,
"trace_id": trace_id,
"principal_id": principal_id,
"provenance_id": provenance_id,
"run_id": run_id,
}
for field_name, expected_value in expected.items():
if getattr(dispatch, field_name) != expected_value:
raise OrchestrationError(f"dispatch {field_name} mismatch")
def hermes_lifecycle_context(
self,
context: RequestContext,
*,
dispatch_id: str,
job_id: str,
tenant_id: str,
task_id: str,
trace_id: str,
principal_id: str,
provenance_id: str,
run_id: str,
) -> tuple[DispatchRecord, HermesJob]:
"""Resolve a runtime lifecycle request against its durable dispatch."""
dispatch = self.dispatch_repository.get(
dispatch_id, tenant_id=context.tenant_id
)
self._assert_dispatch_context(
dispatch,
runtime="hermes",
runtime_execution_id=job_id,
tenant_id=tenant_id,
task_id=task_id,
trace_id=trace_id,
principal_id=principal_id,
provenance_id=provenance_id,
run_id=run_id,
)
if context.tenant_id != tenant_id:
raise OrchestrationError("runtime context tenant_id mismatch")
if context.principal_id != principal_id:
raise OrchestrationError("runtime context principal_id mismatch")
if context.trace_id != trace_id:
raise OrchestrationError("runtime context trace_id mismatch")
if self.hermes is None:
raise OrchestrationError("Hermes runtime is not configured")
job = self.hermes.get(job_id, tenant_id=tenant_id)
if job.task_id != task_id or job.trace_id != trace_id:
raise OrchestrationError("Hermes job identity mismatch")
return dispatch, job
def bind_hermes_worker(
self,
context: RequestContext,
job: HermesJob,
*,
dispatch_id: str,
provenance_id: str,
run_id: str,
worker_id: str,
) -> DispatchRecord:
dispatch, current = self.hermes_lifecycle_context(
context,
dispatch_id=dispatch_id,
job_id=job.job_id,
tenant_id=job.tenant_id,
task_id=job.task_id,
trace_id=job.trace_id,
principal_id=context.principal_id,
provenance_id=provenance_id,
run_id=run_id,
)
if current.assigned_worker_id != worker_id:
raise OrchestrationError("Hermes worker does not own claimed job")
return self.dispatch_repository.update(
dispatch.dispatch_id,
tenant_id=dispatch.tenant_id,
expected_worker_id=worker_id,
)
def reconcile_openclaw(
self,
context: RequestContext,
result: OpenClawResult,
*,
dispatch_id: str,
provenance_id: str,
run_id: str,
) -> TaskRecord:
dispatch = self.dispatch_repository.get(
dispatch_id, tenant_id=context.tenant_id
)
self._assert_dispatch_context(
dispatch,
runtime="openclaw",
runtime_execution_id=result.command_id,
tenant_id=result.tenant_id,
task_id=result.task_id,
trace_id=result.trace_id,
principal_id=context.principal_id,
provenance_id=provenance_id,
run_id=run_id,
)
result_digest = digest_payload(
{
"command_id": result.command_id,
"state": result.state.value,
"output": result.output,
"error": result.error,
"runtime_id": result.runtime_id,
"attempts": result.attempts,
"finished_at": result.finished_at,
}
)
if dispatch.state in TERMINAL_DISPATCH_STATES:
if dispatch.result_digest != result_digest:
raise DispatchConflict("terminal dispatch received a different result")
return self.control_plane.get_task(context, dispatch.task_id)
task = self.control_plane.get_task(context, dispatch.task_id)
if task.state != TaskState.RUNNING:
raise OrchestrationError("task is not RUNNING for active dispatch")
if result.state == CommandState.COMPLETED:
target = TaskState.COMPLETED
values = {"result": result.output or {}}
dispatch_state = DispatchState.COMPLETED
elif result.state in {
CommandState.FAILED,
CommandState.TIMED_OUT,
CommandState.REJECTED,
}:
target = TaskState.FAILED
values = {
"error": result.error or {"code": result.state.value.lower()}
}
dispatch_state = DispatchState.FAILED
elif result.state == CommandState.CANCELLED:
target = TaskState.CANCELLED
values = {}
dispatch_state = DispatchState.CANCELLED
else:
self.dispatch_repository.update(
dispatch.dispatch_id,
tenant_id=dispatch.tenant_id,
state=DispatchState.RUNNING,
result_digest=result_digest,
)
return task
updated_task = self.control_plane.transition_task(
context,
task_id=task.task_id,
target_state=target,
expected_revision=task.revision,
**values,
)
self.dispatch_repository.update(
dispatch.dispatch_id,
tenant_id=dispatch.tenant_id,
state=dispatch_state,
result_digest=result_digest,
)
return updated_task
def reconcile_hermes(
self,
context: RequestContext,
result: HermesResult,
*,
dispatch_id: str,
provenance_id: str,
run_id: str,
) -> TaskRecord:
dispatch = self.dispatch_repository.get(
dispatch_id, tenant_id=context.tenant_id
)
self._assert_dispatch_context(
dispatch,
runtime="hermes",
runtime_execution_id=result.job_id,
tenant_id=result.tenant_id,
task_id=result.task_id,
trace_id=result.trace_id,
principal_id=context.principal_id,
provenance_id=provenance_id,
run_id=run_id,
)
if self.hermes is None:
raise OrchestrationError("Hermes runtime is not configured")
job = self.hermes.get(result.job_id, tenant_id=result.tenant_id)
if result.state in {HermesJobState.COMPLETED, HermesJobState.FAILED}:
if not result.worker_id:
raise OrchestrationError("terminal Hermes result requires worker_id")
if job.assigned_worker_id != result.worker_id:
raise OrchestrationError("Hermes result worker does not own job lease")
if (
dispatch.expected_worker_id is not None
and dispatch.expected_worker_id != result.worker_id
):
raise OrchestrationError("Hermes result worker mismatch")
result_digest = digest_payload(
{
"job_id": result.job_id,
"worker_id": result.worker_id,
"state": result.state.value,
"output": result.output,
"error": result.error,
"finished_at": result.finished_at,
}
)
if dispatch.state in TERMINAL_DISPATCH_STATES:
if dispatch.result_digest != result_digest:
raise DispatchConflict("terminal dispatch received a different result")
return self.control_plane.get_task(context, dispatch.task_id)
task = self.control_plane.get_task(context, dispatch.task_id)
if task.state != TaskState.RUNNING:
raise OrchestrationError("task is not RUNNING for active dispatch")
if result.state == HermesJobState.COMPLETED:
target = TaskState.COMPLETED
values = {"result": result.output or {}}
dispatch_state = DispatchState.COMPLETED
elif result.state == HermesJobState.FAILED:
target = TaskState.FAILED
values = {"error": result.error or {"code": "hermes_failed"}}
dispatch_state = DispatchState.FAILED
elif result.state == HermesJobState.CANCELLED:
target = TaskState.CANCELLED
values = {}
dispatch_state = DispatchState.CANCELLED
else:
raise OrchestrationError("Hermes reconcile only accepts terminal results")
updated_task = self.control_plane.transition_task(
context,
task_id=task.task_id,
target_state=target,
expected_revision=task.revision,
**values,
)
self.dispatch_repository.update(
dispatch.dispatch_id,
tenant_id=dispatch.tenant_id,
state=dispatch_state,
expected_worker_id=result.worker_id,
result_digest=result_digest,
)
return updated_task
def dispatch_snapshot(self, *, tenant_id: str) -> Mapping[str, DispatchRecord]:
return {
item.task_id: item
for item in self.dispatch_repository.list(tenant_id=tenant_id)
}
-181
View File
@@ -1,181 +0,0 @@
"""Durable single-process persistence for the AIOS V2 control plane.
The candidate uses an atomic SQLite snapshot so a gateway restart does not lose
tenants, agents, task state, idempotency records, or audit events. The domain
service remains storage-agnostic; a later HA backend can replace this module
without changing API or runtime-adapter contracts.
"""
from __future__ import annotations
import json
import sqlite3
import threading
from dataclasses import asdict
from pathlib import Path
from typing import Any, Callable
from .control_plane import (
AgentRecord,
AuditEvent,
ControlPlaneService,
InMemoryControlPlaneStore,
TaskRecord,
TaskState,
TenantRecord,
)
class SQLiteSnapshotRepository:
"""Store one versioned control-plane snapshot in SQLite."""
SCHEMA_VERSION = 1
def __init__(self, path: str | Path) -> None:
self.path = Path(path).resolve()
self.path.parent.mkdir(parents=True, exist_ok=True)
self._lock = threading.RLock()
self._initialize()
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.path, timeout=30.0)
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA synchronous=FULL")
connection.execute("PRAGMA foreign_keys=ON")
return connection
def _initialize(self) -> None:
with self._connect() as connection:
connection.execute(
"""
CREATE TABLE IF NOT EXISTS control_plane_snapshot (
singleton_id INTEGER PRIMARY KEY CHECK (singleton_id = 1),
schema_version INTEGER NOT NULL,
payload_json TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
)
"""
)
def load(self) -> dict[str, Any] | None:
with self._lock, self._connect() as connection:
row = connection.execute(
"SELECT schema_version, payload_json FROM control_plane_snapshot WHERE singleton_id = 1"
).fetchone()
if row is None:
return None
schema_version, payload_json = row
if int(schema_version) != self.SCHEMA_VERSION:
raise RuntimeError(
f"unsupported control-plane snapshot version: {schema_version}"
)
decoded = json.loads(str(payload_json))
if not isinstance(decoded, dict):
raise RuntimeError("invalid control-plane snapshot")
return decoded
def save(self, payload: dict[str, Any]) -> None:
encoded = json.dumps(
payload,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
default=str,
)
with self._lock, self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
connection.execute(
"""
INSERT INTO control_plane_snapshot (
singleton_id, schema_version, payload_json, updated_at
) VALUES (1, ?, ?, CURRENT_TIMESTAMP)
ON CONFLICT(singleton_id) DO UPDATE SET
schema_version = excluded.schema_version,
payload_json = excluded.payload_json,
updated_at = CURRENT_TIMESTAMP
""",
(self.SCHEMA_VERSION, encoded),
)
connection.commit()
class DurableControlPlaneService(ControlPlaneService):
"""Control-plane service that persists after every successful mutation."""
def __init__(
self,
repository: SQLiteSnapshotRepository,
*,
policy_evaluator=None,
) -> None:
self.repository = repository
store = InMemoryControlPlaneStore()
self._restore(store, repository.load())
super().__init__(store, policy_evaluator=policy_evaluator)
@staticmethod
def _restore(
store: InMemoryControlPlaneStore,
snapshot: dict[str, Any] | None,
) -> None:
if not snapshot:
return
for item in snapshot.get("tenants", []):
record = TenantRecord(**item)
store.tenants[record.tenant_id] = record
for item in snapshot.get("agents", []):
normalized = dict(item)
normalized["capabilities"] = tuple(normalized.get("capabilities", ()))
record = AgentRecord(**normalized)
store.agents[record.agent_id] = record
for item in snapshot.get("tasks", []):
normalized = dict(item)
normalized["state"] = TaskState(normalized["state"])
record = TaskRecord(**normalized)
store.tasks[record.task_id] = record
store.audit_events.extend(AuditEvent(**item) for item in snapshot.get("audit", []))
for item in snapshot.get("idempotency", []):
store.idempotency[(item["tenant_id"], item["key"])] = (
item["task_id"],
item["digest"],
)
def _snapshot(self) -> dict[str, Any]:
with self.store.lock:
return {
"schema_version": self.repository.SCHEMA_VERSION,
"tenants": [asdict(item) for item in self.store.tenants.values()],
"agents": [asdict(item) for item in self.store.agents.values()],
"tasks": [
{**asdict(item), "state": item.state.value}
for item in self.store.tasks.values()
],
"audit": [asdict(item) for item in self.store.audit_events],
"idempotency": [
{
"tenant_id": tenant_id,
"key": key,
"task_id": task_id,
"digest": digest,
}
for (tenant_id, key), (task_id, digest)
in self.store.idempotency.items()
],
}
def _mutate(self, operation: Callable[[], Any]) -> Any:
with self.store.lock:
result = operation()
self.repository.save(self._snapshot())
return result
def register_tenant(self, *args, **kwargs):
return self._mutate(lambda: super(DurableControlPlaneService, self).register_tenant(*args, **kwargs))
def register_agent(self, *args, **kwargs):
return self._mutate(lambda: super(DurableControlPlaneService, self).register_agent(*args, **kwargs))
def create_task(self, *args, **kwargs):
return self._mutate(lambda: super(DurableControlPlaneService, self).create_task(*args, **kwargs))
def transition_task(self, *args, **kwargs):
return self._mutate(lambda: super(DurableControlPlaneService, self).transition_task(*args, **kwargs))
-831
View File
@@ -1,831 +0,0 @@
"""AIOS V2 可信 Run Provenance 签发、持久化与受控写入。
生产策略要求 Provenance 同时覆盖七类路径:代码变更、Git 操作、部署、
Runtime 执行、Evidence 写入、回滚和审计。任何缺失、篡改、过期、撤销、
重放或上下文不匹配都默认拒绝。
"""
from __future__ import annotations
import base64
import hashlib
import json
import os
import sqlite3
import stat
import threading
import uuid
from dataclasses import asdict, dataclass, field, replace
from datetime import datetime, timedelta, timezone
from enum import Enum
from pathlib import Path
from typing import Any, Mapping, Protocol, Sequence
PROVENANCE_VERSION = "aios.v2.run-provenance.v1"
PRODUCTION_POLICY_VERSION = "aios.v2.production.v1"
def utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def parse_timestamp(value: str) -> datetime:
normalized = str(value or "").strip()
if not normalized:
raise ValueError("timestamp is required")
if normalized.endswith("Z"):
normalized = normalized[:-1] + "+00:00"
parsed = datetime.fromisoformat(normalized)
if parsed.tzinfo is None:
raise ValueError("timestamp must include timezone")
return parsed.astimezone(timezone.utc)
def required(value: str, label: str) -> str:
normalized = str(value or "").strip()
if not normalized:
raise ValueError(f"{label} is required")
return normalized
def canonical_json_bytes(value: Mapping[str, Any]) -> bytes:
return json.dumps(
dict(value),
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
default=str,
).encode("utf-8")
def digest_payload(value: Mapping[str, Any]) -> str:
return hashlib.sha256(canonical_json_bytes(value)).hexdigest()
def _validate_digest(value: str) -> str:
normalized = required(value, "request_digest").lower()
if len(normalized) != 64 or any(ch not in "0123456789abcdef" for ch in normalized):
raise ValueError("request_digest must be a SHA-256 hexadecimal digest")
return normalized
class ExecutionPath(str, Enum):
CODE_CHANGE = "code_change"
GIT_OPERATION = "git_operation"
DEPLOYMENT = "deployment"
RUNTIME_EXECUTE = "runtime_execute"
EVIDENCE_WRITE = "evidence_write"
ROLLBACK = "rollback"
AUDIT = "audit"
REQUIRED_PRODUCTION_PATHS = frozenset(item.value for item in ExecutionPath)
class GitOperation(str, Enum):
READ_REF = "read_ref"
FETCH = "fetch"
CREATE_BRANCH = "create_branch"
COMMIT = "commit"
MERGE = "merge"
PUSH = "push"
UPDATE_EVIDENCE = "update_evidence"
class ProvenanceStatus(str, Enum):
ISSUED = "issued"
VERIFIED = "verified"
USED = "used"
EXPIRED = "expired"
REVOKED = "revoked"
class ProvenanceError(RuntimeError):
"""Run Provenance 基础错误。"""
class ProvenanceDenied(ProvenanceError):
"""策略拒绝当前操作。"""
class ProvenanceNotFound(ProvenanceDenied):
"""未找到 Provenance。"""
class ProvenanceSignatureError(ProvenanceDenied):
"""签名无效。"""
class ProvenanceExpired(ProvenanceDenied):
"""Provenance 已过期。"""
class ProvenanceRevoked(ProvenanceDenied):
"""Provenance 已撤销。"""
class ProvenanceReplay(ProvenanceDenied):
"""检测到相同授权摘要重放。"""
class Signer(Protocol):
key_id: str
def sign(self, payload: bytes) -> str: ...
class SignatureVerifier(Protocol):
key_id: str
fingerprint: str
def verify(self, payload: bytes, signature: str) -> bool: ...
def _load_crypto() -> tuple[Any, Any, Any]:
try:
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import (
Ed25519PrivateKey,
Ed25519PublicKey,
)
except ImportError as exc: # pragma: no cover - depends on deployment image
raise RuntimeError(
"cryptography package with Ed25519 support is required"
) from exc
return serialization, Ed25519PrivateKey, Ed25519PublicKey
def _check_private_key_file(path: Path) -> bytes:
resolved = path.resolve()
metadata = resolved.stat()
if not stat.S_ISREG(metadata.st_mode):
raise RuntimeError("private key reference is not a regular file")
if metadata.st_mode & 0o077:
raise RuntimeError("private key file permissions must be 0600 or stricter")
return resolved.read_bytes()
def _check_public_key_file(path: Path) -> bytes:
resolved = path.resolve()
metadata = resolved.stat()
if not stat.S_ISREG(metadata.st_mode):
raise RuntimeError("public key reference is not a regular file")
if metadata.st_mode & 0o022:
raise RuntimeError("public key file must not be group/world writable")
return resolved.read_bytes()
class Ed25519PrivateKeySigner:
"""Ed25519 Owner/Control Plane 签发器。"""
def __init__(self, private_key: Any, *, key_id: str) -> None:
self.private_key = private_key
self.key_id = required(key_id, "key_id")
@classmethod
def from_file(cls, path: str | Path, *, key_id: str) -> "Ed25519PrivateKeySigner":
serialization, private_cls, _ = _load_crypto()
value = _check_private_key_file(Path(path))
if value.startswith(b"-----BEGIN"):
key = serialization.load_pem_private_key(value, password=None)
elif len(value) == 32:
key = private_cls.from_private_bytes(value)
else:
raise RuntimeError("private key must be PEM or a raw 32-byte Ed25519 seed")
if not isinstance(key, private_cls):
raise RuntimeError("private key is not Ed25519")
return cls(key, key_id=key_id)
def sign(self, payload: bytes) -> str:
return base64.b64encode(self.private_key.sign(bytes(payload))).decode("ascii")
def public_verifier(self) -> "Ed25519PublicKeyVerifier":
return Ed25519PublicKeyVerifier(self.private_key.public_key(), key_id=self.key_id)
class Ed25519PublicKeyVerifier:
"""Ed25519 签名验证器。"""
def __init__(self, public_key: Any, *, key_id: str) -> None:
serialization, _, public_cls = _load_crypto()
if not isinstance(public_key, public_cls):
raise RuntimeError("public key is not Ed25519")
self.public_key = public_key
self.key_id = required(key_id, "key_id")
raw = public_key.public_bytes(
encoding=serialization.Encoding.Raw,
format=serialization.PublicFormat.Raw,
)
self.fingerprint = hashlib.sha256(raw).hexdigest()
@classmethod
def from_file(cls, path: str | Path, *, key_id: str) -> "Ed25519PublicKeyVerifier":
serialization, _, public_cls = _load_crypto()
value = _check_public_key_file(Path(path))
if value.startswith(b"-----BEGIN"):
key = serialization.load_pem_public_key(value)
elif len(value) == 32:
key = public_cls.from_public_bytes(value)
else:
raise RuntimeError("public key must be PEM or raw 32-byte Ed25519 key")
if not isinstance(key, public_cls):
raise RuntimeError("public key is not Ed25519")
return cls(key, key_id=key_id)
def verify(self, payload: bytes, signature: str) -> bool:
try:
decoded = base64.b64decode(required(signature, "signature"), validate=True)
self.public_key.verify(decoded, bytes(payload))
return True
except Exception:
return False
@dataclass(frozen=True)
class RunProvenance:
provenance_id: str
run_id: str
issuer: str
tenant_id: str
task_id: str
agent_id: str
principal_id: str
allowed_paths: Sequence[str]
allowed_git_operations: Sequence[str]
issued_at: str
expires_at: str
policy_version: str
key_id: str
nonce: str
signature: str = ""
schema_version: str = PROVENANCE_VERSION
def __post_init__(self) -> None:
for name in (
"provenance_id",
"run_id",
"issuer",
"tenant_id",
"task_id",
"agent_id",
"principal_id",
"issued_at",
"expires_at",
"policy_version",
"key_id",
"nonce",
"schema_version",
):
object.__setattr__(self, name, required(getattr(self, name), name))
if self.schema_version != PROVENANCE_VERSION:
raise ValueError(f"unsupported schema_version: {self.schema_version}")
issued = parse_timestamp(self.issued_at)
expires = parse_timestamp(self.expires_at)
if expires <= issued:
raise ValueError("expires_at must be later than issued_at")
paths = tuple(sorted({ExecutionPath(item).value for item in self.allowed_paths}))
operations = tuple(
sorted({GitOperation(item).value for item in self.allowed_git_operations})
)
object.__setattr__(self, "allowed_paths", paths)
object.__setattr__(self, "allowed_git_operations", operations)
object.__setattr__(self, "signature", str(self.signature or "").strip())
def unsigned_dict(self) -> dict[str, Any]:
return {
"schema_version": self.schema_version,
"provenance_id": self.provenance_id,
"run_id": self.run_id,
"issuer": self.issuer,
"tenant_id": self.tenant_id,
"task_id": self.task_id,
"agent_id": self.agent_id,
"principal_id": self.principal_id,
"allowed_paths": list(self.allowed_paths),
"allowed_git_operations": list(self.allowed_git_operations),
"issued_at": self.issued_at,
"expires_at": self.expires_at,
"policy_version": self.policy_version,
"key_id": self.key_id,
"nonce": self.nonce,
}
def canonical_bytes(self) -> bytes:
return canonical_json_bytes(self.unsigned_dict())
def as_dict(self) -> dict[str, Any]:
value = self.unsigned_dict()
value["signature"] = self.signature
return value
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "RunProvenance":
return cls(**dict(value))
class ProvenanceIssuer:
"""只由 Owner 或 AIOS Control Plane 持有的签发服务。"""
def __init__(self, signer: Signer, store: "SQLiteProvenanceStore") -> None:
self.signer = signer
self.store = store
def issue(
self,
*,
run_id: str,
issuer: str,
tenant_id: str,
task_id: str,
agent_id: str,
principal_id: str,
allowed_paths: Sequence[str] = tuple(REQUIRED_PRODUCTION_PATHS),
allowed_git_operations: Sequence[str] = tuple(item.value for item in GitOperation),
ttl_seconds: int = 3600,
policy_version: str = PRODUCTION_POLICY_VERSION,
provenance_id: str | None = None,
nonce: str | None = None,
now: datetime | None = None,
) -> RunProvenance:
if ttl_seconds <= 0:
raise ValueError("ttl_seconds must be positive")
normalized_paths = frozenset(ExecutionPath(item).value for item in allowed_paths)
if policy_version == PRODUCTION_POLICY_VERSION and normalized_paths != REQUIRED_PRODUCTION_PATHS:
missing = sorted(REQUIRED_PRODUCTION_PATHS - normalized_paths)
extra = sorted(normalized_paths - REQUIRED_PRODUCTION_PATHS)
raise ProvenanceDenied(
f"production provenance must cover exactly seven paths; missing={missing}, extra={extra}"
)
normalized_operations = tuple(
sorted({GitOperation(item).value for item in allowed_git_operations})
)
if ExecutionPath.GIT_OPERATION.value in normalized_paths and not normalized_operations:
raise ProvenanceDenied("git_operation path requires explicit allowed Git operations")
issued = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
unsigned = RunProvenance(
provenance_id=provenance_id or f"prov_{uuid.uuid4().hex}",
run_id=run_id,
issuer=issuer,
tenant_id=tenant_id,
task_id=task_id,
agent_id=agent_id,
principal_id=principal_id,
allowed_paths=tuple(normalized_paths),
allowed_git_operations=normalized_operations,
issued_at=issued.isoformat().replace("+00:00", "Z"),
expires_at=(issued + timedelta(seconds=ttl_seconds))
.isoformat()
.replace("+00:00", "Z"),
policy_version=policy_version,
key_id=self.signer.key_id,
nonce=nonce or uuid.uuid4().hex,
)
signed = replace(unsigned, signature=self.signer.sign(unsigned.canonical_bytes()))
self.store.save_issued(signed)
return signed
@dataclass(frozen=True)
class ProvenanceAuthorization:
authorization_id: str
provenance_id: str
run_id: str
tenant_id: str
task_id: str
agent_id: str
principal_id: str
execution_path: str
git_operation: str | None
request_digest: str
policy_version: str
key_id: str
public_key_fingerprint: str
authorized_at: str = field(default_factory=utc_now)
def as_dict(self) -> dict[str, Any]:
return asdict(self)
class SQLiteProvenanceStore:
"""持久化 Provenance、状态、授权使用和不可变事件。"""
def __init__(self, path: str | Path) -> None:
self.path = Path(path).resolve()
self.path.parent.mkdir(parents=True, exist_ok=True)
self._lock = threading.RLock()
self._connection = sqlite3.connect(
str(self.path), timeout=30.0, check_same_thread=False
)
self._connection.row_factory = sqlite3.Row
with self._lock:
self._connection.execute("PRAGMA journal_mode=WAL")
self._connection.execute("PRAGMA synchronous=FULL")
self._connection.execute("PRAGMA foreign_keys=ON")
self._connection.executescript(
"""
CREATE TABLE IF NOT EXISTS provenance_records (
provenance_id TEXT PRIMARY KEY,
payload_json TEXT NOT NULL,
status TEXT NOT NULL,
signature_verified INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS provenance_usage (
authorization_id TEXT PRIMARY KEY,
provenance_id TEXT NOT NULL,
execution_path TEXT NOT NULL,
git_operation TEXT NOT NULL DEFAULT '',
request_digest TEXT NOT NULL,
authorized_at TEXT NOT NULL,
receipt_json TEXT NOT NULL,
UNIQUE(provenance_id, execution_path, git_operation, request_digest),
FOREIGN KEY(provenance_id) REFERENCES provenance_records(provenance_id)
);
CREATE TABLE IF NOT EXISTS provenance_events (
event_id TEXT PRIMARY KEY,
provenance_id TEXT NOT NULL,
event_type TEXT NOT NULL,
details_json TEXT NOT NULL,
created_at TEXT NOT NULL,
FOREIGN KEY(provenance_id) REFERENCES provenance_records(provenance_id)
);
CREATE INDEX IF NOT EXISTS idx_provenance_usage_id
ON provenance_usage(provenance_id, authorized_at);
CREATE INDEX IF NOT EXISTS idx_provenance_events_id
ON provenance_events(provenance_id, created_at);
"""
)
self._connection.commit()
def close(self) -> None:
with self._lock:
self._connection.close()
def _event(
self,
provenance_id: str,
event_type: str,
details: Mapping[str, Any] | None = None,
) -> None:
self._connection.execute(
"INSERT INTO provenance_events VALUES (?, ?, ?, ?, ?)",
(
f"pevt_{uuid.uuid4().hex}",
provenance_id,
event_type,
json.dumps(dict(details or {}), ensure_ascii=False, sort_keys=True),
utc_now(),
),
)
def save_issued(self, provenance: RunProvenance) -> None:
payload = json.dumps(
provenance.as_dict(), ensure_ascii=False, sort_keys=True, separators=(",", ":")
)
now = utc_now()
with self._lock:
self._connection.execute("BEGIN IMMEDIATE")
try:
prior = self._connection.execute(
"SELECT payload_json FROM provenance_records WHERE provenance_id = ?",
(provenance.provenance_id,),
).fetchone()
if prior is not None:
if prior["payload_json"] != payload:
raise ProvenanceDenied("provenance_id reused with different payload")
self._connection.rollback()
return
self._connection.execute(
"INSERT INTO provenance_records VALUES (?, ?, ?, 0, ?, ?)",
(
provenance.provenance_id,
payload,
ProvenanceStatus.ISSUED.value,
now,
now,
),
)
self._event(provenance.provenance_id, "issued")
self._connection.commit()
except Exception:
self._connection.rollback()
raise
def get(self, provenance_id: str) -> RunProvenance:
with self._lock:
row = self._connection.execute(
"SELECT payload_json FROM provenance_records WHERE provenance_id = ?",
(required(provenance_id, "provenance_id"),),
).fetchone()
if row is None:
raise ProvenanceNotFound("provenance not found")
value = json.loads(row["payload_json"])
if not isinstance(value, Mapping):
raise ProvenanceError("stored provenance payload is invalid")
return RunProvenance.from_dict(value)
def status(self, provenance_id: str) -> ProvenanceStatus:
with self._lock:
row = self._connection.execute(
"SELECT status FROM provenance_records WHERE provenance_id = ?",
(required(provenance_id, "provenance_id"),),
).fetchone()
if row is None:
raise ProvenanceNotFound("provenance not found")
return ProvenanceStatus(row["status"])
def mark_verified(self, provenance_id: str, *, fingerprint: str) -> None:
with self._lock:
self._connection.execute("BEGIN IMMEDIATE")
try:
current = self.status(provenance_id)
if current in {ProvenanceStatus.REVOKED, ProvenanceStatus.EXPIRED}:
raise ProvenanceDenied(f"cannot verify provenance in state {current.value}")
next_status = (
ProvenanceStatus.USED.value
if current == ProvenanceStatus.USED
else ProvenanceStatus.VERIFIED.value
)
self._connection.execute(
"UPDATE provenance_records SET status = ?, signature_verified = 1, updated_at = ? WHERE provenance_id = ?",
(next_status, utc_now(), provenance_id),
)
self._event(
provenance_id,
"verified",
{"public_key_fingerprint": fingerprint},
)
self._connection.commit()
except Exception:
self._connection.rollback()
raise
def mark_expired(self, provenance_id: str) -> None:
with self._lock:
self._connection.execute("BEGIN IMMEDIATE")
try:
current = self.status(provenance_id)
if current == ProvenanceStatus.REVOKED:
self._connection.rollback()
return
self._connection.execute(
"UPDATE provenance_records SET status = ?, updated_at = ? WHERE provenance_id = ?",
(ProvenanceStatus.EXPIRED.value, utc_now(), provenance_id),
)
self._event(provenance_id, "expired")
self._connection.commit()
except Exception:
self._connection.rollback()
raise
def revoke(self, provenance_id: str, *, reason: str, principal_id: str) -> None:
with self._lock:
self._connection.execute("BEGIN IMMEDIATE")
try:
self.get(provenance_id)
self._connection.execute(
"UPDATE provenance_records SET status = ?, updated_at = ? WHERE provenance_id = ?",
(ProvenanceStatus.REVOKED.value, utc_now(), provenance_id),
)
self._event(
provenance_id,
"revoked",
{
"reason": required(reason, "reason"),
"principal_id": required(principal_id, "principal_id"),
},
)
self._connection.commit()
except Exception:
self._connection.rollback()
raise
def record_authorization(self, authorization: ProvenanceAuthorization) -> None:
receipt = json.dumps(
authorization.as_dict(), ensure_ascii=False, sort_keys=True, separators=(",", ":")
)
with self._lock:
self._connection.execute("BEGIN IMMEDIATE")
try:
current = self.status(authorization.provenance_id)
if current in {ProvenanceStatus.REVOKED, ProvenanceStatus.EXPIRED}:
raise ProvenanceDenied(f"provenance state is {current.value}")
try:
self._connection.execute(
"INSERT INTO provenance_usage VALUES (?, ?, ?, ?, ?, ?, ?)",
(
authorization.authorization_id,
authorization.provenance_id,
authorization.execution_path,
authorization.git_operation or "",
authorization.request_digest,
authorization.authorized_at,
receipt,
),
)
except sqlite3.IntegrityError as exc:
raise ProvenanceReplay(
"identical provenance path/operation/request digest already used"
) from exc
self._connection.execute(
"UPDATE provenance_records SET status = ?, updated_at = ? WHERE provenance_id = ?",
(
ProvenanceStatus.USED.value,
authorization.authorized_at,
authorization.provenance_id,
),
)
self._event(
authorization.provenance_id,
"used",
{
"authorization_id": authorization.authorization_id,
"execution_path": authorization.execution_path,
"git_operation": authorization.git_operation,
"request_digest": authorization.request_digest,
},
)
self._connection.commit()
except Exception:
self._connection.rollback()
raise
def authorization_count(self, provenance_id: str) -> int:
with self._lock:
row = self._connection.execute(
"SELECT COUNT(*) AS total FROM provenance_usage WHERE provenance_id = ?",
(required(provenance_id, "provenance_id"),),
).fetchone()
return int(row["total"])
def events(self, provenance_id: str) -> list[dict[str, Any]]:
with self._lock:
rows = self._connection.execute(
"SELECT event_type, details_json, created_at FROM provenance_events WHERE provenance_id = ? ORDER BY created_at, event_id",
(required(provenance_id, "provenance_id"),),
).fetchall()
return [
{
"event_type": row["event_type"],
"details": json.loads(row["details_json"]),
"created_at": row["created_at"],
}
for row in rows
]
class ProvenanceVerifier:
"""验证 Owner 签名、上下文、路径、Git 操作、时效和重放。"""
def __init__(
self,
store: SQLiteProvenanceStore,
verifiers: Mapping[str, SignatureVerifier],
) -> None:
if not verifiers:
raise ValueError("at least one public key verifier is required")
self.store = store
self.verifiers = {required(key, "key_id"): value for key, value in verifiers.items()}
def authorize(
self,
*,
provenance_id: str,
run_id: str,
tenant_id: str,
task_id: str,
agent_id: str,
principal_id: str,
execution_path: str,
request_digest: str,
git_operation: str | None = None,
now: datetime | None = None,
) -> ProvenanceAuthorization:
provenance = self.store.get(provenance_id)
status = self.store.status(provenance.provenance_id)
if status == ProvenanceStatus.REVOKED:
raise ProvenanceRevoked("provenance is revoked")
current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
if current >= parse_timestamp(provenance.expires_at):
self.store.mark_expired(provenance.provenance_id)
raise ProvenanceExpired("provenance is expired")
verifier = self.verifiers.get(provenance.key_id)
if verifier is None:
raise ProvenanceSignatureError("no trusted public key for provenance key_id")
if not provenance.signature or not verifier.verify(
provenance.canonical_bytes(), provenance.signature
):
raise ProvenanceSignatureError("invalid provenance signature")
expected = {
"run_id": required(run_id, "run_id"),
"tenant_id": required(tenant_id, "tenant_id"),
"task_id": required(task_id, "task_id"),
"agent_id": required(agent_id, "agent_id"),
"principal_id": required(principal_id, "principal_id"),
}
for field_name, expected_value in expected.items():
if getattr(provenance, field_name) != expected_value:
raise ProvenanceDenied(f"provenance {field_name} mismatch")
path = ExecutionPath(execution_path).value
if path not in provenance.allowed_paths:
raise ProvenanceDenied(f"execution path is not allowed: {path}")
operation: str | None = None
if path == ExecutionPath.GIT_OPERATION.value:
if git_operation is None:
raise ProvenanceDenied("git operation is required for git_operation path")
operation = GitOperation(git_operation).value
if operation not in provenance.allowed_git_operations:
raise ProvenanceDenied(f"Git operation is not allowed: {operation}")
elif git_operation is not None:
raise ProvenanceDenied("git_operation is only valid for git_operation path")
digest = _validate_digest(request_digest)
self.store.mark_verified(
provenance.provenance_id, fingerprint=verifier.fingerprint
)
authorization = ProvenanceAuthorization(
authorization_id=f"pauth_{uuid.uuid4().hex}",
provenance_id=provenance.provenance_id,
run_id=provenance.run_id,
tenant_id=provenance.tenant_id,
task_id=provenance.task_id,
agent_id=provenance.agent_id,
principal_id=provenance.principal_id,
execution_path=path,
git_operation=operation,
request_digest=digest,
policy_version=provenance.policy_version,
key_id=provenance.key_id,
public_key_fingerprint=verifier.fingerprint,
)
self.store.record_authorization(authorization)
return authorization
class ProvenanceControlledWriter:
"""七路径统一策略入口并持久写入授权回执。"""
def __init__(
self,
verifier: ProvenanceVerifier,
receipt_root: str | Path,
*,
writer_id: str = "aios-v2-provenance-writer",
) -> None:
self.verifier = verifier
self.receipt_root = Path(receipt_root).resolve()
self.receipt_root.mkdir(parents=True, exist_ok=True)
self.writer_id = required(writer_id, "writer_id")
self._lock = threading.RLock()
@staticmethod
def _component(value: str) -> str:
normalized = required(value, "path component")
if normalized in {".", ".."} or "/" in normalized or "\\" in normalized:
raise ProvenanceDenied("invalid receipt path component")
return normalized
def authorize(self, **values: Any) -> ProvenanceAuthorization:
authorization = self.verifier.authorize(**values)
tenant = self._component(authorization.tenant_id)
provenance_id = self._component(authorization.provenance_id)
target_dir = self.receipt_root / tenant / provenance_id
target = target_dir / f"{authorization.authorization_id}.json"
payload = authorization.as_dict()
payload["writer_id"] = self.writer_id
with self._lock:
target_dir.mkdir(parents=True, exist_ok=True)
temporary = target.with_suffix(f".tmp-{os.getpid()}")
temporary.write_text(
json.dumps(payload, ensure_ascii=False, sort_keys=True, indent=2) + "\n",
encoding="utf-8",
)
os.chmod(temporary, 0o640)
os.replace(temporary, target)
return authorization
def authorize_code_change(self, **values: Any) -> ProvenanceAuthorization:
return self.authorize(execution_path=ExecutionPath.CODE_CHANGE.value, **values)
def authorize_git_operation(
self, *, git_operation: str, **values: Any
) -> ProvenanceAuthorization:
return self.authorize(
execution_path=ExecutionPath.GIT_OPERATION.value,
git_operation=git_operation,
**values,
)
def authorize_deployment(self, **values: Any) -> ProvenanceAuthorization:
return self.authorize(execution_path=ExecutionPath.DEPLOYMENT.value, **values)
def authorize_runtime_execute(self, **values: Any) -> ProvenanceAuthorization:
return self.authorize(execution_path=ExecutionPath.RUNTIME_EXECUTE.value, **values)
def authorize_evidence_write(self, **values: Any) -> ProvenanceAuthorization:
return self.authorize(execution_path=ExecutionPath.EVIDENCE_WRITE.value, **values)
def authorize_rollback(self, **values: Any) -> ProvenanceAuthorization:
return self.authorize(execution_path=ExecutionPath.ROLLBACK.value, **values)
def authorize_audit(self, **values: Any) -> ProvenanceAuthorization:
return self.authorize(execution_path=ExecutionPath.AUDIT.value, **values)
@@ -1,381 +0,0 @@
#!/usr/bin/env python3
"""AIOS V2 Run Provenance 与最小权限 Grant 运维 CLI。
私钥只用于签发;授权命令只加载一个与 ``--path`` 对应的 Grant。所有输出均为
不含 Secret 的 JSON。旧 ``issue`` 仅为候选兼容,生产必须使用
``issue-workflow`` 与 ``issue-grant``。
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from pathlib import Path
from typing import Any, Mapping, Sequence
from .provenance import (
Ed25519PrivateKeySigner,
Ed25519PublicKeyVerifier,
ExecutionPath,
GitOperation,
ProvenanceIssuer,
SQLiteProvenanceStore,
)
from .provenance_grants import (
GrantIssuer,
GrantType,
GrantVerifier,
SQLiteGrantStore,
WorkflowProvenanceIssuer,
)
from .provenance_operations import (
OperationContext,
ProvenanceOperationGate,
context_from_environment,
load_request_file,
)
from .provenance_scoped import ScopedLeastPrivilegeProvenanceWriter
def _json_object(value: str | None, file_path: str | None) -> Mapping[str, Any]:
if value and file_path:
raise ValueError("request-json and request-file are mutually exclusive")
if file_path:
return load_request_file(file_path)
parsed = json.loads(value or "{}")
if not isinstance(parsed, Mapping):
raise ValueError("request JSON must be an object")
return dict(parsed)
def _csv(value: str | None) -> tuple[str, ...]:
return tuple(item.strip() for item in str(value or "").split(",") if item.strip())
def _write_json(path: str | None, payload: Mapping[str, Any]) -> None:
encoded = json.dumps(dict(payload), ensure_ascii=False, sort_keys=True, indent=2) + "\n"
if path:
target = Path(path).resolve()
target.parent.mkdir(parents=True, exist_ok=True)
temporary = target.with_suffix(f".tmp-{os.getpid()}")
temporary.write_text(encoded, encoding="utf-8")
os.chmod(temporary, 0o640)
os.replace(temporary, target)
print(encoded, end="")
def _parent_store(args: argparse.Namespace) -> SQLiteProvenanceStore:
return SQLiteProvenanceStore(args.store_file)
def _grant_store(args: argparse.Namespace) -> SQLiteGrantStore:
return SQLiteGrantStore(args.store_file)
def _context(args: argparse.Namespace) -> OperationContext:
if args.context_from_env:
return context_from_environment()
values = {
"provenance_id": args.provenance_id,
"run_id": args.run_id,
"tenant_id": args.tenant_id,
"task_id": args.task_id,
"agent_id": args.agent_id,
"principal_id": args.principal_id,
}
missing = [name for name, value in values.items() if not str(value or "").strip()]
if missing:
raise ValueError(f"missing context arguments: {missing}")
return OperationContext(**{name: str(value).strip() for name, value in values.items()})
def command_issue_legacy(args: argparse.Namespace) -> int:
store = _parent_store(args)
signer = Ed25519PrivateKeySigner.from_file(args.private_key_file, key_id=args.key_id)
provenance = ProvenanceIssuer(signer, store).issue(
run_id=args.run_id,
issuer=args.issuer,
tenant_id=args.tenant_id,
task_id=args.task_id,
agent_id=args.agent_id,
principal_id=args.principal_id,
allowed_paths=_csv(args.allowed_paths) or tuple(item.value for item in ExecutionPath),
allowed_git_operations=_csv(args.allowed_git_operations)
or tuple(item.value for item in GitOperation),
ttl_seconds=args.ttl_seconds,
policy_version=args.policy_version,
)
_write_json(
args.output,
{
"mode": "legacy-full-rights-candidate-only",
"provenance": provenance.as_dict(),
"status": store.status(provenance.provenance_id).value,
"secret_value_stored": False,
},
)
return 0
def command_issue_workflow(args: argparse.Namespace) -> int:
store = _parent_store(args)
signer = Ed25519PrivateKeySigner.from_file(args.private_key_file, key_id=args.key_id)
workflow = WorkflowProvenanceIssuer(signer, store).issue(
run_id=args.run_id,
issuer=args.issuer,
tenant_id=args.tenant_id,
task_id=args.task_id,
agent_id=args.agent_id,
principal_id=args.principal_id,
ttl_seconds=args.ttl_seconds,
)
_write_json(
args.output,
{
"mode": "least-privilege-workflow",
"workflow_provenance": workflow.as_dict(),
"status": store.status(workflow.provenance_id).value,
"secret_value_stored": False,
},
)
return 0
def command_issue_grant(args: argparse.Namespace) -> int:
parent_store = _parent_store(args)
grant_store = _grant_store(args)
signer = Ed25519PrivateKeySigner.from_file(args.private_key_file, key_id=args.key_id)
grant = GrantIssuer(signer, parent_store, grant_store).issue(
parent_provenance_id=args.parent_provenance_id,
grant_type=GrantType(args.grant_type),
ttl_seconds=args.ttl_seconds,
allowed_git_operations=_csv(args.allowed_git_operations),
)
_write_json(
args.output,
{
"grant": grant.as_dict(),
"status": grant_store.status(grant.grant_id).value,
"secret_value_stored": False,
},
)
return 0
def _single_scope_writer(args: argparse.Namespace) -> ScopedLeastPrivilegeProvenanceWriter:
path = ExecutionPath(args.path).value
parent_store = _parent_store(args)
grant_store = _grant_store(args)
verifier = Ed25519PublicKeyVerifier.from_file(args.public_key_file, key_id=args.key_id)
trusted = GrantVerifier(parent_store, grant_store, {args.key_id: verifier})
return ScopedLeastPrivilegeProvenanceWriter(
trusted,
args.receipt_dir,
{path: args.grant_id},
required_paths=(path,),
writer_id=f"aios-v2-cli-{path}",
)
def command_authorize(args: argparse.Namespace) -> int:
writer = _single_scope_writer(args)
context = _context(args)
receipt = ProvenanceOperationGate(writer).authorize(
context,
execution_path=args.path,
operation=args.operation,
request=_json_object(args.request_json, args.request_file),
)
_write_json(args.output, {"authorization": receipt.as_dict()})
return 0
def command_revoke(args: argparse.Namespace) -> int:
store = _parent_store(args)
store.revoke(args.provenance_id, reason=args.reason, principal_id=args.principal_id)
_write_json(
args.output,
{"provenance_id": args.provenance_id, "status": store.status(args.provenance_id).value},
)
return 0
def command_status(args: argparse.Namespace) -> int:
store = _parent_store(args)
provenance = store.get(args.provenance_id)
grant_store = _grant_store(args)
_write_json(
args.output,
{
"provenance": provenance.as_dict(),
"status": store.status(args.provenance_id).value,
"authorization_count": store.authorization_count(args.provenance_id),
"events": store.events(args.provenance_id),
"grants": [item.as_dict() for item in grant_store.list_for_parent(args.provenance_id)],
},
)
return 0
def command_grant_revoke(args: argparse.Namespace) -> int:
store = _grant_store(args)
store.revoke(args.grant_id, reason=args.reason, principal_id=args.principal_id)
_write_json(
args.output,
{"grant_id": args.grant_id, "status": store.status(args.grant_id).value},
)
return 0
def command_grant_status(args: argparse.Namespace) -> int:
store = _grant_store(args)
grant = store.get(args.grant_id)
_write_json(
args.output,
{"grant": grant.as_dict(), "status": store.status(args.grant_id).value},
)
return 0
def add_common_store(parser: argparse.ArgumentParser) -> None:
parser.add_argument(
"--store-file",
default=os.getenv(
"AIOS_PROVENANCE_STORE_FILE",
"/var/lib/aios-v2/run-provenance.sqlite3",
),
)
parser.add_argument("--output", default=None)
def add_signing_context(parser: argparse.ArgumentParser) -> None:
parser.add_argument("--private-key-file", required=True)
parser.add_argument("--key-id", required=True)
def add_workflow_identity(parser: argparse.ArgumentParser) -> None:
parser.add_argument("--run-id", required=True)
parser.add_argument("--issuer", required=True)
parser.add_argument("--tenant-id", required=True)
parser.add_argument("--task-id", required=True)
parser.add_argument("--agent-id", required=True)
parser.add_argument("--principal-id", required=True)
def add_context(parser: argparse.ArgumentParser) -> None:
parser.add_argument("--context-from-env", action="store_true")
parser.add_argument("--provenance-id")
parser.add_argument("--run-id")
parser.add_argument("--tenant-id")
parser.add_argument("--task-id")
parser.add_argument("--agent-id")
parser.add_argument("--principal-id")
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description="AIOS V2 Run Provenance CLI")
subcommands = parser.add_subparsers(dest="command", required=True)
issue = subcommands.add_parser("issue", help="候选兼容:签发全权限 Provenance;生产禁止")
add_common_store(issue)
add_signing_context(issue)
add_workflow_identity(issue)
issue.add_argument("--allowed-paths")
issue.add_argument("--allowed-git-operations")
issue.add_argument("--ttl-seconds", type=int, default=3600)
issue.add_argument("--policy-version", default="aios.v2.candidate.legacy")
issue.set_defaults(handler=command_issue_legacy)
workflow = subcommands.add_parser(
"issue-workflow", help="签发不含操作权限的父 Workflow Provenance"
)
add_common_store(workflow)
add_signing_context(workflow)
add_workflow_identity(workflow)
workflow.add_argument("--ttl-seconds", type=int, default=14400)
workflow.set_defaults(handler=command_issue_workflow)
grant = subcommands.add_parser("issue-grant", help="签发单一路径最小权限 Grant")
add_common_store(grant)
add_signing_context(grant)
grant.add_argument("--parent-provenance-id", required=True)
grant.add_argument("--grant-type", choices=[item.value for item in GrantType], required=True)
grant.add_argument("--allowed-git-operations")
grant.add_argument("--ttl-seconds", type=int, default=900)
grant.set_defaults(handler=command_issue_grant)
authorize = subcommands.add_parser("authorize", help="使用单一子 Grant 授权一次运维操作")
add_common_store(authorize)
add_context(authorize)
authorize.add_argument("--public-key-file", required=True)
authorize.add_argument("--key-id", required=True)
authorize.add_argument("--grant-id", required=True)
authorize.add_argument(
"--receipt-dir",
default=os.getenv(
"AIOS_PROVENANCE_RECEIPT_DIR",
"/var/lib/aios-v2/provenance-receipts",
),
)
authorize.add_argument(
"--path",
required=True,
choices=[
ExecutionPath.CODE_CHANGE.value,
ExecutionPath.DEPLOYMENT.value,
ExecutionPath.RUNTIME_EXECUTE.value,
ExecutionPath.EVIDENCE_WRITE.value,
ExecutionPath.ROLLBACK.value,
ExecutionPath.AUDIT.value,
],
)
authorize.add_argument("--operation", required=True)
request_group = authorize.add_mutually_exclusive_group()
request_group.add_argument("--request-json")
request_group.add_argument("--request-file")
authorize.set_defaults(handler=command_authorize)
revoke = subcommands.add_parser("revoke", help="撤销父 Workflow Provenance")
add_common_store(revoke)
revoke.add_argument("--provenance-id", required=True)
revoke.add_argument("--reason", required=True)
revoke.add_argument("--principal-id", required=True)
revoke.set_defaults(handler=command_revoke)
status = subcommands.add_parser("status", help="查询父 Workflow 与子 Grant")
add_common_store(status)
status.add_argument("--provenance-id", required=True)
status.set_defaults(handler=command_status)
grant_revoke = subcommands.add_parser("grant-revoke", help="单独撤销子 Grant")
add_common_store(grant_revoke)
grant_revoke.add_argument("--grant-id", required=True)
grant_revoke.add_argument("--reason", required=True)
grant_revoke.add_argument("--principal-id", required=True)
grant_revoke.set_defaults(handler=command_grant_revoke)
grant_status = subcommands.add_parser("grant-status", help="查询子 Grant 状态")
add_common_store(grant_status)
grant_status.add_argument("--grant-id", required=True)
grant_status.set_defaults(handler=command_grant_status)
return parser
def main(argv: Sequence[str] | None = None) -> int:
args = build_parser().parse_args(argv)
try:
return int(args.handler(args))
except Exception as exc:
print(
json.dumps(
{"status": "denied", "error": type(exc).__name__, "message": str(exc)},
ensure_ascii=False,
),
file=sys.stderr,
)
return 1
if __name__ == "__main__":
raise SystemExit(main())
@@ -1,416 +0,0 @@
"""受最小权限 Run Provenance Grant 约束的 Git 写入器。
所有 Git 操作都使用参数数组执行。对于 Commit 和 Evidence Commit,先计算
工作树内容摘要并完成 Provenance 授权,之后才允许执行 ``git add``,避免未授权
请求改变 Git Index。
"""
from __future__ import annotations
import hashlib
import os
import re
import subprocess
import uuid
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Any, Mapping, Protocol, Sequence
from .provenance import GitOperation, ProvenanceDenied, digest_payload, utc_now
_BRANCH_PATTERN = re.compile(r"^(?![-./])(?!.*\.\.)(?!.*//)[A-Za-z0-9._/-]{1,240}$")
_SECRET_URL = re.compile(r"(?i)(https?://)([^/@\s]+)@")
_TOKENISH = re.compile(r"(?i)(token|password|secret|authorization)([=:]\s*)([^\s]+)")
class GitAuthorizationWriter(Protocol):
def authorize_code_change(self, **values: Any) -> Any: ...
def authorize_git_operation(self, *, git_operation: str, **values: Any) -> Any: ...
class ProvenanceGitError(RuntimeError):
"""Git 操作被拒绝或执行失败。"""
@dataclass(frozen=True)
class GitExecutionContext:
provenance_id: str
run_id: str
tenant_id: str
task_id: str
agent_id: str
principal_id: str
@dataclass(frozen=True)
class GitCommandResult:
operation_id: str
operation: str
authorization_ids: Sequence[str]
command: Sequence[str]
exit_code: int
stdout: str
stderr: str
head_before: str | None
head_after: str | None
started_at: str
completed_at: str
def as_dict(self) -> dict[str, Any]:
return asdict(self)
class ProvenanceGitWriter:
"""执行经过单路径 Grant 授权的窄范围 Git 操作。"""
def __init__(
self,
repo: str | Path,
writer: GitAuthorizationWriter,
*,
timeout_seconds: float = 120.0,
output_limit: int = 16000,
) -> None:
self.repo = Path(repo).resolve()
self.writer = writer
self.timeout_seconds = float(timeout_seconds)
self.output_limit = int(output_limit)
if self.timeout_seconds <= 0 or self.output_limit < 1024:
raise ValueError("invalid Git writer limits")
self._run_raw(["rev-parse", "--is-inside-work-tree"], check=True)
@staticmethod
def _validate_branch(branch: str) -> str:
normalized = str(branch or "").strip()
if not _BRANCH_PATTERN.fullmatch(normalized) or normalized.endswith((".", "/")):
raise ProvenanceGitError("invalid Git branch name")
if normalized in {"main", "master"}:
raise ProvenanceGitError("direct main/master operation is prohibited")
return normalized
def _validate_path(self, value: str | Path) -> str:
raw = Path(value)
candidate = (self.repo / raw).resolve() if not raw.is_absolute() else raw.resolve()
try:
relative = candidate.relative_to(self.repo)
except ValueError as exc:
raise ProvenanceGitError("Git path escapes repository") from exc
if str(relative) in {"", "."}:
raise ProvenanceGitError("repository root is not a valid file path")
return relative.as_posix()
@staticmethod
def _redact(value: str, limit: int) -> str:
text = _SECRET_URL.sub(r"\1***@", str(value or ""))
text = _TOKENISH.sub(r"\1\2***", text)
if len(text) > limit:
return text[:limit] + "\n...[truncated]"
return text
def _head(self) -> str | None:
process = self._run_raw(["rev-parse", "HEAD"], check=False)
return process.stdout.strip() if process.returncode == 0 else None
def _run_raw(
self,
args: Sequence[str],
*,
check: bool,
env: Mapping[str, str] | None = None,
) -> subprocess.CompletedProcess[str]:
command = ["git", "-C", str(self.repo), *[str(item) for item in args]]
safe_env = {
**os.environ,
"GIT_TERMINAL_PROMPT": "0",
"GIT_CONFIG_NOSYSTEM": os.environ.get("GIT_CONFIG_NOSYSTEM", "1"),
}
if env:
safe_env.update({str(key): str(value) for key, value in env.items()})
process = subprocess.run(
command,
text=True,
capture_output=True,
timeout=self.timeout_seconds,
env=safe_env,
)
if check and process.returncode != 0:
raise ProvenanceGitError(
self._redact(process.stderr or process.stdout, self.output_limit)
or f"Git command failed with exit code {process.returncode}"
)
return process
def _path_digest(self, relative: str) -> str:
target = self.repo / relative
if target.exists():
if not target.is_file():
raise ProvenanceGitError("Git writer accepts file paths only")
hasher = hashlib.sha256()
with target.open("rb") as handle:
while True:
block = handle.read(65536)
if not block:
break
hasher.update(block)
return hasher.hexdigest()
tracked = self._run_raw(["ls-files", "--error-unmatch", "--", relative], check=False)
if tracked.returncode != 0:
raise ProvenanceGitError(f"path does not exist and is not tracked: {relative}")
return "deleted"
def _content_manifest(self, paths: Sequence[str]) -> Mapping[str, str]:
return {path: self._path_digest(path) for path in sorted(set(paths))}
def _authorize(
self,
context: GitExecutionContext,
*,
operation: GitOperation,
request: Mapping[str, Any],
code_change: bool,
) -> list[str]:
request_digest = digest_payload(
{
"operation": operation.value,
"repo": self.repo.name,
"request": dict(request),
"head": self._head(),
}
)
ids: list[str] = []
try:
if code_change:
code_authorization = self.writer.authorize_code_change(
provenance_id=context.provenance_id,
run_id=context.run_id,
tenant_id=context.tenant_id,
task_id=context.task_id,
agent_id=context.agent_id,
principal_id=context.principal_id,
request_digest=digest_payload(
{
"git_operation": operation.value,
"request_digest": request_digest,
}
),
)
ids.append(code_authorization.authorization_id)
git_authorization = self.writer.authorize_git_operation(
provenance_id=context.provenance_id,
run_id=context.run_id,
tenant_id=context.tenant_id,
task_id=context.task_id,
agent_id=context.agent_id,
principal_id=context.principal_id,
git_operation=operation.value,
request_digest=request_digest,
)
ids.append(git_authorization.authorization_id)
return ids
except ProvenanceDenied as exc:
raise ProvenanceGitError(f"Run Provenance denied Git operation: {exc}") from exc
def _run_authorized(
self,
*,
operation: GitOperation,
args: Sequence[str],
authorization_ids: Sequence[str],
) -> GitCommandResult:
started = utc_now()
before = self._head()
process = self._run_raw(args, check=False)
after = self._head()
result = GitCommandResult(
operation_id=f"gitop_{uuid.uuid4().hex}",
operation=operation.value,
authorization_ids=tuple(authorization_ids),
command=tuple(["git", "-C", "<repo>", *[str(item) for item in args]]),
exit_code=int(process.returncode),
stdout=self._redact(process.stdout, self.output_limit),
stderr=self._redact(process.stderr, self.output_limit),
head_before=before,
head_after=after,
started_at=started,
completed_at=utc_now(),
)
if process.returncode != 0:
raise ProvenanceGitError(
result.stderr or result.stdout or f"Git {operation.value} failed"
)
return result
def _execute(
self,
context: GitExecutionContext,
*,
operation: GitOperation,
args: Sequence[str],
request: Mapping[str, Any],
code_change: bool,
) -> GitCommandResult:
authorization_ids = self._authorize(
context,
operation=operation,
request=request,
code_change=code_change,
)
return self._run_authorized(
operation=operation,
args=args,
authorization_ids=authorization_ids,
)
def read_ref(self, context: GitExecutionContext, ref: str) -> GitCommandResult:
normalized = str(ref or "").strip()
if not normalized or any(ch.isspace() for ch in normalized):
raise ProvenanceGitError("invalid Git ref")
return self._execute(
context,
operation=GitOperation.READ_REF,
args=["rev-parse", "--verify", normalized],
request={"ref": normalized},
code_change=False,
)
def fetch(
self,
context: GitExecutionContext,
*,
remote: str,
refspecs: Sequence[str],
) -> GitCommandResult:
normalized_remote = str(remote or "").strip()
normalized_refspecs = tuple(str(item).strip() for item in refspecs if str(item).strip())
if not normalized_remote or not normalized_refspecs:
raise ProvenanceGitError("remote and refspecs are required")
if any(item.startswith("+") and ":refs/heads/" in item for item in normalized_refspecs):
raise ProvenanceGitError(
"fetching directly into local branch refs is prohibited; use remote-tracking refs"
)
return self._execute(
context,
operation=GitOperation.FETCH,
args=["fetch", "--prune", normalized_remote, *normalized_refspecs],
request={"remote": normalized_remote, "refspecs": normalized_refspecs},
code_change=False,
)
def create_branch(
self,
context: GitExecutionContext,
*,
branch: str,
start_point: str,
) -> GitCommandResult:
normalized = self._validate_branch(branch)
start = str(start_point or "").strip()
if not start:
raise ProvenanceGitError("start_point is required")
return self._execute(
context,
operation=GitOperation.CREATE_BRANCH,
args=["branch", normalized, start],
request={"branch": normalized, "start_point": start},
code_change=False,
)
def commit(
self,
context: GitExecutionContext,
*,
paths: Sequence[str | Path],
message: str,
) -> GitCommandResult:
normalized_paths = tuple(self._validate_path(item) for item in paths)
normalized_message = str(message or "").strip()
if not normalized_paths or not normalized_message:
raise ProvenanceGitError("commit paths and message are required")
if len(normalized_message) > 500:
raise ProvenanceGitError("commit message is too long")
request = {
"paths": normalized_paths,
"message": normalized_message,
"content_manifest": self._content_manifest(normalized_paths),
}
authorization_ids = self._authorize(
context,
operation=GitOperation.COMMIT,
request=request,
code_change=True,
)
self._run_raw(["add", "--", *normalized_paths], check=True)
return self._run_authorized(
operation=GitOperation.COMMIT,
args=["commit", "--no-gpg-sign", "-m", normalized_message, "--", *normalized_paths],
authorization_ids=authorization_ids,
)
def merge(
self,
context: GitExecutionContext,
*,
source_ref: str,
message: str,
) -> GitCommandResult:
source = str(source_ref or "").strip()
normalized_message = str(message or "").strip()
if not source or not normalized_message:
raise ProvenanceGitError("source_ref and message are required")
return self._execute(
context,
operation=GitOperation.MERGE,
args=["merge", "--no-ff", "--no-edit", "-m", normalized_message, source],
request={"source_ref": source, "message": normalized_message},
code_change=True,
)
def push(
self,
context: GitExecutionContext,
*,
remote: str,
branch: str,
) -> GitCommandResult:
normalized_remote = str(remote or "").strip()
normalized_branch = self._validate_branch(branch)
if not normalized_remote:
raise ProvenanceGitError("remote is required")
return self._execute(
context,
operation=GitOperation.PUSH,
args=["push", normalized_remote, f"refs/heads/{normalized_branch}:refs/heads/{normalized_branch}"],
request={"remote": normalized_remote, "branch": normalized_branch},
code_change=False,
)
def update_evidence(
self,
context: GitExecutionContext,
*,
paths: Sequence[str | Path],
message: str,
) -> GitCommandResult:
normalized_paths = tuple(self._validate_path(item) for item in paths)
normalized_message = str(message or "").strip()
if not normalized_paths or not normalized_message:
raise ProvenanceGitError("evidence paths and message are required")
if any(not path.startswith(".agent/") for path in normalized_paths):
raise ProvenanceGitError("UPDATE_EVIDENCE may only commit .agent/ paths")
request = {
"paths": normalized_paths,
"message": normalized_message,
"content_manifest": self._content_manifest(normalized_paths),
}
authorization_ids = self._authorize(
context,
operation=GitOperation.UPDATE_EVIDENCE,
request=request,
code_change=False,
)
self._run_raw(["add", "--", *normalized_paths], check=True)
return self._run_authorized(
operation=GitOperation.UPDATE_EVIDENCE,
args=["commit", "--no-gpg-sign", "-m", normalized_message, "--", *normalized_paths],
authorization_ids=authorization_ids,
)
@@ -1,744 +0,0 @@
"""AIOS V2 最小权限 Run Provenance 子 Grant。
父 Workflow Provenance 只绑定 run/tenant/task/agent/principal 身份,不直接授予
操作权限。每个子 Grant 只允许一个 ExecutionPath;Git Grant 还必须限定具体
GitOperation。子 Grant 具备独立 TTL、nonce、状态、撤销和重放防护。
"""
from __future__ import annotations
import json
import os
import sqlite3
import threading
import uuid
from dataclasses import asdict, dataclass, field, replace
from datetime import datetime, timedelta, timezone
from enum import Enum
from pathlib import Path
from typing import Any, Mapping, Sequence
from .provenance import (
ExecutionPath,
GitOperation,
ProvenanceDenied,
ProvenanceExpired,
ProvenanceIssuer,
ProvenanceNotFound,
ProvenanceReplay,
ProvenanceRevoked,
ProvenanceSignatureError,
ProvenanceStatus,
ProvenanceVerifier,
RunProvenance,
SQLiteProvenanceStore,
Signer,
SignatureVerifier,
_validate_digest,
canonical_json_bytes,
parse_timestamp,
required,
utc_now,
)
WORKFLOW_POLICY_VERSION = "aios.v2.workflow.v1"
GRANT_SCHEMA_VERSION = "aios.v2.provenance-grant.v1"
class GrantType(str, Enum):
CODE_CHANGE = "code_change"
GIT = "git"
DEPLOYMENT = "deployment"
RUNTIME = "runtime"
EVIDENCE = "evidence"
ROLLBACK = "rollback"
AUDIT = "audit"
GRANT_PATHS: Mapping[GrantType, ExecutionPath] = {
GrantType.CODE_CHANGE: ExecutionPath.CODE_CHANGE,
GrantType.GIT: ExecutionPath.GIT_OPERATION,
GrantType.DEPLOYMENT: ExecutionPath.DEPLOYMENT,
GrantType.RUNTIME: ExecutionPath.RUNTIME_EXECUTE,
GrantType.EVIDENCE: ExecutionPath.EVIDENCE_WRITE,
GrantType.ROLLBACK: ExecutionPath.ROLLBACK,
GrantType.AUDIT: ExecutionPath.AUDIT,
}
class GrantStatus(str, Enum):
ISSUED = "issued"
VERIFIED = "verified"
USED = "used"
EXPIRED = "expired"
REVOKED = "revoked"
@dataclass(frozen=True)
class ProvenanceGrant:
grant_id: str
parent_provenance_id: str
grant_type: GrantType
execution_path: ExecutionPath
run_id: str
issuer: str
tenant_id: str
task_id: str
agent_id: str
principal_id: str
allowed_git_operations: Sequence[str]
issued_at: str
expires_at: str
policy_version: str
key_id: str
nonce: str
signature: str = ""
schema_version: str = GRANT_SCHEMA_VERSION
def __post_init__(self) -> None:
for name in (
"grant_id",
"parent_provenance_id",
"run_id",
"issuer",
"tenant_id",
"task_id",
"agent_id",
"principal_id",
"issued_at",
"expires_at",
"policy_version",
"key_id",
"nonce",
"schema_version",
):
object.__setattr__(self, name, required(getattr(self, name), name))
if self.schema_version != GRANT_SCHEMA_VERSION:
raise ValueError(f"unsupported grant schema: {self.schema_version}")
grant_type = GrantType(self.grant_type)
path = ExecutionPath(self.execution_path)
if GRANT_PATHS[grant_type] != path:
raise ValueError("grant_type does not match execution_path")
object.__setattr__(self, "grant_type", grant_type)
object.__setattr__(self, "execution_path", path)
operations = tuple(
sorted({GitOperation(item).value for item in self.allowed_git_operations})
)
if grant_type == GrantType.GIT and not operations:
raise ValueError("Git Grant requires explicit Git operations")
if grant_type != GrantType.GIT and operations:
raise ValueError("only Git Grant may contain Git operations")
object.__setattr__(self, "allowed_git_operations", operations)
issued = parse_timestamp(self.issued_at)
expires = parse_timestamp(self.expires_at)
if expires <= issued:
raise ValueError("grant expires_at must be later than issued_at")
object.__setattr__(self, "signature", str(self.signature or "").strip())
def unsigned_dict(self) -> dict[str, Any]:
return {
"schema_version": self.schema_version,
"grant_id": self.grant_id,
"parent_provenance_id": self.parent_provenance_id,
"grant_type": self.grant_type.value,
"execution_path": self.execution_path.value,
"run_id": self.run_id,
"issuer": self.issuer,
"tenant_id": self.tenant_id,
"task_id": self.task_id,
"agent_id": self.agent_id,
"principal_id": self.principal_id,
"allowed_git_operations": list(self.allowed_git_operations),
"issued_at": self.issued_at,
"expires_at": self.expires_at,
"policy_version": self.policy_version,
"key_id": self.key_id,
"nonce": self.nonce,
}
def canonical_bytes(self) -> bytes:
return canonical_json_bytes(self.unsigned_dict())
def as_dict(self) -> dict[str, Any]:
value = self.unsigned_dict()
value["signature"] = self.signature
return value
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "ProvenanceGrant":
return cls(**dict(value))
@dataclass(frozen=True)
class GrantAuthorization:
authorization_id: str
grant_id: str
parent_provenance_id: str
run_id: str
tenant_id: str
task_id: str
agent_id: str
principal_id: str
execution_path: str
git_operation: str | None
request_digest: str
policy_version: str
key_id: str
public_key_fingerprint: str
authorized_at: str = field(default_factory=utc_now)
@property
def provenance_id(self) -> str:
return self.parent_provenance_id
def as_dict(self) -> dict[str, Any]:
return asdict(self)
class WorkflowProvenanceIssuer:
"""签发不携带操作权限的父 Workflow Provenance。"""
def __init__(self, signer: Signer, store: SQLiteProvenanceStore) -> None:
self._issuer = ProvenanceIssuer(signer, store)
def issue(
self,
*,
run_id: str,
issuer: str,
tenant_id: str,
task_id: str,
agent_id: str,
principal_id: str,
ttl_seconds: int = 14400,
provenance_id: str | None = None,
nonce: str | None = None,
now: datetime | None = None,
) -> RunProvenance:
return self._issuer.issue(
run_id=run_id,
issuer=issuer,
tenant_id=tenant_id,
task_id=task_id,
agent_id=agent_id,
principal_id=principal_id,
allowed_paths=(),
allowed_git_operations=(),
ttl_seconds=ttl_seconds,
policy_version=WORKFLOW_POLICY_VERSION,
provenance_id=provenance_id,
nonce=nonce,
now=now,
)
class SQLiteGrantStore:
"""在 Provenance SQLite 数据库中持久化子 Grant 与使用记录。"""
def __init__(self, path: str | Path) -> None:
self.path = Path(path).resolve()
self.path.parent.mkdir(parents=True, exist_ok=True)
self._lock = threading.RLock()
self._connection = sqlite3.connect(
str(self.path), timeout=30.0, check_same_thread=False
)
self._connection.row_factory = sqlite3.Row
with self._lock:
self._connection.execute("PRAGMA journal_mode=WAL")
self._connection.execute("PRAGMA synchronous=FULL")
self._connection.execute("PRAGMA foreign_keys=ON")
self._connection.executescript(
"""
CREATE TABLE IF NOT EXISTS provenance_grants (
grant_id TEXT PRIMARY KEY,
parent_provenance_id TEXT NOT NULL,
grant_type TEXT NOT NULL,
execution_path TEXT NOT NULL,
payload_json TEXT NOT NULL,
status TEXT NOT NULL,
signature_verified INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_grants_parent
ON provenance_grants(parent_provenance_id, grant_type, status);
CREATE TABLE IF NOT EXISTS provenance_grant_usage (
authorization_id TEXT PRIMARY KEY,
grant_id TEXT NOT NULL,
execution_path TEXT NOT NULL,
git_operation TEXT NOT NULL DEFAULT '',
request_digest TEXT NOT NULL,
authorized_at TEXT NOT NULL,
receipt_json TEXT NOT NULL,
UNIQUE(grant_id, execution_path, git_operation, request_digest),
FOREIGN KEY(grant_id) REFERENCES provenance_grants(grant_id)
);
CREATE TABLE IF NOT EXISTS provenance_grant_events (
event_id TEXT PRIMARY KEY,
grant_id TEXT NOT NULL,
event_type TEXT NOT NULL,
details_json TEXT NOT NULL,
created_at TEXT NOT NULL,
FOREIGN KEY(grant_id) REFERENCES provenance_grants(grant_id)
);
"""
)
self._connection.commit()
def close(self) -> None:
with self._lock:
self._connection.close()
def _event(
self,
grant_id: str,
event_type: str,
details: Mapping[str, Any] | None = None,
) -> None:
self._connection.execute(
"INSERT INTO provenance_grant_events VALUES (?, ?, ?, ?, ?)",
(
f"gevt_{uuid.uuid4().hex}",
grant_id,
event_type,
json.dumps(dict(details or {}), ensure_ascii=False, sort_keys=True),
utc_now(),
),
)
def save_issued(self, grant: ProvenanceGrant) -> None:
payload = json.dumps(
grant.as_dict(), ensure_ascii=False, sort_keys=True, separators=(",", ":")
)
now = utc_now()
with self._lock:
self._connection.execute("BEGIN IMMEDIATE")
try:
self._connection.execute(
"INSERT INTO provenance_grants VALUES (?, ?, ?, ?, ?, ?, 0, ?, ?)",
(
grant.grant_id,
grant.parent_provenance_id,
grant.grant_type.value,
grant.execution_path.value,
payload,
GrantStatus.ISSUED.value,
now,
now,
),
)
self._event(grant.grant_id, "issued")
self._connection.commit()
except sqlite3.IntegrityError as exc:
self._connection.rollback()
raise ProvenanceDenied("grant_id already exists") from exc
except Exception:
self._connection.rollback()
raise
def get(self, grant_id: str) -> ProvenanceGrant:
with self._lock:
row = self._connection.execute(
"SELECT payload_json FROM provenance_grants WHERE grant_id = ?",
(required(grant_id, "grant_id"),),
).fetchone()
if row is None:
raise ProvenanceNotFound("provenance grant not found")
value = json.loads(row["payload_json"])
if not isinstance(value, Mapping):
raise ProvenanceDenied("stored grant is invalid")
return ProvenanceGrant.from_dict(value)
def status(self, grant_id: str) -> GrantStatus:
with self._lock:
row = self._connection.execute(
"SELECT status FROM provenance_grants WHERE grant_id = ?",
(required(grant_id, "grant_id"),),
).fetchone()
if row is None:
raise ProvenanceNotFound("provenance grant not found")
return GrantStatus(row["status"])
def mark_verified(self, grant_id: str, *, fingerprint: str) -> None:
with self._lock:
self._connection.execute("BEGIN IMMEDIATE")
try:
current = self.status(grant_id)
if current in {GrantStatus.REVOKED, GrantStatus.EXPIRED}:
raise ProvenanceDenied(f"cannot verify grant in state {current.value}")
next_status = (
GrantStatus.USED.value
if current == GrantStatus.USED
else GrantStatus.VERIFIED.value
)
self._connection.execute(
"UPDATE provenance_grants SET status = ?, signature_verified = 1, updated_at = ? WHERE grant_id = ?",
(next_status, utc_now(), grant_id),
)
self._event(
grant_id,
"verified",
{"public_key_fingerprint": fingerprint},
)
self._connection.commit()
except Exception:
self._connection.rollback()
raise
def mark_expired(self, grant_id: str) -> None:
with self._lock:
self._connection.execute("BEGIN IMMEDIATE")
try:
if self.status(grant_id) == GrantStatus.REVOKED:
self._connection.rollback()
return
self._connection.execute(
"UPDATE provenance_grants SET status = ?, updated_at = ? WHERE grant_id = ?",
(GrantStatus.EXPIRED.value, utc_now(), grant_id),
)
self._event(grant_id, "expired")
self._connection.commit()
except Exception:
self._connection.rollback()
raise
def revoke(self, grant_id: str, *, reason: str, principal_id: str) -> None:
with self._lock:
self._connection.execute("BEGIN IMMEDIATE")
try:
self.get(grant_id)
self._connection.execute(
"UPDATE provenance_grants SET status = ?, updated_at = ? WHERE grant_id = ?",
(GrantStatus.REVOKED.value, utc_now(), grant_id),
)
self._event(
grant_id,
"revoked",
{
"reason": required(reason, "reason"),
"principal_id": required(principal_id, "principal_id"),
},
)
self._connection.commit()
except Exception:
self._connection.rollback()
raise
def record_authorization(self, authorization: GrantAuthorization) -> None:
receipt = json.dumps(
authorization.as_dict(), ensure_ascii=False, sort_keys=True, separators=(",", ":")
)
with self._lock:
self._connection.execute("BEGIN IMMEDIATE")
try:
current = self.status(authorization.grant_id)
if current in {GrantStatus.REVOKED, GrantStatus.EXPIRED}:
raise ProvenanceDenied(f"grant state is {current.value}")
try:
self._connection.execute(
"INSERT INTO provenance_grant_usage VALUES (?, ?, ?, ?, ?, ?, ?)",
(
authorization.authorization_id,
authorization.grant_id,
authorization.execution_path,
authorization.git_operation or "",
authorization.request_digest,
authorization.authorized_at,
receipt,
),
)
except sqlite3.IntegrityError as exc:
raise ProvenanceReplay(
"identical grant path/operation/request digest already used"
) from exc
self._connection.execute(
"UPDATE provenance_grants SET status = ?, updated_at = ? WHERE grant_id = ?",
(GrantStatus.USED.value, authorization.authorized_at, authorization.grant_id),
)
self._event(
authorization.grant_id,
"used",
{
"authorization_id": authorization.authorization_id,
"execution_path": authorization.execution_path,
"git_operation": authorization.git_operation,
"request_digest": authorization.request_digest,
},
)
self._connection.commit()
except Exception:
self._connection.rollback()
raise
def list_for_parent(self, parent_provenance_id: str) -> list[ProvenanceGrant]:
with self._lock:
rows = self._connection.execute(
"SELECT payload_json FROM provenance_grants WHERE parent_provenance_id = ? ORDER BY grant_type, grant_id",
(required(parent_provenance_id, "parent_provenance_id"),),
).fetchall()
return [ProvenanceGrant.from_dict(json.loads(row["payload_json"])) for row in rows]
class GrantIssuer:
"""从父 Workflow Provenance 派生单一路径最小权限 Grant。"""
def __init__(
self,
signer: Signer,
parent_store: SQLiteProvenanceStore,
grant_store: SQLiteGrantStore,
) -> None:
self.signer = signer
self.parent_store = parent_store
self.grant_store = grant_store
def issue(
self,
*,
parent_provenance_id: str,
grant_type: GrantType | str,
ttl_seconds: int = 900,
allowed_git_operations: Sequence[str] = (),
grant_id: str | None = None,
nonce: str | None = None,
now: datetime | None = None,
) -> ProvenanceGrant:
if ttl_seconds <= 0:
raise ValueError("grant ttl_seconds must be positive")
parent = self.parent_store.get(parent_provenance_id)
if parent.policy_version != WORKFLOW_POLICY_VERSION:
raise ProvenanceDenied("child grants require a workflow provenance parent")
if parent.allowed_paths or parent.allowed_git_operations:
raise ProvenanceDenied("workflow provenance parent must not contain operation rights")
parent_status = self.parent_store.status(parent.provenance_id)
if parent_status == ProvenanceStatus.REVOKED:
raise ProvenanceRevoked("workflow provenance is revoked")
issued = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
parent_expiry = parse_timestamp(parent.expires_at)
expires = min(issued + timedelta(seconds=ttl_seconds), parent_expiry)
if expires <= issued:
raise ProvenanceExpired("workflow provenance has expired")
normalized_type = GrantType(grant_type)
operations = tuple(allowed_git_operations)
unsigned = ProvenanceGrant(
grant_id=grant_id or f"grant_{uuid.uuid4().hex}",
parent_provenance_id=parent.provenance_id,
grant_type=normalized_type,
execution_path=GRANT_PATHS[normalized_type],
run_id=parent.run_id,
issuer=parent.issuer,
tenant_id=parent.tenant_id,
task_id=parent.task_id,
agent_id=parent.agent_id,
principal_id=parent.principal_id,
allowed_git_operations=operations,
issued_at=issued.isoformat().replace("+00:00", "Z"),
expires_at=expires.isoformat().replace("+00:00", "Z"),
policy_version=f"{WORKFLOW_POLICY_VERSION}.{normalized_type.value}",
key_id=self.signer.key_id,
nonce=nonce or uuid.uuid4().hex,
)
signed = replace(unsigned, signature=self.signer.sign(unsigned.canonical_bytes()))
self.grant_store.save_issued(signed)
return signed
class GrantVerifier:
"""验证父 Workflow 与子 Grant,确保单一路径最小权限。"""
def __init__(
self,
parent_store: SQLiteProvenanceStore,
grant_store: SQLiteGrantStore,
verifiers: Mapping[str, SignatureVerifier],
) -> None:
if not verifiers:
raise ValueError("at least one trusted verifier is required")
self.parent_store = parent_store
self.grant_store = grant_store
self.verifiers = {required(key, "key_id"): value for key, value in verifiers.items()}
def _verify_parent(self, parent: RunProvenance, *, now: datetime) -> SignatureVerifier:
status = self.parent_store.status(parent.provenance_id)
if status == ProvenanceStatus.REVOKED:
raise ProvenanceRevoked("workflow provenance is revoked")
if now >= parse_timestamp(parent.expires_at):
self.parent_store.mark_expired(parent.provenance_id)
raise ProvenanceExpired("workflow provenance is expired")
verifier = self.verifiers.get(parent.key_id)
if verifier is None or not parent.signature or not verifier.verify(
parent.canonical_bytes(), parent.signature
):
raise ProvenanceSignatureError("invalid workflow provenance signature")
if parent.policy_version != WORKFLOW_POLICY_VERSION:
raise ProvenanceDenied("parent is not a workflow provenance")
if parent.allowed_paths or parent.allowed_git_operations:
raise ProvenanceDenied("workflow provenance must not contain operation rights")
return verifier
def authorize(
self,
*,
grant_id: str,
parent_provenance_id: str,
run_id: str,
tenant_id: str,
task_id: str,
agent_id: str,
principal_id: str,
execution_path: str,
request_digest: str,
git_operation: str | None = None,
now: datetime | None = None,
) -> GrantAuthorization:
current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
parent = self.parent_store.get(parent_provenance_id)
self._verify_parent(parent, now=current)
grant = self.grant_store.get(grant_id)
grant_status = self.grant_store.status(grant.grant_id)
if grant_status == GrantStatus.REVOKED:
raise ProvenanceRevoked("provenance grant is revoked")
if current >= parse_timestamp(grant.expires_at):
self.grant_store.mark_expired(grant.grant_id)
raise ProvenanceExpired("provenance grant is expired")
verifier = self.verifiers.get(grant.key_id)
if verifier is None or not grant.signature or not verifier.verify(
grant.canonical_bytes(), grant.signature
):
raise ProvenanceSignatureError("invalid provenance grant signature")
expected = {
"parent_provenance_id": parent.provenance_id,
"run_id": required(run_id, "run_id"),
"tenant_id": required(tenant_id, "tenant_id"),
"task_id": required(task_id, "task_id"),
"agent_id": required(agent_id, "agent_id"),
"principal_id": required(principal_id, "principal_id"),
}
for field_name, expected_value in expected.items():
if getattr(grant, field_name) != expected_value:
raise ProvenanceDenied(f"grant {field_name} mismatch")
if field_name != "parent_provenance_id" and getattr(parent, field_name) != expected_value:
raise ProvenanceDenied(f"workflow {field_name} mismatch")
path = ExecutionPath(execution_path)
if grant.execution_path != path:
raise ProvenanceDenied(
f"grant permits {grant.execution_path.value}, not {path.value}"
)
operation: str | None = None
if path == ExecutionPath.GIT_OPERATION:
if git_operation is None:
raise ProvenanceDenied("Git operation is required")
operation = GitOperation(git_operation).value
if operation not in grant.allowed_git_operations:
raise ProvenanceDenied(f"Git operation is not allowed: {operation}")
elif git_operation is not None:
raise ProvenanceDenied("git_operation is only valid for a Git Grant")
digest = _validate_digest(request_digest)
self.grant_store.mark_verified(grant.grant_id, fingerprint=verifier.fingerprint)
authorization = GrantAuthorization(
authorization_id=f"gauth_{uuid.uuid4().hex}",
grant_id=grant.grant_id,
parent_provenance_id=parent.provenance_id,
run_id=grant.run_id,
tenant_id=grant.tenant_id,
task_id=grant.task_id,
agent_id=grant.agent_id,
principal_id=grant.principal_id,
execution_path=path.value,
git_operation=operation,
request_digest=digest,
policy_version=grant.policy_version,
key_id=grant.key_id,
public_key_fingerprint=verifier.fingerprint,
)
self.grant_store.record_authorization(authorization)
return authorization
class LeastPrivilegeProvenanceWriter:
"""按路径选择独立子 Grant 的生产 Writer。
``provenance_id`` 参数始终指父 Workflow Provenance;Grant ID 来自受保护
配置映射,调用方无法通过请求体临时扩大权限。
"""
def __init__(
self,
verifier: GrantVerifier,
receipt_root: str | Path,
grant_ids: Mapping[str, str],
*,
writer_id: str = "aios-v2-least-privilege-writer",
) -> None:
self.verifier = verifier
self.receipt_root = Path(receipt_root).resolve()
self.receipt_root.mkdir(parents=True, exist_ok=True)
self.grant_ids = {
ExecutionPath(path).value: required(grant_id, f"grant_id[{path}]")
for path, grant_id in grant_ids.items()
}
missing = sorted(path.value for path in ExecutionPath if path.value not in self.grant_ids)
if missing:
raise ValueError(f"missing least-privilege Grant mappings: {missing}")
self.writer_id = required(writer_id, "writer_id")
self._lock = threading.RLock()
@staticmethod
def _component(value: str) -> str:
normalized = required(value, "path component")
if normalized in {".", ".."} or "/" in normalized or "\\" in normalized:
raise ProvenanceDenied("invalid receipt path component")
return normalized
def authorize(self, **values: Any) -> GrantAuthorization:
path = ExecutionPath(values["execution_path"]).value
grant_id = self.grant_ids[path]
parent_provenance_id = values.pop("provenance_id")
authorization = self.verifier.authorize(
grant_id=grant_id,
parent_provenance_id=parent_provenance_id,
**values,
)
tenant = self._component(authorization.tenant_id)
parent = self._component(authorization.parent_provenance_id)
grant = self._component(authorization.grant_id)
target_dir = self.receipt_root / tenant / parent / grant
target = target_dir / f"{authorization.authorization_id}.json"
payload = authorization.as_dict()
payload["writer_id"] = self.writer_id
with self._lock:
target_dir.mkdir(parents=True, exist_ok=True)
temporary = target.with_suffix(f".tmp-{os.getpid()}")
temporary.write_text(
json.dumps(payload, ensure_ascii=False, sort_keys=True, indent=2) + "\n",
encoding="utf-8",
)
os.chmod(temporary, 0o640)
os.replace(temporary, target)
return authorization
def authorize_code_change(self, **values: Any) -> GrantAuthorization:
return self.authorize(execution_path=ExecutionPath.CODE_CHANGE.value, **values)
def authorize_git_operation(
self, *, git_operation: str, **values: Any
) -> GrantAuthorization:
return self.authorize(
execution_path=ExecutionPath.GIT_OPERATION.value,
git_operation=git_operation,
**values,
)
def authorize_deployment(self, **values: Any) -> GrantAuthorization:
return self.authorize(execution_path=ExecutionPath.DEPLOYMENT.value, **values)
def authorize_runtime_execute(self, **values: Any) -> GrantAuthorization:
return self.authorize(execution_path=ExecutionPath.RUNTIME_EXECUTE.value, **values)
def authorize_evidence_write(self, **values: Any) -> GrantAuthorization:
return self.authorize(execution_path=ExecutionPath.EVIDENCE_WRITE.value, **values)
def authorize_rollback(self, **values: Any) -> GrantAuthorization:
return self.authorize(execution_path=ExecutionPath.ROLLBACK.value, **values)
def authorize_audit(self, **values: Any) -> GrantAuthorization:
return self.authorize(execution_path=ExecutionPath.AUDIT.value, **values)
@@ -1,265 +0,0 @@
"""AIOS V2 运维操作 Provenance Gate 与服务级 Scoped Writer 工厂。"""
from __future__ import annotations
import json
import os
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Any, Mapping, Protocol, Sequence
from .provenance import (
Ed25519PublicKeyVerifier,
ExecutionPath,
ProvenanceAuthorization,
ProvenanceControlledWriter,
ProvenanceDenied,
ProvenanceVerifier,
SQLiteProvenanceStore,
digest_payload,
)
from .provenance_grants import GrantAuthorization, GrantVerifier, SQLiteGrantStore
from .provenance_scoped import ScopedLeastPrivilegeProvenanceWriter
APPLICATION_PROVENANCE_SCOPE = (
ExecutionPath.RUNTIME_EXECUTE.value,
ExecutionPath.EVIDENCE_WRITE.value,
ExecutionPath.AUDIT.value,
)
class AuthorizationWriter(Protocol):
def authorize_deployment(self, **values: Any) -> Any: ...
def authorize_rollback(self, **values: Any) -> Any: ...
def authorize_audit(self, **values: Any) -> Any: ...
def authorize_code_change(self, **values: Any) -> Any: ...
def authorize_evidence_write(self, **values: Any) -> Any: ...
def authorize_runtime_execute(self, **values: Any) -> Any: ...
@dataclass(frozen=True)
class OperationContext:
provenance_id: str
run_id: str
tenant_id: str
task_id: str
agent_id: str
principal_id: str
def as_dict(self) -> dict[str, str]:
return asdict(self)
@dataclass(frozen=True)
class OperationReceipt:
operation: str
request_digest: str
authorization_id: str
provenance_id: str
grant_id: str | None
run_id: str
tenant_id: str
task_id: str
agent_id: str
principal_id: str
execution_path: str
authorized_at: str
def as_dict(self) -> dict[str, Any]:
return asdict(self)
class ProvenanceOperationGate:
"""部署、回滚、审计和运行操作的统一授权入口。"""
def __init__(self, writer: AuthorizationWriter) -> None:
self.writer = writer
def authorize(
self,
context: OperationContext,
*,
execution_path: str,
operation: str,
request: Mapping[str, Any],
) -> OperationReceipt:
path = ExecutionPath(execution_path).value
normalized_operation = str(operation or "").strip()
if not normalized_operation:
raise ValueError("operation is required")
digest = digest_payload(
{
"operation": normalized_operation,
"execution_path": path,
"request": dict(request),
}
)
values = {
"provenance_id": context.provenance_id,
"run_id": context.run_id,
"tenant_id": context.tenant_id,
"task_id": context.task_id,
"agent_id": context.agent_id,
"principal_id": context.principal_id,
"request_digest": digest,
}
if path == ExecutionPath.DEPLOYMENT.value:
authorization = self.writer.authorize_deployment(**values)
elif path == ExecutionPath.ROLLBACK.value:
authorization = self.writer.authorize_rollback(**values)
elif path == ExecutionPath.AUDIT.value:
authorization = self.writer.authorize_audit(**values)
elif path == ExecutionPath.CODE_CHANGE.value:
authorization = self.writer.authorize_code_change(**values)
elif path == ExecutionPath.EVIDENCE_WRITE.value:
authorization = self.writer.authorize_evidence_write(**values)
elif path == ExecutionPath.RUNTIME_EXECUTE.value:
authorization = self.writer.authorize_runtime_execute(**values)
else:
raise ProvenanceDenied(
"generic operation gate does not authorize git_operation; use ProvenanceGitWriter"
)
return self._receipt(normalized_operation, digest, authorization)
@staticmethod
def _receipt(
operation: str,
request_digest: str,
authorization: ProvenanceAuthorization | GrantAuthorization | Any,
) -> OperationReceipt:
return OperationReceipt(
operation=operation,
request_digest=request_digest,
authorization_id=authorization.authorization_id,
provenance_id=authorization.provenance_id,
grant_id=getattr(authorization, "grant_id", None),
run_id=authorization.run_id,
tenant_id=authorization.tenant_id,
task_id=authorization.task_id,
agent_id=authorization.agent_id,
principal_id=authorization.principal_id,
execution_path=authorization.execution_path,
authorized_at=authorization.authorized_at,
)
_GRANT_ENV_BY_PATH: Mapping[str, str] = {
ExecutionPath.CODE_CHANGE.value: "AIOS_PROVENANCE_GRANT_CODE_CHANGE_ID",
ExecutionPath.GIT_OPERATION.value: "AIOS_PROVENANCE_GRANT_GIT_ID",
ExecutionPath.DEPLOYMENT.value: "AIOS_PROVENANCE_GRANT_DEPLOYMENT_ID",
ExecutionPath.RUNTIME_EXECUTE.value: "AIOS_PROVENANCE_GRANT_RUNTIME_ID",
ExecutionPath.EVIDENCE_WRITE.value: "AIOS_PROVENANCE_GRANT_EVIDENCE_ID",
ExecutionPath.ROLLBACK.value: "AIOS_PROVENANCE_GRANT_ROLLBACK_ID",
ExecutionPath.AUDIT.value: "AIOS_PROVENANCE_GRANT_AUDIT_ID",
}
def grant_ids_from_environment(
*,
required_paths: Sequence[str] | None = None,
) -> dict[str, str]:
paths = tuple(
sorted(
{
ExecutionPath(path).value
for path in (required_paths or tuple(item.value for item in ExecutionPath))
}
)
)
values = {path: os.getenv(_GRANT_ENV_BY_PATH[path], "").strip() for path in paths}
missing = [_GRANT_ENV_BY_PATH[path] for path, value in values.items() if not value]
if missing:
raise RuntimeError(f"missing scoped Provenance Grant configuration: {missing}")
return values
def build_provenance_writer_from_env(
*,
state_dir: str | Path | None = None,
required: bool | None = None,
required_paths: Sequence[str] | None = None,
writer_id: str = "aios-v2-application-grant-writer",
) -> AuthorizationWriter | None:
"""从环境构建服务级最小权限 Writer。
生产应用默认只加载 Runtime、Evidence、Audit Grant。Git、部署和回滚进程
必须显式传入自己的 ``required_paths``,不能继承应用默认范围。旧全权限 Writer
只允许在显式关闭强制模式的候选/测试环境中使用。
"""
require_value = (
bool(required)
if required is not None
else os.getenv("AIOS_REQUIRE_RUN_PROVENANCE", "0").strip().lower()
in {"1", "true", "yes", "on"}
)
public_key_file = os.getenv("AIOS_PROVENANCE_PUBLIC_KEY_FILE", "").strip()
key_id = os.getenv("AIOS_PROVENANCE_KEY_ID", "").strip()
base = Path(state_dir or os.getenv("AIOS_V2_STATE_DIR", "/var/lib/aios-v2")).resolve()
store_file = os.getenv(
"AIOS_PROVENANCE_STORE_FILE", str(base / "run-provenance.sqlite3")
).strip()
receipt_root = os.getenv(
"AIOS_PROVENANCE_RECEIPT_DIR", str(base / "provenance-receipts")
).strip()
configured = bool(public_key_file and key_id)
if not configured:
if require_value:
raise RuntimeError(
"Run Provenance is required but AIOS_PROVENANCE_PUBLIC_KEY_FILE or AIOS_PROVENANCE_KEY_ID is missing"
)
return None
verifier = Ed25519PublicKeyVerifier.from_file(public_key_file, key_id=key_id)
parent_store = SQLiteProvenanceStore(store_file)
normalized_paths = tuple(
sorted(
{
ExecutionPath(path).value
for path in (
required_paths
if required_paths is not None
else APPLICATION_PROVENANCE_SCOPE
)
}
)
)
if require_value:
if not normalized_paths:
raise RuntimeError("production Run Provenance requires a non-empty service scope")
grant_ids = grant_ids_from_environment(required_paths=normalized_paths)
grant_store = SQLiteGrantStore(store_file)
trusted_grants = GrantVerifier(parent_store, grant_store, {key_id: verifier})
return ScopedLeastPrivilegeProvenanceWriter(
trusted_grants,
receipt_root,
grant_ids,
required_paths=normalized_paths,
writer_id=writer_id,
)
legacy = ProvenanceVerifier(parent_store, {key_id: verifier})
return ProvenanceControlledWriter(legacy, receipt_root)
def context_from_environment() -> OperationContext:
values = {
"provenance_id": os.getenv("AIOS_PROVENANCE_ID", ""),
"run_id": os.getenv("AIOS_RUN_ID", ""),
"tenant_id": os.getenv("AIOS_TENANT_ID", ""),
"task_id": os.getenv("AIOS_TASK_ID", ""),
"agent_id": os.getenv("AIOS_AGENT_ID", ""),
"principal_id": os.getenv("AIOS_PRINCIPAL_ID", ""),
}
missing = [name for name, value in values.items() if not str(value).strip()]
if missing:
raise RuntimeError(f"missing Provenance context environment variables: {missing}")
return OperationContext(**{name: str(value).strip() for name, value in values.items()})
def load_request_file(path: str | Path) -> Mapping[str, Any]:
value = json.loads(Path(path).read_text(encoding="utf-8"))
if not isinstance(value, Mapping):
raise ValueError("request file must contain a JSON object")
return dict(value)
@@ -1,131 +0,0 @@
"""按服务范围加载的最小权限 Provenance Writer。
该 Writer 只持有调用进程需要的 Grant ID。未配置路径的方法即使被调用也会
默认拒绝,避免应用进程获得 Git、部署或回滚权限。
"""
from __future__ import annotations
import json
import os
import threading
from pathlib import Path
from typing import Any, Mapping, Sequence
from .provenance import ExecutionPath, ProvenanceDenied, required
from .provenance_grants import GrantAuthorization, GrantVerifier
class ScopedLeastPrivilegeProvenanceWriter:
"""只暴露指定 ExecutionPath 的最小权限 Writer。"""
def __init__(
self,
verifier: GrantVerifier,
receipt_root: str | Path,
grant_ids: Mapping[str, str],
*,
required_paths: Sequence[str],
writer_id: str = "aios-v2-scoped-grant-writer",
) -> None:
self.verifier = verifier
self.receipt_root = Path(receipt_root).resolve()
self.receipt_root.mkdir(parents=True, exist_ok=True)
normalized_required = tuple(
sorted({ExecutionPath(path).value for path in required_paths})
)
if not normalized_required:
raise ValueError("required_paths must not be empty")
self.required_paths = frozenset(normalized_required)
self.grant_ids = {
path: required(grant_ids.get(path, ""), f"grant_id[{path}]")
for path in normalized_required
}
unexpected = sorted(set(grant_ids) - self.required_paths)
if unexpected:
raise ValueError(
f"scoped writer received unnecessary Grant mappings: {unexpected}"
)
self.writer_id = required(writer_id, "writer_id")
self._lock = threading.RLock()
@staticmethod
def _component(value: str) -> str:
normalized = required(value, "path component")
if normalized in {".", ".."} or "/" in normalized or "\\" in normalized:
raise ProvenanceDenied("invalid receipt path component")
return normalized
def _grant_for(self, path: str) -> str:
normalized = ExecutionPath(path).value
grant_id = self.grant_ids.get(normalized)
if grant_id is None:
raise ProvenanceDenied(
f"this process is not allowed to authorize {normalized}"
)
return grant_id
def authorize(self, **values: Any) -> GrantAuthorization:
path = ExecutionPath(values["execution_path"]).value
grant_id = self._grant_for(path)
parent_provenance_id = values.pop("provenance_id")
authorization = self.verifier.authorize(
grant_id=grant_id,
parent_provenance_id=parent_provenance_id,
**values,
)
tenant = self._component(authorization.tenant_id)
parent = self._component(authorization.parent_provenance_id)
grant = self._component(authorization.grant_id)
target_dir = self.receipt_root / tenant / parent / grant
target = target_dir / f"{authorization.authorization_id}.json"
payload = authorization.as_dict()
payload["writer_id"] = self.writer_id
payload["scope"] = sorted(self.required_paths)
with self._lock:
target_dir.mkdir(parents=True, exist_ok=True)
temporary = target.with_suffix(f".tmp-{os.getpid()}")
temporary.write_text(
json.dumps(payload, ensure_ascii=False, sort_keys=True, indent=2)
+ "\n",
encoding="utf-8",
)
os.chmod(temporary, 0o640)
os.replace(temporary, target)
return authorization
def authorize_code_change(self, **values: Any) -> GrantAuthorization:
return self.authorize(
execution_path=ExecutionPath.CODE_CHANGE.value, **values
)
def authorize_git_operation(
self, *, git_operation: str, **values: Any
) -> GrantAuthorization:
return self.authorize(
execution_path=ExecutionPath.GIT_OPERATION.value,
git_operation=git_operation,
**values,
)
def authorize_deployment(self, **values: Any) -> GrantAuthorization:
return self.authorize(
execution_path=ExecutionPath.DEPLOYMENT.value, **values
)
def authorize_runtime_execute(self, **values: Any) -> GrantAuthorization:
return self.authorize(
execution_path=ExecutionPath.RUNTIME_EXECUTE.value, **values
)
def authorize_evidence_write(self, **values: Any) -> GrantAuthorization:
return self.authorize(
execution_path=ExecutionPath.EVIDENCE_WRITE.value, **values
)
def authorize_rollback(self, **values: Any) -> GrantAuthorization:
return self.authorize(
execution_path=ExecutionPath.ROLLBACK.value, **values
)
def authorize_audit(self, **values: Any) -> GrantAuthorization:
return self.authorize(execution_path=ExecutionPath.AUDIT.value, **values)
@@ -1,607 +0,0 @@
"""Durable execution truth registry for externally executed AIOS tasks.
A task manifest being present in Git is not evidence that an executor discovered
or ran it. This registry separates declaration, registration, claim, start,
heartbeat and terminal evidence. Release gates must require a terminal success
record with immutable evidence hashes.
"""
from __future__ import annotations
import json
import sqlite3
import threading
import uuid
from dataclasses import asdict, dataclass, field, replace
from datetime import datetime, timedelta, timezone
from enum import Enum
from pathlib import Path
from typing import Any, Mapping, Sequence
def utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def parse_time(value: str) -> datetime:
normalized = str(value or "").strip()
if normalized.endswith("Z"):
normalized = normalized[:-1] + "+00:00"
parsed = datetime.fromisoformat(normalized)
if parsed.tzinfo is None:
raise ValueError("timestamp requires timezone")
return parsed.astimezone(timezone.utc)
def required(value: str, label: str) -> str:
normalized = str(value or "").strip()
if not normalized:
raise ValueError(f"{label} is required")
return normalized
class ExecutionTruthState(str, Enum):
DECLARED = "DECLARED"
REGISTERED = "REGISTERED"
CLAIMED = "CLAIMED"
RUNNING = "RUNNING"
SUCCEEDED = "SUCCEEDED"
FAILED = "FAILED"
CANCELLED = "CANCELLED"
NOT_STARTED = "NOT_STARTED"
STALLED = "STALLED"
TERMINAL_STATES = frozenset(
{
ExecutionTruthState.SUCCEEDED,
ExecutionTruthState.FAILED,
ExecutionTruthState.CANCELLED,
ExecutionTruthState.NOT_STARTED,
ExecutionTruthState.STALLED,
}
)
@dataclass(frozen=True)
class ExecutionTaskManifest:
task_id: str
executor: str
source_branch: str
source_oid: str
target_version: str
steps: Sequence[str]
start_deadline_seconds: int = 300
heartbeat_timeout_seconds: int = 180
required_evidence: Sequence[str] = ()
notification_events: Sequence[str] = ("STARTED", "SUCCEEDED", "FAILED")
owner_notification_ref: str = "AIOS_OWNER_WECHAT_TARGET_REF"
manifest_version: str = "aios.external-task.v1"
def __post_init__(self) -> None:
for name in (
"task_id",
"executor",
"source_branch",
"source_oid",
"target_version",
"manifest_version",
"owner_notification_ref",
):
object.__setattr__(self, name, required(getattr(self, name), name))
steps = tuple(required(item, "step") for item in self.steps)
if not steps:
raise ValueError("at least one execution step is required")
if self.start_deadline_seconds < 1 or self.heartbeat_timeout_seconds < 1:
raise ValueError("execution deadlines must be positive")
object.__setattr__(self, "steps", steps)
object.__setattr__(
self,
"required_evidence",
tuple(sorted({required(item, "required_evidence") for item in self.required_evidence})),
)
object.__setattr__(
self,
"notification_events",
tuple(sorted({required(item, "notification_event") for item in self.notification_events})),
)
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
value["steps"] = list(self.steps)
value["required_evidence"] = list(self.required_evidence)
value["notification_events"] = list(self.notification_events)
return value
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "ExecutionTaskManifest":
return cls(**dict(value))
@dataclass(frozen=True)
class ExecutionTruthRecord:
task_id: str
manifest_json: str
state: ExecutionTruthState
executor_run_id: str | None = None
executor_id: str | None = None
declared_at: str = field(default_factory=utc_now)
registered_at: str | None = None
claimed_at: str | None = None
started_at: str | None = None
heartbeat_at: str | None = None
completed_at: str | None = None
current_step: int = 0
evidence: Mapping[str, str] = field(default_factory=dict)
error: Mapping[str, Any] = field(default_factory=dict)
revision: int = 1
@property
def manifest(self) -> ExecutionTaskManifest:
return ExecutionTaskManifest.from_dict(json.loads(self.manifest_json))
def as_dict(self) -> dict[str, Any]:
return {
**asdict(self),
"state": self.state.value,
"manifest": self.manifest.as_dict(),
"evidence": dict(self.evidence),
"error": dict(self.error),
}
class ExecutionTruthConflict(RuntimeError):
pass
class SQLiteExecutionTruthRegistry:
"""Persistent registration and run-evidence store."""
def __init__(self, path: str | Path) -> None:
self.path = Path(path).resolve()
self.path.parent.mkdir(parents=True, exist_ok=True)
self._lock = threading.RLock()
with self._connect() as connection:
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS execution_truth (
task_id TEXT PRIMARY KEY,
manifest_json TEXT NOT NULL,
state TEXT NOT NULL,
record_json TEXT NOT NULL,
revision INTEGER NOT NULL,
updated_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS execution_truth_events (
event_id TEXT PRIMARY KEY,
task_id TEXT NOT NULL,
event_type TEXT NOT NULL,
details_json TEXT NOT NULL,
created_at TEXT NOT NULL,
FOREIGN KEY(task_id) REFERENCES execution_truth(task_id)
);
CREATE INDEX IF NOT EXISTS idx_execution_truth_state
ON execution_truth(state, updated_at);
"""
)
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(str(self.path), timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA synchronous=FULL")
connection.execute("PRAGMA foreign_keys=ON")
return connection
@staticmethod
def _manifest_json(manifest: ExecutionTaskManifest) -> str:
return json.dumps(manifest.as_dict(), ensure_ascii=False, sort_keys=True, separators=(",", ":"))
@staticmethod
def _record_json(record: ExecutionTruthRecord) -> str:
value = record.as_dict()
value.pop("manifest", None)
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"), default=str)
@staticmethod
def _decode(row: sqlite3.Row) -> ExecutionTruthRecord:
value = json.loads(row["record_json"])
value["state"] = ExecutionTruthState(value["state"])
value["evidence"] = dict(value.get("evidence") or {})
value["error"] = dict(value.get("error") or {})
return ExecutionTruthRecord(**value)
def _event(
self,
connection: sqlite3.Connection,
task_id: str,
event_type: str,
details: Mapping[str, Any] | None = None,
) -> None:
connection.execute(
"INSERT INTO execution_truth_events VALUES (?, ?, ?, ?, ?)",
(
f"truth_evt_{uuid.uuid4().hex}",
task_id,
event_type,
json.dumps(dict(details or {}), ensure_ascii=False, sort_keys=True, default=str),
utc_now(),
),
)
def declare(self, manifest: ExecutionTaskManifest) -> ExecutionTruthRecord:
manifest_json = self._manifest_json(manifest)
record = ExecutionTruthRecord(
task_id=manifest.task_id,
manifest_json=manifest_json,
state=ExecutionTruthState.DECLARED,
)
with self._lock, self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
existing = connection.execute(
"SELECT record_json FROM execution_truth WHERE task_id = ?",
(manifest.task_id,),
).fetchone()
if existing is not None:
current = self._decode(existing)
if current.manifest_json != manifest_json:
previous_manifest = current.manifest.as_dict()
replacement_manifest = manifest.as_dict()
previous_oid = previous_manifest.pop("source_oid")
replacement_oid = replacement_manifest.pop("source_oid")
if (
current.state != ExecutionTruthState.NOT_STARTED
or previous_manifest != replacement_manifest
):
connection.rollback()
raise ExecutionTruthConflict(
"task_id already declared with another manifest"
)
now = utc_now()
updated = replace(
current,
manifest_json=manifest_json,
revision=current.revision + 1,
)
connection.execute(
"UPDATE execution_truth SET manifest_json = ?, record_json = ?, revision = ?, updated_at = ? WHERE task_id = ?",
(
manifest_json,
self._record_json(updated),
updated.revision,
now,
updated.task_id,
),
)
self._event(
connection,
updated.task_id,
"MANIFEST_SUPERSEDED",
{
"previous_source_oid": previous_oid,
"source_oid": replacement_oid,
"state": updated.state.value,
},
)
connection.commit()
return updated
connection.rollback()
return current
connection.execute(
"INSERT INTO execution_truth VALUES (?, ?, ?, ?, ?, ?)",
(
record.task_id,
manifest_json,
record.state.value,
self._record_json(record),
record.revision,
record.declared_at,
),
)
self._event(connection, record.task_id, "DECLARED", {"source_oid": manifest.source_oid})
connection.commit()
return record
def get(self, task_id: str) -> ExecutionTruthRecord:
with self._lock, self._connect() as connection:
row = connection.execute(
"SELECT record_json FROM execution_truth WHERE task_id = ?",
(required(task_id, "task_id"),),
).fetchone()
if row is None:
raise KeyError("execution task not declared")
return self._decode(row)
def _transition(
self,
task_id: str,
*,
allowed: Sequence[ExecutionTruthState],
target: ExecutionTruthState,
executor_run_id: str | None = None,
executor_id: str | None = None,
current_step: int | None = None,
evidence: Mapping[str, str] | None = None,
error: Mapping[str, Any] | None = None,
) -> ExecutionTruthRecord:
with self._lock, self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
row = connection.execute(
"SELECT record_json FROM execution_truth WHERE task_id = ?",
(required(task_id, "task_id"),),
).fetchone()
if row is None:
connection.rollback()
raise KeyError("execution task not declared")
current = self._decode(row)
if current.state not in set(allowed):
connection.rollback()
raise ExecutionTruthConflict(
f"cannot transition {current.state.value} to {target.value}"
)
now = utc_now()
updates: dict[str, Any] = {
"state": target,
"revision": current.revision + 1,
}
if executor_run_id is not None:
if current.executor_run_id and current.executor_run_id != executor_run_id:
connection.rollback()
raise ExecutionTruthConflict("executor_run_id mismatch")
updates["executor_run_id"] = required(executor_run_id, "executor_run_id")
if executor_id is not None:
if current.executor_id and current.executor_id != executor_id:
connection.rollback()
raise ExecutionTruthConflict("executor_id mismatch")
updates["executor_id"] = required(executor_id, "executor_id")
if target == ExecutionTruthState.REGISTERED:
updates["registered_at"] = now
if target == ExecutionTruthState.CLAIMED:
updates["claimed_at"] = now
updates["heartbeat_at"] = now
if target == ExecutionTruthState.RUNNING:
updates["started_at"] = current.started_at or now
updates["heartbeat_at"] = now
if current_step is not None:
if current_step < current.current_step or current_step > len(current.manifest.steps):
connection.rollback()
raise ExecutionTruthConflict("invalid task step progression")
updates["current_step"] = current_step
updates["heartbeat_at"] = now
if evidence is not None:
updates["evidence"] = {
**dict(current.evidence),
**{required(key, "evidence key"): required(value, "evidence hash") for key, value in evidence.items()},
}
if error is not None:
updates["error"] = json.loads(json.dumps(dict(error), default=str))
if target in TERMINAL_STATES:
updates["completed_at"] = now
updated = replace(current, **updates)
connection.execute(
"UPDATE execution_truth SET state = ?, record_json = ?, revision = ?, updated_at = ? WHERE task_id = ?",
(
updated.state.value,
self._record_json(updated),
updated.revision,
now,
updated.task_id,
),
)
self._event(
connection,
updated.task_id,
target.value,
{
"executor_run_id": updated.executor_run_id,
"executor_id": updated.executor_id,
"current_step": updated.current_step,
},
)
connection.commit()
return updated
def register(self, task_id: str, *, executor_run_id: str) -> ExecutionTruthRecord:
return self._transition(
task_id,
allowed=(ExecutionTruthState.DECLARED,),
target=ExecutionTruthState.REGISTERED,
executor_run_id=executor_run_id,
)
def retry_not_started(
self,
task_id: str,
*,
executor_run_id: str,
) -> ExecutionTruthRecord:
"""Start a new audited attempt after the declaration deadline expired."""
with self._lock, self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
row = connection.execute(
"SELECT record_json FROM execution_truth WHERE task_id = ?",
(required(task_id, "task_id"),),
).fetchone()
if row is None:
connection.rollback()
raise KeyError("execution task not declared")
current = self._decode(row)
if current.state != ExecutionTruthState.NOT_STARTED:
connection.rollback()
raise ExecutionTruthConflict(
f"cannot retry task from {current.state.value}"
)
now = utc_now()
updated = replace(
current,
state=ExecutionTruthState.REGISTERED,
executor_run_id=required(executor_run_id, "executor_run_id"),
executor_id=None,
registered_at=now,
claimed_at=None,
started_at=None,
heartbeat_at=None,
completed_at=None,
current_step=0,
evidence={},
error={},
revision=current.revision + 1,
)
connection.execute(
"UPDATE execution_truth SET state = ?, record_json = ?, revision = ?, updated_at = ? WHERE task_id = ?",
(
updated.state.value,
self._record_json(updated),
updated.revision,
now,
updated.task_id,
),
)
self._event(
connection,
updated.task_id,
"RETRIED",
{
"previous_state": current.state.value,
"previous_executor_run_id": current.executor_run_id,
"executor_run_id": updated.executor_run_id,
"previous_error": dict(current.error),
},
)
connection.commit()
return updated
def claim(self, task_id: str, *, executor_run_id: str, executor_id: str) -> ExecutionTruthRecord:
return self._transition(
task_id,
allowed=(ExecutionTruthState.REGISTERED,),
target=ExecutionTruthState.CLAIMED,
executor_run_id=executor_run_id,
executor_id=executor_id,
)
def start(self, task_id: str, *, executor_run_id: str, executor_id: str) -> ExecutionTruthRecord:
return self._transition(
task_id,
allowed=(ExecutionTruthState.CLAIMED,),
target=ExecutionTruthState.RUNNING,
executor_run_id=executor_run_id,
executor_id=executor_id,
current_step=1,
)
def heartbeat(
self,
task_id: str,
*,
executor_run_id: str,
executor_id: str,
current_step: int,
) -> ExecutionTruthRecord:
return self._transition(
task_id,
allowed=(ExecutionTruthState.RUNNING,),
target=ExecutionTruthState.RUNNING,
executor_run_id=executor_run_id,
executor_id=executor_id,
current_step=current_step,
)
def succeed(
self,
task_id: str,
*,
executor_run_id: str,
executor_id: str,
evidence: Mapping[str, str],
) -> ExecutionTruthRecord:
current = self.get(task_id)
missing = sorted(set(current.manifest.required_evidence) - set(evidence))
if missing:
raise ExecutionTruthConflict(f"missing required evidence: {missing}")
if current.current_step != len(current.manifest.steps):
raise ExecutionTruthConflict("all declared steps must complete before success")
return self._transition(
task_id,
allowed=(ExecutionTruthState.RUNNING,),
target=ExecutionTruthState.SUCCEEDED,
executor_run_id=executor_run_id,
executor_id=executor_id,
evidence=evidence,
)
def fail(
self,
task_id: str,
*,
executor_run_id: str,
executor_id: str,
error: Mapping[str, Any],
evidence: Mapping[str, str] | None = None,
) -> ExecutionTruthRecord:
return self._transition(
task_id,
allowed=(ExecutionTruthState.REGISTERED, ExecutionTruthState.CLAIMED, ExecutionTruthState.RUNNING),
target=ExecutionTruthState.FAILED,
executor_run_id=executor_run_id,
executor_id=executor_id,
evidence=evidence,
error=error,
)
def evaluate_deadlines(self, task_id: str, *, now: datetime | None = None) -> ExecutionTruthRecord:
current = self.get(task_id)
current_time = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
manifest = current.manifest
if current.state in TERMINAL_STATES:
return current
declared = parse_time(current.declared_at)
if current.state in {ExecutionTruthState.DECLARED, ExecutionTruthState.REGISTERED}:
if current_time >= declared + timedelta(seconds=manifest.start_deadline_seconds):
return self._transition(
task_id,
allowed=(current.state,),
target=ExecutionTruthState.NOT_STARTED,
error={"code": "start_deadline_exceeded"},
)
if current.state in {ExecutionTruthState.CLAIMED, ExecutionTruthState.RUNNING}:
heartbeat = parse_time(current.heartbeat_at or current.claimed_at or current.declared_at)
if current_time >= heartbeat + timedelta(seconds=manifest.heartbeat_timeout_seconds):
return self._transition(
task_id,
allowed=(current.state,),
target=ExecutionTruthState.STALLED,
error={"code": "heartbeat_timeout"},
)
return current
def gate(self, task_id: str) -> Mapping[str, Any]:
record = self.evaluate_deadlines(task_id)
passed = record.state == ExecutionTruthState.SUCCEEDED
return {
"task_id": record.task_id,
"state": record.state.value,
"passed": passed,
"release_blocking": not passed,
"executor_run_id": record.executor_run_id,
"executor_id": record.executor_id,
"current_step": record.current_step,
"total_steps": len(record.manifest.steps),
"evidence": dict(record.evidence),
"error": dict(record.error),
}
def events(self, task_id: str) -> list[Mapping[str, Any]]:
with self._lock, self._connect() as connection:
rows = connection.execute(
"SELECT event_id, event_type, details_json, created_at FROM execution_truth_events WHERE task_id = ? ORDER BY created_at, event_id",
(required(task_id, "task_id"),),
).fetchall()
return [
{
"event_id": row["event_id"],
"event_type": row["event_type"],
"details": json.loads(row["details_json"]),
"created_at": row["created_at"],
}
for row in rows
]
-13
View File
@@ -1,13 +0,0 @@
"""AIOS unified, role-aware web portal."""
from .application import UnifiedPortalApplication, build_application
from .capabilities import ROLE_CAPABILITIES, allows, require, resolve
__all__ = [
"ROLE_CAPABILITIES",
"UnifiedPortalApplication",
"allows",
"build_application",
"require",
"resolve",
]
-512
View File
@@ -1,512 +0,0 @@
"""Unified, role-aware AIOS web application.
The portal wraps the production CandidateApplication instead of replacing its
runtime/control-plane routes. Portal endpoints aggregate existing authoritative
stores and require server-side capabilities before delegating any state change.
"""
from __future__ import annotations
import dataclasses
import json
import os
import re
import sqlite3
import urllib.parse
from enum import Enum
from http import HTTPStatus
from pathlib import Path
from typing import Any, Mapping
from aios_v2.app import (
CandidateApplication,
CandidateHTTPServer,
build_application as build_candidate_application,
)
from aios_v2.control_plane import RequestContext, TaskState
from aios_v2.gitops_status import collect_gitops_status
from aios_v2.provenance import SQLiteProvenanceStore
from aios_v2.provenance_grants import SQLiteGrantStore
from .capabilities import allows, require, resolve
def _json_value(value: Any) -> Any:
if dataclasses.is_dataclass(value):
return {
field.name: _json_value(getattr(value, field.name))
for field in dataclasses.fields(value)
}
if isinstance(value, Enum):
return value.value
if isinstance(value, Mapping):
return {str(key): _json_value(item) for key, item in value.items()}
if isinstance(value, (list, tuple, set, frozenset)):
return [_json_value(item) for item in value]
return value
def _mapping(value: Any) -> Mapping[str, Any]:
if not isinstance(value, Mapping):
raise ValueError("JSON body must be an object")
return value
def _required(value: Any, label: str) -> str:
normalized = str(value or "").strip()
if not normalized:
raise ValueError(f"{label} is required")
return normalized
def _safe_count(items: list[Any]) -> int:
return len(items)
class AuthorizationCatalog:
"""Tenant-filtered read/revoke access to existing Provenance stores."""
def __init__(self, store_file: str | Path) -> None:
self.store_file = Path(store_file).resolve()
def _read_connection(self) -> sqlite3.Connection | None:
if not self.store_file.exists():
return None
connection = sqlite3.connect(
f"file:{self.store_file}?mode=ro",
uri=True,
timeout=10.0,
)
connection.row_factory = sqlite3.Row
return connection
def snapshot(self, *, tenant_id: str) -> Mapping[str, Any]:
connection = self._read_connection()
if connection is None:
return {
"configured": False,
"workflows": [],
"grants": [],
"summary": {"workflows": 0, "grants": 0, "active_grants": 0},
}
try:
workflows: list[dict[str, Any]] = []
grants: list[dict[str, Any]] = []
tables = {
row["name"]
for row in connection.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
).fetchall()
}
if "provenance_records" in tables:
rows = connection.execute(
"""
SELECT r.payload_json, r.status, r.signature_verified,
r.created_at, r.updated_at,
COUNT(u.authorization_id) AS usage_count
FROM provenance_records r
LEFT JOIN provenance_usage u
ON u.provenance_id = r.provenance_id
GROUP BY r.provenance_id
ORDER BY r.updated_at DESC, r.provenance_id
"""
).fetchall()
for row in rows:
payload = json.loads(row["payload_json"])
if payload.get("tenant_id") != tenant_id:
continue
workflows.append(
{
**payload,
"status": row["status"],
"signature_verified": bool(row["signature_verified"]),
"usage_count": int(row["usage_count"]),
"created_at": row["created_at"],
"updated_at": row["updated_at"],
}
)
if "provenance_grants" in tables:
rows = connection.execute(
"""
SELECT g.payload_json, g.status, g.signature_verified,
g.created_at, g.updated_at,
COUNT(u.authorization_id) AS usage_count
FROM provenance_grants g
LEFT JOIN provenance_grant_usage u
ON u.grant_id = g.grant_id
GROUP BY g.grant_id
ORDER BY g.updated_at DESC, g.grant_id
"""
).fetchall()
for row in rows:
payload = json.loads(row["payload_json"])
if payload.get("tenant_id") != tenant_id:
continue
grants.append(
{
**payload,
"status": row["status"],
"signature_verified": bool(row["signature_verified"]),
"usage_count": int(row["usage_count"]),
"created_at": row["created_at"],
"updated_at": row["updated_at"],
}
)
active = sum(
1
for item in grants
if item.get("status") not in {"revoked", "expired"}
)
return {
"configured": True,
"workflows": workflows,
"grants": grants,
"summary": {
"workflows": len(workflows),
"grants": len(grants),
"active_grants": active,
"revoked_grants": sum(
1 for item in grants if item.get("status") == "revoked"
),
},
}
finally:
connection.close()
def revoke_workflow(
self,
*,
provenance_id: str,
tenant_id: str,
principal_id: str,
reason: str,
) -> Mapping[str, Any]:
store = SQLiteProvenanceStore(self.store_file)
try:
record = store.get(provenance_id)
if record.tenant_id != tenant_id:
raise KeyError("workflow provenance not found")
store.revoke(
record.provenance_id,
reason=reason,
principal_id=principal_id,
)
return {
"provenance_id": record.provenance_id,
"status": store.status(record.provenance_id).value,
}
finally:
store.close()
def revoke_grant(
self,
*,
grant_id: str,
tenant_id: str,
principal_id: str,
reason: str,
) -> Mapping[str, Any]:
store = SQLiteGrantStore(self.store_file)
try:
grant = store.get(grant_id)
if grant.tenant_id != tenant_id:
raise KeyError("provenance grant not found")
store.revoke(
grant.grant_id,
reason=reason,
principal_id=principal_id,
)
return {
"grant_id": grant.grant_id,
"status": store.status(grant.grant_id).value,
}
finally:
store.close()
class UnifiedPortalApplication:
"""Role-aware portal facade over the existing production application."""
def __init__(
self,
base: CandidateApplication,
*,
provenance_store_file: str | Path,
) -> None:
self.base = base
self.gateway = base.gateway
self.orchestrator = base.orchestrator
self.tracker = base.tracker
self.hermes = base.hermes
self.openclaw = base.openclaw
self.authorization_catalog = AuthorizationCatalog(provenance_store_file)
def context(self, headers: Mapping[str, str]) -> RequestContext:
return self.base.context(headers)
def _runtime(self, headers: Mapping[str, str]) -> Mapping[str, Any]:
status, payload = self.base.handle_json("GET", "/api/v2/runtime", headers, {})
if int(status) != HTTPStatus.OK:
raise RuntimeError("runtime status is unavailable")
return payload
def _gitops(self, context: RequestContext) -> Mapping[str, Any]:
return {
"tenant_id": context.tenant_id,
"gitops": collect_gitops_status().as_dict(),
}
def _tasks_view(self, context: RequestContext) -> list[Mapping[str, Any]]:
service = self.orchestrator.control_plane
tasks = service.list_tasks(context)
dispatches = {
item.task_id: item
for item in self.orchestrator.dispatch_repository.list(
tenant_id=context.tenant_id
)
}
evidence_by_task: dict[str, list[Any]] = {}
for evidence in self.tracker.list(tenant_id=context.tenant_id):
evidence_by_task.setdefault(evidence.task_id, []).append(evidence)
audit_by_task: dict[str, list[Any]] = {}
for event in service.list_audit_events(context):
if event.resource_type == "task":
audit_by_task.setdefault(event.resource_id, []).append(event)
capabilities = set(resolve(context).capabilities)
result: list[Mapping[str, Any]] = []
for task in tasks:
actions: list[str] = []
if task.state == TaskState.CREATED and "task.approve" in capabilities:
actions.append("approve")
if task.state in {TaskState.CREATED, TaskState.APPROVED, TaskState.RUNNING} and "task.cancel" in capabilities:
actions.append("cancel")
if task.state == TaskState.APPROVED and "task.dispatch" in capabilities:
actions.append("dispatch")
result.append(
{
"task": _json_value(task),
"dispatch": _json_value(dispatches.get(task.task_id)),
"evidence": _json_value(evidence_by_task.get(task.task_id, [])),
"audit": _json_value(audit_by_task.get(task.task_id, [])[-12:]),
"available_actions": actions,
}
)
return result
def _overview(
self,
context: RequestContext,
headers: Mapping[str, str],
) -> Mapping[str, Any]:
service = self.orchestrator.control_plane
capabilities = resolve(context)
sections: dict[str, Any] = {}
tasks = service.list_tasks(context) if allows(context, "task.read") else []
dispatches = (
self.orchestrator.dispatch_repository.list(tenant_id=context.tenant_id)
if allows(context, "dispatch.read")
else []
)
evidence = (
self.tracker.list(tenant_id=context.tenant_id)
if allows(context, "evidence.read")
else []
)
audit = (
service.list_audit_events(context)
if allows(context, "audit.read")
else []
)
if allows(context, "runtime.read"):
sections["runtime"] = self._runtime(headers)
if allows(context, "release.read"):
sections["release"] = {
**self._gitops(context),
"version": os.getenv("AIOS_RELEASE_VERSION", "V2R1C1"),
"release_oid": os.getenv("AIOS_RELEASE_OID", "unknown"),
"baseline": os.getenv("AIOS_BASELINE_VERSION", "V2R1C1"),
"status": os.getenv("AIOS_RELEASE_STATUS", "PRODUCTION_DEPLOYED"),
}
if allows(context, "provenance.read"):
sections["authorization"] = self.authorization_catalog.snapshot(
tenant_id=context.tenant_id
)["summary"]
task_states: dict[str, int] = {}
for task in tasks:
task_states[task.state.value] = task_states.get(task.state.value, 0) + 1
return {
"identity": capabilities.as_dict(),
"summary": {
"tasks": _safe_count(tasks),
"task_states": task_states,
"dispatches": _safe_count(dispatches),
"active_dispatches": sum(
1
for item in dispatches
if item.state.value not in {"COMPLETED", "FAILED", "CANCELLED"}
),
"evidence": _safe_count(evidence),
"audit_events": _safe_count(audit),
},
"sections": sections,
"recent": {
"tasks": _json_value(tasks[:8]),
"dispatches": _json_value(dispatches[-8:]),
"audit": _json_value(audit[-12:]),
},
}
def handle_json(
self,
method: str,
raw_path: str,
headers: Mapping[str, str],
body: Any,
) -> tuple[int, Mapping[str, Any]]:
parsed = urllib.parse.urlparse(raw_path)
path = parsed.path.rstrip("/") or "/"
method = method.upper()
if not path.startswith("/api/v2/portal") and path != "/api/v2/me/capabilities":
return self.base.handle_json(method, raw_path, headers, body)
context = self.context(headers)
payload = _mapping(body or {})
if method == "GET" and path == "/api/v2/me/capabilities":
require(context, "portal.access")
return HTTPStatus.OK, resolve(context).as_dict()
if method == "GET" and path == "/api/v2/portal/overview":
require(context, "dashboard.read")
return HTTPStatus.OK, self._overview(context, headers)
if method == "GET" and path == "/api/v2/portal/tasks":
require(context, "task.read")
return HTTPStatus.OK, {"items": self._tasks_view(context)}
if method == "GET" and path == "/api/v2/portal/authorizations":
require(context, "provenance.read")
return HTTPStatus.OK, self.authorization_catalog.snapshot(
tenant_id=context.tenant_id
)
if method == "GET" and path == "/api/v2/portal/runtime":
require(context, "runtime.read")
return HTTPStatus.OK, {
**self._runtime(headers),
"dispatches": _json_value(
self.orchestrator.dispatch_repository.list(
tenant_id=context.tenant_id
)
),
"hermes_jobs": _json_value(
self.hermes.list(tenant_id=context.tenant_id)
),
}
if method == "GET" and path == "/api/v2/portal/evidence":
require(context, "evidence.read")
return HTTPStatus.OK, {
"executions": _json_value(
self.tracker.list(tenant_id=context.tenant_id)
)
}
if method == "GET" and path == "/api/v2/portal/audit":
require(context, "audit.read")
return HTTPStatus.OK, {
"events": _json_value(
self.orchestrator.control_plane.list_audit_events(context)
)
}
if method == "GET" and path == "/api/v2/portal/release":
require(context, "release.read")
return HTTPStatus.OK, {
**self._gitops(context),
"version": os.getenv("AIOS_RELEASE_VERSION", "V2R1C1"),
"release_oid": os.getenv("AIOS_RELEASE_OID", "unknown"),
"baseline": os.getenv("AIOS_BASELINE_VERSION", "V2R1C1"),
"status": os.getenv("AIOS_RELEASE_STATUS", "PRODUCTION_DEPLOYED"),
"operations": (
["review_release", "request_tag", "request_rollback"]
if allows(context, "release.operate")
else []
),
"note": "Release operations remain GitOps approval requests; the portal does not mutate tags or production directly.",
}
if method == "POST" and path == "/api/v2/portal/tasks":
require(context, "task.create")
return self.base.handle_json("POST", "/api/v2/tasks", headers, payload)
task_action = re.fullmatch(
r"/api/v2/portal/tasks/([^/]+)/(approve|cancel|dispatch)", path
)
if method == "POST" and task_action:
action = task_action.group(2)
require(context, f"task.{action}")
task_id = urllib.parse.unquote(task_action.group(1))
target = (
f"/api/v2/tasks/{task_id}/dispatch"
if action == "dispatch"
else f"/api/v2/tasks/{task_id}/{action}"
)
return self.base.handle_json("POST", target, headers, payload)
workflow_revoke = re.fullmatch(
r"/api/v2/portal/provenance/([^/]+)/revoke", path
)
if method == "POST" and workflow_revoke:
require(context, "provenance.revoke")
reason = _required(payload.get("reason"), "reason")
result = self.authorization_catalog.revoke_workflow(
provenance_id=urllib.parse.unquote(workflow_revoke.group(1)),
tenant_id=context.tenant_id,
principal_id=context.principal_id,
reason=reason,
)
return HTTPStatus.OK, {"workflow": result}
grant_revoke = re.fullmatch(r"/api/v2/portal/grants/([^/]+)/revoke", path)
if method == "POST" and grant_revoke:
require(context, "provenance.revoke")
reason = _required(payload.get("reason"), "reason")
result = self.authorization_catalog.revoke_grant(
grant_id=urllib.parse.unquote(grant_revoke.group(1)),
tenant_id=context.tenant_id,
principal_id=context.principal_id,
reason=reason,
)
return HTTPStatus.OK, {"grant": result}
return HTTPStatus.NOT_FOUND, {
"error": "not_found",
"message": "portal route not found",
}
def build_application() -> UnifiedPortalApplication:
state_dir = Path(os.getenv("AIOS_V2_STATE_DIR", "/var/lib/aios-v2")).resolve()
provenance_file = Path(
os.getenv(
"AIOS_PROVENANCE_STORE_FILE",
str(state_dir / "run-provenance.sqlite3"),
)
)
return UnifiedPortalApplication(
build_candidate_application(),
provenance_store_file=provenance_file,
)
def main() -> None:
host = os.getenv("AIOS_V2_HOST", "127.0.0.1")
port = int(os.getenv("AIOS_V2_PORT", "18920"))
server = CandidateHTTPServer((host, port), build_application())
print(f"AIOS V2 unified portal listening on {host}:{port}")
server.serve_forever()
if __name__ == "__main__":
main()
-111
View File
@@ -1,111 +0,0 @@
"""Server-side role and capability resolution for the unified AIOS portal."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Mapping, Sequence
from aios_v2.control_plane import RequestContext, Unauthorized
ROLE_CAPABILITIES: Mapping[str, frozenset[str]] = {
"platform_admin": frozenset({
"portal.access", "dashboard.read", "tenant.read", "tenant.manage",
"agent.read", "agent.manage", "task.read", "task.create", "task.approve",
"task.cancel", "task.dispatch", "policy.read", "audit.read", "evidence.read",
"runtime.read", "runtime.operate", "dispatch.read", "provenance.read",
"provenance.revoke", "external_task.read", "gitops.read", "release.read",
"release.operate", "worktree.read", "system.admin",
}),
"tenant_admin": frozenset({
"portal.access", "dashboard.read", "tenant.read", "agent.read", "agent.manage",
"task.read", "task.create", "task.approve", "task.cancel", "task.dispatch",
"policy.read", "audit.read", "evidence.read", "runtime.read", "dispatch.read",
"provenance.read", "provenance.revoke", "external_task.read", "gitops.read",
"release.read",
}),
"operator": frozenset({
"portal.access", "dashboard.read", "tenant.read", "agent.read", "task.read",
"task.create", "task.cancel", "task.dispatch", "runtime.read", "dispatch.read",
"evidence.read", "provenance.read", "external_task.read", "gitops.read",
"release.read",
}),
"approver": frozenset({
"portal.access", "dashboard.read", "tenant.read", "task.read", "task.approve",
"task.cancel", "policy.read", "audit.read", "provenance.read", "evidence.read",
"external_task.read", "release.read",
}),
"auditor": frozenset({
"portal.access", "dashboard.read", "tenant.read", "agent.read", "task.read",
"audit.read", "evidence.read", "runtime.read", "dispatch.read",
"provenance.read", "external_task.read", "gitops.read", "release.read",
"worktree.read",
}),
"runtime": frozenset({
"portal.access", "dashboard.read", "task.read", "runtime.read", "runtime.operate",
"dispatch.read", "evidence.read", "provenance.read", "external_task.read",
}),
"release_manager": frozenset({
"portal.access", "dashboard.read", "audit.read", "runtime.read", "dispatch.read",
"provenance.read", "external_task.read", "gitops.read", "release.read",
"release.operate", "worktree.read",
}),
"viewer": frozenset({
"portal.access", "dashboard.read", "task.read", "runtime.read",
"external_task.read",
}),
}
NAVIGATION: Sequence[Mapping[str, str]] = (
{"id": "overview", "label": "总览", "capability": "dashboard.read"},
{"id": "resources", "label": "租户与 Agent", "capability": "tenant.read"},
{"id": "tasks", "label": "任务中心", "capability": "task.read"},
{"id": "execution", "label": "执行中心", "capability": "external_task.read"},
{"id": "authorization", "label": "授权中心", "capability": "provenance.read"},
{"id": "runtime", "label": "运行与运维", "capability": "runtime.read"},
{"id": "evidence", "label": "执行证据", "capability": "evidence.read"},
{"id": "audit", "label": "审计中心", "capability": "audit.read"},
{"id": "release", "label": "GitOps 与发布", "capability": "release.read"},
)
@dataclass(frozen=True)
class ResolvedCapabilities:
tenant_id: str
principal_id: str
roles: tuple[str, ...]
capabilities: tuple[str, ...]
navigation: tuple[Mapping[str, str], ...]
def as_dict(self) -> dict[str, object]:
return {
"tenant_id": self.tenant_id,
"principal_id": self.principal_id,
"roles": list(self.roles),
"capabilities": list(self.capabilities),
"navigation": [dict(item) for item in self.navigation],
}
def resolve(context: RequestContext) -> ResolvedCapabilities:
capabilities: set[str] = set()
for role in context.roles:
capabilities.update(ROLE_CAPABILITIES.get(role, ()))
ordered = tuple(sorted(capabilities))
navigation = tuple(item for item in NAVIGATION if item["capability"] in capabilities)
return ResolvedCapabilities(
tenant_id=context.tenant_id,
principal_id=context.principal_id,
roles=tuple(sorted(context.roles)),
capabilities=ordered,
navigation=navigation,
)
def require(context: RequestContext, capability: str) -> None:
if capability not in resolve(context).capabilities:
raise Unauthorized(f"portal capability is required: {capability}")
def allows(context: RequestContext, capability: str) -> bool:
return capability in resolve(context).capabilities
-48
View File
@@ -1,48 +0,0 @@
"""Unified portal HTML export with V2R1C2 resource and execution centers.
The canonical dependency-free portal remains in ``aios_v2.console``. This
module applies deterministic, fail-fast extensions at import time so the
V2R1C1 production entrypoint remains unchanged while the V2R1C2 candidate gets
resource and portable-executor views.
"""
from __future__ import annotations
from aios_v2.console import CONSOLE_HTML as _BASE_PORTAL_HTML
def _extend_portal(source: str) -> str:
replacements = (
(
'<section id="overview" class="view active"></section>\n <section id="tasks" class="view"></section>\n <section id="authorization" class="view"></section>',
'<section id="overview" class="view active"></section>\n <section id="resources" class="view"></section>\n <section id="tasks" class="view"></section>\n <section id="execution" class="view"></section>\n <section id="authorization" class="view"></section>',
),
(
"const labels={overview:'系统总览',tasks:'任务中心',authorization:'授权中心'",
"const labels={overview:'系统总览',resources:'租户与 Agent',tasks:'任务中心',execution:'执行中心',authorization:'授权中心'",
),
(
"if(view==='overview')await overview();if(view==='tasks')await tasks();if(view==='authorization')await authorizations();",
"if(view==='overview')await overview();if(view==='resources')await resources();if(view==='tasks')await tasks();if(view==='execution')await execution();if(view==='authorization')await authorizations();",
),
(
"async function tasks(){const d=await api('/api/v2/portal/tasks');",
"""async function resources(){const d=await api('/api/v2/portal/resources');const t=d.tenant||{},agents=d.agents||[],ops=d.operations||{};const agentButton=ops.agent_create?'<button class=\"btn\" id=\"newAgent\">注册 Agent</button>':'';$('#resources').innerHTML=`<div class=\"cards\">${card('租户',t.tenant_id||'—',t.status||'')}${card('Agent',d.summary?.agents||0,`启用 ${d.summary?.enabled_agents||0}`)}${card('任务',d.summary?.tasks||0)}${card('Runtime 类型',(d.summary?.runtimes||[]).length,(d.summary?.runtimes||[]).join(', '))}</div><div class=\"grid\"><div class=\"panel\"><div class=\"panel-head\"><h2>当前租户</h2></div><div class=\"kv\"><div>Tenant ID</div><div>${esc(t.tenant_id||'—')}</div><div>名称</div><div>${esc(t.display_name||'—')}</div><div>状态</div><div>${pill(t.status||'unknown')}</div><div>创建时间</div><div>${esc(t.created_at||'—')}</div><div>更新时间</div><div>${esc(t.updated_at||'—')}</div></div></div><div class=\"panel\"><div class=\"panel-head\"><h2>Agent Registry</h2><div>${agentButton}</div></div>${table([['Agent',x=>esc(x.agent_id)],['名称',x=>esc(x.display_name)],['Runtime',x=>pill(x.runtime)],['能力',x=>(x.capabilities||[]).map(pill).join('')],['启用',x=>pill(x.enabled?'enabled':'disabled')],['任务数',x=>esc(x.task_count)]],agents)}</div></div>`;$('#newAgent')?.addEventListener('click',()=>{const agent_id=prompt('Agent ID');const display_name=prompt('显示名称',agent_id||'');const runtime=prompt('Runtime Provider','openclaw');const capabilities=prompt('能力,逗号分隔','');if(!agent_id||!display_name||!runtime)return;api('/api/v2/portal/agents',{method:'POST',body:JSON.stringify({agent_id,display_name,runtime,capabilities:String(capabilities||'').split(',').map(x=>x.trim()).filter(Boolean),metadata:{source:'unified-portal'}})}).then(resources).catch(err)})}
function executionStage(x){const s=x.stage||{},status=x.truth||{},record=status.record||{},gate=status.gate||{},d=x.dependency_gate||{},notifications=gate.notifications||[],evidence=record.evidence||{};const stateName=record.state||'WAITING_DEPENDENCY';const deps=(s.depends_on||[]).join(', ')||'根阶段';return `<div class=\"panel\" style=\"background:#fbfcfe\"><div class=\"panel-head\"><h2>${esc(s.stage_id)} · ${esc(s.role)}</h2><div>${pill(stateName)} ${pill(s.executor_type)}</div></div><div class=\"chain\"><div class=\"chain-item\"><b>Provider</b><div>${esc(s.executor_type)}</div><small>${esc(deps)}</small></div><span class=\"chain-arrow\">→</span><div class=\"chain-item\"><b>Run</b><div>${esc(record.executor_run_id||'未注册')}</div><small>${esc(record.executor_id||'')}</small></div><span class=\"chain-arrow\">→</span><div class=\"chain-item\"><b>Progress</b><div>${esc(record.current_step||0)}/${esc((record.manifest?.steps||s.steps||[]).length)}</div><small>${esc(record.heartbeat_at||'无心跳')}</small></div><span class=\"chain-arrow\">→</span><div class=\"chain-item\"><b>Evidence</b><div>${Object.keys(evidence).length} 项</div><small>${Object.keys(evidence).slice(0,2).map(esc).join(', ')}</small></div><span class=\"chain-arrow\">→</span><div class=\"chain-item\"><b>Notification</b><div>${notifications.length} 条</div><small>${notifications.map(n=>n.status).join(', ')||'无'}</small></div><span class=\"chain-arrow\">→</span><div class=\"chain-item\"><b>Gate</b><div>${pill(gate.passed?'PASS':'BLOCKED')}</div><small>${d.passed?'依赖满足':'等待依赖'}</small></div></div>${Object.keys(record.error||{}).length?json(record.error):''}</div>`}
async function execution(){const d=await api('/api/v2/portal/execution');const providers=d.providers||[],plans=d.plans||[];$('#execution').innerHTML=`<div class=\"cards\">${card('Provider',providers.length)}${card('Execution Plan',plans.length)}${card('协议',d.protocol_version||'—')}${card('Plan Schema',d.plan_schema_version||'—')}</div><div class=\"panel\"><div class=\"panel-head\"><h2>Executor Provider Catalog</h2></div>${table([['Provider',x=>esc(x.executor_type)],['Adapter',x=>esc(x.adapter_version)],['协议',x=>esc(x.protocol_version)],['Trust Roles',x=>(x.trust_roles||[]).map(pill).join('')],['Capabilities',x=>(x.capabilities||[]).map(pill).join('')]],providers)}</div>${plans.map(p=>`<div class=\"panel\"><div class=\"panel-head\"><h2>${esc(p.plan_id)} · ${esc(p.target_version)}</h2><div>${pill(p.gate?.passed?'PASS':'BLOCKED')}</div></div><div class=\"kv\"><div>Source Branch</div><div>${esc(p.source_branch)}</div><div>Source OID</div><div>${esc(p.source_oid)}</div><div>Release Gate</div><div>${pill(p.gate?.passed?'PASS':'BLOCKED')}</div></div></div>${(p.snapshot?.stages||[]).map(executionStage).join('')}`).join('')||'<div class=\"empty\">尚未声明执行计划</div>'}`}
async function tasks(){const d=await api('/api/v2/portal/tasks');""",
),
)
result = source
for old, new in replacements:
if old not in result:
raise RuntimeError(f"unified portal extension anchor not found: {old[:100]}")
result = result.replace(old, new, 1)
return result
PORTAL_HTML = _extend_portal(_BASE_PORTAL_HTML)
if "\n+function executionStage" in PORTAL_HTML or "\n+async function execution" in PORTAL_HTML:
raise RuntimeError("execution center contains an invalid diff marker")
__all__ = ["PORTAL_HTML"]
@@ -1,130 +0,0 @@
"""HTTP facade for portable multi-provider execution plans."""
from __future__ import annotations
import urllib.parse
from http import HTTPStatus
from typing import Any, Mapping
from aios_v2.control_plane import RequestContext, Unauthorized
from aios_v2.execution_plan import ExecutionPlanCoordinator
from aios_v2.executor_protocol import ExecutorProviderRegistry, normalize_executor
from .capabilities import require
class ExecutionPlanAPI:
"""Portal read model and provider-authenticated plan discovery API."""
INTERNAL_PREFIX = "/api/v2/internal/execution-plans"
PORTAL_PATH = "/api/v2/portal/execution"
def __init__(
self,
coordinator: ExecutionPlanCoordinator,
*,
providers: ExecutorProviderRegistry,
) -> None:
self.coordinator = coordinator
self.providers = providers
@classmethod
def is_route(cls, raw_path: str) -> bool:
path = urllib.parse.urlparse(raw_path).path.rstrip("/") or "/"
return path == cls.PORTAL_PATH or path.startswith(cls.INTERNAL_PREFIX)
def _require_provider(
self,
context: RequestContext,
executor_type: str,
) -> str:
normalized = normalize_executor(executor_type)
if not self.providers.trusted(
executor_type=normalized,
roles=tuple(context.roles),
platform_admin="platform_admin" in context.roles,
):
raise Unauthorized(
f"trusted role for executor Provider is required: {normalized}"
)
return normalized
def _portal_snapshot(self) -> Mapping[str, Any]:
plans = []
for plan in self.coordinator.plan_store.list():
plans.append(
{
"plan_id": plan.plan_id,
"target_version": plan.target_version,
"source_branch": plan.source_branch,
"source_oid": plan.source_oid,
"snapshot": self.coordinator.snapshot(plan.plan_id),
"gate": self.coordinator.gate(plan.plan_id),
}
)
return {
"protocol_version": "aios.executor.v1",
"plan_schema_version": "aios.execution-plan.v2",
"providers": [
item.as_dict() for item in self.providers.list_descriptors()
],
"plans": plans,
}
def handle(
self,
method: str,
raw_path: str,
context: RequestContext,
body: Any,
) -> tuple[int, Mapping[str, Any]]:
del body
parsed = urllib.parse.urlparse(raw_path)
path = parsed.path.rstrip("/") or "/"
method = method.upper()
if method == "GET" and path == self.PORTAL_PATH:
require(context, "external_task.read")
return HTTPStatus.OK, self._portal_snapshot()
if not path.startswith(f"{self.INTERNAL_PREFIX}/"):
return HTTPStatus.NOT_FOUND, {
"error": "not_found",
"message": "execution plan route not found",
}
remainder = urllib.parse.unquote(path[len(self.INTERNAL_PREFIX) + 1 :])
parts = tuple(item for item in remainder.split("/") if item)
if len(parts) != 2:
return HTTPStatus.NOT_FOUND, {
"error": "not_found",
"message": "execution plan route not found",
}
plan_id, action = parts
query = urllib.parse.parse_qs(parsed.query)
executor_type = self._require_provider(
context,
str(query.get("executor", [""])[0]),
)
if method == "GET" and action == "discover":
items = self.coordinator.discover_ready(
plan_id=plan_id,
executor_type=executor_type,
)
return HTTPStatus.OK, {
"plan_id": plan_id,
"executor_type": executor_type,
"count": len(items),
"items": items,
}
if method == "GET" and action == "snapshot":
return HTTPStatus.OK, self.coordinator.snapshot(plan_id)
if method == "GET" and action == "gate":
return HTTPStatus.OK, self.coordinator.gate(plan_id)
return HTTPStatus.NOT_FOUND, {
"error": "not_found",
"message": "execution plan route not found",
}
@@ -1,276 +0,0 @@
"""Protected HTTP facade for portable executor truth and notification receipts."""
from __future__ import annotations
import urllib.parse
from http import HTTPStatus
from pathlib import Path
from typing import Any, Mapping, Sequence
from aios_v2.control_plane import RequestContext, Unauthorized
from aios_v2.executor_protocol import ExecutorProviderRegistry, normalize_executor
from aios_v2.external_task_control import ExternalTaskCoordinator
from aios_v2.external_task_query import list_records
from aios_v2.task_execution_registry import ExecutionTruthState
from .capabilities import require
def _body(value: Any) -> Mapping[str, Any]:
if not isinstance(value, Mapping):
raise ValueError("JSON body must be an object")
return value
def _required(value: Any, label: str) -> str:
normalized = str(value or "").strip()
if not normalized:
raise ValueError(f"{label} is required")
return normalized
def _mapping_strings(value: Any, label: str) -> dict[str, str]:
if not isinstance(value, Mapping):
raise ValueError(f"{label} must be an object")
return {
_required(key, f"{label} key"): _required(item, f"{label}[{key}]")
for key, item in value.items()
}
class ExternalTaskAPI:
"""Fail-closed external task API shared by every registered Provider."""
INTERNAL_PREFIX = "/api/v2/internal/external-tasks"
def __init__(
self,
coordinator: ExternalTaskCoordinator,
*,
store_file: str | Path,
providers: ExecutorProviderRegistry,
) -> None:
self.coordinator = coordinator
self.store_file = Path(store_file).resolve()
self.providers = providers
@staticmethod
def is_route(path: str) -> bool:
normalized = path.rstrip("/") or "/"
return normalized.startswith(ExternalTaskAPI.INTERNAL_PREFIX) or normalized == "/api/v2/portal/external-tasks"
def _require_executor(self, context: RequestContext, executor_type: str) -> str:
normalized = normalize_executor(executor_type)
if not self.providers.trusted(
executor_type=normalized,
roles=tuple(context.roles),
platform_admin="platform_admin" in context.roles,
):
raise Unauthorized(
f"trusted role for executor Provider is required: {normalized}"
)
return normalized
def _task_executor(self, task_id: str) -> str:
return normalize_executor(
self.coordinator.registry.get(task_id).manifest.executor
)
@staticmethod
def _executor_identity(
context: RequestContext,
payload: Mapping[str, Any],
) -> str:
executor_id = _required(payload.get("executor_id"), "executor_id")
if executor_id != context.principal_id and "platform_admin" not in context.roles:
raise Unauthorized("executor_id must match trusted principal")
return executor_id
def declare_directory(self, directory: str | Path) -> list[Mapping[str, Any]]:
root = Path(directory).resolve()
if not root.exists():
return []
declared: list[Mapping[str, Any]] = []
for manifest in sorted(root.glob("*.json")):
record = self.coordinator.declare_file(manifest)
self.providers.validate_manifest(record.manifest)
declared.append(
{
"manifest": str(manifest),
"task_id": record.task_id,
"executor_type": record.manifest.executor,
"state": record.state.value,
}
)
return declared
def handle(
self,
method: str,
raw_path: str,
context: RequestContext,
body: Any,
) -> tuple[int, Mapping[str, Any]]:
parsed = urllib.parse.urlparse(raw_path)
path = parsed.path.rstrip("/") or "/"
method = method.upper()
payload = _body(body or {})
if method == "GET" and path == "/api/v2/portal/external-tasks":
require(context, "external_task.read")
records = list_records(self.store_file)
return HTTPStatus.OK, {
"count": len(records),
"items": [self.coordinator.status(item.task_id) for item in records],
}
if method == "GET" and path == f"{self.INTERNAL_PREFIX}/discover":
query = urllib.parse.parse_qs(parsed.query)
states = tuple(query.get("state", ()))
executor = self._require_executor(
context,
str(query.get("executor", [""])[0]),
)
records = list_records(
self.store_file,
executor=executor,
states=states,
)
return HTTPStatus.OK, {
"count": len(records),
"executor_type": executor,
"items": [item.as_dict() for item in records],
}
if method == "GET" and path == f"{self.INTERNAL_PREFIX}/notifications":
query = urllib.parse.parse_qs(parsed.query)
executor = self._require_executor(
context,
str(query.get("executor", [""])[0]),
)
task_id = query.get("task_id", [None])[0]
status = query.get("status", [None])[0]
items = self.coordinator.notifications.list(
task_id=task_id,
status=status,
)
filtered = [
item
for item in items
if self._task_executor(item.task_id) == executor
]
return HTTPStatus.OK, {
"count": len(filtered),
"executor_type": executor,
"items": [item.as_dict() for item in filtered],
}
notification_ack = self._match(
path,
f"{self.INTERNAL_PREFIX}/notifications/",
"/ack",
)
if method == "POST" and notification_ack:
notification = self.coordinator.notifications.get(notification_ack)
executor = self._task_executor(notification.task_id)
self._require_executor(context, executor)
self._executor_identity(context, payload)
record = self.coordinator.notifications.acknowledge(
notification_ack,
receipt_sha256=_required(
payload.get("receipt_sha256"),
"receipt_sha256",
),
)
return HTTPStatus.OK, {"notification": record.as_dict()}
truth_task = self._match(path, f"{self.INTERNAL_PREFIX}/", "/truth")
if method == "GET" and truth_task:
self._require_executor(context, self._task_executor(truth_task))
return HTTPStatus.OK, self.coordinator.status(truth_task)
for action in ("register", "retry", "claim", "start", "heartbeat", "succeed", "fail"):
task_id = self._match(path, f"{self.INTERNAL_PREFIX}/", f"/{action}")
if method != "POST" or not task_id:
continue
executor_type = self._task_executor(task_id)
self._require_executor(context, executor_type)
run_id = _required(payload.get("executor_run_id"), "executor_run_id")
if action == "register":
return HTTPStatus.OK, {
"record": self.coordinator.register(
task_id,
executor_run_id=run_id,
).as_dict()
}
if action == "retry":
return HTTPStatus.OK, {
"record": self.coordinator.retry_not_started(
task_id,
executor_run_id=run_id,
).as_dict()
}
executor_id = self._executor_identity(context, payload)
if action == "claim":
record = self.coordinator.claim(
task_id,
executor_run_id=run_id,
executor_id=executor_id,
)
elif action == "start":
record = self.coordinator.start(
task_id,
executor_run_id=run_id,
executor_id=executor_id,
)
elif action == "heartbeat":
record = self.coordinator.heartbeat(
task_id,
executor_run_id=run_id,
executor_id=executor_id,
current_step=int(payload.get("current_step")),
)
elif action == "succeed":
record = self.coordinator.succeed(
task_id,
executor_run_id=run_id,
executor_id=executor_id,
evidence=_mapping_strings(payload.get("evidence"), "evidence"),
)
else:
error = payload.get("error")
if not isinstance(error, Mapping):
raise ValueError("error must be an object")
evidence_value = payload.get("evidence")
evidence = (
_mapping_strings(evidence_value, "evidence")
if evidence_value is not None
else None
)
record = self.coordinator.fail(
task_id,
executor_run_id=run_id,
executor_id=executor_id,
error=dict(error),
evidence=evidence,
)
return HTTPStatus.OK, {"record": record.as_dict()}
return HTTPStatus.NOT_FOUND, {
"error": "not_found",
"message": "external task route not found",
}
@staticmethod
def _match(path: str, prefix: str, suffix: str) -> str | None:
if not path.startswith(prefix) or not path.endswith(suffix):
return None
middle = path[len(prefix) : len(path) - len(suffix)]
if not middle or "/" in middle:
return None
return urllib.parse.unquote(middle)
DISCOVERABLE_STATES: Sequence[str] = (
ExecutionTruthState.DECLARED.value,
ExecutionTruthState.REGISTERED.value,
)
@@ -1,376 +0,0 @@
"""V2R1C2 unified portal and portable executor release entrypoint.
The V2R1C1 production entrypoint remains unchanged. This candidate entrypoint
uses an isolated state directory, loads a versioned executor catalog, declares
portable multi-provider execution plans and exposes both portal read models and
provider-authenticated lifecycle APIs.
"""
from __future__ import annotations
import json
import os
import re
import urllib.parse
from dataclasses import replace
from http import HTTPStatus
from pathlib import Path
from typing import Any, Mapping, Sequence
import aios_v2.app as candidate_app_module
from aios_v2.app import (
CandidateApplication,
CandidateHTTPServer,
build_application as build_candidate_application,
)
from aios_v2.control_plane import RequestContext
from aios_v2.execution_plan import ExecutionPlanCoordinator, ExecutionPlanManifest
from aios_v2.executor_catalog import FileExecutorCatalogStore
from aios_v2.official_executor_providers import official_provider_factories
from .application import UnifiedPortalApplication, _json_value, _mapping
from .capabilities import allows, require
from .console import PORTAL_HTML
from .execution_plan_api import ExecutionPlanAPI
from .external_task_api import ExternalTaskAPI
_READ_BRIDGE_ROLES = frozenset({"approver", "runtime", "viewer", "release_manager"})
_GIT_OID = re.compile(r"^[0-9a-f]{40}$")
def _resolve_environment_value(value: Any, *, label: str) -> Any:
if not isinstance(value, str) or not value.startswith("ENV:"):
return value
variable = value[4:].strip()
if not variable:
raise ValueError(f"{label} environment variable name is empty")
resolved = str(os.getenv(variable, "")).strip()
if not resolved:
raise RuntimeError(f"required environment variable is not set: {variable}")
return resolved
def _load_execution_plan(path: str | Path) -> ExecutionPlanManifest:
source = Path(path).resolve()
value = json.loads(source.read_text(encoding="utf-8"))
if not isinstance(value, Mapping):
raise ValueError(f"execution plan must contain a JSON object: {source}")
payload = dict(value)
for label in ("source_branch", "source_oid", "target_version"):
payload[label] = _resolve_environment_value(payload.get(label), label=label)
source_oid = str(payload.get("source_oid") or "").strip().lower()
if not _GIT_OID.fullmatch(source_oid):
raise ValueError(
"execution plan source_oid must resolve to an exact 40-character Git OID"
)
payload["source_oid"] = source_oid
return ExecutionPlanManifest.from_dict(payload)
def _path_list(value: str) -> tuple[Path, ...]:
return tuple(
Path(item).expanduser().resolve()
for item in value.split(os.pathsep)
if item.strip()
)
class V2R1C2PortalApplication(UnifiedPortalApplication):
"""Unified portal candidate with provider-neutral execution governance."""
def __init__(
self,
base: CandidateApplication,
*,
provenance_store_file: str | Path,
external_task_api: ExternalTaskAPI,
execution_plan_api: ExecutionPlanAPI,
plan_coordinator: ExecutionPlanCoordinator,
catalog_snapshot: Mapping[str, Any],
declared_plans: Sequence[str],
) -> None:
super().__init__(base, provenance_store_file=provenance_store_file)
self.external_task_api = external_task_api
self.execution_plan_api = execution_plan_api
self.plan_coordinator = plan_coordinator
self.catalog_snapshot = dict(catalog_snapshot)
self.declared_plans = tuple(declared_plans)
def context(self, headers: Mapping[str, str]) -> RequestContext:
original = self.base.context(headers)
if original.roles.intersection(_READ_BRIDGE_ROLES):
return replace(original, roles=frozenset((*original.roles, "auditor")))
return original
def _resource_snapshot(self, context: RequestContext) -> Mapping[str, Any]:
service = self.orchestrator.control_plane
tenant = service.get_tenant(context)
agents = service.list_agents(context) if allows(context, "agent.read") else []
tasks = service.list_tasks(context) if allows(context, "task.read") else []
tasks_by_agent: dict[str, int] = {}
for task in tasks:
tasks_by_agent[task.agent_id] = tasks_by_agent.get(task.agent_id, 0) + 1
return {
"tenant": _json_value(tenant),
"agents": [
{
**_json_value(agent),
"task_count": tasks_by_agent.get(agent.agent_id, 0),
}
for agent in agents
],
"summary": {
"agents": len(agents),
"enabled_agents": sum(1 for agent in agents if agent.enabled),
"tasks": len(tasks),
"runtimes": sorted({agent.runtime for agent in agents}),
},
"operations": {
"tenant_create": allows(context, "tenant.manage"),
"agent_create": allows(context, "agent.manage"),
},
}
def _runtime_snapshot(
self,
context: RequestContext,
headers: Mapping[str, str],
) -> Mapping[str, Any]:
return {
**self._runtime(headers),
"dispatches": _json_value(
self.orchestrator.dispatch_repository.list(
tenant_id=context.tenant_id
)
),
"hermes_jobs": _json_value(
self.hermes.list_jobs(tenant_id=context.tenant_id)
),
"contract": {
"openclaw": os.getenv(
"AIOS_OPENCLAW_CONTRACT_VERSION",
"aios.openclaw.command.v2",
),
"signature_algorithm": os.getenv(
"AIOS_OPENCLAW_SIGNATURE_ALGORITHM",
"Ed25519",
),
"plugin_version": os.getenv(
"AIOS_OPENCLAW_PLUGIN_VERSION",
"2.1.0-rc.4",
),
"key_id": os.getenv(
"AIOS_OPENCLAW_SIGNING_KEY_ID",
"openclaw-v2r1-candidate-ed25519",
),
"executor_protocol": "aios.executor.v1",
"execution_plan": "aios.execution-plan.v2",
},
}
def _release_snapshot(self, context: RequestContext) -> Mapping[str, Any]:
plans = []
for plan_id in self.declared_plans:
plans.append(
{
"plan_id": plan_id,
"gate": self.plan_coordinator.gate(plan_id),
}
)
aggregate_passed = bool(plans) and all(
item["gate"]["passed"] for item in plans
)
return {
**self._gitops(context),
"version": os.getenv("AIOS_RELEASE_VERSION", "V2R1C2-CANDIDATE"),
"target_version": os.getenv("AIOS_TARGET_VERSION", "V2R1C2"),
"release_oid": os.getenv("AIOS_RELEASE_OID", "candidate-not-tagged"),
"source_oid": os.getenv("AIOS_RELEASE_SOURCE_OID", "missing"),
"tag": os.getenv("AIOS_RELEASE_TAG", "NOT_CREATED"),
"baseline": os.getenv("AIOS_BASELINE_VERSION", "V2R1C1"),
"previous_version": os.getenv("AIOS_PREVIOUS_VERSION", "V2R1C1"),
"status": os.getenv("AIOS_RELEASE_STATUS", "CANDIDATE_VALIDATION"),
"evidence_manifest_sha256": os.getenv(
"AIOS_RELEASE_EVIDENCE_SHA256",
"pending",
),
"executor_catalog": {
"path": self.catalog_snapshot.get("path"),
"sha256": self.catalog_snapshot.get("catalog_sha256"),
"protocol_version": "aios.executor.v1",
},
"execution_plans": plans,
"execution_gate": {
"passed": aggregate_passed,
"release_blocking": not aggregate_passed,
},
"operations": (
["review_release", "request_tag", "request_rollback"]
if allows(context, "release.operate")
else []
),
"note": (
"Release operations remain GitOps approval requests. The portal "
"never mutates tags, main, or production directly. Tag approval "
"must remain blocked until every required execution-plan gate passes."
),
}
def handle_json(
self,
method: str,
raw_path: str,
headers: Mapping[str, str],
body: Any,
) -> tuple[int, Mapping[str, Any]]:
parsed = urllib.parse.urlparse(raw_path)
path = parsed.path.rstrip("/") or "/"
method = method.upper()
if self.execution_plan_api.is_route(raw_path):
context = (
self.context(headers)
if path == ExecutionPlanAPI.PORTAL_PATH
else self.base.context(headers)
)
return self.execution_plan_api.handle(
method,
raw_path,
context,
body,
)
if self.external_task_api.is_route(path):
context = (
self.context(headers)
if path == "/api/v2/portal/external-tasks"
else self.base.context(headers)
)
return self.external_task_api.handle(
method,
raw_path,
context,
body,
)
context: RequestContext | None = None
if path in {
"/api/v2/portal/resources",
"/api/v2/portal/runtime",
"/api/v2/portal/release",
"/api/v2/portal/tenants",
"/api/v2/portal/agents",
}:
context = self.context(headers)
if method == "GET" and path == "/api/v2/portal/resources":
assert context is not None
require(context, "tenant.read")
return HTTPStatus.OK, self._resource_snapshot(context)
if method == "POST" and path == "/api/v2/portal/tenants":
assert context is not None
require(context, "tenant.manage")
return self.base.handle_json(
"POST",
"/api/v2/tenants",
headers,
_mapping(body),
)
if method == "POST" and path == "/api/v2/portal/agents":
assert context is not None
require(context, "agent.manage")
return self.base.handle_json(
"POST",
"/api/v2/agents",
headers,
_mapping(body),
)
if method == "GET" and path == "/api/v2/portal/runtime":
assert context is not None
require(context, "runtime.read")
return HTTPStatus.OK, self._runtime_snapshot(context, headers)
if method == "GET" and path == "/api/v2/portal/release":
assert context is not None
require(context, "release.read")
return HTTPStatus.OK, self._release_snapshot(context)
return super().handle_json(method, raw_path, headers, body)
def build_application() -> V2R1C2PortalApplication:
state_dir = Path(
os.getenv("AIOS_V2_STATE_DIR", "/var/lib/aios-v2r1c2-portal")
).resolve()
state_dir.mkdir(parents=True, exist_ok=True)
provenance_file = Path(
os.getenv(
"AIOS_PROVENANCE_STORE_FILE",
str(state_dir / "run-provenance.sqlite3"),
)
).resolve()
truth_file = Path(
os.getenv(
"AIOS_EXTERNAL_TASK_STORE_FILE",
str(state_dir / "external-task-truth.sqlite3"),
)
).resolve()
catalog_file = Path(
os.getenv(
"AIOS_EXECUTOR_CATALOG_FILE",
str(state_dir / "executor-catalog.json"),
)
).resolve()
catalog_store = FileExecutorCatalogStore(catalog_file)
catalog_snapshot = catalog_store.initialize_defaults()
providers = catalog_store.registry(
provider_factories=official_provider_factories()
)
plan_coordinator = ExecutionPlanCoordinator.from_file(
truth_file,
providers=providers,
)
declared_plans: list[str] = []
plan_files = _path_list(os.getenv("AIOS_EXECUTION_PLAN_FILES", ""))
for plan_file in plan_files:
plan = _load_execution_plan(plan_file)
plan_coordinator.declare(plan)
declared_plans.append(plan.plan_id)
base: CandidateApplication = build_candidate_application()
external_task_api = ExternalTaskAPI(
plan_coordinator.task_coordinator,
store_file=truth_file,
providers=providers,
)
execution_plan_api = ExecutionPlanAPI(
plan_coordinator,
providers=providers,
)
return V2R1C2PortalApplication(
base,
provenance_store_file=provenance_file,
external_task_api=external_task_api,
execution_plan_api=execution_plan_api,
plan_coordinator=plan_coordinator,
catalog_snapshot=catalog_snapshot.as_dict(),
declared_plans=declared_plans,
)
def main() -> None:
candidate_app_module.CONSOLE_HTML = PORTAL_HTML
host = os.getenv("AIOS_V2_HOST", "127.0.0.1")
port = int(os.getenv("AIOS_V2_PORT", "18921"))
server = CandidateHTTPServer((host, port), build_application())
print(f"AIOS V2R1C2 unified portal listening on {host}:{port}")
server.serve_forever()
if __name__ == "__main__":
main()
+687
View File
@@ -0,0 +1,687 @@
#!/usr/bin/env python3
"""One-shot governed executor for an already approved exact release binding.
This module deliberately does not build release artifacts. The build/staging
evidence is completed before Owner production approval. Production execution
consumes that immutable binding, revalidates it, and performs narrow
side-effects through adapters in a durable fail-closed order.
Adapter protocol:
stdin: JSON request with operation = "probe" | "apply" | "rollback"
stdout: one JSON object on the final non-empty line
shell: never used
The executor never rewrites tags, never writes refs/heads/main, and records
Baseline last.
"""
from __future__ import annotations
import argparse
import dataclasses
import fcntl
import hashlib
import json
import os
import re
import stat
import subprocess
import sys
import tempfile
import time
from pathlib import Path
from typing import Any, Mapping, Sequence
from trusted_release_execution_plan import PlanError, TrustedReleasePlan
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
PLAN_RE = re.compile(r"^sha256:[0-9a-f]{64}$")
REQUIRED_V2R1C7_MEMBERS = {
"payload/VERSION",
"payload/active-runtime/app/deployment_state.py",
"payload/active-runtime/app/v2r1c6_privacy_tools.py",
"payload/active-runtime/app/v2r1c7_health_service.py",
"payload/deploy/aios-v2/systemd/aios-v2r1c7-production.service",
"payload/deploy/aios-v2/v2r1c7-runtime-config.json",
"payload/deploy/aios-v2/v2r1c7-rollback.json",
"payload/scripts/probe_v2r1c7_health.py",
"payload/scripts/rollback_v2r1c7_release.sh",
"payload/BUILD-METADATA.json",
"payload/ARTIFACT-MANIFEST.json",
"payload/ROLLBACK-METADATA.json",
}
class ReleaseBlocked(RuntimeError):
pass
class ManualOwnerReviewRequired(ReleaseBlocked):
pass
class ReconciliationRequired(ReleaseBlocked):
pass
@dataclasses.dataclass(frozen=True)
class Binding:
version: str
target_oid: str
evidence_closure_oid: str
artifact_sha256: str
approval_plan_id: str
artifact_path: str
artifact_size: int
production_service: str
production_port: int
repository_identity: str = "aios/aios"
candidate_id: str | None = None
release_plan_sha256: str | None = None
@classmethod
def from_trusted_plan(cls, plan: TrustedReleasePlan) -> "Binding":
value = plan.value
adapter = value["adapter_policy"]
binding = cls(
version=str(value["version"]), target_oid=str(value["source_commit"]),
evidence_closure_oid=str(value["evidence_closure_oid"]), artifact_sha256=str(value["artifact_sha256"]),
approval_plan_id=str(value["approval_plan_id"]), artifact_path=str(value["artifact"]["path"]),
artifact_size=int(value["artifact"]["size"]), production_service=str(adapter["service"]),
production_port=int(adapter["port"]), candidate_id=str(value["candidate_id"]),
release_plan_sha256=plan.release_plan_sha256,
)
plan.assert_binding(dataclasses.asdict(binding))
return binding
def __post_init__(self) -> None:
if not self.version or not OID_RE.fullmatch(self.target_oid):
raise ValueError("invalid release target binding")
if not OID_RE.fullmatch(self.evidence_closure_oid):
raise ValueError("invalid evidence closure OID")
if not SHA_RE.fullmatch(self.artifact_sha256):
raise ValueError("invalid artifact SHA-256")
if not PLAN_RE.fullmatch(self.approval_plan_id):
raise ValueError("invalid approval plan id")
if self.artifact_size <= 0:
raise ValueError("invalid artifact size")
if not self.production_service:
raise ValueError("production service is required")
if not 1024 <= self.production_port <= 65535:
raise ValueError("invalid production port")
if self.repository_identity != "aios/aios":
raise ValueError("unexpected repository identity")
@property
def tag(self) -> str:
return self.version
@property
def fingerprint(self) -> str:
payload = {
"version": self.version,
"target_oid": self.target_oid,
"evidence_closure_oid": self.evidence_closure_oid,
"artifact_sha256": self.artifact_sha256,
"approval_plan_id": self.approval_plan_id,
"artifact_size": self.artifact_size,
"production_service": self.production_service,
"production_port": self.production_port,
"repository_identity": self.repository_identity,
"candidate_id": self.candidate_id,
"release_plan_sha256": self.release_plan_sha256,
}
return hashlib.sha256(canonical_json(payload)).hexdigest()
def request(self, operation: str) -> dict[str, Any]:
return {
"schema": "aios.production-executor-request.v1",
"operation": operation,
"binding_fingerprint": self.fingerprint,
"version": self.version,
"tag": self.tag,
"target_oid": self.target_oid,
"evidence_closure_oid": self.evidence_closure_oid,
"artifact_sha256": self.artifact_sha256,
"artifact_path": self.artifact_path,
"artifact_size": self.artifact_size,
"approval_plan_id": self.approval_plan_id,
"production_service": self.production_service,
"production_port": self.production_port,
"repository_identity": self.repository_identity,
"candidate_id": self.candidate_id,
"release_plan_sha256": self.release_plan_sha256,
}
def canonical_json(value: object) -> bytes:
return (json.dumps(value, sort_keys=True, separators=(",", ":")) + "\n").encode("utf-8")
def secure_json(path: Path) -> dict[str, Any]:
st = path.lstat()
if not stat.S_ISREG(st.st_mode) or path.is_symlink():
raise ReleaseBlocked(f"non-regular authorization file: {path}")
if st.st_uid != 0:
raise ReleaseBlocked(f"authorization file must be root-owned: {path}")
if st.st_mode & 0o022:
raise ReleaseBlocked(f"authorization file is group/world writable: {path}")
data = json.loads(path.read_text(encoding="utf-8"))
if not isinstance(data, dict):
raise ReleaseBlocked("authorization receipt must be a JSON object")
return data
def sha256_file(path: Path) -> str:
if path.is_symlink() or not path.is_file():
raise ReleaseBlocked("artifact must be a regular non-symlink file")
digest = hashlib.sha256()
with path.open("rb") as source:
for block in iter(lambda: source.read(1024 * 1024), b""):
digest.update(block)
return digest.hexdigest()
def run_git(git_dir: Path, *args: str, check: bool = True) -> subprocess.CompletedProcess[str]:
return subprocess.run(
["git", f"--git-dir={git_dir}", *args],
check=check,
capture_output=True,
text=True,
timeout=30,
env={"PATH": os.environ.get("PATH", "")},
)
def verify_git_lineage(git_dir: Path, binding: Binding) -> dict[str, Any]:
authority_main = run_git(git_dir, "rev-parse", "refs/heads/main").stdout.strip()
if not OID_RE.fullmatch(authority_main):
raise ReleaseBlocked("Authority main OID is invalid")
target_type = run_git(git_dir, "cat-file", "-t", binding.target_oid).stdout.strip()
if target_type != "commit":
raise ReleaseBlocked("target_oid is not a commit")
evidence_type = run_git(
git_dir, "cat-file", "-t", binding.evidence_closure_oid
).stdout.strip()
if evidence_type != "commit":
raise ReleaseBlocked("evidence_closure_oid is not a commit")
# A control-plane-only patch may advance main after release approval.
# The approved release target remains valid only while it is still on main.
target_on_main = run_git(
git_dir,
"merge-base",
"--is-ancestor",
binding.target_oid,
authority_main,
check=False,
).returncode == 0
if not target_on_main:
raise ManualOwnerReviewRequired(
"approved target is no longer an ancestor of Authority main"
)
target_in_evidence = run_git(
git_dir,
"merge-base",
"--is-ancestor",
binding.target_oid,
binding.evidence_closure_oid,
check=False,
).returncode == 0
if not target_in_evidence:
raise ReleaseBlocked("approved target is not an ancestor of evidence closure")
return {
"authority_main_oid": authority_main,
"target_oid": binding.target_oid,
"target_is_ancestor_of_main": True,
"evidence_closure_oid": binding.evidence_closure_oid,
"evidence_object_type": evidence_type,
"target_is_ancestor_of_evidence": True,
}
def verify_artifact(binding: Binding) -> dict[str, Any]:
path = Path(binding.artifact_path)
if path.stat().st_size != binding.artifact_size:
raise ReleaseBlocked("artifact size mismatch")
actual = sha256_file(path)
if actual != binding.artifact_sha256:
raise ReleaseBlocked("artifact SHA-256 mismatch")
if binding.version == "V2R1C7":
# Python tarfile may not have zstd support on the target host, so
# decompress into a private temporary tar and inspect it there.
with tempfile.TemporaryDirectory(prefix="aios-artifact-verify-") as temp_dir:
tar_path = Path(temp_dir) / "release.tar"
with tar_path.open("wb") as output:
completed = subprocess.run(
["zstd", "-dc", str(path)],
check=False,
stdout=output,
stderr=subprocess.PIPE,
timeout=60,
env={"PATH": os.environ.get("PATH", "")},
)
if completed.returncode != 0:
raise ReleaseBlocked("artifact zstd decompression failed")
import tarfile
with tarfile.open(tar_path, "r:") as archive:
names: set[str] = set()
for member in archive.getmembers():
candidate = Path(member.name)
if (
candidate.is_absolute()
or ".." in candidate.parts
or member.issym()
or member.islnk()
or not (member.isfile() or member.isdir())
):
raise ReleaseBlocked(f"unsafe artifact member: {member.name}")
names.add(member.name)
missing = sorted(REQUIRED_V2R1C7_MEMBERS - names)
if missing:
raise ReleaseBlocked(
"artifact missing required members: " + ",".join(missing)
)
return {
"artifact_path": str(path),
"artifact_sha256": actual,
"artifact_size": binding.artifact_size,
"verified": True,
}
def verify_authorization(receipt: Mapping[str, Any], binding: Binding) -> dict[str, Any]:
if receipt.get("accepted") is not True:
raise ReleaseBlocked("Owner production authorization is not accepted")
if receipt.get("binding_replaced") not in (None, False):
raise ManualOwnerReviewRequired("Owner binding was replaced")
expected = {
"version": binding.version,
"target_oid": binding.target_oid,
"evidence_closure_oid": binding.evidence_closure_oid,
"artifact_sha256": binding.artifact_sha256,
"approval_plan_id": binding.approval_plan_id,
}
if binding.candidate_id is not None:
expected["candidate_id"] = binding.candidate_id
if binding.release_plan_sha256 is not None:
expected["release_plan_sha256"] = binding.release_plan_sha256
if receipt.get("candidate_binding_sha256") is None:
raise ManualOwnerReviewRequired("Owner authorization missing candidate binding")
for key, value in expected.items():
if receipt.get(key) != value:
raise ManualOwnerReviewRequired(f"Owner authorization mismatch: {key}")
return {"accepted": True, **expected}
class DurableState:
def __init__(self, root: Path, binding: Binding):
self.root = root
self.root.mkdir(parents=True, exist_ok=True)
os.chmod(self.root, 0o700)
self.path = self.root / "execution.json"
self.binding = binding
self.value: dict[str, Any]
if self.path.exists():
existing = json.loads(self.path.read_text(encoding="utf-8"))
if existing.get("binding_fingerprint") != binding.fingerprint:
raise ManualOwnerReviewRequired(
"state root contains a different production binding"
)
self.value = existing
else:
self.value = {
"schema": "aios.production-execution-state.v1",
"binding_fingerprint": binding.fingerprint,
"version": binding.version,
"target_oid": binding.target_oid,
"state": "AUTHORIZED",
"completed": {},
"history": [],
}
self.save()
def save(self) -> None:
temporary = self.path.with_suffix(".tmp")
temporary.write_bytes(canonical_json(self.value))
os.chmod(temporary, 0o600)
os.replace(temporary, self.path)
def record(self, state: str, step: str, result: Mapping[str, Any]) -> None:
now = time.time()
sanitized = json.loads(json.dumps(result))
self.value["state"] = state
self.value["completed"][step] = sanitized
self.value["history"].append(
{"at": now, "state": state, "step": step, "result": sanitized}
)
self.save()
def fail(self, state: str, reason: str) -> None:
self.value["state"] = state
self.value["history"].append(
{"at": time.time(), "state": state, "reason": reason}
)
self.save()
class FileLock:
def __init__(self, path: Path):
self.path = path
self.handle = None
def __enter__(self) -> "FileLock":
self.path.parent.mkdir(parents=True, exist_ok=True)
self.handle = self.path.open("a+", encoding="utf-8")
os.chmod(self.path, 0o600)
try:
fcntl.flock(
self.handle.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB
)
except BlockingIOError as exc:
raise ReleaseBlocked("production executor is already running") from exc
return self
def __exit__(self, exc_type, exc, tb) -> None:
if self.handle is not None:
fcntl.flock(self.handle.fileno(), fcntl.LOCK_UN)
self.handle.close()
def parse_command(values: Sequence[str]) -> list[str]:
command = [str(x) for x in values if str(x)]
if not command:
raise ValueError("adapter command is required")
if not os.path.isabs(command[0]):
raise ValueError("adapter executable must be an absolute path")
return command
def run_adapter(command: Sequence[str], request: Mapping[str, Any]) -> dict[str, Any]:
completed = subprocess.run(
list(command),
input=canonical_json(request),
check=False,
capture_output=True,
timeout=180,
env={"PATH": os.environ.get("PATH", "")},
)
if completed.returncode != 0:
raise ReleaseBlocked(
f"adapter failed: {Path(command[0]).name} rc={completed.returncode}"
)
lines = [line for line in completed.stdout.decode("utf-8").splitlines() if line.strip()]
if not lines:
raise ReleaseBlocked("adapter returned no JSON result")
try:
value = json.loads(lines[-1])
except ValueError as exc:
raise ReleaseBlocked("adapter returned invalid JSON") from exc
if not isinstance(value, dict):
raise ReleaseBlocked("adapter result must be an object")
return value
def require_exact_response(result: Mapping[str, Any], binding: Binding) -> None:
for key, expected in {
"version": binding.version,
"target_oid": binding.target_oid,
"evidence_closure_oid": binding.evidence_closure_oid,
"artifact_sha256": binding.artifact_sha256,
"approval_plan_id": binding.approval_plan_id,
}.items():
if key in result and result[key] != expected:
raise ManualOwnerReviewRequired(f"adapter binding mismatch: {key}")
def tag_step(command: Sequence[str], binding: Binding) -> dict[str, Any]:
request = binding.request("probe")
probe = run_adapter(command, request)
require_exact_response(probe, binding)
state = probe.get("status")
if state == "CONFLICT":
raise ManualOwnerReviewRequired("same-name tag conflicts with approved binding")
if state == "INDETERMINATE":
raise ReconciliationRequired("tag publication state is indeterminate")
if state not in {"APPLIED_VERIFIED", "NOT_APPLIED", "PARTIAL_LOCAL"}:
raise ReleaseBlocked(f"unsupported tag probe status: {state}")
if state != "APPLIED_VERIFIED":
applied = run_adapter(command, binding.request("apply"))
require_exact_response(applied, binding)
if applied.get("status") == "CONFLICT":
raise ManualOwnerReviewRequired("tag conflict during publication")
probe = run_adapter(command, request)
require_exact_response(probe, binding)
if probe.get("status") != "APPLIED_VERIFIED":
raise ReconciliationRequired("tag was not independently verified")
if probe.get("peeled_target_oid") != binding.target_oid:
raise ManualOwnerReviewRequired("published tag peeled target mismatch")
tag_oid = probe.get("tag_oid")
if not isinstance(tag_oid, str) or not OID_RE.fullmatch(tag_oid):
raise ReleaseBlocked("verified annotated tag object OID missing")
return dict(probe)
def deploy_step(command: Sequence[str], binding: Binding) -> dict[str, Any]:
probe = run_adapter(command, binding.request("probe"))
require_exact_response(probe, binding)
if probe.get("status") == "ACTIVE_VERIFIED":
return dict(probe)
if probe.get("status") == "CONFLICT":
raise ManualOwnerReviewRequired("production runtime has conflicting binding")
applied = run_adapter(command, binding.request("apply"))
require_exact_response(applied, binding)
probe = run_adapter(command, binding.request("probe"))
require_exact_response(probe, binding)
if probe.get("status") != "ACTIVE_VERIFIED":
try:
run_adapter(command, binding.request("rollback"))
except Exception:
pass
raise ReconciliationRequired("production runtime did not verify after apply")
if (
probe.get("deployment_state") != "ACTIVE"
or probe.get("production_deployed") is not True
):
raise ReconciliationRequired("production health contract is not ACTIVE")
return dict(probe)
def finalize_step(
command: Sequence[str],
binding: Binding,
*,
name: str,
tag_oid: str,
) -> dict[str, Any]:
request = binding.request("probe")
request["tag_oid"] = tag_oid
probe = run_adapter(command, request)
require_exact_response(probe, binding)
if probe.get("status") == "CONFLICT":
raise ManualOwnerReviewRequired(f"{name} has conflicting release binding")
if probe.get("status") != "VERIFIED":
apply_request = binding.request("apply")
apply_request["tag_oid"] = tag_oid
applied = run_adapter(command, apply_request)
require_exact_response(applied, binding)
probe = run_adapter(command, request)
require_exact_response(probe, binding)
if probe.get("status") != "VERIFIED":
raise ReconciliationRequired(f"{name} did not verify after apply")
if probe.get("tag_oid") not in (None, tag_oid):
raise ManualOwnerReviewRequired(f"{name} tag object mismatch")
return dict(probe)
def execute(
*,
binding: Binding,
authorization_path: Path,
repository: Path,
state_root: Path,
tag_command: Sequence[str],
deploy_command: Sequence[str],
registry_command: Sequence[str],
baseline_command: Sequence[str],
release_plan_path: Path | None = None,
) -> dict[str, Any]:
plan = TrustedReleasePlan.load(release_plan_path) if release_plan_path is not None else None
if plan is not None:
try:
plan.assert_binding(dataclasses.asdict(binding))
except PlanError as exc:
raise ReleaseBlocked(f"trusted release plan mismatch: {exc}") from exc
authorization = verify_authorization(secure_json(authorization_path), binding)
lineage = verify_git_lineage(repository, binding)
artifact = verify_artifact(binding)
state = DurableState(state_root, binding)
with FileLock(state_root / "executor.lock"):
state.record(
"PREFLIGHT_VERIFIED",
"preflight",
{"authorization": authorization, "lineage": lineage, "artifact": artifact},
)
tag: dict[str, Any] | None = None
runtime: dict[str, Any] | None = None
try:
tag = tag_step(tag_command, binding)
state.record("TAG_VERIFIED", "tag", tag)
runtime = deploy_step(deploy_command, binding)
state.record("RUNTIME_VERIFIED", "runtime", runtime)
registry = finalize_step(
registry_command,
binding,
name="registry",
tag_oid=tag["tag_oid"],
)
state.record("REGISTRY_VERIFIED", "registry", registry)
baseline = finalize_step(
baseline_command,
binding,
name="baseline",
tag_oid=tag["tag_oid"],
)
state.record("BASELINE_VERIFIED", "baseline", baseline)
# Final readback: no mutation, all four planes must still agree.
final_tag = tag_step(tag_command, binding)
final_runtime = run_adapter(deploy_command, binding.request("probe"))
final_registry_request = binding.request("probe")
final_registry_request["tag_oid"] = tag["tag_oid"]
final_registry = run_adapter(registry_command, final_registry_request)
final_baseline_request = binding.request("probe")
final_baseline_request["tag_oid"] = tag["tag_oid"]
final_baseline = run_adapter(baseline_command, final_baseline_request)
for value in (final_runtime, final_registry, final_baseline):
require_exact_response(value, binding)
if final_runtime.get("status") != "ACTIVE_VERIFIED":
raise ReconciliationRequired("final runtime readback failed")
if final_registry.get("status") != "VERIFIED":
raise ReconciliationRequired("final Registry readback failed")
if final_baseline.get("status") != "VERIFIED":
raise ReconciliationRequired("final Baseline readback failed")
final = {
"status": "PRODUCTION_ACCEPTED",
"version": binding.version,
"target_oid": binding.target_oid,
"evidence_closure_oid": binding.evidence_closure_oid,
"artifact_sha256": binding.artifact_sha256,
"approval_plan_id": binding.approval_plan_id,
"tag_oid": tag["tag_oid"],
"tag": final_tag,
"runtime": final_runtime,
"registry": final_registry,
"baseline": final_baseline,
"force_push_used": False,
"tag_rewritten": False,
"direct_main_push": False,
"secrets_committed": False,
}
state.record("COMPLETED", "final_consistency", final)
return final
except ManualOwnerReviewRequired as exc:
state.fail("MANUAL_OWNER_REVIEW_REQUIRED", str(exc))
raise
except ReconciliationRequired as exc:
state.fail("RECONCILIATION_REQUIRED", str(exc))
raise
except Exception as exc:
# Once a formal tag has been verified, no later error is a clean
# pre-side-effect failure. Preserve the tag and require idempotent
# reconciliation rather than pretending the release never started.
if tag is not None:
reason = f"{type(exc).__name__}: {exc}"
state.fail("RECONCILIATION_REQUIRED", reason)
raise ReconciliationRequired(reason) from exc
state.fail("FAIL_CLOSED", f"{type(exc).__name__}: {exc}")
raise
def parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser()
p.add_argument("--authorization", required=True, type=Path)
p.add_argument("--release-plan", type=Path)
p.add_argument("--repository", required=True, type=Path)
p.add_argument("--state-root", required=True, type=Path)
p.add_argument("--artifact", type=Path)
p.add_argument("--artifact-size", type=int)
p.add_argument("--version")
p.add_argument("--target-oid")
p.add_argument("--evidence-closure-oid")
p.add_argument("--artifact-sha256")
p.add_argument("--approval-plan-id")
p.add_argument("--production-service")
p.add_argument("--production-port", type=int)
p.add_argument("--repository-identity", default="aios/aios")
p.add_argument("--tag-adapter", required=True, nargs="+")
p.add_argument("--deploy-adapter", required=True, nargs="+")
p.add_argument("--registry-adapter", required=True, nargs="+")
p.add_argument("--baseline-adapter", required=True, nargs="+")
return p
def main() -> int:
args = parser().parse_args()
plan = TrustedReleasePlan.load(args.release_plan) if args.release_plan else None
if plan is not None:
binding = Binding.from_trusted_plan(plan)
else:
legacy=(args.artifact,args.artifact_size,args.version,args.target_oid,args.evidence_closure_oid,args.artifact_sha256,args.approval_plan_id,args.production_service,args.production_port)
if any(value is None for value in legacy): raise SystemExit("legacy binding is incomplete")
binding = Binding(version=args.version,target_oid=args.target_oid,evidence_closure_oid=args.evidence_closure_oid,artifact_sha256=args.artifact_sha256,approval_plan_id=args.approval_plan_id,artifact_path=str(args.artifact),artifact_size=args.artifact_size,production_service=args.production_service,production_port=args.production_port,repository_identity=args.repository_identity)
try:
result = execute(
binding=binding,
authorization_path=args.authorization,
repository=args.repository,
state_root=args.state_root,
tag_command=parse_command(args.tag_adapter),
deploy_command=parse_command(args.deploy_adapter),
registry_command=parse_command(args.registry_adapter),
baseline_command=parse_command(args.baseline_adapter),
release_plan_path=args.release_plan,
)
print(json.dumps(result, sort_keys=True))
return 0
except ManualOwnerReviewRequired as exc:
print(json.dumps({"status": "MANUAL_OWNER_REVIEW_REQUIRED", "reason": str(exc)}, sort_keys=True))
return 20
except ReconciliationRequired as exc:
print(json.dumps({"status": "RECONCILIATION_REQUIRED", "reason": str(exc)}, sort_keys=True))
return 21
except Exception as exc:
print(json.dumps({"status": "FAIL_CLOSED", "error": type(exc).__name__, "reason": str(exc)}, sort_keys=True))
return 22
if __name__ == "__main__":
raise SystemExit(main())
+65 -2
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import os
import re
import subprocess
import time
import urllib.parse
from pathlib import Path
from typing import Iterable
@@ -306,14 +307,76 @@ def authoritative_diff(
from_oid = validate_git_revision(from_revision, "from", git_dir=path)
to_oid = validate_git_revision(to_revision, "to", git_dir=path)
args = ["diff", "--no-ext-diff", "--no-textconv", from_oid, to_oid]
result = run_authoritative_git(args, git_dir=path)
patch_limit = max(1, min(_MAX_OUTPUT, 256 * 1024))
process = subprocess.Popen(
["git", "--git-dir", str(path), *args],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
env={**os.environ, "GIT_OPTIONAL_LOCKS": "0"},
)
chunks: list[bytes] = []
stored_bytes = 0
total_bytes = 0
started = time.monotonic()
try:
while process.stdout is not None:
chunk = process.stdout.read(64 * 1024)
if not chunk:
break
total_bytes += len(chunk)
if stored_bytes < patch_limit:
remaining = patch_limit - stored_bytes
selected = chunk[:remaining]
chunks.append(selected)
stored_bytes += len(selected)
if time.monotonic() - started > _TIMEOUT:
process.kill()
raise AuthoritativeGitError("GIT_REF_QUERY_TIMEOUT")
stderr = (
process.stderr.read(8192).decode("utf-8", errors="replace")
if process.stderr is not None
else ""
)
returncode = process.wait(timeout=max(0.1, _TIMEOUT - (time.monotonic() - started)))
except (OSError, subprocess.TimeoutExpired) as exc:
process.kill()
process.wait()
raise AuthoritativeGitError("GIT_REF_QUERY_FAILED") from exc
if returncode != 0:
raise AuthoritativeGitError(_redact(stderr.strip() or "GIT_REF_QUERY_FAILED")[:200])
try:
names_result = run_authoritative_git(
["diff", "--no-ext-diff", "--no-textconv", "--name-status", from_oid, to_oid],
git_dir=path,
)
changed_paths = [
line.split("\t", 1)[-1]
for line in names_result.stdout.splitlines()
if line.strip()
]
except AuthoritativeGitOutputTooLarge:
changed_paths = []
try:
stats_result = run_authoritative_git(
["diff", "--no-ext-diff", "--no-textconv", "--stat", from_oid, to_oid],
git_dir=path,
)
stats = stats_result.stdout.strip()
except AuthoritativeGitOutputTooLarge:
stats = "GIT_STAT_OUTPUT_TRUNCATED"
patch = b"".join(chunks).decode("utf-8", errors="replace")
return {
"code": 0,
"stdout": result.stdout,
"stdout": patch,
"stderr": "",
"cmd": ["git", f"--git-dir={path}", *args],
"from_oid": from_oid,
"to_oid": to_oid,
"changed_paths": changed_paths,
"stats": stats,
"total_size_bytes": total_bytes,
"truncated": total_bytes > patch_limit,
"patch_limit_bytes": patch_limit,
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,89 @@
"""Durable Owner/GPT notification outbox with retry and acknowledgement."""
from __future__ import annotations
import json
import sqlite3
import time
import uuid
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Callable, Mapping
@dataclass(frozen=True)
class OutboxEvent:
event_id: str
recipient: str
kind: str
payload: Mapping[str, Any]
attempts: int
available_at: float
acknowledged: bool
class DurableOutbox:
RECIPIENTS = ("owner", "gpt")
def __init__(self, path: str | Path) -> None:
self.path = str(path)
self.db = sqlite3.connect(self.path)
self.db.execute("PRAGMA journal_mode=WAL")
self.db.execute("CREATE TABLE IF NOT EXISTS outbox (event_id TEXT PRIMARY KEY, recipient TEXT NOT NULL, kind TEXT NOT NULL, payload TEXT NOT NULL, attempts INTEGER NOT NULL DEFAULT 0, available_at REAL NOT NULL, delivered_at REAL, acknowledged INTEGER NOT NULL DEFAULT 0, last_error TEXT)")
self.db.commit()
def enqueue(self, kind: str, payload: Mapping[str, Any], *, event_id: str | None = None, recipients: tuple[str, ...] = RECIPIENTS) -> list[str]:
ids: list[str] = []
for recipient in recipients:
if recipient not in self.RECIPIENTS:
raise ValueError("recipient is outside the governed notification plane")
eid = event_id or str(uuid.uuid4())
if event_id is not None and len(recipients) > 1:
eid = f"{event_id}:{recipient}"
self.db.execute("INSERT OR IGNORE INTO outbox(event_id,recipient,kind,payload,available_at) VALUES(?,?,?,?,?)", (eid, recipient, kind, json.dumps(dict(payload), sort_keys=True), time.time()))
ids.append(eid)
self.db.commit()
return ids
def due(self, *, now: float | None = None, limit: int = 20) -> list[OutboxEvent]:
now = time.time() if now is None else now
rows = self.db.execute("SELECT event_id,recipient,kind,payload,attempts,available_at,acknowledged FROM outbox WHERE delivered_at IS NULL AND available_at<=? ORDER BY available_at,event_id LIMIT ?", (now, limit)).fetchall()
return [OutboxEvent(r[0], r[1], r[2], json.loads(r[3]), r[4], r[5], bool(r[6])) for r in rows]
def delivered(self, event_id: str, *, now: float | None = None) -> None:
self.db.execute("UPDATE outbox SET delivered_at=? WHERE event_id=?", (time.time() if now is None else now, event_id))
self.db.commit()
def failed(self, event_id: str, error: str, *, now: float | None = None, max_backoff: float = 300.0) -> None:
now = time.time() if now is None else now
row = self.db.execute("SELECT attempts FROM outbox WHERE event_id=?", (event_id,)).fetchone()
attempts = int(row[0]) + 1 if row else 1
delay = min(max_backoff, 2 ** min(attempts, 8))
self.db.execute("UPDATE outbox SET attempts=?,available_at=?,last_error=? WHERE event_id=?", (attempts, now + delay, error[:200], event_id))
self.db.commit()
def acknowledge(self, event_id: str) -> None:
self.db.execute("UPDATE outbox SET acknowledged=1 WHERE event_id=?", (event_id,))
self.db.commit()
def close(self) -> None:
self.db.close()
class NotificationPlane:
def __init__(self, outbox: DurableOutbox) -> None:
self.outbox = outbox
def emit(self, kind: str, payload: Mapping[str, Any], *, event_id: str | None = None) -> list[str]:
return self.outbox.enqueue(kind, payload, event_id=event_id)
def deliver_once(self, send: Callable[[OutboxEvent], None], *, now: float | None = None) -> list[str]:
delivered: list[str] = []
for event in self.outbox.due(now=now):
try:
send(event)
except Exception as exc:
self.outbox.failed(event.event_id, type(exc).__name__, now=now)
else:
self.outbox.delivered(event.event_id, now=now)
delivered.append(event.event_id)
return delivered
+84
View File
@@ -0,0 +1,84 @@
#!/usr/bin/env python3
"""Single canonical Production Baseline authority for the GitOps reader.
The API and the one-shot writer import this module. Workspace and release
copies are projections only; they are never selected as a fallback source.
"""
from __future__ import annotations
import hashlib
import json
import os
from pathlib import Path
from typing import Any
AUTHORITY_PATH = Path(os.getenv(
"AIOS_GITOPS_BASELINE_AUTHORITY",
"/var/lib/aios-gitops-runtime-anchor/baseline/current.json",
))
AUTHORITY_SHA_PATH = AUTHORITY_PATH.with_suffix(".sha256")
class BaselineAuthorityError(RuntimeError):
"""Fail-closed error raised for an unavailable or inconsistent authority."""
class BaselineAuthority:
def __init__(self, path: Path | str = AUTHORITY_PATH):
self.path = Path(path)
self.sha_path = self.path.with_suffix(".sha256")
def _raw(self) -> bytes:
try:
raw = self.path.read_bytes()
except OSError as exc:
raise BaselineAuthorityError(f"authority_unavailable:{exc}") from exc
if not raw:
raise BaselineAuthorityError("authority_empty")
return raw
def read(self) -> dict[str, Any]:
raw = self._raw()
digest = hashlib.sha256(raw).hexdigest()
try:
value = json.loads(raw.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise BaselineAuthorityError("authority_invalid_json") from exc
if not isinstance(value, dict):
raise BaselineAuthorityError("authority_not_object")
expected = str(value.get("authority_sha256", ""))
if expected and expected != digest:
raise BaselineAuthorityError("authority_embedded_digest_mismatch")
if self.sha_path.exists():
sidecar = self.sha_path.read_text(encoding="utf-8").strip().split()[0]
if sidecar != digest:
raise BaselineAuthorityError("authority_sidecar_digest_mismatch")
required = {
"current_baseline", "previous_baseline", "target_oid", "tag_oid",
"evidence_closure_oid", "approval_plan_id", "status", "generation",
"source_identity",
}
missing = sorted(required.difference(value))
if missing:
raise BaselineAuthorityError("authority_missing:" + ",".join(missing))
value = dict(value)
value["authority_sha256"] = digest
value["authority_path"] = str(self.path)
return value
def read_consistent(self, policy: dict[str, Any] | None = None) -> dict[str, Any]:
value = self.read()
if policy:
expected = policy.get("release_binding", policy)
for key, field in (("version", "current_baseline"),
("previous_version", "previous_baseline"),
("target_oid", "target_oid"),
("tag_oid", "tag_oid"),
("evidence_closure_oid", "evidence_closure_oid"),
("approval_plan_id", "approval_plan_id")):
wanted = expected.get(key)
if wanted is not None and value.get(field) != wanted:
raise BaselineAuthorityError(f"authority_binding_mismatch:{field}")
return value

Some files were not shown because too many files have changed in this diff Show More