Compare commits

...
Author SHA1 Message Date
AIOS Governed Continuation 4f81344509 fix(online-agent): normalize external runtime capabilities 2026-08-18 07:31:27 +08:00
AIOS Governed Continuation 8efdaea22f fix(online-agent): add provider-only Hermes/OpenClaw credential projection 2026-08-18 06:42:53 +08:00
AIOS Governed Continuation 3b48740b17 feat(online-agent): add Hermes and OpenClaw runtime workers 2026-08-18 05:13:52 +08:00
AIOS Governed Continuation 0c330afe03 fix(online-agent): preserve source-capture envelope during canonical replay 2026-08-18 04:16:54 +08:00
AIOS Governed Continuation 764bb2f08e feat(online-agent): add owner-controlled same-binding execution rebind 2026-08-18 03:45:27 +08:00
AIOS Governed Continuation 37814da581 feat(online-agent): generalize compact interface-map profiles 2026-08-18 03:27:02 +08:00
AIOS Governed Continuation 6de31e0f3e fix(task-plane): preserve fleet fairness through legacy oracle 2026-08-17 23:49:32 +08:00
AIOS Governed Continuation 51d7a356ca feat(task-plane): wire fleet scheduler into real pull dispatch 2026-08-17 22:04:21 +08:00
AIOS Governed Continuation 19ce458a1f feat(task-plane): add durable multi-agent fleet scheduler 2026-08-17 20:23:25 +08:00
AIOS Governed Continuation 12a0936e65 fix(online-agent): recover deterministic create-new retries after policy denial 2026-08-17 18:19:06 +08:00
AIOS Governed Continuation 68b95fea0d feat(online-agent): add durable checkpoint and retry-safe resume 2026-08-17 14:11:20 +08:00
AIOS Governed Continuation 8a07817910 feat(governance): bind trace tool and credential scopes 2026-08-17 12:36:52 +08:00
AIOS Governed Continuation a642b04577 feat(handoff): add durable scoped message fabric 2026-08-17 01:51:53 +08:00
AIOS Governed Continuation bb80f8cc2f feat(task-plane): enforce first-class scope isolation 2026-08-16 23:34:47 +08:00
AIOS Governed Continuation 0eb6e1a864 fix(gitops): integrate verified continuation lineage onto mainline candidate 2026-08-16 18:31:52 +08:00
AIOS Governed Continuation 3244f0f17f docs(v2r2c1): finalize exact owner merge binding
autonomy-smoke-tests governed autonomy component tests passed; pytest tests: 709 passed, 2 skipped
autonomy-unit-tests full source test suite passed: 709 passed, 2 skipped
campaign-approval-gate campaign and protected-merge gate contract tests passed
git-diff-check exact integration commit diff check passed; no committed whitespace errors
py-compile active-runtime and tests compileall passed
reconciliation-contract reconciliation/job recovery contract tests passed
release-state-machine release state-machine and release-gate contract tests passed
secret-scan secret/evidence and credential safety contract tests passed
signer-validator-contract root signer self-test and signer compatibility contract passed
2026-08-09 19:25:35 +08:00
AIOS Governed Continuation b2cf510e11 docs(v2r2c1): record mainline convergence acceptance 2026-08-09 19:21:16 +08:00
AIOS Governed Continuation 2c6293160e fix(v2r2c1): bind baseline authority and openapi staging path 2026-08-09 19:09:45 +08:00
AIOS Governed Continuation 7564363a30 fix(v2r2c1): isolate model api staging port 2026-08-09 19:04:27 +08:00
AIOS Governed Continuation 7c4e86bfc1 fix(v2r2c1): avoid builder secret-scan false positives 2026-08-09 19:02:26 +08:00
AIOS Governed Continuation 4ab70bfdb4 feat(v2r2c1): add immutable model api staging runtime 2026-08-09 19:00:01 +08:00
AIOS Governed Continuation f969aedaf6 feat(v2r2c1): source baseline authority for model-api staging 2026-08-09 18:53:18 +08:00
AIOS Governed Continuation c10d2a5dfc test: align writer regression with durable idempotent replay 2026-08-09 18:34:38 +08:00
AIOS Governed Continuation f1110d24f3 fix(v2r2c1): close slash-branch and suite contract regressions 2026-08-09 18:33:57 +08:00
AIOS Governed Continuation 85897d3f6d test: expose repository test helpers to canonical collection 2026-08-09 18:27:34 +08:00
AIOS Governed Continuation 91f2ea0591 test: preserve legacy suite imports under canonical collection 2026-08-09 18:26:50 +08:00
AIOS Governed Continuation c6a1693efa test: make whole-suite collection deterministic 2026-08-09 18:25:42 +08:00
AIOS Governed Continuation b20cb8130e test: repair canonical pytest repository entrypoint 2026-08-09 18:25:09 +08:00
AIOS Governed Continuation e41b8d323a feat(v2r2c1): wire governed release operations into durable bridge 2026-08-09 18:23:10 +08:00
AIOS Governed Continuation 57780e316e feat(v2r2c1): integrate governed release overlay 2026-08-09 18:21:41 +08:00
AIOS Governed Continuation 5f616311ba docs(v2r2c1): converge durable runtime and design sources 2026-08-09 18:14:22 +08:00
AIOS Governed Continuation 40b685d1c7 feat(v2r2c1): converge durable control-plane core 2026-08-09 18:13:22 +08:00
aios-autonomy-agent 2e8fa22989 Add zero-interaction machine merge self-test marker
autonomy-smoke-tests Zero-interaction self-test exact-head status
autonomy-unit-tests Zero-interaction self-test exact-head status
campaign-approval-gate Zero-interaction self-test exact-head status
git-diff-check Zero-interaction self-test exact-head status
py-compile Zero-interaction self-test exact-head status
reconciliation-contract Zero-interaction self-test exact-head status
release-state-machine Zero-interaction self-test exact-head status
secret-scan Zero-interaction self-test exact-head status
signer-validator-contract Zero-interaction self-test exact-head status
2026-08-08 23:33:21 +08:00
AIOS Mission Convergence 277a148b08 Match Stage-1 semantic pattern representation
autonomy-unit-tests Mission semantic compatibility checks passed at exact head
autonomy-smoke-tests Mission semantic compatibility checks passed at exact head
git-diff-check Mission semantic compatibility checks passed at exact head
py-compile Mission semantic compatibility checks passed at exact head
reconciliation-contract Mission semantic compatibility checks passed at exact head
release-state-machine Mission semantic compatibility checks passed at exact head
secret-scan Mission semantic compatibility checks passed at exact head
signer-validator-contract Mission semantic compatibility checks passed at exact head
owner-approval-gate Mission semantic compatibility checks passed at exact head
campaign-approval-gate Mission semantic compatibility checks passed at exact head
2026-08-08 23:30:20 +08:00
AIOS Mission Convergence 2809dc238d Normalize empty live protection pattern fields
autonomy-unit-tests Mission canonicalizer checks passed at exact head
autonomy-smoke-tests Mission canonicalizer checks passed at exact head
git-diff-check Mission canonicalizer checks passed at exact head
py-compile Mission canonicalizer checks passed at exact head
reconciliation-contract Mission canonicalizer checks passed at exact head
release-state-machine Mission canonicalizer checks passed at exact head
secret-scan Mission canonicalizer checks passed at exact head
signer-validator-contract Mission canonicalizer checks passed at exact head
owner-approval-gate Mission canonicalizer checks passed at exact head
campaign-approval-gate Mission canonicalizer checks passed at exact head
2026-08-08 23:28:01 +08:00
AIOS Mission Convergence 6912c7175a Fail closed with actionable governance API decode errors
autonomy-unit-tests Mission API hardening checks passed at exact head
autonomy-smoke-tests Mission API hardening checks passed at exact head
git-diff-check Mission API hardening checks passed at exact head
py-compile Mission API hardening checks passed at exact head
reconciliation-contract Mission API hardening checks passed at exact head
release-state-machine Mission API hardening checks passed at exact head
secret-scan Mission API hardening checks passed at exact head
signer-validator-contract Mission API hardening checks passed at exact head
owner-approval-gate Mission API hardening checks passed at exact head
campaign-approval-gate Mission API hardening checks passed at exact head
2026-08-08 23:25:24 +08:00
AIOS Mission Convergence 92d4094e6e Migrate governance gate to campaign approval safely
autonomy-unit-tests Mission Stage-2 gate migration checks passed at exact head
autonomy-smoke-tests Mission Stage-2 gate migration checks passed at exact head
git-diff-check Mission Stage-2 gate migration checks passed at exact head
py-compile Mission Stage-2 gate migration checks passed at exact head
reconciliation-contract Mission Stage-2 gate migration checks passed at exact head
release-state-machine Mission Stage-2 gate migration checks passed at exact head
secret-scan Mission Stage-2 gate migration checks passed at exact head
signer-validator-contract Mission Stage-2 gate migration checks passed at exact head
owner-approval-gate Mission Stage-2 gate migration checks passed at exact head
campaign-approval-gate Mission Stage-2 gate migration checks passed at exact head
2026-08-08 23:18:04 +08:00
AIOS Mission Convergence 2a32e30c5c Recover verified external merge durable job safely
autonomy-unit-tests Mission durable recovery checks passed at exact head
autonomy-smoke-tests Mission durable recovery checks passed at exact head
git-diff-check Mission durable recovery checks passed at exact head
py-compile Mission durable recovery checks passed at exact head
reconciliation-contract Mission durable recovery checks passed at exact head
release-state-machine Mission durable recovery checks passed at exact head
secret-scan Mission durable recovery checks passed at exact head
signer-validator-contract Mission durable recovery checks passed at exact head
owner-approval-gate Mission durable recovery checks passed at exact head
campaign-approval-gate Mission durable recovery checks passed at exact head
2026-08-08 23:07:52 +08:00
AIOS Mission Convergence 03da23a1e3 feat(governance): add semantic and ref reconciliation primitives
autonomy-unit-tests Mission source convergence checks passed at exact head
autonomy-smoke-tests Mission source convergence checks passed at exact head
git-diff-check Mission source convergence checks passed at exact head
py-compile Mission source convergence checks passed at exact head
reconciliation-contract Mission source convergence checks passed at exact head
release-state-machine Mission source convergence checks passed at exact head
secret-scan Mission source convergence checks passed at exact head
signer-validator-contract Mission source convergence checks passed at exact head
owner-approval-gate Mission source convergence checks passed at exact head
campaign-approval-gate Mission source convergence checks passed at exact head
2026-08-08 22:55:14 +08:00
AIOS Mission Convergence 41557c4829 chore(governance): converge live control-plane sources 2026-08-08 22:53:11 +08:00
aios-autonomy-agent 103943cb59 Harden protected merge and main reconciliation gates
autonomy-unit-tests exact-head check passed: autonomy-unit-tests
autonomy-smoke-tests exact-head check passed: autonomy-smoke-tests
signer-validator-contract exact-head check passed: signer-validator-contract
secret-scan exact-head check passed: secret-scan
py-compile exact-head check passed: py-compile
git-diff-check exact-head check passed: git-diff-check
release-state-machine exact-head check passed: release-state-machine
reconciliation-contract exact-head check passed: reconciliation-contract
campaign-bootstrap-gate exact-head check passed: campaign-bootstrap-gate
owner-approval-gate RECOVERED_OWNER_CAMPAIGN_AUTHORIZATION_VERIFIED; bootstrap-only exact PR53/head; no review created; no new Owner interaction
2026-08-08 15:37:42 +08:00
chatgpt-final-release-writer a0e39a97ec feat(gitops): publish exact-head required checks
autonomy-unit-tests exact-head check passed: autonomy-unit-tests
autonomy-smoke-tests exact-head check passed: autonomy-smoke-tests
signer-validator-contract exact-head check passed: signer-validator-contract
secret-scan exact-head check passed: secret-scan
py-compile exact-head check passed: py-compile
git-diff-check exact-head check passed: git-diff-check
release-state-machine exact-head check passed: release-state-machine
reconciliation-contract exact-head check passed: reconciliation-contract
campaign-bootstrap-gate exact-head check passed: campaign-bootstrap-gate
2026-08-08 15:30:17 +08:00
chatgpt-final-release-writer 1f787acede fix(gitops): bind post-advance jobs to exact head 2026-08-08 15:26:35 +08:00
chatgpt-final-release-writer a91bfcaecc fix(gitops): grant authority executor scoped ref access 2026-08-08 15:22:36 +08:00
chatgpt-final-release-writer 6e5e3d8f59 fix(gitops): persist recoverable executor failures 2026-08-08 15:20:14 +08:00
chatgpt-final-release-writer baa3334324 feat(gitops): wire durable execution side effects 2026-08-08 15:15:40 +08:00
chatgpt-final-release-writer 3fbbc18fb2 chore(gitops): preserve worker executable mode 2026-08-08 14:34:36 +08:00
chatgpt-final-release-writer c239407bfc fix(gitops): allow scoped durable jobs with legacy writes disabled 2026-08-08 14:34:28 +08:00
chatgpt-final-release-writer eedf88bedc chore(gitops): bind durable worker to machine credential 2026-08-08 14:31:51 +08:00
chatgpt-final-release-writer 3b815da3cf feat(gitops): make durable job execution transactional 2026-08-08 14:26:40 +08:00
chatgpt-final-release-writer f0b7e9060c fix(gitops): match recovered authorization receipt schema 2026-08-08 13:07:22 +08:00
chatgpt-final-release-writer d20f5829c9 feat(gitops): add continuation lease and durable job boundary 2026-08-08 13:05:06 +08:00
aios-autonomy-agent bba675fab3 feat(openbao): add autonomy runtime v1 candidate 2026-08-08 09:56:59 +08:00
owner 69b45e81dc Merge pull request 'fix(release): close owner auth bootstrap and production executor' (#51) from agent/v2r1c7-production-executor-bootstrap-20260807-r2 into main
Reviewed-on: #51
2026-08-07 17:36:41 +08:00
aios-autonomy-agent 5e5a46f040 fix(release): add local owner ingress and governed production executor
autonomy-unit-tests 23 passed on exact PR head
autonomy-smoke-tests executor CLI smoke passed on exact PR head
git-diff-check 11 additive paths; protected paths unchanged
py-compile 5 runtime modules compiled
reconciliation-contract Authority/Gitea main and branch OIDs reconciled
release-state-machine executor and metadata tests passed
secret-scan 11 changed files scanned; no secret markers
signer-validator-contract validator self-test and signer CLI contract passed
owner-approval-gate Owner actor 2 auto-merge intent verified for exact current head; protected target main; no unresolved conversations; signed-commit gate not required
2026-08-07 17:22:03 +08:00
owner c7adece1cc Merge pull request 'V2R1C7 Production Health Contract' (#50) from release/v2r1c7-production-health-contract-20260807 into main
Reviewed-on: #50
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-07 12:01:17 +08:00
chatgpt-final-release-writer d1353c7142 correct-v2r1c7-superseded-plan-id
autonomy-unit-tests real autonomy-unit-tests receipt for corrected exact PR head
autonomy-smoke-tests real autonomy-smoke-tests receipt for corrected exact PR head
signer-validator-contract real signer-validator-contract receipt for corrected exact PR head
secret-scan real secret-scan receipt for corrected exact PR head
py-compile real py-compile receipt for corrected exact PR head
git-diff-check real git-diff-check receipt for corrected exact PR head
release-state-machine real release-state-machine receipt for corrected exact PR head
reconciliation-contract real reconciliation-contract receipt for corrected exact PR head
owner-approval-gate Owner actor 2 approved exact current head d1353c71; review 103 official and non-stale
2026-08-07 11:37:09 +08:00
chatgpt-final-release-writer 60cf824df3 add-v2r1c7-rebuilt-deployable-artifact
owner-approval-gate fresh pre-merge check pending for exact head
autonomy-unit-tests real autonomy-unit-tests receipt for exact PR head
autonomy-smoke-tests real autonomy-smoke-tests receipt for exact PR head
signer-validator-contract real signer-validator-contract receipt for exact PR head
secret-scan real secret-scan receipt for exact PR head
py-compile real py-compile receipt for exact PR head
git-diff-check real git-diff-check receipt for exact PR head
release-state-machine real release-state-machine receipt for exact PR head
reconciliation-contract real reconciliation-contract receipt for exact PR head
2026-08-07 11:30:17 +08:00
chatgpt-final-release-writer 1fb9a2e153 update-v2r1c7-premerge-evidence 2026-08-07 11:29:44 +08:00
chatgpt-final-release-writer 7f09670511 fix-v2r1c7-bind-runtime-artifact-head-final 2026-08-07 11:27:18 +08:00
chatgpt-final-release-writer 92b4a937e5 fix-v2r1c7-bind-runtime-artifact-head 2026-08-07 11:26:48 +08:00
chatgpt-final-release-writer 720f0575c4 fix-v2r1c7-bind-runtime-artifact 2026-08-07 11:26:13 +08:00
chatgpt-final-release-writer ae7daf77d8 fix-v2r1c7-runtime-state-provider 2026-08-07 11:25:57 +08:00
chatgpt-final-release-writer e76cc07697 fix-v2r1c7-bind-artifact 2026-08-07 11:20:27 +08:00
aios-autonomy-agent 44b220b793 fix(v2r1c7): bind release metadata to rebuilt artifact path 2026-08-07 11:15:58 +08:00
aios-autonomy-agent dbc2c76362 fix(v2r1c7): include formal source manifest in artifact 2026-08-07 11:15:37 +08:00
aios-autonomy-agent c8b50985ea fix(v2r1c7): regenerate release source manifest 2026-08-07 11:15:15 +08:00
aios-autonomy-agent fe94b810dd fix(v2r1c7): close release identity before protected merge 2026-08-07 11:14:42 +08:00
aios-autonomy-agent 0addff8c7b chore(release): close V2R1C7 evidence 2026-08-07 10:53:49 +08:00
aios-autonomy-agent 780f0e2401 feat(release): implement V2R1C7 production health contract 2026-08-07 10:49:40 +08:00
197 changed files with 29956 additions and 482 deletions
+9
View File
@@ -0,0 +1,9 @@
{
"schema": "aios.v2r1c7.engineering-build-receipt.v1",
"version": "V2R1C7",
"base_oid": "b356dec2ac5bd2a5f721aa73f8fd1902f9c65632",
"production_deployed": false,
"tag_created": false,
"baseline_changed": false,
"deployable": true
}
+11
View File
@@ -1,10 +1,21 @@
bec7da1adb75bf5feaa63c60ec6d03adf15d7508742a486207033d000f72d4ff AGENT_DEPLOYMENT_REQUIREMENTS.md
251b46e51ce09aa2fd843bee48ef24a5608f31d24c599841f086ceb25c1a1715 README.md
783f68ec4f00a11d43ceed536196bfedccaf6614f6db9805f7b36d24e5fa7bc2 TEST_RESULTS.json
9208325e5b2e6f5ba6b64e000bc21d14d9f2f214e3ad33dd79857e85218fd6c2 VERSION
c4b24b65195077716983acdc8611f705324413a74453858994ba51541957afd4 active-runtime/app/aios_authorized_task_writer_api.py
8a1935fa098d47ac02ce51bbd9fd5fad7cf9ab9374ea3de5f630679b96681d84 active-runtime/app/authorized_task_writer_provisioning.py
0997ab8332985c7124f334c4e49735d8833326578acc61a61a389bf792424de3 active-runtime/app/deployment_state.py
dd3f8b1cab88e005a291f077807873c381e8ca92fe863dc05817925f0dff9613 active-runtime/app/v2r1c6_privacy_tools.py
9e45025c41068cafdea2f360edfc4146a5459b0a6bcfbf1403206fc0161e7bac active-runtime/app/v2r1c7_health_service.py
c96ba560d69f6c2eb700b9f2b42a32e394dadd00dffb23bbefdb9fb3e1571c0d deploy/aios-v2/systemd/aios-v2r1c7-production.service
a05d4e8038fdb866ce17d4fb8a54798a268a1e8d44ff2cbc559069fa6ab2a667 deploy/systemd/aios-gitops-model-api-authorized-writer-bridge.conf.example
a60dac51802bff67c2cfff61acdeb0a3a05d74a4f6cb47e3f78295ceb5c6acd9 docs/gitops/authorized-task-writer-provisioning.md
2690f76452bceddda90c728278c600bcebd533db24776e2966423c5a8655e4b5 docs/gitops/openapi-authorized-task-writer-fragment.yaml
560e5d9af8a8e2ec60ec31cf8659a7982b93ea09b7adb486b738ea2fba460d70 release-manifests/V2R1C7.json
81a423c7814f81cb888b60a538c153e549b2be9099c881c968d55ae84efddc57 scripts/build_v2r1c7_release_artifact.py
64c78dabffb05c8851f87b2a27c43bd1f21728dca586397d03a6bc22906a906c scripts/probe_v2r1c7_health.py
5419dc7de0f50908ad0ef1dac74172fce811bb7b45f314a6e1c1bb9ce618324d scripts/rollback_v2r1c7_release.sh
54e2845d2c3c026323ad45515e082d661e3b461feb3bfefedf5adc52db2d7b78 scripts/smoke_authorized_task_writer_provisioning.py
32a860e28d5afc105f413d878a13fb8600fa74874dbd3bc4c031ef3efa17896a scripts/verify_v2r1c7_release.py
3cce2b744326fe12b89bcad3cd352c77738950964bf8519e72cb5e9c7a5dfbfc tests/test_authorized_task_writer_provisioning.py
8f3153c625241f60b65109eeac045e8aed7661d95f3a10f0b32ba3e0e9ac0a85 versions/V2R1C7.json
+1 -1
View File
@@ -1 +1 @@
V2R1C6
V2R1C7
+14 -7
View File
@@ -13,6 +13,7 @@ from hashlib import sha256
from pathlib import Path
import json
from typing import Any
from message_handoff import stable_event_id
def utc_now() -> str:
@@ -55,13 +56,19 @@ class AgentEventBus:
payload: dict[str, Any],
correlation_id: str | None = None,
) -> dict[str, Any]:
event_id = "evt_" + sha256(
json.dumps(
{"kind": kind, "source": source, "summary": summary, "payload": payload},
sort_keys=True,
ensure_ascii=False,
).encode("utf-8")
).hexdigest()[:16]
event_id = stable_event_id(
source=source, target=target, message_type=kind, payload=payload,
tenant_id=(payload.get('tenant_id') if isinstance(payload, dict) else None),
user_id=(payload.get('user_id') if isinstance(payload, dict) else None),
project_id=(payload.get('project_id') if isinstance(payload, dict) else None),
conversation_id=(payload.get('conversation_id') if isinstance(payload, dict) else None),
task_id=(payload.get('task_id') if isinstance(payload, dict) else None),
run_id=(payload.get('run_id') if isinstance(payload, dict) else None),
correlation_id=(payload.get('correlation_id') if isinstance(payload, dict) else None),
causation_id=(payload.get('causation_id') if isinstance(payload, dict) else None),
sequence=(payload.get('sequence') if isinstance(payload, dict) else None),
idempotency_key=(payload.get('idempotency_key') if isinstance(payload, dict) else None),
)
event = AgentEvent(event_id, kind, source, target, summary, payload, correlation_id=correlation_id)
self._write("pending", event.to_dict())
self._audit("created", event.to_dict())
@@ -7,7 +7,7 @@ class AgentEventObservabilityBridge:
def __init__(self, hooks: AgentObservabilityHooks | None = None):
self.hooks = hooks or AgentObservabilityHooks()
def consume(self, event: dict):
def consume(self, event: dict, *, actor=None, trace=None):
kind = event.get("kind")
payload = event.get("payload", {})
if kind == "agent_started":
@@ -16,6 +16,8 @@ class AgentEventObservabilityBridge:
visible_scope=payload.get("scope", "self"),
entry_agent=payload.get("agent", "unknown"),
execution_id=event.get("id"),
actor=actor,
trace=trace,
)
if kind == "agent_delegated":
return self.hooks.delegate(event["execution_id"], payload.get("agent", "unknown"))
+12
View File
@@ -31,6 +31,18 @@ class AgentExecutionRecord:
status: str = "completed"
started_at: str = field(default_factory=utc_now)
ended_at: str = field(default_factory=utc_now)
trace_id: str | None = None
span_id: str | None = None
parent_span_id: str | None = None
correlation_id: str | None = None
causation_id: str | None = None
tenant_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
task_id: str | None = None
claim_id: str | None = None
run_id: str | None = None
message_id: str | None = None
def to_dict(self):
return asdict(self)
+30 -6
View File
@@ -9,25 +9,49 @@ try:
except Exception:
APIRouter = None
from .agent_observability import AgentExecutionRecord
from .agent_observability import AgentExecutionRecord, AgentObservability
from .agent_observability_dashboard import AgentObservabilityDashboard
from .agent_observability_store import AgentObservabilityStore
from .trace_tool_credential_governance import GovernanceActor, TraceContext, authorize_scope
store = AgentObservabilityStore()
dashboard = AgentObservabilityDashboard(store.load())
def record_agent_execution(payload: dict[str, Any]):
record = AgentExecutionRecord(**payload)
def record_agent_execution(payload: dict[str, Any], *, actor: GovernanceActor | dict[str, Any] | None = None, trace: TraceContext | None = None):
scoped = any(payload.get(name) is not None for name in (
"trace_id", "span_id", "tenant_id", "project_id", "conversation_id", "task_id", "run_id"
))
values = dict(payload)
if scoped:
if actor is None or trace is None:
raise RuntimeError("TRACE_ACTOR_CONTEXT_REQUIRED")
trusted = GovernanceActor.from_value(actor)
authorize_scope(trusted, trace.scope)
bound = {
"trace_id": trace.trace_id, "span_id": trace.span_id,
"parent_span_id": trace.parent_span_id, "correlation_id": trace.correlation_id,
"causation_id": trace.causation_id, "tenant_id": trace.scope.tenant_id,
"user_id": trace.scope.user_id, "project_id": trace.scope.project_id,
"conversation_id": trace.scope.conversation_id, "task_id": trace.task_id,
"claim_id": trace.claim_id, "run_id": trace.scope.run_id, "message_id": trace.message_id,
}
for name, authoritative in bound.items():
if values.get(name) not in (None, authoritative):
raise RuntimeError("TRACE_SCOPE_MISMATCH_DENY:" + name)
values[name] = authoritative
record = AgentExecutionRecord(**values)
return {"status": "ok", "record": store.append(record)}
def get_agent_observability(user_id: str | None = None, group_id: str | None = None, role: str = "user"):
records = store.query(user_id, group_id, role)
def get_agent_observability(user_id: str | None = None, group_id: str | None = None, role: str = "user", *, actor: GovernanceActor | dict[str, Any] | None = None):
records = store.query(user_id, group_id, role, actor=actor)
visible = AgentObservability()
visible.records = [AgentExecutionRecord(**record) for record in records]
return {
"status": "ok",
"records": records,
"summary": store.load().summary(user_id if role != "owner" else None),
"summary": visible.summary(None),
}
@@ -14,6 +14,11 @@ from uuid import uuid4
from .agent_observability import AgentExecutionRecord
from .agent_observability_store import AgentObservabilityStore
from .trace_tool_credential_governance import (
GovernanceActor,
TraceContext,
authorize_scope,
)
def utc_now() -> str:
@@ -36,6 +41,7 @@ class AgentRunContext:
input_tokens: int = 0
output_tokens: int = 0
status: str = "running"
trace: TraceContext | None = None
def delegate(self, agent_name: str) -> None:
if agent_name not in self.path:
@@ -59,7 +65,19 @@ class AgentObservabilityHooks:
visible_scope: str,
entry_agent: str,
execution_id: str | None = None,
*,
actor: GovernanceActor | dict[str, Any] | None = None,
trace: TraceContext | None = None,
) -> dict[str, Any]:
if execution_id is not None and execution_id in self.active:
raise RuntimeError("OBSERVABILITY_EXECUTION_CONFLICT")
if trace is not None:
if actor is None:
raise RuntimeError("TRACE_ACTOR_CONTEXT_REQUIRED")
trusted_actor = GovernanceActor.from_value(actor)
authorize_scope(trusted_actor, trace.scope)
if trace.scope.user_id is not None and trace.scope.user_id != user_id:
raise RuntimeError("TRACE_SCOPE_MISMATCH_DENY:user_id")
run = AgentRunContext(
execution_id=execution_id or f"exec_{uuid4().hex[:12]}",
user_id=user_id,
@@ -67,6 +85,7 @@ class AgentObservabilityHooks:
entry_agent=entry_agent,
path=[entry_agent],
used_hermes=entry_agent.lower() == "hermes",
trace=trace,
)
self.active[run.execution_id] = run
return {"status": "started", "execution_id": run.execution_id, "run": run.__dict__}
@@ -106,6 +125,18 @@ class AgentObservabilityHooks:
status=status,
started_at=run.started_at,
ended_at=ended_at,
trace_id=run.trace.trace_id if run.trace else None,
span_id=run.trace.span_id if run.trace else None,
parent_span_id=run.trace.parent_span_id if run.trace else None,
correlation_id=run.trace.correlation_id if run.trace else None,
causation_id=run.trace.causation_id if run.trace else None,
tenant_id=run.trace.scope.tenant_id if run.trace else None,
project_id=run.trace.scope.project_id if run.trace else None,
conversation_id=run.trace.scope.conversation_id if run.trace else None,
task_id=run.trace.task_id if run.trace else None,
claim_id=run.trace.claim_id if run.trace else None,
run_id=run.trace.scope.run_id if run.trace else None,
message_id=run.trace.message_id if run.trace else None,
)
return {"status": "stored", "record": self.store.append(record)}
@@ -12,6 +12,12 @@ from pathlib import Path
from typing import Any
from .agent_observability import AgentExecutionRecord, AgentObservability
from .trace_tool_credential_governance import (
GovernanceActor,
GovernanceError,
GovernanceScope,
authorize_scope,
)
class AgentObservabilityStore:
@@ -23,6 +29,11 @@ class AgentObservabilityStore:
def append(self, record: AgentExecutionRecord) -> dict[str, Any]:
path = self.records_dir / f"{record.execution_id}.json"
data = record.to_dict()
if path.exists():
existing = json.loads(path.read_text(encoding="utf-8"))
if existing == data:
return existing
raise RuntimeError("OBSERVABILITY_EXECUTION_CONFLICT")
path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")
return data
@@ -33,8 +44,36 @@ class AgentObservabilityStore:
observability.records.append(AgentExecutionRecord(**raw))
return observability
def query(self, user_id: str | None = None, group_id: str | None = None, role: str = "user") -> list[dict[str, Any]]:
def query(self, user_id: str | None = None, group_id: str | None = None, role: str = "user", *, actor: GovernanceActor | dict[str, Any] | None = None) -> list[dict[str, Any]]:
records = [record.to_dict() for record in self.load().records]
scoped = [record for record in records if record.get("tenant_id") is not None]
legacy = [record for record in records if record.get("tenant_id") is None]
if actor is not None:
trusted = GovernanceActor.from_value(actor)
visible: list[dict[str, Any]] = []
for record in scoped:
try:
authorize_scope(
trusted,
GovernanceScope(
"tenant",
tenant_id=record.get("tenant_id"),
user_id=record.get("user_id"),
project_id=record.get("project_id"),
conversation_id=record.get("conversation_id"),
run_id=record.get("run_id"),
),
)
except GovernanceError:
continue
visible.append(record)
if trusted.privileged:
visible.extend(legacy)
else:
visible.extend(record for record in legacy if record.get("user_id") == trusted.scope.user_id)
return visible
# Caller-supplied role remains a legacy compatibility path only.
records = legacy
if role == "owner":
return records
if role == "group_admin" and group_id:
@@ -0,0 +1,722 @@
"""Owner-controlled CAS-bound execution-envelope rebind for Online Agent Tasks.
This service exists for a narrow recovery case: a Task was projected/claimed with a read-only
source_capture envelope before its already-registered deterministic plan became authoritative.
It never creates Tasks/Jobs/branches and never accepts arbitrary metadata patches. It only replaces
metadata.execution + the derived execution_projection for the same governed binding, after an
official NEEDS_REWORK transition has returned the Task to QUEUED with no active lease.
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import hmac
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
import os
from pathlib import Path
import re
import sqlite3
from typing import Any, Callable, Mapping, Sequence
import urllib.error
import urllib.request
from aios_online_agent_execution import (
GovernedExecutionBinding,
validate_execution_envelope,
)
TASK_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{2,179}$")
KEY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{7,159}$")
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
MAX_BODY = 64 * 1024
AUTHORIZATION_ID_ENV = "AIOS_AGENT_TASK_EXECUTION_REBIND_AUTHORIZATION_ID"
class ExecutionRebindError(RuntimeError):
def __init__(self, code: str, status: int = 409) -> None:
super().__init__(code)
self.code = code
self.status = status
def _canonical(value: object) -> bytes:
return json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=False
).encode("utf-8")
def _sha(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _require_string(
value: object,
code: str,
pattern: re.Pattern[str] | None = None,
) -> str:
if not isinstance(value, str) or not value.strip():
raise ExecutionRebindError(code, 400)
value = value.strip()
if pattern is not None and pattern.fullmatch(value) is None:
raise ExecutionRebindError(code, 400)
return value
def _required_authorization_id() -> str:
value = os.environ.get(AUTHORIZATION_ID_ENV, "").strip()
if not value:
raise RuntimeError("TASK_EXECUTION_REBIND_AUTHORIZATION_ID_MISSING")
if KEY_RE.fullmatch(value) is None:
raise RuntimeError("TASK_EXECUTION_REBIND_AUTHORIZATION_ID_INVALID")
return value
@dataclass(frozen=True)
class ExecutionRebindRequest:
expected_status: str
expected_revision: int
expected_attempt: int
previous_verification_sha256: str
expected_old_execution_digest: str
expected_new_execution_digest: str
authorization_id: str
idempotency_key: str
reason: str
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "ExecutionRebindRequest":
allowed = {
"expected_status",
"expected_revision",
"expected_attempt",
"previous_verification_sha256",
"expected_old_execution_digest",
"expected_new_execution_digest",
"authorization_id",
"idempotency_key",
"reason",
}
if set(value) != allowed:
raise ExecutionRebindError("EXECUTION_REBIND_FIELDS_INVALID", 400)
status = _require_string(value["expected_status"], "EXPECTED_STATUS_INVALID")
if status != "QUEUED":
raise ExecutionRebindError("EXPECTED_STATUS_INVALID", 400)
revision = value["expected_revision"]
attempt = value["expected_attempt"]
if (
not isinstance(revision, int)
or isinstance(revision, bool)
or revision < 0
or not isinstance(attempt, int)
or isinstance(attempt, bool)
or attempt < 1
):
raise ExecutionRebindError("EXECUTION_REBIND_INTEGER_INVALID", 400)
verification = _require_string(
value["previous_verification_sha256"], "VERIFICATION_SHA256_INVALID"
)
old_digest = _require_string(
value["expected_old_execution_digest"], "OLD_EXECUTION_DIGEST_INVALID"
)
new_digest = _require_string(
value["expected_new_execution_digest"], "NEW_EXECUTION_DIGEST_INVALID"
)
for digest, code in (
(verification, "VERIFICATION_SHA256_INVALID"),
(old_digest, "OLD_EXECUTION_DIGEST_INVALID"),
(new_digest, "NEW_EXECUTION_DIGEST_INVALID"),
):
if SHA_RE.fullmatch(digest) is None:
raise ExecutionRebindError(code, 400)
if old_digest == new_digest:
raise ExecutionRebindError("EXECUTION_REBIND_DIGEST_UNCHANGED", 400)
authorization = _require_string(
value["authorization_id"], "AUTHORIZATION_ID_INVALID", KEY_RE
)
key = _require_string(value["idempotency_key"], "IDEMPOTENCY_KEY_INVALID", KEY_RE)
reason = _require_string(value["reason"], "EXECUTION_REBIND_REASON_INVALID")
if len(reason) > 512:
raise ExecutionRebindError("EXECUTION_REBIND_REASON_INVALID", 400)
return cls(
status,
revision,
attempt,
verification,
old_digest,
new_digest,
authorization,
key,
reason,
)
def public(self) -> dict[str, Any]:
return {
"expected_status": self.expected_status,
"expected_revision": self.expected_revision,
"expected_attempt": self.expected_attempt,
"previous_verification_sha256": self.previous_verification_sha256,
"expected_old_execution_digest": self.expected_old_execution_digest,
"expected_new_execution_digest": self.expected_new_execution_digest,
"authorization_id": self.authorization_id,
"idempotency_key": self.idempotency_key,
"reason": self.reason,
}
class CanonicalBridgeExecutionResolver:
"""Resolve the already-registered plan from the canonical governed Bridge."""
def __init__(self, base_url: str, control_token: str) -> None:
self.base_url = base_url.rstrip("/")
self.control_token = control_token
def __call__(
self,
task_id: str,
required_capabilities: Sequence[str],
) -> dict[str, Any]:
body = json.dumps(
{
"task_id": task_id,
"required_capabilities": list(required_capabilities),
},
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
request = urllib.request.Request(
self.base_url + "/v2/execution-envelope",
method="POST",
data=body,
headers={
"Authorization": "Bearer " + self.control_token,
"Content-Type": "application/json",
"Accept": "application/json",
},
)
try:
with urllib.request.urlopen(request, timeout=15) as response:
raw = response.read()
status = response.status
except urllib.error.HTTPError as exc:
raw = exc.read()
status = exc.code
try:
value = json.loads(raw.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ExecutionRebindError("CANONICAL_EXECUTION_RESPONSE_INVALID", 502) from exc
if not isinstance(value, Mapping):
raise ExecutionRebindError("CANONICAL_EXECUTION_RESPONSE_INVALID", 502)
if status != 200:
raise ExecutionRebindError(
str(value.get("error") or "CANONICAL_EXECUTION_RESOLUTION_FAILED"),
502 if status >= 500 else 409,
)
return dict(value)
class ExecutionRebindStore:
"""Official owner-controlled mutation with CAS, plan resolution and durable audit."""
def __init__(
self,
db_path: Path,
resolver: Callable[[str, Sequence[str]], Mapping[str, Any]],
) -> None:
self.db_path = Path(db_path)
self.resolver = resolver
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.db_path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@staticmethod
def _columns(
connection: sqlite3.Connection,
table: str,
) -> dict[str, sqlite3.Row]:
return {
str(row["name"]): row
for row in connection.execute(f'PRAGMA table_info("{table}")')
}
@staticmethod
def _json_value(raw: object, code: str) -> dict[str, Any]:
if isinstance(raw, Mapping):
return dict(raw)
if not isinstance(raw, str):
raise ExecutionRebindError(code, 409)
try:
value = json.loads(raw)
except json.JSONDecodeError as exc:
raise ExecutionRebindError(code, 409) from exc
if not isinstance(value, Mapping):
raise ExecutionRebindError(code, 409)
return dict(value)
@staticmethod
def _verification_hash(raw: object) -> str:
if raw in (None, ""):
value: object = {}
elif isinstance(raw, str):
try:
value = json.loads(raw)
except json.JSONDecodeError as exc:
raise ExecutionRebindError("TASK_VERIFICATION_INVALID", 409) from exc
else:
value = raw
return _sha(value)
@staticmethod
def _metadata_column(columns: Mapping[str, sqlite3.Row]) -> str:
for name in ("metadata_json", "metadata"):
if name in columns:
return name
raise ExecutionRebindError("TASK_METADATA_SCHEMA_UNSUPPORTED", 503)
@staticmethod
def _required_capabilities(
task: sqlite3.Row,
columns: Mapping[str, sqlite3.Row],
old_execution: Mapping[str, Any],
) -> list[str]:
for field in ("required_capabilities_json", "required_capabilities"):
if field not in columns:
continue
raw = task[field]
if raw in (None, ""):
continue
if isinstance(raw, str):
try:
value = json.loads(raw)
except json.JSONDecodeError:
value = [item for item in raw.split(",") if item.strip()]
else:
value = raw
if isinstance(value, list) and all(isinstance(item, str) for item in value):
return [item.strip() for item in value if item.strip()]
value = old_execution.get("capability_scope")
if isinstance(value, list) and all(isinstance(item, str) for item in value):
return [item for item in value if item]
raise ExecutionRebindError("TASK_CAPABILITIES_UNAVAILABLE", 503)
@staticmethod
def _event_table(
connection: sqlite3.Connection,
) -> tuple[str, dict[str, sqlite3.Row]]:
tables = {
row["name"]
for row in connection.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
)
}
for name in ("task_events", "events"):
if name not in tables:
continue
columns = ExecutionRebindStore._columns(connection, name)
if "task_id" in columns and ("event_type" in columns or "type" in columns):
return name, columns
raise ExecutionRebindError("TASK_EVENT_TABLE_UNSUPPORTED", 503)
@staticmethod
def _insert_event(
connection: sqlite3.Connection,
task_id: str,
payload: Mapping[str, Any],
) -> None:
table, columns = ExecutionRebindStore._event_table(connection)
values: dict[str, Any] = {"task_id": task_id}
if "event_type" in columns:
values["event_type"] = "TASK_EXECUTION_REBIND_AUTHORIZED"
else:
values["type"] = "TASK_EXECUTION_REBIND_AUTHORIZED"
payload_text = json.dumps(payload, sort_keys=True, separators=(",", ":"))
if "payload_json" in columns:
values["payload_json"] = payload_text
elif "payload" in columns:
values["payload"] = payload_text
else:
raise ExecutionRebindError("TASK_EVENT_PAYLOAD_UNSUPPORTED", 503)
if "created_at" in columns:
values["created_at"] = _now()
if "agent_id" in columns:
values["agent_id"] = "owner-control"
if "actor_id" in columns:
values["actor_id"] = "owner-control"
names = list(values)
quoted = ",".join('"' + name + '"' for name in names)
placeholders = ",".join("?" for _ in names)
connection.execute(
f'INSERT INTO "{table}" ({quoted}) VALUES ({placeholders})',
tuple(values[name] for name in names),
)
@staticmethod
def _ensure_rebind_table(connection: sqlite3.Connection) -> None:
connection.execute(
"""
CREATE TABLE IF NOT EXISTS task_execution_rebinds (
task_id TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
request_sha256 TEXT NOT NULL,
authorization_id TEXT NOT NULL,
previous_verification_sha256 TEXT NOT NULL,
old_execution_digest TEXT NOT NULL,
new_execution_digest TEXT NOT NULL,
expected_revision INTEGER NOT NULL,
applied_revision INTEGER NOT NULL,
attempt INTEGER NOT NULL,
reason TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY(task_id,idempotency_key)
)
"""
)
def _load_and_validate(
self,
connection: sqlite3.Connection,
task_id: str,
request: ExecutionRebindRequest,
) -> tuple[sqlite3.Row, dict[str, sqlite3.Row], str, dict[str, Any], dict[str, Any]]:
columns = self._columns(connection, "tasks")
required = {
"task_id",
"status",
"attempt",
"revision",
"claimed_by",
"lease_token_hash",
"lease_expires_at",
"verification_json",
}
if not required.issubset(columns):
raise ExecutionRebindError("TASK_SCHEMA_EXECUTION_REBIND_UNSUPPORTED", 503)
metadata_column = self._metadata_column(columns)
task = connection.execute(
"SELECT * FROM tasks WHERE task_id=?",
(task_id,),
).fetchone()
if task is None:
raise ExecutionRebindError("TASK_NOT_FOUND", 404)
if (
str(task["status"]) != request.expected_status
or int(task["revision"]) != request.expected_revision
or int(task["attempt"]) != request.expected_attempt
):
raise ExecutionRebindError("EXECUTION_REBIND_CAS_MISMATCH", 409)
if task["lease_token_hash"] not in (None, "") or task["lease_expires_at"] not in (None, ""):
raise ExecutionRebindError("EXECUTION_REBIND_ACTIVE_LEASE_DENY", 409)
verification_sha = self._verification_hash(task["verification_json"])
if verification_sha != request.previous_verification_sha256:
raise ExecutionRebindError("EXECUTION_REBIND_VERIFICATION_DRIFT", 409)
metadata = self._json_value(
task[metadata_column],
"TASK_METADATA_INVALID",
)
execution = metadata.get("execution")
if not isinstance(execution, Mapping):
raise ExecutionRebindError("TASK_EXECUTION_METADATA_MISSING", 409)
old_execution = validate_execution_envelope(dict(execution))
if old_execution["execution_digest"] != request.expected_old_execution_digest:
raise ExecutionRebindError("EXECUTION_REBIND_OLD_DIGEST_DRIFT", 409)
if old_execution.get("action") != "source_capture":
raise ExecutionRebindError("EXECUTION_REBIND_SOURCE_CAPTURE_REQUIRED", 409)
projection = metadata.get("execution_projection")
if isinstance(projection, Mapping):
projected_digest = str(projection.get("execution_digest") or "")
if projected_digest and projected_digest != request.expected_old_execution_digest:
raise ExecutionRebindError("EXECUTION_REBIND_PROJECTION_DRIFT", 409)
return task, columns, metadata_column, metadata, old_execution
def rebind(
self,
task_id: str,
request: ExecutionRebindRequest,
) -> dict[str, Any]:
task_id = _require_string(task_id, "TASK_ID_INVALID", TASK_RE)
required_authorization = _required_authorization_id()
if request.authorization_id != required_authorization:
raise ExecutionRebindError("EXECUTION_REBIND_AUTHORIZATION_ID_MISMATCH", 403)
request_sha = _sha(request.public())
# Idempotent replay must win before CAS validation because a successful
# rebind necessarily increments the Task revision and changes the persisted envelope.
with self._connect() as connection:
self._ensure_rebind_table(connection)
replay = connection.execute(
"SELECT * FROM task_execution_rebinds WHERE task_id=? AND idempotency_key=?",
(task_id, request.idempotency_key),
).fetchone()
if replay is not None:
if replay["request_sha256"] != request_sha:
raise ExecutionRebindError("EXECUTION_REBIND_IDEMPOTENCY_CONFLICT", 409)
connection.commit()
return {
"task_id": task_id,
"status": "QUEUED",
"attempt": int(replay["attempt"]),
"revision": int(replay["applied_revision"]),
"old_execution_digest": replay["old_execution_digest"],
"new_execution_digest": replay["new_execution_digest"],
"authorization_id": replay["authorization_id"],
"idempotent": True,
}
# Read/CAS snapshot first. Canonical Bridge resolution happens outside the DB write lock.
with self._connect() as connection:
task, columns, _, _, old_execution = self._load_and_validate(
connection, task_id, request
)
capabilities = self._required_capabilities(task, columns, old_execution)
binding = GovernedExecutionBinding.from_mapping(old_execution)
resolved = dict(self.resolver(task_id, capabilities))
new_execution = validate_execution_envelope(resolved, binding=binding)
if new_execution.get("action") != "deterministic":
raise ExecutionRebindError("EXECUTION_REBIND_DETERMINISTIC_PLAN_REQUIRED", 409)
if new_execution["execution_digest"] != request.expected_new_execution_digest:
raise ExecutionRebindError("EXECUTION_REBIND_NEW_DIGEST_DRIFT", 409)
if new_execution["execution_digest"] == old_execution["execution_digest"]:
raise ExecutionRebindError("EXECUTION_REBIND_DIGEST_UNCHANGED", 409)
# Re-read under BEGIN IMMEDIATE and apply only if the exact snapshot is unchanged.
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
self._ensure_rebind_table(connection)
replay = connection.execute(
"SELECT * FROM task_execution_rebinds WHERE task_id=? AND idempotency_key=?",
(task_id, request.idempotency_key),
).fetchone()
if replay is not None:
if replay["request_sha256"] != request_sha:
raise ExecutionRebindError("EXECUTION_REBIND_IDEMPOTENCY_CONFLICT", 409)
connection.commit()
return {
"task_id": task_id,
"status": "QUEUED",
"attempt": int(replay["attempt"]),
"revision": int(replay["applied_revision"]),
"old_execution_digest": replay["old_execution_digest"],
"new_execution_digest": replay["new_execution_digest"],
"authorization_id": replay["authorization_id"],
"idempotent": True,
}
task, columns, metadata_column, metadata, current_old = self._load_and_validate(
connection, task_id, request
)
if current_old["execution_digest"] != old_execution["execution_digest"]:
raise ExecutionRebindError("EXECUTION_REBIND_OLD_DIGEST_DRIFT", 409)
new_metadata = dict(metadata)
new_metadata["execution"] = new_execution
new_metadata["execution_projection"] = {
"schema": "aios.online-agent.task-projection.v1",
"task_id": binding.task_id,
"job_id": binding.job_id,
"execution_digest": new_execution["execution_digest"],
}
applied_revision = request.expected_revision + 1
previous_claimed_by = task["claimed_by"]
cursor = connection.execute(
f"""
UPDATE tasks
SET "{metadata_column}"=?, revision=?, claimed_by=NULL,
lease_token_hash=NULL, lease_expires_at=NULL
WHERE task_id=? AND status=? AND revision=? AND attempt=?
AND claimed_by IS ? AND lease_token_hash IS NULL AND lease_expires_at IS NULL
""",
(
json.dumps(new_metadata, sort_keys=True, separators=(",", ":")),
applied_revision,
task_id,
request.expected_status,
request.expected_revision,
request.expected_attempt,
previous_claimed_by,
),
)
if cursor.rowcount != 1:
raise ExecutionRebindError("EXECUTION_REBIND_CAS_MISMATCH", 409)
event_payload = {
"schema": "aios.task-execution-rebind.event.v1",
"task_id": task_id,
"job_id": binding.job_id,
"authorization_id": request.authorization_id,
"idempotency_key": request.idempotency_key,
"reason": request.reason,
"previous_verification_sha256": request.previous_verification_sha256,
"old_execution_digest": old_execution["execution_digest"],
"new_execution_digest": new_execution["execution_digest"],
"old_action": old_execution.get("action"),
"new_action": new_execution.get("action"),
"previous_revision": request.expected_revision,
"applied_revision": applied_revision,
"attempt": request.expected_attempt,
"request_sha256": request_sha,
"previous_claimed_by": previous_claimed_by,
"stale_claim_normalized": previous_claimed_by not in (None, ""),
}
self._insert_event(connection, task_id, event_payload)
connection.execute(
"""
INSERT INTO task_execution_rebinds(
task_id,idempotency_key,request_sha256,authorization_id,
previous_verification_sha256,old_execution_digest,new_execution_digest,
expected_revision,applied_revision,attempt,reason,created_at
) VALUES (?,?,?,?,?,?,?,?,?,?,?,?)
""",
(
task_id,
request.idempotency_key,
request_sha,
request.authorization_id,
request.previous_verification_sha256,
old_execution["execution_digest"],
new_execution["execution_digest"],
request.expected_revision,
applied_revision,
request.expected_attempt,
request.reason,
_now(),
),
)
connection.commit()
return {
"task_id": task_id,
"status": "QUEUED",
"attempt": request.expected_attempt,
"revision": applied_revision,
"old_execution_digest": old_execution["execution_digest"],
"new_execution_digest": new_execution["execution_digest"],
"authorization_id": request.authorization_id,
"idempotent": False,
}
class ExecutionRebindHandler(BaseHTTPRequestHandler):
store: ExecutionRebindStore | None = None
control_token: str = ""
def log_message(self, *_args: Any) -> None:
return
def _json(self, value: Mapping[str, Any], status: int) -> None:
data = json.dumps(
dict(value), sort_keys=True, separators=(",", ":")
).encode("utf-8")
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def _authorized(self) -> bool:
header = self.headers.get("Authorization", "")
return (
header.startswith("Bearer ")
and bool(self.control_token)
and hmac.compare_digest(header[7:], self.control_token)
)
def do_GET(self) -> None:
if self.path == "/health":
self._json(
{"status": "ok", "service": "aios-agent-task-execution-rebind"},
200,
)
else:
self._json({"error": "NOT_FOUND"}, 404)
def do_POST(self) -> None:
match = re.fullmatch(r"/v1/tasks/([^/]+)/execution-rebind", self.path)
if match is None:
self._json({"error": "NOT_FOUND"}, 404)
return
if not self._authorized():
self._json({"error": "UNAUTHORIZED"}, 401)
return
try:
length = int(self.headers.get("Content-Length", "0"))
except ValueError:
self._json({"error": "CONTENT_LENGTH_INVALID"}, 400)
return
if length < 1 or length > MAX_BODY:
self._json({"error": "REQUEST_SIZE_INVALID"}, 413)
return
try:
value = json.loads(self.rfile.read(length).decode("utf-8"))
if not isinstance(value, Mapping):
raise ValueError
request = ExecutionRebindRequest.from_mapping(value)
if self.store is None:
raise ExecutionRebindError("RUNTIME_UNAVAILABLE", 503)
result = self.store.rebind(match.group(1), request)
self._json(result, 200 if result["idempotent"] else 201)
except ExecutionRebindError as exc:
self._json({"error": exc.code}, exc.status)
except (UnicodeDecodeError, json.JSONDecodeError, ValueError):
self._json({"error": "REQUEST_INVALID"}, 400)
def load_control_token(path: Path) -> str:
if path.is_symlink() or not path.is_file():
raise RuntimeError("TASK_EXECUTION_REBIND_CONTROL_CREDENTIAL_UNAVAILABLE")
value = path.read_text(encoding="utf-8").strip()
if len(value) < 32:
raise RuntimeError("TASK_EXECUTION_REBIND_CONTROL_CREDENTIAL_INVALID")
return value
def serve() -> None:
db = Path(os.environ.get("AIOS_AGENT_TASK_DB", "/var/lib/aios-agent-task/state.sqlite3"))
_required_authorization_id()
token_path = Path(
os.environ.get(
"AIOS_AGENT_TASK_CONTROL_TOKEN_FILE",
str(
Path(os.environ.get("CREDENTIALS_DIRECTORY", "/run/credentials"))
/ "agent_task_control_token"
),
)
)
token = load_control_token(token_path)
bridge = os.environ.get(
"AIOS_GOVERNED_WRITE_API_URL",
"http://127.0.0.1:18892",
)
host = os.environ.get("AIOS_TASK_EXECUTION_REBIND_HOST", "127.0.0.1")
port = int(os.environ.get("AIOS_TASK_EXECUTION_REBIND_PORT", "18942"))
ExecutionRebindHandler.store = ExecutionRebindStore(
db,
CanonicalBridgeExecutionResolver(bridge, token),
)
ExecutionRebindHandler.control_token = token
ThreadingHTTPServer((host, port), ExecutionRebindHandler).serve_forever()
if __name__ == "__main__":
serve()
@@ -0,0 +1,427 @@
"""Owner-controlled, CAS-bound retry extension for exhausted Online Agent Tasks."""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import hmac
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
import os
from pathlib import Path
import re
import sqlite3
from typing import Any, Mapping
TASK_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{2,179}$")
KEY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{7,159}$")
MAX_BODY = 64 * 1024
AUTHORIZE_EXTENSION_ID_ENV = "AIOS_AGENT_TASK_AUTHORIZATION_ID"
class RetryExtensionError(RuntimeError):
def __init__(self, code: str, status: int = 409) -> None:
super().__init__(code)
self.code = code
self.status = status
def _canonical(value: object) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def _sha(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _required_authorization_id() -> str:
value = os.environ.get(AUTHORIZE_EXTENSION_ID_ENV, "").strip()
if not value:
raise RuntimeError("TASK_RETRY_AUTHORIZATION_ID_MISSING")
if KEY_RE.fullmatch(value) is None:
raise RuntimeError("TASK_RETRY_AUTHORIZATION_ID_INVALID")
return value
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _require_string(value: object, code: str, pattern: re.Pattern[str] | None = None) -> str:
if not isinstance(value, str) or not value.strip():
raise RetryExtensionError(code, 400)
value = value.strip()
if pattern is not None and pattern.fullmatch(value) is None:
raise RetryExtensionError(code, 400)
return value
@dataclass(frozen=True)
class RetryExtensionRequest:
expected_status: str
expected_revision: int
expected_attempt: int
expected_max_attempts: int
next_max_attempts: int
previous_verification_sha256: str
authorization_id: str
idempotency_key: str
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "RetryExtensionRequest":
allowed = {
"expected_status", "expected_revision", "expected_attempt", "expected_max_attempts",
"next_max_attempts", "previous_verification_sha256", "authorization_id", "idempotency_key",
}
if set(value) != allowed:
raise RetryExtensionError("RETRY_EXTENSION_FIELDS_INVALID", 400)
status = _require_string(value["expected_status"], "EXPECTED_STATUS_INVALID")
if status != "FAILED":
raise RetryExtensionError("EXPECTED_STATUS_INVALID", 400)
ints: dict[str, int] = {}
for field in ("expected_revision", "expected_attempt", "expected_max_attempts", "next_max_attempts"):
item = value[field]
if not isinstance(item, int) or isinstance(item, bool) or item < 0:
raise RetryExtensionError("RETRY_EXTENSION_INTEGER_INVALID", 400)
ints[field] = item
if ints["expected_attempt"] != ints["expected_max_attempts"]:
raise RetryExtensionError("RETRY_EXTENSION_NOT_EXHAUSTED", 409)
if ints["next_max_attempts"] != ints["expected_max_attempts"] + 1:
raise RetryExtensionError("RETRY_EXTENSION_MUST_BE_BOUNDED_ONE_STEP", 400)
if ints["next_max_attempts"] > 50:
raise RetryExtensionError("RETRY_EXTENSION_LIMIT_EXCEEDED", 400)
verification = _require_string(value["previous_verification_sha256"], "VERIFICATION_SHA256_INVALID")
if re.fullmatch(r"[0-9a-f]{64}", verification) is None:
raise RetryExtensionError("VERIFICATION_SHA256_INVALID", 400)
authorization = _require_string(value["authorization_id"], "AUTHORIZATION_ID_INVALID", KEY_RE)
key = _require_string(value["idempotency_key"], "IDEMPOTENCY_KEY_INVALID", KEY_RE)
return cls(
status,
ints["expected_revision"],
ints["expected_attempt"],
ints["expected_max_attempts"],
ints["next_max_attempts"],
verification,
authorization,
key,
)
def public(self) -> dict[str, Any]:
return {
"expected_status": self.expected_status,
"expected_revision": self.expected_revision,
"expected_attempt": self.expected_attempt,
"expected_max_attempts": self.expected_max_attempts,
"next_max_attempts": self.next_max_attempts,
"previous_verification_sha256": self.previous_verification_sha256,
"authorization_id": self.authorization_id,
"idempotency_key": self.idempotency_key,
}
class RetryExtensionStore:
"""Owns the official mutation. Direct/manual DB edits remain forbidden."""
def __init__(self, db_path: Path) -> None:
self.db_path = Path(db_path)
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.db_path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@staticmethod
def _columns(connection: sqlite3.Connection, table: str) -> dict[str, sqlite3.Row]:
return {str(row["name"]): row for row in connection.execute(f'PRAGMA table_info("{table}")')}
@staticmethod
def _event_table(connection: sqlite3.Connection) -> tuple[str, dict[str, sqlite3.Row]]:
tables = {
row["name"]
for row in connection.execute("SELECT name FROM sqlite_master WHERE type='table'")
}
for name in ("task_events", "events"):
if name not in tables:
continue
columns = RetryExtensionStore._columns(connection, name)
if "task_id" in columns and ("event_type" in columns or "type" in columns):
return name, columns
raise RetryExtensionError("TASK_EVENT_TABLE_UNSUPPORTED", 503)
@staticmethod
def _verification_hash(raw: object) -> str:
if raw in (None, ""):
value: object = {}
elif isinstance(raw, str):
try:
value = json.loads(raw)
except json.JSONDecodeError as exc:
raise RetryExtensionError("TASK_VERIFICATION_INVALID", 409) from exc
else:
value = raw
return _sha(value)
@staticmethod
def _ensure_extension_table(connection: sqlite3.Connection) -> None:
connection.execute(
"""
CREATE TABLE IF NOT EXISTS task_retry_extensions (
task_id TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
request_sha256 TEXT NOT NULL,
authorization_id TEXT NOT NULL,
previous_verification_sha256 TEXT NOT NULL,
previous_max_attempts INTEGER NOT NULL,
next_max_attempts INTEGER NOT NULL,
expected_revision INTEGER NOT NULL,
applied_revision INTEGER NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY(task_id, idempotency_key)
)
"""
)
@staticmethod
def _insert_event(
connection: sqlite3.Connection,
task_id: str,
payload: Mapping[str, Any],
) -> None:
table, columns = RetryExtensionStore._event_table(connection)
values: dict[str, Any] = {"task_id": task_id}
if "event_type" in columns:
values["event_type"] = "TASK_RETRY_EXTENSION_AUTHORIZED"
else:
values["type"] = "TASK_RETRY_EXTENSION_AUTHORIZED"
payload_text = json.dumps(payload, sort_keys=True, separators=(",", ":"))
if "payload_json" in columns:
values["payload_json"] = payload_text
elif "payload" in columns:
values["payload"] = payload_text
else:
raise RetryExtensionError("TASK_EVENT_PAYLOAD_UNSUPPORTED", 503)
if "created_at" in columns:
values["created_at"] = _now()
if "agent_id" in columns:
values["agent_id"] = "owner-control"
if "actor_id" in columns:
values["actor_id"] = "owner-control"
names = list(values)
sql = f'INSERT INTO "{table}" ({",".join(f"""\"{name}\"""" for name in names)}) VALUES ({",".join("?" for _ in names)})'
connection.execute(sql, tuple(values[name] for name in names))
def extend(self, task_id: str, request: RetryExtensionRequest) -> dict[str, Any]:
task_id = _require_string(task_id, "TASK_ID_INVALID", TASK_RE)
required_authorization_id = _required_authorization_id()
if request.authorization_id != required_authorization_id:
raise RetryExtensionError("RETRY_EXTENSION_AUTHORIZATION_ID_MISMATCH", 403)
request_payload = request.public()
request_sha = _sha(request_payload)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
self._ensure_extension_table(connection)
replay = connection.execute(
"SELECT * FROM task_retry_extensions WHERE task_id=? AND idempotency_key=?",
(task_id, request.idempotency_key),
).fetchone()
if replay is not None:
if replay["request_sha256"] != request_sha:
raise RetryExtensionError("RETRY_EXTENSION_IDEMPOTENCY_CONFLICT", 409)
connection.commit()
return {
"task_id": task_id,
"status": "QUEUED",
"attempt": request.expected_attempt,
"max_attempts": int(replay["next_max_attempts"]),
"revision": int(replay["applied_revision"]),
"idempotent": True,
"authorization_id": replay["authorization_id"],
"previous_verification_sha256": replay["previous_verification_sha256"],
}
columns = self._columns(connection, "tasks")
required = {
"task_id", "status", "attempt", "max_attempts", "revision", "claimed_by",
"lease_token_hash", "lease_expires_at", "verification_json",
}
if not required.issubset(columns):
raise RetryExtensionError("TASK_SCHEMA_RETRY_EXTENSION_UNSUPPORTED", 503)
task = connection.execute("SELECT * FROM tasks WHERE task_id=?", (task_id,)).fetchone()
if task is None:
raise RetryExtensionError("TASK_NOT_FOUND", 404)
actual_verification_sha = self._verification_hash(task["verification_json"])
if actual_verification_sha != request.previous_verification_sha256:
raise RetryExtensionError("RETRY_EXTENSION_VERIFICATION_DRIFT", 409)
if (
task["status"] != request.expected_status
or int(task["revision"]) != request.expected_revision
or int(task["attempt"]) != request.expected_attempt
or int(task["max_attempts"]) != request.expected_max_attempts
):
raise RetryExtensionError("RETRY_EXTENSION_CAS_MISMATCH", 409)
for lease_field in ("lease_token_hash", "lease_expires_at"):
if lease_field in columns and task[lease_field] not in (None, ""):
raise RetryExtensionError("RETRY_EXTENSION_ACTIVE_LEASE_DENY", 409)
previous_claimed_by = task["claimed_by"]
applied_revision = request.expected_revision + 1
cursor = connection.execute(
"""
UPDATE tasks
SET status='QUEUED', max_attempts=?, revision=?, claimed_by=NULL,
lease_token_hash=NULL, lease_expires_at=NULL
WHERE task_id=? AND status=? AND revision=? AND attempt=? AND max_attempts=?
AND claimed_by IS ?
AND lease_token_hash IS NULL AND lease_expires_at IS NULL
""",
(
request.next_max_attempts,
applied_revision,
task_id,
request.expected_status,
request.expected_revision,
request.expected_attempt,
request.expected_max_attempts,
previous_claimed_by,
),
)
if cursor.rowcount != 1:
raise RetryExtensionError("RETRY_EXTENSION_CAS_MISMATCH", 409)
event_payload = {
"schema": "aios.task-retry-extension.event.v1",
"task_id": task_id,
"authorization_id": request.authorization_id,
"idempotency_key": request.idempotency_key,
"previous_verification_sha256": request.previous_verification_sha256,
"previous_max_attempts": request.expected_max_attempts,
"next_max_attempts": request.next_max_attempts,
"previous_revision": request.expected_revision,
"applied_revision": applied_revision,
"request_sha256": request_sha,
"previous_claimed_by": previous_claimed_by,
"stale_terminal_claim_normalized": previous_claimed_by not in (None, ""),
}
self._insert_event(connection, task_id, event_payload)
connection.execute(
"""
INSERT INTO task_retry_extensions(
task_id,idempotency_key,request_sha256,authorization_id,previous_verification_sha256,
previous_max_attempts,next_max_attempts,expected_revision,applied_revision,created_at
) VALUES (?,?,?,?,?,?,?,?,?,?)
""",
(
task_id,
request.idempotency_key,
request_sha,
request.authorization_id,
request.previous_verification_sha256,
request.expected_max_attempts,
request.next_max_attempts,
request.expected_revision,
applied_revision,
_now(),
),
)
connection.commit()
return {
"task_id": task_id,
"status": "QUEUED",
"attempt": request.expected_attempt,
"max_attempts": request.next_max_attempts,
"revision": applied_revision,
"idempotent": False,
"authorization_id": request.authorization_id,
"previous_verification_sha256": request.previous_verification_sha256,
}
class RetryExtensionHandler(BaseHTTPRequestHandler):
store: RetryExtensionStore | None = None
control_token: str = ""
def log_message(self, *_args: Any) -> None:
return
def _json(self, value: Mapping[str, Any], status: int) -> None:
data = json.dumps(dict(value), sort_keys=True, separators=(",", ":")).encode("utf-8")
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def _authorized(self) -> bool:
header = self.headers.get("Authorization", "")
return (
header.startswith("Bearer ")
and bool(self.control_token)
and hmac.compare_digest(header[7:], self.control_token)
)
def do_GET(self) -> None:
if self.path == "/health":
self._json({"status": "ok", "service": "aios-agent-task-retry-extension"}, 200)
else:
self._json({"error": "NOT_FOUND"}, 404)
def do_POST(self) -> None:
match = re.fullmatch(r"/v1/tasks/([^/]+)/retry-extension", self.path)
if match is None:
self._json({"error": "NOT_FOUND"}, 404)
return
if not self._authorized():
self._json({"error": "UNAUTHORIZED"}, 401)
return
try:
length = int(self.headers.get("Content-Length", "0"))
except ValueError:
self._json({"error": "CONTENT_LENGTH_INVALID"}, 400)
return
if length < 1 or length > MAX_BODY:
self._json({"error": "REQUEST_SIZE_INVALID"}, 413)
return
try:
value = json.loads(self.rfile.read(length).decode("utf-8"))
if not isinstance(value, Mapping):
raise ValueError
request = RetryExtensionRequest.from_mapping(value)
if self.store is None:
raise RetryExtensionError("RUNTIME_UNAVAILABLE", 503)
result = self.store.extend(match.group(1), request)
self._json(result, 200 if result["idempotent"] else 201)
except RetryExtensionError as exc:
self._json({"error": exc.code}, exc.status)
except (UnicodeDecodeError, json.JSONDecodeError, ValueError):
self._json({"error": "REQUEST_INVALID"}, 400)
def load_control_token(path: Path) -> str:
if path.is_symlink() or not path.is_file():
raise RuntimeError("TASK_RETRY_CONTROL_CREDENTIAL_UNAVAILABLE")
value = path.read_text(encoding="utf-8").strip()
if len(value) < 32:
raise RuntimeError("TASK_RETRY_CONTROL_CREDENTIAL_INVALID")
return value
def serve() -> None:
db = Path(os.environ.get("AIOS_AGENT_TASK_DB", "/var/lib/aios-agent-task/state.sqlite3"))
_required_authorization_id()
token_path = Path(
os.environ.get(
"AIOS_AGENT_TASK_CONTROL_TOKEN_FILE",
str(Path(os.environ.get("CREDENTIALS_DIRECTORY", "/run/credentials")) / "agent_task_control_token"),
)
)
host = os.environ.get("AIOS_TASK_RETRY_HOST", "127.0.0.1")
port = int(os.environ.get("AIOS_TASK_RETRY_PORT", "18941"))
RetryExtensionHandler.store = RetryExtensionStore(db)
RetryExtensionHandler.control_token = load_control_token(token_path)
ThreadingHTTPServer((host, port), RetryExtensionHandler).serve_forever()
if __name__ == "__main__":
serve()
@@ -0,0 +1,258 @@
"""Fail-closed Authority Git side-effect boundary.
Only this module may issue the narrowly scoped ref CAS primitives used by the
durable executor. It never accepts force updates, arbitrary refs, tags, or
direct production mutations.
"""
from __future__ import annotations
from dataclasses import dataclass
import os
from pathlib import Path
import re
import subprocess
import tempfile
from typing import Mapping, Sequence
OID_RE = re.compile(r"^[0-9a-f]{40}$")
TASK_BRANCH_RE = re.compile(r"^refs/heads/agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$")
class AuthorityGitError(RuntimeError):
pass
def _oid(value: str, field: str) -> str:
if not isinstance(value, str) or not OID_RE.fullmatch(value):
raise AuthorityGitError(f"{field}_INVALID")
return value
@dataclass(frozen=True, slots=True)
class AuthorityGitExecutor:
git_dir: Path
worktree_root: Path = Path("/var/lib/aios-gitops/worktrees")
def __post_init__(self) -> None:
if not self.git_dir.is_absolute() or self.git_dir.is_symlink():
raise AuthorityGitError("AUTHORITY_GIT_DIR_INVALID")
if not self.worktree_root.is_absolute() or self.worktree_root.is_symlink():
raise AuthorityGitError("WORKTREE_ROOT_INVALID")
def _git(self, args: Sequence[str], *, check: bool = True) -> str:
result = subprocess.run(
["git", f"--git-dir={self.git_dir}", *args],
text=True,
capture_output=True,
check=False,
)
if check and result.returncode != 0:
raise AuthorityGitError("AUTHORITY_GIT_COMMAND_FAILED")
return result.stdout.strip()
def read_ref(self, ref: str) -> str | None:
if not ref.startswith("refs/") or ref.startswith(("refs/tags/", "refs/remotes/")):
raise AuthorityGitError("REF_READ_SCOPE_DENIED")
value = self._git(["rev-parse", "--verify", ref], check=False)
return value if value and OID_RE.fullmatch(value) else None
def commit_exists(self, oid: str) -> bool:
_oid(oid, "commit_oid")
result = subprocess.run(["git", f"--git-dir={self.git_dir}", "cat-file", "-e", oid + "^{commit}"], capture_output=True, check=False)
return result.returncode == 0
def ancestry_check(self, older_oid: str, newer_oid: str) -> bool:
_oid(older_oid, "older_oid")
_oid(newer_oid, "newer_oid")
result = subprocess.run(
["git", f"--git-dir={self.git_dir}", "merge-base", "--is-ancestor", older_oid, newer_oid],
capture_output=True,
check=False,
)
if result.returncode not in (0, 1):
raise AuthorityGitError("ANCESTRY_CHECK_FAILED")
return result.returncode == 0
def create_detached_worktree(
self,
base_oid: str,
worktree_name: str,
root: Path | None = None,
) -> Path:
_oid(base_oid, "base_oid")
if not worktree_name or "/" in worktree_name or "\\" in worktree_name or worktree_name in {".", ".."}:
raise AuthorityGitError("WORKTREE_NAME_INVALID")
parent = root or self.worktree_root
if not parent.is_absolute() or parent.is_symlink():
raise AuthorityGitError("WORKTREE_ROOT_INVALID")
path = parent / worktree_name
if path.exists() or path.is_symlink():
raise AuthorityGitError("WORKTREE_ALREADY_EXISTS")
parent.mkdir(parents=True, exist_ok=True)
self._git(["worktree", "add", "--detach", str(path), base_oid])
return path
def recover_failed_detached_worktree_exact(
self,
*,
path: Path,
expected_oid: str,
stale_oid: str,
target_ref: str,
) -> str:
"""Recreate one exact clean, detached, single-attempt worktree at its durable head."""
_oid(expected_oid, "expected_oid")
_oid(stale_oid, "stale_oid")
if not TASK_BRANCH_RE.fullmatch(target_ref):
raise AuthorityGitError("TASK_BRANCH_SCOPE_DENIED")
if self.read_ref(target_ref) is not None:
raise AuthorityGitError("AUTHORITY_TARGET_BRANCH_ALREADY_EXISTS")
if (
not path.is_absolute()
or path.parent != self.worktree_root
or path.name in {"", ".", ".."}
or path.is_symlink()
or not path.is_dir()
):
raise AuthorityGitError("WORKTREE_RECOVERY_PATH_INVALID")
head = subprocess.run(
["git", "-C", str(path), "rev-parse", "HEAD"],
text=True, capture_output=True, check=False,
)
if head.returncode != 0 or head.stdout.strip() != stale_oid or stale_oid == expected_oid:
raise AuthorityGitError("WORKTREE_RECOVERY_HEAD_INVALID")
symbolic = subprocess.run(
["git", "-C", str(path), "symbolic-ref", "-q", "HEAD"],
text=True, capture_output=True, check=False,
)
if symbolic.returncode != 1:
raise AuthorityGitError("WORKTREE_RECOVERY_NOT_DETACHED")
status = subprocess.run(
["git", "-C", str(path), "status", "--porcelain=v1", "--untracked-files=all"],
text=True, capture_output=True, check=False,
)
if status.returncode != 0 or status.stdout:
raise AuthorityGitError("WORKTREE_RECOVERY_NOT_CLEAN")
parents = self._git(["rev-list", "--parents", "-n", "1", stale_oid]).split()
if parents != [stale_oid, expected_oid]:
raise AuthorityGitError("WORKTREE_RECOVERY_LINEAGE_INVALID")
registered = [
line.removeprefix("worktree ")
for line in self._git(["worktree", "list", "--porcelain"]).splitlines()
if line.startswith("worktree ")
]
if registered.count(str(path)) != 1:
raise AuthorityGitError("WORKTREE_RECOVERY_REGISTRATION_INVALID")
self._git(["worktree", "remove", "--force", str(path)])
if path.exists() or path.is_symlink():
raise AuthorityGitError("WORKTREE_RECOVERY_REMOVE_FAILED")
recreated = self.create_detached_worktree(expected_oid, path.name, path.parent)
readback = subprocess.run(
["git", "-C", str(recreated), "rev-parse", "HEAD"],
text=True, capture_output=True, check=False,
)
detached = subprocess.run(
["git", "-C", str(recreated), "symbolic-ref", "-q", "HEAD"],
text=True, capture_output=True, check=False,
)
if readback.returncode != 0 or readback.stdout.strip() != expected_oid or detached.returncode != 1:
raise AuthorityGitError("WORKTREE_RECOVERY_READBACK_MISMATCH")
return stale_oid
def create_task_branch_CAS(self, branch: str, expected_base_oid: str, new_oid: str) -> str:
if not TASK_BRANCH_RE.fullmatch(branch):
raise AuthorityGitError("TASK_BRANCH_SCOPE_DENIED")
_oid(expected_base_oid, "expected_base_oid")
_oid(new_oid, "new_oid")
current = self.read_ref(branch)
if current is not None:
if current == new_oid:
return new_oid
raise AuthorityGitError("AUTHORITY_TARGET_BRANCH_ALREADY_EXISTS")
if not self.ancestry_check(expected_base_oid, new_oid):
raise AuthorityGitError("NEW_OID_NOT_DESCENDANT")
self._git(["update-ref", branch, new_oid, "0" * 40])
readback = self.read_ref(branch)
if readback != new_oid:
raise AuthorityGitError("AUTHORITY_CREATE_READBACK_MISMATCH")
return readback
def advance_task_branch_CAS(self, branch: str, expected_oid: str, new_oid: str) -> str:
if not TASK_BRANCH_RE.fullmatch(branch):
raise AuthorityGitError("TASK_BRANCH_SCOPE_DENIED")
_oid(expected_oid, "expected_oid")
_oid(new_oid, "new_oid")
current = self.read_ref(branch)
if current != expected_oid:
if current == new_oid:
return new_oid
raise AuthorityGitError("AUTHORITY_BRANCH_EXPECTATION_MISMATCH")
if not self.ancestry_check(expected_oid, new_oid):
raise AuthorityGitError("NEW_OID_NOT_DESCENDANT")
self._git(["update-ref", branch, new_oid, expected_oid])
readback = self.read_ref(branch)
if readback != new_oid:
raise AuthorityGitError("AUTHORITY_CAS_READBACK_MISMATCH")
return readback
def ff_CAS_main_after_verified_protected_merge(self, expected_oid: str, new_oid: str) -> str:
_oid(expected_oid, "expected_oid")
_oid(new_oid, "new_oid")
if not self.ancestry_check(expected_oid, new_oid):
raise AuthorityGitError("MAIN_FF_REQUIRED")
current = self.read_ref("refs/heads/main")
if current != expected_oid:
if current == new_oid:
return new_oid
raise AuthorityGitError("AUTHORITY_MAIN_EXPECTATION_MISMATCH")
self._git(["update-ref", "refs/heads/main", new_oid, expected_oid])
readback = self.read_ref("refs/heads/main")
if readback != new_oid:
raise AuthorityGitError("AUTHORITY_MAIN_CAS_READBACK_MISMATCH")
return readback
def commit_file_in_isolated_worktree(
self,
*,
base_oid: str,
files: Mapping[str, bytes],
message: str,
worktree_name: str,
) -> str:
_oid(base_oid, "base_oid")
if not worktree_name or "/" in worktree_name or "\\" in worktree_name or worktree_name in {".", ".."}:
raise AuthorityGitError("WORKTREE_NAME_INVALID")
root = self.worktree_root / worktree_name
if root.exists() or root.is_symlink():
raise AuthorityGitError("WORKTREE_ALREADY_EXISTS")
self.worktree_root.mkdir(parents=True, exist_ok=True)
self._git(["worktree", "add", "--detach", str(root), base_oid])
try:
for relative, content in files.items():
path = Path(relative)
if path.is_absolute() or ".." in path.parts or not path.parts or path.parts[0] == ".git":
raise AuthorityGitError("FILE_PATH_INVALID")
target = root / path
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(content)
result = subprocess.run(
["git", "-C", str(root), "add", "--", *files.keys()],
text=True,
capture_output=True,
check=False,
)
if result.returncode != 0:
raise AuthorityGitError("WORKTREE_ADD_FAILED")
result = subprocess.run(
["git", "-C", str(root), "-c", "user.name=aios-durable-job", "-c", "user.email=aios-durable-job@localhost", "commit", "--no-verify", "-m", message],
text=True,
capture_output=True,
check=False,
)
if result.returncode != 0:
raise AuthorityGitError("WORKTREE_COMMIT_FAILED")
return subprocess.check_output(["git", "-C", str(root), "rev-parse", "HEAD"], text=True).strip()
finally:
subprocess.run(["git", f"--git-dir={self.git_dir}", "worktree", "remove", "--force", str(root)], capture_output=True, check=False)
@@ -0,0 +1,104 @@
"""Read-only validation of recovered campaign authorization evidence."""
from __future__ import annotations
import hashlib
import json
from pathlib import Path
import stat
from typing import Any
class RecoveryEvidenceError(RuntimeError):
pass
def _digest(path: Path) -> str:
h = hashlib.sha256()
with path.open("rb") as stream:
for block in iter(lambda: stream.read(1024 * 1024), b""):
h.update(block)
return h.hexdigest()
def _secure_json(path: Path) -> dict[str, Any]:
if path.is_symlink() or not path.is_file():
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_MISSING")
info = path.stat(follow_symlinks=False)
if info.st_uid != 0 or stat.S_IMODE(info.st_mode) & 0o022:
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_PERMISSIONS")
try:
value = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_INVALID") from exc
if not isinstance(value, dict):
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_OBJECT_REQUIRED")
return value
def load_recovered_authorization(path: str | Path) -> dict[str, Any]:
"""Verify the recovery receipt, sidecar, and all referenced immutable files."""
receipt_path = Path(path)
receipt = _secure_json(receipt_path)
sidecar = receipt_path.with_suffix(receipt_path.suffix + ".sha256")
if sidecar.is_symlink() or not sidecar.is_file():
raise RecoveryEvidenceError("RECOVERY_DIGEST_SIDECAR_MISSING")
expected = sidecar.read_text(encoding="ascii").split()[0]
actual = _digest(receipt_path)
if expected != actual:
raise RecoveryEvidenceError("RECOVERY_DIGEST_MISMATCH")
required = {
"schema",
"campaign_id",
"original_event_id",
"original_authorization_digest",
"signed_grant_id",
"signed_grant_digest",
"task_id",
"repository",
"original_base_oid",
"branch",
"expected_branch_oid",
"recovered_from_existing_evidence",
"new_owner_interaction",
"recovery_state",
"production_side_effect_executed",
"global_writes_enabled",
"grant_consumed",
"owner_authorization_consumed",
"original_receipts_immutable",
"recovered_at",
"evidence_files",
}
if set(receipt) != required or receipt["schema"] != "aios.owner-authorization-recovery.v1":
raise RecoveryEvidenceError("RECOVERY_SCHEMA_INVALID")
if receipt["recovered_from_existing_evidence"] is not True or receipt["new_owner_interaction"] is not False:
raise RecoveryEvidenceError("RECOVERY_NOT_APPEND_ONLY")
if receipt["production_side_effect_executed"] is not False or receipt["global_writes_enabled"] is not False:
raise RecoveryEvidenceError("RECOVERY_SCOPE_INVALID")
if receipt["recovery_state"] != "RECOVERED_PENDING_CONTINUATION_LEASE":
raise RecoveryEvidenceError("RECOVERY_STATE_INVALID")
evidence = receipt["evidence_files"]
if not isinstance(evidence, list) or not evidence:
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_LIST_INVALID")
for item in evidence:
if not isinstance(item, dict) or set(item) != {"path", "sha256"}:
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_ENTRY_INVALID")
target = Path(item["path"])
if _digest(target) != item["sha256"]:
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_DIGEST_MISMATCH")
return {
"schema": receipt["schema"],
"state": receipt["recovery_state"],
"campaign_id": receipt["campaign_id"],
"original_event_id": receipt["original_event_id"],
"original_authorization_digest": receipt["original_authorization_digest"],
"signed_grant_id": receipt["signed_grant_id"],
"task_id": receipt["task_id"],
"repository": receipt["repository"],
"original_base_oid": receipt["original_base_oid"],
"branch": receipt["branch"],
"expected_branch_oid": receipt["expected_branch_oid"],
"digest": actual,
"new_owner_interaction": False,
}
@@ -0,0 +1,163 @@
"""Canonical fail-closed capability semantics for task eligibility."""
from __future__ import annotations
from collections.abc import Iterable
_TASK_ALIASES: dict[str, frozenset[str]] = {
"repository_read": frozenset({"repo.read"}),
"git.read": frozenset({"repo.read"}),
"repo.read": frozenset({"repo.read"}),
"test_execution": frozenset({"test.execute"}),
"git.test": frozenset({"test.execute"}),
"test.execute": frozenset({"test.execute"}),
"governed_git_write": frozenset({"governed.git.write.branch", "governed.git.write.file"}),
"governed.git.write": frozenset({"governed.git.write.branch", "governed.git.write.file"}),
"governed.write.branch": frozenset({"governed.git.write.branch"}),
"governed.write.file": frozenset({"governed.git.write.file"}),
"shell": frozenset({"runtime.shell"}),
"git": frozenset({"runtime.git"}),
"python": frozenset({"runtime.python"}),
"code.read": frozenset({"legacy.code.read"}),
"code.review": frozenset({"legacy.code.review"}),
"code.write": frozenset({"legacy.code.write"}),
"code.write.patch": frozenset({"legacy.code.write.patch"}),
"test.python": frozenset({"legacy.test.python"}),
"test.shell.allowlisted": frozenset({"legacy.test.shell.allowlisted"}),
"git.commit": frozenset({"legacy.git.commit"}),
"git.diff.read": frozenset({"legacy.git.diff.read"}),
"git.force_push": frozenset({"legacy.git.force_push"}),
"deploy.production": frozenset({"legacy.deploy.production"}),
"secret.transform": frozenset({"legacy.secret.transform"}),
"secure_file_transformer": frozenset({"legacy.secure_file_transformer"}),
"runtime.shadow": frozenset({"legacy.runtime.shadow"}),
"review.security": frozenset({"legacy.review.security"}),
"review.api-contract": frozenset({"legacy.review.api-contract"}),
"gpu.train": frozenset({"legacy.gpu.train"}),
"agent.runtime.hermes": frozenset({"agent.runtime.hermes"}),
"agent.runtime.openclaw": frozenset({"agent.runtime.openclaw"}),
}
_WORKER_ALIASES: dict[str, frozenset[str]] = {
"git.read": frozenset({"repo.read"}),
"repo.read": frozenset({"repo.read"}),
"repository_read": frozenset({"repo.read"}),
"git.test": frozenset({"test.execute"}),
"test.execute": frozenset({"test.execute"}),
"test_execution": frozenset({"test.execute"}),
"governed.write.branch": frozenset({"governed.git.write.branch"}),
"governed.git.write.branch": frozenset({"governed.git.write.branch"}),
"governed.write.file": frozenset({"governed.git.write.file"}),
"governed.git.write.file": frozenset({"governed.git.write.file"}),
"shell": frozenset({"runtime.shell"}),
"git": frozenset({"runtime.git"}),
"python": frozenset({"runtime.python"}),
"code.read": frozenset({"legacy.code.read"}),
"code.review": frozenset({"legacy.code.review"}),
"code.write": frozenset({"legacy.code.write"}),
"code.write.patch": frozenset({"legacy.code.write.patch"}),
"test.python": frozenset({"legacy.test.python"}),
"test.shell.allowlisted": frozenset({"legacy.test.shell.allowlisted"}),
"git.commit": frozenset({"legacy.git.commit"}),
"git.diff.read": frozenset({"legacy.git.diff.read"}),
"git.force_push": frozenset({"legacy.git.force_push"}),
"deploy.production": frozenset({"legacy.deploy.production"}),
"secret.transform": frozenset({"legacy.secret.transform"}),
"secure_file_transformer": frozenset({"legacy.secure_file_transformer"}),
"runtime.shadow": frozenset({"legacy.runtime.shadow"}),
"review.security": frozenset({"legacy.review.security"}),
"review.api-contract": frozenset({"legacy.review.api-contract"}),
"gpu.train": frozenset({"legacy.gpu.train"}),
"agent.runtime.hermes": frozenset({"agent.runtime.hermes"}),
"agent.runtime.openclaw": frozenset({"agent.runtime.openclaw"}),
}
_UNKNOWN_PREFIX = "exact:"
def _clean(value: object) -> str:
if not isinstance(value, str):
raise ValueError("capability must be a string")
text = value.strip().lower()
if not text:
raise ValueError("capability must not be empty")
return text
def normalize_required_capabilities(values: Iterable[object]) -> frozenset[str]:
"""Map task-facing names to one semantic vocabulary.
Unknown names remain explicit exact requirements. They are never treated
as wildcards or granted by a failed normalization.
"""
result: set[str] = set()
for value in values:
text = _clean(value)
if "*" in text:
result.add(_UNKNOWN_PREFIX + text)
else:
result.update(_TASK_ALIASES.get(text, frozenset({_UNKNOWN_PREFIX + text})))
return frozenset(result)
def normalize_worker_capabilities(values: Iterable[object]) -> frozenset[str]:
"""Map registered worker names to effective semantic capabilities."""
result: set[str] = set()
for value in values:
text = _clean(value)
if "*" in text:
continue
mapped = _WORKER_ALIASES.get(text)
if mapped is not None:
result.update(mapped)
if {
"governed.git.write.branch",
"governed.git.write.file",
}.issubset(result):
result.add("governed.git.write")
return frozenset(result)
def has_capabilities(
worker_capabilities: Iterable[object],
required_capabilities: Iterable[object],
) -> bool:
"""Return the single eligibility decision used by poll and claim."""
required = normalize_required_capabilities(required_capabilities)
effective = normalize_worker_capabilities(worker_capabilities)
if any(value.startswith(_UNKNOWN_PREFIX) for value in required):
return False
return required.issubset(effective)
def required_capabilities_subset(
granted_requirements: Iterable[object],
required_requirements: Iterable[object],
) -> bool:
"""Check semantic capability containment for delegation constraints."""
granted = normalize_required_capabilities(granted_requirements)
required = normalize_required_capabilities(required_requirements)
if any(value.startswith(_UNKNOWN_PREFIX) for value in granted | required):
return False
return required.issubset(granted)
def normalized_capability_matrix(
worker_capabilities: Iterable[object],
required_capabilities: Iterable[object],
) -> dict[str, object]:
"""Expose non-secret decision evidence for tests and diagnostics."""
worker_values = tuple(worker_capabilities)
required_values = tuple(required_capabilities)
worker = normalize_worker_capabilities(worker_values)
required = normalize_required_capabilities(required_values)
return {
"worker_effective": sorted(worker),
"task_required": sorted(required),
"eligible": has_capabilities(worker_values, required_values),
}
+96 -2
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
from __future__ import annotations
import hashlib, hmac, json, os, re, subprocess, sys, time, urllib.parse, uuid
import hashlib, hmac, json, os, re, subprocess, sys, time, urllib.parse, urllib.request, uuid
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from typing import Any
@@ -33,6 +33,11 @@ from silent_release_hardening import (
SignedReleasePlanApproval,
SilentReleaseActionService,
)
try:
from aios_gitops_job_controller import GitOpsJobController, JobError, ServiceUnavailable, JobStore
except ModuleNotFoundError:
sys.path.insert(0, str(Path(__file__).resolve().parent))
from aios_gitops_job_controller import GitOpsJobController, JobError, ServiceUnavailable, JobStore
try:
from authoritative_git import (
AuthoritativeGitError,
@@ -120,6 +125,12 @@ def reject_non_read_http(handler, method: str, path: str) -> bool:
return False
if path in SILENT_RELEASE_POST_PATHS:
return False
if (
method == 'POST'
and path in ('/api/file', '/api/commit-file')
and os.getenv('AIOS_GITOPS_DURABLE_JOB_EXECUTION_ENABLED', os.getenv('AIOS_GITOPS_JOB_CONTROLLER_ENABLED', 'false')).lower() in {'1', 'true', 'yes', 'on'}
):
return False
decision = evaluate_git_write(WS)
if decision.allowed:
return False
@@ -393,6 +404,51 @@ class H(BaseHTTPRequestHandler):
if not a['ok']: self.sendj({'error':'unauthorized','hint':'Bearer token required'},401); return None
if '*' not in a['scopes'] and scope not in a['scopes']: self.sendj({'error':'forbidden','required_scope':scope,'account':a['account']},403); return None
return a
def bridge_release(self, path, data):
route = {
'/createTagApprovalRequest': '/v2/tag/approval-request',
'/requestOrCreateTag': '/v2/tag/publish',
'/getTagPublicationStatus': '/v2/tag/status',
'/buildProductionArtifact': '/v2/release/artifact',
'/executeApprovedProductionRelease': '/v2/release/execute',
'/getReleaseConsistency': '/v2/release/consistency',
'/runReleaseAcceptance': '/v2/release/acceptance',
}.get(path)
if route is None:
raise ValueError('governed_release_route_invalid')
base = os.getenv('AIOS_BRIDGE_V2_URL', 'http://127.0.0.1:18892').rstrip('/')
request = urllib.request.Request(
base + route,
data=json.dumps(data, sort_keys=True, separators=(',', ':')).encode('utf-8'),
headers={'Content-Type': 'application/json'},
method='POST',
)
try:
with urllib.request.urlopen(request, timeout=30) as response:
result = json.loads(response.read().decode('utf-8'))
return self.sendj(result, response.status)
except urllib.error.HTTPError as exc:
try:
body = json.loads(exc.read().decode('utf-8'))
except Exception:
body = {'error': 'bridge_http_error'}
return self.sendj(body, exc.code)
except (urllib.error.URLError, TimeoutError) as exc:
return self.sendj({'error': 'GOVERNED_BRIDGE_UNAVAILABLE', 'detail': type(exc).__name__}, 503)
def do_POST(self):
path = urllib.parse.urlparse(self.path).path
if path in {
'/createTagApprovalRequest', '/requestOrCreateTag', '/getTagPublicationStatus',
'/buildProductionArtifact', '/executeApprovedProductionRelease',
'/getReleaseConsistency', '/runReleaseAcceptance',
}:
actor = self.need('tag:create')
if not actor:
return
return self.bridge_release(path, self.body())
if reject_non_read_http(self, 'POST', path):
return
return self.sendj({'error': 'not_found', 'path': path}, 404)
def do_GET(self):
u=urllib.parse.urlparse(self.path); path=u.path.rstrip('/') if u.path != '/' else u.path
if path not in READ_ONLY_GET_PATHS: return self.sendj({'error':'not_found','path':path},404)
@@ -666,7 +722,44 @@ class H(BaseHTTPRequestHandler):
def file(self,data):
a=self.need('file:write');
if not a: return
b=checkout(data.get('branch'), data.get('base','main'), bool(data.get('create_branch'))); r=write_file(data.get('path'), data.get('content','')); c=commit([r], data.get('message') or f'chore(chatgpt): update {r}'); audit(a['account'],'file',r,{'branch':b,'commit':c}); self.sendj({'status':'committed','branch':b,'path':r,'commit':c})
if os.getenv('AIOS_GITOPS_DURABLE_JOB_EXECUTION_ENABLED', os.getenv('AIOS_GITOPS_JOB_CONTROLLER_ENABLED', 'false')).lower() not in {'1','true','yes','on'}:
return self.sendj({'error':'DURABLE_JOB_CONTROLLER_UNAVAILABLE','retryable':True},503)
try:
path = relpath(str(data.get('path') or ''))
branch = str(data.get('branch') or '')
expected = str(data.get('expected_branch_oid') or '')
campaign_id = str(data.get('campaign_id') or os.getenv('AIOS_GITOPS_CAMPAIGN_ID') or '')
supplied_digest = str(data.get('payload_digest') or '')
content = data.get('content')
payload_bytes = None
if content is not None:
payload_bytes = content.encode('utf-8') if isinstance(content, str) else json.dumps(content, sort_keys=True, ensure_ascii=False).encode('utf-8')
payload_digest = supplied_digest or hashlib.sha256(payload_bytes or b'').hexdigest()
if not campaign_id or not branch or not expected:
return self.sendj({'error':'JOB_BINDING_REQUIRED','retryable':False},409)
request = {
'campaign_id': campaign_id,
'repository': REPOSITORY_IDENTITY or 'aios/aios',
'operation': 'commit-file',
'branch': branch,
'expected_branch_oid': expected,
'payload_digest': payload_digest,
# The controller persists only the binding and digest. Raw
# content is never copied into durable state or logs.
}
receipt = GitOpsJobController(
JobStore(Path(os.getenv('AIOS_GITOPS_JOB_ROOT', '/var/lib/aios-gitops/jobs')))
).submit(request, payload=payload_bytes)
return self.sendj({
'job_id': receipt.job_id,
'state': receipt.state.value,
'side_effect_state': receipt.side_effect_state,
'retryable': receipt.http_status == 503,
}, receipt.http_status)
except ServiceUnavailable as exc:
return self.sendj({'error':str(exc),'layer':'durable_job','retryable':True,'side_effect_state':'UNKNOWN'},503)
except JobError as exc:
return self.sendj({'error':str(exc),'retryable':False},409)
def commit(self,data):
a=self.need('commit:create');
if not a: return
@@ -857,3 +950,4 @@ def main():
print(f'AIOS ChatGPT GitOps adapter listening on {HOST}:{PORT}')
ThreadingHTTPServer((HOST,PORT), H).serve_forever()
if __name__ == '__main__': main()
@@ -0,0 +1,224 @@
"""ChatGPT GitOps Bridge v2 contract and governed release operations."""
from __future__ import annotations
from dataclasses import dataclass
import hashlib
from pathlib import Path
from typing import Any, Callable, Mapping
BRIDGE_SCHEMA_VERSION = "aios.chatgpt-gitops-bridge.v2"
BRIDGE_OPERATIONS = (
"getRuntimeStatus",
"getGovernedWriteCapabilities",
"createGovernedTask",
"acquireWriterLease",
"bootstrapWriterLease",
"renewWriterLease",
"commitFileGoverned",
"commitFilesGoverned",
"getGitOpsJob",
"resumeGitOpsJob",
"reconcileGitOpsJob",
"createTagApprovalRequest",
"requestOrCreateTag",
"getTagPublicationStatus",
"buildProductionArtifact",
"executeApprovedProductionRelease",
"getReleaseConsistency",
"runReleaseAcceptance",
"getControlPlaneSnapshot",
)
OPERATION_REQUIRED_FIELDS = {
"createGovernedTask": ("task_id", "repository", "base_oid", "target_branch", "risk", "description", "idempotency_key"),
"acquireWriterLease": ("schema", "task_id", "repository", "branch_mode", "base_oid", "expected_branch_oid", "target_branch", "generation", "lease_seconds"),
"bootstrapWriterLease": ("schema", "task_id", "repository", "branch_mode", "base_oid", "expected_branch_oid", "target_branch", "generation", "lease_seconds"),
"renewWriterLease": ("schema", "task_id", "repository", "branch_mode", "base_oid", "expected_branch_oid", "target_branch", "generation", "lease_seconds"),
"commitFileGoverned": ("task_id", "repository", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "path", "content_base64", "message"),
"commitFilesGoverned": ("task_id", "repository", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "files", "message"),
"getGitOpsJob": ("job_id",),
"resumeGitOpsJob": ("job_id", "expected_revision"),
"reconcileGitOpsJob": ("job_id", "expected_revision"),
"createTagApprovalRequest": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id", "owner_approval", "execute"),
"requestOrCreateTag": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id", "execute"),
"getTagPublicationStatus": ("version",),
"buildProductionArtifact": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id"),
"executeApprovedProductionRelease": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id", "execute"),
"getReleaseConsistency": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id"),
"runReleaseAcceptance": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id"),
"getControlPlaneSnapshot": (),
}
BRIDGE_OPERATION_CONTRACTS = {
"acquireWriterLease": {
"method": "POST",
"path": "/v2/writer-lease",
"required_fields": OPERATION_REQUIRED_FIELDS["acquireWriterLease"],
"response_fields": (
"task_id", "job_id", "generation", "lease_seconds", "lease_expires_at",
"create_target_branch", "worktree_mode", "cas_ref_update_required",
),
},
"bootstrapWriterLease": {
"method": "POST",
"path": "/v2/writer-lease/bootstrap",
"required_fields": OPERATION_REQUIRED_FIELDS["bootstrapWriterLease"],
"response_fields": (
"task_id", "job_id", "generation", "lease_seconds", "lease_expires_at",
"create_target_branch", "worktree_mode", "cas_ref_update_required",
"recovered", "bootstrapped", "runtime_rehydrated",
"durable_branch_lease_present", "idempotent",
),
},
"renewWriterLease": {
"method": "POST",
"path": "/v2/writer-lease/renew",
"required_fields": OPERATION_REQUIRED_FIELDS["renewWriterLease"],
"response_fields": (
"task_id", "job_id", "generation", "lease_seconds", "lease_expires_at",
"recovered", "bootstrapped", "runtime_rehydrated",
"durable_branch_lease_present", "idempotent",
),
},
"commitFileGoverned": {
"method": "POST", "path": "/v2/commit-file",
"required_fields": OPERATION_REQUIRED_FIELDS["commitFileGoverned"],
"response_fields": ("job_id", "commit_oid", "commit_visible", "branch_mode", "target_branch", "generation"),
},
"commitFilesGoverned": {
"method": "POST", "path": "/v2/commit-files",
"required_fields": OPERATION_REQUIRED_FIELDS["commitFilesGoverned"],
"response_fields": ("job_id", "commit_oid", "commit_visible", "branch_mode", "target_branch", "generation"),
},
}
class RuntimeDriftError(RuntimeError):
pass
class BridgeV2Error(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class ControlPlaneReleaseManifest:
control_plane_release_version: str
control_plane_target_oid: str
artifact_sha256: str
runtime_path: Path
bridge_schema_version: str
repository_identity: str
authoritative_git_dir: Path
deployment_state: str
def as_dict(self) -> dict[str, object]:
return {
"control_plane_release_version": self.control_plane_release_version,
"control_plane_target_oid": self.control_plane_target_oid,
"artifact_sha256": self.artifact_sha256,
"runtime_path": str(self.runtime_path),
"bridge_schema_version": self.bridge_schema_version,
"repository_identity": self.repository_identity,
"authoritative_git_dir": str(self.authoritative_git_dir),
"deployment_state": self.deployment_state,
}
def validate(self, artifact: Path, *, observed_oid: str, approved_root: Path = Path("/opt/aios/gitops-control-plane")) -> dict[str, object]:
try:
runtime = self.runtime_path.resolve(strict=True)
root = approved_root.resolve(strict=True)
artifact_path = artifact.resolve(strict=True)
runtime.relative_to(root)
artifact_path.relative_to(runtime)
except (OSError, ValueError):
raise RuntimeDriftError("CONTROL_PLANE_RUNTIME_DRIFT") from None
mutable_markers = {"workspace", "workspaces", "worktree", "worktrees"}
if mutable_markers.intersection(part.lower() for part in runtime.parts):
raise RuntimeDriftError("CONTROL_PLANE_RUNTIME_DRIFT")
valid = (
len(self.control_plane_target_oid) == 40
and observed_oid == self.control_plane_target_oid
and len(self.artifact_sha256) == 64
and hashlib.sha256(artifact_path.read_bytes()).hexdigest() == self.artifact_sha256
and self.bridge_schema_version == BRIDGE_SCHEMA_VERSION
and self.repository_identity == "aios/aios"
and self.authoritative_git_dir == Path("/data/aios-gitops/repo.git")
and self.authoritative_git_dir.is_absolute()
and self.deployment_state == "STAGING"
)
if not valid:
raise RuntimeDriftError("CONTROL_PLANE_RUNTIME_DRIFT")
return {**self.as_dict(), "drift_status": "PASS"}
class BridgeV2Adapter:
"""Thin operation adapter; all mutating handlers are injected seams."""
def __init__(self, *, runtime_status: Mapping[str, Any], handlers: Mapping[str, Callable[[Mapping[str, Any]], Any]] | None = None) -> None:
self.runtime_status = dict(runtime_status)
self.handlers = dict(handlers or {})
def getRuntimeStatus(self) -> dict[str, Any]:
return {"bridge_schema_version": BRIDGE_SCHEMA_VERSION, **self.runtime_status}
def getGovernedWriteCapabilities(self) -> dict[str, Any]:
return {
"bridge_schema_version": BRIDGE_SCHEMA_VERSION,
"legacy_writes_enabled": False,
"governed_writes_enabled": True,
"operations": list(BRIDGE_OPERATIONS),
"operation_contracts": {
operation: {
**contract,
"required_fields": list(contract["required_fields"]),
"response_fields": list(contract["response_fields"]),
}
for operation, contract in BRIDGE_OPERATION_CONTRACTS.items()
},
"force_push": False,
"main_write": False,
"tag_write": False,
"baseline_write": False,
"production_write": False,
"approved_release_operations": True,
"approved_release_scope": {
"version": "V2R2C1",
"target_oid": "998d30304787e31e5ae607c0bd86b5edb2b0e5ad",
"evidence_closure_oid": "a8b906d1bc24088c19647e98de6a679268f6d0a2",
"approval_plan_id": "sha256:c0df52008ae7f64cec5ee2d28947aa7b892066c670ab8295e4549140b1a7500e",
},
}
@staticmethod
def legacy_write_disabled() -> dict[str, object]:
return {"error": "LEGACY_WRITE_DISABLED", "replacement": "createGovernedTask", "governed_writes_available": True}
def _call(self, operation: str, request: Mapping[str, Any]) -> Any:
required = OPERATION_REQUIRED_FIELDS[operation]
if any(key not in request for key in required):
raise BridgeV2Error("BRIDGE_REQUEST_FIELDS_MISSING")
handler = self.handlers.get(operation)
if handler is None:
raise BridgeV2Error("GOVERNED_HANDLER_UNAVAILABLE")
return handler(dict(request))
def createGovernedTask(self, request: Mapping[str, Any]) -> Any: return self._call("createGovernedTask", request)
def acquireWriterLease(self, request: Mapping[str, Any]) -> Any: return self._call("acquireWriterLease", request)
def bootstrapWriterLease(self, request: Mapping[str, Any]) -> Any: return self._call("bootstrapWriterLease", request)
def renewWriterLease(self, request: Mapping[str, Any]) -> Any: return self._call("renewWriterLease", request)
def commitFileGoverned(self, request: Mapping[str, Any]) -> Any: return self._call("commitFileGoverned", request)
def commitFilesGoverned(self, request: Mapping[str, Any]) -> Any: return self._call("commitFilesGoverned", request)
def getGitOpsJob(self, request: Mapping[str, Any]) -> Any: return self._call("getGitOpsJob", request)
def resumeGitOpsJob(self, request: Mapping[str, Any]) -> Any: return self._call("resumeGitOpsJob", request)
def reconcileGitOpsJob(self, request: Mapping[str, Any]) -> Any: return self._call("reconcileGitOpsJob", request)
def createTagApprovalRequest(self, request: Mapping[str, Any]) -> Any: return self._call("createTagApprovalRequest", request)
def requestOrCreateTag(self, request: Mapping[str, Any]) -> Any: return self._call("requestOrCreateTag", request)
def getTagPublicationStatus(self, request: Mapping[str, Any]) -> Any: return self._call("getTagPublicationStatus", request)
def buildProductionArtifact(self, request: Mapping[str, Any]) -> Any: return self._call("buildProductionArtifact", request)
def executeApprovedProductionRelease(self, request: Mapping[str, Any]) -> Any: return self._call("executeApprovedProductionRelease", request)
def getReleaseConsistency(self, request: Mapping[str, Any]) -> Any: return self._call("getReleaseConsistency", request)
def runReleaseAcceptance(self, request: Mapping[str, Any]) -> Any: return self._call("runReleaseAcceptance", request)
def getControlPlaneSnapshot(self, request: Mapping[str, Any] | None = None) -> Any: return self._call("getControlPlaneSnapshot", request or {})
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,692 @@
"""Governed continuation executor for exact existing task branches.
The executor composes the existing continuation lease and durable Job store.
All Git and Gitea side effects live behind injected narrow interfaces so the
state machine can prove Authority, Git-network, API-branch, and commit-object
agreement before it reports completion.
"""
from __future__ import annotations
import base64
from dataclasses import asdict, dataclass
from datetime import datetime, timezone
import hashlib
import json
import os
from pathlib import Path, PurePosixPath
import re
import subprocess
from typing import Callable, Mapping, Protocol
from urllib.error import HTTPError, URLError
from urllib.parse import quote, urlsplit
from urllib.request import Request, urlopen
from aios_authority_git_executor import AuthorityGitExecutor
from aios_continuation_lease import WriterLeaseRequestV2
from aios_gitops_job_controller import GitOpsJobController, JobState
from aios_gitops_reconciliation_core import ReconciliationAction, RemoteObservation, decide_reconciliation
from authorized_task_writer_provisioning import validate_writer_lease_request_v2
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SECRET_PATTERNS = (
re.compile(rb"gh[pousr]_[A-Za-z0-9]{20,}"),
re.compile(rb"(?i)(?:token|password|secret|api[_-]?key)\s*[:=]\s*[\"'][^\"'\s]{12,}[\"']"),
re.compile(rb"-----BEGIN " + rb"(?:RSA |EC |OPENSSH )?PRIVATE KEY-----"),
)
class ContinuationExecutionError(RuntimeError):
"""Stable fail-closed continuation execution error."""
class DetachedWorkspace(Protocol):
def is_detached(self) -> bool: ...
def head_oid(self) -> str: ...
def commit_files(self, files: Mapping[str, bytes], message: str) -> str: ...
class AuthorityPort(Protocol):
def branch_exists(self, branch: str) -> bool: ...
def branch_head(self, branch: str) -> str: ...
def is_ancestor(self, older: str, newer: str) -> bool: ...
def create_task_branch_cas(self, branch: str, expected_base: str, new: str) -> str: ...
def compare_and_swap_branch(self, branch: str, expected: str, new: str) -> str: ...
class RemotePort(Protocol):
def repository_identity(self) -> str: ...
def push_new_exact(self, branch: str, new: str) -> None: ...
def push_exact(self, branch: str, expected: str, new: str) -> None: ...
def read_git_ref(self, branch: str) -> str | None: ...
def read_api_branch(self, branch: str) -> str | None: ...
def commit_visible(self, oid: str) -> bool: ...
class CredentialProjection(Protocol):
def project(self) -> None: ...
def revoke(self) -> None: ...
def _run(argv: list[str], *, cwd: Path | None = None, environment: Mapping[str, str] | None = None) -> str:
result = subprocess.run(
argv,
cwd=cwd,
env=None if environment is None else dict(environment),
text=True,
capture_output=True,
check=False,
)
if result.returncode != 0:
raise ContinuationExecutionError("SUBPROCESS_FAILED")
return result.stdout.strip()
@dataclass(slots=True)
class AuthorityGitAdapter:
"""Branch-name adapter over the existing narrow AuthorityGitExecutor."""
executor: AuthorityGitExecutor
@staticmethod
def _ref(branch: str) -> str:
return f"refs/heads/{branch}"
def branch_exists(self, branch: str) -> bool:
return self.executor.read_ref(self._ref(branch)) is not None
def branch_head(self, branch: str) -> str:
value = self.executor.read_ref(self._ref(branch))
if value is None:
raise ContinuationExecutionError("EXISTING_BRANCH_MISSING")
return value
def is_ancestor(self, older: str, newer: str) -> bool:
return self.executor.ancestry_check(older, newer)
def create_task_branch_cas(self, branch: str, expected_base: str, new: str) -> str:
return self.executor.create_task_branch_CAS(self._ref(branch), expected_base, new)
def compare_and_swap_branch(self, branch: str, expected: str, new: str) -> str:
return self.executor.advance_task_branch_CAS(self._ref(branch), expected, new)
@dataclass(slots=True)
class DetachedGitWorkspace:
path: Path
def __post_init__(self) -> None:
if not self.path.is_absolute() or self.path.is_symlink() or not self.path.is_dir():
raise ContinuationExecutionError("WORKTREE_PATH_INVALID")
top = Path(_run(["git", "rev-parse", "--show-toplevel"], cwd=self.path)).resolve()
if top != self.path.resolve():
raise ContinuationExecutionError("WORKTREE_PATH_INVALID")
def is_detached(self) -> bool:
result = subprocess.run(
["git", "symbolic-ref", "-q", "HEAD"],
cwd=self.path,
capture_output=True,
check=False,
)
return result.returncode == 1
def head_oid(self) -> str:
return _run(["git", "rev-parse", "HEAD"], cwd=self.path)
def _dirty_paths(self) -> set[str]:
command = subprocess.run(
["git", "status", "--porcelain=v1", "--untracked-files=all"],
cwd=self.path,
text=True,
capture_output=True,
check=False,
)
if command.returncode != 0:
raise ContinuationExecutionError("SUBPROCESS_FAILED")
output = command.stdout
result: set[str] = set()
for line in output.splitlines():
path = line[3:]
if " -> " in path:
path = path.split(" -> ", 1)[1]
result.add(path)
return result
def materialize_files(self, files: Mapping[str, bytes]) -> None:
dirty = self._dirty_paths()
expected_paths = set(files)
if dirty and dirty != expected_paths:
raise ContinuationExecutionError("WORKTREE_DIRTY_SCOPE_MISMATCH")
for relative, content in files.items():
target = (self.path / relative).resolve()
try:
target.relative_to(self.path.resolve())
except ValueError as exc:
raise ContinuationExecutionError("COMMIT_PATH_REJECTED") from exc
if target.is_symlink():
raise ContinuationExecutionError("COMMIT_PATH_REJECTED")
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(content)
def commit_files(self, files: Mapping[str, bytes], message: str) -> str:
expected_paths = set(files)
if self._dirty_paths() != expected_paths:
raise ContinuationExecutionError("WORKTREE_DIRTY_SCOPE_MISMATCH")
for relative, content in files.items():
target = (self.path / relative).resolve()
try:
target.relative_to(self.path.resolve())
except ValueError as exc:
raise ContinuationExecutionError("COMMIT_PATH_REJECTED") from exc
if not target.is_file() or target.is_symlink() or target.read_bytes() != content:
raise ContinuationExecutionError("AUTHORING_PAYLOAD_MISMATCH")
_run(["git", "add", "--", *sorted(expected_paths)], cwd=self.path)
staged = set(_run(["git", "diff", "--cached", "--name-only"], cwd=self.path).splitlines())
if staged != expected_paths:
raise ContinuationExecutionError("STAGED_SCOPE_MISMATCH")
environment = os.environ.copy()
environment.update(
{
"GIT_AUTHOR_NAME": "AIOS Governed Continuation",
"GIT_AUTHOR_EMAIL": "aios-governed-continuation@localhost",
"GIT_COMMITTER_NAME": "AIOS Governed Continuation",
"GIT_COMMITTER_EMAIL": "aios-governed-continuation@localhost",
}
)
_run(["git", "commit", "--no-gpg-sign", "-m", message, "--", *sorted(expected_paths)], cwd=self.path, environment=environment)
if not self.is_detached():
raise ContinuationExecutionError("WORKTREE_NOT_DETACHED")
return self.head_oid()
@dataclass(slots=True)
class ServerSideCredentialProjection:
ssh_key_path: Path
api_token_path: Path
active: bool = False
def project(self) -> None:
for path in (self.ssh_key_path, self.api_token_path):
if path.is_symlink() or not path.is_file() or path.stat().st_size < 32:
raise ContinuationExecutionError("CREDENTIAL_PROJECTION_MISSING")
self.active = True
def revoke(self) -> None:
self.active = False
@dataclass(slots=True)
class GiteaContinuationRemote:
workspace: Path
credential: ServerSideCredentialProjection
remote_name: str = "gitea"
repository: str = "aios/aios"
api_base: str = "https://g.cnnetmkt.top/git/api/v1"
expected_host: str = "g.cnnetmkt.top"
@staticmethod
def validate_remote_url(value: str, expected_host: str = "g.cnnetmkt.top") -> None:
if value.startswith(("/", "file:")) or value.endswith("/data/aios-gitops/repo.git"):
raise ContinuationExecutionError("GITEA_REMOTE_IDENTITY_INVALID")
if "://" in value:
parsed = urlsplit(value)
host, path = parsed.hostname, parsed.path
else:
location = value.split("@", 1)[-1]
host, separator, path = location.partition(":")
if not separator:
raise ContinuationExecutionError("GITEA_REMOTE_IDENTITY_INVALID")
path = "/" + path
if host != expected_host or path.rstrip("/") not in {"/aios/aios.git", "/git/aios/aios.git"}:
raise ContinuationExecutionError("GITEA_REMOTE_IDENTITY_INVALID")
def repository_identity(self) -> str:
url = _run(["git", "remote", "get-url", self.remote_name], cwd=self.workspace)
self.validate_remote_url(url, self.expected_host)
if self.repository != "aios/aios":
raise ContinuationExecutionError("REMOTE_REPOSITORY_IDENTITY_MISMATCH")
return self.repository
def _require_projected(self) -> None:
if not self.credential.active:
raise ContinuationExecutionError("CREDENTIAL_PROJECTION_MISSING")
def read_git_ref(self, branch: str) -> str | None:
self._require_projected()
output = _run(["git", "ls-remote", self.remote_name, f"refs/heads/{branch}"], cwd=self.workspace)
if not output:
return None
oid, ref = output.split(maxsplit=1)
if ref != f"refs/heads/{branch}" or not OID_RE.fullmatch(oid):
raise ContinuationExecutionError("GITEA_GIT_READBACK_INVALID")
return oid
def push_new_exact(self, branch: str, new: str) -> None:
self._require_projected()
self.repository_identity()
if self.read_git_ref(branch) is not None:
raise ContinuationExecutionError("GITEA_TARGET_BRANCH_ALREADY_EXISTS")
_run(["git", "push", self.remote_name, f"{new}:refs/heads/{branch}"], cwd=self.workspace)
if self.read_git_ref(branch) != new:
raise ContinuationExecutionError("GITEA_CREATE_READBACK_MISMATCH")
def push_exact(self, branch: str, expected: str, new: str) -> None:
self._require_projected()
self.repository_identity()
if self.read_git_ref(branch) != expected:
raise ContinuationExecutionError("GITEA_BRANCH_EXPECTATION_MISMATCH")
_run(["git", "push", self.remote_name, f"{new}:refs/heads/{branch}"], cwd=self.workspace)
def _api(self, path: str) -> object:
self._require_projected()
token = self.credential.api_token_path.read_text(encoding="utf-8").strip()
request = Request(
self.api_base.rstrip("/") + path,
headers={"Accept": "application/json", "Authorization": f"token {token}"},
)
try:
with urlopen(request, timeout=20) as response:
return json.loads(response.read().decode("utf-8"))
except HTTPError as exc:
if exc.code == 404:
return None
raise ContinuationExecutionError(f"GITEA_API_HTTP_{exc.code}") from exc
except (URLError, TimeoutError, OSError, json.JSONDecodeError) as exc:
raise ContinuationExecutionError("GITEA_API_UNAVAILABLE") from exc
def read_api_branch(self, branch: str) -> str | None:
owner, name = self.repository.split("/", 1)
page = 1
while page <= 100:
value = self._api(f"/repos/{quote(owner)}/{quote(name)}/branches?page={page}&limit=50")
if value is None:
return None
if not isinstance(value, list):
raise ContinuationExecutionError("GITEA_API_BRANCH_COLLECTION_INVALID")
for item in value:
if not isinstance(item, dict) or item.get("name") != branch:
continue
oid = item.get("commit", {}).get("id")
if not isinstance(oid, str) or not OID_RE.fullmatch(oid):
raise ContinuationExecutionError("GITEA_API_BRANCH_INVALID")
return oid
if len(value) < 50:
return None
page += 1
raise ContinuationExecutionError("GITEA_API_BRANCH_PAGINATION_LIMIT")
def commit_visible(self, oid: str) -> bool:
owner, name = self.repository.split("/", 1)
value = self._api(f"/repos/{quote(owner)}/{quote(name)}/git/commits/{oid}")
if not isinstance(value, dict):
return False
observed = value.get("sha") or value.get("id")
return observed == oid
@dataclass(frozen=True, slots=True)
class ContinuationPermissions:
create_target_branch: bool = False
force_push: bool = False
tag_write: bool = False
baseline_write: bool = False
production_write: bool = False
main_write: bool = False
def validate(self) -> None:
if any(asdict(self).values()):
raise ContinuationExecutionError("MUTATION_PERMISSION_DENIED")
@dataclass(frozen=True, slots=True)
class ContinuationRuntimeBinding:
request: WriterLeaseRequestV2
job_id: str
workspace_path: Path
lease_not_before: datetime
lease_expires_at: datetime
original_receipt_valid: bool
original_grant_consumed: bool
permissions: ContinuationPermissions
@dataclass(frozen=True, slots=True)
class ContinuationCommit:
idempotency_key: str
files: Mapping[str, bytes]
message: str
def payload(self) -> bytes:
if not self.idempotency_key or not self.message.strip() or not self.files:
raise ContinuationExecutionError("COMMIT_REQUEST_INVALID")
normalized: dict[str, str] = {}
for path, content in self.files.items():
pure = PurePosixPath(path)
if (
not isinstance(path, str)
or pure.is_absolute()
or ".." in pure.parts
or path.startswith((".git/", "baseline/", "production/"))
or not isinstance(content, bytes)
):
raise ContinuationExecutionError("COMMIT_PATH_REJECTED")
if any(pattern.search(content) for pattern in SECRET_PATTERNS):
raise ContinuationExecutionError("SECRET_LIKE_PAYLOAD_REJECTED")
normalized[path] = base64.b64encode(content).decode("ascii")
return json.dumps(
{"files_base64": normalized, "message": self.message},
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
class ContinuationExecutor:
def __init__(
self,
controller: GitOpsJobController,
authority: AuthorityPort,
workspace: DetachedWorkspace,
remote: RemotePort,
credential: CredentialProjection,
*,
now: Callable[[], datetime] = lambda: datetime.now(timezone.utc),
) -> None:
self.controller = controller
self.authority = authority
self.workspace = workspace
self.remote = remote
self.credential = credential
self.now = now
self._observed_generations: dict[str, int] = {}
def observe_generation(self, branch: str, generation: int) -> None:
self._observed_generations[branch] = max(generation, self._observed_generations.get(branch, 0))
def _max_generation(self, branch: str, *, exclude_job_id: str | None = None) -> int:
observed = self._observed_generations.get(branch, 0)
with self.controller.store._connect() as connection:
if exclude_job_id is None:
row = connection.execute(
"SELECT MAX(generation) AS generation FROM branch_leases WHERE branch=?",
(branch,),
).fetchone()
else:
row = connection.execute(
"SELECT MAX(generation) AS generation FROM branch_leases "
"WHERE branch=? AND job_id!=?",
(branch, exclude_job_id),
).fetchone()
durable = int(row["generation"] or 0)
return max(observed, durable)
@staticmethod
def _request_mapping(request: WriterLeaseRequestV2) -> dict[str, object]:
return {
"schema": WriterLeaseRequestV2.schema,
"task_id": request.task_id,
"job_id": request.job_id,
"repository": request.repository,
"branch_mode": request.branch_mode,
"base_oid": request.base_oid,
"expected_branch_oid": request.expected_branch_oid,
"target_branch": request.target_branch,
"generation": request.generation,
"lease_seconds": request.lease_seconds,
}
def _validate_preflight(self, binding: ContinuationRuntimeBinding, *, job_id: str | None = None) -> None:
request = binding.request
request_mapping = self._request_mapping(request)
normalized = validate_writer_lease_request_v2(request_mapping, self.authority)
expected_normalized = {
**request_mapping,
"create_target_branch": request.branch_mode == "CREATE_NEW",
"worktree_mode": (
"detached_at_base_oid"
if request.branch_mode == "CREATE_NEW"
else "detached_at_expected_head"
),
"cas_ref_update_required": True,
}
if normalized != expected_normalized:
raise ContinuationExecutionError("CONTINUATION_NORMALIZATION_INVALID")
if not binding.original_receipt_valid or not binding.original_grant_consumed:
raise ContinuationExecutionError("ORIGINAL_GRANT_LINEAGE_INVALID")
binding.permissions.validate()
now = self.now()
if now < binding.lease_not_before:
raise ContinuationExecutionError("LEASE_NOT_ACTIVE")
if now >= binding.lease_expires_at:
raise ContinuationExecutionError("LEASE_EXPIRED")
if request.generation <= self._max_generation(request.target_branch, exclude_job_id=job_id):
raise ContinuationExecutionError("GENERATION_NOT_MONOTONIC")
if not self.workspace.is_detached():
raise ContinuationExecutionError("WORKTREE_NOT_DETACHED")
if self.workspace.head_oid() != request.expected_branch_oid:
raise ContinuationExecutionError("WORKTREE_HEAD_MISMATCH")
if self.remote.repository_identity() != request.repository:
raise ContinuationExecutionError("REMOTE_REPOSITORY_IDENTITY_MISMATCH")
def _job_request(self, binding: ContinuationRuntimeBinding, commit: ContinuationCommit, payload: bytes) -> dict[str, object]:
request = binding.request
return {
"idempotency_key": commit.idempotency_key,
"campaign_id": request.task_id,
"task_id": request.task_id,
"repository": request.repository,
"operation": "CONTINUATION_COMMIT_FILES",
"branch": request.target_branch,
"expected_branch_oid": request.expected_branch_oid,
"base_oid": request.base_oid,
"branch_mode": request.branch_mode,
"payload_digest": hashlib.sha256(payload).hexdigest(),
"lineage_anchor_oid": request.base_oid,
"generation": request.generation,
}
def execute(
self,
binding: ContinuationRuntimeBinding,
commit: ContinuationCommit,
*,
before_complete: Callable[[str], None] | None = None,
) -> dict[str, object]:
commit.payload()
if not binding.job_id or binding.request.job_id != binding.job_id:
raise ContinuationExecutionError("RUNTIME_LINEAGE_MISSING_JOB_ID")
current = self.controller.store.read(binding.job_id)
current_state = JobState(current["state"])
if current_state == JobState.COMPLETED:
return self.controller.store._receipt(current, duplicate=True).as_dict()
if current_state == JobState.REMOTE_VERIFIED:
receipt = json.loads(str(current["receipt_json"] or "{}"))
commit_oid = str(receipt.get("authority_oid") or "")
if not OID_RE.fullmatch(commit_oid):
raise ContinuationExecutionError("GOVERNED_COMMIT_OID_INVALID")
if before_complete is not None:
before_complete(commit_oid)
return self.controller.advance(
binding.job_id,
JobState.COMPLETED,
side_effect_state="REMOTE_VERIFIED",
receipt=receipt,
).as_dict()
self._validate_preflight(binding, job_id=binding.job_id)
request = binding.request
projected = False
try:
self.credential.project()
projected = True
row = self.controller.store.read(binding.job_id)
state = JobState(row["state"])
if state == JobState.REQUESTED:
self.controller.advance(
binding.job_id,
JobState.AUTHORIZED,
side_effect_state="AUTHORIZED",
receipt={"task_id": request.task_id, "original_grant_consumed": True},
expected_revision=int(row["revision"]),
)
row = self.controller.store.read(binding.job_id)
state = JobState(row["state"])
if state == JobState.AUTHORIZED:
self.controller.lease_job(
binding.job_id,
lease_id=f"lease-{binding.job_id}",
lineage_anchor_oid=request.base_oid,
current_head_oid=request.expected_branch_oid,
generation=request.generation,
lease_expires_at=binding.lease_expires_at.astimezone(timezone.utc).isoformat(),
expected_revision=int(row["revision"]),
)
row = self.controller.store.read(binding.job_id)
state = JobState(row["state"])
if state == JobState.LEASED:
self.controller.advance(binding.job_id, JobState.WORKTREE_READY,
side_effect_state="WORKTREE_READY", expected_revision=int(row["revision"]))
row = self.controller.store.read(binding.job_id)
state = JobState(row["state"])
if state == JobState.WORKTREE_READY:
self.controller.advance(binding.job_id, JobState.EXECUTING,
side_effect_state="IN_PROGRESS", expected_revision=int(row["revision"]))
elif state != JobState.EXECUTING:
raise ContinuationExecutionError("JOB_EXECUTION_STATE_INVALID")
new_oid = self.workspace.commit_files(commit.files, commit.message)
if not OID_RE.fullmatch(new_oid) or not self.authority.is_ancestor(request.expected_branch_oid, new_oid):
raise ContinuationExecutionError("NON_FAST_FORWARD_COMMIT")
if request.branch_mode == "CREATE_NEW":
authority_oid = self.authority.create_task_branch_cas(request.target_branch, request.base_oid, new_oid)
else:
authority_oid = self.authority.compare_and_swap_branch(request.target_branch, request.expected_branch_oid, new_oid)
if authority_oid != new_oid:
raise ContinuationExecutionError("AUTHORITY_CAS_READBACK_MISMATCH")
self.controller.advance(
binding.job_id,
JobState.LOCAL_VERIFIED,
side_effect_state="LOCAL_VERIFIED",
receipt={"authority_oid": authority_oid, "force_push": False},
)
try:
self.controller.advance(binding.job_id, JobState.REMOTE_SYNCING, side_effect_state="IN_PROGRESS")
if request.branch_mode == "CREATE_NEW":
self.remote.push_new_exact(request.target_branch, new_oid)
else:
self.remote.push_exact(request.target_branch, request.expected_branch_oid, new_oid)
except Exception:
return self._remote_pending(binding.job_id, authority_oid, new_oid)
return self._finish_remote_verification(
binding.job_id,
authority_oid,
new_oid,
before_complete=before_complete,
)
finally:
if projected:
self.credential.revoke()
def _observe_remote(self, branch: str, oid: str) -> tuple[str | None, str | None, bool]:
return self.remote.read_git_ref(branch), self.remote.read_api_branch(branch), self.remote.commit_visible(oid)
def _remote_pending(self, job_id: str, authority_oid: str, new_oid: str) -> dict[str, object]:
receipt = {
"authority_oid": authority_oid,
"desired_remote_oid": new_oid,
"remote_verified": False,
"force_push": False,
}
result = self.controller.advance(
job_id,
JobState.RECONCILIATION_REQUIRED,
side_effect_state="REMOTE_PUSH_PENDING",
receipt=receipt,
)
return result.as_dict()
def _finish_remote_verification(
self,
job_id: str,
authority_oid: str,
new_oid: str,
*,
before_complete: Callable[[str], None] | None = None,
) -> dict[str, object]:
row = self.controller.store.read(job_id)
branch = row["branch"]
request = json.loads(row["request_json"])["binding"]
git_oid, api_oid, visible = self._observe_remote(branch, new_oid)
decision = decide_reconciliation(RemoteObservation("aios/aios", branch, new_oid, authority_oid, git_oid, api_oid, visible, datetime.now(timezone.utc).isoformat(), request.get("expected_branch_oid")))
if decision.action is not ReconciliationAction.MARK_REMOTE_VERIFIED:
return self._remote_pending(job_id, authority_oid, new_oid)
receipt = {
"authority_oid": authority_oid,
"git_remote_oid": git_oid,
"api_branch_oid": api_oid,
"commit_visible": True,
"remote_verified": True,
"reconciliation_decision": decision.action.value,
"force_push": False,
}
self.controller.advance(
job_id,
JobState.REMOTE_VERIFIED,
side_effect_state="REMOTE_VERIFIED",
receipt=receipt,
)
if before_complete is not None:
before_complete(new_oid)
return self.controller.advance(
job_id,
JobState.COMPLETED,
side_effect_state="REMOTE_VERIFIED",
receipt=receipt,
).as_dict()
def reconcile(self, job_id: str) -> dict[str, object]:
row = self.controller.store.read(job_id)
state = JobState(row["state"])
if state == JobState.COMPLETED:
return self.controller.store._receipt(row, duplicate=True).as_dict()
if state != JobState.RECONCILIATION_REQUIRED:
raise ContinuationExecutionError("JOB_NOT_RECONCILABLE")
request = json.loads(row["request_json"])["binding"]
receipt = json.loads(row["receipt_json"])
new_oid = receipt["desired_remote_oid"]
authority_oid = self.authority.branch_head(row["branch"])
try:
generation = int(request.get("generation") or 0)
except (TypeError, ValueError) as exc:
raise ContinuationExecutionError("DURABLE_BINDING_GENERATION_INVALID") from exc
if generation > 0:
with self.controller.store._connect() as connection:
earlier = connection.execute(
"SELECT 1 FROM jobs WHERE branch=? AND state=? "
"AND CAST(json_extract(request_json, '$.binding.generation') AS INTEGER) < ? LIMIT 1",
(row["branch"], JobState.RECONCILIATION_REQUIRED.value, generation),
).fetchone()
if earlier is not None:
raise ContinuationExecutionError("EARLIER_RECONCILIATION_REQUIRED")
self.credential.project()
try:
git_oid, api_oid, visible = self._observe_remote(row["branch"], new_oid)
if authority_oid == git_oid == api_oid == new_oid and visible:
self.controller.advance(job_id, JobState.RECONCILING, side_effect_state="OBSERVATION_FIRST")
return self._finish_remote_verification(job_id, new_oid, new_oid)
expected_oid = request["expected_branch_oid"]
if (
self.authority.is_ancestor(new_oid, authority_oid)
and git_oid is not None
and git_oid == api_oid
and self.authority.is_ancestor(git_oid, new_oid)
):
self.controller.advance(job_id, JobState.RECONCILING, side_effect_state="OBSERVATION_FIRST_CHAIN")
self.remote.push_exact(row["branch"], git_oid, new_oid)
return self._finish_remote_verification(job_id, new_oid, new_oid)
if authority_oid == new_oid and git_oid == api_oid == expected_oid:
self.controller.advance(job_id, JobState.RECONCILING, side_effect_state="OBSERVATION_FIRST_RETRY")
self.remote.push_exact(row["branch"], expected_oid, new_oid)
return self._finish_remote_verification(job_id, new_oid, new_oid)
return self.controller.store._receipt(row, duplicate=True).as_dict()
finally:
self.credential.revoke()
@@ -0,0 +1,157 @@
"""Campaign-aware continuation leases for an existing Git branch.
This module is deliberately transport-neutral. It validates branch lineage
and hands ref mutation to an injected compare-and-swap broker; it does not run
Git, inspect credentials, or create a branch itself.
"""
from __future__ import annotations
from dataclasses import dataclass, replace
from datetime import datetime, timedelta, timezone
import re
from typing import Protocol
OID_RE = re.compile(r"^[0-9a-f]{40}$")
TARGET_BRANCH = "agent/autonomous-release-campaign-v1-20260807"
LINEAGE_ANCHOR = "69b45e81dcd2c3707c367003b08c55fd361faae8"
CAMPAIGN_HEAD = "bba675fab3b7e0c5fba7066720252b3c91b9f20d"
class ContinuationError(RuntimeError):
"""Stable fail-closed error for lease or branch binding violations."""
class BranchProbe(Protocol):
def branch_exists(self, branch: str) -> bool: ...
def branch_head(self, branch: str) -> str: ...
def is_ancestor(self, older: str, newer: str) -> bool: ...
class RefCAS(Protocol):
def compare_and_swap_branch(self, branch: str, expected: str, new: str) -> str: ...
def _oid(value: str, field: str) -> str:
if not isinstance(value, str) or not OID_RE.fullmatch(value):
raise ContinuationError(f"{field}_invalid")
return value
@dataclass(frozen=True, slots=True)
class WriterLeaseRequestV2:
task_id: str
repository: str
branch_mode: str
base_oid: str
expected_branch_oid: str
target_branch: str
generation: int
lease_seconds: int = 900
job_id: str = ""
schema = "aios.task-writer-lease.v2"
accepted_schemas = frozenset({"aios.task-writer-lease.v2", "aios.task-writer-lease-request.v2"})
def validate(self, probe: BranchProbe) -> None:
if not self.task_id or self.repository != "aios/aios":
raise ContinuationError("request_identity_invalid")
if not self.job_id or not self.job_id.startswith("job-"):
raise ContinuationError("job_id_invalid")
if self.branch_mode not in {"CREATE_NEW", "CONTINUE_EXISTING"}:
raise ContinuationError("branch_mode_invalid")
_oid(self.base_oid, "base_oid")
_oid(self.expected_branch_oid, "expected_branch_oid")
if self.target_branch in {"main", "master"} or self.target_branch.startswith(("refs/", "tags/")):
raise ContinuationError("protected_branch_rejected")
if not isinstance(self.generation, int) or isinstance(self.generation, bool) or self.generation < 1:
raise ContinuationError("generation_invalid")
if not isinstance(self.lease_seconds, int) or not 1 <= self.lease_seconds <= 86_400:
raise ContinuationError("lease_duration_invalid")
if self.branch_mode == "CONTINUE_EXISTING":
if not probe.branch_exists(self.target_branch):
raise ContinuationError("EXISTING_BRANCH_MISSING")
actual = _oid(probe.branch_head(self.target_branch), "actual_branch_oid")
if actual != self.expected_branch_oid:
raise ContinuationError("EXISTING_BRANCH_OID_CONFLICT")
if not probe.is_ancestor(self.base_oid, actual):
raise ContinuationError("BASE_NOT_ANCESTOR_OF_BRANCH")
elif hasattr(probe, "commit_exists") and not probe.commit_exists(self.base_oid):
raise ContinuationError("BASE_COMMIT_NOT_FOUND")
elif self.branch_mode == "CREATE_NEW" and probe.branch_exists(self.target_branch):
raise ContinuationError("TARGET_BRANCH_ALREADY_EXISTS")
elif self.branch_mode == "CREATE_NEW" and self.base_oid != self.expected_branch_oid:
raise ContinuationError("CREATE_NEW_BASE_EXPECTATION_MISMATCH")
@dataclass(frozen=True, slots=True)
class CampaignWriterLease:
task_id: str
repository: str
branch: str
lineage_anchor_oid: str
current_head_oid: str
generation: int
lease_expires_at: datetime
append_only: bool = True
renewable: bool = True
multi_commit: bool = True
recoverable: bool = True
state: str = "ACTIVE"
branch_bound: bool = True
def __post_init__(self) -> None:
_oid(self.lineage_anchor_oid, "lineage_anchor_oid")
_oid(self.current_head_oid, "current_head_oid")
if not all((self.append_only, self.renewable, self.multi_commit, self.recoverable, self.branch_bound)):
raise ContinuationError("lease_capability_downgrade")
class ContinuationLeaseManager:
"""State machine for a multi-commit lease with CAS-only ref updates."""
def __init__(self, probe: BranchProbe, refs: RefCAS, *, now=datetime.now) -> None:
self.probe = probe
self.refs = refs
self.now = now
def open(self, request: WriterLeaseRequestV2) -> CampaignWriterLease:
request.validate(self.probe)
if request.branch_mode != "CONTINUE_EXISTING":
raise ContinuationError("CREATE_NEW_REQUIRES_SEPARATE_PROVISIONER")
return CampaignWriterLease(
task_id=request.task_id,
repository=request.repository,
branch=request.target_branch,
lineage_anchor_oid=request.base_oid,
current_head_oid=request.expected_branch_oid,
generation=request.generation,
lease_expires_at=self.now(timezone.utc) + timedelta(seconds=request.lease_seconds),
)
def renew(self, lease: CampaignWriterLease, seconds: int) -> CampaignWriterLease:
if lease.state != "ACTIVE" or not lease.renewable or seconds < 1:
raise ContinuationError("lease_not_renewable")
return replace(lease, lease_expires_at=self.now(timezone.utc) + timedelta(seconds=seconds))
def append_commit(self, lease: CampaignWriterLease, new_head_oid: str) -> CampaignWriterLease:
if lease.state != "ACTIVE":
raise ContinuationError("lease_not_active")
_oid(new_head_oid, "new_head_oid")
if not self.probe.is_ancestor(lease.current_head_oid, new_head_oid):
raise ContinuationError("NON_FAST_FORWARD_COMMIT")
try:
actual = self.refs.compare_and_swap_branch(
lease.branch, lease.current_head_oid, new_head_oid
)
except Exception as exc:
raise ContinuationError("BRANCH_LEASE_STALE") from exc
if actual != new_head_oid:
raise ContinuationError("BRANCH_LEASE_STALE")
return replace(lease, current_head_oid=new_head_oid)
def supersede(self, lease: CampaignWriterLease) -> CampaignWriterLease:
if lease.state not in {"ACTIVE", "RECOVERABLE_FAILURE", "RECONCILIATION_REQUIRED"}:
raise ContinuationError("lease_not_supersedable")
return replace(lease, generation=lease.generation + 1, state="SUPERSEDED")
@@ -0,0 +1,25 @@
"""Separate read, write, and release plane readiness evaluation."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Mapping
@dataclass(frozen=True, slots=True)
class RuntimeStatus:
read_plane: Mapping[str, bool]
write_plane: Mapping[str, bool]
release_plane: Mapping[str, bool]
@property
def release_control_plane_ready(self) -> bool:
return all(self.read_plane.values()) and all(self.write_plane.values()) and all(self.release_plane.values())
def as_dict(self) -> dict[str, object]:
return {
"READ_PLANE": dict(self.read_plane),
"WRITE_PLANE": dict(self.write_plane),
"RELEASE_PLANE": dict(self.release_plane),
"release_control_plane_ready": self.release_control_plane_ready,
}
@@ -0,0 +1,58 @@
"""Workspace-independent authority/network/Gitea ref sentinel."""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
import subprocess
from typing import Callable, Mapping
class SentinelError(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class SentinelInputs:
authority_git_dir: Path
gitea_remote: str
gitea_api: str
ref: str
expected_oid: str
@dataclass(frozen=True, slots=True)
class SentinelResult:
authority_oid: str | None
network_oid: str | None
gitea_oid: str | None
equal: bool
error: str | None = None
def authority_oid(inputs: SentinelInputs) -> str:
result = subprocess.run(
["git", "--git-dir", str(inputs.authority_git_dir), "rev-parse", inputs.ref],
text=True,
capture_output=True,
check=False,
)
if result.returncode != 0:
raise SentinelError("AUTHORITY_REF_UNAVAILABLE")
return result.stdout.strip()
def evaluate(
inputs: SentinelInputs,
*,
network_reader: Callable[[str, str], str | None],
gitea_reader: Callable[[str, str], str | None],
) -> SentinelResult:
try:
local = authority_oid(inputs)
network = network_reader(inputs.gitea_remote, inputs.ref)
gitea = gitea_reader(inputs.gitea_api, inputs.ref)
equal = local == network == gitea == inputs.expected_oid
return SentinelResult(local, network, gitea, equal, None if equal else "OID_MISMATCH")
except SentinelError as exc:
return SentinelResult(None, None, None, False, str(exc))
@@ -0,0 +1,26 @@
"""Non-sensitive control-plane snapshot assembly."""
from __future__ import annotations
from typing import Any, Mapping
def _copy(value: Mapping[str, Any] | None, keys: tuple[str, ...]) -> dict[str, Any]:
value = value or {}
return {key: value.get(key) for key in keys}
def build_control_plane_snapshot(*, target_version: str, runtime: Mapping[str, Any] | None = None, repository: Mapping[str, Any] | None = None, production: Mapping[str, Any] | None = None, task: Mapping[str, Any] | None = None, capabilities: Mapping[str, Any] | None = None, credential_broker: Mapping[str, Any] | None = None) -> dict[str, Any]:
return {
"version": {"target_release": target_version},
"runtime": _copy(runtime, ("bridge_schema_version", "control_plane_release_version", "runtime_release_commit", "runtime_path", "artifact_sha256", "drift_status")),
"repository": _copy(repository, ("identity", "authority_git_dir", "authority_main_oid")),
"production": _copy(production, ("baseline_version", "target_oid", "deployment_state")),
"task": _copy(task, ("task_id", "branch", "authority_branch_oid", "gitea_git_oid", "gitea_api_oid", "commit_visible", "generation", "reconciliation_state")),
"capabilities": _copy(capabilities, ("legacy_writes_enabled", "governed_writes_enabled", "force_push", "main_write", "tag_write", "baseline_write", "production_write")),
"credential_broker": _copy(credential_broker, ("status", "provider")),
"secret_material_exposed": False,
}
def getControlPlaneSnapshot(**kwargs: Any) -> dict[str, Any]:
return build_control_plane_snapshot(**kwargs)
@@ -0,0 +1,124 @@
"""Metadata-only, server-side credential lease broker."""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
import os
from pathlib import Path
import secrets
import re
from typing import Any, Callable, Mapping
@dataclass(frozen=True, slots=True)
class CredentialLease:
lease_id: str
task_id: str
repository: str
target_branch: str
generation: int
issued_at: str
expires_at: str
scopes: tuple[str, ...]
provider: str
state: str = "ACTIVE"
scope_type: str = "system"
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
run_id: str | None = None
def as_dict(self) -> dict[str, Any]:
return {"lease_id": self.lease_id, "task_id": self.task_id, "repository": self.repository, "target_branch": self.target_branch, "generation": self.generation, "issued_at": self.issued_at, "expires_at": self.expires_at, "scopes": list(self.scopes), "provider": self.provider, "state": self.state, "scope_type": self.scope_type, "tenant_id": self.tenant_id, "user_id": self.user_id, "project_id": self.project_id, "run_id": self.run_id}
class FileCredentialProvider:
def __init__(self, token_path: str | os.PathLike[str], *, actor: str = "aios-autonomy-agent") -> None:
self.token_path = Path(token_path)
self.actor = actor
def environment(self, lease: CredentialLease) -> dict[str, str]:
if not self.token_path.is_file():
raise RuntimeError("CREDENTIAL_PROVIDER_UNAVAILABLE")
return {"CREDENTIAL_FILE": str(self.token_path), "GITEA_MACHINE_ACTOR": self.actor}
class CredentialBroker:
def __init__(self, provider: Any, *, now: Callable[[], datetime] | None = None) -> None:
self.provider = provider
self.now = now or (lambda: datetime.now(timezone.utc))
self._leases: dict[str, CredentialLease] = {}
def issue(self, *, task_id: str, repository: str, target_branch: str, generation: int, scopes: tuple[str, ...] = ("git:push", "gitea:read"), lease_seconds: int = 900, scope_type: str = "system", tenant_id: str | None = None, user_id: str | None = None, project_id: str | None = None, run_id: str | None = None) -> dict[str, Any]:
if repository != "aios/aios" or not target_branch.startswith("agent/") or generation < 1 or not 1 <= lease_seconds <= 86400:
raise ValueError("CREDENTIAL_LEASE_BINDING_INVALID")
if scope_type == "tenant":
if not tenant_id:
raise ValueError("CREDENTIAL_SCOPE_DENY")
elif scope_type in {"system", "global"}:
if any((tenant_id, user_id, project_id, run_id)):
raise ValueError("CREDENTIAL_SCOPE_DENY")
else:
raise ValueError("CREDENTIAL_SCOPE_DENY")
now = self.now().astimezone(timezone.utc)
lease = CredentialLease("lease-" + secrets.token_hex(16), task_id, repository, target_branch, generation, now.isoformat(), (now + timedelta(seconds=lease_seconds)).isoformat(), tuple(scopes), type(self.provider).__name__, "ACTIVE", scope_type, tenant_id, user_id, project_id, run_id)
self._leases[lease.lease_id] = lease
return lease.as_dict()
def _get(self, lease_id: str) -> CredentialLease:
lease = self._leases.get(lease_id)
if lease is None:
raise KeyError("CREDENTIAL_LEASE_NOT_FOUND")
if lease.state == "ACTIVE" and self.now().astimezone(timezone.utc) >= datetime.fromisoformat(lease.expires_at):
lease = CredentialLease(**{**lease.as_dict(), "scopes": tuple(lease.scopes), "state": "EXPIRED"})
self._leases[lease_id] = lease
return lease
def environment(self, lease_id: str) -> Mapping[str, str]:
lease = self._get(lease_id)
if lease.state != "ACTIVE":
raise RuntimeError("CREDENTIAL_LEASE_NOT_ACTIVE")
environment = self.provider.environment(lease)
for key, value in environment.items():
upper = str(key).upper()
if not isinstance(value, str):
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
sensitive = any(marker in upper for marker in ("CREDENTIAL", "TOKEN", "SECRET", "PASSWORD", "PRIVATE_KEY", "API_KEY"))
if re.search(r"(?i)bearer\s+\S+|(?:token|secret|password|api[_-]?key)\s*[:=]", value):
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
if sensitive and upper.endswith("_FILE"):
candidate = Path(value)
credential_root = Path("/run/credentials")
try:
resolved = candidate.resolve(strict=False)
except OSError as exc:
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY") from exc
if not candidate.is_absolute() or not (resolved == credential_root or credential_root in resolved.parents):
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
elif sensitive and upper.endswith("_REFERENCE"):
if re.fullmatch(r"(?:systemd-credential|vault|secret-manager):[A-Za-z0-9][A-Za-z0-9._:/#-]{0,255}", value) is None:
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
elif sensitive:
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
return environment
def authorize(self, lease_id: str, *, task_id: str, operation: str, scope_type: str = "system", tenant_id: str | None = None, user_id: str | None = None, project_id: str | None = None, run_id: str | None = None) -> Mapping[str, str]:
lease = self._get(lease_id)
if lease.state != "ACTIVE" or lease.task_id != task_id or operation not in lease.scopes:
raise RuntimeError("CREDENTIAL_SCOPE_DENY")
requested = (scope_type, tenant_id, user_id, project_id, run_id)
actual = (lease.scope_type, lease.tenant_id, lease.user_id, lease.project_id, lease.run_id)
if requested != actual:
raise RuntimeError("CREDENTIAL_SCOPE_DENY")
return self.environment(lease_id)
def status(self, lease_id: str) -> dict[str, Any]:
return self._get(lease_id).as_dict()
def revoke(self, lease_id: str) -> dict[str, Any]:
lease = self._get(lease_id)
if lease.state == "REVOKED":
return lease.as_dict()
updated = CredentialLease(**{**lease.as_dict(), "scopes": tuple(lease.scopes), "state": "REVOKED"})
self._leases[lease_id] = updated
return updated.as_dict()
@@ -0,0 +1,710 @@
"""Credential-isolated external Agent runtime executor for Online Agent Tasks.
Task payloads never choose commands. A root-owned runtime profile chooses the executable and
I/O mode. The Online Agent worker writes a hash-bound request into an exchange directory; this
uncredentialed process executes Hermes/OpenClaw and returns a bounded, sanitized result.
"""
from __future__ import annotations
from dataclasses import dataclass
import hashlib
import json
import os
from pathlib import Path
import re
import subprocess
import tempfile
import time
from typing import Any, Mapping, Sequence
REQUEST_SCHEMA = "aios.external-agent-runtime.request.v1"
RESULT_SCHEMA = "aios.external-agent-runtime.result.v1"
PROFILE_SCHEMA = "aios.external-agent-runtime.profile.v1"
SUPPORTED_RUNTIMES = frozenset({"hermes", "openclaw"})
REQUEST_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{15,254}$")
RUNTIME_RE = re.compile(r"^[a-z][a-z0-9_-]{2,31}$")
SENSITIVE_RE = re.compile(
r"(?i)(authorization\s*[:=]\s*bearer\s+\S+|"
r"(?:password|passwd|secret|token|api[_-]?key)\s*[:=]\s*\S+)"
)
SECRET_ENV_MARKERS = (
"TOKEN",
"SECRET",
"PASSWORD",
"CREDENTIAL",
"AUTHORIZATION",
"PRIVATE_KEY",
"API_KEY",
)
PROFILE_ENV_NAME_RE = re.compile(r"^[A-Z][A-Z0-9_]{1,63}$")
PROFILE_CREDENTIAL_NAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.:-]{2,127}$")
PROFILE_CREDENTIAL_KEY_RE = re.compile(r"^[A-Z][A-Z0-9_]{1,63}$")
PROVIDER_CREDENTIAL_ENV_NAMES = frozenset({"OPENAI_API_KEY"})
PROFILE_FIXED_ENV_DENY = frozenset(
{
"PATH",
"HOME",
"NO_COLOR",
"LC_ALL",
"LANG",
"LD_PRELOAD",
"LD_LIBRARY_PATH",
"PYTHONPATH",
"NODE_OPTIONS",
"BASH_ENV",
"SHELL",
"IFS",
}
)
MAX_REQUEST_BYTES = 256 * 1024
MAX_OUTPUT_BYTES = 128 * 1024
MAX_PROFILE_BYTES = 64 * 1024
MAX_CREDENTIAL_BYTES = 64 * 1024
DEFAULT_PROFILE_ROOT = Path("/etc/aios/external-agent-runtime.d")
DEFAULT_EXCHANGE_ROOT = Path("/var/lib/aios-external-agent-executor")
ALLOWED_EXECUTABLE_ROOTS = (
Path("/usr/bin"),
Path("/usr/local/bin"),
Path("/opt/aios/bin"),
Path("/opt/hermes"),
Path("/opt/openclaw"),
Path("/opt/aios-hermes-agent"),
Path("/opt/aios/openclaw-engineering"),
)
class ExternalAgentRuntimeError(RuntimeError):
pass
def canonical(value: object) -> bytes:
return json.dumps(
value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
).encode("utf-8")
def sha256(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def sanitize_text(value: str) -> str:
return SENSITIVE_RE.sub("[REDACTED]", str(value))
def _bounded_text(value: str, limit: int = MAX_OUTPUT_BYTES) -> str:
data = sanitize_text(value).encode("utf-8", "replace")
if len(data) <= limit:
return data.decode("utf-8", "replace")
marker = b"\n...[TRUNCATED]...\n"
half = max(1, (limit - len(marker)) // 2)
return (data[:half] + marker + data[-half:]).decode("utf-8", "replace")
def _safe_runtime(value: object) -> str:
runtime = str(value or "").strip().lower()
if RUNTIME_RE.fullmatch(runtime) is None or runtime not in SUPPORTED_RUNTIMES:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_RUNTIME_UNSUPPORTED")
return runtime
def _assert_no_secret_env() -> None:
leaked = sorted(
key
for key, value in os.environ.items()
if value and any(marker in key.upper() for marker in SECRET_ENV_MARKERS)
)
# CREDENTIALS_DIRECTORY itself is systemd metadata; the directory is expected to be hidden.
leaked = [key for key in leaked if key != "CREDENTIALS_DIRECTORY"]
if leaked:
raise ExternalAgentRuntimeError(
"EXTERNAL_AGENT_CREDENTIAL_ENV_EXPOSED:" + ",".join(leaked)
)
def _safe_executable(value: object) -> str:
if not isinstance(value, str) or not value.startswith("/"):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_EXECUTABLE_ABSOLUTE_REQUIRED")
path = Path(value)
if path.is_symlink():
resolved = path.resolve(strict=True)
else:
resolved = path.resolve(strict=True)
if not resolved.is_file() or not os.access(resolved, os.X_OK):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_EXECUTABLE_INVALID")
if not any(resolved == root or root in resolved.parents for root in ALLOWED_EXECUTABLE_ROOTS):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_EXECUTABLE_ROOT_DENY")
return str(resolved)
def _safe_profile_script(value: object) -> str:
if not isinstance(value, str) or not value.startswith("/"):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_SCRIPT_ABSOLUTE_REQUIRED")
path = Path(value)
if path.is_symlink():
resolved = path.resolve(strict=True)
else:
resolved = path.resolve(strict=True)
if not resolved.is_file():
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_SCRIPT_INVALID")
if not any(resolved == root or root in resolved.parents for root in ALLOWED_EXECUTABLE_ROOTS):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_SCRIPT_ROOT_DENY")
return str(resolved)
def _profile_environment(value: object) -> tuple[tuple[str, str], ...]:
if value is None:
return ()
if not isinstance(value, Mapping):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PROFILE_ENVIRONMENT_INVALID")
result: list[tuple[str, str]] = []
for raw_name, raw_value in value.items():
if not isinstance(raw_name, str) or PROFILE_ENV_NAME_RE.fullmatch(raw_name) is None:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PROFILE_ENVIRONMENT_NAME_INVALID")
if raw_name in PROFILE_FIXED_ENV_DENY:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PROFILE_ENVIRONMENT_NAME_DENY")
if any(marker in raw_name.upper() for marker in SECRET_ENV_MARKERS):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PROFILE_SECRET_ENV_DENY")
if not isinstance(raw_value, str) or "\x00" in raw_value or "\n" in raw_value or "\r" in raw_value:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PROFILE_ENVIRONMENT_VALUE_INVALID")
if len(raw_value) > 4096:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PROFILE_ENVIRONMENT_VALUE_TOO_LARGE")
result.append((raw_name, raw_value))
return tuple(sorted(result))
def _profile_credential_environment(
value: object,
) -> tuple[tuple[str, str, str, str], ...]:
if value is None:
return ()
if not isinstance(value, Mapping):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PROFILE_CREDENTIAL_ENV_INVALID")
result: list[tuple[str, str, str, str]] = []
for raw_env_name, raw_spec in value.items():
if (
not isinstance(raw_env_name, str)
or raw_env_name not in PROVIDER_CREDENTIAL_ENV_NAMES
or PROFILE_ENV_NAME_RE.fullmatch(raw_env_name) is None
):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_ENV_NAME_DENY")
if not isinstance(raw_spec, Mapping):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_SPEC_INVALID")
credential = raw_spec.get("credential")
fmt = str(raw_spec.get("format") or "dotenv")
key = raw_spec.get("key")
if (
not isinstance(credential, str)
or PROFILE_CREDENTIAL_NAME_RE.fullmatch(credential) is None
):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_NAME_INVALID")
if fmt not in {"dotenv", "raw"}:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_FORMAT_INVALID")
if fmt == "dotenv":
if not isinstance(key, str) or PROFILE_CREDENTIAL_KEY_RE.fullmatch(key) is None:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_KEY_INVALID")
elif key is not None:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_KEY_UNEXPECTED")
result.append((raw_env_name, credential, fmt, str(key or "")))
return tuple(sorted(result))
@dataclass(frozen=True)
class RuntimeProfile:
runtime: str
argv: tuple[str, ...]
input_mode: str
output_mode: str
timeout_seconds: int
max_output_bytes: int
environment: tuple[tuple[str, str], ...]
credential_env: tuple[tuple[str, str, str, str], ...]
@classmethod
def load(
cls,
runtime: str,
profile_root: Path = DEFAULT_PROFILE_ROOT,
) -> "RuntimeProfile":
runtime = _safe_runtime(runtime)
root = Path(profile_root)
if root.is_symlink():
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PROFILE_ROOT_SYMLINK_DENY")
path = root / f"{runtime}.json"
if path.is_symlink() or not path.is_file() or path.stat().st_size > MAX_PROFILE_BYTES:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PROFILE_UNAVAILABLE")
try:
value = json.loads(path.read_text(encoding="utf-8"))
except json.JSONDecodeError as exc:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PROFILE_INVALID") from exc
if not isinstance(value, Mapping) or value.get("schema") != PROFILE_SCHEMA:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PROFILE_INVALID")
if _safe_runtime(value.get("runtime")) != runtime:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PROFILE_RUNTIME_MISMATCH")
argv = value.get("argv")
if (
not isinstance(argv, Sequence)
or isinstance(argv, (str, bytes))
or not argv
or not all(isinstance(item, str) and item and "\x00" not in item for item in argv)
):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PROFILE_ARGV_INVALID")
executable = _safe_executable(argv[0])
normalized = [executable, *list(argv[1:])]
if any(item in {"sh", "bash", "-c", "--command"} for item in normalized[1:]):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_SHELL_MODE_DENY")
joined_argv = " ".join(normalized)
if re.search(r"(?i)(token|secret|password|api[_-]?key|authorization)\s*[:=]", joined_argv):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PROFILE_SECRET_ARG_DENY")
if Path(executable).name in {"node", "nodejs"}:
if len(normalized) < 2 or normalized[1].startswith("-"):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_NODE_SCRIPT_REQUIRED")
normalized[1] = _safe_profile_script(normalized[1])
if any(item in {"-e", "--eval", "-p", "--print", "-r", "--require"} for item in normalized[1:]):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_NODE_CODE_MODE_DENY")
input_mode = str(value.get("input_mode") or "stdin_text")
output_mode = str(value.get("output_mode") or "stdout_text")
if input_mode not in {"stdin_text", "stdin_json", "file_json"}:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_INPUT_MODE_INVALID")
if output_mode not in {"stdout_text", "stdout_json", "file_json"}:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_OUTPUT_MODE_INVALID")
timeout = int(value.get("timeout_seconds") or 900)
max_output = int(value.get("max_output_bytes") or MAX_OUTPUT_BYTES)
if not 1 <= timeout <= 3600:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_TIMEOUT_INVALID")
if not 1024 <= max_output <= MAX_OUTPUT_BYTES:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_OUTPUT_LIMIT_INVALID")
environment = _profile_environment(value.get("environment"))
credential_env = _profile_credential_environment(value.get("credential_env"))
return cls(
runtime,
tuple(normalized),
input_mode,
output_mode,
timeout,
max_output,
environment,
credential_env,
)
def validate_request(value: Mapping[str, Any]) -> dict[str, Any]:
if value.get("schema") != REQUEST_SCHEMA:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_REQUEST_SCHEMA_INVALID")
request_id = str(value.get("request_id") or "")
if REQUEST_ID_RE.fullmatch(request_id) is None:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_REQUEST_ID_INVALID")
runtime = _safe_runtime(value.get("runtime"))
task_id = str(value.get("task_id") or "")
execution_digest = str(value.get("execution_digest") or "")
if not task_id or not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._:-]{2,179}", task_id):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_TASK_ID_INVALID")
if re.fullmatch(r"[0-9a-f]{64}", execution_digest) is None:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_EXECUTION_DIGEST_INVALID")
payload = value.get("payload")
if not isinstance(payload, Mapping):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PAYLOAD_INVALID")
encoded = canonical(payload)
if len(encoded) > MAX_REQUEST_BYTES:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_PAYLOAD_TOO_LARGE")
goal = payload.get("goal")
instructions = payload.get("instructions")
if not isinstance(goal, str) or not goal.strip():
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_GOAL_REQUIRED")
if instructions is not None and not isinstance(instructions, str):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_INSTRUCTIONS_INVALID")
return {
"schema": REQUEST_SCHEMA,
"request_id": request_id,
"runtime": runtime,
"task_id": task_id,
"execution_digest": execution_digest,
"payload": dict(payload),
}
def render_prompt(payload: Mapping[str, Any]) -> str:
goal = str(payload.get("goal") or "").strip()
instructions = str(payload.get("instructions") or "").strip()
criteria = payload.get("success_criteria") or []
constraints = payload.get("constraints") or {}
lines = [
"AIOS Online Agent Task",
"",
"Goal:",
goal,
]
if instructions:
lines.extend(["", "Instructions:", instructions])
if criteria:
lines.extend(["", "Success criteria:", json.dumps(criteria, ensure_ascii=False, sort_keys=True)])
if constraints:
lines.extend(["", "Constraints:", json.dumps(constraints, ensure_ascii=False, sort_keys=True)])
lines.extend([
"",
"Return the final task result only. Do not claim repository, production, credential, or control-plane side effects unless they are explicitly represented in the task payload.",
])
return "\n".join(lines)
def _argv_with_files(
profile: RuntimeProfile,
input_path: Path,
output_path: Path,
prompt: str,
) -> list[str]:
replacements = {
"{input_json}": str(input_path),
"{output_json}": str(output_path),
"{prompt}": prompt,
}
result = []
for item in profile.argv:
for marker, replacement in replacements.items():
item = item.replace(marker, replacement)
result.append(item)
if profile.input_mode == "file_json" and not any("{input_json}" in item for item in profile.argv):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_INPUT_PLACEHOLDER_REQUIRED")
if profile.output_mode == "file_json" and not any("{output_json}" in item for item in profile.argv):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_OUTPUT_PLACEHOLDER_REQUIRED")
return result
def _credential_environment(profile: RuntimeProfile) -> dict[str, str]:
if not profile.credential_env:
return {}
raw_root = os.environ.get("CREDENTIALS_DIRECTORY")
if not raw_root or not raw_root.startswith("/"):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_DIRECTORY_MISSING")
root = Path(raw_root)
if root.is_symlink() or not root.is_dir():
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_DIRECTORY_INVALID")
result: dict[str, str] = {}
for env_name, credential, fmt, key in profile.credential_env:
path = root / credential
if path.is_symlink() or not path.is_file():
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_FILE_INVALID")
try:
if path.stat().st_size > MAX_CREDENTIAL_BYTES:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_TOO_LARGE")
raw = path.read_bytes()
except OSError as exc:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_READ_FAILED") from exc
if b"\x00" in raw:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_CONTENT_INVALID")
try:
text = raw.decode("utf-8")
except UnicodeDecodeError as exc:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_CONTENT_INVALID") from exc
selected = ""
if fmt == "raw":
selected = text.strip()
else:
found = False
for line in text.splitlines():
candidate = line.strip()
if not candidate or candidate.startswith("#"):
continue
if candidate.startswith("export "):
candidate = candidate[7:].lstrip()
if "=" not in candidate:
continue
candidate_key, candidate_value = candidate.split("=", 1)
if candidate_key.strip() != key:
continue
if found:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_KEY_DUPLICATE")
selected = candidate_value.strip()
if len(selected) >= 2 and selected[0] == selected[-1] and selected[0] in {"'", '"'}:
selected = selected[1:-1]
found = True
if not found:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_KEY_MISSING")
if not selected or "\x00" in selected or "\n" in selected or "\r" in selected:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_VALUE_INVALID")
if len(selected) > MAX_CREDENTIAL_BYTES:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_CREDENTIAL_VALUE_TOO_LARGE")
result[env_name] = selected
return result
def _parse_output(
profile: RuntimeProfile,
stdout: str,
output_path: Path,
) -> tuple[str, object]:
if profile.output_mode == "file_json":
if output_path.is_symlink() or not output_path.is_file():
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_OUTPUT_FILE_MISSING")
raw = output_path.read_bytes()
if len(raw) > profile.max_output_bytes:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_OUTPUT_TOO_LARGE")
text = raw.decode("utf-8", "replace")
try:
parsed = json.loads(text)
except json.JSONDecodeError as exc:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_OUTPUT_JSON_INVALID") from exc
return _bounded_text(text, profile.max_output_bytes), parsed
if len(stdout.encode("utf-8", "replace")) > profile.max_output_bytes:
stdout = _bounded_text(stdout, profile.max_output_bytes)
if profile.output_mode == "stdout_json":
try:
parsed = json.loads(stdout)
except json.JSONDecodeError as exc:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_OUTPUT_JSON_INVALID") from exc
return _bounded_text(stdout, profile.max_output_bytes), parsed
return _bounded_text(stdout, profile.max_output_bytes), None
def execute_request(
request_path: Path,
result_path: Path,
*,
profile_root: Path = DEFAULT_PROFILE_ROOT,
) -> None:
_assert_no_secret_env()
if request_path.is_symlink() or not request_path.is_file():
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_REQUEST_FILE_INVALID")
raw_request = request_path.read_bytes()
if len(raw_request) > MAX_REQUEST_BYTES:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_REQUEST_TOO_LARGE")
try:
request = validate_request(json.loads(raw_request.decode("utf-8")))
except json.JSONDecodeError as exc:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_REQUEST_JSON_INVALID") from exc
profile = RuntimeProfile.load(request["runtime"], profile_root)
with tempfile.TemporaryDirectory(prefix=f"aios-{request['runtime']}-") as temporary:
temp_root = Path(temporary)
(temp_root / "home").mkdir()
input_path = temp_root / "input.json"
output_path = temp_root / "output.json"
input_path.write_bytes(canonical(request["payload"]))
prompt = render_prompt(request["payload"])
argv = _argv_with_files(profile, input_path, output_path, prompt)
stdin_data = None
if profile.input_mode == "stdin_json":
stdin_data = canonical(request["payload"]).decode("utf-8")
elif profile.input_mode == "stdin_text":
stdin_data = prompt
child_env = {
"PATH": "/usr/bin:/usr/local/bin:/bin",
"LC_ALL": "C.UTF-8",
"LANG": "C.UTF-8",
"HOME": str(temp_root / "home"),
"NO_COLOR": "1",
}
child_env.update(dict(profile.environment))
child_env.update(_credential_environment(profile))
started = time.monotonic()
try:
completed = subprocess.run(
argv,
input=stdin_data,
capture_output=True,
text=True,
check=False,
shell=False,
timeout=profile.timeout_seconds,
cwd=str(temp_root),
env=child_env,
)
except subprocess.TimeoutExpired as exc:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_TIMEOUT") from exc
if completed.returncode != 0:
raise ExternalAgentRuntimeError(
f"EXTERNAL_AGENT_PROCESS_FAILED:{completed.returncode}:"
+ _bounded_text(completed.stderr, 2048)
)
output_text, output_json = _parse_output(profile, completed.stdout, output_path)
result: dict[str, Any] = {
"schema": RESULT_SCHEMA,
"request_id": request["request_id"],
"request_sha256": sha256(canonical(request)),
"runtime": request["runtime"],
"task_id": request["task_id"],
"execution_digest": request["execution_digest"],
"exit_code": completed.returncode,
"duration_ms": int((time.monotonic() - started) * 1000),
"output_text": output_text,
"output_sha256": sha256(output_text.encode("utf-8", "replace")),
"stderr_sha256": sha256(completed.stderr.encode("utf-8", "replace")),
"credential_material_exposed": False,
"control_plane_access_granted": False,
}
if output_json is not None:
result["output_json"] = output_json
result["result_sha256"] = sha256(canonical(result))
result_path.parent.mkdir(parents=True, exist_ok=True)
fd, name = tempfile.mkstemp(prefix=f".{result_path.name}.", dir=result_path.parent)
try:
os.fchmod(fd, 0o640)
with os.fdopen(fd, "w", encoding="utf-8") as handle:
json.dump(result, handle, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
handle.write("\n")
handle.flush()
os.fsync(handle.fileno())
os.replace(name, result_path)
finally:
if os.path.exists(name):
os.unlink(name)
class ExternalAgentExchange:
"""Worker-side idempotent request/result exchange."""
def __init__(self, root: Path = DEFAULT_EXCHANGE_ROOT) -> None:
self.root = Path(root)
def execute(
self,
*,
request_id: str,
runtime: str,
task_id: str,
execution_digest: str,
payload: Mapping[str, Any],
timeout_seconds: int = 1800,
) -> dict[str, Any]:
request = validate_request(
{
"schema": REQUEST_SCHEMA,
"request_id": request_id,
"runtime": runtime,
"task_id": task_id,
"execution_digest": execution_digest,
"payload": dict(payload),
}
)
requests = self.root / "requests"
results = self.root / "results"
failed = self.root / "failed"
for directory in (requests, results, failed):
directory.mkdir(parents=True, exist_ok=True)
request_path = requests / f"{request_id}.json"
result_path = results / f"{request_id}.json"
failed_path = failed / f"{request_id}.json"
request_sha = sha256(canonical(request))
if result_path.is_file():
return self._validated_result(result_path, request, request_sha)
if failed_path.is_file():
value = json.loads(failed_path.read_text(encoding="utf-8"))
raise ExternalAgentRuntimeError(
str(value.get("error") or "EXTERNAL_AGENT_EXECUTOR_FAILED")
)
if request_path.exists():
try:
existing = json.loads(request_path.read_text(encoding="utf-8"))
except json.JSONDecodeError as exc:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_EXISTING_REQUEST_INVALID") from exc
if sha256(canonical(existing)) != request_sha:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_REQUEST_ID_CONFLICT")
else:
fd, name = tempfile.mkstemp(prefix=f".{request_path.name}.", dir=requests)
try:
os.fchmod(fd, 0o640)
with os.fdopen(fd, "w", encoding="utf-8") as handle:
json.dump(request, handle, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
handle.write("\n")
handle.flush()
os.fsync(handle.fileno())
os.replace(name, request_path)
finally:
if os.path.exists(name):
os.unlink(name)
deadline = time.monotonic() + timeout_seconds
while time.monotonic() < deadline:
if result_path.is_file():
return self._validated_result(result_path, request, request_sha)
if failed_path.is_file():
try:
value = json.loads(failed_path.read_text(encoding="utf-8"))
except json.JSONDecodeError:
value = {"error": "EXTERNAL_AGENT_EXECUTOR_FAILED"}
raise ExternalAgentRuntimeError(
str(value.get("error") or "EXTERNAL_AGENT_EXECUTOR_FAILED")
)
time.sleep(0.25)
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_EXCHANGE_TIMEOUT")
@staticmethod
def _validated_result(
result_path: Path,
request: Mapping[str, Any],
request_sha: str,
) -> dict[str, Any]:
try:
value = json.loads(result_path.read_text(encoding="utf-8"))
except json.JSONDecodeError as exc:
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_RESULT_INVALID") from exc
if (
not isinstance(value, Mapping)
or value.get("schema") != RESULT_SCHEMA
or value.get("request_id") != request["request_id"]
or value.get("request_sha256") != request_sha
or value.get("runtime") != request["runtime"]
or value.get("task_id") != request["task_id"]
or value.get("execution_digest") != request["execution_digest"]
):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_RESULT_BINDING_INVALID")
material = dict(value)
declared = str(material.pop("result_sha256", ""))
if declared != sha256(canonical(material)):
raise ExternalAgentRuntimeError("EXTERNAL_AGENT_RESULT_DIGEST_INVALID")
return dict(value)
def serve(
root: Path = DEFAULT_EXCHANGE_ROOT,
profile_root: Path = DEFAULT_PROFILE_ROOT,
) -> None:
_assert_no_secret_env()
root = Path(root)
for directory in (root / "requests", root / "results", root / "failed"):
directory.mkdir(parents=True, exist_ok=True)
while True:
for request_path in sorted((root / "requests").glob("*.json")):
if request_path.is_symlink():
continue
result_path = root / "results" / request_path.name
failed_path = root / "failed" / request_path.name
try:
execute_request(request_path, result_path, profile_root=profile_root)
request_path.unlink(missing_ok=True)
except Exception as exc:
failed_path.write_text(
json.dumps(
{"error": sanitize_text(str(exc) or type(exc).__name__)},
ensure_ascii=False,
sort_keys=True,
)
+ "\n",
encoding="utf-8",
)
request_path.unlink(missing_ok=True)
time.sleep(0.25)
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--root", default=str(DEFAULT_EXCHANGE_ROOT))
parser.add_argument("--profile-root", default=str(DEFAULT_PROFILE_ROOT))
parser.add_argument("--request")
parser.add_argument("--result")
args = parser.parse_args()
if args.request and args.result:
execute_request(
Path(args.request),
Path(args.result),
profile_root=Path(args.profile_root),
)
else:
serve(Path(args.root), Path(args.profile_root))
@@ -0,0 +1,125 @@
"""Non-Owner Gitea machine capability and branch reconciliation boundary."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Mapping, Protocol
class MachineBrokerError(RuntimeError):
pass
class GiteaTransport(Protocol):
def compare_and_swap_branch(self, branch: str, expected_oid: str, new_oid: str) -> str: ...
def create_or_update_pr(self, branch: str, target: str, title: str, body: str) -> Mapping[str, Any]: ...
def publish_status(self, branch: str, state: str, description: str) -> Mapping[str, Any]: ...
def protected_merge(self, pr_number: int) -> Mapping[str, Any]: ...
def read_branch(self, branch: str) -> str | None: ...
def whoami(self) -> Mapping[str, Any]: ...
def repository(self, owner: str, name: str) -> Mapping[str, Any]: ...
@dataclass(frozen=True, slots=True)
class MachineCredentialProjection:
"""Opaque systemd credential reference; the credential value is never accepted."""
reference: str
account: str
owner_account: str = "aiman"
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "MachineCredentialProjection":
if set(value) != {"scheme", "name", "account"}:
raise MachineBrokerError("CREDENTIAL_REFERENCE_INVALID")
if value["scheme"] != "systemd-credential" or value["name"] != "gitea-machine-token":
raise MachineBrokerError("CREDENTIAL_REFERENCE_INVALID")
return cls(f"{value['scheme']}:{value['name']}", str(value["account"]))
def __post_init__(self) -> None:
if self.reference != "systemd-credential:gitea-machine-token":
raise MachineBrokerError("CREDENTIAL_REFERENCE_INVALID")
if not self.account or self.account == self.owner_account:
raise MachineBrokerError("OWNER_MACHINE_IDENTITY_REJECTED")
@dataclass(frozen=True, slots=True)
class CampaignCapabilityReceipt:
campaign_id: str
task_id: str
repository: str
branch: str
expected_oid: str
capabilities: frozenset[str]
denied: frozenset[str]
def require(self, capability: str) -> None:
if capability in self.denied or capability not in self.capabilities:
raise MachineBrokerError("CAPABILITY_DENIED")
if self.branch in {"main", "master"}:
raise MachineBrokerError("PROTECTED_BRANCH_DENIED")
class GiteaMachineBroker:
def __init__(self, transport: GiteaTransport, credential: MachineCredentialProjection, receipt: CampaignCapabilityReceipt) -> None:
self.transport = transport
self.credential = credential
self.receipt = receipt
def verify_identity(self) -> Mapping[str, Any]:
identity = dict(self.transport.whoami())
login = identity.get("login") or identity.get("username")
if not isinstance(login, str) or not login or login == self.credential.owner_account or login != self.credential.account:
raise MachineBrokerError("GITEA_MACHINE_IDENTITY_INVALID")
owner, name = self.receipt.repository.split("/", 1)
repo = dict(self.transport.repository(owner, name))
if repo.get("full_name") != self.receipt.repository:
raise MachineBrokerError("GITEA_REPOSITORY_MISMATCH")
return {"login": login, "repository": self.receipt.repository, "owner_account": self.credential.owner_account}
def push_branch(self, new_oid: str) -> str:
self.receipt.require("exact_task_branch_push")
return self.transport.compare_and_swap_branch(self.receipt.branch, self.receipt.expected_oid, new_oid)
def create_pr(self, title: str, body: str) -> Mapping[str, Any]:
self.receipt.require("pr_create_update")
return self.transport.create_or_update_pr(self.receipt.branch, "main", title, body)
def status(self, state: str, description: str, context: str = "aios/durable-release") -> Mapping[str, Any]:
self.receipt.require("status_publish")
return self.transport.publish_status(self.receipt.branch, state, description, context)
def merge(self, pr_number: int) -> Mapping[str, Any]:
self.receipt.require("protected_merge")
return self.transport.protected_merge(pr_number)
def readback(self) -> str | None:
self.receipt.require("remote_readback")
return self.transport.read_branch(self.receipt.branch)
def reconcile_exact_branch(authority_oid: str, gitea_oid: str | None, expected_oid: str) -> str:
if authority_oid != expected_oid:
return "AUTHORITY_EXPECTATION_MISMATCH"
if gitea_oid is None:
return "CREATE_REQUIRED"
if gitea_oid == expected_oid:
return "ALREADY_IN_SYNC"
return "MANUAL_OWNER_REVIEW_REQUIRED"
def reconcile_fast_forward(
remote_oid: str | None,
expected_old_oid: str,
new_oid: str,
*,
old_is_ancestor_of_new: bool,
) -> str:
"""Return a bounded action for an active Job/lease branch scope."""
if remote_oid is None:
return "CREATE_REQUIRED"
if remote_oid == new_oid:
return "ALREADY_IN_SYNC"
if remote_oid == expected_old_oid and old_is_ancestor_of_new:
return "FAST_FORWARD_REQUIRED"
return "MANUAL_OWNER_REVIEW_REQUIRED"
@@ -0,0 +1,208 @@
"""Gitea HTTP/Git transport using only a projected systemd credential."""
from __future__ import annotations
from dataclasses import dataclass
import json
import os
from pathlib import Path
import subprocess
import tempfile
from typing import Any, Mapping
from urllib.error import HTTPError, URLError
from urllib.parse import quote, urlencode
from urllib.request import Request, urlopen
class GiteaTransportError(RuntimeError):
pass
def _safe_oid(value: str, field: str) -> str:
if not isinstance(value, str) or len(value) != 40 or any(ch not in "0123456789abcdef" for ch in value):
raise GiteaTransportError(f"{field}_INVALID")
return value
@dataclass(slots=True)
class GiteaMachineTransport:
api_base: str
repository_id: str
remote_url: str
authority_git_dir: Path
credential_name: str = "gitea-machine-token"
actor: str = "aios-autonomy-agent"
timeout: int = 20
expected_merge_head: str | None = None
expected_merge_base: str | None = None
def __post_init__(self) -> None:
if self.credential_name != "gitea-machine-token":
raise GiteaTransportError("CREDENTIAL_NAME_INVALID")
if "/" not in self.repository_id or self.repository_id.startswith("/"):
raise GiteaTransportError("REPOSITORY_INVALID")
if not self.api_base.startswith("https://") and not self.api_base.startswith("http://127.0.0.1"):
raise GiteaTransportError("GITEA_API_SCOPE_INVALID")
if not self.remote_url.startswith("https://"):
raise GiteaTransportError("GITEA_REMOTE_SCOPE_INVALID")
@property
def _credential_path(self) -> Path:
directory = os.environ.get("CREDENTIALS_DIRECTORY", "")
if not directory:
raise GiteaTransportError("CREDENTIAL_PROJECTION_MISSING")
path = Path(directory) / self.credential_name
if path.is_symlink() or not path.is_file():
raise GiteaTransportError("CREDENTIAL_PROJECTION_MISSING")
return path
def _token(self) -> str:
value = self._credential_path.read_text(encoding="utf-8").strip()
if not value:
raise GiteaTransportError("CREDENTIAL_EMPTY")
return value
def _request(self, method: str, path: str, body: Mapping[str, Any] | None = None) -> tuple[int, Any]:
data = None if body is None else json.dumps(dict(body), separators=(",", ":")).encode()
headers = {"Accept": "application/json", "Authorization": f"token {self._token()}"}
if data is not None:
headers["Content-Type"] = "application/json"
request = Request(self.api_base.rstrip("/") + path, data=data, headers=headers, method=method)
try:
with urlopen(request, timeout=self.timeout) as response:
raw = response.read()
return response.status, json.loads(raw.decode()) if raw else {}
except HTTPError as exc:
# Never include response bodies: Gitea may echo sensitive request context.
raise GiteaTransportError(f"GITEA_HTTP_{exc.code}") from exc
except (URLError, TimeoutError, OSError) as exc:
raise GiteaTransportError("GITEA_NETWORK_ERROR") from exc
def whoami(self) -> Mapping[str, Any]:
return self._request("GET", "/user")[1]
def repository_info(self) -> Mapping[str, Any]:
owner, name = self.repository_id.split("/", 1)
return self._request("GET", f"/repos/{quote(owner)}/{quote(name)}")[1]
def repository(self, owner: str, name: str) -> Mapping[str, Any]:
if f"{owner}/{name}" != self.repository_id:
raise GiteaTransportError("REPOSITORY_SCOPE_DENIED")
return self.repository_info()
def read_branch(self, branch: str) -> str | None:
owner, name = self.repository_id.split("/", 1)
try:
value = self._request("GET", f"/repos/{quote(owner)}/{quote(name)}/branches/{quote(branch, safe='/')}")[1]
except GiteaTransportError as exc:
if str(exc) == "GITEA_HTTP_404":
return None
raise
sha = value.get("commit", {}).get("id") if isinstance(value, dict) else None
return _safe_oid(sha, "remote_branch_oid")
def _askpass(self) -> tuple[Path, dict[str, str]]:
credential = self._credential_path
fd, name = tempfile.mkstemp(prefix="aios-gitea-askpass-", dir="/tmp")
path = Path(name)
os.fchmod(fd, 0o700)
script = "#!/bin/sh\ncase \"$1\" in *Username*) printf '%s\\n' \"$GITEA_MACHINE_ACTOR\" ;; *) cat \"$CREDENTIAL_FILE\" ;; esac\n"
with os.fdopen(fd, "w", encoding="utf-8") as stream:
stream.write(script)
stream.flush()
os.fsync(stream.fileno())
return path, {
"GIT_ASKPASS": str(path),
"GIT_TERMINAL_PROMPT": "0",
"GITEA_MACHINE_ACTOR": self.actor,
"CREDENTIAL_FILE": str(credential),
}
def _push_branch_fast_forward(self, branch: str, new_oid: str) -> None:
askpass, additions = self._askpass()
environment = os.environ.copy()
environment.update(additions)
try:
result = subprocess.run(
["git", f"--git-dir={self.authority_git_dir}", "push", self.remote_url, f"{new_oid}:refs/heads/{branch}"],
text=True,
capture_output=True,
check=False,
env=environment,
timeout=self.timeout,
)
if result.returncode != 0:
raise GiteaTransportError("GITEA_BRANCH_CREATE_FAILED")
finally:
askpass.unlink(missing_ok=True)
def compare_and_swap_branch(self, branch: str, expected_oid: str, new_oid: str) -> str:
_safe_oid(expected_oid, "expected_oid")
_safe_oid(new_oid, "new_oid")
current = self.read_branch(branch)
if current == new_oid:
return new_oid
if current is None:
self._push_branch_fast_forward(branch, new_oid)
readback = self.read_branch(branch)
if readback != new_oid:
raise GiteaTransportError("GITEA_BRANCH_CREATE_READBACK_MISMATCH")
return readback
if current != expected_oid:
raise GiteaTransportError("GITEA_BRANCH_EXPECTATION_MISMATCH")
# The preflight read binds the expected old tip. The Git receive-pack
# then enforces the actual fast-forward CAS while transferring the new
# objects; no force/lease override or arbitrary ref is permitted.
self._push_branch_fast_forward(branch, new_oid)
readback = self.read_branch(branch)
if readback != new_oid:
raise GiteaTransportError("GITEA_BRANCH_CAS_READBACK_MISMATCH")
return readback
def _pulls(self) -> list[Mapping[str, Any]]:
owner, name = self.repository_id.split("/", 1)
query = urlencode({"state": "all", "base_branch": "main", "limit": 100})
value = self._request("GET", f"/repos/{quote(owner)}/{quote(name)}/pulls?{query}")[1]
return value if isinstance(value, list) else []
def create_or_update_pr(self, branch: str, target: str, title: str, body: str) -> Mapping[str, Any]:
if target != "main":
raise GiteaTransportError("PR_BASE_SCOPE_DENIED")
head = self.read_branch(branch)
if head is None:
raise GiteaTransportError("PR_HEAD_MISSING")
matches = [p for p in self._pulls() if (p.get("head") or {}).get("ref") == branch and (p.get("base") or {}).get("ref") == target]
open_matches = [p for p in matches if p.get("state") == "open"]
if len(open_matches) > 1:
raise GiteaTransportError("MULTIPLE_MATCHING_PRS")
if open_matches:
if (open_matches[0].get("head") or {}).get("sha") != head:
raise GiteaTransportError("PR_HEAD_CONFLICT")
return open_matches[0]
owner, name = self.repository_id.split("/", 1)
return self._request("POST", f"/repos/{quote(owner)}/{quote(name)}/pulls", {"head": branch, "base": target, "title": title, "body": body, "allow_maintainer_edit": False})[1]
def publish_status(self, branch: str, state: str, description: str, context: str = "aios/durable-release") -> Mapping[str, Any]:
oid = self.read_branch(branch)
if oid is None:
raise GiteaTransportError("STATUS_BRANCH_MISSING")
owner, name = self.repository_id.split("/", 1)
return self._request("POST", f"/repos/{quote(owner)}/{quote(name)}/statuses/{oid}", {"state": state, "context": context, "description": description})[1]
def protected_merge(self, pr_number: int) -> Mapping[str, Any]:
if not isinstance(pr_number, int) or pr_number < 1:
raise GiteaTransportError("PR_NUMBER_INVALID")
owner, name = self.repository_id.split("/", 1)
pr = self._request("GET", f"/repos/{quote(owner)}/{quote(name)}/pulls/{pr_number}")[1]
if pr.get("state") != "open" or (pr.get("base") or {}).get("ref") != "main":
raise GiteaTransportError("PR_MERGE_BINDING_INVALID")
if pr.get("mergeable") is False:
raise GiteaTransportError("PR_NOT_MERGEABLE")
head_oid = (pr.get("head") or {}).get("sha")
base_oid = (pr.get("base") or {}).get("sha")
if self.expected_merge_head is not None and head_oid != self.expected_merge_head:
raise GiteaTransportError("PR_HEAD_EXPECTATION_MISMATCH")
if self.expected_merge_base is not None and base_oid != self.expected_merge_base:
raise GiteaTransportError("PR_BASE_EXPECTATION_MISMATCH")
body = {"do": "fast-forward-only", "force_merge": False, "merge_when_checks_succeed": False, "head_commit_id": head_oid}
return self._request("POST", f"/repos/{quote(owner)}/{quote(name)}/pulls/{pr_number}/merge", body)[1]
@@ -0,0 +1,93 @@
"""Canonical Gitea transport and slash-branch verification."""
from __future__ import annotations
from dataclasses import dataclass
import json
import os
from pathlib import Path
import subprocess
from typing import Any, Callable, Mapping, Optional
from urllib.parse import quote
from urllib.request import Request, urlopen
from aios_gitops_reconciliation_core import RemoteObservation
@dataclass(frozen=True, slots=True)
class GiteaPublisherConfig:
repository: str = "aios/aios"
api_base: str = "https://g.cnnetmkt.top/git/api/v1"
remote_name: str = "gitea-http"
class GiteaPublisher:
def __init__(self, workspace: Path, *, config: GiteaPublisherConfig = GiteaPublisherConfig(), credential: Any = None, api_get: Callable[[str], Any] | None = None, git_ref: Callable[[str], Optional[str]] | None = None) -> None:
self.workspace = Path(workspace)
self.config = config
self.credential = credential
self._api_get = api_get
self._git_ref = git_ref
@staticmethod
def validate_repository(repository: str) -> None:
if repository != "aios/aios":
raise ValueError("REPOSITORY_IDENTITY_MISMATCH")
@staticmethod
def validate_branch(branch: str) -> None:
if not isinstance(branch, str) or not branch.startswith("agent/") or branch in {"main", "master"}:
raise ValueError("BRANCH_SCOPE_DENIED")
def repository_identity(self) -> str:
self.validate_repository(self.config.repository)
return self.config.repository
def _environment(self) -> dict[str, str]:
if self.credential is not None and hasattr(self.credential, "environment"):
return dict(self.credential.environment())
return {}
def read_git_ref(self, branch: str) -> Optional[str]:
self.validate_branch(branch)
if self._git_ref is not None:
return self._git_ref(branch)
result = subprocess.run(["git", "-C", str(self.workspace), "ls-remote", self.config.remote_name, f"refs/heads/{branch}"], text=True, capture_output=True, check=False, env={**os.environ, **self._environment(), "GIT_TERMINAL_PROMPT": "0"})
if result.returncode != 0 or not result.stdout.strip():
return None
return result.stdout.split()[0]
def _api(self, path: str) -> Any:
if self._api_get is not None:
return self._api_get(path)
headers = {"Accept": "application/json"}
env = self._environment()
if env.get("AIOS_GITEA_TOKEN"):
headers["Authorization"] = "token " + env["AIOS_GITEA_TOKEN"]
with urlopen(Request(self.config.api_base.rstrip("/") + path, headers=headers), timeout=20) as response:
return json.loads(response.read() or b"{}")
def read_api_branch(self, branch: str) -> Optional[str]:
self.validate_branch(branch)
for page in range(1, 101):
value = self._api(f"/repos/aios/aios/branches?page={page}&limit=50")
if not isinstance(value, list):
return None
for item in value:
if isinstance(item, Mapping) and item.get("name") == branch:
commit = item.get("commit")
return commit.get("id") if isinstance(commit, Mapping) else None
if len(value) < 50:
return None
raise RuntimeError("GITEA_BRANCH_PAGINATION_LIMIT")
def commit_visible(self, oid: str) -> bool:
if not isinstance(oid, str) or len(oid) != 40:
return False
try:
value = self._api(f"/repos/aios/aios/git/commits/{quote(oid, safe='')}")
return isinstance(value, Mapping) and value.get("sha", oid) == oid
except Exception:
return False
def observe(self, branch: str, desired_oid: str, authority_oid: Optional[str], *, expected_oid: Optional[str] = None, observed_at: str = "") -> RemoteObservation:
return RemoteObservation(self.repository_identity(), branch, desired_oid, authority_oid, self.read_git_ref(branch), self.read_api_branch(branch), self.commit_visible(desired_oid), observed_at, expected_oid)
@@ -0,0 +1,707 @@
"""SQLite-backed durable GitOps Job Controller.
The controller owns durable intent, idempotency, lease metadata, state
transitions, receipts, and payload references. A worker owns Git side effects
behind the campaign capability and branch-CAS boundaries.
"""
from __future__ import annotations
from dataclasses import dataclass
from enum import Enum
import hashlib
import json
import os
from pathlib import Path
import sqlite3
import tempfile
from datetime import datetime, timezone
from typing import Any, Callable, Mapping
class JobError(RuntimeError):
pass
class ServiceUnavailable(JobError):
pass
@dataclass(frozen=True, slots=True)
class LeaseRenewalResult:
"""Durable outcome of one writer-lease recovery attempt."""
renewed: bool
idempotent: bool
durable: bool
lease_expires_at: str | None
def __bool__(self) -> bool:
return self.renewed or self.idempotent
class BindingConflict(JobError):
pass
class JobState(str, Enum):
REQUESTED = "REQUESTED"
AUTHORIZED = "AUTHORIZED"
LEASED = "LEASED"
WORKTREE_READY = "WORKTREE_READY"
EXECUTING = "EXECUTING"
LOCAL_VERIFIED = "LOCAL_VERIFIED"
REMOTE_SYNCING = "REMOTE_SYNCING"
REMOTE_VERIFIED = "REMOTE_VERIFIED"
COMPLETED = "COMPLETED"
RECOVERABLE_FAILURE = "RECOVERABLE_FAILURE"
RECONCILING = "RECONCILING"
RECONCILIATION_REQUIRED = "RECONCILIATION_REQUIRED"
MANUAL_OWNER_REVIEW_REQUIRED = "MANUAL_OWNER_REVIEW_REQUIRED"
_ORDER = {
JobState.REQUESTED: 0,
JobState.AUTHORIZED: 1,
JobState.LEASED: 2,
JobState.WORKTREE_READY: 3,
JobState.EXECUTING: 4,
JobState.LOCAL_VERIFIED: 5,
JobState.REMOTE_SYNCING: 6,
JobState.REMOTE_VERIFIED: 7,
JobState.COMPLETED: 8,
}
_RECOVERABLE = {JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}
_TERMINAL = {JobState.COMPLETED, JobState.MANUAL_OWNER_REVIEW_REQUIRED}
def utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def parse_utc(value: str) -> datetime:
normalized = value[:-1] + "+00:00" if value.endswith("Z") else value
parsed = datetime.fromisoformat(normalized)
if parsed.tzinfo is None:
raise ValueError("LEASE_EXPIRY_INVALID")
return parsed.astimezone(timezone.utc)
def canonical_json(value: Any) -> bytes:
return (
json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + "\n"
).encode("utf-8")
def sha256_json(value: Any) -> str:
return hashlib.sha256(canonical_json(value)).hexdigest()
class PayloadStore:
"""Content-addressed filesystem payloads; raw content never enters SQLite."""
def __init__(self, root: str | os.PathLike[str]) -> None:
self.root = Path(root)
if not self.root.is_absolute() or self.root.is_symlink():
raise JobError("PAYLOAD_ROOT_INVALID")
self.root.mkdir(parents=True, exist_ok=True, mode=0o700)
os.chmod(self.root, 0o700)
def put(self, content: bytes, expected_digest: str) -> tuple[str, int]:
digest = hashlib.sha256(content).hexdigest()
if digest != expected_digest:
raise JobError("PAYLOAD_DIGEST_MISMATCH")
if len(expected_digest) != 64 or any(ch not in "0123456789abcdef" for ch in expected_digest):
raise JobError("PAYLOAD_DIGEST_INVALID")
target = self.root / expected_digest
if target.is_symlink():
raise JobError("PAYLOAD_PATH_SYMLINK")
if target.is_file():
if target.stat().st_size != len(content) or hashlib.sha256(target.read_bytes()).hexdigest() != digest:
raise JobError("PAYLOAD_EXISTING_MISMATCH")
return f"sha256/{digest}", len(content)
fd, temporary = tempfile.mkstemp(prefix=".payload-", dir=self.root)
try:
os.fchmod(fd, 0o600)
with os.fdopen(fd, "wb") as stream:
stream.write(content)
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary, target)
os.chmod(target, 0o600)
finally:
if os.path.exists(temporary):
os.unlink(temporary)
return f"sha256/{digest}", len(content)
def read_verified(self, reference: str, expected_digest: str, expected_size: int) -> bytes:
if reference != f"sha256/{expected_digest}":
raise JobError("PAYLOAD_REFERENCE_MISMATCH")
path = self.root / expected_digest
if path.is_symlink() or not path.is_file():
raise JobError("PAYLOAD_MISSING")
content = path.read_bytes()
if len(content) != expected_size or hashlib.sha256(content).hexdigest() != expected_digest:
raise JobError("PAYLOAD_VERIFY_FAILED")
return content
@dataclass(frozen=True, slots=True)
class JobReceipt:
job_id: str
idempotency_key: str
state: JobState
side_effect_state: str
http_status: int
revision: int
receipt: Mapping[str, Any] | None = None
def as_dict(self) -> dict[str, Any]:
return {
"job_id": self.job_id,
"idempotency_key": self.idempotency_key,
"state": self.state.value,
"side_effect_state": self.side_effect_state,
"http_status": self.http_status,
"revision": self.revision,
"receipt": dict(self.receipt) if self.receipt else None,
}
class JobStore:
SCHEMA_VERSION = 1
def __init__(self, root: str | os.PathLike[str], *, payload_root: str | os.PathLike[str] | None = None) -> None:
self.root = Path(root)
if not self.root.is_absolute() or self.root.is_symlink():
raise JobError("JOB_ROOT_INVALID")
self.root.mkdir(parents=True, exist_ok=True, mode=0o700)
os.chmod(self.root, 0o700)
self.db_path = self.root / "state.sqlite3"
self.payloads = PayloadStore(payload_root or self.root.parent / "payloads" / "sha256")
self._initialize()
def _connect(self) -> sqlite3.Connection:
if self.db_path.is_symlink():
raise ServiceUnavailable("JOB_DB_SYMLINK")
connection = sqlite3.connect(self.db_path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA synchronous=FULL")
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
def _initialize(self) -> None:
try:
with self._connect() as connection:
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS metadata (key TEXT PRIMARY KEY, value TEXT NOT NULL);
CREATE TABLE IF NOT EXISTS jobs (
job_id TEXT PRIMARY KEY, idempotency_key TEXT NOT NULL UNIQUE,
campaign_id TEXT NOT NULL, repository TEXT NOT NULL, operation TEXT NOT NULL,
branch TEXT NOT NULL, expected_branch_oid TEXT NOT NULL,
payload_digest TEXT NOT NULL, payload_ref TEXT, payload_size INTEGER,
state TEXT NOT NULL, side_effect_state TEXT NOT NULL,
revision INTEGER NOT NULL, request_json TEXT NOT NULL,
receipt_json TEXT, created_at TEXT NOT NULL, updated_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS idempotency_keys (
key TEXT PRIMARY KEY, job_id TEXT NOT NULL UNIQUE,
binding_json TEXT NOT NULL, created_at TEXT NOT NULL,
FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE TABLE IF NOT EXISTS branch_leases (
lease_id TEXT PRIMARY KEY, job_id TEXT NOT NULL UNIQUE,
branch TEXT NOT NULL, lineage_anchor_oid TEXT NOT NULL,
current_head_oid TEXT NOT NULL, generation INTEGER NOT NULL,
lease_expires_at TEXT NOT NULL, state TEXT NOT NULL,
revision INTEGER NOT NULL, FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE TABLE IF NOT EXISTS state_transitions (
sequence INTEGER PRIMARY KEY AUTOINCREMENT, job_id TEXT NOT NULL,
from_state TEXT, to_state TEXT NOT NULL, revision INTEGER NOT NULL,
side_effect_state TEXT NOT NULL, created_at TEXT NOT NULL,
FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE TABLE IF NOT EXISTS receipts (
receipt_id TEXT PRIMARY KEY, job_id TEXT NOT NULL,
kind TEXT NOT NULL, digest TEXT NOT NULL, payload_json TEXT NOT NULL,
created_at TEXT NOT NULL, FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
INSERT OR REPLACE INTO metadata(key, value) VALUES('schema_version', '1');
"""
)
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
def _mirror(self, job: Mapping[str, Any]) -> None:
directory = self.root / str(job["job_id"])
directory.mkdir(mode=0o700, exist_ok=True)
(directory / "receipts").mkdir(mode=0o700, exist_ok=True)
(directory / "logs-digest").mkdir(mode=0o700, exist_ok=True)
for name, value in (("request.json", job["request"]), ("state.json", job)):
target = directory / name
fd, temporary = tempfile.mkstemp(prefix=".mirror-", dir=directory)
try:
os.fchmod(fd, 0o600)
with os.fdopen(fd, "wb") as stream:
stream.write(canonical_json(value))
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary, target)
finally:
if os.path.exists(temporary):
os.unlink(temporary)
def read(self, job_id: str) -> dict[str, Any]:
try:
with self._connect() as connection:
row = connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone()
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
if row is None:
raise JobError("JOB_NOT_FOUND")
return dict(row)
def _receipt(self, row: Mapping[str, Any], *, duplicate: bool) -> JobReceipt:
state = JobState(row["state"])
if state == JobState.COMPLETED:
status, side = 200, "COMPLETED"
elif state in _RECOVERABLE:
status = 202
side = row["side_effect_state"] if row["side_effect_state"] == "REMOTE_PUSH_PENDING" else "RESUME_SAME_JOB"
elif row["side_effect_state"] == "UNKNOWN":
status, side = 503, "UNKNOWN"
else:
status, side = 202, "SAME_JOB" if duplicate else "QUEUED"
receipt = json.loads(row["receipt_json"]) if row["receipt_json"] else None
return JobReceipt(row["job_id"], row["idempotency_key"], state, side, status, int(row["revision"]), receipt)
class GitOpsJobController:
def __init__(self, store: JobStore) -> None:
self.store = store
@staticmethod
def _binding(request: Mapping[str, Any]) -> dict[str, str]:
fields = ("campaign_id", "repository", "operation", "branch", "expected_branch_oid", "payload_digest")
if any(not isinstance(request.get(field), str) or not request[field].strip() for field in fields):
raise JobError("JOB_BINDING_INVALID")
oid = request["expected_branch_oid"]
digest = request["payload_digest"]
if len(oid) != 40 or any(ch not in "0123456789abcdef" for ch in oid):
raise JobError("EXPECTED_BRANCH_OID_INVALID")
if len(digest) != 64 or any(ch not in "0123456789abcdef" for ch in digest):
raise JobError("PAYLOAD_DIGEST_INVALID")
if request["branch"] in {"main", "master"} or request["branch"].startswith(("refs/", "tags/")):
raise JobError("PROTECTED_BRANCH_REJECTED")
binding = {field: str(request[field]) for field in fields}
# Operation-specific, non-secret binding inputs must survive the
# durable round trip and participate in idempotency. Arbitrary request
# fields are intentionally discarded rather than persisted.
optional = (
"idempotency_key",
"task_id",
"new_branch_oid",
"authorization_recovery_digest",
"lineage_anchor_oid",
"base_oid",
"branch_mode",
"generation",
"expected_main_oid",
"pr_number",
"target_branch",
"check_context",
"check_state",
"check_description",
)
for field in optional:
value = request.get(field)
if value is None:
continue
if isinstance(value, bool) or not isinstance(value, (str, int)):
raise JobError("JOB_BINDING_OPTION_INVALID")
text = str(value).strip()
if not text or len(text) > 512:
raise JobError("JOB_BINDING_OPTION_INVALID")
binding[field] = text
return binding
def submit(self, request: Mapping[str, Any], *, payload: bytes | None = None) -> JobReceipt:
binding = self._binding(request)
payload_ref = request.get("payload_ref")
payload_size = request.get("payload_size")
if payload is not None:
payload_ref, payload_size = self.store.payloads.put(payload, binding["payload_digest"])
if payload_ref is not None and payload_ref != f"sha256/{binding['payload_digest']}":
raise JobError("PAYLOAD_REFERENCE_MISMATCH")
if payload_ref is not None and (not isinstance(payload_size, int) or payload_size < 0):
raise JobError("PAYLOAD_SIZE_INVALID")
client_idempotency_key = binding.get("idempotency_key")
if client_idempotency_key is not None:
if len(client_idempotency_key) > 128 or not all(
character.isalnum() or character in "._:-" for character in client_idempotency_key
):
raise JobError("IDEMPOTENCY_KEY_INVALID")
key = sha256_json({"client_idempotency_key": client_idempotency_key})
else:
key = sha256_json(binding)
job_id = "job-" + key[:32]
durable_request = dict(binding)
if payload_ref is not None:
durable_request.update({"payload_ref": str(payload_ref), "payload_size": int(payload_size)})
now = utc_now()
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
existing = connection.execute("SELECT * FROM jobs WHERE idempotency_key=?", (key,)).fetchone()
if existing is not None:
if json.loads(existing["request_json"]).get("binding") != binding:
raise BindingConflict("BINDING_CONFLICT")
row = dict(existing)
connection.commit()
return self.store._receipt(row, duplicate=True)
connection.execute(
"INSERT INTO jobs(job_id,idempotency_key,campaign_id,repository,operation,branch,expected_branch_oid,payload_digest,payload_ref,payload_size,state,side_effect_state,revision,request_json,receipt_json,created_at,updated_at) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
(job_id, key, binding["campaign_id"], binding["repository"], binding["operation"], binding["branch"], binding["expected_branch_oid"], binding["payload_digest"], payload_ref, payload_size, JobState.REQUESTED.value, "NONE", 0, json.dumps({"binding": binding, **durable_request}, sort_keys=True), None, now, now),
)
connection.execute("INSERT INTO idempotency_keys(key,job_id,binding_json,created_at) VALUES(?,?,?,?)", (key, job_id, json.dumps(binding, sort_keys=True), now))
connection.execute("INSERT INTO state_transitions(job_id,from_state,to_state,revision,side_effect_state,created_at) VALUES(?,?,?,?,?,?)", (job_id, None, JobState.REQUESTED.value, 0, "NONE", now))
row = dict(connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone())
connection.commit()
except BindingConflict:
raise
except sqlite3.IntegrityError:
row = self.store.read(job_id)
return self.store._receipt(row, duplicate=True)
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
row["request"] = json.loads(row["request_json"])
self.store._mirror(row)
return self.store._receipt(row, duplicate=False)
def lease_job(
self,
job_id: str,
*,
lease_id: str,
lineage_anchor_oid: str,
current_head_oid: str,
generation: int,
lease_expires_at: str,
expected_revision: int,
) -> JobReceipt:
"""Atomically create a branch lease and move its Job to LEASED."""
if generation < 1 or any(len(oid) != 40 for oid in (lineage_anchor_oid, current_head_oid)):
raise JobError("LEASE_BINDING_INVALID")
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
row = connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone()
if row is None:
raise JobError("JOB_NOT_FOUND")
if int(row["revision"]) != expected_revision:
raise JobError("REVISION_CAS_CONFLICT")
if row["branch"] in {"main", "master"}:
raise JobError("PROTECTED_BRANCH_REJECTED")
now = utc_now()
connection.execute("INSERT INTO branch_leases(lease_id,job_id,branch,lineage_anchor_oid,current_head_oid,generation,lease_expires_at,state,revision) VALUES(?,?,?,?,?,?,?,?,?)", (lease_id, job_id, row["branch"], lineage_anchor_oid, current_head_oid, generation, lease_expires_at, "ACTIVE", 0))
next_revision = expected_revision + 1
updated = connection.execute("UPDATE jobs SET state=?,side_effect_state=?,revision=?,updated_at=? WHERE job_id=? AND revision=?", (JobState.LEASED.value, "LEASED", next_revision, now, job_id, expected_revision))
if updated.rowcount != 1:
raise JobError("REVISION_CAS_CONFLICT")
connection.execute("INSERT INTO state_transitions(job_id,from_state,to_state,revision,side_effect_state,created_at) VALUES(?,?,?,?,?,?)", (job_id, row["state"], JobState.LEASED.value, next_revision, "LEASED", now))
result = dict(connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone())
connection.commit()
except sqlite3.IntegrityError as exc:
raise JobError("LEASE_CONFLICT") from exc
except (JobError, BindingConflict):
raise
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
result["request"] = json.loads(result["request_json"])
self.store._mirror(result)
return self.store._receipt(result, duplicate=True)
def update_lease(self, lease_id: str, *, expected_revision: int, current_head_oid: str, generation: int, state: str = "ACTIVE", lease_expires_at: str | None = None) -> None:
if len(current_head_oid) != 40 or generation < 1:
raise JobError("LEASE_BINDING_INVALID")
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
if lease_expires_at is None:
updated = connection.execute("UPDATE branch_leases SET current_head_oid=?,generation=?,state=?,revision=revision+1 WHERE lease_id=? AND revision=?", (current_head_oid, generation, state, lease_id, expected_revision))
else:
updated = connection.execute("UPDATE branch_leases SET current_head_oid=?,generation=?,lease_expires_at=?,state=?,revision=revision+1 WHERE lease_id=? AND revision=?", (current_head_oid, generation, lease_expires_at, state, lease_id, expected_revision))
if updated.rowcount != 1:
raise JobError("LEASE_REVISION_CAS_CONFLICT")
connection.commit()
except JobError:
raise
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
def renew_lease(
self,
*,
branch: str,
lineage_anchor_oid: str,
current_head_oid: str,
generation: int,
lease_expires_at: str,
) -> LeaseRenewalResult:
if (
not branch
or len(lineage_anchor_oid) != 40
or len(current_head_oid) != 40
or generation < 1
or not lease_expires_at.strip()
):
raise JobError("LEASE_BINDING_INVALID")
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
rows = connection.execute(
"SELECT lease_id, revision, generation, state, lease_expires_at "
"FROM branch_leases "
"WHERE branch=? AND lineage_anchor_oid=? AND current_head_oid=? "
"AND generation=? AND state='ACTIVE' ORDER BY revision DESC",
(branch, lineage_anchor_oid, current_head_oid, generation),
).fetchall()
if not rows:
connection.commit()
return LeaseRenewalResult(False, False, False, None)
if len(rows) != 1:
raise JobError("LEASE_BINDING_CONFLICT")
row = rows[0]
try:
current_expires_at = parse_utc(str(row["lease_expires_at"]))
requested_expires_at = parse_utc(lease_expires_at)
except (TypeError, ValueError) as exc:
raise JobError("LEASE_EXPIRY_INVALID") from exc
now = datetime.now(timezone.utc)
if requested_expires_at <= now:
raise JobError("LEASE_EXPIRY_INVALID")
if current_expires_at > now:
connection.commit()
return LeaseRenewalResult(
False,
True,
True,
str(row["lease_expires_at"]),
)
updated = connection.execute(
"UPDATE branch_leases SET lease_expires_at=?,revision=revision+1 "
"WHERE lease_id=? AND revision=? AND generation=? "
"AND state='ACTIVE' AND lease_expires_at=?",
(
lease_expires_at,
row["lease_id"],
int(row["revision"]),
generation,
row["lease_expires_at"],
),
)
if updated.rowcount != 1:
current = connection.execute(
"SELECT lease_expires_at, generation, state FROM branch_leases "
"WHERE lease_id=?",
(row["lease_id"],),
).fetchone()
if (
current is not None
and int(current["generation"]) == generation
and current["state"] == "ACTIVE"
and parse_utc(str(current["lease_expires_at"])) > now
):
connection.commit()
return LeaseRenewalResult(
False,
True,
True,
str(current["lease_expires_at"]),
)
raise JobError("LEASE_REVISION_CAS_CONFLICT")
connection.commit()
return LeaseRenewalResult(True, False, True, lease_expires_at)
except JobError:
raise
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
def advance(self, job_id: str, state: JobState, *, side_effect_state: str, receipt: Mapping[str, Any] | None = None, expected_revision: int | None = None) -> JobReceipt:
return self._advance(
job_id,
state,
side_effect_state=side_effect_state,
receipt=receipt,
expected_revision=expected_revision,
)
def _advance(
self,
job_id: str,
state: JobState,
*,
side_effect_state: str,
receipt: Mapping[str, Any] | None = None,
expected_revision: int | None = None,
_allow_verified_external_recovery: bool = False,
) -> JobReceipt:
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
row = connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone()
if row is None:
raise JobError("JOB_NOT_FOUND")
previous = JobState(row["state"])
revision = int(row["revision"])
if expected_revision is not None and expected_revision != revision:
raise JobError("REVISION_CAS_CONFLICT")
recovery_transition = (
_allow_verified_external_recovery
and previous == JobState.MANUAL_OWNER_REVIEW_REQUIRED
and state == JobState.COMPLETED
)
if previous in _TERMINAL and state != previous and not recovery_transition:
raise JobError("JOB_TERMINAL")
if state not in _RECOVERABLE | {JobState.RECONCILING, JobState.MANUAL_OWNER_REVIEW_REQUIRED}:
if previous in _RECOVERABLE:
raise JobError("RECONCILE_REQUIRED")
if previous != JobState.RECONCILING and _ORDER.get(state, -1) < _ORDER.get(previous, -1):
raise JobError("JOB_STATE_REGRESSION")
next_revision = revision + 1
receipt_json = json.dumps(dict(receipt), sort_keys=True) if receipt is not None else row["receipt_json"]
now = utc_now()
updated = connection.execute("UPDATE jobs SET state=?,side_effect_state=?,revision=?,receipt_json=?,updated_at=? WHERE job_id=? AND revision=?", (state.value, side_effect_state, next_revision, receipt_json, now, job_id, revision))
if updated.rowcount != 1:
raise JobError("REVISION_CAS_CONFLICT")
connection.execute("INSERT INTO state_transitions(job_id,from_state,to_state,revision,side_effect_state,created_at) VALUES(?,?,?,?,?,?)", (job_id, previous.value, state.value, next_revision, side_effect_state, now))
if receipt is not None:
digest = hashlib.sha256(canonical_json(receipt)).hexdigest()
connection.execute("INSERT INTO receipts(receipt_id,job_id,kind,digest,payload_json,created_at) VALUES(?,?,?,?,?,?)", (f"receipt-{job_id}-{next_revision}", job_id, state.value, digest, json.dumps(dict(receipt), sort_keys=True), now))
result = dict(connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone())
connection.commit()
except (JobError, BindingConflict):
raise
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
result["request"] = json.loads(result["request_json"])
self.store._mirror(result)
return self.store._receipt(result, duplicate=True)
@staticmethod
def _validate_verified_external_merge_proof(
job_id: str,
binding: Mapping[str, Any],
proof: Mapping[str, Any],
) -> dict[str, Any]:
if not isinstance(proof, Mapping):
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_INVALID")
def require_oid(field: str) -> str:
value = proof.get(field)
if not isinstance(value, str) or len(value) != 40 or any(ch not in "0123456789abcdef" for ch in value):
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_INVALID")
return value
if proof.get("proof_schema") != "aios.job-recovery.v1":
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_INVALID")
if proof.get("job_id") != job_id:
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
if proof.get("operation") != "PROTECTED_MERGE" or binding.get("operation") != "PROTECTED_MERGE":
raise JobError("EXTERNAL_SIDE_EFFECT_OPERATION_REJECTED")
if proof.get("pr_number") != 53 or str(binding.get("pr_number")) != "53":
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
if proof.get("pr_merged") is not True or proof.get("protected_merge") is not True:
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
if proof.get("force_push") is not False:
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
if proof.get("machine_actor") != "aios-autonomy-agent":
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
expected_head = str(binding.get("expected_branch_oid", ""))
expected_base = str(binding.get("expected_main_oid", ""))
head_oid = require_oid("head_oid")
base_oid = require_oid("base_oid")
merged_main = require_oid("main_oid_after_protected_merge")
gitea_merged_main = require_oid("gitea_main_oid_after_protected_merge")
authority_merged_main = require_oid("authority_main_oid_after_protected_merge")
current_gitea = require_oid("current_gitea_main_oid")
current_authority = require_oid("current_authority_main_oid")
if head_oid != expected_head or base_oid != expected_base:
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
if not (merged_main == gitea_merged_main == authority_merged_main == head_oid):
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
if current_gitea != current_authority or proof.get("current_main_descends_from_protected_merge") is not True:
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
return dict(proof)
def complete_verified_external_side_effect(
self,
job_id: str,
proof: Mapping[str, Any],
*,
expected_revision: int | None = None,
) -> JobReceipt:
"""Close one historical manual job from independently verified proof.
This is deliberately narrower than advance: it is only valid for the
known protected PR53 recovery shape and preserves the old receipt.
The actual state change still uses the normal revision-CAS path.
"""
current = self.store.read(job_id)
revision = int(current["revision"])
if expected_revision is not None and expected_revision != revision:
raise JobError("REVISION_CAS_CONFLICT")
request = json.loads(current["request_json"])
binding = request.get("binding", request)
normalized_proof = self._validate_verified_external_merge_proof(job_id, binding, proof)
existing_receipt = json.loads(current["receipt_json"]) if current["receipt_json"] else None
if JobState(current["state"]) == JobState.COMPLETED:
if (
existing_receipt
and existing_receipt.get("recovered_from_historical_manual") is True
and existing_receipt.get("proof") == normalized_proof
):
return self.store._receipt(current, duplicate=True)
raise JobError("JOB_TERMINAL")
if JobState(current["state"]) != JobState.MANUAL_OWNER_REVIEW_REQUIRED:
raise JobError("EXTERNAL_SIDE_EFFECT_RECOVERY_STATE_INVALID")
recovery_receipt = {
"recovered_from_historical_manual": True,
"recovery_kind": "VERIFIED_EXTERNAL_SIDE_EFFECT",
"historical_receipt": existing_receipt,
"proof": normalized_proof,
}
return self._advance(
job_id,
JobState.COMPLETED,
side_effect_state="REMOTE_VERIFIED",
receipt=recovery_receipt,
expected_revision=revision,
_allow_verified_external_recovery=True,
)
def reconcile(self, job_id: str, observer: Callable[[Mapping[str, Any]], Mapping[str, Any]]) -> JobReceipt:
current = self.store.read(job_id)
if JobState(current["state"]) not in _RECOVERABLE:
return self.store._receipt(current, duplicate=True)
self.advance(job_id, JobState.RECONCILING, side_effect_state="IN_PROGRESS")
try:
observed = dict(observer(current))
stable = JobState(observed.get("state", JobState.REQUESTED.value))
if stable in {JobState.RECOVERABLE_FAILURE, JobState.RECONCILING}:
raise JobError("RECONCILIATION_UNSTABLE")
return self.advance(job_id, stable, side_effect_state=str(observed.get("side_effect_state", "NONE")), receipt=observed.get("receipt"))
except Exception as exc:
self.advance(job_id, JobState.RECONCILIATION_REQUIRED, side_effect_state="UNKNOWN", receipt={"error_code": type(exc).__name__})
raise JobError("RECONCILIATION_REQUIRED") from exc
def resume(self, job_id: str) -> JobReceipt:
current = self.store.read(job_id)
if JobState(current["state"]) in _TERMINAL:
raise JobError("JOB_TERMINAL")
return self.reconcile(job_id, lambda _current: {"state": JobState.REQUESTED.value, "side_effect_state": "RESUME_SAME_JOB"})
@@ -0,0 +1,279 @@
"""Durable Job side-effect executor for the AIOS campaign control plane."""
from __future__ import annotations
import argparse
import base64
from dataclasses import dataclass, replace
import json
from pathlib import Path
from typing import Any, Mapping
from aios_authority_git_executor import AuthorityGitExecutor, AuthorityGitError
from aios_campaign_recovery import RecoveryEvidenceError, load_recovered_authorization
from aios_gitea_machine_broker import CampaignCapabilityReceipt, GiteaMachineBroker, MachineBrokerError
from aios_gitops_job_controller import GitOpsJobController, JobError, JobState, JobStore
TASK_ID = "AIOS_AUTONOMOUS_RELEASE_CAMPAIGN_V1"
CAMPAIGN_ID = "AIOS-AUTONOMOUS-RELEASE-CAMPAIGN-20260807"
REPOSITORY = "aios/aios"
BRANCH = "agent/autonomous-release-campaign-v1-20260807"
EXPECTED_BRANCH_OID = "bba675fab3b7e0c5fba7066720252b3c91b9f20d"
AUTHORIZATION_DIGEST = "6b0bf550eb702126e967eeff2d561fa808444b924d8a71f5399accad508ed3b2"
LINEAGE_ANCHOR = "69b45e81dcd2c3707c367003b08c55fd361faae8"
class JobExecutionError(RuntimeError):
pass
@dataclass(slots=True)
class DurableJobExecutor:
controller: GitOpsJobController
authority: AuthorityGitExecutor
broker: GiteaMachineBroker
recovery_path: Path
def _request(self, job_id: str) -> tuple[dict[str, Any], dict[str, Any]]:
row = self.controller.store.read(job_id)
try:
request = json.loads(row["request_json"])
except (KeyError, TypeError, json.JSONDecodeError) as exc:
raise JobExecutionError("JOB_REQUEST_INVALID") from exc
return row, dict(request.get("binding") or {})
def _validate_campaign(self, request: Mapping[str, Any]) -> str:
operation = str(request.get("operation", ""))
expected_branch_oid = request.get("expected_branch_oid")
required = {
"task_id": TASK_ID,
"campaign_id": CAMPAIGN_ID,
"repository": REPOSITORY,
"branch": BRANCH,
"authorization_recovery_digest": AUTHORIZATION_DIGEST,
}
for key, expected in required.items():
if request.get(key) != expected:
raise JobExecutionError(f"JOB_{key.upper()}_MISMATCH")
new_oid = request.get("new_branch_oid")
if not isinstance(new_oid, str) or len(new_oid) != 40 or any(ch not in "0123456789abcdef" for ch in new_oid):
raise JobExecutionError("NEW_BRANCH_OID_INVALID")
if not isinstance(expected_branch_oid, str) or len(expected_branch_oid) != 40 or any(ch not in "0123456789abcdef" for ch in expected_branch_oid):
raise JobExecutionError("EXPECTED_BRANCH_OID_INVALID")
if operation == "ADVANCE_EXISTING_CAMPAIGN_BRANCH" and expected_branch_oid == new_oid:
raise JobExecutionError("EXPECTED_BRANCH_OID_INVALID")
if operation != "ADVANCE_EXISTING_CAMPAIGN_BRANCH" and expected_branch_oid != new_oid:
raise JobExecutionError("POST_ADVANCE_HEAD_BINDING_INVALID")
try:
recovery = load_recovered_authorization(self.recovery_path)
except RecoveryEvidenceError as exc:
raise JobExecutionError(str(exc)) from exc
if recovery["digest"] != AUTHORIZATION_DIGEST:
raise JobExecutionError("RECOVERY_DIGEST_MISMATCH")
for key, expected in required.items():
if key == "campaign_id":
actual = recovery.get(key)
elif key == "authorization_recovery_digest":
continue
else:
actual = recovery.get({"expected_branch_oid": "expected_branch_oid"}.get(key, key))
if actual is not None and actual != expected:
raise JobExecutionError("RECOVERY_BINDING_MISMATCH")
if recovery.get("expected_branch_oid") != EXPECTED_BRANCH_OID:
raise JobExecutionError("RECOVERY_BRANCH_OID_MISMATCH")
authority_head = self.authority.read_ref(f"refs/heads/{BRANCH}")
if authority_head not in {expected_branch_oid, new_oid}:
raise JobExecutionError("AUTHORITY_BRANCH_EXPECTATION_MISMATCH")
if not self.authority.ancestry_check(expected_branch_oid, new_oid):
raise JobExecutionError("NEW_OID_NOT_DESCENDANT")
if not self.authority.ancestry_check(EXPECTED_BRANCH_OID, new_oid):
raise JobExecutionError("NEW_OID_OUTSIDE_AUTHORIZED_LINEAGE")
identity = self.broker.verify_identity()
if identity.get("repository") != REPOSITORY or identity.get("login") == "aiman":
raise JobExecutionError("MACHINE_IDENTITY_SCOPE_INVALID")
return new_oid
def dry_run(self, job_id: str) -> dict[str, Any]:
_row, request = self._request(job_id)
new_oid = self._validate_campaign(request)
remote = self.broker.readback()
if remote is None:
action = "CREATE_REQUIRED"
elif remote == new_oid:
action = "ALREADY_IN_SYNC"
elif remote == request["expected_branch_oid"]:
action = "FAST_FORWARD_REQUIRED"
else:
action = "MANUAL_OWNER_REVIEW_REQUIRED"
if action == "MANUAL_OWNER_REVIEW_REQUIRED":
raise JobExecutionError("GITEA_BRANCH_CONFLICT")
return {
"decision": "SAFE_TO_ADVANCE",
"authority_action": "CAS_REQUIRED",
"gitea_action": action,
"previous_branch_oid": request["expected_branch_oid"],
"final_branch_oid": new_oid,
"force_push": False,
}
def _recover_requested(self, job_id: str) -> dict[str, Any]:
row = self.controller.store.read(job_id)
if JobState(row["state"]) in {JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}:
self.controller.resume(job_id)
row = self.controller.store.read(job_id)
return row
def execute(self, job_id: str, *, dry_run: bool = False) -> dict[str, Any]:
try:
return self._execute_unprotected(job_id, dry_run=dry_run)
except Exception as exc:
try:
row = self.controller.store.read(job_id)
state = JobState(row["state"])
if state not in {JobState.COMPLETED, JobState.MANUAL_OWNER_REVIEW_REQUIRED, JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}:
self.controller.advance(job_id, JobState.RECOVERABLE_FAILURE, side_effect_state="UNKNOWN", receipt={"error_code": type(exc).__name__})
except Exception:
pass
raise
def _execute_unprotected(self, job_id: str, *, dry_run: bool = False) -> dict[str, Any]:
row, request = self._request(job_id)
operation = request.get("operation")
if operation == "ADVANCE_EXISTING_CAMPAIGN_BRANCH":
if dry_run:
return self.dry_run(job_id)
return self._advance_campaign(job_id)
if operation == "COMMIT_FILE":
return self._commit_file(job_id, request)
if operation == "SYNC_TASK_BRANCH_TO_GITEA":
new_oid = self._validate_campaign(request)
final = self.broker.push_branch(new_oid)
return {"operation": operation, "remote_verified": final == new_oid, "final_network_oid": final}
if operation == "CREATE_PR":
self._validate_campaign(request)
return dict(self.broker.create_pr("AIOS autonomous release control plane", "Durable campaign continuation; exact head and recovery evidence are bound in the Job."))
if operation == "PUBLISH_CHECK":
self._validate_campaign(request)
context = request.get("check_context", "aios/durable-release")
state = request.get("check_state", "success")
description = request.get("check_description", "Durable campaign control-plane check passed")
return dict(self.broker.status(str(state), str(description), str(context)))
if operation == "PROTECTED_MERGE":
self._validate_campaign(request)
number = int(request.get("pr_number", "0"))
self.broker.transport.expected_merge_head = request["expected_branch_oid"]
expected_main = request.get("expected_main_oid")
if expected_main is not None:
self.broker.transport.expected_merge_base = expected_main
return dict(self.broker.merge(number))
if operation == "RECONCILE_AUTHORITY_MAIN":
self._validate_campaign(request)
old = request.get("expected_main_oid")
new = request.get("new_branch_oid")
if not isinstance(old, str) or not isinstance(new, str):
raise JobExecutionError("MAIN_RECONCILIATION_BINDING_INVALID")
gitea_main = self.broker.transport.read_branch("main")
if gitea_main != new:
raise JobExecutionError("GITEA_MAIN_EXPECTATION_MISMATCH")
if not self.authority.ancestry_check(old, new):
raise JobExecutionError("MAIN_OID_NOT_DESCENDANT")
return {"authority_main": self.authority.ff_CAS_main_after_verified_protected_merge(old, new)}
raise JobExecutionError("OPERATION_NOT_ALLOWED")
def _advance_campaign(self, job_id: str) -> dict[str, Any]:
self._recover_requested(job_id)
row, request = self._request(job_id)
new_oid = self._validate_campaign(request)
expected_oid = request["expected_branch_oid"]
state = JobState(row["state"])
if state == JobState.REQUESTED:
receipt = self.controller.advance(job_id, JobState.AUTHORIZED, side_effect_state="AUTHORIZED", receipt={"authorization_digest": AUTHORIZATION_DIGEST, "task_id": TASK_ID})
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.AUTHORIZED:
self.controller.lease_job(job_id, lease_id=f"lease-{job_id}", lineage_anchor_oid=LINEAGE_ANCHOR, current_head_oid=expected_oid, generation=1, lease_expires_at="2099-01-01T00:00:00Z", expected_revision=int(row["revision"]))
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.LEASED:
self.controller.advance(job_id, JobState.WORKTREE_READY, side_effect_state="WORKTREE_READY", receipt={"branch": BRANCH, "expected_oid": expected_oid})
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.WORKTREE_READY:
self.controller.advance(job_id, JobState.EXECUTING, side_effect_state="IN_PROGRESS")
row = self.controller.store.read(job_id)
authority_before = self.authority.read_ref(f"refs/heads/{BRANCH}")
if authority_before == expected_oid:
authority_oid = self.authority.advance_task_branch_CAS(f"refs/heads/{BRANCH}", expected_oid, new_oid)
elif authority_before == new_oid:
authority_oid = new_oid
else:
raise JobExecutionError("AUTHORITY_BRANCH_CONFLICT")
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.EXECUTING:
self.controller.advance(job_id, JobState.LOCAL_VERIFIED, side_effect_state="LOCAL_VERIFIED", receipt={"authority": {"branch": BRANCH, "previous_oid": expected_oid, "new_oid": new_oid, "CAS_success": True, "readback_oid": authority_oid}})
row = self.controller.store.read(job_id)
remote_before = self.broker.readback()
if remote_before != new_oid:
if remote_before not in {None, expected_oid}:
raise JobExecutionError("GITEA_BRANCH_CONFLICT")
if JobState(row["state"]) == JobState.LOCAL_VERIFIED:
if hasattr(self.broker, "receipt"):
self.broker.receipt = replace(self.broker.receipt, expected_oid=expected_oid)
self.controller.advance(job_id, JobState.REMOTE_SYNCING, side_effect_state="IN_PROGRESS", receipt={"gitea": {"previous_oid": remote_before, "desired_oid": new_oid, "force_push": False}})
final_network = self.broker.push_branch(new_oid)
else:
final_network = new_oid
final_api = self.broker.readback()
if final_api != new_oid:
raise JobExecutionError("GITEA_BRANCH_READBACK_MISMATCH")
row = self.controller.store.read(job_id)
if JobState(row["state"]) in {JobState.LOCAL_VERIFIED, JobState.REMOTE_SYNCING}:
self.controller.advance(job_id, JobState.REMOTE_VERIFIED, side_effect_state="REMOTE_VERIFIED", receipt={"gitea": {"machine_actor": "aios-autonomy-agent", "previous_oid": remote_before, "desired_oid": new_oid, "final_network_oid": final_network, "final_API_oid": final_api, "force_push": False}})
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.REMOTE_VERIFIED:
self.controller.advance(job_id, JobState.COMPLETED, side_effect_state="REMOTE_VERIFIED", receipt={"authority_oid": authority_oid, "network_oid": final_network, "gitea_api_oid": final_api, "force_push": False})
return self.controller.store._receipt(self.controller.store.read(job_id), duplicate=True).as_dict()
def _commit_file(self, job_id: str, request: Mapping[str, Any]) -> dict[str, Any]:
row = self.controller.store.read(job_id)
payload_ref = request.get("payload_ref")
if not payload_ref:
raise JobExecutionError("PAYLOAD_REQUIRED")
payload = self.controller.store.payloads.read_verified(payload_ref, row["payload_digest"], int(request["payload_size"]))
try:
spec = json.loads(payload.decode())
path = str(spec["path"])
content = base64.b64decode(spec["content_base64"], validate=True)
except Exception as exc:
raise JobExecutionError("COMMIT_FILE_PAYLOAD_INVALID") from exc
new_oid = self.authority.commit_file_in_isolated_worktree(base_oid=row["expected_branch_oid"], files={path: content}, message="chore(gitops): durable commit-file", worktree_name=job_id)
return {"operation": "COMMIT_FILE", "new_oid": new_oid}
def build_from_environment(job_root: Path) -> DurableJobExecutor:
from aios_gitea_machine_transport import GiteaMachineTransport
from aios_gitea_machine_broker import MachineCredentialProjection
account = __import__("os").environ.get("AIOS_GITEA_MACHINE_ACCOUNT", "aios-autonomy-agent")
api = __import__("os").environ.get("AIOS_GITEA_API", "https://g.cnnetmkt.top/git/api/v1")
remote = __import__("os").environ.get("AIOS_GITEA_REMOTE", "https://g.cnnetmkt.top/git/aios/aios.git")
authority_dir = Path(__import__("os").environ.get("AIOS_GITOPS_AUTHORITY_GIT_DIR", "/data/aios-gitops/repo.git"))
worktree_root = Path(__import__("os").environ.get("AIOS_GITOPS_WORKTREE_ROOT", "/var/lib/aios-gitops/worktrees"))
recovery = Path(__import__("os").environ.get("AIOS_RELEASE_RECOVERY_RECEIPT", "/var/lib/aios-gitops/release-campaigns/AIOS-AUTONOMOUS-RELEASE-CAMPAIGN-20260807/authorization-recovery.json"))
transport = GiteaMachineTransport(api, REPOSITORY, remote, authority_dir)
credential = MachineCredentialProjection.from_mapping({"scheme": "systemd-credential", "name": "gitea-machine-token", "account": account})
receipt = CampaignCapabilityReceipt(CAMPAIGN_ID, TASK_ID, REPOSITORY, BRANCH, EXPECTED_BRANCH_OID, frozenset({"exact_task_branch_push", "pr_create_update", "status_publish", "protected_merge", "remote_readback"}), frozenset({"direct_main_push", "force_push", "tag_update", "tag_delete", "baseline_write", "production_deploy"}))
return DurableJobExecutor(GitOpsJobController(JobStore(job_root)), AuthorityGitExecutor(authority_dir, worktree_root), GiteaMachineBroker(transport, credential, receipt), recovery)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--job-root", required=True, type=Path)
parser.add_argument("--job-id", required=True)
parser.add_argument("--dry-run", action="store_true")
args = parser.parse_args(argv)
result = build_from_environment(args.job_root).execute(args.job_id, dry_run=args.dry_run)
print(json.dumps(result, sort_keys=True))
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,80 @@
"""Pure, observation-first decisions for governed GitOps reconciliation."""
from __future__ import annotations
from dataclasses import dataclass
from enum import Enum
import re
from typing import Optional
OID_RE = re.compile(r"^[0-9a-f]{40}$")
class ReconciliationAction(str, Enum):
NOOP = "NOOP"
PUSH_EXACT = "PUSH_EXACT"
VERIFY_REMOTE = "VERIFY_REMOTE"
MARK_REMOTE_VERIFIED = "MARK_REMOTE_VERIFIED"
MARK_RECONCILIATION_REQUIRED = "MARK_RECONCILIATION_REQUIRED"
FAIL_CONFLICT = "FAIL_CONFLICT"
@dataclass(frozen=True, slots=True)
class RemoteObservation:
repository_identity: str
branch: str
desired_oid: str
authority_oid: Optional[str]
remote_git_oid: Optional[str]
remote_api_oid: Optional[str]
commit_visible: bool
observed_at: str
expected_oid: Optional[str] = None
@property
def all_equal(self) -> bool:
values = (self.authority_oid, self.remote_git_oid, self.remote_api_oid, self.desired_oid)
return all(isinstance(value, str) for value in values) and len(set(values)) == 1
@dataclass(frozen=True, slots=True)
class ReconciliationDecision:
action: ReconciliationAction
state: str
reason: str
retry_safe: bool = False
def _valid_oid(value: Optional[str]) -> bool:
return value is None or bool(OID_RE.fullmatch(value))
def decide_reconciliation(observation: RemoteObservation) -> ReconciliationDecision:
"""Decide from a fresh observation; never complete on a missing remote."""
if observation.repository_identity != "aios/aios":
return ReconciliationDecision(ReconciliationAction.FAIL_CONFLICT, "CONFLICT", "REPOSITORY_IDENTITY_MISMATCH")
if not observation.branch.startswith("agent/"):
return ReconciliationDecision(ReconciliationAction.FAIL_CONFLICT, "CONFLICT", "BRANCH_SCOPE_DENIED")
if not _valid_oid(observation.desired_oid) or not _valid_oid(observation.expected_oid):
return ReconciliationDecision(ReconciliationAction.FAIL_CONFLICT, "CONFLICT", "OID_INVALID")
if observation.authority_oid is None:
return ReconciliationDecision(ReconciliationAction.MARK_RECONCILIATION_REQUIRED, "RECONCILIATION_REQUIRED", "AUTHORITY_UNAVAILABLE")
if observation.authority_oid != observation.desired_oid:
return ReconciliationDecision(ReconciliationAction.MARK_RECONCILIATION_REQUIRED, "RECONCILIATION_REQUIRED", "AUTHORITY_CONFLICT")
present = [oid for oid in (observation.remote_git_oid, observation.remote_api_oid) if oid is not None]
if any(oid not in {observation.desired_oid, observation.expected_oid} for oid in present):
return ReconciliationDecision(ReconciliationAction.FAIL_CONFLICT, "CONFLICT", "REMOTE_OID_CONFLICT")
if observation.all_equal and observation.commit_visible:
return ReconciliationDecision(ReconciliationAction.MARK_REMOTE_VERIFIED, "REMOTE_VERIFIED", "ALL_OIDS_EQUAL_AND_VISIBLE")
if observation.remote_git_oid is None and observation.remote_api_oid is None:
return ReconciliationDecision(ReconciliationAction.MARK_RECONCILIATION_REQUIRED, "REMOTE_PUSH_PENDING", "AUTHORITY_DESIRED_REMOTE_MISSING")
if observation.remote_git_oid == observation.desired_oid or observation.remote_api_oid == observation.desired_oid:
return ReconciliationDecision(ReconciliationAction.VERIFY_REMOTE, "REMOTE_VERIFY_PENDING", "REMOTE_PROOF_PARTIAL")
if observation.expected_oid is not None and observation.remote_git_oid == observation.expected_oid and observation.remote_api_oid == observation.expected_oid:
return ReconciliationDecision(ReconciliationAction.PUSH_EXACT, "REMOTE_PUSH_PENDING", "SAFE_EXACT_FAST_FORWARD_RETRY", retry_safe=True)
if not observation.commit_visible:
return ReconciliationDecision(ReconciliationAction.VERIFY_REMOTE, "COMMIT_VISIBILITY_PENDING", "COMMIT_VISIBILITY_LAG")
return ReconciliationDecision(ReconciliationAction.MARK_RECONCILIATION_REQUIRED, "RECONCILIATION_REQUIRED", "PARTIAL_REMOTE_OBSERVATION")
def reconcile_observation(**kwargs: object) -> ReconciliationDecision:
return decide_reconciliation(RemoteObservation(**kwargs))
+100
View File
@@ -0,0 +1,100 @@
"""Single supervised socket-activated worker for durable GitOps Jobs.
The worker consumes durable requests and reconciliation queries. Repository
mutation is a separate injected executor and remains impossible while the
legacy global write gate is false.
"""
from __future__ import annotations
import argparse
import json
import os
from pathlib import Path
import signal
import socket
import threading
from typing import Any, Mapping
from aios_gitops_job_controller import GitOpsJobController, JobError, JobState, JobStore, ServiceUnavailable
def runtime_policy(environment: Mapping[str, str] | None = None) -> dict[str, bool]:
env = environment or os.environ
legacy = env.get("AIOS_GITOPS_WRITES_ENABLED", "false").lower() in {"1", "true", "yes", "on"}
durable = env.get("AIOS_GITOPS_DURABLE_JOB_EXECUTION_ENABLED", "false").lower() in {"1", "true", "yes", "on"}
# Legacy direct Git mutation remains disabled. Authorized durable Jobs use
# their own campaign/task capability and may mutate only through the
# worker's lease/CAS path while the legacy gate stays false.
return {"legacy_global_write": legacy, "durable_job_execution": durable, "repository_mutation_allowed": durable}
def handle_request(controller: GitOpsJobController, request: Mapping[str, Any], executor: Any | None = None) -> dict[str, Any]:
job_id = str(request.get("job_id") or "")
if not job_id:
raise JobError("JOB_ID_REQUIRED")
if request.get("action") == "execute":
if executor is None:
raise JobError("JOB_EXECUTOR_UNAVAILABLE")
result = executor.execute(job_id, dry_run=bool(request.get("dry_run", False)))
receipt = controller.store._receipt(controller.store.read(job_id), duplicate=True)
return {"job": result, **receipt.as_dict()}
if request.get("action") == "resume":
receipt = controller.resume(job_id)
else:
row = controller.store.read(job_id)
receipt = controller.store._receipt(row, duplicate=True)
return receipt.as_dict()
def serve_socket(sock: socket.socket, controller: GitOpsJobController, stopping: threading.Event, executor: Any | None = None) -> None:
sock.listen(32)
sock.settimeout(1.0)
while not stopping.is_set():
try:
connection, _ = sock.accept()
except socket.timeout:
continue
with connection:
try:
raw = connection.makefile("rb").readline(1_048_576)
request = json.loads(raw.decode("utf-8"))
response = {"status": "ok", **handle_request(controller, request, executor)}
except ServiceUnavailable as exc:
response = {"status": "error", "error": str(exc), "retryable": True, "side_effect_state": "UNKNOWN"}
except Exception as exc:
response = {"status": "error", "error": type(exc).__name__, "retryable": False}
connection.sendall((json.dumps(response, sort_keys=True) + "\n").encode("utf-8"))
def socket_from_systemd() -> socket.socket:
if os.environ.get("LISTEN_FDS") != "1":
raise RuntimeError("SOCKET_ACTIVATION_REQUIRED")
return socket.fromfd(3, socket.AF_UNIX, socket.SOCK_STREAM)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--job-root", required=True, type=Path)
args = parser.parse_args(argv)
policy = runtime_policy()
if policy["legacy_global_write"]:
raise RuntimeError("LEGACY_GLOBAL_WRITES_MUST_REMAIN_FALSE")
if not policy["durable_job_execution"]:
raise RuntimeError("DURABLE_JOB_EXECUTION_GATE_CLOSED")
controller = GitOpsJobController(JobStore(args.job_root))
from aios_gitops_job_executor import build_from_environment
executor = build_from_environment(args.job_root)
stopping = threading.Event()
signal.signal(signal.SIGTERM, lambda *_: stopping.set())
signal.signal(signal.SIGINT, lambda *_: stopping.set())
sock = socket_from_systemd()
try:
serve_socket(sock, controller, stopping, executor)
finally:
sock.close()
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,59 @@
"""Central fail-closed policy seam for governed task and continuation requests."""
from __future__ import annotations
from dataclasses import dataclass
import re
from typing import Any, Mapping
OID_RE = re.compile(r"^[0-9a-f]{40}$")
@dataclass(frozen=True, slots=True)
class GovernedPolicyInput:
identity: str
task_id: str
repository: str
operation: str
base_oid: str
expected_branch_oid: str
target_branch: str
generation: int
risk: str = "engineering"
requested_capabilities: tuple[str, ...] = ()
previous_generation: int | None = None
@dataclass(frozen=True, slots=True)
class PolicyDecision:
decision: str
reason: str
def as_dict(self) -> dict[str, str]:
return {"decision": self.decision, "reason": self.reason}
def evaluate_policy(value: GovernedPolicyInput | Mapping[str, Any]) -> PolicyDecision:
if isinstance(value, Mapping):
value = GovernedPolicyInput(**value)
if value.repository != "aios/aios":
return PolicyDecision("DENY", "REPOSITORY_IDENTITY_MISMATCH")
if not OID_RE.fullmatch(value.base_oid) or not OID_RE.fullmatch(value.expected_branch_oid):
return PolicyDecision("DENY", "OID_INVALID")
if not value.target_branch.startswith("agent/") or value.target_branch in {"main", "master"}:
return PolicyDecision("DENY", "BRANCH_SCOPE_DENIED")
if "force_push" in value.requested_capabilities or "force" in value.operation.lower():
return PolicyDecision("DENY", "FORCE_PUSH_DENIED")
if {"main_write", "tag_write", "baseline_write", "production_write", "production_deploy", "tag_mutation", "baseline_mutation"}.intersection(value.requested_capabilities):
return PolicyDecision("DENY", "MUTATION_SCOPE_DENIED")
if not isinstance(value.generation, int) or isinstance(value.generation, bool) or value.generation < 1:
return PolicyDecision("DENY", "GENERATION_INVALID")
if value.previous_generation is not None and value.generation <= value.previous_generation:
return PolicyDecision("MANUAL_OWNER_REVIEW_REQUIRED", "GENERATION_NOT_MONOTONIC")
if not value.task_id:
return PolicyDecision("DENY", "TASK_ID_INVALID")
return PolicyDecision("ALLOW", "GOVERNED_ENGINEERING_SCOPE")
class GovernedPolicy:
def decide(self, request: GovernedPolicyInput | Mapping[str, Any]) -> PolicyDecision:
return evaluate_policy(request)
@@ -0,0 +1,84 @@
"""Durable governed task binding for CREATE_NEW and CONTINUE_EXISTING."""
from __future__ import annotations
from dataclasses import dataclass, asdict
import hashlib
import json
from pathlib import Path
import re
from typing import Any, Mapping
from aios_gitops_job_controller import GitOpsJobController, JobStore
OID_RE = re.compile(r"^[0-9a-f]{40}$")
BRANCH_RE = re.compile(r"^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$")
class BootstrapError(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class GovernedTaskBinding:
task_id: str
repository: str
base_oid: str
target_branch: str
risk: str
description: str
idempotency_key: str
expected_branch_oid: str
branch_mode: str
generation: int
job_id: str | None = None
def response(self, *, idempotent: bool = False) -> dict[str, Any]:
return {"task_id": self.task_id, "repository": self.repository, "base_oid": self.base_oid, "target_branch": self.target_branch, "expected_branch_oid": self.expected_branch_oid, "generation": self.generation, "branch_mode": self.branch_mode, "create_target_branch": self.branch_mode == "CREATE_NEW", "worktree_mode": "detached_at_base_oid" if self.branch_mode == "CREATE_NEW" else "detached_at_expected_head", "cas_ref_update_required": True, "idempotent": idempotent, "state": "BOUND", "job_id": self.job_id}
def _safe(value: str) -> str:
return re.sub(r"[^A-Za-z0-9_.-]+", "-", value).strip("-")[:120] or "task"
class GovernedTaskBootstrap:
def __init__(self, authority: Any, root: Path, *, remote_name: str, token_path: Path, ssh_key_path: Path, api_base: str = "https://g.cnnetmkt.top/git/api/v1") -> None:
self.authority = authority; self.root = Path(root); self.remote_name = remote_name; self.token_path = Path(token_path); self.ssh_key_path = Path(ssh_key_path); self.api_base = api_base; self.task_meta_root = self.root / "task-bindings"; self.job_root = self.root / "jobs"; self.task_meta_root.mkdir(parents=True, exist_ok=True); self.job_root.mkdir(parents=True, exist_ok=True)
def _store(self, task_id: str) -> tuple[JobStore, GitOpsJobController]:
root = self.job_root / _safe(task_id); store = JobStore(root, payload_root=root / "payloads"); return store, GitOpsJobController(store)
def _metadata(self, task_id: str) -> Path:
return self.task_meta_root / (_safe(task_id) + ".json")
def _read_existing(self, task_id: str) -> GovernedTaskBinding | None:
path = self._metadata(task_id)
if not path.is_file() or path.is_symlink(): return None
return GovernedTaskBinding(**json.loads(path.read_text(encoding="utf-8")))
def _write(self, binding: GovernedTaskBinding) -> None:
path = self._metadata(binding.task_id); temporary = path.with_suffix(".tmp"); temporary.write_text(json.dumps(asdict(binding), sort_keys=True), encoding="utf-8"); temporary.replace(path)
def create(self, request: Mapping[str, Any]) -> dict[str, Any]:
values = {key: str(request[key]) for key in ("task_id", "repository", "base_oid", "target_branch", "risk", "description", "idempotency_key")}
if values["repository"] != "aios/aios" or not OID_RE.fullmatch(values["base_oid"]): raise BootstrapError("TASK_BINDING_INVALID")
if not BRANCH_RE.fullmatch(values["target_branch"]): raise BootstrapError("TARGET_BRANCH_INVALID")
existing = self._read_existing(values["task_id"])
if existing is not None:
if any(getattr(existing, key) != values[key] for key in ("repository", "base_oid", "target_branch", "risk", "description")): raise BootstrapError("TASK_BINDING_CONFLICT")
return existing.response(idempotent=True)
if not self.authority.commit_exists(values["base_oid"]): raise BootstrapError("BASE_COMMIT_NOT_FOUND")
current = self.authority.read_ref("refs/heads/" + values["target_branch"])
expected = str(request.get("expected_branch_oid") or values["base_oid"])
if current is None:
mode, generation = "CREATE_NEW", 1
if expected != values["base_oid"]: raise BootstrapError("EXPECTED_BASE_CONFLICT")
else:
if expected != current: raise BootstrapError("EXISTING_BRANCH_OID_CONFLICT")
mode, generation = "CONTINUE_EXISTING", int(request.get("generation", 2))
if generation < 1: raise BootstrapError("GENERATION_INVALID")
payload = json.dumps(values, sort_keys=True, separators=(",", ":")).encode()
_, controller = self._store(values["task_id"])
receipt = controller.submit({"idempotency_key": values["idempotency_key"], "campaign_id": values["task_id"], "task_id": values["task_id"], "repository": values["repository"], "operation": "GOVERNED_TASK_CREATE", "branch": values["target_branch"], "expected_branch_oid": expected, "payload_digest": hashlib.sha256(payload).hexdigest(), "lineage_anchor_oid": values["base_oid"]}, payload=payload)
binding = GovernedTaskBinding(**values, expected_branch_oid=expected, branch_mode=mode, generation=generation, job_id=receipt.job_id)
self._write(binding)
return binding.response()
@@ -0,0 +1,306 @@
"""Uncredentialed deterministic executor for governed Online Agent changes.
This process never receives Agent/Owner/Bridge credentials and never writes Git refs.
It operates only on an exchange bundle and returns hash-bound changed files + test evidence.
"""
from __future__ import annotations
import base64
from dataclasses import dataclass
import errno
import hashlib
import io
import json
import os
from pathlib import Path
import re
import shutil
import socket
import subprocess
import tarfile
import tempfile
import time
from typing import Any, Mapping, Sequence
ALLOWED_PROGRAMS = {"/usr/bin/python3", "/usr/bin/git", "/bin/true", "/usr/bin/true"}
MAX_ARCHIVE_BYTES = 64 * 1024 * 1024
MAX_FILE_BYTES = 4 * 1024 * 1024
SECRET_ENV_TOKENS = ("TOKEN", "SECRET", "PASSWORD", "CREDENTIAL", "AUTHORIZATION", "PRIVATE_KEY")
REQUEST_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{15,254}$")
SENSITIVE_VALUE_RE = re.compile(
r"(?i)(authorization\s*[:=]\s*bearer\s+\S+|authorization\s+bearer\s+\S+|"
r"(?:password|passwd|secret|token|api[_-]?key)\s*[:=]\s*\S+)"
)
class IsolatedExecutorError(RuntimeError):
pass
def sha256(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def canonical(value: object) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def safe_relative(value: object) -> str:
if not isinstance(value, str) or not value or value.startswith("/"):
raise IsolatedExecutorError("PATH_INVALID")
path = Path(value)
if ".." in path.parts or "." in path.parts or any(part in {"", ".git"} for part in path.parts):
raise IsolatedExecutorError("PATH_INVALID")
return path.as_posix()
def resolve_under(root: Path, relative: str) -> Path:
root = root.resolve()
path = (root / safe_relative(relative)).resolve(strict=False)
if path == root or root not in path.parents:
raise IsolatedExecutorError("PATH_ESCAPE")
return path
def _credential_exposure(directory: Path) -> str | None:
"""Return an exposure code; EACCES is the intended credential isolation proof."""
try:
if not directory.exists():
return None
if directory.is_symlink():
return "CREDENTIAL_MOUNT_SYMLINK"
if os.access(directory, os.R_OK):
return "CREDENTIAL_MOUNT_ACCESSIBLE"
if directory.is_dir():
for child in directory.iterdir():
if child.is_file() and os.access(child, os.R_OK):
return "CREDENTIAL_FILE_ACCESSIBLE"
except PermissionError:
return None
return None
def assert_uncredentialed() -> None:
credentials = Path("/run/credentials")
candidates = {credentials}
credentials_directory = Path(os.environ.get("CREDENTIALS_DIRECTORY", "/run/credentials"))
candidates.add(credentials_directory)
leaked = sorted(
key for key, value in os.environ.items()
if value
and (any(token in key.upper() for token in SECRET_ENV_TOKENS) or SENSITIVE_VALUE_RE.search(str(value)))
)
if leaked:
raise IsolatedExecutorError("CREDENTIAL_ENV_EXPOSED:" + ",".join(leaked))
for directory in candidates:
exposure = _credential_exposure(directory)
if exposure is not None:
raise IsolatedExecutorError(exposure)
# In the intended PrivateNetwork namespace host loopback services are unreachable.
try:
probe = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except OSError as exc:
if exc.errno in {errno.EACCES, errno.EPERM, errno.EAFNOSUPPORT}:
return
raise IsolatedExecutorError("NETWORK_ISOLATION_PROBE_FAILED") from exc
probe.settimeout(0.15)
try:
if probe.connect_ex(("127.0.0.1", 18892)) == 0:
raise IsolatedExecutorError("GOVERNED_BRIDGE_REACHABLE")
finally:
probe.close()
def extract_source(archive: Path, root: Path) -> None:
if archive.is_symlink() or not archive.is_file() or archive.stat().st_size > MAX_ARCHIVE_BYTES:
raise IsolatedExecutorError("SOURCE_ARCHIVE_INVALID")
with tarfile.open(archive, "r:gz") as tf:
for member in tf.getmembers():
path = Path(member.name)
if path.is_absolute() or ".." in path.parts or ".git" in path.parts:
raise IsolatedExecutorError("SOURCE_ARCHIVE_PATH_INVALID")
if member.issym() or member.islnk():
raise IsolatedExecutorError("SOURCE_ARCHIVE_LINK_DENY")
tf.extractall(root, filter="data")
def run_command(item: Mapping[str, Any], root: Path) -> dict[str, Any]:
argv = item.get("argv")
if not isinstance(argv, Sequence) or isinstance(argv, (str, bytes)) or not argv:
raise IsolatedExecutorError("COMMAND_INVALID")
argv = [str(part) for part in argv]
if argv[0] not in ALLOWED_PROGRAMS:
raise IsolatedExecutorError("COMMAND_PROGRAM_DENY")
if argv[0] == "/usr/bin/python3" and "-m" in argv:
index = argv.index("-m")
if index + 1 >= len(argv) or argv[index + 1] not in {"pytest", "py_compile"}:
raise IsolatedExecutorError("PYTHON_MODULE_DENY")
if argv[0] == "/usr/bin/git" and any(
token in {"push", "update-ref", "commit", "merge", "rebase", "reset", "clean", "branch"}
for token in argv[1:]
):
raise IsolatedExecutorError("GIT_MUTATION_DENY")
timeout = int(item.get("timeout_seconds") or 300)
if not 1 <= timeout <= 1800:
raise IsolatedExecutorError("COMMAND_TIMEOUT_INVALID")
cwd = root if item.get("cwd", ".") == "." else resolve_under(root, str(item.get("cwd")))
started = time.monotonic()
completed = subprocess.run(
argv,
cwd=cwd,
capture_output=True,
text=True,
check=False,
timeout=timeout,
env={"PATH": "/usr/bin:/bin", "LC_ALL": "C.UTF-8"},
)
result = {
"label": str(item.get("label") or "command"),
"exit_code": completed.returncode,
"stdout_sha256": sha256(completed.stdout.encode("utf-8", "replace")),
"stderr_sha256": sha256(completed.stderr.encode("utf-8", "replace")),
"duration_ms": int((time.monotonic() - started) * 1000),
}
if completed.returncode != 0:
raise IsolatedExecutorError(
"COMMAND_FAILED:" + result["label"] + ":" + str(completed.returncode)
)
return result
def apply_changes(request: Mapping[str, Any], root: Path) -> list[str]:
changes = request.get("change_set")
if not isinstance(changes, list) or not changes:
raise IsolatedExecutorError("CHANGE_SET_INVALID")
changed: list[str] = []
for change in changes:
if not isinstance(change, Mapping):
raise IsolatedExecutorError("CHANGE_INVALID")
relative = safe_relative(change.get("path"))
path = resolve_under(root, relative)
current = path.read_bytes() if path.is_file() else None
expected = change.get("expected_sha256")
if expected is not None and (current is None or sha256(current) != expected):
raise IsolatedExecutorError("SOURCE_DRIFT:" + relative)
operation = change.get("operation")
if operation == "write_file":
content = change.get("content")
if not isinstance(content, str):
raise IsolatedExecutorError("WRITE_CONTENT_INVALID")
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8")
elif operation == "replace_text":
if current is None:
raise IsolatedExecutorError("REPLACE_SOURCE_MISSING")
old = str(change.get("old"))
new = str(change.get("new"))
count = int(change.get("count", 1))
text = current.decode("utf-8")
if text.count(old) != count:
raise IsolatedExecutorError("REPLACE_COUNT_INVALID:" + relative)
path.write_text(text.replace(old, new), encoding="utf-8")
elif operation == "delete_file":
if path.exists():
path.unlink()
else:
raise IsolatedExecutorError("CHANGE_OPERATION_INVALID")
changed.append(relative)
return sorted(set(changed))
def changed_bundle(root: Path, changed: list[str]) -> list[dict[str, Any]]:
files = []
for relative in changed:
path = resolve_under(root, relative)
if path.exists():
data = path.read_bytes()
if len(data) > MAX_FILE_BYTES:
raise IsolatedExecutorError("CHANGED_FILE_TOO_LARGE:" + relative)
files.append({
"path": relative,
"deleted": False,
"sha256": sha256(data),
"content_b64": base64.b64encode(data).decode("ascii"),
})
else:
files.append({"path": relative, "deleted": True, "sha256": None, "content_b64": None})
return files
def execute_request(request_path: Path, result_path: Path) -> None:
assert_uncredentialed()
request = json.loads(request_path.read_text(encoding="utf-8"))
if request.get("schema") != "aios.isolated-execution.request.v1":
raise IsolatedExecutorError("REQUEST_SCHEMA_INVALID")
request_id = request.get("request_id")
if not isinstance(request_id, str) or not REQUEST_ID_RE.fullmatch(request_id):
raise IsolatedExecutorError("REQUEST_ID_INVALID")
source_archive = Path(str(request.get("source_archive") or ""))
if not source_archive.is_absolute():
raise IsolatedExecutorError("SOURCE_ARCHIVE_PATH_INVALID")
expected_archive_sha = str(request.get("source_archive_sha256") or "")
if sha256(source_archive.read_bytes()) != expected_archive_sha:
raise IsolatedExecutorError("SOURCE_ARCHIVE_SHA_MISMATCH")
with tempfile.TemporaryDirectory(prefix="aios-isolated-executor-") as temporary:
root = Path(temporary) / "workspace"
root.mkdir()
extract_source(source_archive, root)
changed = apply_changes(request, root)
commands = [
run_command(item, root)
for item in request.get("verification", [])
if isinstance(item, Mapping)
]
result = {
"schema": "aios.isolated-execution.result.v1",
"request_id": request_id,
"request_sha256": sha256(canonical(request)),
"changed_files": changed_bundle(root, changed),
"verification": commands,
"credential_isolation": {
"credential_env_absent": True,
"credential_mount_inaccessible": True,
"governed_bridge_unreachable": True,
},
}
result["result_sha256"] = sha256(canonical(result))
temporary_result = result_path.with_suffix(".tmp")
temporary_result.write_text(json.dumps(result, sort_keys=True, separators=(",", ":")), encoding="utf-8")
os.replace(temporary_result, result_path)
def serve(root: Path) -> None:
assert_uncredentialed()
requests = root / "requests"
results = root / "results"
failed = root / "failed"
for path in (requests, results, failed):
path.mkdir(parents=True, exist_ok=True)
while True:
for request in sorted(requests.glob("*.json")):
if request.is_symlink():
continue
result = results / request.name
try:
execute_request(request, result)
request.unlink()
except Exception as exc:
(failed / request.name).write_text(
json.dumps({"error": str(exc) or type(exc).__name__}, sort_keys=True),
encoding="utf-8",
)
request.unlink(missing_ok=True)
time.sleep(0.5)
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--root", default="/var/lib/aios-online-agent-executor")
parser.add_argument("--request")
parser.add_argument("--result")
args = parser.parse_args()
if args.request and args.result:
execute_request(Path(args.request), Path(args.result))
else:
serve(Path(args.root))
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,301 @@
"""Durable checkpoint journal for the generic Online Agent worker.
The JSON task state remains a fast cache. This SQLite journal is the durable,
append-only evidence used for restart/resume/replay decisions.
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import json
from pathlib import Path
import sqlite3
from typing import Any, Mapping
PHASE_ORDER = {
"claimed": 10,
"workspace_prepared": 20,
"verification_passed": 30,
"source_capture_done": 40,
"commit_done": 40,
"reconciliation_required": 45,
"result_pending": 50,
"result_submitted": 60,
}
class CheckpointError(RuntimeError):
pass
def canonical(value: object) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False, default=str).encode("utf-8")
def sha256_json(value: object) -> str:
return hashlib.sha256(canonical(value)).hexdigest()
def now() -> str:
return datetime.now(timezone.utc).isoformat()
@dataclass(frozen=True)
class CheckpointBinding:
task_id: str
attempt: int
execution_digest: str
job_id: str
repository: str
base_oid: str
expected_branch_oid: str
target_branch: str
generation: int
def material(self) -> dict[str, Any]:
return {
"task_id": self.task_id,
"attempt": self.attempt,
"execution_digest": self.execution_digest,
"job_id": self.job_id,
"repository": self.repository,
"base_oid": self.base_oid,
"expected_branch_oid": self.expected_branch_oid,
"target_branch": self.target_branch,
"generation": self.generation,
}
@property
def binding_sha256(self) -> str:
return sha256_json(self.material())
class CheckpointStore:
def __init__(self, path: Path) -> None:
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
with self._connect() as connection:
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS checkpoints(
task_id TEXT NOT NULL,
attempt INTEGER NOT NULL,
sequence INTEGER NOT NULL,
phase TEXT NOT NULL,
phase_order INTEGER NOT NULL,
execution_digest TEXT NOT NULL,
job_id TEXT NOT NULL,
repository TEXT NOT NULL,
base_oid TEXT NOT NULL,
expected_branch_oid TEXT NOT NULL,
target_branch TEXT NOT NULL,
generation INTEGER NOT NULL,
binding_sha256 TEXT NOT NULL,
payload_json TEXT NOT NULL,
payload_sha256 TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY(task_id, attempt, sequence)
);
CREATE INDEX IF NOT EXISTS idx_checkpoints_task_latest
ON checkpoints(task_id, attempt DESC, sequence DESC);
CREATE UNIQUE INDEX IF NOT EXISTS idx_checkpoints_exact_replay
ON checkpoints(task_id, attempt, phase, payload_sha256);
CREATE TABLE IF NOT EXISTS checkpoint_heads(
task_id TEXT PRIMARY KEY,
attempt INTEGER NOT NULL,
sequence INTEGER NOT NULL,
phase TEXT NOT NULL,
phase_order INTEGER NOT NULL,
execution_digest TEXT NOT NULL,
binding_sha256 TEXT NOT NULL,
payload_sha256 TEXT NOT NULL,
updated_at TEXT NOT NULL
);
"""
)
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA synchronous=FULL")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@staticmethod
def _phase_order(phase: str) -> int:
try:
return PHASE_ORDER[phase]
except KeyError as exc:
raise CheckpointError("CHECKPOINT_PHASE_INVALID") from exc
def append(
self,
binding: CheckpointBinding,
phase: str,
payload: Mapping[str, Any],
) -> dict[str, Any]:
order = self._phase_order(phase)
payload_value = dict(payload)
payload_sha = sha256_json(payload_value)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
replay = connection.execute(
"""
SELECT * FROM checkpoints
WHERE task_id=? AND attempt=? AND phase=? AND payload_sha256=?
""",
(binding.task_id, binding.attempt, phase, payload_sha),
).fetchone()
if replay is not None:
connection.commit()
return dict(replay)
head = connection.execute(
"SELECT * FROM checkpoint_heads WHERE task_id=?",
(binding.task_id,),
).fetchone()
if head is not None:
head_attempt = int(head["attempt"])
head_order = int(head["phase_order"])
if binding.attempt < head_attempt:
raise CheckpointError("CHECKPOINT_ATTEMPT_REGRESSION")
if binding.attempt == head_attempt:
if head["execution_digest"] != binding.execution_digest:
raise CheckpointError("CHECKPOINT_EXECUTION_DIGEST_CONFLICT")
if head["binding_sha256"] != binding.binding_sha256:
raise CheckpointError("CHECKPOINT_SCOPE_MISMATCH_DENY")
if order < head_order and phase != "reconciliation_required":
raise CheckpointError("CHECKPOINT_PHASE_REGRESSION")
sequence = int(head["sequence"]) + 1
else:
sequence = 1
else:
sequence = 1
created_at = now()
connection.execute(
"""
INSERT INTO checkpoints(
task_id,attempt,sequence,phase,phase_order,execution_digest,job_id,
repository,base_oid,expected_branch_oid,target_branch,generation,
binding_sha256,payload_json,payload_sha256,created_at
) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
""",
(
binding.task_id,
binding.attempt,
sequence,
phase,
order,
binding.execution_digest,
binding.job_id,
binding.repository,
binding.base_oid,
binding.expected_branch_oid,
binding.target_branch,
binding.generation,
binding.binding_sha256,
canonical(payload_value).decode("utf-8"),
payload_sha,
created_at,
),
)
connection.execute(
"""
INSERT INTO checkpoint_heads(
task_id,attempt,sequence,phase,phase_order,execution_digest,
binding_sha256,payload_sha256,updated_at
) VALUES (?,?,?,?,?,?,?,?,?)
ON CONFLICT(task_id) DO UPDATE SET
attempt=excluded.attempt,
sequence=excluded.sequence,
phase=excluded.phase,
phase_order=excluded.phase_order,
execution_digest=excluded.execution_digest,
binding_sha256=excluded.binding_sha256,
payload_sha256=excluded.payload_sha256,
updated_at=excluded.updated_at
""",
(
binding.task_id,
binding.attempt,
sequence,
phase,
order,
binding.execution_digest,
binding.binding_sha256,
payload_sha,
created_at,
),
)
connection.commit()
row = connection.execute(
"SELECT * FROM checkpoints WHERE task_id=? AND attempt=? AND sequence=?",
(binding.task_id, binding.attempt, sequence),
).fetchone()
return dict(row)
def latest(self, task_id: str) -> dict[str, Any] | None:
with self._connect() as connection:
row = connection.execute(
"""
SELECT * FROM checkpoints
WHERE task_id=?
ORDER BY attempt DESC, sequence DESC
LIMIT 1
""",
(task_id,),
).fetchone()
if row is None:
return None
value = dict(row)
value["payload"] = json.loads(value.pop("payload_json"))
return value
def latest_for_attempt(self, task_id: str, attempt: int) -> dict[str, Any] | None:
with self._connect() as connection:
row = connection.execute(
"""
SELECT * FROM checkpoints
WHERE task_id=? AND attempt=?
ORDER BY sequence DESC
LIMIT 1
""",
(task_id, attempt),
).fetchone()
if row is None:
return None
value = dict(row)
value["payload"] = json.loads(value.pop("payload_json"))
return value
def verify_latest_binding(self, binding: CheckpointBinding) -> dict[str, Any] | None:
value = self.latest(binding.task_id)
if value is None:
return None
if int(value["attempt"]) > binding.attempt:
raise CheckpointError("CHECKPOINT_FUTURE_ATTEMPT")
# A previous attempt may be reused only when the immutable execution/scope binding matches.
if value["execution_digest"] != binding.execution_digest:
return None
expected = CheckpointBinding(
task_id=value["task_id"],
attempt=int(value["attempt"]),
execution_digest=value["execution_digest"],
job_id=value["job_id"],
repository=value["repository"],
base_oid=value["base_oid"],
expected_branch_oid=value["expected_branch_oid"],
target_branch=value["target_branch"],
generation=int(value["generation"]),
)
current_scope = binding.material()
historical_scope = expected.material()
current_scope.pop("attempt", None)
historical_scope.pop("attempt", None)
if sha256_json(current_scope) != sha256_json(historical_scope):
raise CheckpointError("CHECKPOINT_SCOPE_MISMATCH_DENY")
return value
@@ -0,0 +1,463 @@
"""Bounded read-only repository interface map for governed Online Agent analysis.
The compact map intentionally returns symbols and short source excerpts rather than a
source archive. It is designed for control-plane consumers that cannot safely ingest a
multi-megabyte repository snapshot.
"""
from __future__ import annotations
import ast
import hashlib
import json
from pathlib import Path
import re
from typing import Any, Iterable, Mapping, Sequence
SCHEMA = "aios.engineering.interface-map.v1"
MAX_RESULT_BYTES = 120 * 1024
TARGET_RESULT_BYTES = 108 * 1024
MAX_MODULES = 40
MAX_SYMBOLS = 80
MAX_TESTS = 40
MAX_EXCERPTS = 25
MAX_EXCERPT_CHARS = 1200
MAX_FILE_BYTES = 2 * 1024 * 1024
ROOTS = (
"active-runtime/app",
"runtime/agent-os",
"tests",
"db/migrations",
"migrations",
"deploy/systemd",
"docs/task-plane",
"docs/superpowers/specs",
)
SUFFIXES = {".py", ".sql", ".service", ".md", ".json", ".yaml", ".yml"}
DEFAULT_TERMS = {
"agent", "worker", "fleet", "scheduler", "scheduling", "task", "claim", "lease",
"heartbeat", "capacity", "concurrency", "priority", "fairness", "queue", "retry",
"backoff", "circuit", "health", "liveness", "scope", "tenant", "project", "trace",
"message", "checkpoint", "recovery", "idempotency", "dispatch", "assignment",
}
FLEET_CONTRACT_TERMS: dict[str, tuple[str, ...]] = {
"FLEET_AGENT_REGISTRATION": ("agent", "register", "worker", "capability", "version"),
"FLEET_CAPACITY_SCHEDULING": ("capacity", "slot", "concurrency", "scheduler", "queue"),
"FLEET_FAIRNESS": ("fair", "priority", "queue", "tenant", "project", "starvation"),
"FLEET_FAULT_ISOLATION": ("failure", "health", "liveness", "timeout", "circuit", "poison"),
"FLEET_WORKER_REASSIGNMENT": ("reassign", "assignment", "claim", "lease", "retry", "worker"),
"FLEET_SCOPE_ISOLATION": ("tenant", "project", "scope", "conversation", "task", "run"),
"FLEET_IDEMPOTENT_DISPATCH": ("idempot", "dispatch", "claim", "lease", "result", "attempt"),
"FLEET_FAILURE_RECOVERY": ("checkpoint", "resume", "recover", "retry", "cleanup", "receipt"),
}
MODEL_GOVERNANCE_CONTRACT_TERMS: dict[str, tuple[str, ...]] = {
"MODEL_PROVIDER_REGISTRY": ("model", "provider", "registry", "endpoint", "capability"),
"MODEL_SELECTION_POLICY": ("model", "select", "route", "policy", "capability"),
"MODEL_VERSION_PINNING": ("model", "version", "pin", "revision", "snapshot"),
"MODEL_BUDGET_ACCOUNTING": ("budget", "token", "cost", "quota", "usage"),
"MODEL_FALLBACK_POLICY": ("fallback", "retry", "route", "provider", "model"),
"MODEL_SCOPE_ISOLATION": ("tenant", "project", "scope", "policy", "model"),
"MODEL_CREDENTIAL_BOUNDARY": ("credential", "secret", "provider", "tool", "token"),
"MODEL_TRACE_AUDIT": ("trace", "audit", "model", "provider", "request"),
"MODEL_EVALUATION_GATE": ("eval", "evaluation", "approval", "quality", "gate"),
"MODEL_ROUTING_RESILIENCE": ("route", "retry", "fallback", "timeout", "health"),
}
PROFILE_CLASSIFICATION = {
"fleet": "P2_INTERFACE_MAP_CAPTURED",
"model_governance": "MODEL_GOVERNANCE_INTERFACE_MAP_CAPTURED",
}
MODEL_PROFILE_MARKERS = {
"model", "provider", "llm", "inference", "routing", "router", "budget",
"token", "cost", "quota", "fallback", "evaluation", "eval", "approval",
"version", "pinning",
}
SENSITIVE_RE = re.compile(
r"(?i)(authorization\s*:\s*bearer\s+\S+|"
r"(?:password|passwd|secret|token|api[_-]?key)\s*[:=]\s*['\"]?[^'\"\s,;]+)"
)
SQL_TABLE_RE = re.compile(
r"(?is)\bcreate\s+table\s+(?:if\s+not\s+exists\s+)?[\"`[]?([A-Za-z_][A-Za-z0-9_.-]*)"
)
class CompactInterfaceMapError(RuntimeError):
pass
def _sha(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def _bounded_excerpt(text: str, start_line: int, end_line: int) -> str:
lines = text.splitlines()
start = max(0, start_line - 2)
end = min(len(lines), max(end_line + 1, start + 1))
excerpt = "\n".join(lines[start:end])
excerpt = SENSITIVE_RE.sub("<redacted-sensitive-source-pattern>", excerpt)
if len(excerpt) > MAX_EXCERPT_CHARS:
excerpt = excerpt[: MAX_EXCERPT_CHARS - 1] + "…"
return excerpt
def _signal_terms(signals: Sequence[str]) -> set[str]:
terms = set(DEFAULT_TERMS)
for signal in signals:
if not isinstance(signal, str):
continue
terms.update(
token.lower()
for token in re.findall(r"[A-Za-z][A-Za-z0-9_.-]{2,63}", signal)
)
return set(sorted(terms)[:96])
def _profile(source_signals: Sequence[str]) -> str:
tokens: set[str] = set()
for signal in source_signals:
if not isinstance(signal, str):
continue
tokens.update(
token.lower()
for token in re.findall(r"[A-Za-z][A-Za-z0-9_.-]{2,63}", signal)
)
model_score = len(tokens & MODEL_PROFILE_MARKERS)
return "model_governance" if model_score >= 2 else "fleet"
def _contract_terms(profile: str) -> Mapping[str, tuple[str, ...]]:
if profile == "model_governance":
return MODEL_GOVERNANCE_CONTRACT_TERMS
if profile == "fleet":
return FLEET_CONTRACT_TERMS
raise CompactInterfaceMapError("INTERFACE_MAP_PROFILE_INVALID")
def _candidate_files(root: Path, terms: set[str]) -> list[dict[str, Any]]:
root = root.resolve()
candidates: list[dict[str, Any]] = []
for relative_root in ROOTS:
directory = (root / relative_root).resolve(strict=False)
if directory == root or root not in directory.parents:
raise CompactInterfaceMapError("INTERFACE_MAP_SCOPE_INVALID")
if not directory.is_dir() or directory.is_symlink():
continue
for path in sorted(directory.rglob("*")):
if path.is_symlink() or not path.is_file() or path.suffix.lower() not in SUFFIXES:
continue
resolved = path.resolve(strict=True)
if root not in resolved.parents:
raise CompactInterfaceMapError("INTERFACE_MAP_PATH_ESCAPE")
data = path.read_bytes()
if len(data) > MAX_FILE_BYTES:
continue
relative = path.relative_to(root).as_posix()
text = data.decode("utf-8", "ignore")
searchable = (relative + "\n" + text[:384000]).lower()
matched = sorted(term for term in terms if term in searchable)
if not matched:
continue
path_bonus = sum(3 for term in terms if term in relative.lower())
test_bonus = 3 if relative.startswith("tests/") else 0
score = min(250, len(matched) + path_bonus + test_bonus)
candidates.append(
{
"path": relative,
"sha256": _sha(data),
"bytes": len(data),
"score": score,
"text": text,
"suffix": path.suffix.lower(),
}
)
candidates.sort(key=lambda item: (-item["score"], item["path"]))
return candidates
def _python_symbols(item: Mapping[str, Any], terms: set[str]) -> list[dict[str, Any]]:
try:
tree = ast.parse(str(item["text"]), filename=str(item["path"]))
except SyntaxError:
return []
result: list[dict[str, Any]] = []
class Visitor(ast.NodeVisitor):
def __init__(self) -> None:
self.stack: list[str] = []
def _record(self, node: ast.AST, kind: str, name: str) -> None:
qualified = ".".join([*self.stack, name]) if self.stack else name
haystack = f"{item['path']} {qualified}".lower()
score = sum(3 if term in haystack else 0 for term in terms)
if name.startswith("test_"):
score += 12
result.append(
{
"path": item["path"],
"symbol": qualified,
"kind": kind,
"line": int(getattr(node, "lineno", 1)),
"end_line": int(getattr(node, "end_lineno", getattr(node, "lineno", 1))),
"score": score + int(item["score"]),
}
)
def visit_ClassDef(self, node: ast.ClassDef) -> Any:
self._record(node, "class", node.name)
self.stack.append(node.name)
self.generic_visit(node)
self.stack.pop()
def visit_FunctionDef(self, node: ast.FunctionDef) -> Any:
self._record(node, "function", node.name)
self.stack.append(node.name)
self.generic_visit(node)
self.stack.pop()
def visit_AsyncFunctionDef(self, node: ast.AsyncFunctionDef) -> Any:
self._record(node, "async_function", node.name)
self.stack.append(node.name)
self.generic_visit(node)
self.stack.pop()
Visitor().visit(tree)
return result
def _surface_ref(symbol: Mapping[str, Any]) -> str:
return f"{symbol['path']}::{symbol['symbol']}"
def _contract_map(
symbols: Sequence[Mapping[str, Any]],
modules: Sequence[Mapping[str, Any]],
contract_terms: Mapping[str, tuple[str, ...]],
) -> dict[str, Any]:
result: dict[str, Any] = {}
module_paths = [str(item["path"]) for item in modules]
for contract, terms in contract_terms.items():
scored: list[tuple[int, str]] = []
for symbol in symbols:
haystack = _surface_ref(symbol).lower()
score = sum(1 for term in terms if term in haystack)
if score:
scored.append((score, _surface_ref(symbol)))
scored.sort(key=lambda item: (-item[0], item[1]))
surfaces = [value for _, value in scored[:6]]
if not surfaces:
for path in module_paths:
score = sum(1 for term in terms if term in path.lower())
if score:
surfaces.append(path)
if len(surfaces) == 6:
break
result[contract] = {
"status": "PARTIAL" if surfaces else "GAP",
"surfaces": surfaces,
}
return result
def _gap_evidence(
symbols: Sequence[Mapping[str, Any]],
profile: str,
) -> tuple[list[str], list[str], list[str]]:
haystack = "\n".join(_surface_ref(item).lower() for item in symbols)
if profile == "model_governance":
invariants = [
"Online Task claim/heartbeat remains the authoritative worker ownership boundary.",
"Provider/model credentials remain outside uncredentialed planner/executor processes.",
"P0-A tenant/project scope and P0-C trace/audit lineage must constrain model routing.",
"P1 retry/checkpoint semantics remain authoritative for model-call execution recovery.",
]
failure_windows: list[str] = []
if "provider" not in haystack and "model" not in haystack:
failure_windows.append("No first-class model/provider registry surface detected.")
if "budget" not in haystack and "cost" not in haystack and "quota" not in haystack:
failure_windows.append("No explicit durable model budget/cost/quota accounting surface detected.")
if "fallback" not in haystack and "route" not in haystack:
failure_windows.append("No explicit model fallback/routing policy surface detected.")
if "eval" not in haystack and "approval" not in haystack:
failure_windows.append("No explicit evaluation/approval gate surface detected for model changes.")
if "version" not in haystack and "pin" not in haystack:
failure_windows.append("No explicit model-version pinning surface detected.")
desired = (
("trace", "P0-C trace/audit lineage"),
("scope", "P0-A tenant/project scope isolation"),
("credential", "P0-C credential governance"),
("checkpoint", "P1 durable execution recovery"),
("message", "P0-B MessageEnvelope/handoff"),
("retry", "P1 retry/idempotency"),
)
else:
invariants = [
"Online Task claim/heartbeat remains the authoritative worker ownership boundary.",
"Governed mutation remains separated from read-only analysis and requires WriterLease.",
"P1 attempt-aware result idempotency and durable checkpoint recovery are reusable fleet primitives.",
"Scope/trace/message lineage must remain attached when work is scheduled or reassigned.",
]
failure_windows = []
if "scheduler" not in haystack:
failure_windows.append("No first-class durable fleet scheduler symbol detected; worker-local polling may still be the scheduling mechanism.")
if "capacity" not in haystack and "slot" not in haystack:
failure_windows.append("No explicit durable worker-capacity/slot primitive detected.")
if "fair" not in haystack and "starvation" not in haystack:
failure_windows.append("No explicit fairness/starvation policy symbol detected.")
if "circuit" not in haystack:
failure_windows.append("No explicit circuit-breaker primitive detected for repeated worker/task failures.")
if "reassign" not in haystack:
failure_windows.append("No explicit worker-reassignment primitive detected beyond lease expiry/retry semantics.")
desired = (
("checkpoint", "P1 durable checkpoint/resume"),
("message", "P0-B MessageEnvelope/handoff"),
("trace", "P0-C trace correlation"),
("scope", "P0-A scope isolation"),
("lease", "claim/WriterLease ownership"),
("heartbeat", "worker liveness"),
("idempot", "retry-safe dispatch/result"),
)
reuse_points: list[str] = []
for token, label in desired:
match = next((item for item in symbols if token in _surface_ref(item).lower()), None)
if match is not None:
reuse_points.append(f"{label}: {_surface_ref(match)}")
return invariants, failure_windows, reuse_points
def _trim_to_budget(value: dict[str, Any]) -> dict[str, Any]:
def size() -> int:
return len(json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":")).encode("utf-8"))
while size() > TARGET_RESULT_BYTES and value["bounded_excerpts"]:
value["bounded_excerpts"].pop()
while size() > TARGET_RESULT_BYTES and len(value["implementation_map"]["functions"]) > 20:
value["implementation_map"]["functions"].pop()
while size() > TARGET_RESULT_BYTES and len(value["implementation_map"]["classes"]) > 15:
value["implementation_map"]["classes"].pop()
while size() > TARGET_RESULT_BYTES and len(value["implementation_map"]["tests"]) > 15:
value["implementation_map"]["tests"].pop()
while size() > TARGET_RESULT_BYTES and len(value["implementation_map"]["modules"]) > 20:
value["implementation_map"]["modules"].pop()
final_size = size()
if final_size > MAX_RESULT_BYTES:
raise CompactInterfaceMapError("INTERFACE_MAP_RESULT_TOO_LARGE")
value["result_bytes"] = final_size
return value
def build_compact_interface_map(
worktree: Path,
source_signals: Sequence[str],
*,
source_commit: str,
) -> dict[str, Any]:
root = Path(worktree)
if not root.is_dir() or root.is_symlink():
raise CompactInterfaceMapError("INTERFACE_MAP_WORKTREE_INVALID")
if re.fullmatch(r"[0-9a-f]{40}", source_commit) is None:
raise CompactInterfaceMapError("INTERFACE_MAP_SOURCE_COMMIT_INVALID")
profile = _profile(source_signals)
terms = _signal_terms(source_signals)
candidates = _candidate_files(root, terms)
if not candidates:
raise CompactInterfaceMapError("INTERFACE_MAP_EMPTY")
module_candidates = candidates[:MAX_MODULES]
modules = [
{
"path": item["path"],
"sha256": item["sha256"],
"bytes": item["bytes"],
"score": item["score"],
}
for item in module_candidates
]
symbols: list[dict[str, Any]] = []
tables: list[dict[str, Any]] = []
systemd_units: list[dict[str, Any]] = []
for item in candidates:
if item["suffix"] == ".py":
symbols.extend(_python_symbols(item, terms))
elif item["suffix"] == ".sql":
for table in SQL_TABLE_RE.findall(str(item["text"])):
tables.append({"path": item["path"], "table": table, "sha256": item["sha256"]})
elif item["suffix"] == ".service":
systemd_units.append({"path": item["path"], "unit": Path(str(item["path"])).name, "sha256": item["sha256"]})
symbols.sort(key=lambda item: (-int(item["score"]), str(item["path"]), str(item["symbol"])))
all_symbols = list(symbols)
test_symbols = [
item for item in all_symbols
if str(item["path"]).startswith("tests/") or str(item["symbol"]).split(".")[-1].startswith("test_")
][: min(MAX_TESTS, 24)]
non_test_symbols = [
item for item in all_symbols
if item not in test_symbols
][: max(0, MAX_SYMBOLS - len(test_symbols))]
symbols = [*test_symbols, *non_test_symbols]
symbols.sort(key=lambda item: (-int(item["score"]), str(item["path"]), str(item["symbol"])))
classes = [
{k: item[k] for k in ("path", "symbol", "line", "end_line")}
for item in symbols
if item["kind"] == "class"
]
functions = [
{k: item[k] for k in ("path", "symbol", "line", "end_line")}
for item in symbols
if item["kind"] in {"function", "async_function"} and not str(item["symbol"]).split(".")[-1].startswith("test_")
]
tests = [
{k: item[k] for k in ("path", "symbol", "line", "end_line")}
for item in symbols
if str(item["path"]).startswith("tests/") or str(item["symbol"]).split(".")[-1].startswith("test_")
][:MAX_TESTS]
excerpts: list[dict[str, Any]] = []
by_path = {str(item["path"]): item for item in candidates}
for symbol in symbols:
if len(excerpts) == MAX_EXCERPTS:
break
item = by_path.get(str(symbol["path"]))
if item is None:
continue
excerpt = _bounded_excerpt(str(item["text"]), int(symbol["line"]), int(symbol["end_line"]))
if not excerpt.strip():
continue
excerpts.append(
{
"path": symbol["path"],
"symbol": symbol["symbol"],
"sha256": item["sha256"],
"excerpt": excerpt,
}
)
invariants, failure_windows, reuse_points = _gap_evidence(all_symbols, profile)
result = {
"schema": SCHEMA,
"profile": profile,
"classification": PROFILE_CLASSIFICATION[profile],
"source_commit": source_commit,
"implementation_map": {
"modules": modules,
"classes": classes[:MAX_SYMBOLS],
"functions": functions[:MAX_SYMBOLS],
"tables": tables[:40],
"systemd_units": systemd_units[:20],
"tests": tests,
"current_invariants": invariants,
"failure_windows": failure_windows,
"reuse_points": reuse_points,
},
"bounded_excerpts": excerpts,
"contracts": _contract_map(all_symbols, modules, _contract_terms(profile)),
}
return _trim_to_budget(result)
@@ -0,0 +1,482 @@
"""Canonical governed execution projection for Online Agent Tasks.
The durable governed binding is authoritative. Natural-language task fields and
worker configuration are never allowed to invent repository, ref, generation, or
idempotency coordinates.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass
import hashlib
import json
import re
from pathlib import Path
from typing import Any, Mapping, Sequence
import urllib.error
import urllib.request
EXECUTION_SCHEMA = "aios.online-agent.execution.v2"
PROJECTION_SCHEMA = "aios.online-agent.task-projection.v1"
OID_RE = re.compile(r"^[0-9a-f]{40}$")
TASK_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{2,179}$")
BRANCH_RE = re.compile(r"^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$")
KEY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{7,159}$")
PROTECTED_PREFIXES = ("main", "master", "refs/", "tags/", "tag/", "release/", "baseline/", "production/")
MUTATION_ACTIONS = frozenset({"deterministic"})
READ_ONLY_ACTIONS = frozenset({"source_capture", "agent_runtime"})
SUPPORTED_ACTIONS = frozenset({"deterministic", "source_capture", "agent_runtime"})
AGENT_RUNTIME_ADAPTERS = frozenset({"hermes", "openclaw"})
MAX_AGENT_RUNTIME_INPUT_BYTES = 64 * 1024
class ExecutionProjectionError(RuntimeError):
"""A canonical fail-closed projection or validation error."""
class OnlineTaskProjectionClient:
"""Official HTTP projection client using only a control credential reference."""
def __init__(self, api_base: str, control_token_path: Path) -> None:
self.api_base = api_base.rstrip("/")
self.control_token_path = Path(control_token_path)
def _token(self) -> str:
path = self.control_token_path
if path.is_symlink() or not path.is_file():
_error("ONLINE_TASK_PROJECTION_CONTROL_UNAVAILABLE")
value = path.read_text(encoding="utf-8").strip()
if len(value) < 32:
_error("ONLINE_TASK_PROJECTION_CONTROL_INVALID")
return value
def create(
self,
projected_task: Mapping[str, Any],
*,
binding: "GovernedExecutionBinding",
) -> dict[str, Any]:
body = dict(projected_task)
payload = _canonical(body).encode("utf-8")
request = urllib.request.Request(
self.api_base + "/v1/tasks",
method="POST",
data=payload,
headers={
"Authorization": "Bearer " + self._token(),
"Content-Type": "application/json",
"Accept": "application/json",
},
)
try:
with urllib.request.urlopen(request, timeout=15) as response:
result = json.loads(response.read().decode("utf-8"))
status = response.status
except urllib.error.HTTPError as exc:
raw = exc.read(8192).decode("utf-8", "replace")
try:
error = json.loads(raw).get("error")
except (AttributeError, json.JSONDecodeError):
error = "HTTP_ERROR"
raise ExecutionProjectionError(f"ONLINE_TASK_PROJECTION_FAILED:{exc.code}:{error}") from exc
except (OSError, urllib.error.URLError, TimeoutError) as exc:
raise ExecutionProjectionError("ONLINE_TASK_PROJECTION_UNAVAILABLE") from exc
if status not in {200, 201} or not isinstance(result, dict):
_error("ONLINE_TASK_PROJECTION_RESPONSE_INVALID")
if result.get("task_id") != binding.task_id or result.get("status") not in {"QUEUED", "VERIFYING", "SUCCEEDED"}:
_error("ONLINE_TASK_PROJECTION_RESPONSE_INVALID")
metadata = result.get("metadata")
if not isinstance(metadata, Mapping):
_error("ONLINE_TASK_PROJECTION_RESPONSE_INVALID")
returned_execution = validate_execution_envelope(
_object(metadata.get("execution"), "ONLINE_TASK_PROJECTION_RESPONSE_INVALID"),
binding=binding,
)
submitted_metadata = _object(body.get("metadata"), "ONLINE_TASK_PROJECTION_REQUEST_INVALID")
submitted_execution = validate_execution_envelope(
_object(submitted_metadata.get("execution"), "ONLINE_TASK_PROJECTION_REQUEST_INVALID"),
binding=binding,
)
if _canonical(returned_execution) != _canonical(submitted_execution):
_error("ONLINE_TASK_PROJECTION_RESPONSE_BINDING_INVALID")
return result
def _error(code: str) -> None:
raise ExecutionProjectionError(code)
def _object(value: object, code: str) -> dict[str, Any]:
if not isinstance(value, Mapping):
_error(code)
return dict(value)
def _string(value: object, code: str) -> str:
if not isinstance(value, str) or not value.strip():
_error(code)
return value.strip()
def _task_id(value: object) -> str:
value = _string(value, "TASK_ID_INVALID")
if TASK_RE.fullmatch(value) is None:
_error("TASK_ID_INVALID")
return value
def _oid(value: object, code: str) -> str:
value = _string(value, code)
if OID_RE.fullmatch(value) is None:
_error(code)
return value
def _branch(value: object) -> str:
value = _string(value, "TARGET_BRANCH_INVALID")
if BRANCH_RE.fullmatch(value) is None or value.startswith(PROTECTED_PREFIXES):
_error("TARGET_BRANCH_INVALID")
return value
def _generation(value: object) -> int:
if not isinstance(value, int) or isinstance(value, bool) or value < 1:
_error("GENERATION_INVALID")
return value
def _canonical(value: object) -> str:
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
def _digest(value: object) -> str:
return hashlib.sha256(_canonical(value).encode("utf-8")).hexdigest()
def _source_signals(value: object) -> tuple[str, ...]:
"""Normalize untrusted natural-language hints before they enter a signed envelope."""
if value is None:
return ()
if not isinstance(value, Sequence) or isinstance(value, (str, bytes)):
_error("SOURCE_CAPTURE_SIGNALS_INVALID")
result: list[str] = []
for item in value:
if not isinstance(item, str):
_error("SOURCE_CAPTURE_SIGNALS_INVALID")
text = " ".join(item.split()).strip()
if not text:
continue
# Projection may accept long task prose, but execution envelopes stay bounded.
text = text[:512].rstrip()
if text not in result:
result.append(text)
if len(result) == 16:
break
return tuple(result)
def _agent_runtime_input(value: object) -> dict[str, Any]:
if not isinstance(value, Mapping):
_error("AGENT_RUNTIME_INPUT_REQUIRED")
raw = dict(value)
allowed = {"goal", "instructions", "success_criteria", "constraints"}
if set(raw) - allowed:
_error("AGENT_RUNTIME_INPUT_FIELDS_INVALID")
goal = _string(raw.get("goal"), "AGENT_RUNTIME_GOAL_REQUIRED")
instructions = raw.get("instructions", "")
if not isinstance(instructions, str):
_error("AGENT_RUNTIME_INSTRUCTIONS_INVALID")
criteria = raw.get("success_criteria", [])
if not isinstance(criteria, list) or len(criteria) > 32:
_error("AGENT_RUNTIME_SUCCESS_CRITERIA_INVALID")
normalized_criteria: list[dict[str, str]] = []
for item in criteria:
if not isinstance(item, Mapping):
_error("AGENT_RUNTIME_SUCCESS_CRITERIA_INVALID")
criterion_id = _string(item.get("id"), "AGENT_RUNTIME_SUCCESS_CRITERIA_INVALID")
description = _string(
item.get("description"), "AGENT_RUNTIME_SUCCESS_CRITERIA_INVALID"
)
normalized_criteria.append(
{"id": criterion_id[:120], "description": description[:1000]}
)
constraints = raw.get("constraints", {})
if not isinstance(constraints, Mapping):
_error("AGENT_RUNTIME_CONSTRAINTS_INVALID")
sensitive_markers = ("token", "secret", "password", "credential", "authorization", "api_key", "apikey")
for key in constraints:
lowered = str(key).lower()
if any(marker in lowered for marker in sensitive_markers):
_error("AGENT_RUNTIME_SECRET_FIELD_DENY")
normalized = {
"goal": goal[:12000],
"instructions": instructions[:32000],
"success_criteria": normalized_criteria,
"constraints": dict(constraints),
}
if len(_canonical(normalized).encode("utf-8")) > MAX_AGENT_RUNTIME_INPUT_BYTES:
_error("AGENT_RUNTIME_INPUT_TOO_LARGE")
return normalized
def _capabilities(value: object) -> tuple[str, ...]:
if value is None:
return ()
if not isinstance(value, Sequence) or isinstance(value, (str, bytes)):
_error("CAPABILITY_SCOPE_INVALID")
result = tuple(sorted({_string(item, "CAPABILITY_SCOPE_INVALID") for item in value}))
if any(len(item) > 120 for item in result):
_error("CAPABILITY_SCOPE_INVALID")
return result
@dataclass(frozen=True, slots=True)
class GovernedExecutionBinding:
task_id: str
job_id: str
repository: str
base_oid: str
expected_branch_oid: str
target_branch: str
branch_mode: str
generation: int
idempotency_key: str
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "GovernedExecutionBinding":
data = dict(value)
task_id = _task_id(data.get("task_id"))
job_id = _string(data.get("job_id"), "JOB_ID_REQUIRED")
if TASK_RE.fullmatch(job_id) is None:
_error("JOB_ID_INVALID")
repository = _string(data.get("repository"), "REPOSITORY_INVALID")
if repository != "aios/aios":
_error("WRONG_REPOSITORY_ENVELOPE_DENY")
base_oid = _oid(data.get("base_oid"), "BASE_OID_INVALID")
expected = _oid(data.get("expected_branch_oid"), "EXPECTED_BRANCH_OID_INVALID")
target = _branch(data.get("target_branch"))
mode = _string(data.get("branch_mode"), "BRANCH_MODE_MISSING")
if mode not in {"CREATE_NEW", "CONTINUE_EXISTING"}:
_error("BRANCH_MODE_INVALID")
generation = _generation(data.get("generation"))
if mode == "CREATE_NEW" and expected != base_oid:
_error("EXPECTED_BASE_CONFLICT")
key = _string(data.get("idempotency_key"), "IDEMPOTENCY_KEY_INVALID")
if KEY_RE.fullmatch(key) is None:
_error("IDEMPOTENCY_KEY_INVALID")
return cls(task_id, job_id, repository, base_oid, expected, target, mode, generation, key)
def public(self) -> dict[str, Any]:
return asdict(self)
def binding_from_bootstrap(value: Mapping[str, Any]) -> GovernedExecutionBinding:
return GovernedExecutionBinding.from_mapping(value)
def validate_execution_envelope(
value: Mapping[str, Any],
*,
binding: GovernedExecutionBinding | None = None,
worker_capabilities: Sequence[str] | None = None,
) -> dict[str, Any]:
envelope = dict(value)
if envelope.get("schema") != EXECUTION_SCHEMA:
_error("EXECUTION_ENVELOPE_SCHEMA_VALIDATION")
candidate = GovernedExecutionBinding.from_mapping(envelope)
if binding is not None and candidate != binding:
comparisons = (
(candidate.task_id, binding.task_id, "TASK_BINDING_CONFLICT"),
(candidate.job_id, binding.job_id, "JOB_BINDING_CONFLICT"),
(candidate.repository, binding.repository, "WRONG_REPOSITORY_ENVELOPE_DENY"),
(candidate.base_oid, binding.base_oid, "WRONG_BASE_ENVELOPE_DENY"),
(candidate.expected_branch_oid, binding.expected_branch_oid, "WRONG_EXPECTED_HEAD_ENVELOPE_DENY"),
(candidate.target_branch, binding.target_branch, "WRONG_BRANCH_ENVELOPE_DENY"),
(candidate.generation, binding.generation, "WRONG_GENERATION_ENVELOPE_DENY"),
(candidate.branch_mode, binding.branch_mode, "BRANCH_MODE_BINDING_CONFLICT"),
(candidate.idempotency_key, binding.idempotency_key, "IDEMPOTENCY_BINDING_CONFLICT"),
)
for actual, expected, code in comparisons:
if actual != expected:
_error(code)
action = _string(envelope.get("action"), "EXECUTION_ACTION_MISSING")
if action not in SUPPORTED_ACTIONS:
_error("EXECUTION_ACTION_UNSUPPORTED")
capability_scope = _capabilities(envelope.get("capability_scope"))
if worker_capabilities is not None and not set(capability_scope).issubset(set(worker_capabilities)):
_error("CAPABILITY_MISMATCH_DENY")
workspace = _object(envelope.get("workspace"), "WORKSPACE_POLICY_REQUIRED")
if workspace.get("authority_git_dir") != "/data/aios-gitops/repo.git" or workspace.get("read_only_repository") is not True:
_error("WORKSPACE_POLICY_DENY")
governance = _object(envelope.get("governance"), "GOVERNANCE_POLICY_REQUIRED")
governed_coordinates = (
(governance.get("task_id"), candidate.task_id, "TASK_BINDING_CONFLICT"),
(governance.get("job_id"), candidate.job_id, "JOB_BINDING_CONFLICT"),
(governance.get("idempotency_key"), candidate.idempotency_key, "IDEMPOTENCY_BINDING_CONFLICT"),
)
for actual_coordinate, expected_coordinate, code in governed_coordinates:
if actual_coordinate != expected_coordinate:
_error(code)
commit_key = _string(governance.get("commit_idempotency_key"), "COMMIT_IDEMPOTENCY_KEY_INVALID")
if KEY_RE.fullmatch(commit_key) is None:
_error("COMMIT_IDEMPOTENCY_KEY_INVALID")
if governance.get("writer_lease") is not (action in MUTATION_ACTIONS):
_error("WRITER_LEASE_POLICY_DENY")
for field in ("main_write", "production_write", "tag_write", "baseline_write", "force_push", "protected_ref_bypass"):
if governance.get(field, False) is not False:
_error("POLICY_DENY")
changes = envelope.get("change_set")
verification = envelope.get("verification")
if not isinstance(changes, list) or not isinstance(verification, list):
_error("EXECUTION_OPERATIONS_INVALID")
if action == "source_capture":
if changes or verification:
_error("SOURCE_CAPTURE_MUTATION_DENY")
signals = list(_source_signals(envelope.get("source_signals") or []))
if envelope.get("source_signals") != signals:
_error("SOURCE_CAPTURE_SIGNALS_INVALID")
if envelope.get("runtime_input") not in (None, {}):
_error("SOURCE_CAPTURE_RUNTIME_INPUT_DENY")
elif action == "agent_runtime":
if changes or verification:
_error("AGENT_RUNTIME_MUTATION_DENY")
adapter = str(envelope.get("adapter") or "")
if adapter not in AGENT_RUNTIME_ADAPTERS:
_error("AGENT_RUNTIME_ADAPTER_INVALID")
runtime_input = _agent_runtime_input(envelope.get("runtime_input"))
if envelope.get("runtime_input") != runtime_input:
_error("AGENT_RUNTIME_INPUT_NOT_CANONICAL")
required_runtime_capability = f"agent.runtime.{adapter}"
if required_runtime_capability not in capability_scope:
_error("AGENT_RUNTIME_CAPABILITY_REQUIRED")
if envelope.get("source_signals") not in (None, []):
_error("AGENT_RUNTIME_SOURCE_SIGNALS_DENY")
elif not changes:
_error("CHANGE_SET_INVALID")
declared = str(envelope.get("execution_digest") or "")
material = {key: item for key, item in envelope.items() if key != "execution_digest"}
actual = _digest(material)
if declared != actual:
_error("EXECUTION_DIGEST_INVALID")
return {**material, "execution_digest": actual}
def project_execution_envelope(
binding: GovernedExecutionBinding,
execution_request: Mapping[str, Any] | None = None,
*,
required_capabilities: Sequence[str] = (),
) -> dict[str, Any]:
request = dict(execution_request or {})
action = str(request.get("action") or "source_capture")
if action not in SUPPORTED_ACTIONS:
_error("EXECUTION_ACTION_UNSUPPORTED")
capabilities = _capabilities(request.get("capability_scope", required_capabilities))
governance = {
"task_id": binding.task_id,
"job_id": binding.job_id,
"idempotency_key": binding.idempotency_key,
"commit_idempotency_key": str(request.get("commit_idempotency_key") or f"{binding.task_id}-commit-g{binding.generation}"),
"writer_lease": action in MUTATION_ACTIONS,
"main_write": False,
"production_write": False,
"tag_write": False,
"baseline_write": False,
"force_push": False,
"protected_ref_bypass": False,
}
supplied_governance = request.get("governance")
if supplied_governance is not None:
governance.update(_object(supplied_governance, "GOVERNANCE_POLICY_REQUIRED"))
material: dict[str, Any] = {
"schema": EXECUTION_SCHEMA,
**binding.public(),
"action": action,
"adapter": str(
request.get("adapter")
or ("deterministic" if action == "deterministic" else "source_capture")
),
"capability_scope": list(capabilities),
"change_set": list(request.get("change_set") or []),
"verification": list(request.get("verification") or []),
"source_signals": list(_source_signals(request.get("source_signals") or [])) if action == "source_capture" else [],
"runtime_input": _agent_runtime_input(request.get("runtime_input")) if action == "agent_runtime" else {},
"governance": governance,
"workspace": {
"authority_git_dir": "/data/aios-gitops/repo.git",
"read_only_repository": True,
"cleanup_policy": "always",
},
}
envelope = {**material, "execution_digest": _digest(material)}
return validate_execution_envelope(envelope, binding=binding)
def project_online_task(
binding: GovernedExecutionBinding,
task_request: Mapping[str, Any],
execution_request: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
request = dict(task_request)
required = _capabilities(request.get("required_capabilities"))
execution_value = dict(execution_request or {})
if str(execution_value.get("action") or "source_capture") == "agent_runtime":
adapter = str(execution_value.get("adapter") or "")
if adapter not in AGENT_RUNTIME_ADAPTERS:
_error("AGENT_RUNTIME_ADAPTER_INVALID")
runtime_input = execution_value.get("runtime_input")
if runtime_input is None:
runtime_input = {
"goal": request.get("goal"),
"instructions": request.get("instructions") or "",
"success_criteria": request.get("success_criteria") or [],
"constraints": request.get("constraints") or {},
}
execution_value["runtime_input"] = runtime_input
execution = project_execution_envelope(
binding,
execution_value,
required_capabilities=required,
)
metadata = dict(request.get("metadata") or {})
if "execution" in metadata:
validate_execution_envelope(_object(metadata["execution"], "EXECUTION_ENVELOPE_INVALID"), binding=binding)
if metadata["execution"] != execution:
_error("EXECUTION_PROJECTION_CONFLICT")
metadata["execution"] = execution
metadata["execution_projection"] = {
"schema": PROJECTION_SCHEMA,
"task_id": binding.task_id,
"job_id": binding.job_id,
"execution_digest": execution["execution_digest"],
}
return {
**request,
"task_id": binding.task_id,
"metadata": metadata,
"required_capabilities": list(required),
"idempotency_key": str(request.get("idempotency_key") or f"online-task:{binding.idempotency_key}"),
}
def resolve_task_execution(
task: Mapping[str, Any],
binding: GovernedExecutionBinding,
*,
worker_capabilities: Sequence[str] | None = None,
) -> tuple[dict[str, Any], str]:
metadata = task.get("metadata")
if isinstance(metadata, Mapping) and "execution" in metadata:
value = validate_execution_envelope(
_object(metadata["execution"], "EXECUTION_ENVELOPE_INVALID"),
binding=binding,
worker_capabilities=worker_capabilities,
)
return value, "persisted"
projected = project_execution_envelope(
binding,
{"action": "source_capture", "capability_scope": task.get("required_capabilities") or []},
required_capabilities=task.get("required_capabilities") or [],
)
validate_execution_envelope(projected, binding=binding, worker_capabilities=worker_capabilities)
return projected, "recovered_projection"
@@ -0,0 +1,187 @@
"""Durable, authority-bound deterministic execution plans for Online Agent Tasks."""
from __future__ import annotations
from contextlib import contextmanager
from dataclasses import dataclass
import fcntl
import hashlib
import json
from pathlib import Path
from typing import Any, Mapping
from aios_online_agent_execution import (
GovernedExecutionBinding,
project_execution_envelope,
)
class ExecutionPlanError(RuntimeError):
pass
def _canonical(value: object) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def _digest(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _task_key(task_id: str) -> str:
readable = "".join(ch if ch.isalnum() or ch in "._-" else "-" for ch in task_id).strip("-")[:72] or "task"
return readable + "-" + hashlib.sha256(task_id.encode("utf-8")).hexdigest()[:24]
@dataclass(frozen=True)
class ExecutionPlanRecord:
task_id: str
job_id: str
source_head_oid: str
generation: int
idempotency_key: str
plan_digest: str
execution: dict[str, Any]
def public(self) -> dict[str, Any]:
return {
"schema": "aios.online-agent.plan-record.v1",
"task_id": self.task_id,
"job_id": self.job_id,
"source_head_oid": self.source_head_oid,
"generation": self.generation,
"idempotency_key": self.idempotency_key,
"plan_digest": self.plan_digest,
"execution": dict(self.execution),
}
class ExecutionPlanStore:
def __init__(self, root: Path) -> None:
self.root = Path(root)
self.root.mkdir(parents=True, exist_ok=True)
def _path(self, task_id: str) -> Path:
return self.root / (_task_key(task_id) + ".json")
@contextmanager
def _lock(self, task_id: str):
path = self.root / (_task_key(task_id) + ".lock")
with path.open("a+b") as handle:
fcntl.flock(handle.fileno(), fcntl.LOCK_EX)
try:
yield
finally:
fcntl.flock(handle.fileno(), fcntl.LOCK_UN)
@staticmethod
def _source_head(binding: GovernedExecutionBinding) -> str:
return binding.expected_branch_oid if binding.branch_mode == "CONTINUE_EXISTING" else binding.base_oid
def _read(self, task_id: str) -> ExecutionPlanRecord | None:
path = self._path(task_id)
if not path.is_file() or path.is_symlink():
return None
value = json.loads(path.read_text(encoding="utf-8"))
if not isinstance(value, Mapping) or value.get("schema") != "aios.online-agent.plan-record.v1":
raise ExecutionPlanError("EXECUTION_PLAN_RECORD_INVALID")
execution = value.get("execution")
if not isinstance(execution, Mapping):
raise ExecutionPlanError("EXECUTION_PLAN_RECORD_INVALID")
return ExecutionPlanRecord(
task_id=str(value.get("task_id") or ""),
job_id=str(value.get("job_id") or ""),
source_head_oid=str(value.get("source_head_oid") or ""),
generation=int(value.get("generation") or 0),
idempotency_key=str(value.get("idempotency_key") or ""),
plan_digest=str(value.get("plan_digest") or ""),
execution=dict(execution),
)
def register(
self,
binding: GovernedExecutionBinding,
request: Mapping[str, Any],
) -> dict[str, Any]:
with self._lock(binding.task_id):
return self._register_locked(binding, request)
def _register_locked(
self,
binding: GovernedExecutionBinding,
request: Mapping[str, Any],
) -> dict[str, Any]:
if request.get("schema") != "aios.online-agent.plan.v1":
raise ExecutionPlanError("EXECUTION_PLAN_SCHEMA_INVALID")
if request.get("task_id") != binding.task_id or request.get("job_id") != binding.job_id:
raise ExecutionPlanError("EXECUTION_PLAN_BINDING_CONFLICT")
if int(request.get("generation") or 0) != binding.generation:
raise ExecutionPlanError("EXECUTION_PLAN_GENERATION_CONFLICT")
source_head = str(request.get("source_head_oid") or "")
if source_head != self._source_head(binding):
raise ExecutionPlanError("EXECUTION_PLAN_SOURCE_HEAD_CONFLICT")
key = str(request.get("idempotency_key") or "")
if len(key) < 8:
raise ExecutionPlanError("EXECUTION_PLAN_IDEMPOTENCY_INVALID")
execution_request = request.get("execution")
if not isinstance(execution_request, Mapping) or execution_request.get("action") != "deterministic":
raise ExecutionPlanError("EXECUTION_PLAN_DETERMINISTIC_REQUIRED")
envelope = project_execution_envelope(binding, execution_request)
material = {
"task_id": binding.task_id,
"job_id": binding.job_id,
"source_head_oid": source_head,
"generation": binding.generation,
"idempotency_key": key,
"execution": dict(execution_request),
}
plan_digest = _digest(material)
existing = self._read(binding.task_id)
expected_previous = request.get("expected_previous_plan_digest")
if existing is not None:
if existing.idempotency_key == key:
if existing.plan_digest != plan_digest:
raise ExecutionPlanError("EXECUTION_PLAN_IDEMPOTENCY_CONFLICT")
return {**existing.public(), "idempotent": True, "execution_digest": envelope["execution_digest"]}
if expected_previous != existing.plan_digest:
raise ExecutionPlanError("EXECUTION_PLAN_REPLACEMENT_CAS_MISMATCH")
if existing.source_head_oid != source_head:
raise ExecutionPlanError("EXECUTION_PLAN_STALE_REPLACEMENT_DENY")
elif expected_previous not in (None, ""):
raise ExecutionPlanError("EXECUTION_PLAN_REPLACEMENT_CAS_MISMATCH")
record = ExecutionPlanRecord(
task_id=binding.task_id,
job_id=binding.job_id,
source_head_oid=source_head,
generation=binding.generation,
idempotency_key=key,
plan_digest=plan_digest,
execution=dict(execution_request),
)
path = self._path(binding.task_id)
temporary = path.with_suffix(".tmp")
temporary.write_text(
json.dumps(record.public(), sort_keys=True, separators=(",", ":")),
encoding="utf-8",
)
temporary.replace(path)
return {**record.public(), "idempotent": False, "execution_digest": envelope["execution_digest"]}
def resolve(self, binding: GovernedExecutionBinding) -> dict[str, Any] | None:
record = self._read(binding.task_id)
if record is None:
return None
if (
record.task_id != binding.task_id
or record.job_id != binding.job_id
or record.generation != binding.generation
):
raise ExecutionPlanError("EXECUTION_PLAN_BINDING_CONFLICT")
if record.source_head_oid != self._source_head(binding):
raise ExecutionPlanError("EXECUTION_PLAN_SOURCE_HEAD_DRIFT")
envelope = project_execution_envelope(binding, record.execution)
return {
"execution": dict(record.execution),
"plan_digest": record.plan_digest,
"execution_digest": envelope["execution_digest"],
}
File diff suppressed because it is too large Load Diff
+652
View File
@@ -0,0 +1,652 @@
#!/usr/bin/env python3
"""One-shot governed executor for an already approved exact release binding.
This module deliberately does not build release artifacts. The build/staging
evidence is completed before Owner production approval. Production execution
consumes that immutable binding, revalidates it, and performs narrow
side-effects through adapters in a durable fail-closed order.
Adapter protocol:
stdin: JSON request with operation = "probe" | "apply" | "rollback"
stdout: one JSON object on the final non-empty line
shell: never used
The executor never rewrites tags, never writes refs/heads/main, and records
Baseline last.
"""
from __future__ import annotations
import argparse
import dataclasses
import fcntl
import hashlib
import json
import os
import re
import stat
import subprocess
import sys
import tempfile
import time
from pathlib import Path
from typing import Any, Mapping, Sequence
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
PLAN_RE = re.compile(r"^sha256:[0-9a-f]{64}$")
REQUIRED_V2R1C7_MEMBERS = {
"payload/VERSION",
"payload/active-runtime/app/deployment_state.py",
"payload/active-runtime/app/v2r1c6_privacy_tools.py",
"payload/active-runtime/app/v2r1c7_health_service.py",
"payload/deploy/aios-v2/systemd/aios-v2r1c7-production.service",
"payload/deploy/aios-v2/v2r1c7-runtime-config.json",
"payload/deploy/aios-v2/v2r1c7-rollback.json",
"payload/scripts/probe_v2r1c7_health.py",
"payload/scripts/rollback_v2r1c7_release.sh",
"payload/BUILD-METADATA.json",
"payload/ARTIFACT-MANIFEST.json",
"payload/ROLLBACK-METADATA.json",
}
class ReleaseBlocked(RuntimeError):
pass
class ManualOwnerReviewRequired(ReleaseBlocked):
pass
class ReconciliationRequired(ReleaseBlocked):
pass
@dataclasses.dataclass(frozen=True)
class Binding:
version: str
target_oid: str
evidence_closure_oid: str
artifact_sha256: str
approval_plan_id: str
artifact_path: str
artifact_size: int
production_service: str
production_port: int
repository_identity: str = "aios/aios"
def __post_init__(self) -> None:
if not self.version or not OID_RE.fullmatch(self.target_oid):
raise ValueError("invalid release target binding")
if not OID_RE.fullmatch(self.evidence_closure_oid):
raise ValueError("invalid evidence closure OID")
if not SHA_RE.fullmatch(self.artifact_sha256):
raise ValueError("invalid artifact SHA-256")
if not PLAN_RE.fullmatch(self.approval_plan_id):
raise ValueError("invalid approval plan id")
if self.artifact_size <= 0:
raise ValueError("invalid artifact size")
if not self.production_service:
raise ValueError("production service is required")
if not 1024 <= self.production_port <= 65535:
raise ValueError("invalid production port")
if self.repository_identity != "aios/aios":
raise ValueError("unexpected repository identity")
@property
def tag(self) -> str:
return self.version
@property
def fingerprint(self) -> str:
payload = {
"version": self.version,
"target_oid": self.target_oid,
"evidence_closure_oid": self.evidence_closure_oid,
"artifact_sha256": self.artifact_sha256,
"approval_plan_id": self.approval_plan_id,
"artifact_size": self.artifact_size,
"production_service": self.production_service,
"production_port": self.production_port,
"repository_identity": self.repository_identity,
}
return hashlib.sha256(canonical_json(payload)).hexdigest()
def request(self, operation: str) -> dict[str, Any]:
return {
"schema": "aios.production-executor-request.v1",
"operation": operation,
"binding_fingerprint": self.fingerprint,
"version": self.version,
"tag": self.tag,
"target_oid": self.target_oid,
"evidence_closure_oid": self.evidence_closure_oid,
"artifact_sha256": self.artifact_sha256,
"artifact_path": self.artifact_path,
"artifact_size": self.artifact_size,
"approval_plan_id": self.approval_plan_id,
"production_service": self.production_service,
"production_port": self.production_port,
"repository_identity": self.repository_identity,
}
def canonical_json(value: object) -> bytes:
return (json.dumps(value, sort_keys=True, separators=(",", ":")) + "\n").encode("utf-8")
def secure_json(path: Path) -> dict[str, Any]:
st = path.lstat()
if not stat.S_ISREG(st.st_mode) or path.is_symlink():
raise ReleaseBlocked(f"non-regular authorization file: {path}")
if st.st_uid != 0:
raise ReleaseBlocked(f"authorization file must be root-owned: {path}")
if st.st_mode & 0o022:
raise ReleaseBlocked(f"authorization file is group/world writable: {path}")
data = json.loads(path.read_text(encoding="utf-8"))
if not isinstance(data, dict):
raise ReleaseBlocked("authorization receipt must be a JSON object")
return data
def sha256_file(path: Path) -> str:
if path.is_symlink() or not path.is_file():
raise ReleaseBlocked("artifact must be a regular non-symlink file")
digest = hashlib.sha256()
with path.open("rb") as source:
for block in iter(lambda: source.read(1024 * 1024), b""):
digest.update(block)
return digest.hexdigest()
def run_git(git_dir: Path, *args: str, check: bool = True) -> subprocess.CompletedProcess[str]:
return subprocess.run(
["git", f"--git-dir={git_dir}", *args],
check=check,
capture_output=True,
text=True,
timeout=30,
env={"PATH": os.environ.get("PATH", "")},
)
def verify_git_lineage(git_dir: Path, binding: Binding) -> dict[str, Any]:
authority_main = run_git(git_dir, "rev-parse", "refs/heads/main").stdout.strip()
if not OID_RE.fullmatch(authority_main):
raise ReleaseBlocked("Authority main OID is invalid")
target_type = run_git(git_dir, "cat-file", "-t", binding.target_oid).stdout.strip()
if target_type != "commit":
raise ReleaseBlocked("target_oid is not a commit")
evidence_type = run_git(
git_dir, "cat-file", "-t", binding.evidence_closure_oid
).stdout.strip()
if evidence_type != "commit":
raise ReleaseBlocked("evidence_closure_oid is not a commit")
# A control-plane-only patch may advance main after release approval.
# The approved release target remains valid only while it is still on main.
target_on_main = run_git(
git_dir,
"merge-base",
"--is-ancestor",
binding.target_oid,
authority_main,
check=False,
).returncode == 0
if not target_on_main:
raise ManualOwnerReviewRequired(
"approved target is no longer an ancestor of Authority main"
)
target_in_evidence = run_git(
git_dir,
"merge-base",
"--is-ancestor",
binding.target_oid,
binding.evidence_closure_oid,
check=False,
).returncode == 0
if not target_in_evidence:
raise ReleaseBlocked("approved target is not an ancestor of evidence closure")
return {
"authority_main_oid": authority_main,
"target_oid": binding.target_oid,
"target_is_ancestor_of_main": True,
"evidence_closure_oid": binding.evidence_closure_oid,
"evidence_object_type": evidence_type,
"target_is_ancestor_of_evidence": True,
}
def verify_artifact(binding: Binding) -> dict[str, Any]:
path = Path(binding.artifact_path)
if path.stat().st_size != binding.artifact_size:
raise ReleaseBlocked("artifact size mismatch")
actual = sha256_file(path)
if actual != binding.artifact_sha256:
raise ReleaseBlocked("artifact SHA-256 mismatch")
if binding.version == "V2R1C7":
# Python tarfile may not have zstd support on the target host, so
# decompress into a private temporary tar and inspect it there.
with tempfile.TemporaryDirectory(prefix="aios-artifact-verify-") as temp_dir:
tar_path = Path(temp_dir) / "release.tar"
with tar_path.open("wb") as output:
completed = subprocess.run(
["zstd", "-dc", str(path)],
check=False,
stdout=output,
stderr=subprocess.PIPE,
timeout=60,
env={"PATH": os.environ.get("PATH", "")},
)
if completed.returncode != 0:
raise ReleaseBlocked("artifact zstd decompression failed")
import tarfile
with tarfile.open(tar_path, "r:") as archive:
names: set[str] = set()
for member in archive.getmembers():
candidate = Path(member.name)
if (
candidate.is_absolute()
or ".." in candidate.parts
or member.issym()
or member.islnk()
or not (member.isfile() or member.isdir())
):
raise ReleaseBlocked(f"unsafe artifact member: {member.name}")
names.add(member.name)
missing = sorted(REQUIRED_V2R1C7_MEMBERS - names)
if missing:
raise ReleaseBlocked(
"artifact missing required members: " + ",".join(missing)
)
return {
"artifact_path": str(path),
"artifact_sha256": actual,
"artifact_size": binding.artifact_size,
"verified": True,
}
def verify_authorization(receipt: Mapping[str, Any], binding: Binding) -> dict[str, Any]:
if receipt.get("accepted") is not True:
raise ReleaseBlocked("Owner production authorization is not accepted")
if receipt.get("binding_replaced") not in (None, False):
raise ManualOwnerReviewRequired("Owner binding was replaced")
expected = {
"version": binding.version,
"target_oid": binding.target_oid,
"evidence_closure_oid": binding.evidence_closure_oid,
"artifact_sha256": binding.artifact_sha256,
"approval_plan_id": binding.approval_plan_id,
}
for key, value in expected.items():
if receipt.get(key) != value:
raise ManualOwnerReviewRequired(f"Owner authorization mismatch: {key}")
return {"accepted": True, **expected}
class DurableState:
def __init__(self, root: Path, binding: Binding):
self.root = root
self.root.mkdir(parents=True, exist_ok=True)
os.chmod(self.root, 0o700)
self.path = self.root / "execution.json"
self.binding = binding
self.value: dict[str, Any]
if self.path.exists():
existing = json.loads(self.path.read_text(encoding="utf-8"))
if existing.get("binding_fingerprint") != binding.fingerprint:
raise ManualOwnerReviewRequired(
"state root contains a different production binding"
)
self.value = existing
else:
self.value = {
"schema": "aios.production-execution-state.v1",
"binding_fingerprint": binding.fingerprint,
"version": binding.version,
"target_oid": binding.target_oid,
"state": "AUTHORIZED",
"completed": {},
"history": [],
}
self.save()
def save(self) -> None:
temporary = self.path.with_suffix(".tmp")
temporary.write_bytes(canonical_json(self.value))
os.chmod(temporary, 0o600)
os.replace(temporary, self.path)
def record(self, state: str, step: str, result: Mapping[str, Any]) -> None:
now = time.time()
sanitized = json.loads(json.dumps(result))
self.value["state"] = state
self.value["completed"][step] = sanitized
self.value["history"].append(
{"at": now, "state": state, "step": step, "result": sanitized}
)
self.save()
def fail(self, state: str, reason: str) -> None:
self.value["state"] = state
self.value["history"].append(
{"at": time.time(), "state": state, "reason": reason}
)
self.save()
class FileLock:
def __init__(self, path: Path):
self.path = path
self.handle = None
def __enter__(self) -> "FileLock":
self.path.parent.mkdir(parents=True, exist_ok=True)
self.handle = self.path.open("a+", encoding="utf-8")
os.chmod(self.path, 0o600)
try:
fcntl.flock(
self.handle.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB
)
except BlockingIOError as exc:
raise ReleaseBlocked("production executor is already running") from exc
return self
def __exit__(self, exc_type, exc, tb) -> None:
if self.handle is not None:
fcntl.flock(self.handle.fileno(), fcntl.LOCK_UN)
self.handle.close()
def parse_command(values: Sequence[str]) -> list[str]:
command = [str(x) for x in values if str(x)]
if not command:
raise ValueError("adapter command is required")
if not os.path.isabs(command[0]):
raise ValueError("adapter executable must be an absolute path")
return command
def run_adapter(command: Sequence[str], request: Mapping[str, Any]) -> dict[str, Any]:
completed = subprocess.run(
list(command),
input=canonical_json(request),
check=False,
capture_output=True,
timeout=180,
env={"PATH": os.environ.get("PATH", "")},
)
if completed.returncode != 0:
raise ReleaseBlocked(
f"adapter failed: {Path(command[0]).name} rc={completed.returncode}"
)
lines = [line for line in completed.stdout.decode("utf-8").splitlines() if line.strip()]
if not lines:
raise ReleaseBlocked("adapter returned no JSON result")
try:
value = json.loads(lines[-1])
except ValueError as exc:
raise ReleaseBlocked("adapter returned invalid JSON") from exc
if not isinstance(value, dict):
raise ReleaseBlocked("adapter result must be an object")
return value
def require_exact_response(result: Mapping[str, Any], binding: Binding) -> None:
for key, expected in {
"version": binding.version,
"target_oid": binding.target_oid,
"evidence_closure_oid": binding.evidence_closure_oid,
"artifact_sha256": binding.artifact_sha256,
"approval_plan_id": binding.approval_plan_id,
}.items():
if key in result and result[key] != expected:
raise ManualOwnerReviewRequired(f"adapter binding mismatch: {key}")
def tag_step(command: Sequence[str], binding: Binding) -> dict[str, Any]:
request = binding.request("probe")
probe = run_adapter(command, request)
require_exact_response(probe, binding)
state = probe.get("status")
if state == "CONFLICT":
raise ManualOwnerReviewRequired("same-name tag conflicts with approved binding")
if state == "INDETERMINATE":
raise ReconciliationRequired("tag publication state is indeterminate")
if state not in {"APPLIED_VERIFIED", "NOT_APPLIED", "PARTIAL_LOCAL"}:
raise ReleaseBlocked(f"unsupported tag probe status: {state}")
if state != "APPLIED_VERIFIED":
applied = run_adapter(command, binding.request("apply"))
require_exact_response(applied, binding)
if applied.get("status") == "CONFLICT":
raise ManualOwnerReviewRequired("tag conflict during publication")
probe = run_adapter(command, request)
require_exact_response(probe, binding)
if probe.get("status") != "APPLIED_VERIFIED":
raise ReconciliationRequired("tag was not independently verified")
if probe.get("peeled_target_oid") != binding.target_oid:
raise ManualOwnerReviewRequired("published tag peeled target mismatch")
tag_oid = probe.get("tag_oid")
if not isinstance(tag_oid, str) or not OID_RE.fullmatch(tag_oid):
raise ReleaseBlocked("verified annotated tag object OID missing")
return dict(probe)
def deploy_step(command: Sequence[str], binding: Binding) -> dict[str, Any]:
probe = run_adapter(command, binding.request("probe"))
require_exact_response(probe, binding)
if probe.get("status") == "ACTIVE_VERIFIED":
return dict(probe)
if probe.get("status") == "CONFLICT":
raise ManualOwnerReviewRequired("production runtime has conflicting binding")
applied = run_adapter(command, binding.request("apply"))
require_exact_response(applied, binding)
probe = run_adapter(command, binding.request("probe"))
require_exact_response(probe, binding)
if probe.get("status") != "ACTIVE_VERIFIED":
try:
run_adapter(command, binding.request("rollback"))
except Exception:
pass
raise ReconciliationRequired("production runtime did not verify after apply")
if (
probe.get("deployment_state") != "ACTIVE"
or probe.get("production_deployed") is not True
):
raise ReconciliationRequired("production health contract is not ACTIVE")
return dict(probe)
def finalize_step(
command: Sequence[str],
binding: Binding,
*,
name: str,
tag_oid: str,
) -> dict[str, Any]:
request = binding.request("probe")
request["tag_oid"] = tag_oid
probe = run_adapter(command, request)
require_exact_response(probe, binding)
if probe.get("status") == "CONFLICT":
raise ManualOwnerReviewRequired(f"{name} has conflicting release binding")
if probe.get("status") != "VERIFIED":
apply_request = binding.request("apply")
apply_request["tag_oid"] = tag_oid
applied = run_adapter(command, apply_request)
require_exact_response(applied, binding)
probe = run_adapter(command, request)
require_exact_response(probe, binding)
if probe.get("status") != "VERIFIED":
raise ReconciliationRequired(f"{name} did not verify after apply")
if probe.get("tag_oid") not in (None, tag_oid):
raise ManualOwnerReviewRequired(f"{name} tag object mismatch")
return dict(probe)
def execute(
*,
binding: Binding,
authorization_path: Path,
repository: Path,
state_root: Path,
tag_command: Sequence[str],
deploy_command: Sequence[str],
registry_command: Sequence[str],
baseline_command: Sequence[str],
) -> dict[str, Any]:
authorization = verify_authorization(secure_json(authorization_path), binding)
lineage = verify_git_lineage(repository, binding)
artifact = verify_artifact(binding)
state = DurableState(state_root, binding)
with FileLock(state_root / "executor.lock"):
state.record(
"PREFLIGHT_VERIFIED",
"preflight",
{"authorization": authorization, "lineage": lineage, "artifact": artifact},
)
tag: dict[str, Any] | None = None
runtime: dict[str, Any] | None = None
try:
tag = tag_step(tag_command, binding)
state.record("TAG_VERIFIED", "tag", tag)
runtime = deploy_step(deploy_command, binding)
state.record("RUNTIME_VERIFIED", "runtime", runtime)
registry = finalize_step(
registry_command,
binding,
name="registry",
tag_oid=tag["tag_oid"],
)
state.record("REGISTRY_VERIFIED", "registry", registry)
baseline = finalize_step(
baseline_command,
binding,
name="baseline",
tag_oid=tag["tag_oid"],
)
state.record("BASELINE_VERIFIED", "baseline", baseline)
# Final readback: no mutation, all four planes must still agree.
final_tag = tag_step(tag_command, binding)
final_runtime = run_adapter(deploy_command, binding.request("probe"))
final_registry_request = binding.request("probe")
final_registry_request["tag_oid"] = tag["tag_oid"]
final_registry = run_adapter(registry_command, final_registry_request)
final_baseline_request = binding.request("probe")
final_baseline_request["tag_oid"] = tag["tag_oid"]
final_baseline = run_adapter(baseline_command, final_baseline_request)
for value in (final_runtime, final_registry, final_baseline):
require_exact_response(value, binding)
if final_runtime.get("status") != "ACTIVE_VERIFIED":
raise ReconciliationRequired("final runtime readback failed")
if final_registry.get("status") != "VERIFIED":
raise ReconciliationRequired("final Registry readback failed")
if final_baseline.get("status") != "VERIFIED":
raise ReconciliationRequired("final Baseline readback failed")
final = {
"status": "PRODUCTION_ACCEPTED",
"version": binding.version,
"target_oid": binding.target_oid,
"evidence_closure_oid": binding.evidence_closure_oid,
"artifact_sha256": binding.artifact_sha256,
"approval_plan_id": binding.approval_plan_id,
"tag_oid": tag["tag_oid"],
"tag": final_tag,
"runtime": final_runtime,
"registry": final_registry,
"baseline": final_baseline,
"force_push_used": False,
"tag_rewritten": False,
"direct_main_push": False,
"secrets_committed": False,
}
state.record("COMPLETED", "final_consistency", final)
return final
except ManualOwnerReviewRequired as exc:
state.fail("MANUAL_OWNER_REVIEW_REQUIRED", str(exc))
raise
except ReconciliationRequired as exc:
state.fail("RECONCILIATION_REQUIRED", str(exc))
raise
except Exception as exc:
# Once a formal tag has been verified, no later error is a clean
# pre-side-effect failure. Preserve the tag and require idempotent
# reconciliation rather than pretending the release never started.
if tag is not None:
reason = f"{type(exc).__name__}: {exc}"
state.fail("RECONCILIATION_REQUIRED", reason)
raise ReconciliationRequired(reason) from exc
state.fail("FAIL_CLOSED", f"{type(exc).__name__}: {exc}")
raise
def parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser()
p.add_argument("--authorization", required=True, type=Path)
p.add_argument("--repository", required=True, type=Path)
p.add_argument("--state-root", required=True, type=Path)
p.add_argument("--artifact", required=True, type=Path)
p.add_argument("--artifact-size", required=True, type=int)
p.add_argument("--version", required=True)
p.add_argument("--target-oid", required=True)
p.add_argument("--evidence-closure-oid", required=True)
p.add_argument("--artifact-sha256", required=True)
p.add_argument("--approval-plan-id", required=True)
p.add_argument("--production-service", required=True)
p.add_argument("--production-port", required=True, type=int)
p.add_argument("--repository-identity", default="aios/aios")
p.add_argument("--tag-adapter", required=True, nargs="+")
p.add_argument("--deploy-adapter", required=True, nargs="+")
p.add_argument("--registry-adapter", required=True, nargs="+")
p.add_argument("--baseline-adapter", required=True, nargs="+")
return p
def main() -> int:
args = parser().parse_args()
binding = Binding(
version=args.version,
target_oid=args.target_oid,
evidence_closure_oid=args.evidence_closure_oid,
artifact_sha256=args.artifact_sha256,
approval_plan_id=args.approval_plan_id,
artifact_path=str(args.artifact),
artifact_size=args.artifact_size,
production_service=args.production_service,
production_port=args.production_port,
repository_identity=args.repository_identity,
)
try:
result = execute(
binding=binding,
authorization_path=args.authorization,
repository=args.repository,
state_root=args.state_root,
tag_command=parse_command(args.tag_adapter),
deploy_command=parse_command(args.deploy_adapter),
registry_command=parse_command(args.registry_adapter),
baseline_command=parse_command(args.baseline_adapter),
)
print(json.dumps(result, sort_keys=True))
return 0
except ManualOwnerReviewRequired as exc:
print(json.dumps({"status": "MANUAL_OWNER_REVIEW_REQUIRED", "reason": str(exc)}, sort_keys=True))
return 20
except ReconciliationRequired as exc:
print(json.dumps({"status": "RECONCILIATION_REQUIRED", "reason": str(exc)}, sort_keys=True))
return 21
except Exception as exc:
print(json.dumps({"status": "FAIL_CLOSED", "error": type(exc).__name__, "reason": str(exc)}, sort_keys=True))
return 22
if __name__ == "__main__":
raise SystemExit(main())
@@ -951,3 +951,54 @@ class AuthorizedTaskWriterProvisioner:
return {**public_result, "prestart": prestart}
finally:
shutil.rmtree(temporary_root, ignore_errors=True)
# v2 continuation validation is intentionally separate from the legacy Owner
# bootstrap path above. It consumes a recovered campaign authorization and
# never creates a branch or asks the Owner to authorize again.
from aios_continuation_lease import WriterLeaseRequestV2
def validate_writer_lease_request_v2(raw: Mapping[str, Any], probe: Any) -> dict[str, Any]:
"""Validate a CONTINUE_EXISTING request against the live exact branch head."""
required = {
"schema",
"task_id",
"job_id",
"repository",
"branch_mode",
"base_oid",
"expected_branch_oid",
"target_branch",
"generation",
"lease_seconds",
}
if set(raw) != required or raw.get("schema") not in WriterLeaseRequestV2.accepted_schemas:
raise ProvisioningError("LEASE_REQUEST_V2_FIELDS_REJECTED", 409)
request = WriterLeaseRequestV2(
task_id=str(raw["task_id"]),
job_id=str(raw["job_id"]),
repository=str(raw["repository"]),
branch_mode=str(raw["branch_mode"]),
base_oid=str(raw["base_oid"]),
expected_branch_oid=str(raw["expected_branch_oid"]),
target_branch=str(raw["target_branch"]),
generation=raw["generation"],
lease_seconds=raw["lease_seconds"],
)
request.validate(probe)
return {
"schema": WriterLeaseRequestV2.schema,
"task_id": request.task_id,
"job_id": request.job_id,
"repository": request.repository,
"branch_mode": request.branch_mode,
"base_oid": request.base_oid,
"expected_branch_oid": request.expected_branch_oid,
"target_branch": request.target_branch,
"generation": request.generation,
"lease_seconds": request.lease_seconds,
"create_target_branch": request.branch_mode == "CREATE_NEW",
"worktree_mode": "detached_at_base_oid" if request.branch_mode == "CREATE_NEW" else "detached_at_expected_head",
"cas_ref_update_required": True,
}
+84
View File
@@ -0,0 +1,84 @@
#!/usr/bin/env python3
"""Single canonical Production Baseline authority for the GitOps reader.
The API and the one-shot writer import this module. Workspace and release
copies are projections only; they are never selected as a fallback source.
"""
from __future__ import annotations
import hashlib
import json
import os
from pathlib import Path
from typing import Any
AUTHORITY_PATH = Path(os.getenv(
"AIOS_GITOPS_BASELINE_AUTHORITY",
"/var/lib/aios-gitops-runtime-anchor/baseline/current.json",
))
AUTHORITY_SHA_PATH = AUTHORITY_PATH.with_suffix(".sha256")
class BaselineAuthorityError(RuntimeError):
"""Fail-closed error raised for an unavailable or inconsistent authority."""
class BaselineAuthority:
def __init__(self, path: Path | str = AUTHORITY_PATH):
self.path = Path(path)
self.sha_path = self.path.with_suffix(".sha256")
def _raw(self) -> bytes:
try:
raw = self.path.read_bytes()
except OSError as exc:
raise BaselineAuthorityError(f"authority_unavailable:{exc}") from exc
if not raw:
raise BaselineAuthorityError("authority_empty")
return raw
def read(self) -> dict[str, Any]:
raw = self._raw()
digest = hashlib.sha256(raw).hexdigest()
try:
value = json.loads(raw.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise BaselineAuthorityError("authority_invalid_json") from exc
if not isinstance(value, dict):
raise BaselineAuthorityError("authority_not_object")
expected = str(value.get("authority_sha256", ""))
if expected and expected != digest:
raise BaselineAuthorityError("authority_embedded_digest_mismatch")
if self.sha_path.exists():
sidecar = self.sha_path.read_text(encoding="utf-8").strip().split()[0]
if sidecar != digest:
raise BaselineAuthorityError("authority_sidecar_digest_mismatch")
required = {
"current_baseline", "previous_baseline", "target_oid", "tag_oid",
"evidence_closure_oid", "approval_plan_id", "status", "generation",
"source_identity",
}
missing = sorted(required.difference(value))
if missing:
raise BaselineAuthorityError("authority_missing:" + ",".join(missing))
value = dict(value)
value["authority_sha256"] = digest
value["authority_path"] = str(self.path)
return value
def read_consistent(self, policy: dict[str, Any] | None = None) -> dict[str, Any]:
value = self.read()
if policy:
expected = policy.get("release_binding", policy)
for key, field in (("version", "current_baseline"),
("previous_version", "previous_baseline"),
("target_oid", "target_oid"),
("tag_oid", "tag_oid"),
("evidence_closure_oid", "evidence_closure_oid"),
("approval_plan_id", "approval_plan_id")):
wanted = expected.get(key)
if wanted is not None and value.get(field) != wanted:
raise BaselineAuthorityError(f"authority_binding_mismatch:{field}")
return value
+108
View File
@@ -0,0 +1,108 @@
"""Atomic, predicate-bound activation of the durable release campaign."""
from __future__ import annotations
import hashlib
import json
import os
from pathlib import Path
import re
import tempfile
from typing import Any, Mapping
from release_campaign_authorization import CampaignError, validate_policy
OID_RE = re.compile(r"^[0-9a-f]{40}$")
CAMPAIGN_ID = "AIOS-AUTONOMOUS-RELEASE-CAMPAIGN-20260807"
class CampaignActivationError(CampaignError):
pass
def canonical_json(value: Any) -> bytes:
return (json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + "\n").encode("utf-8")
def digest_bytes(value: bytes) -> str:
return hashlib.sha256(value).hexdigest()
def build_active_policy(current: Mapping[str, Any], facts: Mapping[str, Any]) -> dict[str, Any]:
policy = validate_policy(dict(current))
required = {
"campaign_id",
"repository",
"source_main_oid",
"gitea_main_oid",
"authority_main_oid",
"runtime_source_oid",
"durable_job_state",
"writer_ready",
"gitea_machine_authenticated",
"governance_executor_ready",
"preapproval_publisher_ready",
"runtime_exact_main",
"main_lineage_verified",
"campaign_revoked",
}
if set(facts) != required:
raise CampaignActivationError("ACTIVATION_FACTS_SCHEMA_INVALID")
if policy["activation_state"] != "PENDING_CONTROL_PLANE_MERGE" or policy["active"] is not False:
raise CampaignActivationError("CAMPAIGN_NOT_PENDING")
if policy["campaign_id"] != CAMPAIGN_ID or facts["campaign_id"] != CAMPAIGN_ID:
raise CampaignActivationError("CAMPAIGN_ID_MISMATCH")
if facts["repository"] != "aios/aios" or policy["repository"] != facts["repository"]:
raise CampaignActivationError("REPOSITORY_MISMATCH")
for name in ("source_main_oid", "gitea_main_oid", "authority_main_oid", "runtime_source_oid"):
if not isinstance(facts[name], str) or not OID_RE.fullmatch(facts[name]):
raise CampaignActivationError("ACTIVATION_OID_INVALID")
if not (
facts["source_main_oid"]
== facts["gitea_main_oid"]
== facts["authority_main_oid"]
== facts["runtime_source_oid"]
):
raise CampaignActivationError("ACTIVATION_MAIN_DIVERGENCE")
if facts["durable_job_state"] != "COMPLETED":
raise CampaignActivationError("DURABLE_JOB_NOT_COMPLETED")
boolean_fields = (
"writer_ready",
"gitea_machine_authenticated",
"governance_executor_ready",
"preapproval_publisher_ready",
"runtime_exact_main",
"main_lineage_verified",
)
if any(facts[name] is not True for name in boolean_fields) or facts["campaign_revoked"] is not False:
raise CampaignActivationError("ACTIVATION_PREDICATE_FAILED")
activated = dict(policy)
activated["activation_state"] = "ACTIVE"
activated["active"] = True
return validate_policy(activated)
def atomic_write_policy(path: Path, sidecar: Path, receipt_path: Path, policy: Mapping[str, Any], receipt: Mapping[str, Any]) -> str:
if path.is_symlink() or sidecar.is_symlink() or receipt_path.exists() or receipt_path.is_symlink():
raise CampaignActivationError("ACTIVATION_PATH_INVALID")
path.parent.mkdir(parents=True, exist_ok=True)
raw = canonical_json(dict(policy))
digest = digest_bytes(raw)
for target, data, mode in (
(path, raw, "wb"),
(sidecar, f"{digest} {path.name}\n".encode("ascii"), "wb"),
(receipt_path, canonical_json(dict(receipt)), "wb"),
):
fd, temporary = tempfile.mkstemp(prefix=f".{target.name}.", dir=target.parent)
try:
os.fchmod(fd, 0o600)
with os.fdopen(fd, mode) as stream:
stream.write(data)
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary, target)
os.chmod(target, 0o600)
finally:
if os.path.exists(temporary):
os.unlink(temporary)
return digest
+106
View File
@@ -0,0 +1,106 @@
#!/usr/bin/env python3
"""Machine campaign approval gate for protected engineering PRs."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import subprocess
from typing import Any, Mapping
from release_campaign_authorization import CampaignAuthorization, CampaignError
class GateError(RuntimeError):
pass
def git_is_ancestor(repository: Path, ancestor: str, descendant: str) -> bool:
result = subprocess.run(
[
"git",
f"--git-dir={repository}",
"merge-base",
"--is-ancestor",
ancestor,
descendant,
],
check=False,
capture_output=True,
)
if result.returncode == 0:
return True
if result.returncode == 1:
return False
raise GateError("git_ancestor_check_failed")
def evaluate(
campaign: CampaignAuthorization,
facts: Mapping[str, Any],
repository: Path,
) -> dict[str, Any]:
required = {
"repository",
"pr_number",
"base_ref",
"base_oid",
"head_oid",
"required_checks_all_success",
"required_checks_same_head",
"unresolved_conversations",
"mergeable",
"protected_target",
}
if set(facts) != required:
raise GateError("pr_facts_fields_invalid")
if facts["repository"] != "aios/aios":
raise GateError("repository_mismatch")
if facts["base_ref"] != "refs/heads/main" or facts["protected_target"] != "main":
raise GateError("protected_main_required")
if facts["required_checks_all_success"] is not True:
raise GateError("required_checks_not_success")
if facts["required_checks_same_head"] is not True:
raise GateError("required_checks_stale")
if facts["unresolved_conversations"] != 0:
raise GateError("unresolved_conversations")
if facts["mergeable"] is not True:
raise GateError("pr_not_mergeable")
if not git_is_ancestor(repository, campaign.anchor_oid, str(facts["base_oid"])):
raise GateError("base_outside_campaign_lineage")
if not git_is_ancestor(repository, str(facts["base_oid"]), str(facts["head_oid"])):
raise GateError("head_outside_pr_lineage")
campaign.assert_operation("protected_merge")
return {
"schema": "aios.campaign-approval-gate-result.v1",
"status": "SUCCESS",
"campaign_id": campaign.campaign_id,
"pr_number": facts["pr_number"],
"head_oid": facts["head_oid"],
"owner_ui_interaction_required": False,
}
def main() -> int:
p = argparse.ArgumentParser()
p.add_argument("--campaign", type=Path, required=True)
p.add_argument("--revocation", type=Path)
p.add_argument("--pr-facts", type=Path, required=True)
p.add_argument("--repository", type=Path, required=True)
a = p.parse_args()
campaign = CampaignAuthorization.load(
a.campaign,
revocation=a.revocation,
require_active=True,
)
facts = json.loads(a.pr_facts.read_text(encoding="utf-8"))
print(json.dumps(evaluate(campaign, facts, a.repository), sort_keys=True))
return 0
if __name__ == "__main__":
try:
raise SystemExit(main())
except (CampaignError, GateError) as exc:
print(json.dumps({"status": "FAILURE", "reason": str(exc)}, sort_keys=True))
raise SystemExit(2)
+150
View File
@@ -0,0 +1,150 @@
#!/usr/bin/env python3
"""Automatically accept an exact release binding under an ACTIVE campaign."""
from __future__ import annotations
import argparse
import hashlib
import json
import os
from pathlib import Path
import stat
import tempfile
from typing import Any, Mapping
from release_campaign_authorization import CampaignAuthorization, CampaignError
class AcceptanceError(RuntimeError):
pass
def secure_json(path: Path) -> dict[str, Any]:
if path.is_symlink() or not path.is_file():
raise AcceptanceError(f"invalid_file:{path}")
st = path.lstat()
if st.st_uid != 0 or stat.S_IMODE(st.st_mode) & 0o022:
raise AcceptanceError(f"unsafe_file:{path}")
value = json.loads(path.read_text(encoding="utf-8"))
if not isinstance(value, dict):
raise AcceptanceError(f"json_object_required:{path}")
return value
def atomic_new(path: Path, value: Mapping[str, Any]) -> None:
if path.exists() or path.is_symlink():
raise AcceptanceError("acceptance_receipt_exists")
path.parent.mkdir(parents=True, exist_ok=True)
fd, name = tempfile.mkstemp(prefix=f".{path.name}.", dir=path.parent)
try:
os.fchmod(fd, 0o600)
if os.geteuid() == 0:
os.fchown(fd, 0, 0)
with os.fdopen(fd, "w", encoding="utf-8", closefd=True) as stream:
json.dump(dict(value), stream, sort_keys=True, indent=2)
stream.write("\n")
stream.flush()
os.fsync(stream.fileno())
os.replace(name, path)
os.chmod(path, 0o600)
finally:
try:
os.unlink(name)
except FileNotFoundError:
pass
def evaluate(
campaign: CampaignAuthorization,
binding: Mapping[str, Any],
evidence: Mapping[str, Any],
artifact: Mapping[str, Any],
staging: Mapping[str, Any],
main: Mapping[str, Any],
tag: Mapping[str, Any],
) -> dict[str, Any]:
required_binding = {
"version",
"target_oid",
"evidence_closure_oid",
"artifact_sha256",
"artifact_size",
"approval_plan_id",
}
if set(binding) != required_binding:
raise AcceptanceError("binding_fields_invalid")
if main.get("repository") != "aios/aios":
raise AcceptanceError("repository_identity_mismatch")
if main.get("target_in_main_lineage") is not True:
raise AcceptanceError("target_outside_main_lineage")
if evidence.get("object_type") != "commit":
raise AcceptanceError("evidence_not_commit")
if evidence.get("target_is_ancestor") is not True:
raise AcceptanceError("target_not_evidence_ancestor")
if evidence.get("closure_oid") != binding["evidence_closure_oid"]:
raise AcceptanceError("evidence_oid_mismatch")
if artifact.get("sha256") != binding["artifact_sha256"]:
raise AcceptanceError("artifact_digest_mismatch")
if artifact.get("size") != binding["artifact_size"]:
raise AcceptanceError("artifact_size_mismatch")
if artifact.get("retained") is not True:
raise AcceptanceError("artifact_not_retained")
if staging.get("install") != "PASS":
raise AcceptanceError("staging_install_failed")
if staging.get("health") != "PASS":
raise AcceptanceError("staging_health_failed")
if staging.get("restart") != "PASS":
raise AcceptanceError("staging_restart_failed")
if staging.get("rollback_readiness") != "PASS":
raise AcceptanceError("rollback_readiness_failed")
if tag.get("conflict") is not False:
raise AcceptanceError("tag_conflict")
campaign.assert_operation("derive_campaign_acceptance")
return {
"schema": "aios.campaign-release-acceptance.v1",
"accepted": True,
"binding_replaced": False,
"campaign_id": campaign.campaign_id,
"campaign_generation": campaign.data["generation"],
**dict(binding),
"owner_interaction_required": False,
}
def main() -> int:
p = argparse.ArgumentParser()
p.add_argument("--campaign", type=Path, required=True)
p.add_argument("--revocation", type=Path)
p.add_argument("--binding", type=Path, required=True)
p.add_argument("--evidence", type=Path, required=True)
p.add_argument("--artifact", type=Path, required=True)
p.add_argument("--staging", type=Path, required=True)
p.add_argument("--main-facts", type=Path, required=True)
p.add_argument("--tag-facts", type=Path, required=True)
p.add_argument("--output", type=Path, required=True)
a = p.parse_args()
campaign = CampaignAuthorization.load(
a.campaign,
revocation=a.revocation,
require_active=True,
)
receipt = evaluate(
campaign,
secure_json(a.binding),
secure_json(a.evidence),
secure_json(a.artifact),
secure_json(a.staging),
secure_json(a.main_facts),
secure_json(a.tag_facts),
)
atomic_new(a.output, receipt)
print(json.dumps(receipt, sort_keys=True))
return 0
if __name__ == "__main__":
try:
raise SystemExit(main())
except (CampaignError, AcceptanceError) as exc:
print(json.dumps({"status": "BLOCKED", "reason": str(exc)}, sort_keys=True))
raise SystemExit(2)
+67
View File
@@ -0,0 +1,67 @@
"""Runtime deployment-state provider for governed health contracts."""
from __future__ import annotations
import datetime as dt
import json
import os
import re
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Mapping
_OID = re.compile(r"^[0-9a-f]{40}$")
_SHA = re.compile(r"^[0-9a-f]{64}$")
_STATES = {"STAGED", "ACTIVE", "ROLLED_BACK"}
@dataclass(frozen=True)
class RuntimeDeploymentState:
version: str
commit_oid: str
artifact_sha256: str
deployment_state: str
@property
def production_deployed(self) -> bool:
return self.deployment_state == "ACTIVE"
def health_payload(self) -> dict[str, Any]:
return {
"status": "ok",
"version": self.version,
"commit_oid": self.commit_oid,
"artifact_sha256": self.artifact_sha256,
"deployment_state": self.deployment_state,
"production_deployed": self.production_deployed,
"health_timestamp": dt.datetime.now(dt.timezone.utc).isoformat(),
}
@classmethod
def from_environment(cls, environ: Mapping[str, str] | None = None) -> "RuntimeDeploymentState":
env = dict(environ or os.environ)
values: dict[str, Any] = {}
state_file = env.get("DEPLOYMENT_STATE_FILE", "").strip()
if state_file:
path = Path(state_file)
if path.is_file() and not path.is_symlink():
loaded = json.loads(path.read_text(encoding="utf-8"))
if not isinstance(loaded, dict):
raise ValueError("deployment_state_file_object_required")
values.update(loaded)
file_deployment_state = values.get("deployment_state")
values.update({k: v for k, v in {
"version": env.get("RELEASE_VERSION", values.get("version", "")),
"commit_oid": env.get("TARGET_OID", values.get("commit_oid", "")),
"artifact_sha256": env.get("ARTIFACT_SHA256", values.get("artifact_sha256", "")),
"deployment_state": env.get("DEPLOYMENT_STATE", values.get("deployment_state", "")),
}.items() if v != ""})
if file_deployment_state not in (None, ""):
values["deployment_state"] = file_deployment_state
version = str(values.get("version", ""))
commit_oid = str(values.get("commit_oid", ""))
artifact_sha256 = str(values.get("artifact_sha256", ""))
deployment_state = str(values.get("deployment_state", ""))
if not version or not _OID.fullmatch(commit_oid) or not _SHA.fullmatch(artifact_sha256):
raise ValueError("deployment_binding_invalid")
if deployment_state not in _STATES:
raise ValueError("deployment_state_invalid")
return cls(version, commit_oid, artifact_sha256, deployment_state)
@@ -0,0 +1 @@
"""Permanent AIOS governance source components."""
@@ -0,0 +1,75 @@
#!/usr/bin/env python3
import json
import pathlib
import socket
import time
SOCKET = '/run/aios-governance-executor/executor.sock'
STATE = pathlib.Path('/var/lib/aios-gitops/governance-executor/drift-monitor.json')
APPROVED = pathlib.Path('/var/lib/aios-gitops/runtime/tasks/AIOS-PROTECTION-SNAPSHOT-DRIFT-FORENSICS-V1/protection-drift/semantic-analysis.json')
def call():
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.settimeout(20)
sock.connect(SOCKET)
request = {'operation': 'GET_BRANCH_PROTECTION', 'payload': {'repository': 'aios/aios', 'branch': 'main'}}
sock.sendall((json.dumps(request, separators=(',', ':')) + '\n').encode())
chunks = []
while True:
part = sock.recv(1024 * 1024)
if not part:
break
chunks.append(part)
if b'\n' in part:
break
sock.close()
return json.loads(b''.join(chunks).splitlines()[0])
def main():
try:
result = call()
if result.get('status') != 'PASS':
raise RuntimeError(str(result.get('code') or 'EXECUTOR_GET_FAILED'))
approved = json.loads(APPROVED.read_text(encoding='utf-8'))
approved_transport = approved['approved_transport_digest_from_evidence']
approved_semantic = approved['approved_semantic_digest']
transport = result['transport_digest']
semantic = result['semantic_digest']
if semantic != approved_semantic:
state = 'GOVERNANCE_POLICY_DRIFT'
elif transport != approved_transport:
state = 'GOVERNANCE_TRANSPORT_METADATA_CHANGED'
else:
state = 'NO_DRIFT'
payload = {
'schema': 'aios.governance-drift-monitor.v1',
'observed_at_unix': int(time.time()),
'status': state,
'release_blocking': False,
'approved_transport_digest': approved_transport,
'observed_transport_digest': transport,
'approved_semantic_digest': approved_semantic,
'observed_semantic_digest': semantic,
'semantic_schema': result.get('semantic_schema'),
'source': 'Dedicated Governance Executor GET_BRANCH_PROTECTION',
}
except Exception as exc:
payload = {
'schema': 'aios.governance-drift-monitor.v1',
'observed_at_unix': int(time.time()),
'status': 'MONITOR_UNAVAILABLE',
'release_blocking': False,
'error_code': str(exc),
}
STATE.parent.mkdir(parents=True, exist_ok=True)
temporary = STATE.with_suffix('.tmp')
temporary.write_text(json.dumps(payload, sort_keys=True, separators=(',', ':'), ensure_ascii=False) + '\n', encoding='utf-8')
temporary.replace(STATE)
STATE.chmod(0o640)
print(json.dumps(payload, sort_keys=True, separators=(',', ':'), ensure_ascii=False))
if __name__ == '__main__':
main()
+417
View File
@@ -0,0 +1,417 @@
#!/usr/bin/env python3
import base64
import copy
import hashlib
import json
import os
import pathlib
import socket
import time
import urllib.error
import urllib.parse
import urllib.request
SOURCE_GOVERNANCE = "/opt/aios/aios-control-plane/current/active-runtime/app/governance"
if SOURCE_GOVERNANCE not in __import__("sys").path:
__import__("sys").path.insert(0, SOURCE_GOVERNANCE)
from stage2_gate_migration import (
CAMPAIGN_GATE,
CAMPAIGN_ID,
OWNER_GATE,
STAGE1_SEMANTIC_SHA,
Stage2GateError,
build_target,
semantic_digest as stage2_semantic_digest,
validate_stage2_facts,
)
API = "http://127.0.0.1:3000"
REPO = "aios/aios"
SOCKET = "/run/aios-governance-executor/executor.sock"
JOURNAL = pathlib.Path("/var/lib/aios-gitops/governance-executor/executor-journal.jsonl")
RECONCILIATION_BINDING_PATH = pathlib.Path("/var/lib/aios-gitops/governance-executor/reconciliation-binding.json")
PR54_HEAD = "015185d7e5191d18c5c1596160807ac228782a0a"
PR54_REVIEW = 104
PR54_MANIFEST_SHA = "1b11ee42d53a9f6c1646e5f5ddd8ce5b7ecf301483b9f3d47a068290d541ba4b"
PR54_PREVIOUS_PROTECTION_SHA = "8b3370add21cd23c2b4f41d8d84d3eaaa09ddf0451549b72c7941bbd92d5d60a"
SEMANTIC_SCHEMA = "aios.semantic-protection.v1"
APPROVED_SEMANTIC_SHA = "ba15d512ed8c1f43e5f72c84125a3a71a7347ef59e18000bbfca24d6e69f1b19"
STAGE1_SEMANTIC_SHA = "d911e856f71de6105b8efa08e13338a448e0b6d37536332decd80c9f409ecf27"
USER_IDS = {"owner": 2, "aios-autonomy-agent": 153}
def canon(value):
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()
def sha(value):
return hashlib.sha256(canon(value)).hexdigest()
def token():
value = (pathlib.Path(os.environ["CREDENTIALS_DIRECTORY"]) / "gitea-governance-executor-token").read_text().strip()
if len(value) < 16:
raise RuntimeError("EXECUTOR_CREDENTIAL_INVALID")
return value
def api(method, path, value, body=None, expected=(200,)):
raw = None if body is None else json.dumps(body, separators=(",", ":")).encode()
req = urllib.request.Request(API + path, data=raw, method=method)
req.add_header("Authorization", "token " + value)
req.add_header("Accept", "application/json")
if raw is not None:
req.add_header("Content-Type", "application/json")
try:
with urllib.request.urlopen(req, timeout=20) as response:
status = response.status
payload = response.read()
except urllib.error.HTTPError as exc:
if exc.code in expected:
payload = exc.read()
return exc.code, json.loads(payload) if payload else None
raise RuntimeError("GITEA_HTTP_" + str(exc.code))
if status not in expected:
raise RuntimeError("GITEA_HTTP_" + str(status))
if not payload or not payload.strip():
raise RuntimeError("GITEA_EMPTY_JSON:" + path)
try:
decoded = json.loads(payload)
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise RuntimeError("GITEA_INVALID_JSON:" + path) from exc
return status, decoded
def journal(event, **fields):
JOURNAL.parent.mkdir(parents=True, exist_ok=True)
record = {"schema": "aios.governance-executor-journal.v2", "event": event, "unix": int(time.time()), **fields}
with JOURNAL.open("a", encoding="utf-8") as stream:
stream.write(json.dumps(record, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + "\n")
stream.flush()
os.fsync(stream.fileno())
def typed_json(value):
if value in (None, "null"):
return []
if isinstance(value, str):
return json.loads(value)
return value
def user_ids(names):
result = []
for name in names or []:
if name not in USER_IDS:
raise RuntimeError("UNKNOWN_GOVERNANCE_USER:" + str(name))
result.append(USER_IDS[name])
return sorted(result)
def semantic_policy(value):
policy = {
"schema_version": SEMANTIC_SCHEMA,
"repo_id": 1,
"protection_id": 2,
"branch_name": value.get("branch_name"),
"can_push": value.get("enable_push"),
"can_force_push": value.get("enable_force_push"),
"enable_merge_whitelist": value.get("enable_merge_whitelist"),
"merge_whitelist_user_ids": user_ids(value.get("merge_whitelist_usernames")),
"merge_whitelist_team_ids": [],
"required_approvals": value.get("required_approvals"),
"enable_status_check": value.get("enable_status_check"),
"status_check_contexts": sorted(value.get("status_check_contexts") or []),
"block_admin_merge_override": value.get("block_admin_merge_override"),
"block_on_official_review_requests": value.get("block_on_official_review_requests"),
"block_on_outdated_branch": value.get("block_on_outdated_branch"),
"block_on_rejected_reviews": value.get("block_on_rejected_reviews"),
"dismiss_stale_approvals": value.get("dismiss_stale_approvals"),
"ignore_stale_approvals": value.get("ignore_stale_approvals"),
"enable_approvals_whitelist": value.get("enable_approvals_whitelist"),
"approvals_whitelist_user_ids": user_ids(value.get("approvals_whitelist_username")),
"approvals_whitelist_team_ids": [],
"enable_bypass_allowlist": value.get("enable_bypass_allowlist"),
"bypass_allowlist_user_ids": user_ids(value.get("bypass_allowlist_usernames")),
"bypass_allowlist_team_ids": [],
"enable_force_push_allowlist": value.get("enable_force_push_allowlist"),
"force_push_allowlist_user_ids": user_ids(value.get("force_push_allowlist_usernames")),
"force_push_allowlist_team_ids": [],
"enable_whitelist": value.get("enable_push_whitelist"),
"whitelist_user_ids": user_ids(value.get("push_whitelist_usernames")),
"whitelist_team_ids": [],
"force_push_allowlist_deploy_keys": value.get("force_push_allowlist_deploy_keys"),
"whitelist_deploy_keys": value.get("push_whitelist_deploy_keys"),
"require_signed_commits": value.get("require_signed_commits"),
"priority": value.get("priority"),
"protected_file_patterns": value.get("protected_file_patterns"),
"unprotected_file_patterns": value.get("unprotected_file_patterns"),
}
return policy
def semantic_sha(value):
return sha(semantic_policy(value))
def verify_binding(binding, value):
required = {
"governance_request_PR", "exact_request_head", "Owner_review_id", "Owner_review_exact_head",
"manifest_sha256", "expected_previous_snapshot_sha256", "approved_delta",
"semantic_reconciliation_request_id", "semantic_reconciliation_head",
"semantic_reconciliation_review_id", "semantic_reconciliation_review_head",
"approved_semantic_digest",
}
if set(binding) != required:
raise RuntimeError("GOVERNANCE_BINDING_SCHEMA_INVALID")
if binding["governance_request_PR"] != 54 or binding["exact_request_head"] != PR54_HEAD:
raise RuntimeError("GOVERNANCE_BINDING_PR54_MISMATCH")
if binding["Owner_review_id"] != PR54_REVIEW or binding["Owner_review_exact_head"] != PR54_HEAD:
raise RuntimeError("GOVERNANCE_BINDING_REVIEW_MISMATCH")
if binding["manifest_sha256"] != PR54_MANIFEST_SHA:
raise RuntimeError("GOVERNANCE_BINDING_MANIFEST_MISMATCH")
if binding["expected_previous_snapshot_sha256"] != PR54_PREVIOUS_PROTECTION_SHA:
raise RuntimeError("GOVERNANCE_BINDING_SNAPSHOT_MISMATCH")
if binding["approved_delta"] != {"add_merge_whitelist_user_id": 153}:
raise RuntimeError("GOVERNANCE_BINDING_DELTA_DENIED")
if binding["approved_semantic_digest"] != APPROVED_SEMANTIC_SHA:
raise RuntimeError("GOVERNANCE_BINDING_SEMANTIC_DIGEST_MISMATCH")
if not RECONCILIATION_BINDING_PATH.is_file():
raise RuntimeError("SEMANTIC_RECONCILIATION_BINDING_REQUIRED")
try:
reconciliation = json.loads(RECONCILIATION_BINDING_PATH.read_text(encoding="utf-8"))
except (OSError, ValueError):
raise RuntimeError("SEMANTIC_RECONCILIATION_BINDING_INVALID") from None
required_reconciliation = {
"schema", "request_id", "pr_number", "request_head", "owner_review_id",
"owner_review_head", "approved_semantic_digest", "approved_delta",
}
if set(reconciliation) != required_reconciliation or reconciliation.get("schema") != "aios.semantic-reconciliation-binding.v1":
raise RuntimeError("SEMANTIC_RECONCILIATION_BINDING_INVALID")
if reconciliation["approved_semantic_digest"] != APPROVED_SEMANTIC_SHA or reconciliation["approved_delta"] != binding["approved_delta"]:
raise RuntimeError("SEMANTIC_RECONCILIATION_BINDING_MISMATCH")
if (
reconciliation["request_id"] != binding["semantic_reconciliation_request_id"]
or reconciliation["request_head"] != binding["semantic_reconciliation_head"]
or reconciliation["owner_review_id"] != binding["semantic_reconciliation_review_id"]
or reconciliation["owner_review_head"] != binding["semantic_reconciliation_review_head"]
):
raise RuntimeError("SEMANTIC_RECONCILIATION_BINDING_MISMATCH")
_, reconciliation_pr = api("GET", f"/api/v1/repos/{REPO}/pulls/{reconciliation['pr_number']}", value)
if (
reconciliation_pr.get("state") != "open"
or reconciliation_pr.get("merged")
or reconciliation_pr.get("head", {}).get("sha") != reconciliation["request_head"]
or reconciliation_pr.get("base", {}).get("ref") != "governance/approvals"
):
raise RuntimeError("SEMANTIC_RECONCILIATION_REQUEST_STALE")
_, reconciliation_reviews = api("GET", f"/api/v1/repos/{REPO}/pulls/{reconciliation['pr_number']}/reviews", value)
reconciliation_review = next((x for x in reconciliation_reviews if x.get("id") == reconciliation["owner_review_id"]), None)
if (
not reconciliation_review
or reconciliation_review.get("user", {}).get("id") != 2
or reconciliation_review.get("state") != "APPROVED"
or reconciliation_review.get("commit_id") != reconciliation["owner_review_head"]
or reconciliation_review.get("dismissed") is not False
):
raise RuntimeError("SEMANTIC_RECONCILIATION_OWNER_REVIEW_STALE")
_, pr = api("GET", f"/api/v1/repos/{REPO}/pulls/54", value)
if pr.get("state") != "open" or pr.get("merged") or pr.get("head", {}).get("sha") != PR54_HEAD:
raise RuntimeError("PR54_APPROVAL_STALE")
_, reviews = api("GET", f"/api/v1/repos/{REPO}/pulls/54/reviews", value)
review = next((x for x in reviews if x.get("id") == PR54_REVIEW), None)
if not review or review.get("user", {}).get("id") != 2 or review.get("state") != "APPROVED" or review.get("commit_id") != PR54_HEAD or review.get("dismissed") is not False:
raise RuntimeError("PR54_APPROVAL_STALE")
manifest_path = "approvals/governance-policy/AIOS-AUTONOMOUS-MERGE-GOVERNANCE-SCOPE-EXTENSION-V1.json"
_, content = api("GET", f"/api/v1/repos/{REPO}/contents/{urllib.parse.quote(manifest_path, safe='/')}?ref={PR54_HEAD}", value)
raw = base64.b64decode("".join(content.get("content", "").split()))
if hashlib.sha256(raw).hexdigest() != PR54_MANIFEST_SHA:
raise RuntimeError("PR54_MANIFEST_STALE")
def get_protection(value, branch):
if branch not in {"main", "governance/approvals"}:
raise RuntimeError("BRANCH_DENIED")
_, protection = api("GET", f"/api/v1/repos/{REPO}/branch_protections/{urllib.parse.quote(branch, safe='')}", value)
return protection
def mutation_receipt(before, after, binding, target_semantic):
return {
"before": {"transport_digest": sha(before), "semantic_digest": semantic_sha(before)},
"approved": {
"mutation_predicate_digest": sha({"operation": "ADD_MERGE_EXECUTOR", "subject_user_id": 153, "preserve_all_unmodified_security_fields": True}),
"Owner_review_id": binding["Owner_review_id"],
"semantic_policy_digest": APPROVED_SEMANTIC_SHA,
},
"after": {"transport_digest": sha(after), "semantic_digest": semantic_sha(after)},
"delta": {"add_merge_whitelist_user_id": 153},
"target_semantic_digest": sha(target_semantic),
"unmodified_security_fields": {"verified": True},
}
def apply_stage1(binding, value):
verify_binding(binding, value)
before = get_protection(value, "main")
before_semantic = semantic_policy(before)
before_semantic_sha = sha(before_semantic)
names = list(before.get("merge_whitelist_usernames") or [])
name_ids = user_ids(names)
if name_ids not in ([2], [2, 153]):
raise RuntimeError("PROTECTION_WHITELIST_PRESTATE_INVALID")
target_names = ["owner", "aios-autonomy-agent"]
target_semantic = copy.deepcopy(before_semantic)
target_semantic["merge_whitelist_user_ids"] = [2, 153]
if name_ids == [2, 153]:
if before_semantic_sha != STAGE1_SEMANTIC_SHA:
journal("APPLY_BLOCKED_STAGE1_IDEMPOTENCY_SEMANTIC_CONFLICT", transport_digest=sha(before), semantic_digest=before_semantic_sha, expected_semantic_digest=STAGE1_SEMANTIC_SHA)
raise RuntimeError("PROTECTION_STAGE1_SEMANTIC_CONFLICT")
receipt = mutation_receipt(before, before, binding, target_semantic)
journal("APPLY_APPROVED_PROTECTION_DELTA_IDEMPOTENT_SUCCESS", receipt=receipt)
return {"status": "IDEMPOTENT_SUCCESS", "protection": before, "transport_digest": sha(before), "semantic_digest": before_semantic_sha, "target_semantic_digest": sha(target_semantic)}
if before_semantic_sha != APPROVED_SEMANTIC_SHA:
journal("APPLY_BLOCKED_SEMANTIC_CAS", transport_digest=sha(before), semantic_digest=before_semantic_sha, expected_semantic_digest=APPROVED_SEMANTIC_SHA)
raise RuntimeError("PROTECTION_SEMANTIC_CAS_CONFLICT")
after_api = api("PATCH", f"/api/v1/repos/{REPO}/branch_protections/main", value, {"merge_whitelist_usernames": target_names})[1]
after = get_protection(value, "main")
for key in before:
if key in {"updated_at", "merge_whitelist_usernames"}:
continue
if after.get(key) != before.get(key):
raise RuntimeError("PROTECTION_UNAPPROVED_FIELD_CHANGED:" + key)
if user_ids(after.get("merge_whitelist_usernames")) != [2, 153]:
raise RuntimeError("PROTECTION_TARGET_READBACK_FAILED")
after_semantic = semantic_policy(after)
if after_semantic != target_semantic:
raise RuntimeError("PROTECTION_SEMANTIC_TARGET_READBACK_FAILED")
receipt = mutation_receipt(before, after, binding, target_semantic)
journal("APPLY_APPROVED_PROTECTION_DELTA_COMPLETED", receipt=receipt)
return {"status": "APPLIED", "protection": after, "previous_transport_digest": sha(before), "previous_semantic_digest": before_semantic_sha, "transport_digest": sha(after), "semantic_digest": sha(after_semantic), "receipt": receipt}
def apply_stage2(binding, facts, value):
verify_binding(binding, value)
validate_stage2_facts(facts)
before = get_protection(value, "main")
if OWNER_GATE in (before.get("status_check_contexts") or []) and stage2_semantic_digest(before) != STAGE1_SEMANTIC_SHA:
raise RuntimeError("STAGE2_SEMANTIC_PRECONDITION_INVALID")
try:
status, target = build_target(before)
except Stage2GateError as exc:
journal("MIGRATE_APPROVED_GATE_BLOCKED", reason=str(exc), semantic_digest=stage2_semantic_digest(before))
raise RuntimeError(str(exc)) from exc
target_digest = stage2_semantic_digest(target)
if status == "IDEMPOTENT_SUCCESS":
journal(
"MIGRATE_APPROVED_GATE_IDEMPOTENT_SUCCESS",
before_semantic_digest=stage2_semantic_digest(before),
target_semantic_digest=target_digest,
campaign_id=CAMPAIGN_ID,
)
return {
"status": status,
"protection": before,
"transport_digest": sha(before),
"semantic_digest": stage2_semantic_digest(before),
"target_semantic_digest": target_digest,
"gate_transition": {"removed": OWNER_GATE, "required": CAMPAIGN_GATE},
}
after_api = api(
"PATCH",
f"/api/v1/repos/{REPO}/branch_protections/main",
value,
{"status_check_contexts": target["status_check_contexts"]},
)[1]
after = get_protection(value, "main")
for key in before:
if key in {"updated_at", "status_check_contexts"}:
continue
if after.get(key) != before.get(key):
raise RuntimeError("PROTECTION_UNAPPROVED_FIELD_CHANGED:" + key)
if after.get("status_check_contexts") != target["status_check_contexts"]:
raise RuntimeError("STAGE2_GATE_TARGET_READBACK_FAILED")
if stage2_semantic_digest(after) != target_digest:
raise RuntimeError("STAGE2_SEMANTIC_TARGET_READBACK_FAILED")
receipt = {
"before_transport_digest": sha(before),
"before_semantic_digest": stage2_semantic_digest(before),
"after_transport_digest": sha(after),
"after_semantic_digest": stage2_semantic_digest(after),
"target_semantic_digest": target_digest,
"campaign_id": CAMPAIGN_ID,
"gate_transition": {"removed": OWNER_GATE, "required": CAMPAIGN_GATE},
"preserved_security_fields": True,
"facts_digest": sha(facts),
}
journal("MIGRATE_APPROVED_GATE_COMPLETED", receipt=receipt)
return {
"status": "APPLIED",
"protection": after,
"previous_transport_digest": sha(before),
"previous_semantic_digest": stage2_semantic_digest(before),
"transport_digest": sha(after),
"semantic_digest": stage2_semantic_digest(after),
"target_semantic_digest": target_digest,
"receipt": receipt,
}
def handle(request, value):
if not isinstance(request, dict) or set(request) != {"operation", "payload"}:
raise RuntimeError("REQUEST_SCHEMA_INVALID")
operation = request["operation"]
payload = request["payload"]
if not isinstance(payload, dict) or payload.get("repository") != REPO:
raise RuntimeError("REPOSITORY_DENIED")
if operation == "GET_BRANCH_PROTECTION":
protection = get_protection(value, payload.get("branch"))
journal("GET_BRANCH_PROTECTION", branch=payload.get("branch"), transport_digest=sha(protection), semantic_digest=semantic_sha(protection))
return {"status": "PASS", "protection": protection, "snapshot_sha256": sha(protection), "transport_digest": sha(protection), "semantic_digest": semantic_sha(protection), "semantic_schema": SEMANTIC_SCHEMA}
if operation == "READBACK_BRANCH_PROTECTION":
protection = get_protection(value, payload.get("branch"))
return {"status": "PASS", "protection": protection, "snapshot_sha256": sha(protection), "transport_digest": sha(protection), "semantic_digest": semantic_sha(protection), "semantic_schema": SEMANTIC_SCHEMA}
if operation == "APPLY_APPROVED_PROTECTION_DELTA":
binding = payload.get("GovernanceExecutionBinding")
if not isinstance(binding, dict):
raise RuntimeError("GOVERNANCE_BINDING_REQUIRED")
return apply_stage1(binding, value)
if operation == "MIGRATE_APPROVED_GATE":
binding = payload.get("GovernanceExecutionBinding")
facts = payload.get("Stage2Facts")
if not isinstance(binding, dict) or not isinstance(facts, dict):
raise RuntimeError("STAGE2_BINDING_OR_FACTS_REQUIRED")
return apply_stage2(binding, facts, value)
raise RuntimeError("OPERATION_DENIED")
def main():
value = token()
sock_path = pathlib.Path(SOCKET)
sock_path.unlink(missing_ok=True)
server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
server.bind(SOCKET)
os.chmod(SOCKET, 0o660)
server.listen(16)
journal("EXECUTOR_STARTED", api_surface=["GET_BRANCH_PROTECTION", "APPLY_APPROVED_PROTECTION_DELTA", "MIGRATE_APPROVED_GATE", "READBACK_BRANCH_PROTECTION"], semantic_schema=SEMANTIC_SCHEMA, approved_semantic_digest=APPROVED_SEMANTIC_SHA, stage1_semantic_digest=STAGE1_SEMANTIC_SHA, campaign_gate=CAMPAIGN_GATE)
try:
while True:
conn, _ = server.accept()
with conn:
data = conn.recv(1024 * 1024 + 1)
if len(data) > 1024 * 1024:
result = {"status": "FAIL", "code": "REQUEST_TOO_LARGE"}
else:
try:
request = json.loads(data.decode())
result = handle(request, value)
except Exception as exc:
result = {"status": "FAIL", "code": str(exc)}
conn.sendall((json.dumps(result, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + "\n").encode())
finally:
server.close()
sock_path.unlink(missing_ok=True)
if __name__ == "__main__":
main()
@@ -0,0 +1,347 @@
#!/usr/bin/env python3
import base64
import hashlib
import json
import os
import pathlib
import shutil
import socket
import subprocess
import tempfile
import time
import urllib.error
import urllib.parse
import urllib.request
LOCAL_API = 'http://127.0.0.1:3000/api/v1'
NETWORK_API = 'https://g.cnnetmkt.top/git/api/v1'
NETWORK_REMOTE = 'https://g.cnnetmkt.top/git/aios/aios.git'
REPO = 'aios/aios'
BASE_BRANCH = 'governance/approvals'
REQUEST_BRANCH = 'governance/request/protection-semantic-reconciliation-v1-20260808'
SOCKET_PATH = '/run/aios-governance-preapproval-publisher/publisher.sock'
CREDENTIAL_NAME = 'gitea-machine-token'
AUTHORITY_REPO = '/data/aios-gitops/repo.git'
TASK_ROOT = pathlib.Path('/var/lib/aios-gitops/runtime/tasks/AIOS-PROTECTION-SNAPSHOT-DRIFT-FORENSICS-V1')
INPUT_ROOT = TASK_ROOT / 'publication-input'
RECEIPT = TASK_ROOT / 'publication-receipt.json'
MANIFEST_REL = 'approvals/governance-policy/AIOS-PROTECTION-SEMANTIC-SNAPSHOT-RECONCILIATION-V1.json'
EVIDENCE_REL = 'approvals/governance-policy/AIOS-PROTECTION-SEMANTIC-SNAPSHOT-RECONCILIATION-V1.evidence.json'
def digest_bytes(data):
return hashlib.sha256(data).hexdigest()
def digest_json(value):
return digest_bytes(json.dumps(value, sort_keys=True, separators=(',', ':'), ensure_ascii=False).encode())
def load_token():
path = pathlib.Path(os.environ['CREDENTIALS_DIRECTORY']) / CREDENTIAL_NAME
value = path.read_text(encoding='utf-8').strip()
if len(value) < 16:
raise RuntimeError('MACHINE_CREDENTIAL_INVALID')
return value
def api(base, method, path, token, body=None, expected=(200,)):
raw = None if body is None else json.dumps(body, separators=(',', ':'), ensure_ascii=False).encode()
request = urllib.request.Request(base + path, data=raw, method=method)
request.add_header('Authorization', 'token ' + token)
request.add_header('Accept', 'application/json')
if raw is not None:
request.add_header('Content-Type', 'application/json')
try:
with urllib.request.urlopen(request, timeout=20) as response:
payload = response.read()
if response.status not in expected:
raise RuntimeError('GITEA_HTTP_' + str(response.status))
return response.status, json.loads(payload) if payload else None
except urllib.error.HTTPError as exc:
payload = exc.read()
if exc.code in expected:
return exc.code, json.loads(payload) if payload else None
raise RuntimeError('GITEA_HTTP_' + str(exc.code) + ':' + method + ':' + path) from None
def authority_head():
result = subprocess.run(
['git', '--git-dir=' + AUTHORITY_REPO, 'rev-parse', 'refs/heads/' + BASE_BRANCH],
text=True, capture_output=True, check=False,
)
if result.returncode != 0:
raise RuntimeError('AUTHORITY_BASE_UNAVAILABLE')
return result.stdout.strip()
def branch_api(base, token, branch, expected=(200,)):
return api(base, 'GET', '/repos/' + REPO + '/branches/' + urllib.parse.quote(branch, safe=''), token, expected=expected)
def verify_base(token):
authority = authority_head()
_, local = branch_api(LOCAL_API, token, BASE_BRANCH)
_, network = branch_api(NETWORK_API, token, BASE_BRANCH)
local_oid = local.get('commit', {}).get('id')
network_oid = network.get('commit', {}).get('id')
if not authority or authority != local_oid or authority != network_oid:
raise RuntimeError('RECONCILIATION_REQUIRED')
return authority
def verify_identity(token):
_, user = api(LOCAL_API, 'GET', '/user', token)
if user.get('login') != 'aios-autonomy-agent' or user.get('id') != 153 or user.get('is_admin') is not False:
raise RuntimeError('MACHINE_IDENTITY_REJECTED')
return {'login': user['login'], 'id': user['id'], 'owner': user.get('is_admin')}
def load_inputs():
manifest = json.dumps(json.loads((INPUT_ROOT / 'manifest.json').read_text(encoding='utf-8')), sort_keys=True, separators=(',', ':'), ensure_ascii=False).encode()
evidence = json.dumps(json.loads((INPUT_ROOT / 'evidence.json').read_text(encoding='utf-8')), sort_keys=True, separators=(',', ':'), ensure_ascii=False).encode()
for data in (manifest, evidence):
if any(secret in data.lower() for secret in (b'authorization:', b'ghp_', b'github_pat_', b'secret=', b'token=')):
raise RuntimeError('SECRET_CONTENT_REJECTED')
return manifest, evidence
def publication_key(base_oid, manifest_sha, evidence_sha):
payload = REPO + base_oid + REQUEST_BRANCH + manifest_sha + evidence_sha
return hashlib.sha256(payload.encode()).hexdigest()
def write_receipt(state, **fields):
record = {
'schema': 'aios.governance-preapproval-publication.v1',
'operation': 'SEMANTIC_RECONCILIATION_REQUEST',
'state': state,
'recorded_unix': int(time.time()),
'request_branch': REQUEST_BRANCH,
'manifest_path': MANIFEST_REL,
'evidence_path': EVIDENCE_REL,
'machine_actor': 'aios-autonomy-agent',
'owner': False,
'credential_ref': {'scheme': 'systemd-credential', 'name': CREDENTIAL_NAME},
'governance_capability_executed': False,
'protection_mutation': False,
'force_push': False,
**fields,
}
RECEIPT.parent.mkdir(parents=True, exist_ok=True)
temporary = RECEIPT.with_suffix('.tmp')
temporary.write_text(json.dumps(record, sort_keys=True, separators=(',', ':'), ensure_ascii=False) + '\n', encoding='utf-8')
temporary.replace(RECEIPT)
RECEIPT.chmod(0o640)
return record
def existing_prs(token):
_, prs = api(LOCAL_API, 'GET', '/repos/' + REPO + '/pulls?state=all&base=' + urllib.parse.quote(BASE_BRANCH, safe='') + '&limit=100', token)
return [pr for pr in prs if pr.get('head', {}).get('ref') == REQUEST_BRANCH and pr.get('base', {}).get('ref') == BASE_BRANCH]
def verify_existing_branch(token, branch_oid, manifest, evidence):
_, commit = api(LOCAL_API, 'GET', '/repos/' + REPO + '/git/commits/' + branch_oid, token)
if commit.get('parents', [{}])[0].get('sha') is None:
raise RuntimeError('REQUEST_BRANCH_COMMIT_INVALID')
for relative, expected in ((MANIFEST_REL, manifest), (EVIDENCE_REL, evidence)):
_, content = api(LOCAL_API, 'GET', '/repos/' + REPO + '/contents/' + urllib.parse.quote(relative, safe='/') + '?ref=' + branch_oid, token)
actual = base64.b64decode(''.join(content.get('content', '').split()))
if actual != expected:
raise RuntimeError('GOVERNANCE_REQUEST_BRANCH_CONFLICT')
def push_one_commit(token, base_oid, manifest, evidence):
work_root = TASK_ROOT / 'publisher-work'
work_root.mkdir(parents=True, exist_ok=True)
temporary = pathlib.Path(tempfile.mkdtemp(prefix='semantic-reconciliation-', dir=work_root))
askpass = temporary / 'askpass.sh'
clone = temporary / 'repo'
credential_path = pathlib.Path(os.environ['CREDENTIALS_DIRECTORY']) / CREDENTIAL_NAME
askpass.write_text(
'#!/bin/sh\ncase "$1" in *Username*) printf "%s\\n" "$GITEA_MACHINE_ACTOR";; *) cat "$CREDENTIAL_FILE";; esac\n',
encoding='utf-8',
)
askpass.chmod(0o700)
env = os.environ.copy()
env.update({
'GIT_ASKPASS': str(askpass),
'GIT_TERMINAL_PROMPT': '0',
'GITEA_MACHINE_ACTOR': 'aios-autonomy-agent',
'CREDENTIAL_FILE': str(credential_path),
'GIT_CONFIG_COUNT': '1',
'GIT_CONFIG_KEY_0': 'safe.directory',
'GIT_CONFIG_VALUE_0': AUTHORITY_REPO,
})
try:
run = subprocess.run(
['git', 'clone', '--branch', BASE_BRANCH, '--single-branch', '--quiet', NETWORK_REMOTE, str(clone)],
text=True, capture_output=True, env=env, check=False,
)
if run.returncode != 0:
detail = ' '.join(run.stderr.strip().split())[:240]
raise RuntimeError('PUBLISHER_CLONE_FAILED:' + detail)
def git(*args, push_env=None):
result = subprocess.run(['git', *args], cwd=clone, text=True, capture_output=True, env=push_env or env, check=False)
if result.returncode != 0:
raise RuntimeError('PUBLISHER_GIT_OPERATION_FAILED')
return result.stdout.strip()
git('checkout', '--detach', base_oid)
git('switch', '--create', REQUEST_BRANCH)
manifest_path = clone / MANIFEST_REL
evidence_path = clone / EVIDENCE_REL
manifest_path.parent.mkdir(parents=True, exist_ok=True)
evidence_path.parent.mkdir(parents=True, exist_ok=True)
manifest_path.write_bytes(manifest)
evidence_path.write_bytes(evidence)
git('add', '--', MANIFEST_REL, EVIDENCE_REL)
staged = git('diff', '--cached', '--name-only').splitlines()
if staged != sorted([MANIFEST_REL, EVIDENCE_REL]):
raise RuntimeError('PUBLISHER_PATH_ALLOWLIST_FAILED')
git('-c', 'user.name=aios-autonomy-agent', '-c', 'user.email=aios-autonomy-agent@aios.invalid', 'commit', '--quiet', '-m', '[governance] publish semantic reconciliation evidence')
if git('rev-list', '--count', base_oid + '..HEAD') != '1':
raise RuntimeError('PUBLISHER_COMMIT_COUNT_FAILED')
git('push', '--porcelain', NETWORK_REMOTE, 'HEAD:refs/heads/' + REQUEST_BRANCH, push_env=env)
head = git('rev-parse', 'HEAD')
return head
finally:
shutil.rmtree(temporary, ignore_errors=True)
def publish():
token = load_token()
identity = verify_identity(token)
base_oid = verify_base(token)
manifest, evidence = load_inputs()
manifest_sha = digest_bytes(manifest)
evidence_sha = digest_bytes(evidence)
key = publication_key(base_oid, manifest_sha, evidence_sha)
write_receipt('REQUESTED', governance_base_oid=base_oid, manifest_sha256=manifest_sha, evidence_sha256=evidence_sha, idempotency_key=key, machine_identity=identity)
write_receipt('LOCAL_CANDIDATE_VERIFIED', governance_base_oid=base_oid, manifest_sha256=manifest_sha, evidence_sha256=evidence_sha, idempotency_key=key, machine_identity=identity)
branch_status, branch = branch_api(LOCAL_API, token, REQUEST_BRANCH, expected=(200, 404))
if branch_status == 200:
branch_oid = branch.get('commit', {}).get('id')
verify_existing_branch(token, branch_oid, manifest, evidence)
else:
branch_oid = push_one_commit(token, base_oid, manifest, evidence)
verify_existing_branch(token, branch_oid, manifest, evidence)
write_receipt('BRANCH_PUSHED', governance_base_oid=base_oid, manifest_sha256=manifest_sha, evidence_sha256=evidence_sha, idempotency_key=key, request_head_oid=branch_oid, machine_identity=identity)
prs = existing_prs(token)
if len(prs) > 1:
raise RuntimeError('MULTIPLE_MATCHING_APPROVAL_PRS')
if prs:
pr = prs[0]
if pr.get('head', {}).get('sha') != branch_oid:
raise RuntimeError('APPROVAL_PR_HEAD_MISMATCH')
pr_number = pr.get('number')
else:
_, pr = api(LOCAL_API, 'POST', '/repos/' + REPO + '/pulls', token, {
'title': '[GOVERNANCE RECONCILIATION] Semantic protection snapshot schema migration',
'head': REQUEST_BRANCH,
'base': BASE_BRANCH,
'body': 'Schema-only reconciliation for the existing PR54 Stage-1 and Stage-2 predicates. No new governance capability; Owner exact-head re-attestation required.',
}, expected=(201,))
pr_number = pr.get('number')
write_receipt('PR_CREATED', governance_base_oid=base_oid, manifest_sha256=manifest_sha, evidence_sha256=evidence_sha, idempotency_key=key, request_head_oid=branch_oid, pr_number=pr_number, machine_identity=identity)
_, remote_branch = branch_api(NETWORK_API, token, REQUEST_BRANCH)
if remote_branch.get('commit', {}).get('id') != branch_oid:
raise RuntimeError('NETWORK_BRANCH_READBACK_MISMATCH')
_, read_pr = api(LOCAL_API, 'GET', '/repos/' + REPO + '/pulls/' + str(pr_number), token)
if read_pr.get('head', {}).get('sha') != branch_oid or read_pr.get('base', {}).get('ref') != BASE_BRANCH:
raise RuntimeError('PR_READBACK_MISMATCH')
result = write_receipt('COMPLETED', governance_base_oid=base_oid, manifest_sha256=manifest_sha, evidence_sha256=evidence_sha, idempotency_key=key, request_head_oid=branch_oid, pr_number=pr_number, pr_head_oid=read_pr.get('head', {}).get('sha'), machine_identity=identity)
return result
def self_test():
token = load_token()
identity = verify_identity(token)
base_oid = verify_base(token)
branch_status, _ = branch_api(LOCAL_API, token, REQUEST_BRANCH, expected=(200, 404))
return {
'status': 'PASS',
'governance_base_oid': base_oid,
'machine_identity': identity,
'credential_ref': {'scheme': 'systemd-credential', 'name': CREDENTIAL_NAME},
'branch_namespace': 'refs/heads/governance/request/*',
'target_base': BASE_BRANCH,
'request_branch_status': 'EXISTS' if branch_status == 200 else 'ABSENT',
'operation_allowlist': ['READ_BASE', 'CREATE_ONE_REQUEST_BRANCH', 'WRITE_TWO_EVIDENCE_PATHS_ONE_COMMIT', 'CREATE_OR_READ_ONE_APPROVAL_PR', 'READBACK'],
'denied': ['MAIN_WRITE', 'DIRECT_APPROVALS_WRITE', 'BRANCH_PROTECTION', 'MERGE', 'FORCE_PUSH', 'TAG', 'PRODUCTION', 'CREDENTIAL_MUTATION'],
'governance_capability_executed': False,
}
def read_publication_status():
token = load_token()
receipt = json.loads(RECEIPT.read_text(encoding='utf-8'))
pr_number = receipt.get('pr_number')
if not pr_number:
raise RuntimeError('PUBLICATION_RECEIPT_PR_MISSING')
_, pr = api(LOCAL_API, 'GET', '/repos/' + REPO + '/pulls/' + str(pr_number), token)
_, reviews = api(LOCAL_API, 'GET', '/repos/' + REPO + '/pulls/' + str(pr_number) + '/reviews', token)
_, branch = branch_api(LOCAL_API, token, REQUEST_BRANCH)
_, network_branch = branch_api(NETWORK_API, token, REQUEST_BRANCH)
return {
'status': 'PASS',
'request_branch': REQUEST_BRANCH,
'branch_head': branch.get('commit', {}).get('id'),
'network_branch_head': network_branch.get('commit', {}).get('id'),
'pr_number': pr.get('number'),
'pr_state': pr.get('state'),
'pr_merged': pr.get('merged'),
'pr_head': pr.get('head', {}).get('sha'),
'pr_base': pr.get('base', {}).get('ref'),
'reviews': [
{
'id': item.get('id'),
'reviewer_id': item.get('user', {}).get('id'),
'state': item.get('state'),
'commit_id': item.get('commit_id'),
'dismissed': item.get('dismissed'),
}
for item in reviews
],
'governance_capability_executed': False,
'protection_mutation': False,
}
def handle(request):
if not isinstance(request, dict) or set(request) != {'operation'}:
raise RuntimeError('REQUEST_SCHEMA_INVALID')
if request['operation'] == 'DRY_RUN_SELF_TEST':
return self_test()
if request['operation'] == 'PUBLISH_SEMANTIC_RECONCILIATION':
return publish()
if request['operation'] == 'READ_SEMANTIC_RECONCILIATION_STATUS':
return read_publication_status()
raise RuntimeError('OPERATION_DENIED')
def serve(listener):
conn, _ = listener.accept()
with conn:
try:
data = conn.recv(1024 * 1024 + 1)
if len(data) > 1024 * 1024:
raise RuntimeError('REQUEST_TOO_LARGE')
result = handle(json.loads(data.decode()))
except Exception as exc:
result = {'status': 'FAIL', 'code': str(exc)}
conn.sendall((json.dumps(result, sort_keys=True, separators=(',', ':'), ensure_ascii=False) + '\n').encode())
def main():
if os.environ.get('LISTEN_FDS') != '1':
raise SystemExit('SOCKET_ACTIVATION_REQUIRED')
listener = socket.fromfd(3, socket.AF_UNIX, socket.SOCK_STREAM)
serve(listener)
if __name__ == '__main__':
main()
@@ -0,0 +1,73 @@
"""Fail-closed Authority/Gitea ref reconciliation after protected merge."""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
import subprocess
class RefReconciliationError(RuntimeError):
pass
def _oid(value: str, field: str) -> str:
if not isinstance(value, str) or len(value) != 40 or any(char not in "0123456789abcdef" for char in value):
raise RefReconciliationError(field + "_INVALID")
return value
@dataclass(frozen=True, slots=True)
class RefReconciler:
git_dir: Path
def read(self, ref: str) -> str:
result = subprocess.run(
["git", f"--git-dir={self.git_dir}", "rev-parse", "--verify", ref],
text=True,
capture_output=True,
check=False,
)
if result.returncode != 0:
raise RefReconciliationError("AUTHORITY_REF_UNAVAILABLE")
return result.stdout.strip()
def is_ancestor(self, ancestor: str, descendant: str) -> bool:
_oid(ancestor, "ancestor")
_oid(descendant, "descendant")
result = subprocess.run(
["git", f"--git-dir={self.git_dir}", "merge-base", "--is-ancestor", ancestor, descendant],
text=True,
capture_output=True,
check=False,
)
if result.returncode == 0:
return True
if result.returncode == 1:
return False
raise RefReconciliationError("AUTHORITY_ANCESTRY_CHECK_FAILED")
def reconcile_main(self, *, expected_oid: str, merged_oid: str, gitea_main_oid: str) -> str:
_oid(expected_oid, "expected_oid")
_oid(merged_oid, "merged_oid")
_oid(gitea_main_oid, "gitea_main_oid")
if gitea_main_oid != merged_oid:
raise RefReconciliationError("GITEA_MAIN_EXPECTATION_MISMATCH")
if not self.is_ancestor(expected_oid, merged_oid):
raise RefReconciliationError("MAIN_FF_REQUIRED")
current = self.read("refs/heads/main")
if current == merged_oid:
return merged_oid
if current != expected_oid:
raise RefReconciliationError("AUTHORITY_MAIN_EXPECTATION_MISMATCH")
result = subprocess.run(
["git", f"--git-dir={self.git_dir}", "update-ref", "refs/heads/main", merged_oid, expected_oid],
text=True,
capture_output=True,
check=False,
)
if result.returncode != 0:
raise RefReconciliationError("AUTHORITY_MAIN_CAS_FAILED")
if self.read("refs/heads/main") != merged_oid:
raise RefReconciliationError("AUTHORITY_MAIN_CAS_READBACK_MISMATCH")
return merged_oid
@@ -0,0 +1,81 @@
"""Canonical semantic branch-protection model used by AIOS governance gates."""
from __future__ import annotations
import hashlib
import json
from typing import Any, Mapping
SCHEMA = "aios.semantic-protection.v1"
USER_IDS = {"owner": 2, "aios-autonomy-agent": 153}
def canonical_json(value: Any) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def _typed_json(value: Any) -> list[Any]:
if value in (None, "null"):
return []
if isinstance(value, str):
return json.loads(value)
return list(value)
def _user_ids(names: Any) -> list[int]:
result = []
for name in names or []:
if name not in USER_IDS:
raise ValueError("UNKNOWN_GOVERNANCE_USER:" + str(name))
result.append(USER_IDS[name])
return sorted(result)
def semantic_policy(protection: Mapping[str, Any]) -> dict[str, Any]:
"""Drop transport metadata and canonicalize semantic security fields."""
return {
"schema_version": SCHEMA,
"repo_id": 1,
"protection_id": 2,
"branch_name": protection.get("branch_name"),
"can_push": protection.get("enable_push"),
"can_force_push": protection.get("enable_force_push"),
"enable_merge_whitelist": protection.get("enable_merge_whitelist"),
"merge_whitelist_user_ids": _user_ids(protection.get("merge_whitelist_usernames")),
"merge_whitelist_team_ids": [],
"required_approvals": protection.get("required_approvals"),
"enable_status_check": protection.get("enable_status_check"),
"status_check_contexts": sorted(protection.get("status_check_contexts") or []),
"block_admin_merge_override": protection.get("block_admin_merge_override"),
"block_on_official_review_requests": protection.get("block_on_official_review_requests"),
"block_on_outdated_branch": protection.get("block_on_outdated_branch"),
"block_on_rejected_reviews": protection.get("block_on_rejected_reviews"),
"dismiss_stale_approvals": protection.get("dismiss_stale_approvals"),
"ignore_stale_approvals": protection.get("ignore_stale_approvals"),
"enable_approvals_whitelist": protection.get("enable_approvals_whitelist"),
"approvals_whitelist_user_ids": _user_ids(protection.get("approvals_whitelist_username")),
"approvals_whitelist_team_ids": [],
"enable_bypass_allowlist": protection.get("enable_bypass_allowlist"),
"bypass_allowlist_user_ids": _user_ids(protection.get("bypass_allowlist_usernames")),
"bypass_allowlist_team_ids": [],
"enable_force_push_allowlist": protection.get("enable_force_push_allowlist"),
"force_push_allowlist_user_ids": _user_ids(protection.get("force_push_allowlist_usernames")),
"enable_whitelist": protection.get("enable_push_whitelist"),
"whitelist_user_ids": _user_ids(protection.get("push_whitelist_usernames")),
"whitelist_team_ids": [],
"force_push_allowlist_deploy_keys": protection.get("force_push_allowlist_deploy_keys"),
"whitelist_deploy_keys": protection.get("push_whitelist_deploy_keys"),
"require_signed_commits": protection.get("require_signed_commits"),
"priority": protection.get("priority"),
"protected_file_patterns": _typed_json(protection.get("protected_file_patterns")),
"unprotected_file_patterns": _typed_json(protection.get("unprotected_file_patterns")),
}
def semantic_digest(protection: Mapping[str, Any]) -> str:
return hashlib.sha256(canonical_json(semantic_policy(protection))).hexdigest()
def transport_digest(protection: Mapping[str, Any]) -> str:
return hashlib.sha256(canonical_json(protection)).hexdigest()
@@ -0,0 +1,141 @@
"""Fail-closed Stage-2 campaign gate migration predicates."""
from __future__ import annotations
import copy
import hashlib
import json
import re
from typing import Any, Mapping
SEMANTIC_SCHEMA = "aios.semantic-protection.v1"
STAGE1_SEMANTIC_SHA = "d911e856f71de6105b8efa08e13338a448e0b6d37536332decd80c9f409ecf27"
CAMPAIGN_ID = "AIOS-AUTONOMOUS-RELEASE-CAMPAIGN-20260807"
OWNER_GATE = "owner-approval-gate"
CAMPAIGN_GATE = "campaign-approval-gate"
OID_RE = re.compile(r"^[0-9a-f]{40}$")
class Stage2GateError(RuntimeError):
pass
def canonical_json(value: Any) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def _typed_json(value: Any) -> list[Any]:
if value in (None, "", "null"):
return []
if isinstance(value, str):
return json.loads(value)
return list(value)
def _user_ids(names: Any) -> list[int]:
mapping = {"owner": 2, "aios-autonomy-agent": 153}
result = []
for name in names or []:
if name not in mapping:
raise Stage2GateError("UNKNOWN_GOVERNANCE_USER")
result.append(mapping[name])
return sorted(result)
def semantic_policy(protection: Mapping[str, Any]) -> dict[str, Any]:
return {
"schema_version": SEMANTIC_SCHEMA,
"repo_id": 1,
"protection_id": 2,
"branch_name": protection.get("branch_name"),
"can_push": protection.get("enable_push"),
"can_force_push": protection.get("enable_force_push"),
"enable_merge_whitelist": protection.get("enable_merge_whitelist"),
"merge_whitelist_user_ids": _user_ids(protection.get("merge_whitelist_usernames")),
"merge_whitelist_team_ids": [],
"required_approvals": protection.get("required_approvals"),
"enable_status_check": protection.get("enable_status_check"),
"status_check_contexts": sorted(protection.get("status_check_contexts") or []),
"block_admin_merge_override": protection.get("block_admin_merge_override"),
"block_on_official_review_requests": protection.get("block_on_official_review_requests"),
"block_on_outdated_branch": protection.get("block_on_outdated_branch"),
"block_on_rejected_reviews": protection.get("block_on_rejected_reviews"),
"dismiss_stale_approvals": protection.get("dismiss_stale_approvals"),
"ignore_stale_approvals": protection.get("ignore_stale_approvals"),
"enable_approvals_whitelist": protection.get("enable_approvals_whitelist"),
"approvals_whitelist_user_ids": _user_ids(protection.get("approvals_whitelist_username")),
"approvals_whitelist_team_ids": [],
"enable_bypass_allowlist": protection.get("enable_bypass_allowlist"),
"bypass_allowlist_user_ids": _user_ids(protection.get("bypass_allowlist_usernames")),
"bypass_allowlist_team_ids": [],
"enable_force_push_allowlist": protection.get("enable_force_push_allowlist"),
"force_push_allowlist_user_ids": _user_ids(protection.get("force_push_allowlist_usernames")),
"force_push_allowlist_team_ids": [],
"enable_whitelist": protection.get("enable_push_whitelist"),
"whitelist_user_ids": _user_ids(protection.get("push_whitelist_usernames")),
"whitelist_team_ids": [],
"force_push_allowlist_deploy_keys": protection.get("force_push_allowlist_deploy_keys"),
"whitelist_deploy_keys": protection.get("push_whitelist_deploy_keys"),
"require_signed_commits": protection.get("require_signed_commits"),
"priority": protection.get("priority"),
"protected_file_patterns": protection.get("protected_file_patterns"),
"unprotected_file_patterns": protection.get("unprotected_file_patterns"),
}
def semantic_digest(protection: Mapping[str, Any]) -> str:
return hashlib.sha256(canonical_json(semantic_policy(protection))).hexdigest()
def build_target(protection: Mapping[str, Any]) -> tuple[str, dict[str, Any]]:
current = list(protection.get("status_check_contexts") or [])
has_owner = OWNER_GATE in current
has_campaign = CAMPAIGN_GATE in current
if has_owner and has_campaign:
raise Stage2GateError("GATE_PRESTATE_CONFLICT")
if not has_owner and not has_campaign:
raise Stage2GateError("OWNER_GATE_MISSING")
if has_campaign:
return "IDEMPOTENT_SUCCESS", copy.deepcopy(dict(protection))
if semantic_digest(protection) != STAGE1_SEMANTIC_SHA:
raise Stage2GateError("SEMANTIC_PRECONDITION_INVALID")
target = copy.deepcopy(dict(protection))
target["status_check_contexts"] = sorted(
[gate for gate in current if gate != OWNER_GATE] + [CAMPAIGN_GATE]
)
return "READY", target
def validate_stage2_facts(facts: Mapping[str, Any]) -> None:
required = {
"campaign_id",
"campaign_active",
"campaign_revoked",
"machine_actor",
"required_checks_all_success",
"required_checks_same_head",
"unresolved_conversations",
"mergeable",
"protected_target",
"current_main_oid",
"gitea_main_oid",
"current_main_lineage_verified",
}
if set(facts) != required:
raise Stage2GateError("STAGE2_FACTS_SCHEMA_INVALID")
if facts["campaign_id"] != CAMPAIGN_ID or facts["campaign_active"] is not True or facts["campaign_revoked"] is not False:
raise Stage2GateError("CAMPAIGN_NOT_ACTIVE")
if facts["machine_actor"] != "aios-autonomy-agent":
raise Stage2GateError("MACHINE_ACTOR_MISMATCH")
if facts["required_checks_all_success"] is not True or facts["required_checks_same_head"] is not True:
raise Stage2GateError("REQUIRED_CHECKS_NOT_CURRENT")
if facts["unresolved_conversations"] != 0 or facts["mergeable"] is not True:
raise Stage2GateError("CAMPAIGN_GATE_FACTS_INVALID")
if facts["protected_target"] != "main" or facts["current_main_lineage_verified"] is not True:
raise Stage2GateError("PROTECTED_TARGET_OR_LINEAGE_INVALID")
if (
not isinstance(facts["current_main_oid"], str)
or not OID_RE.fullmatch(facts["current_main_oid"])
or facts["gitea_main_oid"] != facts["current_main_oid"]
):
raise Stage2GateError("MAIN_READBACK_MISMATCH")
@@ -0,0 +1,568 @@
"""Governed V2R2C1 production release orchestration.
This module is deliberately scoped to the Owner-approved V2R2C1 binding. It
uses the existing TagPublisher and metadata adapters; it does not introduce a
second Git writer, a second durable job store, or an alternate baseline.
"""
from __future__ import annotations
import argparse
import fcntl
import hashlib
import json
import os
import re
import subprocess
import tarfile
import tempfile
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Mapping
from gitea_tag_publication import (
GiteaTagClient,
GitTagBackend,
PublicationManualOwnerReviewRequired,
PublicationPending,
PublicationReconciliationRequired,
PublicationRequest,
TagPublisher,
)
VERSION = "V2R2C1"
TARGET_OID = "998d30304787e31e5ae607c0bd86b5edb2b0e5ad"
EVIDENCE_CLOSURE_OID = "a8b906d1bc24088c19647e98de6a679268f6d0a2"
APPROVAL_PLAN_ID = "sha256:c0df52008ae7f64cec5ee2d28947aa7b892066c670ab8295e4549140b1a7500e"
REPOSITORY = "aios/aios"
SERVICE = "aios-v2r1c7-production.service"
PORT = 18927
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
class ReleaseBlocked(RuntimeError):
"""A release gate failed and no unsafe recovery is permitted."""
class ManualOwnerReviewRequired(ReleaseBlocked):
pass
class ReconciliationRequired(ReleaseBlocked):
pass
def canonical(value: object) -> bytes:
return (json.dumps(value, sort_keys=True, separators=(",", ":")) + "\n").encode()
def sha256_file(path: Path) -> str:
if path.is_symlink() or not path.is_file():
raise ReleaseBlocked("artifact must be a regular file")
digest = hashlib.sha256()
with path.open("rb") as stream:
for block in iter(lambda: stream.read(1024 * 1024), b""):
digest.update(block)
return digest.hexdigest()
def atomic_json(path: Path, value: Mapping[str, Any], mode: int = 0o600) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_name(path.name + ".tmp")
temporary.write_bytes(canonical(dict(value)))
os.chmod(temporary, mode)
os.replace(temporary, path)
@dataclass(frozen=True)
class ReleaseBinding:
version: str = VERSION
target_oid: str = TARGET_OID
evidence_closure_oid: str = EVIDENCE_CLOSURE_OID
approval_plan_id: str = APPROVAL_PLAN_ID
repository: str = REPOSITORY
production_service: str = SERVICE
production_port: int = PORT
def validate(self) -> None:
if self.version != VERSION or self.repository != REPOSITORY:
raise ManualOwnerReviewRequired("release binding is outside V2R2C1 scope")
if not OID_RE.fullmatch(self.target_oid) or self.target_oid != TARGET_OID:
raise ManualOwnerReviewRequired("V2R2C1 target_oid mismatch")
if not OID_RE.fullmatch(self.evidence_closure_oid) or self.evidence_closure_oid != EVIDENCE_CLOSURE_OID:
raise ManualOwnerReviewRequired("V2R2C1 evidence closure mismatch")
if self.approval_plan_id != APPROVAL_PLAN_ID:
raise ManualOwnerReviewRequired("V2R2C1 approval plan mismatch")
if self.production_service != SERVICE or int(self.production_port) != PORT:
raise ManualOwnerReviewRequired("production service binding mismatch")
def as_dict(self) -> dict[str, Any]:
return {
"version": self.version,
"target_oid": self.target_oid,
"evidence_closure_oid": self.evidence_closure_oid,
"approval_plan_id": self.approval_plan_id,
"repository": self.repository,
"production_service": self.production_service,
"production_port": self.production_port,
}
class StateLock:
def __init__(self, path: Path):
self.path = path
self.handle = None
def __enter__(self) -> "StateLock":
self.path.parent.mkdir(parents=True, exist_ok=True)
self.handle = self.path.open("a+", encoding="utf-8")
fcntl.flock(self.handle.fileno(), fcntl.LOCK_EX)
return self
def __exit__(self, exc_type, exc, tb) -> None:
if self.handle is not None:
fcntl.flock(self.handle.fileno(), fcntl.LOCK_UN)
self.handle.close()
class ReleaseState:
def __init__(self, root: Path, binding: ReleaseBinding):
self.root = root
self.path = root / "release-state.json"
self.binding = binding
root.mkdir(parents=True, exist_ok=True)
if self.path.exists():
value = json.loads(self.path.read_text(encoding="utf-8"))
if value.get("binding") != binding.as_dict():
raise ManualOwnerReviewRequired("persisted release binding conflict")
self.value = value
else:
self.value = {
"schema": "aios.v2r2c1.governed-production-release-state.v1",
"binding": binding.as_dict(),
"state": "INITIAL",
"completed": {},
"history": [],
}
self.save()
def save(self) -> None:
atomic_json(self.path, self.value)
def record(self, state: str, step: str, result: Mapping[str, Any]) -> None:
safe = json.loads(json.dumps(dict(result)))
self.value["state"] = state
self.value.setdefault("completed", {})[step] = safe
self.value.setdefault("history", []).append(
{"at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), "state": state, "step": step, "result": safe}
)
self.save()
def fail(self, state: str, reason: str) -> None:
self.value["state"] = state
self.value.setdefault("history", []).append(
{"at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), "state": state, "reason": str(reason)[:500]}
)
self.save()
def _git(git_dir: Path, *args: str, check: bool = True) -> str:
completed = subprocess.run(
["git", f"--git-dir={git_dir}", *args],
check=False,
capture_output=True,
text=True,
timeout=45,
env={"PATH": os.environ.get("PATH", "")},
)
if check and completed.returncode != 0:
raise ReconciliationRequired(completed.stderr.strip() or "authoritative git query failed")
return completed.stdout.strip()
def _ancestor(git_dir: Path, ancestor: str, descendant: str) -> bool:
return subprocess.run(
["git", f"--git-dir={git_dir}", "merge-base", "--is-ancestor", ancestor, descendant],
check=False,
capture_output=True,
timeout=45,
env={"PATH": os.environ.get("PATH", "")},
).returncode == 0
def verify_lineage(git_dir: Path, binding: ReleaseBinding) -> dict[str, Any]:
binding.validate()
if _git(git_dir, "cat-file", "-t", binding.target_oid) != "commit":
raise ReleaseBlocked("approved target_oid is not a commit")
if _git(git_dir, "cat-file", "-t", binding.evidence_closure_oid) != "commit":
raise ReleaseBlocked("evidence closure is not a commit")
if not _ancestor(git_dir, binding.target_oid, binding.evidence_closure_oid):
raise ManualOwnerReviewRequired("approved target is not an ancestor of evidence closure")
return {
"target_oid": binding.target_oid,
"evidence_closure_oid": binding.evidence_closure_oid,
"target_is_ancestor_of_evidence": True,
"main_oid": _git(git_dir, "rev-parse", "refs/heads/main"),
}
def verify_owner_approval(value: Mapping[str, Any], binding: ReleaseBinding) -> dict[str, Any]:
approval = value.get("owner_approval")
if not isinstance(approval, Mapping):
raise ReleaseBlocked("OWNER_APPROVAL_RECEIPT_REQUIRED")
if approval.get("approved") is not True or approval.get("status") != "APPROVED":
raise ReleaseBlocked("OWNER_APPROVAL_NOT_ACCEPTED")
if approval.get("approved_by") != "aiman":
raise ManualOwnerReviewRequired("OWNER_APPROVAL_ACTOR_MISMATCH")
if approval.get("approval_source") != "explicit_owner_approval_in_chat":
raise ManualOwnerReviewRequired("OWNER_APPROVAL_SOURCE_MISMATCH")
for key, expected in binding.as_dict().items():
if key in {"repository", "production_service", "production_port"}:
continue
if approval.get(key) != expected:
raise ManualOwnerReviewRequired("OWNER_APPROVAL_BINDING_MISMATCH:" + key)
return dict(approval)
def _payload_file(git_dir: Path, oid: str, path: str) -> bytes:
completed = subprocess.run(
["git", f"--git-dir={git_dir}", "show", f"{oid}:{path}"],
check=False,
capture_output=True,
timeout=45,
env={"PATH": os.environ.get("PATH", "")},
)
if completed.returncode != 0:
raise ReleaseBlocked("required release source missing:" + path)
return completed.stdout
def _commit_epoch(git_dir: Path, oid: str) -> int:
return int(_git(git_dir, "show", "-s", "--format=%ct", oid))
def build_production_artifact(
git_dir: Path,
binding: ReleaseBinding,
output_root: Path,
) -> dict[str, Any]:
"""Build a deterministic executable V2R2C1 health artifact from target_oid."""
binding.validate()
verify_lineage(git_dir, binding)
epoch = _commit_epoch(git_dir, binding.target_oid)
output_root.mkdir(parents=True, exist_ok=True)
stem = f"aios-{VERSION}-{binding.target_oid[:12]}"
output = output_root / (stem + ".tar.zst")
with tempfile.TemporaryDirectory(prefix="v2r2c1-artifact-") as temp:
root = Path(temp) / "payload"
root.mkdir(parents=True)
source_paths = {
"active-runtime/app/deployment_state.py": "active-runtime/app/deployment_state.py",
"active-runtime/app/v2r1c6_privacy_tools.py": "active-runtime/app/v2r1c6_privacy_tools.py",
}
for destination, source in source_paths.items():
target = root / destination
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(_payload_file(git_dir, binding.target_oid, source))
generated: dict[str, str] = {
"VERSION": VERSION + "\n",
"active-runtime/app/v2r2c1_health_service.py": (
"from v2r1c6_privacy_tools import main\n\n"
"if __name__ == '__main__':\n raise SystemExit(main())\n"
),
"deploy/aios-v2/systemd/aios-v2r1c7-production.service": (
"[Unit]\nDescription=AIOS V2R2C1 governed health contract service\n"
"After=network.target\n\n[Service]\nType=simple\n"
"User=aios-v2r1c7\nGroup=aios-v2r1c7\n"
"WorkingDirectory=/opt/aios/v2r2c1\n"
"EnvironmentFile=/etc/aios/aios-v2r2c1.env\n"
"ExecStart=/usr/bin/python3 /opt/aios/v2r2c1/active-runtime/app/v2r2c1_health_service.py --serve --host 127.0.0.1 --port ${AIOS_V2R1C7_PORT}\n"
"Restart=on-failure\nRestartSec=5\nNoNewPrivileges=true\nPrivateTmp=true\n"
"ProtectHome=true\nProtectSystem=strict\nProtectKernelLogs=true\n"
"ProtectKernelModules=true\nProtectKernelTunables=true\nProtectControlGroups=true\n"
"RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\nIPAddressDeny=any\n"
"IPAddressAllow=localhost\nUMask=0077\n\n[Install]\nWantedBy=multi-user.target\n"
),
"deploy/aios-v2/v2r2c1-runtime-config.json": json.dumps({
"schema": "aios.v2r2c1.production-runtime.v1",
"version": VERSION,
"target_oid": binding.target_oid,
"service": SERVICE,
"port": PORT,
"health_path": "/health",
"deployment_state_source": "environment_and_state_file",
}, sort_keys=True, separators=(",", ":")) + "\n",
"deploy/aios-v2/v2r2c1-rollback.json": json.dumps({
"schema": "aios.v2r2c1.rollback-metadata.v1",
"rollback_version": "V2R1C7",
"rollback_service": SERVICE,
"rollback_target_is_previous_production": True,
"automatic_rollback": False,
}, sort_keys=True, separators=(",", ":")) + "\n",
"scripts/probe_v2r2c1_health.py": (
"import json,sys,urllib.request\n"
"url=sys.argv[1] if len(sys.argv)>1 else 'http://127.0.0.1:18927/health'\n"
"with urllib.request.urlopen(url, timeout=5) as r: print(json.dumps(json.load(r), sort_keys=True))\n"
),
"scripts/rollback_v2r2c1_release.sh": "#!/bin/sh\nset -eu\nprintf '%s\\n' 'rollback requires governed owner-approved executor'\nexit 2\n",
"release-manifests/V2R2C1.json": json.dumps({
"schema": "aios.release-source.v2",
"version": VERSION,
"repository": REPOSITORY,
"target_oid": binding.target_oid,
"evidence_closure_oid": binding.evidence_closure_oid,
"production_deployed": False,
"deployment_state": "STAGING",
"artifact_kind": "executable-production-health",
}, sort_keys=True, separators=(",", ":")) + "\n",
}
for rel, text in generated.items():
target = root / rel
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(text, encoding="utf-8")
if rel.startswith("scripts/"):
target.chmod(0o755)
metadata = {
"schema": "aios.release-artifact-build.v3",
"version": VERSION,
"repository": REPOSITORY,
"source_head_oid": binding.target_oid,
"evidence_closure_oid": binding.evidence_closure_oid,
"source_date_epoch": epoch,
"clean_source": True,
"executable_payload": True,
"production_deployed": False,
"tag_created": False,
"baseline_changed": False,
"secrets_included": False,
}
(root / "BUILD-METADATA.json").write_bytes(canonical(metadata))
(root / "ROLLBACK-METADATA.json").write_bytes(canonical(json.loads((root / "deploy/aios-v2/v2r2c1-rollback.json").read_text(encoding="utf-8"))))
names = sorted("payload/" + str(path.relative_to(root)).replace(os.sep, "/") for path in root.rglob("*") if path.is_file())
(root / "ARTIFACT-MANIFEST.json").write_bytes(canonical({
"schema": "aios.release-artifact-manifest.v3",
"version": VERSION,
"source_head_oid": binding.target_oid,
"evidence_closure_oid": binding.evidence_closure_oid,
"repository": REPOSITORY,
"files": names + ["payload/ARTIFACT-MANIFEST.json"],
"deployable": True,
"executable_payload": True,
"production_deployed": False,
"tag_created": False,
"baseline_changed": False,
}))
tar_path = Path(temp) / "release.tar"
with tarfile.open(tar_path, "w") as archive:
for path in sorted(root.rglob("*")):
relative = str(path.relative_to(root)).replace(os.sep, "/")
info = archive.gettarinfo(str(path), arcname="payload/" + relative)
info.uid = info.gid = 0
info.uname = "root"
info.gname = "root"
info.mtime = epoch
info.mode = 0o755 if path.is_dir() or path.suffix == ".sh" else 0o644
if path.is_file():
with path.open("rb") as stream:
archive.addfile(info, stream)
else:
archive.addfile(info)
subprocess.run(["zstd", "-T1", "-19", "--no-progress", "-q", "-f", str(tar_path), "-o", str(output)], check=True, timeout=120)
digest = sha256_file(output)
receipt = {
"schema": "aios.release-artifact-receipt.v3",
"version": VERSION,
"repository": REPOSITORY,
"source_head_oid": binding.target_oid,
"evidence_closure_oid": binding.evidence_closure_oid,
"artifact": str(output),
"artifact_sha256": digest,
"artifact_size": output.stat().st_size,
"deployable": True,
"executable_payload": True,
"production_deployed": False,
"tag_created": False,
"baseline_changed": False,
}
atomic_json(output.with_suffix(output.suffix + ".receipt.json"), receipt, mode=0o640)
return receipt
class GovernedProductionReleaseExecutor:
def __init__(self, config: Any):
self.config = config
self.binding = ReleaseBinding()
self.binding.validate()
self.root = Path(config.state_root) / "v2r2c1-governed-production-release"
self.state = ReleaseState(self.root, self.binding)
workspace = self.root / "publication-workspace"
workspace.mkdir(parents=True, exist_ok=True)
self.publisher = TagPublisher(
workspace,
GitTagBackend(workspace, authoritative_repo=Path(config.authoritative_git_dir)),
GiteaTagClient(config.api_base, REPOSITORY, Path(config.token_path)),
config.remote_name,
state_root=self.root / "tag-publication",
)
def _request(self, request: Mapping[str, Any]) -> PublicationRequest:
binding = self.binding
for key, expected in binding.as_dict().items():
if key in request and request[key] != expected:
raise ManualOwnerReviewRequired("release request binding mismatch:" + key)
return PublicationRequest(binding.version, binding.target_oid, binding.evidence_closure_oid, binding.approval_plan_id)
def approval_request(self, request: Mapping[str, Any]) -> dict[str, Any]:
if request.get("execute") is not False:
raise ReleaseBlocked("TAG_APPROVAL_REQUEST_MUST_BE_DRY_RUN")
owner = verify_owner_approval(request, self.binding)
with StateLock(self.root / "release.lock"):
current = self.state.value.get("approval")
approval_digest = hashlib.sha256(canonical(owner)).hexdigest()
record = {
"schema": "aios.v2r2c1.tag-approval-request.v1",
"binding": self.binding.as_dict(),
"owner_approval": owner,
"owner_approval_digest_sha256": approval_digest,
"execute": False,
"recorded_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
}
if current and current != record:
raise ManualOwnerReviewRequired("persisted Owner approval differs")
self.state.value["approval"] = record
self.state.record("APPROVAL_REQUEST_RECORDED", "tag_approval_request", record)
return {"status": "RECORDED", "execute": False, "binding": self.binding.as_dict(), "owner_approval_digest_sha256": approval_digest}
def _require_approval(self) -> None:
approval = self.state.value.get("approval")
if not isinstance(approval, Mapping) or approval.get("binding") != self.binding.as_dict():
raise ReleaseBlocked("OWNER_APPROVAL_REQUEST_NOT_RECORDED")
def publish_tag(self, request: Mapping[str, Any]) -> dict[str, Any]:
if request.get("execute") is not True:
raise ReleaseBlocked("TAG_PUBLICATION_REQUIRES_EXECUTE_TRUE")
self._require_approval()
publication = self._request(request)
with StateLock(self.root / "release.lock"):
try:
verified = self.publisher._verify_publication(publication, allow_push=True, previously_deployed=False)
except PublicationManualOwnerReviewRequired as exc:
self.state.fail("MANUAL_OWNER_REVIEW_REQUIRED", str(exc))
raise ManualOwnerReviewRequired(str(exc)) from exc
except PublicationPending as exc:
self.state.fail("REMOTE_PUSH_PENDING", str(exc))
raise ReconciliationRequired(str(exc)) from exc
except PublicationReconciliationRequired as exc:
self.state.fail("RECONCILIATION_REQUIRED", str(exc))
raise ReconciliationRequired(str(exc)) from exc
result = dict(verified)
self.state.record("TAG_VERIFIED", "tag", result)
return {"status": "REMOTE_VERIFIED", **result}
def tag_status(self, request: Mapping[str, Any]) -> dict[str, Any]:
self._request(request)
return self.publisher.get_tag_publication_status(VERSION)
def build_artifact(self, request: Mapping[str, Any]) -> dict[str, Any]:
self._require_approval()
self._request(request)
receipt = build_production_artifact(Path(self.config.authoritative_git_dir), self.binding, self.root / "artifacts")
self.state.record("ARTIFACT_VERIFIED", "production_artifact", receipt)
return {"status": "VERIFIED", **receipt}
def execute_request(self, request: Mapping[str, Any]) -> dict[str, Any]:
self._require_approval()
publication = self._request(request)
tag_state = self.state.value.get("completed", {}).get("tag")
if not isinstance(tag_state, Mapping) or tag_state.get("state") not in {"GITEA_API_CONFIRMED", "REMOTE_VERIFIED"}:
raise ReleaseBlocked("REMOTE_TAG_PROOF_REQUIRED")
artifact = self.state.value.get("completed", {}).get("production_artifact")
if not isinstance(artifact, Mapping):
artifact = self.build_artifact(request)
value = {
**self.binding.as_dict(),
"tag_oid": tag_state.get("tag_oid") or tag_state.get("gitea_tag_oid"),
"artifact_path": artifact["artifact"],
"artifact_size": int(artifact["artifact_size"]),
"artifact_sha256": artifact["artifact_sha256"],
}
# The production adapter is privileged and is run through the root
# approved-release service boundary by the caller. This handler is
# still useful for a root-owned staging invocation and remains
# fail-closed when the adapter is not available.
import v2r2c1_production_adapter
runtime = v2r2c1_production_adapter.execute_request({"schema": "aios.production-executor-request.v1", "operation": "apply", **value})
if runtime.get("status") != "ACTIVE_VERIFIED":
self.state.fail("RECONCILIATION_REQUIRED", "production health did not verify")
raise ReconciliationRequired("production health did not verify")
self.state.record("PRODUCTION_VERIFIED", "production", runtime)
return {"status": "PRODUCTION_VERIFIED", "runtime": runtime, "artifact": artifact, "tag_oid": value["tag_oid"]}
def consistency(self, request: Mapping[str, Any]) -> dict[str, Any]:
publication = self._request(request)
result = self.publisher.get_release_consistency(publication)
result["release_binding"] = self.binding.as_dict()
result["production_health"] = _health(self.config.health_url, self.binding, self.state.value.get("completed", {}).get("production_artifact", {}))
result["registry"] = _read_json(Path(self.config.registry_root) / f"{VERSION}.json")
result["baseline"] = _read_json(Path(self.config.baseline_path))
return result
def acceptance(self, request: Mapping[str, Any]) -> dict[str, Any]:
result = self.consistency(request)
result["accepted"] = result.get("status") == "PASS" and result.get("production_health", {}).get("status") == "ACTIVE_VERIFIED"
self.state.record("ACCEPTED" if result["accepted"] else "RECONCILIATION_REQUIRED", "acceptance", result)
return result
def _read_json(path: Path) -> dict[str, Any]:
try:
value = json.loads(path.read_text(encoding="utf-8"))
except FileNotFoundError:
return {"status": "MISSING", "path": str(path)}
if not isinstance(value, dict):
return {"status": "INVALID", "path": str(path)}
return {"status": "PRESENT", "path": str(path), **value}
def _health(url: str, binding: ReleaseBinding, artifact: Mapping[str, Any]) -> dict[str, Any]:
import urllib.request
try:
with urllib.request.urlopen(url, timeout=5) as response:
body = json.loads(response.read().decode("utf-8"))
status = response.status
except Exception as exc:
return {"status": "UNAVAILABLE", "error": type(exc).__name__}
expected_artifact = artifact.get("artifact_sha256")
exact = (
status == 200
and body.get("version") == binding.version
and body.get("commit_oid") == binding.target_oid
and (not expected_artifact or body.get("artifact_sha256") == expected_artifact)
and body.get("deployment_state") == "ACTIVE"
and body.get("production_deployed") is True
)
return {"status": "ACTIVE_VERIFIED" if exact else "CONFLICT", "http_status": status, **body}
def build_executor_config(config: Any) -> GovernedProductionReleaseExecutor:
return GovernedProductionReleaseExecutor(config)
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("operation", choices=("build-artifact", "status"))
parser.add_argument("--git-dir", type=Path, default=Path("/data/aios-gitops/repo.git"))
parser.add_argument("--output-root", type=Path, default=Path("/var/lib/aios-gitops/approved-artifacts/V2R2C1"))
args = parser.parse_args()
binding = ReleaseBinding()
if args.operation == "build-artifact":
print(json.dumps(build_production_artifact(args.git_dir, binding, args.output_root), sort_keys=True))
else:
print(json.dumps({"binding": binding.as_dict(), "status": "READY"}, sort_keys=True))
return 0
if __name__ == "__main__":
raise SystemExit(main())
+293
View File
@@ -0,0 +1,293 @@
#!/usr/bin/env python3
"""Thin production tag adapter over AIOS's existing mature TagPublisher.
This adapter deliberately does not call TagPublisher.publish(), because that
legacy method also mutates Registry/Baseline. The production executor requires:
Tag -> runtime ACTIVE -> Registry -> Baseline.
`apply` reuses TagPublisher._verify_publication() under the publisher's durable
per-version lock. That method performs the existing fail-closed annotated-tag,
network remote, Gitea API, peeled-target and listing verification, but stops
before Registry/Baseline mutation.
`probe` is read-only.
"""
from __future__ import annotations
import argparse
import json
import os
import re
import sys
from pathlib import Path
from typing import Any, Mapping
from gitea_tag_publication import (
GiteaTagClient,
GitTagBackend,
PublicationConflict,
PublicationManualOwnerReviewRequired,
PublicationPending,
PublicationReconciliationRequired,
PublicationRequest,
PublicationState,
TagPublisher,
)
OID_RE = re.compile(r"^[0-9a-f]{40}$")
PLAN_RE = re.compile(r"^sha256:[0-9a-f]{64}$")
def load_request() -> dict[str, Any]:
value = json.load(sys.stdin)
if not isinstance(value, dict):
raise ValueError("request_object_required")
if value.get("schema") != "aios.production-executor-request.v1":
raise ValueError("request_schema_invalid")
if not re.fullmatch(r"V\d+R\d+C\d+(?:-[A-Za-z0-9_.-]+)?", str(value.get("version", ""))):
raise ValueError("version_invalid")
for name in ("target_oid", "evidence_closure_oid"):
if not OID_RE.fullmatch(str(value.get(name, ""))):
raise ValueError(f"{name}_invalid")
if not PLAN_RE.fullmatch(str(value.get("approval_plan_id", ""))):
raise ValueError("approval_plan_id_invalid")
if value.get("repository_identity") != "aios/aios":
raise ValueError("repository_identity_invalid")
return value
def publication_request(value: Mapping[str, Any]) -> PublicationRequest:
return PublicationRequest(
str(value["version"]),
str(value["target_oid"]),
str(value["evidence_closure_oid"]),
str(value["approval_plan_id"]),
)
def response(value: Mapping[str, Any], status: str, **extra: Any) -> dict[str, Any]:
return {
"status": status,
"version": value["version"],
"target_oid": value["target_oid"],
"evidence_closure_oid": value["evidence_closure_oid"],
"artifact_sha256": value["artifact_sha256"],
"approval_plan_id": value["approval_plan_id"],
**extra,
}
def build(args: argparse.Namespace) -> tuple[TagPublisher, GitTagBackend, GiteaTagClient]:
git = GitTagBackend(args.workspace, authoritative_repo=args.repository)
gitea = GiteaTagClient(
args.gitea_api,
args.gitea_repo,
args.gitea_token_file,
token_line=args.gitea_token_line,
)
publisher = TagPublisher(
args.workspace,
git,
gitea,
args.remote,
state_root=args.state_root,
)
return publisher, git, gitea
def _gitea_lookup(gitea: GiteaTagClient, version: str) -> tuple[str, str] | None:
try:
return gitea.lookup_tag(version)
except RuntimeError as exc:
if "HTTP 404" in str(exc):
return None
raise
def probe(
args: argparse.Namespace,
value: Mapping[str, Any],
publisher: TagPublisher,
git: GitTagBackend,
gitea: GiteaTagClient,
) -> dict[str, Any]:
request = publication_request(value)
request.validate()
git.validate_evidence_lineage(request)
local_oid = git._rev(f"refs/tags/{request.version}")
local_target = git._rev(f"refs/tags/{request.version}^{{commit}}")
remote = git.remote_tag(args.remote, request.version)
api = _gitea_lookup(gitea, request.version)
listing = False
if api is not None:
listing = gitea.tag_visible_in_listing(request.version, request.target_oid)
if local_oid is None and remote is None and api is None:
return response(value, "NOT_APPLIED")
if local_oid is not None:
if local_target is None or local_oid == local_target:
return response(value, "CONFLICT", reason="local_tag_not_annotated")
if local_target != request.target_oid:
return response(
value,
"CONFLICT",
reason="local_peeled_target_mismatch",
tag_oid=local_oid,
peeled_target_oid=local_target,
)
# A remote/Gitea tag without the Authority tag object is not safe to
# recreate, because a new annotated tag would have a different object OID.
if local_oid is None and (remote is not None or api is not None):
return response(
value,
"INDETERMINATE",
reason="remote_or_gitea_tag_exists_without_authority_tag_object",
)
if remote is not None:
if remote[0] != local_oid or remote[1] != request.target_oid:
return response(
value,
"CONFLICT",
reason="network_remote_tag_mismatch",
tag_oid=local_oid,
peeled_target_oid=local_target,
remote_tag_oid=remote[0],
remote_peeled_target_oid=remote[1],
)
if api is not None:
if api[0] != local_oid or api[1] != request.target_oid:
return response(
value,
"CONFLICT",
reason="gitea_api_tag_mismatch",
tag_oid=local_oid,
peeled_target_oid=local_target,
gitea_tag_oid=api[0],
gitea_peeled_target_oid=api[1],
)
if (
local_oid is not None
and remote is not None
and api is not None
and listing
):
return response(
value,
"APPLIED_VERIFIED",
tag_oid=local_oid,
peeled_target_oid=local_target,
remote_tag_oid=remote[0],
remote_peeled_target_oid=remote[1],
gitea_tag_oid=api[0],
gitea_peeled_target_oid=api[1],
gitea_listing_visible=True,
)
if local_oid is not None and remote is None and api is None:
return response(
value,
"PARTIAL_LOCAL",
tag_oid=local_oid,
peeled_target_oid=local_target,
)
return response(
value,
"INDETERMINATE",
reason="tag_visibility_partial",
tag_oid=local_oid,
peeled_target_oid=local_target,
)
def apply(
args: argparse.Namespace,
value: Mapping[str, Any],
publisher: TagPublisher,
git: GitTagBackend,
gitea: GiteaTagClient,
) -> dict[str, Any]:
before = probe(args, value, publisher, git, gitea)
if before["status"] in {"APPLIED_VERIFIED", "CONFLICT", "INDETERMINATE"}:
return before
request = publication_request(value)
with publisher.store.lock(request.version):
request.validate()
publisher._assert_fingerprint(request)
publisher.store.record(request, PublicationState.PENDING)
publisher._verify_publication(
request,
allow_push=True,
previously_deployed=False,
)
return probe(args, value, publisher, git, gitea)
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--workspace", type=Path, default=Path(os.environ["AIOS_GITOPS_CONTROLLER_WORKSPACE"]))
parser.add_argument("--repository", type=Path, default=Path(os.environ["AIOS_GITOPS_AUTHORITATIVE_GIT_DIR"]))
parser.add_argument("--remote", default=os.getenv("AIOS_GITOPS_GITEA_REMOTE", "gitea"))
parser.add_argument("--gitea-api", default=os.getenv("AIOS_GITOPS_GITEA_API_URL", ""))
parser.add_argument("--gitea-repo", default=os.getenv("AIOS_GITOPS_GITEA_REPOSITORY", ""))
parser.add_argument(
"--gitea-token-file",
type=Path,
default=Path(os.getenv("AIOS_GITOPS_GITEA_TOKEN_FILE", "")),
)
parser.add_argument(
"--gitea-token-line",
type=int,
default=int(os.getenv("AIOS_GITOPS_GITEA_TOKEN_LINE", "1")),
)
parser.add_argument(
"--state-root",
type=Path,
default=Path(
os.getenv(
"AIOS_GITOPS_PUBLICATION_STATE_DIR",
"/var/lib/aios-gitops/tag-publications",
)
),
)
args = parser.parse_args()
value = load_request()
publisher, git, gitea = build(args)
operation = value.get("operation")
if operation == "probe":
result = probe(args, value, publisher, git, gitea)
elif operation == "apply":
result = apply(args, value, publisher, git, gitea)
else:
raise ValueError("unsupported_operation")
print(json.dumps(result, sort_keys=True))
return 0
if __name__ == "__main__":
try:
raise SystemExit(main())
except PublicationManualOwnerReviewRequired as exc:
print(json.dumps({"status": "CONFLICT", "reason": str(exc)}, sort_keys=True))
raise SystemExit(20)
except PublicationConflict as exc:
print(json.dumps({"status": "CONFLICT", "reason": str(exc)}, sort_keys=True))
raise SystemExit(20)
except (PublicationPending, PublicationReconciliationRequired) as exc:
print(json.dumps({"status": "INDETERMINATE", "reason": str(exc)}, sort_keys=True))
raise SystemExit(21)
except Exception as exc:
print(
json.dumps(
{"status": "INDETERMINATE", "error": type(exc).__name__, "reason": str(exc)},
sort_keys=True,
)
)
raise SystemExit(22)
+456
View File
@@ -0,0 +1,456 @@
from __future__ import annotations
import dataclasses
import datetime as dt
import hashlib
import json
import sqlite3
import re
from pathlib import Path
from typing import Any, Iterable, Mapping
UTC = dt.timezone.utc
_ALLOWED_CONFIDENTIALITY = frozenset({"public", "tenant", "project", "private", "run"})
_PRIVILEGED = frozenset({"*", "admin", "system.admin", "message.admin", "task.admin"})
class HandoffError(RuntimeError):
code = "HANDOFF_ERROR"
class MessageScopeMismatch(HandoffError):
code = "MESSAGE_SCOPE_MISMATCH_DENY"
class ConfidentialityScopeDenied(HandoffError):
code = "MESSAGE_CONFIDENTIALITY_SCOPE_ENFORCE"
class MessageExpired(HandoffError):
code = "MESSAGE_EXPIRED"
class SequenceGap(HandoffError):
code = "MESSAGE_SEQUENCE_GAP_DETECT"
class SequenceConflict(HandoffError):
code = "MESSAGE_SEQUENCE_CONFLICT"
class IdempotencyConflict(HandoffError):
code = "MESSAGE_IDEMPOTENCY_CONFLICT"
def _utcnow() -> dt.datetime:
return dt.datetime.now(UTC)
def _iso(value: dt.datetime | str | None) -> str | None:
if value is None:
return None
parsed = dt.datetime.fromisoformat(value.replace("Z", "+00:00")) if isinstance(value, str) else value
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=UTC)
return parsed.astimezone(UTC).isoformat()
def _canonical(value: Any) -> str:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False, default=str)
def _sha(value: Any) -> str:
return hashlib.sha256(_canonical(value).encode("utf-8")).hexdigest()
def _clean(value: Any) -> str | None:
if value is None:
return None
text = str(value).strip()
return text or None
def _tuple_scope(value: Iterable[str] | str | None) -> tuple[str, ...]:
if value is None:
return ()
if isinstance(value, str):
value = value.split(",")
return tuple(sorted({str(item).strip() for item in value if str(item).strip()}))
_SECRET_KEY = re.compile(r"(?i)(?:token|secret|password|private[_-]?key|api[_-]?key)")
_SECRET_VALUE = re.compile(r"(?i)bearer\s+[A-Za-z0-9._~+/-]{8,}|(?:token|secret|password|api[_-]?key)\s*[:=]\s*\S+")
_SAFE_REFERENCE = re.compile(r"^(?:systemd-credential|vault|secret-manager):[A-Za-z0-9][A-Za-z0-9._:/#-]{0,255}$")
def assert_payload_secret_safe(value: Any, *, key: str = "") -> None:
if isinstance(value, Mapping):
for child_key, child in value.items():
assert_payload_secret_safe(child, key=str(child_key))
return
if isinstance(value, (list, tuple)):
for child in value:
assert_payload_secret_safe(child, key=key)
return
if isinstance(value, (bytes, bytearray)):
raise ValueError("CREDENTIAL_SECRET_EXPOSURE_DENY")
if isinstance(value, str):
if _SECRET_VALUE.search(value):
raise ValueError("CREDENTIAL_SECRET_EXPOSURE_DENY")
if _SECRET_KEY.search(key) and value and _SAFE_REFERENCE.fullmatch(value) is None:
raise ValueError("CREDENTIAL_SECRET_EXPOSURE_DENY")
@dataclasses.dataclass(frozen=True)
class DeliveryContext:
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
run_id: str | None = None
capabilities: tuple[str, ...] = ()
@classmethod
def from_actor(cls, actor: Any) -> "DeliveryContext":
def get(name: str) -> Any:
if isinstance(actor, Mapping):
return actor.get(name)
return getattr(actor, name, None)
return cls(
tenant_id=_clean(get("tenant_id")),
user_id=_clean(get("user_id")),
project_id=_clean(get("project_id")),
conversation_id=_clean(get("conversation_id")),
run_id=_clean(get("run_id")),
capabilities=_tuple_scope(get("capabilities") or get("capability_scope")),
)
@property
def privileged(self) -> bool:
return bool(set(self.capabilities) & _PRIVILEGED)
@dataclasses.dataclass(frozen=True)
class MessageEnvelope:
source: str
target: str
message_type: str
payload: Mapping[str, Any]
sequence: int
idempotency_key: str
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
task_id: str | None = None
run_id: str | None = None
correlation_id: str | None = None
causation_id: str | None = None
capability_scope: tuple[str, ...] = ()
confidentiality_scope: str = "private"
expires_at: str | None = None
created_at: str = dataclasses.field(default_factory=lambda: _iso(_utcnow()) or "")
envelope_id: str = ""
def __post_init__(self) -> None:
assert_payload_secret_safe(self.payload)
for field in ("source", "target", "message_type", "idempotency_key", "tenant_id", "user_id",
"project_id", "conversation_id", "task_id", "run_id", "correlation_id", "causation_id"):
object.__setattr__(self, field, _clean(getattr(self, field)) if field not in {
"source", "target", "message_type", "idempotency_key"
} else (_clean(getattr(self, field)) or ""))
object.__setattr__(self, "capability_scope", _tuple_scope(self.capability_scope))
object.__setattr__(self, "confidentiality_scope", (_clean(self.confidentiality_scope) or "private").lower())
object.__setattr__(self, "expires_at", _iso(self.expires_at))
object.__setattr__(self, "created_at", _iso(self.created_at) or "")
if not self.source or not self.target or not self.message_type or not self.idempotency_key:
raise ValueError("MESSAGE_ENVELOPE_IDENTITY_REQUIRED")
if not isinstance(self.sequence, int) or self.sequence < 1:
raise ValueError("MESSAGE_SEQUENCE_MUST_BE_POSITIVE")
if self.confidentiality_scope not in _ALLOWED_CONFIDENTIALITY:
raise ValueError("MESSAGE_CONFIDENTIALITY_SCOPE_INVALID")
if self.confidentiality_scope != "public" and self.tenant_id is None:
raise ValueError("MESSAGE_TENANT_REQUIRED")
if self.confidentiality_scope in {"project", "private", "run"} and self.project_id is None:
raise ValueError("MESSAGE_PROJECT_REQUIRED")
if self.confidentiality_scope in {"private", "run"} and self.user_id is None:
raise ValueError("MESSAGE_USER_REQUIRED")
if self.confidentiality_scope == "run" and self.run_id is None:
raise ValueError("MESSAGE_RUN_REQUIRED")
expected = stable_event_id(
source=self.source, target=self.target, message_type=self.message_type, payload=self.payload,
tenant_id=self.tenant_id, user_id=self.user_id, project_id=self.project_id,
conversation_id=self.conversation_id, task_id=self.task_id, run_id=self.run_id,
correlation_id=self.correlation_id, causation_id=self.causation_id,
sequence=self.sequence, idempotency_key=self.idempotency_key,
)
if self.envelope_id and self.envelope_id != expected:
raise ValueError("MESSAGE_ENVELOPE_ID_MISMATCH")
object.__setattr__(self, "envelope_id", expected)
@property
def payload_sha256(self) -> str:
return _sha(self.payload)
@property
def stream_key(self) -> str:
return _sha({
"tenant_id": self.tenant_id, "source": self.source, "target": self.target,
"conversation_id": self.conversation_id, "task_id": self.task_id, "run_id": self.run_id,
})
def is_expired(self, *, now: dt.datetime | None = None) -> bool:
if self.expires_at is None:
return False
return dt.datetime.fromisoformat(self.expires_at) <= (now or _utcnow()).astimezone(UTC)
def as_record(self) -> dict[str, Any]:
return {
"envelope_id": self.envelope_id, "stream_key": self.stream_key, "source": self.source,
"target": self.target, "message_type": self.message_type, "payload_json": _canonical(self.payload),
"payload_sha256": self.payload_sha256, "tenant_id": self.tenant_id, "user_id": self.user_id,
"project_id": self.project_id, "conversation_id": self.conversation_id, "task_id": self.task_id,
"run_id": self.run_id, "correlation_id": self.correlation_id,
"causation_id": self.causation_id, "sequence": self.sequence,
"capability_scope_json": _canonical(self.capability_scope),
"confidentiality_scope": self.confidentiality_scope, "idempotency_key": self.idempotency_key,
"expires_at": self.expires_at, "created_at": self.created_at,
}
def stable_event_id(*, source: Any, target: Any, message_type: Any, payload: Any,
tenant_id: Any = None, user_id: Any = None, project_id: Any = None,
conversation_id: Any = None, task_id: Any = None, run_id: Any = None,
correlation_id: Any = None, causation_id: Any = None,
sequence: Any = None, idempotency_key: Any = None) -> str:
assert_payload_secret_safe(payload)
material = {
"source": _clean(source), "target": _clean(target), "message_type": _clean(message_type),
"payload_sha256": _sha(payload), "tenant_id": _clean(tenant_id), "user_id": _clean(user_id),
"project_id": _clean(project_id), "conversation_id": _clean(conversation_id),
"task_id": _clean(task_id), "run_id": _clean(run_id), "causation_id": _clean(causation_id),
"sequence": sequence, "idempotency_key": _clean(idempotency_key),
}
# Preserve historical envelope IDs for legacy rows that predate correlation_id.
if _clean(correlation_id) is not None:
material["correlation_id"] = _clean(correlation_id)
return _sha(material)
def bind_envelope_scope(envelope: MessageEnvelope, actor: Any) -> MessageEnvelope:
ctx = DeliveryContext.from_actor(actor)
if ctx.privileged:
return envelope
if ctx.tenant_id is None:
if envelope.tenant_id is not None:
raise MessageScopeMismatch(MessageScopeMismatch.code)
return envelope
updates: dict[str, Any] = {}
for field in ("tenant_id", "user_id", "project_id", "conversation_id", "run_id"):
trusted = getattr(ctx, field)
requested = getattr(envelope, field)
if requested is not None and requested != trusted:
raise MessageScopeMismatch(f"{MessageScopeMismatch.code}:{field}")
if requested is None and trusted is not None:
updates[field] = trusted
if not updates:
return envelope
data = dataclasses.asdict(envelope)
data.update(updates)
data["envelope_id"] = ""
return MessageEnvelope(**data)
def authorize_delivery(actor: Any, envelope: MessageEnvelope) -> None:
ctx = DeliveryContext.from_actor(actor)
if ctx.privileged:
return
if not set(envelope.capability_scope).issubset(set(ctx.capabilities)):
raise ConfidentialityScopeDenied(f"{ConfidentialityScopeDenied.code}:capability")
if envelope.confidentiality_scope == "public":
return
if ctx.tenant_id != envelope.tenant_id:
raise MessageScopeMismatch(f"{MessageScopeMismatch.code}:tenant_id")
if envelope.confidentiality_scope in {"project", "private", "run"} and ctx.project_id != envelope.project_id:
raise ConfidentialityScopeDenied(f"{ConfidentialityScopeDenied.code}:project_id")
if envelope.confidentiality_scope in {"private", "run"} and ctx.user_id != envelope.user_id:
raise ConfidentialityScopeDenied(f"{ConfidentialityScopeDenied.code}:user_id")
if envelope.confidentiality_scope == "run" and ctx.run_id != envelope.run_id:
raise ConfidentialityScopeDenied(f"{ConfidentialityScopeDenied.code}:run_id")
_SCHEMA = """
CREATE TABLE IF NOT EXISTS message_handoff (
envelope_id TEXT PRIMARY KEY,
stream_key TEXT NOT NULL,
source TEXT NOT NULL,
target TEXT NOT NULL,
message_type TEXT NOT NULL,
payload_json TEXT NOT NULL,
payload_sha256 TEXT NOT NULL,
tenant_id TEXT,
user_id TEXT,
project_id TEXT,
conversation_id TEXT,
task_id TEXT,
run_id TEXT,
correlation_id TEXT,
causation_id TEXT,
sequence INTEGER NOT NULL CHECK(sequence >= 1),
capability_scope_json TEXT NOT NULL,
confidentiality_scope TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
expires_at TEXT,
created_at TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'PENDING',
delivered_at TEXT,
acked_at TEXT,
UNIQUE(stream_key, idempotency_key),
UNIQUE(stream_key, sequence)
);
CREATE INDEX IF NOT EXISTS idx_message_handoff_target_status
ON message_handoff(target, status, created_at);
CREATE INDEX IF NOT EXISTS idx_message_handoff_scope
ON message_handoff(tenant_id, project_id, user_id, conversation_id, run_id);
"""
class DurableHandoffStore:
def __init__(self, path: str | Path) -> None:
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
with self._connect() as conn:
conn.executescript(_SCHEMA)
columns = {str(row[1]) for row in conn.execute("PRAGMA table_info(message_handoff)")}
if "correlation_id" not in columns:
conn.execute("ALTER TABLE message_handoff ADD COLUMN correlation_id TEXT")
conn.execute(
"CREATE INDEX IF NOT EXISTS idx_message_handoff_lineage "
"ON message_handoff(correlation_id, causation_id)"
)
def _connect(self) -> sqlite3.Connection:
conn = sqlite3.connect(self.path, timeout=30.0)
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA foreign_keys=ON")
conn.execute("PRAGMA busy_timeout=30000")
return conn
@staticmethod
def _from_row(row: sqlite3.Row) -> MessageEnvelope:
return MessageEnvelope(
source=row["source"], target=row["target"], message_type=row["message_type"],
payload=json.loads(row["payload_json"]), tenant_id=row["tenant_id"], user_id=row["user_id"],
project_id=row["project_id"], conversation_id=row["conversation_id"], task_id=row["task_id"],
run_id=row["run_id"], correlation_id=row["correlation_id"],
causation_id=row["causation_id"], sequence=int(row["sequence"]),
capability_scope=tuple(json.loads(row["capability_scope_json"])),
confidentiality_scope=row["confidentiality_scope"], idempotency_key=row["idempotency_key"],
expires_at=row["expires_at"], created_at=row["created_at"], envelope_id=row["envelope_id"],
)
def put(self, envelope: MessageEnvelope) -> tuple[str, MessageEnvelope]:
if envelope.is_expired():
raise MessageExpired(MessageExpired.code)
record = envelope.as_record()
with self._connect() as conn:
conn.execute("BEGIN IMMEDIATE")
replay = conn.execute(
"SELECT * FROM message_handoff WHERE stream_key=? AND idempotency_key=?",
(envelope.stream_key, envelope.idempotency_key),
).fetchone()
if replay is not None:
existing = self._from_row(replay)
if existing.envelope_id != envelope.envelope_id or existing.payload_sha256 != envelope.payload_sha256:
raise IdempotencyConflict(IdempotencyConflict.code)
conn.commit()
return "REPLAY", existing
seq = conn.execute(
"SELECT * FROM message_handoff WHERE stream_key=? AND sequence=?",
(envelope.stream_key, envelope.sequence),
).fetchone()
if seq is not None:
existing = self._from_row(seq)
if existing.envelope_id == envelope.envelope_id:
conn.commit()
return "REPLAY", existing
raise SequenceConflict(SequenceConflict.code)
maximum = int(conn.execute(
"SELECT COALESCE(MAX(sequence),0) FROM message_handoff WHERE stream_key=?",
(envelope.stream_key,),
).fetchone()[0])
expected = maximum + 1
if envelope.sequence != expected:
raise SequenceGap(f"{SequenceGap.code}:expected={expected}:actual={envelope.sequence}")
columns = tuple(record)
placeholders = ",".join("?" for _ in columns)
conn.execute(
f"INSERT INTO message_handoff ({','.join(columns)}) VALUES ({placeholders})",
tuple(record[column] for column in columns),
)
conn.commit()
return "CREATED", envelope
def get(self, envelope_id: str, *, actor: Any) -> MessageEnvelope:
with self._connect() as conn:
row = conn.execute("SELECT * FROM message_handoff WHERE envelope_id=?", (envelope_id,)).fetchone()
if row is None:
raise KeyError(envelope_id)
envelope = self._from_row(row)
if envelope.is_expired():
conn.execute("UPDATE message_handoff SET status='EXPIRED' WHERE envelope_id=?", (envelope_id,))
conn.commit()
raise MessageExpired(MessageExpired.code)
authorize_delivery(actor, envelope)
return envelope
def pending(self, *, target: str, actor: Any, limit: int = 100) -> list[MessageEnvelope]:
if not 1 <= limit <= 1000:
raise ValueError("MESSAGE_PENDING_LIMIT_INVALID")
visible: list[MessageEnvelope] = []
with self._connect() as conn:
rows = conn.execute(
"SELECT * FROM message_handoff WHERE target=? AND status='PENDING' ORDER BY created_at, sequence LIMIT ?",
(target, limit),
).fetchall()
for row in rows:
envelope = self._from_row(row)
if envelope.is_expired():
conn.execute("UPDATE message_handoff SET status='EXPIRED' WHERE envelope_id=?", (envelope.envelope_id,))
continue
try:
authorize_delivery(actor, envelope)
except HandoffError:
continue
visible.append(envelope)
conn.commit()
return visible
def mark_delivered(self, envelope_id: str, *, actor: Any) -> MessageEnvelope:
envelope = self.get(envelope_id, actor=actor)
with self._connect() as conn:
conn.execute(
"UPDATE message_handoff SET status='DELIVERED', delivered_at=? "
"WHERE envelope_id=? AND status='PENDING'",
(_iso(_utcnow()), envelope_id),
)
conn.commit()
return envelope
def acknowledge(self, envelope_id: str, *, actor: Any) -> MessageEnvelope:
envelope = self.get(envelope_id, actor=actor)
with self._connect() as conn:
conn.execute(
"UPDATE message_handoff SET status='ACKED', acked_at=? WHERE envelope_id=?",
(_iso(_utcnow()), envelope_id),
)
conn.commit()
return envelope
class HandoffFabric:
def __init__(self, store: DurableHandoffStore) -> None:
self.store = store
def publish(self, envelope: MessageEnvelope, *, actor: Any | None = None) -> tuple[str, MessageEnvelope]:
return self.store.put(bind_envelope_scope(envelope, actor) if actor is not None else envelope)
@@ -0,0 +1,68 @@
from __future__ import annotations
from typing import Any, Mapping
from message_handoff import MessageEnvelope
def _get(value: Any, *names: str, default: Any = None) -> Any:
for name in names:
if isinstance(value, Mapping) and name in value:
return value[name]
if hasattr(value, name):
return getattr(value, name)
return default
def agent_event_to_envelope(event: Any, *, target: str | None = None, sequence: int = 1,
idempotency_key: str | None = None, **scope: Any) -> MessageEnvelope:
payload = _get(event, "payload", "data", default={}) or {}
target = target or _get(event, "target", "recipient")
if not target:
raise ValueError("MESSAGE_TARGET_REQUIRED")
return MessageEnvelope(
source=str(_get(event, "source", "producer", "agent_id", default="agent-event")),
target=str(target),
message_type=str(_get(event, "event_type", "type", "message_type", default="agent.event")),
payload=dict(payload),
sequence=sequence,
idempotency_key=idempotency_key or str(_get(event, "id", "event_id", default="")) or f"agent-event:{sequence}",
causation_id=_get(event, "causation_id"),
**scope,
)
def taskplane_event_to_envelope(event: Any, *, target: str, sequence: int = 1,
idempotency_key: str | None = None, **scope: Any) -> MessageEnvelope:
data = _get(event, "data", "payload", default={}) or {}
event_id = str(_get(event, "id", "event_id", default=""))
return MessageEnvelope(
source=str(_get(event, "source", default="task-plane")),
target=target,
message_type=str(_get(event, "type", "event_type", default="task.event")),
payload=dict(data),
sequence=sequence,
idempotency_key=idempotency_key or event_id or f"task-event:{sequence}",
causation_id=event_id or None,
**scope,
)
def legacy_message_to_envelope(message: Any, *, target: str | None = None, sequence: int = 1,
idempotency_key: str | None = None, **scope: Any) -> MessageEnvelope:
payload = _get(message, "payload", "data", "body", "content", default={})
if not isinstance(payload, Mapping):
payload = {"value": payload}
target = target or _get(message, "target", "recipient", "receiver")
if not target:
raise ValueError("MESSAGE_TARGET_REQUIRED")
source = _get(message, "source", "sender", "agent_id", default="legacy-message-bus")
kind = _get(message, "message_type", "type", "kind", default="legacy.message")
identifier = _get(message, "id", "message_id", "event_id")
return MessageEnvelope(
source=str(source), target=str(target), message_type=str(kind), payload=dict(payload),
sequence=sequence,
idempotency_key=idempotency_key or (str(identifier) if identifier else f"legacy:{sequence}"),
causation_id=str(identifier) if identifier else None,
**scope,
)
+642
View File
@@ -0,0 +1,642 @@
#!/usr/bin/env python3
"""One-shot local Owner authentication ingress for AIOS.
This module deliberately does not mint a reusable Owner token.
Authentication is based on a real local login session:
- a root-run one-shot server owns a Unix socket;
- the client must run as the configured Owner OS account;
- the server validates SO_PEERCRED, Linux audit loginuid and the controlling TTY;
- the Owner must interactively confirm the exact task/branch/base request;
- the server constructs a narrow OwnerPrincipal and invokes the existing
OwnerAuthorizationBroker;
- one successful authorization consumes the service lifetime.
The resulting principal is fixed to one non-wildcard intake scope
(`commit:create` for the current deployment) and cannot carry production,
tag, baseline, main or history-rewrite capability.
This is a trust-root ingress, not a Git writer and not a production executor.
"""
from __future__ import annotations
import argparse
import fcntl
import hashlib
import json
import os
from pathlib import Path
import pwd
import re
import secrets
import socket
import stat
import struct
import sys
import tempfile
import time
from typing import Any, Mapping
BINDING_SCHEMA = "aios.local-owner-auth-ingress.v1"
CLIENT_BEGIN_SCHEMA = "aios.local-owner-auth-client-begin.v1"
CHALLENGE_SCHEMA = "aios.local-owner-auth-challenge.v1"
CLIENT_CONFIRM_SCHEMA = "aios.local-owner-auth-client-confirm.v1"
RESULT_SCHEMA = "aios.local-owner-auth-result.v1"
ACTION = "AUTHORIZE_TASK_WRITER_PROVISIONING"
OID_RE = re.compile(r"^[0-9a-f]{40}$")
IDENTITY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._@:+/-]{1,254}$")
SCOPE_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{1,127}$")
TASK_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{1,255}$")
BRANCH_RE = re.compile(r"^agent/[A-Za-z0-9._/-]{1,220}$")
MAX_FRAME = 64 * 1024
CHALLENGE_TTL_SECONDS = 120
EXPECTED_ALLOWED_OPERATIONS = frozenset(
{
"create_worktree",
"create_target_branch_if_absent",
"commit_to_target_branch",
"push_target_branch",
}
)
EXPECTED_DENIED_OPERATIONS = frozenset(
{
"direct_main_push",
"force_push",
"tag_create",
"tag_update",
"tag_delete",
"branch_delete",
"repository_admin",
"baseline_write",
"production_deploy",
"production_marker_write",
"history_rewrite",
}
)
SENSITIVE_FRAGMENTS = (
"authorization_header",
"bearer",
"client_secret",
"password",
"private_key",
"raw_secret",
"raw_token",
"secret_value",
)
class BootstrapError(RuntimeError):
def __init__(self, code: str, status: int = 1) -> None:
super().__init__(code)
self.code = code
self.status = status
def canonical_json(value: Any) -> bytes:
return (
json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
+ "\n"
).encode("utf-8")
def sha256_json(value: Any) -> str:
return hashlib.sha256(canonical_json(value)).hexdigest()
def atomic_json(path: Path, value: Mapping[str, Any], mode: int = 0o600) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
if path.parent.is_symlink() or path.is_symlink():
raise BootstrapError("STATE_SYMLINK_REJECTED")
fd, name = tempfile.mkstemp(prefix=f".{path.name}.", dir=str(path.parent))
try:
os.fchmod(fd, mode)
with os.fdopen(fd, "wb", closefd=True) as stream:
stream.write(canonical_json(dict(value)))
stream.flush()
os.fsync(stream.fileno())
os.replace(name, path)
os.chmod(path, mode)
dfd = os.open(path.parent, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
try:
os.fsync(dfd)
finally:
os.close(dfd)
finally:
try:
os.unlink(name)
except FileNotFoundError:
pass
def secure_root_file(path: Path) -> dict[str, Any]:
if path.is_symlink() or not path.is_file():
raise BootstrapError("BINDING_FILE_INVALID")
st = path.lstat()
if st.st_uid != 0:
raise BootstrapError("BINDING_FILE_ROOT_OWNER_REQUIRED")
if stat.S_IMODE(st.st_mode) & 0o077:
raise BootstrapError("BINDING_FILE_PERMISSIONS_INVALID")
if st.st_size <= 0 or st.st_size > 1_048_576:
raise BootstrapError("BINDING_FILE_SIZE_INVALID")
try:
value = json.loads(path.read_text(encoding="utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise BootstrapError("BINDING_JSON_INVALID") from exc
if not isinstance(value, dict):
raise BootstrapError("BINDING_JSON_INVALID")
return value
def validate_binding(value: Mapping[str, Any]) -> dict[str, Any]:
if set(value) != {
"schema",
"owner_identity",
"owner_role",
"authenticated_scope",
"request",
}:
raise BootstrapError("BINDING_FIELDS_REJECTED")
if value["schema"] != BINDING_SCHEMA:
raise BootstrapError("BINDING_SCHEMA_REJECTED")
identity = value["owner_identity"]
if not isinstance(identity, str) or not IDENTITY_RE.fullmatch(identity):
raise BootstrapError("OWNER_IDENTITY_INVALID")
if value["owner_role"] != "owner":
raise BootstrapError("OWNER_ROLE_INVALID")
scope = value["authenticated_scope"]
if (
not isinstance(scope, str)
or not SCOPE_RE.fullmatch(scope)
or scope == "writer:provision"
or scope == "*"
or scope.endswith(":*")
):
raise BootstrapError("OWNER_SCOPE_INVALID")
request = value["request"]
if not isinstance(request, dict):
raise BootstrapError("REQUEST_INVALID")
expected_fields = {
"action",
"task_id",
"repository",
"base_ref",
"base_oid",
"target_branch",
"one_time_use",
"expires_in_seconds",
"production_change_authorized",
"tag_change_authorized",
"baseline_change_authorized",
"main_write_authorized",
"allowed_operations",
"denied_operations",
}
if set(request) != expected_fields:
raise BootstrapError("REQUEST_FIELDS_REJECTED")
if request["action"] != ACTION:
raise BootstrapError("REQUEST_ACTION_REJECTED")
if request["repository"] != "aios/aios":
raise BootstrapError("REPOSITORY_REJECTED")
if request["base_ref"] != "refs/heads/main":
raise BootstrapError("BASE_REF_REJECTED")
if not isinstance(request["base_oid"], str) or not OID_RE.fullmatch(request["base_oid"]):
raise BootstrapError("BASE_OID_REJECTED")
if (
not isinstance(request["target_branch"], str)
or not BRANCH_RE.fullmatch(request["target_branch"])
):
raise BootstrapError("TARGET_BRANCH_REJECTED")
if (
not isinstance(request["task_id"], str)
or not TASK_RE.fullmatch(request["task_id"])
):
raise BootstrapError("TASK_ID_REJECTED")
if request["one_time_use"] is not True:
raise BootstrapError("ONE_TIME_USE_REQUIRED")
expiry = request["expires_in_seconds"]
if (
not isinstance(expiry, int)
or isinstance(expiry, bool)
or not 60 <= expiry <= 900
):
raise BootstrapError("EXPIRY_REJECTED")
for field in (
"production_change_authorized",
"tag_change_authorized",
"baseline_change_authorized",
"main_write_authorized",
):
if request[field] is not False:
raise BootstrapError("CONSEQUENTIAL_CAPABILITY_REJECTED")
if (
not isinstance(request["allowed_operations"], list)
or frozenset(request["allowed_operations"]) != EXPECTED_ALLOWED_OPERATIONS
):
raise BootstrapError("ALLOWED_OPERATIONS_MISMATCH")
if (
not isinstance(request["denied_operations"], list)
or frozenset(request["denied_operations"]) != EXPECTED_DENIED_OPERATIONS
):
raise BootstrapError("DENIED_OPERATIONS_MISMATCH")
return {
"schema": BINDING_SCHEMA,
"owner_identity": identity,
"owner_role": "owner",
"authenticated_scope": scope,
"request": dict(request),
}
def _status_uids(pid: int) -> tuple[int, int, int, int]:
try:
lines = Path(f"/proc/{pid}/status").read_text(encoding="utf-8").splitlines()
except OSError as exc:
raise BootstrapError("PEER_STATUS_UNAVAILABLE") from exc
for line in lines:
if line.startswith("Uid:"):
fields = line.split()[1:]
if len(fields) != 4:
break
return tuple(int(item) for item in fields) # type: ignore[return-value]
raise BootstrapError("PEER_UID_STATUS_INVALID")
def _loginuid(pid: int) -> int:
try:
raw = Path(f"/proc/{pid}/loginuid").read_text(encoding="ascii").strip()
value = int(raw)
except (OSError, ValueError) as exc:
raise BootstrapError("PEER_LOGINUID_UNAVAILABLE") from exc
if value == 4294967295:
raise BootstrapError("PEER_LOGIN_SESSION_REQUIRED")
return value
def _tty_path(pid: int) -> Path:
try:
value = os.readlink(f"/proc/{pid}/fd/0")
except OSError as exc:
raise BootstrapError("PEER_INTERACTIVE_TTY_REQUIRED") from exc
if not (value.startswith("/dev/pts/") or value.startswith("/dev/tty")):
raise BootstrapError("PEER_INTERACTIVE_TTY_REQUIRED")
path = Path(value)
try:
if not stat.S_ISCHR(path.stat().st_mode):
raise BootstrapError("PEER_INTERACTIVE_TTY_REQUIRED")
except OSError as exc:
raise BootstrapError("PEER_INTERACTIVE_TTY_REQUIRED") from exc
return path
def verify_peer_session(pid: int, uid: int, gid: int, owner_identity: str) -> dict[str, Any]:
try:
owner = pwd.getpwnam(owner_identity)
except KeyError as exc:
raise BootstrapError("OWNER_OS_ACCOUNT_MISSING") from exc
if uid != owner.pw_uid:
raise BootstrapError("PEER_OWNER_UID_MISMATCH")
uids = _status_uids(pid)
if any(item != owner.pw_uid for item in uids):
raise BootstrapError("PEER_UID_TRANSITION_REJECTED")
loginuid = _loginuid(pid)
if loginuid != owner.pw_uid:
# This deliberately rejects sudo/runuser from another authenticated
# account; the session itself must have logged in as the Owner.
raise BootstrapError("PEER_LOGINUID_OWNER_MISMATCH")
tty = _tty_path(pid)
if tty.stat().st_uid != owner.pw_uid:
raise BootstrapError("PEER_TTY_OWNER_MISMATCH")
return {
"pid": pid,
"uid": uid,
"gid": gid,
"loginuid": loginuid,
"tty": str(tty),
"owner_identity": owner_identity,
}
def read_frame(connection: socket.socket) -> dict[str, Any]:
data = bytearray()
while len(data) <= MAX_FRAME:
block = connection.recv(4096)
if not block:
break
data.extend(block)
if b"\n" in block:
break
if not data or len(data) > MAX_FRAME:
raise BootstrapError("PROTOCOL_FRAME_INVALID")
raw = bytes(data).split(b"\n", 1)[0]
try:
value = json.loads(raw)
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise BootstrapError("PROTOCOL_JSON_INVALID") from exc
if not isinstance(value, dict):
raise BootstrapError("PROTOCOL_JSON_INVALID")
return value
def write_frame(connection: socket.socket, value: Mapping[str, Any]) -> None:
connection.sendall(canonical_json(dict(value)))
def safe_result(value: Mapping[str, Any]) -> dict[str, Any]:
for key in value:
lowered = str(key).lower()
if any(fragment in lowered for fragment in SENSITIVE_FRAGMENTS):
raise BootstrapError("BROKER_RESPONSE_SENSITIVE_DATA_REJECTED")
allowed = {
"status",
"approval_plan_id",
"grant_id",
"grant_digest_sha256",
"branch_oid",
"branch_ref_created",
"grant_consumed",
"owner_authorization_event_id",
"owner_authorization_reference",
"owner_authorization_digest_sha256",
"owner_authorization_consumed",
"owner_authorization_idempotent",
}
return {key: value.get(key) for key in sorted(allowed) if key in value}
def request_summary(binding: Mapping[str, Any]) -> dict[str, Any]:
request = binding["request"]
return {
"owner_identity": binding["owner_identity"],
"authenticated_scope": binding["authenticated_scope"],
"task_id": request["task_id"],
"repository": request["repository"],
"base_ref": request["base_ref"],
"base_oid": request["base_oid"],
"target_branch": request["target_branch"],
"expires_in_seconds": request["expires_in_seconds"],
"allowed_operations": sorted(request["allowed_operations"]),
"denied_operations": sorted(request["denied_operations"]),
"production_change_authorized": False,
"tag_change_authorized": False,
"baseline_change_authorized": False,
"main_write_authorized": False,
}
def _ensure_runtime_paths(socket_path: Path, state_root: Path, owner_identity: str) -> None:
if os.geteuid() != 0:
raise BootstrapError("SERVER_ROOT_REQUIRED")
owner = pwd.getpwnam(owner_identity)
for directory, mode, uid, gid in (
(socket_path.parent, 0o750, 0, owner.pw_gid),
(state_root, 0o700, 0, 0),
):
if directory.is_symlink():
raise BootstrapError("RUNTIME_DIRECTORY_SYMLINK_REJECTED")
directory.mkdir(parents=True, exist_ok=True)
os.chown(directory, uid, gid)
os.chmod(directory, mode)
if socket_path.exists() or socket_path.is_symlink():
raise BootstrapError("SOCKET_ALREADY_EXISTS")
def _lock(path: Path):
path.parent.mkdir(parents=True, exist_ok=True)
fd = os.open(path, os.O_RDWR | os.O_CREAT | getattr(os, "O_NOFOLLOW", 0), 0o600)
os.fchmod(fd, 0o600)
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
return fd
def invoke_broker(binding: Mapping[str, Any]) -> dict[str, Any]:
# Import only on the root server side. The bootstrap package does not carry
# a second broker/provisioner implementation; it reuses current main.
from owner_authorization_broker import ( # type: ignore
OwnerAuthorizationBroker,
OwnerAuthorizationBrokerConfig,
OwnerPrincipal,
)
config = OwnerAuthorizationBrokerConfig.from_environment()
identity = str(binding["owner_identity"])
scope = str(binding["authenticated_scope"])
if identity not in config.allowed_owner_identities:
raise BootstrapError("OWNER_IDENTITY_NOT_ALLOWED_BY_LIVE_BROKER")
if scope not in config.allowed_intake_scopes:
raise BootstrapError("OWNER_SCOPE_NOT_ALLOWED_BY_LIVE_BROKER")
if config.max_expiry_seconds < int(binding["request"]["expires_in_seconds"]):
raise BootstrapError("LIVE_BROKER_EXPIRY_TOO_SMALL")
principal = OwnerPrincipal(
identity=identity,
role="owner",
authenticated_scope=scope,
token_scopes=frozenset({scope}),
credential_id="",
)
return OwnerAuthorizationBroker(config=config).authorize_and_provision(
binding["request"], principal
)
def serve(binding_path: Path, socket_path: Path, state_root: Path, timeout: int) -> int:
binding = validate_binding(secure_root_file(binding_path))
identity = str(binding["owner_identity"])
_ensure_runtime_paths(socket_path, state_root, identity)
lock_fd = _lock(state_root / "bootstrap.lock")
listener = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
try:
listener.bind(str(socket_path))
owner = pwd.getpwnam(identity)
os.chown(socket_path, 0, owner.pw_gid)
os.chmod(socket_path, 0o660)
listener.listen(1)
listener.settimeout(timeout)
connection, _ = listener.accept()
with connection:
raw_peer = connection.getsockopt(
socket.SOL_SOCKET, socket.SO_PEERCRED, struct.calcsize("3i")
)
pid, uid, gid = struct.unpack("3i", raw_peer)
session = verify_peer_session(pid, uid, gid, identity)
begin = read_frame(connection)
if begin != {"schema": CLIENT_BEGIN_SCHEMA, "action": "BEGIN"}:
raise BootstrapError("CLIENT_BEGIN_REJECTED")
challenge = secrets.token_hex(12)
challenge_issued = time.monotonic()
summary = request_summary(binding)
write_frame(
connection,
{
"schema": CHALLENGE_SCHEMA,
"challenge": challenge,
"expires_in_seconds": CHALLENGE_TTL_SECONDS,
"request_digest_sha256": sha256_json(binding["request"]),
"summary": summary,
},
)
confirm = read_frame(connection)
expected_confirmation = f"AUTHORIZE {challenge}"
if (
confirm.get("schema") != CLIENT_CONFIRM_SCHEMA
or confirm.get("challenge") != challenge
or confirm.get("confirmation") != expected_confirmation
):
raise BootstrapError("OWNER_INTERACTIVE_CONFIRMATION_REJECTED")
if time.monotonic() - challenge_issued > CHALLENGE_TTL_SECONDS:
raise BootstrapError("OWNER_CONFIRMATION_EXPIRED")
result = safe_result(invoke_broker(binding))
receipt = {
"schema": "aios.local-owner-auth-ingress-receipt.v1",
"status": "CONSUMED",
"owner_identity": identity,
"owner_role": "owner",
"authenticated_scope": binding["authenticated_scope"],
"peer": session,
"request_digest_sha256": sha256_json(binding["request"]),
"binding_digest_sha256": sha256_json(binding),
"challenge_digest_sha256": hashlib.sha256(
challenge.encode("ascii")
).hexdigest(),
"broker": result,
"completed_at_unix": int(time.time()),
}
event_id = str(
result.get("owner_authorization_event_id")
or hashlib.sha256(
canonical_json(
{
"peer": session,
"request": sha256_json(binding["request"]),
}
)
).hexdigest()[:32]
)
atomic_json(state_root / "events" / f"{event_id}.json", receipt)
write_frame(
connection,
{
"schema": RESULT_SCHEMA,
"ok": True,
"result": result,
},
)
return 0
except (BootstrapError, TimeoutError, socket.timeout) as exc:
code = exc.code if isinstance(exc, BootstrapError) else "OWNER_BOOTSTRAP_TIMEOUT"
try:
if "connection" in locals():
write_frame(
connection,
{
"schema": RESULT_SCHEMA,
"ok": False,
"error": code,
},
)
except Exception:
pass
atomic_json(
state_root / "last-failure.json",
{
"schema": "aios.local-owner-auth-ingress-failure.v1",
"status": "FAILED",
"error": code,
"binding_digest_sha256": sha256_json(binding),
"failed_at_unix": int(time.time()),
},
)
return 2
finally:
listener.close()
try:
socket_path.unlink()
except FileNotFoundError:
pass
try:
fcntl.flock(lock_fd, fcntl.LOCK_UN)
finally:
os.close(lock_fd)
def client(socket_path: Path) -> int:
# The server performs the authoritative peer/session checks. The client
# also refuses non-interactive execution to make accidental automation hard.
if not sys.stdin.isatty() or not sys.stdout.isatty():
raise BootstrapError("CLIENT_INTERACTIVE_TTY_REQUIRED")
connection = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
connection.connect(str(socket_path))
with connection:
write_frame(connection, {"schema": CLIENT_BEGIN_SCHEMA, "action": "BEGIN"})
challenge = read_frame(connection)
if challenge.get("schema") != CHALLENGE_SCHEMA:
raise BootstrapError("CHALLENGE_INVALID")
summary = challenge.get("summary")
if not isinstance(summary, dict):
raise BootstrapError("CHALLENGE_INVALID")
print("\nAIOS one-shot Owner authorization request:")
print(json.dumps(summary, indent=2, sort_keys=True))
print("\nThis authorizes only the task writer shown above.")
phrase = f"AUTHORIZE {challenge['challenge']}"
typed = input(f"Type exactly:\n{phrase}\n> ").strip()
if typed != phrase:
raise BootstrapError("OWNER_CONFIRMATION_MISMATCH")
write_frame(
connection,
{
"schema": CLIENT_CONFIRM_SCHEMA,
"challenge": challenge["challenge"],
"confirmation": typed,
},
)
result = read_frame(connection)
print(json.dumps(result, indent=2, sort_keys=True))
return 0 if result.get("ok") is True else 3
def parser() -> argparse.ArgumentParser:
value = argparse.ArgumentParser()
mode = value.add_mutually_exclusive_group(required=True)
mode.add_argument("--serve", action="store_true")
mode.add_argument("--client", action="store_true")
value.add_argument("--binding", type=Path)
value.add_argument(
"--socket",
type=Path,
default=Path("/run/aios-owner-bootstrap/owner.sock"),
)
value.add_argument(
"--state-root",
type=Path,
default=Path("/var/lib/aios-gitops/owner-local-auth-ingress"),
)
value.add_argument("--timeout", type=int, default=300)
return value
def main(argv: list[str] | None = None) -> int:
args = parser().parse_args(argv)
if args.serve:
if args.binding is None:
raise SystemExit("--binding is required with --serve")
return serve(args.binding, args.socket, args.state_root, args.timeout)
return client(args.socket)
if __name__ == "__main__":
try:
raise SystemExit(main())
except BootstrapError as exc:
print(json.dumps({"status": "BLOCKED", "error": exc.code}), file=sys.stderr)
raise SystemExit(exc.status)
+225
View File
@@ -0,0 +1,225 @@
#!/usr/bin/env python3
"""AIOS durable autonomous release-campaign authorization.
The campaign receipt represents one explicit Owner policy delegation. It is not
a reusable Owner credential and it is not an unrestricted Git/production token.
Normal release operations are allowed only when all machine gates in the
campaign policy succeed. Formal tag conflicts, non-fast-forward main divergence,
history rewrite, private-signing-material exposure, repository identity conflict
or campaign revocation remain fail-closed/manual-review conditions.
"""
from __future__ import annotations
from dataclasses import dataclass
import hashlib
import json
import os
from pathlib import Path
import re
import stat
from typing import Any, Mapping
OID_RE = re.compile(r"^[0-9a-f]{40}$")
CAMPAIGN_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{3,127}$")
SCHEMA = "aios.release-campaign-authorization.v1"
ALLOWED_ENGINEERING = frozenset({
"create_task_branch",
"create_worktree",
"append_commit",
"push_task_branch",
"repair_task_branch",
"create_pr",
"update_pr",
"run_required_checks",
"publish_truthful_status",
"protected_merge",
})
ALLOWED_RECONCILIATION = frozenset({
"fetch_gitea_main",
"verify_ancestor",
"authority_main_ff_cas",
"refresh_runtime_context",
})
ALLOWED_RELEASE = frozenset({
"deterministic_build",
"retain_artifact",
"materialize_approved_artifact",
"staging_install",
"health_check",
"restart_test",
"rollback_readiness_test",
"create_evidence_commit",
"generate_exact_binding",
"derive_campaign_acceptance",
})
ALLOWED_PRODUCTION = frozenset({
"create_annotated_tag",
"publish_tag",
"verify_tag",
"deploy_exact_artifact",
"verify_production_health",
"update_runtime_registry",
"update_production_baseline",
"update_baseline_sidecar",
})
DENIED = frozenset({
"force_push",
"non_fast_forward_main_write",
"direct_main_push",
"published_history_rewrite",
"formal_tag_rewrite",
"formal_tag_delete",
"overwrite_conflicting_same_name_tag",
"expose_private_signing_material",
"commit_plaintext_secret",
"repository_identity_change",
"weaken_branch_protection",
})
class CampaignError(RuntimeError):
pass
def canonical_json(value: Any) -> bytes:
return (
json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
+ "\n"
).encode("utf-8")
def sha256_bytes(value: bytes) -> str:
return hashlib.sha256(value).hexdigest()
def secure_json(path: Path) -> dict[str, Any]:
if path.is_symlink() or not path.is_file():
raise CampaignError(f"invalid_file:{path}")
st = path.lstat()
if st.st_uid != 0:
raise CampaignError(f"root_owner_required:{path}")
if stat.S_IMODE(st.st_mode) & 0o022:
raise CampaignError(f"group_world_writable_rejected:{path}")
try:
value = json.loads(path.read_text(encoding="utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise CampaignError(f"invalid_json:{path}") from exc
if not isinstance(value, dict):
raise CampaignError(f"json_object_required:{path}")
return value
def _exact_set(value: Any, expected: frozenset[str], name: str) -> None:
if not isinstance(value, list) or frozenset(value) != expected:
raise CampaignError(f"{name}_mismatch")
def validate_policy(value: Mapping[str, Any]) -> dict[str, Any]:
required = {
"schema",
"campaign_id",
"generation",
"owner_identity",
"owner_role",
"repository",
"lineage_anchor_oid",
"control_plane_branch",
"activation_state",
"active",
"revoked",
"valid_until_revoked",
"allowed_engineering",
"allowed_reconciliation",
"allowed_release",
"allowed_production",
"denied",
"normal_owner_interaction_required",
}
if set(value) != required:
raise CampaignError("campaign_fields_invalid")
if value["schema"] != SCHEMA:
raise CampaignError("campaign_schema_invalid")
if not isinstance(value["campaign_id"], str) or not CAMPAIGN_RE.fullmatch(value["campaign_id"]):
raise CampaignError("campaign_id_invalid")
if value["generation"] != 1:
raise CampaignError("campaign_generation_invalid")
if value["owner_identity"] != "aiman" or value["owner_role"] != "owner":
raise CampaignError("campaign_owner_invalid")
if value["repository"] != "aios/aios":
raise CampaignError("campaign_repository_invalid")
if not isinstance(value["lineage_anchor_oid"], str) or not OID_RE.fullmatch(value["lineage_anchor_oid"]):
raise CampaignError("campaign_anchor_invalid")
branch = value["control_plane_branch"]
if branch != "agent/autonomous-release-campaign-v1-20260807":
raise CampaignError("campaign_branch_invalid")
if value["activation_state"] not in {"PENDING_CONTROL_PLANE_MERGE", "ACTIVE"}:
raise CampaignError("campaign_activation_state_invalid")
if value["revoked"] is not False or value["valid_until_revoked"] is not True:
raise CampaignError("campaign_lifecycle_invalid")
if value["normal_owner_interaction_required"] is not False:
raise CampaignError("campaign_owner_interaction_policy_invalid")
if value["activation_state"] == "ACTIVE" and value["active"] is not True:
raise CampaignError("campaign_active_state_invalid")
if value["activation_state"] == "PENDING_CONTROL_PLANE_MERGE" and value["active"] is not False:
raise CampaignError("campaign_pending_state_invalid")
_exact_set(value["allowed_engineering"], ALLOWED_ENGINEERING, "allowed_engineering")
_exact_set(value["allowed_reconciliation"], ALLOWED_RECONCILIATION, "allowed_reconciliation")
_exact_set(value["allowed_release"], ALLOWED_RELEASE, "allowed_release")
_exact_set(value["allowed_production"], ALLOWED_PRODUCTION, "allowed_production")
_exact_set(value["denied"], DENIED, "denied")
return dict(value)
@dataclass(frozen=True)
class CampaignAuthorization:
path: Path
data: dict[str, Any]
digest: str
@classmethod
def load(
cls,
path: Path,
*,
sidecar: Path | None = None,
revocation: Path | None = None,
require_active: bool = True,
) -> "CampaignAuthorization":
data = validate_policy(secure_json(path))
raw = path.read_bytes()
digest = sha256_bytes(raw)
sidecar = sidecar or path.with_suffix(path.suffix + ".sha256")
if not sidecar.is_file() or sidecar.is_symlink():
raise CampaignError("campaign_digest_sidecar_missing")
token = sidecar.read_text(encoding="ascii").strip().split()[0]
if token != digest:
raise CampaignError("campaign_digest_sidecar_mismatch")
if revocation and revocation.exists():
revoked = secure_json(revocation)
if revoked.get("campaign_id") == data["campaign_id"] and revoked.get("revoked") is True:
raise CampaignError("campaign_revoked")
if require_active and (data["activation_state"] != "ACTIVE" or data["active"] is not True):
raise CampaignError("campaign_not_active")
return cls(path=path, data=data, digest=digest)
@property
def campaign_id(self) -> str:
return str(self.data["campaign_id"])
@property
def anchor_oid(self) -> str:
return str(self.data["lineage_anchor_oid"])
def assert_operation(self, operation: str) -> None:
if operation in DENIED:
raise CampaignError(f"campaign_operation_denied:{operation}")
universe = (
ALLOWED_ENGINEERING
| ALLOWED_RECONCILIATION
| ALLOWED_RELEASE
| ALLOWED_PRODUCTION
)
if operation not in universe:
raise CampaignError(f"campaign_operation_out_of_scope:{operation}")
+427
View File
@@ -0,0 +1,427 @@
#!/usr/bin/env python3
"""One-interaction bootstrap for the AIOS autonomous release campaign.
A single real local `aiman` login confirmation does two things atomically at the
policy layer:
1. Writes a root-owned, digest-sidecar protected campaign authorization receipt
in PENDING_CONTROL_PLANE_MERGE state. It grants no immediate production side
effect; the campaign becomes ACTIVE only after the campaign control-plane PR
is merged and machine activation verifies lineage and implementation.
2. Invokes the existing OwnerAuthorizationBroker to issue only the narrow
engineering writer needed to implement that control-plane PR.
No reusable Owner token is minted.
"""
from __future__ import annotations
import argparse
import fcntl
import hashlib
import json
import os
from pathlib import Path
import secrets
import socket
import struct
import tempfile
import time
from typing import Any, Mapping
from owner_local_auth_ingress import (
BootstrapError,
canonical_json,
read_frame,
verify_peer_session,
write_frame,
)
from release_campaign_authorization import (
ALLOWED_ENGINEERING,
ALLOWED_PRODUCTION,
ALLOWED_RECONCILIATION,
ALLOWED_RELEASE,
DENIED,
SCHEMA,
validate_policy,
)
BEGIN = "aios.release-campaign-bootstrap.begin.v1"
CHALLENGE = "aios.release-campaign-bootstrap.challenge.v1"
CONFIRM = "aios.release-campaign-bootstrap.confirm.v1"
RESULT = "aios.release-campaign-bootstrap.result.v1"
TASK_ID = "AIOS_AUTONOMOUS_RELEASE_CAMPAIGN_V1"
TARGET_BRANCH = "agent/autonomous-release-campaign-v1-20260807"
REPOSITORY = "aios/aios"
class CampaignBootstrapError(RuntimeError):
pass
def sha256_file(path: Path) -> str:
h = hashlib.sha256()
with path.open("rb") as stream:
for block in iter(lambda: stream.read(1024 * 1024), b""):
h.update(block)
return h.hexdigest()
def atomic_json_new(path: Path, value: Mapping[str, Any]) -> None:
if path.exists() or path.is_symlink():
raise CampaignBootstrapError(f"target_exists:{path}")
path.parent.mkdir(parents=True, exist_ok=True)
fd, name = tempfile.mkstemp(prefix=f".{path.name}.", dir=path.parent)
try:
os.fchmod(fd, 0o600)
if os.geteuid() == 0:
os.fchown(fd, 0, 0)
with os.fdopen(fd, "wb", closefd=True) as stream:
stream.write(canonical_json(dict(value)))
stream.flush()
os.fsync(stream.fileno())
os.replace(name, path)
os.chmod(path, 0o600)
dfd = os.open(path.parent, os.O_RDONLY | os.O_DIRECTORY)
try:
os.fsync(dfd)
finally:
os.close(dfd)
finally:
try:
os.unlink(name)
except FileNotFoundError:
pass
def atomic_sidecar(path: Path, digest: str, target_name: str) -> None:
if path.exists() or path.is_symlink():
raise CampaignBootstrapError(f"sidecar_exists:{path}")
path.parent.mkdir(parents=True, exist_ok=True)
fd, name = tempfile.mkstemp(prefix=f".{path.name}.", dir=path.parent)
try:
os.fchmod(fd, 0o600)
if os.geteuid() == 0:
os.fchown(fd, 0, 0)
with os.fdopen(fd, "w", encoding="ascii", closefd=True) as stream:
stream.write(f"{digest} {target_name}\n")
stream.flush()
os.fsync(stream.fileno())
os.replace(name, path)
os.chmod(path, 0o600)
dfd = os.open(path.parent, os.O_RDONLY | os.O_DIRECTORY)
try:
os.fsync(dfd)
finally:
os.close(dfd)
finally:
try:
os.unlink(name)
except FileNotFoundError:
pass
def writer_request(base_oid: str) -> dict[str, Any]:
return {
"action": "AUTHORIZE_TASK_WRITER_PROVISIONING",
"task_id": TASK_ID,
"repository": REPOSITORY,
"base_ref": "refs/heads/main",
"base_oid": base_oid,
"target_branch": TARGET_BRANCH,
"one_time_use": True,
"expires_in_seconds": 300,
"production_change_authorized": False,
"tag_change_authorized": False,
"baseline_change_authorized": False,
"main_write_authorized": False,
"allowed_operations": [
"create_worktree",
"create_target_branch_if_absent",
"commit_to_target_branch",
"push_target_branch",
],
"denied_operations": [
"direct_main_push",
"force_push",
"tag_create",
"tag_update",
"tag_delete",
"branch_delete",
"repository_admin",
"baseline_write",
"production_deploy",
"production_marker_write",
"history_rewrite",
],
}
def campaign_policy(campaign_id: str, base_oid: str) -> dict[str, Any]:
value = {
"schema": SCHEMA,
"campaign_id": campaign_id,
"generation": 1,
"owner_identity": "aiman",
"owner_role": "owner",
"repository": REPOSITORY,
"lineage_anchor_oid": base_oid,
"control_plane_branch": TARGET_BRANCH,
"activation_state": "PENDING_CONTROL_PLANE_MERGE",
"active": False,
"revoked": False,
"valid_until_revoked": True,
"allowed_engineering": sorted(ALLOWED_ENGINEERING),
"allowed_reconciliation": sorted(ALLOWED_RECONCILIATION),
"allowed_release": sorted(ALLOWED_RELEASE),
"allowed_production": sorted(ALLOWED_PRODUCTION),
"denied": sorted(DENIED),
"normal_owner_interaction_required": False,
}
return validate_policy(value)
def invoke_writer_broker(base_oid: str) -> dict[str, Any]:
from owner_authorization_broker import (
OwnerAuthorizationBroker,
OwnerAuthorizationBrokerConfig,
OwnerPrincipal,
)
config = OwnerAuthorizationBrokerConfig.from_environment()
if "aiman" not in config.allowed_owner_identities:
raise CampaignBootstrapError("aiman_not_allowed_by_live_broker")
if "commit:create" not in config.allowed_intake_scopes:
raise CampaignBootstrapError("commit_create_not_allowed_by_live_broker")
principal = OwnerPrincipal(
identity="aiman",
role="owner",
authenticated_scope="commit:create",
token_scopes=frozenset({"commit:create"}),
credential_id="",
)
return OwnerAuthorizationBroker(config=config).authorize_and_provision(
writer_request(base_oid),
principal,
)
def safe_broker_result(value: Mapping[str, Any]) -> dict[str, Any]:
allowed = {
"status",
"approval_plan_id",
"grant_id",
"grant_digest_sha256",
"branch_oid",
"branch_ref_created",
"grant_consumed",
"owner_authorization_event_id",
"owner_authorization_reference",
"owner_authorization_digest_sha256",
"owner_authorization_consumed",
"owner_authorization_idempotent",
}
lowered = " ".join(str(k).lower() for k in value)
for secret_word in ("token", "password", "private_key", "secret_value", "bearer"):
if secret_word in lowered:
raise CampaignBootstrapError("broker_response_sensitive_field_rejected")
return {k: value[k] for k in sorted(allowed) if k in value}
def serve(
*,
socket_path: Path,
state_root: Path,
base_oid: str,
campaign_id: str,
timeout: int,
) -> int:
if os.geteuid() != 0:
raise CampaignBootstrapError("server_root_required")
if len(base_oid) != 40 or any(c not in "0123456789abcdef" for c in base_oid):
raise CampaignBootstrapError("base_oid_invalid")
policy = campaign_policy(campaign_id, base_oid)
campaign_dir = state_root / campaign_id
campaign_path = campaign_dir / "authorization.json"
sidecar_path = campaign_dir / "authorization.json.sha256"
socket_path.parent.mkdir(parents=True, exist_ok=True)
if socket_path.exists() or socket_path.is_symlink():
raise CampaignBootstrapError("socket_already_exists")
import pwd
owner = pwd.getpwnam("aiman")
os.chown(socket_path.parent, 0, owner.pw_gid)
os.chmod(socket_path.parent, 0o750)
listener = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
try:
listener.bind(str(socket_path))
os.chown(socket_path, 0, owner.pw_gid)
os.chmod(socket_path, 0o660)
listener.listen(1)
listener.settimeout(timeout)
conn, _ = listener.accept()
with conn:
raw = conn.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, struct.calcsize("3i"))
pid, uid, gid = struct.unpack("3i", raw)
peer = verify_peer_session(pid, uid, gid, "aiman")
if read_frame(conn) != {"schema": BEGIN, "action": "BEGIN"}:
raise CampaignBootstrapError("client_begin_invalid")
challenge = secrets.token_hex(12)
issued = time.monotonic()
summary = {
"campaign_id": campaign_id,
"repository": REPOSITORY,
"lineage_anchor_oid": base_oid,
"control_plane_branch": TARGET_BRANCH,
"normal_owner_interaction_after_activation": False,
"allowed_engineering": sorted(ALLOWED_ENGINEERING),
"allowed_reconciliation": sorted(ALLOWED_RECONCILIATION),
"allowed_release": sorted(ALLOWED_RELEASE),
"allowed_production": sorted(ALLOWED_PRODUCTION),
"denied": sorted(DENIED),
"writer_request": writer_request(base_oid),
}
write_frame(
conn,
{
"schema": CHALLENGE,
"challenge": challenge,
"expires_in_seconds": 120,
"summary": summary,
},
)
confirm = read_frame(conn)
phrase = f"AUTHORIZE CAMPAIGN {campaign_id} {challenge}"
if (
confirm.get("schema") != CONFIRM
or confirm.get("challenge") != challenge
or confirm.get("confirmation") != phrase
):
raise CampaignBootstrapError("campaign_confirmation_invalid")
if time.monotonic() - issued > 120:
raise CampaignBootstrapError("campaign_confirmation_expired")
# First issue the narrow engineering writer. If this fails, do not
# persist a campaign authorization that cannot be implemented.
broker = safe_broker_result(invoke_writer_broker(base_oid))
if broker.get("status") != "READY_FOR_CODE_AUTHORING":
raise CampaignBootstrapError("writer_not_ready")
atomic_json_new(campaign_path, policy)
digest = sha256_file(campaign_path)
atomic_sidecar(sidecar_path, digest, campaign_path.name)
receipt = {
"schema": "aios.release-campaign-bootstrap-receipt.v1",
"status": "CAMPAIGN_PENDING_CONTROL_PLANE_MERGE",
"campaign_id": campaign_id,
"campaign_authorization_path": str(campaign_path),
"campaign_authorization_sha256": digest,
"owner_identity": "aiman",
"owner_role": "owner",
"authentication": {
"method": "local_so_peercred_loginuid_tty",
"uid": peer["uid"],
"loginuid": peer["loginuid"],
"tty": peer["tty"],
},
"writer": broker,
"reusable_owner_token_created": False,
"production_side_effect_executed": False,
"completed_at_unix": int(time.time()),
}
receipt_path = campaign_dir / "bootstrap-receipt.json"
atomic_json_new(receipt_path, receipt)
write_frame(
conn,
{
"schema": RESULT,
"ok": True,
"campaign_id": campaign_id,
"campaign_authorization_path": str(campaign_path),
"campaign_authorization_sha256": digest,
"writer": broker,
},
)
return 0
finally:
listener.close()
try:
socket_path.unlink()
except FileNotFoundError:
pass
def client(socket_path: Path) -> int:
import sys
if not sys.stdin.isatty() or not sys.stdout.isatty():
raise CampaignBootstrapError("interactive_tty_required")
conn = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
conn.connect(str(socket_path))
with conn:
write_frame(conn, {"schema": BEGIN, "action": "BEGIN"})
challenge = read_frame(conn)
if challenge.get("schema") != CHALLENGE:
raise CampaignBootstrapError("challenge_invalid")
print("\nAIOS Autonomous Release Campaign authorization")
print(json.dumps(challenge["summary"], indent=2, sort_keys=True))
phrase = (
f"AUTHORIZE CAMPAIGN "
f"{challenge['summary']['campaign_id']} "
f"{challenge['challenge']}"
)
typed = input(f"\nType exactly:\n{phrase}\n> ").strip()
if typed != phrase:
raise CampaignBootstrapError("confirmation_mismatch")
write_frame(
conn,
{
"schema": CONFIRM,
"challenge": challenge["challenge"],
"confirmation": typed,
},
)
result = read_frame(conn)
print(json.dumps(result, indent=2, sort_keys=True))
return 0 if result.get("ok") is True else 3
def main() -> int:
p = argparse.ArgumentParser()
mode = p.add_mutually_exclusive_group(required=True)
mode.add_argument("--serve", action="store_true")
mode.add_argument("--client", action="store_true")
p.add_argument("--socket", type=Path, default=Path("/run/aios-release-campaign/owner.sock"))
p.add_argument(
"--state-root",
type=Path,
default=Path("/var/lib/aios-gitops/release-campaigns"),
)
p.add_argument("--base-oid")
p.add_argument("--campaign-id")
p.add_argument("--timeout", type=int, default=300)
a = p.parse_args()
if a.serve:
if not a.base_oid or not a.campaign_id:
raise CampaignBootstrapError("server_binding_required")
return serve(
socket_path=a.socket,
state_root=a.state_root,
base_oid=a.base_oid,
campaign_id=a.campaign_id,
timeout=a.timeout,
)
return client(a.socket)
if __name__ == "__main__":
try:
raise SystemExit(main())
except (CampaignBootstrapError, BootstrapError) as exc:
print(json.dumps({"status": "BLOCKED", "reason": str(exc)}, sort_keys=True))
raise SystemExit(2)
+113
View File
@@ -0,0 +1,113 @@
#!/usr/bin/env python3
"""Durable state model for the AIOS autonomous release campaign."""
from __future__ import annotations
from dataclasses import dataclass
import json
import os
from pathlib import Path
import tempfile
from typing import Any
STATES = (
"DEVELOPMENT",
"VALIDATING",
"PR_READY",
"MERGING",
"MAIN_RECONCILING",
"POSTMERGE_BUILDING",
"STAGING",
"EVIDENCE_CLOSING",
"BINDING_ACCEPTING",
"TAGGING",
"DEPLOYING",
"VERIFYING",
"REGISTRY_FINALIZING",
"BASELINE_FINALIZING",
"PRODUCTION_DEPLOYED",
)
RECOVERABLE = "RECOVERABLE_FAILURE"
RECONCILIATION = "RECONCILIATION_REQUIRED"
MANUAL = "MANUAL_OWNER_REVIEW_REQUIRED"
NEXT = {STATES[i]: STATES[i + 1] for i in range(len(STATES) - 1)}
MANUAL_REASONS = frozenset({
"formal_same_name_tag_conflict",
"force_push_required",
"published_history_rewrite_required",
"non_fast_forward_main_divergence",
"private_signing_material_exposure_required",
"repository_identity_conflict",
"campaign_revoked",
"operation_outside_campaign_scope",
})
class OrchestratorError(RuntimeError):
pass
@dataclass
class ReleaseState:
version: str
state: str = "DEVELOPMENT"
last_stable_state: str = "DEVELOPMENT"
recoverable_reason: str | None = None
def advance(self) -> None:
if self.state not in NEXT:
raise OrchestratorError(f"cannot_advance:{self.state}")
self.last_stable_state = self.state
self.state = NEXT[self.state]
self.recoverable_reason = None
def recoverable_failure(self, reason: str) -> None:
self.last_stable_state = self.state
self.state = RECOVERABLE
self.recoverable_reason = reason
def reconciliation_required(self, reason: str) -> None:
self.last_stable_state = self.state
self.state = RECONCILIATION
self.recoverable_reason = reason
def manual_review(self, reason: str) -> None:
if reason not in MANUAL_REASONS:
raise OrchestratorError("reason_not_manual_red_line")
self.last_stable_state = self.state
self.state = MANUAL
self.recoverable_reason = reason
def resume(self) -> None:
if self.state not in {RECOVERABLE, RECONCILIATION}:
raise OrchestratorError("state_not_resumable")
self.state = self.last_stable_state
def as_dict(self) -> dict[str, Any]:
return {
"schema": "aios.autonomous-release-state.v1",
"version": self.version,
"state": self.state,
"last_stable_state": self.last_stable_state,
"recoverable_reason": self.recoverable_reason,
"owner_interaction_required": self.state == MANUAL,
}
def atomic_state(path: Path, state: ReleaseState) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
fd, name = tempfile.mkstemp(prefix=f".{path.name}.", dir=path.parent)
try:
os.fchmod(fd, 0o600)
with os.fdopen(fd, "w", encoding="utf-8", closefd=True) as stream:
json.dump(state.as_dict(), stream, sort_keys=True, indent=2)
stream.write("\n")
stream.flush()
os.fsync(stream.fileno())
os.replace(name, path)
finally:
try:
os.unlink(name)
except FileNotFoundError:
pass
+410
View File
@@ -0,0 +1,410 @@
#!/usr/bin/env python3
"""Atomic Registry/Baseline finalizer for an already ACTIVE exact release.
This adapter never writes Git refs. It is intended for the AIOS runtime
Registry and authoritative runtime-anchor Baseline only.
Registry is finalized before Baseline. Baseline apply independently verifies:
- exact Authority annotated tag and peeled target
- exact ACTIVE production health
- exact V2R1C7 Registry record
- exact previous Baseline file SHA-256 (CAS)
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import re
import stat
import subprocess
import sys
import tempfile
import time
import urllib.error
import urllib.request
from pathlib import Path
from typing import Any, Mapping
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
PLAN_RE = re.compile(r"^sha256:[0-9a-f]{64}$")
def sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as source:
for block in iter(lambda: source.read(1024 * 1024), b""):
digest.update(block)
return digest.hexdigest()
def atomic_json(path: Path, value: Mapping[str, Any], mode: int = 0o640) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
# Existing runtime-authority files may be root:gitops and read-only to
# non-root readers. Preserve their owner/group/mode across os.replace().
if path.exists():
metadata = path.lstat()
if not stat.S_ISREG(metadata.st_mode) or path.is_symlink():
raise RuntimeError("ATOMIC_TARGET_NOT_REGULAR")
target_mode = stat.S_IMODE(metadata.st_mode)
target_uid = metadata.st_uid
target_gid = metadata.st_gid
else:
parent = path.parent.stat()
target_mode = mode
target_uid = 0 if os.geteuid() == 0 else os.geteuid()
target_gid = parent.st_gid
fd, name = tempfile.mkstemp(prefix=f".{path.name}.", dir=str(path.parent))
try:
with os.fdopen(fd, "w", encoding="utf-8") as output:
json.dump(dict(value), output, sort_keys=True, indent=2)
output.write("\n")
output.flush()
os.fsync(output.fileno())
os.chmod(name, target_mode)
if os.geteuid() == 0:
os.chown(name, target_uid, target_gid)
os.replace(name, path)
dfd = os.open(path.parent, os.O_RDONLY | os.O_DIRECTORY)
try:
os.fsync(dfd)
finally:
os.close(dfd)
finally:
try:
os.unlink(name)
except FileNotFoundError:
pass
def load_request() -> dict[str, Any]:
value = json.load(sys.stdin)
if not isinstance(value, dict) or value.get("schema") != "aios.production-executor-request.v1":
raise ValueError("request_schema_invalid")
if value.get("repository_identity") != "aios/aios":
raise ValueError("repository_identity_invalid")
if not re.fullmatch(r"V\d+R\d+C\d+(?:-[A-Za-z0-9_.-]+)?", str(value.get("version", ""))):
raise ValueError("version_invalid")
for name in ("target_oid", "evidence_closure_oid"):
if not OID_RE.fullmatch(str(value.get(name, ""))):
raise ValueError(f"{name}_invalid")
if not SHA_RE.fullmatch(str(value.get("artifact_sha256", ""))):
raise ValueError("artifact_sha256_invalid")
if not PLAN_RE.fullmatch(str(value.get("approval_plan_id", ""))):
raise ValueError("approval_plan_id_invalid")
if "tag_oid" in value and not OID_RE.fullmatch(str(value["tag_oid"])):
raise ValueError("tag_oid_invalid")
return value
def exact_response(value: Mapping[str, Any], status: str, **extra: Any) -> dict[str, Any]:
return {
"status": status,
"version": value["version"],
"target_oid": value["target_oid"],
"evidence_closure_oid": value["evidence_closure_oid"],
"artifact_sha256": value["artifact_sha256"],
"approval_plan_id": value["approval_plan_id"],
**extra,
}
def git(repository: Path, *args: str, check: bool = True) -> subprocess.CompletedProcess[str]:
return subprocess.run(
["git", f"--git-dir={repository}", *args],
check=check,
capture_output=True,
text=True,
timeout=30,
env={"PATH": os.environ.get("PATH", "")},
)
def verify_local_tag(repository: Path, value: Mapping[str, Any]) -> tuple[str, str]:
tag_oid = git(repository, "rev-parse", "--verify", f"refs/tags/{value['version']}").stdout.strip()
peeled = git(repository, "rev-parse", "--verify", f"refs/tags/{value['version']}^{{commit}}").stdout.strip()
if tag_oid == peeled or peeled != value["target_oid"]:
raise RuntimeError("FORMAL_TAG_BINDING_MISMATCH")
requested = value.get("tag_oid")
if requested and requested != tag_oid:
raise RuntimeError("FORMAL_TAG_OBJECT_MISMATCH")
return tag_oid, peeled
def health(url: str, value: Mapping[str, Any]) -> dict[str, Any]:
try:
with urllib.request.urlopen(url, timeout=5) as response:
body = json.loads(response.read().decode("utf-8"))
status = response.status
except (urllib.error.URLError, TimeoutError, ValueError) as exc:
raise RuntimeError("PRODUCTION_HEALTH_UNAVAILABLE") from exc
if not (
status == 200
and body.get("version") == value["version"]
and body.get("commit_oid") == value["target_oid"]
and body.get("artifact_sha256") == value["artifact_sha256"]
and body.get("deployment_state") == "ACTIVE"
and body.get("production_deployed") is True
):
raise RuntimeError("PRODUCTION_HEALTH_BINDING_MISMATCH")
return body
def registry_record(value: Mapping[str, Any], tag_oid: str, production_health: Mapping[str, Any]) -> dict[str, Any]:
return {
"schema": "aios.production-registry.v2",
"version": value["version"],
"repository": "aios/aios",
"target_oid": value["target_oid"],
"tag": value["version"],
"tag_oid": tag_oid,
"tag_object": tag_oid,
"peeled_target_oid": value["target_oid"],
"evidence_closure_oid": value["evidence_closure_oid"],
"artifact_sha256": value["artifact_sha256"],
"approval_plan_id": value["approval_plan_id"],
"production_service": value["production_service"],
"production_port": value["production_port"],
"registered": True,
"enabled": True,
"production_deployed": True,
"status": "PRODUCTION_DEPLOYED",
"deployment_state": production_health["deployment_state"],
"updated_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
}
def registry_matches(record: Mapping[str, Any], value: Mapping[str, Any], tag_oid: str) -> bool:
return all(
record.get(key) == expected
for key, expected in {
"version": value["version"],
"target_oid": value["target_oid"],
"tag_oid": tag_oid,
"evidence_closure_oid": value["evidence_closure_oid"],
"artifact_sha256": value["artifact_sha256"],
"approval_plan_id": value["approval_plan_id"],
"production_service": value["production_service"],
"production_port": value["production_port"],
"production_deployed": True,
}.items()
) and record.get("registered") is True and record.get("enabled") is True
def baseline_matches(record: Mapping[str, Any], value: Mapping[str, Any], tag_oid: str) -> bool:
current = record.get("current_baseline") or record.get("version")
return current == value["version"] and all(
record.get(key) == expected
for key, expected in {
"target_oid": value["target_oid"],
"tag_oid": tag_oid,
"evidence_closure_oid": value["evidence_closure_oid"],
"artifact_sha256": value["artifact_sha256"],
"approval_plan_id": value["approval_plan_id"],
"production_service": value["production_service"],
"production_port": value["production_port"],
"status": "PRODUCTION_DEPLOYED",
}.items()
)
def registry_probe(args: argparse.Namespace, value: Mapping[str, Any]) -> dict[str, Any]:
tag_oid, _ = verify_local_tag(args.repository, value)
path = args.registry_root / f"{value['version']}.json"
if not path.exists():
return exact_response(value, "NOT_APPLIED", tag_oid=tag_oid)
record = json.loads(path.read_text(encoding="utf-8"))
if registry_matches(record, value, tag_oid):
return exact_response(
value,
"VERIFIED",
tag_oid=tag_oid,
registry_path=str(path),
registry_sha256=sha256_file(path),
)
return exact_response(value, "CONFLICT", tag_oid=tag_oid, reason="registry_binding_mismatch")
def registry_apply(args: argparse.Namespace, value: Mapping[str, Any]) -> dict[str, Any]:
existing = registry_probe(args, value)
if existing["status"] in {"VERIFIED", "CONFLICT"}:
return existing
tag_oid, _ = verify_local_tag(args.repository, value)
production_health = health(args.health_url, value)
path = args.registry_root / f"{value['version']}.json"
atomic_json(path, registry_record(value, tag_oid, production_health))
return registry_probe(args, value)
def secure_baseline_authority_file(path: Path) -> None:
if not path.is_file() or path.is_symlink():
raise RuntimeError("BASELINE_AUTHORITY_FILE_INVALID")
baseline_stat = path.lstat()
if baseline_stat.st_uid != 0 or baseline_stat.st_mode & 0o022:
raise RuntimeError("BASELINE_AUTHORITY_PERMISSIONS_INVALID")
def baseline_probe(args: argparse.Namespace, value: Mapping[str, Any]) -> dict[str, Any]:
tag_oid, _ = verify_local_tag(args.repository, value)
secure_baseline_authority_file(args.baseline_path)
record = json.loads(args.baseline_path.read_text(encoding="utf-8"))
if baseline_matches(record, value, tag_oid):
return exact_response(
value,
"VERIFIED",
tag_oid=tag_oid,
baseline_path=str(args.baseline_path),
baseline_sha256=sha256_file(args.baseline_path),
)
current = record.get("current_baseline") or record.get("version")
current_sha = sha256_file(args.baseline_path)
if (
current == args.expected_previous_baseline
and current_sha == args.expected_previous_baseline_sha256
):
return exact_response(
value,
"NOT_APPLIED",
tag_oid=tag_oid,
previous_baseline=current,
previous_baseline_sha256=current_sha,
)
return exact_response(
value,
"CONFLICT",
tag_oid=tag_oid,
reason="baseline_precondition_or_binding_mismatch",
observed_baseline=current,
observed_sha256=current_sha,
)
def baseline_apply(args: argparse.Namespace, value: Mapping[str, Any]) -> dict[str, Any]:
existing = baseline_probe(args, value)
if existing["status"] in {"VERIFIED", "CONFLICT"}:
return existing
tag_oid, _ = verify_local_tag(args.repository, value)
production_health = health(args.health_url, value)
registry_path = args.registry_root / f"{value['version']}.json"
if not registry_path.is_file():
raise RuntimeError("REGISTRY_NOT_FINALIZED")
registry = json.loads(registry_path.read_text(encoding="utf-8"))
if not registry_matches(registry, value, tag_oid):
raise RuntimeError("REGISTRY_BINDING_NOT_VERIFIED")
previous = json.loads(args.baseline_path.read_text(encoding="utf-8"))
if sha256_file(args.baseline_path) != args.expected_previous_baseline_sha256:
raise RuntimeError("BASELINE_CAS_MISMATCH")
generation = max(
int(previous.get("generation", 0) or 0),
int(previous.get("authority_generation", 0) or 0),
) + 1
record = {
"schema": "aios.production-baseline.v2",
"source_identity": "aios-gitops-runtime-anchor-baseline-authority-v1",
"current_baseline": value["version"],
"version": value["version"],
"previous_baseline": args.expected_previous_baseline,
"previous_version": args.expected_previous_baseline,
"status": "PRODUCTION_DEPLOYED",
"target_oid": value["target_oid"],
"peeled_target_oid": value["target_oid"],
"tag_oid": tag_oid,
"evidence_closure_oid": value["evidence_closure_oid"],
"approval_plan_id": value["approval_plan_id"],
"artifact_sha256": value["artifact_sha256"],
"production_artifact_sha256": value["artifact_sha256"],
"production_service": value["production_service"],
"production_port": value["production_port"],
"production_deployed": True,
"deployment_state": production_health["deployment_state"],
"registry_digest": sha256_file(registry_path),
"registry_source": str(registry_path),
"generation": generation,
"authority_generation": generation,
# The actual previously deployed production baseline is the rollback
# target. V2R1C6 has a formal tag but was never production-deployed.
"rollback_version": args.expected_previous_baseline,
"rollback_tag_oid": previous.get("tag_oid"),
"rollback_target_oid": previous.get("target_oid"),
"promoted_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"promoted_by": "aios-approved-production-release-executor",
}
atomic_json(args.baseline_path, record)
return baseline_probe(args, value)
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument(
"--mode",
choices=("registry", "baseline"),
default=os.environ.get("AIOS_RELEASE_METADATA_MODE"),
required=os.environ.get("AIOS_RELEASE_METADATA_MODE") is None,
)
parser.add_argument(
"--repository",
type=Path,
default=Path(os.environ["AIOS_GITOPS_AUTHORITATIVE_GIT_DIR"]),
)
parser.add_argument(
"--registry-root",
type=Path,
default=Path(os.environ["AIOS_RUNTIME_REGISTRY_ROOT"]),
)
parser.add_argument(
"--baseline-path",
type=Path,
default=Path(os.environ["AIOS_RUNTIME_BASELINE_PATH"]),
)
parser.add_argument(
"--expected-previous-baseline",
default=os.environ.get("AIOS_PREVIOUS_BASELINE", "V2R1C5"),
)
parser.add_argument(
"--expected-previous-baseline-sha256",
default=os.environ.get("AIOS_PREVIOUS_BASELINE_SHA256"),
required=os.environ.get("AIOS_PREVIOUS_BASELINE_SHA256") is None,
)
parser.add_argument(
"--health-url",
default=os.environ.get("AIOS_RELEASE_HEALTH_URL", "http://127.0.0.1:18927/health"),
)
args = parser.parse_args()
value = load_request()
operation = value.get("operation")
if args.mode == "registry":
if operation == "probe":
result = registry_probe(args, value)
elif operation == "apply":
result = registry_apply(args, value)
else:
raise ValueError("unsupported_operation")
else:
if operation == "probe":
result = baseline_probe(args, value)
elif operation == "apply":
result = baseline_apply(args, value)
else:
raise ValueError("unsupported_operation")
print(json.dumps(result, sort_keys=True))
return 0
if __name__ == "__main__":
try:
raise SystemExit(main())
except Exception as exc:
print(
json.dumps(
{"status": "FAIL_CLOSED", "error": type(exc).__name__, "reason": str(exc)},
sort_keys=True,
)
)
raise SystemExit(2)
+9 -1
View File
@@ -11,6 +11,7 @@ from dataclasses import asdict, dataclass, field
from typing import Any
from .search_gateway_orchestrator import SearchGatewayOrchestrator
from .trace_tool_credential_governance import GovernanceActor, TraceContext
@dataclass(frozen=True)
@@ -43,6 +44,9 @@ class SearchGatewayAgentAdapter:
domain_profiles: list[str] | None = None,
user_scope: str = "public",
correlation_id: str | None = None,
*,
actor: GovernanceActor | dict[str, Any] | None = None,
trace: TraceContext | None = None,
) -> dict[str, Any]:
payload = {
"query": query,
@@ -59,7 +63,11 @@ class SearchGatewayAgentAdapter:
"session_profile": context.session_profile,
"correlation_id": correlation_id,
}
return self.orchestrator.execute(payload)
if actor is None and trace is None:
# Preserve adapter compatibility; the governed orchestrator itself
# still fails closed when trusted context is absent.
return self.orchestrator.execute(payload)
return self.orchestrator.execute(payload, actor=actor, trace=trace)
class OpenClawSearchClient(SearchGatewayAgentAdapter):
+70 -4
View File
@@ -13,10 +13,18 @@ from __future__ import annotations
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
import hashlib
import json
from typing import Any, Protocol
from uuid import uuid4
from .search_profile_resolver import DomainQueryProfileRegistry, SearchProfile, SearchProfileResolver
from .trace_tool_credential_governance import (
GovernanceActor,
GovernanceError,
GovernanceStore,
TraceContext,
)
def utc_now() -> str:
@@ -156,13 +164,25 @@ class SearchPolicyGuard:
class SearchGateway:
def __init__(self, providers: list[SearchProvider] | None = None):
def __init__(self, providers: list[SearchProvider] | None = None, *, governance_store: GovernanceStore | None = None):
self.providers = providers or []
self.governance_store = governance_store
self.profile_resolver = SearchProfileResolver()
self.planner = QueryPlanner()
self.guard = SearchPolicyGuard()
def search(self, request: SearchRequest) -> SearchResponse:
def search(self, request: SearchRequest, *, actor: GovernanceActor | dict[str, Any] | None = None, trace: TraceContext | None = None) -> SearchResponse:
trusted_actor: GovernanceActor | None = None
if self.governance_store is not None:
if actor is None or trace is None:
raise GovernanceError("TOOL_MCP_AUTHZ_DENY")
trusted_actor = GovernanceActor.from_value(actor)
if request.requester != trusted_actor.identity:
raise GovernanceError("TOOL_MCP_AUTHZ_DENY", field="requester")
if request.correlation_id is not None and request.correlation_id not in {
trace.trace_id, trace.correlation_id
}:
raise GovernanceError("TOOL_SCOPE_PROPAGATION", field="correlation_id")
request_id = request.correlation_id or f"search_{uuid4().hex[:12]}"
profile = self.profile_resolver.resolve(request.group_profile, request.user_profile, request.session_profile)
expanded_queries = self.planner.expand(request, profile)
@@ -184,14 +204,60 @@ class SearchGateway:
attempts: list[SearchProviderAttempt] = []
providers = self._ordered_providers(profile)
for provider in providers:
if request.authorized_sources and provider.name not in request.authorized_sources:
attempts.append(SearchProviderAttempt(provider=provider.name, status="denied", reason="SOURCE_AUTHZ_DENY"))
continue
if request.user_scope != "public" and provider.name not in request.authorized_sources:
attempts.append(SearchProviderAttempt(provider=provider.name, status="denied", reason="SOURCE_AUTHZ_REQUIRED"))
continue
try:
if self.governance_store is not None:
assert trusted_actor is not None and trace is not None
input_digest = hashlib.sha256(
json.dumps(
{"query": request.query, "intent": request.intent, "provider": provider.name},
sort_keys=True, separators=(",", ":"), ensure_ascii=False,
).encode("utf-8")
).hexdigest()
invocation_key = hashlib.sha256(
f"{trace.trace_id}:{trace.span_id}:{provider.name}:{input_digest}".encode("utf-8")
).hexdigest()
authorization = self.governance_store.authorize_tool(
actor=trusted_actor,
tool_name=f"search.{provider.name}",
trace=trace,
requested_scope=trace.scope,
input_digest=input_digest,
invocation_key=invocation_key,
)
if authorization.get("idempotent"):
raise GovernanceError("TOOL_INVOCATION_REPLAY")
provider_evidence, attempt = provider.search(request, expanded_queries)
if self.governance_store is not None:
result_digest = hashlib.sha256(
json.dumps(
{
"evidence": [item.to_dict() for item in provider_evidence],
"attempt": attempt.to_dict(),
},
sort_keys=True, separators=(",", ":"), ensure_ascii=False,
).encode("utf-8")
).hexdigest()
self.governance_store.record_tool_receipt(
invocation_key=invocation_key,
result_digest=result_digest,
actor=trusted_actor,
trace=trace,
)
evidence.extend(provider_evidence)
attempts.append(attempt)
if provider_evidence:
break
except Exception as exc:
attempts.append(SearchProviderAttempt(provider=getattr(provider, "name", "unknown"), status="failed", reason=str(exc)))
except GovernanceError:
raise
except Exception:
# Provider exceptions may contain URLs, tokens, or local paths.
attempts.append(SearchProviderAttempt(provider=getattr(provider, "name", "unknown"), status="failed", reason="PROVIDER_FAILURE"))
if evidence:
return SearchResponse(
+8 -6
View File
@@ -7,27 +7,29 @@ try:
except Exception:
APIRouter = None
from .search_gateway import SearchGateway, SearchRequest
from .search_gateway import SearchRequest
from .search_gateway_orchestrator import SearchGatewayOrchestrator
from .trace_tool_credential_governance import GovernanceActor, TraceContext
from .search_gateway_dashboard import SearchGatewayDashboard
gateway = SearchGateway()
orchestrator = SearchGatewayOrchestrator()
dashboard = SearchGatewayDashboard()
def search_gateway_request(payload: dict[str, Any]) -> dict[str, Any]:
request = SearchRequest(**payload)
return gateway.search(request).to_dict()
def search_gateway_request(payload: dict[str, Any], *, actor: GovernanceActor | dict[str, Any] | None = None, trace: TraceContext | None = None) -> dict[str, Any]:
return orchestrator.execute(payload, actor=actor, trace=trace)
def get_search_gateway_status() -> dict[str, Any]:
return {
"module": "search-gateway",
"status": "ready",
"providers_configured": len(gateway.providers),
"providers_configured": len(orchestrator.gateway.providers),
"policy_guard": "enabled",
"profile_customization": "enabled",
"fallback_required_when_provider_unavailable": True,
"governance": "required",
}
@@ -3,24 +3,49 @@
from __future__ import annotations
from typing import Any
import os
from .search_audit_store import SearchAuditStore
from .search_event_bridge import SearchEventBridge
from .search_gateway import SearchGateway, SearchRequest
from .search_provider_registry import SearchProviderRegistry
from .trace_tool_credential_governance import (
GovernanceActor,
GovernanceScope,
GovernanceStore,
ToolDescriptor,
TraceContext,
)
class SearchGatewayOrchestrator:
def __init__(self):
def __init__(self, *, governance_store: GovernanceStore | None = None):
self.registry = SearchProviderRegistry()
self.audit = SearchAuditStore()
self.events = SearchEventBridge()
self.gateway = SearchGateway(self.registry.ordered())
self.governance = governance_store or GovernanceStore(
os.environ.get("AIOS_SEARCH_GOVERNANCE_DB", "runtime/search-governance.sqlite3")
)
registrar = GovernanceActor(
"search-runtime-bootstrap", GovernanceScope("system"), ("system.admin",)
)
for provider in self.registry.ordered():
self.governance.register_tool(
ToolDescriptor(
f"search.{provider.name}", "tool", GovernanceScope("global"), ("tool.search",)
),
registrar,
)
self.gateway = SearchGateway(self.registry.ordered(), governance_store=self.governance)
def execute(self, payload: dict[str, Any]) -> dict[str, Any]:
def execute(self, payload: dict[str, Any], *, actor: GovernanceActor | dict[str, Any] | None = None, trace: TraceContext | None = None) -> dict[str, Any]:
if actor is None or trace is None:
from .trace_tool_credential_governance import GovernanceError
raise GovernanceError("TOOL_MCP_AUTHZ_DENY")
self.governance.record_trace(trace, actor)
request = SearchRequest(**payload)
self.events.search_started(request.to_dict())
response = self.gateway.search(request)
response = self.gateway.search(request, actor=actor, trace=trace)
result = response.to_dict()
for attempt in result.get("attempts", []):
self.events.provider_attempted(result["request_id"], attempt)
@@ -35,6 +60,7 @@ class SearchGatewayOrchestrator:
"registry": self.registry.status(),
"audit": "enabled",
"events": "enabled",
"governance": "required",
}
+56 -26
View File
@@ -34,6 +34,7 @@ from .models import (
)
from .policy import PolicyEngine
from .store import TaskPlaneStore
from .scope import bind_task_scope
from .worker_registry import WorkerRegistry
@@ -68,28 +69,54 @@ class TaskPlaneService:
self.delegation = DelegationEngine(store)
self.store.probe_attestation_verifier = self.policy.probe_attestation_verifier
def create_task(self, body: dict[str, Any], idempotency_key: str | None = None) -> dict[str, Any]:
def _authorized_task(self, task_id: str, actor: str, action: str) -> TaskRecord:
task = self.store.get_task(task_id, self.policy.actor_context(actor))
self.policy.assert_allowed(action, actor, task=task)
return task
def create_task(
self,
body: dict[str, Any],
idempotency_key: str | None = None,
*,
actor_identity: str | None = None,
) -> dict[str, Any]:
if body.get("parent_task_id"):
raise ForbiddenError("child tasks must use the delegate endpoint")
issuer = body.get("issuer", {})
actor = issuer.get("id", "anon") if isinstance(issuer, dict) else "anon"
declared_actor = issuer.get("id", "anon") if isinstance(issuer, dict) else "anon"
actor = actor_identity or declared_actor
self.policy.assert_allowed("create", actor)
try:
plan = TaskPlan.from_dict(body)
except (TypeError, ValueError) as exc:
raise ValidationError("invalid task plan") from exc
if actor_identity is not None:
plan = bind_task_scope(
plan, self.policy.actor_context(actor_identity), public_create=True
)
task = self.store.create_task_with_event(plan, idempotency_key=idempotency_key)
return task.to_dict()
def list_tasks(self, actor: str | None = None) -> list[dict[str, Any]]:
context = None
if actor is not None:
self.policy.assert_allowed("read", actor)
return [task.to_dict(include_task=False) for task in self.store.list_tasks()]
context = self.policy.actor_context(actor)
return [
task.to_dict(include_task=False)
for task in self.store.list_tasks(context)
]
def get_task(self, task_id: str, actor: str | None = None) -> dict[str, Any]:
context = None
if actor is not None:
self.policy.assert_allowed("read", actor, task=self.store.get_task(task_id))
return self.store.get_task(task_id).to_dict()
self.policy.assert_allowed("read", actor)
context = self.policy.actor_context(actor)
task = self.store.get_task(task_id, context)
if actor is not None:
self.policy.assert_allowed("read", actor, task=task)
return task.to_dict()
def get_worker(self, worker_id: str, actor: str | None = None) -> dict[str, Any]:
if actor is not None:
@@ -163,9 +190,8 @@ class TaskPlaneService:
if not worker_id:
raise ValidationError("worker_id required")
task = self.store.get_task(task_id)
task = self._authorized_task(task_id, actor, "claim")
self.policy.assert_readonly_worker_binding(actor, str(worker_id))
self.policy.assert_allowed("claim", actor, task=task)
lease_seconds = _bounded_int(
body.get("lease_seconds", 30), "lease_seconds", 1, 3600
)
@@ -184,10 +210,9 @@ class TaskPlaneService:
if not claim_id:
raise ValidationError("claim_id required")
actor = body.get("actor_identity") or body.get("worker_identity") or "worker"
task = self.store.get_task(task_id)
task = self._authorized_task(task_id, actor, "heartbeat")
claim = self.store.get_claim(claim_id)
self.policy.assert_readonly_worker_binding(actor, claim.worker_id)
self.policy.assert_allowed("heartbeat", actor, task=task)
if claim.worker_identity != actor:
raise ForbiddenError("claim identity mismatch")
lease_seconds = _bounded_int(
@@ -206,10 +231,9 @@ class TaskPlaneService:
message = body.get("message")
if progress is None:
raise ValidationError("progress required")
task = self.store.get_task(task_id)
task = self._authorized_task(task_id, actor, "progress")
claim = self.store.get_claim(claim_id)
self.policy.assert_readonly_worker_binding(actor, claim.worker_id)
self.policy.assert_allowed("progress", actor, task=task)
if task.task_plan.data_classification == "credential-secret":
if message not in {None, ""} or contains_secret_text(stage):
raise ValidationError(
@@ -234,7 +258,7 @@ class TaskPlaneService:
if not claim_id:
raise ValidationError("claim_id required")
actor = body.get("actor_identity") or body.get("worker_identity") or "worker"
task = self.store.get_task(task_id)
task = self._authorized_task(task_id, actor, "result")
status = body.get("status", "COMPLETED")
outputs = body.get("outputs", [])
@@ -247,7 +271,6 @@ class TaskPlaneService:
raise ValidationError("evidence items must be objects")
claim = self.store.get_claim(claim_id)
self.policy.assert_readonly_worker_binding(actor, claim.worker_id)
self.policy.assert_allowed("result", actor, task=task)
if claim.worker_identity != actor:
raise ForbiddenError("claim identity mismatch")
if task.task_plan.data_classification == "credential-secret":
@@ -326,8 +349,7 @@ class TaskPlaneService:
reason = body.get("reason", "failed")
actor = body.get("actor_identity") or body.get("worker_identity") or "worker"
self.policy.assert_allowed("fail", actor)
task = self.store.get_task(task_id)
self.policy.assert_allowed("fail", actor, task=task)
task = self._authorized_task(task_id, actor, "fail")
claim = self.store.get_claim(claim_id)
if claim.worker_identity != actor:
raise ForbiddenError("claim identity mismatch")
@@ -339,8 +361,7 @@ class TaskPlaneService:
actor = body.get("actor_identity") or body.get("worker_identity") or "owner"
self.policy.assert_allowed("cancel", actor)
reason = body.get("reason", "cancelled")
task = self.store.get_task(task_id)
self.policy.assert_allowed("cancel", actor, task=task)
task = self._authorized_task(task_id, actor, "cancel")
updated = self.store.cancel_task_with_event(
task_id, task.revision, reason=reason
)
@@ -349,8 +370,7 @@ class TaskPlaneService:
def delegate(self, task_id: str, body: dict[str, Any]) -> dict[str, Any]:
actor = body.get("actor_identity") or "owner"
self.policy.assert_allowed("delegate", actor)
parent = self.store.get_task(task_id)
self.policy.assert_allowed("delegate", actor, task=parent)
parent = self._authorized_task(task_id, actor, "delegate")
if "task" in body:
try:
child_plan = TaskPlan.from_dict(body["task"])
@@ -364,25 +384,31 @@ class TaskPlaneService:
except (TypeError, ValueError) as exc:
raise ValidationError("invalid child task plan") from exc
child_plan.parent_task_id = task_id
child_plan = bind_task_scope(
child_plan, self.policy.actor_context(actor), public_create=False
)
child = self.delegation.delegate(task_id, child_plan, caller=actor)
return child.to_dict()
def get_events(self, task_id: str, actor: str | None = None) -> list[dict[str, Any]]:
if actor is not None:
self.policy.assert_allowed("audit", actor, task=self.store.get_task(task_id))
self.store.get_task(task_id)
self._authorized_task(task_id, actor, "audit")
else:
self.store.get_task(task_id)
return self.store.list_events(task_id)
def get_evidence(self, task_id: str, actor: str | None = None) -> list[dict[str, Any]]:
if actor is not None:
self.policy.assert_allowed("audit", actor, task=self.store.get_task(task_id))
self.store.get_task(task_id)
self._authorized_task(task_id, actor, "audit")
else:
self.store.get_task(task_id)
return self.evidence.list(task_id)
def get_children(self, task_id: str, actor: str | None = None) -> list[str]:
if actor is not None:
self.policy.assert_allowed("read", actor, task=self.store.get_task(task_id))
self.store.get_task(task_id)
self._authorized_task(task_id, actor, "read")
else:
self.store.get_task(task_id)
return self.store.list_children(task_id)
@@ -506,7 +532,11 @@ class TaskPlaneRequestHandler(BaseHTTPRequestHandler):
):
raise ForbiddenError("issuer identity mismatch")
body["issuer"] = {"type": "authenticated", "id": identity}
return 201, self.service.create_task(body or {}, headers.get("idempotency-key"))
return 201, self.service.create_task(
body or {},
headers.get("idempotency-key"),
actor_identity=identity,
)
if body.get("actor_identity") not in {None, identity}:
raise ForbiddenError("actor identity mismatch")
body["actor_identity"] = identity
+5
View File
@@ -106,3 +106,8 @@ class DurableOutboxDispatcher:
receipt=str(receipt) if receipt else None,
error=None if receipt else f"{adapter.name} receipt unavailable",
)
# R4.2 P0-B compatibility adapter.
def to_handoff_envelope(event, *, target, sequence=1, idempotency_key=None, **scope):
from message_handoff_adapters import taskplane_event_to_envelope
return taskplane_event_to_envelope(event, target=target, sequence=sequence, idempotency_key=idempotency_key, **scope)
+911
View File
@@ -0,0 +1,911 @@
"""Durable multi-agent fleet scheduling for TaskPlane.
The fleet layer decides *which* eligible worker should receive *which* queued task.
It never replaces TaskPlane claim/lease ownership: callers must still perform the
authoritative TaskPlane claim CAS after a reservation is returned.
"""
from __future__ import annotations
from contextlib import contextmanager
from dataclasses import dataclass
import hashlib
import json
import math
import sqlite3
import threading
import time
from typing import Any, Iterable, Mapping, Sequence
DEFAULT_RESERVATION_SECONDS = 30
DEFAULT_CIRCUIT_THRESHOLD = 3
DEFAULT_CIRCUIT_SECONDS = 30
MAX_CIRCUIT_SECONDS = 15 * 60
MAX_SLOTS = 128
class FleetError(RuntimeError):
pass
class FleetConflict(FleetError):
pass
def _canonical(value: object) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def _sha(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _text(value: object, default: str = "") -> str:
if value is None:
return default
return str(value).strip() or default
def _int(value: object, default: int, minimum: int, maximum: int) -> int:
if value is None:
return default
if isinstance(value, bool):
raise FleetError("FLEET_INTEGER_INVALID")
try:
result = int(value)
except (TypeError, ValueError) as exc:
raise FleetError("FLEET_INTEGER_INVALID") from exc
if result < minimum or result > maximum:
raise FleetError("FLEET_INTEGER_OUT_OF_RANGE")
return result
def _float(value: object, default: float, minimum: float, maximum: float) -> float:
if value is None:
return default
try:
result = float(value)
except (TypeError, ValueError) as exc:
raise FleetError("FLEET_NUMBER_INVALID") from exc
if not math.isfinite(result) or result < minimum or result > maximum:
raise FleetError("FLEET_NUMBER_OUT_OF_RANGE")
return result
def _mapping(value: object) -> Mapping[str, Any]:
if isinstance(value, Mapping):
return value
if hasattr(value, "to_dict"):
data = value.to_dict()
if isinstance(data, Mapping):
return data
if hasattr(value, "__dict__"):
return vars(value)
return {}
def _attr(value: object, name: str, default: Any = None) -> Any:
if isinstance(value, Mapping):
return value.get(name, default)
return getattr(value, name, default)
def _task_plan(task: object) -> object:
return _attr(task, "task_plan", _attr(task, "plan", task))
def _task_id(task: object) -> str:
return _text(_attr(task, "task_id", _attr(_task_plan(task), "task_id", "")))
def _task_status(task: object) -> str:
return _text(_attr(task, "status", "")).upper()
def _task_revision(task: object) -> int:
return _int(_attr(task, "revision", 0), 0, 0, 2**31 - 1)
def _task_created_at(task: object) -> float:
value = _attr(task, "created_at", _attr(task, "updated_at", 0.0))
try:
return float(value or 0.0)
except (TypeError, ValueError):
return 0.0
def _task_required_capabilities(task: object) -> frozenset[str]:
plan = _task_plan(task)
value = _attr(plan, "required_capabilities", ())
if not isinstance(value, Sequence) or isinstance(value, (str, bytes)):
return frozenset()
return frozenset(_text(item) for item in value if _text(item))
def _task_metadata(task: object) -> Mapping[str, Any]:
plan = _task_plan(task)
value = _attr(plan, "metadata", {})
return value if isinstance(value, Mapping) else {}
def _task_scope_key(task: object) -> str:
plan = _task_plan(task)
scope_type = _text(_attr(plan, "scope_type", "legacy"), "legacy")
tenant = _text(_attr(plan, "tenant_id", ""))
project = _text(_attr(plan, "project_id", ""))
user = _text(_attr(plan, "user_id", ""))
if scope_type == "tenant":
return "tenant:" + tenant + "|project:" + (project or "*")
if scope_type == "system":
return "system"
return "legacy:" + (user or "*")
def _worker_id(worker: object) -> str:
return _text(_attr(worker, "worker_id", ""))
def _worker_capabilities(worker: object) -> frozenset[str]:
value = _attr(worker, "capabilities", ())
if isinstance(value, Mapping):
value = value.keys()
if not isinstance(value, Iterable) or isinstance(value, (str, bytes)):
return frozenset()
return frozenset(_text(item) for item in value if _text(item))
@dataclass(frozen=True)
class FleetWorkerPolicy:
worker_id: str
max_slots: int = 1
failure_domain: str = "default"
weight: float = 1.0
labels: Mapping[str, str] | None = None
revision: int = 1
@dataclass(frozen=True)
class FleetDispatchDecision:
reservation_id: str
task_id: str
worker_id: str
task_revision: int
scope_key: str
failure_domain: str
reserved_until: float
score: tuple[float, float, float, str]
def to_dict(self) -> dict[str, Any]:
return {
"reservation_id": self.reservation_id,
"task_id": self.task_id,
"worker_id": self.worker_id,
"task_revision": self.task_revision,
"scope_key": self.scope_key,
"failure_domain": self.failure_domain,
"reserved_until": self.reserved_until,
"score": list(self.score),
}
class FleetStateStore:
"""Small durable fleet journal colocated in the existing TaskPlane SQLite database."""
def __init__(self, connection: sqlite3.Connection, lock: threading.RLock | None = None) -> None:
self.conn = connection
self.lock = lock or threading.RLock()
self._savepoint_counter = 0
self._ensure_schema()
@contextmanager
def _transaction(self):
with self.lock:
self._savepoint_counter += 1
name = f"fleet_{threading.get_ident()}_{self._savepoint_counter}"
self.conn.execute(f"SAVEPOINT {name}")
try:
yield
except Exception:
self.conn.execute(f"ROLLBACK TO {name}")
self.conn.execute(f"RELEASE {name}")
raise
else:
self.conn.execute(f"RELEASE {name}")
def _ensure_schema(self) -> None:
with self.lock:
self.conn.executescript(
"""
CREATE TABLE IF NOT EXISTS fleet_worker_capacity (
worker_id TEXT PRIMARY KEY,
max_slots INTEGER NOT NULL,
failure_domain TEXT NOT NULL,
weight REAL NOT NULL,
labels_json TEXT NOT NULL,
revision INTEGER NOT NULL,
updated_at REAL NOT NULL
);
CREATE TABLE IF NOT EXISTS fleet_scope_state (
scope_key TEXT PRIMARY KEY,
virtual_finish REAL NOT NULL,
dispatch_count INTEGER NOT NULL,
last_dispatched_at REAL,
revision INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS fleet_dispatches (
reservation_id TEXT PRIMARY KEY,
task_id TEXT NOT NULL UNIQUE,
worker_id TEXT NOT NULL,
task_revision INTEGER NOT NULL,
scope_key TEXT NOT NULL,
failure_domain TEXT NOT NULL,
state TEXT NOT NULL,
reserved_until REAL NOT NULL,
claim_id TEXT,
created_at REAL NOT NULL,
updated_at REAL NOT NULL
);
CREATE INDEX IF NOT EXISTS ix_fleet_dispatch_worker_state
ON fleet_dispatches(worker_id, state, reserved_until);
CREATE TABLE IF NOT EXISTS fleet_worker_failures (
worker_id TEXT PRIMARY KEY,
consecutive_failures INTEGER NOT NULL,
circuit_open_until REAL,
last_failure_class TEXT,
revision INTEGER NOT NULL,
updated_at REAL NOT NULL
);
CREATE TABLE IF NOT EXISTS fleet_task_worker_failures (
task_id TEXT NOT NULL,
worker_id TEXT NOT NULL,
failure_count INTEGER NOT NULL,
blocked_until REAL,
last_failure_class TEXT,
revision INTEGER NOT NULL,
updated_at REAL NOT NULL,
PRIMARY KEY(task_id, worker_id)
);
"""
)
def configure_worker(
self,
worker_id: str,
*,
max_slots: int = 1,
failure_domain: str = "default",
weight: float = 1.0,
labels: Mapping[str, str] | None = None,
expected_revision: int | None = None,
now: float | None = None,
) -> FleetWorkerPolicy:
worker_id = _text(worker_id)
if not worker_id:
raise FleetError("FLEET_WORKER_ID_REQUIRED")
max_slots = _int(max_slots, 1, 1, MAX_SLOTS)
weight = _float(weight, 1.0, 0.05, 100.0)
failure_domain = _text(failure_domain, "default")
normalized_labels = {
_text(k): _text(v)
for k, v in dict(labels or {}).items()
if _text(k) and _text(v)
}
now = float(time.time() if now is None else now)
with self._transaction():
row = self.conn.execute(
"SELECT revision FROM fleet_worker_capacity WHERE worker_id=?",
(worker_id,),
).fetchone()
current = int(row[0]) if row else 0
if expected_revision is not None and current != int(expected_revision):
raise FleetConflict("FLEET_WORKER_POLICY_CAS_MISMATCH")
revision = current + 1
self.conn.execute(
"""
INSERT INTO fleet_worker_capacity(
worker_id,max_slots,failure_domain,weight,labels_json,revision,updated_at
) VALUES (?,?,?,?,?,?,?)
ON CONFLICT(worker_id) DO UPDATE SET
max_slots=excluded.max_slots,
failure_domain=excluded.failure_domain,
weight=excluded.weight,
labels_json=excluded.labels_json,
revision=excluded.revision,
updated_at=excluded.updated_at
""",
(
worker_id,
max_slots,
failure_domain,
weight,
json.dumps(normalized_labels, sort_keys=True, separators=(",", ":")),
revision,
now,
),
)
return FleetWorkerPolicy(
worker_id, max_slots, failure_domain, weight, normalized_labels, revision
)
def worker_policy(self, worker_id: str) -> FleetWorkerPolicy:
worker_id = _text(worker_id)
with self.lock:
row = self.conn.execute(
"""
SELECT worker_id,max_slots,failure_domain,weight,labels_json,revision
FROM fleet_worker_capacity WHERE worker_id=?
""",
(worker_id,),
).fetchone()
if row is None:
return FleetWorkerPolicy(worker_id=worker_id)
return FleetWorkerPolicy(
worker_id=str(row[0]),
max_slots=int(row[1]),
failure_domain=str(row[2]),
weight=float(row[3]),
labels=json.loads(row[4] or "{}"),
revision=int(row[5]),
)
def _cleanup_expired(self, now: float) -> None:
self.conn.execute(
"""
UPDATE fleet_dispatches
SET state='EXPIRED', updated_at=?
WHERE state='RESERVED' AND reserved_until<=?
""",
(now, now),
)
def reserved_slots(self, worker_id: str, now: float) -> int:
with self.lock:
self._cleanup_expired(now)
row = self.conn.execute(
"""
SELECT COUNT(*) FROM fleet_dispatches
WHERE worker_id=? AND state='RESERVED' AND reserved_until>?
""",
(worker_id, now),
).fetchone()
return int(row[0] if row else 0)
def scope_virtual_finish(self, scope_key: str) -> float:
with self.lock:
row = self.conn.execute(
"SELECT virtual_finish FROM fleet_scope_state WHERE scope_key=?",
(scope_key,),
).fetchone()
return float(row[0]) if row else 0.0
def reserve(
self,
*,
task_id: str,
worker_id: str,
task_revision: int,
scope_key: str,
failure_domain: str,
score: tuple[float, float, float, str],
reservation_seconds: int,
now: float,
scope_weight: float,
) -> FleetDispatchDecision:
reservation_seconds = _int(
reservation_seconds, DEFAULT_RESERVATION_SECONDS, 1, 300
)
reservation_id = "fleet-" + _sha(
{
"task_id": task_id,
"worker_id": worker_id,
"task_revision": task_revision,
"scope_key": scope_key,
}
)[:32]
reserved_until = now + reservation_seconds
with self._transaction():
self._cleanup_expired(now)
existing = self.conn.execute(
"""
SELECT reservation_id,worker_id,task_revision,scope_key,failure_domain,
reserved_until,state
FROM fleet_dispatches WHERE task_id=?
""",
(task_id,),
).fetchone()
# Capacity is rechecked inside the same durable reservation transaction.
# This closes the cross-thread/process race between eligibility scoring and reserve.
policy_row = self.conn.execute(
"SELECT max_slots FROM fleet_worker_capacity WHERE worker_id=?",
(worker_id,),
).fetchone()
max_slots = int(policy_row[0]) if policy_row else 1
if existing and str(existing[6]) in {"RESERVED", "CLAIMED"} and float(existing[5]) > now:
if str(existing[1]) != worker_id or int(existing[2]) != task_revision:
raise FleetConflict("FLEET_TASK_ALREADY_RESERVED")
return FleetDispatchDecision(
str(existing[0]), task_id, worker_id, task_revision, str(existing[3]),
str(existing[4]), float(existing[5]), score
)
claim_columns = {
str(row[1])
for row in self.conn.execute("PRAGMA table_info(claims)").fetchall()
}
active_claims = 0
if {"worker_id", "status"}.issubset(claim_columns):
sql = "SELECT COUNT(*) FROM claims WHERE worker_id=? AND status='ACTIVE'"
params: list[Any] = [worker_id]
if "lease_expires_at" in claim_columns:
sql += " AND lease_expires_at>?"
params.append(now)
row = self.conn.execute(sql, tuple(params)).fetchone()
active_claims = int(row[0] if row else 0)
row = self.conn.execute(
"""
SELECT COUNT(*) FROM fleet_dispatches
WHERE worker_id=? AND task_id<>?
AND state='RESERVED' AND reserved_until>?
""",
(worker_id, task_id, now),
).fetchone()
active_reservations = int(row[0] if row else 0)
if active_claims + active_reservations >= max_slots:
raise FleetConflict("FLEET_WORKER_CAPACITY_EXHAUSTED")
self.conn.execute(
"""
INSERT INTO fleet_dispatches(
reservation_id,task_id,worker_id,task_revision,scope_key,failure_domain,
state,reserved_until,claim_id,created_at,updated_at
) VALUES (?,?,?,?,?,?,'RESERVED',?,NULL,?,?)
ON CONFLICT(task_id) DO UPDATE SET
reservation_id=excluded.reservation_id,
worker_id=excluded.worker_id,
task_revision=excluded.task_revision,
scope_key=excluded.scope_key,
failure_domain=excluded.failure_domain,
state='RESERVED',
reserved_until=excluded.reserved_until,
claim_id=NULL,
updated_at=excluded.updated_at
""",
(
reservation_id, task_id, worker_id, task_revision, scope_key,
failure_domain, reserved_until, now, now,
),
)
row = self.conn.execute(
"SELECT virtual_finish,dispatch_count,revision FROM fleet_scope_state WHERE scope_key=?",
(scope_key,),
).fetchone()
current_finish = float(row[0]) if row else 0.0
dispatch_count = int(row[1]) if row else 0
revision = int(row[2]) if row else 0
next_finish = max(current_finish, 0.0) + 1.0 / max(scope_weight, 0.05)
self.conn.execute(
"""
INSERT INTO fleet_scope_state(
scope_key,virtual_finish,dispatch_count,last_dispatched_at,revision
) VALUES (?,?,?,?,?)
ON CONFLICT(scope_key) DO UPDATE SET
virtual_finish=excluded.virtual_finish,
dispatch_count=excluded.dispatch_count,
last_dispatched_at=excluded.last_dispatched_at,
revision=excluded.revision
""",
(scope_key, next_finish, dispatch_count + 1, now, revision + 1),
)
return FleetDispatchDecision(
reservation_id, task_id, worker_id, task_revision, scope_key,
failure_domain, reserved_until, score
)
def bind_claim(self, reservation_id: str, claim_id: str, *, now: float | None = None) -> None:
now = float(time.time() if now is None else now)
with self._transaction():
row = self.conn.execute(
"SELECT state FROM fleet_dispatches WHERE reservation_id=?",
(reservation_id,),
).fetchone()
if row is None:
raise FleetError("FLEET_RESERVATION_NOT_FOUND")
if str(row[0]) not in {"RESERVED", "CLAIMED"}:
raise FleetConflict("FLEET_RESERVATION_NOT_ACTIVE")
self.conn.execute(
"""
UPDATE fleet_dispatches
SET state='CLAIMED', claim_id=?, updated_at=?
WHERE reservation_id=?
""",
(claim_id, now, reservation_id),
)
def release(
self,
task_id: str,
*,
state: str = "RELEASED",
now: float | None = None,
) -> None:
if state not in {"RELEASED", "COMPLETED", "FAILED", "EXPIRED"}:
raise FleetError("FLEET_RESERVATION_STATE_INVALID")
now = float(time.time() if now is None else now)
with self._transaction():
self.conn.execute(
"UPDATE fleet_dispatches SET state=?,updated_at=? WHERE task_id=?",
(state, now, task_id),
)
def worker_circuit_open(self, worker_id: str, now: float) -> bool:
with self.lock:
row = self.conn.execute(
"SELECT circuit_open_until FROM fleet_worker_failures WHERE worker_id=?",
(worker_id,),
).fetchone()
return bool(row and row[0] is not None and float(row[0]) > now)
def task_worker_blocked(self, task_id: str, worker_id: str, now: float) -> bool:
with self.lock:
row = self.conn.execute(
"""
SELECT blocked_until FROM fleet_task_worker_failures
WHERE task_id=? AND worker_id=?
""",
(task_id, worker_id),
).fetchone()
return bool(row and row[0] is not None and float(row[0]) > now)
def record_worker_failure(
self,
worker_id: str,
failure_class: str,
*,
threshold: int = DEFAULT_CIRCUIT_THRESHOLD,
base_circuit_seconds: int = DEFAULT_CIRCUIT_SECONDS,
now: float | None = None,
) -> dict[str, Any]:
now = float(time.time() if now is None else now)
threshold = _int(threshold, DEFAULT_CIRCUIT_THRESHOLD, 1, 20)
base_circuit_seconds = _int(base_circuit_seconds, DEFAULT_CIRCUIT_SECONDS, 1, 3600)
with self._transaction():
row = self.conn.execute(
"""
SELECT consecutive_failures,revision FROM fleet_worker_failures
WHERE worker_id=?
""",
(worker_id,),
).fetchone()
failures = (int(row[0]) if row else 0) + 1
revision = (int(row[1]) if row else 0) + 1
open_until = None
if failures >= threshold:
exponent = min(failures - threshold, 8)
open_until = now + min(
MAX_CIRCUIT_SECONDS, base_circuit_seconds * (2**exponent)
)
self.conn.execute(
"""
INSERT INTO fleet_worker_failures(
worker_id,consecutive_failures,circuit_open_until,last_failure_class,
revision,updated_at
) VALUES (?,?,?,?,?,?)
ON CONFLICT(worker_id) DO UPDATE SET
consecutive_failures=excluded.consecutive_failures,
circuit_open_until=excluded.circuit_open_until,
last_failure_class=excluded.last_failure_class,
revision=excluded.revision,
updated_at=excluded.updated_at
""",
(worker_id, failures, open_until, _text(failure_class, "unknown"), revision, now),
)
return {
"worker_id": worker_id,
"consecutive_failures": failures,
"circuit_open_until": open_until,
"revision": revision,
}
def record_worker_success(self, worker_id: str, *, now: float | None = None) -> None:
now = float(time.time() if now is None else now)
with self._transaction():
self.conn.execute(
"""
INSERT INTO fleet_worker_failures(
worker_id,consecutive_failures,circuit_open_until,last_failure_class,
revision,updated_at
) VALUES (?,0,NULL,NULL,1,?)
ON CONFLICT(worker_id) DO UPDATE SET
consecutive_failures=0,
circuit_open_until=NULL,
last_failure_class=NULL,
revision=revision+1,
updated_at=excluded.updated_at
""",
(worker_id, now),
)
def record_task_worker_failure(
self,
task_id: str,
worker_id: str,
failure_class: str,
*,
quarantine_seconds: int = 60,
now: float | None = None,
) -> dict[str, Any]:
now = float(time.time() if now is None else now)
quarantine_seconds = _int(quarantine_seconds, 60, 1, 3600)
with self._transaction():
row = self.conn.execute(
"""
SELECT failure_count,revision FROM fleet_task_worker_failures
WHERE task_id=? AND worker_id=?
""",
(task_id, worker_id),
).fetchone()
count = (int(row[0]) if row else 0) + 1
revision = (int(row[1]) if row else 0) + 1
blocked_until = now + quarantine_seconds
self.conn.execute(
"""
INSERT INTO fleet_task_worker_failures(
task_id,worker_id,failure_count,blocked_until,last_failure_class,
revision,updated_at
) VALUES (?,?,?,?,?,?,?)
ON CONFLICT(task_id,worker_id) DO UPDATE SET
failure_count=excluded.failure_count,
blocked_until=excluded.blocked_until,
last_failure_class=excluded.last_failure_class,
revision=excluded.revision,
updated_at=excluded.updated_at
""",
(
task_id, worker_id, count, blocked_until,
_text(failure_class, "unknown"), revision, now,
),
)
return {
"task_id": task_id,
"worker_id": worker_id,
"failure_count": count,
"blocked_until": blocked_until,
"revision": revision,
}
class FleetScheduler:
"""Deterministic capacity/fairness scheduler over existing TaskPlane records."""
def __init__(
self,
store: Any,
workers: Any,
*,
now_fn: Any = time.time,
reservation_seconds: int = DEFAULT_RESERVATION_SECONDS,
) -> None:
self.store = store
self.workers = workers
self.now_fn = now_fn
self.reservation_seconds = _int(
reservation_seconds, DEFAULT_RESERVATION_SECONDS, 1, 300
)
connection = getattr(store, "conn", None)
if not isinstance(connection, sqlite3.Connection):
raise FleetError("FLEET_TASKPLANE_SQLITE_CONNECTION_REQUIRED")
lock = getattr(store, "_lock", None)
self.state = FleetStateStore(connection, lock)
def configure_worker(self, worker_id: str, **kwargs: Any) -> FleetWorkerPolicy:
return self.state.configure_worker(worker_id, **kwargs)
def _online(self, worker_id: str, now: float) -> bool:
method = getattr(self.workers, "is_online", None)
if not callable(method):
raise FleetError("FLEET_WORKER_REGISTRY_ONLINE_CHECK_REQUIRED")
try:
return bool(method(worker_id, now=now))
except TypeError:
return bool(method(worker_id))
def _worker_record(self, worker_id: str) -> object:
getter = getattr(self.store, "get_worker", None)
if not callable(getter):
raise FleetError("FLEET_WORKER_LOOKUP_REQUIRED")
return getter(worker_id)
def _active_claims(self, worker_id: str, now: float) -> int:
# The TaskPlane store owns one SQLite connection in the current runtime.
# Fleet reads must share its lock just like fleet writes do, otherwise two
# concurrent pullers can issue overlapping operations on the same handle.
with self.state.lock:
columns = {
str(row[1])
for row in self.store.conn.execute("PRAGMA table_info(claims)").fetchall()
}
required = {"worker_id", "status"}
if not required.issubset(columns):
raise FleetError("FLEET_CLAIMS_SCHEMA_UNSUPPORTED")
sql = "SELECT COUNT(*) FROM claims WHERE worker_id=? AND status='ACTIVE'"
params: list[Any] = [worker_id]
if "lease_expires_at" in columns:
sql += " AND lease_expires_at>?"
params.append(now)
row = self.store.conn.execute(sql, tuple(params)).fetchone()
return int(row[0] if row else 0)
@staticmethod
def _labels_match(policy: FleetWorkerPolicy, task: object) -> bool:
metadata = _task_metadata(task)
affinity = metadata.get("fleet_affinity_labels") or {}
if affinity and not isinstance(affinity, Mapping):
return False
labels = dict(policy.labels or {})
for key, value in dict(affinity).items():
if labels.get(_text(key)) != _text(value):
return False
avoid = metadata.get("fleet_avoid_failure_domains") or []
if isinstance(avoid, Sequence) and not isinstance(avoid, (str, bytes)):
if policy.failure_domain in {_text(item) for item in avoid}:
return False
return True
def _eligible_worker(self, worker_id: str, task: object, now: float) -> tuple[bool, int, FleetWorkerPolicy]:
if not self._online(worker_id, now):
return False, 0, self.state.worker_policy(worker_id)
worker = self._worker_record(worker_id)
policy = self.state.worker_policy(worker_id)
if self.state.worker_circuit_open(worker_id, now):
return False, 0, policy
if self.state.task_worker_blocked(_task_id(task), worker_id, now):
return False, 0, policy
if not self._labels_match(policy, task):
return False, 0, policy
required = _task_required_capabilities(task)
capabilities = _worker_capabilities(worker)
if required and not required.issubset(capabilities):
return False, 0, policy
active = self._active_claims(worker_id, now)
reserved = self.state.reserved_slots(worker_id, now)
load = active + reserved
if load >= policy.max_slots:
return False, load, policy
return True, load, policy
@staticmethod
def _task_priority(task: object, now: float) -> float:
metadata = _task_metadata(task)
base = _int(metadata.get("fleet_priority"), 0, -100, 100)
created = _task_created_at(task)
waited = max(0.0, now - created) if created > 0 else 0.0
aging = min(100.0, waited / 60.0)
return float(base) + aging
def _task_score(self, task: object, now: float) -> tuple[float, float, float, str]:
scope = _task_scope_key(task)
virtual_finish = self.state.scope_virtual_finish(scope)
priority = self._task_priority(task, now)
return (virtual_finish, -priority, _task_created_at(task), _task_id(task))
def _tasks(self, actor_context: Any = None) -> list[object]:
lister = getattr(self.store, "list_tasks", None)
if not callable(lister):
raise FleetError("FLEET_TASK_LIST_REQUIRED")
try:
tasks = lister(actor_context)
except TypeError:
tasks = lister()
return [
task for task in tasks
if _task_status(task) in {"CREATED", "QUEUED"}
and _task_id(task)
]
def rank_for_worker(
self,
worker_id: str,
*,
actor_context: Any = None,
now: float | None = None,
) -> list[object]:
"""Return fleet-ranked tasks without reserving or claiming anything."""
now = float(self.now_fn() if now is None else now)
worker_id = _text(worker_id)
if not worker_id:
return []
ranked: list[object] = []
for task in self._tasks(actor_context):
eligible, _, _ = self._eligible_worker(worker_id, task, now)
if eligible:
ranked.append(task)
return sorted(ranked, key=lambda task: self._task_score(task, now))
def reserve_for_worker(
self,
worker_id: str,
task: object,
*,
now: float | None = None,
) -> FleetDispatchDecision:
"""Reserve exactly one already-ranked task, rechecking eligibility and capacity."""
now = float(self.now_fn() if now is None else now)
worker_id = _text(worker_id)
eligible, _, policy = self._eligible_worker(worker_id, task, now)
if not eligible:
raise FleetConflict("FLEET_WORKER_NOT_ELIGIBLE")
metadata = _task_metadata(task)
scope_weight = _float(metadata.get("fleet_scope_weight"), 1.0, 0.05, 100.0)
return self.state.reserve(
task_id=_task_id(task),
worker_id=worker_id,
task_revision=_task_revision(task),
scope_key=_task_scope_key(task),
failure_domain=policy.failure_domain,
score=self._task_score(task, now),
reservation_seconds=self.reservation_seconds,
now=now,
scope_weight=scope_weight,
)
def select(
self,
worker_ids: Sequence[str],
*,
actor_context: Any = None,
limit: int = 1,
now: float | None = None,
) -> list[FleetDispatchDecision]:
now = float(self.now_fn() if now is None else now)
limit = _int(limit, 1, 1, 100)
worker_ids = tuple(dict.fromkeys(_text(item) for item in worker_ids if _text(item)))
tasks = sorted(self._tasks(actor_context), key=lambda task: self._task_score(task, now))
decisions: list[FleetDispatchDecision] = []
local_load: dict[str, int] = {}
for task in tasks:
if len(decisions) >= limit:
break
candidates: list[tuple[int, float, str, FleetWorkerPolicy]] = []
for worker_id in worker_ids:
eligible, load, policy = self._eligible_worker(worker_id, task, now)
if not eligible:
continue
effective_load = local_load.get(worker_id, load)
if effective_load >= policy.max_slots:
continue
candidates.append(
(
effective_load,
-policy.weight,
worker_id,
policy,
)
)
if not candidates:
continue
candidates.sort(key=lambda item: (item[0], item[1], item[2]))
_, _, worker_id, policy = candidates[0]
score = self._task_score(task, now)
metadata = _task_metadata(task)
scope_weight = _float(metadata.get("fleet_scope_weight"), 1.0, 0.05, 100.0)
try:
decision = self.state.reserve(
task_id=_task_id(task),
worker_id=worker_id,
task_revision=_task_revision(task),
scope_key=_task_scope_key(task),
failure_domain=policy.failure_domain,
score=score,
reservation_seconds=self.reservation_seconds,
now=now,
scope_weight=scope_weight,
)
except FleetConflict:
continue
local_load[worker_id] = local_load.get(
worker_id,
self._active_claims(worker_id, now)
+ self.state.reserved_slots(worker_id, now)
- 1,
) + 1
decisions.append(decision)
return decisions
+27
View File
@@ -13,6 +13,8 @@ from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Any
from .scope import normalize_task_scope
SENSITIVE_KEYS = {
"secret",
@@ -66,6 +68,12 @@ TASK_PLAN_KEYS = {
"allowed_input_paths",
"allowed_output_paths",
"output_policy",
"scope_type",
"tenant_id",
"user_id",
"project_id",
"conversation_id",
"run_id",
}
@@ -237,6 +245,12 @@ class TaskPlan:
schema_version: int = 1
task_id: str | None = None
task_type: str = "code.change"
scope_type: str = "legacy"
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
run_id: str | None = None
issuer: dict[str, Any] = field(default_factory=dict)
intent: str | None = None
required_capabilities: list[str] = field(default_factory=list)
@@ -311,10 +325,17 @@ class TaskPlan:
raise ValueError("credential-secret requires metadata-only logging")
if output_policy != "hash-count-status-only":
raise ValueError("credential-secret requires restricted output")
scope = normalize_task_scope(data)
return cls(
schema_version=schema_version,
task_id=validate_identifier(data["task_id"], "task_id"),
task_type=validate_type(data.get("task_type", "code.change"), "task_type"),
scope_type=scope.scope_type,
tenant_id=scope.tenant_id,
user_id=scope.user_id,
project_id=scope.project_id,
conversation_id=scope.conversation_id,
run_id=scope.run_id,
issuer=issuer,
intent=data.get("intent"),
required_capabilities=_normalize_list(data.get("required_capabilities", [])),
@@ -348,6 +369,12 @@ class TaskPlan:
"schema_version": self.schema_version,
"task_id": self.task_id,
"task_type": self.task_type,
"scope_type": self.scope_type,
"tenant_id": self.tenant_id,
"user_id": self.user_id,
"project_id": self.project_id,
"conversation_id": self.conversation_id,
"run_id": self.run_id,
"issuer": self.issuer,
"intent": self.intent,
"required_capabilities": list(self.required_capabilities),
+20 -1
View File
@@ -3,10 +3,11 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable
from typing import Any, Callable, Mapping
from .errors import ForbiddenError, LockedError
from .models import ProbeEvidence, ResultEnvelope, TaskRecord
from .scope import TaskActorContext, actor_context as build_actor_context
RISK_RANK = {"low": 1, "medium": 2, "high": 3}
@@ -93,6 +94,10 @@ class PolicyEngine:
result_signature_verifier: Callable[[ResultEnvelope], bool] | None = None,
probe_attestation_verifier: Callable[[ProbeEvidence], bool] | None = None,
identity_capabilities: dict[str, set[str]] | None = None,
identity_actor_contexts: Mapping[
str, TaskActorContext | Mapping[str, Any]
]
| None = None,
admin_identities: set[str] | None = None,
readonly_worker_execution_enabled: bool = False,
readonly_worker_profiles: list[ReadonlyWorkerProfile] | None = None,
@@ -109,12 +114,26 @@ class PolicyEngine:
identity: set(capabilities)
for identity, capabilities in (identity_capabilities or {}).items()
}
self.identity_actor_contexts = dict(identity_actor_contexts or {})
self.admin_identities = set(admin_identities or ())
self.readonly_worker_execution_enabled = readonly_worker_execution_enabled
self.readonly_worker_profiles = {
profile.identity: profile for profile in (readonly_worker_profiles or [])
}
def actor_context(self, identity: str) -> TaskActorContext:
configured = self.identity_actor_contexts.get(
identity, self.identity_actor_contexts.get("*")
)
capabilities = set(
self.identity_capabilities.get(
identity, self.identity_capabilities.get("*", set())
)
)
if identity in self.admin_identities:
capabilities.add("task.admin")
return build_actor_context(identity, configured, capabilities)
def _readonly_worker_allows(
self, action: str, actor: str, task: TaskRecord | None
) -> bool:
+200
View File
@@ -74,3 +74,203 @@ class TaskScheduler:
operation_digest=task.task_plan.operation_digest(),
)
return claim
# R4.2 P2 durable multi-agent fleet scheduling public surface.
from .fleet import (
FleetConflict,
FleetDispatchDecision,
FleetError,
FleetScheduler,
FleetStateStore,
FleetWorkerPolicy,
)
# R4.2 P2-B fleet-aware pull scheduling.
#
# The original TaskScheduler remains the security oracle and claim-CAS implementation.
# Fleet ordering only controls the order presented to that oracle.
import inspect as _fleet_inspect
import threading as _fleet_threading
from collections.abc import Mapping as _FleetMapping
from .worker_registry import WorkerRegistry as _FleetWorkerRegistry
from .errors import NotFoundError as _FleetNotFoundError
_LegacyTaskScheduler = TaskScheduler
_LegacyScheduleParameters = _fleet_inspect.signature(
_LegacyTaskScheduler.schedule_for_worker
).parameters
_LegacyNextParameters = _fleet_inspect.signature(
_LegacyTaskScheduler.next_queued_task
).parameters
class _FleetOrderedStoreProxy:
def __init__(self, delegate, tasks):
self._delegate = delegate
self._tasks = tuple(tasks)
def list_tasks(self, *args, **kwargs):
return list(self._tasks)
def __getattr__(self, name):
return getattr(self._delegate, name)
def _fleet_claim_field(claim, name, default=None):
if isinstance(claim, _FleetMapping):
return claim.get(name, default)
return getattr(claim, name, default)
class TaskScheduler(_LegacyTaskScheduler):
"""Fleet-integrated pull scheduler preserving legacy security and claim semantics."""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
store = getattr(self, "store", args[0] if args else None)
if store is None:
raise FleetError("FLEET_SCHEDULER_STORE_REQUIRED")
registry = getattr(self, "workers", None)
if registry is None:
registry = _FleetWorkerRegistry(store)
self.fleet_scheduler = FleetScheduler(store, registry)
self._fleet_local = _fleet_threading.local()
def _fleet_legacy_next(self, worker_id, tasks, *args, **kwargs):
proxy = _FleetOrderedStoreProxy(self.store, tasks)
legacy = _LegacyTaskScheduler(proxy)
legacy_kwargs = dict(kwargs)
if "actor_context" not in _LegacyNextParameters:
legacy_kwargs.pop("actor_context", None)
return legacy.next_queued_task(worker_id, *args, **legacy_kwargs)
def next_queued_task(self, worker_id, *args, **kwargs):
actor_context = kwargs.get(
"actor_context", getattr(self._fleet_local, "actor_context", None)
)
# Preserve legacy fail-closed worker checks before fleet ordering.
probe_kwargs = dict(kwargs)
if "actor_context" not in _LegacyNextParameters:
probe_kwargs.pop("actor_context", None)
legacy_probe = _LegacyTaskScheduler(self.store).next_queued_task(
worker_id, *args, **probe_kwargs
)
if legacy_probe is None:
return None
ranked = self.fleet_scheduler.rank_for_worker(
worker_id, actor_context=actor_context
)
# Preserve FleetScheduler fairness order. The legacy scheduler is used
# only as a fail-closed security oracle: present one fleet-ranked task
# at a time so its internal sort cannot reorder across scopes.
# Keep the legacy priority dimension ahead of Fleet fairness for
# compatibility with existing TaskPlane scheduling contracts; the
# Fleet order remains authoritative among equal-priority tasks.
def _legacy_priority(task):
plan = getattr(task, "task_plan", task)
metadata = getattr(plan, "metadata", {}) or {}
if not isinstance(metadata, _FleetMapping):
return 0
try:
return int(metadata.get("priority", 0))
except (TypeError, ValueError):
return 0
ranked = sorted(ranked, key=lambda task: -_legacy_priority(task))
for ranked_candidate in ranked:
try:
candidate = self._fleet_legacy_next(
worker_id, [ranked_candidate], *args, **kwargs
)
except _FleetNotFoundError:
continue
if candidate is None:
continue
ranked_id = getattr(
ranked_candidate,
"task_id",
getattr(getattr(ranked_candidate, "task_plan", None), "task_id", None),
)
candidate_id = getattr(
candidate,
"task_id",
getattr(getattr(candidate, "task_plan", None), "task_id", None),
)
if candidate_id != ranked_id:
raise FleetConflict("FLEET_LEGACY_ORACLE_ORDER_VIOLATION")
try:
decision = self.fleet_scheduler.reserve_for_worker(
worker_id, candidate
)
except FleetConflict:
continue
self._fleet_local.pending = decision
return candidate
return None
def schedule_for_worker(self, worker_id, *args, **kwargs):
actor_context = kwargs.get("actor_context")
legacy_kwargs = dict(kwargs)
if "actor_context" not in _LegacyScheduleParameters:
legacy_kwargs.pop("actor_context", None)
self._fleet_local.actor_context = actor_context
self._fleet_local.pending = None
try:
claim = _LegacyTaskScheduler.schedule_for_worker(
self, worker_id, *args, **legacy_kwargs
)
decision = getattr(self._fleet_local, "pending", None)
if claim is None:
if decision is not None:
self.fleet_scheduler.state.release(
decision.task_id, state="RELEASED"
)
return None
if decision is not None:
claim_id = _fleet_claim_field(claim, "claim_id")
claim_task_id = _fleet_claim_field(claim, "task_id", decision.task_id)
if not claim_id:
self.fleet_scheduler.state.release(
decision.task_id, state="RELEASED"
)
raise FleetError("FLEET_CLAIM_ID_REQUIRED")
if claim_task_id != decision.task_id:
self.fleet_scheduler.state.release(
decision.task_id, state="RELEASED"
)
raise FleetConflict("FLEET_CLAIM_TASK_MISMATCH")
self.fleet_scheduler.state.bind_claim(
decision.reservation_id, str(claim_id)
)
return claim
except Exception:
decision = getattr(self._fleet_local, "pending", None)
if decision is not None:
self.fleet_scheduler.state.release(
decision.task_id, state="RELEASED"
)
raise
finally:
self._fleet_local.pending = None
self._fleet_local.actor_context = None
def record_worker_failure(self, worker_id, failure_class, **kwargs):
return self.fleet_scheduler.state.record_worker_failure(
worker_id, failure_class, **kwargs
)
def record_worker_success(self, worker_id, **kwargs):
return self.fleet_scheduler.state.record_worker_success(
worker_id, **kwargs
)
def quarantine_task_worker(
self, task_id, worker_id, failure_class, **kwargs
):
return self.fleet_scheduler.state.record_task_worker_failure(
task_id, worker_id, failure_class, **kwargs
)
+181
View File
@@ -0,0 +1,181 @@
"""Trusted TaskPlane scope model and SQL authorization helpers."""
from __future__ import annotations
from dataclasses import dataclass, replace
from typing import Any, Mapping
from .errors import ForbiddenError, ValidationError
SCOPE_FIELDS = ("tenant_id", "user_id", "project_id", "conversation_id", "run_id")
PRIVILEGED_CAPABILITIES = frozenset(
{"*", "admin", "system.admin", "task.admin", "task.read.all"}
)
def _value(value: Any, field: str) -> Any:
if hasattr(value, "task_plan"):
value = value.task_plan
if isinstance(value, Mapping):
return value.get(field)
return getattr(value, field, None)
def _text(value: Any) -> str | None:
if value is None:
return None
result = str(value).strip()
return result or None
@dataclass(frozen=True)
class TaskScope:
scope_type: str
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
run_id: str | None = None
@dataclass(frozen=True)
class TaskActorContext:
identity: str
scope_type: str = "legacy"
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
run_id: str | None = None
capabilities: frozenset[str] = frozenset()
def normalize_task_scope(value: Any) -> TaskScope:
fields = {field: _text(_value(value, field)) for field in SCOPE_FIELDS}
declared = _text(_value(value, "scope_type"))
if declared is None:
declared = "tenant" if fields["tenant_id"] else "legacy"
if declared == "tenant":
if fields["tenant_id"] is None:
raise ValidationError(
"tenant scope requires tenant_id", code="TASK_SCOPE_TENANT_ID_REQUIRED"
)
elif declared == "legacy":
if any(fields.values()):
raise ValidationError(
"legacy scope cannot contain scoped ownership",
code="TASK_SCOPE_LEGACY_WITH_SCOPED_FIELDS",
)
elif declared == "system":
if any(fields[field] for field in ("tenant_id", "user_id", "project_id", "conversation_id")):
raise ValidationError(
"system scope cannot contain tenant ownership",
code="TASK_SCOPE_SYSTEM_WITH_TENANT_FIELDS",
)
else:
raise ValidationError("invalid task scope type", code="TASK_SCOPE_TYPE_INVALID")
return TaskScope(declared, **fields)
def actor_is_privileged(actor: TaskActorContext) -> bool:
return bool(actor.capabilities & PRIVILEGED_CAPABILITIES)
def actor_context(
identity: str,
configured: TaskActorContext | Mapping[str, Any] | None,
capabilities: set[str] | frozenset[str],
) -> TaskActorContext:
if isinstance(configured, TaskActorContext):
context = configured
else:
data = dict(configured or {})
data.pop("identity", None)
data.pop("capabilities", None)
context = TaskActorContext(identity=str(identity), **data)
scope = normalize_task_scope(context)
return replace(
context,
identity=str(identity),
scope_type=scope.scope_type,
tenant_id=scope.tenant_id,
user_id=scope.user_id,
project_id=scope.project_id,
conversation_id=scope.conversation_id,
run_id=scope.run_id,
capabilities=frozenset(str(item) for item in capabilities),
)
def bind_task_scope(plan: Any, actor: TaskActorContext, *, public_create: bool) -> Any:
requested = normalize_task_scope(plan)
if actor_is_privileged(actor):
trusted = normalize_task_scope(actor)
if requested.scope_type == "legacy" and trusted.scope_type != "legacy":
plan.scope_type = trusted.scope_type
for field in SCOPE_FIELDS:
setattr(plan, field, getattr(actor, field))
return plan
trusted = normalize_task_scope(actor)
if public_create and trusted.scope_type == "legacy":
raise ForbiddenError(
"authenticated identity has no tenant scope",
code="TASK_ACTOR_SCOPE_UNAVAILABLE",
)
if trusted.scope_type != "tenant":
if requested.scope_type == "legacy":
plan.scope_type = trusted.scope_type
for field in SCOPE_FIELDS:
setattr(plan, field, getattr(actor, field))
return plan
if requested.scope_type != trusted.scope_type:
raise ForbiddenError("task scope mismatch", code="TASK_SCOPE_MISMATCH_DENY")
return plan
if requested.scope_type not in {"legacy", "tenant"}:
raise ForbiddenError("task scope mismatch", code="TASK_SCOPE_MISMATCH_DENY")
updates: dict[str, Any] = {"scope_type": "tenant"}
for field in SCOPE_FIELDS:
declared = getattr(requested, field)
authoritative = getattr(actor, field)
if declared is not None and declared != authoritative:
raise ForbiddenError(
"task scope mismatch",
code="TASK_SCOPE_MISMATCH_DENY",
details={"field": field},
)
updates[field] = authoritative
for field, item in updates.items():
setattr(plan, field, item)
return plan
def actor_scope_sql(actor: TaskActorContext | None) -> tuple[str, tuple[Any, ...]]:
if actor is None or actor_is_privileged(actor):
return "", ()
scope = normalize_task_scope(actor)
if scope.scope_type == "legacy":
return "scope_type = 'legacy'", ()
if scope.scope_type == "system":
return "scope_type = 'system'", ()
clauses = ["scope_type = 'tenant'", "tenant_id = ?"]
params: list[Any] = [scope.tenant_id]
for field in ("user_id", "project_id", "conversation_id", "run_id"):
value = getattr(scope, field)
if value is None:
clauses.append(f"{field} IS NULL")
else:
clauses.append(f"({field} IS NULL OR {field} = ?)")
params.append(value)
return " AND ".join(clauses), tuple(params)
def task_scope_values(value: Any) -> tuple[str | None, ...]:
scope = normalize_task_scope(value)
return (
scope.scope_type,
scope.tenant_id,
scope.user_id,
scope.project_id,
scope.conversation_id,
scope.run_id,
)
+95 -11
View File
@@ -32,6 +32,7 @@ from .models import (
redact_value,
validate_identifier,
)
from .scope import TaskActorContext, actor_scope_sql, task_scope_values
EventEnvelope = tuple[str, str, str, str, str, float]
@@ -134,6 +135,12 @@ class TaskPlaneStore:
status TEXT NOT NULL,
task_json TEXT NOT NULL,
task_type TEXT NOT NULL,
scope_type TEXT NOT NULL DEFAULT 'legacy',
tenant_id TEXT,
user_id TEXT,
project_id TEXT,
conversation_id TEXT,
run_id TEXT,
root_task_id TEXT NOT NULL,
parent_task_id TEXT,
correlation_id TEXT,
@@ -272,6 +279,21 @@ class TaskPlaneStore:
)
"""
)
self._ensure_columns(
"tasks",
{
"scope_type": "TEXT NOT NULL DEFAULT 'legacy'",
"tenant_id": "TEXT",
"user_id": "TEXT",
"project_id": "TEXT",
"conversation_id": "TEXT",
"run_id": "TEXT",
},
)
self.conn.execute(
"UPDATE tasks SET scope_type='legacy' "
"WHERE scope_type IS NULL OR TRIM(scope_type)=''"
)
self._ensure_columns(
"workers",
{
@@ -299,6 +321,14 @@ class TaskPlaneStore:
)
self.conn.execute("CREATE INDEX IF NOT EXISTS ix_tasks_status ON tasks(status)")
self.conn.execute("CREATE INDEX IF NOT EXISTS ix_tasks_parent ON tasks(parent_task_id)")
self.conn.execute(
"CREATE INDEX IF NOT EXISTS ix_tasks_scope_tenant_project "
"ON tasks(scope_type, tenant_id, project_id)"
)
self.conn.execute(
"CREATE INDEX IF NOT EXISTS ix_tasks_scope_owner "
"ON tasks(scope_type, tenant_id, user_id, conversation_id, run_id)"
)
self.conn.execute("CREATE INDEX IF NOT EXISTS ix_workers_status ON workers(status)")
self.conn.execute("CREATE INDEX IF NOT EXISTS ix_claims_task ON claims(task_id)")
self.conn.execute(
@@ -347,6 +377,42 @@ class TaskPlaneStore:
raise NotFoundError("task")
task_json = json.loads(row["task_json"])
task_plan = TaskPlan.from_dict(task_json)
stored_scope = tuple(
row[field]
for field in (
"scope_type",
"tenant_id",
"user_id",
"project_id",
"conversation_id",
"run_id",
)
)
json_has_scope = any(
field in task_json
for field in (
"scope_type",
"tenant_id",
"user_id",
"project_id",
"conversation_id",
"run_id",
)
)
if json_has_scope and stored_scope != task_scope_values(task_plan):
raise ConflictError(
"task scope persistence mismatch",
code="TASK_SCOPE_PERSISTENCE_MISMATCH",
)
if not json_has_scope:
(
task_plan.scope_type,
task_plan.tenant_id,
task_plan.user_id,
task_plan.project_id,
task_plan.conversation_id,
task_plan.run_id,
) = stored_scope
task_plan.parent_task_id = row["parent_task_id"]
task_plan.correlation_id = row["correlation_id"]
return TaskRecord(
@@ -389,6 +455,7 @@ class TaskPlaneStore:
task_id = task_plan.task_id or f"task-{uuid.uuid4()}"
task_plan.task_id = task_id
task_json = redact_value(task_plan.to_dict(redact=False))
scope_values = task_scope_values(task_plan)
now = _now()
with self._transaction():
if idempotency_key:
@@ -397,7 +464,11 @@ class TaskPlaneStore:
(idempotency_key,),
).fetchone()
if existing:
if _canonical(json.loads(existing["task_json"])) != _canonical(task_json):
existing_plan = TaskPlan.from_dict(
json.loads(existing["task_json"])
)
existing_json = redact_value(existing_plan.to_dict(redact=False))
if _canonical(existing_json) != _canonical(task_json):
raise ConflictError("idempotency key payload mismatch")
return self.get_task(existing["task_id"])
root_task_id = task_id
@@ -412,14 +483,18 @@ class TaskPlaneStore:
self.conn.execute(
"""
INSERT INTO tasks
(task_id, revision, status, task_json, task_type, root_task_id, parent_task_id, correlation_id, idempotency_key, created_at, updated_at, failure_reason)
VALUES (?, 1, ?, ?, ?, ?, ?, ?, ?, ?, ?, NULL)
(task_id, revision, status, task_json, task_type,
scope_type, tenant_id, user_id, project_id, conversation_id, run_id,
root_task_id, parent_task_id, correlation_id, idempotency_key,
created_at, updated_at, failure_reason)
VALUES (?, 1, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, NULL)
""",
(
task_id,
"CREATED",
json.dumps(task_json, ensure_ascii=False),
task_plan.task_type,
*scope_values,
root_task_id,
task_plan.parent_task_id,
task_plan.correlation_id,
@@ -451,21 +526,30 @@ class TaskPlaneStore:
)
return self.get_task(task.task_id)
def get_task(self, task_id: str) -> TaskRecord:
def get_task(
self, task_id: str, actor_context: TaskActorContext | None = None
) -> TaskRecord:
task_id = _validated_id(task_id, "task_id")
predicate, scope_params = actor_scope_sql(actor_context)
sql = "SELECT * FROM tasks WHERE task_id=?"
if predicate:
sql += f" AND ({predicate})"
with self._lock:
row = self.conn.execute(
"SELECT * FROM tasks WHERE task_id=?", (task_id,)
).fetchone()
row = self.conn.execute(sql, (task_id, *scope_params)).fetchone()
if row is None:
raise NotFoundError(f"task {task_id} not found")
return self._row_to_task(row)
def list_tasks(self) -> list[TaskRecord]:
def list_tasks(
self, actor_context: TaskActorContext | None = None
) -> list[TaskRecord]:
predicate, scope_params = actor_scope_sql(actor_context)
sql = "SELECT * FROM tasks"
if predicate:
sql += f" WHERE ({predicate})"
sql += " ORDER BY created_at DESC"
with self._lock:
rows = self.conn.execute(
"SELECT * FROM tasks ORDER BY created_at DESC"
).fetchall()
rows = self.conn.execute(sql, scope_params).fetchall()
return [self._row_to_task(row) for row in rows]
def get_task_budget(self, task_id: str) -> Budget:
@@ -0,0 +1,759 @@
"""Scope-bound governance for traces, Tool/MCP calls, and credentials.
This module deliberately stores references and audit metadata only. Secret
material remains in the configured credential provider boundary.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
from contextlib import contextmanager
import hashlib
import json
import re
import sqlite3
from pathlib import Path
from typing import Any, Iterable, Mapping
SCOPE_FIELDS = ("tenant_id", "user_id", "project_id", "conversation_id", "run_id")
PRIVILEGED_CAPABILITIES = frozenset({"*", "admin", "system.admin", "governance.admin"})
REFERENCE_RE = re.compile(r"^(systemd-credential|vault|secret-manager):[A-Za-z0-9][A-Za-z0-9._:/#-]{0,255}$")
IDENTIFIER_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{0,179}$")
class GovernanceError(RuntimeError):
def __init__(self, code: str, *, field: str | None = None) -> None:
super().__init__(code if field is None else f"{code}:{field}")
self.code = code
self.field = field
def _clean(value: Any) -> str | None:
if value is None:
return None
text = str(value).strip()
return text or None
def _scope_tuple(value: Iterable[str] | str | None) -> tuple[str, ...]:
if value is None:
return ()
if isinstance(value, str):
value = value.split(",")
return tuple(sorted({_clean(item) for item in value if _clean(item)}))
def _canonical(value: Any) -> str:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False, default=str)
def _sha(value: Any) -> str:
return hashlib.sha256(_canonical(value).encode("utf-8")).hexdigest()
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _identifier(value: Any, code: str) -> str:
text = _clean(value)
if text is None or IDENTIFIER_RE.fullmatch(text) is None:
raise GovernanceError(code)
return text
@dataclass(frozen=True)
class GovernanceScope:
"""Explicit resource scope; system/global resources never get fake tenants."""
scope_type: str
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
run_id: str | None = None
def __post_init__(self) -> None:
object.__setattr__(self, "scope_type", (_clean(self.scope_type) or "").lower())
for name in SCOPE_FIELDS:
object.__setattr__(self, name, _clean(getattr(self, name)))
if self.scope_type == "tenant":
if self.tenant_id is None:
raise GovernanceError("GOVERNANCE_TENANT_ID_REQUIRED")
elif self.scope_type in {"system", "global"}:
if any(getattr(self, name) is not None for name in SCOPE_FIELDS):
raise GovernanceError("GOVERNANCE_NON_TENANT_SCOPE_INVALID")
elif self.scope_type == "legacy":
if any(getattr(self, name) is not None for name in SCOPE_FIELDS):
raise GovernanceError("GOVERNANCE_LEGACY_SCOPE_INVALID")
else:
raise GovernanceError("GOVERNANCE_SCOPE_TYPE_INVALID")
@classmethod
def from_value(cls, value: Any) -> "GovernanceScope":
def get(name: str) -> Any:
return value.get(name) if isinstance(value, Mapping) else getattr(value, name, None)
declared = _clean(get("scope_type"))
tenant_id = _clean(get("tenant_id"))
return cls(
scope_type=declared or ("tenant" if tenant_id else "legacy"),
tenant_id=tenant_id,
user_id=get("user_id"),
project_id=get("project_id"),
conversation_id=get("conversation_id"),
run_id=get("run_id"),
)
def as_dict(self) -> dict[str, Any]:
return asdict(self)
@dataclass(frozen=True)
class GovernanceActor:
identity: str
scope: GovernanceScope
capabilities: tuple[str, ...] = ()
def __post_init__(self) -> None:
object.__setattr__(self, "identity", _identifier(self.identity, "GOVERNANCE_ACTOR_INVALID"))
object.__setattr__(self, "capabilities", _scope_tuple(self.capabilities))
@classmethod
def from_value(cls, value: Any) -> "GovernanceActor":
if isinstance(value, cls):
return value
if not isinstance(value, Mapping):
raise GovernanceError("GOVERNANCE_ACTOR_INVALID")
return cls(
identity=value.get("identity") or value.get("actor_id") or value.get("user_id"),
scope=GovernanceScope.from_value(value),
capabilities=_scope_tuple(value.get("capabilities") or value.get("capability_scope")),
)
@property
def privileged(self) -> bool:
return bool(set(self.capabilities) & PRIVILEGED_CAPABILITIES)
def authorize_scope(actor: GovernanceActor | Mapping[str, Any], resource: GovernanceScope | Mapping[str, Any]) -> None:
actor = GovernanceActor.from_value(actor)
resource = resource if isinstance(resource, GovernanceScope) else GovernanceScope.from_value(resource)
if actor.privileged:
return
if resource.scope_type in {"system", "global"}:
raise GovernanceError("GOVERNANCE_SYSTEM_SCOPE_AUTHZ_DENY")
if resource.scope_type == "legacy":
if actor.scope.scope_type != "legacy":
raise GovernanceError("GOVERNANCE_LEGACY_SCOPE_AUTHZ_DENY")
return
if actor.scope.scope_type != "tenant":
raise GovernanceError("GOVERNANCE_SCOPE_MISMATCH_DENY", field="scope_type")
for name in SCOPE_FIELDS:
expected = getattr(resource, name)
actual = getattr(actor.scope, name)
if expected is not None and expected != actual:
raise GovernanceError("GOVERNANCE_SCOPE_MISMATCH_DENY", field=name)
@dataclass(frozen=True)
class TraceContext:
trace_id: str
span_id: str
scope: GovernanceScope
parent_span_id: str | None = None
correlation_id: str | None = None
causation_id: str | None = None
task_id: str | None = None
claim_id: str | None = None
message_id: str | None = None
def __post_init__(self) -> None:
object.__setattr__(self, "trace_id", _identifier(self.trace_id, "TRACE_ID_INVALID"))
object.__setattr__(self, "span_id", _identifier(self.span_id, "TRACE_SPAN_ID_INVALID"))
for name in ("parent_span_id", "correlation_id", "causation_id", "task_id", "claim_id", "message_id"):
value = _clean(getattr(self, name))
if value is not None and IDENTIFIER_RE.fullmatch(value) is None:
raise GovernanceError("TRACE_LINEAGE_IDENTIFIER_INVALID", field=name)
object.__setattr__(self, name, value)
def child(self, *, span_id: str, actor: GovernanceActor | Mapping[str, Any], **lineage: Any) -> "TraceContext":
authorize_scope(actor, self.scope)
for name in ("trace_id", "task_id", "claim_id", "message_id", "correlation_id"):
supplied = _clean(lineage.get(name))
expected = getattr(self, name)
if supplied is not None and expected is not None and supplied != expected:
raise GovernanceError("TRACE_LINEAGE_CONTINUITY", field=name)
requested_scope = lineage.pop("scope", self.scope)
requested_scope = requested_scope if isinstance(requested_scope, GovernanceScope) else GovernanceScope.from_value(requested_scope)
if requested_scope != self.scope:
raise GovernanceError("TRACE_SCOPE_MISMATCH_DENY")
return TraceContext(
trace_id=self.trace_id,
span_id=span_id,
parent_span_id=self.span_id,
correlation_id=self.correlation_id,
causation_id=_clean(lineage.get("causation_id")) or self.span_id,
task_id=self.task_id,
claim_id=self.claim_id,
message_id=self.message_id,
scope=self.scope,
)
def as_dict(self) -> dict[str, Any]:
return {
"trace_id": self.trace_id,
"span_id": self.span_id,
"parent_span_id": self.parent_span_id,
"correlation_id": self.correlation_id,
"causation_id": self.causation_id,
"task_id": self.task_id,
"claim_id": self.claim_id,
"message_id": self.message_id,
**self.scope.as_dict(),
}
def bind_trace_lineage(
*,
actor: GovernanceActor | Mapping[str, Any],
task: Any,
message: Any,
trace_id: str,
span_id: str,
parent: TraceContext | None = None,
) -> TraceContext:
actor = GovernanceActor.from_value(actor)
task_scope = GovernanceScope.from_value(task)
message_scope = GovernanceScope.from_value(message)
if task_scope != message_scope:
raise GovernanceError("TRACE_SCOPE_MISMATCH_DENY")
authorize_scope(actor, task_scope)
def get(value: Any, name: str) -> Any:
return value.get(name) if isinstance(value, Mapping) else getattr(value, name, None)
task_id = _clean(get(task, "task_id"))
message_task = _clean(get(message, "task_id"))
if task_id and message_task and task_id != message_task:
raise GovernanceError("TRACE_LINEAGE_CONTINUITY", field="task_id")
context = TraceContext(
trace_id=trace_id,
span_id=span_id,
parent_span_id=parent.span_id if parent else None,
correlation_id=_clean(get(task, "correlation_id")) or _clean(get(message, "correlation_id")),
causation_id=_clean(get(message, "causation_id")) or (parent.span_id if parent else None),
task_id=task_id or message_task,
claim_id=_clean(get(task, "claim_id")) or _clean(get(message, "claim_id")),
message_id=_clean(get(message, "envelope_id")) or _clean(get(message, "id")),
scope=task_scope,
)
if parent is not None:
if parent.trace_id != context.trace_id or parent.scope != context.scope:
raise GovernanceError("TRACE_LINEAGE_CONTINUITY")
if parent.task_id and context.task_id and parent.task_id != context.task_id:
raise GovernanceError("TRACE_LINEAGE_CONTINUITY", field="task_id")
return context
@dataclass(frozen=True)
class ToolDescriptor:
name: str
kind: str
scope: GovernanceScope
required_capabilities: tuple[str, ...] = ()
credential_ref: str | None = None
def __post_init__(self) -> None:
object.__setattr__(self, "name", _identifier(self.name, "TOOL_NAME_INVALID"))
object.__setattr__(self, "kind", (_clean(self.kind) or "").lower())
if self.kind not in {"tool", "mcp"}:
raise GovernanceError("TOOL_KIND_INVALID")
object.__setattr__(self, "required_capabilities", _scope_tuple(self.required_capabilities))
if self.credential_ref is not None and REFERENCE_RE.fullmatch(self.credential_ref) is None:
raise GovernanceError("CREDENTIAL_SECRET_EXPOSURE_DENY")
@dataclass(frozen=True)
class CredentialGrant:
grant_id: str
credential_ref: str
scope: GovernanceScope
operations: tuple[str, ...]
task_id: str | None = None
tool_name: str | None = None
expires_at: str | None = None
def __post_init__(self) -> None:
object.__setattr__(self, "grant_id", _identifier(self.grant_id, "CREDENTIAL_GRANT_ID_INVALID"))
if REFERENCE_RE.fullmatch(self.credential_ref or "") is None:
raise GovernanceError("CREDENTIAL_SECRET_EXPOSURE_DENY")
object.__setattr__(self, "operations", _scope_tuple(self.operations))
if not self.operations:
raise GovernanceError("CREDENTIAL_OPERATIONS_REQUIRED")
if self.expires_at is not None:
parsed = datetime.fromisoformat(self.expires_at.replace("Z", "+00:00"))
if parsed.tzinfo is None:
raise GovernanceError("CREDENTIAL_EXPIRY_INVALID")
def authorize(
self,
actor: GovernanceActor | Mapping[str, Any],
*,
operation: str,
task_id: str | None = None,
tool_name: str | None = None,
now: datetime | None = None,
) -> str:
try:
authorize_scope(actor, self.scope)
except GovernanceError as exc:
raise GovernanceError("CREDENTIAL_SCOPE_DENY", field=exc.field) from exc
if operation not in self.operations:
raise GovernanceError("CREDENTIAL_SCOPE_DENY", field="operation")
if self.task_id is not None and self.task_id != _clean(task_id):
raise GovernanceError("CREDENTIAL_SCOPE_DENY", field="task_id")
if self.tool_name is not None and self.tool_name != _clean(tool_name):
raise GovernanceError("CREDENTIAL_SCOPE_DENY", field="tool_name")
if self.expires_at is not None:
current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
expiry = datetime.fromisoformat(self.expires_at.replace("Z", "+00:00")).astimezone(timezone.utc)
if current >= expiry:
raise GovernanceError("CREDENTIAL_SCOPE_DENY", field="expires_at")
return self.credential_ref
def public(self) -> dict[str, Any]:
return {
"grant_id": self.grant_id,
"credential_ref": self.credential_ref,
"operations": list(self.operations),
"task_id": self.task_id,
"tool_name": self.tool_name,
"expires_at": self.expires_at,
**self.scope.as_dict(),
}
_SCHEMA = """
CREATE TABLE IF NOT EXISTS governance_schema_metadata (
singleton INTEGER PRIMARY KEY CHECK(singleton=1),
schema_version INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS governed_tools (
name TEXT PRIMARY KEY,
kind TEXT NOT NULL,
scope_json TEXT NOT NULL,
required_capabilities_json TEXT NOT NULL,
credential_ref TEXT,
descriptor_sha256 TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS governed_trace_spans (
trace_id TEXT NOT NULL,
span_id TEXT NOT NULL,
parent_span_id TEXT,
context_json TEXT NOT NULL,
context_sha256 TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY(trace_id, span_id)
);
CREATE TABLE IF NOT EXISTS governed_credential_grants (
grant_id TEXT PRIMARY KEY,
grant_json TEXT NOT NULL,
grant_sha256 TEXT NOT NULL,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS governed_tool_invocations (
invocation_key TEXT PRIMARY KEY,
request_sha256 TEXT NOT NULL,
tool_name TEXT NOT NULL,
task_id TEXT NOT NULL,
claim_id TEXT NOT NULL,
trace_id TEXT NOT NULL,
span_id TEXT NOT NULL,
status TEXT NOT NULL,
receipt_sha256 TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS governance_audit_evidence (
sequence INTEGER PRIMARY KEY AUTOINCREMENT,
evidence_id TEXT NOT NULL UNIQUE,
action TEXT NOT NULL,
actor_identity TEXT NOT NULL,
trace_id TEXT,
span_id TEXT,
resource_id TEXT NOT NULL,
outcome TEXT NOT NULL,
details_json TEXT NOT NULL,
previous_sha256 TEXT NOT NULL,
evidence_sha256 TEXT NOT NULL,
created_at TEXT NOT NULL
);
"""
class GovernanceStore:
def __init__(self, path: str | Path) -> None:
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
with self._connection() as connection:
has_metadata = connection.execute(
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='governance_schema_metadata'"
).fetchone()
if has_metadata:
row = connection.execute(
"SELECT schema_version FROM governance_schema_metadata WHERE singleton=1"
).fetchone()
if row is not None and int(row[0]) != 1:
raise GovernanceError("GOVERNANCE_SCHEMA_VERSION_UNSUPPORTED")
connection.executescript(_SCHEMA)
row = connection.execute(
"SELECT schema_version FROM governance_schema_metadata WHERE singleton=1"
).fetchone()
if row is None:
connection.execute(
"INSERT INTO governance_schema_metadata(singleton,schema_version) VALUES(1,1)"
)
connection.commit()
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@contextmanager
def _connection(self):
connection = self._connect()
try:
yield connection
connection.commit()
except Exception:
connection.rollback()
raise
finally:
connection.close()
def _audit(
self,
connection: sqlite3.Connection,
*,
action: str,
actor: GovernanceActor,
resource_id: str,
outcome: str,
trace: TraceContext | None,
details: Mapping[str, Any],
) -> dict[str, Any]:
previous = connection.execute(
"SELECT evidence_sha256 FROM governance_audit_evidence ORDER BY sequence DESC LIMIT 1"
).fetchone()
previous_sha = str(previous[0]) if previous else ""
created_at = _now()
material = {
"action": action,
"actor_identity": actor.identity,
"trace_id": trace.trace_id if trace else None,
"span_id": trace.span_id if trace else None,
"resource_id": resource_id,
"outcome": outcome,
"details": dict(details),
"previous_sha256": previous_sha,
"created_at": created_at,
}
digest = _sha(material)
evidence_id = "gev-" + digest[:32]
connection.execute(
"""INSERT INTO governance_audit_evidence
(evidence_id,action,actor_identity,trace_id,span_id,resource_id,outcome,
details_json,previous_sha256,evidence_sha256,created_at)
VALUES (?,?,?,?,?,?,?,?,?,?,?)""",
(
evidence_id, action, actor.identity, material["trace_id"], material["span_id"],
resource_id, outcome, _canonical(material["details"]), previous_sha, digest, created_at,
),
)
return {"evidence_id": evidence_id, "evidence_sha256": digest, **material}
def record_trace(self, trace: TraceContext, actor: GovernanceActor | Mapping[str, Any]) -> dict[str, Any]:
actor = GovernanceActor.from_value(actor)
authorize_scope(actor, trace.scope)
context = trace.as_dict()
digest = _sha(context)
with self._connection() as connection:
existing = connection.execute(
"SELECT context_sha256 FROM governed_trace_spans WHERE trace_id=? AND span_id=?",
(trace.trace_id, trace.span_id),
).fetchone()
if existing is not None and existing[0] != digest:
raise GovernanceError("TRACE_LINEAGE_CONFLICT")
if existing is not None:
return {"context": context, "context_sha256": digest, "idempotent": True}
if trace.parent_span_id:
parent = connection.execute(
"SELECT context_json FROM governed_trace_spans WHERE trace_id=? AND span_id=?",
(trace.trace_id, trace.parent_span_id),
).fetchone()
if parent is None:
raise GovernanceError("TRACE_LINEAGE_CONTINUITY", field="parent_span_id")
parent_context = json.loads(parent[0])
if GovernanceScope.from_value(parent_context) != trace.scope:
raise GovernanceError("TRACE_SCOPE_MISMATCH_DENY")
connection.execute(
"INSERT OR IGNORE INTO governed_trace_spans VALUES (?,?,?,?,?,?)",
(trace.trace_id, trace.span_id, trace.parent_span_id, _canonical(context), digest, _now()),
)
evidence = self._audit(
connection, action="TRACE_RECORDED", actor=actor,
resource_id=f"{trace.trace_id}:{trace.span_id}", outcome="ALLOW", trace=trace,
details={"context_sha256": digest},
)
return {"context": context, "context_sha256": digest, "evidence": evidence}
def register_tool(self, descriptor: ToolDescriptor, actor: GovernanceActor | Mapping[str, Any]) -> dict[str, Any]:
actor = GovernanceActor.from_value(actor)
if not actor.privileged:
raise GovernanceError("TOOL_MCP_AUTHZ_DENY")
public = {
"name": descriptor.name,
"kind": descriptor.kind,
"required_capabilities": list(descriptor.required_capabilities),
"credential_ref": descriptor.credential_ref,
**descriptor.scope.as_dict(),
}
digest = _sha(public)
with self._connection() as connection:
existing = connection.execute(
"SELECT descriptor_sha256 FROM governed_tools WHERE name=?", (descriptor.name,)
).fetchone()
if existing is not None and existing[0] != digest:
raise GovernanceError("TOOL_DESCRIPTOR_CONFLICT")
if existing is not None:
return {"descriptor": public, "descriptor_sha256": digest, "idempotent": True}
connection.execute(
"INSERT OR IGNORE INTO governed_tools VALUES (?,?,?,?,?,?)",
(descriptor.name, descriptor.kind, _canonical(descriptor.scope.as_dict()),
_canonical(descriptor.required_capabilities), descriptor.credential_ref, digest),
)
evidence = self._audit(
connection, action="TOOL_REGISTERED", actor=actor, resource_id=descriptor.name,
outcome="ALLOW", trace=None, details={"descriptor_sha256": digest},
)
return {"descriptor": public, "descriptor_sha256": digest, "evidence": evidence}
def authorize_tool(
self,
*,
actor: GovernanceActor | Mapping[str, Any],
tool_name: str,
trace: TraceContext,
requested_scope: GovernanceScope | Mapping[str, Any],
input_digest: str,
invocation_key: str,
) -> dict[str, Any]:
actor = GovernanceActor.from_value(actor)
invocation_key = _identifier(invocation_key, "TOOL_INVOCATION_KEY_INVALID")
if not re.fullmatch(r"[0-9a-f]{64}", input_digest):
raise GovernanceError("TOOL_INPUT_DIGEST_INVALID")
if trace.task_id is None or trace.claim_id is None:
raise GovernanceError("TOOL_MCP_AUTHZ_DENY", field="active_claim")
requested_scope = requested_scope if isinstance(requested_scope, GovernanceScope) else GovernanceScope.from_value(requested_scope)
if requested_scope != trace.scope:
raise GovernanceError("TOOL_SCOPE_PROPAGATION")
authorize_scope(actor, requested_scope)
with self._connection() as connection:
persisted_trace = connection.execute(
"SELECT context_sha256 FROM governed_trace_spans WHERE trace_id=? AND span_id=?",
(trace.trace_id, trace.span_id),
).fetchone()
if persisted_trace is None or persisted_trace[0] != _sha(trace.as_dict()):
raise GovernanceError("TOOL_MCP_AUTHZ_DENY", field="trace")
row = connection.execute("SELECT * FROM governed_tools WHERE name=?", (tool_name,)).fetchone()
if row is None:
raise GovernanceError("TOOL_MCP_AUTHZ_DENY")
descriptor_scope = GovernanceScope.from_value(json.loads(row["scope_json"]))
if descriptor_scope.scope_type == "tenant" and descriptor_scope != requested_scope:
raise GovernanceError("TOOL_MCP_AUTHZ_DENY")
required = set(json.loads(row["required_capabilities_json"]))
if not required.issubset(set(actor.capabilities)):
raise GovernanceError("TOOL_MCP_AUTHZ_DENY")
request_sha = _sha({
"tool_name": tool_name, "input_digest": input_digest,
"task_id": trace.task_id, "claim_id": trace.claim_id,
"trace_id": trace.trace_id, "span_id": trace.span_id,
"scope": requested_scope.as_dict(),
})
replay = connection.execute(
"SELECT * FROM governed_tool_invocations WHERE invocation_key=?",
(invocation_key,),
).fetchone()
if replay is not None:
if replay["request_sha256"] != request_sha:
raise GovernanceError("TOOL_INVOCATION_IDEMPOTENCY_CONFLICT")
if replay["status"] != "VERIFIED":
raise GovernanceError("TOOL_INVOCATION_RECONCILIATION_REQUIRED")
return {
"tool_name": tool_name, "trace_id": trace.trace_id,
"span_id": trace.span_id, "invocation_key": invocation_key,
"status": "VERIFIED", "receipt_sha256": replay["receipt_sha256"],
"idempotent": True,
}
created_at = _now()
connection.execute(
"""INSERT INTO governed_tool_invocations
(invocation_key,request_sha256,tool_name,task_id,claim_id,trace_id,span_id,
status,receipt_sha256,created_at,updated_at)
VALUES(?,?,?,?,?,?,?,?,?,?,?)""",
(invocation_key, request_sha, tool_name, trace.task_id, trace.claim_id,
trace.trace_id, trace.span_id, "AUTHORIZED", None, created_at, created_at),
)
evidence = self._audit(
connection, action="TOOL_AUTHORIZED", actor=actor, resource_id=tool_name,
outcome="ALLOW", trace=trace,
details={"input_sha256": input_digest, "scope_sha256": _sha(requested_scope.as_dict()),
"invocation_key": invocation_key},
)
return {
"tool_name": tool_name,
"trace_id": trace.trace_id,
"span_id": trace.span_id,
"invocation_key": invocation_key,
"status": "AUTHORIZED",
"credential_required": row["credential_ref"] is not None,
"evidence": evidence,
}
def record_tool_receipt(
self,
*,
invocation_key: str,
result_digest: str,
actor: GovernanceActor | Mapping[str, Any],
trace: TraceContext,
) -> dict[str, Any]:
actor = GovernanceActor.from_value(actor)
invocation_key = _identifier(invocation_key, "TOOL_INVOCATION_KEY_INVALID")
if not re.fullmatch(r"[0-9a-f]{64}", result_digest):
raise GovernanceError("TOOL_RECEIPT_DIGEST_INVALID")
authorize_scope(actor, trace.scope)
with self._connection() as connection:
row = connection.execute(
"SELECT * FROM governed_tool_invocations WHERE invocation_key=?",
(invocation_key,),
).fetchone()
if row is None or row["trace_id"] != trace.trace_id or row["span_id"] != trace.span_id:
raise GovernanceError("TOOL_RECEIPT_BINDING_INVALID")
if row["status"] == "VERIFIED":
if row["receipt_sha256"] != result_digest:
raise GovernanceError("TOOL_RECEIPT_CONFLICT")
return {"invocation_key": invocation_key, "status": "VERIFIED", "idempotent": True}
if row["status"] != "AUTHORIZED":
raise GovernanceError("TOOL_INVOCATION_RECONCILIATION_REQUIRED")
connection.execute(
"UPDATE governed_tool_invocations SET status='VERIFIED',receipt_sha256=?,updated_at=? "
"WHERE invocation_key=? AND status='AUTHORIZED'",
(result_digest, _now(), invocation_key),
)
evidence = self._audit(
connection, action="TOOL_RESULT_VERIFIED", actor=actor,
resource_id=row["tool_name"], outcome="ALLOW", trace=trace,
details={"invocation_key": invocation_key, "result_sha256": result_digest},
)
return {"invocation_key": invocation_key, "status": "VERIFIED", "evidence": evidence}
def store_grant(self, grant: CredentialGrant, actor: GovernanceActor | Mapping[str, Any]) -> dict[str, Any]:
actor = GovernanceActor.from_value(actor)
if not actor.privileged:
raise GovernanceError("CREDENTIAL_SCOPE_DENY")
public = grant.public()
digest = _sha(public)
with self._connection() as connection:
existing = connection.execute(
"SELECT grant_sha256 FROM governed_credential_grants WHERE grant_id=?", (grant.grant_id,)
).fetchone()
if existing is not None and existing[0] != digest:
raise GovernanceError("CREDENTIAL_GRANT_CONFLICT")
connection.execute(
"INSERT OR IGNORE INTO governed_credential_grants VALUES (?,?,?,?)",
(grant.grant_id, _canonical(public), digest, _now()),
)
evidence = self._audit(
connection, action="CREDENTIAL_GRANT_STORED", actor=actor,
resource_id=grant.grant_id, outcome="ALLOW", trace=None,
details={"grant_sha256": digest},
)
return {"grant": public, "grant_sha256": digest, "evidence": evidence}
def authorize_credential(
self,
*,
grant_id: str,
actor: GovernanceActor | Mapping[str, Any],
trace: TraceContext,
operation: str,
task_id: str | None,
tool_name: str | None,
now: datetime | None = None,
) -> dict[str, Any]:
actor = GovernanceActor.from_value(actor)
with self._connection() as connection:
row = connection.execute(
"SELECT grant_json FROM governed_credential_grants WHERE grant_id=?", (grant_id,)
).fetchone()
if row is None:
raise GovernanceError("CREDENTIAL_SCOPE_DENY")
raw = json.loads(row[0])
grant = CredentialGrant(
grant_id=raw["grant_id"], credential_ref=raw["credential_ref"],
scope=GovernanceScope.from_value(raw), operations=tuple(raw["operations"]),
task_id=raw.get("task_id"), tool_name=raw.get("tool_name"),
expires_at=raw.get("expires_at"),
)
if trace.scope != grant.scope:
raise GovernanceError("CREDENTIAL_SCOPE_DENY", field="trace_scope")
reference = grant.authorize(
actor, operation=operation, task_id=task_id, tool_name=tool_name, now=now
)
evidence = self._audit(
connection, action="CREDENTIAL_AUTHORIZED", actor=actor,
resource_id=grant_id, outcome="ALLOW", trace=trace,
details={"operation": operation, "task_id": task_id, "tool_name": tool_name,
"credential_ref_sha256": _sha(reference)},
)
return {"credential_ref": reference, "evidence": evidence}
def verify_audit_chain(self, required_actions: Iterable[str] = ()) -> bool:
ordered_actions: list[str] = []
previous = ""
with self._connection() as connection:
rows = connection.execute("SELECT * FROM governance_audit_evidence ORDER BY sequence").fetchall()
for row in rows:
details = json.loads(row["details_json"])
material = {
"action": row["action"], "actor_identity": row["actor_identity"],
"trace_id": row["trace_id"], "span_id": row["span_id"],
"resource_id": row["resource_id"], "outcome": row["outcome"],
"details": details, "previous_sha256": row["previous_sha256"],
"created_at": row["created_at"],
}
if row["previous_sha256"] != previous or row["evidence_sha256"] != _sha(material):
return False
previous = row["evidence_sha256"]
ordered_actions.append(row["action"])
position = 0
for required in required_actions:
try:
position = ordered_actions.index(required, position) + 1
except ValueError:
return False
return True
__all__ = [
"CredentialGrant", "GovernanceActor", "GovernanceError", "GovernanceScope",
"GovernanceStore", "ToolDescriptor", "TraceContext", "authorize_scope",
"bind_trace_lineage",
]
+46 -127
View File
@@ -1,9 +1,4 @@
"""Privacy and artifact helpers for the V2R1C6 release candidate.
The module is deliberately dependency-free. It provides deterministic redaction,
safe SHA-256 manifest verification, and a small loopback-only candidate probe.
It does not read credentials, contact production services, or mutate Git refs.
"""
"""Privacy, artifact, and runtime health helpers for V2R1C7."""
from __future__ import annotations
import argparse
@@ -14,152 +9,76 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from typing import Any, Mapping
_SECRET_KEY = re.compile(r"(?i)(?:token|secret|password|private[_-]?key|api[_-]?key)")
_SECRET_VALUE = re.compile(
r"(?i)(?:bearer\s+)[A-Za-z0-9._~+/=-]+|"
r"(?:token|secret|password|private[_-]?key|api[_-]?key)\s*[:=]\s*[^\s,;]+"
)
from deployment_state import RuntimeDeploymentState
_SECRET_KEY = re.compile(r"(?i)(?:token|secret|password|private[_-]?key|api[_-]?key)")
_SECRET_VALUE = re.compile(r"(?i)(?:bearer\s+)[A-Za-z0-9._~+/=-]+|(?:token|secret|password|private[_-]?key|api[_-]?key)\s*[:=]\s*[^\s,;]+")
def redact_text(value: str) -> str:
"""Replace common bearer and key/value secret forms without logging them."""
if not isinstance(value, str):
raise TypeError("text_required")
if not isinstance(value, str): raise TypeError("text_required")
return _SECRET_VALUE.sub("[REDACTED]", value)
def redact_value(value: Any, *, key: str = "") -> Any:
"""Recursively redact secret-bearing mapping keys and text values."""
if isinstance(value, Mapping):
result: dict[str, Any] = {}
for name, item in value.items():
name_text = str(name)
if _SECRET_KEY.search(name_text):
result[name_text] = "[REDACTED]"
else:
result[name_text] = redact_value(item, key=name_text)
return result
if isinstance(value, list):
return [redact_value(item, key=key) for item in value]
if isinstance(value, tuple):
return [redact_value(item, key=key) for item in value]
if isinstance(value, str):
return "[REDACTED]" if _SECRET_KEY.search(key) else redact_text(value)
return {str(k): ("[REDACTED]" if _SECRET_KEY.search(str(k)) else redact_value(v, key=str(k))) for k,v in value.items()}
if isinstance(value, list): return [redact_value(v, key=key) for v in value]
if isinstance(value, tuple): return [redact_value(v, key=key) for v in value]
if isinstance(value, str): return "[REDACTED]" if _SECRET_KEY.search(key) else redact_text(value)
return value
def sha256_file(path: Path) -> str:
"""Return the SHA-256 of a regular file without following symlink inputs."""
if path.is_symlink() or not path.is_file():
raise ValueError("regular_file_required")
digest = hashlib.sha256()
if path.is_symlink() or not path.is_file(): raise ValueError("regular_file_required")
digest=hashlib.sha256()
with path.open("rb") as stream:
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
digest.update(chunk)
for chunk in iter(lambda: stream.read(1024*1024), b""): digest.update(chunk)
return digest.hexdigest()
def _safe_relative(root: Path, name: str) -> Path:
relative = Path(name)
if relative.is_absolute() or ".." in relative.parts or name in {"", "."}:
raise ValueError("unsafe_manifest_path")
candidate = (root / relative).resolve()
candidate.relative_to(root.resolve())
return candidate
relative=Path(name)
if relative.is_absolute() or ".." in relative.parts or name in {"", "."}: raise ValueError("unsafe_manifest_path")
candidate=(root/relative).resolve(); candidate.relative_to(root.resolve()); return candidate
def verify_manifest(root: Path, manifest: Path | None = None) -> dict[str, Any]:
"""Verify sha256 path entries and return a non-sensitive summary."""
root = root.resolve()
manifest = manifest or root / "artifacts" / "MANIFEST.sha256"
if manifest.is_symlink() or not manifest.is_file():
raise ValueError("manifest_missing")
checked = 0
root=root.resolve(); manifest=manifest or root/"artifacts"/"MANIFEST.sha256"
if manifest.is_symlink() or not manifest.is_file(): raise ValueError("manifest_missing")
checked=0
for raw in manifest.read_text(encoding="utf-8").splitlines():
line = raw.strip()
if not line:
continue
parts = line.split(maxsplit=1)
if len(parts) != 2 or not re.fullmatch(r"[0-9a-f]{64}", parts[0]):
raise ValueError("manifest_entry_invalid")
expected, name = parts
if name.startswith("*"):
name = name[1:]
path = _safe_relative(root, name)
if sha256_file(path) != expected:
raise ValueError("manifest_hash_mismatch")
checked += 1
if checked == 0:
raise ValueError("manifest_empty")
return {"status": "PASS", "entries": checked}
line=raw.strip()
if not line: continue
parts=line.split(maxsplit=1)
if len(parts)!=2 or not re.fullmatch(r"[0-9a-f]{64}", parts[0]): raise ValueError("manifest_entry_invalid")
expected,name=parts; name=name[1:] if name.startswith("*") else name
if sha256_file(_safe_relative(root,name))!=expected: raise ValueError("manifest_hash_mismatch")
checked+=1
if not checked: raise ValueError("manifest_empty")
return {"status":"PASS","entries":checked}
def load_release_metadata(path: Path) -> dict[str, Any]:
metadata = json.loads(path.read_text(encoding="utf-8"))
if metadata.get("version") != "V2R1C6":
raise ValueError("version_mismatch")
if metadata.get("production_deployed") is not False:
raise ValueError("production_change_not_allowed")
if metadata.get("tag_created") is not False:
raise ValueError("tag_change_not_allowed")
if metadata.get("baseline_changed") is not False:
raise ValueError("baseline_change_not_allowed")
metadata=json.loads(path.read_text(encoding="utf-8"))
if metadata.get("version") not in {"V2R1C6","V2R1C7"}: raise ValueError("version_mismatch")
if any(metadata.get(k) is not False for k in ("production_deployed","tag_created","baseline_changed")): raise ValueError("production_change_not_allowed")
return metadata
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--root", type=Path, default=Path("."))
parser.add_argument("--manifest", type=Path)
parser.add_argument("--serve", action="store_true")
parser.add_argument("--host", default="127.0.0.1")
parser.add_argument("--port", type=int, default=18926)
args = parser.parse_args()
if args.serve:
serve(args.host, args.port)
return 0
result = verify_manifest(args.root, args.manifest)
print(json.dumps(result, sort_keys=True))
return 0
def health_payload() -> dict[str, Any]:
return {
"status": "ok",
"version": "V2R1C6",
"production_deployed": False,
"gitops_writes_enabled": False,
}
def health_payload(state_provider: RuntimeDeploymentState | None = None) -> dict[str, Any]:
return (state_provider or RuntimeDeploymentState.from_environment()).health_payload()
class _HealthHandler(BaseHTTPRequestHandler):
def do_GET(self) -> None:
if self.path not in {"/", "/health", "/healthz"}:
self.send_error(404)
return
body = json.dumps(health_payload(), sort_keys=True).encode("utf-8")
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, _format: str, *_args: object) -> None:
return
if self.path not in {"/","/health","/healthz"}: self.send_error(404); return
body=json.dumps(health_payload(), sort_keys=True).encode()
self.send_response(200); self.send_header("Content-Type","application/json"); self.send_header("Content-Length",str(len(body))); self.end_headers(); self.wfile.write(body)
def log_message(self, _format: str, *_args: object) -> None: return
def serve(host: str, port: int) -> None:
if host not in {"127.0.0.1", "::1", "localhost"}:
raise ValueError("loopback_only_listener_required")
if not 1024 <= port <= 65535:
raise ValueError("invalid_listener_port")
server = ThreadingHTTPServer((host, port), _HealthHandler)
try:
server.serve_forever()
finally:
server.server_close()
if host not in {"127.0.0.1","::1","localhost"}: raise ValueError("loopback_only_listener_required")
if not 1024 <= port <= 65535: raise ValueError("invalid_listener_port")
server=ThreadingHTTPServer((host,port),_HealthHandler)
try: server.serve_forever()
finally: server.server_close()
def main() -> int:
parser=argparse.ArgumentParser(); parser.add_argument("--root",type=Path,default=Path(".")); parser.add_argument("--manifest",type=Path); parser.add_argument("--serve",action="store_true"); parser.add_argument("--host",default="127.0.0.1"); parser.add_argument("--port",type=int,default=18927); args=parser.parse_args()
if args.serve: serve(args.host,args.port); return 0
print(json.dumps(verify_manifest(args.root,args.manifest),sort_keys=True)); return 0
if __name__ == "__main__":
raise SystemExit(main())
if __name__ == "__main__": raise SystemExit(main())
@@ -0,0 +1,5 @@
"""V2R1C7 production health entry point; state is install/runtime supplied."""
from v2r1c6_privacy_tools import main
if __name__ == "__main__":
raise SystemExit(main())
+455
View File
@@ -0,0 +1,455 @@
#!/usr/bin/env python3
"""Concrete V2R1C7 production deployment adapter.
The adapter consumes only an already approved immutable artifact. It never
writes Git refs, Registry, or Baseline. A rollback only tears down the V2R1C7
runtime and leaves V2R1C5/V2R1C6 untouched.
"""
from __future__ import annotations
import argparse
import grp
import hashlib
import json
import os
import pwd
import shutil
import stat
import subprocess
import sys
import tarfile
import tempfile
import time
import urllib.error
import urllib.request
from pathlib import Path
from typing import Any, Mapping
VERSION = "V2R1C7"
SERVICE = "aios-v2r1c7-production.service"
PORT = 18927
RUNTIME_USER = "aios-v2r1c7"
RUNTIME_GROUP = "aios-v2r1c7"
REQUIRED_MEMBERS = {
"payload/VERSION",
"payload/active-runtime/app/deployment_state.py",
"payload/active-runtime/app/v2r1c6_privacy_tools.py",
"payload/active-runtime/app/v2r1c7_health_service.py",
"payload/deploy/aios-v2/systemd/aios-v2r1c7-production.service",
"payload/deploy/aios-v2/v2r1c7-runtime-config.json",
"payload/deploy/aios-v2/v2r1c7-rollback.json",
"payload/scripts/probe_v2r1c7_health.py",
"payload/scripts/rollback_v2r1c7_release.sh",
"payload/BUILD-METADATA.json",
"payload/ARTIFACT-MANIFEST.json",
"payload/ROLLBACK-METADATA.json",
}
def load_request() -> dict[str, Any]:
value = json.load(sys.stdin)
if not isinstance(value, dict):
raise ValueError("request_object_required")
if value.get("schema") != "aios.production-executor-request.v1":
raise ValueError("request_schema_invalid")
if value.get("version") != VERSION:
raise ValueError("version_mismatch")
if value.get("production_service") != SERVICE:
raise ValueError("service_mismatch")
if int(value.get("production_port", -1)) != PORT:
raise ValueError("port_mismatch")
return value
def sha256_file(path: Path) -> str:
if path.is_symlink() or not path.is_file():
raise ValueError("artifact_regular_file_required")
digest = hashlib.sha256()
with path.open("rb") as stream:
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def atomic_json(path: Path, value: Mapping[str, Any], *, mode: int, gid: int | None = None) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_name(path.name + ".tmp")
temporary.write_text(
json.dumps(dict(value), sort_keys=True, separators=(",", ":")) + "\n",
encoding="utf-8",
)
os.chmod(temporary, mode)
if gid is not None:
os.chown(temporary, 0, gid)
os.replace(temporary, path)
def atomic_text(path: Path, text: str, *, mode: int, gid: int | None = None) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_name(path.name + ".tmp")
temporary.write_text(text, encoding="utf-8")
os.chmod(temporary, mode)
if gid is not None:
os.chown(temporary, 0, gid)
os.replace(temporary, path)
def systemctl(*args: str, check: bool = True) -> subprocess.CompletedProcess[str]:
return subprocess.run(
["systemctl", *args],
check=check,
capture_output=True,
text=True,
timeout=45,
env={"PATH": os.environ.get("PATH", "")},
)
def identity() -> tuple[int, int]:
try:
user = pwd.getpwnam(RUNTIME_USER)
group = grp.getgrnam(RUNTIME_GROUP)
except KeyError as exc:
raise RuntimeError("RUNTIME_IDENTITY_MISSING") from exc
if user.pw_gid != group.gr_gid:
raise RuntimeError("RUNTIME_IDENTITY_GROUP_MISMATCH")
return user.pw_uid, group.gr_gid
def safe_extract(artifact: Path, destination: Path) -> set[str]:
with tempfile.TemporaryDirectory(prefix="aios-v2r1c7-unpack-") as temp:
tar_path = Path(temp) / "release.tar"
with tar_path.open("wb") as output:
completed = subprocess.run(
["zstd", "-dc", str(artifact)],
check=False,
stdout=output,
stderr=subprocess.PIPE,
timeout=60,
env={"PATH": os.environ.get("PATH", "")},
)
if completed.returncode != 0:
raise RuntimeError("ARTIFACT_DECOMPRESSION_FAILED")
with tarfile.open(tar_path, "r:") as archive:
members = archive.getmembers()
names = {m.name for m in members}
missing = REQUIRED_MEMBERS - names
if missing:
raise RuntimeError("ARTIFACT_MEMBERS_MISSING:" + ",".join(sorted(missing)))
for member in members:
relative = Path(member.name)
if (
relative.is_absolute()
or ".." in relative.parts
or member.issym()
or member.islnk()
or not (member.isfile() or member.isdir())
):
raise RuntimeError("UNSAFE_ARTIFACT_MEMBER:" + member.name)
if not member.name.startswith("payload/"):
raise RuntimeError("UNEXPECTED_ARTIFACT_ROOT:" + member.name)
payload = destination
payload.mkdir(parents=True, exist_ok=True)
for member in members:
rel = Path(member.name).relative_to("payload")
if str(rel) == ".":
continue
target = payload / rel
resolved_parent = target.parent.resolve()
resolved_parent.relative_to(payload.resolve())
if member.isdir():
target.mkdir(parents=True, exist_ok=True)
os.chmod(target, 0o755)
continue
target.parent.mkdir(parents=True, exist_ok=True)
source = archive.extractfile(member)
if source is None:
raise RuntimeError("ARTIFACT_MEMBER_READ_FAILED")
with target.open("wb") as output:
shutil.copyfileobj(source, output)
# Scripts must be executable; all other payload files are read-only.
mode = 0o755 if rel.parts and rel.parts[0] == "scripts" else 0o644
os.chmod(target, mode)
return names
def health(url: str, expected: Mapping[str, Any]) -> dict[str, Any]:
try:
with urllib.request.urlopen(url, timeout=5) as response:
body = json.loads(response.read().decode("utf-8"))
status = response.status
except (urllib.error.URLError, TimeoutError, ValueError):
return {"status": "UNAVAILABLE"}
exact = (
status == 200
and body.get("version") == expected["version"]
and body.get("commit_oid") == expected["target_oid"]
and body.get("artifact_sha256") == expected["artifact_sha256"]
and body.get("deployment_state") == "ACTIVE"
and body.get("production_deployed") is True
)
if not exact:
return {
"status": "CONFLICT",
"http_status": status,
"version": body.get("version"),
"target_oid": body.get("commit_oid"),
"artifact_sha256": body.get("artifact_sha256"),
"deployment_state": body.get("deployment_state"),
"production_deployed": body.get("production_deployed"),
}
return {
"status": "ACTIVE_VERIFIED",
"http_status": 200,
"version": body["version"],
"target_oid": body["commit_oid"],
"artifact_sha256": body["artifact_sha256"],
"deployment_state": body["deployment_state"],
"production_deployed": True,
}
def paths(args: argparse.Namespace, value: Mapping[str, Any]) -> dict[str, Path]:
release_id = (
f"{VERSION}-{str(value['target_oid'])[:12]}-"
f"{str(value['artifact_sha256'])[:12]}"
)
return {
"release_dir": args.release_root / release_id,
"current_link": args.current_link,
"env_file": args.env_file,
"state_file": args.state_file,
"unit_file": args.unit_file,
}
def probe(args: argparse.Namespace, value: Mapping[str, Any]) -> dict[str, Any]:
p = paths(args, value)
url = f"http://127.0.0.1:{PORT}/health"
active = systemctl("is-active", "--quiet", SERVICE, check=False).returncode == 0
if not active:
# A stale exact release directory is safe to reuse; it is not a deployment.
return {
"status": "NOT_APPLIED",
"version": VERSION,
"target_oid": value["target_oid"],
"artifact_sha256": value["artifact_sha256"],
"production_deployed": False,
"deployment_state": "ROLLED_BACK"
if p["state_file"].exists()
and "ROLLED_BACK" in p["state_file"].read_text(encoding="utf-8")
else "ABSENT",
}
result = health(url, value)
if result["status"] == "UNAVAILABLE":
return {
"status": "CONFLICT",
"version": VERSION,
"target_oid": value["target_oid"],
"artifact_sha256": value["artifact_sha256"],
"reason": "service_active_health_unavailable",
}
return result
def install_release(args: argparse.Namespace, value: Mapping[str, Any]) -> dict[str, Any]:
_, gid = identity()
artifact = Path(str(value["artifact_path"]))
if artifact.stat().st_size != int(value["artifact_size"]):
raise RuntimeError("ARTIFACT_SIZE_MISMATCH")
if sha256_file(artifact) != value["artifact_sha256"]:
raise RuntimeError("ARTIFACT_DIGEST_MISMATCH")
p = paths(args, value)
release_dir = p["release_dir"]
marker = release_dir / "DEPLOYMENT-BINDING.json"
marker_value = {
"schema": "aios.runtime-release-binding.v1",
"version": VERSION,
"target_oid": value["target_oid"],
"artifact_sha256": value["artifact_sha256"],
"approval_plan_id": value["approval_plan_id"],
}
if release_dir.exists():
if not marker.is_file():
raise RuntimeError("EXISTING_RELEASE_DIR_UNTRUSTED")
current_marker = json.loads(marker.read_text(encoding="utf-8"))
if current_marker != marker_value:
raise RuntimeError("EXISTING_RELEASE_DIR_BINDING_CONFLICT")
else:
release_dir.parent.mkdir(parents=True, exist_ok=True)
temp_dir = Path(
tempfile.mkdtemp(prefix=release_dir.name + ".", dir=str(release_dir.parent))
)
try:
safe_extract(artifact, temp_dir)
atomic_json(
temp_dir / "DEPLOYMENT-BINDING.json",
marker_value,
mode=0o644,
)
os.rename(temp_dir, release_dir)
except Exception:
shutil.rmtree(temp_dir, ignore_errors=True)
raise
if (release_dir / "VERSION").read_text(encoding="utf-8").strip() != VERSION:
raise RuntimeError("RELEASE_VERSION_FILE_MISMATCH")
# Only replace our dedicated V2R1C7 link. Never touch previous versions.
current_link = p["current_link"]
if current_link.exists() and not current_link.is_symlink():
raise RuntimeError("CURRENT_RELEASE_PATH_NOT_SYMLINK")
if current_link.is_symlink():
existing = current_link.resolve()
if existing != release_dir.resolve():
raise RuntimeError("CURRENT_RELEASE_LINK_CONFLICT")
else:
current_link.parent.mkdir(parents=True, exist_ok=True)
new_link = current_link.with_name(current_link.name + ".new")
try:
new_link.unlink()
except FileNotFoundError:
pass
new_link.symlink_to(release_dir)
os.replace(new_link, current_link)
state_value = {
"version": VERSION,
"commit_oid": value["target_oid"],
"artifact_sha256": value["artifact_sha256"],
"deployment_state": "ACTIVE",
}
atomic_json(p["state_file"], state_value, mode=0o640, gid=gid)
env = (
f"RELEASE_VERSION={VERSION}\n"
f"TARGET_OID={value['target_oid']}\n"
f"ARTIFACT_SHA256={value['artifact_sha256']}\n"
"DEPLOYMENT_STATE=ACTIVE\n"
f"AIOS_V2R1C7_PORT={PORT}\n"
f"DEPLOYMENT_STATE_FILE={p['state_file']}\n"
"AIOS_GITOPS_WRITES_ENABLED=false\n"
)
atomic_text(p["env_file"], env, mode=0o640, gid=gid)
unit_source = (
release_dir
/ "deploy"
/ "aios-v2"
/ "systemd"
/ "aios-v2r1c7-production.service"
)
unit_text = unit_source.read_text(encoding="utf-8")
if "aios-v2r1c7-production.service" in unit_text:
# Description text is irrelevant; service identity is the destination.
pass
atomic_text(p["unit_file"], unit_text, mode=0o644)
verify = subprocess.run(
["systemd-analyze", "verify", str(p["unit_file"])],
check=False,
capture_output=True,
text=True,
timeout=30,
env={"PATH": os.environ.get("PATH", "")},
)
if verify.returncode != 0:
raise RuntimeError("PRODUCTION_SYSTEMD_VERIFY_FAILED")
systemctl("daemon-reload")
systemctl("enable", SERVICE)
systemctl("restart", SERVICE)
deadline = time.time() + args.health_timeout
result = {"status": "UNAVAILABLE"}
while time.time() < deadline:
result = health(f"http://127.0.0.1:{PORT}/health", value)
if result["status"] == "ACTIVE_VERIFIED":
break
if result["status"] == "CONFLICT":
break
time.sleep(1)
if result["status"] != "ACTIVE_VERIFIED":
raise RuntimeError("PRODUCTION_HEALTH_VERIFY_FAILED")
# Restart recovery is part of production acceptance, not just staging.
systemctl("restart", SERVICE)
deadline = time.time() + args.health_timeout
while time.time() < deadline:
result = health(f"http://127.0.0.1:{PORT}/health", value)
if result["status"] == "ACTIVE_VERIFIED":
break
if result["status"] == "CONFLICT":
break
time.sleep(1)
if result["status"] != "ACTIVE_VERIFIED":
raise RuntimeError("PRODUCTION_RESTART_RECOVERY_FAILED")
return result
def rollback(args: argparse.Namespace, value: Mapping[str, Any]) -> dict[str, Any]:
_, gid = identity()
p = paths(args, value)
systemctl("stop", SERVICE, check=False)
systemctl("disable", SERVICE, check=False)
state = {
"version": VERSION,
"commit_oid": value["target_oid"],
"artifact_sha256": value["artifact_sha256"],
"deployment_state": "ROLLED_BACK",
}
atomic_json(p["state_file"], state, mode=0o640, gid=gid)
return {
"status": "ROLLED_BACK",
"version": VERSION,
"target_oid": value["target_oid"],
"artifact_sha256": value["artifact_sha256"],
"deployment_state": "ROLLED_BACK",
"production_deployed": False,
}
def main() -> int:
p = argparse.ArgumentParser()
p.add_argument("--release-root", type=Path, default=Path("/opt/aios/releases"))
p.add_argument("--current-link", type=Path, default=Path("/opt/aios/v2r1c7"))
p.add_argument("--env-file", type=Path, default=Path("/etc/aios/aios-v2r1c7.env"))
p.add_argument(
"--state-file",
type=Path,
default=Path("/var/lib/aios/v2r1c7/deployment-state.json"),
)
p.add_argument(
"--unit-file",
type=Path,
default=Path("/etc/systemd/system/aios-v2r1c7-production.service"),
)
p.add_argument("--health-timeout", type=int, default=30)
args = p.parse_args()
value = load_request()
operation = value.get("operation")
if operation == "probe":
result = probe(args, value)
elif operation == "apply":
result = install_release(args, value)
elif operation == "rollback":
result = rollback(args, value)
else:
raise ValueError("unsupported_operation")
print(json.dumps(result, sort_keys=True))
return 0
if __name__ == "__main__":
try:
raise SystemExit(main())
except Exception as exc:
print(
json.dumps(
{"status": "FAIL_CLOSED", "error": type(exc).__name__, "reason": str(exc)},
sort_keys=True,
)
)
raise SystemExit(2)
@@ -0,0 +1,135 @@
"""V2R2C1 adapter using the existing V2R1C7 deployment primitives.
The service identity and loopback port remain stable for a safe cutover. The
new release is installed under its own immutable directory and keeps the
previous V2R1C7 release available for an explicit, owner-approved rollback.
"""
from __future__ import annotations
import json
import os
import shutil
import sys
from argparse import Namespace
from pathlib import Path
from typing import Any, Mapping
import v2r1c7_production_adapter as base
VERSION = "V2R2C1"
SERVICE = "aios-v2r1c7-production.service"
PORT = 18927
RELEASE_ROOT = Path("/opt/aios/releases")
CURRENT_LINK = Path("/opt/aios/v2r2c1")
ENV_FILE = Path("/etc/aios/aios-v2r2c1.env")
STATE_FILE = Path("/var/lib/aios/v2r2c1/deployment-state.json")
UNIT_FILE = Path("/etc/systemd/system/aios-v2r1c7-production.service")
ROLLBACK_ROOT = Path("/var/lib/aios/v2r2c1/rollback")
REQUIRED_MEMBERS = {
"payload/VERSION",
"payload/active-runtime/app/deployment_state.py",
"payload/active-runtime/app/v2r1c6_privacy_tools.py",
"payload/active-runtime/app/v2r2c1_health_service.py",
"payload/deploy/aios-v2/systemd/aios-v2r1c7-production.service",
"payload/deploy/aios-v2/v2r2c1-runtime-config.json",
"payload/deploy/aios-v2/v2r2c1-rollback.json",
"payload/scripts/probe_v2r2c1_health.py",
"payload/scripts/rollback_v2r2c1_release.sh",
"payload/BUILD-METADATA.json",
"payload/ARTIFACT-MANIFEST.json",
"payload/ROLLBACK-METADATA.json",
}
def _configure_base() -> None:
base.VERSION = VERSION
base.SERVICE = SERVICE
base.PORT = PORT
base.REQUIRED_MEMBERS = REQUIRED_MEMBERS
def _args() -> Namespace:
return Namespace(
release_root=RELEASE_ROOT,
current_link=CURRENT_LINK,
env_file=ENV_FILE,
state_file=STATE_FILE,
unit_file=UNIT_FILE,
health_timeout=45,
)
def load_request() -> dict[str, Any]:
_configure_base()
value = base.load_request()
if value.get("version") != VERSION or value.get("production_service") != SERVICE or int(value.get("production_port", -1)) != PORT:
raise ValueError("V2R2C1 production binding mismatch")
return value
def _backup(path: Path, name: str) -> None:
if path.is_file() and not path.is_symlink():
ROLLBACK_ROOT.mkdir(parents=True, exist_ok=True)
destination = ROLLBACK_ROOT / name
if not destination.exists():
shutil.copy2(path, destination)
def execute_request(value: Mapping[str, Any]) -> dict[str, Any]:
_configure_base()
if value.get("schema") != "aios.production-executor-request.v1":
raise ValueError("request_schema_invalid")
if value.get("operation") == "apply":
_backup(UNIT_FILE, "aios-v2r1c7-production.service.previous")
_backup(Path("/etc/aios/aios-v2r1c7.env"), "aios-v2r1c7.env.previous")
_backup(Path("/var/lib/aios/v2r1c7/deployment-state.json"), "v2r1c7-deployment-state.previous")
return base.install_release(_args(), value)
if value.get("operation") == "probe":
return base.probe(_args(), value)
if value.get("operation") == "rollback":
return rollback(value)
raise ValueError("unsupported_operation")
def rollback(value: Mapping[str, Any]) -> dict[str, Any]:
"""Explicit rollback only; this adapter never invokes it automatically."""
import subprocess
subprocess.run(["systemctl", "stop", SERVICE], check=False, capture_output=True, text=True, timeout=45)
previous_unit = ROLLBACK_ROOT / "aios-v2r1c7-production.service.previous"
previous_env = ROLLBACK_ROOT / "aios-v2r1c7.env.previous"
previous_state = ROLLBACK_ROOT / "v2r1c7-deployment-state.previous"
if not previous_unit.is_file() or not previous_env.is_file():
raise RuntimeError("ROLLBACK_RECEIPT_MISSING")
UNIT_FILE.parent.mkdir(parents=True, exist_ok=True)
ENV_FILE.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(previous_unit, UNIT_FILE)
shutil.copy2(previous_env, Path("/etc/aios/aios-v2r1c7.env"))
if previous_state.is_file():
Path("/var/lib/aios/v2r1c7").mkdir(parents=True, exist_ok=True)
shutil.copy2(previous_state, Path("/var/lib/aios/v2r1c7/deployment-state.json"))
subprocess.run(["systemctl", "daemon-reload"], check=True, capture_output=True, text=True, timeout=45)
subprocess.run(["systemctl", "restart", SERVICE], check=True, capture_output=True, text=True, timeout=45)
result = base.health("http://127.0.0.1:18927/health", {
"version": "V2R1C7",
"target_oid": str(json.loads(Path("/var/lib/aios/v2r1c7/deployment-state.json").read_text(encoding="utf-8")).get("commit_oid")),
"artifact_sha256": str(json.loads(Path("/var/lib/aios/v2r1c7/deployment-state.json").read_text(encoding="utf-8")).get("artifact_sha256")),
})
return {"status": "ROLLED_BACK", "rollback_health": result, "automatic_rollback": False}
def main() -> int:
try:
value = load_request()
result = execute_request(value)
print(json.dumps(result, sort_keys=True))
return 0
except Exception as exc:
print(json.dumps({"status": "FAIL_CLOSED", "error": type(exc).__name__, "reason": str(exc)}, sort_keys=True))
return 2
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,9 @@
{
"schema": "aios.approval.supersession.v1",
"version": "V2R1C7",
"superseded_approval_plan_id": "sha256:e62afc9946bcd0badc8e75291baf979e915dccaa50c2c77912c77f1ddeb2aa65",
"status": "SUPERSEDED_PREMERGE_IDENTITY_CHANGED",
"execute": false,
"reason": "release identity and artifact source head changed before protected merge",
"production_deployed": false
}
+1
View File
@@ -0,0 +1 @@
{"artifact":"aios-V2R1C7-92b4a937e56b.tar.zst","artifact_sha256":"6c7c763dc9a63a8411be3fc458881fc9bca1f403355d2f3a1cb338edbf9f24da","artifact_size":9672,"base_oid":"b356dec2ac5bd2a5f721aa73f8fd1902f9c65632","baseline_changed":false,"deployable":true,"executable_payload":true,"production_deployed":false,"schema":"aios.release-artifact-receipt.v3","source_head_oid":"7f09670511138f1f1f74f6b40329cd35d6eb2e55","tag_created":false,"version":"V2R1C7"}
+79
View File
@@ -0,0 +1,79 @@
{
"duplicates": 0,
"files": [
{
"path": "VERSION",
"sha256": "9208325e5b2e6f5ba6b64e000bc21d14d9f2f214e3ad33dd79857e85218fd6c2",
"size": 7
},
{
"path": "versions/V2R1C7.json",
"sha256": "8f3153c625241f60b65109eeac045e8aed7661d95f3a10f0b32ba3e0e9ac0a85",
"size": 801
},
{
"path": "release-manifests/V2R1C7.json",
"sha256": "560e5d9af8a8e2ec60ec31cf8659a7982b93ea09b7adb486b738ea2fba460d70",
"size": 735
},
{
"path": "deploy/aios-v2/systemd/aios-v2r1c7-production.service",
"sha256": "c96ba560d69f6c2eb700b9f2b42a32e394dadd00dffb23bbefdb9fb3e1571c0d",
"size": 709
},
{
"path": "active-runtime/app/deployment_state.py",
"sha256": "0997ab8332985c7124f334c4e49735d8833326578acc61a61a389bf792424de3",
"size": 2657
},
{
"path": "active-runtime/app/v2r1c6_privacy_tools.py",
"sha256": "dd3f8b1cab88e005a291f077807873c381e8ca92fe863dc05817925f0dff9613",
"size": 4603
},
{
"path": "active-runtime/app/v2r1c7_health_service.py",
"sha256": "9e45025c41068cafdea2f360edfc4146a5459b0a6bcfbf1403206fc0161e7bac",
"size": 174
},
{
"path": "scripts/build_v2r1c7_release_artifact.py",
"sha256": "81a423c7814f81cb888b60a538c153e549b2be9099c881c968d55ae84efddc57",
"size": 4787
},
{
"path": "scripts/verify_v2r1c7_release.py",
"sha256": "32a860e28d5afc105f413d878a13fb8600fa74874dbd3bc4c031ef3efa17896a",
"size": 1228
},
{
"path": "scripts/probe_v2r1c7_health.py",
"sha256": "64c78dabffb05c8851f87b2a27c43bd1f21728dca586397d03a6bc22906a906c",
"size": 681
},
{
"path": "scripts/rollback_v2r1c7_release.sh",
"sha256": "5419dc7de0f50908ad0ef1dac74172fce811bb7b45f314a6e1c1bb9ce618324d",
"size": 97
}
],
"mismatched": 0,
"missing": 0,
"missing_paths": [],
"required_files": [
"VERSION",
"versions/V2R1C7.json",
"release-manifests/V2R1C7.json",
"deploy/aios-v2/systemd/aios-v2r1c7-production.service",
"active-runtime/app/deployment_state.py",
"active-runtime/app/v2r1c6_privacy_tools.py",
"active-runtime/app/v2r1c7_health_service.py",
"scripts/build_v2r1c7_release_artifact.py",
"scripts/verify_v2r1c7_release.py",
"scripts/probe_v2r1c7_health.py",
"scripts/rollback_v2r1c7_release.sh"
],
"schema": "aios.v2r1c7.source-manifest.v1",
"unexpected": 0,
"version": "V2R1C7"
}
@@ -0,0 +1 @@
{"artifact":"aios-V2R1C7-92b4a937e56b.tar.zst","artifact_sha256":"6c7c763dc9a63a8411be3fc458881fc9bca1f403355d2f3a1cb338edbf9f24da","artifact_size":9672,"base_oid":"b356dec2ac5bd2a5f721aa73f8fd1902f9c65632","baseline_changed":false,"deployable":true,"executable_payload":true,"production_deployed":false,"schema":"aios.release-artifact-receipt.v3","source_head_oid":"7f09670511138f1f1f74f6b40329cd35d6eb2e55","tag_created":false,"version":"V2R1C7"}
Binary file not shown.
@@ -0,0 +1 @@
{"artifact":"aios-V2R1C7-v2r1c7.tar.zst","artifact_sha256":"08864f3f2d7e674a908a1d6eb7436db8747bd917200bf206b01276ec3da1e4be","artifact_size":7993,"baseline_changed":false,"deployable":true,"executable_payload":true,"production_deployed":false,"schema":"aios.release-artifact-receipt.v2","source_base_oid":"b356dec2ac5bd2a5f721aa73f8fd1902f9c65632","tag_created":false,"version":"V2R1C7"}
Binary file not shown.
@@ -0,0 +1,9 @@
providers:
oneapi:
name: oneapi
base_url: http://172.18.0.2:3000/v1
key_env: OPENAI_API_KEY
default_model: deepseek-v4-flash
model:
provider: oneapi
name: deepseek-v4-flash
@@ -0,0 +1,13 @@
{
"argv": [
"/usr/local/bin/hermes",
"run",
"--non-interactive"
],
"input_mode": "stdin_text",
"max_output_bytes": 131072,
"output_mode": "stdout_text",
"runtime": "hermes",
"schema": "aios.external-agent-runtime.profile.v1",
"timeout_seconds": 900
}
@@ -0,0 +1,28 @@
{
"argv": [
"/opt/aios-hermes-agent/hermes-agent/.venv/bin/hermes",
"-z",
"{prompt}",
"--provider",
"oneapi",
"-m",
"deepseek-v4-flash",
"--no-restore-cwd"
],
"credential_env": {
"OPENAI_API_KEY": {
"credential": "aios-external-agent-oneapi-env",
"format": "dotenv",
"key": "ONEAPI_REGULAR_TOKEN"
}
},
"environment": {
"HERMES_HOME": "/etc/aios/external-agent-runtime.d/hermes-home"
},
"input_mode": "stdin_text",
"max_output_bytes": 131072,
"output_mode": "stdout_text",
"runtime": "hermes",
"schema": "aios.external-agent-runtime.profile.v1",
"timeout_seconds": 180
}
@@ -0,0 +1,41 @@
{
"tools": {
"profile": "minimal"
},
"memory": {
"search": {
"enabled": false
}
},
"agents": {
"defaults": {
"workspace": "/var/lib/aios-external-agent-executor/openclaw-workspace",
"model": {
"primary": "openai/deepseek-v4-flash"
},
"thinkingDefault": "off"
},
"entries": {
"main": {
"default": true,
"workspace": "/var/lib/aios-external-agent-executor/openclaw-workspace"
}
}
},
"models": {
"providers": {
"openai": {
"baseUrl": "http://172.18.0.2:3000/v1",
"api": "openai-completions",
"models": [
{
"id": "deepseek-v4-flash",
"name": "deepseek-v4-flash",
"contextWindow": 32768,
"maxTokens": 256
}
]
}
}
}
}
@@ -0,0 +1,13 @@
{
"argv": [
"/usr/local/bin/openclaw",
"run",
"--non-interactive"
],
"input_mode": "stdin_text",
"max_output_bytes": 131072,
"output_mode": "stdout_text",
"runtime": "openclaw",
"schema": "aios.external-agent-runtime.profile.v1",
"timeout_seconds": 900
}
@@ -0,0 +1,37 @@
{
"argv": [
"/usr/bin/node",
"/opt/aios/openclaw-engineering/v2026.7.2-beta.7/openclaw.mjs",
"agent",
"--local",
"--json",
"--agent",
"main",
"--thinking",
"off",
"--message",
"{prompt}",
"--model",
"openai/deepseek-v4-flash",
"--timeout",
"180"
],
"credential_env": {
"OPENAI_API_KEY": {
"credential": "aios-external-agent-oneapi-env",
"format": "dotenv",
"key": "ONEAPI_REGULAR_TOKEN"
}
},
"environment": {
"OPENCLAW_CONFIG_PATH": "/etc/aios/external-agent-runtime.d/openclaw-config.json",
"OPENCLAW_HOME": "/var/lib/aios-external-agent-executor/openclaw-home",
"OPENCLAW_STATE_DIR": "/var/lib/aios-external-agent-executor/openclaw-state"
},
"input_mode": "stdin_text",
"max_output_bytes": 131072,
"output_mode": "stdout_json",
"runtime": "openclaw",
"schema": "aios.external-agent-runtime.profile.v1",
"timeout_seconds": 240
}
@@ -0,0 +1,36 @@
-- R4.2 P0-B durable Message/Handoff Fabric.
-- Forward-only. The existing 20260707 message_outbox remains a compatibility source.
CREATE TABLE IF NOT EXISTS message_handoff (
envelope_id TEXT PRIMARY KEY,
stream_key TEXT NOT NULL,
source TEXT NOT NULL,
target TEXT NOT NULL,
message_type TEXT NOT NULL,
payload_json TEXT NOT NULL,
payload_sha256 TEXT NOT NULL,
tenant_id TEXT,
user_id TEXT,
project_id TEXT,
conversation_id TEXT,
task_id TEXT,
run_id TEXT,
causation_id TEXT,
sequence INTEGER NOT NULL CHECK(sequence >= 1),
capability_scope_json TEXT NOT NULL,
confidentiality_scope TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
expires_at TEXT,
created_at TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'PENDING',
delivered_at TEXT,
acked_at TEXT,
UNIQUE(stream_key, idempotency_key),
UNIQUE(stream_key, sequence)
);
CREATE INDEX IF NOT EXISTS idx_message_handoff_target_status
ON message_handoff(target, status, created_at);
CREATE INDEX IF NOT EXISTS idx_message_handoff_scope
ON message_handoff(tenant_id, project_id, user_id, conversation_id, run_id);
@@ -0,0 +1,8 @@
# Installer generates these bindings; STAGED is the only pre-acceptance state.
RELEASE_VERSION=V2R1C7
TARGET_OID=__INSTALL_TARGET_OID__
ARTIFACT_SHA256=__INSTALL_ARTIFACT_SHA256__
DEPLOYMENT_STATE=STAGED
AIOS_V2R1C7_PORT=18927
DEPLOYMENT_STATE_FILE=/var/lib/aios/v2r1c7/deployment-state.json
AIOS_GITOPS_WRITES_ENABLED=false
@@ -0,0 +1,28 @@
[Unit]
Description=AIOS V2R1C7 governed health contract service
After=network.target
[Service]
Type=simple
User=aios-v2r1c7
Group=aios-v2r1c7
WorkingDirectory=/opt/aios/v2r1c7
EnvironmentFile=/etc/aios/aios-v2r1c7.env
ExecStart=/usr/bin/python3 /opt/aios/v2r1c7/active-runtime/app/v2r1c7_health_service.py --serve --host 127.0.0.1 --port ${AIOS_V2R1C7_PORT}
Restart=on-failure
RestartSec=5
NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectControlGroups=true
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
IPAddressDeny=any
IPAddressAllow=localhost
UMask=0077
[Install]
WantedBy=multi-user.target
+9
View File
@@ -0,0 +1,9 @@
{
"schema": "aios.release-rollback.v1",
"version": "V2R1C7",
"previous_version": "V2R1C6",
"previous_tag": "V2R1C6",
"rollback_state": "ROLLED_BACK",
"production_deploy_authorized": false,
"execution": "manual_owner_approval_required"
}
@@ -0,0 +1,8 @@
{
"schema": "aios.v2r1c7.runtime-config.v1",
"install_state": "STAGED",
"acceptance_transition": "STAGED -> ACTIVE only after staging gates",
"production_deploy_authorized": false,
"registry_write_authorized": false,
"baseline_write_authorized": false
}
@@ -0,0 +1,37 @@
[Unit]
Description=AIOS owner-controlled Task execution-rebind API
After=aios-agent-task-api.service
Wants=aios-agent-task-api.service
[Service]
Type=simple
User=aios-agent-task
Group=aios-agent-task
LoadCredential=agent_task_control_token:/etc/aios-gitops/agent-task-control.token
EnvironmentFile=/etc/aios-gitops/agent-task-execution-rebind.env
Environment=AIOS_AGENT_TASK_DB=/var/lib/aios-agent-task/state.sqlite3
Environment=AIOS_AGENT_TASK_CONTROL_TOKEN_FILE=%d/agent_task_control_token
Environment=AIOS_GOVERNED_WRITE_API_URL=http://127.0.0.1:18892
Environment=AIOS_TASK_EXECUTION_REBIND_HOST=127.0.0.1
Environment=AIOS_TASK_EXECUTION_REBIND_PORT=18942
ExecStart=/usr/bin/python3 -B /opt/aios/current/active-runtime/app/aios_agent_task_execution_rebind.py
Restart=on-failure
RestartSec=2
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
LockPersonality=true
RestrictSUIDSGID=true
RestrictRealtime=true
RestrictNamespaces=true
MemoryDenyWriteExecute=true
RestrictAddressFamilies=AF_UNIX AF_INET
ReadWritePaths=/var/lib/aios-agent-task
UMask=0077
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,36 @@
[Unit]
Description=AIOS owner-controlled Task retry-extension API
After=aios-agent-task-api.service
Wants=aios-agent-task-api.service
[Service]
Type=simple
User=aios-agent-task
Group=aios-agent-task
LoadCredential=agent_task_control_token:/etc/aios-gitops/agent-task-control.token
EnvironmentFile=/etc/aios-gitops/agent-task-retry-extension.env
Environment=AIOS_AGENT_TASK_DB=/var/lib/aios-agent-task/state.sqlite3
Environment=AIOS_AGENT_TASK_CONTROL_TOKEN_FILE=%d/agent_task_control_token
Environment=AIOS_TASK_RETRY_HOST=127.0.0.1
Environment=AIOS_TASK_RETRY_PORT=18941
ExecStart=/usr/bin/python3 -B /opt/aios/current/active-runtime/app/aios_agent_task_retry_extension.py
Restart=on-failure
RestartSec=2
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
LockPersonality=true
RestrictSUIDSGID=true
RestrictRealtime=true
RestrictNamespaces=true
MemoryDenyWriteExecute=true
RestrictAddressFamilies=AF_UNIX AF_INET
ReadWritePaths=/var/lib/aios-agent-task
UMask=0077
[Install]
WantedBy=multi-user.target

Some files were not shown because too many files have changed in this diff Show More