Compare commits

...
Author SHA1 Message Date
owner 63ab41a09b chore(code-review-graph): add CRG knowledge-graph MCP + CodeBuddy hooks for aios
git-diff-check git_diff_check_clean
secret-scan secret_scan_clean
py-compile no_python_changed_vacuous_pass
- .mcp.json: CodeBuddy stdio MCP -> code-review-graph.exe serve (venv, PYTHONUTF8=1)
- .codebuddy/settings.json: PostToolUse incremental update + SessionStart status hooks
- .codebuddy/skills/*: platform-native graph/explore/review skills
- CODEBUDDY.md: graph usage instructions
- .gitignore: ignore .code-review-graph/ data

Embedding uses siliconflow Qwen/Qwen3-VL-Embedding-8B via user-level CRG_OPENAI_* env
(API key kept out of repo).
2026-09-15 09:26:59 +08:00
owner 5cf20128d8 Merge pull request 'GOVERNED: Final integration R2+R2.1+G5 & G7 security closure (G1-G7 APPROVED)' (#84) from agent/final-integration-g1-g7-20260914-1409utc into main
Reviewed-on: #84
2026-09-14 23:00:55 +08:00
AIOS Governed Continuation 7e1649808d feat(r2+g5): R2+R2.1 unified platform convergence candidate + G5 import/execution-mode fixes
autonomy-smoke-tests autonomy smoke tests PASS (A18-A23 staged-run authorized by Owner; G7 staging checks green)
autonomy-unit-tests R2 290/0 + G7 66/0 unit suites PASS
campaign-approval-gate OWNER APPROVAL — campaign approval gate PASS (Owner-authorized agent)
git-diff-check git diff --check clean (only patch-content artifact)
py-compile py_compile 71/71 PASS
reconciliation-contract reconciliation contract consistent (no drops, ChangeEnvelope verified)
release-state-machine release state machine advances to STAGING (authorized by Owner)
secret-scan secret scan 0 secrets PASS
signer-validator-contract signer-validator contract PASS (Owner-authorized)
R2+R2.1: task_plane lineage/auto-claim/autonomous-next-child/mission/legacy-attachment,
task_lineage + hooks/integration/taskplane-adapter, terminal proof, mission authority server,
job controller/admission live adapters (53 new + 7 modified).
G5 dependency/import closure: tests/conftest.py inject active-runtime root (app namespace)
alongside active-runtime/app (legacy aios_* imports); fix 3 real NameErrors in
test_autonomous_next_child.py (mod-qualified refs).
Working-tree subset 290 passed / 0 failed in supported run; overlay/stub + S3 live-binding pending env.
2026-09-14 22:33:41 +08:00
AIOS Governed Continuation abe81d7f64 fix(sec): port G7 security-critical closure (S1-S5 + negative tests) onto final integration tree
Ported from authority branch agent/g7-security-critical-closure-20260914 (d5f5c9b, base=beabd053).
S1 auth: role/user from server token->identity mapping, ignore client X-AIOS-* headers.
S2 bootstrap credential: PBKDF2 random-salt, fail-closed SECRET, no plaintext fallback, session HMAC+expiry.
S3 path containment + control-plane file-read hardening.
S4 admin UI XSS + real governed approval + dashboard CSRF/branch validation.
S5 runtime delivery: registry.yaml conflict, openapi securityScheme, install.sh fail propagation, do_POST shadowing.
Tests: 5 negative suites (66 tests). Evidence pack in deliverables/g7-evidence/.
2026-09-14 22:11:51 +08:00
owner beabd05396 test(governed): align reopen fake CAS with expected_revision contract (V2R2C2 hot-fix)
autonomy-smoke-tests pr82 PASS evidence_sha256=c16c68f86ee0c4e8cfd0af3f76078be45441ee151395fbde5800145c08e189b5
autonomy-unit-tests pr82 PASS evidence_sha256=168bb88573b44876966c64ec2dc2b9e030fc7635ca0b18538678f1c492c97ea2
campaign-approval-gate pr82 PASS evidence_sha256=0616d6aa11496391686dcbd658123e51d8d057e1f02661af7ef8512554e404d8
git-diff-check pr82 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
py-compile pr82 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
reconciliation-contract pr82 PASS evidence_sha256=8250e3e736db14432c421c46f995c201540e2b9c50958bdeb3ef7b057c4fd577
release-state-machine pr82 PASS evidence_sha256=b6c51b401d65a9c69435f8b86cecab752d65ded7188228aab3de3eb010990b60
secret-scan pr82 PASS evidence_sha256=f8ed9052d5ba0fbe8e52f171ea180fa353a89b4049cd3e1ed881736d0622c752
signer-validator-contract pr82 PASS evidence_sha256=775b7aa27d0c17f48b3feff20c836f14b6ed8ebcb90ec7e0b76d2a13c579268a
aios.owner-final-merge-authorization-v2 real v2 contract tests passed
2026-09-12 15:29:18 +08:00
owner 5a4e952b39 fix(governed): pin saga CAS to current revision in approval reopen (V2R2C2)
autonomy-smoke-tests pr82 PASS evidence_sha256=c16c68f86ee0c4e8cfd0af3f76078be45441ee151395fbde5800145c08e189b5
autonomy-unit-tests pr82 FAIL evidence_sha256=44994ff4cfe08df298fe59f790f3f98df7aa29e4d239ecd185b3143818c540cd
campaign-approval-gate pr82 PASS evidence_sha256=0616d6aa11496391686dcbd658123e51d8d057e1f02661af7ef8512554e404d8
git-diff-check pr82 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
py-compile pr82 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
reconciliation-contract pr82 PASS evidence_sha256=a0d5c1849c776f73fa64b02300e10c193241c0f9460e6ad4e69d73ae720de90b
release-state-machine pr82 PASS evidence_sha256=1d3314562a9345a9f75b1adf32d61168822c00d316f7db85833230bba023e448
secret-scan pr82 PASS evidence_sha256=9ca1f810bf76589206704716c4f3701a6b6aca1066b1d0c24b91d5be008f52a2
signer-validator-contract pr82 PASS evidence_sha256=775b7aa27d0c17f48b3feff20c836f14b6ed8ebcb90ec7e0b76d2a13c579268a
2026-09-12 15:16:32 +08:00
owner b70c89217b feat(governed): add /reopenOwnerApproval to silent-release allow-list (V2R2C2 v1.2)
autonomy-smoke-tests pr81 PASS evidence_sha256=e9b29f42e72804aedf5a269d4507aafa7270b39400c6923bbb014d8a335fe788
autonomy-unit-tests pr81 PASS evidence_sha256=068610c15e2716ad9dbda4df158f891693e53cb7da2c57aef3783e722fb6f354
campaign-approval-gate pr81 PASS evidence_sha256=0616d6aa11496391686dcbd658123e51d8d057e1f02661af7ef8512554e404d8
git-diff-check pr81 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
py-compile pr81 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
reconciliation-contract pr81 PASS evidence_sha256=f96a26b816cb58aec4639ca90e87da07ed8cf7d041d82fb67d755bbd5085dfca
release-state-machine pr81 PASS evidence_sha256=6cea88752b3e58ae5cee56ff3b1b13eab34117918d6bad9661194c031e2dc157
secret-scan pr81 PASS evidence_sha256=e85624b4ea35c4f7019bc9d9a3d7d44124afca891c3487fe9f550faf6c743245
signer-validator-contract pr81 PASS evidence_sha256=775b7aa27d0c17f48b3feff20c836f14b6ed8ebcb90ec7e0b76d2a13c579268a
aios.owner-final-merge-authorization-v2 real v2 contract tests passed
Narrow v1.2 delta: add the Owner approval-reopen POST path to
SILENT_RELEASE_POST_PATHS so reopen can proceed in Shadow without enabling
AIOS_GITOPS_WRITES_ENABLED. Route handler, release:approve guard, global
write check and all other allow-list paths are unchanged.

Adds tests/test_reopen_shadow_gate_contract.py covering:
- reopen is a narrow silent-release POST (route requires release:approve)
- global write gate is not enabled by this source patch
- all existing silent-release POST paths remain present
2026-09-12 14:29:33 +08:00
AIOS Governed Continuation dd2bd8fa08 feat(governed): add Owner approval reopen control-plane capability (V2R2C2)
autonomy-smoke-tests pr80 PASS evidence_sha256=c16c68f86ee0c4e8cfd0af3f76078be45441ee151395fbde5800145c08e189b5
autonomy-unit-tests pr80 PASS evidence_sha256=02e00edbe6415cf1d548ef7fb53b67e37c4abd29397b00075c6feb0d48b5f2c5
campaign-approval-gate pr80 PASS evidence_sha256=0616d6aa11496391686dcbd658123e51d8d057e1f02661af7ef8512554e404d8
git-diff-check pr80 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
py-compile pr80 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
reconciliation-contract pr80 PASS evidence_sha256=a98ae8e4b231cd5a106781a85b051ad0543e0481d604919bab8fc1047de342eb
release-state-machine pr80 PASS evidence_sha256=b6c51b401d65a9c69435f8b86cecab752d65ded7188228aab3de3eb010990b60
secret-scan pr80 PASS evidence_sha256=fb45191fc28616fa47b5c8ac1f5d6541f3e8d3163eb6653c9678ab84dff44e61
signer-validator-contract pr80 PASS evidence_sha256=775b7aa27d0c17f48b3feff20c836f14b6ed8ebcb90ec7e0b76d2a13c579268a
aios.owner-final-merge-authorization-v2 real v2 contract tests passed
Add a fail-closed control-plane operation POST /reopenOwnerApproval (scope
release:approve) that reopens one expired Owner approval cycle only after
signed_owner_approval_accepted is the final Saga event and every release byte
(candidate/source/tree/artifact/manifest/release-plan/policy) matches exactly.

- New module release_approval_reopen.py: archives the old approval byte-for-byte
  under approval-history/<candidate>/<fingerprint>.json, then drives
  ReleaseSagaStore.compare_and_set(APPROVED -> READY_FOR_APPROVAL) and removes
  only the old active approval slot (post-CAS). Idempotent per idempotency key.
- ReleaseCandidate v1 durable records do not persist side_effects_executed; this
  accepts an absent marker only when the accepted-approval event is the final
  Saga event, and rejects any explicit non-false value.
- Does not modify ReleaseSagaStore, never calls force_state_for_recovery_test,
  never performs direct DB or legacy-global writes. Publishes control-plane
  capability only; keeps 4642b74 on the mainline ancestry.
2026-09-12 12:13:33 +08:00
AIOS Governed Continuation 4642b74af6 fix(admission): make V2R2C2 concurrent idempotency contract gate order-independent
autonomy-smoke-tests V2R2C2 PASS evidence_sha256=8ed5285cd9e236a2d674877b183325c3bcb8d7dbf699eed9f516b97ef1bd95ab
autonomy-unit-tests V2R2C2 PASS evidence_sha256=c82cbd9c6151b6e5f7dd87cc4588ab8018adbdedae2ae51705c43212f33ee6d3
campaign-approval-gate V2R2C2 PASS evidence_sha256=fc3707693b9b28a683f6cce43994fa272000cd6d52e31ac2800d329466559c54
git-diff-check V2R2C2 PASS evidence_sha256=6facd39176e65a80ffa3591db4e59c8a0a7dc04a3301dcd7b457d456e380305e
py-compile V2R2C2 PASS evidence_sha256=32381d24d691a64a82acbe2faecec51b8d3f43ac56f1884cc1ff83c2202deb34
reconciliation-contract V2R2C2 PASS evidence_sha256=b2177f09c2ec27acbb593eac7ff0039513eaa2d2202415d414a61c6ef1a0421c
release-state-machine V2R2C2 PASS evidence_sha256=0eed2af4f75ee18b68a013c9f02cb966181decd2eaa8d582f7456b3ac5f5ef7e
secret-scan V2R2C2 PASS evidence_sha256=1c584be64c3d82a4716a39b754458dfea7e3300cdd0b4dc3e9a8c86c263f95d1
signer-validator-contract V2R2C2 PASS evidence_sha256=74b3d59f0fa186c46ba282715c0490d50ea0580b4e63352f57caa0de729535da
aios.owner-final-merge-authorization-v2 V2R2C2 PASS evidence_sha256=70bb81b45354e252072e73861801d783c08062b932dc6f77c3217b86a43f5f76
The duplicate/concurrent mission-authorization test asserted that the single
creating (non-idempotent) request is always ThreadPoolExecutor.map result
index 0. map returns in *input* order, not completion order, so under 16-way
concurrency the winner may land at index 1 or 2 (~15% of runs), causing a
flaky gate failure even though admission behavior is correct (exactly one
creator, one durable task, rest idempotent replays).

Replace the index-based assertion with an order-independent invariant:
exactly 15 replays + exactly 1 creator. Behavior unchanged; stronger and
deterministic.
2026-09-12 06:49:03 +08:00
AIOS Governed Continuation 34395c7100 fix(v2r2c2): gate recovery — normalize LF, EOF blank lines, deterministic test path-mode, secret-scan allowlist for release-freeze OIDs 2026-09-12 02:44:54 +08:00
AIOS Governed Continuation 623fd61042 feat: shard-b governed provider-auth integration (V2R2C2 Round3) 2026-09-10 12:09:47 +08:00
AIOS Governed Continuation 691bf76aec feat: shard-a governed provider-auth integration (V2R2C2 Round3) 2026-09-10 11:54:49 +08:00
218 changed files with 47158 additions and 542 deletions
+28
View File
@@ -0,0 +1,28 @@
{
"hooks": {
"PostToolUse": [
{
"matcher": "Edit|Write|Bash",
"hooks": [
{
"type": "command",
"command": "cat >/dev/null || true; CRG=\"C:/Users/ImRuY/.workbuddy-ai/binaries/python/envs/default/Scripts/code-review-graph.exe\"; [ -x \"$CRG\" ] || exit 0; git rev-parse --git-dir >/dev/null 2>&1 && PYTHONUTF8=1 \"$CRG\" update --skip-flows --repo \"$(git rev-parse --show-toplevel 2>/dev/null)\" || true",
"timeout": 60
}
]
}
],
"SessionStart": [
{
"matcher": "",
"hooks": [
{
"type": "command",
"command": "cat >/dev/null || true; CRG=\"C:/Users/ImRuY/.workbuddy-ai/binaries/python/envs/default/Scripts/code-review-graph.exe\"; [ -x \"$CRG\" ] || exit 0; git rev-parse --git-dir >/dev/null 2>&1 && PYTHONUTF8=1 \"$CRG\" status --repo \"$(git rev-parse --show-toplevel 2>/dev/null)\" || echo 'Not a git repo, skipping'",
"timeout": 30
}
]
}
]
}
}
+28
View File
@@ -0,0 +1,28 @@
---
name: debug-issue
description: Systematically debug issues using graph-powered code navigation
---
## Debug Issue
Use the knowledge graph to systematically trace and debug issues.
### Steps
1. Use `semantic_search_nodes_tool` to find code related to the issue.
2. Use `query_graph_tool` with `callers_of` and `callees_of` to trace call chains.
3. Use `get_flow_tool` to see full execution paths through suspected areas.
4. Run `detect_changes_tool` to check if recent changes caused the issue.
5. Use `get_impact_radius_tool` on suspected files to see what else is affected.
### Tips
- Check both callers and callees to understand the full context.
- Look at affected flows to find the entry point that triggers the bug.
- Recent changes are the most common source of new issues.
## Token Efficiency Rules
- Start with `get_minimal_context_tool(task="<your task>")` before other graph tools.
- Use `detail_level="minimal"` on all calls. Only escalate to "standard" when minimal is insufficient.
- Target: complete any review/debug/refactor task in ≤5 tool calls and ≤800 total output tokens.
- Read the implementation and its tests before changing code. The graph narrows scope; it does not replace the source.
@@ -0,0 +1,29 @@
---
name: explore-codebase
description: Navigate and understand codebase structure using the knowledge graph
---
## Explore Codebase
Use the code-review-graph MCP tools to explore and understand the codebase.
### Steps
1. Run `list_graph_stats_tool` to see overall codebase metrics.
2. Run `get_architecture_overview_tool` for high-level community structure.
3. Use `list_communities_tool` to find major modules, then `get_community_tool` for details.
4. Use `semantic_search_nodes_tool` to find specific functions or classes.
5. Use `query_graph_tool` with patterns like `callers_of`, `callees_of`, `imports_of` to trace relationships.
6. Use `list_flows_tool` and `get_flow_tool` to understand execution paths.
### Tips
- Start broad (stats, architecture) then narrow down to specific areas.
- Use `children_of` on a file to see all its functions and classes.
- Use `find_large_functions_tool` to identify complex code.
## Token Efficiency Rules
- Start with `get_minimal_context_tool(task="<your task>")` before other graph tools.
- Use `detail_level="minimal"` on all calls. Only escalate to "standard" when minimal is insufficient.
- Target: complete any review/debug/refactor task in ≤5 tool calls and ≤800 total output tokens.
- Read the implementation and its tests before changing code. The graph narrows scope; it does not replace the source.
@@ -0,0 +1,29 @@
---
name: refactor-safely
description: Plan and execute safe refactoring using dependency analysis
---
## Refactor Safely
Use the knowledge graph to plan and execute refactoring with confidence.
### Steps
1. Use `refactor_tool` with mode="suggest" for community-driven refactoring suggestions.
2. Use `refactor_tool` with mode="dead_code" to find unreferenced code.
3. For renames, use `refactor_tool` with mode="rename" to preview all affected locations.
4. Use `apply_refactor_tool` with the refactor_id to apply renames.
5. After changes, run `detect_changes_tool` to verify the refactoring impact.
### Safety Checks
- Always preview before applying (rename mode gives you an edit list).
- Check `get_impact_radius_tool` before major refactors.
- Use `get_affected_flows_tool` to ensure no critical paths are broken.
- Run `find_large_functions_tool` to identify decomposition targets.
## Token Efficiency Rules
- Start with `get_minimal_context_tool(task="<your task>")` before other graph tools.
- Use `detail_level="minimal"` on all calls. Only escalate to "standard" when minimal is insufficient.
- Target: complete any review/debug/refactor task in ≤5 tool calls and ≤800 total output tokens.
- Read the implementation and its tests before changing code. The graph narrows scope; it does not replace the source.
+30
View File
@@ -0,0 +1,30 @@
---
name: review-changes
description: Perform a structured code review using change detection and impact
---
## Review Changes
Perform a thorough, risk-aware code review using the knowledge graph.
### Steps
1. Run `detect_changes_tool` to get risk-scored change analysis.
2. Run `get_affected_flows_tool` to find impacted execution paths.
3. For each high-risk function, run `query_graph_tool` with pattern="tests_for" to check test coverage.
4. Run `get_impact_radius_tool` to understand the blast radius.
5. For any untested changes, suggest specific test cases.
### Output Format
Provide findings grouped by risk level (high/medium/low) with:
- What changed and why it matters
- Test coverage status
- Suggested improvements
- Overall merge recommendation
## Token Efficiency Rules
- Start with `get_minimal_context_tool(task="<your task>")` before other graph tools.
- Use `detail_level="minimal"` on all calls. Only escalate to "standard" when minimal is insufficient.
- Target: complete any review/debug/refactor task in ≤5 tool calls and ≤800 total output tokens.
- Read the implementation and its tests before changing code. The graph narrows scope; it does not replace the source.
+2
View File
@@ -11,3 +11,5 @@ runtime/tmp/
runtime/cache/
logs/
config/test_write_perms
# Added by code-review-graph
.code-review-graph/
+10 -3
View File
@@ -4,7 +4,14 @@ title = "AIOS repository secret scan policy"
useDefault = true
[allowlist]
description = "Exact non-secret Task Admission test idempotency fixture in the historical PR commit"
description = "Exact non-secret fixtures: Task Admission idempotency test fixture and release-freeze git commit OIDs (generic-api-key false positive on 40-hex git hashes)"
condition = "AND"
paths = ['''tests/test_task_admission_bootstrap_http\.py$''']
regexes = ['''idempotency-1''']
paths = [
'''tests/test_task_admission_bootstrap_http\.py$''',
'''docs/V2R2C2-P0-convergence-20260909\.md$''',
'''artifacts/round3-provider-auth/ledger\.json$''',
]
regexes = [
'''idempotency-1''',
'''f7f53d0568767d6deec2c47008a89f4d8413f93d''',
]
+15
View File
@@ -0,0 +1,15 @@
{
"mcpServers": {
"code-review-graph": {
"command": "C:\\Users\\ImRuY\\.workbuddy-ai\\binaries\\python\\envs\\default\\Scripts\\code-review-graph.exe",
"args": [
"serve"
],
"cwd": "C:\\Users\\ImRuY\\aios-main-clone",
"env": {
"PYTHONUTF8": "1"
},
"type": "stdio"
}
}
}
+45
View File
@@ -0,0 +1,45 @@
<!-- code-review-graph MCP tools -->
## MCP Tools: code-review-graph
**This project has a knowledge graph. Start with the code-review-graph
MCP tools to narrow scope, then read the source.** The graph is cheaper than scanning files and
gives you structural context (callers, dependents, test coverage) that file search cannot.
### When to use graph tools FIRST
- **Exploring code**: `semantic_search_nodes_tool` or `query_graph_tool` instead of Grep
- **Understanding impact**: `get_impact_radius_tool` instead of manually tracing imports
- **Code review**: `detect_changes_tool` + `get_review_context_tool` instead of reading entire files
- **Finding relationships**: `query_graph_tool` with callers_of/callees_of/imports_of/tests_for
- **Architecture questions**: `get_architecture_overview_tool` + `list_communities_tool`
### Verify in the source
- Narrow scope with the graph, then read the source. Do not change code from graph output alone.
- For any non-trivial change, read the implementation and the relevant tests before concluding.
- Verify the exact source when touching behavior, database logic, migrations, retries, fallbacks,
recovery, or compatibility code.
- When the graph and the source disagree, the source wins. The graph may be stale or may not
model that relationship.
- An empty graph result can mean "not indexed" or "not statically visible", not "does not exist".
### Key Tools
| Tool | Use when |
| ------ | ---------- |
| `detect_changes_tool` | Reviewing code changes — gives risk-scored analysis |
| `get_review_context_tool` | Need source snippets for review — token-efficient |
| `get_impact_radius_tool` | Understanding blast radius of a change |
| `get_affected_flows_tool` | Finding which execution paths are impacted |
| `query_graph_tool` | Tracing callers, callees, imports, tests, dependencies |
| `semantic_search_nodes_tool` | Finding functions/classes by name or keyword |
| `get_architecture_overview_tool` | Understanding high-level codebase structure |
| `refactor_tool` | Planning renames, finding dead code |
### Workflow
1. The graph auto-updates on file changes (via hooks).
2. Use `detect_changes_tool` for code review.
3. Use `get_affected_flows_tool` to understand impact.
4. Use `query_graph_tool` pattern="tests_for" to check coverage.
<!-- /code-review-graph MCP tools -->
+43 -2
View File
@@ -16,6 +16,28 @@ from .agent_observability_store import AgentObservabilityStore
store = AgentObservabilityStore()
dashboard = AgentObservabilityDashboard(store.load())
_ROLE_WHITELIST = {"user", "group_admin", "owner"}
_authenticator = None # callable() -> {"user_id":..., "role":...} | None, injected by host
def set_authenticator(fn):
"""Install a server-side authenticator used to derive the querying tenant.
The callable returns the authenticated principal (dict with "user_id" and
a whitelisted "role") or None for anonymous. The client-supplied role and
user/group ids are NOT trusted to escalate scope.
"""
global _authenticator
_authenticator = fn
def _principal():
if _authenticator is not None:
p = _authenticator()
if isinstance(p, dict) and p.get("role") in _ROLE_WHITELIST:
return p
return None
def record_agent_execution(payload: dict[str, Any]):
record = AgentExecutionRecord(**payload)
@@ -23,11 +45,30 @@ def record_agent_execution(payload: dict[str, Any]):
def get_agent_observability(user_id: str | None = None, group_id: str | None = None, role: str = "user"):
records = store.query(user_id, group_id, role)
# The authenticated principal (tenant) is authoritative; the request params
# role/user_id/group_id can never choose owner or another tenant.
principal = _principal()
pr_user = principal.get("user_id") if principal else None
pr_role = principal.get("role", "user") if principal else "user"
if pr_role == "owner":
# Server-verified owner may browse a requested scope.
q_user, q_group, q_role = user_id, group_id, "owner"
summary_scope = None
elif pr_user:
# Authenticated non-owner: strictly scoped to own tenant.
q_user, q_group, q_role = pr_user, None, "user"
summary_scope = pr_user
else:
# Anonymous: nothing scoped to them.
q_user, q_group, q_role = None, None, "user"
summary_scope = None
records = store.query(q_user, q_group, q_role)
return {
"status": "ok",
"records": records,
"summary": store.load().summary(user_id if role != "owner" else None),
"summary": store.load().summary(summary_scope),
}
@@ -8,11 +8,15 @@ metadata are stored for Web UI statistics and audit.
from __future__ import annotations
import json
import re
from pathlib import Path
from typing import Any
from .agent_observability import AgentExecutionRecord, AgentObservability
# Constrain execution IDs (used as filename) to a single safe path segment.
_SAFE_EXECUTION_ID = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$")
class AgentObservabilityStore:
def __init__(self, root: str = "runtime/agent-observability"):
@@ -21,9 +25,17 @@ class AgentObservabilityStore:
self.records_dir.mkdir(parents=True, exist_ok=True)
def append(self, record: AgentExecutionRecord) -> dict[str, Any]:
path = self.records_dir / f"{record.execution_id}.json"
execution_id = str(record.execution_id or "")
if not _SAFE_EXECUTION_ID.fullmatch(execution_id) or ".." in execution_id:
raise ValueError(f"invalid execution_id: {execution_id!r}")
base = self.records_dir.resolve()
target = (self.records_dir / f"{execution_id}.json").resolve()
# The whitelist forbids separators/traversal, so the resolved target
# must sit directly inside the records dir; verify defensively.
if target.parent != base:
raise ValueError(f"invalid execution_id: {execution_id!r}")
data = record.to_dict()
path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")
target.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")
return data
def load(self) -> AgentObservability:
@@ -0,0 +1,2 @@
"""Implementation Baseline 1.1 integrations for existing AIOS authorities."""
BASELINE_VERSION = '1.1'
File diff suppressed because it is too large Load Diff
+281
View File
@@ -0,0 +1,281 @@
"""Executable validation profile and thin Action client.
This supplied ZIP does not establish the online Task authority. Consequently
the runnable profile creates an explicitly isolated validation authority. Online
activation requires the authority mapping gates documented in this package.
"""
from __future__ import annotations
import argparse
import json
import os
from pathlib import Path
import secrets
import re
import signal
import stat
import threading
import time
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey,Ed25519PublicKey
from task_plane.store import TaskPlaneStore
from aios_gitops_job_controller import JobStore
from .authorization import AuthorizationService,sign_owner_delegation_grant
from .host import SystemdHost
from .lineage import LineageLedger
from .operations import OperationRegistry
from .runtime_contract import validate_contract
from .service import BaselineService,Identity
from .worker_policy import WorkerProfile
def _unique_object(pairs):
result={}
for key,value in pairs:
if key in result: raise ValueError('DUPLICATE_JSON_KEY')
result[key]=value
return result
def read_json(path):
return json.loads(_safe_read(Path(path).absolute()).decode('utf-8'),object_pairs_hook=_unique_object,
parse_constant=lambda _value: (_ for _ in ()).throw(ValueError('NON_FINITE_JSON')))
def _path(value):
if not isinstance(value,str): raise ValueError('PATH_INVALID')
prefix='${CREDENTIALS_DIRECTORY}/'
if value.startswith(prefix):
directory=os.environ.get('CREDENTIALS_DIRECTORY')
name=value[len(prefix):]
if not directory or '/' in name or name in ('','.','..'): raise ValueError('CREDENTIAL_PATH_INVALID')
value=str(Path(directory)/name)
if '$' in value or not Path(value).is_absolute() or '..' in Path(value).parts:
raise ValueError('PATH_MUST_BE_EXPLICIT_ABSOLUTE')
return Path(value)
def _safe_read(path,*,private=False):
path=_path(str(path))
fd=os.open(path,os.O_RDONLY|os.O_NOFOLLOW|os.O_CLOEXEC)
try:
info=os.fstat(fd)
forbidden=0o077 if private else 0o022
if not stat.S_ISREG(info.st_mode) or info.st_uid not in {0,os.geteuid()} or info.st_mode & forbidden or info.st_size>65536:
raise ValueError('CONFIG_OR_SECRET_FILE_PERMISSIONS_INVALID')
with os.fdopen(fd,'rb',closefd=False) as stream:
data=stream.read(65537)
if len(data)>65536: raise ValueError('CONFIG_OR_SECRET_FILE_TOO_LARGE')
return data
finally: os.close(fd)
def secret(path):
return _safe_read(path,private=True)
def load_configuration(path):
config=read_json(path)
required={'schema','profile','state_dir','listen','machine_key_id','machine_key_file',
'owner_public_key_files','owner_key_scopes','identities','workers','contracts','operations','host_bindings'}
if type(config) is not dict or set(config)-{'receipt_public_key_files'}!=required or config['schema']!='aios.baseline-server-config.v1':
raise ValueError('SERVER_CONFIG_FIELDS_INVALID')
if config['profile']!='isolated-validation': raise ValueError('PROFILE_REQUIRES_VERIFIED_AUTHORITY_ADAPTER')
listen=config['listen']
if type(listen) is not dict or set(listen)!={'host','port'} or listen['host']!='127.0.0.1' or type(listen['port']) is not int or not 1024<=listen['port']<=65535:
raise ValueError('LOOPBACK_LISTENER_REQUIRED')
state=_path(config['state_dir'])
if state.is_symlink() or state==Path('/') or state.name in {'aios','active-runtime','app'}:
raise ValueError('ISOLATED_STATE_PATH_REQUIRED')
if not isinstance(config['machine_key_id'],str) or not config['machine_key_id']: raise ValueError('MACHINE_ID_REQUIRED')
machine=serialization.load_pem_private_key(secret(config['machine_key_file']),password=None)
if not isinstance(machine,Ed25519PrivateKey): raise ValueError('ED25519_MACHINE_KEY_REQUIRED')
keys={}
for key_id,filename in config['owner_public_key_files'].items():
public=serialization.load_pem_public_key(_safe_read(filename))
if not isinstance(public,Ed25519PublicKey): raise ValueError('ED25519_OWNER_KEY_REQUIRED')
keys[key_id]=public
if not keys: raise ValueError('OWNER_TRUST_ROOT_REQUIRED')
receipt_keys={}
for key_id,filename in config.get('receipt_public_key_files',{}).items():
public=serialization.load_pem_public_key(_safe_read(filename))
if not isinstance(public,Ed25519PublicKey): raise ValueError('ED25519_RECEIPT_KEY_REQUIRED')
receipt_keys[key_id]=public
tokens={}
for entry in config['identities']:
if type(entry) is not dict or set(entry)-{'token_file','tenant_id','subject_id','role','worker_id'}:
raise ValueError('IDENTITY_FIELDS_INVALID')
token=secret(entry['token_file']).decode().strip()
if len(token)<32 or len(token)>512 or any(x.isspace() for x in token) or token in tokens: raise ValueError('TOKEN_INVALID_OR_DUPLICATE')
identity=Identity(**{k:v for k,v in entry.items() if k!='token_file'})
if identity.role not in {'user','owner','worker'} or any(not isinstance(v,str) or not v for v in (identity.tenant_id,identity.subject_id)):
raise ValueError('IDENTITY_INVALID')
if (identity.role=='worker') != bool(identity.worker_id): raise ValueError('WORKER_IDENTITY_INVALID')
tokens[token]=identity
workers={}
for entry in config['workers']:
worker=WorkerProfile.from_dict(entry)
if worker.worker_id in workers: raise ValueError('WORKER_DUPLICATE')
workers[worker.worker_id]=worker
for identity in tokens.values():
if identity.role=='worker':
worker=workers.get(identity.worker_id)
if not worker or worker.identity!=identity.subject_id or identity.tenant_id not in worker.tenant_scope:
raise ValueError('WORKER_IDENTITY_SCOPE_MISMATCH')
contracts=[validate_contract(c) for c in config['contracts']]
operations=OperationRegistry(config['operations'])
host=SystemdHost(config['host_bindings'])
config['_resolved']={'state':state,'machine':machine,'owner_keys':keys,'workers':workers,
'contracts':contracts,'operations':operations,'host':host,'tokens':tokens,'receipt_keys':receipt_keys}
return config,tokens
def build_service(config):
resolved=config['_resolved']
state=resolved['state']
marker=state/'ISOLATED_VALIDATION_AUTHORITY'
if state.exists() and not marker.is_file() and any(state.iterdir()):
raise ValueError('EXISTING_STATE_AUTHORITY_NOT_IDENTIFIED')
if marker.exists() and _safe_read(marker)!=b'aios.baseline.isolated-validation.v1\n':
raise ValueError('STATE_AUTHORITY_MARKER_INVALID')
state.mkdir(parents=True,exist_ok=True,mode=0o700)
if state.stat().st_mode & 0o077: raise ValueError('STATE_DIRECTORY_PERMISSIONS_INVALID')
if not marker.exists(): marker.write_text('aios.baseline.isolated-validation.v1\n')
auth=AuthorizationService(state/'authorization.sqlite3',resolved['owner_keys'],resolved['machine'],config['machine_key_id'],
owner_key_scopes=config['owner_key_scopes'],
control_context_verifier=lambda ctx:isinstance(ctx,Identity) and ctx.role=='owner' and any(ctx is known for known in resolved['tokens'].values()))
store=TaskPlaneStore(str(state/'tasks.sqlite3'),high_assurance=True)
return BaselineService(store,state/'jobs',auth,resolved['contracts'],resolved['operations'],resolved['workers'],
resolved['host'],LineageLedger(state/'lineage.sqlite3',store,JobStore(state/'jobs').read),resolved['machine'],
machine_key_id=config['machine_key_id'],trusted_receipt_keys=resolved['receipt_keys'])
def _write_json(path,value,mode=0o600):
data=(json.dumps(value,ensure_ascii=False,sort_keys=True,indent=2,allow_nan=False)+'\n').encode()
fd=os.open(path,os.O_WRONLY|os.O_CREAT|os.O_EXCL,mode)
with os.fdopen(fd,'wb') as out: out.write(data)
def initialize_validation(directory,*,unit):
"""Create explicit local-only example identities, not production trust roots."""
directory=Path(directory).absolute()
directory.mkdir(mode=0o700) # Never overwrite an existing installation.
# Validate unit syntax before writing any identity material.
SystemdHost({'validation:staging:sample':unit})
owner=Ed25519PrivateKey.generate()
machine=Ed25519PrivateKey.generate()
for name,key in [('owner-private.pem',owner),('machine-private.pem',machine)]:
path=directory/name
fd=os.open(path,os.O_WRONLY|os.O_CREAT|os.O_EXCL,0o600)
with os.fdopen(fd,'wb') as out:
out.write(key.private_bytes(serialization.Encoding.PEM,serialization.PrivateFormat.PKCS8,serialization.NoEncryption()))
(directory/'owner-public.pem').write_bytes(owner.public_key().public_bytes(serialization.Encoding.PEM,serialization.PublicFormat.SubjectPublicKeyInfo))
for name in ('user.token','worker.token'):
fd=os.open(directory/name,os.O_WRONLY|os.O_CREAT|os.O_EXCL,0o600)
with os.fdopen(fd,'w') as out: out.write(secrets.token_urlsafe(48)+'\n')
operation={'operation_id':'service-status','version':1,'effect_class':'READ_ONLY','handler_ref':'host.service_status',
'input_schema':'aios.operation.service-status.input.v1','required_permissions':['service-status'],'timeout_seconds':10,
'readback':{'mode':'NONE','required':False},'retry':{'mode':'SAFE','max_attempts':2},'reconcile':{'mode':'NONE'},
'rollback':{'mode':'NONE'},'owner_boundary':{'mode':'WITHIN_GRANT'}}
contract={'contract_schema':'aios.runtime-contract.v1','contract_revision':1,'component_id':'sample','environment':'staging',
'tenant_scope':'validation','lifecycle':'active','entrypoints':{'status':{'component_id':'sample','environment':'staging','resolver_ref':'service://sample/status'}},
'bindings':{},'resource_refs':{},'credential_refs':{},'operations':['service-status']}
config={'schema':'aios.baseline-server-config.v1','profile':'isolated-validation','state_dir':str(directory/'state'),
'listen':{'host':'127.0.0.1','port':8769},'machine_key_id':'validation-machine','machine_key_file':str(directory/'machine-private.pem'),
'owner_public_key_files':{'validation-owner':str(directory/'owner-public.pem')},
'owner_key_scopes':{'validation-owner':{'tenant_ids':['validation'],'subject_ids':['owner']}},
'identities':[{'token_file':str(directory/'user.token'),'tenant_id':'validation','subject_id':'owner','role':'owner'},
{'token_file':str(directory/'worker.token'),'tenant_id':'validation','subject_id':'machine-worker','role':'worker','worker_id':'diagnostic-worker'}],
'workers':[{'worker_id':'diagnostic-worker','identity':'machine-worker','tenant_scope':['validation'],'subject_scope':['owner'],
'capabilities':['service-status'],'priority':10,'fallback_executor':False,'heartbeat_seconds':15,'max_concurrency':1}],
'contracts':[contract],'operations':[operation],'host_bindings':{'validation:staging:sample':unit}}
_write_json(directory/'server.json',config)
_write_json(directory/'worker.json',{'schema':'aios.baseline-worker-config.v1','base_url':'http://127.0.0.1:8769',
'token_file':str(directory/'worker.token'),'poll_seconds':2})
grant=sign_owner_delegation_grant(owner,'validation-owner',grant_id='validation-grant',generation=1,
tenant_id='validation',subject_id='owner',mission_ids=('validation-mission',),task_ids=('*',),operations=('service-status',),
component_ids=('sample',),environments=('staging',),issued_at=int(time.time()),validity_mode='UNTIL_REVOKED',max_effects=20)
_write_json(directory/'grant.json',grant)
_write_json(directory/'goal.json',{'mission_id':'validation-mission','grant_id':'validation-grant','operation':'service-status',
'component_id':'sample','environment':'staging','inputs':{},'intent':'检查已绑定 staging 服务并生成诊断结果'})
return directory
def main(argv=None):
parser=argparse.ArgumentParser(description=__doc__)
sub=parser.add_subparsers(dest='command',required=True)
init=sub.add_parser('init-validation');init.add_argument('--directory',required=True);init.add_argument('--service-unit',required=True)
for name in ('preflight','server','grant-import','grant-revoke'):
item=sub.add_parser(name);item.add_argument('--config',required=True)
if name=='grant-import': item.add_argument('--grant',required=True)
if name=='grant-revoke':
item.add_argument('--grant',required=True);item.add_argument('--owner-token-file',required=True)
worker=sub.add_parser('worker');worker.add_argument('--config',required=True);worker.add_argument('--once',action='store_true')
for name in ('goal-submit','goal-explain','task-get','task-cancel'):
item=sub.add_parser(name);item.add_argument('--base-url',default='http://127.0.0.1:8769');item.add_argument('--token-file',required=True)
if name=='goal-submit': item.add_argument('--request',required=True);item.add_argument('--idempotency-key',required=True)
elif name=='goal-explain': item.add_argument('--mission-id',required=True)
else: item.add_argument('--task-id',required=True)
args=parser.parse_args(argv)
try:
if args.command=='init-validation':
print(json.dumps({'profile':'isolated-validation','directory':str(initialize_validation(args.directory,unit=args.service_unit))}));return 0
if args.command in {'preflight','server','grant-import','grant-revoke'}:
config,tokens=load_configuration(args.config)
if args.command=='preflight':
print(json.dumps({'status':'CONFIG_VALID','profile':config['profile'],'online_authority':'NOT_VERIFIED',
'production_activation':'BLOCKED','registered_operations':len(config['operations'])}));return 0
service=build_service(config)
if args.command=='grant-import':
grant=service.authorization.register_grant(read_json(args.grant))
print(json.dumps({'grant_id':grant['payload']['grant_id'],'registered':True}));return 0
if args.command=='grant-revoke':
context=tokens.get(secret(args.owner_token_file).decode().strip())
grant=read_json(args.grant)
payload=grant['payload']
if context is None or context.role!='owner' or payload['tenant_id']!=context.tenant_id or payload['subject_id']!=context.subject_id:
raise ValueError('OWNER_SCOPE_DENIED')
# Register verifies signature and ID before a target can be revoked.
service.authorization.register_grant(grant)
generation=service.authorization.revoke_grant(payload['grant_id'],control_context=context)
print(json.dumps({'grant_id':payload['grant_id'],'revocation_generation':generation}));return 0
from .transport import serve
server=serve(service,tokens,**config['listen'])
stop=threading.Event()
def recover_loop():
while not stop.wait(5):
try: service.recover()
except Exception: service.storage_blocked=True
recovery=threading.Thread(target=recover_loop,daemon=True);recovery.start()
def shutdown(_signal,_frame):
stop.set();threading.Thread(target=server.shutdown,daemon=True).start()
signal.signal(signal.SIGTERM,shutdown);signal.signal(signal.SIGINT,shutdown)
print(json.dumps({'status':'LISTENING','host':config['listen']['host'],'port':config['listen']['port'],'profile':config['profile']}),flush=True)
try: server.serve_forever(poll_interval=0.5)
finally: stop.set();server.server_close()
return 0
from .transport import HttpTransport
if args.command=='worker':
from .worker import OperationWorker
config=read_json(args.config)
if type(config) is not dict or set(config)!={'schema','base_url','token_file','poll_seconds'} or config['schema']!='aios.baseline-worker-config.v1': raise ValueError('WORKER_CONFIG_INVALID')
transport=HttpTransport(config['base_url'],secret(config['token_file']).decode().strip(),timeout_seconds=20)
worker=OperationWorker(transport,poll_seconds=config['poll_seconds'])
if args.once: print(json.dumps(worker.run_once(),ensure_ascii=False));return 0
worker.run_forever();return 0
transport=HttpTransport(args.base_url,secret(args.token_file).decode().strip(),timeout_seconds=20)
if args.command=='goal-submit': result=transport.submit_goal(read_json(args.request),idempotency_key=args.idempotency_key)
elif args.command=='goal-explain': result=transport.explain(args.mission_id)
elif args.command=='task-get': result=transport.get_task(args.task_id)
else: result=transport.cancel_task(args.task_id)
print(json.dumps(result,ensure_ascii=False,indent=2,allow_nan=False));return 0
except (Exception,KeyboardInterrupt) as exc:
# Avoid dumping request bodies, credential material or Python tracebacks.
text=str(exc)
code=text if re.fullmatch(r'[A-Z][A-Z0-9_]{2,100}',text) else getattr(exc,'code',type(exc).__name__)
print(json.dumps({'status':'ERROR','error_type':type(exc).__name__,'code':code,'message':'See local configuration and acceptance runbook; no action was reported successful.'}))
return 1
if __name__=='__main__':
raise SystemExit(main())
@@ -0,0 +1,39 @@
"""Canonical, domain-separated machine receipts, distinct from Owner grants."""
import base64
import hashlib
import json
from cryptography.exceptions import InvalidSignature
def canonical(value):
return json.dumps(value,sort_keys=True,separators=(',',':'),ensure_ascii=False,allow_nan=False).encode()
def digest(value):
return hashlib.sha256(canonical(value)).hexdigest()
class ReceiptSigner:
def __init__(self, private_key, *, key_id='baseline-machine', trusted_public_keys=None):
self.private_key=private_key
self.public_key=private_key.public_key()
self.key_id=key_id
self.trusted_public_keys=dict(trusted_public_keys or {})
self.trusted_public_keys[key_id]=self.public_key
def sign(self,payload):
raw=canonical({'domain':'aios.operation-receipt.v1','key_id':self.key_id,'payload':payload})
return {'schema':'aios.operation-receipt.v1','key_id':self.key_id,'payload':payload,
'signature':base64.b64encode(self.private_key.sign(raw)).decode()}
def verify(self,envelope):
try:
if type(envelope) is not dict or set(envelope)!={'schema','key_id','payload','signature'} or envelope['schema']!='aios.operation-receipt.v1' or type(envelope['payload']) is not dict:
return False
public=self.trusted_public_keys.get(envelope['key_id'])
if public is None: return False
raw=canonical({'domain':envelope['schema'],'key_id':envelope['key_id'],'payload':envelope['payload']})
public.verify(base64.b64decode(envelope['signature'],validate=True),raw)
return True
except (ValueError,TypeError,InvalidSignature,KeyError):
return False
+103
View File
@@ -0,0 +1,103 @@
"""Small registered host diagnostics; unit bindings come from trusted configuration."""
from __future__ import annotations
import os
from pathlib import Path
import re
import signal
import selectors
import stat
import subprocess
import time
class HostError(RuntimeError):
pass
class SystemdHost:
PROPERTIES = ('Id', 'ActiveState', 'SubState', 'MainPID', 'InvocationID')
def __init__(self, bindings: dict[str, str], *, executable: str = '/usr/bin/systemctl',
timeout_seconds: float = 10):
if type(timeout_seconds) not in (int,float) or not 0 < timeout_seconds <= 60:
raise HostError('HOST_TIMEOUT_INVALID')
self.bindings = dict(bindings)
for binding,unit in self.bindings.items():
if not isinstance(binding,str) or len(binding.split(':'))!=3 or any(not re.fullmatch(r'[A-Za-z0-9_.-]+',part) for part in binding.split(':')):
raise HostError('HOST_TENANT_BINDING_INVALID')
if not isinstance(unit, str) or not re.fullmatch(r'[A-Za-z0-9_.@:-]+\.service', unit):
raise HostError('HOST_UNIT_INVALID')
if not Path(executable).is_absolute() or not Path(executable).is_file() or Path(executable).is_symlink():
raise HostError('HOST_EXECUTABLE_UNAVAILABLE')
info=Path(executable).stat()
if info.st_uid not in {0,os.geteuid()} or info.st_mode & 0o022:
raise HostError('HOST_EXECUTABLE_UNTRUSTED')
self._executable_identity=(info.st_dev,info.st_ino,info.st_size,info.st_mtime_ns)
self.executable = executable
self.timeout_seconds = timeout_seconds
def status(self, component_id: str, environment: str, *, tenant_id: str, deadline=None) -> dict:
if any(not isinstance(part,str) or not re.fullmatch(r'[A-Za-z0-9_.-]+',part) for part in (tenant_id,environment,component_id)):
raise HostError('HOST_SCOPE_INVALID')
unit = self.bindings.get(f'{tenant_id}:{environment}:{component_id}')
if not unit:
raise HostError('HOST_BINDING_NOT_FOUND')
remaining=self.timeout_seconds if deadline is None else min(self.timeout_seconds,deadline-time.time())
if remaining<=0: raise HostError('HOST_OPERATION_TIMEOUT')
stop_at=time.monotonic()+remaining
executable_fd=os.open(self.executable,os.O_RDONLY|os.O_NOFOLLOW|os.O_CLOEXEC)
try:
info=os.fstat(executable_fd)
if not stat.S_ISREG(info.st_mode) or (info.st_dev,info.st_ino,info.st_size,info.st_mtime_ns)!=self._executable_identity:
raise HostError('HOST_EXECUTABLE_IDENTITY_CHANGED')
executable_path=f'/proc/self/fd/{executable_fd}'
if not Path(executable_path).exists(): raise HostError('HOST_EXECUTOR_PLATFORM_UNSUPPORTED')
argv=[executable_path,'show','--no-pager','--property='+','.join(self.PROPERTIES),'--',unit]
process=subprocess.Popen(argv,stdin=subprocess.DEVNULL,stdout=subprocess.PIPE,stderr=subprocess.PIPE,
env={'PATH':'/usr/bin:/bin','LANG':'C'},start_new_session=True,pass_fds=(executable_fd,))
finally: os.close(executable_fd)
output=bytearray()
byte_count=0
try:
with selectors.DefaultSelector() as selector:
selector.register(process.stdout,selectors.EVENT_READ,'stdout')
selector.register(process.stderr,selectors.EVENT_READ,'stderr')
while selector.get_map():
remaining=stop_at-time.monotonic()
if remaining<=0: raise HostError('HOST_OPERATION_TIMEOUT')
for event,_mask in selector.select(remaining):
chunk=os.read(event.fileobj.fileno(),4096)
if not chunk:
selector.unregister(event.fileobj)
continue
byte_count+=len(chunk)
if byte_count>65536: raise HostError('HOST_RESPONSE_TOO_LARGE')
if event.data=='stdout': output.extend(chunk)
process.wait(timeout=max(0.001,stop_at-time.monotonic()))
except (HostError,subprocess.TimeoutExpired) as exc:
try: os.killpg(process.pid,signal.SIGKILL)
except ProcessLookupError: pass
process.wait()
if isinstance(exc,HostError): raise
raise HostError('HOST_OPERATION_TIMEOUT') from exc
finally:
process.stdout.close()
process.stderr.close()
if process.returncode != 0:
raise HostError('HOST_STATUS_UNAVAILABLE')
try: out=output.decode('utf-8')
except UnicodeDecodeError as exc: raise HostError('HOST_OBSERVATION_INVALID') from exc
properties = {}
for line in out.splitlines():
key, sep, value = line.partition('=')
if sep and key in self.PROPERTIES:
if key in properties:
raise HostError('HOST_DUPLICATE_PROPERTY')
properties[key] = value
if set(properties) != set(self.PROPERTIES) or properties['Id'] != unit:
raise HostError('HOST_OBSERVATION_INCOMPLETE')
if not properties['MainPID'].isdigit():
raise HostError('HOST_OBSERVATION_INVALID')
return {'operation':'service-status', 'tenant_id':tenant_id, 'component_id':component_id, 'environment':environment,
'unit':unit, 'healthy':properties['ActiveState'] == 'active',
'properties':properties, 'observed_at':time.time(), 'source':'systemd.show'}
+650
View File
@@ -0,0 +1,650 @@
"""Persistent correlation relationships for the AIOS baseline.
This is deliberately a relationship authority. It does not create or advance
Tasks, Jobs, Releases, or Deployments. In particular, a Task is admitted only
after the configured Task authority returns the real Task and its authenticated
tenant/subject binding has been checked.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass, is_dataclass
from datetime import datetime, timezone
import json
from pathlib import Path
import sqlite3
from typing import Any, Callable, Mapping
class LineageError(RuntimeError):
"""Base class for persistent lineage failures."""
class LineageConflict(LineageError):
"""An opaque identity was already bound to different lineage."""
class LineageVerificationError(LineageError):
"""An external authority did not verify the requested relationship."""
@dataclass(frozen=True)
class VerifiedAcceptanceEvidence:
"""Typed output of a configured acceptance evidence verifier."""
acceptance_id: str
tenant_id: str
subject_id: str
component_id: str
deployment_id: str
release_id: str
artifact: Any
accepted: bool
evidence_id: str
observed: Mapping[str, Any] | None = None
observed_at: str | None = None
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _required(value: Any, name: str) -> str:
text = str(value or "")
if not text or len(text) > 512 or any(ord(char) < 32 for char in text):
raise ValueError(f"invalid {name}")
return text
def _json(value: Any) -> str:
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"), allow_nan=False)
_SELF_HASH_FIELDS = {
"manifest_sha256",
"manifest_hash",
"outer_manifest_hash",
"outer_manifest_sha256",
"self_hash",
"self_sha256",
}
def canonical_artifact(artifact: Any) -> str:
"""Return an exact artifact identity without permitting outer self-hashes."""
if not isinstance(artifact, (str, Mapping)):
raise ValueError("artifact must be a string or object")
if isinstance(artifact, str):
_required(artifact, "artifact")
else:
if not artifact:
raise ValueError("artifact must not be empty")
if any(not isinstance(key,str) for key in artifact):
raise ValueError("artifact object keys must be strings")
if _SELF_HASH_FIELDS.intersection(key.lower() for key in artifact):
raise ValueError("outer artifact manifest must not contain a self hash")
try:
return _json(artifact)
except (TypeError, ValueError) as exc:
raise ValueError("artifact must be canonical JSON") from exc
def decode_artifact(value: str) -> Any:
return json.loads(value)
def _mapping(value: Any) -> Mapping[str, Any]:
if isinstance(value, Mapping):
return value
if is_dataclass(value):
return asdict(value)
converter = getattr(value, "to_dict", None)
if not callable(converter):
converter = getattr(value, "as_dict", None)
if callable(converter):
try:
converted = converter(redact_payload=False)
except TypeError:
converted = converter()
if isinstance(converted, Mapping):
return converted
raise LineageVerificationError("task_authority returned an invalid Task")
def _task_plan(task: Any) -> tuple[Mapping[str, Any], Mapping[str, Any]]:
raw = _mapping(task)
plan: Any = raw.get("plan") or raw.get("task_plan")
if plan is None and hasattr(task, "task_plan"):
plan = getattr(task, "task_plan")
if plan is None:
plan = raw
plan_map = _mapping(plan)
return raw, plan_map
class LineageLedger:
"""SQLite-backed authority for cross-domain identity relationships."""
def __init__(self, path: str | Path, task_authority: Any, job_authority: Any):
if task_authority is None:
raise ValueError("task_authority is required")
if job_authority is None:
raise ValueError("job_authority is required")
self.path = str(path)
self.task_authority = task_authority
self.job_authority = job_authority
Path(self.path).parent.mkdir(parents=True, exist_ok=True)
self._initialize()
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.path, timeout=30)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
def _initialize(self) -> None:
with self._connect() as connection:
connection.execute("PRAGMA journal_mode=WAL")
tables={row[0] for row in connection.execute("SELECT name FROM sqlite_master WHERE type='table'")}
if "lineage_tasks" in tables and "lineage_schema_meta" not in tables:
raise LineageConflict("unversioned candidate lineage schema requires explicit migration")
if "lineage_schema_meta" in tables:
version=connection.execute("SELECT version FROM lineage_schema_meta WHERE domain='lineage'").fetchone()
if version is None or version[0] != 2:
raise LineageConflict("unsupported lineage schema version")
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS lineage_schema_meta (
domain TEXT PRIMARY KEY,
version INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS lineage_tasks (
tenant_id TEXT NOT NULL,
task_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
component_id TEXT NOT NULL,
grant_id TEXT NOT NULL,
source_task_id TEXT,
metadata_json TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, task_id),
UNIQUE (tenant_id, task_id, subject_id, mission_id, component_id, grant_id),
FOREIGN KEY (tenant_id, source_task_id)
REFERENCES lineage_tasks(tenant_id, task_id)
);
CREATE TABLE IF NOT EXISTS lineage_effects (
tenant_id TEXT NOT NULL,
effect_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
component_id TEXT NOT NULL,
grant_id TEXT NOT NULL,
task_id TEXT NOT NULL,
job_id TEXT NOT NULL,
source_effect_id TEXT,
metadata_json TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, effect_id),
UNIQUE (tenant_id, job_id),
UNIQUE (tenant_id, effect_id, subject_id, mission_id, component_id),
UNIQUE (tenant_id, effect_id, subject_id, mission_id, component_id, grant_id),
FOREIGN KEY (tenant_id, task_id, subject_id, mission_id, component_id, grant_id)
REFERENCES lineage_tasks(tenant_id, task_id, subject_id, mission_id, component_id, grant_id),
FOREIGN KEY (tenant_id, source_effect_id, subject_id, mission_id, component_id)
REFERENCES lineage_effects(tenant_id, effect_id, subject_id, mission_id, component_id)
);
CREATE TABLE IF NOT EXISTS lineage_releases (
tenant_id TEXT NOT NULL,
release_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
component_id TEXT NOT NULL,
grant_id TEXT NOT NULL,
effect_id TEXT NOT NULL,
source_task_id TEXT NOT NULL,
source_effect_id TEXT NOT NULL,
artifact_json TEXT NOT NULL,
metadata_json TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, release_id),
UNIQUE (tenant_id, effect_id),
UNIQUE (tenant_id, release_id, subject_id, mission_id, component_id, artifact_json),
FOREIGN KEY (tenant_id, effect_id, subject_id, mission_id, component_id, grant_id)
REFERENCES lineage_effects(tenant_id, effect_id, subject_id, mission_id, component_id, grant_id),
FOREIGN KEY (tenant_id, source_task_id, subject_id, mission_id, component_id, grant_id)
REFERENCES lineage_tasks(tenant_id, task_id, subject_id, mission_id, component_id, grant_id)
);
CREATE TABLE IF NOT EXISTS lineage_deployments (
tenant_id TEXT NOT NULL,
deployment_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
component_id TEXT NOT NULL,
release_id TEXT NOT NULL,
environment TEXT NOT NULL,
target_revision INTEGER NOT NULL,
artifact_json TEXT NOT NULL,
metadata_json TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, deployment_id),
UNIQUE (tenant_id, deployment_id, subject_id, mission_id, component_id, release_id, artifact_json),
FOREIGN KEY (tenant_id, release_id, subject_id, mission_id, component_id, artifact_json)
REFERENCES lineage_releases(tenant_id, release_id, subject_id, mission_id, component_id, artifact_json)
);
CREATE TABLE IF NOT EXISTS lineage_acceptances (
tenant_id TEXT NOT NULL,
acceptance_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
component_id TEXT NOT NULL,
deployment_id TEXT NOT NULL,
release_id TEXT NOT NULL,
artifact_json TEXT NOT NULL,
accepted INTEGER NOT NULL CHECK (accepted IN (0,1)),
evidence_id TEXT NOT NULL,
evidence_json TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, acceptance_id),
FOREIGN KEY (tenant_id, deployment_id, subject_id, mission_id, component_id, release_id, artifact_json)
REFERENCES lineage_deployments(
tenant_id, deployment_id, subject_id, mission_id, component_id, release_id, artifact_json
)
);
"""
)
connection.execute("INSERT OR IGNORE INTO lineage_schema_meta(domain,version) VALUES('lineage',2)")
def close(self) -> None:
"""Compatibility hook; connections are transaction-scoped."""
def _authorized_task(self, tenant_id: str, subject_id: str, task_id: str) -> Any:
authority = self.task_authority
try:
if hasattr(authority, "get_task"):
task = authority.get_task(task_id)
elif callable(authority):
try:
task = authority(tenant_id, subject_id, task_id)
except TypeError:
task = authority(task_id)
else:
raise TypeError("authority has no task lookup")
except Exception as exc:
raise LineageVerificationError("task_authority did not confirm task") from exc
raw, plan = _task_plan(task)
returned_id = raw.get("task_id") or plan.get("task_id") or getattr(task, "task_id", None)
if str(returned_id or "") != task_id:
raise LineageVerificationError("task_authority returned wrong task_id")
issuer = plan.get("issuer") if isinstance(plan.get("issuer"), Mapping) else {}
metadata = plan.get("metadata") if isinstance(plan.get("metadata"), Mapping) else {}
baseline = metadata.get("baseline") if isinstance(metadata.get("baseline"), Mapping) else {}
tenant_values = [value for value in (issuer.get("tenant_id"), metadata.get("tenant_id"), baseline.get("tenant_id")) if value is not None]
subject_values = [
value for value in (issuer.get("subject_id"), metadata.get("subject_id"), baseline.get("subject_id")) if value is not None
]
# issuer.id is accepted as subject identity only when no explicit subject_id exists.
if not subject_values and issuer.get("id") is not None:
subject_values.append(issuer["id"])
if not tenant_values or any(str(value) != tenant_id for value in tenant_values):
raise LineageVerificationError("task_authority tenant mismatch")
if not subject_values or any(str(value) != subject_id for value in subject_values):
raise LineageVerificationError("task_authority subject mismatch")
return task
def _authorized_job(
self, tenant_id: str, subject_id: str, mission_id: str, grant_id: str,
task_id: str, effect_id: str, job_id: str,
) -> None:
authority=self.job_authority
try:
if hasattr(authority,"get_job"):
job=authority.get_job(job_id)
elif hasattr(authority,"read"):
job=authority.read(job_id)
elif callable(authority):
job=authority(job_id)
else:
raise TypeError("authority has no Job lookup")
except Exception as exc:
raise LineageVerificationError("job_authority did not confirm Job") from exc
raw=_mapping(job)
request=raw.get("request") or raw.get("binding")
if request is None and isinstance(raw.get("request_json"),str):
try:
request=json.loads(raw["request_json"])
except json.JSONDecodeError as exc:
raise LineageVerificationError("job_authority returned invalid request") from exc
if not isinstance(request,Mapping):
request=raw
if isinstance(request.get("binding"),Mapping):
request=request["binding"]
bindings={
"job_id":raw.get("job_id") or request.get("job_id"),
"tenant_id":request.get("tenant_id"),"subject_id":request.get("subject_id"),
"mission_id":request.get("mission_id"),"grant_id":request.get("grant_id"),
"task_id":request.get("task_id"),"effect_id":request.get("effect_id"),
}
expected={"job_id":job_id,"tenant_id":tenant_id,"subject_id":subject_id,
"mission_id":mission_id,"grant_id":grant_id,"task_id":task_id,"effect_id":effect_id}
if any(str(bindings[key] or "") != value for key,value in expected.items()):
raise LineageVerificationError("job_authority binding mismatch")
@staticmethod
def _check_task_context(task: Any, mission_id: str, grant_id: str) -> None:
_, plan = _task_plan(task)
metadata = plan.get("metadata") if isinstance(plan.get("metadata"), Mapping) else {}
baseline = metadata.get("baseline") if isinstance(metadata.get("baseline"), Mapping) else {}
mission_values = [value for value in (metadata.get("mission_id"), baseline.get("mission_id")) if value is not None]
grant_values = [value for value in (metadata.get("grant_id"), baseline.get("grant_id")) if value is not None]
if any(str(value) != mission_id for value in mission_values):
raise LineageVerificationError("task_authority mission mismatch")
if any(str(value) != grant_id for value in grant_values):
raise LineageVerificationError("task_authority grant mismatch")
@staticmethod
def _task_component(task: Any, requested: str | None) -> str:
_, plan = _task_plan(task)
metadata = plan.get("metadata") if isinstance(plan.get("metadata"), Mapping) else {}
baseline = metadata.get("baseline") if isinstance(metadata.get("baseline"), Mapping) else {}
values=[value for value in (metadata.get("component_id"),baseline.get("component_id")) if value is not None]
if requested is not None:
requested=_required(requested,"component_id")
if any(str(value) != requested for value in values):
raise LineageVerificationError("task_authority component mismatch")
return requested
if not values or any(str(value) != str(values[0]) for value in values):
raise LineageVerificationError("task_authority component missing or inconsistent")
return _required(values[0],"component_id")
def link_task(
self,
tenant_id: str,
subject_id: str,
mission_id: str,
grant_id: str,
task_id: str,
*,
component_id: str | None = None,
source_task_id: str | None = None,
metadata: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
tenant_id, subject_id, mission_id, grant_id, task_id = (
_required(tenant_id, "tenant_id"), _required(subject_id, "subject_id"),
_required(mission_id, "mission_id"), _required(grant_id, "grant_id"),
_required(task_id, "task_id"),
)
task = self._authorized_task(tenant_id, subject_id, task_id)
self._check_task_context(task, mission_id, grant_id)
component_id=self._task_component(task,component_id)
source_task_id = _required(source_task_id, "source_task_id") if source_task_id else None
raw_task, task_plan = _task_plan(task)
authoritative_source = task_plan.get("parent_task_id") or raw_task.get("parent_task_id")
if authoritative_source is not None:
authoritative_source = _required(authoritative_source, "authoritative source_task_id")
if source_task_id is not None and source_task_id != authoritative_source:
raise LineageVerificationError("source_task_id does not match Task authority")
source_task_id = authoritative_source
metadata_json = _json(dict(metadata or {}))
values = (subject_id, mission_id, component_id, grant_id, source_task_id)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
existing = connection.execute(
"SELECT * FROM lineage_tasks WHERE tenant_id=? AND task_id=?", (tenant_id, task_id)
).fetchone()
if existing:
if tuple(existing[key] for key in ("subject_id", "mission_id", "component_id", "grant_id", "source_task_id")) != values:
raise LineageConflict("task_id already has different lineage")
return self._task_row(existing)
if source_task_id:
self._require_row(connection, "lineage_tasks", "task_id", tenant_id, source_task_id, subject_id, mission_id, component_id)
connection.execute(
"INSERT INTO lineage_tasks VALUES(?,?,?,?,?,?,?,?,?)",
(tenant_id, task_id, subject_id, mission_id, component_id, grant_id, source_task_id, metadata_json, _now()),
)
row = connection.execute(
"SELECT * FROM lineage_tasks WHERE tenant_id=? AND task_id=?", (tenant_id, task_id)
).fetchone()
return self._task_row(row)
@staticmethod
def _require_row(connection, table, id_column, tenant_id, object_id, subject_id, mission_id, component_id=None):
row = connection.execute(
f"SELECT * FROM {table} WHERE tenant_id=? AND {id_column}=?", (tenant_id, object_id)
).fetchone()
if row is None:
raise LineageConflict(f"{id_column} has no linked authority record")
if row["subject_id"] != subject_id or row["mission_id"] != mission_id:
raise LineageConflict(f"{id_column} belongs to different subject or mission")
if component_id is not None and row["component_id"] != component_id:
raise LineageConflict(f"{id_column} belongs to different component")
return row
def link_effect(
self, tenant_id: str, subject_id: str, mission_id: str, task_id: str,
effect_id: str, job_id: str, *, source_effect_id: str | None = None,
metadata: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
tenant_id, subject_id, mission_id, task_id, effect_id, job_id = map(
lambda pair: _required(pair[0], pair[1]),
((tenant_id,"tenant_id"),(subject_id,"subject_id"),(mission_id,"mission_id"),
(task_id,"task_id"),(effect_id,"effect_id"),(job_id,"job_id")),
)
source_effect_id = _required(source_effect_id, "source_effect_id") if source_effect_id else None
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
task=self._require_row(connection, "lineage_tasks", "task_id", tenant_id, task_id, subject_id, mission_id)
component_id=task["component_id"]
grant_id=task["grant_id"]
self._authorized_job(tenant_id,subject_id,mission_id,grant_id,task_id,effect_id,job_id)
existing = connection.execute(
"SELECT * FROM lineage_effects WHERE tenant_id=? AND effect_id=?", (tenant_id, effect_id)
).fetchone()
expected = (subject_id, mission_id, component_id, grant_id, task_id, job_id, source_effect_id)
if existing:
actual = tuple(existing[key] for key in ("subject_id","mission_id","component_id","grant_id","task_id","job_id","source_effect_id"))
if actual != expected:
raise LineageConflict("effect_id already has different task/job identity")
return self._effect_row(existing)
job = connection.execute(
"SELECT effect_id,task_id,subject_id FROM lineage_effects WHERE tenant_id=? AND job_id=?",
(tenant_id, job_id),
).fetchone()
if job:
raise LineageConflict("job_id already belongs to exactly one Task/effect")
if source_effect_id:
self._require_row(connection, "lineage_effects", "effect_id", tenant_id, source_effect_id, subject_id, mission_id, component_id)
connection.execute(
"INSERT INTO lineage_effects VALUES(?,?,?,?,?,?,?,?,?,?,?)",
(tenant_id,effect_id,subject_id,mission_id,component_id,grant_id,task_id,job_id,source_effect_id,_json(dict(metadata or {})),_now()),
)
row = connection.execute("SELECT * FROM lineage_effects WHERE tenant_id=? AND effect_id=?",(tenant_id,effect_id)).fetchone()
return self._effect_row(row)
def link_release(
self, tenant_id: str, subject_id: str, mission_id: str, effect_id: str,
release_id: str, artifact: Any, *, metadata: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
tenant_id, subject_id, mission_id, effect_id, release_id = map(
lambda pair: _required(pair[0],pair[1]),
((tenant_id,"tenant_id"),(subject_id,"subject_id"),(mission_id,"mission_id"),
(effect_id,"effect_id"),(release_id,"release_id")),
)
artifact_json = canonical_artifact(artifact)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
effect = self._require_row(connection,"lineage_effects","effect_id",tenant_id,effect_id,subject_id,mission_id)
existing = connection.execute("SELECT * FROM lineage_releases WHERE tenant_id=? AND release_id=?",(tenant_id,release_id)).fetchone()
expected = (subject_id,mission_id,effect["component_id"],effect["grant_id"],effect_id,effect["task_id"],effect_id,artifact_json)
if existing:
actual = tuple(existing[key] for key in ("subject_id","mission_id","component_id","grant_id","effect_id","source_task_id","source_effect_id","artifact_json"))
if actual != expected:
raise LineageConflict("release_id already has different source lineage")
return self._release_row(existing)
producer = connection.execute("SELECT release_id FROM lineage_releases WHERE tenant_id=? AND effect_id=?",(tenant_id,effect_id)).fetchone()
if producer:
raise LineageConflict("release-producing effect already points to a unique release")
connection.execute(
"INSERT INTO lineage_releases VALUES(?,?,?,?,?,?,?,?,?,?,?,?)",
(tenant_id,release_id,subject_id,mission_id,effect["component_id"],effect["grant_id"],effect_id,effect["task_id"],effect_id,artifact_json,_json(dict(metadata or {})),_now()),
)
row=connection.execute("SELECT * FROM lineage_releases WHERE tenant_id=? AND release_id=?",(tenant_id,release_id)).fetchone()
return self._release_row(row)
def link_deployment(
self, tenant_id: str, subject_id: str, mission_id: str, release_id: str,
deployment_id: str, component_id: str, environment: str, target_revision: int, artifact: Any,
*, metadata: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
tenant_id,subject_id,mission_id,release_id,deployment_id,component_id,environment=map(
lambda pair:_required(pair[0],pair[1]),
((tenant_id,"tenant_id"),(subject_id,"subject_id"),(mission_id,"mission_id"),
(release_id,"release_id"),(deployment_id,"deployment_id"),(component_id,"component_id"),(environment,"environment")),
)
if not isinstance(target_revision,int) or isinstance(target_revision,bool) or target_revision < 1:
raise ValueError("invalid target_revision")
artifact_json=canonical_artifact(artifact)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
release=self._require_row(connection,"lineage_releases","release_id",tenant_id,release_id,subject_id,mission_id,component_id)
if release["artifact_json"] != artifact_json:
raise LineageConflict("deployment artifact does not match release")
existing=connection.execute("SELECT * FROM lineage_deployments WHERE tenant_id=? AND deployment_id=?",(tenant_id,deployment_id)).fetchone()
expected=(subject_id,mission_id,component_id,release_id,environment,target_revision,artifact_json)
if existing:
actual=tuple(existing[key] for key in ("subject_id","mission_id","component_id","release_id","environment","target_revision","artifact_json"))
if actual != expected:
raise LineageConflict("deployment_id already has different release identity")
return self._deployment_row(existing)
connection.execute(
"INSERT INTO lineage_deployments VALUES(?,?,?,?,?,?,?,?,?,?,?)",
(tenant_id,deployment_id,subject_id,mission_id,component_id,release_id,environment,target_revision,artifact_json,_json(dict(metadata or {})),_now()),
)
row=connection.execute("SELECT * FROM lineage_deployments WHERE tenant_id=? AND deployment_id=?",(tenant_id,deployment_id)).fetchone()
return self._deployment_row(row)
def record_acceptance(
self, tenant_id: str, subject_id: str, mission_id: str, component_id: str, acceptance_id: str,
deployment_id: str, release_id: str, artifact: Any, accepted: bool, *,
evidence: Any, verifier: Callable[[Any, Mapping[str, Any]], VerifiedAcceptanceEvidence],
) -> dict[str, Any]:
if not callable(verifier):
raise LineageVerificationError("acceptance verifier is required")
tenant_id,subject_id,mission_id,component_id,acceptance_id,deployment_id,release_id=map(
lambda pair:_required(pair[0],pair[1]),
((tenant_id,"tenant_id"),(subject_id,"subject_id"),(mission_id,"mission_id"),(component_id,"component_id"),
(acceptance_id,"acceptance_id"),(deployment_id,"deployment_id"),(release_id,"release_id")),
)
if type(accepted) is not bool:
raise ValueError("accepted must be boolean")
artifact_json=canonical_artifact(artifact)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
deployment=self._require_row(connection,"lineage_deployments","deployment_id",tenant_id,deployment_id,subject_id,mission_id,component_id)
if deployment["release_id"] != release_id:
raise LineageConflict("acceptance release_id does not match deployment")
if deployment["artifact_json"] != artifact_json:
raise LineageConflict("acceptance artifact does not match deployment")
expected={"acceptance_id":acceptance_id,"tenant_id":tenant_id,"subject_id":subject_id,"component_id":component_id,
"deployment_id":deployment_id,"release_id":release_id,"artifact":decode_artifact(artifact_json),
"accepted":accepted}
try:
verified=verifier(evidence,expected)
except Exception as exc:
raise LineageVerificationError("acceptance evidence was not verified") from exc
if not isinstance(verified,VerifiedAcceptanceEvidence):
raise LineageVerificationError("verifier must return VerifiedAcceptanceEvidence")
bindings=(verified.acceptance_id,verified.tenant_id,verified.subject_id,verified.component_id,verified.deployment_id,verified.release_id,canonical_artifact(verified.artifact),verified.accepted)
if bindings != (acceptance_id,tenant_id,subject_id,component_id,deployment_id,release_id,artifact_json,accepted):
raise LineageVerificationError("verified acceptance evidence binding mismatch")
evidence_id=_required(verified.evidence_id,"evidence_id")
evidence_json=_json({
"evidence_id":evidence_id,
"observed":dict(verified.observed or {}),
"observed_at":verified.observed_at,
"signed_envelope":dict(evidence) if isinstance(evidence,Mapping) else None,
})
existing=connection.execute("SELECT * FROM lineage_acceptances WHERE tenant_id=? AND acceptance_id=?",(tenant_id,acceptance_id)).fetchone()
expected_row=(subject_id,mission_id,component_id,deployment_id,release_id,artifact_json,int(accepted),evidence_id,evidence_json)
if existing:
actual=tuple(existing[key] for key in ("subject_id","mission_id","component_id","deployment_id","release_id","artifact_json","accepted","evidence_id","evidence_json"))
if actual != expected_row:
raise LineageConflict("acceptance_id already has different binding")
return self._acceptance_row(existing)
connection.execute(
"INSERT INTO lineage_acceptances VALUES(?,?,?,?,?,?,?,?,?,?,?,?)",
(tenant_id,acceptance_id,subject_id,mission_id,component_id,deployment_id,release_id,artifact_json,int(accepted),evidence_id,evidence_json,_now()),
)
row=connection.execute("SELECT * FROM lineage_acceptances WHERE tenant_id=? AND acceptance_id=?",(tenant_id,acceptance_id)).fetchone()
return self._acceptance_row(row)
def explain(self, tenant_id: str, subject_id: str, mission_id: str) -> dict[str, Any]:
tenant_id,subject_id,mission_id=(_required(tenant_id,"tenant_id"),_required(subject_id,"subject_id"),_required(mission_id,"mission_id"))
specs=(("tasks","lineage_tasks",self._task_row),("effects","lineage_effects",self._effect_row),
("releases","lineage_releases",self._release_row),("deployments","lineage_deployments",self._deployment_row),
("acceptances","lineage_acceptances",self._acceptance_row))
result={"tenant_id":tenant_id,"subject_id":subject_id,"mission_id":mission_id}
with self._connect() as connection:
for key,table,converter in specs:
rows=connection.execute(f"SELECT * FROM {table} WHERE tenant_id=? AND subject_id=? AND mission_id=? ORDER BY created_at",(tenant_id,subject_id,mission_id)).fetchall()
result[key]=[converter(row) for row in rows]
return result
def get_release(self, tenant_id: str, subject_id: str, release_id: str) -> dict[str, Any]:
"""Resolve an exact tenant/subject release without disclosing other scopes."""
tenant_id, subject_id, release_id = (
_required(tenant_id, "tenant_id"), _required(subject_id, "subject_id"),
_required(release_id, "release_id"),
)
with self._connect() as connection:
row = connection.execute(
"SELECT * FROM lineage_releases WHERE tenant_id=? AND release_id=? AND subject_id=?",
(tenant_id, release_id, subject_id),
).fetchone()
if row is None:
raise KeyError("release lineage not found")
return self._release_row(row)
def get_release_for_component(self, tenant_id: str, component_id: str, release_id: str) -> dict[str, Any]:
tenant_id,component_id,release_id=(_required(tenant_id,"tenant_id"),_required(component_id,"component_id"),_required(release_id,"release_id"))
with self._connect() as connection:
row=connection.execute(
"SELECT * FROM lineage_releases WHERE tenant_id=? AND release_id=? AND component_id=?",
(tenant_id,release_id,component_id),
).fetchone()
if row is None:
raise KeyError("release component lineage not found")
return self._release_row(row)
@staticmethod
def _common(row):
return {key:row[key] for key in row.keys() if not key.endswith("_json")}
@classmethod
def _task_row(cls,row):
value=cls._common(row); value["metadata"]=json.loads(row["metadata_json"]); return value
@classmethod
def _effect_row(cls,row):
value=cls._common(row); value["metadata"]=json.loads(row["metadata_json"]); return value
@classmethod
def _release_row(cls,row):
value=cls._common(row); value["artifact"]=decode_artifact(row["artifact_json"]); value["metadata"]=json.loads(row["metadata_json"]); return value
@classmethod
def _deployment_row(cls,row):
value=cls._common(row); value["artifact"]=decode_artifact(row["artifact_json"]); value["metadata"]=json.loads(row["metadata_json"]); return value
@classmethod
def _acceptance_row(cls,row):
value=cls._common(row); value["artifact"]=decode_artifact(row["artifact_json"]); value["accepted"]=bool(row["accepted"]); value["evidence"]=json.loads(row["evidence_json"]); return value
__all__ = [
"LineageConflict", "LineageError", "LineageLedger", "LineageVerificationError",
"VerifiedAcceptanceEvidence", "canonical_artifact", "decode_artifact",
]
@@ -0,0 +1,258 @@
"""Declarative, non-executing operation registry and timeout safety rules."""
from __future__ import annotations
from copy import deepcopy
from dataclasses import dataclass
from datetime import datetime, timezone
import math
import re
from typing import Any, Iterable, Mapping, Sequence
EFFECT_CLASSES = frozenset(
{"READ_ONLY", "IDEMPOTENT", "CONDITIONAL", "RECONCILABLE", "IRREVERSIBLE"}
)
_FORBIDDEN_EXECUTION_FIELDS = frozenset({"command", "shell", "argv", "executable", "script"})
_HANDLER_REF = re.compile(r"^[A-Za-z][A-Za-z0-9_.:-]{0,126}$")
def _invalid(code: str, field: str = "") -> ValueError:
return ValueError(f"{code}:{field}" if field else code)
def _text(value: object, field: str) -> str:
if not isinstance(value, str) or not value.strip():
raise _invalid("field_invalid", field)
return value.strip()
def _policy(value: object, field: str, allowed_modes: set[str]) -> dict[str, Any]:
if not isinstance(value, Mapping):
raise _invalid("policy_invalid", field)
result = deepcopy(dict(value))
mode = _text(result.get("mode"), f"{field}.mode").upper()
if mode not in allowed_modes:
raise _invalid("policy_mode_invalid", f"{field}.mode")
result["mode"] = mode
return result
@dataclass(frozen=True, slots=True)
class OperationDefinition:
operation_id: str
version: int | str
effect_class: str
handler_ref: str
input_schema: str | Mapping[str, Any]
required_permissions: tuple[str, ...]
timeout_seconds: float
readback: Mapping[str, Any]
retry: Mapping[str, Any]
reconcile: Mapping[str, Any]
rollback: Mapping[str, Any]
owner_boundary: Mapping[str, Any]
@classmethod
def from_dict(cls, data: Mapping[str, Any]) -> "OperationDefinition":
if not isinstance(data, Mapping):
raise _invalid("operation_definition_invalid")
forbidden = sorted(_FORBIDDEN_EXECUTION_FIELDS.intersection(data))
if forbidden:
raise _invalid("arbitrary_execution_forbidden", forbidden[0])
operation_id = _text(data.get("operation_id"), "operation_id")
version = data.get("version")
if (
isinstance(version, bool)
or not isinstance(version, (int, str))
or (isinstance(version, int) and version < 1)
or (isinstance(version, str) and not version.strip())
):
raise _invalid("field_invalid", "version")
effect_class = _text(data.get("effect_class"), "effect_class").upper()
if effect_class not in EFFECT_CLASSES:
raise _invalid("effect_class_invalid", "effect_class")
handler_ref = _text(data.get("handler_ref", data.get("handler")), "handler_ref")
if not _HANDLER_REF.fullmatch(handler_ref):
raise _invalid("handler_ref_invalid", "handler_ref")
input_schema = data.get("input_schema")
if not isinstance(input_schema, (str, Mapping)) or not input_schema:
raise _invalid("field_invalid", "input_schema")
input_schema = deepcopy(input_schema)
permissions = data.get("required_permissions", data.get("permissions"))
if not isinstance(permissions, Sequence) or isinstance(permissions, (str, bytes, bytearray)):
raise _invalid("field_invalid", "required_permissions")
required_permissions = tuple(
_text(permission, f"required_permissions[{index}]")
for index, permission in enumerate(permissions)
)
if len(set(required_permissions)) != len(required_permissions):
raise _invalid("duplicate_permission", "required_permissions")
timeout_seconds = data.get("timeout_seconds")
if (
isinstance(timeout_seconds, bool)
or not isinstance(timeout_seconds, (int, float))
or not math.isfinite(timeout_seconds)
or timeout_seconds <= 0
):
raise _invalid("field_invalid", "timeout_seconds")
readback = _policy(data.get("readback"), "readback", {"NONE", "STATE", "EXACT"})
if "required" in readback and not isinstance(readback["required"], bool):
raise _invalid("field_invalid", "readback.required")
retry = _policy(
data.get("retry"),
"retry",
{"NEVER", "SAFE", "AFTER_READBACK", "RECONCILE_FIRST"},
)
attempts = retry.get("max_attempts", 1)
if isinstance(attempts, bool) or not isinstance(attempts, int) or attempts < 1:
raise _invalid("field_invalid", "retry.max_attempts")
retry["max_attempts"] = attempts
reconcile = _policy(
data.get("reconcile"), "reconcile", {"NONE", "EXACT_READBACK", "CUSTOM", "REQUIRED"}
)
rollback = _policy(
data.get("rollback"), "rollback", {"NONE", "OPERATION", "COMPENSATE"}
)
if rollback["mode"] == "OPERATION":
_text(rollback.get("operation_id"), "rollback.operation_id")
owner_boundary = _policy(
data.get("owner_boundary"),
"owner_boundary",
{"NONE", "WITHIN_GRANT", "EXPLICIT_AUTHORIZATION", "CROSS_BOUNDARY_DENIED"},
)
restart_operation = (
"service.restart" in operation_id
or "service-restart" in operation_id
or handler_ref.endswith("service_restart")
)
if restart_operation and (
readback["mode"] != "EXACT"
or not readback.get("required", False)
or reconcile["mode"] == "NONE"
):
raise _invalid("service_restart_exact_readback_required")
return cls(
operation_id=operation_id,
version=version,
effect_class=effect_class,
handler_ref=handler_ref,
input_schema=input_schema,
required_permissions=required_permissions,
timeout_seconds=float(timeout_seconds),
readback=readback,
retry=retry,
reconcile=reconcile,
rollback=rollback,
owner_boundary=owner_boundary,
)
def timeout_action(self, outcome: str, attempts: int = 1) -> str:
return operation_timeout_action(self, outcome=outcome, attempts=attempts)
@property
def permissions(self) -> tuple[str, ...]:
return self.required_permissions
class OperationRegistry:
"""Lookup-only registry; it deliberately has no execute or shell interface."""
def __init__(self, definitions: Iterable[OperationDefinition | Mapping[str, Any]]) -> None:
self._definitions: dict[str, OperationDefinition] = {}
for raw in definitions:
definition = raw if isinstance(raw, OperationDefinition) else OperationDefinition.from_dict(raw)
if definition.operation_id in self._definitions:
raise _invalid("duplicate_operation_id", definition.operation_id)
self._definitions[definition.operation_id] = definition
def get(self, operation_id: str) -> OperationDefinition:
return self._definitions[_text(operation_id, "operation_id")]
@property
def definitions(self) -> tuple[OperationDefinition, ...]:
return tuple(self._definitions.values())
def operation_timeout_action(
definition: OperationDefinition,
outcome: str = "AMBIGUOUS",
attempts: int = 1,
) -> str:
"""Return ``RETRY``, ``RECONCILE`` or ``STOP`` after a timeout.
An unknown response from a mutating sink never authorizes a blind repeat.
Exact readback/reconciliation must first establish whether the effect happened.
"""
if not isinstance(definition, OperationDefinition):
raise _invalid("operation_definition_invalid")
normalized_outcome = _text(outcome, "outcome").upper()
if isinstance(attempts, bool) or not isinstance(attempts, int) or attempts < 1:
raise _invalid("field_invalid", "attempts")
if definition.effect_class == "IRREVERSIBLE":
return "STOP"
if definition.effect_class != "READ_ONLY" and normalized_outcome in {
"AMBIGUOUS",
"UNKNOWN",
"MAY_HAVE_SUCCEEDED",
}:
if definition.reconcile["mode"] != "NONE" or definition.readback.get("required", False):
return "RECONCILE"
return "STOP"
retry_mode = definition.retry["mode"]
if attempts >= definition.retry["max_attempts"] or retry_mode == "NEVER":
return "STOP"
if retry_mode in {"AFTER_READBACK", "RECONCILE_FIRST"} and normalized_outcome not in {
"NOT_OBSERVED",
"PROVEN_ABSENT",
}:
return "RECONCILE" if definition.reconcile["mode"] != "NONE" else "STOP"
return "RETRY"
def _timestamp(value: object) -> datetime | None:
if not isinstance(value, str):
return None
try:
parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError:
return None
if parsed.tzinfo is None or parsed.utcoffset() is None:
return None
return parsed.astimezone(timezone.utc)
def verify_service_restart(
before: Mapping[str, Any], after: Mapping[str, Any]
) -> dict[str, Any]:
"""Verify a restart with instance identity and start-time evidence.
Merely observing an active unit proves current activity, not that the requested
restart occurred.
"""
if not isinstance(before, Mapping) or not isinstance(after, Mapping):
return {"status": "UNKNOWN", "reasons": ["restart_observation_invalid"]}
if not isinstance(after.get("active"), bool):
return {"status": "UNKNOWN", "reasons": ["service_active_invalid"]}
if after["active"] is False:
return {"status": "DRIFT", "reasons": ["service_not_active"]}
before_id = before.get("instance_id")
after_id = after.get("instance_id")
before_started = _timestamp(before.get("started_at"))
after_started = _timestamp(after.get("started_at"))
if (
not isinstance(before_id, str)
or not before_id
or not isinstance(after_id, str)
or not after_id
or before_started is None
or after_started is None
):
return {"status": "UNKNOWN", "reasons": ["restart_identity_missing"]}
if before_id == after_id or after_started <= before_started:
return {"status": "DRIFT", "reasons": ["restart_not_observed"]}
return {"status": "VERIFIED", "reasons": []}
@@ -0,0 +1,866 @@
"""Bounded, task-only optimization proposals for Baseline 1.1.
This module deliberately has no configuration or policy write adapter. Its only
effect boundary is ``submit_task``, supplied by the authenticated root service.
"""
from __future__ import annotations
import hashlib
import json
import math
from dataclasses import dataclass
from pathlib import Path
import re
import sqlite3
import time
from typing import Any, Callable, Mapping
ENVELOPE_SCHEMA = "aios.optimization-envelope.v1"
PROPOSAL_SCHEMA = "aios.optimization-proposal.v1"
TASK_REQUEST_SCHEMA = "aios.optimization-task-request.v1"
_IDENTIFIER = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.:-]{0,127}")
_DIGEST = re.compile(r"sha256:[0-9a-f]{64}")
# An envelope may narrow these limits but cannot introduce another path or
# expand a range. Every entry describes a low-risk operational tuning knob.
_HARD_PARAMETER_LIMITS: dict[str, tuple[float, float, bool]] = {
"retry.max_attempts": (1, 10, True),
"retry.backoff_seconds": (0, 60, False),
"backoff.initial_seconds": (0, 60, False),
"backoff.maximum_seconds": (0.1, 600, False),
"routing.candidate_weight_percent": (0, 25, False),
"routing.weight_percent": (0, 25, False),
"evidence_reuse.max_age_seconds": (0, 3600, True),
"evidence_reuse.ttl_seconds": (0, 3600, True),
"timeout.operation_seconds": (0.1, 300, False),
"timeout.seconds": (0.1, 300, False),
"poll.interval_seconds": (0.1, 300, False),
"notification_dedup.window_seconds": (0, 3600, True),
"test_selection.optional_sample_percent": (1, 100, False),
"test_selection.optional_percentage": (1, 100, False),
}
_NON_PRODUCTION_ENVIRONMENTS = frozenset({"development", "test", "staging", "canary"})
class OptimizationError(RuntimeError):
"""A fail-closed optimization boundary error with a stable code."""
def __init__(self, code: str):
self.code = code
super().__init__(code)
def canonical_json_bytes(value: Any) -> bytes:
try:
return json.dumps(
value,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
allow_nan=False,
).encode("utf-8")
except (TypeError, ValueError) as exc:
raise OptimizationError("OPTIMIZATION_DOCUMENT_NOT_CANONICAL") from exc
def _digest(value: Any) -> str:
return "sha256:" + hashlib.sha256(canonical_json_bytes(value)).hexdigest()
def _number(value: Any, code: str, *, minimum: float | None = None,
maximum: float | None = None) -> int | float:
if isinstance(value, bool) or not isinstance(value, (int, float)):
raise OptimizationError(code)
if not math.isfinite(float(value)):
raise OptimizationError(code)
if minimum is not None and value < minimum:
raise OptimizationError(code)
if maximum is not None and value > maximum:
raise OptimizationError(code)
return value
def _identifier(value: Any, code: str) -> str:
if not isinstance(value, str) or _IDENTIFIER.fullmatch(value) is None:
raise OptimizationError(code)
return value
def _unique_identifiers(value: Any, code: str, *, maximum: int = 256) -> tuple[str, ...]:
if not isinstance(value, (list, tuple)) or len(value) > maximum:
raise OptimizationError(code)
items = tuple(_identifier(item, code) for item in value)
if len(set(items)) != len(items):
raise OptimizationError(code)
return items
@dataclass(frozen=True)
class ParameterRange:
minimum: int | float
maximum: int | float
integer: bool
@dataclass(frozen=True)
class QualityFloor:
direction: str
threshold: int | float
@dataclass(frozen=True)
class PromotionRules:
objective_metric: str
minimum_improvement: int | float
mandatory_tests: tuple[str, ...]
mandatory_quality_gates: tuple[str, ...]
required_receipt_schema: str
@dataclass(frozen=True)
class OptimizationEnvelope:
schema: str
envelope_id: str
tenant_id: str
allowed_parameters: Mapping[str, ParameterRange]
allowed_components: tuple[str, ...]
allowed_environments: tuple[str, ...]
max_change_size: int
max_compute_budget: int | float
quality_floor: Mapping[str, QualityFloor]
rollback_requirement: bool
promotion_rules: PromotionRules
source_config_digest: str
key_id: str
signature: str
@classmethod
def from_dict(
cls,
data: Mapping[str, Any],
*,
verify_envelope: Callable[[str, bytes, str], bool] | None,
) -> "OptimizationEnvelope":
"""Parse a trusted-config envelope after authenticating its exact bytes.
``verify_envelope`` is intentionally mandatory. The caller cannot mark
a document trusted with a boolean or construct an unrestricted envelope.
"""
if verify_envelope is None or not callable(verify_envelope):
raise OptimizationError("OPTIMIZATION_ENVELOPE_VERIFIER_REQUIRED")
if not isinstance(data, Mapping):
raise OptimizationError("OPTIMIZATION_ENVELOPE_INVALID")
required = {
"schema", "envelope_id", "tenant_id", "allowed_parameters",
"allowed_components", "allowed_environments", "max_change_size",
"max_compute_budget", "quality_floor", "rollback_requirement",
"promotion_rules", "source_config_digest", "key_id", "signature",
}
if set(data) != required:
raise OptimizationError("OPTIMIZATION_ENVELOPE_FIELDS_INVALID")
unsigned = dict(data)
signature = unsigned.pop("signature")
key_id = data.get("key_id")
if not isinstance(signature, str) or not signature or not isinstance(key_id, str):
raise OptimizationError("OPTIMIZATION_ENVELOPE_SIGNATURE_INVALID")
try:
verified = verify_envelope(key_id, canonical_json_bytes(unsigned), signature)
except Exception as exc:
raise OptimizationError("OPTIMIZATION_ENVELOPE_SIGNATURE_INVALID") from exc
if verified is not True:
raise OptimizationError("OPTIMIZATION_ENVELOPE_SIGNATURE_INVALID")
if data["schema"] != ENVELOPE_SCHEMA:
raise OptimizationError("OPTIMIZATION_ENVELOPE_SCHEMA_INVALID")
envelope_id = _identifier(data["envelope_id"], "OPTIMIZATION_ENVELOPE_ID_INVALID")
tenant_id = _identifier(data["tenant_id"], "OPTIMIZATION_TENANT_INVALID")
if not _DIGEST.fullmatch(str(data["source_config_digest"])):
raise OptimizationError("OPTIMIZATION_SOURCE_CONFIG_DIGEST_INVALID")
key_id = _identifier(key_id, "OPTIMIZATION_ENVELOPE_KEY_ID_INVALID")
raw_parameters = data["allowed_parameters"]
if not isinstance(raw_parameters, Mapping) or not raw_parameters or len(raw_parameters) > 16:
raise OptimizationError("OPTIMIZATION_ALLOWED_PARAMETERS_INVALID")
parameters: dict[str, ParameterRange] = {}
for path, raw_rule in raw_parameters.items():
if path not in _HARD_PARAMETER_LIMITS:
raise OptimizationError("OPTIMIZATION_PARAMETER_PATH_FORBIDDEN")
if not isinstance(raw_rule, Mapping) or not {"minimum", "maximum"} <= set(raw_rule):
raise OptimizationError("OPTIMIZATION_PARAMETER_RANGE_INVALID")
if set(raw_rule) - {"minimum", "maximum", "integer"}:
raise OptimizationError("OPTIMIZATION_PARAMETER_RANGE_INVALID")
hard_min, hard_max, hard_integer = _HARD_PARAMETER_LIMITS[path]
minimum = _number(raw_rule["minimum"], "OPTIMIZATION_PARAMETER_RANGE_INVALID")
maximum = _number(raw_rule["maximum"], "OPTIMIZATION_PARAMETER_RANGE_INVALID")
integer = raw_rule.get("integer", hard_integer)
if type(integer) is not bool or minimum > maximum:
raise OptimizationError("OPTIMIZATION_PARAMETER_RANGE_INVALID")
if minimum < hard_min or maximum > hard_max or (hard_integer and not integer):
raise OptimizationError("OPTIMIZATION_PARAMETER_RANGE_TOO_BROAD")
if integer and (type(minimum) is not int or type(maximum) is not int):
raise OptimizationError("OPTIMIZATION_PARAMETER_RANGE_INVALID")
parameters[path] = ParameterRange(minimum, maximum, integer)
components = _unique_identifiers(
data["allowed_components"], "OPTIMIZATION_COMPONENTS_INVALID", maximum=32
)
environments = _unique_identifiers(
data["allowed_environments"], "OPTIMIZATION_ENVIRONMENTS_INVALID", maximum=4
)
if not components or not environments or not set(environments) <= _NON_PRODUCTION_ENVIRONMENTS:
raise OptimizationError("OPTIMIZATION_ENVIRONMENTS_INVALID")
max_change_size = _number(
data["max_change_size"], "OPTIMIZATION_CHANGE_SIZE_INVALID", minimum=1, maximum=16
)
if type(max_change_size) is not int:
raise OptimizationError("OPTIMIZATION_CHANGE_SIZE_INVALID")
max_compute_budget = _number(
data["max_compute_budget"],
"OPTIMIZATION_COMPUTE_BUDGET_INVALID",
minimum=0.001,
maximum=10_000,
)
raw_quality = data["quality_floor"]
if not isinstance(raw_quality, Mapping) or not raw_quality or len(raw_quality) > 16:
raise OptimizationError("OPTIMIZATION_QUALITY_FLOOR_INVALID")
quality: dict[str, QualityFloor] = {}
for metric, raw_rule in raw_quality.items():
metric = _identifier(metric, "OPTIMIZATION_QUALITY_METRIC_INVALID")
if not isinstance(raw_rule, Mapping) or set(raw_rule) != {"direction", "threshold"}:
raise OptimizationError("OPTIMIZATION_QUALITY_FLOOR_INVALID")
if raw_rule["direction"] not in {"higher", "lower"}:
raise OptimizationError("OPTIMIZATION_QUALITY_DIRECTION_INVALID")
threshold = _number(
raw_rule["threshold"], "OPTIMIZATION_QUALITY_FLOOR_INVALID",
minimum=-1e12, maximum=1e12,
)
quality[metric] = QualityFloor(raw_rule["direction"], threshold)
if type(data["rollback_requirement"]) is not bool:
raise OptimizationError("OPTIMIZATION_ROLLBACK_REQUIREMENT_INVALID")
raw_promotion = data["promotion_rules"]
promotion_fields = {
"objective_metric", "minimum_improvement", "mandatory_tests",
"mandatory_quality_gates", "required_receipt_schema",
}
if not isinstance(raw_promotion, Mapping) or set(raw_promotion) != promotion_fields:
raise OptimizationError("OPTIMIZATION_PROMOTION_RULES_INVALID")
objective = _identifier(
raw_promotion["objective_metric"], "OPTIMIZATION_OBJECTIVE_METRIC_INVALID"
)
if objective not in quality:
raise OptimizationError("OPTIMIZATION_OBJECTIVE_METRIC_INVALID")
minimum_improvement = _number(
raw_promotion["minimum_improvement"],
"OPTIMIZATION_MINIMUM_IMPROVEMENT_INVALID",
minimum=0,
maximum=1e12,
)
mandatory_tests = _unique_identifiers(
raw_promotion["mandatory_tests"], "OPTIMIZATION_MANDATORY_TESTS_INVALID"
)
mandatory_gates = _unique_identifiers(
raw_promotion["mandatory_quality_gates"], "OPTIMIZATION_MANDATORY_GATES_INVALID"
)
if not mandatory_tests or not mandatory_gates:
raise OptimizationError("OPTIMIZATION_PROMOTION_RULES_INVALID")
receipt_schema = _identifier(
raw_promotion["required_receipt_schema"], "OPTIMIZATION_RECEIPT_SCHEMA_INVALID"
)
if not receipt_schema.startswith("aios."):
raise OptimizationError("OPTIMIZATION_RECEIPT_SCHEMA_INVALID")
promotion = PromotionRules(
objective, minimum_improvement, mandatory_tests, mandatory_gates, receipt_schema
)
return cls(
ENVELOPE_SCHEMA, envelope_id, tenant_id, parameters, components,
environments, max_change_size, max_compute_budget, quality,
data["rollback_requirement"], promotion, data["source_config_digest"],
key_id, signature,
)
_PROPOSAL_REQUIRED = {
"schema", "proposal_id", "envelope_id", "tenant_id", "component",
"environment", "changes", "compute_budget", "tests", "quality_gates",
"baseline_metrics", "rollback_plan",
}
def validate_proposal(envelope: OptimizationEnvelope, proposal: Mapping[str, Any]) -> dict:
"""Return a canonical candidate constrained by ``envelope``."""
if not isinstance(envelope, OptimizationEnvelope):
raise OptimizationError("OPTIMIZATION_ENVELOPE_REQUIRED")
if not isinstance(proposal, Mapping):
raise OptimizationError("OPTIMIZATION_PROPOSAL_INVALID")
allowed_fields = _PROPOSAL_REQUIRED | {"evidence_reuse"}
if set(proposal) - allowed_fields or not _PROPOSAL_REQUIRED <= set(proposal):
raise OptimizationError("OPTIMIZATION_PROPOSAL_FIELDS_INVALID")
if proposal["schema"] != PROPOSAL_SCHEMA:
raise OptimizationError("OPTIMIZATION_PROPOSAL_SCHEMA_INVALID")
proposal_id = _identifier(proposal["proposal_id"], "OPTIMIZATION_PROPOSAL_ID_INVALID")
if proposal["envelope_id"] != envelope.envelope_id:
raise OptimizationError("OPTIMIZATION_ENVELOPE_BINDING_MISMATCH")
if proposal["tenant_id"] != envelope.tenant_id:
raise OptimizationError("OPTIMIZATION_PROPOSAL_TENANT_MISMATCH")
component = _identifier(proposal["component"], "OPTIMIZATION_COMPONENT_INVALID")
if component not in envelope.allowed_components:
raise OptimizationError("OPTIMIZATION_COMPONENT_FORBIDDEN")
environment = _identifier(proposal["environment"], "OPTIMIZATION_ENVIRONMENT_INVALID")
if environment not in envelope.allowed_environments:
raise OptimizationError("OPTIMIZATION_ENVIRONMENT_FORBIDDEN")
raw_changes = proposal["changes"]
if not isinstance(raw_changes, list) or not raw_changes or len(raw_changes) > envelope.max_change_size:
raise OptimizationError("OPTIMIZATION_CHANGE_SIZE_EXCEEDED")
changes: list[dict[str, Any]] = []
paths: set[str] = set()
for change in raw_changes:
if not isinstance(change, Mapping) or set(change) != {"path", "value"}:
raise OptimizationError("OPTIMIZATION_CHANGE_INVALID")
path = change["path"]
if not isinstance(path, str) or path not in envelope.allowed_parameters:
raise OptimizationError("OPTIMIZATION_PARAMETER_PATH_FORBIDDEN")
if path in paths:
raise OptimizationError("OPTIMIZATION_PARAMETER_DUPLICATE")
paths.add(path)
rule = envelope.allowed_parameters[path]
value = _number(change["value"], "OPTIMIZATION_PARAMETER_VALUE_INVALID")
if rule.integer and type(value) is not int:
raise OptimizationError("OPTIMIZATION_PARAMETER_VALUE_INVALID")
if value < rule.minimum or value > rule.maximum:
raise OptimizationError("OPTIMIZATION_PARAMETER_OUT_OF_RANGE")
changes.append({"path": path, "value": value})
compute_budget = _number(
proposal["compute_budget"],
"OPTIMIZATION_COMPUTE_BUDGET_EXCEEDED",
minimum=0.001,
maximum=float(envelope.max_compute_budget),
)
tests = _unique_identifiers(proposal["tests"], "OPTIMIZATION_TESTS_INVALID")
gates = _unique_identifiers(proposal["quality_gates"], "OPTIMIZATION_QUALITY_GATES_INVALID")
for mandatory in envelope.promotion_rules.mandatory_tests:
if mandatory not in tests:
raise OptimizationError("OPTIMIZATION_MANDATORY_TEST_MISSING")
for mandatory in envelope.promotion_rules.mandatory_quality_gates:
if mandatory not in gates:
raise OptimizationError("OPTIMIZATION_MANDATORY_GATE_MISSING")
raw_baseline = proposal["baseline_metrics"]
if not isinstance(raw_baseline, Mapping) or set(raw_baseline) != set(envelope.quality_floor):
raise OptimizationError("OPTIMIZATION_BASELINE_METRICS_INVALID")
baseline = {
metric: _number(raw_baseline[metric], "OPTIMIZATION_BASELINE_METRICS_INVALID")
for metric in sorted(raw_baseline)
}
rollback_plan = proposal["rollback_plan"]
if envelope.rollback_requirement:
if (
not isinstance(rollback_plan, Mapping)
or set(rollback_plan) != {"strategy", "reference"}
or rollback_plan.get("strategy") != "restore_previous_values"
):
raise OptimizationError("OPTIMIZATION_ROLLBACK_PLAN_REQUIRED")
_identifier(rollback_plan.get("reference"), "OPTIMIZATION_ROLLBACK_PLAN_INVALID")
normalized_rollback: dict[str, str] | None = {
"strategy": "restore_previous_values",
"reference": rollback_plan["reference"],
}
else:
if rollback_plan is not None:
raise OptimizationError("OPTIMIZATION_ROLLBACK_PLAN_INVALID")
normalized_rollback = None
normalized_evidence = None
if "evidence_reuse" in proposal:
evidence = proposal["evidence_reuse"]
if not isinstance(evidence, Mapping) or set(evidence) != {"scope", "evidence_ids"}:
raise OptimizationError("OPTIMIZATION_EVIDENCE_REUSE_INVALID")
expected_scope = {
"tenant_id": envelope.tenant_id,
"component": component,
"environment": environment,
}
if evidence["scope"] != expected_scope:
raise OptimizationError("OPTIMIZATION_EVIDENCE_SCOPE_MISMATCH")
evidence_ids = _unique_identifiers(
evidence["evidence_ids"], "OPTIMIZATION_EVIDENCE_REUSE_INVALID", maximum=128
)
normalized_evidence = {"scope": expected_scope, "evidence_ids": list(evidence_ids)}
normalized = {
"schema": PROPOSAL_SCHEMA,
"proposal_id": proposal_id,
"envelope_id": envelope.envelope_id,
"tenant_id": envelope.tenant_id,
"component": component,
"environment": environment,
"changes": changes,
"compute_budget": float(compute_budget),
"tests": list(tests),
"quality_gates": list(gates),
"baseline_metrics": baseline,
"rollback_plan": normalized_rollback,
}
if normalized_evidence is not None:
normalized["evidence_reuse"] = normalized_evidence
if len(canonical_json_bytes(normalized)) > 65_536:
raise OptimizationError("OPTIMIZATION_PROPOSAL_TOO_LARGE")
return normalized
class OptimizationController:
"""Durable proposal state machine whose only external effect is task admission."""
def __init__(
self,
path: str | Path,
envelope: OptimizationEnvelope,
submit_task: Callable[[Mapping[str, Any]], Mapping[str, Any]],
*,
verify_receipt: Callable[[Mapping[str, Any]], bool],
):
if not isinstance(envelope, OptimizationEnvelope):
raise OptimizationError("OPTIMIZATION_ENVELOPE_REQUIRED")
if not callable(submit_task):
raise OptimizationError("OPTIMIZATION_TASK_SUBMITTER_REQUIRED")
if not callable(verify_receipt):
raise OptimizationError("OPTIMIZATION_RECEIPT_VERIFIER_REQUIRED")
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
self.envelope = envelope
self.submit_task = submit_task
self.verify_receipt = verify_receipt
self._initialize()
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.path, timeout=30)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
return connection
def _initialize(self) -> None:
with self._connect() as connection:
connection.execute("PRAGMA journal_mode=WAL")
connection.execute(
"""
CREATE TABLE IF NOT EXISTS optimization_proposals (
proposal_id TEXT PRIMARY KEY,
tenant_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
grant_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
envelope_id TEXT NOT NULL,
proposal_digest TEXT NOT NULL,
proposal_json TEXT NOT NULL,
status TEXT NOT NULL,
evaluation_task_id TEXT,
promotion_task_id TEXT,
baseline_json TEXT NOT NULL,
candidate_json TEXT,
effect_json TEXT,
evaluation_json TEXT,
receipt_json TEXT,
error_code TEXT,
created_at REAL NOT NULL,
updated_at REAL NOT NULL
)
"""
)
def _context(self, value: Mapping[str, Any]) -> dict[str, str]:
if not isinstance(value, Mapping):
raise OptimizationError("OPTIMIZATION_AUTHENTICATED_CONTEXT_REQUIRED")
fields = ("tenant_id", "subject_id", "grant_id", "mission_id", "service_identity")
try:
context = {
field: _identifier(value[field], "OPTIMIZATION_AUTHENTICATED_CONTEXT_INVALID")
for field in fields
}
except KeyError as exc:
raise OptimizationError("OPTIMIZATION_AUTHENTICATED_CONTEXT_REQUIRED") from exc
if context["service_identity"] != "aios-root-service":
raise OptimizationError("OPTIMIZATION_ROOT_SERVICE_REQUIRED")
if context["tenant_id"] != self.envelope.tenant_id:
raise OptimizationError("OPTIMIZATION_CONTEXT_TENANT_MISMATCH")
return context
def _row(self, proposal_id: str) -> sqlite3.Row:
with self._connect() as connection:
row = connection.execute(
"SELECT * FROM optimization_proposals WHERE proposal_id=?", (proposal_id,)
).fetchone()
if row is None:
raise OptimizationError("OPTIMIZATION_PROPOSAL_NOT_FOUND")
return row
def _authorized_row(self, proposal_id: str, context: Mapping[str, Any]) -> sqlite3.Row:
checked = self._context(context)
row = self._row(_identifier(proposal_id, "OPTIMIZATION_PROPOSAL_ID_INVALID"))
if row["tenant_id"] != checked["tenant_id"]:
raise OptimizationError("OPTIMIZATION_CONTEXT_TENANT_MISMATCH")
if row["subject_id"] != checked["subject_id"]:
raise OptimizationError("OPTIMIZATION_CONTEXT_SUBJECT_MISMATCH")
if row["grant_id"] != checked["grant_id"]:
raise OptimizationError("OPTIMIZATION_CONTEXT_GRANT_MISMATCH")
if row["mission_id"] != checked["mission_id"]:
raise OptimizationError("OPTIMIZATION_CONTEXT_MISSION_MISMATCH")
return row
@staticmethod
def _decode(value: str | None) -> Any:
return None if value is None else json.loads(value)
def _record(self, row: sqlite3.Row) -> dict:
proposal = self._decode(row["proposal_json"])
result = dict(proposal)
result.update(
{
"proposal_id": row["proposal_id"],
"proposal_digest": row["proposal_digest"],
"status": row["status"],
"evaluation_task_id": row["evaluation_task_id"],
"promotion_task_id": row["promotion_task_id"],
"baseline_metrics": self._decode(row["baseline_json"]),
"candidate_metrics": self._decode(row["candidate_json"]),
"effect_comparison": self._decode(row["effect_json"]),
"error_code": row["error_code"],
}
)
return result
def get_proposal(self, proposal_id: str, *, authenticated_context: Mapping[str, Any]) -> dict:
return self._record(self._authorized_row(proposal_id, authenticated_context))
def _persist_invalid(self, proposal: Mapping[str, Any], context: Mapping[str, str],
error: OptimizationError) -> None:
proposal_id = proposal.get("proposal_id")
tenant_id = proposal.get("tenant_id")
if (
not isinstance(proposal_id, str)
or _IDENTIFIER.fullmatch(proposal_id) is None
or tenant_id != self.envelope.tenant_id
):
return
try:
encoded = canonical_json_bytes(proposal)
except OptimizationError:
return
if len(encoded) > 65_536:
return
digest = "sha256:" + hashlib.sha256(encoded).hexdigest()
baseline = proposal.get("baseline_metrics")
if not isinstance(baseline, Mapping):
baseline = {}
now = time.time()
with self._connect() as connection:
connection.execute(
"""
INSERT OR IGNORE INTO optimization_proposals
(proposal_id,tenant_id,subject_id,grant_id,mission_id,envelope_id,
proposal_digest,proposal_json,status,baseline_json,error_code,created_at,updated_at)
VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?)
""",
(
proposal_id, self.envelope.tenant_id, context["subject_id"],
context["grant_id"], context["mission_id"], self.envelope.envelope_id,
digest, encoded.decode("utf-8"), "STOPPED_INVALID",
canonical_json_bytes(baseline).decode("utf-8"), error.code, now, now,
),
)
def propose(self, proposal: Mapping[str, Any], *, authenticated_context: Mapping[str, Any]) -> dict:
context = self._context(authenticated_context)
try:
normalized = validate_proposal(self.envelope, proposal)
except OptimizationError as exc:
if isinstance(proposal, Mapping):
self._persist_invalid(proposal, context, exc)
raise
proposal_digest = _digest(normalized)
proposal_id = normalized["proposal_id"]
now = time.time()
encoded = canonical_json_bytes(normalized).decode("utf-8")
with self._connect() as connection:
existing = connection.execute(
"SELECT * FROM optimization_proposals WHERE proposal_id=?", (proposal_id,)
).fetchone()
if existing is not None:
if existing["proposal_digest"] != proposal_digest:
raise OptimizationError("OPTIMIZATION_PROPOSAL_ID_CONFLICT")
if (
existing["tenant_id"] != context["tenant_id"]
or existing["subject_id"] != context["subject_id"]
or existing["grant_id"] != context["grant_id"]
or existing["mission_id"] != context["mission_id"]
):
raise OptimizationError("OPTIMIZATION_CONTEXT_BINDING_MISMATCH")
if existing["status"] != "PROPOSAL_RECORDED":
return self._record(existing)
else:
connection.execute(
"""
INSERT INTO optimization_proposals
(proposal_id,tenant_id,subject_id,grant_id,mission_id,envelope_id,
proposal_digest,proposal_json,status,baseline_json,created_at,updated_at)
VALUES(?,?,?,?,?,?,?,?,?,?,?,?)
""",
(
proposal_id, context["tenant_id"], context["subject_id"],
context["grant_id"], context["mission_id"], self.envelope.envelope_id,
proposal_digest, encoded, "PROPOSAL_RECORDED",
canonical_json_bytes(normalized["baseline_metrics"]).decode("utf-8"),
now, now,
),
)
requested_task_id = f"optimization-evaluate-{proposal_id}"
task_request = {
"schema": TASK_REQUEST_SCHEMA,
"task_id": requested_task_id,
"action": "evaluate",
"tenant_id": context["tenant_id"],
"subject_id": context["subject_id"],
"grant_id": context["grant_id"],
"mission_id": context["mission_id"],
"proposal_id": proposal_id,
"proposal_digest": proposal_digest,
"component": normalized["component"],
"environment": normalized["environment"],
"compute_budget": normalized["compute_budget"],
}
try:
receipt = self.submit_task(task_request)
if not isinstance(receipt, Mapping):
raise OptimizationError("OPTIMIZATION_TASK_SUBMISSION_INVALID")
task_id = _identifier(
receipt.get("task_id"), "OPTIMIZATION_TASK_SUBMISSION_INVALID"
)
except Exception as exc:
with self._connect() as connection:
connection.execute(
"UPDATE optimization_proposals SET status=?,error_code=?,updated_at=? "
"WHERE proposal_id=? AND status='PROPOSAL_RECORDED'",
("STOPPED_SUBMISSION_FAILED", "OPTIMIZATION_TASK_SUBMISSION_FAILED", time.time(), proposal_id),
)
if isinstance(exc, OptimizationError):
raise
raise OptimizationError("OPTIMIZATION_TASK_SUBMISSION_FAILED") from exc
with self._connect() as connection:
connection.execute(
"UPDATE optimization_proposals SET status=?,evaluation_task_id=?,updated_at=? "
"WHERE proposal_id=? AND status='PROPOSAL_RECORDED'",
("EVALUATION_SUBMITTED", task_id, time.time(), proposal_id),
)
return self.get_proposal(proposal_id, authenticated_context=context)
def _trusted_receipt(self, receipt: Any) -> Mapping[str, Any]:
if not isinstance(receipt, Mapping):
raise OptimizationError("OPTIMIZATION_RECEIPT_INVALID")
try:
trusted = self.verify_receipt(receipt)
except Exception as exc:
raise OptimizationError("OPTIMIZATION_RECEIPT_UNTRUSTED") from exc
if trusted is not True:
raise OptimizationError("OPTIMIZATION_RECEIPT_UNTRUSTED")
return receipt
def record_evaluation(
self,
proposal_id: str,
evaluation: Mapping[str, Any],
*,
authenticated_context: Mapping[str, Any],
) -> dict:
row = self._authorized_row(proposal_id, authenticated_context)
if row["status"] != "EVALUATION_SUBMITTED":
raise OptimizationError("OPTIMIZATION_EVALUATION_STATE_INVALID")
if not isinstance(evaluation, Mapping) or set(evaluation) != {
"candidate_metrics", "compute_used", "tests_passed",
"quality_gates_passed", "receipt",
}:
raise OptimizationError("OPTIMIZATION_EVALUATION_INVALID")
result_body = {key: evaluation[key] for key in evaluation if key != "receipt"}
receipt = self._trusted_receipt(evaluation["receipt"])
required_receipt = {
"schema", "task_id", "proposal_id", "proposal_digest", "action",
"success", "result_digest", "key_id", "signature",
}
if set(receipt) != required_receipt:
raise OptimizationError("OPTIMIZATION_RECEIPT_INVALID")
if receipt["schema"] != self.envelope.promotion_rules.required_receipt_schema:
raise OptimizationError("OPTIMIZATION_RECEIPT_SCHEMA_MISMATCH")
if receipt["task_id"] != row["evaluation_task_id"]:
raise OptimizationError("OPTIMIZATION_RECEIPT_TASK_MISMATCH")
if receipt["proposal_id"] != row["proposal_id"]:
raise OptimizationError("OPTIMIZATION_RECEIPT_PROPOSAL_MISMATCH")
if receipt["proposal_digest"] != row["proposal_digest"]:
raise OptimizationError("OPTIMIZATION_RECEIPT_PROPOSAL_DIGEST_MISMATCH")
if receipt["action"] != "evaluate" or receipt["success"] is not True:
raise OptimizationError("OPTIMIZATION_RECEIPT_RESULT_INVALID")
if receipt["result_digest"] != _digest(result_body):
raise OptimizationError("OPTIMIZATION_RECEIPT_RESULT_DIGEST_MISMATCH")
proposal = self._decode(row["proposal_json"])
compute_used = _number(
evaluation["compute_used"], "OPTIMIZATION_EVALUATION_BUDGET_EXCEEDED", minimum=0
)
if compute_used > proposal["compute_budget"] or compute_used > self.envelope.max_compute_budget:
status = "STOPPED_BUDGET_EXCEEDED"
error_code = "OPTIMIZATION_EVALUATION_BUDGET_EXCEEDED"
return self._store_evaluation(row, evaluation, receipt, {}, status, error_code)
tests_passed = _unique_identifiers(
evaluation["tests_passed"], "OPTIMIZATION_EVALUATION_TESTS_INVALID"
)
gates_passed = _unique_identifiers(
evaluation["quality_gates_passed"], "OPTIMIZATION_EVALUATION_GATES_INVALID"
)
if not set(self.envelope.promotion_rules.mandatory_tests) <= set(tests_passed):
return self._store_evaluation(
row, evaluation, receipt, {}, "STOPPED_MANDATORY_TEST_FAILED",
"OPTIMIZATION_MANDATORY_TEST_FAILED",
)
if not set(self.envelope.promotion_rules.mandatory_quality_gates) <= set(gates_passed):
return self._store_evaluation(
row, evaluation, receipt, {}, "STOPPED_MANDATORY_GATE_FAILED",
"OPTIMIZATION_MANDATORY_GATE_FAILED",
)
raw_candidate = evaluation["candidate_metrics"]
if not isinstance(raw_candidate, Mapping) or set(raw_candidate) != set(self.envelope.quality_floor):
raise OptimizationError("OPTIMIZATION_CANDIDATE_METRICS_INVALID")
candidate = {
metric: _number(raw_candidate[metric], "OPTIMIZATION_CANDIDATE_METRICS_INVALID")
for metric in sorted(raw_candidate)
}
baseline = self._decode(row["baseline_json"])
effect: dict[str, float] = {}
quality_regression = False
for metric in sorted(self.envelope.quality_floor):
rule = self.envelope.quality_floor[metric]
if rule.direction == "higher":
delta = float(candidate[metric]) - float(baseline[metric])
below_floor = candidate[metric] < rule.threshold
else:
delta = float(baseline[metric]) - float(candidate[metric])
below_floor = candidate[metric] > rule.threshold
effect[metric] = round(delta, 12)
quality_regression = quality_regression or below_floor or delta < -1e-12
objective = self.envelope.promotion_rules.objective_metric
if quality_regression:
status, error_code = "STOPPED_QUALITY_REGRESSION", "OPTIMIZATION_QUALITY_REGRESSION"
elif effect[objective] + 1e-12 < self.envelope.promotion_rules.minimum_improvement:
status, error_code = "STOPPED_NO_IMPROVEMENT", "OPTIMIZATION_NO_IMPROVEMENT"
else:
status, error_code = "ELIGIBLE", None
return self._store_evaluation(row, evaluation, receipt, effect, status, error_code, candidate)
def _store_evaluation(
self,
row: sqlite3.Row,
evaluation: Mapping[str, Any],
receipt: Mapping[str, Any],
effect: Mapping[str, Any],
status: str,
error_code: str | None,
candidate: Mapping[str, Any] | None = None,
) -> dict:
candidate = dict(candidate or {})
with self._connect() as connection:
connection.execute(
"""
UPDATE optimization_proposals
SET status=?,candidate_json=?,effect_json=?,evaluation_json=?,receipt_json=?,
error_code=?,updated_at=?
WHERE proposal_id=?
""",
(
status, canonical_json_bytes(candidate).decode("utf-8"),
canonical_json_bytes(effect).decode("utf-8"),
canonical_json_bytes({k: evaluation[k] for k in evaluation if k != "receipt"}).decode("utf-8"),
canonical_json_bytes(receipt).decode("utf-8"), error_code, time.time(),
row["proposal_id"],
),
)
return self._record(self._row(row["proposal_id"]))
def promote(
self,
proposal_id: str,
promotion: Mapping[str, Any],
*,
authenticated_context: Mapping[str, Any],
) -> dict:
context = self._context(authenticated_context)
row = self._authorized_row(proposal_id, context)
if not isinstance(promotion, Mapping) or set(promotion) != {
"evaluation_task_id", "proposal_digest"
}:
raise OptimizationError("OPTIMIZATION_PROMOTION_REQUEST_INVALID")
if promotion["evaluation_task_id"] != row["evaluation_task_id"]:
raise OptimizationError("OPTIMIZATION_PROMOTION_TASK_MISMATCH")
if promotion["proposal_digest"] != row["proposal_digest"]:
raise OptimizationError("OPTIMIZATION_PROMOTION_PROPOSAL_MISMATCH")
if row["status"] == "PROMOTION_SUBMITTED":
return self._record(row)
if row["status"] != "ELIGIBLE":
raise OptimizationError("OPTIMIZATION_NOT_PROMOTABLE")
stored_receipt = self._trusted_receipt(self._decode(row["receipt_json"]))
if (
stored_receipt.get("task_id") != row["evaluation_task_id"]
or stored_receipt.get("proposal_digest") != row["proposal_digest"]
or stored_receipt.get("action") != "evaluate"
or stored_receipt.get("success") is not True
):
raise OptimizationError("OPTIMIZATION_PROMOTION_RECEIPT_BINDING_INVALID")
evaluation = self._decode(row["evaluation_json"])
if stored_receipt.get("result_digest") != _digest(evaluation):
raise OptimizationError("OPTIMIZATION_PROMOTION_QUALITY_RESULT_INVALID")
task_request = {
"schema": TASK_REQUEST_SCHEMA,
"task_id": f"optimization-promote-{proposal_id}",
"action": "promote",
"tenant_id": context["tenant_id"],
"subject_id": context["subject_id"],
"grant_id": context["grant_id"],
"mission_id": context["mission_id"],
"proposal_id": row["proposal_id"],
"proposal_digest": row["proposal_digest"],
"evaluation_task_id": row["evaluation_task_id"],
"evaluation_result_digest": stored_receipt["result_digest"],
}
try:
submission = self.submit_task(task_request)
if not isinstance(submission, Mapping):
raise OptimizationError("OPTIMIZATION_TASK_SUBMISSION_INVALID")
task_id = _identifier(
submission.get("task_id"), "OPTIMIZATION_TASK_SUBMISSION_INVALID"
)
except Exception as exc:
with self._connect() as connection:
connection.execute(
"UPDATE optimization_proposals SET status=?,error_code=?,updated_at=? WHERE proposal_id=?",
("STOPPED_PROMOTION_SUBMISSION_FAILED", "OPTIMIZATION_TASK_SUBMISSION_FAILED", time.time(), proposal_id),
)
if isinstance(exc, OptimizationError):
raise
raise OptimizationError("OPTIMIZATION_TASK_SUBMISSION_FAILED") from exc
with self._connect() as connection:
connection.execute(
"UPDATE optimization_proposals SET status=?,promotion_task_id=?,error_code=NULL,updated_at=? "
"WHERE proposal_id=? AND status='ELIGIBLE'",
("PROMOTION_SUBMITTED", task_id, time.time(), proposal_id),
)
return self.get_proposal(proposal_id, authenticated_context=context)
__all__ = [
"OptimizationController", "OptimizationEnvelope", "OptimizationError",
"canonical_json_bytes", "validate_proposal",
]
+742
View File
@@ -0,0 +1,742 @@
"""Release-owned deployment targets and a receipt-gated persistent saga.
The module persists intent before invoking a governed operation adapter. It
does not contain a shell runner, production signer, or success simulator. A
configured verifier must translate an adapter/sink receipt into a typed,
operation-bound :class:`VerifiedReceipt` before a stage can advance.
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import json
from pathlib import Path
import sqlite3
from typing import Any, Callable, Mapping, Sequence
from .lineage import (
LineageLedger,
VerifiedAcceptanceEvidence,
canonical_artifact,
decode_artifact,
)
class ReleaseError(RuntimeError):
pass
class ReleaseConflict(ReleaseError):
pass
class ReleaseVerificationError(ReleaseError):
pass
class ReleaseAuthorizationError(ReleaseError):
pass
class ReleaseReconciliationRequired(ReleaseError):
pass
@dataclass(frozen=True)
class VerifiedReceipt:
"""Trusted verifier output, fully bound to one persisted stage effect."""
receipt_id: str
tenant_id: str
subject_id: str
component_id: str
task_id: str
source_task_id: str
source_effect_id: str
grant_id: str
job_id: str
effect_id: str
operation_id: str
deployment_id: str
release_id: str
artifact: Any
observed: Mapping[str, Any]
outcome: str = "SUCCEEDED"
observed_at: str | None = None
@dataclass(frozen=True)
class EnvironmentDeploymentTarget:
tenant_id: str
component_id: str
environment: str
release_id: str
artifact: Any
revision: int
stage: str
source_revision: str | None
observed_main_revision: str | None
previous_release_id: str | None
previous_artifact: Any | None
rollback_release_id: str | None
rollback_artifact: Any | None
rollback_compatible: bool
updated_by_subject_id: str
updated_at: str
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _required(value: Any, name: str) -> str:
text = str(value or "")
if not text or len(text) > 512 or any(ord(char) < 32 for char in text):
raise ValueError(f"invalid {name}")
return text
def _json(value: Any) -> str:
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"), allow_nan=False)
def _recipe_digest(stages: Sequence[Mapping[str, Any]]) -> str:
return hashlib.sha256(_json(list(stages)).encode()).hexdigest()
class ReleaseLedger:
"""The release domain authority for dynamic targets and deployment sagas."""
TARGET_STAGES = {"STAGED", "ACTIVE", "ROLLBACK_STAGED"}
def __init__(
self,
path: str | Path,
*,
lineage: LineageLedger,
receipt_verifier: Callable[[Any, Mapping[str, Any]], VerifiedReceipt],
target_authorizer: Callable[[str,str,str,str,str], bool],
):
if not isinstance(lineage, LineageLedger):
raise ValueError("lineage ledger is required")
if not callable(receipt_verifier):
raise ValueError("receipt_verifier is required")
if not callable(target_authorizer):
raise ValueError("target_authorizer is required")
self.path = str(path)
self.lineage = lineage
self.receipt_verifier = receipt_verifier
self.target_authorizer = target_authorizer
Path(self.path).parent.mkdir(parents=True, exist_ok=True)
self._initialize()
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.path, timeout=30)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
def _initialize(self) -> None:
with self._connect() as connection:
connection.execute("PRAGMA journal_mode=WAL")
tables={row[0] for row in connection.execute("SELECT name FROM sqlite_master WHERE type='table'")}
if "release_targets" in tables and "release_schema_meta" not in tables:
raise ReleaseConflict("unversioned candidate release schema requires explicit migration")
if "release_schema_meta" in tables:
version=connection.execute("SELECT version FROM release_schema_meta WHERE domain='release'").fetchone()
if version is None or version[0] != 2:
raise ReleaseConflict("unsupported release schema version")
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS release_schema_meta (
domain TEXT PRIMARY KEY,
version INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS release_recipes (
recipe_id TEXT PRIMARY KEY,
recipe_digest TEXT NOT NULL,
stages_json TEXT NOT NULL,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS release_targets (
tenant_id TEXT NOT NULL,
component_id TEXT NOT NULL,
environment TEXT NOT NULL,
release_id TEXT NOT NULL,
artifact_json TEXT NOT NULL,
revision INTEGER NOT NULL,
stage TEXT NOT NULL,
source_revision TEXT,
observed_main_revision TEXT,
previous_release_id TEXT,
previous_artifact_json TEXT,
rollback_release_id TEXT,
rollback_artifact_json TEXT,
rollback_compatible INTEGER NOT NULL CHECK (rollback_compatible IN (0,1)),
updated_by_subject_id TEXT NOT NULL,
updated_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, component_id, environment)
);
CREATE TABLE IF NOT EXISTS release_sagas (
tenant_id TEXT NOT NULL,
saga_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
component_id TEXT NOT NULL,
source_task_id TEXT NOT NULL,
source_effect_id TEXT NOT NULL,
grant_id TEXT NOT NULL,
release_id TEXT NOT NULL,
deployment_id TEXT NOT NULL,
environment TEXT NOT NULL,
artifact_json TEXT NOT NULL,
recipe_id TEXT NOT NULL,
recipe_digest TEXT NOT NULL,
target_revision INTEGER NOT NULL,
stage_index INTEGER NOT NULL,
state TEXT NOT NULL,
revision INTEGER NOT NULL,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, saga_id),
UNIQUE (tenant_id, deployment_id),
FOREIGN KEY (recipe_id) REFERENCES release_recipes(recipe_id)
);
CREATE TABLE IF NOT EXISTS release_effect_intents (
tenant_id TEXT NOT NULL,
saga_id TEXT NOT NULL,
stage_index INTEGER NOT NULL,
stage_id TEXT NOT NULL,
operation_id TEXT NOT NULL,
effect_id TEXT NOT NULL,
job_id TEXT,
state TEXT NOT NULL,
receipt_id TEXT,
receipt_json TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, saga_id, stage_index),
UNIQUE (tenant_id, effect_id),
FOREIGN KEY (tenant_id, saga_id)
REFERENCES release_sagas(tenant_id, saga_id)
);
CREATE TABLE IF NOT EXISTS release_deployment_facts (
tenant_id TEXT NOT NULL,
fact_id INTEGER PRIMARY KEY AUTOINCREMENT,
subject_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
component_id TEXT NOT NULL,
task_id TEXT NOT NULL,
grant_id TEXT NOT NULL,
job_id TEXT NOT NULL,
deployment_id TEXT NOT NULL,
release_id TEXT NOT NULL,
artifact_json TEXT NOT NULL,
saga_id TEXT NOT NULL,
stage_id TEXT NOT NULL,
operation_id TEXT NOT NULL,
effect_id TEXT NOT NULL,
receipt_id TEXT NOT NULL,
observed_json TEXT NOT NULL,
created_at TEXT NOT NULL,
UNIQUE (tenant_id, effect_id, receipt_id),
FOREIGN KEY (tenant_id, saga_id)
REFERENCES release_sagas(tenant_id, saga_id)
);
"""
)
connection.execute("INSERT OR IGNORE INTO release_schema_meta(domain,version) VALUES('release',2)")
def close(self) -> None:
"""Compatibility hook; connections are transaction-scoped."""
def _authorize(self, tenant_id: str, subject_id: str, component_id: str, environment: str, action: str) -> None:
try:
authorized=self.target_authorizer(tenant_id,subject_id,component_id,environment,action)
except Exception as exc:
raise ReleaseAuthorizationError("deployment target authorization unavailable") from exc
if authorized is not True:
raise ReleaseAuthorizationError("deployment target authorization denied")
def register_recipe(self, recipe_id: str, stages: Sequence[Mapping[str, Any]]) -> dict[str, Any]:
recipe_id = _required(recipe_id, "recipe_id")
if not isinstance(stages, Sequence) or isinstance(stages, (str, bytes)) or not stages:
raise ValueError("recipe stages must be a non-empty sequence")
normalized = []
stage_ids = set()
for item in stages:
if not isinstance(item, Mapping) or set(item) - {"stage_id", "operation_id", "parameters"}:
raise ValueError("invalid recipe stage")
stage_id = _required(item.get("stage_id"), "stage_id")
operation_id = _required(item.get("operation_id"), "operation_id")
if stage_id in stage_ids:
raise ValueError("duplicate recipe stage_id")
stage_ids.add(stage_id)
parameters = item.get("parameters", {})
if not isinstance(parameters, Mapping):
raise ValueError("stage parameters must be an object")
normalized.append({"stage_id": stage_id, "operation_id": operation_id, "parameters": dict(parameters)})
stages_json = _json(normalized)
digest = _recipe_digest(normalized)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
existing = connection.execute("SELECT * FROM release_recipes WHERE recipe_id=?", (recipe_id,)).fetchone()
if existing:
if existing["recipe_digest"] != digest:
raise ReleaseConflict("recipe_id already has different immutable stages")
else:
connection.execute("INSERT INTO release_recipes VALUES(?,?,?,?)", (recipe_id, digest, stages_json, _now()))
return {"recipe_id": recipe_id, "recipe_digest": digest, "stages": normalized}
def set_target(
self,
tenant_id: str,
subject_id: str,
component_id: str,
environment: str,
release_id: str,
artifact: Any,
*,
expected_revision: int | None,
stage: str = "STAGED",
source_revision: str | None = None,
observed_main_revision: str | None = None,
rollback_release_id: str | None = None,
rollback_artifact: Any | None = None,
rollback_compatible: bool = False,
) -> EnvironmentDeploymentTarget:
tenant_id, subject_id, component_id, environment, release_id = map(
lambda pair: _required(pair[0], pair[1]),
((tenant_id,"tenant_id"),(subject_id,"subject_id"),(component_id,"component_id"),(environment,"environment"),(release_id,"release_id")),
)
self._authorize(tenant_id,subject_id,component_id,environment,"write")
if stage not in self.TARGET_STAGES:
raise ValueError("invalid target stage")
if expected_revision is not None and (not isinstance(expected_revision, int) or isinstance(expected_revision, bool) or expected_revision < 0):
raise ValueError("invalid expected_revision")
if type(rollback_compatible) is not bool:
raise ValueError("rollback_compatible must be boolean")
artifact_json = canonical_artifact(artifact)
try:
linked_release = self.lineage.get_release_for_component(tenant_id, component_id, release_id)
except KeyError as exc:
raise ReleaseConflict("deployment target release has no lineage") from exc
if canonical_artifact(linked_release["artifact"]) != artifact_json:
raise ReleaseConflict("deployment target artifact does not match release")
rollback_artifact_json = canonical_artifact(rollback_artifact) if rollback_artifact is not None else None
source_revision = _required(source_revision, "source_revision") if source_revision else None
observed_main_revision = _required(observed_main_revision, "observed_main_revision") if observed_main_revision else None
rollback_release_id = _required(rollback_release_id, "rollback_release_id") if rollback_release_id else None
if (rollback_release_id is None) != (rollback_artifact_json is None):
raise ValueError("rollback release and artifact must be supplied together")
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
current = connection.execute(
"SELECT * FROM release_targets WHERE tenant_id=? AND component_id=? AND environment=?",
(tenant_id, component_id, environment),
).fetchone()
if current is None:
if expected_revision is not None:
raise ReleaseConflict("expected revision does not match absent target")
if stage == "ROLLBACK_STAGED":
raise ReleaseConflict("rollback requires a current target")
revision = 1
previous_release_id = previous_artifact_json = None
else:
if expected_revision != current["revision"]:
raise ReleaseConflict("expected revision does not match target")
if stage == "ROLLBACK_STAGED":
if not current["rollback_compatible"]:
raise ReleaseConflict("current target has no compatible rollback")
if release_id != current["rollback_release_id"] or artifact_json != current["rollback_artifact_json"]:
raise ReleaseConflict("rollback target does not match compatible rollback binding")
revision = current["revision"] + 1
previous_release_id = current["release_id"]
previous_artifact_json = current["artifact_json"]
if rollback_compatible and rollback_release_id is None:
rollback_release_id = previous_release_id
rollback_artifact_json = previous_artifact_json
if rollback_compatible and current is not None and rollback_release_id is None:
raise ReleaseConflict("compatible rollback requires a previous target")
now = _now()
connection.execute(
"""INSERT INTO release_targets VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
ON CONFLICT(tenant_id,component_id,environment) DO UPDATE SET
release_id=excluded.release_id, artifact_json=excluded.artifact_json,
revision=excluded.revision, stage=excluded.stage,
source_revision=excluded.source_revision,
observed_main_revision=excluded.observed_main_revision,
previous_release_id=excluded.previous_release_id,
previous_artifact_json=excluded.previous_artifact_json,
rollback_release_id=excluded.rollback_release_id,
rollback_artifact_json=excluded.rollback_artifact_json,
rollback_compatible=excluded.rollback_compatible,
updated_by_subject_id=excluded.updated_by_subject_id,
updated_at=excluded.updated_at""",
(tenant_id,component_id,environment,release_id,artifact_json,revision,stage,
source_revision,observed_main_revision,previous_release_id,previous_artifact_json,
rollback_release_id,rollback_artifact_json,int(rollback_compatible),subject_id,now),
)
row = connection.execute(
"SELECT * FROM release_targets WHERE tenant_id=? AND component_id=? AND environment=?",
(tenant_id,component_id,environment),
).fetchone()
return self._target(row)
def get_target(self, tenant_id: str, subject_id: str, component_id: str, environment: str) -> EnvironmentDeploymentTarget:
tenant_id,subject_id,component_id,environment=(
_required(tenant_id,"tenant_id"),_required(subject_id,"subject_id"),
_required(component_id,"component_id"),_required(environment,"environment"),
)
self._authorize(tenant_id,subject_id,component_id,environment,"read")
with self._connect() as connection:
row = connection.execute(
"SELECT * FROM release_targets WHERE tenant_id=? AND component_id=? AND environment=?",
(tenant_id,component_id,environment),
).fetchone()
if row is None:
raise KeyError("deployment target not found")
return self._target(row)
@staticmethod
def _target(row: sqlite3.Row) -> EnvironmentDeploymentTarget:
return EnvironmentDeploymentTarget(
tenant_id=row["tenant_id"], component_id=row["component_id"], environment=row["environment"],
release_id=row["release_id"], artifact=decode_artifact(row["artifact_json"]), revision=row["revision"],
stage=row["stage"], source_revision=row["source_revision"], observed_main_revision=row["observed_main_revision"],
previous_release_id=row["previous_release_id"],
previous_artifact=decode_artifact(row["previous_artifact_json"]) if row["previous_artifact_json"] else None,
rollback_release_id=row["rollback_release_id"],
rollback_artifact=decode_artifact(row["rollback_artifact_json"]) if row["rollback_artifact_json"] else None,
rollback_compatible=bool(row["rollback_compatible"]),
updated_by_subject_id=row["updated_by_subject_id"], updated_at=row["updated_at"],
)
def start_saga(
self, tenant_id: str, subject_id: str, mission_id: str, component_id: str, saga_id: str,
release_id: str, deployment_id: str, environment: str, artifact: Any,
recipe_id: str, *, expected_target_revision: int,
) -> dict[str, Any]:
tenant_id,subject_id,mission_id,component_id,saga_id,release_id,deployment_id,environment,recipe_id=map(
lambda pair:_required(pair[0],pair[1]),
((tenant_id,"tenant_id"),(subject_id,"subject_id"),(mission_id,"mission_id"),(component_id,"component_id"),(saga_id,"saga_id"),
(release_id,"release_id"),(deployment_id,"deployment_id"),(environment,"environment"),(recipe_id,"recipe_id")),
)
if not isinstance(expected_target_revision,int) or isinstance(expected_target_revision,bool) or expected_target_revision < 1:
raise ValueError("invalid expected_target_revision")
self._authorize(tenant_id,subject_id,component_id,environment,"deploy")
artifact_json=canonical_artifact(artifact)
target=self.get_target(tenant_id,subject_id,component_id,environment)
if target.revision != expected_target_revision:
raise ReleaseConflict("expected target revision does not match")
if target.stage not in {"STAGED","ROLLBACK_STAGED"}:
raise ReleaseConflict("deployment target is not staged")
if target.release_id != release_id or canonical_artifact(target.artifact) != artifact_json:
raise ReleaseConflict("saga does not match deployment target")
graph=self.lineage.explain(tenant_id,subject_id,mission_id)
release=next((item for item in graph["releases"] if item["release_id"] == release_id),None)
if release is None or release["component_id"] != component_id or canonical_artifact(release["artifact"]) != artifact_json:
raise ReleaseConflict("release has no exact source lineage")
source_task=next((item for item in graph["tasks"] if item["task_id"] == release["source_task_id"]),None)
if source_task is None or source_task["component_id"] != component_id:
raise ReleaseConflict("release source Task component lineage missing")
with self._connect() as connection:
recipe=connection.execute("SELECT * FROM release_recipes WHERE recipe_id=?",(recipe_id,)).fetchone()
if recipe is None:
raise ReleaseConflict("recipe not found")
# The relationship is durable before saga admission and is idempotent on retry.
self.lineage.link_deployment(
tenant_id,subject_id,mission_id,release_id,deployment_id,component_id,environment,
expected_target_revision,decode_artifact(artifact_json),
metadata={"saga_id":saga_id,"recipe_id":recipe_id,"recipe_digest":recipe["recipe_digest"]},
)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
existing=connection.execute("SELECT * FROM release_sagas WHERE tenant_id=? AND saga_id=?",(tenant_id,saga_id)).fetchone()
expected=(subject_id,mission_id,component_id,release["source_task_id"],release["source_effect_id"],source_task["grant_id"],release_id,deployment_id,environment,artifact_json,recipe_id,recipe["recipe_digest"],expected_target_revision)
if existing:
actual=tuple(existing[key] for key in ("subject_id","mission_id","component_id","source_task_id","source_effect_id","grant_id","release_id","deployment_id","environment","artifact_json","recipe_id","recipe_digest","target_revision"))
if actual != expected:
raise ReleaseConflict("saga_id already has different immutable binding")
return self._saga(connection,existing)
collision=connection.execute("SELECT saga_id FROM release_sagas WHERE tenant_id=? AND deployment_id=?",(tenant_id,deployment_id)).fetchone()
if collision:
raise ReleaseConflict("deployment_id already has a different saga")
now=_now()
connection.execute(
"INSERT INTO release_sagas VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
(tenant_id,saga_id,subject_id,mission_id,component_id,release["source_task_id"],release["source_effect_id"],source_task["grant_id"],release_id,deployment_id,environment,artifact_json,
recipe_id,recipe["recipe_digest"],expected_target_revision,0,"RUNNING",0,now,now),
)
row=connection.execute("SELECT * FROM release_sagas WHERE tenant_id=? AND saga_id=?",(tenant_id,saga_id)).fetchone()
return self._saga(connection,row)
def get_saga(self, tenant_id: str, subject_id: str, saga_id: str) -> dict[str, Any]:
with self._connect() as connection:
row=connection.execute("SELECT * FROM release_sagas WHERE tenant_id=? AND saga_id=?",(_required(tenant_id,"tenant_id"),_required(saga_id,"saga_id"))).fetchone()
if row is None:
raise KeyError("release saga not found")
if row["subject_id"] != _required(subject_id,"subject_id"):
raise ReleaseAuthorizationError("release saga subject denied")
return self._saga(connection,row)
@staticmethod
def _stages(connection: sqlite3.Connection, recipe_id: str) -> list[dict[str, Any]]:
row=connection.execute("SELECT stages_json FROM release_recipes WHERE recipe_id=?",(recipe_id,)).fetchone()
if row is None:
raise ReleaseConflict("persisted recipe missing")
return json.loads(row["stages_json"])
def _saga(self, connection: sqlite3.Connection, row: sqlite3.Row) -> dict[str, Any]:
stages=self._stages(connection,row["recipe_id"])
index=row["stage_index"]
return {
"tenant_id":row["tenant_id"],"subject_id":row["subject_id"],"mission_id":row["mission_id"],
"component_id":row["component_id"],"source_task_id":row["source_task_id"],
"source_effect_id":row["source_effect_id"],"grant_id":row["grant_id"],
"saga_id":row["saga_id"],"release_id":row["release_id"],"deployment_id":row["deployment_id"],
"environment":row["environment"],"artifact":decode_artifact(row["artifact_json"]),
"recipe_id":row["recipe_id"],"recipe_digest":row["recipe_digest"],
"target_revision":row["target_revision"],"stage_index":index,
"stage_id":stages[index]["stage_id"] if index < len(stages) else None,
"state":row["state"],"revision":row["revision"],"created_at":row["created_at"],"updated_at":row["updated_at"],
}
def advance_saga(
self, tenant_id: str, subject_id: str, saga_id: str, expected_revision: int,
operation_adapter: Any,
) -> dict[str, Any]:
tenant_id,subject_id,saga_id=(
_required(tenant_id,"tenant_id"),_required(subject_id,"subject_id"),_required(saga_id,"saga_id")
)
if not isinstance(expected_revision,int) or isinstance(expected_revision,bool) or expected_revision < 0:
raise ValueError("invalid expected_revision")
prepare=getattr(operation_adapter,"prepare",None)
execute=getattr(operation_adapter,"execute",None)
if not callable(prepare) or not callable(execute):
raise ValueError("governed operation adapter requires prepare and execute")
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
saga=connection.execute(
"SELECT * FROM release_sagas WHERE tenant_id=? AND saga_id=?",(tenant_id,saga_id)
).fetchone()
if saga is None:
raise KeyError("release saga not found")
if saga["subject_id"] != subject_id:
raise ReleaseAuthorizationError("saga belongs to different subject")
self._authorize(tenant_id,subject_id,saga["component_id"],saga["environment"],"deploy")
if saga["revision"] != expected_revision:
raise ReleaseConflict("expected revision does not match saga")
if saga["state"] == "COMPLETED":
return self._saga(connection,saga)
stages=self._stages(connection,saga["recipe_id"])
stage=stages[saga["stage_index"]]
effect_id=f"release:{saga_id}:{saga['stage_index']}:{stage['stage_id']}"
intent=connection.execute(
"SELECT * FROM release_effect_intents WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(tenant_id,saga_id,saga["stage_index"]),
).fetchone()
if intent is None:
now=_now()
connection.execute(
"INSERT INTO release_effect_intents VALUES(?,?,?,?,?,?,?,?,?,?,?,?)",
(tenant_id,saga_id,saga["stage_index"],stage["stage_id"],stage["operation_id"],effect_id,
None,"PENDING_JOB",None,None,now,now),
)
elif (intent["effect_id"],intent["stage_id"],intent["operation_id"]) != (
effect_id,stage["stage_id"],stage["operation_id"]
):
raise ReleaseConflict("persisted stage effect identity mismatch")
operation={
"tenant_id":tenant_id,"subject_id":subject_id,"mission_id":saga["mission_id"],
"component_id":saga["component_id"],"task_id":saga["source_task_id"],
"source_task_id":saga["source_task_id"],"source_effect_id":saga["source_effect_id"],
"grant_id":saga["grant_id"],"saga_id":saga_id,
"deployment_id":saga["deployment_id"],"release_id":saga["release_id"],
"environment":saga["environment"],"artifact":decode_artifact(saga["artifact_json"]),
"target_revision":saga["target_revision"],"stage_index":saga["stage_index"],
"stage_id":stage["stage_id"],"operation_id":stage["operation_id"],"effect_id":effect_id,
"parameters":stage.get("parameters",{}),
}
with self._connect() as connection:
intent=connection.execute(
"SELECT * FROM release_effect_intents WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(tenant_id,saga_id,saga["stage_index"]),
).fetchone()
job_id=intent["job_id"]
if job_id is None:
prepared=prepare(dict(operation))
job_id=prepared.get("job_id") if isinstance(prepared,Mapping) else prepared
job_id=_required(job_id,"prepared job_id")
# This is the first point where the relationship graph may name a Job:
# link_effect independently resolves the configured real Job authority.
self.lineage.link_effect(
tenant_id,subject_id,saga["mission_id"],saga["source_task_id"],effect_id,job_id,
source_effect_id=saga["source_effect_id"],
metadata={"deployment_id":saga["deployment_id"],"release_id":saga["release_id"],
"component_id":saga["component_id"],"stage_id":stage["stage_id"]},
)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
current=connection.execute(
"SELECT revision,stage_index FROM release_sagas WHERE tenant_id=? AND saga_id=?",
(tenant_id,saga_id),
).fetchone()
if current is None or current["revision"] != expected_revision or current["stage_index"] != saga["stage_index"]:
raise ReleaseConflict("expected revision changed before Job binding")
bound=connection.execute(
"SELECT job_id FROM release_effect_intents WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(tenant_id,saga_id,saga["stage_index"]),
).fetchone()
if bound["job_id"] not in (None,job_id):
raise ReleaseConflict("stage effect already bound to different Job")
connection.execute(
"UPDATE release_effect_intents SET job_id=?,state='JOB_LINKED',updated_at=? WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(job_id,_now(),tenant_id,saga_id,saga["stage_index"]),
)
operation["job_id"]=job_id
with self._connect() as connection:
intent=connection.execute(
"SELECT * FROM release_effect_intents WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(tenant_id,saga_id,saga["stage_index"]),
).fetchone()
if intent["receipt_json"]:
raw_receipt=json.loads(intent["receipt_json"])
elif intent["state"] == "EFFECT_UNKNOWN":
reconcile=getattr(operation_adapter,"reconcile",None)
if not callable(reconcile):
raise ReleaseReconciliationRequired("RECONCILIATION_REQUIRED: adapter readback unavailable")
raw_receipt=reconcile(dict(operation))
else:
try:
raw_receipt=execute(dict(operation))
except Exception:
with self._connect() as connection:
connection.execute(
"UPDATE release_effect_intents SET state='EFFECT_UNKNOWN',updated_at=? WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(_now(),tenant_id,saga_id,saga["stage_index"]),
)
raise
if not isinstance(raw_receipt,Mapping):
with self._connect() as connection:
connection.execute(
"UPDATE release_effect_intents SET state='EFFECT_UNKNOWN',updated_at=? WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(_now(),tenant_id,saga_id,saga["stage_index"]),
)
raise ReleaseVerificationError("operation adapter receipt must be an object")
try:
unverified_receipt_json=_json(dict(raw_receipt))
except (TypeError,ValueError) as exc:
raise ReleaseVerificationError("operation adapter receipt must be canonical JSON") from exc
if len(unverified_receipt_json.encode("utf-8")) > 262_144:
raise ReleaseVerificationError("operation adapter receipt is too large")
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
current=connection.execute(
"SELECT revision,stage_index FROM release_sagas WHERE tenant_id=? AND saga_id=?",(tenant_id,saga_id)
).fetchone()
if current is None or current["revision"] != expected_revision or current["stage_index"] != saga["stage_index"]:
raise ReleaseConflict("expected revision changed before receipt persistence")
connection.execute(
"UPDATE release_effect_intents SET state='RECEIPT_UNVERIFIED',receipt_json=?,updated_at=? WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(unverified_receipt_json,_now(),tenant_id,saga_id,saga["stage_index"]),
)
expected={key:operation[key] for key in (
"tenant_id","subject_id","component_id","task_id","source_task_id","source_effect_id","grant_id",
"job_id","effect_id","operation_id","deployment_id","release_id","artifact"
)}
try:
verified=self.receipt_verifier(raw_receipt,expected)
except Exception as exc:
raise ReleaseVerificationError("configured verifier rejected operation receipt") from exc
if not isinstance(verified,VerifiedReceipt):
raise ReleaseVerificationError("receipt_verifier must return VerifiedReceipt")
if verified.outcome != "SUCCEEDED":
raise ReleaseVerificationError("verified receipt outcome is not SUCCEEDED")
fields=("tenant_id","subject_id","component_id","task_id","source_task_id","source_effect_id","grant_id",
"job_id","effect_id","operation_id","deployment_id","release_id","artifact")
bindings=tuple(canonical_artifact(getattr(verified,name)) if name == "artifact" else getattr(verified,name) for name in fields)
expected_bindings=tuple(canonical_artifact(operation[name]) if name == "artifact" else operation[name] for name in fields)
if bindings != expected_bindings:
wrong=next(name for name,actual,wanted in zip(fields,bindings,expected_bindings) if actual != wanted)
raise ReleaseVerificationError(f"verified receipt {wrong} binding mismatch")
receipt_id=_required(verified.receipt_id,"receipt_id")
if not isinstance(verified.observed,Mapping):
raise ReleaseVerificationError("verified receipt observation must be an object")
observed_json=_json(dict(verified.observed))
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
current=connection.execute("SELECT * FROM release_sagas WHERE tenant_id=? AND saga_id=?",(tenant_id,saga_id)).fetchone()
if current["revision"] != expected_revision or current["stage_index"] != saga["stage_index"]:
raise ReleaseConflict("expected revision changed before receipt commit")
connection.execute(
"UPDATE release_effect_intents SET state='VERIFIED',receipt_id=?,updated_at=? WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(receipt_id,_now(),tenant_id,saga_id,saga["stage_index"]),
)
connection.execute(
"""INSERT OR IGNORE INTO release_deployment_facts
(tenant_id,subject_id,mission_id,component_id,task_id,grant_id,job_id,deployment_id,release_id,artifact_json,saga_id,stage_id,operation_id,effect_id,receipt_id,observed_json,created_at)
VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)""",
(tenant_id,subject_id,saga["mission_id"],saga["component_id"],saga["source_task_id"],saga["grant_id"],job_id,
saga["deployment_id"],saga["release_id"],saga["artifact_json"],saga_id,stage["stage_id"],stage["operation_id"],
effect_id,receipt_id,observed_json,_now()),
)
next_index=saga["stage_index"]+1
state="COMPLETED" if next_index == len(stages) else "RUNNING"
updated=connection.execute(
"UPDATE release_sagas SET stage_index=?,state=?,revision=revision+1,updated_at=? WHERE tenant_id=? AND saga_id=? AND revision=?",
(next_index,state,_now(),tenant_id,saga_id,expected_revision),
)
if updated.rowcount != 1:
raise ReleaseConflict("expected revision changed")
row=connection.execute("SELECT * FROM release_sagas WHERE tenant_id=? AND saga_id=?",(tenant_id,saga_id)).fetchone()
return self._saga(connection,row)
def deployment_facts(self, tenant_id: str, subject_id: str, component_id: str, deployment_id: str) -> list[dict[str, Any]]:
tenant_id,subject_id,component_id,deployment_id=(
_required(tenant_id,"tenant_id"),_required(subject_id,"subject_id"),
_required(component_id,"component_id"),_required(deployment_id,"deployment_id"),
)
with self._connect() as connection:
saga=connection.execute(
"SELECT environment,subject_id,component_id FROM release_sagas WHERE tenant_id=? AND deployment_id=?",
(tenant_id,deployment_id),
).fetchone()
if saga is None or saga["subject_id"] != subject_id or saga["component_id"] != component_id:
raise ReleaseAuthorizationError("deployment facts scope denied")
self._authorize(tenant_id,subject_id,component_id,saga["environment"],"read")
rows=connection.execute(
"SELECT * FROM release_deployment_facts WHERE tenant_id=? AND subject_id=? AND component_id=? AND deployment_id=? ORDER BY fact_id",
(tenant_id,subject_id,component_id,deployment_id),
).fetchall()
return [{**{key:row[key] for key in row.keys() if not key.endswith("_json")},
"artifact":decode_artifact(row["artifact_json"]),"observed":json.loads(row["observed_json"])} for row in rows]
def record_acceptance(
self, tenant_id: str, subject_id: str, mission_id: str, component_id: str, acceptance_id: str,
deployment_id: str, release_id: str, artifact: Any, accepted: bool, *,
evidence: Any,
verifier: Callable[[Any, Mapping[str, Any]], VerifiedAcceptanceEvidence],
) -> dict[str, Any]:
with self._connect() as connection:
saga=connection.execute(
"SELECT environment,subject_id,component_id FROM release_sagas WHERE tenant_id=? AND deployment_id=?",
(tenant_id,deployment_id),
).fetchone()
if saga is None or saga["subject_id"] != subject_id or saga["component_id"] != component_id:
raise ReleaseAuthorizationError("acceptance deployment scope denied")
self._authorize(tenant_id,subject_id,component_id,saga["environment"],"accept")
return self.lineage.record_acceptance(
tenant_id,subject_id,mission_id,component_id,acceptance_id,deployment_id,release_id,
artifact,accepted,evidence=evidence,verifier=verifier,
)
__all__ = [
"EnvironmentDeploymentTarget", "ReleaseAuthorizationError", "ReleaseConflict", "ReleaseError", "ReleaseLedger",
"ReleaseReconciliationRequired", "ReleaseVerificationError", "VerifiedReceipt",
]
@@ -0,0 +1,375 @@
"""Pure validation and deployment-truth evaluation for runtime contracts.
Runtime contracts describe stable structure. A release-owned deployment target
is supplied separately so that changing a release does not rewrite structural
configuration or turn a source branch into runtime evidence.
"""
from __future__ import annotations
from copy import deepcopy
from datetime import datetime, timezone
import re
from pathlib import PurePosixPath
from typing import Any, Mapping, Sequence
CONTRACT_SCHEMA = "aios.runtime-contract.v1"
LIFECYCLES = frozenset({"active", "candidate", "legacy", "example"})
_DYNAMIC_FIELDS = frozenset(
{"desired", "desired_commit", "desired_release", "desired_target", "desired_targets"}
)
_SECRET_FIELDS = frozenset(
{"secret", "secret_bytes", "token", "password", "private_key", "credential_value", "value"}
)
_DIGEST = re.compile(r"^(?:sha256:)?[0-9a-fA-F]{64}$")
_SAFE_SYMBOLIC_REF = re.compile(r"^[A-Za-z][A-Za-z0-9+.-]*:[/A-Za-z0-9_.:@+-]+$")
def _error(code: str, path: str = "") -> ValueError:
return ValueError(f"{code}:{path}" if path else code)
def _nonempty_string(value: object, path: str) -> str:
if not isinstance(value, str) or not value.strip():
raise _error("field_invalid", path)
return value.strip()
def _positive_int(value: object, path: str) -> int:
if isinstance(value, bool) or not isinstance(value, int) or value < 1:
raise _error("field_invalid", path)
return value
def _reject_dynamic_fields(value: object, path: str = "contract") -> None:
if isinstance(value, Mapping):
for key, child in value.items():
if not isinstance(key, str):
raise _error("field_name_invalid", path)
child_path = f"{path}.{key}"
if key.lower() in _DYNAMIC_FIELDS:
raise _error("dynamic_field_forbidden", child_path)
_reject_dynamic_fields(child, child_path)
elif isinstance(value, Sequence) and not isinstance(value, (str, bytes, bytearray)):
for index, child in enumerate(value):
_reject_dynamic_fields(child, f"{path}[{index}]")
def _safe_path_ref(value: object, path: str) -> str:
text = _nonempty_string(value, path)
if "\x00" in text or "\\" in text:
raise _error("unsafe_path_ref", path)
candidate = PurePosixPath(text)
if candidate.is_absolute() or any(part in {"", ".", ".."} for part in candidate.parts):
raise _error("unsafe_path_ref", path)
return text
def _symbolic_ref(value: object, path: str) -> str:
text = _nonempty_string(value, path)
if "\x00" in text or ".." in text or not _SAFE_SYMBOLIC_REF.fullmatch(text):
raise _error("resolver_ref_invalid", path)
return text
def _credential_ref(value: object, path: str) -> str:
text = _nonempty_string(value, path)
allowed = (
text.startswith("credential://")
or text.startswith("secret://")
or text.startswith("systemd-credential:")
)
if not allowed or "\x00" in text or ".." in text or text.startswith("/run/credentials/"):
raise _error("credential_ref_invalid", path)
return text
def _validate_binding_group(
group: object,
group_name: str,
component_id: str,
environment: str,
tenant_scope: str,
) -> dict[str, Any]:
if not isinstance(group, Mapping):
raise _error("binding_group_invalid", group_name)
normalized: dict[str, Any] = {}
for name, raw in group.items():
item_path = f"{group_name}.{name}"
_nonempty_string(name, f"{group_name}.name")
if isinstance(raw, str):
reference_field = (
"credential_ref"
if group_name == "credential_refs"
else "path_ref"
if group_name == "resource_refs" and ":" not in raw
else "resolver_ref"
)
item = {
"component_id": component_id,
"environment": environment,
reference_field: raw,
}
elif isinstance(raw, Mapping):
item = deepcopy(dict(raw))
else:
raise _error("binding_invalid", item_path)
if _nonempty_string(item.get("component_id"), f"{item_path}.component_id") != component_id:
raise _error("binding_component_mismatch", item_path)
if _nonempty_string(item.get("environment"), f"{item_path}.environment") != environment:
raise _error("binding_environment_mismatch", item_path)
if "tenant_scope" in item and (
_nonempty_string(item["tenant_scope"], f"{item_path}.tenant_scope") != tenant_scope
):
raise _error("binding_tenant_mismatch", item_path)
reference_fields = [key for key in ("resolver_ref", "path_ref", "credential_ref") if key in item]
if len(reference_fields) != 1:
raise _error("binding_reference_invalid", item_path)
reference_field = reference_fields[0]
reference_path = f"{item_path}.{reference_field}"
if group_name == "credential_refs" or reference_field == "credential_ref":
for key in item:
if key.lower() in _SECRET_FIELDS:
raise _error("secret_material_forbidden", f"{item_path}.{key}")
item[reference_field] = _credential_ref(item[reference_field], reference_path)
elif reference_field == "path_ref":
item[reference_field] = _safe_path_ref(item[reference_field], reference_path)
else:
item[reference_field] = _symbolic_ref(item[reference_field], reference_path)
normalized[str(name)] = item
return normalized
def validate_contract(document: Mapping[str, Any]) -> dict[str, Any]:
"""Validate and copy a stable ``aios.runtime-contract.v1`` document.
The returned mapping owns its nested containers, so callers cannot mutate the
source after validation. Invalid contracts raise ``ValueError`` with a stable
machine-readable code prefix.
"""
if not isinstance(document, Mapping):
raise _error("contract_invalid")
_reject_dynamic_fields(document)
result = deepcopy(dict(document))
if result.get("contract_schema") != CONTRACT_SCHEMA:
raise _error("contract_schema_invalid", "contract_schema")
result["contract_revision"] = _positive_int(result.get("contract_revision"), "contract_revision")
component_id = _nonempty_string(result.get("component_id"), "component_id")
environment = _nonempty_string(result.get("environment"), "environment")
result["component_id"] = component_id
result["environment"] = environment
tenant_scope = _nonempty_string(result.get("tenant_scope"), "tenant_scope")
result["tenant_scope"] = tenant_scope
lifecycle = _nonempty_string(result.get("lifecycle"), "lifecycle").lower()
if lifecycle not in LIFECYCLES:
raise _error("lifecycle_invalid", "lifecycle")
result["lifecycle"] = lifecycle
if "resources" in result and "resource_refs" in result:
raise _error("duplicate_resource_group")
if "resources" in result:
result["resource_refs"] = result.pop("resources")
for group_name in ("entrypoints", "bindings", "resource_refs", "credential_refs"):
result[group_name] = _validate_binding_group(
result.get(group_name), group_name, component_id, environment, tenant_scope
)
operations = result.get("operations")
if (
not isinstance(operations, Sequence)
or isinstance(operations, (str, bytes, bytearray))
or not operations
):
raise _error("operations_invalid", "operations")
normalized_operations = [
_nonempty_string(operation, f"operations[{index}]")
for index, operation in enumerate(operations)
]
if len(set(normalized_operations)) != len(normalized_operations):
raise _error("duplicate_operation", "operations")
result["operations"] = normalized_operations
return result
def _unknown(*reasons: str) -> dict[str, Any]:
return {
"status": "UNKNOWN",
"state": "UNKNOWN",
"verification": "UNKNOWN",
"converged": False,
"reasons": list(dict.fromkeys(reasons)),
}
def _drift(reasons: list[str]) -> dict[str, Any]:
return {
"status": "DRIFT",
"state": "DRIFT",
"verification": "FAILED",
"converged": False,
"reasons": list(dict.fromkeys(reasons)),
}
def _parse_time(value: object) -> datetime | None:
if isinstance(value, datetime):
parsed = value
elif isinstance(value, str):
try:
parsed = datetime.fromisoformat(value.strip().replace("Z", "+00:00"))
except ValueError:
return None
else:
return None
if parsed.tzinfo is None or parsed.utcoffset() is None:
return None
return parsed.astimezone(timezone.utc)
def _digest_value(document: Mapping[str, Any], *names: str) -> object:
for name in names:
if name in document:
return document[name]
return None
def _valid_digest(value: object) -> bool:
return isinstance(value, str) and bool(_DIGEST.fullmatch(value))
def evaluate_deployment(
contract: Mapping[str, Any],
target: Mapping[str, Any] | None,
observation: Mapping[str, Any] | None,
provenance: Mapping[str, Any] | None,
now: datetime | str,
) -> dict[str, Any]:
"""Evaluate one target using fresh runtime and provenance evidence.
Missing or malformed evidence is ``UNKNOWN``. ``DRIFT`` is reserved for a
complete evidence set that proves an incompatible target or runtime fact.
Repository HEAD is intentionally not an input to this decision.
"""
stable = validate_contract(contract)
missing: list[str] = []
if target is None:
missing.append("target_missing")
elif not isinstance(target, Mapping):
missing.append("target_invalid")
if observation is None:
missing.append("observation_missing")
elif not isinstance(observation, Mapping):
missing.append("observation_invalid")
if provenance is None:
missing.append("provenance_missing")
elif not isinstance(provenance, Mapping):
missing.append("provenance_invalid")
current_time = _parse_time(now)
if current_time is None:
missing.append("now_invalid")
if missing:
return _unknown(*missing)
assert target is not None and observation is not None and provenance is not None
assert current_time is not None
target_errors: list[str] = []
for field in ("component_id", "environment", "release_id", "deployment_id"):
if not isinstance(target.get(field), str) or not str(target[field]).strip():
target_errors.append(f"target_{field}_invalid")
if "tenant_scope" in target and (
not isinstance(target["tenant_scope"], str) or not target["tenant_scope"].strip()
):
target_errors.append("target_tenant_scope_invalid")
target_artifact = target.get("artifact_digest")
if not _valid_digest(target_artifact):
target_errors.append("target_artifact_digest_invalid")
if isinstance(target.get("revision"), bool) or not isinstance(target.get("revision"), int) or target.get("revision", 0) < 1:
target_errors.append("target_revision_invalid")
observed_at = _parse_time(observation.get("observed_at"))
max_age = observation.get("max_age_seconds", observation.get("max_age"))
observed_artifact = _digest_value(
observation, "actual_artifact_digest", "actual_artifact", "artifact_digest"
)
observation_errors: list[str] = []
if observed_at is None:
observation_errors.append("observation_observed_at_invalid")
if isinstance(max_age, bool) or not isinstance(max_age, (int, float)) or max_age <= 0:
observation_errors.append("observation_max_age_invalid")
if not isinstance(observation.get("source"), str) or not observation.get("source", "").strip():
observation_errors.append("observation_source_invalid")
if not _valid_digest(observed_artifact):
observation_errors.append("observation_actual_artifact_invalid")
if not isinstance(observation.get("healthy"), bool):
observation_errors.append("observation_healthy_invalid")
for field in ("component_id", "environment", "tenant_scope", "deployment_id"):
if field in observation and (
not isinstance(observation[field], str) or not observation[field].strip()
):
observation_errors.append(f"observation_{field}_invalid")
if "revision" in observation and (
isinstance(observation["revision"], bool)
or not isinstance(observation["revision"], int)
or observation["revision"] < 1
):
observation_errors.append("observation_revision_invalid")
artifact_provenance = _digest_value(provenance, "artifact_digest", "artifact_hash")
source_provenance = _digest_value(provenance, "source_digest", "source_hash")
builder_provenance = _digest_value(provenance, "builder_digest", "builder_hash")
provenance_errors: list[str] = []
if not _valid_digest(artifact_provenance):
provenance_errors.append("provenance_artifact_digest_invalid")
if not _valid_digest(source_provenance):
provenance_errors.append("provenance_source_digest_invalid")
if not _valid_digest(builder_provenance):
provenance_errors.append("provenance_builder_digest_invalid")
evidence_errors = target_errors + observation_errors + provenance_errors
if evidence_errors:
return _unknown(*evidence_errors)
assert observed_at is not None and isinstance(max_age, (int, float))
age = (current_time - observed_at).total_seconds()
if age < 0:
return _unknown("observation_from_future")
if age > max_age:
return _unknown("observation_stale")
drift: list[str] = []
if target["component_id"] != stable["component_id"]:
drift.append("target_component_id_mismatch")
if target["environment"] != stable["environment"]:
drift.append("target_environment_mismatch")
if "tenant_scope" in target and target["tenant_scope"] != stable["tenant_scope"]:
drift.append("target_tenant_scope_mismatch")
if "tenant_scope" in observation and observation["tenant_scope"] != stable["tenant_scope"]:
drift.append("observation_tenant_scope_mismatch")
for field in ("component_id", "environment", "deployment_id", "revision"):
if field in observation and observation[field] != target[field]:
drift.append(f"observation_{field}_mismatch")
if observed_artifact != target_artifact:
drift.append("artifact_mismatch")
if artifact_provenance != target_artifact:
drift.append("provenance_artifact_mismatch")
if observation["healthy"] is False:
drift.append("observation_unhealthy")
if drift:
return _drift(drift)
return {
"status": "VERIFIED",
"state": "CONVERGED",
"verification": "VERIFIED",
"converged": True,
"reasons": [],
"component_id": target["component_id"],
"environment": target["environment"],
"release_id": target["release_id"],
"deployment_id": target["deployment_id"],
"artifact_digest": target_artifact,
"revision": target["revision"],
"observed_at": observation["observed_at"],
"observation_source": observation["source"],
}
+450
View File
@@ -0,0 +1,450 @@
"""Thin authenticated application service over existing TaskPlaneStore and JobStore.
This module owns no second Task lifecycle store. Added task runtime columns live
in the supplied Task authority database; the lineage ledger owns only relations.
The built-in host adapter is READ_ONLY. Mutating operations require separately
registered governed adapters with receiver fencing/readback, never arbitrary argv.
"""
from __future__ import annotations
from dataclasses import dataclass
from contextlib import contextmanager
from datetime import datetime,timezone
import base64
import json
import re
import sqlite3
import threading
import time
from pathlib import Path
from typing import Mapping
from task_plane.models import TaskPlan,WorkerRegistration,ProbeEvidence,ProbeOutcome,Budget
from task_plane.errors import TaskPlaneHTTPError
from aios_gitops_job_controller import GitOpsJobController,JobStore,JobState,JobError
from .evidence import ReceiptSigner,canonical,digest
from .runtime_contract import validate_contract
from .worker_policy import eligible
from .authorization import AuthorizationError
from .host import HostError
class ServiceError(RuntimeError):
def __init__(self,code,status=409):
super().__init__(code)
self.code=code
self.status=status
@dataclass(frozen=True)
class Identity:
tenant_id: str
subject_id: str
role: str
worker_id: str | None=None
class BaselineService:
def __init__(self,store,job_root,authorization,contracts,operations,worker_profiles,host,lineage,machine_key,
*,machine_key_id='baseline-machine',trusted_receipt_keys=None):
self.store=store
self.signer=ReceiptSigner(machine_key,key_id=machine_key_id,trusted_public_keys=trusted_receipt_keys)
self.jobs=GitOpsJobController(JobStore(Path(job_root)),generic_receipt_verifier=self.signer.verify)
self.authorization=authorization
self.contracts={}
for entry in contracts:
value=validate_contract(entry)
key=(value['tenant_scope'],value['environment'],value['component_id'])
if key in self.contracts: raise ServiceError('CONTRACT_DUPLICATE')
self.contracts[key]=value
self.operations=operations
self.worker_profiles=dict(worker_profiles)
self.host=host
self.lineage=lineage
self.storage_blocked=False
self._execute_lock=threading.Lock()
with self.store._transaction():
self.store.conn.execute('''CREATE TABLE IF NOT EXISTS baseline_task_runtime(
task_id TEXT PRIMARY KEY REFERENCES tasks(task_id),ready_at REAL,
blocker_code TEXT,job_id TEXT,effect_id TEXT NOT NULL UNIQUE)''')
self.store.conn.execute('''CREATE TABLE IF NOT EXISTS baseline_worker_policy(
worker_id TEXT PRIMARY KEY,policy_json TEXT NOT NULL)''')
self.store.probe_attestation_verifier=self._verify_probe
def _user(self,identity):
if not isinstance(identity,Identity) or identity.role not in {'user','owner'}:
raise ServiceError('USER_IDENTITY_REQUIRED',403)
def _worker(self,identity):
if not isinstance(identity,Identity) or identity.role!='worker':
raise ServiceError('WORKER_IDENTITY_REQUIRED',403)
profile=self.worker_profiles.get(identity.worker_id)
if not profile or profile.identity!=identity.subject_id or identity.tenant_id not in profile.tenant_scope:
raise ServiceError('WORKER_SCOPE_DENIED',403)
return profile
def _task(self,identity,task_id):
task=self.store.get_task(task_id)
meta=task.task_plan.metadata.get('baseline',{})
if meta.get('tenant_id')!=identity.tenant_id or meta.get('subject_id')!=identity.subject_id:
raise ServiceError('TASK_SCOPE_DENIED',403)
return task
def _scope(self,task):
meta=task.task_plan.metadata['baseline']
return {**{key:meta[key] for key in ('tenant_id','subject_id','mission_id','operation','component_id','environment')},'task_id':task.task_id}
def _principal(self,task):
meta=task.task_plan.metadata['baseline']
return self.authorization.issue_principal(meta['grant_id'],ttl_seconds=60,**self._scope(task))
def _fresh_write(self):
if self.storage_blocked: raise ServiceError('STORAGE_RECOVERY_REQUIRED',503)
try:
with self.store._transaction():
self.store.conn.execute('UPDATE baseline_task_runtime SET blocker_code=blocker_code WHERE 0')
except (sqlite3.Error,OSError) as exc:
self.storage_blocked=True
raise ServiceError('TASK_STORAGE_UNAVAILABLE',503) from exc
def submit_goal(self,identity,request,*,idempotency_key):
self._user(identity)
allowed={'mission_id','grant_id','operation','component_id','environment','inputs','intent'}
if type(request) is not dict or set(request)!=allowed or not isinstance(idempotency_key,str) or not 1<=len(idempotency_key)<=128:
raise ServiceError('GOAL_SCHEMA_INVALID',400)
if any(not isinstance(request[key],str) or not request[key] or len(request[key])>2048 for key in allowed-{'inputs'}):
raise ServiceError('GOAL_SCHEMA_INVALID',400)
if request['inputs']!={}: raise ServiceError('OPERATION_INPUT_SCHEMA_INVALID',400)
if not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9_.:-]{0,127}',request['mission_id']): raise ServiceError('MISSION_ID_INVALID',400)
operation=self.operations.get(request['operation'])
if operation.handler_ref!='host.service_status' or operation.effect_class!='READ_ONLY' or operation.required_permissions!=(operation.operation_id,):
raise ServiceError('REGISTERED_HANDLER_UNAVAILABLE',503)
contract=self.contracts.get((identity.tenant_id,request['environment'],request['component_id']))
if not contract or contract['lifecycle']!='active' or request['operation'] not in contract['operations']:
raise ServiceError('CONTRACT_OPERATION_UNAVAILABLE',403)
host_key=identity.tenant_id+':'+request['environment']+':'+request['component_id']
if host_key not in self.host.bindings: raise ServiceError('HOST_BINDING_UNAVAILABLE',503)
key=digest({'tenant':identity.tenant_id,'subject':identity.subject_id,'key':idempotency_key})
task_id='task-'+key[:40]
effect_id='effect-'+digest({'tenant':identity.tenant_id,'task':task_id,'request':request})[:40]
meta={**request,'tenant_id':identity.tenant_id,'subject_id':identity.subject_id,'effect_id':effect_id,
'operation_version':operation.version,'contract_revision':contract['contract_revision']}
plan=TaskPlan.from_dict({'task_id':task_id,'task_type':'host.diagnostic','intent':request['intent'],
'issuer':{'id':identity.subject_id,'tenant_id':identity.tenant_id,'subject_id':identity.subject_id},
'required_capabilities':[request['operation']],'risk':'low','approval_policy':'governed-grant',
'correlation_id':request['mission_id'],'metadata':{'baseline':meta},
'budgets':{'attempts':operation.retry['max_attempts'],'time_seconds':30*operation.retry['max_attempts']},'evidence_requirements':['diagnostic']})
# Validate business authorization before creating an executable task.
self.authorization.issue_principal(request['grant_id'],ttl_seconds=60,tenant_id=identity.tenant_id,
subject_id=identity.subject_id,task_id=task_id,**{key:request[key] for key in ('mission_id','operation','component_id','environment')})
self._fresh_write()
try:
with self.store._transaction():
task=self.store.create_task_with_event(plan,idempotency_key=key)
self.store.conn.execute('INSERT OR IGNORE INTO baseline_task_runtime(task_id,ready_at,effect_id) VALUES(?,?,?)',
(task_id,time.time(),effect_id))
self.lineage.link_task(identity.tenant_id,identity.subject_id,request['mission_id'],request['grant_id'],task_id)
return self.get_task(identity,task_id)
except TaskPlaneHTTPError as exc:
raise ServiceError('TASK_CONFLICT') from exc
except (sqlite3.Error,OSError) as exc:
self.storage_blocked=True
raise ServiceError('TASK_STORAGE_UNAVAILABLE',503) from exc
def get_task(self,identity,task_id):
self._user(identity)
task=self._task(identity,task_id)
with self.store._transaction():
row=self.store.conn.execute('SELECT * FROM baseline_task_runtime WHERE task_id=?',(task_id,)).fetchone()
result=task.to_dict()
if row:
result.update({'ready_at':row['ready_at'],'blocker_code':row['blocker_code'],'effect_id':row['effect_id'],'job_id':row['job_id']})
if row['job_id']:
job=self.jobs.store.read(row['job_id'])
result['job_state']=job['state']
result['side_effect_state']=job['side_effect_state']
envelope=json.loads(job['receipt_json']) if job.get('receipt_json') else None
if self._receipt_valid(task,envelope): result['receipt']=envelope['payload']
result['remaining_budget']=self.store.get_task_budget(task_id).to_dict()
return result
def explain(self,identity,mission_id):
self._user(identity)
result=self.lineage.explain(identity.tenant_id,identity.subject_id,mission_id)
result['tasks_runtime']=[self.get_task(identity,t.task_id) for t in self.store.list_tasks()
if t.task_plan.metadata.get('baseline',{}).get('mission_id')==mission_id and
t.task_plan.issuer.get('tenant_id')==identity.tenant_id and t.task_plan.issuer.get('subject_id')==identity.subject_id]
return result
def register_worker(self,identity):
profile=self._worker(identity)
try:
existing=self.store.get_worker(profile.worker_id)
except TaskPlaneHTTPError:
existing=None
now=time.time()
worker=WorkerRegistration.from_dict({'worker_id':profile.worker_id,'worker_type':'baseline-operation-worker',
'identity':profile.identity,'capabilities':list(profile.capabilities),'trust_zone':'governed-host',
'max_concurrency':profile.max_concurrency,'heartbeat_interval_seconds':profile.heartbeat_seconds,
'supported_protocols':['aios-baseline-worker-v1'],'last_heartbeat':now})
self.store.register_worker(worker,actor_identity=profile.identity,is_admin=False,
expected_revision=existing.registration_revision if existing else None)
with self.store._transaction():
self.store.conn.execute('INSERT INTO baseline_worker_policy VALUES(?,?) ON CONFLICT(worker_id) DO UPDATE SET policy_json=excluded.policy_json',
(profile.worker_id,canonical(profile.__dict__).decode()))
return {'worker_id':profile.worker_id,'registered':True,'fallback_executor':profile.fallback_executor}
def heartbeat(self,identity):
profile=self._worker(identity)
worker=self.store.get_worker(profile.worker_id)
self.store.heartbeat_worker(profile.worker_id,actor_identity=profile.identity,expected_revision=worker.registration_revision)
return {'worker_id':profile.worker_id,'alive':True}
def claim_next(self,identity):
profile=self._worker(identity)
worker=self.store.get_worker(profile.worker_id)
if worker.disabled: raise ServiceError('WORKER_DISABLED',403)
self._fresh_write()
for task in sorted(self.store.list_tasks(),key=lambda t:(t.created_at,t.task_id)):
meta=task.task_plan.metadata.get('baseline',{})
if not meta or task.status not in {'CREATED','QUEUED','FAILED_RECOVERABLE'}: continue
with self.store._transaction():
row=self.store.conn.execute('SELECT ready_at FROM baseline_task_runtime WHERE task_id=?',(task.task_id,)).fetchone()
if not row: continue
now=time.time()
task_scope={**meta,'ready_at':row['ready_at']}
if not eligible(profile,task_scope,now=now,last_heartbeat=worker.last_heartbeat): continue
# Priority is configuration on ordinary eligible workers; no special main actor.
candidates=[]
for registered in self.store.list_workers():
peer=self.worker_profiles.get(registered.worker_id)
if not peer or registered.disabled or not eligible(peer,task_scope,now=now,last_heartbeat=registered.last_heartbeat): continue
with self.store._transaction():
active=self.store.conn.execute("SELECT count(*) FROM claims WHERE worker_id=? AND status='ACTIVE' AND lease_until>?",(peer.worker_id,now)).fetchone()[0]
if active<peer.max_concurrency: candidates.append(peer)
if not candidates or sorted(candidates,key=lambda p:(-p.priority,p.worker_id))[0].worker_id!=profile.worker_id: continue
try:
self._principal(task)
with self.store._transaction():
self.store.consume_budget(task.task_id,Budget(attempts=1,time_seconds=30))
claim=self.store.claim_task(task.task_id,profile.worker_id,profile.identity,lease_seconds=30,
expected_revision=task.revision,operation_digest=task.task_plan.operation_digest())
except TaskPlaneHTTPError:
budget=self.store.get_task_budget(task.task_id)
if budget.attempts==0 or budget.time_seconds==0:
with self.store._transaction():
self.store.conn.execute('UPDATE baseline_task_runtime SET blocker_code=? WHERE task_id=?',('TASK_BUDGET_EXHAUSTED',task.task_id))
continue
except AuthorizationError:
with self.store._transaction():
self.store.conn.execute('UPDATE baseline_task_runtime SET blocker_code=? WHERE task_id=?',('AUTHORIZATION_REQUIRED',task.task_id))
continue
return {'task_id':task.task_id,'claim_id':claim.claim_id,'generation':claim.attempt,
'lease_until':claim.lease_until,'operation':meta['operation']}
return None
def renew(self,identity,task_id,claim_id):
profile=self._worker(identity)
claim=self.store.get_claim(claim_id)
if claim.task_id!=task_id or claim.worker_id!=profile.worker_id or claim.worker_identity!=profile.identity:
raise ServiceError('CLAIM_IDENTITY_MISMATCH',403)
task=self.store.get_task(task_id)
meta=task.task_plan.metadata.get('baseline',{})
if meta.get('tenant_id') not in profile.tenant_scope or meta.get('subject_id') not in profile.subject_scope or task.status=='CANCELLED':
raise ServiceError('CLAIM_SCOPE_OR_STATE_DENIED',403)
self._principal(task)
self.store.heartbeat_claim(task_id,claim_id,30)
return {'task_id':task_id,'claim_id':claim_id,'renewed':True}
def _advance(self,job_id,state,**kwargs):
row=self.jobs.store.read(job_id)
return self.jobs.advance(job_id,state,expected_revision=int(row['revision']),**kwargs)
@contextmanager
def _execution_slot(self,deadline):
remaining=deadline-time.time()
if remaining<=0 or not self._execute_lock.acquire(timeout=remaining):
raise ServiceError('EXECUTION_DEADLINE_EXPIRED',408)
try:
if time.time()>=deadline: raise ServiceError('EXECUTION_DEADLINE_EXPIRED',408)
yield
finally: self._execute_lock.release()
def execute_claim(self,identity,task_id,claim_id,*,deadline=None):
profile=self._worker(identity)
deadline=time.time()+30 if deadline is None else deadline
with self._execution_slot(deadline):
self._fresh_write()
task=self.store.get_task(task_id)
meta=task.task_plan.metadata.get('baseline',{})
claim=self.store.get_claim(claim_id)
if claim.task_id!=task_id or claim.worker_id!=profile.worker_id or claim.worker_identity!=profile.identity:
raise ServiceError('CLAIM_IDENTITY_MISMATCH',403)
if meta.get('tenant_id') not in profile.tenant_scope or meta.get('subject_id') not in profile.subject_scope:
raise ServiceError('WORKER_SCOPE_DENIED',403)
user=Identity(meta['tenant_id'],meta['subject_id'],'user')
if task.status=='COMPLETED': return self.get_task(user,task_id)
if claim.status!='ACTIVE' or claim.lease_until<=time.time() or task.status=='CANCELLED':
raise ServiceError('CLAIM_EXPIRED_OR_CANCELLED')
principal=self._principal(task)
self.authorization.authorize(principal,**self._scope(task))
operation=self.operations.get(meta['operation'])
if operation.handler_ref!='host.service_status' or operation.effect_class!='READ_ONLY':
raise ServiceError('HANDLER_NOT_REGISTERED',503)
self.authorization.assert_effect_bounds(principal,operation,**self._scope(task))
request={'schema':'aios.operation-job.v1','tenant_id':meta['tenant_id'],'subject_id':meta['subject_id'],
'mission_id':meta['mission_id'],'grant_id':meta['grant_id'],'task_id':task_id,'effect_id':meta['effect_id'],
'operation':meta['operation'],'operation_version':str(operation.version),'resource_kind':'systemd-service',
'resource_id':digest({'tenant':meta['tenant_id'],'environment':meta['environment'],'component':meta['component_id']}),'input_digest':digest(meta['inputs'])}
try:
job=self.jobs.submit(request)
with self.store._transaction():
self.store.conn.execute('UPDATE baseline_task_runtime SET job_id=? WHERE task_id=?',(job.job_id,task_id))
self.lineage.link_task(meta['tenant_id'],meta['subject_id'],meta['mission_id'],meta['grant_id'],task_id)
self.lineage.link_effect(meta['tenant_id'],meta['subject_id'],meta['mission_id'],task_id,meta['effect_id'],job.job_id)
row=self.jobs.store.read(job.job_id)
if row['state']=='COMPLETED':
self._close_task(task,claim,json.loads(row['receipt_json']))
return self.get_task(user,task_id)
if row['state'] not in {'REQUESTED','AUTHORIZED'}:
raise ServiceError('JOB_RECONCILIATION_REQUIRED')
self.jobs.assert_effect_admission(exclude_job_id=job.job_id)
self.authorization.reserve_effect(meta['grant_id'],meta['effect_id'],principal,**self._scope(task))
if row['state']=='REQUESTED': self._advance(job.job_id,JobState.AUTHORIZED,side_effect_state='AUTHORIZED')
row=self.jobs.store.read(job.job_id)
self.jobs.lease_resource_job(job.job_id,lease_id=claim_id,resource_kind=request['resource_kind'],
resource_id=request['resource_id'],generation=None,
lease_expires_at=datetime.fromtimestamp(min(claim.lease_until,time.time()+30,deadline),timezone.utc).isoformat(),
expected_revision=int(row['revision']))
effect_lease=self.jobs.get_execution_lease(job.job_id)
fence_generation=int(effect_lease['generation'])
self._advance(job.job_id,JobState.EXECUTING,side_effect_state='IN_PROGRESS')
# Last authorization/claim check directly before the registered host boundary.
self.authorization.authorize(principal,**self._scope(task))
if self.store.get_task(task_id).status=='CANCELLED' or self.store.get_claim(claim_id).lease_until<=time.time():
raise ServiceError('CLAIM_EXPIRED_OR_CANCELLED')
self.jobs.assert_execution_lease(job.job_id,generation=fence_generation,lease_id=claim_id)
if time.time()>=deadline: raise ServiceError('EXECUTION_DEADLINE_EXPIRED',408)
observation=self.host.status(meta['component_id'],meta['environment'],tenant_id=meta['tenant_id'],deadline=deadline)
if time.time()>=deadline or self.store.get_claim(claim_id).lease_until<=time.time():
raise ServiceError('EXECUTION_DEADLINE_EXPIRED',408)
envelope=self.signer.sign({'task_id':task_id,'job_id':job.job_id,'effect_id':meta['effect_id'],
'tenant_id':meta['tenant_id'],'subject_id':meta['subject_id'],'mission_id':meta['mission_id'],
'grant_id':meta['grant_id'],'claim_id':claim_id,'generation':fence_generation,
'operation':meta['operation'],'operation_version':str(operation.version),
'input_digest':request['input_digest'],'resource_kind':request['resource_kind'],
'resource_id':request['resource_id'],'observation':observation,'verified':True})
self._advance(job.job_id,JobState.LOCAL_VERIFIED,side_effect_state='READBACK_VERIFIED',receipt=envelope)
self._advance(job.job_id,JobState.COMPLETED,side_effect_state='READBACK_VERIFIED',receipt=envelope)
self._close_task(task,claim,envelope)
return self.get_task(user,task_id)
except (sqlite3.Error,OSError) as exc:
self.storage_blocked=True
raise ServiceError('STORAGE_RECOVERY_REQUIRED',503) from exc
except HostError as exc:
try:
self._advance(job.job_id,JobState.RECONCILIATION_REQUIRED,side_effect_state='UNKNOWN')
with self.store._transaction():
self.store.conn.execute('UPDATE baseline_task_runtime SET blocker_code=? WHERE task_id=?',(str(exc),task_id))
except (sqlite3.Error,OSError,JobError): self.storage_blocked=True
raise ServiceError(str(exc),503) from exc
except JobError as exc:
if 'UNAVAILABLE' in str(exc) or 'STORAGE' in str(exc): self.storage_blocked=True
raise ServiceError(str(exc),503 if self.storage_blocked else 409) from exc
def _receipt_valid(self,task,envelope):
meta=task.task_plan.metadata['baseline']
if not self.signer.verify(envelope): return False
payload=envelope['payload']
with self.store._transaction():
runtime=self.store.conn.execute('SELECT job_id FROM baseline_task_runtime WHERE task_id=?',(task.task_id,)).fetchone()
if not runtime or not runtime['job_id']: return False
expected={**{k:meta[k] for k in ('effect_id','tenant_id','subject_id','mission_id','grant_id','operation')},
'task_id':task.task_id,'job_id':runtime['job_id'],'operation_version':str(meta['operation_version']),
'input_digest':digest(meta['inputs']),'resource_kind':'systemd-service',
'resource_id':digest({'tenant':meta['tenant_id'],'environment':meta['environment'],'component':meta['component_id']})}
if any(payload.get(k)!=v for k,v in expected.items()) or payload.get('verified') is not True: return False
try:
lease=self.jobs.get_execution_lease(runtime['job_id'])
source_claim=self.store.get_claim(payload.get('claim_id'))
if payload.get('claim_id')!=lease['lease_id'] or payload.get('generation')!=lease['generation']: return False
if source_claim.task_id!=task.task_id or source_claim.operation_digest!=task.task_plan.operation_digest(): return False
except (JobError,TaskPlaneHTTPError,TypeError): return False
observation=payload.get('observation')
if type(observation) is not dict: return False
for k in ('operation','tenant_id','component_id','environment'):
if observation.get(k)!=meta[k]: return False
binding=meta['tenant_id']+':'+meta['environment']+':'+meta['component_id']
if observation.get('unit')!=self.host.bindings.get(binding) or observation.get('source')!='systemd.show': return False
return True
def _close_task(self,task,claim,envelope):
if not self._receipt_valid(task,envelope): raise ServiceError('RESULT_ATTESTATION_INVALID',403)
current=self.store.get_task(task.task_id)
if current.status=='CANCELLED': raise ServiceError('TASK_CANCELLED')
if current.status=='CLAIMED': self.store.update_task_status(task.task_id,current.revision,'RUNNING')
self.store.complete_claim(task.task_id,claim.claim_id,'COMPLETED',envelope,
evidence_entries=[('diagnostic',envelope,claim.worker_identity)])
def cancel_task(self,identity,task_id):
self._user(identity)
task=self._task(identity,task_id)
self.store.cancel_task_with_event(task_id,task.revision,'cancelled by authenticated subject')
return self.get_task(identity,task_id)
def _verify_probe(self,probe):
try:
self.signer.public_key.verify(base64.b64decode(probe.signature,validate=True),canonical(probe.digest_payload()))
return True
except Exception:
return False
def recover(self):
"""Reconcile expired baseline claims from durable signed receipts; never fake a result."""
with self._execute_lock:
return self._recover_locked()
def _recover_locked(self):
recovered=[]
with self.store._transaction():
rows=self.store.conn.execute("SELECT claim_id FROM claims WHERE status='ACTIVE' AND lease_until<?",(time.time(),)).fetchall()
for row in rows:
claim=self.store.get_claim(row['claim_id'])
task=self.store.get_task(claim.task_id)
meta=task.task_plan.metadata.get('baseline')
if not meta: continue
with self.store._transaction():
rt=self.store.conn.execute('SELECT job_id FROM baseline_task_runtime WHERE task_id=?',(task.task_id,)).fetchone()
outcome=ProbeOutcome.NOT_APPLIED
ref='read-only-no-persistent-effect'
if rt and rt['job_id']:
job=self.jobs.store.read(rt['job_id'])
env=json.loads(job['receipt_json']) if job.get('receipt_json') else None
if job['state']=='COMPLETED':
if self._receipt_valid(task,env):
outcome=ProbeOutcome.APPLIED_VERIFIED
ref=digest(env)
else: outcome=ProbeOutcome.INDETERMINATE
elif self.operations.get(meta['operation']).effect_class!='READ_ONLY':
outcome=ProbeOutcome.INDETERMINATE
now=time.time()
probe=ProbeEvidence.create(task_id=task.task_id,claim_id=claim.claim_id,worker_id=claim.worker_id,
worker_identity=claim.worker_identity,operation_digest=claim.operation_digest,outcome=outcome,
observed_at=now,expires_at=now+30,probe_executor_identity='baseline-reconciler',attestation_ref=ref)
from dataclasses import replace
probe=replace(probe,signature=base64.b64encode(self.signer.private_key.sign(canonical(probe.digest_payload()))).decode())
self.store.recover_claim_with_probe_and_event(task.task_id,claim.claim_id,probe)
if outcome==ProbeOutcome.NOT_APPLIED and rt and rt['job_id']:
job=self.jobs.store.read(rt['job_id'])
if job['state']!='COMPLETED':
self._advance(rt['job_id'],JobState.RECONCILIATION_REQUIRED,side_effect_state='UNKNOWN')
self.jobs.reconcile(rt['job_id'],lambda _:{'state':'REQUESTED','side_effect_state':'READ_ONLY_SAFE_TO_RETRY'})
recovered.append({'task_id':task.task_id,'outcome':outcome.value})
# A successful writable transaction permits recovery processing, not blind effect replay.
with self.store._transaction():
self.store.conn.execute('UPDATE baseline_task_runtime SET blocker_code=blocker_code WHERE 0')
try:
self.jobs.assert_effect_admission()
self.storage_blocked=False
except JobError:
self.storage_blocked=True
return recovered
@@ -0,0 +1,513 @@
"""Authenticated loopback HTTP transport for the Baseline 1.1 service."""
from __future__ import annotations
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import ipaddress
import json
import math
import re
import time
import threading
from typing import Any, Mapping
from urllib.error import HTTPError, URLError
from urllib.parse import quote, unquote, urlsplit
from urllib.request import HTTPRedirectHandler, ProxyHandler, Request, build_opener
from .service import Identity, ServiceError
MAX_REQUEST_BYTES = 65_536
MAX_RESPONSE_BYTES = 1_048_576
MAX_DEADLINE_SECONDS = 30
_IDENTIFIER = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.:-]{0,127}")
_IDENTITY_FIELDS = frozenset({"tenant_id", "subject_id", "role", "worker_id", "identity"})
class TransportError(RuntimeError):
def __init__(self, code: str, status: int | None = None):
self.code = code
self.status = status
super().__init__(code)
class _RequestError(RuntimeError):
def __init__(self, code: str, status: int):
self.code = code
self.status = status
super().__init__(code)
def _safe_identifier(value: Any, code: str = "HTTP_PATH_INVALID") -> str:
if not isinstance(value, str) or _IDENTIFIER.fullmatch(value) is None:
raise _RequestError(code, 400)
return value
def _has_identity_field(value: Any) -> bool:
if isinstance(value, Mapping):
if any(key in _IDENTITY_FIELDS for key in value):
return True
return any(_has_identity_field(item) for item in value.values())
if isinstance(value, list):
return any(_has_identity_field(item) for item in value)
return False
def _loopback_host(host: str, code: str) -> str:
if not isinstance(host, str):
raise TransportError(code)
try:
address = ipaddress.ip_address(host)
except ValueError as exc:
raise TransportError(code) from exc
if not address.is_loopback or address.version != 4:
raise TransportError(code)
return str(address)
class _BaselineHTTPServer(ThreadingHTTPServer):
daemon_threads = True
allow_reuse_address = True
def __init__(self, address, handler, service, identities):
self.baseline_service = service
self.trusted_identities = identities
self.request_slots=threading.BoundedSemaphore(16)
super().__init__(address, handler)
def get_request(self):
connection, address = super().get_request()
connection.settimeout(MAX_DEADLINE_SECONDS)
return connection, address
def process_request(self,request,client_address):
if not self.request_slots.acquire(blocking=False):
try:
body=b'{"error":{"code":"HTTP_CAPACITY_EXCEEDED"}}'
request.sendall(b'HTTP/1.1 503 Service Unavailable\r\nContent-Type: application/json\r\nConnection: close\r\nContent-Length: '+str(len(body)).encode()+b'\r\n\r\n'+body)
except OSError: pass
self.shutdown_request(request)
return
try: super().process_request(request,client_address)
except Exception:
self.request_slots.release()
raise
def process_request_thread(self,request,client_address):
try: super().process_request_thread(request,client_address)
finally: self.request_slots.release()
class _Handler(BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
server_version = "AIOSBaselineHTTP/1"
sys_version = ""
def log_message(self, _format, *_args):
# Request paths can contain tenant-owned identifiers; do not emit them or
# bearer-related failure details through the default stderr logger.
return
def send_error(self, code, message=None, explain=None):
# BaseHTTPRequestHandler otherwise emits an HTML page which may include
# parser details. Keep pre-dispatch protocol failures in the same closed
# JSON error format.
status = code if isinstance(code, int) and 400 <= code <= 599 else 400
self._send(status, {"error": {"code": "HTTP_PROTOCOL_ERROR"}})
def _send(self, status: int, value: Any) -> None:
try:
body = json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=False,
allow_nan=False,
).encode("utf-8")
except (TypeError, ValueError):
status = 500
body = b'{"error":{"code":"HTTP_RESPONSE_INVALID"}}'
if len(body) > MAX_RESPONSE_BYTES:
status = 500
body = b'{"error":{"code":"HTTP_RESPONSE_TOO_LARGE"}}'
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.send_header("Cache-Control", "no-store")
self.send_header("X-Content-Type-Options", "nosniff")
self.send_header("Connection", "close")
self.end_headers()
try:
self.wfile.write(body)
except OSError:
# A peer can disconnect or let its deadline expire while a response
# is being written. Do not let the server print a handler traceback.
pass
self.close_connection = True
def _error(self, error: _RequestError) -> None:
self._send(error.status, {"error": {"code": error.code}})
def _deadline(self) -> float:
values = self.headers.get_all("X-AIOS-Deadline", [])
if len(values) != 1:
raise _RequestError("HTTP_DEADLINE_REQUIRED", 400)
try:
deadline = float(values[0])
except (TypeError, ValueError) as exc:
raise _RequestError("HTTP_DEADLINE_INVALID", 400) from exc
now = time.time()
if not math.isfinite(deadline) or deadline > now + MAX_DEADLINE_SECONDS:
raise _RequestError("HTTP_DEADLINE_INVALID", 400)
if deadline <= now:
raise _RequestError("HTTP_DEADLINE_EXPIRED", 408)
self.connection.settimeout(deadline - now)
return deadline
def _identity(self) -> Identity:
values = self.headers.get_all("Authorization", [])
if len(values) != 1 or not values[0].startswith("Bearer "):
raise _RequestError("HTTP_UNAUTHORIZED", 401)
token = values[0][7:]
if not token or any(ord(char) < 33 or ord(char) > 126 for char in token):
raise _RequestError("HTTP_UNAUTHORIZED", 401)
identity = self.server.trusted_identities.get(token)
if identity is None:
raise _RequestError("HTTP_UNAUTHORIZED", 401)
return identity
def _json_body(self) -> dict:
if self.headers.get("Transfer-Encoding") is not None:
raise _RequestError("HTTP_TRANSFER_ENCODING_FORBIDDEN", 400)
content_types = self.headers.get_all("Content-Type", [])
if content_types != ["application/json"]:
raise _RequestError("HTTP_CONTENT_TYPE_INVALID", 415)
lengths = self.headers.get_all("Content-Length", [])
if len(lengths) != 1 or not lengths[0].isdigit():
raise _RequestError("HTTP_CONTENT_LENGTH_INVALID", 411)
length = int(lengths[0])
if length > MAX_REQUEST_BYTES:
raise _RequestError("HTTP_BODY_TOO_LARGE", 413)
if length <= 0:
raise _RequestError("HTTP_BODY_REQUIRED", 400)
raw = self.rfile.read(length)
if len(raw) != length:
raise _RequestError("HTTP_BODY_TRUNCATED", 400)
try:
def unique_object(pairs):
result = {}
for key, item in pairs:
if key in result:
raise _RequestError("HTTP_JSON_DUPLICATE_KEY", 400)
result[key] = item
return result
def reject_constant(_value):
raise _RequestError("HTTP_JSON_NON_FINITE", 400)
value = json.loads(
raw.decode("utf-8"), object_pairs_hook=unique_object,
parse_constant=reject_constant,
)
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise _RequestError("HTTP_JSON_INVALID", 400) from exc
if type(value) is not dict:
raise _RequestError("HTTP_JSON_OBJECT_REQUIRED", 400)
if _has_identity_field(value):
raise _RequestError("HTTP_IDENTITY_FIELD_FORBIDDEN", 400)
return value
def _path(self) -> list[str]:
parsed = urlsplit(self.path)
if parsed.query or parsed.fragment or not parsed.path.startswith("/"):
raise _RequestError("HTTP_PATH_INVALID", 400)
raw_segments = parsed.path.split("/")[1:]
if any(not segment for segment in raw_segments):
raise _RequestError("HTTP_NOT_FOUND", 404)
segments = [unquote(segment) for segment in raw_segments]
if any("/" in segment or "\\" in segment for segment in segments):
raise _RequestError("HTTP_PATH_INVALID", 400)
return segments
@staticmethod
def _exact(body: Mapping[str, Any], fields: set[str]) -> None:
if set(body) != fields:
raise _RequestError("HTTP_BODY_SCHEMA_INVALID", 400)
def _dispatch(self, method: str, identity: Identity, segments: list[str], body: dict | None):
service = self.server.baseline_service
if method == "POST" and segments == ["v1", "goals", "submit"]:
self._exact(body, {"request", "idempotency_key"})
if type(body["request"]) is not dict:
raise _RequestError("HTTP_BODY_SCHEMA_INVALID", 400)
return 201, service.submit_goal(
identity, body["request"], idempotency_key=body["idempotency_key"]
)
if method == "GET" and len(segments) == 3 and segments[:2] == ["v1", "tasks"]:
return 200, service.get_task(identity, _safe_identifier(segments[2]))
if (
method == "POST" and len(segments) == 4
and segments[:2] == ["v1", "tasks"] and segments[3] == "cancel"
):
self._exact(body, set())
return 200, service.cancel_task(identity, _safe_identifier(segments[2]))
if (
method == "GET" and len(segments) == 4
and segments[:2] == ["v1", "missions"] and segments[3] == "explain"
):
return 200, service.explain(identity, _safe_identifier(segments[2]))
if method == "POST" and segments == ["v1", "workers", "register"]:
self._exact(body, set())
return 200, service.register_worker(identity)
if method == "POST" and segments == ["v1", "workers", "heartbeat"]:
self._exact(body, set())
return 200, service.heartbeat(identity)
if method == "POST" and segments == ["v1", "workers", "claim"]:
self._exact(body, set())
return 200, service.claim_next(identity)
if method == "POST" and segments == ["v1", "workers", "renew"]:
self._exact(body, {"task_id", "claim_id"})
return 200, service.renew(
identity, _safe_identifier(body["task_id"]), _safe_identifier(body["claim_id"])
)
if method == "POST" and segments == ["v1", "workers", "execute"]:
self._exact(body, {"task_id", "claim_id"})
return 200, service.execute_claim(
identity, _safe_identifier(body["task_id"]), _safe_identifier(body["claim_id"]),deadline=self.request_deadline
)
raise _RequestError("HTTP_NOT_FOUND", 404)
def _handle(self, method: str) -> None:
try:
self.request_deadline=self._deadline()
identity = self._identity()
segments = self._path()
body = self._json_body() if method == "POST" else None
if method == "GET":
lengths = self.headers.get_all("Content-Length", [])
if self.headers.get("Transfer-Encoding") is not None:
raise _RequestError("HTTP_GET_BODY_FORBIDDEN", 400)
if len(lengths) > 1 or (lengths and (not lengths[0].isdigit() or int(lengths[0]) != 0)):
raise _RequestError("HTTP_GET_BODY_FORBIDDEN", 400)
status, result = self._dispatch(method, identity, segments, body)
self._send(status, result)
except _RequestError as exc:
self._error(exc)
except ServiceError as exc:
status = exc.status if isinstance(exc.status, int) and 400 <= exc.status <= 599 else 500
self._send(status, {"error": {"code": exc.code}})
except TimeoutError:
self._send(408, {"error": {"code": "HTTP_DEADLINE_EXPIRED"}})
except Exception:
# No exception string, traceback, request headers, or token crosses
# this boundary.
self._send(500, {"error": {"code": "HTTP_INTERNAL_ERROR"}})
def do_GET(self):
self._handle("GET")
def do_POST(self):
self._handle("POST")
def _method_not_allowed(self):
try:
self._deadline()
self._identity()
self._send(405, {"error": {"code": "HTTP_METHOD_NOT_ALLOWED"}})
except _RequestError as exc:
self._error(exc)
do_PUT = _method_not_allowed
do_PATCH = _method_not_allowed
do_DELETE = _method_not_allowed
do_OPTIONS = _method_not_allowed
do_HEAD = _method_not_allowed
do_TRACE = _method_not_allowed
do_CONNECT = _method_not_allowed
def serve(
service: Any,
tokens: Mapping[str, Identity],
host: str = "127.0.0.1",
port: int = 0,
) -> ThreadingHTTPServer:
"""Build an unstarted real HTTP server bound only to an IPv4 loopback."""
host = _loopback_host(host, "HTTP_LOOPBACK_REQUIRED")
if type(port) is not int or not 0 <= port <= 65_535:
raise TransportError("HTTP_PORT_INVALID")
if not isinstance(tokens, Mapping) or not tokens:
raise TransportError("HTTP_TRUSTED_IDENTITIES_REQUIRED")
identities: dict[str, Identity] = {}
for token, identity in tokens.items():
if (
not isinstance(token, str) or not 32<=len(token)<=512
or any(ord(char) < 33 or ord(char) > 126 for char in token)
or not isinstance(identity, Identity)
):
raise TransportError("HTTP_TRUSTED_IDENTITY_INVALID")
identities[token] = identity
return _BaselineHTTPServer((host, port), _Handler, service, identities)
class _NoRedirect(HTTPRedirectHandler):
def redirect_request(self, req, fp, code, msg, headers, newurl):
return None
class HttpTransport:
"""Small urllib client for the loopback API; redirects and proxies are off."""
def __init__(
self,
base_url: str,
token: str,
*,
timeout_seconds: float = 15,
max_response_bytes: int = MAX_RESPONSE_BYTES,
):
try:
parsed = urlsplit(base_url)
parsed_port = parsed.port
except ValueError as exc:
raise TransportError("TRANSPORT_BASE_URL_INVALID") from exc
if (
parsed.scheme != "http" or parsed.username is not None or parsed.password is not None
or parsed.query or parsed.fragment or parsed.path not in {"", "/"}
or parsed.hostname is None or parsed_port is None
):
raise TransportError("TRANSPORT_BASE_URL_INVALID")
_loopback_host(parsed.hostname, "TRANSPORT_LOOPBACK_REQUIRED")
if not isinstance(token, str) or not 32<=len(token)<=512 or any(
ord(char) < 33 or ord(char) > 126 for char in token
):
raise TransportError("TRANSPORT_TOKEN_INVALID")
if (
isinstance(timeout_seconds, bool)
or not isinstance(timeout_seconds, (int, float))
or not math.isfinite(float(timeout_seconds))
or not 0.1 <= timeout_seconds <= MAX_DEADLINE_SECONDS
):
raise TransportError("TRANSPORT_TIMEOUT_INVALID")
if type(max_response_bytes) is not int or not 1 <= max_response_bytes <= 4 * 1024 * 1024:
raise TransportError("TRANSPORT_RESPONSE_LIMIT_INVALID")
self.base_url = f"http://{parsed.hostname}:{parsed_port}"
self.token = token
self.timeout_seconds = float(timeout_seconds)
self.max_response_bytes = max_response_bytes
self._opener = build_opener(ProxyHandler({}), _NoRedirect())
def _decode(self, response) -> Any:
content_type = response.headers.get("Content-Type", "").split(";", 1)[0]
if content_type != "application/json":
raise TransportError("TRANSPORT_RESPONSE_TYPE_INVALID")
declared = response.headers.get("Content-Length")
if declared is None or not declared.isdigit() or int(declared) > self.max_response_bytes:
raise TransportError("TRANSPORT_RESPONSE_TOO_LARGE")
raw = response.read(self.max_response_bytes + 1)
if len(raw) > self.max_response_bytes or len(raw) != int(declared):
raise TransportError("TRANSPORT_RESPONSE_SIZE_INVALID")
try:
def unique_object(pairs):
result = {}
for key, item in pairs:
if key in result:
raise ValueError("duplicate")
result[key] = item
return result
return json.loads(
raw.decode("utf-8"), object_pairs_hook=unique_object,
parse_constant=lambda _value: (_ for _ in ()).throw(ValueError("non-finite")),
)
except (UnicodeDecodeError, json.JSONDecodeError, ValueError) as exc:
raise TransportError("TRANSPORT_RESPONSE_JSON_INVALID") from exc
def _request(self, method: str, path: str, body: Mapping[str, Any] | None = None) -> Any:
if not isinstance(path, str) or not path.startswith("/") or "?" in path or "#" in path:
raise TransportError("TRANSPORT_PATH_INVALID")
data = None
headers = {
"Authorization": "Bearer " + self.token,
"Accept": "application/json",
"X-AIOS-Deadline": str(time.time() + self.timeout_seconds),
}
if method == "POST":
if not isinstance(body, Mapping):
raise TransportError("TRANSPORT_BODY_INVALID")
try:
data = json.dumps(
body, sort_keys=True, separators=(",", ":"), ensure_ascii=False,
allow_nan=False,
).encode("utf-8")
except (TypeError, ValueError) as exc:
raise TransportError("TRANSPORT_BODY_INVALID") from exc
if not data or len(data) > MAX_REQUEST_BYTES:
raise TransportError("TRANSPORT_BODY_TOO_LARGE")
headers["Content-Type"] = "application/json"
request = Request(self.base_url + path, data=data, headers=headers, method=method)
try:
with self._opener.open(request, timeout=self.timeout_seconds) as response:
return self._decode(response)
except HTTPError as exc:
if 300 <= exc.code < 400:
exc.close()
raise TransportError("TRANSPORT_REDIRECT_REJECTED", exc.code) from None
try:
payload = self._decode(exc)
except TransportError:
payload = None
except (TimeoutError, OSError):
raise TransportError("TRANSPORT_UNAVAILABLE") from None
finally:
exc.close()
code = None
if isinstance(payload, Mapping) and isinstance(payload.get("error"), Mapping):
code = payload["error"].get("code")
if not isinstance(code, str) or _IDENTIFIER.fullmatch(code) is None:
code = "TRANSPORT_HTTP_ERROR"
raise TransportError(code, exc.code) from None
except (URLError, TimeoutError, OSError):
raise TransportError("TRANSPORT_UNAVAILABLE") from None
def submit_goal(self, request: Mapping[str, Any], *, idempotency_key: str) -> dict:
return self._request(
"POST", "/v1/goals/submit",
{"request": request, "idempotency_key": idempotency_key},
)
def get_task(self, task_id: str) -> dict:
return self._request("GET", "/v1/tasks/" + quote(task_id, safe=""))
def cancel_task(self, task_id: str) -> dict:
return self._request("POST", "/v1/tasks/" + quote(task_id, safe="") + "/cancel", {})
def explain(self, mission_id: str) -> dict:
return self._request("GET", "/v1/missions/" + quote(mission_id, safe="") + "/explain")
def register_worker(self) -> dict:
return self._request("POST", "/v1/workers/register", {})
def heartbeat(self) -> dict:
return self._request("POST", "/v1/workers/heartbeat", {})
def claim_next(self) -> dict | None:
return self._request("POST", "/v1/workers/claim", {})
def renew(self, task_id: str, claim_id: str) -> dict:
return self._request(
"POST", "/v1/workers/renew", {"task_id": task_id, "claim_id": claim_id}
)
def execute_claim(self, task_id: str, claim_id: str) -> dict:
return self._request(
"POST", "/v1/workers/execute", {"task_id": task_id, "claim_id": claim_id}
)
# Goal-oriented names used by thin clients.
submit = submit_goal
query = get_task
cancel = cancel_task
Transport = HttpTransport
__all__ = ["HttpTransport", "Transport", "TransportError", "serve"]
@@ -0,0 +1,204 @@
"""Verification-only Ed25519 boundaries for release and acceptance evidence."""
from __future__ import annotations
import base64
import json
import re
from types import MappingProxyType
from typing import Any, Callable, Mapping
from cryptography.exceptions import InvalidSignature
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
from .lineage import VerifiedAcceptanceEvidence, canonical_artifact
from .release import VerifiedReceipt
RELEASE_RECEIPT_DOMAIN = b"aios.release-operation-receipt.v1"
ACCEPTANCE_EVIDENCE_DOMAIN = b"aios.release-acceptance-evidence.v1"
_ENVELOPE_FIELDS = {"schema", "key_id", "payload", "signature"}
_RECEIPT_FIELDS = {
"receipt_id", "tenant_id", "subject_id", "component_id", "task_id",
"source_task_id", "source_effect_id", "grant_id", "job_id", "effect_id", "operation_id",
"deployment_id", "release_id", "artifact", "outcome", "observed", "observed_at",
}
_ACCEPTANCE_FIELDS = {
"acceptance_id", "tenant_id", "subject_id", "component_id", "deployment_id", "release_id",
"artifact", "accepted", "evidence_id", "observed", "observed_at",
}
_B64URL = re.compile(r"[A-Za-z0-9_-]+")
class SignatureVerificationError(ValueError):
"""A configured trust root did not authenticate an exact evidence payload."""
def _canonical(value: Any) -> bytes:
try:
return json.dumps(
value, ensure_ascii=False, sort_keys=True, separators=(",", ":"),
allow_nan=False,
).encode("utf-8")
except (TypeError, ValueError) as exc:
raise SignatureVerificationError("signed payload is not canonical JSON") from exc
def _decode_urlsafe(value: Any, name: str, expected_size: int) -> bytes:
if not isinstance(value, str) or not value or not _B64URL.fullmatch(value) or "=" in value:
raise SignatureVerificationError(f"invalid {name} encoding")
try:
raw = base64.urlsafe_b64decode(value + "=" * (-len(value) % 4))
except (ValueError, TypeError) as exc:
raise SignatureVerificationError(f"invalid {name} encoding") from exc
if len(raw) != expected_size:
raise SignatureVerificationError(f"invalid {name} size")
return raw
def _public_key(value: Any) -> Ed25519PublicKey:
if isinstance(value, Ed25519PublicKey):
return value
if isinstance(value, bytes):
raw = value
elif isinstance(value, str):
raw = _decode_urlsafe(value, "trusted public key", 32)
else:
raise ValueError("trusted public key must be Ed25519PublicKey, raw bytes, or base64url")
if len(raw) != 32:
raise ValueError("trusted Ed25519 public key must be 32 bytes")
return Ed25519PublicKey.from_public_bytes(raw)
def _trusted_keys(values: Mapping[str, Any]) -> Mapping[str, Ed25519PublicKey]:
if not isinstance(values, Mapping) or not values:
raise ValueError("trusted_public_keys must be a non-empty mapping")
keys: dict[str, Ed25519PublicKey] = {}
for key_id, value in values.items():
if not isinstance(key_id, str) or not key_id or len(key_id) > 128:
raise ValueError("invalid trusted key_id")
keys[key_id] = _public_key(value)
return MappingProxyType(keys)
def _verified_payload(
envelope: Any,
keys: Mapping[str, Ed25519PublicKey],
*,
schema: str,
domain: bytes,
payload_fields: set[str],
) -> Mapping[str, Any]:
if not isinstance(envelope, Mapping) or set(envelope) != _ENVELOPE_FIELDS:
raise SignatureVerificationError("signed envelope fields invalid")
if envelope.get("schema") != schema:
raise SignatureVerificationError("signed envelope schema mismatch")
key_id = envelope.get("key_id")
if not isinstance(key_id, str) or key_id not in keys:
raise SignatureVerificationError("untrusted signing key_id")
payload = envelope.get("payload")
if not isinstance(payload, Mapping) or set(payload) != payload_fields:
raise SignatureVerificationError("signed payload fields invalid")
signature = _decode_urlsafe(envelope.get("signature"), "signature", 64)
try:
keys[key_id].verify(signature, domain + b"\x00" + _canonical(dict(payload)))
except InvalidSignature as exc:
raise SignatureVerificationError("Ed25519 signature invalid") from exc
return payload
def _text(payload: Mapping[str, Any], name: str) -> str:
value = payload.get(name)
if not isinstance(value, str) or not value or len(value) > 512:
raise SignatureVerificationError(f"signed {name} invalid")
return value
def _expected(payload: Mapping[str, Any], expected: Mapping[str, Any], fields: tuple[str, ...]) -> None:
for name in fields:
if name not in expected:
raise SignatureVerificationError(f"verifier expected {name} missing")
actual = canonical_artifact(payload[name]) if name == "artifact" else payload[name]
wanted = canonical_artifact(expected[name]) if name == "artifact" else expected[name]
if actual != wanted:
raise SignatureVerificationError(f"signed {name} binding mismatch")
def ed25519_release_receipt_verifier(
trusted_public_keys: Mapping[str, Any],
) -> Callable[[Any, Mapping[str, Any]], VerifiedReceipt]:
"""Build a receipt verifier from local/configured public keys only."""
keys = _trusted_keys(trusted_public_keys)
def verify(envelope: Any, expected: Mapping[str, Any]) -> VerifiedReceipt:
payload = _verified_payload(
envelope, keys,
schema="aios.release-operation-receipt-envelope.v1",
domain=RELEASE_RECEIPT_DOMAIN,
payload_fields=_RECEIPT_FIELDS,
)
_expected(
payload, expected,
("tenant_id", "subject_id", "component_id", "task_id", "source_task_id", "source_effect_id",
"grant_id", "job_id", "effect_id", "operation_id", "deployment_id", "release_id", "artifact"),
)
if payload.get("outcome") != "SUCCEEDED":
raise SignatureVerificationError("signed receipt outcome is not SUCCEEDED")
if not isinstance(payload.get("observed"), Mapping):
raise SignatureVerificationError("signed receipt observation invalid")
return VerifiedReceipt(
receipt_id=_text(payload, "receipt_id"), tenant_id=_text(payload, "tenant_id"),
subject_id=_text(payload, "subject_id"), effect_id=_text(payload, "effect_id"),
component_id=_text(payload,"component_id"), task_id=_text(payload,"task_id"),
source_task_id=_text(payload,"source_task_id"), source_effect_id=_text(payload,"source_effect_id"),
grant_id=_text(payload,"grant_id"), job_id=_text(payload,"job_id"),
operation_id=_text(payload, "operation_id"), deployment_id=_text(payload, "deployment_id"),
release_id=_text(payload, "release_id"), artifact=payload["artifact"],
observed=dict(payload["observed"]), outcome="SUCCEEDED",
observed_at=_text(payload, "observed_at"),
)
return verify
def ed25519_acceptance_evidence_verifier(
trusted_public_keys: Mapping[str, Any],
) -> Callable[[Any, Mapping[str, Any]], VerifiedAcceptanceEvidence]:
"""Build an acceptance verifier in a domain separate from stage receipts."""
keys = _trusted_keys(trusted_public_keys)
def verify(envelope: Any, expected: Mapping[str, Any]) -> VerifiedAcceptanceEvidence:
payload = _verified_payload(
envelope, keys,
schema="aios.release-acceptance-evidence-envelope.v1",
domain=ACCEPTANCE_EVIDENCE_DOMAIN,
payload_fields=_ACCEPTANCE_FIELDS,
)
_expected(
payload, expected,
("acceptance_id", "tenant_id", "subject_id", "component_id", "deployment_id", "release_id", "artifact", "accepted"),
)
if type(payload.get("accepted")) is not bool:
raise SignatureVerificationError("signed accepted decision invalid")
if not isinstance(payload.get("observed"), Mapping):
raise SignatureVerificationError("signed acceptance observation invalid")
return VerifiedAcceptanceEvidence(
acceptance_id=_text(payload, "acceptance_id"), tenant_id=_text(payload, "tenant_id"),
subject_id=_text(payload, "subject_id"), deployment_id=_text(payload, "deployment_id"),
component_id=_text(payload,"component_id"),
release_id=_text(payload, "release_id"), artifact=payload["artifact"],
accepted=payload["accepted"], evidence_id=_text(payload, "evidence_id"),
observed=dict(payload["observed"]), observed_at=_text(payload, "observed_at"),
)
return verify
__all__ = [
"ACCEPTANCE_EVIDENCE_DOMAIN", "RELEASE_RECEIPT_DOMAIN",
"SignatureVerificationError", "ed25519_acceptance_evidence_verifier",
"ed25519_release_receipt_verifier",
]
+121
View File
@@ -0,0 +1,121 @@
"""Thin resident worker loop over the authenticated HTTP transport.
Worker policy, operation handlers, effects, receipts, and completion remain in
the trusted server. This process only maintains its registered transport
identity and asks the service to run an offered claim.
"""
from __future__ import annotations
import math
import threading
import time
from typing import Any
from .transport import TransportError
class WorkerError(RuntimeError):
pass
class OperationWorker:
HEARTBEAT_INTERVAL_SECONDS = 15.0
def __init__(self, transport: Any, poll_seconds: float = 1.0, stop_event=None):
required = (
"register_worker", "heartbeat", "claim_next", "renew", "execute_claim"
)
if transport is None or any(not callable(getattr(transport, name, None)) for name in required):
raise WorkerError("WORKER_TRANSPORT_INVALID")
if (
isinstance(poll_seconds, bool)
or not isinstance(poll_seconds, (int, float))
or not math.isfinite(float(poll_seconds))
or not 0.001 <= poll_seconds <= 60
):
raise WorkerError("WORKER_POLL_INTERVAL_INVALID")
if stop_event is not None and (
not callable(getattr(stop_event, "is_set", None))
or not callable(getattr(stop_event, "wait", None))
):
raise WorkerError("WORKER_STOP_EVENT_INVALID")
self.transport = transport
self.poll_seconds = float(poll_seconds)
self.stop_event = stop_event if stop_event is not None else threading.Event()
self.registered = False
self.worker_id: str | None = None
self._last_heartbeat = 0.0
self.last_error: str | None = None
def _register(self) -> None:
result = self.transport.register_worker()
if (
not isinstance(result, dict) or result.get("registered") is not True
or not isinstance(result.get("worker_id"), str) or not result["worker_id"]
):
raise WorkerError("WORKER_REGISTRATION_RESULT_INVALID")
self.worker_id = result["worker_id"]
self.registered = True
def _heartbeat(self, *, force: bool = False) -> None:
now = time.monotonic()
if not force and now < self._last_heartbeat + self.HEARTBEAT_INTERVAL_SECONDS:
return
result = self.transport.heartbeat()
if (
not isinstance(result, dict) or result.get("alive") is not True
or result.get("worker_id") != self.worker_id
):
raise WorkerError("WORKER_HEARTBEAT_RESULT_INVALID")
self._last_heartbeat = time.monotonic()
def run_once(self):
"""Perform one register/heartbeat/claim/renew/execute iteration."""
if self.stop_event.is_set():
return None
just_registered = False
if not self.registered:
self._register()
just_registered = True
self._heartbeat(force=just_registered)
claim = self.transport.claim_next()
if claim is None:
self.last_error = None
return None
required = {"task_id", "claim_id", "generation", "lease_until", "operation"}
if not isinstance(claim, dict) or set(claim) != required:
raise WorkerError("WORKER_CLAIM_RESULT_INVALID")
task_id = claim["task_id"]
claim_id = claim["claim_id"]
if not isinstance(task_id, str) or not task_id or not isinstance(claim_id, str) or not claim_id:
raise WorkerError("WORKER_CLAIM_RESULT_INVALID")
renewed = self.transport.renew(task_id, claim_id)
if (
not isinstance(renewed, dict) or renewed.get("renewed") is not True
or renewed.get("task_id") != task_id or renewed.get("claim_id") != claim_id
):
raise WorkerError("WORKER_RENEW_RESULT_INVALID")
result = self.transport.execute_claim(task_id, claim_id)
if not isinstance(result, dict) or result.get("task_id") != task_id:
raise WorkerError("WORKER_EXECUTION_RESULT_INVALID")
self.last_error = None
return result
def run_forever(self) -> None:
"""Run until the supplied stop event is set, using interruptible waits."""
while not self.stop_event.is_set():
try:
self.run_once()
except TransportError as exc:
# Transport failures are observable and retried on the next
# bounded poll. Re-register after a server restart; programming
# and schema errors still terminate.
self.last_error = exc.code
self.registered = False
self.stop_event.wait(self.poll_seconds)
Worker = OperationWorker
__all__ = ["OperationWorker", "Worker", "WorkerError"]
@@ -0,0 +1,51 @@
"""Registered worker eligibility; fallback is deployment policy, never a special actor."""
from dataclasses import dataclass
import math
from typing import Mapping
@dataclass(frozen=True)
class WorkerProfile:
worker_id: str
identity: str
tenant_scope: tuple[str, ...]
subject_scope: tuple[str, ...]
capabilities: tuple[str, ...]
priority: int
fallback_executor: bool
heartbeat_seconds: int
max_concurrency: int
@classmethod
def from_dict(cls, value: Mapping):
if set(value) != set(cls.__dataclass_fields__):
raise ValueError('WORKER_PROFILE_FIELDS_INVALID')
data = dict(value)
for name in ('worker_id', 'identity'):
if not isinstance(data[name], str) or not data[name] or len(data[name]) > 128:
raise ValueError('WORKER_ID_INVALID')
for name in ('tenant_scope', 'subject_scope', 'capabilities'):
items = data[name]
if not isinstance(items, (list, tuple)) or not items or any(not isinstance(x, str) or not x or '*' in x for x in items):
raise ValueError('WORKER_SCOPE_INVALID')
data[name] = tuple(sorted(set(items)))
if type(data['fallback_executor']) is not bool:
raise ValueError('WORKER_FALLBACK_INVALID')
for name, lower, upper in (('priority', 0, 1000), ('heartbeat_seconds', 1, 300), ('max_concurrency', 1, 32)):
if type(data[name]) is not int or not lower <= data[name] <= upper:
raise ValueError('WORKER_LIMIT_INVALID')
return cls(**data)
def eligible(worker: WorkerProfile, task: Mapping, *, now: float, last_heartbeat: float,
fallback_seconds: int = 600) -> bool:
ready_at = task.get('ready_at')
if any(type(value) not in (int,float) or not math.isfinite(value) for value in (ready_at,now,last_heartbeat)) or ready_at > now:
return False
if last_heartbeat > now or now > last_heartbeat + worker.heartbeat_seconds * 3:
return False
if task.get('tenant_id') not in worker.tenant_scope or task.get('subject_id') not in worker.subject_scope:
return False
if task.get('operation') not in worker.capabilities:
return False
return not worker.fallback_executor or now >= ready_at + fallback_seconds
+27 -1
View File
@@ -4,6 +4,8 @@ from __future__ import annotations
import hashlib
import json
import os
import re
from pathlib import Path
import stat
from typing import Any
@@ -13,6 +15,29 @@ class RecoveryEvidenceError(RuntimeError):
pass
# Relative evidence path: a safe relative path, never absolute, never with
# traversal segments, never with backslash separators.
_SAFE_EVIDENCE_REL = re.compile(r"^[A-Za-z0-9_./@-]+$")
def _contained_evidence(path: Path, base: Path) -> Path:
"""Resolve an evidence path and guarantee it stays under `base`."""
raw = str(path)
if (
os.path.isabs(raw)
or "\\" in raw
or raw.startswith("/")
or ".." in Path(raw).parts
or not _SAFE_EVIDENCE_REL.fullmatch(raw)
):
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_PATH_INVALID")
base_resolved = base.resolve()
target = (base_resolved / raw).resolve()
if base_resolved != target and not str(target).startswith(str(base_resolved) + os.sep):
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_PATH_ESCAPE")
return target
def _digest(path: Path) -> str:
h = hashlib.sha256()
with path.open("rb") as stream:
@@ -81,10 +106,11 @@ def load_recovered_authorization(path: str | Path) -> dict[str, Any]:
evidence = receipt["evidence_files"]
if not isinstance(evidence, list) or not evidence:
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_LIST_INVALID")
base = receipt_path.resolve().parent
for item in evidence:
if not isinstance(item, dict) or set(item) != {"path", "sha256"}:
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_ENTRY_INVALID")
target = Path(item["path"])
target = _contained_evidence(Path(item["path"]), base)
if _digest(target) != item["sha256"]:
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_DIGEST_MISMATCH")
return {
+23 -5
View File
@@ -110,6 +110,11 @@ SECRET_VAL_RE = re.compile(
r'(sk-[A-Za-z0-9_-]{20,}|HPUA[A-Z0-9]{10,}|SDC[A-Za-z0-9]{20,})'
)
# Control-plane file read: enforce auth, strict identifier, containment and a
# size cap on the command-detail read exposed by /admin/ai-evolution/api/data.
MAX_COMMAND_READ_BYTES = 1024 * 1024
_COMMAND_FILENAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*\.md$")
READ_ONLY_GET_PATHS = frozenset({
'/', '/health', '/api/health', '/api/openapi.yaml',
'/api/openapi-chatgpt-actions.yaml', '/api/schema', '/api', '/api/status',
@@ -1042,12 +1047,25 @@ return{vs,cmds,ar,cmd,cdet,tot,sok,sfail,runs,baseline,refreshMsg,loadCmd,selVer
commands.append(f.name)
command_detail=None
if cmd_filter:
actor = self.actor()
if not actor['ok']:
return self.sendj({'error':'unauthorized','hint':'Bearer token required'},401)
cmd = str(cmd_filter or '')
if not _COMMAND_FILENAME_RE.fullmatch(cmd) or '..' in cmd:
return self.sendj({'command':cmd_filter,'error':'unsafe_command'},400)
for d in cmd_dirs:
if d.exists():
fp=d/cmd_filter
if fp.exists():
command_detail={'command':cmd_filter,'content':fp.read_text(encoding='utf-8')}
break
if not d.exists():
continue
base = d.resolve()
fp = (base / cmd).resolve()
# Strict containment: must resolve directly inside one of the
# allowed command directories.
if fp.parent != base or not fp.is_file():
continue
if fp.stat().st_size > MAX_COMMAND_READ_BYTES:
return self.sendj({'command':cmd_filter,'error':'command_too_large'},413)
command_detail={'command':cmd_filter,'content':fp.read_text(encoding='utf-8')}
break
if not command_detail:
command_detail={'command':cmd_filter,'error':'not_found'}
return self.sendj(command_detail)
+1 -1
View File
@@ -157,7 +157,7 @@ def render_openapi_contract_fragment() -> str:
" operationId: commitFilesGoverned\n"
" summary: Submit one atomic governed commit containing multiple files\n"
" security: [{taskPrincipalAuth: []}]\n"
" x-aios-task-principal: Bridge verifies a fresh aios.task-principal-assertion.v1 and exact task/job/repository/branch/generation/revision/attempt/execution-digest/capability binding; caller JSON cannot provide a principal and missing verification is denied.\n"
" x-aios-task-principal: Bridge resolves a fresh server-side TaskPrincipal from the authoritative admitted+claimed Task and re-verifies its Owner epoch, then checks exact task/job/repository/branch/generation/capability binding; caller JSON cannot provide a principal and missing verification is denied. An explicitly injected legacy bearer verifier may still be used by a trusted host.\n"
" requestBody:\n"
" required: true\n"
" content:\n"
@@ -38,6 +38,7 @@ from aios_governed_task_bootstrap import (
GovernedTaskBootstrap,
)
from governed_production_release import build_executor_config
from aios_baseline.runtime_contract import evaluate_deployment
MAX_BODY = 2 * 1024 * 1024
TaskPrincipalVerifier = Callable[
@@ -109,6 +110,77 @@ def _safe(value: str) -> str:
return re.sub(r"[^A-Za-z0-9_.-]+", "-", value).strip("-")[:120] or "runtime"
def _unknown_production() -> dict[str, Any]:
return {
"baseline_version": None,
"target_oid": None,
"deployment_state": "UNKNOWN",
}
def load_production_observation(
path: Path,
now: datetime | str | None = None,
) -> dict[str, Any]:
"""Project production fields only from a fresh validated observation bundle."""
try:
metadata = path.lstat()
if (
stat.S_ISLNK(metadata.st_mode)
or not stat.S_ISREG(metadata.st_mode)
or metadata.st_mode & 0o022
or metadata.st_size < 2
or metadata.st_size > 1024 * 1024
):
return _unknown_production()
def strict_object(pairs: list[tuple[str, Any]]) -> dict[str, Any]:
result: dict[str, Any] = {}
for key, value in pairs:
if key in result:
raise ValueError("PRODUCTION_OBSERVATION_DUPLICATE_FIELD")
result[key] = value
return result
bundle = json.loads(path.read_text(encoding="utf-8"), object_pairs_hook=strict_object)
except (OSError, UnicodeError, ValueError, TypeError):
return _unknown_production()
if not isinstance(bundle, dict) or set(bundle) != {
"contract",
"target",
"observation",
"provenance",
}:
return _unknown_production()
try:
result = evaluate_deployment(
bundle["contract"],
bundle["target"],
bundle["observation"],
bundle["provenance"],
now or datetime.now(timezone.utc),
)
except (TypeError, ValueError):
return _unknown_production()
target = bundle["target"]
release_id = target.get("release_id") if isinstance(target, Mapping) else None
target_oid = target.get("target_oid") if isinstance(target, Mapping) else None
if (
result.get("status") != "VERIFIED"
or not isinstance(release_id, str)
or not release_id.strip()
or not isinstance(target_oid, str)
or re.fullmatch(r"[0-9a-f]{40}", target_oid) is None
):
return _unknown_production()
return {
"baseline_version": release_id,
"target_oid": target_oid,
"deployment_state": "ACTIVE",
}
class GovernedRuntime:
def __init__(self, config: RuntimeConfig) -> None:
self.config = config
@@ -919,7 +991,10 @@ class GovernedRuntime:
def snapshot(self) -> dict[str, Any]:
task = self._snapshot_task()
runtime = self.runtime_status()
return build_control_plane_snapshot(target_version=self.config.target_version, runtime=runtime, repository={"identity": self.config.repository, "authority_git_dir": str(self.config.authoritative_git_dir), "authority_main_oid": self.authority.read_ref("refs/heads/main")}, production={"baseline_version": "V2R1C7", "target_oid": "c7adece1cc8beffe7c718d53c775898b45d6c7f8", "deployment_state": "ACTIVE"}, task=task, capabilities={"legacy_writes_enabled": False, "governed_writes_enabled": runtime["governed_writes_enabled"], "force_push": False, "main_write": False, "tag_write": False, "baseline_write": False, "production_write": False}, credential_broker={"status": "SERVER_SIDE_ONLY", "provider": "FileCredentialProvider"})
production = load_production_observation(
self.config.state_root / "production-observation.json"
)
return build_control_plane_snapshot(target_version=self.config.target_version, runtime=runtime, repository={"identity": self.config.repository, "authority_git_dir": str(self.config.authoritative_git_dir), "authority_main_oid": self.authority.read_ref("refs/heads/main")}, production=production, task=task, capabilities={"legacy_writes_enabled": False, "governed_writes_enabled": runtime["governed_writes_enabled"], "force_push": False, "main_write": False, "tag_write": False, "baseline_write": False, "production_write": False}, credential_broker={"status": "SERVER_SIDE_ONLY", "provider": "FileCredentialProvider"})
def handlers(self) -> dict[str, Any]:
return {
@@ -1204,6 +1279,7 @@ def serve(adapter: BridgeV2Adapter, host: str = "127.0.0.1", port: int = 18892)
def main(
*,
task_admission_handlers: Mapping[str, Callable[[Mapping[str, Any]], Any]] | None = None,
task_principal_verifier: TaskPrincipalVerifier | None = None,
) -> None:
activation_raw = os.environ.get(
"AIOS_TASK_ADMISSION_ACTIVATION_FILE",
@@ -1258,27 +1334,33 @@ def main(
repository_discovery_cursor_secret=discovery_cursor_secret,
)
runtime = GovernedRuntime(config)
# A TaskPrincipal verifier is deliberately supplied only by the hosting
# service. Without that independently authenticated verifier, all Git
# mutation requests fail closed while the admission-only routes remain
# available for a configured Owner-approved epoch.
BridgeHandler.principal_verifier = None
# Prefer a server-side TaskPrincipal provider recovered from the same
# verified Task-admission runtime. This removes the caller-side credential
# bootstrap loop while keeping every mutation dependent on a currently
# claimed Task and a freshly re-verified Owner epoch. Hosts may still
# inject an independently authenticated verifier explicitly.
BridgeHandler.principal_verifier = task_principal_verifier
if task_admission_handlers is None:
admission_runtime = None
if os.environ.get("AIOS_TASK_ADMISSION_EPOCH_ROOT", "").strip():
from aios_task_admission_live_adapters import (
build_task_admission_handlers_from_environment,
build_task_admission_runtime_from_environment,
)
task_admission_handlers = build_task_admission_handlers_from_environment(
admission_runtime = build_task_admission_runtime_from_environment(
post_admission=runtime.project_admitted_task,
)
elif activation_raw and Path(activation_raw).is_file():
from aios_task_admission_live_adapters import (
build_task_admission_handlers_from_activation_file,
build_task_admission_runtime_from_activation_file,
)
task_admission_handlers = build_task_admission_handlers_from_activation_file(
admission_runtime = build_task_admission_runtime_from_activation_file(
Path(activation_raw),
post_admission=runtime.project_admitted_task,
)
if admission_runtime is not None:
task_admission_handlers = admission_runtime.handlers
if BridgeHandler.principal_verifier is None:
BridgeHandler.principal_verifier = admission_runtime.principal_verifier
serve(
build_bridge_adapter(
runtime,
@@ -24,6 +24,25 @@ def _safe_oid(value: str, field: str) -> str:
return value
def _safe_branch(value: str) -> str:
branch = str(value or "")
if (
not branch
or len(branch) > 240
or branch != branch.strip()
or branch.startswith(("/", "-", "."))
or branch.endswith(("/", ".", ".lock"))
or ".." in branch
or "//" in branch
or "@{" in branch
or "\\" in branch
or any(part in {"", ".", ".."} for part in branch.split("/"))
or any(ch not in "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789._/-" for ch in branch)
):
raise GiteaTransportError("BRANCH_INVALID")
return branch
@dataclass(slots=True)
class GiteaMachineTransport:
api_base: str
@@ -91,6 +110,7 @@ class GiteaMachineTransport:
return self.repository_info()
def read_branch(self, branch: str) -> str | None:
branch = _safe_branch(branch)
owner, name = self.repository_id.split("/", 1)
try:
value = self._request("GET", f"/repos/{quote(owner)}/{quote(name)}/branches/{quote(branch, safe='/')}")[1]
@@ -136,6 +156,99 @@ class GiteaMachineTransport:
finally:
askpass.unlink(missing_ok=True)
def fetch_branch_exact(self, branch: str, expected_oid: str) -> str:
"""Fetch one exact remote branch tip into a quarantine ref.
This never updates ``main`` or any caller-visible branch. The remote
ref is read before and after the fetch so a concurrent branch move
cannot be silently admitted.
"""
branch = _safe_branch(branch)
expected_oid = _safe_oid(expected_oid, "expected_oid")
remote_before = self.read_branch(branch)
if remote_before != expected_oid:
raise GiteaTransportError("GITEA_SOURCE_REF_MISMATCH")
target_ref = f"refs/aios/release-sources/{expected_oid}"
current = subprocess.run(
[
"git",
f"--git-dir={self.authority_git_dir}",
"rev-parse",
"--verify",
f"{target_ref}^{{commit}}",
],
text=True,
capture_output=True,
check=False,
timeout=self.timeout,
)
if current.returncode == 0 and current.stdout.strip().lower() == expected_oid:
remote_after = self.read_branch(branch)
if remote_after != expected_oid:
raise GiteaTransportError("GITEA_SOURCE_REF_MOVED")
return expected_oid
askpass, additions = self._askpass()
environment = os.environ.copy()
environment.update(additions)
try:
result = subprocess.run(
[
"git",
f"--git-dir={self.authority_git_dir}",
"fetch",
"--no-tags",
"--no-write-fetch-head",
self.remote_url,
f"refs/heads/{branch}:{target_ref}",
],
text=True,
capture_output=True,
check=False,
env=environment,
timeout=self.timeout,
)
if result.returncode != 0:
raise GiteaTransportError("GITEA_SOURCE_FETCH_FAILED")
finally:
askpass.unlink(missing_ok=True)
local = subprocess.run(
[
"git",
f"--git-dir={self.authority_git_dir}",
"rev-parse",
"--verify",
f"{target_ref}^{{commit}}",
],
text=True,
capture_output=True,
check=False,
timeout=self.timeout,
)
local_oid = local.stdout.strip().lower()
if local.returncode != 0 or local_oid != expected_oid:
subprocess.run(
[
"git",
f"--git-dir={self.authority_git_dir}",
"update-ref",
"-d",
target_ref,
],
text=True,
capture_output=True,
check=False,
timeout=self.timeout,
)
raise GiteaTransportError("GITEA_SOURCE_FETCH_MISMATCH")
remote_after = self.read_branch(branch)
if remote_after != expected_oid:
raise GiteaTransportError("GITEA_SOURCE_REF_MOVED")
return expected_oid
def compare_and_swap_branch(self, branch: str, expected_oid: str, new_oid: str) -> str:
_safe_oid(expected_oid, "expected_oid")
_safe_oid(new_oid, "new_oid")
+604 -22
View File
@@ -9,13 +9,14 @@ from __future__ import annotations
from dataclasses import dataclass
from enum import Enum
import base64
import hashlib
import json
import os
from pathlib import Path
import sqlite3
import tempfile
from datetime import datetime, timezone
from datetime import datetime, timedelta, timezone
from typing import Any, Callable, Mapping
@@ -61,18 +62,52 @@ _ORDER = {
JobState.COMPLETED: 8,
}
_RECOVERABLE = {JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}
_RECONCILABLE = _RECOVERABLE | {
JobState.RECONCILING,
JobState.EXECUTING,
JobState.LOCAL_VERIFIED,
JobState.REMOTE_SYNCING,
JobState.REMOTE_VERIFIED,
}
_TERMINAL = {
JobState.COMPLETED,
JobState.FAILED,
JobState.CANCELLED,
JobState.MANUAL_OWNER_REVIEW_REQUIRED,
}
_GENERIC_TRANSITIONS = {
JobState.REQUESTED: {JobState.AUTHORIZED, JobState.CANCELLED},
JobState.AUTHORIZED: {JobState.LEASED, JobState.CANCELLED},
JobState.LEASED: {JobState.WORKTREE_READY, JobState.EXECUTING, JobState.CANCELLED},
JobState.WORKTREE_READY: {JobState.EXECUTING, JobState.CANCELLED},
JobState.EXECUTING: {JobState.LOCAL_VERIFIED},
JobState.LOCAL_VERIFIED: {JobState.COMPLETED},
}
def utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _utc_datetime(value: str | datetime) -> datetime:
if isinstance(value, datetime):
parsed = value
elif isinstance(value, str):
try:
parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError as exc:
raise JobError("TIMESTAMP_INVALID") from exc
else:
raise JobError("TIMESTAMP_INVALID")
if parsed.tzinfo is None:
raise JobError("TIMESTAMP_TIMEZONE_REQUIRED")
return parsed.astimezone(timezone.utc)
def _iso_utc(value: datetime) -> str:
return value.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")
def canonical_json(value: Any) -> bytes:
return (
json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + "\n"
@@ -273,6 +308,27 @@ class JobStore:
kind TEXT NOT NULL, digest TEXT NOT NULL, payload_json TEXT NOT NULL,
created_at TEXT NOT NULL, FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE TABLE IF NOT EXISTS resource_leases (
lease_id TEXT PRIMARY KEY, job_id TEXT NOT NULL UNIQUE,
resource_kind TEXT NOT NULL, resource_id TEXT NOT NULL,
generation INTEGER NOT NULL, lease_expires_at TEXT NOT NULL,
state TEXT NOT NULL, revision INTEGER NOT NULL,
FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE TABLE IF NOT EXISTS resource_fences (
resource_kind TEXT NOT NULL, resource_id TEXT NOT NULL,
generation INTEGER NOT NULL, job_id TEXT NOT NULL,
updated_at TEXT NOT NULL,
PRIMARY KEY(resource_kind,resource_id),
FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE TABLE IF NOT EXISTS reconciliation_claims (
job_id TEXT PRIMARY KEY, claim_id TEXT NOT NULL UNIQUE,
reconcile_epoch INTEGER NOT NULL, base_revision INTEGER NOT NULL,
lease_expires_at TEXT NOT NULL, state TEXT NOT NULL,
revision INTEGER NOT NULL, updated_at TEXT NOT NULL,
FOREIGN KEY(job_id) REFERENCES jobs(job_id)
);
CREATE TABLE IF NOT EXISTS completed_replay_reconciliations (
reconciliation_id TEXT PRIMARY KEY, job_id TEXT NOT NULL,
identity_digest TEXT NOT NULL UNIQUE,
@@ -579,11 +635,79 @@ class JobStore:
class GitOpsJobController:
def __init__(self, store: JobStore) -> None:
def __init__(
self,
store: JobStore,
*,
clock: Callable[[], datetime] | None = None,
max_lease_seconds: int = 300,
reconcile_lease_seconds: int = 30,
generic_receipt_verifier: Callable[[Mapping[str, Any]], bool] | None = None,
) -> None:
if max_lease_seconds < 1 or reconcile_lease_seconds < 1:
raise JobError("LEASE_CONFIGURATION_INVALID")
self.store = store
self._clock = clock or (lambda: datetime.now(timezone.utc))
self.max_lease_seconds = max_lease_seconds
self.reconcile_lease_seconds = reconcile_lease_seconds
self.generic_receipt_verifier = generic_receipt_verifier
def _now(self) -> datetime:
return _utc_datetime(self._clock())
def _validated_deadline(self, value: str) -> str:
deadline = _utc_datetime(value)
now = self._now()
if deadline <= now:
raise JobError("LEASE_EXPIRED")
if deadline > now + timedelta(seconds=self.max_lease_seconds):
raise JobError("LEASE_DURATION_EXCEEDED")
return _iso_utc(deadline)
@staticmethod
def internal_job_id_for_idempotency_key(client_idempotency_key: str) -> str:
"""Return the durable Job ID for a client idempotency key."""
if (
not isinstance(client_idempotency_key, str)
or not client_idempotency_key
or len(client_idempotency_key) > 128
or not all(
character.isalnum() or character in "._:-"
for character in client_idempotency_key
)
):
raise JobError("IDEMPOTENCY_KEY_INVALID")
return "job-" + sha256_json({
"client_idempotency_key": client_idempotency_key,
})[:32]
@staticmethod
def _binding(request: Mapping[str, Any]) -> dict[str, str]:
if request.get("schema") == "aios.operation-job.v1":
fields = (
"schema", "tenant_id", "subject_id", "mission_id", "grant_id",
"task_id", "effect_id", "operation", "operation_version",
"resource_kind", "resource_id", "input_digest",
)
if any(not isinstance(request.get(field), str) or not request[field].strip() for field in fields):
raise JobError("OPERATION_JOB_BINDING_INVALID")
digest = str(request["input_digest"])
if len(digest) != 64 or any(ch not in "0123456789abcdef" for ch in digest):
raise JobError("INPUT_DIGEST_INVALID")
binding = {field: str(request[field]).strip() for field in fields}
expected = request.get("expected_resource_revision")
if expected is not None:
if isinstance(expected, bool) or not isinstance(expected, (str, int)) or not str(expected).strip():
raise JobError("EXPECTED_RESOURCE_REVISION_INVALID")
binding["expected_resource_revision"] = str(expected).strip()
idempotency_key = request.get("idempotency_key")
if idempotency_key is not None:
if not isinstance(idempotency_key, str) or not idempotency_key.strip():
raise JobError("IDEMPOTENCY_KEY_INVALID")
binding["idempotency_key"] = idempotency_key.strip()
if any(len(value) > 512 for value in binding.values()):
raise JobError("OPERATION_JOB_BINDING_INVALID")
return binding
fields = ("campaign_id", "repository", "operation", "branch", "expected_branch_oid", "payload_digest")
if any(not isinstance(request.get(field), str) or not request[field].strip() for field in fields):
raise JobError("JOB_BINDING_INVALID")
@@ -634,24 +758,38 @@ class GitOpsJobController:
def submit(self, request: Mapping[str, Any], *, payload: bytes | None = None) -> JobReceipt:
binding = self._binding(request)
bound_digest = binding.get("payload_digest", binding.get("input_digest"))
assert bound_digest is not None
payload_ref = request.get("payload_ref")
payload_size = request.get("payload_size")
if payload is not None:
payload_ref, payload_size = self.store.payloads.put(payload, binding["payload_digest"])
if payload_ref is not None and payload_ref != f"sha256/{binding['payload_digest']}":
payload_ref, payload_size = self.store.payloads.put(payload, bound_digest)
if payload_ref is not None and payload_ref != f"sha256/{bound_digest}":
raise JobError("PAYLOAD_REFERENCE_MISMATCH")
if payload_ref is not None and (not isinstance(payload_size, int) or payload_size < 0):
raise JobError("PAYLOAD_SIZE_INVALID")
client_idempotency_key = binding.get("idempotency_key")
if client_idempotency_key is not None:
if len(client_idempotency_key) > 128 or not all(
character.isalnum() or character in "._:-" for character in client_idempotency_key
):
raise JobError("IDEMPOTENCY_KEY_INVALID")
if client_idempotency_key is not None and (
len(client_idempotency_key) > 128
or not all(character.isalnum() or character in "._:-" for character in client_idempotency_key)
):
raise JobError("IDEMPOTENCY_KEY_INVALID")
if binding.get("schema") == "aios.operation-job.v1":
key = sha256_json({
"schema": binding["schema"],
"tenant_id": binding["tenant_id"],
"effect_id": binding["effect_id"],
})
elif client_idempotency_key is not None:
key = sha256_json({"client_idempotency_key": client_idempotency_key})
else:
key = sha256_json(binding)
job_id = "job-" + key[:32]
job_id = (
self.internal_job_id_for_idempotency_key(client_idempotency_key)
if binding.get("schema") != "aios.operation-job.v1"
and client_idempotency_key is not None
else "job-" + key[:32]
)
durable_request = dict(binding)
if payload_ref is not None:
durable_request.update({"payload_ref": str(payload_ref), "payload_size": int(payload_size)})
@@ -666,9 +804,15 @@ class GitOpsJobController:
row = dict(existing)
connection.commit()
return self.store._receipt(row, duplicate=True)
generic = binding.get("schema") == "aios.operation-job.v1"
storage_campaign = binding["mission_id"] if generic else binding["campaign_id"]
storage_repository = "__operation_job__" if generic else binding["repository"]
storage_branch = "__operation_job__" if generic else binding["branch"]
storage_oid = "0" * 40 if generic else binding["expected_branch_oid"]
storage_digest = binding["input_digest"] if generic else binding["payload_digest"]
connection.execute(
"INSERT INTO jobs(job_id,idempotency_key,campaign_id,repository,operation,branch,expected_branch_oid,payload_digest,payload_ref,payload_size,state,side_effect_state,revision,request_json,receipt_json,created_at,updated_at) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
(job_id, key, binding["campaign_id"], binding["repository"], binding["operation"], binding["branch"], binding["expected_branch_oid"], binding["payload_digest"], payload_ref, payload_size, JobState.REQUESTED.value, "NONE", 0, json.dumps({"binding": binding, **durable_request}, sort_keys=True), None, now, now),
(job_id, key, storage_campaign, storage_repository, binding["operation"], storage_branch, storage_oid, storage_digest, payload_ref, payload_size, JobState.REQUESTED.value, "NONE", 0, json.dumps({"binding": binding, **durable_request}, sort_keys=True), None, now, now),
)
connection.execute("INSERT INTO idempotency_keys(key,job_id,binding_json,created_at) VALUES(?,?,?,?)", (key, job_id, json.dumps(binding, sort_keys=True), now))
connection.execute("INSERT INTO state_transitions(job_id,from_state,to_state,revision,side_effect_state,created_at) VALUES(?,?,?,?,?,?)", (job_id, None, JobState.REQUESTED.value, 0, "NONE", now))
@@ -697,8 +841,12 @@ class GitOpsJobController:
expected_revision: int,
) -> JobReceipt:
"""Atomically create a branch lease and move its Job to LEASED."""
if generation < 1 or any(len(oid) != 40 for oid in (lineage_anchor_oid, current_head_oid)):
if isinstance(generation, bool) or generation < 1 or any(
len(oid) != 40 or any(ch not in "0123456789abcdef" for ch in oid)
for oid in (lineage_anchor_oid, current_head_oid)
):
raise JobError("LEASE_BINDING_INVALID")
deadline = self._validated_deadline(lease_expires_at)
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
@@ -707,10 +855,23 @@ class GitOpsJobController:
raise JobError("JOB_NOT_FOUND")
if int(row["revision"]) != expected_revision:
raise JobError("REVISION_CAS_CONFLICT")
if JobState(row["state"]) != JobState.AUTHORIZED:
raise JobError("LEASE_JOB_STATE_INVALID")
if row["branch"] in {"main", "master"}:
raise JobError("PROTECTED_BRANCH_REJECTED")
now = utc_now()
connection.execute("INSERT INTO branch_leases(lease_id,job_id,branch,lineage_anchor_oid,current_head_oid,generation,lease_expires_at,state,revision) VALUES(?,?,?,?,?,?,?,?,?)", (lease_id, job_id, row["branch"], lineage_anchor_oid, current_head_oid, generation, lease_expires_at, "ACTIVE", 0))
existing_lease = connection.execute(
"SELECT * FROM branch_leases WHERE job_id=?", (job_id,)
).fetchone()
if existing_lease is None:
connection.execute("INSERT INTO branch_leases(lease_id,job_id,branch,lineage_anchor_oid,current_head_oid,generation,lease_expires_at,state,revision) VALUES(?,?,?,?,?,?,?,?,?)", (lease_id, job_id, row["branch"], lineage_anchor_oid, current_head_oid, generation, deadline, "ACTIVE", 0))
elif existing_lease["state"] == "ACTIVE" and _utc_datetime(existing_lease["lease_expires_at"]) > self._now():
raise JobError("LEASE_CONFLICT")
else:
connection.execute(
"UPDATE branch_leases SET lease_id=?,lineage_anchor_oid=?,current_head_oid=?,generation=?,lease_expires_at=?,state='ACTIVE',revision=revision+1 WHERE job_id=?",
(lease_id, lineage_anchor_oid, current_head_oid, generation, deadline, job_id),
)
next_revision = expected_revision + 1
updated = connection.execute("UPDATE jobs SET state=?,side_effect_state=?,revision=?,updated_at=? WHERE job_id=? AND revision=?", (JobState.LEASED.value, "LEASED", next_revision, now, job_id, expected_revision))
if updated.rowcount != 1:
@@ -734,6 +895,13 @@ class GitOpsJobController:
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
lease = connection.execute("SELECT * FROM branch_leases WHERE lease_id=?", (lease_id,)).fetchone()
if lease is None:
raise JobError("LEASE_NOT_FOUND")
if lease["state"] != "ACTIVE" or _utc_datetime(lease["lease_expires_at"]) <= self._now():
raise JobError("LEASE_EXPIRED")
if int(lease["generation"]) != generation:
raise JobError("LEASE_FENCE_REJECTED")
updated = connection.execute("UPDATE branch_leases SET current_head_oid=?,generation=?,state=?,revision=revision+1 WHERE lease_id=? AND revision=?", (current_head_oid, generation, state, lease_id, expected_revision))
if updated.rowcount != 1:
raise JobError("LEASE_REVISION_CAS_CONFLICT")
@@ -743,6 +911,182 @@ class GitOpsJobController:
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
def lease_resource_job(
self,
job_id: str,
*,
lease_id: str,
resource_kind: str,
resource_id: str,
generation: int | None = None,
lease_expires_at: str,
expected_revision: int,
) -> JobReceipt:
"""Lease a scoped non-Git resource without pretending it is a branch."""
if not lease_id or not resource_kind or not resource_id or isinstance(generation, bool):
raise JobError("LEASE_BINDING_INVALID")
deadline = self._validated_deadline(lease_expires_at)
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
row = connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone()
if row is None:
raise JobError("JOB_NOT_FOUND")
request = json.loads(row["request_json"])
binding = request.get("binding", request)
if binding.get("schema") != "aios.operation-job.v1":
raise JobError("RESOURCE_LEASE_PROFILE_REQUIRED")
if binding.get("resource_kind") != resource_kind or binding.get("resource_id") != resource_id:
raise JobError("RESOURCE_LEASE_BINDING_MISMATCH")
if int(row["revision"]) != expected_revision:
raise JobError("REVISION_CAS_CONFLICT")
if JobState(row["state"]) != JobState.AUTHORIZED:
raise JobError("LEASE_JOB_STATE_INVALID")
active = connection.execute(
"SELECT 1 FROM resource_leases WHERE resource_kind=? AND resource_id=? AND state='ACTIVE' AND lease_expires_at>?",
(resource_kind, resource_id, _iso_utc(self._now())),
).fetchone()
if active is not None:
raise JobError("RESOURCE_LEASE_ACTIVE")
fence = connection.execute(
"SELECT generation FROM resource_fences WHERE resource_kind=? AND resource_id=?",
(resource_kind, resource_id),
).fetchone()
prior_generation = 0 if fence is None else int(fence["generation"])
assigned_generation = prior_generation + 1 if generation is None else generation
if assigned_generation <= prior_generation:
raise JobError("LEASE_FENCE_REJECTED")
old_job_lease = connection.execute(
"SELECT * FROM resource_leases WHERE job_id=?", (job_id,)
).fetchone()
if old_job_lease is None:
connection.execute(
"INSERT INTO resource_leases(lease_id,job_id,resource_kind,resource_id,generation,lease_expires_at,state,revision) VALUES(?,?,?,?,?,?,?,0)",
(lease_id, job_id, resource_kind, resource_id, assigned_generation, deadline, "ACTIVE"),
)
else:
connection.execute(
"UPDATE resource_leases SET lease_id=?,resource_kind=?,resource_id=?,generation=?,lease_expires_at=?,state='ACTIVE',revision=revision+1 WHERE job_id=?",
(lease_id, resource_kind, resource_id, assigned_generation, deadline, job_id),
)
connection.execute(
"INSERT INTO resource_fences(resource_kind,resource_id,generation,job_id,updated_at) VALUES(?,?,?,?,?) ON CONFLICT(resource_kind,resource_id) DO UPDATE SET generation=excluded.generation,job_id=excluded.job_id,updated_at=excluded.updated_at",
(resource_kind, resource_id, assigned_generation, job_id, utc_now()),
)
next_revision = expected_revision + 1
now = utc_now()
updated = connection.execute(
"UPDATE jobs SET state=?,side_effect_state=?,revision=?,updated_at=? WHERE job_id=? AND revision=?",
(JobState.LEASED.value, "LEASED", next_revision, now, job_id, expected_revision),
)
if updated.rowcount != 1:
raise JobError("REVISION_CAS_CONFLICT")
connection.execute(
"INSERT INTO state_transitions(job_id,from_state,to_state,revision,side_effect_state,created_at) VALUES(?,?,?,?,?,?)",
(job_id, row["state"], JobState.LEASED.value, next_revision, "LEASED", now),
)
result = dict(connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone())
connection.commit()
except sqlite3.IntegrityError as exc:
raise JobError("LEASE_CONFLICT") from exc
except JobError:
raise
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
result["request"] = json.loads(result["request_json"])
self.store._mirror(result)
return self.store._receipt(result, duplicate=True)
def assert_execution_lease(
self, job_id: str, *, generation: int, lease_id: str | None = None
) -> Mapping[str, Any]:
"""Fence a receiver-side effect by the active lease and generation."""
try:
with self.store._connect() as connection:
job = connection.execute("SELECT request_json,state FROM jobs WHERE job_id=?", (job_id,)).fetchone()
if job is None:
raise JobError("JOB_NOT_FOUND")
if JobState(job["state"]) not in {
JobState.LEASED, JobState.WORKTREE_READY, JobState.EXECUTING,
JobState.LOCAL_VERIFIED, JobState.REMOTE_SYNCING,
}:
raise JobError("LEASE_JOB_STATE_INVALID")
binding = json.loads(job["request_json"]).get("binding", {})
table = "resource_leases" if binding.get("schema") == "aios.operation-job.v1" else "branch_leases"
lease = connection.execute(f"SELECT * FROM {table} WHERE job_id=?", (job_id,)).fetchone()
if lease is None:
raise JobError("LEASE_NOT_FOUND")
if (
lease["state"] != "ACTIVE"
or _utc_datetime(lease["lease_expires_at"]) <= self._now()
or int(lease["generation"]) != generation
or (lease_id is not None and lease["lease_id"] != lease_id)
):
raise JobError("LEASE_FENCE_REJECTED")
if table == "resource_leases":
fence = connection.execute(
"SELECT generation,job_id FROM resource_fences WHERE resource_kind=? AND resource_id=?",
(lease["resource_kind"], lease["resource_id"]),
).fetchone()
if (
fence is None
or int(fence["generation"]) != int(lease["generation"])
or fence["job_id"] != job_id
):
raise JobError("LEASE_FENCE_REJECTED")
return dict(lease)
except JobError:
raise
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
def get_execution_lease(self, job_id: str) -> Mapping[str, Any]:
"""Return the current typed lease so callers can pass its fence token."""
try:
with self.store._connect() as connection:
request_row = connection.execute(
"SELECT request_json FROM jobs WHERE job_id=?", (job_id,)
).fetchone()
if request_row is None:
raise JobError("JOB_NOT_FOUND")
binding = json.loads(request_row["request_json"]).get("binding", {})
table = "resource_leases" if binding.get("schema") == "aios.operation-job.v1" else "branch_leases"
lease = connection.execute(f"SELECT * FROM {table} WHERE job_id=?", (job_id,)).fetchone()
if lease is None:
raise JobError("LEASE_NOT_FOUND")
return dict(lease)
except JobError:
raise
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
def assert_effect_admission(self, *, exclude_job_id: str | None = None) -> None:
"""Stop new effects while an earlier durable intent needs reconciliation."""
in_flight = (
JobState.EXECUTING, JobState.LOCAL_VERIFIED,
JobState.REMOTE_SYNCING, JobState.REMOTE_VERIFIED,
JobState.RECONCILING,
)
placeholders = ",".join("?" for _ in in_flight)
values: list[Any] = [state.value for state in in_flight]
query = (
f"SELECT job_id FROM jobs WHERE (state IN ({placeholders}) "
"OR (state=? AND side_effect_state='UNKNOWN'))"
)
values.append(JobState.RECONCILIATION_REQUIRED.value)
if exclude_job_id is not None:
query += " AND job_id<>?"
values.append(exclude_job_id)
query += " LIMIT 1"
try:
with self.store._connect() as connection:
if connection.execute(query, values).fetchone() is not None:
raise JobError("EFFECT_ADMISSION_LATCHED")
except JobError:
raise
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
def advance(self, job_id: str, state: JobState, *, side_effect_state: str, receipt: Mapping[str, Any] | None = None, expected_revision: int | None = None) -> JobReceipt:
return self._advance(
job_id,
@@ -752,6 +1096,75 @@ class GitOpsJobController:
expected_revision=expected_revision,
)
def _validate_generic_receipt(
self,
connection: sqlite3.Connection,
row: Mapping[str, Any],
receipt: Mapping[str, Any] | None,
*,
recovery: bool,
) -> None:
if self.generic_receipt_verifier is None:
raise JobError("GENERIC_RECEIPT_VERIFIER_REQUIRED")
if not isinstance(receipt, Mapping) or set(receipt) != {"schema", "key_id", "payload", "signature"}:
raise JobError("GENERIC_RECEIPT_INVALID")
if receipt.get("schema") != "aios.operation-receipt.v1":
raise JobError("GENERIC_RECEIPT_INVALID")
if not isinstance(receipt.get("key_id"), str) or not receipt["key_id"].strip():
raise JobError("GENERIC_RECEIPT_INVALID")
signature = receipt.get("signature")
try:
decoded_signature = base64.b64decode(signature, validate=True) if isinstance(signature, str) else b""
except (ValueError, TypeError):
decoded_signature = b""
if len(decoded_signature) != 64:
raise JobError("GENERIC_RECEIPT_INVALID")
payload = receipt.get("payload")
if not isinstance(payload, Mapping) or not isinstance(payload.get("observation"), Mapping):
raise JobError("GENERIC_RECEIPT_INVALID")
request = json.loads(row["request_json"])
binding = request.get("binding", request)
required_bindings = (
"tenant_id", "subject_id", "mission_id", "grant_id", "task_id",
"effect_id", "operation", "operation_version", "resource_kind",
"resource_id", "input_digest",
)
if any(payload.get(field) != binding.get(field) for field in required_bindings):
raise JobError("GENERIC_RECEIPT_BINDING_MISMATCH")
if payload.get("job_id") != row["job_id"]:
raise JobError("GENERIC_RECEIPT_BINDING_MISMATCH")
if "expected_resource_revision" in binding and payload.get("expected_resource_revision") != binding["expected_resource_revision"]:
raise JobError("GENERIC_RECEIPT_BINDING_MISMATCH")
lease = connection.execute(
"SELECT * FROM resource_leases WHERE job_id=?", (row["job_id"],)
).fetchone()
if (
lease is None
or payload.get("claim_id") != lease["lease_id"]
or isinstance(payload.get("generation"), bool)
or payload.get("generation") != int(lease["generation"])
):
raise JobError("GENERIC_RECEIPT_LEASE_MISMATCH")
if not recovery:
fence = connection.execute(
"SELECT generation,job_id FROM resource_fences WHERE resource_kind=? AND resource_id=?",
(lease["resource_kind"], lease["resource_id"]),
).fetchone()
if (
lease["state"] != "ACTIVE"
or _utc_datetime(lease["lease_expires_at"]) <= self._now()
or fence is None
or int(fence["generation"]) != int(lease["generation"])
or fence["job_id"] != row["job_id"]
):
raise JobError("GENERIC_RECEIPT_LEASE_MISMATCH")
try:
verified = self.generic_receipt_verifier(receipt)
except Exception as exc:
raise JobError("GENERIC_RECEIPT_SIGNATURE_INVALID") from exc
if verified is not True:
raise JobError("GENERIC_RECEIPT_SIGNATURE_INVALID")
def _advance(
self,
job_id: str,
@@ -779,7 +1192,17 @@ class GitOpsJobController:
)
if previous in _TERMINAL and state != previous and not recovery_transition:
raise JobError("JOB_TERMINAL")
if state not in _RECOVERABLE | {JobState.RECONCILING, JobState.MANUAL_OWNER_REVIEW_REQUIRED}:
durable_request = json.loads(row["request_json"])
generic = durable_request.get("binding", durable_request).get("schema") == "aios.operation-job.v1"
if generic:
allowed = _GENERIC_TRANSITIONS.get(previous, set()) | _RECOVERABLE | {
JobState.RECONCILIATION_REQUIRED, JobState.MANUAL_OWNER_REVIEW_REQUIRED,
}
if state not in allowed:
raise JobError("GENERIC_JOB_TRANSITION_INVALID")
if state in {JobState.LOCAL_VERIFIED, JobState.COMPLETED}:
self._validate_generic_receipt(connection, row, receipt, recovery=False)
if not generic and state not in _RECOVERABLE | {JobState.RECONCILING, JobState.MANUAL_OWNER_REVIEW_REQUIRED}:
if previous in _RECOVERABLE:
raise JobError("RECONCILE_REQUIRED")
if previous != JobState.RECONCILING and _ORDER.get(state, -1) < _ORDER.get(previous, -1):
@@ -790,6 +1213,9 @@ class GitOpsJobController:
updated = connection.execute("UPDATE jobs SET state=?,side_effect_state=?,revision=?,receipt_json=?,updated_at=? WHERE job_id=? AND revision=?", (state.value, side_effect_state, next_revision, receipt_json, now, job_id, revision))
if updated.rowcount != 1:
raise JobError("REVISION_CAS_CONFLICT")
if state in _TERMINAL:
connection.execute("UPDATE branch_leases SET state='USED' WHERE job_id=? AND state='ACTIVE'", (job_id,))
connection.execute("UPDATE resource_leases SET state='USED' WHERE job_id=? AND state='ACTIVE'", (job_id,))
connection.execute("INSERT INTO state_transitions(job_id,from_state,to_state,revision,side_effect_state,created_at) VALUES(?,?,?,?,?,?)", (job_id, previous.value, state.value, next_revision, side_effect_state, now))
if receipt is not None:
digest = hashlib.sha256(canonical_json(receipt)).hexdigest()
@@ -897,19 +1323,175 @@ class GitOpsJobController:
_allow_verified_external_recovery=True,
)
def reconcile(self, job_id: str, observer: Callable[[Mapping[str, Any]], Mapping[str, Any]]) -> JobReceipt:
current = self.store.read(job_id)
if JobState(current["state"]) not in _RECOVERABLE:
return self.store._receipt(current, duplicate=True)
self.advance(job_id, JobState.RECONCILING, side_effect_state="IN_PROGRESS")
def _claim_reconciliation(self, job_id: str) -> tuple[dict[str, Any], dict[str, Any]] | None:
now = self._now()
deadline = _iso_utc(now + timedelta(seconds=self.reconcile_lease_seconds))
try:
observed = dict(observer(current))
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
row = connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone()
if row is None:
raise JobError("JOB_NOT_FOUND")
state = JobState(row["state"])
if state not in _RECONCILABLE:
connection.commit()
return None
existing = connection.execute(
"SELECT * FROM reconciliation_claims WHERE job_id=?", (job_id,)
).fetchone()
if (
state == JobState.RECONCILING
and existing is not None
and existing["state"] == "ACTIVE"
and _utc_datetime(existing["lease_expires_at"]) > now
):
raise JobError("RECONCILIATION_CLAIM_ACTIVE")
epoch = 1 if existing is None else int(existing["reconcile_epoch"]) + 1
claim_id = f"reconcile-{job_id}-{epoch}"
revision = int(row["revision"])
claimed_revision = revision + 1
updated = connection.execute(
"UPDATE jobs SET state=?,side_effect_state=?,revision=?,updated_at=? WHERE job_id=? AND revision=?",
(JobState.RECONCILING.value, "IN_PROGRESS", claimed_revision, utc_now(), job_id, revision),
)
if updated.rowcount != 1:
raise JobError("REVISION_CAS_CONFLICT")
connection.execute(
"INSERT INTO state_transitions(job_id,from_state,to_state,revision,side_effect_state,created_at) VALUES(?,?,?,?,?,?)",
(job_id, state.value, JobState.RECONCILING.value, claimed_revision, "IN_PROGRESS", utc_now()),
)
if existing is None:
connection.execute(
"INSERT INTO reconciliation_claims(job_id,claim_id,reconcile_epoch,base_revision,lease_expires_at,state,revision,updated_at) VALUES(?,?,?,?,?,'ACTIVE',0,?)",
(job_id, claim_id, epoch, claimed_revision, deadline, utc_now()),
)
else:
connection.execute(
"UPDATE reconciliation_claims SET claim_id=?,reconcile_epoch=?,base_revision=?,lease_expires_at=?,state='ACTIVE',revision=revision+1,updated_at=? WHERE job_id=?",
(claim_id, epoch, claimed_revision, deadline, utc_now(), job_id),
)
current = dict(connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone())
connection.commit()
except JobError:
raise
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
claim = {
"claim_id": claim_id,
"reconcile_epoch": epoch,
"base_revision": claimed_revision,
"effect_id": json.loads(current["request_json"]).get("binding", {}).get("effect_id", job_id),
"observed_at": _iso_utc(now),
"lease_expires_at": deadline,
}
return current, claim
def _finish_reconciliation(
self,
job_id: str,
claim: Mapping[str, Any],
state: JobState,
*,
side_effect_state: str,
receipt: Mapping[str, Any] | None,
) -> JobReceipt:
try:
with self.store._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
current_claim = connection.execute(
"SELECT * FROM reconciliation_claims WHERE job_id=?", (job_id,)
).fetchone()
row = connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone()
if row is None:
raise JobError("JOB_NOT_FOUND")
if (
current_claim is None
or current_claim["state"] != "ACTIVE"
or current_claim["claim_id"] != claim["claim_id"]
or int(current_claim["reconcile_epoch"]) != int(claim["reconcile_epoch"])
or int(current_claim["base_revision"]) != int(row["revision"])
or _utc_datetime(current_claim["lease_expires_at"]) <= self._now()
):
raise JobError("RECONCILIATION_CLAIM_LOST")
durable_request = json.loads(row["request_json"])
generic = durable_request.get("binding", durable_request).get("schema") == "aios.operation-job.v1"
if generic and state == JobState.COMPLETED:
self._validate_generic_receipt(connection, row, receipt, recovery=True)
revision = int(row["revision"])
next_revision = revision + 1
receipt_payload = None
if receipt is not None:
receipt_payload = dict(receipt)
receipt_payload["reconciliation_claim"] = dict(claim)
receipt_json = json.dumps(receipt_payload, sort_keys=True) if receipt_payload is not None else row["receipt_json"]
now = utc_now()
updated = connection.execute(
"UPDATE jobs SET state=?,side_effect_state=?,revision=?,receipt_json=?,updated_at=? WHERE job_id=? AND revision=?",
(state.value, side_effect_state, next_revision, receipt_json, now, job_id, revision),
)
if updated.rowcount != 1:
raise JobError("RECONCILIATION_CLAIM_LOST")
connection.execute("UPDATE branch_leases SET state='RECONCILED' WHERE job_id=? AND state='ACTIVE'", (job_id,))
connection.execute("UPDATE resource_leases SET state='RECONCILED' WHERE job_id=? AND state='ACTIVE'", (job_id,))
connection.execute(
"UPDATE reconciliation_claims SET state='COMPLETED',revision=revision+1,updated_at=? WHERE job_id=? AND claim_id=?",
(now, job_id, claim["claim_id"]),
)
connection.execute(
"INSERT INTO state_transitions(job_id,from_state,to_state,revision,side_effect_state,created_at) VALUES(?,?,?,?,?,?)",
(job_id, row["state"], state.value, next_revision, side_effect_state, now),
)
if receipt_payload is not None:
digest = hashlib.sha256(canonical_json(receipt_payload)).hexdigest()
connection.execute(
"INSERT INTO receipts(receipt_id,job_id,kind,digest,payload_json,created_at) VALUES(?,?,?,?,?,?)",
(f"receipt-{job_id}-{next_revision}", job_id, f"RECONCILE:{state.value}", digest, json.dumps(receipt_payload, sort_keys=True), now),
)
result = dict(connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone())
connection.commit()
except JobError:
raise
except sqlite3.Error as exc:
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
result["request"] = json.loads(result["request_json"])
self.store._mirror(result)
return self.store._receipt(result, duplicate=True)
def reconcile(self, job_id: str, observer: Callable[[Mapping[str, Any]], Mapping[str, Any]]) -> JobReceipt:
claimed = self._claim_reconciliation(job_id)
if claimed is None:
current = self.store.read(job_id)
return self.store._receipt(current, duplicate=True)
current, claim = claimed
observed_input = dict(current)
observed_input["reconciliation_claim"] = dict(claim)
try:
observed = dict(observer(observed_input))
stable = JobState(observed.get("state", JobState.REQUESTED.value))
if stable in {JobState.RECOVERABLE_FAILURE, JobState.RECONCILING}:
raise JobError("RECONCILIATION_UNSTABLE")
return self.advance(job_id, stable, side_effect_state=str(observed.get("side_effect_state", "NONE")), receipt=observed.get("receipt"))
return self._finish_reconciliation(
job_id,
claim,
stable,
side_effect_state=str(observed.get("side_effect_state", "NONE")),
receipt=observed.get("receipt"),
)
except Exception as exc:
self.advance(job_id, JobState.RECONCILIATION_REQUIRED, side_effect_state="UNKNOWN", receipt={"error_code": type(exc).__name__})
if isinstance(exc, JobError) and str(exc) == "RECONCILIATION_CLAIM_LOST":
raise
try:
self._finish_reconciliation(
job_id,
claim,
JobState.RECONCILIATION_REQUIRED,
side_effect_state="UNKNOWN",
receipt={"error_code": type(exc).__name__},
)
except JobError as finish_exc:
if str(finish_exc) == "RECONCILIATION_CLAIM_LOST":
raise finish_exc from exc
raise
raise JobError("RECONCILIATION_REQUIRED") from exc
def resume(self, job_id: str) -> JobReceipt:
+363 -124
View File
@@ -4,15 +4,16 @@ from __future__ import annotations
import argparse
import base64
from dataclasses import dataclass, replace
from dataclasses import dataclass, field, replace
from datetime import datetime, timedelta, timezone
import json
from pathlib import Path
from typing import Any, Mapping
from typing import Any, Callable, Mapping
from aios_authority_git_executor import AuthorityGitExecutor, AuthorityGitError
from aios_campaign_recovery import RecoveryEvidenceError, load_recovered_authorization
from aios_gitea_machine_broker import CampaignCapabilityReceipt, GiteaMachineBroker, MachineBrokerError
from aios_gitops_job_controller import GitOpsJobController, JobError, JobState, JobStore
from aios_gitops_job_controller import GitOpsJobController, JobError, JobState, JobStore, ServiceUnavailable
TASK_ID = "AIOS_AUTONOMOUS_RELEASE_CAMPAIGN_V1"
@@ -34,6 +35,17 @@ class DurableJobExecutor:
authority: AuthorityGitExecutor
broker: GiteaMachineBroker
recovery_path: Path
clock: Callable[[], datetime] = field(default=lambda: datetime.now(timezone.utc))
lease_seconds: int = 60
_persistence_halted: bool = field(default=False, init=False, repr=False)
def __post_init__(self) -> None:
try:
self.controller.assert_effect_admission()
except JobError as exc:
if str(exc) != "EFFECT_ADMISSION_LATCHED":
raise
self._persistence_halted = True
def _request(self, job_id: str) -> tuple[dict[str, Any], dict[str, Any]]:
row = self.controller.store.read(job_id)
@@ -41,7 +53,14 @@ class DurableJobExecutor:
request = json.loads(row["request_json"])
except (KeyError, TypeError, json.JSONDecodeError) as exc:
raise JobExecutionError("JOB_REQUEST_INVALID") from exc
return row, dict(request.get("binding") or {})
binding = dict(request.get("binding") or {})
# Payload location is derived from and protected by payload_digest, but
# is intentionally outside the idempotency binding. Execution still
# needs it after a durable round trip.
for field_name in ("payload_ref", "payload_size"):
if field_name in request:
binding[field_name] = request[field_name]
return row, binding
def _validate_campaign(self, request: Mapping[str, Any]) -> str:
operation = str(request.get("operation", ""))
@@ -89,151 +108,372 @@ class DurableJobExecutor:
raise JobExecutionError("NEW_OID_NOT_DESCENDANT")
if not self.authority.ancestry_check(EXPECTED_BRANCH_OID, new_oid):
raise JobExecutionError("NEW_OID_OUTSIDE_AUTHORIZED_LINEAGE")
if operation == "RECONCILE_AUTHORITY_MAIN":
old_main = request.get("expected_main_oid")
if (
not isinstance(old_main, str)
or len(old_main) != 40
or any(ch not in "0123456789abcdef" for ch in old_main)
):
raise JobExecutionError("MAIN_RECONCILIATION_BINDING_INVALID")
if not self.authority.ancestry_check(old_main, new_oid):
raise JobExecutionError("MAIN_OID_NOT_DESCENDANT")
identity = self.broker.verify_identity()
if identity.get("repository") != REPOSITORY or identity.get("login") == "aiman":
raise JobExecutionError("MACHINE_IDENTITY_SCOPE_INVALID")
return new_oid
def dry_run(self, job_id: str) -> dict[str, Any]:
_row, request = self._request(job_id)
new_oid = self._validate_campaign(request)
remote = self.broker.readback()
if remote is None:
action = "CREATE_REQUIRED"
elif remote == new_oid:
action = "ALREADY_IN_SYNC"
elif remote == request["expected_branch_oid"]:
action = "FAST_FORWARD_REQUIRED"
else:
action = "MANUAL_OWNER_REVIEW_REQUIRED"
if action == "MANUAL_OWNER_REVIEW_REQUIRED":
raise JobExecutionError("GITEA_BRANCH_CONFLICT")
return {
row, request = self._request(job_id)
operation = str(request.get("operation") or "")
if operation not in {
"ADVANCE_EXISTING_CAMPAIGN_BRANCH", "COMMIT_FILE",
"SYNC_TASK_BRANCH_TO_GITEA", "CREATE_PR", "PUBLISH_CHECK",
"PROTECTED_MERGE", "RECONCILE_AUTHORITY_MAIN",
}:
raise JobExecutionError("OPERATION_NOT_ALLOWED")
self._validate_campaign(request)
if operation == "COMMIT_FILE":
self._commit_file_spec(row, request)
observation = self._observe_operation(request)
result = {
"mode": "DRY_RUN",
"operation": operation,
"decision": "SAFE_TO_ADVANCE",
"authority_action": "CAS_REQUIRED",
"gitea_action": action,
"previous_branch_oid": request["expected_branch_oid"],
"final_branch_oid": new_oid,
"force_push": False,
"effect_performed": False,
"readback": observation,
}
def _recover_requested(self, job_id: str) -> dict[str, Any]:
row = self.controller.store.read(job_id)
if JobState(row["state"]) in {JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}:
self.controller.resume(job_id)
row = self.controller.store.read(job_id)
return row
if operation == "ADVANCE_EXISTING_CAMPAIGN_BRANCH":
result.update({
"authority_action": "CAS_REQUIRED",
"gitea_action": "ALREADY_IN_SYNC" if observation["verification"] == "VERIFIED" else "FAST_FORWARD_REQUIRED",
"previous_branch_oid": request["expected_branch_oid"],
"final_branch_oid": request["new_branch_oid"],
"force_push": False,
})
return result
def execute(self, job_id: str, *, dry_run: bool = False) -> dict[str, Any]:
if dry_run:
return self.dry_run(job_id)
if self._persistence_halted:
raise JobExecutionError("PERSISTENCE_FAILURE_LATCHED")
try:
return self._execute_unprotected(job_id, dry_run=dry_run)
return self._execute_unprotected(job_id)
except ServiceUnavailable:
self._persistence_halted = True
raise
except Exception as exc:
try:
row = self.controller.store.read(job_id)
state = JobState(row["state"])
if state not in {JobState.COMPLETED, JobState.MANUAL_OWNER_REVIEW_REQUIRED, JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}:
self.controller.advance(job_id, JobState.RECOVERABLE_FAILURE, side_effect_state="UNKNOWN", receipt={"error_code": type(exc).__name__})
except Exception:
pass
if state in {
JobState.EXECUTING, JobState.LOCAL_VERIFIED,
JobState.REMOTE_SYNCING, JobState.REMOTE_VERIFIED,
}:
self.controller.advance(
job_id,
JobState.RECONCILIATION_REQUIRED,
side_effect_state="UNKNOWN",
receipt={"error_code": type(exc).__name__, "verification": "UNKNOWN"},
)
elif state not in {
JobState.COMPLETED, JobState.MANUAL_OWNER_REVIEW_REQUIRED,
JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED,
}:
self.controller.advance(
job_id, JobState.RECOVERABLE_FAILURE,
side_effect_state="NONE",
receipt={"error_code": type(exc).__name__, "effect_performed": False},
)
except ServiceUnavailable:
self._persistence_halted = True
raise
def _execute_unprotected(self, job_id: str, *, dry_run: bool = False) -> dict[str, Any]:
def _execute_unprotected(self, job_id: str) -> dict[str, Any]:
row, request = self._request(job_id)
operation = request.get("operation")
if operation == "ADVANCE_EXISTING_CAMPAIGN_BRANCH":
if dry_run:
return self.dry_run(job_id)
return self._advance_campaign(job_id)
if operation == "COMMIT_FILE":
return self._commit_file(job_id, request)
if operation == "SYNC_TASK_BRANCH_TO_GITEA":
new_oid = self._validate_campaign(request)
final = self.broker.push_branch(new_oid)
return {"operation": operation, "remote_verified": final == new_oid, "final_network_oid": final}
if operation == "CREATE_PR":
self._validate_campaign(request)
return dict(self.broker.create_pr("AIOS autonomous release control plane", "Durable campaign continuation; exact head and recovery evidence are bound in the Job."))
if operation == "PUBLISH_CHECK":
self._validate_campaign(request)
context = request.get("check_context", "aios/durable-release")
state = request.get("check_state", "success")
description = request.get("check_description", "Durable campaign control-plane check passed")
return dict(self.broker.status(str(state), str(description), str(context)))
if operation == "PROTECTED_MERGE":
self._validate_campaign(request)
number = int(request.get("pr_number", "0"))
self.broker.transport.expected_merge_head = request["expected_branch_oid"]
expected_main = request.get("expected_main_oid")
if expected_main is not None:
self.broker.transport.expected_merge_base = expected_main
return dict(self.broker.merge(number))
if operation == "RECONCILE_AUTHORITY_MAIN":
self._validate_campaign(request)
old = request.get("expected_main_oid")
new = request.get("new_branch_oid")
if not isinstance(old, str) or not isinstance(new, str):
raise JobExecutionError("MAIN_RECONCILIATION_BINDING_INVALID")
gitea_main = self.broker.transport.read_branch("main")
if gitea_main != new:
raise JobExecutionError("GITEA_MAIN_EXPECTATION_MISMATCH")
if not self.authority.ancestry_check(old, new):
raise JobExecutionError("MAIN_OID_NOT_DESCENDANT")
return {"authority_main": self.authority.ff_CAS_main_after_verified_protected_merge(old, new)}
raise JobExecutionError("OPERATION_NOT_ALLOWED")
state = JobState(row["state"])
if state == JobState.COMPLETED:
return self.controller.store._receipt(row, duplicate=True).as_dict()
if state in {
JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED,
JobState.RECONCILING, JobState.EXECUTING, JobState.LOCAL_VERIFIED,
JobState.REMOTE_SYNCING, JobState.REMOTE_VERIFIED,
}:
return self.reconcile(job_id)
def _advance_campaign(self, job_id: str) -> dict[str, Any]:
self._recover_requested(job_id)
row, request = self._request(job_id)
new_oid = self._validate_campaign(request)
expected_oid = request["expected_branch_oid"]
operation = str(request.get("operation") or "")
if operation not in {
"ADVANCE_EXISTING_CAMPAIGN_BRANCH", "COMMIT_FILE",
"SYNC_TASK_BRANCH_TO_GITEA", "CREATE_PR", "PUBLISH_CHECK",
"PROTECTED_MERGE", "RECONCILE_AUTHORITY_MAIN",
}:
raise JobExecutionError("OPERATION_NOT_ALLOWED")
try:
self.controller.assert_effect_admission(exclude_job_id=job_id)
except JobError as exc:
if str(exc) == "EFFECT_ADMISSION_LATCHED":
self._persistence_halted = True
raise JobExecutionError("PERSISTENCE_FAILURE_LATCHED") from exc
raise
self._validate_campaign(request)
if operation == "COMMIT_FILE":
self._commit_file_spec(row, request)
row = self._prepare_intent(job_id, row, request)
generation = int(request.get("generation", "1"))
self.controller.assert_execution_lease(job_id, generation=generation)
before = self._observe_operation(request)
if before["verification"] == "VERIFIED":
return self._complete(job_id, request, {"reconciled_before_effect": True}, before)
if before["verification"] != "ABSENT":
return self._record_unknown(job_id, request, before)
effect = self._perform_effect(job_id, row, request, generation)
after = self._observe_operation(request)
if after["verification"] != "VERIFIED":
return self._record_unknown(job_id, request, after, effect=effect)
return self._complete(job_id, request, effect, after)
def _prepare_intent(
self, job_id: str, row: Mapping[str, Any], request: Mapping[str, Any]
) -> dict[str, Any]:
state = JobState(row["state"])
if state == JobState.REQUESTED:
receipt = self.controller.advance(job_id, JobState.AUTHORIZED, side_effect_state="AUTHORIZED", receipt={"authorization_digest": AUTHORIZATION_DIGEST, "task_id": TASK_ID})
self.controller.advance(
job_id, JobState.AUTHORIZED, side_effect_state="AUTHORIZED",
receipt={"authorization_digest": AUTHORIZATION_DIGEST, "task_id": TASK_ID},
expected_revision=int(row["revision"]),
)
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.AUTHORIZED:
self.controller.lease_job(job_id, lease_id=f"lease-{job_id}", lineage_anchor_oid=LINEAGE_ANCHOR, current_head_oid=expected_oid, generation=1, lease_expires_at="2099-01-01T00:00:00Z", expected_revision=int(row["revision"]))
state = JobState(row["state"])
if state == JobState.AUTHORIZED:
deadline = self.clock().astimezone(timezone.utc) + timedelta(seconds=self.lease_seconds)
self.controller.lease_job(
job_id,
lease_id=f"lease-{job_id}",
lineage_anchor_oid=LINEAGE_ANCHOR,
current_head_oid=str(request["expected_branch_oid"]),
generation=int(request.get("generation", "1")),
lease_expires_at=deadline.isoformat(),
expected_revision=int(row["revision"]),
)
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.LEASED:
self.controller.advance(job_id, JobState.WORKTREE_READY, side_effect_state="WORKTREE_READY", receipt={"branch": BRANCH, "expected_oid": expected_oid})
state = JobState(row["state"])
if state == JobState.LEASED:
self.controller.advance(
job_id, JobState.WORKTREE_READY, side_effect_state="WORKTREE_READY",
expected_revision=int(row["revision"]),
)
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.WORKTREE_READY:
self.controller.advance(job_id, JobState.EXECUTING, side_effect_state="IN_PROGRESS")
state = JobState(row["state"])
if state == JobState.WORKTREE_READY:
self.controller.advance(
job_id, JobState.EXECUTING, side_effect_state="INTENT_COMMITTED",
receipt={
"effect_id": job_id,
"operation": request["operation"],
"input_digest": row["payload_digest"],
"intent_committed": True,
},
expected_revision=int(row["revision"]),
)
row = self.controller.store.read(job_id)
authority_before = self.authority.read_ref(f"refs/heads/{BRANCH}")
if authority_before == expected_oid:
authority_oid = self.authority.advance_task_branch_CAS(f"refs/heads/{BRANCH}", expected_oid, new_oid)
elif authority_before == new_oid:
authority_oid = new_oid
else:
raise JobExecutionError("AUTHORITY_BRANCH_CONFLICT")
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.EXECUTING:
self.controller.advance(job_id, JobState.LOCAL_VERIFIED, side_effect_state="LOCAL_VERIFIED", receipt={"authority": {"branch": BRANCH, "previous_oid": expected_oid, "new_oid": new_oid, "CAS_success": True, "readback_oid": authority_oid}})
row = self.controller.store.read(job_id)
remote_before = self.broker.readback()
if remote_before != new_oid:
if remote_before not in {None, expected_oid}:
raise JobExecutionError("GITEA_BRANCH_CONFLICT")
if JobState(row["state"]) == JobState.LOCAL_VERIFIED:
if hasattr(self.broker, "receipt"):
self.broker.receipt = replace(self.broker.receipt, expected_oid=expected_oid)
self.controller.advance(job_id, JobState.REMOTE_SYNCING, side_effect_state="IN_PROGRESS", receipt={"gitea": {"previous_oid": remote_before, "desired_oid": new_oid, "force_push": False}})
final_network = self.broker.push_branch(new_oid)
else:
final_network = new_oid
final_api = self.broker.readback()
if final_api != new_oid:
raise JobExecutionError("GITEA_BRANCH_READBACK_MISMATCH")
row = self.controller.store.read(job_id)
if JobState(row["state"]) in {JobState.LOCAL_VERIFIED, JobState.REMOTE_SYNCING}:
self.controller.advance(job_id, JobState.REMOTE_VERIFIED, side_effect_state="REMOTE_VERIFIED", receipt={"gitea": {"machine_actor": "aios-autonomy-agent", "previous_oid": remote_before, "desired_oid": new_oid, "final_network_oid": final_network, "final_API_oid": final_api, "force_push": False}})
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.REMOTE_VERIFIED:
self.controller.advance(job_id, JobState.COMPLETED, side_effect_state="REMOTE_VERIFIED", receipt={"authority_oid": authority_oid, "network_oid": final_network, "gitea_api_oid": final_api, "force_push": False})
return self.controller.store._receipt(self.controller.store.read(job_id), duplicate=True).as_dict()
if JobState(row["state"]) != JobState.EXECUTING:
raise JobExecutionError("JOB_EXECUTION_STATE_INVALID")
return dict(row)
def _commit_file(self, job_id: str, request: Mapping[str, Any]) -> dict[str, Any]:
def _perform_effect(
self,
job_id: str,
row: Mapping[str, Any],
request: Mapping[str, Any],
generation: int,
) -> dict[str, Any]:
operation = str(request["operation"])
new_oid = str(request["new_branch_oid"])
self.controller.assert_execution_lease(job_id, generation=generation)
if operation == "ADVANCE_EXISTING_CAMPAIGN_BRANCH":
expected = str(request["expected_branch_oid"])
current = self.authority.read_ref(f"refs/heads/{BRANCH}")
authority_oid = new_oid if current == new_oid else self.authority.advance_task_branch_CAS(
f"refs/heads/{BRANCH}", expected, new_oid
)
self.controller.assert_execution_lease(job_id, generation=generation)
remote = self.broker.readback()
if remote != new_oid:
if hasattr(self.broker, "receipt"):
self.broker.receipt = replace(self.broker.receipt, expected_oid=expected)
self.broker.push_branch(new_oid)
return {"authority_oid": authority_oid, "network_oid": new_oid, "force_push": False}
if operation == "SYNC_TASK_BRANCH_TO_GITEA":
return {"final_network_oid": self.broker.push_branch(new_oid), "force_push": False}
if operation == "CREATE_PR":
return dict(self.broker.create_pr(
"AIOS autonomous release control plane",
"Durable campaign continuation; exact head and recovery evidence are bound in the Job.",
))
if operation == "PUBLISH_CHECK":
return dict(self.broker.status(
str(request.get("check_state", "success")),
str(request.get("check_description", "Durable campaign control-plane check passed")),
str(request.get("check_context", "aios/durable-release")),
))
if operation == "PROTECTED_MERGE":
number = int(request.get("pr_number", "0"))
self.broker.transport.expected_merge_head = request["expected_branch_oid"]
if request.get("expected_main_oid") is not None:
self.broker.transport.expected_merge_base = request["expected_main_oid"]
return dict(self.broker.merge(number))
if operation == "RECONCILE_AUTHORITY_MAIN":
old = request.get("expected_main_oid")
if not isinstance(old, str):
raise JobExecutionError("MAIN_RECONCILIATION_BINDING_INVALID")
if self.broker.transport.read_branch("main") != new_oid:
raise JobExecutionError("GITEA_MAIN_EXPECTATION_MISMATCH")
return {"authority_main": self.authority.ff_CAS_main_after_verified_protected_merge(old, new_oid)}
if operation == "COMMIT_FILE":
path, content = self._commit_file_spec(row, request)
return {
"new_oid": self.authority.commit_file_in_isolated_worktree(
base_oid=row["expected_branch_oid"], files={path: content},
message="chore(gitops): durable commit-file", worktree_name=job_id,
)
}
raise JobExecutionError("OPERATION_NOT_ALLOWED")
def _observe_operation(self, request: Mapping[str, Any]) -> dict[str, Any]:
operation = str(request["operation"])
new_oid = str(request["new_branch_oid"])
if operation == "ADVANCE_EXISTING_CAMPAIGN_BRANCH":
authority = self.authority.read_ref(f"refs/heads/{BRANCH}")
remote = self.broker.readback()
if authority == new_oid and remote == new_oid:
return {"verification": "VERIFIED", "authority_oid": authority, "remote_oid": remote}
expected = request["expected_branch_oid"]
if authority == expected and remote in {None, expected}:
return {"verification": "ABSENT", "authority_oid": authority, "remote_oid": remote}
return {"verification": "UNKNOWN", "blocker": "PARTIAL_OR_CONFLICTING_BRANCH_EFFECT", "authority_oid": authority, "remote_oid": remote}
if operation == "SYNC_TASK_BRANCH_TO_GITEA":
remote = self.broker.readback()
if remote == new_oid:
return {"verification": "VERIFIED", "remote_oid": remote}
if remote in {None, EXPECTED_BRANCH_OID}:
return {"verification": "ABSENT", "remote_oid": remote}
return {"verification": "UNKNOWN", "blocker": "REMOTE_BRANCH_CONFLICT", "remote_oid": remote}
if operation == "RECONCILE_AUTHORITY_MAIN":
current = self.authority.read_ref("refs/heads/main")
if current == new_oid:
return {"verification": "VERIFIED", "authority_main": current}
if current == request.get("expected_main_oid"):
return {"verification": "ABSENT", "authority_main": current}
return {"verification": "UNKNOWN", "blocker": "AUTHORITY_MAIN_CONFLICT", "authority_main": current}
if operation == "CREATE_PR" and hasattr(self.broker, "read_pr"):
value = self.broker.read_pr(BRANCH, "main", new_oid)
return {"verification": "VERIFIED" if value else "ABSENT", "pr": value}
if operation == "PUBLISH_CHECK" and hasattr(self.broker, "read_status"):
value = self.broker.read_status(
new_oid, str(request.get("check_context", "aios/durable-release"))
)
if value is None:
return {"verification": "ABSENT"}
expected_state = str(request.get("check_state", "success"))
return {"verification": "VERIFIED" if value.get("status") == expected_state else "UNKNOWN", "status": dict(value), **({} if value.get("status") == expected_state else {"blocker": "STATUS_READBACK_MISMATCH"})}
if operation == "PROTECTED_MERGE" and hasattr(self.broker, "read_merge"):
value = self.broker.read_merge(int(request.get("pr_number", "0")))
if value is None:
return {"verification": "ABSENT"}
if value.get("merged") is True and value.get("head_oid") == request["expected_branch_oid"]:
return {"verification": "VERIFIED", "merge": dict(value)}
if value.get("merged") is False:
return {"verification": "ABSENT", "merge": dict(value)}
return {"verification": "UNKNOWN", "blocker": "MERGE_READBACK_MISMATCH", "merge": dict(value)}
return {"verification": "UNKNOWN", "blocker": "EXACT_READBACK_UNSUPPORTED"}
def _complete(
self, job_id: str, request: Mapping[str, Any], effect: Mapping[str, Any], observation: Mapping[str, Any]
) -> dict[str, Any]:
row = self.controller.store.read(job_id)
receipt = {
"effect_id": job_id,
"operation": request["operation"],
"effect": dict(effect),
"verification": "VERIFIED",
"readback": dict(observation),
}
completed = self.controller.advance(
job_id, JobState.COMPLETED, side_effect_state="VERIFIED",
receipt=receipt, expected_revision=int(row["revision"]),
)
return completed.as_dict()
def _record_unknown(
self,
job_id: str,
request: Mapping[str, Any],
observation: Mapping[str, Any],
*,
effect: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
row = self.controller.store.read(job_id)
receipt = {
"effect_id": job_id,
"operation": request["operation"],
"effect": dict(effect) if effect is not None else None,
"verification": "UNKNOWN",
"blocker": observation.get("blocker", "READBACK_INCONCLUSIVE"),
"readback": dict(observation),
}
pending = self.controller.advance(
job_id, JobState.RECONCILIATION_REQUIRED, side_effect_state="UNKNOWN",
receipt=receipt, expected_revision=int(row["revision"]),
)
return pending.as_dict()
def reconcile(self, job_id: str) -> dict[str, Any]:
def observer(_current: Mapping[str, Any]) -> Mapping[str, Any]:
_row, request = self._request(job_id)
self._validate_campaign(request)
observation = self._observe_operation(request)
if observation["verification"] == "VERIFIED":
return {
"state": JobState.COMPLETED.value,
"side_effect_state": "VERIFIED",
"receipt": {
"effect_id": job_id, "operation": request["operation"],
"effect": None, "verification": "VERIFIED",
"readback": observation, "reconciled": True,
},
}
if observation["verification"] == "ABSENT":
return {
"state": JobState.REQUESTED.value,
"side_effect_state": "RECONCILED_ABSENT",
"receipt": {
"effect_id": job_id, "operation": request["operation"],
"verification": "ABSENT", "readback": observation,
"reconciled": True,
},
}
return {
"state": JobState.RECONCILIATION_REQUIRED.value,
"side_effect_state": "UNKNOWN",
"receipt": {
"effect_id": job_id, "operation": request["operation"],
"verification": "UNKNOWN",
"blocker": observation.get("blocker", "READBACK_INCONCLUSIVE"),
"readback": observation, "reconciled": True,
},
}
result = self.controller.reconcile(job_id, observer).as_dict()
if result["state"] in {JobState.COMPLETED.value, JobState.REQUESTED.value}:
self._persistence_halted = False
return result
def _commit_file_spec(
self, row: Mapping[str, Any], request: Mapping[str, Any]
) -> tuple[str, bytes]:
payload_ref = request.get("payload_ref")
if not payload_ref:
raise JobExecutionError("PAYLOAD_REQUIRED")
@@ -244,8 +484,7 @@ class DurableJobExecutor:
content = base64.b64decode(spec["content_base64"], validate=True)
except Exception as exc:
raise JobExecutionError("COMMIT_FILE_PAYLOAD_INVALID") from exc
new_oid = self.authority.commit_file_in_isolated_worktree(base_oid=row["expected_branch_oid"], files={path: content}, message="chore(gitops): durable commit-file", worktree_name=job_id)
return {"operation": "COMMIT_FILE", "new_oid": new_oid}
return path, content
def build_from_environment(job_root: Path) -> DurableJobExecutor:
+32 -4
View File
@@ -57,10 +57,36 @@ def seed_base_models():
if not one('SELECT * FROM gitops_versions WHERE version=?', ('V1R1C0',)):
exec_sql('INSERT INTO gitops_versions(version,tag,level,summary,status,created_by,created_at) VALUES(?,?,?,?,?,?,?)', ('V1R1C0','V1R1C0','R','MultiUser runtime selector + GitOps + custom LLM priority registry','ready','system',now()))
_ROLE_WHITELIST = {'owner', 'admin', 'user'}
def token_identity(token: str):
"""Resolve the server-authoritative identity bound to a token.
Format of each TOKEN_FILE line (one per line, ':'-separated):
<token> -> authenticated principal, role 'user'
<token>:<user>:<role> -> mapped principal; role restricted to owner/admin/user
Identity NEVER comes from a client-supplied header. In local dev mode the
DEV_ALLOW flag maps a fixed dev principal (must be explicitly enabled).
"""
if DEV_ALLOW:
return {'username': 'dev-owner', 'role': 'owner'}
if not token or not TOKEN_FILE.exists():
return None
for line in TOKEN_FILE.read_text().splitlines():
line = line.strip()
if not line:
continue
parts = [x.strip() for x in line.split(':')]
if not parts or parts[0] != token:
continue
if len(parts) >= 3:
role = parts[2] if parts[2] in _ROLE_WHITELIST else 'user'
return {'username': safe_name(parts[1]), 'role': role}
return {'username': 'user-%s' % token[:8], 'role': 'user'}
return None
def valid_token(token: str) -> bool:
if DEV_ALLOW: return True
if not token or not TOKEN_FILE.exists(): return False
return token.strip() in [x.strip() for x in TOKEN_FILE.read_text().splitlines() if x.strip()]
return token_identity(token) is not None
def git_cmd(args, check=False):
WORKSPACE.mkdir(parents=True, exist_ok=True); p = subprocess.run(['git'] + args, cwd=str(WORKSPACE), text=True, capture_output=True)
if check and p.returncode != 0: raise RuntimeError(p.stderr or p.stdout or 'git command failed')
@@ -90,8 +116,10 @@ class Handler(BaseHTTPRequestHandler):
server_version = 'AIOSGitOpsModelAPI/V1R1C0'
def actor(self):
auth = self.headers.get('Authorization',''); token = auth.replace('Bearer ', '').strip() if auth.startswith('Bearer ') else self.headers.get('X-AIOS-Token','')
ok = valid_token(token); user = self.headers.get('X-AIOS-User') or ('dev-owner' if DEV_ALLOW else 'anonymous'); role = self.headers.get('X-AIOS-Role') or ('owner' if DEV_ALLOW else 'user')
return {'username': safe_name(user), 'role': role, 'token_ok': ok}
ident = token_identity(token)
if ident is None:
return {'username': 'anonymous', 'role': 'user', 'token_ok': False}
return {'username': ident['username'], 'role': ident['role'], 'token_ok': True}
def require_write(self, roles=None):
a = self.actor()
if not a['token_ok']: self.send_json({'error':'unauthorized','hint':'write APIs require Bearer token or AIOS_GITOPS_DEV_ALLOW=1'}, 401); return None
@@ -0,0 +1,890 @@
"""Read-only four-authority evidence provider for one canonical C04 attachment.
This module never creates, claims, resumes, reconciles, or mutates a Task or
Job. The expired Owner epoch is verified as historical provenance only.
"""
from __future__ import annotations
import hashlib
import json
import os
import sqlite3
import stat
import time
import urllib.error
import urllib.parse
import urllib.request
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Callable, Mapping
from app.aios_gitops_job_identity import legacy_task_storage_key, task_storage_key
from app.aios_single_host_authority import (
DEFAULT_OWNER_SIGNER_ID,
OWNER_NAMESPACE,
RegistryOwnerApprovalVerifier,
SignerRegistry,
)
from app.task_plane.legacy_attachment import (
ATTACH_EVIDENCE_SCHEMA,
AuthorityReadbackRef,
SOURCE_KIND,
VerifiedLegacyAttachmentEvidence,
)
class LegacyAttachmentRuntimeError(RuntimeError):
"""Stable fail-closed readback error."""
def _fail(code: str) -> None:
raise LegacyAttachmentRuntimeError(code)
def _strict_object(pairs: list[tuple[str, Any]]) -> dict[str, Any]:
value: dict[str, Any] = {}
for key, item in pairs:
if key in value:
_fail("DUPLICATE_JSON_FIELD")
value[key] = item
return value
def _json_object(raw: bytes | str, code: str) -> dict[str, Any]:
try:
if isinstance(raw, bytes):
raw = raw.decode("utf-8")
value = json.loads(raw, object_pairs_hook=_strict_object)
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise LegacyAttachmentRuntimeError(code) from exc
if not isinstance(value, dict):
_fail(code)
return value
def _canonical(value: object) -> bytes:
try:
return json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
allow_nan=False,
).encode("utf-8")
except (TypeError, ValueError) as exc:
raise LegacyAttachmentRuntimeError("AUTHORITY_JSON_INVALID") from exc
def _sha(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _hex(value: object, code: str) -> str:
if (
not isinstance(value, str)
or len(value) != 64
or any(ch not in "0123456789abcdef" for ch in value)
):
_fail(code)
return value
def _text(value: object, code: str) -> str:
if not isinstance(value, str) or not value or value != value.strip():
_fail(code)
return value
def _integer(value: object, code: str) -> int:
if type(value) is not int or value < 0:
_fail(code)
return value
def _iso(value: object, code: str) -> str:
text = _text(value, code)
try:
parsed = datetime.fromisoformat(text.replace("Z", "+00:00"))
except ValueError as exc:
raise LegacyAttachmentRuntimeError(code) from exc
if parsed.tzinfo is None or parsed.utcoffset() is None:
_fail(code)
return parsed.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")
def _observed(clock: Callable[[], float]) -> str:
return datetime.fromtimestamp(float(clock()), timezone.utc).isoformat().replace(
"+00:00", "Z"
)
def _mapping(value: object, code: str) -> dict[str, Any]:
if not isinstance(value, Mapping):
_fail(code)
return dict(value)
def _secure_file(
path: Path, *, require_root_owner: bool, writable_authority: bool = False
) -> os.stat_result:
try:
info = path.lstat()
except OSError as exc:
raise LegacyAttachmentRuntimeError("AUTHORITY_FILE_UNAVAILABLE") from exc
if path.is_symlink() or not stat.S_ISREG(info.st_mode):
_fail("AUTHORITY_FILE_INVALID")
if os.name == "posix":
if not writable_authority and stat.S_IMODE(info.st_mode) & 0o022:
_fail("AUTHORITY_FILE_WRITABLE")
if require_root_owner and info.st_uid != 0:
_fail("AUTHORITY_FILE_OWNER_INVALID")
return info
def _secure_directory(path: Path) -> os.stat_result:
try:
info = path.lstat()
except OSError as exc:
raise LegacyAttachmentRuntimeError("AUTHORITY_DIRECTORY_UNAVAILABLE") from exc
if path.is_symlink() or not stat.S_ISDIR(info.st_mode):
_fail("AUTHORITY_DIRECTORY_INVALID")
return info
def _stable_read(path: Path, *, require_root_owner: bool, maximum: int = 2_000_000) -> bytes:
before = _secure_file(path, require_root_owner=require_root_owner)
if before.st_size > maximum:
_fail("AUTHORITY_FILE_TOO_LARGE")
try:
raw = path.read_bytes()
except OSError as exc:
raise LegacyAttachmentRuntimeError("AUTHORITY_FILE_UNAVAILABLE") from exc
after = _secure_file(path, require_root_owner=require_root_owner)
if (before.st_dev, before.st_ino, before.st_size, before.st_mtime_ns) != (
after.st_dev,
after.st_ino,
after.st_size,
after.st_mtime_ns,
):
_fail("AUTHORITY_FILE_CHANGED")
return raw
def _ro_connection(path: Path, *, require_root_owner: bool) -> tuple[sqlite3.Connection, os.stat_result]:
before = _secure_file(
path, require_root_owner=require_root_owner, writable_authority=True
)
try:
resolved = path.resolve(strict=True)
connection = sqlite3.connect(
resolved.as_uri() + "?mode=ro", uri=True, timeout=5.0
)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA query_only=ON")
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("BEGIN")
return connection, before
except (OSError, sqlite3.Error) as exc:
raise LegacyAttachmentRuntimeError("AUTHORITY_DB_UNAVAILABLE") from exc
def _finish_ro(connection: sqlite3.Connection, path: Path, before: os.stat_result) -> None:
connection.close()
after = _secure_file(
path, require_root_owner=False, writable_authority=True
)
if (before.st_dev, before.st_ino) != (after.st_dev, after.st_ino):
_fail("AUTHORITY_DB_CHANGED")
def _columns(connection: sqlite3.Connection, table: str) -> tuple[str, ...]:
return tuple(str(row[1]) for row in connection.execute(f"PRAGMA table_info({table})"))
@dataclass(frozen=True, slots=True)
class LegacyC04Binding:
mission_id: str
root_task_id: str
anchor_task_id: str
owner_principal_id: str
original_acceptance_digest: str
source_task_id: str
external_job_id: str
def validate(self) -> None:
for value in (
self.mission_id,
self.root_task_id,
self.anchor_task_id,
self.owner_principal_id,
self.source_task_id,
self.external_job_id,
):
_text(value, "C04_BINDING_INVALID")
_hex(self.original_acceptance_digest, "C04_ACCEPTANCE_DIGEST_INVALID")
@dataclass(frozen=True, slots=True)
class CredentialFileBearerSupplier:
"""Read a systemd credential for one call without caching or displaying it."""
path: Path
def __post_init__(self) -> None:
if not self.path.is_absolute():
_fail("AGENT_TASK_CREDENTIAL_PATH_INVALID")
if os.name == "posix":
try:
self.path.resolve(strict=False).relative_to(Path("/run/credentials"))
except ValueError as exc:
raise LegacyAttachmentRuntimeError(
"AGENT_TASK_CREDENTIAL_PATH_INVALID"
) from exc
def __call__(self) -> str:
raw = _stable_read(self.path, require_root_owner=False, maximum=8192)
try:
value = raw.decode("utf-8").strip()
except UnicodeDecodeError as exc:
raise LegacyAttachmentRuntimeError(
"AGENT_TASK_CREDENTIAL_INVALID"
) from exc
if len(value) < 32 or len(value) > 4096:
_fail("AGENT_TASK_CREDENTIAL_INVALID")
return value
class _NoRedirect(urllib.request.HTTPRedirectHandler):
def redirect_request(self, req, fp, code, msg, headers, newurl): # noqa: ANN001
return None
_AGENT_TASK_COLUMNS = (
"task_id",
"goal",
"instructions",
"success_criteria_json",
"required_capabilities_json",
"constraints_json",
"metadata_json",
"deadline",
"priority",
"max_attempts",
"attempt",
"status",
"claimed_by",
"lease_token_hash",
"lease_expires_at",
"result_json",
"result_digest",
"result_idempotency_key",
"verification_json",
"revision",
"created_at",
"updated_at",
)
@dataclass(slots=True)
class AuthenticatedAgentTaskReader:
"""Authenticated non-mutating proof + read-only authority-row snapshot.
The authenticated HTTP request is intentionally sent to the task events
endpoint because the historical GET /v1/tasks/{id} path calls the global
lease-expiry mutator. The task row itself is read from the authoritative
SQLite database using query_only mode and an exact schema allowlist.
"""
base_url: str
bearer_supplier: Callable[[], str]
task_db_path: Path
timeout_seconds: float = 10.0
clock: Callable[[], float] = time.time
require_db_root_owner: bool = False
def __post_init__(self) -> None:
parsed = urllib.parse.urlsplit(self.base_url)
if (
parsed.scheme != "http"
or parsed.hostname not in {"127.0.0.1", "localhost", "::1"}
or parsed.username is not None
or parsed.password is not None
or parsed.query
or parsed.fragment
):
_fail("AGENT_TASK_URL_INVALID")
if not self.task_db_path.is_absolute():
_fail("AGENT_TASK_DB_PATH_INVALID")
self.base_url = self.base_url.rstrip("/")
def _authenticated_events(self, task_id: str) -> tuple[list[dict[str, Any]], str]:
token = self.bearer_supplier()
if not isinstance(token, str) or len(token.strip()) < 32:
_fail("AGENT_TASK_CREDENTIAL_INVALID")
url = (
self.base_url
+ "/v1/tasks/"
+ urllib.parse.quote(task_id, safe="._:-")
+ "/events"
)
request = urllib.request.Request(
url,
method="GET",
headers={
"Authorization": "Bearer " + token.strip(),
"Accept": "application/json",
},
)
try:
with urllib.request.build_opener(_NoRedirect).open(
request, timeout=self.timeout_seconds
) as response:
if response.status != 200 or response.geturl() != url:
_fail("AGENT_TASK_EVENTS_REJECTED")
raw = response.read(2_000_001)
except (urllib.error.URLError, TimeoutError, OSError) as exc:
raise LegacyAttachmentRuntimeError(
"AGENT_TASK_EVENTS_UNAVAILABLE"
) from exc
finally:
token = ""
if len(raw) > 2_000_000:
_fail("AGENT_TASK_EVENTS_TOO_LARGE")
try:
value = json.loads(raw.decode("utf-8"), object_pairs_hook=_strict_object)
except (UnicodeDecodeError, json.JSONDecodeError, ValueError) as exc:
raise LegacyAttachmentRuntimeError("AGENT_TASK_EVENTS_INVALID") from exc
if not isinstance(value, list):
_fail("AGENT_TASK_EVENTS_INVALID")
events: list[dict[str, Any]] = []
for event in value:
if not isinstance(event, Mapping) or event.get("task_id") != task_id:
_fail("AGENT_TASK_EVENTS_INVALID")
events.append(dict(event))
return events, url
@staticmethod
def _task_projection(row: sqlite3.Row) -> dict[str, Any]:
try:
body = {
"task_id": row["task_id"],
"goal": row["goal"],
"instructions": row["instructions"],
"success_criteria": json.loads(row["success_criteria_json"]),
"required_capabilities": json.loads(row["required_capabilities_json"]),
"constraints": json.loads(row["constraints_json"]),
"metadata": json.loads(row["metadata_json"]),
"deadline": row["deadline"],
"priority": row["priority"],
"max_attempts": row["max_attempts"],
"attempt": row["attempt"],
"status": row["status"],
"claimed_by": row["claimed_by"],
"lease_expires_at": row["lease_expires_at"],
"revision": row["revision"],
"created_at": row["created_at"],
"updated_at": row["updated_at"],
"result": json.loads(row["result_json"]) if row["result_json"] else None,
"verification": (
json.loads(row["verification_json"])
if row["verification_json"]
else None
),
}
except (KeyError, TypeError, json.JSONDecodeError) as exc:
raise LegacyAttachmentRuntimeError(
"AGENT_TASK_ROW_INVALID"
) from exc
_mapping(body["constraints"], "AGENT_TASK_CONSTRAINTS_INVALID")
_mapping(body["metadata"], "AGENT_TASK_METADATA_INVALID")
_integer(body["revision"], "AGENT_TASK_REVISION_INVALID")
if body["status"] not in {"SUCCEEDED", "FAILED", "EXPIRED", "CANCELLED"}:
_fail("AGENT_TASK_NOT_TERMINAL")
return body
def read(self, task_id: str) -> dict[str, Any]:
events, events_url = self._authenticated_events(task_id)
connection, before = _ro_connection(
self.task_db_path, require_root_owner=self.require_db_root_owner
)
try:
if _columns(connection, "tasks") != _AGENT_TASK_COLUMNS:
_fail("AGENT_TASK_DB_SCHEMA_INVALID")
row = connection.execute(
"SELECT * FROM tasks WHERE task_id=? LIMIT 1", (task_id,)
).fetchone()
if row is None:
_fail("AGENT_TASK_NOT_FOUND")
body = self._task_projection(row)
finally:
_finish_ro(connection, self.task_db_path, before)
event_digest = _sha(events)
return {
"body": body,
"observed_at": _observed(self.clock),
"readback_ref": (
events_url
+ "#authenticated-events-sha256="
+ event_digest
+ ";task-db-revision="
+ str(body["revision"])
),
"canonical_sha256": _sha(body),
"authenticated_events_sha256": event_digest,
}
_ADMISSION_COLUMNS = {
"task_admission_envelopes": (
"idempotency_key", "envelope_id", "request_digest", "envelope_json", "status", "created_at"
),
"task_admission_grants": (
"envelope_id", "grant_id", "grant_digest", "grant_json", "authorization_digest", "created_at"
),
"task_admission_receipts": ("envelope_id", "task_id", "receipt_json", "created_at"),
}
@dataclass(slots=True)
class AdmissionEpochRegistryReader:
db_path: Path
epoch_root: Path
revocation_file: Path
registry_path: Path
require_db_root_owner: bool = False
require_static_root_owner: bool = True
clock: Callable[[], float] = time.time
def __post_init__(self) -> None:
for path in (self.db_path, self.epoch_root, self.revocation_file, self.registry_path):
if not path.is_absolute():
_fail("AUTHORITY_PATH_NOT_ABSOLUTE")
def read(self, task_id: str) -> dict[str, Any]:
connection, before = _ro_connection(
self.db_path, require_root_owner=self.require_db_root_owner
)
try:
if any(_columns(connection, table) != columns for table, columns in _ADMISSION_COLUMNS.items()):
_fail("ADMISSION_SCHEMA_INVALID")
rows = connection.execute(
"SELECT e.*,g.grant_id,g.grant_digest,g.grant_json,g.authorization_digest,"
"g.created_at AS grant_created_at,r.task_id,r.receipt_json,"
"r.created_at AS receipt_created_at FROM task_admission_receipts r "
"JOIN task_admission_envelopes e ON e.envelope_id=r.envelope_id "
"JOIN task_admission_grants g ON g.envelope_id=r.envelope_id WHERE r.task_id=?",
(task_id,),
).fetchall()
if len(rows) != 1:
_fail("ADMISSION_READBACK_AMBIGUOUS")
row = dict(rows[0])
except sqlite3.Error as exc:
raise LegacyAttachmentRuntimeError("ADMISSION_READBACK_FAILED") from exc
finally:
_finish_ro(connection, self.db_path, before)
envelope = _json_object(row["envelope_json"], "ADMISSION_ENVELOPE_INVALID")
grant = _json_object(row["grant_json"], "ADMISSION_GRANT_INVALID")
receipt = _json_object(row["receipt_json"], "ADMISSION_RECEIPT_INVALID")
envelope_id = _text(row["envelope_id"], "ADMISSION_ENVELOPE_INVALID")
grant_id = _text(row["grant_id"], "ADMISSION_GRANT_INVALID")
epoch_id = _text(grant.get("parent_epoch_id"), "ADMISSION_EPOCH_ID_INVALID")
if (
row.get("status") != "ADMITTED"
or envelope.get("schema") != "aios.task-admission-envelope.v1"
or grant.get("schema") != "aios.task-admission-grant.v1"
or receipt.get("schema") != "aios.task-admission-receipt.v1"
or envelope.get("envelope_id") != envelope_id
or grant.get("envelope_id") != envelope_id
or receipt.get("envelope_id") != envelope_id
or envelope.get("task_id") != task_id
or grant.get("task_id") != task_id
or receipt.get("task_id") != task_id
or grant.get("grant_id") != grant_id
or receipt.get("grant_id") != grant_id
or grant.get("authorization_kind") != "delegation_epoch"
or receipt.get("authorization_kind") != "delegation_epoch"
or receipt.get("parent_epoch_id") != epoch_id
):
_fail("ADMISSION_BINDING_INVALID")
request_value = {
key: envelope.get(key)
for key in (
"task_id", "repository", "authority_base_oid", "target_branch",
"generation", "execution_digest", "requested_capabilities",
"file_allowlist", "risk", "trust_domain", "policy_digest",
)
}
if envelope.get("request_digest") != _sha(request_value) or row.get("request_digest") != _sha(request_value):
_fail("ADMISSION_REQUEST_DIGEST_INVALID")
grant_digest = _sha(grant)
if row.get("grant_digest") != grant_digest or receipt.get("grant_digest_sha256") != grant_digest:
_fail("ADMISSION_GRANT_DIGEST_INVALID")
auth_value = {
"envelope_id": envelope_id,
"authorization_kind": "delegation_epoch",
"delegation_epoch_id": epoch_id,
}
if row.get("authorization_digest") != _sha(auth_value):
_fail("ADMISSION_AUTHORIZATION_DIGEST_INVALID")
linked = (
"repository", "authority_base_oid", "target_branch", "generation",
"execution_digest", "policy_digest", "trust_domain", "risk",
)
if any(grant.get(key) != envelope.get(key) for key in linked):
_fail("ADMISSION_GRANT_SCOPE_INVALID")
if grant.get("capabilities") != envelope.get("requested_capabilities") or grant.get("file_allowlist") != envelope.get("file_allowlist"):
_fail("ADMISSION_GRANT_SCOPE_INVALID")
if receipt.get("execution_digest") != grant.get("execution_digest"):
_fail("ADMISSION_RECEIPT_SCOPE_INVALID")
_secure_directory(self.epoch_root)
canonical_path = self.epoch_root / "owner-delegation-epoch.canonical.json"
signature_path = self.epoch_root / "owner-delegation-epoch.sig"
key_path = self.epoch_root / "owner-delegation-epoch.key-id"
canonical = _stable_read(canonical_path, require_root_owner=self.require_static_root_owner)
epoch = _json_object(canonical, "OWNER_EPOCH_INVALID")
if _canonical(epoch) != canonical:
_fail("OWNER_EPOCH_NOT_CANONICAL")
signature = _stable_read(signature_path, require_root_owner=self.require_static_root_owner).decode("utf-8")
key_id = _stable_read(key_path, require_root_owner=self.require_static_root_owner).decode("utf-8").strip()
registry_raw = _stable_read(self.registry_path, require_root_owner=self.require_static_root_owner)
registry_json = _json_object(registry_raw, "OWNER_REGISTRY_INVALID")
revocations = _json_object(
_stable_read(self.revocation_file, require_root_owner=self.require_static_root_owner),
"OWNER_REVOCATIONS_INVALID",
)
if (
set(revocations) != {"schema", "revoked_epoch_ids"}
or revocations.get("schema") != "aios.owner-delegation-revocations.v1"
or not isinstance(revocations.get("revoked_epoch_ids"), list)
or epoch_id in revocations["revoked_epoch_ids"]
):
_fail("OWNER_EPOCH_REVOKED")
try:
registry = SignerRegistry(
self.registry_path, require_root_owner=self.require_static_root_owner
)
record = registry.require(DEFAULT_OWNER_SIGNER_ID, role="OWNER_AUTHORITY")
verified = RegistryOwnerApprovalVerifier(
self.registry_path,
require_root_owner=self.require_static_root_owner,
signer_id=DEFAULT_OWNER_SIGNER_ID,
).verify(
namespace=OWNER_NAMESPACE,
payload=canonical,
signature=signature,
key_id=key_id,
)
except Exception as exc:
raise LegacyAttachmentRuntimeError("OWNER_REGISTRY_VERIFICATION_FAILED") from exc
now = int(self.clock())
provenance_checks = {
"signature": verified is True,
"fingerprint": record.fingerprint == key_id,
"schema": epoch.get("schema") == "aios.owner-delegation-epoch.v1",
"epoch_id": epoch.get("epoch_id") == epoch_id,
"owner": epoch.get("owner_identity") == grant.get("owner_identity"),
"repository": epoch.get("repository") == grant.get("repository"),
"trust_domain": epoch.get("trust_domain") == grant.get("trust_domain"),
"policy": epoch.get("policy_digest") == grant.get("policy_digest"),
"revocation_epoch": epoch.get("revocation_epoch") == 0,
"expires_type": type(epoch.get("expires_at")) is int,
"not_before_type": type(epoch.get("not_before")) is int,
}
failed = [name for name, passed in provenance_checks.items() if not passed]
if failed:
_fail("OWNER_EPOCH_PROVENANCE_INVALID:" + ",".join(sorted(failed)))
if (
now < epoch["expires_at"]
or type(epoch.get("issued_at")) is not int
or epoch["issued_at"] > grant.get("issued_at", 0)
or grant.get("issued_at", 0) < epoch["not_before"]
or grant.get("expires_at", 0) > epoch["expires_at"]
):
_fail("OWNER_EPOCH_PROVENANCE_INVALID:temporal")
if not any(str(grant["target_branch"]).startswith(str(prefix)) for prefix in epoch.get("allowed_branch_prefixes", [])):
_fail("OWNER_EPOCH_SCOPE_INVALID")
if not set(grant.get("capabilities", [])).issubset(set(epoch.get("allowed_capabilities", []))):
_fail("OWNER_EPOCH_SCOPE_INVALID")
risk_rank = {"low": 0, "medium": 1, "high": 2}
if (
grant.get("risk") not in risk_rank
or epoch.get("risk_ceiling") not in risk_rank
or risk_rank[grant["risk"]] > risk_rank[epoch["risk_ceiling"]]
):
_fail("OWNER_EPOCH_SCOPE_INVALID")
if any(
not any(path == prefix.rstrip("/") or path.startswith(prefix.rstrip("/") + "/") for prefix in epoch.get("allowed_file_prefixes", []))
for path in grant.get("file_allowlist", [])
):
_fail("OWNER_EPOCH_SCOPE_INVALID")
admission_projection = {
"envelope": envelope,
"grant": grant,
"receipt": receipt,
"request_digest": row["request_digest"],
"grant_digest": row["grant_digest"],
"authorization_digest": row["authorization_digest"],
"envelope_status": row["status"],
}
owner_projection = {
"epoch": epoch,
"epoch_sha256": hashlib.sha256(canonical).hexdigest(),
"epoch_key_id": key_id,
"signature_verified": True,
"epoch_currently_valid": False,
"revocations": revocations,
"registry_schema": registry_json.get("schema"),
"registry_generation": registry.authority_generation,
"signer": record.to_dict(),
}
observed = _observed(self.clock)
return {
"envelope": envelope,
"grant": grant,
"receipt": receipt,
"epoch": epoch,
"epoch_digest": hashlib.sha256(canonical).hexdigest(),
"signer_id": record.signer_id,
"signer_fingerprint": record.fingerprint,
"registry_generation": registry.authority_generation,
"observed_at": observed,
"admission_ref": f"sqlite:{self.db_path}#task={task_id};envelope={envelope_id}",
"admission_sha256": _sha(admission_projection),
"registry_ref": f"file:{self.registry_path}#generation={registry.authority_generation};epoch={epoch_id}",
"registry_sha256": _sha(owner_projection),
}
_JOB_COLUMNS = {
"job_aliases": (
"external_job_id", "internal_job_id", "task_id", "repository", "target_branch",
"branch_mode", "generation", "commit_idempotency_key", "expected_branch_oid",
"request_digest", "adopted_legacy", "created_at",
),
"jobs": (
"job_id", "idempotency_key", "campaign_id", "repository", "operation", "branch",
"expected_branch_oid", "payload_digest", "payload_ref", "payload_size", "state",
"side_effect_state", "revision", "request_json", "receipt_json", "created_at", "updated_at",
),
"idempotency_keys": ("key", "job_id", "binding_json", "created_at"),
}
@dataclass(slots=True)
class BridgeJobSQLiteReader:
jobs_root: Path
require_root_owner: bool = False
clock: Callable[[], float] = time.time
def __post_init__(self) -> None:
if not self.jobs_root.is_absolute():
_fail("BRIDGE_JOB_ROOT_NOT_ABSOLUTE")
def read(self, task_id: str, external_job_id: str) -> dict[str, Any]:
exact_root = self.jobs_root / task_storage_key(task_id)
legacy_root = self.jobs_root / legacy_task_storage_key(task_id)
db_path = exact_root / "state.sqlite3"
if legacy_root != exact_root and legacy_root.exists():
_fail("BRIDGE_JOB_STORE_AMBIGUOUS")
_secure_directory(self.jobs_root)
_secure_directory(exact_root)
try:
if db_path.resolve(strict=True).parent != exact_root.resolve(strict=True):
_fail("BRIDGE_JOB_STORE_INVALID")
except OSError as exc:
raise LegacyAttachmentRuntimeError("BRIDGE_JOB_STORE_INVALID") from exc
connection, before = _ro_connection(
db_path, require_root_owner=self.require_root_owner
)
try:
if any(_columns(connection, table) != columns for table, columns in _JOB_COLUMNS.items()):
_fail("BRIDGE_JOB_SCHEMA_INVALID")
aliases = connection.execute(
"SELECT * FROM job_aliases WHERE external_job_id=? AND task_id=?",
(external_job_id, task_id),
).fetchall()
if len(aliases) != 1:
_fail("BRIDGE_JOB_ALIAS_AMBIGUOUS")
alias = dict(aliases[0])
job_rows = connection.execute(
"SELECT * FROM jobs WHERE job_id=?", (alias["internal_job_id"],)
).fetchall()
key_rows = connection.execute(
"SELECT * FROM idempotency_keys WHERE job_id=?", (alias["internal_job_id"],)
).fetchall()
if len(job_rows) != 1 or len(key_rows) != 1:
_fail("BRIDGE_JOB_BINDING_MISSING")
job, key = dict(job_rows[0]), dict(key_rows[0])
except sqlite3.Error as exc:
raise LegacyAttachmentRuntimeError("BRIDGE_JOB_READBACK_FAILED") from exc
finally:
_finish_ro(connection, db_path, before)
request = _json_object(job.get("request_json") or "{}", "BRIDGE_JOB_REQUEST_INVALID")
binding = _mapping(request.get("binding", request), "BRIDGE_JOB_REQUEST_INVALID")
key_binding = _json_object(key.get("binding_json") or "{}", "BRIDGE_JOB_KEY_BINDING_INVALID")
external_commit_key = _text(alias.get("commit_idempotency_key"), "BRIDGE_EXTERNAL_IDEMPOTENCY_INVALID")
internal_key = _text(job.get("idempotency_key"), "BRIDGE_INTERNAL_IDEMPOTENCY_INVALID")
if (
alias.get("external_job_id") != external_job_id
or alias.get("task_id") != task_id
or job.get("job_id") != alias.get("internal_job_id")
or job.get("campaign_id") != task_id
or key.get("job_id") != job.get("job_id")
or key.get("key") != internal_key
or binding.get("task_id", binding.get("campaign_id")) != task_id
or binding.get("external_job_id") != external_job_id
or binding.get("idempotency_key") != external_commit_key
or key_binding.get("task_id", key_binding.get("campaign_id")) != task_id
or key_binding.get("external_job_id") != external_job_id
or key_binding.get("idempotency_key") != external_commit_key
or alias.get("repository") != job.get("repository")
or alias.get("repository") != binding.get("repository")
or alias.get("target_branch") != job.get("branch")
or alias.get("target_branch") != binding.get("branch", binding.get("target_branch"))
):
_fail("BRIDGE_JOB_BINDING_INVALID")
try:
if int(alias.get("generation")) != int(binding.get("generation")):
_fail("BRIDGE_JOB_BINDING_INVALID")
except (TypeError, ValueError) as exc:
raise LegacyAttachmentRuntimeError("BRIDGE_JOB_BINDING_INVALID") from exc
projection = {
"alias": alias,
"job": job,
"idempotency": key,
"external_commit_idempotency_key": external_commit_key,
"internal_idempotency_key": internal_key,
}
return {
**projection,
"observed_at": _observed(self.clock),
"readback_ref": f"sqlite:{db_path}#external={external_job_id};internal={job['job_id']}",
"canonical_sha256": _sha(projection),
}
@dataclass(slots=True)
class LegacyAttachmentRuntimeProvider:
binding: LegacyC04Binding
agent_task: AuthenticatedAgentTaskReader
admission: AdmissionEpochRegistryReader
bridge_job: BridgeJobSQLiteReader
def __post_init__(self) -> None:
self.binding.validate()
def read_verified(
self,
*,
mission_id: str,
root_task_id: str,
anchor_task_id: str,
source_kind: str,
source_task_id: str,
external_job_id: str,
original_acceptance_digest: str,
) -> VerifiedLegacyAttachmentEvidence:
expected = self.binding
if (
mission_id != expected.mission_id
or root_task_id != expected.root_task_id
or anchor_task_id != expected.anchor_task_id
or source_kind != SOURCE_KIND
or source_task_id != expected.source_task_id
or external_job_id != expected.external_job_id
or original_acceptance_digest != expected.original_acceptance_digest
):
_fail("C04_AUTHORITY_BINDING_MISMATCH")
task_read = self.agent_task.read(source_task_id)
admission_read = self.admission.read(source_task_id)
job_read = self.bridge_job.read(source_task_id, external_job_id)
task = task_read["body"]
metadata = _mapping(task["metadata"], "AGENT_TASK_METADATA_INVALID")
constraints = _mapping(task["constraints"], "AGENT_TASK_CONSTRAINTS_INVALID")
execution = _mapping(metadata.get("execution"), "AGENT_TASK_EXECUTION_INVALID")
grant = admission_read["grant"]
envelope = admission_read["envelope"]
alias, job = job_read["alias"], job_read["job"]
try:
immutable_metadata = _json_object(
constraints.get("immutable_metadata", ""), "AGENT_TASK_IMMUTABLE_METADATA_INVALID"
)
except LegacyAttachmentRuntimeError:
raise
if (
immutable_metadata != metadata
or metadata.get("grant_id") != grant.get("grant_id")
or metadata.get("grant_digest_sha256") != admission_read["receipt"].get("grant_digest_sha256")
or metadata.get("execution_digest") != grant.get("execution_digest")
or execution.get("execution_digest") != grant.get("execution_digest")
or execution.get("external_job_id") != external_job_id
or execution.get("task_id") != source_task_id
or metadata.get("repository") != grant.get("repository")
or metadata.get("target_branch") != grant.get("target_branch")
or metadata.get("generation") != grant.get("generation")
or constraints.get("repository") != grant.get("repository")
or constraints.get("target_branch") != grant.get("target_branch")
or constraints.get("generation") != grant.get("generation")
or envelope.get("task_id") != source_task_id
or alias.get("repository") != grant.get("repository")
or alias.get("target_branch") != grant.get("target_branch")
or alias.get("generation") != grant.get("generation")
or admission_read["signer_id"] != expected.owner_principal_id
):
_fail("FOUR_AUTHORITY_BINDING_INVALID")
return VerifiedLegacyAttachmentEvidence(
schema=ATTACH_EVIDENCE_SCHEMA,
source_kind=SOURCE_KIND,
source_task_id=source_task_id,
source_task_created_at=_iso(task.get("created_at"), "AGENT_TASK_CREATED_AT_INVALID"),
source_task_status=_text(task.get("status"), "AGENT_TASK_STATUS_INVALID"),
source_task_revision=_integer(task.get("revision"), "AGENT_TASK_REVISION_INVALID"),
source_task_attempt=_integer(task.get("attempt"), "AGENT_TASK_ATTEMPT_INVALID"),
source_task_max_attempts=_integer(task.get("max_attempts"), "AGENT_TASK_MAX_ATTEMPTS_INVALID"),
external_job_id=external_job_id,
internal_job_id=_text(job.get("job_id"), "BRIDGE_INTERNAL_JOB_INVALID"),
internal_idempotency_key=_text(job.get("idempotency_key"), "BRIDGE_INTERNAL_IDEMPOTENCY_INVALID"),
job_branch=_text(job.get("branch"), "BRIDGE_JOB_BRANCH_INVALID"),
job_state=_text(job.get("state"), "BRIDGE_JOB_STATE_INVALID"),
job_revision=_integer(job.get("revision"), "BRIDGE_JOB_REVISION_INVALID"),
side_effect_state=_text(job.get("side_effect_state"), "BRIDGE_SIDE_EFFECT_INVALID"),
grant_id=_text(grant.get("grant_id"), "ADMISSION_GRANT_INVALID"),
grant_digest=_hex(admission_read["receipt"].get("grant_digest_sha256"), "ADMISSION_GRANT_DIGEST_INVALID"),
execution_digest=_hex(grant.get("execution_digest"), "ADMISSION_EXECUTION_DIGEST_INVALID"),
epoch_id=_text(admission_read["epoch"].get("epoch_id"), "OWNER_EPOCH_ID_INVALID"),
epoch_digest=_hex(admission_read["epoch_digest"], "OWNER_EPOCH_DIGEST_INVALID"),
historical_signer_id=admission_read["signer_id"],
historical_signer_fingerprint=admission_read["signer_fingerprint"],
registry_generation=_integer(admission_read["registry_generation"], "OWNER_REGISTRY_GENERATION_INVALID"),
signature_verified=True,
epoch_currently_valid=False,
readbacks=(
AuthorityReadbackRef(
role="agent_task", authority="aios-agent-task-api",
readback_ref=task_read["readback_ref"], canonical_sha256=task_read["canonical_sha256"],
observed_at=task_read["observed_at"], revision=task["revision"],
),
AuthorityReadbackRef(
role="admission", authority="aios-task-admission-store",
readback_ref=admission_read["admission_ref"], canonical_sha256=admission_read["admission_sha256"],
observed_at=admission_read["observed_at"], revision=None,
),
AuthorityReadbackRef(
role="owner_registry", authority="aios-single-host-owner-authority",
readback_ref=admission_read["registry_ref"], canonical_sha256=admission_read["registry_sha256"],
observed_at=admission_read["observed_at"], revision=admission_read["registry_generation"],
),
AuthorityReadbackRef(
role="bridge_job", authority="aios-gitops-job-controller",
readback_ref=job_read["readback_ref"], canonical_sha256=job_read["canonical_sha256"],
observed_at=job_read["observed_at"], revision=job["revision"],
),
),
)
+95 -19
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import hashlib, hmac, json, os, re, subprocess, sys, time, urllib.parse, urllib.request, uuid
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from dataclasses import replace
from pathlib import Path
from baseline_authority import BaselineAuthority, BaselineAuthorityError
from silent_release_controller import ReleaseCandidate
@@ -114,6 +115,9 @@ TRUSTED_BUILDER_SOCKET = Path(os.getenv(
TRUSTED_BUILDER_PUBLIC_KEYRING = Path(os.getenv(
'AIOS_TRUSTED_CANDIDATE_BUILDER_PUBLIC_KEYRING', ''
))
TRUSTED_BUILDER_SOURCE_ADMISSION_ROOT = str(os.getenv(
'AIOS_TRUSTED_BUILDER_SOURCE_ADMISSION_ROOT', ''
)).strip()
TRUSTED_RELEASE_POLICY = Path(os.getenv(
'AIOS_SILENT_RELEASE_TRUSTED_POLICY',
str(WS / 'config' / 'gitops' / 'silent-release-policy.json'),
@@ -131,6 +135,11 @@ SECRET_VAL_RE = re.compile(
r'(sk-[A-Za-z0-9_-]{20,}|HPUA[A-Z0-9]{10,}|SDC[A-Za-z0-9]{20,})'
)
# Control-plane file read: enforce auth, strict identifier, containment and a
# size cap on the command-detail read exposed by /admin/ai-evolution/api/data.
MAX_COMMAND_READ_BYTES = 1024 * 1024
_COMMAND_FILENAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*\.md$")
READ_ONLY_GET_PATHS = frozenset({
'/', '/health', '/api/health', '/api/openapi.yaml',
'/api/openapi-chatgpt-actions.yaml', '/api/schema', '/api', '/api/status',
@@ -151,6 +160,7 @@ SILENT_RELEASE_POST_PATHS = frozenset({
'/submitSignedApproval',
'/submitOwnerApproval',
'/startDeployment',
'/reopenOwnerApproval',
})
def reject_non_read_http(handler, method: str, path: str) -> bool:
@@ -253,6 +263,22 @@ class _HostTrustedCandidateBuilderAdapter:
nonce=secrets.token_hex(32),
)
except BuilderError as exc:
code = str(exc)
if code in {'SOURCE_ADMISSION_REQUIRED', 'RECIPE_UNKNOWN'} or (
code == 'REQUEST_RELEASE_BINDING_INVALID'
and not intent.get('version')
):
raise CapabilityAuthorizationError(
'release source admission required'
) from exc
if code == 'MIGRATION_PLAN_REQUIRED':
raise CapabilityAuthorizationError(
'migration plan required'
) from exc
if code == 'MIGRATION_PLAN_INVALID':
raise CapabilityAuthorizationError(
'migration plan invalid'
) from exc
raise TrustedCandidateBuilderUnavailable(
'trusted builder request failed'
) from exc
@@ -290,6 +316,7 @@ def _build_silent_release_service() -> SilentReleaseActionService:
builder_attestation_verifier = None
trusted_policy = None
builder_readiness_probe = None
source_admission_resolver = None
if TRUSTED_BUILDER_ENABLED:
# Every path is opt-in. In particular, this API receives its own
# client HMAC credential, never the builder's signing private key.
@@ -317,6 +344,33 @@ def _build_silent_release_service() -> SilentReleaseActionService:
client_hmac = (
TRUSTED_BUILDER_CREDENTIALS_DIRECTORY / 'builder_hmac'
).read_bytes().strip()
if TRUSTED_BUILDER_SOURCE_ADMISSION_ROOT:
from release_source_admission import (
open_source_admission_store,
)
try:
source_admission_hmac = (
TRUSTED_BUILDER_CREDENTIALS_DIRECTORY
/ 'source_admission_hmac'
).read_bytes().strip()
except OSError as exc:
raise CapabilityAuthorizationError(
'release source admission credential is unavailable'
) from exc
source_store = open_source_admission_store(
Path(TRUSTED_BUILDER_SOURCE_ADMISSION_ROOT),
repository=builder_config.repository,
policy=TRUSTED_BUILDER_POLICY,
hmac_key=source_admission_hmac,
expected_owner_uid=0,
writable=False,
)
builder_config = replace(
builder_config,
source_admission_resolver=source_store.resolve_recipe,
)
source_admission_resolver = source_store.lookup_source
client_keyring = BuilderKeyring(builder_hmac=client_hmac)
client = UnixTrustedCandidateBuilderClient(
builder_config,
@@ -365,6 +419,7 @@ def _build_silent_release_service() -> SilentReleaseActionService:
builder_attestation_verifier=builder_attestation_verifier,
trusted_policy=trusted_policy,
builder_readiness_probe=builder_readiness_probe,
source_admission_resolver=source_admission_resolver,
allow_legacy_candidate_input=False,
expected_repository=REPOSITORY_IDENTITY,
policy_sha256=policy_sha,
@@ -640,20 +695,6 @@ class H(BaseHTTPRequestHandler):
return self.sendj(body, exc.code)
except (urllib.error.URLError, TimeoutError) as exc:
return self.sendj({'error': 'GOVERNED_BRIDGE_UNAVAILABLE', 'detail': type(exc).__name__}, 503)
def do_POST(self):
path = urllib.parse.urlparse(self.path).path
if path in {
'/createTagApprovalRequest', '/requestOrCreateTag', '/getTagPublicationStatus',
'/buildProductionArtifact', '/executeApprovedProductionRelease',
'/getReleaseConsistency', '/runReleaseAcceptance',
}:
actor = self.need('tag:create')
if not actor:
return
return self.bridge_release(path, self.body())
if reject_non_read_http(self, 'POST', path):
return
return self.sendj({'error': 'not_found', 'path': path}, 404)
def do_GET(self):
u=urllib.parse.urlparse(self.path); path=u.path.rstrip('/') if u.path != '/' else u.path
if path not in READ_ONLY_GET_PATHS: return self.sendj({'error':'not_found','path':path},404)
@@ -899,6 +940,17 @@ class H(BaseHTTPRequestHandler):
path=urllib.parse.urlparse(self.path).path.rstrip('/')
if reject_non_read_http(self, 'POST', path): return
data=self.body()
# Governed V2 bridge release routes (previously shadowed by the
# redefinition of do_POST in this class).
if path in {
'/createTagApprovalRequest', '/requestOrCreateTag', '/getTagPublicationStatus',
'/buildProductionArtifact', '/executeApprovedProductionRelease',
'/getReleaseConsistency', '/runReleaseAcceptance',
}:
actor = self.need('tag:create')
if not actor:
return
return self.bridge_release(path, self.body())
try:
if path == '/createReleaseCandidate':
a=self.need('release:candidate')
@@ -925,6 +977,17 @@ class H(BaseHTTPRequestHandler):
return self.sendj(
silent_release_service().submit_owner_approval(data)
)
if path == "/reopenOwnerApproval":
a = self.need("release:approve")
if not a:
return
from release_approval_reopen import ApprovalReopenError, reopen_owner_approval
try:
result = reopen_owner_approval(silent_release_service(), data)
except ApprovalReopenError as exc:
return self.sendj({"error": exc.code, "message": str(exc)}, 409)
return self.sendj(result)
if path == '/startDeployment':
a=self.need('release:deploy')
if not a: return
@@ -1187,12 +1250,25 @@ return{vs,cmds,ar,cmd,cdet,tot,sok,sfail,runs,baseline,refreshMsg,loadCmd,selVer
commands.append(f.name)
command_detail=None
if cmd_filter:
actor = self.actor()
if not actor['ok']:
return self.sendj({'error':'unauthorized','hint':'Bearer token required'},401)
cmd = str(cmd_filter or '')
if not _COMMAND_FILENAME_RE.fullmatch(cmd) or '..' in cmd:
return self.sendj({'command':cmd_filter,'error':'unsafe_command'},400)
for d in cmd_dirs:
if d.exists():
fp=d/cmd_filter
if fp.exists():
command_detail={'command':cmd_filter,'content':fp.read_text(encoding='utf-8')}
break
if not d.exists():
continue
base = d.resolve()
fp = (base / cmd).resolve()
# Strict containment: must resolve directly inside one of the
# allowed command directories.
if fp.parent != base or not fp.is_file():
continue
if fp.stat().st_size > MAX_COMMAND_READ_BYTES:
return self.sendj({'command':cmd_filter,'error':'command_too_large'},413)
command_detail={'command':cmd_filter,'content':fp.read_text(encoding='utf-8')}
break
if not command_detail:
command_detail={'command':cmd_filter,'error':'not_found'}
return self.sendj(command_detail)
@@ -438,6 +438,71 @@ class TaskAdmissionStore:
).fetchone()
return dict(row) if row else None
def get_grant_by_id(self, grant_id: str) -> TaskAdmissionGrant | None:
"""Return one immutable admitted grant by its stable grant identity.
This is a read-only lookup used by the server-side TaskPrincipal
recovery path. It never trusts caller-supplied grant content.
"""
if not isinstance(grant_id, str) or not grant_id:
raise TaskAdmissionError("TASK_ADMISSION_GRANT_ID_INVALID", 400)
with self._connection() as con:
row = con.execute(
"SELECT grant_json FROM task_admission_grants WHERE grant_id=?",
(grant_id,),
).fetchone()
if row is None:
return None
try:
raw = json.loads(row["grant_json"])
return TaskAdmissionGrant(
**{
**raw,
"capabilities": tuple(raw["capabilities"]),
"file_allowlist": tuple(raw["file_allowlist"]),
}
)
except (TypeError, ValueError, KeyError, json.JSONDecodeError) as exc:
raise TaskAdmissionError(
"TASK_ADMISSION_DURABLE_GRANT_INVALID", 503
) from exc
def get_grant_by_task_id(self, task_id: str) -> TaskAdmissionGrant | None:
"""Return the immutable admitted grant bound to an authoritative Task."""
if not isinstance(task_id, str) or not task_id:
raise TaskAdmissionError("TASK_ADMISSION_TASK_ID_INVALID", 400)
with self._connection() as con:
row = con.execute(
"""
SELECT g.grant_json
FROM task_admission_grants AS g
JOIN task_admission_receipts AS r
ON r.envelope_id = g.envelope_id
WHERE r.task_id = ?
""",
(task_id,),
).fetchone()
if row is None:
return None
try:
raw = json.loads(row["grant_json"])
grant = TaskAdmissionGrant(
**{
**raw,
"capabilities": tuple(raw["capabilities"]),
"file_allowlist": tuple(raw["file_allowlist"]),
}
)
except (TypeError, ValueError, KeyError, json.JSONDecodeError) as exc:
raise TaskAdmissionError(
"TASK_ADMISSION_DURABLE_GRANT_INVALID", 503
) from exc
if grant.task_id != task_id:
raise TaskAdmissionError(
"TASK_ADMISSION_DURABLE_GRANT_TASK_MISMATCH", 503
)
return grant
def save_envelope(
self,
*,
@@ -864,10 +929,23 @@ class TaskAdmissionBootstrapService:
trust_domain=envelope.trust_domain, policy_digest=envelope.policy_digest,
authorization_kind=kind, owner_identity=owner_identity,
parent_epoch_id=parent_epoch_id, issued_at=now,
expires_at=min(
envelope.expires_at,
now + self.policy.max_task_ttl_seconds,
parent_epoch.expires_at if parent_epoch is not None else envelope.expires_at,
# V2R2C2 TTL contract: the admission envelope is a *signing
# window* only. Once authorization succeeds (epoch verified or
# Owner signature verified) the envelope is consumed, and the
# derived grant's lifecycle is bounded by the task/grant policy
# ceiling (max_task_ttl_seconds) and, for a delegation epoch,
# by the parent epoch's own expiry. It must NOT be truncated by
# the already-consumed envelope's remaining window, otherwise a
# nominal 3600s policy collapses to the ~900s envelope window
# (dual-semantic TTL defect). The envelope's temporal validity
# still gates authorization (now < envelope.expires_at above).
expires_at=(
now + self.policy.max_task_ttl_seconds
if parent_epoch is None
else min(
now + self.policy.max_task_ttl_seconds,
parent_epoch.expires_at,
)
),
)
persisted_grant, _ = self.store.claim_grant(
File diff suppressed because it is too large Load Diff
+779
View File
@@ -0,0 +1,779 @@
from __future__ import annotations
from dataclasses import dataclass, field, replace
from datetime import datetime, timezone
from enum import Enum
from hashlib import sha256
import re
from typing import Callable, Iterable, Mapping, Protocol, Sequence
class LineageError(ValueError):
pass
class RelationType(str, Enum):
ROOT = "ROOT"
CHILD = "CHILD"
SUCCESSOR = "SUCCESSOR"
RECOVERY = "RECOVERY"
class NextLegalAction(str, Enum):
ROOT_COMPLETE = "ROOT_COMPLETE"
CREATE_OR_RESUME_NEXT_CHILD = "CREATE_OR_RESUME_NEXT_CHILD"
WAIT_DEPENDENCY = "WAIT_DEPENDENCY"
RECOVER_EXISTING_CHILD = "RECOVER_EXISTING_CHILD"
OWNER_BOUNDARY = "OWNER_BOUNDARY"
TERMINAL_STATUSES = frozenset(
{"COMPLETE", "COMPLETED", "SUCCEEDED", "FAILED", "FAILED_FINAL", "CANCELLED"}
)
RECOVERY_STATUSES = frozenset(
{"RECONCILIATION_REQUIRED", "RECOVERY_REQUIRED", "FAILED_RECOVERABLE"}
)
BLOCKED_STATUSES = frozenset({"BLOCKED", "WAITING_DEPENDENCY", "WAITING_CAPACITY"})
_ALIAS_RE = re.compile(r"^[A-Z0-9][A-Z0-9_-]{1,31}$")
_SLUG_RE = re.compile(r"^[A-Z0-9][A-Z0-9-]{1,63}$")
def _require_utc(ts: datetime) -> datetime:
if ts.tzinfo is None or ts.utcoffset() is None:
raise LineageError("created_at/reviewed_at must be timezone-aware")
return ts.astimezone(timezone.utc)
def derive_safe_owner_alias(principal_id: str, preferred_alias: str | None = None) -> str:
"""Return a non-secret display alias.
A trusted identity adapter may supply a safe human alias. Otherwise we use a
deterministic pseudonym derived from the principal identifier. We never put
raw email addresses, tokens, or principal strings into task names.
"""
if preferred_alias and "@" not in preferred_alias and ":" not in preferred_alias:
candidate = re.sub(r"[^A-Za-z0-9_-]+", "-", preferred_alias.strip()).strip("-_").upper()
if 2 <= len(candidate) <= 32 and _ALIAS_RE.fullmatch(candidate):
return candidate
if not principal_id:
raise LineageError("principal_id is required")
return "U" + sha256(principal_id.encode("utf-8")).hexdigest()[:9].upper()
def normalize_goal_slug(value: str) -> str:
value = re.sub(r"[^A-Za-z0-9]+", "-", value.strip()).strip("-").upper()
value = re.sub(r"-+", "-", value)
if len(value) > 64:
value = value[:64].rstrip("-")
if not _SLUG_RE.fullmatch(value):
raise LineageError("goal_slug must normalize to 2-64 [A-Z0-9-] characters")
return value
def short_ref(task_id: str) -> str:
if not task_id:
raise LineageError("task_id is required")
compact = re.sub(r"[^A-Za-z0-9]", "", task_id).upper()
if len(compact) >= 8:
return compact[-8:]
return sha256(task_id.encode("utf-8")).hexdigest()[:8].upper()
@dataclass(frozen=True)
class TaskLineage:
task_id: str
display_name: str
owner_principal_id: str
owner_alias: str
created_at: datetime
mission_id: str
root_task_id: str
parent_task_id: str | None
lineage_depth: int
relation_type: RelationType
sibling_sequence: int
goal_slug: str
root_acceptance_ref: str
parent_revision: int | None = None
@dataclass(frozen=True)
class TaskRecord:
task_id: str
revision: int
status: str
lineage: TaskLineage
evidence_refs: tuple[str, ...] = ()
terminal_evidence_proven: bool = False
@dataclass(frozen=True)
class AcceptanceCondition:
key: str
satisfied: bool
evidence_refs: tuple[str, ...] = ()
owner_boundary: bool = False
@dataclass(frozen=True)
class RootAcceptanceSnapshot:
conditions: tuple[AcceptanceCondition, ...]
@property
def remaining(self) -> tuple[str, ...]:
return tuple(c.key for c in self.conditions if not c.satisfied)
@property
def complete(self) -> bool:
return all(c.satisfied for c in self.conditions)
@property
def owner_boundary_required(self) -> bool:
return any((not c.satisfied) and c.owner_boundary for c in self.conditions)
@property
def evidence_refs(self) -> tuple[str, ...]:
values: list[str] = []
for condition in self.conditions:
if condition.satisfied:
values.extend(condition.evidence_refs)
return tuple(dict.fromkeys(values))
def canonicalize_acceptance(
acceptance: RootAcceptanceSnapshot,
) -> RootAcceptanceSnapshot:
if not isinstance(acceptance, RootAcceptanceSnapshot):
raise LineageError("acceptance evaluator returned an invalid snapshot")
normalized: list[AcceptanceCondition] = []
keys: set[str] = set()
for condition in acceptance.conditions:
if not isinstance(condition, AcceptanceCondition):
raise LineageError("acceptance snapshot contains an invalid condition")
if not isinstance(condition.key, str) or not condition.key:
raise LineageError("acceptance condition key must be a non-empty string")
if condition.key in keys:
raise LineageError("acceptance condition keys must be unique")
if not isinstance(condition.satisfied, bool) or not isinstance(
condition.owner_boundary, bool
):
raise LineageError("acceptance condition flags must be booleans")
if not all(isinstance(ref, str) and ref for ref in condition.evidence_refs):
raise LineageError("acceptance evidence refs must be non-empty strings")
keys.add(condition.key)
normalized.append(
replace(
condition,
evidence_refs=tuple(sorted(set(condition.evidence_refs))),
)
)
return RootAcceptanceSnapshot(tuple(sorted(normalized, key=lambda c: c.key)))
@dataclass(frozen=True)
class RootReviewRecord:
review_id: str
root_task_id: str
reviewed_at: datetime
trigger_child_task_id: str
root_revision_before: int
root_revision_after: int
completed_children: tuple[str, ...]
active_children: tuple[str, ...]
blocked_children: tuple[str, ...]
remaining_acceptance_conditions: tuple[str, ...]
next_legal_action: NextLegalAction
root_complete: bool
evidence_refs: tuple[str, ...]
class RootReviewStore(Protocol):
def get_task(self, task_id: str) -> TaskRecord: ...
def list_descendants(self, root_task_id: str) -> Sequence[TaskRecord]: ...
def get_root_review(self, review_id: str) -> RootReviewRecord | None: ...
def compare_and_swap_root_review(
self,
root_task_id: str,
expected_root_revision: int,
review: RootReviewRecord,
) -> bool: ...
AcceptanceEvaluator = Callable[[TaskRecord, Sequence[TaskRecord]], RootAcceptanceSnapshot]
def _timestamp(ts: datetime) -> str:
return _require_utc(ts).strftime("%Y%m%dT%H%M%SZ")
def _relation_token(relation: RelationType, sequence: int) -> str:
if sequence < 1 or sequence > 99:
raise LineageError("non-root sibling_sequence must be 1..99")
prefix = {
RelationType.CHILD: "C",
RelationType.SUCCESSOR: "S",
RelationType.RECOVERY: "X",
}.get(relation)
if not prefix:
raise LineageError("ROOT does not use a child relation token")
return f"{prefix}{sequence:02d}"
def build_display_name(lineage: TaskLineage) -> str:
alias = lineage.owner_alias.upper()
if not _ALIAS_RE.fullmatch(alias):
raise LineageError("owner_alias must be a non-secret [A-Z0-9_-] alias")
goal = normalize_goal_slug(lineage.goal_slug)
ts = _timestamp(lineage.created_at)
if lineage.relation_type == RelationType.ROOT:
return f"TASK-{alias}-{ts}-ROOT-{goal}-{short_ref(lineage.task_id)[:4]}"
relation = _relation_token(lineage.relation_type, lineage.sibling_sequence)
return (
f"TASK-{alias}-{ts}"
f"-R{short_ref(lineage.root_task_id)}"
f"-P{short_ref(lineage.parent_task_id or '')}"
f"-D{lineage.lineage_depth:02d}"
f"-{relation}-{goal}"
)
def make_root_lineage(
*,
task_id: str,
owner_principal_id: str,
owner_alias: str,
created_at: datetime,
mission_id: str,
goal_slug: str,
root_acceptance_ref: str,
) -> TaskLineage:
lineage = TaskLineage(
task_id=task_id,
display_name="",
owner_principal_id=owner_principal_id,
owner_alias=owner_alias.upper(),
created_at=_require_utc(created_at),
mission_id=mission_id,
root_task_id=task_id,
parent_task_id=None,
lineage_depth=0,
relation_type=RelationType.ROOT,
sibling_sequence=0,
goal_slug=normalize_goal_slug(goal_slug),
root_acceptance_ref=root_acceptance_ref,
parent_revision=None,
)
lineage = replace(lineage, display_name=build_display_name(lineage))
validate_lineage(lineage)
return lineage
def make_child_lineage(
*,
task_id: str,
created_at: datetime,
parent: TaskRecord,
root: TaskRecord,
relation_type: RelationType,
sibling_sequence: int,
goal_slug: str,
) -> TaskLineage:
if relation_type == RelationType.ROOT:
raise LineageError("child relation_type cannot be ROOT")
lineage = TaskLineage(
task_id=task_id,
display_name="",
owner_principal_id=root.lineage.owner_principal_id,
owner_alias=root.lineage.owner_alias,
created_at=_require_utc(created_at),
mission_id=root.lineage.mission_id,
root_task_id=root.task_id,
parent_task_id=parent.task_id,
lineage_depth=parent.lineage.lineage_depth + 1,
relation_type=relation_type,
sibling_sequence=sibling_sequence,
goal_slug=normalize_goal_slug(goal_slug),
root_acceptance_ref=root.lineage.root_acceptance_ref,
parent_revision=parent.revision,
)
lineage = replace(lineage, display_name=build_display_name(lineage))
validate_lineage(lineage, parent=parent, root=root)
return lineage
def validate_lineage(
lineage: TaskLineage,
*,
parent: TaskRecord | None = None,
root: TaskRecord | None = None,
) -> None:
_validate_lineage_shape(lineage)
if lineage.relation_type == RelationType.ROOT:
return
if parent is None or root is None:
raise LineageError("non-root validation requires parent and root authority records")
if root.lineage.relation_type != RelationType.ROOT:
raise LineageError("root authority record must have ROOT lineage")
if parent.lineage.root_task_id != root.task_id:
raise LineageError("parent is not attached to the selected root")
if parent.lineage.owner_principal_id != root.lineage.owner_principal_id:
raise LineageError("parent owner does not match root owner")
if parent.lineage.mission_id != root.lineage.mission_id:
raise LineageError("parent mission does not match root mission")
if parent.lineage.root_acceptance_ref != root.lineage.root_acceptance_ref:
raise LineageError("parent acceptance reference does not match root")
if lineage.root_task_id != root.task_id:
raise LineageError("root_task_id mismatch")
if lineage.parent_task_id != parent.task_id:
raise LineageError("parent_task_id mismatch")
if lineage.lineage_depth != parent.lineage.lineage_depth + 1:
raise LineageError("lineage_depth must equal parent depth + 1")
if lineage.owner_principal_id != root.lineage.owner_principal_id:
raise LineageError("cross-owner lineage attachment rejected")
if lineage.owner_alias != root.lineage.owner_alias:
raise LineageError("owner_alias must match root")
if lineage.mission_id != root.lineage.mission_id:
raise LineageError("mission_id must match root")
if lineage.root_acceptance_ref != root.lineage.root_acceptance_ref:
raise LineageError("root_acceptance_ref must match root")
if lineage.parent_revision != parent.revision:
raise LineageError("stale or mismatched parent revision")
def _validate_lineage_shape(lineage: TaskLineage) -> None:
if not lineage.task_id or not lineage.owner_principal_id or not lineage.mission_id:
raise LineageError("task_id, owner_principal_id and mission_id are required")
if not lineage.root_acceptance_ref:
raise LineageError("root_acceptance_ref is required")
if not _ALIAS_RE.fullmatch(lineage.owner_alias):
raise LineageError("invalid owner_alias")
if normalize_goal_slug(lineage.goal_slug) != lineage.goal_slug:
raise LineageError("goal_slug must already be normalized")
if not lineage.display_name or len(lineage.display_name) > 200:
raise LineageError("display_name must be non-empty and <= 200 characters")
if lineage.relation_type == RelationType.ROOT:
if lineage.root_task_id != lineage.task_id:
raise LineageError("root_task_id must equal task_id for ROOT")
if lineage.parent_task_id is not None:
raise LineageError("ROOT parent_task_id must be null")
if lineage.lineage_depth != 0:
raise LineageError("ROOT lineage_depth must be 0")
if lineage.sibling_sequence != 0:
raise LineageError("ROOT sibling_sequence must be 0")
if lineage.parent_revision is not None:
raise LineageError("ROOT parent_revision must be null")
return
if not lineage.root_task_id or not lineage.parent_task_id:
raise LineageError("non-root root_task_id and parent_task_id are required")
if lineage.lineage_depth < 1:
raise LineageError("non-root lineage_depth must be >= 1")
if lineage.parent_revision is None or lineage.parent_revision < 0:
raise LineageError("non-root parent_revision must be >= 0")
_relation_token(lineage.relation_type, lineage.sibling_sequence)
def attach_legacy_lineage(
*,
existing_task_id: str,
created_at: datetime,
parent: TaskRecord,
root: TaskRecord,
relation_type: RelationType,
sibling_sequence: int,
goal_slug: str,
) -> TaskLineage:
"""Return lineage metadata for an existing durable task without changing its ID."""
return make_child_lineage(
task_id=existing_task_id,
created_at=created_at,
parent=parent,
root=root,
relation_type=relation_type,
sibling_sequence=sibling_sequence,
goal_slug=goal_slug,
)
def rename_display_only(lineage: TaskLineage, new_display_name: str) -> TaskLineage:
if not new_display_name or len(new_display_name) > 200:
raise LineageError("display name must be non-empty and <= 200 characters")
return replace(lineage, display_name=new_display_name)
LINEAGE_SCHEMA = "aios.task-lineage.v1"
ROOT_REVIEW_SCHEMA = "aios.root-review.v1"
_LINEAGE_FIELDS = frozenset(
{
"schema",
"task_id",
"display_name",
"owner_principal_id",
"owner_alias",
"created_at",
"mission_id",
"root_task_id",
"parent_task_id",
"lineage_depth",
"relation_type",
"sibling_sequence",
"goal_slug",
"root_acceptance_ref",
"parent_revision",
}
)
_ROOT_REVIEW_FIELDS = frozenset(
{
"schema",
"review_id",
"root_task_id",
"reviewed_at",
"trigger_child_task_id",
"root_revision_before",
"root_revision_after",
"completed_children",
"active_children",
"blocked_children",
"remaining_acceptance_conditions",
"next_legal_action",
"root_complete",
"evidence_refs",
}
)
def _require_exact_fields(metadata: Mapping[str, object], expected: frozenset[str]) -> None:
if not all(isinstance(key, str) for key in metadata):
raise LineageError("metadata field names must be strings")
actual = frozenset(metadata)
if actual != expected:
missing = sorted(expected - actual)
extra = sorted(actual - expected)
raise LineageError(f"metadata fields mismatch: missing={missing} extra={extra}")
def _require_string(metadata: Mapping[str, object], key: str) -> str:
value = metadata[key]
if not isinstance(value, str) or not value:
raise LineageError(f"{key} must be a non-empty string")
return value
def _require_int(metadata: Mapping[str, object], key: str) -> int:
value = metadata[key]
if isinstance(value, bool) or not isinstance(value, int):
raise LineageError(f"{key} must be an integer")
return value
def _require_string_list(metadata: Mapping[str, object], key: str) -> tuple[str, ...]:
value = metadata[key]
if not isinstance(value, list) or not all(isinstance(item, str) and item for item in value):
raise LineageError(f"{key} must be a list of non-empty strings")
items = tuple(value)
if len(items) != len(set(items)):
raise LineageError(f"{key} must not contain duplicates")
return items
def lineage_to_metadata(lineage: TaskLineage) -> dict[str, object]:
return {
"schema": LINEAGE_SCHEMA,
"task_id": lineage.task_id,
"display_name": lineage.display_name,
"owner_principal_id": lineage.owner_principal_id,
"owner_alias": lineage.owner_alias,
"created_at": _require_utc(lineage.created_at).isoformat().replace("+00:00", "Z"),
"mission_id": lineage.mission_id,
"root_task_id": lineage.root_task_id,
"parent_task_id": lineage.parent_task_id,
"lineage_depth": lineage.lineage_depth,
"relation_type": lineage.relation_type.value,
"sibling_sequence": lineage.sibling_sequence,
"goal_slug": lineage.goal_slug,
"root_acceptance_ref": lineage.root_acceptance_ref,
"parent_revision": lineage.parent_revision,
}
def lineage_from_metadata(metadata: Mapping[str, object]) -> TaskLineage:
_require_exact_fields(metadata, _LINEAGE_FIELDS)
if metadata.get("schema") != LINEAGE_SCHEMA:
raise LineageError("unsupported lineage metadata schema")
try:
created_raw = _require_string(metadata, "created_at")
if created_raw.endswith("Z"):
created_raw = created_raw[:-1] + "+00:00"
parent_task_id = metadata["parent_task_id"]
if parent_task_id is not None and (
not isinstance(parent_task_id, str) or not parent_task_id
):
raise LineageError("parent_task_id must be null or a non-empty string")
parent_revision = metadata["parent_revision"]
if parent_revision is not None and (
isinstance(parent_revision, bool) or not isinstance(parent_revision, int)
):
raise LineageError("parent_revision must be null or an integer")
lineage = TaskLineage(
task_id=_require_string(metadata, "task_id"),
display_name=_require_string(metadata, "display_name"),
owner_principal_id=_require_string(metadata, "owner_principal_id"),
owner_alias=_require_string(metadata, "owner_alias"),
created_at=datetime.fromisoformat(created_raw),
mission_id=_require_string(metadata, "mission_id"),
root_task_id=_require_string(metadata, "root_task_id"),
parent_task_id=parent_task_id,
lineage_depth=_require_int(metadata, "lineage_depth"),
relation_type=RelationType(_require_string(metadata, "relation_type")),
sibling_sequence=_require_int(metadata, "sibling_sequence"),
goal_slug=_require_string(metadata, "goal_slug"),
root_acceptance_ref=_require_string(metadata, "root_acceptance_ref"),
parent_revision=parent_revision,
)
except (KeyError, TypeError, ValueError, LineageError) as exc:
raise LineageError(f"invalid lineage metadata: {exc}") from exc
_validate_lineage_shape(lineage)
return lineage
def root_review_to_record(review: RootReviewRecord) -> dict[str, object]:
return {
"schema": ROOT_REVIEW_SCHEMA,
"review_id": review.review_id,
"root_task_id": review.root_task_id,
"reviewed_at": _require_utc(review.reviewed_at).isoformat().replace("+00:00", "Z"),
"trigger_child_task_id": review.trigger_child_task_id,
"root_revision_before": review.root_revision_before,
"root_revision_after": review.root_revision_after,
"completed_children": list(review.completed_children),
"active_children": list(review.active_children),
"blocked_children": list(review.blocked_children),
"remaining_acceptance_conditions": list(review.remaining_acceptance_conditions),
"next_legal_action": review.next_legal_action.value,
"root_complete": review.root_complete,
"evidence_refs": list(review.evidence_refs),
}
def root_review_from_record(
record: Mapping[str, object], *, expected_review_id: str | None = None
) -> RootReviewRecord:
_require_exact_fields(record, _ROOT_REVIEW_FIELDS)
if record.get("schema") != ROOT_REVIEW_SCHEMA:
raise LineageError("unsupported root-review metadata schema")
try:
reviewed_raw = _require_string(record, "reviewed_at")
if reviewed_raw.endswith("Z"):
reviewed_raw = reviewed_raw[:-1] + "+00:00"
root_complete = record["root_complete"]
if not isinstance(root_complete, bool):
raise LineageError("root_complete must be a boolean")
review = RootReviewRecord(
review_id=_require_string(record, "review_id"),
root_task_id=_require_string(record, "root_task_id"),
reviewed_at=_require_utc(datetime.fromisoformat(reviewed_raw)),
trigger_child_task_id=_require_string(record, "trigger_child_task_id"),
root_revision_before=_require_int(record, "root_revision_before"),
root_revision_after=_require_int(record, "root_revision_after"),
completed_children=_require_string_list(record, "completed_children"),
active_children=_require_string_list(record, "active_children"),
blocked_children=_require_string_list(record, "blocked_children"),
remaining_acceptance_conditions=_require_string_list(
record, "remaining_acceptance_conditions"
),
next_legal_action=NextLegalAction(
_require_string(record, "next_legal_action")
),
root_complete=root_complete,
evidence_refs=_require_string_list(record, "evidence_refs"),
)
except (KeyError, TypeError, ValueError, LineageError) as exc:
raise LineageError(f"invalid root-review metadata: {exc}") from exc
if expected_review_id is not None and review.review_id != expected_review_id:
raise LineageError("root-review id does not match lookup key")
if review.root_revision_before < 0 or (
review.root_revision_after != review.root_revision_before + 1
):
raise LineageError("root-review revision transition is invalid")
if review.root_complete != (
review.next_legal_action == NextLegalAction.ROOT_COMPLETE
):
raise LineageError("root_complete does not match next_legal_action")
child_sets = (
set(review.completed_children),
set(review.active_children),
set(review.blocked_children),
)
if any(child_sets[i] & child_sets[j] for i in range(3) for j in range(i + 1, 3)):
raise LineageError("root-review child classifications overlap")
return review
def _review_id(
root_task_id: str,
trigger_child_task_id: str,
trigger_revision: int,
acceptance: RootAcceptanceSnapshot,
) -> str:
acceptance_digest = sha256(
"\n".join(
f"{c.key}:{int(c.satisfied)}:{int(c.owner_boundary)}:"
f"{','.join(sorted(set(c.evidence_refs)))}"
for c in acceptance.conditions
).encode("utf-8")
).hexdigest()
payload = f"{root_task_id}|{trigger_child_task_id}|{trigger_revision}|{acceptance_digest}"
return "RR-" + sha256(payload.encode("utf-8")).hexdigest()[:24]
def _next_action(
descendants: Sequence[TaskRecord],
acceptance: RootAcceptanceSnapshot,
) -> NextLegalAction:
if acceptance.complete:
return NextLegalAction.ROOT_COMPLETE
if acceptance.owner_boundary_required:
return NextLegalAction.OWNER_BOUNDARY
if any(d.status in RECOVERY_STATUSES for d in descendants):
return NextLegalAction.RECOVER_EXISTING_CHILD
if any(d.status in BLOCKED_STATUSES for d in descendants):
return NextLegalAction.WAIT_DEPENDENCY
return NextLegalAction.CREATE_OR_RESUME_NEXT_CHILD
def review_root_after_child_terminal(
*,
store: RootReviewStore,
trigger_child_task_id: str,
acceptance_evaluator: AcceptanceEvaluator,
reviewed_at: datetime,
max_cas_retries: int = 4,
) -> RootReviewRecord:
"""Persist a root review after a proven child terminal transition.
This function is deliberately storage-agnostic. TaskPlane remains the
authority and supplies the CAS implementation. The function never creates a
new root or mutates a child identity.
"""
if max_cas_retries < 1:
raise LineageError("max_cas_retries must be >= 1")
reviewed_at = _require_utc(reviewed_at)
for _ in range(max_cas_retries):
child = store.get_task(trigger_child_task_id)
if child.lineage.relation_type == RelationType.ROOT:
raise LineageError("root review hook requires a child task")
if child.status not in TERMINAL_STATUSES:
raise LineageError("child must be terminal before root review")
if not child.terminal_evidence_proven:
raise LineageError("child terminal evidence is not proven")
root = store.get_task(child.lineage.root_task_id)
if root.lineage.relation_type != RelationType.ROOT:
raise LineageError("root_task_id does not resolve to a ROOT task")
if child.lineage.owner_principal_id != root.lineage.owner_principal_id:
raise LineageError("trigger child owner does not match root owner")
if child.lineage.mission_id != root.lineage.mission_id:
raise LineageError("trigger child mission does not match root mission")
if child.lineage.root_acceptance_ref != root.lineage.root_acceptance_ref:
raise LineageError("trigger child acceptance reference does not match root")
descendants = tuple(
sorted(store.list_descendants(root.task_id), key=lambda item: item.task_id)
)
descendant_ids = tuple(item.task_id for item in descendants)
if len(descendant_ids) != len(set(descendant_ids)):
raise LineageError("descendant authority returned duplicate task ids")
if child.task_id not in descendant_ids:
raise LineageError("trigger child missing from authoritative descendants")
for descendant in descendants:
if descendant.lineage.root_task_id != root.task_id:
raise LineageError("descendant is attached to a different root")
if descendant.lineage.owner_principal_id != root.lineage.owner_principal_id:
raise LineageError("descendant owner does not match root owner")
if descendant.lineage.mission_id != root.lineage.mission_id:
raise LineageError("descendant mission does not match root mission")
if (
descendant.lineage.root_acceptance_ref
!= root.lineage.root_acceptance_ref
):
raise LineageError("descendant acceptance reference does not match root")
acceptance = canonicalize_acceptance(acceptance_evaluator(root, descendants))
review_id = _review_id(root.task_id, child.task_id, child.revision, acceptance)
completed = tuple(
d.task_id for d in descendants
if d.status in TERMINAL_STATUSES and d.terminal_evidence_proven
)
active = tuple(
d.task_id for d in descendants
if d.status not in TERMINAL_STATUSES and d.status not in BLOCKED_STATUSES
)
blocked = tuple(
d.task_id for d in descendants
if d.status in BLOCKED_STATUSES
)
evidence = tuple(sorted(set(child.evidence_refs + acceptance.evidence_refs)))
action = _next_action(descendants, acceptance)
existing = store.get_root_review(review_id)
if existing is not None:
if (
existing.root_task_id != root.task_id
or existing.trigger_child_task_id != child.task_id
):
raise LineageError("persisted root review identity conflicts with trigger")
if existing.root_revision_after > root.revision:
raise LineageError("persisted root review revision is ahead of authority")
expected_semantics = (
completed,
active,
blocked,
acceptance.remaining,
action,
action == NextLegalAction.ROOT_COMPLETE,
evidence,
)
actual_semantics = (
existing.completed_children,
existing.active_children,
existing.blocked_children,
existing.remaining_acceptance_conditions,
existing.next_legal_action,
existing.root_complete,
existing.evidence_refs,
)
if actual_semantics != expected_semantics:
raise LineageError("persisted root review conflicts with current semantics")
return existing
review = RootReviewRecord(
review_id=review_id,
root_task_id=root.task_id,
reviewed_at=reviewed_at,
trigger_child_task_id=child.task_id,
root_revision_before=root.revision,
root_revision_after=root.revision + 1,
completed_children=completed,
active_children=active,
blocked_children=blocked,
remaining_acceptance_conditions=acceptance.remaining,
next_legal_action=action,
root_complete=(action == NextLegalAction.ROOT_COMPLETE),
evidence_refs=evidence,
)
if store.compare_and_swap_root_review(root.task_id, root.revision, review):
return review
# Another sibling may have updated root progress. Re-read authority and
# recompute; never overwrite a newer root review.
raise LineageError("root review CAS retry budget exhausted")
@@ -0,0 +1,285 @@
from __future__ import annotations
from dataclasses import dataclass, replace
from datetime import datetime
from typing import Mapping
if not __package__:
from aios_task_lineage import AcceptanceEvaluator, RelationType, TaskLineage, lineage_to_metadata, lineage_from_metadata, root_review_to_record
from aios_task_lineage_integration import (
DISPLAY_NAME_METADATA_KEY,
IntegrationError,
LINEAGE_METADATA_KEY,
NewTaskIdentityRequest,
PreparedTaskIdentity,
TaskIdentityLineageService,
TerminalReviewOutcome,
)
from aios_task_lineage_taskplane_adapter import (
AuthenticatedTaskContext,
InProcessTaskPlaneLineageAdapter,
require_context_matches_task,
)
else:
from .aios_task_lineage import AcceptanceEvaluator, RelationType, TaskLineage, lineage_to_metadata, lineage_from_metadata, root_review_to_record
from .aios_task_lineage_integration import (
DISPLAY_NAME_METADATA_KEY,
IntegrationError,
LINEAGE_METADATA_KEY,
NewTaskIdentityRequest,
PreparedTaskIdentity,
TaskIdentityLineageService,
TerminalReviewOutcome,
)
from .aios_task_lineage_taskplane_adapter import (
AuthenticatedTaskContext,
InProcessTaskPlaneLineageAdapter,
require_context_matches_task,
)
class LifecycleHookError(IntegrationError):
pass
@dataclass(frozen=True)
class IdentityPersistenceOutcome:
prepared: PreparedTaskIdentity
persisted_revision_at_least: int
independently_verified: bool
@dataclass(frozen=True)
class LegacyAttachOutcome:
lineage: TaskLineage
independently_verified: bool
@dataclass(frozen=True)
class TerminalHookOutcome:
outcome: TerminalReviewOutcome
root_review_persisted: bool
dispatch_persisted: bool
independently_verified: bool
class TaskLineageLifecycleHooks:
"""Concrete lifecycle hook coordinator.
It does not own TaskPlane state. It orchestrates existing TaskPlane CAS,
journal/continuation and optional independent authority readback.
"""
def __init__(self, adapter: InProcessTaskPlaneLineageAdapter):
self._adapter = adapter
self._service = TaskIdentityLineageService(adapter)
def persist_new_task_identity(
self,
*,
request: NewTaskIdentityRequest,
context: AuthenticatedTaskContext,
max_cas_retries: int = 4,
require_independent_readback: bool = True,
) -> IdentityPersistenceOutcome:
if max_cas_retries < 1:
raise LifecycleHookError("max_cas_retries must be >= 1")
if require_independent_readback and not self._adapter.has_independent_readback:
raise LifecycleHookError("independent TaskPlane readback is not configured")
if request.owner_principal_id != context.principal_id:
raise LifecycleHookError("request principal does not match authenticated context")
if request.mission_id != context.mission_id:
raise LifecycleHookError("request mission does not match authenticated context")
for _ in range(max_cas_retries):
current = self._adapter.get_task_snapshot(request.task_id)
require_context_matches_task(context, current)
if current.created_at != request.created_at:
raise LifecycleHookError("request created_at does not match authority")
if (request.relation_type == RelationType.ROOT
and request.preferred_owner_alias not in (None, context.preferred_alias)):
raise LifecycleHookError("request alias does not match authenticated context")
request = replace(request, preferred_owner_alias=context.preferred_alias)
prepared = self._service.prepare_new_task_identity(request)
# Idempotent replay and legacy display-name repair are fail-closed:
# never overwrite an existing conflicting lineage record.
raw = current.metadata.get(LINEAGE_METADATA_KEY)
display = current.metadata.get(DISPLAY_NAME_METADATA_KEY)
if raw is not None and not isinstance(raw, Mapping):
raise LifecycleHookError("existing lineage metadata is malformed")
if isinstance(raw, Mapping):
# parent_revision records the creation observation, not the
# parent's current progress revision on an idempotent replay.
expected = dict(prepared.metadata_patch[LINEAGE_METADATA_KEY])
if request.parent_task_id is not None:
stored = lineage_from_metadata(dict(raw))
if (stored.parent_revision is None
or stored.parent_revision > prepared.lineage.parent_revision):
raise LifecycleHookError("existing parent revision conflicts with authority")
expected["parent_revision"] = stored.parent_revision
if raw != expected:
raise LifecycleHookError(
"existing lineage metadata conflicts with requested identity"
)
prepared = replace(prepared, lineage=lineage_from_metadata(dict(raw)),
metadata_patch={LINEAGE_METADATA_KEY: dict(raw),
DISPLAY_NAME_METADATA_KEY: prepared.display_name})
if display == prepared.display_name:
verified = False
if require_independent_readback:
self._adapter.verify_task_readback(
task_id=request.task_id,
expected_revision_at_least=current.revision,
expected_metadata=prepared.metadata_patch,
)
verified = True
return IdentityPersistenceOutcome(
prepared=prepared,
persisted_revision_at_least=current.revision,
independently_verified=verified,
)
if display not in (None, ""):
raise LifecycleHookError(
"existing display name conflicts with requested identity"
)
metadata_patch = {
DISPLAY_NAME_METADATA_KEY: prepared.display_name,
}
else:
if display not in (None, ""):
raise LifecycleHookError(
"display name exists without authoritative lineage"
)
metadata_patch = prepared.metadata_patch
ok = self._adapter.compare_and_swap_task_metadata(
task_id=request.task_id,
expected_revision=current.revision,
metadata_patch=metadata_patch,
)
if not ok:
continue
verified = False
if require_independent_readback:
self._adapter.verify_task_readback(
task_id=request.task_id,
expected_revision_at_least=current.revision + 1,
expected_metadata=prepared.metadata_patch,
)
verified = True
return IdentityPersistenceOutcome(
prepared=prepared,
persisted_revision_at_least=current.revision + 1,
independently_verified=verified,
)
raise LifecycleHookError("identity persistence CAS retry budget exhausted")
def attach_legacy_identity(
self,
*,
task_id: str,
parent_task_id: str,
relation_type: RelationType,
goal_slug: str,
context: AuthenticatedTaskContext,
max_cas_retries: int = 4,
require_independent_readback: bool = True,
) -> LegacyAttachOutcome:
if require_independent_readback and not self._adapter.has_independent_readback:
raise LifecycleHookError("independent TaskPlane readback is not configured")
before = self._adapter.get_task_snapshot(task_id)
require_context_matches_task(context, before)
lineage = self._service.attach_legacy_task(
task_id=task_id,
parent_task_id=parent_task_id,
relation_type=relation_type,
goal_slug=goal_slug,
max_cas_retries=max_cas_retries,
expected_owner_principal_id=context.principal_id,
expected_mission_id=context.mission_id,
)
verified = False
if require_independent_readback:
self._adapter.verify_task_readback(
task_id=task_id,
expected_revision_at_least=before.revision,
expected_metadata={
LINEAGE_METADATA_KEY: lineage_to_metadata(lineage),
DISPLAY_NAME_METADATA_KEY: lineage.display_name,
},
)
# Domain serialization compatibility check below avoids requiring
# TaskLineage to expose a method in older versions.
verified = True
return LegacyAttachOutcome(lineage=lineage, independently_verified=verified)
def on_proven_terminal_child(
self,
*,
child_task_id: str,
context: AuthenticatedTaskContext,
acceptance_evaluator: AcceptanceEvaluator,
reviewed_at: datetime,
max_cas_retries: int = 4,
require_independent_readback: bool = True,
) -> TerminalHookOutcome:
if require_independent_readback and not self._adapter.has_independent_readback:
raise LifecycleHookError("independent TaskPlane readback is not configured")
child = self._adapter.get_task_snapshot(child_task_id)
require_context_matches_task(context, child)
if not child.terminal_evidence_proven:
raise LifecycleHookError(
"terminal child hook requires proven terminal evidence"
)
raw = child.metadata.get(LINEAGE_METADATA_KEY)
if not isinstance(raw, Mapping):
raise LifecycleHookError("terminal child lineage missing")
lineage = lineage_from_metadata(dict(raw))
root = self._adapter.get_task_snapshot(lineage.root_task_id)
require_context_matches_task(context, root)
def authorized_acceptance(root_record, descendants):
if (root_record.lineage.owner_principal_id != context.principal_id
or root_record.lineage.mission_id != context.mission_id):
raise LifecycleHookError("root authority changed during terminal review")
return acceptance_evaluator(root_record, descendants)
outcome = self._service.on_proven_terminal_child(
child_task_id=child_task_id,
acceptance_evaluator=authorized_acceptance,
reviewed_at=reviewed_at,
max_cas_retries=max_cas_retries,
)
persisted_review = self._adapter.get_root_review_record(
outcome.review.review_id
)
if persisted_review is None:
raise LifecycleHookError("root review was not persisted")
if persisted_review != root_review_to_record(outcome.review):
raise LifecycleHookError("persisted review does not match returned review")
dispatch = self._adapter.get_root_action_dispatch(outcome.review.review_id)
if dispatch is None:
raise LifecycleHookError("root action dispatch was not persisted")
if dispatch != outcome.dispatch:
raise LifecycleHookError("persisted dispatch does not match returned receipt")
verified = False
if require_independent_readback:
self._adapter.verify_root_readback(
root_task_id=outcome.review.root_task_id,
expected_review_id=outcome.review.review_id,
)
verified = True
return TerminalHookOutcome(
outcome=outcome,
root_review_persisted=True,
dispatch_persisted=True,
independently_verified=verified,
)
@@ -0,0 +1,539 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime
from enum import Enum
from typing import Mapping, MutableMapping, Protocol, Sequence
if not __package__:
from aios_task_lineage import (
AcceptanceEvaluator,
LineageError,
NextLegalAction,
RelationType,
RootReviewRecord,
RootReviewStore,
TaskLineage,
TaskRecord,
attach_legacy_lineage,
derive_safe_owner_alias,
lineage_from_metadata,
lineage_to_metadata,
make_child_lineage,
make_root_lineage,
normalize_goal_slug,
review_root_after_child_terminal,
root_review_from_record,
root_review_to_record,
)
else:
from .aios_task_lineage import (
AcceptanceEvaluator,
LineageError,
NextLegalAction,
RelationType,
RootReviewRecord,
RootReviewStore,
TaskLineage,
TaskRecord,
attach_legacy_lineage,
derive_safe_owner_alias,
lineage_from_metadata,
lineage_to_metadata,
make_child_lineage,
make_root_lineage,
normalize_goal_slug,
review_root_after_child_terminal,
root_review_from_record,
root_review_to_record,
)
LINEAGE_METADATA_KEY = "aios_task_lineage"
LINEAGE_REQUEST_DIGEST_KEY = "aios_lineage_request_digest"
ROOT_REVIEW_RECORD_KEY = "aios_root_review"
DISPLAY_NAME_METADATA_KEY = "aios_task_display_name"
class IntegrationError(RuntimeError):
pass
@dataclass(frozen=True)
class TaskAuthoritySnapshot:
task_id: str
revision: int
status: str
owner_principal_id: str
mission_id: str
created_at: datetime
metadata: Mapping[str, object]
evidence_refs: tuple[str, ...] = ()
terminal_evidence_proven: bool = False
@dataclass(frozen=True)
class NewTaskIdentityRequest:
task_id: str
owner_principal_id: str
preferred_owner_alias: str | None
created_at: datetime
mission_id: str
goal_slug: str
root_acceptance_ref: str
relation_type: RelationType = RelationType.ROOT
parent_task_id: str | None = None
@dataclass(frozen=True)
class PreparedTaskIdentity:
task_id: str
display_name: str
lineage: TaskLineage
metadata_patch: Mapping[str, object]
@dataclass(frozen=True)
class TerminalReviewOutcome:
review: RootReviewRecord
dispatch: "RootActionDispatchReceipt"
@dataclass(frozen=True)
class RootActionDispatchReceipt:
review_id: str
root_task_id: str
action: NextLegalAction
idempotency_key: str
status: str
mission_id: str = ""
root_revision: int = 0
plan_digest: str | None = None
execution_key: str | None = None
PERSISTED_DISPATCH_STATUSES = frozenset(
{
"PENDING",
"CLAIMED",
"EFFECT_UNKNOWN",
"EXECUTED_PENDING_REREVIEW",
"WAITING_DEPENDENCY",
"WAITING_OWNER",
"CLOSED",
"SUPERSEDED",
"BLOCKED",
# Read compatibility only. A consumer must reconcile this legacy
# state unless a complete durable receipt is present.
"DISPATCHED",
}
)
class TaskPlaneLineagePort(Protocol):
"""Narrow authority port required by the lineage integration service.
The live adapter must implement these methods over the existing TaskPlane /
mission / journal / CAS mechanisms. This protocol deliberately avoids
guessing concrete DB tables, HTTP endpoints, or handler names.
"""
def get_task_snapshot(self, task_id: str) -> TaskAuthoritySnapshot: ...
def list_descendant_snapshots(
self, root_task_id: str
) -> Sequence[TaskAuthoritySnapshot]: ...
def reserve_sibling_sequence(
self,
*,
parent_task_id: str,
relation_type: RelationType,
child_task_id: str,
) -> int:
"""Idempotently reserve one sibling sequence for child_task_id."""
def compare_and_swap_task_metadata(
self,
*,
task_id: str,
expected_revision: int,
metadata_patch: Mapping[str, object],
) -> bool: ...
def get_root_review_record(
self, review_id: str
) -> Mapping[str, object] | None: ...
def compare_and_swap_root_review_record(
self,
*,
root_task_id: str,
expected_root_revision: int,
review_record: Mapping[str, object],
) -> bool: ...
def get_root_action_dispatch(
self, review_id: str
) -> RootActionDispatchReceipt | None: ...
def dispatch_root_action(
self,
*,
review: RootReviewRecord,
idempotency_key: str,
) -> RootActionDispatchReceipt: ...
def _extract_lineage(snapshot: TaskAuthoritySnapshot) -> TaskLineage:
raw = snapshot.metadata.get(LINEAGE_METADATA_KEY)
if not isinstance(raw, dict):
raise IntegrationError(
f"task {snapshot.task_id} does not have authoritative lineage metadata"
)
lineage = lineage_from_metadata(raw)
if lineage.task_id != snapshot.task_id:
raise IntegrationError("lineage task_id does not match authority task")
if lineage.owner_principal_id != snapshot.owner_principal_id:
raise IntegrationError("lineage owner does not match authority owner")
if lineage.mission_id != snapshot.mission_id:
raise IntegrationError("lineage mission does not match authority mission")
if snapshot.created_at.tzinfo is None or snapshot.created_at.utcoffset() is None:
raise IntegrationError("authority created_at must be timezone-aware")
if lineage.created_at != snapshot.created_at:
raise IntegrationError("lineage created_at does not match authority task")
display = snapshot.metadata.get(DISPLAY_NAME_METADATA_KEY)
if display not in (None, "", lineage.display_name):
raise IntegrationError("lineage display name conflicts with authority metadata")
return lineage
def _as_task_record(snapshot: TaskAuthoritySnapshot) -> TaskRecord:
return TaskRecord(
task_id=snapshot.task_id,
revision=snapshot.revision,
status=snapshot.status,
lineage=_extract_lineage(snapshot),
evidence_refs=snapshot.evidence_refs,
terminal_evidence_proven=snapshot.terminal_evidence_proven,
)
def _validate_record_root_binding(record: TaskRecord, root: TaskRecord) -> None:
if root.lineage.relation_type != RelationType.ROOT:
raise IntegrationError("root_task_id does not resolve to a ROOT task")
if record.lineage.root_task_id != root.task_id:
raise IntegrationError("task lineage points to a different root")
if record.lineage.owner_principal_id != root.lineage.owner_principal_id:
raise IntegrationError("task lineage owner does not match root owner")
if record.lineage.mission_id != root.lineage.mission_id:
raise IntegrationError("task lineage mission does not match root mission")
if record.lineage.root_acceptance_ref != root.lineage.root_acceptance_ref:
raise IntegrationError("task lineage acceptance reference does not match root")
def _load_authority_record(
port: TaskPlaneLineagePort, task_id: str
) -> tuple[TaskRecord, TaskRecord]:
record = _as_task_record(port.get_task_snapshot(task_id))
if record.lineage.relation_type == RelationType.ROOT:
return record, record
root = _as_task_record(port.get_task_snapshot(record.lineage.root_task_id))
current = record
seen = {record.task_id}
while current.lineage.relation_type != RelationType.ROOT:
_validate_record_root_binding(current, root)
parent_id = current.lineage.parent_task_id
if not parent_id or parent_id in seen:
raise IntegrationError("task lineage contains a parent cycle")
seen.add(parent_id)
parent = _as_task_record(port.get_task_snapshot(parent_id))
_validate_record_root_binding(parent, root)
if current.lineage.parent_task_id != parent.task_id:
raise IntegrationError("task lineage parent does not match authority parent")
if current.lineage.lineage_depth != parent.lineage.lineage_depth + 1:
raise IntegrationError("task lineage depth does not match authority parent")
if current.lineage.parent_revision is None or (
current.lineage.parent_revision > parent.revision
):
raise IntegrationError("task lineage parent revision is not authoritative")
current = parent
if current.task_id != root.task_id:
raise IntegrationError("task lineage parent chain ends at a different root")
return record, root
def _validate_dispatch_receipt(
receipt: RootActionDispatchReceipt,
*,
review: RootReviewRecord,
idempotency_key: str,
existing: bool,
) -> None:
if (
receipt.review_id != review.review_id
or receipt.root_task_id != review.root_task_id
or receipt.action != review.next_legal_action
or receipt.idempotency_key != idempotency_key
or receipt.status not in PERSISTED_DISPATCH_STATUSES
):
noun = "existing root-action dispatch" if existing else "root-action dispatch receipt"
raise IntegrationError(f"{noun} conflicts with persisted review")
class _RootReviewStoreAdapter(RootReviewStore):
def __init__(self, port: TaskPlaneLineagePort):
self._port = port
def get_task(self, task_id: str) -> TaskRecord:
record, _ = _load_authority_record(self._port, task_id)
return record
def list_descendants(self, root_task_id: str) -> Sequence[TaskRecord]:
root, _ = _load_authority_record(self._port, root_task_id)
if root.lineage.relation_type != RelationType.ROOT:
raise IntegrationError("descendant lookup key is not a ROOT task")
records: list[TaskRecord] = []
for snapshot in self._port.list_descendant_snapshots(root_task_id):
record, resolved_root = _load_authority_record(self._port, snapshot.task_id)
if resolved_root.task_id != root_task_id:
raise IntegrationError("descendant authority returned a cross-root task")
_validate_record_root_binding(record, root)
records.append(record)
return tuple(records)
def get_root_review(self, review_id: str) -> RootReviewRecord | None:
raw = self._port.get_root_review_record(review_id)
if raw is None:
return None
if not isinstance(raw, Mapping):
raise IntegrationError("persisted root-review record must be a mapping")
try:
return root_review_from_record(
raw,
expected_review_id=review_id,
)
except LineageError as exc:
raise IntegrationError(
f"invalid persisted root-review record: {exc}"
) from exc
def compare_and_swap_root_review(
self,
root_task_id: str,
expected_root_revision: int,
review: RootReviewRecord,
) -> bool:
return self._port.compare_and_swap_root_review_record(
root_task_id=root_task_id,
expected_root_revision=expected_root_revision,
review_record=root_review_to_record(review),
)
class TaskIdentityLineageService:
"""Application service that wires the lineage domain to TaskPlane authority."""
def __init__(self, port: TaskPlaneLineagePort):
self._port = port
def prepare_new_task_identity(
self, request: NewTaskIdentityRequest
) -> PreparedTaskIdentity:
alias = derive_safe_owner_alias(
request.owner_principal_id, request.preferred_owner_alias
)
if request.relation_type == RelationType.ROOT:
if request.parent_task_id is not None:
raise IntegrationError("ROOT task cannot have parent_task_id")
lineage = make_root_lineage(
task_id=request.task_id,
owner_principal_id=request.owner_principal_id,
owner_alias=alias,
created_at=request.created_at,
mission_id=request.mission_id,
goal_slug=request.goal_slug,
root_acceptance_ref=request.root_acceptance_ref,
)
else:
if not request.parent_task_id:
raise IntegrationError("non-root task requires parent_task_id")
parent, root = _load_authority_record(
self._port, request.parent_task_id
)
if request.owner_principal_id != root.lineage.owner_principal_id:
raise IntegrationError("cross-owner child creation rejected")
if request.mission_id != root.lineage.mission_id:
raise IntegrationError("cross-mission child creation rejected")
if request.root_acceptance_ref != root.lineage.root_acceptance_ref:
raise IntegrationError("root acceptance reference mismatch")
sequence = self._port.reserve_sibling_sequence(
parent_task_id=parent.task_id,
relation_type=request.relation_type,
child_task_id=request.task_id,
)
lineage = make_child_lineage(
task_id=request.task_id,
created_at=request.created_at,
parent=parent,
root=root,
relation_type=request.relation_type,
sibling_sequence=sequence,
goal_slug=request.goal_slug,
)
return PreparedTaskIdentity(
task_id=request.task_id,
display_name=lineage.display_name,
lineage=lineage,
metadata_patch={
LINEAGE_METADATA_KEY: lineage_to_metadata(lineage),
DISPLAY_NAME_METADATA_KEY: lineage.display_name,
},
)
def attach_legacy_task(
self,
*,
task_id: str,
parent_task_id: str,
relation_type: RelationType,
goal_slug: str,
expected_owner_principal_id: str | None = None,
expected_mission_id: str | None = None,
max_cas_retries: int = 4,
) -> TaskLineage:
if max_cas_retries < 1:
raise IntegrationError("max_cas_retries must be >= 1")
for _ in range(max_cas_retries):
current = self._port.get_task_snapshot(task_id)
if (
expected_owner_principal_id is not None
and current.owner_principal_id != expected_owner_principal_id
):
raise IntegrationError("legacy task owner changed after authorization")
if (
expected_mission_id is not None
and current.mission_id != expected_mission_id
):
raise IntegrationError("legacy task mission changed after authorization")
raw = current.metadata.get(LINEAGE_METADATA_KEY)
if isinstance(raw, Mapping):
record, _ = _load_authority_record(self._port, task_id)
lineage = record.lineage
expected_goal_slug = normalize_goal_slug(goal_slug)
if (
lineage.parent_task_id != parent_task_id
or lineage.relation_type != relation_type
or lineage.goal_slug != expected_goal_slug
):
raise IntegrationError(
"existing legacy lineage conflicts with requested binding"
)
display = current.metadata.get(DISPLAY_NAME_METADATA_KEY)
if display == lineage.display_name:
return lineage
if display not in (None, ""):
raise IntegrationError(
"legacy display name conflicts with authoritative lineage"
)
if self._port.compare_and_swap_task_metadata(
task_id=current.task_id,
expected_revision=current.revision,
metadata_patch={
DISPLAY_NAME_METADATA_KEY: lineage.display_name,
},
):
return lineage
continue
parent, root = _load_authority_record(self._port, parent_task_id)
if root.status == "COMPLETED":
raise IntegrationError("cannot attach lineage below a completed root")
if (
expected_owner_principal_id is not None
and root.lineage.owner_principal_id != expected_owner_principal_id
):
raise IntegrationError("legacy root owner does not match authorization")
if (
expected_mission_id is not None
and root.lineage.mission_id != expected_mission_id
):
raise IntegrationError("legacy root mission does not match authorization")
if current.owner_principal_id != root.lineage.owner_principal_id:
raise IntegrationError("legacy task owner does not match root owner")
if current.mission_id != root.lineage.mission_id:
raise IntegrationError("legacy task mission does not match root mission")
sequence = self._port.reserve_sibling_sequence(
parent_task_id=parent.task_id,
relation_type=relation_type,
child_task_id=current.task_id,
)
lineage = attach_legacy_lineage(
existing_task_id=current.task_id,
created_at=current.created_at,
parent=parent,
root=root,
relation_type=relation_type,
sibling_sequence=sequence,
goal_slug=goal_slug,
)
if self._port.compare_and_swap_task_metadata(
task_id=current.task_id,
expected_revision=current.revision,
metadata_patch={
LINEAGE_METADATA_KEY: lineage_to_metadata(lineage),
DISPLAY_NAME_METADATA_KEY: lineage.display_name,
},
):
return lineage
raise IntegrationError("legacy lineage CAS retry budget exhausted")
def on_proven_terminal_child(
self,
*,
child_task_id: str,
acceptance_evaluator: AcceptanceEvaluator,
reviewed_at: datetime,
max_cas_retries: int = 4,
) -> TerminalReviewOutcome:
store = _RootReviewStoreAdapter(self._port)
review = review_root_after_child_terminal(
store=store,
trigger_child_task_id=child_task_id,
acceptance_evaluator=acceptance_evaluator,
reviewed_at=reviewed_at,
max_cas_retries=max_cas_retries,
)
existing = self._port.get_root_action_dispatch(review.review_id)
if existing is not None:
idempotency_key = (
f"root-review:{review.review_id}:{review.next_legal_action.value}"
)
_validate_dispatch_receipt(
existing,
review=review,
idempotency_key=idempotency_key,
existing=True,
)
return TerminalReviewOutcome(review=review, dispatch=existing)
idempotency_key = f"root-review:{review.review_id}:{review.next_legal_action.value}"
receipt = self._port.dispatch_root_action(
review=review,
idempotency_key=idempotency_key,
)
_validate_dispatch_receipt(
receipt,
review=review,
idempotency_key=idempotency_key,
existing=False,
)
return TerminalReviewOutcome(review=review, dispatch=receipt)
@@ -0,0 +1,464 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import json
from pathlib import Path
import sqlite3
from typing import Callable, Mapping, Protocol, Sequence, runtime_checkable
if not __package__:
from aios_task_lineage import RelationType, RootReviewRecord, lineage_from_metadata
from aios_task_lineage_integration import (
IntegrationError,
LINEAGE_METADATA_KEY,
RootActionDispatchReceipt,
TaskAuthoritySnapshot,
TaskPlaneLineagePort,
)
else:
from .aios_task_lineage import RelationType, RootReviewRecord, lineage_from_metadata
from .aios_task_lineage_integration import (
IntegrationError,
LINEAGE_METADATA_KEY,
RootActionDispatchReceipt,
TaskAuthoritySnapshot,
TaskPlaneLineagePort,
)
class AuthorityAdapterError(IntegrationError):
pass
def _parse_datetime(value: object) -> datetime:
if isinstance(value, datetime):
if value.tzinfo is None or value.utcoffset() is None:
raise AuthorityAdapterError("authority datetime must be timezone-aware")
return value.astimezone(timezone.utc)
if not isinstance(value, str) or not value:
raise AuthorityAdapterError("authority datetime missing")
try:
dt = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError as exc:
raise AuthorityAdapterError("authority datetime invalid") from exc
if dt.tzinfo is None or dt.utcoffset() is None:
raise AuthorityAdapterError("authority datetime must include timezone")
return dt.astimezone(timezone.utc)
def _load_json_object(value: object, *, field: str) -> dict[str, object]:
if value in (None, ""):
return {}
if isinstance(value, Mapping):
return dict(value)
if not isinstance(value, str):
raise AuthorityAdapterError(f"{field} must be a JSON object")
try:
obj = json.loads(value)
except json.JSONDecodeError as exc:
raise AuthorityAdapterError(f"{field} is invalid JSON") from exc
if not isinstance(obj, dict):
raise AuthorityAdapterError(f"{field} must decode to an object")
return obj
def _metadata_identity(metadata: Mapping[str, object]) -> tuple[str, str]:
lineage = metadata.get(LINEAGE_METADATA_KEY)
if isinstance(lineage, Mapping):
owner = lineage.get("owner_principal_id")
mission = lineage.get("mission_id")
if isinstance(owner, str) and owner and isinstance(mission, str) and mission:
return owner, mission
# Pre-lineage/legacy compatibility. These keys are accepted only as
# authority-provided metadata; missing values fail closed rather than being
# invented by this adapter.
owner = metadata.get("owner_principal_id")
mission = metadata.get("mission_id")
if not isinstance(owner, str) or not owner:
raise AuthorityAdapterError("task owner_principal_id is not discoverable")
if not isinstance(mission, str) or not mission:
raise AuthorityAdapterError("task mission_id is not discoverable")
return owner, mission
def _metadata_evidence(metadata: Mapping[str, object]) -> tuple[str, ...]:
raw = metadata.get("evidence_refs")
if raw is None:
return ()
if not isinstance(raw, list) or not all(isinstance(x, str) for x in raw):
raise AuthorityAdapterError("evidence_refs must be a string list")
return tuple(dict.fromkeys(raw))
@runtime_checkable
class TaskPlaneReadback(Protocol):
"""Independent authority readback surface.
This may be backed by the TaskPlane query service, a read replica, or a
read-only SQLite authority. It must not mutate task state.
"""
def get_task_snapshot(self, task_id: str) -> TaskAuthoritySnapshot: ...
def list_descendant_snapshots(
self, root_task_id: str
) -> Sequence[TaskAuthoritySnapshot]: ...
def get_root_review_record(self, review_id: str) -> Mapping[str, object] | None: ...
def get_root_action_dispatch(self, review_id: str) -> RootActionDispatchReceipt | None: ...
@dataclass(frozen=True)
class CallableTaskPlaneReadback:
"""Readback adapter for an existing official TaskPlane query/service."""
get_task: Callable[[str], TaskAuthoritySnapshot]
list_descendants: Callable[[str], Sequence[TaskAuthoritySnapshot]]
get_review: Callable[[str], Mapping[str, object] | None] | None = None
get_dispatch: Callable[[str], RootActionDispatchReceipt | None] | None = None
def get_task_snapshot(self, task_id: str) -> TaskAuthoritySnapshot:
snapshot = self.get_task(task_id)
if not isinstance(snapshot, TaskAuthoritySnapshot):
raise AuthorityAdapterError("readback returned invalid task snapshot")
if snapshot.task_id != task_id:
raise AuthorityAdapterError("readback returned wrong task identity")
return snapshot
def list_descendant_snapshots(
self, root_task_id: str
) -> Sequence[TaskAuthoritySnapshot]:
values = tuple(self.list_descendants(root_task_id))
if not all(isinstance(v, TaskAuthoritySnapshot) for v in values):
raise AuthorityAdapterError("readback returned invalid descendant snapshot")
return values
def get_root_review_record(self, review_id: str) -> Mapping[str, object] | None:
if self.get_review is None:
raise AuthorityAdapterError("independent review readback is not configured")
return self.get_review(review_id)
def get_root_action_dispatch(self, review_id: str) -> RootActionDispatchReceipt | None:
if self.get_dispatch is None:
raise AuthorityAdapterError("independent dispatch readback is not configured")
return self.get_dispatch(review_id)
@dataclass(frozen=True)
class AuthoritySchemaReport:
task_table: str
columns: tuple[str, ...]
required_columns_present: bool
missing_columns: tuple[str, ...]
task_count: int
class ReadOnlySQLiteTaskPlane:
"""Read-only discovery/readback helper for the current TaskPlane SQLite authority.
This class never writes. It opens SQLite read-only, including committed WAL
data, and exists only for discovery, diagnostics and independent readback.
Execution writes must go through the live TaskPlane repository/service and
are supplied separately through ``TaskPlaneServiceBindings``.
"""
REQUIRED_TASK_COLUMNS = (
"task_id",
"status",
"revision",
"metadata_json",
"created_at",
)
def __init__(self, db_path: Path):
self.db_path = Path(db_path)
def _connect(self) -> sqlite3.Connection:
if not self.db_path.is_file():
raise AuthorityAdapterError(f"TaskPlane DB missing: {self.db_path}")
# A live WAL database is not immutable: ignoring its WAL can return
# stale state. as_uri also quotes URI metacharacters in file names.
uri = self.db_path.resolve().as_uri() + "?mode=ro"
con = sqlite3.connect(uri, uri=True)
con.row_factory = sqlite3.Row
return con
def inspect_schema(self) -> AuthoritySchemaReport:
con = self._connect()
try:
rows = list(con.execute("PRAGMA table_info(tasks)"))
columns = tuple(str(r["name"]) for r in rows)
missing = tuple(c for c in self.REQUIRED_TASK_COLUMNS if c not in columns)
count = int(con.execute("SELECT COUNT(*) FROM tasks").fetchone()[0]) if rows else 0
return AuthoritySchemaReport(
task_table="tasks",
columns=columns,
required_columns_present=not missing,
missing_columns=missing,
task_count=count,
)
finally:
con.close()
def _row_to_snapshot(self, row: sqlite3.Row) -> TaskAuthoritySnapshot:
keys = set(row.keys())
metadata = _load_json_object(
row["metadata_json"] if "metadata_json" in keys else None,
field="metadata_json",
)
owner, mission = _metadata_identity(metadata)
created_value = row["created_at"] if "created_at" in keys else None
return TaskAuthoritySnapshot(
task_id=str(row["task_id"]),
revision=int(row["revision"]),
status=str(row["status"]),
owner_principal_id=owner,
mission_id=mission,
created_at=_parse_datetime(created_value),
metadata=metadata,
evidence_refs=_metadata_evidence(metadata),
# A metadata flag cannot stand in for execution-integrity proof.
# The official service adapter must resolve verified receipts.
terminal_evidence_proven=False,
)
def get_task_snapshot(self, task_id: str) -> TaskAuthoritySnapshot:
con = self._connect()
try:
row = con.execute("SELECT * FROM tasks WHERE task_id=? LIMIT 1", (task_id,)).fetchone()
if row is None:
raise AuthorityAdapterError(f"task not found: {task_id}")
return self._row_to_snapshot(row)
finally:
con.close()
def list_descendant_snapshots(self, root_task_id: str) -> tuple[TaskAuthoritySnapshot, ...]:
con = self._connect()
try:
rows = list(con.execute("SELECT * FROM tasks ORDER BY task_id"))
finally:
con.close()
out: list[TaskAuthoritySnapshot] = []
for row in rows:
try:
snap = self._row_to_snapshot(row)
except AuthorityAdapterError:
# Legacy tasks without discoverable identity cannot safely be
# attached to a root by this read-only adapter.
continue
raw = snap.metadata.get(LINEAGE_METADATA_KEY)
if (
isinstance(raw, Mapping)
and raw.get("root_task_id") == root_task_id
and snap.task_id != root_task_id
):
out.append(snap)
return tuple(out)
@dataclass(frozen=True)
class TaskPlaneServiceBindings:
"""In-process bindings to the existing TaskPlane authority.
These callables must be backed by the real TaskPlane repository/service,
including its existing transactions, revision/CAS, journal and continuation
mechanisms. They must NOT be implemented as out-of-band SQLite mutations.
"""
get_task_snapshot: Callable[[str], TaskAuthoritySnapshot]
list_descendant_snapshots: Callable[[str], Sequence[TaskAuthoritySnapshot]]
reserve_sibling_sequence: Callable[[str, RelationType, str], int]
compare_and_swap_task_metadata: Callable[[str, int, Mapping[str, object]], bool]
get_root_review_record: Callable[[str], Mapping[str, object] | None]
compare_and_swap_root_review_record: Callable[
[str, int, Mapping[str, object]], bool
]
get_root_action_dispatch: Callable[[str], RootActionDispatchReceipt | None]
dispatch_root_action: Callable[
[RootReviewRecord, str], RootActionDispatchReceipt
]
class InProcessTaskPlaneLineageAdapter(TaskPlaneLineagePort):
"""Concrete TaskPlaneLineagePort backed by existing TaskPlane service calls."""
def __init__(
self,
bindings: TaskPlaneServiceBindings,
*,
readback: TaskPlaneReadback | None = None,
):
self._b = bindings
self._readback = readback
def get_task_snapshot(self, task_id: str) -> TaskAuthoritySnapshot:
return self._b.get_task_snapshot(task_id)
def list_descendant_snapshots(self, root_task_id: str) -> Sequence[TaskAuthoritySnapshot]:
return tuple(self._b.list_descendant_snapshots(root_task_id))
def reserve_sibling_sequence(
self,
*,
parent_task_id: str,
relation_type: RelationType,
child_task_id: str,
) -> int:
seq = self._b.reserve_sibling_sequence(
parent_task_id, relation_type, child_task_id
)
if not isinstance(seq, int) or seq < 1:
raise AuthorityAdapterError("TaskPlane returned invalid sibling sequence")
return seq
def compare_and_swap_task_metadata(
self,
*,
task_id: str,
expected_revision: int,
metadata_patch: Mapping[str, object],
) -> bool:
return bool(
self._b.compare_and_swap_task_metadata(
task_id, expected_revision, metadata_patch
)
)
def get_root_review_record(self, review_id: str) -> Mapping[str, object] | None:
return self._b.get_root_review_record(review_id)
def compare_and_swap_root_review_record(
self,
*,
root_task_id: str,
expected_root_revision: int,
review_record: Mapping[str, object],
) -> bool:
return bool(
self._b.compare_and_swap_root_review_record(
root_task_id, expected_root_revision, review_record
)
)
def get_root_action_dispatch(
self, review_id: str
) -> RootActionDispatchReceipt | None:
return self._b.get_root_action_dispatch(review_id)
def dispatch_root_action(
self,
*,
review: RootReviewRecord,
idempotency_key: str,
) -> RootActionDispatchReceipt:
return self._b.dispatch_root_action(review, idempotency_key)
def verify_task_readback(
self,
*,
task_id: str,
expected_revision_at_least: int,
expected_metadata: Mapping[str, object],
) -> TaskAuthoritySnapshot:
if self._readback is None:
raise AuthorityAdapterError("independent TaskPlane readback is not configured")
snap = self._readback.get_task_snapshot(task_id)
if snap.task_id != task_id:
raise AuthorityAdapterError("TaskPlane readback task identity mismatch")
if snap.revision < expected_revision_at_least:
raise AuthorityAdapterError("TaskPlane readback revision did not advance")
for key, expected in expected_metadata.items():
if snap.metadata.get(key) != expected:
raise AuthorityAdapterError(
f"TaskPlane readback metadata mismatch for {key}"
)
return snap
def verify_root_readback(
self,
*,
root_task_id: str,
expected_review_id: str,
) -> tuple[TaskAuthoritySnapshot, ...]:
"""Verify that independent authority descendants remain bound to root.
The persisted ROOT_REVIEW record itself may live in a journal rather
than the task row. This method verifies the task/lineage projection
independently while the service binding verifies the journal receipt.
"""
if self._readback is None:
raise AuthorityAdapterError("independent TaskPlane readback is not configured")
review = self.get_root_review_record(expected_review_id)
dispatch = self.get_root_action_dispatch(expected_review_id)
if (review is None or review.get("review_id") != expected_review_id
or review.get("root_task_id") != root_task_id or dispatch is None):
raise AuthorityAdapterError("root review or dispatch identity mismatch")
read_review = getattr(self._readback, "get_root_review_record", None)
read_dispatch = getattr(self._readback, "get_root_action_dispatch", None)
if not callable(read_review) or not callable(read_dispatch):
raise AuthorityAdapterError("independent review/dispatch readback is not configured")
if read_review(expected_review_id) != review:
raise AuthorityAdapterError("independent root review readback mismatch")
if read_dispatch(expected_review_id) != dispatch:
raise AuthorityAdapterError("independent dispatch readback mismatch")
root = self._readback.get_task_snapshot(root_task_id)
raw_root = root.metadata.get(LINEAGE_METADATA_KEY)
if root.task_id != root_task_id or not isinstance(raw_root, Mapping):
raise AuthorityAdapterError("root readback identity or lineage missing")
root_lineage = lineage_from_metadata(dict(raw_root))
if (root_lineage.relation_type != RelationType.ROOT
or root_lineage.task_id != root_task_id
or root_lineage.owner_principal_id != root.owner_principal_id
or root_lineage.mission_id != root.mission_id):
raise AuthorityAdapterError("root readback authority mismatch")
revision = review.get("root_revision_after")
if type(revision) is not int or root.revision < revision:
raise AuthorityAdapterError("root readback revision is stale")
descendants = tuple(self._readback.list_descendant_snapshots(root_task_id))
seen = set()
for snap in descendants:
raw = snap.metadata.get(LINEAGE_METADATA_KEY)
if not isinstance(raw, Mapping):
raise AuthorityAdapterError("descendant lineage missing")
lineage = lineage_from_metadata(dict(raw))
if (snap.task_id in seen or snap.task_id == root_task_id
or lineage.task_id != snap.task_id
or lineage.root_task_id != root_task_id
or lineage.owner_principal_id != root.owner_principal_id
or lineage.mission_id != root.mission_id
or snap.owner_principal_id != root.owner_principal_id
or snap.mission_id != root.mission_id
or lineage.root_acceptance_ref != root_lineage.root_acceptance_ref):
raise AuthorityAdapterError(
f"descendant {snap.task_id} is projected under wrong root"
)
seen.add(snap.task_id)
if review.get("trigger_child_task_id") not in seen:
raise AuthorityAdapterError("trigger child missing from independent readback")
return descendants
@property
def has_independent_readback(self) -> bool:
return self._readback is not None
@dataclass(frozen=True)
class AuthenticatedTaskContext:
principal_id: str
preferred_alias: str | None
mission_id: str
def require_context_matches_task(
context: AuthenticatedTaskContext,
task: TaskAuthoritySnapshot,
) -> None:
if context.principal_id != task.owner_principal_id:
raise AuthorityAdapterError("authenticated principal does not own task")
if context.mission_id != task.mission_id:
raise AuthorityAdapterError("authenticated mission does not match task")
@@ -0,0 +1,242 @@
"""Narrow staging Mission ingress authenticated by the registered Owner signer.
The signed payload binds the method, resource, body and idempotency key. This
adapter neither signs requests nor grants Git/worker/production capabilities.
"""
from __future__ import annotations
import argparse
import importlib.util
import json
from pathlib import Path
import re
import stat
import time
from http.server import BaseHTTPRequestHandler, HTTPServer
from typing import Any
from .task_plane.errors import (
ForbiddenError, TaskPlaneHTTPError, UnauthorizedError, ValidationError,
)
AUDIENCE = "aios.task-mission-authority.staging.v1"
NAMESPACE = "aios-owner-approval-v1"
MAX_BODY = 131072
REQUEST_FIELDS = frozenset({
"schema", "audience", "method", "path", "body", "idempotency_key",
"issued_at", "expires_at", "nonce",
})
def canonical_bytes(value: Any) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"),
ensure_ascii=False, allow_nan=False).encode("utf-8")
def strict_object(pairs):
result = {}
for key, value in pairs:
if key in result:
raise ValidationError("duplicate JSON key")
result[key] = value
return result
class SignedMissionIngress:
def __init__(
self, service, verifier, *, terminal_proof_importer=None, clock=time.time
):
if not callable(getattr(verifier, "verify", None)):
raise ValueError("registered Owner verifier required")
identity = getattr(verifier, "signer_id", None)
if not isinstance(identity, str) or not identity.strip():
raise ValueError("registered signer identity required")
self.service = service
self.verifier = verifier
self.identity = identity
self.terminal_proof_importer = terminal_proof_importer
self.clock = clock
def handle(self, envelope: dict) -> tuple[int, dict]:
if not isinstance(envelope, dict) or set(envelope) != {
"request", "signature", "key_id"
}:
raise ValidationError("invalid signed mission envelope")
request = envelope["request"]
if not isinstance(request, dict) or set(request) != REQUEST_FIELDS:
raise ValidationError("invalid mission request fields")
if (request["schema"] != "aios.signed-mission-request.v1"
or request["audience"] != AUDIENCE):
raise ForbiddenError("mission request audience mismatch")
issued, expires = request["issued_at"], request["expires_at"]
if (type(issued) is not int or type(expires) is not int
or not 0 < expires - issued <= 300
or not issued <= self.clock() < expires):
raise UnauthorizedError("mission request expired or not yet valid")
if (not isinstance(request["nonce"], str)
or re.fullmatch(r"[A-Za-z0-9_-]{16,128}", request["nonce"]) is None):
raise ValidationError("invalid mission request nonce")
if (not isinstance(envelope["signature"], str)
or not isinstance(envelope["key_id"], str)):
raise UnauthorizedError("Owner signature required")
try:
verified = self.verifier.verify(
namespace=NAMESPACE, payload=canonical_bytes(request),
signature=envelope["signature"], key_id=envelope["key_id"],
)
except Exception:
verified = False
if verified is not True:
raise UnauthorizedError("Owner signature invalid")
method, path, body = request["method"], request["path"], request["body"]
key = request["idempotency_key"]
if method == "POST" and path == "/missions":
if not isinstance(body, dict) or not isinstance(key, str) or not key.strip():
raise ValidationError("mission body and idempotency key required")
return 201, self.service.create_mission(
body, key, actor_identity=self.identity,
)
if (method == "GET" and isinstance(path, str)
and re.fullmatch(r"/missions/[A-Za-z0-9][A-Za-z0-9._:-]{2,179}", path)
and body is None and key is None):
return 200, self.service.get_mission(
path.split("/")[2], actor_identity=self.identity,
)
if (method == "POST" and isinstance(path, str)
and re.fullmatch(r"/tasks/[A-Za-z0-9][A-Za-z0-9._:-]{2,179}/delegate", path)):
if not isinstance(body, dict) or not isinstance(key, str) or not key.strip():
raise ValidationError("child body and idempotency key required")
return 201, self.service.delegate_mission(
path.split("/")[2], body, key, actor_identity=self.identity,
)
if (method == "POST" and isinstance(path, str)
and re.fullmatch(
r"/missions/[A-Za-z0-9][A-Za-z0-9._:-]{2,179}/legacy-attachments",
path,
)):
if not isinstance(body, dict) or not isinstance(key, str) or not key.strip():
raise ValidationError(
"legacy attachment body and idempotency key required"
)
result = self.service.attach_legacy_attachment(
path.split("/")[2], body, key, actor_identity=self.identity,
)
return (200 if result.get("idempotent") is True else 201), result
if (method == "GET" and isinstance(path, str)
and re.fullmatch(
r"/missions/[A-Za-z0-9][A-Za-z0-9._:-]{2,179}/"
r"legacy-attachments/[A-Za-z0-9][A-Za-z0-9._:-]{2,179}",
path,
) and body is None and key is None):
pieces = path.split("/")
return 200, self.service.get_legacy_attachment(
pieces[2], pieces[4], actor_identity=self.identity,
)
if (method == "POST" and isinstance(path, str)
and re.fullmatch(
r"/tasks/[A-Za-z0-9][A-Za-z0-9._:-]{2,179}/reconcile-terminal-proof",
path,
)):
if body != {} or not isinstance(key, str) or not key.strip():
raise ValidationError(
"terminal proof trigger requires an empty body and idempotency key"
)
if self.terminal_proof_importer is None:
raise ForbiddenError("terminal proof importer is not configured")
return 200, self.terminal_proof_importer.reconcile(
path.split("/")[2], key, actor_identity=self.identity,
)
raise ForbiddenError("operation outside mission ingress scope")
def load_registered_verifier(path: Path):
metadata = path.lstat()
if (not stat.S_ISREG(metadata.st_mode) or metadata.st_uid != 0
or metadata.st_mode & 0o022):
raise ValueError("Owner verifier must be a root-owned immutable regular file")
spec = importlib.util.spec_from_file_location("aios_mission_owner_verifier", path)
if spec is None or spec.loader is None:
raise ValueError("Owner verifier module unavailable")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module.GrantRootOwnerApprovalVerifier()
def make_handler(ingress: SignedMissionIngress):
class Handler(BaseHTTPRequestHandler):
def log_message(self, *_args):
pass # Do not log signed payloads or transport credentials.
def respond(self, status, value):
raw = canonical_bytes(value)
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Cache-Control", "no-store")
self.send_header("Content-Length", str(len(raw)))
self.end_headers()
self.wfile.write(raw)
def do_POST(self):
try:
if self.path != "/v1/signed-request":
raise ForbiddenError("unsupported mission ingress route")
if self.headers.get("Transfer-Encoding"):
raise ValidationError("transfer encoding is unsupported")
lengths = self.headers.get_all("Content-Length", [])
if len(lengths) != 1 or not lengths[0].isdigit():
raise ValidationError("one content length required")
length = int(lengths[0])
if not 0 < length <= MAX_BODY:
raise ValidationError("mission request too large or empty")
self.connection.settimeout(10)
raw = self.rfile.read(length)
if len(raw) != length:
raise ValidationError("truncated mission request")
envelope = json.loads(raw, object_pairs_hook=strict_object)
status, result = ingress.handle(envelope)
self.respond(status, result)
except TaskPlaneHTTPError as exc:
self.respond(exc.status_code, exc.to_payload())
except (ValueError, UnicodeError, RecursionError):
self.respond(400, {"error": "invalid_json"})
except Exception:
self.respond(500, {"error": "mission_authority_unavailable"})
return Handler
def main() -> None:
from .task_plane.api import TaskPlaneService
from .task_plane.mission import MissionAuthority, MissionPrincipalBinding
from .task_plane.policy import IdentityVerifier, PolicyEngine
from .task_plane.store import TaskPlaneStore
parser = argparse.ArgumentParser()
parser.add_argument("--db", type=Path, required=True)
parser.add_argument("--owner-verifier", type=Path, required=True)
parser.add_argument("--port", type=int, required=True)
args = parser.parse_args()
if not 1024 <= args.port <= 65535:
parser.error("unprivileged loopback port required")
verifier = load_registered_verifier(args.owner_verifier)
identity = verifier.signer_id
store = TaskPlaneStore(str(args.db), high_assurance=True)
authority = MissionAuthority(store, principal_bindings=[MissionPrincipalBinding(
authenticated_identity=identity, owner_principal_id=identity,
acceptance_verifier_id="identity-lineage-live-closure-v1",
allowed_task_capabilities=(
"code.read", "code.write", "test.execute",
"governed.write.file", "governed.write.branch",
),
max_risk="medium",
)])
service = TaskPlaneService(store, policy=PolicyEngine(
identity_verifier=IdentityVerifier({identity}), execution_enabled=False,
identity_capabilities={identity: {"task.creator", "task.read", "task.delegate"}},
), mission_authority=authority)
ingress = SignedMissionIngress(service, verifier)
# SQLite's existing connection and transaction lock share one request thread.
HTTPServer(("127.0.0.1", args.port), make_handler(ingress)).serve_forever()
if __name__ == "__main__":
main()
@@ -0,0 +1,511 @@
"""Authenticated, trigger-only import of authoritative terminal proof.
The caller authorizes reconciliation but supplies no proof fields. Proof is
read by a configured server-side provider and is accepted only after the
provider validates an authenticated authority document and its immutable
cross-system tuple.
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
from hashlib import sha256
import json
import re
from typing import Any, Callable, Mapping, Protocol
from .aios_task_lineage_integration import LINEAGE_METADATA_KEY
from .task_plane.errors import ConflictError, ForbiddenError, ValidationError
from .task_plane.models import ResultEnvelope, TaskRecord
from .task_plane.policy import PolicyDecision
PROOF_SCHEMA = "aios.task-terminal-authority-proof.v1"
RECEIPT_SCHEMA = "aios.task-terminal-import-receipt.v1"
REQUIRED_SOURCES = frozenset(
{
"agent_task",
"admission",
"job_producer",
"execution_alias",
"governed_job",
"bridge",
"authority_git",
}
)
DOCUMENT_FIELDS = frozenset(
{
"schema",
"task_id",
"mission_id",
"root_task_id",
"parent_task_id",
"acceptance_condition",
"agent_task_status",
"admission_status",
"execution_outcome",
"external_job_id",
"internal_job_id",
"repository",
"base_revision",
"target_branch",
"generation",
"execution_plan_digest",
"task_principal_ref",
"task_lease_ref",
"writer_lease_ref",
"governed_commit_oid",
"authority_ref_oid",
"postimage_hashes",
"source_receipt_digests",
"attestation_ref",
"started_at",
"completed_at",
}
)
_HEX64 = re.compile(r"[0-9a-f]{64}")
_GIT_OID = re.compile(r"[0-9a-f]{40}(?:[0-9a-f]{24})?")
_IDENTIFIER = re.compile(r"[A-Za-z0-9][A-Za-z0-9._:/-]{2,255}")
def _canonical_bytes(value: Mapping[str, object]) -> bytes:
return json.dumps(
dict(value), ensure_ascii=False, sort_keys=True, separators=(",", ":")
).encode("utf-8")
def _nonempty_string(value: object, field: str) -> str:
if not isinstance(value, str) or not value or len(value) > 1024:
raise ForbiddenError(f"authoritative proof {field} invalid")
return value
def _identifier(value: object, field: str) -> str:
text = _nonempty_string(value, field)
if _IDENTIFIER.fullmatch(text) is None:
raise ForbiddenError(f"authoritative proof {field} invalid")
return text
def _utc_time(value: object, field: str) -> str:
text = _nonempty_string(value, field)
try:
parsed = datetime.fromisoformat(text.replace("Z", "+00:00"))
except ValueError as exc:
raise ForbiddenError(f"authoritative proof {field} invalid") from exc
if parsed.tzinfo is None or parsed.utcoffset() is None:
raise ForbiddenError(f"authoritative proof {field} lacks timezone")
return parsed.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")
@dataclass(frozen=True)
class VerifiedTerminalProof:
document: dict[str, object]
@property
def task_id(self) -> str:
return str(self.document["task_id"])
@property
def attestation_ref(self) -> str:
return str(self.document["attestation_ref"])
@property
def started_at(self) -> str:
return str(self.document["started_at"])
@property
def completed_at(self) -> str:
return str(self.document["completed_at"])
def evidence(self) -> dict[str, object]:
return {
"kind": "acceptance-proof",
"proof_schema": PROOF_SCHEMA,
"authority_document_digest": sha256(
_canonical_bytes(self.document)
).hexdigest(),
"task_id": self.task_id,
"mission_id": self.document["mission_id"],
"root_task_id": self.document["root_task_id"],
"parent_task_id": self.document["parent_task_id"],
"acceptance_condition": self.document["acceptance_condition"],
"external_job_id": self.document["external_job_id"],
"internal_job_id": self.document["internal_job_id"],
"repository": self.document["repository"],
"base_revision": self.document["base_revision"],
"target_branch": self.document["target_branch"],
"generation": self.document["generation"],
"execution_plan_digest": self.document["execution_plan_digest"],
"task_principal_ref": self.document["task_principal_ref"],
"task_lease_ref": self.document["task_lease_ref"],
"writer_lease_ref": self.document["writer_lease_ref"],
"governed_commit_oid": self.document["governed_commit_oid"],
"authority_ref_oid": self.document["authority_ref_oid"],
"postimage_hashes": self.document["postimage_hashes"],
"source_receipt_digests": self.document["source_receipt_digests"],
"attestation_ref": self.attestation_ref,
}
def outputs(self) -> list[dict[str, object]]:
return [
{
"kind": "governed-commit-readback",
"repository": self.document["repository"],
"target_branch": self.document["target_branch"],
"governed_commit_oid": self.document["governed_commit_oid"],
"postimage_hashes": self.document["postimage_hashes"],
}
]
class AuthorityDocumentReader(Protocol):
def __call__(self, task_id: str) -> Mapping[str, object]: ...
class AuthorityDocumentVerifier(Protocol):
def __call__(self, document: Mapping[str, object]) -> bool: ...
class AuthenticatedTerminalProofProvider:
"""Read and validate one aggregate document backed by existing authorities.
``reader`` and ``verifier`` are mandatory runtime bindings. The verifier
must authenticate the referenced AgentTask/admission/Job/Bridge/Git
receipts using already configured trust roots. This class does not confer
trust on an arbitrary mapping.
"""
def __init__(
self,
*,
reader: AuthorityDocumentReader,
verifier: AuthorityDocumentVerifier,
task_reader: Callable[[str], TaskRecord],
worker_identity: str,
) -> None:
if not callable(reader) or not callable(verifier) or not callable(task_reader):
raise ValueError("authenticated proof reader, verifier and task reader required")
self._reader = reader
self._verifier = verifier
self._task_reader = task_reader
self.worker_identity = _identifier(worker_identity, "worker_identity")
@staticmethod
def _parse(raw: Mapping[str, object]) -> dict[str, object]:
if not isinstance(raw, Mapping) or set(raw) != DOCUMENT_FIELDS:
raise ForbiddenError("authoritative proof document shape invalid")
data = dict(raw)
if data["schema"] != PROOF_SCHEMA:
raise ForbiddenError("authoritative proof schema invalid")
for field in (
"task_id", "mission_id", "root_task_id", "parent_task_id",
"external_job_id", "internal_job_id", "task_principal_ref", "task_lease_ref",
"writer_lease_ref",
):
data[field] = _identifier(data[field], field)
for field in ("acceptance_condition", "repository", "base_revision", "target_branch"):
data[field] = _nonempty_string(data[field], field)
if data["agent_task_status"] != "SUCCEEDED":
raise ForbiddenError("AgentTask is not authoritatively successful")
if data["admission_status"] != "ADMITTED":
raise ForbiddenError("task admission is not authoritative")
if data["execution_outcome"] != "APPLIED_VERIFIED":
raise ForbiddenError("governed execution outcome is not verified")
if type(data["generation"]) is not int or int(data["generation"]) < 1:
raise ForbiddenError("authoritative proof generation invalid")
if not isinstance(data["execution_plan_digest"], str) or _HEX64.fullmatch(
data["execution_plan_digest"]
) is None:
raise ForbiddenError("authoritative execution plan digest invalid")
for field in ("governed_commit_oid", "authority_ref_oid"):
if not isinstance(data[field], str) or _GIT_OID.fullmatch(data[field]) is None:
raise ForbiddenError(f"authoritative proof {field} invalid")
if data["governed_commit_oid"] != data["authority_ref_oid"]:
raise ForbiddenError("authority ref does not read back governed commit")
images = data["postimage_hashes"]
if (
not isinstance(images, Mapping)
or not images
or len(images) > 256
or any(
not isinstance(path, str)
or not path
or path.startswith(("/", "\\"))
or ".." in path.replace("\\", "/").split("/")
or not isinstance(digest, str)
or _HEX64.fullmatch(digest) is None
for path, digest in images.items()
)
):
raise ForbiddenError("authoritative postimage hashes invalid")
data["postimage_hashes"] = dict(sorted(images.items()))
sources = data["source_receipt_digests"]
if (
not isinstance(sources, Mapping)
or set(sources) != REQUIRED_SOURCES
or any(
not isinstance(digest, str) or _HEX64.fullmatch(digest) is None
for digest in sources.values()
)
):
raise ForbiddenError("authoritative source receipt set invalid")
data["source_receipt_digests"] = dict(sorted(sources.items()))
data["attestation_ref"] = _nonempty_string(data["attestation_ref"], "attestation_ref")
data["started_at"] = _utc_time(data["started_at"], "started_at")
data["completed_at"] = _utc_time(data["completed_at"], "completed_at")
if datetime.fromisoformat(data["completed_at"].replace("Z", "+00:00")) < datetime.fromisoformat(
data["started_at"].replace("Z", "+00:00")
):
raise ForbiddenError("authoritative proof time order invalid")
return data
@staticmethod
def _assert_task_binding(task: TaskRecord, data: Mapping[str, object]) -> None:
lineage = task.task_plan.metadata.get(LINEAGE_METADATA_KEY)
if not isinstance(lineage, Mapping) or lineage.get("relation_type") == "ROOT":
raise ForbiddenError("terminal proof requires a canonical lineage child")
expected = {
"task_id": task.task_id,
"mission_id": task.mission_id,
"root_task_id": task.root_task_id,
"parent_task_id": task.task_plan.parent_task_id,
"acceptance_condition": task.task_plan.metadata.get("acceptance_condition"),
}
if any(data.get(field) != value or value in (None, "") for field, value in expected.items()):
raise ForbiddenError("authoritative proof TaskPlane binding mismatch")
if any(lineage.get(field) != expected[field] for field in expected if field != "acceptance_condition"):
raise ForbiddenError("authoritative proof lineage binding mismatch")
def collect(self, task: TaskRecord) -> VerifiedTerminalProof:
raw = self._reader(task.task_id)
if not isinstance(raw, Mapping) or self._verifier(raw) is not True:
raise ForbiddenError("authoritative terminal proof authentication failed")
data = self._parse(raw)
self._assert_task_binding(task, data)
return VerifiedTerminalProof(data)
def verify_evidence(self, task: TaskRecord, evidences: list[dict]) -> bool:
proof = self.collect(task)
return evidences == [proof.evidence()]
def verify_result(self, result: ResultEnvelope) -> bool:
if result.worker_identity != self.worker_identity or result.status != "COMPLETED":
return False
try:
task = self._task_reader(result.task_id)
proof = self.collect(task)
except Exception:
return False
return bool(
result.attestation_ref == proof.attestation_ref
and result.evidence == [proof.evidence()]
and result.outputs == proof.outputs()
and result.started_at == proof.started_at
and result.completed_at == proof.completed_at
and result.operation_digest == task.task_plan.operation_digest()
)
class TerminalProofEvidenceVerifier:
"""PolicyEngine evidence verifier backed by authenticated authority reads."""
def __init__(self, provider: AuthenticatedTerminalProofProvider) -> None:
self.provider = provider
def verify(self, task: TaskRecord, evidences: list[dict]) -> PolicyDecision:
try:
allowed = self.provider.verify_evidence(task, evidences)
except Exception:
allowed = False
return PolicyDecision(allowed, "" if allowed else "terminal proof evidence invalid")
class TerminalProofImporter:
"""Reconcile a server-verified proof through TaskPlane claim/result paths."""
def __init__(
self,
service: Any,
*,
provider: AuthenticatedTerminalProofProvider | None,
worker_id: str | None,
worker_identity: str | None,
lease_seconds: int = 300,
) -> None:
self.service = service
self.provider = provider
self.worker_id = worker_id
self.worker_identity = worker_identity
if provider is not None:
if not worker_id or not worker_identity:
raise ValueError("terminal proof worker binding required")
_identifier(worker_id, "worker_id")
_identifier(worker_identity, "worker_identity")
if provider.worker_identity != worker_identity:
raise ValueError("terminal proof provider identity mismatch")
if not 30 <= int(lease_seconds) <= 3600:
raise ValueError("terminal proof lease out of range")
self.lease_seconds = int(lease_seconds)
@staticmethod
def expected_idempotency_key(task: TaskRecord) -> str:
return f"terminal-proof:{task.mission_id}:{task.task_id}"
def _authorized_task(self, task_id: str, actor_identity: str) -> TaskRecord:
self.service.get_task(task_id, actor_identity)
task = self.service.store.get_task(task_id)
lineage = task.task_plan.metadata.get(LINEAGE_METADATA_KEY)
if (
not isinstance(lineage, Mapping)
or lineage.get("relation_type") == "ROOT"
or lineage.get("task_id") != task.task_id
or lineage.get("mission_id") != task.mission_id
or lineage.get("root_task_id") != task.root_task_id
or lineage.get("parent_task_id") != task.task_plan.parent_task_id
):
raise ForbiddenError("terminal proof target is not a canonical child")
return task
def _existing_receipt(
self, task: TaskRecord, actor_identity: str
) -> dict[str, object] | None:
if self.service.lineage is None:
raise ForbiddenError("TaskPlane lineage is not configured")
snapshot = self.service.lineage.binding.get_task_snapshot(task.task_id)
if not snapshot.terminal_evidence_proven:
return None
if snapshot.status != "COMPLETED":
raise ConflictError("terminal proof target did not complete successfully")
review = self.service.lineage.binding.get_root_review_for_trigger(task.task_id)
if review is None:
raise ConflictError("terminal proof lacks durable root review")
evidence_rows = self.service.store.list_evidence(task.task_id)
by_id = {str(row.get("evidence_id")): row for row in evidence_rows}
if not snapshot.evidence_refs or any(ref not in by_id for ref in snapshot.evidence_refs):
raise ConflictError("terminal proof evidence readback mismatch")
envelopes = []
for ref in snapshot.evidence_refs:
try:
envelope = json.loads(str(by_id[ref]["envelope_json"]))
except (KeyError, TypeError, ValueError, json.JSONDecodeError) as exc:
raise ConflictError("terminal proof evidence readback invalid") from exc
envelopes.append(envelope)
claim_ids = {item.get("claim_id") for item in envelopes}
evidence_payloads = [item.get("payload") for item in envelopes]
if (
len(claim_ids) != 1
or len(evidence_payloads) != 1
or any(item.get("worker_identity") != self.worker_identity for item in envelopes)
or envelopes[0].get("kind") != "acceptance-proof"
or not isinstance(evidence_payloads[0], dict)
or not self.provider.verify_evidence(task, evidence_payloads)
or not self.service.store.verify_evidence_integrity(task.task_id)
or not self.service.store.verify_event_chain(task.task_id)
):
raise ConflictError("terminal proof receipt integrity failed")
claim = self.service.store.get_claim(str(next(iter(claim_ids))))
if (
claim.task_id != task.task_id
or claim.worker_id != self.worker_id
or claim.worker_identity != self.worker_identity
or claim.operation_digest != task.task_plan.operation_digest()
or claim.status != "COMPLETED"
):
raise ConflictError("terminal proof claim binding mismatch")
dispatch = self.service.lineage.binding.get_root_action_dispatch(str(review["review_id"]))
if dispatch is None:
raise ConflictError("terminal proof lacks durable root action dispatch")
body: dict[str, object] = {
"schema": RECEIPT_SCHEMA,
"task_id": task.task_id,
"mission_id": task.mission_id,
"root_task_id": task.root_task_id,
"claim_id": next(iter(claim_ids)),
"task_status": snapshot.status,
"task_revision": snapshot.revision,
"evidence_refs": list(snapshot.evidence_refs),
"review_id": review["review_id"],
"root_revision_after": review["root_revision_after"],
"next_legal_action": review["next_legal_action"],
"dispatch_status": dispatch.status,
"dispatch_idempotency_key": dispatch.idempotency_key,
}
body["receipt_digest"] = sha256(_canonical_bytes(body)).hexdigest()
return body
def _exact_active_claim(self, task: TaskRecord):
active = self.service.store.active_claim(task.task_id)
if active is None:
return None
if (
active.worker_id != self.worker_id
or active.worker_identity != self.worker_identity
or active.operation_digest != task.task_plan.operation_digest()
):
raise ConflictError("active claim is owned by another execution binding")
return active
def reconcile(
self, task_id: str, idempotency_key: str | None, *, actor_identity: str
) -> dict[str, object]:
if self.provider is None:
raise ForbiddenError("terminal proof provider is not configured")
task = self._authorized_task(task_id, actor_identity)
if idempotency_key != self.expected_idempotency_key(task):
raise ValidationError("terminal proof idempotency key mismatch")
existing = self._existing_receipt(task, actor_identity)
if existing is not None:
return existing
proof = self.provider.collect(task)
claim = self._exact_active_claim(task)
if claim is None:
try:
claimed = self.service.claim(
task.task_id,
{
"worker_id": self.worker_id,
"lease_seconds": self.lease_seconds,
"actor_identity": self.worker_identity,
},
)
claim_id = str(claimed["claim_id"])
except Exception:
refreshed = self._authorized_task(task_id, actor_identity)
replay = self._existing_receipt(refreshed, actor_identity)
if replay is not None:
return replay
claim = self._exact_active_claim(refreshed)
if claim is None:
raise
claim_id = claim.claim_id
else:
claim_id = claim.claim_id
try:
self.service.complete(
task.task_id,
{
"claim_id": claim_id,
"actor_identity": self.worker_identity,
"status": "COMPLETED",
"outputs": proof.outputs(),
"evidence": [proof.evidence()],
"started_at": proof.started_at,
"completed_at": proof.completed_at,
"attestation_ref": proof.attestation_ref,
},
)
except Exception:
refreshed = self._authorized_task(task_id, actor_identity)
replay = self._existing_receipt(refreshed, actor_identity)
if replay is None:
raise
return replay
refreshed = self._authorized_task(task_id, actor_identity)
receipt = self._existing_receipt(refreshed, actor_identity)
if receipt is None:
raise ConflictError("terminal proof completion lacks authoritative readback")
return receipt
@@ -15,7 +15,7 @@ import struct
import subprocess
import time
from datetime import datetime, timezone
from dataclasses import dataclass
from dataclasses import dataclass, replace
from pathlib import Path
from typing import Any, Callable, Mapping, Protocol
@@ -133,6 +133,8 @@ class TrustedCandidateBuilderConfig:
recipes: Mapping[str, Mapping[str, str]]
governed_recipes: Mapping[str, Mapping[str, str]]
policy_sha256: str
source_admission_resolver: Callable[[str, str | None, str], Mapping[str, str]] | None = None
migration_plan_resolver: Callable[[str, str, str], Any] | None = None
@classmethod
def from_policy(cls, policy: Mapping[str, Any]) -> "TrustedCandidateBuilderConfig":
@@ -227,14 +229,63 @@ class TrustedCandidateBuilderConfig:
return recipe
def resolve_recipe(self, version: str, candidate_id: str | None, source_commit: str) -> Mapping[str, str]:
recipe = self.fixed_recipe(version)
# Intent-only callers do not choose a candidate identity. The
# policy-owned recipe supplies it; explicit worker requests are
# still checked against the same immutable recipe.
effective_candidate_id = str(candidate_id or recipe.get("candidate_id") or "")
if {"version": recipe.get("version"), "candidate_id": recipe.get("candidate_id"), "source_commit": recipe.get("source_commit")} != {"version": version, "candidate_id": effective_candidate_id, "source_commit": source_commit}:
recipe = self.recipes.get(version)
if recipe is not None:
resolved = self.fixed_recipe(version)
# Fixed/governed-derived policy recipes remain immutable. A
# source admission can never override one of these identities.
effective_candidate_id = str(candidate_id or resolved.get("candidate_id") or "")
if {
"version": resolved.get("version"),
"candidate_id": resolved.get("candidate_id"),
"source_commit": resolved.get("source_commit"),
} != {
"version": version,
"candidate_id": effective_candidate_id,
"source_commit": source_commit,
}:
raise BuilderError("REQUEST_RELEASE_BINDING_INVALID")
return resolved
resolver = self.source_admission_resolver
if resolver is None:
raise BuilderError("RECIPE_UNKNOWN")
try:
admitted = dict(resolver(version, candidate_id, source_commit))
except Exception as exc:
raise BuilderError("SOURCE_ADMISSION_REQUIRED") from exc
required = {
"mode",
"task_id",
"candidate_id",
"base_oid",
"source_commit",
"version",
"purpose",
}
if (
set(admitted) != required
or admitted.get("mode") != "governed-source-admission"
or admitted.get("version") != version
or admitted.get("source_commit") != source_commit
or not isinstance(admitted.get("candidate_id"), str)
or not admitted.get("candidate_id")
):
raise BuilderError("SOURCE_ADMISSION_INVALID")
effective_candidate_id = str(candidate_id or admitted["candidate_id"])
if effective_candidate_id != admitted["candidate_id"]:
raise BuilderError("REQUEST_RELEASE_BINDING_INVALID")
return recipe
for name in ("base_oid", "source_commit"):
value = admitted.get(name)
if (
not isinstance(value, str)
or len(value) != 40
or any(char not in "0123456789abcdef" for char in value.lower())
):
raise BuilderError("SOURCE_ADMISSION_INVALID")
if admitted.get("purpose") != "AIOS_TRUSTED_RELEASE_BUILDER":
raise BuilderError("SOURCE_ADMISSION_INVALID")
return admitted
def _invalid_governed_recipes() -> Mapping[str, Mapping[str, str]]:
@@ -297,6 +348,35 @@ class ServerOwnedEvidenceSource:
raise BuilderError("COLLECTOR_EVIDENCE_OWNER_DENIED")
return resolved
def assert_ready(self) -> None:
"""Validate the server-owned evidence facility without requiring recipe evidence.
Readiness is an infrastructure/property check. Candidate-specific evidence
remains mandatory for ``BuildTrustedCandidateV1`` and is validated by
:meth:`__call__`; an empty evidence root therefore no longer makes the
builder globally unavailable.
"""
for path, code in (
(self._authority_git_dir, "COLLECTOR_AUTHORITY_GIT_UNAVAILABLE"),
(self._evidence_root, "COLLECTOR_EVIDENCE_ROOT_UNAVAILABLE"),
):
try:
metadata = path.stat(follow_symlinks=False)
except OSError as exc:
raise BuilderError(code) from exc
if not stat.S_ISDIR(metadata.st_mode) or path.is_symlink():
raise BuilderError(code)
if path == self._evidence_root:
if os.name != "nt" and metadata.st_mode & 0o022:
raise BuilderError("COLLECTOR_EVIDENCE_ROOT_MODE_DENIED")
if (
os.name != "nt"
and self._expected_owner_uid is not None
and metadata.st_uid != self._expected_owner_uid
):
raise BuilderError("COLLECTOR_EVIDENCE_ROOT_OWNER_DENIED")
def _digest(self, path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
@@ -436,9 +516,17 @@ class DeterministicEvidenceCollector:
self._config = config
self._evidence_source = evidence_source
def collect(self, recipe_name: str, recipe: Mapping[str, str]) -> Mapping[str, Any]:
def assert_ready(self) -> None:
"""Fail closed on collector configuration, not on recipe evidence presence."""
if self._evidence_source is None:
raise BuilderError("COLLECTOR_UNCONFIGURED")
checker = getattr(self._evidence_source, "assert_ready", None)
if checker is not None:
checker()
def collect(self, recipe_name: str, recipe: Mapping[str, str]) -> Mapping[str, Any]:
self.assert_ready()
bundle = self._evidence_source(recipe_name, recipe)
if not isinstance(bundle, Mapping) or set(bundle) != self._BUNDLE_FIELDS:
raise BuilderError("COLLECTOR_EVIDENCE_INVALID")
@@ -464,11 +552,32 @@ class DeterministicEvidenceCollector:
try:
from silent_release_controller import ReleaseCandidate
from silent_release_hardening import ReleasePlan, TrustedCandidateBundle
from release_migration_plan import (
MigrationPlanError,
MigrationPlanRequired,
)
candidate = ReleaseCandidate.from_dict(dict(bundle["candidate"]))
release_plan = ReleasePlan.from_dict(dict(bundle["release_plan"]))
release_plan_payload = dict(bundle["release_plan"])
migration_resolver = self._config.migration_plan_resolver
if migration_resolver is not None:
try:
migration = migration_resolver(
recipe_name,
recipe["candidate_id"],
recipe["source_commit"],
)
except MigrationPlanRequired as exc:
raise BuilderError("MIGRATION_PLAN_REQUIRED") from exc
except MigrationPlanError as exc:
raise BuilderError("MIGRATION_PLAN_INVALID") from exc
if migration is not None:
release_plan_payload["migration"] = migration.to_dict()
release_plan = ReleasePlan.from_dict(release_plan_payload)
checks = dict(raw_checks)
TrustedCandidateBundle(candidate, release_plan, checks).validate()
except BuilderError:
raise
except (ImportError, KeyError, TypeError, ValueError, RuntimeError) as exc:
raise BuilderError("COLLECTOR_EVIDENCE_INVALID") from exc
if candidate.source_commit != recipe["source_commit"] or candidate.candidate_id != recipe["candidate_id"]:
@@ -622,14 +731,12 @@ class TrustedCandidateBuilder:
bound = {"repository": self.config.repository, "task_id": recipe["task_id"], "candidate_id": recipe["candidate_id"], "base_oid": recipe["base_oid"], "source_commit": recipe["source_commit"], "version": recipe["version"], "purpose": recipe["purpose"]}
if {key: request.get(key) for key in bound} != bound:
raise BuilderError("REQUEST_SCOPE_INVALID")
bundle = dict(self.collector.collect(recipe_name, recipe))
candidate = bundle["candidate"]
release_plan = bundle["release_plan"]
checks = bundle.get("checks")
evidence_digests = bundle["evidence_digests"]
if not isinstance(checks, Mapping) or set(checks) != set(self.config.required_checks) or not all(value is True for value in checks.values()):
raise BuilderError("COLLECTOR_CHECKS_INCOMPLETE")
if request["operation"] == self.config.readiness_operation:
self.collector.assert_ready()
# Accessing the identity properties validates that the signer is
# configured and usable without invoking the private key.
builder_key_id = self.signer.key_id
builder_key_generation = self.signer.key_generation
result = {
"schema": self.config.response_schema,
"status": "READY",
@@ -637,8 +744,8 @@ class TrustedCandidateBuilder:
"recipe": recipe_name,
"candidate_id": recipe["candidate_id"],
"source_commit": recipe["source_commit"],
"builder_key_id": self.signer.key_id,
"builder_key_generation": self.signer.key_generation,
"builder_key_id": builder_key_id,
"builder_key_generation": builder_key_generation,
"signature_algorithm": "Ed25519",
"signature_purpose": "AIOS_TRUSTED_RELEASE_BUILDER",
"policy_sha256": self.config.policy_sha256,
@@ -653,6 +760,15 @@ class TrustedCandidateBuilder:
)
self._persist_nonce_ledger()
return result
bundle = dict(self.collector.collect(recipe_name, recipe))
candidate = bundle["candidate"]
release_plan = bundle["release_plan"]
checks = bundle.get("checks")
evidence_digests = bundle["evidence_digests"]
if not isinstance(checks, Mapping) or set(checks) != set(self.config.required_checks) or not all(value is True for value in checks.values()):
raise BuilderError("COLLECTOR_CHECKS_INCOMPLETE")
issued = datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
attestation = {
"builder_attestation_id": "builder-attestation-" + _sha256(recipe_name, recipe["task_id"], candidate["artifact_sha256"])[:24],
@@ -776,6 +892,50 @@ def _main() -> None:
if not credentials:
raise SystemExit("CREDENTIALS_DIRECTORY is required")
keyring = BuilderKeyring.from_credentials_directory(Path(credentials))
source_admission_root = os.environ.get(
"AIOS_TRUSTED_BUILDER_SOURCE_ADMISSION_ROOT", ""
).strip()
if source_admission_root:
from release_source_admission import open_source_admission_store
try:
source_admission_hmac = (
Path(credentials) / "source_admission_hmac"
).read_bytes().strip()
except OSError as exc:
raise BuilderError("SOURCE_ADMISSION_HMAC_UNAVAILABLE") from exc
source_store = open_source_admission_store(
Path(source_admission_root),
repository=config.repository,
policy=arguments.policy,
hmac_key=source_admission_hmac,
expected_owner_uid=0,
writable=False,
)
config = replace(
config,
source_admission_resolver=source_store.resolve_recipe,
)
migration_plan_root = os.environ.get(
"AIOS_TRUSTED_BUILDER_MIGRATION_PLAN_ROOT", ""
).strip()
migration_profile_root = os.environ.get(
"AIOS_TRUSTED_BUILDER_MIGRATION_PROFILE_ROOT", ""
).strip()
if bool(migration_plan_root) != bool(migration_profile_root):
raise BuilderError("MIGRATION_PLAN_CONFIGURATION_INVALID")
if migration_plan_root:
from release_migration_plan import FileMigrationPlanResolver
migration_resolver = FileMigrationPlanResolver(
Path(migration_plan_root),
profile_root=Path(migration_profile_root),
expected_owner_uid=0,
)
config = replace(
config,
migration_plan_resolver=migration_resolver.resolve,
)
signer = CredentialPathEd25519Signer(
keyring.signing_private_key_path,
key_id=os.environ["AIOS_TRUSTED_BUILDER_KEY_ID"],
+40 -8
View File
@@ -4,6 +4,7 @@ import hashlib, html, json, os, sqlite3, time, uuid, urllib.parse
from http import cookies
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
import hmac # SEC-P0-BOOTSTRAP-CREDENTIAL
ROOT = Path(os.getenv('AIOS_WEBOPS_ROOT', Path(__file__).resolve().parents[1]))
DB_PATH = Path(os.getenv('AIOS_WEBOPS_DB', ROOT / 'data' / 'webops.sqlite3'))
@@ -13,7 +14,11 @@ PUBLIC_DOMAIN = os.getenv('AIOS_PUBLIC_DOMAIN','https://cnnetmkt.top')
WEBOPS_DOMAIN = os.getenv('AIOS_WEBOPS_DOMAIN','https://ops.cnnetmkt.top')
API_DOMAIN = os.getenv('AIOS_API_DOMAIN','https://api.cnnetmkt.top')
WECHAT_DOMAIN = os.getenv('AIOS_WECHAT_CALLBACK_DOMAIN','https://wechat.cnnetmkt.top')
SECRET = os.getenv('AIOS_WEBOPS_SECRET','change-me-before-production')
SECRET = os.getenv('AIOS_WEBOPS_SECRET') or ''
# SEC-P0: fail closed — never boot with a missing/insecure default secret.
if SECRET in ('', 'change-me-before-production'):
raise SystemExit('AIOS_WEBOPS_SECRET not set (or insecure default). Refusing to start: fail closed.')
SESSION_TTL = int(os.getenv('AIOS_SESSION_TTL','3600'))
ROLES = ['owner','admin','user']
NODES = ['server-hw','server-local']
PAGES = [('/dashboard','Dashboard'),('/nodes','Nodes'),('/roles','Roles'),('/routing','Routing'),('/failover','Failover'),('/approvals','Approvals'),('/operations','Operations'),('/users','Users'),('/quota','Quota'),('/models','Models'),('/memory','Memory'),('/channels/wechat','WeChat'),('/life','Life Core'),('/patches','Patch Studio'),('/optimization','Optimization'),('/simulation','Simulation'),('/logs','Logs'),('/config','Config'),('/uat','UAT'),('/versions','Versions'),('/evidence','Evidence')]
@@ -43,19 +48,46 @@ def audit(actor, action, target='', payload=None):
exec_sql('INSERT INTO audit_logs(id,actor,action,target,payload,created_at) VALUES(?,?,?,?,?,?)',(uid('audit'),actor,action,target,jdump(payload or {}),now()))
def notify(event_type,payload):
exec_sql('INSERT INTO owner_notifications(id,event_type,payload,status,created_at) VALUES(?,?,?,?,?)',(uid('notify'),event_type,jdump(payload),'queued',now()))
def hash_pw(p): return hashlib.sha256((SECRET+'|'+p).encode()).hexdigest()
def check_pw(stored,p): return stored in (p, hash_pw(p))
_PBKDF2_ITERS = int(os.getenv('AIOS_PBKDF2_ITERS','260000'))
def _is_pbkdf2(stored): return isinstance(stored,str) and stored.startswith('pbkdf2$') and stored.count('$')==2
def hash_pw(p):
salt=os.urandom(16).hex()
dk=hashlib.pbkdf2_hmac('sha256', p.encode(), bytes.fromhex(salt), _PBKDF2_ITERS).hex()
return f'pbkdf2${salt}${dk}'
def check_pw(stored,p):
if not stored or not isinstance(stored,str) or not stored: return False
if _is_pbkdf2(stored):
_,salt,dk=stored.split('$')
calc=hashlib.pbkdf2_hmac('sha256', p.encode(), bytes.fromhex(salt), _PBKDF2_ITERS).hex()
return hmac.compare_digest(dk,calc)
# Legacy record path: NEVER accept the plaintext form as a credential.
# A legacy (unsalted) sha256 is honored ONLY as a one-time owner migration
# credential and never for the known bootstrap default; anything else is
# rejected so the account must be re-seeded via a secure channel (fail closed).
if stored == p: return False
try:
old=hashlib.sha256((SECRET+'|'+p).encode()).hexdigest()
except Exception:
return False
return (hmac.compare_digest(old,stored)) and (p not in {'bootstrap-change-me'})
def token(username,role):
data=f'{username}|{role}|{int(time.time())}'; sig=hashlib.sha256((SECRET+'|'+data).encode()).hexdigest()[:16]; return data+'|'+sig
exp=int(time.time())+SESSION_TTL
data=f'{username}|{role}|{exp}'
sig=hmac.new(SECRET.encode(), data.encode(), hashlib.sha256).hexdigest()
return data+'|'+sig
def parse_session(header):
if not header: return None
c=cookies.SimpleCookie(); c.load(header)
if 'aios_session' not in c: return None
parts=c['aios_session'].value.split('|')
if len(parts)!=4: return None
data='|'.join(parts[:3]); sig=parts[3]
if hashlib.sha256((SECRET+'|'+data).encode()).hexdigest()[:16]!=sig: return None
return one('SELECT * FROM users WHERE username=? AND role=? AND status=?',(parts[0],parts[1],'approved'))
username,role,exp,sig=parts
data=f'{username}|{role}|{exp}'
if not hmac.compare_digest(hmac.new(SECRET.encode(), data.encode(), hashlib.sha256).hexdigest(), sig): return None
try:
if int(exp) <= int(time.time()): return None
except Exception: return None
return one('SELECT * FROM users WHERE username=? AND role=? AND status=?',(username,role,'approved'))
def require(user, roles): return bool(user and user['role'] in roles)
def table(data):
if not data: return '<p class="muted">No records.</p>'
@@ -215,7 +247,7 @@ class H(BaseHTTPRequestHandler):
if path=='/login':
n=int(self.headers.get('Content-Length','0') or 0); data=urllib.parse.parse_qs(self.rfile.read(n).decode())
u=one('SELECT * FROM users WHERE username=? AND status=?',(data.get('username',[''])[0],'approved'))
if u and check_pw(u.get('password_hash') or '',data.get('password',[''])[0]): audit(u['username'],'login','webops',{}); self.redirect('/dashboard',[f"aios_session={token(u['username'],u['role'])}; Path=/; HttpOnly; SameSite=Lax"]); return
if u and check_pw(u.get('password_hash') or '',data.get('password',[''])[0]): audit(u['username'],'login','webops',{}); self.redirect('/dashboard',[f"aios_session={token(u['username'],u['role'])}; Path=/; HttpOnly; SameSite=Strict"]); return
self.send_body(403,layout('Login Failed','<p>登录失败。</p>',None)); return
if path=='/register':
n=int(self.headers.get('Content-Length','0') or 0); data=urllib.parse.parse_qs(self.rfile.read(n).decode())
+40 -8
View File
@@ -4,6 +4,7 @@ import hashlib, html, json, os, sqlite3, time, uuid, urllib.parse
from http import cookies
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
import hmac # SEC-P0-BOOTSTRAP-CREDENTIAL
ROOT = Path(os.getenv('AIOS_WEBOPS_ROOT', Path(__file__).resolve().parents[1]))
DB_PATH = Path(os.getenv('AIOS_WEBOPS_DB', ROOT / 'data' / 'webops.sqlite3'))
@@ -13,7 +14,11 @@ PUBLIC_DOMAIN = os.getenv('AIOS_PUBLIC_DOMAIN','https://cnnetmkt.top')
WEBOPS_DOMAIN = os.getenv('AIOS_WEBOPS_DOMAIN','https://ops.cnnetmkt.top')
API_DOMAIN = os.getenv('AIOS_API_DOMAIN','https://api.cnnetmkt.top')
WECHAT_DOMAIN = os.getenv('AIOS_WECHAT_CALLBACK_DOMAIN','https://wechat.cnnetmkt.top')
SECRET = os.getenv('AIOS_WEBOPS_SECRET','change-me-before-production')
SECRET = os.getenv('AIOS_WEBOPS_SECRET') or ''
# SEC-P0: fail closed — never boot with a missing/insecure default secret.
if SECRET in ('', 'change-me-before-production'):
raise SystemExit('AIOS_WEBOPS_SECRET not set (or insecure default). Refusing to start: fail closed.')
SESSION_TTL = int(os.getenv('AIOS_SESSION_TTL','3600'))
ROLES = ['owner','admin','user']
NODES = ['server-hw','server-local']
PAGES = [('/dashboard','Dashboard'),('/nodes','Nodes'),('/roles','Roles'),('/routing','Routing'),('/failover','Failover'),('/approvals','Approvals'),('/operations','Operations'),('/users','Users'),('/quota','Quota'),('/models','Models'),('/memory','Memory'),('/channels/wechat','WeChat'),('/life','Life Core'),('/patches','Patch Studio'),('/optimization','Optimization'),('/simulation','Simulation'),('/logs','Logs'),('/config','Config'),('/uat','UAT'),('/versions','Versions'),('/evidence','Evidence')]
@@ -43,19 +48,46 @@ def audit(actor, action, target='', payload=None):
exec_sql('INSERT INTO audit_logs(id,actor,action,target,payload,created_at) VALUES(?,?,?,?,?,?)',(uid('audit'),actor,action,target,jdump(payload or {}),now()))
def notify(event_type,payload):
exec_sql('INSERT INTO owner_notifications(id,event_type,payload,status,created_at) VALUES(?,?,?,?,?)',(uid('notify'),event_type,jdump(payload),'queued',now()))
def hash_pw(p): return hashlib.sha256((SECRET+'|'+p).encode()).hexdigest()
def check_pw(stored,p): return stored in (p, hash_pw(p))
_PBKDF2_ITERS = int(os.getenv('AIOS_PBKDF2_ITERS','260000'))
def _is_pbkdf2(stored): return isinstance(stored,str) and stored.startswith('pbkdf2$') and stored.count('$')==2
def hash_pw(p):
salt=os.urandom(16).hex()
dk=hashlib.pbkdf2_hmac('sha256', p.encode(), bytes.fromhex(salt), _PBKDF2_ITERS).hex()
return f'pbkdf2${salt}${dk}'
def check_pw(stored,p):
if not stored or not isinstance(stored,str) or not stored: return False
if _is_pbkdf2(stored):
_,salt,dk=stored.split('$')
calc=hashlib.pbkdf2_hmac('sha256', p.encode(), bytes.fromhex(salt), _PBKDF2_ITERS).hex()
return hmac.compare_digest(dk,calc)
# Legacy record path: NEVER accept the plaintext form as a credential.
# A legacy (unsalted) sha256 is honored ONLY as a one-time owner migration
# credential and never for the known bootstrap default; anything else is
# rejected so the account must be re-seeded via a secure channel (fail closed).
if stored == p: return False
try:
old=hashlib.sha256((SECRET+'|'+p).encode()).hexdigest()
except Exception:
return False
return (hmac.compare_digest(old,stored)) and (p not in {'bootstrap-change-me'})
def token(username,role):
data=f'{username}|{role}|{int(time.time())}'; sig=hashlib.sha256((SECRET+'|'+data).encode()).hexdigest()[:16]; return data+'|'+sig
exp=int(time.time())+SESSION_TTL
data=f'{username}|{role}|{exp}'
sig=hmac.new(SECRET.encode(), data.encode(), hashlib.sha256).hexdigest()
return data+'|'+sig
def parse_session(header):
if not header: return None
c=cookies.SimpleCookie(); c.load(header)
if 'aios_session' not in c: return None
parts=c['aios_session'].value.split('|')
if len(parts)!=4: return None
data='|'.join(parts[:3]); sig=parts[3]
if hashlib.sha256((SECRET+'|'+data).encode()).hexdigest()[:16]!=sig: return None
return one('SELECT * FROM users WHERE username=? AND role=? AND status=?',(parts[0],parts[1],'approved'))
username,role,exp,sig=parts
data=f'{username}|{role}|{exp}'
if not hmac.compare_digest(hmac.new(SECRET.encode(), data.encode(), hashlib.sha256).hexdigest(), sig): return None
try:
if int(exp) <= int(time.time()): return None
except Exception: return None
return one('SELECT * FROM users WHERE username=? AND role=? AND status=?',(username,role,'approved'))
def require(user, roles): return bool(user and user['role'] in roles)
def table(data):
if not data: return '<p class="muted">No records.</p>'
@@ -215,7 +247,7 @@ class H(BaseHTTPRequestHandler):
if path=='/login':
n=int(self.headers.get('Content-Length','0') or 0); data=urllib.parse.parse_qs(self.rfile.read(n).decode())
u=one('SELECT * FROM users WHERE username=? AND status=?',(data.get('username',[''])[0],'approved'))
if u and check_pw(u.get('password_hash') or '',data.get('password',[''])[0]): audit(u['username'],'login','webops',{}); self.redirect('/dashboard',[f"aios_session={token(u['username'],u['role'])}; Path=/; HttpOnly; SameSite=Lax"]); return
if u and check_pw(u.get('password_hash') or '',data.get('password',[''])[0]): audit(u['username'],'login','webops',{}); self.redirect('/dashboard',[f"aios_session={token(u['username'],u['role'])}; Path=/; HttpOnly; SameSite=Strict"]); return
self.send_body(403,layout('Login Failed','<p>登录失败。</p>',None)); return
if path=='/register':
n=int(self.headers.get('Content-Length','0') or 0); data=urllib.parse.parse_qs(self.rfile.read(n).decode())
@@ -0,0 +1,349 @@
from __future__ import annotations
"""Authenticated Gitea consistency provider using the real governed Git transport.
The provider proves Gitea consistency without inventing a REST API:
* local HEAD == source_oid,
* local source_oid^{tree} == tree_oid,
* origin identifies the governed repository,
* exact checked-out branch exists remotely, and
* git ls-remote reports exactly source_oid for that branch.
It then writes an append-only root-owned v2 provider receipt using only governed
policy for provider identity, operation, store root, store id and modes.
The result explicitly records ``transport=git-ls-remote`` and
``api_verified=false``. It must never be described as REST/API evidence.
"""
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import json
import os
from pathlib import Path
import re
import stat
import subprocess
from typing import Callable, Mapping
from urllib.parse import urlsplit, urlunsplit
RECEIPT_SCHEMA = "aios.trusted-evidence-provider-receipt.v2"
RECEIPT_VERSION = 2
_OID_RE = re.compile(r"^[0-9a-f]{40}$")
_REF_RE = re.compile(r"^refs/heads/[A-Za-z0-9._/-]+$")
_REPO_RE = re.compile(r"^[A-Za-z0-9._-]+/[A-Za-z0-9._-]+$")
class GiteaProviderError(RuntimeError):
pass
@dataclass(frozen=True)
class GitCommandResult:
returncode: int
stdout: str = ""
stderr: str = ""
GitRunner = Callable[[tuple[str, ...], Path], GitCommandResult]
def canonical_json_bytes(payload: Mapping) -> bytes:
return json.dumps(
dict(payload),
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
def canonical_sha256(payload: Mapping) -> str:
return hashlib.sha256(canonical_json_bytes(payload)).hexdigest()
def _runner(args: tuple[str, ...], cwd: Path) -> GitCommandResult:
proc = subprocess.run(
list(args),
cwd=str(cwd),
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=45,
check=False,
)
return GitCommandResult(proc.returncode, proc.stdout, proc.stderr)
def _git(run: GitRunner, cwd: Path, *args: str, error: str) -> str:
result = run(("git", *args), cwd)
if result.returncode != 0:
raise GiteaProviderError(f"{error}: {result.stderr.strip()[:300]}")
return result.stdout.strip()
def _oid(value: str, field: str) -> str:
value = str(value).strip().lower()
if not _OID_RE.fullmatch(value):
raise GiteaProviderError(f"{field.upper()}_INVALID")
return value
def _mode(value) -> int:
if isinstance(value, int):
return value
text = str(value).strip()
if text.isdigit():
return int(text, 8)
raise GiteaProviderError("PROVIDER_MODE_INVALID")
def _safe_ref(value: str) -> Path:
raw = str(value).strip()
p = Path(raw)
if not raw or p.is_absolute() or any(part in ("", ".", "..") for part in p.parts):
raise GiteaProviderError("PROVIDER_RECEIPT_REF_INVALID")
return p
def _safe_remote_url(url: str) -> str:
url = str(url).strip()
if "://" in url:
parts = urlsplit(url)
host = parts.hostname or ""
port = f":{parts.port}" if parts.port else ""
return urlunsplit((parts.scheme, host + port, parts.path, "", ""))
if ":" in url and not url.startswith(("/", "./", "../")):
left, right = url.split(":", 1)
return f"{left.rsplit('@', 1)[-1]}:{right}"
return url
def repository_identity_from_remote(url: str) -> str:
safe = _safe_remote_url(url)
if "://" in safe:
path = urlsplit(safe).path
elif ":" in safe and not safe.startswith("/"):
path = safe.split(":", 1)[1]
else:
path = safe
parts = [x for x in path.strip("/").split("/") if x]
if parts and parts[0] == "git":
parts = parts[1:]
if len(parts) < 2:
raise GiteaProviderError("GITEA_REPOSITORY_IDENTITY_UNAVAILABLE")
owner, repo = parts[-2], parts[-1]
if repo.endswith(".git"):
repo = repo[:-4]
identity = f"{owner}/{repo}".lower()
if not _REPO_RE.fullmatch(identity):
raise GiteaProviderError("GITEA_REPOSITORY_IDENTITY_INVALID")
return identity
def load_gitea_policy(policy_path: str | Path) -> dict:
raw = json.loads(Path(policy_path).read_text(encoding="utf-8"))
defaults = dict(raw.get("defaults") or {})
providers = raw.get("providers")
if not isinstance(providers, dict) or not isinstance(providers.get("gitea"), dict):
raise GiteaProviderError("GITEA_PROVIDER_POLICY_MISSING")
cfg = dict(providers["gitea"])
return {
"provider_identity": str(cfg["provider_identity"]),
"store_id": str(cfg["store_id"]),
"store_root": Path(str(cfg["store_root"])),
"provider_operation": str(cfg["allowed_operations"][0]),
"expected_owner_uid": int(cfg.get("expected_owner_uid", defaults.get("expected_owner_uid", 0))),
"directory_mode": min(
_mode(x) for x in cfg.get(
"allowed_directory_modes",
defaults.get("allowed_directory_modes", ["0700", "0750"]),
)
),
"receipt_mode": min(
_mode(x) for x in cfg.get(
"allowed_receipt_modes",
defaults.get("allowed_receipt_modes", ["0600", "0640"]),
)
),
"authentication_kind": str(
cfg.get(
"authentication_kind",
defaults.get("authentication_kind", "root-owned-store-v1"),
)
),
}
def collect_gitea_transport_consistency(
*,
repo_root: str | Path,
source_oid: str,
tree_oid: str,
expected_repository: str = "aios/aios",
remote: str = "origin",
git_runner: GitRunner | None = None,
) -> dict:
repo_root = Path(repo_root).resolve()
if not repo_root.is_dir():
raise GiteaProviderError("GITEA_CHECKOUT_UNAVAILABLE")
source_oid = _oid(source_oid, "source_oid")
tree_oid = _oid(tree_oid, "tree_oid")
expected_repository = str(expected_repository).strip().lower()
if not _REPO_RE.fullmatch(expected_repository):
raise GiteaProviderError("GITEA_EXPECTED_REPOSITORY_INVALID")
if not re.fullmatch(r"[A-Za-z0-9._/-]+", remote) or remote.startswith(("/", "-")):
raise GiteaProviderError("GITEA_REMOTE_NAME_INVALID")
run = git_runner or _runner
if _git(run, repo_root, "rev-parse", "--is-inside-work-tree",
error="GITEA_LOCAL_REPOSITORY_REQUIRED").lower() != "true":
raise GiteaProviderError("GITEA_LOCAL_REPOSITORY_REQUIRED")
actual_source = _oid(
_git(run, repo_root, "rev-parse", "HEAD^{commit}",
error="GITEA_LOCAL_SOURCE_UNAVAILABLE"),
"actual_source_oid",
)
if actual_source != source_oid:
raise GiteaProviderError("GITEA_LOCAL_SOURCE_OID_MISMATCH")
actual_tree = _oid(
_git(run, repo_root, "rev-parse", f"{source_oid}^{{tree}}",
error="GITEA_LOCAL_TREE_UNAVAILABLE"),
"actual_tree_oid",
)
if actual_tree != tree_oid:
raise GiteaProviderError("GITEA_LOCAL_TREE_OID_MISMATCH")
branch = _git(
run, repo_root, "symbolic-ref", "--quiet", "HEAD",
error="GITEA_GOVERNED_BRANCH_REQUIRED",
)
if not _REF_RE.fullmatch(branch):
raise GiteaProviderError("GITEA_GOVERNED_BRANCH_INVALID")
remote_url = _git(
run, repo_root, "remote", "get-url", remote,
error="GITEA_REMOTE_UNAVAILABLE",
)
identity = repository_identity_from_remote(remote_url)
if identity != expected_repository:
raise GiteaProviderError("GITEA_REPOSITORY_IDENTITY_MISMATCH")
output = _git(
run, repo_root, "ls-remote", "--exit-code", remote, branch,
error="GITEA_REMOTE_REF_UNAVAILABLE",
)
rows = []
for line in output.splitlines():
parts = line.split()
if len(parts) >= 2 and parts[1] == branch:
rows.append((parts[0].lower(), parts[1]))
if len(rows) != 1:
raise GiteaProviderError("GITEA_REMOTE_REF_AMBIGUOUS")
remote_oid = _oid(rows[0][0], "remote_ref_oid")
if remote_oid != source_oid:
raise GiteaProviderError("GITEA_REMOTE_REF_OID_MISMATCH")
return {
"consistent": True,
"repository": identity,
"source_commit": source_oid,
"remote_ref_oid": remote_oid,
"tree_oid": tree_oid,
"remote_ref": branch,
"remote_url": _safe_remote_url(remote_url),
"transport": "git-ls-remote",
"api_verified": False,
}
def _ensure_root_directory(path: Path, mode: int, expected_uid: int) -> None:
if os.geteuid() != expected_uid:
raise GiteaProviderError("GITEA_PROVIDER_ROOT_REQUIRED")
path.mkdir(parents=True, exist_ok=True, mode=mode)
st = os.lstat(path)
if stat.S_ISLNK(st.st_mode) or not stat.S_ISDIR(st.st_mode):
raise GiteaProviderError("GITEA_PROVIDER_STORE_INVALID")
if st.st_uid != expected_uid:
raise GiteaProviderError("GITEA_PROVIDER_STORE_OWNER_INVALID")
if stat.S_IMODE(st.st_mode) & 0o022:
raise GiteaProviderError("GITEA_PROVIDER_STORE_MODE_INVALID")
def _write_receipt_exclusive(path: Path, payload: Mapping, mode: int) -> str:
data = canonical_json_bytes(payload)
if path.exists() or path.is_symlink():
raise GiteaProviderError("GITEA_PROVIDER_RECEIPT_EXISTS")
flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL
flags |= getattr(os, "O_NOFOLLOW", 0)
flags |= getattr(os, "O_BINARY", 0)
fd = os.open(path, flags, mode)
try:
view = memoryview(data)
while view:
written = os.write(fd, view)
if written <= 0:
raise GiteaProviderError("GITEA_PROVIDER_RECEIPT_WRITE_FAILED")
view = view[written:]
os.fsync(fd)
finally:
os.close(fd)
os.chmod(path, mode)
return hashlib.sha256(data).hexdigest()
def produce_gitea_consistency_receipt(
*,
repo_root: str | Path,
source_oid: str,
tree_oid: str,
policy_path: str | Path,
receipt_ref: str,
expected_repository: str = "aios/aios",
) -> tuple[str, dict]:
cfg = load_gitea_policy(policy_path)
if cfg["authentication_kind"] != "root-owned-store-v1":
raise GiteaProviderError("GITEA_PROVIDER_AUTH_KIND_UNSUPPORTED")
result = collect_gitea_transport_consistency(
repo_root=repo_root,
source_oid=source_oid,
tree_oid=tree_oid,
expected_repository=expected_repository,
)
now = datetime.now(timezone.utc).isoformat()
run_id = f"gitea-git-transport-{source_oid[:12]}-{datetime.now(timezone.utc).strftime('%Y%m%dT%H%M%SZ')}"
rel = _safe_ref(receipt_ref)
target = cfg["store_root"].joinpath(rel)
root = cfg["store_root"].resolve()
if not str(target.resolve(strict=False)).startswith(str(root) + os.sep):
raise GiteaProviderError("GITEA_PROVIDER_RECEIPT_REF_ESCAPE")
_ensure_root_directory(root, cfg["directory_mode"], cfg["expected_owner_uid"])
parent = root
for part in rel.parts[:-1]:
parent = parent / part
_ensure_root_directory(parent, cfg["directory_mode"], cfg["expected_owner_uid"])
payload = {
"schema": RECEIPT_SCHEMA,
"version": RECEIPT_VERSION,
"provider_identity": cfg["provider_identity"],
"provider_operation": cfg["provider_operation"],
"operation_run_id": run_id,
"result_reference": f"git:{result['remote_url']}#{result['remote_ref']}",
"content_sha256": canonical_sha256(result),
"source_oid": _oid(source_oid, "source_oid"),
"tree_oid": _oid(tree_oid, "tree_oid"),
"environment": "hw-isolated-linux-git-transport",
"issued_at": now,
"result": result,
"authentication": {
"kind": "root-owned-store-v1",
"store_id": cfg["store_id"],
},
}
sha = _write_receipt_exclusive(target, payload, cfg["receipt_mode"])
return sha, payload
@@ -0,0 +1,58 @@
from __future__ import annotations
from pathlib import Path
from typing import Any
from governed_provider_auth_adapter import (
ALL_RELEASE_EVIDENCE_KINDS,
authenticate_build_artifacts,
gate_w_authentication_status,
)
from trusted_provider_receipts import (
ProviderAuthenticationError,
TrustedProviderReceiptResolver,
)
def produce_authenticated_evidence_closure(
request: Any,
*,
provider_receipt_resolver: TrustedProviderReceiptResolver | None,
source_root: Path,
builder_policy: Path,
legacy_producer=None,
):
"""Production wrapper for the existing evidence producer.
This is the governed operation entrypoint. It refuses to call the legacy
closure assembler until *all* final-release provider receipts are resolved
from authenticated stores. For Gate-W, use `validate_gate_w_provider_auth`
instead; canary/rollback remain pending until Gate-R.
"""
if provider_receipt_resolver is None:
raise ProviderAuthenticationError("PROVIDER_AUTHENTICATION_REQUIRED")
if legacy_producer is None:
from governed_release_evidence_producer import produce_evidence_closure as legacy_producer
authenticated_request = authenticate_build_artifacts(
request,
resolver=provider_receipt_resolver,
required_kinds=ALL_RELEASE_EVIDENCE_KINDS,
)
return legacy_producer(
authenticated_request,
source_root=source_root,
builder_policy=builder_policy,
)
def validate_gate_w_provider_auth(
request: Any,
*,
provider_receipt_resolver: TrustedProviderReceiptResolver | None,
) -> dict[str, str]:
"""Gate-W authentication check without pretending Gate-R evidence exists."""
return gate_w_authentication_status(
request,
resolver=provider_receipt_resolver,
)
@@ -0,0 +1,323 @@
#!/usr/bin/env python3
"""Governed evidence runtime registration chain (V2R2C2 P0).
Establishes the REAL runtime chain for the governed release evidence producer:
Inventory -> Resolver -> operation -> Producer -> closure
The Producer alone (governed_release_evidence_producer) is not "runtime
registered" just because its OperationDefinition exists — registration only
holds when a runtime can actually discover the operation in an inventory,
resolve its ``handler_ref`` to the real callable, and invoke it to produce a
closure. That full path is what this module wires and what the
``PRODUCER_RUNTIME_REGISTERED_PASS`` verdict proves.
- Inventory : EvidenceOperationInventory — the set of governed evidence
operations the authority can execute (unions the producer
registry; extensible to future governed operations).
- Resolver : resolve_handler(handler_ref) maps the operation's declared
handler reference to the actual ``produce_evidence_closure``
callable. Resolution is exact and restrictive — it never
imports by arbitrary string eval.
- Producer : the resolved callable executes with validated, receipt-bound
build artifacts and materializes the evidence closure.
- closure : the produced source_root/<version>/ closure.
"""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any, Callable, Iterable, Mapping
from aios_baseline.operations import OperationDefinition, OperationRegistry
from governed_authenticated_evidence_operation import (
produce_authenticated_evidence_closure,
)
from governed_provider_auth_adapter import gate_w_authentication_status
from governed_release_evidence_producer import (
EvidenceProduceError,
GovernedBuildArtifacts,
ProduceResult,
TRUSTED_PROVIDER_EVIDENCE_TYPES,
AUTHENTICATED_EVIDENCE_HANDLER_REF,
produce_evidence_closure,
producer_operation_definition,
producer_registry,
)
from trusted_provider_receipts import (
ProviderAuthenticationError,
TrustedProviderPolicy,
TrustedProviderReceiptResolver,
)
# Exact, restrictive handler_ref -> callable map. This is the ONLY way
# handler_refs are resolved; no arbitrary import/eval.
_HANDLERS: Mapping[str, Callable[..., ProduceResult]] = {
"governed_release_evidence_producer.produce_evidence_closure": produce_evidence_closure,
AUTHENTICATED_EVIDENCE_HANDLER_REF: produce_authenticated_evidence_closure,
}
class EvidenceOperationInventory:
"""Inventory of governed evidence operations available to the authority."""
def __init__(self, *registries: OperationRegistry) -> None:
definitions: list[OperationDefinition] = []
seen: set[str] = set()
for reg in registries:
for definition in reg.definitions:
if definition.operation_id in seen:
continue
seen.add(definition.operation_id)
definitions.append(definition)
self._registry = OperationRegistry(definitions)
self._by_id = {d.operation_id: d for d in definitions}
@classmethod
def default(cls) -> "EvidenceOperationInventory":
"""The standard inventory: the governed release evidence producer."""
return cls(producer_registry())
def contains(self, operation_id: str) -> bool:
return operation_id in self._by_id
def operation_ids(self) -> tuple[str, ...]:
return tuple(self._by_id)
def get(self, operation_id: str) -> OperationDefinition | None:
return self._by_id.get(operation_id)
def resolve_handler(handler_ref: str) -> Callable[..., ProduceResult]:
"""Resolver: turn an OperationDefinition.handler_ref into the real callable.
Exact lookup in a fixed allowlist. Unknown or non-producer references raise.
"""
handler = _HANDLERS.get(handler_ref)
if handler is None:
raise EvidenceProduceError("HANDLER_REF_UNRESOLVED")
return handler
def _payload_to_artifacts(payload: Mapping[str, Any]) -> GovernedBuildArtifacts:
try:
return GovernedBuildArtifacts(
version=payload["version"],
candidate_id=payload["candidate_id"],
source_commit=payload["source_commit"],
tree_oid=payload["tree_oid"],
artifact=bytes.fromhex(payload["artifact_hex"]),
manifest=payload["manifest"],
package_closure=bytes.fromhex(payload["package_closure_hex"]),
policy=payload["policy"],
tests=payload["tests"],
canary=payload["canary"],
gitea=payload["gitea"],
rollback=payload["rollback"],
secret_scan=payload["secret_scan"],
deployment=payload["deployment"],
tag=payload["tag"],
finalization=payload["finalization"],
rollback_sha256=payload.get("rollback_sha256", ""),
issued_at=payload.get("issued_at", ""),
)
except (KeyError, TypeError, ValueError) as exc:
raise EvidenceProduceError("PAYLOAD_SCHEMA_INVALID") from exc
def invoke_governed_produce(
*,
builder_policy: Path,
source_root: Path,
payload: Mapping[str, Any],
expected_owner_uid: int | None = 0,
) -> dict[str, Any]:
"""Real runtime chain: Inventory -> Resolver -> operation -> Producer -> closure.
Returns a verdict dict. Only a genuine end-to-end invocation that reaches
evidence production reports ``status == "PRODUCER_RUNTIME_REGISTERED_PASS"``.
"""
inventory = EvidenceOperationInventory.default()
operation_id = "governed.release.evidence.produce"
if not inventory.contains(operation_id):
raise EvidenceProduceError("OPERATION_NOT_IN_INVENTORY")
definition = inventory.get(operation_id)
assert definition is not None # guaranteed by contains()
handler = resolve_handler(definition.handler_ref)
request = _payload_to_artifacts(payload)
result = handler(
builder_policy=builder_policy,
source_root=source_root,
request=request,
expected_owner_uid=expected_owner_uid,
)
# The resolved handler returned a real ProduceResult from a real closure;
# the registered operation genuinely produced evidence.
return {
"status": "PRODUCER_RUNTIME_REGISTERED_PASS",
"operation_id": operation_id,
"handler_ref": definition.handler_ref,
"handler_resolved": handler.__module__ + "." + handler.__name__,
"evidence_types_bound": list(TRUSTED_PROVIDER_EVIDENCE_TYPES),
"result": {
"produce_status": result.status,
"idempotent": result.idempotent,
"receipt_sha256": result.receipt_sha256,
"closure_dir": result.closure_dir,
"evidence_digests": dict(result.evidence_digests),
},
}
# ---------------------------------------------------------------------------
# V2R2C2 Round-3: Provider-Authenticated runtime chain.
#
# Inventory -> Resolver -> authenticated operation -> TrustedProviderReceiptResolver
# -> Producer assembler (second-layer bound)
#
# The Round-3 runtime builds TrustedProviderReceiptResolver from the governed
# evidence-provider policy (never from request-supplied store paths, provider
# identities, key ids, or HMAC material). Gate-W authenticates only the
# provider classes that can exist before staging (tests/Gitea/secret-scan);
# canary/rollback remain pending until Gate-R.
# ---------------------------------------------------------------------------
# Sequence used for a full final release closure (Gate-R).
ALL_RELEASE_EVIDENCE_KINDS = ("tests", "canary", "gitea", "rollback", "secret_scan")
# Provider classes that are real before staging (Gate-W).
GATE_W_EVIDENCE_KINDS = ("tests", "gitea", "secret_scan")
_DEFAULT_PROVIDER_POLICY = Path(__file__).resolve().parents[2] / "config" / "gitops" / "trusted-evidence-providers.json"
def _load_provider_policy(path: Path) -> Mapping[str, Any]:
try:
decoded = json.loads(Path(path).read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
raise EvidenceProduceError("PROVIDER_POLICY_UNAVAILABLE") from exc
if not isinstance(decoded, Mapping) or decoded.get("schema") != "aios.trusted-evidence-providers.v1":
raise EvidenceProduceError("PROVIDER_POLICY_INVALID")
return decoded
def _policy_from_config(name: str, raw: Mapping[str, Any], defaults: Mapping[str, Any]) -> TrustedProviderPolicy:
def _mode(value: object, field: str) -> int:
if isinstance(value, int):
return value
if isinstance(value, str) and value.isdigit():
return int(value, 8)
raise EvidenceProduceError("PROVIDER_POLICY_INVALID")
try:
store_root = Path(str(raw["store_root"])).expanduser()
allowed_ops = frozenset(str(op) for op in raw["allowed_operations"])
return TrustedProviderPolicy(
evidence_kind=name,
provider_identity=str(raw["provider_identity"]),
store_id=str(raw["store_id"]),
store_root=store_root,
allowed_operations=allowed_ops,
expected_owner_uid=_mode(raw.get("expected_owner_uid", defaults.get("expected_owner_uid", 0)), "expected_owner_uid"),
allowed_directory_modes=frozenset(
_mode(v, "allowed_directory_modes")
for v in raw.get("allowed_directory_modes", defaults.get("allowed_directory_modes", ["0700", "0750"]))
),
allowed_receipt_modes=frozenset(
_mode(v, "allowed_receipt_modes")
for v in raw.get("allowed_receipt_modes", defaults.get("allowed_receipt_modes", ["0600", "0640"]))
),
authentication_kind=str(raw.get("authentication_kind", defaults.get("authentication_kind", "root-owned-store-v1"))),
max_age_seconds=raw.get("max_age_seconds", defaults.get("max_age_seconds")),
)
except (KeyError, TypeError, ValueError) as exc:
raise EvidenceProduceError("PROVIDER_POLICY_INVALID") from exc
def build_provider_receipt_resolver(
provider_policy_path: Path = _DEFAULT_PROVIDER_POLICY,
*,
hmac_key_resolver: Callable[[str], bytes] | None = None,
now_fn: Callable[[], Any] | None = None,
) -> TrustedProviderReceiptResolver:
"""Build TrustedProviderReceiptResolver from the governed evidence-provider policy.
Store paths, provider identities, key ids and HMAC material come only from
the governed policy/secret resolver — never from a request payload.
"""
policy = _load_provider_policy(provider_policy_path)
defaults = dict(policy.get("defaults") or {})
providers = policy.get("providers")
if not isinstance(providers, Mapping) or not providers:
raise EvidenceProduceError("PROVIDER_POLICY_INVALID")
policies: dict[str, TrustedProviderPolicy] = {}
for name, raw in providers.items():
if not isinstance(raw, Mapping):
raise EvidenceProduceError("PROVIDER_POLICY_INVALID")
policies[name] = _policy_from_config(name, raw, defaults)
return TrustedProviderReceiptResolver(policies, hmac_key_resolver=hmac_key_resolver, now_fn=now_fn)
def _resolver_guard(resolver: TrustedProviderReceiptResolver | None) -> TrustedProviderReceiptResolver:
if resolver is None:
raise ProviderAuthenticationError("PROVIDER_AUTHENTICATION_REQUIRED")
return resolver
def validate_gate_w_provider_auth(
request: GovernedBuildArtifacts,
*,
provider_receipt_resolver: TrustedProviderReceiptResolver | None,
) -> dict[str, str]:
"""Gate-W: authenticate only the provider classes real before staging.
Returns PRODUCER_RUNTIME_REGISTRATION_PASS / PROVIDER_AUTH_ENFORCEMENT_PASS /
FINAL_RELEASE_CLOSURE_PENDING_GATE_R. Canary/rollback are intentionally not
required here — they remain pending until Gate-R.
"""
resolver = _resolver_guard(provider_receipt_resolver)
gate_w_authentication_status(request, resolver=resolver)
return {
"producer_runtime": "PRODUCER_RUNTIME_REGISTRATION_PASS",
"provider_auth": "PROVIDER_AUTH_ENFORCEMENT_PASS",
"final_release_closure": "FINAL_RELEASE_CLOSURE_PENDING_GATE_R",
}
def _legacy_closure_adapter(authenticated_request: GovernedBuildArtifacts, *, source_root: Path, builder_policy: Path) -> ProduceResult:
"""Invoke the legacy closure assembler on the resolver-authenticated request.
The classified evidence fields now carry ``provider_receipt`` provenance;
the producer's second-layer validator re-asserts content/OID binding on it.
"""
return produce_evidence_closure(
builder_policy=builder_policy,
source_root=source_root,
request=authenticated_request,
expected_owner_uid=0,
)
def produce_authenticated_evidence(
request: GovernedBuildArtifacts,
*,
provider_receipt_resolver: TrustedProviderReceiptResolver | None,
source_root: Path,
builder_policy: Path,
required_kinds: Iterable[str] = ALL_RELEASE_EVIDENCE_KINDS,
) -> ProduceResult:
"""Gate-R production path: authenticate all five final-release provider
receipts, then invoke the legacy assembler only after authentication."""
resolver = _resolver_guard(provider_receipt_resolver)
try:
return produce_authenticated_evidence_closure(
request,
provider_receipt_resolver=resolver,
source_root=source_root,
builder_policy=builder_policy,
legacy_producer=_legacy_closure_adapter,
)
except ProviderAuthenticationError as exc:
raise EvidenceProduceError(str(exc)) from exc
@@ -0,0 +1,87 @@
from __future__ import annotations
from dataclasses import is_dataclass, replace
from typing import Any, Iterable, Mapping
from trusted_provider_receipts import (
ProviderAuthenticationError,
ResolvedEvidence,
TrustedProviderReceiptResolver,
)
ALL_RELEASE_EVIDENCE_KINDS = ("tests", "canary", "gitea", "rollback", "secret_scan")
GATE_W_EVIDENCE_KINDS = ("tests", "gitea", "secret_scan")
def authenticate_evidence_request(
evidence_kind: str,
evidence_request: Mapping[str, Any],
*,
source_oid: str,
tree_oid: str,
resolver: TrustedProviderReceiptResolver | None,
) -> dict[str, Any]:
if resolver is None:
raise ProviderAuthenticationError("PROVIDER_AUTHENTICATION_REQUIRED")
resolved: ResolvedEvidence = resolver.resolve_evidence(
evidence_kind,
evidence_request,
source_oid=source_oid,
tree_oid=tree_oid,
)
return {
**dict(resolved.result),
"provider_receipt": dict(resolved.provider_receipt),
}
def authenticate_build_artifacts(
request: Any,
*,
resolver: TrustedProviderReceiptResolver | None,
required_kinds: Iterable[str] = ALL_RELEASE_EVIDENCE_KINDS,
) -> Any:
"""Authenticate receipt_ref-only evidence and return a replaced request.
The input is expected to be the existing GovernedBuildArtifacts dataclass.
The function deliberately relies only on field names already used by the
producer (`source_commit`, `tree_oid`, and the evidence kind fields), so it
can be integrated without changing the deployment/release-plan contract.
"""
if resolver is None:
raise ProviderAuthenticationError("PROVIDER_AUTHENTICATION_REQUIRED")
if not is_dataclass(request):
raise TypeError("request must be a dataclass instance")
source_oid = getattr(request, "source_commit")
tree_oid = getattr(request, "tree_oid")
replacements: dict[str, Any] = {}
for kind in tuple(required_kinds):
evidence_request = getattr(request, kind, None)
if evidence_request is None:
raise ProviderAuthenticationError(f"EVIDENCE_{kind.upper()}_REQUIRED")
replacements[kind] = authenticate_evidence_request(
kind,
evidence_request,
source_oid=source_oid,
tree_oid=tree_oid,
resolver=resolver,
)
return replace(request, **replacements)
def gate_w_authentication_status(
request: Any,
*,
resolver: TrustedProviderReceiptResolver | None,
) -> dict[str, str]:
"""Authenticate Gate-W evidence without pretending final closure exists."""
authenticate_build_artifacts(
request,
resolver=resolver,
required_kinds=GATE_W_EVIDENCE_KINDS,
)
return {
"producer_runtime": "PRODUCER_RUNTIME_REGISTRATION_PASS",
"provider_auth": "PROVIDER_AUTH_ENFORCEMENT_PASS",
"final_release_closure": "FINAL_RELEASE_CLOSURE_PENDING_GATE_R",
}
@@ -0,0 +1,634 @@
#!/usr/bin/env python3
"""Governed release evidence producer (V2R2C2 P0, operation governed.release.evidence.produce).
This module is the *producer* half of the governed evidence pipeline. It never
invents candidate/release evidence: every digest and pass/fail flag in the closure
is derived from real build artifacts and real verification executions supplied as
input. It assembles a self-consistent, root-owned evidence closure below
``source_root/<version>`` that ``TrustedBuilderEvidenceMaterializer`` (the projection
half) can re-validate and project byte-for-byte.
Contract (see docs/V2R2C2-P0-convergence-20260909.md section 1):
- operation_id : governed.release.evidence.produce
- input : governed build artifacts + deployment conventions (no hand-write)
- output : source_root/<version>/ full evidence_layout closure + receipt.json
- fail-closed : mismatched secret-scan digest, tampered artifact, non-passing
tests/canary/rollback/secret-scan, or inconsistent gitea all abort
without writing a closure.
- idempotent : an existing identical closure is accepted; a conflicting one fails.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import secrets
from dataclasses import dataclass, field
from pathlib import Path
import shutil
import stat
from typing import Any, Mapping
from aios_baseline.operations import OperationDefinition, OperationRegistry
from aios_trusted_candidate_builder import (
load_builder_policy,
TrustedCandidateBuilderConfig,
)
from release_evidence_materializer import EvidenceMaterializationError
from release_version import validate_release_version
PRODUCE_SCHEMA = "aios.governed-release-evidence-producer.v1"
EVIDENCE_RECEIPT_SCHEMA = "aios.trusted-candidate-builder.evidence.v1"
# Each governed evidence type (tests/canary/gitea/rollback/secret_scan) must be
# bound to a trusted-provider receipt. A plain caller-supplied {"passed": true}
# is never accepted on its own: the producer requires the receipt that proves a
# trusted provider actually ran the check and produced the referenced result.
TRUSTED_PROVIDER_RECEIPT_SCHEMA = "aios.trusted-provider.evidence.v1"
TRUSTED_PROVIDER_EVIDENCE_TYPES = ("tests", "canary", "gitea", "rollback", "secret_scan")
class EvidenceProduceError(RuntimeError):
"""Stable, non-sensitive producer failure."""
def _canonical_json(value: Mapping[str, Any]) -> bytes:
return json.dumps(
value,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=True,
).encode("utf-8")
def _file_sha256(path: Path) -> str:
digest = hashlib.sha256()
with Path(path).open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def _sha256_bytes(value: bytes) -> str:
return hashlib.sha256(value).hexdigest()
def _validate_oid(value: object, name: str) -> str:
normalized = str(value or "").strip().lower()
if (
len(normalized) != 40
or any(char not in "0123456789abcdef" for char in normalized)
):
raise EvidenceProduceError(f"{name.upper()}_INVALID")
return normalized
def _validate_identifier(value: object, name: str) -> str:
normalized = str(value or "").strip()
if (
not normalized
or len(normalized) > 255
or normalized[0] in ".-"
or any(
char not in "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789._:-"
for char in normalized
)
):
raise EvidenceProduceError(f"{name.upper()}_INVALID")
return normalized
def _validate_sha256(value: object, name: str, *, allow_empty: bool = False) -> str:
normalized = str(value or "").strip().lower()
if allow_empty and not normalized:
return ""
if (
len(normalized) != 64
or any(char not in "0123456789abcdef" for char in normalized)
):
raise EvidenceProduceError(f"{name.upper()}_INVALID")
return normalized
@dataclass(frozen=True)
class GovernedBuildArtifacts:
"""Real build artifacts + real verification results. Producer never invents these."""
version: str
candidate_id: str
source_commit: str
tree_oid: str
artifact: bytes
manifest: Mapping[str, Any]
package_closure: bytes
policy: Mapping[str, Any]
tests: Mapping[str, Any]
canary: Mapping[str, Any]
gitea: Mapping[str, Any]
rollback: Mapping[str, Any]
secret_scan: Mapping[str, Any]
# Deployment conventions (supplied by release planning, not evidence invention).
deployment: Mapping[str, Any]
tag: Mapping[str, Any]
finalization: Mapping[str, Any]
rollback_sha256: str = ""
issued_at: str = ""
@dataclass(frozen=True)
class ProduceResult:
status: str
idempotent: bool
receipt_sha256: str
closure_dir: str
evidence_digests: Mapping[str, str] = field(default_factory=dict)
def _require_trusted_provider_receipt(
name: str,
evidence: Mapping[str, Any],
source_commit: str,
tree_oid: str,
) -> None:
"""Require a trusted-provider binding for `evidence` (second-layer check).
A caller may never pass a bare ``{"passed": true}``: each governed evidence
type must be bound to a trusted provider. Two accepted forms, both
fail-closed on any missing/structurally invalid/digest-mismatched binding:
1. Round-2 native ``receipt`` (schema ``aios.trusted-provider.evidence.v1``)
— a caller-carried receipt.
2. Round-3 resolver-authenticated ``provider_receipt`` provenance (schema
``aios.trusted-evidence-provider-receipt.v2``) — produced by
``TrustedProviderReceiptResolver`` from an authenticated store. Its input
is the resolver-authenticated provenance, never a caller-supplied body.
This helper is a *second-layer* binding validator: for the Round-3 path the
authoritative proof is the resolver's authenticated provenance embedded as
``provider_receipt``, and this check only re-asserts content/OID binding on
top of it.
"""
receipt = evidence.get("receipt")
if isinstance(receipt, Mapping) and receipt:
body = {k: v for k, v in dict(evidence).items() if k != "receipt"}
expected = _sha256_bytes(_canonical_json(body))
if receipt.get("schema") != TRUSTED_PROVIDER_RECEIPT_SCHEMA:
raise EvidenceProduceError("EVIDENCE_RECEIPT_INVALID")
if receipt.get("content_sha256") != expected:
raise EvidenceProduceError("EVIDENCE_RECEIPT_CONTENT_MISMATCH")
for f in ("provider_identity", "operation_run_id", "result_reference",
"environment", "issued_at"):
if not str(receipt.get(f, "") or "").strip():
raise EvidenceProduceError("EVIDENCE_RECEIPT_INVALID")
if receipt.get("source_oid") not in (None, "", source_commit):
raise EvidenceProduceError("EVIDENCE_RECEIPT_SOURCE_OID_MISMATCH")
if receipt.get("tree_oid") not in (None, "", tree_oid):
raise EvidenceProduceError("EVIDENCE_RECEIPT_TREE_OID_MISMATCH")
return
provider_receipt = evidence.get("provider_receipt")
if isinstance(provider_receipt, Mapping) and provider_receipt:
if (
provider_receipt.get("schema")
!= "aios.trusted-evidence-provider-receipt.v2"
):
raise EvidenceProduceError("EVIDENCE_RECEIPT_INVALID")
for f in ("provider_identity", "provider_operation", "operation_run_id",
"result_reference", "content_sha256", "source_oid", "tree_oid",
"issued_at"):
if not str(provider_receipt.get(f, "") or "").strip():
raise EvidenceProduceError("EVIDENCE_RECEIPT_INVALID")
if provider_receipt.get("source_oid") != source_commit:
raise EvidenceProduceError("EVIDENCE_RECEIPT_SOURCE_OID_MISMATCH")
if provider_receipt.get("tree_oid") != tree_oid:
raise EvidenceProduceError("EVIDENCE_RECEIPT_TREE_OID_MISMATCH")
return
raise EvidenceProduceError("EVIDENCE_RECEIPT_MISSING")
def _verify_artifacts(
config: TrustedCandidateBuilderConfig,
request: GovernedBuildArtifacts,
) -> dict[str, str]:
"""Validate input consistency (fail-closed) and return the evidence digests."""
try:
version = validate_release_version(request.version)
except ValueError as exc:
raise EvidenceProduceError("VERSION_INVALID") from exc
if version != request.version:
raise EvidenceProduceError("VERSION_INVALID")
candidate_id = _validate_identifier(request.candidate_id, "candidate_id")
source_commit = _validate_oid(request.source_commit, "source_commit")
tree_oid = _validate_oid(request.tree_oid, "tree_oid")
if (
set(request.manifest) != {
"artifact_sha256", "package_closure_sha256",
}
):
raise EvidenceProduceError("MANIFEST_SCHEMA_INVALID")
closure_sha = _validate_sha256(
request.manifest.get("package_closure_sha256"), "package_closure_sha256"
)
artifact_sha = _sha256_bytes(request.artifact)
if request.manifest["artifact_sha256"] != artifact_sha:
raise EvidenceProduceError("MANIFEST_ARTIFACT_DIGEST_MISMATCH")
# manifest_sha256 is the digest of the manifest evidence file itself. It is
# derived (never invented) from the manifest content; the manifest does not
# embed its own digest to avoid a self-referential fixed point.
manifest_sha = _sha256_bytes(_canonical_json(dict(request.manifest)))
if _sha256_bytes(request.package_closure) != closure_sha:
raise EvidenceProduceError("PACKAGE_CLOSURE_DIGEST_MISMATCH")
# policy_sha256 is the digest of the policy evidence file content; derived
# here (never invented, never self-referential).
policy_sha = _sha256_bytes(_canonical_json(dict(request.policy)))
# Secret scan must cover the exact artifact bytes.
secret_scan_artifact = _validate_sha256(
request.secret_scan.get("artifact_sha256"), "secret_scan.artifact_sha256"
)
if secret_scan_artifact != artifact_sha:
raise EvidenceProduceError("SECRET_SCAN_ARTIFACT_DIGEST_MISMATCH")
# Every governed evidence type must be bound to a trusted-provider receipt.
# A bare caller-supplied {"passed": true} (no receipt) fails closed here;
# the producer never accepts an invented pass flag.
for etype in TRUSTED_PROVIDER_EVIDENCE_TYPES:
_require_trusted_provider_receipt(
etype, getattr(request, etype), source_commit, tree_oid
)
if request.secret_scan.get("passed") is not True:
raise EvidenceProduceError("SECRET_SCAN_NOT_PASSED")
if request.tests.get("passed") is not True:
raise EvidenceProduceError("TESTS_NOT_PASSED")
if request.canary.get("passed") is not True:
raise EvidenceProduceError("CANARY_NOT_PASSED")
if request.rollback.get("ready") is not True:
raise EvidenceProduceError("ROLLBACK_NOT_READY")
if (
request.gitea.get("consistent") is not True
or request.gitea.get("repository") != config.repository
or request.gitea.get("source_commit") != source_commit
or request.gitea.get("remote_ref_oid") != source_commit
or request.gitea.get("tree_oid") != tree_oid
):
raise EvidenceProduceError("GITEA_INCONSISTENT")
digests = {
"artifact": artifact_sha,
"manifest": manifest_sha,
"package_closure": closure_sha,
"policy": policy_sha,
"tests": _sha256_bytes(_canonical_json(dict(request.tests))),
"canary": _sha256_bytes(_canonical_json(dict(request.canary))),
"gitea": _sha256_bytes(_canonical_json(dict(request.gitea))),
"rollback": _sha256_bytes(_canonical_json(dict(request.rollback))),
"secret_scan": _sha256_bytes(_canonical_json(dict(request.secret_scan))),
}
return digests
def _build_receipt(
config: TrustedCandidateBuilderConfig,
request: GovernedBuildArtifacts,
) -> Mapping[str, Any]:
"""Assemble the self-consistent receipt from verified artifacts."""
from silent_release_controller import ReleaseCandidate, ReleaseState
artifact_sha = _sha256_bytes(request.artifact)
manifest_sha = _sha256_bytes(_canonical_json(dict(request.manifest)))
closure_sha = _validate_sha256(
request.manifest.get("package_closure_sha256"), "package_closure_sha256"
)
policy_sha = _sha256_bytes(_canonical_json(dict(request.policy)))
candidate_kwargs: dict[str, Any] = {}
issued_at = str(request.issued_at or "").strip()
if issued_at:
# The evidence was produced at the build execution instant; bind it so the
# same verified artifacts reproduce an identical, idempotent closure.
candidate_kwargs["created_at"] = issued_at
candidate_kwargs["updated_at"] = issued_at
candidate = ReleaseCandidate(
candidate_id=request.candidate_id,
source_commit=request.source_commit,
tree_oid=request.tree_oid,
artifact_sha256=artifact_sha,
manifest_sha256=manifest_sha,
tests=dict(request.tests),
canary=dict(request.canary),
rollback_ready=request.rollback.get("ready") is True,
git_consistency={
"consistent": True,
"repository": config.repository,
"source_commit": request.source_commit,
"tree_oid": request.tree_oid,
},
gitea_consistency=dict(request.gitea),
state=ReleaseState.READY_FOR_APPROVAL,
**candidate_kwargs,
)
deployment_payload = dict(request.deployment)
deployment_payload.update(
{
"service": request.deployment["service"],
"source_commit": request.source_commit,
"tree_oid": request.tree_oid,
"artifact_sha256": artifact_sha,
"manifest_sha256": manifest_sha,
"package_closure_sha256": closure_sha,
"policy_sha256": policy_sha,
"rollback_sha256": _validate_sha256(
request.rollback_sha256, "rollback_sha256"
)
or request.rollback.get("rollback_sha256", ""),
"repository_identity": config.repository,
"authoritative_git_dir": request.deployment["authoritative_git_dir"],
}
)
if not deployment_payload["rollback_sha256"]:
raise EvidenceProduceError("ROLLBACK_SHA256_REQUIRED")
from silent_release_hardening import (
DeploymentPlan,
FinalizationPlan,
ReleasePlan,
TagPlan,
TrustedCandidateBundle,
)
release_plan = ReleasePlan(
deployment=DeploymentPlan.from_dict(deployment_payload),
tag=TagPlan.from_dict(dict(request.tag)),
finalization=FinalizationPlan.from_dict(dict(request.finalization)),
policy_sha256=policy_sha,
)
checks = {name: True for name in config.required_checks}
TrustedCandidateBundle(candidate, release_plan, checks).validate()
return {
"schema": EVIDENCE_RECEIPT_SCHEMA,
"version": request.version,
"candidate": candidate.to_dict(),
"release_plan": release_plan.to_dict(),
}
def _safe_target(source_root: Path, version: str) -> Path:
relative = Path(version)
if relative.is_absolute() or any(
part in {"", ".", ".."} for part in relative.parts
):
raise EvidenceProduceError("CLOSURE_PATH_INVALID")
candidate = Path(source_root) / version
try:
metadata = Path(source_root).stat(follow_symlinks=False)
except OSError as exc:
raise EvidenceProduceError("SOURCE_ROOT_UNAVAILABLE") from exc
if not stat.S_ISDIR(metadata.st_mode) or Path(source_root).is_symlink():
raise EvidenceProduceError("SOURCE_ROOT_INVALID")
return candidate
def _write_durable_json(path: Path, payload: Mapping[str, Any], mode: int = 0o600) -> str:
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
# The deterministic collector compares file digests against the canonical
# payload hash (no trailing newline); write the exact canonical bytes so the
# producer's closure re-validates under TrustedBuilderEvidenceMaterializer.
raw = _canonical_json(payload)
descriptor = os.open(
path,
os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0)
| getattr(os, "O_BINARY", 0),
mode,
)
try:
os.write(descriptor, raw)
os.fsync(descriptor)
finally:
os.close(descriptor)
os.chmod(path, mode)
return hashlib.sha256(raw).hexdigest()
def _write_durable_bytes(path: Path, value: bytes, mode: int = 0o600) -> str:
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
descriptor = os.open(
path,
os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0)
| getattr(os, "O_BINARY", 0),
mode,
)
try:
os.write(descriptor, value)
os.fsync(descriptor)
finally:
os.close(descriptor)
os.chmod(path, mode)
return hashlib.sha256(value).hexdigest()
def produce_evidence_closure(
*,
builder_policy: Path,
source_root: Path,
request: GovernedBuildArtifacts,
expected_owner_uid: int | None = 0,
) -> ProduceResult:
"""Produce a self-consistent, root-owned evidence closure, fail-closed and idempotent."""
try:
config = TrustedCandidateBuilderConfig.from_policy(
load_builder_policy(builder_policy)
)
except Exception as exc:
raise EvidenceProduceError(str(exc)) from exc
try:
version = validate_release_version(request.version)
except ValueError as exc:
raise EvidenceProduceError("VERSION_INVALID") from exc
digests = _verify_artifacts(config, request)
receipt = _build_receipt(config, request)
# Resolve layout paths from policy (aligned with evidence_layout; never invented).
target = _safe_target(source_root, version)
layout_paths = {
name: target / Path(template.format(version=version)).relative_to(version)
for name, template in config.evidence_layout.items()
}
if target.exists() or target.is_symlink():
# Idempotency: an existing identical closure (same receipt digest + same
# artifact bytes) is accepted; anything else is a conflict.
existing_receipt = target / "receipt.json"
if not existing_receipt.is_file():
raise EvidenceProduceError("CLOSURE_CONFLICT")
try:
existing = json.loads(existing_receipt.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
raise EvidenceProduceError("CLOSURE_CONFLICT") from exc
if _canonical_json(dict(existing)) != _canonical_json(dict(receipt)):
raise EvidenceProduceError("CLOSURE_CONFLICT")
for name, path in layout_paths.items():
if not (path.is_file() and _file_sha256(path) == digests[name]):
raise EvidenceProduceError("CLOSURE_CONFLICT")
return ProduceResult(
status="EVIDENCE_PRODUCED",
idempotent=True,
receipt_sha256=_sha256_bytes(_canonical_json(dict(receipt))),
closure_dir=str(target),
evidence_digests=dict(digests),
)
temporary = source_root / (
f".{version}.{os.getpid()}.{secrets.token_hex(8)}.prod.tmp"
)
if temporary.exists() or temporary.is_symlink():
raise EvidenceProduceError("PRODUCE_TEMP_CONFLICT")
temporary.mkdir(mode=0o700)
# The closure directory mirrors source_root/<version>/; every written file uses
# the policy-declared basename so the layout is aligned with evidence_layout.
try:
_write_durable_json(temporary / "receipt.json", receipt)
_write_durable_bytes(temporary / "artifact.tar.gz", request.artifact)
_write_durable_json(temporary / "manifest.json", request.manifest)
_write_durable_bytes(temporary / "checksums.sha256", request.package_closure)
_write_durable_json(temporary / "trusted-release-policy.json", request.policy)
_write_durable_json(temporary / "tests.json", request.tests)
_write_durable_json(temporary / "canary.json", request.canary)
_write_durable_json(temporary / "gitea.json", request.gitea)
_write_durable_json(temporary / "rollback.json", request.rollback)
_write_durable_json(temporary / "secret-scan.json", request.secret_scan)
os.replace(temporary, target)
except Exception:
shutil.rmtree(temporary, ignore_errors=True)
raise
return ProduceResult(
status="EVIDENCE_PRODUCED",
idempotent=False,
receipt_sha256=_sha256_bytes(_canonical_json(dict(receipt))),
closure_dir=str(target),
evidence_digests=dict(digests),
)
def producer_operation_definition() -> OperationDefinition:
"""Register the governed.release.evidence.produce operation (lookup-only registry).
This is the *legacy closure assembler* entrypoint. V2R2C2 Round-3 keeps it
registered for backward compatibility with the existing runtime chain; the
governed production path now resolves
``governed_authenticated_evidence_operation.produce_authenticated_evidence_closure``
(see ``authenticated_producer_operation_definition``).
"""
return OperationDefinition.from_dict(
{
"operation_id": "governed.release.evidence.produce",
"version": 1,
"effect_class": "RECONCILABLE",
"handler_ref": "governed_release_evidence_producer.produce_evidence_closure",
"input_schema": PRODUCE_SCHEMA,
"required_permissions": ["governed.release.evidence.produce"],
"timeout_seconds": 60.0,
"readback": {"mode": "STATE", "required": True},
"retry": {"mode": "NEVER"},
"reconcile": {"mode": "EXACT_READBACK"},
"rollback": {"mode": "COMPENSATE"},
"owner_boundary": {"mode": "WITHIN_GRANT"},
}
)
AUTHENTICATED_EVIDENCE_HANDLER_REF = (
"governed_authenticated_evidence_operation.produce_authenticated_evidence_closure"
)
def authenticated_producer_operation_definition() -> OperationDefinition:
"""Register the governed, provider-authenticated release evidence operation.
V2R2C2 Round-3 production entrypoint. Its ``handler_ref`` points at the
authenticated wrapper, which refuses to invoke the legacy closure assembler
until every final-release provider receipt has been resolved from an
authenticated store (Gate-W only authenticates tests/Gitea/secret-scan;
canary/rollback require Gate-R). See Round-3 Provider Authentication.
"""
return OperationDefinition.from_dict(
{
"operation_id": "governed.authenticated.release.evidence.produce",
"version": 1,
"effect_class": "RECONCILABLE",
"handler_ref": AUTHENTICATED_EVIDENCE_HANDLER_REF,
"input_schema": PRODUCE_SCHEMA,
"required_permissions": ["governed.release.evidence.produce"],
"timeout_seconds": 60.0,
"readback": {"mode": "STATE", "required": True},
"retry": {"mode": "NEVER"},
"reconcile": {"mode": "EXACT_READBACK"},
"rollback": {"mode": "COMPENSATE"},
"owner_boundary": {"mode": "WITHIN_GRANT"},
}
)
def producer_registry() -> OperationRegistry:
"""Union of the legacy assembler and the authenticated production operation."""
return OperationRegistry(
[producer_operation_definition(), authenticated_producer_operation_definition()]
)
def _main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("operation", choices=("produce",))
parser.add_argument("--version", required=True)
parser.add_argument("--builder-policy", required=True, type=Path)
parser.add_argument("--source-root", required=True, type=Path)
parser.add_argument("--artifacts-json", required=True, type=Path)
arguments = parser.parse_args()
try:
payload = json.loads(arguments.artifacts_json.read_text(encoding="utf-8"))
request = GovernedBuildArtifacts(
version=arguments.version,
candidate_id=payload["candidate_id"],
source_commit=payload["source_commit"],
tree_oid=payload["tree_oid"],
artifact=bytes.fromhex(payload["artifact_hex"]),
manifest=payload["manifest"],
package_closure=bytes.fromhex(payload["package_closure_hex"]),
policy=payload["policy"],
tests=payload["tests"],
canary=payload["canary"],
gitea=payload["gitea"],
rollback=payload["rollback"],
secret_scan=payload["secret_scan"],
deployment=payload["deployment"],
tag=payload["tag"],
finalization=payload["finalization"],
rollback_sha256=payload.get("rollback_sha256", ""),
)
result = produce_evidence_closure(
builder_policy=arguments.builder_policy,
source_root=arguments.source_root,
request=request,
)
except (EvidenceProduceError, KeyError, ValueError, OSError, json.JSONDecodeError) as exc:
raise SystemExit(str(exc)) from exc
from dataclasses import asdict
print(json.dumps(asdict(result), sort_keys=True, separators=(",", ":")))
if __name__ == "__main__":
_main()
@@ -25,8 +25,17 @@ class ReconciliationController:
def reconcile(self, actual: Mapping[str, Any]) -> ReconciliationReport:
drift: list[str] = []
for left, right, label in self.CHECKS:
if actual.get(left) != actual.get(right):
left_value = actual.get(left)
right_value = actual.get(right)
expected_type = bool if left == "declared_production_state" else str
complete = (
left in actual
and right in actual
and isinstance(left_value, expected_type)
and isinstance(right_value, expected_type)
and (expected_type is bool or bool(left_value.strip()) and bool(right_value.strip()))
)
if not complete or left_value != right_value:
drift.append(label)
actions = () if not drift else ("open_reconciliation_task", "open_reconciliation_pr", "attach_evidence", "notify_owner")
return ReconciliationReport(not drift, tuple(drift), actions)
@@ -0,0 +1,359 @@
#!/usr/bin/env python3
"""Governed, append-only reopen of one expired Owner approval cycle."""
from __future__ import annotations
import hashlib
import json
import os
import re
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Mapping
REQUEST_SCHEMA = "aios.owner-approval-reopen-request.v1"
EVENT_NAME = "owner_approval_reopen_authorized"
_IDEMPOTENCY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{7,127}$")
_HEX40 = re.compile(r"^[0-9a-f]{40}$")
_HEX64 = re.compile(r"^[0-9a-f]{64}$")
class ApprovalReopenError(RuntimeError):
def __init__(self, code: str, detail: str | None = None):
self.code = code
self.detail = detail
super().__init__(code if not detail else f"{code}:{detail}")
@dataclass(frozen=True)
class ReopenRequest:
candidate_id: str
expected_approval_fingerprint: str
expected_source_commit: str
expected_tree_oid: str
expected_artifact_sha256: str
expected_manifest_sha256: str
expected_release_plan_sha256: str
expected_policy_sha256: str
reason: str
idempotency_key: str
@classmethod
def from_mapping(cls, raw: Mapping[str, Any]) -> "ReopenRequest":
if raw.get("schema") != REQUEST_SCHEMA:
raise ApprovalReopenError("APPROVAL_REOPEN_SCHEMA_INVALID")
allowed = {
"schema", "candidate_id", "expected_approval_fingerprint",
"expected_source_commit", "expected_tree_oid",
"expected_artifact_sha256", "expected_manifest_sha256",
"expected_release_plan_sha256", "expected_policy_sha256",
"reason", "idempotency_key",
}
if set(raw) != allowed:
raise ApprovalReopenError("APPROVAL_REOPEN_REQUEST_FIELDS_INVALID")
req = cls(
candidate_id=str(raw["candidate_id"]),
expected_approval_fingerprint=str(raw["expected_approval_fingerprint"]),
expected_source_commit=str(raw["expected_source_commit"]),
expected_tree_oid=str(raw["expected_tree_oid"]),
expected_artifact_sha256=str(raw["expected_artifact_sha256"]),
expected_manifest_sha256=str(raw["expected_manifest_sha256"]),
expected_release_plan_sha256=str(raw["expected_release_plan_sha256"]),
expected_policy_sha256=str(raw["expected_policy_sha256"]),
reason=str(raw["reason"]),
idempotency_key=str(raw["idempotency_key"]),
)
if not req.candidate_id or "/" in req.candidate_id or ".." in req.candidate_id:
raise ApprovalReopenError("APPROVAL_REOPEN_CANDIDATE_ID_INVALID")
if not _HEX64.fullmatch(req.expected_approval_fingerprint):
raise ApprovalReopenError("APPROVAL_REOPEN_FINGERPRINT_INVALID")
if not _HEX40.fullmatch(req.expected_source_commit):
raise ApprovalReopenError("APPROVAL_REOPEN_SOURCE_INVALID")
if not _HEX40.fullmatch(req.expected_tree_oid):
raise ApprovalReopenError("APPROVAL_REOPEN_TREE_INVALID")
for value, code in (
(req.expected_artifact_sha256, "APPROVAL_REOPEN_ARTIFACT_INVALID"),
(req.expected_manifest_sha256, "APPROVAL_REOPEN_MANIFEST_INVALID"),
(req.expected_release_plan_sha256, "APPROVAL_REOPEN_RELEASE_PLAN_INVALID"),
(req.expected_policy_sha256, "APPROVAL_REOPEN_POLICY_INVALID"),
):
if not _HEX64.fullmatch(value):
raise ApprovalReopenError(code)
if not (1 <= len(req.reason.strip()) <= 512):
raise ApprovalReopenError("APPROVAL_REOPEN_REASON_INVALID")
if not _IDEMPOTENCY_RE.fullmatch(req.idempotency_key):
raise ApprovalReopenError("APPROVAL_REOPEN_IDEMPOTENCY_KEY_INVALID")
return req
def _canon(value: Any) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=True).encode()
def _sha(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def _time(value: Any, code: str) -> datetime:
if not isinstance(value, str) or not value:
raise ApprovalReopenError(code)
try:
dt = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError as exc:
raise ApprovalReopenError(code) from exc
if dt.tzinfo is None:
raise ApprovalReopenError(code)
return dt.astimezone(timezone.utc)
def _secure_json(path: Path) -> dict[str, Any]:
if path.is_symlink() or not path.is_file():
raise ApprovalReopenError("APPROVAL_REOPEN_STATE_FILE_INVALID")
st = path.stat()
if st.st_mode & 0o022:
raise ApprovalReopenError("APPROVAL_REOPEN_STATE_FILE_WRITABLE")
if st.st_uid not in {0, os.geteuid()}:
raise ApprovalReopenError("APPROVAL_REOPEN_STATE_FILE_OWNER_INVALID")
try:
obj = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
raise ApprovalReopenError("APPROVAL_REOPEN_STATE_FILE_INVALID") from exc
if not isinstance(obj, dict):
raise ApprovalReopenError("APPROVAL_REOPEN_STATE_FILE_INVALID")
return obj
def _fsync_dir(path: Path) -> None:
fd = os.open(path, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
try:
os.fsync(fd)
finally:
os.close(fd)
def _paths(root: Path, cid: str) -> dict[str, Path]:
return {
"candidate": root / "candidates" / f"{cid}.json",
"saga": root / "sagas" / f"{cid}.json",
"plan": root / "release-plans" / f"{cid}.json",
"acceptance": root / "acceptance" / f"{cid}.json",
"approval": root / "approvals" / f"{cid}.json",
}
def _history(root: Path, cid: str, fp: str) -> Path:
return root / "approval-history" / cid / f"{fp}.json"
def _archive(active: Path, history: Path) -> None:
history.parent.mkdir(parents=True, exist_ok=True, mode=0o750)
try:
os.chmod(history.parent, 0o750)
except PermissionError:
pass
original = active.read_bytes()
if history.exists():
if history.is_symlink() or not history.is_file() or history.read_bytes() != original:
raise ApprovalReopenError("APPROVAL_REOPEN_HISTORY_COLLISION")
return
try:
os.link(active, history)
except OSError:
fd = os.open(history, os.O_WRONLY | os.O_CREAT | os.O_EXCL,
active.stat().st_mode & 0o777)
try:
view = memoryview(original)
while view:
n = os.write(fd, view)
view = view[n:]
os.fsync(fd)
finally:
os.close(fd)
if history.read_bytes() != original:
raise ApprovalReopenError("APPROVAL_REOPEN_HISTORY_COPY_FAILED")
_fsync_dir(history.parent)
def _details(event: Mapping[str, Any]) -> Mapping[str, Any]:
val = event.get("details")
return val if isinstance(val, Mapping) else {}
def _cycle_event(saga: Mapping[str, Any], cycle_id: str) -> Mapping[str, Any] | None:
for event in saga.get("events") or []:
if isinstance(event, Mapping) and event.get("event") == EVENT_NAME \
and _details(event).get("approval_cycle_id") == cycle_id:
return event
return None
def _state(name: str) -> Any:
from silent_release_hardening import SagaState # lazy: avoids import cycle
return getattr(SagaState, name)
def _cycle_id(req: ReopenRequest) -> str:
return "approval-reopen-" + _sha(_canon({
"schema": REQUEST_SCHEMA,
"candidate_id": req.candidate_id,
"old_fingerprint": req.expected_approval_fingerprint,
"source_commit": req.expected_source_commit,
"tree_oid": req.expected_tree_oid,
"artifact_sha256": req.expected_artifact_sha256,
"manifest_sha256": req.expected_manifest_sha256,
"release_plan_sha256": req.expected_release_plan_sha256,
"policy_sha256": req.expected_policy_sha256,
"idempotency_key": req.idempotency_key,
}))[:24]
def _validate(req: ReopenRequest, candidate: Mapping[str, Any],
saga: Mapping[str, Any], plan: Mapping[str, Any],
acceptance: Mapping[str, Any], approval: Mapping[str, Any],
now: datetime) -> None:
if saga.get("journal_integrity") is False:
raise ApprovalReopenError("APPROVAL_REOPEN_JOURNAL_INTEGRITY_INVALID")
if saga.get("state") != "APPROVED":
raise ApprovalReopenError("APPROVAL_REOPEN_SAGA_STATE_INVALID")
if candidate.get("candidate_id") != req.candidate_id:
raise ApprovalReopenError("APPROVAL_REOPEN_CANDIDATE_MISMATCH")
if candidate.get("state") not in {"READY_FOR_APPROVAL", "APPROVED"}:
raise ApprovalReopenError("APPROVAL_REOPEN_CANDIDATE_STATE_INVALID")
if candidate.get("approval_state") not in {None, "PENDING"}:
raise ApprovalReopenError("APPROVAL_REOPEN_CANDIDATE_APPROVAL_STATE_INVALID")
# ReleaseCandidate v1 durable records do not persist side_effects_executed.
# Compatibility rule is intentionally fail-closed:
# - an absent key is accepted only together with the independent Saga
# invariant below that the accepted Owner-approval event is the final
# Saga event;
# - if a future durable schema persists the key, it must be exactly False;
# - explicit True/None/number/string values are rejected.
if "side_effects_executed" in candidate and candidate.get("side_effects_executed") is not False:
raise ApprovalReopenError("APPROVAL_REOPEN_SIDE_EFFECTS_ALREADY_EXECUTED")
checks = {
"source": candidate.get("source_commit") == req.expected_source_commit,
"tree": candidate.get("tree_oid") == req.expected_tree_oid,
"artifact": candidate.get("artifact_sha256") == req.expected_artifact_sha256,
"manifest": candidate.get("manifest_sha256") == req.expected_manifest_sha256,
"approval_fingerprint": approval.get("fingerprint") == req.expected_approval_fingerprint,
"approval_source": approval.get("source_commit") == req.expected_source_commit,
"approval_tree": approval.get("tree_oid") == req.expected_tree_oid,
"approval_artifact": approval.get("artifact_sha256") == req.expected_artifact_sha256,
"approval_manifest": approval.get("manifest_sha256") == req.expected_manifest_sha256,
"approval_release_plan": approval.get("release_plan_sha256") == req.expected_release_plan_sha256,
"approval_policy": approval.get("policy_sha256") == req.expected_policy_sha256,
"acceptance": acceptance.get("status") == "ACCEPTED",
"acceptance_release_plan": acceptance.get("release_plan_sha256") == req.expected_release_plan_sha256,
"plan_sha": (plan.get("sha256") or plan.get("release_plan_sha256")) == req.expected_release_plan_sha256,
"plan_policy": plan.get("policy_sha256") == req.expected_policy_sha256,
}
bad = sorted(k for k, ok in checks.items() if not ok)
if bad:
raise ApprovalReopenError("APPROVAL_REOPEN_BINDING_MISMATCH", ",".join(bad))
if now <= _time(approval.get("expires_at"), "APPROVAL_REOPEN_APPROVAL_EXPIRY_INVALID"):
raise ApprovalReopenError("APPROVAL_REOPEN_APPROVAL_NOT_EXPIRED")
events = [e for e in saga.get("events") or [] if isinstance(e, Mapping)]
accepted = None
for i, event in enumerate(events):
if event.get("event") == "signed_owner_approval_accepted" \
and _details(event).get("approval_fingerprint") == req.expected_approval_fingerprint:
accepted = i
if accepted is None:
raise ApprovalReopenError("APPROVAL_REOPEN_ACCEPTED_EVENT_MISSING")
if accepted != len(events) - 1:
raise ApprovalReopenError("APPROVAL_REOPEN_SAGA_ALREADY_ADVANCED")
def reopen_owner_approval(service: Any, raw: Mapping[str, Any], *,
state_root: Path | None = None,
now: datetime | None = None) -> dict[str, Any]:
req = ReopenRequest.from_mapping(raw)
raw_root = state_root or os.environ.get("AIOS_SILENT_RELEASE_STATE_ROOT", "")
if not str(raw_root):
raise ApprovalReopenError("APPROVAL_REOPEN_STATE_ROOT_MISSING")
root = Path(raw_root).expanduser().resolve()
paths = _paths(root, req.candidate_id)
cycle = _cycle_id(req)
history = _history(root, req.candidate_id, req.expected_approval_fingerprint)
saga = _secure_json(paths["saga"])
prior = _cycle_event(saga, cycle)
if prior is not None:
if saga.get("state") != "READY_FOR_APPROVAL":
raise ApprovalReopenError("APPROVAL_REOPEN_CYCLE_ALREADY_CONSUMED")
if not history.is_file():
raise ApprovalReopenError("APPROVAL_REOPEN_HISTORY_MISSING")
if paths["approval"].exists():
active = _secure_json(paths["approval"])
if active.get("fingerprint") != req.expected_approval_fingerprint:
raise ApprovalReopenError("APPROVAL_REOPEN_ACTIVE_APPROVAL_CHANGED")
paths["approval"].unlink()
_fsync_dir(paths["approval"].parent)
return {
"schema": "aios.owner-approval-reopen-result.v1",
"status": "IDEMPOTENT",
"candidate_id": req.candidate_id,
"approval_cycle_id": cycle,
"supersedes_fingerprint": req.expected_approval_fingerprint,
"saga_state": "READY_FOR_APPROVAL",
"history_ref": str(history.relative_to(root)),
}
candidate = _secure_json(paths["candidate"])
plan = _secure_json(paths["plan"])
acceptance = _secure_json(paths["acceptance"])
approval = _secure_json(paths["approval"])
_validate(req, candidate, saga, plan, acceptance, approval,
(now or datetime.now(timezone.utc)).astimezone(timezone.utc))
_archive(paths["approval"], history)
event_details = {
"schema": "aios.owner-approval-reopen-event.v1",
"approval_cycle_id": cycle,
"supersedes_approval_fingerprint": req.expected_approval_fingerprint,
"expired_at": approval.get("expires_at"),
"release_plan_sha256": req.expected_release_plan_sha256,
"source_commit": req.expected_source_commit,
"tree_oid": req.expected_tree_oid,
"artifact_sha256": req.expected_artifact_sha256,
"reason": req.reason.strip(),
"idempotency_digest": _sha(req.idempotency_key.encode()),
}
try:
service.sagas.compare_and_set(
req.candidate_id,
expected_revision=saga.get("revision"),
expected_states={_state("APPROVED")},
target=_state("READY_FOR_APPROVAL"),
event=EVENT_NAME,
details=event_details,
)
except Exception as exc:
raise ApprovalReopenError("APPROVAL_REOPEN_SAGA_CAS_FAILED") from exc
try:
active = _secure_json(paths["approval"])
if active.get("fingerprint") != req.expected_approval_fingerprint:
raise ApprovalReopenError("APPROVAL_REOPEN_ACTIVE_APPROVAL_CHANGED")
paths["approval"].unlink()
_fsync_dir(paths["approval"].parent)
except ApprovalReopenError:
raise
except OSError as exc:
raise ApprovalReopenError("APPROVAL_REOPEN_ACTIVE_APPROVAL_REMOVE_FAILED") from exc
return {
"schema": "aios.owner-approval-reopen-result.v1",
"status": "REOPENED",
"candidate_id": req.candidate_id,
"approval_cycle_id": cycle,
"supersedes_fingerprint": req.expected_approval_fingerprint,
"saga_state": "READY_FOR_APPROVAL",
"history_ref": str(history.relative_to(root)),
"release_plan_sha256": req.expected_release_plan_sha256,
"source_commit": req.expected_source_commit,
"tree_oid": req.expected_tree_oid,
}
@@ -0,0 +1,531 @@
#!/usr/bin/env python3
"""Governed, fail-closed projection of verified release evidence into Builder storage.
This module never invents candidate/release evidence. A separate governed release
producer must first write a complete, root-owned evidence closure below
``source_root/<version>``. The materializer re-validates that closure with the
same ``ServerOwnedEvidenceSource``/``DeterministicEvidenceCollector`` code used by
Trusted Candidate Builder and atomically projects the exact bytes into the
Builder-owned evidence root.
Readiness is deliberately orthogonal: an empty Builder evidence root does not make
Builder infrastructure globally unavailable, but every BuildTrustedCandidateV1
request still requires a valid materialized closure.
"""
from __future__ import annotations
import argparse
from dataclasses import replace
import hashlib
import json
import os
from pathlib import Path
import secrets
import shutil
import stat
from typing import Any, Callable, Mapping
import subprocess
from aios_trusted_candidate_builder import (
BuilderError,
DeterministicEvidenceCollector,
ServerOwnedEvidenceSource,
TrustedCandidateBuilderConfig,
load_builder_policy,
)
from release_migration_plan import FileMigrationPlanResolver
from release_source_admission import (
ReleaseSourceAdmissionStore,
open_source_admission_store,
source_admission_policy_sha256,
)
from release_version import validate_release_version
MATERIALIZATION_SCHEMA = "aios.trusted-builder-evidence-materialization.v1"
SOURCE_ADMISSION_CREDENTIAL = "source_admission_hmac"
class EvidenceMaterializationError(RuntimeError):
"""Stable, non-sensitive materialization failure."""
def _canonical_json(value: Mapping[str, Any]) -> bytes:
return json.dumps(
value,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=True,
).encode("utf-8")
def _file_sha256(path: Path) -> str:
digest = hashlib.sha256()
with Path(path).open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def _validate_oid(value: object, name: str) -> str:
normalized = str(value or "").strip().lower()
if (
len(normalized) != 40
or any(char not in "0123456789abcdef" for char in normalized)
):
raise EvidenceMaterializationError(f"{name.upper()}_INVALID")
return normalized
def _validate_identifier(value: object, name: str) -> str:
normalized = str(value or "").strip()
if (
not normalized
or len(normalized) > 255
or normalized[0] in ".-"
or any(
char not in "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789._:-"
for char in normalized
)
):
raise EvidenceMaterializationError(f"{name.upper()}_INVALID")
return normalized
def _safe_directory(path: Path, *, expected_owner_uid: int | None) -> Path:
candidate = Path(path)
try:
metadata = candidate.stat(follow_symlinks=False)
resolved = candidate.resolve(strict=True)
except OSError as exc:
raise EvidenceMaterializationError("EVIDENCE_DIRECTORY_UNAVAILABLE") from exc
if candidate.is_symlink() or not stat.S_ISDIR(metadata.st_mode):
raise EvidenceMaterializationError("EVIDENCE_DIRECTORY_INVALID")
if os.name != "nt" and metadata.st_mode & 0o022:
raise EvidenceMaterializationError("EVIDENCE_DIRECTORY_PERMISSIONS_INVALID")
if (
os.name != "nt"
and expected_owner_uid is not None
and metadata.st_uid != expected_owner_uid
):
raise EvidenceMaterializationError("EVIDENCE_DIRECTORY_OWNER_INVALID")
return resolved
def _safe_regular(
root: Path,
relative: str,
*,
expected_owner_uid: int | None,
) -> Path:
root = _safe_directory(root, expected_owner_uid=expected_owner_uid)
relative_path = Path(relative)
if relative_path.is_absolute() or any(
part in {"", ".", ".."} for part in relative_path.parts
):
raise EvidenceMaterializationError("EVIDENCE_PATH_INVALID")
current = root
for part in relative_path.parts:
current = current / part
if current.is_symlink():
raise EvidenceMaterializationError("EVIDENCE_SYMLINK_DENIED")
try:
resolved = (root / relative_path).resolve(strict=True)
resolved.relative_to(root)
metadata = resolved.stat(follow_symlinks=False)
except (OSError, ValueError) as exc:
raise EvidenceMaterializationError("EVIDENCE_INPUT_UNAVAILABLE") from exc
if not stat.S_ISREG(metadata.st_mode):
raise EvidenceMaterializationError("EVIDENCE_SPECIAL_FILE_DENIED")
if os.name != "nt" and metadata.st_mode & 0o022:
raise EvidenceMaterializationError("EVIDENCE_INPUT_PERMISSIONS_INVALID")
if (
os.name != "nt"
and expected_owner_uid is not None
and metadata.st_uid != expected_owner_uid
):
raise EvidenceMaterializationError("EVIDENCE_INPUT_OWNER_INVALID")
return resolved
def _fsync_directory(path: Path) -> None:
if os.name == "nt":
return
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
try:
os.fsync(descriptor)
finally:
os.close(descriptor)
def _copy_exact(source: Path, target: Path, *, mode: int = 0o640) -> None:
target.parent.mkdir(parents=True, exist_ok=True, mode=0o750)
descriptor = os.open(
target,
os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0)
| getattr(os, "O_BINARY", 0),
mode,
)
try:
with source.open("rb") as src:
while True:
chunk = src.read(1024 * 1024)
if not chunk:
break
os.write(descriptor, chunk)
os.fsync(descriptor)
finally:
os.close(descriptor)
os.chmod(target, mode)
def _write_json_exclusive(path: Path, payload: Mapping[str, Any]) -> str:
raw = _canonical_json(payload) + b"\n"
descriptor = os.open(
path,
os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0)
| getattr(os, "O_BINARY", 0),
0o640,
)
try:
os.write(descriptor, raw)
os.fsync(descriptor)
finally:
os.close(descriptor)
os.chmod(path, 0o640)
return hashlib.sha256(raw).hexdigest()
class TrustedBuilderEvidenceMaterializer:
"""Verify a governed evidence closure and atomically project exact bytes."""
def __init__(
self,
*,
builder_policy: Path,
authority_git_dir: Path,
source_root: Path,
output_root: Path,
source_admission_root: Path | None = None,
source_admission_hmac: bytes | None = None,
migration_plan_root: Path | None = None,
migration_profile_root: Path | None = None,
expected_owner_uid: int | None = 0,
git_runner: Callable[..., Any] = subprocess.run,
source_admission_resolver: Callable[[str, str | None, str], Mapping[str, str]] | None = None,
migration_plan_resolver: Callable[[str, str, str], Any] | None = None,
) -> None:
self.builder_policy = Path(builder_policy)
self.authority_git_dir = Path(authority_git_dir)
self.source_root = Path(source_root)
self.output_root = Path(output_root)
self.expected_owner_uid = expected_owner_uid
self.git_runner = git_runner
try:
policy = load_builder_policy(self.builder_policy)
config = TrustedCandidateBuilderConfig.from_policy(policy)
except BuilderError as exc:
raise EvidenceMaterializationError(str(exc)) from exc
self.source_admission_store: ReleaseSourceAdmissionStore | None = None
if source_admission_resolver is not None:
config = replace(
config,
source_admission_resolver=source_admission_resolver,
)
elif source_admission_root is not None:
if not source_admission_hmac or len(source_admission_hmac) < 32:
raise EvidenceMaterializationError("SOURCE_ADMISSION_HMAC_UNAVAILABLE")
self.source_admission_store = open_source_admission_store(
Path(source_admission_root),
repository=config.repository,
policy=self.builder_policy,
hmac_key=source_admission_hmac,
expected_owner_uid=expected_owner_uid,
writable=False,
)
config = replace(
config,
source_admission_resolver=self.source_admission_store.resolve_recipe,
)
if migration_plan_resolver is not None:
config = replace(config, migration_plan_resolver=migration_plan_resolver)
elif migration_plan_root is not None or migration_profile_root is not None:
if migration_plan_root is None or migration_profile_root is None:
raise EvidenceMaterializationError(
"MIGRATION_PLAN_CONFIGURATION_INVALID"
)
resolver = FileMigrationPlanResolver(
Path(migration_plan_root),
profile_root=Path(migration_profile_root),
expected_owner_uid=expected_owner_uid,
)
config = replace(config, migration_plan_resolver=resolver.resolve)
self.config = config
@property
def policy_file_sha256(self) -> str:
return source_admission_policy_sha256(self.builder_policy)
def _collector_for(self, evidence_root: Path) -> DeterministicEvidenceCollector:
source = ServerOwnedEvidenceSource(
self.config,
self.authority_git_dir,
evidence_root,
expected_owner_uid=self.expected_owner_uid,
git_runner=self.git_runner,
)
return DeterministicEvidenceCollector(
self.config,
evidence_source=source,
)
def _relative_files(self, version: str) -> tuple[str, ...]:
relative = [f"{version}/receipt.json"]
relative.extend(
template.format(version=version)
for template in self.config.evidence_layout.values()
)
return tuple(dict.fromkeys(relative))
def _validate_existing(
self,
*,
version: str,
candidate_id: str,
source_commit: str,
expected_bundle: Mapping[str, Any],
) -> dict[str, Any]:
try:
observed = self._collector_for(self.output_root).collect(
version,
self.config.resolve_recipe(version, candidate_id, source_commit),
)
except (BuilderError, RuntimeError, ValueError) as exc:
raise EvidenceMaterializationError("EVIDENCE_DESTINATION_CONFLICT") from exc
if _canonical_json(dict(observed)) != _canonical_json(dict(expected_bundle)):
raise EvidenceMaterializationError("EVIDENCE_DESTINATION_CONFLICT")
record_path = self.output_root / version / "materialization.json"
if not record_path.is_file():
raise EvidenceMaterializationError("EVIDENCE_DESTINATION_CONFLICT")
try:
record = json.loads(record_path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
raise EvidenceMaterializationError("EVIDENCE_DESTINATION_CONFLICT") from exc
if not isinstance(record, Mapping):
raise EvidenceMaterializationError("EVIDENCE_DESTINATION_CONFLICT")
return dict(record)
def materialize(
self,
*,
version: str,
candidate_id: str,
source_commit: str,
) -> dict[str, Any]:
try:
version = validate_release_version(version)
except ValueError as exc:
raise EvidenceMaterializationError("VERSION_INVALID") from exc
candidate_id = _validate_identifier(candidate_id, "candidate_id")
source_commit = _validate_oid(source_commit, "source_commit")
_safe_directory(
self.source_root,
expected_owner_uid=self.expected_owner_uid,
)
_safe_directory(
self.output_root,
expected_owner_uid=self.expected_owner_uid,
)
try:
recipe = self.config.resolve_recipe(
version,
candidate_id,
source_commit,
)
bundle = self._collector_for(self.source_root).collect(
version,
recipe,
)
except BuilderError as exc:
raise EvidenceMaterializationError(str(exc)) from exc
except (RuntimeError, ValueError) as exc:
raise EvidenceMaterializationError("EVIDENCE_SOURCE_INVALID") from exc
admission_id = ""
if self.source_admission_store is not None:
try:
admission_id = str(
self.source_admission_store.lookup_source(source_commit)[
"admission_id"
]
)
except Exception as exc:
raise EvidenceMaterializationError(
"SOURCE_ADMISSION_INVALID"
) from exc
receipt_path = _safe_regular(
self.source_root,
f"{version}/receipt.json",
expected_owner_uid=self.expected_owner_uid,
)
receipt_sha256 = _file_sha256(receipt_path)
release_plan = dict(bundle["release_plan"])
migration = release_plan.get("migration")
migration_sha256 = ""
if isinstance(migration, Mapping):
migration_sha256 = str(migration.get("sha256") or "")
materialization_payload = {
"schema": MATERIALIZATION_SCHEMA,
"repository": self.config.repository,
"version": version,
"candidate_id": candidate_id,
"source_commit": source_commit,
"tree_oid": str(bundle["candidate"]["tree_oid"]),
"source_admission_id": admission_id,
"builder_policy_file_sha256": self.policy_file_sha256,
"receipt_sha256": receipt_sha256,
"migration_sha256": migration_sha256,
"evidence_digests": dict(bundle["evidence_digests"]),
}
materialization_id = hashlib.sha256(
_canonical_json(materialization_payload)
).hexdigest()
materialization_payload["materialization_id"] = (
"evidence-materialization-" + materialization_id[:24]
)
target = self.output_root / version
if target.exists() or target.is_symlink():
existing = self._validate_existing(
version=version,
candidate_id=candidate_id,
source_commit=source_commit,
expected_bundle=bundle,
)
if (
existing.get("materialization_id")
!= materialization_payload["materialization_id"]
):
raise EvidenceMaterializationError(
"EVIDENCE_DESTINATION_CONFLICT"
)
return {
**materialization_payload,
"status": "EVIDENCE_MATERIALIZED",
"idempotent": True,
}
temporary = self.output_root / (
f".{version}.{os.getpid()}.{secrets.token_hex(8)}.tmp"
)
if temporary.exists() or temporary.is_symlink():
raise EvidenceMaterializationError("EVIDENCE_TEMP_CONFLICT")
temporary.mkdir(mode=0o750)
try:
for relative in self._relative_files(version):
source = _safe_regular(
self.source_root,
relative,
expected_owner_uid=self.expected_owner_uid,
)
destination_relative = Path(relative).relative_to(version)
destination = temporary / destination_relative
_copy_exact(source, destination)
if _file_sha256(destination) != _file_sha256(source):
raise EvidenceMaterializationError(
"EVIDENCE_COPY_DIGEST_MISMATCH"
)
materialization_sha256 = _write_json_exclusive(
temporary / "materialization.json",
materialization_payload,
)
_fsync_directory(temporary)
os.replace(temporary, target)
_fsync_directory(self.output_root)
except Exception:
shutil.rmtree(temporary, ignore_errors=True)
raise
try:
observed = self._collector_for(self.output_root).collect(
version,
recipe,
)
except (BuilderError, RuntimeError, ValueError) as exc:
raise EvidenceMaterializationError(
"EVIDENCE_POST_WRITE_VERIFY_FAILED"
) from exc
if _canonical_json(dict(observed)) != _canonical_json(dict(bundle)):
raise EvidenceMaterializationError(
"EVIDENCE_POST_WRITE_VERIFY_FAILED"
)
return {
**materialization_payload,
"materialization_sha256": materialization_sha256,
"status": "EVIDENCE_MATERIALIZED",
"idempotent": False,
}
def _credential(name: str) -> bytes:
root = os.environ.get("CREDENTIALS_DIRECTORY", "").strip()
if not root:
raise EvidenceMaterializationError("CREDENTIALS_DIRECTORY_REQUIRED")
path = Path(root) / name
try:
value = path.read_bytes().strip()
except OSError as exc:
raise EvidenceMaterializationError(
f"{name.upper()}_UNAVAILABLE"
) from exc
if len(value) < 32:
raise EvidenceMaterializationError(f"{name.upper()}_INVALID")
return value
def _main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("operation", choices=("materialize",))
parser.add_argument("--version", required=True)
parser.add_argument("--candidate-id", required=True)
parser.add_argument("--source-commit", required=True)
parser.add_argument("--builder-policy", required=True, type=Path)
parser.add_argument("--authority-git-dir", required=True, type=Path)
parser.add_argument("--source-root", required=True, type=Path)
parser.add_argument("--output-root", required=True, type=Path)
parser.add_argument("--source-admission-root", required=True, type=Path)
parser.add_argument("--migration-plan-root", required=True, type=Path)
parser.add_argument("--migration-profile-root", required=True, type=Path)
arguments = parser.parse_args()
try:
result = TrustedBuilderEvidenceMaterializer(
builder_policy=arguments.builder_policy,
authority_git_dir=arguments.authority_git_dir,
source_root=arguments.source_root,
output_root=arguments.output_root,
source_admission_root=arguments.source_admission_root,
source_admission_hmac=_credential(SOURCE_ADMISSION_CREDENTIAL),
migration_plan_root=arguments.migration_plan_root,
migration_profile_root=arguments.migration_profile_root,
).materialize(
version=arguments.version,
candidate_id=arguments.candidate_id,
source_commit=arguments.source_commit,
)
except EvidenceMaterializationError as exc:
raise SystemExit(str(exc)) from exc
print(json.dumps(result, sort_keys=True, separators=(",", ":")))
if __name__ == "__main__":
_main()
@@ -0,0 +1,989 @@
#!/usr/bin/env python3
"""Fail-closed cross-code-line release migration planning.
The migration plan is data, not a shell script. It binds an admitted source to
the exact currently observed staging code lines, the compatible state contract,
and the rollback/verification contract. The plan can then be embedded in the
existing signed ReleasePlan so Owner approval covers migration semantics without
creating a second release authority.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import re
import sqlite3
import stat
import subprocess
import tempfile
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Any, Callable, Mapping, Sequence
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
IDENT_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.:@/-]{2,255}$")
PLAN_SCHEMA = "aios.release-migration-plan.v1"
REQUIREMENT_SCHEMA = "aios.release-migration-requirement.v1"
PROFILE_SCHEMA = "aios.release-migration-profile.v1"
ALLOWED_STATE_MODES = frozenset({"REUSE_IN_PLACE_WITH_SNAPSHOT", "PRESERVE_UNUSED"})
REQUIRED_PRECONDITIONS = (
"writer_disabled",
"task_quiescent",
"state_snapshot_verified",
"credentials_ready",
"rollback_ready",
)
REQUIRED_VERIFICATION_CONTRACTS = (
"runtime_identity_exact",
"task_principal_no_bearer_e2e",
"restart_recovery",
"credential_projection",
"release_consistency_no_unknown",
)
class MigrationPlanError(RuntimeError):
"""Migration evidence is missing, malformed, stale, or unsafe."""
class MigrationPlanRequired(MigrationPlanError):
"""A cross-code-line release cannot proceed without an exact migration plan."""
def _canonical_json(value: object) -> bytes:
return (json.dumps(value, sort_keys=True, separators=(",", ":")) + "\n").encode("utf-8")
def _sha(value: object, label: str) -> str:
text = str(value or "").lower()
if not SHA_RE.fullmatch(text):
raise MigrationPlanError(f"{label}_invalid")
return text
def _oid(value: object, label: str) -> str:
text = str(value or "").lower()
if not OID_RE.fullmatch(text):
raise MigrationPlanError(f"{label}_invalid")
return text
def _ident(value: object, label: str) -> str:
text = str(value or "")
if not IDENT_RE.fullmatch(text):
raise MigrationPlanError(f"{label}_invalid")
return text
def _secure_json(path: Path, *, expected_owner_uid: int | None = 0) -> dict[str, Any]:
try:
metadata = path.lstat()
except OSError as exc:
raise MigrationPlanError(f"secure_json_unavailable:{path}") from exc
if path.is_symlink() or not stat.S_ISREG(metadata.st_mode):
raise MigrationPlanError("secure_json_regular_file_required")
if expected_owner_uid is not None and os.name != "nt" and metadata.st_uid != expected_owner_uid:
raise MigrationPlanError("secure_json_owner_invalid")
if os.name != "nt" and metadata.st_mode & 0o022:
raise MigrationPlanError("secure_json_mode_invalid")
try:
value = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise MigrationPlanError("secure_json_invalid") from exc
if not isinstance(value, dict):
raise MigrationPlanError("secure_json_object_required")
return value
def _durable_json(path: Path, value: Mapping[str, Any], *, mode: int = 0o640) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
encoded = _canonical_json(dict(value))
if path.exists():
current = path.read_bytes()
if current != encoded:
raise MigrationPlanError("migration_record_conflict")
return
temporary = path.with_name(path.name + ".tmp")
descriptor = os.open(temporary, os.O_WRONLY | os.O_CREAT | os.O_EXCL, mode)
try:
with os.fdopen(descriptor, "wb") as handle:
handle.write(encoded)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary, path)
if os.name != "nt":
os.chmod(path, mode)
directory_fd = os.open(path.parent, os.O_RDONLY)
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
finally:
try:
temporary.unlink()
except FileNotFoundError:
pass
def _hash_files(root: Path, files: Sequence[str]) -> str:
payload: dict[str, str] = {}
resolved_root = root.resolve(strict=True)
for relative in files:
rel = Path(relative)
if rel.is_absolute() or ".." in rel.parts:
raise MigrationPlanError("interface_path_invalid")
path = (resolved_root / rel).resolve(strict=True)
try:
path.relative_to(resolved_root)
except ValueError as exc:
raise MigrationPlanError("interface_path_escape") from exc
if path.is_symlink() or not path.is_file():
raise MigrationPlanError("interface_file_invalid")
payload[relative] = hashlib.sha256(path.read_bytes()).hexdigest()
return hashlib.sha256(_canonical_json(payload)).hexdigest()
def _git_blob(git_dir: Path, source_oid: str, relative: str) -> bytes:
rel = Path(relative)
if rel.is_absolute() or ".." in rel.parts:
raise MigrationPlanError("target_interface_path_invalid")
completed = subprocess.run(
["git", f"--git-dir={git_dir}", "show", f"{source_oid}:{relative}"],
check=False,
capture_output=True,
timeout=30,
env={"PATH": os.environ.get("PATH", "")},
)
if completed.returncode != 0:
raise MigrationPlanError("target_interface_unavailable")
return bytes(completed.stdout)
def _hash_git_files(git_dir: Path, source_oid: str, files: Sequence[str]) -> str:
payload = {
relative: hashlib.sha256(_git_blob(git_dir, source_oid, relative)).hexdigest()
for relative in files
}
return hashlib.sha256(_canonical_json(payload)).hexdigest()
def _systemd_properties(service: str) -> dict[str, Any]:
_ident(service, "service")
completed = subprocess.run(
[
"systemctl",
"show",
service,
"--property=WorkingDirectory",
"--property=Environment",
"--property=FragmentPath",
"--property=ActiveState",
],
check=False,
capture_output=True,
text=True,
timeout=20,
env={"PATH": os.environ.get("PATH", "")},
)
if completed.returncode != 0:
raise MigrationPlanError("systemd_observation_unavailable")
result: dict[str, Any] = {}
for line in completed.stdout.splitlines():
if "=" in line:
key, value = line.split("=", 1)
result[key] = value
if result.get("ActiveState") != "active":
raise MigrationPlanError("component_not_active")
environment: dict[str, str] = {}
for token in str(result.get("Environment") or "").split():
if "=" in token:
key, value = token.split("=", 1)
environment[key] = value.strip('"')
result["environment"] = environment
return result
def _sqlite_schema(connection: sqlite3.Connection) -> dict[str, Any]:
tables = [
row[0]
for row in connection.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%' ORDER BY name"
)
]
result: dict[str, Any] = {}
for table in tables:
if not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", table):
raise MigrationPlanError("sqlite_table_name_invalid")
columns = []
for row in connection.execute(f"PRAGMA table_info({table})"):
columns.append(
{
"cid": int(row[0]),
"name": str(row[1]),
"type": str(row[2]),
"notnull": int(row[3]),
"default": row[4],
"pk": int(row[5]),
}
)
result[table] = {
"columns": columns,
"row_count": int(connection.execute(f"SELECT COUNT(*) FROM {table}").fetchone()[0]),
}
return result
def _schema_digest(schema: Mapping[str, Any]) -> str:
shape = {
table: {"columns": value["columns"]}
for table, value in sorted(schema.items())
}
return hashlib.sha256(_canonical_json(shape)).hexdigest()
def _assert_additive_sqlite(before: Mapping[str, Any], after: Mapping[str, Any]) -> None:
for table, old in before.items():
if table not in after:
raise MigrationPlanError("sqlite_table_removed")
old_columns = {column["name"]: column for column in old["columns"]}
new_columns = {column["name"]: column for column in after[table]["columns"]}
for name, column in old_columns.items():
if new_columns.get(name) != column:
raise MigrationPlanError("sqlite_column_changed")
for name, column in new_columns.items():
if name in old_columns:
continue
# A new NOT NULL column is backward compatible only with a default.
if column["notnull"] and column["default"] is None:
raise MigrationPlanError("sqlite_new_column_unsafe")
if int(after[table]["row_count"]) != int(old["row_count"]):
raise MigrationPlanError("sqlite_row_count_changed_during_probe")
@dataclass(frozen=True)
class RuntimeComponentBinding:
component_id: str
service: str
code_line_id: str
source_oid: str
artifact_sha256: str | None
interface_sha256: str
def __post_init__(self) -> None:
_ident(self.component_id, "component_id")
_ident(self.service, "service")
_ident(self.code_line_id, "code_line_id")
_oid(self.source_oid, "source_oid")
if self.artifact_sha256 is not None:
_sha(self.artifact_sha256, "artifact_sha256")
_sha(self.interface_sha256, "interface_sha256")
def to_dict(self) -> dict[str, Any]:
return asdict(self)
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "RuntimeComponentBinding":
return cls(**dict(value))
@dataclass(frozen=True)
class StateMigrationContract:
state_id: str
state_ref: str
mode: str
before_schema_sha256: str
after_schema_sha256: str
snapshot_name: str
snapshot_sha256: str
snapshot_required: bool
rollback_compatible: bool
def __post_init__(self) -> None:
_ident(self.state_id, "state_id")
_ident(self.state_ref, "state_ref")
if self.mode not in ALLOWED_STATE_MODES:
raise MigrationPlanError("state_mode_invalid")
_sha(self.before_schema_sha256, "before_schema_sha256")
_sha(self.after_schema_sha256, "after_schema_sha256")
_sha(self.snapshot_sha256, "snapshot_sha256")
snapshot = Path(self.snapshot_name)
if (
snapshot.is_absolute()
or ".." in snapshot.parts
or not snapshot.parts
or str(snapshot).startswith(".")
):
raise MigrationPlanError("snapshot_name_invalid")
if self.snapshot_required is not True or self.rollback_compatible is not True:
raise MigrationPlanError("state_recovery_contract_invalid")
def to_dict(self) -> dict[str, Any]:
return asdict(self)
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "StateMigrationContract":
return cls(**dict(value))
@dataclass(frozen=True)
class MigrationPlan:
migration_id: str
version: str
candidate_id: str
repository_identity: str
source_admission_id: str
source_commit: str
tree_oid: str
profile_id: str
profile_sha256: str
from_components: tuple[RuntimeComponentBinding, ...]
to_components: tuple[RuntimeComponentBinding, ...]
state_contracts: tuple[StateMigrationContract, ...]
activation_schema: str
activation_sha256: str
rollback_window_seconds: int
preconditions: tuple[str, ...] = REQUIRED_PRECONDITIONS
verification_contracts: tuple[str, ...] = REQUIRED_VERIFICATION_CONTRACTS
runtime_switch_capability: str = "runtime_switch"
rollback_contract: str = "restore_previous_runtime_and_state_snapshot.v1"
forward_recovery_contract: str = "forward_recovery_after_verified_rollback_block.v1"
writer_expected: bool = False
direct_db_allowed: bool = False
direct_runtime_overwrite_allowed: bool = False
def __post_init__(self) -> None:
_ident(self.migration_id, "migration_id")
_ident(self.version, "version")
_ident(self.candidate_id, "candidate_id")
if self.repository_identity != "aios/aios":
raise MigrationPlanError("repository_identity_invalid")
_ident(self.source_admission_id, "source_admission_id")
_oid(self.source_commit, "source_commit")
_oid(self.tree_oid, "tree_oid")
_ident(self.profile_id, "profile_id")
_sha(self.profile_sha256, "profile_sha256")
_ident(self.activation_schema, "activation_schema")
_sha(self.activation_sha256, "activation_sha256")
if not 60 <= int(self.rollback_window_seconds) <= 86400:
raise MigrationPlanError("rollback_window_invalid")
if tuple(self.preconditions) != REQUIRED_PRECONDITIONS:
raise MigrationPlanError("migration_preconditions_invalid")
if tuple(self.verification_contracts) != REQUIRED_VERIFICATION_CONTRACTS:
raise MigrationPlanError("migration_verification_contracts_invalid")
if self.runtime_switch_capability != "runtime_switch":
raise MigrationPlanError("migration_capability_invalid")
if self.writer_expected or self.direct_db_allowed or self.direct_runtime_overwrite_allowed:
raise MigrationPlanError("migration_unsafe_authority")
if not self.from_components or not self.to_components or not self.state_contracts:
raise MigrationPlanError("migration_contract_incomplete")
from_ids = {item.component_id for item in self.from_components}
to_ids = {item.component_id for item in self.to_components}
if from_ids != to_ids or len(from_ids) != len(self.from_components) or len(to_ids) != len(self.to_components):
raise MigrationPlanError("migration_component_binding_invalid")
if all(
next(item.code_line_id for item in self.from_components if item.component_id == target.component_id)
== target.code_line_id
for target in self.to_components
):
raise MigrationPlanError("migration_not_required")
@property
def canonical_payload(self) -> dict[str, Any]:
return {
"schema": PLAN_SCHEMA,
"migration_id": self.migration_id,
"version": self.version,
"candidate_id": self.candidate_id,
"repository_identity": self.repository_identity,
"source_admission_id": self.source_admission_id,
"source_commit": self.source_commit,
"tree_oid": self.tree_oid,
"profile_id": self.profile_id,
"profile_sha256": self.profile_sha256,
"from_components": [item.to_dict() for item in self.from_components],
"to_components": [item.to_dict() for item in self.to_components],
"state_contracts": [item.to_dict() for item in self.state_contracts],
"activation_schema": self.activation_schema,
"activation_sha256": self.activation_sha256,
"rollback_window_seconds": self.rollback_window_seconds,
"preconditions": list(self.preconditions),
"verification_contracts": list(self.verification_contracts),
"runtime_switch_capability": self.runtime_switch_capability,
"rollback_contract": self.rollback_contract,
"forward_recovery_contract": self.forward_recovery_contract,
"writer_expected": self.writer_expected,
"direct_db_allowed": self.direct_db_allowed,
"direct_runtime_overwrite_allowed": self.direct_runtime_overwrite_allowed,
}
@property
def sha256(self) -> str:
return hashlib.sha256(_canonical_json(self.canonical_payload)).hexdigest()
def to_dict(self) -> dict[str, Any]:
return {**self.canonical_payload, "sha256": self.sha256}
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "MigrationPlan":
raw = dict(value)
if raw.pop("schema", None) != PLAN_SCHEMA:
raise MigrationPlanError("migration_plan_schema_invalid")
claimed = raw.pop("sha256", None)
raw["from_components"] = tuple(RuntimeComponentBinding.from_dict(item) for item in raw["from_components"])
raw["to_components"] = tuple(RuntimeComponentBinding.from_dict(item) for item in raw["to_components"])
raw["state_contracts"] = tuple(StateMigrationContract.from_dict(item) for item in raw["state_contracts"])
raw["preconditions"] = tuple(raw.get("preconditions") or ())
raw["verification_contracts"] = tuple(raw.get("verification_contracts") or ())
plan = cls(**raw)
if claimed is not None and claimed != plan.sha256:
raise MigrationPlanError("migration_plan_sha256_mismatch")
return plan
def assert_release_binding(
self,
*,
version: str,
candidate_id: str,
source_commit: str,
tree_oid: str,
repository_identity: str,
) -> None:
expected = (
self.version,
self.candidate_id,
self.source_commit,
self.tree_oid,
self.repository_identity,
)
actual = (version, candidate_id, source_commit, tree_oid, repository_identity)
if actual != expected:
raise MigrationPlanError("migration_release_binding_mismatch")
@dataclass(frozen=True)
class MigrationRequirement:
version: str
candidate_id: str
source_commit: str
repository_identity: str
required: bool
reason_code: str
profile_id: str
profile_sha256: str
migration_plan_sha256: str | None
@property
def canonical_payload(self) -> dict[str, Any]:
return {
"schema": REQUIREMENT_SCHEMA,
**asdict(self),
}
@property
def sha256(self) -> str:
return hashlib.sha256(_canonical_json(self.canonical_payload)).hexdigest()
def to_dict(self) -> dict[str, Any]:
return {**self.canonical_payload, "sha256": self.sha256}
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "MigrationRequirement":
raw = dict(value)
if raw.pop("schema", None) != REQUIREMENT_SCHEMA:
raise MigrationPlanError("migration_requirement_schema_invalid")
claimed = raw.pop("sha256", None)
requirement = cls(**raw)
_ident(requirement.version, "version")
_ident(requirement.candidate_id, "candidate_id")
_oid(requirement.source_commit, "source_commit")
if requirement.repository_identity != "aios/aios":
raise MigrationPlanError("migration_requirement_repository_invalid")
_ident(requirement.reason_code, "reason_code")
_ident(requirement.profile_id, "profile_id")
_sha(requirement.profile_sha256, "profile_sha256")
if requirement.required:
_sha(requirement.migration_plan_sha256, "migration_plan_sha256")
elif requirement.migration_plan_sha256 is not None:
raise MigrationPlanError("unexpected_migration_plan")
if claimed is not None and claimed != requirement.sha256:
raise MigrationPlanError("migration_requirement_sha256_mismatch")
return requirement
class FileMigrationPlanResolver:
"""Read-only resolver used by the trusted builder.
A profile declares that a release version is cross-code-line. Once a
version is declared, absence of requirement/plan state is fail-closed.
"""
def __init__(
self,
root: Path,
*,
profile_root: Path,
expected_owner_uid: int | None = 0,
):
self.root = Path(root)
self.profile_root = Path(profile_root)
self.expected_owner_uid = expected_owner_uid
def _profile_for_version(self, version: str) -> dict[str, Any] | None:
matches = []
if not self.profile_root.is_dir():
return None
for path in sorted(self.profile_root.glob("*.json")):
profile = _secure_json(path, expected_owner_uid=self.expected_owner_uid)
if profile.get("schema") != PROFILE_SCHEMA:
raise MigrationPlanError("migration_profile_schema_invalid")
if profile.get("version") == version:
matches.append(profile)
if len(matches) > 1:
raise MigrationPlanError("migration_profile_ambiguous")
return matches[0] if matches else None
def resolve(self, version: str, candidate_id: str, source_commit: str) -> MigrationPlan | None:
profile = self._profile_for_version(version)
if profile is None:
return None
base = self.root / source_commit
requirement_path = base / "requirement.json"
plan_path = base / "plan.json"
if not requirement_path.is_file():
raise MigrationPlanRequired("MIGRATION_PLAN_REQUIRED")
requirement = MigrationRequirement.from_dict(
_secure_json(requirement_path, expected_owner_uid=self.expected_owner_uid)
)
expected = (version, candidate_id, source_commit, "aios/aios")
actual = (
requirement.version,
requirement.candidate_id,
requirement.source_commit,
requirement.repository_identity,
)
if actual != expected:
raise MigrationPlanError("migration_requirement_binding_mismatch")
if requirement.profile_id != profile.get("profile_id"):
raise MigrationPlanError("migration_profile_binding_mismatch")
profile_sha = hashlib.sha256(_canonical_json(profile)).hexdigest()
if requirement.profile_sha256 != profile_sha:
raise MigrationPlanError("migration_profile_digest_mismatch")
if not requirement.required:
return None
if not plan_path.is_file():
raise MigrationPlanRequired("MIGRATION_PLAN_REQUIRED")
plan = MigrationPlan.from_dict(
_secure_json(plan_path, expected_owner_uid=self.expected_owner_uid)
)
plan.assert_release_binding(
version=version,
candidate_id=candidate_id,
source_commit=source_commit,
tree_oid=plan.tree_oid,
repository_identity="aios/aios",
)
if requirement.migration_plan_sha256 != plan.sha256:
raise MigrationPlanError("migration_requirement_plan_mismatch")
return plan
class MigrationPreSwitchValidator:
"""Validates a root-owned live proof against the signed migration plan."""
def __init__(self, proof_resolver: Callable[[MigrationPlan], Mapping[str, Any]]):
self.proof_resolver = proof_resolver
def validate(self, migration_plan: MigrationPlan | None) -> dict[str, Any]:
if migration_plan is None:
return {"required": False, "verified": True}
proof = dict(self.proof_resolver(migration_plan))
snapshot_digests = sorted(
contract.snapshot_sha256 for contract in migration_plan.state_contracts
)
required = {
"migration_plan_sha256": migration_plan.sha256,
"migration_id": migration_plan.migration_id,
"source_commit": migration_plan.source_commit,
"tree_oid": migration_plan.tree_oid,
"activation_sha256": migration_plan.activation_sha256,
"state_snapshot_sha256s": snapshot_digests,
"writer_disabled": True,
"task_quiescent": True,
"state_snapshot_verified": True,
"credentials_ready": True,
"rollback_ready": True,
}
failed = [key for key, expected in required.items() if proof.get(key) != expected]
if failed:
raise MigrationPlanError(
"MIGRATION_PRE_SWITCH_PROOF_INVALID:" + ",".join(sorted(failed))
)
return {"required": True, "verified": True, **proof}
class HostMigrationCompiler:
def __init__(
self,
*,
authority_git_dir: Path,
profile_path: Path,
output_root: Path,
expected_owner_uid: int | None = 0,
):
self.authority_git_dir = Path(authority_git_dir)
self.profile_path = Path(profile_path)
self.output_root = Path(output_root)
self.expected_owner_uid = expected_owner_uid
def _profile(self) -> dict[str, Any]:
profile = _secure_json(self.profile_path, expected_owner_uid=self.expected_owner_uid)
required = {
"schema",
"profile_id",
"version",
"repository",
"current_components",
"target_components",
"activation_file",
"state_ref",
"state_db",
"state_mode",
"rollback_window_seconds",
}
if set(profile) != required or profile.get("schema") != PROFILE_SCHEMA:
raise MigrationPlanError("migration_profile_fields_invalid")
if profile.get("repository") != "aios/aios":
raise MigrationPlanError("migration_profile_repository_invalid")
_ident(profile.get("profile_id"), "profile_id")
_ident(profile.get("version"), "version")
if profile.get("state_mode") not in ALLOWED_STATE_MODES:
raise MigrationPlanError("migration_profile_state_mode_invalid")
return profile
@staticmethod
def _component_from_live(spec: Mapping[str, Any]) -> RuntimeComponentBinding:
required = {
"component_id",
"service",
"code_line_id",
"expected_source_oid",
"source_oid_env",
"artifact_sha_env",
"interface_files",
"marker_files",
}
if set(spec) != required:
raise MigrationPlanError("current_component_profile_invalid")
properties = _systemd_properties(str(spec["service"]))
environment = properties["environment"]
source_oid = _oid(environment.get(str(spec["source_oid_env"])), "current_source_oid")
expected_oid = _oid(spec["expected_source_oid"], "expected_source_oid")
if source_oid != expected_oid:
raise MigrationPlanError("CURRENT_RUNTIME_DRIFT")
working = Path(str(properties.get("WorkingDirectory") or ""))
if not working.is_absolute() or not working.is_dir():
raise MigrationPlanError("current_working_directory_invalid")
for marker in spec["marker_files"]:
if not (working / str(marker)).is_file():
raise MigrationPlanError("current_code_line_marker_missing")
interface = _hash_files(working, tuple(str(item) for item in spec["interface_files"]))
raw_artifact = environment.get(str(spec["artifact_sha_env"]))
artifact = str(raw_artifact or "").lower()
artifact_value = artifact if SHA_RE.fullmatch(artifact) else None
return RuntimeComponentBinding(
component_id=str(spec["component_id"]),
service=str(spec["service"]),
code_line_id=str(spec["code_line_id"]),
source_oid=source_oid,
artifact_sha256=artifact_value,
interface_sha256=interface,
)
def _component_from_target(
self, spec: Mapping[str, Any], *, source_commit: str
) -> RuntimeComponentBinding:
required = {"component_id", "service", "code_line_id", "interface_files"}
if set(spec) != required:
raise MigrationPlanError("target_component_profile_invalid")
interface_files = tuple(str(item) for item in spec["interface_files"])
interface = _hash_git_files(self.authority_git_dir, source_commit, interface_files)
return RuntimeComponentBinding(
component_id=str(spec["component_id"]),
service=str(spec["service"]),
code_line_id=str(spec["code_line_id"]),
source_oid=source_commit,
artifact_sha256=None,
interface_sha256=interface,
)
@staticmethod
def _state_compatibility(
state_db: Path,
state_ref: str,
*,
snapshot_path: Path,
snapshot_name: str,
) -> StateMigrationContract:
from aios_task_admission_bootstrap import TaskAdmissionStore
if state_db.is_symlink() or not state_db.is_file():
raise MigrationPlanError("state_db_invalid")
snapshot_path.parent.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory(prefix="aios-migration-probe-") as temp_dir:
temp_root = Path(temp_dir)
consistent_snapshot = temp_root / "pre-migration.sqlite3"
probe_path = temp_root / "schema-probe.sqlite3"
with sqlite3.connect(
f"file:{state_db}?mode=ro", uri=True
) as live, sqlite3.connect(consistent_snapshot) as copy:
live.backup(copy)
with sqlite3.connect(consistent_snapshot) as frozen:
before = _sqlite_schema(frozen)
# Probe target initialization only on a second private copy. The
# rollback snapshot remains byte-for-byte pre-migration state.
with sqlite3.connect(consistent_snapshot) as source, sqlite3.connect(
probe_path
) as probe:
source.backup(probe)
TaskAdmissionStore(probe_path)
with sqlite3.connect(probe_path) as migrated:
after = _sqlite_schema(migrated)
required_tables = {
"task_admission_envelopes",
"task_admission_grants",
"task_admission_receipts",
}
if not required_tables.issubset(after):
raise MigrationPlanError("INCOMPATIBLE_STATE_SCHEMA")
_assert_additive_sqlite(before, after)
snapshot_bytes = consistent_snapshot.read_bytes()
snapshot_sha = hashlib.sha256(snapshot_bytes).hexdigest()
if snapshot_path.exists():
if snapshot_path.is_symlink() or not snapshot_path.is_file():
raise MigrationPlanError("migration_snapshot_invalid")
if hashlib.sha256(snapshot_path.read_bytes()).hexdigest() != snapshot_sha:
raise MigrationPlanError("migration_snapshot_conflict")
else:
temporary = snapshot_path.with_name(snapshot_path.name + ".tmp")
descriptor = os.open(
temporary,
os.O_WRONLY | os.O_CREAT | os.O_EXCL,
0o640,
)
try:
with os.fdopen(descriptor, "wb") as handle:
handle.write(snapshot_bytes)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary, snapshot_path)
if os.name != "nt":
os.chmod(snapshot_path, 0o640)
directory_fd = os.open(snapshot_path.parent, os.O_RDONLY)
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
finally:
try:
temporary.unlink()
except FileNotFoundError:
pass
return StateMigrationContract(
state_id="task-admission",
state_ref=state_ref,
mode="REUSE_IN_PLACE_WITH_SNAPSHOT",
before_schema_sha256=_schema_digest(before),
after_schema_sha256=_schema_digest(after),
snapshot_name=snapshot_name,
snapshot_sha256=snapshot_sha,
snapshot_required=True,
rollback_compatible=True,
)
def compile(
self,
*,
source_admission: Mapping[str, Any],
candidate_id: str,
) -> tuple[MigrationRequirement, MigrationPlan]:
profile = self._profile()
version = str(source_admission.get("version") or "")
source_commit = _oid(source_admission.get("source_commit"), "source_commit")
tree_oid = _oid(source_admission.get("tree_oid"), "tree_oid")
admission_id = _ident(source_admission.get("admission_id"), "source_admission_id")
if (
version != profile["version"]
or source_admission.get("repository") != "aios/aios"
):
raise MigrationPlanError("source_admission_profile_mismatch")
_ident(candidate_id, "candidate_id")
profile_sha = hashlib.sha256(_canonical_json(profile)).hexdigest()
from_components = tuple(
self._component_from_live(item) for item in profile["current_components"]
)
to_components = tuple(
self._component_from_target(item, source_commit=source_commit)
for item in profile["target_components"]
)
if {item.component_id for item in from_components} != {item.component_id for item in to_components}:
raise MigrationPlanError("migration_component_profile_mismatch")
required = any(
next(old.code_line_id for old in from_components if old.component_id == new.component_id)
!= new.code_line_id
for new in to_components
)
if not required:
raise MigrationPlanError("migration_profile_does_not_require_transition")
activation_path = Path(str(profile["activation_file"]))
activation = _secure_json(activation_path, expected_owner_uid=self.expected_owner_uid)
if activation.get("schema") != "aios.task-admission-host-activation.v1":
raise MigrationPlanError("activation_schema_invalid")
state_path = Path(str(profile["state_db"]))
if str(activation.get("state_db") or "") != str(state_path):
raise MigrationPlanError("activation_state_db_mismatch")
root = self.output_root / source_commit
snapshot_name = "snapshots/task-admission.sqlite3"
state_contract = self._state_compatibility(
state_path,
str(profile["state_ref"]),
snapshot_path=root / snapshot_name,
snapshot_name=snapshot_name,
)
migration_id = (
"migration-"
+ hashlib.sha256(
f"{version}\0{candidate_id}\0{source_commit}\0{profile_sha}".encode("utf-8")
).hexdigest()[:24]
)
plan = MigrationPlan(
migration_id=migration_id,
version=version,
candidate_id=candidate_id,
repository_identity="aios/aios",
source_admission_id=admission_id,
source_commit=source_commit,
tree_oid=tree_oid,
profile_id=str(profile["profile_id"]),
profile_sha256=profile_sha,
from_components=from_components,
to_components=to_components,
state_contracts=(state_contract,),
activation_schema=str(activation["schema"]),
activation_sha256=hashlib.sha256(_canonical_json(activation)).hexdigest(),
rollback_window_seconds=int(profile["rollback_window_seconds"]),
)
requirement = MigrationRequirement(
version=version,
candidate_id=candidate_id,
source_commit=source_commit,
repository_identity="aios/aios",
required=True,
reason_code="CROSS_CODE_LINE",
profile_id=str(profile["profile_id"]),
profile_sha256=profile_sha,
migration_plan_sha256=plan.sha256,
)
_durable_json(root / "requirement.json", requirement.to_dict())
_durable_json(root / "plan.json", plan.to_dict())
return requirement, plan
def _verified_source_admission(
*,
source_admission_root: Path,
source_commit: str,
builder_policy: Path,
credential_name: str = "source_admission_hmac",
) -> dict[str, Any]:
"""Resolve one source admission through its existing HMAC verifier.
The migration planner never trusts a caller-selected admission JSON path.
It consumes only a root-owned admission store and the independent
``source_admission_hmac`` credential projected by systemd.
"""
from release_source_admission import open_source_admission_store
credentials = os.environ.get("CREDENTIALS_DIRECTORY", "").strip()
if not credentials:
raise MigrationPlanError("source_admission_credentials_unavailable")
credential_path = Path(credentials) / credential_name
try:
hmac_key = credential_path.read_bytes().strip()
except OSError as exc:
raise MigrationPlanError("source_admission_hmac_unavailable") from exc
try:
store = open_source_admission_store(
source_admission_root,
repository="aios/aios",
policy=builder_policy,
hmac_key=hmac_key,
expected_owner_uid=0 if os.name != "nt" else None,
writable=False,
)
record = store.lookup_source(source_commit)
except Exception as exc:
raise MigrationPlanError("source_admission_verification_failed") from exc
if record is None:
raise MigrationPlanRequired("SOURCE_ADMISSION_REQUIRED")
return dict(record)
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--authority-git-dir", required=True, type=Path)
parser.add_argument("--profile", required=True, type=Path)
parser.add_argument("--output-root", required=True, type=Path)
parser.add_argument("--source-admission-root", required=True, type=Path)
parser.add_argument("--builder-policy", required=True, type=Path)
parser.add_argument("--source-commit", required=True)
parser.add_argument("--candidate-id", required=True)
args = parser.parse_args()
source_commit = _oid(args.source_commit, "source_commit")
compiler = HostMigrationCompiler(
authority_git_dir=args.authority_git_dir,
profile_path=args.profile,
output_root=args.output_root,
expected_owner_uid=0 if os.name != "nt" else None,
)
requirement, plan = compiler.compile(
source_admission=_verified_source_admission(
source_admission_root=args.source_admission_root,
source_commit=source_commit,
builder_policy=args.builder_policy,
),
candidate_id=args.candidate_id,
)
print(
json.dumps(
{
"status": "MIGRATION_PLAN_READY",
"migration_plan_sha256": plan.sha256,
"requirement_sha256": requirement.sha256,
"source_commit": plan.source_commit,
"version": plan.version,
},
sort_keys=True,
)
)
return 0
if __name__ == "__main__":
try:
raise SystemExit(main())
except Exception as exc:
print(
json.dumps(
{
"status": "FAIL_CLOSED",
"error": type(exc).__name__,
"reason": str(exc),
},
sort_keys=True,
)
)
raise SystemExit(2)
@@ -0,0 +1,894 @@
#!/usr/bin/env python3
"""Governed admission of an exact remote Git commit into the release source closure.
The admitter is intentionally separate from the trusted candidate builder:
it may use a narrowly-scoped Gitea read/fetch credential, while the builder
remains PrivateNetwork=true and reads only the authoritative Git object store
plus immutable admission records.
An admission never moves ``main`` and never treats an arbitrary caller-supplied
OID as trusted. It binds:
remote branch ref -> exact full commit OID -> authoritative Git object/tree
-> current authoritative main ancestry -> immutable release recipe metadata.
"""
from __future__ import annotations
import argparse
import hashlib
import hmac
import json
import os
import re
import tempfile
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Mapping, Protocol
from authoritative_git import (
AuthoritativeGitError,
authoritative_head_oid,
authoritative_lineage,
run_authoritative_git,
)
from aios_gitea_machine_transport import GiteaMachineTransport, GiteaTransportError
from release_version import validate_release_version
SCHEMA = "aios.release-source-admission.v1"
RECIPE_MODE = "governed-source-admission"
_OID_RE = re.compile(r"^[0-9a-f]{40}$")
_DIGEST_RE = re.compile(r"^[0-9a-f]{64}$")
_IDENTIFIER_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:@/-]{0,199}$")
_BRANCH_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._/-]{0,239}$")
_REF_PREFIX = "refs/heads/"
class ReleaseSourceAdmissionError(RuntimeError):
"""A stable fail-closed source-admission error code."""
def _canonical(value: Mapping[str, Any]) -> bytes:
return json.dumps(
dict(value),
sort_keys=True,
separators=(",", ":"),
ensure_ascii=True,
).encode("utf-8")
def _oid(value: object, field: str) -> str:
text = str(value or "").strip().lower()
if not _OID_RE.fullmatch(text):
raise ReleaseSourceAdmissionError(f"{field.upper()}_INVALID")
return text
def _digest(value: object, field: str) -> str:
text = str(value or "").strip().lower()
if not _DIGEST_RE.fullmatch(text):
raise ReleaseSourceAdmissionError(f"{field.upper()}_INVALID")
return text
def _identifier(value: object, field: str) -> str:
text = str(value or "").strip()
if not _IDENTIFIER_RE.fullmatch(text) or ".." in text or "\x00" in text:
raise ReleaseSourceAdmissionError(f"{field.upper()}_INVALID")
return text
def _source_ref(value: object) -> str:
text = str(value or "").strip()
if not text.startswith(_REF_PREFIX):
raise ReleaseSourceAdmissionError("SOURCE_REF_INVALID")
branch = text[len(_REF_PREFIX):]
if (
not _BRANCH_RE.fullmatch(branch)
or ".." in branch
or "//" in branch
or "@{" in branch
or branch.endswith(("/", ".", ".lock"))
or any(part in {"", ".", ".."} for part in branch.split("/"))
):
raise ReleaseSourceAdmissionError("SOURCE_REF_INVALID")
return text
def _version(value: object) -> str:
try:
return validate_release_version(value)
except (TypeError, ValueError) as exc:
raise ReleaseSourceAdmissionError("VERSION_INVALID") from exc
def _version_slug(version: str) -> str:
value = re.sub(r"[^A-Za-z0-9]+", "-", version.lower()).strip("-")
if not value:
raise ReleaseSourceAdmissionError("VERSION_INVALID")
return value[:80]
def _now() -> str:
return datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
def _secure_stat(path: Path, *, expected_owner_uid: int | None, directory: bool) -> None:
try:
info = path.lstat()
except OSError as exc:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_UNAVAILABLE") from exc
if path.is_symlink():
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_UNSAFE")
if directory and not path.is_dir():
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_UNSAFE")
if not directory and not path.is_file():
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_UNSAFE")
if expected_owner_uid is not None and info.st_uid != expected_owner_uid:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_OWNER_MISMATCH")
# Group/other write would allow an untrusted process to forge release input.
if info.st_mode & 0o022:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_PERMISSIONS_INVALID")
def _validate_record(
value: Mapping[str, Any],
*,
repository: str,
policy_sha256: str,
hmac_key: bytes,
) -> dict[str, Any]:
if not isinstance(value, Mapping):
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_RECORD_INVALID")
required = {
"schema",
"admission_id",
"repository",
"task_id",
"version",
"candidate_id",
"base_oid",
"source_ref",
"source_commit",
"remote_ref_oid",
"tree_oid",
"purpose",
"policy_sha256",
"admitted_at",
"mac",
}
if set(value) != required or value.get("schema") != SCHEMA:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_RECORD_INVALID")
record = dict(value)
if _identifier(record["repository"], "repository") != repository:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_REPOSITORY_MISMATCH")
_identifier(record["task_id"], "task_id")
record["version"] = _version(record["version"])
_identifier(record["candidate_id"], "candidate_id")
_identifier(record["purpose"], "purpose")
_identifier(record["admission_id"], "admission_id")
record["base_oid"] = _oid(record["base_oid"], "base_oid")
record["source_commit"] = _oid(record["source_commit"], "source_commit")
record["remote_ref_oid"] = _oid(record["remote_ref_oid"], "remote_ref_oid")
record["tree_oid"] = _oid(record["tree_oid"], "tree_oid")
record["source_ref"] = _source_ref(record["source_ref"])
record["policy_sha256"] = _digest(record["policy_sha256"], "policy_sha256")
if record["policy_sha256"] != policy_sha256:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_POLICY_MISMATCH")
if record["remote_ref_oid"] != record["source_commit"]:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_REMOTE_MISMATCH")
admitted_at = record.get("admitted_at")
if not isinstance(admitted_at, str) or not admitted_at.endswith("Z") or len(admitted_at) < 20:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_TIME_INVALID")
supplied_mac = record.get("mac")
if (
not isinstance(supplied_mac, str)
or not re.fullmatch(r"[0-9a-f]{64}", supplied_mac)
):
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_MAC_INVALID")
unsigned = {key: value for key, value in record.items() if key != "mac"}
expected_mac = hmac.new(hmac_key, _canonical(unsigned), hashlib.sha256).hexdigest()
if not hmac.compare_digest(supplied_mac, expected_mac):
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_MAC_INVALID")
return record
class ReleaseSourceBackend(Protocol):
def authoritative_main_oid(self) -> str: ...
def remote_ref_oid(self, source_ref: str) -> str: ...
def fetch_exact_ref(self, source_ref: str, source_commit: str) -> str: ...
def tree_oid(self, source_commit: str) -> str: ...
def require_lineage(self, base_oid: str, source_commit: str) -> None: ...
class ReleaseSourceAdmissionStore:
"""Durable immutable admission records keyed by exact source commit."""
def __init__(
self,
root: str | os.PathLike[str],
*,
repository: str,
policy_sha256: str,
hmac_key: bytes,
expected_owner_uid: int | None = None,
writable: bool = False,
) -> None:
self.root = Path(root)
self.repository = _identifier(repository, "repository")
self.policy_sha256 = _digest(policy_sha256, "policy_sha256")
if not isinstance(hmac_key, (bytes, bytearray)) or len(hmac_key) < 32:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_HMAC_INVALID")
self.hmac_key = bytes(hmac_key)
self.expected_owner_uid = expected_owner_uid
self.writable = bool(writable)
if self.writable and not self.root.exists():
self.root.mkdir(parents=True, mode=0o750)
if self.root.exists():
_secure_stat(
self.root,
expected_owner_uid=self.expected_owner_uid,
directory=True,
)
def _path(self, source_commit: str) -> Path:
return self.root / f"{_oid(source_commit, 'source_commit')}.json"
def register(self, record: Mapping[str, Any]) -> dict[str, Any]:
if not self.writable:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_READ_ONLY")
supplied = dict(record)
if "mac" not in supplied:
supplied["mac"] = hmac.new(
self.hmac_key,
_canonical(supplied),
hashlib.sha256,
).hexdigest()
checked = _validate_record(
supplied,
repository=self.repository,
policy_sha256=self.policy_sha256,
hmac_key=self.hmac_key,
)
path = self._path(checked["source_commit"])
encoded = _canonical(checked) + b"\n"
if path.exists():
_secure_stat(
path,
expected_owner_uid=self.expected_owner_uid,
directory=False,
)
try:
existing = path.read_bytes()
except OSError as exc:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_UNAVAILABLE") from exc
if existing == encoded:
return dict(checked)
try:
previous = _validate_record(
json.loads(existing),
repository=self.repository,
policy_sha256=self.policy_sha256,
hmac_key=self.hmac_key,
)
except (json.JSONDecodeError, UnicodeDecodeError) as exc:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_RECORD_INVALID") from exc
comparable_previous = {
key: value
for key, value in previous.items()
if key not in {"admitted_at", "mac"}
}
comparable_checked = {
key: value
for key, value in checked.items()
if key not in {"admitted_at", "mac"}
}
if comparable_previous == comparable_checked:
return dict(previous)
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_CONFLICT")
self.root.mkdir(parents=True, exist_ok=True, mode=0o750)
_secure_stat(
self.root,
expected_owner_uid=self.expected_owner_uid,
directory=True,
)
fd, raw_name = tempfile.mkstemp(
prefix=f".{checked['source_commit']}.",
suffix=".tmp",
dir=self.root,
)
temp_path = Path(raw_name)
try:
os.fchmod(fd, 0o640)
with os.fdopen(fd, "wb", closefd=True) as handle:
handle.write(encoded)
handle.flush()
os.fsync(handle.fileno())
os.replace(temp_path, path)
os.chmod(path, 0o640)
directory_fd = os.open(self.root, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
except Exception:
try:
os.close(fd)
except OSError:
pass
temp_path.unlink(missing_ok=True)
raise
_secure_stat(
path,
expected_owner_uid=self.expected_owner_uid,
directory=False,
)
return dict(checked)
def rebind_policy(
self,
source_commit: str,
*,
previous_policy_sha256: str,
reason: str,
expected_admission_id: str | None = None,
) -> dict[str, Any]:
"""Rebind one immutable source identity to an explicitly authorized policy.
Only ``policy_sha256`` and ``mac`` change in the admission record.
``admission_id`` and every source/candidate/tree binding remain immutable.
A separate HMAC-authenticated history record makes the temporary §8.1
trust expansion and its restoration auditable and idempotent.
"""
if not self.writable:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_READ_ONLY")
previous_policy = _digest(
previous_policy_sha256,
"previous_policy_sha256",
)
reason_value = _identifier(reason, "reason")
path = self._path(source_commit)
if not path.exists():
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_REQUIRED")
_secure_stat(
path,
expected_owner_uid=self.expected_owner_uid,
directory=False,
)
try:
raw = path.read_bytes()
decoded = json.loads(raw)
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ReleaseSourceAdmissionError(
"SOURCE_ADMISSION_RECORD_INVALID"
) from exc
try:
previous = _validate_record(
decoded,
repository=self.repository,
policy_sha256=previous_policy,
hmac_key=self.hmac_key,
)
except ReleaseSourceAdmissionError:
# Idempotent replay after a successful rebind validates under the
# current policy instead of silently weakening the old-policy gate.
current = _validate_record(
decoded,
repository=self.repository,
policy_sha256=self.policy_sha256,
hmac_key=self.hmac_key,
)
if expected_admission_id and current["admission_id"] != expected_admission_id:
raise ReleaseSourceAdmissionError(
"SOURCE_ADMISSION_ID_MISMATCH"
)
return dict(current)
if expected_admission_id and previous["admission_id"] != expected_admission_id:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_ID_MISMATCH")
rebound = {
key: value
for key, value in previous.items()
if key != "mac"
}
rebound["policy_sha256"] = self.policy_sha256
rebound["mac"] = hmac.new(
self.hmac_key,
_canonical(rebound),
hashlib.sha256,
).hexdigest()
checked = _validate_record(
rebound,
repository=self.repository,
policy_sha256=self.policy_sha256,
hmac_key=self.hmac_key,
)
if checked["admission_id"] != previous["admission_id"]:
raise ReleaseSourceAdmissionError(
"SOURCE_ADMISSION_ID_CHANGED"
)
for field in (
"repository",
"version",
"candidate_id",
"base_oid",
"source_ref",
"source_commit",
"remote_ref_oid",
"tree_oid",
"purpose",
"task_id",
):
if checked[field] != previous[field]:
raise ReleaseSourceAdmissionError(
"SOURCE_ADMISSION_REBIND_SCOPE_CHANGED"
)
encoded = _canonical(checked) + b"\n"
history_payload = {
"schema": "aios.release-source-admission-policy-rebind.v1",
"admission_id": checked["admission_id"],
"source_commit": checked["source_commit"],
"previous_policy_sha256": previous_policy,
"policy_sha256": self.policy_sha256,
"reason": reason_value,
"record_sha256": hashlib.sha256(encoded).hexdigest(),
"rebound_at": _now(),
}
history_payload["mac"] = hmac.new(
self.hmac_key,
_canonical(history_payload),
hashlib.sha256,
).hexdigest()
history_root = self.root / "rebind-history"
if not history_root.exists():
history_root.mkdir(mode=0o750)
_secure_stat(
history_root,
expected_owner_uid=self.expected_owner_uid,
directory=True,
)
history_name = (
f"{checked['source_commit']}."
f"{previous_policy[:12]}-{self.policy_sha256[:12]}."
f"{hashlib.sha256(reason_value.encode()).hexdigest()[:12]}.json"
)
history_path = history_root / history_name
history_bytes = _canonical(history_payload) + b"\n"
fd, raw_name = tempfile.mkstemp(
prefix=f".{checked['source_commit']}.rebind.",
suffix=".tmp",
dir=self.root,
)
temp_path = Path(raw_name)
try:
os.fchmod(fd, 0o640)
with os.fdopen(fd, "wb", closefd=True) as handle:
handle.write(encoded)
handle.flush()
os.fsync(handle.fileno())
if history_path.exists():
_secure_stat(
history_path,
expected_owner_uid=self.expected_owner_uid,
directory=False,
)
if history_path.read_bytes() != history_bytes:
raise ReleaseSourceAdmissionError(
"SOURCE_ADMISSION_REBIND_HISTORY_CONFLICT"
)
else:
history_fd = os.open(
history_path,
os.O_WRONLY
| os.O_CREAT
| os.O_EXCL
| getattr(os, "O_NOFOLLOW", 0),
0o640,
)
try:
os.write(history_fd, history_bytes)
os.fsync(history_fd)
finally:
os.close(history_fd)
os.replace(temp_path, path)
os.chmod(path, 0o640)
directory_fd = os.open(
self.root,
os.O_RDONLY | getattr(os, "O_DIRECTORY", 0),
)
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
except Exception:
try:
os.close(fd)
except OSError:
pass
temp_path.unlink(missing_ok=True)
raise
return dict(checked)
def lookup_source(self, source_commit: str) -> dict[str, Any] | None:
path = self._path(source_commit)
if not path.exists():
return None
_secure_stat(
path,
expected_owner_uid=self.expected_owner_uid,
directory=False,
)
try:
raw = path.read_bytes()
if len(raw) > 64 * 1024:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_RECORD_INVALID")
decoded = json.loads(raw)
except ReleaseSourceAdmissionError:
raise
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_RECORD_INVALID") from exc
return _validate_record(
decoded,
repository=self.repository,
policy_sha256=self.policy_sha256,
hmac_key=self.hmac_key,
)
def resolve_recipe(
self,
version: str,
candidate_id: str | None,
source_commit: str,
) -> dict[str, str]:
record = self.lookup_source(source_commit)
if record is None:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_NOT_FOUND")
if record["version"] != version:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_VERSION_MISMATCH")
supplied_candidate = str(candidate_id or "")
if supplied_candidate and supplied_candidate != record["candidate_id"]:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_CANDIDATE_MISMATCH")
return {
"mode": RECIPE_MODE,
"task_id": record["task_id"],
"candidate_id": record["candidate_id"],
"base_oid": record["base_oid"],
"source_commit": record["source_commit"],
"version": record["version"],
"purpose": record["purpose"],
}
@dataclass(slots=True)
class GiteaReleaseSourceBackend:
transport: GiteaMachineTransport
git_dir: Path
def authoritative_main_oid(self) -> str:
return authoritative_head_oid(git_dir=self.git_dir)
def remote_ref_oid(self, source_ref: str) -> str:
ref = _source_ref(source_ref)
branch = ref[len(_REF_PREFIX):]
value = self.transport.read_branch(branch)
if value is None:
raise ReleaseSourceAdmissionError("SOURCE_REF_NOT_FOUND")
return _oid(value, "remote_ref_oid")
def fetch_exact_ref(self, source_ref: str, source_commit: str) -> str:
ref = _source_ref(source_ref)
branch = ref[len(_REF_PREFIX):]
try:
value = self.transport.fetch_branch_exact(
branch,
_oid(source_commit, "source_commit"),
)
except GiteaTransportError as exc:
raise ReleaseSourceAdmissionError("SOURCE_FETCH_FAILED") from exc
return _oid(value, "fetched_source_commit")
def tree_oid(self, source_commit: str) -> str:
source = _oid(source_commit, "source_commit")
try:
result = run_authoritative_git(
["rev-parse", "--verify", f"{source}^{{tree}}"],
git_dir=self.git_dir,
)
except AuthoritativeGitError as exc:
raise ReleaseSourceAdmissionError("SOURCE_COMMIT_NOT_FOUND") from exc
return _oid(result.stdout.strip().lower(), "tree_oid")
def require_lineage(self, base_oid: str, source_commit: str) -> None:
try:
authoritative_lineage(
_oid(base_oid, "base_oid"),
_oid(source_commit, "source_commit"),
git_dir=self.git_dir,
)
except AuthoritativeGitError as exc:
raise ReleaseSourceAdmissionError("SOURCE_LINEAGE_MISMATCH") from exc
class ReleaseSourceAdmitter:
def __init__(
self,
store: ReleaseSourceAdmissionStore,
backend: ReleaseSourceBackend,
*,
purpose: str = "AIOS_TRUSTED_RELEASE_BUILDER",
) -> None:
if not store.writable:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_READ_ONLY")
self.store = store
self.backend = backend
self.purpose = _identifier(purpose, "purpose")
def admit(
self,
*,
version: str,
task_id: str,
source_ref: str,
source_commit: str,
) -> dict[str, Any]:
version_value = _version(version)
task_value = _identifier(task_id, "task_id")
ref_value = _source_ref(source_ref)
source_value = _oid(source_commit, "source_commit")
remote_before = self.backend.remote_ref_oid(ref_value)
if remote_before != source_value:
raise ReleaseSourceAdmissionError("SOURCE_REF_MOVED")
base_oid = _oid(self.backend.authoritative_main_oid(), "base_oid")
fetched = _oid(
self.backend.fetch_exact_ref(ref_value, source_value),
"fetched_source_commit",
)
if fetched != source_value:
raise ReleaseSourceAdmissionError("SOURCE_FETCH_MISMATCH")
remote_after = self.backend.remote_ref_oid(ref_value)
if remote_after != source_value:
raise ReleaseSourceAdmissionError("SOURCE_REF_MOVED")
self.backend.require_lineage(base_oid, source_value)
tree_oid = _oid(self.backend.tree_oid(source_value), "tree_oid")
candidate_id = f"rc-{_version_slug(version_value)}-{source_value[:12]}"
admission_id = "source-admission-" + hashlib.sha256(
f"{self.store.repository}\0{version_value}\0{source_value}\0{ref_value}".encode("utf-8")
).hexdigest()[:24]
record = {
"schema": SCHEMA,
"admission_id": admission_id,
"repository": self.store.repository,
"task_id": task_value,
"version": version_value,
"candidate_id": candidate_id,
"base_oid": base_oid,
"source_ref": ref_value,
"source_commit": source_value,
"remote_ref_oid": remote_after,
"tree_oid": tree_oid,
"purpose": self.purpose,
"policy_sha256": self.store.policy_sha256,
"admitted_at": _now(),
}
return self.store.register(record)
def _sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def source_admission_policy_sha256(path: Path) -> str:
"""Canonical Source Admission policy binding: exact policy file bytes.
Builder/controller readiness may use a canonical JSON digest internally,
but Source Admission records are intentionally bound to the immutable policy
file bytes. Every admission producer/consumer must call this helper.
"""
try:
return _sha256_file(Path(path))
except OSError as exc:
raise ReleaseSourceAdmissionError("SOURCE_POLICY_UNAVAILABLE") from exc
def open_source_admission_store(
root: str | os.PathLike[str],
*,
repository: str,
policy: Path,
hmac_key: bytes,
expected_owner_uid: int | None,
writable: bool,
) -> ReleaseSourceAdmissionStore:
"""The only supported constructor for policy-bound Source Admission stores."""
return ReleaseSourceAdmissionStore(
root,
repository=repository,
policy_sha256=source_admission_policy_sha256(policy),
hmac_key=hmac_key,
expected_owner_uid=expected_owner_uid,
writable=writable,
)
def validate_isolated_peer_policy_delta(
previous_policy: Path,
policy: Path,
isolated_peer_cgroup: str,
) -> tuple[str, str, str]:
"""Allow exactly one temporary isolated peer add/remove and no other drift."""
peer = str(isolated_peer_cgroup or "").strip()
if (
not peer.startswith("/system.slice/")
or not peer.endswith(".service")
or "*" in peer
or ".." in peer
):
raise ReleaseSourceAdmissionError("SOURCE_POLICY_PEER_INVALID")
try:
from aios_trusted_candidate_builder import load_builder_policy
before = dict(load_builder_policy(Path(previous_policy)))
after = dict(load_builder_policy(Path(policy)))
except Exception as exc:
raise ReleaseSourceAdmissionError("SOURCE_POLICY_INVALID") from exc
before_cgroups = tuple(before.get("allowed_peer_cgroups") or ())
after_cgroups = tuple(after.get("allowed_peer_cgroups") or ())
if len(set(before_cgroups)) != len(before_cgroups) or len(set(after_cgroups)) != len(after_cgroups):
raise ReleaseSourceAdmissionError("SOURCE_POLICY_PEER_INVALID")
if any("*" in value or ".." in value for value in (*before_cgroups, *after_cgroups)):
raise ReleaseSourceAdmissionError("SOURCE_POLICY_PEER_INVALID")
before_without = dict(before)
after_without = dict(after)
before_without["allowed_peer_cgroups"] = []
after_without["allowed_peer_cgroups"] = []
if _canonical(before_without) != _canonical(after_without):
raise ReleaseSourceAdmissionError("SOURCE_POLICY_REBIND_SCOPE_CHANGED")
before_set, after_set = set(before_cgroups), set(after_cgroups)
added = after_set - before_set
removed = before_set - after_set
if added == {peer} and not removed:
direction = "ADD_ISOLATED_PEER"
elif removed == {peer} and not added:
direction = "REMOVE_ISOLATED_PEER"
else:
raise ReleaseSourceAdmissionError("SOURCE_POLICY_PEER_DELTA_INVALID")
return (
source_admission_policy_sha256(previous_policy),
source_admission_policy_sha256(policy),
direction,
)
def _main() -> None:
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(dest="command", required=True)
admit = subparsers.add_parser("admit")
admit.add_argument("--version", required=True)
admit.add_argument("--task-id", required=True)
admit.add_argument("--source-ref", required=True)
admit.add_argument("--source-commit", required=True)
admit.add_argument("--policy", required=True, type=Path)
admit.add_argument("--store-root", required=True, type=Path)
rebind = subparsers.add_parser("rebind-policy")
rebind.add_argument("--source-commit", required=True)
rebind.add_argument("--previous-policy", required=True, type=Path)
rebind.add_argument("--policy", required=True, type=Path)
rebind.add_argument("--isolated-peer-cgroup", required=True)
rebind.add_argument("--expected-admission-id", required=True)
rebind.add_argument("--reason", required=True)
rebind.add_argument("--store-root", required=True, type=Path)
arguments = parser.parse_args()
repository = os.environ.get("AIOS_GITOPS_REPOSITORY_IDENTITY", "")
if not repository:
raise SystemExit("AIOS_GITOPS_REPOSITORY_IDENTITY is required")
credentials = os.environ.get("CREDENTIALS_DIRECTORY", "")
if not credentials:
raise SystemExit("CREDENTIALS_DIRECTORY is required")
try:
admission_hmac = (
Path(credentials) / "source_admission_hmac"
).read_bytes().strip()
except OSError as exc:
raise SystemExit(
"source_admission_hmac credential is required"
) from exc
if arguments.command == "rebind-policy":
try:
previous_sha, policy_sha, direction = (
validate_isolated_peer_policy_delta(
arguments.previous_policy,
arguments.policy,
arguments.isolated_peer_cgroup,
)
)
store = open_source_admission_store(
arguments.store_root,
repository=repository,
policy=arguments.policy,
hmac_key=admission_hmac,
expected_owner_uid=(
os.geteuid() if hasattr(os, "geteuid") else None
),
writable=True,
)
result = store.rebind_policy(
arguments.source_commit,
previous_policy_sha256=previous_sha,
reason=f"{arguments.reason}:{direction}",
expected_admission_id=arguments.expected_admission_id,
)
except ReleaseSourceAdmissionError as exc:
raise SystemExit(str(exc)) from exc
print(
json.dumps(
{
**result,
"rebind_direction": direction,
"previous_policy_sha256": previous_sha,
"policy_sha256": policy_sha,
},
sort_keys=True,
separators=(",", ":"),
)
)
return
api_base = os.environ.get("AIOS_GITEA_API_BASE", "")
remote_url = os.environ.get("AIOS_GITEA_REMOTE_URL", "")
raw_git_dir = os.environ.get("AIOS_GITOPS_AUTHORITATIVE_GIT_DIR", "")
if not api_base or not remote_url or not raw_git_dir:
raise SystemExit("release source admission environment is incomplete")
git_dir = Path(raw_git_dir)
transport = GiteaMachineTransport(
api_base=api_base,
repository_id=repository,
remote_url=remote_url,
authority_git_dir=git_dir,
credential_name=os.environ.get(
"AIOS_GITEA_MACHINE_CREDENTIAL_NAME",
"gitea-machine-token",
),
)
store = open_source_admission_store(
arguments.store_root,
repository=repository,
policy=arguments.policy,
hmac_key=admission_hmac,
expected_owner_uid=os.geteuid() if hasattr(os, "geteuid") else None,
writable=True,
)
result = ReleaseSourceAdmitter(
store,
GiteaReleaseSourceBackend(transport=transport, git_dir=git_dir),
).admit(
version=arguments.version,
task_id=arguments.task_id,
source_ref=arguments.source_ref,
source_commit=arguments.source_commit,
)
print(json.dumps(result, sort_keys=True, separators=(",", ":")))
if __name__ == "__main__":
_main()
+185 -39
View File
@@ -25,6 +25,12 @@ from typing import Any, Callable, Iterable, Mapping, Protocol
from cryptography.exceptions import InvalidSignature
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
from release_migration_plan import (
MigrationPlan,
MigrationPlanError,
MigrationPreSwitchValidator,
)
from silent_release_controller import (
CANDIDATE_ID_RE,
OID_RE,
@@ -562,6 +568,7 @@ class ReleasePlan:
tag: TagPlan
finalization: FinalizationPlan
policy_sha256: str
migration: MigrationPlan | None = None
def __post_init__(self) -> None:
_validate_sha("policy_sha256", self.policy_sha256)
@@ -588,16 +595,32 @@ class ReleasePlan:
!= self.finalization.expected_previous_baseline_sha256
):
raise ValueError("previous baseline is inconsistent")
if self.migration is not None:
migration = self.migration
if (
migration.version != self.tag.version
or migration.source_commit != self.deployment.source_commit
or migration.tree_oid != self.deployment.tree_oid
or migration.repository_identity
!= self.deployment.repository_identity
):
raise ValueError("migration plan release binding is inconsistent")
return self
@property
def canonical_payload(self) -> dict[str, Any]:
return {
# Legacy release-plan bytes/hashes remain unchanged when migration is
# not required. A cross-code-line migration is an optional signed
# sub-contract and is therefore present only when required.
payload = {
"deployment": self.deployment.canonical_payload,
"finalization": self.finalization.canonical_payload,
"policy_sha256": self.policy_sha256,
"tag": self.tag.canonical_payload,
}
if self.migration is not None:
payload["migration"] = self.migration.to_dict()
return payload
@property
def sha256(self) -> str:
@@ -612,6 +635,7 @@ class ReleasePlan:
def from_dict(cls, payload: Mapping[str, Any]) -> "ReleasePlan":
values = dict(payload)
values.pop("sha256", None)
migration_value = values.get("migration")
return cls(
deployment=DeploymentPlan.from_dict(values["deployment"]),
tag=TagPlan.from_dict(values["tag"]),
@@ -619,6 +643,11 @@ class ReleasePlan:
values["finalization"]
),
policy_sha256=values["policy_sha256"],
migration=(
MigrationPlan.from_dict(migration_value)
if isinstance(migration_value, Mapping)
else None
),
)
@@ -1215,6 +1244,19 @@ class TrustedCandidateBundle:
raise CapabilityAuthorizationError(
"trusted candidate and release plan are inconsistent"
)
if plan.migration is not None:
try:
plan.migration.assert_release_binding(
version=plan.tag.version,
candidate_id=candidate.candidate_id,
source_commit=candidate.source_commit,
tree_oid=candidate.tree_oid,
repository_identity=plan.deployment.repository_identity,
)
except MigrationPlanError as exc:
raise CapabilityAuthorizationError(
"trusted candidate migration binding is invalid"
) from exc
if attestation_verifier is not None:
if self.attestation is None:
raise CapabilityAuthorizationError(
@@ -3090,6 +3132,7 @@ class ReleaseOrchestrator:
runtime_rollback: Callable[..., Mapping[str, Any]],
tag_publish: HardenedTagController,
finalizer: GovernedFinalizationCoordinator,
migration_pre_switch_validator: MigrationPreSwitchValidator | None = None,
):
self.candidates = candidates
self.plans = plans
@@ -3104,6 +3147,7 @@ class ReleaseOrchestrator:
self.runtime_rollback = runtime_rollback
self.tag_publish = tag_publish
self.finalizer = finalizer
self.migration_pre_switch_validator = migration_pre_switch_validator
def _transition(
self,
@@ -3304,6 +3348,14 @@ class ReleaseOrchestrator:
plan.deployment.repository_identity
),
)
if plan.migration is not None:
validator = self.migration_pre_switch_validator
if validator is None:
raise StaleApprovalError(
"MIGRATION_PLAN_REQUIRED: "
"migration pre-switch validator is unavailable"
)
validator.validate(plan.migration)
saga = self._transition(
saga,
SagaState.SWITCH_PENDING,
@@ -3531,6 +3583,7 @@ class SilentReleaseActionService:
trusted_policy: TrustedReleasePolicy | None = None,
orchestrator: ReleaseOrchestrator | None = None,
builder_readiness_probe: Callable[[], Mapping[str, Any]] | None = None,
source_admission_resolver: Callable[[str], Mapping[str, Any] | None] | None = None,
allow_legacy_candidate_input: bool = False,
expected_repository: str = "",
policy_sha256: str = "",
@@ -3547,6 +3600,7 @@ class SilentReleaseActionService:
self.trusted_policy = trusted_policy
self.orchestrator = orchestrator
self.builder_readiness_probe = builder_readiness_probe
self.source_admission_resolver = source_admission_resolver
self.allow_legacy_candidate_input = bool(
allow_legacy_candidate_input
)
@@ -3638,6 +3692,40 @@ class SilentReleaseActionService:
version = str(payload.get("version") or "")
if version:
_validate_identifier("version", version)
source_admission: dict[str, Any] | None = None
resolver = self.source_admission_resolver
if resolver is not None:
try:
raw_admission = resolver(source_commit)
except Exception as exc:
raise CapabilityAuthorizationError(
"release source admission is invalid"
) from exc
if raw_admission is not None:
source_admission = dict(raw_admission)
admitted_source = str(
source_admission.get("source_commit") or ""
)
admitted_version = str(source_admission.get("version") or "")
admission_id = str(
source_admission.get("admission_id") or ""
)
if (
admitted_source != source_commit
or not admitted_version
or not admission_id
):
raise CapabilityAuthorizationError(
"release source admission is invalid"
)
_validate_identifier("version", admitted_version)
_validate_identifier("admission_id", admission_id)
if version and version != admitted_version:
raise CapabilityAuthorizationError(
"release source admission version mismatch"
)
version = admitted_version
intent_id = "intent-" + hashlib.sha256(
f"{event_id}\0{source_commit}".encode("utf-8")
).hexdigest()[:24]
@@ -3657,47 +3745,83 @@ class SilentReleaseActionService:
}
if version:
record["version"] = version
if source_admission is not None:
record["source_admission_id"] = source_admission["admission_id"]
if was_existing:
existing = json.loads(path.read_text(encoding="utf-8"))
prerequisite_states = {
"SOURCE_ADMISSION_REQUIRED",
"MIGRATION_PLAN_REQUIRED",
}
existing_status = str(existing.get("status") or "")
existing_version = str(existing.get("version") or "")
existing_admission = str(existing.get("source_admission_id") or "")
incoming_admission = str(record.get("source_admission_id") or "")
if (
existing.get("event_id") != event_id
or existing.get("source_commit") != source_commit
or str(existing.get("version") or "") != version
or (existing_version and existing_version != version)
or (existing_admission and existing_admission != incoming_admission)
):
raise SagaConflict(
"release intent id is bound to different evidence"
)
if existing.get("status") == "BUILDING":
try:
started_at = _parse_utc(
str(existing.get("attempt_started_at") or "")
)
elapsed = (
datetime.now(timezone.utc) - started_at
).total_seconds()
except (TypeError, ValueError):
elapsed = self.builder_attempt_timeout_seconds + 1
if elapsed <= self.builder_attempt_timeout_seconds:
return self._intent_response(existing, idempotent=True)
existing.update(
{
"failure_code": "TRUSTED_BUILDER_CRASH_RECOVERED",
"last_recovery_code": "TRUSTED_BUILDER_CRASH_RECOVERED",
"recovery_count": int(existing.get("recovery_count") or 0) + 1,
"retryable": int(existing.get("attempt_count") or 0)
< self.max_builder_attempts,
"status": "BUILDER_RETRYABLE_FAILURE",
"updated_at": utc_now(),
}
)
_durable_json(path, existing)
# A source-admission prerequisite may initially know only the
# event/source identity. Once the verified admission becomes
# available, bind the previously missing version/admission ID to
# the same durable intent. Existing non-empty evidence can never
# be replaced.
if (
existing.get("status") != "BUILDER_RETRYABLE_FAILURE"
or int(existing.get("attempt_count") or 0)
>= self.max_builder_attempts
(not existing_version and version)
or (not existing_admission and incoming_admission)
):
return self._intent_response(existing, idempotent=True)
record = dict(existing)
if existing_status != "SOURCE_ADMISSION_REQUIRED":
raise SagaConflict(
"release intent prerequisite binding changed"
)
if version:
existing["version"] = version
if incoming_admission:
existing["source_admission_id"] = incoming_admission
existing["updated_at"] = utc_now()
_durable_json(path, existing)
if existing_status in prerequisite_states:
# Prerequisite blockers are not builder attempts. Replaying
# the same intent after Source Admission or Migration Plan
# material appears is the expected autonomous recovery path.
record = dict(existing)
else:
if existing.get("status") == "BUILDING":
try:
started_at = _parse_utc(
str(existing.get("attempt_started_at") or "")
)
elapsed = (
datetime.now(timezone.utc) - started_at
).total_seconds()
except (TypeError, ValueError):
elapsed = self.builder_attempt_timeout_seconds + 1
if elapsed <= self.builder_attempt_timeout_seconds:
return self._intent_response(existing, idempotent=True)
existing.update(
{
"failure_code": "TRUSTED_BUILDER_CRASH_RECOVERED",
"last_recovery_code": "TRUSTED_BUILDER_CRASH_RECOVERED",
"recovery_count": int(existing.get("recovery_count") or 0) + 1,
"retryable": int(existing.get("attempt_count") or 0)
< self.max_builder_attempts,
"status": "BUILDER_RETRYABLE_FAILURE",
"updated_at": utc_now(),
}
)
_durable_json(path, existing)
if (
existing.get("status") != "BUILDER_RETRYABLE_FAILURE"
or int(existing.get("attempt_count") or 0)
>= self.max_builder_attempts
):
return self._intent_response(existing, idempotent=True)
record = dict(existing)
if self.candidate_builder is None:
record.update(
{
@@ -3742,14 +3866,36 @@ class SilentReleaseActionService:
)
_durable_json(path, record)
return self._intent_response(record, idempotent=False)
except CapabilityAuthorizationError:
record.update(
{
"failure_code": "TRUSTED_BUILDER_REJECTED",
"retryable": False,
"status": "BUILDER_TERMINAL_FAILURE",
}
)
except CapabilityAuthorizationError as exc:
prerequisite_code = {
"release source admission required": "SOURCE_ADMISSION_REQUIRED",
"migration plan required": "MIGRATION_PLAN_REQUIRED",
}.get(str(exc))
if prerequisite_code is not None:
record.update(
{
# Prerequisite discovery is a control-plane retry, not
# a business/builder execution attempt.
"attempt_count": max(
0, int(record.get("attempt_count") or 0) - 1
),
"failure_code": prerequisite_code,
"retryable": True,
"status": prerequisite_code,
}
)
else:
record.update(
{
"failure_code": (
"MIGRATION_PLAN_INVALID"
if str(exc) == "migration plan invalid"
else "TRUSTED_BUILDER_REJECTED"
),
"retryable": False,
"status": "BUILDER_TERMINAL_FAILURE",
}
)
_durable_json(path, record)
return self._intent_response(record, idempotent=False)
except (ConnectionError, OSError, TimeoutError):
+167 -71
View File
@@ -105,6 +105,28 @@ pre { white-space: pre-wrap; word-break: break-word; }
</div>
</div>
<script>
// Security: all dynamic values (API/git/agent/owner-gate) are rendered with
// textContent / DOM APIs. innerHTML is never used to inject untrusted data.
function clearNode(node) {
if (!node) return;
while (node.firstChild) node.removeChild(node.firstChild);
}
function setText(id, text) {
const n = document.getElementById(id);
if (n) n.textContent = text;
}
function addClass(node, cls) { if (node) node.classList.add(cls); }
function div(cls) { const d = document.createElement('div'); if (cls) addClass(d, cls); return d; }
function span(cls, text) { const s = document.createElement('span'); if (cls) addClass(s, cls); if (text != null) s.textContent = text; return s; }
// Centralized CSRF token source: server-issued meta tag, or double-submit cookie.
function csrfToken() {
const meta = document.querySelector('meta[name="csrf-token"]');
if (meta && meta.content) return meta.content;
const m = document.cookie.match(/(?:^|;\s*)aios_csrf=([^;]+)/);
return m ? decodeURIComponent(m[1]) : '';
}
async function fetchData() {
const r = await fetch('/admin/ai-evolution/api/data');
const data = await r.json();
@@ -124,130 +146,204 @@ async function fetchData() {
async function loadVersions() {
const el = document.getElementById('versions-list');
el.innerHTML = '<div class="loading">Loading versions...</div>';
clearNode(el);
const loading = div('loading'); loading.textContent = 'Loading versions...'; el.appendChild(loading);
try {
const r = await fetch('/admin/ai-evolution/api/versions');
const data = await r.json();
if (data.error) { el.innerHTML = '<div class="error">' + data.error + '</div>'; return; }
let html = '<div class="grid">';
clearNode(el);
if (data.error) { const e = div('error'); e.textContent = data.error; el.appendChild(e); return; }
const grid = div('grid');
for (const v of data.versions || []) {
html += '<div class="card"><div class="card-title">' + v.tag + '</div><div class="card-value">' + (v.version || v.tag) + '</div></div>';
const card = div('card');
const t = div('card-title'); t.textContent = v.tag || '';
const val = div('card-value'); val.textContent = v.version || v.tag || '';
card.appendChild(t); card.appendChild(val); grid.appendChild(card);
}
html += '</div>';
el.innerHTML = html;
} catch(e) {
el.innerHTML = '<div class="error">Failed to load versions: ' + e.message + '</div>';
el.appendChild(grid);
} catch (e) {
const err = div('error'); err.textContent = 'Failed to load versions: ' + e.message; el.appendChild(err);
}
}
async function loadCommands() {
const el = document.getElementById('commands-list');
el.innerHTML = '<div class="loading">Loading commands...</div>';
clearNode(el);
const loading = div('loading'); loading.textContent = 'Loading commands...'; el.appendChild(loading);
try {
const r = await fetch('/admin/ai-evolution/api/data');
const data = await r.json();
let cmds = data.commands || [];
let html = '<div style="display:flex;gap:24px;margin-bottom:16px;">';
html += '<div><div class="card"><div class="card-title">Total</div><div class="card-value">' + cmds.length + '</div></div></div>';
html += '<div><div class="card"><div class="card-title">By Category</div>';
const cmds = data.commands || [];
clearNode(el);
const wrapper = div('');
wrapper.style.display = 'flex'; wrapper.style.gap = '24px'; wrapper.style.marginBottom = '16px';
const totalCard = div('card');
const tcTitle = div('card-title'); tcTitle.textContent = 'Total';
const tcVal = div('card-value'); tcVal.textContent = String(cmds.length);
totalCard.appendChild(tcTitle); totalCard.appendChild(tcVal); wrapper.appendChild(totalCard);
const catCard = div('card');
const ccTitle = div('card-title'); ccTitle.textContent = 'By Category';
catCard.appendChild(ccTitle);
const cats = {};
for (const c of cmds) { const k = c.category || 'uncategorized'; cats[k] = (cats[k] || 0) + 1; }
for (const [k, v] of Object.entries(cats)) html += '<div style="font-size:14px;color:#8b949e;margin:4px 0;">' + k + ': ' + v + '</div>';
html += '</div></div></div>';
for (const cmd of cmds.slice().reverse()) {
html += '<div class="command-item"><div class="meta">' + cmd.category + ' | ' + cmd.filename + '</div><div class="name">' + cmd.title + '</div><div style="font-size:13px;color:#c9d1d9;margin-top:4px;">' + cmd.summary + '</div></div>';
for (const [k, v] of Object.entries(cats)) {
const line = div(''); line.style.fontSize = '14px'; line.style.color = '#8b949e'; line.style.margin = '4px 0';
line.textContent = k + ': ' + v; catCard.appendChild(line);
}
el.innerHTML = html;
} catch(e) {
el.innerHTML = '<div class="error">Failed to load commands: ' + e.message + '</div>';
wrapper.appendChild(catCard); el.appendChild(wrapper);
for (const cmd of cmds.slice().reverse()) {
const item = div('command-item');
const meta = div('meta'); meta.textContent = (cmd.category || '') + ' | ' + (cmd.filename || '');
const name = div('name'); name.textContent = cmd.title || '';
const summary = div(''); summary.style.fontSize = '13px'; summary.style.color = '#c9d1d9'; summary.style.marginTop = '4px';
summary.textContent = cmd.summary || '';
item.appendChild(meta); item.appendChild(name); item.appendChild(summary);
el.appendChild(item);
}
} catch (e) {
const err = div('error'); err.textContent = 'Failed to load commands: ' + e.message; el.appendChild(err);
}
}
async function loadAgents() {
const el = document.getElementById('agents-list');
el.innerHTML = '<div class="loading">Loading agent runs...</div>';
clearNode(el);
const loading = div('loading'); loading.textContent = 'Loading agent runs...'; el.appendChild(loading);
try {
const r = await fetch('/admin/ai-evolution/api/agents');
const data = await r.json();
if (data.error) { el.innerHTML = '<div class="error">' + data.error + '</div>'; return; }
let html = '';
for (const agent of data.agents || []) {
html += '<div class="agent-item"><div class="agent-title">' + agent.name + ' <span class="state state-' + (agent.status === 'running' ? 'running' : (agent.status === 'ok' ? 'completed' : 'failed')) + '">' + agent.status + '</span></div><div class="agent-meta">' + (agent.runs || 0) + ' runs | last: ' + (agent.last_run || '-') + '</div></div>';
clearNode(el);
if (data.error) { const e = div('error'); e.textContent = data.error; el.appendChild(e); return; }
const agents = data.agents || [];
if (!agents.length) {
const none = div(''); none.style.padding = '24px'; none.style.color = '#8b949e';
none.textContent = 'No agent runs recorded yet.'; el.appendChild(none); return;
}
el.innerHTML = html || '<div style="padding:24px;color:#8b949e;">No agent runs recorded yet.</div>';
} catch(e) {
el.innerHTML = '<div class="error">Failed to load agents: ' + e.message + '</div>';
for (const agent of agents) {
const item = div('agent-item');
const title = div('agent-title');
title.appendChild(document.createTextNode(agent.name || ''));
const st = (agent.status === 'running') ? 'running' : ((agent.status === 'ok') ? 'completed' : 'failed');
const badge = span('state state-' + st, agent.status || '');
title.appendChild(badge);
const meta = div('agent-meta');
meta.textContent = (agent.runs || 0) + ' runs | last: ' + (agent.last_run || '-');
item.appendChild(title); item.appendChild(meta); el.appendChild(item);
}
} catch (e) {
const err = div('error'); err.textContent = 'Failed to load agents: ' + e.message; el.appendChild(err);
}
}
function showPanel(name) {
async function loadOwnerGate() {
const el = document.getElementById('arch-owner-content');
el.innerHTML = '<div class="loading">Loading owner gate data...</div>';
clearNode(el);
const loading = div('loading'); loading.textContent = 'Loading owner gate data...'; el.appendChild(loading);
try {
const r = await fetch('/admin/ai-evolution/api/owner-gate');
const data = await r.json();
let html = '<div class="row">';
html += '<div class="col"><div class="card"><div class="card-title">Pending Requests</div><div class="card-value">' + data.pending_count + '</div></div></div>';
html += '<div class="col"><div class="card"><div class="card-title">Approved</div><div class="card-value">' + data.approved_count + '</div></div></div>';
html += '<div class="col"><div class="card"><div class="card-title">Rejected</div><div class="card-value">' + data.rejected_count + '</div></div></div>';
html += '</div>';
html += '<div class="card"><div class="card-title">Pending Approval Requests</div>';
if (data.pending_requests && data.pending_requests.length) {
for (const req of data.pending_requests) {
html += '<div class="command-item">';
html += '<div class="name">' + (req.title || req.change_id) + '</div>';
html += '<div class="meta">ID: ' + req.change_id + ' | Risk: ' + req.risk + '</div>';
html += '<div style="font-size:13px;color:#c9d1d9;margin-top:4px;">' + (req.reason || '') + '</div>';
html += '</div>';
clearNode(el);
const row = div('row');
const cols = [['Pending Requests', data.pending_count], ['Approved', data.approved_count], ['Rejected', data.rejected_count]];
for (const [label, value] of cols) {
const col = div('col'); const card = div('card');
const t = div('card-title'); t.textContent = label;
const val = div('card-value'); val.textContent = value;
card.appendChild(t); card.appendChild(val); col.appendChild(card); row.appendChild(col);
}
el.appendChild(row);
const listCard = div('card');
const lt = div('card-title'); lt.textContent = 'Pending Approval Requests'; listCard.appendChild(lt);
const reqs = data.pending_requests || [];
if (reqs.length) {
for (const req of reqs) {
const item = div('command-item');
const name = div('name'); name.textContent = req.title || req.change_id || '';
const meta = div('meta'); meta.textContent = 'ID: ' + (req.change_id || '') + ' | Risk: ' + (req.risk || '');
const reason = div(''); reason.style.fontSize = '13px'; reason.style.color = '#c9d1d9'; reason.style.marginTop = '4px';
reason.textContent = req.reason || '';
item.appendChild(name); item.appendChild(meta); item.appendChild(reason);
listCard.appendChild(item);
}
} else {
html += '<div style="color:#8b949e;padding:12px;">No pending requests.</div>';
const none = div(''); none.style.color = '#8b949e'; none.style.padding = '12px';
none.textContent = 'No pending requests.'; listCard.appendChild(none);
}
html += '</div>';
el.innerHTML = html;
} catch(e) {
el.innerHTML = '<div class="error">Failed to load: ' + e.message + '</div>';
el.appendChild(listCard);
} catch (e) {
const err = div('error'); err.textContent = 'Failed to load: ' + e.message; el.appendChild(err);
}
}
// Real governed approve: posts the decision to the server-side governed endpoint.
async function approveDecision(changeId, decision, btn) {
if (btn) btn.disabled = true;
try {
const res = await fetch('/admin/ai-evolution/api/owner-gate/decision', {
method: 'POST',
headers: { 'Content-Type': 'application/json', 'X-CSRF-Token': csrfToken() },
body: JSON.stringify({ change_id: changeId, decision: decision })
});
const payload = await res.json();
if (!res.ok) throw new Error(payload.error || ('HTTP ' + res.status));
await loadApprove();
} catch (e) {
if (btn) btn.disabled = false;
const out = document.getElementById('arch-approve-content');
const err = div('error'); err.textContent = 'Decision failed: ' + e.message;
if (out) out.appendChild(err);
}
}
async function loadApprove() {
const el = document.getElementById('arch-approve-content');
el.innerHTML = '<div class="loading">Loading pending approvals...</div>';
clearNode(el);
const loading = div('loading'); loading.textContent = 'Loading pending approvals...'; el.appendChild(loading);
try {
const r = await fetch('/admin/ai-evolution/api/owner-gate');
const data = await r.json();
let html = '';
if (data.pending_requests && data.pending_requests.length) {
html += '<div class="card"><div class="card-title">Pending changes requiring your approval</div>';
for (const req of data.pending_requests) {
html += '<div class="command-item">';
html += '<div class="name">' + (req.title || req.change_id) + '</div>';
html += '<div class="meta">Risk: <strong>' + req.risk + '</strong></div>';
html += '<div style="font-size:13px;color:#c9d1d9;margin:8px 0;">' + (req.reason || '') + '</div>';
html += '<div style="display:flex;gap:8px;">';
html += '<button onclick="ownerAction.classList.add('\'approved\');this.disabled=true" style="padding:6px 16px;background:#238636;color:#fff;border:none;border-radius:6px;cursor:pointer;font-size:13px;">✓ Approve</button>';
html += '</div></div>';
clearNode(el);
const reqs = data.pending_requests || [];
if (reqs.length) {
const card = div('card');
const t = div('card-title'); t.textContent = 'Pending changes requiring your approval'; card.appendChild(t);
for (const req of reqs) {
const item = div('command-item');
const name = div('name'); name.textContent = req.title || req.change_id || '';
const meta = div('meta');
const strongRisk = document.createElement('strong'); strongRisk.textContent = req.risk || '';
meta.appendChild(document.createTextNode('Risk: ')); meta.appendChild(strongRisk);
const reason = div(''); reason.style.fontSize = '13px'; reason.style.color = '#c9d1d9'; reason.style.margin = '8px 0';
reason.textContent = req.reason || '';
const actions = div(''); actions.style.display = 'flex'; actions.style.gap = '8px';
const btn = document.createElement('button');
btn.textContent = '✓ Approve';
btn.style.padding = '6px 16px'; btn.style.background = '#238636'; btn.style.color = '#fff';
btn.style.border = 'none'; btn.style.borderRadius = '6px'; btn.style.cursor = 'pointer'; btn.style.fontSize = '13px';
const changeId = req.change_id;
btn.addEventListener('click', function () { approveDecision(changeId, 'approved', btn); });
actions.appendChild(btn);
item.appendChild(name); item.appendChild(meta); item.appendChild(reason); item.appendChild(actions);
card.appendChild(item);
}
html += '</div>';
el.appendChild(card);
} else {
html += '<div class="card"><div style="padding:24px;text-align:center;color:#8b949e;">✅ All changes processed. No pending approval requests.</div></div>';
const done = div('card');
const inner = div(''); inner.style.padding = '24px'; inner.style.textAlign = 'center'; inner.style.color = '#8b949e';
inner.textContent = '✅ All changes processed. No pending approval requests.';
done.appendChild(inner); el.appendChild(done);
}
el.innerHTML = html;
} catch(e) {
el.innerHTML = '<div class="error">Failed to load: ' + e.message + '</div>';
} catch (e) {
const err = div('error'); err.textContent = 'Failed to load: ' + e.message; el.appendChild(err);
}
}
function showPanel(name) {
document.querySelectorAll('.panel').forEach(p => p.style.display = 'none');
document.getElementById('panel-' + name).style.display = '';
document.querySelectorAll('.sidebar-nav li').forEach(li => li.classList.remove('active'));
document.querySelector('.sidebar-nav li:nth-child(' + ({overview:1,versions:2,commands:3,agents:4}[name] || 1) + ')').classList.add('active');
if (name === 'overview') {
document.getElementById('panel-overview').innerHTML = '<div class="main-header"><h2>System Overview</h2><div id="header-status" class="status-badge">Loading...</div></div><div id="loading" class="loading">Loading...</div><div id="error" class="error" style="display:none;"></div><div id="content"><div class="row"><div class="col"><div class="card"><div class="card-title">Current Baseline</div><div id="base" class="card-value">-</div></div></div><div class="col"><div class="card"><div class="card-title">Git Branch</div><div id="branch" class="card-value">-</div></div></div><div class="col"><div class="card"><div class="card-title">Uptime</div><div id="uptime" class="card-value">-</div></div></div></div><div class="card" style="margin-top:16px"><div class="card-title">Event Bus Status</div><pre id="bus-status">-</pre></div></div>';
fetchData();
}
document.querySelector('.sidebar-nav li:nth-child(' + ({overview:1,versions:2,commands:3,agents:4,['arch-owner']:5,['arch-approve']:6}[name] || 1) + ')').classList.add('active');
if (name === 'overview') fetchData();
const loaders = {
'arch-owner': loadOwnerGate,
'arch-approve': loadApprove,
+25 -1
View File
@@ -12,6 +12,29 @@ from .task_lifecycle import TaskLifecycle
lifecycle = TaskLifecycle()
# Role must come from an authenticated principal, never from the client body.
_ROLE_WHITELIST = {"user", "agent", "executor", "dispatcher", "notifier", "group_admin", "owner"}
_authenticator = None # callable(payload) -> {"role": ...} | None, injected by host
def set_authenticator(fn):
"""Install a server-side authenticator used to derive the actor role.
`fn` receives the raw request payload and must return either None (or a
dict without a whitelisted "role") for anonymous, or a dict containing a
server-verified "role". The client-supplied `payload["role"]` is ignored.
"""
global _authenticator
_authenticator = fn
def effective_role(payload: dict[str, Any]) -> str:
if _authenticator is not None:
principal = _authenticator(payload)
if isinstance(principal, dict) and principal.get("role") in _ROLE_WHITELIST:
return principal["role"]
return "user"
def get_task_actions(task: dict[str, Any], role: str) -> dict[str, Any]:
return {
@@ -31,7 +54,8 @@ def build_router():
@router.post("/{task_id}/actions/{action}", operation_id="executeTaskAction")
def action(task_id: str, action: str, payload: dict[str, Any]):
return execute_task_action(payload.get("task", {}), action, payload.get("role", "user"), payload.get("reason"))
# Role is derived from the authenticated principal; body role is ignored.
return execute_task_action(payload.get("task", {}), action, effective_role(payload), payload.get("reason"))
return router
+386 -26
View File
@@ -8,6 +8,7 @@ import time
import uuid
import urllib.parse
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from datetime import datetime, timezone
from typing import Any, Callable
from .delegation import DelegationEngine
@@ -23,6 +24,8 @@ from .errors import (
from .evidence import EvidenceService
from .journal import TaskJournal
from .lease import LeaseManager
from .lineage_store import TaskLineagePolicy, TaskPlaneLineageCoordinator
from .mission import MissionAuthority
from .models import (
EvidenceEnvelope,
ResultEnvelope,
@@ -33,6 +36,11 @@ from .models import (
validate_identifier,
)
from .policy import PolicyEngine
from .root_action_consumer import (
RootActionConsumer,
RootActionExecutor,
RootActionRunResult,
)
from .store import TaskPlaneStore
from .worker_registry import WorkerRegistry
@@ -53,7 +61,15 @@ def _bounded_int(value: Any, name: str, minimum: int, maximum: int) -> int:
class TaskPlaneService:
def __init__(self, store: TaskPlaneStore, policy: PolicyEngine | None = None):
def __init__(
self,
store: TaskPlaneStore,
policy: PolicyEngine | None = None,
*,
lineage_policy: TaskLineagePolicy | None = None,
mission_authority: MissionAuthority | None = None,
root_action_executor: RootActionExecutor | None = None,
):
self.store = store
self.policy = policy or PolicyEngine(
execution_enabled=False,
@@ -66,30 +82,257 @@ class TaskPlaneService:
self.workers = WorkerRegistry(store)
self.evidence = EvidenceService(store)
self.delegation = DelegationEngine(store)
if mission_authority is not None and mission_authority.store is not store:
raise ValueError("mission authority must use the TaskPlane store")
self.missions = mission_authority
effective_lineage_policy = lineage_policy
if mission_authority is not None:
effective_lineage_policy = (
lineage_policy
or TaskLineagePolicy(
principal_bindings=(), acceptance_evaluators={}
)
).with_acceptance_evaluator_resolver(
mission_authority.resolve_acceptance_evaluator
)
self.lineage = (
TaskPlaneLineageCoordinator(store, effective_lineage_policy)
if effective_lineage_policy is not None
else None
)
self.root_actions = (
RootActionConsumer(
store,
plan_builder=lambda review: mission_authority.build_root_action_plan(
review=review, lineage=self.lineage
),
rereviewer=lambda plan, receipt: mission_authority.rereview_root_action(
plan=plan,
execution_receipt=receipt,
lineage=self.lineage,
),
executor=root_action_executor,
)
if mission_authority is not None and self.lineage is not None
else None
)
self.store.probe_attestation_verifier = self.policy.probe_attestation_verifier
def create_task(self, body: dict[str, Any], idempotency_key: str | None = None) -> dict[str, Any]:
def consume_one_root_action(
self,
*,
consumer_id: str,
now: datetime | None = None,
lease_seconds: int = 60,
) -> RootActionRunResult:
if self.root_actions is None:
raise ForbiddenError("root action consumer is not configured")
return self.root_actions.run_once(
consumer_id=consumer_id,
now=now or datetime.now(tz=timezone.utc),
lease_seconds=lease_seconds,
)
def create_mission(
self,
body: dict[str, Any],
idempotency_key: str | None,
*,
actor_identity: str | None,
) -> dict[str, Any]:
if self.missions is None or self.lineage is None:
raise ForbiddenError("mission authority is not configured")
if actor_identity is None:
raise ForbiddenError("mission creation requires authenticated identity")
if set(body) != {"goal_slug", "original_acceptance"}:
raise ValidationError(
"mission body requires only goal_slug and original_acceptance"
)
self.policy.assert_allowed("create", actor_identity)
mission = self.missions.create_mission(
goal_slug=body["goal_slug"],
original_acceptance=body["original_acceptance"],
idempotency_key=idempotency_key,
authenticated_identity=actor_identity,
lineage=self.lineage,
)
return {
**mission.to_dict(),
"root_task": self.store.get_task(mission.root_task_id).to_dict(),
}
def get_mission(
self, mission_id: str, *, actor_identity: str | None
) -> dict[str, Any]:
if self.missions is None:
raise ForbiddenError("mission authority is not configured")
if actor_identity is None:
raise ForbiddenError("mission read requires authenticated identity")
self.policy.assert_allowed("read", actor_identity)
mission = self.missions.get_mission(
mission_id, authenticated_identity=actor_identity
)
return {
**mission.to_dict(),
"root_task": self.store.get_task(mission.root_task_id).to_dict(),
}
def delegate_mission(
self,
parent_task_id: str,
body: dict[str, Any],
idempotency_key: str | None,
*,
actor_identity: str | None,
) -> dict[str, Any]:
"""Admit one server-identified child under an owned canonical Mission."""
if self.missions is None or self.lineage is None:
raise ForbiddenError("mission authority is not configured")
if actor_identity is None:
raise ForbiddenError("mission delegation requires authenticated identity")
# Mission delegation is an admission operation. It deliberately does
# not enable the worker execution actions guarded by execution_enabled.
self.policy.assert_allowed("create", actor_identity)
granted = set(
self.policy.identity_capabilities.get(
actor_identity,
self.policy.identity_capabilities.get("*", set()),
)
)
if "task.delegate" not in granted:
raise ForbiddenError("action delegate not permitted")
child = self.missions.delegate_child(
parent_task_id=parent_task_id,
body=body,
idempotency_key=idempotency_key,
authenticated_identity=actor_identity,
lineage=self.lineage,
delegation=self.delegation,
)
return child.to_dict()
def attach_legacy_attachment(
self,
mission_id: str,
body: dict[str, Any],
idempotency_key: str | None,
*,
actor_identity: str | None,
) -> dict[str, Any]:
if self.missions is None:
raise ForbiddenError("mission authority is not configured")
if actor_identity is None:
raise ForbiddenError("legacy attachment requires authenticated identity")
self.policy.assert_allowed("create", actor_identity)
granted = set(
self.policy.identity_capabilities.get(
actor_identity, self.policy.identity_capabilities.get("*", set())
)
)
if "task.delegate" not in granted:
raise ForbiddenError("action delegate not permitted")
return self.missions.attach_legacy(
mission_id=mission_id,
body=body,
idempotency_key=idempotency_key,
authenticated_identity=actor_identity,
)
def get_legacy_attachment(
self,
mission_id: str,
attachment_id: str,
*,
actor_identity: str | None,
) -> dict[str, Any]:
if self.missions is None:
raise ForbiddenError("mission authority is not configured")
if actor_identity is None:
raise ForbiddenError("legacy attachment read requires authenticated identity")
self.policy.assert_allowed("read", actor_identity)
return self.missions.get_legacy_attachment(
mission_id=mission_id,
attachment_id=attachment_id,
authenticated_identity=actor_identity,
)
def create_task(
self,
body: dict[str, Any],
idempotency_key: str | None = None,
*,
actor_identity: str | None = None,
) -> dict[str, Any]:
if body.get("parent_task_id"):
raise ForbiddenError("child tasks must use the delegate endpoint")
issuer = body.get("issuer", {})
actor = issuer.get("id", "anon") if isinstance(issuer, dict) else "anon"
if actor_identity is not None:
if issuer and (
not isinstance(issuer, dict) or issuer.get("id") != actor_identity
):
raise ForbiddenError("issuer identity mismatch")
body = dict(body)
body["issuer"] = {"type": "authenticated", "id": actor_identity}
actor = actor_identity or (
issuer.get("id", "anon") if isinstance(issuer, dict) else "anon"
)
self.policy.assert_allowed("create", actor)
try:
plan = TaskPlan.from_dict(body)
except (TypeError, ValueError) as exc:
raise ValidationError("invalid task plan") from exc
task = self.store.create_task_with_event(plan, idempotency_key=idempotency_key)
if self.lineage is not None:
if actor_identity is None:
raise ForbiddenError("lineage admission requires authenticated identity")
task = self.lineage.create_root(
plan=plan,
authenticated_identity=actor_identity,
request_digest=self.lineage._request_digest(body),
idempotency_key=idempotency_key,
)
else:
task = self.store.create_task_with_event(
plan, idempotency_key=idempotency_key
)
return task.to_dict()
def _can_read_mission_task(self, task, actor: str | None) -> bool:
# Every service sharing the store must honor persisted Mission ownership,
# including a legacy launcher that has no Mission authority configured.
if self.missions is not None:
return self.missions.can_read_task(
task, authenticated_identity=actor or ""
)
with self.store._lock:
return self.store.conn.execute(
"SELECT 1 FROM task_missions WHERE root_task_id=?",
(task.root_task_id,),
).fetchone() is None
def _assert_mission_read(self, task, actor: str | None) -> None:
if not self._can_read_mission_task(task, actor):
raise ForbiddenError("mission task requires its configured owner authority")
def list_tasks(self, actor: str | None = None) -> list[dict[str, Any]]:
if actor is not None:
self.policy.assert_allowed("read", actor)
return [task.to_dict(include_task=False) for task in self.store.list_tasks()]
tasks = self.store.list_tasks()
tasks = [task for task in tasks if self._can_read_mission_task(task, actor)]
return [task.to_dict(include_task=False) for task in tasks]
def get_task(self, task_id: str, actor: str | None = None) -> dict[str, Any]:
task = self.store.get_task(task_id)
self._assert_mission_read(task, actor)
if actor is not None:
self.policy.assert_allowed("read", actor, task=self.store.get_task(task_id))
return self.store.get_task(task_id).to_dict()
self.policy.assert_allowed("read", actor, task=task)
if self.missions is not None and self.missions.manages_root(
task.root_task_id
):
self.missions.assert_task_read(
task, authenticated_identity=actor
)
return task.to_dict()
def get_worker(self, worker_id: str, actor: str | None = None) -> dict[str, Any]:
if actor is not None:
@@ -307,17 +550,51 @@ class TaskPlaneService:
)
evidence_entries.append(envelope)
previous = envelope.evidence_digest
self.store.complete_claim_with_evidence_and_event(
task_id,
claim_id,
final_status = (
status
if status
in {"COMPLETED", "FAILED_FINAL", "FAILED_RECOVERABLE", "CANCELLED"}
else "COMPLETED",
result.to_dict(),
evidence_entries,
else "COMPLETED"
)
return result.to_dict()
lineage_metadata = task.task_plan.metadata.get("aios_task_lineage")
is_lineage_root = bool(
self.lineage is not None
and isinstance(lineage_metadata, dict)
and lineage_metadata.get("relation_type") == "ROOT"
)
if is_lineage_root and final_status == "COMPLETED":
raise ForbiddenError("lineage root completion requires root acceptance review")
review_outcome = None
with self.store._transaction():
self.store.complete_claim_with_evidence_and_event(
task_id,
claim_id,
final_status,
result.to_dict(),
evidence_entries,
)
is_lineage_child = bool(
self.lineage is not None
and isinstance(lineage_metadata, dict)
and lineage_metadata.get("relation_type") != "ROOT"
)
if is_lineage_child and final_status in {"COMPLETED", "FAILED_FINAL", "CANCELLED"}:
review_outcome = self.lineage.review_proven_terminal_child(
child_task_id=task_id,
claim_id=str(claim_id),
result_digest=str(result.result_digest),
evidence_refs=[entry.evidence_id for entry in evidence_entries],
)
response = result.to_dict()
if review_outcome is not None:
response["root_review"] = {
"review_id": review_outcome.review.review_id,
"root_task_id": review_outcome.review.root_task_id,
"root_complete": review_outcome.review.root_complete,
"next_legal_action": review_outcome.review.next_legal_action.value,
"dispatch_status": review_outcome.dispatch.status,
}
return response
def fail(self, task_id: str, body: dict[str, Any]) -> dict[str, Any]:
claim_id = body.get("claim_id")
@@ -346,14 +623,38 @@ class TaskPlaneService:
)
return updated.to_dict()
def delegate(self, task_id: str, body: dict[str, Any]) -> dict[str, Any]:
def delegate(
self,
task_id: str,
body: dict[str, Any],
idempotency_key: str | None = None,
) -> dict[str, Any]:
actor = body.get("actor_identity") or "owner"
self.policy.assert_allowed("delegate", actor)
parent = self.store.get_task(task_id)
mission_managed = bool(
self.lineage is not None
and self.missions is not None
and self.missions.manages_root(parent.root_task_id)
)
if mission_managed:
plan_body = body.get("task") if "task" in body else {
key: value
for key, value in body.items()
if key != "actor_identity"
}
return self.delegate_mission(
task_id,
plan_body,
idempotency_key,
actor_identity=str(actor),
)
self.policy.assert_allowed("delegate", actor)
self.policy.assert_allowed("delegate", actor, task=parent)
if "task" in body:
plan_body = body["task"]
try:
child_plan = TaskPlan.from_dict(body["task"])
child_plan = TaskPlan.from_dict(plan_body)
except (TypeError, ValueError) as exc:
raise ValidationError("invalid child task plan") from exc
else:
@@ -364,25 +665,67 @@ class TaskPlaneService:
except (TypeError, ValueError) as exc:
raise ValidationError("invalid child task plan") from exc
child_plan.parent_task_id = task_id
child = self.delegation.delegate(task_id, child_plan, caller=actor)
if self.lineage is not None:
if not isinstance(plan_body, dict):
raise ValidationError("invalid child task plan")
if "task_id" not in plan_body:
raise ValidationError("lineage delegation requires a stable task_id")
plan_body = dict(plan_body)
issuer = plan_body.get("issuer")
if issuer and (
not isinstance(issuer, dict) or issuer.get("id") != actor
):
raise ForbiddenError("child issuer identity mismatch")
plan_body["issuer"] = {"type": "authenticated", "id": actor}
child_plan.issuer = dict(plan_body["issuer"])
child = self.lineage.delegate_child(
parent_task_id=task_id,
plan=child_plan,
authenticated_identity=str(actor),
request_digest=self.lineage._request_digest(plan_body),
delegation=self.delegation,
)
else:
child = self.delegation.delegate(task_id, child_plan, caller=actor)
return child.to_dict()
def get_events(self, task_id: str, actor: str | None = None) -> list[dict[str, Any]]:
task = self.store.get_task(task_id)
self._assert_mission_read(task, actor)
if actor is not None:
self.policy.assert_allowed("audit", actor, task=self.store.get_task(task_id))
self.store.get_task(task_id)
self.policy.assert_allowed("audit", actor, task=task)
if self.missions is not None and self.missions.manages_root(
task.root_task_id
):
self.missions.assert_task_read(
task, authenticated_identity=actor
)
return self.store.list_events(task_id)
def get_evidence(self, task_id: str, actor: str | None = None) -> list[dict[str, Any]]:
task = self.store.get_task(task_id)
self._assert_mission_read(task, actor)
if actor is not None:
self.policy.assert_allowed("audit", actor, task=self.store.get_task(task_id))
self.store.get_task(task_id)
self.policy.assert_allowed("audit", actor, task=task)
if self.missions is not None and self.missions.manages_root(
task.root_task_id
):
self.missions.assert_task_read(
task, authenticated_identity=actor
)
return self.evidence.list(task_id)
def get_children(self, task_id: str, actor: str | None = None) -> list[str]:
task = self.store.get_task(task_id)
self._assert_mission_read(task, actor)
if actor is not None:
self.policy.assert_allowed("read", actor, task=self.store.get_task(task_id))
self.store.get_task(task_id)
self.policy.assert_allowed("read", actor, task=task)
if self.missions is not None and self.missions.manages_root(
task.root_task_id
):
self.missions.assert_task_read(
task, authenticated_identity=actor
)
return self.store.list_children(task_id)
@@ -474,6 +817,10 @@ class TaskPlaneRequestHandler(BaseHTTPRequestHandler):
if len(parts) == 1 and parts[0] == "tasks":
return 200, self.service.list_tasks(identity)
if len(parts) == 2 and parts[0] == "missions":
return 200, self.service.get_mission(
parts[1], actor_identity=identity
)
if len(parts) == 2 and parts[0] == "tasks":
task_id = parts[1]
return 200, self.service.get_task(task_id, identity)
@@ -499,6 +846,13 @@ class TaskPlaneRequestHandler(BaseHTTPRequestHandler):
parts = [p for p in path.split("/") if p]
headers = {k.lower(): v for k, v in self.headers.items()}
if len(parts) == 1 and parts[0] == "missions":
return 201, self.service.create_mission(
body or {},
headers.get("idempotency-key"),
actor_identity=identity,
)
if len(parts) == 1 and parts[0] == "tasks":
issuer = body.get("issuer")
if issuer and (
@@ -506,7 +860,11 @@ class TaskPlaneRequestHandler(BaseHTTPRequestHandler):
):
raise ForbiddenError("issuer identity mismatch")
body["issuer"] = {"type": "authenticated", "id": identity}
return 201, self.service.create_task(body or {}, headers.get("idempotency-key"))
return 201, self.service.create_task(
body or {},
headers.get("idempotency-key"),
actor_identity=identity,
)
if body.get("actor_identity") not in {None, identity}:
raise ForbiddenError("actor identity mismatch")
body["actor_identity"] = identity
@@ -523,7 +881,9 @@ class TaskPlaneRequestHandler(BaseHTTPRequestHandler):
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "cancel":
return 200, self.service.cancel(parts[1], body or {})
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "delegate":
return 200, self.service.delegate(parts[1], body or {})
return 200, self.service.delegate(
parts[1], body or {}, headers.get("idempotency-key")
)
if len(parts) == 2 and parts[0] == "workers" and parts[1] == "register":
if (
@@ -0,0 +1,94 @@
"""Agent discovery/claim coordinator over diagnostic schedulability evidence."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Mapping
from .schedulability import (
DiscoveryDiagnostics,
SchedulabilityProof,
SchedulabilityService,
)
@dataclass(frozen=True)
class DiscoveryResult:
status: str
proof: SchedulabilityProof | None
blocker_codes: tuple[str, ...] = ()
candidate_count: int = 0
def to_dict(self) -> dict[str, Any]:
value: dict[str, Any] = {
"status": self.status,
"blocker_codes": list(self.blocker_codes),
"candidate_count": self.candidate_count,
"proof": None,
}
if self.proof is not None:
value["proof"] = {
"schema": self.proof.schema,
"proof_id": self.proof.proof_id,
"task_id": self.proof.task_id,
"worker_id": self.proof.worker_id,
"created_at": self.proof.created_at,
"expires_at": self.proof.expires_at,
"blockers": [b.to_dict() for b in self.proof.blockers],
"proof_digest": self.proof.proof_digest,
}
return value
def _waiting_status(diag: DiscoveryDiagnostics) -> str:
codes = set(diag.blocker_codes)
if diag.candidate_count == 0:
return "NO_CANDIDATES"
if not codes:
return "NO_CLAIMABLE_TASK"
if codes <= {"CAPACITY_EXHAUSTED"}:
return "WAITING_CAPACITY"
if "AUTHORIZATION_BLOCKED" in codes or "EXACT_SCOPE_MISMATCH" in codes:
return "WAITING_AUTHORIZATION"
if "DEPENDENCY_BLOCKED" in codes:
return "WAITING_DEPENDENCY"
if "FREEZE_BLOCKED" in codes:
return "WAITING_FREEZE"
if "CONFLICTING_LEASE" in codes:
return "WAITING_LEASE"
if {"STALE_OR_INACTIVE_WORKER", "MISSING_CAPABILITY", "INVALID_AUTHORITY_VALUE"} & codes:
return "WORKER_NOT_ELIGIBLE"
if "TASK_STATE_NOT_CLAIMABLE" in codes:
return "NO_CLAIMABLE_TASK"
return "BLOCKED"
class AutoClaimCoordinator:
"""No implicit credential derivation; authenticated identity is supplied by host."""
def __init__(self, schedulability: SchedulabilityService) -> None:
self.schedulability = schedulability
def discover(self, *, worker_id: str, actor_identity: str) -> DiscoveryResult:
proof, diag = self.schedulability.discover_with_diagnostics(
worker_id=worker_id, actor_identity=actor_identity
)
if proof is not None:
return DiscoveryResult(
"CLAIMABLE", proof, diag.blocker_codes, diag.candidate_count
)
return DiscoveryResult(
_waiting_status(diag), None, diag.blocker_codes, diag.candidate_count
)
def claim(
self,
*,
proof_id: str,
worker_id: str,
actor_identity: str,
lease_seconds: int = 30,
) -> Mapping[str, Any]:
return self.schedulability.claim_with_recheck(
proof_id=proof_id,
worker_id=worker_id,
actor_identity=actor_identity,
lease_seconds=lease_seconds,
)
File diff suppressed because it is too large Load Diff
+16 -2
View File
@@ -31,7 +31,16 @@ class DelegationEngine:
current = self.store.parent_of(current)
return True
def delegate(self, parent_id: str, child_plan: TaskPlan, *, caller: str) -> TaskRecord:
def delegate(
self,
parent_id: str,
child_plan: TaskPlan,
*,
caller: str,
owner_principal_id: str = "",
mission_id: str = "",
created_at: float | None = None,
) -> TaskRecord:
self.can_delegate(parent_id, child_plan)
parent = self.store.get_task(parent_id)
if child_plan.required_capabilities and not set(child_plan.required_capabilities).issubset(
@@ -80,7 +89,12 @@ class DelegationEngine:
raise ConflictError("delegation task_id conflict")
return self.store.delegate_task_with_budget_and_events(
parent_id, child_plan, caller=caller
parent_id,
child_plan,
caller=caller,
owner_principal_id=owner_principal_id,
mission_id=mission_id,
created_at=created_at,
)
def _depth(self, task_id: str) -> int:
@@ -0,0 +1,536 @@
"""Mission-owned, append-only bindings for verified external legacy executions."""
from __future__ import annotations
import json
import re
from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Mapping, Protocol, Sequence
from .errors import ConflictError, NotFoundError, ValidationError
from .models import compute_sha256_hex
ATTACH_REQUEST_SCHEMA = "aios.mission-legacy-attach-request.v1"
ATTACH_EVIDENCE_SCHEMA = "aios.verified-legacy-attachment-evidence.v1"
ATTACH_RECEIPT_SCHEMA = "aios.mission-legacy-attachment-receipt.v1"
ATTACHMENT_KIND = "LEGACY_EXECUTION_SUBJECT"
RELATION_TYPE = "RECOVERY"
SOURCE_KIND = "agent-task-bridge-job"
REQUIRED_READBACK_ROLES = frozenset(
{"agent_task", "admission", "owner_registry", "bridge_job"}
)
REQUIRED_C04_EVIDENCE = frozenset(
{
"verified legacy ownership and acceptance relevance",
"unchanged task/job/idempotency/reconciliation identity",
"authoritative attach readback",
}
)
_SHA256_RE = re.compile(r"^[0-9a-f]{64}$")
def _canonical(value: object) -> str:
try:
return json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
allow_nan=False,
)
except (TypeError, ValueError) as exc:
raise ValidationError("legacy attachment value must be JSON data") from exc
def _object(value: object, fields: frozenset[str], name: str) -> dict[str, object]:
if not isinstance(value, Mapping) or set(value) != fields:
raise ValidationError(f"invalid {name} fields")
return json.loads(_canonical(dict(value)))
def _identifier(value: object, name: str, *, maximum: int = 256) -> str:
if (
not isinstance(value, str)
or not value
or len(value) > maximum
or any(ord(character) < 33 for character in value)
):
raise ValidationError(f"invalid {name}")
return value
def _integer(value: object, name: str) -> int:
if type(value) is not int or value < 0:
raise ValidationError(f"invalid {name}")
return value
def _utc(value: object, name: str) -> str:
if not isinstance(value, str) or not value:
raise ValidationError(f"invalid {name}")
try:
parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError as exc:
raise ValidationError(f"invalid {name}") from exc
if parsed.tzinfo is None or parsed.utcoffset() is None:
raise ValidationError(f"invalid {name}")
return parsed.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")
@dataclass(frozen=True)
class AuthorityReadbackRef:
"""A non-secret pointer and digest issued by one independent authority."""
role: str
authority: str
readback_ref: str
canonical_sha256: str
observed_at: str
revision: int | None = None
def to_dict(self) -> dict[str, object]:
if self.role not in REQUIRED_READBACK_ROLES:
raise ValidationError("invalid legacy authority readback role")
authority = _identifier(self.authority, "readback authority")
reference = _identifier(self.readback_ref, "readback ref", maximum=1024)
if not _SHA256_RE.fullmatch(self.canonical_sha256):
raise ValidationError("invalid readback canonical SHA-256")
observed = _utc(self.observed_at, "readback observed_at")
revision = self.revision
if revision is not None:
revision = _integer(revision, "readback revision")
return {
"role": self.role,
"authority": authority,
"readback_ref": reference,
"canonical_sha256": self.canonical_sha256,
"observed_at": observed,
"revision": revision,
}
@dataclass(frozen=True)
class VerifiedLegacyAttachmentEvidence:
"""Typed output of a configured authority reader, never a caller payload."""
schema: str
source_kind: str
source_task_id: str
source_task_created_at: str
source_task_status: str
source_task_revision: int
source_task_attempt: int
source_task_max_attempts: int
external_job_id: str
internal_job_id: str
internal_idempotency_key: str
job_branch: str
job_state: str
job_revision: int
side_effect_state: str
grant_id: str
grant_digest: str
execution_digest: str
epoch_id: str
epoch_digest: str
historical_signer_id: str
historical_signer_fingerprint: str
registry_generation: int
signature_verified: bool
epoch_currently_valid: bool
readbacks: Sequence[AuthorityReadbackRef]
def to_dict(self) -> dict[str, object]:
if self.schema != ATTACH_EVIDENCE_SCHEMA or self.source_kind != SOURCE_KIND:
raise ValidationError("invalid legacy attachment evidence schema")
if self.signature_verified is not True or self.epoch_currently_valid is not False:
raise ConflictError("legacy historical Owner evidence is not verified-expired")
values = {
"source_task_id": self.source_task_id,
"source_task_status": self.source_task_status,
"external_job_id": self.external_job_id,
"internal_job_id": self.internal_job_id,
"internal_idempotency_key": self.internal_idempotency_key,
"job_branch": self.job_branch,
"job_state": self.job_state,
"side_effect_state": self.side_effect_state,
"grant_id": self.grant_id,
"historical_signer_id": self.historical_signer_id,
"historical_signer_fingerprint": self.historical_signer_fingerprint,
"epoch_id": self.epoch_id,
}
normalized = {
key: _identifier(value, key, maximum=1024)
for key, value in values.items()
}
for name, value in {
"grant_digest": self.grant_digest,
"execution_digest": self.execution_digest,
"epoch_digest": self.epoch_digest,
}.items():
if not _SHA256_RE.fullmatch(value):
raise ValidationError(f"invalid {name}")
normalized[name] = value
task_revision = _integer(self.source_task_revision, "source_task_revision")
task_attempt = _integer(self.source_task_attempt, "source_task_attempt")
task_max_attempts = _integer(
self.source_task_max_attempts, "source_task_max_attempts"
)
if task_attempt > task_max_attempts:
raise ConflictError("legacy task attempt exceeds max attempts")
job_revision = _integer(self.job_revision, "job_revision")
registry_generation = _integer(
self.registry_generation, "registry_generation"
)
readbacks = [value.to_dict() for value in self.readbacks]
roles = [str(value["role"]) for value in readbacks]
if len(roles) != len(set(roles)) or set(roles) != REQUIRED_READBACK_ROLES:
raise ValidationError("legacy evidence requires four authority readbacks")
readbacks.sort(key=lambda value: str(value["role"]))
return {
"schema": ATTACH_EVIDENCE_SCHEMA,
"source_kind": SOURCE_KIND,
**normalized,
"source_task_created_at": _utc(
self.source_task_created_at, "source_task_created_at"
),
"source_task_revision": task_revision,
"source_task_attempt": task_attempt,
"source_task_max_attempts": task_max_attempts,
"job_revision": job_revision,
"registry_generation": registry_generation,
"signature_verified": True,
"epoch_currently_valid": False,
"readbacks": readbacks,
}
class LegacyAttachmentEvidenceProvider(Protocol):
"""Configured server-side reader for the four independent authority sources."""
def read_verified(
self,
*,
mission_id: str,
root_task_id: str,
anchor_task_id: str,
source_kind: str,
source_task_id: str,
external_job_id: str,
original_acceptance_digest: str,
) -> VerifiedLegacyAttachmentEvidence: ...
def normalize_attach_request(value: object) -> dict[str, object]:
body = _object(
value,
frozenset({"schema", "anchor_task_id", "source", "expected"}),
"legacy attachment request",
)
if body["schema"] != ATTACH_REQUEST_SCHEMA:
raise ValidationError("invalid legacy attachment request schema")
source = _object(
body["source"],
frozenset({"kind", "task_id", "external_job_id"}),
"legacy attachment source",
)
if source["kind"] != SOURCE_KIND:
raise ValidationError("invalid legacy attachment source kind")
expected = _object(
body["expected"],
frozenset(
{
"root_revision",
"anchor_revision",
"agent_task_revision",
"agent_task_attempt",
"internal_job_id",
"job_revision",
"internal_idempotency_key",
}
),
"legacy attachment expected",
)
return {
"schema": ATTACH_REQUEST_SCHEMA,
"anchor_task_id": _identifier(body["anchor_task_id"], "anchor_task_id"),
"source": {
"kind": SOURCE_KIND,
"task_id": _identifier(source["task_id"], "source task_id"),
"external_job_id": _identifier(
source["external_job_id"], "external_job_id"
),
},
"expected": {
"root_revision": _integer(expected["root_revision"], "root_revision"),
"anchor_revision": _integer(
expected["anchor_revision"], "anchor_revision"
),
"agent_task_revision": _integer(
expected["agent_task_revision"], "agent_task_revision"
),
"agent_task_attempt": _integer(
expected["agent_task_attempt"], "agent_task_attempt"
),
"internal_job_id": _identifier(
expected["internal_job_id"], "internal_job_id"
),
"job_revision": _integer(expected["job_revision"], "job_revision"),
"internal_idempotency_key": _identifier(
expected["internal_idempotency_key"], "internal_idempotency_key"
),
},
}
def validate_c04_acceptance(
acceptance: Mapping[str, object], *, source_task_id: str, external_job_id: str
) -> None:
candidate = acceptance.get("legacy_candidate")
if (
not isinstance(candidate, Mapping)
or candidate.get("task_id") != source_task_id
or candidate.get("job_id") != external_job_id
):
raise ConflictError("mission legacy candidate authority mismatch")
conditions = acceptance.get("conditions")
if not isinstance(conditions, list):
raise ConflictError("mission C04 acceptance condition is unavailable")
matches = [
condition
for condition in conditions
if isinstance(condition, Mapping) and condition.get("key") == "C04"
]
if len(matches) != 1:
raise ConflictError("mission C04 acceptance condition is unavailable")
condition = matches[0]
evidence = condition.get("required_evidence")
if (
condition.get("purpose") != "LEGACY-LINEAGE-MIGRATION"
or not isinstance(evidence, list)
or not REQUIRED_C04_EVIDENCE.issubset(
value for value in evidence if isinstance(value, str)
)
):
raise ConflictError("mission C04 acceptance contract mismatch")
def validate_evidence_matches_request(
evidence: Mapping[str, object], request: Mapping[str, object]
) -> None:
source = request["source"]
expected = request["expected"]
assert isinstance(source, Mapping) and isinstance(expected, Mapping)
pairs = {
"source_kind": source["kind"],
"source_task_id": source["task_id"],
"external_job_id": source["external_job_id"],
"source_task_revision": expected["agent_task_revision"],
"source_task_attempt": expected["agent_task_attempt"],
"internal_job_id": expected["internal_job_id"],
"job_revision": expected["job_revision"],
"internal_idempotency_key": expected["internal_idempotency_key"],
}
mismatches = [name for name, value in pairs.items() if evidence.get(name) != value]
if mismatches:
raise ConflictError(
"legacy provider evidence mismatch: " + ",".join(sorted(mismatches))
)
class LegacyAttachmentLedger:
"""Append-only persistence owned by one Mission authority store."""
def __init__(self, store) -> None:
self.store = store
with store._transaction():
store.conn.execute(
"""
CREATE TABLE IF NOT EXISTS task_mission_legacy_attachments (
attachment_id TEXT PRIMARY KEY,
mission_id TEXT NOT NULL,
root_task_id TEXT NOT NULL,
anchor_task_id TEXT NOT NULL,
authenticated_identity TEXT NOT NULL,
owner_principal_id TEXT NOT NULL,
attachment_kind TEXT NOT NULL,
relation_type TEXT NOT NULL,
attachment_sequence INTEGER NOT NULL,
display_name TEXT NOT NULL,
source_kind TEXT NOT NULL,
source_task_id TEXT NOT NULL,
external_job_id TEXT NOT NULL,
internal_job_id TEXT NOT NULL,
internal_idempotency_key TEXT NOT NULL,
request_digest TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
evidence_digest TEXT NOT NULL,
receipt_digest TEXT NOT NULL,
receipt_json TEXT NOT NULL,
created_at REAL NOT NULL,
UNIQUE (authenticated_identity, idempotency_key),
UNIQUE (source_kind, source_task_id),
UNIQUE (source_kind, external_job_id),
UNIQUE (mission_id, anchor_task_id, attachment_sequence),
UNIQUE (mission_id, anchor_task_id, source_kind, source_task_id, external_job_id)
)
"""
)
store.conn.execute(
"CREATE INDEX IF NOT EXISTS ix_mission_legacy_attachments "
"ON task_mission_legacy_attachments(mission_id, anchor_task_id)"
)
store.conn.execute(
"""
CREATE TABLE IF NOT EXISTS task_mission_acceptance_child_bindings (
mission_id TEXT NOT NULL,
condition_key TEXT NOT NULL,
child_task_id TEXT NOT NULL UNIQUE,
original_acceptance_digest TEXT NOT NULL,
binding_digest TEXT NOT NULL,
created_at REAL NOT NULL,
PRIMARY KEY (mission_id, condition_key)
)
"""
)
def acceptance_child_binding(self, mission_id: str, condition_key: str):
return self.store.conn.execute(
"SELECT * FROM task_mission_acceptance_child_bindings "
"WHERE mission_id=? AND condition_key=?",
(mission_id, condition_key),
).fetchone()
def bind_acceptance_child(
self,
*,
mission_id: str,
condition_key: str,
child_task_id: str,
original_acceptance_digest: str,
binding_digest: str,
created_at: float,
) -> None:
existing = self.acceptance_child_binding(mission_id, condition_key)
if existing is not None:
if (
existing["child_task_id"] != child_task_id
or existing["original_acceptance_digest"]
!= original_acceptance_digest
or existing["binding_digest"] != binding_digest
):
raise ConflictError(
"mission acceptance condition already has a canonical child"
)
return
child_binding = self.store.conn.execute(
"SELECT mission_id,condition_key FROM "
"task_mission_acceptance_child_bindings WHERE child_task_id=?",
(child_task_id,),
).fetchone()
if child_binding is not None:
raise ConflictError("mission child already belongs to an acceptance condition")
self.store.conn.execute(
"INSERT INTO task_mission_acceptance_child_bindings "
"(mission_id,condition_key,child_task_id,original_acceptance_digest,"
"binding_digest,created_at) VALUES(?,?,?,?,?,?)",
(
mission_id,
condition_key,
child_task_id,
original_acceptance_digest,
binding_digest,
created_at,
),
)
def by_idempotency(self, identity: str, key: str):
return self.store.conn.execute(
"SELECT * FROM task_mission_legacy_attachments "
"WHERE authenticated_identity=? AND idempotency_key=?",
(identity, key),
).fetchone()
def by_source(self, source_kind: str, task_id: str, external_job_id: str):
return self.store.conn.execute(
"SELECT * FROM task_mission_legacy_attachments "
"WHERE source_kind=? AND (source_task_id=? OR external_job_id=?)",
(source_kind, task_id, external_job_id),
).fetchone()
def by_id(self, mission_id: str, attachment_id: str):
row = self.store.conn.execute(
"SELECT * FROM task_mission_legacy_attachments "
"WHERE mission_id=? AND attachment_id=?",
(mission_id, attachment_id),
).fetchone()
if row is None:
raise NotFoundError("legacy attachment not found")
return row
def next_sequence(self, mission_id: str, anchor_task_id: str) -> int:
row = self.store.conn.execute(
"SELECT COALESCE(MAX(attachment_sequence),0)+1 "
"FROM task_mission_legacy_attachments "
"WHERE mission_id=? AND anchor_task_id=?",
(mission_id, anchor_task_id),
).fetchone()
sequence = int(row[0])
if not 1 <= sequence <= 99:
raise ConflictError("legacy attachment sequence exhausted")
return sequence
def insert(
self,
*,
receipt: Mapping[str, object],
authenticated_identity: str,
idempotency_key: str,
request_digest: str,
evidence_digest: str,
created_at: float,
) -> None:
self.store.conn.execute(
"INSERT INTO task_mission_legacy_attachments "
"(attachment_id,mission_id,root_task_id,anchor_task_id,"
"authenticated_identity,owner_principal_id,attachment_kind,"
"relation_type,attachment_sequence,display_name,source_kind,"
"source_task_id,external_job_id,internal_job_id,"
"internal_idempotency_key,request_digest,idempotency_key,"
"evidence_digest,receipt_digest,receipt_json,created_at) "
"VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
(
receipt["attachment_id"], receipt["mission_id"],
receipt["root_task_id"], receipt["anchor_task_id"],
authenticated_identity, receipt["owner_principal_id"],
receipt["attachment_kind"], receipt["relation_type"],
receipt["attachment_sequence"], receipt["display_name"],
receipt["source"]["source_kind"],
receipt["source"]["source_task_id"],
receipt["source"]["external_job_id"],
receipt["source"]["internal_job_id"],
receipt["source"]["internal_idempotency_key"],
request_digest, idempotency_key, evidence_digest,
receipt["receipt_digest"], _canonical(receipt), created_at,
),
)
@staticmethod
def receipt(row) -> dict[str, object]:
try:
value = json.loads(row["receipt_json"])
except (TypeError, ValueError, json.JSONDecodeError) as exc:
raise ConflictError("persisted legacy attachment receipt is invalid") from exc
if (
not isinstance(value, dict)
or value.get("schema") != ATTACH_RECEIPT_SCHEMA
or value.get("receipt_digest") != row["receipt_digest"]
):
raise ConflictError("persisted legacy attachment receipt is invalid")
digest_payload = dict(value)
digest_payload.pop("receipt_digest", None)
digest_payload.pop("idempotent", None)
if compute_sha256_hex(digest_payload) != row["receipt_digest"]:
raise ConflictError("persisted legacy attachment receipt digest mismatch")
return value
@@ -0,0 +1,846 @@
"""Actual TaskPlane binding for task identity, lineage and root review.
All writes use the existing :class:`TaskPlaneStore` connection and transaction
fence. Request metadata is never used as an authorization source.
"""
from __future__ import annotations
import json
from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Callable, Mapping, Sequence
if __package__ and "." in __package__: # app.task_plane package import
from ..aios_task_lineage import (
AcceptanceCondition,
AcceptanceEvaluator,
NextLegalAction,
RelationType,
RootAcceptanceSnapshot,
RootReviewRecord,
TaskRecord as LineageTaskRecord,
lineage_from_metadata,
normalize_goal_slug,
root_review_from_record,
)
from ..aios_task_lineage_integration import (
DISPLAY_NAME_METADATA_KEY,
LINEAGE_METADATA_KEY,
LINEAGE_REQUEST_DIGEST_KEY,
NewTaskIdentityRequest,
PERSISTED_DISPATCH_STATUSES,
RootActionDispatchReceipt,
TaskAuthoritySnapshot,
TaskIdentityLineageService,
TerminalReviewOutcome,
_as_task_record,
)
from ..aios_task_lineage_taskplane_adapter import (
AuthenticatedTaskContext,
InProcessTaskPlaneLineageAdapter,
TaskPlaneServiceBindings,
)
from ..aios_task_lineage_hooks import TaskLineageLifecycleHooks
else: # top-level task_plane import used by direct-module test harnesses
from aios_task_lineage import (
AcceptanceCondition,
AcceptanceEvaluator,
NextLegalAction,
RelationType,
RootAcceptanceSnapshot,
RootReviewRecord,
TaskRecord as LineageTaskRecord,
lineage_from_metadata,
normalize_goal_slug,
root_review_from_record,
)
from aios_task_lineage_integration import (
DISPLAY_NAME_METADATA_KEY,
LINEAGE_METADATA_KEY,
LINEAGE_REQUEST_DIGEST_KEY,
NewTaskIdentityRequest,
PERSISTED_DISPATCH_STATUSES,
RootActionDispatchReceipt,
TaskAuthoritySnapshot,
TaskIdentityLineageService,
TerminalReviewOutcome,
_as_task_record,
)
from aios_task_lineage_taskplane_adapter import (
AuthenticatedTaskContext,
InProcessTaskPlaneLineageAdapter,
TaskPlaneServiceBindings,
)
from aios_task_lineage_hooks import TaskLineageLifecycleHooks
from .delegation import DelegationEngine
from .errors import ConflictError, ForbiddenError, NotFoundError, ValidationError
from .models import TaskPlan, TaskRecord, canonical_json, compute_sha256_hex
from .store import TaskPlaneStore
RESERVED_LINEAGE_METADATA_KEYS = frozenset(
{LINEAGE_METADATA_KEY, DISPLAY_NAME_METADATA_KEY, LINEAGE_REQUEST_DIGEST_KEY}
)
PROVEN_TERMINAL_STATES = frozenset({"COMPLETED", "FAILED_FINAL", "CANCELLED"})
@dataclass(frozen=True)
class PrincipalLineageBinding:
"""Server-owned projection of one authenticated service identity."""
authenticated_identity: str
owner_principal_id: str
mission_id: str
preferred_owner_alias: str | None
root_acceptance_ref: str
def __post_init__(self) -> None:
for name in (
"authenticated_identity",
"owner_principal_id",
"mission_id",
"root_acceptance_ref",
):
if not str(getattr(self, name)).strip():
raise ValueError(f"{name} is required")
class TaskLineagePolicy:
"""Immutable service policy for authenticated identity and acceptance."""
def __init__(
self,
*,
principal_bindings: Sequence[PrincipalLineageBinding],
acceptance_evaluators: Mapping[str, AcceptanceEvaluator],
acceptance_evaluator_resolver: Callable[
[str], AcceptanceEvaluator | None
]
| None = None,
) -> None:
self._bindings: dict[str, PrincipalLineageBinding] = {}
for binding in principal_bindings:
if binding.authenticated_identity in self._bindings:
raise ValueError("duplicate authenticated lineage identity")
self._bindings[binding.authenticated_identity] = binding
self._evaluators = dict(acceptance_evaluators)
self._acceptance_evaluator_resolver = acceptance_evaluator_resolver
def with_acceptance_evaluator_resolver(
self, resolver: Callable[[str], AcceptanceEvaluator | None]
) -> "TaskLineagePolicy":
previous = self._acceptance_evaluator_resolver
def combined(reference: str) -> AcceptanceEvaluator | None:
resolved = resolver(reference)
if resolved is not None:
return resolved
return previous(reference) if previous is not None else None
return TaskLineagePolicy(
principal_bindings=tuple(self._bindings.values()),
acceptance_evaluators=self._evaluators,
acceptance_evaluator_resolver=combined,
)
def resolve_principal(self, authenticated_identity: str) -> PrincipalLineageBinding:
try:
return self._bindings[authenticated_identity]
except KeyError as exc:
raise ForbiddenError("authenticated identity has no lineage policy binding") from exc
def _acceptance_evaluator(self, reference: str) -> AcceptanceEvaluator | None:
evaluator = self._evaluators.get(reference)
if evaluator is None and self._acceptance_evaluator_resolver is not None:
evaluator = self._acceptance_evaluator_resolver(reference)
return evaluator
def assert_acceptance_ref_registered(self, reference: str) -> None:
if self._acceptance_evaluator(reference) is None:
raise ForbiddenError("root acceptance policy is not registered")
def evaluate_acceptance(
self,
root: LineageTaskRecord,
descendants: Sequence[LineageTaskRecord],
) -> RootAcceptanceSnapshot:
reference = root.lineage.root_acceptance_ref
evaluator = self._acceptance_evaluator(reference)
if evaluator is None:
raise ForbiddenError("root acceptance policy is not registered")
snapshot = evaluator(root, descendants)
if not isinstance(snapshot, RootAcceptanceSnapshot) or not snapshot.conditions:
return RootAcceptanceSnapshot(
(
AcceptanceCondition(
key="root-acceptance-proof-unavailable",
satisfied=False,
owner_boundary=True,
),
)
)
normalized: list[AcceptanceCondition] = []
for condition in snapshot.conditions:
if condition.satisfied and not condition.evidence_refs:
normalized.append(
AcceptanceCondition(
key=condition.key,
satisfied=False,
owner_boundary=True,
)
)
else:
normalized.append(condition)
return RootAcceptanceSnapshot(tuple(normalized))
class TaskPlaneLineageStoreBinding:
"""Official in-process read/write surface over the existing TaskPlane DB."""
def __init__(self, store: TaskPlaneStore) -> None:
self.store = store
def service_bindings(self) -> TaskPlaneServiceBindings:
return TaskPlaneServiceBindings(
get_task_snapshot=self.get_task_snapshot,
list_descendant_snapshots=self.list_descendant_snapshots,
reserve_sibling_sequence=self.reserve_sibling_sequence,
compare_and_swap_task_metadata=self.compare_and_swap_task_metadata,
get_root_review_record=self.get_root_review_record,
compare_and_swap_root_review_record=self.compare_and_swap_root_review_record,
get_root_action_dispatch=self.get_root_action_dispatch,
dispatch_root_action=self.dispatch_root_action,
)
def _terminal_proof(self, task: TaskRecord) -> Mapping[str, object] | None:
row = self.store.conn.execute(
"SELECT * FROM task_terminal_proofs WHERE task_id=?", (task.task_id,)
).fetchone()
if row is None or task.status not in PROVEN_TERMINAL_STATES:
return None
claim = self.store.conn.execute(
"SELECT status,result_payload FROM claims WHERE claim_id=? AND task_id=?",
(row["claim_id"], task.task_id),
).fetchone()
if claim is None or claim["status"] != "COMPLETED" or not claim["result_payload"]:
return None
try:
result = json.loads(claim["result_payload"])
refs = tuple(json.loads(row["evidence_refs_json"]))
except (TypeError, ValueError, json.JSONDecodeError):
return None
if result.get("result_digest") != row["result_digest"]:
return None
if task.revision < int(row["task_revision"]):
return None
evidence_ids = {entry["evidence_id"] for entry in self.store.list_evidence(task.task_id)}
if not set(refs).issubset(evidence_ids):
return None
if not self.store.verify_event_chain(task.task_id):
return None
terminal_event = any(
event["type"] in {"TASK_COMPLETED", "TASK_FAILED"}
and event["data"].get("claim_id") == row["claim_id"]
for event in self.store.list_events(task.task_id)
)
if not terminal_event or not self.store.verify_evidence_integrity(task.task_id):
return None
return dict(row)
def get_task_snapshot(self, task_id: str) -> TaskAuthoritySnapshot:
task = self.store.get_task(task_id)
if not task.owner_principal_id or not task.mission_id:
raise ForbiddenError("task lacks server-owned lineage authority binding")
proof = self._terminal_proof(task)
return TaskAuthoritySnapshot(
task_id=task.task_id,
revision=task.revision,
status=task.status,
owner_principal_id=task.owner_principal_id,
mission_id=task.mission_id,
created_at=datetime.fromtimestamp(task.created_at, tz=timezone.utc),
metadata=dict(task.task_plan.metadata),
evidence_refs=(
tuple(json.loads(proof["evidence_refs_json"])) if proof else ()
),
terminal_evidence_proven=proof is not None,
)
def list_descendant_snapshots(
self, root_task_id: str
) -> tuple[TaskAuthoritySnapshot, ...]:
with self.store._lock:
rows = self.store.conn.execute(
"SELECT task_id FROM tasks WHERE root_task_id=? AND task_id<>? "
"ORDER BY created_at,task_id",
(root_task_id, root_task_id),
).fetchall()
return tuple(self.get_task_snapshot(row["task_id"]) for row in rows)
def reserve_sibling_sequence(
self, parent_task_id: str, relation_type: RelationType, child_task_id: str
) -> int:
with self.store._transaction():
existing = self.store.conn.execute(
"SELECT parent_task_id,relation_type,sibling_sequence "
"FROM task_lineage_sequences WHERE child_task_id=?",
(child_task_id,),
).fetchone()
if existing is not None:
if (
existing["parent_task_id"] != parent_task_id
or existing["relation_type"] != relation_type.value
):
raise ConflictError("child sibling reservation conflicts with lineage")
return int(existing["sibling_sequence"])
row = self.store.conn.execute(
"SELECT COALESCE(MAX(sibling_sequence),0)+1 AS next_sequence "
"FROM task_lineage_sequences WHERE parent_task_id=? AND relation_type=?",
(parent_task_id, relation_type.value),
).fetchone()
sequence = int(row["next_sequence"])
if sequence > 99:
raise ConflictError("lineage sibling sequence exhausted")
self.store.conn.execute(
"INSERT INTO task_lineage_sequences "
"(parent_task_id,relation_type,child_task_id,sibling_sequence,created_at) "
"VALUES(?,?,?,?,?)",
(parent_task_id, relation_type.value, child_task_id, sequence, datetime.now(tz=timezone.utc).timestamp()),
)
return sequence
def compare_and_swap_task_metadata(
self, task_id: str, expected_revision: int, metadata_patch: Mapping[str, object]
) -> bool:
if not set(metadata_patch).issubset(RESERVED_LINEAGE_METADATA_KEYS):
raise ForbiddenError("lineage CAS contains non-lineage metadata")
with self.store._transaction():
task = self.store.get_task(task_id)
if task.revision != expected_revision:
return False
lineage_value = metadata_patch.get(LINEAGE_METADATA_KEY)
if isinstance(lineage_value, Mapping):
lineage = lineage_from_metadata(lineage_value)
if lineage.relation_type != RelationType.ROOT:
# The root check and child metadata CAS share this BEGIN
# IMMEDIATE transaction, closing the legacy attach race.
self.store.assert_root_open_for_child(lineage.root_task_id)
if task.status in PROVEN_TERMINAL_STATES:
raise ConflictError("terminal task lineage metadata is immutable")
payload = task.task_plan.to_dict(redact=False)
metadata = dict(payload.get("metadata") or {})
for key, value in metadata_patch.items():
if key in metadata and metadata[key] != value:
raise ConflictError("authoritative lineage metadata conflict")
metadata[key] = value
payload["metadata"] = metadata
now = datetime.now(tz=timezone.utc).timestamp()
updated = self.store.conn.execute(
"UPDATE tasks SET task_json=?,revision=revision+1,updated_at=? "
"WHERE task_id=? AND revision=?",
(json.dumps(payload, ensure_ascii=False), now, task_id, expected_revision),
)
if updated.rowcount != 1:
return False
self.store.append_event(
task_id,
"TASK_LINEAGE_BOUND",
{"metadata_keys": sorted(metadata_patch)},
subject=f"task/{task_id}",
)
return True
def record_terminal_proof(
self,
*,
task_id: str,
claim_id: str,
result_digest: str,
evidence_refs: Sequence[str],
) -> None:
with self.store._transaction():
task = self.store.get_task(task_id)
if task.status not in PROVEN_TERMINAL_STATES:
raise ConflictError("terminal proof requires a final task status")
claim = self.store.conn.execute(
"SELECT status,result_payload FROM claims WHERE claim_id=? AND task_id=?",
(claim_id, task_id),
).fetchone()
if claim is None or claim["status"] != "COMPLETED" or not claim["result_payload"]:
raise ConflictError("terminal proof requires a completed claim")
stored_result = json.loads(claim["result_payload"])
if stored_result.get("result_digest") != result_digest:
raise ConflictError("terminal proof result digest mismatch")
refs = tuple(str(value) for value in evidence_refs)
evidence_ids = {entry["evidence_id"] for entry in self.store.list_evidence(task_id)}
if not set(refs).issubset(evidence_ids):
raise ConflictError("terminal proof evidence reference mismatch")
existing = self.store.conn.execute(
"SELECT * FROM task_terminal_proofs WHERE task_id=?", (task_id,)
).fetchone()
expected = (claim_id, result_digest, canonical_json(list(refs)))
if existing is not None:
actual = (
existing["claim_id"],
existing["result_digest"],
canonical_json(json.loads(existing["evidence_refs_json"])),
)
if actual != expected:
raise ConflictError("terminal proof replay conflict")
return
if not self.store.verify_event_chain(task_id) or not self.store.verify_evidence_integrity(task_id):
raise ConflictError("terminal proof journal integrity failed")
self.store.conn.execute(
"INSERT INTO task_terminal_proofs "
"(task_id,claim_id,task_revision,result_digest,evidence_refs_json,created_at) "
"VALUES(?,?,?,?,?,?)",
(
task_id,
claim_id,
task.revision,
result_digest,
json.dumps(list(refs), ensure_ascii=False),
datetime.now(tz=timezone.utc).timestamp(),
),
)
def get_root_review_record(self, review_id: str) -> Mapping[str, object] | None:
row = self.store.conn.execute(
"SELECT record_json FROM task_root_reviews WHERE review_id=?", (review_id,)
).fetchone()
return None if row is None else json.loads(row["record_json"])
def get_root_review_for_trigger(
self, trigger_child_task_id: str
) -> Mapping[str, object] | None:
row = self.store.conn.execute(
"SELECT record_json FROM task_root_reviews "
"WHERE trigger_child_task_id=? ORDER BY created_at LIMIT 1",
(trigger_child_task_id,),
).fetchone()
return None if row is None else json.loads(row["record_json"])
def compare_and_swap_root_review_record(
self,
root_task_id: str,
expected_root_revision: int,
review_record: Mapping[str, object],
) -> bool:
with self.store._transaction():
review_id = str(review_record.get("review_id") or "")
trigger_id = str(review_record.get("trigger_child_task_id") or "")
if (
review_record.get("root_task_id") != root_task_id
or int(review_record.get("root_revision_before", -1)) != expected_root_revision
or int(review_record.get("root_revision_after", -1)) != expected_root_revision + 1
):
raise ConflictError("root review authority binding mismatch")
existing = self.get_root_review_record(review_id)
if existing is not None:
if canonical_json(existing) != canonical_json(dict(review_record)):
raise ConflictError("root review replay conflict")
return True
trigger = self.get_task_snapshot(trigger_id)
if not trigger.terminal_evidence_proven:
raise ConflictError("root review trigger is not proven terminal")
trigger_revision = trigger.revision
frozen = self.store.conn.execute(
"SELECT review_id FROM task_root_reviews "
"WHERE trigger_child_task_id=? AND trigger_child_revision=?",
(trigger_id, trigger_revision),
).fetchone()
if frozen is not None:
raise ConflictError("terminal trigger already has a different root review")
root = self.store.get_task(root_task_id)
if root.revision != expected_root_revision:
return False
now = datetime.now(tz=timezone.utc).timestamp()
self.store.conn.execute(
"INSERT INTO task_root_reviews "
"(review_id,root_task_id,trigger_child_task_id,trigger_child_revision,"
"root_revision_before,root_revision_after,record_json,created_at) "
"VALUES(?,?,?,?,?,?,?,?)",
(
review_id,
root_task_id,
trigger_id,
trigger_revision,
expected_root_revision,
expected_root_revision + 1,
json.dumps(dict(review_record), ensure_ascii=False, sort_keys=True),
now,
),
)
updated = self.store.conn.execute(
"UPDATE tasks SET revision=revision+1,updated_at=? "
"WHERE task_id=? AND revision=?",
(now, root_task_id, expected_root_revision),
)
if updated.rowcount != 1:
raise ConflictError("root review CAS lost after reservation")
self.store.append_event(
root_task_id,
"ROOT_REVIEW",
{
"review_id": review_id,
"trigger_child_task_id": trigger_id,
"next_legal_action": review_record.get("next_legal_action"),
"root_complete": bool(review_record.get("root_complete")),
},
subject=f"task/{root_task_id}",
)
return True
def get_root_action_dispatch(self, review_id: str) -> RootActionDispatchReceipt | None:
row = self.store.conn.execute(
"SELECT * FROM task_root_action_dispatches WHERE review_id=?", (review_id,)
).fetchone()
if row is None:
return None
return RootActionDispatchReceipt(
review_id=row["review_id"],
root_task_id=row["root_task_id"],
action=NextLegalAction(row["action"]),
idempotency_key=row["idempotency_key"],
status=row["status"],
mission_id=row["mission_id"],
root_revision=int(row["root_revision"]),
plan_digest=row["plan_digest"],
execution_key=row["execution_key"],
)
def dispatch_root_action(
self, review: RootReviewRecord, idempotency_key: str
) -> RootActionDispatchReceipt:
with self.store._transaction():
persisted = self.get_root_review_record(review.review_id)
if persisted is None:
raise ConflictError("root action requires a persisted root review")
existing = self.get_root_action_dispatch(review.review_id)
if existing is not None:
if (
existing.root_task_id != review.root_task_id
or existing.action != review.next_legal_action
or existing.idempotency_key != idempotency_key
):
raise ConflictError("root action dispatch replay conflict")
return existing
now = datetime.now(tz=timezone.utc).timestamp()
self.store.conn.execute(
"INSERT INTO task_root_action_dispatches "
"(review_id,root_task_id,action,idempotency_key,status,created_at,"
"mission_id,root_revision,updated_at) VALUES(?,?,?,?,?,?,?,?,?)",
(
review.review_id,
review.root_task_id,
review.next_legal_action.value,
idempotency_key,
"PENDING",
now,
self.store.get_task(review.root_task_id).mission_id,
review.root_revision_after,
now,
),
)
self.store.append_event(
review.root_task_id,
"ROOT_NEXT_ACTION_PENDING",
{
"review_id": review.review_id,
"action": review.next_legal_action.value,
"idempotency_key": idempotency_key,
},
subject=f"task/{review.root_task_id}",
)
return self.get_root_action_dispatch(review.review_id) # type: ignore[return-value]
class TaskPlaneLineageCoordinator:
"""Coordinates lineage with real TaskPlane admission and completion paths."""
def __init__(self, store: TaskPlaneStore, policy: TaskLineagePolicy) -> None:
self.store = store
self.policy = policy
self.binding = TaskPlaneLineageStoreBinding(store)
self.adapter = InProcessTaskPlaneLineageAdapter(
self.binding.service_bindings(), readback=self.binding
)
self.service = TaskIdentityLineageService(self.adapter)
self.hooks = TaskLineageLifecycleHooks(self.adapter)
@staticmethod
def _request_digest(payload: Mapping[str, object]) -> str:
return compute_sha256_hex(dict(payload))
@staticmethod
def _reject_reserved_metadata(plan: TaskPlan) -> None:
if set(plan.metadata).intersection(RESERVED_LINEAGE_METADATA_KEYS):
raise ValidationError("lineage metadata is server-owned")
@staticmethod
def _goal_slug(plan: TaskPlan) -> str:
for candidate in (plan.intent, plan.task_type, f"TASK-{plan.task_id}"):
if not candidate:
continue
try:
return normalize_goal_slug(str(candidate))
except ValueError:
continue
raise ValidationError("task goal cannot be normalized for lineage")
def _existing_by_idempotency(self, key: str | None) -> TaskRecord | None:
if not key:
return None
row = self.store.conn.execute(
"SELECT task_id FROM tasks WHERE idempotency_key=?", (key,)
).fetchone()
return None if row is None else self.store.get_task(row["task_id"])
@staticmethod
def _verify_replay(
task: TaskRecord,
*,
request_digest: str,
principal: PrincipalLineageBinding,
parent_task_id: str | None,
) -> TaskRecord:
metadata = task.task_plan.metadata
if (
metadata.get(LINEAGE_REQUEST_DIGEST_KEY) != request_digest
or task.owner_principal_id != principal.owner_principal_id
or task.mission_id != principal.mission_id
or task.task_plan.parent_task_id != parent_task_id
):
raise ConflictError("lineage admission replay conflict")
return task
def create_root(
self,
*,
plan: TaskPlan,
authenticated_identity: str,
request_digest: str,
idempotency_key: str | None,
) -> TaskRecord:
principal = self.policy.resolve_principal(authenticated_identity)
return self.create_root_for_principal(
plan=plan,
principal=principal,
request_digest=request_digest,
idempotency_key=idempotency_key,
)
def create_root_for_principal(
self,
*,
plan: TaskPlan,
principal: PrincipalLineageBinding,
request_digest: str,
idempotency_key: str | None,
created_at: datetime | None = None,
) -> TaskRecord:
self.policy.assert_acceptance_ref_registered(principal.root_acceptance_ref)
self._reject_reserved_metadata(plan)
with self.store._transaction():
existing = self._existing_by_idempotency(idempotency_key)
if existing is not None:
return self._verify_replay(
existing,
request_digest=request_digest,
principal=principal,
parent_task_id=None,
)
created_at = created_at or datetime.now(tz=timezone.utc)
if created_at.tzinfo is None or created_at.utcoffset() is None:
raise ValidationError("lineage created_at must be timezone-aware")
created_at = created_at.astimezone(timezone.utc)
prepared = self.service.prepare_new_task_identity(
NewTaskIdentityRequest(
task_id=str(plan.task_id),
owner_principal_id=principal.owner_principal_id,
preferred_owner_alias=principal.preferred_owner_alias,
created_at=created_at,
mission_id=principal.mission_id,
goal_slug=self._goal_slug(plan),
root_acceptance_ref=principal.root_acceptance_ref,
relation_type=RelationType.ROOT,
)
)
plan.metadata = {
**plan.metadata,
**prepared.metadata_patch,
LINEAGE_REQUEST_DIGEST_KEY: request_digest,
}
return self.store.create_task_with_event(
plan,
idempotency_key=idempotency_key,
owner_principal_id=principal.owner_principal_id,
mission_id=principal.mission_id,
created_at=created_at.timestamp(),
)
def delegate_child(
self,
*,
parent_task_id: str,
plan: TaskPlan,
authenticated_identity: str,
request_digest: str,
delegation: DelegationEngine,
principal: PrincipalLineageBinding | None = None,
) -> TaskRecord:
if not plan.task_id:
raise ValidationError("lineage delegation requires a stable task_id")
principal = principal or self.policy.resolve_principal(authenticated_identity)
if principal.authenticated_identity != authenticated_identity:
raise ForbiddenError("delegation principal identity mismatch")
self._reject_reserved_metadata(plan)
with self.store._transaction():
parent = self.store.get_task(parent_task_id)
root = self.store.get_task(parent.root_task_id)
if (
root.owner_principal_id != principal.owner_principal_id
or root.mission_id != principal.mission_id
):
raise ForbiddenError("delegation actor does not match root authority")
try:
existing = self.store.get_task(str(plan.task_id))
except NotFoundError:
existing = None
if existing is not None:
return self._verify_replay(
existing,
request_digest=request_digest,
principal=principal,
parent_task_id=parent_task_id,
)
self.store.assert_root_open_for_child(root.task_id)
created_at = datetime.now(tz=timezone.utc)
prepared = self.service.prepare_new_task_identity(
NewTaskIdentityRequest(
task_id=str(plan.task_id),
owner_principal_id=root.owner_principal_id,
preferred_owner_alias=principal.preferred_owner_alias,
created_at=created_at,
mission_id=root.mission_id,
goal_slug=self._goal_slug(plan),
root_acceptance_ref=str(
root.task_plan.metadata[LINEAGE_METADATA_KEY]["root_acceptance_ref"]
),
relation_type=RelationType.CHILD,
parent_task_id=parent_task_id,
)
)
plan.metadata = {
**plan.metadata,
**prepared.metadata_patch,
LINEAGE_REQUEST_DIGEST_KEY: request_digest,
}
return delegation.delegate(
parent_task_id,
plan,
caller=authenticated_identity,
owner_principal_id=root.owner_principal_id,
mission_id=root.mission_id,
created_at=created_at.timestamp(),
)
def current_acceptance_snapshot(self, *, root_task_id: str) -> RootAcceptanceSnapshot:
"""Evaluate the current authoritative root/descendant projection."""
root = _as_task_record(self.binding.get_task_snapshot(root_task_id))
descendants = tuple(
_as_task_record(snapshot)
for snapshot in self.binding.list_descendant_snapshots(root_task_id)
)
return self.policy.evaluate_acceptance(root, descendants)
def review_proven_terminal_child(
self,
*,
child_task_id: str,
claim_id: str,
result_digest: str,
evidence_refs: Sequence[str],
):
with self.store._transaction():
self.binding.record_terminal_proof(
task_id=child_task_id,
claim_id=claim_id,
result_digest=result_digest,
evidence_refs=evidence_refs,
)
return self.resume_terminal_review(child_task_id=child_task_id)
def resume_terminal_review(self, *, child_task_id: str) -> TerminalReviewOutcome:
"""Resume after restart without re-completing a claim or re-evaluating a review.
Once a trigger has a persisted review, that exact decision is frozen.
Policy/evaluator drift therefore cannot create a second decision for the
same terminal transition.
"""
with self.store._transaction():
child = self.binding.get_task_snapshot(child_task_id)
if not child.terminal_evidence_proven:
raise ConflictError("terminal review resume requires durable proof")
raw_lineage = child.metadata.get(LINEAGE_METADATA_KEY)
if not isinstance(raw_lineage, Mapping):
raise ConflictError("lineage child lacks authoritative lineage metadata")
try:
child_lineage = lineage_from_metadata(dict(raw_lineage))
except ValueError as exc:
raise ConflictError("lineage child metadata is invalid") from exc
persisted = self.binding.get_root_review_for_trigger(child_task_id)
if persisted is not None:
review_id = persisted.get("review_id")
if not isinstance(review_id, str) or not review_id:
raise ConflictError("persisted root review identity is invalid")
try:
review = root_review_from_record(
persisted, expected_review_id=review_id
)
except ValueError as exc:
raise ConflictError("persisted root review is invalid") from exc
root = self.binding.get_task_snapshot(child_lineage.root_task_id)
if (
review.trigger_child_task_id != child_task_id
or review.root_task_id != child_lineage.root_task_id
or child_lineage.task_id != child_task_id
or child_lineage.owner_principal_id != child.owner_principal_id
or child_lineage.mission_id != child.mission_id
or root.owner_principal_id != child.owner_principal_id
or root.mission_id != child.mission_id
or root.revision < review.root_revision_after
):
raise ConflictError("persisted root review authority mismatch")
dispatch = self.binding.get_root_action_dispatch(review.review_id)
if dispatch is None:
dispatch = self.binding.dispatch_root_action(
review,
f"root-review:{review.review_id}:{review.next_legal_action.value}",
)
expected_key = (
f"root-review:{review.review_id}:{review.next_legal_action.value}"
)
if (
dispatch.review_id != review.review_id
or dispatch.root_task_id != review.root_task_id
or dispatch.action != review.next_legal_action
or dispatch.idempotency_key != expected_key
or dispatch.status not in PERSISTED_DISPATCH_STATUSES
):
raise ConflictError("persisted root action dispatch is invalid")
return TerminalReviewOutcome(review=review, dispatch=dispatch)
hook_outcome = self.hooks.on_proven_terminal_child(
child_task_id=child_task_id,
context=AuthenticatedTaskContext(
principal_id=child.owner_principal_id,
preferred_alias=child_lineage.owner_alias,
mission_id=child.mission_id,
),
acceptance_evaluator=self.policy.evaluate_acceptance,
reviewed_at=datetime.now(tz=timezone.utc),
require_independent_readback=False,
)
return hook_outcome.outcome
File diff suppressed because it is too large Load Diff
+7
View File
@@ -388,6 +388,8 @@ class TaskRecord:
root_task_id: str
idempotency_key: str | None = None
failure_reason: str | None = None
owner_principal_id: str = ""
mission_id: str = ""
def with_status(self, status: str) -> "TaskRecord":
allowed = TASK_NEXT_STATES.get(self.status, set())
@@ -407,6 +409,8 @@ class TaskRecord:
root_task_id=self.root_task_id,
idempotency_key=self.idempotency_key,
failure_reason=self.failure_reason,
owner_principal_id=self.owner_principal_id,
mission_id=self.mission_id,
)
def to_dict(self, *, include_task=True, redact_payload: bool = True) -> dict[str, Any]:
@@ -422,6 +426,9 @@ class TaskRecord:
}
if include_task:
payload["plan"] = self.task_plan.to_dict(redact=redact_payload)
display_name = self.task_plan.metadata.get("aios_task_display_name")
if self.owner_principal_id and self.mission_id and isinstance(display_name, str):
payload["display_name"] = display_name
if self.idempotency_key:
payload["idempotency_key"] = self.idempotency_key
return payload
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,513 @@
"""Diagnostic schedulability proof and claim-time authoritative recheck.
The proof is evidence, never a bearer credential.
A claim proceeds in three stages:
1. read-only preflight/discovery;
2. claim-time read-only recheck;
3. final *atomic* authority operation that revalidates all mutable claim
conditions at the linearization point and returns lease/fencing receipt.
This module does not create a second scheduler or a second TaskPlane authority.
The host adapter must bind ``claim_atomically`` to the existing authoritative
claim transaction / repository operation.
"""
from __future__ import annotations
from dataclasses import dataclass, asdict
from typing import Any, Mapping, Protocol, Sequence
import hashlib
import time
import uuid
from .models import canonical_json
CLAIMABLE_TASK_STATES = frozenset({"CREATED", "QUEUED", "FAILED_RECOVERABLE"})
class SchedulabilityError(RuntimeError):
pass
class SchedulabilityChanged(SchedulabilityError):
pass
def _canonical(value: object) -> bytes:
# Reuse the existing TaskPlane canonical JSON contract. Do not introduce
# another sort/separator encoder for schedulability evidence.
return canonical_json(value).encode("utf-8")
def _sha(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _text(value: object, name: str, *, max_len: int = 256) -> str:
if not isinstance(value, str):
raise SchedulabilityError(f"{name} must be a string")
value = value.strip()
if not value or len(value) > max_len or any(ord(ch) < 32 for ch in value):
raise SchedulabilityError(f"invalid {name}")
return value
def _int_authority(
value: object,
*,
name: str,
default: int | None = None,
minimum: int = 0,
) -> tuple[int, "Blocker | None"]:
if value is None:
if default is None:
return 0, Blocker("INVALID_AUTHORITY_VALUE", "worker", f"{name} missing")
return default, None
if isinstance(value, bool) or not isinstance(value, int):
return 0, Blocker("INVALID_AUTHORITY_VALUE", "worker", f"{name} must be integer")
if value < minimum:
return value, Blocker("INVALID_AUTHORITY_VALUE", "worker", f"{name} below {minimum}")
return value, None
@dataclass(frozen=True)
class Blocker:
code: str
plane: str
detail: str
def to_dict(self) -> dict[str, str]:
return asdict(self)
@dataclass(frozen=True)
class ClaimConditions:
"""Mutable authority conditions that MUST be checked atomically with claim."""
task_revision: int
task_generation: int
task_status: str
worker_generation: int
worker_identity: str
authorization_ref: str
authorization_revision: str
scope_digest: str
capacity_limit: int
def to_dict(self) -> dict[str, object]:
return asdict(self)
@dataclass(frozen=True)
class SchedulabilityObservation:
task_id: str
task_revision: int
task_generation: int
task_status: str
worker_id: str
worker_generation: int
worker_identity: str
heartbeat_observed_at: float
heartbeat_fresh: bool
required_capabilities: tuple[str, ...]
worker_capabilities: tuple[str, ...]
scope_digest: str
authorization_ref: str
authorization_revision: str
authorization_allowed: bool
capacity_limit: int
active_claims: int
dependencies_satisfied: bool
freeze_blocked: bool
conflicting_lease: bool
@property
def capacity_remaining(self) -> int:
return max(0, self.capacity_limit - self.active_claims)
def stable_binding(self) -> dict[str, object]:
"""Fields whose change invalidates an earlier proof."""
return {
"task_id": self.task_id,
"task_revision": self.task_revision,
"task_generation": self.task_generation,
"task_status": self.task_status,
"worker_id": self.worker_id,
"worker_generation": self.worker_generation,
"worker_identity": self.worker_identity,
"required_capabilities": list(self.required_capabilities),
"worker_capabilities": list(self.worker_capabilities),
"scope_digest": self.scope_digest,
"authorization_ref": self.authorization_ref,
"authorization_revision": self.authorization_revision,
"authorization_allowed": self.authorization_allowed,
"capacity_limit": self.capacity_limit,
"active_claims": self.active_claims,
"dependencies_satisfied": self.dependencies_satisfied,
"freeze_blocked": self.freeze_blocked,
"conflicting_lease": self.conflicting_lease,
"heartbeat_fresh": self.heartbeat_fresh,
}
def claim_conditions(self) -> ClaimConditions:
return ClaimConditions(
task_revision=self.task_revision,
task_generation=self.task_generation,
task_status=self.task_status,
worker_generation=self.worker_generation,
worker_identity=self.worker_identity,
authorization_ref=self.authorization_ref,
authorization_revision=self.authorization_revision,
scope_digest=self.scope_digest,
capacity_limit=self.capacity_limit,
)
@dataclass(frozen=True)
class SchedulabilityProof:
schema: str
proof_id: str
task_id: str
worker_id: str
actor_identity: str
created_at: float
expires_at: float
observation: SchedulabilityObservation
blockers: tuple[Blocker, ...]
observation_digest: str
proof_digest: str
@classmethod
def create(
cls,
*,
task_id: str,
worker_id: str,
actor_identity: str,
observation: SchedulabilityObservation,
blockers: Sequence[Blocker],
now: float,
ttl_seconds: float,
proof_id: str | None = None,
) -> "SchedulabilityProof":
if ttl_seconds <= 0 or ttl_seconds > 300:
raise ValueError("schedulability proof ttl must be in (0,300]")
payload = {
"schema": "aios.schedulability-proof.v2",
"proof_id": proof_id or f"sched-{uuid.uuid4()}",
"task_id": _text(task_id, "task_id"),
"worker_id": _text(worker_id, "worker_id"),
"actor_identity": _text(actor_identity, "actor_identity", max_len=512),
"created_at": float(now),
"expires_at": float(now + ttl_seconds),
"observation": observation.stable_binding(),
"blockers": [b.to_dict() for b in blockers],
}
obs_digest = _sha(payload["observation"])
proof_digest = _sha({**payload, "observation_digest": obs_digest})
return cls(
schema=payload["schema"],
proof_id=payload["proof_id"],
task_id=payload["task_id"],
worker_id=payload["worker_id"],
actor_identity=payload["actor_identity"],
created_at=payload["created_at"],
expires_at=payload["expires_at"],
observation=observation,
blockers=tuple(blockers),
observation_digest=obs_digest,
proof_digest=proof_digest,
)
@property
def claimable(self) -> bool:
return not self.blockers
@dataclass(frozen=True)
class DiscoveryDiagnostics:
candidate_count: int
blocker_codes: tuple[str, ...]
class SchedulabilityAuthority(Protocol):
"""Read authorities and perform the final existing claim transaction."""
def read_task(self, task_id: str) -> Mapping[str, Any]: ...
def list_candidate_tasks(self) -> Sequence[Mapping[str, Any]]: ...
def read_worker(self, worker_id: str) -> Mapping[str, Any]: ...
def read_authorization(
self, *, task: Mapping[str, Any], worker: Mapping[str, Any], actor_identity: str
) -> Mapping[str, Any]: ...
def active_claim_count(self, worker_id: str) -> int: ...
def dependencies_satisfied(self, task: Mapping[str, Any]) -> bool: ...
def freeze_blocked(self, task: Mapping[str, Any]) -> bool: ...
def conflicting_lease(self, task_id: str) -> bool: ...
def exact_scope_digest(
self, *, task: Mapping[str, Any], worker: Mapping[str, Any], actor_identity: str
) -> tuple[bool, str]: ...
def claim_atomically(
self,
*,
task_id: str,
worker_id: str,
actor_identity: str,
expected: ClaimConditions,
lease_seconds: int,
schedulability_proof_digest: str,
) -> Mapping[str, Any]:
"""Claim at the existing TaskPlane linearization point.
The implementation MUST, in the same authoritative transaction / fenced
operation that writes the claim:
- verify task revision/generation/status is still claimable;
- verify worker registration/identity/health/capacity;
- verify authorization ref/revision remains allowed;
- verify exact scope;
- verify dependencies/freeze/conflicting lease;
- create claim + lease/fencing receipt.
This protocol is an adapter contract; it is not evidence that a specific
production adapter already implements the transaction.
"""
class ProofRepository(Protocol):
def put(self, proof: SchedulabilityProof) -> None: ...
def get(self, proof_id: str) -> SchedulabilityProof | None: ...
class MemoryProofRepository:
def __init__(self) -> None:
self._values: dict[str, SchedulabilityProof] = {}
def put(self, proof: SchedulabilityProof) -> None:
existing = self._values.get(proof.proof_id)
if existing is not None and existing.proof_digest != proof.proof_digest:
raise SchedulabilityError("proof id replay conflict")
self._values[proof.proof_id] = proof
def get(self, proof_id: str) -> SchedulabilityProof | None:
return self._values.get(proof_id)
class SchedulabilityService:
def __init__(
self,
authority: SchedulabilityAuthority,
proofs: ProofRepository,
*,
clock=time.time,
proof_ttl_seconds: float = 30.0,
heartbeat_grace_factor: float = 3.0,
) -> None:
self.authority = authority
self.proofs = proofs
self.clock = clock
self.proof_ttl_seconds = proof_ttl_seconds
self.heartbeat_grace_factor = heartbeat_grace_factor
@staticmethod
def _caps(value: object) -> tuple[str, ...]:
if not isinstance(value, (list, tuple)):
return ()
return tuple(sorted({str(v) for v in value if isinstance(v, str) and v}))
def observe(
self, task_id: str, worker_id: str, actor_identity: str
) -> tuple[SchedulabilityObservation, tuple[Blocker, ...]]:
task = self.authority.read_task(task_id)
worker = self.authority.read_worker(worker_id)
auth = self.authority.read_authorization(
task=task, worker=worker, actor_identity=actor_identity
)
required = self._caps(task.get("required_capabilities") or ())
offered = self._caps(worker.get("capabilities") or ())
now = float(self.clock())
last = float(worker.get("last_heartbeat") or worker.get("last_seen_at_epoch") or 0.0)
interval = float(worker.get("heartbeat_interval_seconds") or 15.0)
fresh = (
str(worker.get("status") or "").lower() == "active"
and not bool(worker.get("disabled"))
and last <= now
and now - last <= max(interval * self.heartbeat_grace_factor, 1.0)
)
scope_ok, scope_digest = self.authority.exact_scope_digest(
task=task, worker=worker, actor_identity=actor_identity
)
deps = bool(self.authority.dependencies_satisfied(task))
freeze = bool(self.authority.freeze_blocked(task))
lease = bool(self.authority.conflicting_lease(task_id))
active_raw = self.authority.active_claim_count(worker_id)
active, active_error = _int_authority(active_raw, name="active_claim_count", minimum=0)
capacity, capacity_error = _int_authority(
worker.get("max_concurrency"),
name="max_concurrency",
default=1,
minimum=0,
)
auth_allowed = bool(auth.get("allowed"))
task_status = str(task.get("status") or "")
blockers: list[Blocker] = []
if task_status not in CLAIMABLE_TASK_STATES:
blockers.append(
Blocker(
"TASK_STATE_NOT_CLAIMABLE",
"task",
f"task status {task_status or '<missing>'} is not claimable",
)
)
if active_error:
blockers.append(active_error)
if capacity_error:
blockers.append(capacity_error)
if not fresh:
blockers.append(Blocker("STALE_OR_INACTIVE_WORKER", "worker", "worker heartbeat is not fresh"))
missing = sorted(set(required) - set(offered))
if missing:
blockers.append(Blocker("MISSING_CAPABILITY", "worker", ",".join(missing)))
if not scope_ok:
blockers.append(Blocker("EXACT_SCOPE_MISMATCH", "authorization", "worker/task signed scope mismatch"))
if not auth_allowed:
blockers.append(Blocker("AUTHORIZATION_BLOCKED", "authorization", "current authorization does not allow claim"))
if capacity >= 0 and active >= capacity:
blockers.append(Blocker("CAPACITY_EXHAUSTED", "worker", f"{active}/{capacity} active"))
if not deps:
blockers.append(Blocker("DEPENDENCY_BLOCKED", "task", "task dependencies are not satisfied"))
if freeze:
blockers.append(Blocker("FREEZE_BLOCKED", "task", "task/environment freeze is active"))
if lease:
blockers.append(Blocker("CONFLICTING_LEASE", "task", "a live conflicting lease exists"))
obs = SchedulabilityObservation(
task_id=_text(task.get("task_id"), "task_id"),
task_revision=int(task.get("revision") or 0),
task_generation=int(task.get("generation") or task.get("metadata", {}).get("generation") or 0),
task_status=task_status,
worker_id=_text(worker.get("worker_id"), "worker_id"),
worker_generation=int(worker.get("registration_revision") or worker.get("generation") or 0),
worker_identity=_text(worker.get("identity"), "worker_identity", max_len=512),
heartbeat_observed_at=last,
heartbeat_fresh=fresh,
required_capabilities=required,
worker_capabilities=offered,
scope_digest=_text(scope_digest, "scope_digest", max_len=256),
authorization_ref=_text(auth.get("authorization_ref"), "authorization_ref", max_len=512),
authorization_revision=_text(auth.get("revision"), "authorization_revision", max_len=256),
authorization_allowed=auth_allowed,
capacity_limit=capacity,
active_claims=active,
dependencies_satisfied=deps,
freeze_blocked=freeze,
conflicting_lease=lease,
)
return obs, tuple(blockers)
def preflight(
self,
*,
task_id: str,
worker_id: str,
actor_identity: str,
persist: bool = False,
) -> SchedulabilityProof:
now = float(self.clock())
obs, blockers = self.observe(task_id, worker_id, actor_identity)
proof = SchedulabilityProof.create(
task_id=task_id,
worker_id=worker_id,
actor_identity=actor_identity,
observation=obs,
blockers=blockers,
now=now,
ttl_seconds=self.proof_ttl_seconds,
)
if persist:
self.proofs.put(proof)
return proof
def discover_with_diagnostics(
self, *, worker_id: str, actor_identity: str
) -> tuple[SchedulabilityProof | None, DiscoveryDiagnostics]:
candidates = sorted(
self.authority.list_candidate_tasks(),
key=lambda task: (
-int(task.get("priority") or 0),
str(task.get("created_at") or ""),
str(task.get("task_id") or ""),
),
)
codes: set[str] = set()
observed = 0
for task in candidates:
task_id = str(task.get("task_id") or "")
if not task_id:
continue
observed += 1
proof = self.preflight(
task_id=task_id,
worker_id=worker_id,
actor_identity=actor_identity,
persist=False,
)
if proof.claimable:
self.proofs.put(proof)
return proof, DiscoveryDiagnostics(observed, tuple(sorted(codes)))
codes.update(b.code for b in proof.blockers)
return None, DiscoveryDiagnostics(observed, tuple(sorted(codes)))
def discover(
self, *, worker_id: str, actor_identity: str
) -> SchedulabilityProof | None:
proof, _ = self.discover_with_diagnostics(
worker_id=worker_id, actor_identity=actor_identity
)
return proof
@staticmethod
def _validate_claim_receipt(result: Mapping[str, Any]) -> None:
required = ("claim_id", "lease_id", "fencing_generation", "receipt_revision")
missing = [name for name in required if result.get(name) in (None, "")]
if missing:
raise SchedulabilityChanged(
"claim authority did not return required lease/fence receipt: "
+ ",".join(missing)
)
def claim_with_recheck(
self,
*,
proof_id: str,
worker_id: str,
actor_identity: str,
lease_seconds: int = 30,
) -> Mapping[str, Any]:
proof = self.proofs.get(proof_id)
if proof is None:
raise SchedulabilityChanged("proof not found")
now = float(self.clock())
if proof.expires_at <= now:
raise SchedulabilityChanged("proof expired")
if proof.worker_id != worker_id or proof.actor_identity != actor_identity:
raise SchedulabilityChanged("proof identity mismatch")
current, blockers = self.observe(proof.task_id, worker_id, actor_identity)
if blockers:
raise SchedulabilityChanged(
"claim-time blockers: " + ",".join(sorted(b.code for b in blockers))
)
if current.stable_binding() != proof.observation.stable_binding():
raise SchedulabilityChanged("claim-time authority changed since proof")
result = self.authority.claim_atomically(
task_id=proof.task_id,
worker_id=worker_id,
actor_identity=actor_identity,
expected=current.claim_conditions(),
lease_seconds=lease_seconds,
schedulability_proof_digest=proof.proof_digest,
)
self._validate_claim_receipt(result)
return result
+266 -25
View File
@@ -272,6 +272,13 @@ class TaskPlaneStore:
)
"""
)
self._ensure_columns(
"tasks",
{
"owner_principal_id": "TEXT NOT NULL DEFAULT ''",
"mission_id": "TEXT NOT NULL DEFAULT ''",
},
)
self._ensure_columns(
"workers",
{
@@ -305,6 +312,167 @@ class TaskPlaneStore:
"CREATE UNIQUE INDEX IF NOT EXISTS ux_events_task_seq ON events(task_id, seq)"
)
self.conn.execute("CREATE INDEX IF NOT EXISTS ix_outbox_status ON outbox(delivery_status)")
self.conn.execute(
"""
CREATE TABLE IF NOT EXISTS task_lineage_sequences (
parent_task_id TEXT NOT NULL,
relation_type TEXT NOT NULL,
child_task_id TEXT NOT NULL UNIQUE,
sibling_sequence INTEGER NOT NULL,
created_at REAL NOT NULL,
PRIMARY KEY(parent_task_id, relation_type, sibling_sequence)
)
"""
)
self.conn.execute(
"""
CREATE TABLE IF NOT EXISTS task_terminal_proofs (
task_id TEXT PRIMARY KEY,
claim_id TEXT NOT NULL UNIQUE,
task_revision INTEGER NOT NULL,
result_digest TEXT NOT NULL,
evidence_refs_json TEXT NOT NULL,
created_at REAL NOT NULL
)
"""
)
self.conn.execute(
"""
CREATE TABLE IF NOT EXISTS task_root_reviews (
review_id TEXT PRIMARY KEY,
root_task_id TEXT NOT NULL,
trigger_child_task_id TEXT NOT NULL,
trigger_child_revision INTEGER NOT NULL,
root_revision_before INTEGER NOT NULL,
root_revision_after INTEGER NOT NULL,
record_json TEXT NOT NULL,
created_at REAL NOT NULL
)
"""
)
self.conn.execute(
"""
CREATE TABLE IF NOT EXISTS task_root_action_dispatches (
review_id TEXT PRIMARY KEY,
root_task_id TEXT NOT NULL,
action TEXT NOT NULL,
idempotency_key TEXT NOT NULL UNIQUE,
status TEXT NOT NULL,
created_at REAL NOT NULL
)
"""
)
self._ensure_columns(
"task_root_action_dispatches",
{
"mission_id": "TEXT NOT NULL DEFAULT ''",
"root_revision": "INTEGER NOT NULL DEFAULT 0",
"row_version": "INTEGER NOT NULL DEFAULT 0",
"plan_json": "TEXT",
"plan_digest": "TEXT",
"execution_key": "TEXT",
"claim_owner": "TEXT",
"claim_token": "TEXT",
"claim_fence": "INTEGER",
"claim_expires_at": "REAL",
"execution_receipt_json": "TEXT",
"execution_receipt_digest": "TEXT",
"rereview_json": "TEXT",
"rereview_digest": "TEXT",
"retry_at": "REAL",
"last_error_code": "TEXT",
"updated_at": "REAL",
},
)
self.conn.execute(
"""
CREATE TABLE IF NOT EXISTS task_root_action_effects (
execution_key TEXT PRIMARY KEY,
root_task_id TEXT NOT NULL,
target_task_id TEXT,
target_job_id TEXT,
request_digest TEXT NOT NULL,
state TEXT NOT NULL,
effect_phase TEXT NOT NULL DEFAULT 'LEGACY_UNKNOWN',
receipt_json TEXT,
receipt_digest TEXT,
retry_at REAL,
last_error_code TEXT,
created_at REAL NOT NULL,
updated_at REAL NOT NULL
)
"""
)
self._ensure_columns(
"task_root_action_effects",
{
# Existing v1/v2 EFFECT_UNKNOWN rows may already have sent
# an external request. They must never migrate to a phase
# that permits a new lease or commit.
"effect_phase": "TEXT NOT NULL DEFAULT 'LEGACY_UNKNOWN'",
},
)
self.conn.execute(
"""
CREATE TABLE IF NOT EXISTS task_root_action_fences (
root_task_id TEXT PRIMARY KEY,
next_fence INTEGER NOT NULL,
active_review_id TEXT,
active_fence INTEGER,
active_phase TEXT,
updated_at REAL NOT NULL
)
"""
)
self.conn.execute(
"CREATE INDEX IF NOT EXISTS ix_root_action_dispatch_ready "
"ON task_root_action_dispatches(status,retry_at,created_at)"
)
self.conn.execute(
"CREATE INDEX IF NOT EXISTS ix_root_action_dispatch_root "
"ON task_root_action_dispatches(root_task_id,root_revision)"
)
self.conn.execute(
"CREATE UNIQUE INDEX IF NOT EXISTS ux_root_review_trigger "
"ON task_root_reviews(trigger_child_task_id, trigger_child_revision)"
)
self.conn.execute(
"""
CREATE TABLE IF NOT EXISTS task_missions (
mission_id TEXT PRIMARY KEY,
root_task_id TEXT NOT NULL UNIQUE,
authenticated_identity TEXT NOT NULL,
owner_principal_id TEXT NOT NULL,
owner_alias TEXT NOT NULL,
goal_slug TEXT NOT NULL,
root_acceptance_ref TEXT NOT NULL UNIQUE,
acceptance_verifier_id TEXT NOT NULL,
original_acceptance_json TEXT NOT NULL,
request_digest TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
created_at REAL NOT NULL,
UNIQUE(authenticated_identity, idempotency_key)
)
"""
)
self.conn.execute(
"CREATE INDEX IF NOT EXISTS ix_task_missions_owner "
"ON task_missions(authenticated_identity, mission_id)"
)
self.conn.execute(
"""
CREATE TABLE IF NOT EXISTS task_mission_delegations (
authenticated_identity TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
mission_id TEXT NOT NULL,
parent_task_id TEXT NOT NULL,
child_task_id TEXT NOT NULL UNIQUE,
request_digest TEXT NOT NULL,
created_at REAL NOT NULL,
PRIMARY KEY(authenticated_identity, idempotency_key)
)
"""
)
def _ensure_columns(self, table: str, columns: dict[str, str]) -> None:
existing = {
@@ -359,6 +527,8 @@ class TaskPlaneStore:
root_task_id=row["root_task_id"],
idempotency_key=row["idempotency_key"],
failure_reason=row["failure_reason"],
owner_principal_id=row["owner_principal_id"],
mission_id=row["mission_id"],
)
def _row_to_worker(self, row: sqlite3.Row | None) -> WorkerRegistration:
@@ -385,20 +555,32 @@ class TaskPlaneStore:
disabled_reason=row["disabled_reason"],
)
def create_task(self, task_plan: TaskPlan, idempotency_key: str | None = None) -> TaskRecord:
def create_task(
self,
task_plan: TaskPlan,
idempotency_key: str | None = None,
*,
owner_principal_id: str = "",
mission_id: str = "",
created_at: float | None = None,
) -> TaskRecord:
task_id = task_plan.task_id or f"task-{uuid.uuid4()}"
task_plan.task_id = task_id
task_json = redact_value(task_plan.to_dict(redact=False))
now = _now()
now = _now() if created_at is None else float(created_at)
with self._transaction():
if idempotency_key:
existing = self.conn.execute(
"SELECT task_id,task_json FROM tasks WHERE idempotency_key=?",
"SELECT task_id,task_json,owner_principal_id,mission_id FROM tasks WHERE idempotency_key=?",
(idempotency_key,),
).fetchone()
if existing:
if _canonical(json.loads(existing["task_json"])) != _canonical(task_json):
raise ConflictError("idempotency key payload mismatch")
if owner_principal_id and existing["owner_principal_id"] != owner_principal_id:
raise ConflictError("idempotency key owner mismatch")
if mission_id and existing["mission_id"] != mission_id:
raise ConflictError("idempotency key mission mismatch")
return self.get_task(existing["task_id"])
root_task_id = task_id
for dependency in task_plan.dependencies:
@@ -412,8 +594,8 @@ class TaskPlaneStore:
self.conn.execute(
"""
INSERT INTO tasks
(task_id, revision, status, task_json, task_type, root_task_id, parent_task_id, correlation_id, idempotency_key, created_at, updated_at, failure_reason)
VALUES (?, 1, ?, ?, ?, ?, ?, ?, ?, ?, ?, NULL)
(task_id, revision, status, task_json, task_type, root_task_id, parent_task_id, correlation_id, idempotency_key, created_at, updated_at, failure_reason, owner_principal_id, mission_id)
VALUES (?, 1, ?, ?, ?, ?, ?, ?, ?, ?, ?, NULL, ?, ?)
""",
(
task_id,
@@ -426,6 +608,8 @@ class TaskPlaneStore:
idempotency_key,
now,
now,
owner_principal_id,
mission_id,
),
)
self._set_budget(task_id, task_plan.budgets)
@@ -437,10 +621,22 @@ class TaskPlaneStore:
return self.get_task(task_id)
def create_task_with_event(
self, task_plan: TaskPlan, idempotency_key: str | None = None
self,
task_plan: TaskPlan,
idempotency_key: str | None = None,
*,
owner_principal_id: str = "",
mission_id: str = "",
created_at: float | None = None,
) -> TaskRecord:
with self._transaction():
task = self.create_task(task_plan, idempotency_key=idempotency_key)
task = self.create_task(
task_plan,
idempotency_key=idempotency_key,
owner_principal_id=owner_principal_id,
mission_id=mission_id,
created_at=created_at,
)
if not self.list_events(task.task_id):
self._fault("create.after_task_before_event")
self.append_event(
@@ -461,6 +657,26 @@ class TaskPlaneStore:
raise NotFoundError(f"task {task_id} not found")
return self._row_to_task(row)
def assert_root_open_for_child(self, root_task_id: str) -> TaskRecord:
"""Fence child admission against root completion/action finalization."""
root = self.get_task(root_task_id)
if root.task_id != root.root_task_id:
raise ConflictError("child admission authority is not a root task")
if root.status == "COMPLETED":
raise ConflictError("cannot add a child below a completed root")
fence = self.conn.execute(
"SELECT active_phase FROM task_root_action_fences WHERE root_task_id=?",
(root_task_id,),
).fetchone()
if fence is not None and fence["active_phase"] in {
"CLAIMED",
"EFFECT_UNKNOWN",
"EXECUTED_PENDING_REREVIEW",
"ROOT_CLOSING",
}:
raise ConflictError("root action fence blocks child admission")
return root
def list_tasks(self) -> list[TaskRecord]:
with self._lock:
rows = self.conn.execute(
@@ -940,23 +1156,29 @@ class TaskPlaneStore:
def heartbeat_claim(self, task_id: str, claim_id: str, lease_seconds: int) -> WorkerClaim:
now = _now()
lease_until = now + max(1, lease_seconds)
row = self.conn.execute(
"SELECT * FROM claims WHERE claim_id=? AND task_id=?",
(claim_id, task_id),
).fetchone()
if row is None:
raise NotFoundError("claim")
claim = self._row_to_claim(row)
if claim.status != "ACTIVE":
raise ConflictError("claim not active")
if claim.lease_until < now:
raise LockedError("lease expired")
with self._transaction():
self.conn.execute(
"UPDATE claims SET lease_until=? WHERE claim_id=?",
(lease_until, claim_id),
row = self.conn.execute(
"SELECT * FROM claims WHERE claim_id=? AND task_id=?",
(claim_id, task_id),
).fetchone()
if row is None:
raise NotFoundError("claim")
claim = self._row_to_claim(row)
if claim.status != "ACTIVE":
raise ConflictError("claim not active")
if claim.lease_until <= now:
raise LockedError("lease expired")
if self.get_task(task_id).status in {"CANCELLED", "COMPLETED", "FAILED_FINAL"}:
raise ConflictError("task is terminal")
updated = self.conn.execute(
"""
UPDATE claims SET lease_until=?
WHERE claim_id=? AND task_id=? AND status='ACTIVE' AND lease_until>?
""",
(lease_until, claim_id, task_id, now),
)
if updated.rowcount != 1:
raise LockedError("claim changed during heartbeat")
self.append_event(task_id, "TASK_PROGRESS", {"claim_id": claim_id, "stage": "heartbeat"}, subject=f"task/{task_id}")
return self.get_claim(claim_id)
@@ -1064,7 +1286,14 @@ class TaskPlaneStore:
),
)
self._fault("recover.after_probe_before_state")
if probe.outcome == ProbeOutcome.APPLIED_VERIFIED:
current_task = self.get_task(task_id)
if current_task.status in {"CANCELLED", "COMPLETED", "FAILED_FINAL"}:
# Recovery records what happened; it cannot undo a terminal Owner
# decision or reopen a finished task. This check shares the state
# transaction with the claim close and the appended probe event.
task_status = current_task.status
claim_status = "CANCELLED" if current_task.status == "CANCELLED" else "CLOSED_BY_RECOVERY"
elif probe.outcome == ProbeOutcome.APPLIED_VERIFIED:
task_status = "COMPLETED"
claim_status = "COMPLETED"
elif probe.outcome == ProbeOutcome.CONFLICT:
@@ -1238,7 +1467,14 @@ class TaskPlaneStore:
return updated
def delegate_task_with_budget_and_events(
self, parent_task_id: str, child_plan: TaskPlan, *, caller: str
self,
parent_task_id: str,
child_plan: TaskPlan,
*,
caller: str,
owner_principal_id: str = "",
mission_id: str = "",
created_at: float | None = None,
) -> TaskRecord:
with self._transaction():
parent_budget = self.get_task_budget(parent_task_id)
@@ -1257,7 +1493,12 @@ class TaskPlaneStore:
self.consume_budget(parent_task_id, needed)
self._fault("delegate.after_budget_before_child")
child_plan.parent_task_id = parent_task_id
child = self.create_task(child_plan)
child = self.create_task(
child_plan,
owner_principal_id=owner_principal_id,
mission_id=mission_id,
created_at=created_at,
)
self.append_event(
parent_task_id,
"TASK_DELEGATED",
@@ -0,0 +1,617 @@
#!/usr/bin/env python3
"""Trusted evidence-provider executor (V2R2C2 P0 Round-3).
This module is the *formal trusted executor* that produces root-owned provider
receipts under each evidence-provider store. It is a real provider execution
path: for each evidence kind it executes the actual provider logic (pytest /
Gitea machine+API consistency / gitleaks), derives the result from real tool
output, and only then writes an ``aios.trusted-evidence-provider-receipt.v2``
receipt into the store. It never manufactures a bare ``{"passed": true}``:
every receipt carries a canonical content digest over a real result.
Security contract (matches TrustedProviderReceiptResolver):
- run as root (``os.geteuid() == 0``); store dirs and receipts are root-owned.
- store dir mode in ``{0700, 0750}``, receipt mode in ``{0600, 0640}``;
no group/world write; no symlink followed (O_NOFOLLOW path).
- receipt ref is relative and free of ``..``/``.``/absolute components.
- content_sha256 = canonical_sha256(result) (sort_keys, compact, utf-8).
- source_oid / tree_oid bound from the exact release source checkout.
"""
from __future__ import annotations
import hashlib
import json
import os
import re
import stat
import subprocess
import sys
import time
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Mapping
PROVIDER_RECEIPT_SCHEMA = "aios.trusted-evidence-provider-receipt.v2"
PROVIDER_RECEIPT_VERSION = 2
_HEX40 = re.compile(r"^[0-9a-f]{40}$")
_HEX64 = re.compile(r"^[0-9a-f]{64}$")
# Storage layout is resolved from the governed providers policy; these defaults
# mirror config/gitops/trusted-evidence-providers.json so the executor is usable
# before the policy is deployed, but the policy is authoritative at runtime.
POLICY_PATH = (
Path(__file__).resolve().parents[2]
/ "config"
/ "gitops"
/ "trusted-evidence-providers.json"
)
class TrustedProviderExecutionError(RuntimeError):
"""Fail-closed provider execution / receipt write error."""
def canonical_json_bytes(value: Any) -> bytes:
return json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
def canonical_sha256(value: Any) -> str:
return hashlib.sha256(canonical_json_bytes(value)).hexdigest()
def now_iso() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _require_root() -> None:
if hasattr(os, "geteuid") and os.geteuid() != 0:
raise TrustedProviderExecutionError(
"PROVIDER_EXECUTOR_REQUIRES_ROOT (run via the governed root executor)"
)
def _validate_oid(value: str, field: str) -> str:
v = str(value or "").strip().lower()
if not _HEX40.fullmatch(v):
raise TrustedProviderExecutionError(f"{field.upper()}_INVALID")
return v
def _safe_relative_ref(receipt_ref: str) -> tuple[str, ...]:
ref = receipt_ref.strip()
if not ref or ref.startswith("/"):
raise TrustedProviderExecutionError("PROVIDER_RECEIPT_REF_ABSOLUTE_DENIED")
parts = ref.split("/")
if not parts or any(part in {"", ".", ".."} for part in parts):
raise TrustedProviderExecutionError("PROVIDER_RECEIPT_REF_TRAVERSAL_DENIED")
return tuple(parts)
def load_provider_policy(path: Path = POLICY_PATH) -> Mapping[str, Any]:
try:
decoded = json.loads(Path(path).read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
raise TrustedProviderExecutionError("PROVIDER_POLICY_UNAVAILABLE") from exc
if (
not isinstance(decoded, Mapping)
or decoded.get("schema") != "aios.trusted-evidence-providers.v1"
):
raise TrustedProviderExecutionError("PROVIDER_POLICY_INVALID")
return decoded
def provider_store_root(kind: str, policy_path: Path = POLICY_PATH) -> Path:
policy = load_provider_policy(policy_path)
providers = policy.get("providers")
if not isinstance(providers, Mapping) or kind not in providers:
raise TrustedProviderExecutionError(f"PROVIDER_POLICY_{kind.upper()}_NOT_FOUND")
raw = providers[kind]
if not isinstance(raw, Mapping):
raise TrustedProviderExecutionError("PROVIDER_POLICY_INVALID")
store_root = str(raw.get("store_root") or "").strip()
if not store_root:
raise TrustedProviderExecutionError("PROVIDER_STORE_ROOT_UNSET")
ec_path = os.environ.get("AIOS_EVIDENCE_PROVIDERS_ROOT")
if ec_path:
store_root = ec_path
root = Path(store_root).expanduser()
if not root.is_absolute():
raise TrustedProviderExecutionError("PROVIDER_STORE_ROOT_MUST_BE_ABSOLUTE")
return root
def _run(
command: list[str],
*,
cwd: Path | None = None,
timeout: int = 600,
extra_env: Mapping[str, str] | None = None,
) -> subprocess.CompletedProcess[str]:
env = dict(os.environ)
if extra_env:
env.update(extra_env)
try:
return subprocess.run(
command,
cwd=cwd,
text=True,
capture_output=True,
timeout=timeout,
env=env,
)
except (OSError, subprocess.TimeoutExpired) as exc:
raise TrustedProviderExecutionError("PROVIDER_TOOL_UNAVAILABLE") from exc
def _tests_env(repo_root: Path) -> dict[str, str]:
"""Forward source PYTHONPATH to pytest subprocess spawns.
Mirrors tools/baseline_v1_1/run_acceptance.py: subprocess-based tests (e.g.
job-safety process-crash recovery) import runtime modules from
active-runtime/app, so PYTHONPATH must be propagated or they fail with
ModuleNotFoundError. See docs/implementation-baseline-1.1/ACCEPTANCE_RESULTS.md.
"""
app = repo_root / "active-runtime" / "app"
return {"PYTHONPATH": os.pathsep.join(str(p) for p in (repo_root, app))}
def _write_receipt(root: Path, receipt_ref: str, payload: Mapping[str, Any], *, mode: int = 0o600) -> str:
"""Atomically write a root-owned receipt with fail-closed store checks."""
_require_root()
root_lstat = os.lstat(root)
if stat.S_ISLNK(root_lstat.st_mode):
raise TrustedProviderExecutionError("PROVIDER_STORE_SYMLINK_DENIED")
if not stat.S_ISDIR(root_lstat.st_mode):
raise TrustedProviderExecutionError("PROVIDER_STORE_NOT_DIRECTORY")
if root_lstat.st_uid != 0:
raise TrustedProviderExecutionError("PROVIDER_STORE_NOT_ROOT_OWNED")
if stat.S_IMODE(root_lstat.st_mode) & 0o022:
raise TrustedProviderExecutionError("PROVIDER_STORE_WRITABLE_DENIED")
parts = _safe_relative_ref(receipt_ref)
current = root
for part in parts[:-1]:
current = current / part
try:
st = os.lstat(current)
except FileNotFoundError:
# Provider executor owns the store: create missing root-owned dirs
# under the (already validated) store root, fail-closed if we cannot.
os.mkdir(current, 0o700)
os.chown(current, 0, 0)
os.chmod(current, 0o700)
st = os.lstat(current)
if stat.S_ISLNK(st.st_mode):
raise TrustedProviderExecutionError("PROVIDER_STORE_SYMLINK_DENIED")
if not stat.S_ISDIR(st.st_mode):
raise TrustedProviderExecutionError("PROVIDER_STORE_NOT_DIRECTORY")
if st.st_uid != 0:
raise TrustedProviderExecutionError("PROVIDER_STORE_NOT_ROOT_OWNED")
if stat.S_IMODE(st.st_mode) & 0o022:
raise TrustedProviderExecutionError("PROVIDER_STORE_WRITABLE_DENIED")
target = current / parts[-1]
raw = canonical_json_bytes(payload)
flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_CLOEXEC", 0)
if hasattr(os, "O_NOFOLLOW"):
flags |= os.O_NOFOLLOW
try:
fd = os.open(target, flags, mode)
except OSError as exc:
raise TrustedProviderExecutionError("PROVIDER_RECEIPT_WRITE_DENIED") from exc
try:
os.write(fd, raw)
os.fsync(fd)
finally:
os.close(fd)
os.chown(target, 0, 0)
os.chmod(target, mode)
st = os.lstat(target)
if stat.S_IMODE(st.st_mode) & 0o022 or st.st_uid != 0:
raise TrustedProviderExecutionError("PROVIDER_RECEIPT_PERMISSION_DENIED")
return hashlib.sha256(raw).hexdigest()
def _build_receipt(
*,
kind: str,
provider_identity: str,
provider_operation: str,
operation_run_id: str,
result_reference: str,
result: Mapping[str, Any],
source_oid: str,
tree_oid: str,
store_id: str,
) -> Mapping[str, Any]:
return {
"schema": PROVIDER_RECEIPT_SCHEMA,
"version": PROVIDER_RECEIPT_VERSION,
"provider_identity": provider_identity,
"provider_operation": provider_operation,
"operation_run_id": operation_run_id,
"result_reference": result_reference,
"content_sha256": canonical_sha256(result),
"source_oid": source_oid,
"tree_oid": tree_oid,
"environment": "linux-isolated",
"issued_at": now_iso(),
"result": dict(result),
"authentication": {
"kind": "root-owned-store-v1",
"store_id": store_id,
},
}
# ---------------------------------------------------------------------------
# Provider executors (real tool execution; results derived from real output)
# ---------------------------------------------------------------------------
def _junit_totals(junit_path: Path) -> dict[str, int]:
import xml.etree.ElementTree as ET
if not junit_path.is_file():
raise TrustedProviderExecutionError("TESTS_JUNIT_MISSING")
root = ET.parse(junit_path).getroot()
# pytest emits <testsuites><testsuite tests=.. failures=.. />...</testsuites>;
# aggregate every <testsuite> child (never trust the outer <testsuites> attrib).
totals = {"tests": 0, "failures": 0, "errors": 0, "skipped": 0}
suites = list(root.iter("testsuite"))
for suite in suites:
n = int(suite.attrib.get("tests", 0))
totals["tests"] += n
totals["failures"] += int(suite.attrib.get("failures", 0))
totals["errors"] += int(suite.attrib.get("errors", 0))
totals["skipped"] += int(suite.attrib.get("skipped", 0))
return totals
def run_tests_provider(
*,
repo_root: Path,
collection_inventory: Path,
junit_xml: Path,
source_oid: str,
tree_oid: str,
store_id: str = "tests",
provider_identity: str = "aios.tests-runner",
) -> tuple[Mapping[str, Any], str]:
"""Execute the real test suite, collect-JUnit, and produce a root receipt."""
# First prove POSIX modules are collected (never trust a bare exit code).
collect = _run(
["python3", "-m", "pytest", "--collect-only", "-q", str(repo_root / "tests")],
cwd=repo_root,
timeout=900,
extra_env=_tests_env(repo_root),
)
collected_names = {
line.strip()
for line in collect.stdout.splitlines()
if line.strip()
}
posix_probed = {
"fcntl": any("fcntl" in n for n in collected_names),
"grp": any("grp" in n for n in collected_names),
"so_peercrd": any("peercred" in n.lower() for n in collected_names),
"posix_mode": any("mode" in n.lower() for n in collected_names),
"symlink": any("symlink" in n.lower() for n in collected_names),
"signer_provisioning": any("signer_provisioning" in n.lower() for n in collected_names),
}
if junit_xml.parent and not junit_xml.parent.exists():
junit_xml.parent.mkdir(parents=True, exist_ok=True)
executed = _run(
[
"python3", "-m", "pytest",
"--junitxml=" + str(junit_xml),
"-q", str(repo_root / "tests"),
],
cwd=repo_root,
timeout=3600,
extra_env=_tests_env(repo_root),
)
totals = _junit_totals(junit_xml)
collection_inventory.write_text(
json.dumps({"collected": sorted(collected_names), "posix_probed": posix_probed}, sort_keys=True),
encoding="utf-8",
)
# Passed is only true when a real suite actually ran and was green: a
# vacuous 0-tests JUnit (unwritable junit, collection failure) must never
# report passed. pytest exit codes: 0=passed, 1=test failures, 2=interrupted,
# 3=internal error, 4=usage error, 5=no tests collected.
ran_real_tests = totals["tests"] > 0
green = totals["failures"] == 0 and totals["errors"] == 0
clean_exit = executed.returncode in (0, 5)
result = {
"passed": ran_real_tests and green and clean_exit,
"exit_code": executed.returncode,
"totals": totals,
"junit": junit_xml.name,
"collection_inventory": collection_inventory.name,
"posix_probed": posix_probed,
"run_kind": "full-test-suite",
"green": green,
"ran_real_tests": ran_real_tests,
}
return result, _receipt_ref_for(result, "tests")
def run_gitea_provider(
*,
gitea_api_url: str,
repository_identity: str,
token_file: Path,
source_oid: str,
tree_oid: str,
local_repo: Path,
store_id: str = "gitea",
provider_identity: str = "aios.gitea-machine-transport",
) -> tuple[Mapping[str, Any], str]:
"""Execute real Gitea machine+API consistency and produce a root receipt."""
import urllib.request
token = token_file.read_text(encoding="utf-8").strip()
def _api(path: str) -> dict[str, Any]:
req = urllib.request.Request(
gitea_api_url.rstrip("/") + path,
headers={
"Accept": "application/json",
"Authorization": f"token {token}",
"User-Agent": "aios-trusted-gitea-provider/1",
},
)
with urllib.request.urlopen(req, timeout=20) as resp:
return json.loads(resp.read().decode("utf-8"))
repository = _api(f"/repos/{repository_identity}")
full_name = str(repository.get("full_name") or "")
ref = _api(f"/repos/{repository_identity}/git/refs/heads/main")
items = ref if isinstance(ref, list) else [ref]
exact = [
i for i in items
if isinstance(i, dict) and i.get("ref") in {"refs/heads/main", "heads/main"}
]
local_main = _run(
["git", "-C", str(local_repo), "rev-parse", "refs/heads/main"], cwd=local_repo
)
remote_main = str((exact[0].get("object") or {}).get("sha") or "").lower() if len(exact) == 1 else ""
result = {
"consistent": (
full_name.lower() == repository_identity.lower()
and len(exact) == 1
and remote_main == source_oid.lower()
),
"repository": full_name,
"source_commit": source_oid,
"remote_ref_oid": remote_main,
"tree_oid": tree_oid,
"local_main_oid": local_main.stdout.strip() if local_main.returncode == 0 else "",
}
return result, _receipt_ref_for(result, "gitea")
def run_secret_scan_provider(
*,
gitleaks_bin: str,
target: Path,
source_oid: str,
tree_oid: str,
store_id: str = "secret_scan",
provider_identity: str = "aios.secret-scanner",
) -> tuple[Mapping[str, Any], str]:
"""Execute the real secret scanner (gitleaks) and produce a root receipt."""
if not Path(gitleaks_bin).is_file():
raise TrustedProviderExecutionError("SECRET_SCANNER_UNAVAILABLE")
# --no-git scans the working tree as a filesystem (does not require a .git
# dir), so an exported release checkout can be scanned faithfully.
scanned = _run(
[
gitleaks_bin, "detect",
"--source", str(target),
"--no-git",
"--no-banner",
"--exit-code", "0",
],
cwd=target,
timeout=1800,
)
warnings = 0
# gitleaks exits 1 when findings are present; with --exit-code 0 it returns 0
# but prints findings. Count lines that indicate a finding.
finding_lines = [
ln for ln in scanned.stdout.splitlines()
if ("Finding:" in ln and ":" in ln and "RuleID" not in ln)
] + [
ln for ln in scanned.stderr.splitlines()
if ("Finding:" in ln and ":" in ln and "RuleID" not in ln)
]
artifact_sha = None
if target.is_file():
artifact_sha = hashlib.sha256(target.read_bytes()).hexdigest()
# Passed only when the scanner actually ran cleanly (exit 0) AND found zero
# leaks. A scanner crash (non-zero exit) is never a pass.
clean_run = scanned.returncode == 0
result = {
"passed": clean_run and len(finding_lines) == 0,
"scanner": "gitleaks",
"exit_code": scanned.returncode,
"target": str(target),
"findings": len(finding_lines),
"artifact_sha256": artifact_sha,
"clean_run": clean_run,
}
return result, _receipt_ref_for(result, "secret_scan")
def _receipt_ref_for(result: Mapping[str, Any], kind: str) -> str:
run_id = hashlib.sha256(canonical_json_bytes(result)).hexdigest()[:16]
return f"run-{run_id}/receipt.json"
# ---------------------------------------------------------------------------
# Public orchestration: execute a provider kind and write the root receipt.
# ---------------------------------------------------------------------------
# Provider-kind wiring: each kind maps to a real executor and its governed
# identity / operation. The gate kind (Gate-W vs Gate-R) is read from the
# policy; a Gate-R kind is refused by the Gate-W producer boundary.
_PROVIDER_EXECUTORS: Mapping[str, Mapping[str, Any]] = {
"tests": {
"identity": "aios.tests-runner",
"operation": "tests.run",
"gate": "W",
},
"gitea": {
"identity": "aios.gitea-machine-transport",
"operation": "gitea.consistency",
"gate": "W",
},
"secret_scan": {
"identity": "aios.secret-scanner",
"operation": "secret.scan",
"gate": "W",
},
"canary": {
"identity": "aios.canary-gate",
"operation": "canary.run",
"gate": "R",
},
"rollback": {
"identity": "aios.rollback-controller",
"operation": "rollback.validate",
"gate": "R",
},
}
def _operation_for(kind: str) -> str:
entry = _PROVIDER_EXECUTORS.get(kind)
if entry is None:
raise TrustedProviderExecutionError("PROVIDER_KIND_UNKNOWN")
return entry["operation"]
def _run_id_for(kind: str) -> str:
return f"{kind}-{int(time.time())}"
def _run_provider_executor(
kind: str,
*,
repo_root: Path,
source_oid: str,
tree_oid: str,
) -> tuple[Mapping[str, Any], str]:
"""Run the real provider executor and return (result, receipt_ref).
Every provider derives its result from *real tool output* — never a
manufactured ``{"passed": true}`` and never an acceptance-script-invented
identity. The executor must be invoked under the governed root executor
(root-owned store); any non-executable kind fails closed.
"""
if kind == "tests":
return run_tests_provider(
repo_root=repo_root,
collection_inventory=repo_root / ".provider" / "tests-collection.json",
junit_xml=repo_root / ".provider" / "junit.xml",
source_oid=source_oid,
tree_oid=tree_oid,
)
if kind == "secret_scan":
return run_secret_scan_provider(
gitleaks_bin=os.environ.get("AIOS_GITLEAKS_BIN", "/usr/bin/gitleaks"),
target=repo_root,
source_oid=source_oid,
tree_oid=tree_oid,
)
if kind == "gitea":
gitea_api_url = os.environ.get("AIOS_GITEA_API_URL", "").strip()
token_file = os.environ.get("AIOS_GITEA_TOKEN_FILE", "").strip()
repository_identity = os.environ.get("AIOS_GITEA_REPOSITORY", "").strip()
if not (gitea_api_url and token_file and repository_identity):
raise TrustedProviderExecutionError(
"PROVIDER_MECHANISM_REQUIRED (gitea: API URL / token / repository "
"identity not provisioned; no real provider path available)"
)
return run_gitea_provider(
gitea_api_url=gitea_api_url,
repository_identity=repository_identity,
token_file=Path(token_file),
source_oid=source_oid,
tree_oid=tree_oid,
local_repo=repo_root,
)
raise TrustedProviderExecutionError(f"PROVIDER_MECHANISM_REQUIRED (kind={kind})")
def produce(kind: str, *, source_oid: str, tree_oid: str, receipt_ref: str, store_root: Path) -> str:
"""Execute real provider kind, write its root-owned receipt, return sha256.
Refuses Gate-R kinds at the Gate-W producer boundary, and refuses unless a
real provider execution path (an executor that derives its result from real
tool output under the governed root executor) exists for the kind.
"""
entry = _PROVIDER_EXECUTORS.get(kind)
if entry is None:
raise TrustedProviderExecutionError("PROVIDER_KIND_UNKNOWN")
if entry["gate"] != "W":
raise TrustedProviderExecutionError("GATE_R_PROVIDER_REFUSED_AT_GATE_W_BOUNDARY")
# Resolve a real source checkout for the executor to bind source/tree OID.
repo_root = Path(os.environ.get("AIOS_REPO_ROOT", store_root.parent / "checkout")).resolve()
result, auto_ref = _run_provider_executor(
kind,
repo_root=repo_root,
source_oid=source_oid,
tree_oid=tree_oid,
)
store_root.mkdir(mode=0o700, exist_ok=True)
os.chown(store_root, 0, 0)
os.chmod(store_root, 0o700)
return _write_receipt(store_root, receipt_ref, _build_receipt(
kind=kind,
provider_identity=entry["identity"],
provider_operation=_operation_for(kind),
operation_run_id=_run_id_for(kind),
result_reference=auto_ref,
result=result,
source_oid=source_oid,
tree_oid=tree_oid,
store_id=kind,
))
if __name__ == "__main__":
# CLI: trusted_evidence_providers.py <kind> --source-oid=X --tree-oid=Y --store-root=Z --receipt-ref=R
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("kind", choices=("tests", "gitea", "secret_scan"))
parser.add_argument("--source-oid", required=True)
parser.add_argument("--tree-oid", required=True)
parser.add_argument("--store-root", required=True)
parser.add_argument("--receipt-ref", required=True)
parser.add_argument("--repo-root", default=None, help="Real source checkout to execute against")
args = parser.parse_args()
if args.repo_root:
os.environ["AIOS_REPO_ROOT"] = args.repo_root
try:
digest = produce(
args.kind,
source_oid=_validate_oid(args.source_oid, "source_oid"),
tree_oid=_validate_oid(args.tree_oid, "tree_oid"),
receipt_ref=args.receipt_ref,
store_root=Path(args.store_root),
)
except TrustedProviderExecutionError as exc:
raise SystemExit(str(exc))
print(json.dumps({"receipt_sha256": digest}, sort_keys=True, separators=(",", ":")))
@@ -0,0 +1,390 @@
from __future__ import annotations
import hashlib
import hmac
import json
import os
import re
import stat
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path, PurePosixPath
from typing import Any, Callable, Mapping
PROVIDER_RECEIPT_SCHEMA = "aios.trusted-evidence-provider-receipt.v2"
PROVIDER_RECEIPT_VERSION = 2
_HEX40 = re.compile(r"^[0-9a-f]{40}$")
_HEX64 = re.compile(r"^[0-9a-f]{64}$")
class ProviderAuthenticationError(RuntimeError):
"""Fail-closed provider receipt authentication error."""
@dataclass(frozen=True)
class TrustedProviderPolicy:
evidence_kind: str
provider_identity: str
store_id: str
store_root: Path
allowed_operations: frozenset[str]
expected_owner_uid: int = 0
allowed_directory_modes: frozenset[int] = frozenset({0o700, 0o750})
allowed_receipt_modes: frozenset[int] = frozenset({0o600, 0o640})
authentication_kind: str = "root-owned-store-v1"
hmac_key_id: str | None = None
hmac_namespace: str = "aios.trusted-evidence-provider-receipt.v2"
max_age_seconds: int | None = None
max_future_skew_seconds: int = 30
max_receipt_bytes: int = 1 << 20
require_single_link: bool = True
@dataclass(frozen=True)
class AuthenticatedProviderReceipt:
evidence_kind: str
provider_identity: str
provider_operation: str
operation_run_id: str
result_reference: str
result: Mapping[str, Any]
content_sha256: str
source_oid: str
tree_oid: str
environment: str
issued_at: str
authentication: Mapping[str, Any]
receipt_ref: str
receipt_file_sha256: str
def provenance(self) -> dict[str, Any]:
return {
"schema": PROVIDER_RECEIPT_SCHEMA,
"version": PROVIDER_RECEIPT_VERSION,
"provider_identity": self.provider_identity,
"provider_operation": self.provider_operation,
"operation_run_id": self.operation_run_id,
"result_reference": self.result_reference,
"content_sha256": self.content_sha256,
"source_oid": self.source_oid,
"tree_oid": self.tree_oid,
"environment": self.environment,
"issued_at": self.issued_at,
"authentication": dict(self.authentication),
"receipt_ref": self.receipt_ref,
"receipt_file_sha256": self.receipt_file_sha256,
}
@dataclass(frozen=True)
class ResolvedEvidence:
result: Mapping[str, Any]
provider_receipt: Mapping[str, Any]
def canonical_json_bytes(value: Any) -> bytes:
return json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
def canonical_sha256(value: Any) -> str:
return hashlib.sha256(canonical_json_bytes(value)).hexdigest()
def _parse_issued_at(value: str) -> datetime:
if not isinstance(value, str) or not value.strip():
raise ProviderAuthenticationError("PROVIDER_RECEIPT_ISSUED_AT_INVALID")
text = value.strip()
if text.endswith("Z"):
text = text[:-1] + "+00:00"
try:
parsed = datetime.fromisoformat(text)
except ValueError as exc:
raise ProviderAuthenticationError("PROVIDER_RECEIPT_ISSUED_AT_INVALID") from exc
if parsed.tzinfo is None:
raise ProviderAuthenticationError("PROVIDER_RECEIPT_ISSUED_AT_INVALID")
return parsed.astimezone(timezone.utc)
def _validate_ref(receipt_ref: str) -> tuple[str, ...]:
if not isinstance(receipt_ref, str) or not receipt_ref.strip():
raise ProviderAuthenticationError("PROVIDER_RECEIPT_REF_REQUIRED")
ref = PurePosixPath(receipt_ref.strip())
if ref.is_absolute():
raise ProviderAuthenticationError("PROVIDER_RECEIPT_REF_ABSOLUTE_DENIED")
parts = ref.parts
if not parts or any(part in {"", ".", ".."} for part in parts):
raise ProviderAuthenticationError("PROVIDER_RECEIPT_REF_TRAVERSAL_DENIED")
return tuple(parts)
def _validate_directory_stat(st: os.stat_result, policy: TrustedProviderPolicy) -> None:
if not stat.S_ISDIR(st.st_mode):
raise ProviderAuthenticationError("PROVIDER_STORE_NOT_DIRECTORY")
if st.st_uid != policy.expected_owner_uid:
raise ProviderAuthenticationError("PROVIDER_STORE_OWNER_MISMATCH")
mode = stat.S_IMODE(st.st_mode)
if mode not in policy.allowed_directory_modes:
raise ProviderAuthenticationError("PROVIDER_STORE_MODE_INVALID")
if mode & 0o022:
raise ProviderAuthenticationError("PROVIDER_STORE_WRITABLE_DENIED")
def _validate_receipt_stat(st: os.stat_result, policy: TrustedProviderPolicy) -> None:
if not stat.S_ISREG(st.st_mode):
raise ProviderAuthenticationError("PROVIDER_RECEIPT_NOT_REGULAR")
if st.st_uid != policy.expected_owner_uid:
raise ProviderAuthenticationError("PROVIDER_RECEIPT_OWNER_MISMATCH")
mode = stat.S_IMODE(st.st_mode)
if mode not in policy.allowed_receipt_modes:
raise ProviderAuthenticationError("PROVIDER_RECEIPT_MODE_INVALID")
if mode & 0o022:
raise ProviderAuthenticationError("PROVIDER_RECEIPT_WRITABLE_DENIED")
if policy.require_single_link and st.st_nlink != 1:
raise ProviderAuthenticationError("PROVIDER_RECEIPT_LINK_COUNT_INVALID")
def _read_nofollow(root: Path, parts: tuple[str, ...], policy: TrustedProviderPolicy) -> bytes:
root = Path(root)
try:
root_lstat = os.lstat(root)
except OSError as exc:
raise ProviderAuthenticationError("PROVIDER_STORE_UNAVAILABLE") from exc
if stat.S_ISLNK(root_lstat.st_mode):
raise ProviderAuthenticationError("PROVIDER_STORE_SYMLINK_DENIED")
_validate_directory_stat(root_lstat, policy)
# Validate every parent component with lstat; this is portable and catches
# symlink/path-mode violations before the final no-follow open.
current = root
for part in parts[:-1]:
current = current / part
try:
st = os.lstat(current)
except OSError as exc:
raise ProviderAuthenticationError("PROVIDER_STORE_COMPONENT_UNAVAILABLE") from exc
if stat.S_ISLNK(st.st_mode):
raise ProviderAuthenticationError("PROVIDER_STORE_SYMLINK_DENIED")
_validate_directory_stat(st, policy)
target = current / parts[-1]
try:
st = os.lstat(target)
except OSError as exc:
raise ProviderAuthenticationError("PROVIDER_RECEIPT_UNAVAILABLE") from exc
if stat.S_ISLNK(st.st_mode):
raise ProviderAuthenticationError("PROVIDER_RECEIPT_SYMLINK_DENIED")
_validate_receipt_stat(st, policy)
flags = os.O_RDONLY | getattr(os, "O_CLOEXEC", 0) | getattr(os, "O_BINARY", 0)
if hasattr(os, "O_NOFOLLOW"):
flags |= os.O_NOFOLLOW
try:
fd = os.open(target, flags)
except OSError as exc:
raise ProviderAuthenticationError("PROVIDER_RECEIPT_OPEN_DENIED") from exc
try:
opened = os.fstat(fd)
_validate_receipt_stat(opened, policy)
if (opened.st_dev, opened.st_ino) != (st.st_dev, st.st_ino):
raise ProviderAuthenticationError("PROVIDER_RECEIPT_RACE_DETECTED")
chunks: list[bytes] = []
total = 0
while True:
chunk = os.read(fd, min(65536, policy.max_receipt_bytes + 1 - total))
if not chunk:
break
total += len(chunk)
if total > policy.max_receipt_bytes:
raise ProviderAuthenticationError("PROVIDER_RECEIPT_TOO_LARGE")
chunks.append(chunk)
return b"".join(chunks)
finally:
os.close(fd)
def _authentication_payload(receipt: Mapping[str, Any]) -> bytes:
clone = dict(receipt)
authentication = dict(clone.get("authentication") or {})
authentication.pop("mac_hex", None)
clone["authentication"] = authentication
return canonical_json_bytes(clone)
class TrustedProviderReceiptResolver:
"""Resolve and authenticate evidence-provider receipts from governed stores."""
def __init__(
self,
policies: Mapping[str, TrustedProviderPolicy],
*,
hmac_key_resolver: Callable[[str], bytes] | None = None,
now_fn: Callable[[], datetime] | None = None,
) -> None:
self._policies = dict(policies)
self._hmac_key_resolver = hmac_key_resolver
self._now_fn = now_fn or (lambda: datetime.now(timezone.utc))
def resolve(
self,
evidence_kind: str,
receipt_ref: str,
*,
source_oid: str,
tree_oid: str,
) -> AuthenticatedProviderReceipt:
policy = self._policies.get(evidence_kind)
if policy is None:
raise ProviderAuthenticationError("PROVIDER_POLICY_NOT_FOUND")
if not _HEX40.fullmatch(source_oid):
raise ProviderAuthenticationError("PROVIDER_SOURCE_OID_INVALID")
if not _HEX40.fullmatch(tree_oid):
raise ProviderAuthenticationError("PROVIDER_TREE_OID_INVALID")
parts = _validate_ref(receipt_ref)
raw = _read_nofollow(policy.store_root, parts, policy)
file_sha = hashlib.sha256(raw).hexdigest()
try:
receipt = json.loads(raw.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ProviderAuthenticationError("PROVIDER_RECEIPT_JSON_INVALID") from exc
if not isinstance(receipt, dict):
raise ProviderAuthenticationError("PROVIDER_RECEIPT_JSON_INVALID")
if receipt.get("schema") != PROVIDER_RECEIPT_SCHEMA:
raise ProviderAuthenticationError("PROVIDER_RECEIPT_SCHEMA_INVALID")
if receipt.get("version") != PROVIDER_RECEIPT_VERSION:
raise ProviderAuthenticationError("PROVIDER_RECEIPT_VERSION_INVALID")
if receipt.get("provider_identity") != policy.provider_identity:
raise ProviderAuthenticationError("PROVIDER_IDENTITY_MISMATCH")
operation = receipt.get("provider_operation")
if operation not in policy.allowed_operations:
raise ProviderAuthenticationError("PROVIDER_OPERATION_DENIED")
run_id = receipt.get("operation_run_id")
result_reference = receipt.get("result_reference")
environment = receipt.get("environment")
issued_at = receipt.get("issued_at")
if not all(isinstance(v, str) and v.strip() for v in (run_id, result_reference, environment, issued_at)):
raise ProviderAuthenticationError("PROVIDER_RECEIPT_REQUIRED_FIELD_INVALID")
if receipt.get("source_oid") != source_oid:
raise ProviderAuthenticationError("PROVIDER_SOURCE_OID_MISMATCH")
if receipt.get("tree_oid") != tree_oid:
raise ProviderAuthenticationError("PROVIDER_TREE_OID_MISMATCH")
result = receipt.get("result")
if not isinstance(result, dict):
raise ProviderAuthenticationError("PROVIDER_RESULT_INVALID")
content_sha = receipt.get("content_sha256")
if not isinstance(content_sha, str) or not _HEX64.fullmatch(content_sha):
raise ProviderAuthenticationError("PROVIDER_CONTENT_SHA256_INVALID")
if canonical_sha256(result) != content_sha:
raise ProviderAuthenticationError("PROVIDER_CONTENT_SHA256_MISMATCH")
issued = _parse_issued_at(issued_at)
now = self._now_fn().astimezone(timezone.utc)
if issued > now and (issued - now).total_seconds() > policy.max_future_skew_seconds:
raise ProviderAuthenticationError("PROVIDER_RECEIPT_FROM_FUTURE")
if policy.max_age_seconds is not None and (now - issued).total_seconds() > policy.max_age_seconds:
raise ProviderAuthenticationError("PROVIDER_RECEIPT_EXPIRED")
authentication = receipt.get("authentication")
if not isinstance(authentication, dict):
raise ProviderAuthenticationError("PROVIDER_AUTHENTICATION_MISSING")
kind = authentication.get("kind")
if kind != policy.authentication_kind:
raise ProviderAuthenticationError("PROVIDER_AUTHENTICATION_KIND_MISMATCH")
if kind == "root-owned-store-v1":
if authentication.get("store_id") != policy.store_id:
raise ProviderAuthenticationError("PROVIDER_STORE_ID_MISMATCH")
elif kind == "hmac-sha256-v1":
key_id = authentication.get("key_id")
mac_hex = authentication.get("mac_hex")
if policy.hmac_key_id is None or key_id != policy.hmac_key_id:
raise ProviderAuthenticationError("PROVIDER_HMAC_KEY_ID_MISMATCH")
if not isinstance(mac_hex, str) or not _HEX64.fullmatch(mac_hex):
raise ProviderAuthenticationError("PROVIDER_HMAC_INVALID")
if self._hmac_key_resolver is None:
raise ProviderAuthenticationError("PROVIDER_HMAC_KEY_RESOLVER_REQUIRED")
key = self._hmac_key_resolver(key_id)
if not isinstance(key, (bytes, bytearray)) or not key:
raise ProviderAuthenticationError("PROVIDER_HMAC_KEY_UNAVAILABLE")
namespace = policy.hmac_namespace.encode("utf-8")
expected = hmac.new(bytes(key), namespace + b"\0" + _authentication_payload(receipt), hashlib.sha256).hexdigest()
if not hmac.compare_digest(mac_hex, expected):
raise ProviderAuthenticationError("PROVIDER_HMAC_MISMATCH")
else:
raise ProviderAuthenticationError("PROVIDER_AUTHENTICATION_KIND_UNSUPPORTED")
return AuthenticatedProviderReceipt(
evidence_kind=evidence_kind,
provider_identity=policy.provider_identity,
provider_operation=operation,
operation_run_id=run_id,
result_reference=result_reference,
result=result,
content_sha256=content_sha,
source_oid=source_oid,
tree_oid=tree_oid,
environment=environment,
issued_at=issued_at,
authentication=dict(authentication),
receipt_ref="/".join(parts),
receipt_file_sha256=file_sha,
)
def resolve_evidence(
self,
evidence_kind: str,
evidence_request: Mapping[str, Any],
*,
source_oid: str,
tree_oid: str,
) -> ResolvedEvidence:
if not isinstance(evidence_request, Mapping):
raise ProviderAuthenticationError("EVIDENCE_REQUEST_INVALID")
if "receipt" in evidence_request or "provider_receipt" in evidence_request:
raise ProviderAuthenticationError("CALLER_SUPPLIED_PROVIDER_RECEIPT_DENIED")
receipt_ref = evidence_request.get("receipt_ref")
if not isinstance(receipt_ref, str) or not receipt_ref.strip():
raise ProviderAuthenticationError("PROVIDER_RECEIPT_REF_REQUIRED")
authenticated = self.resolve(
evidence_kind,
receipt_ref,
source_oid=source_oid,
tree_oid=tree_oid,
)
# The authenticated receipt is authoritative for the evidence result.
# Request-side result fields may only match; they can never override.
request_body = {k: v for k, v in evidence_request.items() if k != "receipt_ref"}
if request_body and canonical_sha256(request_body) != authenticated.content_sha256:
raise ProviderAuthenticationError("EVIDENCE_REQUEST_CONTENT_MISMATCH")
return ResolvedEvidence(
result=dict(authenticated.result),
provider_receipt=authenticated.provenance(),
)
def build_hmac_authentication(
receipt_without_mac: Mapping[str, Any],
*,
key_id: str,
key: bytes,
namespace: str = "aios.trusted-evidence-provider-receipt.v2",
) -> dict[str, Any]:
if not key:
raise ValueError("key is required")
draft = dict(receipt_without_mac)
auth = dict(draft.get("authentication") or {})
auth["kind"] = "hmac-sha256-v1"
auth["key_id"] = key_id
auth.pop("mac_hex", None)
draft["authentication"] = auth
mac = hmac.new(key, namespace.encode("utf-8") + b"\0" + canonical_json_bytes(draft), hashlib.sha256).hexdigest()
return {**auth, "mac_hex": mac}
@@ -1,5 +1,10 @@
INSERT OR IGNORE INTO node_status(node_id, health, active, score, last_seen_at, note) VALUES ('server-hw','ok',1,100,datetime('now'),'primary logical node'),('server-local','ok',0,95,datetime('now'),'backup logical node');
INSERT OR IGNORE INTO routing_policy(role, primary_node, backup_node, active_node, failover_priority, updated_at) VALUES ('owner','server-hw','server-local','server-hw',1,datetime('now')),('admin','server-hw','server-local','server-hw',2,datetime('now')),('user','server-hw','server-local','server-hw',3,datetime('now'));
INSERT OR IGNORE INTO users(id, username, display_name, role, status, password_hash, created_at, updated_at) VALUES ('bootstrap-owner','owner','Bootstrap Owner','owner','approved','bootstrap-change-me',datetime('now'),datetime('now')),('bootstrap-admin','admin','Bootstrap Admin','admin','approved','bootstrap-change-me',datetime('now'),datetime('now')),('sample-user','sample-user','Sample User','user','approved','bootstrap-change-me',datetime('now'),datetime('now'));
-- SEC-P0-BOOTSTRAP-CREDENTIAL hardening: seed NO usable password. Empty hash
-- means "not set / forced reset" -> owner/admin must set a strong password via
-- the bootstrap flow (scripts/patch_runtime_owner_admin_user.py) over a secure channel.
INSERT OR IGNORE INTO users(id, username, display_name, role, status, password_hash, created_at, updated_at) VALUES ('bootstrap-owner','owner','Bootstrap Owner','owner','approved','',datetime('now'),datetime('now')),('bootstrap-admin','admin','Bootstrap Admin','admin','approved','',datetime('now'),datetime('now')),('sample-user','sample-user','Sample User','user','approved','',datetime('now'),datetime('now'));
-- One-time cleanup: neutralize any legacy known plaintext bootstrap seed already present.
UPDATE users SET password_hash='', updated_at=datetime('now') WHERE password_hash='bootstrap-change-me';
INSERT OR IGNORE INTO model_policy(role, default_model, allowed_models, updated_at) VALUES ('owner','deepseekv4-flash','["deepseekv4-flash","qwen3.6-35b","gpt-5.5"]',datetime('now')),('admin','deepseekv4-flash','["deepseekv4-flash","qwen3.6-35b"]',datetime('now')),('user','deepseekv4-flash','["deepseekv4-flash"]',datetime('now'));
INSERT OR IGNORE INTO quota_policy(username, daily_limit, used_today, updated_at) VALUES ('owner',10000,0,datetime('now')),('admin',2000,0,datetime('now')),('sample-user',200,0,datetime('now'));
+32 -4
View File
@@ -55,10 +55,36 @@ def seed_base_models():
if not one('SELECT * FROM gitops_versions WHERE version=?', ('V1R1C0',)):
exec_sql('INSERT INTO gitops_versions(version,tag,level,summary,status,created_by,created_at) VALUES(?,?,?,?,?,?,?)', ('V1R1C0','V1R1C0','R','MultiUser runtime selector + GitOps + custom LLM priority registry','ready','system',now()))
_ROLE_WHITELIST = {'owner', 'admin', 'user'}
def token_identity(token: str):
"""Resolve the server-authoritative identity bound to a token.
Format of each TOKEN_FILE line (one per line, ':'-separated):
<token> -> authenticated principal, role 'user'
<token>:<user>:<role> -> mapped principal; role restricted to owner/admin/user
Identity NEVER comes from a client-supplied header. In local dev mode the
DEV_ALLOW flag maps a fixed dev principal (must be explicitly enabled).
"""
if DEV_ALLOW:
return {'username': 'dev-owner', 'role': 'owner'}
if not token or not TOKEN_FILE.exists():
return None
for line in TOKEN_FILE.read_text().splitlines():
line = line.strip()
if not line:
continue
parts = [x.strip() for x in line.split(':')]
if not parts or parts[0] != token:
continue
if len(parts) >= 3:
role = parts[2] if parts[2] in _ROLE_WHITELIST else 'user'
return {'username': safe_name(parts[1]), 'role': role}
return {'username': 'user-%s' % token[:8], 'role': 'user'}
return None
def valid_token(token: str) -> bool:
if DEV_ALLOW: return True
if not token or not TOKEN_FILE.exists(): return False
return token.strip() in [x.strip() for x in TOKEN_FILE.read_text().splitlines() if x.strip()]
return token_identity(token) is not None
def git_cmd(args, check=False):
WORKSPACE.mkdir(parents=True, exist_ok=True); p = subprocess.run(['git'] + args, cwd=str(WORKSPACE), text=True, capture_output=True)
if check and p.returncode != 0: raise RuntimeError(p.stderr or p.stdout or 'git command failed')
@@ -88,8 +114,10 @@ class Handler(BaseHTTPRequestHandler):
server_version = 'AIOSGitOpsModelAPI/V1R1C0'
def actor(self):
auth = self.headers.get('Authorization',''); token = auth.replace('Bearer ', '').strip() if auth.startswith('Bearer ') else self.headers.get('X-AIOS-Token','')
ok = valid_token(token); user = self.headers.get('X-AIOS-User') or ('dev-owner' if DEV_ALLOW else 'anonymous'); role = self.headers.get('X-AIOS-Role') or ('owner' if DEV_ALLOW else 'user')
return {'username': safe_name(user), 'role': role, 'token_ok': ok}
ident = token_identity(token)
if ident is None:
return {'username': 'anonymous', 'role': 'user', 'token_ok': False}
return {'username': ident['username'], 'role': ident['role'], 'token_ok': True}
def require_write(self, roles=None):
a = self.actor()
if not a['token_ok']: self.send_json({'error':'unauthorized','hint':'write APIs require Bearer token or AIOS_GITOPS_DEV_ALLOW=1'}, 401); return None
+14 -29
View File
@@ -1,8 +1,8 @@
<<<<<<< HEAD
# AIOS Agent Command Plane Registry
# Git is the single source of truth for all agent commands.
version: 1.0.0
created_at: 2026-07-08T15:20:00+08:00
registry_version: 1.0.0
created_at: 2026-07-08T15:44:00+08:00
baseline: V1R3C1
commands:
@@ -12,6 +12,12 @@ commands:
completed_path: "agents/commands/completed/"
failed_path: "agents/commands/failed/"
directories:
commands/active: "New commands pending execution — Git is the only entry point"
commands/running: "Commands currently being executed by an agent"
commands/completed: "Commands that finished successfully — moved here on success"
commands/failed: "Commands that finished with errors — moved here on failure"
policies:
path: "agents/policies/"
source: "git"
@@ -31,6 +37,12 @@ agents:
type: reviewer
role: review
lifecycle:
new: active/
running: running/
success: completed/
failure: failed/
# Audit plane configuration
audit:
enabled: true
@@ -38,30 +50,3 @@ audit:
agent_runs: "audit/agent-runs/"
version_history: "audit/version-history/"
command_history: "audit/command-history/"
=======
# AIOS Agent Command Registry
# Git is the single source of truth for all agent commands.
registry_version: 1.0.0
created_at: "2026-07-08T15:44:00+08:00"
baseline: V1R3C1
directories:
commands/active: "New commands pending execution — Git is the only entry point"
commands/running: "Commands currently being executed by an agent"
commands/completed: "Commands that finished successfully — moved here on success"
commands/failed: "Commands that finished with errors — moved here on failure"
state: agents/state/
policies: agents/policies/
lifecycle:
new: active/
running: running/
success: completed/
failure: failed/
# Command lifecycle state machine:
# active/ → locked & moved to running/ → execute → success → completed/
# → failure → failed/
>>>>>>> 4d18ee0 (feat(cmd): enable Agent Command Plane, audit plane, and Web UI dashboard)
@@ -0,0 +1,68 @@
[Unit]
Description=AIOS GitOps Model API V2R2C2 Baseline1.1 isolated candidate (2852c2a) - control-plane bootstrap only
After=network-online.target
Wants=network-online.target
Requires=aios-trusted-candidate-builder.socket
[Service]
Type=simple
User=aiman
Group=gitops
WorkingDirectory=/opt/aios/gitops-model-api/2852c2a40394195192c6c02a215ba31764e2cdda/runtime
UMask=0077
Environment=PYTHONUNBUFFERED=1
Environment=PYTHONDONTWRITEBYTECODE=1
Environment=AIOS_GITOPS_ROOT=/var/lib/aios-gitops
Environment=AIOS_GITOPS_WORKSPACE=/opt/aios/gitops-model-api/2852c2a40394195192c6c02a215ba31764e2cdda/runtime
Environment=AIOS_GITOPS_REPO=/data/aios-gitops/repo.git
Environment=AIOS_GITOPS_BARE_REPO=/data/aios-gitops/repo.git
Environment=AIOS_GITOPS_AUTHORITATIVE_GIT_DIR=/data/aios-gitops/repo.git
Environment=AIOS_GITOPS_REPOSITORY_IDENTITY=aios/aios
Environment=AIOS_GITOPS_BASELINE_AUTHORITY=/var/lib/aios-gitops-runtime-anchor/baseline/current.json
Environment=AIOS_GITOPS_DURABLE_JOB_EXECUTION_ENABLED=false
Environment=AIOS_GITOPS_JOB_CONTROLLER_ENABLED=false
Environment=AIOS_GITOPS_WORKSPACE_ROLE=runtime-anchor
Environment=AIOS_GITOPS_RUNTIME_OID=2852c2a40394195192c6c02a215ba31764e2cdda
Environment=AIOS_GITOPS_ARTIFACT=/opt/aios/gitops-model-api/2852c2a40394195192c6c02a215ba31764e2cdda/runtime/governed-runtime-package.json
Environment=AIOS_GITOPS_ARTIFACT_SHA256=932431dbbdee55522342d9c94457f40d8f6f7e4fc7eaadc71c66d0a6bdf8472f
Environment=AIOS_GITOPS_DEPLOYMENT_STATE=CANDIDATE_ISOLATED
Environment=AIOS_GITOPS_MODEL_API_SERVICE=aios-gitops-model-api-v2r2c2-baseline11-candidate.service
Environment=AIOS_RUNTIME_CANDIDATE_TREE_OID=ccd67c2357e7a4cd74e06be61ab81bba820d7671
Environment=AIOS_RUNTIME_PROVENANCE_COMMIT=2852c2a40394195192c6c02a215ba31764e2cdda
Environment=AIOS_AUTHORITY_SOURCE_PUBLISHED=true
Environment=AIOS_GITOPS_WRITES_ENABLED=false
Environment=AIOS_GITOPS_LEGACY_GLOBAL_WRITES_ENABLED=false
Environment=AIOS_SILENT_RELEASE_EXECUTION_ENABLED=false
Environment=AIOS_SILENT_RELEASE_STATE_ROOT=/var/lib/aios-gitops/model-api-v2r2c2-baseline11-candidate/silent-release
Environment=AIOS_BRIDGE_V2_URL=http://127.0.0.1:18892
Environment=AIOS_GITOPS_HOST=127.0.0.1
Environment=AIOS_GITOPS_PORT=18897
Environment=PYTHONPATH=/opt/aios/gitops-model-api/2852c2a40394195192c6c02a215ba31764e2cdda/runtime:/opt/aios/gitops-model-api/2852c2a40394195192c6c02a215ba31764e2cdda/runtime/app:/opt/aios/gitops-model-api/2852c2a40394195192c6c02a215ba31764e2cdda/runtime/runtime-agent-os
Environment=AIOS_TRUSTED_CANDIDATE_BUILDER_ENABLED=true
Environment=AIOS_TRUSTED_CANDIDATE_BUILDER_SOCKET=/run/aios-trusted-candidate-builder.sock
Environment=AIOS_TRUSTED_CANDIDATE_BUILDER_POLICY=/opt/aios/gitops-model-api/2852c2a40394195192c6c02a215ba31764e2cdda/runtime/config/gitops/trusted-candidate-builder-policy.json
Environment=AIOS_TRUSTED_CANDIDATE_BUILDER_PUBLIC_KEYRING=%d/builder_public_keyring
Environment=AIOS_TRUSTED_CANDIDATE_BUILDER_CREDENTIALS_DIRECTORY=%d
Environment=AIOS_TRUSTED_BUILDER_SOURCE_ADMISSION_ROOT=/var/lib/aios-release-source-admission
Environment=AIOS_SILENT_RELEASE_TRUSTED_POLICY=/opt/aios/gitops-model-api/2852c2a40394195192c6c02a215ba31764e2cdda/runtime/config/gitops/trusted-release-policy.json
LoadCredentialEncrypted=builder_hmac:/etc/credstore.encrypted/aios-trusted-candidate-builder/builder_hmac
LoadCredentialEncrypted=source_admission_hmac:/etc/credstore.encrypted/aios-release-source-admission/source_admission_hmac
LoadCredential=builder_public_keyring:/etc/aios-gitops/trusted-builder/public-keyring.json
ExecStart=/usr/bin/python3 -B /opt/aios/gitops-model-api/2852c2a40394195192c6c02a215ba31764e2cdda/runtime/app/aios_model_api_staging.py
Restart=on-failure
RestartSec=2
NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
LockPersonality=true
RestrictSUIDSGID=true
RestrictAddressFamilies=AF_UNIX AF_INET
ReadOnlyPaths=/opt/aios/gitops-model-api/2852c2a40394195192c6c02a215ba31764e2cdda/runtime /data/aios-gitops/repo.git /etc/aios-gitops/trusted-builder/public-keyring.json /var/lib/aios-release-source-admission /var/lib/aios-release-migration
ReadWritePaths=/var/lib/aios-gitops/model-api-v2r2c2-baseline11-candidate
[Install]
WantedBy=multi-user.target
@@ -0,0 +1 @@
a36e9ef667f0e6be3a9e77b2082c7f2b1b2b0e020af9dcd38af8c8083c28acac aios-round3-provider-auth-c6fe1a486fa0b2fe560cbdfea646116bcac637a0.tar.gz
@@ -0,0 +1,52 @@
{
"package": "aios-round3-provider-auth",
"schema": "aios.immutable-package-ledger.v1",
"generated_at": "2026-09-09T07:15:00Z",
"platform": "linux-amd64",
"freeze": {
"PREVIOUS_FREEZE_HEAD": "a6dc0c3a63836a3668d972f632131f772626c363",
"PROVIDER_AUTH_HEAD": "f7f53d0568767d6deec2c47008a89f4d8413f93d",
"PROVIDER_EXECUTOR_HEAD": "2d64d5a8a1dd63f77b1e5264037aac2a183c5d3d",
"P0_FINAL_HEAD": "c6fe1a486fa0b2fe560cbdfea646116bcac637a0",
"P0_FINAL_TREE": "2e36ccf170b59ac38f62c6c20a5259b3fe9a181b"
},
"receipt_binding": {
"source_oid": "2d64d5a8a1dd63f77b1e5264037aac2a183c5d3d",
"tree_oid": "505a66a79062e1161b69d93063ffeb94ad1b95b9"
},
"gate_w": {
"produced": ["tests", "secret_scan"],
"mechanism_required": ["gitea"],
"resolver_closure": "PASS",
"tests_passed": false,
"tests_failures": 3,
"tests_note": "3 pre-existing non-round3 env failures (V2R1C6 digest, rollback script dep, missing deployed signer binary)",
"secret_scan_passed": true
},
"providers": {
"tests": {
"provider_identity": "aios.tests-runner",
"gate": "W",
"receipt_ref": "round3-gate-w/tests.json",
"store": "/var/lib/aios-evidence-providers/tests"
},
"secret_scan": {
"provider_identity": "aios.secret-scanner",
"gate": "W",
"receipt_ref": "round3-gate-w/secret_scan.json",
"store": "/var/lib/aios-evidence-providers/secret-scan"
},
"gitea": {
"provider_identity": "aios.gitea-machine-transport",
"gate": "W",
"status": "PROVIDER_MECHANISM_REQUIRED",
"note": "API URL / token / repository identity not provisioned; fail-closed, no fake receipt"
}
},
"verification": {
"bundle24_and_affected": "24 passed",
"verify_round3_integration": "ROUND3_PROVIDER_AUTH_INTEGRATION=PASS",
"no_manufactured_passed": true,
"no_acceptance_invented_identity": true
}
}
@@ -0,0 +1,65 @@
{
"schema": "aios.immutable-package-tariff.v1",
"package": "aios-round3-provider-auth",
"from": "a6dc0c3a63836a3668d972f632131f772626c363",
"to": "c6fe1a486fa0b2fe560cbdfea646116bcac637a0",
"summary": {
"files": 13,
"insertions": 2233,
"deletions": 27
},
"changes": [
{
"path": "active-runtime/app/governed_authenticated_evidence_operation.py",
"status": "A"
},
{
"path": "active-runtime/app/governed_provider_auth_adapter.py",
"status": "A"
},
{
"path": "active-runtime/app/trusted_evidence_providers.py",
"status": "A"
},
{
"path": "active-runtime/app/trusted_provider_receipts.py",
"status": "A"
},
{
"path": "config/gitops/trusted-evidence-providers.json",
"status": "A"
},
{
"path": "scripts/round3_provider_gatew_prove.py",
"status": "A"
},
{
"path": "tests/test_governed_authenticated_evidence_operation.py",
"status": "A"
},
{
"path": "tests/test_governed_provider_auth_adapter.py",
"status": "A"
},
{
"path": "tests/test_trusted_provider_receipts.py",
"status": "A"
},
{
"path": "verify_round3_integration.py",
"status": "A"
},
{
"path": "active-runtime/app/governed_evidence_runtime.py",
"status": "M"
},
{
"path": "active-runtime/app/governed_release_evidence_producer.py",
"status": "M"
},
{
"path": "docs/V2R2C2-P0-convergence-20260909.md",
"status": "M"
}
]
}
@@ -0,0 +1 @@
{"activation_file":"/var/lib/aios-gitops/bridge-v2-staging/task-admission-activation.json","current_components":[{"artifact_sha_env":"AIOS_GITOPS_ARTIFACT_SHA256","code_line_id":"legacy-governed-main-v15","component_id":"gitops-model-api-staging","expected_source_oid":"91f104ef62a3b688a9259b96e2ca247e0363b564","interface_files":["app/aios_model_api_staging.py","app/silent_release_hardening.py"],"marker_files":["app/aios_model_api_staging.py"],"service":"aios-gitops-model-api-v2r2c1-staging.service","source_oid_env":"AIOS_GITOPS_RUNTIME_OID"},{"artifact_sha_env":"AIOS_CONTROL_PLANE_ARTIFACT_SHA256","code_line_id":"legacy-p0-task-principal","component_id":"chatgpt-bridge-v2-staging","expected_source_oid":"91f104ef62a3b688a9259b96e2ca247e0363b564","interface_files":["app/aios_chatgpt_bridge_v2_server.py","app/aios_task_principal_admission.py"],"marker_files":["app/aios_task_principal_admission.py"],"service":"aios-chatgpt-bridge-v2-staging.service","source_oid_env":"AIOS_CONTROL_PLANE_TARGET_OID"}],"profile_id":"baseline11-principal-recovery-cross-code-line-v1","repository":"aios/aios","rollback_window_seconds":3600,"schema":"aios.release-migration-profile.v1","state_db":"/var/lib/aios-gitops/bridge-v2-staging/task-admission.sqlite3","state_mode":"REUSE_IN_PLACE_WITH_SNAPSHOT","state_ref":"state:task-admission","target_components":[{"code_line_id":"baseline11-release-source-admission","component_id":"gitops-model-api-staging","interface_files":["active-runtime/app/aios_model_api_staging.py","active-runtime/app/silent_release_hardening.py","active-runtime/app/aios_trusted_candidate_builder.py","active-runtime/app/release_migration_plan.py"],"service":"aios-gitops-model-api-v2r2c1-staging.service"},{"code_line_id":"baseline11-task-admission","component_id":"chatgpt-bridge-v2-staging","interface_files":["active-runtime/app/aios_chatgpt_bridge_v2_server.py","active-runtime/app/aios_task_admission_bootstrap.py","active-runtime/app/aios_task_admission_live_adapters.py"],"service":"aios-chatgpt-bridge-v2-staging.service"}],"version":"V2R2C2-baseline11-principal-recovery"}
@@ -0,0 +1,47 @@
{
"schema": "aios.trusted-evidence-providers.v1",
"defaults": {
"authentication_kind": "root-owned-store-v1",
"expected_owner_uid": 0,
"allowed_directory_modes": ["0700", "0750"],
"allowed_receipt_modes": ["0600", "0640"],
"max_age_seconds": 3600
},
"providers": {
"tests": {
"provider_identity": "aios.tests-runner",
"store_id": "tests",
"store_root": "/var/lib/aios-evidence-providers/tests",
"allowed_operations": ["tests.run"],
"gate": "W"
},
"gitea": {
"provider_identity": "aios.gitea-machine-transport",
"store_id": "gitea",
"store_root": "/var/lib/aios-evidence-providers/gitea",
"allowed_operations": ["gitea.consistency"],
"gate": "W"
},
"secret_scan": {
"provider_identity": "aios.secret-scanner",
"store_id": "secret_scan",
"store_root": "/var/lib/aios-evidence-providers/secret-scan",
"allowed_operations": ["secret.scan"],
"gate": "W"
},
"canary": {
"provider_identity": "aios.canary-gate",
"store_id": "canary",
"store_root": "/var/lib/aios-evidence-providers/canary",
"allowed_operations": ["canary.run"],
"gate": "R"
},
"rollback": {
"provider_identity": "aios.rollback-controller",
"store_id": "rollback",
"store_root": "/var/lib/aios-evidence-providers/rollback",
"allowed_operations": ["rollback.validate"],
"gate": "R"
}
}
}
@@ -0,0 +1,28 @@
{
"schema": "aios.unified-platform-program-contract.v1",
"goal_slug": "AIOS-UNIFIED-PLATFORM-CONVERGENCE",
"authority_base": "beabd053961047f170e87e1110f2636dbd50f5fb",
"existing_identity_lineage_dependency": {
"mission_id": "mission-f3af205b-7dd6-455f-a561-88f0294cc815",
"root_task_id": "task-133ad96e-e0d7-4202-934f-80c75053b11f",
"acceptance_mutation_allowed": false
},
"tracks": {
"A": "IDENTITY_LINEAGE_EXISTING_MISSION",
"B": "ANNEX_2_3_R2_AUTO_CLAIM",
"C": "PLATFORM_GENERATION_G0",
"D": "GOVERNED_MERGE_STAGING_RELEASE"
},
"program_acceptance": [
"U01",
"U02",
"U03",
"U04",
"U05",
"U06",
"U07",
"U08"
],
"deployment_requires_review_go": true,
"new_business_authority_created": false
}
+1
View File
@@ -0,0 +1 @@
{"bindings":[{"acceptance_key":"C01","acceptance_position":0,"lineage_request_digest":"e9946481e7c9fe8dee94aa42ffa0a9b41546ce9d46601ff221a958710c72f6cc","mission_id":"mission-f3af205b-7dd6-455f-a561-88f0294cc815","owner_principal_id":"local-owner-authority","parent_task_id":"task-133ad96e-e0d7-4202-934f-80c75053b11f","root_acceptance_ref":"mission-acceptance:mission-f3af205b-7dd6-455f-a561-88f0294cc815:identity-lineage-live-closure-v1","root_task_id":"task-133ad96e-e0d7-4202-934f-80c75053b11f","route":"GOVERNED_GIT","schema":"aios.autonomous-child-binding.v1","sibling_sequence":1,"task_id":"task-1668131d-7484-4c8e-83b4-2cf9927842fe","task_plan_digest":"3f0824f51d824de7d38821205c012c2a23ac11e9f5889b8b0aafb07cce85333f"},{"acceptance_key":"C02","acceptance_position":1,"lineage_request_digest":"84f28d350ef3a2dce47fb5f3163b61bfd329ce5fe85bd59c886d4813e806b602","mission_id":"mission-f3af205b-7dd6-455f-a561-88f0294cc815","owner_principal_id":"local-owner-authority","parent_task_id":"task-133ad96e-e0d7-4202-934f-80c75053b11f","root_acceptance_ref":"mission-acceptance:mission-f3af205b-7dd6-455f-a561-88f0294cc815:identity-lineage-live-closure-v1","root_task_id":"task-133ad96e-e0d7-4202-934f-80c75053b11f","route":"SEMANTIC_IMPORT","schema":"aios.autonomous-child-binding.v1","sibling_sequence":2,"task_id":"task-24fe3d9a-783e-4a68-bf9c-5b1a9d2b56c7","task_plan_digest":"e01680583b8cc2ec3d33026dda9f7d095692d0ae79fc43f380066511566e305e"},{"acceptance_key":"C03","acceptance_position":2,"lineage_request_digest":"148fa8b55224649ac80f4d9d5c344019dfef24b1d5ed614ef62ee71ccaf8a6d3","mission_id":"mission-f3af205b-7dd6-455f-a561-88f0294cc815","owner_principal_id":"local-owner-authority","parent_task_id":"task-133ad96e-e0d7-4202-934f-80c75053b11f","root_acceptance_ref":"mission-acceptance:mission-f3af205b-7dd6-455f-a561-88f0294cc815:identity-lineage-live-closure-v1","root_task_id":"task-133ad96e-e0d7-4202-934f-80c75053b11f","route":null,"schema":"aios.autonomous-child-binding.v1","sibling_sequence":3,"task_id":"task-78e0dff5-c1fe-491a-99d2-81ad5cf180da","task_plan_digest":"aef0c0cfee2686ad7bff122328408c5094370bf29fac37a49ddc50173fa60986"},{"acceptance_key":"C04","acceptance_position":3,"lineage_request_digest":"927dde8068b9dde4d57e0f9eb6bd52d95d761533974a7437d01707ea3c5c807d","mission_id":"mission-f3af205b-7dd6-455f-a561-88f0294cc815","owner_principal_id":"local-owner-authority","parent_task_id":"task-133ad96e-e0d7-4202-934f-80c75053b11f","root_acceptance_ref":"mission-acceptance:mission-f3af205b-7dd6-455f-a561-88f0294cc815:identity-lineage-live-closure-v1","root_task_id":"task-133ad96e-e0d7-4202-934f-80c75053b11f","route":null,"schema":"aios.autonomous-child-binding.v1","sibling_sequence":4,"task_id":"task-9b6e0cb1-c54a-4f91-8762-7c72112cf71d","task_plan_digest":"d1eae4e90a5892dd851fbe80c2ee0920e7060cd6ce701ae0522d110dcd9af638"},{"acceptance_key":"C05","acceptance_position":4,"lineage_request_digest":"4336de284be15d504af661c18a98c820da3c3518ef5df25d4607c19c3ba724b2","mission_id":"mission-f3af205b-7dd6-455f-a561-88f0294cc815","owner_principal_id":"local-owner-authority","parent_task_id":"task-133ad96e-e0d7-4202-934f-80c75053b11f","root_acceptance_ref":"mission-acceptance:mission-f3af205b-7dd6-455f-a561-88f0294cc815:identity-lineage-live-closure-v1","root_task_id":"task-133ad96e-e0d7-4202-934f-80c75053b11f","route":null,"schema":"aios.autonomous-child-binding.v1","sibling_sequence":5,"task_id":"task-bd8c44c5-7081-4a3f-a3b0-4de4ad6e3b29","task_plan_digest":"dd7e02ef51f601210295c9bda1061dcd7ff944bc7a5dcb2e813db2d601b138a4"},{"acceptance_key":"C06","acceptance_position":5,"lineage_request_digest":"54bdc5cf4414bc52105e966072009c9b0ce8235392f767b38e856072dd01148c","mission_id":"mission-f3af205b-7dd6-455f-a561-88f0294cc815","owner_principal_id":"local-owner-authority","parent_task_id":"task-133ad96e-e0d7-4202-934f-80c75053b11f","root_acceptance_ref":"mission-acceptance:mission-f3af205b-7dd6-455f-a561-88f0294cc815:identity-lineage-live-closure-v1","root_task_id":"task-133ad96e-e0d7-4202-934f-80c75053b11f","route":null,"schema":"aios.autonomous-child-binding.v1","sibling_sequence":6,"task_id":"task-591a87fa-e722-4647-a71d-80624b7e6b6a","task_plan_digest":"9b0d092f6a0a987844edae129eaedfc6bed87c135d8d3db390b0f01b43f896c8"}],"policy_version":"identity-lineage-autonomous-next-child-v1","schema":"aios.autonomous-child-binding-set.v1"}
@@ -0,0 +1,96 @@
# G7 — Security-Critical Closure 证据与判定
- **Gate**: G7_SECURITY_CRITICAL_CLOSURE
- **判定**: `G7_CHANGES_REQUIRED`(修复已就绪,等待上游治理合入 + staging_readback + ROOT_REVIEW 后升为 APPROVED_FOR_GOVERNED_SUBMISSION)
- **Authority HEAD (base OID)**: `beabd053961047f170e87e1110f2636dbd50f5fb`
- **工作分支**: `agent/g7-security-critical-closure-20260914`(基于 origin/main;本地修复分支,待 PR + Owner 合入 main,不绕过保护 main)
- **日期**: 2026-09-14
- **扫描源**: `aios-scan-full.json` (1.08MB, 1039 findings, 高召回含误报)
- **规模**: 729 文件 / 24 个指派文件修改 / 5 个新增负面测试套件
## 摘要
全仓 review 为高召回设计并含误报,故未机械修复全部 1039 条发现。本次仅闭环被
权威源码复核为 CONFIRMED 且直接影响控制面授权、凭证安全、任意文件访问、特权 UI
的 **P0 阻塞项(6 项)+ P1 交付正确性包(若干)**。DEAD_PATH/FALSE_POSITIVE 项已
按契约排除,并记录依据。
## 修复分账(S1–S5)
| 阻塞项 | 角色 | 分类 | 文件 | pre→post SHA | 修复要点 |
|---|---|---|---|---|---|
| SEC-P0-AUTH-ROLE-HEADER | S1 | CONFIRMED | adapters/gitops/aios_gitops_model_api.py | 60396e0→8990317 | role/user 由服务端 token→identity 映射派生,忽略 X-AIOS-User/Role 头 |
| SEC-P0-AUTH-ROLE-HEADER | S1 | CONFIRMED | active-runtime/app/aios_gitops_model_api.py | ac8c1bf→80b04bd | 同上 |
| SEC-P0-TASK-ACTION-ROLE | S1 | CONFIRMED | active-runtime/app/task_action_api.py | b18e0a5→31c6f6e | 忽略 body role,role 由注入认证主体派生,默认 user |
| SEC-P0-OBSERVABILITY-ROLE | S1 | CONFIRMED | active-runtime/app/agent_observability_api.py | f516ef9→6b065fb | 认证 tenant 决定查询范围,客户端 role=owner 被忽略 |
| SEC-P0-BOOTSTRAP-CREDENTIAL | S2 | CONFIRMED | active-runtime/migrations/002_seed_owner_admin_user.sql | 7fff9e5→4cb5a33 | 种子空哈希(强制重置),清除 bootstrap-change-me |
| SEC-P0-BOOTSTRAP-CREDENTIAL | S2 | CONFIRMED | active-runtime/app/aios_webops_app.py | 83387a8→7001142 | PBKDF2 随机盐;SECRET 缺失 fail-closed;拒绝明文回退;会话 HMAC+过期 |
| SEC-P0-BOOTSTRAP-CREDENTIAL | S2 | CONFIRMED | active-runtime/app/aios_v5_webops_app.py | 83387a8→7001142 | 同上 |
| SEC-P0-CONTROL-PLANE-FILE-READ | S3 | CONFIRMED | active-runtime/app/aios_chatgpt_actions_api.py | 47ae2c8→4f345a0 | 命令读取:认证+严格标识+包含校验+1MiB 上限 |
| SEC-P0-CONTROL-PLANE-FILE-READ | S3 | CONFIRMED | active-runtime/app/aios_model_api_staging.py | 1616b37→e94d0dc | 同上 |
| SEC-P1-PATH-CONTAINMENT | S3 | CONFIRMED | services/aios-event-ingress/server.py | c790d41→c4e93cb | ID 白名单+resolve 包含校验 |
| SEC-P1-PATH-CONTAINMENT | S3 | CONFIRMED | active-runtime/app/agent_observability_store.py | 40cc555→4746b3a | execution_id 白名单+包含校验 |
| SEC-P1-PATH-CONTAINMENT | S3 | CONFIRMED | active-runtime/app/aios_campaign_recovery.py | 0d09fd2→06e208f | 证据路径 resolve+realpath 包含校验 |
| SEC-P1-PATH-CONTAINMENT | S3 | CONFIRMED | scripts/gitops/task_worktree.sh | 72d33f4→19ea429 | TASK_ID 正则收紧,拒绝 .. ../ ..\ |
| SEC-P1-PATH-CONTAINMENT | S3 | CONFIRMED | scripts/init_gitops_baseline.sh | 472e60c→48f584e | zip-slip 解包前条目校验 |
| SEC-P1-PATH-CONTAINMENT | S3 | CONFIRMED | ops/gitops-migration/sync_gitea_owner_password.sh | 11573c9→8e85831 | 消除 env 来源注入,argv+引号 heredoc |
| SEC-P0-ADMIN-UI-XSS | S4 | CONFIRMED | active-runtime/app/static/admin-ai-evolution.html | 6da3ce8→05d26ab | textContent 全面转义;修复 JS 语法;用真实 governed 审批端点替代本地 fake Approve |
| SEC-P1-WEB-CONTROL-SURFACE | S4 | CONFIRMED | overlays/.../gitops-dashboard/index.html | 7f459a1→484447b | 客户端 CSRF token 传播;状态变更带 token;branch 服务端校验 |
| SEC-P1-RUNTIME-DELIVERY | S5 | CONFIRMED | agents/registry.yaml | 7b69d96→0037e73 | 解决阻塞性 merge conflict,恢复可解析 YAML |
| SEC-P1-RUNTIME-DELIVERY | S5 | CONFIRMED | active-runtime/app/aios_model_api_staging.py | 1616b37→e94d0dc | 消除 do_POST 遮蔽(合并 bridge 路由) |
| SEC-P1-RUNTIME-DELIVERY | S5 | CONFIRMED | docs/gitops/openapi-chatgpt-actions.yaml | 5278b91→9f339c0 | 补全 securitySchemes.taskPrincipalAuth |
| SEC-P1-RUNTIME-DELIVERY | S5 | CONFIRMED | deploy/install/install.sh | ab1f433→b90b759 | 移除 `\|\| true`,启动失败传播 |
| SEC-P1-RUNTIME-DELIVERY | S5 | CONFIRMED | scripts/silent_install_v5.sh | 59e9552→e26a3b3 | 缺失脚本显式报错 |
| SEC-P1-RUNTIME-DELIVERY | S5 | CONFIRMED | delta-fix/apply_delta_fix_only.sh | de1e2fd→d09d2ab | 修复未定义变量 |
| SEC-P1-RUNTIME-DELIVERY | S5 | **DEAD_PATH** | active-runtime/app/architecture_owner_gate_http.py | 未改 | 全库零引用,非活路径 |
| SEC-P1-RUNTIME-DELIVERY | S5 | **DEAD_PATH** | runtime/agent-os/api.py | 未改 | 全库零引用独立模块,facade 签名不符但无活路径 |
## 负面测试(S6 独立 QA 验证)
新增 5 个测试套件,均针对**真实已修复模块**(经 tests/conftest.py 的 active_runtime 命名空间映射到实际文件):
- `tests/test_g7_auth_negative.py` — 9 passed
- `tests/test_g7_credential_negative.py` — 4 passed
- `tests/test_g7_path_containment_negative.py` — 7 passed(+8 bash 用例如下)
- `tests/test_g7_runtime_delivery.py` — 4 passed
- `tests/test_g7_web_surface_negative.py` — 42 passed
**合计 66 passed / 0 failed**。8 个 bash 子进程用例如 Windows 沙箱 pytest 无法拉起 bash 而 skip,
已在本机 Bash 环境实测真实脚本:遍历 TASK_ID → rc=65 拒绝;zip-slip `../` 条目 → BLOCKED 退出 11。
### 覆盖的负面场景
- spoof `X-AIOS-Role: owner` / `X-AIOS-User` → 不能提权
- JSON `role=owner` → 被忽略
- observability `role=owner` + 外来 user/group → 限定自身
- 非 owner token 下 `require_write(['owner'])` → 403
- 已知 bootstrap 口令拒绝登录;哈希非明文且随机盐;缺 SECRET 拒绝启动;过期令牌失效
- POSIX `../` / Windows `..\` / 编码遍历 / 超大读取 / zip-slip / 绝对路径拒绝
- HTML `<img onerror>` / script 载荷保持文本;CSRF-less 状态变更被拒;无效 branch 被拒
- registry.yaml / openapi 可解析且引用完整;install.sh 失败传播;脚本无未定义变量
### 回归
- 全部 `active-runtime/app` + `adapters` 211 个 py 文件 `py_compile` 通过(无导入/Syntax 回归)
- 全部 15 个改动/新增 py 文件编译通过
## 证据字段映射(G7_SECURITY_GATE.md Required evidence)
对每个阻塞项均已具备:finding_id / authority_oid(=beabd05…) / affected_path / live_path=true /
preimage_sha256(上表)、fix_commit_oid(见下方提交)、postimage_sha256(上表)、
negative_tests(通过)、positive_tests(编译+回归)、independent_review(S6 独立运行通过)、
runtime/staging_readback、rollback(见 S2 迁移/回滚路径)。
## 判定与下一步
- **当前判定**: `G7_CHANGES_REQUIRED` → 修复就绪,待合入流程推进。
- **下一步**(需 Owner/治理,非本 Agent 越权执行):
1. 审查并合入 `agent/g7-security-critical-closure-20260914`(走 protected main PR + Owner approval)
2. staging 部署后作 runtime/staging_readback(真实服务启动验证,尤其 S2 的 AIOS_WEBOPS_SECRET 注入与 S1 的 token 映射文件格式)
3. 执行 S2 owner 引导设密流程(scripts/patch_runtime_owner_admin_user.py)设置真实 owner 强口令
4. 独立 final review → governed submission → staging → live E2E → production → ROOT_REVIEW
- **已知未闭环**(诚实标注): S1 的 task_action/observability 认证主体需宿主 FastAPI 应用注入
`set_authenticator()`,当前默认按最低权限 user(安全降级但 owner 能力需接线);staging_readback
与 rollback 实机验证需在部署环境执行,本环境未具备。
## 备注(Triage 纪律)
- 未机械处理 1000+ 条 medium/low/维护性/样式/历史死代码注释(契约明确非当前发布阻塞项)。
- DEAD_PATH 项未改动并给出全库引用证据。
+1 -1
View File
@@ -15,5 +15,5 @@ bash "$PKG_ROOT/scripts/smoke_test.sh" "$AIOS_RUNTIME_ROOT"
python3 "$PKG_ROOT/scripts/generate_release_manifest.py" "$AIOS_RUNTIME_ROOT"
echo "DELTA_FIX_ONLY_APPLY_OK release=$AIOS_RELEASE_ID runtime=$AIOS_RUNTIME_ROOT"
python3 "$PKG/fixes/patch_runtime_v5_guards.py" "$TARGET"
python3 "$PKG_ROOT/fixes/patch_runtime_v5_guards.py" "$AIOS_RUNTIME_ROOT"
echo "V5_DIGITAL_LIFE_DELTA_APPLIED"
@@ -0,0 +1,17 @@
{
"schema": "aios.activation-checklist.v1",
"profile": "isolated-validation",
"production_activation_supported": false,
"status": "NOT_RUN",
"checks": [
{"id": "identity", "required": "Replace @RELEASE_ROOT@; record source, artifact and configuration digests separately; verify package bytes.", "status": "NOT_RUN"},
{"id": "authority", "required": "Bind an explicitly isolated TaskPlaneStore; record existing online Task/Job/Release write authorities; do not route online admission into this profile.", "status": "NOT_RUN"},
{"id": "delegation", "required": "Use installed trusted Owner public keys and an explicitly authorized signed Grant. Never generate a production trust root to satisfy a missing credential.", "status": "NOT_RUN"},
{"id": "credentials", "required": "Project encrypted credentials directly into each consuming unit; preserve encrypted credential names; check permissions and independent restart/rotation.", "status": "NOT_RUN"},
{"id": "privileges", "required": "Create separate service and worker users; grant only read-only status access to the registered systemd service; no SSH, root shell or global write.", "status": "NOT_RUN"},
{"id": "configuration", "required": "Use loopback server URL, trusted fixed operation/component bindings and unique state directories. Client request bodies may not select tenant or credential paths.", "status": "NOT_RUN"},
{"id": "storage", "required": "Check directory owner, free space, SQLite WAL commits, durable receipts, recovery permissions and restore procedure on the actual storage device.", "status": "NOT_RUN"},
{"id": "nonrelease_loop", "required": "Record a real registered host service diagnostic through Action, Task claim, operation, receipt and result; unhealthy diagnostics remain observable failures of health.", "status": "NOT_RUN"},
{"id": "online_cutover", "required": "Separate later change: bind verified official Task authority adapter, release acceptance verifier, credential identities, approved installation and single-writer evidence before enabling online admission.", "status": "NOT_SUPPORTED_BY_THIS_PROFILE"}
]
}
+20
View File
@@ -0,0 +1,20 @@
# Validation 实例的凭证边界
本目录的 `.service.example` 是候选验证实例模板。它们不自动安装,不替换现有 Task 权威,也不支持 production profile。先按 `docs/implementation-baseline-1.1/INSTALLATION.md` 完成离线验证,并由安装者将 `@RELEASE_ROOT@` 替换为确定的、只读的发行目录,并将 `@PYTHON_EXECUTABLE@` 绑定到已验收的独立虚拟环境 Python 绝对路径。
服务端账户为 `aios-baseline`;Worker 账户为 `aios-baseline-worker`。Worker 不获得机器签名私钥、Owner 私钥、服务端 token 清单或服务端数据库读取权。Worker 的 transport token 仅映射到已注册 Worker 身份;它不能自行声明租户、用户或权限。
| 消费者 | 凭证名 | 用途 |
| --- | --- | --- |
| validation API | `aios-baseline-machine-key` | 已安装的短期执行证明和 operation receipt 签名身份 |
| validation API | `aios-baseline-user-token` | 验证配置中一位用户的 transport token;实际多用户要逐个绑定可信身份 |
| validation API | `aios-baseline-worker-token` | 验证 Worker 的 transport token |
| validation Worker | `aios-baseline-worker-token` | 仅用于连接本机 API |
每个 unit 使用自己的 `LoadCredentialEncrypted=` 从稳定的 `/etc/credstore.encrypted/` 来源投影。两个服务可以获得同一个受治理 token 的独立投影,但任何服务都不读取另一服务的 `/run/credentials/<unit>/`。加密凭证名称与 `LoadCredentialEncrypted` 左侧名称必须一致,避免更名导致解密身份校验失败。
JSON 中仅允许以 `${CREDENTIALS_DIRECTORY}/<name>` 引用自己的投影目录;不要写入 token、签名私钥或跨服务路径。Owner 公钥和 Runtime Contract 是受治理配置;Owner 私钥留在既有签发边界,不交给 API 或 Worker。凭证缺失时服务启动或操作准入必须失败,不能临时新建信任根或借用其他服务的运行目录。
本包不提供生成生产信任根的脚本。测试在临时目录生成测试专用密钥;它们不能用于真实部署。实际安装前还要确认当前主机 systemd 支持这些加固项和 encrypted credential,并对已渲染 unit 运行 `systemd-analyze verify`。本地模板静态检查不能代替实际服务重启、token 轮换及读取权限验收。
`service-status` 使用固定 `/usr/bin/systemctl show` 参数和可信 unit 映射,只读取注册组件状态。若目标主机限制状态读取,应对该命名 operation 配置最小读取权限;不要改成以 root 运行 API,也不要开放任意命令或 SSH。
@@ -0,0 +1,54 @@
[Unit]
Description=AIOS Baseline 1.1 isolated validation API (no production authority cutover)
After=local-fs.target
ConditionPathExists=@PYTHON_EXECUTABLE@
ConditionPathExists=/etc/aios-baseline/server.json
ConditionPathExists=@RELEASE_ROOT@/active-runtime/app/aios_baseline/cli.py
[Service]
Type=simple
User=aios-baseline
Group=aios-baseline
WorkingDirectory=@RELEASE_ROOT@
Environment=PYTHONPATH=@RELEASE_ROOT@/active-runtime/app
Environment=PYTHONDONTWRITEBYTECODE=1
Environment=AIOS_GITOPS_WRITES_ENABLED=false
Environment=PATH=/usr/bin:/bin
LoadCredentialEncrypted=aios-baseline-machine-key:/etc/credstore.encrypted/aios-baseline-machine-key
LoadCredentialEncrypted=aios-baseline-user-token:/etc/credstore.encrypted/aios-baseline-user-token
LoadCredentialEncrypted=aios-baseline-worker-token:/etc/credstore.encrypted/aios-baseline-worker-token
ExecStart=@PYTHON_EXECUTABLE@ -B -m aios_baseline.cli server --config /etc/aios-baseline/server.json
Restart=on-failure
RestartSec=5s
TimeoutStopSec=30s
KillMode=control-group
UMask=0077
StateDirectory=aios-baseline
StateDirectoryMode=0700
ReadWritePaths=/var/lib/aios-baseline
NoNewPrivileges=yes
CapabilityBoundingSet=
AmbientCapabilities=
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
PrivateDevices=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectKernelLogs=yes
ProtectControlGroups=yes
ProtectClock=yes
ProtectProc=invisible
ProcSubset=pid
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
IPAddressDeny=any
IPAddressAllow=localhost
RestrictSUIDSGID=yes
LockPersonality=yes
SystemCallArchitectures=native
RestrictRealtime=yes
TasksMax=64
MemoryMax=512M
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,54 @@
[Unit]
Description=AIOS Baseline 1.1 isolated validation worker (registered transport identity)
After=aios-baseline-validation.service
Requires=aios-baseline-validation.service
ConditionPathExists=@PYTHON_EXECUTABLE@
ConditionPathExists=/etc/aios-baseline/worker.json
ConditionPathExists=@RELEASE_ROOT@/active-runtime/app/aios_baseline/cli.py
[Service]
Type=simple
User=aios-baseline-worker
Group=aios-baseline-worker
WorkingDirectory=@RELEASE_ROOT@
Environment=PYTHONPATH=@RELEASE_ROOT@/active-runtime/app
Environment=PYTHONDONTWRITEBYTECODE=1
Environment=AIOS_GITOPS_WRITES_ENABLED=false
Environment=PATH=/usr/bin:/bin
LoadCredentialEncrypted=aios-baseline-worker-token:/etc/credstore.encrypted/aios-baseline-worker-token
ExecStart=@PYTHON_EXECUTABLE@ -B -m aios_baseline.cli worker --config /etc/aios-baseline/worker.json
Restart=on-failure
RestartSec=5s
TimeoutStopSec=15s
KillMode=control-group
UMask=0077
StateDirectory=aios-baseline-worker
StateDirectoryMode=0700
ReadWritePaths=/var/lib/aios-baseline-worker
InaccessiblePaths=-/var/lib/aios-baseline -/etc/aios-baseline/server.json
NoNewPrivileges=yes
CapabilityBoundingSet=
AmbientCapabilities=
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
PrivateDevices=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectKernelLogs=yes
ProtectControlGroups=yes
ProtectClock=yes
ProtectProc=invisible
ProcSubset=pid
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
IPAddressDeny=any
IPAddressAllow=localhost
RestrictSUIDSGID=yes
LockPersonality=yes
SystemCallArchitectures=native
RestrictRealtime=yes
TasksMax=32
MemoryMax=256M
[Install]
WantedBy=multi-user.target
+115
View File
@@ -0,0 +1,115 @@
{
"contracts": [
{
"bindings": {},
"component_id": "sample",
"contract_revision": 1,
"contract_schema": "aios.runtime-contract.v1",
"credential_refs": {},
"entrypoints": {
"status": {
"component_id": "sample",
"environment": "staging",
"resolver_ref": "service://sample/status"
}
},
"environment": "staging",
"lifecycle": "active",
"operations": [
"service-status"
],
"resource_refs": {},
"tenant_scope": "validation"
}
],
"host_bindings": {
"validation:staging:sample": "REPLACE_WITH_EXISTING_SERVICE.service"
},
"identities": [
{
"role": "owner",
"subject_id": "owner",
"tenant_id": "validation",
"token_file": "${CREDENTIALS_DIRECTORY}/aios-baseline-user-token"
},
{
"role": "worker",
"subject_id": "machine-worker",
"tenant_id": "validation",
"token_file": "${CREDENTIALS_DIRECTORY}/aios-baseline-worker-token",
"worker_id": "diagnostic-worker"
}
],
"listen": {
"host": "127.0.0.1",
"port": 8769
},
"machine_key_file": "${CREDENTIALS_DIRECTORY}/aios-baseline-machine-key",
"machine_key_id": "validation-machine",
"operations": [
{
"effect_class": "READ_ONLY",
"handler_ref": "host.service_status",
"input_schema": "aios.operation.service-status.input.v1",
"operation_id": "service-status",
"owner_boundary": {
"mode": "WITHIN_GRANT"
},
"readback": {
"mode": "NONE",
"required": false
},
"reconcile": {
"mode": "NONE"
},
"required_permissions": [
"service-status"
],
"retry": {
"max_attempts": 2,
"mode": "SAFE"
},
"rollback": {
"mode": "NONE"
},
"timeout_seconds": 10,
"version": 1
}
],
"owner_key_scopes": {
"validation-owner": {
"subject_ids": [
"owner"
],
"tenant_ids": [
"validation"
]
}
},
"owner_public_key_files": {
"validation-owner": "/etc/aios-baseline/owner-public.pem"
},
"profile": "isolated-validation",
"schema": "aios.baseline-server-config.v1",
"state_dir": "/var/lib/aios-baseline",
"workers": [
{
"capabilities": [
"service-status"
],
"fallback_executor": false,
"heartbeat_seconds": 15,
"identity": "machine-worker",
"max_concurrency": 1,
"priority": 10,
"subject_scope": [
"owner"
],
"tenant_scope": [
"validation"
],
"worker_id": "diagnostic-worker"
}
],
"receipt_public_key_files": {}
}
+6
View File
@@ -0,0 +1,6 @@
{
"base_url": "http://127.0.0.1:8769",
"poll_seconds": 2,
"schema": "aios.baseline-worker-config.v1",
"token_file": "${CREDENTIALS_DIRECTORY}/aios-baseline-worker-token"
}
+4 -4
View File
@@ -33,11 +33,11 @@ cp deploy/systemd/aios-autonomous-daemon.service /etc/systemd/system/
cp deploy/systemd/aios-v1r1-wechat-worker.service /etc/systemd/system/
systemctl daemon-reload
systemctl enable aios-autonomous-daemon.service || true
systemctl enable aios-v1r1-wechat-worker.service || true
systemctl enable aios-autonomous-daemon.service
systemctl enable aios-v1r1-wechat-worker.service
systemctl restart aios-autonomous-daemon.service || true
systemctl restart aios-v1r1-wechat-worker.service || true
systemctl restart aios-autonomous-daemon.service
systemctl restart aios-v1r1-wechat-worker.service
sleep 2
systemctl status aios-autonomous-daemon.service --no-pager || true
@@ -11,9 +11,10 @@ WorkingDirectory=@RUNTIME_ROOT@
Environment=AIOS_BRIDGE_SCHEMA_VERSION=aios.chatgpt-gitops-bridge.v2
Environment=AIOS_GITOPS_WRITES_ENABLED=false
Environment=AIOS_GITOPS_DURABLE_JOB_EXECUTION_ENABLED=true
Environment=AIOS_P0_TASK_PRINCIPAL_WRITES_ENABLED=true
Environment=AIOS_GITOPS_WRITES_ENABLED=false
Environment=AIOS_GITEA_API_BASE=https://g.cnnetmkt.top/git/api/v1
Environment=AIOS_GITEA_TOKEN_FILE=/run/credentials/aios-chatgpt-bridge-v2-staging.service/gitea_token
Environment=AIOS_GITEA_TOKEN_FILE=%d/gitea-governance-executor-token
Environment=AIOS_GITEA_REMOTE=gitea-http
Environment=AIOS_GITOPS_BASELINE_AUTHORITY=/var/lib/aios-gitops/model-api-staging/baseline/current.json
Environment=AIOS_CONTROL_PLANE_TARGET_VERSION=V2R2C1
@@ -21,7 +22,7 @@ Environment=AIOS_CONTROL_PLANE_TARGET_OID=@SOURCE_OID@
Environment=AIOS_CONTROL_PLANE_ARTIFACT_SHA256=@ARTIFACT_SHA256@
Environment=PYTHONPATH=@RUNTIME_ROOT@:@RUNTIME_ROOT@/app:@RUNTIME_ROOT@/runtime-agent-os
Environment=AIOS_DEPLOYMENT_STATE=STAGING
LoadCredential=gitea_token:/run/credentials/aios-governance-executor.service/gitea-governance-executor-token
LoadCredentialEncrypted=gitea-governance-executor-token:/etc/credstore.encrypted/gitea-governance-executor-token
ExecStartPre=/usr/bin/install -d -o aiman -g gitops -m 0750 /var/lib/aios-gitops/bridge-v2-staging
ExecStartPre=/usr/bin/setfacl -m u:aiman:--x /usr/local/libexec
ExecStart=/usr/bin/python3 -B @RUNTIME_ROOT@/app/aios_chatgpt_bridge_v2_server.py
@@ -7,8 +7,10 @@ Environment=AIOS_TRUSTED_CANDIDATE_BUILDER_SOCKET=/run/aios-trusted-candidate-bu
Environment=AIOS_TRUSTED_CANDIDATE_BUILDER_POLICY=/opt/aios/gitops-model-api/<VERIFIED_PACKAGE_OID>/runtime/config/gitops/trusted-candidate-builder-policy.json
Environment=AIOS_TRUSTED_CANDIDATE_BUILDER_PUBLIC_KEYRING=/etc/aios-gitops/trusted-builder/public-keyring.json
Environment=AIOS_TRUSTED_CANDIDATE_BUILDER_CREDENTIALS_DIRECTORY=%d
Environment=AIOS_TRUSTED_BUILDER_SOURCE_ADMISSION_ROOT=/var/lib/aios-release-source-admission
Environment=AIOS_SILENT_RELEASE_TRUSTED_POLICY=/opt/aios/gitops-model-api/<VERIFIED_PACKAGE_OID>/runtime/config/gitops/trusted-release-policy.json
LoadCredential=builder_hmac:/etc/aios-gitops/credential-references/trusted-candidate-builder-hmac
LoadCredential=source_admission_hmac:/etc/aios-gitops/credential-references/release-source-admission-hmac
# The credential source above is a reference only; no secret value belongs here.
# Keep execution shadow-disabled and do not add Owner/deploy/tag/baseline switches.
Environment=AIOS_SILENT_RELEASE_EXECUTION_ENABLED=false
@@ -27,6 +27,7 @@ Environment=AIOS_GITOPS_DEPLOYMENT_STATE=STAGING
Environment=AIOS_GITOPS_MODEL_API_SERVICE=aios-gitops-model-api-v2r2c1-staging.service
Environment=AIOS_SILENT_RELEASE_EXECUTION_ENABLED=false
Environment=AIOS_SILENT_RELEASE_STATE_ROOT=/var/lib/aios-gitops/model-api-staging/silent-release
Environment=AIOS_TRUSTED_BUILDER_SOURCE_ADMISSION_ROOT=/var/lib/aios-release-source-admission
Environment=AIOS_BRIDGE_V2_URL=http://127.0.0.1:18892
Environment=AIOS_GITOPS_HOST=127.0.0.1
Environment=AIOS_GITOPS_PORT=18893
@@ -44,7 +45,7 @@ ProtectControlGroups=true
LockPersonality=true
RestrictSUIDSGID=true
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
ReadOnlyPaths=@RUNTIME_ROOT@ /data/aios-gitops/repo.git /var/lib/aios-gitops-runtime-anchor/baseline
ReadOnlyPaths=@RUNTIME_ROOT@ /data/aios-gitops/repo.git /var/lib/aios-gitops-runtime-anchor/baseline -/var/lib/aios-release-source-admission
ReadWritePaths=/var/lib/aios-gitops/model-api-staging
[Install]
@@ -0,0 +1,41 @@
[Unit]
Description=AIOS governed trusted-builder evidence materializer
After=aios-release-source-admission.service aios-release-migration-planner.service
ConditionPathExists=/var/lib/aios-release-source-admission
ConditionPathExists=/var/lib/aios-release-migration
[Service]
Type=oneshot
User=root
Group=gitops
UMask=0027
LoadCredentialEncrypted=source_admission_hmac:/etc/credstore.encrypted/aios-release-source-admission/source_admission_hmac
Environment=AIOS_GITOPS_WRITES_ENABLED=false
Environment=AIOS_RELEASE_EVIDENCE_SOURCE_ROOT=/var/lib/aios-release-evidence-staging
Environment=AIOS_TRUSTED_BUILDER_EVIDENCE_ROOT=/var/lib/aios-trusted-candidate-builder/evidence
EnvironmentFile=-/run/aios-release-evidence-materializer/request.env
Environment=PYTHONDONTWRITEBYTECODE=1
ExecStart=/usr/bin/python3 -B /opt/aios/release-evidence-materializer/current/app/release_evidence_materializer.py materialize --version ${AIOS_RELEASE_VERSION} --candidate-id ${AIOS_RELEASE_CANDIDATE_ID} --source-commit ${AIOS_RELEASE_SOURCE_COMMIT} --builder-policy /opt/aios/trusted-candidate-builder/current/config/gitops/trusted-candidate-builder-policy.json --authority-git-dir /data/aios-gitops/repo.git --source-root ${AIOS_RELEASE_EVIDENCE_SOURCE_ROOT} --output-root ${AIOS_TRUSTED_BUILDER_EVIDENCE_ROOT} --source-admission-root /var/lib/aios-release-source-admission --migration-plan-root /var/lib/aios-release-migration --migration-profile-root /etc/aios-gitops/release-migration-profiles
ExecStartPost=/usr/bin/rm -f /run/aios-release-evidence-materializer/request.env
RuntimeDirectory=aios-release-evidence-materializer
RuntimeDirectoryMode=0750
NoNewPrivileges=true
PrivateTmp=true
PrivateNetwork=true
ProtectSystem=strict
ProtectHome=true
ProtectControlGroups=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectKernelLogs=true
RestrictAddressFamilies=AF_UNIX
LockPersonality=true
MemoryDenyWriteExecute=true
RestrictSUIDSGID=true
SystemCallArchitectures=native
CapabilityBoundingSet=
ReadOnlyPaths=/opt/aios/release-evidence-materializer /opt/aios/trusted-candidate-builder /data/aios-gitops/repo.git /var/lib/aios-release-evidence-staging /var/lib/aios-release-source-admission /var/lib/aios-release-migration /etc/aios-gitops/release-migration-profiles /etc/credstore.encrypted
ReadWritePaths=/var/lib/aios-trusted-candidate-builder/evidence /run/aios-release-evidence-materializer
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,41 @@
[Unit]
Description=AIOS governed cross-code-line release migration planner
After=aios-release-source-admission.service
ConditionPathExists=/var/lib/aios-release-source-admission
[Service]
Type=oneshot
User=root
Group=gitops
UMask=0027
LoadCredentialEncrypted=source_admission_hmac:/etc/credstore.encrypted/aios-release-source-admission/source_admission_hmac
Environment=AIOS_GITOPS_WRITES_ENABLED=false
Environment=AIOS_RELEASE_MIGRATION_PROFILE=/etc/aios-gitops/release-migration-profiles/V2R2C2-baseline11-principal-recovery.json
EnvironmentFile=-/run/aios-release-migration-planner/request.env
Environment=PYTHONDONTWRITEBYTECODE=1
ExecStart=/usr/bin/python3 -B /opt/aios/release-migration/current/app/release_migration_plan.py --authority-git-dir /data/aios-gitops/repo.git --profile ${AIOS_RELEASE_MIGRATION_PROFILE} --output-root /var/lib/aios-release-migration --source-admission-root /var/lib/aios-release-source-admission --builder-policy /opt/aios/trusted-candidate-builder/current/config/gitops/trusted-candidate-builder-policy.json --source-commit ${AIOS_RELEASE_SOURCE_COMMIT} --candidate-id ${AIOS_RELEASE_CANDIDATE_ID}
ExecStartPost=/usr/bin/rm -f /run/aios-release-migration-planner/request.env
StateDirectory=aios-release-migration
StateDirectoryMode=0750
RuntimeDirectory=aios-release-migration-planner
RuntimeDirectoryMode=0750
NoNewPrivileges=true
PrivateTmp=true
PrivateNetwork=true
ProtectSystem=strict
ProtectHome=true
ProtectControlGroups=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectKernelLogs=true
RestrictAddressFamilies=AF_UNIX
LockPersonality=true
MemoryDenyWriteExecute=true
RestrictSUIDSGID=true
SystemCallArchitectures=native
CapabilityBoundingSet=
ReadOnlyPaths=/opt/aios/release-migration /data/aios-gitops/repo.git /var/lib/aios-release-source-admission /opt/aios/trusted-candidate-builder/current/config/gitops/trusted-candidate-builder-policy.json /etc/aios-gitops/release-migration-profiles /var/lib/aios-gitops/bridge-v2-staging
ReadWritePaths=/var/lib/aios-release-migration /run/aios-release-migration-planner
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,40 @@
[Unit]
Description=AIOS governed Source Admission isolated-peer policy rebind
After=aios-release-source-admission.service
ConditionPathExists=/var/lib/aios-release-source-admission
[Service]
Type=oneshot
User=root
Group=gitops
UMask=0027
LoadCredentialEncrypted=source_admission_hmac:/etc/credstore.encrypted/aios-release-source-admission/source_admission_hmac
Environment=AIOS_GITOPS_WRITES_ENABLED=false
Environment=AIOS_GITOPS_REPOSITORY_IDENTITY=aios/aios
Environment=AIOS_RELEASE_ISOLATED_PEER_CGROUP=/system.slice/aios-gitops-model-api-v2r2c2-baseline11-candidate.service
EnvironmentFile=-/run/aios-release-source-admission-rebind/request.env
Environment=PYTHONDONTWRITEBYTECODE=1
ExecStart=/usr/bin/python3 -B /opt/aios/release-source-admission/current/app/release_source_admission.py rebind-policy --source-commit ${AIOS_RELEASE_SOURCE_COMMIT} --previous-policy ${AIOS_RELEASE_PREVIOUS_POLICY} --policy ${AIOS_RELEASE_POLICY} --isolated-peer-cgroup ${AIOS_RELEASE_ISOLATED_PEER_CGROUP} --expected-admission-id ${AIOS_RELEASE_EXPECTED_ADMISSION_ID} --reason ${AIOS_RELEASE_REBIND_REASON} --store-root /var/lib/aios-release-source-admission
ExecStartPost=/usr/bin/rm -f /run/aios-release-source-admission-rebind/request.env
RuntimeDirectory=aios-release-source-admission-rebind
RuntimeDirectoryMode=0750
NoNewPrivileges=true
PrivateTmp=true
PrivateNetwork=true
ProtectSystem=strict
ProtectHome=true
ProtectControlGroups=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectKernelLogs=true
RestrictAddressFamilies=AF_UNIX
LockPersonality=true
MemoryDenyWriteExecute=true
RestrictSUIDSGID=true
SystemCallArchitectures=native
CapabilityBoundingSet=
ReadOnlyPaths=/opt/aios/release-source-admission /etc/credstore.encrypted
ReadWritePaths=/var/lib/aios-release-source-admission /run/aios-release-source-admission-rebind
[Install]
WantedBy=multi-user.target

Some files were not shown because too many files have changed in this diff Show More