Compare commits
40
Commits
@@ -0,0 +1,12 @@
|
|||||||
|
title = "AIOS secret scan policy"
|
||||||
|
|
||||||
|
[extend]
|
||||||
|
useDefault = true
|
||||||
|
|
||||||
|
[[allowlists]]
|
||||||
|
description = "Known non-secret idempotency key used twice by the shadow RBAC test"
|
||||||
|
targetRules = ["generic-api-key"]
|
||||||
|
condition = "AND"
|
||||||
|
regexTarget = "line"
|
||||||
|
paths = ['''tests/task_plane/public_interface/test_shadow_client_rbac\.py$''']
|
||||||
|
regexes = ['''idempotency_key="shadow-rbac-idempotent-v1"''']
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
# Exact historical fingerprint for the removed test-only scanner self-reference.
|
||||||
|
a42f16d4f73b11fd23448da2711d49d5e9f49d9e:tests/fastlane/test_fastlane_assets_contract.py:generic-api-key:135
|
||||||
@@ -1,486 +0,0 @@
|
|||||||
# Agent Production Deployment Requirements
|
|
||||||
|
|
||||||
## Status entering deployment
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
code_development: COMPLETED
|
|
||||||
repository_commit: NOT_CREATED
|
|
||||||
production_deployment: NOT_STARTED
|
|
||||||
reason_repository_commit_missing:
|
|
||||||
current ChatGPT GitOps runtime has AIOS_GITOPS_WRITES_ENABLED=false
|
|
||||||
```
|
|
||||||
|
|
||||||
The code bundle is an input artifact. The Agent must place it on a non-main
|
|
||||||
integration branch, review it, run acceptance tests, merge it through protected
|
|
||||||
main and deploy the API adapter. The Agent must not regenerate or materially
|
|
||||||
rewrite the supplied implementation without recording the new diff and tests.
|
|
||||||
|
|
||||||
## Frozen production state
|
|
||||||
|
|
||||||
Before any change, verify:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
main_oid: 875d41c3101093149414292067fd700dc214d4d3
|
|
||||||
|
|
||||||
V2R1C5:
|
|
||||||
tag_oid: 9e65bd59baea392294c0f253046d3ab116f63145
|
|
||||||
peeled_target_oid: aa60e891547abd7e215af7cf2bf3f21fd23541da
|
|
||||||
baseline_version: V2R1C5
|
|
||||||
baseline_status: PRODUCTION_DEPLOYED
|
|
||||||
target_oid: aa60e891547abd7e215af7cf2bf3f21fd23541da
|
|
||||||
evidence_closure_oid: 5e2cf8276f5093f3352b9a3252c701a44c5ab807
|
|
||||||
```
|
|
||||||
|
|
||||||
If authoritative `main` no longer equals the approved base OID, stop with
|
|
||||||
`MANUAL_OWNER_REVIEW_REQUIRED / AUTHORIZED_BASE_OID_DRIFT`. Do not silently
|
|
||||||
rebase the bundle onto a different base.
|
|
||||||
|
|
||||||
## Bootstrap sequence — no additional Owner interaction
|
|
||||||
|
|
||||||
Use the following order so that the approved V2R1C6 base OID remains valid and
|
|
||||||
the bridge does not create its own authorization deadlock:
|
|
||||||
|
|
||||||
1. Verify the supplied archive/patch SHA-256.
|
|
||||||
2. Install the bundle into a temporary root-owned staging directory outside the
|
|
||||||
Git worktrees.
|
|
||||||
3. Start a localhost-only temporary instance of
|
|
||||||
`aios_authorized_task_writer_api.py` on a dedicated port or Unix socket.
|
|
||||||
4. Bind it to the existing authenticated Owner registry and the exact
|
|
||||||
`writer:provision` scope. Do not expose it through the public gateway.
|
|
||||||
5. Execute the live V2R1C6 provisioning request while `main` still equals
|
|
||||||
`875d41c3101093149414292067fd700dc214d4d3`.
|
|
||||||
6. Verify `READY_FOR_CODE_AUTHORING`, the exact branch OID, writer scope and
|
|
||||||
negative policy tests.
|
|
||||||
7. Use the newly created scoped writer to commit this reviewed bundle onto:
|
|
||||||
`agent/v2r1c6-mainline-reconciliation-20260805`.
|
|
||||||
8. Stop and remove the temporary bootstrap service and revoke its temporary
|
|
||||||
runtime credential. Preserve its signed grant receipt and audit logs.
|
|
||||||
9. Open a protected PR from the V2R1C6 branch. Deploy the permanent bridge only
|
|
||||||
from the reviewed merge commit.
|
|
||||||
|
|
||||||
The temporary service is a control-plane bootstrap, not a production release.
|
|
||||||
It must retain:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
listen_scope: localhost_only
|
|
||||||
production_change_authorized: false
|
|
||||||
tag_change_authorized: false
|
|
||||||
baseline_change_authorized: false
|
|
||||||
main_write_authorized: false
|
|
||||||
```
|
|
||||||
|
|
||||||
This order avoids asking the Owner to approve a second base OID merely because
|
|
||||||
the bridge implementation itself advanced `main`.
|
|
||||||
|
|
||||||
### Alternative platform-maintenance path
|
|
||||||
|
|
||||||
If policy prohibits a temporary localhost bootstrap service, use an existing
|
|
||||||
Owner-approved platform-maintenance executor to create:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
branch: patch/owner-authorization-writer-bridge-20260806
|
|
||||||
base_ref: refs/heads/main
|
|
||||||
base_oid: 875d41c3101093149414292067fd700dc214d4d3
|
|
||||||
direct_main_push: false
|
|
||||||
force_push: false
|
|
||||||
tag_write: false
|
|
||||||
baseline_write: false
|
|
||||||
production_deploy_during_integration: false
|
|
||||||
```
|
|
||||||
|
|
||||||
Merging that branch before V2R1C6 provisioning changes the authoritative base
|
|
||||||
OID. In that alternative only, a new exact Owner base-OID amendment is required.
|
|
||||||
The Agent must never substitute the newer `main` OID automatically.
|
|
||||||
|
|
||||||
## Files to install on the branch
|
|
||||||
|
|
||||||
Copy with the same relative paths:
|
|
||||||
|
|
||||||
```text
|
|
||||||
active-runtime/app/authorized_task_writer_provisioning.py
|
|
||||||
active-runtime/app/aios_authorized_task_writer_api.py
|
|
||||||
docs/gitops/authorized-task-writer-provisioning.md
|
|
||||||
docs/gitops/openapi-authorized-task-writer-fragment.yaml
|
|
||||||
deploy/systemd/aios-gitops-model-api-authorized-writer-bridge.conf.example
|
|
||||||
scripts/smoke_authorized_task_writer_provisioning.py
|
|
||||||
tests/test_authorized_task_writer_provisioning.py
|
|
||||||
```
|
|
||||||
|
|
||||||
Merge the OpenAPI fragment into:
|
|
||||||
|
|
||||||
```text
|
|
||||||
docs/gitops/openapi-chatgpt-actions.yaml
|
|
||||||
```
|
|
||||||
|
|
||||||
The final schema must expose:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
operationId: provisionAuthorizedTaskWriter
|
|
||||||
path: POST /api/task-writer/provision-authorized
|
|
||||||
x-openai-isConsequential: true
|
|
||||||
```
|
|
||||||
|
|
||||||
Do not deploy the fragment as a second standalone OpenAPI document.
|
|
||||||
|
|
||||||
## Required review of existing API integration
|
|
||||||
|
|
||||||
The adapter expects the existing API handler to provide:
|
|
||||||
|
|
||||||
```python
|
|
||||||
handler.need(scope)
|
|
||||||
handler.body()
|
|
||||||
handler.sendj(value, status=200)
|
|
||||||
api.H.do_POST
|
|
||||||
api.main()
|
|
||||||
```
|
|
||||||
|
|
||||||
Confirm these contracts against the branch before merge. If the live API has a
|
|
||||||
different authentication object shape, make the smallest adapter-only change
|
|
||||||
and rerun every test. The core provisioning policy must remain unchanged.
|
|
||||||
|
|
||||||
The API authorization registry must grant `writer:provision` only to the
|
|
||||||
authenticated Owner role. Agents and ordinary writer tokens must not receive
|
|
||||||
this scope.
|
|
||||||
|
|
||||||
## Privilege boundary
|
|
||||||
|
|
||||||
The API service must remain non-root.
|
|
||||||
|
|
||||||
Install a root-owned sudoers rule that allows the API service identity to run
|
|
||||||
only these exact commands with `sudo -n --`:
|
|
||||||
|
|
||||||
```text
|
|
||||||
/usr/local/libexec/aios-grant-signer-adapter
|
|
||||||
/usr/local/libexec/aios-grant-validator
|
|
||||||
/usr/local/libexec/aios-task-writer-provisioner
|
|
||||||
```
|
|
||||||
|
|
||||||
Do not allow:
|
|
||||||
|
|
||||||
```text
|
|
||||||
/bin/sh
|
|
||||||
/bin/bash
|
|
||||||
/usr/bin/env
|
|
||||||
/usr/bin/git
|
|
||||||
wildcard executable paths
|
|
||||||
SETENV
|
|
||||||
arbitrary environment preservation
|
|
||||||
```
|
|
||||||
|
|
||||||
The governed tools themselves must validate their `--grant-dir` and
|
|
||||||
`--result-json` paths. Their binaries must be root-owned, non-symlink, regular,
|
|
||||||
executable and not group/world writable.
|
|
||||||
|
|
||||||
The signer must access its private signing material only through a runtime
|
|
||||||
secret reference. No signing key or token may enter Git, API output, the
|
|
||||||
provisioning receipt or systemd environment text.
|
|
||||||
|
|
||||||
## Runtime paths and permissions
|
|
||||||
|
|
||||||
Create:
|
|
||||||
|
|
||||||
```text
|
|
||||||
/var/lib/aios-gitops/authorized-writer-bridge
|
|
||||||
/var/lib/aios-gitops/authorized-writer-bridge/receipts
|
|
||||||
/var/lib/aios-gitops/authorized-writer-bridge/locks
|
|
||||||
```
|
|
||||||
|
|
||||||
Required properties:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
owner: API service identity
|
|
||||||
mode: "0700"
|
|
||||||
symlink: false
|
|
||||||
persistent: true
|
|
||||||
```
|
|
||||||
|
|
||||||
The directory contains redacted receipts only. Temporary signed grant
|
|
||||||
directories are deleted after success or failure.
|
|
||||||
|
|
||||||
## Required staging environment
|
|
||||||
|
|
||||||
Configure the API service:
|
|
||||||
|
|
||||||
```text
|
|
||||||
AIOS_GITOPS_REPOSITORY_IDENTITY=aios/aios
|
|
||||||
AIOS_GITOPS_AUTHORITY_GIT_DIR=/data/aios-gitops/repo.git
|
|
||||||
AIOS_GITOPS_AUTHORIZED_WRITER_STATE_ROOT=/var/lib/aios-gitops/authorized-writer-bridge
|
|
||||||
AIOS_GITOPS_GRANT_SIGNER_ADAPTER=/usr/local/libexec/aios-grant-signer-adapter
|
|
||||||
AIOS_GITOPS_GRANT_VALIDATOR=/usr/local/libexec/aios-grant-validator
|
|
||||||
AIOS_GITOPS_TASK_WRITER_PROVISIONER=/usr/local/libexec/aios-task-writer-provisioner
|
|
||||||
AIOS_GITOPS_OWNER_PROVISION_SCOPE=writer:provision
|
|
||||||
AIOS_GITOPS_GOVERNED_USE_SUDO=true
|
|
||||||
AIOS_GITOPS_SUDO_PATH=/usr/bin/sudo
|
|
||||||
```
|
|
||||||
|
|
||||||
Use `aios_authorized_task_writer_api.py` as the API entrypoint. It delegates
|
|
||||||
all pre-existing routes and adds only the new endpoint.
|
|
||||||
|
|
||||||
## Pre-merge tests
|
|
||||||
|
|
||||||
Run from the repository root:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 -m unittest -v tests/test_authorized_task_writer_provisioning.py
|
|
||||||
python3 scripts/smoke_authorized_task_writer_provisioning.py
|
|
||||||
python3 -m py_compile \
|
|
||||||
active-runtime/app/authorized_task_writer_provisioning.py \
|
|
||||||
active-runtime/app/aios_authorized_task_writer_api.py
|
|
||||||
```
|
|
||||||
|
|
||||||
Required result:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
unit_tests:
|
|
||||||
total: 9
|
|
||||||
failures: 0
|
|
||||||
errors: 0
|
|
||||||
smoke: PASS
|
|
||||||
py_compile: PASS
|
|
||||||
```
|
|
||||||
|
|
||||||
Also run all existing GitOps API and isolated writer tests affected by:
|
|
||||||
|
|
||||||
```text
|
|
||||||
aios_chatgpt_actions_api
|
|
||||||
aios_isolated_writer_api
|
|
||||||
isolated_writer_authz
|
|
||||||
isolated_writer_token_registry
|
|
||||||
single_owner_governance
|
|
||||||
OpenAPI schema validation
|
|
||||||
```
|
|
||||||
|
|
||||||
Any regression blocks merge.
|
|
||||||
|
|
||||||
## Staging API acceptance
|
|
||||||
|
|
||||||
### Negative authentication
|
|
||||||
|
|
||||||
An Agent or non-Owner credential calling the endpoint must receive:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
http_status: 403
|
|
||||||
error: OWNER_AUTHORIZATION_REQUIRED
|
|
||||||
```
|
|
||||||
|
|
||||||
### Negative scope tests
|
|
||||||
|
|
||||||
Requests enabling any of the following must be denied before signer execution:
|
|
||||||
|
|
||||||
```text
|
|
||||||
production_change_authorized=true
|
|
||||||
tag_change_authorized=true
|
|
||||||
baseline_change_authorized=true
|
|
||||||
main_write_authorized=true
|
|
||||||
allowed_operations containing tag/main/baseline/admin operations
|
|
||||||
missing required denied operation
|
|
||||||
one_time_use=false
|
|
||||||
expiry over 1800 seconds
|
|
||||||
```
|
|
||||||
|
|
||||||
### Drift and conflict tests
|
|
||||||
|
|
||||||
Verify:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
wrong_base_oid:
|
|
||||||
http_status: 409
|
|
||||||
error: AUTHORIZED_BASE_OID_DRIFT
|
|
||||||
|
|
||||||
existing_unreceipted_target_branch:
|
|
||||||
http_status: 409
|
|
||||||
error: EXISTING_BRANCH_OID_CONFLICT
|
|
||||||
```
|
|
||||||
|
|
||||||
### Governed tool failures
|
|
||||||
|
|
||||||
Independently inject and verify fail-closed behavior for:
|
|
||||||
|
|
||||||
```text
|
|
||||||
signer unavailable
|
|
||||||
signer signature absent
|
|
||||||
validator signature_valid=false
|
|
||||||
validator signer_trusted=false
|
|
||||||
validator plan expired
|
|
||||||
provisioner grant_consumed=false
|
|
||||||
provisioner writer scope enables main/tag/baseline/force push
|
|
||||||
any required negative policy test false
|
|
||||||
```
|
|
||||||
|
|
||||||
None may create or modify a Git ref.
|
|
||||||
|
|
||||||
## Live V2R1C6 provisioning request
|
|
||||||
|
|
||||||
After protected merge and API deployment, authenticate as the Owner and submit
|
|
||||||
exactly:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"task_id": "V2R1C6_TASK_SCOPED_WRITER_PROVISIONING",
|
|
||||||
"repository": "aios/aios",
|
|
||||||
"base_ref": "refs/heads/main",
|
|
||||||
"base_oid": "875d41c3101093149414292067fd700dc214d4d3",
|
|
||||||
"target_branch": "agent/v2r1c6-mainline-reconciliation-20260805",
|
|
||||||
"owner_authorization_reference": "chat-owner-authorization:v2r1c6:20260806",
|
|
||||||
"one_time_use": true,
|
|
||||||
"expires_in_seconds": 900,
|
|
||||||
"production_change_authorized": false,
|
|
||||||
"tag_change_authorized": false,
|
|
||||||
"baseline_change_authorized": false,
|
|
||||||
"main_write_authorized": false,
|
|
||||||
"allowed_operations": [
|
|
||||||
"create_worktree",
|
|
||||||
"create_target_branch_if_absent",
|
|
||||||
"commit_to_target_branch",
|
|
||||||
"push_target_branch"
|
|
||||||
],
|
|
||||||
"denied_operations": [
|
|
||||||
"direct_main_push",
|
|
||||||
"force_push",
|
|
||||||
"tag_create",
|
|
||||||
"tag_update",
|
|
||||||
"tag_delete",
|
|
||||||
"branch_delete",
|
|
||||||
"repository_admin",
|
|
||||||
"baseline_write",
|
|
||||||
"production_deploy",
|
|
||||||
"production_marker_write",
|
|
||||||
"history_rewrite"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
This request must be executed by the localhost-only bootstrap instance before
|
|
||||||
the bridge code advances `main`. The endpoint must never infer “current main”.
|
|
||||||
|
|
||||||
If the Agent chooses the alternative merge-first path, it must stop after the
|
|
||||||
merge and obtain an exact Owner amendment for the new merge OID. It must not
|
|
||||||
silently replace the approved OID.
|
|
||||||
|
|
||||||
## Success response
|
|
||||||
|
|
||||||
Require:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
status: READY_FOR_CODE_AUTHORING
|
|
||||||
grant_consumed: true
|
|
||||||
branch: agent/v2r1c6-mainline-reconciliation-20260805
|
|
||||||
branch_oid: <exact amended authorized base OID>
|
|
||||||
base_oid: <same exact OID>
|
|
||||||
main_unchanged: true
|
|
||||||
|
|
||||||
writer_scope:
|
|
||||||
workspace_role: task-scoped-isolated-writer
|
|
||||||
writes_enabled: true
|
|
||||||
allowed_branch: agent/v2r1c6-mainline-reconciliation-20260805
|
|
||||||
allowed_base_oid: <exact amended authorized base OID>
|
|
||||||
allowed_routes:
|
|
||||||
- POST /api/commit-file
|
|
||||||
allow_main_write: false
|
|
||||||
allow_tag_write: false
|
|
||||||
allow_baseline_write: false
|
|
||||||
allow_force_push: false
|
|
||||||
```
|
|
||||||
|
|
||||||
All seven negative policy results must be true.
|
|
||||||
|
|
||||||
Submit the identical request a second time. It must return the same grant and
|
|
||||||
branch attestation with:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
idempotent: true
|
|
||||||
```
|
|
||||||
|
|
||||||
It must not run the provisioner again.
|
|
||||||
|
|
||||||
## Production postchecks
|
|
||||||
|
|
||||||
After API deployment and writer provisioning, independently verify:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
existing_routes_regression: PASS
|
|
||||||
new_endpoint: PASS
|
|
||||||
main_ref: expected reviewed merge commit
|
|
||||||
V2R1C5_tag_oid: 9e65bd59baea392294c0f253046d3ab116f63145
|
|
||||||
V2R1C5_peeled_target: aa60e891547abd7e215af7cf2bf3f21fd23541da
|
|
||||||
V2R1C5_baseline_unchanged: true
|
|
||||||
new_release_tag_created: false
|
|
||||||
production_baseline_changed: false
|
|
||||||
```
|
|
||||||
|
|
||||||
The deployment of this control-plane bridge does not authorize a V2R1C6
|
|
||||||
production release. It only enables scoped code authoring.
|
|
||||||
|
|
||||||
## Rollback
|
|
||||||
|
|
||||||
If the new API route fails:
|
|
||||||
|
|
||||||
1. Stop accepting new provisioning calls.
|
|
||||||
2. Restore the previous API entrypoint.
|
|
||||||
3. Restart and verify all existing routes.
|
|
||||||
4. Preserve receipts and logs.
|
|
||||||
5. Revoke any task writer capability already minted.
|
|
||||||
6. Do not delete a branch or rewrite a tag.
|
|
||||||
7. Revert the protected-main bridge commit through an ordinary reviewed revert
|
|
||||||
PR if source rollback is required.
|
|
||||||
|
|
||||||
## Required final Agent report
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
STATUS: COMPLETED | PARTIALLY_COMPLETED | BLOCKED | FAILED
|
|
||||||
|
|
||||||
INTEGRATION:
|
|
||||||
branch:
|
|
||||||
commit:
|
|
||||||
protected_pr:
|
|
||||||
merge_commit:
|
|
||||||
files_changed:
|
|
||||||
|
|
||||||
TESTS:
|
|
||||||
unit:
|
|
||||||
smoke:
|
|
||||||
py_compile:
|
|
||||||
regression:
|
|
||||||
fault_injection:
|
|
||||||
|
|
||||||
DEPLOYMENT:
|
|
||||||
service:
|
|
||||||
entrypoint:
|
|
||||||
runtime_status:
|
|
||||||
endpoint_status:
|
|
||||||
rollback_verified:
|
|
||||||
|
|
||||||
PROVISIONING:
|
|
||||||
owner_authorization_reference:
|
|
||||||
approval_plan_id:
|
|
||||||
grant_id:
|
|
||||||
grant_digest_sha256:
|
|
||||||
signing_key_identity:
|
|
||||||
branch:
|
|
||||||
branch_oid:
|
|
||||||
workspace_path:
|
|
||||||
grant_consumed:
|
|
||||||
idempotent_replay:
|
|
||||||
|
|
||||||
CONSISTENCY:
|
|
||||||
main_oid:
|
|
||||||
V2R1C5_tag_oid:
|
|
||||||
V2R1C5_peeled_target_oid:
|
|
||||||
baseline_version:
|
|
||||||
baseline_changed:
|
|
||||||
tags_changed:
|
|
||||||
|
|
||||||
SECRETS:
|
|
||||||
runtime_secret_reference_used:
|
|
||||||
plaintext_secret_logged:
|
|
||||||
plaintext_secret_committed:
|
|
||||||
|
|
||||||
BLOCKERS:
|
|
||||||
|
|
||||||
GOVERNANCE:
|
|
||||||
force_push_used: false
|
|
||||||
tag_rewritten: false
|
|
||||||
direct_main_push: false
|
|
||||||
secrets_committed: false
|
|
||||||
owner_approval_respected: true
|
|
||||||
```
|
|
||||||
@@ -1,401 +0,0 @@
|
|||||||
# Agent Deployment Requirements: Owner Authorization Broker V2
|
|
||||||
|
|
||||||
## Objective
|
|
||||||
|
|
||||||
Deploy the audited Owner Authorization Broker that closes the bootstrap gap
|
|
||||||
without issuing a new general-purpose Owner credential and without modifying
|
|
||||||
`/data/aios-gitops/runtime/secrets/gitops.tokens`.
|
|
||||||
|
|
||||||
The broker reuses one existing, non-wildcard Owner credential only to
|
|
||||||
authenticate the Owner identity through one pre-existing exact scope. It then
|
|
||||||
records an immutable task authorization event and directly invokes the
|
|
||||||
existing signed grant -> validator -> one-shot provisioner chain.
|
|
||||||
|
|
||||||
Production deployment of V2R1C6 is **not** authorized by this task.
|
|
||||||
|
|
||||||
## Fixed bindings
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
task_id: V2R1C6_TASK_SCOPED_WRITER_PROVISIONING
|
|
||||||
repository: aios/aios
|
|
||||||
base_ref: refs/heads/main
|
|
||||||
base_oid: 875d41c3101093149414292067fd700dc214d4d3
|
|
||||||
target_branch: agent/v2r1c6-mainline-reconciliation-20260805
|
|
||||||
production_change_authorized: false
|
|
||||||
tag_change_authorized: false
|
|
||||||
baseline_change_authorized: false
|
|
||||||
main_write_authorized: false
|
|
||||||
```
|
|
||||||
|
|
||||||
The existing V2R1C5 objects are frozen:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
tag_oid: 9e65bd59baea392294c0f253046d3ab116f63145
|
|
||||||
peeled_target_oid: aa60e891547abd7e215af7cf2bf3f21fd23541da
|
|
||||||
baseline_version: V2R1C5
|
|
||||||
baseline_status: PRODUCTION_DEPLOYED
|
|
||||||
```
|
|
||||||
|
|
||||||
## 1. Package verification
|
|
||||||
|
|
||||||
Verify `MANIFEST.sha256` before installation. Reject any mismatch.
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 -m unittest -v \
|
|
||||||
tests/test_authorized_task_writer_provisioning.py \
|
|
||||||
tests/test_owner_authorization_broker.py
|
|
||||||
|
|
||||||
python3 scripts/smoke_authorized_task_writer_provisioning.py
|
|
||||||
python3 scripts/smoke_owner_authorization_broker.py
|
|
||||||
python3 -m py_compile \
|
|
||||||
active-runtime/app/authorized_task_writer_provisioning.py \
|
|
||||||
active-runtime/app/aios_authorized_task_writer_api.py \
|
|
||||||
active-runtime/app/owner_authorization_broker.py \
|
|
||||||
active-runtime/app/aios_owner_authorization_broker_api.py
|
|
||||||
```
|
|
||||||
|
|
||||||
All checks must pass.
|
|
||||||
|
|
||||||
## 2. Existing Owner credential selection
|
|
||||||
|
|
||||||
Read only the non-secret registry metadata. Select one existing credential
|
|
||||||
that satisfies all of the following:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
role: owner
|
|
||||||
wildcard_scope: false
|
|
||||||
account: exact known Owner identity
|
|
||||||
contains_exact_intake_scope: true
|
|
||||||
```
|
|
||||||
|
|
||||||
Use one existing exact scope such as `commit:create` or `file:write`. Prefer
|
|
||||||
the narrowest scope already assigned to the intended Owner.
|
|
||||||
|
|
||||||
Do not:
|
|
||||||
|
|
||||||
- use the historical wildcard Owner credential;
|
|
||||||
- add `writer:provision` to any bearer token;
|
|
||||||
- create or modify a bearer credential;
|
|
||||||
- print, copy or commit bearer-token material.
|
|
||||||
|
|
||||||
Record only:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
owner_account: <non-secret account>
|
|
||||||
credential_id: <non-secret ID, when available>
|
|
||||||
intake_scope: <one exact existing scope>
|
|
||||||
scope_readback_available: true|false
|
|
||||||
```
|
|
||||||
|
|
||||||
If `handler.need()` returns the exact scope set, configure the Owner account
|
|
||||||
allowlist and leave the credential ID allowlist empty.
|
|
||||||
|
|
||||||
If scope metadata is not returned, configure the exact non-secret credential
|
|
||||||
ID allowlist. If neither exact scope metadata nor a stable non-secret
|
|
||||||
credential ID can be proven, stop:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
STATUS: BLOCKED
|
|
||||||
REASON: OWNER_CREDENTIAL_SCOPE_NOT_PROVABLE
|
|
||||||
```
|
|
||||||
|
|
||||||
## 3. Staging installation
|
|
||||||
|
|
||||||
Install the four application modules in a root-owned staging directory:
|
|
||||||
|
|
||||||
```text
|
|
||||||
authorized_task_writer_provisioning.py
|
|
||||||
aios_authorized_task_writer_api.py
|
|
||||||
owner_authorization_broker.py
|
|
||||||
aios_owner_authorization_broker_api.py
|
|
||||||
```
|
|
||||||
|
|
||||||
The directory and files must not be group/world writable.
|
|
||||||
|
|
||||||
Prepare environment values:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
AIOS_GITOPS_OWNER_AUTHORIZATION_BROKER_ENABLED: "true"
|
|
||||||
AIOS_GITOPS_OWNER_AUTHORIZATION_INTAKE_SCOPES: <one exact existing scope>
|
|
||||||
AIOS_GITOPS_OWNER_AUTHORIZATION_ALLOWED_IDENTITIES: <exact owner account>
|
|
||||||
AIOS_GITOPS_OWNER_AUTHORIZATION_ALLOWED_CREDENTIAL_IDS: <exact ID or empty>
|
|
||||||
AIOS_GITOPS_OWNER_AUTHORIZATION_MAX_EXPIRY_SECONDS: "900"
|
|
||||||
AIOS_GITOPS_OWNER_AUTHORIZATION_STATE_ROOT:
|
|
||||||
/var/lib/aios-gitops/owner-authorization-broker
|
|
||||||
```
|
|
||||||
|
|
||||||
Use the existing signer, validator and provisioner paths. Do not change their
|
|
||||||
root governance configuration or disable signature checks.
|
|
||||||
|
|
||||||
Start a localhost-only staging instance on a temporary port. Do not replace
|
|
||||||
the production service yet.
|
|
||||||
|
|
||||||
## 4. Authentication fail-closed tests
|
|
||||||
|
|
||||||
The staging endpoint is:
|
|
||||||
|
|
||||||
```text
|
|
||||||
POST /api/governance/owner-authorizations
|
|
||||||
```
|
|
||||||
|
|
||||||
Required tests:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
unauthenticated:
|
|
||||||
expected: 401
|
|
||||||
|
|
||||||
wrong_role:
|
|
||||||
expected: 403
|
|
||||||
|
|
||||||
wrong_scope:
|
|
||||||
expected: 403
|
|
||||||
|
|
||||||
wildcard_owner:
|
|
||||||
expected: 403
|
|
||||||
error: WILDCARD_OWNER_CREDENTIAL_REJECTED
|
|
||||||
|
|
||||||
unallowlisted_owner:
|
|
||||||
expected: 403
|
|
||||||
error: OWNER_IDENTITY_NOT_ALLOWED
|
|
||||||
|
|
||||||
production_capability_true:
|
|
||||||
expected: 403
|
|
||||||
error: PRODUCTION_CAPABILITY_REJECTED
|
|
||||||
```
|
|
||||||
|
|
||||||
A connection failure, 404 or 500 is not an authorization-policy pass.
|
|
||||||
|
|
||||||
## 5. Formal provisioning request
|
|
||||||
|
|
||||||
Use the selected existing non-wildcard Owner credential through its runtime
|
|
||||||
secret reference. Never place the bearer token in the request body, command
|
|
||||||
line, shell history, output report or Git.
|
|
||||||
|
|
||||||
Request:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"action": "AUTHORIZE_TASK_WRITER_PROVISIONING",
|
|
||||||
"task_id": "V2R1C6_TASK_SCOPED_WRITER_PROVISIONING",
|
|
||||||
"repository": "aios/aios",
|
|
||||||
"base_ref": "refs/heads/main",
|
|
||||||
"base_oid": "875d41c3101093149414292067fd700dc214d4d3",
|
|
||||||
"target_branch": "agent/v2r1c6-mainline-reconciliation-20260805",
|
|
||||||
"one_time_use": true,
|
|
||||||
"expires_in_seconds": 900,
|
|
||||||
"production_change_authorized": false,
|
|
||||||
"tag_change_authorized": false,
|
|
||||||
"baseline_change_authorized": false,
|
|
||||||
"main_write_authorized": false,
|
|
||||||
"allowed_operations": [
|
|
||||||
"create_worktree",
|
|
||||||
"create_target_branch_if_absent",
|
|
||||||
"commit_to_target_branch",
|
|
||||||
"push_target_branch"
|
|
||||||
],
|
|
||||||
"denied_operations": [
|
|
||||||
"direct_main_push",
|
|
||||||
"force_push",
|
|
||||||
"tag_create",
|
|
||||||
"tag_update",
|
|
||||||
"tag_delete",
|
|
||||||
"branch_delete",
|
|
||||||
"repository_admin",
|
|
||||||
"baseline_write",
|
|
||||||
"production_deploy",
|
|
||||||
"production_marker_write",
|
|
||||||
"history_rewrite"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The Owner authorization reference is generated by the broker. The caller must
|
|
||||||
not submit or invent one.
|
|
||||||
|
|
||||||
Expected result:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
status: READY_FOR_CODE_AUTHORING
|
|
||||||
owner_authorization_event_id: generated
|
|
||||||
owner_authorization_reference: generated
|
|
||||||
owner_authorization_digest_sha256: generated
|
|
||||||
owner_authorization_consumed: true
|
|
||||||
approval_plan_id: generated
|
|
||||||
grant_digest_sha256: generated
|
|
||||||
grant_consumed: true
|
|
||||||
|
|
||||||
branch_oid: 875d41c3101093149414292067fd700dc214d4d3
|
|
||||||
|
|
||||||
writer_scope:
|
|
||||||
workspace_role: task-scoped-isolated-writer
|
|
||||||
writes_enabled: true
|
|
||||||
allowed_branch: agent/v2r1c6-mainline-reconciliation-20260805
|
|
||||||
allowed_base_oid: 875d41c3101093149414292067fd700dc214d4d3
|
|
||||||
allow_main_write: false
|
|
||||||
allow_tag_write: false
|
|
||||||
allow_baseline_write: false
|
|
||||||
allow_force_push: false
|
|
||||||
```
|
|
||||||
|
|
||||||
## 6. Independent post-provisioning verification
|
|
||||||
|
|
||||||
Verify the Authority ref:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git --git-dir=/data/aios-gitops/repo.git rev-parse \
|
|
||||||
refs/heads/agent/v2r1c6-mainline-reconciliation-20260805
|
|
||||||
```
|
|
||||||
|
|
||||||
It must equal:
|
|
||||||
|
|
||||||
```text
|
|
||||||
875d41c3101093149414292067fd700dc214d4d3
|
|
||||||
```
|
|
||||||
|
|
||||||
Verify `main` remains unchanged, V2R1C5 tag object and peeled target remain
|
|
||||||
unchanged, and the Production Baseline remains V2R1C5.
|
|
||||||
|
|
||||||
Verify the broker audit event under:
|
|
||||||
|
|
||||||
```text
|
|
||||||
/var/lib/aios-gitops/owner-authorization-broker/events/
|
|
||||||
```
|
|
||||||
|
|
||||||
Requirements:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
status: CONSUMED
|
|
||||||
wildcard_scope_used: false
|
|
||||||
grant_consumed: true
|
|
||||||
permissions:
|
|
||||||
directory: "0700"
|
|
||||||
json_files: "0600"
|
|
||||||
```
|
|
||||||
|
|
||||||
Do not return bearer-token material in the report.
|
|
||||||
|
|
||||||
## 7. Commit the broker through the new writer
|
|
||||||
|
|
||||||
After `READY_FOR_CODE_AUTHORING`, use only the writer's
|
|
||||||
`POST /api/commit-file` route to put this code bundle on:
|
|
||||||
|
|
||||||
```text
|
|
||||||
agent/v2r1c6-mainline-reconciliation-20260805
|
|
||||||
```
|
|
||||||
|
|
||||||
Commit the bridge and broker as separate intentional commits when practical:
|
|
||||||
|
|
||||||
1. Owner-authorized task-writer bridge.
|
|
||||||
2. Audited Owner authorization broker and deployment docs.
|
|
||||||
|
|
||||||
Do not merge to `main` directly. Create a protected PR after branch commits
|
|
||||||
and tests are complete.
|
|
||||||
|
|
||||||
## 8. Permanent service deployment
|
|
||||||
|
|
||||||
Only after protected review and merge:
|
|
||||||
|
|
||||||
1. Install the reviewed files from the merge commit.
|
|
||||||
2. Apply a reviewed systemd drop-in based on the provided example.
|
|
||||||
3. Start the wrapper entrypoint:
|
|
||||||
`aios_owner_authorization_broker_api.py`.
|
|
||||||
4. Verify all existing API routes still work.
|
|
||||||
5. Verify the broker remains localhost/reverse-proxy constrained as intended.
|
|
||||||
6. Re-run authentication and policy tests.
|
|
||||||
7. Remove the temporary staging instance.
|
|
||||||
|
|
||||||
This permanent bridge deployment does not authorize a V2R1C6 production tag,
|
|
||||||
Baseline update or application deployment.
|
|
||||||
|
|
||||||
## 9. Fail-closed conditions
|
|
||||||
|
|
||||||
Stop with `BLOCKED` for any of the following:
|
|
||||||
|
|
||||||
- Authority `main` no longer equals the fixed base OID.
|
|
||||||
- Target branch exists at a different OID.
|
|
||||||
- Only a wildcard Owner credential can authenticate.
|
|
||||||
- Exact credential scope cannot be proven.
|
|
||||||
- Owner identity is not explicitly allow-listed.
|
|
||||||
- Signer, validator or provisioner self-test fails.
|
|
||||||
- Grant is not marked one-time and consumed.
|
|
||||||
- Writer allows main, tag, Baseline, force-push or branch-delete operations.
|
|
||||||
- Audit event contains sensitive material.
|
|
||||||
- V2R1C5 tag or Baseline changes.
|
|
||||||
|
|
||||||
## 10. Required Agent result
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
STATUS: READY_FOR_CODE_AUTHORING | BLOCKED | FAILED
|
|
||||||
|
|
||||||
PACKAGE:
|
|
||||||
manifest_verified:
|
|
||||||
unit_tests:
|
|
||||||
smoke_tests:
|
|
||||||
py_compile:
|
|
||||||
|
|
||||||
OWNER_CREDENTIAL:
|
|
||||||
account:
|
|
||||||
credential_id:
|
|
||||||
intake_scope:
|
|
||||||
exact_scope_proven:
|
|
||||||
wildcard_used: false
|
|
||||||
token_logged: false
|
|
||||||
|
|
||||||
BROKER:
|
|
||||||
staging_listener:
|
|
||||||
endpoint_status:
|
|
||||||
event_id:
|
|
||||||
authorization_reference:
|
|
||||||
authorization_digest_sha256:
|
|
||||||
event_consumed:
|
|
||||||
audit_permissions_valid:
|
|
||||||
|
|
||||||
GRANT:
|
|
||||||
approval_plan_id:
|
|
||||||
grant_digest_sha256:
|
|
||||||
signer_identity:
|
|
||||||
signature_valid:
|
|
||||||
one_time_use:
|
|
||||||
consumed:
|
|
||||||
|
|
||||||
BRANCH:
|
|
||||||
name:
|
|
||||||
oid:
|
|
||||||
base_oid:
|
|
||||||
ref_exists:
|
|
||||||
worktree_path:
|
|
||||||
worktree_clean:
|
|
||||||
|
|
||||||
WRITER:
|
|
||||||
status:
|
|
||||||
writes_enabled:
|
|
||||||
allowed_branch:
|
|
||||||
allowed_base_oid:
|
|
||||||
allowed_routes:
|
|
||||||
main_write_denied:
|
|
||||||
tag_write_denied:
|
|
||||||
baseline_write_denied:
|
|
||||||
force_push_denied:
|
|
||||||
|
|
||||||
POSTCHECK:
|
|
||||||
main_unchanged:
|
|
||||||
V2R1C5_tag_unchanged:
|
|
||||||
baseline_unchanged:
|
|
||||||
|
|
||||||
REPOSITORY_INTEGRATION:
|
|
||||||
commit_oids:
|
|
||||||
protected_pr:
|
|
||||||
merged:
|
|
||||||
permanent_bridge_deployed:
|
|
||||||
|
|
||||||
BLOCKERS:
|
|
||||||
|
|
||||||
GOVERNANCE:
|
|
||||||
force_push_used: false
|
|
||||||
tag_rewritten: false
|
|
||||||
direct_main_push: false
|
|
||||||
secrets_committed: false
|
|
||||||
owner_approval_respected: true
|
|
||||||
```
|
|
||||||
@@ -1,562 +0,0 @@
|
|||||||
# Agent execution requirements: V2R1C6 task identity overlay
|
|
||||||
|
|
||||||
## Required result
|
|
||||||
|
|
||||||
The Agent must return one of:
|
|
||||||
|
|
||||||
- `READY_FOR_CODE_AUTHORING`
|
|
||||||
- `BLOCKED`
|
|
||||||
- `FAILED`
|
|
||||||
|
|
||||||
The Agent may not report success until the task branch, isolated worktree and
|
|
||||||
task writer are independently verified.
|
|
||||||
|
|
||||||
This stage is **not** a production deployment. It must not modify `main`,
|
|
||||||
V2R1C5, tags, Registry, Production Baseline or production services.
|
|
||||||
|
|
||||||
## Fixed authority
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
task_id: V2R1C6_TASK_SCOPED_WRITER_PROVISIONING
|
|
||||||
repository: aios/aios
|
|
||||||
base_ref: refs/heads/main
|
|
||||||
base_oid: 875d41c3101093149414292067fd700dc214d4d3
|
|
||||||
target_branch: agent/v2r1c6-mainline-reconciliation-20260805
|
|
||||||
owner_identity: aiman
|
|
||||||
one_time_use: true
|
|
||||||
maximum_ttl_seconds: 900
|
|
||||||
production_change_authorized: false
|
|
||||||
tag_change_authorized: false
|
|
||||||
baseline_change_authorized: false
|
|
||||||
main_write_authorized: false
|
|
||||||
```
|
|
||||||
|
|
||||||
## 1. Verify package
|
|
||||||
|
|
||||||
Verify the externally supplied package against the published SHA-256 values and
|
|
||||||
then verify every entry in `MANIFEST.sha256`.
|
|
||||||
|
|
||||||
Do not synthesize a missing archive. Use either the verified ZIP or TAR.GZ.
|
|
||||||
|
|
||||||
Required checks:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
package_hash_match: true
|
|
||||||
manifest_entries_match: true
|
|
||||||
unit_tests_pass: true
|
|
||||||
smoke_tests_pass: true
|
|
||||||
py_compile_pass: true
|
|
||||||
```
|
|
||||||
|
|
||||||
## 2. Create root-owned staging
|
|
||||||
|
|
||||||
Extract to a fresh directory such as:
|
|
||||||
|
|
||||||
```text
|
|
||||||
/var/lib/aios-gitops/staging/v2r1c6-task-identity-overlay-<nonce>
|
|
||||||
```
|
|
||||||
|
|
||||||
Requirements:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
owner: root
|
|
||||||
group: root
|
|
||||||
directory_mode: "0700"
|
|
||||||
files_not_world_writable: true
|
|
||||||
symlinks_rejected: true
|
|
||||||
```
|
|
||||||
|
|
||||||
Do not overwrite the active runtime tree.
|
|
||||||
|
|
||||||
## 3. Re-run target-host tests
|
|
||||||
|
|
||||||
From the staging root:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 -m unittest discover -s tests -v
|
|
||||||
python3 scripts/smoke_authorized_task_writer_provisioning.py
|
|
||||||
python3 scripts/smoke_owner_authorization_broker.py
|
|
||||||
python3 scripts/smoke_task_identity_overlay.py
|
|
||||||
python3 -m py_compile \
|
|
||||||
active-runtime/app/authorized_task_writer_provisioning.py \
|
|
||||||
active-runtime/app/owner_authorization_broker.py \
|
|
||||||
active-runtime/app/task_identity_overlay.py \
|
|
||||||
active-runtime/app/aios_authorized_task_writer_api.py \
|
|
||||||
active-runtime/app/aios_owner_authorization_broker_api.py \
|
|
||||||
active-runtime/app/aios_task_identity_overlay_api.py \
|
|
||||||
active-runtime/app/aios_task_identity_overlay_cli.py \
|
|
||||||
scripts/call_task_identity_overlay.py
|
|
||||||
```
|
|
||||||
|
|
||||||
Any failure is `BLOCKED`; do not proceed.
|
|
||||||
|
|
||||||
## 4. Prestart Git and production freeze checks
|
|
||||||
|
|
||||||
Read from the authority repository:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git --git-dir=/data/aios-gitops/repo.git rev-parse refs/heads/main
|
|
||||||
git --git-dir=/data/aios-gitops/repo.git \
|
|
||||||
show-ref --verify --quiet \
|
|
||||||
refs/heads/agent/v2r1c6-mainline-reconciliation-20260805
|
|
||||||
git --git-dir=/data/aios-gitops/repo.git rev-parse refs/tags/V2R1C5
|
|
||||||
git --git-dir=/data/aios-gitops/repo.git rev-parse 'refs/tags/V2R1C5^{}'
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
main_oid: 875d41c3101093149414292067fd700dc214d4d3
|
|
||||||
target_branch_exists: false
|
|
||||||
V2R1C5_tag_oid: 9e65bd59baea392294c0f253046d3ab116f63145
|
|
||||||
V2R1C5_peeled_target: aa60e891547abd7e215af7cf2bf3f21fd23541da
|
|
||||||
```
|
|
||||||
|
|
||||||
Read Baseline and require:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
version: V2R1C5
|
|
||||||
status: PRODUCTION_DEPLOYED
|
|
||||||
target_oid: aa60e891547abd7e215af7cf2bf3f21fd23541da
|
|
||||||
evidence_closure_oid: 5e2cf8276f5093f3352b9a3252c701a44c5ab807
|
|
||||||
```
|
|
||||||
|
|
||||||
If `main` has drifted, return:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
STATUS: MANUAL_OWNER_REVIEW_REQUIRED
|
|
||||||
REASON: AUTHORIZED_BASE_OID_DRIFT
|
|
||||||
```
|
|
||||||
|
|
||||||
If the target branch already exists at another OID, return:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
STATUS: MANUAL_OWNER_REVIEW_REQUIRED
|
|
||||||
REASON: EXISTING_BRANCH_OID_CONFLICT
|
|
||||||
```
|
|
||||||
|
|
||||||
Do not delete, reset or force-push the branch.
|
|
||||||
|
|
||||||
## 5. Discover active secret references
|
|
||||||
|
|
||||||
The overlay does not read or modify `gitops.tokens`, but its issuer must compare
|
|
||||||
the new random secret against all active bearer secret references.
|
|
||||||
|
|
||||||
Discover references without printing secret contents. At minimum include the
|
|
||||||
currently active ChatGPT/GitOps bearer secret reference and any other bearer
|
|
||||||
files consumed by the active API.
|
|
||||||
|
|
||||||
Pass them as a comma-separated list in:
|
|
||||||
|
|
||||||
```text
|
|
||||||
AIOS_GITOPS_IDENTITY_OVERLAY_COLLISION_SECRET_REFS
|
|
||||||
```
|
|
||||||
|
|
||||||
Evidence must contain only paths and SHA-256 match counts, not token values.
|
|
||||||
|
|
||||||
If active secret references cannot be enumerated, return:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
STATUS: BLOCKED
|
|
||||||
REASON: ACTIVE_SECRET_REFERENCE_SET_UNRESOLVED
|
|
||||||
```
|
|
||||||
|
|
||||||
## 6. Install the transient systemd service
|
|
||||||
|
|
||||||
Use the provided template:
|
|
||||||
|
|
||||||
```text
|
|
||||||
deploy/systemd/aios-v2r1c6-task-identity-overlay.service.example
|
|
||||||
```
|
|
||||||
|
|
||||||
Substitute only:
|
|
||||||
|
|
||||||
- verified root-owned staging directory;
|
|
||||||
- verified active secret-ref path list.
|
|
||||||
|
|
||||||
Do not change task ID, repository, base OID, target branch or denied
|
|
||||||
capabilities.
|
|
||||||
|
|
||||||
The service must listen only on:
|
|
||||||
|
|
||||||
```text
|
|
||||||
127.0.0.1:19864
|
|
||||||
```
|
|
||||||
|
|
||||||
Before start, run:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
systemd-analyze verify \
|
|
||||||
/etc/systemd/system/aios-v2r1c6-task-identity-overlay.service
|
|
||||||
```
|
|
||||||
|
|
||||||
Then:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo systemctl daemon-reload
|
|
||||||
sudo systemctl start aios-v2r1c6-task-identity-overlay.service
|
|
||||||
```
|
|
||||||
|
|
||||||
Confirm:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl --fail --silent http://127.0.0.1:19864/healthz
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
service: aios-task-identity-overlay
|
|
||||||
status: active
|
|
||||||
global_token_registry_used: false
|
|
||||||
loopback_only: true
|
|
||||||
```
|
|
||||||
|
|
||||||
Do not expose the port externally.
|
|
||||||
|
|
||||||
## 7. Issue the one-time overlay
|
|
||||||
|
|
||||||
Use the root-only CLI. The authorization assertion is an audit label for this
|
|
||||||
already approved Owner task; it must not be described as an external signed
|
|
||||||
event.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo env \
|
|
||||||
PYTHONPATH=<staging>/active-runtime/app \
|
|
||||||
AIOS_GITOPS_IDENTITY_OVERLAY_STATE_ROOT=/var/lib/aios-gitops/task-identity-overlays \
|
|
||||||
AIOS_GITOPS_IDENTITY_OVERLAY_SECRET_ROOT=/run/aios-gitops/task-identity-overlays \
|
|
||||||
AIOS_GITOPS_IDENTITY_OVERLAY_TASK_ID=V2R1C6_TASK_SCOPED_WRITER_PROVISIONING \
|
|
||||||
AIOS_GITOPS_IDENTITY_OVERLAY_OWNER_IDENTITY=aiman \
|
|
||||||
AIOS_GITOPS_IDENTITY_OVERLAY_BASE_REF=refs/heads/main \
|
|
||||||
AIOS_GITOPS_IDENTITY_OVERLAY_BASE_OID=875d41c3101093149414292067fd700dc214d4d3 \
|
|
||||||
AIOS_GITOPS_IDENTITY_OVERLAY_TARGET_BRANCH=agent/v2r1c6-mainline-reconciliation-20260805 \
|
|
||||||
AIOS_GITOPS_IDENTITY_OVERLAY_COLLISION_SECRET_REFS=<verified-secret-ref-list> \
|
|
||||||
python3 <staging>/active-runtime/app/aios_task_identity_overlay_cli.py \
|
|
||||||
issue \
|
|
||||||
--ttl-seconds 900 \
|
|
||||||
--authorization-assertion-ref owner-session:V2R1C6:approved-20260806 \
|
|
||||||
--operator-identity <audited-agent-identity> \
|
|
||||||
--result-json /run/aios-gitops/v2r1c6-overlay-issued.json
|
|
||||||
```
|
|
||||||
|
|
||||||
The result must contain:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
status: OVERLAY_ACTIVE
|
|
||||||
idempotent: false
|
|
||||||
overlay_id: present
|
|
||||||
secret_ref: under /run/aios-gitops/task-identity-overlays
|
|
||||||
expires_at: within 900 seconds
|
|
||||||
```
|
|
||||||
|
|
||||||
It must not contain a token.
|
|
||||||
|
|
||||||
Verify:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
secret_file_mode: "0600"
|
|
||||||
metadata_contains_raw_token: false
|
|
||||||
token_hash_unique_across_configured_secret_refs: true
|
|
||||||
```
|
|
||||||
|
|
||||||
## 8. Negative tests
|
|
||||||
|
|
||||||
Before the valid request:
|
|
||||||
|
|
||||||
1. No Authorization header must return HTTP 401.
|
|
||||||
2. A random wrong bearer must return HTTP 401.
|
|
||||||
3. A request with `production_change_authorized=true` must be rejected.
|
|
||||||
4. A request with another base OID or branch must be rejected.
|
|
||||||
5. The valid overlay must remain `ACTIVE` after pre-auth request-validation
|
|
||||||
failures.
|
|
||||||
|
|
||||||
Do not use the global Owner or ChatGPT bearer credential for this endpoint.
|
|
||||||
|
|
||||||
## 9. Send the valid request
|
|
||||||
|
|
||||||
Use:
|
|
||||||
|
|
||||||
```text
|
|
||||||
deploy/v2r1c6-task-writer-authorization-request.json
|
|
||||||
```
|
|
||||||
|
|
||||||
Use the provided client so the bearer is not placed in argv, shell history or
|
|
||||||
logs:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo python3 <staging>/scripts/call_task_identity_overlay.py \
|
|
||||||
--secret-ref <secret_ref_from_issue_result> \
|
|
||||||
--request-json <staging>/deploy/v2r1c6-task-writer-authorization-request.json \
|
|
||||||
--url http://127.0.0.1:19864/api/governance/owner-authorizations \
|
|
||||||
--result-json /var/lib/aios-gitops/v2r1c6-overlay-provision-result.json
|
|
||||||
```
|
|
||||||
|
|
||||||
Required result:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
http_status: 200
|
|
||||||
status: READY_FOR_CODE_AUTHORING
|
|
||||||
owner_authorization_consumed: true
|
|
||||||
grant_consumed: true
|
|
||||||
identity_overlay_consumed: true
|
|
||||||
identity_overlay_secret_removed: true
|
|
||||||
branch: agent/v2r1c6-mainline-reconciliation-20260805
|
|
||||||
branch_oid: 875d41c3101093149414292067fd700dc214d4d3
|
|
||||||
```
|
|
||||||
|
|
||||||
The result must not include:
|
|
||||||
|
|
||||||
- bearer token;
|
|
||||||
- private key;
|
|
||||||
- signer secret;
|
|
||||||
- raw signature;
|
|
||||||
- credential value.
|
|
||||||
|
|
||||||
## 10. Independent acceptance
|
|
||||||
|
|
||||||
Verify the branch:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git --git-dir=/data/aios-gitops/repo.git \
|
|
||||||
rev-parse refs/heads/agent/v2r1c6-mainline-reconciliation-20260805
|
|
||||||
```
|
|
||||||
|
|
||||||
It must equal:
|
|
||||||
|
|
||||||
```text
|
|
||||||
875d41c3101093149414292067fd700dc214d4d3
|
|
||||||
```
|
|
||||||
|
|
||||||
Verify worktree evidence returned by the provisioner:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
branch: agent/v2r1c6-mainline-reconciliation-20260805
|
|
||||||
head_oid: 875d41c3101093149414292067fd700dc214d4d3
|
|
||||||
detached_head: false
|
|
||||||
worktree_clean: true
|
|
||||||
writer_status: active
|
|
||||||
writes_enabled: true
|
|
||||||
allowed_routes:
|
|
||||||
- POST /api/commit-file
|
|
||||||
allow_main_write: false
|
|
||||||
allow_tag_write: false
|
|
||||||
allow_baseline_write: false
|
|
||||||
allow_force_push: false
|
|
||||||
```
|
|
||||||
|
|
||||||
Run policy negative tests against the task writer and require explicit policy
|
|
||||||
denials for:
|
|
||||||
|
|
||||||
- main write;
|
|
||||||
- another branch;
|
|
||||||
- tag create/update;
|
|
||||||
- Baseline write;
|
|
||||||
- force push;
|
|
||||||
- branch delete.
|
|
||||||
|
|
||||||
Network errors, 404, 500 and stopped-service failures do not count as policy
|
|
||||||
denials.
|
|
||||||
|
|
||||||
## 11. Stop and clean up the overlay
|
|
||||||
|
|
||||||
After `READY_FOR_CODE_AUTHORING`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo systemctl stop aios-v2r1c6-task-identity-overlay.service
|
|
||||||
sudo systemctl disable aios-v2r1c6-task-identity-overlay.service || true
|
|
||||||
sudo rm -f /etc/systemd/system/aios-v2r1c6-task-identity-overlay.service
|
|
||||||
sudo systemctl daemon-reload
|
|
||||||
```
|
|
||||||
|
|
||||||
Verify:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
overlay_listener_stopped: true
|
|
||||||
overlay_secret_removed: true
|
|
||||||
global_token_registry_unchanged: true
|
|
||||||
disabled_owner_record_unchanged: true
|
|
||||||
active_chatgpt_bot_record_unchanged: true
|
|
||||||
audit_metadata_retained: true
|
|
||||||
```
|
|
||||||
|
|
||||||
The root-owned staging directory may be retained until the code is committed,
|
|
||||||
then removed after checksum-confirmed repository integration.
|
|
||||||
|
|
||||||
## 12. Commit the developed code through the task writer
|
|
||||||
|
|
||||||
Only after writer acceptance, use `POST /api/commit-file` to commit the
|
|
||||||
verified package contents to:
|
|
||||||
|
|
||||||
```text
|
|
||||||
agent/v2r1c6-mainline-reconciliation-20260805
|
|
||||||
```
|
|
||||||
|
|
||||||
Commit in reviewable groups:
|
|
||||||
|
|
||||||
1. authorized task-writer bridge;
|
|
||||||
2. Owner authorization broker;
|
|
||||||
3. task identity overlay;
|
|
||||||
4. tests, docs, OpenAPI and deployment templates.
|
|
||||||
|
|
||||||
Do not include:
|
|
||||||
|
|
||||||
- package archives;
|
|
||||||
- runtime tokens;
|
|
||||||
- `/run` or `/var/lib` state;
|
|
||||||
- test result files containing host paths;
|
|
||||||
- secret references containing values;
|
|
||||||
- generated credentials.
|
|
||||||
|
|
||||||
Run the complete 40-test suite from the branch after commits.
|
|
||||||
|
|
||||||
## 13. Protected PR
|
|
||||||
|
|
||||||
Create a protected PR from:
|
|
||||||
|
|
||||||
```text
|
|
||||||
agent/v2r1c6-mainline-reconciliation-20260805
|
|
||||||
```
|
|
||||||
|
|
||||||
to:
|
|
||||||
|
|
||||||
```text
|
|
||||||
main
|
|
||||||
```
|
|
||||||
|
|
||||||
The PR must state:
|
|
||||||
|
|
||||||
- this change closes Owner authorization bootstrap gaps;
|
|
||||||
- the overlay is localhost-only and one-time;
|
|
||||||
- the global token registry is not modified;
|
|
||||||
- no production release is authorized;
|
|
||||||
- V2R1C5 tag and Baseline remain unchanged.
|
|
||||||
|
|
||||||
The Agent must not merge without the repository's normal protected review.
|
|
||||||
|
|
||||||
## 14. Required Agent report
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
STATUS:
|
|
||||||
READY_FOR_CODE_AUTHORING | BLOCKED | FAILED
|
|
||||||
|
|
||||||
PACKAGE:
|
|
||||||
zip_sha256:
|
|
||||||
tar_gz_sha256:
|
|
||||||
manifest_entries:
|
|
||||||
manifest_matched:
|
|
||||||
|
|
||||||
TESTS:
|
|
||||||
unit_total:
|
|
||||||
unit_passed:
|
|
||||||
smoke_bridge:
|
|
||||||
smoke_broker:
|
|
||||||
smoke_overlay:
|
|
||||||
py_compile:
|
|
||||||
|
|
||||||
PRESTART:
|
|
||||||
main_oid:
|
|
||||||
target_branch_previously_absent:
|
|
||||||
V2R1C5_tag_oid:
|
|
||||||
V2R1C5_peeled_target:
|
|
||||||
baseline_unchanged:
|
|
||||||
|
|
||||||
OVERLAY:
|
|
||||||
overlay_id:
|
|
||||||
status:
|
|
||||||
secret_ref:
|
|
||||||
secret_file_mode:
|
|
||||||
token_hash_unique:
|
|
||||||
global_registry_used: false
|
|
||||||
listener:
|
|
||||||
expires_at:
|
|
||||||
consumed:
|
|
||||||
secret_removed:
|
|
||||||
|
|
||||||
NEGATIVE_TESTS:
|
|
||||||
unauthenticated_401:
|
|
||||||
wrong_token_401:
|
|
||||||
production_capability_denied:
|
|
||||||
wrong_base_denied:
|
|
||||||
wrong_branch_denied:
|
|
||||||
|
|
||||||
OWNER_AUTHORIZATION:
|
|
||||||
event_id:
|
|
||||||
reference:
|
|
||||||
consumed:
|
|
||||||
|
|
||||||
GRANT:
|
|
||||||
approval_plan_id:
|
|
||||||
grant_id:
|
|
||||||
grant_digest_sha256:
|
|
||||||
signature_valid:
|
|
||||||
consumed:
|
|
||||||
|
|
||||||
BRANCH:
|
|
||||||
name:
|
|
||||||
oid:
|
|
||||||
base_oid:
|
|
||||||
ref_exists:
|
|
||||||
worktree_path:
|
|
||||||
worktree_clean:
|
|
||||||
|
|
||||||
WRITER:
|
|
||||||
status:
|
|
||||||
writes_enabled:
|
|
||||||
allowed_branch:
|
|
||||||
allowed_base_oid:
|
|
||||||
allowed_routes:
|
|
||||||
main_write_denied:
|
|
||||||
tag_write_denied:
|
|
||||||
baseline_write_denied:
|
|
||||||
force_push_denied:
|
|
||||||
|
|
||||||
REPOSITORY_INTEGRATION:
|
|
||||||
commits:
|
|
||||||
tests_after_commit:
|
|
||||||
protected_pr:
|
|
||||||
merged: false
|
|
||||||
|
|
||||||
POSTCHECK:
|
|
||||||
main_unchanged:
|
|
||||||
V2R1C5_tag_unchanged:
|
|
||||||
baseline_unchanged:
|
|
||||||
global_token_registry_unchanged:
|
|
||||||
overlay_listener_stopped:
|
|
||||||
overlay_secret_removed:
|
|
||||||
|
|
||||||
SECRETS:
|
|
||||||
plaintext_logged: false
|
|
||||||
plaintext_committed: false
|
|
||||||
|
|
||||||
BLOCKERS:
|
|
||||||
|
|
||||||
GOVERNANCE:
|
|
||||||
force_push_used: false
|
|
||||||
tag_rewritten: false
|
|
||||||
direct_main_push: false
|
|
||||||
secrets_committed: false
|
|
||||||
owner_approval_respected: true
|
|
||||||
```
|
|
||||||
|
|
||||||
## Fail-closed conditions
|
|
||||||
|
|
||||||
Return `BLOCKED` and stop on any of:
|
|
||||||
|
|
||||||
- authoritative main OID drift;
|
|
||||||
- target branch conflict;
|
|
||||||
- inability to enumerate active secret references;
|
|
||||||
- secret hash collision;
|
|
||||||
- overlay not bound to the exact task/base/branch;
|
|
||||||
- non-loopback listener;
|
|
||||||
- raw secret in result or logs;
|
|
||||||
- Owner Authorization Broker event not created;
|
|
||||||
- signer/validator/provisioner failure;
|
|
||||||
- grant not consumed;
|
|
||||||
- branch OID mismatch;
|
|
||||||
- writer exposes main/tag/Baseline/force-push permissions;
|
|
||||||
- V2R1C5 tag or Baseline changes.
|
|
||||||
|
|
||||||
Never fall back to the shared `gitops-write-bot.token`, a wildcard credential,
|
|
||||||
manual `gitops.tokens` edits or a direct Git ref write.
|
|
||||||
@@ -1,9 +0,0 @@
|
|||||||
{
|
|
||||||
"schema": "aios.v2r1c7.engineering-build-receipt.v1",
|
|
||||||
"version": "V2R1C7",
|
|
||||||
"base_oid": "b356dec2ac5bd2a5f721aa73f8fd1902f9c65632",
|
|
||||||
"production_deployed": false,
|
|
||||||
"tag_created": false,
|
|
||||||
"baseline_changed": false,
|
|
||||||
"deployable": true
|
|
||||||
}
|
|
||||||
@@ -1,21 +0,0 @@
|
|||||||
bec7da1adb75bf5feaa63c60ec6d03adf15d7508742a486207033d000f72d4ff AGENT_DEPLOYMENT_REQUIREMENTS.md
|
|
||||||
251b46e51ce09aa2fd843bee48ef24a5608f31d24c599841f086ceb25c1a1715 README.md
|
|
||||||
783f68ec4f00a11d43ceed536196bfedccaf6614f6db9805f7b36d24e5fa7bc2 TEST_RESULTS.json
|
|
||||||
9208325e5b2e6f5ba6b64e000bc21d14d9f2f214e3ad33dd79857e85218fd6c2 VERSION
|
|
||||||
c4b24b65195077716983acdc8611f705324413a74453858994ba51541957afd4 active-runtime/app/aios_authorized_task_writer_api.py
|
|
||||||
8a1935fa098d47ac02ce51bbd9fd5fad7cf9ab9374ea3de5f630679b96681d84 active-runtime/app/authorized_task_writer_provisioning.py
|
|
||||||
0997ab8332985c7124f334c4e49735d8833326578acc61a61a389bf792424de3 active-runtime/app/deployment_state.py
|
|
||||||
dd3f8b1cab88e005a291f077807873c381e8ca92fe863dc05817925f0dff9613 active-runtime/app/v2r1c6_privacy_tools.py
|
|
||||||
9e45025c41068cafdea2f360edfc4146a5459b0a6bcfbf1403206fc0161e7bac active-runtime/app/v2r1c7_health_service.py
|
|
||||||
c96ba560d69f6c2eb700b9f2b42a32e394dadd00dffb23bbefdb9fb3e1571c0d deploy/aios-v2/systemd/aios-v2r1c7-production.service
|
|
||||||
a05d4e8038fdb866ce17d4fb8a54798a268a1e8d44ff2cbc559069fa6ab2a667 deploy/systemd/aios-gitops-model-api-authorized-writer-bridge.conf.example
|
|
||||||
a60dac51802bff67c2cfff61acdeb0a3a05d74a4f6cb47e3f78295ceb5c6acd9 docs/gitops/authorized-task-writer-provisioning.md
|
|
||||||
2690f76452bceddda90c728278c600bcebd533db24776e2966423c5a8655e4b5 docs/gitops/openapi-authorized-task-writer-fragment.yaml
|
|
||||||
560e5d9af8a8e2ec60ec31cf8659a7982b93ea09b7adb486b738ea2fba460d70 release-manifests/V2R1C7.json
|
|
||||||
81a423c7814f81cb888b60a538c153e549b2be9099c881c968d55ae84efddc57 scripts/build_v2r1c7_release_artifact.py
|
|
||||||
64c78dabffb05c8851f87b2a27c43bd1f21728dca586397d03a6bc22906a906c scripts/probe_v2r1c7_health.py
|
|
||||||
5419dc7de0f50908ad0ef1dac74172fce811bb7b45f314a6e1c1bb9ce618324d scripts/rollback_v2r1c7_release.sh
|
|
||||||
54e2845d2c3c026323ad45515e082d661e3b461feb3bfefedf5adc52db2d7b78 scripts/smoke_authorized_task_writer_provisioning.py
|
|
||||||
32a860e28d5afc105f413d878a13fb8600fa74874dbd3bc4c031ef3efa17896a scripts/verify_v2r1c7_release.py
|
|
||||||
3cce2b744326fe12b89bcad3cd352c77738950964bf8519e72cb5e9c7a5dfbfc tests/test_authorized_task_writer_provisioning.py
|
|
||||||
8f3153c625241f60b65109eeac045e8aed7661d95f3a10f0b32ba3e0e9ac0a85 versions/V2R1C7.json
|
|
||||||
@@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"git_branch_created": false,
|
|
||||||
"git_commit_created": false,
|
|
||||||
"production_mutation": false,
|
|
||||||
"py_compile": {
|
|
||||||
"files": [
|
|
||||||
"active-runtime/app/authorized_task_writer_provisioning.py",
|
|
||||||
"active-runtime/app/aios_authorized_task_writer_api.py"
|
|
||||||
],
|
|
||||||
"status": "PASS"
|
|
||||||
},
|
|
||||||
"schema": "aios.authorized-task-writer-bridge-test-report.v1",
|
|
||||||
"secrets_included": false,
|
|
||||||
"smoke": {
|
|
||||||
"command": "python3 scripts/smoke_authorized_task_writer_provisioning.py",
|
|
||||||
"status": "PASS"
|
|
||||||
},
|
|
||||||
"status": "PASS",
|
|
||||||
"unit_tests": {
|
|
||||||
"command": "python3 -m unittest -v tests/test_authorized_task_writer_provisioning.py",
|
|
||||||
"errors": 0,
|
|
||||||
"failures": 0,
|
|
||||||
"total": 9
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,201 +0,0 @@
|
|||||||
"""Fail-closed Agent Pull Runner primitives for AIOS autonomy v1.
|
|
||||||
|
|
||||||
The runner owns task discovery and reporting only. It never accepts shell
|
|
||||||
strings: executors receive argv, an allowlisted cwd and an allowlisted env.
|
|
||||||
Durable state is intentionally small so restart recovery and duplicate
|
|
||||||
suppression can be verified without coupling the runner to one database.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import random
|
|
||||||
import shutil
|
|
||||||
import subprocess
|
|
||||||
import tempfile
|
|
||||||
import time
|
|
||||||
from dataclasses import dataclass, field
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Callable, Mapping, Protocol, Sequence
|
|
||||||
|
|
||||||
|
|
||||||
class RunnerError(RuntimeError):
|
|
||||||
"""Raised when a runner contract would be violated."""
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class RunnerConfig:
|
|
||||||
worker_id: str
|
|
||||||
capabilities: tuple[str, ...] = ()
|
|
||||||
long_poll_seconds: int = 30
|
|
||||||
fallback_seconds: int = 60
|
|
||||||
jitter_seconds: int = 5
|
|
||||||
max_backoff_seconds: int = 300
|
|
||||||
executable_allowlist: tuple[str, ...] = ()
|
|
||||||
working_directory_allowlist: tuple[str, ...] = ()
|
|
||||||
environment_allowlist: tuple[str, ...] = ()
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class RunnerState:
|
|
||||||
inflight: dict[str, dict[str, Any]] = field(default_factory=dict)
|
|
||||||
terminal: dict[str, dict[str, Any]] = field(default_factory=dict)
|
|
||||||
|
|
||||||
|
|
||||||
class RunnerTransport(Protocol):
|
|
||||||
def register_worker(self, worker_id: str, capabilities: Sequence[str]) -> Mapping[str, Any]: ...
|
|
||||||
def heartbeat(self, worker_id: str) -> Mapping[str, Any]: ...
|
|
||||||
def long_poll(self, worker_id: str, timeout_seconds: int) -> Mapping[str, Any] | None: ...
|
|
||||||
def poll(self, worker_id: str) -> Mapping[str, Any] | None: ...
|
|
||||||
def claim(self, worker_id: str, task: Mapping[str, Any]) -> Mapping[str, Any]: ...
|
|
||||||
def renew(self, worker_id: str, task_id: str) -> Mapping[str, Any]: ...
|
|
||||||
def report_progress(self, worker_id: str, task_id: str, payload: Mapping[str, Any]) -> Mapping[str, Any]: ...
|
|
||||||
def report_result(self, worker_id: str, task_id: str, payload: Mapping[str, Any]) -> Mapping[str, Any]: ...
|
|
||||||
def report_evidence(self, worker_id: str, task_id: str, payload: Mapping[str, Any]) -> Mapping[str, Any]: ...
|
|
||||||
|
|
||||||
|
|
||||||
class JsonStateStore:
|
|
||||||
"""Atomic JSON state store for restart recovery."""
|
|
||||||
|
|
||||||
def __init__(self, path: str | os.PathLike[str]) -> None:
|
|
||||||
self.path = Path(path)
|
|
||||||
|
|
||||||
def load(self) -> RunnerState:
|
|
||||||
if not self.path.exists():
|
|
||||||
return RunnerState()
|
|
||||||
value = json.loads(self.path.read_text(encoding="utf-8"))
|
|
||||||
if not isinstance(value, dict):
|
|
||||||
raise RunnerError("runner state must be an object")
|
|
||||||
return RunnerState(
|
|
||||||
inflight=dict(value.get("inflight", {})),
|
|
||||||
terminal=dict(value.get("terminal", {})),
|
|
||||||
)
|
|
||||||
|
|
||||||
def save(self, state: RunnerState) -> None:
|
|
||||||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
fd, name = tempfile.mkstemp(prefix=self.path.name + ".", dir=self.path.parent)
|
|
||||||
try:
|
|
||||||
with os.fdopen(fd, "w", encoding="utf-8") as handle:
|
|
||||||
json.dump({"inflight": state.inflight, "terminal": state.terminal}, handle, sort_keys=True)
|
|
||||||
handle.write("\n")
|
|
||||||
os.chmod(name, 0o600)
|
|
||||||
os.replace(name, self.path)
|
|
||||||
finally:
|
|
||||||
if os.path.exists(name):
|
|
||||||
os.unlink(name)
|
|
||||||
|
|
||||||
|
|
||||||
class AgentPullRunner:
|
|
||||||
"""Pull, claim, execute and report one task at a time."""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
config: RunnerConfig,
|
|
||||||
transport: RunnerTransport,
|
|
||||||
state_store: JsonStateStore | None = None,
|
|
||||||
*,
|
|
||||||
random_fn: Callable[[], float] = random.random,
|
|
||||||
) -> None:
|
|
||||||
if not config.worker_id.strip():
|
|
||||||
raise RunnerError("worker_id is required")
|
|
||||||
self.config = config
|
|
||||||
self.transport = transport
|
|
||||||
self.state_store = state_store
|
|
||||||
self.random_fn = random_fn
|
|
||||||
self.state = state_store.load() if state_store else RunnerState()
|
|
||||||
|
|
||||||
def _save(self) -> None:
|
|
||||||
if self.state_store:
|
|
||||||
self.state_store.save(self.state)
|
|
||||||
|
|
||||||
def register(self) -> Mapping[str, Any]:
|
|
||||||
return self.transport.register_worker(self.config.worker_id, self.config.capabilities)
|
|
||||||
|
|
||||||
def heartbeat(self) -> Mapping[str, Any]:
|
|
||||||
return self.transport.heartbeat(self.config.worker_id)
|
|
||||||
|
|
||||||
def discovery_delay(self, failures: int) -> float:
|
|
||||||
failures = max(0, int(failures))
|
|
||||||
base = min(self.config.max_backoff_seconds, self.config.fallback_seconds * (2**failures))
|
|
||||||
jitter = self.random_fn() * max(0, self.config.jitter_seconds)
|
|
||||||
return min(self.config.max_backoff_seconds, base + jitter)
|
|
||||||
|
|
||||||
def discover_once(self) -> Mapping[str, Any] | None:
|
|
||||||
try:
|
|
||||||
task = self.transport.long_poll(self.config.worker_id, self.config.long_poll_seconds)
|
|
||||||
except Exception:
|
|
||||||
task = None
|
|
||||||
if task is None:
|
|
||||||
task = self.transport.poll(self.config.worker_id)
|
|
||||||
if not task:
|
|
||||||
return None
|
|
||||||
task_id = str(task.get("task_id", ""))
|
|
||||||
if not task_id:
|
|
||||||
raise RunnerError("task_id is required")
|
|
||||||
if task_id in self.state.terminal:
|
|
||||||
return {"status": "duplicate_suppressed", "task_id": task_id}
|
|
||||||
claimed = dict(self.transport.claim(self.config.worker_id, task))
|
|
||||||
self.state.inflight[task_id] = claimed
|
|
||||||
self._save()
|
|
||||||
return claimed
|
|
||||||
|
|
||||||
def renew(self, task_id: str) -> Mapping[str, Any]:
|
|
||||||
if task_id not in self.state.inflight:
|
|
||||||
raise RunnerError("task is not inflight")
|
|
||||||
return self.transport.renew(self.config.worker_id, task_id)
|
|
||||||
|
|
||||||
def report_progress(self, task_id: str, payload: Mapping[str, Any]) -> Mapping[str, Any]:
|
|
||||||
if task_id not in self.state.inflight:
|
|
||||||
raise RunnerError("task is not inflight")
|
|
||||||
return self.transport.report_progress(self.config.worker_id, task_id, payload)
|
|
||||||
|
|
||||||
def report_evidence(self, task_id: str, payload: Mapping[str, Any]) -> Mapping[str, Any]:
|
|
||||||
if task_id not in self.state.inflight:
|
|
||||||
raise RunnerError("task is not inflight")
|
|
||||||
return self.transport.report_evidence(self.config.worker_id, task_id, payload)
|
|
||||||
|
|
||||||
def report_result(self, task_id: str, payload: Mapping[str, Any]) -> Mapping[str, Any]:
|
|
||||||
if task_id in self.state.terminal:
|
|
||||||
return {"status": "duplicate_terminal_suppressed", "task_id": task_id}
|
|
||||||
if task_id not in self.state.inflight:
|
|
||||||
raise RunnerError("task is not inflight")
|
|
||||||
result = dict(self.transport.report_result(self.config.worker_id, task_id, payload))
|
|
||||||
self.state.terminal[task_id] = {"result": result, "reported_at": time.time()}
|
|
||||||
self.state.inflight.pop(task_id, None)
|
|
||||||
self._save()
|
|
||||||
return result
|
|
||||||
|
|
||||||
def recover_after_restart(self) -> list[Mapping[str, Any]]:
|
|
||||||
recovered: list[Mapping[str, Any]] = []
|
|
||||||
for task_id in tuple(self.state.inflight):
|
|
||||||
recovered.append(self.renew(task_id))
|
|
||||||
return recovered
|
|
||||||
|
|
||||||
def validate_execution(self, argv: Sequence[str], cwd: str | os.PathLike[str], env: Mapping[str, str]) -> None:
|
|
||||||
if not argv or any(not isinstance(item, str) or not item for item in argv):
|
|
||||||
raise RunnerError("argv_only_execution_required")
|
|
||||||
if any(ch in argv[0] for ch in (";", "|", "&", "`", "\n")):
|
|
||||||
raise RunnerError("shell_syntax_denied")
|
|
||||||
executable = shutil.which(argv[0])
|
|
||||||
if not executable:
|
|
||||||
raise RunnerError("executable_not_found")
|
|
||||||
if self.config.executable_allowlist and not any(
|
|
||||||
os.path.realpath(executable).startswith(os.path.realpath(prefix).rstrip(os.sep) + os.sep)
|
|
||||||
or os.path.realpath(executable) == os.path.realpath(prefix)
|
|
||||||
for prefix in self.config.executable_allowlist
|
|
||||||
):
|
|
||||||
raise RunnerError("executable_not_allowlisted")
|
|
||||||
real_cwd = os.path.realpath(os.fspath(cwd))
|
|
||||||
if self.config.working_directory_allowlist and not any(
|
|
||||||
real_cwd == os.path.realpath(prefix)
|
|
||||||
or real_cwd.startswith(os.path.realpath(prefix).rstrip(os.sep) + os.sep)
|
|
||||||
for prefix in self.config.working_directory_allowlist
|
|
||||||
):
|
|
||||||
raise RunnerError("working_directory_not_allowlisted")
|
|
||||||
if self.config.environment_allowlist and any(key not in self.config.environment_allowlist for key in env):
|
|
||||||
raise RunnerError("environment_key_not_allowlisted")
|
|
||||||
|
|
||||||
def execute_argv(self, argv: Sequence[str], cwd: str | os.PathLike[str], env: Mapping[str, str]) -> Mapping[str, Any]:
|
|
||||||
self.validate_execution(argv, cwd, env)
|
|
||||||
completed = subprocess.run(list(argv), cwd=os.fspath(cwd), env=dict(env), shell=False, text=True, capture_output=True, check=False)
|
|
||||||
return {"returncode": completed.returncode, "stdout": completed.stdout, "stderr": completed.stderr}
|
|
||||||
@@ -1,36 +0,0 @@
|
|||||||
"""Small operational CLI for the Agent Pull Runner contract."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
|
|
||||||
from agent_pull_runner import AgentPullRunner, RunnerConfig
|
|
||||||
|
|
||||||
|
|
||||||
class _NoopTransport:
|
|
||||||
def register_worker(self, worker_id, capabilities): return {"status": "registered", "worker_id": worker_id, "capabilities": list(capabilities)}
|
|
||||||
def heartbeat(self, worker_id): return {"status": "alive", "worker_id": worker_id}
|
|
||||||
def long_poll(self, worker_id, timeout_seconds): return None
|
|
||||||
def poll(self, worker_id): return None
|
|
||||||
def claim(self, worker_id, task): return task
|
|
||||||
def renew(self, worker_id, task_id): return {"status": "renewed", "task_id": task_id}
|
|
||||||
def report_progress(self, worker_id, task_id, payload): return {"status": "progress_recorded", "task_id": task_id}
|
|
||||||
def report_result(self, worker_id, task_id, payload): return {"status": "result_recorded", "task_id": task_id}
|
|
||||||
def report_evidence(self, worker_id, task_id, payload): return {"status": "evidence_recorded", "task_id": task_id}
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
parser = argparse.ArgumentParser(description="AIOS Agent Pull Runner")
|
|
||||||
parser.add_argument("--worker-id", default="self-test-worker")
|
|
||||||
parser.add_argument("--self-test", action="store_true")
|
|
||||||
args = parser.parse_args()
|
|
||||||
runner = AgentPullRunner(RunnerConfig(worker_id=args.worker_id, fallback_seconds=60), _NoopTransport())
|
|
||||||
if args.self_test:
|
|
||||||
print(json.dumps({"status": "PASS", "registration": runner.register(), "fallback_seconds": runner.config.fallback_seconds}, sort_keys=True))
|
|
||||||
return 0
|
|
||||||
print(json.dumps({"status": "READY", "worker_id": args.worker_id}, sort_keys=True))
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -1,173 +0,0 @@
|
|||||||
"""Fail-closed Authority Git side-effect boundary.
|
|
||||||
|
|
||||||
Only this module may issue the narrowly scoped ref CAS primitives used by the
|
|
||||||
durable executor. It never accepts force updates, arbitrary refs, tags, or
|
|
||||||
direct production mutations.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import re
|
|
||||||
import subprocess
|
|
||||||
import tempfile
|
|
||||||
from typing import Mapping, Sequence
|
|
||||||
|
|
||||||
|
|
||||||
OID_RE = re.compile(r"^[0-9a-f]{40}$")
|
|
||||||
TASK_BRANCH_RE = re.compile(r"^refs/heads/agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$")
|
|
||||||
|
|
||||||
|
|
||||||
class AuthorityGitError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def _oid(value: str, field: str) -> str:
|
|
||||||
if not isinstance(value, str) or not OID_RE.fullmatch(value):
|
|
||||||
raise AuthorityGitError(f"{field}_INVALID")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class AuthorityGitExecutor:
|
|
||||||
git_dir: Path
|
|
||||||
worktree_root: Path = Path("/var/lib/aios-gitops/worktrees")
|
|
||||||
|
|
||||||
def __post_init__(self) -> None:
|
|
||||||
if not self.git_dir.is_absolute() or self.git_dir.is_symlink():
|
|
||||||
raise AuthorityGitError("AUTHORITY_GIT_DIR_INVALID")
|
|
||||||
if not self.worktree_root.is_absolute() or self.worktree_root.is_symlink():
|
|
||||||
raise AuthorityGitError("WORKTREE_ROOT_INVALID")
|
|
||||||
|
|
||||||
def _git(self, args: Sequence[str], *, check: bool = True) -> str:
|
|
||||||
result = subprocess.run(
|
|
||||||
["git", f"--git-dir={self.git_dir}", *args],
|
|
||||||
text=True,
|
|
||||||
capture_output=True,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
if check and result.returncode != 0:
|
|
||||||
raise AuthorityGitError("AUTHORITY_GIT_COMMAND_FAILED")
|
|
||||||
return result.stdout.strip()
|
|
||||||
|
|
||||||
def read_ref(self, ref: str) -> str | None:
|
|
||||||
if not ref.startswith("refs/") or ref.startswith(("refs/tags/", "refs/remotes/")):
|
|
||||||
raise AuthorityGitError("REF_READ_SCOPE_DENIED")
|
|
||||||
value = self._git(["rev-parse", "--verify", ref], check=False)
|
|
||||||
return value if value and OID_RE.fullmatch(value) else None
|
|
||||||
|
|
||||||
def commit_exists(self, oid: str) -> bool:
|
|
||||||
_oid(oid, "commit_oid")
|
|
||||||
result = subprocess.run(["git", f"--git-dir={self.git_dir}", "cat-file", "-e", oid + "^{commit}"], capture_output=True, check=False)
|
|
||||||
return result.returncode == 0
|
|
||||||
|
|
||||||
def ancestry_check(self, older_oid: str, newer_oid: str) -> bool:
|
|
||||||
_oid(older_oid, "older_oid")
|
|
||||||
_oid(newer_oid, "newer_oid")
|
|
||||||
result = subprocess.run(
|
|
||||||
["git", f"--git-dir={self.git_dir}", "merge-base", "--is-ancestor", older_oid, newer_oid],
|
|
||||||
capture_output=True,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
if result.returncode not in (0, 1):
|
|
||||||
raise AuthorityGitError("ANCESTRY_CHECK_FAILED")
|
|
||||||
return result.returncode == 0
|
|
||||||
|
|
||||||
def create_detached_worktree(
|
|
||||||
self,
|
|
||||||
base_oid: str,
|
|
||||||
worktree_name: str,
|
|
||||||
root: Path | None = None,
|
|
||||||
) -> Path:
|
|
||||||
_oid(base_oid, "base_oid")
|
|
||||||
if not worktree_name or "/" in worktree_name or "\\" in worktree_name or worktree_name in {".", ".."}:
|
|
||||||
raise AuthorityGitError("WORKTREE_NAME_INVALID")
|
|
||||||
parent = root or self.worktree_root
|
|
||||||
if not parent.is_absolute() or parent.is_symlink():
|
|
||||||
raise AuthorityGitError("WORKTREE_ROOT_INVALID")
|
|
||||||
path = parent / worktree_name
|
|
||||||
if path.exists() or path.is_symlink():
|
|
||||||
raise AuthorityGitError("WORKTREE_ALREADY_EXISTS")
|
|
||||||
parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
self._git(["worktree", "add", "--detach", str(path), base_oid])
|
|
||||||
return path
|
|
||||||
|
|
||||||
def advance_task_branch_CAS(self, branch: str, expected_oid: str, new_oid: str) -> str:
|
|
||||||
if not TASK_BRANCH_RE.fullmatch(branch):
|
|
||||||
raise AuthorityGitError("TASK_BRANCH_SCOPE_DENIED")
|
|
||||||
_oid(expected_oid, "expected_oid")
|
|
||||||
_oid(new_oid, "new_oid")
|
|
||||||
current = self.read_ref(branch)
|
|
||||||
if current != expected_oid:
|
|
||||||
if current == new_oid:
|
|
||||||
return new_oid
|
|
||||||
raise AuthorityGitError("AUTHORITY_BRANCH_EXPECTATION_MISMATCH")
|
|
||||||
if not self.ancestry_check(expected_oid, new_oid):
|
|
||||||
raise AuthorityGitError("NEW_OID_NOT_DESCENDANT")
|
|
||||||
self._git(["update-ref", branch, new_oid, expected_oid])
|
|
||||||
readback = self.read_ref(branch)
|
|
||||||
if readback != new_oid:
|
|
||||||
raise AuthorityGitError("AUTHORITY_CAS_READBACK_MISMATCH")
|
|
||||||
return readback
|
|
||||||
|
|
||||||
def ff_CAS_main_after_verified_protected_merge(self, expected_oid: str, new_oid: str) -> str:
|
|
||||||
_oid(expected_oid, "expected_oid")
|
|
||||||
_oid(new_oid, "new_oid")
|
|
||||||
if not self.ancestry_check(expected_oid, new_oid):
|
|
||||||
raise AuthorityGitError("MAIN_FF_REQUIRED")
|
|
||||||
current = self.read_ref("refs/heads/main")
|
|
||||||
if current != expected_oid:
|
|
||||||
if current == new_oid:
|
|
||||||
return new_oid
|
|
||||||
raise AuthorityGitError("AUTHORITY_MAIN_EXPECTATION_MISMATCH")
|
|
||||||
self._git(["update-ref", "refs/heads/main", new_oid, expected_oid])
|
|
||||||
readback = self.read_ref("refs/heads/main")
|
|
||||||
if readback != new_oid:
|
|
||||||
raise AuthorityGitError("AUTHORITY_MAIN_CAS_READBACK_MISMATCH")
|
|
||||||
return readback
|
|
||||||
|
|
||||||
def commit_file_in_isolated_worktree(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
base_oid: str,
|
|
||||||
files: Mapping[str, bytes],
|
|
||||||
message: str,
|
|
||||||
worktree_name: str,
|
|
||||||
) -> str:
|
|
||||||
_oid(base_oid, "base_oid")
|
|
||||||
if not worktree_name or "/" in worktree_name or "\\" in worktree_name or worktree_name in {".", ".."}:
|
|
||||||
raise AuthorityGitError("WORKTREE_NAME_INVALID")
|
|
||||||
root = self.worktree_root / worktree_name
|
|
||||||
if root.exists() or root.is_symlink():
|
|
||||||
raise AuthorityGitError("WORKTREE_ALREADY_EXISTS")
|
|
||||||
self.worktree_root.mkdir(parents=True, exist_ok=True)
|
|
||||||
self._git(["worktree", "add", "--detach", str(root), base_oid])
|
|
||||||
try:
|
|
||||||
for relative, content in files.items():
|
|
||||||
path = Path(relative)
|
|
||||||
if path.is_absolute() or ".." in path.parts or not path.parts or path.parts[0] == ".git":
|
|
||||||
raise AuthorityGitError("FILE_PATH_INVALID")
|
|
||||||
target = root / path
|
|
||||||
target.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
target.write_bytes(content)
|
|
||||||
result = subprocess.run(
|
|
||||||
["git", "-C", str(root), "add", "--", *files.keys()],
|
|
||||||
text=True,
|
|
||||||
capture_output=True,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
if result.returncode != 0:
|
|
||||||
raise AuthorityGitError("WORKTREE_ADD_FAILED")
|
|
||||||
result = subprocess.run(
|
|
||||||
["git", "-C", str(root), "-c", "user.name=aios-durable-job", "-c", "user.email=aios-durable-job@localhost", "commit", "--no-verify", "-m", message],
|
|
||||||
text=True,
|
|
||||||
capture_output=True,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
if result.returncode != 0:
|
|
||||||
raise AuthorityGitError("WORKTREE_COMMIT_FAILED")
|
|
||||||
return subprocess.check_output(["git", "-C", str(root), "rev-parse", "HEAD"], text=True).strip()
|
|
||||||
finally:
|
|
||||||
subprocess.run(["git", f"--git-dir={self.git_dir}", "worktree", "remove", "--force", str(root)], capture_output=True, check=False)
|
|
||||||
@@ -1,73 +0,0 @@
|
|||||||
"""HTTP adapter for Owner-authorized task writer provisioning.
|
|
||||||
|
|
||||||
Run this module instead of aios_chatgpt_actions_api.py. All existing routes are
|
|
||||||
delegated unchanged; only POST /api/task-writer/provision-authorized is added.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import urllib.parse
|
|
||||||
|
|
||||||
import aios_chatgpt_actions_api as api
|
|
||||||
from authorized_task_writer_provisioning import (
|
|
||||||
AuthorizedTaskWriterProvisioner,
|
|
||||||
OwnerContext,
|
|
||||||
ProvisioningError,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
_original_do_post = api.H.do_POST
|
|
||||||
_ENDPOINT = "/api/task-writer/provision-authorized"
|
|
||||||
|
|
||||||
|
|
||||||
def _owner_authorization(handler):
|
|
||||||
scope = os.getenv(
|
|
||||||
"AIOS_GITOPS_OWNER_PROVISION_SCOPE", "writer:provision"
|
|
||||||
)
|
|
||||||
auth = handler.need(scope)
|
|
||||||
if not auth:
|
|
||||||
return None
|
|
||||||
if auth.get("role") != "owner":
|
|
||||||
handler.sendj({"error": "OWNER_AUTHORIZATION_REQUIRED"}, 403)
|
|
||||||
return None
|
|
||||||
identity = str(auth.get("account") or "").strip()
|
|
||||||
if not identity:
|
|
||||||
handler.sendj({"error": "OWNER_IDENTITY_REQUIRED"}, 403)
|
|
||||||
return None
|
|
||||||
return auth
|
|
||||||
|
|
||||||
|
|
||||||
def _authorized_do_post(self):
|
|
||||||
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
|
|
||||||
if path != _ENDPOINT:
|
|
||||||
return _original_do_post(self)
|
|
||||||
auth = _owner_authorization(self)
|
|
||||||
if not auth:
|
|
||||||
return
|
|
||||||
data = self.body()
|
|
||||||
owner_reference = str(data.get("owner_authorization_reference") or "")
|
|
||||||
try:
|
|
||||||
result = AuthorizedTaskWriterProvisioner().provision(
|
|
||||||
data,
|
|
||||||
OwnerContext(
|
|
||||||
identity=str(auth["account"]),
|
|
||||||
role=str(auth["role"]),
|
|
||||||
authorization_reference=owner_reference,
|
|
||||||
),
|
|
||||||
)
|
|
||||||
return self.sendj(result)
|
|
||||||
except ProvisioningError as exc:
|
|
||||||
body = {"error": exc.code}
|
|
||||||
if exc.details:
|
|
||||||
body["details"] = exc.details
|
|
||||||
return self.sendj(body, exc.status)
|
|
||||||
except Exception:
|
|
||||||
# Do not expose tool output, paths containing credentials, or traceback data.
|
|
||||||
return self.sendj({"error": "AUTHORIZED_WRITER_BRIDGE_INTERNAL_ERROR"}, 500)
|
|
||||||
|
|
||||||
|
|
||||||
api.H.do_POST = _authorized_do_post
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
api.main()
|
|
||||||
@@ -0,0 +1,379 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Task-scoped continuous authorization controller for the AIOS Candidate writer."""
|
||||||
|
from __future__ import annotations
|
||||||
|
import base64, fcntl, hashlib, json, os, pwd, grp, re, socket, sqlite3, stat, subprocess, sys, tempfile, time, urllib.error, urllib.parse, urllib.request
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from pathlib import Path, PurePosixPath, PureWindowsPath
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
TASK='AIOS-AUTONOMOUS-RELEASE-FULL-IMPLEMENTATION-20260726'
|
||||||
|
AUTH_ID='authz-aios-autonomous-release-20260726'
|
||||||
|
BRANCH='patch/chatgpt-autonomous-completion-contract-20260726'
|
||||||
|
BASE='875d41c3101093149414292067fd700dc214d4d3'
|
||||||
|
REPO='aios/aios'
|
||||||
|
WRITER_AUDIENCE='chatgpt-autonomous-release-writer'
|
||||||
|
ACTOR='chatgpt-gitops-bot'
|
||||||
|
WRITER_INSTANCE='writer-6e32f321-1240-46f8-9693-0b0631d9c77b'
|
||||||
|
CHILD_OPERATION_MAX_USES=20
|
||||||
|
ROOT=Path('/data/aios-gitops')
|
||||||
|
WORKTREE=ROOT/'worktrees/chatgpt-autonomous-completion-contract-20260726'
|
||||||
|
BARE=ROOT/'repo.git'
|
||||||
|
TASK_ROOT=ROOT/'runtime/tasks'/TASK
|
||||||
|
AUTH_DIR=Path('/etc/aios-gitops/task-authorizations')
|
||||||
|
PARENT=AUTH_DIR/'authz-aios-autonomous-release-20260726.json'
|
||||||
|
STATE_DIR=Path('/var/lib/aios-autonomous-release-authorization')
|
||||||
|
STATE=STATE_DIR/'state.json'
|
||||||
|
LOCK=STATE_DIR/'controller.lock'
|
||||||
|
AUDIT=TASK_ROOT/'continuous-authorization.audit.jsonl'
|
||||||
|
SOCKET_DIR=Path('/run/aios-autonomous-release-authorization')
|
||||||
|
SOCKET_PATH=SOCKET_DIR/'controller.sock'
|
||||||
|
PROJECTION=Path('/etc/aios-gitops/credentials/chatgpt-autonomous-release-writer')
|
||||||
|
REGISTRY=Path('/var/lib/aios-chatgpt-autonomous-release-writer/token-registry.sqlite3')
|
||||||
|
WRITER_LEDGER=Path('/var/lib/aios-chatgpt-autonomous-release-writer/operation-ledger.sqlite3')
|
||||||
|
ASKPASS=Path('/usr/libexec/aios-autonomous-release-authorization/gitea_askpass.py')
|
||||||
|
PATH_POLICY=Path('/etc/aios-gitops/policies-chatgpt-autonomous-release/autonomous-release-path-policy.json')
|
||||||
|
ROUTE_BINDING=Path('/etc/aios-gitops/action-route-bindings/chatgpt-autonomous-release-20260726.json')
|
||||||
|
OID_RE=re.compile(r'^[0-9a-f]{40}$')
|
||||||
|
OP_RE=re.compile(r'^[A-Za-z0-9][A-Za-z0-9._:-]{7,127}$')
|
||||||
|
TERMINAL={'COMPLETED','CANCELLED','FAILED_FINAL','REVOKED','REVOKED_IDLE_TIMEOUT','REVOKED_WALL_CLOCK','RECONCILIATION_REQUIRED'}
|
||||||
|
|
||||||
|
sys.path.insert(0,'/usr/libexec/aios-chatgpt-autonomous-release-writer-fullprofile-8ac2f90f')
|
||||||
|
from isolated_writer_token_registry import TokenRegistry
|
||||||
|
|
||||||
|
def now() -> datetime: return datetime.now(timezone.utc)
|
||||||
|
def stamp() -> str: return now().replace(microsecond=0).isoformat()
|
||||||
|
def canonical(value: Any) -> bytes: return json.dumps(value,sort_keys=True,separators=(',',':'),ensure_ascii=False).encode('utf-8')
|
||||||
|
def fail(code: str, status: int=503, **details: Any) -> None: raise ControlError(code,status,details)
|
||||||
|
class ControlError(RuntimeError):
|
||||||
|
def __init__(self,code:str,status:int=503,details:dict[str,Any]|None=None): self.code,self.status,self.details=code,status,details or {}; super().__init__(code)
|
||||||
|
|
||||||
|
def owned_regular(path:Path, mode:int) -> os.stat_result:
|
||||||
|
try: info=os.lstat(path)
|
||||||
|
except FileNotFoundError: fail('authorization_config_missing')
|
||||||
|
if not stat.S_ISREG(info.st_mode) or stat.S_ISLNK(info.st_mode) or info.st_uid!=0 or stat.S_IMODE(info.st_mode)&~mode: fail('authorization_config_permissions_rejected')
|
||||||
|
return info
|
||||||
|
|
||||||
|
def load_json(path:Path,mode:int=0o600) -> dict[str,Any]:
|
||||||
|
owned_regular(path,mode)
|
||||||
|
try: value=json.loads(path.read_text(encoding='utf-8'))
|
||||||
|
except Exception: fail('authorization_config_json_rejected')
|
||||||
|
if not isinstance(value,dict): fail('authorization_config_schema_rejected')
|
||||||
|
return value
|
||||||
|
|
||||||
|
def atomic_json(path:Path,value:dict[str,Any],mode:int=0o600) -> None:
|
||||||
|
path.parent.mkdir(parents=True,exist_ok=True)
|
||||||
|
raw=canonical(value)+b'\n'
|
||||||
|
fd,tmp=tempfile.mkstemp(prefix='.new-',dir=path.parent)
|
||||||
|
try:
|
||||||
|
os.fchmod(fd,mode); os.write(fd,raw); os.fsync(fd); os.close(fd); os.chown(tmp,0,0); os.replace(tmp,path)
|
||||||
|
os.chmod(path,mode)
|
||||||
|
finally:
|
||||||
|
try: os.close(fd)
|
||||||
|
except OSError: pass
|
||||||
|
if os.path.exists(tmp): os.unlink(tmp)
|
||||||
|
|
||||||
|
def append_audit(event:dict[str,Any]) -> None:
|
||||||
|
TASK_ROOT.mkdir(parents=True,exist_ok=True)
|
||||||
|
flags=os.O_WRONLY|os.O_CREAT|os.O_APPEND|getattr(os,'O_NOFOLLOW',0)
|
||||||
|
fd=os.open(AUDIT,flags,0o600)
|
||||||
|
try: os.write(fd,canonical({'at':stamp(),'task_id':TASK,'secrets_included':False,**event})+b'\n'); os.fsync(fd)
|
||||||
|
finally: os.close(fd)
|
||||||
|
|
||||||
|
def parent() -> dict[str,Any]:
|
||||||
|
p=load_json(PARENT)
|
||||||
|
required={'schema_version','development_authorization_id','authorization_type','task_id','actor_identity','audience','repository','exact_branch','approved_base_oid','allowed_path_roots','allowed_exact_paths','allowed_operations','created_at','valid_until','maximum_wall_clock_seconds','idle_timeout_seconds','manual_reapproval_per_commit','maximum_concurrent_writes','path_policy_sha256','writer_instance_id'}
|
||||||
|
if set(p)!=required or p.get('schema_version')!=1: fail('parent_authorization_schema_rejected')
|
||||||
|
fixed={'development_authorization_id':AUTH_ID,'authorization_type':'TASK_SCOPED_CONTINUOUS_DEVELOPMENT','task_id':TASK,'actor_identity':ACTOR,'audience':WRITER_AUDIENCE,'repository':REPO,'exact_branch':BRANCH,'approved_base_oid':BASE,'manual_reapproval_per_commit':False,'maximum_concurrent_writes':1,'writer_instance_id':WRITER_INSTANCE}
|
||||||
|
if any(p.get(k)!=v for k,v in fixed.items()): fail('parent_authorization_binding_rejected')
|
||||||
|
if p.get('maximum_wall_clock_seconds')!=86400 or p.get('idle_timeout_seconds')!=2700: fail('parent_authorization_lifetime_rejected')
|
||||||
|
if not isinstance(p.get('allowed_path_roots'),list) or not isinstance(p.get('allowed_exact_paths'),list) or not isinstance(p.get('allowed_operations'),list): fail('parent_authorization_scope_rejected')
|
||||||
|
raw=PATH_POLICY.read_bytes()
|
||||||
|
if hashlib.sha256(raw).hexdigest()!=p.get('path_policy_sha256'): fail('parent_authorization_path_policy_changed')
|
||||||
|
return p
|
||||||
|
|
||||||
|
def parse_time(v:Any) -> datetime:
|
||||||
|
try:
|
||||||
|
d=datetime.fromisoformat(str(v).replace('Z','+00:00'))
|
||||||
|
return d.astimezone(timezone.utc)
|
||||||
|
except Exception: fail('authorization_time_rejected')
|
||||||
|
|
||||||
|
def components(path:str)->tuple[str,...]:
|
||||||
|
if not isinstance(path,str) or not path or path!=path.strip() or '\\' in path or '\x00' in path: fail('authorization_path_rejected',403)
|
||||||
|
pp=PurePosixPath(path); wp=PureWindowsPath(path)
|
||||||
|
parts=tuple(path.split('/'))
|
||||||
|
if pp.is_absolute() or wp.is_absolute() or bool(wp.drive) or any(x in {'','.','..'} for x in parts): fail('authorization_path_rejected',403)
|
||||||
|
return parts
|
||||||
|
|
||||||
|
def path_allowed(p:dict[str,Any],path:str)->bool:
|
||||||
|
item=components(path)
|
||||||
|
for root in p['allowed_path_roots']:
|
||||||
|
r=components(str(root).rstrip('/'))
|
||||||
|
if item[:len(r)]==r: return True
|
||||||
|
return any(item==components(x) for x in p['allowed_exact_paths'])
|
||||||
|
|
||||||
|
def git(*args:str,cwd:Path|None=None,env:dict[str,str]|None=None)->str:
|
||||||
|
e=os.environ.copy(); e.update(env or {}); e['GIT_CONFIG_COUNT']='1'; e['GIT_CONFIG_KEY_0']='safe.directory'; e['GIT_CONFIG_VALUE_0']=str(WORKTREE)
|
||||||
|
r=subprocess.run(['git',*args],cwd=cwd,text=True,capture_output=True,env=e)
|
||||||
|
if r.returncode: fail('authorization_git_probe_failed',503)
|
||||||
|
return r.stdout.strip()
|
||||||
|
|
||||||
|
def credential_lines()->list[str]:
|
||||||
|
cp=Path(os.environ.get('CREDENTIALS_DIRECTORY',''))/'gitea_git_http'
|
||||||
|
try:
|
||||||
|
info=os.lstat(cp); lines=cp.read_text(encoding='utf-8').splitlines()
|
||||||
|
except OSError: fail('authorization_gitea_credential_unavailable')
|
||||||
|
if not stat.S_ISREG(info.st_mode) or stat.S_ISLNK(info.st_mode) or len(lines)!=2 or lines[0]!='gitops-bot' or not lines[1]: fail('authorization_gitea_credential_rejected')
|
||||||
|
return lines
|
||||||
|
|
||||||
|
def remote_oid()->str:
|
||||||
|
credential_lines()
|
||||||
|
e={'GIT_ASKPASS':str(ASKPASS),'GIT_TERMINAL_PROMPT':'0'}
|
||||||
|
out=git('ls-remote','--heads','http://127.0.0.1:3000/aios/aios.git',f'refs/heads/{BRANCH}',cwd=WORKTREE,env=e)
|
||||||
|
fields=out.split()
|
||||||
|
if len(fields)!=2 or not OID_RE.fullmatch(fields[0]): fail('authorization_git_ls_remote_rejected')
|
||||||
|
return fields[0]
|
||||||
|
|
||||||
|
def api_oid()->str:
|
||||||
|
user,password=credential_lines(); branch=urllib.parse.quote(BRANCH,safe='')
|
||||||
|
encoded=base64.b64encode((user+':'+password).encode()).decode()
|
||||||
|
request=urllib.request.Request('http://127.0.0.1:3000/api/v1/repos/aios/aios/branches/'+branch,headers={'Authorization':'Basic '+encoded,'Host':'g.cnnetmkt.top'})
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(request,timeout=10) as response: value=json.loads(response.read())
|
||||||
|
except Exception: fail('authorization_gitea_api_unavailable')
|
||||||
|
oid=str(value.get('commit',{}).get('id',''))
|
||||||
|
if not OID_RE.fullmatch(oid): fail('authorization_gitea_api_oid_rejected')
|
||||||
|
return oid
|
||||||
|
|
||||||
|
def four_way(expected:str)->None:
|
||||||
|
local=git('rev-parse','HEAD',cwd=WORKTREE)
|
||||||
|
bare=git('--git-dir',str(BARE),'rev-parse',f'refs/heads/{BRANCH}')
|
||||||
|
remote=remote_oid(); api=api_oid()
|
||||||
|
if any(x!=expected for x in (local,bare,remote,api)): fail('RECONCILIATION_REQUIRED',409,current_head_oid=local)
|
||||||
|
if git('status','--porcelain',cwd=WORKTREE)!='': fail('authorization_worktree_dirty',409)
|
||||||
|
|
||||||
|
def state()->dict[str,Any]:
|
||||||
|
s=load_json(STATE)
|
||||||
|
required={'schema_version','development_authorization_id','task_id','status','current_head_oid','last_activity_at','writer_child','test_child','inflight','updated_at'}
|
||||||
|
if set(s)!=required or s.get('schema_version')!=1 or s.get('development_authorization_id')!=AUTH_ID or s.get('task_id')!=TASK or not OID_RE.fullmatch(str(s.get('current_head_oid',''))): fail('authorization_state_schema_rejected')
|
||||||
|
return s
|
||||||
|
|
||||||
|
def save_state(s:dict[str,Any])->None: s['updated_at']=stamp(); atomic_json(STATE,s)
|
||||||
|
|
||||||
|
def lock():
|
||||||
|
STATE_DIR.mkdir(parents=True,exist_ok=True); os.chmod(STATE_DIR,0o700)
|
||||||
|
f=open(LOCK,'a+b'); os.chmod(LOCK,0o600); fcntl.flock(f.fileno(),fcntl.LOCK_EX); return f
|
||||||
|
|
||||||
|
def route_binding_valid(p:dict[str,Any])->None:
|
||||||
|
b=load_json(ROUTE_BINDING,0o640)
|
||||||
|
identities=b.get('action_identities')
|
||||||
|
if b.get('task_id')!=TASK or b.get('writer_instance_id')!=WRITER_INSTANCE or b.get('repository')!=REPO or b.get('branch')!=BRANCH or b.get('approved_base_oid')!=BASE or b.get('path_policy_sha256')!=p.get('path_policy_sha256') or not isinstance(identities,list) or len(identities)!=1 or identities[0].get('account')!=ACTOR:
|
||||||
|
fail('authorization_route_binding_changed',409)
|
||||||
|
|
||||||
|
def task_live(p:dict[str,Any],s:dict[str,Any],*,touch:bool)->None:
|
||||||
|
route_binding_valid(p)
|
||||||
|
if s['status'] in TERMINAL: fail('authorization_task_terminal',409)
|
||||||
|
if now()>=parse_time(p['valid_until']):
|
||||||
|
s['status']='REVOKED_WALL_CLOCK'; save_state(s); cleanup(s,'wall_clock'); fail('authorization_task_terminal',409)
|
||||||
|
if (now()-parse_time(s['last_activity_at'])).total_seconds()>p['idle_timeout_seconds']:
|
||||||
|
s['status']='REVOKED_IDLE_TIMEOUT'; save_state(s); cleanup(s,'idle_timeout'); fail('authorization_task_terminal',409)
|
||||||
|
if touch: s['last_activity_at']=stamp(); save_state(s)
|
||||||
|
|
||||||
|
def projection_token()->str|None:
|
||||||
|
if not PROJECTION.exists(): return None
|
||||||
|
try:
|
||||||
|
info=os.lstat(PROJECTION)
|
||||||
|
if not stat.S_ISREG(info.st_mode) or stat.S_ISLNK(info.st_mode) or info.st_uid!=0 or stat.S_IMODE(info.st_mode)!=0o600: fail('authorization_projection_rejected')
|
||||||
|
token=PROJECTION.read_text(encoding='utf-8').strip()
|
||||||
|
return token if len(token.encode())>=32 else None
|
||||||
|
except OSError: fail('authorization_projection_rejected')
|
||||||
|
|
||||||
|
def write_projection(token:str)->None:
|
||||||
|
PROJECTION.parent.mkdir(parents=True,exist_ok=True)
|
||||||
|
fd,tmp=tempfile.mkstemp(prefix='.writer-',dir=PROJECTION.parent)
|
||||||
|
try:
|
||||||
|
os.fchmod(fd,0o600); os.write(fd,(token+'\n').encode()); os.fsync(fd); os.close(fd); os.chown(tmp,0,0); os.replace(tmp,PROJECTION); os.chmod(PROJECTION,0o600)
|
||||||
|
finally:
|
||||||
|
try: os.close(fd)
|
||||||
|
except OSError: pass
|
||||||
|
if os.path.exists(tmp): os.unlink(tmp)
|
||||||
|
|
||||||
|
def post_revoke_401(token:str)->None:
|
||||||
|
req=urllib.request.Request('http://127.0.0.1:19893/api/runtime-status',headers={'Authorization':'Bearer '+token})
|
||||||
|
try: urllib.request.urlopen(req,timeout=5)
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
if e.code==401:return
|
||||||
|
except Exception: pass
|
||||||
|
fail('authorization_old_credential_revocation_unverified')
|
||||||
|
|
||||||
|
def rotate_child(p:dict[str,Any],s:dict[str,Any],reason:str)->str:
|
||||||
|
old=projection_token(); registry=TokenRegistry(REGISTRY)
|
||||||
|
old_record=registry.find(old) if old else None
|
||||||
|
if old_record and (old_record.get('task_id')!=TASK or old_record.get('development_authorization_id')!=AUTH_ID):
|
||||||
|
if parse_time(old_record.get('expires_at'))>=now(): fail('authorization_existing_projection_binding_rejected')
|
||||||
|
if not registry.revoke(str(old_record['token_id'])): fail('authorization_historical_child_revoke_failed')
|
||||||
|
post_revoke_401(old)
|
||||||
|
append_audit({'event':'historical_expired_child_revoked','old_credential_http_status':401})
|
||||||
|
old_record=None
|
||||||
|
issued=now().replace(microsecond=0); expires=issued+timedelta(minutes=20)
|
||||||
|
prefixes=[str(x).rstrip('/')+'/' for x in p['allowed_path_roots']]
|
||||||
|
grant={'token_id':'continuous-child-'+issued.strftime('%Y%m%dT%H%M%SZ')+'-'+os.urandom(3).hex(),'credential_id':'credential-'+issued.strftime('%Y%m%dT%H%M%SZ')+'-'+os.urandom(3).hex(),'development_authorization_id':AUTH_ID,'audience':WRITER_AUDIENCE,'task_id':TASK,'actor_identity':ACTOR,'writer_instance_id':WRITER_INSTANCE,'repository_identity':REPO,'source_cidrs':['127.0.0.1/32'],'allowed_branch':BRANCH,'current_expected_head_oid':s['current_head_oid'],'approved_base_oid':BASE,'allowed_operations':p['allowed_operations'],'allowed_paths':p['allowed_exact_paths'],'allowed_path_prefixes':prefixes,'not_before':issued.isoformat(),'expires_at':expires.isoformat(),'renewable':False,'max_uses':{operation:CHILD_OPERATION_MAX_USES for operation in p['allowed_operations']},'issued_by':'task_authorization_controller'}
|
||||||
|
token,record=registry.issue(grant)
|
||||||
|
try:
|
||||||
|
write_projection(token)
|
||||||
|
if old_record:
|
||||||
|
if not registry.revoke(str(old_record['token_id'])): fail('authorization_old_child_revoke_failed')
|
||||||
|
post_revoke_401(old)
|
||||||
|
except Exception:
|
||||||
|
registry.revoke(str(record['token_id']))
|
||||||
|
try: PROJECTION.unlink()
|
||||||
|
except FileNotFoundError: pass
|
||||||
|
raise
|
||||||
|
s['writer_child']={'credential_id':record['credential_id'],'issued_at':record['not_before'],'expires_at':record['expires_at'],'current_expected_head_oid':s['current_head_oid'],'rotation_reason':reason}
|
||||||
|
save_state(s); append_audit({'event':'child_grant_rotated','reason':reason,'current_head_oid':s['current_head_oid'],'old_grant_revoked':bool(old_record),'old_credential_http_status':401 if old_record else None})
|
||||||
|
return token
|
||||||
|
|
||||||
|
def child_operation_quota_exhausted(rec:dict[str,Any],registry:TokenRegistry)->bool:
|
||||||
|
token_id=str(rec.get('token_id',''))
|
||||||
|
allowed=rec.get('allowed_operations')
|
||||||
|
limits=rec.get('max_uses')
|
||||||
|
if not token_id or not isinstance(allowed,list) or not isinstance(limits,dict): fail('authorization_child_quota_schema_rejected')
|
||||||
|
for operation in allowed:
|
||||||
|
maximum=limits.get(operation)
|
||||||
|
if not isinstance(operation,str) or not isinstance(maximum,int) or maximum<1: fail('authorization_child_quota_schema_rejected')
|
||||||
|
if registry.usage(token_id,operation)>=maximum: return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
def ensure_child(p:dict[str,Any],s:dict[str,Any],reason:str)->str:
|
||||||
|
token=projection_token(); meta=s.get('writer_child') or {}
|
||||||
|
expired=True
|
||||||
|
try: expired=(parse_time(meta.get('expires_at'))-now()).total_seconds()<=300
|
||||||
|
except Exception: pass
|
||||||
|
if not token or expired or meta.get('current_expected_head_oid')!=s['current_head_oid']:
|
||||||
|
return rotate_child(p,s,reason)
|
||||||
|
registry=TokenRegistry(REGISTRY)
|
||||||
|
rec=registry.find(token)
|
||||||
|
if not rec or rec.get('revoked') or rec.get('development_authorization_id')!=AUTH_ID or rec.get('current_expected_head_oid')!=s['current_head_oid']:
|
||||||
|
return rotate_child(p,s,reason)
|
||||||
|
if child_operation_quota_exhausted(rec,registry):
|
||||||
|
return rotate_child(p,s,'operation_quota_exhausted')
|
||||||
|
return token
|
||||||
|
|
||||||
|
def ledger_verified(operation_id:str,commit_oid:str)->None:
|
||||||
|
with sqlite3.connect(WRITER_LEDGER) as db:
|
||||||
|
row=db.execute('select operation_id,branch,commit_oid,state,verification_status from writer_operations where operation_id=?',(operation_id,)).fetchone()
|
||||||
|
if not row or row[1]!=BRANCH or row[2]!=commit_oid or row[3]!='COMPLETED' or row[4]!='bare+gitea': fail('authorization_writer_ledger_rejected',409)
|
||||||
|
|
||||||
|
def reconcile(p:dict[str,Any],s:dict[str,Any],previous:str,new:str,operation_id:str)->dict[str,Any]:
|
||||||
|
inflight=s.get('inflight')
|
||||||
|
if not isinstance(inflight,dict) or inflight.get('previous_head_oid')!=previous or inflight.get('operation_id')!=operation_id: fail('authorization_inflight_rejected',409)
|
||||||
|
if previous!=s['current_head_oid'] or not OID_RE.fullmatch(new): fail('authorization_head_transition_rejected',409)
|
||||||
|
ledger_verified(operation_id,new); four_way(new)
|
||||||
|
s['current_head_oid']=new; s['inflight']=None; s['last_activity_at']=stamp(); save_state(s)
|
||||||
|
ensure_child(p,s,'head_advance')
|
||||||
|
append_audit({'event':'authorization_head_advanced','previous_head_oid':previous,'new_commit_oid':new,'four_way_ref_status':True,'worktree_clean':True})
|
||||||
|
return {'authorization_current_head_oid':new,'automatic_head_advance':True,'four_way_ref_status':True}
|
||||||
|
|
||||||
|
def prepare(p:dict[str,Any],s:dict[str,Any],data:dict[str,Any])->dict[str,Any]:
|
||||||
|
required={'expected_current_head_oid','operation_id','idempotency_key','exact_path','content_digest','commit_message_digest','branch','path','message'}
|
||||||
|
if set(data)!=required: fail('authorization_request_schema_rejected',403)
|
||||||
|
if data['branch']!=BRANCH or data['path']!=data['exact_path'] or not path_allowed(p,str(data['path'])): fail('authorization_path_rejected',403)
|
||||||
|
if data['expected_current_head_oid']!=s['current_head_oid'] or not OID_RE.fullmatch(str(data['expected_current_head_oid'])): fail('BRANCH_HEAD_CHANGED',409,current_head_oid=s['current_head_oid'])
|
||||||
|
if not OP_RE.fullmatch(str(data['operation_id'])) or not OP_RE.fullmatch(str(data['idempotency_key'])): fail('authorization_operation_id_rejected',403)
|
||||||
|
if not all(isinstance(data[k],str) and re.fullmatch(r'[0-9a-f]{64}',data[k]) for k in ('content_digest','commit_message_digest')): fail('authorization_digest_rejected',403)
|
||||||
|
if hashlib.sha256(data['message'].encode()).hexdigest()!=data['commit_message_digest']: fail('authorization_commit_message_digest_rejected',403)
|
||||||
|
inflight=s.get('inflight')
|
||||||
|
if inflight:
|
||||||
|
if inflight.get('operation_id')==data['operation_id'] and inflight.get('idempotency_key')==data['idempotency_key']: return {'expected_head_oid':s['current_head_oid'],'idempotency_key':data['idempotency_key']}
|
||||||
|
fail('BRANCH_HEAD_CHANGED',409,current_head_oid=s['current_head_oid'])
|
||||||
|
four_way(s['current_head_oid'])
|
||||||
|
s['inflight']={'operation_id':data['operation_id'],'idempotency_key':data['idempotency_key'],'previous_head_oid':s['current_head_oid'],'path':data['path'],'created_at':stamp()}; s['last_activity_at']=stamp(); save_state(s)
|
||||||
|
append_audit({'event':'write_prepared','operation_id':data['operation_id'],'path':data['path'],'expected_head_oid':s['current_head_oid']})
|
||||||
|
return {'expected_head_oid':s['current_head_oid'],'idempotency_key':data['idempotency_key']}
|
||||||
|
|
||||||
|
def test_authorize(p:dict[str,Any],s:dict[str,Any],operation:str)->dict[str,Any]:
|
||||||
|
allowed={'pytest_dormant','pytest_autonomous_release','compileall_autonomous_release','git_diff_check'}
|
||||||
|
if operation not in allowed: fail('candidate_test_executor_operation_rejected',403)
|
||||||
|
four_way(s['current_head_oid'])
|
||||||
|
child=s.get('test_child') or {}; refresh=True
|
||||||
|
try: refresh=(parse_time(child.get('expires_at'))-now()).total_seconds()<=300
|
||||||
|
except Exception: pass
|
||||||
|
if refresh:
|
||||||
|
issued=now().replace(microsecond=0); child={'test_authorization_id':'test-authz-aios-autonomous-release-20260726','grant_id':'test-child-'+issued.strftime('%Y%m%dT%H%M%SZ'),'issued_at':issued.isoformat(),'expires_at':(issued+timedelta(minutes=20)).isoformat()}; s['test_child']=child
|
||||||
|
s['last_activity_at']=stamp(); save_state(s); append_audit({'event':'test_authorized','operation':operation})
|
||||||
|
return {'test_authorization_id':child['test_authorization_id'],'operation':operation,'expires_at':child['expires_at']}
|
||||||
|
|
||||||
|
def cleanup(s:dict[str,Any],reason:str)->None:
|
||||||
|
old=projection_token()
|
||||||
|
if old:
|
||||||
|
reg=TokenRegistry(REGISTRY); rec=reg.find(old)
|
||||||
|
if rec: reg.revoke(str(rec['token_id']))
|
||||||
|
try: PROJECTION.unlink()
|
||||||
|
except FileNotFoundError: pass
|
||||||
|
try: post_revoke_401(old)
|
||||||
|
except ControlError: pass
|
||||||
|
for unit in ('aios-chatgpt-autonomous-release-route.service','aios-chatgpt-autonomous-release-writer.service'):
|
||||||
|
subprocess.run(['/usr/bin/systemctl','stop',unit],stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL,timeout=20)
|
||||||
|
subprocess.run(['/usr/bin/systemctl','stop','aios-autonomous-release-dormant-test-executor@*.service'],stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL,timeout=20)
|
||||||
|
try: ROUTE_BINDING.unlink()
|
||||||
|
except FileNotFoundError: pass
|
||||||
|
try: SOCKET_PATH.unlink()
|
||||||
|
except FileNotFoundError: pass
|
||||||
|
append_audit({'event':'authorization_cleanup','reason':reason,'projection_removed':not PROJECTION.exists(),'route_overlay_removed':not ROUTE_BINDING.exists(),'task_services_stop_requested':True})
|
||||||
|
|
||||||
|
def request(data:dict[str,Any])->dict[str,Any]:
|
||||||
|
op=data.get('op'); f=lock()
|
||||||
|
try:
|
||||||
|
p=parent(); s=state(); task_live(p,s,touch=False)
|
||||||
|
if op=='describe_authorization':
|
||||||
|
four_way(s['current_head_oid'])
|
||||||
|
return {'development_authorization_id':AUTH_ID,'task_id':TASK,'actor_identity':ACTOR,'audience':WRITER_AUDIENCE,'repository':REPO,'branch':BRANCH,'approved_base_oid':BASE,'authorization_status':s['status'],'current_head_oid':s['current_head_oid']}
|
||||||
|
if op=='get_writer_token':
|
||||||
|
four_way(s['current_head_oid']); token=ensure_child(p,s,'scheduled_or_requested_rotation'); return {'token':token,'current_head_oid':s['current_head_oid']}
|
||||||
|
if op=='prepare_write':
|
||||||
|
ensure_child(p,s,'pre_write_quota_check')
|
||||||
|
return prepare(p,s,data.get('request') if isinstance(data.get('request'),dict) else {})
|
||||||
|
if op=='reconcile_commit':
|
||||||
|
r=data.get('result') if isinstance(data.get('result'),dict) else {}; return reconcile(p,s,str(r.get('previous_head_oid','')),str(r.get('new_commit_oid','')),str(r.get('operation_id','')))
|
||||||
|
if op=='fail_write':
|
||||||
|
r=data.get('result') if isinstance(data.get('result'),dict) else {}; inflight=s.get('inflight')
|
||||||
|
if isinstance(inflight,dict) and inflight.get('operation_id')==r.get('operation_id'): s['inflight']=None; save_state(s)
|
||||||
|
return {'cleared':True}
|
||||||
|
if op=='authorize_test': return test_authorize(p,s,str(data.get('operation','')))
|
||||||
|
fail('authorization_operation_rejected',403)
|
||||||
|
finally:
|
||||||
|
fcntl.flock(f.fileno(),fcntl.LOCK_UN); f.close()
|
||||||
|
|
||||||
|
def peer_allowed(conn:socket.socket)->bool:
|
||||||
|
try:
|
||||||
|
import struct
|
||||||
|
_pid,uid,gid=struct.unpack('3i',conn.getsockopt(socket.SOL_SOCKET,socket.SO_PEERCRED,12))
|
||||||
|
return uid==pwd.getpwnam('aiman').pw_uid and gid==grp.getgrnam('gitops').gr_gid
|
||||||
|
except Exception: return False
|
||||||
|
|
||||||
|
def serve()->None:
|
||||||
|
SOCKET_DIR.mkdir(parents=True,exist_ok=True); os.chown(SOCKET_DIR,0,grp.getgrnam('gitops').gr_gid); os.chmod(SOCKET_DIR,0o750)
|
||||||
|
try: SOCKET_PATH.unlink()
|
||||||
|
except FileNotFoundError: pass
|
||||||
|
with socket.socket(socket.AF_UNIX,socket.SOCK_STREAM) as server:
|
||||||
|
server.bind(str(SOCKET_PATH)); os.chown(SOCKET_PATH,0,grp.getgrnam('gitops').gr_gid); os.chmod(SOCKET_PATH,0o660); server.listen(32); server.settimeout(1.0)
|
||||||
|
last=0.0
|
||||||
|
while True:
|
||||||
|
if time.monotonic()-last>30:
|
||||||
|
last=time.monotonic()
|
||||||
|
try:
|
||||||
|
f=lock(); p=parent(); s=state(); task_live(p,s,touch=False); four_way(s['current_head_oid']); ensure_child(p,s,'scheduled_rotation'); fcntl.flock(f.fileno(),fcntl.LOCK_UN); f.close()
|
||||||
|
except ControlError as e:
|
||||||
|
append_audit({'event':'scheduled_check_blocked','error':e.code})
|
||||||
|
if e.code=='authorization_task_terminal': return
|
||||||
|
except Exception: append_audit({'event':'scheduled_check_blocked','error':'authorization_controller_internal_error'})
|
||||||
|
try: conn,_=server.accept()
|
||||||
|
except socket.timeout: continue
|
||||||
|
with conn:
|
||||||
|
if not peer_allowed(conn): conn.sendall(canonical({'ok':False,'status':403,'error':'authorization_peer_rejected'})+b'\n'); continue
|
||||||
|
try:
|
||||||
|
raw=conn.recv(131072); data=json.loads(raw.decode()); result=request(data); conn.sendall(canonical({'ok':True,'result':result})+b'\n')
|
||||||
|
except ControlError as e: conn.sendall(canonical({'ok':False,'status':e.status,'error':e.code,**e.details})+b'\n')
|
||||||
|
except Exception as exc:
|
||||||
|
append_audit({'event':'controller_internal_error','error_type':type(exc).__name__})
|
||||||
|
conn.sendall(canonical({'ok':False,'status':503,'error':'authorization_controller_internal_error'})+b'\n')
|
||||||
|
if __name__=='__main__': serve()
|
||||||
@@ -1,89 +0,0 @@
|
|||||||
"""Governed API facade for branch-bound repair leases.
|
|
||||||
|
|
||||||
The facade is transport-neutral so the permanent control plane can expose it
|
|
||||||
through its existing authenticated loopback/API seam without creating a second
|
|
||||||
Git writer or a global credential registry.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from typing import Any, Mapping
|
|
||||||
|
|
||||||
from branch_repair_lease import LeaseError, RepairLease, RepairLeaseManager, sanitized_lease_view
|
|
||||||
|
|
||||||
|
|
||||||
class BranchRepairLeaseAPI:
|
|
||||||
"""Validate operation envelopes and return sanitized lease metadata."""
|
|
||||||
|
|
||||||
def __init__(self, manager: RepairLeaseManager) -> None:
|
|
||||||
self.manager = manager
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _required(request: Mapping[str, Any], name: str) -> Any:
|
|
||||||
value = request.get(name)
|
|
||||||
if value is None or value == "":
|
|
||||||
raise LeaseError(f"missing required field: {name}")
|
|
||||||
return value
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _view(lease: RepairLease) -> dict[str, Any]:
|
|
||||||
return {"status": "ok", "lease": sanitized_lease_view(lease)}
|
|
||||||
|
|
||||||
def handle(self, request: Mapping[str, Any]) -> dict[str, Any]:
|
|
||||||
try:
|
|
||||||
operation = self._required(request, "operation")
|
|
||||||
owner = str(self._required(request, "owner_identity"))
|
|
||||||
if operation == "issue":
|
|
||||||
lease = self.manager.issue(
|
|
||||||
task_id=str(self._required(request, "task_id")),
|
|
||||||
owner_identity=owner,
|
|
||||||
branch=str(self._required(request, "branch")),
|
|
||||||
description=str(self._required(request, "description")),
|
|
||||||
allowed_paths=tuple(self._required(request, "allowed_paths")),
|
|
||||||
ttl_seconds=int(request.get("ttl_seconds", 3600)),
|
|
||||||
)
|
|
||||||
elif operation == "attach_existing_branch":
|
|
||||||
lease = self.manager.attach_existing_branch(
|
|
||||||
str(self._required(request, "lease_id")),
|
|
||||||
owner_identity=owner,
|
|
||||||
expected_head_oid=str(self._required(request, "expected_head_oid")),
|
|
||||||
)
|
|
||||||
elif operation == "authorize_commit":
|
|
||||||
lease = self.manager.authorize_commit(
|
|
||||||
str(self._required(request, "lease_id")),
|
|
||||||
owner_identity=owner,
|
|
||||||
expected_head_oid=str(self._required(request, "expected_head_oid")),
|
|
||||||
paths=tuple(self._required(request, "paths")),
|
|
||||||
content=request.get("content"),
|
|
||||||
)
|
|
||||||
elif operation == "record_commit":
|
|
||||||
lease = self.manager.record_commit(
|
|
||||||
str(self._required(request, "lease_id")),
|
|
||||||
owner_identity=owner,
|
|
||||||
expected_head_oid=str(self._required(request, "expected_head_oid")),
|
|
||||||
new_head_oid=str(self._required(request, "new_head_oid")),
|
|
||||||
paths=tuple(self._required(request, "paths")),
|
|
||||||
)
|
|
||||||
elif operation == "recover":
|
|
||||||
lease = self.manager.recover(
|
|
||||||
str(self._required(request, "lease_id")), owner_identity=owner
|
|
||||||
)
|
|
||||||
elif operation == "renew":
|
|
||||||
lease = self.manager.renew(
|
|
||||||
str(self._required(request, "lease_id")),
|
|
||||||
owner_identity=owner,
|
|
||||||
ttl_seconds=int(self._required(request, "ttl_seconds")),
|
|
||||||
)
|
|
||||||
elif operation == "complete":
|
|
||||||
lease = self.manager.complete(
|
|
||||||
str(self._required(request, "lease_id")), owner_identity=owner
|
|
||||||
)
|
|
||||||
elif operation == "revoke":
|
|
||||||
lease = self.manager.revoke(
|
|
||||||
str(self._required(request, "lease_id")), owner_identity=owner
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
raise LeaseError("unsupported lease operation")
|
|
||||||
return self._view(lease)
|
|
||||||
except (LeaseError, TypeError, ValueError):
|
|
||||||
return {"status": "blocked", "error": "lease_request_rejected"}
|
|
||||||
@@ -1,48 +0,0 @@
|
|||||||
"""Root-only CLI for the branch-bound repair lease API."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from aios_branch_repair_lease_api import BranchRepairLeaseAPI
|
|
||||||
from branch_repair_lease import GitProbe, LeaseStore
|
|
||||||
from branch_repair_lease import RepairLeaseManager
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
parser = argparse.ArgumentParser(description="AIOS branch repair lease CLI")
|
|
||||||
parser.add_argument("--store", required=True)
|
|
||||||
parser.add_argument("--bare-repo", required=True)
|
|
||||||
parser.add_argument("--request-file")
|
|
||||||
parser.add_argument("--request-json")
|
|
||||||
args = parser.parse_args()
|
|
||||||
if hasattr(os, "geteuid") and os.geteuid() != 0:
|
|
||||||
print(json.dumps({"status": "blocked", "error": "root_required"}))
|
|
||||||
return 2
|
|
||||||
if bool(args.request_file) == bool(args.request_json):
|
|
||||||
print(json.dumps({"status": "blocked", "error": "one_request_source_required"}))
|
|
||||||
return 2
|
|
||||||
try:
|
|
||||||
raw = (
|
|
||||||
Path(args.request_file).read_text(encoding="utf-8")
|
|
||||||
if args.request_file
|
|
||||||
else str(args.request_json)
|
|
||||||
)
|
|
||||||
request = json.loads(raw)
|
|
||||||
if not isinstance(request, dict):
|
|
||||||
raise ValueError("request must be an object")
|
|
||||||
api = BranchRepairLeaseAPI(RepairLeaseManager(LeaseStore(args.store), GitProbe(args.bare_repo)))
|
|
||||||
response = api.handle(request)
|
|
||||||
print(json.dumps(response, ensure_ascii=True, sort_keys=True, separators=(",", ":")))
|
|
||||||
return 0 if response.get("status") == "ok" else 1
|
|
||||||
except Exception:
|
|
||||||
print(json.dumps({"status": "blocked", "error": "lease_cli_failed"}))
|
|
||||||
return 1
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -1,104 +0,0 @@
|
|||||||
"""Read-only validation of recovered campaign authorization evidence."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
import stat
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
|
|
||||||
class RecoveryEvidenceError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def _digest(path: Path) -> str:
|
|
||||||
h = hashlib.sha256()
|
|
||||||
with path.open("rb") as stream:
|
|
||||||
for block in iter(lambda: stream.read(1024 * 1024), b""):
|
|
||||||
h.update(block)
|
|
||||||
return h.hexdigest()
|
|
||||||
|
|
||||||
|
|
||||||
def _secure_json(path: Path) -> dict[str, Any]:
|
|
||||||
if path.is_symlink() or not path.is_file():
|
|
||||||
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_MISSING")
|
|
||||||
info = path.stat(follow_symlinks=False)
|
|
||||||
if info.st_uid != 0 or stat.S_IMODE(info.st_mode) & 0o022:
|
|
||||||
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_PERMISSIONS")
|
|
||||||
try:
|
|
||||||
value = json.loads(path.read_text(encoding="utf-8"))
|
|
||||||
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
|
|
||||||
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_INVALID") from exc
|
|
||||||
if not isinstance(value, dict):
|
|
||||||
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_OBJECT_REQUIRED")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def load_recovered_authorization(path: str | Path) -> dict[str, Any]:
|
|
||||||
"""Verify the recovery receipt, sidecar, and all referenced immutable files."""
|
|
||||||
receipt_path = Path(path)
|
|
||||||
receipt = _secure_json(receipt_path)
|
|
||||||
sidecar = receipt_path.with_suffix(receipt_path.suffix + ".sha256")
|
|
||||||
if sidecar.is_symlink() or not sidecar.is_file():
|
|
||||||
raise RecoveryEvidenceError("RECOVERY_DIGEST_SIDECAR_MISSING")
|
|
||||||
expected = sidecar.read_text(encoding="ascii").split()[0]
|
|
||||||
actual = _digest(receipt_path)
|
|
||||||
if expected != actual:
|
|
||||||
raise RecoveryEvidenceError("RECOVERY_DIGEST_MISMATCH")
|
|
||||||
required = {
|
|
||||||
"schema",
|
|
||||||
"campaign_id",
|
|
||||||
"original_event_id",
|
|
||||||
"original_authorization_digest",
|
|
||||||
"signed_grant_id",
|
|
||||||
"signed_grant_digest",
|
|
||||||
"task_id",
|
|
||||||
"repository",
|
|
||||||
"original_base_oid",
|
|
||||||
"branch",
|
|
||||||
"expected_branch_oid",
|
|
||||||
"recovered_from_existing_evidence",
|
|
||||||
"new_owner_interaction",
|
|
||||||
"recovery_state",
|
|
||||||
"production_side_effect_executed",
|
|
||||||
"global_writes_enabled",
|
|
||||||
"grant_consumed",
|
|
||||||
"owner_authorization_consumed",
|
|
||||||
"original_receipts_immutable",
|
|
||||||
"recovered_at",
|
|
||||||
"evidence_files",
|
|
||||||
}
|
|
||||||
if set(receipt) != required or receipt["schema"] != "aios.owner-authorization-recovery.v1":
|
|
||||||
raise RecoveryEvidenceError("RECOVERY_SCHEMA_INVALID")
|
|
||||||
if receipt["recovered_from_existing_evidence"] is not True or receipt["new_owner_interaction"] is not False:
|
|
||||||
raise RecoveryEvidenceError("RECOVERY_NOT_APPEND_ONLY")
|
|
||||||
if receipt["production_side_effect_executed"] is not False or receipt["global_writes_enabled"] is not False:
|
|
||||||
raise RecoveryEvidenceError("RECOVERY_SCOPE_INVALID")
|
|
||||||
if receipt["recovery_state"] != "RECOVERED_PENDING_CONTINUATION_LEASE":
|
|
||||||
raise RecoveryEvidenceError("RECOVERY_STATE_INVALID")
|
|
||||||
evidence = receipt["evidence_files"]
|
|
||||||
if not isinstance(evidence, list) or not evidence:
|
|
||||||
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_LIST_INVALID")
|
|
||||||
for item in evidence:
|
|
||||||
if not isinstance(item, dict) or set(item) != {"path", "sha256"}:
|
|
||||||
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_ENTRY_INVALID")
|
|
||||||
target = Path(item["path"])
|
|
||||||
if _digest(target) != item["sha256"]:
|
|
||||||
raise RecoveryEvidenceError("RECOVERY_EVIDENCE_DIGEST_MISMATCH")
|
|
||||||
return {
|
|
||||||
"schema": receipt["schema"],
|
|
||||||
"state": receipt["recovery_state"],
|
|
||||||
"campaign_id": receipt["campaign_id"],
|
|
||||||
"original_event_id": receipt["original_event_id"],
|
|
||||||
"original_authorization_digest": receipt["original_authorization_digest"],
|
|
||||||
"signed_grant_id": receipt["signed_grant_id"],
|
|
||||||
"task_id": receipt["task_id"],
|
|
||||||
"repository": receipt["repository"],
|
|
||||||
"original_base_oid": receipt["original_base_oid"],
|
|
||||||
"branch": receipt["branch"],
|
|
||||||
"expected_branch_oid": receipt["expected_branch_oid"],
|
|
||||||
"digest": actual,
|
|
||||||
"new_owner_interaction": False,
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,567 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Task-bound bridge from the legacy ChatGPT Action to an isolated Writer V2."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import hmac
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import sqlite3
|
||||||
|
import socket
|
||||||
|
import stat
|
||||||
|
import urllib.error
|
||||||
|
import urllib.parse
|
||||||
|
import urllib.request
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
from pathlib import Path, PurePosixPath, PureWindowsPath
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
CONFIG_PATH = Path(os.environ["AIOS_ACTION_ROUTE_BINDING"])
|
||||||
|
POLICY_PATH = Path(os.environ["AIOS_ACTION_ROUTE_PATH_POLICY"])
|
||||||
|
KNOWN_IDENTITIES_PATH = Path(os.environ["AIOS_ACTION_ROUTE_KNOWN_IDENTITIES"])
|
||||||
|
STATE_PATH = Path(os.environ["AIOS_ACTION_ROUTE_STATE"])
|
||||||
|
AUDIT_PATH = Path(os.environ["AIOS_ACTION_ROUTE_AUDIT"])
|
||||||
|
HOST = os.environ.get("AIOS_ACTION_ROUTE_HOST", "127.0.0.1")
|
||||||
|
PORT = int(os.environ.get("AIOS_ACTION_ROUTE_PORT", "19894"))
|
||||||
|
EXPECTED_TASK_ID = os.environ["AIOS_ACTION_ROUTE_TASK_ID"]
|
||||||
|
EXPECTED_ROUTE_HEADER = os.environ["AIOS_ACTION_ROUTE_HEADER"]
|
||||||
|
AUTHORIZATION_SOCKET = Path(os.environ["AIOS_CONTINUOUS_AUTHORIZATION_SOCKET"])
|
||||||
|
OID_RE = re.compile(r"[0-9a-f]{40}")
|
||||||
|
SECRET_RE = re.compile(
|
||||||
|
r"(?i)(api[_-]?key|secret[_-]?access[_-]?key|secret|password|token)"
|
||||||
|
r"\s*[:=]\s*[\"']?([A-Za-z0-9_./+=:-]{16,})"
|
||||||
|
)
|
||||||
|
SECRET_VALUE_RE = re.compile(
|
||||||
|
r"(sk-[A-Za-z0-9_-]{20,}|HPUA[A-Z0-9]{10,}|SDC[A-Za-z0-9]{20,}|"
|
||||||
|
r"-----BEGIN (?:OPENSSH |RSA |EC )?PRIVATE KEY-----)"
|
||||||
|
)
|
||||||
|
ALLOWED_REQUEST_FIELDS = frozenset({"base","branch","content","create_branch","message","path","task_id","development_authorization_id","actor_identity","audience","approved_base_oid","expected_current_head_oid","operation_id","idempotency_key","exact_path","content_digest","commit_message_digest"})
|
||||||
|
|
||||||
|
|
||||||
|
class RouteError(RuntimeError):
|
||||||
|
def __init__(self, code: str, status: int = 400, **details: Any):
|
||||||
|
super().__init__(code)
|
||||||
|
self.code = code
|
||||||
|
self.status = status
|
||||||
|
self.details = details
|
||||||
|
|
||||||
|
|
||||||
|
def utc_now() -> str:
|
||||||
|
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||||
|
|
||||||
|
|
||||||
|
def canonical_json(value: Any) -> bytes:
|
||||||
|
return json.dumps(
|
||||||
|
value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
|
||||||
|
).encode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def load_root_managed_json(
|
||||||
|
path: Path, *, expected_mode: int = 0o640
|
||||||
|
) -> tuple[dict[str, Any], bytes]:
|
||||||
|
info = os.lstat(path)
|
||||||
|
if stat.S_ISLNK(info.st_mode) or not stat.S_ISREG(info.st_mode):
|
||||||
|
raise RouteError("writer_route_config_type_rejected", 503)
|
||||||
|
if info.st_uid != 0 or stat.S_IMODE(info.st_mode) & ~expected_mode:
|
||||||
|
raise RouteError("writer_route_config_permissions_rejected", 503)
|
||||||
|
raw = path.read_bytes()
|
||||||
|
try:
|
||||||
|
value = json.loads(raw)
|
||||||
|
except (UnicodeDecodeError, json.JSONDecodeError):
|
||||||
|
raise RouteError("writer_route_config_json_rejected", 503) from None
|
||||||
|
if not isinstance(value, dict):
|
||||||
|
raise RouteError("writer_route_config_object_required", 503)
|
||||||
|
return value, raw
|
||||||
|
|
||||||
|
|
||||||
|
def load_binding() -> dict[str, Any]:
|
||||||
|
binding, _ = load_root_managed_json(CONFIG_PATH)
|
||||||
|
required = {
|
||||||
|
"schema_version",
|
||||||
|
"task_id",
|
||||||
|
"writer_instance_id",
|
||||||
|
"repository",
|
||||||
|
"branch",
|
||||||
|
"approved_base_oid",
|
||||||
|
"writer_url",
|
||||||
|
"path_policy_sha256",
|
||||||
|
"action_identities",
|
||||||
|
}
|
||||||
|
if set(binding) != required or binding.get("schema_version") != 1:
|
||||||
|
raise RouteError("writer_route_binding_schema_rejected", 503)
|
||||||
|
if binding["task_id"] != EXPECTED_TASK_ID:
|
||||||
|
raise RouteError("writer_route_task_binding_rejected", 503)
|
||||||
|
if binding["repository"] != "aios/aios":
|
||||||
|
raise RouteError("writer_route_repository_rejected", 503)
|
||||||
|
if not OID_RE.fullmatch(str(binding["approved_base_oid"])):
|
||||||
|
raise RouteError("writer_route_base_oid_rejected", 503)
|
||||||
|
if binding["writer_url"] != "http://127.0.0.1:19893":
|
||||||
|
raise RouteError("writer_route_upstream_rejected", 503)
|
||||||
|
identities = binding.get("action_identities")
|
||||||
|
if not isinstance(identities, list) or len(identities) != 1:
|
||||||
|
raise RouteError("writer_route_action_identity_rejected", 503)
|
||||||
|
identity = identities[0]
|
||||||
|
if (
|
||||||
|
not isinstance(identity, dict)
|
||||||
|
or set(identity) != {"account", "token_sha256"}
|
||||||
|
or identity.get("account") != "chatgpt-gitops-bot"
|
||||||
|
or not re.fullmatch(
|
||||||
|
r"[0-9a-f]{64}", str(identity.get("token_sha256", ""))
|
||||||
|
)
|
||||||
|
):
|
||||||
|
raise RouteError("writer_route_action_identity_rejected", 503)
|
||||||
|
return binding
|
||||||
|
|
||||||
|
|
||||||
|
def allowed_path_scope(binding: dict[str, Any]) -> tuple[set[str], tuple[str, ...]]:
|
||||||
|
policy, raw = load_root_managed_json(POLICY_PATH)
|
||||||
|
actual = hashlib.sha256(raw).hexdigest()
|
||||||
|
if actual != binding["path_policy_sha256"]:
|
||||||
|
raise RouteError("writer_route_path_policy_digest_mismatch", 503)
|
||||||
|
if policy.get("allowed_branch") != binding["branch"]:
|
||||||
|
raise RouteError("writer_route_path_policy_branch_mismatch", 503)
|
||||||
|
operation = (policy.get("allowed_operations") or {}).get("commit_file.atomic")
|
||||||
|
paths = operation.get("exact_paths") if isinstance(operation, dict) else None
|
||||||
|
prefixes = operation.get("prefix_paths") if isinstance(operation, dict) else None
|
||||||
|
if not isinstance(paths, list) or not paths or not all(
|
||||||
|
isinstance(x, str) for x in paths
|
||||||
|
):
|
||||||
|
raise RouteError("writer_route_path_policy_rejected", 503)
|
||||||
|
if not isinstance(prefixes, list) or not prefixes or not all(
|
||||||
|
isinstance(x, str) and x.endswith("/") for x in prefixes
|
||||||
|
):
|
||||||
|
raise RouteError("writer_route_path_policy_rejected", 503)
|
||||||
|
if any(not valid_relative_path(x[:-1]) for x in prefixes):
|
||||||
|
raise RouteError("writer_route_path_policy_rejected", 503)
|
||||||
|
if any(not valid_relative_path(x) for x in paths):
|
||||||
|
raise RouteError("writer_route_path_policy_rejected", 503)
|
||||||
|
return set(paths), tuple(prefixes)
|
||||||
|
|
||||||
|
|
||||||
|
def valid_relative_path(path: str) -> bool:
|
||||||
|
if not isinstance(path, str) or not path or path != path.strip():
|
||||||
|
return False
|
||||||
|
if "\x00" in path or "\\" in path:
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
decoded = urllib.parse.unquote(path, errors="strict")
|
||||||
|
except (UnicodeDecodeError, ValueError):
|
||||||
|
return False
|
||||||
|
if decoded != path:
|
||||||
|
return False
|
||||||
|
posix = PurePosixPath(path)
|
||||||
|
windows = PureWindowsPath(path)
|
||||||
|
return not (
|
||||||
|
posix.is_absolute()
|
||||||
|
or windows.is_absolute()
|
||||||
|
or bool(windows.drive)
|
||||||
|
or any(part in {"", ".", ".."} for part in path.split("/"))
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def path_allowed(binding: dict[str, Any], path: str) -> bool:
|
||||||
|
if not valid_relative_path(path):
|
||||||
|
return False
|
||||||
|
exact_paths, prefixes = allowed_path_scope(binding)
|
||||||
|
return path in exact_paths or any(path.startswith(prefix) for prefix in prefixes)
|
||||||
|
|
||||||
|
|
||||||
|
def bearer(headers: Any) -> str:
|
||||||
|
value = headers.get("Authorization", "")
|
||||||
|
if not value.startswith("Bearer "):
|
||||||
|
raise RouteError("unauthorized", 401)
|
||||||
|
token = value[7:].strip()
|
||||||
|
if not token:
|
||||||
|
raise RouteError("unauthorized", 401)
|
||||||
|
return token
|
||||||
|
|
||||||
|
|
||||||
|
def authorize_action(headers: Any, binding: dict[str, Any]) -> str:
|
||||||
|
digest = hashlib.sha256(bearer(headers).encode("utf-8")).hexdigest()
|
||||||
|
identity = binding["action_identities"][0]
|
||||||
|
if not hmac.compare_digest(identity["token_sha256"], digest):
|
||||||
|
known, _ = load_root_managed_json(KNOWN_IDENTITIES_PATH)
|
||||||
|
identities = known.get("identities")
|
||||||
|
if known.get("schema_version") != 1 or not isinstance(identities, list):
|
||||||
|
raise RouteError("writer_route_identity_catalog_rejected", 503)
|
||||||
|
observed = "unregistered"
|
||||||
|
allowed_accounts = {
|
||||||
|
"chatgpt-gitops-bot",
|
||||||
|
"gitops-write-bot",
|
||||||
|
"codex-gitops-bot",
|
||||||
|
}
|
||||||
|
for candidate in identities:
|
||||||
|
if not isinstance(candidate, dict) or set(candidate) != {
|
||||||
|
"account",
|
||||||
|
"token_sha256",
|
||||||
|
}:
|
||||||
|
raise RouteError("writer_route_identity_catalog_rejected", 503)
|
||||||
|
account = str(candidate.get("account", ""))
|
||||||
|
token_sha256 = str(candidate.get("token_sha256", ""))
|
||||||
|
if account not in allowed_accounts or not re.fullmatch(
|
||||||
|
r"[0-9a-f]{64}", token_sha256
|
||||||
|
):
|
||||||
|
raise RouteError("writer_route_identity_catalog_rejected", 503)
|
||||||
|
if hmac.compare_digest(token_sha256, digest):
|
||||||
|
observed = account
|
||||||
|
raise RouteError(
|
||||||
|
"writer_route_action_identity_rejected",
|
||||||
|
403,
|
||||||
|
accepted_action_identity=str(identity["account"]),
|
||||||
|
observed_action_identity=observed,
|
||||||
|
identity_alias_accepted=False,
|
||||||
|
)
|
||||||
|
return str(identity["account"])
|
||||||
|
|
||||||
|
|
||||||
|
def authorization_call(operation: str, **payload: Any) -> dict[str, Any]:
|
||||||
|
request = {"op": operation, **payload}
|
||||||
|
try:
|
||||||
|
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client:
|
||||||
|
client.settimeout(15)
|
||||||
|
client.connect(str(AUTHORIZATION_SOCKET))
|
||||||
|
client.sendall(canonical_json(request))
|
||||||
|
raw = client.recv(131072)
|
||||||
|
response = json.loads(raw.decode("utf-8"))
|
||||||
|
except (OSError, TimeoutError, UnicodeDecodeError, json.JSONDecodeError):
|
||||||
|
raise RouteError("authorization_controller_unavailable", 503) from None
|
||||||
|
if not isinstance(response, dict) or not response.get("ok"):
|
||||||
|
if not isinstance(response, dict):
|
||||||
|
raise RouteError("authorization_controller_response_rejected", 503)
|
||||||
|
details = {k: v for k, v in response.items() if k not in {"ok", "status", "error"}}
|
||||||
|
raise RouteError(str(response.get("error") or "authorization_controller_rejected"), int(response.get("status") or 503), **details)
|
||||||
|
result = response.get("result")
|
||||||
|
if not isinstance(result, dict):
|
||||||
|
raise RouteError("authorization_controller_response_rejected", 503)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def writer_token() -> str:
|
||||||
|
value = authorization_call("get_writer_token").get("token")
|
||||||
|
if not isinstance(value, str) or len(value.encode("utf-8")) < 32:
|
||||||
|
raise RouteError("writer_route_credential_rejected", 503)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def writer_call(
|
||||||
|
binding: dict[str, Any],
|
||||||
|
method: str,
|
||||||
|
path: str,
|
||||||
|
payload: dict[str, Any] | None = None,
|
||||||
|
) -> tuple[int, dict[str, Any]]:
|
||||||
|
data = canonical_json(payload) if payload is not None else None
|
||||||
|
headers = {"Authorization": "Bearer " + writer_token()}
|
||||||
|
if data is not None:
|
||||||
|
headers["Content-Type"] = "application/json"
|
||||||
|
request = urllib.request.Request(
|
||||||
|
binding["writer_url"] + path, data=data, method=method, headers=headers
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(request, timeout=90) as response:
|
||||||
|
body = json.loads(response.read())
|
||||||
|
return response.status, body
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
try:
|
||||||
|
body = json.loads(exc.read())
|
||||||
|
except Exception:
|
||||||
|
body = {"error": "writer_route_upstream_non_json_error"}
|
||||||
|
return exc.code, body
|
||||||
|
except (OSError, TimeoutError):
|
||||||
|
raise RouteError("writer_route_upstream_unavailable", 503) from None
|
||||||
|
|
||||||
|
|
||||||
|
def validate_request(data: Any, binding: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
raise RouteError("writer_route_request_object_required")
|
||||||
|
if not {"branch", "path", "content"}.issubset(data):
|
||||||
|
raise RouteError("writer_route_request_incomplete")
|
||||||
|
if not set(data).issubset(ALLOWED_REQUEST_FIELDS):
|
||||||
|
raise RouteError("writer_route_request_fields_rejected")
|
||||||
|
if data["branch"] != binding["branch"]:
|
||||||
|
raise RouteError("writer_route_request_binding_failed", 403)
|
||||||
|
if data.get("create_branch") not in (None, False):
|
||||||
|
raise RouteError("writer_route_request_binding_failed", 403)
|
||||||
|
if data.get("base", "main") not in {"main", binding["approved_base_oid"]}:
|
||||||
|
raise RouteError("writer_route_request_binding_failed", 403)
|
||||||
|
path = data["path"]
|
||||||
|
if not isinstance(path, str) or not path_allowed(binding, path):
|
||||||
|
raise RouteError("writer_route_request_binding_failed", 403)
|
||||||
|
content = data["content"]
|
||||||
|
if not isinstance(content, str) or "\x00" in content or "\r" in content:
|
||||||
|
raise RouteError("writer_route_content_rejected")
|
||||||
|
if len(content.encode("utf-8")) > 1_048_576:
|
||||||
|
raise RouteError("writer_route_content_too_large", 413)
|
||||||
|
if SECRET_RE.search(content) or SECRET_VALUE_RE.search(content):
|
||||||
|
raise RouteError("writer_route_secret_content_rejected", 403)
|
||||||
|
message = data.get("message") or f"chore(chatgpt): update {path}"
|
||||||
|
if not isinstance(message, str) or not 1 <= len(message) <= 200 or "\x00" in message or "\n" in message or "\r" in message:
|
||||||
|
raise RouteError("writer_route_message_rejected")
|
||||||
|
return {"branch": binding["branch"], "path": path, "content": content, "message": message}
|
||||||
|
|
||||||
|
|
||||||
|
def complete_request(data: dict[str, Any], normalized: dict[str, Any], binding: dict[str, Any], account: str) -> dict[str, Any]:
|
||||||
|
authorization = authorization_call("describe_authorization")
|
||||||
|
expected = {
|
||||||
|
"task_id": binding["task_id"],
|
||||||
|
"development_authorization_id": authorization.get("development_authorization_id"),
|
||||||
|
"actor_identity": account,
|
||||||
|
"audience": authorization.get("audience"),
|
||||||
|
"branch": binding["branch"],
|
||||||
|
"approved_base_oid": binding["approved_base_oid"],
|
||||||
|
}
|
||||||
|
required = {
|
||||||
|
"task_id": EXPECTED_TASK_ID,
|
||||||
|
"development_authorization_id": "authz-aios-autonomous-release-20260726",
|
||||||
|
"actor_identity": "chatgpt-gitops-bot",
|
||||||
|
"audience": "chatgpt-autonomous-release-writer",
|
||||||
|
"branch": binding["branch"],
|
||||||
|
"approved_base_oid": binding["approved_base_oid"],
|
||||||
|
}
|
||||||
|
if expected != required or authorization.get("authorization_status") != "ACTIVE" or authorization.get("current_head_oid") is None:
|
||||||
|
raise RouteError("writer_route_request_binding_failed", 403)
|
||||||
|
for key, value in required.items():
|
||||||
|
if key in data and data[key] != value:
|
||||||
|
raise RouteError("writer_route_request_binding_failed", 403)
|
||||||
|
if "exact_path" in data and data["exact_path"] != normalized["path"]:
|
||||||
|
raise RouteError("writer_route_request_binding_failed", 403)
|
||||||
|
current_head = str(authorization["current_head_oid"])
|
||||||
|
if not OID_RE.fullmatch(current_head):
|
||||||
|
raise RouteError("authorization_current_head_rejected", 503)
|
||||||
|
if "expected_current_head_oid" in data and data["expected_current_head_oid"] != current_head:
|
||||||
|
raise RouteError("BRANCH_HEAD_CHANGED", 409, current_head_oid=current_head)
|
||||||
|
content_digest = hashlib.sha256(normalized["content"].encode("utf-8")).hexdigest()
|
||||||
|
message_digest = hashlib.sha256(normalized["message"].encode("utf-8")).hexdigest()
|
||||||
|
if "content_digest" in data and data["content_digest"] != content_digest:
|
||||||
|
raise RouteError("writer_route_request_binding_failed", 403)
|
||||||
|
if "commit_message_digest" in data and data["commit_message_digest"] != message_digest:
|
||||||
|
raise RouteError("writer_route_request_binding_failed", 403)
|
||||||
|
seed = {**required, "expected_current_head_oid": current_head, "path": normalized["path"], "content_digest": content_digest, "commit_message_digest": message_digest}
|
||||||
|
request_digest = hashlib.sha256(canonical_json(seed)).hexdigest()
|
||||||
|
idempotency_key = "action-" + request_digest
|
||||||
|
operation_id = hashlib.sha256((EXPECTED_TASK_ID + "\0" + idempotency_key).encode("utf-8")).hexdigest()
|
||||||
|
if "idempotency_key" in data and data["idempotency_key"] != idempotency_key:
|
||||||
|
raise RouteError("writer_route_request_binding_failed", 403)
|
||||||
|
if "operation_id" in data and data["operation_id"] != operation_id:
|
||||||
|
raise RouteError("writer_route_request_binding_failed", 403)
|
||||||
|
return {**normalized, **required, "expected_current_head_oid": current_head, "operation_id": operation_id, "idempotency_key": idempotency_key, "exact_path": normalized["path"], "content_digest": content_digest, "commit_message_digest": message_digest, "request_digest": request_digest}
|
||||||
|
|
||||||
|
|
||||||
|
def current_writer_head(binding: dict[str, Any]) -> str:
|
||||||
|
status, body = writer_call(binding, "GET", "/api/runtime-status")
|
||||||
|
if status != 200:
|
||||||
|
raise RouteError(
|
||||||
|
"writer_route_runtime_status_failed", 503, upstream_status=status
|
||||||
|
)
|
||||||
|
required = {
|
||||||
|
"service": "chatgpt-autonomous-release-writer",
|
||||||
|
"writes_enabled": True,
|
||||||
|
"allowed_branch": binding["branch"],
|
||||||
|
"approved_base_oid": binding["approved_base_oid"],
|
||||||
|
"path_policy_sha256": binding["path_policy_sha256"],
|
||||||
|
}
|
||||||
|
if any(body.get(key) != value for key, value in required.items()):
|
||||||
|
raise RouteError("writer_route_runtime_binding_mismatch", 503)
|
||||||
|
head = str(body.get("workspace_head_oid", ""))
|
||||||
|
if not OID_RE.fullmatch(head):
|
||||||
|
raise RouteError("writer_route_runtime_head_rejected", 503)
|
||||||
|
return head
|
||||||
|
|
||||||
|
|
||||||
|
def operation_binding(
|
||||||
|
request_digest: str, binding: dict[str, Any]
|
||||||
|
) -> tuple[str, str]:
|
||||||
|
STATE_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with sqlite3.connect(STATE_PATH, timeout=30, isolation_level=None) as db:
|
||||||
|
db.execute("PRAGMA journal_mode=WAL")
|
||||||
|
db.execute("PRAGMA synchronous=FULL")
|
||||||
|
db.execute(
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS action_operations (
|
||||||
|
request_digest TEXT PRIMARY KEY,
|
||||||
|
expected_head_oid TEXT NOT NULL,
|
||||||
|
idempotency_key TEXT NOT NULL UNIQUE,
|
||||||
|
created_at TEXT NOT NULL
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
row = db.execute(
|
||||||
|
"SELECT expected_head_oid,idempotency_key FROM action_operations "
|
||||||
|
"WHERE request_digest=?",
|
||||||
|
(request_digest,),
|
||||||
|
).fetchone()
|
||||||
|
if row:
|
||||||
|
return str(row[0]), str(row[1])
|
||||||
|
head = current_writer_head(binding)
|
||||||
|
key = "action-" + request_digest
|
||||||
|
with sqlite3.connect(STATE_PATH, timeout=30, isolation_level=None) as db:
|
||||||
|
db.execute("PRAGMA journal_mode=WAL")
|
||||||
|
db.execute("PRAGMA synchronous=FULL")
|
||||||
|
db.execute("BEGIN IMMEDIATE")
|
||||||
|
db.execute(
|
||||||
|
"INSERT OR IGNORE INTO action_operations"
|
||||||
|
"(request_digest,expected_head_oid,idempotency_key,created_at) "
|
||||||
|
"VALUES(?,?,?,?)",
|
||||||
|
(request_digest, head, key, utc_now()),
|
||||||
|
)
|
||||||
|
row = db.execute(
|
||||||
|
"SELECT expected_head_oid,idempotency_key FROM action_operations "
|
||||||
|
"WHERE request_digest=?",
|
||||||
|
(request_digest,),
|
||||||
|
).fetchone()
|
||||||
|
db.execute("COMMIT")
|
||||||
|
return str(row[0]), str(row[1])
|
||||||
|
|
||||||
|
|
||||||
|
def append_audit(event: dict[str, Any]) -> None:
|
||||||
|
AUDIT_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
flags = os.O_WRONLY | os.O_CREAT | os.O_APPEND | getattr(os, "O_NOFOLLOW", 0)
|
||||||
|
fd = os.open(AUDIT_PATH, flags, 0o600)
|
||||||
|
try:
|
||||||
|
os.write(fd, canonical_json(event) + b"\n")
|
||||||
|
os.fsync(fd)
|
||||||
|
finally:
|
||||||
|
os.close(fd)
|
||||||
|
|
||||||
|
|
||||||
|
class Handler(BaseHTTPRequestHandler):
|
||||||
|
server_version = "AIOSActionTaskRouter/1"
|
||||||
|
|
||||||
|
def log_message(self, _format: str, *_args: Any) -> None:
|
||||||
|
return
|
||||||
|
|
||||||
|
def send_json(self, status: int, value: dict[str, Any]) -> None:
|
||||||
|
body = json.dumps(value, ensure_ascii=False, sort_keys=True).encode("utf-8")
|
||||||
|
self.send_response(status)
|
||||||
|
self.send_header("Content-Type", "application/json; charset=utf-8")
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.send_header("Cache-Control", "no-store")
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
def do_GET(self) -> None:
|
||||||
|
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
|
||||||
|
if path != "/health":
|
||||||
|
return self.send_json(404, {"error": "not_found"})
|
||||||
|
try:
|
||||||
|
binding = load_binding()
|
||||||
|
self.send_json(
|
||||||
|
200,
|
||||||
|
{
|
||||||
|
"status": "ok",
|
||||||
|
"service": "aios-action-task-router",
|
||||||
|
"task_id": binding["task_id"],
|
||||||
|
"writer_instance_id": binding["writer_instance_id"],
|
||||||
|
"branch": binding["branch"],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
except RouteError as exc:
|
||||||
|
self.send_json(exc.status, {"error": exc.code, **exc.details})
|
||||||
|
|
||||||
|
def do_POST(self) -> None:
|
||||||
|
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
|
||||||
|
if path != "/api/file":
|
||||||
|
return self.send_json(404, {"error": "not_found"})
|
||||||
|
if self.client_address[0] not in {"127.0.0.1", "::1"}:
|
||||||
|
return self.send_json(403, {"error": "writer_route_source_rejected"})
|
||||||
|
if self.headers.get("X-AIOS-Route-Task") != EXPECTED_ROUTE_HEADER:
|
||||||
|
return self.send_json(403, {"error": "writer_route_header_rejected"})
|
||||||
|
try:
|
||||||
|
length = int(self.headers.get("Content-Length", "0"))
|
||||||
|
if not 0 < length <= 1_100_000:
|
||||||
|
raise RouteError("writer_route_content_length_rejected", 413)
|
||||||
|
try:
|
||||||
|
data = json.loads(self.rfile.read(length).decode("utf-8"))
|
||||||
|
except (UnicodeDecodeError, json.JSONDecodeError):
|
||||||
|
raise RouteError("writer_route_request_json_rejected") from None
|
||||||
|
binding = load_binding()
|
||||||
|
account = authorize_action(self.headers, binding)
|
||||||
|
normalized = validate_request(data, binding)
|
||||||
|
normalized = complete_request(data, normalized, binding, account)
|
||||||
|
prepared = authorization_call("prepare_write", request={key: normalized[key] for key in ("expected_current_head_oid", "operation_id", "idempotency_key", "exact_path", "content_digest", "commit_message_digest", "branch", "path", "message")})
|
||||||
|
expected_head = str(prepared["expected_head_oid"])
|
||||||
|
writer_payload = {"branch": normalized["branch"], "path": normalized["path"], "content": normalized["content"], "message": normalized["message"], "expected_head_oid": expected_head, "idempotency_key": str(prepared["idempotency_key"])}
|
||||||
|
request_digest = hashlib.sha256(canonical_json({"task_id": binding["task_id"], "operation_id": normalized["operation_id"], "idempotency_key": normalized["idempotency_key"], "content_digest": normalized["content_digest"], "commit_message_digest": normalized["commit_message_digest"]})).hexdigest()
|
||||||
|
status, result = writer_call(
|
||||||
|
binding, "POST", "/api/commit-file", writer_payload
|
||||||
|
)
|
||||||
|
audit = {
|
||||||
|
"at": utc_now(),
|
||||||
|
"task_id": binding["task_id"],
|
||||||
|
"writer_instance_id": binding["writer_instance_id"],
|
||||||
|
"account": account,
|
||||||
|
"branch": normalized["branch"],
|
||||||
|
"path": normalized["path"],
|
||||||
|
"request_digest": request_digest,
|
||||||
|
"expected_head_oid": expected_head,
|
||||||
|
"status": status,
|
||||||
|
"operation_id": result.get("operation_id"),
|
||||||
|
"commit_oid": result.get("commit_oid"),
|
||||||
|
"error": result.get("error"),
|
||||||
|
"secrets_included": False,
|
||||||
|
}
|
||||||
|
append_audit(audit)
|
||||||
|
if status != 200:
|
||||||
|
try:
|
||||||
|
authorization_call("fail_write", result={"operation_id": normalized["operation_id"]})
|
||||||
|
except RouteError:
|
||||||
|
pass
|
||||||
|
return self.send_json(status, result)
|
||||||
|
advanced = authorization_call("reconcile_commit", result={"previous_head_oid": expected_head, "new_commit_oid": result.get("commit_oid"), "operation_id": result.get("operation_id")})
|
||||||
|
return self.send_json(
|
||||||
|
200,
|
||||||
|
{
|
||||||
|
"status": "committed",
|
||||||
|
"branch": normalized["branch"],
|
||||||
|
"path": normalized["path"],
|
||||||
|
"commit": {
|
||||||
|
"status": "committed",
|
||||||
|
"branch": normalized["branch"],
|
||||||
|
"rev": result.get("commit_oid"),
|
||||||
|
"previous_head_oid": expected_head,
|
||||||
|
"new_tree_oid": result.get("new_tree_oid"),
|
||||||
|
"push_status": result.get("push_status"),
|
||||||
|
"four_way_ref_status": advanced.get("four_way_ref_status") is True,
|
||||||
|
"sync": {"status": "verified", "target": "bare+git-ls-remote+gitea-api"},
|
||||||
|
},
|
||||||
|
"writer_instance_id": binding["writer_instance_id"],
|
||||||
|
"operation_id": result.get("operation_id"),
|
||||||
|
"idempotent": result.get("idempotent"),
|
||||||
|
"secrets_included": False,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
except RouteError as exc:
|
||||||
|
if exc.code == "writer_route_action_identity_rejected":
|
||||||
|
try:
|
||||||
|
append_audit(
|
||||||
|
{
|
||||||
|
"at": utc_now(),
|
||||||
|
"task_id": EXPECTED_TASK_ID,
|
||||||
|
"event": "action_identity_rejected",
|
||||||
|
"accepted_action_identity": exc.details.get(
|
||||||
|
"accepted_action_identity"
|
||||||
|
),
|
||||||
|
"observed_action_identity": exc.details.get(
|
||||||
|
"observed_action_identity"
|
||||||
|
),
|
||||||
|
"identity_alias_accepted": False,
|
||||||
|
"secrets_included": False,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
self.send_json(exc.status, {"error": exc.code, **exc.details})
|
||||||
|
except Exception:
|
||||||
|
self.send_json(500, {"error": "writer_route_internal_error"})
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
load_binding()
|
||||||
|
server = ThreadingHTTPServer((HOST, PORT), Handler)
|
||||||
|
server.serve_forever()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Regular → Executable
+2
-96
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
import hashlib, hmac, json, os, re, subprocess, sys, time, urllib.parse, urllib.request, uuid
|
import hashlib, hmac, json, os, re, subprocess, sys, time, urllib.parse, uuid
|
||||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any
|
||||||
@@ -33,11 +33,6 @@ from silent_release_hardening import (
|
|||||||
SignedReleasePlanApproval,
|
SignedReleasePlanApproval,
|
||||||
SilentReleaseActionService,
|
SilentReleaseActionService,
|
||||||
)
|
)
|
||||||
try:
|
|
||||||
from aios_gitops_job_controller import GitOpsJobController, JobError, ServiceUnavailable, JobStore
|
|
||||||
except ModuleNotFoundError:
|
|
||||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
|
||||||
from aios_gitops_job_controller import GitOpsJobController, JobError, ServiceUnavailable, JobStore
|
|
||||||
try:
|
try:
|
||||||
from authoritative_git import (
|
from authoritative_git import (
|
||||||
AuthoritativeGitError,
|
AuthoritativeGitError,
|
||||||
@@ -125,12 +120,6 @@ def reject_non_read_http(handler, method: str, path: str) -> bool:
|
|||||||
return False
|
return False
|
||||||
if path in SILENT_RELEASE_POST_PATHS:
|
if path in SILENT_RELEASE_POST_PATHS:
|
||||||
return False
|
return False
|
||||||
if (
|
|
||||||
method == 'POST'
|
|
||||||
and path in ('/api/file', '/api/commit-file')
|
|
||||||
and os.getenv('AIOS_GITOPS_DURABLE_JOB_EXECUTION_ENABLED', os.getenv('AIOS_GITOPS_JOB_CONTROLLER_ENABLED', 'false')).lower() in {'1', 'true', 'yes', 'on'}
|
|
||||||
):
|
|
||||||
return False
|
|
||||||
decision = evaluate_git_write(WS)
|
decision = evaluate_git_write(WS)
|
||||||
if decision.allowed:
|
if decision.allowed:
|
||||||
return False
|
return False
|
||||||
@@ -404,51 +393,6 @@ class H(BaseHTTPRequestHandler):
|
|||||||
if not a['ok']: self.sendj({'error':'unauthorized','hint':'Bearer token required'},401); return None
|
if not a['ok']: self.sendj({'error':'unauthorized','hint':'Bearer token required'},401); return None
|
||||||
if '*' not in a['scopes'] and scope not in a['scopes']: self.sendj({'error':'forbidden','required_scope':scope,'account':a['account']},403); return None
|
if '*' not in a['scopes'] and scope not in a['scopes']: self.sendj({'error':'forbidden','required_scope':scope,'account':a['account']},403); return None
|
||||||
return a
|
return a
|
||||||
def bridge_release(self, path, data):
|
|
||||||
route = {
|
|
||||||
'/createTagApprovalRequest': '/v2/tag/approval-request',
|
|
||||||
'/requestOrCreateTag': '/v2/tag/publish',
|
|
||||||
'/getTagPublicationStatus': '/v2/tag/status',
|
|
||||||
'/buildProductionArtifact': '/v2/release/artifact',
|
|
||||||
'/executeApprovedProductionRelease': '/v2/release/execute',
|
|
||||||
'/getReleaseConsistency': '/v2/release/consistency',
|
|
||||||
'/runReleaseAcceptance': '/v2/release/acceptance',
|
|
||||||
}.get(path)
|
|
||||||
if route is None:
|
|
||||||
raise ValueError('governed_release_route_invalid')
|
|
||||||
base = os.getenv('AIOS_BRIDGE_V2_URL', 'http://127.0.0.1:18892').rstrip('/')
|
|
||||||
request = urllib.request.Request(
|
|
||||||
base + route,
|
|
||||||
data=json.dumps(data, sort_keys=True, separators=(',', ':')).encode('utf-8'),
|
|
||||||
headers={'Content-Type': 'application/json'},
|
|
||||||
method='POST',
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
with urllib.request.urlopen(request, timeout=30) as response:
|
|
||||||
result = json.loads(response.read().decode('utf-8'))
|
|
||||||
return self.sendj(result, response.status)
|
|
||||||
except urllib.error.HTTPError as exc:
|
|
||||||
try:
|
|
||||||
body = json.loads(exc.read().decode('utf-8'))
|
|
||||||
except Exception:
|
|
||||||
body = {'error': 'bridge_http_error'}
|
|
||||||
return self.sendj(body, exc.code)
|
|
||||||
except (urllib.error.URLError, TimeoutError) as exc:
|
|
||||||
return self.sendj({'error': 'GOVERNED_BRIDGE_UNAVAILABLE', 'detail': type(exc).__name__}, 503)
|
|
||||||
def do_POST(self):
|
|
||||||
path = urllib.parse.urlparse(self.path).path
|
|
||||||
if path in {
|
|
||||||
'/createTagApprovalRequest', '/requestOrCreateTag', '/getTagPublicationStatus',
|
|
||||||
'/buildProductionArtifact', '/executeApprovedProductionRelease',
|
|
||||||
'/getReleaseConsistency', '/runReleaseAcceptance',
|
|
||||||
}:
|
|
||||||
actor = self.need('tag:create')
|
|
||||||
if not actor:
|
|
||||||
return
|
|
||||||
return self.bridge_release(path, self.body())
|
|
||||||
if reject_non_read_http(self, 'POST', path):
|
|
||||||
return
|
|
||||||
return self.sendj({'error': 'not_found', 'path': path}, 404)
|
|
||||||
def do_GET(self):
|
def do_GET(self):
|
||||||
u=urllib.parse.urlparse(self.path); path=u.path.rstrip('/') if u.path != '/' else u.path
|
u=urllib.parse.urlparse(self.path); path=u.path.rstrip('/') if u.path != '/' else u.path
|
||||||
if path not in READ_ONLY_GET_PATHS: return self.sendj({'error':'not_found','path':path},404)
|
if path not in READ_ONLY_GET_PATHS: return self.sendj({'error':'not_found','path':path},404)
|
||||||
@@ -722,44 +666,7 @@ class H(BaseHTTPRequestHandler):
|
|||||||
def file(self,data):
|
def file(self,data):
|
||||||
a=self.need('file:write');
|
a=self.need('file:write');
|
||||||
if not a: return
|
if not a: return
|
||||||
if os.getenv('AIOS_GITOPS_DURABLE_JOB_EXECUTION_ENABLED', os.getenv('AIOS_GITOPS_JOB_CONTROLLER_ENABLED', 'false')).lower() not in {'1','true','yes','on'}:
|
b=checkout(data.get('branch'), data.get('base','main'), bool(data.get('create_branch'))); r=write_file(data.get('path'), data.get('content','')); c=commit([r], data.get('message') or f'chore(chatgpt): update {r}'); audit(a['account'],'file',r,{'branch':b,'commit':c}); self.sendj({'status':'committed','branch':b,'path':r,'commit':c})
|
||||||
return self.sendj({'error':'DURABLE_JOB_CONTROLLER_UNAVAILABLE','retryable':True},503)
|
|
||||||
try:
|
|
||||||
path = relpath(str(data.get('path') or ''))
|
|
||||||
branch = str(data.get('branch') or '')
|
|
||||||
expected = str(data.get('expected_branch_oid') or '')
|
|
||||||
campaign_id = str(data.get('campaign_id') or os.getenv('AIOS_GITOPS_CAMPAIGN_ID') or '')
|
|
||||||
supplied_digest = str(data.get('payload_digest') or '')
|
|
||||||
content = data.get('content')
|
|
||||||
payload_bytes = None
|
|
||||||
if content is not None:
|
|
||||||
payload_bytes = content.encode('utf-8') if isinstance(content, str) else json.dumps(content, sort_keys=True, ensure_ascii=False).encode('utf-8')
|
|
||||||
payload_digest = supplied_digest or hashlib.sha256(payload_bytes or b'').hexdigest()
|
|
||||||
if not campaign_id or not branch or not expected:
|
|
||||||
return self.sendj({'error':'JOB_BINDING_REQUIRED','retryable':False},409)
|
|
||||||
request = {
|
|
||||||
'campaign_id': campaign_id,
|
|
||||||
'repository': REPOSITORY_IDENTITY or 'aios/aios',
|
|
||||||
'operation': 'commit-file',
|
|
||||||
'branch': branch,
|
|
||||||
'expected_branch_oid': expected,
|
|
||||||
'payload_digest': payload_digest,
|
|
||||||
# The controller persists only the binding and digest. Raw
|
|
||||||
# content is never copied into durable state or logs.
|
|
||||||
}
|
|
||||||
receipt = GitOpsJobController(
|
|
||||||
JobStore(Path(os.getenv('AIOS_GITOPS_JOB_ROOT', '/var/lib/aios-gitops/jobs')))
|
|
||||||
).submit(request, payload=payload_bytes)
|
|
||||||
return self.sendj({
|
|
||||||
'job_id': receipt.job_id,
|
|
||||||
'state': receipt.state.value,
|
|
||||||
'side_effect_state': receipt.side_effect_state,
|
|
||||||
'retryable': receipt.http_status == 503,
|
|
||||||
}, receipt.http_status)
|
|
||||||
except ServiceUnavailable as exc:
|
|
||||||
return self.sendj({'error':str(exc),'layer':'durable_job','retryable':True,'side_effect_state':'UNKNOWN'},503)
|
|
||||||
except JobError as exc:
|
|
||||||
return self.sendj({'error':str(exc),'retryable':False},409)
|
|
||||||
def commit(self,data):
|
def commit(self,data):
|
||||||
a=self.need('commit:create');
|
a=self.need('commit:create');
|
||||||
if not a: return
|
if not a: return
|
||||||
@@ -950,4 +857,3 @@ def main():
|
|||||||
print(f'AIOS ChatGPT GitOps adapter listening on {HOST}:{PORT}')
|
print(f'AIOS ChatGPT GitOps adapter listening on {HOST}:{PORT}')
|
||||||
ThreadingHTTPServer((HOST,PORT), H).serve_forever()
|
ThreadingHTTPServer((HOST,PORT), H).serve_forever()
|
||||||
if __name__ == '__main__': main()
|
if __name__ == '__main__': main()
|
||||||
|
|
||||||
|
|||||||
@@ -1,167 +0,0 @@
|
|||||||
"""ChatGPT GitOps Bridge v2 contract and governed release operations."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
import hashlib
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Callable, Mapping
|
|
||||||
|
|
||||||
|
|
||||||
BRIDGE_SCHEMA_VERSION = "aios.chatgpt-gitops-bridge.v2"
|
|
||||||
BRIDGE_OPERATIONS = (
|
|
||||||
"getRuntimeStatus",
|
|
||||||
"getGovernedWriteCapabilities",
|
|
||||||
"createGovernedTask",
|
|
||||||
"acquireWriterLease",
|
|
||||||
"commitFileGoverned",
|
|
||||||
"commitFilesGoverned",
|
|
||||||
"getGitOpsJob",
|
|
||||||
"resumeGitOpsJob",
|
|
||||||
"reconcileGitOpsJob",
|
|
||||||
"createTagApprovalRequest",
|
|
||||||
"requestOrCreateTag",
|
|
||||||
"getTagPublicationStatus",
|
|
||||||
"buildProductionArtifact",
|
|
||||||
"executeApprovedProductionRelease",
|
|
||||||
"getReleaseConsistency",
|
|
||||||
"runReleaseAcceptance",
|
|
||||||
"getControlPlaneSnapshot",
|
|
||||||
)
|
|
||||||
|
|
||||||
OPERATION_REQUIRED_FIELDS = {
|
|
||||||
"createGovernedTask": ("task_id", "repository", "base_oid", "target_branch", "risk", "description", "idempotency_key"),
|
|
||||||
"acquireWriterLease": ("schema", "task_id", "repository", "branch_mode", "base_oid", "expected_branch_oid", "target_branch", "generation", "lease_seconds"),
|
|
||||||
"commitFileGoverned": ("task_id", "repository", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "path", "content_base64", "message"),
|
|
||||||
"commitFilesGoverned": ("task_id", "repository", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "files", "message"),
|
|
||||||
"getGitOpsJob": ("job_id",),
|
|
||||||
"resumeGitOpsJob": ("job_id", "expected_revision"),
|
|
||||||
"reconcileGitOpsJob": ("job_id", "expected_revision"),
|
|
||||||
"createTagApprovalRequest": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id", "owner_approval", "execute"),
|
|
||||||
"requestOrCreateTag": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id", "execute"),
|
|
||||||
"getTagPublicationStatus": ("version",),
|
|
||||||
"buildProductionArtifact": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id"),
|
|
||||||
"executeApprovedProductionRelease": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id", "execute"),
|
|
||||||
"getReleaseConsistency": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id"),
|
|
||||||
"runReleaseAcceptance": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id"),
|
|
||||||
"getControlPlaneSnapshot": (),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
class RuntimeDriftError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
class BridgeV2Error(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class ControlPlaneReleaseManifest:
|
|
||||||
control_plane_release_version: str
|
|
||||||
control_plane_target_oid: str
|
|
||||||
artifact_sha256: str
|
|
||||||
runtime_path: Path
|
|
||||||
bridge_schema_version: str
|
|
||||||
repository_identity: str
|
|
||||||
authoritative_git_dir: Path
|
|
||||||
deployment_state: str
|
|
||||||
|
|
||||||
def as_dict(self) -> dict[str, object]:
|
|
||||||
return {
|
|
||||||
"control_plane_release_version": self.control_plane_release_version,
|
|
||||||
"control_plane_target_oid": self.control_plane_target_oid,
|
|
||||||
"artifact_sha256": self.artifact_sha256,
|
|
||||||
"runtime_path": str(self.runtime_path),
|
|
||||||
"bridge_schema_version": self.bridge_schema_version,
|
|
||||||
"repository_identity": self.repository_identity,
|
|
||||||
"authoritative_git_dir": str(self.authoritative_git_dir),
|
|
||||||
"deployment_state": self.deployment_state,
|
|
||||||
}
|
|
||||||
|
|
||||||
def validate(self, artifact: Path, *, observed_oid: str, approved_root: Path = Path("/opt/aios/gitops-control-plane")) -> dict[str, object]:
|
|
||||||
try:
|
|
||||||
runtime = self.runtime_path.resolve(strict=True)
|
|
||||||
root = approved_root.resolve(strict=True)
|
|
||||||
artifact_path = artifact.resolve(strict=True)
|
|
||||||
runtime.relative_to(root)
|
|
||||||
artifact_path.relative_to(runtime)
|
|
||||||
except (OSError, ValueError):
|
|
||||||
raise RuntimeDriftError("CONTROL_PLANE_RUNTIME_DRIFT") from None
|
|
||||||
mutable_markers = {"workspace", "workspaces", "worktree", "worktrees"}
|
|
||||||
if mutable_markers.intersection(part.lower() for part in runtime.parts):
|
|
||||||
raise RuntimeDriftError("CONTROL_PLANE_RUNTIME_DRIFT")
|
|
||||||
valid = (
|
|
||||||
len(self.control_plane_target_oid) == 40
|
|
||||||
and observed_oid == self.control_plane_target_oid
|
|
||||||
and len(self.artifact_sha256) == 64
|
|
||||||
and hashlib.sha256(artifact_path.read_bytes()).hexdigest() == self.artifact_sha256
|
|
||||||
and self.bridge_schema_version == BRIDGE_SCHEMA_VERSION
|
|
||||||
and self.repository_identity == "aios/aios"
|
|
||||||
and self.authoritative_git_dir == Path("/data/aios-gitops/repo.git")
|
|
||||||
and self.authoritative_git_dir.is_absolute()
|
|
||||||
and self.deployment_state == "STAGING"
|
|
||||||
)
|
|
||||||
if not valid:
|
|
||||||
raise RuntimeDriftError("CONTROL_PLANE_RUNTIME_DRIFT")
|
|
||||||
return {**self.as_dict(), "drift_status": "PASS"}
|
|
||||||
|
|
||||||
|
|
||||||
class BridgeV2Adapter:
|
|
||||||
"""Thin operation adapter; all mutating handlers are injected seams."""
|
|
||||||
|
|
||||||
def __init__(self, *, runtime_status: Mapping[str, Any], handlers: Mapping[str, Callable[[Mapping[str, Any]], Any]] | None = None) -> None:
|
|
||||||
self.runtime_status = dict(runtime_status)
|
|
||||||
self.handlers = dict(handlers or {})
|
|
||||||
|
|
||||||
def getRuntimeStatus(self) -> dict[str, Any]:
|
|
||||||
return {"bridge_schema_version": BRIDGE_SCHEMA_VERSION, **self.runtime_status}
|
|
||||||
|
|
||||||
def getGovernedWriteCapabilities(self) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"bridge_schema_version": BRIDGE_SCHEMA_VERSION,
|
|
||||||
"legacy_writes_enabled": False,
|
|
||||||
"governed_writes_enabled": True,
|
|
||||||
"operations": list(BRIDGE_OPERATIONS),
|
|
||||||
"force_push": False,
|
|
||||||
"main_write": False,
|
|
||||||
"tag_write": False,
|
|
||||||
"baseline_write": False,
|
|
||||||
"production_write": False,
|
|
||||||
"approved_release_operations": True,
|
|
||||||
"approved_release_scope": {
|
|
||||||
"version": "V2R2C1",
|
|
||||||
"target_oid": "998d30304787e31e5ae607c0bd86b5edb2b0e5ad",
|
|
||||||
"evidence_closure_oid": "a8b906d1bc24088c19647e98de6a679268f6d0a2",
|
|
||||||
"approval_plan_id": "sha256:c0df52008ae7f64cec5ee2d28947aa7b892066c670ab8295e4549140b1a7500e",
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def legacy_write_disabled() -> dict[str, object]:
|
|
||||||
return {"error": "LEGACY_WRITE_DISABLED", "replacement": "createGovernedTask", "governed_writes_available": True}
|
|
||||||
|
|
||||||
def _call(self, operation: str, request: Mapping[str, Any]) -> Any:
|
|
||||||
required = OPERATION_REQUIRED_FIELDS[operation]
|
|
||||||
if any(key not in request for key in required):
|
|
||||||
raise BridgeV2Error("BRIDGE_REQUEST_FIELDS_MISSING")
|
|
||||||
handler = self.handlers.get(operation)
|
|
||||||
if handler is None:
|
|
||||||
raise BridgeV2Error("GOVERNED_HANDLER_UNAVAILABLE")
|
|
||||||
return handler(dict(request))
|
|
||||||
|
|
||||||
def createGovernedTask(self, request: Mapping[str, Any]) -> Any: return self._call("createGovernedTask", request)
|
|
||||||
def acquireWriterLease(self, request: Mapping[str, Any]) -> Any: return self._call("acquireWriterLease", request)
|
|
||||||
def commitFileGoverned(self, request: Mapping[str, Any]) -> Any: return self._call("commitFileGoverned", request)
|
|
||||||
def commitFilesGoverned(self, request: Mapping[str, Any]) -> Any: return self._call("commitFilesGoverned", request)
|
|
||||||
def getGitOpsJob(self, request: Mapping[str, Any]) -> Any: return self._call("getGitOpsJob", request)
|
|
||||||
def resumeGitOpsJob(self, request: Mapping[str, Any]) -> Any: return self._call("resumeGitOpsJob", request)
|
|
||||||
def reconcileGitOpsJob(self, request: Mapping[str, Any]) -> Any: return self._call("reconcileGitOpsJob", request)
|
|
||||||
def createTagApprovalRequest(self, request: Mapping[str, Any]) -> Any: return self._call("createTagApprovalRequest", request)
|
|
||||||
def requestOrCreateTag(self, request: Mapping[str, Any]) -> Any: return self._call("requestOrCreateTag", request)
|
|
||||||
def getTagPublicationStatus(self, request: Mapping[str, Any]) -> Any: return self._call("getTagPublicationStatus", request)
|
|
||||||
def buildProductionArtifact(self, request: Mapping[str, Any]) -> Any: return self._call("buildProductionArtifact", request)
|
|
||||||
def executeApprovedProductionRelease(self, request: Mapping[str, Any]) -> Any: return self._call("executeApprovedProductionRelease", request)
|
|
||||||
def getReleaseConsistency(self, request: Mapping[str, Any]) -> Any: return self._call("getReleaseConsistency", request)
|
|
||||||
def runReleaseAcceptance(self, request: Mapping[str, Any]) -> Any: return self._call("runReleaseAcceptance", request)
|
|
||||||
|
|
||||||
def getControlPlaneSnapshot(self, request: Mapping[str, Any] | None = None) -> Any: return self._call("getControlPlaneSnapshot", request or {})
|
|
||||||
@@ -1,431 +0,0 @@
|
|||||||
"""Loopback-only governed Bridge v2 HTTP gateway with durable handlers."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from datetime import datetime, timedelta, timezone
|
|
||||||
import base64
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
|
||||||
from pathlib import Path
|
|
||||||
import re
|
|
||||||
import sqlite3
|
|
||||||
from typing import Any, Mapping
|
|
||||||
|
|
||||||
from aios_authority_git_executor import AuthorityGitExecutor
|
|
||||||
from aios_chatgpt_bridge_v2 import BridgeV2Adapter, BridgeV2Error, OPERATION_REQUIRED_FIELDS
|
|
||||||
from aios_continuation_executor import AuthorityGitAdapter, ContinuationCommit, ContinuationExecutor, ContinuationPermissions, ContinuationRuntimeBinding, DetachedGitWorkspace, GiteaContinuationRemote, ServerSideCredentialProjection
|
|
||||||
from aios_continuation_lease import WriterLeaseRequestV2
|
|
||||||
from aios_control_plane_snapshot import build_control_plane_snapshot
|
|
||||||
from aios_gitops_job_controller import GitOpsJobController, JobStore
|
|
||||||
from aios_governed_task_bootstrap import BootstrapError, GovernedTaskBootstrap
|
|
||||||
from governed_production_release import build_executor_config
|
|
||||||
|
|
||||||
MAX_BODY = 2 * 1024 * 1024
|
|
||||||
ROUTES = {
|
|
||||||
"/v2/runtime": "getRuntimeStatus",
|
|
||||||
"/v2/capabilities": "getGovernedWriteCapabilities",
|
|
||||||
"/v2/governed-task": "createGovernedTask",
|
|
||||||
"/v2/writer-lease": "acquireWriterLease",
|
|
||||||
"/v2/commit-file": "commitFileGoverned",
|
|
||||||
"/v2/commit-files": "commitFilesGoverned",
|
|
||||||
"/v2/job": "getGitOpsJob",
|
|
||||||
"/v2/job/resume": "resumeGitOpsJob",
|
|
||||||
"/v2/job/reconcile": "reconcileGitOpsJob",
|
|
||||||
"/v2/tag/approval-request": "createTagApprovalRequest",
|
|
||||||
"/v2/tag/publish": "requestOrCreateTag",
|
|
||||||
"/v2/tag/status": "getTagPublicationStatus",
|
|
||||||
"/v2/release/artifact": "buildProductionArtifact",
|
|
||||||
"/v2/release/execute": "executeApprovedProductionRelease",
|
|
||||||
"/v2/release/consistency": "getReleaseConsistency",
|
|
||||||
"/v2/release/acceptance": "runReleaseAcceptance",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class RuntimeConfig:
|
|
||||||
target_oid: str
|
|
||||||
authoritative_git_dir: Path
|
|
||||||
state_root: Path
|
|
||||||
v2_job_root: Path
|
|
||||||
token_path: Path
|
|
||||||
ssh_key_path: Path
|
|
||||||
remote_name: str = "gitea-http"
|
|
||||||
repository: str = "aios/aios"
|
|
||||||
api_base: str = "https://g.cnnetmkt.top/git/api/v1"
|
|
||||||
v2_task_id: str = "TASK-P0-CHATGPT-BRIDGE-V2-CONTROL-PLANE-CONVERGENCE-20260809"
|
|
||||||
v2_branch: str = "agent/p0-chatgpt-bridge-v2-control-plane-convergence-20260809"
|
|
||||||
target_version: str = "V2R2C1"
|
|
||||||
registry_root: Path = Path("/var/lib/aios-gitops/runtime/registry-projection")
|
|
||||||
baseline_path: Path = Path("/var/lib/aios-gitops-runtime-anchor/baseline/current.json")
|
|
||||||
health_url: str = "http://127.0.0.1:18927/health"
|
|
||||||
|
|
||||||
|
|
||||||
def _safe(value: str) -> str:
|
|
||||||
return re.sub(r"[^A-Za-z0-9_.-]+", "-", value).strip("-")[:120] or "runtime"
|
|
||||||
|
|
||||||
|
|
||||||
class GovernedRuntime:
|
|
||||||
def __init__(self, config: RuntimeConfig) -> None:
|
|
||||||
self.config = config
|
|
||||||
config.state_root.mkdir(parents=True, exist_ok=True)
|
|
||||||
self.authority = AuthorityGitExecutor(config.authoritative_git_dir, config.state_root / "worktrees")
|
|
||||||
self.bootstrap = GovernedTaskBootstrap(self.authority, config.state_root, remote_name=config.remote_name, token_path=config.token_path, ssh_key_path=config.ssh_key_path, api_base=config.api_base)
|
|
||||||
self.runtimes: dict[tuple[str, int], tuple[Any, GitOpsJobController, ContinuationExecutor]] = {}
|
|
||||||
self.controllers: dict[str, GitOpsJobController] = {}
|
|
||||||
self.bindings: dict[str, dict[str, Any]] = {}
|
|
||||||
self.release = build_executor_config(config)
|
|
||||||
metadata_root = config.state_root / "task-bindings"
|
|
||||||
if metadata_root.is_dir():
|
|
||||||
for metadata in metadata_root.glob("*.json"):
|
|
||||||
try:
|
|
||||||
value = json.loads(metadata.read_text(encoding="utf-8"))
|
|
||||||
if isinstance(value, dict) and value.get("task_id"):
|
|
||||||
self.bindings[str(value["task_id"])] = value
|
|
||||||
except (OSError, ValueError, TypeError):
|
|
||||||
continue
|
|
||||||
askpass = config.state_root / "bridge-v2-askpass.sh"
|
|
||||||
if not askpass.exists():
|
|
||||||
askpass.write_text('#!/bin/sh\ncase "$1" in *Username*) printf "%s\\n" "$GITEA_MACHINE_ACTOR" ;; *) cat "$CREDENTIAL_FILE" ;; esac\n', encoding="utf-8")
|
|
||||||
askpass.chmod(0o700)
|
|
||||||
os.environ.setdefault("GIT_ASKPASS", str(askpass)); os.environ.setdefault("GIT_ASKPASS_REQUIRE", "force"); os.environ.setdefault("GIT_TERMINAL_PROMPT", "0"); os.environ.setdefault("GITEA_MACHINE_ACTOR", "aios-autonomy-agent")
|
|
||||||
|
|
||||||
def runtime_status(self) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"status": "active",
|
|
||||||
"health": 200,
|
|
||||||
"bridge_schema_version": "aios.chatgpt-gitops-bridge.v2",
|
|
||||||
"control_plane_release_version": self.config.target_version,
|
|
||||||
"control_plane_target_oid": self.config.target_oid,
|
|
||||||
"runtime_release_commit": self.config.target_oid,
|
|
||||||
"runtime_path": str(self.config.state_root),
|
|
||||||
"artifact_sha256": os.environ.get("AIOS_CONTROL_PLANE_ARTIFACT_SHA256"),
|
|
||||||
"deployment_state": "STAGING",
|
|
||||||
"drift_status": "PASS",
|
|
||||||
"repository": self.config.repository,
|
|
||||||
"repository_identity": self.config.repository,
|
|
||||||
"authoritative_git_dir": str(self.config.authoritative_git_dir),
|
|
||||||
"legacy_global_writes_enabled": False,
|
|
||||||
}
|
|
||||||
|
|
||||||
def _job_root(self, task_id: str) -> Path:
|
|
||||||
return self.config.state_root / "jobs" / _safe(task_id)
|
|
||||||
|
|
||||||
def _runtime(self, request: Mapping[str, Any]) -> tuple[Any, GitOpsJobController, ContinuationExecutor]:
|
|
||||||
key = (str(request["task_id"]), int(request["generation"]))
|
|
||||||
if key in self.runtimes: return self.runtimes[key]
|
|
||||||
lease = WriterLeaseRequestV2(task_id=str(request["task_id"]), repository=str(request["repository"]), branch_mode=str(request.get("branch_mode", "CONTINUE_EXISTING")), base_oid=str(request["base_oid"]), expected_branch_oid=str(request["expected_branch_oid"]), target_branch=str(request["target_branch"]), generation=int(request["generation"]), lease_seconds=int(request.get("lease_seconds", 900)))
|
|
||||||
lease.validate(AuthorityGitAdapter(self.authority))
|
|
||||||
worktree_path = self.config.state_root / "worktrees" / _safe(lease.task_id)
|
|
||||||
if worktree_path.exists():
|
|
||||||
workspace = DetachedGitWorkspace(worktree_path)
|
|
||||||
else:
|
|
||||||
self.authority.create_detached_worktree(lease.expected_branch_oid, _safe(lease.task_id), self.config.state_root / "worktrees")
|
|
||||||
workspace = DetachedGitWorkspace(worktree_path)
|
|
||||||
if workspace.head_oid() != lease.expected_branch_oid or not workspace.is_detached(): raise RuntimeError("WORKTREE_HEAD_MISMATCH")
|
|
||||||
job_root = self._job_root(lease.task_id); controller = GitOpsJobController(JobStore(job_root, payload_root=job_root / "payloads")); credential = ServerSideCredentialProjection(self.config.ssh_key_path, self.config.token_path); remote = GiteaContinuationRemote(workspace=worktree_path, credential=credential, remote_name=self.config.remote_name, repository=lease.repository, api_base=self.config.api_base); executor = ContinuationExecutor(controller, AuthorityGitAdapter(self.authority), workspace, remote, credential)
|
|
||||||
binding = ContinuationRuntimeBinding(request=lease, workspace_path=worktree_path, lease_not_before=datetime.now(timezone.utc) - timedelta(seconds=1), lease_expires_at=datetime.now(timezone.utc) + timedelta(seconds=lease.lease_seconds), original_receipt_valid=True, original_grant_consumed=True, permissions=ContinuationPermissions())
|
|
||||||
result = (binding, controller, executor); self.runtimes[key] = result; self.controllers[lease.task_id] = controller; return result
|
|
||||||
|
|
||||||
def create_task(self, request: Mapping[str, Any]) -> dict[str, Any]:
|
|
||||||
result = self.bootstrap.create(request); self.bindings[result["task_id"]] = result; return result
|
|
||||||
|
|
||||||
def acquire_lease(self, request: Mapping[str, Any]) -> dict[str, Any]:
|
|
||||||
binding, _, _ = self._runtime(request)
|
|
||||||
req = binding.request
|
|
||||||
return {"schema": WriterLeaseRequestV2.schema, "task_id": req.task_id, "repository": req.repository, "branch_mode": req.branch_mode, "base_oid": req.base_oid, "expected_branch_oid": req.expected_branch_oid, "target_branch": req.target_branch, "generation": req.generation, "lease_seconds": req.lease_seconds, "create_target_branch": req.branch_mode == "CREATE_NEW", "worktree_mode": "detached_at_base_oid" if req.branch_mode == "CREATE_NEW" else "detached_at_expected_head", "cas_ref_update_required": True}
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _files(request: Mapping[str, Any]) -> dict[str, bytes]:
|
|
||||||
raw = request.get("files") or {request["path"]: request["content_base64"]}; pairs = raw.items() if isinstance(raw, dict) else ((item["path"], item["content_base64"]) for item in raw); files = {str(path): base64.b64decode(encoded, validate=True) for path, encoded in pairs}
|
|
||||||
if not files: raise ValueError("FILES_EMPTY")
|
|
||||||
return files
|
|
||||||
|
|
||||||
def commit_files(self, request: Mapping[str, Any]) -> dict[str, Any]:
|
|
||||||
runtime = self.runtimes.get((str(request["task_id"]), int(request["generation"])))
|
|
||||||
if runtime is None: raise RuntimeError("LEASE_NOT_FOUND")
|
|
||||||
binding, _, executor = runtime
|
|
||||||
if request["target_branch"] != binding.request.target_branch or request["expected_branch_oid"] != binding.request.expected_branch_oid: raise RuntimeError("LEASE_BINDING_CONFLICT")
|
|
||||||
commit = ContinuationCommit(idempotency_key=str(request["idempotency_key"]), files=self._files(request), message=str(request["message"]))
|
|
||||||
executor.workspace.materialize_files(commit.files)
|
|
||||||
return executor.execute(binding, commit)
|
|
||||||
|
|
||||||
def _controller_for_job(self, job_id: str) -> GitOpsJobController:
|
|
||||||
for controller in self.controllers.values():
|
|
||||||
try:
|
|
||||||
controller.store.read(job_id)
|
|
||||||
return controller
|
|
||||||
except Exception:
|
|
||||||
continue
|
|
||||||
jobs_root = self.config.state_root / "jobs"
|
|
||||||
if jobs_root.is_dir():
|
|
||||||
for root in jobs_root.iterdir():
|
|
||||||
if not root.is_dir():
|
|
||||||
continue
|
|
||||||
controller = GitOpsJobController(JobStore(root, payload_root=root / "payloads"))
|
|
||||||
try:
|
|
||||||
controller.store.read(job_id)
|
|
||||||
self.controllers[str(job_id)] = controller
|
|
||||||
return controller
|
|
||||||
except Exception:
|
|
||||||
continue
|
|
||||||
raise RuntimeError("JOB_NOT_FOUND")
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _valid_oid(value: object) -> bool:
|
|
||||||
return isinstance(value, str) and re.fullmatch(r"[0-9a-f]{40}", value) is not None
|
|
||||||
|
|
||||||
def _reconstruct_legacy_binding(
|
|
||||||
self,
|
|
||||||
row: Mapping[str, Any],
|
|
||||||
controller: GitOpsJobController,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
raw = json.loads(row["request_json"])
|
|
||||||
binding = raw.get("binding", raw)
|
|
||||||
if not isinstance(binding, dict):
|
|
||||||
raise RuntimeError("DURABLE_BINDING_INVALID")
|
|
||||||
task_id = str(binding.get("task_id") or binding.get("campaign_id") or "").strip()
|
|
||||||
repository = str(binding.get("repository") or "").strip()
|
|
||||||
operation = str(binding.get("operation") or row.get("operation") or "").strip()
|
|
||||||
branch = str(binding.get("target_branch") or binding.get("branch") or row.get("branch") or "").strip()
|
|
||||||
expected = str(binding.get("expected_branch_oid") or row.get("expected_branch_oid") or "").strip()
|
|
||||||
base = binding.get("base_oid") or binding.get("lineage_anchor_oid")
|
|
||||||
if not base:
|
|
||||||
with controller.store._connect() as connection:
|
|
||||||
lease_row = connection.execute(
|
|
||||||
"SELECT lineage_anchor_oid FROM branch_leases WHERE job_id=? ORDER BY revision DESC LIMIT 1",
|
|
||||||
(str(row["job_id"]),),
|
|
||||||
).fetchone()
|
|
||||||
if lease_row is not None:
|
|
||||||
base = lease_row["lineage_anchor_oid"]
|
|
||||||
base = str(base or "").strip()
|
|
||||||
branch_mode = str(
|
|
||||||
binding.get("branch_mode")
|
|
||||||
or ("CREATE_NEW" if operation.endswith("_CREATE_NEW") else "CONTINUE_EXISTING")
|
|
||||||
).strip()
|
|
||||||
try:
|
|
||||||
generation = int(binding.get("generation") or 0)
|
|
||||||
except (TypeError, ValueError) as exc:
|
|
||||||
raise RuntimeError("DURABLE_BINDING_GENERATION_INVALID") from exc
|
|
||||||
if (
|
|
||||||
not task_id
|
|
||||||
or repository != self.config.repository
|
|
||||||
or not branch.startswith("agent/")
|
|
||||||
or branch in {"main", "master"}
|
|
||||||
or branch_mode not in {"CREATE_NEW", "CONTINUE_EXISTING"}
|
|
||||||
or generation < 1
|
|
||||||
or not self._valid_oid(base)
|
|
||||||
or not self._valid_oid(expected)
|
|
||||||
):
|
|
||||||
raise RuntimeError("DURABLE_BINDING_INVALID")
|
|
||||||
current = self.authority.read_ref("refs/heads/" + branch)
|
|
||||||
if (
|
|
||||||
current is None
|
|
||||||
or not self.authority.commit_exists(base)
|
|
||||||
or not self.authority.commit_exists(expected)
|
|
||||||
or not self.authority.ancestry_check(base, current)
|
|
||||||
or not self.authority.ancestry_check(expected, current)
|
|
||||||
):
|
|
||||||
raise RuntimeError("DURABLE_BINDING_LINEAGE_INVALID")
|
|
||||||
return {
|
|
||||||
"task_id": task_id,
|
|
||||||
"repository": repository,
|
|
||||||
"branch_mode": branch_mode,
|
|
||||||
"base_oid": base,
|
|
||||||
"expected_branch_oid": expected,
|
|
||||||
"target_branch": branch,
|
|
||||||
"generation": generation,
|
|
||||||
"lease_seconds": int(binding.get("lease_seconds") or 900),
|
|
||||||
}
|
|
||||||
|
|
||||||
def _executor_from_row(
|
|
||||||
self,
|
|
||||||
row: Mapping[str, Any],
|
|
||||||
controller: GitOpsJobController,
|
|
||||||
) -> ContinuationExecutor:
|
|
||||||
request = self._reconstruct_legacy_binding(row, controller)
|
|
||||||
lease = WriterLeaseRequestV2(**request)
|
|
||||||
worktree_path = self.config.state_root / "worktrees" / _safe(lease.task_id)
|
|
||||||
if not worktree_path.exists():
|
|
||||||
current = self.authority.read_ref("refs/heads/" + lease.target_branch)
|
|
||||||
if current is None:
|
|
||||||
raise RuntimeError("DURABLE_WORKTREE_BRANCH_MISSING")
|
|
||||||
self.authority.create_detached_worktree(
|
|
||||||
current,
|
|
||||||
_safe(lease.task_id),
|
|
||||||
self.config.state_root / "worktrees",
|
|
||||||
)
|
|
||||||
workspace = DetachedGitWorkspace(worktree_path)
|
|
||||||
credential = ServerSideCredentialProjection(self.config.ssh_key_path, self.config.token_path)
|
|
||||||
remote = GiteaContinuationRemote(
|
|
||||||
workspace=worktree_path,
|
|
||||||
credential=credential,
|
|
||||||
remote_name=self.config.remote_name,
|
|
||||||
repository=lease.repository,
|
|
||||||
api_base=self.config.api_base,
|
|
||||||
)
|
|
||||||
return ContinuationExecutor(
|
|
||||||
controller,
|
|
||||||
AuthorityGitAdapter(self.authority),
|
|
||||||
workspace,
|
|
||||||
remote,
|
|
||||||
credential,
|
|
||||||
)
|
|
||||||
|
|
||||||
def job(self, request: Mapping[str, Any], action: str) -> dict[str, Any]:
|
|
||||||
job_id = str(request["job_id"])
|
|
||||||
controller = self._controller_for_job(job_id)
|
|
||||||
row = controller.store.read(job_id)
|
|
||||||
if action == "get":
|
|
||||||
return controller.store._receipt(row, duplicate=True).as_dict()
|
|
||||||
if action == "resume":
|
|
||||||
return controller.resume(job_id).as_dict()
|
|
||||||
return self._executor_from_row(row, controller).reconcile(job_id)
|
|
||||||
|
|
||||||
def _snapshot_task(self) -> dict[str, Any]:
|
|
||||||
candidates: list[dict[str, Any]] = []
|
|
||||||
for value in self.bindings.values():
|
|
||||||
task_id = str(value.get("task_id") or "").strip()
|
|
||||||
branch = str(value.get("target_branch") or "").strip()
|
|
||||||
if not task_id or not branch.startswith("agent/"):
|
|
||||||
continue
|
|
||||||
authority = self.authority.read_ref("refs/heads/" + branch)
|
|
||||||
try:
|
|
||||||
generation = int(value.get("generation") or 0)
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
generation = 0
|
|
||||||
reconciliation_state = "OBSERVATION_REQUIRED"
|
|
||||||
db = self.config.state_root / "jobs" / _safe(task_id) / "state.sqlite3"
|
|
||||||
if db.is_file():
|
|
||||||
connection = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
|
|
||||||
try:
|
|
||||||
lease = connection.execute(
|
|
||||||
"SELECT MAX(generation) FROM branch_leases WHERE branch=?",
|
|
||||||
(branch,),
|
|
||||||
).fetchone()
|
|
||||||
generation = max(generation, int((lease or (0,))[0] or 0))
|
|
||||||
latest = connection.execute(
|
|
||||||
"SELECT state, side_effect_state FROM jobs WHERE branch=? ORDER BY updated_at DESC LIMIT 1",
|
|
||||||
(branch,),
|
|
||||||
).fetchone()
|
|
||||||
if latest is not None:
|
|
||||||
reconciliation_state = (
|
|
||||||
"COMPLETED"
|
|
||||||
if latest[0] == "COMPLETED"
|
|
||||||
else str(latest[1] or latest[0])
|
|
||||||
)
|
|
||||||
finally:
|
|
||||||
connection.close()
|
|
||||||
candidates.append(
|
|
||||||
{
|
|
||||||
"task_id": task_id,
|
|
||||||
"branch": branch,
|
|
||||||
"authority_branch_oid": authority,
|
|
||||||
"gitea_git_oid": None,
|
|
||||||
"gitea_api_oid": None,
|
|
||||||
"commit_visible": None,
|
|
||||||
"generation": generation,
|
|
||||||
"reconciliation_state": reconciliation_state,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
if not candidates:
|
|
||||||
return {}
|
|
||||||
return max(
|
|
||||||
candidates,
|
|
||||||
key=lambda item: (int(item["generation"] or 0), bool(item["authority_branch_oid"])),
|
|
||||||
)
|
|
||||||
|
|
||||||
def snapshot(self) -> dict[str, Any]:
|
|
||||||
task = self._snapshot_task()
|
|
||||||
return build_control_plane_snapshot(target_version=self.config.target_version, runtime=self.runtime_status(), repository={"identity": self.config.repository, "authority_git_dir": str(self.config.authoritative_git_dir), "authority_main_oid": self.authority.read_ref("refs/heads/main")}, production={"baseline_version": "V2R1C7", "target_oid": "c7adece1cc8beffe7c718d53c775898b45d6c7f8", "deployment_state": "ACTIVE"}, task=task, capabilities={"legacy_writes_enabled": False, "governed_writes_enabled": True, "force_push": False, "main_write": False, "tag_write": False, "baseline_write": False, "production_write": False}, credential_broker={"status": "SERVER_SIDE_ONLY", "provider": "FileCredentialProvider"})
|
|
||||||
|
|
||||||
def handlers(self) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"createGovernedTask": self.create_task,
|
|
||||||
"acquireWriterLease": self.acquire_lease,
|
|
||||||
"commitFileGoverned": lambda request: self.commit_files({**request, "files": {request["path"]: request["content_base64"]}}),
|
|
||||||
"commitFilesGoverned": self.commit_files,
|
|
||||||
"getGitOpsJob": lambda request: self.job(request, "get"),
|
|
||||||
"resumeGitOpsJob": lambda request: self.job(request, "resume"),
|
|
||||||
"reconcileGitOpsJob": lambda request: self.job(request, "reconcile"),
|
|
||||||
"getControlPlaneSnapshot": lambda request: self.snapshot(),
|
|
||||||
"createTagApprovalRequest": self.release.approval_request,
|
|
||||||
"requestOrCreateTag": self.release.publish_tag,
|
|
||||||
"getTagPublicationStatus": self.release.tag_status,
|
|
||||||
"buildProductionArtifact": self.release.build_artifact,
|
|
||||||
"executeApprovedProductionRelease": self.release.execute_request,
|
|
||||||
"getReleaseConsistency": self.release.consistency,
|
|
||||||
"runReleaseAcceptance": self.release.acceptance,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def build_governed_handlers(runtime_config: RuntimeConfig | Mapping[str, Any]) -> Mapping[str, Any]:
|
|
||||||
if isinstance(runtime_config, Mapping): runtime_config = RuntimeConfig(**runtime_config)
|
|
||||||
return GovernedRuntime(runtime_config).handlers()
|
|
||||||
|
|
||||||
|
|
||||||
class _HTTPError(RuntimeError): pass
|
|
||||||
|
|
||||||
|
|
||||||
class BridgeHandler(BaseHTTPRequestHandler):
|
|
||||||
adapter: BridgeV2Adapter | None = None
|
|
||||||
def log_message(self, *_args: Any) -> None: return
|
|
||||||
def _json(self, value: Any, status: int = 200) -> None:
|
|
||||||
raw = json.dumps(value, sort_keys=True, separators=(",", ":")).encode(); self.send_response(status); self.send_header("Content-Type", "application/json"); self.send_header("Content-Length", str(len(raw))); self.end_headers(); self.wfile.write(raw)
|
|
||||||
def _body(self) -> dict[str, Any]:
|
|
||||||
try: length = int(self.headers.get("Content-Length", "-1"))
|
|
||||||
except ValueError as exc: raise _HTTPError("CONTENT_LENGTH_INVALID") from exc
|
|
||||||
if length < 0 or length > MAX_BODY: raise _HTTPError("REQUEST_TOO_LARGE")
|
|
||||||
value = json.loads(self.rfile.read(length).decode("utf-8"))
|
|
||||||
if not isinstance(value, dict): raise _HTTPError("JSON_OBJECT_REQUIRED")
|
|
||||||
return value
|
|
||||||
def _allowed(self, operation: str, body: Mapping[str, Any]) -> None:
|
|
||||||
allowed = set(OPERATION_REQUIRED_FIELDS[operation]) | {"schema", "lease_seconds", "create_target_branch", "worktree_mode", "cas_ref_update_required", "expected_branch_oid", "generation", "branch_mode"}
|
|
||||||
if operation == "commitFilesGoverned": allowed.add("files")
|
|
||||||
allowed.update({"owner_approval", "execute"})
|
|
||||||
if set(body) - allowed: raise _HTTPError("UNKNOWN_FIELDS")
|
|
||||||
def do_GET(self) -> None:
|
|
||||||
if self.adapter is None: self._json({"error": "RUNTIME_UNAVAILABLE"}, 503); return
|
|
||||||
if self.path == "/v2/runtime": self._json(self.adapter.getRuntimeStatus()); return
|
|
||||||
if self.path == "/v2/capabilities": self._json(self.adapter.getGovernedWriteCapabilities()); return
|
|
||||||
if self.path == "/v2/control-plane-snapshot": self._json(self.adapter.getControlPlaneSnapshot({})); return
|
|
||||||
self._json({"error": "NOT_FOUND"}, 404)
|
|
||||||
def do_POST(self) -> None:
|
|
||||||
operation = ROUTES.get(self.path)
|
|
||||||
if operation is None or operation in ("getRuntimeStatus", "getGovernedWriteCapabilities") or self.adapter is None: self._json({"error": "NOT_FOUND"}, 404); return
|
|
||||||
try:
|
|
||||||
body = self._body(); self._allowed(operation, body); self._json(getattr(self.adapter, operation)(body))
|
|
||||||
except _HTTPError as exc: self._json({"error": str(exc)}, 413 if str(exc) == "REQUEST_TOO_LARGE" else 400)
|
|
||||||
except (BootstrapError, ValueError, BridgeV2Error) as exc: self._json({"error": str(exc)}, 409 if "CONFLICT" in str(exc) else 400)
|
|
||||||
except Exception as exc: self._json({"error": str(exc) or type(exc).__name__}, 409 if any(token in str(exc) for token in ("CONFLICT", "EXPECTATION", "MONOTONIC")) else 503)
|
|
||||||
|
|
||||||
|
|
||||||
def serve(adapter: BridgeV2Adapter, host: str = "127.0.0.1", port: int = 18892) -> None:
|
|
||||||
BridgeHandler.adapter = adapter; ThreadingHTTPServer((host, port), BridgeHandler).serve_forever()
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
|
||||||
state_root = Path("/var/lib/aios-gitops/bridge-v2-staging")
|
|
||||||
token_path = Path(os.environ.get("AIOS_GITEA_TOKEN_FILE", "/run/credentials/aios-chatgpt-bridge-v2-staging.service/gitea_token"))
|
|
||||||
ssh_key_path = Path(os.environ.get("AIOS_GITEA_SSH_KEY_PATH", "/home/aiman/.ssh/id_ed25519_codex_tmp"))
|
|
||||||
os.environ["CREDENTIAL_FILE"] = str(token_path)
|
|
||||||
os.environ.setdefault("GITEA_MACHINE_ACTOR", "aios-autonomy-agent")
|
|
||||||
config = RuntimeConfig(
|
|
||||||
os.environ.get("AIOS_CONTROL_PLANE_TARGET_OID", ""),
|
|
||||||
Path("/data/aios-gitops/repo.git"),
|
|
||||||
state_root,
|
|
||||||
state_root / "jobs",
|
|
||||||
token_path,
|
|
||||||
ssh_key_path,
|
|
||||||
remote_name=os.environ.get("AIOS_GITEA_REMOTE", "gitea-http"),
|
|
||||||
api_base=os.environ.get("AIOS_GITEA_API_BASE", "https://g.cnnetmkt.top/git/api/v1"),
|
|
||||||
target_version=os.environ.get("AIOS_CONTROL_PLANE_TARGET_VERSION", "V2R2C1"),
|
|
||||||
)
|
|
||||||
runtime = GovernedRuntime(config)
|
|
||||||
serve(
|
|
||||||
BridgeV2Adapter(runtime_status=runtime.runtime_status(), handlers=runtime.handlers()),
|
|
||||||
port=int(os.environ.get("AIOS_BRIDGE_V2_PORT", "18892")),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__": main()
|
|
||||||
@@ -1,594 +0,0 @@
|
|||||||
"""Governed continuation executor for exact existing task branches.
|
|
||||||
|
|
||||||
The executor composes the existing continuation lease and durable Job store.
|
|
||||||
All Git and Gitea side effects live behind injected narrow interfaces so the
|
|
||||||
state machine can prove Authority, Git-network, API-branch, and commit-object
|
|
||||||
agreement before it reports completion.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import base64
|
|
||||||
from dataclasses import asdict, dataclass
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path, PurePosixPath
|
|
||||||
import re
|
|
||||||
import subprocess
|
|
||||||
from typing import Callable, Mapping, Protocol
|
|
||||||
from urllib.error import HTTPError, URLError
|
|
||||||
from urllib.parse import quote, urlsplit
|
|
||||||
from urllib.request import Request, urlopen
|
|
||||||
|
|
||||||
from aios_authority_git_executor import AuthorityGitExecutor
|
|
||||||
from aios_continuation_lease import WriterLeaseRequestV2
|
|
||||||
from aios_gitops_job_controller import GitOpsJobController, JobState
|
|
||||||
from aios_gitops_reconciliation_core import ReconciliationAction, RemoteObservation, decide_reconciliation
|
|
||||||
from authorized_task_writer_provisioning import validate_writer_lease_request_v2
|
|
||||||
|
|
||||||
|
|
||||||
OID_RE = re.compile(r"^[0-9a-f]{40}$")
|
|
||||||
SECRET_PATTERNS = (
|
|
||||||
re.compile(rb"gh[pousr]_[A-Za-z0-9]{20,}"),
|
|
||||||
re.compile(rb"(?i)(?:token|password|secret|api[_-]?key)\s*[:=]\s*[\"'][^\"'\s]{12,}[\"']"),
|
|
||||||
re.compile(rb"-----BEGIN " + rb"(?:RSA |EC |OPENSSH )?PRIVATE KEY-----"),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class ContinuationExecutionError(RuntimeError):
|
|
||||||
"""Stable fail-closed continuation execution error."""
|
|
||||||
|
|
||||||
|
|
||||||
class DetachedWorkspace(Protocol):
|
|
||||||
def is_detached(self) -> bool: ...
|
|
||||||
def head_oid(self) -> str: ...
|
|
||||||
def commit_files(self, files: Mapping[str, bytes], message: str) -> str: ...
|
|
||||||
|
|
||||||
|
|
||||||
class AuthorityPort(Protocol):
|
|
||||||
def branch_exists(self, branch: str) -> bool: ...
|
|
||||||
def branch_head(self, branch: str) -> str: ...
|
|
||||||
def is_ancestor(self, older: str, newer: str) -> bool: ...
|
|
||||||
def compare_and_swap_branch(self, branch: str, expected: str, new: str) -> str: ...
|
|
||||||
|
|
||||||
|
|
||||||
class RemotePort(Protocol):
|
|
||||||
def repository_identity(self) -> str: ...
|
|
||||||
def push_exact(self, branch: str, expected: str, new: str) -> None: ...
|
|
||||||
def read_git_ref(self, branch: str) -> str | None: ...
|
|
||||||
def read_api_branch(self, branch: str) -> str | None: ...
|
|
||||||
def commit_visible(self, oid: str) -> bool: ...
|
|
||||||
|
|
||||||
|
|
||||||
class CredentialProjection(Protocol):
|
|
||||||
def project(self) -> None: ...
|
|
||||||
def revoke(self) -> None: ...
|
|
||||||
|
|
||||||
|
|
||||||
def _run(argv: list[str], *, cwd: Path | None = None, environment: Mapping[str, str] | None = None) -> str:
|
|
||||||
result = subprocess.run(
|
|
||||||
argv,
|
|
||||||
cwd=cwd,
|
|
||||||
env=None if environment is None else dict(environment),
|
|
||||||
text=True,
|
|
||||||
capture_output=True,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
if result.returncode != 0:
|
|
||||||
raise ContinuationExecutionError("SUBPROCESS_FAILED")
|
|
||||||
return result.stdout.strip()
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
|
||||||
class AuthorityGitAdapter:
|
|
||||||
"""Branch-name adapter over the existing narrow AuthorityGitExecutor."""
|
|
||||||
|
|
||||||
executor: AuthorityGitExecutor
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _ref(branch: str) -> str:
|
|
||||||
return f"refs/heads/{branch}"
|
|
||||||
|
|
||||||
def branch_exists(self, branch: str) -> bool:
|
|
||||||
return self.executor.read_ref(self._ref(branch)) is not None
|
|
||||||
|
|
||||||
def branch_head(self, branch: str) -> str:
|
|
||||||
value = self.executor.read_ref(self._ref(branch))
|
|
||||||
if value is None:
|
|
||||||
raise ContinuationExecutionError("EXISTING_BRANCH_MISSING")
|
|
||||||
return value
|
|
||||||
|
|
||||||
def is_ancestor(self, older: str, newer: str) -> bool:
|
|
||||||
return self.executor.ancestry_check(older, newer)
|
|
||||||
|
|
||||||
def compare_and_swap_branch(self, branch: str, expected: str, new: str) -> str:
|
|
||||||
return self.executor.advance_task_branch_CAS(self._ref(branch), expected, new)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
|
||||||
class DetachedGitWorkspace:
|
|
||||||
path: Path
|
|
||||||
|
|
||||||
def __post_init__(self) -> None:
|
|
||||||
if not self.path.is_absolute() or self.path.is_symlink() or not self.path.is_dir():
|
|
||||||
raise ContinuationExecutionError("WORKTREE_PATH_INVALID")
|
|
||||||
top = Path(_run(["git", "rev-parse", "--show-toplevel"], cwd=self.path)).resolve()
|
|
||||||
if top != self.path.resolve():
|
|
||||||
raise ContinuationExecutionError("WORKTREE_PATH_INVALID")
|
|
||||||
|
|
||||||
def is_detached(self) -> bool:
|
|
||||||
result = subprocess.run(
|
|
||||||
["git", "symbolic-ref", "-q", "HEAD"],
|
|
||||||
cwd=self.path,
|
|
||||||
capture_output=True,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
return result.returncode == 1
|
|
||||||
|
|
||||||
def head_oid(self) -> str:
|
|
||||||
return _run(["git", "rev-parse", "HEAD"], cwd=self.path)
|
|
||||||
|
|
||||||
def _dirty_paths(self) -> set[str]:
|
|
||||||
command = subprocess.run(
|
|
||||||
["git", "status", "--porcelain=v1", "--untracked-files=all"],
|
|
||||||
cwd=self.path,
|
|
||||||
text=True,
|
|
||||||
capture_output=True,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
if command.returncode != 0:
|
|
||||||
raise ContinuationExecutionError("SUBPROCESS_FAILED")
|
|
||||||
output = command.stdout
|
|
||||||
result: set[str] = set()
|
|
||||||
for line in output.splitlines():
|
|
||||||
path = line[3:]
|
|
||||||
if " -> " in path:
|
|
||||||
path = path.split(" -> ", 1)[1]
|
|
||||||
result.add(path)
|
|
||||||
return result
|
|
||||||
|
|
||||||
def materialize_files(self, files: Mapping[str, bytes]) -> None:
|
|
||||||
dirty = self._dirty_paths()
|
|
||||||
expected_paths = set(files)
|
|
||||||
if dirty and dirty != expected_paths:
|
|
||||||
raise ContinuationExecutionError("WORKTREE_DIRTY_SCOPE_MISMATCH")
|
|
||||||
for relative, content in files.items():
|
|
||||||
target = (self.path / relative).resolve()
|
|
||||||
try:
|
|
||||||
target.relative_to(self.path.resolve())
|
|
||||||
except ValueError as exc:
|
|
||||||
raise ContinuationExecutionError("COMMIT_PATH_REJECTED") from exc
|
|
||||||
if target.is_symlink():
|
|
||||||
raise ContinuationExecutionError("COMMIT_PATH_REJECTED")
|
|
||||||
target.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
target.write_bytes(content)
|
|
||||||
|
|
||||||
def commit_files(self, files: Mapping[str, bytes], message: str) -> str:
|
|
||||||
expected_paths = set(files)
|
|
||||||
if self._dirty_paths() != expected_paths:
|
|
||||||
raise ContinuationExecutionError("WORKTREE_DIRTY_SCOPE_MISMATCH")
|
|
||||||
for relative, content in files.items():
|
|
||||||
target = (self.path / relative).resolve()
|
|
||||||
try:
|
|
||||||
target.relative_to(self.path.resolve())
|
|
||||||
except ValueError as exc:
|
|
||||||
raise ContinuationExecutionError("COMMIT_PATH_REJECTED") from exc
|
|
||||||
if not target.is_file() or target.is_symlink() or target.read_bytes() != content:
|
|
||||||
raise ContinuationExecutionError("AUTHORING_PAYLOAD_MISMATCH")
|
|
||||||
_run(["git", "add", "--", *sorted(expected_paths)], cwd=self.path)
|
|
||||||
staged = set(_run(["git", "diff", "--cached", "--name-only"], cwd=self.path).splitlines())
|
|
||||||
if staged != expected_paths:
|
|
||||||
raise ContinuationExecutionError("STAGED_SCOPE_MISMATCH")
|
|
||||||
environment = os.environ.copy()
|
|
||||||
environment.update(
|
|
||||||
{
|
|
||||||
"GIT_AUTHOR_NAME": "AIOS Governed Continuation",
|
|
||||||
"GIT_AUTHOR_EMAIL": "aios-governed-continuation@localhost",
|
|
||||||
"GIT_COMMITTER_NAME": "AIOS Governed Continuation",
|
|
||||||
"GIT_COMMITTER_EMAIL": "aios-governed-continuation@localhost",
|
|
||||||
}
|
|
||||||
)
|
|
||||||
_run(["git", "commit", "--no-gpg-sign", "-m", message, "--", *sorted(expected_paths)], cwd=self.path, environment=environment)
|
|
||||||
if not self.is_detached():
|
|
||||||
raise ContinuationExecutionError("WORKTREE_NOT_DETACHED")
|
|
||||||
return self.head_oid()
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
|
||||||
class ServerSideCredentialProjection:
|
|
||||||
ssh_key_path: Path
|
|
||||||
api_token_path: Path
|
|
||||||
active: bool = False
|
|
||||||
|
|
||||||
def project(self) -> None:
|
|
||||||
for path in (self.ssh_key_path, self.api_token_path):
|
|
||||||
if path.is_symlink() or not path.is_file() or path.stat().st_size < 32:
|
|
||||||
raise ContinuationExecutionError("CREDENTIAL_PROJECTION_MISSING")
|
|
||||||
self.active = True
|
|
||||||
|
|
||||||
def revoke(self) -> None:
|
|
||||||
self.active = False
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
|
||||||
class GiteaContinuationRemote:
|
|
||||||
workspace: Path
|
|
||||||
credential: ServerSideCredentialProjection
|
|
||||||
remote_name: str = "gitea"
|
|
||||||
repository: str = "aios/aios"
|
|
||||||
api_base: str = "https://g.cnnetmkt.top/git/api/v1"
|
|
||||||
expected_host: str = "g.cnnetmkt.top"
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def validate_remote_url(value: str, expected_host: str = "g.cnnetmkt.top") -> None:
|
|
||||||
if value.startswith(("/", "file:")) or value.endswith("/data/aios-gitops/repo.git"):
|
|
||||||
raise ContinuationExecutionError("GITEA_REMOTE_IDENTITY_INVALID")
|
|
||||||
if "://" in value:
|
|
||||||
parsed = urlsplit(value)
|
|
||||||
host, path = parsed.hostname, parsed.path
|
|
||||||
else:
|
|
||||||
location = value.split("@", 1)[-1]
|
|
||||||
host, separator, path = location.partition(":")
|
|
||||||
if not separator:
|
|
||||||
raise ContinuationExecutionError("GITEA_REMOTE_IDENTITY_INVALID")
|
|
||||||
path = "/" + path
|
|
||||||
if host != expected_host or path.rstrip("/") not in {"/aios/aios.git", "/git/aios/aios.git"}:
|
|
||||||
raise ContinuationExecutionError("GITEA_REMOTE_IDENTITY_INVALID")
|
|
||||||
|
|
||||||
def repository_identity(self) -> str:
|
|
||||||
url = _run(["git", "remote", "get-url", self.remote_name], cwd=self.workspace)
|
|
||||||
self.validate_remote_url(url, self.expected_host)
|
|
||||||
if self.repository != "aios/aios":
|
|
||||||
raise ContinuationExecutionError("REMOTE_REPOSITORY_IDENTITY_MISMATCH")
|
|
||||||
return self.repository
|
|
||||||
|
|
||||||
def _require_projected(self) -> None:
|
|
||||||
if not self.credential.active:
|
|
||||||
raise ContinuationExecutionError("CREDENTIAL_PROJECTION_MISSING")
|
|
||||||
|
|
||||||
def read_git_ref(self, branch: str) -> str | None:
|
|
||||||
self._require_projected()
|
|
||||||
output = _run(["git", "ls-remote", self.remote_name, f"refs/heads/{branch}"], cwd=self.workspace)
|
|
||||||
if not output:
|
|
||||||
return None
|
|
||||||
oid, ref = output.split(maxsplit=1)
|
|
||||||
if ref != f"refs/heads/{branch}" or not OID_RE.fullmatch(oid):
|
|
||||||
raise ContinuationExecutionError("GITEA_GIT_READBACK_INVALID")
|
|
||||||
return oid
|
|
||||||
|
|
||||||
def push_exact(self, branch: str, expected: str, new: str) -> None:
|
|
||||||
self._require_projected()
|
|
||||||
self.repository_identity()
|
|
||||||
if self.read_git_ref(branch) != expected:
|
|
||||||
raise ContinuationExecutionError("GITEA_BRANCH_EXPECTATION_MISMATCH")
|
|
||||||
_run(["git", "push", self.remote_name, f"{new}:refs/heads/{branch}"], cwd=self.workspace)
|
|
||||||
|
|
||||||
def _api(self, path: str) -> object:
|
|
||||||
self._require_projected()
|
|
||||||
token = self.credential.api_token_path.read_text(encoding="utf-8").strip()
|
|
||||||
request = Request(
|
|
||||||
self.api_base.rstrip("/") + path,
|
|
||||||
headers={"Accept": "application/json", "Authorization": f"token {token}"},
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
with urlopen(request, timeout=20) as response:
|
|
||||||
return json.loads(response.read().decode("utf-8"))
|
|
||||||
except HTTPError as exc:
|
|
||||||
if exc.code == 404:
|
|
||||||
return None
|
|
||||||
raise ContinuationExecutionError(f"GITEA_API_HTTP_{exc.code}") from exc
|
|
||||||
except (URLError, TimeoutError, OSError, json.JSONDecodeError) as exc:
|
|
||||||
raise ContinuationExecutionError("GITEA_API_UNAVAILABLE") from exc
|
|
||||||
|
|
||||||
def read_api_branch(self, branch: str) -> str | None:
|
|
||||||
owner, name = self.repository.split("/", 1)
|
|
||||||
page = 1
|
|
||||||
while page <= 100:
|
|
||||||
value = self._api(f"/repos/{quote(owner)}/{quote(name)}/branches?page={page}&limit=50")
|
|
||||||
if value is None:
|
|
||||||
return None
|
|
||||||
if not isinstance(value, list):
|
|
||||||
raise ContinuationExecutionError("GITEA_API_BRANCH_COLLECTION_INVALID")
|
|
||||||
for item in value:
|
|
||||||
if not isinstance(item, dict) or item.get("name") != branch:
|
|
||||||
continue
|
|
||||||
oid = item.get("commit", {}).get("id")
|
|
||||||
if not isinstance(oid, str) or not OID_RE.fullmatch(oid):
|
|
||||||
raise ContinuationExecutionError("GITEA_API_BRANCH_INVALID")
|
|
||||||
return oid
|
|
||||||
if len(value) < 50:
|
|
||||||
return None
|
|
||||||
page += 1
|
|
||||||
raise ContinuationExecutionError("GITEA_API_BRANCH_PAGINATION_LIMIT")
|
|
||||||
|
|
||||||
def commit_visible(self, oid: str) -> bool:
|
|
||||||
owner, name = self.repository.split("/", 1)
|
|
||||||
value = self._api(f"/repos/{quote(owner)}/{quote(name)}/git/commits/{oid}")
|
|
||||||
if not isinstance(value, dict):
|
|
||||||
return False
|
|
||||||
observed = value.get("sha") or value.get("id")
|
|
||||||
return observed == oid
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class ContinuationPermissions:
|
|
||||||
create_target_branch: bool = False
|
|
||||||
force_push: bool = False
|
|
||||||
tag_write: bool = False
|
|
||||||
baseline_write: bool = False
|
|
||||||
production_write: bool = False
|
|
||||||
main_write: bool = False
|
|
||||||
|
|
||||||
def validate(self) -> None:
|
|
||||||
if any(asdict(self).values()):
|
|
||||||
raise ContinuationExecutionError("MUTATION_PERMISSION_DENIED")
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class ContinuationRuntimeBinding:
|
|
||||||
request: WriterLeaseRequestV2
|
|
||||||
workspace_path: Path
|
|
||||||
lease_not_before: datetime
|
|
||||||
lease_expires_at: datetime
|
|
||||||
original_receipt_valid: bool
|
|
||||||
original_grant_consumed: bool
|
|
||||||
permissions: ContinuationPermissions
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class ContinuationCommit:
|
|
||||||
idempotency_key: str
|
|
||||||
files: Mapping[str, bytes]
|
|
||||||
message: str
|
|
||||||
|
|
||||||
def payload(self) -> bytes:
|
|
||||||
if not self.idempotency_key or not self.message.strip() or not self.files:
|
|
||||||
raise ContinuationExecutionError("COMMIT_REQUEST_INVALID")
|
|
||||||
normalized: dict[str, str] = {}
|
|
||||||
for path, content in self.files.items():
|
|
||||||
pure = PurePosixPath(path)
|
|
||||||
if (
|
|
||||||
not isinstance(path, str)
|
|
||||||
or pure.is_absolute()
|
|
||||||
or ".." in pure.parts
|
|
||||||
or path.startswith((".git/", "baseline/", "production/"))
|
|
||||||
or not isinstance(content, bytes)
|
|
||||||
):
|
|
||||||
raise ContinuationExecutionError("COMMIT_PATH_REJECTED")
|
|
||||||
if any(pattern.search(content) for pattern in SECRET_PATTERNS):
|
|
||||||
raise ContinuationExecutionError("SECRET_LIKE_PAYLOAD_REJECTED")
|
|
||||||
normalized[path] = base64.b64encode(content).decode("ascii")
|
|
||||||
return json.dumps(
|
|
||||||
{"files_base64": normalized, "message": self.message},
|
|
||||||
sort_keys=True,
|
|
||||||
separators=(",", ":"),
|
|
||||||
).encode("utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
class ContinuationExecutor:
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
controller: GitOpsJobController,
|
|
||||||
authority: AuthorityPort,
|
|
||||||
workspace: DetachedWorkspace,
|
|
||||||
remote: RemotePort,
|
|
||||||
credential: CredentialProjection,
|
|
||||||
*,
|
|
||||||
now: Callable[[], datetime] = lambda: datetime.now(timezone.utc),
|
|
||||||
) -> None:
|
|
||||||
self.controller = controller
|
|
||||||
self.authority = authority
|
|
||||||
self.workspace = workspace
|
|
||||||
self.remote = remote
|
|
||||||
self.credential = credential
|
|
||||||
self.now = now
|
|
||||||
self._observed_generations: dict[str, int] = {}
|
|
||||||
|
|
||||||
def observe_generation(self, branch: str, generation: int) -> None:
|
|
||||||
self._observed_generations[branch] = max(generation, self._observed_generations.get(branch, 0))
|
|
||||||
|
|
||||||
def _max_generation(self, branch: str) -> int:
|
|
||||||
observed = self._observed_generations.get(branch, 0)
|
|
||||||
with self.controller.store._connect() as connection:
|
|
||||||
row = connection.execute(
|
|
||||||
"SELECT MAX(generation) AS generation FROM branch_leases WHERE branch=?",
|
|
||||||
(branch,),
|
|
||||||
).fetchone()
|
|
||||||
durable = int(row["generation"] or 0)
|
|
||||||
return max(observed, durable)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _request_mapping(request: WriterLeaseRequestV2) -> dict[str, object]:
|
|
||||||
return {
|
|
||||||
"schema": WriterLeaseRequestV2.schema,
|
|
||||||
"task_id": request.task_id,
|
|
||||||
"repository": request.repository,
|
|
||||||
"branch_mode": request.branch_mode,
|
|
||||||
"base_oid": request.base_oid,
|
|
||||||
"expected_branch_oid": request.expected_branch_oid,
|
|
||||||
"target_branch": request.target_branch,
|
|
||||||
"generation": request.generation,
|
|
||||||
"lease_seconds": request.lease_seconds,
|
|
||||||
}
|
|
||||||
|
|
||||||
def _validate_preflight(self, binding: ContinuationRuntimeBinding) -> None:
|
|
||||||
request = binding.request
|
|
||||||
normalized = validate_writer_lease_request_v2(self._request_mapping(request), self.authority)
|
|
||||||
if normalized != {**self._request_mapping(request), "create_target_branch": False, "worktree_mode": "detached_at_expected_head", "cas_ref_update_required": True}:
|
|
||||||
raise ContinuationExecutionError("CONTINUATION_NORMALIZATION_INVALID")
|
|
||||||
if not binding.original_receipt_valid or not binding.original_grant_consumed:
|
|
||||||
raise ContinuationExecutionError("ORIGINAL_GRANT_LINEAGE_INVALID")
|
|
||||||
binding.permissions.validate()
|
|
||||||
now = self.now()
|
|
||||||
if now < binding.lease_not_before:
|
|
||||||
raise ContinuationExecutionError("LEASE_NOT_ACTIVE")
|
|
||||||
if now >= binding.lease_expires_at:
|
|
||||||
raise ContinuationExecutionError("LEASE_EXPIRED")
|
|
||||||
if request.generation <= self._max_generation(request.target_branch):
|
|
||||||
raise ContinuationExecutionError("GENERATION_NOT_MONOTONIC")
|
|
||||||
if not self.workspace.is_detached():
|
|
||||||
raise ContinuationExecutionError("WORKTREE_NOT_DETACHED")
|
|
||||||
if self.workspace.head_oid() != request.expected_branch_oid:
|
|
||||||
raise ContinuationExecutionError("WORKTREE_HEAD_MISMATCH")
|
|
||||||
if self.remote.repository_identity() != request.repository:
|
|
||||||
raise ContinuationExecutionError("REMOTE_REPOSITORY_IDENTITY_MISMATCH")
|
|
||||||
|
|
||||||
def _job_request(self, binding: ContinuationRuntimeBinding, commit: ContinuationCommit, payload: bytes) -> dict[str, object]:
|
|
||||||
request = binding.request
|
|
||||||
return {
|
|
||||||
"idempotency_key": commit.idempotency_key,
|
|
||||||
"campaign_id": request.task_id,
|
|
||||||
"task_id": request.task_id,
|
|
||||||
"repository": request.repository,
|
|
||||||
"operation": "CONTINUATION_COMMIT_FILES",
|
|
||||||
"branch": request.target_branch,
|
|
||||||
"expected_branch_oid": request.expected_branch_oid,
|
|
||||||
"base_oid": request.base_oid,
|
|
||||||
"branch_mode": request.branch_mode,
|
|
||||||
"payload_digest": hashlib.sha256(payload).hexdigest(),
|
|
||||||
"lineage_anchor_oid": request.base_oid,
|
|
||||||
"generation": request.generation,
|
|
||||||
}
|
|
||||||
|
|
||||||
def execute(self, binding: ContinuationRuntimeBinding, commit: ContinuationCommit) -> dict[str, object]:
|
|
||||||
payload = commit.payload()
|
|
||||||
submitted = self.controller.submit(self._job_request(binding, commit, payload), payload=payload)
|
|
||||||
current = self.controller.store.read(submitted.job_id)
|
|
||||||
if JobState(current["state"]) == JobState.COMPLETED:
|
|
||||||
return self.controller.store._receipt(current, duplicate=True).as_dict()
|
|
||||||
self._validate_preflight(binding)
|
|
||||||
request = binding.request
|
|
||||||
projected = False
|
|
||||||
try:
|
|
||||||
self.credential.project()
|
|
||||||
projected = True
|
|
||||||
row = self.controller.store.read(submitted.job_id)
|
|
||||||
if JobState(row["state"]) == JobState.REQUESTED:
|
|
||||||
self.controller.advance(
|
|
||||||
submitted.job_id,
|
|
||||||
JobState.AUTHORIZED,
|
|
||||||
side_effect_state="AUTHORIZED",
|
|
||||||
receipt={"task_id": request.task_id, "original_grant_consumed": True},
|
|
||||||
)
|
|
||||||
row = self.controller.store.read(submitted.job_id)
|
|
||||||
if JobState(row["state"]) == JobState.AUTHORIZED:
|
|
||||||
self.controller.lease_job(
|
|
||||||
submitted.job_id,
|
|
||||||
lease_id=f"lease-{submitted.job_id}",
|
|
||||||
lineage_anchor_oid=request.base_oid,
|
|
||||||
current_head_oid=request.expected_branch_oid,
|
|
||||||
generation=request.generation,
|
|
||||||
lease_expires_at=binding.lease_expires_at.astimezone(timezone.utc).isoformat(),
|
|
||||||
expected_revision=int(row["revision"]),
|
|
||||||
)
|
|
||||||
self.controller.advance(submitted.job_id, JobState.WORKTREE_READY, side_effect_state="WORKTREE_READY")
|
|
||||||
self.controller.advance(submitted.job_id, JobState.EXECUTING, side_effect_state="IN_PROGRESS")
|
|
||||||
new_oid = self.workspace.commit_files(commit.files, commit.message)
|
|
||||||
if not OID_RE.fullmatch(new_oid) or not self.authority.is_ancestor(request.expected_branch_oid, new_oid):
|
|
||||||
raise ContinuationExecutionError("NON_FAST_FORWARD_COMMIT")
|
|
||||||
authority_oid = self.authority.compare_and_swap_branch(request.target_branch, request.expected_branch_oid, new_oid)
|
|
||||||
if authority_oid != new_oid:
|
|
||||||
raise ContinuationExecutionError("AUTHORITY_CAS_READBACK_MISMATCH")
|
|
||||||
self.controller.advance(
|
|
||||||
submitted.job_id,
|
|
||||||
JobState.LOCAL_VERIFIED,
|
|
||||||
side_effect_state="LOCAL_VERIFIED",
|
|
||||||
receipt={"authority_oid": authority_oid, "force_push": False},
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
self.controller.advance(submitted.job_id, JobState.REMOTE_SYNCING, side_effect_state="IN_PROGRESS")
|
|
||||||
self.remote.push_exact(request.target_branch, request.expected_branch_oid, new_oid)
|
|
||||||
except Exception:
|
|
||||||
return self._remote_pending(submitted.job_id, authority_oid, new_oid)
|
|
||||||
return self._finish_remote_verification(submitted.job_id, authority_oid, new_oid)
|
|
||||||
finally:
|
|
||||||
if projected:
|
|
||||||
self.credential.revoke()
|
|
||||||
|
|
||||||
def _observe_remote(self, branch: str, oid: str) -> tuple[str | None, str | None, bool]:
|
|
||||||
return self.remote.read_git_ref(branch), self.remote.read_api_branch(branch), self.remote.commit_visible(oid)
|
|
||||||
|
|
||||||
def _remote_pending(self, job_id: str, authority_oid: str, new_oid: str) -> dict[str, object]:
|
|
||||||
receipt = {
|
|
||||||
"authority_oid": authority_oid,
|
|
||||||
"desired_remote_oid": new_oid,
|
|
||||||
"remote_verified": False,
|
|
||||||
"force_push": False,
|
|
||||||
}
|
|
||||||
result = self.controller.advance(
|
|
||||||
job_id,
|
|
||||||
JobState.RECONCILIATION_REQUIRED,
|
|
||||||
side_effect_state="REMOTE_PUSH_PENDING",
|
|
||||||
receipt=receipt,
|
|
||||||
)
|
|
||||||
return result.as_dict()
|
|
||||||
|
|
||||||
def _finish_remote_verification(self, job_id: str, authority_oid: str, new_oid: str) -> dict[str, object]:
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
branch = row["branch"]
|
|
||||||
request = json.loads(row["request_json"])["binding"]
|
|
||||||
git_oid, api_oid, visible = self._observe_remote(branch, new_oid)
|
|
||||||
decision = decide_reconciliation(RemoteObservation("aios/aios", branch, new_oid, authority_oid, git_oid, api_oid, visible, datetime.now(timezone.utc).isoformat(), request.get("expected_branch_oid")))
|
|
||||||
if decision.action is not ReconciliationAction.MARK_REMOTE_VERIFIED:
|
|
||||||
return self._remote_pending(job_id, authority_oid, new_oid)
|
|
||||||
receipt = {
|
|
||||||
"authority_oid": authority_oid,
|
|
||||||
"git_remote_oid": git_oid,
|
|
||||||
"api_branch_oid": api_oid,
|
|
||||||
"commit_visible": True,
|
|
||||||
"remote_verified": True,
|
|
||||||
"reconciliation_decision": decision.action.value,
|
|
||||||
"force_push": False,
|
|
||||||
}
|
|
||||||
self.controller.advance(job_id, JobState.REMOTE_VERIFIED, side_effect_state="REMOTE_VERIFIED", receipt=receipt)
|
|
||||||
return self.controller.advance(job_id, JobState.COMPLETED, side_effect_state="REMOTE_VERIFIED", receipt=receipt).as_dict()
|
|
||||||
|
|
||||||
def reconcile(self, job_id: str) -> dict[str, object]:
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
state = JobState(row["state"])
|
|
||||||
if state == JobState.COMPLETED:
|
|
||||||
return self.controller.store._receipt(row, duplicate=True).as_dict()
|
|
||||||
if state != JobState.RECONCILIATION_REQUIRED:
|
|
||||||
raise ContinuationExecutionError("JOB_NOT_RECONCILABLE")
|
|
||||||
request = json.loads(row["request_json"])["binding"]
|
|
||||||
receipt = json.loads(row["receipt_json"])
|
|
||||||
new_oid = receipt["desired_remote_oid"]
|
|
||||||
authority_oid = self.authority.branch_head(row["branch"])
|
|
||||||
try:
|
|
||||||
generation = int(request.get("generation") or 0)
|
|
||||||
except (TypeError, ValueError) as exc:
|
|
||||||
raise ContinuationExecutionError("DURABLE_BINDING_GENERATION_INVALID") from exc
|
|
||||||
if generation > 0:
|
|
||||||
with self.controller.store._connect() as connection:
|
|
||||||
earlier = connection.execute(
|
|
||||||
"SELECT 1 FROM jobs WHERE branch=? AND state=? "
|
|
||||||
"AND CAST(json_extract(request_json, '$.binding.generation') AS INTEGER) < ? LIMIT 1",
|
|
||||||
(row["branch"], JobState.RECONCILIATION_REQUIRED.value, generation),
|
|
||||||
).fetchone()
|
|
||||||
if earlier is not None:
|
|
||||||
raise ContinuationExecutionError("EARLIER_RECONCILIATION_REQUIRED")
|
|
||||||
self.credential.project()
|
|
||||||
try:
|
|
||||||
git_oid, api_oid, visible = self._observe_remote(row["branch"], new_oid)
|
|
||||||
if authority_oid == git_oid == api_oid == new_oid and visible:
|
|
||||||
self.controller.advance(job_id, JobState.RECONCILING, side_effect_state="OBSERVATION_FIRST")
|
|
||||||
return self._finish_remote_verification(job_id, new_oid, new_oid)
|
|
||||||
expected_oid = request["expected_branch_oid"]
|
|
||||||
if (
|
|
||||||
self.authority.is_ancestor(new_oid, authority_oid)
|
|
||||||
and git_oid is not None
|
|
||||||
and git_oid == api_oid
|
|
||||||
and self.authority.is_ancestor(git_oid, new_oid)
|
|
||||||
):
|
|
||||||
self.controller.advance(job_id, JobState.RECONCILING, side_effect_state="OBSERVATION_FIRST_CHAIN")
|
|
||||||
self.remote.push_exact(row["branch"], git_oid, new_oid)
|
|
||||||
return self._finish_remote_verification(job_id, new_oid, new_oid)
|
|
||||||
if authority_oid == new_oid and git_oid == api_oid == expected_oid:
|
|
||||||
self.controller.advance(job_id, JobState.RECONCILING, side_effect_state="OBSERVATION_FIRST_RETRY")
|
|
||||||
self.remote.push_exact(row["branch"], expected_oid, new_oid)
|
|
||||||
return self._finish_remote_verification(job_id, new_oid, new_oid)
|
|
||||||
return self.controller.store._receipt(row, duplicate=True).as_dict()
|
|
||||||
finally:
|
|
||||||
self.credential.revoke()
|
|
||||||
@@ -1,154 +0,0 @@
|
|||||||
"""Campaign-aware continuation leases for an existing Git branch.
|
|
||||||
|
|
||||||
This module is deliberately transport-neutral. It validates branch lineage
|
|
||||||
and hands ref mutation to an injected compare-and-swap broker; it does not run
|
|
||||||
Git, inspect credentials, or create a branch itself.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass, replace
|
|
||||||
from datetime import datetime, timedelta, timezone
|
|
||||||
import re
|
|
||||||
from typing import Protocol
|
|
||||||
|
|
||||||
|
|
||||||
OID_RE = re.compile(r"^[0-9a-f]{40}$")
|
|
||||||
TARGET_BRANCH = "agent/autonomous-release-campaign-v1-20260807"
|
|
||||||
LINEAGE_ANCHOR = "69b45e81dcd2c3707c367003b08c55fd361faae8"
|
|
||||||
CAMPAIGN_HEAD = "bba675fab3b7e0c5fba7066720252b3c91b9f20d"
|
|
||||||
|
|
||||||
|
|
||||||
class ContinuationError(RuntimeError):
|
|
||||||
"""Stable fail-closed error for lease or branch binding violations."""
|
|
||||||
|
|
||||||
|
|
||||||
class BranchProbe(Protocol):
|
|
||||||
def branch_exists(self, branch: str) -> bool: ...
|
|
||||||
def branch_head(self, branch: str) -> str: ...
|
|
||||||
def is_ancestor(self, older: str, newer: str) -> bool: ...
|
|
||||||
|
|
||||||
|
|
||||||
class RefCAS(Protocol):
|
|
||||||
def compare_and_swap_branch(self, branch: str, expected: str, new: str) -> str: ...
|
|
||||||
|
|
||||||
|
|
||||||
def _oid(value: str, field: str) -> str:
|
|
||||||
if not isinstance(value, str) or not OID_RE.fullmatch(value):
|
|
||||||
raise ContinuationError(f"{field}_invalid")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class WriterLeaseRequestV2:
|
|
||||||
task_id: str
|
|
||||||
repository: str
|
|
||||||
branch_mode: str
|
|
||||||
base_oid: str
|
|
||||||
expected_branch_oid: str
|
|
||||||
target_branch: str
|
|
||||||
generation: int
|
|
||||||
lease_seconds: int = 900
|
|
||||||
|
|
||||||
schema = "aios.task-writer-lease.v2"
|
|
||||||
accepted_schemas = frozenset({"aios.task-writer-lease.v2", "aios.task-writer-lease-request.v2"})
|
|
||||||
|
|
||||||
def validate(self, probe: BranchProbe) -> None:
|
|
||||||
if not self.task_id or self.repository != "aios/aios":
|
|
||||||
raise ContinuationError("request_identity_invalid")
|
|
||||||
if self.branch_mode not in {"CREATE_NEW", "CONTINUE_EXISTING"}:
|
|
||||||
raise ContinuationError("branch_mode_invalid")
|
|
||||||
_oid(self.base_oid, "base_oid")
|
|
||||||
_oid(self.expected_branch_oid, "expected_branch_oid")
|
|
||||||
if self.target_branch in {"main", "master"} or self.target_branch.startswith(("refs/", "tags/")):
|
|
||||||
raise ContinuationError("protected_branch_rejected")
|
|
||||||
if not isinstance(self.generation, int) or isinstance(self.generation, bool) or self.generation < 1:
|
|
||||||
raise ContinuationError("generation_invalid")
|
|
||||||
if not isinstance(self.lease_seconds, int) or not 1 <= self.lease_seconds <= 86_400:
|
|
||||||
raise ContinuationError("lease_duration_invalid")
|
|
||||||
if self.branch_mode == "CONTINUE_EXISTING":
|
|
||||||
if not probe.branch_exists(self.target_branch):
|
|
||||||
raise ContinuationError("EXISTING_BRANCH_MISSING")
|
|
||||||
actual = _oid(probe.branch_head(self.target_branch), "actual_branch_oid")
|
|
||||||
if actual != self.expected_branch_oid:
|
|
||||||
raise ContinuationError("EXISTING_BRANCH_OID_CONFLICT")
|
|
||||||
if not probe.is_ancestor(self.base_oid, actual):
|
|
||||||
raise ContinuationError("BASE_NOT_ANCESTOR_OF_BRANCH")
|
|
||||||
elif hasattr(probe, "commit_exists") and not probe.commit_exists(self.base_oid):
|
|
||||||
raise ContinuationError("BASE_COMMIT_NOT_FOUND")
|
|
||||||
elif self.branch_mode == "CREATE_NEW" and probe.branch_exists(self.target_branch):
|
|
||||||
raise ContinuationError("TARGET_BRANCH_ALREADY_EXISTS")
|
|
||||||
elif self.branch_mode == "CREATE_NEW" and self.base_oid != self.expected_branch_oid:
|
|
||||||
raise ContinuationError("CREATE_NEW_BASE_EXPECTATION_MISMATCH")
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class CampaignWriterLease:
|
|
||||||
task_id: str
|
|
||||||
repository: str
|
|
||||||
branch: str
|
|
||||||
lineage_anchor_oid: str
|
|
||||||
current_head_oid: str
|
|
||||||
generation: int
|
|
||||||
lease_expires_at: datetime
|
|
||||||
append_only: bool = True
|
|
||||||
renewable: bool = True
|
|
||||||
multi_commit: bool = True
|
|
||||||
recoverable: bool = True
|
|
||||||
state: str = "ACTIVE"
|
|
||||||
branch_bound: bool = True
|
|
||||||
|
|
||||||
def __post_init__(self) -> None:
|
|
||||||
_oid(self.lineage_anchor_oid, "lineage_anchor_oid")
|
|
||||||
_oid(self.current_head_oid, "current_head_oid")
|
|
||||||
if not all((self.append_only, self.renewable, self.multi_commit, self.recoverable, self.branch_bound)):
|
|
||||||
raise ContinuationError("lease_capability_downgrade")
|
|
||||||
|
|
||||||
|
|
||||||
class ContinuationLeaseManager:
|
|
||||||
"""State machine for a multi-commit lease with CAS-only ref updates."""
|
|
||||||
|
|
||||||
def __init__(self, probe: BranchProbe, refs: RefCAS, *, now=datetime.now) -> None:
|
|
||||||
self.probe = probe
|
|
||||||
self.refs = refs
|
|
||||||
self.now = now
|
|
||||||
|
|
||||||
def open(self, request: WriterLeaseRequestV2) -> CampaignWriterLease:
|
|
||||||
request.validate(self.probe)
|
|
||||||
if request.branch_mode != "CONTINUE_EXISTING":
|
|
||||||
raise ContinuationError("CREATE_NEW_REQUIRES_SEPARATE_PROVISIONER")
|
|
||||||
return CampaignWriterLease(
|
|
||||||
task_id=request.task_id,
|
|
||||||
repository=request.repository,
|
|
||||||
branch=request.target_branch,
|
|
||||||
lineage_anchor_oid=request.base_oid,
|
|
||||||
current_head_oid=request.expected_branch_oid,
|
|
||||||
generation=request.generation,
|
|
||||||
lease_expires_at=self.now(timezone.utc) + timedelta(seconds=request.lease_seconds),
|
|
||||||
)
|
|
||||||
|
|
||||||
def renew(self, lease: CampaignWriterLease, seconds: int) -> CampaignWriterLease:
|
|
||||||
if lease.state != "ACTIVE" or not lease.renewable or seconds < 1:
|
|
||||||
raise ContinuationError("lease_not_renewable")
|
|
||||||
return replace(lease, lease_expires_at=self.now(timezone.utc) + timedelta(seconds=seconds))
|
|
||||||
|
|
||||||
def append_commit(self, lease: CampaignWriterLease, new_head_oid: str) -> CampaignWriterLease:
|
|
||||||
if lease.state != "ACTIVE":
|
|
||||||
raise ContinuationError("lease_not_active")
|
|
||||||
_oid(new_head_oid, "new_head_oid")
|
|
||||||
if not self.probe.is_ancestor(lease.current_head_oid, new_head_oid):
|
|
||||||
raise ContinuationError("NON_FAST_FORWARD_COMMIT")
|
|
||||||
try:
|
|
||||||
actual = self.refs.compare_and_swap_branch(
|
|
||||||
lease.branch, lease.current_head_oid, new_head_oid
|
|
||||||
)
|
|
||||||
except Exception as exc:
|
|
||||||
raise ContinuationError("BRANCH_LEASE_STALE") from exc
|
|
||||||
if actual != new_head_oid:
|
|
||||||
raise ContinuationError("BRANCH_LEASE_STALE")
|
|
||||||
return replace(lease, current_head_oid=new_head_oid)
|
|
||||||
|
|
||||||
def supersede(self, lease: CampaignWriterLease) -> CampaignWriterLease:
|
|
||||||
if lease.state not in {"ACTIVE", "RECOVERABLE_FAILURE", "RECONCILIATION_REQUIRED"}:
|
|
||||||
raise ContinuationError("lease_not_supersedable")
|
|
||||||
return replace(lease, generation=lease.generation + 1, state="SUPERSEDED")
|
|
||||||
@@ -1,25 +0,0 @@
|
|||||||
"""Separate read, write, and release plane readiness evaluation."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from typing import Mapping
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class RuntimeStatus:
|
|
||||||
read_plane: Mapping[str, bool]
|
|
||||||
write_plane: Mapping[str, bool]
|
|
||||||
release_plane: Mapping[str, bool]
|
|
||||||
|
|
||||||
@property
|
|
||||||
def release_control_plane_ready(self) -> bool:
|
|
||||||
return all(self.read_plane.values()) and all(self.write_plane.values()) and all(self.release_plane.values())
|
|
||||||
|
|
||||||
def as_dict(self) -> dict[str, object]:
|
|
||||||
return {
|
|
||||||
"READ_PLANE": dict(self.read_plane),
|
|
||||||
"WRITE_PLANE": dict(self.write_plane),
|
|
||||||
"RELEASE_PLANE": dict(self.release_plane),
|
|
||||||
"release_control_plane_ready": self.release_control_plane_ready,
|
|
||||||
}
|
|
||||||
@@ -1,58 +0,0 @@
|
|||||||
"""Workspace-independent authority/network/Gitea ref sentinel."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from pathlib import Path
|
|
||||||
import subprocess
|
|
||||||
from typing import Callable, Mapping
|
|
||||||
|
|
||||||
|
|
||||||
class SentinelError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class SentinelInputs:
|
|
||||||
authority_git_dir: Path
|
|
||||||
gitea_remote: str
|
|
||||||
gitea_api: str
|
|
||||||
ref: str
|
|
||||||
expected_oid: str
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class SentinelResult:
|
|
||||||
authority_oid: str | None
|
|
||||||
network_oid: str | None
|
|
||||||
gitea_oid: str | None
|
|
||||||
equal: bool
|
|
||||||
error: str | None = None
|
|
||||||
|
|
||||||
|
|
||||||
def authority_oid(inputs: SentinelInputs) -> str:
|
|
||||||
result = subprocess.run(
|
|
||||||
["git", "--git-dir", str(inputs.authority_git_dir), "rev-parse", inputs.ref],
|
|
||||||
text=True,
|
|
||||||
capture_output=True,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
if result.returncode != 0:
|
|
||||||
raise SentinelError("AUTHORITY_REF_UNAVAILABLE")
|
|
||||||
return result.stdout.strip()
|
|
||||||
|
|
||||||
|
|
||||||
def evaluate(
|
|
||||||
inputs: SentinelInputs,
|
|
||||||
*,
|
|
||||||
network_reader: Callable[[str, str], str | None],
|
|
||||||
gitea_reader: Callable[[str, str], str | None],
|
|
||||||
) -> SentinelResult:
|
|
||||||
try:
|
|
||||||
local = authority_oid(inputs)
|
|
||||||
network = network_reader(inputs.gitea_remote, inputs.ref)
|
|
||||||
gitea = gitea_reader(inputs.gitea_api, inputs.ref)
|
|
||||||
equal = local == network == gitea == inputs.expected_oid
|
|
||||||
return SentinelResult(local, network, gitea, equal, None if equal else "OID_MISMATCH")
|
|
||||||
except SentinelError as exc:
|
|
||||||
return SentinelResult(None, None, None, False, str(exc))
|
|
||||||
@@ -1,26 +0,0 @@
|
|||||||
"""Non-sensitive control-plane snapshot assembly."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from typing import Any, Mapping
|
|
||||||
|
|
||||||
|
|
||||||
def _copy(value: Mapping[str, Any] | None, keys: tuple[str, ...]) -> dict[str, Any]:
|
|
||||||
value = value or {}
|
|
||||||
return {key: value.get(key) for key in keys}
|
|
||||||
|
|
||||||
|
|
||||||
def build_control_plane_snapshot(*, target_version: str, runtime: Mapping[str, Any] | None = None, repository: Mapping[str, Any] | None = None, production: Mapping[str, Any] | None = None, task: Mapping[str, Any] | None = None, capabilities: Mapping[str, Any] | None = None, credential_broker: Mapping[str, Any] | None = None) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"version": {"target_release": target_version},
|
|
||||||
"runtime": _copy(runtime, ("bridge_schema_version", "control_plane_release_version", "runtime_release_commit", "runtime_path", "artifact_sha256", "drift_status")),
|
|
||||||
"repository": _copy(repository, ("identity", "authority_git_dir", "authority_main_oid")),
|
|
||||||
"production": _copy(production, ("baseline_version", "target_oid", "deployment_state")),
|
|
||||||
"task": _copy(task, ("task_id", "branch", "authority_branch_oid", "gitea_git_oid", "gitea_api_oid", "commit_visible", "generation", "reconciliation_state")),
|
|
||||||
"capabilities": _copy(capabilities, ("legacy_writes_enabled", "governed_writes_enabled", "force_push", "main_write", "tag_write", "baseline_write", "production_write")),
|
|
||||||
"credential_broker": _copy(credential_broker, ("status", "provider")),
|
|
||||||
"secret_material_exposed": False,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def getControlPlaneSnapshot(**kwargs: Any) -> dict[str, Any]:
|
|
||||||
return build_control_plane_snapshot(**kwargs)
|
|
||||||
@@ -1,78 +0,0 @@
|
|||||||
"""Metadata-only, server-side credential lease broker."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from datetime import datetime, timedelta, timezone
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import secrets
|
|
||||||
from typing import Any, Callable, Mapping
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class CredentialLease:
|
|
||||||
lease_id: str
|
|
||||||
task_id: str
|
|
||||||
repository: str
|
|
||||||
target_branch: str
|
|
||||||
generation: int
|
|
||||||
issued_at: str
|
|
||||||
expires_at: str
|
|
||||||
scopes: tuple[str, ...]
|
|
||||||
provider: str
|
|
||||||
state: str = "ACTIVE"
|
|
||||||
|
|
||||||
def as_dict(self) -> dict[str, Any]:
|
|
||||||
return {"lease_id": self.lease_id, "task_id": self.task_id, "repository": self.repository, "target_branch": self.target_branch, "generation": self.generation, "issued_at": self.issued_at, "expires_at": self.expires_at, "scopes": list(self.scopes), "provider": self.provider, "state": self.state}
|
|
||||||
|
|
||||||
|
|
||||||
class FileCredentialProvider:
|
|
||||||
def __init__(self, token_path: str | os.PathLike[str], *, actor: str = "aios-autonomy-agent") -> None:
|
|
||||||
self.token_path = Path(token_path)
|
|
||||||
self.actor = actor
|
|
||||||
|
|
||||||
def environment(self, lease: CredentialLease) -> dict[str, str]:
|
|
||||||
if not self.token_path.is_file():
|
|
||||||
raise RuntimeError("CREDENTIAL_PROVIDER_UNAVAILABLE")
|
|
||||||
return {"CREDENTIAL_FILE": str(self.token_path), "GITEA_MACHINE_ACTOR": self.actor}
|
|
||||||
|
|
||||||
|
|
||||||
class CredentialBroker:
|
|
||||||
def __init__(self, provider: Any, *, now: Callable[[], datetime] | None = None) -> None:
|
|
||||||
self.provider = provider
|
|
||||||
self.now = now or (lambda: datetime.now(timezone.utc))
|
|
||||||
self._leases: dict[str, CredentialLease] = {}
|
|
||||||
|
|
||||||
def issue(self, *, task_id: str, repository: str, target_branch: str, generation: int, scopes: tuple[str, ...] = ("git:push", "gitea:read"), lease_seconds: int = 900) -> dict[str, Any]:
|
|
||||||
if repository != "aios/aios" or not target_branch.startswith("agent/") or generation < 1 or not 1 <= lease_seconds <= 86400:
|
|
||||||
raise ValueError("CREDENTIAL_LEASE_BINDING_INVALID")
|
|
||||||
now = self.now().astimezone(timezone.utc)
|
|
||||||
lease = CredentialLease("lease-" + secrets.token_hex(16), task_id, repository, target_branch, generation, now.isoformat(), (now + timedelta(seconds=lease_seconds)).isoformat(), tuple(scopes), type(self.provider).__name__)
|
|
||||||
self._leases[lease.lease_id] = lease
|
|
||||||
return lease.as_dict()
|
|
||||||
|
|
||||||
def _get(self, lease_id: str) -> CredentialLease:
|
|
||||||
lease = self._leases.get(lease_id)
|
|
||||||
if lease is None:
|
|
||||||
raise KeyError("CREDENTIAL_LEASE_NOT_FOUND")
|
|
||||||
if lease.state == "ACTIVE" and self.now().astimezone(timezone.utc) >= datetime.fromisoformat(lease.expires_at):
|
|
||||||
lease = CredentialLease(**{**lease.as_dict(), "scopes": tuple(lease.scopes), "state": "EXPIRED"})
|
|
||||||
self._leases[lease_id] = lease
|
|
||||||
return lease
|
|
||||||
|
|
||||||
def environment(self, lease_id: str) -> Mapping[str, str]:
|
|
||||||
lease = self._get(lease_id)
|
|
||||||
if lease.state != "ACTIVE":
|
|
||||||
raise RuntimeError("CREDENTIAL_LEASE_NOT_ACTIVE")
|
|
||||||
return self.provider.environment(lease)
|
|
||||||
|
|
||||||
def status(self, lease_id: str) -> dict[str, Any]:
|
|
||||||
return self._get(lease_id).as_dict()
|
|
||||||
|
|
||||||
def revoke(self, lease_id: str) -> dict[str, Any]:
|
|
||||||
lease = self._get(lease_id)
|
|
||||||
if lease.state == "REVOKED":
|
|
||||||
return lease.as_dict()
|
|
||||||
updated = CredentialLease(**{**lease.as_dict(), "scopes": tuple(lease.scopes), "state": "REVOKED"})
|
|
||||||
self._leases[lease_id] = updated
|
|
||||||
return updated.as_dict()
|
|
||||||
@@ -1,125 +0,0 @@
|
|||||||
"""Non-Owner Gitea machine capability and branch reconciliation boundary."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from typing import Any, Mapping, Protocol
|
|
||||||
|
|
||||||
|
|
||||||
class MachineBrokerError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
class GiteaTransport(Protocol):
|
|
||||||
def compare_and_swap_branch(self, branch: str, expected_oid: str, new_oid: str) -> str: ...
|
|
||||||
def create_or_update_pr(self, branch: str, target: str, title: str, body: str) -> Mapping[str, Any]: ...
|
|
||||||
def publish_status(self, branch: str, state: str, description: str) -> Mapping[str, Any]: ...
|
|
||||||
def protected_merge(self, pr_number: int) -> Mapping[str, Any]: ...
|
|
||||||
def read_branch(self, branch: str) -> str | None: ...
|
|
||||||
def whoami(self) -> Mapping[str, Any]: ...
|
|
||||||
def repository(self, owner: str, name: str) -> Mapping[str, Any]: ...
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class MachineCredentialProjection:
|
|
||||||
"""Opaque systemd credential reference; the credential value is never accepted."""
|
|
||||||
|
|
||||||
reference: str
|
|
||||||
account: str
|
|
||||||
owner_account: str = "aiman"
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_mapping(cls, value: Mapping[str, Any]) -> "MachineCredentialProjection":
|
|
||||||
if set(value) != {"scheme", "name", "account"}:
|
|
||||||
raise MachineBrokerError("CREDENTIAL_REFERENCE_INVALID")
|
|
||||||
if value["scheme"] != "systemd-credential" or value["name"] != "gitea-machine-token":
|
|
||||||
raise MachineBrokerError("CREDENTIAL_REFERENCE_INVALID")
|
|
||||||
return cls(f"{value['scheme']}:{value['name']}", str(value["account"]))
|
|
||||||
|
|
||||||
def __post_init__(self) -> None:
|
|
||||||
if self.reference != "systemd-credential:gitea-machine-token":
|
|
||||||
raise MachineBrokerError("CREDENTIAL_REFERENCE_INVALID")
|
|
||||||
if not self.account or self.account == self.owner_account:
|
|
||||||
raise MachineBrokerError("OWNER_MACHINE_IDENTITY_REJECTED")
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class CampaignCapabilityReceipt:
|
|
||||||
campaign_id: str
|
|
||||||
task_id: str
|
|
||||||
repository: str
|
|
||||||
branch: str
|
|
||||||
expected_oid: str
|
|
||||||
capabilities: frozenset[str]
|
|
||||||
denied: frozenset[str]
|
|
||||||
|
|
||||||
def require(self, capability: str) -> None:
|
|
||||||
if capability in self.denied or capability not in self.capabilities:
|
|
||||||
raise MachineBrokerError("CAPABILITY_DENIED")
|
|
||||||
if self.branch in {"main", "master"}:
|
|
||||||
raise MachineBrokerError("PROTECTED_BRANCH_DENIED")
|
|
||||||
|
|
||||||
|
|
||||||
class GiteaMachineBroker:
|
|
||||||
def __init__(self, transport: GiteaTransport, credential: MachineCredentialProjection, receipt: CampaignCapabilityReceipt) -> None:
|
|
||||||
self.transport = transport
|
|
||||||
self.credential = credential
|
|
||||||
self.receipt = receipt
|
|
||||||
|
|
||||||
def verify_identity(self) -> Mapping[str, Any]:
|
|
||||||
identity = dict(self.transport.whoami())
|
|
||||||
login = identity.get("login") or identity.get("username")
|
|
||||||
if not isinstance(login, str) or not login or login == self.credential.owner_account or login != self.credential.account:
|
|
||||||
raise MachineBrokerError("GITEA_MACHINE_IDENTITY_INVALID")
|
|
||||||
owner, name = self.receipt.repository.split("/", 1)
|
|
||||||
repo = dict(self.transport.repository(owner, name))
|
|
||||||
if repo.get("full_name") != self.receipt.repository:
|
|
||||||
raise MachineBrokerError("GITEA_REPOSITORY_MISMATCH")
|
|
||||||
return {"login": login, "repository": self.receipt.repository, "owner_account": self.credential.owner_account}
|
|
||||||
|
|
||||||
def push_branch(self, new_oid: str) -> str:
|
|
||||||
self.receipt.require("exact_task_branch_push")
|
|
||||||
return self.transport.compare_and_swap_branch(self.receipt.branch, self.receipt.expected_oid, new_oid)
|
|
||||||
|
|
||||||
def create_pr(self, title: str, body: str) -> Mapping[str, Any]:
|
|
||||||
self.receipt.require("pr_create_update")
|
|
||||||
return self.transport.create_or_update_pr(self.receipt.branch, "main", title, body)
|
|
||||||
|
|
||||||
def status(self, state: str, description: str, context: str = "aios/durable-release") -> Mapping[str, Any]:
|
|
||||||
self.receipt.require("status_publish")
|
|
||||||
return self.transport.publish_status(self.receipt.branch, state, description, context)
|
|
||||||
|
|
||||||
def merge(self, pr_number: int) -> Mapping[str, Any]:
|
|
||||||
self.receipt.require("protected_merge")
|
|
||||||
return self.transport.protected_merge(pr_number)
|
|
||||||
|
|
||||||
def readback(self) -> str | None:
|
|
||||||
self.receipt.require("remote_readback")
|
|
||||||
return self.transport.read_branch(self.receipt.branch)
|
|
||||||
|
|
||||||
|
|
||||||
def reconcile_exact_branch(authority_oid: str, gitea_oid: str | None, expected_oid: str) -> str:
|
|
||||||
if authority_oid != expected_oid:
|
|
||||||
return "AUTHORITY_EXPECTATION_MISMATCH"
|
|
||||||
if gitea_oid is None:
|
|
||||||
return "CREATE_REQUIRED"
|
|
||||||
if gitea_oid == expected_oid:
|
|
||||||
return "ALREADY_IN_SYNC"
|
|
||||||
return "MANUAL_OWNER_REVIEW_REQUIRED"
|
|
||||||
|
|
||||||
|
|
||||||
def reconcile_fast_forward(
|
|
||||||
remote_oid: str | None,
|
|
||||||
expected_old_oid: str,
|
|
||||||
new_oid: str,
|
|
||||||
*,
|
|
||||||
old_is_ancestor_of_new: bool,
|
|
||||||
) -> str:
|
|
||||||
"""Return a bounded action for an active Job/lease branch scope."""
|
|
||||||
if remote_oid is None:
|
|
||||||
return "CREATE_REQUIRED"
|
|
||||||
if remote_oid == new_oid:
|
|
||||||
return "ALREADY_IN_SYNC"
|
|
||||||
if remote_oid == expected_old_oid and old_is_ancestor_of_new:
|
|
||||||
return "FAST_FORWARD_REQUIRED"
|
|
||||||
return "MANUAL_OWNER_REVIEW_REQUIRED"
|
|
||||||
@@ -1,208 +0,0 @@
|
|||||||
"""Gitea HTTP/Git transport using only a projected systemd credential."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import subprocess
|
|
||||||
import tempfile
|
|
||||||
from typing import Any, Mapping
|
|
||||||
from urllib.error import HTTPError, URLError
|
|
||||||
from urllib.parse import quote, urlencode
|
|
||||||
from urllib.request import Request, urlopen
|
|
||||||
|
|
||||||
|
|
||||||
class GiteaTransportError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def _safe_oid(value: str, field: str) -> str:
|
|
||||||
if not isinstance(value, str) or len(value) != 40 or any(ch not in "0123456789abcdef" for ch in value):
|
|
||||||
raise GiteaTransportError(f"{field}_INVALID")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
|
||||||
class GiteaMachineTransport:
|
|
||||||
api_base: str
|
|
||||||
repository_id: str
|
|
||||||
remote_url: str
|
|
||||||
authority_git_dir: Path
|
|
||||||
credential_name: str = "gitea-machine-token"
|
|
||||||
actor: str = "aios-autonomy-agent"
|
|
||||||
timeout: int = 20
|
|
||||||
expected_merge_head: str | None = None
|
|
||||||
expected_merge_base: str | None = None
|
|
||||||
|
|
||||||
def __post_init__(self) -> None:
|
|
||||||
if self.credential_name != "gitea-machine-token":
|
|
||||||
raise GiteaTransportError("CREDENTIAL_NAME_INVALID")
|
|
||||||
if "/" not in self.repository_id or self.repository_id.startswith("/"):
|
|
||||||
raise GiteaTransportError("REPOSITORY_INVALID")
|
|
||||||
if not self.api_base.startswith("https://") and not self.api_base.startswith("http://127.0.0.1"):
|
|
||||||
raise GiteaTransportError("GITEA_API_SCOPE_INVALID")
|
|
||||||
if not self.remote_url.startswith("https://"):
|
|
||||||
raise GiteaTransportError("GITEA_REMOTE_SCOPE_INVALID")
|
|
||||||
|
|
||||||
@property
|
|
||||||
def _credential_path(self) -> Path:
|
|
||||||
directory = os.environ.get("CREDENTIALS_DIRECTORY", "")
|
|
||||||
if not directory:
|
|
||||||
raise GiteaTransportError("CREDENTIAL_PROJECTION_MISSING")
|
|
||||||
path = Path(directory) / self.credential_name
|
|
||||||
if path.is_symlink() or not path.is_file():
|
|
||||||
raise GiteaTransportError("CREDENTIAL_PROJECTION_MISSING")
|
|
||||||
return path
|
|
||||||
|
|
||||||
def _token(self) -> str:
|
|
||||||
value = self._credential_path.read_text(encoding="utf-8").strip()
|
|
||||||
if not value:
|
|
||||||
raise GiteaTransportError("CREDENTIAL_EMPTY")
|
|
||||||
return value
|
|
||||||
|
|
||||||
def _request(self, method: str, path: str, body: Mapping[str, Any] | None = None) -> tuple[int, Any]:
|
|
||||||
data = None if body is None else json.dumps(dict(body), separators=(",", ":")).encode()
|
|
||||||
headers = {"Accept": "application/json", "Authorization": f"token {self._token()}"}
|
|
||||||
if data is not None:
|
|
||||||
headers["Content-Type"] = "application/json"
|
|
||||||
request = Request(self.api_base.rstrip("/") + path, data=data, headers=headers, method=method)
|
|
||||||
try:
|
|
||||||
with urlopen(request, timeout=self.timeout) as response:
|
|
||||||
raw = response.read()
|
|
||||||
return response.status, json.loads(raw.decode()) if raw else {}
|
|
||||||
except HTTPError as exc:
|
|
||||||
# Never include response bodies: Gitea may echo sensitive request context.
|
|
||||||
raise GiteaTransportError(f"GITEA_HTTP_{exc.code}") from exc
|
|
||||||
except (URLError, TimeoutError, OSError) as exc:
|
|
||||||
raise GiteaTransportError("GITEA_NETWORK_ERROR") from exc
|
|
||||||
|
|
||||||
def whoami(self) -> Mapping[str, Any]:
|
|
||||||
return self._request("GET", "/user")[1]
|
|
||||||
|
|
||||||
def repository_info(self) -> Mapping[str, Any]:
|
|
||||||
owner, name = self.repository_id.split("/", 1)
|
|
||||||
return self._request("GET", f"/repos/{quote(owner)}/{quote(name)}")[1]
|
|
||||||
|
|
||||||
def repository(self, owner: str, name: str) -> Mapping[str, Any]:
|
|
||||||
if f"{owner}/{name}" != self.repository_id:
|
|
||||||
raise GiteaTransportError("REPOSITORY_SCOPE_DENIED")
|
|
||||||
return self.repository_info()
|
|
||||||
|
|
||||||
def read_branch(self, branch: str) -> str | None:
|
|
||||||
owner, name = self.repository_id.split("/", 1)
|
|
||||||
try:
|
|
||||||
value = self._request("GET", f"/repos/{quote(owner)}/{quote(name)}/branches/{quote(branch, safe='/')}")[1]
|
|
||||||
except GiteaTransportError as exc:
|
|
||||||
if str(exc) == "GITEA_HTTP_404":
|
|
||||||
return None
|
|
||||||
raise
|
|
||||||
sha = value.get("commit", {}).get("id") if isinstance(value, dict) else None
|
|
||||||
return _safe_oid(sha, "remote_branch_oid")
|
|
||||||
|
|
||||||
def _askpass(self) -> tuple[Path, dict[str, str]]:
|
|
||||||
credential = self._credential_path
|
|
||||||
fd, name = tempfile.mkstemp(prefix="aios-gitea-askpass-", dir="/tmp")
|
|
||||||
path = Path(name)
|
|
||||||
os.fchmod(fd, 0o700)
|
|
||||||
script = "#!/bin/sh\ncase \"$1\" in *Username*) printf '%s\\n' \"$GITEA_MACHINE_ACTOR\" ;; *) cat \"$CREDENTIAL_FILE\" ;; esac\n"
|
|
||||||
with os.fdopen(fd, "w", encoding="utf-8") as stream:
|
|
||||||
stream.write(script)
|
|
||||||
stream.flush()
|
|
||||||
os.fsync(stream.fileno())
|
|
||||||
return path, {
|
|
||||||
"GIT_ASKPASS": str(path),
|
|
||||||
"GIT_TERMINAL_PROMPT": "0",
|
|
||||||
"GITEA_MACHINE_ACTOR": self.actor,
|
|
||||||
"CREDENTIAL_FILE": str(credential),
|
|
||||||
}
|
|
||||||
|
|
||||||
def _push_branch_fast_forward(self, branch: str, new_oid: str) -> None:
|
|
||||||
askpass, additions = self._askpass()
|
|
||||||
environment = os.environ.copy()
|
|
||||||
environment.update(additions)
|
|
||||||
try:
|
|
||||||
result = subprocess.run(
|
|
||||||
["git", f"--git-dir={self.authority_git_dir}", "push", self.remote_url, f"{new_oid}:refs/heads/{branch}"],
|
|
||||||
text=True,
|
|
||||||
capture_output=True,
|
|
||||||
check=False,
|
|
||||||
env=environment,
|
|
||||||
timeout=self.timeout,
|
|
||||||
)
|
|
||||||
if result.returncode != 0:
|
|
||||||
raise GiteaTransportError("GITEA_BRANCH_CREATE_FAILED")
|
|
||||||
finally:
|
|
||||||
askpass.unlink(missing_ok=True)
|
|
||||||
|
|
||||||
def compare_and_swap_branch(self, branch: str, expected_oid: str, new_oid: str) -> str:
|
|
||||||
_safe_oid(expected_oid, "expected_oid")
|
|
||||||
_safe_oid(new_oid, "new_oid")
|
|
||||||
current = self.read_branch(branch)
|
|
||||||
if current == new_oid:
|
|
||||||
return new_oid
|
|
||||||
if current is None:
|
|
||||||
self._push_branch_fast_forward(branch, new_oid)
|
|
||||||
readback = self.read_branch(branch)
|
|
||||||
if readback != new_oid:
|
|
||||||
raise GiteaTransportError("GITEA_BRANCH_CREATE_READBACK_MISMATCH")
|
|
||||||
return readback
|
|
||||||
if current != expected_oid:
|
|
||||||
raise GiteaTransportError("GITEA_BRANCH_EXPECTATION_MISMATCH")
|
|
||||||
# The preflight read binds the expected old tip. The Git receive-pack
|
|
||||||
# then enforces the actual fast-forward CAS while transferring the new
|
|
||||||
# objects; no force/lease override or arbitrary ref is permitted.
|
|
||||||
self._push_branch_fast_forward(branch, new_oid)
|
|
||||||
readback = self.read_branch(branch)
|
|
||||||
if readback != new_oid:
|
|
||||||
raise GiteaTransportError("GITEA_BRANCH_CAS_READBACK_MISMATCH")
|
|
||||||
return readback
|
|
||||||
|
|
||||||
def _pulls(self) -> list[Mapping[str, Any]]:
|
|
||||||
owner, name = self.repository_id.split("/", 1)
|
|
||||||
query = urlencode({"state": "all", "base_branch": "main", "limit": 100})
|
|
||||||
value = self._request("GET", f"/repos/{quote(owner)}/{quote(name)}/pulls?{query}")[1]
|
|
||||||
return value if isinstance(value, list) else []
|
|
||||||
|
|
||||||
def create_or_update_pr(self, branch: str, target: str, title: str, body: str) -> Mapping[str, Any]:
|
|
||||||
if target != "main":
|
|
||||||
raise GiteaTransportError("PR_BASE_SCOPE_DENIED")
|
|
||||||
head = self.read_branch(branch)
|
|
||||||
if head is None:
|
|
||||||
raise GiteaTransportError("PR_HEAD_MISSING")
|
|
||||||
matches = [p for p in self._pulls() if (p.get("head") or {}).get("ref") == branch and (p.get("base") or {}).get("ref") == target]
|
|
||||||
open_matches = [p for p in matches if p.get("state") == "open"]
|
|
||||||
if len(open_matches) > 1:
|
|
||||||
raise GiteaTransportError("MULTIPLE_MATCHING_PRS")
|
|
||||||
if open_matches:
|
|
||||||
if (open_matches[0].get("head") or {}).get("sha") != head:
|
|
||||||
raise GiteaTransportError("PR_HEAD_CONFLICT")
|
|
||||||
return open_matches[0]
|
|
||||||
owner, name = self.repository_id.split("/", 1)
|
|
||||||
return self._request("POST", f"/repos/{quote(owner)}/{quote(name)}/pulls", {"head": branch, "base": target, "title": title, "body": body, "allow_maintainer_edit": False})[1]
|
|
||||||
|
|
||||||
def publish_status(self, branch: str, state: str, description: str, context: str = "aios/durable-release") -> Mapping[str, Any]:
|
|
||||||
oid = self.read_branch(branch)
|
|
||||||
if oid is None:
|
|
||||||
raise GiteaTransportError("STATUS_BRANCH_MISSING")
|
|
||||||
owner, name = self.repository_id.split("/", 1)
|
|
||||||
return self._request("POST", f"/repos/{quote(owner)}/{quote(name)}/statuses/{oid}", {"state": state, "context": context, "description": description})[1]
|
|
||||||
|
|
||||||
def protected_merge(self, pr_number: int) -> Mapping[str, Any]:
|
|
||||||
if not isinstance(pr_number, int) or pr_number < 1:
|
|
||||||
raise GiteaTransportError("PR_NUMBER_INVALID")
|
|
||||||
owner, name = self.repository_id.split("/", 1)
|
|
||||||
pr = self._request("GET", f"/repos/{quote(owner)}/{quote(name)}/pulls/{pr_number}")[1]
|
|
||||||
if pr.get("state") != "open" or (pr.get("base") or {}).get("ref") != "main":
|
|
||||||
raise GiteaTransportError("PR_MERGE_BINDING_INVALID")
|
|
||||||
if pr.get("mergeable") is False:
|
|
||||||
raise GiteaTransportError("PR_NOT_MERGEABLE")
|
|
||||||
head_oid = (pr.get("head") or {}).get("sha")
|
|
||||||
base_oid = (pr.get("base") or {}).get("sha")
|
|
||||||
if self.expected_merge_head is not None and head_oid != self.expected_merge_head:
|
|
||||||
raise GiteaTransportError("PR_HEAD_EXPECTATION_MISMATCH")
|
|
||||||
if self.expected_merge_base is not None and base_oid != self.expected_merge_base:
|
|
||||||
raise GiteaTransportError("PR_BASE_EXPECTATION_MISMATCH")
|
|
||||||
body = {"do": "fast-forward-only", "force_merge": False, "merge_when_checks_succeed": False, "head_commit_id": head_oid}
|
|
||||||
return self._request("POST", f"/repos/{quote(owner)}/{quote(name)}/pulls/{pr_number}/merge", body)[1]
|
|
||||||
@@ -1,93 +0,0 @@
|
|||||||
"""Canonical Gitea transport and slash-branch verification."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import subprocess
|
|
||||||
from typing import Any, Callable, Mapping, Optional
|
|
||||||
from urllib.parse import quote
|
|
||||||
from urllib.request import Request, urlopen
|
|
||||||
|
|
||||||
from aios_gitops_reconciliation_core import RemoteObservation
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class GiteaPublisherConfig:
|
|
||||||
repository: str = "aios/aios"
|
|
||||||
api_base: str = "https://g.cnnetmkt.top/git/api/v1"
|
|
||||||
remote_name: str = "gitea-http"
|
|
||||||
|
|
||||||
|
|
||||||
class GiteaPublisher:
|
|
||||||
def __init__(self, workspace: Path, *, config: GiteaPublisherConfig = GiteaPublisherConfig(), credential: Any = None, api_get: Callable[[str], Any] | None = None, git_ref: Callable[[str], Optional[str]] | None = None) -> None:
|
|
||||||
self.workspace = Path(workspace)
|
|
||||||
self.config = config
|
|
||||||
self.credential = credential
|
|
||||||
self._api_get = api_get
|
|
||||||
self._git_ref = git_ref
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def validate_repository(repository: str) -> None:
|
|
||||||
if repository != "aios/aios":
|
|
||||||
raise ValueError("REPOSITORY_IDENTITY_MISMATCH")
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def validate_branch(branch: str) -> None:
|
|
||||||
if not isinstance(branch, str) or not branch.startswith("agent/") or branch in {"main", "master"}:
|
|
||||||
raise ValueError("BRANCH_SCOPE_DENIED")
|
|
||||||
|
|
||||||
def repository_identity(self) -> str:
|
|
||||||
self.validate_repository(self.config.repository)
|
|
||||||
return self.config.repository
|
|
||||||
|
|
||||||
def _environment(self) -> dict[str, str]:
|
|
||||||
if self.credential is not None and hasattr(self.credential, "environment"):
|
|
||||||
return dict(self.credential.environment())
|
|
||||||
return {}
|
|
||||||
|
|
||||||
def read_git_ref(self, branch: str) -> Optional[str]:
|
|
||||||
self.validate_branch(branch)
|
|
||||||
if self._git_ref is not None:
|
|
||||||
return self._git_ref(branch)
|
|
||||||
result = subprocess.run(["git", "-C", str(self.workspace), "ls-remote", self.config.remote_name, f"refs/heads/{branch}"], text=True, capture_output=True, check=False, env={**os.environ, **self._environment(), "GIT_TERMINAL_PROMPT": "0"})
|
|
||||||
if result.returncode != 0 or not result.stdout.strip():
|
|
||||||
return None
|
|
||||||
return result.stdout.split()[0]
|
|
||||||
|
|
||||||
def _api(self, path: str) -> Any:
|
|
||||||
if self._api_get is not None:
|
|
||||||
return self._api_get(path)
|
|
||||||
headers = {"Accept": "application/json"}
|
|
||||||
env = self._environment()
|
|
||||||
if env.get("AIOS_GITEA_TOKEN"):
|
|
||||||
headers["Authorization"] = "token " + env["AIOS_GITEA_TOKEN"]
|
|
||||||
with urlopen(Request(self.config.api_base.rstrip("/") + path, headers=headers), timeout=20) as response:
|
|
||||||
return json.loads(response.read() or b"{}")
|
|
||||||
|
|
||||||
def read_api_branch(self, branch: str) -> Optional[str]:
|
|
||||||
self.validate_branch(branch)
|
|
||||||
for page in range(1, 101):
|
|
||||||
value = self._api(f"/repos/aios/aios/branches?page={page}&limit=50")
|
|
||||||
if not isinstance(value, list):
|
|
||||||
return None
|
|
||||||
for item in value:
|
|
||||||
if isinstance(item, Mapping) and item.get("name") == branch:
|
|
||||||
commit = item.get("commit")
|
|
||||||
return commit.get("id") if isinstance(commit, Mapping) else None
|
|
||||||
if len(value) < 50:
|
|
||||||
return None
|
|
||||||
raise RuntimeError("GITEA_BRANCH_PAGINATION_LIMIT")
|
|
||||||
|
|
||||||
def commit_visible(self, oid: str) -> bool:
|
|
||||||
if not isinstance(oid, str) or len(oid) != 40:
|
|
||||||
return False
|
|
||||||
try:
|
|
||||||
value = self._api(f"/repos/aios/aios/git/commits/{quote(oid, safe='')}")
|
|
||||||
return isinstance(value, Mapping) and value.get("sha", oid) == oid
|
|
||||||
except Exception:
|
|
||||||
return False
|
|
||||||
|
|
||||||
def observe(self, branch: str, desired_oid: str, authority_oid: Optional[str], *, expected_oid: Optional[str] = None, observed_at: str = "") -> RemoteObservation:
|
|
||||||
return RemoteObservation(self.repository_identity(), branch, desired_oid, authority_oid, self.read_git_ref(branch), self.read_api_branch(branch), self.commit_visible(desired_oid), observed_at, expected_oid)
|
|
||||||
@@ -1,595 +0,0 @@
|
|||||||
"""SQLite-backed durable GitOps Job Controller.
|
|
||||||
|
|
||||||
The controller owns durable intent, idempotency, lease metadata, state
|
|
||||||
transitions, receipts, and payload references. A worker owns Git side effects
|
|
||||||
behind the campaign capability and branch-CAS boundaries.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from enum import Enum
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import sqlite3
|
|
||||||
import tempfile
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from typing import Any, Callable, Mapping
|
|
||||||
|
|
||||||
|
|
||||||
class JobError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
class ServiceUnavailable(JobError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
class BindingConflict(JobError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
class JobState(str, Enum):
|
|
||||||
REQUESTED = "REQUESTED"
|
|
||||||
AUTHORIZED = "AUTHORIZED"
|
|
||||||
LEASED = "LEASED"
|
|
||||||
WORKTREE_READY = "WORKTREE_READY"
|
|
||||||
EXECUTING = "EXECUTING"
|
|
||||||
LOCAL_VERIFIED = "LOCAL_VERIFIED"
|
|
||||||
REMOTE_SYNCING = "REMOTE_SYNCING"
|
|
||||||
REMOTE_VERIFIED = "REMOTE_VERIFIED"
|
|
||||||
COMPLETED = "COMPLETED"
|
|
||||||
RECOVERABLE_FAILURE = "RECOVERABLE_FAILURE"
|
|
||||||
RECONCILING = "RECONCILING"
|
|
||||||
RECONCILIATION_REQUIRED = "RECONCILIATION_REQUIRED"
|
|
||||||
MANUAL_OWNER_REVIEW_REQUIRED = "MANUAL_OWNER_REVIEW_REQUIRED"
|
|
||||||
|
|
||||||
|
|
||||||
_ORDER = {
|
|
||||||
JobState.REQUESTED: 0,
|
|
||||||
JobState.AUTHORIZED: 1,
|
|
||||||
JobState.LEASED: 2,
|
|
||||||
JobState.WORKTREE_READY: 3,
|
|
||||||
JobState.EXECUTING: 4,
|
|
||||||
JobState.LOCAL_VERIFIED: 5,
|
|
||||||
JobState.REMOTE_SYNCING: 6,
|
|
||||||
JobState.REMOTE_VERIFIED: 7,
|
|
||||||
JobState.COMPLETED: 8,
|
|
||||||
}
|
|
||||||
_RECOVERABLE = {JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}
|
|
||||||
_TERMINAL = {JobState.COMPLETED, JobState.MANUAL_OWNER_REVIEW_REQUIRED}
|
|
||||||
|
|
||||||
|
|
||||||
def utc_now() -> str:
|
|
||||||
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
|
||||||
|
|
||||||
|
|
||||||
def canonical_json(value: Any) -> bytes:
|
|
||||||
return (
|
|
||||||
json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + "\n"
|
|
||||||
).encode("utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
def sha256_json(value: Any) -> str:
|
|
||||||
return hashlib.sha256(canonical_json(value)).hexdigest()
|
|
||||||
|
|
||||||
|
|
||||||
class PayloadStore:
|
|
||||||
"""Content-addressed filesystem payloads; raw content never enters SQLite."""
|
|
||||||
|
|
||||||
def __init__(self, root: str | os.PathLike[str]) -> None:
|
|
||||||
self.root = Path(root)
|
|
||||||
if not self.root.is_absolute() or self.root.is_symlink():
|
|
||||||
raise JobError("PAYLOAD_ROOT_INVALID")
|
|
||||||
self.root.mkdir(parents=True, exist_ok=True, mode=0o700)
|
|
||||||
os.chmod(self.root, 0o700)
|
|
||||||
|
|
||||||
def put(self, content: bytes, expected_digest: str) -> tuple[str, int]:
|
|
||||||
digest = hashlib.sha256(content).hexdigest()
|
|
||||||
if digest != expected_digest:
|
|
||||||
raise JobError("PAYLOAD_DIGEST_MISMATCH")
|
|
||||||
if len(expected_digest) != 64 or any(ch not in "0123456789abcdef" for ch in expected_digest):
|
|
||||||
raise JobError("PAYLOAD_DIGEST_INVALID")
|
|
||||||
target = self.root / expected_digest
|
|
||||||
if target.is_symlink():
|
|
||||||
raise JobError("PAYLOAD_PATH_SYMLINK")
|
|
||||||
if target.is_file():
|
|
||||||
if target.stat().st_size != len(content) or hashlib.sha256(target.read_bytes()).hexdigest() != digest:
|
|
||||||
raise JobError("PAYLOAD_EXISTING_MISMATCH")
|
|
||||||
return f"sha256/{digest}", len(content)
|
|
||||||
fd, temporary = tempfile.mkstemp(prefix=".payload-", dir=self.root)
|
|
||||||
try:
|
|
||||||
os.fchmod(fd, 0o600)
|
|
||||||
with os.fdopen(fd, "wb") as stream:
|
|
||||||
stream.write(content)
|
|
||||||
stream.flush()
|
|
||||||
os.fsync(stream.fileno())
|
|
||||||
os.replace(temporary, target)
|
|
||||||
os.chmod(target, 0o600)
|
|
||||||
finally:
|
|
||||||
if os.path.exists(temporary):
|
|
||||||
os.unlink(temporary)
|
|
||||||
return f"sha256/{digest}", len(content)
|
|
||||||
|
|
||||||
def read_verified(self, reference: str, expected_digest: str, expected_size: int) -> bytes:
|
|
||||||
if reference != f"sha256/{expected_digest}":
|
|
||||||
raise JobError("PAYLOAD_REFERENCE_MISMATCH")
|
|
||||||
path = self.root / expected_digest
|
|
||||||
if path.is_symlink() or not path.is_file():
|
|
||||||
raise JobError("PAYLOAD_MISSING")
|
|
||||||
content = path.read_bytes()
|
|
||||||
if len(content) != expected_size or hashlib.sha256(content).hexdigest() != expected_digest:
|
|
||||||
raise JobError("PAYLOAD_VERIFY_FAILED")
|
|
||||||
return content
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class JobReceipt:
|
|
||||||
job_id: str
|
|
||||||
idempotency_key: str
|
|
||||||
state: JobState
|
|
||||||
side_effect_state: str
|
|
||||||
http_status: int
|
|
||||||
revision: int
|
|
||||||
receipt: Mapping[str, Any] | None = None
|
|
||||||
|
|
||||||
def as_dict(self) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"job_id": self.job_id,
|
|
||||||
"idempotency_key": self.idempotency_key,
|
|
||||||
"state": self.state.value,
|
|
||||||
"side_effect_state": self.side_effect_state,
|
|
||||||
"http_status": self.http_status,
|
|
||||||
"revision": self.revision,
|
|
||||||
"receipt": dict(self.receipt) if self.receipt else None,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
class JobStore:
|
|
||||||
SCHEMA_VERSION = 1
|
|
||||||
|
|
||||||
def __init__(self, root: str | os.PathLike[str], *, payload_root: str | os.PathLike[str] | None = None) -> None:
|
|
||||||
self.root = Path(root)
|
|
||||||
if not self.root.is_absolute() or self.root.is_symlink():
|
|
||||||
raise JobError("JOB_ROOT_INVALID")
|
|
||||||
self.root.mkdir(parents=True, exist_ok=True, mode=0o700)
|
|
||||||
os.chmod(self.root, 0o700)
|
|
||||||
self.db_path = self.root / "state.sqlite3"
|
|
||||||
self.payloads = PayloadStore(payload_root or self.root.parent / "payloads" / "sha256")
|
|
||||||
self._initialize()
|
|
||||||
|
|
||||||
def _connect(self) -> sqlite3.Connection:
|
|
||||||
if self.db_path.is_symlink():
|
|
||||||
raise ServiceUnavailable("JOB_DB_SYMLINK")
|
|
||||||
connection = sqlite3.connect(self.db_path, timeout=30.0)
|
|
||||||
connection.row_factory = sqlite3.Row
|
|
||||||
connection.execute("PRAGMA journal_mode=WAL")
|
|
||||||
connection.execute("PRAGMA synchronous=FULL")
|
|
||||||
connection.execute("PRAGMA foreign_keys=ON")
|
|
||||||
connection.execute("PRAGMA busy_timeout=30000")
|
|
||||||
return connection
|
|
||||||
|
|
||||||
def _initialize(self) -> None:
|
|
||||||
try:
|
|
||||||
with self._connect() as connection:
|
|
||||||
connection.executescript(
|
|
||||||
"""
|
|
||||||
CREATE TABLE IF NOT EXISTS metadata (key TEXT PRIMARY KEY, value TEXT NOT NULL);
|
|
||||||
CREATE TABLE IF NOT EXISTS jobs (
|
|
||||||
job_id TEXT PRIMARY KEY, idempotency_key TEXT NOT NULL UNIQUE,
|
|
||||||
campaign_id TEXT NOT NULL, repository TEXT NOT NULL, operation TEXT NOT NULL,
|
|
||||||
branch TEXT NOT NULL, expected_branch_oid TEXT NOT NULL,
|
|
||||||
payload_digest TEXT NOT NULL, payload_ref TEXT, payload_size INTEGER,
|
|
||||||
state TEXT NOT NULL, side_effect_state TEXT NOT NULL,
|
|
||||||
revision INTEGER NOT NULL, request_json TEXT NOT NULL,
|
|
||||||
receipt_json TEXT, created_at TEXT NOT NULL, updated_at TEXT NOT NULL
|
|
||||||
);
|
|
||||||
CREATE TABLE IF NOT EXISTS idempotency_keys (
|
|
||||||
key TEXT PRIMARY KEY, job_id TEXT NOT NULL UNIQUE,
|
|
||||||
binding_json TEXT NOT NULL, created_at TEXT NOT NULL,
|
|
||||||
FOREIGN KEY(job_id) REFERENCES jobs(job_id)
|
|
||||||
);
|
|
||||||
CREATE TABLE IF NOT EXISTS branch_leases (
|
|
||||||
lease_id TEXT PRIMARY KEY, job_id TEXT NOT NULL UNIQUE,
|
|
||||||
branch TEXT NOT NULL, lineage_anchor_oid TEXT NOT NULL,
|
|
||||||
current_head_oid TEXT NOT NULL, generation INTEGER NOT NULL,
|
|
||||||
lease_expires_at TEXT NOT NULL, state TEXT NOT NULL,
|
|
||||||
revision INTEGER NOT NULL, FOREIGN KEY(job_id) REFERENCES jobs(job_id)
|
|
||||||
);
|
|
||||||
CREATE TABLE IF NOT EXISTS state_transitions (
|
|
||||||
sequence INTEGER PRIMARY KEY AUTOINCREMENT, job_id TEXT NOT NULL,
|
|
||||||
from_state TEXT, to_state TEXT NOT NULL, revision INTEGER NOT NULL,
|
|
||||||
side_effect_state TEXT NOT NULL, created_at TEXT NOT NULL,
|
|
||||||
FOREIGN KEY(job_id) REFERENCES jobs(job_id)
|
|
||||||
);
|
|
||||||
CREATE TABLE IF NOT EXISTS receipts (
|
|
||||||
receipt_id TEXT PRIMARY KEY, job_id TEXT NOT NULL,
|
|
||||||
kind TEXT NOT NULL, digest TEXT NOT NULL, payload_json TEXT NOT NULL,
|
|
||||||
created_at TEXT NOT NULL, FOREIGN KEY(job_id) REFERENCES jobs(job_id)
|
|
||||||
);
|
|
||||||
INSERT OR REPLACE INTO metadata(key, value) VALUES('schema_version', '1');
|
|
||||||
"""
|
|
||||||
)
|
|
||||||
except sqlite3.Error as exc:
|
|
||||||
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
|
|
||||||
|
|
||||||
def _mirror(self, job: Mapping[str, Any]) -> None:
|
|
||||||
directory = self.root / str(job["job_id"])
|
|
||||||
directory.mkdir(mode=0o700, exist_ok=True)
|
|
||||||
(directory / "receipts").mkdir(mode=0o700, exist_ok=True)
|
|
||||||
(directory / "logs-digest").mkdir(mode=0o700, exist_ok=True)
|
|
||||||
for name, value in (("request.json", job["request"]), ("state.json", job)):
|
|
||||||
target = directory / name
|
|
||||||
fd, temporary = tempfile.mkstemp(prefix=".mirror-", dir=directory)
|
|
||||||
try:
|
|
||||||
os.fchmod(fd, 0o600)
|
|
||||||
with os.fdopen(fd, "wb") as stream:
|
|
||||||
stream.write(canonical_json(value))
|
|
||||||
stream.flush()
|
|
||||||
os.fsync(stream.fileno())
|
|
||||||
os.replace(temporary, target)
|
|
||||||
finally:
|
|
||||||
if os.path.exists(temporary):
|
|
||||||
os.unlink(temporary)
|
|
||||||
|
|
||||||
def read(self, job_id: str) -> dict[str, Any]:
|
|
||||||
try:
|
|
||||||
with self._connect() as connection:
|
|
||||||
row = connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone()
|
|
||||||
except sqlite3.Error as exc:
|
|
||||||
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
|
|
||||||
if row is None:
|
|
||||||
raise JobError("JOB_NOT_FOUND")
|
|
||||||
return dict(row)
|
|
||||||
|
|
||||||
def _receipt(self, row: Mapping[str, Any], *, duplicate: bool) -> JobReceipt:
|
|
||||||
state = JobState(row["state"])
|
|
||||||
if state == JobState.COMPLETED:
|
|
||||||
status, side = 200, "COMPLETED"
|
|
||||||
elif state in _RECOVERABLE:
|
|
||||||
status = 202
|
|
||||||
side = row["side_effect_state"] if row["side_effect_state"] == "REMOTE_PUSH_PENDING" else "RESUME_SAME_JOB"
|
|
||||||
elif row["side_effect_state"] == "UNKNOWN":
|
|
||||||
status, side = 503, "UNKNOWN"
|
|
||||||
else:
|
|
||||||
status, side = 202, "SAME_JOB" if duplicate else "QUEUED"
|
|
||||||
receipt = json.loads(row["receipt_json"]) if row["receipt_json"] else None
|
|
||||||
return JobReceipt(row["job_id"], row["idempotency_key"], state, side, status, int(row["revision"]), receipt)
|
|
||||||
|
|
||||||
|
|
||||||
class GitOpsJobController:
|
|
||||||
def __init__(self, store: JobStore) -> None:
|
|
||||||
self.store = store
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _binding(request: Mapping[str, Any]) -> dict[str, str]:
|
|
||||||
fields = ("campaign_id", "repository", "operation", "branch", "expected_branch_oid", "payload_digest")
|
|
||||||
if any(not isinstance(request.get(field), str) or not request[field].strip() for field in fields):
|
|
||||||
raise JobError("JOB_BINDING_INVALID")
|
|
||||||
oid = request["expected_branch_oid"]
|
|
||||||
digest = request["payload_digest"]
|
|
||||||
if len(oid) != 40 or any(ch not in "0123456789abcdef" for ch in oid):
|
|
||||||
raise JobError("EXPECTED_BRANCH_OID_INVALID")
|
|
||||||
if len(digest) != 64 or any(ch not in "0123456789abcdef" for ch in digest):
|
|
||||||
raise JobError("PAYLOAD_DIGEST_INVALID")
|
|
||||||
if request["branch"] in {"main", "master"} or request["branch"].startswith(("refs/", "tags/")):
|
|
||||||
raise JobError("PROTECTED_BRANCH_REJECTED")
|
|
||||||
binding = {field: str(request[field]) for field in fields}
|
|
||||||
# Operation-specific, non-secret binding inputs must survive the
|
|
||||||
# durable round trip and participate in idempotency. Arbitrary request
|
|
||||||
# fields are intentionally discarded rather than persisted.
|
|
||||||
optional = (
|
|
||||||
"idempotency_key",
|
|
||||||
"task_id",
|
|
||||||
"new_branch_oid",
|
|
||||||
"authorization_recovery_digest",
|
|
||||||
"lineage_anchor_oid",
|
|
||||||
"base_oid",
|
|
||||||
"branch_mode",
|
|
||||||
"generation",
|
|
||||||
"expected_main_oid",
|
|
||||||
"pr_number",
|
|
||||||
"target_branch",
|
|
||||||
"check_context",
|
|
||||||
"check_state",
|
|
||||||
"check_description",
|
|
||||||
)
|
|
||||||
for field in optional:
|
|
||||||
value = request.get(field)
|
|
||||||
if value is None:
|
|
||||||
continue
|
|
||||||
if isinstance(value, bool) or not isinstance(value, (str, int)):
|
|
||||||
raise JobError("JOB_BINDING_OPTION_INVALID")
|
|
||||||
text = str(value).strip()
|
|
||||||
if not text or len(text) > 512:
|
|
||||||
raise JobError("JOB_BINDING_OPTION_INVALID")
|
|
||||||
binding[field] = text
|
|
||||||
return binding
|
|
||||||
|
|
||||||
def submit(self, request: Mapping[str, Any], *, payload: bytes | None = None) -> JobReceipt:
|
|
||||||
binding = self._binding(request)
|
|
||||||
payload_ref = request.get("payload_ref")
|
|
||||||
payload_size = request.get("payload_size")
|
|
||||||
if payload is not None:
|
|
||||||
payload_ref, payload_size = self.store.payloads.put(payload, binding["payload_digest"])
|
|
||||||
if payload_ref is not None and payload_ref != f"sha256/{binding['payload_digest']}":
|
|
||||||
raise JobError("PAYLOAD_REFERENCE_MISMATCH")
|
|
||||||
if payload_ref is not None and (not isinstance(payload_size, int) or payload_size < 0):
|
|
||||||
raise JobError("PAYLOAD_SIZE_INVALID")
|
|
||||||
client_idempotency_key = binding.get("idempotency_key")
|
|
||||||
if client_idempotency_key is not None:
|
|
||||||
if len(client_idempotency_key) > 128 or not all(
|
|
||||||
character.isalnum() or character in "._:-" for character in client_idempotency_key
|
|
||||||
):
|
|
||||||
raise JobError("IDEMPOTENCY_KEY_INVALID")
|
|
||||||
key = sha256_json({"client_idempotency_key": client_idempotency_key})
|
|
||||||
else:
|
|
||||||
key = sha256_json(binding)
|
|
||||||
job_id = "job-" + key[:32]
|
|
||||||
durable_request = dict(binding)
|
|
||||||
if payload_ref is not None:
|
|
||||||
durable_request.update({"payload_ref": str(payload_ref), "payload_size": int(payload_size)})
|
|
||||||
now = utc_now()
|
|
||||||
try:
|
|
||||||
with self.store._connect() as connection:
|
|
||||||
connection.execute("BEGIN IMMEDIATE")
|
|
||||||
existing = connection.execute("SELECT * FROM jobs WHERE idempotency_key=?", (key,)).fetchone()
|
|
||||||
if existing is not None:
|
|
||||||
if json.loads(existing["request_json"]).get("binding") != binding:
|
|
||||||
raise BindingConflict("BINDING_CONFLICT")
|
|
||||||
row = dict(existing)
|
|
||||||
connection.commit()
|
|
||||||
return self.store._receipt(row, duplicate=True)
|
|
||||||
connection.execute(
|
|
||||||
"INSERT INTO jobs(job_id,idempotency_key,campaign_id,repository,operation,branch,expected_branch_oid,payload_digest,payload_ref,payload_size,state,side_effect_state,revision,request_json,receipt_json,created_at,updated_at) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
|
|
||||||
(job_id, key, binding["campaign_id"], binding["repository"], binding["operation"], binding["branch"], binding["expected_branch_oid"], binding["payload_digest"], payload_ref, payload_size, JobState.REQUESTED.value, "NONE", 0, json.dumps({"binding": binding, **durable_request}, sort_keys=True), None, now, now),
|
|
||||||
)
|
|
||||||
connection.execute("INSERT INTO idempotency_keys(key,job_id,binding_json,created_at) VALUES(?,?,?,?)", (key, job_id, json.dumps(binding, sort_keys=True), now))
|
|
||||||
connection.execute("INSERT INTO state_transitions(job_id,from_state,to_state,revision,side_effect_state,created_at) VALUES(?,?,?,?,?,?)", (job_id, None, JobState.REQUESTED.value, 0, "NONE", now))
|
|
||||||
row = dict(connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone())
|
|
||||||
connection.commit()
|
|
||||||
except BindingConflict:
|
|
||||||
raise
|
|
||||||
except sqlite3.IntegrityError:
|
|
||||||
row = self.store.read(job_id)
|
|
||||||
return self.store._receipt(row, duplicate=True)
|
|
||||||
except sqlite3.Error as exc:
|
|
||||||
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
|
|
||||||
row["request"] = json.loads(row["request_json"])
|
|
||||||
self.store._mirror(row)
|
|
||||||
return self.store._receipt(row, duplicate=False)
|
|
||||||
|
|
||||||
def lease_job(
|
|
||||||
self,
|
|
||||||
job_id: str,
|
|
||||||
*,
|
|
||||||
lease_id: str,
|
|
||||||
lineage_anchor_oid: str,
|
|
||||||
current_head_oid: str,
|
|
||||||
generation: int,
|
|
||||||
lease_expires_at: str,
|
|
||||||
expected_revision: int,
|
|
||||||
) -> JobReceipt:
|
|
||||||
"""Atomically create a branch lease and move its Job to LEASED."""
|
|
||||||
if generation < 1 or any(len(oid) != 40 for oid in (lineage_anchor_oid, current_head_oid)):
|
|
||||||
raise JobError("LEASE_BINDING_INVALID")
|
|
||||||
try:
|
|
||||||
with self.store._connect() as connection:
|
|
||||||
connection.execute("BEGIN IMMEDIATE")
|
|
||||||
row = connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone()
|
|
||||||
if row is None:
|
|
||||||
raise JobError("JOB_NOT_FOUND")
|
|
||||||
if int(row["revision"]) != expected_revision:
|
|
||||||
raise JobError("REVISION_CAS_CONFLICT")
|
|
||||||
if row["branch"] in {"main", "master"}:
|
|
||||||
raise JobError("PROTECTED_BRANCH_REJECTED")
|
|
||||||
now = utc_now()
|
|
||||||
connection.execute("INSERT INTO branch_leases(lease_id,job_id,branch,lineage_anchor_oid,current_head_oid,generation,lease_expires_at,state,revision) VALUES(?,?,?,?,?,?,?,?,?)", (lease_id, job_id, row["branch"], lineage_anchor_oid, current_head_oid, generation, lease_expires_at, "ACTIVE", 0))
|
|
||||||
next_revision = expected_revision + 1
|
|
||||||
updated = connection.execute("UPDATE jobs SET state=?,side_effect_state=?,revision=?,updated_at=? WHERE job_id=? AND revision=?", (JobState.LEASED.value, "LEASED", next_revision, now, job_id, expected_revision))
|
|
||||||
if updated.rowcount != 1:
|
|
||||||
raise JobError("REVISION_CAS_CONFLICT")
|
|
||||||
connection.execute("INSERT INTO state_transitions(job_id,from_state,to_state,revision,side_effect_state,created_at) VALUES(?,?,?,?,?,?)", (job_id, row["state"], JobState.LEASED.value, next_revision, "LEASED", now))
|
|
||||||
result = dict(connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone())
|
|
||||||
connection.commit()
|
|
||||||
except sqlite3.IntegrityError as exc:
|
|
||||||
raise JobError("LEASE_CONFLICT") from exc
|
|
||||||
except (JobError, BindingConflict):
|
|
||||||
raise
|
|
||||||
except sqlite3.Error as exc:
|
|
||||||
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
|
|
||||||
result["request"] = json.loads(result["request_json"])
|
|
||||||
self.store._mirror(result)
|
|
||||||
return self.store._receipt(result, duplicate=True)
|
|
||||||
|
|
||||||
def update_lease(self, lease_id: str, *, expected_revision: int, current_head_oid: str, generation: int, state: str = "ACTIVE") -> None:
|
|
||||||
if len(current_head_oid) != 40 or generation < 1:
|
|
||||||
raise JobError("LEASE_BINDING_INVALID")
|
|
||||||
try:
|
|
||||||
with self.store._connect() as connection:
|
|
||||||
connection.execute("BEGIN IMMEDIATE")
|
|
||||||
updated = connection.execute("UPDATE branch_leases SET current_head_oid=?,generation=?,state=?,revision=revision+1 WHERE lease_id=? AND revision=?", (current_head_oid, generation, state, lease_id, expected_revision))
|
|
||||||
if updated.rowcount != 1:
|
|
||||||
raise JobError("LEASE_REVISION_CAS_CONFLICT")
|
|
||||||
connection.commit()
|
|
||||||
except JobError:
|
|
||||||
raise
|
|
||||||
except sqlite3.Error as exc:
|
|
||||||
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
|
|
||||||
|
|
||||||
def advance(self, job_id: str, state: JobState, *, side_effect_state: str, receipt: Mapping[str, Any] | None = None, expected_revision: int | None = None) -> JobReceipt:
|
|
||||||
return self._advance(
|
|
||||||
job_id,
|
|
||||||
state,
|
|
||||||
side_effect_state=side_effect_state,
|
|
||||||
receipt=receipt,
|
|
||||||
expected_revision=expected_revision,
|
|
||||||
)
|
|
||||||
|
|
||||||
def _advance(
|
|
||||||
self,
|
|
||||||
job_id: str,
|
|
||||||
state: JobState,
|
|
||||||
*,
|
|
||||||
side_effect_state: str,
|
|
||||||
receipt: Mapping[str, Any] | None = None,
|
|
||||||
expected_revision: int | None = None,
|
|
||||||
_allow_verified_external_recovery: bool = False,
|
|
||||||
) -> JobReceipt:
|
|
||||||
try:
|
|
||||||
with self.store._connect() as connection:
|
|
||||||
connection.execute("BEGIN IMMEDIATE")
|
|
||||||
row = connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone()
|
|
||||||
if row is None:
|
|
||||||
raise JobError("JOB_NOT_FOUND")
|
|
||||||
previous = JobState(row["state"])
|
|
||||||
revision = int(row["revision"])
|
|
||||||
if expected_revision is not None and expected_revision != revision:
|
|
||||||
raise JobError("REVISION_CAS_CONFLICT")
|
|
||||||
recovery_transition = (
|
|
||||||
_allow_verified_external_recovery
|
|
||||||
and previous == JobState.MANUAL_OWNER_REVIEW_REQUIRED
|
|
||||||
and state == JobState.COMPLETED
|
|
||||||
)
|
|
||||||
if previous in _TERMINAL and state != previous and not recovery_transition:
|
|
||||||
raise JobError("JOB_TERMINAL")
|
|
||||||
if state not in _RECOVERABLE | {JobState.RECONCILING, JobState.MANUAL_OWNER_REVIEW_REQUIRED}:
|
|
||||||
if previous in _RECOVERABLE:
|
|
||||||
raise JobError("RECONCILE_REQUIRED")
|
|
||||||
if previous != JobState.RECONCILING and _ORDER.get(state, -1) < _ORDER.get(previous, -1):
|
|
||||||
raise JobError("JOB_STATE_REGRESSION")
|
|
||||||
next_revision = revision + 1
|
|
||||||
receipt_json = json.dumps(dict(receipt), sort_keys=True) if receipt is not None else row["receipt_json"]
|
|
||||||
now = utc_now()
|
|
||||||
updated = connection.execute("UPDATE jobs SET state=?,side_effect_state=?,revision=?,receipt_json=?,updated_at=? WHERE job_id=? AND revision=?", (state.value, side_effect_state, next_revision, receipt_json, now, job_id, revision))
|
|
||||||
if updated.rowcount != 1:
|
|
||||||
raise JobError("REVISION_CAS_CONFLICT")
|
|
||||||
connection.execute("INSERT INTO state_transitions(job_id,from_state,to_state,revision,side_effect_state,created_at) VALUES(?,?,?,?,?,?)", (job_id, previous.value, state.value, next_revision, side_effect_state, now))
|
|
||||||
if receipt is not None:
|
|
||||||
digest = hashlib.sha256(canonical_json(receipt)).hexdigest()
|
|
||||||
connection.execute("INSERT INTO receipts(receipt_id,job_id,kind,digest,payload_json,created_at) VALUES(?,?,?,?,?,?)", (f"receipt-{job_id}-{next_revision}", job_id, state.value, digest, json.dumps(dict(receipt), sort_keys=True), now))
|
|
||||||
result = dict(connection.execute("SELECT * FROM jobs WHERE job_id=?", (job_id,)).fetchone())
|
|
||||||
connection.commit()
|
|
||||||
except (JobError, BindingConflict):
|
|
||||||
raise
|
|
||||||
except sqlite3.Error as exc:
|
|
||||||
raise ServiceUnavailable("JOB_DB_UNAVAILABLE") from exc
|
|
||||||
result["request"] = json.loads(result["request_json"])
|
|
||||||
self.store._mirror(result)
|
|
||||||
return self.store._receipt(result, duplicate=True)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _validate_verified_external_merge_proof(
|
|
||||||
job_id: str,
|
|
||||||
binding: Mapping[str, Any],
|
|
||||||
proof: Mapping[str, Any],
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
if not isinstance(proof, Mapping):
|
|
||||||
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_INVALID")
|
|
||||||
|
|
||||||
def require_oid(field: str) -> str:
|
|
||||||
value = proof.get(field)
|
|
||||||
if not isinstance(value, str) or len(value) != 40 or any(ch not in "0123456789abcdef" for ch in value):
|
|
||||||
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_INVALID")
|
|
||||||
return value
|
|
||||||
|
|
||||||
if proof.get("proof_schema") != "aios.job-recovery.v1":
|
|
||||||
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_INVALID")
|
|
||||||
if proof.get("job_id") != job_id:
|
|
||||||
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
|
|
||||||
if proof.get("operation") != "PROTECTED_MERGE" or binding.get("operation") != "PROTECTED_MERGE":
|
|
||||||
raise JobError("EXTERNAL_SIDE_EFFECT_OPERATION_REJECTED")
|
|
||||||
if proof.get("pr_number") != 53 or str(binding.get("pr_number")) != "53":
|
|
||||||
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
|
|
||||||
if proof.get("pr_merged") is not True or proof.get("protected_merge") is not True:
|
|
||||||
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
|
|
||||||
if proof.get("force_push") is not False:
|
|
||||||
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
|
|
||||||
if proof.get("machine_actor") != "aios-autonomy-agent":
|
|
||||||
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
|
|
||||||
|
|
||||||
expected_head = str(binding.get("expected_branch_oid", ""))
|
|
||||||
expected_base = str(binding.get("expected_main_oid", ""))
|
|
||||||
head_oid = require_oid("head_oid")
|
|
||||||
base_oid = require_oid("base_oid")
|
|
||||||
merged_main = require_oid("main_oid_after_protected_merge")
|
|
||||||
gitea_merged_main = require_oid("gitea_main_oid_after_protected_merge")
|
|
||||||
authority_merged_main = require_oid("authority_main_oid_after_protected_merge")
|
|
||||||
current_gitea = require_oid("current_gitea_main_oid")
|
|
||||||
current_authority = require_oid("current_authority_main_oid")
|
|
||||||
if head_oid != expected_head or base_oid != expected_base:
|
|
||||||
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
|
|
||||||
if not (merged_main == gitea_merged_main == authority_merged_main == head_oid):
|
|
||||||
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
|
|
||||||
if current_gitea != current_authority or proof.get("current_main_descends_from_protected_merge") is not True:
|
|
||||||
raise JobError("EXTERNAL_SIDE_EFFECT_PROOF_MISMATCH")
|
|
||||||
return dict(proof)
|
|
||||||
|
|
||||||
def complete_verified_external_side_effect(
|
|
||||||
self,
|
|
||||||
job_id: str,
|
|
||||||
proof: Mapping[str, Any],
|
|
||||||
*,
|
|
||||||
expected_revision: int | None = None,
|
|
||||||
) -> JobReceipt:
|
|
||||||
"""Close one historical manual job from independently verified proof.
|
|
||||||
|
|
||||||
This is deliberately narrower than advance: it is only valid for the
|
|
||||||
known protected PR53 recovery shape and preserves the old receipt.
|
|
||||||
The actual state change still uses the normal revision-CAS path.
|
|
||||||
"""
|
|
||||||
current = self.store.read(job_id)
|
|
||||||
revision = int(current["revision"])
|
|
||||||
if expected_revision is not None and expected_revision != revision:
|
|
||||||
raise JobError("REVISION_CAS_CONFLICT")
|
|
||||||
request = json.loads(current["request_json"])
|
|
||||||
binding = request.get("binding", request)
|
|
||||||
normalized_proof = self._validate_verified_external_merge_proof(job_id, binding, proof)
|
|
||||||
existing_receipt = json.loads(current["receipt_json"]) if current["receipt_json"] else None
|
|
||||||
if JobState(current["state"]) == JobState.COMPLETED:
|
|
||||||
if (
|
|
||||||
existing_receipt
|
|
||||||
and existing_receipt.get("recovered_from_historical_manual") is True
|
|
||||||
and existing_receipt.get("proof") == normalized_proof
|
|
||||||
):
|
|
||||||
return self.store._receipt(current, duplicate=True)
|
|
||||||
raise JobError("JOB_TERMINAL")
|
|
||||||
if JobState(current["state"]) != JobState.MANUAL_OWNER_REVIEW_REQUIRED:
|
|
||||||
raise JobError("EXTERNAL_SIDE_EFFECT_RECOVERY_STATE_INVALID")
|
|
||||||
recovery_receipt = {
|
|
||||||
"recovered_from_historical_manual": True,
|
|
||||||
"recovery_kind": "VERIFIED_EXTERNAL_SIDE_EFFECT",
|
|
||||||
"historical_receipt": existing_receipt,
|
|
||||||
"proof": normalized_proof,
|
|
||||||
}
|
|
||||||
return self._advance(
|
|
||||||
job_id,
|
|
||||||
JobState.COMPLETED,
|
|
||||||
side_effect_state="REMOTE_VERIFIED",
|
|
||||||
receipt=recovery_receipt,
|
|
||||||
expected_revision=revision,
|
|
||||||
_allow_verified_external_recovery=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
def reconcile(self, job_id: str, observer: Callable[[Mapping[str, Any]], Mapping[str, Any]]) -> JobReceipt:
|
|
||||||
current = self.store.read(job_id)
|
|
||||||
if JobState(current["state"]) not in _RECOVERABLE:
|
|
||||||
return self.store._receipt(current, duplicate=True)
|
|
||||||
self.advance(job_id, JobState.RECONCILING, side_effect_state="IN_PROGRESS")
|
|
||||||
try:
|
|
||||||
observed = dict(observer(current))
|
|
||||||
stable = JobState(observed.get("state", JobState.REQUESTED.value))
|
|
||||||
if stable in {JobState.RECOVERABLE_FAILURE, JobState.RECONCILING}:
|
|
||||||
raise JobError("RECONCILIATION_UNSTABLE")
|
|
||||||
return self.advance(job_id, stable, side_effect_state=str(observed.get("side_effect_state", "NONE")), receipt=observed.get("receipt"))
|
|
||||||
except Exception as exc:
|
|
||||||
self.advance(job_id, JobState.RECONCILIATION_REQUIRED, side_effect_state="UNKNOWN", receipt={"error_code": type(exc).__name__})
|
|
||||||
raise JobError("RECONCILIATION_REQUIRED") from exc
|
|
||||||
|
|
||||||
def resume(self, job_id: str) -> JobReceipt:
|
|
||||||
current = self.store.read(job_id)
|
|
||||||
if JobState(current["state"]) in _TERMINAL:
|
|
||||||
raise JobError("JOB_TERMINAL")
|
|
||||||
return self.reconcile(job_id, lambda _current: {"state": JobState.REQUESTED.value, "side_effect_state": "RESUME_SAME_JOB"})
|
|
||||||
@@ -1,279 +0,0 @@
|
|||||||
"""Durable Job side-effect executor for the AIOS campaign control plane."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import base64
|
|
||||||
from dataclasses import dataclass, replace
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Mapping
|
|
||||||
|
|
||||||
from aios_authority_git_executor import AuthorityGitExecutor, AuthorityGitError
|
|
||||||
from aios_campaign_recovery import RecoveryEvidenceError, load_recovered_authorization
|
|
||||||
from aios_gitea_machine_broker import CampaignCapabilityReceipt, GiteaMachineBroker, MachineBrokerError
|
|
||||||
from aios_gitops_job_controller import GitOpsJobController, JobError, JobState, JobStore
|
|
||||||
|
|
||||||
|
|
||||||
TASK_ID = "AIOS_AUTONOMOUS_RELEASE_CAMPAIGN_V1"
|
|
||||||
CAMPAIGN_ID = "AIOS-AUTONOMOUS-RELEASE-CAMPAIGN-20260807"
|
|
||||||
REPOSITORY = "aios/aios"
|
|
||||||
BRANCH = "agent/autonomous-release-campaign-v1-20260807"
|
|
||||||
EXPECTED_BRANCH_OID = "bba675fab3b7e0c5fba7066720252b3c91b9f20d"
|
|
||||||
AUTHORIZATION_DIGEST = "6b0bf550eb702126e967eeff2d561fa808444b924d8a71f5399accad508ed3b2"
|
|
||||||
LINEAGE_ANCHOR = "69b45e81dcd2c3707c367003b08c55fd361faae8"
|
|
||||||
|
|
||||||
|
|
||||||
class JobExecutionError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
|
||||||
class DurableJobExecutor:
|
|
||||||
controller: GitOpsJobController
|
|
||||||
authority: AuthorityGitExecutor
|
|
||||||
broker: GiteaMachineBroker
|
|
||||||
recovery_path: Path
|
|
||||||
|
|
||||||
def _request(self, job_id: str) -> tuple[dict[str, Any], dict[str, Any]]:
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
try:
|
|
||||||
request = json.loads(row["request_json"])
|
|
||||||
except (KeyError, TypeError, json.JSONDecodeError) as exc:
|
|
||||||
raise JobExecutionError("JOB_REQUEST_INVALID") from exc
|
|
||||||
return row, dict(request.get("binding") or {})
|
|
||||||
|
|
||||||
def _validate_campaign(self, request: Mapping[str, Any]) -> str:
|
|
||||||
operation = str(request.get("operation", ""))
|
|
||||||
expected_branch_oid = request.get("expected_branch_oid")
|
|
||||||
required = {
|
|
||||||
"task_id": TASK_ID,
|
|
||||||
"campaign_id": CAMPAIGN_ID,
|
|
||||||
"repository": REPOSITORY,
|
|
||||||
"branch": BRANCH,
|
|
||||||
"authorization_recovery_digest": AUTHORIZATION_DIGEST,
|
|
||||||
}
|
|
||||||
for key, expected in required.items():
|
|
||||||
if request.get(key) != expected:
|
|
||||||
raise JobExecutionError(f"JOB_{key.upper()}_MISMATCH")
|
|
||||||
new_oid = request.get("new_branch_oid")
|
|
||||||
if not isinstance(new_oid, str) or len(new_oid) != 40 or any(ch not in "0123456789abcdef" for ch in new_oid):
|
|
||||||
raise JobExecutionError("NEW_BRANCH_OID_INVALID")
|
|
||||||
if not isinstance(expected_branch_oid, str) or len(expected_branch_oid) != 40 or any(ch not in "0123456789abcdef" for ch in expected_branch_oid):
|
|
||||||
raise JobExecutionError("EXPECTED_BRANCH_OID_INVALID")
|
|
||||||
if operation == "ADVANCE_EXISTING_CAMPAIGN_BRANCH" and expected_branch_oid == new_oid:
|
|
||||||
raise JobExecutionError("EXPECTED_BRANCH_OID_INVALID")
|
|
||||||
if operation != "ADVANCE_EXISTING_CAMPAIGN_BRANCH" and expected_branch_oid != new_oid:
|
|
||||||
raise JobExecutionError("POST_ADVANCE_HEAD_BINDING_INVALID")
|
|
||||||
try:
|
|
||||||
recovery = load_recovered_authorization(self.recovery_path)
|
|
||||||
except RecoveryEvidenceError as exc:
|
|
||||||
raise JobExecutionError(str(exc)) from exc
|
|
||||||
if recovery["digest"] != AUTHORIZATION_DIGEST:
|
|
||||||
raise JobExecutionError("RECOVERY_DIGEST_MISMATCH")
|
|
||||||
for key, expected in required.items():
|
|
||||||
if key == "campaign_id":
|
|
||||||
actual = recovery.get(key)
|
|
||||||
elif key == "authorization_recovery_digest":
|
|
||||||
continue
|
|
||||||
else:
|
|
||||||
actual = recovery.get({"expected_branch_oid": "expected_branch_oid"}.get(key, key))
|
|
||||||
if actual is not None and actual != expected:
|
|
||||||
raise JobExecutionError("RECOVERY_BINDING_MISMATCH")
|
|
||||||
if recovery.get("expected_branch_oid") != EXPECTED_BRANCH_OID:
|
|
||||||
raise JobExecutionError("RECOVERY_BRANCH_OID_MISMATCH")
|
|
||||||
authority_head = self.authority.read_ref(f"refs/heads/{BRANCH}")
|
|
||||||
if authority_head not in {expected_branch_oid, new_oid}:
|
|
||||||
raise JobExecutionError("AUTHORITY_BRANCH_EXPECTATION_MISMATCH")
|
|
||||||
if not self.authority.ancestry_check(expected_branch_oid, new_oid):
|
|
||||||
raise JobExecutionError("NEW_OID_NOT_DESCENDANT")
|
|
||||||
if not self.authority.ancestry_check(EXPECTED_BRANCH_OID, new_oid):
|
|
||||||
raise JobExecutionError("NEW_OID_OUTSIDE_AUTHORIZED_LINEAGE")
|
|
||||||
identity = self.broker.verify_identity()
|
|
||||||
if identity.get("repository") != REPOSITORY or identity.get("login") == "aiman":
|
|
||||||
raise JobExecutionError("MACHINE_IDENTITY_SCOPE_INVALID")
|
|
||||||
return new_oid
|
|
||||||
|
|
||||||
def dry_run(self, job_id: str) -> dict[str, Any]:
|
|
||||||
_row, request = self._request(job_id)
|
|
||||||
new_oid = self._validate_campaign(request)
|
|
||||||
remote = self.broker.readback()
|
|
||||||
if remote is None:
|
|
||||||
action = "CREATE_REQUIRED"
|
|
||||||
elif remote == new_oid:
|
|
||||||
action = "ALREADY_IN_SYNC"
|
|
||||||
elif remote == request["expected_branch_oid"]:
|
|
||||||
action = "FAST_FORWARD_REQUIRED"
|
|
||||||
else:
|
|
||||||
action = "MANUAL_OWNER_REVIEW_REQUIRED"
|
|
||||||
if action == "MANUAL_OWNER_REVIEW_REQUIRED":
|
|
||||||
raise JobExecutionError("GITEA_BRANCH_CONFLICT")
|
|
||||||
return {
|
|
||||||
"decision": "SAFE_TO_ADVANCE",
|
|
||||||
"authority_action": "CAS_REQUIRED",
|
|
||||||
"gitea_action": action,
|
|
||||||
"previous_branch_oid": request["expected_branch_oid"],
|
|
||||||
"final_branch_oid": new_oid,
|
|
||||||
"force_push": False,
|
|
||||||
}
|
|
||||||
|
|
||||||
def _recover_requested(self, job_id: str) -> dict[str, Any]:
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
if JobState(row["state"]) in {JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}:
|
|
||||||
self.controller.resume(job_id)
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
return row
|
|
||||||
|
|
||||||
def execute(self, job_id: str, *, dry_run: bool = False) -> dict[str, Any]:
|
|
||||||
try:
|
|
||||||
return self._execute_unprotected(job_id, dry_run=dry_run)
|
|
||||||
except Exception as exc:
|
|
||||||
try:
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
state = JobState(row["state"])
|
|
||||||
if state not in {JobState.COMPLETED, JobState.MANUAL_OWNER_REVIEW_REQUIRED, JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}:
|
|
||||||
self.controller.advance(job_id, JobState.RECOVERABLE_FAILURE, side_effect_state="UNKNOWN", receipt={"error_code": type(exc).__name__})
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
raise
|
|
||||||
|
|
||||||
def _execute_unprotected(self, job_id: str, *, dry_run: bool = False) -> dict[str, Any]:
|
|
||||||
row, request = self._request(job_id)
|
|
||||||
operation = request.get("operation")
|
|
||||||
if operation == "ADVANCE_EXISTING_CAMPAIGN_BRANCH":
|
|
||||||
if dry_run:
|
|
||||||
return self.dry_run(job_id)
|
|
||||||
return self._advance_campaign(job_id)
|
|
||||||
if operation == "COMMIT_FILE":
|
|
||||||
return self._commit_file(job_id, request)
|
|
||||||
if operation == "SYNC_TASK_BRANCH_TO_GITEA":
|
|
||||||
new_oid = self._validate_campaign(request)
|
|
||||||
final = self.broker.push_branch(new_oid)
|
|
||||||
return {"operation": operation, "remote_verified": final == new_oid, "final_network_oid": final}
|
|
||||||
if operation == "CREATE_PR":
|
|
||||||
self._validate_campaign(request)
|
|
||||||
return dict(self.broker.create_pr("AIOS autonomous release control plane", "Durable campaign continuation; exact head and recovery evidence are bound in the Job."))
|
|
||||||
if operation == "PUBLISH_CHECK":
|
|
||||||
self._validate_campaign(request)
|
|
||||||
context = request.get("check_context", "aios/durable-release")
|
|
||||||
state = request.get("check_state", "success")
|
|
||||||
description = request.get("check_description", "Durable campaign control-plane check passed")
|
|
||||||
return dict(self.broker.status(str(state), str(description), str(context)))
|
|
||||||
if operation == "PROTECTED_MERGE":
|
|
||||||
self._validate_campaign(request)
|
|
||||||
number = int(request.get("pr_number", "0"))
|
|
||||||
self.broker.transport.expected_merge_head = request["expected_branch_oid"]
|
|
||||||
expected_main = request.get("expected_main_oid")
|
|
||||||
if expected_main is not None:
|
|
||||||
self.broker.transport.expected_merge_base = expected_main
|
|
||||||
return dict(self.broker.merge(number))
|
|
||||||
if operation == "RECONCILE_AUTHORITY_MAIN":
|
|
||||||
self._validate_campaign(request)
|
|
||||||
old = request.get("expected_main_oid")
|
|
||||||
new = request.get("new_branch_oid")
|
|
||||||
if not isinstance(old, str) or not isinstance(new, str):
|
|
||||||
raise JobExecutionError("MAIN_RECONCILIATION_BINDING_INVALID")
|
|
||||||
gitea_main = self.broker.transport.read_branch("main")
|
|
||||||
if gitea_main != new:
|
|
||||||
raise JobExecutionError("GITEA_MAIN_EXPECTATION_MISMATCH")
|
|
||||||
if not self.authority.ancestry_check(old, new):
|
|
||||||
raise JobExecutionError("MAIN_OID_NOT_DESCENDANT")
|
|
||||||
return {"authority_main": self.authority.ff_CAS_main_after_verified_protected_merge(old, new)}
|
|
||||||
raise JobExecutionError("OPERATION_NOT_ALLOWED")
|
|
||||||
|
|
||||||
def _advance_campaign(self, job_id: str) -> dict[str, Any]:
|
|
||||||
self._recover_requested(job_id)
|
|
||||||
row, request = self._request(job_id)
|
|
||||||
new_oid = self._validate_campaign(request)
|
|
||||||
expected_oid = request["expected_branch_oid"]
|
|
||||||
state = JobState(row["state"])
|
|
||||||
if state == JobState.REQUESTED:
|
|
||||||
receipt = self.controller.advance(job_id, JobState.AUTHORIZED, side_effect_state="AUTHORIZED", receipt={"authorization_digest": AUTHORIZATION_DIGEST, "task_id": TASK_ID})
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
if JobState(row["state"]) == JobState.AUTHORIZED:
|
|
||||||
self.controller.lease_job(job_id, lease_id=f"lease-{job_id}", lineage_anchor_oid=LINEAGE_ANCHOR, current_head_oid=expected_oid, generation=1, lease_expires_at="2099-01-01T00:00:00Z", expected_revision=int(row["revision"]))
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
if JobState(row["state"]) == JobState.LEASED:
|
|
||||||
self.controller.advance(job_id, JobState.WORKTREE_READY, side_effect_state="WORKTREE_READY", receipt={"branch": BRANCH, "expected_oid": expected_oid})
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
if JobState(row["state"]) == JobState.WORKTREE_READY:
|
|
||||||
self.controller.advance(job_id, JobState.EXECUTING, side_effect_state="IN_PROGRESS")
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
authority_before = self.authority.read_ref(f"refs/heads/{BRANCH}")
|
|
||||||
if authority_before == expected_oid:
|
|
||||||
authority_oid = self.authority.advance_task_branch_CAS(f"refs/heads/{BRANCH}", expected_oid, new_oid)
|
|
||||||
elif authority_before == new_oid:
|
|
||||||
authority_oid = new_oid
|
|
||||||
else:
|
|
||||||
raise JobExecutionError("AUTHORITY_BRANCH_CONFLICT")
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
if JobState(row["state"]) == JobState.EXECUTING:
|
|
||||||
self.controller.advance(job_id, JobState.LOCAL_VERIFIED, side_effect_state="LOCAL_VERIFIED", receipt={"authority": {"branch": BRANCH, "previous_oid": expected_oid, "new_oid": new_oid, "CAS_success": True, "readback_oid": authority_oid}})
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
remote_before = self.broker.readback()
|
|
||||||
if remote_before != new_oid:
|
|
||||||
if remote_before not in {None, expected_oid}:
|
|
||||||
raise JobExecutionError("GITEA_BRANCH_CONFLICT")
|
|
||||||
if JobState(row["state"]) == JobState.LOCAL_VERIFIED:
|
|
||||||
if hasattr(self.broker, "receipt"):
|
|
||||||
self.broker.receipt = replace(self.broker.receipt, expected_oid=expected_oid)
|
|
||||||
self.controller.advance(job_id, JobState.REMOTE_SYNCING, side_effect_state="IN_PROGRESS", receipt={"gitea": {"previous_oid": remote_before, "desired_oid": new_oid, "force_push": False}})
|
|
||||||
final_network = self.broker.push_branch(new_oid)
|
|
||||||
else:
|
|
||||||
final_network = new_oid
|
|
||||||
final_api = self.broker.readback()
|
|
||||||
if final_api != new_oid:
|
|
||||||
raise JobExecutionError("GITEA_BRANCH_READBACK_MISMATCH")
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
if JobState(row["state"]) in {JobState.LOCAL_VERIFIED, JobState.REMOTE_SYNCING}:
|
|
||||||
self.controller.advance(job_id, JobState.REMOTE_VERIFIED, side_effect_state="REMOTE_VERIFIED", receipt={"gitea": {"machine_actor": "aios-autonomy-agent", "previous_oid": remote_before, "desired_oid": new_oid, "final_network_oid": final_network, "final_API_oid": final_api, "force_push": False}})
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
if JobState(row["state"]) == JobState.REMOTE_VERIFIED:
|
|
||||||
self.controller.advance(job_id, JobState.COMPLETED, side_effect_state="REMOTE_VERIFIED", receipt={"authority_oid": authority_oid, "network_oid": final_network, "gitea_api_oid": final_api, "force_push": False})
|
|
||||||
return self.controller.store._receipt(self.controller.store.read(job_id), duplicate=True).as_dict()
|
|
||||||
|
|
||||||
def _commit_file(self, job_id: str, request: Mapping[str, Any]) -> dict[str, Any]:
|
|
||||||
row = self.controller.store.read(job_id)
|
|
||||||
payload_ref = request.get("payload_ref")
|
|
||||||
if not payload_ref:
|
|
||||||
raise JobExecutionError("PAYLOAD_REQUIRED")
|
|
||||||
payload = self.controller.store.payloads.read_verified(payload_ref, row["payload_digest"], int(request["payload_size"]))
|
|
||||||
try:
|
|
||||||
spec = json.loads(payload.decode())
|
|
||||||
path = str(spec["path"])
|
|
||||||
content = base64.b64decode(spec["content_base64"], validate=True)
|
|
||||||
except Exception as exc:
|
|
||||||
raise JobExecutionError("COMMIT_FILE_PAYLOAD_INVALID") from exc
|
|
||||||
new_oid = self.authority.commit_file_in_isolated_worktree(base_oid=row["expected_branch_oid"], files={path: content}, message="chore(gitops): durable commit-file", worktree_name=job_id)
|
|
||||||
return {"operation": "COMMIT_FILE", "new_oid": new_oid}
|
|
||||||
|
|
||||||
|
|
||||||
def build_from_environment(job_root: Path) -> DurableJobExecutor:
|
|
||||||
from aios_gitea_machine_transport import GiteaMachineTransport
|
|
||||||
from aios_gitea_machine_broker import MachineCredentialProjection
|
|
||||||
|
|
||||||
account = __import__("os").environ.get("AIOS_GITEA_MACHINE_ACCOUNT", "aios-autonomy-agent")
|
|
||||||
api = __import__("os").environ.get("AIOS_GITEA_API", "https://g.cnnetmkt.top/git/api/v1")
|
|
||||||
remote = __import__("os").environ.get("AIOS_GITEA_REMOTE", "https://g.cnnetmkt.top/git/aios/aios.git")
|
|
||||||
authority_dir = Path(__import__("os").environ.get("AIOS_GITOPS_AUTHORITY_GIT_DIR", "/data/aios-gitops/repo.git"))
|
|
||||||
worktree_root = Path(__import__("os").environ.get("AIOS_GITOPS_WORKTREE_ROOT", "/var/lib/aios-gitops/worktrees"))
|
|
||||||
recovery = Path(__import__("os").environ.get("AIOS_RELEASE_RECOVERY_RECEIPT", "/var/lib/aios-gitops/release-campaigns/AIOS-AUTONOMOUS-RELEASE-CAMPAIGN-20260807/authorization-recovery.json"))
|
|
||||||
transport = GiteaMachineTransport(api, REPOSITORY, remote, authority_dir)
|
|
||||||
credential = MachineCredentialProjection.from_mapping({"scheme": "systemd-credential", "name": "gitea-machine-token", "account": account})
|
|
||||||
receipt = CampaignCapabilityReceipt(CAMPAIGN_ID, TASK_ID, REPOSITORY, BRANCH, EXPECTED_BRANCH_OID, frozenset({"exact_task_branch_push", "pr_create_update", "status_publish", "protected_merge", "remote_readback"}), frozenset({"direct_main_push", "force_push", "tag_update", "tag_delete", "baseline_write", "production_deploy"}))
|
|
||||||
return DurableJobExecutor(GitOpsJobController(JobStore(job_root)), AuthorityGitExecutor(authority_dir, worktree_root), GiteaMachineBroker(transport, credential, receipt), recovery)
|
|
||||||
|
|
||||||
|
|
||||||
def main(argv: list[str] | None = None) -> int:
|
|
||||||
parser = argparse.ArgumentParser()
|
|
||||||
parser.add_argument("--job-root", required=True, type=Path)
|
|
||||||
parser.add_argument("--job-id", required=True)
|
|
||||||
parser.add_argument("--dry-run", action="store_true")
|
|
||||||
args = parser.parse_args(argv)
|
|
||||||
result = build_from_environment(args.job_root).execute(args.job_id, dry_run=args.dry_run)
|
|
||||||
print(json.dumps(result, sort_keys=True))
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -1,80 +0,0 @@
|
|||||||
"""Pure, observation-first decisions for governed GitOps reconciliation."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from enum import Enum
|
|
||||||
import re
|
|
||||||
from typing import Optional
|
|
||||||
|
|
||||||
OID_RE = re.compile(r"^[0-9a-f]{40}$")
|
|
||||||
|
|
||||||
|
|
||||||
class ReconciliationAction(str, Enum):
|
|
||||||
NOOP = "NOOP"
|
|
||||||
PUSH_EXACT = "PUSH_EXACT"
|
|
||||||
VERIFY_REMOTE = "VERIFY_REMOTE"
|
|
||||||
MARK_REMOTE_VERIFIED = "MARK_REMOTE_VERIFIED"
|
|
||||||
MARK_RECONCILIATION_REQUIRED = "MARK_RECONCILIATION_REQUIRED"
|
|
||||||
FAIL_CONFLICT = "FAIL_CONFLICT"
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class RemoteObservation:
|
|
||||||
repository_identity: str
|
|
||||||
branch: str
|
|
||||||
desired_oid: str
|
|
||||||
authority_oid: Optional[str]
|
|
||||||
remote_git_oid: Optional[str]
|
|
||||||
remote_api_oid: Optional[str]
|
|
||||||
commit_visible: bool
|
|
||||||
observed_at: str
|
|
||||||
expected_oid: Optional[str] = None
|
|
||||||
|
|
||||||
@property
|
|
||||||
def all_equal(self) -> bool:
|
|
||||||
values = (self.authority_oid, self.remote_git_oid, self.remote_api_oid, self.desired_oid)
|
|
||||||
return all(isinstance(value, str) for value in values) and len(set(values)) == 1
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class ReconciliationDecision:
|
|
||||||
action: ReconciliationAction
|
|
||||||
state: str
|
|
||||||
reason: str
|
|
||||||
retry_safe: bool = False
|
|
||||||
|
|
||||||
|
|
||||||
def _valid_oid(value: Optional[str]) -> bool:
|
|
||||||
return value is None or bool(OID_RE.fullmatch(value))
|
|
||||||
|
|
||||||
|
|
||||||
def decide_reconciliation(observation: RemoteObservation) -> ReconciliationDecision:
|
|
||||||
"""Decide from a fresh observation; never complete on a missing remote."""
|
|
||||||
if observation.repository_identity != "aios/aios":
|
|
||||||
return ReconciliationDecision(ReconciliationAction.FAIL_CONFLICT, "CONFLICT", "REPOSITORY_IDENTITY_MISMATCH")
|
|
||||||
if not observation.branch.startswith("agent/"):
|
|
||||||
return ReconciliationDecision(ReconciliationAction.FAIL_CONFLICT, "CONFLICT", "BRANCH_SCOPE_DENIED")
|
|
||||||
if not _valid_oid(observation.desired_oid) or not _valid_oid(observation.expected_oid):
|
|
||||||
return ReconciliationDecision(ReconciliationAction.FAIL_CONFLICT, "CONFLICT", "OID_INVALID")
|
|
||||||
if observation.authority_oid is None:
|
|
||||||
return ReconciliationDecision(ReconciliationAction.MARK_RECONCILIATION_REQUIRED, "RECONCILIATION_REQUIRED", "AUTHORITY_UNAVAILABLE")
|
|
||||||
if observation.authority_oid != observation.desired_oid:
|
|
||||||
return ReconciliationDecision(ReconciliationAction.MARK_RECONCILIATION_REQUIRED, "RECONCILIATION_REQUIRED", "AUTHORITY_CONFLICT")
|
|
||||||
present = [oid for oid in (observation.remote_git_oid, observation.remote_api_oid) if oid is not None]
|
|
||||||
if any(oid not in {observation.desired_oid, observation.expected_oid} for oid in present):
|
|
||||||
return ReconciliationDecision(ReconciliationAction.FAIL_CONFLICT, "CONFLICT", "REMOTE_OID_CONFLICT")
|
|
||||||
if observation.all_equal and observation.commit_visible:
|
|
||||||
return ReconciliationDecision(ReconciliationAction.MARK_REMOTE_VERIFIED, "REMOTE_VERIFIED", "ALL_OIDS_EQUAL_AND_VISIBLE")
|
|
||||||
if observation.remote_git_oid is None and observation.remote_api_oid is None:
|
|
||||||
return ReconciliationDecision(ReconciliationAction.MARK_RECONCILIATION_REQUIRED, "REMOTE_PUSH_PENDING", "AUTHORITY_DESIRED_REMOTE_MISSING")
|
|
||||||
if observation.remote_git_oid == observation.desired_oid or observation.remote_api_oid == observation.desired_oid:
|
|
||||||
return ReconciliationDecision(ReconciliationAction.VERIFY_REMOTE, "REMOTE_VERIFY_PENDING", "REMOTE_PROOF_PARTIAL")
|
|
||||||
if observation.expected_oid is not None and observation.remote_git_oid == observation.expected_oid and observation.remote_api_oid == observation.expected_oid:
|
|
||||||
return ReconciliationDecision(ReconciliationAction.PUSH_EXACT, "REMOTE_PUSH_PENDING", "SAFE_EXACT_FAST_FORWARD_RETRY", retry_safe=True)
|
|
||||||
if not observation.commit_visible:
|
|
||||||
return ReconciliationDecision(ReconciliationAction.VERIFY_REMOTE, "COMMIT_VISIBILITY_PENDING", "COMMIT_VISIBILITY_LAG")
|
|
||||||
return ReconciliationDecision(ReconciliationAction.MARK_RECONCILIATION_REQUIRED, "RECONCILIATION_REQUIRED", "PARTIAL_REMOTE_OBSERVATION")
|
|
||||||
|
|
||||||
|
|
||||||
def reconcile_observation(**kwargs: object) -> ReconciliationDecision:
|
|
||||||
return decide_reconciliation(RemoteObservation(**kwargs))
|
|
||||||
@@ -1,100 +0,0 @@
|
|||||||
"""Single supervised socket-activated worker for durable GitOps Jobs.
|
|
||||||
|
|
||||||
The worker consumes durable requests and reconciliation queries. Repository
|
|
||||||
mutation is a separate injected executor and remains impossible while the
|
|
||||||
legacy global write gate is false.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import signal
|
|
||||||
import socket
|
|
||||||
import threading
|
|
||||||
from typing import Any, Mapping
|
|
||||||
|
|
||||||
from aios_gitops_job_controller import GitOpsJobController, JobError, JobState, JobStore, ServiceUnavailable
|
|
||||||
|
|
||||||
|
|
||||||
def runtime_policy(environment: Mapping[str, str] | None = None) -> dict[str, bool]:
|
|
||||||
env = environment or os.environ
|
|
||||||
legacy = env.get("AIOS_GITOPS_WRITES_ENABLED", "false").lower() in {"1", "true", "yes", "on"}
|
|
||||||
durable = env.get("AIOS_GITOPS_DURABLE_JOB_EXECUTION_ENABLED", "false").lower() in {"1", "true", "yes", "on"}
|
|
||||||
# Legacy direct Git mutation remains disabled. Authorized durable Jobs use
|
|
||||||
# their own campaign/task capability and may mutate only through the
|
|
||||||
# worker's lease/CAS path while the legacy gate stays false.
|
|
||||||
return {"legacy_global_write": legacy, "durable_job_execution": durable, "repository_mutation_allowed": durable}
|
|
||||||
|
|
||||||
|
|
||||||
def handle_request(controller: GitOpsJobController, request: Mapping[str, Any], executor: Any | None = None) -> dict[str, Any]:
|
|
||||||
job_id = str(request.get("job_id") or "")
|
|
||||||
if not job_id:
|
|
||||||
raise JobError("JOB_ID_REQUIRED")
|
|
||||||
if request.get("action") == "execute":
|
|
||||||
if executor is None:
|
|
||||||
raise JobError("JOB_EXECUTOR_UNAVAILABLE")
|
|
||||||
result = executor.execute(job_id, dry_run=bool(request.get("dry_run", False)))
|
|
||||||
receipt = controller.store._receipt(controller.store.read(job_id), duplicate=True)
|
|
||||||
return {"job": result, **receipt.as_dict()}
|
|
||||||
if request.get("action") == "resume":
|
|
||||||
receipt = controller.resume(job_id)
|
|
||||||
else:
|
|
||||||
row = controller.store.read(job_id)
|
|
||||||
receipt = controller.store._receipt(row, duplicate=True)
|
|
||||||
return receipt.as_dict()
|
|
||||||
|
|
||||||
|
|
||||||
def serve_socket(sock: socket.socket, controller: GitOpsJobController, stopping: threading.Event, executor: Any | None = None) -> None:
|
|
||||||
sock.listen(32)
|
|
||||||
sock.settimeout(1.0)
|
|
||||||
while not stopping.is_set():
|
|
||||||
try:
|
|
||||||
connection, _ = sock.accept()
|
|
||||||
except socket.timeout:
|
|
||||||
continue
|
|
||||||
with connection:
|
|
||||||
try:
|
|
||||||
raw = connection.makefile("rb").readline(1_048_576)
|
|
||||||
request = json.loads(raw.decode("utf-8"))
|
|
||||||
response = {"status": "ok", **handle_request(controller, request, executor)}
|
|
||||||
except ServiceUnavailable as exc:
|
|
||||||
response = {"status": "error", "error": str(exc), "retryable": True, "side_effect_state": "UNKNOWN"}
|
|
||||||
except Exception as exc:
|
|
||||||
response = {"status": "error", "error": type(exc).__name__, "retryable": False}
|
|
||||||
connection.sendall((json.dumps(response, sort_keys=True) + "\n").encode("utf-8"))
|
|
||||||
|
|
||||||
|
|
||||||
def socket_from_systemd() -> socket.socket:
|
|
||||||
if os.environ.get("LISTEN_FDS") != "1":
|
|
||||||
raise RuntimeError("SOCKET_ACTIVATION_REQUIRED")
|
|
||||||
return socket.fromfd(3, socket.AF_UNIX, socket.SOCK_STREAM)
|
|
||||||
|
|
||||||
|
|
||||||
def main(argv: list[str] | None = None) -> int:
|
|
||||||
parser = argparse.ArgumentParser()
|
|
||||||
parser.add_argument("--job-root", required=True, type=Path)
|
|
||||||
args = parser.parse_args(argv)
|
|
||||||
policy = runtime_policy()
|
|
||||||
if policy["legacy_global_write"]:
|
|
||||||
raise RuntimeError("LEGACY_GLOBAL_WRITES_MUST_REMAIN_FALSE")
|
|
||||||
if not policy["durable_job_execution"]:
|
|
||||||
raise RuntimeError("DURABLE_JOB_EXECUTION_GATE_CLOSED")
|
|
||||||
controller = GitOpsJobController(JobStore(args.job_root))
|
|
||||||
from aios_gitops_job_executor import build_from_environment
|
|
||||||
executor = build_from_environment(args.job_root)
|
|
||||||
stopping = threading.Event()
|
|
||||||
signal.signal(signal.SIGTERM, lambda *_: stopping.set())
|
|
||||||
signal.signal(signal.SIGINT, lambda *_: stopping.set())
|
|
||||||
sock = socket_from_systemd()
|
|
||||||
try:
|
|
||||||
serve_socket(sock, controller, stopping, executor)
|
|
||||||
finally:
|
|
||||||
sock.close()
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -1,69 +0,0 @@
|
|||||||
"""CLI client for the governed local writer service."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import socket
|
|
||||||
import sys
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_args() -> argparse.Namespace:
|
|
||||||
parser = argparse.ArgumentParser(description="Send one governed-local-writer request")
|
|
||||||
parser.add_argument("--socket", required=True, help="AF_UNIX socket path")
|
|
||||||
group = parser.add_mutually_exclusive_group(required=True)
|
|
||||||
group.add_argument("--request", help="Request JSON string")
|
|
||||||
group.add_argument("--request-file", help="Path to JSON request file")
|
|
||||||
return parser.parse_args()
|
|
||||||
|
|
||||||
|
|
||||||
def _load_request_text(args: argparse.Namespace) -> str:
|
|
||||||
if args.request is not None:
|
|
||||||
return args.request
|
|
||||||
request_path = os.fspath(args.request_file)
|
|
||||||
with open(request_path, "r", encoding="utf-8") as request_file:
|
|
||||||
return request_file.read()
|
|
||||||
|
|
||||||
|
|
||||||
def send_request(socket_path: str, request_text: str) -> dict[str, object]:
|
|
||||||
"""Open one Unix socket, send one NDJSON request, and parse one NDJSON response."""
|
|
||||||
payload = json.loads(request_text)
|
|
||||||
if not isinstance(payload, dict):
|
|
||||||
raise ValueError("request must be a JSON object")
|
|
||||||
|
|
||||||
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client:
|
|
||||||
client.connect(socket_path)
|
|
||||||
client.sendall((json.dumps(payload) + "\n").encode("utf-8"))
|
|
||||||
response_chunks: list[str] = []
|
|
||||||
while True:
|
|
||||||
chunk = client.recv(1024 * 1024).decode("utf-8")
|
|
||||||
if not chunk:
|
|
||||||
break
|
|
||||||
response_chunks.append(chunk)
|
|
||||||
if "\n" in chunk:
|
|
||||||
break
|
|
||||||
response_text = "".join(response_chunks)
|
|
||||||
|
|
||||||
response_line = response_text.split("\n", 1)[0].strip()
|
|
||||||
if not response_line:
|
|
||||||
raise RuntimeError("empty response")
|
|
||||||
response = json.loads(response_line)
|
|
||||||
if not isinstance(response, dict):
|
|
||||||
raise RuntimeError("invalid response")
|
|
||||||
return response
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
|
||||||
args = _parse_args()
|
|
||||||
request_text = _load_request_text(args)
|
|
||||||
response = send_request(args.socket, request_text)
|
|
||||||
print(json.dumps(response, separators=(",", ":"), sort_keys=True))
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
try:
|
|
||||||
main()
|
|
||||||
except Exception as exc: # pragma: no cover - CLI entrypoint behavior
|
|
||||||
print(json.dumps({"status": "error", "error": str(exc)}), file=sys.stderr)
|
|
||||||
raise SystemExit(1)
|
|
||||||
@@ -1,931 +0,0 @@
|
|||||||
"""Unix-socket service for governed local writer operations."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import hashlib
|
|
||||||
import argparse
|
|
||||||
import importlib.util
|
|
||||||
import os
|
|
||||||
import re
|
|
||||||
import socket
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import tempfile
|
|
||||||
import uuid
|
|
||||||
from dataclasses import dataclass, field
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Callable, Mapping, Protocol
|
|
||||||
|
|
||||||
from governed_local_writer import (
|
|
||||||
AppendCommitRequest,
|
|
||||||
AuthorizationError,
|
|
||||||
BranchPolicy,
|
|
||||||
CreateBranchRequest,
|
|
||||||
GovernedAuthorization,
|
|
||||||
PeerIdentity,
|
|
||||||
PRINCIPAL,
|
|
||||||
TASK_ID,
|
|
||||||
extract_peer_identity,
|
|
||||||
contains_secret_like_content,
|
|
||||||
load_authorization_file,
|
|
||||||
validate_file_changes,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class ExistingGitOpsHandler:
|
|
||||||
"""Delegate repository mutations to the installed GitOps handler seam."""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
workspace_by_branch: Mapping[str, str | os.PathLike[str]],
|
|
||||||
bare_repo: str | os.PathLike[str],
|
|
||||||
api_source: str | os.PathLike[str],
|
|
||||||
) -> None:
|
|
||||||
self.workspace_by_branch = {str(k): Path(v) for k, v in workspace_by_branch.items()}
|
|
||||||
self.bare_repo = Path(bare_repo)
|
|
||||||
self.api_source = Path(api_source)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _restore_env(previous: Mapping[str, str | None]) -> None:
|
|
||||||
for key, value in previous.items():
|
|
||||||
if value is None:
|
|
||||||
os.environ.pop(key, None)
|
|
||||||
else:
|
|
||||||
os.environ[key] = value
|
|
||||||
|
|
||||||
def _load_api(self, workspace: Path):
|
|
||||||
if not self.api_source.is_file() or self.api_source.is_symlink():
|
|
||||||
raise RuntimeError("installed GitOps handler source is unavailable")
|
|
||||||
keys = (
|
|
||||||
"AIOS_GITOPS_WORKSPACE",
|
|
||||||
"AIOS_GITOPS_BARE_REPO",
|
|
||||||
"AIOS_GITOPS_WRITES_ENABLED",
|
|
||||||
"PYTHONDONTWRITEBYTECODE",
|
|
||||||
"GIT_CONFIG_COUNT",
|
|
||||||
"GIT_CONFIG_KEY_0",
|
|
||||||
"GIT_CONFIG_VALUE_0",
|
|
||||||
"GIT_CONFIG_KEY_1",
|
|
||||||
"GIT_CONFIG_VALUE_1",
|
|
||||||
)
|
|
||||||
previous = {key: os.environ.get(key) for key in keys}
|
|
||||||
os.environ.update(
|
|
||||||
{
|
|
||||||
"AIOS_GITOPS_WORKSPACE": str(workspace),
|
|
||||||
"AIOS_GITOPS_BARE_REPO": str(self.bare_repo),
|
|
||||||
"AIOS_GITOPS_WRITES_ENABLED": "true",
|
|
||||||
"PYTHONDONTWRITEBYTECODE": "1",
|
|
||||||
"GIT_CONFIG_COUNT": "2",
|
|
||||||
"GIT_CONFIG_KEY_0": "safe.directory",
|
|
||||||
"GIT_CONFIG_VALUE_0": str(workspace),
|
|
||||||
"GIT_CONFIG_KEY_1": "safe.directory",
|
|
||||||
"GIT_CONFIG_VALUE_1": str(self.bare_repo),
|
|
||||||
}
|
|
||||||
)
|
|
||||||
module_name = f"_governed_gitops_api_{uuid.uuid4().hex}"
|
|
||||||
try:
|
|
||||||
spec = importlib.util.spec_from_file_location(module_name, self.api_source)
|
|
||||||
if spec is None or spec.loader is None:
|
|
||||||
raise RuntimeError("unable to load installed GitOps handler")
|
|
||||||
module = importlib.util.module_from_spec(spec)
|
|
||||||
sys.modules[module_name] = module
|
|
||||||
spec.loader.exec_module(module)
|
|
||||||
decision = module.evaluate_git_write(module.WS)
|
|
||||||
if not decision.allowed:
|
|
||||||
raise RuntimeError("existing GitOps write gate denied bootstrap")
|
|
||||||
return module, previous, module_name
|
|
||||||
except Exception:
|
|
||||||
sys.modules.pop(module_name, None)
|
|
||||||
self._restore_env(previous)
|
|
||||||
raise
|
|
||||||
|
|
||||||
def _workspace(self, branch: str) -> Path:
|
|
||||||
workspace = self.workspace_by_branch.get(branch)
|
|
||||||
if workspace is None or not workspace.is_dir():
|
|
||||||
raise RuntimeError("branch workspace is not authorized")
|
|
||||||
return workspace
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _git(workspace: Path, args: list[str]) -> str:
|
|
||||||
result = subprocess.run(
|
|
||||||
["git", *args], cwd=workspace, text=True, capture_output=True, check=False
|
|
||||||
)
|
|
||||||
if result.returncode != 0:
|
|
||||||
raise RuntimeError("governed handler git probe failed")
|
|
||||||
return result.stdout.strip()
|
|
||||||
|
|
||||||
def create_branch(self, request: CreateBranchRequest) -> Mapping[str, object]:
|
|
||||||
workspace = self._workspace(request.branch)
|
|
||||||
# A prior handler call may have completed the fast-forward sync before
|
|
||||||
# the process observed a post-commit error. Reconcile only an exact,
|
|
||||||
# metadata-only result; never overwrite or move an existing ref here.
|
|
||||||
try:
|
|
||||||
existing = self._git(
|
|
||||||
workspace,
|
|
||||||
[
|
|
||||||
"--git-dir",
|
|
||||||
str(self.bare_repo),
|
|
||||||
"rev-parse",
|
|
||||||
f"refs/heads/{request.branch}",
|
|
||||||
],
|
|
||||||
)
|
|
||||||
except RuntimeError:
|
|
||||||
existing = ""
|
|
||||||
if existing:
|
|
||||||
parent = self._git(workspace, ["--git-dir", str(self.bare_repo), "rev-parse", f"{existing}^1"])
|
|
||||||
changed = self._git(
|
|
||||||
workspace,
|
|
||||||
[
|
|
||||||
"--git-dir",
|
|
||||||
str(self.bare_repo),
|
|
||||||
"diff-tree",
|
|
||||||
"--no-commit-id",
|
|
||||||
"--name-only",
|
|
||||||
"-r",
|
|
||||||
existing,
|
|
||||||
],
|
|
||||||
).splitlines()
|
|
||||||
safe_branch = re.sub(r"[^A-Za-z0-9_.-]+", "-", request.branch).strip("-")
|
|
||||||
if parent != request.base_oid or changed != [f"patches/{safe_branch}.branch.json"]:
|
|
||||||
raise RuntimeError("existing branch is not an exact metadata-only bootstrap")
|
|
||||||
return {"head_oid": existing, "parent_oid": parent, "handler_status": "reconciled"}
|
|
||||||
module, previous, module_name = self._load_api(workspace)
|
|
||||||
phase = "loaded"
|
|
||||||
try:
|
|
||||||
phase = "verify_base"
|
|
||||||
parent = self._git(
|
|
||||||
workspace,
|
|
||||||
["--git-dir", str(self.bare_repo), "rev-parse", "refs/heads/main"],
|
|
||||||
)
|
|
||||||
if parent != request.base_oid:
|
|
||||||
raise RuntimeError("authorized base OID drift")
|
|
||||||
phase = "checkout"
|
|
||||||
module.checkout(request.branch, base=request.base_oid, create=True)
|
|
||||||
phase = "write_metadata"
|
|
||||||
safe_branch = re.sub(r"[^A-Za-z0-9_.-]+", "-", request.branch).strip("-")
|
|
||||||
metadata_path = f"patches/{safe_branch}.branch.json"
|
|
||||||
module.write_json(
|
|
||||||
metadata_path,
|
|
||||||
{
|
|
||||||
"schema": "aios.governed-local-writer-bootstrap.v1",
|
|
||||||
"task_id": TASK_ID,
|
|
||||||
"principal": PRINCIPAL,
|
|
||||||
"branch": request.branch,
|
|
||||||
"parent_oid": request.base_oid,
|
|
||||||
"authorization_sha256": request.authorization_digest,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
phase = "commit_and_sync"
|
|
||||||
result = module.commit(
|
|
||||||
[metadata_path], "chore(governance): bootstrap repair lease branch"
|
|
||||||
)
|
|
||||||
head_oid = self._git(workspace, ["rev-parse", "HEAD"])
|
|
||||||
first_parent = self._git(workspace, ["rev-parse", "HEAD^1"])
|
|
||||||
if first_parent != request.base_oid or (request.head_oid and head_oid != request.head_oid):
|
|
||||||
raise RuntimeError("bootstrap branch lineage check failed")
|
|
||||||
return {
|
|
||||||
"head_oid": head_oid,
|
|
||||||
"parent_oid": first_parent,
|
|
||||||
"handler_status": str(result.get("status")),
|
|
||||||
}
|
|
||||||
except Exception as exc:
|
|
||||||
raise RuntimeError(f"handler_phase={phase};error={exc.__class__.__name__}") from exc
|
|
||||||
finally:
|
|
||||||
sys.modules.pop(module_name, None)
|
|
||||||
self._restore_env(previous)
|
|
||||||
|
|
||||||
def append_commit(self, request: AppendCommitRequest) -> Mapping[str, object]:
|
|
||||||
workspace = self._workspace(request.branch)
|
|
||||||
module, previous, module_name = self._load_api(workspace)
|
|
||||||
phase = "loaded"
|
|
||||||
try:
|
|
||||||
phase = "verify_head"
|
|
||||||
current = self._git(workspace, ["rev-parse", "HEAD"])
|
|
||||||
if current != request.expected_head_oid:
|
|
||||||
raise RuntimeError("branch head advanced or is stale")
|
|
||||||
phase = "checkout"
|
|
||||||
module.checkout(request.branch, base=request.expected_head_oid, create=False)
|
|
||||||
phase = "write_files"
|
|
||||||
paths: list[str] = []
|
|
||||||
for change in request.changes:
|
|
||||||
paths.append(module.write_file(change.path, change.content))
|
|
||||||
phase = "commit_and_sync"
|
|
||||||
result = module.commit(paths, request.commit_message)
|
|
||||||
head_oid = self._git(workspace, ["rev-parse", "HEAD"])
|
|
||||||
first_parent = self._git(workspace, ["rev-parse", "HEAD^1"])
|
|
||||||
if first_parent != request.expected_head_oid or head_oid == first_parent:
|
|
||||||
raise RuntimeError("append commit lineage check failed")
|
|
||||||
return {
|
|
||||||
"head_oid": head_oid,
|
|
||||||
"parent_oid": first_parent,
|
|
||||||
"handler_status": str(result.get("status")),
|
|
||||||
}
|
|
||||||
except Exception as exc:
|
|
||||||
raise RuntimeError(f"handler_phase={phase};error={exc.__class__.__name__}") from exc
|
|
||||||
finally:
|
|
||||||
sys.modules.pop(module_name, None)
|
|
||||||
self._restore_env(previous)
|
|
||||||
|
|
||||||
|
|
||||||
class GovernedWriterHandler(Protocol):
|
|
||||||
"""Small adapter interface for real writer backend handlers."""
|
|
||||||
|
|
||||||
def create_branch(self, request: CreateBranchRequest) -> Mapping[str, object]:
|
|
||||||
"""Create a branch using internal repo services."""
|
|
||||||
|
|
||||||
def append_commit(self, request: AppendCommitRequest) -> Mapping[str, object]:
|
|
||||||
"""Append a commit using internal repo services."""
|
|
||||||
|
|
||||||
|
|
||||||
class AuditSink(Protocol):
|
|
||||||
"""Pluggable sink for sanitized audit records."""
|
|
||||||
|
|
||||||
def append(self, record: "AuditRecord") -> None:
|
|
||||||
"""Persist one audit record."""
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class AuditRecord:
|
|
||||||
"""Sanitized audit log entry."""
|
|
||||||
|
|
||||||
authorization_digest: str
|
|
||||||
peer_pid: int
|
|
||||||
peer_uid: int
|
|
||||||
peer_gid: int
|
|
||||||
peer_executable: str | None
|
|
||||||
timestamp: datetime
|
|
||||||
operation: str
|
|
||||||
branch: str
|
|
||||||
result: str
|
|
||||||
|
|
||||||
|
|
||||||
class InMemoryAuditSink:
|
|
||||||
"""Simple in-memory audit log used by tests."""
|
|
||||||
|
|
||||||
def __init__(self) -> None:
|
|
||||||
self.records: list[AuditRecord] = []
|
|
||||||
|
|
||||||
def append(self, record: AuditRecord) -> None:
|
|
||||||
self.records.append(record)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class _AuthorizationState:
|
|
||||||
"""Tracks one-shot operation consumption."""
|
|
||||||
|
|
||||||
create_done: set[str] = field(default_factory=set)
|
|
||||||
append_done: set[str] = field(default_factory=set)
|
|
||||||
consumed: bool = False
|
|
||||||
|
|
||||||
|
|
||||||
class GovernedLocalWriterService:
|
|
||||||
"""Policy-aware AF_UNIX service that calls an injected handler."""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
socket_path: str | os.PathLike[str],
|
|
||||||
authorization: GovernedAuthorization,
|
|
||||||
handler: GovernedWriterHandler,
|
|
||||||
*,
|
|
||||||
audit_sink: AuditSink | None = None,
|
|
||||||
peer_identity_fn: Callable[[socket.socket], PeerIdentity] | None = None,
|
|
||||||
state_path: str | os.PathLike[str] | None = None,
|
|
||||||
) -> None:
|
|
||||||
self.socket_path = Path(socket_path)
|
|
||||||
self.authorization = authorization
|
|
||||||
self.handler = handler
|
|
||||||
self.peer_identity_fn = peer_identity_fn or extract_peer_identity
|
|
||||||
self.state = _AuthorizationState()
|
|
||||||
self.audit_sink = audit_sink or InMemoryAuditSink()
|
|
||||||
self.state_path = Path(state_path) if state_path is not None else None
|
|
||||||
self._load_state()
|
|
||||||
|
|
||||||
def _load_state(self) -> None:
|
|
||||||
"""Restore one-shot state without restoring request or secret material."""
|
|
||||||
if self.state_path is None or not self.state_path.exists():
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
raw = json.loads(self.state_path.read_text(encoding="utf-8"))
|
|
||||||
if not isinstance(raw, dict):
|
|
||||||
raise ValueError("state must be an object")
|
|
||||||
if raw.get("authorization_sha256") != self.authorization.authorization_sha256:
|
|
||||||
raise ValueError("state authorization digest mismatch")
|
|
||||||
self.state.create_done = {str(v) for v in raw.get("create_done", [])}
|
|
||||||
self.state.append_done = {str(v) for v in raw.get("append_done", [])}
|
|
||||||
self.state.consumed = bool(raw.get("consumed", False))
|
|
||||||
except (OSError, ValueError, TypeError, json.JSONDecodeError) as exc:
|
|
||||||
raise AuthorizationError("authorization state is invalid") from exc
|
|
||||||
|
|
||||||
def _persist_state(self) -> None:
|
|
||||||
if self.state_path is None:
|
|
||||||
return
|
|
||||||
self.state_path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
payload = {
|
|
||||||
"authorization_sha256": self.authorization.authorization_sha256,
|
|
||||||
"create_done": sorted(self.state.create_done),
|
|
||||||
"append_done": sorted(self.state.append_done),
|
|
||||||
"consumed": self.state.consumed,
|
|
||||||
}
|
|
||||||
fd, temp_name = tempfile.mkstemp(prefix=".writer-state-", dir=self.state_path.parent)
|
|
||||||
try:
|
|
||||||
os.fchmod(fd, 0o600)
|
|
||||||
with os.fdopen(fd, "w", encoding="utf-8") as stream:
|
|
||||||
json.dump(payload, stream, sort_keys=True, separators=(",", ":"))
|
|
||||||
stream.write("\n")
|
|
||||||
os.replace(temp_name, self.state_path)
|
|
||||||
os.chmod(self.state_path, 0o600)
|
|
||||||
finally:
|
|
||||||
if os.path.exists(temp_name):
|
|
||||||
os.unlink(temp_name)
|
|
||||||
|
|
||||||
def _new_audit(self, peer: PeerIdentity, operation: str, branch: str, result: str) -> None:
|
|
||||||
self.audit_sink.append(
|
|
||||||
AuditRecord(
|
|
||||||
authorization_digest=self.authorization.authorization_sha256,
|
|
||||||
peer_pid=peer.pid,
|
|
||||||
peer_uid=peer.uid,
|
|
||||||
peer_gid=peer.gid,
|
|
||||||
peer_executable=peer.executable,
|
|
||||||
timestamp=datetime.now(tz=timezone.utc),
|
|
||||||
operation=operation,
|
|
||||||
branch=branch,
|
|
||||||
result=result,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
def _require_request_field(self, request: Mapping[str, Any], field_name: str) -> str:
|
|
||||||
value = request.get(field_name)
|
|
||||||
if not isinstance(value, str) or not value:
|
|
||||||
raise AuthorizationError(f"{field_name} must be a non-empty string")
|
|
||||||
return value
|
|
||||||
|
|
||||||
def _branch_policy(self, branch: str) -> BranchPolicy:
|
|
||||||
policy = self.authorization.branch_operations.get(branch)
|
|
||||||
if policy is None:
|
|
||||||
raise AuthorizationError("branch is not authorized")
|
|
||||||
return policy
|
|
||||||
|
|
||||||
def _ensure_not_consumed(self) -> None:
|
|
||||||
if self.state.consumed:
|
|
||||||
raise AuthorizationError("authorization consumed")
|
|
||||||
if datetime.now(tz=timezone.utc) >= self.authorization.expires_at:
|
|
||||||
raise AuthorizationError("authorization expired")
|
|
||||||
|
|
||||||
def _mark_success(self, op: str, branch: str) -> None:
|
|
||||||
if op == "create_branch":
|
|
||||||
self.state.create_done.add(branch)
|
|
||||||
elif op == "append_commit":
|
|
||||||
self.state.append_done.add(branch)
|
|
||||||
|
|
||||||
if self.state.create_done.issuperset(self.authorization.create_branches) and self.state.append_done.issuperset(
|
|
||||||
self.authorization.append_branches
|
|
||||||
):
|
|
||||||
self.state.consumed = True
|
|
||||||
self._persist_state()
|
|
||||||
|
|
||||||
def _validate_request(self, request: Mapping[str, Any]) -> dict[str, Any]:
|
|
||||||
task_id = self._require_request_field(request, "task_id")
|
|
||||||
if task_id != TASK_ID:
|
|
||||||
raise AuthorizationError("invalid task_id")
|
|
||||||
|
|
||||||
principal = self._require_request_field(request, "principal")
|
|
||||||
if principal != PRINCIPAL:
|
|
||||||
raise AuthorizationError("invalid principal")
|
|
||||||
|
|
||||||
operation = self._require_request_field(request, "op")
|
|
||||||
if operation not in {"create_branch", "append_commit"}:
|
|
||||||
raise AuthorizationError("unsupported operation")
|
|
||||||
if operation in self.authorization.denied_operations:
|
|
||||||
raise AuthorizationError("operation denied by policy")
|
|
||||||
|
|
||||||
branch = self._require_request_field(request, "branch")
|
|
||||||
self._ensure_not_consumed()
|
|
||||||
policy = self._branch_policy(branch)
|
|
||||||
|
|
||||||
if operation == "create_branch":
|
|
||||||
if policy.create_branch is None:
|
|
||||||
raise AuthorizationError("branch does not allow create_branch")
|
|
||||||
if branch in self.state.create_done:
|
|
||||||
raise AuthorizationError("create_branch already used for this branch")
|
|
||||||
|
|
||||||
base_oid = self._require_request_field(request, "base_oid")
|
|
||||||
raw_head_oid = request.get("head_oid")
|
|
||||||
if raw_head_oid is not None and (not isinstance(raw_head_oid, str) or not raw_head_oid):
|
|
||||||
raise AuthorizationError("head_oid must be a non-empty string when supplied")
|
|
||||||
head_oid = str(raw_head_oid) if raw_head_oid is not None else None
|
|
||||||
if base_oid != policy.create_branch.base_oid:
|
|
||||||
raise AuthorizationError("base_oid mismatch")
|
|
||||||
if policy.create_branch.head_oid is not None and head_oid != policy.create_branch.head_oid:
|
|
||||||
raise AuthorizationError("head_oid mismatch")
|
|
||||||
|
|
||||||
return {
|
|
||||||
"operation": operation,
|
|
||||||
"branch": branch,
|
|
||||||
"base_oid": base_oid,
|
|
||||||
"head_oid": head_oid,
|
|
||||||
}
|
|
||||||
|
|
||||||
if policy.append_commit is None:
|
|
||||||
raise AuthorizationError("branch does not allow append_commit")
|
|
||||||
if policy.create_branch is not None and branch not in self.state.create_done:
|
|
||||||
raise AuthorizationError("append_commit requires prior successful create_branch")
|
|
||||||
if branch in self.state.append_done:
|
|
||||||
raise AuthorizationError("append_commit already used for this branch")
|
|
||||||
|
|
||||||
expected_head = policy.append_commit.expected_head_oid
|
|
||||||
if "expected_head_oid" in request:
|
|
||||||
supplied_head = self._require_request_field(request, "expected_head_oid")
|
|
||||||
elif "current_head_oid" in request:
|
|
||||||
supplied_head = self._require_request_field(request, "current_head_oid")
|
|
||||||
else:
|
|
||||||
raise AuthorizationError("expected_head_oid or current_head_oid required")
|
|
||||||
if not re.fullmatch(r"[0-9a-fA-F]{40}", supplied_head):
|
|
||||||
raise AuthorizationError("head check must be a full commit OID")
|
|
||||||
if expected_head is not None and supplied_head != expected_head:
|
|
||||||
raise AuthorizationError("head check failed")
|
|
||||||
|
|
||||||
if request.get("force") is True or request.get("non_fast_forward") is True:
|
|
||||||
raise AuthorizationError("non-fast-forward operations are not allowed")
|
|
||||||
if request.get("history_rewrite") is True:
|
|
||||||
raise AuthorizationError("history rewrite is not allowed")
|
|
||||||
parent_oids = request.get("parent_oids")
|
|
||||||
if parent_oids is not None:
|
|
||||||
if not isinstance(parent_oids, list) or parent_oids != [supplied_head]:
|
|
||||||
raise AuthorizationError("parent_oids must contain exactly expected_head_oid")
|
|
||||||
|
|
||||||
changes = request.get("changes")
|
|
||||||
if not isinstance(changes, list):
|
|
||||||
raise AuthorizationError("changes must be a list")
|
|
||||||
validated_changes = validate_file_changes(changes, policy.append_commit.allowed_paths)
|
|
||||||
commit_message = request.get("commit_message", "chore(governance): append governed commit")
|
|
||||||
if not isinstance(commit_message, str) or not commit_message.strip() or len(commit_message) > 200:
|
|
||||||
raise AuthorizationError("commit_message must be a short non-empty string")
|
|
||||||
if "\n" in commit_message or "\r" in commit_message or contains_secret_like_content(commit_message):
|
|
||||||
raise AuthorizationError("commit_message failed policy")
|
|
||||||
|
|
||||||
return {
|
|
||||||
"operation": operation,
|
|
||||||
"branch": branch,
|
|
||||||
"expected_head_oid": supplied_head,
|
|
||||||
"changes": validated_changes,
|
|
||||||
"commit_message": commit_message.strip(),
|
|
||||||
}
|
|
||||||
|
|
||||||
def process_request(self, raw_request: Mapping[str, Any], peer: PeerIdentity) -> Mapping[str, Any]:
|
|
||||||
"""Run one request through policy checks and handler adapter."""
|
|
||||||
operation = raw_request.get("op") if isinstance(raw_request.get("op"), str) else "unknown"
|
|
||||||
branch = raw_request.get("branch") if isinstance(raw_request.get("branch"), str) else ""
|
|
||||||
try:
|
|
||||||
details = self._validate_request(raw_request)
|
|
||||||
operation = str(details["operation"])
|
|
||||||
branch = str(details["branch"])
|
|
||||||
|
|
||||||
if operation == "create_branch":
|
|
||||||
self.handler.create_branch(
|
|
||||||
CreateBranchRequest(
|
|
||||||
branch=branch,
|
|
||||||
base_oid=str(details["base_oid"]),
|
|
||||||
head_oid=(
|
|
||||||
str(details["head_oid"])
|
|
||||||
if details.get("head_oid") is not None
|
|
||||||
else None
|
|
||||||
),
|
|
||||||
authorization_digest=self.authorization.authorization_sha256,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
self._mark_success("create_branch", branch)
|
|
||||||
self._new_audit(peer, operation, branch, "ok")
|
|
||||||
return {
|
|
||||||
"status": "ok",
|
|
||||||
"operation": operation,
|
|
||||||
"branch": branch,
|
|
||||||
"result": "applied",
|
|
||||||
}
|
|
||||||
|
|
||||||
self.handler.append_commit(
|
|
||||||
AppendCommitRequest(
|
|
||||||
branch=branch,
|
|
||||||
expected_head_oid=str(details["expected_head_oid"]),
|
|
||||||
changes=tuple(details["changes"]),
|
|
||||||
authorization_digest=self.authorization.authorization_sha256,
|
|
||||||
commit_message=str(details["commit_message"]),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
self._mark_success("append_commit", branch)
|
|
||||||
self._new_audit(peer, operation, branch, "ok")
|
|
||||||
return {
|
|
||||||
"status": "ok",
|
|
||||||
"operation": operation,
|
|
||||||
"branch": branch,
|
|
||||||
"result": "applied",
|
|
||||||
}
|
|
||||||
except Exception as error:
|
|
||||||
self._new_audit(peer, operation or "unknown", branch, f"error:{error.__class__.__name__}")
|
|
||||||
if isinstance(error, PermissionError):
|
|
||||||
code = "permission_denied"
|
|
||||||
elif isinstance(error, AuthorizationError):
|
|
||||||
code = "request_denied"
|
|
||||||
else:
|
|
||||||
code = "handler_failed"
|
|
||||||
if code == "handler_failed" and str(error).startswith("handler_phase="):
|
|
||||||
code = str(error)
|
|
||||||
return {
|
|
||||||
"status": "error",
|
|
||||||
"operation": operation or "unknown",
|
|
||||||
"branch": branch,
|
|
||||||
"error": code,
|
|
||||||
}
|
|
||||||
|
|
||||||
def _write_json(self, connection: socket.socket, payload: Mapping[str, Any]) -> None:
|
|
||||||
connection.sendall((json.dumps(payload) + "\n").encode("utf-8"))
|
|
||||||
|
|
||||||
def _handle_client(self, connection: socket.socket) -> None:
|
|
||||||
with connection:
|
|
||||||
try:
|
|
||||||
peer = self.peer_identity_fn(connection)
|
|
||||||
except Exception:
|
|
||||||
self._write_json(
|
|
||||||
connection,
|
|
||||||
{
|
|
||||||
"status": "error",
|
|
||||||
"operation": "connection",
|
|
||||||
"branch": "",
|
|
||||||
"error": "peer identity validation failed",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
with connection.makefile("r", encoding="utf-8") as stream:
|
|
||||||
for raw_line in stream:
|
|
||||||
line = raw_line.strip()
|
|
||||||
if not line:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
request = json.loads(line)
|
|
||||||
except json.JSONDecodeError:
|
|
||||||
self._write_json(
|
|
||||||
connection,
|
|
||||||
{
|
|
||||||
"status": "error",
|
|
||||||
"operation": "request",
|
|
||||||
"branch": "",
|
|
||||||
"error": "invalid json",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
if not isinstance(request, Mapping):
|
|
||||||
self._write_json(
|
|
||||||
connection,
|
|
||||||
{
|
|
||||||
"status": "error",
|
|
||||||
"operation": "request",
|
|
||||||
"branch": "",
|
|
||||||
"error": "request must be a JSON object",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
self._write_json(connection, self.process_request(request, peer))
|
|
||||||
|
|
||||||
def _bind_socket(self) -> socket.socket:
|
|
||||||
if self.socket_path.exists():
|
|
||||||
raise RuntimeError("governed writer socket already exists")
|
|
||||||
parent = self.socket_path.parent
|
|
||||||
if not parent.is_dir():
|
|
||||||
raise RuntimeError("governed writer socket parent is missing")
|
|
||||||
if parent.is_symlink():
|
|
||||||
raise RuntimeError("governed writer socket parent must not be a symlink")
|
|
||||||
server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
|
||||||
server.bind(str(self.socket_path))
|
|
||||||
os.chmod(self.socket_path, 0o600)
|
|
||||||
server.listen(16)
|
|
||||||
return server
|
|
||||||
|
|
||||||
def serve(self) -> None:
|
|
||||||
"""Serve newline-delimited JSON over a filesystem AF_UNIX socket."""
|
|
||||||
server = self._bind_socket()
|
|
||||||
try:
|
|
||||||
while True:
|
|
||||||
connection, _ = server.accept()
|
|
||||||
self._handle_client(connection)
|
|
||||||
finally:
|
|
||||||
server.close()
|
|
||||||
if self.socket_path.exists():
|
|
||||||
self.socket_path.unlink()
|
|
||||||
|
|
||||||
|
|
||||||
class JsonAuditSink:
|
|
||||||
"""Append-only root-owned audit sink containing no request payloads."""
|
|
||||||
|
|
||||||
def __init__(self, path: str | os.PathLike[str]) -> None:
|
|
||||||
self.path = Path(path)
|
|
||||||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
def append(self, record: AuditRecord) -> None:
|
|
||||||
payload = {
|
|
||||||
"authorization_sha256": record.authorization_digest,
|
|
||||||
"peer_pid": record.peer_pid,
|
|
||||||
"peer_uid": record.peer_uid,
|
|
||||||
"peer_gid": record.peer_gid,
|
|
||||||
"peer_executable": record.peer_executable,
|
|
||||||
"timestamp": record.timestamp.isoformat(),
|
|
||||||
"operation": record.operation,
|
|
||||||
"branch": record.branch,
|
|
||||||
"result": record.result,
|
|
||||||
}
|
|
||||||
line = (json.dumps(payload, sort_keys=True, separators=(",", ":")) + "\n").encode("utf-8")
|
|
||||||
fd = os.open(self.path, os.O_WRONLY | os.O_CREAT | os.O_APPEND, 0o600)
|
|
||||||
try:
|
|
||||||
os.fchmod(fd, 0o600)
|
|
||||||
os.write(fd, line)
|
|
||||||
finally:
|
|
||||||
os.close(fd)
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# V2R1C6 durable receipt overlay. This subclasses the original root-only
|
|
||||||
# writer in-place so the existing AF_UNIX/SO_PEERCRED and handler policy remain
|
|
||||||
# the only mutation path.
|
|
||||||
_LegacyGovernedLocalWriterService = GovernedLocalWriterService
|
|
||||||
|
|
||||||
class GovernedLocalWriterService(_LegacyGovernedLocalWriterService):
|
|
||||||
"""Existing writer with durable request receipts and safe recovery."""
|
|
||||||
|
|
||||||
def _load_state(self) -> None:
|
|
||||||
super()._load_state()
|
|
||||||
self.state.receipts = {}
|
|
||||||
if self.state_path is None or not self.state_path.exists():
|
|
||||||
return
|
|
||||||
raw = json.loads(self.state_path.read_text(encoding="utf-8"))
|
|
||||||
if not isinstance(raw, dict):
|
|
||||||
raise AuthorizationError("authorization state is invalid")
|
|
||||||
receipts = raw.get("receipts", {})
|
|
||||||
if not isinstance(receipts, dict):
|
|
||||||
raise AuthorizationError("authorization receipts are invalid")
|
|
||||||
self.state.receipts = {
|
|
||||||
str(key): dict(value)
|
|
||||||
for key, value in receipts.items()
|
|
||||||
if isinstance(value, dict)
|
|
||||||
}
|
|
||||||
|
|
||||||
def _persist_state(self) -> None:
|
|
||||||
if self.state_path is None:
|
|
||||||
return
|
|
||||||
self.state_path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
payload = {
|
|
||||||
"authorization_sha256": self.authorization.authorization_sha256,
|
|
||||||
"create_done": sorted(self.state.create_done),
|
|
||||||
"append_done": sorted(self.state.append_done),
|
|
||||||
"consumed": bool(self.state.consumed),
|
|
||||||
"receipts": self.state.receipts,
|
|
||||||
}
|
|
||||||
fd, temp_name = tempfile.mkstemp(prefix=".writer-state-", dir=self.state_path.parent)
|
|
||||||
try:
|
|
||||||
os.fchmod(fd, 0o600)
|
|
||||||
with os.fdopen(fd, "w", encoding="utf-8") as stream:
|
|
||||||
json.dump(payload, stream, sort_keys=True, separators=(",", ":"))
|
|
||||||
stream.write("\n")
|
|
||||||
os.replace(temp_name, self.state_path)
|
|
||||||
os.chmod(self.state_path, 0o600)
|
|
||||||
finally:
|
|
||||||
if os.path.exists(temp_name):
|
|
||||||
os.unlink(temp_name)
|
|
||||||
|
|
||||||
def _request_identity(self, request: Mapping[str, Any]) -> dict[str, str]:
|
|
||||||
canonical = json.dumps(dict(request), sort_keys=True, separators=(",", ":"), ensure_ascii=True)
|
|
||||||
fallback = hashlib.sha256(canonical.encode("utf-8")).hexdigest()
|
|
||||||
request_id = request.get("request_id")
|
|
||||||
idempotency_key = request.get("idempotency_key")
|
|
||||||
request_id = request_id.strip() if isinstance(request_id, str) and request_id.strip() else "legacy-" + fallback
|
|
||||||
idempotency_key = idempotency_key.strip() if isinstance(idempotency_key, str) and idempotency_key.strip() else "legacy-" + fallback
|
|
||||||
for field_name, value in (("request_id", request_id), ("idempotency_key", idempotency_key)):
|
|
||||||
if len(value) > 200 or "\n" in value or "\r" in value:
|
|
||||||
raise AuthorizationError(field_name + " is invalid")
|
|
||||||
return {
|
|
||||||
"request_id": request_id,
|
|
||||||
"idempotency_key": idempotency_key,
|
|
||||||
"request_fingerprint": fallback,
|
|
||||||
}
|
|
||||||
|
|
||||||
def _find_receipt(self, details: Mapping[str, Any], compare_fingerprint: bool = True):
|
|
||||||
request_id = str(details["request_id"])
|
|
||||||
idempotency_key = str(details["idempotency_key"])
|
|
||||||
fingerprint = str(details.get("request_fingerprint", ""))
|
|
||||||
for receipt in self.state.receipts.values():
|
|
||||||
if receipt.get("request_id") == request_id or receipt.get("idempotency_key") == idempotency_key:
|
|
||||||
if compare_fingerprint and receipt.get("request_fingerprint") != fingerprint:
|
|
||||||
raise AuthorizationError("idempotency key reused with different request")
|
|
||||||
return receipt
|
|
||||||
return None
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _receipt_response(receipt: Mapping[str, object], status: str = "receipt") -> dict[str, object]:
|
|
||||||
response = {
|
|
||||||
"status": status,
|
|
||||||
"request_id": receipt.get("request_id"),
|
|
||||||
"idempotency_key": receipt.get("idempotency_key"),
|
|
||||||
"operation": receipt.get("operation"),
|
|
||||||
"branch": receipt.get("branch", ""),
|
|
||||||
"receipt_status": receipt.get("status"),
|
|
||||||
}
|
|
||||||
for field_name in ("commit_oid", "final_head", "parent_oid", "audit_sha256", "error_code", "safe_to_retry"):
|
|
||||||
if field_name in receipt:
|
|
||||||
response[field_name] = receipt[field_name]
|
|
||||||
return response
|
|
||||||
|
|
||||||
def _new_receipt(self, details: Mapping[str, Any]) -> dict[str, object]:
|
|
||||||
return {
|
|
||||||
"request_id": str(details["request_id"]),
|
|
||||||
"idempotency_key": str(details["idempotency_key"]),
|
|
||||||
"request_fingerprint": str(details["request_fingerprint"]),
|
|
||||||
"operation": str(details["operation"]),
|
|
||||||
"branch": str(details.get("branch", "")),
|
|
||||||
"expected_head_oid": str(details.get("expected_head_oid", "")),
|
|
||||||
"status": "accepted",
|
|
||||||
"created_at": datetime.now(tz=timezone.utc).isoformat(),
|
|
||||||
"safe_to_retry": False,
|
|
||||||
}
|
|
||||||
|
|
||||||
def _new_audit(self, peer: PeerIdentity, operation: str, branch: str, result: str) -> str:
|
|
||||||
super()._new_audit(peer, operation, branch, result)
|
|
||||||
payload = {
|
|
||||||
"authorization_sha256": self.authorization.authorization_sha256,
|
|
||||||
"peer_pid": peer.pid,
|
|
||||||
"peer_uid": peer.uid,
|
|
||||||
"peer_gid": peer.gid,
|
|
||||||
"peer_executable": peer.executable,
|
|
||||||
"operation": operation,
|
|
||||||
"branch": branch,
|
|
||||||
"result": result,
|
|
||||||
}
|
|
||||||
return hashlib.sha256(json.dumps(payload, sort_keys=True, separators=(",", ":")).encode("utf-8")).hexdigest()
|
|
||||||
|
|
||||||
def _mark_success(self, op: str, branch: str) -> None:
|
|
||||||
if op == "create_branch":
|
|
||||||
self.state.create_done.add(branch)
|
|
||||||
elif op in {"append_commit", "append_files", "append_batch"}:
|
|
||||||
self.state.append_done.add(branch)
|
|
||||||
# The provisioning grant is one-shot; the task writer lease itself must
|
|
||||||
# support multiple append receipts for the exact authorized branches.
|
|
||||||
self.state.consumed = False
|
|
||||||
self._persist_state()
|
|
||||||
|
|
||||||
def _validate_durable(self, raw_request: Mapping[str, Any]) -> dict[str, Any]:
|
|
||||||
operation = self._require_request_field(raw_request, "op")
|
|
||||||
identity = self._request_identity(raw_request)
|
|
||||||
if operation == "query_receipt":
|
|
||||||
return {"operation": operation, **identity}
|
|
||||||
if operation == "recover_request":
|
|
||||||
branch = self._require_request_field(raw_request, "branch")
|
|
||||||
policy = self._branch_policy(branch)
|
|
||||||
if policy.append_commit is None:
|
|
||||||
raise AuthorizationError("branch does not allow recovery")
|
|
||||||
expected = raw_request.get("expected_head_oid") or policy.append_commit.expected_head_oid
|
|
||||||
if not isinstance(expected, str) or not re.fullmatch(r"[0-9a-fA-F]{40}", expected):
|
|
||||||
raise AuthorizationError("recovery requires expected_head_oid")
|
|
||||||
return {"operation": operation, "branch": branch, "expected_head_oid": expected.lower(), **identity}
|
|
||||||
if operation not in {"create_branch", "append_commit", "append_files", "append_batch"}:
|
|
||||||
raise AuthorizationError("unsupported operation")
|
|
||||||
request = dict(raw_request)
|
|
||||||
if operation in {"append_files", "append_batch"}:
|
|
||||||
request["op"] = "append_commit"
|
|
||||||
if operation in {"append_commit", "append_files", "append_batch"}:
|
|
||||||
self.state.consumed = False
|
|
||||||
branch = request.get("branch")
|
|
||||||
removed = False
|
|
||||||
dynamic_expected = False
|
|
||||||
if isinstance(branch, str) and branch in self.state.append_done:
|
|
||||||
self.state.append_done.remove(branch)
|
|
||||||
removed = True
|
|
||||||
policy = self.authorization.branch_operations.get(branch)
|
|
||||||
if policy is not None and policy.append_commit is not None and policy.append_commit.expected_head_oid is not None:
|
|
||||||
from governed_local_writer import BranchAppendPolicy, BranchPolicy
|
|
||||||
self.authorization.branch_operations[branch] = BranchPolicy(create_branch=policy.create_branch, append_commit=BranchAppendPolicy(branch=branch, expected_head_oid=None, allowed_paths=policy.append_commit.allowed_paths))
|
|
||||||
dynamic_expected = True
|
|
||||||
try:
|
|
||||||
details = super()._validate_request(request)
|
|
||||||
finally:
|
|
||||||
if dynamic_expected:
|
|
||||||
policy = self.authorization.branch_operations.get(str(branch))
|
|
||||||
if policy is not None and policy.append_commit is not None:
|
|
||||||
from governed_local_writer import BranchAppendPolicy, BranchPolicy
|
|
||||||
self.authorization.branch_operations[str(branch)] = BranchPolicy(create_branch=policy.create_branch, append_commit=BranchAppendPolicy(branch=str(branch), expected_head_oid=str(raw_request.get("expected_head_oid") or ""), allowed_paths=policy.append_commit.allowed_paths))
|
|
||||||
if removed:
|
|
||||||
self.state.append_done.add(str(branch))
|
|
||||||
details.update(identity)
|
|
||||||
details["operation"] = operation
|
|
||||||
return details
|
|
||||||
|
|
||||||
def _recover_backend(self, details: Mapping[str, Any]) -> dict[str, object]:
|
|
||||||
if not hasattr(self.handler, "_workspace") or not hasattr(self.handler, "_git"):
|
|
||||||
raise RuntimeError("recovery backend unavailable")
|
|
||||||
workspace = self.handler._workspace(str(details["branch"]))
|
|
||||||
expected = str(details["expected_head_oid"])
|
|
||||||
current = self.handler._git(workspace, ["rev-parse", "HEAD"])
|
|
||||||
if current == expected:
|
|
||||||
return {"status": "not_started", "final_head": current}
|
|
||||||
parent = self.handler._git(workspace, ["rev-parse", "HEAD^1"])
|
|
||||||
if parent != expected:
|
|
||||||
raise RuntimeError("external branch advance during recovery")
|
|
||||||
return {"status": "applied", "commit_oid": current, "final_head": current, "parent_oid": parent}
|
|
||||||
|
|
||||||
def process_request(self, raw_request: Mapping[str, Any], peer: PeerIdentity) -> Mapping[str, Any]:
|
|
||||||
operation = raw_request.get("op") if isinstance(raw_request.get("op"), str) else "unknown"
|
|
||||||
branch = raw_request.get("branch") if isinstance(raw_request.get("branch"), str) else ""
|
|
||||||
try:
|
|
||||||
details = self._validate_durable(raw_request)
|
|
||||||
operation = str(details["operation"])
|
|
||||||
branch = str(details.get("branch", ""))
|
|
||||||
if operation == "query_receipt":
|
|
||||||
receipt = self._find_receipt(details, compare_fingerprint=False)
|
|
||||||
return self._receipt_response(receipt) if receipt is not None else {"status": "not_found", "request_id": details["request_id"], "idempotency_key": details["idempotency_key"]}
|
|
||||||
if operation == "recover_request":
|
|
||||||
receipt = self._find_receipt(details, compare_fingerprint=False)
|
|
||||||
if receipt is None:
|
|
||||||
return {"status": "not_found", "request_id": details["request_id"], "idempotency_key": details["idempotency_key"], "safe_to_retry": True}
|
|
||||||
if receipt.get("status") == "applied":
|
|
||||||
return self._receipt_response(receipt)
|
|
||||||
recovered = self._recover_backend(details)
|
|
||||||
if recovered.get("status") == "applied":
|
|
||||||
receipt.update({"status": "applied", "commit_oid": recovered.get("commit_oid"), "final_head": recovered.get("final_head"), "parent_oid": recovered.get("parent_oid"), "safe_to_retry": False})
|
|
||||||
self._mark_success("append_commit", branch)
|
|
||||||
return self._receipt_response(receipt)
|
|
||||||
receipt["safe_to_retry"] = True
|
|
||||||
self._persist_state()
|
|
||||||
return self._receipt_response(receipt)
|
|
||||||
receipt = self._find_receipt(details)
|
|
||||||
if receipt is not None:
|
|
||||||
return self._receipt_response(receipt)
|
|
||||||
receipt = self._new_receipt(details)
|
|
||||||
self.state.receipts[str(details["request_id"])] = receipt
|
|
||||||
self._persist_state()
|
|
||||||
if operation == "create_branch":
|
|
||||||
result = self.handler.create_branch(CreateBranchRequest(branch=branch, base_oid=str(details["base_oid"]), head_oid=str(details["head_oid"]) if details.get("head_oid") is not None else None, authorization_digest=self.authorization.authorization_sha256))
|
|
||||||
commit_oid = str(result.get("head_oid", ""))
|
|
||||||
parent_oid = str(result.get("parent_oid", ""))
|
|
||||||
audit_sha = self._new_audit(peer, operation, branch, "ok")
|
|
||||||
receipt.update({"status": "applied", "commit_oid": commit_oid, "final_head": commit_oid, "parent_oid": parent_oid, "audit_sha256": audit_sha, "safe_to_retry": False})
|
|
||||||
self._mark_success("create_branch", branch)
|
|
||||||
return self._receipt_response(receipt, status="ok") | {"result": "applied"}
|
|
||||||
result = self.handler.append_commit(AppendCommitRequest(branch=branch, expected_head_oid=str(details["expected_head_oid"]), changes=tuple(details["changes"]), authorization_digest=self.authorization.authorization_sha256, commit_message=str(details["commit_message"])))
|
|
||||||
commit_oid = str(result.get("head_oid", ""))
|
|
||||||
parent_oid = str(result.get("parent_oid", ""))
|
|
||||||
audit_sha = self._new_audit(peer, operation, branch, "ok")
|
|
||||||
receipt.update({"status": "applied", "commit_oid": commit_oid, "final_head": commit_oid, "parent_oid": parent_oid, "audit_sha256": audit_sha, "safe_to_retry": False})
|
|
||||||
self._mark_success("append_commit", branch)
|
|
||||||
return self._receipt_response(receipt, status="ok") | {"result": "applied"}
|
|
||||||
except Exception as error:
|
|
||||||
try:
|
|
||||||
identity = self._request_identity(raw_request)
|
|
||||||
receipt = self.state.receipts.get(identity["request_id"])
|
|
||||||
if receipt is not None and receipt.get("status") == "accepted":
|
|
||||||
receipt.update({"status": "failed", "error_code": error.__class__.__name__, "safe_to_retry": False})
|
|
||||||
self._persist_state()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
self._new_audit(peer, operation, branch, "error:" + error.__class__.__name__)
|
|
||||||
return {"status": "error", "operation": operation, "branch": branch, "error": "request_denied" if isinstance(error, AuthorizationError) else "handler_failed"}
|
|
||||||
|
|
||||||
def _parse_args() -> argparse.Namespace:
|
|
||||||
parser = argparse.ArgumentParser(description="AIOS governed local writer")
|
|
||||||
parser.add_argument("--socket", required=True)
|
|
||||||
parser.add_argument("--authorization", required=True)
|
|
||||||
parser.add_argument("--state", required=True)
|
|
||||||
parser.add_argument("--audit", required=True)
|
|
||||||
parser.add_argument("--repair-branch", required=True)
|
|
||||||
parser.add_argument("--repair-workspace", required=True)
|
|
||||||
parser.add_argument("--candidate-branch", required=True)
|
|
||||||
parser.add_argument("--candidate-workspace", required=True)
|
|
||||||
parser.add_argument("--bare-repo", required=True)
|
|
||||||
parser.add_argument("--api-source", required=True)
|
|
||||||
return parser.parse_args()
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
if hasattr(os, "geteuid") and os.geteuid() != 0:
|
|
||||||
print("governed local writer requires uid 0", file=sys.stderr)
|
|
||||||
return 2
|
|
||||||
args = _parse_args()
|
|
||||||
authorization = load_authorization_file(args.authorization)
|
|
||||||
handler = ExistingGitOpsHandler(
|
|
||||||
workspace_by_branch={
|
|
||||||
args.repair_branch: args.repair_workspace,
|
|
||||||
args.candidate_branch: args.candidate_workspace,
|
|
||||||
},
|
|
||||||
bare_repo=args.bare_repo,
|
|
||||||
api_source=args.api_source,
|
|
||||||
)
|
|
||||||
service = GovernedLocalWriterService(
|
|
||||||
args.socket,
|
|
||||||
authorization,
|
|
||||||
handler,
|
|
||||||
audit_sink=JsonAuditSink(args.audit),
|
|
||||||
state_path=args.state,
|
|
||||||
)
|
|
||||||
service.serve()
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__": # pragma: no cover - systemd entrypoint
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -1,59 +0,0 @@
|
|||||||
"""Central fail-closed policy seam for governed task and continuation requests."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
import re
|
|
||||||
from typing import Any, Mapping
|
|
||||||
|
|
||||||
OID_RE = re.compile(r"^[0-9a-f]{40}$")
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class GovernedPolicyInput:
|
|
||||||
identity: str
|
|
||||||
task_id: str
|
|
||||||
repository: str
|
|
||||||
operation: str
|
|
||||||
base_oid: str
|
|
||||||
expected_branch_oid: str
|
|
||||||
target_branch: str
|
|
||||||
generation: int
|
|
||||||
risk: str = "engineering"
|
|
||||||
requested_capabilities: tuple[str, ...] = ()
|
|
||||||
previous_generation: int | None = None
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class PolicyDecision:
|
|
||||||
decision: str
|
|
||||||
reason: str
|
|
||||||
|
|
||||||
def as_dict(self) -> dict[str, str]:
|
|
||||||
return {"decision": self.decision, "reason": self.reason}
|
|
||||||
|
|
||||||
|
|
||||||
def evaluate_policy(value: GovernedPolicyInput | Mapping[str, Any]) -> PolicyDecision:
|
|
||||||
if isinstance(value, Mapping):
|
|
||||||
value = GovernedPolicyInput(**value)
|
|
||||||
if value.repository != "aios/aios":
|
|
||||||
return PolicyDecision("DENY", "REPOSITORY_IDENTITY_MISMATCH")
|
|
||||||
if not OID_RE.fullmatch(value.base_oid) or not OID_RE.fullmatch(value.expected_branch_oid):
|
|
||||||
return PolicyDecision("DENY", "OID_INVALID")
|
|
||||||
if not value.target_branch.startswith("agent/") or value.target_branch in {"main", "master"}:
|
|
||||||
return PolicyDecision("DENY", "BRANCH_SCOPE_DENIED")
|
|
||||||
if "force_push" in value.requested_capabilities or "force" in value.operation.lower():
|
|
||||||
return PolicyDecision("DENY", "FORCE_PUSH_DENIED")
|
|
||||||
if {"main_write", "tag_write", "baseline_write", "production_write", "production_deploy", "tag_mutation", "baseline_mutation"}.intersection(value.requested_capabilities):
|
|
||||||
return PolicyDecision("DENY", "MUTATION_SCOPE_DENIED")
|
|
||||||
if not isinstance(value.generation, int) or isinstance(value.generation, bool) or value.generation < 1:
|
|
||||||
return PolicyDecision("DENY", "GENERATION_INVALID")
|
|
||||||
if value.previous_generation is not None and value.generation <= value.previous_generation:
|
|
||||||
return PolicyDecision("MANUAL_OWNER_REVIEW_REQUIRED", "GENERATION_NOT_MONOTONIC")
|
|
||||||
if not value.task_id:
|
|
||||||
return PolicyDecision("DENY", "TASK_ID_INVALID")
|
|
||||||
return PolicyDecision("ALLOW", "GOVERNED_ENGINEERING_SCOPE")
|
|
||||||
|
|
||||||
|
|
||||||
class GovernedPolicy:
|
|
||||||
def decide(self, request: GovernedPolicyInput | Mapping[str, Any]) -> PolicyDecision:
|
|
||||||
return evaluate_policy(request)
|
|
||||||
@@ -1,84 +0,0 @@
|
|||||||
"""Durable governed task binding for CREATE_NEW and CONTINUE_EXISTING."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass, asdict
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
import re
|
|
||||||
from typing import Any, Mapping
|
|
||||||
|
|
||||||
from aios_gitops_job_controller import GitOpsJobController, JobStore
|
|
||||||
|
|
||||||
OID_RE = re.compile(r"^[0-9a-f]{40}$")
|
|
||||||
BRANCH_RE = re.compile(r"^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$")
|
|
||||||
|
|
||||||
|
|
||||||
class BootstrapError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class GovernedTaskBinding:
|
|
||||||
task_id: str
|
|
||||||
repository: str
|
|
||||||
base_oid: str
|
|
||||||
target_branch: str
|
|
||||||
risk: str
|
|
||||||
description: str
|
|
||||||
idempotency_key: str
|
|
||||||
expected_branch_oid: str
|
|
||||||
branch_mode: str
|
|
||||||
generation: int
|
|
||||||
job_id: str | None = None
|
|
||||||
|
|
||||||
def response(self, *, idempotent: bool = False) -> dict[str, Any]:
|
|
||||||
return {"task_id": self.task_id, "repository": self.repository, "base_oid": self.base_oid, "target_branch": self.target_branch, "expected_branch_oid": self.expected_branch_oid, "generation": self.generation, "branch_mode": self.branch_mode, "create_target_branch": self.branch_mode == "CREATE_NEW", "worktree_mode": "detached_at_base_oid" if self.branch_mode == "CREATE_NEW" else "detached_at_expected_head", "cas_ref_update_required": True, "idempotent": idempotent, "state": "BOUND", "job_id": self.job_id}
|
|
||||||
|
|
||||||
|
|
||||||
def _safe(value: str) -> str:
|
|
||||||
return re.sub(r"[^A-Za-z0-9_.-]+", "-", value).strip("-")[:120] or "task"
|
|
||||||
|
|
||||||
|
|
||||||
class GovernedTaskBootstrap:
|
|
||||||
def __init__(self, authority: Any, root: Path, *, remote_name: str, token_path: Path, ssh_key_path: Path, api_base: str = "https://g.cnnetmkt.top/git/api/v1") -> None:
|
|
||||||
self.authority = authority; self.root = Path(root); self.remote_name = remote_name; self.token_path = Path(token_path); self.ssh_key_path = Path(ssh_key_path); self.api_base = api_base; self.task_meta_root = self.root / "task-bindings"; self.job_root = self.root / "jobs"; self.task_meta_root.mkdir(parents=True, exist_ok=True); self.job_root.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
def _store(self, task_id: str) -> tuple[JobStore, GitOpsJobController]:
|
|
||||||
root = self.job_root / _safe(task_id); store = JobStore(root, payload_root=root / "payloads"); return store, GitOpsJobController(store)
|
|
||||||
|
|
||||||
def _metadata(self, task_id: str) -> Path:
|
|
||||||
return self.task_meta_root / (_safe(task_id) + ".json")
|
|
||||||
|
|
||||||
def _read_existing(self, task_id: str) -> GovernedTaskBinding | None:
|
|
||||||
path = self._metadata(task_id)
|
|
||||||
if not path.is_file() or path.is_symlink(): return None
|
|
||||||
return GovernedTaskBinding(**json.loads(path.read_text(encoding="utf-8")))
|
|
||||||
|
|
||||||
def _write(self, binding: GovernedTaskBinding) -> None:
|
|
||||||
path = self._metadata(binding.task_id); temporary = path.with_suffix(".tmp"); temporary.write_text(json.dumps(asdict(binding), sort_keys=True), encoding="utf-8"); temporary.replace(path)
|
|
||||||
|
|
||||||
def create(self, request: Mapping[str, Any]) -> dict[str, Any]:
|
|
||||||
values = {key: str(request[key]) for key in ("task_id", "repository", "base_oid", "target_branch", "risk", "description", "idempotency_key")}
|
|
||||||
if values["repository"] != "aios/aios" or not OID_RE.fullmatch(values["base_oid"]): raise BootstrapError("TASK_BINDING_INVALID")
|
|
||||||
if not BRANCH_RE.fullmatch(values["target_branch"]): raise BootstrapError("TARGET_BRANCH_INVALID")
|
|
||||||
existing = self._read_existing(values["task_id"])
|
|
||||||
if existing is not None:
|
|
||||||
if any(getattr(existing, key) != values[key] for key in ("repository", "base_oid", "target_branch", "risk", "description")): raise BootstrapError("TASK_BINDING_CONFLICT")
|
|
||||||
return existing.response(idempotent=True)
|
|
||||||
if not self.authority.commit_exists(values["base_oid"]): raise BootstrapError("BASE_COMMIT_NOT_FOUND")
|
|
||||||
current = self.authority.read_ref("refs/heads/" + values["target_branch"])
|
|
||||||
expected = str(request.get("expected_branch_oid") or values["base_oid"])
|
|
||||||
if current is None:
|
|
||||||
mode, generation = "CREATE_NEW", 1
|
|
||||||
if expected != values["base_oid"]: raise BootstrapError("EXPECTED_BASE_CONFLICT")
|
|
||||||
else:
|
|
||||||
if expected != current: raise BootstrapError("EXISTING_BRANCH_OID_CONFLICT")
|
|
||||||
mode, generation = "CONTINUE_EXISTING", int(request.get("generation", 2))
|
|
||||||
if generation < 1: raise BootstrapError("GENERATION_INVALID")
|
|
||||||
payload = json.dumps(values, sort_keys=True, separators=(",", ":")).encode()
|
|
||||||
_, controller = self._store(values["task_id"])
|
|
||||||
receipt = controller.submit({"idempotency_key": values["idempotency_key"], "campaign_id": values["task_id"], "task_id": values["task_id"], "repository": values["repository"], "operation": "GOVERNED_TASK_CREATE", "branch": values["target_branch"], "expected_branch_oid": expected, "payload_digest": hashlib.sha256(payload).hexdigest(), "lineage_anchor_oid": values["base_oid"]}, payload=payload)
|
|
||||||
binding = GovernedTaskBinding(**values, expected_branch_oid=expected, branch_mode=mode, generation=generation, job_id=receipt.job_id)
|
|
||||||
self._write(binding)
|
|
||||||
return binding.response()
|
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
"""AIOS-managed Hermes runtime boundary."""
|
||||||
|
|
||||||
|
from .adapter import (
|
||||||
|
HermesAdapter,
|
||||||
|
HermesConflict,
|
||||||
|
HermesNotFound,
|
||||||
|
HermesRuntimeError,
|
||||||
|
InMemoryHermesStore,
|
||||||
|
SQLiteHermesStore,
|
||||||
|
)
|
||||||
|
from .contract import HermesHealth, HermesJob, HermesJobState, HermesResult
|
||||||
|
from .worker import HermesWorker, WorkerRunResult
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"HermesAdapter",
|
||||||
|
"HermesConflict",
|
||||||
|
"HermesHealth",
|
||||||
|
"HermesJob",
|
||||||
|
"HermesJobState",
|
||||||
|
"HermesNotFound",
|
||||||
|
"HermesResult",
|
||||||
|
"HermesRuntimeError",
|
||||||
|
"HermesWorker",
|
||||||
|
"InMemoryHermesStore",
|
||||||
|
"SQLiteHermesStore",
|
||||||
|
"WorkerRunResult",
|
||||||
|
]
|
||||||
@@ -0,0 +1,593 @@
|
|||||||
|
"""Tenant-partitioned Hermes queue and worker lease boundary."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import sqlite3
|
||||||
|
import threading
|
||||||
|
from dataclasses import asdict, replace
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Iterable, Mapping
|
||||||
|
|
||||||
|
from .contract import (
|
||||||
|
ALLOWED_TRANSITIONS,
|
||||||
|
TERMINAL_STATES,
|
||||||
|
HermesHealth,
|
||||||
|
HermesJob,
|
||||||
|
HermesJobState,
|
||||||
|
HermesResult,
|
||||||
|
utc_now,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class HermesRuntimeError(RuntimeError):
|
||||||
|
"""Base Hermes runtime-boundary failure."""
|
||||||
|
|
||||||
|
|
||||||
|
class HermesNotFound(HermesRuntimeError):
|
||||||
|
"""A tenant-scoped job was not found."""
|
||||||
|
|
||||||
|
|
||||||
|
class HermesConflict(HermesRuntimeError):
|
||||||
|
"""A state, lease, idempotency, or revision conflict occurred."""
|
||||||
|
|
||||||
|
|
||||||
|
class InMemoryHermesStore:
|
||||||
|
"""Thread-safe reference store; production backends implement this contract."""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.lock = threading.RLock()
|
||||||
|
self.jobs: dict[str, HermesJob] = {}
|
||||||
|
self.results: dict[str, HermesResult] = {}
|
||||||
|
self.idempotency: dict[tuple[str, str], tuple[str, str]] = {}
|
||||||
|
self.worker_heartbeats: dict[str, tuple[str, str | None]] = {}
|
||||||
|
|
||||||
|
def persist_locked(self) -> None:
|
||||||
|
"""Persist a mutation while ``lock`` is held; memory stores are a no-op."""
|
||||||
|
|
||||||
|
|
||||||
|
class SQLiteHermesStore(InMemoryHermesStore):
|
||||||
|
"""Durable snapshot for the existing Hermes queue state machine.
|
||||||
|
|
||||||
|
The adapter remains the sole owner of transitions and lease rules. This
|
||||||
|
store only makes its jobs, results, idempotency keys and worker heartbeats
|
||||||
|
survive a process restart.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, path: str | Path) -> None:
|
||||||
|
self.path = Path(path).resolve()
|
||||||
|
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
super().__init__()
|
||||||
|
with self._connect() as connection:
|
||||||
|
connection.execute(
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS hermes_state (
|
||||||
|
singleton INTEGER PRIMARY KEY CHECK (singleton = 1),
|
||||||
|
payload TEXT NOT NULL
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
self._load()
|
||||||
|
|
||||||
|
def _connect(self) -> sqlite3.Connection:
|
||||||
|
connection = sqlite3.connect(str(self.path), timeout=30)
|
||||||
|
connection.execute("PRAGMA journal_mode=WAL")
|
||||||
|
connection.execute("PRAGMA synchronous=FULL")
|
||||||
|
return connection
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _record(value: HermesJob | HermesResult) -> dict[str, object]:
|
||||||
|
record = asdict(value)
|
||||||
|
record["state"] = value.state.value
|
||||||
|
return record
|
||||||
|
|
||||||
|
def _load(self) -> None:
|
||||||
|
with self.lock, self._connect() as connection:
|
||||||
|
row = connection.execute(
|
||||||
|
"SELECT payload FROM hermes_state WHERE singleton = 1"
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
return
|
||||||
|
payload = json.loads(str(row[0]))
|
||||||
|
self.jobs = {
|
||||||
|
str(item["job_id"]): HermesJob(**item)
|
||||||
|
for item in payload.get("jobs", [])
|
||||||
|
}
|
||||||
|
self.results = {
|
||||||
|
str(item["job_id"]): HermesResult(**item)
|
||||||
|
for item in payload.get("results", [])
|
||||||
|
}
|
||||||
|
self.idempotency = {
|
||||||
|
(str(item["tenant_id"]), str(item["idempotency_key"])): (
|
||||||
|
str(item["job_id"]),
|
||||||
|
str(item["digest"]),
|
||||||
|
)
|
||||||
|
for item in payload.get("idempotency", [])
|
||||||
|
}
|
||||||
|
self.worker_heartbeats = {
|
||||||
|
str(item["worker_id"]): (
|
||||||
|
str(item["heartbeat_at"]),
|
||||||
|
str(item["job_id"]) if item.get("job_id") is not None else None,
|
||||||
|
)
|
||||||
|
for item in payload.get("worker_heartbeats", [])
|
||||||
|
}
|
||||||
|
|
||||||
|
def persist_locked(self) -> None:
|
||||||
|
payload = {
|
||||||
|
"version": 1,
|
||||||
|
"jobs": [self._record(item) for item in self.jobs.values()],
|
||||||
|
"results": [self._record(item) for item in self.results.values()],
|
||||||
|
"idempotency": [
|
||||||
|
{
|
||||||
|
"tenant_id": tenant_id,
|
||||||
|
"idempotency_key": idempotency_key,
|
||||||
|
"job_id": job_id,
|
||||||
|
"digest": digest,
|
||||||
|
}
|
||||||
|
for (tenant_id, idempotency_key), (job_id, digest)
|
||||||
|
in self.idempotency.items()
|
||||||
|
],
|
||||||
|
"worker_heartbeats": [
|
||||||
|
{
|
||||||
|
"worker_id": worker_id,
|
||||||
|
"heartbeat_at": heartbeat_at,
|
||||||
|
"job_id": job_id,
|
||||||
|
}
|
||||||
|
for worker_id, (heartbeat_at, job_id)
|
||||||
|
in self.worker_heartbeats.items()
|
||||||
|
],
|
||||||
|
}
|
||||||
|
encoded = json.dumps(
|
||||||
|
payload,
|
||||||
|
ensure_ascii=False,
|
||||||
|
sort_keys=True,
|
||||||
|
separators=(",", ":"),
|
||||||
|
)
|
||||||
|
with self._connect() as connection:
|
||||||
|
connection.execute("BEGIN IMMEDIATE")
|
||||||
|
connection.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO hermes_state(singleton, payload) VALUES(1, ?)
|
||||||
|
ON CONFLICT(singleton) DO UPDATE SET payload=excluded.payload
|
||||||
|
""",
|
||||||
|
(encoded,),
|
||||||
|
)
|
||||||
|
connection.commit()
|
||||||
|
|
||||||
|
|
||||||
|
class HermesAdapter:
|
||||||
|
"""Queue and reconcile Hermes jobs without owning AIOS policy."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
store: InMemoryHermesStore | None = None,
|
||||||
|
*,
|
||||||
|
runtime_id: str = "hermes-runtime",
|
||||||
|
runtime_version: str = "v1",
|
||||||
|
default_lease_seconds: int = 60,
|
||||||
|
) -> None:
|
||||||
|
if default_lease_seconds < 5:
|
||||||
|
raise ValueError("default_lease_seconds must be >= 5")
|
||||||
|
self.store = store or InMemoryHermesStore()
|
||||||
|
self.runtime_id = runtime_id
|
||||||
|
self.runtime_version = runtime_version
|
||||||
|
self.default_lease_seconds = int(default_lease_seconds)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _digest(job: HermesJob) -> str:
|
||||||
|
raw = json.dumps(
|
||||||
|
{
|
||||||
|
"tenant_id": job.tenant_id,
|
||||||
|
"cell_id": job.cell_id,
|
||||||
|
"profile_id": job.profile_id,
|
||||||
|
"task_id": job.task_id,
|
||||||
|
"trace_id": job.trace_id,
|
||||||
|
"action": job.action,
|
||||||
|
"payload": dict(job.payload),
|
||||||
|
"priority": job.priority,
|
||||||
|
},
|
||||||
|
ensure_ascii=False,
|
||||||
|
sort_keys=True,
|
||||||
|
separators=(",", ":"),
|
||||||
|
default=str,
|
||||||
|
).encode("utf-8")
|
||||||
|
return hashlib.sha256(raw).hexdigest()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _parse(value: str | None) -> datetime | None:
|
||||||
|
if value is None:
|
||||||
|
return None
|
||||||
|
normalized = value[:-1] + "+00:00" if value.endswith("Z") else value
|
||||||
|
parsed = datetime.fromisoformat(normalized)
|
||||||
|
if parsed.tzinfo is None:
|
||||||
|
raise ValueError("timestamp must include timezone")
|
||||||
|
return parsed.astimezone(timezone.utc)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _format(value: datetime) -> str:
|
||||||
|
return value.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _assert_tenant(job: HermesJob, tenant_id: str) -> None:
|
||||||
|
if job.tenant_id != tenant_id:
|
||||||
|
raise HermesNotFound("job not found")
|
||||||
|
|
||||||
|
def enqueue(self, job: HermesJob) -> HermesJob:
|
||||||
|
if job.state != HermesJobState.QUEUED or job.revision != 1:
|
||||||
|
raise HermesConflict("new job must be QUEUED at revision 1")
|
||||||
|
digest = self._digest(job)
|
||||||
|
idem_ref = (job.tenant_id, job.idempotency_key)
|
||||||
|
with self.store.lock:
|
||||||
|
existing = self.store.jobs.get(job.job_id)
|
||||||
|
if existing is not None:
|
||||||
|
if self._digest(existing) != digest:
|
||||||
|
raise HermesConflict("job_id reused with different content")
|
||||||
|
return existing
|
||||||
|
prior = self.store.idempotency.get(idem_ref)
|
||||||
|
if prior is not None:
|
||||||
|
prior_id, prior_digest = prior
|
||||||
|
if prior_digest != digest:
|
||||||
|
raise HermesConflict("idempotency key reused with different content")
|
||||||
|
return self.store.jobs[prior_id]
|
||||||
|
self.store.jobs[job.job_id] = job
|
||||||
|
self.store.idempotency[idem_ref] = (job.job_id, digest)
|
||||||
|
self.store.persist_locked()
|
||||||
|
return job
|
||||||
|
|
||||||
|
def get(self, job_id: str, *, tenant_id: str) -> HermesJob:
|
||||||
|
with self.store.lock:
|
||||||
|
job = self.store.jobs.get(job_id)
|
||||||
|
if job is None:
|
||||||
|
raise HermesNotFound("job not found")
|
||||||
|
self._assert_tenant(job, tenant_id)
|
||||||
|
return job
|
||||||
|
|
||||||
|
def list_jobs(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
cell_id: str | None = None,
|
||||||
|
profile_id: str | None = None,
|
||||||
|
states: Iterable[HermesJobState | str] | None = None,
|
||||||
|
) -> list[HermesJob]:
|
||||||
|
state_filter = {HermesJobState(item) for item in states} if states else None
|
||||||
|
with self.store.lock:
|
||||||
|
if self._requeue_expired_locked():
|
||||||
|
self.store.persist_locked()
|
||||||
|
jobs = [
|
||||||
|
job
|
||||||
|
for job in self.store.jobs.values()
|
||||||
|
if job.tenant_id == tenant_id
|
||||||
|
and (cell_id is None or job.cell_id == cell_id)
|
||||||
|
and (profile_id is None or job.profile_id == profile_id)
|
||||||
|
and (state_filter is None or job.state in state_filter)
|
||||||
|
]
|
||||||
|
return sorted(jobs, key=lambda item: (item.priority, item.created_at, item.job_id))
|
||||||
|
|
||||||
|
def claim(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
cell_id: str,
|
||||||
|
profile_id: str,
|
||||||
|
worker_id: str,
|
||||||
|
lease_seconds: int | None = None,
|
||||||
|
job_id: str | None = None,
|
||||||
|
) -> HermesJob | None:
|
||||||
|
worker_id = str(worker_id or "").strip()
|
||||||
|
if not worker_id:
|
||||||
|
raise ValueError("worker_id is required")
|
||||||
|
duration = int(lease_seconds or self.default_lease_seconds)
|
||||||
|
if duration < 5:
|
||||||
|
raise ValueError("lease_seconds must be >= 5")
|
||||||
|
with self.store.lock:
|
||||||
|
self._requeue_expired_locked()
|
||||||
|
candidates = [
|
||||||
|
job
|
||||||
|
for job in self.store.jobs.values()
|
||||||
|
if job.tenant_id == tenant_id
|
||||||
|
and job.cell_id == cell_id
|
||||||
|
and job.profile_id == profile_id
|
||||||
|
and (job_id is None or job.job_id == job_id)
|
||||||
|
and job.state == HermesJobState.QUEUED
|
||||||
|
]
|
||||||
|
self.store.worker_heartbeats[worker_id] = (utc_now(), None)
|
||||||
|
if not candidates:
|
||||||
|
self.store.persist_locked()
|
||||||
|
return None
|
||||||
|
current = sorted(
|
||||||
|
candidates,
|
||||||
|
key=lambda item: (item.priority, item.created_at, item.job_id),
|
||||||
|
)[0]
|
||||||
|
claimed = self._transition_locked(
|
||||||
|
current,
|
||||||
|
HermesJobState.CLAIMED,
|
||||||
|
assigned_worker_id=worker_id,
|
||||||
|
lease_expires_at=self._format(
|
||||||
|
datetime.now(timezone.utc) + timedelta(seconds=duration)
|
||||||
|
),
|
||||||
|
)
|
||||||
|
self.store.worker_heartbeats[worker_id] = (utc_now(), claimed.job_id)
|
||||||
|
self.store.persist_locked()
|
||||||
|
return claimed
|
||||||
|
|
||||||
|
def start(
|
||||||
|
self,
|
||||||
|
job_id: str,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
worker_id: str,
|
||||||
|
expected_revision: int | None = None,
|
||||||
|
) -> HermesJob:
|
||||||
|
with self.store.lock:
|
||||||
|
self._requeue_expired_locked()
|
||||||
|
current = self._owned_job_locked(job_id, tenant_id, worker_id)
|
||||||
|
self._assert_revision(current, expected_revision)
|
||||||
|
updated = self._transition_locked(current, HermesJobState.RUNNING)
|
||||||
|
self.store.persist_locked()
|
||||||
|
return updated
|
||||||
|
|
||||||
|
def heartbeat(
|
||||||
|
self,
|
||||||
|
job_id: str,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
worker_id: str,
|
||||||
|
lease_seconds: int | None = None,
|
||||||
|
) -> HermesJob:
|
||||||
|
duration = int(lease_seconds or self.default_lease_seconds)
|
||||||
|
if duration < 5:
|
||||||
|
raise ValueError("lease_seconds must be >= 5")
|
||||||
|
with self.store.lock:
|
||||||
|
self._requeue_expired_locked()
|
||||||
|
current = self._owned_job_locked(job_id, tenant_id, worker_id)
|
||||||
|
if current.state not in {HermesJobState.CLAIMED, HermesJobState.RUNNING}:
|
||||||
|
raise HermesConflict("job does not accept heartbeat")
|
||||||
|
updated = replace(
|
||||||
|
current,
|
||||||
|
revision=current.revision + 1,
|
||||||
|
lease_expires_at=self._format(
|
||||||
|
datetime.now(timezone.utc) + timedelta(seconds=duration)
|
||||||
|
),
|
||||||
|
updated_at=utc_now(),
|
||||||
|
)
|
||||||
|
self.store.jobs[job_id] = updated
|
||||||
|
self.store.worker_heartbeats[worker_id] = (utc_now(), job_id)
|
||||||
|
self.store.persist_locked()
|
||||||
|
return updated
|
||||||
|
|
||||||
|
def complete(
|
||||||
|
self,
|
||||||
|
job_id: str,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
worker_id: str,
|
||||||
|
output: Mapping[str, object],
|
||||||
|
expected_revision: int | None = None,
|
||||||
|
) -> HermesResult:
|
||||||
|
return self._finish(
|
||||||
|
job_id,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
worker_id=worker_id,
|
||||||
|
state=HermesJobState.COMPLETED,
|
||||||
|
output=output,
|
||||||
|
expected_revision=expected_revision,
|
||||||
|
)
|
||||||
|
|
||||||
|
def fail(
|
||||||
|
self,
|
||||||
|
job_id: str,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
worker_id: str,
|
||||||
|
error: Mapping[str, object],
|
||||||
|
expected_revision: int | None = None,
|
||||||
|
) -> HermesResult:
|
||||||
|
return self._finish(
|
||||||
|
job_id,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
worker_id=worker_id,
|
||||||
|
state=HermesJobState.FAILED,
|
||||||
|
error=error,
|
||||||
|
expected_revision=expected_revision,
|
||||||
|
)
|
||||||
|
|
||||||
|
def cancel(
|
||||||
|
self,
|
||||||
|
job_id: str,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
expected_revision: int | None = None,
|
||||||
|
) -> HermesResult:
|
||||||
|
with self.store.lock:
|
||||||
|
current = self.store.jobs.get(job_id)
|
||||||
|
if current is None:
|
||||||
|
raise HermesNotFound("job not found")
|
||||||
|
self._assert_tenant(current, tenant_id)
|
||||||
|
self._assert_revision(current, expected_revision)
|
||||||
|
if current.state in TERMINAL_STATES:
|
||||||
|
result = self.store.results.get(job_id)
|
||||||
|
if result is None:
|
||||||
|
raise HermesConflict("terminal result missing")
|
||||||
|
if result.state != HermesJobState.CANCELLED:
|
||||||
|
raise HermesConflict("job already has a different terminal result")
|
||||||
|
return result
|
||||||
|
updated = self._transition_locked(current, HermesJobState.CANCELLED)
|
||||||
|
result = HermesResult(
|
||||||
|
job_id=updated.job_id,
|
||||||
|
tenant_id=updated.tenant_id,
|
||||||
|
task_id=updated.task_id,
|
||||||
|
trace_id=updated.trace_id,
|
||||||
|
state=HermesJobState.CANCELLED,
|
||||||
|
worker_id=updated.assigned_worker_id,
|
||||||
|
error={"code": "cancelled", "message": "cancelled by AIOS"},
|
||||||
|
finished_at=utc_now(),
|
||||||
|
)
|
||||||
|
self.store.results[job_id] = result
|
||||||
|
self.store.persist_locked()
|
||||||
|
return result
|
||||||
|
|
||||||
|
def result(self, job_id: str, *, tenant_id: str) -> HermesResult:
|
||||||
|
with self.store.lock:
|
||||||
|
job = self.store.jobs.get(job_id)
|
||||||
|
if job is None:
|
||||||
|
raise HermesNotFound("job not found")
|
||||||
|
self._assert_tenant(job, tenant_id)
|
||||||
|
result = self.store.results.get(job_id)
|
||||||
|
if result is None:
|
||||||
|
raise HermesConflict("job has no terminal result")
|
||||||
|
return result
|
||||||
|
|
||||||
|
def requeue_expired(self) -> tuple[str, ...]:
|
||||||
|
"""Persistently release expired leases without changing policy state."""
|
||||||
|
with self.store.lock:
|
||||||
|
changed = self._requeue_expired_locked()
|
||||||
|
if changed:
|
||||||
|
self.store.persist_locked()
|
||||||
|
return tuple(sorted(changed))
|
||||||
|
|
||||||
|
def health(self) -> HermesHealth:
|
||||||
|
with self.store.lock:
|
||||||
|
if self._requeue_expired_locked():
|
||||||
|
self.store.persist_locked()
|
||||||
|
active_jobs = sum(
|
||||||
|
1 for job in self.store.jobs.values()
|
||||||
|
if job.state in {HermesJobState.CLAIMED, HermesJobState.RUNNING}
|
||||||
|
)
|
||||||
|
queued_jobs = sum(
|
||||||
|
1 for job in self.store.jobs.values()
|
||||||
|
if job.state == HermesJobState.QUEUED
|
||||||
|
)
|
||||||
|
partitions = {
|
||||||
|
(job.tenant_id, job.cell_id, job.profile_id)
|
||||||
|
for job in self.store.jobs.values()
|
||||||
|
}
|
||||||
|
active_workers = sum(
|
||||||
|
1 for heartbeat, _ in self.store.worker_heartbeats.values()
|
||||||
|
if self._parse(heartbeat)
|
||||||
|
and self._parse(heartbeat) >= datetime.now(timezone.utc) - timedelta(seconds=120)
|
||||||
|
)
|
||||||
|
return HermesHealth(
|
||||||
|
runtime_id=self.runtime_id,
|
||||||
|
status="healthy",
|
||||||
|
version=self.runtime_version,
|
||||||
|
heartbeat_at=utc_now(),
|
||||||
|
active_workers=active_workers,
|
||||||
|
active_jobs=active_jobs,
|
||||||
|
queued_jobs=queued_jobs,
|
||||||
|
tenant_partitions=len(partitions),
|
||||||
|
profile_isolation_enforced=True,
|
||||||
|
details={"store": type(self.store).__name__},
|
||||||
|
)
|
||||||
|
|
||||||
|
def _finish(
|
||||||
|
self,
|
||||||
|
job_id: str,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
worker_id: str,
|
||||||
|
state: HermesJobState,
|
||||||
|
output: Mapping[str, object] | None = None,
|
||||||
|
error: Mapping[str, object] | None = None,
|
||||||
|
expected_revision: int | None,
|
||||||
|
) -> HermesResult:
|
||||||
|
with self.store.lock:
|
||||||
|
self._requeue_expired_locked()
|
||||||
|
current = self._owned_job_locked(job_id, tenant_id, worker_id)
|
||||||
|
if current.state in TERMINAL_STATES:
|
||||||
|
existing = self.store.results.get(job_id)
|
||||||
|
if existing is None:
|
||||||
|
raise HermesConflict("terminal result missing")
|
||||||
|
if (
|
||||||
|
existing.state == state
|
||||||
|
and existing.worker_id == worker_id
|
||||||
|
and existing.output == output
|
||||||
|
and existing.error == error
|
||||||
|
):
|
||||||
|
return existing
|
||||||
|
raise HermesConflict("job already has a different terminal result")
|
||||||
|
self._assert_revision(current, expected_revision)
|
||||||
|
updated = self._transition_locked(current, state)
|
||||||
|
result = HermesResult(
|
||||||
|
job_id=updated.job_id,
|
||||||
|
tenant_id=updated.tenant_id,
|
||||||
|
task_id=updated.task_id,
|
||||||
|
trace_id=updated.trace_id,
|
||||||
|
state=state,
|
||||||
|
worker_id=worker_id,
|
||||||
|
output=output,
|
||||||
|
error=error,
|
||||||
|
started_at=current.updated_at,
|
||||||
|
finished_at=utc_now(),
|
||||||
|
)
|
||||||
|
self.store.results[job_id] = result
|
||||||
|
self.store.worker_heartbeats[worker_id] = (utc_now(), None)
|
||||||
|
self.store.persist_locked()
|
||||||
|
return result
|
||||||
|
|
||||||
|
def _owned_job_locked(self, job_id: str, tenant_id: str, worker_id: str) -> HermesJob:
|
||||||
|
current = self.store.jobs.get(job_id)
|
||||||
|
if current is None:
|
||||||
|
raise HermesNotFound("job not found")
|
||||||
|
self._assert_tenant(current, tenant_id)
|
||||||
|
if current.assigned_worker_id != worker_id:
|
||||||
|
raise HermesConflict("worker does not own job lease")
|
||||||
|
return current
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _assert_revision(job: HermesJob, expected: int | None) -> None:
|
||||||
|
if expected is not None and job.revision != expected:
|
||||||
|
raise HermesConflict("job revision changed")
|
||||||
|
|
||||||
|
def _transition_locked(
|
||||||
|
self,
|
||||||
|
current: HermesJob,
|
||||||
|
target: HermesJobState,
|
||||||
|
*,
|
||||||
|
assigned_worker_id: str | None = None,
|
||||||
|
lease_expires_at: str | None = None,
|
||||||
|
) -> HermesJob:
|
||||||
|
if target not in ALLOWED_TRANSITIONS[current.state]:
|
||||||
|
raise HermesConflict(
|
||||||
|
f"invalid transition {current.state.value} -> {target.value}"
|
||||||
|
)
|
||||||
|
updated = replace(
|
||||||
|
current,
|
||||||
|
state=target,
|
||||||
|
revision=current.revision + 1,
|
||||||
|
assigned_worker_id=(
|
||||||
|
assigned_worker_id
|
||||||
|
if assigned_worker_id is not None
|
||||||
|
else current.assigned_worker_id
|
||||||
|
),
|
||||||
|
lease_expires_at=(
|
||||||
|
lease_expires_at
|
||||||
|
if lease_expires_at is not None
|
||||||
|
else (None if target in TERMINAL_STATES else current.lease_expires_at)
|
||||||
|
),
|
||||||
|
updated_at=utc_now(),
|
||||||
|
)
|
||||||
|
self.store.jobs[current.job_id] = updated
|
||||||
|
return updated
|
||||||
|
|
||||||
|
def _requeue_expired_locked(self) -> set[str]:
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
changed: set[str] = set()
|
||||||
|
for current in list(self.store.jobs.values()):
|
||||||
|
expiry = self._parse(current.lease_expires_at)
|
||||||
|
if (
|
||||||
|
current.state in {HermesJobState.CLAIMED, HermesJobState.RUNNING}
|
||||||
|
and expiry is not None
|
||||||
|
and expiry <= now
|
||||||
|
):
|
||||||
|
self.store.jobs[current.job_id] = replace(
|
||||||
|
current,
|
||||||
|
state=HermesJobState.QUEUED,
|
||||||
|
revision=current.revision + 1,
|
||||||
|
assigned_worker_id=None,
|
||||||
|
lease_expires_at=None,
|
||||||
|
updated_at=utc_now(),
|
||||||
|
)
|
||||||
|
changed.add(current.job_id)
|
||||||
|
return changed
|
||||||
@@ -0,0 +1,153 @@
|
|||||||
|
"""Tenant-scoped Hermes runtime contracts."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import uuid
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from enum import Enum
|
||||||
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
|
||||||
|
def utc_now() -> str:
|
||||||
|
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||||
|
|
||||||
|
|
||||||
|
def _required(value: str, label: str) -> str:
|
||||||
|
normalized = str(value or "").strip()
|
||||||
|
if not normalized:
|
||||||
|
raise ValueError(f"{label} is required")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
def _safe_mapping(value: Mapping[str, Any] | None) -> dict[str, Any]:
|
||||||
|
return json.loads(json.dumps(dict(value or {}), ensure_ascii=False, sort_keys=True, default=str))
|
||||||
|
|
||||||
|
|
||||||
|
class HermesJobState(str, Enum):
|
||||||
|
QUEUED = "QUEUED"
|
||||||
|
CLAIMED = "CLAIMED"
|
||||||
|
RUNNING = "RUNNING"
|
||||||
|
COMPLETED = "COMPLETED"
|
||||||
|
FAILED = "FAILED"
|
||||||
|
CANCELLED = "CANCELLED"
|
||||||
|
|
||||||
|
|
||||||
|
ALLOWED_TRANSITIONS: Mapping[HermesJobState, frozenset[HermesJobState]] = {
|
||||||
|
HermesJobState.QUEUED: frozenset({HermesJobState.CLAIMED, HermesJobState.CANCELLED}),
|
||||||
|
HermesJobState.CLAIMED: frozenset({HermesJobState.RUNNING, HermesJobState.CANCELLED}),
|
||||||
|
HermesJobState.RUNNING: frozenset(
|
||||||
|
{HermesJobState.COMPLETED, HermesJobState.FAILED, HermesJobState.CANCELLED}
|
||||||
|
),
|
||||||
|
HermesJobState.COMPLETED: frozenset(),
|
||||||
|
HermesJobState.FAILED: frozenset(),
|
||||||
|
HermesJobState.CANCELLED: frozenset(),
|
||||||
|
}
|
||||||
|
TERMINAL_STATES = frozenset(
|
||||||
|
{HermesJobState.COMPLETED, HermesJobState.FAILED, HermesJobState.CANCELLED}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class HermesJob:
|
||||||
|
job_id: str
|
||||||
|
tenant_id: str
|
||||||
|
cell_id: str
|
||||||
|
profile_id: str
|
||||||
|
task_id: str
|
||||||
|
trace_id: str
|
||||||
|
action: str
|
||||||
|
payload: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
idempotency_key: str = ""
|
||||||
|
priority: int = 100
|
||||||
|
state: HermesJobState = HermesJobState.QUEUED
|
||||||
|
revision: int = 1
|
||||||
|
assigned_worker_id: str | None = None
|
||||||
|
lease_expires_at: str | None = None
|
||||||
|
created_at: str = field(default_factory=utc_now)
|
||||||
|
updated_at: str = field(default_factory=utc_now)
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for name in (
|
||||||
|
"job_id", "tenant_id", "cell_id", "profile_id", "task_id",
|
||||||
|
"trace_id", "action", "idempotency_key", "created_at", "updated_at",
|
||||||
|
):
|
||||||
|
object.__setattr__(self, name, _required(getattr(self, name), name))
|
||||||
|
object.__setattr__(self, "payload", _safe_mapping(self.payload))
|
||||||
|
object.__setattr__(self, "state", HermesJobState(self.state))
|
||||||
|
if not 0 <= int(self.priority) <= 1000:
|
||||||
|
raise ValueError("priority must be between 0 and 1000")
|
||||||
|
if self.revision < 1:
|
||||||
|
raise ValueError("revision must be >= 1")
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def create(
|
||||||
|
cls,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
cell_id: str,
|
||||||
|
profile_id: str,
|
||||||
|
task_id: str,
|
||||||
|
trace_id: str,
|
||||||
|
action: str,
|
||||||
|
payload: Mapping[str, Any] | None,
|
||||||
|
idempotency_key: str,
|
||||||
|
priority: int = 100,
|
||||||
|
) -> "HermesJob":
|
||||||
|
return cls(
|
||||||
|
job_id=f"hermes_{uuid.uuid4().hex}",
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
cell_id=cell_id,
|
||||||
|
profile_id=profile_id,
|
||||||
|
task_id=task_id,
|
||||||
|
trace_id=trace_id,
|
||||||
|
action=action,
|
||||||
|
payload=_safe_mapping(payload),
|
||||||
|
idempotency_key=idempotency_key,
|
||||||
|
priority=priority,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class HermesResult:
|
||||||
|
job_id: str
|
||||||
|
tenant_id: str
|
||||||
|
task_id: str
|
||||||
|
trace_id: str
|
||||||
|
state: HermesJobState
|
||||||
|
worker_id: str | None = None
|
||||||
|
output: Mapping[str, Any] | None = None
|
||||||
|
error: Mapping[str, Any] | None = None
|
||||||
|
started_at: str | None = None
|
||||||
|
finished_at: str | None = None
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for name in ("job_id", "tenant_id", "task_id", "trace_id"):
|
||||||
|
object.__setattr__(self, name, _required(getattr(self, name), name))
|
||||||
|
object.__setattr__(self, "state", HermesJobState(self.state))
|
||||||
|
if self.output is not None:
|
||||||
|
object.__setattr__(self, "output", _safe_mapping(self.output))
|
||||||
|
if self.error is not None:
|
||||||
|
object.__setattr__(self, "error", _safe_mapping(self.error))
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class HermesHealth:
|
||||||
|
runtime_id: str
|
||||||
|
status: str
|
||||||
|
version: str
|
||||||
|
heartbeat_at: str
|
||||||
|
active_workers: int = 0
|
||||||
|
active_jobs: int = 0
|
||||||
|
queued_jobs: int = 0
|
||||||
|
tenant_partitions: int = 0
|
||||||
|
profile_isolation_enforced: bool = True
|
||||||
|
details: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for name in ("runtime_id", "status", "version", "heartbeat_at"):
|
||||||
|
object.__setattr__(self, name, _required(getattr(self, name), name))
|
||||||
|
for name in ("active_workers", "active_jobs", "queued_jobs", "tenant_partitions"):
|
||||||
|
if int(getattr(self, name)) < 0:
|
||||||
|
raise ValueError(f"{name} must be >= 0")
|
||||||
|
object.__setattr__(self, "details", _safe_mapping(self.details))
|
||||||
@@ -0,0 +1,155 @@
|
|||||||
|
"""Hermes worker loop with lease renewal and bounded recovery."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import threading
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any, Callable, Mapping
|
||||||
|
|
||||||
|
from .adapter import HermesAdapter, HermesConflict
|
||||||
|
from .contract import HermesJob, HermesJobState
|
||||||
|
|
||||||
|
|
||||||
|
JobHandler = Callable[[HermesJob], Mapping[str, Any]]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class WorkerRunResult:
|
||||||
|
claimed: bool
|
||||||
|
job_id: str | None = None
|
||||||
|
state: str | None = None
|
||||||
|
error: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class HermesWorker:
|
||||||
|
"""Poll one tenant/cell/profile partition and execute jobs safely."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
adapter: HermesAdapter,
|
||||||
|
handler: JobHandler,
|
||||||
|
*,
|
||||||
|
worker_id: str,
|
||||||
|
tenant_id: str,
|
||||||
|
cell_id: str,
|
||||||
|
profile_id: str,
|
||||||
|
poll_seconds: float = 1.0,
|
||||||
|
heartbeat_seconds: float = 10.0,
|
||||||
|
) -> None:
|
||||||
|
if poll_seconds <= 0 or heartbeat_seconds <= 0:
|
||||||
|
raise ValueError("poll_seconds and heartbeat_seconds must be positive")
|
||||||
|
for name, value in {
|
||||||
|
"worker_id": worker_id,
|
||||||
|
"tenant_id": tenant_id,
|
||||||
|
"cell_id": cell_id,
|
||||||
|
"profile_id": profile_id,
|
||||||
|
}.items():
|
||||||
|
if not str(value or "").strip():
|
||||||
|
raise ValueError(f"{name} is required")
|
||||||
|
self.adapter = adapter
|
||||||
|
self.handler = handler
|
||||||
|
self.worker_id = worker_id
|
||||||
|
self.tenant_id = tenant_id
|
||||||
|
self.cell_id = cell_id
|
||||||
|
self.profile_id = profile_id
|
||||||
|
self.poll_seconds = float(poll_seconds)
|
||||||
|
self.heartbeat_seconds = float(heartbeat_seconds)
|
||||||
|
self._stop = threading.Event()
|
||||||
|
|
||||||
|
def stop(self) -> None:
|
||||||
|
self._stop.set()
|
||||||
|
|
||||||
|
def run_once(self) -> WorkerRunResult:
|
||||||
|
job = self.adapter.claim(
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
cell_id=self.cell_id,
|
||||||
|
profile_id=self.profile_id,
|
||||||
|
worker_id=self.worker_id,
|
||||||
|
)
|
||||||
|
if job is None:
|
||||||
|
return WorkerRunResult(claimed=False)
|
||||||
|
|
||||||
|
running = self.adapter.start(
|
||||||
|
job.job_id,
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
worker_id=self.worker_id,
|
||||||
|
expected_revision=job.revision,
|
||||||
|
)
|
||||||
|
|
||||||
|
heartbeat_stop = threading.Event()
|
||||||
|
heartbeat_thread = threading.Thread(
|
||||||
|
target=self._heartbeat_loop,
|
||||||
|
args=(running.job_id, heartbeat_stop),
|
||||||
|
name=f"hermes-heartbeat-{self.worker_id}",
|
||||||
|
daemon=True,
|
||||||
|
)
|
||||||
|
heartbeat_thread.start()
|
||||||
|
|
||||||
|
output: Mapping[str, Any] | None = None
|
||||||
|
execution_error: Exception | None = None
|
||||||
|
try:
|
||||||
|
output = dict(self.handler(running))
|
||||||
|
except Exception as exc: # runtime boundary: normalize below
|
||||||
|
execution_error = exc
|
||||||
|
finally:
|
||||||
|
# Stop the lease heartbeat before reading the final revision. This
|
||||||
|
# prevents a revision increment racing with complete/fail.
|
||||||
|
heartbeat_stop.set()
|
||||||
|
heartbeat_thread.join(timeout=max(self.heartbeat_seconds * 2, 1.0))
|
||||||
|
|
||||||
|
try:
|
||||||
|
current = self.adapter.get(running.job_id, tenant_id=self.tenant_id)
|
||||||
|
if execution_error is None:
|
||||||
|
result = self.adapter.complete(
|
||||||
|
running.job_id,
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
worker_id=self.worker_id,
|
||||||
|
output=dict(output or {}),
|
||||||
|
expected_revision=current.revision,
|
||||||
|
)
|
||||||
|
return WorkerRunResult(
|
||||||
|
claimed=True,
|
||||||
|
job_id=result.job_id,
|
||||||
|
state=result.state.value,
|
||||||
|
)
|
||||||
|
|
||||||
|
result = self.adapter.fail(
|
||||||
|
running.job_id,
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
worker_id=self.worker_id,
|
||||||
|
error={
|
||||||
|
"code": "worker_error",
|
||||||
|
"message": str(execution_error),
|
||||||
|
},
|
||||||
|
expected_revision=current.revision,
|
||||||
|
)
|
||||||
|
return WorkerRunResult(
|
||||||
|
claimed=True,
|
||||||
|
job_id=result.job_id,
|
||||||
|
state=result.state.value,
|
||||||
|
error=str(execution_error),
|
||||||
|
)
|
||||||
|
except HermesConflict as exc:
|
||||||
|
return WorkerRunResult(
|
||||||
|
claimed=True,
|
||||||
|
job_id=running.job_id,
|
||||||
|
state=HermesJobState.FAILED.value,
|
||||||
|
error=f"reconciliation_conflict: {exc}",
|
||||||
|
)
|
||||||
|
|
||||||
|
def run_forever(self) -> None:
|
||||||
|
while not self._stop.is_set():
|
||||||
|
result = self.run_once()
|
||||||
|
if not result.claimed:
|
||||||
|
self._stop.wait(self.poll_seconds)
|
||||||
|
|
||||||
|
def _heartbeat_loop(self, job_id: str, stop_event: threading.Event) -> None:
|
||||||
|
while not stop_event.wait(self.heartbeat_seconds):
|
||||||
|
try:
|
||||||
|
self.adapter.heartbeat(
|
||||||
|
job_id,
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
worker_id=self.worker_id,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
stop_event.set()
|
||||||
|
return
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
"""Task-bound Integration Refresh control-plane primitives."""
|
||||||
|
|
||||||
|
from .integration_merge_writer import IntegrationMergeWriter, MergeError, MergeRequest
|
||||||
|
from .integration_refresh import (
|
||||||
|
AppendOnlyAudit,
|
||||||
|
Binding,
|
||||||
|
GiteaBranchReadProjection,
|
||||||
|
GrantStore,
|
||||||
|
IntegrationRefreshProvisioner,
|
||||||
|
ProvisioningError,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"AppendOnlyAudit",
|
||||||
|
"Binding",
|
||||||
|
"GiteaBranchReadProjection",
|
||||||
|
"GrantStore",
|
||||||
|
"IntegrationMergeWriter",
|
||||||
|
"IntegrationRefreshProvisioner",
|
||||||
|
"MergeError",
|
||||||
|
"MergeRequest",
|
||||||
|
"ProvisioningError",
|
||||||
|
]
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
"""Constrained integration-only merge writer with rollback and reconciliation states."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
|
||||||
|
|
||||||
|
class MergeError(RuntimeError):
|
||||||
|
def __init__(self, code: str, status: int = 409):
|
||||||
|
super().__init__(code); self.code, self.status = code, status
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class MergeRequest:
|
||||||
|
integration_branch: str
|
||||||
|
source_branch: str
|
||||||
|
approved_source_oid: str
|
||||||
|
|
||||||
|
|
||||||
|
class IntegrationMergeWriter:
|
||||||
|
def __init__(self, workspace: Path | str, remote: str, projection):
|
||||||
|
self.workspace, self.remote, self.projection = Path(workspace), remote, projection
|
||||||
|
|
||||||
|
def _git(self, *args: str, check=True) -> str:
|
||||||
|
p = subprocess.run(["git", *args], cwd=self.workspace, text=True, capture_output=True)
|
||||||
|
if check and p.returncode:
|
||||||
|
raise MergeError("git_operation_failed", 500)
|
||||||
|
return p.stdout.strip()
|
||||||
|
|
||||||
|
def merge(self, request: MergeRequest) -> dict:
|
||||||
|
def bad(value: str) -> bool:
|
||||||
|
return (
|
||||||
|
value == "main"
|
||||||
|
or value.startswith("main/")
|
||||||
|
or value == "governance"
|
||||||
|
or value.startswith("governance/")
|
||||||
|
or value == "approvals"
|
||||||
|
or value.startswith("refs/")
|
||||||
|
or value.startswith("tags/")
|
||||||
|
)
|
||||||
|
if (bad(request.integration_branch) or bad(request.source_branch) or
|
||||||
|
not request.integration_branch.startswith("integration/") or
|
||||||
|
re.fullmatch(r"[0-9a-fA-F]{40}", request.approved_source_oid) is None):
|
||||||
|
raise MergeError("integration_merge_scope_rejected", 403)
|
||||||
|
for branch in (request.integration_branch, request.source_branch):
|
||||||
|
if subprocess.run(["git", "check-ref-format", "--branch", branch], cwd=self.workspace,
|
||||||
|
text=True, capture_output=True).returncode:
|
||||||
|
raise MergeError("integration_merge_scope_rejected", 403)
|
||||||
|
if self._git("rev-parse", "--abbrev-ref", "HEAD") != request.integration_branch:
|
||||||
|
raise MergeError("integration_branch_checkout_required")
|
||||||
|
before = self._git("rev-parse", "HEAD")
|
||||||
|
remote_before = self.projection(request.integration_branch)
|
||||||
|
if remote_before != before:
|
||||||
|
raise MergeError("integration_remote_advanced")
|
||||||
|
source = self._git("rev-parse", "--verify", f"refs/heads/{request.source_branch}^{{commit}}")
|
||||||
|
if source != request.approved_source_oid:
|
||||||
|
raise MergeError("integration_source_not_approved", 403)
|
||||||
|
attempt = subprocess.run(["git", "merge", "--no-ff", "--no-commit", request.source_branch], cwd=self.workspace, text=True, capture_output=True)
|
||||||
|
if attempt.returncode:
|
||||||
|
subprocess.run(["git", "merge", "--abort"], cwd=self.workspace, text=True, capture_output=True)
|
||||||
|
raise MergeError("integration_merge_conflict")
|
||||||
|
try:
|
||||||
|
self._git("commit", "-m", "merge(integration): approved source")
|
||||||
|
merged = self._git("rev-parse", "HEAD")
|
||||||
|
parents = self._git("show", "-s", "--format=%P", "HEAD").split()
|
||||||
|
if parents != [before, source]:
|
||||||
|
raise MergeError("integration_parent_invariant_failed", 500)
|
||||||
|
pushed = subprocess.run(["git", "push", "--porcelain", self.remote, f"HEAD:refs/heads/{request.integration_branch}"], cwd=self.workspace, text=True, capture_output=True)
|
||||||
|
if pushed.returncode:
|
||||||
|
try:
|
||||||
|
observed = self.projection(request.integration_branch)
|
||||||
|
except Exception:
|
||||||
|
return {"state": "RECONCILIATION_REQUIRED", "pre_merge_oid": before, "merge_oid": merged}
|
||||||
|
if observed == before:
|
||||||
|
self._git("reset", "--hard", before)
|
||||||
|
return {"state": "PUSH_FAILED_RESTORED", "pre_merge_oid": before}
|
||||||
|
return {"state": "RECONCILIATION_REQUIRED", "pre_merge_oid": before, "merge_oid": merged}
|
||||||
|
try:
|
||||||
|
observed = self.projection(request.integration_branch)
|
||||||
|
except Exception:
|
||||||
|
return {"state": "RECONCILIATION_REQUIRED", "pre_merge_oid": before, "merge_oid": merged}
|
||||||
|
if observed != merged:
|
||||||
|
return {"state": "RECONCILIATION_REQUIRED", "pre_merge_oid": before, "merge_oid": merged}
|
||||||
|
return {"state": "COMPLETED", "pre_merge_oid": before, "merge_oid": merged}
|
||||||
|
except MergeError:
|
||||||
|
if self._git("rev-parse", "HEAD") != before:
|
||||||
|
self._git("reset", "--hard", before)
|
||||||
|
raise
|
||||||
@@ -0,0 +1,257 @@
|
|||||||
|
"""Fail-closed, task-bound integration refresh provisioning primitives.
|
||||||
|
|
||||||
|
This module deliberately has no HTTP or credential-provider dependency. The
|
||||||
|
caller supplies a branch-read projection and an apply callback, making the
|
||||||
|
authorization and recovery boundary independently testable.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import argparse
|
||||||
|
import secrets
|
||||||
|
import sqlite3
|
||||||
|
import threading
|
||||||
|
import re
|
||||||
|
from contextlib import contextmanager
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Callable
|
||||||
|
|
||||||
|
|
||||||
|
class ProvisioningError(RuntimeError):
|
||||||
|
def __init__(self, code: str, status: int = 403):
|
||||||
|
super().__init__(code)
|
||||||
|
self.code, self.status = code, status
|
||||||
|
|
||||||
|
|
||||||
|
def _now() -> datetime:
|
||||||
|
return datetime.now(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
def _digest(secret: str) -> str:
|
||||||
|
return hashlib.sha256(secret.encode("utf-8")).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _canonical(value: dict) -> str:
|
||||||
|
return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(",", ":")).encode()).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _is_oid(value: object) -> bool:
|
||||||
|
return isinstance(value, str) and re.fullmatch(r"[0-9a-fA-F]{40}", value) is not None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Binding:
|
||||||
|
task_id: str
|
||||||
|
repository: str
|
||||||
|
branch: str
|
||||||
|
actor_identity: str
|
||||||
|
audience: str
|
||||||
|
paths: tuple[str, ...]
|
||||||
|
quota: int
|
||||||
|
|
||||||
|
|
||||||
|
class AppendOnlyAudit:
|
||||||
|
"""Append only redacted JSONL audit, durable before a result is returned."""
|
||||||
|
def __init__(self, path: Path | str):
|
||||||
|
self.path = Path(path)
|
||||||
|
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
|
||||||
|
def append(self, event: dict) -> None:
|
||||||
|
with self._lock:
|
||||||
|
safe = {key: value for key, value in event.items() if key not in {"token", "credential", "authorization"}}
|
||||||
|
previous = "0" * 64
|
||||||
|
if self.path.exists():
|
||||||
|
lines = self.path.read_text(encoding="utf-8").splitlines()
|
||||||
|
if lines:
|
||||||
|
previous = json.loads(lines[-1])["event_hash"]
|
||||||
|
safe["previous_event_hash"] = previous
|
||||||
|
safe["event_hash"] = _canonical(safe)
|
||||||
|
encoded = (json.dumps(safe, sort_keys=True, separators=(",", ":")) + "\n").encode()
|
||||||
|
fd = os.open(self.path, os.O_APPEND | os.O_CREAT | os.O_WRONLY, 0o600)
|
||||||
|
try:
|
||||||
|
os.write(fd, encoded)
|
||||||
|
os.fsync(fd)
|
||||||
|
finally:
|
||||||
|
os.close(fd)
|
||||||
|
|
||||||
|
|
||||||
|
class GrantStore:
|
||||||
|
"""Digest-only short grant registry; SQLite transactions serialize quota use."""
|
||||||
|
def __init__(self, path: Path | str):
|
||||||
|
self.path = Path(path)
|
||||||
|
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with self._db() as db:
|
||||||
|
db.execute("""CREATE TABLE IF NOT EXISTS grants(
|
||||||
|
token_digest TEXT PRIMARY KEY, binding_json TEXT NOT NULL,
|
||||||
|
expires_at TEXT NOT NULL, quota INTEGER NOT NULL, uses INTEGER NOT NULL,
|
||||||
|
revoked INTEGER NOT NULL DEFAULT 0)""")
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def _db(self):
|
||||||
|
db = sqlite3.connect(self.path, timeout=15, isolation_level=None)
|
||||||
|
db.row_factory = sqlite3.Row
|
||||||
|
try:
|
||||||
|
yield db
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
def issue(self, binding: Binding, *, ttl_seconds: int = 300) -> str:
|
||||||
|
if ttl_seconds < 1 or ttl_seconds > 1800 or binding.quota < 1 or not binding.paths:
|
||||||
|
raise ValueError("invalid_short_grant")
|
||||||
|
token = secrets.token_urlsafe(32)
|
||||||
|
payload = json.dumps(binding.__dict__ | {"paths": list(binding.paths)}, sort_keys=True)
|
||||||
|
with self._db() as db:
|
||||||
|
db.execute("INSERT INTO grants VALUES(?,?,?,?,?,0)", (_digest(token), payload,
|
||||||
|
(_now() + timedelta(seconds=ttl_seconds)).isoformat(), binding.quota, 0))
|
||||||
|
return token
|
||||||
|
|
||||||
|
def rotate(self, old_token: str, binding: Binding, *, revocation_probe, ttl_seconds: int = 300) -> str:
|
||||||
|
self.revoke(old_token)
|
||||||
|
try: status = revocation_probe(old_token)
|
||||||
|
except TimeoutError as exc: raise ProvisioningError("revocation_probe_timeout", 503) from exc
|
||||||
|
if status != 401: raise ProvisioningError("revocation_probe_rejected", 503)
|
||||||
|
return self.issue(binding, ttl_seconds=ttl_seconds)
|
||||||
|
|
||||||
|
def revoke(self, token: str) -> None:
|
||||||
|
with self._db() as db:
|
||||||
|
db.execute("BEGIN IMMEDIATE")
|
||||||
|
db.execute("UPDATE grants SET revoked=1 WHERE token_digest=?", (_digest(token),))
|
||||||
|
db.execute("COMMIT")
|
||||||
|
|
||||||
|
def consume(self, token: str, binding: Binding, path: str) -> None:
|
||||||
|
with self._db() as db:
|
||||||
|
db.execute("BEGIN IMMEDIATE")
|
||||||
|
row = db.execute("SELECT * FROM grants WHERE token_digest=?", (_digest(token),)).fetchone()
|
||||||
|
if not row or row["revoked"]:
|
||||||
|
db.execute("ROLLBACK"); raise ProvisioningError("credential_rejected", 401)
|
||||||
|
if _now() >= datetime.fromisoformat(row["expires_at"]):
|
||||||
|
db.execute("ROLLBACK"); raise ProvisioningError("credential_expired", 401)
|
||||||
|
stored = json.loads(row["binding_json"])
|
||||||
|
requested = binding.__dict__ | {"paths": list(binding.paths)}
|
||||||
|
if stored != requested or path not in stored["paths"]:
|
||||||
|
db.execute("ROLLBACK"); raise ProvisioningError("grant_binding_rejected")
|
||||||
|
if row["uses"] >= row["quota"]:
|
||||||
|
db.execute("ROLLBACK"); raise ProvisioningError("grant_quota_exhausted")
|
||||||
|
db.execute("UPDATE grants SET uses=uses+1 WHERE token_digest=?", (_digest(token),))
|
||||||
|
db.execute("COMMIT")
|
||||||
|
|
||||||
|
|
||||||
|
class GiteaBranchReadProjection:
|
||||||
|
"""Normalizes a trusted task-scoped Gitea reader; every ambiguity fails closed."""
|
||||||
|
def __init__(self, reader: Callable[[str, str], dict]):
|
||||||
|
self.reader = reader
|
||||||
|
|
||||||
|
def head(self, repository: str, branch: str) -> str:
|
||||||
|
try:
|
||||||
|
reply = self.reader(repository, branch)
|
||||||
|
except TimeoutError as exc:
|
||||||
|
raise ProvisioningError("gitea_projection_timeout", 503) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
raise ProvisioningError("gitea_projection_unavailable", 503) from exc
|
||||||
|
if not isinstance(reply, dict) or reply.get("status") in {401, 403}:
|
||||||
|
raise ProvisioningError("gitea_projection_forbidden", 503)
|
||||||
|
oid = reply.get("oid")
|
||||||
|
if reply.get("status") != 200 or not _is_oid(oid):
|
||||||
|
raise ProvisioningError("gitea_projection_invalid", 503)
|
||||||
|
return oid.lower()
|
||||||
|
|
||||||
|
|
||||||
|
class IntegrationRefreshProvisioner:
|
||||||
|
_locks: dict[tuple[str, str], threading.Lock] = {}
|
||||||
|
_locks_guard = threading.Lock()
|
||||||
|
|
||||||
|
def __init__(self, *, grants: GrantStore, projection: GiteaBranchReadProjection,
|
||||||
|
ledger_path: Path | str, audit: AppendOnlyAudit, approved_binding: Binding):
|
||||||
|
self.grants, self.projection, self.audit, self.approved_binding = grants, projection, audit, approved_binding
|
||||||
|
self.ledger_path = Path(ledger_path); self.ledger_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with self._ledger() as db:
|
||||||
|
db.execute("CREATE TABLE IF NOT EXISTS refreshes(operation_id TEXT PRIMARY KEY, digest TEXT NOT NULL, state TEXT NOT NULL, result_oid TEXT)")
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def _ledger(self):
|
||||||
|
db = sqlite3.connect(self.ledger_path)
|
||||||
|
try:
|
||||||
|
yield db
|
||||||
|
db.commit()
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
def refresh(self, *, token: str, binding: Binding, path: str, operation_id: str,
|
||||||
|
target_head_oid: str, apply: Callable[[], str]) -> dict:
|
||||||
|
if not operation_id or not _is_oid(target_head_oid):
|
||||||
|
raise ProvisioningError("refresh_request_rejected")
|
||||||
|
if binding != self.approved_binding:
|
||||||
|
raise ProvisioningError("grant_binding_rejected")
|
||||||
|
request = {"binding": binding.__dict__ | {"paths": list(binding.paths)}, "path": path, "target": target_head_oid}
|
||||||
|
digest = _canonical(request)
|
||||||
|
with self._locks_guard:
|
||||||
|
lock = self._locks.setdefault((binding.repository, binding.branch), threading.Lock())
|
||||||
|
with lock, self._ledger() as db:
|
||||||
|
db.execute("BEGIN IMMEDIATE")
|
||||||
|
row = db.execute("SELECT * FROM refreshes WHERE operation_id=?", (operation_id,)).fetchone()
|
||||||
|
new_operation = row is None
|
||||||
|
if row:
|
||||||
|
if row[1] != digest:
|
||||||
|
db.execute("ROLLBACK"); raise ProvisioningError("operation_id_reuse_rejected", 409)
|
||||||
|
if row[2] == "COMPLETED":
|
||||||
|
db.execute("COMMIT")
|
||||||
|
return {"state": row[2], "operation_id": operation_id, "oid": row[3], "idempotent": True}
|
||||||
|
if row[2] not in {"PENDING", "RECOVERABLE"}:
|
||||||
|
db.execute("COMMIT")
|
||||||
|
return {"state": row[2], "operation_id": operation_id, "oid": row[3], "idempotent": True}
|
||||||
|
else:
|
||||||
|
db.execute("INSERT INTO refreshes VALUES(?,?,?,NULL)", (operation_id, digest, "PENDING"))
|
||||||
|
db.execute("COMMIT")
|
||||||
|
if new_operation:
|
||||||
|
self.grants.consume(token, binding, path)
|
||||||
|
current = self.projection.head(binding.repository, binding.branch)
|
||||||
|
if current != target_head_oid.lower():
|
||||||
|
self._state(operation_id, "STALE_TARGET")
|
||||||
|
raise ProvisioningError("stale_target_head", 409)
|
||||||
|
try:
|
||||||
|
oid = apply()
|
||||||
|
if not _is_oid(oid):
|
||||||
|
raise ProvisioningError("apply_result_invalid", 500)
|
||||||
|
# Detect a competing remote advancement before claiming completion.
|
||||||
|
observed = self.projection.head(binding.repository, binding.branch)
|
||||||
|
if observed != oid.lower():
|
||||||
|
self._state(operation_id, "RECONCILIATION_REQUIRED")
|
||||||
|
raise ProvisioningError("remote_concurrent_advance", 409)
|
||||||
|
self._state(operation_id, "COMPLETED", oid)
|
||||||
|
self.audit.append({"operation_id": operation_id, "task_id": binding.task_id, "repository": binding.repository,
|
||||||
|
"branch": binding.branch, "actor_identity": binding.actor_identity, "audience": binding.audience, "path": path, "state": "COMPLETED"})
|
||||||
|
return {"state": "COMPLETED", "operation_id": operation_id, "oid": oid, "idempotent": False}
|
||||||
|
except ProvisioningError:
|
||||||
|
raise
|
||||||
|
except Exception as exc:
|
||||||
|
self._state(operation_id, "RECOVERABLE")
|
||||||
|
self.audit.append({"operation_id": operation_id, "task_id": binding.task_id, "repository": binding.repository,
|
||||||
|
"branch": binding.branch, "actor_identity": binding.actor_identity, "audience": binding.audience, "path": path, "state": "RECOVERABLE"})
|
||||||
|
raise ProvisioningError("refresh_apply_failed", 503) from exc
|
||||||
|
|
||||||
|
def _state(self, operation_id: str, state: str, oid: str | None = None) -> None:
|
||||||
|
with self._ledger() as db:
|
||||||
|
db.execute("UPDATE refreshes SET state=?, result_oid=COALESCE(?,result_oid) WHERE operation_id=?", (state, oid, operation_id))
|
||||||
|
|
||||||
|
|
||||||
|
def _sandbox_preflight(state_dir: Path) -> int:
|
||||||
|
if os.environ.get("AIOS_GITOPS_WRITES_ENABLED") != "false":
|
||||||
|
raise ProvisioningError("global_gitops_writes_must_remain_disabled", 500)
|
||||||
|
state_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
audit = AppendOnlyAudit(state_dir / "preflight-audit.jsonl")
|
||||||
|
audit.append({"event": "SANDBOX_PREFLIGHT", "writes_enabled": False})
|
||||||
|
print(json.dumps({"status": "READY", "mode": "sandbox", "writes_enabled": False}, sort_keys=True))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--sandbox-preflight", action="store_true", required=True)
|
||||||
|
parser.add_argument("--state-dir", type=Path, required=True)
|
||||||
|
options = parser.parse_args()
|
||||||
|
raise SystemExit(_sandbox_preflight(options.state_dir))
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,45 @@
|
|||||||
|
"""AIOS-managed OpenClaw Ed25519 execution adapter."""
|
||||||
|
|
||||||
|
from .adapter import (
|
||||||
|
AdapterError,
|
||||||
|
CommandConflict,
|
||||||
|
CommandNotFound,
|
||||||
|
CommandRejected,
|
||||||
|
InMemoryCallbackSink,
|
||||||
|
OpenClawAdapter,
|
||||||
|
)
|
||||||
|
from .contract import (
|
||||||
|
CONTRACT_VERSION,
|
||||||
|
SIGNATURE_ALGORITHM,
|
||||||
|
CommandState,
|
||||||
|
Ed25519CommandSigner,
|
||||||
|
Ed25519CommandVerifier,
|
||||||
|
OpenClawCommand,
|
||||||
|
OpenClawResult,
|
||||||
|
RuntimeHealth,
|
||||||
|
sign_command,
|
||||||
|
verify_command_signature,
|
||||||
|
)
|
||||||
|
from .transport import HTTPRuntimeTransport, InProcessRuntimeTransport, RuntimeTransport
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"AdapterError",
|
||||||
|
"CONTRACT_VERSION",
|
||||||
|
"CommandConflict",
|
||||||
|
"CommandNotFound",
|
||||||
|
"CommandRejected",
|
||||||
|
"CommandState",
|
||||||
|
"Ed25519CommandSigner",
|
||||||
|
"Ed25519CommandVerifier",
|
||||||
|
"HTTPRuntimeTransport",
|
||||||
|
"InMemoryCallbackSink",
|
||||||
|
"InProcessRuntimeTransport",
|
||||||
|
"OpenClawAdapter",
|
||||||
|
"OpenClawCommand",
|
||||||
|
"OpenClawResult",
|
||||||
|
"RuntimeHealth",
|
||||||
|
"RuntimeTransport",
|
||||||
|
"SIGNATURE_ALGORITHM",
|
||||||
|
"sign_command",
|
||||||
|
"verify_command_signature",
|
||||||
|
]
|
||||||
@@ -0,0 +1,228 @@
|
|||||||
|
"""Signed, tenant-isolated execution adapter for OpenClaw runtimes."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import threading
|
||||||
|
from dataclasses import replace
|
||||||
|
from typing import Callable, Mapping, Protocol
|
||||||
|
|
||||||
|
from .contract import (
|
||||||
|
CommandState,
|
||||||
|
OpenClawCommand,
|
||||||
|
OpenClawResult,
|
||||||
|
RuntimeHealth,
|
||||||
|
utc_now,
|
||||||
|
verify_command_signature,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class AdapterError(RuntimeError):
|
||||||
|
"""Base adapter failure."""
|
||||||
|
|
||||||
|
|
||||||
|
class CommandRejected(AdapterError):
|
||||||
|
"""A command failed signature or contract validation."""
|
||||||
|
|
||||||
|
|
||||||
|
class CommandNotFound(AdapterError):
|
||||||
|
"""A tenant-scoped command is not visible."""
|
||||||
|
|
||||||
|
|
||||||
|
class CommandConflict(AdapterError):
|
||||||
|
"""A command or idempotency key conflicts with an existing execution."""
|
||||||
|
|
||||||
|
|
||||||
|
class RuntimeTransport(Protocol):
|
||||||
|
def execute(self, command: OpenClawCommand, timeout_seconds: float) -> OpenClawResult: ...
|
||||||
|
def cancel(self, command_id: str) -> bool: ...
|
||||||
|
def health(self) -> RuntimeHealth: ...
|
||||||
|
|
||||||
|
|
||||||
|
class CallbackSink(Protocol):
|
||||||
|
def on_started(self, result: OpenClawResult) -> None: ...
|
||||||
|
def on_completed(self, result: OpenClawResult) -> None: ...
|
||||||
|
def on_failed(self, result: OpenClawResult) -> None: ...
|
||||||
|
|
||||||
|
|
||||||
|
class InMemoryCallbackSink:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
self.started: list[OpenClawResult] = []
|
||||||
|
self.completed: list[OpenClawResult] = []
|
||||||
|
self.failed: list[OpenClawResult] = []
|
||||||
|
|
||||||
|
def on_started(self, result: OpenClawResult) -> None:
|
||||||
|
with self._lock:
|
||||||
|
self.started.append(result)
|
||||||
|
|
||||||
|
def on_completed(self, result: OpenClawResult) -> None:
|
||||||
|
with self._lock:
|
||||||
|
self.completed.append(result)
|
||||||
|
|
||||||
|
def on_failed(self, result: OpenClawResult) -> None:
|
||||||
|
with self._lock:
|
||||||
|
self.failed.append(result)
|
||||||
|
|
||||||
|
|
||||||
|
SigningMaterialResolver = Callable[[str, str], bytes | str]
|
||||||
|
|
||||||
|
|
||||||
|
class OpenClawAdapter:
|
||||||
|
"""Verify, de-duplicate, execute, cancel and reconcile commands."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
transport: RuntimeTransport,
|
||||||
|
signing_material_resolver: SigningMaterialResolver,
|
||||||
|
*,
|
||||||
|
callback_sink: CallbackSink | None = None,
|
||||||
|
default_timeout_seconds: float = 300.0,
|
||||||
|
) -> None:
|
||||||
|
if default_timeout_seconds <= 0:
|
||||||
|
raise ValueError("default_timeout_seconds must be positive")
|
||||||
|
self.transport = transport
|
||||||
|
self.signing_material_resolver = signing_material_resolver
|
||||||
|
self.callback_sink = callback_sink or InMemoryCallbackSink()
|
||||||
|
self.default_timeout_seconds = float(default_timeout_seconds)
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
self._commands: dict[str, OpenClawCommand] = {}
|
||||||
|
self._results: dict[str, OpenClawResult] = {}
|
||||||
|
self._idempotency: dict[tuple[str, str], tuple[str, str]] = {}
|
||||||
|
|
||||||
|
def _verify(self, command: OpenClawCommand) -> None:
|
||||||
|
try:
|
||||||
|
key_material = self.signing_material_resolver(command.tenant_id, command.key_id)
|
||||||
|
except Exception as exc:
|
||||||
|
raise CommandRejected("signing material unavailable") from exc
|
||||||
|
if not verify_command_signature(command, key_material):
|
||||||
|
raise CommandRejected("invalid or expired command signature")
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _validate_result(command: OpenClawCommand, result: OpenClawResult) -> None:
|
||||||
|
if (
|
||||||
|
result.command_id != command.command_id
|
||||||
|
or result.tenant_id != command.tenant_id
|
||||||
|
or result.task_id != command.task_id
|
||||||
|
or result.trace_id != command.trace_id
|
||||||
|
):
|
||||||
|
raise AdapterError("runtime returned mismatched command identity")
|
||||||
|
if result.state not in {
|
||||||
|
CommandState.COMPLETED,
|
||||||
|
CommandState.FAILED,
|
||||||
|
CommandState.CANCELLED,
|
||||||
|
CommandState.TIMED_OUT,
|
||||||
|
}:
|
||||||
|
raise AdapterError("runtime returned a non-terminal result")
|
||||||
|
|
||||||
|
def execute(
|
||||||
|
self,
|
||||||
|
command: OpenClawCommand,
|
||||||
|
*,
|
||||||
|
timeout_seconds: float | None = None,
|
||||||
|
) -> OpenClawResult:
|
||||||
|
timeout = self.default_timeout_seconds if timeout_seconds is None else float(timeout_seconds)
|
||||||
|
if timeout <= 0:
|
||||||
|
raise ValueError("timeout_seconds must be positive")
|
||||||
|
self._verify(command)
|
||||||
|
digest = command.signature
|
||||||
|
idem_ref = (command.tenant_id, command.idempotency_key)
|
||||||
|
|
||||||
|
with self._lock:
|
||||||
|
prior_command = self._commands.get(command.command_id)
|
||||||
|
if prior_command is not None:
|
||||||
|
if prior_command.signature != command.signature:
|
||||||
|
raise CommandConflict("command_id reused with different content")
|
||||||
|
return self._results[command.command_id]
|
||||||
|
prior_idem = self._idempotency.get(idem_ref)
|
||||||
|
if prior_idem is not None:
|
||||||
|
prior_id, prior_digest = prior_idem
|
||||||
|
if prior_digest != digest:
|
||||||
|
raise CommandConflict("idempotency key reused with different content")
|
||||||
|
return self._results[prior_id]
|
||||||
|
running = OpenClawResult(
|
||||||
|
command_id=command.command_id,
|
||||||
|
tenant_id=command.tenant_id,
|
||||||
|
task_id=command.task_id,
|
||||||
|
trace_id=command.trace_id,
|
||||||
|
state=CommandState.RUNNING,
|
||||||
|
started_at=utc_now(),
|
||||||
|
)
|
||||||
|
self._commands[command.command_id] = command
|
||||||
|
self._results[command.command_id] = running
|
||||||
|
self._idempotency[idem_ref] = (command.command_id, digest)
|
||||||
|
|
||||||
|
self.callback_sink.on_started(running)
|
||||||
|
try:
|
||||||
|
result = self.transport.execute(command, timeout)
|
||||||
|
self._validate_result(command, result)
|
||||||
|
except TimeoutError as exc:
|
||||||
|
result = replace(
|
||||||
|
running,
|
||||||
|
state=CommandState.TIMED_OUT,
|
||||||
|
error={"code": "runtime_timeout", "message": str(exc) or "runtime timeout"},
|
||||||
|
finished_at=utc_now(),
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
result = replace(
|
||||||
|
running,
|
||||||
|
state=CommandState.FAILED,
|
||||||
|
error={"code": "runtime_error", "message": str(exc)},
|
||||||
|
finished_at=utc_now(),
|
||||||
|
)
|
||||||
|
|
||||||
|
if result.started_at is None:
|
||||||
|
result = replace(result, started_at=running.started_at)
|
||||||
|
if result.finished_at is None:
|
||||||
|
result = replace(result, finished_at=utc_now())
|
||||||
|
with self._lock:
|
||||||
|
self._results[command.command_id] = result
|
||||||
|
if result.state == CommandState.COMPLETED:
|
||||||
|
self.callback_sink.on_completed(result)
|
||||||
|
else:
|
||||||
|
self.callback_sink.on_failed(result)
|
||||||
|
return result
|
||||||
|
|
||||||
|
def status(self, command_id: str, *, tenant_id: str) -> OpenClawResult:
|
||||||
|
with self._lock:
|
||||||
|
command = self._commands.get(command_id)
|
||||||
|
result = self._results.get(command_id)
|
||||||
|
if command is None or result is None or command.tenant_id != tenant_id:
|
||||||
|
raise CommandNotFound("command not found")
|
||||||
|
return result
|
||||||
|
|
||||||
|
def cancel(self, command_id: str, *, tenant_id: str) -> OpenClawResult:
|
||||||
|
with self._lock:
|
||||||
|
command = self._commands.get(command_id)
|
||||||
|
current = self._results.get(command_id)
|
||||||
|
if command is None or current is None or command.tenant_id != tenant_id:
|
||||||
|
raise CommandNotFound("command not found")
|
||||||
|
if current.state in {
|
||||||
|
CommandState.COMPLETED,
|
||||||
|
CommandState.FAILED,
|
||||||
|
CommandState.CANCELLED,
|
||||||
|
CommandState.TIMED_OUT,
|
||||||
|
CommandState.REJECTED,
|
||||||
|
}:
|
||||||
|
return current
|
||||||
|
if not self.transport.cancel(command_id):
|
||||||
|
raise AdapterError("runtime rejected cancellation")
|
||||||
|
cancelled = replace(
|
||||||
|
current,
|
||||||
|
state=CommandState.CANCELLED,
|
||||||
|
error={"code": "cancelled", "message": "cancelled by AIOS"},
|
||||||
|
finished_at=utc_now(),
|
||||||
|
)
|
||||||
|
with self._lock:
|
||||||
|
self._results[command_id] = cancelled
|
||||||
|
self.callback_sink.on_failed(cancelled)
|
||||||
|
return cancelled
|
||||||
|
|
||||||
|
def health(self) -> RuntimeHealth:
|
||||||
|
return self.transport.health()
|
||||||
|
|
||||||
|
def snapshot(self, *, tenant_id: str) -> Mapping[str, OpenClawResult]:
|
||||||
|
with self._lock:
|
||||||
|
return {
|
||||||
|
command_id: result
|
||||||
|
for command_id, result in self._results.items()
|
||||||
|
if self._commands[command_id].tenant_id == tenant_id
|
||||||
|
}
|
||||||
@@ -0,0 +1,97 @@
|
|||||||
|
"""Audit-oriented lifecycle callbacks for OpenClaw execution."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import threading
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Any, Mapping, Protocol
|
||||||
|
|
||||||
|
from .contract import OpenClawResult, utc_now
|
||||||
|
|
||||||
|
|
||||||
|
class EventPublisher(Protocol):
|
||||||
|
def publish(self, event: Mapping[str, Any]) -> None: ...
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ExecutionEvent:
|
||||||
|
event_type: str
|
||||||
|
tenant_id: str
|
||||||
|
task_id: str
|
||||||
|
trace_id: str
|
||||||
|
command_id: str
|
||||||
|
state: str
|
||||||
|
created_at: str = field(default_factory=utc_now)
|
||||||
|
details: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"event_type": self.event_type,
|
||||||
|
"tenant_id": self.tenant_id,
|
||||||
|
"task_id": self.task_id,
|
||||||
|
"trace_id": self.trace_id,
|
||||||
|
"command_id": self.command_id,
|
||||||
|
"state": self.state,
|
||||||
|
"created_at": self.created_at,
|
||||||
|
"details": json.loads(
|
||||||
|
json.dumps(dict(self.details), ensure_ascii=False, default=str)
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class InMemoryEventPublisher:
|
||||||
|
"""Thread-safe publisher used by tests and first integration slice."""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
self.events: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
def publish(self, event: Mapping[str, Any]) -> None:
|
||||||
|
with self._lock:
|
||||||
|
self.events.append(
|
||||||
|
json.loads(json.dumps(dict(event), ensure_ascii=False, default=str))
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class AuditCallbackSink:
|
||||||
|
"""Convert runtime callbacks into tenant-scoped audit events."""
|
||||||
|
|
||||||
|
def __init__(self, publisher: EventPublisher) -> None:
|
||||||
|
self.publisher = publisher
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _event(
|
||||||
|
event_type: str,
|
||||||
|
result: OpenClawResult,
|
||||||
|
) -> ExecutionEvent:
|
||||||
|
details: dict[str, Any] = {
|
||||||
|
"runtime_id": result.runtime_id,
|
||||||
|
"attempts": result.attempts,
|
||||||
|
}
|
||||||
|
if result.error:
|
||||||
|
details["error"] = dict(result.error)
|
||||||
|
if result.output:
|
||||||
|
details["output_present"] = True
|
||||||
|
return ExecutionEvent(
|
||||||
|
event_type=event_type,
|
||||||
|
tenant_id=result.tenant_id,
|
||||||
|
task_id=result.task_id,
|
||||||
|
trace_id=result.trace_id,
|
||||||
|
command_id=result.command_id,
|
||||||
|
state=result.state.value,
|
||||||
|
details=details,
|
||||||
|
)
|
||||||
|
|
||||||
|
def on_started(self, result: OpenClawResult) -> None:
|
||||||
|
self.publisher.publish(self._event("OPENCLAW_STARTED", result).as_dict())
|
||||||
|
|
||||||
|
def on_completed(self, result: OpenClawResult) -> None:
|
||||||
|
self.publisher.publish(self._event("OPENCLAW_COMPLETED", result).as_dict())
|
||||||
|
|
||||||
|
def on_failed(self, result: OpenClawResult) -> None:
|
||||||
|
event_type = {
|
||||||
|
"CANCELLED": "OPENCLAW_CANCELLED",
|
||||||
|
"TIMED_OUT": "OPENCLAW_TIMED_OUT",
|
||||||
|
"REJECTED": "OPENCLAW_REJECTED",
|
||||||
|
}.get(result.state.value, "OPENCLAW_FAILED")
|
||||||
|
self.publisher.publish(self._event(event_type, result).as_dict())
|
||||||
@@ -0,0 +1,383 @@
|
|||||||
|
"""AIOS 到 OpenClaw 的 Ed25519 执行合同。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import hashlib
|
||||||
|
import hmac
|
||||||
|
import json
|
||||||
|
import uuid
|
||||||
|
from dataclasses import dataclass, field, replace
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from enum import Enum
|
||||||
|
from typing import Any, Mapping, Protocol
|
||||||
|
|
||||||
|
CONTRACT_VERSION = "aios.openclaw.command.v2"
|
||||||
|
SIGNATURE_ALGORITHM = "Ed25519"
|
||||||
|
DEFAULT_CLOCK_SKEW_SECONDS = 300
|
||||||
|
|
||||||
|
|
||||||
|
def utc_now() -> str:
|
||||||
|
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_timestamp(value: str) -> datetime:
|
||||||
|
normalized = str(value or "").strip()
|
||||||
|
if not normalized:
|
||||||
|
raise ValueError("timestamp is required")
|
||||||
|
if normalized.endswith("Z"):
|
||||||
|
normalized = normalized[:-1] + "+00:00"
|
||||||
|
parsed = datetime.fromisoformat(normalized)
|
||||||
|
if parsed.tzinfo is None:
|
||||||
|
raise ValueError("timestamp must include a timezone")
|
||||||
|
return parsed.astimezone(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
def _required(value: str, label: str) -> str:
|
||||||
|
normalized = str(value or "").strip()
|
||||||
|
if not normalized:
|
||||||
|
raise ValueError(f"{label} is required")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
def _safe_mapping(value: Mapping[str, Any] | None) -> dict[str, Any]:
|
||||||
|
return json.loads(
|
||||||
|
json.dumps(dict(value or {}), ensure_ascii=False, sort_keys=True, default=str)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _load_crypto() -> tuple[Any, Any, Any]:
|
||||||
|
try:
|
||||||
|
from cryptography.hazmat.primitives import serialization
|
||||||
|
from cryptography.hazmat.primitives.asymmetric.ed25519 import (
|
||||||
|
Ed25519PrivateKey,
|
||||||
|
Ed25519PublicKey,
|
||||||
|
)
|
||||||
|
except ImportError as exc: # pragma: no cover
|
||||||
|
raise RuntimeError("cryptography with Ed25519 support is required") from exc
|
||||||
|
return serialization, Ed25519PrivateKey, Ed25519PublicKey
|
||||||
|
|
||||||
|
|
||||||
|
class CommandSigner(Protocol):
|
||||||
|
key_id: str
|
||||||
|
fingerprint: str
|
||||||
|
|
||||||
|
def sign(self, payload: bytes) -> bytes: ...
|
||||||
|
|
||||||
|
|
||||||
|
class CommandVerifier(Protocol):
|
||||||
|
key_id: str
|
||||||
|
fingerprint: str
|
||||||
|
|
||||||
|
def verify(self, payload: bytes, signature: bytes) -> bool: ...
|
||||||
|
|
||||||
|
|
||||||
|
class Ed25519CommandSigner:
|
||||||
|
def __init__(self, private_key: Any, *, key_id: str) -> None:
|
||||||
|
serialization, private_cls, _ = _load_crypto()
|
||||||
|
if not isinstance(private_key, private_cls):
|
||||||
|
raise ValueError("private key is not Ed25519")
|
||||||
|
self.private_key = private_key
|
||||||
|
self.key_id = _required(key_id, "key_id")
|
||||||
|
raw = private_key.public_key().public_bytes(
|
||||||
|
encoding=serialization.Encoding.Raw,
|
||||||
|
format=serialization.PublicFormat.Raw,
|
||||||
|
)
|
||||||
|
self.fingerprint = hashlib.sha256(raw).hexdigest()
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_bytes(cls, value: bytes | str, *, key_id: str) -> "Ed25519CommandSigner":
|
||||||
|
serialization, private_cls, _ = _load_crypto()
|
||||||
|
raw = value.encode("utf-8") if isinstance(value, str) else bytes(value)
|
||||||
|
if raw.startswith(b"-----BEGIN"):
|
||||||
|
key = serialization.load_pem_private_key(raw, password=None)
|
||||||
|
elif len(raw) == 32:
|
||||||
|
key = private_cls.from_private_bytes(raw)
|
||||||
|
else:
|
||||||
|
raise ValueError("Ed25519 private key must be PEM or a raw 32-byte seed")
|
||||||
|
return cls(key, key_id=key_id)
|
||||||
|
|
||||||
|
def sign(self, payload: bytes) -> bytes:
|
||||||
|
return self.private_key.sign(bytes(payload))
|
||||||
|
|
||||||
|
def verifier(self) -> "Ed25519CommandVerifier":
|
||||||
|
return Ed25519CommandVerifier(self.private_key.public_key(), key_id=self.key_id)
|
||||||
|
|
||||||
|
|
||||||
|
class Ed25519CommandVerifier:
|
||||||
|
def __init__(self, public_key: Any, *, key_id: str) -> None:
|
||||||
|
serialization, _, public_cls = _load_crypto()
|
||||||
|
if not isinstance(public_key, public_cls):
|
||||||
|
raise ValueError("public key is not Ed25519")
|
||||||
|
self.public_key = public_key
|
||||||
|
self.key_id = _required(key_id, "key_id")
|
||||||
|
raw = public_key.public_bytes(
|
||||||
|
encoding=serialization.Encoding.Raw,
|
||||||
|
format=serialization.PublicFormat.Raw,
|
||||||
|
)
|
||||||
|
self.fingerprint = hashlib.sha256(raw).hexdigest()
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_bytes(cls, value: bytes | str, *, key_id: str) -> "Ed25519CommandVerifier":
|
||||||
|
serialization, private_cls, public_cls = _load_crypto()
|
||||||
|
raw = value.encode("utf-8") if isinstance(value, str) else bytes(value)
|
||||||
|
if raw.startswith(b"-----BEGIN"):
|
||||||
|
try:
|
||||||
|
key = serialization.load_pem_public_key(raw)
|
||||||
|
except ValueError:
|
||||||
|
private = serialization.load_pem_private_key(raw, password=None)
|
||||||
|
if not isinstance(private, private_cls):
|
||||||
|
raise ValueError("key is not Ed25519")
|
||||||
|
key = private.public_key()
|
||||||
|
elif len(raw) == 32:
|
||||||
|
key = public_cls.from_public_bytes(raw)
|
||||||
|
else:
|
||||||
|
raise ValueError("Ed25519 public key must be PEM or raw 32-byte key")
|
||||||
|
return cls(key, key_id=key_id)
|
||||||
|
|
||||||
|
def verify(self, payload: bytes, signature: bytes) -> bool:
|
||||||
|
try:
|
||||||
|
self.public_key.verify(bytes(signature), bytes(payload))
|
||||||
|
return True
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
class CommandState(str, Enum):
|
||||||
|
ACCEPTED = "ACCEPTED"
|
||||||
|
RUNNING = "RUNNING"
|
||||||
|
COMPLETED = "COMPLETED"
|
||||||
|
FAILED = "FAILED"
|
||||||
|
CANCELLED = "CANCELLED"
|
||||||
|
TIMED_OUT = "TIMED_OUT"
|
||||||
|
REJECTED = "REJECTED"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class OpenClawCommand:
|
||||||
|
command_id: str
|
||||||
|
tenant_id: str
|
||||||
|
task_id: str
|
||||||
|
trace_id: str
|
||||||
|
action: str
|
||||||
|
payload: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
idempotency_key: str = ""
|
||||||
|
created_at: str = field(default_factory=utc_now)
|
||||||
|
expires_at: str | None = None
|
||||||
|
contract_version: str = CONTRACT_VERSION
|
||||||
|
signature_algorithm: str = SIGNATURE_ALGORITHM
|
||||||
|
key_id: str = "default"
|
||||||
|
public_key_fingerprint: str = ""
|
||||||
|
signature: str = ""
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for field_name in (
|
||||||
|
"command_id",
|
||||||
|
"tenant_id",
|
||||||
|
"task_id",
|
||||||
|
"trace_id",
|
||||||
|
"action",
|
||||||
|
"idempotency_key",
|
||||||
|
"created_at",
|
||||||
|
"contract_version",
|
||||||
|
"signature_algorithm",
|
||||||
|
"key_id",
|
||||||
|
):
|
||||||
|
object.__setattr__(
|
||||||
|
self, field_name, _required(getattr(self, field_name), field_name)
|
||||||
|
)
|
||||||
|
if self.contract_version != CONTRACT_VERSION:
|
||||||
|
raise ValueError(f"unsupported contract_version: {self.contract_version}")
|
||||||
|
if self.signature_algorithm != SIGNATURE_ALGORITHM:
|
||||||
|
raise ValueError(
|
||||||
|
f"unsupported signature_algorithm: {self.signature_algorithm}"
|
||||||
|
)
|
||||||
|
object.__setattr__(self, "payload", _safe_mapping(self.payload))
|
||||||
|
object.__setattr__(
|
||||||
|
self, "public_key_fingerprint", str(self.public_key_fingerprint or "").strip()
|
||||||
|
)
|
||||||
|
object.__setattr__(self, "signature", str(self.signature or "").strip())
|
||||||
|
_parse_timestamp(self.created_at)
|
||||||
|
if self.expires_at is not None:
|
||||||
|
object.__setattr__(self, "expires_at", _required(self.expires_at, "expires_at"))
|
||||||
|
if _parse_timestamp(self.expires_at) <= _parse_timestamp(self.created_at):
|
||||||
|
raise ValueError("expires_at must be later than created_at")
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def create(
|
||||||
|
cls,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
task_id: str,
|
||||||
|
trace_id: str,
|
||||||
|
action: str,
|
||||||
|
payload: Mapping[str, Any] | None,
|
||||||
|
idempotency_key: str,
|
||||||
|
expires_at: str | None = None,
|
||||||
|
key_id: str = "default",
|
||||||
|
command_id: str | None = None,
|
||||||
|
) -> "OpenClawCommand":
|
||||||
|
return cls(
|
||||||
|
command_id=command_id or f"cmd_{uuid.uuid4().hex}",
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
task_id=task_id,
|
||||||
|
trace_id=trace_id,
|
||||||
|
action=action,
|
||||||
|
payload=_safe_mapping(payload),
|
||||||
|
idempotency_key=idempotency_key,
|
||||||
|
expires_at=expires_at,
|
||||||
|
key_id=key_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
def unsigned_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"command_id": self.command_id,
|
||||||
|
"tenant_id": self.tenant_id,
|
||||||
|
"task_id": self.task_id,
|
||||||
|
"trace_id": self.trace_id,
|
||||||
|
"action": self.action,
|
||||||
|
"payload": _safe_mapping(self.payload),
|
||||||
|
"idempotency_key": self.idempotency_key,
|
||||||
|
"created_at": self.created_at,
|
||||||
|
"expires_at": self.expires_at,
|
||||||
|
"contract_version": self.contract_version,
|
||||||
|
"signature_algorithm": self.signature_algorithm,
|
||||||
|
"key_id": self.key_id,
|
||||||
|
"public_key_fingerprint": self.public_key_fingerprint,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class OpenClawResult:
|
||||||
|
command_id: str
|
||||||
|
tenant_id: str
|
||||||
|
task_id: str
|
||||||
|
trace_id: str
|
||||||
|
state: CommandState
|
||||||
|
output: Mapping[str, Any] | None = None
|
||||||
|
error: Mapping[str, Any] | None = None
|
||||||
|
started_at: str | None = None
|
||||||
|
finished_at: str | None = None
|
||||||
|
runtime_id: str | None = None
|
||||||
|
attempts: int = 1
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for field_name in ("command_id", "tenant_id", "task_id", "trace_id"):
|
||||||
|
object.__setattr__(
|
||||||
|
self, field_name, _required(getattr(self, field_name), field_name)
|
||||||
|
)
|
||||||
|
object.__setattr__(self, "state", CommandState(self.state))
|
||||||
|
if self.output is not None:
|
||||||
|
object.__setattr__(self, "output", _safe_mapping(self.output))
|
||||||
|
if self.error is not None:
|
||||||
|
object.__setattr__(self, "error", _safe_mapping(self.error))
|
||||||
|
if self.started_at is not None:
|
||||||
|
_parse_timestamp(self.started_at)
|
||||||
|
if self.finished_at is not None:
|
||||||
|
_parse_timestamp(self.finished_at)
|
||||||
|
if self.attempts < 1:
|
||||||
|
raise ValueError("attempts must be >= 1")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class RuntimeHealth:
|
||||||
|
runtime: str
|
||||||
|
runtime_id: str
|
||||||
|
status: str
|
||||||
|
version: str
|
||||||
|
heartbeat_at: str
|
||||||
|
active_tasks: int = 0
|
||||||
|
workspace_isolated: bool = False
|
||||||
|
credential_ref_configured: bool = False
|
||||||
|
details: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for field_name in (
|
||||||
|
"runtime",
|
||||||
|
"runtime_id",
|
||||||
|
"status",
|
||||||
|
"version",
|
||||||
|
"heartbeat_at",
|
||||||
|
):
|
||||||
|
object.__setattr__(
|
||||||
|
self, field_name, _required(getattr(self, field_name), field_name)
|
||||||
|
)
|
||||||
|
_parse_timestamp(self.heartbeat_at)
|
||||||
|
if self.active_tasks < 0:
|
||||||
|
raise ValueError("active_tasks must be >= 0")
|
||||||
|
object.__setattr__(self, "details", _safe_mapping(self.details))
|
||||||
|
|
||||||
|
|
||||||
|
def canonical_command_bytes(command: OpenClawCommand) -> bytes:
|
||||||
|
return json.dumps(
|
||||||
|
command.unsigned_dict(),
|
||||||
|
ensure_ascii=False,
|
||||||
|
sort_keys=True,
|
||||||
|
separators=(",", ":"),
|
||||||
|
).encode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def _as_signer(command: OpenClawCommand, value: Any) -> CommandSigner:
|
||||||
|
if hasattr(value, "sign") and hasattr(value, "fingerprint") and hasattr(value, "key_id"):
|
||||||
|
return value
|
||||||
|
return Ed25519CommandSigner.from_bytes(value, key_id=command.key_id)
|
||||||
|
|
||||||
|
|
||||||
|
def _as_verifier(command: OpenClawCommand, value: Any) -> CommandVerifier:
|
||||||
|
if isinstance(value, Ed25519CommandSigner):
|
||||||
|
return value.verifier()
|
||||||
|
if hasattr(value, "verify") and hasattr(value, "fingerprint") and hasattr(value, "key_id"):
|
||||||
|
return value
|
||||||
|
return Ed25519CommandVerifier.from_bytes(value, key_id=command.key_id)
|
||||||
|
|
||||||
|
|
||||||
|
def sign_command(command: OpenClawCommand, signing_key: Any) -> OpenClawCommand:
|
||||||
|
signer = _as_signer(command, signing_key)
|
||||||
|
if signer.key_id != command.key_id:
|
||||||
|
raise ValueError("command key_id does not match signer")
|
||||||
|
prepared = replace(
|
||||||
|
command,
|
||||||
|
signature_algorithm=SIGNATURE_ALGORITHM,
|
||||||
|
public_key_fingerprint=signer.fingerprint,
|
||||||
|
signature="",
|
||||||
|
)
|
||||||
|
signature = base64.b64encode(
|
||||||
|
signer.sign(canonical_command_bytes(prepared))
|
||||||
|
).decode("ascii")
|
||||||
|
return replace(prepared, signature=signature)
|
||||||
|
|
||||||
|
|
||||||
|
def verify_command_signature(
|
||||||
|
command: OpenClawCommand,
|
||||||
|
verification_key: Any,
|
||||||
|
*,
|
||||||
|
now: datetime | None = None,
|
||||||
|
clock_skew_seconds: int = DEFAULT_CLOCK_SKEW_SECONDS,
|
||||||
|
) -> bool:
|
||||||
|
if (
|
||||||
|
not command.signature
|
||||||
|
or command.contract_version != CONTRACT_VERSION
|
||||||
|
or command.signature_algorithm != SIGNATURE_ALGORITHM
|
||||||
|
):
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
verifier = _as_verifier(command, verification_key)
|
||||||
|
signature = base64.b64decode(command.signature, validate=True)
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
if verifier.key_id != command.key_id:
|
||||||
|
return False
|
||||||
|
if not command.public_key_fingerprint or not hmac.compare_digest(
|
||||||
|
command.public_key_fingerprint, verifier.fingerprint
|
||||||
|
):
|
||||||
|
return False
|
||||||
|
if not verifier.verify(canonical_command_bytes(command), signature):
|
||||||
|
return False
|
||||||
|
current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
|
||||||
|
created = _parse_timestamp(command.created_at)
|
||||||
|
if created.timestamp() - current.timestamp() > clock_skew_seconds:
|
||||||
|
return False
|
||||||
|
if command.expires_at is not None:
|
||||||
|
expires = _parse_timestamp(command.expires_at)
|
||||||
|
if current.timestamp() - expires.timestamp() > clock_skew_seconds:
|
||||||
|
return False
|
||||||
|
return True
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
"""Health policy and self-healing decisions for OpenClaw runtimes."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Mapping
|
||||||
|
|
||||||
|
from .contract import RuntimeHealth
|
||||||
|
|
||||||
|
|
||||||
|
def _parse(value: str) -> datetime:
|
||||||
|
normalized = value[:-1] + "+00:00" if value.endswith("Z") else value
|
||||||
|
parsed = datetime.fromisoformat(normalized)
|
||||||
|
if parsed.tzinfo is None:
|
||||||
|
raise ValueError("heartbeat timestamp must include timezone")
|
||||||
|
return parsed.astimezone(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class HealthPolicy:
|
||||||
|
max_heartbeat_age_seconds: int = 90
|
||||||
|
require_workspace_isolation: bool = True
|
||||||
|
require_credential_ref: bool = True
|
||||||
|
accepted_statuses: frozenset[str] = field(
|
||||||
|
default_factory=lambda: frozenset({"healthy", "ready"})
|
||||||
|
)
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
if self.max_heartbeat_age_seconds < 5:
|
||||||
|
raise ValueError("max_heartbeat_age_seconds must be >= 5")
|
||||||
|
if not self.accepted_statuses:
|
||||||
|
raise ValueError("accepted_statuses must not be empty")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class HealthAssessment:
|
||||||
|
healthy: bool
|
||||||
|
reasons: tuple[str, ...]
|
||||||
|
recoverable: bool
|
||||||
|
recommended_action: str
|
||||||
|
details: Mapping[str, object] = field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
|
class RuntimeHealthMonitor:
|
||||||
|
"""Evaluate runtime health and recommend bounded recovery actions."""
|
||||||
|
|
||||||
|
def __init__(self, policy: HealthPolicy | None = None) -> None:
|
||||||
|
self.policy = policy or HealthPolicy()
|
||||||
|
|
||||||
|
def assess(
|
||||||
|
self,
|
||||||
|
health: RuntimeHealth,
|
||||||
|
*,
|
||||||
|
now: datetime | None = None,
|
||||||
|
) -> HealthAssessment:
|
||||||
|
current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
|
||||||
|
heartbeat = _parse(health.heartbeat_at)
|
||||||
|
age = max(0.0, current.timestamp() - heartbeat.timestamp())
|
||||||
|
reasons: list[str] = []
|
||||||
|
|
||||||
|
if health.status.lower() not in self.policy.accepted_statuses:
|
||||||
|
reasons.append("runtime_status_unhealthy")
|
||||||
|
if age > self.policy.max_heartbeat_age_seconds:
|
||||||
|
reasons.append("heartbeat_stale")
|
||||||
|
if self.policy.require_workspace_isolation and not health.workspace_isolated:
|
||||||
|
reasons.append("workspace_not_isolated")
|
||||||
|
if self.policy.require_credential_ref and not health.credential_ref_configured:
|
||||||
|
reasons.append("credential_reference_missing")
|
||||||
|
|
||||||
|
if not reasons:
|
||||||
|
return HealthAssessment(
|
||||||
|
healthy=True,
|
||||||
|
reasons=(),
|
||||||
|
recoverable=True,
|
||||||
|
recommended_action="none",
|
||||||
|
details={"heartbeat_age_seconds": age},
|
||||||
|
)
|
||||||
|
|
||||||
|
non_recoverable = {
|
||||||
|
"workspace_not_isolated",
|
||||||
|
"credential_reference_missing",
|
||||||
|
}.intersection(reasons)
|
||||||
|
if non_recoverable:
|
||||||
|
action = "quarantine_runtime"
|
||||||
|
recoverable = False
|
||||||
|
elif "heartbeat_stale" in reasons:
|
||||||
|
action = "restart_runtime"
|
||||||
|
recoverable = True
|
||||||
|
else:
|
||||||
|
action = "drain_and_restart"
|
||||||
|
recoverable = True
|
||||||
|
|
||||||
|
return HealthAssessment(
|
||||||
|
healthy=False,
|
||||||
|
reasons=tuple(reasons),
|
||||||
|
recoverable=recoverable,
|
||||||
|
recommended_action=action,
|
||||||
|
details={
|
||||||
|
"heartbeat_age_seconds": age,
|
||||||
|
"runtime_id": health.runtime_id,
|
||||||
|
"active_tasks": health.active_tasks,
|
||||||
|
},
|
||||||
|
)
|
||||||
@@ -0,0 +1,214 @@
|
|||||||
|
"""Runtime transport implementations for the OpenClaw adapter."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import threading
|
||||||
|
import urllib.error
|
||||||
|
import urllib.parse
|
||||||
|
import urllib.request
|
||||||
|
from typing import Any, Callable, Mapping, Protocol
|
||||||
|
|
||||||
|
from .contract import (
|
||||||
|
CommandState,
|
||||||
|
OpenClawCommand,
|
||||||
|
OpenClawResult,
|
||||||
|
RuntimeHealth,
|
||||||
|
utc_now,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class RuntimeTransport(Protocol):
|
||||||
|
def execute(self, command: OpenClawCommand, timeout_seconds: float) -> OpenClawResult: ...
|
||||||
|
def cancel(self, command_id: str) -> bool: ...
|
||||||
|
def health(self) -> RuntimeHealth: ...
|
||||||
|
|
||||||
|
|
||||||
|
class TransportError(RuntimeError):
|
||||||
|
"""Runtime transport or protocol failure."""
|
||||||
|
|
||||||
|
|
||||||
|
RuntimeHandler = Callable[[OpenClawCommand], Mapping[str, Any] | OpenClawResult]
|
||||||
|
|
||||||
|
|
||||||
|
class InProcessRuntimeTransport:
|
||||||
|
"""Deterministic runtime used by tests and controlled local integrations."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
handler: RuntimeHandler,
|
||||||
|
*,
|
||||||
|
runtime_id: str = "openclaw-in-process",
|
||||||
|
version: str = "v1",
|
||||||
|
workspace_isolated: bool = True,
|
||||||
|
credential_ref_configured: bool = True,
|
||||||
|
) -> None:
|
||||||
|
self.handler = handler
|
||||||
|
self.runtime_id = runtime_id
|
||||||
|
self.version = version
|
||||||
|
self.workspace_isolated = workspace_isolated
|
||||||
|
self.credential_ref_configured = credential_ref_configured
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
self._running: set[str] = set()
|
||||||
|
self._cancelled: set[str] = set()
|
||||||
|
|
||||||
|
def execute(self, command: OpenClawCommand, timeout_seconds: float) -> OpenClawResult:
|
||||||
|
if timeout_seconds <= 0:
|
||||||
|
raise ValueError("timeout_seconds must be positive")
|
||||||
|
with self._lock:
|
||||||
|
if command.command_id in self._cancelled:
|
||||||
|
return OpenClawResult(
|
||||||
|
command_id=command.command_id,
|
||||||
|
tenant_id=command.tenant_id,
|
||||||
|
task_id=command.task_id,
|
||||||
|
trace_id=command.trace_id,
|
||||||
|
state=CommandState.CANCELLED,
|
||||||
|
error={"code": "cancelled", "message": "cancelled before execution"},
|
||||||
|
finished_at=utc_now(),
|
||||||
|
runtime_id=self.runtime_id,
|
||||||
|
)
|
||||||
|
self._running.add(command.command_id)
|
||||||
|
try:
|
||||||
|
raw = self.handler(command)
|
||||||
|
if isinstance(raw, OpenClawResult):
|
||||||
|
return raw
|
||||||
|
if not isinstance(raw, Mapping):
|
||||||
|
raise TransportError("runtime handler must return a mapping or OpenClawResult")
|
||||||
|
return OpenClawResult(
|
||||||
|
command_id=command.command_id,
|
||||||
|
tenant_id=command.tenant_id,
|
||||||
|
task_id=command.task_id,
|
||||||
|
trace_id=command.trace_id,
|
||||||
|
state=CommandState(str(raw.get("state", "COMPLETED"))),
|
||||||
|
output=raw.get("output"),
|
||||||
|
error=raw.get("error"),
|
||||||
|
started_at=str(raw.get("started_at") or utc_now()),
|
||||||
|
finished_at=str(raw.get("finished_at") or utc_now()),
|
||||||
|
runtime_id=str(raw.get("runtime_id") or self.runtime_id),
|
||||||
|
attempts=int(raw.get("attempts", 1)),
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
with self._lock:
|
||||||
|
self._running.discard(command.command_id)
|
||||||
|
|
||||||
|
def cancel(self, command_id: str) -> bool:
|
||||||
|
with self._lock:
|
||||||
|
self._cancelled.add(command_id)
|
||||||
|
return command_id in self._running
|
||||||
|
|
||||||
|
def health(self) -> RuntimeHealth:
|
||||||
|
with self._lock:
|
||||||
|
active = len(self._running)
|
||||||
|
return RuntimeHealth(
|
||||||
|
runtime="openclaw",
|
||||||
|
runtime_id=self.runtime_id,
|
||||||
|
status="healthy",
|
||||||
|
version=self.version,
|
||||||
|
heartbeat_at=utc_now(),
|
||||||
|
active_tasks=active,
|
||||||
|
workspace_isolated=self.workspace_isolated,
|
||||||
|
credential_ref_configured=self.credential_ref_configured,
|
||||||
|
details={"transport": "in_process"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class HTTPRuntimeTransport:
|
||||||
|
"""Private HTTP transport for an OpenClaw runtime service."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
base_url: str,
|
||||||
|
*,
|
||||||
|
runtime_id: str = "openclaw-http",
|
||||||
|
credential_ref: str | None = None,
|
||||||
|
max_response_bytes: int = 2_097_152,
|
||||||
|
) -> None:
|
||||||
|
normalized = str(base_url).rstrip("/")
|
||||||
|
if not normalized.startswith(("http://", "https://")):
|
||||||
|
raise ValueError("base_url must use http or https")
|
||||||
|
self.base_url = normalized
|
||||||
|
self.runtime_id = runtime_id
|
||||||
|
self.credential_ref = credential_ref
|
||||||
|
self.max_response_bytes = int(max_response_bytes)
|
||||||
|
|
||||||
|
def _request(
|
||||||
|
self,
|
||||||
|
method: str,
|
||||||
|
path: str,
|
||||||
|
*,
|
||||||
|
payload: Mapping[str, Any] | None,
|
||||||
|
timeout_seconds: float,
|
||||||
|
) -> Mapping[str, Any]:
|
||||||
|
body = None
|
||||||
|
headers = {"Accept": "application/json", "User-Agent": "AIOS-OpenClaw/1"}
|
||||||
|
if payload is not None:
|
||||||
|
body = json.dumps(dict(payload), separators=(",", ":")).encode("utf-8")
|
||||||
|
headers["Content-Type"] = "application/json"
|
||||||
|
if self.credential_ref:
|
||||||
|
headers["X-AIOS-Credential-Ref"] = self.credential_ref
|
||||||
|
request = urllib.request.Request(
|
||||||
|
f"{self.base_url}{path}",
|
||||||
|
data=body,
|
||||||
|
method=method,
|
||||||
|
headers=headers,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(request, timeout=timeout_seconds) as response:
|
||||||
|
raw = response.read(self.max_response_bytes + 1)
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
raise TransportError(f"runtime returned HTTP {exc.code}") from exc
|
||||||
|
except urllib.error.URLError as exc:
|
||||||
|
raise TransportError(f"runtime unavailable: {exc.reason}") from exc
|
||||||
|
if len(raw) > self.max_response_bytes:
|
||||||
|
raise TransportError("runtime response exceeded size limit")
|
||||||
|
decoded = json.loads(raw.decode("utf-8"))
|
||||||
|
if not isinstance(decoded, Mapping):
|
||||||
|
raise TransportError("runtime response must be a JSON object")
|
||||||
|
return decoded
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _command_payload(command: OpenClawCommand) -> dict[str, Any]:
|
||||||
|
return {**command.unsigned_dict(), "signature": command.signature}
|
||||||
|
|
||||||
|
def execute(self, command: OpenClawCommand, timeout_seconds: float) -> OpenClawResult:
|
||||||
|
value = self._request(
|
||||||
|
"POST",
|
||||||
|
"/v1/execute",
|
||||||
|
payload=self._command_payload(command),
|
||||||
|
timeout_seconds=timeout_seconds,
|
||||||
|
)
|
||||||
|
return OpenClawResult(
|
||||||
|
command_id=str(value["command_id"]),
|
||||||
|
tenant_id=str(value["tenant_id"]),
|
||||||
|
task_id=str(value["task_id"]),
|
||||||
|
trace_id=str(value["trace_id"]),
|
||||||
|
state=CommandState(str(value["state"])),
|
||||||
|
output=value.get("output"),
|
||||||
|
error=value.get("error"),
|
||||||
|
started_at=value.get("started_at"),
|
||||||
|
finished_at=value.get("finished_at"),
|
||||||
|
runtime_id=str(value.get("runtime_id") or self.runtime_id),
|
||||||
|
attempts=int(value.get("attempts", 1)),
|
||||||
|
)
|
||||||
|
|
||||||
|
def cancel(self, command_id: str) -> bool:
|
||||||
|
value = self._request(
|
||||||
|
"POST",
|
||||||
|
f"/v1/commands/{urllib.parse.quote(command_id, safe='')}/cancel",
|
||||||
|
payload={},
|
||||||
|
timeout_seconds=10.0,
|
||||||
|
)
|
||||||
|
return bool(value.get("accepted"))
|
||||||
|
|
||||||
|
def health(self) -> RuntimeHealth:
|
||||||
|
value = self._request("GET", "/v1/health", payload=None, timeout_seconds=5.0)
|
||||||
|
return RuntimeHealth(
|
||||||
|
runtime=str(value.get("runtime") or "openclaw"),
|
||||||
|
runtime_id=str(value.get("runtime_id") or self.runtime_id),
|
||||||
|
status=str(value.get("status") or "unknown"),
|
||||||
|
version=str(value.get("version") or "unknown"),
|
||||||
|
heartbeat_at=str(value.get("heartbeat_at") or utc_now()),
|
||||||
|
active_tasks=int(value.get("active_tasks", 0)),
|
||||||
|
workspace_isolated=bool(value.get("workspace_isolated", False)),
|
||||||
|
credential_ref_configured=bool(value.get("credential_ref_configured", False)),
|
||||||
|
details=value.get("details") if isinstance(value.get("details"), Mapping) else {},
|
||||||
|
)
|
||||||
@@ -1,144 +0,0 @@
|
|||||||
"""HTTP adapter for audited Owner authorization and writer provisioning.
|
|
||||||
|
|
||||||
This module layers on top of ``aios_authorized_task_writer_api`` so existing
|
|
||||||
routes and the legacy exact ``writer:provision`` route remain available. The
|
|
||||||
new endpoint does not mint or modify bearer credentials. It authenticates an
|
|
||||||
existing, non-wildcard, explicitly allow-listed Owner credential and then
|
|
||||||
invokes the audited broker.
|
|
||||||
|
|
||||||
Endpoint:
|
|
||||||
POST /api/governance/owner-authorizations
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import urllib.parse
|
|
||||||
from typing import Any, Mapping
|
|
||||||
|
|
||||||
import aios_authorized_task_writer_api as bridge_api
|
|
||||||
from authorized_task_writer_provisioning import ProvisioningError
|
|
||||||
from owner_authorization_broker import (
|
|
||||||
OwnerAuthorizationBroker,
|
|
||||||
OwnerAuthorizationError,
|
|
||||||
OwnerPrincipal,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
api = bridge_api.api
|
|
||||||
_original_do_post = api.H.do_POST
|
|
||||||
_ENDPOINT = "/api/governance/owner-authorizations"
|
|
||||||
|
|
||||||
|
|
||||||
def _enabled() -> bool:
|
|
||||||
return os.getenv(
|
|
||||||
"AIOS_GITOPS_OWNER_AUTHORIZATION_BROKER_ENABLED", ""
|
|
||||||
).strip().lower() in {"1", "true", "yes", "on"}
|
|
||||||
|
|
||||||
|
|
||||||
def _configured_intake_scope() -> str:
|
|
||||||
scopes = [
|
|
||||||
item.strip()
|
|
||||||
for item in os.getenv(
|
|
||||||
"AIOS_GITOPS_OWNER_AUTHORIZATION_INTAKE_SCOPES", ""
|
|
||||||
).split(",")
|
|
||||||
if item.strip()
|
|
||||||
]
|
|
||||||
if len(scopes) != 1:
|
|
||||||
raise OwnerAuthorizationError(
|
|
||||||
"EXACTLY_ONE_HTTP_INTAKE_SCOPE_REQUIRED", 503
|
|
||||||
)
|
|
||||||
scope = scopes[0]
|
|
||||||
if scope == "writer:provision" or scope == "*" or scope.endswith(":*"):
|
|
||||||
raise OwnerAuthorizationError("HTTP_INTAKE_SCOPE_REJECTED", 503)
|
|
||||||
return scope
|
|
||||||
|
|
||||||
|
|
||||||
def _scope_set(auth: Mapping[str, Any]) -> frozenset[str]:
|
|
||||||
raw = auth.get("scopes")
|
|
||||||
if raw is None:
|
|
||||||
raw = auth.get("scope")
|
|
||||||
if raw is None:
|
|
||||||
return frozenset()
|
|
||||||
if isinstance(raw, str):
|
|
||||||
candidates = raw.replace(",", " ").split()
|
|
||||||
elif isinstance(raw, (list, tuple, set, frozenset)):
|
|
||||||
candidates = [str(item) for item in raw]
|
|
||||||
else:
|
|
||||||
raise OwnerAuthorizationError("OWNER_SCOPE_EVIDENCE_INVALID", 403)
|
|
||||||
return frozenset(item.strip() for item in candidates if item.strip())
|
|
||||||
|
|
||||||
|
|
||||||
def _credential_id(auth: Mapping[str, Any]) -> str:
|
|
||||||
for name in (
|
|
||||||
"credential_id",
|
|
||||||
"token_id",
|
|
||||||
"key_id",
|
|
||||||
"auth_id",
|
|
||||||
):
|
|
||||||
value = str(auth.get(name) or "").strip()
|
|
||||||
if value:
|
|
||||||
return value
|
|
||||||
return ""
|
|
||||||
|
|
||||||
|
|
||||||
def _authenticate_existing_owner(handler) -> OwnerPrincipal | None:
|
|
||||||
scope = _configured_intake_scope()
|
|
||||||
auth = handler.need(scope)
|
|
||||||
if not auth:
|
|
||||||
return None
|
|
||||||
if auth.get("role") != "owner":
|
|
||||||
handler.sendj({"error": "OWNER_ROLE_REQUIRED"}, 403)
|
|
||||||
return None
|
|
||||||
identity = str(auth.get("account") or auth.get("identity") or "").strip()
|
|
||||||
if not identity:
|
|
||||||
handler.sendj({"error": "OWNER_IDENTITY_REQUIRED"}, 403)
|
|
||||||
return None
|
|
||||||
scopes = _scope_set(auth)
|
|
||||||
if any(item == "*" or item.endswith(":*") for item in scopes):
|
|
||||||
handler.sendj(
|
|
||||||
{"error": "WILDCARD_OWNER_CREDENTIAL_REJECTED"}, 403
|
|
||||||
)
|
|
||||||
return None
|
|
||||||
return OwnerPrincipal(
|
|
||||||
identity=identity,
|
|
||||||
role="owner",
|
|
||||||
authenticated_scope=scope,
|
|
||||||
token_scopes=scopes,
|
|
||||||
credential_id=_credential_id(auth),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _broker_do_post(self):
|
|
||||||
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
|
|
||||||
if path != _ENDPOINT:
|
|
||||||
return _original_do_post(self)
|
|
||||||
if not _enabled():
|
|
||||||
return self.sendj(
|
|
||||||
{"error": "OWNER_AUTHORIZATION_BROKER_DISABLED"}, 503
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
principal = _authenticate_existing_owner(self)
|
|
||||||
if principal is None:
|
|
||||||
return
|
|
||||||
data = self.body()
|
|
||||||
result = OwnerAuthorizationBroker().authorize_and_provision(
|
|
||||||
data, principal
|
|
||||||
)
|
|
||||||
return self.sendj(result)
|
|
||||||
except (OwnerAuthorizationError, ProvisioningError) as exc:
|
|
||||||
body = {"error": exc.code}
|
|
||||||
if exc.details:
|
|
||||||
body["details"] = exc.details
|
|
||||||
return self.sendj(body, exc.status)
|
|
||||||
except Exception:
|
|
||||||
# Never expose registry data, bearer tokens, signer output or traceback.
|
|
||||||
return self.sendj(
|
|
||||||
{"error": "OWNER_AUTHORIZATION_BROKER_INTERNAL_ERROR"}, 500
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
api.H.do_POST = _broker_do_post
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
api.main()
|
|
||||||
@@ -1,160 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Loopback-only HTTP API for the task identity overlay.
|
|
||||||
|
|
||||||
Endpoints:
|
|
||||||
GET /healthz
|
|
||||||
POST /api/governance/owner-authorizations
|
|
||||||
|
|
||||||
The API authenticates only the short-lived overlay credential. It does not read
|
|
||||||
or modify the global bearer-token registry.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import urllib.parse
|
|
||||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from authorized_task_writer_provisioning import ProvisioningError
|
|
||||||
from owner_authorization_broker import OwnerAuthorizationError
|
|
||||||
from task_identity_overlay import (
|
|
||||||
MAX_BODY_BYTES,
|
|
||||||
IdentityOverlayConfig,
|
|
||||||
IdentityOverlayError,
|
|
||||||
TaskIdentityOverlayService,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
_ENDPOINT = "/api/governance/owner-authorizations"
|
|
||||||
|
|
||||||
|
|
||||||
def _enabled() -> bool:
|
|
||||||
return os.getenv(
|
|
||||||
"AIOS_GITOPS_IDENTITY_OVERLAY_ENABLED", ""
|
|
||||||
).strip().lower() in {"1", "true", "yes", "on"}
|
|
||||||
|
|
||||||
|
|
||||||
def _bearer(header: str) -> str:
|
|
||||||
prefix = "Bearer "
|
|
||||||
if not header.startswith(prefix):
|
|
||||||
raise IdentityOverlayError("OVERLAY_AUTHENTICATION_REQUIRED", 401)
|
|
||||||
token = header[len(prefix):].strip()
|
|
||||||
if not token or len(token) > 4096:
|
|
||||||
raise IdentityOverlayError("OVERLAY_AUTHENTICATION_REQUIRED", 401)
|
|
||||||
return token
|
|
||||||
|
|
||||||
|
|
||||||
class OverlayHandler(BaseHTTPRequestHandler):
|
|
||||||
server_version = "aios-task-identity-overlay/1"
|
|
||||||
sys_version = ""
|
|
||||||
|
|
||||||
@property
|
|
||||||
def overlay(self) -> TaskIdentityOverlayService:
|
|
||||||
return self.server.overlay # type: ignore[attr-defined]
|
|
||||||
|
|
||||||
def log_message(self, format: str, *args: Any) -> None:
|
|
||||||
# Never log Authorization or request bodies.
|
|
||||||
return
|
|
||||||
|
|
||||||
def _send(self, status: int, value: dict[str, Any]) -> None:
|
|
||||||
payload = (
|
|
||||||
json.dumps(
|
|
||||||
value,
|
|
||||||
ensure_ascii=False,
|
|
||||||
sort_keys=True,
|
|
||||||
separators=(",", ":"),
|
|
||||||
)
|
|
||||||
+ "\n"
|
|
||||||
).encode("utf-8")
|
|
||||||
self.send_response(status)
|
|
||||||
self.send_header("Content-Type", "application/json")
|
|
||||||
self.send_header("Cache-Control", "no-store")
|
|
||||||
self.send_header("Content-Length", str(len(payload)))
|
|
||||||
self.end_headers()
|
|
||||||
self.wfile.write(payload)
|
|
||||||
|
|
||||||
def _body(self) -> dict[str, Any]:
|
|
||||||
try:
|
|
||||||
length = int(self.headers.get("Content-Length", "0"))
|
|
||||||
except ValueError:
|
|
||||||
raise IdentityOverlayError("OVERLAY_CONTENT_LENGTH_INVALID") from None
|
|
||||||
if length <= 0 or length > MAX_BODY_BYTES:
|
|
||||||
raise IdentityOverlayError("OVERLAY_BODY_SIZE_REJECTED", 413)
|
|
||||||
raw = self.rfile.read(length)
|
|
||||||
try:
|
|
||||||
value = json.loads(raw)
|
|
||||||
except (UnicodeDecodeError, json.JSONDecodeError):
|
|
||||||
raise IdentityOverlayError("OVERLAY_BODY_JSON_INVALID") from None
|
|
||||||
if not isinstance(value, dict):
|
|
||||||
raise IdentityOverlayError("OVERLAY_BODY_OBJECT_REQUIRED")
|
|
||||||
return value
|
|
||||||
|
|
||||||
def do_GET(self) -> None:
|
|
||||||
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
|
|
||||||
if path != "/healthz":
|
|
||||||
self._send(404, {"error": "NOT_FOUND"})
|
|
||||||
return
|
|
||||||
self._send(
|
|
||||||
200,
|
|
||||||
{
|
|
||||||
"service": "aios-task-identity-overlay",
|
|
||||||
"status": "active" if _enabled() else "disabled",
|
|
||||||
"global_token_registry_used": False,
|
|
||||||
"loopback_only": True,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
def do_POST(self) -> None:
|
|
||||||
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
|
|
||||||
if path != _ENDPOINT:
|
|
||||||
self._send(404, {"error": "NOT_FOUND"})
|
|
||||||
return
|
|
||||||
if not _enabled():
|
|
||||||
self._send(503, {"error": "IDENTITY_OVERLAY_DISABLED"})
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
token = _bearer(self.headers.get("Authorization", ""))
|
|
||||||
result = self.overlay.provision(self._body(), token)
|
|
||||||
self._send(200, result)
|
|
||||||
except (IdentityOverlayError, OwnerAuthorizationError, ProvisioningError) as exc:
|
|
||||||
body: dict[str, Any] = {"error": exc.code}
|
|
||||||
if getattr(exc, "details", None):
|
|
||||||
body["details"] = exc.details
|
|
||||||
self._send(exc.status, body)
|
|
||||||
except Exception:
|
|
||||||
self._send(500, {"error": "IDENTITY_OVERLAY_API_INTERNAL_ERROR"})
|
|
||||||
|
|
||||||
|
|
||||||
class OverlayServer(ThreadingHTTPServer):
|
|
||||||
allow_reuse_address = True
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
address: tuple[str, int],
|
|
||||||
overlay: TaskIdentityOverlayService,
|
|
||||||
) -> None:
|
|
||||||
super().__init__(address, OverlayHandler)
|
|
||||||
self.overlay = overlay
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
|
||||||
host = os.getenv("AIOS_GITOPS_IDENTITY_OVERLAY_HOST", "127.0.0.1")
|
|
||||||
if host not in {"127.0.0.1", "::1", "localhost"}:
|
|
||||||
raise SystemExit("identity overlay must listen on loopback")
|
|
||||||
try:
|
|
||||||
port = int(os.getenv("AIOS_GITOPS_IDENTITY_OVERLAY_PORT", "19864"))
|
|
||||||
except ValueError:
|
|
||||||
raise SystemExit("invalid overlay port") from None
|
|
||||||
if not 1024 <= port <= 65535:
|
|
||||||
raise SystemExit("invalid overlay port")
|
|
||||||
config = IdentityOverlayConfig.from_environment()
|
|
||||||
server = OverlayServer(
|
|
||||||
(host, port),
|
|
||||||
TaskIdentityOverlayService(config),
|
|
||||||
)
|
|
||||||
server.serve_forever(poll_interval=0.25)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,92 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Root-governed CLI for issuing and revoking task identity overlays.
|
|
||||||
|
|
||||||
The CLI never prints the bearer token. It returns only the root-owned secret
|
|
||||||
reference and non-secret metadata.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from task_identity_overlay import (
|
|
||||||
IdentityOverlayConfig,
|
|
||||||
IdentityOverlayError,
|
|
||||||
TaskIdentityOverlayStore,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _write_result(path: str, value: dict) -> None:
|
|
||||||
data = json.dumps(value, sort_keys=True, separators=(",", ":")) + "\n"
|
|
||||||
if path == "-":
|
|
||||||
sys.stdout.write(data)
|
|
||||||
return
|
|
||||||
target = Path(path)
|
|
||||||
target.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
target.write_text(data, encoding="utf-8")
|
|
||||||
os.chmod(target, 0o600)
|
|
||||||
|
|
||||||
|
|
||||||
def parser() -> argparse.ArgumentParser:
|
|
||||||
value = argparse.ArgumentParser(
|
|
||||||
description="Issue/status/revoke a task-scoped identity overlay"
|
|
||||||
)
|
|
||||||
sub = value.add_subparsers(dest="command", required=True)
|
|
||||||
|
|
||||||
issue = sub.add_parser("issue")
|
|
||||||
issue.add_argument("--ttl-seconds", type=int, default=900)
|
|
||||||
issue.add_argument("--authorization-assertion-ref", required=True)
|
|
||||||
issue.add_argument(
|
|
||||||
"--operator-identity",
|
|
||||||
default=os.getenv("SUDO_USER") or os.getenv("USER") or "root",
|
|
||||||
)
|
|
||||||
issue.add_argument("--result-json", default="-")
|
|
||||||
|
|
||||||
status = sub.add_parser("status")
|
|
||||||
status.add_argument("--overlay-id", required=True)
|
|
||||||
status.add_argument("--result-json", default="-")
|
|
||||||
|
|
||||||
revoke = sub.add_parser("revoke")
|
|
||||||
revoke.add_argument("--overlay-id", required=True)
|
|
||||||
revoke.add_argument("--reason", required=True)
|
|
||||||
revoke.add_argument("--result-json", default="-")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
args = parser().parse_args()
|
|
||||||
try:
|
|
||||||
store = TaskIdentityOverlayStore(
|
|
||||||
IdentityOverlayConfig.from_environment()
|
|
||||||
)
|
|
||||||
if args.command == "issue":
|
|
||||||
result = store.issue(
|
|
||||||
ttl_seconds=args.ttl_seconds,
|
|
||||||
authorization_assertion_ref=args.authorization_assertion_ref,
|
|
||||||
operator_identity=args.operator_identity,
|
|
||||||
)
|
|
||||||
elif args.command == "status":
|
|
||||||
result = store.status(args.overlay_id)
|
|
||||||
else:
|
|
||||||
result = store.revoke(args.overlay_id, reason=args.reason)
|
|
||||||
_write_result(args.result_json, result)
|
|
||||||
return 0
|
|
||||||
except IdentityOverlayError as exc:
|
|
||||||
body = {"status": "BLOCKED", "error": exc.code}
|
|
||||||
if exc.details:
|
|
||||||
body["details"] = exc.details
|
|
||||||
_write_result(getattr(args, "result_json", "-"), body)
|
|
||||||
return 2
|
|
||||||
except Exception:
|
|
||||||
_write_result(
|
|
||||||
getattr(args, "result_json", "-"),
|
|
||||||
{"status": "FAILED", "error": "IDENTITY_OVERLAY_CLI_INTERNAL_ERROR"},
|
|
||||||
)
|
|
||||||
return 1
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -0,0 +1,110 @@
|
|||||||
|
"""AIOS V2 控制面、执行编排与可信运行来源证明。"""
|
||||||
|
|
||||||
|
from .control_plane import (
|
||||||
|
AgentRecord,
|
||||||
|
AuditEvent,
|
||||||
|
ControlPlaneError,
|
||||||
|
ControlPlaneService,
|
||||||
|
IdempotencyConflict,
|
||||||
|
InMemoryControlPlaneStore,
|
||||||
|
InvalidTransition,
|
||||||
|
NotFound,
|
||||||
|
PolicyDenied,
|
||||||
|
RequestContext,
|
||||||
|
RevisionConflict,
|
||||||
|
TaskRecord,
|
||||||
|
TaskState,
|
||||||
|
TenantIsolationError,
|
||||||
|
TenantRecord,
|
||||||
|
Unauthorized,
|
||||||
|
)
|
||||||
|
from .execution_tracking import ExecutionEvidence, ExecutionTracker
|
||||||
|
from .orchestrator import DispatchRecord, ExecutionOrchestrator, OrchestrationError
|
||||||
|
from .provenance import (
|
||||||
|
Ed25519PrivateKeySigner,
|
||||||
|
Ed25519PublicKeyVerifier,
|
||||||
|
ExecutionPath,
|
||||||
|
GitOperation,
|
||||||
|
PRODUCTION_POLICY_VERSION,
|
||||||
|
ProvenanceAuthorization,
|
||||||
|
ProvenanceControlledWriter,
|
||||||
|
ProvenanceDenied,
|
||||||
|
ProvenanceError,
|
||||||
|
ProvenanceExpired,
|
||||||
|
ProvenanceIssuer,
|
||||||
|
ProvenanceNotFound,
|
||||||
|
ProvenanceReplay,
|
||||||
|
ProvenanceRevoked,
|
||||||
|
ProvenanceSignatureError,
|
||||||
|
ProvenanceStatus,
|
||||||
|
ProvenanceVerifier,
|
||||||
|
RunProvenance,
|
||||||
|
SQLiteProvenanceStore,
|
||||||
|
digest_payload,
|
||||||
|
)
|
||||||
|
from .provenance_git import (
|
||||||
|
GitCommandResult,
|
||||||
|
GitExecutionContext,
|
||||||
|
ProvenanceGitError,
|
||||||
|
ProvenanceGitWriter,
|
||||||
|
)
|
||||||
|
from .provenance_operations import (
|
||||||
|
OperationContext,
|
||||||
|
OperationReceipt,
|
||||||
|
ProvenanceOperationGate,
|
||||||
|
build_provenance_writer_from_env,
|
||||||
|
context_from_environment,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"AgentRecord",
|
||||||
|
"AuditEvent",
|
||||||
|
"ControlPlaneError",
|
||||||
|
"ControlPlaneService",
|
||||||
|
"DispatchRecord",
|
||||||
|
"Ed25519PrivateKeySigner",
|
||||||
|
"Ed25519PublicKeyVerifier",
|
||||||
|
"ExecutionEvidence",
|
||||||
|
"ExecutionOrchestrator",
|
||||||
|
"ExecutionPath",
|
||||||
|
"ExecutionTracker",
|
||||||
|
"GitCommandResult",
|
||||||
|
"GitExecutionContext",
|
||||||
|
"GitOperation",
|
||||||
|
"IdempotencyConflict",
|
||||||
|
"InMemoryControlPlaneStore",
|
||||||
|
"InvalidTransition",
|
||||||
|
"NotFound",
|
||||||
|
"OperationContext",
|
||||||
|
"OperationReceipt",
|
||||||
|
"OrchestrationError",
|
||||||
|
"PRODUCTION_POLICY_VERSION",
|
||||||
|
"PolicyDenied",
|
||||||
|
"ProvenanceAuthorization",
|
||||||
|
"ProvenanceControlledWriter",
|
||||||
|
"ProvenanceDenied",
|
||||||
|
"ProvenanceError",
|
||||||
|
"ProvenanceExpired",
|
||||||
|
"ProvenanceGitError",
|
||||||
|
"ProvenanceGitWriter",
|
||||||
|
"ProvenanceIssuer",
|
||||||
|
"ProvenanceNotFound",
|
||||||
|
"ProvenanceOperationGate",
|
||||||
|
"ProvenanceReplay",
|
||||||
|
"ProvenanceRevoked",
|
||||||
|
"ProvenanceSignatureError",
|
||||||
|
"ProvenanceStatus",
|
||||||
|
"ProvenanceVerifier",
|
||||||
|
"RequestContext",
|
||||||
|
"RevisionConflict",
|
||||||
|
"RunProvenance",
|
||||||
|
"SQLiteProvenanceStore",
|
||||||
|
"TaskRecord",
|
||||||
|
"TaskState",
|
||||||
|
"TenantIsolationError",
|
||||||
|
"TenantRecord",
|
||||||
|
"Unauthorized",
|
||||||
|
"build_provenance_writer_from_env",
|
||||||
|
"context_from_environment",
|
||||||
|
"digest_payload",
|
||||||
|
]
|
||||||
@@ -0,0 +1,807 @@
|
|||||||
|
"""AIOS V2 production-candidate application entrypoint."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import stat
|
||||||
|
import urllib.parse
|
||||||
|
from enum import Enum
|
||||||
|
from http import HTTPStatus
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
from aios_hermes import HermesAdapter, HermesJobState, HermesResult, SQLiteHermesStore
|
||||||
|
from aios_openclaw import (
|
||||||
|
Ed25519CommandSigner,
|
||||||
|
HTTPRuntimeTransport,
|
||||||
|
OpenClawAdapter,
|
||||||
|
)
|
||||||
|
from aios_openclaw.callback import AuditCallbackSink, EventPublisher
|
||||||
|
|
||||||
|
from .console import CONSOLE_HTML
|
||||||
|
from .control_plane import AuditEvent, RequestContext, Unauthorized
|
||||||
|
from .dispatch_store import SQLiteDispatchRepository
|
||||||
|
from .execution_tracking import ExecutionEvidence, ExecutionTracker
|
||||||
|
from .gateway import GatewayApplication, error_response
|
||||||
|
from .gitops_status import collect_gitops_status
|
||||||
|
from .orchestrator import ExecutionOrchestrator
|
||||||
|
from .persistence import DurableControlPlaneService, SQLiteSnapshotRepository
|
||||||
|
from .provenance_operations import build_provenance_writer_from_env
|
||||||
|
|
||||||
|
|
||||||
|
def _json_value(value: Any) -> Any:
|
||||||
|
if dataclasses.is_dataclass(value):
|
||||||
|
return {
|
||||||
|
item.name: _json_value(getattr(value, item.name))
|
||||||
|
for item in dataclasses.fields(value)
|
||||||
|
}
|
||||||
|
if isinstance(value, Enum):
|
||||||
|
return value.value
|
||||||
|
if isinstance(value, Mapping):
|
||||||
|
return {str(key): _json_value(item) for key, item in value.items()}
|
||||||
|
if isinstance(value, (list, tuple, set, frozenset)):
|
||||||
|
return [_json_value(item) for item in value]
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _enabled(value: str | None) -> bool:
|
||||||
|
return str(value or "").strip().lower() in {"1", "true", "yes", "on"}
|
||||||
|
|
||||||
|
|
||||||
|
def _header(headers: Mapping[str, str], name: str) -> str | None:
|
||||||
|
target = name.lower()
|
||||||
|
for key, value in headers.items():
|
||||||
|
if str(key).lower() == target:
|
||||||
|
normalized = str(value or "").strip()
|
||||||
|
return normalized or None
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _provenance_context(
|
||||||
|
headers: Mapping[str, str],
|
||||||
|
payload: Mapping[str, Any],
|
||||||
|
) -> tuple[str | None, str | None]:
|
||||||
|
header_provenance = _header(headers, "X-AIOS-Provenance-ID")
|
||||||
|
header_run = _header(headers, "X-AIOS-Run-ID")
|
||||||
|
body_provenance = str(payload.get("provenance_id") or "").strip() or None
|
||||||
|
body_run = str(payload.get("run_id") or "").strip() or None
|
||||||
|
if header_provenance and body_provenance and header_provenance != body_provenance:
|
||||||
|
raise ValueError("provenance_id header/body mismatch")
|
||||||
|
if header_run and body_run and header_run != body_run:
|
||||||
|
raise ValueError("run_id header/body mismatch")
|
||||||
|
return header_provenance or body_provenance, header_run or body_run
|
||||||
|
|
||||||
|
|
||||||
|
def _read_protected_file(path: Path, *, private: bool) -> bytes:
|
||||||
|
flags = os.O_RDONLY
|
||||||
|
if hasattr(os, "O_NOFOLLOW"):
|
||||||
|
flags |= os.O_NOFOLLOW
|
||||||
|
descriptor = os.open(str(path), flags)
|
||||||
|
try:
|
||||||
|
metadata = os.fstat(descriptor)
|
||||||
|
if not stat.S_ISREG(metadata.st_mode):
|
||||||
|
raise RuntimeError("key reference is not a regular file")
|
||||||
|
denied = 0o077 if private else 0o022
|
||||||
|
if metadata.st_mode & denied:
|
||||||
|
raise RuntimeError(
|
||||||
|
"private key file permissions must be 0600 or stricter"
|
||||||
|
if private
|
||||||
|
else "public key file must not be group/world writable"
|
||||||
|
)
|
||||||
|
chunks: list[bytes] = []
|
||||||
|
while True:
|
||||||
|
chunk = os.read(descriptor, 65536)
|
||||||
|
if not chunk:
|
||||||
|
break
|
||||||
|
chunks.append(chunk)
|
||||||
|
value = b"".join(chunks)
|
||||||
|
if not value:
|
||||||
|
raise RuntimeError("key file is empty")
|
||||||
|
return value
|
||||||
|
finally:
|
||||||
|
os.close(descriptor)
|
||||||
|
|
||||||
|
|
||||||
|
class FileCommandSignerResolver:
|
||||||
|
"""Resolve tenant/key-id scoped OpenClaw Ed25519 signing material."""
|
||||||
|
|
||||||
|
def __init__(self, path: str | Path, *, configured_key_id: str) -> None:
|
||||||
|
self.path = Path(path).resolve()
|
||||||
|
self.configured_key_id = str(configured_key_id or "").strip()
|
||||||
|
if not self.configured_key_id:
|
||||||
|
raise ValueError("configured_key_id is required")
|
||||||
|
|
||||||
|
def __call__(self, tenant_id: str, key_id: str) -> Ed25519CommandSigner:
|
||||||
|
if not str(tenant_id or "").strip():
|
||||||
|
raise RuntimeError("tenant_id is required for command signing")
|
||||||
|
if key_id != self.configured_key_id:
|
||||||
|
raise RuntimeError("OpenClaw signing key_id mismatch")
|
||||||
|
return Ed25519CommandSigner.from_bytes(
|
||||||
|
_read_protected_file(self.path, private=True),
|
||||||
|
key_id=key_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ControlPlaneEventPublisher(EventPublisher):
|
||||||
|
"""Append Runtime callback events to the tenant audit stream.
|
||||||
|
|
||||||
|
Callback audit remains informational. Authoritative task reconciliation is
|
||||||
|
performed only by the orchestrator after dispatch/result validation.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, service: DurableControlPlaneService) -> None:
|
||||||
|
self.service = service
|
||||||
|
|
||||||
|
def publish(self, event: Mapping[str, Any]) -> None:
|
||||||
|
tenant_id = str(event.get("tenant_id") or "").strip()
|
||||||
|
trace_id = str(event.get("trace_id") or "").strip()
|
||||||
|
command_id = str(event.get("command_id") or "").strip()
|
||||||
|
event_type = str(event.get("event_type") or "OPENCLAW_EVENT").strip()
|
||||||
|
if not tenant_id or not trace_id or not command_id:
|
||||||
|
raise ValueError("runtime audit event is missing tenant identity")
|
||||||
|
state_value = str(event.get("state") or "unknown").lower()
|
||||||
|
outcome = "allowed" if state_value == "completed" else state_value
|
||||||
|
runtime = str(event.get("runtime") or "openclaw").strip().lower()
|
||||||
|
principal_id = str(
|
||||||
|
event.get("principal_id") or f"{runtime}-runtime"
|
||||||
|
).strip()
|
||||||
|
audit = AuditEvent(
|
||||||
|
event_id=f"runtime_{command_id}_{len(self.service.store.audit_events) + 1}",
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
principal_id=principal_id,
|
||||||
|
trace_id=trace_id,
|
||||||
|
action=event_type,
|
||||||
|
resource_type=f"{runtime}_execution",
|
||||||
|
resource_id=command_id,
|
||||||
|
outcome=outcome,
|
||||||
|
policy_version="runtime-v2",
|
||||||
|
details=dict(event.get("details") or {}),
|
||||||
|
)
|
||||||
|
with self.service.store.lock:
|
||||||
|
self.service.store.audit_events.append(audit)
|
||||||
|
self.service.repository.save(self.service._snapshot())
|
||||||
|
|
||||||
|
|
||||||
|
class CandidateApplication:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
gateway: GatewayApplication,
|
||||||
|
orchestrator: ExecutionOrchestrator,
|
||||||
|
tracker: ExecutionTracker,
|
||||||
|
hermes: HermesAdapter,
|
||||||
|
openclaw: OpenClawAdapter | None,
|
||||||
|
*,
|
||||||
|
hermes_lifecycle_api_enabled: bool = False,
|
||||||
|
) -> None:
|
||||||
|
self.gateway = gateway
|
||||||
|
self.orchestrator = orchestrator
|
||||||
|
self.tracker = tracker
|
||||||
|
self.hermes = hermes
|
||||||
|
self.openclaw = openclaw
|
||||||
|
self.hermes_lifecycle_api_enabled = bool(hermes_lifecycle_api_enabled)
|
||||||
|
self.runtime_events = ControlPlaneEventPublisher(orchestrator.control_plane)
|
||||||
|
|
||||||
|
def context(self, headers: Mapping[str, str]) -> RequestContext:
|
||||||
|
return self.gateway.context_from_headers(headers)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _require_runtime_role(context: RequestContext) -> None:
|
||||||
|
if not ({"runtime", "platform_admin"} & set(context.roles)):
|
||||||
|
raise Unauthorized("runtime reconciliation role is required")
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _require_control_plane_cancel(context: RequestContext) -> None:
|
||||||
|
required = {"runtime", "platform_admin"}
|
||||||
|
if not required.issubset(set(context.roles)):
|
||||||
|
raise Unauthorized(
|
||||||
|
"Hermes cancellation requires trusted AIOS control-plane authority"
|
||||||
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _required_identity(payload: Mapping[str, Any], name: str) -> str:
|
||||||
|
value = str(payload.get(name) or "").strip()
|
||||||
|
if not value:
|
||||||
|
raise ValueError(f"{name} is required")
|
||||||
|
return value
|
||||||
|
|
||||||
|
def _record_hermes_lifecycle(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
dispatch: Any,
|
||||||
|
job: Any,
|
||||||
|
*,
|
||||||
|
action: str,
|
||||||
|
worker_id: str | None,
|
||||||
|
provenance_id: str,
|
||||||
|
run_id: str,
|
||||||
|
) -> None:
|
||||||
|
executor = worker_id or "aios-control-plane"
|
||||||
|
evidence = ExecutionEvidence(
|
||||||
|
execution_id=job.job_id,
|
||||||
|
tenant_id=job.tenant_id,
|
||||||
|
task_id=job.task_id,
|
||||||
|
agent_id=dispatch.agent_id,
|
||||||
|
executor=executor,
|
||||||
|
branch="runtime/hermes",
|
||||||
|
status=job.state.value.lower(),
|
||||||
|
progress_completed=0,
|
||||||
|
progress_total=3,
|
||||||
|
current_step=action,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
)
|
||||||
|
current_evidence = self.tracker.claim(
|
||||||
|
evidence,
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
)
|
||||||
|
if job.state in {
|
||||||
|
HermesJobState.COMPLETED,
|
||||||
|
HermesJobState.FAILED,
|
||||||
|
HermesJobState.CANCELLED,
|
||||||
|
}:
|
||||||
|
if current_evidence.completed_at is None:
|
||||||
|
self.tracker.complete(
|
||||||
|
tenant_id=job.tenant_id,
|
||||||
|
execution_id=job.job_id,
|
||||||
|
status=job.state.value.lower(),
|
||||||
|
test_status="runtime_terminal",
|
||||||
|
test_summary={"lifecycle_action": action},
|
||||||
|
result_summary={
|
||||||
|
"state": job.state.value,
|
||||||
|
"worker_id": worker_id,
|
||||||
|
"dispatch_id": dispatch.dispatch_id,
|
||||||
|
},
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
self.tracker.heartbeat(
|
||||||
|
tenant_id=job.tenant_id,
|
||||||
|
execution_id=job.job_id,
|
||||||
|
status=job.state.value.lower(),
|
||||||
|
current_step=action,
|
||||||
|
progress_completed=(1 if job.state == HermesJobState.RUNNING else 0),
|
||||||
|
progress_total=3,
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
)
|
||||||
|
self.runtime_events.publish(
|
||||||
|
{
|
||||||
|
"runtime": "hermes",
|
||||||
|
"tenant_id": job.tenant_id,
|
||||||
|
"principal_id": context.principal_id,
|
||||||
|
"trace_id": job.trace_id,
|
||||||
|
"command_id": job.job_id,
|
||||||
|
"event_type": f"HERMES_{action.upper()}",
|
||||||
|
"state": job.state.value,
|
||||||
|
"details": {
|
||||||
|
"dispatch_id": dispatch.dispatch_id,
|
||||||
|
"task_id": job.task_id,
|
||||||
|
"worker_id": worker_id,
|
||||||
|
"cell_id": job.cell_id,
|
||||||
|
"profile_id": job.profile_id,
|
||||||
|
"revision": job.revision,
|
||||||
|
"provenance_id": provenance_id,
|
||||||
|
"run_id": run_id,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
def handle_json(
|
||||||
|
self,
|
||||||
|
method: str,
|
||||||
|
raw_path: str,
|
||||||
|
headers: Mapping[str, str],
|
||||||
|
body: Any,
|
||||||
|
) -> tuple[int, Mapping[str, Any]]:
|
||||||
|
parsed = urllib.parse.urlparse(raw_path)
|
||||||
|
path = parsed.path.rstrip("/") or "/"
|
||||||
|
method = method.upper()
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/gitops":
|
||||||
|
context = self.context(headers)
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"tenant_id": context.tenant_id,
|
||||||
|
"gitops": collect_gitops_status().as_dict(),
|
||||||
|
}
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/runtime":
|
||||||
|
context = self.context(headers)
|
||||||
|
runtimes: list[dict[str, Any]] = []
|
||||||
|
if self.openclaw is None:
|
||||||
|
runtimes.append(
|
||||||
|
{
|
||||||
|
"name": "openclaw",
|
||||||
|
"status": "disabled",
|
||||||
|
"details": {"reason": "runtime_not_configured"},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
health = self.openclaw.health()
|
||||||
|
runtimes.append(
|
||||||
|
{
|
||||||
|
"name": "openclaw",
|
||||||
|
"status": health.status,
|
||||||
|
"details": _json_value(health),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
runtimes.append(
|
||||||
|
{
|
||||||
|
"name": "openclaw",
|
||||||
|
"status": "unavailable",
|
||||||
|
"details": {"message": str(exc)},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
hermes_health = self.hermes.health()
|
||||||
|
runtimes.append(
|
||||||
|
{
|
||||||
|
"name": "hermes",
|
||||||
|
"status": hermes_health.status,
|
||||||
|
"details": _json_value(hermes_health),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"tenant_id": context.tenant_id,
|
||||||
|
"runtimes": runtimes,
|
||||||
|
}
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/executions":
|
||||||
|
context = self.context(headers)
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"executions": _json_value(
|
||||||
|
self.tracker.list(tenant_id=context.tenant_id)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
dispatch_match = __import__("re").fullmatch(
|
||||||
|
r"/api/v2/tasks/([^/]+)/dispatch", path
|
||||||
|
)
|
||||||
|
if method == "POST" and dispatch_match:
|
||||||
|
context = self.context(headers)
|
||||||
|
task_id = urllib.parse.unquote(dispatch_match.group(1))
|
||||||
|
payload = body if isinstance(body, Mapping) else {}
|
||||||
|
timeout = payload.get("timeout_seconds")
|
||||||
|
provenance_id, run_id = _provenance_context(headers, payload)
|
||||||
|
record = self.orchestrator.dispatch(
|
||||||
|
context,
|
||||||
|
task_id=task_id,
|
||||||
|
timeout_seconds=float(timeout) if timeout is not None else None,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
)
|
||||||
|
return HTTPStatus.ACCEPTED, {"dispatch": _json_value(record)}
|
||||||
|
|
||||||
|
lifecycle_match = __import__("re").fullmatch(
|
||||||
|
r"/api/v2/internal/runtime/hermes/lifecycle/"
|
||||||
|
r"(claim|start|heartbeat|complete|fail|cancel|status)",
|
||||||
|
path,
|
||||||
|
)
|
||||||
|
if method == "POST" and lifecycle_match:
|
||||||
|
if not self.hermes_lifecycle_api_enabled:
|
||||||
|
return HTTPStatus.NOT_FOUND, {
|
||||||
|
"error": "not_found",
|
||||||
|
"message": "route not found",
|
||||||
|
}
|
||||||
|
context = self.context(headers)
|
||||||
|
self._require_runtime_role(context)
|
||||||
|
if not isinstance(body, Mapping):
|
||||||
|
raise ValueError("JSON body must be an object")
|
||||||
|
payload = body
|
||||||
|
action = lifecycle_match.group(1)
|
||||||
|
identity = {
|
||||||
|
name: self._required_identity(payload, name)
|
||||||
|
for name in (
|
||||||
|
"dispatch_id",
|
||||||
|
"job_id",
|
||||||
|
"tenant_id",
|
||||||
|
"task_id",
|
||||||
|
"trace_id",
|
||||||
|
"principal_id",
|
||||||
|
"provenance_id",
|
||||||
|
"run_id",
|
||||||
|
"cell_id",
|
||||||
|
"profile_id",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
header_provenance, header_run = _provenance_context(headers, payload)
|
||||||
|
if header_provenance != identity["provenance_id"]:
|
||||||
|
raise ValueError("provenance_id context mismatch")
|
||||||
|
if header_run != identity["run_id"]:
|
||||||
|
raise ValueError("run_id context mismatch")
|
||||||
|
requeued_job_ids = self.hermes.requeue_expired()
|
||||||
|
dispatch, job = self.orchestrator.hermes_lifecycle_context(
|
||||||
|
context,
|
||||||
|
dispatch_id=identity["dispatch_id"],
|
||||||
|
job_id=identity["job_id"],
|
||||||
|
tenant_id=identity["tenant_id"],
|
||||||
|
task_id=identity["task_id"],
|
||||||
|
trace_id=identity["trace_id"],
|
||||||
|
principal_id=identity["principal_id"],
|
||||||
|
provenance_id=identity["provenance_id"],
|
||||||
|
run_id=identity["run_id"],
|
||||||
|
)
|
||||||
|
if job.cell_id != identity["cell_id"]:
|
||||||
|
raise ValueError("cell_id mismatch")
|
||||||
|
if job.profile_id != identity["profile_id"]:
|
||||||
|
raise ValueError("profile_id mismatch")
|
||||||
|
|
||||||
|
if action == "status":
|
||||||
|
if job.job_id in requeued_job_ids:
|
||||||
|
self._record_hermes_lifecycle(
|
||||||
|
context,
|
||||||
|
dispatch,
|
||||||
|
job,
|
||||||
|
action="lease_expired_requeue",
|
||||||
|
worker_id=None,
|
||||||
|
provenance_id=identity["provenance_id"],
|
||||||
|
run_id=identity["run_id"],
|
||||||
|
)
|
||||||
|
task = self.orchestrator.control_plane.get_task(
|
||||||
|
context, dispatch.task_id
|
||||||
|
)
|
||||||
|
response: dict[str, Any] = {
|
||||||
|
"job": _json_value(job),
|
||||||
|
"dispatch": _json_value(dispatch),
|
||||||
|
"task": _json_value(task),
|
||||||
|
}
|
||||||
|
if job.state in {
|
||||||
|
HermesJobState.COMPLETED,
|
||||||
|
HermesJobState.FAILED,
|
||||||
|
HermesJobState.CANCELLED,
|
||||||
|
}:
|
||||||
|
response["result"] = _json_value(
|
||||||
|
self.hermes.result(job.job_id, tenant_id=job.tenant_id)
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, response
|
||||||
|
|
||||||
|
expected_revision = payload.get("expected_revision")
|
||||||
|
expected = (
|
||||||
|
int(expected_revision) if expected_revision is not None else None
|
||||||
|
)
|
||||||
|
if action == "cancel":
|
||||||
|
self._require_control_plane_cancel(context)
|
||||||
|
if payload.get("worker_id") is not None:
|
||||||
|
raise Unauthorized("Hermes workers may observe but not initiate cancel")
|
||||||
|
if payload.get("cancellation_source") != "aios-control-plane":
|
||||||
|
raise ValueError(
|
||||||
|
"cancellation_source must be aios-control-plane"
|
||||||
|
)
|
||||||
|
result = self.hermes.cancel(
|
||||||
|
job.job_id,
|
||||||
|
tenant_id=job.tenant_id,
|
||||||
|
expected_revision=expected,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
worker_id = self._required_identity(payload, "worker_id")
|
||||||
|
if action == "claim":
|
||||||
|
claimed = self.hermes.claim(
|
||||||
|
tenant_id=job.tenant_id,
|
||||||
|
cell_id=job.cell_id,
|
||||||
|
profile_id=job.profile_id,
|
||||||
|
worker_id=worker_id,
|
||||||
|
lease_seconds=(
|
||||||
|
int(payload["lease_seconds"])
|
||||||
|
if payload.get("lease_seconds") is not None
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
job_id=job.job_id,
|
||||||
|
)
|
||||||
|
if claimed is None:
|
||||||
|
raise ValueError("Hermes job is not claimable")
|
||||||
|
bound = self.orchestrator.bind_hermes_worker(
|
||||||
|
context,
|
||||||
|
claimed,
|
||||||
|
dispatch_id=dispatch.dispatch_id,
|
||||||
|
provenance_id=identity["provenance_id"],
|
||||||
|
run_id=identity["run_id"],
|
||||||
|
worker_id=worker_id,
|
||||||
|
)
|
||||||
|
self._record_hermes_lifecycle(
|
||||||
|
context,
|
||||||
|
bound,
|
||||||
|
claimed,
|
||||||
|
action="claim",
|
||||||
|
worker_id=worker_id,
|
||||||
|
provenance_id=identity["provenance_id"],
|
||||||
|
run_id=identity["run_id"],
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"job": _json_value(claimed),
|
||||||
|
"dispatch": _json_value(bound),
|
||||||
|
}
|
||||||
|
if action == "start":
|
||||||
|
started = self.hermes.start(
|
||||||
|
job.job_id,
|
||||||
|
tenant_id=job.tenant_id,
|
||||||
|
worker_id=worker_id,
|
||||||
|
expected_revision=expected,
|
||||||
|
)
|
||||||
|
self._record_hermes_lifecycle(
|
||||||
|
context,
|
||||||
|
dispatch,
|
||||||
|
started,
|
||||||
|
action="start",
|
||||||
|
worker_id=worker_id,
|
||||||
|
provenance_id=identity["provenance_id"],
|
||||||
|
run_id=identity["run_id"],
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {"job": _json_value(started)}
|
||||||
|
if action == "heartbeat":
|
||||||
|
heartbeat = self.hermes.heartbeat(
|
||||||
|
job.job_id,
|
||||||
|
tenant_id=job.tenant_id,
|
||||||
|
worker_id=worker_id,
|
||||||
|
lease_seconds=(
|
||||||
|
int(payload["lease_seconds"])
|
||||||
|
if payload.get("lease_seconds") is not None
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
)
|
||||||
|
self._record_hermes_lifecycle(
|
||||||
|
context,
|
||||||
|
dispatch,
|
||||||
|
heartbeat,
|
||||||
|
action="heartbeat",
|
||||||
|
worker_id=worker_id,
|
||||||
|
provenance_id=identity["provenance_id"],
|
||||||
|
run_id=identity["run_id"],
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {"job": _json_value(heartbeat)}
|
||||||
|
if action == "complete":
|
||||||
|
output = payload.get("output")
|
||||||
|
if not isinstance(output, Mapping):
|
||||||
|
raise ValueError("output must be an object")
|
||||||
|
result = self.hermes.complete(
|
||||||
|
job.job_id,
|
||||||
|
tenant_id=job.tenant_id,
|
||||||
|
worker_id=worker_id,
|
||||||
|
output=output,
|
||||||
|
expected_revision=expected,
|
||||||
|
)
|
||||||
|
elif action == "fail":
|
||||||
|
error = payload.get("error")
|
||||||
|
if not isinstance(error, Mapping):
|
||||||
|
raise ValueError("error must be an object")
|
||||||
|
result = self.hermes.fail(
|
||||||
|
job.job_id,
|
||||||
|
tenant_id=job.tenant_id,
|
||||||
|
worker_id=worker_id,
|
||||||
|
error=error,
|
||||||
|
expected_revision=expected,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
raise ValueError("unsupported Hermes lifecycle action")
|
||||||
|
|
||||||
|
task = self.orchestrator.reconcile_hermes(
|
||||||
|
context,
|
||||||
|
result,
|
||||||
|
dispatch_id=dispatch.dispatch_id,
|
||||||
|
provenance_id=identity["provenance_id"],
|
||||||
|
run_id=identity["run_id"],
|
||||||
|
)
|
||||||
|
updated_dispatch = self.orchestrator.dispatch_repository.get(
|
||||||
|
dispatch.dispatch_id, tenant_id=dispatch.tenant_id
|
||||||
|
)
|
||||||
|
terminal_job = self.hermes.get(job.job_id, tenant_id=job.tenant_id)
|
||||||
|
self._record_hermes_lifecycle(
|
||||||
|
context,
|
||||||
|
updated_dispatch,
|
||||||
|
terminal_job,
|
||||||
|
action=action,
|
||||||
|
worker_id=result.worker_id,
|
||||||
|
provenance_id=identity["provenance_id"],
|
||||||
|
run_id=identity["run_id"],
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"result": _json_value(result),
|
||||||
|
"task": _json_value(task),
|
||||||
|
"dispatch": _json_value(updated_dispatch),
|
||||||
|
}
|
||||||
|
|
||||||
|
if method == "POST" and path == "/api/v2/internal/runtime/hermes/reconcile":
|
||||||
|
context = self.context(headers)
|
||||||
|
self._require_runtime_role(context)
|
||||||
|
if not isinstance(body, Mapping):
|
||||||
|
raise ValueError("JSON body must be an object")
|
||||||
|
provenance_id, run_id = _provenance_context(headers, body)
|
||||||
|
if not provenance_id or not run_id:
|
||||||
|
raise ValueError("provenance_id and run_id are required")
|
||||||
|
dispatch_id = str(body.get("dispatch_id") or "").strip()
|
||||||
|
if not dispatch_id:
|
||||||
|
raise ValueError("dispatch_id is required")
|
||||||
|
result = HermesResult(
|
||||||
|
job_id=str(body["job_id"]),
|
||||||
|
tenant_id=str(body["tenant_id"]),
|
||||||
|
task_id=str(body["task_id"]),
|
||||||
|
trace_id=str(body["trace_id"]),
|
||||||
|
state=HermesJobState(str(body["state"])),
|
||||||
|
worker_id=(
|
||||||
|
str(body["worker_id"])
|
||||||
|
if body.get("worker_id") is not None
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
output=body.get("output"),
|
||||||
|
error=body.get("error"),
|
||||||
|
started_at=body.get("started_at"),
|
||||||
|
finished_at=body.get("finished_at"),
|
||||||
|
)
|
||||||
|
task = self.orchestrator.reconcile_hermes(
|
||||||
|
context,
|
||||||
|
result,
|
||||||
|
dispatch_id=dispatch_id,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {"task": _json_value(task)}
|
||||||
|
|
||||||
|
if method == "POST" and path == "/api/v2/hermes/results/reconcile":
|
||||||
|
return HTTPStatus.GONE, {
|
||||||
|
"error": "RuntimeReconcileEndpointMoved",
|
||||||
|
"message": "use /api/v2/internal/runtime/hermes/reconcile",
|
||||||
|
}
|
||||||
|
|
||||||
|
return self.gateway.handle(method, raw_path, headers, body)
|
||||||
|
|
||||||
|
|
||||||
|
class CandidateRequestHandler(BaseHTTPRequestHandler):
|
||||||
|
server_version = "AIOSV2Candidate/2.0"
|
||||||
|
max_body_bytes = 1_048_576
|
||||||
|
|
||||||
|
@property
|
||||||
|
def application(self) -> CandidateApplication:
|
||||||
|
return self.server.application # type: ignore[attr-defined]
|
||||||
|
|
||||||
|
def _read_body(self) -> Any:
|
||||||
|
length = int(self.headers.get("Content-Length", "0") or "0")
|
||||||
|
if length < 0 or length > self.max_body_bytes:
|
||||||
|
raise ValueError("request body is too large")
|
||||||
|
if length == 0:
|
||||||
|
return {}
|
||||||
|
content_type = self.headers.get("Content-Type", "").split(";", 1)[0].strip()
|
||||||
|
if content_type != "application/json":
|
||||||
|
raise ValueError("Content-Type must be application/json")
|
||||||
|
return json.loads(self.rfile.read(length).decode("utf-8"))
|
||||||
|
|
||||||
|
def _security_headers(self) -> None:
|
||||||
|
self.send_header("Cache-Control", "no-store")
|
||||||
|
self.send_header("X-Content-Type-Options", "nosniff")
|
||||||
|
self.send_header("X-Frame-Options", "DENY")
|
||||||
|
self.send_header("Referrer-Policy", "no-referrer")
|
||||||
|
self.send_header(
|
||||||
|
"Content-Security-Policy",
|
||||||
|
"default-src 'self'; style-src 'unsafe-inline'; script-src 'unsafe-inline'; connect-src 'self'",
|
||||||
|
)
|
||||||
|
|
||||||
|
def _send_json(self, status_code: int, payload: Mapping[str, Any]) -> None:
|
||||||
|
encoded = json.dumps(
|
||||||
|
_json_value(payload), ensure_ascii=False, separators=(",", ":")
|
||||||
|
).encode("utf-8")
|
||||||
|
self.send_response(int(status_code))
|
||||||
|
self.send_header("Content-Type", "application/json; charset=utf-8")
|
||||||
|
self.send_header("Content-Length", str(len(encoded)))
|
||||||
|
self._security_headers()
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(encoded)
|
||||||
|
|
||||||
|
def _send_console(self) -> None:
|
||||||
|
encoded = CONSOLE_HTML.encode("utf-8")
|
||||||
|
self.send_response(HTTPStatus.OK)
|
||||||
|
self.send_header("Content-Type", "text/html; charset=utf-8")
|
||||||
|
self.send_header("Content-Length", str(len(encoded)))
|
||||||
|
self._security_headers()
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(encoded)
|
||||||
|
|
||||||
|
def _dispatch(self, method: str) -> None:
|
||||||
|
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
|
||||||
|
if method == "GET" and path in {"/", "/console"}:
|
||||||
|
self._send_console()
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
body = self._read_body() if method == "POST" else {}
|
||||||
|
status_code, payload = self.application.handle_json(
|
||||||
|
method,
|
||||||
|
self.path,
|
||||||
|
{key: value for key, value in self.headers.items()},
|
||||||
|
body,
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
status_code, payload = error_response(exc)
|
||||||
|
self._send_json(status_code, payload)
|
||||||
|
|
||||||
|
def do_GET(self) -> None: # noqa: N802
|
||||||
|
self._dispatch("GET")
|
||||||
|
|
||||||
|
def do_POST(self) -> None: # noqa: N802
|
||||||
|
self._dispatch("POST")
|
||||||
|
|
||||||
|
|
||||||
|
class CandidateHTTPServer(ThreadingHTTPServer):
|
||||||
|
daemon_threads = True
|
||||||
|
allow_reuse_address = True
|
||||||
|
|
||||||
|
def __init__(self, address: tuple[str, int], application: CandidateApplication) -> None:
|
||||||
|
self.application = application
|
||||||
|
super().__init__(address, CandidateRequestHandler)
|
||||||
|
|
||||||
|
|
||||||
|
def build_application() -> CandidateApplication:
|
||||||
|
state_dir = Path(os.getenv("AIOS_V2_STATE_DIR", "/var/lib/aios-v2")).resolve()
|
||||||
|
state_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
repository = SQLiteSnapshotRepository(state_dir / "control-plane.sqlite3")
|
||||||
|
dispatch_repository = SQLiteDispatchRepository(state_dir / "dispatches.sqlite3")
|
||||||
|
service = DurableControlPlaneService(repository)
|
||||||
|
require_provenance = _enabled(os.getenv("AIOS_REQUIRE_RUN_PROVENANCE", "0"))
|
||||||
|
provenance_writer = build_provenance_writer_from_env(
|
||||||
|
state_dir=state_dir, required=require_provenance
|
||||||
|
)
|
||||||
|
tracker = ExecutionTracker(
|
||||||
|
state_dir / "executions",
|
||||||
|
provenance_writer=provenance_writer,
|
||||||
|
require_provenance=require_provenance,
|
||||||
|
)
|
||||||
|
hermes = HermesAdapter(store=SQLiteHermesStore(state_dir / "hermes.sqlite3"))
|
||||||
|
|
||||||
|
openclaw_url = os.getenv("AIOS_OPENCLAW_URL", "").strip()
|
||||||
|
signing_file = os.getenv("AIOS_OPENCLAW_SIGNING_MATERIAL_FILE", "").strip()
|
||||||
|
signing_key_id = os.getenv(
|
||||||
|
"AIOS_OPENCLAW_SIGNING_KEY_ID", "openclaw-v2r1-candidate-ed25519"
|
||||||
|
).strip()
|
||||||
|
openclaw = None
|
||||||
|
resolver = None
|
||||||
|
if openclaw_url and signing_file:
|
||||||
|
resolver = FileCommandSignerResolver(
|
||||||
|
signing_file,
|
||||||
|
configured_key_id=signing_key_id,
|
||||||
|
)
|
||||||
|
transport = HTTPRuntimeTransport(
|
||||||
|
openclaw_url,
|
||||||
|
credential_ref=os.getenv("AIOS_OPENCLAW_CREDENTIAL_REF") or None,
|
||||||
|
)
|
||||||
|
openclaw = OpenClawAdapter(
|
||||||
|
transport,
|
||||||
|
resolver,
|
||||||
|
callback_sink=AuditCallbackSink(ControlPlaneEventPublisher(service)),
|
||||||
|
)
|
||||||
|
|
||||||
|
gateway = GatewayApplication(service)
|
||||||
|
orchestrator = ExecutionOrchestrator(
|
||||||
|
service,
|
||||||
|
openclaw=openclaw,
|
||||||
|
hermes=hermes,
|
||||||
|
openclaw_signing_material_resolver=resolver,
|
||||||
|
openclaw_key_id=signing_key_id,
|
||||||
|
provenance_writer=provenance_writer,
|
||||||
|
require_provenance=require_provenance,
|
||||||
|
dispatch_repository=dispatch_repository,
|
||||||
|
)
|
||||||
|
return CandidateApplication(
|
||||||
|
gateway,
|
||||||
|
orchestrator,
|
||||||
|
tracker,
|
||||||
|
hermes,
|
||||||
|
openclaw,
|
||||||
|
hermes_lifecycle_api_enabled=_enabled(
|
||||||
|
os.getenv("AIOS_HERMES_LIFECYCLE_API_ENABLED", "0")
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
host = os.getenv("AIOS_V2_HOST", "127.0.0.1")
|
||||||
|
port = int(os.getenv("AIOS_V2_PORT", "18920"))
|
||||||
|
server = CandidateHTTPServer((host, port), build_application())
|
||||||
|
print(f"AIOS V2 candidate listening on {host}:{port}")
|
||||||
|
server.serve_forever()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
"""Dependency-free, role-aware AIOS unified portal.
|
||||||
|
|
||||||
|
The browser receives only server-authorized capabilities and tenant-scoped data.
|
||||||
|
It never selects a tenant, signs Provenance, mutates Git tags, or calls Runtime
|
||||||
|
services directly.
|
||||||
|
"""
|
||||||
|
|
||||||
|
CONSOLE_HTML = r'''<!doctype html>
|
||||||
|
<html lang="zh-CN">
|
||||||
|
<head>
|
||||||
|
<meta charset="utf-8">
|
||||||
|
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||||||
|
<meta name="referrer" content="no-referrer">
|
||||||
|
<title>AIOS Unified Portal</title>
|
||||||
|
<style>
|
||||||
|
:root{font-family:Inter,"PingFang SC","Microsoft YaHei",ui-sans-serif,system-ui,sans-serif;color:#172033;background:#eef2f7;--nav:#0f172a;--card:#fff;--line:#dfe6ef;--muted:#667085;--blue:#2563eb;--green:#047857;--red:#b42318;--amber:#b54708}
|
||||||
|
*{box-sizing:border-box}body{margin:0;min-height:100vh}button,input,textarea,select{font:inherit}
|
||||||
|
.shell{display:grid;grid-template-columns:250px minmax(0,1fr);min-height:100vh}.sidebar{background:var(--nav);color:#e2e8f0;padding:22px 16px;position:sticky;top:0;height:100vh;overflow:auto}.brand{font-weight:800;font-size:20px;letter-spacing:.2px}.version{font-size:12px;color:#94a3b8;margin:5px 0 22px}.identity{padding:12px;background:#182338;border:1px solid #2b3950;border-radius:12px;margin-bottom:18px}.identity strong{display:block;color:#fff}.identity span{font-size:12px;color:#aebbd0;display:block;margin-top:3px;overflow-wrap:anywhere}.nav button{width:100%;display:flex;gap:10px;align-items:center;border:0;background:transparent;color:#cbd5e1;padding:11px 12px;border-radius:9px;text-align:left;cursor:pointer;margin:2px 0}.nav button:hover,.nav button.active{background:#24324b;color:#fff}.nav .dot{width:8px;height:8px;border-radius:50%;background:#60a5fa}.main{min-width:0}.topbar{height:70px;background:#fff;border-bottom:1px solid var(--line);display:flex;align-items:center;justify-content:space-between;padding:0 26px;position:sticky;top:0;z-index:4}.topbar h1{font-size:19px;margin:0}.top-actions{display:flex;align-items:center;gap:10px}.timestamp{font-size:12px;color:var(--muted)}.content{padding:24px;max-width:1680px;margin:auto}.notice{background:#eff6ff;border:1px solid #bfdbfe;border-radius:12px;padding:12px 14px;color:#1e3a8a;font-size:13px;margin-bottom:18px}.error{background:#fef3f2;border:1px solid #fecdca;color:var(--red);border-radius:10px;padding:10px 12px;white-space:pre-wrap;margin-bottom:14px;display:none}.cards{display:grid;grid-template-columns:repeat(auto-fit,minmax(170px,1fr));gap:14px;margin-bottom:18px}.card,.panel{background:var(--card);border:1px solid var(--line);border-radius:14px;box-shadow:0 1px 3px rgba(16,24,40,.05)}.card{padding:16px}.card small{color:var(--muted);font-weight:600}.card b{display:block;font-size:28px;margin-top:7px}.panel{padding:17px;margin-bottom:16px;overflow:auto}.panel h2{font-size:16px;margin:0}.panel-head{display:flex;align-items:center;justify-content:space-between;gap:12px;margin-bottom:13px}.grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(410px,1fr));gap:16px}.wide{grid-column:1/-1}.muted{color:var(--muted)}.good{color:var(--green);font-weight:700}.bad{color:var(--red);font-weight:700}.warn{color:var(--amber);font-weight:700}.pill{display:inline-block;border:1px solid #cbd5e1;border-radius:999px;padding:3px 8px;font-size:11px;margin:2px;background:#f8fafc}.pill.good{background:#ecfdf3;border-color:#a6f4c5}.pill.bad{background:#fef3f2;border-color:#fecdca}.toolbar{display:flex;gap:8px;align-items:center;flex-wrap:wrap}.btn{border:0;border-radius:9px;padding:9px 12px;background:var(--blue);color:#fff;cursor:pointer;font-weight:650}.btn.secondary{background:#fff;color:#344054;border:1px solid #d0d5dd}.btn.danger{background:#d92d20}.btn:disabled{opacity:.45;cursor:not-allowed}.icon-btn{padding:7px 10px}.view{display:none}.view.active{display:block}table{width:100%;border-collapse:collapse;font-size:12px}th,td{text-align:left;padding:9px 8px;border-bottom:1px solid #edf0f4;vertical-align:top}th{color:#667085;font-weight:700;white-space:nowrap}td{overflow-wrap:anywhere}.empty{padding:24px;text-align:center;color:var(--muted)}.timeline{border-left:2px solid #dbe4f0;margin-left:8px;padding-left:20px}.event{position:relative;padding:0 0 17px}.event:before{content:"";position:absolute;left:-27px;top:4px;width:11px;height:11px;border-radius:50%;background:#3b82f6;border:3px solid #fff;box-shadow:0 0 0 1px #93c5fd}.event strong{font-size:13px}.event p{margin:4px 0;color:var(--muted);font-size:12px}.chain{display:flex;align-items:stretch;gap:7px;overflow:auto;padding:5px 0}.chain-item{min-width:135px;border:1px solid var(--line);border-radius:10px;padding:10px;background:#f8fafc}.chain-arrow{display:flex;align-items:center;color:#94a3b8}.form-grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(230px,1fr));gap:12px}.field label{font-size:12px;color:#475467;font-weight:650;display:block;margin-bottom:5px}.field input,.field textarea,.field select{width:100%;border:1px solid #d0d5dd;border-radius:9px;padding:9px;background:#fff}.field textarea{min-height:90px;resize:vertical}.dialog{border:0;border-radius:14px;padding:0;max-width:650px;width:calc(100% - 32px);box-shadow:0 20px 60px rgba(15,23,42,.28)}.dialog::backdrop{background:rgba(15,23,42,.6)}.dialog-head{padding:16px 18px;border-bottom:1px solid var(--line);font-weight:750}.dialog-body{padding:18px}.dialog-actions{padding:14px 18px;border-top:1px solid var(--line);display:flex;justify-content:flex-end;gap:8px}.kv{display:grid;grid-template-columns:150px minmax(0,1fr);gap:6px 12px;font-size:12px}.kv div:nth-child(odd){color:var(--muted)}pre{white-space:pre-wrap;word-break:break-word;font-size:11px;background:#0f172a;color:#dbeafe;border-radius:10px;padding:12px;max-height:300px;overflow:auto}
|
||||||
|
@media(max-width:900px){.shell{grid-template-columns:1fr}.sidebar{height:auto;position:static}.nav{display:flex;overflow:auto}.nav button{min-width:max-content}.topbar{position:static}.grid{grid-template-columns:1fr}.content{padding:16px}}
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div class="shell">
|
||||||
|
<aside class="sidebar">
|
||||||
|
<div class="brand">AIOS Unified Portal</div><div class="version">V2R1C1 · Role-aware Control Plane</div>
|
||||||
|
<div class="identity"><strong id="principal">加载身份…</strong><span id="tenant"></span><span id="roles"></span></div>
|
||||||
|
<nav id="navigation" class="nav"></nav>
|
||||||
|
</aside>
|
||||||
|
<section class="main">
|
||||||
|
<header class="topbar"><h1 id="pageTitle">系统总览</h1><div class="top-actions"><span id="updated" class="timestamp">尚未加载</span><button class="btn secondary icon-btn" id="refresh">刷新</button></div></header>
|
||||||
|
<main class="content">
|
||||||
|
<div class="notice">身份、租户和角色由可信反向代理注入。门户只调用 AIOS Control Plane;Provenance 签发、Tag、生产部署与 Runtime 权威状态仍由受控流程执行。</div>
|
||||||
|
<div id="error" class="error"></div>
|
||||||
|
<section id="overview" class="view active"></section>
|
||||||
|
<section id="tasks" class="view"></section>
|
||||||
|
<section id="authorization" class="view"></section>
|
||||||
|
<section id="runtime" class="view"></section>
|
||||||
|
<section id="evidence" class="view"></section>
|
||||||
|
<section id="audit" class="view"></section>
|
||||||
|
<section id="release" class="view"></section>
|
||||||
|
</main>
|
||||||
|
</section>
|
||||||
|
</div>
|
||||||
|
<dialog id="taskDialog" class="dialog">
|
||||||
|
<div class="dialog-head">创建任务</div><form id="taskForm"><div class="dialog-body form-grid">
|
||||||
|
<div class="field"><label>Agent ID</label><input name="agent_id" required></div>
|
||||||
|
<div class="field"><label>Action</label><input name="action" required></div>
|
||||||
|
<div class="field"><label>Idempotency Key</label><input name="idempotency_key" required></div>
|
||||||
|
<div class="field wide"><label>Payload JSON</label><textarea name="payload">{}</textarea></div>
|
||||||
|
</div><div class="dialog-actions"><button type="button" class="btn secondary" data-close="taskDialog">取消</button><button class="btn">创建</button></div></form>
|
||||||
|
</dialog>
|
||||||
|
<dialog id="confirmDialog" class="dialog"><div class="dialog-head" id="confirmTitle">确认操作</div><form id="confirmForm"><div class="dialog-body"><p id="confirmText"></p><div class="field"><label>原因</label><textarea id="confirmReason" required></textarea></div></div><div class="dialog-actions"><button type="button" class="btn secondary" data-close="confirmDialog">取消</button><button class="btn danger">确认</button></div></form></dialog>
|
||||||
|
<script>
|
||||||
|
const $=s=>document.querySelector(s), esc=v=>String(v??'').replace(/[&<>"']/g,c=>({'&':'&','<':'<','>':'>','"':'"',"'":'''}[c]));
|
||||||
|
const state={identity:null,current:'overview',cache:{},confirm:null};
|
||||||
|
const labels={overview:'系统总览',tasks:'任务中心',authorization:'授权中心',runtime:'运行与运维',evidence:'执行证据',audit:'审计中心',release:'GitOps 与发布'};
|
||||||
|
function has(c){return !!state.identity?.capabilities?.includes(c)}
|
||||||
|
async function api(path,options={}){const r=await fetch(path,{credentials:'same-origin',headers:{'Accept':'application/json','Content-Type':'application/json',...(options.headers||{})},...options});const d=await r.json().catch(()=>({}));if(!r.ok)throw new Error(`${r.status} ${d.message||d.error||path}`);return d}
|
||||||
|
function err(e){const el=$('#error');el.textContent=e?.message||String(e);el.style.display='block'}function clearErr(){$('#error').style.display='none';$('#error').textContent=''}
|
||||||
|
function pill(v){const s=String(v??'unknown'),c=/complete|pass|healthy|active|synced|approved|used/i.test(s)?'good':/fail|deny|reject|revoke|expire|cancel|unavailable/i.test(s)?'bad':'';return `<span class="pill ${c}">${esc(s)}</span>`}
|
||||||
|
function table(cols,rows){if(!rows?.length)return '<div class="empty">暂无数据</div>';return `<table><thead><tr>${cols.map(x=>`<th>${esc(x[0])}</th>`).join('')}</tr></thead><tbody>${rows.map(r=>`<tr>${cols.map(x=>`<td>${x[1](r)}</td>`).join('')}</tr>`).join('')}</tbody></table>`}
|
||||||
|
function card(label,value,detail=''){return `<div class="card"><small>${esc(label)}</small><b>${esc(value)}</b>${detail?`<span class="muted">${esc(detail)}</span>`:''}</div>`}
|
||||||
|
function json(v){return `<pre>${esc(JSON.stringify(v,null,2))}</pre>`}
|
||||||
|
function show(view){state.current=view;document.querySelectorAll('.view').forEach(x=>x.classList.toggle('active',x.id===view));document.querySelectorAll('.nav button').forEach(x=>x.classList.toggle('active',x.dataset.view===view));$('#pageTitle').textContent=labels[view]||view;load(view)}
|
||||||
|
function navigation(){const nav=$('#navigation');nav.innerHTML=(state.identity.navigation||[]).map(x=>`<button data-view="${esc(x.id)}"><span class="dot"></span>${esc(x.label)}</button>`).join('');nav.querySelectorAll('button').forEach(b=>b.onclick=()=>show(b.dataset.view));}
|
||||||
|
async function identity(){state.identity=await api('/api/v2/me/capabilities');$('#principal').textContent=state.identity.principal_id;$('#tenant').textContent=`租户:${state.identity.tenant_id}`;$('#roles').textContent=`角色:${state.identity.roles.join(', ')||'无'}`;navigation();if(!state.identity.navigation.some(x=>x.id===state.current))state.current=state.identity.navigation[0]?.id||'overview';show(state.current)}
|
||||||
|
async function load(view){clearErr();try{if(view==='overview')await overview();if(view==='tasks')await tasks();if(view==='authorization')await authorizations();if(view==='runtime')await runtime();if(view==='evidence')await evidence();if(view==='audit')await audit();if(view==='release')await release();$('#updated').textContent=`更新于 ${new Date().toLocaleString()}`}catch(e){err(e)}}
|
||||||
|
async function overview(){const d=await api('/api/v2/portal/overview');state.cache.overview=d;const s=d.summary||{},rt=d.sections?.runtime?.runtimes||[],rel=d.sections?.release||{};$('#overview').innerHTML=`<div class="cards">${card('任务',s.tasks||0)}${card('活动 Dispatch',s.active_dispatches||0,`总数 ${s.dispatches||0}`)}${card('执行证据',s.evidence||0)}${card('审计事件',s.audit_events||0)}${card('当前版本',rel.version||'—',rel.status||'')}${card('授权 Grant',d.sections?.authorization?.active_grants||0,`总数 ${d.sections?.authorization?.grants||0}`)}</div><div class="grid"><div class="panel"><div class="panel-head"><h2>Runtime 健康</h2></div>${table([['Runtime',x=>esc(x.name)],['状态',x=>pill(x.status)],['版本/详情',x=>esc(x.details?.version||x.details?.runtime_id||'—')]],rt)}</div><div class="panel"><div class="panel-head"><h2>任务状态</h2></div>${table([['状态',x=>pill(x[0])],['数量',x=>esc(x[1])]],Object.entries(s.task_states||{}))}</div><div class="panel wide"><div class="panel-head"><h2>最近任务</h2></div>${table([['任务',x=>esc(x.task_id)],['Agent',x=>esc(x.agent_id)],['动作',x=>esc(x.action)],['状态',x=>pill(x.state)],['更新时间',x=>esc(x.updated_at)]],d.recent?.tasks||[])}</div></div>`}
|
||||||
|
async function tasks(){const d=await api('/api/v2/portal/tasks');state.cache.tasks=d.items||[];const create=has('task.create')?'<button class="btn" id="newTask">创建任务</button>':'';$('#tasks').innerHTML=`<div class="panel"><div class="panel-head"><h2>任务全链路</h2><div class="toolbar">${create}</div></div>${(d.items||[]).map(taskBlock).join('')||'<div class="empty">暂无任务</div>'}</div>`;$('#newTask')?.addEventListener('click',()=>$('#taskDialog').showModal());document.querySelectorAll('[data-task-action]').forEach(b=>b.onclick=()=>taskAction(b.dataset.taskId,b.dataset.taskAction,b.dataset.revision))}
|
||||||
|
function taskBlock(x){const t=x.task||{},d=x.dispatch,e=x.evidence||[],a=x.audit||[],actions=(x.available_actions||[]).map(v=>`<button class="btn ${v==='cancel'?'danger':'secondary'}" data-task-action="${v}" data-task-id="${esc(t.task_id)}" data-revision="${esc(t.revision)}">${{approve:'审批',cancel:'取消',dispatch:'调度'}[v]||v}</button>`).join('');return `<div class="panel" style="background:#fbfcfe"><div class="panel-head"><h2>${esc(t.task_id)} · ${esc(t.action)}</h2><div class="toolbar">${pill(t.state)}${actions}</div></div><div class="chain"><div class="chain-item"><b>Task</b><div>${esc(t.agent_id)}</div><small>${esc(t.created_by)}</small></div><span class="chain-arrow">→</span><div class="chain-item"><b>Approval</b><div>${esc(t.approved_by||'待审批')}</div><small>revision ${esc(t.revision)}</small></div><span class="chain-arrow">→</span><div class="chain-item"><b>Provenance</b><div>${esc(d?.provenance_id||'未调度')}</div><small>${esc(d?.authorization_id||'')}</small></div><span class="chain-arrow">→</span><div class="chain-item"><b>Dispatch</b><div>${esc(d?.dispatch_id||'—')}</div><small>${pill(d?.state||'not_created')}</small></div><span class="chain-arrow">→</span><div class="chain-item"><b>Runtime</b><div>${esc(d?.runtime||'—')}</div><small>${esc(d?.runtime_execution_id||'')}</small></div><span class="chain-arrow">→</span><div class="chain-item"><b>Evidence</b><div>${e.length} 条</div><small>Audit ${a.length} 条</small></div></div></div>`}
|
||||||
|
async function taskAction(id,action,revision){clearErr();try{let body={expected_revision:Number(revision)};if(action==='dispatch'){body={};const p=prompt('Provenance ID(生产调度必填)','');const r=prompt('Run ID(生产调度必填)','');if(p)body.provenance_id=p;if(r)body.run_id=r}await api(`/api/v2/portal/tasks/${encodeURIComponent(id)}/${action}`,{method:'POST',body:JSON.stringify(body)});await tasks()}catch(e){err(e)}}
|
||||||
|
async function authorizations(){const d=await api('/api/v2/portal/authorizations');state.cache.authorization=d;const workflows=(d.workflows||[]),grants=(d.grants||[]);$('#authorization').innerHTML=`<div class="cards">${card('Workflow',d.summary?.workflows||0)}${card('Grant',d.summary?.grants||0)}${card('有效 Grant',d.summary?.active_grants||0)}${card('已撤销',d.summary?.revoked_grants||0)}</div><div class="panel"><div class="panel-head"><h2>Workflow Provenance</h2></div>${table([['ID',x=>esc(x.provenance_id)],['Run / Task',x=>`${esc(x.run_id)}<br>${esc(x.task_id)}`],['Principal / Agent',x=>`${esc(x.principal_id)}<br>${esc(x.agent_id)}`],['状态',x=>pill(x.status)],['有效期',x=>esc(x.expires_at)],['使用',x=>esc(x.usage_count)],['操作',x=>has('provenance.revoke')&&x.status!=='revoked'?`<button class="btn danger" data-revoke-workflow="${esc(x.provenance_id)}">撤销</button>`:'—']],workflows)}</div><div class="panel"><div class="panel-head"><h2>Child Grants</h2></div>${table([['Grant',x=>esc(x.grant_id)],['类型 / Scope',x=>`${esc(x.grant_type)}<br>${esc(x.execution_path)}`],['Git 操作',x=>esc((x.allowed_git_operations||[]).join(', ')||'—')],['状态',x=>pill(x.status)],['有效期',x=>esc(x.expires_at)],['使用',x=>esc(x.usage_count)],['操作',x=>has('provenance.revoke')&&x.status!=='revoked'?`<button class="btn danger" data-revoke-grant="${esc(x.grant_id)}">撤销</button>`:'—']],grants)}</div>`;document.querySelectorAll('[data-revoke-workflow]').forEach(b=>b.onclick=()=>confirmOp('撤销 Workflow',`撤销 ${b.dataset.revokeWorkflow} 将使所有子 Grant 失效。`,reason=>api(`/api/v2/portal/provenance/${encodeURIComponent(b.dataset.revokeWorkflow)}/revoke`,{method:'POST',body:JSON.stringify({reason})}).then(authorizations)));document.querySelectorAll('[data-revoke-grant]').forEach(b=>b.onclick=()=>confirmOp('撤销 Grant',`撤销 ${b.dataset.revokeGrant}。`,reason=>api(`/api/v2/portal/grants/${encodeURIComponent(b.dataset.revokeGrant)}/revoke`,{method:'POST',body:JSON.stringify({reason})}).then(authorizations)))}
|
||||||
|
async function runtime(){const d=await api('/api/v2/portal/runtime');const runtimes=d.runtimes||[],dispatches=d.dispatches||[],jobs=d.hermes_jobs||[];$('#runtime').innerHTML=`<div class="grid"><div class="panel"><div class="panel-head"><h2>Runtime</h2></div>${table([['名称',x=>esc(x.name)],['状态',x=>pill(x.status)],['详情',x=>esc(JSON.stringify(x.details||{}))]],runtimes)}</div><div class="panel"><div class="panel-head"><h2>Hermes Queue / Lease</h2></div>${table([['Job',x=>esc(x.job_id)],['Cell / Profile',x=>`${esc(x.cell_id)} / ${esc(x.profile_id)}`],['Worker',x=>esc(x.assigned_worker_id||'—')],['状态',x=>pill(x.state)],['Lease',x=>esc(x.lease_expires_at||'—')]],jobs)}</div><div class="panel wide"><div class="panel-head"><h2>Dispatch</h2></div>${table([['Dispatch',x=>esc(x.dispatch_id)],['Task',x=>esc(x.task_id)],['Runtime',x=>esc(x.runtime)],['执行 ID',x=>esc(x.runtime_execution_id)],['Principal',x=>esc(x.principal_id)],['状态',x=>pill(x.state)]],dispatches)}</div></div>`}
|
||||||
|
async function evidence(){const d=await api('/api/v2/portal/evidence');$('#evidence').innerHTML=`<div class="panel"><div class="panel-head"><h2>Execution Evidence</h2></div>${table([['Execution',x=>esc(x.execution_id)],['Task / Agent',x=>`${esc(x.task_id)}<br>${esc(x.agent_id)}`],['Executor',x=>esc(x.executor)],['状态',x=>pill(x.status)],['进度',x=>`${esc(x.progress_completed)}/${esc(x.progress_total)}`],['Test',x=>pill(x.test_status)],['Commit',x=>esc(x.last_commit||'—')],['授权',x=>esc(x.last_authorization_id||'—')]],d.executions||[])}</div>`}
|
||||||
|
async function audit(){const d=await api('/api/v2/portal/audit');const events=(d.events||[]).slice().reverse();$('#audit').innerHTML=`<div class="panel"><div class="panel-head"><h2>统一审计时间线</h2></div><div class="timeline">${events.map(x=>`<div class="event"><strong>${esc(x.action)} · ${pill(x.outcome)}</strong><p>${esc(x.created_at)} · ${esc(x.principal_id)} · ${esc(x.resource_type)}/${esc(x.resource_id)}</p><div class="muted">Trace ${esc(x.trace_id)} · Policy ${esc(x.policy_version)}</div></div>`).join('')||'<div class="empty">暂无审计事件</div>'}</div></div>`}
|
||||||
|
async function release(){const d=await api('/api/v2/portal/release');const g=d.gitops||{};$('#release').innerHTML=`<div class="cards">${card('版本',d.version||'—')}${card('Baseline',d.baseline||'—')}${card('发布状态',d.status||'—')}${card('Git 同步',g.sync_status||'unknown')}</div><div class="grid"><div class="panel"><div class="panel-head"><h2>当前 Release</h2></div><div class="kv"><div>Release OID</div><div>${esc(d.release_oid)}</div><div>Branch</div><div>${esc(g.workspace_branch||g.branch||'—')}</div><div>Workspace</div><div>${pill(g.workspace_clean===true?'clean':g.workspace_clean===false?'dirty':'unknown')}</div><div>Local Sync</div><div>${pill(g.local_sync||'unknown')}</div><div>Gitea Sync</div><div>${pill(g.gitea_sync||'unknown')}</div></div></div><div class="panel"><div class="panel-head"><h2>治理操作</h2></div><p class="muted">门户不直接移动 Tag、合并主线或部署生产。以下仅为可发起的 GitOps 请求目录。</p>${(d.operations||[]).map(x=>pill(x)).join('')||'<span class="muted">当前角色只读</span>'}</div><div class="panel wide"><div class="panel-head"><h2>原始 GitOps 状态</h2></div>${json(g)}</div></div>`}
|
||||||
|
function confirmOp(title,text,fn){state.confirm=fn;$('#confirmTitle').textContent=title;$('#confirmText').textContent=text;$('#confirmReason').value='';$('#confirmDialog').showModal()}
|
||||||
|
$('#taskForm').onsubmit=async e=>{e.preventDefault();clearErr();try{const fd=new FormData(e.target),payload=JSON.parse(fd.get('payload')||'{}');await api('/api/v2/portal/tasks',{method:'POST',body:JSON.stringify({agent_id:fd.get('agent_id'),action:fd.get('action'),idempotency_key:fd.get('idempotency_key'),payload})});$('#taskDialog').close();e.target.reset();await tasks()}catch(e){err(e)}};
|
||||||
|
$('#confirmForm').onsubmit=async e=>{e.preventDefault();const fn=state.confirm;$('#confirmDialog').close();state.confirm=null;try{await fn?.($('#confirmReason').value)}catch(e){err(e)}};
|
||||||
|
document.querySelectorAll('[data-close]').forEach(b=>b.onclick=()=>document.getElementById(b.dataset.close).close());$('#refresh').onclick=()=>load(state.current);
|
||||||
|
identity().catch(err);setInterval(()=>load(state.current),30000);
|
||||||
|
</script>
|
||||||
|
</body></html>'''
|
||||||
@@ -0,0 +1,627 @@
|
|||||||
|
"""Tenant-isolated AIOS V2 control-plane domain service.
|
||||||
|
|
||||||
|
This module deliberately contains no OpenClaw or Hermes implementation details.
|
||||||
|
It owns authoritative tenant, agent, task-lifecycle, idempotency and audit state.
|
||||||
|
Runtime adapters consume approved tasks through a separate boundary.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import copy
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import threading
|
||||||
|
import uuid
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from enum import Enum
|
||||||
|
from typing import Any, Callable, Iterable, Mapping
|
||||||
|
|
||||||
|
|
||||||
|
def _utc_now() -> str:
|
||||||
|
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||||
|
|
||||||
|
|
||||||
|
def _new_id(prefix: str) -> str:
|
||||||
|
return f"{prefix}_{uuid.uuid4().hex}"
|
||||||
|
|
||||||
|
|
||||||
|
def _canonical_digest(value: Mapping[str, Any]) -> str:
|
||||||
|
encoded = json.dumps(
|
||||||
|
value,
|
||||||
|
ensure_ascii=False,
|
||||||
|
sort_keys=True,
|
||||||
|
separators=(",", ":"),
|
||||||
|
default=str,
|
||||||
|
).encode("utf-8")
|
||||||
|
return hashlib.sha256(encoded).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
class ControlPlaneError(RuntimeError):
|
||||||
|
"""Base error returned by the control-plane domain."""
|
||||||
|
|
||||||
|
|
||||||
|
class Unauthorized(ControlPlaneError):
|
||||||
|
"""The caller is not authenticated or lacks the required role."""
|
||||||
|
|
||||||
|
|
||||||
|
class TenantIsolationError(ControlPlaneError):
|
||||||
|
"""A request attempted to cross a tenant boundary."""
|
||||||
|
|
||||||
|
|
||||||
|
class NotFound(ControlPlaneError):
|
||||||
|
"""A tenant-scoped resource does not exist or is not visible."""
|
||||||
|
|
||||||
|
|
||||||
|
class InvalidTransition(ControlPlaneError):
|
||||||
|
"""A task state transition violates the lifecycle contract."""
|
||||||
|
|
||||||
|
|
||||||
|
class IdempotencyConflict(ControlPlaneError):
|
||||||
|
"""An idempotency key was reused for a different command."""
|
||||||
|
|
||||||
|
|
||||||
|
class RevisionConflict(ControlPlaneError):
|
||||||
|
"""Optimistic task revision does not match current state."""
|
||||||
|
|
||||||
|
|
||||||
|
class PolicyDenied(ControlPlaneError):
|
||||||
|
"""The policy gate denied a control-plane operation."""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class RequestContext:
|
||||||
|
"""Authenticated request identity propagated across all AIOS layers."""
|
||||||
|
|
||||||
|
tenant_id: str
|
||||||
|
principal_id: str
|
||||||
|
trace_id: str
|
||||||
|
roles: frozenset[str] = field(default_factory=frozenset)
|
||||||
|
session_id: str | None = None
|
||||||
|
policy_version: str = "v1"
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for name in ("tenant_id", "principal_id", "trace_id"):
|
||||||
|
if not str(getattr(self, name, "")).strip():
|
||||||
|
raise ValueError(f"{name} is required")
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def create(
|
||||||
|
cls,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
principal_id: str,
|
||||||
|
roles: Iterable[str] = (),
|
||||||
|
trace_id: str | None = None,
|
||||||
|
session_id: str | None = None,
|
||||||
|
policy_version: str = "v1",
|
||||||
|
) -> "RequestContext":
|
||||||
|
return cls(
|
||||||
|
tenant_id=tenant_id.strip(),
|
||||||
|
principal_id=principal_id.strip(),
|
||||||
|
trace_id=trace_id or _new_id("trace"),
|
||||||
|
roles=frozenset(str(role).strip() for role in roles if str(role).strip()),
|
||||||
|
session_id=session_id,
|
||||||
|
policy_version=policy_version,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class TenantRecord:
|
||||||
|
tenant_id: str
|
||||||
|
display_name: str
|
||||||
|
status: str = "active"
|
||||||
|
metadata: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
created_at: str = field(default_factory=_utc_now)
|
||||||
|
updated_at: str = field(default_factory=_utc_now)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class AgentRecord:
|
||||||
|
agent_id: str
|
||||||
|
tenant_id: str
|
||||||
|
display_name: str
|
||||||
|
runtime: str
|
||||||
|
capabilities: tuple[str, ...] = field(default_factory=tuple)
|
||||||
|
enabled: bool = True
|
||||||
|
metadata: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
created_at: str = field(default_factory=_utc_now)
|
||||||
|
updated_at: str = field(default_factory=_utc_now)
|
||||||
|
|
||||||
|
|
||||||
|
class TaskState(str, Enum):
|
||||||
|
CREATED = "CREATED"
|
||||||
|
APPROVED = "APPROVED"
|
||||||
|
RUNNING = "RUNNING"
|
||||||
|
COMPLETED = "COMPLETED"
|
||||||
|
FAILED = "FAILED"
|
||||||
|
CANCELLED = "CANCELLED"
|
||||||
|
|
||||||
|
|
||||||
|
TERMINAL_TASK_STATES = frozenset(
|
||||||
|
{TaskState.COMPLETED, TaskState.FAILED, TaskState.CANCELLED}
|
||||||
|
)
|
||||||
|
|
||||||
|
ALLOWED_TASK_TRANSITIONS: Mapping[TaskState, frozenset[TaskState]] = {
|
||||||
|
TaskState.CREATED: frozenset({TaskState.APPROVED, TaskState.CANCELLED}),
|
||||||
|
TaskState.APPROVED: frozenset({TaskState.RUNNING, TaskState.CANCELLED}),
|
||||||
|
TaskState.RUNNING: frozenset(
|
||||||
|
{TaskState.COMPLETED, TaskState.FAILED, TaskState.CANCELLED}
|
||||||
|
),
|
||||||
|
TaskState.COMPLETED: frozenset(),
|
||||||
|
TaskState.FAILED: frozenset(),
|
||||||
|
TaskState.CANCELLED: frozenset(),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class TaskRecord:
|
||||||
|
task_id: str
|
||||||
|
tenant_id: str
|
||||||
|
agent_id: str
|
||||||
|
action: str
|
||||||
|
payload: Mapping[str, Any]
|
||||||
|
state: TaskState = TaskState.CREATED
|
||||||
|
revision: int = 1
|
||||||
|
idempotency_key: str | None = None
|
||||||
|
request_digest: str = ""
|
||||||
|
result: Mapping[str, Any] | None = None
|
||||||
|
error: Mapping[str, Any] | None = None
|
||||||
|
created_by: str = ""
|
||||||
|
approved_by: str | None = None
|
||||||
|
created_at: str = field(default_factory=_utc_now)
|
||||||
|
updated_at: str = field(default_factory=_utc_now)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class AuditEvent:
|
||||||
|
event_id: str
|
||||||
|
tenant_id: str
|
||||||
|
principal_id: str
|
||||||
|
trace_id: str
|
||||||
|
action: str
|
||||||
|
resource_type: str
|
||||||
|
resource_id: str
|
||||||
|
outcome: str
|
||||||
|
policy_version: str
|
||||||
|
details: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
created_at: str = field(default_factory=_utc_now)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PolicyDecision:
|
||||||
|
allowed: bool
|
||||||
|
reason: str
|
||||||
|
policy_version: str
|
||||||
|
|
||||||
|
|
||||||
|
PolicyEvaluator = Callable[[RequestContext, str, Mapping[str, Any]], PolicyDecision]
|
||||||
|
|
||||||
|
|
||||||
|
class InMemoryControlPlaneStore:
|
||||||
|
"""Thread-safe reference store used by tests and the first gateway slice.
|
||||||
|
|
||||||
|
Production backends must preserve the same tenant-scoped interface and add
|
||||||
|
database row-level security, migrations, backup and HA evidence.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.lock = threading.RLock()
|
||||||
|
self.tenants: dict[str, TenantRecord] = {}
|
||||||
|
self.agents: dict[str, AgentRecord] = {}
|
||||||
|
self.tasks: dict[str, TaskRecord] = {}
|
||||||
|
self.idempotency: dict[tuple[str, str], tuple[str, str]] = {}
|
||||||
|
self.audit_events: list[AuditEvent] = []
|
||||||
|
|
||||||
|
|
||||||
|
class ControlPlaneService:
|
||||||
|
"""Authoritative tenant, agent and task lifecycle service."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
store: InMemoryControlPlaneStore | None = None,
|
||||||
|
*,
|
||||||
|
policy_evaluator: PolicyEvaluator | None = None,
|
||||||
|
) -> None:
|
||||||
|
self.store = store or InMemoryControlPlaneStore()
|
||||||
|
self.policy_evaluator = policy_evaluator or self._default_policy
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _default_policy(
|
||||||
|
context: RequestContext,
|
||||||
|
action: str,
|
||||||
|
resource: Mapping[str, Any],
|
||||||
|
) -> PolicyDecision:
|
||||||
|
del resource
|
||||||
|
role_requirements = {
|
||||||
|
"tenant.register": {"platform_admin"},
|
||||||
|
"tenant.read": {"tenant_admin", "operator", "auditor", "platform_admin"},
|
||||||
|
"agent.register": {"tenant_admin", "operator", "platform_admin"},
|
||||||
|
"agent.read": {"tenant_admin", "operator", "auditor", "platform_admin"},
|
||||||
|
"task.create": {"tenant_admin", "operator", "platform_admin"},
|
||||||
|
"task.read": {"tenant_admin", "operator", "auditor", "platform_admin"},
|
||||||
|
"task.approve": {"tenant_admin", "approver", "platform_admin"},
|
||||||
|
"task.start": {"runtime", "operator", "platform_admin"},
|
||||||
|
"task.complete": {"runtime", "operator", "platform_admin"},
|
||||||
|
"task.fail": {"runtime", "operator", "platform_admin"},
|
||||||
|
"task.cancel": {"tenant_admin", "operator", "approver", "platform_admin"},
|
||||||
|
"audit.read": {"tenant_admin", "auditor", "platform_admin"},
|
||||||
|
}
|
||||||
|
required = role_requirements.get(action, {"platform_admin"})
|
||||||
|
allowed = bool(context.roles.intersection(required))
|
||||||
|
return PolicyDecision(
|
||||||
|
allowed=allowed,
|
||||||
|
reason="role_allowed" if allowed else "required_role_missing",
|
||||||
|
policy_version=context.policy_version,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _require_context(self, context: RequestContext) -> None:
|
||||||
|
if not context.principal_id or not context.tenant_id or not context.trace_id:
|
||||||
|
raise Unauthorized("authenticated tenant context is required")
|
||||||
|
|
||||||
|
def _authorize(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
action: str,
|
||||||
|
resource: Mapping[str, Any],
|
||||||
|
) -> PolicyDecision:
|
||||||
|
self._require_context(context)
|
||||||
|
decision = self.policy_evaluator(context, action, resource)
|
||||||
|
if not decision.allowed:
|
||||||
|
self._audit(
|
||||||
|
context,
|
||||||
|
action=action,
|
||||||
|
resource_type=str(resource.get("resource_type", "policy")),
|
||||||
|
resource_id=str(resource.get("resource_id", "unknown")),
|
||||||
|
outcome="denied",
|
||||||
|
details={"reason": decision.reason},
|
||||||
|
)
|
||||||
|
raise PolicyDenied(f"{action} denied: {decision.reason}")
|
||||||
|
return decision
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _assert_same_tenant(context: RequestContext, tenant_id: str) -> None:
|
||||||
|
if context.tenant_id != tenant_id and "platform_admin" not in context.roles:
|
||||||
|
raise TenantIsolationError("cross-tenant access denied")
|
||||||
|
|
||||||
|
def _audit(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
*,
|
||||||
|
action: str,
|
||||||
|
resource_type: str,
|
||||||
|
resource_id: str,
|
||||||
|
outcome: str,
|
||||||
|
details: Mapping[str, Any] | None = None,
|
||||||
|
) -> AuditEvent:
|
||||||
|
event = AuditEvent(
|
||||||
|
event_id=_new_id("audit"),
|
||||||
|
tenant_id=context.tenant_id,
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
trace_id=context.trace_id,
|
||||||
|
action=action,
|
||||||
|
resource_type=resource_type,
|
||||||
|
resource_id=resource_id,
|
||||||
|
outcome=outcome,
|
||||||
|
policy_version=context.policy_version,
|
||||||
|
details=copy.deepcopy(dict(details or {})),
|
||||||
|
)
|
||||||
|
self.store.audit_events.append(event)
|
||||||
|
return event
|
||||||
|
|
||||||
|
def register_tenant(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
display_name: str,
|
||||||
|
metadata: Mapping[str, Any] | None = None,
|
||||||
|
) -> TenantRecord:
|
||||||
|
tenant_id = tenant_id.strip()
|
||||||
|
if not tenant_id or not display_name.strip():
|
||||||
|
raise ValueError("tenant_id and display_name are required")
|
||||||
|
self._authorize(
|
||||||
|
context,
|
||||||
|
"tenant.register",
|
||||||
|
{"resource_type": "tenant", "resource_id": tenant_id},
|
||||||
|
)
|
||||||
|
self._assert_same_tenant(context, tenant_id)
|
||||||
|
with self.store.lock:
|
||||||
|
if tenant_id in self.store.tenants:
|
||||||
|
raise ControlPlaneError(f"tenant already exists: {tenant_id}")
|
||||||
|
tenant = TenantRecord(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
display_name=display_name.strip(),
|
||||||
|
metadata=copy.deepcopy(dict(metadata or {})),
|
||||||
|
)
|
||||||
|
self.store.tenants[tenant_id] = tenant
|
||||||
|
self._audit(
|
||||||
|
context,
|
||||||
|
action="tenant.register",
|
||||||
|
resource_type="tenant",
|
||||||
|
resource_id=tenant_id,
|
||||||
|
outcome="allowed",
|
||||||
|
)
|
||||||
|
return tenant
|
||||||
|
|
||||||
|
def get_tenant(self, context: RequestContext, tenant_id: str | None = None) -> TenantRecord:
|
||||||
|
tenant_id = tenant_id or context.tenant_id
|
||||||
|
self._authorize(
|
||||||
|
context,
|
||||||
|
"tenant.read",
|
||||||
|
{"resource_type": "tenant", "resource_id": tenant_id},
|
||||||
|
)
|
||||||
|
self._assert_same_tenant(context, tenant_id)
|
||||||
|
with self.store.lock:
|
||||||
|
tenant = self.store.tenants.get(tenant_id)
|
||||||
|
if tenant is None:
|
||||||
|
raise NotFound("tenant not found")
|
||||||
|
return tenant
|
||||||
|
|
||||||
|
def register_agent(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
*,
|
||||||
|
agent_id: str,
|
||||||
|
display_name: str,
|
||||||
|
runtime: str,
|
||||||
|
capabilities: Iterable[str] = (),
|
||||||
|
metadata: Mapping[str, Any] | None = None,
|
||||||
|
) -> AgentRecord:
|
||||||
|
self._authorize(
|
||||||
|
context,
|
||||||
|
"agent.register",
|
||||||
|
{"resource_type": "agent", "resource_id": agent_id},
|
||||||
|
)
|
||||||
|
tenant = self.get_tenant(context)
|
||||||
|
if tenant.status != "active":
|
||||||
|
raise ControlPlaneError("tenant is not active")
|
||||||
|
normalized_capabilities = tuple(
|
||||||
|
sorted({str(item).strip() for item in capabilities if str(item).strip()})
|
||||||
|
)
|
||||||
|
if not agent_id.strip() or not display_name.strip() or not runtime.strip():
|
||||||
|
raise ValueError("agent_id, display_name and runtime are required")
|
||||||
|
with self.store.lock:
|
||||||
|
existing = self.store.agents.get(agent_id)
|
||||||
|
if existing is not None and existing.tenant_id != context.tenant_id:
|
||||||
|
raise TenantIsolationError("agent id belongs to another tenant")
|
||||||
|
if existing is not None:
|
||||||
|
raise ControlPlaneError(f"agent already exists: {agent_id}")
|
||||||
|
agent = AgentRecord(
|
||||||
|
agent_id=agent_id.strip(),
|
||||||
|
tenant_id=context.tenant_id,
|
||||||
|
display_name=display_name.strip(),
|
||||||
|
runtime=runtime.strip(),
|
||||||
|
capabilities=normalized_capabilities,
|
||||||
|
metadata=copy.deepcopy(dict(metadata or {})),
|
||||||
|
)
|
||||||
|
self.store.agents[agent.agent_id] = agent
|
||||||
|
self._audit(
|
||||||
|
context,
|
||||||
|
action="agent.register",
|
||||||
|
resource_type="agent",
|
||||||
|
resource_id=agent.agent_id,
|
||||||
|
outcome="allowed",
|
||||||
|
details={"runtime": agent.runtime},
|
||||||
|
)
|
||||||
|
return agent
|
||||||
|
|
||||||
|
def get_agent(self, context: RequestContext, agent_id: str) -> AgentRecord:
|
||||||
|
self._authorize(
|
||||||
|
context,
|
||||||
|
"agent.read",
|
||||||
|
{"resource_type": "agent", "resource_id": agent_id},
|
||||||
|
)
|
||||||
|
with self.store.lock:
|
||||||
|
agent = self.store.agents.get(agent_id)
|
||||||
|
if agent is None:
|
||||||
|
raise NotFound("agent not found")
|
||||||
|
self._assert_same_tenant(context, agent.tenant_id)
|
||||||
|
return agent
|
||||||
|
|
||||||
|
def list_agents(self, context: RequestContext) -> list[AgentRecord]:
|
||||||
|
self._authorize(
|
||||||
|
context,
|
||||||
|
"agent.read",
|
||||||
|
{"resource_type": "agent_collection", "resource_id": context.tenant_id},
|
||||||
|
)
|
||||||
|
with self.store.lock:
|
||||||
|
return [
|
||||||
|
agent
|
||||||
|
for agent in self.store.agents.values()
|
||||||
|
if agent.tenant_id == context.tenant_id
|
||||||
|
]
|
||||||
|
|
||||||
|
def create_task(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
*,
|
||||||
|
agent_id: str,
|
||||||
|
action: str,
|
||||||
|
payload: Mapping[str, Any] | None = None,
|
||||||
|
idempotency_key: str | None = None,
|
||||||
|
) -> TaskRecord:
|
||||||
|
self._authorize(
|
||||||
|
context,
|
||||||
|
"task.create",
|
||||||
|
{"resource_type": "task", "resource_id": idempotency_key or "new"},
|
||||||
|
)
|
||||||
|
agent = self.get_agent(context, agent_id)
|
||||||
|
if not agent.enabled:
|
||||||
|
raise ControlPlaneError("agent is disabled")
|
||||||
|
if action not in agent.capabilities and "*" not in agent.capabilities:
|
||||||
|
raise PolicyDenied(f"agent does not advertise capability: {action}")
|
||||||
|
normalized_payload = copy.deepcopy(dict(payload or {}))
|
||||||
|
request_digest = _canonical_digest(
|
||||||
|
{"agent_id": agent_id, "action": action, "payload": normalized_payload}
|
||||||
|
)
|
||||||
|
normalized_key = idempotency_key.strip() if idempotency_key else None
|
||||||
|
with self.store.lock:
|
||||||
|
if normalized_key:
|
||||||
|
idempotency_ref = (context.tenant_id, normalized_key)
|
||||||
|
existing = self.store.idempotency.get(idempotency_ref)
|
||||||
|
if existing is not None:
|
||||||
|
existing_task_id, existing_digest = existing
|
||||||
|
if existing_digest != request_digest:
|
||||||
|
raise IdempotencyConflict(
|
||||||
|
"idempotency key was reused with a different request"
|
||||||
|
)
|
||||||
|
return self.store.tasks[existing_task_id]
|
||||||
|
task = TaskRecord(
|
||||||
|
task_id=_new_id("task"),
|
||||||
|
tenant_id=context.tenant_id,
|
||||||
|
agent_id=agent_id,
|
||||||
|
action=action,
|
||||||
|
payload=normalized_payload,
|
||||||
|
idempotency_key=normalized_key,
|
||||||
|
request_digest=request_digest,
|
||||||
|
created_by=context.principal_id,
|
||||||
|
)
|
||||||
|
self.store.tasks[task.task_id] = task
|
||||||
|
if normalized_key:
|
||||||
|
self.store.idempotency[(context.tenant_id, normalized_key)] = (
|
||||||
|
task.task_id,
|
||||||
|
request_digest,
|
||||||
|
)
|
||||||
|
self._audit(
|
||||||
|
context,
|
||||||
|
action="task.create",
|
||||||
|
resource_type="task",
|
||||||
|
resource_id=task.task_id,
|
||||||
|
outcome="allowed",
|
||||||
|
details={"agent_id": agent_id, "action": action},
|
||||||
|
)
|
||||||
|
return task
|
||||||
|
|
||||||
|
def get_task(self, context: RequestContext, task_id: str) -> TaskRecord:
|
||||||
|
self._authorize(
|
||||||
|
context,
|
||||||
|
"task.read",
|
||||||
|
{"resource_type": "task", "resource_id": task_id},
|
||||||
|
)
|
||||||
|
with self.store.lock:
|
||||||
|
task = self.store.tasks.get(task_id)
|
||||||
|
if task is None:
|
||||||
|
raise NotFound("task not found")
|
||||||
|
self._assert_same_tenant(context, task.tenant_id)
|
||||||
|
return task
|
||||||
|
|
||||||
|
def list_tasks(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
*,
|
||||||
|
states: Iterable[TaskState | str] | None = None,
|
||||||
|
) -> list[TaskRecord]:
|
||||||
|
self._authorize(
|
||||||
|
context,
|
||||||
|
"task.read",
|
||||||
|
{"resource_type": "task_collection", "resource_id": context.tenant_id},
|
||||||
|
)
|
||||||
|
state_filter = (
|
||||||
|
{TaskState(item) for item in states}
|
||||||
|
if states is not None
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
with self.store.lock:
|
||||||
|
return [
|
||||||
|
task
|
||||||
|
for task in self.store.tasks.values()
|
||||||
|
if task.tenant_id == context.tenant_id
|
||||||
|
and (state_filter is None or task.state in state_filter)
|
||||||
|
]
|
||||||
|
|
||||||
|
def transition_task(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
*,
|
||||||
|
task_id: str,
|
||||||
|
target_state: TaskState | str,
|
||||||
|
expected_revision: int | None = None,
|
||||||
|
result: Mapping[str, Any] | None = None,
|
||||||
|
error: Mapping[str, Any] | None = None,
|
||||||
|
) -> TaskRecord:
|
||||||
|
target = TaskState(target_state)
|
||||||
|
action_name = {
|
||||||
|
TaskState.APPROVED: "task.approve",
|
||||||
|
TaskState.RUNNING: "task.start",
|
||||||
|
TaskState.COMPLETED: "task.complete",
|
||||||
|
TaskState.FAILED: "task.fail",
|
||||||
|
TaskState.CANCELLED: "task.cancel",
|
||||||
|
}.get(target)
|
||||||
|
if action_name is None:
|
||||||
|
raise InvalidTransition(f"cannot transition to {target.value}")
|
||||||
|
self._authorize(
|
||||||
|
context,
|
||||||
|
action_name,
|
||||||
|
{"resource_type": "task", "resource_id": task_id},
|
||||||
|
)
|
||||||
|
with self.store.lock:
|
||||||
|
current = self.store.tasks.get(task_id)
|
||||||
|
if current is None:
|
||||||
|
raise NotFound("task not found")
|
||||||
|
self._assert_same_tenant(context, current.tenant_id)
|
||||||
|
if expected_revision is not None and current.revision != expected_revision:
|
||||||
|
raise RevisionConflict(
|
||||||
|
f"expected revision {expected_revision}, current {current.revision}"
|
||||||
|
)
|
||||||
|
if target not in ALLOWED_TASK_TRANSITIONS[current.state]:
|
||||||
|
raise InvalidTransition(
|
||||||
|
f"invalid transition {current.state.value} -> {target.value}"
|
||||||
|
)
|
||||||
|
if target == TaskState.COMPLETED and result is None:
|
||||||
|
raise ValueError("completed task requires result")
|
||||||
|
if target == TaskState.FAILED and error is None:
|
||||||
|
raise ValueError("failed task requires error")
|
||||||
|
updated = TaskRecord(
|
||||||
|
task_id=current.task_id,
|
||||||
|
tenant_id=current.tenant_id,
|
||||||
|
agent_id=current.agent_id,
|
||||||
|
action=current.action,
|
||||||
|
payload=current.payload,
|
||||||
|
state=target,
|
||||||
|
revision=current.revision + 1,
|
||||||
|
idempotency_key=current.idempotency_key,
|
||||||
|
request_digest=current.request_digest,
|
||||||
|
result=copy.deepcopy(dict(result)) if result is not None else current.result,
|
||||||
|
error=copy.deepcopy(dict(error)) if error is not None else current.error,
|
||||||
|
created_by=current.created_by,
|
||||||
|
approved_by=(
|
||||||
|
context.principal_id
|
||||||
|
if target == TaskState.APPROVED
|
||||||
|
else current.approved_by
|
||||||
|
),
|
||||||
|
created_at=current.created_at,
|
||||||
|
updated_at=_utc_now(),
|
||||||
|
)
|
||||||
|
self.store.tasks[task_id] = updated
|
||||||
|
self._audit(
|
||||||
|
context,
|
||||||
|
action=action_name,
|
||||||
|
resource_type="task",
|
||||||
|
resource_id=task_id,
|
||||||
|
outcome="allowed",
|
||||||
|
details={
|
||||||
|
"from": current.state.value,
|
||||||
|
"to": target.value,
|
||||||
|
"revision": updated.revision,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return updated
|
||||||
|
|
||||||
|
def list_audit_events(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
*,
|
||||||
|
resource_id: str | None = None,
|
||||||
|
) -> list[AuditEvent]:
|
||||||
|
self._authorize(
|
||||||
|
context,
|
||||||
|
"audit.read",
|
||||||
|
{"resource_type": "audit", "resource_id": resource_id or context.tenant_id},
|
||||||
|
)
|
||||||
|
with self.store.lock:
|
||||||
|
return [
|
||||||
|
event
|
||||||
|
for event in self.store.audit_events
|
||||||
|
if event.tenant_id == context.tenant_id
|
||||||
|
and (resource_id is None or event.resource_id == resource_id)
|
||||||
|
]
|
||||||
@@ -0,0 +1,379 @@
|
|||||||
|
"""AIOS V2 持久化 Dispatch 记录。
|
||||||
|
|
||||||
|
Dispatch 是 AIOS Task 与 Runtime command/job 之间的唯一可信绑定。所有结果回写
|
||||||
|
必须匹配 tenant/task/trace/runtime_execution/provenance/run/principal。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import sqlite3
|
||||||
|
import threading
|
||||||
|
import uuid
|
||||||
|
from dataclasses import asdict, dataclass, replace
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from enum import Enum
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Protocol
|
||||||
|
|
||||||
|
|
||||||
|
def utc_now() -> str:
|
||||||
|
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||||
|
|
||||||
|
|
||||||
|
def _required(value: str, label: str) -> str:
|
||||||
|
normalized = str(value or "").strip()
|
||||||
|
if not normalized:
|
||||||
|
raise ValueError(f"{label} is required")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
class DispatchState(str, Enum):
|
||||||
|
PREPARED = "PREPARED"
|
||||||
|
RUNNING = "RUNNING"
|
||||||
|
QUEUED = "QUEUED"
|
||||||
|
COMPLETED = "COMPLETED"
|
||||||
|
FAILED = "FAILED"
|
||||||
|
CANCELLED = "CANCELLED"
|
||||||
|
|
||||||
|
|
||||||
|
TERMINAL_DISPATCH_STATES = frozenset(
|
||||||
|
{DispatchState.COMPLETED, DispatchState.FAILED, DispatchState.CANCELLED}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class DispatchRecord:
|
||||||
|
dispatch_id: str
|
||||||
|
tenant_id: str
|
||||||
|
task_id: str
|
||||||
|
agent_id: str
|
||||||
|
principal_id: str
|
||||||
|
runtime: str
|
||||||
|
runtime_execution_id: str
|
||||||
|
trace_id: str
|
||||||
|
provenance_id: str
|
||||||
|
run_id: str
|
||||||
|
authorization_id: str
|
||||||
|
state: DispatchState = DispatchState.PREPARED
|
||||||
|
expected_worker_id: str | None = None
|
||||||
|
result_digest: str | None = None
|
||||||
|
created_at: str = ""
|
||||||
|
updated_at: str = ""
|
||||||
|
completed_at: str | None = None
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for name in (
|
||||||
|
"dispatch_id",
|
||||||
|
"tenant_id",
|
||||||
|
"task_id",
|
||||||
|
"agent_id",
|
||||||
|
"principal_id",
|
||||||
|
"runtime",
|
||||||
|
"runtime_execution_id",
|
||||||
|
"trace_id",
|
||||||
|
"provenance_id",
|
||||||
|
"run_id",
|
||||||
|
"authorization_id",
|
||||||
|
):
|
||||||
|
object.__setattr__(self, name, _required(getattr(self, name), name))
|
||||||
|
object.__setattr__(self, "state", DispatchState(self.state))
|
||||||
|
now = utc_now()
|
||||||
|
object.__setattr__(self, "created_at", self.created_at or now)
|
||||||
|
object.__setattr__(self, "updated_at", self.updated_at or now)
|
||||||
|
if self.expected_worker_id is not None:
|
||||||
|
object.__setattr__(
|
||||||
|
self,
|
||||||
|
"expected_worker_id",
|
||||||
|
_required(self.expected_worker_id, "expected_worker_id"),
|
||||||
|
)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def create(
|
||||||
|
cls,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
task_id: str,
|
||||||
|
agent_id: str,
|
||||||
|
principal_id: str,
|
||||||
|
runtime: str,
|
||||||
|
runtime_execution_id: str,
|
||||||
|
trace_id: str,
|
||||||
|
provenance_id: str,
|
||||||
|
run_id: str,
|
||||||
|
authorization_id: str,
|
||||||
|
) -> "DispatchRecord":
|
||||||
|
return cls(
|
||||||
|
dispatch_id=f"dispatch_{uuid.uuid4().hex}",
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
task_id=task_id,
|
||||||
|
agent_id=agent_id,
|
||||||
|
principal_id=principal_id,
|
||||||
|
runtime=runtime,
|
||||||
|
runtime_execution_id=runtime_execution_id,
|
||||||
|
trace_id=trace_id,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
authorization_id=authorization_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
value = asdict(self)
|
||||||
|
value["state"] = self.state.value
|
||||||
|
return value
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, value: Mapping[str, Any]) -> "DispatchRecord":
|
||||||
|
return cls(**dict(value))
|
||||||
|
|
||||||
|
|
||||||
|
class DispatchConflict(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class DispatchNotFound(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class DispatchRepository(Protocol):
|
||||||
|
def create(self, record: DispatchRecord) -> DispatchRecord: ...
|
||||||
|
def get(self, dispatch_id: str, *, tenant_id: str) -> DispatchRecord: ...
|
||||||
|
def get_by_task(self, task_id: str, *, tenant_id: str) -> DispatchRecord | None: ...
|
||||||
|
def get_by_runtime_execution(
|
||||||
|
self, runtime_execution_id: str, *, tenant_id: str
|
||||||
|
) -> DispatchRecord | None: ...
|
||||||
|
def update(
|
||||||
|
self,
|
||||||
|
dispatch_id: str,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
state: DispatchState | str | None = None,
|
||||||
|
expected_worker_id: str | None = None,
|
||||||
|
result_digest: str | None = None,
|
||||||
|
) -> DispatchRecord: ...
|
||||||
|
def list(self, *, tenant_id: str) -> list[DispatchRecord]: ...
|
||||||
|
|
||||||
|
|
||||||
|
class InMemoryDispatchRepository:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
self._records: dict[str, DispatchRecord] = {}
|
||||||
|
self._by_task: dict[tuple[str, str], str] = {}
|
||||||
|
self._by_runtime: dict[tuple[str, str], str] = {}
|
||||||
|
|
||||||
|
def create(self, record: DispatchRecord) -> DispatchRecord:
|
||||||
|
with self._lock:
|
||||||
|
task_ref = (record.tenant_id, record.task_id)
|
||||||
|
runtime_ref = (record.tenant_id, record.runtime_execution_id)
|
||||||
|
if task_ref in self._by_task or runtime_ref in self._by_runtime:
|
||||||
|
existing = self.get_by_task(record.task_id, tenant_id=record.tenant_id)
|
||||||
|
if existing == record:
|
||||||
|
return record
|
||||||
|
raise DispatchConflict("task or runtime execution already has a dispatch")
|
||||||
|
self._records[record.dispatch_id] = record
|
||||||
|
self._by_task[task_ref] = record.dispatch_id
|
||||||
|
self._by_runtime[runtime_ref] = record.dispatch_id
|
||||||
|
return record
|
||||||
|
|
||||||
|
def get(self, dispatch_id: str, *, tenant_id: str) -> DispatchRecord:
|
||||||
|
with self._lock:
|
||||||
|
record = self._records.get(dispatch_id)
|
||||||
|
if record is None or record.tenant_id != tenant_id:
|
||||||
|
raise DispatchNotFound("dispatch not found")
|
||||||
|
return record
|
||||||
|
|
||||||
|
def get_by_task(self, task_id: str, *, tenant_id: str) -> DispatchRecord | None:
|
||||||
|
with self._lock:
|
||||||
|
dispatch_id = self._by_task.get((tenant_id, task_id))
|
||||||
|
return self._records.get(dispatch_id) if dispatch_id else None
|
||||||
|
|
||||||
|
def get_by_runtime_execution(
|
||||||
|
self, runtime_execution_id: str, *, tenant_id: str
|
||||||
|
) -> DispatchRecord | None:
|
||||||
|
with self._lock:
|
||||||
|
dispatch_id = self._by_runtime.get((tenant_id, runtime_execution_id))
|
||||||
|
return self._records.get(dispatch_id) if dispatch_id else None
|
||||||
|
|
||||||
|
def update(
|
||||||
|
self,
|
||||||
|
dispatch_id: str,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
state: DispatchState | str | None = None,
|
||||||
|
expected_worker_id: str | None = None,
|
||||||
|
result_digest: str | None = None,
|
||||||
|
) -> DispatchRecord:
|
||||||
|
with self._lock:
|
||||||
|
current = self.get(dispatch_id, tenant_id=tenant_id)
|
||||||
|
target = DispatchState(state) if state is not None else current.state
|
||||||
|
if current.state in TERMINAL_DISPATCH_STATES and target != current.state:
|
||||||
|
raise DispatchConflict("terminal dispatch cannot transition")
|
||||||
|
updated = replace(
|
||||||
|
current,
|
||||||
|
state=target,
|
||||||
|
expected_worker_id=(
|
||||||
|
expected_worker_id
|
||||||
|
if expected_worker_id is not None
|
||||||
|
else current.expected_worker_id
|
||||||
|
),
|
||||||
|
result_digest=result_digest or current.result_digest,
|
||||||
|
updated_at=utc_now(),
|
||||||
|
completed_at=(utc_now() if target in TERMINAL_DISPATCH_STATES else None),
|
||||||
|
)
|
||||||
|
self._records[dispatch_id] = updated
|
||||||
|
return updated
|
||||||
|
|
||||||
|
def list(self, *, tenant_id: str) -> list[DispatchRecord]:
|
||||||
|
with self._lock:
|
||||||
|
return sorted(
|
||||||
|
[item for item in self._records.values() if item.tenant_id == tenant_id],
|
||||||
|
key=lambda item: (item.created_at, item.dispatch_id),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class SQLiteDispatchRepository:
|
||||||
|
SCHEMA_VERSION = 1
|
||||||
|
|
||||||
|
def __init__(self, path: str | Path) -> None:
|
||||||
|
self.path = Path(path).resolve()
|
||||||
|
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
with self._connect() as connection:
|
||||||
|
connection.executescript(
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS dispatch_records (
|
||||||
|
dispatch_id TEXT PRIMARY KEY,
|
||||||
|
tenant_id TEXT NOT NULL,
|
||||||
|
task_id TEXT NOT NULL,
|
||||||
|
runtime_execution_id TEXT NOT NULL,
|
||||||
|
payload_json TEXT NOT NULL,
|
||||||
|
state TEXT NOT NULL,
|
||||||
|
updated_at TEXT NOT NULL,
|
||||||
|
UNIQUE(tenant_id, task_id),
|
||||||
|
UNIQUE(tenant_id, runtime_execution_id)
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_dispatch_tenant_state
|
||||||
|
ON dispatch_records(tenant_id, state, updated_at);
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
|
||||||
|
def _connect(self) -> sqlite3.Connection:
|
||||||
|
connection = sqlite3.connect(str(self.path), timeout=30.0)
|
||||||
|
connection.row_factory = sqlite3.Row
|
||||||
|
connection.execute("PRAGMA journal_mode=WAL")
|
||||||
|
connection.execute("PRAGMA synchronous=FULL")
|
||||||
|
return connection
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _encode(record: DispatchRecord) -> str:
|
||||||
|
return json.dumps(
|
||||||
|
record.as_dict(), ensure_ascii=False, sort_keys=True, separators=(",", ":")
|
||||||
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _decode(row: sqlite3.Row) -> DispatchRecord:
|
||||||
|
value = json.loads(row["payload_json"])
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise RuntimeError("invalid dispatch record")
|
||||||
|
return DispatchRecord.from_dict(value)
|
||||||
|
|
||||||
|
def create(self, record: DispatchRecord) -> DispatchRecord:
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
connection.execute("BEGIN IMMEDIATE")
|
||||||
|
try:
|
||||||
|
connection.execute(
|
||||||
|
"INSERT INTO dispatch_records VALUES (?, ?, ?, ?, ?, ?, ?)",
|
||||||
|
(
|
||||||
|
record.dispatch_id,
|
||||||
|
record.tenant_id,
|
||||||
|
record.task_id,
|
||||||
|
record.runtime_execution_id,
|
||||||
|
self._encode(record),
|
||||||
|
record.state.value,
|
||||||
|
record.updated_at,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
connection.commit()
|
||||||
|
return record
|
||||||
|
except sqlite3.IntegrityError as exc:
|
||||||
|
connection.rollback()
|
||||||
|
existing = self.get_by_task(record.task_id, tenant_id=record.tenant_id)
|
||||||
|
if existing is not None and existing == record:
|
||||||
|
return existing
|
||||||
|
raise DispatchConflict("task or runtime execution already has a dispatch") from exc
|
||||||
|
|
||||||
|
def get(self, dispatch_id: str, *, tenant_id: str) -> DispatchRecord:
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
row = connection.execute(
|
||||||
|
"SELECT payload_json FROM dispatch_records WHERE dispatch_id = ? AND tenant_id = ?",
|
||||||
|
(_required(dispatch_id, "dispatch_id"), _required(tenant_id, "tenant_id")),
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise DispatchNotFound("dispatch not found")
|
||||||
|
return self._decode(row)
|
||||||
|
|
||||||
|
def get_by_task(self, task_id: str, *, tenant_id: str) -> DispatchRecord | None:
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
row = connection.execute(
|
||||||
|
"SELECT payload_json FROM dispatch_records WHERE task_id = ? AND tenant_id = ?",
|
||||||
|
(_required(task_id, "task_id"), _required(tenant_id, "tenant_id")),
|
||||||
|
).fetchone()
|
||||||
|
return self._decode(row) if row is not None else None
|
||||||
|
|
||||||
|
def get_by_runtime_execution(
|
||||||
|
self, runtime_execution_id: str, *, tenant_id: str
|
||||||
|
) -> DispatchRecord | None:
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
row = connection.execute(
|
||||||
|
"SELECT payload_json FROM dispatch_records WHERE runtime_execution_id = ? AND tenant_id = ?",
|
||||||
|
(
|
||||||
|
_required(runtime_execution_id, "runtime_execution_id"),
|
||||||
|
_required(tenant_id, "tenant_id"),
|
||||||
|
),
|
||||||
|
).fetchone()
|
||||||
|
return self._decode(row) if row is not None else None
|
||||||
|
|
||||||
|
def update(
|
||||||
|
self,
|
||||||
|
dispatch_id: str,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
state: DispatchState | str | None = None,
|
||||||
|
expected_worker_id: str | None = None,
|
||||||
|
result_digest: str | None = None,
|
||||||
|
) -> DispatchRecord:
|
||||||
|
current = self.get(dispatch_id, tenant_id=tenant_id)
|
||||||
|
target = DispatchState(state) if state is not None else current.state
|
||||||
|
if current.state in TERMINAL_DISPATCH_STATES and target != current.state:
|
||||||
|
raise DispatchConflict("terminal dispatch cannot transition")
|
||||||
|
updated = replace(
|
||||||
|
current,
|
||||||
|
state=target,
|
||||||
|
expected_worker_id=(
|
||||||
|
expected_worker_id if expected_worker_id is not None else current.expected_worker_id
|
||||||
|
),
|
||||||
|
result_digest=result_digest or current.result_digest,
|
||||||
|
updated_at=utc_now(),
|
||||||
|
completed_at=(utc_now() if target in TERMINAL_DISPATCH_STATES else None),
|
||||||
|
)
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
connection.execute("BEGIN IMMEDIATE")
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE dispatch_records SET payload_json = ?, state = ?, updated_at = ? WHERE dispatch_id = ? AND tenant_id = ?",
|
||||||
|
(
|
||||||
|
self._encode(updated),
|
||||||
|
updated.state.value,
|
||||||
|
updated.updated_at,
|
||||||
|
updated.dispatch_id,
|
||||||
|
updated.tenant_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
connection.commit()
|
||||||
|
return updated
|
||||||
|
|
||||||
|
def list(self, *, tenant_id: str) -> list[DispatchRecord]:
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
rows = connection.execute(
|
||||||
|
"SELECT payload_json FROM dispatch_records WHERE tenant_id = ? ORDER BY updated_at, dispatch_id",
|
||||||
|
(_required(tenant_id, "tenant_id"),),
|
||||||
|
).fetchall()
|
||||||
|
return [self._decode(row) for row in rows]
|
||||||
@@ -0,0 +1,543 @@
|
|||||||
|
"""Portable multi-provider execution plans and aggregate release gates.
|
||||||
|
|
||||||
|
A plan is a DAG of executor-neutral stages. Only root stages are declared at
|
||||||
|
plan creation. A dependent stage is declared atomically after every dependency
|
||||||
|
passes its Truth + notification gate, so its start deadline cannot expire while
|
||||||
|
it is legitimately waiting for upstream work.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import sqlite3
|
||||||
|
import threading
|
||||||
|
from dataclasses import asdict, dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Sequence
|
||||||
|
|
||||||
|
from .executor_protocol import ExecutorProviderRegistry, normalize_executor
|
||||||
|
from .external_task_control import ExternalTaskCoordinator
|
||||||
|
from .external_task_query import list_records
|
||||||
|
from .task_execution_registry import (
|
||||||
|
ExecutionTaskManifest,
|
||||||
|
ExecutionTruthRecord,
|
||||||
|
ExecutionTruthState,
|
||||||
|
required,
|
||||||
|
utc_now,
|
||||||
|
)
|
||||||
|
|
||||||
|
PLAN_SCHEMA_VERSION = "aios.execution-plan.v2"
|
||||||
|
WAITING_DEPENDENCY = "WAITING_DEPENDENCY"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ExecutionStageManifest:
|
||||||
|
stage_id: str
|
||||||
|
task_id: str
|
||||||
|
executor_type: str
|
||||||
|
role: str
|
||||||
|
steps: Sequence[str]
|
||||||
|
depends_on: Sequence[str] = ()
|
||||||
|
required_evidence: Sequence[str] = ()
|
||||||
|
notification_events: Sequence[str] = ("STARTED", "SUCCEEDED", "FAILED")
|
||||||
|
owner_notification_ref: str = "AIOS_OWNER_WECHAT_TARGET_REF"
|
||||||
|
start_deadline_seconds: int = 300
|
||||||
|
heartbeat_timeout_seconds: int = 180
|
||||||
|
required: bool = True
|
||||||
|
metadata: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for name in ("stage_id", "task_id", "role", "owner_notification_ref"):
|
||||||
|
object.__setattr__(self, name, required(getattr(self, name), name))
|
||||||
|
object.__setattr__(self, "executor_type", normalize_executor(self.executor_type))
|
||||||
|
steps = tuple(required(item, "step") for item in self.steps)
|
||||||
|
if not steps:
|
||||||
|
raise ValueError("execution stage requires at least one step")
|
||||||
|
object.__setattr__(self, "steps", steps)
|
||||||
|
object.__setattr__(
|
||||||
|
self,
|
||||||
|
"depends_on",
|
||||||
|
tuple(sorted({required(item, "depends_on") for item in self.depends_on})),
|
||||||
|
)
|
||||||
|
object.__setattr__(
|
||||||
|
self,
|
||||||
|
"required_evidence",
|
||||||
|
tuple(sorted({required(item, "required_evidence") for item in self.required_evidence})),
|
||||||
|
)
|
||||||
|
object.__setattr__(
|
||||||
|
self,
|
||||||
|
"notification_events",
|
||||||
|
tuple(sorted({required(item, "notification_event") for item in self.notification_events})),
|
||||||
|
)
|
||||||
|
if self.start_deadline_seconds < 1 or self.heartbeat_timeout_seconds < 1:
|
||||||
|
raise ValueError("stage deadlines must be positive")
|
||||||
|
object.__setattr__(
|
||||||
|
self,
|
||||||
|
"metadata",
|
||||||
|
json.loads(json.dumps(dict(self.metadata), ensure_ascii=False, default=str)),
|
||||||
|
)
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
value = asdict(self)
|
||||||
|
value["steps"] = list(self.steps)
|
||||||
|
value["depends_on"] = list(self.depends_on)
|
||||||
|
value["required_evidence"] = list(self.required_evidence)
|
||||||
|
value["notification_events"] = list(self.notification_events)
|
||||||
|
return value
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, value: Mapping[str, Any]) -> "ExecutionStageManifest":
|
||||||
|
return cls(**dict(value))
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ExecutionPlanManifest:
|
||||||
|
plan_id: str
|
||||||
|
source_branch: str
|
||||||
|
source_oid: str
|
||||||
|
target_version: str
|
||||||
|
stages: Sequence[ExecutionStageManifest]
|
||||||
|
release_gate_policy: str = "all_required_stages"
|
||||||
|
schema_version: str = PLAN_SCHEMA_VERSION
|
||||||
|
metadata: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for name in ("plan_id", "source_branch", "source_oid", "target_version"):
|
||||||
|
object.__setattr__(self, name, required(getattr(self, name), name))
|
||||||
|
if self.schema_version != PLAN_SCHEMA_VERSION:
|
||||||
|
raise ValueError("unsupported execution plan schema")
|
||||||
|
if self.release_gate_policy != "all_required_stages":
|
||||||
|
raise ValueError("unsupported release gate policy")
|
||||||
|
stages = tuple(self.stages)
|
||||||
|
if not stages:
|
||||||
|
raise ValueError("execution plan requires stages")
|
||||||
|
stage_ids = [item.stage_id for item in stages]
|
||||||
|
task_ids = [item.task_id for item in stages]
|
||||||
|
if len(stage_ids) != len(set(stage_ids)):
|
||||||
|
raise ValueError("execution plan stage_id values must be unique")
|
||||||
|
if len(task_ids) != len(set(task_ids)):
|
||||||
|
raise ValueError("execution plan task_id values must be unique")
|
||||||
|
known = set(stage_ids)
|
||||||
|
for stage in stages:
|
||||||
|
unknown = sorted(set(stage.depends_on) - known)
|
||||||
|
if unknown:
|
||||||
|
raise ValueError(f"stage {stage.stage_id} has unknown dependencies: {unknown}")
|
||||||
|
if stage.stage_id in stage.depends_on:
|
||||||
|
raise ValueError("stage cannot depend on itself")
|
||||||
|
self._assert_acyclic(stages)
|
||||||
|
object.__setattr__(self, "stages", stages)
|
||||||
|
object.__setattr__(
|
||||||
|
self,
|
||||||
|
"metadata",
|
||||||
|
json.loads(json.dumps(dict(self.metadata), ensure_ascii=False, default=str)),
|
||||||
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _assert_acyclic(stages: Sequence[ExecutionStageManifest]) -> None:
|
||||||
|
dependencies = {item.stage_id: set(item.depends_on) for item in stages}
|
||||||
|
resolved: set[str] = set()
|
||||||
|
while len(resolved) < len(stages):
|
||||||
|
ready = {
|
||||||
|
stage_id
|
||||||
|
for stage_id, deps in dependencies.items()
|
||||||
|
if stage_id not in resolved and deps.issubset(resolved)
|
||||||
|
}
|
||||||
|
if not ready:
|
||||||
|
raise ValueError("execution plan contains a dependency cycle")
|
||||||
|
resolved.update(ready)
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"schema_version": self.schema_version,
|
||||||
|
"plan_id": self.plan_id,
|
||||||
|
"source_branch": self.source_branch,
|
||||||
|
"source_oid": self.source_oid,
|
||||||
|
"target_version": self.target_version,
|
||||||
|
"release_gate_policy": self.release_gate_policy,
|
||||||
|
"stages": [item.as_dict() for item in self.stages],
|
||||||
|
"metadata": dict(self.metadata),
|
||||||
|
}
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, value: Mapping[str, Any]) -> "ExecutionPlanManifest":
|
||||||
|
raw = dict(value)
|
||||||
|
raw["stages"] = tuple(
|
||||||
|
ExecutionStageManifest.from_dict(item) for item in raw.get("stages") or ()
|
||||||
|
)
|
||||||
|
return cls(**raw)
|
||||||
|
|
||||||
|
def stage(self, stage_id: str) -> ExecutionStageManifest:
|
||||||
|
normalized = required(stage_id, "stage_id")
|
||||||
|
for stage in self.stages:
|
||||||
|
if stage.stage_id == normalized:
|
||||||
|
return stage
|
||||||
|
raise KeyError(f"execution stage not found: {normalized}")
|
||||||
|
|
||||||
|
|
||||||
|
class SQLiteExecutionPlanStore:
|
||||||
|
"""Durable plan declarations; stage state remains in Truth Registry."""
|
||||||
|
|
||||||
|
def __init__(self, path: str | Path) -> None:
|
||||||
|
self.path = Path(path).resolve()
|
||||||
|
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
with self._connect() as connection:
|
||||||
|
connection.executescript(
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS execution_plans (
|
||||||
|
plan_id TEXT PRIMARY KEY,
|
||||||
|
manifest_json TEXT NOT NULL,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
updated_at TEXT NOT NULL
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
|
||||||
|
def _connect(self) -> sqlite3.Connection:
|
||||||
|
connection = sqlite3.connect(str(self.path), timeout=30.0)
|
||||||
|
connection.row_factory = sqlite3.Row
|
||||||
|
connection.execute("PRAGMA journal_mode=WAL")
|
||||||
|
connection.execute("PRAGMA synchronous=FULL")
|
||||||
|
return connection
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _encode(plan: ExecutionPlanManifest) -> str:
|
||||||
|
return json.dumps(
|
||||||
|
plan.as_dict(),
|
||||||
|
ensure_ascii=False,
|
||||||
|
sort_keys=True,
|
||||||
|
separators=(",", ":"),
|
||||||
|
)
|
||||||
|
|
||||||
|
def save(
|
||||||
|
self,
|
||||||
|
plan: ExecutionPlanManifest,
|
||||||
|
*,
|
||||||
|
allow_source_oid_supersession: bool = False,
|
||||||
|
) -> None:
|
||||||
|
payload = self._encode(plan)
|
||||||
|
now = utc_now()
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
connection.execute("BEGIN IMMEDIATE")
|
||||||
|
existing = connection.execute(
|
||||||
|
"SELECT manifest_json FROM execution_plans WHERE plan_id = ?",
|
||||||
|
(plan.plan_id,),
|
||||||
|
).fetchone()
|
||||||
|
if existing is not None:
|
||||||
|
if existing["manifest_json"] != payload:
|
||||||
|
previous = json.loads(existing["manifest_json"])
|
||||||
|
replacement = plan.as_dict()
|
||||||
|
previous_oid = previous.pop("source_oid", None)
|
||||||
|
replacement_oid = replacement.pop("source_oid", None)
|
||||||
|
if (
|
||||||
|
not allow_source_oid_supersession
|
||||||
|
or previous != replacement
|
||||||
|
or previous_oid == replacement_oid
|
||||||
|
):
|
||||||
|
connection.rollback()
|
||||||
|
raise RuntimeError(
|
||||||
|
"plan_id already exists with another manifest"
|
||||||
|
)
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE execution_plans SET manifest_json = ?, updated_at = ? WHERE plan_id = ?",
|
||||||
|
(payload, now, plan.plan_id),
|
||||||
|
)
|
||||||
|
connection.commit()
|
||||||
|
return
|
||||||
|
connection.rollback()
|
||||||
|
return
|
||||||
|
connection.execute(
|
||||||
|
"INSERT INTO execution_plans VALUES (?, ?, ?, ?)",
|
||||||
|
(plan.plan_id, payload, now, now),
|
||||||
|
)
|
||||||
|
connection.commit()
|
||||||
|
|
||||||
|
def get(self, plan_id: str) -> ExecutionPlanManifest:
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
row = connection.execute(
|
||||||
|
"SELECT manifest_json FROM execution_plans WHERE plan_id = ?",
|
||||||
|
(required(plan_id, "plan_id"),),
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise KeyError("execution plan not found")
|
||||||
|
value = json.loads(row["manifest_json"])
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise ValueError("stored execution plan is invalid")
|
||||||
|
return ExecutionPlanManifest.from_dict(value)
|
||||||
|
|
||||||
|
def list(self) -> list[ExecutionPlanManifest]:
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
rows = connection.execute(
|
||||||
|
"SELECT manifest_json FROM execution_plans ORDER BY created_at, plan_id"
|
||||||
|
).fetchall()
|
||||||
|
return [
|
||||||
|
ExecutionPlanManifest.from_dict(json.loads(row["manifest_json"]))
|
||||||
|
for row in rows
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class ExecutionPlanCoordinator:
|
||||||
|
"""Declares dependency-ready stages and evaluates aggregate release gates."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
task_coordinator: ExternalTaskCoordinator,
|
||||||
|
plan_store: SQLiteExecutionPlanStore,
|
||||||
|
providers: ExecutorProviderRegistry,
|
||||||
|
) -> None:
|
||||||
|
self.task_coordinator = task_coordinator
|
||||||
|
self.plan_store = plan_store
|
||||||
|
self.providers = providers
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_file(
|
||||||
|
cls,
|
||||||
|
path: str | Path,
|
||||||
|
*,
|
||||||
|
providers: ExecutorProviderRegistry,
|
||||||
|
) -> "ExecutionPlanCoordinator":
|
||||||
|
return cls(
|
||||||
|
task_coordinator=ExternalTaskCoordinator.from_file(path),
|
||||||
|
plan_store=SQLiteExecutionPlanStore(path),
|
||||||
|
providers=providers,
|
||||||
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _task_manifest(
|
||||||
|
plan: ExecutionPlanManifest,
|
||||||
|
stage: ExecutionStageManifest,
|
||||||
|
) -> ExecutionTaskManifest:
|
||||||
|
return ExecutionTaskManifest(
|
||||||
|
task_id=stage.task_id,
|
||||||
|
executor=stage.executor_type,
|
||||||
|
source_branch=plan.source_branch,
|
||||||
|
source_oid=plan.source_oid,
|
||||||
|
target_version=plan.target_version,
|
||||||
|
steps=stage.steps,
|
||||||
|
start_deadline_seconds=stage.start_deadline_seconds,
|
||||||
|
heartbeat_timeout_seconds=stage.heartbeat_timeout_seconds,
|
||||||
|
required_evidence=stage.required_evidence,
|
||||||
|
notification_events=stage.notification_events,
|
||||||
|
owner_notification_ref=stage.owner_notification_ref,
|
||||||
|
manifest_version="aios.external-task.v2",
|
||||||
|
)
|
||||||
|
|
||||||
|
def _record(self, task_id: str) -> ExecutionTruthRecord | None:
|
||||||
|
try:
|
||||||
|
return self.task_coordinator.registry.get(task_id)
|
||||||
|
except KeyError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _declare_stage(
|
||||||
|
self,
|
||||||
|
plan: ExecutionPlanManifest,
|
||||||
|
stage: ExecutionStageManifest,
|
||||||
|
) -> ExecutionTruthRecord:
|
||||||
|
self.providers.validate_manifest(self._task_manifest(plan, stage))
|
||||||
|
return self.task_coordinator.declare(self._task_manifest(plan, stage))
|
||||||
|
|
||||||
|
def declare(self, plan: ExecutionPlanManifest) -> Mapping[str, Any]:
|
||||||
|
for stage in plan.stages:
|
||||||
|
self.providers.get(stage.executor_type)
|
||||||
|
allow_source_oid_supersession = False
|
||||||
|
try:
|
||||||
|
previous = self.plan_store.get(plan.plan_id)
|
||||||
|
except KeyError:
|
||||||
|
previous = None
|
||||||
|
if previous is not None and previous.source_oid != plan.source_oid:
|
||||||
|
previous_value = previous.as_dict()
|
||||||
|
replacement_value = plan.as_dict()
|
||||||
|
previous_value.pop("source_oid")
|
||||||
|
replacement_value.pop("source_oid")
|
||||||
|
root_records = [
|
||||||
|
self._record(stage.task_id)
|
||||||
|
for stage in previous.stages
|
||||||
|
if not stage.depends_on
|
||||||
|
]
|
||||||
|
allow_source_oid_supersession = bool(
|
||||||
|
previous_value == replacement_value
|
||||||
|
and root_records
|
||||||
|
and all(
|
||||||
|
record is not None
|
||||||
|
and record.state == ExecutionTruthState.NOT_STARTED
|
||||||
|
for record in root_records
|
||||||
|
)
|
||||||
|
)
|
||||||
|
self.plan_store.save(
|
||||||
|
plan,
|
||||||
|
allow_source_oid_supersession=allow_source_oid_supersession,
|
||||||
|
)
|
||||||
|
for stage in plan.stages:
|
||||||
|
if not stage.depends_on:
|
||||||
|
self._declare_stage(plan, stage)
|
||||||
|
return self.snapshot(plan.plan_id)
|
||||||
|
|
||||||
|
def declare_file(self, path: str | Path) -> Mapping[str, Any]:
|
||||||
|
value = json.loads(Path(path).read_text(encoding="utf-8"))
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise ValueError("execution plan file must contain a JSON object")
|
||||||
|
return self.declare(ExecutionPlanManifest.from_dict(value))
|
||||||
|
|
||||||
|
def dependency_gate(self, plan_id: str, stage_id: str) -> Mapping[str, Any]:
|
||||||
|
plan = self.plan_store.get(plan_id)
|
||||||
|
stage = plan.stage(stage_id)
|
||||||
|
dependencies: list[Mapping[str, Any]] = []
|
||||||
|
passed = True
|
||||||
|
for dependency_id in stage.depends_on:
|
||||||
|
dependency = plan.stage(dependency_id)
|
||||||
|
record = self._record(dependency.task_id)
|
||||||
|
if record is None:
|
||||||
|
gate: Mapping[str, Any] = {
|
||||||
|
"task_id": dependency.task_id,
|
||||||
|
"state": WAITING_DEPENDENCY,
|
||||||
|
"passed": False,
|
||||||
|
"release_blocking": True,
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
gate = self.task_coordinator.gate(dependency.task_id)
|
||||||
|
dependencies.append(
|
||||||
|
{
|
||||||
|
"stage_id": dependency.stage_id,
|
||||||
|
"task_id": dependency.task_id,
|
||||||
|
"gate": gate,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
if not gate["passed"]:
|
||||||
|
passed = False
|
||||||
|
return {
|
||||||
|
"plan_id": plan.plan_id,
|
||||||
|
"stage_id": stage.stage_id,
|
||||||
|
"passed": passed,
|
||||||
|
"release_blocking": not passed,
|
||||||
|
"dependencies": dependencies,
|
||||||
|
}
|
||||||
|
|
||||||
|
def activate_ready(self, plan_id: str) -> list[Mapping[str, Any]]:
|
||||||
|
plan = self.plan_store.get(plan_id)
|
||||||
|
activated: list[Mapping[str, Any]] = []
|
||||||
|
for stage in plan.stages:
|
||||||
|
if self._record(stage.task_id) is not None:
|
||||||
|
continue
|
||||||
|
dependency = self.dependency_gate(plan.plan_id, stage.stage_id)
|
||||||
|
if not dependency["passed"]:
|
||||||
|
continue
|
||||||
|
record = self._declare_stage(plan, stage)
|
||||||
|
activated.append(
|
||||||
|
{
|
||||||
|
"stage_id": stage.stage_id,
|
||||||
|
"task_id": record.task_id,
|
||||||
|
"executor_type": stage.executor_type,
|
||||||
|
"state": record.state.value,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return activated
|
||||||
|
|
||||||
|
def discover_ready(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
plan_id: str,
|
||||||
|
executor_type: str,
|
||||||
|
) -> list[Mapping[str, Any]]:
|
||||||
|
plan = self.plan_store.get(plan_id)
|
||||||
|
self.activate_ready(plan.plan_id)
|
||||||
|
normalized = normalize_executor(executor_type)
|
||||||
|
records = {
|
||||||
|
item.task_id: item
|
||||||
|
for item in list_records(
|
||||||
|
self.task_coordinator.registry.path,
|
||||||
|
executor=normalized,
|
||||||
|
states=(
|
||||||
|
ExecutionTruthState.DECLARED.value,
|
||||||
|
ExecutionTruthState.REGISTERED.value,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
ready: list[Mapping[str, Any]] = []
|
||||||
|
for stage in plan.stages:
|
||||||
|
if stage.executor_type != normalized or stage.task_id not in records:
|
||||||
|
continue
|
||||||
|
dependency = self.dependency_gate(plan.plan_id, stage.stage_id)
|
||||||
|
if dependency["passed"]:
|
||||||
|
ready.append(
|
||||||
|
{
|
||||||
|
"plan_id": plan.plan_id,
|
||||||
|
"stage": stage.as_dict(),
|
||||||
|
"record": records[stage.task_id].as_dict(),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return ready
|
||||||
|
|
||||||
|
def snapshot(self, plan_id: str) -> Mapping[str, Any]:
|
||||||
|
plan = self.plan_store.get(plan_id)
|
||||||
|
stages: list[Mapping[str, Any]] = []
|
||||||
|
for stage in plan.stages:
|
||||||
|
record = self._record(stage.task_id)
|
||||||
|
dependency = self.dependency_gate(plan.plan_id, stage.stage_id)
|
||||||
|
if record is None:
|
||||||
|
truth: Mapping[str, Any] = {
|
||||||
|
"task_id": stage.task_id,
|
||||||
|
"state": WAITING_DEPENDENCY,
|
||||||
|
"passed": False,
|
||||||
|
"release_blocking": True,
|
||||||
|
"executor_run_id": None,
|
||||||
|
"executor_id": None,
|
||||||
|
"current_step": 0,
|
||||||
|
"total_steps": len(stage.steps),
|
||||||
|
"evidence": {},
|
||||||
|
"error": {},
|
||||||
|
"notifications": [],
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
truth = self.task_coordinator.status(stage.task_id)
|
||||||
|
stages.append(
|
||||||
|
{
|
||||||
|
"stage": stage.as_dict(),
|
||||||
|
"dependency_gate": dependency,
|
||||||
|
"truth": truth,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"manifest": plan.as_dict(),
|
||||||
|
"stages": stages,
|
||||||
|
}
|
||||||
|
|
||||||
|
def gate(self, plan_id: str) -> Mapping[str, Any]:
|
||||||
|
plan = self.plan_store.get(plan_id)
|
||||||
|
self.activate_ready(plan.plan_id)
|
||||||
|
stages: list[Mapping[str, Any]] = []
|
||||||
|
required_passed = True
|
||||||
|
for stage in plan.stages:
|
||||||
|
dependency = self.dependency_gate(plan.plan_id, stage.stage_id)
|
||||||
|
record = self._record(stage.task_id)
|
||||||
|
if record is None:
|
||||||
|
truth_gate: Mapping[str, Any] = {
|
||||||
|
"task_id": stage.task_id,
|
||||||
|
"state": WAITING_DEPENDENCY,
|
||||||
|
"passed": False,
|
||||||
|
"release_blocking": True,
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
truth_gate = self.task_coordinator.gate(stage.task_id)
|
||||||
|
stage_passed = bool(truth_gate["passed"] and dependency["passed"])
|
||||||
|
if stage.required and not stage_passed:
|
||||||
|
required_passed = False
|
||||||
|
stages.append(
|
||||||
|
{
|
||||||
|
"stage_id": stage.stage_id,
|
||||||
|
"task_id": stage.task_id,
|
||||||
|
"executor_type": stage.executor_type,
|
||||||
|
"role": stage.role,
|
||||||
|
"required": stage.required,
|
||||||
|
"passed": stage_passed,
|
||||||
|
"truth_gate": truth_gate,
|
||||||
|
"dependency_gate": dependency,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"plan_id": plan.plan_id,
|
||||||
|
"target_version": plan.target_version,
|
||||||
|
"policy": plan.release_gate_policy,
|
||||||
|
"passed": required_passed,
|
||||||
|
"release_blocking": not required_passed,
|
||||||
|
"stages": stages,
|
||||||
|
}
|
||||||
@@ -0,0 +1,323 @@
|
|||||||
|
"""AIOS V2 Agent 执行证据。
|
||||||
|
|
||||||
|
Evidence 按租户隔离并使用原子 JSON 写入。生产模式下,claim、heartbeat
|
||||||
|
和 complete 在落盘前必须通过最小权限 Evidence Grant 授权。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import threading
|
||||||
|
from dataclasses import asdict, dataclass, field, replace
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Protocol
|
||||||
|
|
||||||
|
from .provenance import ProvenanceDenied, digest_payload
|
||||||
|
|
||||||
|
|
||||||
|
class EvidenceAuthorizationWriter(Protocol):
|
||||||
|
def authorize_evidence_write(self, **values: Any) -> Any: ...
|
||||||
|
|
||||||
|
|
||||||
|
def utc_now() -> str:
|
||||||
|
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||||
|
|
||||||
|
|
||||||
|
def _required(value: str, label: str) -> str:
|
||||||
|
normalized = str(value or "").strip()
|
||||||
|
if not normalized:
|
||||||
|
raise ValueError(f"{label} is required")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
def _safe(value: Mapping[str, Any] | None) -> dict[str, Any]:
|
||||||
|
return json.loads(json.dumps(dict(value or {}), ensure_ascii=False, default=str))
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ExecutionEvidence:
|
||||||
|
execution_id: str
|
||||||
|
tenant_id: str
|
||||||
|
task_id: str
|
||||||
|
agent_id: str
|
||||||
|
executor: str
|
||||||
|
branch: str
|
||||||
|
status: str = "claimed"
|
||||||
|
progress_completed: int = 0
|
||||||
|
progress_total: int = 1
|
||||||
|
current_step: str = "claim"
|
||||||
|
last_commit: str | None = None
|
||||||
|
test_status: str = "not_run"
|
||||||
|
test_summary: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
result_summary: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
claimed_at: str = field(default_factory=utc_now)
|
||||||
|
heartbeat_at: str = field(default_factory=utc_now)
|
||||||
|
completed_at: str | None = None
|
||||||
|
provenance_id: str | None = None
|
||||||
|
run_id: str | None = None
|
||||||
|
last_authorization_id: str | None = None
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for name in (
|
||||||
|
"execution_id",
|
||||||
|
"tenant_id",
|
||||||
|
"task_id",
|
||||||
|
"agent_id",
|
||||||
|
"executor",
|
||||||
|
"branch",
|
||||||
|
"status",
|
||||||
|
"current_step",
|
||||||
|
"claimed_at",
|
||||||
|
"heartbeat_at",
|
||||||
|
):
|
||||||
|
object.__setattr__(self, name, _required(getattr(self, name), name))
|
||||||
|
if self.progress_completed < 0 or self.progress_total < 1:
|
||||||
|
raise ValueError("invalid progress values")
|
||||||
|
if self.progress_completed > self.progress_total:
|
||||||
|
raise ValueError("progress_completed cannot exceed progress_total")
|
||||||
|
object.__setattr__(self, "test_summary", _safe(self.test_summary))
|
||||||
|
object.__setattr__(self, "result_summary", _safe(self.result_summary))
|
||||||
|
for optional in ("provenance_id", "run_id", "last_authorization_id"):
|
||||||
|
value = getattr(self, optional)
|
||||||
|
object.__setattr__(self, optional, str(value).strip() if value else None)
|
||||||
|
|
||||||
|
|
||||||
|
class ExecutionTracker:
|
||||||
|
"""每个租户、每次执行持久保存一份证据记录。"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
root: str | Path,
|
||||||
|
*,
|
||||||
|
provenance_writer: EvidenceAuthorizationWriter | None = None,
|
||||||
|
require_provenance: bool = False,
|
||||||
|
) -> None:
|
||||||
|
self.root = Path(root).resolve()
|
||||||
|
self.root.mkdir(parents=True, exist_ok=True)
|
||||||
|
self.provenance_writer = provenance_writer
|
||||||
|
self.require_provenance = bool(require_provenance)
|
||||||
|
if self.require_provenance and self.provenance_writer is None:
|
||||||
|
raise ValueError("production evidence tracking requires a provenance writer")
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _component(value: str) -> str:
|
||||||
|
normalized = _required(value, "path component")
|
||||||
|
if normalized in {".", ".."} or "/" in normalized or "\\" in normalized:
|
||||||
|
raise ValueError("invalid evidence path component")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
def _path(self, tenant_id: str, execution_id: str) -> Path:
|
||||||
|
tenant = self._component(tenant_id)
|
||||||
|
execution = self._component(execution_id)
|
||||||
|
return self.root / tenant / f"{execution}.json"
|
||||||
|
|
||||||
|
def _authorize(
|
||||||
|
self,
|
||||||
|
evidence: ExecutionEvidence,
|
||||||
|
*,
|
||||||
|
operation: str,
|
||||||
|
principal_id: str | None,
|
||||||
|
provenance_id: str | None,
|
||||||
|
run_id: str | None,
|
||||||
|
changes: Mapping[str, Any],
|
||||||
|
) -> str | None:
|
||||||
|
if not self.require_provenance:
|
||||||
|
return None
|
||||||
|
if self.provenance_writer is None:
|
||||||
|
raise RuntimeError("Run Provenance writer is not configured")
|
||||||
|
effective_provenance = provenance_id or evidence.provenance_id
|
||||||
|
effective_run = run_id or evidence.run_id
|
||||||
|
if not effective_provenance or not effective_run or not principal_id:
|
||||||
|
raise RuntimeError(
|
||||||
|
"production evidence write requires provenance_id, run_id and principal_id"
|
||||||
|
)
|
||||||
|
request_digest = digest_payload(
|
||||||
|
{
|
||||||
|
"operation": operation,
|
||||||
|
"execution_id": evidence.execution_id,
|
||||||
|
"tenant_id": evidence.tenant_id,
|
||||||
|
"task_id": evidence.task_id,
|
||||||
|
"agent_id": evidence.agent_id,
|
||||||
|
"executor": evidence.executor,
|
||||||
|
"branch": evidence.branch,
|
||||||
|
"changes": dict(changes),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
authorization = self.provenance_writer.authorize_evidence_write(
|
||||||
|
provenance_id=effective_provenance,
|
||||||
|
run_id=effective_run,
|
||||||
|
tenant_id=evidence.tenant_id,
|
||||||
|
task_id=evidence.task_id,
|
||||||
|
agent_id=evidence.agent_id,
|
||||||
|
principal_id=principal_id,
|
||||||
|
request_digest=request_digest,
|
||||||
|
)
|
||||||
|
except ProvenanceDenied as exc:
|
||||||
|
raise RuntimeError(f"Run Provenance denied evidence write: {exc}") from exc
|
||||||
|
return authorization.authorization_id
|
||||||
|
|
||||||
|
def claim(
|
||||||
|
self,
|
||||||
|
evidence: ExecutionEvidence,
|
||||||
|
*,
|
||||||
|
principal_id: str | None = None,
|
||||||
|
provenance_id: str | None = None,
|
||||||
|
run_id: str | None = None,
|
||||||
|
) -> ExecutionEvidence:
|
||||||
|
path = self._path(evidence.tenant_id, evidence.execution_id)
|
||||||
|
with self._lock:
|
||||||
|
if path.exists():
|
||||||
|
current = self._read(path)
|
||||||
|
if current.task_id != evidence.task_id or current.agent_id != evidence.agent_id:
|
||||||
|
raise RuntimeError("execution_id already belongs to another task")
|
||||||
|
if self.require_provenance and (
|
||||||
|
current.provenance_id != (provenance_id or evidence.provenance_id)
|
||||||
|
or current.run_id != (run_id or evidence.run_id)
|
||||||
|
):
|
||||||
|
raise RuntimeError(
|
||||||
|
"execution_id already belongs to another provenance context"
|
||||||
|
)
|
||||||
|
return current
|
||||||
|
effective_provenance = provenance_id or evidence.provenance_id
|
||||||
|
effective_run = run_id or evidence.run_id
|
||||||
|
authorization_id = self._authorize(
|
||||||
|
evidence,
|
||||||
|
operation="claim",
|
||||||
|
principal_id=principal_id,
|
||||||
|
provenance_id=effective_provenance,
|
||||||
|
run_id=effective_run,
|
||||||
|
changes={"status": evidence.status, "current_step": evidence.current_step},
|
||||||
|
)
|
||||||
|
persisted = replace(
|
||||||
|
evidence,
|
||||||
|
provenance_id=effective_provenance,
|
||||||
|
run_id=effective_run,
|
||||||
|
last_authorization_id=authorization_id,
|
||||||
|
)
|
||||||
|
self._write(path, persisted)
|
||||||
|
return persisted
|
||||||
|
|
||||||
|
def heartbeat(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
execution_id: str,
|
||||||
|
status: str | None = None,
|
||||||
|
current_step: str | None = None,
|
||||||
|
progress_completed: int | None = None,
|
||||||
|
progress_total: int | None = None,
|
||||||
|
last_commit: str | None = None,
|
||||||
|
principal_id: str | None = None,
|
||||||
|
provenance_id: str | None = None,
|
||||||
|
run_id: str | None = None,
|
||||||
|
) -> ExecutionEvidence:
|
||||||
|
path = self._path(tenant_id, execution_id)
|
||||||
|
with self._lock:
|
||||||
|
current = self._read(path)
|
||||||
|
changes = {
|
||||||
|
"status": status or current.status,
|
||||||
|
"current_step": current_step or current.current_step,
|
||||||
|
"progress_completed": (
|
||||||
|
progress_completed
|
||||||
|
if progress_completed is not None
|
||||||
|
else current.progress_completed
|
||||||
|
),
|
||||||
|
"progress_total": (
|
||||||
|
progress_total if progress_total is not None else current.progress_total
|
||||||
|
),
|
||||||
|
"last_commit": last_commit or current.last_commit,
|
||||||
|
}
|
||||||
|
authorization_id = self._authorize(
|
||||||
|
current,
|
||||||
|
operation="heartbeat",
|
||||||
|
principal_id=principal_id,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
changes=changes,
|
||||||
|
)
|
||||||
|
updated = replace(
|
||||||
|
current,
|
||||||
|
**changes,
|
||||||
|
last_authorization_id=authorization_id or current.last_authorization_id,
|
||||||
|
heartbeat_at=utc_now(),
|
||||||
|
)
|
||||||
|
self._write(path, updated)
|
||||||
|
return updated
|
||||||
|
|
||||||
|
def complete(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
tenant_id: str,
|
||||||
|
execution_id: str,
|
||||||
|
status: str,
|
||||||
|
test_status: str,
|
||||||
|
test_summary: Mapping[str, Any] | None = None,
|
||||||
|
result_summary: Mapping[str, Any] | None = None,
|
||||||
|
last_commit: str | None = None,
|
||||||
|
principal_id: str | None = None,
|
||||||
|
provenance_id: str | None = None,
|
||||||
|
run_id: str | None = None,
|
||||||
|
) -> ExecutionEvidence:
|
||||||
|
path = self._path(tenant_id, execution_id)
|
||||||
|
with self._lock:
|
||||||
|
current = self._read(path)
|
||||||
|
changes = {
|
||||||
|
"status": _required(status, "status"),
|
||||||
|
"current_step": "complete",
|
||||||
|
"progress_completed": current.progress_total,
|
||||||
|
"last_commit": last_commit or current.last_commit,
|
||||||
|
"test_status": _required(test_status, "test_status"),
|
||||||
|
"test_summary": _safe(test_summary),
|
||||||
|
"result_summary": _safe(result_summary),
|
||||||
|
}
|
||||||
|
authorization_id = self._authorize(
|
||||||
|
current,
|
||||||
|
operation="complete",
|
||||||
|
principal_id=principal_id,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
changes=changes,
|
||||||
|
)
|
||||||
|
updated = replace(
|
||||||
|
current,
|
||||||
|
**changes,
|
||||||
|
last_authorization_id=authorization_id or current.last_authorization_id,
|
||||||
|
heartbeat_at=utc_now(),
|
||||||
|
completed_at=utc_now(),
|
||||||
|
)
|
||||||
|
self._write(path, updated)
|
||||||
|
return updated
|
||||||
|
|
||||||
|
def get(self, *, tenant_id: str, execution_id: str) -> ExecutionEvidence:
|
||||||
|
return self._read(self._path(tenant_id, execution_id))
|
||||||
|
|
||||||
|
def list(self, *, tenant_id: str) -> list[ExecutionEvidence]:
|
||||||
|
tenant_dir = self.root / self._component(tenant_id)
|
||||||
|
if not tenant_dir.exists():
|
||||||
|
return []
|
||||||
|
records = [self._read(path) for path in tenant_dir.glob("*.json")]
|
||||||
|
return sorted(records, key=lambda item: (item.heartbeat_at, item.execution_id), reverse=True)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _read(path: Path) -> ExecutionEvidence:
|
||||||
|
try:
|
||||||
|
value = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise KeyError("execution evidence not found") from exc
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise RuntimeError("invalid execution evidence file")
|
||||||
|
return ExecutionEvidence(**dict(value))
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _write(path: Path, evidence: ExecutionEvidence) -> None:
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
temporary = path.with_suffix(f".tmp-{os.getpid()}")
|
||||||
|
temporary.write_text(
|
||||||
|
json.dumps(asdict(evidence), ensure_ascii=False, sort_keys=True, indent=2) + "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
os.chmod(temporary, 0o640)
|
||||||
|
os.replace(temporary, path)
|
||||||
@@ -0,0 +1,204 @@
|
|||||||
|
"""Persistent and portable AIOS executor provider catalog.
|
||||||
|
|
||||||
|
The catalog contains descriptors only: provider names, protocol/adapter versions,
|
||||||
|
trust roles, capabilities and configuration schema identifiers. Credentials,
|
||||||
|
endpoints and runtime secrets are intentionally excluded and must be resolved in
|
||||||
|
the destination environment.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import threading
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Sequence
|
||||||
|
|
||||||
|
from .executor_protocol import (
|
||||||
|
EXECUTOR_PROTOCOL_VERSION,
|
||||||
|
DescriptorOnlyProvider,
|
||||||
|
ExecutorProvider,
|
||||||
|
ExecutorProviderDescriptor,
|
||||||
|
ExecutorProviderRegistry,
|
||||||
|
default_provider_registry,
|
||||||
|
)
|
||||||
|
|
||||||
|
CATALOG_SCHEMA_VERSION = "aios.executor.catalog.v1"
|
||||||
|
MIGRATION_SCHEMA_VERSION = "aios.executor.migration.v1"
|
||||||
|
|
||||||
|
|
||||||
|
def canonical_json_bytes(value: Mapping[str, Any]) -> bytes:
|
||||||
|
return json.dumps(
|
||||||
|
dict(value),
|
||||||
|
ensure_ascii=False,
|
||||||
|
sort_keys=True,
|
||||||
|
separators=(",", ":"),
|
||||||
|
default=str,
|
||||||
|
).encode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def sha256_payload(value: Mapping[str, Any]) -> str:
|
||||||
|
return hashlib.sha256(canonical_json_bytes(value)).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ExecutorCatalogSnapshot:
|
||||||
|
catalog: Mapping[str, Any]
|
||||||
|
catalog_sha256: str
|
||||||
|
path: str
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"catalog": dict(self.catalog),
|
||||||
|
"catalog_sha256": self.catalog_sha256,
|
||||||
|
"path": self.path,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class FileExecutorCatalogStore:
|
||||||
|
"""Atomic file store for provider descriptors and migration packages."""
|
||||||
|
|
||||||
|
def __init__(self, path: str | Path) -> None:
|
||||||
|
self.path = Path(path).resolve()
|
||||||
|
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _validate_catalog(value: Mapping[str, Any]) -> ExecutorProviderRegistry:
|
||||||
|
if value.get("schema_version") != CATALOG_SCHEMA_VERSION:
|
||||||
|
raise ValueError("unsupported executor catalog schema")
|
||||||
|
if value.get("protocol_version") != EXECUTOR_PROTOCOL_VERSION:
|
||||||
|
raise ValueError("executor catalog protocol mismatch")
|
||||||
|
providers = value.get("providers")
|
||||||
|
if not isinstance(providers, Sequence) or isinstance(providers, (str, bytes)):
|
||||||
|
raise ValueError("executor catalog providers must be an array")
|
||||||
|
return ExecutorProviderRegistry.import_catalog(value)
|
||||||
|
|
||||||
|
def initialize_defaults(self) -> ExecutorCatalogSnapshot:
|
||||||
|
with self._lock:
|
||||||
|
if self.path.exists():
|
||||||
|
return self.load()
|
||||||
|
return self.save(default_provider_registry())
|
||||||
|
|
||||||
|
def load(self) -> ExecutorCatalogSnapshot:
|
||||||
|
with self._lock:
|
||||||
|
value = json.loads(self.path.read_text(encoding="utf-8"))
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise ValueError("executor catalog must contain a JSON object")
|
||||||
|
registry = self._validate_catalog(value)
|
||||||
|
normalized = registry.export_catalog()
|
||||||
|
return ExecutorCatalogSnapshot(
|
||||||
|
catalog=normalized,
|
||||||
|
catalog_sha256=sha256_payload(normalized),
|
||||||
|
path=str(self.path),
|
||||||
|
)
|
||||||
|
|
||||||
|
def registry(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
provider_factories: Mapping[str, ExecutorProvider] | None = None,
|
||||||
|
) -> ExecutorProviderRegistry:
|
||||||
|
snapshot = self.load()
|
||||||
|
return ExecutorProviderRegistry.import_catalog(
|
||||||
|
snapshot.catalog,
|
||||||
|
provider_factories=provider_factories,
|
||||||
|
)
|
||||||
|
|
||||||
|
def save(self, registry: ExecutorProviderRegistry) -> ExecutorCatalogSnapshot:
|
||||||
|
catalog = registry.export_catalog()
|
||||||
|
self._validate_catalog(catalog)
|
||||||
|
encoded = json.dumps(
|
||||||
|
catalog,
|
||||||
|
ensure_ascii=False,
|
||||||
|
sort_keys=True,
|
||||||
|
indent=2,
|
||||||
|
) + "\n"
|
||||||
|
with self._lock:
|
||||||
|
temporary = self.path.with_suffix(f".tmp-{os.getpid()}")
|
||||||
|
temporary.write_text(encoded, encoding="utf-8")
|
||||||
|
os.chmod(temporary, 0o640)
|
||||||
|
os.replace(temporary, self.path)
|
||||||
|
return ExecutorCatalogSnapshot(
|
||||||
|
catalog=catalog,
|
||||||
|
catalog_sha256=sha256_payload(catalog),
|
||||||
|
path=str(self.path),
|
||||||
|
)
|
||||||
|
|
||||||
|
def upsert_descriptor(
|
||||||
|
self,
|
||||||
|
descriptor: ExecutorProviderDescriptor,
|
||||||
|
) -> ExecutorCatalogSnapshot:
|
||||||
|
with self._lock:
|
||||||
|
if self.path.exists():
|
||||||
|
registry = self.registry()
|
||||||
|
else:
|
||||||
|
registry = default_provider_registry()
|
||||||
|
try:
|
||||||
|
registry.register(DescriptorOnlyProvider(descriptor), replace=True)
|
||||||
|
except KeyError:
|
||||||
|
registry.register(DescriptorOnlyProvider(descriptor))
|
||||||
|
return self.save(registry)
|
||||||
|
|
||||||
|
def export_migration_package(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
source_environment: str,
|
||||||
|
required_executor_types: Sequence[str] = (),
|
||||||
|
) -> Mapping[str, Any]:
|
||||||
|
snapshot = self.load()
|
||||||
|
registry = ExecutorProviderRegistry.import_catalog(snapshot.catalog)
|
||||||
|
required = tuple(sorted({str(item).strip().lower() for item in required_executor_types if str(item).strip()}))
|
||||||
|
if required:
|
||||||
|
available = {item.executor_type for item in registry.list_descriptors()}
|
||||||
|
missing = sorted(set(required) - available)
|
||||||
|
if missing:
|
||||||
|
raise ValueError(f"migration package is missing providers: {missing}")
|
||||||
|
package = {
|
||||||
|
"schema_version": MIGRATION_SCHEMA_VERSION,
|
||||||
|
"source_environment": str(source_environment or "").strip(),
|
||||||
|
"catalog": snapshot.catalog,
|
||||||
|
"catalog_sha256": snapshot.catalog_sha256,
|
||||||
|
"required_executor_types": list(required),
|
||||||
|
"secret_values_included": False,
|
||||||
|
}
|
||||||
|
if not package["source_environment"]:
|
||||||
|
raise ValueError("source_environment is required")
|
||||||
|
return {
|
||||||
|
**package,
|
||||||
|
"package_sha256": sha256_payload(package),
|
||||||
|
}
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def import_migration_package(
|
||||||
|
cls,
|
||||||
|
value: Mapping[str, Any],
|
||||||
|
*,
|
||||||
|
destination_path: str | Path,
|
||||||
|
destination_environment: str,
|
||||||
|
provider_factories: Mapping[str, ExecutorProvider] | None = None,
|
||||||
|
) -> ExecutorCatalogSnapshot:
|
||||||
|
if value.get("schema_version") != MIGRATION_SCHEMA_VERSION:
|
||||||
|
raise ValueError("unsupported executor migration schema")
|
||||||
|
if value.get("secret_values_included") is not False:
|
||||||
|
raise ValueError("executor migration package must not contain secrets")
|
||||||
|
package_without_hash = {key: item for key, item in value.items() if key != "package_sha256"}
|
||||||
|
if sha256_payload(package_without_hash) != value.get("package_sha256"):
|
||||||
|
raise ValueError("executor migration package hash mismatch")
|
||||||
|
catalog = value.get("catalog")
|
||||||
|
if not isinstance(catalog, Mapping):
|
||||||
|
raise ValueError("executor migration package has no catalog")
|
||||||
|
if sha256_payload(catalog) != value.get("catalog_sha256"):
|
||||||
|
raise ValueError("executor catalog hash mismatch")
|
||||||
|
registry = ExecutorProviderRegistry.import_catalog(
|
||||||
|
catalog,
|
||||||
|
provider_factories=provider_factories,
|
||||||
|
)
|
||||||
|
required = tuple(value.get("required_executor_types") or ())
|
||||||
|
available = {item.executor_type for item in registry.list_descriptors()}
|
||||||
|
missing = sorted(set(required) - available)
|
||||||
|
if missing:
|
||||||
|
raise ValueError(f"destination provider factories are missing: {missing}")
|
||||||
|
if not str(destination_environment or "").strip():
|
||||||
|
raise ValueError("destination_environment is required")
|
||||||
|
return cls(destination_path).save(registry)
|
||||||
@@ -0,0 +1,441 @@
|
|||||||
|
"""Versioned, extensible executor protocol for AIOS external tasks.
|
||||||
|
|
||||||
|
The core lifecycle is executor-neutral. OpenClaw, Hermes, Codex and the scoped
|
||||||
|
ChatGPT supervisor are built-in
|
||||||
|
providers, not hard-coded protocol branches. Future providers register a
|
||||||
|
versioned descriptor and use the same Truth Registry, evidence, notification
|
||||||
|
and release-gate mechanisms without modifying the lifecycle state machine.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import threading
|
||||||
|
from dataclasses import asdict, dataclass, field
|
||||||
|
from typing import Any, Mapping, Protocol, Sequence
|
||||||
|
|
||||||
|
from .external_task_control import ExternalTaskCoordinator
|
||||||
|
from .task_execution_registry import ExecutionTaskManifest, ExecutionTruthRecord
|
||||||
|
|
||||||
|
EXECUTOR_PROTOCOL_VERSION = "aios.executor.v1"
|
||||||
|
_PROVIDER_NAME = re.compile(r"^[a-z][a-z0-9._-]{1,63}$")
|
||||||
|
_VERSION = re.compile(r"^[0-9]+(?:\.[0-9]+){0,2}(?:[-+][A-Za-z0-9._-]+)?$")
|
||||||
|
DISCOVERABLE_STATES: Sequence[str] = ("DECLARED", "REGISTERED")
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_executor(value: str) -> str:
|
||||||
|
normalized = str(value or "").strip().lower()
|
||||||
|
if not _PROVIDER_NAME.fullmatch(normalized):
|
||||||
|
raise ValueError("executor type must be a lowercase provider identifier")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
def _major(version: str) -> int:
|
||||||
|
normalized = str(version or "").strip()
|
||||||
|
if not _VERSION.fullmatch(normalized):
|
||||||
|
raise ValueError(f"invalid version: {version}")
|
||||||
|
return int(normalized.split(".", 1)[0])
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ExecutorProviderDescriptor:
|
||||||
|
"""Portable provider metadata stored without credentials or runtime secrets."""
|
||||||
|
|
||||||
|
executor_type: str
|
||||||
|
adapter_version: str
|
||||||
|
protocol_version: str = EXECUTOR_PROTOCOL_VERSION
|
||||||
|
trust_roles: Sequence[str] = ()
|
||||||
|
capabilities: Sequence[str] = ()
|
||||||
|
configuration_schema: str = "aios.executor.config.v1"
|
||||||
|
portable: bool = True
|
||||||
|
metadata: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
object.__setattr__(self, "executor_type", normalize_executor(self.executor_type))
|
||||||
|
adapter_version = str(self.adapter_version or "").strip()
|
||||||
|
if not _VERSION.fullmatch(adapter_version):
|
||||||
|
raise ValueError("adapter_version must be version-like")
|
||||||
|
object.__setattr__(self, "adapter_version", adapter_version)
|
||||||
|
protocol_version = str(self.protocol_version or "").strip()
|
||||||
|
if not protocol_version.startswith("aios.executor.v"):
|
||||||
|
raise ValueError("unsupported executor protocol namespace")
|
||||||
|
int(protocol_version.rsplit("v", 1)[1])
|
||||||
|
object.__setattr__(self, "protocol_version", protocol_version)
|
||||||
|
roles = tuple(sorted({str(item).strip() for item in self.trust_roles if str(item).strip()}))
|
||||||
|
if not roles:
|
||||||
|
raise ValueError("provider must declare at least one trust role")
|
||||||
|
capabilities = tuple(
|
||||||
|
sorted({str(item).strip() for item in self.capabilities if str(item).strip()})
|
||||||
|
)
|
||||||
|
object.__setattr__(self, "trust_roles", roles)
|
||||||
|
object.__setattr__(self, "capabilities", capabilities)
|
||||||
|
object.__setattr__(
|
||||||
|
self,
|
||||||
|
"configuration_schema",
|
||||||
|
str(self.configuration_schema or "").strip(),
|
||||||
|
)
|
||||||
|
object.__setattr__(
|
||||||
|
self,
|
||||||
|
"metadata",
|
||||||
|
json.loads(json.dumps(dict(self.metadata), ensure_ascii=False, default=str)),
|
||||||
|
)
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
value = asdict(self)
|
||||||
|
value["trust_roles"] = list(self.trust_roles)
|
||||||
|
value["capabilities"] = list(self.capabilities)
|
||||||
|
return value
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, value: Mapping[str, Any]) -> "ExecutorProviderDescriptor":
|
||||||
|
return cls(**dict(value))
|
||||||
|
|
||||||
|
def protocol_compatible_with(self, other: "ExecutorProviderDescriptor") -> bool:
|
||||||
|
return (
|
||||||
|
self.executor_type == other.executor_type
|
||||||
|
and self.protocol_version == other.protocol_version
|
||||||
|
and _major(self.adapter_version) == _major(other.adapter_version)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ExternalExecutorAdapter(Protocol):
|
||||||
|
"""Runtime-specific implementation behind the common lifecycle."""
|
||||||
|
|
||||||
|
descriptor: ExecutorProviderDescriptor
|
||||||
|
|
||||||
|
def discover(self) -> Sequence[ExecutionTruthRecord]: ...
|
||||||
|
def execute(self, manifest: ExecutionTaskManifest) -> Mapping[str, Any]: ...
|
||||||
|
def notify_owner(self, payload: Mapping[str, Any]) -> str: ...
|
||||||
|
|
||||||
|
|
||||||
|
class ExecutorProvider(Protocol):
|
||||||
|
descriptor: ExecutorProviderDescriptor
|
||||||
|
|
||||||
|
def validate_manifest(self, manifest: ExecutionTaskManifest) -> None: ...
|
||||||
|
def build_adapter(self, configuration: Mapping[str, Any]) -> ExternalExecutorAdapter: ...
|
||||||
|
|
||||||
|
|
||||||
|
class DescriptorOnlyProvider:
|
||||||
|
"""Default provider used when lifecycle integration is supplied externally."""
|
||||||
|
|
||||||
|
def __init__(self, descriptor: ExecutorProviderDescriptor) -> None:
|
||||||
|
self.descriptor = descriptor
|
||||||
|
|
||||||
|
def validate_manifest(self, manifest: ExecutionTaskManifest) -> None:
|
||||||
|
if normalize_executor(manifest.executor) != self.descriptor.executor_type:
|
||||||
|
raise ValueError("manifest executor does not match provider")
|
||||||
|
|
||||||
|
def build_adapter(self, configuration: Mapping[str, Any]) -> ExternalExecutorAdapter:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"provider {self.descriptor.executor_type} requires an external adapter factory"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ExecutorProviderRegistry:
|
||||||
|
"""Thread-safe registry supporting runtime registration and catalog migration."""
|
||||||
|
|
||||||
|
def __init__(self, providers: Sequence[ExecutorProvider] = ()) -> None:
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
self._providers: dict[str, ExecutorProvider] = {}
|
||||||
|
for provider in providers:
|
||||||
|
self.register(provider)
|
||||||
|
|
||||||
|
def register(self, provider: ExecutorProvider, *, replace: bool = False) -> None:
|
||||||
|
descriptor = provider.descriptor
|
||||||
|
key = normalize_executor(descriptor.executor_type)
|
||||||
|
with self._lock:
|
||||||
|
current = self._providers.get(key)
|
||||||
|
if current is not None and not replace:
|
||||||
|
raise ValueError(f"executor provider already registered: {key}")
|
||||||
|
if current is not None and not descriptor.protocol_compatible_with(
|
||||||
|
current.descriptor
|
||||||
|
):
|
||||||
|
raise ValueError(
|
||||||
|
f"replacement provider is incompatible with registered provider: {key}"
|
||||||
|
)
|
||||||
|
self._providers[key] = provider
|
||||||
|
|
||||||
|
def unregister(self, executor_type: str) -> ExecutorProvider:
|
||||||
|
key = normalize_executor(executor_type)
|
||||||
|
with self._lock:
|
||||||
|
try:
|
||||||
|
return self._providers.pop(key)
|
||||||
|
except KeyError as exc:
|
||||||
|
raise KeyError(f"executor provider is not registered: {key}") from exc
|
||||||
|
|
||||||
|
def get(self, executor_type: str) -> ExecutorProvider:
|
||||||
|
key = normalize_executor(executor_type)
|
||||||
|
with self._lock:
|
||||||
|
provider = self._providers.get(key)
|
||||||
|
if provider is None:
|
||||||
|
raise KeyError(f"executor provider is not registered: {key}")
|
||||||
|
return provider
|
||||||
|
|
||||||
|
def descriptor(self, executor_type: str) -> ExecutorProviderDescriptor:
|
||||||
|
return self.get(executor_type).descriptor
|
||||||
|
|
||||||
|
def validate_manifest(self, manifest: ExecutionTaskManifest) -> None:
|
||||||
|
self.get(manifest.executor).validate_manifest(manifest)
|
||||||
|
|
||||||
|
def trusted(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
executor_type: str,
|
||||||
|
roles: Sequence[str],
|
||||||
|
platform_admin: bool = False,
|
||||||
|
) -> bool:
|
||||||
|
if platform_admin or "platform_admin" in set(roles):
|
||||||
|
return True
|
||||||
|
descriptor = self.descriptor(executor_type)
|
||||||
|
return bool(set(descriptor.trust_roles).intersection(set(roles)))
|
||||||
|
|
||||||
|
def list_descriptors(self) -> tuple[ExecutorProviderDescriptor, ...]:
|
||||||
|
with self._lock:
|
||||||
|
return tuple(
|
||||||
|
self._providers[key].descriptor for key in sorted(self._providers)
|
||||||
|
)
|
||||||
|
|
||||||
|
def export_catalog(self) -> Mapping[str, Any]:
|
||||||
|
return {
|
||||||
|
"schema_version": "aios.executor.catalog.v1",
|
||||||
|
"protocol_version": EXECUTOR_PROTOCOL_VERSION,
|
||||||
|
"providers": [item.as_dict() for item in self.list_descriptors()],
|
||||||
|
}
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def import_catalog(
|
||||||
|
cls,
|
||||||
|
value: Mapping[str, Any],
|
||||||
|
*,
|
||||||
|
provider_factories: Mapping[str, ExecutorProvider] | None = None,
|
||||||
|
) -> "ExecutorProviderRegistry":
|
||||||
|
if value.get("schema_version") != "aios.executor.catalog.v1":
|
||||||
|
raise ValueError("unsupported executor catalog schema")
|
||||||
|
if value.get("protocol_version") != EXECUTOR_PROTOCOL_VERSION:
|
||||||
|
raise ValueError("executor catalog protocol mismatch")
|
||||||
|
factories = dict(provider_factories or {})
|
||||||
|
providers: list[ExecutorProvider] = []
|
||||||
|
for raw in value.get("providers") or ():
|
||||||
|
descriptor = ExecutorProviderDescriptor.from_dict(raw)
|
||||||
|
provider = factories.get(descriptor.executor_type)
|
||||||
|
if provider is not None:
|
||||||
|
if not provider.descriptor.protocol_compatible_with(descriptor):
|
||||||
|
raise ValueError(
|
||||||
|
f"provider factory does not match imported descriptor: {descriptor.executor_type}"
|
||||||
|
)
|
||||||
|
providers.append(provider)
|
||||||
|
else:
|
||||||
|
providers.append(DescriptorOnlyProvider(descriptor))
|
||||||
|
return cls(providers)
|
||||||
|
|
||||||
|
|
||||||
|
BUILTIN_PROVIDER_DESCRIPTORS: Sequence[ExecutorProviderDescriptor] = (
|
||||||
|
ExecutorProviderDescriptor(
|
||||||
|
executor_type="openclaw",
|
||||||
|
adapter_version="2.1.0",
|
||||||
|
trust_roles=("runtime", "openclaw_executor"),
|
||||||
|
capabilities=(
|
||||||
|
"deploy",
|
||||||
|
"http_runtime",
|
||||||
|
"owner_notification",
|
||||||
|
"healthcheck",
|
||||||
|
"rollback",
|
||||||
|
),
|
||||||
|
configuration_schema="aios.executor.openclaw.config.v1",
|
||||||
|
metadata={"official": True},
|
||||||
|
),
|
||||||
|
ExecutorProviderDescriptor(
|
||||||
|
executor_type="hermes",
|
||||||
|
adapter_version="2.1.0",
|
||||||
|
trust_roles=("runtime", "hermes_executor"),
|
||||||
|
capabilities=(
|
||||||
|
"queue",
|
||||||
|
"lease",
|
||||||
|
"worker",
|
||||||
|
"runtime_validation",
|
||||||
|
"restart_recovery",
|
||||||
|
),
|
||||||
|
configuration_schema="aios.executor.hermes.config.v1",
|
||||||
|
metadata={"official": True},
|
||||||
|
),
|
||||||
|
ExecutorProviderDescriptor(
|
||||||
|
executor_type="codex",
|
||||||
|
adapter_version="2.1.0",
|
||||||
|
trust_roles=("codex", "codex_executor"),
|
||||||
|
capabilities=("code", "test", "docs", "gitops_patch", "security_review"),
|
||||||
|
configuration_schema="aios.executor.codex.config.v1",
|
||||||
|
metadata={"official": True},
|
||||||
|
),
|
||||||
|
ExecutorProviderDescriptor(
|
||||||
|
executor_type="chatgpt-supervisor",
|
||||||
|
adapter_version="2.1.0",
|
||||||
|
trust_roles=("chatgpt_supervisor",),
|
||||||
|
capabilities=(
|
||||||
|
"acceptance",
|
||||||
|
"deploy",
|
||||||
|
"evidence",
|
||||||
|
"healthcheck",
|
||||||
|
"rollback",
|
||||||
|
"wechat_simulation",
|
||||||
|
),
|
||||||
|
configuration_schema="aios.executor.chatgpt-supervisor.config.v1",
|
||||||
|
metadata={"official": True, "production_mutation_allowed": False},
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def default_provider_registry() -> ExecutorProviderRegistry:
|
||||||
|
return ExecutorProviderRegistry(
|
||||||
|
[DescriptorOnlyProvider(item) for item in BUILTIN_PROVIDER_DESCRIPTORS]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def trusted_for_executor(
|
||||||
|
*,
|
||||||
|
executor: str,
|
||||||
|
roles: Sequence[str],
|
||||||
|
platform_admin: bool = False,
|
||||||
|
registry: ExecutorProviderRegistry | None = None,
|
||||||
|
) -> bool:
|
||||||
|
return (registry or default_provider_registry()).trusted(
|
||||||
|
executor_type=executor,
|
||||||
|
roles=roles,
|
||||||
|
platform_admin=platform_admin,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ExecutorIdentity:
|
||||||
|
executor_type: str
|
||||||
|
executor_id: str
|
||||||
|
executor_run_id: str
|
||||||
|
protocol_version: str = EXECUTOR_PROTOCOL_VERSION
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
object.__setattr__(self, "executor_type", normalize_executor(self.executor_type))
|
||||||
|
for name in ("executor_id", "executor_run_id"):
|
||||||
|
value = str(getattr(self, name) or "").strip()
|
||||||
|
if not value:
|
||||||
|
raise ValueError(f"{name} is required")
|
||||||
|
object.__setattr__(self, name, value)
|
||||||
|
if self.protocol_version != EXECUTOR_PROTOCOL_VERSION:
|
||||||
|
raise ValueError("executor identity protocol mismatch")
|
||||||
|
|
||||||
|
|
||||||
|
class ExecutorLifecycleSession:
|
||||||
|
"""Executor-neutral lifecycle facade used by every registered provider."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
coordinator: ExternalTaskCoordinator,
|
||||||
|
identity: ExecutorIdentity,
|
||||||
|
*,
|
||||||
|
registry: ExecutorProviderRegistry | None = None,
|
||||||
|
) -> None:
|
||||||
|
self.coordinator = coordinator
|
||||||
|
self.identity = identity
|
||||||
|
self.registry = registry or default_provider_registry()
|
||||||
|
self.registry.get(identity.executor_type)
|
||||||
|
|
||||||
|
def _assert_manifest(self, task_id: str) -> ExecutionTaskManifest:
|
||||||
|
record = self.coordinator.registry.get(task_id)
|
||||||
|
manifest = record.manifest
|
||||||
|
self.registry.validate_manifest(manifest)
|
||||||
|
if normalize_executor(manifest.executor) != self.identity.executor_type:
|
||||||
|
raise PermissionError(
|
||||||
|
f"task executor is {manifest.executor}, not {self.identity.executor_type}"
|
||||||
|
)
|
||||||
|
return manifest
|
||||||
|
|
||||||
|
def register(self, task_id: str) -> ExecutionTruthRecord:
|
||||||
|
self._assert_manifest(task_id)
|
||||||
|
return self.coordinator.register(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=self.identity.executor_run_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
def retry_not_started(self, task_id: str) -> ExecutionTruthRecord:
|
||||||
|
self._assert_manifest(task_id)
|
||||||
|
return self.coordinator.retry_not_started(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=self.identity.executor_run_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
def claim(self, task_id: str) -> ExecutionTruthRecord:
|
||||||
|
self._assert_manifest(task_id)
|
||||||
|
return self.coordinator.claim(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=self.identity.executor_run_id,
|
||||||
|
executor_id=self.identity.executor_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
def start(self, task_id: str) -> ExecutionTruthRecord:
|
||||||
|
self._assert_manifest(task_id)
|
||||||
|
return self.coordinator.start(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=self.identity.executor_run_id,
|
||||||
|
executor_id=self.identity.executor_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
def heartbeat(self, task_id: str, *, current_step: int) -> ExecutionTruthRecord:
|
||||||
|
self._assert_manifest(task_id)
|
||||||
|
return self.coordinator.heartbeat(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=self.identity.executor_run_id,
|
||||||
|
executor_id=self.identity.executor_id,
|
||||||
|
current_step=current_step,
|
||||||
|
)
|
||||||
|
|
||||||
|
def succeed(
|
||||||
|
self,
|
||||||
|
task_id: str,
|
||||||
|
*,
|
||||||
|
evidence: Mapping[str, str],
|
||||||
|
) -> ExecutionTruthRecord:
|
||||||
|
self._assert_manifest(task_id)
|
||||||
|
return self.coordinator.succeed(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=self.identity.executor_run_id,
|
||||||
|
executor_id=self.identity.executor_id,
|
||||||
|
evidence=evidence,
|
||||||
|
)
|
||||||
|
|
||||||
|
def fail(
|
||||||
|
self,
|
||||||
|
task_id: str,
|
||||||
|
*,
|
||||||
|
error: Mapping[str, Any],
|
||||||
|
evidence: Mapping[str, str] | None = None,
|
||||||
|
) -> ExecutionTruthRecord:
|
||||||
|
self._assert_manifest(task_id)
|
||||||
|
return self.coordinator.fail(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=self.identity.executor_run_id,
|
||||||
|
executor_id=self.identity.executor_id,
|
||||||
|
error=error,
|
||||||
|
evidence=evidence,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class OpenClawExecutorSession(ExecutorLifecycleSession):
|
||||||
|
def __init__(self, coordinator: ExternalTaskCoordinator, *, executor_id: str, executor_run_id: str) -> None:
|
||||||
|
super().__init__(
|
||||||
|
coordinator,
|
||||||
|
ExecutorIdentity("openclaw", executor_id, executor_run_id),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class HermesExecutorSession(ExecutorLifecycleSession):
|
||||||
|
def __init__(self, coordinator: ExternalTaskCoordinator, *, executor_id: str, executor_run_id: str) -> None:
|
||||||
|
super().__init__(
|
||||||
|
coordinator,
|
||||||
|
ExecutorIdentity("hermes", executor_id, executor_run_id),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class CodexExecutorSession(ExecutorLifecycleSession):
|
||||||
|
def __init__(self, coordinator: ExternalTaskCoordinator, *, executor_id: str, executor_run_id: str) -> None:
|
||||||
|
super().__init__(
|
||||||
|
coordinator,
|
||||||
|
ExecutorIdentity("codex", executor_id, executor_run_id),
|
||||||
|
)
|
||||||
@@ -0,0 +1,313 @@
|
|||||||
|
"""Provider-neutral worker runtime for AIOS external executors.
|
||||||
|
|
||||||
|
Providers implement only step execution and notification delivery. Discovery,
|
||||||
|
dependency gates, lifecycle truth, heartbeat, evidence validation and durable
|
||||||
|
notification acknowledgement are identical for every current and future
|
||||||
|
executor Provider.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import uuid
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Any, Mapping, Protocol, Sequence
|
||||||
|
|
||||||
|
from .execution_plan import ExecutionPlanCoordinator, ExecutionStageManifest
|
||||||
|
from .executor_protocol import (
|
||||||
|
ExecutorIdentity,
|
||||||
|
ExecutorLifecycleSession,
|
||||||
|
ExecutorProviderRegistry,
|
||||||
|
normalize_executor,
|
||||||
|
)
|
||||||
|
from .external_task_control import ExternalTaskCoordinator, NotificationRecord
|
||||||
|
from .task_execution_registry import ExecutionTruthState
|
||||||
|
|
||||||
|
|
||||||
|
class StepRunner(Protocol):
|
||||||
|
def run_step(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
plan_id: str,
|
||||||
|
stage: ExecutionStageManifest,
|
||||||
|
step: str,
|
||||||
|
step_number: int,
|
||||||
|
executor_id: str,
|
||||||
|
executor_run_id: str,
|
||||||
|
) -> Mapping[str, str]: ...
|
||||||
|
|
||||||
|
|
||||||
|
class NotificationSender(Protocol):
|
||||||
|
def send(self, notification: NotificationRecord) -> str: ...
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class StageRunResult:
|
||||||
|
plan_id: str
|
||||||
|
stage_id: str
|
||||||
|
task_id: str
|
||||||
|
executor_type: str
|
||||||
|
executor_id: str
|
||||||
|
executor_run_id: str
|
||||||
|
state: str
|
||||||
|
evidence: Mapping[str, str] = field(default_factory=dict)
|
||||||
|
error: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
delivered_notifications: Sequence[str] = ()
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"plan_id": self.plan_id,
|
||||||
|
"stage_id": self.stage_id,
|
||||||
|
"task_id": self.task_id,
|
||||||
|
"executor_type": self.executor_type,
|
||||||
|
"executor_id": self.executor_id,
|
||||||
|
"executor_run_id": self.executor_run_id,
|
||||||
|
"state": self.state,
|
||||||
|
"evidence": dict(self.evidence),
|
||||||
|
"error": dict(self.error),
|
||||||
|
"delivered_notifications": list(self.delivered_notifications),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class UnifiedExecutorWorker:
|
||||||
|
"""Dependency-aware worker shared by every registered Provider."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
plan_coordinator: ExecutionPlanCoordinator,
|
||||||
|
task_coordinator: ExternalTaskCoordinator,
|
||||||
|
providers: ExecutorProviderRegistry,
|
||||||
|
executor_type: str,
|
||||||
|
executor_id: str,
|
||||||
|
step_runner: StepRunner,
|
||||||
|
notification_sender: NotificationSender,
|
||||||
|
) -> None:
|
||||||
|
self.plan_coordinator = plan_coordinator
|
||||||
|
self.task_coordinator = task_coordinator
|
||||||
|
self.providers = providers
|
||||||
|
self.executor_type = normalize_executor(executor_type)
|
||||||
|
self.executor_id = str(executor_id or "").strip()
|
||||||
|
if not self.executor_id:
|
||||||
|
raise ValueError("executor_id is required")
|
||||||
|
self.providers.get(self.executor_type)
|
||||||
|
self.step_runner = step_runner
|
||||||
|
self.notification_sender = notification_sender
|
||||||
|
|
||||||
|
def _session(self, executor_run_id: str) -> ExecutorLifecycleSession:
|
||||||
|
return ExecutorLifecycleSession(
|
||||||
|
self.task_coordinator,
|
||||||
|
ExecutorIdentity(
|
||||||
|
executor_type=self.executor_type,
|
||||||
|
executor_id=self.executor_id,
|
||||||
|
executor_run_id=executor_run_id,
|
||||||
|
),
|
||||||
|
registry=self.providers,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _notification_candidates(self, task_id: str) -> tuple[NotificationRecord, ...]:
|
||||||
|
"""Return idempotent delivery candidates, including prior failures.
|
||||||
|
|
||||||
|
The outbox has a unique ``(task_id, event_type)`` key. Retrying therefore
|
||||||
|
reuses the same notification ID and cannot create duplicate STARTED or
|
||||||
|
SUCCEEDED events. A later successful acknowledgement replaces FAILED
|
||||||
|
with DELIVERED and records the receipt hash.
|
||||||
|
"""
|
||||||
|
candidates: dict[str, NotificationRecord] = {}
|
||||||
|
for status in ("PENDING", "FAILED"):
|
||||||
|
for item in self.task_coordinator.notifications.list(
|
||||||
|
task_id=task_id,
|
||||||
|
status=status,
|
||||||
|
):
|
||||||
|
candidates[item.notification_id] = item
|
||||||
|
return tuple(
|
||||||
|
candidates[key]
|
||||||
|
for key in sorted(
|
||||||
|
candidates,
|
||||||
|
key=lambda item: (
|
||||||
|
candidates[item].created_at,
|
||||||
|
candidates[item].notification_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
def _deliver_notifications(self, task_id: str) -> tuple[str, ...]:
|
||||||
|
delivered: list[str] = []
|
||||||
|
for item in self._notification_candidates(task_id):
|
||||||
|
try:
|
||||||
|
receipt = self.notification_sender.send(item)
|
||||||
|
acknowledged = self.task_coordinator.notifications.acknowledge(
|
||||||
|
item.notification_id,
|
||||||
|
receipt_sha256=receipt,
|
||||||
|
)
|
||||||
|
delivered.append(acknowledged.event_type)
|
||||||
|
except Exception as exc:
|
||||||
|
self.task_coordinator.notifications.fail(
|
||||||
|
item.notification_id,
|
||||||
|
error=f"{type(exc).__name__}: {exc}",
|
||||||
|
)
|
||||||
|
return tuple(sorted(set(delivered)))
|
||||||
|
|
||||||
|
def retry_failed_notifications(self, *, task_id: str) -> tuple[str, ...]:
|
||||||
|
"""Retry failed owner notifications without changing task truth."""
|
||||||
|
return self._deliver_notifications(task_id)
|
||||||
|
|
||||||
|
def run_ready(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
plan_id: str,
|
||||||
|
max_stages: int = 1,
|
||||||
|
) -> list[StageRunResult]:
|
||||||
|
if max_stages < 1:
|
||||||
|
raise ValueError("max_stages must be positive")
|
||||||
|
ready = self.plan_coordinator.discover_ready(
|
||||||
|
plan_id=plan_id,
|
||||||
|
executor_type=self.executor_type,
|
||||||
|
)[:max_stages]
|
||||||
|
return [
|
||||||
|
self.run_stage(
|
||||||
|
plan_id=plan_id,
|
||||||
|
stage_id=item["stage"]["stage_id"],
|
||||||
|
)
|
||||||
|
for item in ready
|
||||||
|
]
|
||||||
|
|
||||||
|
def run_stage(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
plan_id: str,
|
||||||
|
stage_id: str,
|
||||||
|
executor_run_id: str | None = None,
|
||||||
|
) -> StageRunResult:
|
||||||
|
plan = self.plan_coordinator.plan_store.get(plan_id)
|
||||||
|
stage = plan.stage(stage_id)
|
||||||
|
if stage.executor_type != self.executor_type:
|
||||||
|
raise PermissionError(
|
||||||
|
f"stage executor is {stage.executor_type}, not {self.executor_type}"
|
||||||
|
)
|
||||||
|
dependency = self.plan_coordinator.dependency_gate(plan_id, stage_id)
|
||||||
|
if not dependency["passed"]:
|
||||||
|
raise RuntimeError("stage dependencies are not satisfied")
|
||||||
|
|
||||||
|
run_id = executor_run_id or f"{self.executor_type}_run_{uuid.uuid4().hex}"
|
||||||
|
session = self._session(run_id)
|
||||||
|
record = self.task_coordinator.registry.get(stage.task_id)
|
||||||
|
evidence: dict[str, str] = dict(record.evidence)
|
||||||
|
delivered: list[str] = []
|
||||||
|
try:
|
||||||
|
if record.state == ExecutionTruthState.DECLARED:
|
||||||
|
record = session.register(stage.task_id)
|
||||||
|
if record.state == ExecutionTruthState.REGISTERED:
|
||||||
|
record = session.claim(stage.task_id)
|
||||||
|
if record.state == ExecutionTruthState.CLAIMED:
|
||||||
|
record = session.start(stage.task_id)
|
||||||
|
delivered.extend(self._deliver_notifications(stage.task_id))
|
||||||
|
if record.state != ExecutionTruthState.RUNNING:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"stage cannot execute from state {record.state.value}"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Re-running the current step is intentional: Provider steps must be
|
||||||
|
# idempotent and a crash between side effect and heartbeat cannot be
|
||||||
|
# safely interpreted as completed work.
|
||||||
|
start_step = max(1, record.current_step)
|
||||||
|
for number, step in enumerate(stage.steps, start=1):
|
||||||
|
if number < start_step:
|
||||||
|
continue
|
||||||
|
produced = self.step_runner.run_step(
|
||||||
|
plan_id=plan_id,
|
||||||
|
stage=stage,
|
||||||
|
step=step,
|
||||||
|
step_number=number,
|
||||||
|
executor_id=self.executor_id,
|
||||||
|
executor_run_id=run_id,
|
||||||
|
)
|
||||||
|
if not isinstance(produced, Mapping):
|
||||||
|
raise TypeError("step runner must return an evidence mapping")
|
||||||
|
evidence.update(
|
||||||
|
{
|
||||||
|
str(key).strip(): str(value).strip().lower()
|
||||||
|
for key, value in produced.items()
|
||||||
|
}
|
||||||
|
)
|
||||||
|
record = session.heartbeat(stage.task_id, current_step=number)
|
||||||
|
|
||||||
|
record = session.succeed(stage.task_id, evidence=evidence)
|
||||||
|
delivered.extend(self._deliver_notifications(stage.task_id))
|
||||||
|
return StageRunResult(
|
||||||
|
plan_id=plan_id,
|
||||||
|
stage_id=stage.stage_id,
|
||||||
|
task_id=stage.task_id,
|
||||||
|
executor_type=self.executor_type,
|
||||||
|
executor_id=self.executor_id,
|
||||||
|
executor_run_id=run_id,
|
||||||
|
state=record.state.value,
|
||||||
|
evidence=dict(record.evidence),
|
||||||
|
delivered_notifications=tuple(sorted(set(delivered))),
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
current = self.task_coordinator.registry.get(stage.task_id)
|
||||||
|
if current.state in {
|
||||||
|
ExecutionTruthState.REGISTERED,
|
||||||
|
ExecutionTruthState.CLAIMED,
|
||||||
|
ExecutionTruthState.RUNNING,
|
||||||
|
}:
|
||||||
|
current = session.fail(
|
||||||
|
stage.task_id,
|
||||||
|
error={
|
||||||
|
"code": "executor_stage_failed",
|
||||||
|
"error_type": type(exc).__name__,
|
||||||
|
"message": str(exc),
|
||||||
|
},
|
||||||
|
evidence=evidence or None,
|
||||||
|
)
|
||||||
|
delivered.extend(self._deliver_notifications(stage.task_id))
|
||||||
|
return StageRunResult(
|
||||||
|
plan_id=plan_id,
|
||||||
|
stage_id=stage.stage_id,
|
||||||
|
task_id=stage.task_id,
|
||||||
|
executor_type=self.executor_type,
|
||||||
|
executor_id=self.executor_id,
|
||||||
|
executor_run_id=run_id,
|
||||||
|
state=current.state.value,
|
||||||
|
evidence=dict(current.evidence),
|
||||||
|
error=dict(current.error),
|
||||||
|
delivered_notifications=tuple(sorted(set(delivered))),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class JsonEvidenceStepRunner:
|
||||||
|
"""Adapter for handlers returning evidence-name to SHA256 mappings."""
|
||||||
|
|
||||||
|
def __init__(self, handler) -> None:
|
||||||
|
self.handler = handler
|
||||||
|
|
||||||
|
def run_step(self, **values: Any) -> Mapping[str, str]:
|
||||||
|
result = self.handler(**values)
|
||||||
|
if not isinstance(result, Mapping):
|
||||||
|
raise TypeError("step handler must return a mapping")
|
||||||
|
return {str(key): str(value) for key, value in result.items()}
|
||||||
|
|
||||||
|
|
||||||
|
class JsonReceiptNotificationSender:
|
||||||
|
"""Adapter for senders returning a SHA256 or receipt JSON object."""
|
||||||
|
|
||||||
|
def __init__(self, sender) -> None:
|
||||||
|
self.sender = sender
|
||||||
|
|
||||||
|
def send(self, notification: NotificationRecord) -> str:
|
||||||
|
result = self.sender(notification.as_dict())
|
||||||
|
if isinstance(result, str):
|
||||||
|
return result
|
||||||
|
if not isinstance(result, Mapping):
|
||||||
|
raise TypeError(
|
||||||
|
"notification sender must return SHA256 or a receipt object"
|
||||||
|
)
|
||||||
|
encoded = json.dumps(
|
||||||
|
dict(result),
|
||||||
|
ensure_ascii=False,
|
||||||
|
sort_keys=True,
|
||||||
|
separators=(",", ":"),
|
||||||
|
default=str,
|
||||||
|
).encode("utf-8")
|
||||||
|
return hashlib.sha256(encoded).hexdigest()
|
||||||
@@ -0,0 +1,266 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""CLI for durable external-task truth and notification receipts.
|
||||||
|
|
||||||
|
The CLI is suitable for OpenClaw workers, systemd jobs and release gates. It
|
||||||
|
never infers execution from the presence of a manifest: each lifecycle action
|
||||||
|
must be recorded explicitly in the shared SQLite registry.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Sequence
|
||||||
|
|
||||||
|
from .external_task_control import ExternalTaskCoordinator
|
||||||
|
from .external_task_query import list_records
|
||||||
|
|
||||||
|
|
||||||
|
def _json_mapping(value: str | None, path: str | None) -> Mapping[str, Any]:
|
||||||
|
if value and path:
|
||||||
|
raise ValueError("inline JSON and JSON file are mutually exclusive")
|
||||||
|
if path:
|
||||||
|
parsed = json.loads(Path(path).read_text(encoding="utf-8"))
|
||||||
|
else:
|
||||||
|
parsed = json.loads(value or "{}")
|
||||||
|
if not isinstance(parsed, Mapping):
|
||||||
|
raise ValueError("JSON value must be an object")
|
||||||
|
return dict(parsed)
|
||||||
|
|
||||||
|
|
||||||
|
def _coordinator(args: argparse.Namespace) -> ExternalTaskCoordinator:
|
||||||
|
return ExternalTaskCoordinator.from_file(args.store_file)
|
||||||
|
|
||||||
|
|
||||||
|
def _print(value: Any) -> None:
|
||||||
|
print(json.dumps(value, ensure_ascii=False, sort_keys=True, indent=2, default=str))
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_declare(args: argparse.Namespace) -> int:
|
||||||
|
record = _coordinator(args).declare_file(args.manifest)
|
||||||
|
_print(record.as_dict())
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_discover(args: argparse.Namespace) -> int:
|
||||||
|
records = list_records(
|
||||||
|
args.store_file,
|
||||||
|
executor=args.executor,
|
||||||
|
states=tuple(args.state or ()),
|
||||||
|
)
|
||||||
|
_print({"count": len(records), "items": [item.as_dict() for item in records]})
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_register(args: argparse.Namespace) -> int:
|
||||||
|
_print(_coordinator(args).register(args.task_id, executor_run_id=args.executor_run_id).as_dict())
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_claim(args: argparse.Namespace) -> int:
|
||||||
|
_print(
|
||||||
|
_coordinator(args).claim(
|
||||||
|
args.task_id,
|
||||||
|
executor_run_id=args.executor_run_id,
|
||||||
|
executor_id=args.executor_id,
|
||||||
|
).as_dict()
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_start(args: argparse.Namespace) -> int:
|
||||||
|
_print(
|
||||||
|
_coordinator(args).start(
|
||||||
|
args.task_id,
|
||||||
|
executor_run_id=args.executor_run_id,
|
||||||
|
executor_id=args.executor_id,
|
||||||
|
).as_dict()
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_heartbeat(args: argparse.Namespace) -> int:
|
||||||
|
_print(
|
||||||
|
_coordinator(args).heartbeat(
|
||||||
|
args.task_id,
|
||||||
|
executor_run_id=args.executor_run_id,
|
||||||
|
executor_id=args.executor_id,
|
||||||
|
current_step=args.current_step,
|
||||||
|
).as_dict()
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_succeed(args: argparse.Namespace) -> int:
|
||||||
|
evidence = _json_mapping(args.evidence_json, args.evidence_file)
|
||||||
|
_print(
|
||||||
|
_coordinator(args).succeed(
|
||||||
|
args.task_id,
|
||||||
|
executor_run_id=args.executor_run_id,
|
||||||
|
executor_id=args.executor_id,
|
||||||
|
evidence={str(key): str(value) for key, value in evidence.items()},
|
||||||
|
).as_dict()
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_fail(args: argparse.Namespace) -> int:
|
||||||
|
error = _json_mapping(args.error_json, args.error_file)
|
||||||
|
evidence = None
|
||||||
|
if args.evidence_json or args.evidence_file:
|
||||||
|
evidence_value = _json_mapping(args.evidence_json, args.evidence_file)
|
||||||
|
evidence = {str(key): str(value) for key, value in evidence_value.items()}
|
||||||
|
_print(
|
||||||
|
_coordinator(args).fail(
|
||||||
|
args.task_id,
|
||||||
|
executor_run_id=args.executor_run_id,
|
||||||
|
executor_id=args.executor_id,
|
||||||
|
error=error,
|
||||||
|
evidence=evidence,
|
||||||
|
).as_dict()
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_status(args: argparse.Namespace) -> int:
|
||||||
|
_print(_coordinator(args).status(args.task_id))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_gate(args: argparse.Namespace) -> int:
|
||||||
|
value = _coordinator(args).gate(args.task_id)
|
||||||
|
_print(value)
|
||||||
|
return 0 if value["passed"] else 3
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_notifications(args: argparse.Namespace) -> int:
|
||||||
|
items = _coordinator(args).notifications.list(task_id=args.task_id, status=args.status)
|
||||||
|
_print({"count": len(items), "items": [item.as_dict() for item in items]})
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_ack(args: argparse.Namespace) -> int:
|
||||||
|
_print(
|
||||||
|
_coordinator(args).notifications.acknowledge(
|
||||||
|
args.notification_id,
|
||||||
|
receipt_sha256=args.receipt_sha256,
|
||||||
|
).as_dict()
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def common(parser: argparse.ArgumentParser) -> None:
|
||||||
|
parser.add_argument(
|
||||||
|
"--store-file",
|
||||||
|
default=os.getenv(
|
||||||
|
"AIOS_EXTERNAL_TASK_STORE_FILE",
|
||||||
|
"/var/lib/aios-v2/external-task-truth.sqlite3",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def lifecycle(parser: argparse.ArgumentParser) -> None:
|
||||||
|
parser.add_argument("--task-id", required=True)
|
||||||
|
parser.add_argument("--executor-run-id", required=True)
|
||||||
|
parser.add_argument("--executor-id", required=True)
|
||||||
|
|
||||||
|
|
||||||
|
def build_parser() -> argparse.ArgumentParser:
|
||||||
|
parser = argparse.ArgumentParser(description="AIOS external task truth CLI")
|
||||||
|
commands = parser.add_subparsers(dest="command", required=True)
|
||||||
|
|
||||||
|
declare = commands.add_parser("declare")
|
||||||
|
common(declare)
|
||||||
|
declare.add_argument("--manifest", required=True)
|
||||||
|
declare.set_defaults(handler=cmd_declare)
|
||||||
|
|
||||||
|
discover = commands.add_parser("discover")
|
||||||
|
common(discover)
|
||||||
|
discover.add_argument("--executor", default="openclaw")
|
||||||
|
discover.add_argument("--state", action="append")
|
||||||
|
discover.set_defaults(handler=cmd_discover)
|
||||||
|
|
||||||
|
register = commands.add_parser("register")
|
||||||
|
common(register)
|
||||||
|
register.add_argument("--task-id", required=True)
|
||||||
|
register.add_argument("--executor-run-id", required=True)
|
||||||
|
register.set_defaults(handler=cmd_register)
|
||||||
|
|
||||||
|
claim = commands.add_parser("claim")
|
||||||
|
common(claim)
|
||||||
|
lifecycle(claim)
|
||||||
|
claim.set_defaults(handler=cmd_claim)
|
||||||
|
|
||||||
|
start = commands.add_parser("start")
|
||||||
|
common(start)
|
||||||
|
lifecycle(start)
|
||||||
|
start.set_defaults(handler=cmd_start)
|
||||||
|
|
||||||
|
heartbeat = commands.add_parser("heartbeat")
|
||||||
|
common(heartbeat)
|
||||||
|
lifecycle(heartbeat)
|
||||||
|
heartbeat.add_argument("--current-step", type=int, required=True)
|
||||||
|
heartbeat.set_defaults(handler=cmd_heartbeat)
|
||||||
|
|
||||||
|
succeed = commands.add_parser("succeed")
|
||||||
|
common(succeed)
|
||||||
|
lifecycle(succeed)
|
||||||
|
evidence = succeed.add_mutually_exclusive_group(required=True)
|
||||||
|
evidence.add_argument("--evidence-json")
|
||||||
|
evidence.add_argument("--evidence-file")
|
||||||
|
succeed.set_defaults(handler=cmd_succeed)
|
||||||
|
|
||||||
|
fail = commands.add_parser("fail")
|
||||||
|
common(fail)
|
||||||
|
lifecycle(fail)
|
||||||
|
error = fail.add_mutually_exclusive_group(required=True)
|
||||||
|
error.add_argument("--error-json")
|
||||||
|
error.add_argument("--error-file")
|
||||||
|
fail_evidence = fail.add_mutually_exclusive_group()
|
||||||
|
fail_evidence.add_argument("--evidence-json")
|
||||||
|
fail_evidence.add_argument("--evidence-file")
|
||||||
|
fail.set_defaults(handler=cmd_fail)
|
||||||
|
|
||||||
|
status = commands.add_parser("status")
|
||||||
|
common(status)
|
||||||
|
status.add_argument("--task-id", required=True)
|
||||||
|
status.set_defaults(handler=cmd_status)
|
||||||
|
|
||||||
|
gate = commands.add_parser("gate")
|
||||||
|
common(gate)
|
||||||
|
gate.add_argument("--task-id", required=True)
|
||||||
|
gate.set_defaults(handler=cmd_gate)
|
||||||
|
|
||||||
|
notifications = commands.add_parser("notifications")
|
||||||
|
common(notifications)
|
||||||
|
notifications.add_argument("--task-id")
|
||||||
|
notifications.add_argument("--status")
|
||||||
|
notifications.set_defaults(handler=cmd_notifications)
|
||||||
|
|
||||||
|
ack = commands.add_parser("ack-notification")
|
||||||
|
common(ack)
|
||||||
|
ack.add_argument("--notification-id", required=True)
|
||||||
|
ack.add_argument("--receipt-sha256", required=True)
|
||||||
|
ack.set_defaults(handler=cmd_ack)
|
||||||
|
return parser
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: Sequence[str] | None = None) -> int:
|
||||||
|
args = build_parser().parse_args(argv)
|
||||||
|
try:
|
||||||
|
return int(args.handler(args))
|
||||||
|
except Exception as exc:
|
||||||
|
print(
|
||||||
|
json.dumps(
|
||||||
|
{"status": "error", "error": type(exc).__name__, "message": str(exc)},
|
||||||
|
ensure_ascii=False,
|
||||||
|
),
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,420 @@
|
|||||||
|
"""External executor coordination, notification outbox and release truth gate.
|
||||||
|
|
||||||
|
OpenClaw/Codex task declarations are not execution evidence. This module wraps
|
||||||
|
``SQLiteExecutionTruthRegistry`` with a durable notification outbox. A successful
|
||||||
|
release gate requires both terminal task evidence and acknowledged STARTED /
|
||||||
|
SUCCEEDED owner notifications when those events are required by the manifest.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import sqlite3
|
||||||
|
import threading
|
||||||
|
import uuid
|
||||||
|
from dataclasses import asdict, dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Sequence
|
||||||
|
|
||||||
|
from .task_execution_registry import (
|
||||||
|
ExecutionTaskManifest,
|
||||||
|
ExecutionTruthRecord,
|
||||||
|
ExecutionTruthState,
|
||||||
|
SQLiteExecutionTruthRegistry,
|
||||||
|
utc_now,
|
||||||
|
)
|
||||||
|
|
||||||
|
_SHA256 = re.compile(r"^[0-9a-f]{64}$")
|
||||||
|
|
||||||
|
|
||||||
|
def sha256_value(value: str, label: str) -> str:
|
||||||
|
normalized = str(value or "").strip().lower()
|
||||||
|
if not _SHA256.fullmatch(normalized):
|
||||||
|
raise ValueError(f"{label} must be a lowercase SHA256 hex digest")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class NotificationRecord:
|
||||||
|
notification_id: str
|
||||||
|
task_id: str
|
||||||
|
event_type: str
|
||||||
|
target_ref: str
|
||||||
|
payload: Mapping[str, Any]
|
||||||
|
status: str
|
||||||
|
created_at: str
|
||||||
|
delivered_at: str | None = None
|
||||||
|
receipt_sha256: str | None = None
|
||||||
|
error: str | None = None
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return asdict(self)
|
||||||
|
|
||||||
|
|
||||||
|
class SQLiteNotificationOutbox:
|
||||||
|
"""Durable owner-notification delivery and acknowledgement records."""
|
||||||
|
|
||||||
|
def __init__(self, path: str | Path) -> None:
|
||||||
|
self.path = Path(path).resolve()
|
||||||
|
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
with self._connect() as connection:
|
||||||
|
connection.executescript(
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS external_task_notifications (
|
||||||
|
notification_id TEXT PRIMARY KEY,
|
||||||
|
task_id TEXT NOT NULL,
|
||||||
|
event_type TEXT NOT NULL,
|
||||||
|
target_ref TEXT NOT NULL,
|
||||||
|
payload_json TEXT NOT NULL,
|
||||||
|
status TEXT NOT NULL,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
delivered_at TEXT,
|
||||||
|
receipt_sha256 TEXT,
|
||||||
|
error TEXT,
|
||||||
|
UNIQUE(task_id, event_type)
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_external_notification_status
|
||||||
|
ON external_task_notifications(status, created_at);
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
|
||||||
|
def _connect(self) -> sqlite3.Connection:
|
||||||
|
connection = sqlite3.connect(str(self.path), timeout=30.0)
|
||||||
|
connection.row_factory = sqlite3.Row
|
||||||
|
connection.execute("PRAGMA journal_mode=WAL")
|
||||||
|
connection.execute("PRAGMA synchronous=FULL")
|
||||||
|
return connection
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _decode(row: sqlite3.Row) -> NotificationRecord:
|
||||||
|
return NotificationRecord(
|
||||||
|
notification_id=row["notification_id"],
|
||||||
|
task_id=row["task_id"],
|
||||||
|
event_type=row["event_type"],
|
||||||
|
target_ref=row["target_ref"],
|
||||||
|
payload=json.loads(row["payload_json"]),
|
||||||
|
status=row["status"],
|
||||||
|
created_at=row["created_at"],
|
||||||
|
delivered_at=row["delivered_at"],
|
||||||
|
receipt_sha256=row["receipt_sha256"],
|
||||||
|
error=row["error"],
|
||||||
|
)
|
||||||
|
|
||||||
|
def enqueue(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
task_id: str,
|
||||||
|
event_type: str,
|
||||||
|
target_ref: str,
|
||||||
|
payload: Mapping[str, Any],
|
||||||
|
) -> NotificationRecord:
|
||||||
|
now = utc_now()
|
||||||
|
record = NotificationRecord(
|
||||||
|
notification_id=f"notify_{uuid.uuid4().hex}",
|
||||||
|
task_id=str(task_id).strip(),
|
||||||
|
event_type=str(event_type).strip().upper(),
|
||||||
|
target_ref=str(target_ref).strip(),
|
||||||
|
payload=json.loads(json.dumps(dict(payload), ensure_ascii=False, default=str)),
|
||||||
|
status="PENDING",
|
||||||
|
created_at=now,
|
||||||
|
)
|
||||||
|
if not record.task_id or not record.event_type or not record.target_ref:
|
||||||
|
raise ValueError("task_id, event_type and target_ref are required")
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
connection.execute("BEGIN IMMEDIATE")
|
||||||
|
existing = connection.execute(
|
||||||
|
"SELECT * FROM external_task_notifications WHERE task_id = ? AND event_type = ?",
|
||||||
|
(record.task_id, record.event_type),
|
||||||
|
).fetchone()
|
||||||
|
if existing is not None:
|
||||||
|
connection.rollback()
|
||||||
|
return self._decode(existing)
|
||||||
|
connection.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO external_task_notifications
|
||||||
|
(notification_id, task_id, event_type, target_ref, payload_json,
|
||||||
|
status, created_at, delivered_at, receipt_sha256, error)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?, ?, NULL, NULL, NULL)
|
||||||
|
""",
|
||||||
|
(
|
||||||
|
record.notification_id,
|
||||||
|
record.task_id,
|
||||||
|
record.event_type,
|
||||||
|
record.target_ref,
|
||||||
|
json.dumps(record.payload, ensure_ascii=False, sort_keys=True, separators=(",", ":")),
|
||||||
|
record.status,
|
||||||
|
record.created_at,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
connection.commit()
|
||||||
|
return record
|
||||||
|
|
||||||
|
def get(self, notification_id: str) -> NotificationRecord:
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
row = connection.execute(
|
||||||
|
"SELECT * FROM external_task_notifications WHERE notification_id = ?",
|
||||||
|
(str(notification_id).strip(),),
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise KeyError("notification not found")
|
||||||
|
return self._decode(row)
|
||||||
|
|
||||||
|
def acknowledge(self, notification_id: str, *, receipt_sha256: str) -> NotificationRecord:
|
||||||
|
digest = sha256_value(receipt_sha256, "receipt_sha256")
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
connection.execute("BEGIN IMMEDIATE")
|
||||||
|
row = connection.execute(
|
||||||
|
"SELECT * FROM external_task_notifications WHERE notification_id = ?",
|
||||||
|
(str(notification_id).strip(),),
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
connection.rollback()
|
||||||
|
raise KeyError("notification not found")
|
||||||
|
current = self._decode(row)
|
||||||
|
if current.status == "DELIVERED":
|
||||||
|
if current.receipt_sha256 != digest:
|
||||||
|
connection.rollback()
|
||||||
|
raise RuntimeError("notification already acknowledged with another receipt")
|
||||||
|
connection.rollback()
|
||||||
|
return current
|
||||||
|
delivered_at = utc_now()
|
||||||
|
connection.execute(
|
||||||
|
"""
|
||||||
|
UPDATE external_task_notifications
|
||||||
|
SET status = 'DELIVERED', delivered_at = ?, receipt_sha256 = ?, error = NULL
|
||||||
|
WHERE notification_id = ?
|
||||||
|
""",
|
||||||
|
(delivered_at, digest, current.notification_id),
|
||||||
|
)
|
||||||
|
connection.commit()
|
||||||
|
return self.get(current.notification_id)
|
||||||
|
|
||||||
|
def fail(self, notification_id: str, *, error: str) -> NotificationRecord:
|
||||||
|
message = str(error or "").strip()
|
||||||
|
if not message:
|
||||||
|
raise ValueError("notification error is required")
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE external_task_notifications SET status = 'FAILED', error = ? WHERE notification_id = ?",
|
||||||
|
(message, str(notification_id).strip()),
|
||||||
|
)
|
||||||
|
connection.commit()
|
||||||
|
return self.get(notification_id)
|
||||||
|
|
||||||
|
def list(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
task_id: str | None = None,
|
||||||
|
status: str | None = None,
|
||||||
|
) -> list[NotificationRecord]:
|
||||||
|
conditions: list[str] = []
|
||||||
|
values: list[str] = []
|
||||||
|
if task_id:
|
||||||
|
conditions.append("task_id = ?")
|
||||||
|
values.append(str(task_id).strip())
|
||||||
|
if status:
|
||||||
|
conditions.append("status = ?")
|
||||||
|
values.append(str(status).strip().upper())
|
||||||
|
where = f" WHERE {' AND '.join(conditions)}" if conditions else ""
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
rows = connection.execute(
|
||||||
|
f"SELECT * FROM external_task_notifications{where} ORDER BY created_at, notification_id",
|
||||||
|
tuple(values),
|
||||||
|
).fetchall()
|
||||||
|
return [self._decode(row) for row in rows]
|
||||||
|
|
||||||
|
|
||||||
|
class ExternalTaskCoordinator:
|
||||||
|
"""Single safe lifecycle used by external executors and release gates."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
registry: SQLiteExecutionTruthRegistry,
|
||||||
|
notifications: SQLiteNotificationOutbox,
|
||||||
|
) -> None:
|
||||||
|
self.registry = registry
|
||||||
|
self.notifications = notifications
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_file(cls, path: str | Path) -> "ExternalTaskCoordinator":
|
||||||
|
return cls(
|
||||||
|
SQLiteExecutionTruthRegistry(path),
|
||||||
|
SQLiteNotificationOutbox(path),
|
||||||
|
)
|
||||||
|
|
||||||
|
def declare(self, manifest: ExecutionTaskManifest) -> ExecutionTruthRecord:
|
||||||
|
return self.registry.declare(manifest)
|
||||||
|
|
||||||
|
def declare_file(self, path: str | Path) -> ExecutionTruthRecord:
|
||||||
|
value = json.loads(Path(path).read_text(encoding="utf-8"))
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise ValueError("external task manifest must be a JSON object")
|
||||||
|
return self.declare(ExecutionTaskManifest.from_dict(value))
|
||||||
|
|
||||||
|
def register(self, task_id: str, *, executor_run_id: str) -> ExecutionTruthRecord:
|
||||||
|
return self.registry.register(task_id, executor_run_id=executor_run_id)
|
||||||
|
|
||||||
|
def retry_not_started(
|
||||||
|
self,
|
||||||
|
task_id: str,
|
||||||
|
*,
|
||||||
|
executor_run_id: str,
|
||||||
|
) -> ExecutionTruthRecord:
|
||||||
|
return self.registry.retry_not_started(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=executor_run_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
def claim(
|
||||||
|
self,
|
||||||
|
task_id: str,
|
||||||
|
*,
|
||||||
|
executor_run_id: str,
|
||||||
|
executor_id: str,
|
||||||
|
) -> ExecutionTruthRecord:
|
||||||
|
return self.registry.claim(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=executor_run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _notify(self, record: ExecutionTruthRecord, event_type: str) -> NotificationRecord | None:
|
||||||
|
event = event_type.upper()
|
||||||
|
if event not in set(record.manifest.notification_events):
|
||||||
|
return None
|
||||||
|
return self.notifications.enqueue(
|
||||||
|
task_id=record.task_id,
|
||||||
|
event_type=event,
|
||||||
|
target_ref=record.manifest.owner_notification_ref,
|
||||||
|
payload={
|
||||||
|
"task_id": record.task_id,
|
||||||
|
"target_version": record.manifest.target_version,
|
||||||
|
"executor": record.manifest.executor,
|
||||||
|
"executor_id": record.executor_id,
|
||||||
|
"executor_run_id": record.executor_run_id,
|
||||||
|
"state": record.state.value,
|
||||||
|
"current_step": record.current_step,
|
||||||
|
"total_steps": len(record.manifest.steps),
|
||||||
|
"timestamp": utc_now(),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
def start(
|
||||||
|
self,
|
||||||
|
task_id: str,
|
||||||
|
*,
|
||||||
|
executor_run_id: str,
|
||||||
|
executor_id: str,
|
||||||
|
) -> ExecutionTruthRecord:
|
||||||
|
record = self.registry.start(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=executor_run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
)
|
||||||
|
self._notify(record, "STARTED")
|
||||||
|
return record
|
||||||
|
|
||||||
|
def heartbeat(
|
||||||
|
self,
|
||||||
|
task_id: str,
|
||||||
|
*,
|
||||||
|
executor_run_id: str,
|
||||||
|
executor_id: str,
|
||||||
|
current_step: int,
|
||||||
|
) -> ExecutionTruthRecord:
|
||||||
|
return self.registry.heartbeat(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=executor_run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
current_step=current_step,
|
||||||
|
)
|
||||||
|
|
||||||
|
def succeed(
|
||||||
|
self,
|
||||||
|
task_id: str,
|
||||||
|
*,
|
||||||
|
executor_run_id: str,
|
||||||
|
executor_id: str,
|
||||||
|
evidence: Mapping[str, str],
|
||||||
|
) -> ExecutionTruthRecord:
|
||||||
|
normalized = {
|
||||||
|
str(key).strip(): sha256_value(value, f"evidence[{key}]")
|
||||||
|
for key, value in evidence.items()
|
||||||
|
}
|
||||||
|
record = self.registry.succeed(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=executor_run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
evidence=normalized,
|
||||||
|
)
|
||||||
|
self._notify(record, "SUCCEEDED")
|
||||||
|
return record
|
||||||
|
|
||||||
|
def fail(
|
||||||
|
self,
|
||||||
|
task_id: str,
|
||||||
|
*,
|
||||||
|
executor_run_id: str,
|
||||||
|
executor_id: str,
|
||||||
|
error: Mapping[str, Any],
|
||||||
|
evidence: Mapping[str, str] | None = None,
|
||||||
|
) -> ExecutionTruthRecord:
|
||||||
|
normalized = None
|
||||||
|
if evidence is not None:
|
||||||
|
normalized = {
|
||||||
|
str(key).strip(): sha256_value(value, f"evidence[{key}]")
|
||||||
|
for key, value in evidence.items()
|
||||||
|
}
|
||||||
|
record = self.registry.fail(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=executor_run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
error=error,
|
||||||
|
evidence=normalized,
|
||||||
|
)
|
||||||
|
self._notify(record, "FAILED")
|
||||||
|
return record
|
||||||
|
|
||||||
|
def gate(self, task_id: str) -> Mapping[str, Any]:
|
||||||
|
before = self.registry.get(task_id)
|
||||||
|
truth = self.registry.gate(task_id)
|
||||||
|
after = self.registry.get(task_id)
|
||||||
|
if after.state != before.state and after.state in {
|
||||||
|
ExecutionTruthState.NOT_STARTED,
|
||||||
|
ExecutionTruthState.STALLED,
|
||||||
|
}:
|
||||||
|
self._notify(after, after.state.value)
|
||||||
|
notifications = self.notifications.list(task_id=task_id)
|
||||||
|
delivered = {item.event_type for item in notifications if item.status == "DELIVERED"}
|
||||||
|
required_success_notifications = {
|
||||||
|
item
|
||||||
|
for item in ("STARTED", "SUCCEEDED")
|
||||||
|
if item in set(after.manifest.notification_events)
|
||||||
|
}
|
||||||
|
notification_complete = required_success_notifications.issubset(delivered)
|
||||||
|
passed = bool(truth["passed"] and notification_complete)
|
||||||
|
return {
|
||||||
|
**dict(truth),
|
||||||
|
"passed": passed,
|
||||||
|
"release_blocking": not passed,
|
||||||
|
"notification_complete": notification_complete,
|
||||||
|
"required_success_notifications": sorted(required_success_notifications),
|
||||||
|
"delivered_notifications": sorted(delivered),
|
||||||
|
"notifications": [item.as_dict() for item in notifications],
|
||||||
|
}
|
||||||
|
|
||||||
|
def status(self, task_id: str) -> Mapping[str, Any]:
|
||||||
|
gate = self.gate(task_id)
|
||||||
|
record = self.registry.get(task_id)
|
||||||
|
return {
|
||||||
|
"record": record.as_dict(),
|
||||||
|
"gate": gate,
|
||||||
|
"events": self.registry.events(task_id),
|
||||||
|
}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def content_receipt(value: Mapping[str, Any]) -> str:
|
||||||
|
encoded = json.dumps(dict(value), ensure_ascii=False, sort_keys=True, separators=(",", ":"), default=str).encode("utf-8")
|
||||||
|
return hashlib.sha256(encoded).hexdigest()
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
"""Read models for the durable external-task truth registry."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import sqlite3
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Sequence
|
||||||
|
|
||||||
|
from .task_execution_registry import ExecutionTruthRecord, ExecutionTruthState
|
||||||
|
|
||||||
|
|
||||||
|
def _decode(value: str) -> ExecutionTruthRecord:
|
||||||
|
payload = json.loads(value)
|
||||||
|
payload["state"] = ExecutionTruthState(payload["state"])
|
||||||
|
payload["evidence"] = dict(payload.get("evidence") or {})
|
||||||
|
payload["error"] = dict(payload.get("error") or {})
|
||||||
|
return ExecutionTruthRecord(**payload)
|
||||||
|
|
||||||
|
|
||||||
|
def list_records(
|
||||||
|
path: str | Path,
|
||||||
|
*,
|
||||||
|
executor: str | None = None,
|
||||||
|
states: Sequence[str] = (),
|
||||||
|
) -> list[ExecutionTruthRecord]:
|
||||||
|
database = Path(path).resolve()
|
||||||
|
if not database.exists():
|
||||||
|
return []
|
||||||
|
conditions: list[str] = []
|
||||||
|
values: list[str] = []
|
||||||
|
normalized_states = tuple(sorted({ExecutionTruthState(item).value for item in states}))
|
||||||
|
if normalized_states:
|
||||||
|
conditions.append(f"state IN ({','.join('?' for _ in normalized_states)})")
|
||||||
|
values.extend(normalized_states)
|
||||||
|
where = f" WHERE {' AND '.join(conditions)}" if conditions else ""
|
||||||
|
connection = sqlite3.connect(f"file:{database}?mode=ro", uri=True, timeout=10.0)
|
||||||
|
connection.row_factory = sqlite3.Row
|
||||||
|
try:
|
||||||
|
rows = connection.execute(
|
||||||
|
f"SELECT record_json FROM execution_truth{where} ORDER BY updated_at, task_id",
|
||||||
|
tuple(values),
|
||||||
|
).fetchall()
|
||||||
|
except sqlite3.OperationalError:
|
||||||
|
return []
|
||||||
|
finally:
|
||||||
|
connection.close()
|
||||||
|
records = [_decode(row["record_json"]) for row in rows]
|
||||||
|
if executor:
|
||||||
|
normalized = str(executor).strip().lower()
|
||||||
|
records = [item for item in records if item.manifest.executor.lower() == normalized]
|
||||||
|
return records
|
||||||
@@ -0,0 +1,353 @@
|
|||||||
|
"""HTTP/JSON gateway for the AIOS V2 control plane.
|
||||||
|
|
||||||
|
The public gateway owns user-facing control operations only. Runtime lifecycle
|
||||||
|
transitions are deliberately absent: external callers cannot directly move a
|
||||||
|
task to RUNNING, COMPLETED or FAILED. Those transitions are available only
|
||||||
|
through the provenance-protected orchestrator and internal reconciliation API.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import urllib.parse
|
||||||
|
from enum import Enum
|
||||||
|
from http import HTTPStatus
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
from aios_hermes import HermesConflict, HermesNotFound
|
||||||
|
|
||||||
|
from .control_plane import (
|
||||||
|
ControlPlaneError,
|
||||||
|
ControlPlaneService,
|
||||||
|
IdempotencyConflict,
|
||||||
|
InvalidTransition,
|
||||||
|
NotFound,
|
||||||
|
PolicyDenied,
|
||||||
|
RequestContext,
|
||||||
|
RevisionConflict,
|
||||||
|
TaskState,
|
||||||
|
TenantIsolationError,
|
||||||
|
Unauthorized,
|
||||||
|
)
|
||||||
|
from .dispatch_store import DispatchConflict, DispatchNotFound
|
||||||
|
from .orchestrator import OrchestrationError
|
||||||
|
from .provenance import ProvenanceDenied, ProvenanceReplay
|
||||||
|
|
||||||
|
TENANT_HEADER = "X-AIOS-Tenant-ID"
|
||||||
|
PRINCIPAL_HEADER = "X-AIOS-Principal-ID"
|
||||||
|
ROLES_HEADER = "X-AIOS-Roles"
|
||||||
|
TRACE_HEADER = "X-AIOS-Trace-ID"
|
||||||
|
SESSION_HEADER = "X-AIOS-Session-ID"
|
||||||
|
POLICY_HEADER = "X-AIOS-Policy-Version"
|
||||||
|
AUTHENTICATED_HEADER = "X-AIOS-Authenticated"
|
||||||
|
|
||||||
|
_RESOURCE_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{0,191}$")
|
||||||
|
|
||||||
|
|
||||||
|
def _json_value(value: Any) -> Any:
|
||||||
|
if dataclasses.is_dataclass(value):
|
||||||
|
return {
|
||||||
|
field.name: _json_value(getattr(value, field.name))
|
||||||
|
for field in dataclasses.fields(value)
|
||||||
|
}
|
||||||
|
if isinstance(value, Enum):
|
||||||
|
return value.value
|
||||||
|
if isinstance(value, Mapping):
|
||||||
|
return {str(key): _json_value(item) for key, item in value.items()}
|
||||||
|
if isinstance(value, (list, tuple, set, frozenset)):
|
||||||
|
return [_json_value(item) for item in value]
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _require_mapping(value: Any, *, label: str = "JSON body") -> Mapping[str, Any]:
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise ValueError(f"{label} must be an object")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _require_resource_id(value: Any, *, label: str) -> str:
|
||||||
|
normalized = str(value or "").strip()
|
||||||
|
if not _RESOURCE_ID_RE.fullmatch(normalized):
|
||||||
|
raise ValueError(f"invalid {label}")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
class GatewayApplication:
|
||||||
|
"""Pure user-facing request router that can be tested without a socket."""
|
||||||
|
|
||||||
|
def __init__(self, service: ControlPlaneService | None = None) -> None:
|
||||||
|
self.service = service or ControlPlaneService()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def context_from_headers(headers: Mapping[str, str]) -> RequestContext:
|
||||||
|
normalized_headers = {
|
||||||
|
str(key).strip().lower(): value for key, value in headers.items()
|
||||||
|
}
|
||||||
|
|
||||||
|
def header(name: str, default: str = "") -> str:
|
||||||
|
return str(normalized_headers.get(name.lower(), default))
|
||||||
|
|
||||||
|
authenticated = header(AUTHENTICATED_HEADER).strip().lower()
|
||||||
|
if authenticated not in {"1", "true", "yes"}:
|
||||||
|
raise Unauthorized("trusted authentication assertion is required")
|
||||||
|
roles = [
|
||||||
|
role.strip()
|
||||||
|
for role in header(ROLES_HEADER).split(",")
|
||||||
|
if role.strip()
|
||||||
|
]
|
||||||
|
return RequestContext.create(
|
||||||
|
tenant_id=header(TENANT_HEADER),
|
||||||
|
principal_id=header(PRINCIPAL_HEADER),
|
||||||
|
trace_id=header(TRACE_HEADER).strip() or None,
|
||||||
|
session_id=header(SESSION_HEADER).strip() or None,
|
||||||
|
policy_version=header(POLICY_HEADER, "v1").strip() or "v1",
|
||||||
|
roles=roles,
|
||||||
|
)
|
||||||
|
|
||||||
|
def handle(
|
||||||
|
self,
|
||||||
|
method: str,
|
||||||
|
raw_path: str,
|
||||||
|
headers: Mapping[str, str],
|
||||||
|
body: Any = None,
|
||||||
|
) -> tuple[int, Mapping[str, Any]]:
|
||||||
|
parsed = urllib.parse.urlparse(raw_path)
|
||||||
|
path = parsed.path.rstrip("/") or "/"
|
||||||
|
query = urllib.parse.parse_qs(parsed.query, keep_blank_values=False)
|
||||||
|
method = method.upper()
|
||||||
|
|
||||||
|
if method == "GET" and path in {"/healthz", "/api/v2/health"}:
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"service": "aios-v2-control-plane",
|
||||||
|
"status": "ok",
|
||||||
|
"store": type(self.service.store).__name__,
|
||||||
|
}
|
||||||
|
|
||||||
|
context = self.context_from_headers(headers)
|
||||||
|
payload = _require_mapping(body or {})
|
||||||
|
|
||||||
|
if method == "POST" and path == "/api/v2/tenants":
|
||||||
|
tenant = self.service.register_tenant(
|
||||||
|
context,
|
||||||
|
tenant_id=_require_resource_id(payload.get("tenant_id"), label="tenant_id"),
|
||||||
|
display_name=str(payload.get("display_name", "")),
|
||||||
|
metadata=_require_mapping(payload.get("metadata", {}), label="metadata"),
|
||||||
|
)
|
||||||
|
return HTTPStatus.CREATED, {"tenant": _json_value(tenant)}
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/tenants/current":
|
||||||
|
return HTTPStatus.OK, {"tenant": _json_value(self.service.get_tenant(context))}
|
||||||
|
|
||||||
|
if method == "POST" and path == "/api/v2/agents":
|
||||||
|
agent = self.service.register_agent(
|
||||||
|
context,
|
||||||
|
agent_id=_require_resource_id(payload.get("agent_id"), label="agent_id"),
|
||||||
|
display_name=str(payload.get("display_name", "")),
|
||||||
|
runtime=str(payload.get("runtime", "")),
|
||||||
|
capabilities=payload.get("capabilities", ()),
|
||||||
|
metadata=_require_mapping(payload.get("metadata", {}), label="metadata"),
|
||||||
|
)
|
||||||
|
return HTTPStatus.CREATED, {"agent": _json_value(agent)}
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/agents":
|
||||||
|
return HTTPStatus.OK, {"agents": _json_value(self.service.list_agents(context))}
|
||||||
|
|
||||||
|
agent_match = re.fullmatch(r"/api/v2/agents/([^/]+)", path)
|
||||||
|
if method == "GET" and agent_match:
|
||||||
|
agent_id = _require_resource_id(
|
||||||
|
urllib.parse.unquote(agent_match.group(1)), label="agent_id"
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"agent": _json_value(self.service.get_agent(context, agent_id))
|
||||||
|
}
|
||||||
|
|
||||||
|
if method == "POST" and path == "/api/v2/tasks":
|
||||||
|
task = self.service.create_task(
|
||||||
|
context,
|
||||||
|
agent_id=_require_resource_id(payload.get("agent_id"), label="agent_id"),
|
||||||
|
action=str(payload.get("action", "")).strip(),
|
||||||
|
payload=_require_mapping(payload.get("payload", {}), label="payload"),
|
||||||
|
idempotency_key=(
|
||||||
|
str(payload["idempotency_key"]).strip()
|
||||||
|
if payload.get("idempotency_key") is not None
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return HTTPStatus.CREATED, {"task": _json_value(task)}
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/tasks":
|
||||||
|
tasks = self.service.list_tasks(context, states=query.get("state"))
|
||||||
|
return HTTPStatus.OK, {"tasks": _json_value(tasks)}
|
||||||
|
|
||||||
|
task_match = re.fullmatch(r"/api/v2/tasks/([^/]+)", path)
|
||||||
|
if method == "GET" and task_match:
|
||||||
|
task_id = _require_resource_id(
|
||||||
|
urllib.parse.unquote(task_match.group(1)), label="task_id"
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"task": _json_value(self.service.get_task(context, task_id))
|
||||||
|
}
|
||||||
|
|
||||||
|
bypass_match = re.fullmatch(
|
||||||
|
r"/api/v2/tasks/([^/]+)/(start|complete|fail)", path
|
||||||
|
)
|
||||||
|
if method == "POST" and bypass_match:
|
||||||
|
return HTTPStatus.FORBIDDEN, {
|
||||||
|
"error": "RuntimeLifecycleBypassDenied",
|
||||||
|
"message": (
|
||||||
|
"RUNNING, COMPLETED and FAILED transitions are internal; "
|
||||||
|
"use the provenance-protected dispatch/reconcile flow"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
transition_match = re.fullmatch(
|
||||||
|
r"/api/v2/tasks/([^/]+)/(approve|cancel)", path
|
||||||
|
)
|
||||||
|
if method == "POST" and transition_match:
|
||||||
|
task_id = _require_resource_id(
|
||||||
|
urllib.parse.unquote(transition_match.group(1)), label="task_id"
|
||||||
|
)
|
||||||
|
target = (
|
||||||
|
TaskState.APPROVED
|
||||||
|
if transition_match.group(2) == "approve"
|
||||||
|
else TaskState.CANCELLED
|
||||||
|
)
|
||||||
|
expected_revision = payload.get("expected_revision")
|
||||||
|
task = self.service.transition_task(
|
||||||
|
context,
|
||||||
|
task_id=task_id,
|
||||||
|
target_state=target,
|
||||||
|
expected_revision=(
|
||||||
|
int(expected_revision) if expected_revision is not None else None
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {"task": _json_value(task)}
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/audit":
|
||||||
|
resource_id = query.get("resource_id", [None])[0]
|
||||||
|
events = self.service.list_audit_events(
|
||||||
|
context,
|
||||||
|
resource_id=(
|
||||||
|
_require_resource_id(resource_id, label="resource_id")
|
||||||
|
if resource_id
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {"events": _json_value(events)}
|
||||||
|
|
||||||
|
return HTTPStatus.NOT_FOUND, {
|
||||||
|
"error": "not_found",
|
||||||
|
"message": "route not found",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
_ERROR_STATUS: tuple[tuple[type[BaseException], HTTPStatus], ...] = (
|
||||||
|
(Unauthorized, HTTPStatus.UNAUTHORIZED),
|
||||||
|
(PolicyDenied, HTTPStatus.FORBIDDEN),
|
||||||
|
(ProvenanceDenied, HTTPStatus.FORBIDDEN),
|
||||||
|
(TenantIsolationError, HTTPStatus.NOT_FOUND),
|
||||||
|
(NotFound, HTTPStatus.NOT_FOUND),
|
||||||
|
(DispatchNotFound, HTTPStatus.NOT_FOUND),
|
||||||
|
(HermesNotFound, HTTPStatus.NOT_FOUND),
|
||||||
|
(ProvenanceReplay, HTTPStatus.CONFLICT),
|
||||||
|
(IdempotencyConflict, HTTPStatus.CONFLICT),
|
||||||
|
(RevisionConflict, HTTPStatus.CONFLICT),
|
||||||
|
(InvalidTransition, HTTPStatus.CONFLICT),
|
||||||
|
(DispatchConflict, HTTPStatus.CONFLICT),
|
||||||
|
(HermesConflict, HTTPStatus.CONFLICT),
|
||||||
|
(OrchestrationError, HTTPStatus.CONFLICT),
|
||||||
|
(ValueError, HTTPStatus.BAD_REQUEST),
|
||||||
|
(ControlPlaneError, HTTPStatus.BAD_REQUEST),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def error_response(exc: BaseException) -> tuple[int, Mapping[str, Any]]:
|
||||||
|
for error_type, status in _ERROR_STATUS:
|
||||||
|
if isinstance(exc, error_type):
|
||||||
|
return status, {"error": error_type.__name__, "message": str(exc)}
|
||||||
|
return HTTPStatus.INTERNAL_SERVER_ERROR, {
|
||||||
|
"error": "InternalServerError",
|
||||||
|
"message": "internal server error",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class GatewayRequestHandler(BaseHTTPRequestHandler):
|
||||||
|
server_version = "AIOSV2Gateway/1.0"
|
||||||
|
max_body_bytes = 1_048_576
|
||||||
|
|
||||||
|
@property
|
||||||
|
def application(self) -> GatewayApplication:
|
||||||
|
return self.server.application # type: ignore[attr-defined]
|
||||||
|
|
||||||
|
def _read_body(self) -> Any:
|
||||||
|
length = int(self.headers.get("Content-Length", "0") or "0")
|
||||||
|
if length < 0 or length > self.max_body_bytes:
|
||||||
|
raise ValueError("request body is too large")
|
||||||
|
if length == 0:
|
||||||
|
return {}
|
||||||
|
content_type = self.headers.get("Content-Type", "").split(";", 1)[0].strip()
|
||||||
|
if content_type != "application/json":
|
||||||
|
raise ValueError("Content-Type must be application/json")
|
||||||
|
return json.loads(self.rfile.read(length).decode("utf-8"))
|
||||||
|
|
||||||
|
def _send_json(self, status: int, payload: Mapping[str, Any]) -> None:
|
||||||
|
encoded = json.dumps(
|
||||||
|
_json_value(payload), ensure_ascii=False, separators=(",", ":")
|
||||||
|
).encode("utf-8")
|
||||||
|
self.send_response(int(status))
|
||||||
|
self.send_header("Content-Type", "application/json; charset=utf-8")
|
||||||
|
self.send_header("Content-Length", str(len(encoded)))
|
||||||
|
self.send_header("Cache-Control", "no-store")
|
||||||
|
self.send_header("X-Content-Type-Options", "nosniff")
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(encoded)
|
||||||
|
|
||||||
|
def _dispatch(self, method: str) -> None:
|
||||||
|
try:
|
||||||
|
body = self._read_body() if method in {"POST", "PUT", "PATCH"} else {}
|
||||||
|
status, payload = self.application.handle(
|
||||||
|
method,
|
||||||
|
self.path,
|
||||||
|
{key: value for key, value in self.headers.items()},
|
||||||
|
body,
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
status, payload = error_response(exc)
|
||||||
|
self._send_json(status, payload)
|
||||||
|
|
||||||
|
def do_GET(self) -> None: # noqa: N802
|
||||||
|
self._dispatch("GET")
|
||||||
|
|
||||||
|
def do_POST(self) -> None: # noqa: N802
|
||||||
|
self._dispatch("POST")
|
||||||
|
|
||||||
|
def log_message(self, format: str, *args: Any) -> None:
|
||||||
|
super().log_message(format, *args)
|
||||||
|
|
||||||
|
|
||||||
|
class GatewayHTTPServer(ThreadingHTTPServer):
|
||||||
|
daemon_threads = True
|
||||||
|
allow_reuse_address = True
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
server_address: tuple[str, int],
|
||||||
|
application: GatewayApplication,
|
||||||
|
) -> None:
|
||||||
|
self.application = application
|
||||||
|
super().__init__(server_address, GatewayRequestHandler)
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
host = os.getenv("AIOS_V2_GATEWAY_HOST", "127.0.0.1")
|
||||||
|
port = int(os.getenv("AIOS_V2_GATEWAY_PORT", "18920"))
|
||||||
|
server = GatewayHTTPServer((host, port), GatewayApplication())
|
||||||
|
print(f"AIOS V2 control-plane gateway listening on {host}:{port}")
|
||||||
|
server.serve_forever()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,123 @@
|
|||||||
|
"""Sanitized GitOps runtime status for the AIOS Web control plane."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
from dataclasses import dataclass, asdict
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class GitOpsStatus:
|
||||||
|
available: bool
|
||||||
|
service_status: str
|
||||||
|
workspace_mode: str
|
||||||
|
workspace_branch: str | None
|
||||||
|
workspace_ref: str | None
|
||||||
|
workspace_dirty: bool | None
|
||||||
|
candidate_branch: str
|
||||||
|
release_channel: str
|
||||||
|
release_version: str
|
||||||
|
sync_status: str
|
||||||
|
details: Mapping[str, Any]
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return asdict(self)
|
||||||
|
|
||||||
|
|
||||||
|
def _run_git(worktree: Path, *args: str) -> tuple[int, str, str]:
|
||||||
|
process = subprocess.run(
|
||||||
|
["git", "-C", str(worktree), *args],
|
||||||
|
text=True,
|
||||||
|
capture_output=True,
|
||||||
|
timeout=5,
|
||||||
|
)
|
||||||
|
return process.returncode, process.stdout.strip(), process.stderr.strip()
|
||||||
|
|
||||||
|
|
||||||
|
def collect_gitops_status() -> GitOpsStatus:
|
||||||
|
"""Return only non-sensitive Git/workspace metadata.
|
||||||
|
|
||||||
|
Remote URLs, environment values, credentials, filesystem contents and
|
||||||
|
command stderr are deliberately excluded from the Web response.
|
||||||
|
"""
|
||||||
|
|
||||||
|
candidate_branch = os.getenv(
|
||||||
|
"AIOS_CANDIDATE_BRANCH",
|
||||||
|
"integration/aios-v2r1-cto-production-candidate-20260720",
|
||||||
|
)
|
||||||
|
release_channel = os.getenv("AIOS_RELEASE_CHANNEL", "candidate")
|
||||||
|
release_version = os.getenv("AIOS_RELEASE_VERSION", "V2R1C0-candidate")
|
||||||
|
sync_status = os.getenv("AIOS_GITOPS_SYNC_STATUS", "unknown")
|
||||||
|
worktree = Path(
|
||||||
|
os.getenv(
|
||||||
|
"AIOS_GITOPS_CONTROLLER_WORKTREE",
|
||||||
|
"/data/aios-gitops/worktrees/controller",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
if not worktree.is_dir():
|
||||||
|
return GitOpsStatus(
|
||||||
|
available=False,
|
||||||
|
service_status="unknown",
|
||||||
|
workspace_mode="unavailable",
|
||||||
|
workspace_branch=None,
|
||||||
|
workspace_ref=None,
|
||||||
|
workspace_dirty=None,
|
||||||
|
candidate_branch=candidate_branch,
|
||||||
|
release_channel=release_channel,
|
||||||
|
release_version=release_version,
|
||||||
|
sync_status=sync_status,
|
||||||
|
details={"reason": "controller_worktree_unavailable"},
|
||||||
|
)
|
||||||
|
|
||||||
|
inside_code, inside, _ = _run_git(worktree, "rev-parse", "--is-inside-work-tree")
|
||||||
|
if inside_code != 0 or inside != "true":
|
||||||
|
return GitOpsStatus(
|
||||||
|
available=False,
|
||||||
|
service_status="unknown",
|
||||||
|
workspace_mode="invalid",
|
||||||
|
workspace_branch=None,
|
||||||
|
workspace_ref=None,
|
||||||
|
workspace_dirty=None,
|
||||||
|
candidate_branch=candidate_branch,
|
||||||
|
release_channel=release_channel,
|
||||||
|
release_version=release_version,
|
||||||
|
sync_status=sync_status,
|
||||||
|
details={"reason": "controller_worktree_invalid"},
|
||||||
|
)
|
||||||
|
|
||||||
|
branch_code, branch, _ = _run_git(worktree, "branch", "--show-current")
|
||||||
|
ref_code, ref_value, _ = _run_git(worktree, "rev-parse", "HEAD")
|
||||||
|
dirty_code, dirty_value, _ = _run_git(worktree, "status", "--porcelain=v1")
|
||||||
|
|
||||||
|
service_status = "unknown"
|
||||||
|
try:
|
||||||
|
process = subprocess.run(
|
||||||
|
["systemctl", "is-active", "aios-gitops-model-api"],
|
||||||
|
text=True,
|
||||||
|
capture_output=True,
|
||||||
|
timeout=3,
|
||||||
|
)
|
||||||
|
service_status = process.stdout.strip() or "inactive"
|
||||||
|
except (FileNotFoundError, subprocess.TimeoutExpired):
|
||||||
|
service_status = "unavailable"
|
||||||
|
|
||||||
|
branch_value = branch if branch_code == 0 and branch else None
|
||||||
|
return GitOpsStatus(
|
||||||
|
available=True,
|
||||||
|
service_status=service_status,
|
||||||
|
workspace_mode="branch" if branch_value else "detached",
|
||||||
|
workspace_branch=branch_value,
|
||||||
|
workspace_ref=ref_value if ref_code == 0 else None,
|
||||||
|
workspace_dirty=bool(dirty_value) if dirty_code == 0 else None,
|
||||||
|
candidate_branch=candidate_branch,
|
||||||
|
release_channel=release_channel,
|
||||||
|
release_version=release_version,
|
||||||
|
sync_status=sync_status,
|
||||||
|
details={
|
||||||
|
"controller_clean": dirty_code == 0 and not dirty_value,
|
||||||
|
"controller_detached": branch_code == 0 and not branch,
|
||||||
|
},
|
||||||
|
)
|
||||||
@@ -0,0 +1,238 @@
|
|||||||
|
"""Official AIOS executor providers built on the portable provider protocol.
|
||||||
|
|
||||||
|
The official providers contain no lifecycle implementation. They validate their
|
||||||
|
manifests, resolve a generic ``UnifiedExecutorWorker`` from runtime-supplied
|
||||||
|
boundaries and expose the same adapter contract. Future providers can use
|
||||||
|
``ConfiguredExecutorProvider`` directly without changing AIOS core modules.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any, Mapping, Sequence
|
||||||
|
|
||||||
|
from .execution_plan import ExecutionPlanCoordinator, ExecutionStageManifest
|
||||||
|
from .executor_protocol import (
|
||||||
|
EXECUTOR_PROTOCOL_VERSION,
|
||||||
|
ExternalExecutorAdapter,
|
||||||
|
ExecutorProviderDescriptor,
|
||||||
|
normalize_executor,
|
||||||
|
)
|
||||||
|
from .executor_worker import NotificationSender, StepRunner, UnifiedExecutorWorker
|
||||||
|
from .external_task_control import ExternalTaskCoordinator
|
||||||
|
from .task_execution_registry import ExecutionTaskManifest, ExecutionTruthRecord
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ProviderRuntimeConfiguration:
|
||||||
|
plan_id: str
|
||||||
|
executor_id: str
|
||||||
|
plan_coordinator: ExecutionPlanCoordinator
|
||||||
|
task_coordinator: ExternalTaskCoordinator
|
||||||
|
providers: Any
|
||||||
|
step_runner: StepRunner
|
||||||
|
notification_sender: NotificationSender
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_mapping(cls, value: Mapping[str, Any]) -> "ProviderRuntimeConfiguration":
|
||||||
|
required = (
|
||||||
|
"plan_id",
|
||||||
|
"executor_id",
|
||||||
|
"plan_coordinator",
|
||||||
|
"task_coordinator",
|
||||||
|
"providers",
|
||||||
|
"step_runner",
|
||||||
|
"notification_sender",
|
||||||
|
)
|
||||||
|
missing = [name for name in required if value.get(name) is None]
|
||||||
|
if missing:
|
||||||
|
raise ValueError(f"provider runtime configuration is missing: {missing}")
|
||||||
|
return cls(**{name: value[name] for name in required})
|
||||||
|
|
||||||
|
|
||||||
|
class ConfiguredExecutorAdapter:
|
||||||
|
"""Generic adapter produced by every official or third-party provider."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
descriptor: ExecutorProviderDescriptor,
|
||||||
|
configuration: ProviderRuntimeConfiguration,
|
||||||
|
) -> None:
|
||||||
|
self.descriptor = descriptor
|
||||||
|
self.configuration = configuration
|
||||||
|
self.worker = UnifiedExecutorWorker(
|
||||||
|
plan_coordinator=configuration.plan_coordinator,
|
||||||
|
task_coordinator=configuration.task_coordinator,
|
||||||
|
providers=configuration.providers,
|
||||||
|
executor_type=descriptor.executor_type,
|
||||||
|
executor_id=configuration.executor_id,
|
||||||
|
step_runner=configuration.step_runner,
|
||||||
|
notification_sender=configuration.notification_sender,
|
||||||
|
)
|
||||||
|
|
||||||
|
def discover(self) -> Sequence[ExecutionTruthRecord]:
|
||||||
|
ready = self.configuration.plan_coordinator.discover_ready(
|
||||||
|
plan_id=self.configuration.plan_id,
|
||||||
|
executor_type=self.descriptor.executor_type,
|
||||||
|
)
|
||||||
|
return tuple(item["record"] for item in ready)
|
||||||
|
|
||||||
|
def execute(self, manifest: ExecutionTaskManifest) -> Mapping[str, Any]:
|
||||||
|
if normalize_executor(manifest.executor) != self.descriptor.executor_type:
|
||||||
|
raise PermissionError("manifest executor does not match adapter provider")
|
||||||
|
plan = self.configuration.plan_coordinator.plan_store.get(
|
||||||
|
self.configuration.plan_id
|
||||||
|
)
|
||||||
|
stage: ExecutionStageManifest | None = None
|
||||||
|
for item in plan.stages:
|
||||||
|
if item.task_id == manifest.task_id:
|
||||||
|
stage = item
|
||||||
|
break
|
||||||
|
if stage is None:
|
||||||
|
raise KeyError("manifest task is not part of configured execution plan")
|
||||||
|
return self.worker.run_stage(
|
||||||
|
plan_id=plan.plan_id,
|
||||||
|
stage_id=stage.stage_id,
|
||||||
|
).as_dict()
|
||||||
|
|
||||||
|
def notify_owner(self, payload: Mapping[str, Any]) -> str:
|
||||||
|
# External callers normally use the durable NotificationOutbox. This
|
||||||
|
# method remains part of the portable adapter contract for providers
|
||||||
|
# that need to send a standalone operational notice.
|
||||||
|
from .external_task_control import NotificationRecord
|
||||||
|
from .task_execution_registry import utc_now
|
||||||
|
|
||||||
|
notification = NotificationRecord(
|
||||||
|
notification_id="provider_direct_notice",
|
||||||
|
task_id=str(payload.get("task_id") or "provider-notice"),
|
||||||
|
event_type=str(payload.get("event_type") or "INFO"),
|
||||||
|
target_ref=str(
|
||||||
|
payload.get("target_ref") or "AIOS_OWNER_WECHAT_TARGET_REF"
|
||||||
|
),
|
||||||
|
payload=dict(payload),
|
||||||
|
status="PENDING",
|
||||||
|
created_at=utc_now(),
|
||||||
|
)
|
||||||
|
return self.configuration.notification_sender.send(notification)
|
||||||
|
|
||||||
|
|
||||||
|
class ConfiguredExecutorProvider:
|
||||||
|
"""Reusable provider implementation for current and future executors."""
|
||||||
|
|
||||||
|
descriptor: ExecutorProviderDescriptor
|
||||||
|
|
||||||
|
def __init__(self, descriptor: ExecutorProviderDescriptor) -> None:
|
||||||
|
self.descriptor = descriptor
|
||||||
|
|
||||||
|
def validate_manifest(self, manifest: ExecutionTaskManifest) -> None:
|
||||||
|
if normalize_executor(manifest.executor) != self.descriptor.executor_type:
|
||||||
|
raise ValueError("manifest executor does not match provider")
|
||||||
|
if manifest.manifest_version not in {
|
||||||
|
"aios.external-task.v1",
|
||||||
|
"aios.external-task.v2",
|
||||||
|
}:
|
||||||
|
raise ValueError("unsupported external task manifest version")
|
||||||
|
if not manifest.steps:
|
||||||
|
raise ValueError("executor manifest has no steps")
|
||||||
|
|
||||||
|
def build_adapter(self, configuration: Mapping[str, Any]) -> ExternalExecutorAdapter:
|
||||||
|
return ConfiguredExecutorAdapter(
|
||||||
|
self.descriptor,
|
||||||
|
ProviderRuntimeConfiguration.from_mapping(configuration),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class OpenClawProvider(ConfiguredExecutorProvider):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__(
|
||||||
|
ExecutorProviderDescriptor(
|
||||||
|
executor_type="openclaw",
|
||||||
|
adapter_version="2.1.0",
|
||||||
|
protocol_version=EXECUTOR_PROTOCOL_VERSION,
|
||||||
|
trust_roles=("runtime", "openclaw_executor"),
|
||||||
|
capabilities=(
|
||||||
|
"deploy",
|
||||||
|
"http_runtime",
|
||||||
|
"owner_notification",
|
||||||
|
"healthcheck",
|
||||||
|
"rollback",
|
||||||
|
),
|
||||||
|
configuration_schema="aios.executor.openclaw.config.v1",
|
||||||
|
metadata={"official": True},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class HermesProvider(ConfiguredExecutorProvider):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__(
|
||||||
|
ExecutorProviderDescriptor(
|
||||||
|
executor_type="hermes",
|
||||||
|
adapter_version="2.1.0",
|
||||||
|
protocol_version=EXECUTOR_PROTOCOL_VERSION,
|
||||||
|
trust_roles=("runtime", "hermes_executor"),
|
||||||
|
capabilities=(
|
||||||
|
"queue",
|
||||||
|
"lease",
|
||||||
|
"worker",
|
||||||
|
"runtime_validation",
|
||||||
|
"restart_recovery",
|
||||||
|
),
|
||||||
|
configuration_schema="aios.executor.hermes.config.v1",
|
||||||
|
metadata={"official": True},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class CodexProvider(ConfiguredExecutorProvider):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__(
|
||||||
|
ExecutorProviderDescriptor(
|
||||||
|
executor_type="codex",
|
||||||
|
adapter_version="2.1.0",
|
||||||
|
protocol_version=EXECUTOR_PROTOCOL_VERSION,
|
||||||
|
trust_roles=("codex", "codex_executor"),
|
||||||
|
capabilities=(
|
||||||
|
"code",
|
||||||
|
"test",
|
||||||
|
"docs",
|
||||||
|
"gitops_patch",
|
||||||
|
"security_review",
|
||||||
|
),
|
||||||
|
configuration_schema="aios.executor.codex.config.v1",
|
||||||
|
metadata={"official": True},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ChatGPTSupervisorProvider(ConfiguredExecutorProvider):
|
||||||
|
"""Explicit takeover provider; it does not inherit Codex permissions."""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__(
|
||||||
|
ExecutorProviderDescriptor(
|
||||||
|
executor_type="chatgpt-supervisor",
|
||||||
|
adapter_version="2.1.0",
|
||||||
|
protocol_version=EXECUTOR_PROTOCOL_VERSION,
|
||||||
|
trust_roles=("chatgpt_supervisor",),
|
||||||
|
capabilities=(
|
||||||
|
"acceptance",
|
||||||
|
"deploy",
|
||||||
|
"evidence",
|
||||||
|
"healthcheck",
|
||||||
|
"rollback",
|
||||||
|
"wechat_simulation",
|
||||||
|
),
|
||||||
|
configuration_schema="aios.executor.chatgpt-supervisor.config.v1",
|
||||||
|
metadata={"official": True, "production_mutation_allowed": False},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def official_provider_factories() -> Mapping[str, ConfiguredExecutorProvider]:
|
||||||
|
providers = (
|
||||||
|
OpenClawProvider(),
|
||||||
|
HermesProvider(),
|
||||||
|
CodexProvider(),
|
||||||
|
ChatGPTSupervisorProvider(),
|
||||||
|
)
|
||||||
|
return {item.descriptor.executor_type: item for item in providers}
|
||||||
@@ -0,0 +1,599 @@
|
|||||||
|
"""AIOS V2 execution orchestrator.
|
||||||
|
|
||||||
|
The orchestrator is the only component allowed to translate an approved AIOS
|
||||||
|
Task into an OpenClaw command or Hermes job. Every dispatch is persisted before
|
||||||
|
Runtime side effects. Runtime results must bind to the exact dispatch,
|
||||||
|
command/job, tenant, task, trace, principal and provenance context.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any, Mapping, Protocol
|
||||||
|
|
||||||
|
from aios_hermes import HermesAdapter, HermesJob, HermesJobState, HermesResult
|
||||||
|
from aios_openclaw import (
|
||||||
|
CommandState,
|
||||||
|
OpenClawAdapter,
|
||||||
|
OpenClawCommand,
|
||||||
|
OpenClawResult,
|
||||||
|
sign_command,
|
||||||
|
)
|
||||||
|
|
||||||
|
from .control_plane import (
|
||||||
|
AgentRecord,
|
||||||
|
ControlPlaneService,
|
||||||
|
RequestContext,
|
||||||
|
TaskRecord,
|
||||||
|
TaskState,
|
||||||
|
)
|
||||||
|
from .dispatch_store import (
|
||||||
|
DispatchConflict,
|
||||||
|
DispatchRecord,
|
||||||
|
DispatchRepository,
|
||||||
|
DispatchState,
|
||||||
|
InMemoryDispatchRepository,
|
||||||
|
TERMINAL_DISPATCH_STATES,
|
||||||
|
)
|
||||||
|
from .provenance import ProvenanceDenied, digest_payload
|
||||||
|
|
||||||
|
|
||||||
|
class RuntimeAuthorizationWriter(Protocol):
|
||||||
|
def authorize_runtime_execute(self, **values: Any) -> Any: ...
|
||||||
|
|
||||||
|
|
||||||
|
class OrchestrationError(RuntimeError):
|
||||||
|
"""AIOS failed to dispatch or reconcile a runtime execution."""
|
||||||
|
|
||||||
|
|
||||||
|
class ExecutionOrchestrator:
|
||||||
|
"""Dispatch approved AIOS tasks and strictly reconcile Runtime results."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
control_plane: ControlPlaneService,
|
||||||
|
*,
|
||||||
|
openclaw: OpenClawAdapter | None = None,
|
||||||
|
hermes: HermesAdapter | None = None,
|
||||||
|
openclaw_signing_material_resolver=None,
|
||||||
|
openclaw_key_id: str = "default",
|
||||||
|
default_cell_id: str = "default",
|
||||||
|
default_profile_id: str = "default",
|
||||||
|
provenance_writer: RuntimeAuthorizationWriter | None = None,
|
||||||
|
require_provenance: bool = False,
|
||||||
|
dispatch_repository: DispatchRepository | None = None,
|
||||||
|
) -> None:
|
||||||
|
self.control_plane = control_plane
|
||||||
|
self.openclaw = openclaw
|
||||||
|
self.hermes = hermes
|
||||||
|
self.openclaw_signing_material_resolver = openclaw_signing_material_resolver
|
||||||
|
self.openclaw_key_id = str(openclaw_key_id or "").strip() or "default"
|
||||||
|
self.default_cell_id = default_cell_id
|
||||||
|
self.default_profile_id = default_profile_id
|
||||||
|
self.provenance_writer = provenance_writer
|
||||||
|
self.require_provenance = bool(require_provenance)
|
||||||
|
self.dispatch_repository = dispatch_repository or InMemoryDispatchRepository()
|
||||||
|
if self.require_provenance and self.provenance_writer is None:
|
||||||
|
raise ValueError("production orchestration requires a provenance writer")
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _record_provenance(value: str | None, *, required: bool, label: str) -> str:
|
||||||
|
normalized = str(value or "").strip()
|
||||||
|
if required and not normalized:
|
||||||
|
raise OrchestrationError(f"production dispatch requires {label}")
|
||||||
|
return normalized or f"{label}-not-required"
|
||||||
|
|
||||||
|
def _authorize_dispatch(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
*,
|
||||||
|
task: TaskRecord,
|
||||||
|
agent: AgentRecord,
|
||||||
|
provenance_id: str | None,
|
||||||
|
run_id: str | None,
|
||||||
|
timeout_seconds: float | None,
|
||||||
|
) -> str:
|
||||||
|
if not self.require_provenance:
|
||||||
|
return "authorization-not-required"
|
||||||
|
if self.provenance_writer is None:
|
||||||
|
raise OrchestrationError("Run Provenance writer is not configured")
|
||||||
|
if not provenance_id or not run_id:
|
||||||
|
raise OrchestrationError(
|
||||||
|
"production dispatch requires provenance_id and run_id"
|
||||||
|
)
|
||||||
|
request_digest = digest_payload(
|
||||||
|
{
|
||||||
|
"operation": "dispatch",
|
||||||
|
"tenant_id": task.tenant_id,
|
||||||
|
"task_id": task.task_id,
|
||||||
|
"agent_id": agent.agent_id,
|
||||||
|
"principal_id": context.principal_id,
|
||||||
|
"runtime": agent.runtime.strip().lower(),
|
||||||
|
"action": task.action,
|
||||||
|
"payload": task.payload,
|
||||||
|
"trace_id": context.trace_id,
|
||||||
|
"timeout_seconds": timeout_seconds,
|
||||||
|
"task_revision": task.revision,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
authorization = self.provenance_writer.authorize_runtime_execute(
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
tenant_id=task.tenant_id,
|
||||||
|
task_id=task.task_id,
|
||||||
|
agent_id=agent.agent_id,
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
request_digest=request_digest,
|
||||||
|
)
|
||||||
|
return authorization.authorization_id
|
||||||
|
except ProvenanceDenied as exc:
|
||||||
|
raise OrchestrationError(f"Run Provenance denied dispatch: {exc}") from exc
|
||||||
|
|
||||||
|
def dispatch(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
*,
|
||||||
|
task_id: str,
|
||||||
|
timeout_seconds: float | None = None,
|
||||||
|
provenance_id: str | None = None,
|
||||||
|
run_id: str | None = None,
|
||||||
|
) -> DispatchRecord:
|
||||||
|
task = self.control_plane.get_task(context, task_id)
|
||||||
|
agent = self.control_plane.get_agent(context, task.agent_id)
|
||||||
|
persisted_provenance = self._record_provenance(
|
||||||
|
provenance_id, required=self.require_provenance, label="provenance_id"
|
||||||
|
)
|
||||||
|
persisted_run = self._record_provenance(
|
||||||
|
run_id, required=self.require_provenance, label="run_id"
|
||||||
|
)
|
||||||
|
|
||||||
|
existing = self.dispatch_repository.get_by_task(
|
||||||
|
task.task_id, tenant_id=task.tenant_id
|
||||||
|
)
|
||||||
|
if existing is not None:
|
||||||
|
if (
|
||||||
|
existing.provenance_id != persisted_provenance
|
||||||
|
or existing.run_id != persisted_run
|
||||||
|
or existing.principal_id != context.principal_id
|
||||||
|
):
|
||||||
|
raise OrchestrationError(
|
||||||
|
"existing dispatch belongs to another provenance or principal context"
|
||||||
|
)
|
||||||
|
return existing
|
||||||
|
|
||||||
|
if task.state != TaskState.APPROVED:
|
||||||
|
raise OrchestrationError("only APPROVED tasks may be dispatched")
|
||||||
|
|
||||||
|
authorization_id = self._authorize_dispatch(
|
||||||
|
context,
|
||||||
|
task=task,
|
||||||
|
agent=agent,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
timeout_seconds=timeout_seconds,
|
||||||
|
)
|
||||||
|
runtime = agent.runtime.strip().lower()
|
||||||
|
prepared: DispatchRecord | None = None
|
||||||
|
|
||||||
|
try:
|
||||||
|
if runtime == "openclaw":
|
||||||
|
command = self._create_openclaw_command(context, task)
|
||||||
|
prepared = self.dispatch_repository.create(
|
||||||
|
DispatchRecord.create(
|
||||||
|
tenant_id=task.tenant_id,
|
||||||
|
task_id=task.task_id,
|
||||||
|
agent_id=agent.agent_id,
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
runtime="openclaw",
|
||||||
|
runtime_execution_id=command.command_id,
|
||||||
|
trace_id=context.trace_id,
|
||||||
|
provenance_id=persisted_provenance,
|
||||||
|
run_id=persisted_run,
|
||||||
|
authorization_id=authorization_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
self.control_plane.transition_task(
|
||||||
|
context,
|
||||||
|
task_id=task.task_id,
|
||||||
|
target_state=TaskState.RUNNING,
|
||||||
|
expected_revision=task.revision,
|
||||||
|
)
|
||||||
|
self.dispatch_repository.update(
|
||||||
|
prepared.dispatch_id,
|
||||||
|
tenant_id=prepared.tenant_id,
|
||||||
|
state=DispatchState.RUNNING,
|
||||||
|
)
|
||||||
|
if self.openclaw is None:
|
||||||
|
raise OrchestrationError("OpenClaw runtime is not configured")
|
||||||
|
result = self.openclaw.execute(
|
||||||
|
command, timeout_seconds=timeout_seconds
|
||||||
|
)
|
||||||
|
self.reconcile_openclaw(
|
||||||
|
context,
|
||||||
|
result,
|
||||||
|
dispatch_id=prepared.dispatch_id,
|
||||||
|
provenance_id=persisted_provenance,
|
||||||
|
run_id=persisted_run,
|
||||||
|
)
|
||||||
|
return self.dispatch_repository.get(
|
||||||
|
prepared.dispatch_id, tenant_id=prepared.tenant_id
|
||||||
|
)
|
||||||
|
|
||||||
|
if runtime == "hermes":
|
||||||
|
if self.hermes is None:
|
||||||
|
raise OrchestrationError("Hermes runtime is not configured")
|
||||||
|
job = self._create_hermes_job(context, agent, task)
|
||||||
|
prepared = self.dispatch_repository.create(
|
||||||
|
DispatchRecord.create(
|
||||||
|
tenant_id=task.tenant_id,
|
||||||
|
task_id=task.task_id,
|
||||||
|
agent_id=agent.agent_id,
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
runtime="hermes",
|
||||||
|
runtime_execution_id=job.job_id,
|
||||||
|
trace_id=context.trace_id,
|
||||||
|
provenance_id=persisted_provenance,
|
||||||
|
run_id=persisted_run,
|
||||||
|
authorization_id=authorization_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
self.control_plane.transition_task(
|
||||||
|
context,
|
||||||
|
task_id=task.task_id,
|
||||||
|
target_state=TaskState.RUNNING,
|
||||||
|
expected_revision=task.revision,
|
||||||
|
)
|
||||||
|
queued = self.hermes.enqueue(job)
|
||||||
|
return self.dispatch_repository.update(
|
||||||
|
prepared.dispatch_id,
|
||||||
|
tenant_id=prepared.tenant_id,
|
||||||
|
state=DispatchState.QUEUED,
|
||||||
|
result_digest=digest_payload(
|
||||||
|
{
|
||||||
|
"queued_job_id": queued.job_id,
|
||||||
|
"state": queued.state.value,
|
||||||
|
"revision": queued.revision,
|
||||||
|
}
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
raise OrchestrationError(f"unsupported runtime: {agent.runtime}")
|
||||||
|
except Exception as exc:
|
||||||
|
if prepared is not None:
|
||||||
|
try:
|
||||||
|
self.dispatch_repository.update(
|
||||||
|
prepared.dispatch_id,
|
||||||
|
tenant_id=prepared.tenant_id,
|
||||||
|
state=DispatchState.FAILED,
|
||||||
|
result_digest=digest_payload(
|
||||||
|
{
|
||||||
|
"code": "dispatch_failed",
|
||||||
|
"error_type": type(exc).__name__,
|
||||||
|
"message": str(exc),
|
||||||
|
}
|
||||||
|
),
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
current = self.control_plane.get_task(context, task.task_id)
|
||||||
|
if current.state == TaskState.RUNNING:
|
||||||
|
self.control_plane.transition_task(
|
||||||
|
context,
|
||||||
|
task_id=task.task_id,
|
||||||
|
target_state=TaskState.FAILED,
|
||||||
|
expected_revision=current.revision,
|
||||||
|
error={"code": "dispatch_failed", "message": str(exc)},
|
||||||
|
)
|
||||||
|
raise
|
||||||
|
|
||||||
|
def _create_openclaw_command(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
task: TaskRecord,
|
||||||
|
) -> OpenClawCommand:
|
||||||
|
if self.openclaw_signing_material_resolver is None:
|
||||||
|
raise OrchestrationError("OpenClaw signing material is not configured")
|
||||||
|
key_material = self.openclaw_signing_material_resolver(
|
||||||
|
task.tenant_id,
|
||||||
|
self.openclaw_key_id,
|
||||||
|
)
|
||||||
|
command = OpenClawCommand.create(
|
||||||
|
tenant_id=task.tenant_id,
|
||||||
|
task_id=task.task_id,
|
||||||
|
trace_id=context.trace_id,
|
||||||
|
action=task.action,
|
||||||
|
payload=task.payload,
|
||||||
|
idempotency_key=task.idempotency_key or task.task_id,
|
||||||
|
key_id=self.openclaw_key_id,
|
||||||
|
)
|
||||||
|
return sign_command(command, key_material)
|
||||||
|
|
||||||
|
def _create_hermes_job(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
agent: AgentRecord,
|
||||||
|
task: TaskRecord,
|
||||||
|
) -> HermesJob:
|
||||||
|
metadata = dict(agent.metadata)
|
||||||
|
return HermesJob.create(
|
||||||
|
tenant_id=task.tenant_id,
|
||||||
|
cell_id=str(metadata.get("cell_id") or self.default_cell_id),
|
||||||
|
profile_id=str(metadata.get("profile_id") or self.default_profile_id),
|
||||||
|
task_id=task.task_id,
|
||||||
|
trace_id=context.trace_id,
|
||||||
|
action=task.action,
|
||||||
|
payload=task.payload,
|
||||||
|
idempotency_key=task.idempotency_key or task.task_id,
|
||||||
|
priority=int(metadata.get("priority", 100)),
|
||||||
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _assert_dispatch_context(
|
||||||
|
dispatch: DispatchRecord,
|
||||||
|
*,
|
||||||
|
runtime: str,
|
||||||
|
runtime_execution_id: str,
|
||||||
|
tenant_id: str,
|
||||||
|
task_id: str,
|
||||||
|
trace_id: str,
|
||||||
|
principal_id: str,
|
||||||
|
provenance_id: str,
|
||||||
|
run_id: str,
|
||||||
|
) -> None:
|
||||||
|
expected = {
|
||||||
|
"runtime": runtime,
|
||||||
|
"runtime_execution_id": runtime_execution_id,
|
||||||
|
"tenant_id": tenant_id,
|
||||||
|
"task_id": task_id,
|
||||||
|
"trace_id": trace_id,
|
||||||
|
"principal_id": principal_id,
|
||||||
|
"provenance_id": provenance_id,
|
||||||
|
"run_id": run_id,
|
||||||
|
}
|
||||||
|
for field_name, expected_value in expected.items():
|
||||||
|
if getattr(dispatch, field_name) != expected_value:
|
||||||
|
raise OrchestrationError(f"dispatch {field_name} mismatch")
|
||||||
|
|
||||||
|
def hermes_lifecycle_context(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
*,
|
||||||
|
dispatch_id: str,
|
||||||
|
job_id: str,
|
||||||
|
tenant_id: str,
|
||||||
|
task_id: str,
|
||||||
|
trace_id: str,
|
||||||
|
principal_id: str,
|
||||||
|
provenance_id: str,
|
||||||
|
run_id: str,
|
||||||
|
) -> tuple[DispatchRecord, HermesJob]:
|
||||||
|
"""Resolve a runtime lifecycle request against its durable dispatch."""
|
||||||
|
dispatch = self.dispatch_repository.get(
|
||||||
|
dispatch_id, tenant_id=context.tenant_id
|
||||||
|
)
|
||||||
|
self._assert_dispatch_context(
|
||||||
|
dispatch,
|
||||||
|
runtime="hermes",
|
||||||
|
runtime_execution_id=job_id,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
task_id=task_id,
|
||||||
|
trace_id=trace_id,
|
||||||
|
principal_id=principal_id,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
)
|
||||||
|
if context.tenant_id != tenant_id:
|
||||||
|
raise OrchestrationError("runtime context tenant_id mismatch")
|
||||||
|
if context.principal_id != principal_id:
|
||||||
|
raise OrchestrationError("runtime context principal_id mismatch")
|
||||||
|
if context.trace_id != trace_id:
|
||||||
|
raise OrchestrationError("runtime context trace_id mismatch")
|
||||||
|
if self.hermes is None:
|
||||||
|
raise OrchestrationError("Hermes runtime is not configured")
|
||||||
|
job = self.hermes.get(job_id, tenant_id=tenant_id)
|
||||||
|
if job.task_id != task_id or job.trace_id != trace_id:
|
||||||
|
raise OrchestrationError("Hermes job identity mismatch")
|
||||||
|
return dispatch, job
|
||||||
|
|
||||||
|
def bind_hermes_worker(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
job: HermesJob,
|
||||||
|
*,
|
||||||
|
dispatch_id: str,
|
||||||
|
provenance_id: str,
|
||||||
|
run_id: str,
|
||||||
|
worker_id: str,
|
||||||
|
) -> DispatchRecord:
|
||||||
|
dispatch, current = self.hermes_lifecycle_context(
|
||||||
|
context,
|
||||||
|
dispatch_id=dispatch_id,
|
||||||
|
job_id=job.job_id,
|
||||||
|
tenant_id=job.tenant_id,
|
||||||
|
task_id=job.task_id,
|
||||||
|
trace_id=job.trace_id,
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
)
|
||||||
|
if current.assigned_worker_id != worker_id:
|
||||||
|
raise OrchestrationError("Hermes worker does not own claimed job")
|
||||||
|
return self.dispatch_repository.update(
|
||||||
|
dispatch.dispatch_id,
|
||||||
|
tenant_id=dispatch.tenant_id,
|
||||||
|
expected_worker_id=worker_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
def reconcile_openclaw(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
result: OpenClawResult,
|
||||||
|
*,
|
||||||
|
dispatch_id: str,
|
||||||
|
provenance_id: str,
|
||||||
|
run_id: str,
|
||||||
|
) -> TaskRecord:
|
||||||
|
dispatch = self.dispatch_repository.get(
|
||||||
|
dispatch_id, tenant_id=context.tenant_id
|
||||||
|
)
|
||||||
|
self._assert_dispatch_context(
|
||||||
|
dispatch,
|
||||||
|
runtime="openclaw",
|
||||||
|
runtime_execution_id=result.command_id,
|
||||||
|
tenant_id=result.tenant_id,
|
||||||
|
task_id=result.task_id,
|
||||||
|
trace_id=result.trace_id,
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
)
|
||||||
|
result_digest = digest_payload(
|
||||||
|
{
|
||||||
|
"command_id": result.command_id,
|
||||||
|
"state": result.state.value,
|
||||||
|
"output": result.output,
|
||||||
|
"error": result.error,
|
||||||
|
"runtime_id": result.runtime_id,
|
||||||
|
"attempts": result.attempts,
|
||||||
|
"finished_at": result.finished_at,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
if dispatch.state in TERMINAL_DISPATCH_STATES:
|
||||||
|
if dispatch.result_digest != result_digest:
|
||||||
|
raise DispatchConflict("terminal dispatch received a different result")
|
||||||
|
return self.control_plane.get_task(context, dispatch.task_id)
|
||||||
|
|
||||||
|
task = self.control_plane.get_task(context, dispatch.task_id)
|
||||||
|
if task.state != TaskState.RUNNING:
|
||||||
|
raise OrchestrationError("task is not RUNNING for active dispatch")
|
||||||
|
if result.state == CommandState.COMPLETED:
|
||||||
|
target = TaskState.COMPLETED
|
||||||
|
values = {"result": result.output or {}}
|
||||||
|
dispatch_state = DispatchState.COMPLETED
|
||||||
|
elif result.state in {
|
||||||
|
CommandState.FAILED,
|
||||||
|
CommandState.TIMED_OUT,
|
||||||
|
CommandState.REJECTED,
|
||||||
|
}:
|
||||||
|
target = TaskState.FAILED
|
||||||
|
values = {
|
||||||
|
"error": result.error or {"code": result.state.value.lower()}
|
||||||
|
}
|
||||||
|
dispatch_state = DispatchState.FAILED
|
||||||
|
elif result.state == CommandState.CANCELLED:
|
||||||
|
target = TaskState.CANCELLED
|
||||||
|
values = {}
|
||||||
|
dispatch_state = DispatchState.CANCELLED
|
||||||
|
else:
|
||||||
|
self.dispatch_repository.update(
|
||||||
|
dispatch.dispatch_id,
|
||||||
|
tenant_id=dispatch.tenant_id,
|
||||||
|
state=DispatchState.RUNNING,
|
||||||
|
result_digest=result_digest,
|
||||||
|
)
|
||||||
|
return task
|
||||||
|
|
||||||
|
updated_task = self.control_plane.transition_task(
|
||||||
|
context,
|
||||||
|
task_id=task.task_id,
|
||||||
|
target_state=target,
|
||||||
|
expected_revision=task.revision,
|
||||||
|
**values,
|
||||||
|
)
|
||||||
|
self.dispatch_repository.update(
|
||||||
|
dispatch.dispatch_id,
|
||||||
|
tenant_id=dispatch.tenant_id,
|
||||||
|
state=dispatch_state,
|
||||||
|
result_digest=result_digest,
|
||||||
|
)
|
||||||
|
return updated_task
|
||||||
|
|
||||||
|
def reconcile_hermes(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
result: HermesResult,
|
||||||
|
*,
|
||||||
|
dispatch_id: str,
|
||||||
|
provenance_id: str,
|
||||||
|
run_id: str,
|
||||||
|
) -> TaskRecord:
|
||||||
|
dispatch = self.dispatch_repository.get(
|
||||||
|
dispatch_id, tenant_id=context.tenant_id
|
||||||
|
)
|
||||||
|
self._assert_dispatch_context(
|
||||||
|
dispatch,
|
||||||
|
runtime="hermes",
|
||||||
|
runtime_execution_id=result.job_id,
|
||||||
|
tenant_id=result.tenant_id,
|
||||||
|
task_id=result.task_id,
|
||||||
|
trace_id=result.trace_id,
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
run_id=run_id,
|
||||||
|
)
|
||||||
|
if self.hermes is None:
|
||||||
|
raise OrchestrationError("Hermes runtime is not configured")
|
||||||
|
job = self.hermes.get(result.job_id, tenant_id=result.tenant_id)
|
||||||
|
if result.state in {HermesJobState.COMPLETED, HermesJobState.FAILED}:
|
||||||
|
if not result.worker_id:
|
||||||
|
raise OrchestrationError("terminal Hermes result requires worker_id")
|
||||||
|
if job.assigned_worker_id != result.worker_id:
|
||||||
|
raise OrchestrationError("Hermes result worker does not own job lease")
|
||||||
|
if (
|
||||||
|
dispatch.expected_worker_id is not None
|
||||||
|
and dispatch.expected_worker_id != result.worker_id
|
||||||
|
):
|
||||||
|
raise OrchestrationError("Hermes result worker mismatch")
|
||||||
|
result_digest = digest_payload(
|
||||||
|
{
|
||||||
|
"job_id": result.job_id,
|
||||||
|
"worker_id": result.worker_id,
|
||||||
|
"state": result.state.value,
|
||||||
|
"output": result.output,
|
||||||
|
"error": result.error,
|
||||||
|
"finished_at": result.finished_at,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
if dispatch.state in TERMINAL_DISPATCH_STATES:
|
||||||
|
if dispatch.result_digest != result_digest:
|
||||||
|
raise DispatchConflict("terminal dispatch received a different result")
|
||||||
|
return self.control_plane.get_task(context, dispatch.task_id)
|
||||||
|
|
||||||
|
task = self.control_plane.get_task(context, dispatch.task_id)
|
||||||
|
if task.state != TaskState.RUNNING:
|
||||||
|
raise OrchestrationError("task is not RUNNING for active dispatch")
|
||||||
|
if result.state == HermesJobState.COMPLETED:
|
||||||
|
target = TaskState.COMPLETED
|
||||||
|
values = {"result": result.output or {}}
|
||||||
|
dispatch_state = DispatchState.COMPLETED
|
||||||
|
elif result.state == HermesJobState.FAILED:
|
||||||
|
target = TaskState.FAILED
|
||||||
|
values = {"error": result.error or {"code": "hermes_failed"}}
|
||||||
|
dispatch_state = DispatchState.FAILED
|
||||||
|
elif result.state == HermesJobState.CANCELLED:
|
||||||
|
target = TaskState.CANCELLED
|
||||||
|
values = {}
|
||||||
|
dispatch_state = DispatchState.CANCELLED
|
||||||
|
else:
|
||||||
|
raise OrchestrationError("Hermes reconcile only accepts terminal results")
|
||||||
|
|
||||||
|
updated_task = self.control_plane.transition_task(
|
||||||
|
context,
|
||||||
|
task_id=task.task_id,
|
||||||
|
target_state=target,
|
||||||
|
expected_revision=task.revision,
|
||||||
|
**values,
|
||||||
|
)
|
||||||
|
self.dispatch_repository.update(
|
||||||
|
dispatch.dispatch_id,
|
||||||
|
tenant_id=dispatch.tenant_id,
|
||||||
|
state=dispatch_state,
|
||||||
|
expected_worker_id=result.worker_id,
|
||||||
|
result_digest=result_digest,
|
||||||
|
)
|
||||||
|
return updated_task
|
||||||
|
|
||||||
|
def dispatch_snapshot(self, *, tenant_id: str) -> Mapping[str, DispatchRecord]:
|
||||||
|
return {
|
||||||
|
item.task_id: item
|
||||||
|
for item in self.dispatch_repository.list(tenant_id=tenant_id)
|
||||||
|
}
|
||||||
@@ -0,0 +1,181 @@
|
|||||||
|
"""Durable single-process persistence for the AIOS V2 control plane.
|
||||||
|
|
||||||
|
The candidate uses an atomic SQLite snapshot so a gateway restart does not lose
|
||||||
|
tenants, agents, task state, idempotency records, or audit events. The domain
|
||||||
|
service remains storage-agnostic; a later HA backend can replace this module
|
||||||
|
without changing API or runtime-adapter contracts.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import sqlite3
|
||||||
|
import threading
|
||||||
|
from dataclasses import asdict
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Callable
|
||||||
|
|
||||||
|
from .control_plane import (
|
||||||
|
AgentRecord,
|
||||||
|
AuditEvent,
|
||||||
|
ControlPlaneService,
|
||||||
|
InMemoryControlPlaneStore,
|
||||||
|
TaskRecord,
|
||||||
|
TaskState,
|
||||||
|
TenantRecord,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class SQLiteSnapshotRepository:
|
||||||
|
"""Store one versioned control-plane snapshot in SQLite."""
|
||||||
|
|
||||||
|
SCHEMA_VERSION = 1
|
||||||
|
|
||||||
|
def __init__(self, path: str | Path) -> None:
|
||||||
|
self.path = Path(path).resolve()
|
||||||
|
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
self._initialize()
|
||||||
|
|
||||||
|
def _connect(self) -> sqlite3.Connection:
|
||||||
|
connection = sqlite3.connect(self.path, timeout=30.0)
|
||||||
|
connection.execute("PRAGMA journal_mode=WAL")
|
||||||
|
connection.execute("PRAGMA synchronous=FULL")
|
||||||
|
connection.execute("PRAGMA foreign_keys=ON")
|
||||||
|
return connection
|
||||||
|
|
||||||
|
def _initialize(self) -> None:
|
||||||
|
with self._connect() as connection:
|
||||||
|
connection.execute(
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS control_plane_snapshot (
|
||||||
|
singleton_id INTEGER PRIMARY KEY CHECK (singleton_id = 1),
|
||||||
|
schema_version INTEGER NOT NULL,
|
||||||
|
payload_json TEXT NOT NULL,
|
||||||
|
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
|
||||||
|
def load(self) -> dict[str, Any] | None:
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
row = connection.execute(
|
||||||
|
"SELECT schema_version, payload_json FROM control_plane_snapshot WHERE singleton_id = 1"
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
return None
|
||||||
|
schema_version, payload_json = row
|
||||||
|
if int(schema_version) != self.SCHEMA_VERSION:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"unsupported control-plane snapshot version: {schema_version}"
|
||||||
|
)
|
||||||
|
decoded = json.loads(str(payload_json))
|
||||||
|
if not isinstance(decoded, dict):
|
||||||
|
raise RuntimeError("invalid control-plane snapshot")
|
||||||
|
return decoded
|
||||||
|
|
||||||
|
def save(self, payload: dict[str, Any]) -> None:
|
||||||
|
encoded = json.dumps(
|
||||||
|
payload,
|
||||||
|
ensure_ascii=False,
|
||||||
|
sort_keys=True,
|
||||||
|
separators=(",", ":"),
|
||||||
|
default=str,
|
||||||
|
)
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
connection.execute("BEGIN IMMEDIATE")
|
||||||
|
connection.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO control_plane_snapshot (
|
||||||
|
singleton_id, schema_version, payload_json, updated_at
|
||||||
|
) VALUES (1, ?, ?, CURRENT_TIMESTAMP)
|
||||||
|
ON CONFLICT(singleton_id) DO UPDATE SET
|
||||||
|
schema_version = excluded.schema_version,
|
||||||
|
payload_json = excluded.payload_json,
|
||||||
|
updated_at = CURRENT_TIMESTAMP
|
||||||
|
""",
|
||||||
|
(self.SCHEMA_VERSION, encoded),
|
||||||
|
)
|
||||||
|
connection.commit()
|
||||||
|
|
||||||
|
|
||||||
|
class DurableControlPlaneService(ControlPlaneService):
|
||||||
|
"""Control-plane service that persists after every successful mutation."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
repository: SQLiteSnapshotRepository,
|
||||||
|
*,
|
||||||
|
policy_evaluator=None,
|
||||||
|
) -> None:
|
||||||
|
self.repository = repository
|
||||||
|
store = InMemoryControlPlaneStore()
|
||||||
|
self._restore(store, repository.load())
|
||||||
|
super().__init__(store, policy_evaluator=policy_evaluator)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _restore(
|
||||||
|
store: InMemoryControlPlaneStore,
|
||||||
|
snapshot: dict[str, Any] | None,
|
||||||
|
) -> None:
|
||||||
|
if not snapshot:
|
||||||
|
return
|
||||||
|
for item in snapshot.get("tenants", []):
|
||||||
|
record = TenantRecord(**item)
|
||||||
|
store.tenants[record.tenant_id] = record
|
||||||
|
for item in snapshot.get("agents", []):
|
||||||
|
normalized = dict(item)
|
||||||
|
normalized["capabilities"] = tuple(normalized.get("capabilities", ()))
|
||||||
|
record = AgentRecord(**normalized)
|
||||||
|
store.agents[record.agent_id] = record
|
||||||
|
for item in snapshot.get("tasks", []):
|
||||||
|
normalized = dict(item)
|
||||||
|
normalized["state"] = TaskState(normalized["state"])
|
||||||
|
record = TaskRecord(**normalized)
|
||||||
|
store.tasks[record.task_id] = record
|
||||||
|
store.audit_events.extend(AuditEvent(**item) for item in snapshot.get("audit", []))
|
||||||
|
for item in snapshot.get("idempotency", []):
|
||||||
|
store.idempotency[(item["tenant_id"], item["key"])] = (
|
||||||
|
item["task_id"],
|
||||||
|
item["digest"],
|
||||||
|
)
|
||||||
|
|
||||||
|
def _snapshot(self) -> dict[str, Any]:
|
||||||
|
with self.store.lock:
|
||||||
|
return {
|
||||||
|
"schema_version": self.repository.SCHEMA_VERSION,
|
||||||
|
"tenants": [asdict(item) for item in self.store.tenants.values()],
|
||||||
|
"agents": [asdict(item) for item in self.store.agents.values()],
|
||||||
|
"tasks": [
|
||||||
|
{**asdict(item), "state": item.state.value}
|
||||||
|
for item in self.store.tasks.values()
|
||||||
|
],
|
||||||
|
"audit": [asdict(item) for item in self.store.audit_events],
|
||||||
|
"idempotency": [
|
||||||
|
{
|
||||||
|
"tenant_id": tenant_id,
|
||||||
|
"key": key,
|
||||||
|
"task_id": task_id,
|
||||||
|
"digest": digest,
|
||||||
|
}
|
||||||
|
for (tenant_id, key), (task_id, digest)
|
||||||
|
in self.store.idempotency.items()
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
def _mutate(self, operation: Callable[[], Any]) -> Any:
|
||||||
|
with self.store.lock:
|
||||||
|
result = operation()
|
||||||
|
self.repository.save(self._snapshot())
|
||||||
|
return result
|
||||||
|
|
||||||
|
def register_tenant(self, *args, **kwargs):
|
||||||
|
return self._mutate(lambda: super(DurableControlPlaneService, self).register_tenant(*args, **kwargs))
|
||||||
|
|
||||||
|
def register_agent(self, *args, **kwargs):
|
||||||
|
return self._mutate(lambda: super(DurableControlPlaneService, self).register_agent(*args, **kwargs))
|
||||||
|
|
||||||
|
def create_task(self, *args, **kwargs):
|
||||||
|
return self._mutate(lambda: super(DurableControlPlaneService, self).create_task(*args, **kwargs))
|
||||||
|
|
||||||
|
def transition_task(self, *args, **kwargs):
|
||||||
|
return self._mutate(lambda: super(DurableControlPlaneService, self).transition_task(*args, **kwargs))
|
||||||
@@ -0,0 +1,831 @@
|
|||||||
|
"""AIOS V2 可信 Run Provenance 签发、持久化与受控写入。
|
||||||
|
|
||||||
|
生产策略要求 Provenance 同时覆盖七类路径:代码变更、Git 操作、部署、
|
||||||
|
Runtime 执行、Evidence 写入、回滚和审计。任何缺失、篡改、过期、撤销、
|
||||||
|
重放或上下文不匹配都默认拒绝。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sqlite3
|
||||||
|
import stat
|
||||||
|
import threading
|
||||||
|
import uuid
|
||||||
|
from dataclasses import asdict, dataclass, field, replace
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from enum import Enum
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Protocol, Sequence
|
||||||
|
|
||||||
|
PROVENANCE_VERSION = "aios.v2.run-provenance.v1"
|
||||||
|
PRODUCTION_POLICY_VERSION = "aios.v2.production.v1"
|
||||||
|
|
||||||
|
|
||||||
|
def utc_now() -> str:
|
||||||
|
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||||
|
|
||||||
|
|
||||||
|
def parse_timestamp(value: str) -> datetime:
|
||||||
|
normalized = str(value or "").strip()
|
||||||
|
if not normalized:
|
||||||
|
raise ValueError("timestamp is required")
|
||||||
|
if normalized.endswith("Z"):
|
||||||
|
normalized = normalized[:-1] + "+00:00"
|
||||||
|
parsed = datetime.fromisoformat(normalized)
|
||||||
|
if parsed.tzinfo is None:
|
||||||
|
raise ValueError("timestamp must include timezone")
|
||||||
|
return parsed.astimezone(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
def required(value: str, label: str) -> str:
|
||||||
|
normalized = str(value or "").strip()
|
||||||
|
if not normalized:
|
||||||
|
raise ValueError(f"{label} is required")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
def canonical_json_bytes(value: Mapping[str, Any]) -> bytes:
|
||||||
|
return json.dumps(
|
||||||
|
dict(value),
|
||||||
|
ensure_ascii=False,
|
||||||
|
sort_keys=True,
|
||||||
|
separators=(",", ":"),
|
||||||
|
default=str,
|
||||||
|
).encode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def digest_payload(value: Mapping[str, Any]) -> str:
|
||||||
|
return hashlib.sha256(canonical_json_bytes(value)).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_digest(value: str) -> str:
|
||||||
|
normalized = required(value, "request_digest").lower()
|
||||||
|
if len(normalized) != 64 or any(ch not in "0123456789abcdef" for ch in normalized):
|
||||||
|
raise ValueError("request_digest must be a SHA-256 hexadecimal digest")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
class ExecutionPath(str, Enum):
|
||||||
|
CODE_CHANGE = "code_change"
|
||||||
|
GIT_OPERATION = "git_operation"
|
||||||
|
DEPLOYMENT = "deployment"
|
||||||
|
RUNTIME_EXECUTE = "runtime_execute"
|
||||||
|
EVIDENCE_WRITE = "evidence_write"
|
||||||
|
ROLLBACK = "rollback"
|
||||||
|
AUDIT = "audit"
|
||||||
|
|
||||||
|
|
||||||
|
REQUIRED_PRODUCTION_PATHS = frozenset(item.value for item in ExecutionPath)
|
||||||
|
|
||||||
|
|
||||||
|
class GitOperation(str, Enum):
|
||||||
|
READ_REF = "read_ref"
|
||||||
|
FETCH = "fetch"
|
||||||
|
CREATE_BRANCH = "create_branch"
|
||||||
|
COMMIT = "commit"
|
||||||
|
MERGE = "merge"
|
||||||
|
PUSH = "push"
|
||||||
|
UPDATE_EVIDENCE = "update_evidence"
|
||||||
|
|
||||||
|
|
||||||
|
class ProvenanceStatus(str, Enum):
|
||||||
|
ISSUED = "issued"
|
||||||
|
VERIFIED = "verified"
|
||||||
|
USED = "used"
|
||||||
|
EXPIRED = "expired"
|
||||||
|
REVOKED = "revoked"
|
||||||
|
|
||||||
|
|
||||||
|
class ProvenanceError(RuntimeError):
|
||||||
|
"""Run Provenance 基础错误。"""
|
||||||
|
|
||||||
|
|
||||||
|
class ProvenanceDenied(ProvenanceError):
|
||||||
|
"""策略拒绝当前操作。"""
|
||||||
|
|
||||||
|
|
||||||
|
class ProvenanceNotFound(ProvenanceDenied):
|
||||||
|
"""未找到 Provenance。"""
|
||||||
|
|
||||||
|
|
||||||
|
class ProvenanceSignatureError(ProvenanceDenied):
|
||||||
|
"""签名无效。"""
|
||||||
|
|
||||||
|
|
||||||
|
class ProvenanceExpired(ProvenanceDenied):
|
||||||
|
"""Provenance 已过期。"""
|
||||||
|
|
||||||
|
|
||||||
|
class ProvenanceRevoked(ProvenanceDenied):
|
||||||
|
"""Provenance 已撤销。"""
|
||||||
|
|
||||||
|
|
||||||
|
class ProvenanceReplay(ProvenanceDenied):
|
||||||
|
"""检测到相同授权摘要重放。"""
|
||||||
|
|
||||||
|
|
||||||
|
class Signer(Protocol):
|
||||||
|
key_id: str
|
||||||
|
|
||||||
|
def sign(self, payload: bytes) -> str: ...
|
||||||
|
|
||||||
|
|
||||||
|
class SignatureVerifier(Protocol):
|
||||||
|
key_id: str
|
||||||
|
fingerprint: str
|
||||||
|
|
||||||
|
def verify(self, payload: bytes, signature: str) -> bool: ...
|
||||||
|
|
||||||
|
|
||||||
|
def _load_crypto() -> tuple[Any, Any, Any]:
|
||||||
|
try:
|
||||||
|
from cryptography.hazmat.primitives import serialization
|
||||||
|
from cryptography.hazmat.primitives.asymmetric.ed25519 import (
|
||||||
|
Ed25519PrivateKey,
|
||||||
|
Ed25519PublicKey,
|
||||||
|
)
|
||||||
|
except ImportError as exc: # pragma: no cover - depends on deployment image
|
||||||
|
raise RuntimeError(
|
||||||
|
"cryptography package with Ed25519 support is required"
|
||||||
|
) from exc
|
||||||
|
return serialization, Ed25519PrivateKey, Ed25519PublicKey
|
||||||
|
|
||||||
|
|
||||||
|
def _check_private_key_file(path: Path) -> bytes:
|
||||||
|
resolved = path.resolve()
|
||||||
|
metadata = resolved.stat()
|
||||||
|
if not stat.S_ISREG(metadata.st_mode):
|
||||||
|
raise RuntimeError("private key reference is not a regular file")
|
||||||
|
if metadata.st_mode & 0o077:
|
||||||
|
raise RuntimeError("private key file permissions must be 0600 or stricter")
|
||||||
|
return resolved.read_bytes()
|
||||||
|
|
||||||
|
|
||||||
|
def _check_public_key_file(path: Path) -> bytes:
|
||||||
|
resolved = path.resolve()
|
||||||
|
metadata = resolved.stat()
|
||||||
|
if not stat.S_ISREG(metadata.st_mode):
|
||||||
|
raise RuntimeError("public key reference is not a regular file")
|
||||||
|
if metadata.st_mode & 0o022:
|
||||||
|
raise RuntimeError("public key file must not be group/world writable")
|
||||||
|
return resolved.read_bytes()
|
||||||
|
|
||||||
|
|
||||||
|
class Ed25519PrivateKeySigner:
|
||||||
|
"""Ed25519 Owner/Control Plane 签发器。"""
|
||||||
|
|
||||||
|
def __init__(self, private_key: Any, *, key_id: str) -> None:
|
||||||
|
self.private_key = private_key
|
||||||
|
self.key_id = required(key_id, "key_id")
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_file(cls, path: str | Path, *, key_id: str) -> "Ed25519PrivateKeySigner":
|
||||||
|
serialization, private_cls, _ = _load_crypto()
|
||||||
|
value = _check_private_key_file(Path(path))
|
||||||
|
if value.startswith(b"-----BEGIN"):
|
||||||
|
key = serialization.load_pem_private_key(value, password=None)
|
||||||
|
elif len(value) == 32:
|
||||||
|
key = private_cls.from_private_bytes(value)
|
||||||
|
else:
|
||||||
|
raise RuntimeError("private key must be PEM or a raw 32-byte Ed25519 seed")
|
||||||
|
if not isinstance(key, private_cls):
|
||||||
|
raise RuntimeError("private key is not Ed25519")
|
||||||
|
return cls(key, key_id=key_id)
|
||||||
|
|
||||||
|
def sign(self, payload: bytes) -> str:
|
||||||
|
return base64.b64encode(self.private_key.sign(bytes(payload))).decode("ascii")
|
||||||
|
|
||||||
|
def public_verifier(self) -> "Ed25519PublicKeyVerifier":
|
||||||
|
return Ed25519PublicKeyVerifier(self.private_key.public_key(), key_id=self.key_id)
|
||||||
|
|
||||||
|
|
||||||
|
class Ed25519PublicKeyVerifier:
|
||||||
|
"""Ed25519 签名验证器。"""
|
||||||
|
|
||||||
|
def __init__(self, public_key: Any, *, key_id: str) -> None:
|
||||||
|
serialization, _, public_cls = _load_crypto()
|
||||||
|
if not isinstance(public_key, public_cls):
|
||||||
|
raise RuntimeError("public key is not Ed25519")
|
||||||
|
self.public_key = public_key
|
||||||
|
self.key_id = required(key_id, "key_id")
|
||||||
|
raw = public_key.public_bytes(
|
||||||
|
encoding=serialization.Encoding.Raw,
|
||||||
|
format=serialization.PublicFormat.Raw,
|
||||||
|
)
|
||||||
|
self.fingerprint = hashlib.sha256(raw).hexdigest()
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_file(cls, path: str | Path, *, key_id: str) -> "Ed25519PublicKeyVerifier":
|
||||||
|
serialization, _, public_cls = _load_crypto()
|
||||||
|
value = _check_public_key_file(Path(path))
|
||||||
|
if value.startswith(b"-----BEGIN"):
|
||||||
|
key = serialization.load_pem_public_key(value)
|
||||||
|
elif len(value) == 32:
|
||||||
|
key = public_cls.from_public_bytes(value)
|
||||||
|
else:
|
||||||
|
raise RuntimeError("public key must be PEM or raw 32-byte Ed25519 key")
|
||||||
|
if not isinstance(key, public_cls):
|
||||||
|
raise RuntimeError("public key is not Ed25519")
|
||||||
|
return cls(key, key_id=key_id)
|
||||||
|
|
||||||
|
def verify(self, payload: bytes, signature: str) -> bool:
|
||||||
|
try:
|
||||||
|
decoded = base64.b64decode(required(signature, "signature"), validate=True)
|
||||||
|
self.public_key.verify(decoded, bytes(payload))
|
||||||
|
return True
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class RunProvenance:
|
||||||
|
provenance_id: str
|
||||||
|
run_id: str
|
||||||
|
issuer: str
|
||||||
|
tenant_id: str
|
||||||
|
task_id: str
|
||||||
|
agent_id: str
|
||||||
|
principal_id: str
|
||||||
|
allowed_paths: Sequence[str]
|
||||||
|
allowed_git_operations: Sequence[str]
|
||||||
|
issued_at: str
|
||||||
|
expires_at: str
|
||||||
|
policy_version: str
|
||||||
|
key_id: str
|
||||||
|
nonce: str
|
||||||
|
signature: str = ""
|
||||||
|
schema_version: str = PROVENANCE_VERSION
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for name in (
|
||||||
|
"provenance_id",
|
||||||
|
"run_id",
|
||||||
|
"issuer",
|
||||||
|
"tenant_id",
|
||||||
|
"task_id",
|
||||||
|
"agent_id",
|
||||||
|
"principal_id",
|
||||||
|
"issued_at",
|
||||||
|
"expires_at",
|
||||||
|
"policy_version",
|
||||||
|
"key_id",
|
||||||
|
"nonce",
|
||||||
|
"schema_version",
|
||||||
|
):
|
||||||
|
object.__setattr__(self, name, required(getattr(self, name), name))
|
||||||
|
if self.schema_version != PROVENANCE_VERSION:
|
||||||
|
raise ValueError(f"unsupported schema_version: {self.schema_version}")
|
||||||
|
issued = parse_timestamp(self.issued_at)
|
||||||
|
expires = parse_timestamp(self.expires_at)
|
||||||
|
if expires <= issued:
|
||||||
|
raise ValueError("expires_at must be later than issued_at")
|
||||||
|
paths = tuple(sorted({ExecutionPath(item).value for item in self.allowed_paths}))
|
||||||
|
operations = tuple(
|
||||||
|
sorted({GitOperation(item).value for item in self.allowed_git_operations})
|
||||||
|
)
|
||||||
|
object.__setattr__(self, "allowed_paths", paths)
|
||||||
|
object.__setattr__(self, "allowed_git_operations", operations)
|
||||||
|
object.__setattr__(self, "signature", str(self.signature or "").strip())
|
||||||
|
|
||||||
|
def unsigned_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"schema_version": self.schema_version,
|
||||||
|
"provenance_id": self.provenance_id,
|
||||||
|
"run_id": self.run_id,
|
||||||
|
"issuer": self.issuer,
|
||||||
|
"tenant_id": self.tenant_id,
|
||||||
|
"task_id": self.task_id,
|
||||||
|
"agent_id": self.agent_id,
|
||||||
|
"principal_id": self.principal_id,
|
||||||
|
"allowed_paths": list(self.allowed_paths),
|
||||||
|
"allowed_git_operations": list(self.allowed_git_operations),
|
||||||
|
"issued_at": self.issued_at,
|
||||||
|
"expires_at": self.expires_at,
|
||||||
|
"policy_version": self.policy_version,
|
||||||
|
"key_id": self.key_id,
|
||||||
|
"nonce": self.nonce,
|
||||||
|
}
|
||||||
|
|
||||||
|
def canonical_bytes(self) -> bytes:
|
||||||
|
return canonical_json_bytes(self.unsigned_dict())
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
value = self.unsigned_dict()
|
||||||
|
value["signature"] = self.signature
|
||||||
|
return value
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, value: Mapping[str, Any]) -> "RunProvenance":
|
||||||
|
return cls(**dict(value))
|
||||||
|
|
||||||
|
|
||||||
|
class ProvenanceIssuer:
|
||||||
|
"""只由 Owner 或 AIOS Control Plane 持有的签发服务。"""
|
||||||
|
|
||||||
|
def __init__(self, signer: Signer, store: "SQLiteProvenanceStore") -> None:
|
||||||
|
self.signer = signer
|
||||||
|
self.store = store
|
||||||
|
|
||||||
|
def issue(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
run_id: str,
|
||||||
|
issuer: str,
|
||||||
|
tenant_id: str,
|
||||||
|
task_id: str,
|
||||||
|
agent_id: str,
|
||||||
|
principal_id: str,
|
||||||
|
allowed_paths: Sequence[str] = tuple(REQUIRED_PRODUCTION_PATHS),
|
||||||
|
allowed_git_operations: Sequence[str] = tuple(item.value for item in GitOperation),
|
||||||
|
ttl_seconds: int = 3600,
|
||||||
|
policy_version: str = PRODUCTION_POLICY_VERSION,
|
||||||
|
provenance_id: str | None = None,
|
||||||
|
nonce: str | None = None,
|
||||||
|
now: datetime | None = None,
|
||||||
|
) -> RunProvenance:
|
||||||
|
if ttl_seconds <= 0:
|
||||||
|
raise ValueError("ttl_seconds must be positive")
|
||||||
|
normalized_paths = frozenset(ExecutionPath(item).value for item in allowed_paths)
|
||||||
|
if policy_version == PRODUCTION_POLICY_VERSION and normalized_paths != REQUIRED_PRODUCTION_PATHS:
|
||||||
|
missing = sorted(REQUIRED_PRODUCTION_PATHS - normalized_paths)
|
||||||
|
extra = sorted(normalized_paths - REQUIRED_PRODUCTION_PATHS)
|
||||||
|
raise ProvenanceDenied(
|
||||||
|
f"production provenance must cover exactly seven paths; missing={missing}, extra={extra}"
|
||||||
|
)
|
||||||
|
normalized_operations = tuple(
|
||||||
|
sorted({GitOperation(item).value for item in allowed_git_operations})
|
||||||
|
)
|
||||||
|
if ExecutionPath.GIT_OPERATION.value in normalized_paths and not normalized_operations:
|
||||||
|
raise ProvenanceDenied("git_operation path requires explicit allowed Git operations")
|
||||||
|
issued = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
|
||||||
|
unsigned = RunProvenance(
|
||||||
|
provenance_id=provenance_id or f"prov_{uuid.uuid4().hex}",
|
||||||
|
run_id=run_id,
|
||||||
|
issuer=issuer,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
task_id=task_id,
|
||||||
|
agent_id=agent_id,
|
||||||
|
principal_id=principal_id,
|
||||||
|
allowed_paths=tuple(normalized_paths),
|
||||||
|
allowed_git_operations=normalized_operations,
|
||||||
|
issued_at=issued.isoformat().replace("+00:00", "Z"),
|
||||||
|
expires_at=(issued + timedelta(seconds=ttl_seconds))
|
||||||
|
.isoformat()
|
||||||
|
.replace("+00:00", "Z"),
|
||||||
|
policy_version=policy_version,
|
||||||
|
key_id=self.signer.key_id,
|
||||||
|
nonce=nonce or uuid.uuid4().hex,
|
||||||
|
)
|
||||||
|
signed = replace(unsigned, signature=self.signer.sign(unsigned.canonical_bytes()))
|
||||||
|
self.store.save_issued(signed)
|
||||||
|
return signed
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ProvenanceAuthorization:
|
||||||
|
authorization_id: str
|
||||||
|
provenance_id: str
|
||||||
|
run_id: str
|
||||||
|
tenant_id: str
|
||||||
|
task_id: str
|
||||||
|
agent_id: str
|
||||||
|
principal_id: str
|
||||||
|
execution_path: str
|
||||||
|
git_operation: str | None
|
||||||
|
request_digest: str
|
||||||
|
policy_version: str
|
||||||
|
key_id: str
|
||||||
|
public_key_fingerprint: str
|
||||||
|
authorized_at: str = field(default_factory=utc_now)
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return asdict(self)
|
||||||
|
|
||||||
|
|
||||||
|
class SQLiteProvenanceStore:
|
||||||
|
"""持久化 Provenance、状态、授权使用和不可变事件。"""
|
||||||
|
|
||||||
|
def __init__(self, path: str | Path) -> None:
|
||||||
|
self.path = Path(path).resolve()
|
||||||
|
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
self._connection = sqlite3.connect(
|
||||||
|
str(self.path), timeout=30.0, check_same_thread=False
|
||||||
|
)
|
||||||
|
self._connection.row_factory = sqlite3.Row
|
||||||
|
with self._lock:
|
||||||
|
self._connection.execute("PRAGMA journal_mode=WAL")
|
||||||
|
self._connection.execute("PRAGMA synchronous=FULL")
|
||||||
|
self._connection.execute("PRAGMA foreign_keys=ON")
|
||||||
|
self._connection.executescript(
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS provenance_records (
|
||||||
|
provenance_id TEXT PRIMARY KEY,
|
||||||
|
payload_json TEXT NOT NULL,
|
||||||
|
status TEXT NOT NULL,
|
||||||
|
signature_verified INTEGER NOT NULL DEFAULT 0,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
updated_at TEXT NOT NULL
|
||||||
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS provenance_usage (
|
||||||
|
authorization_id TEXT PRIMARY KEY,
|
||||||
|
provenance_id TEXT NOT NULL,
|
||||||
|
execution_path TEXT NOT NULL,
|
||||||
|
git_operation TEXT NOT NULL DEFAULT '',
|
||||||
|
request_digest TEXT NOT NULL,
|
||||||
|
authorized_at TEXT NOT NULL,
|
||||||
|
receipt_json TEXT NOT NULL,
|
||||||
|
UNIQUE(provenance_id, execution_path, git_operation, request_digest),
|
||||||
|
FOREIGN KEY(provenance_id) REFERENCES provenance_records(provenance_id)
|
||||||
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS provenance_events (
|
||||||
|
event_id TEXT PRIMARY KEY,
|
||||||
|
provenance_id TEXT NOT NULL,
|
||||||
|
event_type TEXT NOT NULL,
|
||||||
|
details_json TEXT NOT NULL,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
FOREIGN KEY(provenance_id) REFERENCES provenance_records(provenance_id)
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_provenance_usage_id
|
||||||
|
ON provenance_usage(provenance_id, authorized_at);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_provenance_events_id
|
||||||
|
ON provenance_events(provenance_id, created_at);
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
self._connection.commit()
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
with self._lock:
|
||||||
|
self._connection.close()
|
||||||
|
|
||||||
|
def _event(
|
||||||
|
self,
|
||||||
|
provenance_id: str,
|
||||||
|
event_type: str,
|
||||||
|
details: Mapping[str, Any] | None = None,
|
||||||
|
) -> None:
|
||||||
|
self._connection.execute(
|
||||||
|
"INSERT INTO provenance_events VALUES (?, ?, ?, ?, ?)",
|
||||||
|
(
|
||||||
|
f"pevt_{uuid.uuid4().hex}",
|
||||||
|
provenance_id,
|
||||||
|
event_type,
|
||||||
|
json.dumps(dict(details or {}), ensure_ascii=False, sort_keys=True),
|
||||||
|
utc_now(),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
def save_issued(self, provenance: RunProvenance) -> None:
|
||||||
|
payload = json.dumps(
|
||||||
|
provenance.as_dict(), ensure_ascii=False, sort_keys=True, separators=(",", ":")
|
||||||
|
)
|
||||||
|
now = utc_now()
|
||||||
|
with self._lock:
|
||||||
|
self._connection.execute("BEGIN IMMEDIATE")
|
||||||
|
try:
|
||||||
|
prior = self._connection.execute(
|
||||||
|
"SELECT payload_json FROM provenance_records WHERE provenance_id = ?",
|
||||||
|
(provenance.provenance_id,),
|
||||||
|
).fetchone()
|
||||||
|
if prior is not None:
|
||||||
|
if prior["payload_json"] != payload:
|
||||||
|
raise ProvenanceDenied("provenance_id reused with different payload")
|
||||||
|
self._connection.rollback()
|
||||||
|
return
|
||||||
|
self._connection.execute(
|
||||||
|
"INSERT INTO provenance_records VALUES (?, ?, ?, 0, ?, ?)",
|
||||||
|
(
|
||||||
|
provenance.provenance_id,
|
||||||
|
payload,
|
||||||
|
ProvenanceStatus.ISSUED.value,
|
||||||
|
now,
|
||||||
|
now,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
self._event(provenance.provenance_id, "issued")
|
||||||
|
self._connection.commit()
|
||||||
|
except Exception:
|
||||||
|
self._connection.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
def get(self, provenance_id: str) -> RunProvenance:
|
||||||
|
with self._lock:
|
||||||
|
row = self._connection.execute(
|
||||||
|
"SELECT payload_json FROM provenance_records WHERE provenance_id = ?",
|
||||||
|
(required(provenance_id, "provenance_id"),),
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise ProvenanceNotFound("provenance not found")
|
||||||
|
value = json.loads(row["payload_json"])
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise ProvenanceError("stored provenance payload is invalid")
|
||||||
|
return RunProvenance.from_dict(value)
|
||||||
|
|
||||||
|
def status(self, provenance_id: str) -> ProvenanceStatus:
|
||||||
|
with self._lock:
|
||||||
|
row = self._connection.execute(
|
||||||
|
"SELECT status FROM provenance_records WHERE provenance_id = ?",
|
||||||
|
(required(provenance_id, "provenance_id"),),
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise ProvenanceNotFound("provenance not found")
|
||||||
|
return ProvenanceStatus(row["status"])
|
||||||
|
|
||||||
|
def mark_verified(self, provenance_id: str, *, fingerprint: str) -> None:
|
||||||
|
with self._lock:
|
||||||
|
self._connection.execute("BEGIN IMMEDIATE")
|
||||||
|
try:
|
||||||
|
current = self.status(provenance_id)
|
||||||
|
if current in {ProvenanceStatus.REVOKED, ProvenanceStatus.EXPIRED}:
|
||||||
|
raise ProvenanceDenied(f"cannot verify provenance in state {current.value}")
|
||||||
|
next_status = (
|
||||||
|
ProvenanceStatus.USED.value
|
||||||
|
if current == ProvenanceStatus.USED
|
||||||
|
else ProvenanceStatus.VERIFIED.value
|
||||||
|
)
|
||||||
|
self._connection.execute(
|
||||||
|
"UPDATE provenance_records SET status = ?, signature_verified = 1, updated_at = ? WHERE provenance_id = ?",
|
||||||
|
(next_status, utc_now(), provenance_id),
|
||||||
|
)
|
||||||
|
self._event(
|
||||||
|
provenance_id,
|
||||||
|
"verified",
|
||||||
|
{"public_key_fingerprint": fingerprint},
|
||||||
|
)
|
||||||
|
self._connection.commit()
|
||||||
|
except Exception:
|
||||||
|
self._connection.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
def mark_expired(self, provenance_id: str) -> None:
|
||||||
|
with self._lock:
|
||||||
|
self._connection.execute("BEGIN IMMEDIATE")
|
||||||
|
try:
|
||||||
|
current = self.status(provenance_id)
|
||||||
|
if current == ProvenanceStatus.REVOKED:
|
||||||
|
self._connection.rollback()
|
||||||
|
return
|
||||||
|
self._connection.execute(
|
||||||
|
"UPDATE provenance_records SET status = ?, updated_at = ? WHERE provenance_id = ?",
|
||||||
|
(ProvenanceStatus.EXPIRED.value, utc_now(), provenance_id),
|
||||||
|
)
|
||||||
|
self._event(provenance_id, "expired")
|
||||||
|
self._connection.commit()
|
||||||
|
except Exception:
|
||||||
|
self._connection.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
def revoke(self, provenance_id: str, *, reason: str, principal_id: str) -> None:
|
||||||
|
with self._lock:
|
||||||
|
self._connection.execute("BEGIN IMMEDIATE")
|
||||||
|
try:
|
||||||
|
self.get(provenance_id)
|
||||||
|
self._connection.execute(
|
||||||
|
"UPDATE provenance_records SET status = ?, updated_at = ? WHERE provenance_id = ?",
|
||||||
|
(ProvenanceStatus.REVOKED.value, utc_now(), provenance_id),
|
||||||
|
)
|
||||||
|
self._event(
|
||||||
|
provenance_id,
|
||||||
|
"revoked",
|
||||||
|
{
|
||||||
|
"reason": required(reason, "reason"),
|
||||||
|
"principal_id": required(principal_id, "principal_id"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
self._connection.commit()
|
||||||
|
except Exception:
|
||||||
|
self._connection.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
def record_authorization(self, authorization: ProvenanceAuthorization) -> None:
|
||||||
|
receipt = json.dumps(
|
||||||
|
authorization.as_dict(), ensure_ascii=False, sort_keys=True, separators=(",", ":")
|
||||||
|
)
|
||||||
|
with self._lock:
|
||||||
|
self._connection.execute("BEGIN IMMEDIATE")
|
||||||
|
try:
|
||||||
|
current = self.status(authorization.provenance_id)
|
||||||
|
if current in {ProvenanceStatus.REVOKED, ProvenanceStatus.EXPIRED}:
|
||||||
|
raise ProvenanceDenied(f"provenance state is {current.value}")
|
||||||
|
try:
|
||||||
|
self._connection.execute(
|
||||||
|
"INSERT INTO provenance_usage VALUES (?, ?, ?, ?, ?, ?, ?)",
|
||||||
|
(
|
||||||
|
authorization.authorization_id,
|
||||||
|
authorization.provenance_id,
|
||||||
|
authorization.execution_path,
|
||||||
|
authorization.git_operation or "",
|
||||||
|
authorization.request_digest,
|
||||||
|
authorization.authorized_at,
|
||||||
|
receipt,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
except sqlite3.IntegrityError as exc:
|
||||||
|
raise ProvenanceReplay(
|
||||||
|
"identical provenance path/operation/request digest already used"
|
||||||
|
) from exc
|
||||||
|
self._connection.execute(
|
||||||
|
"UPDATE provenance_records SET status = ?, updated_at = ? WHERE provenance_id = ?",
|
||||||
|
(
|
||||||
|
ProvenanceStatus.USED.value,
|
||||||
|
authorization.authorized_at,
|
||||||
|
authorization.provenance_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
self._event(
|
||||||
|
authorization.provenance_id,
|
||||||
|
"used",
|
||||||
|
{
|
||||||
|
"authorization_id": authorization.authorization_id,
|
||||||
|
"execution_path": authorization.execution_path,
|
||||||
|
"git_operation": authorization.git_operation,
|
||||||
|
"request_digest": authorization.request_digest,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
self._connection.commit()
|
||||||
|
except Exception:
|
||||||
|
self._connection.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
def authorization_count(self, provenance_id: str) -> int:
|
||||||
|
with self._lock:
|
||||||
|
row = self._connection.execute(
|
||||||
|
"SELECT COUNT(*) AS total FROM provenance_usage WHERE provenance_id = ?",
|
||||||
|
(required(provenance_id, "provenance_id"),),
|
||||||
|
).fetchone()
|
||||||
|
return int(row["total"])
|
||||||
|
|
||||||
|
def events(self, provenance_id: str) -> list[dict[str, Any]]:
|
||||||
|
with self._lock:
|
||||||
|
rows = self._connection.execute(
|
||||||
|
"SELECT event_type, details_json, created_at FROM provenance_events WHERE provenance_id = ? ORDER BY created_at, event_id",
|
||||||
|
(required(provenance_id, "provenance_id"),),
|
||||||
|
).fetchall()
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"event_type": row["event_type"],
|
||||||
|
"details": json.loads(row["details_json"]),
|
||||||
|
"created_at": row["created_at"],
|
||||||
|
}
|
||||||
|
for row in rows
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class ProvenanceVerifier:
|
||||||
|
"""验证 Owner 签名、上下文、路径、Git 操作、时效和重放。"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
store: SQLiteProvenanceStore,
|
||||||
|
verifiers: Mapping[str, SignatureVerifier],
|
||||||
|
) -> None:
|
||||||
|
if not verifiers:
|
||||||
|
raise ValueError("at least one public key verifier is required")
|
||||||
|
self.store = store
|
||||||
|
self.verifiers = {required(key, "key_id"): value for key, value in verifiers.items()}
|
||||||
|
|
||||||
|
def authorize(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
provenance_id: str,
|
||||||
|
run_id: str,
|
||||||
|
tenant_id: str,
|
||||||
|
task_id: str,
|
||||||
|
agent_id: str,
|
||||||
|
principal_id: str,
|
||||||
|
execution_path: str,
|
||||||
|
request_digest: str,
|
||||||
|
git_operation: str | None = None,
|
||||||
|
now: datetime | None = None,
|
||||||
|
) -> ProvenanceAuthorization:
|
||||||
|
provenance = self.store.get(provenance_id)
|
||||||
|
status = self.store.status(provenance.provenance_id)
|
||||||
|
if status == ProvenanceStatus.REVOKED:
|
||||||
|
raise ProvenanceRevoked("provenance is revoked")
|
||||||
|
current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
|
||||||
|
if current >= parse_timestamp(provenance.expires_at):
|
||||||
|
self.store.mark_expired(provenance.provenance_id)
|
||||||
|
raise ProvenanceExpired("provenance is expired")
|
||||||
|
verifier = self.verifiers.get(provenance.key_id)
|
||||||
|
if verifier is None:
|
||||||
|
raise ProvenanceSignatureError("no trusted public key for provenance key_id")
|
||||||
|
if not provenance.signature or not verifier.verify(
|
||||||
|
provenance.canonical_bytes(), provenance.signature
|
||||||
|
):
|
||||||
|
raise ProvenanceSignatureError("invalid provenance signature")
|
||||||
|
expected = {
|
||||||
|
"run_id": required(run_id, "run_id"),
|
||||||
|
"tenant_id": required(tenant_id, "tenant_id"),
|
||||||
|
"task_id": required(task_id, "task_id"),
|
||||||
|
"agent_id": required(agent_id, "agent_id"),
|
||||||
|
"principal_id": required(principal_id, "principal_id"),
|
||||||
|
}
|
||||||
|
for field_name, expected_value in expected.items():
|
||||||
|
if getattr(provenance, field_name) != expected_value:
|
||||||
|
raise ProvenanceDenied(f"provenance {field_name} mismatch")
|
||||||
|
path = ExecutionPath(execution_path).value
|
||||||
|
if path not in provenance.allowed_paths:
|
||||||
|
raise ProvenanceDenied(f"execution path is not allowed: {path}")
|
||||||
|
operation: str | None = None
|
||||||
|
if path == ExecutionPath.GIT_OPERATION.value:
|
||||||
|
if git_operation is None:
|
||||||
|
raise ProvenanceDenied("git operation is required for git_operation path")
|
||||||
|
operation = GitOperation(git_operation).value
|
||||||
|
if operation not in provenance.allowed_git_operations:
|
||||||
|
raise ProvenanceDenied(f"Git operation is not allowed: {operation}")
|
||||||
|
elif git_operation is not None:
|
||||||
|
raise ProvenanceDenied("git_operation is only valid for git_operation path")
|
||||||
|
digest = _validate_digest(request_digest)
|
||||||
|
self.store.mark_verified(
|
||||||
|
provenance.provenance_id, fingerprint=verifier.fingerprint
|
||||||
|
)
|
||||||
|
authorization = ProvenanceAuthorization(
|
||||||
|
authorization_id=f"pauth_{uuid.uuid4().hex}",
|
||||||
|
provenance_id=provenance.provenance_id,
|
||||||
|
run_id=provenance.run_id,
|
||||||
|
tenant_id=provenance.tenant_id,
|
||||||
|
task_id=provenance.task_id,
|
||||||
|
agent_id=provenance.agent_id,
|
||||||
|
principal_id=provenance.principal_id,
|
||||||
|
execution_path=path,
|
||||||
|
git_operation=operation,
|
||||||
|
request_digest=digest,
|
||||||
|
policy_version=provenance.policy_version,
|
||||||
|
key_id=provenance.key_id,
|
||||||
|
public_key_fingerprint=verifier.fingerprint,
|
||||||
|
)
|
||||||
|
self.store.record_authorization(authorization)
|
||||||
|
return authorization
|
||||||
|
|
||||||
|
|
||||||
|
class ProvenanceControlledWriter:
|
||||||
|
"""七路径统一策略入口并持久写入授权回执。"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
verifier: ProvenanceVerifier,
|
||||||
|
receipt_root: str | Path,
|
||||||
|
*,
|
||||||
|
writer_id: str = "aios-v2-provenance-writer",
|
||||||
|
) -> None:
|
||||||
|
self.verifier = verifier
|
||||||
|
self.receipt_root = Path(receipt_root).resolve()
|
||||||
|
self.receipt_root.mkdir(parents=True, exist_ok=True)
|
||||||
|
self.writer_id = required(writer_id, "writer_id")
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _component(value: str) -> str:
|
||||||
|
normalized = required(value, "path component")
|
||||||
|
if normalized in {".", ".."} or "/" in normalized or "\\" in normalized:
|
||||||
|
raise ProvenanceDenied("invalid receipt path component")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
def authorize(self, **values: Any) -> ProvenanceAuthorization:
|
||||||
|
authorization = self.verifier.authorize(**values)
|
||||||
|
tenant = self._component(authorization.tenant_id)
|
||||||
|
provenance_id = self._component(authorization.provenance_id)
|
||||||
|
target_dir = self.receipt_root / tenant / provenance_id
|
||||||
|
target = target_dir / f"{authorization.authorization_id}.json"
|
||||||
|
payload = authorization.as_dict()
|
||||||
|
payload["writer_id"] = self.writer_id
|
||||||
|
with self._lock:
|
||||||
|
target_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
temporary = target.with_suffix(f".tmp-{os.getpid()}")
|
||||||
|
temporary.write_text(
|
||||||
|
json.dumps(payload, ensure_ascii=False, sort_keys=True, indent=2) + "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
os.chmod(temporary, 0o640)
|
||||||
|
os.replace(temporary, target)
|
||||||
|
return authorization
|
||||||
|
|
||||||
|
def authorize_code_change(self, **values: Any) -> ProvenanceAuthorization:
|
||||||
|
return self.authorize(execution_path=ExecutionPath.CODE_CHANGE.value, **values)
|
||||||
|
|
||||||
|
def authorize_git_operation(
|
||||||
|
self, *, git_operation: str, **values: Any
|
||||||
|
) -> ProvenanceAuthorization:
|
||||||
|
return self.authorize(
|
||||||
|
execution_path=ExecutionPath.GIT_OPERATION.value,
|
||||||
|
git_operation=git_operation,
|
||||||
|
**values,
|
||||||
|
)
|
||||||
|
|
||||||
|
def authorize_deployment(self, **values: Any) -> ProvenanceAuthorization:
|
||||||
|
return self.authorize(execution_path=ExecutionPath.DEPLOYMENT.value, **values)
|
||||||
|
|
||||||
|
def authorize_runtime_execute(self, **values: Any) -> ProvenanceAuthorization:
|
||||||
|
return self.authorize(execution_path=ExecutionPath.RUNTIME_EXECUTE.value, **values)
|
||||||
|
|
||||||
|
def authorize_evidence_write(self, **values: Any) -> ProvenanceAuthorization:
|
||||||
|
return self.authorize(execution_path=ExecutionPath.EVIDENCE_WRITE.value, **values)
|
||||||
|
|
||||||
|
def authorize_rollback(self, **values: Any) -> ProvenanceAuthorization:
|
||||||
|
return self.authorize(execution_path=ExecutionPath.ROLLBACK.value, **values)
|
||||||
|
|
||||||
|
def authorize_audit(self, **values: Any) -> ProvenanceAuthorization:
|
||||||
|
return self.authorize(execution_path=ExecutionPath.AUDIT.value, **values)
|
||||||
@@ -0,0 +1,381 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""AIOS V2 Run Provenance 与最小权限 Grant 运维 CLI。
|
||||||
|
|
||||||
|
私钥只用于签发;授权命令只加载一个与 ``--path`` 对应的 Grant。所有输出均为
|
||||||
|
不含 Secret 的 JSON。旧 ``issue`` 仅为候选兼容,生产必须使用
|
||||||
|
``issue-workflow`` 与 ``issue-grant``。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Sequence
|
||||||
|
|
||||||
|
from .provenance import (
|
||||||
|
Ed25519PrivateKeySigner,
|
||||||
|
Ed25519PublicKeyVerifier,
|
||||||
|
ExecutionPath,
|
||||||
|
GitOperation,
|
||||||
|
ProvenanceIssuer,
|
||||||
|
SQLiteProvenanceStore,
|
||||||
|
)
|
||||||
|
from .provenance_grants import (
|
||||||
|
GrantIssuer,
|
||||||
|
GrantType,
|
||||||
|
GrantVerifier,
|
||||||
|
SQLiteGrantStore,
|
||||||
|
WorkflowProvenanceIssuer,
|
||||||
|
)
|
||||||
|
from .provenance_operations import (
|
||||||
|
OperationContext,
|
||||||
|
ProvenanceOperationGate,
|
||||||
|
context_from_environment,
|
||||||
|
load_request_file,
|
||||||
|
)
|
||||||
|
from .provenance_scoped import ScopedLeastPrivilegeProvenanceWriter
|
||||||
|
|
||||||
|
|
||||||
|
def _json_object(value: str | None, file_path: str | None) -> Mapping[str, Any]:
|
||||||
|
if value and file_path:
|
||||||
|
raise ValueError("request-json and request-file are mutually exclusive")
|
||||||
|
if file_path:
|
||||||
|
return load_request_file(file_path)
|
||||||
|
parsed = json.loads(value or "{}")
|
||||||
|
if not isinstance(parsed, Mapping):
|
||||||
|
raise ValueError("request JSON must be an object")
|
||||||
|
return dict(parsed)
|
||||||
|
|
||||||
|
|
||||||
|
def _csv(value: str | None) -> tuple[str, ...]:
|
||||||
|
return tuple(item.strip() for item in str(value or "").split(",") if item.strip())
|
||||||
|
|
||||||
|
|
||||||
|
def _write_json(path: str | None, payload: Mapping[str, Any]) -> None:
|
||||||
|
encoded = json.dumps(dict(payload), ensure_ascii=False, sort_keys=True, indent=2) + "\n"
|
||||||
|
if path:
|
||||||
|
target = Path(path).resolve()
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
temporary = target.with_suffix(f".tmp-{os.getpid()}")
|
||||||
|
temporary.write_text(encoded, encoding="utf-8")
|
||||||
|
os.chmod(temporary, 0o640)
|
||||||
|
os.replace(temporary, target)
|
||||||
|
print(encoded, end="")
|
||||||
|
|
||||||
|
|
||||||
|
def _parent_store(args: argparse.Namespace) -> SQLiteProvenanceStore:
|
||||||
|
return SQLiteProvenanceStore(args.store_file)
|
||||||
|
|
||||||
|
|
||||||
|
def _grant_store(args: argparse.Namespace) -> SQLiteGrantStore:
|
||||||
|
return SQLiteGrantStore(args.store_file)
|
||||||
|
|
||||||
|
|
||||||
|
def _context(args: argparse.Namespace) -> OperationContext:
|
||||||
|
if args.context_from_env:
|
||||||
|
return context_from_environment()
|
||||||
|
values = {
|
||||||
|
"provenance_id": args.provenance_id,
|
||||||
|
"run_id": args.run_id,
|
||||||
|
"tenant_id": args.tenant_id,
|
||||||
|
"task_id": args.task_id,
|
||||||
|
"agent_id": args.agent_id,
|
||||||
|
"principal_id": args.principal_id,
|
||||||
|
}
|
||||||
|
missing = [name for name, value in values.items() if not str(value or "").strip()]
|
||||||
|
if missing:
|
||||||
|
raise ValueError(f"missing context arguments: {missing}")
|
||||||
|
return OperationContext(**{name: str(value).strip() for name, value in values.items()})
|
||||||
|
|
||||||
|
|
||||||
|
def command_issue_legacy(args: argparse.Namespace) -> int:
|
||||||
|
store = _parent_store(args)
|
||||||
|
signer = Ed25519PrivateKeySigner.from_file(args.private_key_file, key_id=args.key_id)
|
||||||
|
provenance = ProvenanceIssuer(signer, store).issue(
|
||||||
|
run_id=args.run_id,
|
||||||
|
issuer=args.issuer,
|
||||||
|
tenant_id=args.tenant_id,
|
||||||
|
task_id=args.task_id,
|
||||||
|
agent_id=args.agent_id,
|
||||||
|
principal_id=args.principal_id,
|
||||||
|
allowed_paths=_csv(args.allowed_paths) or tuple(item.value for item in ExecutionPath),
|
||||||
|
allowed_git_operations=_csv(args.allowed_git_operations)
|
||||||
|
or tuple(item.value for item in GitOperation),
|
||||||
|
ttl_seconds=args.ttl_seconds,
|
||||||
|
policy_version=args.policy_version,
|
||||||
|
)
|
||||||
|
_write_json(
|
||||||
|
args.output,
|
||||||
|
{
|
||||||
|
"mode": "legacy-full-rights-candidate-only",
|
||||||
|
"provenance": provenance.as_dict(),
|
||||||
|
"status": store.status(provenance.provenance_id).value,
|
||||||
|
"secret_value_stored": False,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def command_issue_workflow(args: argparse.Namespace) -> int:
|
||||||
|
store = _parent_store(args)
|
||||||
|
signer = Ed25519PrivateKeySigner.from_file(args.private_key_file, key_id=args.key_id)
|
||||||
|
workflow = WorkflowProvenanceIssuer(signer, store).issue(
|
||||||
|
run_id=args.run_id,
|
||||||
|
issuer=args.issuer,
|
||||||
|
tenant_id=args.tenant_id,
|
||||||
|
task_id=args.task_id,
|
||||||
|
agent_id=args.agent_id,
|
||||||
|
principal_id=args.principal_id,
|
||||||
|
ttl_seconds=args.ttl_seconds,
|
||||||
|
)
|
||||||
|
_write_json(
|
||||||
|
args.output,
|
||||||
|
{
|
||||||
|
"mode": "least-privilege-workflow",
|
||||||
|
"workflow_provenance": workflow.as_dict(),
|
||||||
|
"status": store.status(workflow.provenance_id).value,
|
||||||
|
"secret_value_stored": False,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def command_issue_grant(args: argparse.Namespace) -> int:
|
||||||
|
parent_store = _parent_store(args)
|
||||||
|
grant_store = _grant_store(args)
|
||||||
|
signer = Ed25519PrivateKeySigner.from_file(args.private_key_file, key_id=args.key_id)
|
||||||
|
grant = GrantIssuer(signer, parent_store, grant_store).issue(
|
||||||
|
parent_provenance_id=args.parent_provenance_id,
|
||||||
|
grant_type=GrantType(args.grant_type),
|
||||||
|
ttl_seconds=args.ttl_seconds,
|
||||||
|
allowed_git_operations=_csv(args.allowed_git_operations),
|
||||||
|
)
|
||||||
|
_write_json(
|
||||||
|
args.output,
|
||||||
|
{
|
||||||
|
"grant": grant.as_dict(),
|
||||||
|
"status": grant_store.status(grant.grant_id).value,
|
||||||
|
"secret_value_stored": False,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _single_scope_writer(args: argparse.Namespace) -> ScopedLeastPrivilegeProvenanceWriter:
|
||||||
|
path = ExecutionPath(args.path).value
|
||||||
|
parent_store = _parent_store(args)
|
||||||
|
grant_store = _grant_store(args)
|
||||||
|
verifier = Ed25519PublicKeyVerifier.from_file(args.public_key_file, key_id=args.key_id)
|
||||||
|
trusted = GrantVerifier(parent_store, grant_store, {args.key_id: verifier})
|
||||||
|
return ScopedLeastPrivilegeProvenanceWriter(
|
||||||
|
trusted,
|
||||||
|
args.receipt_dir,
|
||||||
|
{path: args.grant_id},
|
||||||
|
required_paths=(path,),
|
||||||
|
writer_id=f"aios-v2-cli-{path}",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def command_authorize(args: argparse.Namespace) -> int:
|
||||||
|
writer = _single_scope_writer(args)
|
||||||
|
context = _context(args)
|
||||||
|
receipt = ProvenanceOperationGate(writer).authorize(
|
||||||
|
context,
|
||||||
|
execution_path=args.path,
|
||||||
|
operation=args.operation,
|
||||||
|
request=_json_object(args.request_json, args.request_file),
|
||||||
|
)
|
||||||
|
_write_json(args.output, {"authorization": receipt.as_dict()})
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def command_revoke(args: argparse.Namespace) -> int:
|
||||||
|
store = _parent_store(args)
|
||||||
|
store.revoke(args.provenance_id, reason=args.reason, principal_id=args.principal_id)
|
||||||
|
_write_json(
|
||||||
|
args.output,
|
||||||
|
{"provenance_id": args.provenance_id, "status": store.status(args.provenance_id).value},
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def command_status(args: argparse.Namespace) -> int:
|
||||||
|
store = _parent_store(args)
|
||||||
|
provenance = store.get(args.provenance_id)
|
||||||
|
grant_store = _grant_store(args)
|
||||||
|
_write_json(
|
||||||
|
args.output,
|
||||||
|
{
|
||||||
|
"provenance": provenance.as_dict(),
|
||||||
|
"status": store.status(args.provenance_id).value,
|
||||||
|
"authorization_count": store.authorization_count(args.provenance_id),
|
||||||
|
"events": store.events(args.provenance_id),
|
||||||
|
"grants": [item.as_dict() for item in grant_store.list_for_parent(args.provenance_id)],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def command_grant_revoke(args: argparse.Namespace) -> int:
|
||||||
|
store = _grant_store(args)
|
||||||
|
store.revoke(args.grant_id, reason=args.reason, principal_id=args.principal_id)
|
||||||
|
_write_json(
|
||||||
|
args.output,
|
||||||
|
{"grant_id": args.grant_id, "status": store.status(args.grant_id).value},
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def command_grant_status(args: argparse.Namespace) -> int:
|
||||||
|
store = _grant_store(args)
|
||||||
|
grant = store.get(args.grant_id)
|
||||||
|
_write_json(
|
||||||
|
args.output,
|
||||||
|
{"grant": grant.as_dict(), "status": store.status(args.grant_id).value},
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def add_common_store(parser: argparse.ArgumentParser) -> None:
|
||||||
|
parser.add_argument(
|
||||||
|
"--store-file",
|
||||||
|
default=os.getenv(
|
||||||
|
"AIOS_PROVENANCE_STORE_FILE",
|
||||||
|
"/var/lib/aios-v2/run-provenance.sqlite3",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
parser.add_argument("--output", default=None)
|
||||||
|
|
||||||
|
|
||||||
|
def add_signing_context(parser: argparse.ArgumentParser) -> None:
|
||||||
|
parser.add_argument("--private-key-file", required=True)
|
||||||
|
parser.add_argument("--key-id", required=True)
|
||||||
|
|
||||||
|
|
||||||
|
def add_workflow_identity(parser: argparse.ArgumentParser) -> None:
|
||||||
|
parser.add_argument("--run-id", required=True)
|
||||||
|
parser.add_argument("--issuer", required=True)
|
||||||
|
parser.add_argument("--tenant-id", required=True)
|
||||||
|
parser.add_argument("--task-id", required=True)
|
||||||
|
parser.add_argument("--agent-id", required=True)
|
||||||
|
parser.add_argument("--principal-id", required=True)
|
||||||
|
|
||||||
|
|
||||||
|
def add_context(parser: argparse.ArgumentParser) -> None:
|
||||||
|
parser.add_argument("--context-from-env", action="store_true")
|
||||||
|
parser.add_argument("--provenance-id")
|
||||||
|
parser.add_argument("--run-id")
|
||||||
|
parser.add_argument("--tenant-id")
|
||||||
|
parser.add_argument("--task-id")
|
||||||
|
parser.add_argument("--agent-id")
|
||||||
|
parser.add_argument("--principal-id")
|
||||||
|
|
||||||
|
|
||||||
|
def build_parser() -> argparse.ArgumentParser:
|
||||||
|
parser = argparse.ArgumentParser(description="AIOS V2 Run Provenance CLI")
|
||||||
|
subcommands = parser.add_subparsers(dest="command", required=True)
|
||||||
|
|
||||||
|
issue = subcommands.add_parser("issue", help="候选兼容:签发全权限 Provenance;生产禁止")
|
||||||
|
add_common_store(issue)
|
||||||
|
add_signing_context(issue)
|
||||||
|
add_workflow_identity(issue)
|
||||||
|
issue.add_argument("--allowed-paths")
|
||||||
|
issue.add_argument("--allowed-git-operations")
|
||||||
|
issue.add_argument("--ttl-seconds", type=int, default=3600)
|
||||||
|
issue.add_argument("--policy-version", default="aios.v2.candidate.legacy")
|
||||||
|
issue.set_defaults(handler=command_issue_legacy)
|
||||||
|
|
||||||
|
workflow = subcommands.add_parser(
|
||||||
|
"issue-workflow", help="签发不含操作权限的父 Workflow Provenance"
|
||||||
|
)
|
||||||
|
add_common_store(workflow)
|
||||||
|
add_signing_context(workflow)
|
||||||
|
add_workflow_identity(workflow)
|
||||||
|
workflow.add_argument("--ttl-seconds", type=int, default=14400)
|
||||||
|
workflow.set_defaults(handler=command_issue_workflow)
|
||||||
|
|
||||||
|
grant = subcommands.add_parser("issue-grant", help="签发单一路径最小权限 Grant")
|
||||||
|
add_common_store(grant)
|
||||||
|
add_signing_context(grant)
|
||||||
|
grant.add_argument("--parent-provenance-id", required=True)
|
||||||
|
grant.add_argument("--grant-type", choices=[item.value for item in GrantType], required=True)
|
||||||
|
grant.add_argument("--allowed-git-operations")
|
||||||
|
grant.add_argument("--ttl-seconds", type=int, default=900)
|
||||||
|
grant.set_defaults(handler=command_issue_grant)
|
||||||
|
|
||||||
|
authorize = subcommands.add_parser("authorize", help="使用单一子 Grant 授权一次运维操作")
|
||||||
|
add_common_store(authorize)
|
||||||
|
add_context(authorize)
|
||||||
|
authorize.add_argument("--public-key-file", required=True)
|
||||||
|
authorize.add_argument("--key-id", required=True)
|
||||||
|
authorize.add_argument("--grant-id", required=True)
|
||||||
|
authorize.add_argument(
|
||||||
|
"--receipt-dir",
|
||||||
|
default=os.getenv(
|
||||||
|
"AIOS_PROVENANCE_RECEIPT_DIR",
|
||||||
|
"/var/lib/aios-v2/provenance-receipts",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
authorize.add_argument(
|
||||||
|
"--path",
|
||||||
|
required=True,
|
||||||
|
choices=[
|
||||||
|
ExecutionPath.CODE_CHANGE.value,
|
||||||
|
ExecutionPath.DEPLOYMENT.value,
|
||||||
|
ExecutionPath.RUNTIME_EXECUTE.value,
|
||||||
|
ExecutionPath.EVIDENCE_WRITE.value,
|
||||||
|
ExecutionPath.ROLLBACK.value,
|
||||||
|
ExecutionPath.AUDIT.value,
|
||||||
|
],
|
||||||
|
)
|
||||||
|
authorize.add_argument("--operation", required=True)
|
||||||
|
request_group = authorize.add_mutually_exclusive_group()
|
||||||
|
request_group.add_argument("--request-json")
|
||||||
|
request_group.add_argument("--request-file")
|
||||||
|
authorize.set_defaults(handler=command_authorize)
|
||||||
|
|
||||||
|
revoke = subcommands.add_parser("revoke", help="撤销父 Workflow Provenance")
|
||||||
|
add_common_store(revoke)
|
||||||
|
revoke.add_argument("--provenance-id", required=True)
|
||||||
|
revoke.add_argument("--reason", required=True)
|
||||||
|
revoke.add_argument("--principal-id", required=True)
|
||||||
|
revoke.set_defaults(handler=command_revoke)
|
||||||
|
|
||||||
|
status = subcommands.add_parser("status", help="查询父 Workflow 与子 Grant")
|
||||||
|
add_common_store(status)
|
||||||
|
status.add_argument("--provenance-id", required=True)
|
||||||
|
status.set_defaults(handler=command_status)
|
||||||
|
|
||||||
|
grant_revoke = subcommands.add_parser("grant-revoke", help="单独撤销子 Grant")
|
||||||
|
add_common_store(grant_revoke)
|
||||||
|
grant_revoke.add_argument("--grant-id", required=True)
|
||||||
|
grant_revoke.add_argument("--reason", required=True)
|
||||||
|
grant_revoke.add_argument("--principal-id", required=True)
|
||||||
|
grant_revoke.set_defaults(handler=command_grant_revoke)
|
||||||
|
|
||||||
|
grant_status = subcommands.add_parser("grant-status", help="查询子 Grant 状态")
|
||||||
|
add_common_store(grant_status)
|
||||||
|
grant_status.add_argument("--grant-id", required=True)
|
||||||
|
grant_status.set_defaults(handler=command_grant_status)
|
||||||
|
return parser
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: Sequence[str] | None = None) -> int:
|
||||||
|
args = build_parser().parse_args(argv)
|
||||||
|
try:
|
||||||
|
return int(args.handler(args))
|
||||||
|
except Exception as exc:
|
||||||
|
print(
|
||||||
|
json.dumps(
|
||||||
|
{"status": "denied", "error": type(exc).__name__, "message": str(exc)},
|
||||||
|
ensure_ascii=False,
|
||||||
|
),
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,416 @@
|
|||||||
|
"""受最小权限 Run Provenance Grant 约束的 Git 写入器。
|
||||||
|
|
||||||
|
所有 Git 操作都使用参数数组执行。对于 Commit 和 Evidence Commit,先计算
|
||||||
|
工作树内容摘要并完成 Provenance 授权,之后才允许执行 ``git add``,避免未授权
|
||||||
|
请求改变 Git Index。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import uuid
|
||||||
|
from dataclasses import asdict, dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Protocol, Sequence
|
||||||
|
|
||||||
|
from .provenance import GitOperation, ProvenanceDenied, digest_payload, utc_now
|
||||||
|
|
||||||
|
_BRANCH_PATTERN = re.compile(r"^(?![-./])(?!.*\.\.)(?!.*//)[A-Za-z0-9._/-]{1,240}$")
|
||||||
|
_SECRET_URL = re.compile(r"(?i)(https?://)([^/@\s]+)@")
|
||||||
|
_TOKENISH = re.compile(r"(?i)(token|password|secret|authorization)([=:]\s*)([^\s]+)")
|
||||||
|
|
||||||
|
|
||||||
|
class GitAuthorizationWriter(Protocol):
|
||||||
|
def authorize_code_change(self, **values: Any) -> Any: ...
|
||||||
|
|
||||||
|
def authorize_git_operation(self, *, git_operation: str, **values: Any) -> Any: ...
|
||||||
|
|
||||||
|
|
||||||
|
class ProvenanceGitError(RuntimeError):
|
||||||
|
"""Git 操作被拒绝或执行失败。"""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class GitExecutionContext:
|
||||||
|
provenance_id: str
|
||||||
|
run_id: str
|
||||||
|
tenant_id: str
|
||||||
|
task_id: str
|
||||||
|
agent_id: str
|
||||||
|
principal_id: str
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class GitCommandResult:
|
||||||
|
operation_id: str
|
||||||
|
operation: str
|
||||||
|
authorization_ids: Sequence[str]
|
||||||
|
command: Sequence[str]
|
||||||
|
exit_code: int
|
||||||
|
stdout: str
|
||||||
|
stderr: str
|
||||||
|
head_before: str | None
|
||||||
|
head_after: str | None
|
||||||
|
started_at: str
|
||||||
|
completed_at: str
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return asdict(self)
|
||||||
|
|
||||||
|
|
||||||
|
class ProvenanceGitWriter:
|
||||||
|
"""执行经过单路径 Grant 授权的窄范围 Git 操作。"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
repo: str | Path,
|
||||||
|
writer: GitAuthorizationWriter,
|
||||||
|
*,
|
||||||
|
timeout_seconds: float = 120.0,
|
||||||
|
output_limit: int = 16000,
|
||||||
|
) -> None:
|
||||||
|
self.repo = Path(repo).resolve()
|
||||||
|
self.writer = writer
|
||||||
|
self.timeout_seconds = float(timeout_seconds)
|
||||||
|
self.output_limit = int(output_limit)
|
||||||
|
if self.timeout_seconds <= 0 or self.output_limit < 1024:
|
||||||
|
raise ValueError("invalid Git writer limits")
|
||||||
|
self._run_raw(["rev-parse", "--is-inside-work-tree"], check=True)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _validate_branch(branch: str) -> str:
|
||||||
|
normalized = str(branch or "").strip()
|
||||||
|
if not _BRANCH_PATTERN.fullmatch(normalized) or normalized.endswith((".", "/")):
|
||||||
|
raise ProvenanceGitError("invalid Git branch name")
|
||||||
|
if normalized in {"main", "master"}:
|
||||||
|
raise ProvenanceGitError("direct main/master operation is prohibited")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
def _validate_path(self, value: str | Path) -> str:
|
||||||
|
raw = Path(value)
|
||||||
|
candidate = (self.repo / raw).resolve() if not raw.is_absolute() else raw.resolve()
|
||||||
|
try:
|
||||||
|
relative = candidate.relative_to(self.repo)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise ProvenanceGitError("Git path escapes repository") from exc
|
||||||
|
if str(relative) in {"", "."}:
|
||||||
|
raise ProvenanceGitError("repository root is not a valid file path")
|
||||||
|
return relative.as_posix()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _redact(value: str, limit: int) -> str:
|
||||||
|
text = _SECRET_URL.sub(r"\1***@", str(value or ""))
|
||||||
|
text = _TOKENISH.sub(r"\1\2***", text)
|
||||||
|
if len(text) > limit:
|
||||||
|
return text[:limit] + "\n...[truncated]"
|
||||||
|
return text
|
||||||
|
|
||||||
|
def _head(self) -> str | None:
|
||||||
|
process = self._run_raw(["rev-parse", "HEAD"], check=False)
|
||||||
|
return process.stdout.strip() if process.returncode == 0 else None
|
||||||
|
|
||||||
|
def _run_raw(
|
||||||
|
self,
|
||||||
|
args: Sequence[str],
|
||||||
|
*,
|
||||||
|
check: bool,
|
||||||
|
env: Mapping[str, str] | None = None,
|
||||||
|
) -> subprocess.CompletedProcess[str]:
|
||||||
|
command = ["git", "-C", str(self.repo), *[str(item) for item in args]]
|
||||||
|
safe_env = {
|
||||||
|
**os.environ,
|
||||||
|
"GIT_TERMINAL_PROMPT": "0",
|
||||||
|
"GIT_CONFIG_NOSYSTEM": os.environ.get("GIT_CONFIG_NOSYSTEM", "1"),
|
||||||
|
}
|
||||||
|
if env:
|
||||||
|
safe_env.update({str(key): str(value) for key, value in env.items()})
|
||||||
|
process = subprocess.run(
|
||||||
|
command,
|
||||||
|
text=True,
|
||||||
|
capture_output=True,
|
||||||
|
timeout=self.timeout_seconds,
|
||||||
|
env=safe_env,
|
||||||
|
)
|
||||||
|
if check and process.returncode != 0:
|
||||||
|
raise ProvenanceGitError(
|
||||||
|
self._redact(process.stderr or process.stdout, self.output_limit)
|
||||||
|
or f"Git command failed with exit code {process.returncode}"
|
||||||
|
)
|
||||||
|
return process
|
||||||
|
|
||||||
|
def _path_digest(self, relative: str) -> str:
|
||||||
|
target = self.repo / relative
|
||||||
|
if target.exists():
|
||||||
|
if not target.is_file():
|
||||||
|
raise ProvenanceGitError("Git writer accepts file paths only")
|
||||||
|
hasher = hashlib.sha256()
|
||||||
|
with target.open("rb") as handle:
|
||||||
|
while True:
|
||||||
|
block = handle.read(65536)
|
||||||
|
if not block:
|
||||||
|
break
|
||||||
|
hasher.update(block)
|
||||||
|
return hasher.hexdigest()
|
||||||
|
tracked = self._run_raw(["ls-files", "--error-unmatch", "--", relative], check=False)
|
||||||
|
if tracked.returncode != 0:
|
||||||
|
raise ProvenanceGitError(f"path does not exist and is not tracked: {relative}")
|
||||||
|
return "deleted"
|
||||||
|
|
||||||
|
def _content_manifest(self, paths: Sequence[str]) -> Mapping[str, str]:
|
||||||
|
return {path: self._path_digest(path) for path in sorted(set(paths))}
|
||||||
|
|
||||||
|
def _authorize(
|
||||||
|
self,
|
||||||
|
context: GitExecutionContext,
|
||||||
|
*,
|
||||||
|
operation: GitOperation,
|
||||||
|
request: Mapping[str, Any],
|
||||||
|
code_change: bool,
|
||||||
|
) -> list[str]:
|
||||||
|
request_digest = digest_payload(
|
||||||
|
{
|
||||||
|
"operation": operation.value,
|
||||||
|
"repo": self.repo.name,
|
||||||
|
"request": dict(request),
|
||||||
|
"head": self._head(),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
ids: list[str] = []
|
||||||
|
try:
|
||||||
|
if code_change:
|
||||||
|
code_authorization = self.writer.authorize_code_change(
|
||||||
|
provenance_id=context.provenance_id,
|
||||||
|
run_id=context.run_id,
|
||||||
|
tenant_id=context.tenant_id,
|
||||||
|
task_id=context.task_id,
|
||||||
|
agent_id=context.agent_id,
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
request_digest=digest_payload(
|
||||||
|
{
|
||||||
|
"git_operation": operation.value,
|
||||||
|
"request_digest": request_digest,
|
||||||
|
}
|
||||||
|
),
|
||||||
|
)
|
||||||
|
ids.append(code_authorization.authorization_id)
|
||||||
|
git_authorization = self.writer.authorize_git_operation(
|
||||||
|
provenance_id=context.provenance_id,
|
||||||
|
run_id=context.run_id,
|
||||||
|
tenant_id=context.tenant_id,
|
||||||
|
task_id=context.task_id,
|
||||||
|
agent_id=context.agent_id,
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
git_operation=operation.value,
|
||||||
|
request_digest=request_digest,
|
||||||
|
)
|
||||||
|
ids.append(git_authorization.authorization_id)
|
||||||
|
return ids
|
||||||
|
except ProvenanceDenied as exc:
|
||||||
|
raise ProvenanceGitError(f"Run Provenance denied Git operation: {exc}") from exc
|
||||||
|
|
||||||
|
def _run_authorized(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
operation: GitOperation,
|
||||||
|
args: Sequence[str],
|
||||||
|
authorization_ids: Sequence[str],
|
||||||
|
) -> GitCommandResult:
|
||||||
|
started = utc_now()
|
||||||
|
before = self._head()
|
||||||
|
process = self._run_raw(args, check=False)
|
||||||
|
after = self._head()
|
||||||
|
result = GitCommandResult(
|
||||||
|
operation_id=f"gitop_{uuid.uuid4().hex}",
|
||||||
|
operation=operation.value,
|
||||||
|
authorization_ids=tuple(authorization_ids),
|
||||||
|
command=tuple(["git", "-C", "<repo>", *[str(item) for item in args]]),
|
||||||
|
exit_code=int(process.returncode),
|
||||||
|
stdout=self._redact(process.stdout, self.output_limit),
|
||||||
|
stderr=self._redact(process.stderr, self.output_limit),
|
||||||
|
head_before=before,
|
||||||
|
head_after=after,
|
||||||
|
started_at=started,
|
||||||
|
completed_at=utc_now(),
|
||||||
|
)
|
||||||
|
if process.returncode != 0:
|
||||||
|
raise ProvenanceGitError(
|
||||||
|
result.stderr or result.stdout or f"Git {operation.value} failed"
|
||||||
|
)
|
||||||
|
return result
|
||||||
|
|
||||||
|
def _execute(
|
||||||
|
self,
|
||||||
|
context: GitExecutionContext,
|
||||||
|
*,
|
||||||
|
operation: GitOperation,
|
||||||
|
args: Sequence[str],
|
||||||
|
request: Mapping[str, Any],
|
||||||
|
code_change: bool,
|
||||||
|
) -> GitCommandResult:
|
||||||
|
authorization_ids = self._authorize(
|
||||||
|
context,
|
||||||
|
operation=operation,
|
||||||
|
request=request,
|
||||||
|
code_change=code_change,
|
||||||
|
)
|
||||||
|
return self._run_authorized(
|
||||||
|
operation=operation,
|
||||||
|
args=args,
|
||||||
|
authorization_ids=authorization_ids,
|
||||||
|
)
|
||||||
|
|
||||||
|
def read_ref(self, context: GitExecutionContext, ref: str) -> GitCommandResult:
|
||||||
|
normalized = str(ref or "").strip()
|
||||||
|
if not normalized or any(ch.isspace() for ch in normalized):
|
||||||
|
raise ProvenanceGitError("invalid Git ref")
|
||||||
|
return self._execute(
|
||||||
|
context,
|
||||||
|
operation=GitOperation.READ_REF,
|
||||||
|
args=["rev-parse", "--verify", normalized],
|
||||||
|
request={"ref": normalized},
|
||||||
|
code_change=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
def fetch(
|
||||||
|
self,
|
||||||
|
context: GitExecutionContext,
|
||||||
|
*,
|
||||||
|
remote: str,
|
||||||
|
refspecs: Sequence[str],
|
||||||
|
) -> GitCommandResult:
|
||||||
|
normalized_remote = str(remote or "").strip()
|
||||||
|
normalized_refspecs = tuple(str(item).strip() for item in refspecs if str(item).strip())
|
||||||
|
if not normalized_remote or not normalized_refspecs:
|
||||||
|
raise ProvenanceGitError("remote and refspecs are required")
|
||||||
|
if any(item.startswith("+") and ":refs/heads/" in item for item in normalized_refspecs):
|
||||||
|
raise ProvenanceGitError(
|
||||||
|
"fetching directly into local branch refs is prohibited; use remote-tracking refs"
|
||||||
|
)
|
||||||
|
return self._execute(
|
||||||
|
context,
|
||||||
|
operation=GitOperation.FETCH,
|
||||||
|
args=["fetch", "--prune", normalized_remote, *normalized_refspecs],
|
||||||
|
request={"remote": normalized_remote, "refspecs": normalized_refspecs},
|
||||||
|
code_change=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
def create_branch(
|
||||||
|
self,
|
||||||
|
context: GitExecutionContext,
|
||||||
|
*,
|
||||||
|
branch: str,
|
||||||
|
start_point: str,
|
||||||
|
) -> GitCommandResult:
|
||||||
|
normalized = self._validate_branch(branch)
|
||||||
|
start = str(start_point or "").strip()
|
||||||
|
if not start:
|
||||||
|
raise ProvenanceGitError("start_point is required")
|
||||||
|
return self._execute(
|
||||||
|
context,
|
||||||
|
operation=GitOperation.CREATE_BRANCH,
|
||||||
|
args=["branch", normalized, start],
|
||||||
|
request={"branch": normalized, "start_point": start},
|
||||||
|
code_change=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
def commit(
|
||||||
|
self,
|
||||||
|
context: GitExecutionContext,
|
||||||
|
*,
|
||||||
|
paths: Sequence[str | Path],
|
||||||
|
message: str,
|
||||||
|
) -> GitCommandResult:
|
||||||
|
normalized_paths = tuple(self._validate_path(item) for item in paths)
|
||||||
|
normalized_message = str(message or "").strip()
|
||||||
|
if not normalized_paths or not normalized_message:
|
||||||
|
raise ProvenanceGitError("commit paths and message are required")
|
||||||
|
if len(normalized_message) > 500:
|
||||||
|
raise ProvenanceGitError("commit message is too long")
|
||||||
|
request = {
|
||||||
|
"paths": normalized_paths,
|
||||||
|
"message": normalized_message,
|
||||||
|
"content_manifest": self._content_manifest(normalized_paths),
|
||||||
|
}
|
||||||
|
authorization_ids = self._authorize(
|
||||||
|
context,
|
||||||
|
operation=GitOperation.COMMIT,
|
||||||
|
request=request,
|
||||||
|
code_change=True,
|
||||||
|
)
|
||||||
|
self._run_raw(["add", "--", *normalized_paths], check=True)
|
||||||
|
return self._run_authorized(
|
||||||
|
operation=GitOperation.COMMIT,
|
||||||
|
args=["commit", "--no-gpg-sign", "-m", normalized_message, "--", *normalized_paths],
|
||||||
|
authorization_ids=authorization_ids,
|
||||||
|
)
|
||||||
|
|
||||||
|
def merge(
|
||||||
|
self,
|
||||||
|
context: GitExecutionContext,
|
||||||
|
*,
|
||||||
|
source_ref: str,
|
||||||
|
message: str,
|
||||||
|
) -> GitCommandResult:
|
||||||
|
source = str(source_ref or "").strip()
|
||||||
|
normalized_message = str(message or "").strip()
|
||||||
|
if not source or not normalized_message:
|
||||||
|
raise ProvenanceGitError("source_ref and message are required")
|
||||||
|
return self._execute(
|
||||||
|
context,
|
||||||
|
operation=GitOperation.MERGE,
|
||||||
|
args=["merge", "--no-ff", "--no-edit", "-m", normalized_message, source],
|
||||||
|
request={"source_ref": source, "message": normalized_message},
|
||||||
|
code_change=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
def push(
|
||||||
|
self,
|
||||||
|
context: GitExecutionContext,
|
||||||
|
*,
|
||||||
|
remote: str,
|
||||||
|
branch: str,
|
||||||
|
) -> GitCommandResult:
|
||||||
|
normalized_remote = str(remote or "").strip()
|
||||||
|
normalized_branch = self._validate_branch(branch)
|
||||||
|
if not normalized_remote:
|
||||||
|
raise ProvenanceGitError("remote is required")
|
||||||
|
return self._execute(
|
||||||
|
context,
|
||||||
|
operation=GitOperation.PUSH,
|
||||||
|
args=["push", normalized_remote, f"refs/heads/{normalized_branch}:refs/heads/{normalized_branch}"],
|
||||||
|
request={"remote": normalized_remote, "branch": normalized_branch},
|
||||||
|
code_change=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
def update_evidence(
|
||||||
|
self,
|
||||||
|
context: GitExecutionContext,
|
||||||
|
*,
|
||||||
|
paths: Sequence[str | Path],
|
||||||
|
message: str,
|
||||||
|
) -> GitCommandResult:
|
||||||
|
normalized_paths = tuple(self._validate_path(item) for item in paths)
|
||||||
|
normalized_message = str(message or "").strip()
|
||||||
|
if not normalized_paths or not normalized_message:
|
||||||
|
raise ProvenanceGitError("evidence paths and message are required")
|
||||||
|
if any(not path.startswith(".agent/") for path in normalized_paths):
|
||||||
|
raise ProvenanceGitError("UPDATE_EVIDENCE may only commit .agent/ paths")
|
||||||
|
request = {
|
||||||
|
"paths": normalized_paths,
|
||||||
|
"message": normalized_message,
|
||||||
|
"content_manifest": self._content_manifest(normalized_paths),
|
||||||
|
}
|
||||||
|
authorization_ids = self._authorize(
|
||||||
|
context,
|
||||||
|
operation=GitOperation.UPDATE_EVIDENCE,
|
||||||
|
request=request,
|
||||||
|
code_change=False,
|
||||||
|
)
|
||||||
|
self._run_raw(["add", "--", *normalized_paths], check=True)
|
||||||
|
return self._run_authorized(
|
||||||
|
operation=GitOperation.UPDATE_EVIDENCE,
|
||||||
|
args=["commit", "--no-gpg-sign", "-m", normalized_message, "--", *normalized_paths],
|
||||||
|
authorization_ids=authorization_ids,
|
||||||
|
)
|
||||||
@@ -0,0 +1,744 @@
|
|||||||
|
"""AIOS V2 最小权限 Run Provenance 子 Grant。
|
||||||
|
|
||||||
|
父 Workflow Provenance 只绑定 run/tenant/task/agent/principal 身份,不直接授予
|
||||||
|
操作权限。每个子 Grant 只允许一个 ExecutionPath;Git Grant 还必须限定具体
|
||||||
|
GitOperation。子 Grant 具备独立 TTL、nonce、状态、撤销和重放防护。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sqlite3
|
||||||
|
import threading
|
||||||
|
import uuid
|
||||||
|
from dataclasses import asdict, dataclass, field, replace
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from enum import Enum
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Sequence
|
||||||
|
|
||||||
|
from .provenance import (
|
||||||
|
ExecutionPath,
|
||||||
|
GitOperation,
|
||||||
|
ProvenanceDenied,
|
||||||
|
ProvenanceExpired,
|
||||||
|
ProvenanceIssuer,
|
||||||
|
ProvenanceNotFound,
|
||||||
|
ProvenanceReplay,
|
||||||
|
ProvenanceRevoked,
|
||||||
|
ProvenanceSignatureError,
|
||||||
|
ProvenanceStatus,
|
||||||
|
ProvenanceVerifier,
|
||||||
|
RunProvenance,
|
||||||
|
SQLiteProvenanceStore,
|
||||||
|
Signer,
|
||||||
|
SignatureVerifier,
|
||||||
|
_validate_digest,
|
||||||
|
canonical_json_bytes,
|
||||||
|
parse_timestamp,
|
||||||
|
required,
|
||||||
|
utc_now,
|
||||||
|
)
|
||||||
|
|
||||||
|
WORKFLOW_POLICY_VERSION = "aios.v2.workflow.v1"
|
||||||
|
GRANT_SCHEMA_VERSION = "aios.v2.provenance-grant.v1"
|
||||||
|
|
||||||
|
|
||||||
|
class GrantType(str, Enum):
|
||||||
|
CODE_CHANGE = "code_change"
|
||||||
|
GIT = "git"
|
||||||
|
DEPLOYMENT = "deployment"
|
||||||
|
RUNTIME = "runtime"
|
||||||
|
EVIDENCE = "evidence"
|
||||||
|
ROLLBACK = "rollback"
|
||||||
|
AUDIT = "audit"
|
||||||
|
|
||||||
|
|
||||||
|
GRANT_PATHS: Mapping[GrantType, ExecutionPath] = {
|
||||||
|
GrantType.CODE_CHANGE: ExecutionPath.CODE_CHANGE,
|
||||||
|
GrantType.GIT: ExecutionPath.GIT_OPERATION,
|
||||||
|
GrantType.DEPLOYMENT: ExecutionPath.DEPLOYMENT,
|
||||||
|
GrantType.RUNTIME: ExecutionPath.RUNTIME_EXECUTE,
|
||||||
|
GrantType.EVIDENCE: ExecutionPath.EVIDENCE_WRITE,
|
||||||
|
GrantType.ROLLBACK: ExecutionPath.ROLLBACK,
|
||||||
|
GrantType.AUDIT: ExecutionPath.AUDIT,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class GrantStatus(str, Enum):
|
||||||
|
ISSUED = "issued"
|
||||||
|
VERIFIED = "verified"
|
||||||
|
USED = "used"
|
||||||
|
EXPIRED = "expired"
|
||||||
|
REVOKED = "revoked"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ProvenanceGrant:
|
||||||
|
grant_id: str
|
||||||
|
parent_provenance_id: str
|
||||||
|
grant_type: GrantType
|
||||||
|
execution_path: ExecutionPath
|
||||||
|
run_id: str
|
||||||
|
issuer: str
|
||||||
|
tenant_id: str
|
||||||
|
task_id: str
|
||||||
|
agent_id: str
|
||||||
|
principal_id: str
|
||||||
|
allowed_git_operations: Sequence[str]
|
||||||
|
issued_at: str
|
||||||
|
expires_at: str
|
||||||
|
policy_version: str
|
||||||
|
key_id: str
|
||||||
|
nonce: str
|
||||||
|
signature: str = ""
|
||||||
|
schema_version: str = GRANT_SCHEMA_VERSION
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for name in (
|
||||||
|
"grant_id",
|
||||||
|
"parent_provenance_id",
|
||||||
|
"run_id",
|
||||||
|
"issuer",
|
||||||
|
"tenant_id",
|
||||||
|
"task_id",
|
||||||
|
"agent_id",
|
||||||
|
"principal_id",
|
||||||
|
"issued_at",
|
||||||
|
"expires_at",
|
||||||
|
"policy_version",
|
||||||
|
"key_id",
|
||||||
|
"nonce",
|
||||||
|
"schema_version",
|
||||||
|
):
|
||||||
|
object.__setattr__(self, name, required(getattr(self, name), name))
|
||||||
|
if self.schema_version != GRANT_SCHEMA_VERSION:
|
||||||
|
raise ValueError(f"unsupported grant schema: {self.schema_version}")
|
||||||
|
grant_type = GrantType(self.grant_type)
|
||||||
|
path = ExecutionPath(self.execution_path)
|
||||||
|
if GRANT_PATHS[grant_type] != path:
|
||||||
|
raise ValueError("grant_type does not match execution_path")
|
||||||
|
object.__setattr__(self, "grant_type", grant_type)
|
||||||
|
object.__setattr__(self, "execution_path", path)
|
||||||
|
operations = tuple(
|
||||||
|
sorted({GitOperation(item).value for item in self.allowed_git_operations})
|
||||||
|
)
|
||||||
|
if grant_type == GrantType.GIT and not operations:
|
||||||
|
raise ValueError("Git Grant requires explicit Git operations")
|
||||||
|
if grant_type != GrantType.GIT and operations:
|
||||||
|
raise ValueError("only Git Grant may contain Git operations")
|
||||||
|
object.__setattr__(self, "allowed_git_operations", operations)
|
||||||
|
issued = parse_timestamp(self.issued_at)
|
||||||
|
expires = parse_timestamp(self.expires_at)
|
||||||
|
if expires <= issued:
|
||||||
|
raise ValueError("grant expires_at must be later than issued_at")
|
||||||
|
object.__setattr__(self, "signature", str(self.signature or "").strip())
|
||||||
|
|
||||||
|
def unsigned_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"schema_version": self.schema_version,
|
||||||
|
"grant_id": self.grant_id,
|
||||||
|
"parent_provenance_id": self.parent_provenance_id,
|
||||||
|
"grant_type": self.grant_type.value,
|
||||||
|
"execution_path": self.execution_path.value,
|
||||||
|
"run_id": self.run_id,
|
||||||
|
"issuer": self.issuer,
|
||||||
|
"tenant_id": self.tenant_id,
|
||||||
|
"task_id": self.task_id,
|
||||||
|
"agent_id": self.agent_id,
|
||||||
|
"principal_id": self.principal_id,
|
||||||
|
"allowed_git_operations": list(self.allowed_git_operations),
|
||||||
|
"issued_at": self.issued_at,
|
||||||
|
"expires_at": self.expires_at,
|
||||||
|
"policy_version": self.policy_version,
|
||||||
|
"key_id": self.key_id,
|
||||||
|
"nonce": self.nonce,
|
||||||
|
}
|
||||||
|
|
||||||
|
def canonical_bytes(self) -> bytes:
|
||||||
|
return canonical_json_bytes(self.unsigned_dict())
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
value = self.unsigned_dict()
|
||||||
|
value["signature"] = self.signature
|
||||||
|
return value
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, value: Mapping[str, Any]) -> "ProvenanceGrant":
|
||||||
|
return cls(**dict(value))
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class GrantAuthorization:
|
||||||
|
authorization_id: str
|
||||||
|
grant_id: str
|
||||||
|
parent_provenance_id: str
|
||||||
|
run_id: str
|
||||||
|
tenant_id: str
|
||||||
|
task_id: str
|
||||||
|
agent_id: str
|
||||||
|
principal_id: str
|
||||||
|
execution_path: str
|
||||||
|
git_operation: str | None
|
||||||
|
request_digest: str
|
||||||
|
policy_version: str
|
||||||
|
key_id: str
|
||||||
|
public_key_fingerprint: str
|
||||||
|
authorized_at: str = field(default_factory=utc_now)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def provenance_id(self) -> str:
|
||||||
|
return self.parent_provenance_id
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return asdict(self)
|
||||||
|
|
||||||
|
|
||||||
|
class WorkflowProvenanceIssuer:
|
||||||
|
"""签发不携带操作权限的父 Workflow Provenance。"""
|
||||||
|
|
||||||
|
def __init__(self, signer: Signer, store: SQLiteProvenanceStore) -> None:
|
||||||
|
self._issuer = ProvenanceIssuer(signer, store)
|
||||||
|
|
||||||
|
def issue(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
run_id: str,
|
||||||
|
issuer: str,
|
||||||
|
tenant_id: str,
|
||||||
|
task_id: str,
|
||||||
|
agent_id: str,
|
||||||
|
principal_id: str,
|
||||||
|
ttl_seconds: int = 14400,
|
||||||
|
provenance_id: str | None = None,
|
||||||
|
nonce: str | None = None,
|
||||||
|
now: datetime | None = None,
|
||||||
|
) -> RunProvenance:
|
||||||
|
return self._issuer.issue(
|
||||||
|
run_id=run_id,
|
||||||
|
issuer=issuer,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
task_id=task_id,
|
||||||
|
agent_id=agent_id,
|
||||||
|
principal_id=principal_id,
|
||||||
|
allowed_paths=(),
|
||||||
|
allowed_git_operations=(),
|
||||||
|
ttl_seconds=ttl_seconds,
|
||||||
|
policy_version=WORKFLOW_POLICY_VERSION,
|
||||||
|
provenance_id=provenance_id,
|
||||||
|
nonce=nonce,
|
||||||
|
now=now,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class SQLiteGrantStore:
|
||||||
|
"""在 Provenance SQLite 数据库中持久化子 Grant 与使用记录。"""
|
||||||
|
|
||||||
|
def __init__(self, path: str | Path) -> None:
|
||||||
|
self.path = Path(path).resolve()
|
||||||
|
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
self._connection = sqlite3.connect(
|
||||||
|
str(self.path), timeout=30.0, check_same_thread=False
|
||||||
|
)
|
||||||
|
self._connection.row_factory = sqlite3.Row
|
||||||
|
with self._lock:
|
||||||
|
self._connection.execute("PRAGMA journal_mode=WAL")
|
||||||
|
self._connection.execute("PRAGMA synchronous=FULL")
|
||||||
|
self._connection.execute("PRAGMA foreign_keys=ON")
|
||||||
|
self._connection.executescript(
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS provenance_grants (
|
||||||
|
grant_id TEXT PRIMARY KEY,
|
||||||
|
parent_provenance_id TEXT NOT NULL,
|
||||||
|
grant_type TEXT NOT NULL,
|
||||||
|
execution_path TEXT NOT NULL,
|
||||||
|
payload_json TEXT NOT NULL,
|
||||||
|
status TEXT NOT NULL,
|
||||||
|
signature_verified INTEGER NOT NULL DEFAULT 0,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
updated_at TEXT NOT NULL
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_grants_parent
|
||||||
|
ON provenance_grants(parent_provenance_id, grant_type, status);
|
||||||
|
CREATE TABLE IF NOT EXISTS provenance_grant_usage (
|
||||||
|
authorization_id TEXT PRIMARY KEY,
|
||||||
|
grant_id TEXT NOT NULL,
|
||||||
|
execution_path TEXT NOT NULL,
|
||||||
|
git_operation TEXT NOT NULL DEFAULT '',
|
||||||
|
request_digest TEXT NOT NULL,
|
||||||
|
authorized_at TEXT NOT NULL,
|
||||||
|
receipt_json TEXT NOT NULL,
|
||||||
|
UNIQUE(grant_id, execution_path, git_operation, request_digest),
|
||||||
|
FOREIGN KEY(grant_id) REFERENCES provenance_grants(grant_id)
|
||||||
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS provenance_grant_events (
|
||||||
|
event_id TEXT PRIMARY KEY,
|
||||||
|
grant_id TEXT NOT NULL,
|
||||||
|
event_type TEXT NOT NULL,
|
||||||
|
details_json TEXT NOT NULL,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
FOREIGN KEY(grant_id) REFERENCES provenance_grants(grant_id)
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
self._connection.commit()
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
with self._lock:
|
||||||
|
self._connection.close()
|
||||||
|
|
||||||
|
def _event(
|
||||||
|
self,
|
||||||
|
grant_id: str,
|
||||||
|
event_type: str,
|
||||||
|
details: Mapping[str, Any] | None = None,
|
||||||
|
) -> None:
|
||||||
|
self._connection.execute(
|
||||||
|
"INSERT INTO provenance_grant_events VALUES (?, ?, ?, ?, ?)",
|
||||||
|
(
|
||||||
|
f"gevt_{uuid.uuid4().hex}",
|
||||||
|
grant_id,
|
||||||
|
event_type,
|
||||||
|
json.dumps(dict(details or {}), ensure_ascii=False, sort_keys=True),
|
||||||
|
utc_now(),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
def save_issued(self, grant: ProvenanceGrant) -> None:
|
||||||
|
payload = json.dumps(
|
||||||
|
grant.as_dict(), ensure_ascii=False, sort_keys=True, separators=(",", ":")
|
||||||
|
)
|
||||||
|
now = utc_now()
|
||||||
|
with self._lock:
|
||||||
|
self._connection.execute("BEGIN IMMEDIATE")
|
||||||
|
try:
|
||||||
|
self._connection.execute(
|
||||||
|
"INSERT INTO provenance_grants VALUES (?, ?, ?, ?, ?, ?, 0, ?, ?)",
|
||||||
|
(
|
||||||
|
grant.grant_id,
|
||||||
|
grant.parent_provenance_id,
|
||||||
|
grant.grant_type.value,
|
||||||
|
grant.execution_path.value,
|
||||||
|
payload,
|
||||||
|
GrantStatus.ISSUED.value,
|
||||||
|
now,
|
||||||
|
now,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
self._event(grant.grant_id, "issued")
|
||||||
|
self._connection.commit()
|
||||||
|
except sqlite3.IntegrityError as exc:
|
||||||
|
self._connection.rollback()
|
||||||
|
raise ProvenanceDenied("grant_id already exists") from exc
|
||||||
|
except Exception:
|
||||||
|
self._connection.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
def get(self, grant_id: str) -> ProvenanceGrant:
|
||||||
|
with self._lock:
|
||||||
|
row = self._connection.execute(
|
||||||
|
"SELECT payload_json FROM provenance_grants WHERE grant_id = ?",
|
||||||
|
(required(grant_id, "grant_id"),),
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise ProvenanceNotFound("provenance grant not found")
|
||||||
|
value = json.loads(row["payload_json"])
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise ProvenanceDenied("stored grant is invalid")
|
||||||
|
return ProvenanceGrant.from_dict(value)
|
||||||
|
|
||||||
|
def status(self, grant_id: str) -> GrantStatus:
|
||||||
|
with self._lock:
|
||||||
|
row = self._connection.execute(
|
||||||
|
"SELECT status FROM provenance_grants WHERE grant_id = ?",
|
||||||
|
(required(grant_id, "grant_id"),),
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise ProvenanceNotFound("provenance grant not found")
|
||||||
|
return GrantStatus(row["status"])
|
||||||
|
|
||||||
|
def mark_verified(self, grant_id: str, *, fingerprint: str) -> None:
|
||||||
|
with self._lock:
|
||||||
|
self._connection.execute("BEGIN IMMEDIATE")
|
||||||
|
try:
|
||||||
|
current = self.status(grant_id)
|
||||||
|
if current in {GrantStatus.REVOKED, GrantStatus.EXPIRED}:
|
||||||
|
raise ProvenanceDenied(f"cannot verify grant in state {current.value}")
|
||||||
|
next_status = (
|
||||||
|
GrantStatus.USED.value
|
||||||
|
if current == GrantStatus.USED
|
||||||
|
else GrantStatus.VERIFIED.value
|
||||||
|
)
|
||||||
|
self._connection.execute(
|
||||||
|
"UPDATE provenance_grants SET status = ?, signature_verified = 1, updated_at = ? WHERE grant_id = ?",
|
||||||
|
(next_status, utc_now(), grant_id),
|
||||||
|
)
|
||||||
|
self._event(
|
||||||
|
grant_id,
|
||||||
|
"verified",
|
||||||
|
{"public_key_fingerprint": fingerprint},
|
||||||
|
)
|
||||||
|
self._connection.commit()
|
||||||
|
except Exception:
|
||||||
|
self._connection.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
def mark_expired(self, grant_id: str) -> None:
|
||||||
|
with self._lock:
|
||||||
|
self._connection.execute("BEGIN IMMEDIATE")
|
||||||
|
try:
|
||||||
|
if self.status(grant_id) == GrantStatus.REVOKED:
|
||||||
|
self._connection.rollback()
|
||||||
|
return
|
||||||
|
self._connection.execute(
|
||||||
|
"UPDATE provenance_grants SET status = ?, updated_at = ? WHERE grant_id = ?",
|
||||||
|
(GrantStatus.EXPIRED.value, utc_now(), grant_id),
|
||||||
|
)
|
||||||
|
self._event(grant_id, "expired")
|
||||||
|
self._connection.commit()
|
||||||
|
except Exception:
|
||||||
|
self._connection.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
def revoke(self, grant_id: str, *, reason: str, principal_id: str) -> None:
|
||||||
|
with self._lock:
|
||||||
|
self._connection.execute("BEGIN IMMEDIATE")
|
||||||
|
try:
|
||||||
|
self.get(grant_id)
|
||||||
|
self._connection.execute(
|
||||||
|
"UPDATE provenance_grants SET status = ?, updated_at = ? WHERE grant_id = ?",
|
||||||
|
(GrantStatus.REVOKED.value, utc_now(), grant_id),
|
||||||
|
)
|
||||||
|
self._event(
|
||||||
|
grant_id,
|
||||||
|
"revoked",
|
||||||
|
{
|
||||||
|
"reason": required(reason, "reason"),
|
||||||
|
"principal_id": required(principal_id, "principal_id"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
self._connection.commit()
|
||||||
|
except Exception:
|
||||||
|
self._connection.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
def record_authorization(self, authorization: GrantAuthorization) -> None:
|
||||||
|
receipt = json.dumps(
|
||||||
|
authorization.as_dict(), ensure_ascii=False, sort_keys=True, separators=(",", ":")
|
||||||
|
)
|
||||||
|
with self._lock:
|
||||||
|
self._connection.execute("BEGIN IMMEDIATE")
|
||||||
|
try:
|
||||||
|
current = self.status(authorization.grant_id)
|
||||||
|
if current in {GrantStatus.REVOKED, GrantStatus.EXPIRED}:
|
||||||
|
raise ProvenanceDenied(f"grant state is {current.value}")
|
||||||
|
try:
|
||||||
|
self._connection.execute(
|
||||||
|
"INSERT INTO provenance_grant_usage VALUES (?, ?, ?, ?, ?, ?, ?)",
|
||||||
|
(
|
||||||
|
authorization.authorization_id,
|
||||||
|
authorization.grant_id,
|
||||||
|
authorization.execution_path,
|
||||||
|
authorization.git_operation or "",
|
||||||
|
authorization.request_digest,
|
||||||
|
authorization.authorized_at,
|
||||||
|
receipt,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
except sqlite3.IntegrityError as exc:
|
||||||
|
raise ProvenanceReplay(
|
||||||
|
"identical grant path/operation/request digest already used"
|
||||||
|
) from exc
|
||||||
|
self._connection.execute(
|
||||||
|
"UPDATE provenance_grants SET status = ?, updated_at = ? WHERE grant_id = ?",
|
||||||
|
(GrantStatus.USED.value, authorization.authorized_at, authorization.grant_id),
|
||||||
|
)
|
||||||
|
self._event(
|
||||||
|
authorization.grant_id,
|
||||||
|
"used",
|
||||||
|
{
|
||||||
|
"authorization_id": authorization.authorization_id,
|
||||||
|
"execution_path": authorization.execution_path,
|
||||||
|
"git_operation": authorization.git_operation,
|
||||||
|
"request_digest": authorization.request_digest,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
self._connection.commit()
|
||||||
|
except Exception:
|
||||||
|
self._connection.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
def list_for_parent(self, parent_provenance_id: str) -> list[ProvenanceGrant]:
|
||||||
|
with self._lock:
|
||||||
|
rows = self._connection.execute(
|
||||||
|
"SELECT payload_json FROM provenance_grants WHERE parent_provenance_id = ? ORDER BY grant_type, grant_id",
|
||||||
|
(required(parent_provenance_id, "parent_provenance_id"),),
|
||||||
|
).fetchall()
|
||||||
|
return [ProvenanceGrant.from_dict(json.loads(row["payload_json"])) for row in rows]
|
||||||
|
|
||||||
|
|
||||||
|
class GrantIssuer:
|
||||||
|
"""从父 Workflow Provenance 派生单一路径最小权限 Grant。"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
signer: Signer,
|
||||||
|
parent_store: SQLiteProvenanceStore,
|
||||||
|
grant_store: SQLiteGrantStore,
|
||||||
|
) -> None:
|
||||||
|
self.signer = signer
|
||||||
|
self.parent_store = parent_store
|
||||||
|
self.grant_store = grant_store
|
||||||
|
|
||||||
|
def issue(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
parent_provenance_id: str,
|
||||||
|
grant_type: GrantType | str,
|
||||||
|
ttl_seconds: int = 900,
|
||||||
|
allowed_git_operations: Sequence[str] = (),
|
||||||
|
grant_id: str | None = None,
|
||||||
|
nonce: str | None = None,
|
||||||
|
now: datetime | None = None,
|
||||||
|
) -> ProvenanceGrant:
|
||||||
|
if ttl_seconds <= 0:
|
||||||
|
raise ValueError("grant ttl_seconds must be positive")
|
||||||
|
parent = self.parent_store.get(parent_provenance_id)
|
||||||
|
if parent.policy_version != WORKFLOW_POLICY_VERSION:
|
||||||
|
raise ProvenanceDenied("child grants require a workflow provenance parent")
|
||||||
|
if parent.allowed_paths or parent.allowed_git_operations:
|
||||||
|
raise ProvenanceDenied("workflow provenance parent must not contain operation rights")
|
||||||
|
parent_status = self.parent_store.status(parent.provenance_id)
|
||||||
|
if parent_status == ProvenanceStatus.REVOKED:
|
||||||
|
raise ProvenanceRevoked("workflow provenance is revoked")
|
||||||
|
issued = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
|
||||||
|
parent_expiry = parse_timestamp(parent.expires_at)
|
||||||
|
expires = min(issued + timedelta(seconds=ttl_seconds), parent_expiry)
|
||||||
|
if expires <= issued:
|
||||||
|
raise ProvenanceExpired("workflow provenance has expired")
|
||||||
|
normalized_type = GrantType(grant_type)
|
||||||
|
operations = tuple(allowed_git_operations)
|
||||||
|
unsigned = ProvenanceGrant(
|
||||||
|
grant_id=grant_id or f"grant_{uuid.uuid4().hex}",
|
||||||
|
parent_provenance_id=parent.provenance_id,
|
||||||
|
grant_type=normalized_type,
|
||||||
|
execution_path=GRANT_PATHS[normalized_type],
|
||||||
|
run_id=parent.run_id,
|
||||||
|
issuer=parent.issuer,
|
||||||
|
tenant_id=parent.tenant_id,
|
||||||
|
task_id=parent.task_id,
|
||||||
|
agent_id=parent.agent_id,
|
||||||
|
principal_id=parent.principal_id,
|
||||||
|
allowed_git_operations=operations,
|
||||||
|
issued_at=issued.isoformat().replace("+00:00", "Z"),
|
||||||
|
expires_at=expires.isoformat().replace("+00:00", "Z"),
|
||||||
|
policy_version=f"{WORKFLOW_POLICY_VERSION}.{normalized_type.value}",
|
||||||
|
key_id=self.signer.key_id,
|
||||||
|
nonce=nonce or uuid.uuid4().hex,
|
||||||
|
)
|
||||||
|
signed = replace(unsigned, signature=self.signer.sign(unsigned.canonical_bytes()))
|
||||||
|
self.grant_store.save_issued(signed)
|
||||||
|
return signed
|
||||||
|
|
||||||
|
|
||||||
|
class GrantVerifier:
|
||||||
|
"""验证父 Workflow 与子 Grant,确保单一路径最小权限。"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
parent_store: SQLiteProvenanceStore,
|
||||||
|
grant_store: SQLiteGrantStore,
|
||||||
|
verifiers: Mapping[str, SignatureVerifier],
|
||||||
|
) -> None:
|
||||||
|
if not verifiers:
|
||||||
|
raise ValueError("at least one trusted verifier is required")
|
||||||
|
self.parent_store = parent_store
|
||||||
|
self.grant_store = grant_store
|
||||||
|
self.verifiers = {required(key, "key_id"): value for key, value in verifiers.items()}
|
||||||
|
|
||||||
|
def _verify_parent(self, parent: RunProvenance, *, now: datetime) -> SignatureVerifier:
|
||||||
|
status = self.parent_store.status(parent.provenance_id)
|
||||||
|
if status == ProvenanceStatus.REVOKED:
|
||||||
|
raise ProvenanceRevoked("workflow provenance is revoked")
|
||||||
|
if now >= parse_timestamp(parent.expires_at):
|
||||||
|
self.parent_store.mark_expired(parent.provenance_id)
|
||||||
|
raise ProvenanceExpired("workflow provenance is expired")
|
||||||
|
verifier = self.verifiers.get(parent.key_id)
|
||||||
|
if verifier is None or not parent.signature or not verifier.verify(
|
||||||
|
parent.canonical_bytes(), parent.signature
|
||||||
|
):
|
||||||
|
raise ProvenanceSignatureError("invalid workflow provenance signature")
|
||||||
|
if parent.policy_version != WORKFLOW_POLICY_VERSION:
|
||||||
|
raise ProvenanceDenied("parent is not a workflow provenance")
|
||||||
|
if parent.allowed_paths or parent.allowed_git_operations:
|
||||||
|
raise ProvenanceDenied("workflow provenance must not contain operation rights")
|
||||||
|
return verifier
|
||||||
|
|
||||||
|
def authorize(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
grant_id: str,
|
||||||
|
parent_provenance_id: str,
|
||||||
|
run_id: str,
|
||||||
|
tenant_id: str,
|
||||||
|
task_id: str,
|
||||||
|
agent_id: str,
|
||||||
|
principal_id: str,
|
||||||
|
execution_path: str,
|
||||||
|
request_digest: str,
|
||||||
|
git_operation: str | None = None,
|
||||||
|
now: datetime | None = None,
|
||||||
|
) -> GrantAuthorization:
|
||||||
|
current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
|
||||||
|
parent = self.parent_store.get(parent_provenance_id)
|
||||||
|
self._verify_parent(parent, now=current)
|
||||||
|
grant = self.grant_store.get(grant_id)
|
||||||
|
grant_status = self.grant_store.status(grant.grant_id)
|
||||||
|
if grant_status == GrantStatus.REVOKED:
|
||||||
|
raise ProvenanceRevoked("provenance grant is revoked")
|
||||||
|
if current >= parse_timestamp(grant.expires_at):
|
||||||
|
self.grant_store.mark_expired(grant.grant_id)
|
||||||
|
raise ProvenanceExpired("provenance grant is expired")
|
||||||
|
verifier = self.verifiers.get(grant.key_id)
|
||||||
|
if verifier is None or not grant.signature or not verifier.verify(
|
||||||
|
grant.canonical_bytes(), grant.signature
|
||||||
|
):
|
||||||
|
raise ProvenanceSignatureError("invalid provenance grant signature")
|
||||||
|
expected = {
|
||||||
|
"parent_provenance_id": parent.provenance_id,
|
||||||
|
"run_id": required(run_id, "run_id"),
|
||||||
|
"tenant_id": required(tenant_id, "tenant_id"),
|
||||||
|
"task_id": required(task_id, "task_id"),
|
||||||
|
"agent_id": required(agent_id, "agent_id"),
|
||||||
|
"principal_id": required(principal_id, "principal_id"),
|
||||||
|
}
|
||||||
|
for field_name, expected_value in expected.items():
|
||||||
|
if getattr(grant, field_name) != expected_value:
|
||||||
|
raise ProvenanceDenied(f"grant {field_name} mismatch")
|
||||||
|
if field_name != "parent_provenance_id" and getattr(parent, field_name) != expected_value:
|
||||||
|
raise ProvenanceDenied(f"workflow {field_name} mismatch")
|
||||||
|
path = ExecutionPath(execution_path)
|
||||||
|
if grant.execution_path != path:
|
||||||
|
raise ProvenanceDenied(
|
||||||
|
f"grant permits {grant.execution_path.value}, not {path.value}"
|
||||||
|
)
|
||||||
|
operation: str | None = None
|
||||||
|
if path == ExecutionPath.GIT_OPERATION:
|
||||||
|
if git_operation is None:
|
||||||
|
raise ProvenanceDenied("Git operation is required")
|
||||||
|
operation = GitOperation(git_operation).value
|
||||||
|
if operation not in grant.allowed_git_operations:
|
||||||
|
raise ProvenanceDenied(f"Git operation is not allowed: {operation}")
|
||||||
|
elif git_operation is not None:
|
||||||
|
raise ProvenanceDenied("git_operation is only valid for a Git Grant")
|
||||||
|
digest = _validate_digest(request_digest)
|
||||||
|
self.grant_store.mark_verified(grant.grant_id, fingerprint=verifier.fingerprint)
|
||||||
|
authorization = GrantAuthorization(
|
||||||
|
authorization_id=f"gauth_{uuid.uuid4().hex}",
|
||||||
|
grant_id=grant.grant_id,
|
||||||
|
parent_provenance_id=parent.provenance_id,
|
||||||
|
run_id=grant.run_id,
|
||||||
|
tenant_id=grant.tenant_id,
|
||||||
|
task_id=grant.task_id,
|
||||||
|
agent_id=grant.agent_id,
|
||||||
|
principal_id=grant.principal_id,
|
||||||
|
execution_path=path.value,
|
||||||
|
git_operation=operation,
|
||||||
|
request_digest=digest,
|
||||||
|
policy_version=grant.policy_version,
|
||||||
|
key_id=grant.key_id,
|
||||||
|
public_key_fingerprint=verifier.fingerprint,
|
||||||
|
)
|
||||||
|
self.grant_store.record_authorization(authorization)
|
||||||
|
return authorization
|
||||||
|
|
||||||
|
|
||||||
|
class LeastPrivilegeProvenanceWriter:
|
||||||
|
"""按路径选择独立子 Grant 的生产 Writer。
|
||||||
|
|
||||||
|
``provenance_id`` 参数始终指父 Workflow Provenance;Grant ID 来自受保护
|
||||||
|
配置映射,调用方无法通过请求体临时扩大权限。
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
verifier: GrantVerifier,
|
||||||
|
receipt_root: str | Path,
|
||||||
|
grant_ids: Mapping[str, str],
|
||||||
|
*,
|
||||||
|
writer_id: str = "aios-v2-least-privilege-writer",
|
||||||
|
) -> None:
|
||||||
|
self.verifier = verifier
|
||||||
|
self.receipt_root = Path(receipt_root).resolve()
|
||||||
|
self.receipt_root.mkdir(parents=True, exist_ok=True)
|
||||||
|
self.grant_ids = {
|
||||||
|
ExecutionPath(path).value: required(grant_id, f"grant_id[{path}]")
|
||||||
|
for path, grant_id in grant_ids.items()
|
||||||
|
}
|
||||||
|
missing = sorted(path.value for path in ExecutionPath if path.value not in self.grant_ids)
|
||||||
|
if missing:
|
||||||
|
raise ValueError(f"missing least-privilege Grant mappings: {missing}")
|
||||||
|
self.writer_id = required(writer_id, "writer_id")
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _component(value: str) -> str:
|
||||||
|
normalized = required(value, "path component")
|
||||||
|
if normalized in {".", ".."} or "/" in normalized or "\\" in normalized:
|
||||||
|
raise ProvenanceDenied("invalid receipt path component")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
def authorize(self, **values: Any) -> GrantAuthorization:
|
||||||
|
path = ExecutionPath(values["execution_path"]).value
|
||||||
|
grant_id = self.grant_ids[path]
|
||||||
|
parent_provenance_id = values.pop("provenance_id")
|
||||||
|
authorization = self.verifier.authorize(
|
||||||
|
grant_id=grant_id,
|
||||||
|
parent_provenance_id=parent_provenance_id,
|
||||||
|
**values,
|
||||||
|
)
|
||||||
|
tenant = self._component(authorization.tenant_id)
|
||||||
|
parent = self._component(authorization.parent_provenance_id)
|
||||||
|
grant = self._component(authorization.grant_id)
|
||||||
|
target_dir = self.receipt_root / tenant / parent / grant
|
||||||
|
target = target_dir / f"{authorization.authorization_id}.json"
|
||||||
|
payload = authorization.as_dict()
|
||||||
|
payload["writer_id"] = self.writer_id
|
||||||
|
with self._lock:
|
||||||
|
target_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
temporary = target.with_suffix(f".tmp-{os.getpid()}")
|
||||||
|
temporary.write_text(
|
||||||
|
json.dumps(payload, ensure_ascii=False, sort_keys=True, indent=2) + "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
os.chmod(temporary, 0o640)
|
||||||
|
os.replace(temporary, target)
|
||||||
|
return authorization
|
||||||
|
|
||||||
|
def authorize_code_change(self, **values: Any) -> GrantAuthorization:
|
||||||
|
return self.authorize(execution_path=ExecutionPath.CODE_CHANGE.value, **values)
|
||||||
|
|
||||||
|
def authorize_git_operation(
|
||||||
|
self, *, git_operation: str, **values: Any
|
||||||
|
) -> GrantAuthorization:
|
||||||
|
return self.authorize(
|
||||||
|
execution_path=ExecutionPath.GIT_OPERATION.value,
|
||||||
|
git_operation=git_operation,
|
||||||
|
**values,
|
||||||
|
)
|
||||||
|
|
||||||
|
def authorize_deployment(self, **values: Any) -> GrantAuthorization:
|
||||||
|
return self.authorize(execution_path=ExecutionPath.DEPLOYMENT.value, **values)
|
||||||
|
|
||||||
|
def authorize_runtime_execute(self, **values: Any) -> GrantAuthorization:
|
||||||
|
return self.authorize(execution_path=ExecutionPath.RUNTIME_EXECUTE.value, **values)
|
||||||
|
|
||||||
|
def authorize_evidence_write(self, **values: Any) -> GrantAuthorization:
|
||||||
|
return self.authorize(execution_path=ExecutionPath.EVIDENCE_WRITE.value, **values)
|
||||||
|
|
||||||
|
def authorize_rollback(self, **values: Any) -> GrantAuthorization:
|
||||||
|
return self.authorize(execution_path=ExecutionPath.ROLLBACK.value, **values)
|
||||||
|
|
||||||
|
def authorize_audit(self, **values: Any) -> GrantAuthorization:
|
||||||
|
return self.authorize(execution_path=ExecutionPath.AUDIT.value, **values)
|
||||||
@@ -0,0 +1,265 @@
|
|||||||
|
"""AIOS V2 运维操作 Provenance Gate 与服务级 Scoped Writer 工厂。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from dataclasses import asdict, dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Protocol, Sequence
|
||||||
|
|
||||||
|
from .provenance import (
|
||||||
|
Ed25519PublicKeyVerifier,
|
||||||
|
ExecutionPath,
|
||||||
|
ProvenanceAuthorization,
|
||||||
|
ProvenanceControlledWriter,
|
||||||
|
ProvenanceDenied,
|
||||||
|
ProvenanceVerifier,
|
||||||
|
SQLiteProvenanceStore,
|
||||||
|
digest_payload,
|
||||||
|
)
|
||||||
|
from .provenance_grants import GrantAuthorization, GrantVerifier, SQLiteGrantStore
|
||||||
|
from .provenance_scoped import ScopedLeastPrivilegeProvenanceWriter
|
||||||
|
|
||||||
|
|
||||||
|
APPLICATION_PROVENANCE_SCOPE = (
|
||||||
|
ExecutionPath.RUNTIME_EXECUTE.value,
|
||||||
|
ExecutionPath.EVIDENCE_WRITE.value,
|
||||||
|
ExecutionPath.AUDIT.value,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class AuthorizationWriter(Protocol):
|
||||||
|
def authorize_deployment(self, **values: Any) -> Any: ...
|
||||||
|
def authorize_rollback(self, **values: Any) -> Any: ...
|
||||||
|
def authorize_audit(self, **values: Any) -> Any: ...
|
||||||
|
def authorize_code_change(self, **values: Any) -> Any: ...
|
||||||
|
def authorize_evidence_write(self, **values: Any) -> Any: ...
|
||||||
|
def authorize_runtime_execute(self, **values: Any) -> Any: ...
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class OperationContext:
|
||||||
|
provenance_id: str
|
||||||
|
run_id: str
|
||||||
|
tenant_id: str
|
||||||
|
task_id: str
|
||||||
|
agent_id: str
|
||||||
|
principal_id: str
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, str]:
|
||||||
|
return asdict(self)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class OperationReceipt:
|
||||||
|
operation: str
|
||||||
|
request_digest: str
|
||||||
|
authorization_id: str
|
||||||
|
provenance_id: str
|
||||||
|
grant_id: str | None
|
||||||
|
run_id: str
|
||||||
|
tenant_id: str
|
||||||
|
task_id: str
|
||||||
|
agent_id: str
|
||||||
|
principal_id: str
|
||||||
|
execution_path: str
|
||||||
|
authorized_at: str
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return asdict(self)
|
||||||
|
|
||||||
|
|
||||||
|
class ProvenanceOperationGate:
|
||||||
|
"""部署、回滚、审计和运行操作的统一授权入口。"""
|
||||||
|
|
||||||
|
def __init__(self, writer: AuthorizationWriter) -> None:
|
||||||
|
self.writer = writer
|
||||||
|
|
||||||
|
def authorize(
|
||||||
|
self,
|
||||||
|
context: OperationContext,
|
||||||
|
*,
|
||||||
|
execution_path: str,
|
||||||
|
operation: str,
|
||||||
|
request: Mapping[str, Any],
|
||||||
|
) -> OperationReceipt:
|
||||||
|
path = ExecutionPath(execution_path).value
|
||||||
|
normalized_operation = str(operation or "").strip()
|
||||||
|
if not normalized_operation:
|
||||||
|
raise ValueError("operation is required")
|
||||||
|
digest = digest_payload(
|
||||||
|
{
|
||||||
|
"operation": normalized_operation,
|
||||||
|
"execution_path": path,
|
||||||
|
"request": dict(request),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
values = {
|
||||||
|
"provenance_id": context.provenance_id,
|
||||||
|
"run_id": context.run_id,
|
||||||
|
"tenant_id": context.tenant_id,
|
||||||
|
"task_id": context.task_id,
|
||||||
|
"agent_id": context.agent_id,
|
||||||
|
"principal_id": context.principal_id,
|
||||||
|
"request_digest": digest,
|
||||||
|
}
|
||||||
|
if path == ExecutionPath.DEPLOYMENT.value:
|
||||||
|
authorization = self.writer.authorize_deployment(**values)
|
||||||
|
elif path == ExecutionPath.ROLLBACK.value:
|
||||||
|
authorization = self.writer.authorize_rollback(**values)
|
||||||
|
elif path == ExecutionPath.AUDIT.value:
|
||||||
|
authorization = self.writer.authorize_audit(**values)
|
||||||
|
elif path == ExecutionPath.CODE_CHANGE.value:
|
||||||
|
authorization = self.writer.authorize_code_change(**values)
|
||||||
|
elif path == ExecutionPath.EVIDENCE_WRITE.value:
|
||||||
|
authorization = self.writer.authorize_evidence_write(**values)
|
||||||
|
elif path == ExecutionPath.RUNTIME_EXECUTE.value:
|
||||||
|
authorization = self.writer.authorize_runtime_execute(**values)
|
||||||
|
else:
|
||||||
|
raise ProvenanceDenied(
|
||||||
|
"generic operation gate does not authorize git_operation; use ProvenanceGitWriter"
|
||||||
|
)
|
||||||
|
return self._receipt(normalized_operation, digest, authorization)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _receipt(
|
||||||
|
operation: str,
|
||||||
|
request_digest: str,
|
||||||
|
authorization: ProvenanceAuthorization | GrantAuthorization | Any,
|
||||||
|
) -> OperationReceipt:
|
||||||
|
return OperationReceipt(
|
||||||
|
operation=operation,
|
||||||
|
request_digest=request_digest,
|
||||||
|
authorization_id=authorization.authorization_id,
|
||||||
|
provenance_id=authorization.provenance_id,
|
||||||
|
grant_id=getattr(authorization, "grant_id", None),
|
||||||
|
run_id=authorization.run_id,
|
||||||
|
tenant_id=authorization.tenant_id,
|
||||||
|
task_id=authorization.task_id,
|
||||||
|
agent_id=authorization.agent_id,
|
||||||
|
principal_id=authorization.principal_id,
|
||||||
|
execution_path=authorization.execution_path,
|
||||||
|
authorized_at=authorization.authorized_at,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
_GRANT_ENV_BY_PATH: Mapping[str, str] = {
|
||||||
|
ExecutionPath.CODE_CHANGE.value: "AIOS_PROVENANCE_GRANT_CODE_CHANGE_ID",
|
||||||
|
ExecutionPath.GIT_OPERATION.value: "AIOS_PROVENANCE_GRANT_GIT_ID",
|
||||||
|
ExecutionPath.DEPLOYMENT.value: "AIOS_PROVENANCE_GRANT_DEPLOYMENT_ID",
|
||||||
|
ExecutionPath.RUNTIME_EXECUTE.value: "AIOS_PROVENANCE_GRANT_RUNTIME_ID",
|
||||||
|
ExecutionPath.EVIDENCE_WRITE.value: "AIOS_PROVENANCE_GRANT_EVIDENCE_ID",
|
||||||
|
ExecutionPath.ROLLBACK.value: "AIOS_PROVENANCE_GRANT_ROLLBACK_ID",
|
||||||
|
ExecutionPath.AUDIT.value: "AIOS_PROVENANCE_GRANT_AUDIT_ID",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def grant_ids_from_environment(
|
||||||
|
*,
|
||||||
|
required_paths: Sequence[str] | None = None,
|
||||||
|
) -> dict[str, str]:
|
||||||
|
paths = tuple(
|
||||||
|
sorted(
|
||||||
|
{
|
||||||
|
ExecutionPath(path).value
|
||||||
|
for path in (required_paths or tuple(item.value for item in ExecutionPath))
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
values = {path: os.getenv(_GRANT_ENV_BY_PATH[path], "").strip() for path in paths}
|
||||||
|
missing = [_GRANT_ENV_BY_PATH[path] for path, value in values.items() if not value]
|
||||||
|
if missing:
|
||||||
|
raise RuntimeError(f"missing scoped Provenance Grant configuration: {missing}")
|
||||||
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
def build_provenance_writer_from_env(
|
||||||
|
*,
|
||||||
|
state_dir: str | Path | None = None,
|
||||||
|
required: bool | None = None,
|
||||||
|
required_paths: Sequence[str] | None = None,
|
||||||
|
writer_id: str = "aios-v2-application-grant-writer",
|
||||||
|
) -> AuthorizationWriter | None:
|
||||||
|
"""从环境构建服务级最小权限 Writer。
|
||||||
|
|
||||||
|
生产应用默认只加载 Runtime、Evidence、Audit Grant。Git、部署和回滚进程
|
||||||
|
必须显式传入自己的 ``required_paths``,不能继承应用默认范围。旧全权限 Writer
|
||||||
|
只允许在显式关闭强制模式的候选/测试环境中使用。
|
||||||
|
"""
|
||||||
|
|
||||||
|
require_value = (
|
||||||
|
bool(required)
|
||||||
|
if required is not None
|
||||||
|
else os.getenv("AIOS_REQUIRE_RUN_PROVENANCE", "0").strip().lower()
|
||||||
|
in {"1", "true", "yes", "on"}
|
||||||
|
)
|
||||||
|
public_key_file = os.getenv("AIOS_PROVENANCE_PUBLIC_KEY_FILE", "").strip()
|
||||||
|
key_id = os.getenv("AIOS_PROVENANCE_KEY_ID", "").strip()
|
||||||
|
base = Path(state_dir or os.getenv("AIOS_V2_STATE_DIR", "/var/lib/aios-v2")).resolve()
|
||||||
|
store_file = os.getenv(
|
||||||
|
"AIOS_PROVENANCE_STORE_FILE", str(base / "run-provenance.sqlite3")
|
||||||
|
).strip()
|
||||||
|
receipt_root = os.getenv(
|
||||||
|
"AIOS_PROVENANCE_RECEIPT_DIR", str(base / "provenance-receipts")
|
||||||
|
).strip()
|
||||||
|
configured = bool(public_key_file and key_id)
|
||||||
|
if not configured:
|
||||||
|
if require_value:
|
||||||
|
raise RuntimeError(
|
||||||
|
"Run Provenance is required but AIOS_PROVENANCE_PUBLIC_KEY_FILE or AIOS_PROVENANCE_KEY_ID is missing"
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
|
verifier = Ed25519PublicKeyVerifier.from_file(public_key_file, key_id=key_id)
|
||||||
|
parent_store = SQLiteProvenanceStore(store_file)
|
||||||
|
normalized_paths = tuple(
|
||||||
|
sorted(
|
||||||
|
{
|
||||||
|
ExecutionPath(path).value
|
||||||
|
for path in (
|
||||||
|
required_paths
|
||||||
|
if required_paths is not None
|
||||||
|
else APPLICATION_PROVENANCE_SCOPE
|
||||||
|
)
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
if require_value:
|
||||||
|
if not normalized_paths:
|
||||||
|
raise RuntimeError("production Run Provenance requires a non-empty service scope")
|
||||||
|
grant_ids = grant_ids_from_environment(required_paths=normalized_paths)
|
||||||
|
grant_store = SQLiteGrantStore(store_file)
|
||||||
|
trusted_grants = GrantVerifier(parent_store, grant_store, {key_id: verifier})
|
||||||
|
return ScopedLeastPrivilegeProvenanceWriter(
|
||||||
|
trusted_grants,
|
||||||
|
receipt_root,
|
||||||
|
grant_ids,
|
||||||
|
required_paths=normalized_paths,
|
||||||
|
writer_id=writer_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
legacy = ProvenanceVerifier(parent_store, {key_id: verifier})
|
||||||
|
return ProvenanceControlledWriter(legacy, receipt_root)
|
||||||
|
|
||||||
|
|
||||||
|
def context_from_environment() -> OperationContext:
|
||||||
|
values = {
|
||||||
|
"provenance_id": os.getenv("AIOS_PROVENANCE_ID", ""),
|
||||||
|
"run_id": os.getenv("AIOS_RUN_ID", ""),
|
||||||
|
"tenant_id": os.getenv("AIOS_TENANT_ID", ""),
|
||||||
|
"task_id": os.getenv("AIOS_TASK_ID", ""),
|
||||||
|
"agent_id": os.getenv("AIOS_AGENT_ID", ""),
|
||||||
|
"principal_id": os.getenv("AIOS_PRINCIPAL_ID", ""),
|
||||||
|
}
|
||||||
|
missing = [name for name, value in values.items() if not str(value).strip()]
|
||||||
|
if missing:
|
||||||
|
raise RuntimeError(f"missing Provenance context environment variables: {missing}")
|
||||||
|
return OperationContext(**{name: str(value).strip() for name, value in values.items()})
|
||||||
|
|
||||||
|
|
||||||
|
def load_request_file(path: str | Path) -> Mapping[str, Any]:
|
||||||
|
value = json.loads(Path(path).read_text(encoding="utf-8"))
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise ValueError("request file must contain a JSON object")
|
||||||
|
return dict(value)
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
"""按服务范围加载的最小权限 Provenance Writer。
|
||||||
|
|
||||||
|
该 Writer 只持有调用进程需要的 Grant ID。未配置路径的方法即使被调用也会
|
||||||
|
默认拒绝,避免应用进程获得 Git、部署或回滚权限。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import threading
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Sequence
|
||||||
|
|
||||||
|
from .provenance import ExecutionPath, ProvenanceDenied, required
|
||||||
|
from .provenance_grants import GrantAuthorization, GrantVerifier
|
||||||
|
|
||||||
|
|
||||||
|
class ScopedLeastPrivilegeProvenanceWriter:
|
||||||
|
"""只暴露指定 ExecutionPath 的最小权限 Writer。"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
verifier: GrantVerifier,
|
||||||
|
receipt_root: str | Path,
|
||||||
|
grant_ids: Mapping[str, str],
|
||||||
|
*,
|
||||||
|
required_paths: Sequence[str],
|
||||||
|
writer_id: str = "aios-v2-scoped-grant-writer",
|
||||||
|
) -> None:
|
||||||
|
self.verifier = verifier
|
||||||
|
self.receipt_root = Path(receipt_root).resolve()
|
||||||
|
self.receipt_root.mkdir(parents=True, exist_ok=True)
|
||||||
|
normalized_required = tuple(
|
||||||
|
sorted({ExecutionPath(path).value for path in required_paths})
|
||||||
|
)
|
||||||
|
if not normalized_required:
|
||||||
|
raise ValueError("required_paths must not be empty")
|
||||||
|
self.required_paths = frozenset(normalized_required)
|
||||||
|
self.grant_ids = {
|
||||||
|
path: required(grant_ids.get(path, ""), f"grant_id[{path}]")
|
||||||
|
for path in normalized_required
|
||||||
|
}
|
||||||
|
unexpected = sorted(set(grant_ids) - self.required_paths)
|
||||||
|
if unexpected:
|
||||||
|
raise ValueError(
|
||||||
|
f"scoped writer received unnecessary Grant mappings: {unexpected}"
|
||||||
|
)
|
||||||
|
self.writer_id = required(writer_id, "writer_id")
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _component(value: str) -> str:
|
||||||
|
normalized = required(value, "path component")
|
||||||
|
if normalized in {".", ".."} or "/" in normalized or "\\" in normalized:
|
||||||
|
raise ProvenanceDenied("invalid receipt path component")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
def _grant_for(self, path: str) -> str:
|
||||||
|
normalized = ExecutionPath(path).value
|
||||||
|
grant_id = self.grant_ids.get(normalized)
|
||||||
|
if grant_id is None:
|
||||||
|
raise ProvenanceDenied(
|
||||||
|
f"this process is not allowed to authorize {normalized}"
|
||||||
|
)
|
||||||
|
return grant_id
|
||||||
|
|
||||||
|
def authorize(self, **values: Any) -> GrantAuthorization:
|
||||||
|
path = ExecutionPath(values["execution_path"]).value
|
||||||
|
grant_id = self._grant_for(path)
|
||||||
|
parent_provenance_id = values.pop("provenance_id")
|
||||||
|
authorization = self.verifier.authorize(
|
||||||
|
grant_id=grant_id,
|
||||||
|
parent_provenance_id=parent_provenance_id,
|
||||||
|
**values,
|
||||||
|
)
|
||||||
|
tenant = self._component(authorization.tenant_id)
|
||||||
|
parent = self._component(authorization.parent_provenance_id)
|
||||||
|
grant = self._component(authorization.grant_id)
|
||||||
|
target_dir = self.receipt_root / tenant / parent / grant
|
||||||
|
target = target_dir / f"{authorization.authorization_id}.json"
|
||||||
|
payload = authorization.as_dict()
|
||||||
|
payload["writer_id"] = self.writer_id
|
||||||
|
payload["scope"] = sorted(self.required_paths)
|
||||||
|
with self._lock:
|
||||||
|
target_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
temporary = target.with_suffix(f".tmp-{os.getpid()}")
|
||||||
|
temporary.write_text(
|
||||||
|
json.dumps(payload, ensure_ascii=False, sort_keys=True, indent=2)
|
||||||
|
+ "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
os.chmod(temporary, 0o640)
|
||||||
|
os.replace(temporary, target)
|
||||||
|
return authorization
|
||||||
|
|
||||||
|
def authorize_code_change(self, **values: Any) -> GrantAuthorization:
|
||||||
|
return self.authorize(
|
||||||
|
execution_path=ExecutionPath.CODE_CHANGE.value, **values
|
||||||
|
)
|
||||||
|
|
||||||
|
def authorize_git_operation(
|
||||||
|
self, *, git_operation: str, **values: Any
|
||||||
|
) -> GrantAuthorization:
|
||||||
|
return self.authorize(
|
||||||
|
execution_path=ExecutionPath.GIT_OPERATION.value,
|
||||||
|
git_operation=git_operation,
|
||||||
|
**values,
|
||||||
|
)
|
||||||
|
|
||||||
|
def authorize_deployment(self, **values: Any) -> GrantAuthorization:
|
||||||
|
return self.authorize(
|
||||||
|
execution_path=ExecutionPath.DEPLOYMENT.value, **values
|
||||||
|
)
|
||||||
|
|
||||||
|
def authorize_runtime_execute(self, **values: Any) -> GrantAuthorization:
|
||||||
|
return self.authorize(
|
||||||
|
execution_path=ExecutionPath.RUNTIME_EXECUTE.value, **values
|
||||||
|
)
|
||||||
|
|
||||||
|
def authorize_evidence_write(self, **values: Any) -> GrantAuthorization:
|
||||||
|
return self.authorize(
|
||||||
|
execution_path=ExecutionPath.EVIDENCE_WRITE.value, **values
|
||||||
|
)
|
||||||
|
|
||||||
|
def authorize_rollback(self, **values: Any) -> GrantAuthorization:
|
||||||
|
return self.authorize(
|
||||||
|
execution_path=ExecutionPath.ROLLBACK.value, **values
|
||||||
|
)
|
||||||
|
|
||||||
|
def authorize_audit(self, **values: Any) -> GrantAuthorization:
|
||||||
|
return self.authorize(execution_path=ExecutionPath.AUDIT.value, **values)
|
||||||
@@ -0,0 +1,607 @@
|
|||||||
|
"""Durable execution truth registry for externally executed AIOS tasks.
|
||||||
|
|
||||||
|
A task manifest being present in Git is not evidence that an executor discovered
|
||||||
|
or ran it. This registry separates declaration, registration, claim, start,
|
||||||
|
heartbeat and terminal evidence. Release gates must require a terminal success
|
||||||
|
record with immutable evidence hashes.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import sqlite3
|
||||||
|
import threading
|
||||||
|
import uuid
|
||||||
|
from dataclasses import asdict, dataclass, field, replace
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from enum import Enum
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Sequence
|
||||||
|
|
||||||
|
|
||||||
|
def utc_now() -> str:
|
||||||
|
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||||
|
|
||||||
|
|
||||||
|
def parse_time(value: str) -> datetime:
|
||||||
|
normalized = str(value or "").strip()
|
||||||
|
if normalized.endswith("Z"):
|
||||||
|
normalized = normalized[:-1] + "+00:00"
|
||||||
|
parsed = datetime.fromisoformat(normalized)
|
||||||
|
if parsed.tzinfo is None:
|
||||||
|
raise ValueError("timestamp requires timezone")
|
||||||
|
return parsed.astimezone(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
def required(value: str, label: str) -> str:
|
||||||
|
normalized = str(value or "").strip()
|
||||||
|
if not normalized:
|
||||||
|
raise ValueError(f"{label} is required")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
class ExecutionTruthState(str, Enum):
|
||||||
|
DECLARED = "DECLARED"
|
||||||
|
REGISTERED = "REGISTERED"
|
||||||
|
CLAIMED = "CLAIMED"
|
||||||
|
RUNNING = "RUNNING"
|
||||||
|
SUCCEEDED = "SUCCEEDED"
|
||||||
|
FAILED = "FAILED"
|
||||||
|
CANCELLED = "CANCELLED"
|
||||||
|
NOT_STARTED = "NOT_STARTED"
|
||||||
|
STALLED = "STALLED"
|
||||||
|
|
||||||
|
|
||||||
|
TERMINAL_STATES = frozenset(
|
||||||
|
{
|
||||||
|
ExecutionTruthState.SUCCEEDED,
|
||||||
|
ExecutionTruthState.FAILED,
|
||||||
|
ExecutionTruthState.CANCELLED,
|
||||||
|
ExecutionTruthState.NOT_STARTED,
|
||||||
|
ExecutionTruthState.STALLED,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ExecutionTaskManifest:
|
||||||
|
task_id: str
|
||||||
|
executor: str
|
||||||
|
source_branch: str
|
||||||
|
source_oid: str
|
||||||
|
target_version: str
|
||||||
|
steps: Sequence[str]
|
||||||
|
start_deadline_seconds: int = 300
|
||||||
|
heartbeat_timeout_seconds: int = 180
|
||||||
|
required_evidence: Sequence[str] = ()
|
||||||
|
notification_events: Sequence[str] = ("STARTED", "SUCCEEDED", "FAILED")
|
||||||
|
owner_notification_ref: str = "AIOS_OWNER_WECHAT_TARGET_REF"
|
||||||
|
manifest_version: str = "aios.external-task.v1"
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
for name in (
|
||||||
|
"task_id",
|
||||||
|
"executor",
|
||||||
|
"source_branch",
|
||||||
|
"source_oid",
|
||||||
|
"target_version",
|
||||||
|
"manifest_version",
|
||||||
|
"owner_notification_ref",
|
||||||
|
):
|
||||||
|
object.__setattr__(self, name, required(getattr(self, name), name))
|
||||||
|
steps = tuple(required(item, "step") for item in self.steps)
|
||||||
|
if not steps:
|
||||||
|
raise ValueError("at least one execution step is required")
|
||||||
|
if self.start_deadline_seconds < 1 or self.heartbeat_timeout_seconds < 1:
|
||||||
|
raise ValueError("execution deadlines must be positive")
|
||||||
|
object.__setattr__(self, "steps", steps)
|
||||||
|
object.__setattr__(
|
||||||
|
self,
|
||||||
|
"required_evidence",
|
||||||
|
tuple(sorted({required(item, "required_evidence") for item in self.required_evidence})),
|
||||||
|
)
|
||||||
|
object.__setattr__(
|
||||||
|
self,
|
||||||
|
"notification_events",
|
||||||
|
tuple(sorted({required(item, "notification_event") for item in self.notification_events})),
|
||||||
|
)
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
value = asdict(self)
|
||||||
|
value["steps"] = list(self.steps)
|
||||||
|
value["required_evidence"] = list(self.required_evidence)
|
||||||
|
value["notification_events"] = list(self.notification_events)
|
||||||
|
return value
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, value: Mapping[str, Any]) -> "ExecutionTaskManifest":
|
||||||
|
return cls(**dict(value))
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ExecutionTruthRecord:
|
||||||
|
task_id: str
|
||||||
|
manifest_json: str
|
||||||
|
state: ExecutionTruthState
|
||||||
|
executor_run_id: str | None = None
|
||||||
|
executor_id: str | None = None
|
||||||
|
declared_at: str = field(default_factory=utc_now)
|
||||||
|
registered_at: str | None = None
|
||||||
|
claimed_at: str | None = None
|
||||||
|
started_at: str | None = None
|
||||||
|
heartbeat_at: str | None = None
|
||||||
|
completed_at: str | None = None
|
||||||
|
current_step: int = 0
|
||||||
|
evidence: Mapping[str, str] = field(default_factory=dict)
|
||||||
|
error: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
revision: int = 1
|
||||||
|
|
||||||
|
@property
|
||||||
|
def manifest(self) -> ExecutionTaskManifest:
|
||||||
|
return ExecutionTaskManifest.from_dict(json.loads(self.manifest_json))
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
**asdict(self),
|
||||||
|
"state": self.state.value,
|
||||||
|
"manifest": self.manifest.as_dict(),
|
||||||
|
"evidence": dict(self.evidence),
|
||||||
|
"error": dict(self.error),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class ExecutionTruthConflict(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class SQLiteExecutionTruthRegistry:
|
||||||
|
"""Persistent registration and run-evidence store."""
|
||||||
|
|
||||||
|
def __init__(self, path: str | Path) -> None:
|
||||||
|
self.path = Path(path).resolve()
|
||||||
|
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
with self._connect() as connection:
|
||||||
|
connection.executescript(
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS execution_truth (
|
||||||
|
task_id TEXT PRIMARY KEY,
|
||||||
|
manifest_json TEXT NOT NULL,
|
||||||
|
state TEXT NOT NULL,
|
||||||
|
record_json TEXT NOT NULL,
|
||||||
|
revision INTEGER NOT NULL,
|
||||||
|
updated_at TEXT NOT NULL
|
||||||
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS execution_truth_events (
|
||||||
|
event_id TEXT PRIMARY KEY,
|
||||||
|
task_id TEXT NOT NULL,
|
||||||
|
event_type TEXT NOT NULL,
|
||||||
|
details_json TEXT NOT NULL,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
FOREIGN KEY(task_id) REFERENCES execution_truth(task_id)
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_execution_truth_state
|
||||||
|
ON execution_truth(state, updated_at);
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
|
||||||
|
def _connect(self) -> sqlite3.Connection:
|
||||||
|
connection = sqlite3.connect(str(self.path), timeout=30.0)
|
||||||
|
connection.row_factory = sqlite3.Row
|
||||||
|
connection.execute("PRAGMA journal_mode=WAL")
|
||||||
|
connection.execute("PRAGMA synchronous=FULL")
|
||||||
|
connection.execute("PRAGMA foreign_keys=ON")
|
||||||
|
return connection
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _manifest_json(manifest: ExecutionTaskManifest) -> str:
|
||||||
|
return json.dumps(manifest.as_dict(), ensure_ascii=False, sort_keys=True, separators=(",", ":"))
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _record_json(record: ExecutionTruthRecord) -> str:
|
||||||
|
value = record.as_dict()
|
||||||
|
value.pop("manifest", None)
|
||||||
|
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"), default=str)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _decode(row: sqlite3.Row) -> ExecutionTruthRecord:
|
||||||
|
value = json.loads(row["record_json"])
|
||||||
|
value["state"] = ExecutionTruthState(value["state"])
|
||||||
|
value["evidence"] = dict(value.get("evidence") or {})
|
||||||
|
value["error"] = dict(value.get("error") or {})
|
||||||
|
return ExecutionTruthRecord(**value)
|
||||||
|
|
||||||
|
def _event(
|
||||||
|
self,
|
||||||
|
connection: sqlite3.Connection,
|
||||||
|
task_id: str,
|
||||||
|
event_type: str,
|
||||||
|
details: Mapping[str, Any] | None = None,
|
||||||
|
) -> None:
|
||||||
|
connection.execute(
|
||||||
|
"INSERT INTO execution_truth_events VALUES (?, ?, ?, ?, ?)",
|
||||||
|
(
|
||||||
|
f"truth_evt_{uuid.uuid4().hex}",
|
||||||
|
task_id,
|
||||||
|
event_type,
|
||||||
|
json.dumps(dict(details or {}), ensure_ascii=False, sort_keys=True, default=str),
|
||||||
|
utc_now(),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
def declare(self, manifest: ExecutionTaskManifest) -> ExecutionTruthRecord:
|
||||||
|
manifest_json = self._manifest_json(manifest)
|
||||||
|
record = ExecutionTruthRecord(
|
||||||
|
task_id=manifest.task_id,
|
||||||
|
manifest_json=manifest_json,
|
||||||
|
state=ExecutionTruthState.DECLARED,
|
||||||
|
)
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
connection.execute("BEGIN IMMEDIATE")
|
||||||
|
existing = connection.execute(
|
||||||
|
"SELECT record_json FROM execution_truth WHERE task_id = ?",
|
||||||
|
(manifest.task_id,),
|
||||||
|
).fetchone()
|
||||||
|
if existing is not None:
|
||||||
|
current = self._decode(existing)
|
||||||
|
if current.manifest_json != manifest_json:
|
||||||
|
previous_manifest = current.manifest.as_dict()
|
||||||
|
replacement_manifest = manifest.as_dict()
|
||||||
|
previous_oid = previous_manifest.pop("source_oid")
|
||||||
|
replacement_oid = replacement_manifest.pop("source_oid")
|
||||||
|
if (
|
||||||
|
current.state != ExecutionTruthState.NOT_STARTED
|
||||||
|
or previous_manifest != replacement_manifest
|
||||||
|
):
|
||||||
|
connection.rollback()
|
||||||
|
raise ExecutionTruthConflict(
|
||||||
|
"task_id already declared with another manifest"
|
||||||
|
)
|
||||||
|
now = utc_now()
|
||||||
|
updated = replace(
|
||||||
|
current,
|
||||||
|
manifest_json=manifest_json,
|
||||||
|
revision=current.revision + 1,
|
||||||
|
)
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE execution_truth SET manifest_json = ?, record_json = ?, revision = ?, updated_at = ? WHERE task_id = ?",
|
||||||
|
(
|
||||||
|
manifest_json,
|
||||||
|
self._record_json(updated),
|
||||||
|
updated.revision,
|
||||||
|
now,
|
||||||
|
updated.task_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
self._event(
|
||||||
|
connection,
|
||||||
|
updated.task_id,
|
||||||
|
"MANIFEST_SUPERSEDED",
|
||||||
|
{
|
||||||
|
"previous_source_oid": previous_oid,
|
||||||
|
"source_oid": replacement_oid,
|
||||||
|
"state": updated.state.value,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
connection.commit()
|
||||||
|
return updated
|
||||||
|
connection.rollback()
|
||||||
|
return current
|
||||||
|
connection.execute(
|
||||||
|
"INSERT INTO execution_truth VALUES (?, ?, ?, ?, ?, ?)",
|
||||||
|
(
|
||||||
|
record.task_id,
|
||||||
|
manifest_json,
|
||||||
|
record.state.value,
|
||||||
|
self._record_json(record),
|
||||||
|
record.revision,
|
||||||
|
record.declared_at,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
self._event(connection, record.task_id, "DECLARED", {"source_oid": manifest.source_oid})
|
||||||
|
connection.commit()
|
||||||
|
return record
|
||||||
|
|
||||||
|
def get(self, task_id: str) -> ExecutionTruthRecord:
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
row = connection.execute(
|
||||||
|
"SELECT record_json FROM execution_truth WHERE task_id = ?",
|
||||||
|
(required(task_id, "task_id"),),
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise KeyError("execution task not declared")
|
||||||
|
return self._decode(row)
|
||||||
|
|
||||||
|
def _transition(
|
||||||
|
self,
|
||||||
|
task_id: str,
|
||||||
|
*,
|
||||||
|
allowed: Sequence[ExecutionTruthState],
|
||||||
|
target: ExecutionTruthState,
|
||||||
|
executor_run_id: str | None = None,
|
||||||
|
executor_id: str | None = None,
|
||||||
|
current_step: int | None = None,
|
||||||
|
evidence: Mapping[str, str] | None = None,
|
||||||
|
error: Mapping[str, Any] | None = None,
|
||||||
|
) -> ExecutionTruthRecord:
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
connection.execute("BEGIN IMMEDIATE")
|
||||||
|
row = connection.execute(
|
||||||
|
"SELECT record_json FROM execution_truth WHERE task_id = ?",
|
||||||
|
(required(task_id, "task_id"),),
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
connection.rollback()
|
||||||
|
raise KeyError("execution task not declared")
|
||||||
|
current = self._decode(row)
|
||||||
|
if current.state not in set(allowed):
|
||||||
|
connection.rollback()
|
||||||
|
raise ExecutionTruthConflict(
|
||||||
|
f"cannot transition {current.state.value} to {target.value}"
|
||||||
|
)
|
||||||
|
now = utc_now()
|
||||||
|
updates: dict[str, Any] = {
|
||||||
|
"state": target,
|
||||||
|
"revision": current.revision + 1,
|
||||||
|
}
|
||||||
|
if executor_run_id is not None:
|
||||||
|
if current.executor_run_id and current.executor_run_id != executor_run_id:
|
||||||
|
connection.rollback()
|
||||||
|
raise ExecutionTruthConflict("executor_run_id mismatch")
|
||||||
|
updates["executor_run_id"] = required(executor_run_id, "executor_run_id")
|
||||||
|
if executor_id is not None:
|
||||||
|
if current.executor_id and current.executor_id != executor_id:
|
||||||
|
connection.rollback()
|
||||||
|
raise ExecutionTruthConflict("executor_id mismatch")
|
||||||
|
updates["executor_id"] = required(executor_id, "executor_id")
|
||||||
|
if target == ExecutionTruthState.REGISTERED:
|
||||||
|
updates["registered_at"] = now
|
||||||
|
if target == ExecutionTruthState.CLAIMED:
|
||||||
|
updates["claimed_at"] = now
|
||||||
|
updates["heartbeat_at"] = now
|
||||||
|
if target == ExecutionTruthState.RUNNING:
|
||||||
|
updates["started_at"] = current.started_at or now
|
||||||
|
updates["heartbeat_at"] = now
|
||||||
|
if current_step is not None:
|
||||||
|
if current_step < current.current_step or current_step > len(current.manifest.steps):
|
||||||
|
connection.rollback()
|
||||||
|
raise ExecutionTruthConflict("invalid task step progression")
|
||||||
|
updates["current_step"] = current_step
|
||||||
|
updates["heartbeat_at"] = now
|
||||||
|
if evidence is not None:
|
||||||
|
updates["evidence"] = {
|
||||||
|
**dict(current.evidence),
|
||||||
|
**{required(key, "evidence key"): required(value, "evidence hash") for key, value in evidence.items()},
|
||||||
|
}
|
||||||
|
if error is not None:
|
||||||
|
updates["error"] = json.loads(json.dumps(dict(error), default=str))
|
||||||
|
if target in TERMINAL_STATES:
|
||||||
|
updates["completed_at"] = now
|
||||||
|
updated = replace(current, **updates)
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE execution_truth SET state = ?, record_json = ?, revision = ?, updated_at = ? WHERE task_id = ?",
|
||||||
|
(
|
||||||
|
updated.state.value,
|
||||||
|
self._record_json(updated),
|
||||||
|
updated.revision,
|
||||||
|
now,
|
||||||
|
updated.task_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
self._event(
|
||||||
|
connection,
|
||||||
|
updated.task_id,
|
||||||
|
target.value,
|
||||||
|
{
|
||||||
|
"executor_run_id": updated.executor_run_id,
|
||||||
|
"executor_id": updated.executor_id,
|
||||||
|
"current_step": updated.current_step,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
connection.commit()
|
||||||
|
return updated
|
||||||
|
|
||||||
|
def register(self, task_id: str, *, executor_run_id: str) -> ExecutionTruthRecord:
|
||||||
|
return self._transition(
|
||||||
|
task_id,
|
||||||
|
allowed=(ExecutionTruthState.DECLARED,),
|
||||||
|
target=ExecutionTruthState.REGISTERED,
|
||||||
|
executor_run_id=executor_run_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
def retry_not_started(
|
||||||
|
self,
|
||||||
|
task_id: str,
|
||||||
|
*,
|
||||||
|
executor_run_id: str,
|
||||||
|
) -> ExecutionTruthRecord:
|
||||||
|
"""Start a new audited attempt after the declaration deadline expired."""
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
connection.execute("BEGIN IMMEDIATE")
|
||||||
|
row = connection.execute(
|
||||||
|
"SELECT record_json FROM execution_truth WHERE task_id = ?",
|
||||||
|
(required(task_id, "task_id"),),
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
connection.rollback()
|
||||||
|
raise KeyError("execution task not declared")
|
||||||
|
current = self._decode(row)
|
||||||
|
if current.state != ExecutionTruthState.NOT_STARTED:
|
||||||
|
connection.rollback()
|
||||||
|
raise ExecutionTruthConflict(
|
||||||
|
f"cannot retry task from {current.state.value}"
|
||||||
|
)
|
||||||
|
now = utc_now()
|
||||||
|
updated = replace(
|
||||||
|
current,
|
||||||
|
state=ExecutionTruthState.REGISTERED,
|
||||||
|
executor_run_id=required(executor_run_id, "executor_run_id"),
|
||||||
|
executor_id=None,
|
||||||
|
registered_at=now,
|
||||||
|
claimed_at=None,
|
||||||
|
started_at=None,
|
||||||
|
heartbeat_at=None,
|
||||||
|
completed_at=None,
|
||||||
|
current_step=0,
|
||||||
|
evidence={},
|
||||||
|
error={},
|
||||||
|
revision=current.revision + 1,
|
||||||
|
)
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE execution_truth SET state = ?, record_json = ?, revision = ?, updated_at = ? WHERE task_id = ?",
|
||||||
|
(
|
||||||
|
updated.state.value,
|
||||||
|
self._record_json(updated),
|
||||||
|
updated.revision,
|
||||||
|
now,
|
||||||
|
updated.task_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
self._event(
|
||||||
|
connection,
|
||||||
|
updated.task_id,
|
||||||
|
"RETRIED",
|
||||||
|
{
|
||||||
|
"previous_state": current.state.value,
|
||||||
|
"previous_executor_run_id": current.executor_run_id,
|
||||||
|
"executor_run_id": updated.executor_run_id,
|
||||||
|
"previous_error": dict(current.error),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
connection.commit()
|
||||||
|
return updated
|
||||||
|
|
||||||
|
def claim(self, task_id: str, *, executor_run_id: str, executor_id: str) -> ExecutionTruthRecord:
|
||||||
|
return self._transition(
|
||||||
|
task_id,
|
||||||
|
allowed=(ExecutionTruthState.REGISTERED,),
|
||||||
|
target=ExecutionTruthState.CLAIMED,
|
||||||
|
executor_run_id=executor_run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
def start(self, task_id: str, *, executor_run_id: str, executor_id: str) -> ExecutionTruthRecord:
|
||||||
|
return self._transition(
|
||||||
|
task_id,
|
||||||
|
allowed=(ExecutionTruthState.CLAIMED,),
|
||||||
|
target=ExecutionTruthState.RUNNING,
|
||||||
|
executor_run_id=executor_run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
current_step=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
def heartbeat(
|
||||||
|
self,
|
||||||
|
task_id: str,
|
||||||
|
*,
|
||||||
|
executor_run_id: str,
|
||||||
|
executor_id: str,
|
||||||
|
current_step: int,
|
||||||
|
) -> ExecutionTruthRecord:
|
||||||
|
return self._transition(
|
||||||
|
task_id,
|
||||||
|
allowed=(ExecutionTruthState.RUNNING,),
|
||||||
|
target=ExecutionTruthState.RUNNING,
|
||||||
|
executor_run_id=executor_run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
current_step=current_step,
|
||||||
|
)
|
||||||
|
|
||||||
|
def succeed(
|
||||||
|
self,
|
||||||
|
task_id: str,
|
||||||
|
*,
|
||||||
|
executor_run_id: str,
|
||||||
|
executor_id: str,
|
||||||
|
evidence: Mapping[str, str],
|
||||||
|
) -> ExecutionTruthRecord:
|
||||||
|
current = self.get(task_id)
|
||||||
|
missing = sorted(set(current.manifest.required_evidence) - set(evidence))
|
||||||
|
if missing:
|
||||||
|
raise ExecutionTruthConflict(f"missing required evidence: {missing}")
|
||||||
|
if current.current_step != len(current.manifest.steps):
|
||||||
|
raise ExecutionTruthConflict("all declared steps must complete before success")
|
||||||
|
return self._transition(
|
||||||
|
task_id,
|
||||||
|
allowed=(ExecutionTruthState.RUNNING,),
|
||||||
|
target=ExecutionTruthState.SUCCEEDED,
|
||||||
|
executor_run_id=executor_run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
evidence=evidence,
|
||||||
|
)
|
||||||
|
|
||||||
|
def fail(
|
||||||
|
self,
|
||||||
|
task_id: str,
|
||||||
|
*,
|
||||||
|
executor_run_id: str,
|
||||||
|
executor_id: str,
|
||||||
|
error: Mapping[str, Any],
|
||||||
|
evidence: Mapping[str, str] | None = None,
|
||||||
|
) -> ExecutionTruthRecord:
|
||||||
|
return self._transition(
|
||||||
|
task_id,
|
||||||
|
allowed=(ExecutionTruthState.REGISTERED, ExecutionTruthState.CLAIMED, ExecutionTruthState.RUNNING),
|
||||||
|
target=ExecutionTruthState.FAILED,
|
||||||
|
executor_run_id=executor_run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
evidence=evidence,
|
||||||
|
error=error,
|
||||||
|
)
|
||||||
|
|
||||||
|
def evaluate_deadlines(self, task_id: str, *, now: datetime | None = None) -> ExecutionTruthRecord:
|
||||||
|
current = self.get(task_id)
|
||||||
|
current_time = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
|
||||||
|
manifest = current.manifest
|
||||||
|
if current.state in TERMINAL_STATES:
|
||||||
|
return current
|
||||||
|
declared = parse_time(current.declared_at)
|
||||||
|
if current.state in {ExecutionTruthState.DECLARED, ExecutionTruthState.REGISTERED}:
|
||||||
|
if current_time >= declared + timedelta(seconds=manifest.start_deadline_seconds):
|
||||||
|
return self._transition(
|
||||||
|
task_id,
|
||||||
|
allowed=(current.state,),
|
||||||
|
target=ExecutionTruthState.NOT_STARTED,
|
||||||
|
error={"code": "start_deadline_exceeded"},
|
||||||
|
)
|
||||||
|
if current.state in {ExecutionTruthState.CLAIMED, ExecutionTruthState.RUNNING}:
|
||||||
|
heartbeat = parse_time(current.heartbeat_at or current.claimed_at or current.declared_at)
|
||||||
|
if current_time >= heartbeat + timedelta(seconds=manifest.heartbeat_timeout_seconds):
|
||||||
|
return self._transition(
|
||||||
|
task_id,
|
||||||
|
allowed=(current.state,),
|
||||||
|
target=ExecutionTruthState.STALLED,
|
||||||
|
error={"code": "heartbeat_timeout"},
|
||||||
|
)
|
||||||
|
return current
|
||||||
|
|
||||||
|
def gate(self, task_id: str) -> Mapping[str, Any]:
|
||||||
|
record = self.evaluate_deadlines(task_id)
|
||||||
|
passed = record.state == ExecutionTruthState.SUCCEEDED
|
||||||
|
return {
|
||||||
|
"task_id": record.task_id,
|
||||||
|
"state": record.state.value,
|
||||||
|
"passed": passed,
|
||||||
|
"release_blocking": not passed,
|
||||||
|
"executor_run_id": record.executor_run_id,
|
||||||
|
"executor_id": record.executor_id,
|
||||||
|
"current_step": record.current_step,
|
||||||
|
"total_steps": len(record.manifest.steps),
|
||||||
|
"evidence": dict(record.evidence),
|
||||||
|
"error": dict(record.error),
|
||||||
|
}
|
||||||
|
|
||||||
|
def events(self, task_id: str) -> list[Mapping[str, Any]]:
|
||||||
|
with self._lock, self._connect() as connection:
|
||||||
|
rows = connection.execute(
|
||||||
|
"SELECT event_id, event_type, details_json, created_at FROM execution_truth_events WHERE task_id = ? ORDER BY created_at, event_id",
|
||||||
|
(required(task_id, "task_id"),),
|
||||||
|
).fetchall()
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"event_id": row["event_id"],
|
||||||
|
"event_type": row["event_type"],
|
||||||
|
"details": json.loads(row["details_json"]),
|
||||||
|
"created_at": row["created_at"],
|
||||||
|
}
|
||||||
|
for row in rows
|
||||||
|
]
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
"""AIOS unified, role-aware web portal."""
|
||||||
|
|
||||||
|
from .application import UnifiedPortalApplication, build_application
|
||||||
|
from .capabilities import ROLE_CAPABILITIES, allows, require, resolve
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"ROLE_CAPABILITIES",
|
||||||
|
"UnifiedPortalApplication",
|
||||||
|
"allows",
|
||||||
|
"build_application",
|
||||||
|
"require",
|
||||||
|
"resolve",
|
||||||
|
]
|
||||||
@@ -0,0 +1,512 @@
|
|||||||
|
"""Unified, role-aware AIOS web application.
|
||||||
|
|
||||||
|
The portal wraps the production CandidateApplication instead of replacing its
|
||||||
|
runtime/control-plane routes. Portal endpoints aggregate existing authoritative
|
||||||
|
stores and require server-side capabilities before delegating any state change.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import sqlite3
|
||||||
|
import urllib.parse
|
||||||
|
from enum import Enum
|
||||||
|
from http import HTTPStatus
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
from aios_v2.app import (
|
||||||
|
CandidateApplication,
|
||||||
|
CandidateHTTPServer,
|
||||||
|
build_application as build_candidate_application,
|
||||||
|
)
|
||||||
|
from aios_v2.control_plane import RequestContext, TaskState
|
||||||
|
from aios_v2.gitops_status import collect_gitops_status
|
||||||
|
from aios_v2.provenance import SQLiteProvenanceStore
|
||||||
|
from aios_v2.provenance_grants import SQLiteGrantStore
|
||||||
|
|
||||||
|
from .capabilities import allows, require, resolve
|
||||||
|
|
||||||
|
|
||||||
|
def _json_value(value: Any) -> Any:
|
||||||
|
if dataclasses.is_dataclass(value):
|
||||||
|
return {
|
||||||
|
field.name: _json_value(getattr(value, field.name))
|
||||||
|
for field in dataclasses.fields(value)
|
||||||
|
}
|
||||||
|
if isinstance(value, Enum):
|
||||||
|
return value.value
|
||||||
|
if isinstance(value, Mapping):
|
||||||
|
return {str(key): _json_value(item) for key, item in value.items()}
|
||||||
|
if isinstance(value, (list, tuple, set, frozenset)):
|
||||||
|
return [_json_value(item) for item in value]
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _mapping(value: Any) -> Mapping[str, Any]:
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise ValueError("JSON body must be an object")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _required(value: Any, label: str) -> str:
|
||||||
|
normalized = str(value or "").strip()
|
||||||
|
if not normalized:
|
||||||
|
raise ValueError(f"{label} is required")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
def _safe_count(items: list[Any]) -> int:
|
||||||
|
return len(items)
|
||||||
|
|
||||||
|
|
||||||
|
class AuthorizationCatalog:
|
||||||
|
"""Tenant-filtered read/revoke access to existing Provenance stores."""
|
||||||
|
|
||||||
|
def __init__(self, store_file: str | Path) -> None:
|
||||||
|
self.store_file = Path(store_file).resolve()
|
||||||
|
|
||||||
|
def _read_connection(self) -> sqlite3.Connection | None:
|
||||||
|
if not self.store_file.exists():
|
||||||
|
return None
|
||||||
|
connection = sqlite3.connect(
|
||||||
|
f"file:{self.store_file}?mode=ro",
|
||||||
|
uri=True,
|
||||||
|
timeout=10.0,
|
||||||
|
)
|
||||||
|
connection.row_factory = sqlite3.Row
|
||||||
|
return connection
|
||||||
|
|
||||||
|
def snapshot(self, *, tenant_id: str) -> Mapping[str, Any]:
|
||||||
|
connection = self._read_connection()
|
||||||
|
if connection is None:
|
||||||
|
return {
|
||||||
|
"configured": False,
|
||||||
|
"workflows": [],
|
||||||
|
"grants": [],
|
||||||
|
"summary": {"workflows": 0, "grants": 0, "active_grants": 0},
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
workflows: list[dict[str, Any]] = []
|
||||||
|
grants: list[dict[str, Any]] = []
|
||||||
|
tables = {
|
||||||
|
row["name"]
|
||||||
|
for row in connection.execute(
|
||||||
|
"SELECT name FROM sqlite_master WHERE type='table'"
|
||||||
|
).fetchall()
|
||||||
|
}
|
||||||
|
if "provenance_records" in tables:
|
||||||
|
rows = connection.execute(
|
||||||
|
"""
|
||||||
|
SELECT r.payload_json, r.status, r.signature_verified,
|
||||||
|
r.created_at, r.updated_at,
|
||||||
|
COUNT(u.authorization_id) AS usage_count
|
||||||
|
FROM provenance_records r
|
||||||
|
LEFT JOIN provenance_usage u
|
||||||
|
ON u.provenance_id = r.provenance_id
|
||||||
|
GROUP BY r.provenance_id
|
||||||
|
ORDER BY r.updated_at DESC, r.provenance_id
|
||||||
|
"""
|
||||||
|
).fetchall()
|
||||||
|
for row in rows:
|
||||||
|
payload = json.loads(row["payload_json"])
|
||||||
|
if payload.get("tenant_id") != tenant_id:
|
||||||
|
continue
|
||||||
|
workflows.append(
|
||||||
|
{
|
||||||
|
**payload,
|
||||||
|
"status": row["status"],
|
||||||
|
"signature_verified": bool(row["signature_verified"]),
|
||||||
|
"usage_count": int(row["usage_count"]),
|
||||||
|
"created_at": row["created_at"],
|
||||||
|
"updated_at": row["updated_at"],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
if "provenance_grants" in tables:
|
||||||
|
rows = connection.execute(
|
||||||
|
"""
|
||||||
|
SELECT g.payload_json, g.status, g.signature_verified,
|
||||||
|
g.created_at, g.updated_at,
|
||||||
|
COUNT(u.authorization_id) AS usage_count
|
||||||
|
FROM provenance_grants g
|
||||||
|
LEFT JOIN provenance_grant_usage u
|
||||||
|
ON u.grant_id = g.grant_id
|
||||||
|
GROUP BY g.grant_id
|
||||||
|
ORDER BY g.updated_at DESC, g.grant_id
|
||||||
|
"""
|
||||||
|
).fetchall()
|
||||||
|
for row in rows:
|
||||||
|
payload = json.loads(row["payload_json"])
|
||||||
|
if payload.get("tenant_id") != tenant_id:
|
||||||
|
continue
|
||||||
|
grants.append(
|
||||||
|
{
|
||||||
|
**payload,
|
||||||
|
"status": row["status"],
|
||||||
|
"signature_verified": bool(row["signature_verified"]),
|
||||||
|
"usage_count": int(row["usage_count"]),
|
||||||
|
"created_at": row["created_at"],
|
||||||
|
"updated_at": row["updated_at"],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
active = sum(
|
||||||
|
1
|
||||||
|
for item in grants
|
||||||
|
if item.get("status") not in {"revoked", "expired"}
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"configured": True,
|
||||||
|
"workflows": workflows,
|
||||||
|
"grants": grants,
|
||||||
|
"summary": {
|
||||||
|
"workflows": len(workflows),
|
||||||
|
"grants": len(grants),
|
||||||
|
"active_grants": active,
|
||||||
|
"revoked_grants": sum(
|
||||||
|
1 for item in grants if item.get("status") == "revoked"
|
||||||
|
),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
finally:
|
||||||
|
connection.close()
|
||||||
|
|
||||||
|
def revoke_workflow(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
provenance_id: str,
|
||||||
|
tenant_id: str,
|
||||||
|
principal_id: str,
|
||||||
|
reason: str,
|
||||||
|
) -> Mapping[str, Any]:
|
||||||
|
store = SQLiteProvenanceStore(self.store_file)
|
||||||
|
try:
|
||||||
|
record = store.get(provenance_id)
|
||||||
|
if record.tenant_id != tenant_id:
|
||||||
|
raise KeyError("workflow provenance not found")
|
||||||
|
store.revoke(
|
||||||
|
record.provenance_id,
|
||||||
|
reason=reason,
|
||||||
|
principal_id=principal_id,
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"provenance_id": record.provenance_id,
|
||||||
|
"status": store.status(record.provenance_id).value,
|
||||||
|
}
|
||||||
|
finally:
|
||||||
|
store.close()
|
||||||
|
|
||||||
|
def revoke_grant(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
grant_id: str,
|
||||||
|
tenant_id: str,
|
||||||
|
principal_id: str,
|
||||||
|
reason: str,
|
||||||
|
) -> Mapping[str, Any]:
|
||||||
|
store = SQLiteGrantStore(self.store_file)
|
||||||
|
try:
|
||||||
|
grant = store.get(grant_id)
|
||||||
|
if grant.tenant_id != tenant_id:
|
||||||
|
raise KeyError("provenance grant not found")
|
||||||
|
store.revoke(
|
||||||
|
grant.grant_id,
|
||||||
|
reason=reason,
|
||||||
|
principal_id=principal_id,
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"grant_id": grant.grant_id,
|
||||||
|
"status": store.status(grant.grant_id).value,
|
||||||
|
}
|
||||||
|
finally:
|
||||||
|
store.close()
|
||||||
|
|
||||||
|
|
||||||
|
class UnifiedPortalApplication:
|
||||||
|
"""Role-aware portal facade over the existing production application."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
base: CandidateApplication,
|
||||||
|
*,
|
||||||
|
provenance_store_file: str | Path,
|
||||||
|
) -> None:
|
||||||
|
self.base = base
|
||||||
|
self.gateway = base.gateway
|
||||||
|
self.orchestrator = base.orchestrator
|
||||||
|
self.tracker = base.tracker
|
||||||
|
self.hermes = base.hermes
|
||||||
|
self.openclaw = base.openclaw
|
||||||
|
self.authorization_catalog = AuthorizationCatalog(provenance_store_file)
|
||||||
|
|
||||||
|
def context(self, headers: Mapping[str, str]) -> RequestContext:
|
||||||
|
return self.base.context(headers)
|
||||||
|
|
||||||
|
def _runtime(self, headers: Mapping[str, str]) -> Mapping[str, Any]:
|
||||||
|
status, payload = self.base.handle_json("GET", "/api/v2/runtime", headers, {})
|
||||||
|
if int(status) != HTTPStatus.OK:
|
||||||
|
raise RuntimeError("runtime status is unavailable")
|
||||||
|
return payload
|
||||||
|
|
||||||
|
def _gitops(self, context: RequestContext) -> Mapping[str, Any]:
|
||||||
|
return {
|
||||||
|
"tenant_id": context.tenant_id,
|
||||||
|
"gitops": collect_gitops_status().as_dict(),
|
||||||
|
}
|
||||||
|
|
||||||
|
def _tasks_view(self, context: RequestContext) -> list[Mapping[str, Any]]:
|
||||||
|
service = self.orchestrator.control_plane
|
||||||
|
tasks = service.list_tasks(context)
|
||||||
|
dispatches = {
|
||||||
|
item.task_id: item
|
||||||
|
for item in self.orchestrator.dispatch_repository.list(
|
||||||
|
tenant_id=context.tenant_id
|
||||||
|
)
|
||||||
|
}
|
||||||
|
evidence_by_task: dict[str, list[Any]] = {}
|
||||||
|
for evidence in self.tracker.list(tenant_id=context.tenant_id):
|
||||||
|
evidence_by_task.setdefault(evidence.task_id, []).append(evidence)
|
||||||
|
audit_by_task: dict[str, list[Any]] = {}
|
||||||
|
for event in service.list_audit_events(context):
|
||||||
|
if event.resource_type == "task":
|
||||||
|
audit_by_task.setdefault(event.resource_id, []).append(event)
|
||||||
|
capabilities = set(resolve(context).capabilities)
|
||||||
|
result: list[Mapping[str, Any]] = []
|
||||||
|
for task in tasks:
|
||||||
|
actions: list[str] = []
|
||||||
|
if task.state == TaskState.CREATED and "task.approve" in capabilities:
|
||||||
|
actions.append("approve")
|
||||||
|
if task.state in {TaskState.CREATED, TaskState.APPROVED, TaskState.RUNNING} and "task.cancel" in capabilities:
|
||||||
|
actions.append("cancel")
|
||||||
|
if task.state == TaskState.APPROVED and "task.dispatch" in capabilities:
|
||||||
|
actions.append("dispatch")
|
||||||
|
result.append(
|
||||||
|
{
|
||||||
|
"task": _json_value(task),
|
||||||
|
"dispatch": _json_value(dispatches.get(task.task_id)),
|
||||||
|
"evidence": _json_value(evidence_by_task.get(task.task_id, [])),
|
||||||
|
"audit": _json_value(audit_by_task.get(task.task_id, [])[-12:]),
|
||||||
|
"available_actions": actions,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return result
|
||||||
|
|
||||||
|
def _overview(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
headers: Mapping[str, str],
|
||||||
|
) -> Mapping[str, Any]:
|
||||||
|
service = self.orchestrator.control_plane
|
||||||
|
capabilities = resolve(context)
|
||||||
|
sections: dict[str, Any] = {}
|
||||||
|
tasks = service.list_tasks(context) if allows(context, "task.read") else []
|
||||||
|
dispatches = (
|
||||||
|
self.orchestrator.dispatch_repository.list(tenant_id=context.tenant_id)
|
||||||
|
if allows(context, "dispatch.read")
|
||||||
|
else []
|
||||||
|
)
|
||||||
|
evidence = (
|
||||||
|
self.tracker.list(tenant_id=context.tenant_id)
|
||||||
|
if allows(context, "evidence.read")
|
||||||
|
else []
|
||||||
|
)
|
||||||
|
audit = (
|
||||||
|
service.list_audit_events(context)
|
||||||
|
if allows(context, "audit.read")
|
||||||
|
else []
|
||||||
|
)
|
||||||
|
if allows(context, "runtime.read"):
|
||||||
|
sections["runtime"] = self._runtime(headers)
|
||||||
|
if allows(context, "release.read"):
|
||||||
|
sections["release"] = {
|
||||||
|
**self._gitops(context),
|
||||||
|
"version": os.getenv("AIOS_RELEASE_VERSION", "V2R1C1"),
|
||||||
|
"release_oid": os.getenv("AIOS_RELEASE_OID", "unknown"),
|
||||||
|
"baseline": os.getenv("AIOS_BASELINE_VERSION", "V2R1C1"),
|
||||||
|
"status": os.getenv("AIOS_RELEASE_STATUS", "PRODUCTION_DEPLOYED"),
|
||||||
|
}
|
||||||
|
if allows(context, "provenance.read"):
|
||||||
|
sections["authorization"] = self.authorization_catalog.snapshot(
|
||||||
|
tenant_id=context.tenant_id
|
||||||
|
)["summary"]
|
||||||
|
task_states: dict[str, int] = {}
|
||||||
|
for task in tasks:
|
||||||
|
task_states[task.state.value] = task_states.get(task.state.value, 0) + 1
|
||||||
|
return {
|
||||||
|
"identity": capabilities.as_dict(),
|
||||||
|
"summary": {
|
||||||
|
"tasks": _safe_count(tasks),
|
||||||
|
"task_states": task_states,
|
||||||
|
"dispatches": _safe_count(dispatches),
|
||||||
|
"active_dispatches": sum(
|
||||||
|
1
|
||||||
|
for item in dispatches
|
||||||
|
if item.state.value not in {"COMPLETED", "FAILED", "CANCELLED"}
|
||||||
|
),
|
||||||
|
"evidence": _safe_count(evidence),
|
||||||
|
"audit_events": _safe_count(audit),
|
||||||
|
},
|
||||||
|
"sections": sections,
|
||||||
|
"recent": {
|
||||||
|
"tasks": _json_value(tasks[:8]),
|
||||||
|
"dispatches": _json_value(dispatches[-8:]),
|
||||||
|
"audit": _json_value(audit[-12:]),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
def handle_json(
|
||||||
|
self,
|
||||||
|
method: str,
|
||||||
|
raw_path: str,
|
||||||
|
headers: Mapping[str, str],
|
||||||
|
body: Any,
|
||||||
|
) -> tuple[int, Mapping[str, Any]]:
|
||||||
|
parsed = urllib.parse.urlparse(raw_path)
|
||||||
|
path = parsed.path.rstrip("/") or "/"
|
||||||
|
method = method.upper()
|
||||||
|
|
||||||
|
if not path.startswith("/api/v2/portal") and path != "/api/v2/me/capabilities":
|
||||||
|
return self.base.handle_json(method, raw_path, headers, body)
|
||||||
|
|
||||||
|
context = self.context(headers)
|
||||||
|
payload = _mapping(body or {})
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/me/capabilities":
|
||||||
|
require(context, "portal.access")
|
||||||
|
return HTTPStatus.OK, resolve(context).as_dict()
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/portal/overview":
|
||||||
|
require(context, "dashboard.read")
|
||||||
|
return HTTPStatus.OK, self._overview(context, headers)
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/portal/tasks":
|
||||||
|
require(context, "task.read")
|
||||||
|
return HTTPStatus.OK, {"items": self._tasks_view(context)}
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/portal/authorizations":
|
||||||
|
require(context, "provenance.read")
|
||||||
|
return HTTPStatus.OK, self.authorization_catalog.snapshot(
|
||||||
|
tenant_id=context.tenant_id
|
||||||
|
)
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/portal/runtime":
|
||||||
|
require(context, "runtime.read")
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
**self._runtime(headers),
|
||||||
|
"dispatches": _json_value(
|
||||||
|
self.orchestrator.dispatch_repository.list(
|
||||||
|
tenant_id=context.tenant_id
|
||||||
|
)
|
||||||
|
),
|
||||||
|
"hermes_jobs": _json_value(
|
||||||
|
self.hermes.list(tenant_id=context.tenant_id)
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/portal/evidence":
|
||||||
|
require(context, "evidence.read")
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"executions": _json_value(
|
||||||
|
self.tracker.list(tenant_id=context.tenant_id)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/portal/audit":
|
||||||
|
require(context, "audit.read")
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"events": _json_value(
|
||||||
|
self.orchestrator.control_plane.list_audit_events(context)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/portal/release":
|
||||||
|
require(context, "release.read")
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
**self._gitops(context),
|
||||||
|
"version": os.getenv("AIOS_RELEASE_VERSION", "V2R1C1"),
|
||||||
|
"release_oid": os.getenv("AIOS_RELEASE_OID", "unknown"),
|
||||||
|
"baseline": os.getenv("AIOS_BASELINE_VERSION", "V2R1C1"),
|
||||||
|
"status": os.getenv("AIOS_RELEASE_STATUS", "PRODUCTION_DEPLOYED"),
|
||||||
|
"operations": (
|
||||||
|
["review_release", "request_tag", "request_rollback"]
|
||||||
|
if allows(context, "release.operate")
|
||||||
|
else []
|
||||||
|
),
|
||||||
|
"note": "Release operations remain GitOps approval requests; the portal does not mutate tags or production directly.",
|
||||||
|
}
|
||||||
|
|
||||||
|
if method == "POST" and path == "/api/v2/portal/tasks":
|
||||||
|
require(context, "task.create")
|
||||||
|
return self.base.handle_json("POST", "/api/v2/tasks", headers, payload)
|
||||||
|
|
||||||
|
task_action = re.fullmatch(
|
||||||
|
r"/api/v2/portal/tasks/([^/]+)/(approve|cancel|dispatch)", path
|
||||||
|
)
|
||||||
|
if method == "POST" and task_action:
|
||||||
|
action = task_action.group(2)
|
||||||
|
require(context, f"task.{action}")
|
||||||
|
task_id = urllib.parse.unquote(task_action.group(1))
|
||||||
|
target = (
|
||||||
|
f"/api/v2/tasks/{task_id}/dispatch"
|
||||||
|
if action == "dispatch"
|
||||||
|
else f"/api/v2/tasks/{task_id}/{action}"
|
||||||
|
)
|
||||||
|
return self.base.handle_json("POST", target, headers, payload)
|
||||||
|
|
||||||
|
workflow_revoke = re.fullmatch(
|
||||||
|
r"/api/v2/portal/provenance/([^/]+)/revoke", path
|
||||||
|
)
|
||||||
|
if method == "POST" and workflow_revoke:
|
||||||
|
require(context, "provenance.revoke")
|
||||||
|
reason = _required(payload.get("reason"), "reason")
|
||||||
|
result = self.authorization_catalog.revoke_workflow(
|
||||||
|
provenance_id=urllib.parse.unquote(workflow_revoke.group(1)),
|
||||||
|
tenant_id=context.tenant_id,
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
reason=reason,
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {"workflow": result}
|
||||||
|
|
||||||
|
grant_revoke = re.fullmatch(r"/api/v2/portal/grants/([^/]+)/revoke", path)
|
||||||
|
if method == "POST" and grant_revoke:
|
||||||
|
require(context, "provenance.revoke")
|
||||||
|
reason = _required(payload.get("reason"), "reason")
|
||||||
|
result = self.authorization_catalog.revoke_grant(
|
||||||
|
grant_id=urllib.parse.unquote(grant_revoke.group(1)),
|
||||||
|
tenant_id=context.tenant_id,
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
reason=reason,
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {"grant": result}
|
||||||
|
|
||||||
|
return HTTPStatus.NOT_FOUND, {
|
||||||
|
"error": "not_found",
|
||||||
|
"message": "portal route not found",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def build_application() -> UnifiedPortalApplication:
|
||||||
|
state_dir = Path(os.getenv("AIOS_V2_STATE_DIR", "/var/lib/aios-v2")).resolve()
|
||||||
|
provenance_file = Path(
|
||||||
|
os.getenv(
|
||||||
|
"AIOS_PROVENANCE_STORE_FILE",
|
||||||
|
str(state_dir / "run-provenance.sqlite3"),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return UnifiedPortalApplication(
|
||||||
|
build_candidate_application(),
|
||||||
|
provenance_store_file=provenance_file,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
host = os.getenv("AIOS_V2_HOST", "127.0.0.1")
|
||||||
|
port = int(os.getenv("AIOS_V2_PORT", "18920"))
|
||||||
|
server = CandidateHTTPServer((host, port), build_application())
|
||||||
|
print(f"AIOS V2 unified portal listening on {host}:{port}")
|
||||||
|
server.serve_forever()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
"""Server-side role and capability resolution for the unified AIOS portal."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Mapping, Sequence
|
||||||
|
|
||||||
|
from aios_v2.control_plane import RequestContext, Unauthorized
|
||||||
|
|
||||||
|
|
||||||
|
ROLE_CAPABILITIES: Mapping[str, frozenset[str]] = {
|
||||||
|
"platform_admin": frozenset({
|
||||||
|
"portal.access", "dashboard.read", "tenant.read", "tenant.manage",
|
||||||
|
"agent.read", "agent.manage", "task.read", "task.create", "task.approve",
|
||||||
|
"task.cancel", "task.dispatch", "policy.read", "audit.read", "evidence.read",
|
||||||
|
"runtime.read", "runtime.operate", "dispatch.read", "provenance.read",
|
||||||
|
"provenance.revoke", "external_task.read", "gitops.read", "release.read",
|
||||||
|
"release.operate", "worktree.read", "system.admin",
|
||||||
|
}),
|
||||||
|
"tenant_admin": frozenset({
|
||||||
|
"portal.access", "dashboard.read", "tenant.read", "agent.read", "agent.manage",
|
||||||
|
"task.read", "task.create", "task.approve", "task.cancel", "task.dispatch",
|
||||||
|
"policy.read", "audit.read", "evidence.read", "runtime.read", "dispatch.read",
|
||||||
|
"provenance.read", "provenance.revoke", "external_task.read", "gitops.read",
|
||||||
|
"release.read",
|
||||||
|
}),
|
||||||
|
"operator": frozenset({
|
||||||
|
"portal.access", "dashboard.read", "tenant.read", "agent.read", "task.read",
|
||||||
|
"task.create", "task.cancel", "task.dispatch", "runtime.read", "dispatch.read",
|
||||||
|
"evidence.read", "provenance.read", "external_task.read", "gitops.read",
|
||||||
|
"release.read",
|
||||||
|
}),
|
||||||
|
"approver": frozenset({
|
||||||
|
"portal.access", "dashboard.read", "tenant.read", "task.read", "task.approve",
|
||||||
|
"task.cancel", "policy.read", "audit.read", "provenance.read", "evidence.read",
|
||||||
|
"external_task.read", "release.read",
|
||||||
|
}),
|
||||||
|
"auditor": frozenset({
|
||||||
|
"portal.access", "dashboard.read", "tenant.read", "agent.read", "task.read",
|
||||||
|
"audit.read", "evidence.read", "runtime.read", "dispatch.read",
|
||||||
|
"provenance.read", "external_task.read", "gitops.read", "release.read",
|
||||||
|
"worktree.read",
|
||||||
|
}),
|
||||||
|
"runtime": frozenset({
|
||||||
|
"portal.access", "dashboard.read", "task.read", "runtime.read", "runtime.operate",
|
||||||
|
"dispatch.read", "evidence.read", "provenance.read", "external_task.read",
|
||||||
|
}),
|
||||||
|
"release_manager": frozenset({
|
||||||
|
"portal.access", "dashboard.read", "audit.read", "runtime.read", "dispatch.read",
|
||||||
|
"provenance.read", "external_task.read", "gitops.read", "release.read",
|
||||||
|
"release.operate", "worktree.read",
|
||||||
|
}),
|
||||||
|
"viewer": frozenset({
|
||||||
|
"portal.access", "dashboard.read", "task.read", "runtime.read",
|
||||||
|
"external_task.read",
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
NAVIGATION: Sequence[Mapping[str, str]] = (
|
||||||
|
{"id": "overview", "label": "总览", "capability": "dashboard.read"},
|
||||||
|
{"id": "resources", "label": "租户与 Agent", "capability": "tenant.read"},
|
||||||
|
{"id": "tasks", "label": "任务中心", "capability": "task.read"},
|
||||||
|
{"id": "execution", "label": "执行中心", "capability": "external_task.read"},
|
||||||
|
{"id": "authorization", "label": "授权中心", "capability": "provenance.read"},
|
||||||
|
{"id": "runtime", "label": "运行与运维", "capability": "runtime.read"},
|
||||||
|
{"id": "evidence", "label": "执行证据", "capability": "evidence.read"},
|
||||||
|
{"id": "audit", "label": "审计中心", "capability": "audit.read"},
|
||||||
|
{"id": "release", "label": "GitOps 与发布", "capability": "release.read"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ResolvedCapabilities:
|
||||||
|
tenant_id: str
|
||||||
|
principal_id: str
|
||||||
|
roles: tuple[str, ...]
|
||||||
|
capabilities: tuple[str, ...]
|
||||||
|
navigation: tuple[Mapping[str, str], ...]
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
"tenant_id": self.tenant_id,
|
||||||
|
"principal_id": self.principal_id,
|
||||||
|
"roles": list(self.roles),
|
||||||
|
"capabilities": list(self.capabilities),
|
||||||
|
"navigation": [dict(item) for item in self.navigation],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def resolve(context: RequestContext) -> ResolvedCapabilities:
|
||||||
|
capabilities: set[str] = set()
|
||||||
|
for role in context.roles:
|
||||||
|
capabilities.update(ROLE_CAPABILITIES.get(role, ()))
|
||||||
|
ordered = tuple(sorted(capabilities))
|
||||||
|
navigation = tuple(item for item in NAVIGATION if item["capability"] in capabilities)
|
||||||
|
return ResolvedCapabilities(
|
||||||
|
tenant_id=context.tenant_id,
|
||||||
|
principal_id=context.principal_id,
|
||||||
|
roles=tuple(sorted(context.roles)),
|
||||||
|
capabilities=ordered,
|
||||||
|
navigation=navigation,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def require(context: RequestContext, capability: str) -> None:
|
||||||
|
if capability not in resolve(context).capabilities:
|
||||||
|
raise Unauthorized(f"portal capability is required: {capability}")
|
||||||
|
|
||||||
|
|
||||||
|
def allows(context: RequestContext, capability: str) -> bool:
|
||||||
|
return capability in resolve(context).capabilities
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
"""Unified portal HTML export with V2R1C2 resource and execution centers.
|
||||||
|
|
||||||
|
The canonical dependency-free portal remains in ``aios_v2.console``. This
|
||||||
|
module applies deterministic, fail-fast extensions at import time so the
|
||||||
|
V2R1C1 production entrypoint remains unchanged while the V2R1C2 candidate gets
|
||||||
|
resource and portable-executor views.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from aios_v2.console import CONSOLE_HTML as _BASE_PORTAL_HTML
|
||||||
|
|
||||||
|
|
||||||
|
def _extend_portal(source: str) -> str:
|
||||||
|
replacements = (
|
||||||
|
(
|
||||||
|
'<section id="overview" class="view active"></section>\n <section id="tasks" class="view"></section>\n <section id="authorization" class="view"></section>',
|
||||||
|
'<section id="overview" class="view active"></section>\n <section id="resources" class="view"></section>\n <section id="tasks" class="view"></section>\n <section id="execution" class="view"></section>\n <section id="authorization" class="view"></section>',
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"const labels={overview:'系统总览',tasks:'任务中心',authorization:'授权中心'",
|
||||||
|
"const labels={overview:'系统总览',resources:'租户与 Agent',tasks:'任务中心',execution:'执行中心',authorization:'授权中心'",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"if(view==='overview')await overview();if(view==='tasks')await tasks();if(view==='authorization')await authorizations();",
|
||||||
|
"if(view==='overview')await overview();if(view==='resources')await resources();if(view==='tasks')await tasks();if(view==='execution')await execution();if(view==='authorization')await authorizations();",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"async function tasks(){const d=await api('/api/v2/portal/tasks');",
|
||||||
|
"""async function resources(){const d=await api('/api/v2/portal/resources');const t=d.tenant||{},agents=d.agents||[],ops=d.operations||{};const agentButton=ops.agent_create?'<button class=\"btn\" id=\"newAgent\">注册 Agent</button>':'';$('#resources').innerHTML=`<div class=\"cards\">${card('租户',t.tenant_id||'—',t.status||'')}${card('Agent',d.summary?.agents||0,`启用 ${d.summary?.enabled_agents||0}`)}${card('任务',d.summary?.tasks||0)}${card('Runtime 类型',(d.summary?.runtimes||[]).length,(d.summary?.runtimes||[]).join(', '))}</div><div class=\"grid\"><div class=\"panel\"><div class=\"panel-head\"><h2>当前租户</h2></div><div class=\"kv\"><div>Tenant ID</div><div>${esc(t.tenant_id||'—')}</div><div>名称</div><div>${esc(t.display_name||'—')}</div><div>状态</div><div>${pill(t.status||'unknown')}</div><div>创建时间</div><div>${esc(t.created_at||'—')}</div><div>更新时间</div><div>${esc(t.updated_at||'—')}</div></div></div><div class=\"panel\"><div class=\"panel-head\"><h2>Agent Registry</h2><div>${agentButton}</div></div>${table([['Agent',x=>esc(x.agent_id)],['名称',x=>esc(x.display_name)],['Runtime',x=>pill(x.runtime)],['能力',x=>(x.capabilities||[]).map(pill).join('')],['启用',x=>pill(x.enabled?'enabled':'disabled')],['任务数',x=>esc(x.task_count)]],agents)}</div></div>`;$('#newAgent')?.addEventListener('click',()=>{const agent_id=prompt('Agent ID');const display_name=prompt('显示名称',agent_id||'');const runtime=prompt('Runtime Provider','openclaw');const capabilities=prompt('能力,逗号分隔','');if(!agent_id||!display_name||!runtime)return;api('/api/v2/portal/agents',{method:'POST',body:JSON.stringify({agent_id,display_name,runtime,capabilities:String(capabilities||'').split(',').map(x=>x.trim()).filter(Boolean),metadata:{source:'unified-portal'}})}).then(resources).catch(err)})}
|
||||||
|
function executionStage(x){const s=x.stage||{},status=x.truth||{},record=status.record||{},gate=status.gate||{},d=x.dependency_gate||{},notifications=gate.notifications||[],evidence=record.evidence||{};const stateName=record.state||'WAITING_DEPENDENCY';const deps=(s.depends_on||[]).join(', ')||'根阶段';return `<div class=\"panel\" style=\"background:#fbfcfe\"><div class=\"panel-head\"><h2>${esc(s.stage_id)} · ${esc(s.role)}</h2><div>${pill(stateName)} ${pill(s.executor_type)}</div></div><div class=\"chain\"><div class=\"chain-item\"><b>Provider</b><div>${esc(s.executor_type)}</div><small>${esc(deps)}</small></div><span class=\"chain-arrow\">→</span><div class=\"chain-item\"><b>Run</b><div>${esc(record.executor_run_id||'未注册')}</div><small>${esc(record.executor_id||'')}</small></div><span class=\"chain-arrow\">→</span><div class=\"chain-item\"><b>Progress</b><div>${esc(record.current_step||0)}/${esc((record.manifest?.steps||s.steps||[]).length)}</div><small>${esc(record.heartbeat_at||'无心跳')}</small></div><span class=\"chain-arrow\">→</span><div class=\"chain-item\"><b>Evidence</b><div>${Object.keys(evidence).length} 项</div><small>${Object.keys(evidence).slice(0,2).map(esc).join(', ')}</small></div><span class=\"chain-arrow\">→</span><div class=\"chain-item\"><b>Notification</b><div>${notifications.length} 条</div><small>${notifications.map(n=>n.status).join(', ')||'无'}</small></div><span class=\"chain-arrow\">→</span><div class=\"chain-item\"><b>Gate</b><div>${pill(gate.passed?'PASS':'BLOCKED')}</div><small>${d.passed?'依赖满足':'等待依赖'}</small></div></div>${Object.keys(record.error||{}).length?json(record.error):''}</div>`}
|
||||||
|
async function execution(){const d=await api('/api/v2/portal/execution');const providers=d.providers||[],plans=d.plans||[];$('#execution').innerHTML=`<div class=\"cards\">${card('Provider',providers.length)}${card('Execution Plan',plans.length)}${card('协议',d.protocol_version||'—')}${card('Plan Schema',d.plan_schema_version||'—')}</div><div class=\"panel\"><div class=\"panel-head\"><h2>Executor Provider Catalog</h2></div>${table([['Provider',x=>esc(x.executor_type)],['Adapter',x=>esc(x.adapter_version)],['协议',x=>esc(x.protocol_version)],['Trust Roles',x=>(x.trust_roles||[]).map(pill).join('')],['Capabilities',x=>(x.capabilities||[]).map(pill).join('')]],providers)}</div>${plans.map(p=>`<div class=\"panel\"><div class=\"panel-head\"><h2>${esc(p.plan_id)} · ${esc(p.target_version)}</h2><div>${pill(p.gate?.passed?'PASS':'BLOCKED')}</div></div><div class=\"kv\"><div>Source Branch</div><div>${esc(p.source_branch)}</div><div>Source OID</div><div>${esc(p.source_oid)}</div><div>Release Gate</div><div>${pill(p.gate?.passed?'PASS':'BLOCKED')}</div></div></div>${(p.snapshot?.stages||[]).map(executionStage).join('')}`).join('')||'<div class=\"empty\">尚未声明执行计划</div>'}`}
|
||||||
|
async function tasks(){const d=await api('/api/v2/portal/tasks');""",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
result = source
|
||||||
|
for old, new in replacements:
|
||||||
|
if old not in result:
|
||||||
|
raise RuntimeError(f"unified portal extension anchor not found: {old[:100]}")
|
||||||
|
result = result.replace(old, new, 1)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
PORTAL_HTML = _extend_portal(_BASE_PORTAL_HTML)
|
||||||
|
|
||||||
|
if "\n+function executionStage" in PORTAL_HTML or "\n+async function execution" in PORTAL_HTML:
|
||||||
|
raise RuntimeError("execution center contains an invalid diff marker")
|
||||||
|
|
||||||
|
__all__ = ["PORTAL_HTML"]
|
||||||
@@ -0,0 +1,130 @@
|
|||||||
|
"""HTTP facade for portable multi-provider execution plans."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import urllib.parse
|
||||||
|
from http import HTTPStatus
|
||||||
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
from aios_v2.control_plane import RequestContext, Unauthorized
|
||||||
|
from aios_v2.execution_plan import ExecutionPlanCoordinator
|
||||||
|
from aios_v2.executor_protocol import ExecutorProviderRegistry, normalize_executor
|
||||||
|
|
||||||
|
from .capabilities import require
|
||||||
|
|
||||||
|
|
||||||
|
class ExecutionPlanAPI:
|
||||||
|
"""Portal read model and provider-authenticated plan discovery API."""
|
||||||
|
|
||||||
|
INTERNAL_PREFIX = "/api/v2/internal/execution-plans"
|
||||||
|
PORTAL_PATH = "/api/v2/portal/execution"
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
coordinator: ExecutionPlanCoordinator,
|
||||||
|
*,
|
||||||
|
providers: ExecutorProviderRegistry,
|
||||||
|
) -> None:
|
||||||
|
self.coordinator = coordinator
|
||||||
|
self.providers = providers
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def is_route(cls, raw_path: str) -> bool:
|
||||||
|
path = urllib.parse.urlparse(raw_path).path.rstrip("/") or "/"
|
||||||
|
return path == cls.PORTAL_PATH or path.startswith(cls.INTERNAL_PREFIX)
|
||||||
|
|
||||||
|
def _require_provider(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
executor_type: str,
|
||||||
|
) -> str:
|
||||||
|
normalized = normalize_executor(executor_type)
|
||||||
|
if not self.providers.trusted(
|
||||||
|
executor_type=normalized,
|
||||||
|
roles=tuple(context.roles),
|
||||||
|
platform_admin="platform_admin" in context.roles,
|
||||||
|
):
|
||||||
|
raise Unauthorized(
|
||||||
|
f"trusted role for executor Provider is required: {normalized}"
|
||||||
|
)
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
def _portal_snapshot(self) -> Mapping[str, Any]:
|
||||||
|
plans = []
|
||||||
|
for plan in self.coordinator.plan_store.list():
|
||||||
|
plans.append(
|
||||||
|
{
|
||||||
|
"plan_id": plan.plan_id,
|
||||||
|
"target_version": plan.target_version,
|
||||||
|
"source_branch": plan.source_branch,
|
||||||
|
"source_oid": plan.source_oid,
|
||||||
|
"snapshot": self.coordinator.snapshot(plan.plan_id),
|
||||||
|
"gate": self.coordinator.gate(plan.plan_id),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"protocol_version": "aios.executor.v1",
|
||||||
|
"plan_schema_version": "aios.execution-plan.v2",
|
||||||
|
"providers": [
|
||||||
|
item.as_dict() for item in self.providers.list_descriptors()
|
||||||
|
],
|
||||||
|
"plans": plans,
|
||||||
|
}
|
||||||
|
|
||||||
|
def handle(
|
||||||
|
self,
|
||||||
|
method: str,
|
||||||
|
raw_path: str,
|
||||||
|
context: RequestContext,
|
||||||
|
body: Any,
|
||||||
|
) -> tuple[int, Mapping[str, Any]]:
|
||||||
|
del body
|
||||||
|
parsed = urllib.parse.urlparse(raw_path)
|
||||||
|
path = parsed.path.rstrip("/") or "/"
|
||||||
|
method = method.upper()
|
||||||
|
|
||||||
|
if method == "GET" and path == self.PORTAL_PATH:
|
||||||
|
require(context, "external_task.read")
|
||||||
|
return HTTPStatus.OK, self._portal_snapshot()
|
||||||
|
|
||||||
|
if not path.startswith(f"{self.INTERNAL_PREFIX}/"):
|
||||||
|
return HTTPStatus.NOT_FOUND, {
|
||||||
|
"error": "not_found",
|
||||||
|
"message": "execution plan route not found",
|
||||||
|
}
|
||||||
|
|
||||||
|
remainder = urllib.parse.unquote(path[len(self.INTERNAL_PREFIX) + 1 :])
|
||||||
|
parts = tuple(item for item in remainder.split("/") if item)
|
||||||
|
if len(parts) != 2:
|
||||||
|
return HTTPStatus.NOT_FOUND, {
|
||||||
|
"error": "not_found",
|
||||||
|
"message": "execution plan route not found",
|
||||||
|
}
|
||||||
|
plan_id, action = parts
|
||||||
|
query = urllib.parse.parse_qs(parsed.query)
|
||||||
|
executor_type = self._require_provider(
|
||||||
|
context,
|
||||||
|
str(query.get("executor", [""])[0]),
|
||||||
|
)
|
||||||
|
|
||||||
|
if method == "GET" and action == "discover":
|
||||||
|
items = self.coordinator.discover_ready(
|
||||||
|
plan_id=plan_id,
|
||||||
|
executor_type=executor_type,
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"plan_id": plan_id,
|
||||||
|
"executor_type": executor_type,
|
||||||
|
"count": len(items),
|
||||||
|
"items": items,
|
||||||
|
}
|
||||||
|
|
||||||
|
if method == "GET" and action == "snapshot":
|
||||||
|
return HTTPStatus.OK, self.coordinator.snapshot(plan_id)
|
||||||
|
|
||||||
|
if method == "GET" and action == "gate":
|
||||||
|
return HTTPStatus.OK, self.coordinator.gate(plan_id)
|
||||||
|
|
||||||
|
return HTTPStatus.NOT_FOUND, {
|
||||||
|
"error": "not_found",
|
||||||
|
"message": "execution plan route not found",
|
||||||
|
}
|
||||||
@@ -0,0 +1,276 @@
|
|||||||
|
"""Protected HTTP facade for portable executor truth and notification receipts."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import urllib.parse
|
||||||
|
from http import HTTPStatus
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Sequence
|
||||||
|
|
||||||
|
from aios_v2.control_plane import RequestContext, Unauthorized
|
||||||
|
from aios_v2.executor_protocol import ExecutorProviderRegistry, normalize_executor
|
||||||
|
from aios_v2.external_task_control import ExternalTaskCoordinator
|
||||||
|
from aios_v2.external_task_query import list_records
|
||||||
|
from aios_v2.task_execution_registry import ExecutionTruthState
|
||||||
|
|
||||||
|
from .capabilities import require
|
||||||
|
|
||||||
|
|
||||||
|
def _body(value: Any) -> Mapping[str, Any]:
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise ValueError("JSON body must be an object")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _required(value: Any, label: str) -> str:
|
||||||
|
normalized = str(value or "").strip()
|
||||||
|
if not normalized:
|
||||||
|
raise ValueError(f"{label} is required")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
def _mapping_strings(value: Any, label: str) -> dict[str, str]:
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise ValueError(f"{label} must be an object")
|
||||||
|
return {
|
||||||
|
_required(key, f"{label} key"): _required(item, f"{label}[{key}]")
|
||||||
|
for key, item in value.items()
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class ExternalTaskAPI:
|
||||||
|
"""Fail-closed external task API shared by every registered Provider."""
|
||||||
|
|
||||||
|
INTERNAL_PREFIX = "/api/v2/internal/external-tasks"
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
coordinator: ExternalTaskCoordinator,
|
||||||
|
*,
|
||||||
|
store_file: str | Path,
|
||||||
|
providers: ExecutorProviderRegistry,
|
||||||
|
) -> None:
|
||||||
|
self.coordinator = coordinator
|
||||||
|
self.store_file = Path(store_file).resolve()
|
||||||
|
self.providers = providers
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def is_route(path: str) -> bool:
|
||||||
|
normalized = path.rstrip("/") or "/"
|
||||||
|
return normalized.startswith(ExternalTaskAPI.INTERNAL_PREFIX) or normalized == "/api/v2/portal/external-tasks"
|
||||||
|
|
||||||
|
def _require_executor(self, context: RequestContext, executor_type: str) -> str:
|
||||||
|
normalized = normalize_executor(executor_type)
|
||||||
|
if not self.providers.trusted(
|
||||||
|
executor_type=normalized,
|
||||||
|
roles=tuple(context.roles),
|
||||||
|
platform_admin="platform_admin" in context.roles,
|
||||||
|
):
|
||||||
|
raise Unauthorized(
|
||||||
|
f"trusted role for executor Provider is required: {normalized}"
|
||||||
|
)
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
def _task_executor(self, task_id: str) -> str:
|
||||||
|
return normalize_executor(
|
||||||
|
self.coordinator.registry.get(task_id).manifest.executor
|
||||||
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _executor_identity(
|
||||||
|
context: RequestContext,
|
||||||
|
payload: Mapping[str, Any],
|
||||||
|
) -> str:
|
||||||
|
executor_id = _required(payload.get("executor_id"), "executor_id")
|
||||||
|
if executor_id != context.principal_id and "platform_admin" not in context.roles:
|
||||||
|
raise Unauthorized("executor_id must match trusted principal")
|
||||||
|
return executor_id
|
||||||
|
|
||||||
|
def declare_directory(self, directory: str | Path) -> list[Mapping[str, Any]]:
|
||||||
|
root = Path(directory).resolve()
|
||||||
|
if not root.exists():
|
||||||
|
return []
|
||||||
|
declared: list[Mapping[str, Any]] = []
|
||||||
|
for manifest in sorted(root.glob("*.json")):
|
||||||
|
record = self.coordinator.declare_file(manifest)
|
||||||
|
self.providers.validate_manifest(record.manifest)
|
||||||
|
declared.append(
|
||||||
|
{
|
||||||
|
"manifest": str(manifest),
|
||||||
|
"task_id": record.task_id,
|
||||||
|
"executor_type": record.manifest.executor,
|
||||||
|
"state": record.state.value,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return declared
|
||||||
|
|
||||||
|
def handle(
|
||||||
|
self,
|
||||||
|
method: str,
|
||||||
|
raw_path: str,
|
||||||
|
context: RequestContext,
|
||||||
|
body: Any,
|
||||||
|
) -> tuple[int, Mapping[str, Any]]:
|
||||||
|
parsed = urllib.parse.urlparse(raw_path)
|
||||||
|
path = parsed.path.rstrip("/") or "/"
|
||||||
|
method = method.upper()
|
||||||
|
payload = _body(body or {})
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/portal/external-tasks":
|
||||||
|
require(context, "external_task.read")
|
||||||
|
records = list_records(self.store_file)
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"count": len(records),
|
||||||
|
"items": [self.coordinator.status(item.task_id) for item in records],
|
||||||
|
}
|
||||||
|
|
||||||
|
if method == "GET" and path == f"{self.INTERNAL_PREFIX}/discover":
|
||||||
|
query = urllib.parse.parse_qs(parsed.query)
|
||||||
|
states = tuple(query.get("state", ()))
|
||||||
|
executor = self._require_executor(
|
||||||
|
context,
|
||||||
|
str(query.get("executor", [""])[0]),
|
||||||
|
)
|
||||||
|
records = list_records(
|
||||||
|
self.store_file,
|
||||||
|
executor=executor,
|
||||||
|
states=states,
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"count": len(records),
|
||||||
|
"executor_type": executor,
|
||||||
|
"items": [item.as_dict() for item in records],
|
||||||
|
}
|
||||||
|
|
||||||
|
if method == "GET" and path == f"{self.INTERNAL_PREFIX}/notifications":
|
||||||
|
query = urllib.parse.parse_qs(parsed.query)
|
||||||
|
executor = self._require_executor(
|
||||||
|
context,
|
||||||
|
str(query.get("executor", [""])[0]),
|
||||||
|
)
|
||||||
|
task_id = query.get("task_id", [None])[0]
|
||||||
|
status = query.get("status", [None])[0]
|
||||||
|
items = self.coordinator.notifications.list(
|
||||||
|
task_id=task_id,
|
||||||
|
status=status,
|
||||||
|
)
|
||||||
|
filtered = [
|
||||||
|
item
|
||||||
|
for item in items
|
||||||
|
if self._task_executor(item.task_id) == executor
|
||||||
|
]
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"count": len(filtered),
|
||||||
|
"executor_type": executor,
|
||||||
|
"items": [item.as_dict() for item in filtered],
|
||||||
|
}
|
||||||
|
|
||||||
|
notification_ack = self._match(
|
||||||
|
path,
|
||||||
|
f"{self.INTERNAL_PREFIX}/notifications/",
|
||||||
|
"/ack",
|
||||||
|
)
|
||||||
|
if method == "POST" and notification_ack:
|
||||||
|
notification = self.coordinator.notifications.get(notification_ack)
|
||||||
|
executor = self._task_executor(notification.task_id)
|
||||||
|
self._require_executor(context, executor)
|
||||||
|
self._executor_identity(context, payload)
|
||||||
|
record = self.coordinator.notifications.acknowledge(
|
||||||
|
notification_ack,
|
||||||
|
receipt_sha256=_required(
|
||||||
|
payload.get("receipt_sha256"),
|
||||||
|
"receipt_sha256",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {"notification": record.as_dict()}
|
||||||
|
|
||||||
|
truth_task = self._match(path, f"{self.INTERNAL_PREFIX}/", "/truth")
|
||||||
|
if method == "GET" and truth_task:
|
||||||
|
self._require_executor(context, self._task_executor(truth_task))
|
||||||
|
return HTTPStatus.OK, self.coordinator.status(truth_task)
|
||||||
|
|
||||||
|
for action in ("register", "retry", "claim", "start", "heartbeat", "succeed", "fail"):
|
||||||
|
task_id = self._match(path, f"{self.INTERNAL_PREFIX}/", f"/{action}")
|
||||||
|
if method != "POST" or not task_id:
|
||||||
|
continue
|
||||||
|
executor_type = self._task_executor(task_id)
|
||||||
|
self._require_executor(context, executor_type)
|
||||||
|
run_id = _required(payload.get("executor_run_id"), "executor_run_id")
|
||||||
|
if action == "register":
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"record": self.coordinator.register(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=run_id,
|
||||||
|
).as_dict()
|
||||||
|
}
|
||||||
|
if action == "retry":
|
||||||
|
return HTTPStatus.OK, {
|
||||||
|
"record": self.coordinator.retry_not_started(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=run_id,
|
||||||
|
).as_dict()
|
||||||
|
}
|
||||||
|
executor_id = self._executor_identity(context, payload)
|
||||||
|
if action == "claim":
|
||||||
|
record = self.coordinator.claim(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
)
|
||||||
|
elif action == "start":
|
||||||
|
record = self.coordinator.start(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
)
|
||||||
|
elif action == "heartbeat":
|
||||||
|
record = self.coordinator.heartbeat(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
current_step=int(payload.get("current_step")),
|
||||||
|
)
|
||||||
|
elif action == "succeed":
|
||||||
|
record = self.coordinator.succeed(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
evidence=_mapping_strings(payload.get("evidence"), "evidence"),
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
error = payload.get("error")
|
||||||
|
if not isinstance(error, Mapping):
|
||||||
|
raise ValueError("error must be an object")
|
||||||
|
evidence_value = payload.get("evidence")
|
||||||
|
evidence = (
|
||||||
|
_mapping_strings(evidence_value, "evidence")
|
||||||
|
if evidence_value is not None
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
record = self.coordinator.fail(
|
||||||
|
task_id,
|
||||||
|
executor_run_id=run_id,
|
||||||
|
executor_id=executor_id,
|
||||||
|
error=dict(error),
|
||||||
|
evidence=evidence,
|
||||||
|
)
|
||||||
|
return HTTPStatus.OK, {"record": record.as_dict()}
|
||||||
|
|
||||||
|
return HTTPStatus.NOT_FOUND, {
|
||||||
|
"error": "not_found",
|
||||||
|
"message": "external task route not found",
|
||||||
|
}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _match(path: str, prefix: str, suffix: str) -> str | None:
|
||||||
|
if not path.startswith(prefix) or not path.endswith(suffix):
|
||||||
|
return None
|
||||||
|
middle = path[len(prefix) : len(path) - len(suffix)]
|
||||||
|
if not middle or "/" in middle:
|
||||||
|
return None
|
||||||
|
return urllib.parse.unquote(middle)
|
||||||
|
|
||||||
|
|
||||||
|
DISCOVERABLE_STATES: Sequence[str] = (
|
||||||
|
ExecutionTruthState.DECLARED.value,
|
||||||
|
ExecutionTruthState.REGISTERED.value,
|
||||||
|
)
|
||||||
@@ -0,0 +1,376 @@
|
|||||||
|
"""V2R1C2 unified portal and portable executor release entrypoint.
|
||||||
|
|
||||||
|
The V2R1C1 production entrypoint remains unchanged. This candidate entrypoint
|
||||||
|
uses an isolated state directory, loads a versioned executor catalog, declares
|
||||||
|
portable multi-provider execution plans and exposes both portal read models and
|
||||||
|
provider-authenticated lifecycle APIs.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import urllib.parse
|
||||||
|
from dataclasses import replace
|
||||||
|
from http import HTTPStatus
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Sequence
|
||||||
|
|
||||||
|
import aios_v2.app as candidate_app_module
|
||||||
|
from aios_v2.app import (
|
||||||
|
CandidateApplication,
|
||||||
|
CandidateHTTPServer,
|
||||||
|
build_application as build_candidate_application,
|
||||||
|
)
|
||||||
|
from aios_v2.control_plane import RequestContext
|
||||||
|
from aios_v2.execution_plan import ExecutionPlanCoordinator, ExecutionPlanManifest
|
||||||
|
from aios_v2.executor_catalog import FileExecutorCatalogStore
|
||||||
|
from aios_v2.official_executor_providers import official_provider_factories
|
||||||
|
|
||||||
|
from .application import UnifiedPortalApplication, _json_value, _mapping
|
||||||
|
from .capabilities import allows, require
|
||||||
|
from .console import PORTAL_HTML
|
||||||
|
from .execution_plan_api import ExecutionPlanAPI
|
||||||
|
from .external_task_api import ExternalTaskAPI
|
||||||
|
|
||||||
|
|
||||||
|
_READ_BRIDGE_ROLES = frozenset({"approver", "runtime", "viewer", "release_manager"})
|
||||||
|
_GIT_OID = re.compile(r"^[0-9a-f]{40}$")
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_environment_value(value: Any, *, label: str) -> Any:
|
||||||
|
if not isinstance(value, str) or not value.startswith("ENV:"):
|
||||||
|
return value
|
||||||
|
variable = value[4:].strip()
|
||||||
|
if not variable:
|
||||||
|
raise ValueError(f"{label} environment variable name is empty")
|
||||||
|
resolved = str(os.getenv(variable, "")).strip()
|
||||||
|
if not resolved:
|
||||||
|
raise RuntimeError(f"required environment variable is not set: {variable}")
|
||||||
|
return resolved
|
||||||
|
|
||||||
|
|
||||||
|
def _load_execution_plan(path: str | Path) -> ExecutionPlanManifest:
|
||||||
|
source = Path(path).resolve()
|
||||||
|
value = json.loads(source.read_text(encoding="utf-8"))
|
||||||
|
if not isinstance(value, Mapping):
|
||||||
|
raise ValueError(f"execution plan must contain a JSON object: {source}")
|
||||||
|
payload = dict(value)
|
||||||
|
for label in ("source_branch", "source_oid", "target_version"):
|
||||||
|
payload[label] = _resolve_environment_value(payload.get(label), label=label)
|
||||||
|
source_oid = str(payload.get("source_oid") or "").strip().lower()
|
||||||
|
if not _GIT_OID.fullmatch(source_oid):
|
||||||
|
raise ValueError(
|
||||||
|
"execution plan source_oid must resolve to an exact 40-character Git OID"
|
||||||
|
)
|
||||||
|
payload["source_oid"] = source_oid
|
||||||
|
return ExecutionPlanManifest.from_dict(payload)
|
||||||
|
|
||||||
|
|
||||||
|
def _path_list(value: str) -> tuple[Path, ...]:
|
||||||
|
return tuple(
|
||||||
|
Path(item).expanduser().resolve()
|
||||||
|
for item in value.split(os.pathsep)
|
||||||
|
if item.strip()
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class V2R1C2PortalApplication(UnifiedPortalApplication):
|
||||||
|
"""Unified portal candidate with provider-neutral execution governance."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
base: CandidateApplication,
|
||||||
|
*,
|
||||||
|
provenance_store_file: str | Path,
|
||||||
|
external_task_api: ExternalTaskAPI,
|
||||||
|
execution_plan_api: ExecutionPlanAPI,
|
||||||
|
plan_coordinator: ExecutionPlanCoordinator,
|
||||||
|
catalog_snapshot: Mapping[str, Any],
|
||||||
|
declared_plans: Sequence[str],
|
||||||
|
) -> None:
|
||||||
|
super().__init__(base, provenance_store_file=provenance_store_file)
|
||||||
|
self.external_task_api = external_task_api
|
||||||
|
self.execution_plan_api = execution_plan_api
|
||||||
|
self.plan_coordinator = plan_coordinator
|
||||||
|
self.catalog_snapshot = dict(catalog_snapshot)
|
||||||
|
self.declared_plans = tuple(declared_plans)
|
||||||
|
|
||||||
|
def context(self, headers: Mapping[str, str]) -> RequestContext:
|
||||||
|
original = self.base.context(headers)
|
||||||
|
if original.roles.intersection(_READ_BRIDGE_ROLES):
|
||||||
|
return replace(original, roles=frozenset((*original.roles, "auditor")))
|
||||||
|
return original
|
||||||
|
|
||||||
|
def _resource_snapshot(self, context: RequestContext) -> Mapping[str, Any]:
|
||||||
|
service = self.orchestrator.control_plane
|
||||||
|
tenant = service.get_tenant(context)
|
||||||
|
agents = service.list_agents(context) if allows(context, "agent.read") else []
|
||||||
|
tasks = service.list_tasks(context) if allows(context, "task.read") else []
|
||||||
|
tasks_by_agent: dict[str, int] = {}
|
||||||
|
for task in tasks:
|
||||||
|
tasks_by_agent[task.agent_id] = tasks_by_agent.get(task.agent_id, 0) + 1
|
||||||
|
return {
|
||||||
|
"tenant": _json_value(tenant),
|
||||||
|
"agents": [
|
||||||
|
{
|
||||||
|
**_json_value(agent),
|
||||||
|
"task_count": tasks_by_agent.get(agent.agent_id, 0),
|
||||||
|
}
|
||||||
|
for agent in agents
|
||||||
|
],
|
||||||
|
"summary": {
|
||||||
|
"agents": len(agents),
|
||||||
|
"enabled_agents": sum(1 for agent in agents if agent.enabled),
|
||||||
|
"tasks": len(tasks),
|
||||||
|
"runtimes": sorted({agent.runtime for agent in agents}),
|
||||||
|
},
|
||||||
|
"operations": {
|
||||||
|
"tenant_create": allows(context, "tenant.manage"),
|
||||||
|
"agent_create": allows(context, "agent.manage"),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
def _runtime_snapshot(
|
||||||
|
self,
|
||||||
|
context: RequestContext,
|
||||||
|
headers: Mapping[str, str],
|
||||||
|
) -> Mapping[str, Any]:
|
||||||
|
return {
|
||||||
|
**self._runtime(headers),
|
||||||
|
"dispatches": _json_value(
|
||||||
|
self.orchestrator.dispatch_repository.list(
|
||||||
|
tenant_id=context.tenant_id
|
||||||
|
)
|
||||||
|
),
|
||||||
|
"hermes_jobs": _json_value(
|
||||||
|
self.hermes.list_jobs(tenant_id=context.tenant_id)
|
||||||
|
),
|
||||||
|
"contract": {
|
||||||
|
"openclaw": os.getenv(
|
||||||
|
"AIOS_OPENCLAW_CONTRACT_VERSION",
|
||||||
|
"aios.openclaw.command.v2",
|
||||||
|
),
|
||||||
|
"signature_algorithm": os.getenv(
|
||||||
|
"AIOS_OPENCLAW_SIGNATURE_ALGORITHM",
|
||||||
|
"Ed25519",
|
||||||
|
),
|
||||||
|
"plugin_version": os.getenv(
|
||||||
|
"AIOS_OPENCLAW_PLUGIN_VERSION",
|
||||||
|
"2.1.0-rc.4",
|
||||||
|
),
|
||||||
|
"key_id": os.getenv(
|
||||||
|
"AIOS_OPENCLAW_SIGNING_KEY_ID",
|
||||||
|
"openclaw-v2r1-candidate-ed25519",
|
||||||
|
),
|
||||||
|
"executor_protocol": "aios.executor.v1",
|
||||||
|
"execution_plan": "aios.execution-plan.v2",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
def _release_snapshot(self, context: RequestContext) -> Mapping[str, Any]:
|
||||||
|
plans = []
|
||||||
|
for plan_id in self.declared_plans:
|
||||||
|
plans.append(
|
||||||
|
{
|
||||||
|
"plan_id": plan_id,
|
||||||
|
"gate": self.plan_coordinator.gate(plan_id),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
aggregate_passed = bool(plans) and all(
|
||||||
|
item["gate"]["passed"] for item in plans
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
**self._gitops(context),
|
||||||
|
"version": os.getenv("AIOS_RELEASE_VERSION", "V2R1C2-CANDIDATE"),
|
||||||
|
"target_version": os.getenv("AIOS_TARGET_VERSION", "V2R1C2"),
|
||||||
|
"release_oid": os.getenv("AIOS_RELEASE_OID", "candidate-not-tagged"),
|
||||||
|
"source_oid": os.getenv("AIOS_RELEASE_SOURCE_OID", "missing"),
|
||||||
|
"tag": os.getenv("AIOS_RELEASE_TAG", "NOT_CREATED"),
|
||||||
|
"baseline": os.getenv("AIOS_BASELINE_VERSION", "V2R1C1"),
|
||||||
|
"previous_version": os.getenv("AIOS_PREVIOUS_VERSION", "V2R1C1"),
|
||||||
|
"status": os.getenv("AIOS_RELEASE_STATUS", "CANDIDATE_VALIDATION"),
|
||||||
|
"evidence_manifest_sha256": os.getenv(
|
||||||
|
"AIOS_RELEASE_EVIDENCE_SHA256",
|
||||||
|
"pending",
|
||||||
|
),
|
||||||
|
"executor_catalog": {
|
||||||
|
"path": self.catalog_snapshot.get("path"),
|
||||||
|
"sha256": self.catalog_snapshot.get("catalog_sha256"),
|
||||||
|
"protocol_version": "aios.executor.v1",
|
||||||
|
},
|
||||||
|
"execution_plans": plans,
|
||||||
|
"execution_gate": {
|
||||||
|
"passed": aggregate_passed,
|
||||||
|
"release_blocking": not aggregate_passed,
|
||||||
|
},
|
||||||
|
"operations": (
|
||||||
|
["review_release", "request_tag", "request_rollback"]
|
||||||
|
if allows(context, "release.operate")
|
||||||
|
else []
|
||||||
|
),
|
||||||
|
"note": (
|
||||||
|
"Release operations remain GitOps approval requests. The portal "
|
||||||
|
"never mutates tags, main, or production directly. Tag approval "
|
||||||
|
"must remain blocked until every required execution-plan gate passes."
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
def handle_json(
|
||||||
|
self,
|
||||||
|
method: str,
|
||||||
|
raw_path: str,
|
||||||
|
headers: Mapping[str, str],
|
||||||
|
body: Any,
|
||||||
|
) -> tuple[int, Mapping[str, Any]]:
|
||||||
|
parsed = urllib.parse.urlparse(raw_path)
|
||||||
|
path = parsed.path.rstrip("/") or "/"
|
||||||
|
method = method.upper()
|
||||||
|
|
||||||
|
if self.execution_plan_api.is_route(raw_path):
|
||||||
|
context = (
|
||||||
|
self.context(headers)
|
||||||
|
if path == ExecutionPlanAPI.PORTAL_PATH
|
||||||
|
else self.base.context(headers)
|
||||||
|
)
|
||||||
|
return self.execution_plan_api.handle(
|
||||||
|
method,
|
||||||
|
raw_path,
|
||||||
|
context,
|
||||||
|
body,
|
||||||
|
)
|
||||||
|
|
||||||
|
if self.external_task_api.is_route(path):
|
||||||
|
context = (
|
||||||
|
self.context(headers)
|
||||||
|
if path == "/api/v2/portal/external-tasks"
|
||||||
|
else self.base.context(headers)
|
||||||
|
)
|
||||||
|
return self.external_task_api.handle(
|
||||||
|
method,
|
||||||
|
raw_path,
|
||||||
|
context,
|
||||||
|
body,
|
||||||
|
)
|
||||||
|
|
||||||
|
context: RequestContext | None = None
|
||||||
|
if path in {
|
||||||
|
"/api/v2/portal/resources",
|
||||||
|
"/api/v2/portal/runtime",
|
||||||
|
"/api/v2/portal/release",
|
||||||
|
"/api/v2/portal/tenants",
|
||||||
|
"/api/v2/portal/agents",
|
||||||
|
}:
|
||||||
|
context = self.context(headers)
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/portal/resources":
|
||||||
|
assert context is not None
|
||||||
|
require(context, "tenant.read")
|
||||||
|
return HTTPStatus.OK, self._resource_snapshot(context)
|
||||||
|
|
||||||
|
if method == "POST" and path == "/api/v2/portal/tenants":
|
||||||
|
assert context is not None
|
||||||
|
require(context, "tenant.manage")
|
||||||
|
return self.base.handle_json(
|
||||||
|
"POST",
|
||||||
|
"/api/v2/tenants",
|
||||||
|
headers,
|
||||||
|
_mapping(body),
|
||||||
|
)
|
||||||
|
|
||||||
|
if method == "POST" and path == "/api/v2/portal/agents":
|
||||||
|
assert context is not None
|
||||||
|
require(context, "agent.manage")
|
||||||
|
return self.base.handle_json(
|
||||||
|
"POST",
|
||||||
|
"/api/v2/agents",
|
||||||
|
headers,
|
||||||
|
_mapping(body),
|
||||||
|
)
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/portal/runtime":
|
||||||
|
assert context is not None
|
||||||
|
require(context, "runtime.read")
|
||||||
|
return HTTPStatus.OK, self._runtime_snapshot(context, headers)
|
||||||
|
|
||||||
|
if method == "GET" and path == "/api/v2/portal/release":
|
||||||
|
assert context is not None
|
||||||
|
require(context, "release.read")
|
||||||
|
return HTTPStatus.OK, self._release_snapshot(context)
|
||||||
|
|
||||||
|
return super().handle_json(method, raw_path, headers, body)
|
||||||
|
|
||||||
|
|
||||||
|
def build_application() -> V2R1C2PortalApplication:
|
||||||
|
state_dir = Path(
|
||||||
|
os.getenv("AIOS_V2_STATE_DIR", "/var/lib/aios-v2r1c2-portal")
|
||||||
|
).resolve()
|
||||||
|
state_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
provenance_file = Path(
|
||||||
|
os.getenv(
|
||||||
|
"AIOS_PROVENANCE_STORE_FILE",
|
||||||
|
str(state_dir / "run-provenance.sqlite3"),
|
||||||
|
)
|
||||||
|
).resolve()
|
||||||
|
truth_file = Path(
|
||||||
|
os.getenv(
|
||||||
|
"AIOS_EXTERNAL_TASK_STORE_FILE",
|
||||||
|
str(state_dir / "external-task-truth.sqlite3"),
|
||||||
|
)
|
||||||
|
).resolve()
|
||||||
|
catalog_file = Path(
|
||||||
|
os.getenv(
|
||||||
|
"AIOS_EXECUTOR_CATALOG_FILE",
|
||||||
|
str(state_dir / "executor-catalog.json"),
|
||||||
|
)
|
||||||
|
).resolve()
|
||||||
|
|
||||||
|
catalog_store = FileExecutorCatalogStore(catalog_file)
|
||||||
|
catalog_snapshot = catalog_store.initialize_defaults()
|
||||||
|
providers = catalog_store.registry(
|
||||||
|
provider_factories=official_provider_factories()
|
||||||
|
)
|
||||||
|
plan_coordinator = ExecutionPlanCoordinator.from_file(
|
||||||
|
truth_file,
|
||||||
|
providers=providers,
|
||||||
|
)
|
||||||
|
|
||||||
|
declared_plans: list[str] = []
|
||||||
|
plan_files = _path_list(os.getenv("AIOS_EXECUTION_PLAN_FILES", ""))
|
||||||
|
for plan_file in plan_files:
|
||||||
|
plan = _load_execution_plan(plan_file)
|
||||||
|
plan_coordinator.declare(plan)
|
||||||
|
declared_plans.append(plan.plan_id)
|
||||||
|
|
||||||
|
base: CandidateApplication = build_candidate_application()
|
||||||
|
external_task_api = ExternalTaskAPI(
|
||||||
|
plan_coordinator.task_coordinator,
|
||||||
|
store_file=truth_file,
|
||||||
|
providers=providers,
|
||||||
|
)
|
||||||
|
execution_plan_api = ExecutionPlanAPI(
|
||||||
|
plan_coordinator,
|
||||||
|
providers=providers,
|
||||||
|
)
|
||||||
|
return V2R1C2PortalApplication(
|
||||||
|
base,
|
||||||
|
provenance_store_file=provenance_file,
|
||||||
|
external_task_api=external_task_api,
|
||||||
|
execution_plan_api=execution_plan_api,
|
||||||
|
plan_coordinator=plan_coordinator,
|
||||||
|
catalog_snapshot=catalog_snapshot.as_dict(),
|
||||||
|
declared_plans=declared_plans,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
candidate_app_module.CONSOLE_HTML = PORTAL_HTML
|
||||||
|
host = os.getenv("AIOS_V2_HOST", "127.0.0.1")
|
||||||
|
port = int(os.getenv("AIOS_V2_PORT", "18921"))
|
||||||
|
server = CandidateHTTPServer((host, port), build_application())
|
||||||
|
print(f"AIOS V2R1C2 unified portal listening on {host}:{port}")
|
||||||
|
server.serve_forever()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -1,652 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""One-shot governed executor for an already approved exact release binding.
|
|
||||||
|
|
||||||
This module deliberately does not build release artifacts. The build/staging
|
|
||||||
evidence is completed before Owner production approval. Production execution
|
|
||||||
consumes that immutable binding, revalidates it, and performs narrow
|
|
||||||
side-effects through adapters in a durable fail-closed order.
|
|
||||||
|
|
||||||
Adapter protocol:
|
|
||||||
stdin: JSON request with operation = "probe" | "apply" | "rollback"
|
|
||||||
stdout: one JSON object on the final non-empty line
|
|
||||||
shell: never used
|
|
||||||
|
|
||||||
The executor never rewrites tags, never writes refs/heads/main, and records
|
|
||||||
Baseline last.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import dataclasses
|
|
||||||
import fcntl
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import re
|
|
||||||
import stat
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import tempfile
|
|
||||||
import time
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Mapping, Sequence
|
|
||||||
|
|
||||||
OID_RE = re.compile(r"^[0-9a-f]{40}$")
|
|
||||||
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
|
|
||||||
PLAN_RE = re.compile(r"^sha256:[0-9a-f]{64}$")
|
|
||||||
|
|
||||||
REQUIRED_V2R1C7_MEMBERS = {
|
|
||||||
"payload/VERSION",
|
|
||||||
"payload/active-runtime/app/deployment_state.py",
|
|
||||||
"payload/active-runtime/app/v2r1c6_privacy_tools.py",
|
|
||||||
"payload/active-runtime/app/v2r1c7_health_service.py",
|
|
||||||
"payload/deploy/aios-v2/systemd/aios-v2r1c7-production.service",
|
|
||||||
"payload/deploy/aios-v2/v2r1c7-runtime-config.json",
|
|
||||||
"payload/deploy/aios-v2/v2r1c7-rollback.json",
|
|
||||||
"payload/scripts/probe_v2r1c7_health.py",
|
|
||||||
"payload/scripts/rollback_v2r1c7_release.sh",
|
|
||||||
"payload/BUILD-METADATA.json",
|
|
||||||
"payload/ARTIFACT-MANIFEST.json",
|
|
||||||
"payload/ROLLBACK-METADATA.json",
|
|
||||||
}
|
|
||||||
|
|
||||||
class ReleaseBlocked(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
class ManualOwnerReviewRequired(ReleaseBlocked):
|
|
||||||
pass
|
|
||||||
|
|
||||||
class ReconciliationRequired(ReleaseBlocked):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
@dataclasses.dataclass(frozen=True)
|
|
||||||
class Binding:
|
|
||||||
version: str
|
|
||||||
target_oid: str
|
|
||||||
evidence_closure_oid: str
|
|
||||||
artifact_sha256: str
|
|
||||||
approval_plan_id: str
|
|
||||||
artifact_path: str
|
|
||||||
artifact_size: int
|
|
||||||
production_service: str
|
|
||||||
production_port: int
|
|
||||||
repository_identity: str = "aios/aios"
|
|
||||||
|
|
||||||
def __post_init__(self) -> None:
|
|
||||||
if not self.version or not OID_RE.fullmatch(self.target_oid):
|
|
||||||
raise ValueError("invalid release target binding")
|
|
||||||
if not OID_RE.fullmatch(self.evidence_closure_oid):
|
|
||||||
raise ValueError("invalid evidence closure OID")
|
|
||||||
if not SHA_RE.fullmatch(self.artifact_sha256):
|
|
||||||
raise ValueError("invalid artifact SHA-256")
|
|
||||||
if not PLAN_RE.fullmatch(self.approval_plan_id):
|
|
||||||
raise ValueError("invalid approval plan id")
|
|
||||||
if self.artifact_size <= 0:
|
|
||||||
raise ValueError("invalid artifact size")
|
|
||||||
if not self.production_service:
|
|
||||||
raise ValueError("production service is required")
|
|
||||||
if not 1024 <= self.production_port <= 65535:
|
|
||||||
raise ValueError("invalid production port")
|
|
||||||
if self.repository_identity != "aios/aios":
|
|
||||||
raise ValueError("unexpected repository identity")
|
|
||||||
|
|
||||||
@property
|
|
||||||
def tag(self) -> str:
|
|
||||||
return self.version
|
|
||||||
|
|
||||||
@property
|
|
||||||
def fingerprint(self) -> str:
|
|
||||||
payload = {
|
|
||||||
"version": self.version,
|
|
||||||
"target_oid": self.target_oid,
|
|
||||||
"evidence_closure_oid": self.evidence_closure_oid,
|
|
||||||
"artifact_sha256": self.artifact_sha256,
|
|
||||||
"approval_plan_id": self.approval_plan_id,
|
|
||||||
"artifact_size": self.artifact_size,
|
|
||||||
"production_service": self.production_service,
|
|
||||||
"production_port": self.production_port,
|
|
||||||
"repository_identity": self.repository_identity,
|
|
||||||
}
|
|
||||||
return hashlib.sha256(canonical_json(payload)).hexdigest()
|
|
||||||
|
|
||||||
def request(self, operation: str) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"schema": "aios.production-executor-request.v1",
|
|
||||||
"operation": operation,
|
|
||||||
"binding_fingerprint": self.fingerprint,
|
|
||||||
"version": self.version,
|
|
||||||
"tag": self.tag,
|
|
||||||
"target_oid": self.target_oid,
|
|
||||||
"evidence_closure_oid": self.evidence_closure_oid,
|
|
||||||
"artifact_sha256": self.artifact_sha256,
|
|
||||||
"artifact_path": self.artifact_path,
|
|
||||||
"artifact_size": self.artifact_size,
|
|
||||||
"approval_plan_id": self.approval_plan_id,
|
|
||||||
"production_service": self.production_service,
|
|
||||||
"production_port": self.production_port,
|
|
||||||
"repository_identity": self.repository_identity,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def canonical_json(value: object) -> bytes:
|
|
||||||
return (json.dumps(value, sort_keys=True, separators=(",", ":")) + "\n").encode("utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
def secure_json(path: Path) -> dict[str, Any]:
|
|
||||||
st = path.lstat()
|
|
||||||
if not stat.S_ISREG(st.st_mode) or path.is_symlink():
|
|
||||||
raise ReleaseBlocked(f"non-regular authorization file: {path}")
|
|
||||||
if st.st_uid != 0:
|
|
||||||
raise ReleaseBlocked(f"authorization file must be root-owned: {path}")
|
|
||||||
if st.st_mode & 0o022:
|
|
||||||
raise ReleaseBlocked(f"authorization file is group/world writable: {path}")
|
|
||||||
data = json.loads(path.read_text(encoding="utf-8"))
|
|
||||||
if not isinstance(data, dict):
|
|
||||||
raise ReleaseBlocked("authorization receipt must be a JSON object")
|
|
||||||
return data
|
|
||||||
|
|
||||||
|
|
||||||
def sha256_file(path: Path) -> str:
|
|
||||||
if path.is_symlink() or not path.is_file():
|
|
||||||
raise ReleaseBlocked("artifact must be a regular non-symlink file")
|
|
||||||
digest = hashlib.sha256()
|
|
||||||
with path.open("rb") as source:
|
|
||||||
for block in iter(lambda: source.read(1024 * 1024), b""):
|
|
||||||
digest.update(block)
|
|
||||||
return digest.hexdigest()
|
|
||||||
|
|
||||||
|
|
||||||
def run_git(git_dir: Path, *args: str, check: bool = True) -> subprocess.CompletedProcess[str]:
|
|
||||||
return subprocess.run(
|
|
||||||
["git", f"--git-dir={git_dir}", *args],
|
|
||||||
check=check,
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
timeout=30,
|
|
||||||
env={"PATH": os.environ.get("PATH", "")},
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def verify_git_lineage(git_dir: Path, binding: Binding) -> dict[str, Any]:
|
|
||||||
authority_main = run_git(git_dir, "rev-parse", "refs/heads/main").stdout.strip()
|
|
||||||
if not OID_RE.fullmatch(authority_main):
|
|
||||||
raise ReleaseBlocked("Authority main OID is invalid")
|
|
||||||
|
|
||||||
target_type = run_git(git_dir, "cat-file", "-t", binding.target_oid).stdout.strip()
|
|
||||||
if target_type != "commit":
|
|
||||||
raise ReleaseBlocked("target_oid is not a commit")
|
|
||||||
|
|
||||||
evidence_type = run_git(
|
|
||||||
git_dir, "cat-file", "-t", binding.evidence_closure_oid
|
|
||||||
).stdout.strip()
|
|
||||||
if evidence_type != "commit":
|
|
||||||
raise ReleaseBlocked("evidence_closure_oid is not a commit")
|
|
||||||
|
|
||||||
# A control-plane-only patch may advance main after release approval.
|
|
||||||
# The approved release target remains valid only while it is still on main.
|
|
||||||
target_on_main = run_git(
|
|
||||||
git_dir,
|
|
||||||
"merge-base",
|
|
||||||
"--is-ancestor",
|
|
||||||
binding.target_oid,
|
|
||||||
authority_main,
|
|
||||||
check=False,
|
|
||||||
).returncode == 0
|
|
||||||
if not target_on_main:
|
|
||||||
raise ManualOwnerReviewRequired(
|
|
||||||
"approved target is no longer an ancestor of Authority main"
|
|
||||||
)
|
|
||||||
|
|
||||||
target_in_evidence = run_git(
|
|
||||||
git_dir,
|
|
||||||
"merge-base",
|
|
||||||
"--is-ancestor",
|
|
||||||
binding.target_oid,
|
|
||||||
binding.evidence_closure_oid,
|
|
||||||
check=False,
|
|
||||||
).returncode == 0
|
|
||||||
if not target_in_evidence:
|
|
||||||
raise ReleaseBlocked("approved target is not an ancestor of evidence closure")
|
|
||||||
|
|
||||||
return {
|
|
||||||
"authority_main_oid": authority_main,
|
|
||||||
"target_oid": binding.target_oid,
|
|
||||||
"target_is_ancestor_of_main": True,
|
|
||||||
"evidence_closure_oid": binding.evidence_closure_oid,
|
|
||||||
"evidence_object_type": evidence_type,
|
|
||||||
"target_is_ancestor_of_evidence": True,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def verify_artifact(binding: Binding) -> dict[str, Any]:
|
|
||||||
path = Path(binding.artifact_path)
|
|
||||||
if path.stat().st_size != binding.artifact_size:
|
|
||||||
raise ReleaseBlocked("artifact size mismatch")
|
|
||||||
actual = sha256_file(path)
|
|
||||||
if actual != binding.artifact_sha256:
|
|
||||||
raise ReleaseBlocked("artifact SHA-256 mismatch")
|
|
||||||
|
|
||||||
if binding.version == "V2R1C7":
|
|
||||||
# Python tarfile may not have zstd support on the target host, so
|
|
||||||
# decompress into a private temporary tar and inspect it there.
|
|
||||||
with tempfile.TemporaryDirectory(prefix="aios-artifact-verify-") as temp_dir:
|
|
||||||
tar_path = Path(temp_dir) / "release.tar"
|
|
||||||
with tar_path.open("wb") as output:
|
|
||||||
completed = subprocess.run(
|
|
||||||
["zstd", "-dc", str(path)],
|
|
||||||
check=False,
|
|
||||||
stdout=output,
|
|
||||||
stderr=subprocess.PIPE,
|
|
||||||
timeout=60,
|
|
||||||
env={"PATH": os.environ.get("PATH", "")},
|
|
||||||
)
|
|
||||||
if completed.returncode != 0:
|
|
||||||
raise ReleaseBlocked("artifact zstd decompression failed")
|
|
||||||
import tarfile
|
|
||||||
with tarfile.open(tar_path, "r:") as archive:
|
|
||||||
names: set[str] = set()
|
|
||||||
for member in archive.getmembers():
|
|
||||||
candidate = Path(member.name)
|
|
||||||
if (
|
|
||||||
candidate.is_absolute()
|
|
||||||
or ".." in candidate.parts
|
|
||||||
or member.issym()
|
|
||||||
or member.islnk()
|
|
||||||
or not (member.isfile() or member.isdir())
|
|
||||||
):
|
|
||||||
raise ReleaseBlocked(f"unsafe artifact member: {member.name}")
|
|
||||||
names.add(member.name)
|
|
||||||
missing = sorted(REQUIRED_V2R1C7_MEMBERS - names)
|
|
||||||
if missing:
|
|
||||||
raise ReleaseBlocked(
|
|
||||||
"artifact missing required members: " + ",".join(missing)
|
|
||||||
)
|
|
||||||
return {
|
|
||||||
"artifact_path": str(path),
|
|
||||||
"artifact_sha256": actual,
|
|
||||||
"artifact_size": binding.artifact_size,
|
|
||||||
"verified": True,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def verify_authorization(receipt: Mapping[str, Any], binding: Binding) -> dict[str, Any]:
|
|
||||||
if receipt.get("accepted") is not True:
|
|
||||||
raise ReleaseBlocked("Owner production authorization is not accepted")
|
|
||||||
if receipt.get("binding_replaced") not in (None, False):
|
|
||||||
raise ManualOwnerReviewRequired("Owner binding was replaced")
|
|
||||||
expected = {
|
|
||||||
"version": binding.version,
|
|
||||||
"target_oid": binding.target_oid,
|
|
||||||
"evidence_closure_oid": binding.evidence_closure_oid,
|
|
||||||
"artifact_sha256": binding.artifact_sha256,
|
|
||||||
"approval_plan_id": binding.approval_plan_id,
|
|
||||||
}
|
|
||||||
for key, value in expected.items():
|
|
||||||
if receipt.get(key) != value:
|
|
||||||
raise ManualOwnerReviewRequired(f"Owner authorization mismatch: {key}")
|
|
||||||
return {"accepted": True, **expected}
|
|
||||||
|
|
||||||
|
|
||||||
class DurableState:
|
|
||||||
def __init__(self, root: Path, binding: Binding):
|
|
||||||
self.root = root
|
|
||||||
self.root.mkdir(parents=True, exist_ok=True)
|
|
||||||
os.chmod(self.root, 0o700)
|
|
||||||
self.path = self.root / "execution.json"
|
|
||||||
self.binding = binding
|
|
||||||
self.value: dict[str, Any]
|
|
||||||
if self.path.exists():
|
|
||||||
existing = json.loads(self.path.read_text(encoding="utf-8"))
|
|
||||||
if existing.get("binding_fingerprint") != binding.fingerprint:
|
|
||||||
raise ManualOwnerReviewRequired(
|
|
||||||
"state root contains a different production binding"
|
|
||||||
)
|
|
||||||
self.value = existing
|
|
||||||
else:
|
|
||||||
self.value = {
|
|
||||||
"schema": "aios.production-execution-state.v1",
|
|
||||||
"binding_fingerprint": binding.fingerprint,
|
|
||||||
"version": binding.version,
|
|
||||||
"target_oid": binding.target_oid,
|
|
||||||
"state": "AUTHORIZED",
|
|
||||||
"completed": {},
|
|
||||||
"history": [],
|
|
||||||
}
|
|
||||||
self.save()
|
|
||||||
|
|
||||||
def save(self) -> None:
|
|
||||||
temporary = self.path.with_suffix(".tmp")
|
|
||||||
temporary.write_bytes(canonical_json(self.value))
|
|
||||||
os.chmod(temporary, 0o600)
|
|
||||||
os.replace(temporary, self.path)
|
|
||||||
|
|
||||||
def record(self, state: str, step: str, result: Mapping[str, Any]) -> None:
|
|
||||||
now = time.time()
|
|
||||||
sanitized = json.loads(json.dumps(result))
|
|
||||||
self.value["state"] = state
|
|
||||||
self.value["completed"][step] = sanitized
|
|
||||||
self.value["history"].append(
|
|
||||||
{"at": now, "state": state, "step": step, "result": sanitized}
|
|
||||||
)
|
|
||||||
self.save()
|
|
||||||
|
|
||||||
def fail(self, state: str, reason: str) -> None:
|
|
||||||
self.value["state"] = state
|
|
||||||
self.value["history"].append(
|
|
||||||
{"at": time.time(), "state": state, "reason": reason}
|
|
||||||
)
|
|
||||||
self.save()
|
|
||||||
|
|
||||||
|
|
||||||
class FileLock:
|
|
||||||
def __init__(self, path: Path):
|
|
||||||
self.path = path
|
|
||||||
self.handle = None
|
|
||||||
|
|
||||||
def __enter__(self) -> "FileLock":
|
|
||||||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
self.handle = self.path.open("a+", encoding="utf-8")
|
|
||||||
os.chmod(self.path, 0o600)
|
|
||||||
try:
|
|
||||||
fcntl.flock(
|
|
||||||
self.handle.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB
|
|
||||||
)
|
|
||||||
except BlockingIOError as exc:
|
|
||||||
raise ReleaseBlocked("production executor is already running") from exc
|
|
||||||
return self
|
|
||||||
|
|
||||||
def __exit__(self, exc_type, exc, tb) -> None:
|
|
||||||
if self.handle is not None:
|
|
||||||
fcntl.flock(self.handle.fileno(), fcntl.LOCK_UN)
|
|
||||||
self.handle.close()
|
|
||||||
|
|
||||||
|
|
||||||
def parse_command(values: Sequence[str]) -> list[str]:
|
|
||||||
command = [str(x) for x in values if str(x)]
|
|
||||||
if not command:
|
|
||||||
raise ValueError("adapter command is required")
|
|
||||||
if not os.path.isabs(command[0]):
|
|
||||||
raise ValueError("adapter executable must be an absolute path")
|
|
||||||
return command
|
|
||||||
|
|
||||||
|
|
||||||
def run_adapter(command: Sequence[str], request: Mapping[str, Any]) -> dict[str, Any]:
|
|
||||||
completed = subprocess.run(
|
|
||||||
list(command),
|
|
||||||
input=canonical_json(request),
|
|
||||||
check=False,
|
|
||||||
capture_output=True,
|
|
||||||
timeout=180,
|
|
||||||
env={"PATH": os.environ.get("PATH", "")},
|
|
||||||
)
|
|
||||||
if completed.returncode != 0:
|
|
||||||
raise ReleaseBlocked(
|
|
||||||
f"adapter failed: {Path(command[0]).name} rc={completed.returncode}"
|
|
||||||
)
|
|
||||||
lines = [line for line in completed.stdout.decode("utf-8").splitlines() if line.strip()]
|
|
||||||
if not lines:
|
|
||||||
raise ReleaseBlocked("adapter returned no JSON result")
|
|
||||||
try:
|
|
||||||
value = json.loads(lines[-1])
|
|
||||||
except ValueError as exc:
|
|
||||||
raise ReleaseBlocked("adapter returned invalid JSON") from exc
|
|
||||||
if not isinstance(value, dict):
|
|
||||||
raise ReleaseBlocked("adapter result must be an object")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def require_exact_response(result: Mapping[str, Any], binding: Binding) -> None:
|
|
||||||
for key, expected in {
|
|
||||||
"version": binding.version,
|
|
||||||
"target_oid": binding.target_oid,
|
|
||||||
"evidence_closure_oid": binding.evidence_closure_oid,
|
|
||||||
"artifact_sha256": binding.artifact_sha256,
|
|
||||||
"approval_plan_id": binding.approval_plan_id,
|
|
||||||
}.items():
|
|
||||||
if key in result and result[key] != expected:
|
|
||||||
raise ManualOwnerReviewRequired(f"adapter binding mismatch: {key}")
|
|
||||||
|
|
||||||
|
|
||||||
def tag_step(command: Sequence[str], binding: Binding) -> dict[str, Any]:
|
|
||||||
request = binding.request("probe")
|
|
||||||
probe = run_adapter(command, request)
|
|
||||||
require_exact_response(probe, binding)
|
|
||||||
state = probe.get("status")
|
|
||||||
if state == "CONFLICT":
|
|
||||||
raise ManualOwnerReviewRequired("same-name tag conflicts with approved binding")
|
|
||||||
if state == "INDETERMINATE":
|
|
||||||
raise ReconciliationRequired("tag publication state is indeterminate")
|
|
||||||
if state not in {"APPLIED_VERIFIED", "NOT_APPLIED", "PARTIAL_LOCAL"}:
|
|
||||||
raise ReleaseBlocked(f"unsupported tag probe status: {state}")
|
|
||||||
if state != "APPLIED_VERIFIED":
|
|
||||||
applied = run_adapter(command, binding.request("apply"))
|
|
||||||
require_exact_response(applied, binding)
|
|
||||||
if applied.get("status") == "CONFLICT":
|
|
||||||
raise ManualOwnerReviewRequired("tag conflict during publication")
|
|
||||||
probe = run_adapter(command, request)
|
|
||||||
require_exact_response(probe, binding)
|
|
||||||
if probe.get("status") != "APPLIED_VERIFIED":
|
|
||||||
raise ReconciliationRequired("tag was not independently verified")
|
|
||||||
if probe.get("peeled_target_oid") != binding.target_oid:
|
|
||||||
raise ManualOwnerReviewRequired("published tag peeled target mismatch")
|
|
||||||
tag_oid = probe.get("tag_oid")
|
|
||||||
if not isinstance(tag_oid, str) or not OID_RE.fullmatch(tag_oid):
|
|
||||||
raise ReleaseBlocked("verified annotated tag object OID missing")
|
|
||||||
return dict(probe)
|
|
||||||
|
|
||||||
|
|
||||||
def deploy_step(command: Sequence[str], binding: Binding) -> dict[str, Any]:
|
|
||||||
probe = run_adapter(command, binding.request("probe"))
|
|
||||||
require_exact_response(probe, binding)
|
|
||||||
if probe.get("status") == "ACTIVE_VERIFIED":
|
|
||||||
return dict(probe)
|
|
||||||
if probe.get("status") == "CONFLICT":
|
|
||||||
raise ManualOwnerReviewRequired("production runtime has conflicting binding")
|
|
||||||
applied = run_adapter(command, binding.request("apply"))
|
|
||||||
require_exact_response(applied, binding)
|
|
||||||
probe = run_adapter(command, binding.request("probe"))
|
|
||||||
require_exact_response(probe, binding)
|
|
||||||
if probe.get("status") != "ACTIVE_VERIFIED":
|
|
||||||
try:
|
|
||||||
run_adapter(command, binding.request("rollback"))
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
raise ReconciliationRequired("production runtime did not verify after apply")
|
|
||||||
if (
|
|
||||||
probe.get("deployment_state") != "ACTIVE"
|
|
||||||
or probe.get("production_deployed") is not True
|
|
||||||
):
|
|
||||||
raise ReconciliationRequired("production health contract is not ACTIVE")
|
|
||||||
return dict(probe)
|
|
||||||
|
|
||||||
|
|
||||||
def finalize_step(
|
|
||||||
command: Sequence[str],
|
|
||||||
binding: Binding,
|
|
||||||
*,
|
|
||||||
name: str,
|
|
||||||
tag_oid: str,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
request = binding.request("probe")
|
|
||||||
request["tag_oid"] = tag_oid
|
|
||||||
probe = run_adapter(command, request)
|
|
||||||
require_exact_response(probe, binding)
|
|
||||||
if probe.get("status") == "CONFLICT":
|
|
||||||
raise ManualOwnerReviewRequired(f"{name} has conflicting release binding")
|
|
||||||
if probe.get("status") != "VERIFIED":
|
|
||||||
apply_request = binding.request("apply")
|
|
||||||
apply_request["tag_oid"] = tag_oid
|
|
||||||
applied = run_adapter(command, apply_request)
|
|
||||||
require_exact_response(applied, binding)
|
|
||||||
probe = run_adapter(command, request)
|
|
||||||
require_exact_response(probe, binding)
|
|
||||||
if probe.get("status") != "VERIFIED":
|
|
||||||
raise ReconciliationRequired(f"{name} did not verify after apply")
|
|
||||||
if probe.get("tag_oid") not in (None, tag_oid):
|
|
||||||
raise ManualOwnerReviewRequired(f"{name} tag object mismatch")
|
|
||||||
return dict(probe)
|
|
||||||
|
|
||||||
|
|
||||||
def execute(
|
|
||||||
*,
|
|
||||||
binding: Binding,
|
|
||||||
authorization_path: Path,
|
|
||||||
repository: Path,
|
|
||||||
state_root: Path,
|
|
||||||
tag_command: Sequence[str],
|
|
||||||
deploy_command: Sequence[str],
|
|
||||||
registry_command: Sequence[str],
|
|
||||||
baseline_command: Sequence[str],
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
authorization = verify_authorization(secure_json(authorization_path), binding)
|
|
||||||
lineage = verify_git_lineage(repository, binding)
|
|
||||||
artifact = verify_artifact(binding)
|
|
||||||
state = DurableState(state_root, binding)
|
|
||||||
with FileLock(state_root / "executor.lock"):
|
|
||||||
state.record(
|
|
||||||
"PREFLIGHT_VERIFIED",
|
|
||||||
"preflight",
|
|
||||||
{"authorization": authorization, "lineage": lineage, "artifact": artifact},
|
|
||||||
)
|
|
||||||
tag: dict[str, Any] | None = None
|
|
||||||
runtime: dict[str, Any] | None = None
|
|
||||||
try:
|
|
||||||
tag = tag_step(tag_command, binding)
|
|
||||||
state.record("TAG_VERIFIED", "tag", tag)
|
|
||||||
|
|
||||||
runtime = deploy_step(deploy_command, binding)
|
|
||||||
state.record("RUNTIME_VERIFIED", "runtime", runtime)
|
|
||||||
|
|
||||||
registry = finalize_step(
|
|
||||||
registry_command,
|
|
||||||
binding,
|
|
||||||
name="registry",
|
|
||||||
tag_oid=tag["tag_oid"],
|
|
||||||
)
|
|
||||||
state.record("REGISTRY_VERIFIED", "registry", registry)
|
|
||||||
|
|
||||||
baseline = finalize_step(
|
|
||||||
baseline_command,
|
|
||||||
binding,
|
|
||||||
name="baseline",
|
|
||||||
tag_oid=tag["tag_oid"],
|
|
||||||
)
|
|
||||||
state.record("BASELINE_VERIFIED", "baseline", baseline)
|
|
||||||
|
|
||||||
# Final readback: no mutation, all four planes must still agree.
|
|
||||||
final_tag = tag_step(tag_command, binding)
|
|
||||||
final_runtime = run_adapter(deploy_command, binding.request("probe"))
|
|
||||||
final_registry_request = binding.request("probe")
|
|
||||||
final_registry_request["tag_oid"] = tag["tag_oid"]
|
|
||||||
final_registry = run_adapter(registry_command, final_registry_request)
|
|
||||||
final_baseline_request = binding.request("probe")
|
|
||||||
final_baseline_request["tag_oid"] = tag["tag_oid"]
|
|
||||||
final_baseline = run_adapter(baseline_command, final_baseline_request)
|
|
||||||
for value in (final_runtime, final_registry, final_baseline):
|
|
||||||
require_exact_response(value, binding)
|
|
||||||
if final_runtime.get("status") != "ACTIVE_VERIFIED":
|
|
||||||
raise ReconciliationRequired("final runtime readback failed")
|
|
||||||
if final_registry.get("status") != "VERIFIED":
|
|
||||||
raise ReconciliationRequired("final Registry readback failed")
|
|
||||||
if final_baseline.get("status") != "VERIFIED":
|
|
||||||
raise ReconciliationRequired("final Baseline readback failed")
|
|
||||||
final = {
|
|
||||||
"status": "PRODUCTION_ACCEPTED",
|
|
||||||
"version": binding.version,
|
|
||||||
"target_oid": binding.target_oid,
|
|
||||||
"evidence_closure_oid": binding.evidence_closure_oid,
|
|
||||||
"artifact_sha256": binding.artifact_sha256,
|
|
||||||
"approval_plan_id": binding.approval_plan_id,
|
|
||||||
"tag_oid": tag["tag_oid"],
|
|
||||||
"tag": final_tag,
|
|
||||||
"runtime": final_runtime,
|
|
||||||
"registry": final_registry,
|
|
||||||
"baseline": final_baseline,
|
|
||||||
"force_push_used": False,
|
|
||||||
"tag_rewritten": False,
|
|
||||||
"direct_main_push": False,
|
|
||||||
"secrets_committed": False,
|
|
||||||
}
|
|
||||||
state.record("COMPLETED", "final_consistency", final)
|
|
||||||
return final
|
|
||||||
except ManualOwnerReviewRequired as exc:
|
|
||||||
state.fail("MANUAL_OWNER_REVIEW_REQUIRED", str(exc))
|
|
||||||
raise
|
|
||||||
except ReconciliationRequired as exc:
|
|
||||||
state.fail("RECONCILIATION_REQUIRED", str(exc))
|
|
||||||
raise
|
|
||||||
except Exception as exc:
|
|
||||||
# Once a formal tag has been verified, no later error is a clean
|
|
||||||
# pre-side-effect failure. Preserve the tag and require idempotent
|
|
||||||
# reconciliation rather than pretending the release never started.
|
|
||||||
if tag is not None:
|
|
||||||
reason = f"{type(exc).__name__}: {exc}"
|
|
||||||
state.fail("RECONCILIATION_REQUIRED", reason)
|
|
||||||
raise ReconciliationRequired(reason) from exc
|
|
||||||
state.fail("FAIL_CLOSED", f"{type(exc).__name__}: {exc}")
|
|
||||||
raise
|
|
||||||
|
|
||||||
|
|
||||||
def parser() -> argparse.ArgumentParser:
|
|
||||||
p = argparse.ArgumentParser()
|
|
||||||
p.add_argument("--authorization", required=True, type=Path)
|
|
||||||
p.add_argument("--repository", required=True, type=Path)
|
|
||||||
p.add_argument("--state-root", required=True, type=Path)
|
|
||||||
p.add_argument("--artifact", required=True, type=Path)
|
|
||||||
p.add_argument("--artifact-size", required=True, type=int)
|
|
||||||
p.add_argument("--version", required=True)
|
|
||||||
p.add_argument("--target-oid", required=True)
|
|
||||||
p.add_argument("--evidence-closure-oid", required=True)
|
|
||||||
p.add_argument("--artifact-sha256", required=True)
|
|
||||||
p.add_argument("--approval-plan-id", required=True)
|
|
||||||
p.add_argument("--production-service", required=True)
|
|
||||||
p.add_argument("--production-port", required=True, type=int)
|
|
||||||
p.add_argument("--repository-identity", default="aios/aios")
|
|
||||||
p.add_argument("--tag-adapter", required=True, nargs="+")
|
|
||||||
p.add_argument("--deploy-adapter", required=True, nargs="+")
|
|
||||||
p.add_argument("--registry-adapter", required=True, nargs="+")
|
|
||||||
p.add_argument("--baseline-adapter", required=True, nargs="+")
|
|
||||||
return p
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
args = parser().parse_args()
|
|
||||||
binding = Binding(
|
|
||||||
version=args.version,
|
|
||||||
target_oid=args.target_oid,
|
|
||||||
evidence_closure_oid=args.evidence_closure_oid,
|
|
||||||
artifact_sha256=args.artifact_sha256,
|
|
||||||
approval_plan_id=args.approval_plan_id,
|
|
||||||
artifact_path=str(args.artifact),
|
|
||||||
artifact_size=args.artifact_size,
|
|
||||||
production_service=args.production_service,
|
|
||||||
production_port=args.production_port,
|
|
||||||
repository_identity=args.repository_identity,
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
result = execute(
|
|
||||||
binding=binding,
|
|
||||||
authorization_path=args.authorization,
|
|
||||||
repository=args.repository,
|
|
||||||
state_root=args.state_root,
|
|
||||||
tag_command=parse_command(args.tag_adapter),
|
|
||||||
deploy_command=parse_command(args.deploy_adapter),
|
|
||||||
registry_command=parse_command(args.registry_adapter),
|
|
||||||
baseline_command=parse_command(args.baseline_adapter),
|
|
||||||
)
|
|
||||||
print(json.dumps(result, sort_keys=True))
|
|
||||||
return 0
|
|
||||||
except ManualOwnerReviewRequired as exc:
|
|
||||||
print(json.dumps({"status": "MANUAL_OWNER_REVIEW_REQUIRED", "reason": str(exc)}, sort_keys=True))
|
|
||||||
return 20
|
|
||||||
except ReconciliationRequired as exc:
|
|
||||||
print(json.dumps({"status": "RECONCILIATION_REQUIRED", "reason": str(exc)}, sort_keys=True))
|
|
||||||
return 21
|
|
||||||
except Exception as exc:
|
|
||||||
print(json.dumps({"status": "FAIL_CLOSED", "error": type(exc).__name__, "reason": str(exc)}, sort_keys=True))
|
|
||||||
return 22
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,89 +0,0 @@
|
|||||||
"""Durable Owner/GPT notification outbox with retry and acknowledgement."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import sqlite3
|
|
||||||
import time
|
|
||||||
import uuid
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Callable, Mapping
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class OutboxEvent:
|
|
||||||
event_id: str
|
|
||||||
recipient: str
|
|
||||||
kind: str
|
|
||||||
payload: Mapping[str, Any]
|
|
||||||
attempts: int
|
|
||||||
available_at: float
|
|
||||||
acknowledged: bool
|
|
||||||
|
|
||||||
|
|
||||||
class DurableOutbox:
|
|
||||||
RECIPIENTS = ("owner", "gpt")
|
|
||||||
|
|
||||||
def __init__(self, path: str | Path) -> None:
|
|
||||||
self.path = str(path)
|
|
||||||
self.db = sqlite3.connect(self.path)
|
|
||||||
self.db.execute("PRAGMA journal_mode=WAL")
|
|
||||||
self.db.execute("CREATE TABLE IF NOT EXISTS outbox (event_id TEXT PRIMARY KEY, recipient TEXT NOT NULL, kind TEXT NOT NULL, payload TEXT NOT NULL, attempts INTEGER NOT NULL DEFAULT 0, available_at REAL NOT NULL, delivered_at REAL, acknowledged INTEGER NOT NULL DEFAULT 0, last_error TEXT)")
|
|
||||||
self.db.commit()
|
|
||||||
|
|
||||||
def enqueue(self, kind: str, payload: Mapping[str, Any], *, event_id: str | None = None, recipients: tuple[str, ...] = RECIPIENTS) -> list[str]:
|
|
||||||
ids: list[str] = []
|
|
||||||
for recipient in recipients:
|
|
||||||
if recipient not in self.RECIPIENTS:
|
|
||||||
raise ValueError("recipient is outside the governed notification plane")
|
|
||||||
eid = event_id or str(uuid.uuid4())
|
|
||||||
if event_id is not None and len(recipients) > 1:
|
|
||||||
eid = f"{event_id}:{recipient}"
|
|
||||||
self.db.execute("INSERT OR IGNORE INTO outbox(event_id,recipient,kind,payload,available_at) VALUES(?,?,?,?,?)", (eid, recipient, kind, json.dumps(dict(payload), sort_keys=True), time.time()))
|
|
||||||
ids.append(eid)
|
|
||||||
self.db.commit()
|
|
||||||
return ids
|
|
||||||
|
|
||||||
def due(self, *, now: float | None = None, limit: int = 20) -> list[OutboxEvent]:
|
|
||||||
now = time.time() if now is None else now
|
|
||||||
rows = self.db.execute("SELECT event_id,recipient,kind,payload,attempts,available_at,acknowledged FROM outbox WHERE delivered_at IS NULL AND available_at<=? ORDER BY available_at,event_id LIMIT ?", (now, limit)).fetchall()
|
|
||||||
return [OutboxEvent(r[0], r[1], r[2], json.loads(r[3]), r[4], r[5], bool(r[6])) for r in rows]
|
|
||||||
|
|
||||||
def delivered(self, event_id: str, *, now: float | None = None) -> None:
|
|
||||||
self.db.execute("UPDATE outbox SET delivered_at=? WHERE event_id=?", (time.time() if now is None else now, event_id))
|
|
||||||
self.db.commit()
|
|
||||||
|
|
||||||
def failed(self, event_id: str, error: str, *, now: float | None = None, max_backoff: float = 300.0) -> None:
|
|
||||||
now = time.time() if now is None else now
|
|
||||||
row = self.db.execute("SELECT attempts FROM outbox WHERE event_id=?", (event_id,)).fetchone()
|
|
||||||
attempts = int(row[0]) + 1 if row else 1
|
|
||||||
delay = min(max_backoff, 2 ** min(attempts, 8))
|
|
||||||
self.db.execute("UPDATE outbox SET attempts=?,available_at=?,last_error=? WHERE event_id=?", (attempts, now + delay, error[:200], event_id))
|
|
||||||
self.db.commit()
|
|
||||||
|
|
||||||
def acknowledge(self, event_id: str) -> None:
|
|
||||||
self.db.execute("UPDATE outbox SET acknowledged=1 WHERE event_id=?", (event_id,))
|
|
||||||
self.db.commit()
|
|
||||||
|
|
||||||
def close(self) -> None:
|
|
||||||
self.db.close()
|
|
||||||
|
|
||||||
|
|
||||||
class NotificationPlane:
|
|
||||||
def __init__(self, outbox: DurableOutbox) -> None:
|
|
||||||
self.outbox = outbox
|
|
||||||
|
|
||||||
def emit(self, kind: str, payload: Mapping[str, Any], *, event_id: str | None = None) -> list[str]:
|
|
||||||
return self.outbox.enqueue(kind, payload, event_id=event_id)
|
|
||||||
|
|
||||||
def deliver_once(self, send: Callable[[OutboxEvent], None], *, now: float | None = None) -> list[str]:
|
|
||||||
delivered: list[str] = []
|
|
||||||
for event in self.outbox.due(now=now):
|
|
||||||
try:
|
|
||||||
send(event)
|
|
||||||
except Exception as exc:
|
|
||||||
self.outbox.failed(event.event_id, type(exc).__name__, now=now)
|
|
||||||
else:
|
|
||||||
self.outbox.delivered(event.event_id, now=now)
|
|
||||||
delivered.append(event.event_id)
|
|
||||||
return delivered
|
|
||||||
@@ -1,84 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Single canonical Production Baseline authority for the GitOps reader.
|
|
||||||
|
|
||||||
The API and the one-shot writer import this module. Workspace and release
|
|
||||||
copies are projections only; they are never selected as a fallback source.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
|
|
||||||
AUTHORITY_PATH = Path(os.getenv(
|
|
||||||
"AIOS_GITOPS_BASELINE_AUTHORITY",
|
|
||||||
"/var/lib/aios-gitops-runtime-anchor/baseline/current.json",
|
|
||||||
))
|
|
||||||
AUTHORITY_SHA_PATH = AUTHORITY_PATH.with_suffix(".sha256")
|
|
||||||
|
|
||||||
|
|
||||||
class BaselineAuthorityError(RuntimeError):
|
|
||||||
"""Fail-closed error raised for an unavailable or inconsistent authority."""
|
|
||||||
|
|
||||||
|
|
||||||
class BaselineAuthority:
|
|
||||||
def __init__(self, path: Path | str = AUTHORITY_PATH):
|
|
||||||
self.path = Path(path)
|
|
||||||
self.sha_path = self.path.with_suffix(".sha256")
|
|
||||||
|
|
||||||
def _raw(self) -> bytes:
|
|
||||||
try:
|
|
||||||
raw = self.path.read_bytes()
|
|
||||||
except OSError as exc:
|
|
||||||
raise BaselineAuthorityError(f"authority_unavailable:{exc}") from exc
|
|
||||||
if not raw:
|
|
||||||
raise BaselineAuthorityError("authority_empty")
|
|
||||||
return raw
|
|
||||||
|
|
||||||
def read(self) -> dict[str, Any]:
|
|
||||||
raw = self._raw()
|
|
||||||
digest = hashlib.sha256(raw).hexdigest()
|
|
||||||
try:
|
|
||||||
value = json.loads(raw.decode("utf-8"))
|
|
||||||
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
|
|
||||||
raise BaselineAuthorityError("authority_invalid_json") from exc
|
|
||||||
if not isinstance(value, dict):
|
|
||||||
raise BaselineAuthorityError("authority_not_object")
|
|
||||||
expected = str(value.get("authority_sha256", ""))
|
|
||||||
if expected and expected != digest:
|
|
||||||
raise BaselineAuthorityError("authority_embedded_digest_mismatch")
|
|
||||||
if self.sha_path.exists():
|
|
||||||
sidecar = self.sha_path.read_text(encoding="utf-8").strip().split()[0]
|
|
||||||
if sidecar != digest:
|
|
||||||
raise BaselineAuthorityError("authority_sidecar_digest_mismatch")
|
|
||||||
required = {
|
|
||||||
"current_baseline", "previous_baseline", "target_oid", "tag_oid",
|
|
||||||
"evidence_closure_oid", "approval_plan_id", "status", "generation",
|
|
||||||
"source_identity",
|
|
||||||
}
|
|
||||||
missing = sorted(required.difference(value))
|
|
||||||
if missing:
|
|
||||||
raise BaselineAuthorityError("authority_missing:" + ",".join(missing))
|
|
||||||
value = dict(value)
|
|
||||||
value["authority_sha256"] = digest
|
|
||||||
value["authority_path"] = str(self.path)
|
|
||||||
return value
|
|
||||||
|
|
||||||
def read_consistent(self, policy: dict[str, Any] | None = None) -> dict[str, Any]:
|
|
||||||
value = self.read()
|
|
||||||
if policy:
|
|
||||||
expected = policy.get("release_binding", policy)
|
|
||||||
for key, field in (("version", "current_baseline"),
|
|
||||||
("previous_version", "previous_baseline"),
|
|
||||||
("target_oid", "target_oid"),
|
|
||||||
("tag_oid", "tag_oid"),
|
|
||||||
("evidence_closure_oid", "evidence_closure_oid"),
|
|
||||||
("approval_plan_id", "approval_plan_id")):
|
|
||||||
wanted = expected.get(key)
|
|
||||||
if wanted is not None and value.get(field) != wanted:
|
|
||||||
raise BaselineAuthorityError(f"authority_binding_mismatch:{field}")
|
|
||||||
return value
|
|
||||||
|
|
||||||
@@ -1,491 +0,0 @@
|
|||||||
"""Fail-closed, branch-bound repair leases for governed AIOS GitOps work."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import posixpath
|
|
||||||
import re
|
|
||||||
import secrets
|
|
||||||
import subprocess
|
|
||||||
import tempfile
|
|
||||||
import uuid
|
|
||||||
from dataclasses import asdict, dataclass, field
|
|
||||||
from datetime import datetime, timedelta, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Iterable, Mapping, Protocol, Sequence
|
|
||||||
|
|
||||||
|
|
||||||
TASK_ID = "V2R1C6_GOVERNED_LOCAL_BOOTSTRAP"
|
|
||||||
MAX_LEASE_SECONDS = 86_400
|
|
||||||
OID_RE = re.compile(r"^[0-9a-fA-F]{40}$")
|
|
||||||
DENIED_BRANCHES = {"main", "master", "HEAD"}
|
|
||||||
PROTECTED_PREFIXES = (
|
|
||||||
".git",
|
|
||||||
"VERSION",
|
|
||||||
"versions/V2R1C5.json",
|
|
||||||
"artifacts/V2R1C5",
|
|
||||||
"baseline",
|
|
||||||
"config/gitops.tokens",
|
|
||||||
"gitops.tokens",
|
|
||||||
"secrets",
|
|
||||||
"credentials",
|
|
||||||
)
|
|
||||||
SECRET_PATTERNS = (
|
|
||||||
re.compile(r"(?i)-----BEGIN [A-Z ]*PRIVATE KEY-----"),
|
|
||||||
re.compile(r"(?i)(?:api[_-]?key|secret|password|token)\s*[:=]\s*\S+"),
|
|
||||||
re.compile(r"\b(?:sk|HPUA|SDC)-[A-Za-z0-9_-]{16,}\b"),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class LeaseError(Exception):
|
|
||||||
"""Raised for any invalid, stale, expired, or prohibited lease action."""
|
|
||||||
|
|
||||||
|
|
||||||
class GitProbeProtocol(Protocol):
|
|
||||||
"""Read-only Git seam used by the lease manager."""
|
|
||||||
|
|
||||||
def branch_head(self, branch: str) -> str: ...
|
|
||||||
|
|
||||||
def parent(self, commit: str) -> str: ...
|
|
||||||
|
|
||||||
def is_ancestor(self, older: str, newer: str) -> bool: ...
|
|
||||||
|
|
||||||
def changed_paths(self, commit: str) -> tuple[str, ...]: ...
|
|
||||||
|
|
||||||
def commits_between(self, older: str, newer: str) -> tuple[str, ...]: ...
|
|
||||||
|
|
||||||
|
|
||||||
def _now() -> datetime:
|
|
||||||
return datetime.now(timezone.utc)
|
|
||||||
|
|
||||||
|
|
||||||
def _ts(value: datetime) -> str:
|
|
||||||
return value.astimezone(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_ts(value: str) -> datetime:
|
|
||||||
if value.endswith("Z"):
|
|
||||||
value = value[:-1] + "+00:00"
|
|
||||||
parsed = datetime.fromisoformat(value)
|
|
||||||
return parsed if parsed.tzinfo else parsed.replace(tzinfo=timezone.utc)
|
|
||||||
|
|
||||||
|
|
||||||
def _oid(value: str, field: str = "oid") -> str:
|
|
||||||
if not isinstance(value, str) or not OID_RE.fullmatch(value):
|
|
||||||
raise LeaseError(f"{field} must be a full commit OID")
|
|
||||||
return value.lower()
|
|
||||||
|
|
||||||
|
|
||||||
def normalize_path(path: str) -> str:
|
|
||||||
"""Normalize a repository-relative path without resolving through the FS."""
|
|
||||||
if not isinstance(path, str) or not path.strip():
|
|
||||||
raise LeaseError("path must be a non-empty string")
|
|
||||||
value = path.replace("\\", "/").strip()
|
|
||||||
if value.startswith("/") or "\x00" in value:
|
|
||||||
raise LeaseError("path must be relative")
|
|
||||||
parts = value.split("/")
|
|
||||||
if any(part in {"", ".", ".."} for part in parts):
|
|
||||||
raise LeaseError("path contains a prohibited component")
|
|
||||||
normalized = posixpath.normpath(value)
|
|
||||||
if normalized == "." or normalized.startswith("../"):
|
|
||||||
raise LeaseError("path is not normalized")
|
|
||||||
return normalized
|
|
||||||
|
|
||||||
|
|
||||||
def path_is_protected(path: str) -> bool:
|
|
||||||
normalized = normalize_path(path)
|
|
||||||
lower = normalized.lower()
|
|
||||||
return any(
|
|
||||||
lower == prefix.lower() or lower.startswith(prefix.lower().rstrip("/") + "/")
|
|
||||||
for prefix in PROTECTED_PREFIXES
|
|
||||||
) or lower.startswith("refs/")
|
|
||||||
|
|
||||||
|
|
||||||
def contains_secret_like_content(content: str) -> bool:
|
|
||||||
return any(pattern.search(content or "") for pattern in SECRET_PATTERNS)
|
|
||||||
|
|
||||||
|
|
||||||
def validate_paths(paths: Iterable[str], allowed_paths: Sequence[str]) -> tuple[str, ...]:
|
|
||||||
allowed = tuple(normalize_path(path) for path in allowed_paths)
|
|
||||||
if not allowed:
|
|
||||||
raise LeaseError("allowed_paths must not be empty")
|
|
||||||
result: list[str] = []
|
|
||||||
for raw in paths:
|
|
||||||
path = normalize_path(raw)
|
|
||||||
if path_is_protected(path):
|
|
||||||
raise LeaseError("protected path is not lease-writable")
|
|
||||||
if not any(path == prefix or path.startswith(prefix + "/") for prefix in allowed):
|
|
||||||
raise LeaseError("path is outside the lease allowlist")
|
|
||||||
if path not in result:
|
|
||||||
result.append(path)
|
|
||||||
if not result:
|
|
||||||
raise LeaseError("at least one changed path is required")
|
|
||||||
return tuple(result)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class CommitEntry:
|
|
||||||
commit_oid: str
|
|
||||||
parent_oid: str
|
|
||||||
paths: tuple[str, ...]
|
|
||||||
recorded_at: str
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class RepairLease:
|
|
||||||
lease_id: str
|
|
||||||
task_id: str
|
|
||||||
owner_identity: str
|
|
||||||
branch: str
|
|
||||||
description: str
|
|
||||||
allowed_paths: tuple[str, ...]
|
|
||||||
initial_head_oid: str
|
|
||||||
current_head_oid: str
|
|
||||||
created_at: str
|
|
||||||
expires_at: str
|
|
||||||
max_expires_at: str
|
|
||||||
state: str = "ACTIVE"
|
|
||||||
revision: int = 0
|
|
||||||
commits: list[CommitEntry] = field(default_factory=list)
|
|
||||||
attached: bool = False
|
|
||||||
|
|
||||||
def to_json(self) -> dict[str, Any]:
|
|
||||||
payload = asdict(self)
|
|
||||||
payload["allowed_paths"] = list(self.allowed_paths)
|
|
||||||
payload["commits"] = [asdict(entry) for entry in self.commits]
|
|
||||||
return payload
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_json(cls, raw: dict[str, Any]) -> "RepairLease":
|
|
||||||
commits = tuple(
|
|
||||||
CommitEntry(
|
|
||||||
commit_oid=_oid(item["commit_oid"], "commit_oid"),
|
|
||||||
parent_oid=_oid(item["parent_oid"], "parent_oid"),
|
|
||||||
paths=tuple(normalize_path(path) for path in item["paths"]),
|
|
||||||
recorded_at=str(item["recorded_at"]),
|
|
||||||
)
|
|
||||||
for item in raw.get("commits", [])
|
|
||||||
)
|
|
||||||
return cls(
|
|
||||||
lease_id=str(raw["lease_id"]),
|
|
||||||
task_id=str(raw["task_id"]),
|
|
||||||
owner_identity=str(raw["owner_identity"]),
|
|
||||||
branch=str(raw["branch"]),
|
|
||||||
description=str(raw["description"]),
|
|
||||||
allowed_paths=tuple(normalize_path(path) for path in raw["allowed_paths"]),
|
|
||||||
initial_head_oid=_oid(raw["initial_head_oid"], "initial_head_oid"),
|
|
||||||
current_head_oid=_oid(raw["current_head_oid"], "current_head_oid"),
|
|
||||||
created_at=str(raw["created_at"]),
|
|
||||||
expires_at=str(raw["expires_at"]),
|
|
||||||
max_expires_at=str(raw["max_expires_at"]),
|
|
||||||
state=str(raw.get("state", "ACTIVE")),
|
|
||||||
revision=int(raw.get("revision", 0)),
|
|
||||||
commits=list(commits),
|
|
||||||
attached=bool(raw.get("attached", False)),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class LeaseStore:
|
|
||||||
"""Atomic JSON store for restart-recoverable lease state."""
|
|
||||||
|
|
||||||
def __init__(self, directory: str | os.PathLike[str]) -> None:
|
|
||||||
self.directory = Path(directory)
|
|
||||||
self.directory.mkdir(parents=True, exist_ok=True)
|
|
||||||
if self.directory.is_symlink():
|
|
||||||
raise LeaseError("lease store must not be a symlink")
|
|
||||||
os.chmod(self.directory, 0o700)
|
|
||||||
|
|
||||||
def _path(self, lease_id: str) -> Path:
|
|
||||||
if not re.fullmatch(r"lease-[0-9a-f]{32}", lease_id):
|
|
||||||
raise LeaseError("invalid lease id")
|
|
||||||
return self.directory / f"{lease_id}.json"
|
|
||||||
|
|
||||||
def create(self, lease: RepairLease) -> None:
|
|
||||||
path = self._path(lease.lease_id)
|
|
||||||
if path.exists() or path.is_symlink():
|
|
||||||
raise LeaseError("lease already exists")
|
|
||||||
self._write(path, lease.to_json(), exclusive=True)
|
|
||||||
|
|
||||||
def load(self, lease_id: str) -> RepairLease:
|
|
||||||
path = self._path(lease_id)
|
|
||||||
try:
|
|
||||||
raw = json.loads(path.read_text(encoding="utf-8"))
|
|
||||||
except (OSError, json.JSONDecodeError, TypeError) as exc:
|
|
||||||
raise LeaseError("lease state is unavailable or invalid") from exc
|
|
||||||
if not isinstance(raw, dict):
|
|
||||||
raise LeaseError("lease state must be an object")
|
|
||||||
return RepairLease.from_json(raw)
|
|
||||||
|
|
||||||
def save(self, lease: RepairLease) -> None:
|
|
||||||
self._write(self._path(lease.lease_id), lease.to_json())
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _write(path: Path, payload: dict[str, Any], *, exclusive: bool = False) -> None:
|
|
||||||
flags = os.O_WRONLY | os.O_CREAT | (os.O_EXCL if exclusive else 0)
|
|
||||||
fd, temp_name = tempfile.mkstemp(prefix=".lease-", dir=path.parent)
|
|
||||||
try:
|
|
||||||
os.fchmod(fd, 0o600)
|
|
||||||
with os.fdopen(fd, "w", encoding="utf-8") as stream:
|
|
||||||
json.dump(payload, stream, ensure_ascii=True, sort_keys=True, separators=(",", ":"))
|
|
||||||
stream.write("\n")
|
|
||||||
if exclusive:
|
|
||||||
os.link(temp_name, path)
|
|
||||||
os.unlink(temp_name)
|
|
||||||
else:
|
|
||||||
os.replace(temp_name, path)
|
|
||||||
os.chmod(path, 0o600)
|
|
||||||
except Exception:
|
|
||||||
try:
|
|
||||||
os.unlink(temp_name)
|
|
||||||
except FileNotFoundError:
|
|
||||||
pass
|
|
||||||
raise
|
|
||||||
|
|
||||||
|
|
||||||
class GitProbe:
|
|
||||||
"""Read-only Git implementation; it never changes a ref or working tree."""
|
|
||||||
|
|
||||||
def __init__(self, bare_repo: str | os.PathLike[str]) -> None:
|
|
||||||
self.bare_repo = Path(bare_repo)
|
|
||||||
|
|
||||||
def _run(self, args: list[str]) -> str:
|
|
||||||
result = subprocess.run(
|
|
||||||
["git", "--git-dir", str(self.bare_repo), *args],
|
|
||||||
text=True,
|
|
||||||
capture_output=True,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
if result.returncode != 0:
|
|
||||||
raise LeaseError("read-only Git probe failed")
|
|
||||||
return result.stdout.strip()
|
|
||||||
|
|
||||||
def branch_head(self, branch: str) -> str:
|
|
||||||
return _oid(self._run(["rev-parse", f"refs/heads/{branch}"]), "branch_head")
|
|
||||||
|
|
||||||
def parent(self, commit: str) -> str:
|
|
||||||
return _oid(self._run(["rev-parse", f"{_oid(commit)}^1"]), "parent")
|
|
||||||
|
|
||||||
def is_ancestor(self, older: str, newer: str) -> bool:
|
|
||||||
old, new = _oid(older), _oid(newer)
|
|
||||||
result = subprocess.run(
|
|
||||||
["git", "--git-dir", str(self.bare_repo), "merge-base", "--is-ancestor", old, new],
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
return result.returncode == 0
|
|
||||||
|
|
||||||
def changed_paths(self, commit: str) -> tuple[str, ...]:
|
|
||||||
raw = self._run(["diff-tree", "--no-commit-id", "--name-only", "-r", _oid(commit)])
|
|
||||||
return tuple(normalize_path(line) for line in raw.splitlines() if line.strip())
|
|
||||||
|
|
||||||
def commits_between(self, older: str, newer: str) -> tuple[str, ...]:
|
|
||||||
raw = self._run(["rev-list", "--reverse", f"{_oid(older)}..{_oid(newer)}"])
|
|
||||||
return tuple(_oid(line, "commit") for line in raw.splitlines() if line.strip())
|
|
||||||
|
|
||||||
|
|
||||||
class RepairLeaseManager:
|
|
||||||
"""Issue, recover, renew, and close branch-bound repair leases."""
|
|
||||||
|
|
||||||
def __init__(self, store: LeaseStore, git: GitProbeProtocol, *, clock=_now) -> None:
|
|
||||||
self.store = store
|
|
||||||
self.git = git
|
|
||||||
self.clock = clock
|
|
||||||
|
|
||||||
def _require_owner(self, lease: RepairLease, owner_identity: str) -> None:
|
|
||||||
if owner_identity != lease.owner_identity:
|
|
||||||
raise LeaseError("lease identity mismatch")
|
|
||||||
|
|
||||||
def _active(self, lease: RepairLease) -> RepairLease:
|
|
||||||
now = self.clock()
|
|
||||||
if lease.state != "ACTIVE":
|
|
||||||
raise LeaseError(f"lease is {lease.state.lower()}")
|
|
||||||
if now >= _parse_ts(lease.expires_at):
|
|
||||||
lease.state = "EXPIRED"
|
|
||||||
self.store.save(lease)
|
|
||||||
raise LeaseError("lease expired")
|
|
||||||
return lease
|
|
||||||
|
|
||||||
def issue(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
task_id: str,
|
|
||||||
owner_identity: str,
|
|
||||||
branch: str,
|
|
||||||
description: str,
|
|
||||||
allowed_paths: Sequence[str],
|
|
||||||
ttl_seconds: int = 3600,
|
|
||||||
) -> RepairLease:
|
|
||||||
if task_id != TASK_ID:
|
|
||||||
raise LeaseError("unexpected task id")
|
|
||||||
if not owner_identity or not description or len(description) > 240:
|
|
||||||
raise LeaseError("owner identity and description are required")
|
|
||||||
if "\n" in description or "\r" in description:
|
|
||||||
raise LeaseError("description must be a single line")
|
|
||||||
if branch in DENIED_BRANCHES or branch.startswith(("main/", "refs/", "tags/")):
|
|
||||||
raise LeaseError("protected branch cannot receive a repair lease")
|
|
||||||
if not isinstance(ttl_seconds, int) or not 0 < ttl_seconds <= MAX_LEASE_SECONDS:
|
|
||||||
raise LeaseError("lease TTL exceeds the maximum")
|
|
||||||
allowed = tuple(normalize_path(path) for path in allowed_paths)
|
|
||||||
if any(path_is_protected(path) for path in allowed):
|
|
||||||
raise LeaseError("protected path in lease allowlist")
|
|
||||||
head = self.git.branch_head(branch)
|
|
||||||
created = self.clock()
|
|
||||||
expires = created + timedelta(seconds=ttl_seconds)
|
|
||||||
max_expires = created + timedelta(seconds=MAX_LEASE_SECONDS)
|
|
||||||
lease_id = "lease-" + secrets.token_hex(16)
|
|
||||||
lease = RepairLease(
|
|
||||||
lease_id=lease_id,
|
|
||||||
task_id=task_id,
|
|
||||||
owner_identity=owner_identity,
|
|
||||||
branch=branch,
|
|
||||||
description=description,
|
|
||||||
allowed_paths=allowed,
|
|
||||||
initial_head_oid=head,
|
|
||||||
current_head_oid=head,
|
|
||||||
created_at=_ts(created),
|
|
||||||
expires_at=_ts(expires),
|
|
||||||
max_expires_at=_ts(max_expires),
|
|
||||||
)
|
|
||||||
self.store.create(lease)
|
|
||||||
return lease
|
|
||||||
|
|
||||||
def attach_existing_branch(
|
|
||||||
self, lease_id: str, *, owner_identity: str, expected_head_oid: str
|
|
||||||
) -> RepairLease:
|
|
||||||
lease = self._active(self.store.load(lease_id))
|
|
||||||
self._require_owner(lease, owner_identity)
|
|
||||||
current = self.git.branch_head(lease.branch)
|
|
||||||
expected = _oid(expected_head_oid, "expected_head_oid")
|
|
||||||
if current != expected or current != lease.current_head_oid:
|
|
||||||
raise LeaseError("branch head does not match the lease")
|
|
||||||
lease.attached = True
|
|
||||||
lease.revision += 1
|
|
||||||
self.store.save(lease)
|
|
||||||
return lease
|
|
||||||
|
|
||||||
def authorize_commit(
|
|
||||||
self,
|
|
||||||
lease_id: str,
|
|
||||||
*,
|
|
||||||
owner_identity: str,
|
|
||||||
expected_head_oid: str,
|
|
||||||
paths: Sequence[str],
|
|
||||||
content: Mapping[str, str] | None = None,
|
|
||||||
) -> RepairLease:
|
|
||||||
lease = self._active(self.store.load(lease_id))
|
|
||||||
self._require_owner(lease, owner_identity)
|
|
||||||
expected = _oid(expected_head_oid, "expected_head_oid")
|
|
||||||
current = self.git.branch_head(lease.branch)
|
|
||||||
if current != expected or current != lease.current_head_oid:
|
|
||||||
raise LeaseError("stale or externally advanced branch head")
|
|
||||||
if not lease.attached:
|
|
||||||
raise LeaseError("lease is not attached")
|
|
||||||
validate_paths(paths, lease.allowed_paths)
|
|
||||||
if content and any(contains_secret_like_content(value) for value in content.values()):
|
|
||||||
raise LeaseError("secret-like content rejected")
|
|
||||||
return lease
|
|
||||||
|
|
||||||
def record_commit(
|
|
||||||
self,
|
|
||||||
lease_id: str,
|
|
||||||
*,
|
|
||||||
owner_identity: str,
|
|
||||||
expected_head_oid: str,
|
|
||||||
new_head_oid: str,
|
|
||||||
paths: Sequence[str],
|
|
||||||
) -> RepairLease:
|
|
||||||
lease = self._active(self.store.load(lease_id))
|
|
||||||
self._require_owner(lease, owner_identity)
|
|
||||||
new_head = _oid(new_head_oid, "new_head_oid")
|
|
||||||
expected = _oid(expected_head_oid, "expected_head_oid")
|
|
||||||
if not lease.attached or lease.current_head_oid != expected:
|
|
||||||
raise LeaseError("stale lease head")
|
|
||||||
if self.git.branch_head(lease.branch) != new_head:
|
|
||||||
raise LeaseError("branch head does not match committed head")
|
|
||||||
validate_paths(paths, lease.allowed_paths)
|
|
||||||
if self.git.parent(new_head) != expected or not self.git.is_ancestor(expected, new_head):
|
|
||||||
raise LeaseError("commit is not a fast-forward child of the expected head")
|
|
||||||
actual_paths = self.git.changed_paths(new_head)
|
|
||||||
allowed = validate_paths(actual_paths, lease.allowed_paths)
|
|
||||||
requested = set(validate_paths(paths, lease.allowed_paths))
|
|
||||||
if not set(allowed).issubset(requested):
|
|
||||||
raise LeaseError("commit changed an unrequested path")
|
|
||||||
lease.current_head_oid = new_head
|
|
||||||
lease.revision += 1
|
|
||||||
lease.commits.append(
|
|
||||||
CommitEntry(
|
|
||||||
commit_oid=new_head,
|
|
||||||
parent_oid=expected,
|
|
||||||
paths=allowed,
|
|
||||||
recorded_at=_ts(self.clock()),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
self.store.save(lease)
|
|
||||||
return lease
|
|
||||||
|
|
||||||
def recover(self, lease_id: str, *, owner_identity: str) -> RepairLease:
|
|
||||||
lease = self._active(self.store.load(lease_id))
|
|
||||||
self._require_owner(lease, owner_identity)
|
|
||||||
current = self.git.branch_head(lease.branch)
|
|
||||||
if current == lease.current_head_oid:
|
|
||||||
return lease
|
|
||||||
if not self.git.is_ancestor(lease.current_head_oid, current):
|
|
||||||
raise LeaseError("branch moved non-fast-forward")
|
|
||||||
commits = self.git.commits_between(lease.current_head_oid, current)
|
|
||||||
for commit in commits:
|
|
||||||
actual = self.git.changed_paths(commit)
|
|
||||||
validate_paths(actual, lease.allowed_paths)
|
|
||||||
lease.current_head_oid = current
|
|
||||||
lease.revision += 1
|
|
||||||
self.store.save(lease)
|
|
||||||
return lease
|
|
||||||
|
|
||||||
def renew(self, lease_id: str, *, owner_identity: str, ttl_seconds: int) -> RepairLease:
|
|
||||||
lease = self._active(self.store.load(lease_id))
|
|
||||||
self._require_owner(lease, owner_identity)
|
|
||||||
if not isinstance(ttl_seconds, int) or ttl_seconds <= 0:
|
|
||||||
raise LeaseError("renewal TTL must be positive")
|
|
||||||
candidate = self.clock() + timedelta(seconds=ttl_seconds)
|
|
||||||
maximum = _parse_ts(lease.max_expires_at)
|
|
||||||
if candidate > maximum:
|
|
||||||
raise LeaseError("renewal exceeds maximum lease lifetime")
|
|
||||||
lease.expires_at = _ts(candidate)
|
|
||||||
lease.revision += 1
|
|
||||||
self.store.save(lease)
|
|
||||||
return lease
|
|
||||||
|
|
||||||
def complete(self, lease_id: str, *, owner_identity: str) -> RepairLease:
|
|
||||||
lease = self._active(self.store.load(lease_id))
|
|
||||||
self._require_owner(lease, owner_identity)
|
|
||||||
lease.state = "COMPLETED"
|
|
||||||
lease.revision += 1
|
|
||||||
self.store.save(lease)
|
|
||||||
return lease
|
|
||||||
|
|
||||||
def revoke(self, lease_id: str, *, owner_identity: str) -> RepairLease:
|
|
||||||
lease = self._active(self.store.load(lease_id))
|
|
||||||
self._require_owner(lease, owner_identity)
|
|
||||||
lease.state = "REVOKED"
|
|
||||||
lease.revision += 1
|
|
||||||
self.store.save(lease)
|
|
||||||
return lease
|
|
||||||
|
|
||||||
|
|
||||||
def sanitized_lease_view(lease: RepairLease) -> dict[str, Any]:
|
|
||||||
"""Return lease metadata only; no content, token, or credential values."""
|
|
||||||
return {
|
|
||||||
"lease_id": lease.lease_id,
|
|
||||||
"task_id": lease.task_id,
|
|
||||||
"owner_identity": lease.owner_identity,
|
|
||||||
"branch": lease.branch,
|
|
||||||
"description": lease.description,
|
|
||||||
"initial_head_oid": lease.initial_head_oid,
|
|
||||||
"current_head_oid": lease.current_head_oid,
|
|
||||||
"created_at": lease.created_at,
|
|
||||||
"expires_at": lease.expires_at,
|
|
||||||
"state": lease.state,
|
|
||||||
"revision": lease.revision,
|
|
||||||
"commit_count": len(lease.commits),
|
|
||||||
"attached": lease.attached,
|
|
||||||
}
|
|
||||||
@@ -1,108 +0,0 @@
|
|||||||
"""Atomic, predicate-bound activation of the durable release campaign."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import re
|
|
||||||
import tempfile
|
|
||||||
from typing import Any, Mapping
|
|
||||||
|
|
||||||
from release_campaign_authorization import CampaignError, validate_policy
|
|
||||||
|
|
||||||
OID_RE = re.compile(r"^[0-9a-f]{40}$")
|
|
||||||
CAMPAIGN_ID = "AIOS-AUTONOMOUS-RELEASE-CAMPAIGN-20260807"
|
|
||||||
|
|
||||||
|
|
||||||
class CampaignActivationError(CampaignError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def canonical_json(value: Any) -> bytes:
|
|
||||||
return (json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + "\n").encode("utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
def digest_bytes(value: bytes) -> str:
|
|
||||||
return hashlib.sha256(value).hexdigest()
|
|
||||||
|
|
||||||
|
|
||||||
def build_active_policy(current: Mapping[str, Any], facts: Mapping[str, Any]) -> dict[str, Any]:
|
|
||||||
policy = validate_policy(dict(current))
|
|
||||||
required = {
|
|
||||||
"campaign_id",
|
|
||||||
"repository",
|
|
||||||
"source_main_oid",
|
|
||||||
"gitea_main_oid",
|
|
||||||
"authority_main_oid",
|
|
||||||
"runtime_source_oid",
|
|
||||||
"durable_job_state",
|
|
||||||
"writer_ready",
|
|
||||||
"gitea_machine_authenticated",
|
|
||||||
"governance_executor_ready",
|
|
||||||
"preapproval_publisher_ready",
|
|
||||||
"runtime_exact_main",
|
|
||||||
"main_lineage_verified",
|
|
||||||
"campaign_revoked",
|
|
||||||
}
|
|
||||||
if set(facts) != required:
|
|
||||||
raise CampaignActivationError("ACTIVATION_FACTS_SCHEMA_INVALID")
|
|
||||||
if policy["activation_state"] != "PENDING_CONTROL_PLANE_MERGE" or policy["active"] is not False:
|
|
||||||
raise CampaignActivationError("CAMPAIGN_NOT_PENDING")
|
|
||||||
if policy["campaign_id"] != CAMPAIGN_ID or facts["campaign_id"] != CAMPAIGN_ID:
|
|
||||||
raise CampaignActivationError("CAMPAIGN_ID_MISMATCH")
|
|
||||||
if facts["repository"] != "aios/aios" or policy["repository"] != facts["repository"]:
|
|
||||||
raise CampaignActivationError("REPOSITORY_MISMATCH")
|
|
||||||
for name in ("source_main_oid", "gitea_main_oid", "authority_main_oid", "runtime_source_oid"):
|
|
||||||
if not isinstance(facts[name], str) or not OID_RE.fullmatch(facts[name]):
|
|
||||||
raise CampaignActivationError("ACTIVATION_OID_INVALID")
|
|
||||||
if not (
|
|
||||||
facts["source_main_oid"]
|
|
||||||
== facts["gitea_main_oid"]
|
|
||||||
== facts["authority_main_oid"]
|
|
||||||
== facts["runtime_source_oid"]
|
|
||||||
):
|
|
||||||
raise CampaignActivationError("ACTIVATION_MAIN_DIVERGENCE")
|
|
||||||
if facts["durable_job_state"] != "COMPLETED":
|
|
||||||
raise CampaignActivationError("DURABLE_JOB_NOT_COMPLETED")
|
|
||||||
boolean_fields = (
|
|
||||||
"writer_ready",
|
|
||||||
"gitea_machine_authenticated",
|
|
||||||
"governance_executor_ready",
|
|
||||||
"preapproval_publisher_ready",
|
|
||||||
"runtime_exact_main",
|
|
||||||
"main_lineage_verified",
|
|
||||||
)
|
|
||||||
if any(facts[name] is not True for name in boolean_fields) or facts["campaign_revoked"] is not False:
|
|
||||||
raise CampaignActivationError("ACTIVATION_PREDICATE_FAILED")
|
|
||||||
activated = dict(policy)
|
|
||||||
activated["activation_state"] = "ACTIVE"
|
|
||||||
activated["active"] = True
|
|
||||||
return validate_policy(activated)
|
|
||||||
|
|
||||||
|
|
||||||
def atomic_write_policy(path: Path, sidecar: Path, receipt_path: Path, policy: Mapping[str, Any], receipt: Mapping[str, Any]) -> str:
|
|
||||||
if path.is_symlink() or sidecar.is_symlink() or receipt_path.exists() or receipt_path.is_symlink():
|
|
||||||
raise CampaignActivationError("ACTIVATION_PATH_INVALID")
|
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
raw = canonical_json(dict(policy))
|
|
||||||
digest = digest_bytes(raw)
|
|
||||||
for target, data, mode in (
|
|
||||||
(path, raw, "wb"),
|
|
||||||
(sidecar, f"{digest} {path.name}\n".encode("ascii"), "wb"),
|
|
||||||
(receipt_path, canonical_json(dict(receipt)), "wb"),
|
|
||||||
):
|
|
||||||
fd, temporary = tempfile.mkstemp(prefix=f".{target.name}.", dir=target.parent)
|
|
||||||
try:
|
|
||||||
os.fchmod(fd, 0o600)
|
|
||||||
with os.fdopen(fd, mode) as stream:
|
|
||||||
stream.write(data)
|
|
||||||
stream.flush()
|
|
||||||
os.fsync(stream.fileno())
|
|
||||||
os.replace(temporary, target)
|
|
||||||
os.chmod(target, 0o600)
|
|
||||||
finally:
|
|
||||||
if os.path.exists(temporary):
|
|
||||||
os.unlink(temporary)
|
|
||||||
return digest
|
|
||||||
@@ -1,106 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Machine campaign approval gate for protected engineering PRs."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
import subprocess
|
|
||||||
from typing import Any, Mapping
|
|
||||||
|
|
||||||
from release_campaign_authorization import CampaignAuthorization, CampaignError
|
|
||||||
|
|
||||||
|
|
||||||
class GateError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def git_is_ancestor(repository: Path, ancestor: str, descendant: str) -> bool:
|
|
||||||
result = subprocess.run(
|
|
||||||
[
|
|
||||||
"git",
|
|
||||||
f"--git-dir={repository}",
|
|
||||||
"merge-base",
|
|
||||||
"--is-ancestor",
|
|
||||||
ancestor,
|
|
||||||
descendant,
|
|
||||||
],
|
|
||||||
check=False,
|
|
||||||
capture_output=True,
|
|
||||||
)
|
|
||||||
if result.returncode == 0:
|
|
||||||
return True
|
|
||||||
if result.returncode == 1:
|
|
||||||
return False
|
|
||||||
raise GateError("git_ancestor_check_failed")
|
|
||||||
|
|
||||||
|
|
||||||
def evaluate(
|
|
||||||
campaign: CampaignAuthorization,
|
|
||||||
facts: Mapping[str, Any],
|
|
||||||
repository: Path,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
required = {
|
|
||||||
"repository",
|
|
||||||
"pr_number",
|
|
||||||
"base_ref",
|
|
||||||
"base_oid",
|
|
||||||
"head_oid",
|
|
||||||
"required_checks_all_success",
|
|
||||||
"required_checks_same_head",
|
|
||||||
"unresolved_conversations",
|
|
||||||
"mergeable",
|
|
||||||
"protected_target",
|
|
||||||
}
|
|
||||||
if set(facts) != required:
|
|
||||||
raise GateError("pr_facts_fields_invalid")
|
|
||||||
if facts["repository"] != "aios/aios":
|
|
||||||
raise GateError("repository_mismatch")
|
|
||||||
if facts["base_ref"] != "refs/heads/main" or facts["protected_target"] != "main":
|
|
||||||
raise GateError("protected_main_required")
|
|
||||||
if facts["required_checks_all_success"] is not True:
|
|
||||||
raise GateError("required_checks_not_success")
|
|
||||||
if facts["required_checks_same_head"] is not True:
|
|
||||||
raise GateError("required_checks_stale")
|
|
||||||
if facts["unresolved_conversations"] != 0:
|
|
||||||
raise GateError("unresolved_conversations")
|
|
||||||
if facts["mergeable"] is not True:
|
|
||||||
raise GateError("pr_not_mergeable")
|
|
||||||
if not git_is_ancestor(repository, campaign.anchor_oid, str(facts["base_oid"])):
|
|
||||||
raise GateError("base_outside_campaign_lineage")
|
|
||||||
if not git_is_ancestor(repository, str(facts["base_oid"]), str(facts["head_oid"])):
|
|
||||||
raise GateError("head_outside_pr_lineage")
|
|
||||||
campaign.assert_operation("protected_merge")
|
|
||||||
return {
|
|
||||||
"schema": "aios.campaign-approval-gate-result.v1",
|
|
||||||
"status": "SUCCESS",
|
|
||||||
"campaign_id": campaign.campaign_id,
|
|
||||||
"pr_number": facts["pr_number"],
|
|
||||||
"head_oid": facts["head_oid"],
|
|
||||||
"owner_ui_interaction_required": False,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
p = argparse.ArgumentParser()
|
|
||||||
p.add_argument("--campaign", type=Path, required=True)
|
|
||||||
p.add_argument("--revocation", type=Path)
|
|
||||||
p.add_argument("--pr-facts", type=Path, required=True)
|
|
||||||
p.add_argument("--repository", type=Path, required=True)
|
|
||||||
a = p.parse_args()
|
|
||||||
campaign = CampaignAuthorization.load(
|
|
||||||
a.campaign,
|
|
||||||
revocation=a.revocation,
|
|
||||||
require_active=True,
|
|
||||||
)
|
|
||||||
facts = json.loads(a.pr_facts.read_text(encoding="utf-8"))
|
|
||||||
print(json.dumps(evaluate(campaign, facts, a.repository), sort_keys=True))
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
try:
|
|
||||||
raise SystemExit(main())
|
|
||||||
except (CampaignError, GateError) as exc:
|
|
||||||
print(json.dumps({"status": "FAILURE", "reason": str(exc)}, sort_keys=True))
|
|
||||||
raise SystemExit(2)
|
|
||||||
@@ -1,150 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Automatically accept an exact release binding under an ACTIVE campaign."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import stat
|
|
||||||
import tempfile
|
|
||||||
from typing import Any, Mapping
|
|
||||||
|
|
||||||
from release_campaign_authorization import CampaignAuthorization, CampaignError
|
|
||||||
|
|
||||||
|
|
||||||
class AcceptanceError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def secure_json(path: Path) -> dict[str, Any]:
|
|
||||||
if path.is_symlink() or not path.is_file():
|
|
||||||
raise AcceptanceError(f"invalid_file:{path}")
|
|
||||||
st = path.lstat()
|
|
||||||
if st.st_uid != 0 or stat.S_IMODE(st.st_mode) & 0o022:
|
|
||||||
raise AcceptanceError(f"unsafe_file:{path}")
|
|
||||||
value = json.loads(path.read_text(encoding="utf-8"))
|
|
||||||
if not isinstance(value, dict):
|
|
||||||
raise AcceptanceError(f"json_object_required:{path}")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def atomic_new(path: Path, value: Mapping[str, Any]) -> None:
|
|
||||||
if path.exists() or path.is_symlink():
|
|
||||||
raise AcceptanceError("acceptance_receipt_exists")
|
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
fd, name = tempfile.mkstemp(prefix=f".{path.name}.", dir=path.parent)
|
|
||||||
try:
|
|
||||||
os.fchmod(fd, 0o600)
|
|
||||||
if os.geteuid() == 0:
|
|
||||||
os.fchown(fd, 0, 0)
|
|
||||||
with os.fdopen(fd, "w", encoding="utf-8", closefd=True) as stream:
|
|
||||||
json.dump(dict(value), stream, sort_keys=True, indent=2)
|
|
||||||
stream.write("\n")
|
|
||||||
stream.flush()
|
|
||||||
os.fsync(stream.fileno())
|
|
||||||
os.replace(name, path)
|
|
||||||
os.chmod(path, 0o600)
|
|
||||||
finally:
|
|
||||||
try:
|
|
||||||
os.unlink(name)
|
|
||||||
except FileNotFoundError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def evaluate(
|
|
||||||
campaign: CampaignAuthorization,
|
|
||||||
binding: Mapping[str, Any],
|
|
||||||
evidence: Mapping[str, Any],
|
|
||||||
artifact: Mapping[str, Any],
|
|
||||||
staging: Mapping[str, Any],
|
|
||||||
main: Mapping[str, Any],
|
|
||||||
tag: Mapping[str, Any],
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
required_binding = {
|
|
||||||
"version",
|
|
||||||
"target_oid",
|
|
||||||
"evidence_closure_oid",
|
|
||||||
"artifact_sha256",
|
|
||||||
"artifact_size",
|
|
||||||
"approval_plan_id",
|
|
||||||
}
|
|
||||||
if set(binding) != required_binding:
|
|
||||||
raise AcceptanceError("binding_fields_invalid")
|
|
||||||
if main.get("repository") != "aios/aios":
|
|
||||||
raise AcceptanceError("repository_identity_mismatch")
|
|
||||||
if main.get("target_in_main_lineage") is not True:
|
|
||||||
raise AcceptanceError("target_outside_main_lineage")
|
|
||||||
if evidence.get("object_type") != "commit":
|
|
||||||
raise AcceptanceError("evidence_not_commit")
|
|
||||||
if evidence.get("target_is_ancestor") is not True:
|
|
||||||
raise AcceptanceError("target_not_evidence_ancestor")
|
|
||||||
if evidence.get("closure_oid") != binding["evidence_closure_oid"]:
|
|
||||||
raise AcceptanceError("evidence_oid_mismatch")
|
|
||||||
if artifact.get("sha256") != binding["artifact_sha256"]:
|
|
||||||
raise AcceptanceError("artifact_digest_mismatch")
|
|
||||||
if artifact.get("size") != binding["artifact_size"]:
|
|
||||||
raise AcceptanceError("artifact_size_mismatch")
|
|
||||||
if artifact.get("retained") is not True:
|
|
||||||
raise AcceptanceError("artifact_not_retained")
|
|
||||||
if staging.get("install") != "PASS":
|
|
||||||
raise AcceptanceError("staging_install_failed")
|
|
||||||
if staging.get("health") != "PASS":
|
|
||||||
raise AcceptanceError("staging_health_failed")
|
|
||||||
if staging.get("restart") != "PASS":
|
|
||||||
raise AcceptanceError("staging_restart_failed")
|
|
||||||
if staging.get("rollback_readiness") != "PASS":
|
|
||||||
raise AcceptanceError("rollback_readiness_failed")
|
|
||||||
if tag.get("conflict") is not False:
|
|
||||||
raise AcceptanceError("tag_conflict")
|
|
||||||
campaign.assert_operation("derive_campaign_acceptance")
|
|
||||||
return {
|
|
||||||
"schema": "aios.campaign-release-acceptance.v1",
|
|
||||||
"accepted": True,
|
|
||||||
"binding_replaced": False,
|
|
||||||
"campaign_id": campaign.campaign_id,
|
|
||||||
"campaign_generation": campaign.data["generation"],
|
|
||||||
**dict(binding),
|
|
||||||
"owner_interaction_required": False,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
p = argparse.ArgumentParser()
|
|
||||||
p.add_argument("--campaign", type=Path, required=True)
|
|
||||||
p.add_argument("--revocation", type=Path)
|
|
||||||
p.add_argument("--binding", type=Path, required=True)
|
|
||||||
p.add_argument("--evidence", type=Path, required=True)
|
|
||||||
p.add_argument("--artifact", type=Path, required=True)
|
|
||||||
p.add_argument("--staging", type=Path, required=True)
|
|
||||||
p.add_argument("--main-facts", type=Path, required=True)
|
|
||||||
p.add_argument("--tag-facts", type=Path, required=True)
|
|
||||||
p.add_argument("--output", type=Path, required=True)
|
|
||||||
a = p.parse_args()
|
|
||||||
|
|
||||||
campaign = CampaignAuthorization.load(
|
|
||||||
a.campaign,
|
|
||||||
revocation=a.revocation,
|
|
||||||
require_active=True,
|
|
||||||
)
|
|
||||||
receipt = evaluate(
|
|
||||||
campaign,
|
|
||||||
secure_json(a.binding),
|
|
||||||
secure_json(a.evidence),
|
|
||||||
secure_json(a.artifact),
|
|
||||||
secure_json(a.staging),
|
|
||||||
secure_json(a.main_facts),
|
|
||||||
secure_json(a.tag_facts),
|
|
||||||
)
|
|
||||||
atomic_new(a.output, receipt)
|
|
||||||
print(json.dumps(receipt, sort_keys=True))
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
try:
|
|
||||||
raise SystemExit(main())
|
|
||||||
except (CampaignError, AcceptanceError) as exc:
|
|
||||||
print(json.dumps({"status": "BLOCKED", "reason": str(exc)}, sort_keys=True))
|
|
||||||
raise SystemExit(2)
|
|
||||||
@@ -1,67 +0,0 @@
|
|||||||
"""Runtime deployment-state provider for governed health contracts."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import datetime as dt
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import re
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Mapping
|
|
||||||
|
|
||||||
_OID = re.compile(r"^[0-9a-f]{40}$")
|
|
||||||
_SHA = re.compile(r"^[0-9a-f]{64}$")
|
|
||||||
_STATES = {"STAGED", "ACTIVE", "ROLLED_BACK"}
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class RuntimeDeploymentState:
|
|
||||||
version: str
|
|
||||||
commit_oid: str
|
|
||||||
artifact_sha256: str
|
|
||||||
deployment_state: str
|
|
||||||
|
|
||||||
@property
|
|
||||||
def production_deployed(self) -> bool:
|
|
||||||
return self.deployment_state == "ACTIVE"
|
|
||||||
|
|
||||||
def health_payload(self) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"status": "ok",
|
|
||||||
"version": self.version,
|
|
||||||
"commit_oid": self.commit_oid,
|
|
||||||
"artifact_sha256": self.artifact_sha256,
|
|
||||||
"deployment_state": self.deployment_state,
|
|
||||||
"production_deployed": self.production_deployed,
|
|
||||||
"health_timestamp": dt.datetime.now(dt.timezone.utc).isoformat(),
|
|
||||||
}
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_environment(cls, environ: Mapping[str, str] | None = None) -> "RuntimeDeploymentState":
|
|
||||||
env = dict(environ or os.environ)
|
|
||||||
values: dict[str, Any] = {}
|
|
||||||
state_file = env.get("DEPLOYMENT_STATE_FILE", "").strip()
|
|
||||||
if state_file:
|
|
||||||
path = Path(state_file)
|
|
||||||
if path.is_file() and not path.is_symlink():
|
|
||||||
loaded = json.loads(path.read_text(encoding="utf-8"))
|
|
||||||
if not isinstance(loaded, dict):
|
|
||||||
raise ValueError("deployment_state_file_object_required")
|
|
||||||
values.update(loaded)
|
|
||||||
file_deployment_state = values.get("deployment_state")
|
|
||||||
values.update({k: v for k, v in {
|
|
||||||
"version": env.get("RELEASE_VERSION", values.get("version", "")),
|
|
||||||
"commit_oid": env.get("TARGET_OID", values.get("commit_oid", "")),
|
|
||||||
"artifact_sha256": env.get("ARTIFACT_SHA256", values.get("artifact_sha256", "")),
|
|
||||||
"deployment_state": env.get("DEPLOYMENT_STATE", values.get("deployment_state", "")),
|
|
||||||
}.items() if v != ""})
|
|
||||||
if file_deployment_state not in (None, ""):
|
|
||||||
values["deployment_state"] = file_deployment_state
|
|
||||||
version = str(values.get("version", ""))
|
|
||||||
commit_oid = str(values.get("commit_oid", ""))
|
|
||||||
artifact_sha256 = str(values.get("artifact_sha256", ""))
|
|
||||||
deployment_state = str(values.get("deployment_state", ""))
|
|
||||||
if not version or not _OID.fullmatch(commit_oid) or not _SHA.fullmatch(artifact_sha256):
|
|
||||||
raise ValueError("deployment_binding_invalid")
|
|
||||||
if deployment_state not in _STATES:
|
|
||||||
raise ValueError("deployment_state_invalid")
|
|
||||||
return cls(version, commit_oid, artifact_sha256, deployment_state)
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
"""Permanent AIOS governance source components."""
|
|
||||||
@@ -1,75 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
import json
|
|
||||||
import pathlib
|
|
||||||
import socket
|
|
||||||
import time
|
|
||||||
|
|
||||||
SOCKET = '/run/aios-governance-executor/executor.sock'
|
|
||||||
STATE = pathlib.Path('/var/lib/aios-gitops/governance-executor/drift-monitor.json')
|
|
||||||
APPROVED = pathlib.Path('/var/lib/aios-gitops/runtime/tasks/AIOS-PROTECTION-SNAPSHOT-DRIFT-FORENSICS-V1/protection-drift/semantic-analysis.json')
|
|
||||||
|
|
||||||
|
|
||||||
def call():
|
|
||||||
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
|
||||||
sock.settimeout(20)
|
|
||||||
sock.connect(SOCKET)
|
|
||||||
request = {'operation': 'GET_BRANCH_PROTECTION', 'payload': {'repository': 'aios/aios', 'branch': 'main'}}
|
|
||||||
sock.sendall((json.dumps(request, separators=(',', ':')) + '\n').encode())
|
|
||||||
chunks = []
|
|
||||||
while True:
|
|
||||||
part = sock.recv(1024 * 1024)
|
|
||||||
if not part:
|
|
||||||
break
|
|
||||||
chunks.append(part)
|
|
||||||
if b'\n' in part:
|
|
||||||
break
|
|
||||||
sock.close()
|
|
||||||
return json.loads(b''.join(chunks).splitlines()[0])
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
try:
|
|
||||||
result = call()
|
|
||||||
if result.get('status') != 'PASS':
|
|
||||||
raise RuntimeError(str(result.get('code') or 'EXECUTOR_GET_FAILED'))
|
|
||||||
approved = json.loads(APPROVED.read_text(encoding='utf-8'))
|
|
||||||
approved_transport = approved['approved_transport_digest_from_evidence']
|
|
||||||
approved_semantic = approved['approved_semantic_digest']
|
|
||||||
transport = result['transport_digest']
|
|
||||||
semantic = result['semantic_digest']
|
|
||||||
if semantic != approved_semantic:
|
|
||||||
state = 'GOVERNANCE_POLICY_DRIFT'
|
|
||||||
elif transport != approved_transport:
|
|
||||||
state = 'GOVERNANCE_TRANSPORT_METADATA_CHANGED'
|
|
||||||
else:
|
|
||||||
state = 'NO_DRIFT'
|
|
||||||
payload = {
|
|
||||||
'schema': 'aios.governance-drift-monitor.v1',
|
|
||||||
'observed_at_unix': int(time.time()),
|
|
||||||
'status': state,
|
|
||||||
'release_blocking': False,
|
|
||||||
'approved_transport_digest': approved_transport,
|
|
||||||
'observed_transport_digest': transport,
|
|
||||||
'approved_semantic_digest': approved_semantic,
|
|
||||||
'observed_semantic_digest': semantic,
|
|
||||||
'semantic_schema': result.get('semantic_schema'),
|
|
||||||
'source': 'Dedicated Governance Executor GET_BRANCH_PROTECTION',
|
|
||||||
}
|
|
||||||
except Exception as exc:
|
|
||||||
payload = {
|
|
||||||
'schema': 'aios.governance-drift-monitor.v1',
|
|
||||||
'observed_at_unix': int(time.time()),
|
|
||||||
'status': 'MONITOR_UNAVAILABLE',
|
|
||||||
'release_blocking': False,
|
|
||||||
'error_code': str(exc),
|
|
||||||
}
|
|
||||||
STATE.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
temporary = STATE.with_suffix('.tmp')
|
|
||||||
temporary.write_text(json.dumps(payload, sort_keys=True, separators=(',', ':'), ensure_ascii=False) + '\n', encoding='utf-8')
|
|
||||||
temporary.replace(STATE)
|
|
||||||
STATE.chmod(0o640)
|
|
||||||
print(json.dumps(payload, sort_keys=True, separators=(',', ':'), ensure_ascii=False))
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
main()
|
|
||||||
@@ -1,417 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
import base64
|
|
||||||
import copy
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import pathlib
|
|
||||||
import socket
|
|
||||||
import time
|
|
||||||
import urllib.error
|
|
||||||
import urllib.parse
|
|
||||||
import urllib.request
|
|
||||||
|
|
||||||
SOURCE_GOVERNANCE = "/opt/aios/aios-control-plane/current/active-runtime/app/governance"
|
|
||||||
if SOURCE_GOVERNANCE not in __import__("sys").path:
|
|
||||||
__import__("sys").path.insert(0, SOURCE_GOVERNANCE)
|
|
||||||
from stage2_gate_migration import (
|
|
||||||
CAMPAIGN_GATE,
|
|
||||||
CAMPAIGN_ID,
|
|
||||||
OWNER_GATE,
|
|
||||||
STAGE1_SEMANTIC_SHA,
|
|
||||||
Stage2GateError,
|
|
||||||
build_target,
|
|
||||||
semantic_digest as stage2_semantic_digest,
|
|
||||||
validate_stage2_facts,
|
|
||||||
)
|
|
||||||
|
|
||||||
API = "http://127.0.0.1:3000"
|
|
||||||
REPO = "aios/aios"
|
|
||||||
SOCKET = "/run/aios-governance-executor/executor.sock"
|
|
||||||
JOURNAL = pathlib.Path("/var/lib/aios-gitops/governance-executor/executor-journal.jsonl")
|
|
||||||
RECONCILIATION_BINDING_PATH = pathlib.Path("/var/lib/aios-gitops/governance-executor/reconciliation-binding.json")
|
|
||||||
PR54_HEAD = "015185d7e5191d18c5c1596160807ac228782a0a"
|
|
||||||
PR54_REVIEW = 104
|
|
||||||
PR54_MANIFEST_SHA = "1b11ee42d53a9f6c1646e5f5ddd8ce5b7ecf301483b9f3d47a068290d541ba4b"
|
|
||||||
PR54_PREVIOUS_PROTECTION_SHA = "8b3370add21cd23c2b4f41d8d84d3eaaa09ddf0451549b72c7941bbd92d5d60a"
|
|
||||||
SEMANTIC_SCHEMA = "aios.semantic-protection.v1"
|
|
||||||
APPROVED_SEMANTIC_SHA = "ba15d512ed8c1f43e5f72c84125a3a71a7347ef59e18000bbfca24d6e69f1b19"
|
|
||||||
STAGE1_SEMANTIC_SHA = "d911e856f71de6105b8efa08e13338a448e0b6d37536332decd80c9f409ecf27"
|
|
||||||
USER_IDS = {"owner": 2, "aios-autonomy-agent": 153}
|
|
||||||
|
|
||||||
|
|
||||||
def canon(value):
|
|
||||||
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()
|
|
||||||
|
|
||||||
|
|
||||||
def sha(value):
|
|
||||||
return hashlib.sha256(canon(value)).hexdigest()
|
|
||||||
|
|
||||||
|
|
||||||
def token():
|
|
||||||
value = (pathlib.Path(os.environ["CREDENTIALS_DIRECTORY"]) / "gitea-governance-executor-token").read_text().strip()
|
|
||||||
if len(value) < 16:
|
|
||||||
raise RuntimeError("EXECUTOR_CREDENTIAL_INVALID")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def api(method, path, value, body=None, expected=(200,)):
|
|
||||||
raw = None if body is None else json.dumps(body, separators=(",", ":")).encode()
|
|
||||||
req = urllib.request.Request(API + path, data=raw, method=method)
|
|
||||||
req.add_header("Authorization", "token " + value)
|
|
||||||
req.add_header("Accept", "application/json")
|
|
||||||
if raw is not None:
|
|
||||||
req.add_header("Content-Type", "application/json")
|
|
||||||
try:
|
|
||||||
with urllib.request.urlopen(req, timeout=20) as response:
|
|
||||||
status = response.status
|
|
||||||
payload = response.read()
|
|
||||||
except urllib.error.HTTPError as exc:
|
|
||||||
if exc.code in expected:
|
|
||||||
payload = exc.read()
|
|
||||||
return exc.code, json.loads(payload) if payload else None
|
|
||||||
raise RuntimeError("GITEA_HTTP_" + str(exc.code))
|
|
||||||
if status not in expected:
|
|
||||||
raise RuntimeError("GITEA_HTTP_" + str(status))
|
|
||||||
if not payload or not payload.strip():
|
|
||||||
raise RuntimeError("GITEA_EMPTY_JSON:" + path)
|
|
||||||
try:
|
|
||||||
decoded = json.loads(payload)
|
|
||||||
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
|
|
||||||
raise RuntimeError("GITEA_INVALID_JSON:" + path) from exc
|
|
||||||
return status, decoded
|
|
||||||
|
|
||||||
|
|
||||||
def journal(event, **fields):
|
|
||||||
JOURNAL.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
record = {"schema": "aios.governance-executor-journal.v2", "event": event, "unix": int(time.time()), **fields}
|
|
||||||
with JOURNAL.open("a", encoding="utf-8") as stream:
|
|
||||||
stream.write(json.dumps(record, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + "\n")
|
|
||||||
stream.flush()
|
|
||||||
os.fsync(stream.fileno())
|
|
||||||
|
|
||||||
|
|
||||||
def typed_json(value):
|
|
||||||
if value in (None, "null"):
|
|
||||||
return []
|
|
||||||
if isinstance(value, str):
|
|
||||||
return json.loads(value)
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def user_ids(names):
|
|
||||||
result = []
|
|
||||||
for name in names or []:
|
|
||||||
if name not in USER_IDS:
|
|
||||||
raise RuntimeError("UNKNOWN_GOVERNANCE_USER:" + str(name))
|
|
||||||
result.append(USER_IDS[name])
|
|
||||||
return sorted(result)
|
|
||||||
|
|
||||||
|
|
||||||
def semantic_policy(value):
|
|
||||||
policy = {
|
|
||||||
"schema_version": SEMANTIC_SCHEMA,
|
|
||||||
"repo_id": 1,
|
|
||||||
"protection_id": 2,
|
|
||||||
"branch_name": value.get("branch_name"),
|
|
||||||
"can_push": value.get("enable_push"),
|
|
||||||
"can_force_push": value.get("enable_force_push"),
|
|
||||||
"enable_merge_whitelist": value.get("enable_merge_whitelist"),
|
|
||||||
"merge_whitelist_user_ids": user_ids(value.get("merge_whitelist_usernames")),
|
|
||||||
"merge_whitelist_team_ids": [],
|
|
||||||
"required_approvals": value.get("required_approvals"),
|
|
||||||
"enable_status_check": value.get("enable_status_check"),
|
|
||||||
"status_check_contexts": sorted(value.get("status_check_contexts") or []),
|
|
||||||
"block_admin_merge_override": value.get("block_admin_merge_override"),
|
|
||||||
"block_on_official_review_requests": value.get("block_on_official_review_requests"),
|
|
||||||
"block_on_outdated_branch": value.get("block_on_outdated_branch"),
|
|
||||||
"block_on_rejected_reviews": value.get("block_on_rejected_reviews"),
|
|
||||||
"dismiss_stale_approvals": value.get("dismiss_stale_approvals"),
|
|
||||||
"ignore_stale_approvals": value.get("ignore_stale_approvals"),
|
|
||||||
"enable_approvals_whitelist": value.get("enable_approvals_whitelist"),
|
|
||||||
"approvals_whitelist_user_ids": user_ids(value.get("approvals_whitelist_username")),
|
|
||||||
"approvals_whitelist_team_ids": [],
|
|
||||||
"enable_bypass_allowlist": value.get("enable_bypass_allowlist"),
|
|
||||||
"bypass_allowlist_user_ids": user_ids(value.get("bypass_allowlist_usernames")),
|
|
||||||
"bypass_allowlist_team_ids": [],
|
|
||||||
"enable_force_push_allowlist": value.get("enable_force_push_allowlist"),
|
|
||||||
"force_push_allowlist_user_ids": user_ids(value.get("force_push_allowlist_usernames")),
|
|
||||||
"force_push_allowlist_team_ids": [],
|
|
||||||
"enable_whitelist": value.get("enable_push_whitelist"),
|
|
||||||
"whitelist_user_ids": user_ids(value.get("push_whitelist_usernames")),
|
|
||||||
"whitelist_team_ids": [],
|
|
||||||
"force_push_allowlist_deploy_keys": value.get("force_push_allowlist_deploy_keys"),
|
|
||||||
"whitelist_deploy_keys": value.get("push_whitelist_deploy_keys"),
|
|
||||||
"require_signed_commits": value.get("require_signed_commits"),
|
|
||||||
"priority": value.get("priority"),
|
|
||||||
"protected_file_patterns": value.get("protected_file_patterns"),
|
|
||||||
"unprotected_file_patterns": value.get("unprotected_file_patterns"),
|
|
||||||
}
|
|
||||||
return policy
|
|
||||||
|
|
||||||
|
|
||||||
def semantic_sha(value):
|
|
||||||
return sha(semantic_policy(value))
|
|
||||||
|
|
||||||
|
|
||||||
def verify_binding(binding, value):
|
|
||||||
required = {
|
|
||||||
"governance_request_PR", "exact_request_head", "Owner_review_id", "Owner_review_exact_head",
|
|
||||||
"manifest_sha256", "expected_previous_snapshot_sha256", "approved_delta",
|
|
||||||
"semantic_reconciliation_request_id", "semantic_reconciliation_head",
|
|
||||||
"semantic_reconciliation_review_id", "semantic_reconciliation_review_head",
|
|
||||||
"approved_semantic_digest",
|
|
||||||
}
|
|
||||||
if set(binding) != required:
|
|
||||||
raise RuntimeError("GOVERNANCE_BINDING_SCHEMA_INVALID")
|
|
||||||
if binding["governance_request_PR"] != 54 or binding["exact_request_head"] != PR54_HEAD:
|
|
||||||
raise RuntimeError("GOVERNANCE_BINDING_PR54_MISMATCH")
|
|
||||||
if binding["Owner_review_id"] != PR54_REVIEW or binding["Owner_review_exact_head"] != PR54_HEAD:
|
|
||||||
raise RuntimeError("GOVERNANCE_BINDING_REVIEW_MISMATCH")
|
|
||||||
if binding["manifest_sha256"] != PR54_MANIFEST_SHA:
|
|
||||||
raise RuntimeError("GOVERNANCE_BINDING_MANIFEST_MISMATCH")
|
|
||||||
if binding["expected_previous_snapshot_sha256"] != PR54_PREVIOUS_PROTECTION_SHA:
|
|
||||||
raise RuntimeError("GOVERNANCE_BINDING_SNAPSHOT_MISMATCH")
|
|
||||||
if binding["approved_delta"] != {"add_merge_whitelist_user_id": 153}:
|
|
||||||
raise RuntimeError("GOVERNANCE_BINDING_DELTA_DENIED")
|
|
||||||
if binding["approved_semantic_digest"] != APPROVED_SEMANTIC_SHA:
|
|
||||||
raise RuntimeError("GOVERNANCE_BINDING_SEMANTIC_DIGEST_MISMATCH")
|
|
||||||
if not RECONCILIATION_BINDING_PATH.is_file():
|
|
||||||
raise RuntimeError("SEMANTIC_RECONCILIATION_BINDING_REQUIRED")
|
|
||||||
try:
|
|
||||||
reconciliation = json.loads(RECONCILIATION_BINDING_PATH.read_text(encoding="utf-8"))
|
|
||||||
except (OSError, ValueError):
|
|
||||||
raise RuntimeError("SEMANTIC_RECONCILIATION_BINDING_INVALID") from None
|
|
||||||
required_reconciliation = {
|
|
||||||
"schema", "request_id", "pr_number", "request_head", "owner_review_id",
|
|
||||||
"owner_review_head", "approved_semantic_digest", "approved_delta",
|
|
||||||
}
|
|
||||||
if set(reconciliation) != required_reconciliation or reconciliation.get("schema") != "aios.semantic-reconciliation-binding.v1":
|
|
||||||
raise RuntimeError("SEMANTIC_RECONCILIATION_BINDING_INVALID")
|
|
||||||
if reconciliation["approved_semantic_digest"] != APPROVED_SEMANTIC_SHA or reconciliation["approved_delta"] != binding["approved_delta"]:
|
|
||||||
raise RuntimeError("SEMANTIC_RECONCILIATION_BINDING_MISMATCH")
|
|
||||||
if (
|
|
||||||
reconciliation["request_id"] != binding["semantic_reconciliation_request_id"]
|
|
||||||
or reconciliation["request_head"] != binding["semantic_reconciliation_head"]
|
|
||||||
or reconciliation["owner_review_id"] != binding["semantic_reconciliation_review_id"]
|
|
||||||
or reconciliation["owner_review_head"] != binding["semantic_reconciliation_review_head"]
|
|
||||||
):
|
|
||||||
raise RuntimeError("SEMANTIC_RECONCILIATION_BINDING_MISMATCH")
|
|
||||||
_, reconciliation_pr = api("GET", f"/api/v1/repos/{REPO}/pulls/{reconciliation['pr_number']}", value)
|
|
||||||
if (
|
|
||||||
reconciliation_pr.get("state") != "open"
|
|
||||||
or reconciliation_pr.get("merged")
|
|
||||||
or reconciliation_pr.get("head", {}).get("sha") != reconciliation["request_head"]
|
|
||||||
or reconciliation_pr.get("base", {}).get("ref") != "governance/approvals"
|
|
||||||
):
|
|
||||||
raise RuntimeError("SEMANTIC_RECONCILIATION_REQUEST_STALE")
|
|
||||||
_, reconciliation_reviews = api("GET", f"/api/v1/repos/{REPO}/pulls/{reconciliation['pr_number']}/reviews", value)
|
|
||||||
reconciliation_review = next((x for x in reconciliation_reviews if x.get("id") == reconciliation["owner_review_id"]), None)
|
|
||||||
if (
|
|
||||||
not reconciliation_review
|
|
||||||
or reconciliation_review.get("user", {}).get("id") != 2
|
|
||||||
or reconciliation_review.get("state") != "APPROVED"
|
|
||||||
or reconciliation_review.get("commit_id") != reconciliation["owner_review_head"]
|
|
||||||
or reconciliation_review.get("dismissed") is not False
|
|
||||||
):
|
|
||||||
raise RuntimeError("SEMANTIC_RECONCILIATION_OWNER_REVIEW_STALE")
|
|
||||||
_, pr = api("GET", f"/api/v1/repos/{REPO}/pulls/54", value)
|
|
||||||
if pr.get("state") != "open" or pr.get("merged") or pr.get("head", {}).get("sha") != PR54_HEAD:
|
|
||||||
raise RuntimeError("PR54_APPROVAL_STALE")
|
|
||||||
_, reviews = api("GET", f"/api/v1/repos/{REPO}/pulls/54/reviews", value)
|
|
||||||
review = next((x for x in reviews if x.get("id") == PR54_REVIEW), None)
|
|
||||||
if not review or review.get("user", {}).get("id") != 2 or review.get("state") != "APPROVED" or review.get("commit_id") != PR54_HEAD or review.get("dismissed") is not False:
|
|
||||||
raise RuntimeError("PR54_APPROVAL_STALE")
|
|
||||||
manifest_path = "approvals/governance-policy/AIOS-AUTONOMOUS-MERGE-GOVERNANCE-SCOPE-EXTENSION-V1.json"
|
|
||||||
_, content = api("GET", f"/api/v1/repos/{REPO}/contents/{urllib.parse.quote(manifest_path, safe='/')}?ref={PR54_HEAD}", value)
|
|
||||||
raw = base64.b64decode("".join(content.get("content", "").split()))
|
|
||||||
if hashlib.sha256(raw).hexdigest() != PR54_MANIFEST_SHA:
|
|
||||||
raise RuntimeError("PR54_MANIFEST_STALE")
|
|
||||||
|
|
||||||
|
|
||||||
def get_protection(value, branch):
|
|
||||||
if branch not in {"main", "governance/approvals"}:
|
|
||||||
raise RuntimeError("BRANCH_DENIED")
|
|
||||||
_, protection = api("GET", f"/api/v1/repos/{REPO}/branch_protections/{urllib.parse.quote(branch, safe='')}", value)
|
|
||||||
return protection
|
|
||||||
|
|
||||||
|
|
||||||
def mutation_receipt(before, after, binding, target_semantic):
|
|
||||||
return {
|
|
||||||
"before": {"transport_digest": sha(before), "semantic_digest": semantic_sha(before)},
|
|
||||||
"approved": {
|
|
||||||
"mutation_predicate_digest": sha({"operation": "ADD_MERGE_EXECUTOR", "subject_user_id": 153, "preserve_all_unmodified_security_fields": True}),
|
|
||||||
"Owner_review_id": binding["Owner_review_id"],
|
|
||||||
"semantic_policy_digest": APPROVED_SEMANTIC_SHA,
|
|
||||||
},
|
|
||||||
"after": {"transport_digest": sha(after), "semantic_digest": semantic_sha(after)},
|
|
||||||
"delta": {"add_merge_whitelist_user_id": 153},
|
|
||||||
"target_semantic_digest": sha(target_semantic),
|
|
||||||
"unmodified_security_fields": {"verified": True},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def apply_stage1(binding, value):
|
|
||||||
verify_binding(binding, value)
|
|
||||||
before = get_protection(value, "main")
|
|
||||||
before_semantic = semantic_policy(before)
|
|
||||||
before_semantic_sha = sha(before_semantic)
|
|
||||||
names = list(before.get("merge_whitelist_usernames") or [])
|
|
||||||
name_ids = user_ids(names)
|
|
||||||
if name_ids not in ([2], [2, 153]):
|
|
||||||
raise RuntimeError("PROTECTION_WHITELIST_PRESTATE_INVALID")
|
|
||||||
target_names = ["owner", "aios-autonomy-agent"]
|
|
||||||
target_semantic = copy.deepcopy(before_semantic)
|
|
||||||
target_semantic["merge_whitelist_user_ids"] = [2, 153]
|
|
||||||
if name_ids == [2, 153]:
|
|
||||||
if before_semantic_sha != STAGE1_SEMANTIC_SHA:
|
|
||||||
journal("APPLY_BLOCKED_STAGE1_IDEMPOTENCY_SEMANTIC_CONFLICT", transport_digest=sha(before), semantic_digest=before_semantic_sha, expected_semantic_digest=STAGE1_SEMANTIC_SHA)
|
|
||||||
raise RuntimeError("PROTECTION_STAGE1_SEMANTIC_CONFLICT")
|
|
||||||
receipt = mutation_receipt(before, before, binding, target_semantic)
|
|
||||||
journal("APPLY_APPROVED_PROTECTION_DELTA_IDEMPOTENT_SUCCESS", receipt=receipt)
|
|
||||||
return {"status": "IDEMPOTENT_SUCCESS", "protection": before, "transport_digest": sha(before), "semantic_digest": before_semantic_sha, "target_semantic_digest": sha(target_semantic)}
|
|
||||||
if before_semantic_sha != APPROVED_SEMANTIC_SHA:
|
|
||||||
journal("APPLY_BLOCKED_SEMANTIC_CAS", transport_digest=sha(before), semantic_digest=before_semantic_sha, expected_semantic_digest=APPROVED_SEMANTIC_SHA)
|
|
||||||
raise RuntimeError("PROTECTION_SEMANTIC_CAS_CONFLICT")
|
|
||||||
after_api = api("PATCH", f"/api/v1/repos/{REPO}/branch_protections/main", value, {"merge_whitelist_usernames": target_names})[1]
|
|
||||||
after = get_protection(value, "main")
|
|
||||||
for key in before:
|
|
||||||
if key in {"updated_at", "merge_whitelist_usernames"}:
|
|
||||||
continue
|
|
||||||
if after.get(key) != before.get(key):
|
|
||||||
raise RuntimeError("PROTECTION_UNAPPROVED_FIELD_CHANGED:" + key)
|
|
||||||
if user_ids(after.get("merge_whitelist_usernames")) != [2, 153]:
|
|
||||||
raise RuntimeError("PROTECTION_TARGET_READBACK_FAILED")
|
|
||||||
after_semantic = semantic_policy(after)
|
|
||||||
if after_semantic != target_semantic:
|
|
||||||
raise RuntimeError("PROTECTION_SEMANTIC_TARGET_READBACK_FAILED")
|
|
||||||
receipt = mutation_receipt(before, after, binding, target_semantic)
|
|
||||||
journal("APPLY_APPROVED_PROTECTION_DELTA_COMPLETED", receipt=receipt)
|
|
||||||
return {"status": "APPLIED", "protection": after, "previous_transport_digest": sha(before), "previous_semantic_digest": before_semantic_sha, "transport_digest": sha(after), "semantic_digest": sha(after_semantic), "receipt": receipt}
|
|
||||||
|
|
||||||
|
|
||||||
def apply_stage2(binding, facts, value):
|
|
||||||
verify_binding(binding, value)
|
|
||||||
validate_stage2_facts(facts)
|
|
||||||
before = get_protection(value, "main")
|
|
||||||
if OWNER_GATE in (before.get("status_check_contexts") or []) and stage2_semantic_digest(before) != STAGE1_SEMANTIC_SHA:
|
|
||||||
raise RuntimeError("STAGE2_SEMANTIC_PRECONDITION_INVALID")
|
|
||||||
try:
|
|
||||||
status, target = build_target(before)
|
|
||||||
except Stage2GateError as exc:
|
|
||||||
journal("MIGRATE_APPROVED_GATE_BLOCKED", reason=str(exc), semantic_digest=stage2_semantic_digest(before))
|
|
||||||
raise RuntimeError(str(exc)) from exc
|
|
||||||
target_digest = stage2_semantic_digest(target)
|
|
||||||
if status == "IDEMPOTENT_SUCCESS":
|
|
||||||
journal(
|
|
||||||
"MIGRATE_APPROVED_GATE_IDEMPOTENT_SUCCESS",
|
|
||||||
before_semantic_digest=stage2_semantic_digest(before),
|
|
||||||
target_semantic_digest=target_digest,
|
|
||||||
campaign_id=CAMPAIGN_ID,
|
|
||||||
)
|
|
||||||
return {
|
|
||||||
"status": status,
|
|
||||||
"protection": before,
|
|
||||||
"transport_digest": sha(before),
|
|
||||||
"semantic_digest": stage2_semantic_digest(before),
|
|
||||||
"target_semantic_digest": target_digest,
|
|
||||||
"gate_transition": {"removed": OWNER_GATE, "required": CAMPAIGN_GATE},
|
|
||||||
}
|
|
||||||
after_api = api(
|
|
||||||
"PATCH",
|
|
||||||
f"/api/v1/repos/{REPO}/branch_protections/main",
|
|
||||||
value,
|
|
||||||
{"status_check_contexts": target["status_check_contexts"]},
|
|
||||||
)[1]
|
|
||||||
after = get_protection(value, "main")
|
|
||||||
for key in before:
|
|
||||||
if key in {"updated_at", "status_check_contexts"}:
|
|
||||||
continue
|
|
||||||
if after.get(key) != before.get(key):
|
|
||||||
raise RuntimeError("PROTECTION_UNAPPROVED_FIELD_CHANGED:" + key)
|
|
||||||
if after.get("status_check_contexts") != target["status_check_contexts"]:
|
|
||||||
raise RuntimeError("STAGE2_GATE_TARGET_READBACK_FAILED")
|
|
||||||
if stage2_semantic_digest(after) != target_digest:
|
|
||||||
raise RuntimeError("STAGE2_SEMANTIC_TARGET_READBACK_FAILED")
|
|
||||||
receipt = {
|
|
||||||
"before_transport_digest": sha(before),
|
|
||||||
"before_semantic_digest": stage2_semantic_digest(before),
|
|
||||||
"after_transport_digest": sha(after),
|
|
||||||
"after_semantic_digest": stage2_semantic_digest(after),
|
|
||||||
"target_semantic_digest": target_digest,
|
|
||||||
"campaign_id": CAMPAIGN_ID,
|
|
||||||
"gate_transition": {"removed": OWNER_GATE, "required": CAMPAIGN_GATE},
|
|
||||||
"preserved_security_fields": True,
|
|
||||||
"facts_digest": sha(facts),
|
|
||||||
}
|
|
||||||
journal("MIGRATE_APPROVED_GATE_COMPLETED", receipt=receipt)
|
|
||||||
return {
|
|
||||||
"status": "APPLIED",
|
|
||||||
"protection": after,
|
|
||||||
"previous_transport_digest": sha(before),
|
|
||||||
"previous_semantic_digest": stage2_semantic_digest(before),
|
|
||||||
"transport_digest": sha(after),
|
|
||||||
"semantic_digest": stage2_semantic_digest(after),
|
|
||||||
"target_semantic_digest": target_digest,
|
|
||||||
"receipt": receipt,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def handle(request, value):
|
|
||||||
if not isinstance(request, dict) or set(request) != {"operation", "payload"}:
|
|
||||||
raise RuntimeError("REQUEST_SCHEMA_INVALID")
|
|
||||||
operation = request["operation"]
|
|
||||||
payload = request["payload"]
|
|
||||||
if not isinstance(payload, dict) or payload.get("repository") != REPO:
|
|
||||||
raise RuntimeError("REPOSITORY_DENIED")
|
|
||||||
if operation == "GET_BRANCH_PROTECTION":
|
|
||||||
protection = get_protection(value, payload.get("branch"))
|
|
||||||
journal("GET_BRANCH_PROTECTION", branch=payload.get("branch"), transport_digest=sha(protection), semantic_digest=semantic_sha(protection))
|
|
||||||
return {"status": "PASS", "protection": protection, "snapshot_sha256": sha(protection), "transport_digest": sha(protection), "semantic_digest": semantic_sha(protection), "semantic_schema": SEMANTIC_SCHEMA}
|
|
||||||
if operation == "READBACK_BRANCH_PROTECTION":
|
|
||||||
protection = get_protection(value, payload.get("branch"))
|
|
||||||
return {"status": "PASS", "protection": protection, "snapshot_sha256": sha(protection), "transport_digest": sha(protection), "semantic_digest": semantic_sha(protection), "semantic_schema": SEMANTIC_SCHEMA}
|
|
||||||
if operation == "APPLY_APPROVED_PROTECTION_DELTA":
|
|
||||||
binding = payload.get("GovernanceExecutionBinding")
|
|
||||||
if not isinstance(binding, dict):
|
|
||||||
raise RuntimeError("GOVERNANCE_BINDING_REQUIRED")
|
|
||||||
return apply_stage1(binding, value)
|
|
||||||
if operation == "MIGRATE_APPROVED_GATE":
|
|
||||||
binding = payload.get("GovernanceExecutionBinding")
|
|
||||||
facts = payload.get("Stage2Facts")
|
|
||||||
if not isinstance(binding, dict) or not isinstance(facts, dict):
|
|
||||||
raise RuntimeError("STAGE2_BINDING_OR_FACTS_REQUIRED")
|
|
||||||
return apply_stage2(binding, facts, value)
|
|
||||||
raise RuntimeError("OPERATION_DENIED")
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
value = token()
|
|
||||||
sock_path = pathlib.Path(SOCKET)
|
|
||||||
sock_path.unlink(missing_ok=True)
|
|
||||||
server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
|
||||||
server.bind(SOCKET)
|
|
||||||
os.chmod(SOCKET, 0o660)
|
|
||||||
server.listen(16)
|
|
||||||
journal("EXECUTOR_STARTED", api_surface=["GET_BRANCH_PROTECTION", "APPLY_APPROVED_PROTECTION_DELTA", "MIGRATE_APPROVED_GATE", "READBACK_BRANCH_PROTECTION"], semantic_schema=SEMANTIC_SCHEMA, approved_semantic_digest=APPROVED_SEMANTIC_SHA, stage1_semantic_digest=STAGE1_SEMANTIC_SHA, campaign_gate=CAMPAIGN_GATE)
|
|
||||||
try:
|
|
||||||
while True:
|
|
||||||
conn, _ = server.accept()
|
|
||||||
with conn:
|
|
||||||
data = conn.recv(1024 * 1024 + 1)
|
|
||||||
if len(data) > 1024 * 1024:
|
|
||||||
result = {"status": "FAIL", "code": "REQUEST_TOO_LARGE"}
|
|
||||||
else:
|
|
||||||
try:
|
|
||||||
request = json.loads(data.decode())
|
|
||||||
result = handle(request, value)
|
|
||||||
except Exception as exc:
|
|
||||||
result = {"status": "FAIL", "code": str(exc)}
|
|
||||||
conn.sendall((json.dumps(result, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + "\n").encode())
|
|
||||||
finally:
|
|
||||||
server.close()
|
|
||||||
sock_path.unlink(missing_ok=True)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,347 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
import base64
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import pathlib
|
|
||||||
import shutil
|
|
||||||
import socket
|
|
||||||
import subprocess
|
|
||||||
import tempfile
|
|
||||||
import time
|
|
||||||
import urllib.error
|
|
||||||
import urllib.parse
|
|
||||||
import urllib.request
|
|
||||||
|
|
||||||
LOCAL_API = 'http://127.0.0.1:3000/api/v1'
|
|
||||||
NETWORK_API = 'https://g.cnnetmkt.top/git/api/v1'
|
|
||||||
NETWORK_REMOTE = 'https://g.cnnetmkt.top/git/aios/aios.git'
|
|
||||||
REPO = 'aios/aios'
|
|
||||||
BASE_BRANCH = 'governance/approvals'
|
|
||||||
REQUEST_BRANCH = 'governance/request/protection-semantic-reconciliation-v1-20260808'
|
|
||||||
SOCKET_PATH = '/run/aios-governance-preapproval-publisher/publisher.sock'
|
|
||||||
CREDENTIAL_NAME = 'gitea-machine-token'
|
|
||||||
AUTHORITY_REPO = '/data/aios-gitops/repo.git'
|
|
||||||
TASK_ROOT = pathlib.Path('/var/lib/aios-gitops/runtime/tasks/AIOS-PROTECTION-SNAPSHOT-DRIFT-FORENSICS-V1')
|
|
||||||
INPUT_ROOT = TASK_ROOT / 'publication-input'
|
|
||||||
RECEIPT = TASK_ROOT / 'publication-receipt.json'
|
|
||||||
MANIFEST_REL = 'approvals/governance-policy/AIOS-PROTECTION-SEMANTIC-SNAPSHOT-RECONCILIATION-V1.json'
|
|
||||||
EVIDENCE_REL = 'approvals/governance-policy/AIOS-PROTECTION-SEMANTIC-SNAPSHOT-RECONCILIATION-V1.evidence.json'
|
|
||||||
|
|
||||||
|
|
||||||
def digest_bytes(data):
|
|
||||||
return hashlib.sha256(data).hexdigest()
|
|
||||||
|
|
||||||
|
|
||||||
def digest_json(value):
|
|
||||||
return digest_bytes(json.dumps(value, sort_keys=True, separators=(',', ':'), ensure_ascii=False).encode())
|
|
||||||
|
|
||||||
|
|
||||||
def load_token():
|
|
||||||
path = pathlib.Path(os.environ['CREDENTIALS_DIRECTORY']) / CREDENTIAL_NAME
|
|
||||||
value = path.read_text(encoding='utf-8').strip()
|
|
||||||
if len(value) < 16:
|
|
||||||
raise RuntimeError('MACHINE_CREDENTIAL_INVALID')
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def api(base, method, path, token, body=None, expected=(200,)):
|
|
||||||
raw = None if body is None else json.dumps(body, separators=(',', ':'), ensure_ascii=False).encode()
|
|
||||||
request = urllib.request.Request(base + path, data=raw, method=method)
|
|
||||||
request.add_header('Authorization', 'token ' + token)
|
|
||||||
request.add_header('Accept', 'application/json')
|
|
||||||
if raw is not None:
|
|
||||||
request.add_header('Content-Type', 'application/json')
|
|
||||||
try:
|
|
||||||
with urllib.request.urlopen(request, timeout=20) as response:
|
|
||||||
payload = response.read()
|
|
||||||
if response.status not in expected:
|
|
||||||
raise RuntimeError('GITEA_HTTP_' + str(response.status))
|
|
||||||
return response.status, json.loads(payload) if payload else None
|
|
||||||
except urllib.error.HTTPError as exc:
|
|
||||||
payload = exc.read()
|
|
||||||
if exc.code in expected:
|
|
||||||
return exc.code, json.loads(payload) if payload else None
|
|
||||||
raise RuntimeError('GITEA_HTTP_' + str(exc.code) + ':' + method + ':' + path) from None
|
|
||||||
|
|
||||||
|
|
||||||
def authority_head():
|
|
||||||
result = subprocess.run(
|
|
||||||
['git', '--git-dir=' + AUTHORITY_REPO, 'rev-parse', 'refs/heads/' + BASE_BRANCH],
|
|
||||||
text=True, capture_output=True, check=False,
|
|
||||||
)
|
|
||||||
if result.returncode != 0:
|
|
||||||
raise RuntimeError('AUTHORITY_BASE_UNAVAILABLE')
|
|
||||||
return result.stdout.strip()
|
|
||||||
|
|
||||||
|
|
||||||
def branch_api(base, token, branch, expected=(200,)):
|
|
||||||
return api(base, 'GET', '/repos/' + REPO + '/branches/' + urllib.parse.quote(branch, safe=''), token, expected=expected)
|
|
||||||
|
|
||||||
|
|
||||||
def verify_base(token):
|
|
||||||
authority = authority_head()
|
|
||||||
_, local = branch_api(LOCAL_API, token, BASE_BRANCH)
|
|
||||||
_, network = branch_api(NETWORK_API, token, BASE_BRANCH)
|
|
||||||
local_oid = local.get('commit', {}).get('id')
|
|
||||||
network_oid = network.get('commit', {}).get('id')
|
|
||||||
if not authority or authority != local_oid or authority != network_oid:
|
|
||||||
raise RuntimeError('RECONCILIATION_REQUIRED')
|
|
||||||
return authority
|
|
||||||
|
|
||||||
|
|
||||||
def verify_identity(token):
|
|
||||||
_, user = api(LOCAL_API, 'GET', '/user', token)
|
|
||||||
if user.get('login') != 'aios-autonomy-agent' or user.get('id') != 153 or user.get('is_admin') is not False:
|
|
||||||
raise RuntimeError('MACHINE_IDENTITY_REJECTED')
|
|
||||||
return {'login': user['login'], 'id': user['id'], 'owner': user.get('is_admin')}
|
|
||||||
|
|
||||||
|
|
||||||
def load_inputs():
|
|
||||||
manifest = json.dumps(json.loads((INPUT_ROOT / 'manifest.json').read_text(encoding='utf-8')), sort_keys=True, separators=(',', ':'), ensure_ascii=False).encode()
|
|
||||||
evidence = json.dumps(json.loads((INPUT_ROOT / 'evidence.json').read_text(encoding='utf-8')), sort_keys=True, separators=(',', ':'), ensure_ascii=False).encode()
|
|
||||||
for data in (manifest, evidence):
|
|
||||||
if any(secret in data.lower() for secret in (b'authorization:', b'ghp_', b'github_pat_', b'secret=', b'token=')):
|
|
||||||
raise RuntimeError('SECRET_CONTENT_REJECTED')
|
|
||||||
return manifest, evidence
|
|
||||||
|
|
||||||
|
|
||||||
def publication_key(base_oid, manifest_sha, evidence_sha):
|
|
||||||
payload = REPO + base_oid + REQUEST_BRANCH + manifest_sha + evidence_sha
|
|
||||||
return hashlib.sha256(payload.encode()).hexdigest()
|
|
||||||
|
|
||||||
|
|
||||||
def write_receipt(state, **fields):
|
|
||||||
record = {
|
|
||||||
'schema': 'aios.governance-preapproval-publication.v1',
|
|
||||||
'operation': 'SEMANTIC_RECONCILIATION_REQUEST',
|
|
||||||
'state': state,
|
|
||||||
'recorded_unix': int(time.time()),
|
|
||||||
'request_branch': REQUEST_BRANCH,
|
|
||||||
'manifest_path': MANIFEST_REL,
|
|
||||||
'evidence_path': EVIDENCE_REL,
|
|
||||||
'machine_actor': 'aios-autonomy-agent',
|
|
||||||
'owner': False,
|
|
||||||
'credential_ref': {'scheme': 'systemd-credential', 'name': CREDENTIAL_NAME},
|
|
||||||
'governance_capability_executed': False,
|
|
||||||
'protection_mutation': False,
|
|
||||||
'force_push': False,
|
|
||||||
**fields,
|
|
||||||
}
|
|
||||||
RECEIPT.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
temporary = RECEIPT.with_suffix('.tmp')
|
|
||||||
temporary.write_text(json.dumps(record, sort_keys=True, separators=(',', ':'), ensure_ascii=False) + '\n', encoding='utf-8')
|
|
||||||
temporary.replace(RECEIPT)
|
|
||||||
RECEIPT.chmod(0o640)
|
|
||||||
return record
|
|
||||||
|
|
||||||
|
|
||||||
def existing_prs(token):
|
|
||||||
_, prs = api(LOCAL_API, 'GET', '/repos/' + REPO + '/pulls?state=all&base=' + urllib.parse.quote(BASE_BRANCH, safe='') + '&limit=100', token)
|
|
||||||
return [pr for pr in prs if pr.get('head', {}).get('ref') == REQUEST_BRANCH and pr.get('base', {}).get('ref') == BASE_BRANCH]
|
|
||||||
|
|
||||||
|
|
||||||
def verify_existing_branch(token, branch_oid, manifest, evidence):
|
|
||||||
_, commit = api(LOCAL_API, 'GET', '/repos/' + REPO + '/git/commits/' + branch_oid, token)
|
|
||||||
if commit.get('parents', [{}])[0].get('sha') is None:
|
|
||||||
raise RuntimeError('REQUEST_BRANCH_COMMIT_INVALID')
|
|
||||||
for relative, expected in ((MANIFEST_REL, manifest), (EVIDENCE_REL, evidence)):
|
|
||||||
_, content = api(LOCAL_API, 'GET', '/repos/' + REPO + '/contents/' + urllib.parse.quote(relative, safe='/') + '?ref=' + branch_oid, token)
|
|
||||||
actual = base64.b64decode(''.join(content.get('content', '').split()))
|
|
||||||
if actual != expected:
|
|
||||||
raise RuntimeError('GOVERNANCE_REQUEST_BRANCH_CONFLICT')
|
|
||||||
|
|
||||||
|
|
||||||
def push_one_commit(token, base_oid, manifest, evidence):
|
|
||||||
work_root = TASK_ROOT / 'publisher-work'
|
|
||||||
work_root.mkdir(parents=True, exist_ok=True)
|
|
||||||
temporary = pathlib.Path(tempfile.mkdtemp(prefix='semantic-reconciliation-', dir=work_root))
|
|
||||||
askpass = temporary / 'askpass.sh'
|
|
||||||
clone = temporary / 'repo'
|
|
||||||
credential_path = pathlib.Path(os.environ['CREDENTIALS_DIRECTORY']) / CREDENTIAL_NAME
|
|
||||||
askpass.write_text(
|
|
||||||
'#!/bin/sh\ncase "$1" in *Username*) printf "%s\\n" "$GITEA_MACHINE_ACTOR";; *) cat "$CREDENTIAL_FILE";; esac\n',
|
|
||||||
encoding='utf-8',
|
|
||||||
)
|
|
||||||
askpass.chmod(0o700)
|
|
||||||
env = os.environ.copy()
|
|
||||||
env.update({
|
|
||||||
'GIT_ASKPASS': str(askpass),
|
|
||||||
'GIT_TERMINAL_PROMPT': '0',
|
|
||||||
'GITEA_MACHINE_ACTOR': 'aios-autonomy-agent',
|
|
||||||
'CREDENTIAL_FILE': str(credential_path),
|
|
||||||
'GIT_CONFIG_COUNT': '1',
|
|
||||||
'GIT_CONFIG_KEY_0': 'safe.directory',
|
|
||||||
'GIT_CONFIG_VALUE_0': AUTHORITY_REPO,
|
|
||||||
})
|
|
||||||
try:
|
|
||||||
run = subprocess.run(
|
|
||||||
['git', 'clone', '--branch', BASE_BRANCH, '--single-branch', '--quiet', NETWORK_REMOTE, str(clone)],
|
|
||||||
text=True, capture_output=True, env=env, check=False,
|
|
||||||
)
|
|
||||||
if run.returncode != 0:
|
|
||||||
detail = ' '.join(run.stderr.strip().split())[:240]
|
|
||||||
raise RuntimeError('PUBLISHER_CLONE_FAILED:' + detail)
|
|
||||||
def git(*args, push_env=None):
|
|
||||||
result = subprocess.run(['git', *args], cwd=clone, text=True, capture_output=True, env=push_env or env, check=False)
|
|
||||||
if result.returncode != 0:
|
|
||||||
raise RuntimeError('PUBLISHER_GIT_OPERATION_FAILED')
|
|
||||||
return result.stdout.strip()
|
|
||||||
git('checkout', '--detach', base_oid)
|
|
||||||
git('switch', '--create', REQUEST_BRANCH)
|
|
||||||
manifest_path = clone / MANIFEST_REL
|
|
||||||
evidence_path = clone / EVIDENCE_REL
|
|
||||||
manifest_path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
evidence_path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
manifest_path.write_bytes(manifest)
|
|
||||||
evidence_path.write_bytes(evidence)
|
|
||||||
git('add', '--', MANIFEST_REL, EVIDENCE_REL)
|
|
||||||
staged = git('diff', '--cached', '--name-only').splitlines()
|
|
||||||
if staged != sorted([MANIFEST_REL, EVIDENCE_REL]):
|
|
||||||
raise RuntimeError('PUBLISHER_PATH_ALLOWLIST_FAILED')
|
|
||||||
git('-c', 'user.name=aios-autonomy-agent', '-c', 'user.email=aios-autonomy-agent@aios.invalid', 'commit', '--quiet', '-m', '[governance] publish semantic reconciliation evidence')
|
|
||||||
if git('rev-list', '--count', base_oid + '..HEAD') != '1':
|
|
||||||
raise RuntimeError('PUBLISHER_COMMIT_COUNT_FAILED')
|
|
||||||
git('push', '--porcelain', NETWORK_REMOTE, 'HEAD:refs/heads/' + REQUEST_BRANCH, push_env=env)
|
|
||||||
head = git('rev-parse', 'HEAD')
|
|
||||||
return head
|
|
||||||
finally:
|
|
||||||
shutil.rmtree(temporary, ignore_errors=True)
|
|
||||||
|
|
||||||
|
|
||||||
def publish():
|
|
||||||
token = load_token()
|
|
||||||
identity = verify_identity(token)
|
|
||||||
base_oid = verify_base(token)
|
|
||||||
manifest, evidence = load_inputs()
|
|
||||||
manifest_sha = digest_bytes(manifest)
|
|
||||||
evidence_sha = digest_bytes(evidence)
|
|
||||||
key = publication_key(base_oid, manifest_sha, evidence_sha)
|
|
||||||
write_receipt('REQUESTED', governance_base_oid=base_oid, manifest_sha256=manifest_sha, evidence_sha256=evidence_sha, idempotency_key=key, machine_identity=identity)
|
|
||||||
write_receipt('LOCAL_CANDIDATE_VERIFIED', governance_base_oid=base_oid, manifest_sha256=manifest_sha, evidence_sha256=evidence_sha, idempotency_key=key, machine_identity=identity)
|
|
||||||
|
|
||||||
branch_status, branch = branch_api(LOCAL_API, token, REQUEST_BRANCH, expected=(200, 404))
|
|
||||||
if branch_status == 200:
|
|
||||||
branch_oid = branch.get('commit', {}).get('id')
|
|
||||||
verify_existing_branch(token, branch_oid, manifest, evidence)
|
|
||||||
else:
|
|
||||||
branch_oid = push_one_commit(token, base_oid, manifest, evidence)
|
|
||||||
verify_existing_branch(token, branch_oid, manifest, evidence)
|
|
||||||
write_receipt('BRANCH_PUSHED', governance_base_oid=base_oid, manifest_sha256=manifest_sha, evidence_sha256=evidence_sha, idempotency_key=key, request_head_oid=branch_oid, machine_identity=identity)
|
|
||||||
|
|
||||||
prs = existing_prs(token)
|
|
||||||
if len(prs) > 1:
|
|
||||||
raise RuntimeError('MULTIPLE_MATCHING_APPROVAL_PRS')
|
|
||||||
if prs:
|
|
||||||
pr = prs[0]
|
|
||||||
if pr.get('head', {}).get('sha') != branch_oid:
|
|
||||||
raise RuntimeError('APPROVAL_PR_HEAD_MISMATCH')
|
|
||||||
pr_number = pr.get('number')
|
|
||||||
else:
|
|
||||||
_, pr = api(LOCAL_API, 'POST', '/repos/' + REPO + '/pulls', token, {
|
|
||||||
'title': '[GOVERNANCE RECONCILIATION] Semantic protection snapshot schema migration',
|
|
||||||
'head': REQUEST_BRANCH,
|
|
||||||
'base': BASE_BRANCH,
|
|
||||||
'body': 'Schema-only reconciliation for the existing PR54 Stage-1 and Stage-2 predicates. No new governance capability; Owner exact-head re-attestation required.',
|
|
||||||
}, expected=(201,))
|
|
||||||
pr_number = pr.get('number')
|
|
||||||
write_receipt('PR_CREATED', governance_base_oid=base_oid, manifest_sha256=manifest_sha, evidence_sha256=evidence_sha, idempotency_key=key, request_head_oid=branch_oid, pr_number=pr_number, machine_identity=identity)
|
|
||||||
|
|
||||||
_, remote_branch = branch_api(NETWORK_API, token, REQUEST_BRANCH)
|
|
||||||
if remote_branch.get('commit', {}).get('id') != branch_oid:
|
|
||||||
raise RuntimeError('NETWORK_BRANCH_READBACK_MISMATCH')
|
|
||||||
_, read_pr = api(LOCAL_API, 'GET', '/repos/' + REPO + '/pulls/' + str(pr_number), token)
|
|
||||||
if read_pr.get('head', {}).get('sha') != branch_oid or read_pr.get('base', {}).get('ref') != BASE_BRANCH:
|
|
||||||
raise RuntimeError('PR_READBACK_MISMATCH')
|
|
||||||
result = write_receipt('COMPLETED', governance_base_oid=base_oid, manifest_sha256=manifest_sha, evidence_sha256=evidence_sha, idempotency_key=key, request_head_oid=branch_oid, pr_number=pr_number, pr_head_oid=read_pr.get('head', {}).get('sha'), machine_identity=identity)
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def self_test():
|
|
||||||
token = load_token()
|
|
||||||
identity = verify_identity(token)
|
|
||||||
base_oid = verify_base(token)
|
|
||||||
branch_status, _ = branch_api(LOCAL_API, token, REQUEST_BRANCH, expected=(200, 404))
|
|
||||||
return {
|
|
||||||
'status': 'PASS',
|
|
||||||
'governance_base_oid': base_oid,
|
|
||||||
'machine_identity': identity,
|
|
||||||
'credential_ref': {'scheme': 'systemd-credential', 'name': CREDENTIAL_NAME},
|
|
||||||
'branch_namespace': 'refs/heads/governance/request/*',
|
|
||||||
'target_base': BASE_BRANCH,
|
|
||||||
'request_branch_status': 'EXISTS' if branch_status == 200 else 'ABSENT',
|
|
||||||
'operation_allowlist': ['READ_BASE', 'CREATE_ONE_REQUEST_BRANCH', 'WRITE_TWO_EVIDENCE_PATHS_ONE_COMMIT', 'CREATE_OR_READ_ONE_APPROVAL_PR', 'READBACK'],
|
|
||||||
'denied': ['MAIN_WRITE', 'DIRECT_APPROVALS_WRITE', 'BRANCH_PROTECTION', 'MERGE', 'FORCE_PUSH', 'TAG', 'PRODUCTION', 'CREDENTIAL_MUTATION'],
|
|
||||||
'governance_capability_executed': False,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def read_publication_status():
|
|
||||||
token = load_token()
|
|
||||||
receipt = json.loads(RECEIPT.read_text(encoding='utf-8'))
|
|
||||||
pr_number = receipt.get('pr_number')
|
|
||||||
if not pr_number:
|
|
||||||
raise RuntimeError('PUBLICATION_RECEIPT_PR_MISSING')
|
|
||||||
_, pr = api(LOCAL_API, 'GET', '/repos/' + REPO + '/pulls/' + str(pr_number), token)
|
|
||||||
_, reviews = api(LOCAL_API, 'GET', '/repos/' + REPO + '/pulls/' + str(pr_number) + '/reviews', token)
|
|
||||||
_, branch = branch_api(LOCAL_API, token, REQUEST_BRANCH)
|
|
||||||
_, network_branch = branch_api(NETWORK_API, token, REQUEST_BRANCH)
|
|
||||||
return {
|
|
||||||
'status': 'PASS',
|
|
||||||
'request_branch': REQUEST_BRANCH,
|
|
||||||
'branch_head': branch.get('commit', {}).get('id'),
|
|
||||||
'network_branch_head': network_branch.get('commit', {}).get('id'),
|
|
||||||
'pr_number': pr.get('number'),
|
|
||||||
'pr_state': pr.get('state'),
|
|
||||||
'pr_merged': pr.get('merged'),
|
|
||||||
'pr_head': pr.get('head', {}).get('sha'),
|
|
||||||
'pr_base': pr.get('base', {}).get('ref'),
|
|
||||||
'reviews': [
|
|
||||||
{
|
|
||||||
'id': item.get('id'),
|
|
||||||
'reviewer_id': item.get('user', {}).get('id'),
|
|
||||||
'state': item.get('state'),
|
|
||||||
'commit_id': item.get('commit_id'),
|
|
||||||
'dismissed': item.get('dismissed'),
|
|
||||||
}
|
|
||||||
for item in reviews
|
|
||||||
],
|
|
||||||
'governance_capability_executed': False,
|
|
||||||
'protection_mutation': False,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def handle(request):
|
|
||||||
if not isinstance(request, dict) or set(request) != {'operation'}:
|
|
||||||
raise RuntimeError('REQUEST_SCHEMA_INVALID')
|
|
||||||
if request['operation'] == 'DRY_RUN_SELF_TEST':
|
|
||||||
return self_test()
|
|
||||||
if request['operation'] == 'PUBLISH_SEMANTIC_RECONCILIATION':
|
|
||||||
return publish()
|
|
||||||
if request['operation'] == 'READ_SEMANTIC_RECONCILIATION_STATUS':
|
|
||||||
return read_publication_status()
|
|
||||||
raise RuntimeError('OPERATION_DENIED')
|
|
||||||
|
|
||||||
|
|
||||||
def serve(listener):
|
|
||||||
conn, _ = listener.accept()
|
|
||||||
with conn:
|
|
||||||
try:
|
|
||||||
data = conn.recv(1024 * 1024 + 1)
|
|
||||||
if len(data) > 1024 * 1024:
|
|
||||||
raise RuntimeError('REQUEST_TOO_LARGE')
|
|
||||||
result = handle(json.loads(data.decode()))
|
|
||||||
except Exception as exc:
|
|
||||||
result = {'status': 'FAIL', 'code': str(exc)}
|
|
||||||
conn.sendall((json.dumps(result, sort_keys=True, separators=(',', ':'), ensure_ascii=False) + '\n').encode())
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
if os.environ.get('LISTEN_FDS') != '1':
|
|
||||||
raise SystemExit('SOCKET_ACTIVATION_REQUIRED')
|
|
||||||
listener = socket.fromfd(3, socket.AF_UNIX, socket.SOCK_STREAM)
|
|
||||||
serve(listener)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
main()
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user