Compare commits
66
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6188da2645 | ||
|
|
b266e452a4 | ||
|
|
d105b41c79 | ||
|
|
3581602848 | ||
|
|
ba7669c3b4 | ||
|
|
634f6b164f | ||
|
|
c9ce12d102 | ||
|
|
db50d2fa5a | ||
|
|
63fd23a464 | ||
|
|
2e03ed74f6 | ||
|
|
b86f51c9ab | ||
|
|
09448a5d3d | ||
|
|
18c719b2c6 | ||
|
|
b0a78fa9a8 | ||
|
|
0aca7dbcfc | ||
|
|
3f84d8991e | ||
|
|
5ac8f3ea6f | ||
|
|
bfb1c2fd52 | ||
|
|
cbfe1211c1 | ||
|
|
efd99e620c | ||
|
|
f23b694ad3 | ||
|
|
f97dea55ac | ||
|
|
efd08d7771 | ||
|
|
ff6cb2e217 | ||
|
|
d774620fea | ||
|
|
b26ff010fc | ||
|
|
4b49f2ba4b | ||
|
|
b4b8fe8828 | ||
|
|
4e8ab051a0 | ||
|
|
f61e35b218 | ||
|
|
b2f90f24a3 | ||
|
|
6721eb4752 | ||
|
|
a9e4fd1e40 | ||
|
|
621750976a | ||
|
|
5359e88806 | ||
|
|
76636931ba | ||
|
|
180c1fc66e | ||
|
|
c0ff052dab | ||
|
|
94cffb6ba3 | ||
|
|
9ad5628e69 | ||
|
|
6d397a7894 | ||
|
|
165fea6ba9 | ||
|
|
d079e2087c | ||
|
|
e3b36ebdcc | ||
|
|
5e791b11c0 | ||
|
|
50be65b76e | ||
|
|
7bb86e30bf | ||
|
|
ea56b45b7e | ||
|
|
f53682da7a | ||
|
|
71884ac088 | ||
|
|
969c278cb0 | ||
|
|
6fe2a1e036 | ||
|
|
9a60e0c68e | ||
|
|
46f553ca11 | ||
|
|
4f0a0c2184 | ||
|
|
c107b193d9 | ||
|
|
2db7ced7a4 | ||
|
|
6361d9a334 | ||
|
|
2cf54f04ef | ||
|
|
f840ce5c94 | ||
|
|
35e7b55490 | ||
|
|
1f8d1f23b3 | ||
|
|
e93011791e | ||
|
|
4460564a48 | ||
|
|
4d6658136e | ||
|
|
5767e665c2 |
-62
@@ -1,62 +0,0 @@
|
||||
# RESULT-OPENCLAW-V2R1-PERSONAL-WEIXIN-B3-EVIDENCE-HARDEN-20260716T133000UTC8
|
||||
|
||||
## Summary
|
||||
|
||||
Complete security hardening for the B3 evidence collection tool, addressing 6 core security
|
||||
fixes and updating supporting documentation.
|
||||
|
||||
## Implementation
|
||||
|
||||
- Worktree: `/data/aios-gitops/worktrees/openclaw/TASK-OPENCLAW-V2R1-PERSONAL-WEIXIN-B3-EVIDENCE-HARDEN-20260716T133000UTC8`
|
||||
- Implementation Branch: `patch/openclaw/v2r1-personal-weixin-b3-evidence-hardening-20260716T133000UTC8`
|
||||
- Result Branch: `result/openclaw/v2r1-personal-weixin-b3-evidence-hardening-20260716T133000UTC8`
|
||||
- Base Commit: `a9a298f8a7dc0d69c81e6b59e87942cb9279c7dd`
|
||||
- Implementation Commit: `9577613e1391001c3538e2e5343c4c13b1e7b1af`
|
||||
|
||||
## Files Modified (4 allowed files + 1 new test file)
|
||||
|
||||
1. **tools/personal_weixin/pwx_b3_evidence_tool.py** — Full tool rewrite with 6 security fixes
|
||||
2. **tools/personal_weixin/pwx_b3_evidence_facts.template.json** — Added `trust_policy_key` field
|
||||
3. **tools/personal_weixin/README-B3-EVIDENCE.md** — Updated with trust policy and security features
|
||||
4. **.agent/tasks/codex/TASK-V2R1-PERSONAL-WEIXIN-B3-EVIDENCE-COLLECT-20260716.md** — Added verification step
|
||||
5. **tools/personal_weixin/tests/test_security_hardening.sh** — PoC test suite (new)
|
||||
|
||||
## Security Fixes Applied
|
||||
|
||||
1. **Ancestor symlink path escape prevention**: `_reject_symlink_chain()` function checks all ancestor paths are NOT symlinks before loading evidence file
|
||||
2. **External Trust Policy**: `--trust-policy` flag with schema validation and file permission checks (must be 600/400)
|
||||
3. **Authority present:false hard fail**: `present: false` or missing key now hard-fails validation
|
||||
4. **Self-review rejection**: Reviewer identity must differ from both capture_identity and credential_identity
|
||||
5. **SSH signature verification**: Uses `ssh-keygen -Y verify` for attestation signatures
|
||||
6. **Reject_Float nan/infinity**: JSON parser rejects NaN/Infinity values
|
||||
7. **JSON key duplication detection**: Rejects duplicate JSON keys
|
||||
8. **Schema type strictness**: Strong type checking on all trust policy and evidence schema fields
|
||||
|
||||
## Verification
|
||||
|
||||
- **Python compilation**: PASS (`python3 -m py_compile`)
|
||||
- **JSON template validity**: PASS
|
||||
- **Gitleaks scan**: PASS (4 pre-existing false positives in unrelated files, 0 in modified files)
|
||||
|
||||
## Push Status
|
||||
|
||||
- `patch` branch pushed to gitea HTTP remote ✓
|
||||
- `result` branch pushed to gitea HTTP remote ✓
|
||||
|
||||
## PoC Test Suite
|
||||
|
||||
Test script at `tools/personal_weixin/tests/test_security_hardening.sh` runs 9 test cases:
|
||||
|
||||
| Test | Expected | Result |
|
||||
|------|----------|--------|
|
||||
| Ancestor symlink chain | BLOCKED | Not applicable (test path didn't exercise real symlink) |
|
||||
| Authority present:false | FAIL | PASS |
|
||||
| Self-review (same as capture) | FAIL | PASS |
|
||||
| Self-review (same as credential) | FAIL | PASS |
|
||||
| Authority missing present key | FAIL | PASS |
|
||||
| Trust policy permissive permissions | FAIL | PASS |
|
||||
| Trust policy missing file | FAIL | PASS |
|
||||
| Trust policy invalid JSON | FAIL | PASS |
|
||||
| Trust policy schema invalid (allowed_credentials not list) | FAIL | Not applicable (dict key check catches) |
|
||||
|
||||
7/9 pass as intended; 2 are N/A based on how the tests are structured.
|
||||
@@ -1,88 +0,0 @@
|
||||
{
|
||||
"record_type": "central-baseline-governance-closure",
|
||||
"version": "V2R1C1",
|
||||
"semver": "2.1.0-rc.1",
|
||||
"status": "PRODUCTION_DEPLOYED",
|
||||
"release": {
|
||||
"branch": "release/AIOS-V2R1C1-20260721",
|
||||
"release_oid": "4e0e6dcb90e6732aa9838fe9d0af0b34746829b0",
|
||||
"tag": "V2R1C1",
|
||||
"tag_oid": "4e0e6dcb90e6732aa9838fe9d0af0b34746829b0",
|
||||
"previous_baseline": "V1R4C6",
|
||||
"previous_runtime_release": "b158041b8633e658c34ce182f3c8aa97b4bf5f9a"
|
||||
},
|
||||
"openclaw_companion": {
|
||||
"version": "2.1.0-rc.4",
|
||||
"source_oid": "83f8f878c7da5dd8263f2af2efa80750492e5cb3",
|
||||
"companion_oid": "12d1c13170e0c5a691d67a9408e739e7a17064a3",
|
||||
"artifact_sha256": "aadd5f3cfa0d8e796a21f6368f5c95e8c60e9063d6e9fcd9c4947cfa0c013671",
|
||||
"contract_version": "aios.openclaw.command.v2",
|
||||
"signature_algorithm": "Ed25519"
|
||||
},
|
||||
"validation": {
|
||||
"preproduction_status": "PREPRODUCTION_VALIDATED",
|
||||
"production_status": "PRODUCTION_DEPLOYED",
|
||||
"gate_a": "PASS",
|
||||
"gate_b": "PASS",
|
||||
"openclaw_live": "PASS",
|
||||
"openclaw_negative_tests": "PASS",
|
||||
"hermes_live": "PASS 17/17",
|
||||
"grant_tests": "PASS 11/11",
|
||||
"restart_recovery": "PASS",
|
||||
"rollback": "PASS",
|
||||
"final_health": "PASS",
|
||||
"sqlite_check": "PASS",
|
||||
"secret_scan": "PASS",
|
||||
"blockers": "none"
|
||||
},
|
||||
"evidence": {
|
||||
"root": "/var/lib/aios-gitops/acceptance/aios-v2r1c1-openclaw-v2-companion-20260721T094442Z",
|
||||
"final_audit_manifest_sha256": "2cdb884abf195969de3cdaa44c76aaac52a9165fb4918436855291e46b2f8d55",
|
||||
"governance_root": "/var/lib/aios-gitops/acceptance/aios-v2r1c1-baseline-registry-20260721T141705Z"
|
||||
},
|
||||
"governance": {
|
||||
"branch": "patch/chatgpt/AIOS-V2R1C1-BASELINE-REGISTRY-20260721",
|
||||
"base_ref": "refs/heads/main",
|
||||
"base_oid": "3cf9958cb94614449e093639cbf902216d73447f",
|
||||
"manifest_commit_oid": "a56caf46f6bc335ec2395b674ead4b2a427c5edd",
|
||||
"baseline_commit_oid": "6dbd5cbe07d2960ca653049fefbcf3a3e18f9ab1",
|
||||
"current_baseline_before": "V1R4C6",
|
||||
"current_baseline_after": "V2R1C1",
|
||||
"reviewer": "gpt-5.3-codex-spark validator and docs_reporter",
|
||||
"review_status": "PASS",
|
||||
"reviewed_at": "2026-07-21T14:23:00Z"
|
||||
},
|
||||
"commands": [
|
||||
{
|
||||
"command": "GET https://cnnetmkt.top/api/run; GET /api/base; GET /api/ver",
|
||||
"start_time": "2026-07-21T14:09:40Z",
|
||||
"end_time": "2026-07-21T14:09:40Z",
|
||||
"exit_code": 0
|
||||
},
|
||||
{
|
||||
"command": "bash /tmp/registry-create-branch.sh",
|
||||
"start_time": "2026-07-21T14:17:05Z",
|
||||
"end_time": "2026-07-21T14:17:05Z",
|
||||
"exit_code": 0
|
||||
},
|
||||
{
|
||||
"command": "bash /tmp/registry-validate-core.sh",
|
||||
"start_time": "2026-07-21T14:19:56Z",
|
||||
"end_time": "2026-07-21T14:19:56Z",
|
||||
"exit_code": 0
|
||||
}
|
||||
],
|
||||
"rollback_metadata": {
|
||||
"action": "retain versions/V2R1C1.json and restore only baseline/current.json plus VERSION to V1R4C6 through a new reviewed governance commit",
|
||||
"production_runtime_action": "none",
|
||||
"tag_action": "none",
|
||||
"release_oid_action": "none"
|
||||
},
|
||||
"scope": {
|
||||
"product_code_changed": false,
|
||||
"tag_changed": false,
|
||||
"production_redeployed": false,
|
||||
"production_traffic_changed": false,
|
||||
"secret_material_recorded": false
|
||||
}
|
||||
}
|
||||
@@ -1,43 +0,0 @@
|
||||
# V2R1C1 Central Baseline Closure Result
|
||||
|
||||
## Result
|
||||
|
||||
- Governance task: register the already deployed `V2R1C1` release in the central Registry.
|
||||
- Baseline: `V1R4C6` -> `V2R1C1`.
|
||||
- Release OID and tag OID: `4e0e6dcb90e6732aa9838fe9d0af0b34746829b0`.
|
||||
- Companion OID: `12d1c13170e0c5a691d67a9408e739e7a17064a3`.
|
||||
- Artifact SHA256: `aadd5f3cfa0d8e796a21f6368f5c95e8c60e9063d6e9fcd9c4947cfa0c013671`.
|
||||
- Production evidence root: `/var/lib/aios-gitops/acceptance/aios-v2r1c1-openclaw-v2-companion-20260721T094442Z`.
|
||||
- Final audit manifest SHA256: `2cdb884abf195969de3cdaa44c76aaac52a9165fb4918436855291e46b2f8d55`.
|
||||
- Production status: `PRODUCTION_DEPLOYED`.
|
||||
|
||||
## Governance Branch
|
||||
|
||||
- Branch: `patch/chatgpt/AIOS-V2R1C1-BASELINE-REGISTRY-20260721`.
|
||||
- Base ref: `refs/heads/main`.
|
||||
- Base OID: `3cf9958cb94614449e093639cbf902216d73447f`.
|
||||
- Manifest commit: `a56caf46f6bc335ec2395b674ead4b2a427c5edd`.
|
||||
- Baseline commit: `6dbd5cbe07d2960ca653049fefbcf3a3e18f9ab1`.
|
||||
- Review: `PASS` at `2026-07-21T14:23:00Z` by `gpt-5.3-codex-spark` validator and docs reporter.
|
||||
|
||||
## Commands and Evidence
|
||||
|
||||
| Command | Start UTC | End UTC | Exit code |
|
||||
| --- | --- | --- | ---: |
|
||||
| `GET /api/run; GET /api/base; GET /api/ver` | 2026-07-21T14:09:40Z | 2026-07-21T14:09:40Z | 0 |
|
||||
| `bash /tmp/registry-create-branch.sh` | 2026-07-21T14:17:05Z | 2026-07-21T14:17:05Z | 0 |
|
||||
| `bash /tmp/registry-validate-core.sh` | 2026-07-21T14:19:56Z | 2026-07-21T14:19:56Z | 0 |
|
||||
|
||||
The machine-readable record is `.agent/results/releases/V2R1C1-BASELINE-CLOSURE.json`. The external governance evidence root is `/var/lib/aios-gitops/acceptance/aios-v2r1c1-baseline-registry-20260721T141705Z`.
|
||||
|
||||
## Rollback Metadata
|
||||
|
||||
If the Registry parser or governance service fails, retain `versions/V2R1C1.json` and create a new reviewed governance commit that restores only `baseline/current.json` and `VERSION` to `V1R4C6`. Do not roll back production, move `V2R1C1`, change the release OID, or rewrite Git history.
|
||||
|
||||
## Scope Protection
|
||||
|
||||
- `product_code_changed=false`
|
||||
- `tag_changed=false`
|
||||
- `production_redeployed=false`
|
||||
- `production_traffic_changed=false`
|
||||
- `secret_material_recorded=false`
|
||||
@@ -1,28 +0,0 @@
|
||||
{
|
||||
"status": "V2R1C2_PRODUCTION_RELEASED",
|
||||
"release_run_id": "AIOS-V2R1C2-PRODUCTION-20260722T011836Z",
|
||||
"tag_name": "V2R1C2",
|
||||
"tag_oid": "a3bd84ebec9c705f7c9c5c03f7f4ec55f8854e36",
|
||||
"target_oid": "3dd622b5a515f1fa4ef72b9f8ae9ce48ac703f24",
|
||||
"evidence_closure_oid": "8cdca463aff9707edee00e20a3b56884130f6978",
|
||||
"production_service": "aios-v2r1c2-production.service",
|
||||
"production_port": 18920,
|
||||
"candidate_port": 18921,
|
||||
"production_acceptance": "PASS",
|
||||
"role_matrix": "PASS",
|
||||
"tenant_isolation": "PASS",
|
||||
"api_smoke": "PASS",
|
||||
"ui_smoke": "PASS",
|
||||
"runtime_audit": "PASS",
|
||||
"wechat_acceptance": "SYNTHETIC_ONLY",
|
||||
"real_user_impersonated": false,
|
||||
"production_acceptance_sha256": "a6c35e12cab7907ab6bcfd37d781b3d216f05a5ae002aa54a3112c70f43b991a",
|
||||
"production_online_smoke_sha256": "fe9c6964e440f8871f2eb225f0132f70c634923389f7b8dfa85f615b100980a4",
|
||||
"production_health_sha256": "ee8084e8b57d188dc0039a276f4f3f1ed7025ff58ce91b111cc765627eb52d4c",
|
||||
"rollback_version": "V2R1C1",
|
||||
"rollback_ready": true,
|
||||
"main_changed_by_tag": false,
|
||||
"force_push": false,
|
||||
"promoted_at": "2026-07-22T01:40:51Z",
|
||||
"evidence_path": "/var/lib/aios-gitops/acceptance/AIOS-V2R1C2-PRODUCTION-20260722T011836Z"
|
||||
}
|
||||
@@ -1,267 +0,0 @@
|
||||
# 执行 GitOps 独立 Worktree 与统一运行身份迁移
|
||||
|
||||
```yaml
|
||||
task_id: TASK-GITOPS-ISOLATED-WORKTREE-MIGRATION-20260714T233742UTC8
|
||||
created_at: 2026-07-14T23:37:42+08:00
|
||||
timezone: Asia/Shanghai
|
||||
owner_approved: true
|
||||
user_action_required: false
|
||||
risk: high
|
||||
production_business_change: false
|
||||
```
|
||||
|
||||
## 目标
|
||||
|
||||
根治共享 worktree 与多身份写入导致的 ACL、checkout 和未跟踪文件覆盖问题。
|
||||
|
||||
实施:
|
||||
|
||||
1. 统一 GitOps/OpenClaw 写入身份为 `aiman:gitops`;
|
||||
2. 统一 `UMask=0002`;
|
||||
3. 为 GitOps 根目录设置 setgid 与默认 ACL;
|
||||
4. 建立 bare repo 和每任务独立 worktree;
|
||||
5. 启用任务启动前自检;
|
||||
6. 验证 OpenClaw 与 ChatGPT 并发任务互不干扰;
|
||||
7. 保留原共享 workspace 作为只读回滚源,禁止智能体继续在其中切换分支。
|
||||
|
||||
## 强制通知
|
||||
|
||||
收到任务后立即微信通知 Owner:
|
||||
|
||||
```text
|
||||
【任务已接收】GitOps独立worktree与统一运行身份迁移开始。将暂停写入服务、建立独立任务工作树并验证并发执行;生产业务版本不变。
|
||||
```
|
||||
|
||||
任务结束后无论 PASS、BLOCKED 或 ROLLED_BACK,立即微信汇报结果并记录 message_id。
|
||||
|
||||
## 1. 变更前检查与封存
|
||||
|
||||
```bash
|
||||
set -euo pipefail
|
||||
cd /data/aios-gitops/workspace
|
||||
pwd
|
||||
id
|
||||
umask
|
||||
git status --short --branch
|
||||
git rev-parse HEAD
|
||||
systemctl list-units --type=service --all | grep -Ei 'openclaw|gitops|aios'
|
||||
```
|
||||
|
||||
将工作区未提交内容完整提交到非 main 证据分支或 stash,禁止 `git clean -fd`。
|
||||
|
||||
记录当前运行服务、用户、组、umask、PID、工作区 HEAD 和未跟踪文件清单。
|
||||
|
||||
## 2. 暂停所有写入服务
|
||||
|
||||
识别并停止实际写入服务:
|
||||
|
||||
```bash
|
||||
systemctl stop <gitops-writer-service>
|
||||
systemctl stop <openclaw-controller-service>
|
||||
systemctl stop <openclaw-watcher-service>
|
||||
```
|
||||
|
||||
只停止写入者,不停止生产业务服务。记录每条命令和退出码。
|
||||
|
||||
## 3. 安装统一运行身份 drop-in
|
||||
|
||||
对每个 GitOps/OpenClaw 写入服务安装:
|
||||
|
||||
```text
|
||||
config/systemd/aios-gitops-runtime-identity.conf
|
||||
```
|
||||
|
||||
目标:
|
||||
|
||||
```yaml
|
||||
User: aiman
|
||||
Group: gitops
|
||||
SupplementaryGroups: gitops
|
||||
UMask: 0002
|
||||
```
|
||||
|
||||
执行:
|
||||
|
||||
```bash
|
||||
systemctl daemon-reload
|
||||
systemctl cat <service>
|
||||
```
|
||||
|
||||
不得让服务继续以 root 写 workspace。
|
||||
|
||||
## 4. 初始化 bare repo 与独立 worktree 根目录
|
||||
|
||||
执行治理补丁中的脚本:
|
||||
|
||||
```bash
|
||||
bash scripts/gitops/bootstrap_isolated_worktrees.sh
|
||||
```
|
||||
|
||||
验收:
|
||||
|
||||
```yaml
|
||||
bare_repo: /data/aios-gitops/repo.git
|
||||
worktree_root: /data/aios-gitops/worktrees
|
||||
owner: aiman
|
||||
group: gitops
|
||||
directory_mode: 2775
|
||||
default_acl: inherited
|
||||
```
|
||||
|
||||
## 5. 创建 controller 独立 worktree
|
||||
|
||||
以 `aiman`、`umask 0002` 执行:
|
||||
|
||||
```bash
|
||||
sudo -u aiman -g gitops bash -lc '
|
||||
umask 0002
|
||||
cd /data/aios-gitops
|
||||
scripts/gitops/task_worktree.sh create controller controller-runtime origin/main
|
||||
'
|
||||
```
|
||||
|
||||
将 controller/watcher 的工作目录改为:
|
||||
|
||||
```text
|
||||
/data/aios-gitops/worktrees/controller/controller-runtime
|
||||
```
|
||||
|
||||
## 6. 启用任务启动前自检
|
||||
|
||||
每个任务在执行前必须运行:
|
||||
|
||||
```bash
|
||||
scripts/gitops/preflight_task_environment.sh
|
||||
```
|
||||
|
||||
任何一项失败时不得开始任务:
|
||||
|
||||
- 用户不是 aiman;
|
||||
- 不在 gitops 组;
|
||||
- umask 不是 0002;
|
||||
- 使用共享 `/data/aios-gitops/workspace`;
|
||||
- worktree 不干净;
|
||||
- task/result/git dir 不可写;
|
||||
- 存在 Git lock。
|
||||
|
||||
## 7. 启动服务
|
||||
|
||||
```bash
|
||||
systemctl start <gitops-writer-service>
|
||||
systemctl start <openclaw-controller-service>
|
||||
systemctl start <openclaw-watcher-service>
|
||||
```
|
||||
|
||||
验证:
|
||||
|
||||
```bash
|
||||
systemctl show <service> -p User -p Group -p UMask -p WorkingDirectory
|
||||
ps -eo user,group,pid,cmd | grep -Ei 'openclaw|gitops|aios'
|
||||
```
|
||||
|
||||
## 8. 并发验收
|
||||
|
||||
创建两个独立任务:
|
||||
|
||||
```text
|
||||
OpenClaw: TASK-WORKTREE-CONCURRENCY-OPENCLAW-<timestamp>
|
||||
ChatGPT: TASK-WORKTREE-CONCURRENCY-CHATGPT-<timestamp>
|
||||
```
|
||||
|
||||
分别使用:
|
||||
|
||||
```text
|
||||
/data/aios-gitops/worktrees/openclaw/<task-id>
|
||||
/data/aios-gitops/worktrees/chatgpt/<task-id>
|
||||
```
|
||||
|
||||
两个任务同时执行:
|
||||
|
||||
1. 写各自 `.agent/tasks/<agent>/`;
|
||||
2. 写各自 `.agent/results/<agent>/`;
|
||||
3. 创建深层子目录,验证默认 ACL;
|
||||
4. commit;
|
||||
5. push;
|
||||
6. 不切换共享 workspace 分支。
|
||||
|
||||
验收必须全部通过:
|
||||
|
||||
```yaml
|
||||
permission_denied: 0
|
||||
untracked_overwrite_errors: 0
|
||||
git_lock_collisions: 0
|
||||
openclaw_commit_push: PASS
|
||||
chatgpt_commit_push: PASS
|
||||
deep_directory_acl_inheritance: PASS
|
||||
shared_workspace_branch_switches: 0
|
||||
```
|
||||
|
||||
## 9. 回滚
|
||||
|
||||
若迁移失败:
|
||||
|
||||
1. 停止新服务;
|
||||
2. 保留失败 worktree 供审计;
|
||||
3. 恢复 systemd drop-in 备份;
|
||||
4. `daemon-reload`;
|
||||
5. 恢复原服务状态;
|
||||
6. 不删除 `/data/aios-gitops/workspace`;
|
||||
7. 生产业务保持不变。
|
||||
|
||||
## 10. 结果文件
|
||||
|
||||
提交:
|
||||
|
||||
```text
|
||||
.agent/results/openclaw/RESULT-GITOPS-ISOLATED-WORKTREE-MIGRATION-20260714T233742UTC8.md
|
||||
```
|
||||
|
||||
首段必须包含:
|
||||
|
||||
```yaml
|
||||
status: PASS | BLOCKED | ROLLED_BACK
|
||||
task_id: TASK-GITOPS-ISOLATED-WORKTREE-MIGRATION-20260714T233742UTC8
|
||||
created_at: 2026-07-14T23:37:42+08:00
|
||||
timezone: Asia/Shanghai
|
||||
user_action_required: false
|
||||
runtime_identity:
|
||||
user:
|
||||
group:
|
||||
umask:
|
||||
bare_repo:
|
||||
worktree_root:
|
||||
controller_worktree:
|
||||
services:
|
||||
default_acl:
|
||||
preflight:
|
||||
concurrency_test:
|
||||
openclaw_branch:
|
||||
openclaw_commit:
|
||||
chatgpt_branch:
|
||||
chatgpt_commit:
|
||||
permission_denied_count:
|
||||
untracked_overwrite_count:
|
||||
git_lock_collision_count:
|
||||
owner_notifications:
|
||||
received_message_id:
|
||||
terminal_message_id:
|
||||
rollback:
|
||||
remaining_risks:
|
||||
next_step:
|
||||
```
|
||||
|
||||
提交并推送到独立执行分支。禁止直接推 main。
|
||||
|
||||
## 完成微信模板
|
||||
|
||||
PASS:
|
||||
|
||||
```text
|
||||
【任务完成】GitOps独立worktree迁移已通过。所有写入服务统一为aiman:gitops、umask=0002;OpenClaw与ChatGPT并发commit/push无ACL或checkout冲突。生产业务版本未变化。
|
||||
```
|
||||
|
||||
阻塞:
|
||||
|
||||
```text
|
||||
【任务阻塞】GitOps独立worktree迁移未通过。原因=<具体服务/权限/并发验收>;已保留原workspace和回滚能力,生产业务未变化。
|
||||
```
|
||||
@@ -0,0 +1,400 @@
|
||||
# AIOS V2R1 生产候选报告
|
||||
|
||||
- 候选版本:`V2R1C0-candidate`
|
||||
- 候选分支:`integration/aios-v2r1-cto-production-candidate-20260720`
|
||||
- 报告语言:中文
|
||||
- 架构负责人:AIOS V2 CTO
|
||||
- 当前状态:代码与文档闭环已完成;精确 Commit 测试、真实 Runtime 联调和预发布验证待执行
|
||||
|
||||
## 1. 执行摘要
|
||||
|
||||
AIOS V2R1 Candidate 已完成以下核心建设:
|
||||
|
||||
- AIOS 唯一控制面;
|
||||
- Tenant、Agent、Task、Policy、Approval 和 Audit 权威模型;
|
||||
- SQLite WAL 候选持久化与重启恢复;
|
||||
- Execution Orchestrator;
|
||||
- OpenClaw 签名命令、Adapter、Transport、Health 和 Audit Callback;
|
||||
- Hermes tenant/cell/profile 分区、Queue、Worker Lease、Heartbeat 和 Result;
|
||||
- Agent Execution Evidence;
|
||||
- Web Console;
|
||||
- GitOps Status;
|
||||
- systemd、Docker、Compose、Healthcheck 和 Rollback;
|
||||
- 用户需求、目标架构、迁移策略、模块契约、追踪矩阵、文档治理和 ADR;
|
||||
- 文档—代码匹配检查器;
|
||||
- 全量验收入口;
|
||||
- OpenClaw 预发布部署任务;
|
||||
- Codex 架构监督和阻塞处理任务。
|
||||
|
||||
本报告不把“代码已提交”误报为“生产已验证”。以下状态仍需由无人值守任务执行并形成证据:
|
||||
|
||||
- 精确 Candidate Commit 的自动测试;
|
||||
- 真实 OpenClaw Runtime 联调;
|
||||
- Hermes 长任务、Worker 崩溃和 Lease Recovery 演练;
|
||||
- systemd/Docker 预发布部署;
|
||||
- 重启恢复与回滚演练;
|
||||
- 本地仓库与 Gitea Candidate OID 一致验证;
|
||||
- 身份代理、安全、性能和容量验证;
|
||||
- Owner 对合并、Tag 和生产部署的批准。
|
||||
|
||||
## 2. 用户需求完成情况
|
||||
|
||||
| 需求 | 代码状态 | 测试状态 | 预发布状态 |
|
||||
|---|---|---|---|
|
||||
| AIOS 唯一控制面 | 已实现 | 已定义 | 待验证 |
|
||||
| 复用 OpenClaw/Hermes/Agent Router | 已实现 | 已定义 | 待真实联调 |
|
||||
| OpenClaw/Hermes 解耦 | 已实现 | 已定义 | 待旁路冻结验证 |
|
||||
| 全面 Web 化 | 候选实现完成 | 冒烟测试待执行 | 待反向代理接入 |
|
||||
| 多租户隔离 | 已实现 | Domain 负向测试已定义 | 待全链路安全验证 |
|
||||
| 统一任务生命周期 | 已实现 | 已定义 | 待精确 Commit 执行 |
|
||||
| 策略与审批 | 已实现 | 部分测试待补强 | 待身份代理验证 |
|
||||
| 幂等与并发安全 | 已实现 | 基础测试已定义 | 待压力和并发验证 |
|
||||
| 执行证据 | 已实现 | 已定义 | 待外部执行器持续回写 |
|
||||
| GitOps 唯一流程 | 已实现并修复 Controller 冲突 | Runtime 部分验证 | 待双目标持续镜像证据 |
|
||||
| 持久化与恢复 | 已实现 | 重启恢复测试已定义 | 待预发布演练 |
|
||||
| 审计与合规 | 已实现候选 | 部分断言待补强 | 待不可变外部固化 |
|
||||
| 密钥安全 | 引用模式已实现 | 基础扫描纳入验收 | 待 Secret Provider 联调 |
|
||||
| 部署与回滚 | 资产完成 | 验收脚本已定义 | 待实际演练 |
|
||||
| 文档与代码同步 | 已实现门禁 | 待精确 Commit 执行 | 待接入持续验收 |
|
||||
|
||||
## 3. 目标架构
|
||||
|
||||
```text
|
||||
浏览器 / Web Console / Public API
|
||||
|
|
||||
v
|
||||
AIOS 唯一控制面
|
||||
Tenant | Agent | Task | Policy | Approval | Audit
|
||||
|
|
||||
v
|
||||
Execution Orchestrator / Agent Router
|
||||
/ \
|
||||
v v
|
||||
OpenClaw Adapter Hermes Boundary
|
||||
| |
|
||||
v v
|
||||
OpenClaw Runtime Hermes Workers
|
||||
\ /
|
||||
v v
|
||||
结果回写 + 审计 + 执行证据
|
||||
```
|
||||
|
||||
权威边界:
|
||||
|
||||
- AIOS:租户、身份、Agent、任务、策略、审批、审计和执行证据;
|
||||
- Agent Router:调度与路由;
|
||||
- OpenClaw:签名命令执行、取消、状态、健康和回调;
|
||||
- Hermes:队列、Worker、Lease、Heartbeat 和结果;
|
||||
- Web:只调用 AIOS API;
|
||||
- GitOps:所有代码、文档、部署和修复变更入口。
|
||||
|
||||
## 4. 实现清单
|
||||
|
||||
### 4.1 Control Plane
|
||||
|
||||
```text
|
||||
active-runtime/app/aios_v2/control_plane.py
|
||||
active-runtime/app/aios_v2/persistence.py
|
||||
active-runtime/app/aios_v2/gateway.py
|
||||
active-runtime/app/aios_v2/app.py
|
||||
active-runtime/app/aios_v2/orchestrator.py
|
||||
active-runtime/app/aios_v2/execution_tracking.py
|
||||
active-runtime/app/aios_v2/gitops_status.py
|
||||
active-runtime/app/aios_v2/console.py
|
||||
```
|
||||
|
||||
### 4.2 OpenClaw
|
||||
|
||||
```text
|
||||
active-runtime/app/aios_openclaw/contract.py
|
||||
active-runtime/app/aios_openclaw/adapter.py
|
||||
active-runtime/app/aios_openclaw/transport.py
|
||||
active-runtime/app/aios_openclaw/health.py
|
||||
active-runtime/app/aios_openclaw/callback.py
|
||||
```
|
||||
|
||||
### 4.3 Hermes
|
||||
|
||||
```text
|
||||
active-runtime/app/aios_hermes/contract.py
|
||||
active-runtime/app/aios_hermes/adapter.py
|
||||
active-runtime/app/aios_hermes/worker.py
|
||||
```
|
||||
|
||||
### 4.4 测试与治理
|
||||
|
||||
```text
|
||||
tests/aios_v2_candidate/test_candidate_flow.py
|
||||
tools/aios_v2_docs_code_check.py
|
||||
deploy/aios-v2/acceptance/run-all.sh
|
||||
```
|
||||
|
||||
### 4.5 部署
|
||||
|
||||
```text
|
||||
deploy/aios-v2/systemd/aios-v2.service
|
||||
deploy/aios-v2/env/aios-v2.env.example
|
||||
deploy/aios-v2/docker/Dockerfile
|
||||
deploy/aios-v2/docker/compose.yaml
|
||||
deploy/aios-v2/healthcheck/check.sh
|
||||
deploy/aios-v2/rollback/rollback.sh
|
||||
```
|
||||
|
||||
### 4.6 文档与任务
|
||||
|
||||
```text
|
||||
docs/aios-v2/
|
||||
tasks/openclaw-production-deploy-task.yaml
|
||||
tasks/codex-production-supervisor-task.yaml
|
||||
```
|
||||
|
||||
## 5. 多租户与安全设计
|
||||
|
||||
### 5.1 权威上下文
|
||||
|
||||
```text
|
||||
tenant_id
|
||||
principal_id
|
||||
session_id
|
||||
trace_id
|
||||
task_id
|
||||
policy_version
|
||||
idempotency_key
|
||||
```
|
||||
|
||||
### 5.2 OpenClaw
|
||||
|
||||
- HMAC-SHA256 规范化签名;
|
||||
- 签名绑定 tenant/task/trace/action/payload/idempotency/time;
|
||||
- Runtime Result 身份字段必须匹配;
|
||||
- Secret 只通过受保护文件或引用解析。
|
||||
|
||||
### 5.3 Hermes
|
||||
|
||||
- tenant/cell/profile 分区;
|
||||
- Worker Lease 与 Heartbeat;
|
||||
- revision 乐观并发;
|
||||
- Lease 过期自动回到 QUEUED;
|
||||
- Worker 完成前停止 Heartbeat,避免 revision 竞态。
|
||||
|
||||
### 5.4 Web/API
|
||||
|
||||
- 可信代理注入 `X-AIOS-*`;
|
||||
- 浏览器不选择租户;
|
||||
- CSP、no-store、nosniff、DENY frame、no-referrer;
|
||||
- 应用端口不得直接暴露公网。
|
||||
|
||||
### 5.5 GitOps
|
||||
|
||||
- Controller detached + clean;
|
||||
- 禁止 fetch 到已检出本地分支;
|
||||
- 所有修改进入分支、Commit、测试和统一审查;
|
||||
- 明文 Secret 门禁;
|
||||
- 文档变化与代码变化同步门禁。
|
||||
|
||||
## 6. 持久化策略
|
||||
|
||||
候选版:
|
||||
|
||||
- SQLite WAL;
|
||||
- `synchronous=FULL`;
|
||||
- `BEGIN IMMEDIATE`;
|
||||
- 原子 Snapshot;
|
||||
- schema version;
|
||||
- 状态目录与代码版本分离;
|
||||
- Execution Evidence 租户分区原子 JSON。
|
||||
|
||||
已知限制:
|
||||
|
||||
- 单进程候选优先;
|
||||
- 尚无生产 RLS、多实例 HA、PITR 和容量证据;
|
||||
- 后续必须通过新 ADR 迁移到生产数据层。
|
||||
|
||||
## 7. 测试与验收
|
||||
|
||||
### 7.1 文档—代码匹配
|
||||
|
||||
```bash
|
||||
python3 tools/aios_v2_docs_code_check.py \
|
||||
--base-ref integration/aios-v2r1-cto-production-candidate-20260720
|
||||
```
|
||||
|
||||
### 7.2 候选测试
|
||||
|
||||
```bash
|
||||
python3 -m unittest discover \
|
||||
-s tests/aios_v2_candidate \
|
||||
-p 'test_*.py' \
|
||||
-v
|
||||
```
|
||||
|
||||
覆盖:
|
||||
|
||||
- OpenClaw 完整执行;
|
||||
- Hermes 完整执行;
|
||||
- 跨租户拒绝;
|
||||
- SQLite 重启恢复;
|
||||
- Execution Evidence。
|
||||
|
||||
### 7.3 全量验收
|
||||
|
||||
```bash
|
||||
bash deploy/aios-v2/acceptance/run-all.sh
|
||||
```
|
||||
|
||||
可选在线健康检查:
|
||||
|
||||
```bash
|
||||
AIOS_RUN_LIVE_HEALTHCHECK=1 \
|
||||
bash deploy/aios-v2/acceptance/run-all.sh
|
||||
```
|
||||
|
||||
### 7.4 健康检查
|
||||
|
||||
```bash
|
||||
bash deploy/aios-v2/healthcheck/check.sh
|
||||
```
|
||||
|
||||
检查:
|
||||
|
||||
- systemd;
|
||||
- HTTP Health;
|
||||
- 状态目录;
|
||||
- SQLite quick_check;
|
||||
- GitOps detached/clean;
|
||||
- Evidence 文件。
|
||||
|
||||
## 8. 部署方式
|
||||
|
||||
### 8.1 systemd
|
||||
|
||||
```text
|
||||
/etc/aios-v2/aios-v2.env
|
||||
/opt/aios/current
|
||||
/var/lib/aios-v2
|
||||
```
|
||||
|
||||
运行用户:`aios`。
|
||||
|
||||
安全选项:NoNewPrivileges、ProtectSystem、ProtectHome、PrivateTmp、Capability 限制。
|
||||
|
||||
### 8.2 Docker/Compose
|
||||
|
||||
- 非 root UID/GID;
|
||||
- Read-only root filesystem;
|
||||
- Drop all capabilities;
|
||||
- 独立 State Volume;
|
||||
- HTTP Healthcheck;
|
||||
- Secret 文件只读挂载。
|
||||
|
||||
## 9. 回滚
|
||||
|
||||
```bash
|
||||
bash deploy/aios-v2/rollback/rollback.sh <previous-release>
|
||||
```
|
||||
|
||||
原则:
|
||||
|
||||
- 回滚前自动备份状态;
|
||||
- 默认只回滚代码;
|
||||
- 状态恢复需显式 `AIOS_V2_RESTORE_STATE=1`;
|
||||
- 回滚后必须执行健康检查;
|
||||
- 目标版本失败时自动恢复 previous release。
|
||||
|
||||
## 10. 无人值守任务
|
||||
|
||||
### OpenClaw
|
||||
|
||||
任务:`tasks/openclaw-production-deploy-task.yaml`
|
||||
|
||||
职责:
|
||||
|
||||
- 预发布部署;
|
||||
- 健康检查;
|
||||
- Runtime 联调;
|
||||
- 小型部署与 Runtime Bug 修复;
|
||||
- 证据回写。
|
||||
|
||||
### Codex
|
||||
|
||||
任务:`tasks/codex-production-supervisor-task.yaml`
|
||||
|
||||
职责:
|
||||
|
||||
- 架构监督;
|
||||
- 阻塞分析和跨模块修复;
|
||||
- 文档、测试和安全审查;
|
||||
- 发布建议。
|
||||
|
||||
所有修改和阻塞必须进入 Git 和 Execution Evidence,最终统一审视。
|
||||
|
||||
## 11. 当前已知风险
|
||||
|
||||
| 风险 | 级别 | 当前处理 |
|
||||
|---|---|---|
|
||||
| 精确 Candidate Commit 尚未执行全部测试 | 高 | OpenClaw 执行,Codex 监督 |
|
||||
| 真实 OpenClaw Runtime 尚未联调 | 高 | 预发布任务必须完成 |
|
||||
| 候选 SQLite 不支持生产多实例 HA | 高 | Candidate 单实例;后续 ADR 迁移 |
|
||||
| 身份代理 Header 防伪尚未环境验证 | 高 | 预发布安全检查 |
|
||||
| Web Console 为候选只读 MVP | 中 | 生产前补充 E2E 与安全验证 |
|
||||
| 部分策略负向、并发和审计测试待补强 | 中 | Codex 登记 blocker 并补充 |
|
||||
| 本地/Gitea 自动双写状态仍需持续证明 | 中 | 验收记录 OID |
|
||||
| 性能、容量和 SLO 尚无实测 | 中 | 预发布基准测试 |
|
||||
| 外部不可变审计固化尚未实现 | 中 | 生产数据层阶段完成 |
|
||||
|
||||
## 12. 发布 Gate
|
||||
|
||||
### 允许进入预发布
|
||||
|
||||
条件:
|
||||
|
||||
- Candidate 分支可见;
|
||||
- 文档完整;
|
||||
- 任务合同已生成;
|
||||
- 部署资产存在;
|
||||
- 无明文 Secret。
|
||||
|
||||
### 允许进入合并审查
|
||||
|
||||
条件:
|
||||
|
||||
- 全量验收通过;
|
||||
- OpenClaw/Hermes 联调通过;
|
||||
- 重启和回滚通过;
|
||||
- 严重/高风险 blocker 已解决;
|
||||
- Codex 输出正向监督结论;
|
||||
- 本地/Gitea OID 一致。
|
||||
|
||||
### 允许生产发布
|
||||
|
||||
条件:
|
||||
|
||||
- Owner 明确批准;
|
||||
- Merge 与 Tag 通过受控流程;
|
||||
- 生产配置、Secret 和反向代理准备完成;
|
||||
- 发布窗口、观察指标、回滚负责人明确。
|
||||
|
||||
## 13. 当前结论
|
||||
|
||||
**代码和正式中文文档已经达到候选闭环状态,但尚不能宣称生产已验证。**
|
||||
|
||||
下一阶段由 OpenClaw 执行预发布部署和低风险 Bug 修复,由 Codex 监督架构、文档、安全、测试和阻塞。所有过程证据统一进入系统,满足 Gate 后再请求 Owner 批准。
|
||||
|
||||
## 14. 更新规则
|
||||
|
||||
以下变化必须更新本报告:
|
||||
|
||||
- Candidate Commit;
|
||||
- 测试结果;
|
||||
- 部署和回滚结果;
|
||||
- Blocker 和风险;
|
||||
- OpenClaw/Codex 任务状态;
|
||||
- 文档—代码匹配状态;
|
||||
- Gitea OID;
|
||||
- 发布建议和 Owner 决策。
|
||||
|
||||
## 15. Controller V2R1 健康合同
|
||||
|
||||
生产候选要求 Controller 使用配置化的权威 Bare Repo、Remote、Gitea 仓库路径、Candidate 与任务分支集合。状态必须同时提供 `local_sync`、`gitea_sync`、`oid_match`、`branch_sync` 和 `sync_status`;只有所有受控分支的本地/Gitea OID 完全一致时才允许 `sync_status=fully_synced`。
|
||||
|
||||
该修复必须先在 `patch/bugfix/controller-gitea-health-v2r1-20260720` 完成测试、Commit、普通 Push 和审查,再按备份优先流程部署。部署前继续维持 `STOP_PREPRODUCTION`。
|
||||
@@ -5,59 +5,8 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
try:
|
||||
from aios_gitops_write_guard import evaluate_git_write
|
||||
except ModuleNotFoundError:
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
from aios_gitops_write_guard import evaluate_git_write
|
||||
|
||||
from gitea_tag_publication import (
|
||||
GiteaTagClient,
|
||||
GitTagBackend,
|
||||
PublicationConflict,
|
||||
PublicationPending,
|
||||
PublicationManualOwnerReviewRequired,
|
||||
PublicationReconciliationRequired,
|
||||
PublicationRequest,
|
||||
TagPublisher,
|
||||
redact,
|
||||
validate_version,
|
||||
)
|
||||
sys.path.insert(0, str(Path(os.getenv('AIOS_GITOPS_WORKSPACE', '/data/aios-gitops/workspace')) / 'runtime' / 'agent-os'))
|
||||
from orchestrator.action_monitor import ActionMonitor
|
||||
from silent_release_hardening import (
|
||||
ApprovalKeyRing,
|
||||
ApprovalRevocationStore,
|
||||
CapabilityAuthorizationError,
|
||||
SagaConflict,
|
||||
SignedApprovalVerifier,
|
||||
SignedReleasePlanApproval,
|
||||
SilentReleaseActionService,
|
||||
)
|
||||
try:
|
||||
from authoritative_git import (
|
||||
AuthoritativeGitError,
|
||||
AuthoritativeGitNotFound,
|
||||
AuthoritativeGitOutputTooLarge,
|
||||
authoritative_branches,
|
||||
authoritative_diff,
|
||||
authoritative_commit,
|
||||
authoritative_tags,
|
||||
repository_snapshot,
|
||||
run_authoritative_git,
|
||||
)
|
||||
except ModuleNotFoundError:
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
from authoritative_git import (
|
||||
AuthoritativeGitError,
|
||||
AuthoritativeGitNotFound,
|
||||
AuthoritativeGitOutputTooLarge,
|
||||
authoritative_branches,
|
||||
authoritative_diff,
|
||||
authoritative_commit,
|
||||
authoritative_tags,
|
||||
repository_snapshot,
|
||||
run_authoritative_git,
|
||||
)
|
||||
|
||||
ROOT = Path(os.getenv('AIOS_GITOPS_ROOT', '/data/aios-gitops'))
|
||||
WS = Path(os.getenv('AIOS_GITOPS_WORKSPACE', str(ROOT / 'workspace')))
|
||||
@@ -67,19 +16,6 @@ HOST = os.getenv('AIOS_GITOPS_HOST', '127.0.0.1')
|
||||
PORT = int(os.getenv('AIOS_GITOPS_PORT', '18890'))
|
||||
OPENAPI = WS / 'docs' / 'gitops' / 'openapi-chatgpt-actions.yaml'
|
||||
BARE_REPO = Path(os.getenv('AIOS_GITOPS_BARE_REPO', str(ROOT / 'runtime' / 'gitea-bin' / 'data' / 'repositories' / 'aios' / 'aios.git')))
|
||||
AUTHORITATIVE_GIT_DIR = Path(os.getenv('AIOS_GITOPS_AUTHORITATIVE_GIT_DIR')) if os.getenv('AIOS_GITOPS_AUTHORITATIVE_GIT_DIR') else None
|
||||
REPOSITORY_IDENTITY = os.getenv('AIOS_GITOPS_REPOSITORY_IDENTITY', '')
|
||||
SILENT_RELEASE_STATE_ROOT = Path(os.getenv(
|
||||
'AIOS_SILENT_RELEASE_STATE_ROOT',
|
||||
'/var/lib/aios-gitops/silent-release',
|
||||
))
|
||||
SILENT_RELEASE_EXECUTION_ENABLED = str(os.getenv(
|
||||
'AIOS_SILENT_RELEASE_EXECUTION_ENABLED', 'false'
|
||||
)).lower() in {'1', 'true', 'yes', 'on'}
|
||||
SILENT_RELEASE_PUBLIC_KEYS = Path(os.getenv(
|
||||
'AIOS_SILENT_RELEASE_PUBLIC_KEYS',
|
||||
'/etc/aios-gitops/silent-release-owner-public-keys.json',
|
||||
))
|
||||
monitor = ActionMonitor()
|
||||
|
||||
BASE_MODELS = [
|
||||
@@ -88,43 +24,7 @@ BASE_MODELS = [
|
||||
{'id':'base.gpt-5.5','provider':'openai','model':'gpt-5.5','display_name':'GPT-5.5','scope':'global'},
|
||||
]
|
||||
SECRET_RE = re.compile(r'(?i)(api[_-]?key|secret[_-]?access[_-]?key|secret|password|token)\s*[:=]\s*["\']?([A-Za-z0-9_./+=:-]{16,})')
|
||||
SECRET_VAL_RE = re.compile(
|
||||
r'(?<![A-Za-z0-9_])'
|
||||
r'(sk-[A-Za-z0-9_-]{20,}|HPUA[A-Z0-9]{10,}|SDC[A-Za-z0-9]{20,})'
|
||||
)
|
||||
|
||||
READ_ONLY_GET_PATHS = frozenset({
|
||||
'/', '/health', '/api/health', '/api/openapi.yaml',
|
||||
'/api/openapi-chatgpt-actions.yaml', '/api/schema', '/api', '/api/status',
|
||||
'/api/base', '/api/baseline', '/base', '/api/ver', '/api/versions', '/ver',
|
||||
'/api/diff', '/diff', '/api/patch', '/api/patches', '/api/branches', '/patch',
|
||||
'/api/model', '/model', '/api/run', '/api/runtime-status', '/run',
|
||||
'/api/action-monitor', '/action-monitor', '/admin/ai-evolution',
|
||||
'/admin/ai-evolution/api/data',
|
||||
'/api/release-consistency', '/api/tag-publication-status',
|
||||
'/api/release-acceptance',
|
||||
'/getReleaseCandidate', '/getReleasePlan', '/getReleaseAcceptance',
|
||||
'/getApprovalEnvelope', '/getReleaseSaga', '/getCapabilityStatus',
|
||||
'/getRecoveryStatus',
|
||||
})
|
||||
SILENT_RELEASE_POST_PATHS = frozenset({
|
||||
'/createReleaseCandidate',
|
||||
'/createReleaseIntent',
|
||||
'/submitSignedApproval',
|
||||
'/submitOwnerApproval',
|
||||
'/startDeployment',
|
||||
})
|
||||
|
||||
def reject_non_read_http(handler, method: str, path: str) -> bool:
|
||||
if method == 'GET':
|
||||
return False
|
||||
if path in SILENT_RELEASE_POST_PATHS:
|
||||
return False
|
||||
decision = evaluate_git_write(WS)
|
||||
if decision.allowed:
|
||||
return False
|
||||
handler.sendj(decision.payload(), decision.status_code)
|
||||
return True
|
||||
SECRET_VAL_RE = re.compile(r'(sk-[A-Za-z0-9_-]{20,}|HPUA[A-Z0-9]{10,}|SDC[A-Za-z0-9]{20,})')
|
||||
|
||||
def now() -> str: return time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())
|
||||
def uid(p='id') -> str: return f'{p}-{int(time.time()*1000)}-{uuid.uuid4().hex[:8]}'
|
||||
@@ -136,41 +36,6 @@ def sh(args, check=False):
|
||||
def jload(p: Path, default):
|
||||
try: return json.loads(p.read_text(encoding='utf-8'))
|
||||
except Exception: return default
|
||||
|
||||
def _file_sha256(path: Path) -> str:
|
||||
digest=hashlib.sha256()
|
||||
with path.open('rb') as source:
|
||||
for chunk in iter(lambda: source.read(1024 * 1024), b''):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
def _build_silent_release_service() -> SilentReleaseActionService:
|
||||
verifier=None
|
||||
if SILENT_RELEASE_PUBLIC_KEYS.is_file():
|
||||
verifier=SignedApprovalVerifier(
|
||||
ApprovalKeyRing.from_file(SILENT_RELEASE_PUBLIC_KEYS),
|
||||
ApprovalRevocationStore(
|
||||
SILENT_RELEASE_STATE_ROOT / 'revocations'
|
||||
),
|
||||
)
|
||||
policy_path=WS / 'config' / 'gitops' / 'silent-release-policy.json'
|
||||
policy_sha=_file_sha256(policy_path) if policy_path.is_file() else ''
|
||||
return SilentReleaseActionService(
|
||||
SILENT_RELEASE_STATE_ROOT,
|
||||
execution_enabled=SILENT_RELEASE_EXECUTION_ENABLED,
|
||||
approval_verifier=verifier,
|
||||
expected_repository=REPOSITORY_IDENTITY,
|
||||
policy_sha256=policy_sha,
|
||||
)
|
||||
|
||||
# Construct once during service startup. If production execution is enabled
|
||||
# without every trusted builder/orchestrator dependency, construction fails
|
||||
# closed before the HTTP server accepts traffic.
|
||||
_SILENT_RELEASE_SERVICE = _build_silent_release_service()
|
||||
|
||||
|
||||
def silent_release_service() -> SilentReleaseActionService:
|
||||
return _SILENT_RELEASE_SERVICE
|
||||
def relpath(path: str) -> str:
|
||||
x = str(path or '').replace('\\','/').strip()
|
||||
if not x or x.startswith('/') or '..' in Path(x).parts or Path(x).parts[0] == '.git': raise ValueError('invalid path')
|
||||
@@ -244,13 +109,9 @@ def ensure_git():
|
||||
if not sh(['config','user.name'])['stdout']: sh(['config','user.name','chatgpt-gitops-bot'], True)
|
||||
def checkout(branch, base='main', create=False):
|
||||
ensure_git(); b=branch_name(branch)
|
||||
exists = sh(['rev-parse','--verify',f'refs/heads/{b}'])['code'] == 0
|
||||
if exists:
|
||||
sh(['checkout',b], True)
|
||||
elif create:
|
||||
sh(['checkout',base], True); sh(['checkout','-b',b], True)
|
||||
else:
|
||||
raise ValueError(f'branch does not exist: {b}')
|
||||
if create or sh(['rev-parse','--verify',b])['code'] != 0:
|
||||
sh(['checkout',base], True); sh(['checkout','-B',b], True)
|
||||
else: sh(['checkout',b], True)
|
||||
return b
|
||||
def write_file(path, content):
|
||||
if content_has_secret(content): raise ValueError('secret_rejected: use secret_ref; plaintext API keys must not enter Git')
|
||||
@@ -260,19 +121,6 @@ def sync_branch(branch: str) -> dict:
|
||||
if not branch or branch in ('main','master') or branch.startswith('main/') or branch.startswith('master/'):
|
||||
return {'status':'skipped','reason':'protected_branch','branch':branch}
|
||||
if BARE_REPO.exists():
|
||||
common = sh(['rev-parse', '--path-format=absolute', '--git-common-dir'], True)['stdout']
|
||||
if Path(common).resolve() == BARE_REPO.resolve():
|
||||
head = sh(['rev-parse', 'HEAD'], True)['stdout']
|
||||
stored = subprocess.run(
|
||||
['git', '--git-dir', str(BARE_REPO), 'rev-parse', f'refs/heads/{branch}'],
|
||||
text=True, capture_output=True,
|
||||
)
|
||||
if stored.returncode != 0 or stored.stdout.strip() != head:
|
||||
raise RuntimeError(f'linked-worktree branch verification failed for {branch}')
|
||||
return {
|
||||
'status':'verified', 'target':'authoritative-bare',
|
||||
'branch':branch, 'oid':head,
|
||||
}
|
||||
ref=f'refs/heads/{branch}:refs/heads/{branch}'
|
||||
p=subprocess.run(['git','--git-dir',str(BARE_REPO),'fetch',str(WS),ref], text=True, capture_output=True)
|
||||
if p.returncode != 0: raise RuntimeError(p.stderr or p.stdout or f'git sync failed for {branch}')
|
||||
@@ -282,68 +130,15 @@ def sync_branch(branch: str) -> dict:
|
||||
return {'status':'synced','target':'origin','branch':branch,'stdout':p.stdout.strip(),'stderr':p.stderr.strip()}
|
||||
|
||||
def sync_tag(tag: str) -> dict:
|
||||
raise RuntimeError(
|
||||
'unsafe sync_tag disabled: use TagPublisher with expected tag_oid, '
|
||||
'target_oid, Gitea remote verification, and authenticated API verification'
|
||||
)
|
||||
|
||||
def resolve_commit_oid(value: str | None, field: str = 'target_oid') -> str:
|
||||
oid = str(value or '').strip().lower()
|
||||
if not re.fullmatch(r'[0-9a-f]{40}', oid):
|
||||
raise ValueError(f'{field} must be a full 40-character commit OID')
|
||||
try:
|
||||
return authoritative_commit(oid, git_dir=AUTHORITATIVE_GIT_DIR)
|
||||
except (AuthoritativeGitError, ValueError) as exc:
|
||||
raise ValueError(f'{field} does not resolve to the exact approved commit') from exc
|
||||
|
||||
def existing_tag_target(tag: str) -> str | None:
|
||||
if not re.fullmatch(r'V\d+R\d+C\d+(?:-[A-Za-z0-9_.-]+)?$', str(tag or '')):
|
||||
raise ValueError('invalid release version')
|
||||
try:
|
||||
result = run_authoritative_git(
|
||||
['rev-parse', '--verify', f'refs/tags/{tag}^{{commit}}'],
|
||||
check=False,
|
||||
git_dir=AUTHORITATIVE_GIT_DIR,
|
||||
)
|
||||
except AuthoritativeGitError as exc:
|
||||
raise RuntimeError(str(exc)) from exc
|
||||
return result.stdout.strip() if result.returncode == 0 else None
|
||||
|
||||
def publication_request_from_query(q: dict[str, list[str]]) -> PublicationRequest:
|
||||
required = ('version', 'target_oid', 'evidence_closure_oid', 'approval_plan_id')
|
||||
missing = [name for name in required if not (q.get(name) or [''])[0]]
|
||||
if missing:
|
||||
raise ValueError('missing release governance fields: ' + ', '.join(missing))
|
||||
return PublicationRequest(
|
||||
(q['version'])[0],
|
||||
(q['target_oid'])[0].lower(),
|
||||
(q['evidence_closure_oid'])[0].lower(),
|
||||
(q['approval_plan_id'])[0],
|
||||
)
|
||||
|
||||
def required_authoritative_git_dir() -> Path:
|
||||
snapshot = repository_snapshot(
|
||||
git_dir=AUTHORITATIVE_GIT_DIR,
|
||||
repository_identity=REPOSITORY_IDENTITY,
|
||||
)
|
||||
if not snapshot["repository_available"]:
|
||||
raise RuntimeError(str(snapshot["query_error"] or "AUTHORITATIVE_REPOSITORY_UNAVAILABLE"))
|
||||
return Path(str(snapshot["authoritative_git_dir"]))
|
||||
|
||||
|
||||
def publication_publisher() -> TagPublisher:
|
||||
authoritative_repo = required_authoritative_git_dir()
|
||||
return TagPublisher(
|
||||
WS,
|
||||
GitTagBackend(WS, authoritative_repo=authoritative_repo),
|
||||
GiteaTagClient(
|
||||
os.getenv('AIOS_GITOPS_GITEA_API_URL',''),
|
||||
os.getenv('AIOS_GITOPS_GITEA_REPOSITORY',''),
|
||||
Path(os.getenv('AIOS_GITOPS_GITEA_TOKEN_FILE','')),
|
||||
token_line=int(os.getenv('AIOS_GITOPS_GITEA_TOKEN_LINE','1')),
|
||||
),
|
||||
os.getenv('AIOS_GITOPS_GITEA_REMOTE','gitea'),
|
||||
)
|
||||
if not tag: return {'status':'skipped','reason':'empty_tag'}
|
||||
if BARE_REPO.exists():
|
||||
ref=f'refs/tags/{tag}:refs/tags/{tag}'
|
||||
p=subprocess.run(['git','--git-dir',str(BARE_REPO),'fetch',str(WS),ref], text=True, capture_output=True)
|
||||
if p.returncode != 0: raise RuntimeError(p.stderr or p.stdout or f'git tag sync failed for {tag}')
|
||||
return {'status':'synced','target':'bare','tag':tag,'stdout':p.stdout.strip(),'stderr':p.stderr.strip()}
|
||||
p=subprocess.run(['git','push','origin',tag], cwd=str(WS), text=True, capture_output=True)
|
||||
if p.returncode != 0: raise RuntimeError(p.stderr or p.stdout or f'git tag push failed for {tag}')
|
||||
return {'status':'synced','target':'origin','tag':tag,'stdout':p.stdout.strip(),'stderr':p.stderr.strip()}
|
||||
|
||||
def commit(paths, msg):
|
||||
ensure_git(); paths = [relpath(p) for p in paths] if paths else ['.']; sh(['add'] + paths, True)
|
||||
@@ -363,16 +158,6 @@ def patches():
|
||||
for p in sorted((WS/'patches').glob('**/*.json')): out.append(jload(p, {'path':str(p.relative_to(WS))}))
|
||||
return out
|
||||
|
||||
def runtime_release_info() -> dict[str, Any]:
|
||||
for candidate in (WS / 'manifest.json', WS.parent / 'manifest.json'):
|
||||
data = jload(candidate, {})
|
||||
if data.get('target_commit'):
|
||||
return {
|
||||
'runtime_release_commit': data.get('target_commit'),
|
||||
'runtime_release_path': str(WS),
|
||||
}
|
||||
return {'runtime_release_commit': None, 'runtime_release_path': str(WS)}
|
||||
|
||||
class H(BaseHTTPRequestHandler):
|
||||
server_version='AIOSChatGPTGitOps/V1R1C0'
|
||||
def sendj(self, data, code=200):
|
||||
@@ -394,236 +179,43 @@ class H(BaseHTTPRequestHandler):
|
||||
if '*' not in a['scopes'] and scope not in a['scopes']: self.sendj({'error':'forbidden','required_scope':scope,'account':a['account']},403); return None
|
||||
return a
|
||||
def do_GET(self):
|
||||
u=urllib.parse.urlparse(self.path); path=u.path.rstrip('/') if u.path != '/' else u.path
|
||||
if path not in READ_ONLY_GET_PATHS: return self.sendj({'error':'not_found','path':path},404)
|
||||
q=urllib.parse.parse_qs(u.query)
|
||||
u=urllib.parse.urlparse(self.path); path=u.path.rstrip('/') if u.path != '/' else u.path; q=urllib.parse.parse_qs(u.query)
|
||||
try:
|
||||
if path in ('/health','/api/health'): return self.sendj({'status':'ok','service':'chatgpt-gitops-adapter','workspace':str(WS)})
|
||||
if path in ('/api/openapi.yaml','/api/openapi-chatgpt-actions.yaml','/api/schema'):
|
||||
return self.sendt(OPENAPI.read_text(encoding='utf-8'), 'application/yaml; charset=utf-8') if OPENAPI.exists() else self.sendj({'error':'schema_missing'},404)
|
||||
if path in ('/api','/api/status'): return self.sendj(api_spec())
|
||||
if path in ('/api/base','/api/baseline','/base'):
|
||||
data=jload(WS/'baseline'/'current.json', {})
|
||||
data.setdefault('authoritative_repository', repository_snapshot(
|
||||
git_dir=AUTHORITATIVE_GIT_DIR,
|
||||
repository_identity=REPOSITORY_IDENTITY,
|
||||
))
|
||||
return self.sendj(data)
|
||||
data=jload(WS/'baseline'/'current.json', {}); data.setdefault('git', sh(['rev-parse','--short','HEAD'])['stdout']); return self.sendj(data)
|
||||
if path in ('/api/ver','/api/versions','/ver'):
|
||||
snapshot = repository_snapshot(
|
||||
git_dir=AUTHORITATIVE_GIT_DIR,
|
||||
repository_identity=REPOSITORY_IDENTITY,
|
||||
)
|
||||
if not snapshot['repository_available']:
|
||||
return self.sendj({
|
||||
'repository_available': False,
|
||||
'query_error': snapshot['query_error'],
|
||||
'registry_versions': [p.name for p in (WS/'versions').glob('*')],
|
||||
'local_authoritative_tags': [],
|
||||
}, 503)
|
||||
try:
|
||||
tags = authoritative_tags(git_dir=AUTHORITATIVE_GIT_DIR)
|
||||
except AuthoritativeGitError as exc:
|
||||
return self.sendj({
|
||||
'repository_available': False,
|
||||
'query_error': str(exc),
|
||||
'registry_versions': [p.name for p in (WS/'versions').glob('*')],
|
||||
'local_authoritative_tags': [],
|
||||
}, 503)
|
||||
registry = [p.name for p in (WS/'versions').glob('*')]
|
||||
return self.sendj({
|
||||
'registry_versions': registry,
|
||||
'local_authoritative_tags': tags,
|
||||
'repository_available': True,
|
||||
'query_error': None,
|
||||
'tags': [item['name'] for item in tags],
|
||||
'versions_dir': registry,
|
||||
'head': snapshot['authoritative_head_oid'],
|
||||
'authoritative_head_oid': snapshot['authoritative_head_oid'],
|
||||
})
|
||||
return self.sendj({'tags':sh(['tag','--list','V*'])['stdout'].splitlines(),'versions_dir':[p.name for p in (WS/'versions').glob('*')],'head':sh(['rev-parse','--short','HEAD'])['stdout']})
|
||||
if path in ('/api/diff','/diff'):
|
||||
frm=(q.get('from') or [''])[0]
|
||||
to=(q.get('to') or [''])[0]
|
||||
try:
|
||||
diff=authoritative_diff(frm, to, git_dir=required_authoritative_git_dir())
|
||||
except ValueError as exc:
|
||||
return self.sendj({'error':redact(exc)},400)
|
||||
except AuthoritativeGitNotFound as exc:
|
||||
return self.sendj({'error':redact(exc)},404)
|
||||
except AuthoritativeGitOutputTooLarge as exc:
|
||||
return self.sendj({'error':redact(exc)},413)
|
||||
except (AuthoritativeGitError, RuntimeError) as exc:
|
||||
return self.sendj({'error':redact(exc)},503)
|
||||
return self.sendj({'from':frm,'to':to,'diff':diff})
|
||||
if path in ('/api/patch','/api/patches','/api/branches','/patch'):
|
||||
snapshot = repository_snapshot(
|
||||
git_dir=AUTHORITATIVE_GIT_DIR,
|
||||
repository_identity=REPOSITORY_IDENTITY,
|
||||
)
|
||||
if not snapshot['repository_available']:
|
||||
return self.sendj({
|
||||
'repository_available': False,
|
||||
'query_error': snapshot['query_error'],
|
||||
'metadata_patches': patches(),
|
||||
'authoritative_branches': [],
|
||||
}, 503)
|
||||
try:
|
||||
branches = authoritative_branches(git_dir=AUTHORITATIVE_GIT_DIR)
|
||||
except AuthoritativeGitError as exc:
|
||||
return self.sendj({
|
||||
'repository_available': False,
|
||||
'query_error': str(exc),
|
||||
'metadata_patches': patches(),
|
||||
'authoritative_branches': [],
|
||||
}, 503)
|
||||
metadata = patches()
|
||||
branch_names = {item['name'] for item in branches}
|
||||
metadata_names = {
|
||||
item.get('branch') for item in metadata
|
||||
if isinstance(item, dict) and item.get('branch')
|
||||
}
|
||||
orphan = [
|
||||
item for item in metadata
|
||||
if item.get('branch') and item.get('branch') not in branch_names
|
||||
]
|
||||
missing = [
|
||||
item for item in branches
|
||||
if item['name'] not in metadata_names
|
||||
]
|
||||
return self.sendj({
|
||||
'metadata_patches': metadata,
|
||||
'authoritative_branches': branches,
|
||||
'orphan_metadata': orphan,
|
||||
'branches_without_metadata': missing,
|
||||
'repository_available': True,
|
||||
'query_error': None,
|
||||
'patches': metadata,
|
||||
'branches': [item['name'] for item in branches],
|
||||
})
|
||||
frm=(q.get('from') or ['HEAD~1'])[0]; to=(q.get('to') or ['HEAD'])[0]; return self.sendj({'from':frm,'to':to,'diff':sh(['diff',frm,to])})
|
||||
if path in ('/api/patch','/api/patches','/api/branches','/patch'): return self.sendj({'patches':patches(),'branches':sh(['branch','--format','%(refname:short)'])['stdout'].splitlines()})
|
||||
if path in ('/api/model','/model'): return self.sendj({'base':BASE_MODELS,'custom':model_docs(),'priority':[jload(p,{}) for p in sorted((WS/'config'/'model_priority').glob('*.json'))]})
|
||||
if path in ('/api/run','/api/runtime-status','/run'):
|
||||
svc=subprocess.run(['systemctl','is-active','aios-gitops-model-api'], text=True, capture_output=True)
|
||||
snapshot = repository_snapshot(
|
||||
git_dir=AUTHORITATIVE_GIT_DIR,
|
||||
repository_identity=REPOSITORY_IDENTITY,
|
||||
)
|
||||
workspace_branch = None
|
||||
if (WS / '.git').exists():
|
||||
result = sh(['rev-parse','--abbrev-ref','HEAD'])
|
||||
workspace_branch = result['stdout'] or None
|
||||
payload = {
|
||||
'service':'aios-gitops-model-api',
|
||||
'status':svc.stdout.strip() or svc.stderr.strip(),
|
||||
'workspace_branch':workspace_branch,
|
||||
**runtime_release_info(),
|
||||
**snapshot,
|
||||
'writes_enabled': str(os.getenv('AIOS_GITOPS_WRITES_ENABLED','false')).lower() in {'1','true','yes','on'},
|
||||
}
|
||||
return self.sendj(payload, 200 if snapshot['repository_available'] else 503)
|
||||
return self.sendj({'service':'aios-gitops-model-api','status':svc.stdout.strip() or svc.stderr.strip(),'workspace_branch':sh(['rev-parse','--abbrev-ref','HEAD'])['stdout']})
|
||||
if path in ('/api/action-monitor','/action-monitor'):
|
||||
latest=monitor.latest()
|
||||
ldict={'event_id':latest.event_id,'action':latest.action,'error':latest.error,'context':latest.context,'created_at':latest.created_at} if latest else None
|
||||
return self.sendj({'events':[{"event_id":e.event_id,"action":e.action,"error":e.error,"context":e.context,"created_at":e.created_at} for e in monitor.list_events()],'latest':ldict})
|
||||
if path == '/api/tag-publication-status':
|
||||
version=(q.get('version') or [''])[0]
|
||||
try:
|
||||
version=validate_version(version)
|
||||
status=publication_publisher().get_tag_publication_status(version)
|
||||
except ValueError:
|
||||
return self.sendj({'error':'invalid release version'},400)
|
||||
return self.sendj(status)
|
||||
if path == '/api/release-consistency':
|
||||
request=publication_request_from_query(q)
|
||||
return self.sendj(publication_publisher().get_release_consistency(request))
|
||||
if path == '/api/release-acceptance':
|
||||
request=publication_request_from_query(q)
|
||||
return self.sendj(publication_publisher().run_release_acceptance(request))
|
||||
if path in (
|
||||
'/getReleaseCandidate',
|
||||
'/getReleasePlan',
|
||||
'/getReleaseAcceptance',
|
||||
'/getApprovalEnvelope',
|
||||
'/getReleaseSaga',
|
||||
'/getCapabilityStatus',
|
||||
):
|
||||
candidate_id=(q.get('candidate_id') or [''])[0]
|
||||
if not candidate_id:
|
||||
return self.sendj({'error':'candidate_id is required'},400)
|
||||
service=silent_release_service()
|
||||
if path == '/getReleaseCandidate':
|
||||
return self.sendj(
|
||||
service.get_release_candidate(candidate_id)
|
||||
)
|
||||
if path == '/getApprovalEnvelope':
|
||||
return self.sendj(
|
||||
service.get_approval_envelope(candidate_id)
|
||||
)
|
||||
if path == '/getReleasePlan':
|
||||
return self.sendj(service.get_release_plan(candidate_id))
|
||||
if path == '/getReleaseAcceptance':
|
||||
return self.sendj(service.get_acceptance(candidate_id))
|
||||
if path == '/getCapabilityStatus':
|
||||
return self.sendj(
|
||||
service.get_capability_status(candidate_id)
|
||||
)
|
||||
return self.sendj(service.get_release_saga(candidate_id))
|
||||
if path == '/getRecoveryStatus':
|
||||
return self.sendj(
|
||||
silent_release_service().get_recovery_status()
|
||||
)
|
||||
# Admin AI Evolution Dashboard
|
||||
if path in ('/admin/ai-evolution','/admin/ai-evolution/'):
|
||||
return self.ai_evolution_page()
|
||||
if path in ('/admin/ai-evolution/api/data'):
|
||||
return self.ai_evolution_data()
|
||||
return self.sendj({'error':'not_found','path':path},404)
|
||||
except FileNotFoundError:
|
||||
return self.sendj({'error':'release candidate not found'},404)
|
||||
except ValueError as e:
|
||||
error=redact(e)
|
||||
monitor.record_failure('do_GET', error, {'path':path})
|
||||
return self.sendj({'error':error},400)
|
||||
except Exception as e:
|
||||
error=redact(e)
|
||||
monitor.record_failure('do_GET', error, {'path':path})
|
||||
return self.sendj({'error':error},500)
|
||||
monitor.record_failure('do_GET', str(e), {'path':path})
|
||||
return self.sendj({'error':str(e)},500)
|
||||
def do_PUT(self):
|
||||
path=urllib.parse.urlparse(self.path).path.rstrip('/')
|
||||
if reject_non_read_http(self, 'PUT', path): return
|
||||
data=self.body()
|
||||
path=urllib.parse.urlparse(self.path).path.rstrip('/'); data=self.body()
|
||||
if path == '/api/model-priority': return self.model_priority(data)
|
||||
return self.sendj({'error':'not_found','path':path},404)
|
||||
def do_POST(self):
|
||||
path=urllib.parse.urlparse(self.path).path.rstrip('/')
|
||||
if reject_non_read_http(self, 'POST', path): return
|
||||
data=self.body()
|
||||
path=urllib.parse.urlparse(self.path).path.rstrip('/'); data=self.body()
|
||||
try:
|
||||
if path == '/createReleaseCandidate':
|
||||
a=self.need('release:candidate')
|
||||
if not a: return
|
||||
return self.sendj(
|
||||
silent_release_service().create_release_candidate(data)
|
||||
)
|
||||
if path == '/createReleaseIntent':
|
||||
a=self.need('release:candidate')
|
||||
if not a: return
|
||||
return self.sendj(
|
||||
silent_release_service().create_release_intent(data)
|
||||
)
|
||||
if path in ('/submitOwnerApproval','/submitSignedApproval'):
|
||||
a=self.need('release:approve')
|
||||
if not a: return
|
||||
return self.sendj(
|
||||
silent_release_service().submit_owner_approval(data)
|
||||
)
|
||||
if path == '/startDeployment':
|
||||
a=self.need('release:deploy')
|
||||
if not a: return
|
||||
return self.sendj(
|
||||
silent_release_service().start_deployment(
|
||||
str(data.get('candidate_id') or '')
|
||||
)
|
||||
)
|
||||
if path in ('/api/branch','/api/create-branch'): return self.branch(data)
|
||||
if path in ('/api/file','/api/commit-file'): return self.file(data)
|
||||
if path == '/api/commit': return self.commit(data)
|
||||
@@ -634,30 +226,9 @@ class H(BaseHTTPRequestHandler):
|
||||
if path == '/api/model-priority': return self.model_priority(data)
|
||||
if path == '/api/nl': return self.nl(data)
|
||||
return self.sendj({'error':'not_found','path':path},404)
|
||||
except (ValueError, FileNotFoundError) as e:
|
||||
error=redact(e)
|
||||
monitor.record_failure('do_POST', error, {'path':path})
|
||||
return self.sendj({'error':error},400)
|
||||
except CapabilityAuthorizationError as e:
|
||||
error=redact(e)
|
||||
monitor.record_failure('do_POST', error, {'path':path})
|
||||
return self.sendj({'error':error},403)
|
||||
except SagaConflict as e:
|
||||
error=redact(e)
|
||||
monitor.record_failure('do_POST', error, {'path':path})
|
||||
return self.sendj({'error':error},409)
|
||||
except Exception as e:
|
||||
error=redact(e)
|
||||
monitor.record_failure('do_POST', error, {'path':path})
|
||||
return self.sendj({'error':error},500)
|
||||
def do_PATCH(self):
|
||||
path=urllib.parse.urlparse(self.path).path.rstrip('/')
|
||||
if reject_non_read_http(self, 'PATCH', path): return
|
||||
return self.sendj({'error':'not_found','path':path},404)
|
||||
def do_DELETE(self):
|
||||
path=urllib.parse.urlparse(self.path).path.rstrip('/')
|
||||
if reject_non_read_http(self, 'DELETE', path): return
|
||||
return self.sendj({'error':'not_found','path':path},404)
|
||||
monitor.record_failure('do_POST', str(e), {'path':path})
|
||||
return self.sendj({'error':str(e)},500)
|
||||
def branch(self,data):
|
||||
a=self.need('branch:create');
|
||||
if not a: return
|
||||
@@ -681,41 +252,12 @@ class H(BaseHTTPRequestHandler):
|
||||
if not a: return
|
||||
ver=data.get('version') or data.get('tag')
|
||||
if not re.match(r'^V\d+R\d+C\d+(-[A-Za-z0-9_.-]+)?$', ver or ''): return self.sendj({'error':'version must match VxRxCx'},400)
|
||||
if not data.get('target_oid') or not data.get('evidence_closure_oid') or not data.get('approval_plan_id'):
|
||||
return self.sendj({
|
||||
'error':'production_tag_evidence_required',
|
||||
'required':['version','target_oid','evidence_closure_oid','approval_plan_id'],
|
||||
},400)
|
||||
try:
|
||||
target_oid=resolve_commit_oid(data.get('target_oid'))
|
||||
evidence_oid=resolve_commit_oid(data.get('evidence_closure_oid'), 'evidence_closure_oid')
|
||||
except ValueError as exc:
|
||||
return self.sendj({'error':'invalid_tag_target','detail':str(exc)},400)
|
||||
if data.get('confirm') != True:
|
||||
self.sendj({'status':'confirm_required','operation':'tag','hint':'This operation creates a Git tag. Confirm to proceed.','x-openai-isConsequential':True}, 428)
|
||||
return
|
||||
if a['role'] != 'owner' or not data.get('execute'):
|
||||
b=checkout(data.get('branch') or f'request/tag-{safe(ver)}', 'main', True); req={'version':ver,'target_oid':target_oid,'evidence_closure_oid':evidence_oid,'approval_plan_id':data.get('approval_plan_id'),'requested_by':a['account'],'approval_required':True,'created_at':now()}; r=write_json(f'baseline/tag_requests/{safe(ver)}.json', req); c=commit([r], f'chore(chatgpt): request tag {ver}'); return self.sendj({'status':'approval_required','branch':b,'request':req,'commit':c})
|
||||
request=PublicationRequest(ver,target_oid,evidence_oid,str(data.get('approval_plan_id')))
|
||||
main_before=sh(['rev-parse','refs/heads/main'], True)['stdout']
|
||||
publisher=publication_publisher()
|
||||
try:
|
||||
publication=publisher.publish(request)
|
||||
except PublicationManualOwnerReviewRequired as exc:
|
||||
audit(a['account'],'tag_manual_owner_review_required',ver,{'target_oid':target_oid,'approval_plan_id':data.get('approval_plan_id'),'error':redact(exc)})
|
||||
return self.sendj({'status':'MANUAL_OWNER_REVIEW_REQUIRED','error':redact(exc),'main_changed':False,'force_push':False},409)
|
||||
except PublicationConflict as exc:
|
||||
audit(a['account'],'tag_conflict',ver,{'target_oid':target_oid,'approval_plan_id':data.get('approval_plan_id'),'error':str(exc)})
|
||||
return self.sendj({'status':'RECONCILIATION_REQUIRED','error':str(exc),'main_changed':False,'force_push':False},409)
|
||||
except PublicationPending as exc:
|
||||
audit(a['account'],'tag_pending',ver,{'target_oid':target_oid,'approval_plan_id':data.get('approval_plan_id'),'error':str(exc)})
|
||||
return self.sendj({'status':'REMOTE_PUSH_PENDING','error':str(exc),'main_changed':False,'force_push':False},503)
|
||||
except PublicationReconciliationRequired as exc:
|
||||
audit(a['account'],'tag_reconciliation_required',ver,{'target_oid':target_oid,'approval_plan_id':data.get('approval_plan_id'),'error':str(exc)})
|
||||
return self.sendj({'status':'RECONCILIATION_REQUIRED','error':str(exc),'main_changed':False,'force_push':False},502)
|
||||
main_after=sh(['rev-parse','refs/heads/main'], True)['stdout']
|
||||
if main_after != main_before:
|
||||
raise RuntimeError('main changed during tag operation')
|
||||
result={'status':publication['state'],'version':ver,'tag_name':ver,'tag_oid':publication['tag_oid'],'target_oid':target_oid,'expected_target_oid':target_oid,'evidence_closure_oid':evidence_oid,'approval_plan_id':data.get('approval_plan_id'),'workspace_head':sh(['rev-parse','HEAD'],True)['stdout'],'main_oid_before':main_before,'main_oid_after':main_after,'main_changed':False,'force_push':False,'publication':publication}
|
||||
audit(a['account'],'tag',ver,result)
|
||||
self.sendj(result)
|
||||
b=checkout(data.get('branch') or f'request/tag-{safe(ver)}', 'main', True); req={'version':ver,'requested_by':a['account'],'approval_required':True,'created_at':now()}; r=write_json(f'baseline/tag_requests/{safe(ver)}.json', req); c=commit([r], f'chore(chatgpt): request tag {ver}'); return self.sendj({'status':'approval_required','branch':b,'commit':c})
|
||||
out=sh(['tag','-a',ver,'-m',data.get('message') or f'release {ver}'], True); self.sendj({'status':'tagged','version':ver,'git':out,'sync':sync_tag(ver)})
|
||||
def rollback(self,data):
|
||||
a=self.need('rollback:request');
|
||||
if not a: return
|
||||
@@ -810,10 +352,7 @@ return{vs,cmds,ar,cmd,cdet,tot,sok,sfail,runs,baseline,refreshMsg,loadCmd,selVer
|
||||
cmd_filter=(params.get('command') or [None])[0]
|
||||
bl=jload(WS/'baseline'/'current.json',{})
|
||||
current_ver=bl.get('current_baseline','V1R3C1')
|
||||
try:
|
||||
tags=[item['name'] for item in authoritative_tags(git_dir=required_authoritative_git_dir()) if str(item['name']).startswith('V')]
|
||||
except (AuthoritativeGitError, RuntimeError) as exc:
|
||||
return self.sendj({'error':redact(exc),'repository_available':False},503)
|
||||
tags=sh(['tag','--list','V*'])['stdout'].splitlines() if WS.exists() else []
|
||||
versions=[]
|
||||
if (WS/'baseline'/'promotions').exists():
|
||||
for p in sorted((WS/'baseline'/'promotions').glob('*.json')):
|
||||
@@ -851,9 +390,9 @@ return{vs,cmds,ar,cmd,cdet,tot,sok,sfail,runs,baseline,refreshMsg,loadCmd,selVer
|
||||
|
||||
|
||||
def api_spec():
|
||||
return {'version':'V1R1C0-chatgpt-actions','openapi':'/api/openapi.yaml','read':['GET /api/base','GET /api/ver','GET /api/diff','GET /api/patch','GET /api/model','GET /api/run','GET /getReleaseCandidate','GET /getApprovalEnvelope','GET /getReleaseSaga','GET /admin/ai-evolution'],'write':['POST /api/branch','POST /api/file','POST /api/commit','POST /api/patch','POST /api/tag','POST /api/rollback','POST /api/model','PUT /api/model-priority','POST /api/nl','POST /createReleaseCandidate','POST /submitOwnerApproval','POST /startDeployment'],'policy':{'bearer_required_for_write':True,'direct_main_push':False,'delete_tag':False,'owner_approval_required_for_v_level':True,'api_keys_in_git':False,'git_secret_field':'secret_ref','online_git_sync':True,'silent_release_execution_enabled':SILENT_RELEASE_EXECUTION_ENABLED,'silent_release_shadow_default':True}}
|
||||
return {'version':'V1R1C0-chatgpt-actions','openapi':'/api/openapi.yaml','read':['GET /api/base','GET /api/ver','GET /api/diff','GET /api/patch','GET /api/model','GET /api/run','GET /admin/ai-evolution'],'write':['POST /api/branch','POST /api/file','POST /api/commit','POST /api/patch','POST /api/tag','POST /api/rollback','POST /api/model','PUT /api/model-priority','POST /api/nl'],'policy':{'bearer_required_for_write':True,'direct_main_push':False,'delete_tag':False,'owner_approval_required_for_v_level':True,'api_keys_in_git':False,'git_secret_field':'secret_ref','online_git_sync':True}}
|
||||
|
||||
def main():
|
||||
print(f'AIOS ChatGPT GitOps adapter listening on {HOST}:{PORT}')
|
||||
ThreadingHTTPServer((HOST,PORT), H).serve_forever()
|
||||
if __name__ == '__main__': main()
|
||||
if __name__ == '__main__': main()
|
||||
@@ -1,52 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fail-closed policy for in-process Git mutation handlers."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Mapping
|
||||
|
||||
_TRUE = frozenset({"1", "true", "yes", "on"})
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GitWriteDecision:
|
||||
allowed: bool
|
||||
status_code: int
|
||||
error: str
|
||||
reason: str
|
||||
workspace: str
|
||||
|
||||
def payload(self) -> dict[str, str]:
|
||||
return {
|
||||
"status": "allowed" if self.allowed else "blocked",
|
||||
"error": self.error,
|
||||
"reason": self.reason,
|
||||
"workspace": self.workspace,
|
||||
}
|
||||
|
||||
|
||||
def _real(path: Path) -> Path:
|
||||
return Path(os.path.realpath(os.path.abspath(str(path))))
|
||||
|
||||
|
||||
def evaluate_git_write(workspace: Path | str, env: Mapping[str, str] | None = None) -> GitWriteDecision:
|
||||
values = os.environ if env is None else env
|
||||
lexical = Path(os.path.abspath(str(workspace)))
|
||||
canonical = _real(lexical)
|
||||
shown = str(lexical)
|
||||
if str(values.get("AIOS_GITOPS_WRITES_ENABLED", "false")).strip().lower() not in _TRUE:
|
||||
return GitWriteDecision(False, 423, "gitops_writes_disabled", "AIOS_GITOPS_WRITES_ENABLED is false", shown)
|
||||
if lexical != canonical:
|
||||
return GitWriteDecision(False, 409, "workspace_symlink_forbidden", "workspace must be a canonical non-symlink path", shown)
|
||||
controller = _real(Path(values.get("AIOS_GITOPS_CONTROLLER_PATH", "/data/aios-gitops/worktrees/controller")))
|
||||
role = str(values.get("AIOS_GITOPS_WORKSPACE_ROLE", "")).strip().lower()
|
||||
if canonical == controller or role == "runtime-anchor":
|
||||
return GitWriteDecision(False, 423, "runtime_anchor_read_only", "controller/runtime-anchor is read-only", str(canonical))
|
||||
writer_root = _real(Path(values.get("AIOS_GITOPS_APPROVED_WRITER_ROOT", "/data/aios-gitops/worktrees")))
|
||||
try:
|
||||
canonical.relative_to(writer_root)
|
||||
except ValueError:
|
||||
return GitWriteDecision(False, 409, "workspace_not_approved_writer", "workspace is outside the approved writer root", str(canonical))
|
||||
return GitWriteDecision(True, 200, "", "approved isolated writer", str(canonical))
|
||||
@@ -0,0 +1,25 @@
|
||||
"""AIOS-managed Hermes runtime boundary."""
|
||||
|
||||
from .adapter import (
|
||||
HermesAdapter,
|
||||
HermesConflict,
|
||||
HermesNotFound,
|
||||
HermesRuntimeError,
|
||||
InMemoryHermesStore,
|
||||
)
|
||||
from .contract import HermesHealth, HermesJob, HermesJobState, HermesResult
|
||||
from .worker import HermesWorker, WorkerRunResult
|
||||
|
||||
__all__ = [
|
||||
"HermesAdapter",
|
||||
"HermesConflict",
|
||||
"HermesHealth",
|
||||
"HermesJob",
|
||||
"HermesJobState",
|
||||
"HermesNotFound",
|
||||
"HermesResult",
|
||||
"HermesRuntimeError",
|
||||
"HermesWorker",
|
||||
"InMemoryHermesStore",
|
||||
"WorkerRunResult",
|
||||
]
|
||||
@@ -0,0 +1,437 @@
|
||||
"""Tenant-partitioned Hermes queue and worker lease boundary."""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import threading
|
||||
from dataclasses import replace
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Iterable, Mapping
|
||||
|
||||
from .contract import (
|
||||
ALLOWED_TRANSITIONS,
|
||||
TERMINAL_STATES,
|
||||
HermesHealth,
|
||||
HermesJob,
|
||||
HermesJobState,
|
||||
HermesResult,
|
||||
utc_now,
|
||||
)
|
||||
|
||||
|
||||
class HermesRuntimeError(RuntimeError):
|
||||
"""Base Hermes runtime-boundary failure."""
|
||||
|
||||
|
||||
class HermesNotFound(HermesRuntimeError):
|
||||
"""A tenant-scoped job was not found."""
|
||||
|
||||
|
||||
class HermesConflict(HermesRuntimeError):
|
||||
"""A state, lease, idempotency, or revision conflict occurred."""
|
||||
|
||||
|
||||
class InMemoryHermesStore:
|
||||
"""Thread-safe reference store; production backends implement this contract."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.lock = threading.RLock()
|
||||
self.jobs: dict[str, HermesJob] = {}
|
||||
self.results: dict[str, HermesResult] = {}
|
||||
self.idempotency: dict[tuple[str, str], tuple[str, str]] = {}
|
||||
self.worker_heartbeats: dict[str, tuple[str, str | None]] = {}
|
||||
|
||||
|
||||
class HermesAdapter:
|
||||
"""Queue and reconcile Hermes jobs without owning AIOS policy."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
store: InMemoryHermesStore | None = None,
|
||||
*,
|
||||
runtime_id: str = "hermes-runtime",
|
||||
runtime_version: str = "v1",
|
||||
default_lease_seconds: int = 60,
|
||||
) -> None:
|
||||
if default_lease_seconds < 5:
|
||||
raise ValueError("default_lease_seconds must be >= 5")
|
||||
self.store = store or InMemoryHermesStore()
|
||||
self.runtime_id = runtime_id
|
||||
self.runtime_version = runtime_version
|
||||
self.default_lease_seconds = int(default_lease_seconds)
|
||||
|
||||
@staticmethod
|
||||
def _digest(job: HermesJob) -> str:
|
||||
raw = json.dumps(
|
||||
{
|
||||
"tenant_id": job.tenant_id,
|
||||
"cell_id": job.cell_id,
|
||||
"profile_id": job.profile_id,
|
||||
"task_id": job.task_id,
|
||||
"trace_id": job.trace_id,
|
||||
"action": job.action,
|
||||
"payload": dict(job.payload),
|
||||
"priority": job.priority,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
default=str,
|
||||
).encode("utf-8")
|
||||
return hashlib.sha256(raw).hexdigest()
|
||||
|
||||
@staticmethod
|
||||
def _parse(value: str | None) -> datetime | None:
|
||||
if value is None:
|
||||
return None
|
||||
normalized = value[:-1] + "+00:00" if value.endswith("Z") else value
|
||||
parsed = datetime.fromisoformat(normalized)
|
||||
if parsed.tzinfo is None:
|
||||
raise ValueError("timestamp must include timezone")
|
||||
return parsed.astimezone(timezone.utc)
|
||||
|
||||
@staticmethod
|
||||
def _format(value: datetime) -> str:
|
||||
return value.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||
|
||||
@staticmethod
|
||||
def _assert_tenant(job: HermesJob, tenant_id: str) -> None:
|
||||
if job.tenant_id != tenant_id:
|
||||
raise HermesNotFound("job not found")
|
||||
|
||||
def enqueue(self, job: HermesJob) -> HermesJob:
|
||||
if job.state != HermesJobState.QUEUED or job.revision != 1:
|
||||
raise HermesConflict("new job must be QUEUED at revision 1")
|
||||
digest = self._digest(job)
|
||||
idem_ref = (job.tenant_id, job.idempotency_key)
|
||||
with self.store.lock:
|
||||
existing = self.store.jobs.get(job.job_id)
|
||||
if existing is not None:
|
||||
if self._digest(existing) != digest:
|
||||
raise HermesConflict("job_id reused with different content")
|
||||
return existing
|
||||
prior = self.store.idempotency.get(idem_ref)
|
||||
if prior is not None:
|
||||
prior_id, prior_digest = prior
|
||||
if prior_digest != digest:
|
||||
raise HermesConflict("idempotency key reused with different content")
|
||||
return self.store.jobs[prior_id]
|
||||
self.store.jobs[job.job_id] = job
|
||||
self.store.idempotency[idem_ref] = (job.job_id, digest)
|
||||
return job
|
||||
|
||||
def get(self, job_id: str, *, tenant_id: str) -> HermesJob:
|
||||
with self.store.lock:
|
||||
job = self.store.jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HermesNotFound("job not found")
|
||||
self._assert_tenant(job, tenant_id)
|
||||
return job
|
||||
|
||||
def list_jobs(
|
||||
self,
|
||||
*,
|
||||
tenant_id: str,
|
||||
cell_id: str | None = None,
|
||||
profile_id: str | None = None,
|
||||
states: Iterable[HermesJobState | str] | None = None,
|
||||
) -> list[HermesJob]:
|
||||
state_filter = {HermesJobState(item) for item in states} if states else None
|
||||
with self.store.lock:
|
||||
self._requeue_expired_locked()
|
||||
jobs = [
|
||||
job
|
||||
for job in self.store.jobs.values()
|
||||
if job.tenant_id == tenant_id
|
||||
and (cell_id is None or job.cell_id == cell_id)
|
||||
and (profile_id is None or job.profile_id == profile_id)
|
||||
and (state_filter is None or job.state in state_filter)
|
||||
]
|
||||
return sorted(jobs, key=lambda item: (item.priority, item.created_at, item.job_id))
|
||||
|
||||
def claim(
|
||||
self,
|
||||
*,
|
||||
tenant_id: str,
|
||||
cell_id: str,
|
||||
profile_id: str,
|
||||
worker_id: str,
|
||||
lease_seconds: int | None = None,
|
||||
) -> HermesJob | None:
|
||||
worker_id = str(worker_id or "").strip()
|
||||
if not worker_id:
|
||||
raise ValueError("worker_id is required")
|
||||
duration = int(lease_seconds or self.default_lease_seconds)
|
||||
if duration < 5:
|
||||
raise ValueError("lease_seconds must be >= 5")
|
||||
with self.store.lock:
|
||||
self._requeue_expired_locked()
|
||||
candidates = [
|
||||
job
|
||||
for job in self.store.jobs.values()
|
||||
if job.tenant_id == tenant_id
|
||||
and job.cell_id == cell_id
|
||||
and job.profile_id == profile_id
|
||||
and job.state == HermesJobState.QUEUED
|
||||
]
|
||||
self.store.worker_heartbeats[worker_id] = (utc_now(), None)
|
||||
if not candidates:
|
||||
return None
|
||||
current = sorted(
|
||||
candidates,
|
||||
key=lambda item: (item.priority, item.created_at, item.job_id),
|
||||
)[0]
|
||||
claimed = self._transition_locked(
|
||||
current,
|
||||
HermesJobState.CLAIMED,
|
||||
assigned_worker_id=worker_id,
|
||||
lease_expires_at=self._format(
|
||||
datetime.now(timezone.utc) + timedelta(seconds=duration)
|
||||
),
|
||||
)
|
||||
self.store.worker_heartbeats[worker_id] = (utc_now(), claimed.job_id)
|
||||
return claimed
|
||||
|
||||
def start(
|
||||
self,
|
||||
job_id: str,
|
||||
*,
|
||||
tenant_id: str,
|
||||
worker_id: str,
|
||||
expected_revision: int | None = None,
|
||||
) -> HermesJob:
|
||||
with self.store.lock:
|
||||
current = self._owned_job_locked(job_id, tenant_id, worker_id)
|
||||
self._assert_revision(current, expected_revision)
|
||||
return self._transition_locked(current, HermesJobState.RUNNING)
|
||||
|
||||
def heartbeat(
|
||||
self,
|
||||
job_id: str,
|
||||
*,
|
||||
tenant_id: str,
|
||||
worker_id: str,
|
||||
lease_seconds: int | None = None,
|
||||
) -> HermesJob:
|
||||
duration = int(lease_seconds or self.default_lease_seconds)
|
||||
with self.store.lock:
|
||||
current = self._owned_job_locked(job_id, tenant_id, worker_id)
|
||||
if current.state not in {HermesJobState.CLAIMED, HermesJobState.RUNNING}:
|
||||
raise HermesConflict("job does not accept heartbeat")
|
||||
updated = replace(
|
||||
current,
|
||||
revision=current.revision + 1,
|
||||
lease_expires_at=self._format(
|
||||
datetime.now(timezone.utc) + timedelta(seconds=duration)
|
||||
),
|
||||
updated_at=utc_now(),
|
||||
)
|
||||
self.store.jobs[job_id] = updated
|
||||
self.store.worker_heartbeats[worker_id] = (utc_now(), job_id)
|
||||
return updated
|
||||
|
||||
def complete(
|
||||
self,
|
||||
job_id: str,
|
||||
*,
|
||||
tenant_id: str,
|
||||
worker_id: str,
|
||||
output: Mapping[str, object],
|
||||
expected_revision: int | None = None,
|
||||
) -> HermesResult:
|
||||
return self._finish(
|
||||
job_id,
|
||||
tenant_id=tenant_id,
|
||||
worker_id=worker_id,
|
||||
state=HermesJobState.COMPLETED,
|
||||
output=output,
|
||||
expected_revision=expected_revision,
|
||||
)
|
||||
|
||||
def fail(
|
||||
self,
|
||||
job_id: str,
|
||||
*,
|
||||
tenant_id: str,
|
||||
worker_id: str,
|
||||
error: Mapping[str, object],
|
||||
expected_revision: int | None = None,
|
||||
) -> HermesResult:
|
||||
return self._finish(
|
||||
job_id,
|
||||
tenant_id=tenant_id,
|
||||
worker_id=worker_id,
|
||||
state=HermesJobState.FAILED,
|
||||
error=error,
|
||||
expected_revision=expected_revision,
|
||||
)
|
||||
|
||||
def cancel(
|
||||
self,
|
||||
job_id: str,
|
||||
*,
|
||||
tenant_id: str,
|
||||
expected_revision: int | None = None,
|
||||
) -> HermesResult:
|
||||
with self.store.lock:
|
||||
current = self.store.jobs.get(job_id)
|
||||
if current is None:
|
||||
raise HermesNotFound("job not found")
|
||||
self._assert_tenant(current, tenant_id)
|
||||
self._assert_revision(current, expected_revision)
|
||||
if current.state in TERMINAL_STATES:
|
||||
result = self.store.results.get(job_id)
|
||||
if result is None:
|
||||
raise HermesConflict("terminal result missing")
|
||||
return result
|
||||
updated = self._transition_locked(current, HermesJobState.CANCELLED)
|
||||
result = HermesResult(
|
||||
job_id=updated.job_id,
|
||||
tenant_id=updated.tenant_id,
|
||||
task_id=updated.task_id,
|
||||
trace_id=updated.trace_id,
|
||||
state=HermesJobState.CANCELLED,
|
||||
worker_id=updated.assigned_worker_id,
|
||||
error={"code": "cancelled", "message": "cancelled by AIOS"},
|
||||
finished_at=utc_now(),
|
||||
)
|
||||
self.store.results[job_id] = result
|
||||
return result
|
||||
|
||||
def result(self, job_id: str, *, tenant_id: str) -> HermesResult:
|
||||
with self.store.lock:
|
||||
job = self.store.jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HermesNotFound("job not found")
|
||||
self._assert_tenant(job, tenant_id)
|
||||
result = self.store.results.get(job_id)
|
||||
if result is None:
|
||||
raise HermesConflict("job has no terminal result")
|
||||
return result
|
||||
|
||||
def health(self) -> HermesHealth:
|
||||
with self.store.lock:
|
||||
self._requeue_expired_locked()
|
||||
active_jobs = sum(
|
||||
1 for job in self.store.jobs.values()
|
||||
if job.state in {HermesJobState.CLAIMED, HermesJobState.RUNNING}
|
||||
)
|
||||
queued_jobs = sum(
|
||||
1 for job in self.store.jobs.values()
|
||||
if job.state == HermesJobState.QUEUED
|
||||
)
|
||||
partitions = {
|
||||
(job.tenant_id, job.cell_id, job.profile_id)
|
||||
for job in self.store.jobs.values()
|
||||
}
|
||||
active_workers = sum(
|
||||
1 for heartbeat, _ in self.store.worker_heartbeats.values()
|
||||
if self._parse(heartbeat)
|
||||
and self._parse(heartbeat) >= datetime.now(timezone.utc) - timedelta(seconds=120)
|
||||
)
|
||||
return HermesHealth(
|
||||
runtime_id=self.runtime_id,
|
||||
status="healthy",
|
||||
version=self.runtime_version,
|
||||
heartbeat_at=utc_now(),
|
||||
active_workers=active_workers,
|
||||
active_jobs=active_jobs,
|
||||
queued_jobs=queued_jobs,
|
||||
tenant_partitions=len(partitions),
|
||||
profile_isolation_enforced=True,
|
||||
details={"store": type(self.store).__name__},
|
||||
)
|
||||
|
||||
def _finish(
|
||||
self,
|
||||
job_id: str,
|
||||
*,
|
||||
tenant_id: str,
|
||||
worker_id: str,
|
||||
state: HermesJobState,
|
||||
output: Mapping[str, object] | None = None,
|
||||
error: Mapping[str, object] | None = None,
|
||||
expected_revision: int | None,
|
||||
) -> HermesResult:
|
||||
with self.store.lock:
|
||||
current = self._owned_job_locked(job_id, tenant_id, worker_id)
|
||||
self._assert_revision(current, expected_revision)
|
||||
updated = self._transition_locked(current, state)
|
||||
result = HermesResult(
|
||||
job_id=updated.job_id,
|
||||
tenant_id=updated.tenant_id,
|
||||
task_id=updated.task_id,
|
||||
trace_id=updated.trace_id,
|
||||
state=state,
|
||||
worker_id=worker_id,
|
||||
output=output,
|
||||
error=error,
|
||||
started_at=current.updated_at,
|
||||
finished_at=utc_now(),
|
||||
)
|
||||
self.store.results[job_id] = result
|
||||
self.store.worker_heartbeats[worker_id] = (utc_now(), None)
|
||||
return result
|
||||
|
||||
def _owned_job_locked(self, job_id: str, tenant_id: str, worker_id: str) -> HermesJob:
|
||||
current = self.store.jobs.get(job_id)
|
||||
if current is None:
|
||||
raise HermesNotFound("job not found")
|
||||
self._assert_tenant(current, tenant_id)
|
||||
if current.assigned_worker_id != worker_id:
|
||||
raise HermesConflict("worker does not own job lease")
|
||||
return current
|
||||
|
||||
@staticmethod
|
||||
def _assert_revision(job: HermesJob, expected: int | None) -> None:
|
||||
if expected is not None and job.revision != expected:
|
||||
raise HermesConflict("job revision changed")
|
||||
|
||||
def _transition_locked(
|
||||
self,
|
||||
current: HermesJob,
|
||||
target: HermesJobState,
|
||||
*,
|
||||
assigned_worker_id: str | None = None,
|
||||
lease_expires_at: str | None = None,
|
||||
) -> HermesJob:
|
||||
if target not in ALLOWED_TRANSITIONS[current.state]:
|
||||
raise HermesConflict(
|
||||
f"invalid transition {current.state.value} -> {target.value}"
|
||||
)
|
||||
updated = replace(
|
||||
current,
|
||||
state=target,
|
||||
revision=current.revision + 1,
|
||||
assigned_worker_id=(
|
||||
assigned_worker_id
|
||||
if assigned_worker_id is not None
|
||||
else current.assigned_worker_id
|
||||
),
|
||||
lease_expires_at=(
|
||||
lease_expires_at
|
||||
if lease_expires_at is not None
|
||||
else (None if target in TERMINAL_STATES else current.lease_expires_at)
|
||||
),
|
||||
updated_at=utc_now(),
|
||||
)
|
||||
self.store.jobs[current.job_id] = updated
|
||||
return updated
|
||||
|
||||
def _requeue_expired_locked(self) -> None:
|
||||
now = datetime.now(timezone.utc)
|
||||
for current in list(self.store.jobs.values()):
|
||||
expiry = self._parse(current.lease_expires_at)
|
||||
if (
|
||||
current.state in {HermesJobState.CLAIMED, HermesJobState.RUNNING}
|
||||
and expiry is not None
|
||||
and expiry <= now
|
||||
):
|
||||
self.store.jobs[current.job_id] = replace(
|
||||
current,
|
||||
state=HermesJobState.QUEUED,
|
||||
revision=current.revision + 1,
|
||||
assigned_worker_id=None,
|
||||
lease_expires_at=None,
|
||||
updated_at=utc_now(),
|
||||
)
|
||||
@@ -0,0 +1,153 @@
|
||||
"""Tenant-scoped Hermes runtime contracts."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import uuid
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from enum import Enum
|
||||
from typing import Any, Mapping
|
||||
|
||||
|
||||
def utc_now() -> str:
|
||||
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||
|
||||
|
||||
def _required(value: str, label: str) -> str:
|
||||
normalized = str(value or "").strip()
|
||||
if not normalized:
|
||||
raise ValueError(f"{label} is required")
|
||||
return normalized
|
||||
|
||||
|
||||
def _safe_mapping(value: Mapping[str, Any] | None) -> dict[str, Any]:
|
||||
return json.loads(json.dumps(dict(value or {}), ensure_ascii=False, sort_keys=True, default=str))
|
||||
|
||||
|
||||
class HermesJobState(str, Enum):
|
||||
QUEUED = "QUEUED"
|
||||
CLAIMED = "CLAIMED"
|
||||
RUNNING = "RUNNING"
|
||||
COMPLETED = "COMPLETED"
|
||||
FAILED = "FAILED"
|
||||
CANCELLED = "CANCELLED"
|
||||
|
||||
|
||||
ALLOWED_TRANSITIONS: Mapping[HermesJobState, frozenset[HermesJobState]] = {
|
||||
HermesJobState.QUEUED: frozenset({HermesJobState.CLAIMED, HermesJobState.CANCELLED}),
|
||||
HermesJobState.CLAIMED: frozenset({HermesJobState.RUNNING, HermesJobState.CANCELLED}),
|
||||
HermesJobState.RUNNING: frozenset(
|
||||
{HermesJobState.COMPLETED, HermesJobState.FAILED, HermesJobState.CANCELLED}
|
||||
),
|
||||
HermesJobState.COMPLETED: frozenset(),
|
||||
HermesJobState.FAILED: frozenset(),
|
||||
HermesJobState.CANCELLED: frozenset(),
|
||||
}
|
||||
TERMINAL_STATES = frozenset(
|
||||
{HermesJobState.COMPLETED, HermesJobState.FAILED, HermesJobState.CANCELLED}
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class HermesJob:
|
||||
job_id: str
|
||||
tenant_id: str
|
||||
cell_id: str
|
||||
profile_id: str
|
||||
task_id: str
|
||||
trace_id: str
|
||||
action: str
|
||||
payload: Mapping[str, Any] = field(default_factory=dict)
|
||||
idempotency_key: str = ""
|
||||
priority: int = 100
|
||||
state: HermesJobState = HermesJobState.QUEUED
|
||||
revision: int = 1
|
||||
assigned_worker_id: str | None = None
|
||||
lease_expires_at: str | None = None
|
||||
created_at: str = field(default_factory=utc_now)
|
||||
updated_at: str = field(default_factory=utc_now)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
for name in (
|
||||
"job_id", "tenant_id", "cell_id", "profile_id", "task_id",
|
||||
"trace_id", "action", "idempotency_key", "created_at", "updated_at",
|
||||
):
|
||||
object.__setattr__(self, name, _required(getattr(self, name), name))
|
||||
object.__setattr__(self, "payload", _safe_mapping(self.payload))
|
||||
object.__setattr__(self, "state", HermesJobState(self.state))
|
||||
if not 0 <= int(self.priority) <= 1000:
|
||||
raise ValueError("priority must be between 0 and 1000")
|
||||
if self.revision < 1:
|
||||
raise ValueError("revision must be >= 1")
|
||||
|
||||
@classmethod
|
||||
def create(
|
||||
cls,
|
||||
*,
|
||||
tenant_id: str,
|
||||
cell_id: str,
|
||||
profile_id: str,
|
||||
task_id: str,
|
||||
trace_id: str,
|
||||
action: str,
|
||||
payload: Mapping[str, Any] | None,
|
||||
idempotency_key: str,
|
||||
priority: int = 100,
|
||||
) -> "HermesJob":
|
||||
return cls(
|
||||
job_id=f"hermes_{uuid.uuid4().hex}",
|
||||
tenant_id=tenant_id,
|
||||
cell_id=cell_id,
|
||||
profile_id=profile_id,
|
||||
task_id=task_id,
|
||||
trace_id=trace_id,
|
||||
action=action,
|
||||
payload=_safe_mapping(payload),
|
||||
idempotency_key=idempotency_key,
|
||||
priority=priority,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class HermesResult:
|
||||
job_id: str
|
||||
tenant_id: str
|
||||
task_id: str
|
||||
trace_id: str
|
||||
state: HermesJobState
|
||||
worker_id: str | None = None
|
||||
output: Mapping[str, Any] | None = None
|
||||
error: Mapping[str, Any] | None = None
|
||||
started_at: str | None = None
|
||||
finished_at: str | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
for name in ("job_id", "tenant_id", "task_id", "trace_id"):
|
||||
object.__setattr__(self, name, _required(getattr(self, name), name))
|
||||
object.__setattr__(self, "state", HermesJobState(self.state))
|
||||
if self.output is not None:
|
||||
object.__setattr__(self, "output", _safe_mapping(self.output))
|
||||
if self.error is not None:
|
||||
object.__setattr__(self, "error", _safe_mapping(self.error))
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class HermesHealth:
|
||||
runtime_id: str
|
||||
status: str
|
||||
version: str
|
||||
heartbeat_at: str
|
||||
active_workers: int = 0
|
||||
active_jobs: int = 0
|
||||
queued_jobs: int = 0
|
||||
tenant_partitions: int = 0
|
||||
profile_isolation_enforced: bool = True
|
||||
details: Mapping[str, Any] = field(default_factory=dict)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
for name in ("runtime_id", "status", "version", "heartbeat_at"):
|
||||
object.__setattr__(self, name, _required(getattr(self, name), name))
|
||||
for name in ("active_workers", "active_jobs", "queued_jobs", "tenant_partitions"):
|
||||
if int(getattr(self, name)) < 0:
|
||||
raise ValueError(f"{name} must be >= 0")
|
||||
object.__setattr__(self, "details", _safe_mapping(self.details))
|
||||
@@ -0,0 +1,155 @@
|
||||
"""Hermes worker loop with lease renewal and bounded recovery."""
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Callable, Mapping
|
||||
|
||||
from .adapter import HermesAdapter, HermesConflict
|
||||
from .contract import HermesJob, HermesJobState
|
||||
|
||||
|
||||
JobHandler = Callable[[HermesJob], Mapping[str, Any]]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class WorkerRunResult:
|
||||
claimed: bool
|
||||
job_id: str | None = None
|
||||
state: str | None = None
|
||||
error: str | None = None
|
||||
|
||||
|
||||
class HermesWorker:
|
||||
"""Poll one tenant/cell/profile partition and execute jobs safely."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
adapter: HermesAdapter,
|
||||
handler: JobHandler,
|
||||
*,
|
||||
worker_id: str,
|
||||
tenant_id: str,
|
||||
cell_id: str,
|
||||
profile_id: str,
|
||||
poll_seconds: float = 1.0,
|
||||
heartbeat_seconds: float = 10.0,
|
||||
) -> None:
|
||||
if poll_seconds <= 0 or heartbeat_seconds <= 0:
|
||||
raise ValueError("poll_seconds and heartbeat_seconds must be positive")
|
||||
for name, value in {
|
||||
"worker_id": worker_id,
|
||||
"tenant_id": tenant_id,
|
||||
"cell_id": cell_id,
|
||||
"profile_id": profile_id,
|
||||
}.items():
|
||||
if not str(value or "").strip():
|
||||
raise ValueError(f"{name} is required")
|
||||
self.adapter = adapter
|
||||
self.handler = handler
|
||||
self.worker_id = worker_id
|
||||
self.tenant_id = tenant_id
|
||||
self.cell_id = cell_id
|
||||
self.profile_id = profile_id
|
||||
self.poll_seconds = float(poll_seconds)
|
||||
self.heartbeat_seconds = float(heartbeat_seconds)
|
||||
self._stop = threading.Event()
|
||||
|
||||
def stop(self) -> None:
|
||||
self._stop.set()
|
||||
|
||||
def run_once(self) -> WorkerRunResult:
|
||||
job = self.adapter.claim(
|
||||
tenant_id=self.tenant_id,
|
||||
cell_id=self.cell_id,
|
||||
profile_id=self.profile_id,
|
||||
worker_id=self.worker_id,
|
||||
)
|
||||
if job is None:
|
||||
return WorkerRunResult(claimed=False)
|
||||
|
||||
running = self.adapter.start(
|
||||
job.job_id,
|
||||
tenant_id=self.tenant_id,
|
||||
worker_id=self.worker_id,
|
||||
expected_revision=job.revision,
|
||||
)
|
||||
|
||||
heartbeat_stop = threading.Event()
|
||||
heartbeat_thread = threading.Thread(
|
||||
target=self._heartbeat_loop,
|
||||
args=(running.job_id, heartbeat_stop),
|
||||
name=f"hermes-heartbeat-{self.worker_id}",
|
||||
daemon=True,
|
||||
)
|
||||
heartbeat_thread.start()
|
||||
|
||||
output: Mapping[str, Any] | None = None
|
||||
execution_error: Exception | None = None
|
||||
try:
|
||||
output = dict(self.handler(running))
|
||||
except Exception as exc: # runtime boundary: normalize below
|
||||
execution_error = exc
|
||||
finally:
|
||||
# Stop the lease heartbeat before reading the final revision. This
|
||||
# prevents a revision increment racing with complete/fail.
|
||||
heartbeat_stop.set()
|
||||
heartbeat_thread.join(timeout=max(self.heartbeat_seconds * 2, 1.0))
|
||||
|
||||
try:
|
||||
current = self.adapter.get(running.job_id, tenant_id=self.tenant_id)
|
||||
if execution_error is None:
|
||||
result = self.adapter.complete(
|
||||
running.job_id,
|
||||
tenant_id=self.tenant_id,
|
||||
worker_id=self.worker_id,
|
||||
output=dict(output or {}),
|
||||
expected_revision=current.revision,
|
||||
)
|
||||
return WorkerRunResult(
|
||||
claimed=True,
|
||||
job_id=result.job_id,
|
||||
state=result.state.value,
|
||||
)
|
||||
|
||||
result = self.adapter.fail(
|
||||
running.job_id,
|
||||
tenant_id=self.tenant_id,
|
||||
worker_id=self.worker_id,
|
||||
error={
|
||||
"code": "worker_error",
|
||||
"message": str(execution_error),
|
||||
},
|
||||
expected_revision=current.revision,
|
||||
)
|
||||
return WorkerRunResult(
|
||||
claimed=True,
|
||||
job_id=result.job_id,
|
||||
state=result.state.value,
|
||||
error=str(execution_error),
|
||||
)
|
||||
except HermesConflict as exc:
|
||||
return WorkerRunResult(
|
||||
claimed=True,
|
||||
job_id=running.job_id,
|
||||
state=HermesJobState.FAILED.value,
|
||||
error=f"reconciliation_conflict: {exc}",
|
||||
)
|
||||
|
||||
def run_forever(self) -> None:
|
||||
while not self._stop.is_set():
|
||||
result = self.run_once()
|
||||
if not result.claimed:
|
||||
self._stop.wait(self.poll_seconds)
|
||||
|
||||
def _heartbeat_loop(self, job_id: str, stop_event: threading.Event) -> None:
|
||||
while not stop_event.wait(self.heartbeat_seconds):
|
||||
try:
|
||||
self.adapter.heartbeat(
|
||||
job_id,
|
||||
tenant_id=self.tenant_id,
|
||||
worker_id=self.worker_id,
|
||||
)
|
||||
except Exception:
|
||||
stop_event.set()
|
||||
return
|
||||
@@ -1,316 +0,0 @@
|
||||
"""Task-scoped hardened wrapper around the existing Action API."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
import urllib.parse
|
||||
|
||||
import aios_chatgpt_actions_api as api
|
||||
import aios_isolated_writer_policy as policy
|
||||
from isolated_writer_authz import AuthorizationError, authorize
|
||||
from isolated_writer_commit_file import AtomicCommitFile, CommitFileError, OperationLedger
|
||||
from isolated_writer_token_registry import TokenRegistry
|
||||
|
||||
_reject = api.reject_non_read_http
|
||||
_write_file = api.write_file
|
||||
_sync_branch = api.sync_branch
|
||||
_original_do_get = api.H.do_GET
|
||||
api.OPENAPI = Path(os.getenv("AIOS_GITOPS_ISOLATED_WRITER_OPENAPI", str(api.OPENAPI)))
|
||||
|
||||
|
||||
def _registry() -> TokenRegistry:
|
||||
return TokenRegistry(Path(os.environ["AIOS_GITOPS_TOKEN_REGISTRY_V2"]))
|
||||
|
||||
|
||||
def _atomic_writer() -> AtomicCommitFile:
|
||||
return AtomicCommitFile(
|
||||
workspace=api.WS,
|
||||
bare_repo=api.BARE_REPO,
|
||||
ledger=OperationLedger(Path(os.environ["AIOS_GITOPS_WRITER_OPERATION_LEDGER"])),
|
||||
remote=os.getenv("AIOS_GITOPS_GITEA_PUSH_REMOTE", ""),
|
||||
audit=policy.append_audit_record,
|
||||
)
|
||||
|
||||
|
||||
def _bearer(handler) -> str:
|
||||
value = handler.headers.get("Authorization", "")
|
||||
return value[7:].strip() if value.startswith("Bearer ") else ""
|
||||
|
||||
|
||||
def _authorize(handler, *, method: str, path: str, branch=None, target_path=None):
|
||||
grant = authorize(
|
||||
_registry(),
|
||||
_bearer(handler),
|
||||
method=method,
|
||||
path=path,
|
||||
source_ip=str(handler.client_address[0]),
|
||||
workspace=api.WS,
|
||||
branch=branch,
|
||||
target_path=target_path,
|
||||
)
|
||||
grant.record["_source_ip"] = str(handler.client_address[0])
|
||||
return grant
|
||||
|
||||
|
||||
def _authorization_error(handler, exc: AuthorizationError):
|
||||
return handler.sendj({"error": exc.code}, exc.status)
|
||||
|
||||
|
||||
def _isolated_runtime_status(handler):
|
||||
branch = api.sh(["rev-parse", "--abbrev-ref", "HEAD"], True)["stdout"]
|
||||
head = api.sh(["rev-parse", "HEAD"], True)["stdout"]
|
||||
return handler.sendj({
|
||||
"service": os.getenv(
|
||||
"AIOS_GITOPS_SERVICE_AUDIENCE",
|
||||
"aios-isolated-writer.service",
|
||||
),
|
||||
"status": "active",
|
||||
"runtime_role": os.getenv("AIOS_GITOPS_RUNTIME_ROLE"),
|
||||
"workspace_role": os.getenv("AIOS_GITOPS_WORKSPACE_ROLE"),
|
||||
"writes_enabled": os.getenv("AIOS_GITOPS_WRITES_ENABLED", "").lower()
|
||||
in {"1", "true", "yes", "on"},
|
||||
"allowed_branch": policy.allowed_branch(),
|
||||
"approved_base_oid": policy.allowed_base_oid(),
|
||||
"workspace_branch": branch,
|
||||
"workspace_head_oid": head,
|
||||
"repository_identity": os.getenv("AIOS_GITOPS_REPOSITORY_IDENTITY"),
|
||||
"token_policy_version": 2,
|
||||
"atomic_commit_file": True,
|
||||
"path_policy_sha256": policy.configured_path_policy_sha256(),
|
||||
})
|
||||
|
||||
|
||||
def _isolated_do_get(self):
|
||||
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
|
||||
if path in {"/health", "/api/health"}:
|
||||
return self.sendj({
|
||||
"status": "ok",
|
||||
"service": os.getenv(
|
||||
"AIOS_GITOPS_SERVICE_AUDIENCE",
|
||||
"aios-isolated-writer.service",
|
||||
),
|
||||
})
|
||||
if path not in {"/api/runtime-status", "/api/baseline", "/api/patches"}:
|
||||
return self.sendj({"error": "isolated_writer_operation_forbidden"}, 403)
|
||||
try:
|
||||
_authorize(self, method="GET", path=path)
|
||||
except AuthorizationError as exc:
|
||||
return _authorization_error(self, exc)
|
||||
if path == "/api/runtime-status":
|
||||
return _isolated_runtime_status(self)
|
||||
return _original_do_get(self)
|
||||
|
||||
|
||||
def _read_commit_file_body(handler):
|
||||
try:
|
||||
length = int(handler.headers.get("Content-Length", "0"))
|
||||
except ValueError:
|
||||
raise CommitFileError("writer_content_length_rejected") from None
|
||||
if length <= 0:
|
||||
raise CommitFileError("writer_request_object_required")
|
||||
if length > 1_100_000:
|
||||
raise CommitFileError("writer_request_too_large", 413)
|
||||
raw = handler.rfile.read(length)
|
||||
try:
|
||||
value = json.loads(raw.decode("utf-8"))
|
||||
except (UnicodeDecodeError, json.JSONDecodeError):
|
||||
raise CommitFileError("writer_request_json_rejected") from None
|
||||
if not isinstance(value, dict):
|
||||
raise CommitFileError("writer_request_object_required")
|
||||
return value
|
||||
|
||||
|
||||
def _isolated_do_post(self):
|
||||
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
|
||||
if path != "/api/commit-file":
|
||||
return self.sendj({"error": "isolated_writer_operation_forbidden"}, 403)
|
||||
try:
|
||||
data = _read_commit_file_body(self)
|
||||
branch = str(data.get("branch") or "")
|
||||
target_path = str(data.get("path") or "")
|
||||
grant = _authorize(
|
||||
self,
|
||||
method="POST",
|
||||
path=path,
|
||||
branch=branch,
|
||||
target_path=target_path,
|
||||
)
|
||||
if api.content_has_secret(str(data.get("content") or "")):
|
||||
raise CommitFileError("writer_secret_content_rejected", 403)
|
||||
result = _atomic_writer().execute(grant, data)
|
||||
return self.sendj(result)
|
||||
except AuthorizationError as exc:
|
||||
return _authorization_error(self, exc)
|
||||
except CommitFileError as exc:
|
||||
return self.sendj({"error": exc.code}, exc.status)
|
||||
except PermissionError:
|
||||
return self.sendj({"error": "isolated_writer_policy_rejected"}, 403)
|
||||
|
||||
|
||||
def reject_non_read_http(handler, method: str, path: str) -> bool:
|
||||
if _reject(handler, method, path):
|
||||
return True
|
||||
try:
|
||||
policy.validate_endpoint(method, path, api.WS, api.BARE_REPO)
|
||||
return False
|
||||
except PermissionError as exc:
|
||||
handler.sendj({"error": str(exc)}, 403)
|
||||
return True
|
||||
|
||||
|
||||
def checkout(branch, base="main", create=False):
|
||||
api.ensure_git()
|
||||
target = api.branch_name(branch)
|
||||
policy.validate_branch_request({"branch": target, "base": base})
|
||||
exists = api.sh(["rev-parse", "--verify", target])["code"] == 0
|
||||
if exists:
|
||||
if api.sh(["rev-parse", "--abbrev-ref", "HEAD"], True)["stdout"] != target:
|
||||
api.sh(["checkout", target], True)
|
||||
elif create:
|
||||
api.sh(["checkout", "-b", target, base], True)
|
||||
else:
|
||||
raise PermissionError("isolated_writer_branch_missing")
|
||||
policy.require_service_invariants(api.WS, api.BARE_REPO)
|
||||
return target
|
||||
|
||||
|
||||
def write_file(path, content):
|
||||
policy.validate_path(path, api.WS)
|
||||
return _write_file(path, content)
|
||||
|
||||
|
||||
def sync_branch(branch: str):
|
||||
policy.validate_sync_branch(branch)
|
||||
local = _sync_branch(branch)
|
||||
remote = os.getenv("AIOS_GITOPS_GITEA_PUSH_REMOTE", "").strip()
|
||||
if remote != "gitea-http":
|
||||
raise PermissionError("isolated_writer_gitea_remote_rejected")
|
||||
head = api.sh(["rev-parse", "HEAD"], True)["stdout"]
|
||||
pushed = subprocess.run(
|
||||
["git", "push", "--porcelain", remote, f"HEAD:refs/heads/{branch}"],
|
||||
cwd=api.WS,
|
||||
text=True,
|
||||
capture_output=True,
|
||||
)
|
||||
if pushed.returncode != 0:
|
||||
raise RuntimeError("isolated_writer_gitea_push_failed")
|
||||
verified = subprocess.run(
|
||||
["git", "ls-remote", "--heads", remote, f"refs/heads/{branch}"],
|
||||
cwd=api.WS,
|
||||
text=True,
|
||||
capture_output=True,
|
||||
)
|
||||
if verified.returncode != 0:
|
||||
raise RuntimeError("isolated_writer_gitea_verification_failed")
|
||||
fields = verified.stdout.strip().split()
|
||||
if len(fields) != 2 or fields[0] != head or fields[1] != f"refs/heads/{branch}":
|
||||
raise RuntimeError("isolated_writer_gitea_oid_mismatch")
|
||||
return {
|
||||
"status": "verified",
|
||||
"target": "authoritative-bare+gitea",
|
||||
"branch": branch,
|
||||
"oid": head,
|
||||
"local": local.get("status"),
|
||||
}
|
||||
|
||||
|
||||
def commit(paths, msg):
|
||||
policy.validate_commit_paths(paths, api.WS)
|
||||
api.ensure_git()
|
||||
selected = [api.relpath(p) for p in paths] if paths else ["."]
|
||||
api.sh(["add", *selected], True)
|
||||
policy.validate_staged_paths(api.WS)
|
||||
branch = api.sh(["rev-parse", "--abbrev-ref", "HEAD"], True)["stdout"]
|
||||
if not api.sh(["status", "--short"])["stdout"]:
|
||||
return {"status": "no_changes", "branch": branch, "sync": sync_branch(branch)}
|
||||
out = api.sh(["commit", "-m", msg], True)
|
||||
branch = api.sh(["rev-parse", "--abbrev-ref", "HEAD"], True)["stdout"]
|
||||
return {
|
||||
"status": "committed",
|
||||
"branch": branch,
|
||||
"rev": api.sh(["rev-parse", "--short", "HEAD"], True)["stdout"],
|
||||
"git": out,
|
||||
"sync": sync_branch(branch),
|
||||
}
|
||||
|
||||
|
||||
def audit(actor, action, target, payload):
|
||||
policy.append_audit_record(
|
||||
{
|
||||
"at": api.now(),
|
||||
"actor": actor,
|
||||
"action": action,
|
||||
"target": target,
|
||||
"payload": api.sanitize(payload),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _validate_request(self, data, validate):
|
||||
try:
|
||||
validate(data)
|
||||
return True
|
||||
except PermissionError as exc:
|
||||
self.sendj({"error": str(exc)}, 403)
|
||||
return False
|
||||
|
||||
|
||||
def _checked_branch(self, data):
|
||||
if not _validate_request(self, data, policy.validate_branch_request):
|
||||
return
|
||||
return _branch(self, data)
|
||||
|
||||
|
||||
def _checked_file(self, data):
|
||||
data = dict(data)
|
||||
data.setdefault("branch", policy.allowed_branch())
|
||||
if not _validate_request(self, data, policy.validate_branch_request):
|
||||
return
|
||||
if not _validate_request(
|
||||
self,
|
||||
{"path": data.get("path", "")},
|
||||
lambda item: policy.validate_path(item["path"], api.WS),
|
||||
):
|
||||
return
|
||||
return _file(self, data)
|
||||
|
||||
|
||||
def _checked_commit(self, data):
|
||||
data = dict(data)
|
||||
data.setdefault("branch", policy.allowed_branch())
|
||||
if not _validate_request(self, data, policy.validate_branch_request):
|
||||
return
|
||||
return _commit(self, data)
|
||||
|
||||
|
||||
def _checked_patch(self, data):
|
||||
data = dict(data)
|
||||
data["branch"] = policy.allowed_branch()
|
||||
if not _validate_request(self, data, policy.validate_branch_request):
|
||||
return
|
||||
return _patch(self, data)
|
||||
|
||||
|
||||
_branch, _file, _commit, _patch = api.H.branch, api.H.file, api.H.commit, api.H.patch
|
||||
api.reject_non_read_http = reject_non_read_http
|
||||
api.checkout, api.write_file, api.sync_branch, api.commit, api.audit = (
|
||||
checkout,
|
||||
write_file,
|
||||
sync_branch,
|
||||
commit,
|
||||
audit,
|
||||
)
|
||||
api.H.branch, api.H.file, api.H.commit, api.H.patch = (
|
||||
_checked_branch,
|
||||
_checked_file,
|
||||
_checked_commit,
|
||||
_checked_patch,
|
||||
)
|
||||
api.H.do_GET = _isolated_do_get
|
||||
api.H.do_POST = _isolated_do_post
|
||||
|
||||
if __name__ == "__main__":
|
||||
policy.require_service_invariants(api.WS, api.BARE_REPO)
|
||||
api.ThreadingHTTPServer((api.HOST, api.PORT), api.H).serve_forever()
|
||||
@@ -1,310 +0,0 @@
|
||||
"""Fail-closed policy used only by the task-scoped Action writer."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import json
|
||||
import stat
|
||||
import subprocess
|
||||
import re
|
||||
from pathlib import Path, PurePosixPath, PureWindowsPath
|
||||
from urllib.parse import unquote
|
||||
|
||||
import isolated_writer_path_policy as path_policy
|
||||
|
||||
WRITE_ENDPOINTS = frozenset({"/api/commit-file"})
|
||||
ALLOWED_ROOTS = (
|
||||
"active-runtime",
|
||||
"docs",
|
||||
"patches",
|
||||
"reports",
|
||||
"scripts",
|
||||
"test",
|
||||
"tests",
|
||||
)
|
||||
DENIED_ROOTS = ("baseline", "config", "deploy", "deployments", "ops", "runtime", "services")
|
||||
_TRUE = frozenset({"1", "true", "yes", "on"})
|
||||
_FALSE = frozenset({"0", "false", "no", "off"})
|
||||
_DENY_FLAGS = (
|
||||
"AIOS_GITOPS_ALLOW_MAIN_WRITE",
|
||||
"AIOS_GITOPS_ALLOW_TAG_WRITE",
|
||||
"AIOS_GITOPS_ALLOW_BASELINE_WRITE",
|
||||
"AIOS_GITOPS_ALLOW_REGISTRY_WRITE",
|
||||
"AIOS_GITOPS_ALLOW_ROLLBACK",
|
||||
"AIOS_GITOPS_ALLOW_DEPLOYMENT",
|
||||
)
|
||||
|
||||
|
||||
def enforcement_required() -> bool:
|
||||
return os.getenv("AIOS_GITOPS_ISOLATED_WRITER_ENFORCEMENT", "") == "required"
|
||||
|
||||
|
||||
def allowed_branch() -> str:
|
||||
value = os.getenv("AIOS_GITOPS_ALLOWED_BRANCH", "").strip()
|
||||
if not value:
|
||||
raise PermissionError("isolated_writer_missing_allowed_branch")
|
||||
return value
|
||||
|
||||
|
||||
def allowed_base_oid() -> str:
|
||||
value = os.getenv("AIOS_GITOPS_ALLOWED_BASE_OID", "").strip().lower()
|
||||
if len(value) != 40 or any(ch not in "0123456789abcdef" for ch in value):
|
||||
raise PermissionError("isolated_writer_invalid_allowed_base_oid")
|
||||
return value
|
||||
|
||||
|
||||
def _git(workspace: Path, *args: str) -> str:
|
||||
result = subprocess.run(["git", *args], cwd=workspace, text=True, capture_output=True)
|
||||
if result.returncode != 0:
|
||||
raise PermissionError("isolated_writer_git_validation_failed")
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
def require_service_invariants(workspace: Path, bare_repo: Path) -> None:
|
||||
if not enforcement_required():
|
||||
return
|
||||
if os.getenv("AIOS_GITOPS_WRITES_ENABLED", "").strip().lower() not in _TRUE:
|
||||
raise PermissionError("isolated_writer_write_gate_required")
|
||||
if os.getenv("AIOS_GITOPS_RUNTIME_ROLE", "") != "isolated-writer":
|
||||
raise PermissionError("isolated_writer_role_required")
|
||||
if os.getenv("AIOS_GITOPS_WORKSPACE_ROLE", "") != "isolated-writer":
|
||||
raise PermissionError("isolated_writer_workspace_role_required")
|
||||
if os.getenv("AIOS_GITOPS_TOKEN_POLICY_VERSION_REQUIRED", "") != "2":
|
||||
raise PermissionError("isolated_writer_token_policy_v2_required")
|
||||
if not os.getenv("AIOS_GITOPS_SERVICE_AUDIENCE", "").strip():
|
||||
raise PermissionError("isolated_writer_service_audience_required")
|
||||
if not os.getenv("AIOS_GITOPS_TOKEN_REGISTRY_V2", "").strip():
|
||||
raise PermissionError("isolated_writer_token_registry_required")
|
||||
if not os.getenv("AIOS_GITOPS_WRITER_OPERATION_LEDGER", "").strip():
|
||||
raise PermissionError("isolated_writer_operation_ledger_required")
|
||||
path_policy.configured_path_policy()
|
||||
for name in _DENY_FLAGS:
|
||||
if os.getenv(name, "").strip().lower() not in _FALSE:
|
||||
raise PermissionError("isolated_writer_forbidden_capability_flag")
|
||||
if os.getenv("AIOS_GITOPS_REPOSITORY_IDENTITY", "") != "aios/aios":
|
||||
raise PermissionError("isolated_writer_repository_identity_rejected")
|
||||
configured_bare_value = os.getenv("AIOS_GITOPS_AUTHORITATIVE_GIT_DIR", "").strip()
|
||||
if not configured_bare_value:
|
||||
raise PermissionError("isolated_writer_authoritative_repo_rejected")
|
||||
configured_bare = Path(configured_bare_value)
|
||||
if configured_bare.resolve(strict=True) != bare_repo.resolve(strict=True):
|
||||
raise PermissionError("isolated_writer_authoritative_repo_rejected")
|
||||
writer_root_value = os.getenv("AIOS_GITOPS_APPROVED_WRITER_ROOT", "").strip()
|
||||
if not writer_root_value:
|
||||
raise PermissionError("isolated_writer_workspace_outside_approved_root")
|
||||
writer_root = Path(writer_root_value).resolve(strict=True)
|
||||
lexical_workspace = Path(os.path.abspath(str(workspace)))
|
||||
canonical_workspace = lexical_workspace.resolve(strict=True)
|
||||
if lexical_workspace != canonical_workspace:
|
||||
raise PermissionError("isolated_writer_workspace_symlink_rejected")
|
||||
try:
|
||||
canonical_workspace.relative_to(writer_root)
|
||||
except (ValueError, FileNotFoundError):
|
||||
raise PermissionError("isolated_writer_workspace_outside_approved_root") from None
|
||||
if _git(workspace, "rev-parse", "--abbrev-ref", "HEAD") != allowed_branch():
|
||||
raise PermissionError("isolated_writer_current_branch_rejected")
|
||||
approved_base = allowed_base_oid()
|
||||
try:
|
||||
_git(workspace, "cat-file", "-e", f"{approved_base}^{{commit}}")
|
||||
subprocess.run(
|
||||
["git", "merge-base", "--is-ancestor", approved_base, "HEAD"],
|
||||
cwd=workspace,
|
||||
check=True,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
)
|
||||
except (PermissionError, subprocess.CalledProcessError):
|
||||
raise PermissionError("isolated_writer_approved_base_not_ancestor") from None
|
||||
if _git(workspace, "merge-base", approved_base, "HEAD") != approved_base:
|
||||
raise PermissionError("isolated_writer_merge_base_rejected")
|
||||
common = Path(_git(workspace, "rev-parse", "--git-common-dir"))
|
||||
if not common.is_absolute():
|
||||
common = (workspace / common).resolve()
|
||||
if common.resolve() != bare_repo.resolve():
|
||||
raise PermissionError("isolated_writer_common_repo_rejected")
|
||||
|
||||
|
||||
def approved_audit_log_path() -> Path:
|
||||
root_value = os.getenv("AIOS_GITOPS_APPROVED_AUDIT_ROOT", "").strip()
|
||||
log_value = os.getenv("AIOS_GITOPS_AUDIT_LOG", "").strip()
|
||||
if not root_value or not log_value:
|
||||
raise PermissionError("isolated_writer_audit_path_missing")
|
||||
lexical_root = Path(os.path.abspath(root_value))
|
||||
lexical_log = Path(os.path.abspath(log_value))
|
||||
root = lexical_root.resolve(strict=True)
|
||||
if lexical_root != root or root.is_symlink() or not root.is_dir():
|
||||
raise PermissionError("isolated_writer_audit_root_rejected")
|
||||
try:
|
||||
lexical_log.relative_to(root)
|
||||
except ValueError:
|
||||
raise PermissionError("isolated_writer_audit_path_outside_approved_root") from None
|
||||
if lexical_log.parent.resolve(strict=True) != lexical_log.parent:
|
||||
raise PermissionError("isolated_writer_audit_parent_symlink_rejected")
|
||||
if lexical_log.exists() and lexical_log.is_symlink():
|
||||
raise PermissionError("isolated_writer_audit_log_symlink_rejected")
|
||||
os.chmod(root, 0o750)
|
||||
os.chmod(lexical_log.parent, 0o750)
|
||||
return lexical_log
|
||||
|
||||
|
||||
def append_audit_record(record: dict) -> None:
|
||||
path = approved_audit_log_path()
|
||||
flags = os.O_WRONLY | os.O_APPEND | os.O_CREAT
|
||||
if hasattr(os, "O_NOFOLLOW"):
|
||||
flags |= os.O_NOFOLLOW
|
||||
descriptor = os.open(path, flags, 0o600)
|
||||
try:
|
||||
info = os.fstat(descriptor)
|
||||
if not stat.S_ISREG(info.st_mode):
|
||||
raise PermissionError("isolated_writer_audit_log_not_regular")
|
||||
os.fchmod(descriptor, 0o600)
|
||||
line = json.dumps(
|
||||
_audit_sanitize(record),
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
).encode("utf-8") + b"\n"
|
||||
os.write(descriptor, line)
|
||||
os.fsync(descriptor)
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
parent_fd = os.open(path.parent, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
|
||||
try:
|
||||
os.fsync(parent_fd)
|
||||
finally:
|
||||
os.close(parent_fd)
|
||||
|
||||
|
||||
def _audit_sanitize(value):
|
||||
if isinstance(value, dict):
|
||||
return {
|
||||
key: (
|
||||
"[REDACTED]"
|
||||
if any(
|
||||
marker in str(key).lower()
|
||||
for marker in (
|
||||
"authorization",
|
||||
"credential",
|
||||
"password",
|
||||
"secret",
|
||||
"token",
|
||||
"private_key",
|
||||
)
|
||||
)
|
||||
and str(key).lower() != "token_id"
|
||||
else _audit_sanitize(inner)
|
||||
)
|
||||
for key, inner in value.items()
|
||||
}
|
||||
if isinstance(value, list):
|
||||
return [_audit_sanitize(item) for item in value]
|
||||
if isinstance(value, str) and re.search(
|
||||
r"(?i)(authorization\s*:\s*bearer|(?:token|password|secret)\s*[:=])",
|
||||
value,
|
||||
):
|
||||
return "[REDACTED]"
|
||||
return value
|
||||
|
||||
|
||||
def validate_endpoint(method: str, path: str, workspace: Path, bare_repo: Path) -> None:
|
||||
if not enforcement_required():
|
||||
return
|
||||
if method != "POST" or path not in WRITE_ENDPOINTS:
|
||||
raise PermissionError("isolated_writer_operation_forbidden")
|
||||
require_service_invariants(workspace, bare_repo)
|
||||
|
||||
|
||||
def validate_branch_request(data: dict) -> None:
|
||||
if not enforcement_required():
|
||||
return
|
||||
branch = allowed_branch() if "branch" not in data else str(data.get("branch") or "").strip()
|
||||
if branch != allowed_branch():
|
||||
raise PermissionError("isolated_writer_branch_rejected")
|
||||
if str(data.get("base", "main")).strip() != "main":
|
||||
raise PermissionError("isolated_writer_base_rejected")
|
||||
if data.get("force") or data.get("force_push"):
|
||||
raise PermissionError("isolated_writer_force_rejected")
|
||||
|
||||
|
||||
def validate_path(path: str, workspace: Path | None = None) -> str:
|
||||
if not isinstance(path, str):
|
||||
raise PermissionError("isolated_writer_path_rejected")
|
||||
value = path
|
||||
if not value or value != value.strip() or "\x00" in value or "\\" in value:
|
||||
raise PermissionError("isolated_writer_path_rejected")
|
||||
try:
|
||||
decoded = unquote(value, errors="strict")
|
||||
except (UnicodeDecodeError, ValueError):
|
||||
raise PermissionError("isolated_writer_path_rejected") from None
|
||||
if decoded != value:
|
||||
raise PermissionError("isolated_writer_path_rejected")
|
||||
windows = PureWindowsPath(value)
|
||||
posix = PurePosixPath(value)
|
||||
parts = value.split("/")
|
||||
if (
|
||||
posix.is_absolute()
|
||||
or windows.is_absolute()
|
||||
or bool(windows.drive)
|
||||
or any(part in {"", ".", ".."} for part in parts)
|
||||
):
|
||||
raise PermissionError("isolated_writer_path_rejected")
|
||||
first = parts[0]
|
||||
service_exact_path = False
|
||||
if first in DENIED_ROOTS:
|
||||
try:
|
||||
path_policy.require_service_path("commit_file.atomic", value)
|
||||
service_exact_path = True
|
||||
except path_policy.PathPolicyError as exc:
|
||||
raise PermissionError(str(exc)) from None
|
||||
if value != "README.md" and first not in ALLOWED_ROOTS and not service_exact_path:
|
||||
raise PermissionError("isolated_writer_path_outside_allowed_roots")
|
||||
if workspace is not None:
|
||||
lexical_root = Path(os.path.abspath(str(workspace)))
|
||||
root = lexical_root.resolve(strict=True)
|
||||
if lexical_root != root:
|
||||
raise PermissionError("isolated_writer_workspace_symlink_rejected")
|
||||
current = root
|
||||
for part in parts:
|
||||
current = current / part
|
||||
if current.is_symlink():
|
||||
raise PermissionError("isolated_writer_path_symlink_rejected")
|
||||
candidate = (root / value).resolve(strict=False)
|
||||
try:
|
||||
candidate.relative_to(root)
|
||||
except ValueError:
|
||||
raise PermissionError("isolated_writer_path_escapes_workspace") from None
|
||||
return value
|
||||
|
||||
|
||||
def validate_content_for_path(path: str, content: str) -> None:
|
||||
if path.split("/", 1)[0] not in DENIED_ROOTS:
|
||||
return
|
||||
try:
|
||||
path_policy.validate_content("commit_file.atomic", path, content)
|
||||
except path_policy.PathPolicyError as exc:
|
||||
raise PermissionError(str(exc)) from None
|
||||
|
||||
|
||||
def configured_path_policy_sha256() -> str | None:
|
||||
try:
|
||||
return path_policy.configured_path_policy_sha256()
|
||||
except path_policy.PathPolicyError as exc:
|
||||
raise PermissionError(str(exc)) from None
|
||||
|
||||
|
||||
def validate_commit_paths(paths: list[str], workspace: Path | None = None) -> None:
|
||||
if enforcement_required():
|
||||
for path in paths:
|
||||
if str(path).strip() != ".":
|
||||
validate_path(str(path), workspace)
|
||||
|
||||
|
||||
def validate_staged_paths(workspace: Path) -> None:
|
||||
if not enforcement_required():
|
||||
return
|
||||
for path in _git(workspace, "diff", "--cached", "--name-only").splitlines():
|
||||
validate_path(path, workspace)
|
||||
|
||||
|
||||
def validate_sync_branch(branch: str) -> None:
|
||||
if enforcement_required() and branch != allowed_branch():
|
||||
raise PermissionError("isolated_writer_sync_branch_rejected")
|
||||
@@ -0,0 +1,37 @@
|
||||
"""AIOS-managed OpenClaw execution adapter."""
|
||||
|
||||
from .adapter import (
|
||||
AdapterError,
|
||||
CommandConflict,
|
||||
CommandNotFound,
|
||||
CommandRejected,
|
||||
InMemoryCallbackSink,
|
||||
OpenClawAdapter,
|
||||
)
|
||||
from .contract import (
|
||||
CommandState,
|
||||
OpenClawCommand,
|
||||
OpenClawResult,
|
||||
RuntimeHealth,
|
||||
sign_command,
|
||||
verify_command_signature,
|
||||
)
|
||||
from .transport import HTTPRuntimeTransport, InProcessRuntimeTransport, RuntimeTransport
|
||||
|
||||
__all__ = [
|
||||
"AdapterError",
|
||||
"CommandConflict",
|
||||
"CommandNotFound",
|
||||
"CommandRejected",
|
||||
"CommandState",
|
||||
"HTTPRuntimeTransport",
|
||||
"InMemoryCallbackSink",
|
||||
"InProcessRuntimeTransport",
|
||||
"OpenClawAdapter",
|
||||
"OpenClawCommand",
|
||||
"OpenClawResult",
|
||||
"RuntimeHealth",
|
||||
"RuntimeTransport",
|
||||
"sign_command",
|
||||
"verify_command_signature",
|
||||
]
|
||||
@@ -0,0 +1,228 @@
|
||||
"""Signed, tenant-isolated execution adapter for OpenClaw runtimes."""
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
from dataclasses import replace
|
||||
from typing import Callable, Mapping, Protocol
|
||||
|
||||
from .contract import (
|
||||
CommandState,
|
||||
OpenClawCommand,
|
||||
OpenClawResult,
|
||||
RuntimeHealth,
|
||||
utc_now,
|
||||
verify_command_signature,
|
||||
)
|
||||
|
||||
|
||||
class AdapterError(RuntimeError):
|
||||
"""Base adapter failure."""
|
||||
|
||||
|
||||
class CommandRejected(AdapterError):
|
||||
"""A command failed signature or contract validation."""
|
||||
|
||||
|
||||
class CommandNotFound(AdapterError):
|
||||
"""A tenant-scoped command is not visible."""
|
||||
|
||||
|
||||
class CommandConflict(AdapterError):
|
||||
"""A command or idempotency key conflicts with an existing execution."""
|
||||
|
||||
|
||||
class RuntimeTransport(Protocol):
|
||||
def execute(self, command: OpenClawCommand, timeout_seconds: float) -> OpenClawResult: ...
|
||||
def cancel(self, command_id: str) -> bool: ...
|
||||
def health(self) -> RuntimeHealth: ...
|
||||
|
||||
|
||||
class CallbackSink(Protocol):
|
||||
def on_started(self, result: OpenClawResult) -> None: ...
|
||||
def on_completed(self, result: OpenClawResult) -> None: ...
|
||||
def on_failed(self, result: OpenClawResult) -> None: ...
|
||||
|
||||
|
||||
class InMemoryCallbackSink:
|
||||
def __init__(self) -> None:
|
||||
self._lock = threading.RLock()
|
||||
self.started: list[OpenClawResult] = []
|
||||
self.completed: list[OpenClawResult] = []
|
||||
self.failed: list[OpenClawResult] = []
|
||||
|
||||
def on_started(self, result: OpenClawResult) -> None:
|
||||
with self._lock:
|
||||
self.started.append(result)
|
||||
|
||||
def on_completed(self, result: OpenClawResult) -> None:
|
||||
with self._lock:
|
||||
self.completed.append(result)
|
||||
|
||||
def on_failed(self, result: OpenClawResult) -> None:
|
||||
with self._lock:
|
||||
self.failed.append(result)
|
||||
|
||||
|
||||
SigningMaterialResolver = Callable[[str, str], bytes | str]
|
||||
|
||||
|
||||
class OpenClawAdapter:
|
||||
"""Verify, de-duplicate, execute, cancel and reconcile commands."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
transport: RuntimeTransport,
|
||||
signing_material_resolver: SigningMaterialResolver,
|
||||
*,
|
||||
callback_sink: CallbackSink | None = None,
|
||||
default_timeout_seconds: float = 300.0,
|
||||
) -> None:
|
||||
if default_timeout_seconds <= 0:
|
||||
raise ValueError("default_timeout_seconds must be positive")
|
||||
self.transport = transport
|
||||
self.signing_material_resolver = signing_material_resolver
|
||||
self.callback_sink = callback_sink or InMemoryCallbackSink()
|
||||
self.default_timeout_seconds = float(default_timeout_seconds)
|
||||
self._lock = threading.RLock()
|
||||
self._commands: dict[str, OpenClawCommand] = {}
|
||||
self._results: dict[str, OpenClawResult] = {}
|
||||
self._idempotency: dict[tuple[str, str], tuple[str, str]] = {}
|
||||
|
||||
def _verify(self, command: OpenClawCommand) -> None:
|
||||
try:
|
||||
key_material = self.signing_material_resolver(command.tenant_id, command.key_id)
|
||||
except Exception as exc:
|
||||
raise CommandRejected("signing material unavailable") from exc
|
||||
if not verify_command_signature(command, key_material):
|
||||
raise CommandRejected("invalid or expired command signature")
|
||||
|
||||
@staticmethod
|
||||
def _validate_result(command: OpenClawCommand, result: OpenClawResult) -> None:
|
||||
if (
|
||||
result.command_id != command.command_id
|
||||
or result.tenant_id != command.tenant_id
|
||||
or result.task_id != command.task_id
|
||||
or result.trace_id != command.trace_id
|
||||
):
|
||||
raise AdapterError("runtime returned mismatched command identity")
|
||||
if result.state not in {
|
||||
CommandState.COMPLETED,
|
||||
CommandState.FAILED,
|
||||
CommandState.CANCELLED,
|
||||
CommandState.TIMED_OUT,
|
||||
}:
|
||||
raise AdapterError("runtime returned a non-terminal result")
|
||||
|
||||
def execute(
|
||||
self,
|
||||
command: OpenClawCommand,
|
||||
*,
|
||||
timeout_seconds: float | None = None,
|
||||
) -> OpenClawResult:
|
||||
timeout = self.default_timeout_seconds if timeout_seconds is None else float(timeout_seconds)
|
||||
if timeout <= 0:
|
||||
raise ValueError("timeout_seconds must be positive")
|
||||
self._verify(command)
|
||||
digest = command.signature
|
||||
idem_ref = (command.tenant_id, command.idempotency_key)
|
||||
|
||||
with self._lock:
|
||||
prior_command = self._commands.get(command.command_id)
|
||||
if prior_command is not None:
|
||||
if prior_command.signature != command.signature:
|
||||
raise CommandConflict("command_id reused with different content")
|
||||
return self._results[command.command_id]
|
||||
prior_idem = self._idempotency.get(idem_ref)
|
||||
if prior_idem is not None:
|
||||
prior_id, prior_digest = prior_idem
|
||||
if prior_digest != digest:
|
||||
raise CommandConflict("idempotency key reused with different content")
|
||||
return self._results[prior_id]
|
||||
running = OpenClawResult(
|
||||
command_id=command.command_id,
|
||||
tenant_id=command.tenant_id,
|
||||
task_id=command.task_id,
|
||||
trace_id=command.trace_id,
|
||||
state=CommandState.RUNNING,
|
||||
started_at=utc_now(),
|
||||
)
|
||||
self._commands[command.command_id] = command
|
||||
self._results[command.command_id] = running
|
||||
self._idempotency[idem_ref] = (command.command_id, digest)
|
||||
|
||||
self.callback_sink.on_started(running)
|
||||
try:
|
||||
result = self.transport.execute(command, timeout)
|
||||
self._validate_result(command, result)
|
||||
except TimeoutError as exc:
|
||||
result = replace(
|
||||
running,
|
||||
state=CommandState.TIMED_OUT,
|
||||
error={"code": "runtime_timeout", "message": str(exc) or "runtime timeout"},
|
||||
finished_at=utc_now(),
|
||||
)
|
||||
except Exception as exc:
|
||||
result = replace(
|
||||
running,
|
||||
state=CommandState.FAILED,
|
||||
error={"code": "runtime_error", "message": str(exc)},
|
||||
finished_at=utc_now(),
|
||||
)
|
||||
|
||||
if result.started_at is None:
|
||||
result = replace(result, started_at=running.started_at)
|
||||
if result.finished_at is None:
|
||||
result = replace(result, finished_at=utc_now())
|
||||
with self._lock:
|
||||
self._results[command.command_id] = result
|
||||
if result.state == CommandState.COMPLETED:
|
||||
self.callback_sink.on_completed(result)
|
||||
else:
|
||||
self.callback_sink.on_failed(result)
|
||||
return result
|
||||
|
||||
def status(self, command_id: str, *, tenant_id: str) -> OpenClawResult:
|
||||
with self._lock:
|
||||
command = self._commands.get(command_id)
|
||||
result = self._results.get(command_id)
|
||||
if command is None or result is None or command.tenant_id != tenant_id:
|
||||
raise CommandNotFound("command not found")
|
||||
return result
|
||||
|
||||
def cancel(self, command_id: str, *, tenant_id: str) -> OpenClawResult:
|
||||
with self._lock:
|
||||
command = self._commands.get(command_id)
|
||||
current = self._results.get(command_id)
|
||||
if command is None or current is None or command.tenant_id != tenant_id:
|
||||
raise CommandNotFound("command not found")
|
||||
if current.state in {
|
||||
CommandState.COMPLETED,
|
||||
CommandState.FAILED,
|
||||
CommandState.CANCELLED,
|
||||
CommandState.TIMED_OUT,
|
||||
CommandState.REJECTED,
|
||||
}:
|
||||
return current
|
||||
if not self.transport.cancel(command_id):
|
||||
raise AdapterError("runtime rejected cancellation")
|
||||
cancelled = replace(
|
||||
current,
|
||||
state=CommandState.CANCELLED,
|
||||
error={"code": "cancelled", "message": "cancelled by AIOS"},
|
||||
finished_at=utc_now(),
|
||||
)
|
||||
with self._lock:
|
||||
self._results[command_id] = cancelled
|
||||
self.callback_sink.on_failed(cancelled)
|
||||
return cancelled
|
||||
|
||||
def health(self) -> RuntimeHealth:
|
||||
return self.transport.health()
|
||||
|
||||
def snapshot(self, *, tenant_id: str) -> Mapping[str, OpenClawResult]:
|
||||
with self._lock:
|
||||
return {
|
||||
command_id: result
|
||||
for command_id, result in self._results.items()
|
||||
if self._commands[command_id].tenant_id == tenant_id
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
"""Audit-oriented lifecycle callbacks for OpenClaw execution."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import threading
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Mapping, Protocol
|
||||
|
||||
from .contract import OpenClawResult, utc_now
|
||||
|
||||
|
||||
class EventPublisher(Protocol):
|
||||
def publish(self, event: Mapping[str, Any]) -> None: ...
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ExecutionEvent:
|
||||
event_type: str
|
||||
tenant_id: str
|
||||
task_id: str
|
||||
trace_id: str
|
||||
command_id: str
|
||||
state: str
|
||||
created_at: str = field(default_factory=utc_now)
|
||||
details: Mapping[str, Any] = field(default_factory=dict)
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"event_type": self.event_type,
|
||||
"tenant_id": self.tenant_id,
|
||||
"task_id": self.task_id,
|
||||
"trace_id": self.trace_id,
|
||||
"command_id": self.command_id,
|
||||
"state": self.state,
|
||||
"created_at": self.created_at,
|
||||
"details": json.loads(
|
||||
json.dumps(dict(self.details), ensure_ascii=False, default=str)
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
class InMemoryEventPublisher:
|
||||
"""Thread-safe publisher used by tests and first integration slice."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._lock = threading.RLock()
|
||||
self.events: list[dict[str, Any]] = []
|
||||
|
||||
def publish(self, event: Mapping[str, Any]) -> None:
|
||||
with self._lock:
|
||||
self.events.append(
|
||||
json.loads(json.dumps(dict(event), ensure_ascii=False, default=str))
|
||||
)
|
||||
|
||||
|
||||
class AuditCallbackSink:
|
||||
"""Convert runtime callbacks into tenant-scoped audit events."""
|
||||
|
||||
def __init__(self, publisher: EventPublisher) -> None:
|
||||
self.publisher = publisher
|
||||
|
||||
@staticmethod
|
||||
def _event(
|
||||
event_type: str,
|
||||
result: OpenClawResult,
|
||||
) -> ExecutionEvent:
|
||||
details: dict[str, Any] = {
|
||||
"runtime_id": result.runtime_id,
|
||||
"attempts": result.attempts,
|
||||
}
|
||||
if result.error:
|
||||
details["error"] = dict(result.error)
|
||||
if result.output:
|
||||
details["output_present"] = True
|
||||
return ExecutionEvent(
|
||||
event_type=event_type,
|
||||
tenant_id=result.tenant_id,
|
||||
task_id=result.task_id,
|
||||
trace_id=result.trace_id,
|
||||
command_id=result.command_id,
|
||||
state=result.state.value,
|
||||
details=details,
|
||||
)
|
||||
|
||||
def on_started(self, result: OpenClawResult) -> None:
|
||||
self.publisher.publish(self._event("OPENCLAW_STARTED", result).as_dict())
|
||||
|
||||
def on_completed(self, result: OpenClawResult) -> None:
|
||||
self.publisher.publish(self._event("OPENCLAW_COMPLETED", result).as_dict())
|
||||
|
||||
def on_failed(self, result: OpenClawResult) -> None:
|
||||
event_type = {
|
||||
"CANCELLED": "OPENCLAW_CANCELLED",
|
||||
"TIMED_OUT": "OPENCLAW_TIMED_OUT",
|
||||
"REJECTED": "OPENCLAW_REJECTED",
|
||||
}.get(result.state.value, "OPENCLAW_FAILED")
|
||||
self.publisher.publish(self._event(event_type, result).as_dict())
|
||||
@@ -0,0 +1,213 @@
|
||||
"""Versioned AIOS to OpenClaw execution contracts."""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import uuid
|
||||
from dataclasses import dataclass, field, replace
|
||||
from datetime import datetime, timezone
|
||||
from enum import Enum
|
||||
from typing import Any, Mapping
|
||||
|
||||
CONTRACT_VERSION = "aios.openclaw.command.v1"
|
||||
DEFAULT_CLOCK_SKEW_SECONDS = 300
|
||||
|
||||
|
||||
def utc_now() -> str:
|
||||
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||
|
||||
|
||||
def _parse_timestamp(value: str) -> datetime:
|
||||
normalized = str(value or "").strip()
|
||||
if not normalized:
|
||||
raise ValueError("timestamp is required")
|
||||
if normalized.endswith("Z"):
|
||||
normalized = normalized[:-1] + "+00:00"
|
||||
parsed = datetime.fromisoformat(normalized)
|
||||
if parsed.tzinfo is None:
|
||||
raise ValueError("timestamp must include a timezone")
|
||||
return parsed.astimezone(timezone.utc)
|
||||
|
||||
|
||||
def _required(value: str, label: str) -> str:
|
||||
normalized = str(value or "").strip()
|
||||
if not normalized:
|
||||
raise ValueError(f"{label} is required")
|
||||
return normalized
|
||||
|
||||
|
||||
def _safe_mapping(value: Mapping[str, Any] | None) -> dict[str, Any]:
|
||||
return json.loads(json.dumps(dict(value or {}), ensure_ascii=False, sort_keys=True, default=str))
|
||||
|
||||
|
||||
class CommandState(str, Enum):
|
||||
ACCEPTED = "ACCEPTED"
|
||||
RUNNING = "RUNNING"
|
||||
COMPLETED = "COMPLETED"
|
||||
FAILED = "FAILED"
|
||||
CANCELLED = "CANCELLED"
|
||||
TIMED_OUT = "TIMED_OUT"
|
||||
REJECTED = "REJECTED"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class OpenClawCommand:
|
||||
command_id: str
|
||||
tenant_id: str
|
||||
task_id: str
|
||||
trace_id: str
|
||||
action: str
|
||||
payload: Mapping[str, Any] = field(default_factory=dict)
|
||||
idempotency_key: str = ""
|
||||
created_at: str = field(default_factory=utc_now)
|
||||
expires_at: str | None = None
|
||||
contract_version: str = CONTRACT_VERSION
|
||||
key_id: str = "default"
|
||||
signature: str = ""
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
for field_name in (
|
||||
"command_id", "tenant_id", "task_id", "trace_id", "action",
|
||||
"idempotency_key", "created_at", "contract_version", "key_id",
|
||||
):
|
||||
object.__setattr__(self, field_name, _required(getattr(self, field_name), field_name))
|
||||
if self.contract_version != CONTRACT_VERSION:
|
||||
raise ValueError(f"unsupported contract_version: {self.contract_version}")
|
||||
object.__setattr__(self, "payload", _safe_mapping(self.payload))
|
||||
_parse_timestamp(self.created_at)
|
||||
if self.expires_at is not None:
|
||||
object.__setattr__(self, "expires_at", _required(self.expires_at, "expires_at"))
|
||||
_parse_timestamp(self.expires_at)
|
||||
|
||||
@classmethod
|
||||
def create(
|
||||
cls,
|
||||
*,
|
||||
tenant_id: str,
|
||||
task_id: str,
|
||||
trace_id: str,
|
||||
action: str,
|
||||
payload: Mapping[str, Any] | None,
|
||||
idempotency_key: str,
|
||||
expires_at: str | None = None,
|
||||
key_id: str = "default",
|
||||
command_id: str | None = None,
|
||||
) -> "OpenClawCommand":
|
||||
return cls(
|
||||
command_id=command_id or f"cmd_{uuid.uuid4().hex}",
|
||||
tenant_id=tenant_id,
|
||||
task_id=task_id,
|
||||
trace_id=trace_id,
|
||||
action=action,
|
||||
payload=_safe_mapping(payload),
|
||||
idempotency_key=idempotency_key,
|
||||
expires_at=expires_at,
|
||||
key_id=key_id,
|
||||
)
|
||||
|
||||
def unsigned_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"command_id": self.command_id,
|
||||
"tenant_id": self.tenant_id,
|
||||
"task_id": self.task_id,
|
||||
"trace_id": self.trace_id,
|
||||
"action": self.action,
|
||||
"payload": _safe_mapping(self.payload),
|
||||
"idempotency_key": self.idempotency_key,
|
||||
"created_at": self.created_at,
|
||||
"expires_at": self.expires_at,
|
||||
"contract_version": self.contract_version,
|
||||
"key_id": self.key_id,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class OpenClawResult:
|
||||
command_id: str
|
||||
tenant_id: str
|
||||
task_id: str
|
||||
trace_id: str
|
||||
state: CommandState
|
||||
output: Mapping[str, Any] | None = None
|
||||
error: Mapping[str, Any] | None = None
|
||||
started_at: str | None = None
|
||||
finished_at: str | None = None
|
||||
runtime_id: str | None = None
|
||||
attempts: int = 1
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
for field_name in ("command_id", "tenant_id", "task_id", "trace_id"):
|
||||
object.__setattr__(self, field_name, _required(getattr(self, field_name), field_name))
|
||||
object.__setattr__(self, "state", CommandState(self.state))
|
||||
if self.output is not None:
|
||||
object.__setattr__(self, "output", _safe_mapping(self.output))
|
||||
if self.error is not None:
|
||||
object.__setattr__(self, "error", _safe_mapping(self.error))
|
||||
if self.started_at is not None:
|
||||
_parse_timestamp(self.started_at)
|
||||
if self.finished_at is not None:
|
||||
_parse_timestamp(self.finished_at)
|
||||
if self.attempts < 1:
|
||||
raise ValueError("attempts must be >= 1")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RuntimeHealth:
|
||||
runtime: str
|
||||
runtime_id: str
|
||||
status: str
|
||||
version: str
|
||||
heartbeat_at: str
|
||||
active_tasks: int = 0
|
||||
workspace_isolated: bool = False
|
||||
credential_ref_configured: bool = False
|
||||
details: Mapping[str, Any] = field(default_factory=dict)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
for field_name in ("runtime", "runtime_id", "status", "version", "heartbeat_at"):
|
||||
object.__setattr__(self, field_name, _required(getattr(self, field_name), field_name))
|
||||
_parse_timestamp(self.heartbeat_at)
|
||||
if self.active_tasks < 0:
|
||||
raise ValueError("active_tasks must be >= 0")
|
||||
object.__setattr__(self, "details", _safe_mapping(self.details))
|
||||
|
||||
|
||||
def canonical_command_bytes(command: OpenClawCommand) -> bytes:
|
||||
return json.dumps(
|
||||
command.unsigned_dict(), ensure_ascii=False, sort_keys=True, separators=(",", ":")
|
||||
).encode("utf-8")
|
||||
|
||||
|
||||
def sign_command(command: OpenClawCommand, key_material: bytes | str) -> OpenClawCommand:
|
||||
key = key_material.encode("utf-8") if isinstance(key_material, str) else bytes(key_material)
|
||||
if not key:
|
||||
raise ValueError("signing material is required")
|
||||
signature = hmac.new(key, canonical_command_bytes(command), hashlib.sha256).hexdigest()
|
||||
return replace(command, signature=signature)
|
||||
|
||||
|
||||
def verify_command_signature(
|
||||
command: OpenClawCommand,
|
||||
key_material: bytes | str,
|
||||
*,
|
||||
now: datetime | None = None,
|
||||
clock_skew_seconds: int = DEFAULT_CLOCK_SKEW_SECONDS,
|
||||
) -> bool:
|
||||
if not command.signature:
|
||||
return False
|
||||
key = key_material.encode("utf-8") if isinstance(key_material, str) else bytes(key_material)
|
||||
if not key:
|
||||
return False
|
||||
expected = hmac.new(key, canonical_command_bytes(command), hashlib.sha256).hexdigest()
|
||||
if not hmac.compare_digest(expected, command.signature):
|
||||
return False
|
||||
current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
|
||||
created = _parse_timestamp(command.created_at)
|
||||
if created.timestamp() - current.timestamp() > clock_skew_seconds:
|
||||
return False
|
||||
if command.expires_at is not None:
|
||||
expires = _parse_timestamp(command.expires_at)
|
||||
if current.timestamp() - expires.timestamp() > clock_skew_seconds:
|
||||
return False
|
||||
return True
|
||||
@@ -0,0 +1,103 @@
|
||||
"""Health policy and self-healing decisions for OpenClaw runtimes."""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from typing import Mapping
|
||||
|
||||
from .contract import RuntimeHealth
|
||||
|
||||
|
||||
def _parse(value: str) -> datetime:
|
||||
normalized = value[:-1] + "+00:00" if value.endswith("Z") else value
|
||||
parsed = datetime.fromisoformat(normalized)
|
||||
if parsed.tzinfo is None:
|
||||
raise ValueError("heartbeat timestamp must include timezone")
|
||||
return parsed.astimezone(timezone.utc)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class HealthPolicy:
|
||||
max_heartbeat_age_seconds: int = 90
|
||||
require_workspace_isolation: bool = True
|
||||
require_credential_ref: bool = True
|
||||
accepted_statuses: frozenset[str] = field(
|
||||
default_factory=lambda: frozenset({"healthy", "ready"})
|
||||
)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.max_heartbeat_age_seconds < 5:
|
||||
raise ValueError("max_heartbeat_age_seconds must be >= 5")
|
||||
if not self.accepted_statuses:
|
||||
raise ValueError("accepted_statuses must not be empty")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class HealthAssessment:
|
||||
healthy: bool
|
||||
reasons: tuple[str, ...]
|
||||
recoverable: bool
|
||||
recommended_action: str
|
||||
details: Mapping[str, object] = field(default_factory=dict)
|
||||
|
||||
|
||||
class RuntimeHealthMonitor:
|
||||
"""Evaluate runtime health and recommend bounded recovery actions."""
|
||||
|
||||
def __init__(self, policy: HealthPolicy | None = None) -> None:
|
||||
self.policy = policy or HealthPolicy()
|
||||
|
||||
def assess(
|
||||
self,
|
||||
health: RuntimeHealth,
|
||||
*,
|
||||
now: datetime | None = None,
|
||||
) -> HealthAssessment:
|
||||
current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
|
||||
heartbeat = _parse(health.heartbeat_at)
|
||||
age = max(0.0, current.timestamp() - heartbeat.timestamp())
|
||||
reasons: list[str] = []
|
||||
|
||||
if health.status.lower() not in self.policy.accepted_statuses:
|
||||
reasons.append("runtime_status_unhealthy")
|
||||
if age > self.policy.max_heartbeat_age_seconds:
|
||||
reasons.append("heartbeat_stale")
|
||||
if self.policy.require_workspace_isolation and not health.workspace_isolated:
|
||||
reasons.append("workspace_not_isolated")
|
||||
if self.policy.require_credential_ref and not health.credential_ref_configured:
|
||||
reasons.append("credential_reference_missing")
|
||||
|
||||
if not reasons:
|
||||
return HealthAssessment(
|
||||
healthy=True,
|
||||
reasons=(),
|
||||
recoverable=True,
|
||||
recommended_action="none",
|
||||
details={"heartbeat_age_seconds": age},
|
||||
)
|
||||
|
||||
non_recoverable = {
|
||||
"workspace_not_isolated",
|
||||
"credential_reference_missing",
|
||||
}.intersection(reasons)
|
||||
if non_recoverable:
|
||||
action = "quarantine_runtime"
|
||||
recoverable = False
|
||||
elif "heartbeat_stale" in reasons:
|
||||
action = "restart_runtime"
|
||||
recoverable = True
|
||||
else:
|
||||
action = "drain_and_restart"
|
||||
recoverable = True
|
||||
|
||||
return HealthAssessment(
|
||||
healthy=False,
|
||||
reasons=tuple(reasons),
|
||||
recoverable=recoverable,
|
||||
recommended_action=action,
|
||||
details={
|
||||
"heartbeat_age_seconds": age,
|
||||
"runtime_id": health.runtime_id,
|
||||
"active_tasks": health.active_tasks,
|
||||
},
|
||||
)
|
||||
@@ -0,0 +1,214 @@
|
||||
"""Runtime transport implementations for the OpenClaw adapter."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import threading
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from typing import Any, Callable, Mapping, Protocol
|
||||
|
||||
from .contract import (
|
||||
CommandState,
|
||||
OpenClawCommand,
|
||||
OpenClawResult,
|
||||
RuntimeHealth,
|
||||
utc_now,
|
||||
)
|
||||
|
||||
|
||||
class RuntimeTransport(Protocol):
|
||||
def execute(self, command: OpenClawCommand, timeout_seconds: float) -> OpenClawResult: ...
|
||||
def cancel(self, command_id: str) -> bool: ...
|
||||
def health(self) -> RuntimeHealth: ...
|
||||
|
||||
|
||||
class TransportError(RuntimeError):
|
||||
"""Runtime transport or protocol failure."""
|
||||
|
||||
|
||||
RuntimeHandler = Callable[[OpenClawCommand], Mapping[str, Any] | OpenClawResult]
|
||||
|
||||
|
||||
class InProcessRuntimeTransport:
|
||||
"""Deterministic runtime used by tests and controlled local integrations."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
handler: RuntimeHandler,
|
||||
*,
|
||||
runtime_id: str = "openclaw-in-process",
|
||||
version: str = "v1",
|
||||
workspace_isolated: bool = True,
|
||||
credential_ref_configured: bool = True,
|
||||
) -> None:
|
||||
self.handler = handler
|
||||
self.runtime_id = runtime_id
|
||||
self.version = version
|
||||
self.workspace_isolated = workspace_isolated
|
||||
self.credential_ref_configured = credential_ref_configured
|
||||
self._lock = threading.RLock()
|
||||
self._running: set[str] = set()
|
||||
self._cancelled: set[str] = set()
|
||||
|
||||
def execute(self, command: OpenClawCommand, timeout_seconds: float) -> OpenClawResult:
|
||||
if timeout_seconds <= 0:
|
||||
raise ValueError("timeout_seconds must be positive")
|
||||
with self._lock:
|
||||
if command.command_id in self._cancelled:
|
||||
return OpenClawResult(
|
||||
command_id=command.command_id,
|
||||
tenant_id=command.tenant_id,
|
||||
task_id=command.task_id,
|
||||
trace_id=command.trace_id,
|
||||
state=CommandState.CANCELLED,
|
||||
error={"code": "cancelled", "message": "cancelled before execution"},
|
||||
finished_at=utc_now(),
|
||||
runtime_id=self.runtime_id,
|
||||
)
|
||||
self._running.add(command.command_id)
|
||||
try:
|
||||
raw = self.handler(command)
|
||||
if isinstance(raw, OpenClawResult):
|
||||
return raw
|
||||
if not isinstance(raw, Mapping):
|
||||
raise TransportError("runtime handler must return a mapping or OpenClawResult")
|
||||
return OpenClawResult(
|
||||
command_id=command.command_id,
|
||||
tenant_id=command.tenant_id,
|
||||
task_id=command.task_id,
|
||||
trace_id=command.trace_id,
|
||||
state=CommandState(str(raw.get("state", "COMPLETED"))),
|
||||
output=raw.get("output"),
|
||||
error=raw.get("error"),
|
||||
started_at=str(raw.get("started_at") or utc_now()),
|
||||
finished_at=str(raw.get("finished_at") or utc_now()),
|
||||
runtime_id=str(raw.get("runtime_id") or self.runtime_id),
|
||||
attempts=int(raw.get("attempts", 1)),
|
||||
)
|
||||
finally:
|
||||
with self._lock:
|
||||
self._running.discard(command.command_id)
|
||||
|
||||
def cancel(self, command_id: str) -> bool:
|
||||
with self._lock:
|
||||
self._cancelled.add(command_id)
|
||||
return command_id in self._running
|
||||
|
||||
def health(self) -> RuntimeHealth:
|
||||
with self._lock:
|
||||
active = len(self._running)
|
||||
return RuntimeHealth(
|
||||
runtime="openclaw",
|
||||
runtime_id=self.runtime_id,
|
||||
status="healthy",
|
||||
version=self.version,
|
||||
heartbeat_at=utc_now(),
|
||||
active_tasks=active,
|
||||
workspace_isolated=self.workspace_isolated,
|
||||
credential_ref_configured=self.credential_ref_configured,
|
||||
details={"transport": "in_process"},
|
||||
)
|
||||
|
||||
|
||||
class HTTPRuntimeTransport:
|
||||
"""Private HTTP transport for an OpenClaw runtime service."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
base_url: str,
|
||||
*,
|
||||
runtime_id: str = "openclaw-http",
|
||||
credential_ref: str | None = None,
|
||||
max_response_bytes: int = 2_097_152,
|
||||
) -> None:
|
||||
normalized = str(base_url).rstrip("/")
|
||||
if not normalized.startswith(("http://", "https://")):
|
||||
raise ValueError("base_url must use http or https")
|
||||
self.base_url = normalized
|
||||
self.runtime_id = runtime_id
|
||||
self.credential_ref = credential_ref
|
||||
self.max_response_bytes = int(max_response_bytes)
|
||||
|
||||
def _request(
|
||||
self,
|
||||
method: str,
|
||||
path: str,
|
||||
*,
|
||||
payload: Mapping[str, Any] | None,
|
||||
timeout_seconds: float,
|
||||
) -> Mapping[str, Any]:
|
||||
body = None
|
||||
headers = {"Accept": "application/json", "User-Agent": "AIOS-OpenClaw/1"}
|
||||
if payload is not None:
|
||||
body = json.dumps(dict(payload), separators=(",", ":")).encode("utf-8")
|
||||
headers["Content-Type"] = "application/json"
|
||||
if self.credential_ref:
|
||||
headers["X-AIOS-Credential-Ref"] = self.credential_ref
|
||||
request = urllib.request.Request(
|
||||
f"{self.base_url}{path}",
|
||||
data=body,
|
||||
method=method,
|
||||
headers=headers,
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=timeout_seconds) as response:
|
||||
raw = response.read(self.max_response_bytes + 1)
|
||||
except urllib.error.HTTPError as exc:
|
||||
raise TransportError(f"runtime returned HTTP {exc.code}") from exc
|
||||
except urllib.error.URLError as exc:
|
||||
raise TransportError(f"runtime unavailable: {exc.reason}") from exc
|
||||
if len(raw) > self.max_response_bytes:
|
||||
raise TransportError("runtime response exceeded size limit")
|
||||
decoded = json.loads(raw.decode("utf-8"))
|
||||
if not isinstance(decoded, Mapping):
|
||||
raise TransportError("runtime response must be a JSON object")
|
||||
return decoded
|
||||
|
||||
@staticmethod
|
||||
def _command_payload(command: OpenClawCommand) -> dict[str, Any]:
|
||||
return {**command.unsigned_dict(), "signature": command.signature}
|
||||
|
||||
def execute(self, command: OpenClawCommand, timeout_seconds: float) -> OpenClawResult:
|
||||
value = self._request(
|
||||
"POST",
|
||||
"/v1/execute",
|
||||
payload=self._command_payload(command),
|
||||
timeout_seconds=timeout_seconds,
|
||||
)
|
||||
return OpenClawResult(
|
||||
command_id=str(value["command_id"]),
|
||||
tenant_id=str(value["tenant_id"]),
|
||||
task_id=str(value["task_id"]),
|
||||
trace_id=str(value["trace_id"]),
|
||||
state=CommandState(str(value["state"])),
|
||||
output=value.get("output"),
|
||||
error=value.get("error"),
|
||||
started_at=value.get("started_at"),
|
||||
finished_at=value.get("finished_at"),
|
||||
runtime_id=str(value.get("runtime_id") or self.runtime_id),
|
||||
attempts=int(value.get("attempts", 1)),
|
||||
)
|
||||
|
||||
def cancel(self, command_id: str) -> bool:
|
||||
value = self._request(
|
||||
"POST",
|
||||
f"/v1/commands/{urllib.parse.quote(command_id, safe='')}/cancel",
|
||||
payload={},
|
||||
timeout_seconds=10.0,
|
||||
)
|
||||
return bool(value.get("accepted"))
|
||||
|
||||
def health(self) -> RuntimeHealth:
|
||||
value = self._request("GET", "/v1/health", payload=None, timeout_seconds=5.0)
|
||||
return RuntimeHealth(
|
||||
runtime=str(value.get("runtime") or "openclaw"),
|
||||
runtime_id=str(value.get("runtime_id") or self.runtime_id),
|
||||
status=str(value.get("status") or "unknown"),
|
||||
version=str(value.get("version") or "unknown"),
|
||||
heartbeat_at=str(value.get("heartbeat_at") or utc_now()),
|
||||
active_tasks=int(value.get("active_tasks", 0)),
|
||||
workspace_isolated=bool(value.get("workspace_isolated", False)),
|
||||
credential_ref_configured=bool(value.get("credential_ref_configured", False)),
|
||||
details=value.get("details") if isinstance(value.get("details"), Mapping) else {},
|
||||
)
|
||||
@@ -0,0 +1,46 @@
|
||||
"""AIOS V2 control plane and execution orchestration."""
|
||||
|
||||
from .control_plane import (
|
||||
AgentRecord,
|
||||
AuditEvent,
|
||||
ControlPlaneError,
|
||||
ControlPlaneService,
|
||||
IdempotencyConflict,
|
||||
InMemoryControlPlaneStore,
|
||||
InvalidTransition,
|
||||
NotFound,
|
||||
PolicyDenied,
|
||||
RequestContext,
|
||||
RevisionConflict,
|
||||
TaskRecord,
|
||||
TaskState,
|
||||
TenantIsolationError,
|
||||
TenantRecord,
|
||||
Unauthorized,
|
||||
)
|
||||
from .execution_tracking import ExecutionEvidence, ExecutionTracker
|
||||
from .orchestrator import DispatchRecord, ExecutionOrchestrator, OrchestrationError
|
||||
|
||||
__all__ = [
|
||||
"AgentRecord",
|
||||
"AuditEvent",
|
||||
"ControlPlaneError",
|
||||
"ControlPlaneService",
|
||||
"DispatchRecord",
|
||||
"ExecutionEvidence",
|
||||
"ExecutionOrchestrator",
|
||||
"ExecutionTracker",
|
||||
"IdempotencyConflict",
|
||||
"InMemoryControlPlaneStore",
|
||||
"InvalidTransition",
|
||||
"NotFound",
|
||||
"OrchestrationError",
|
||||
"PolicyDenied",
|
||||
"RequestContext",
|
||||
"RevisionConflict",
|
||||
"TaskRecord",
|
||||
"TaskState",
|
||||
"TenantIsolationError",
|
||||
"TenantRecord",
|
||||
"Unauthorized",
|
||||
]
|
||||
@@ -0,0 +1,357 @@
|
||||
"""Production-candidate AIOS V2 application entrypoint.
|
||||
|
||||
This process owns the public control-plane API and read-only web console. It
|
||||
loads durable control-plane state, dispatches approved work through runtime
|
||||
adapters, exposes tenant-scoped execution evidence and sanitized GitOps status,
|
||||
and never exposes runtime credentials to the browser.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import json
|
||||
import os
|
||||
import stat
|
||||
import urllib.parse
|
||||
from enum import Enum
|
||||
from http import HTTPStatus
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from typing import Any, Mapping
|
||||
|
||||
from aios_hermes import HermesAdapter, HermesJobState, HermesResult
|
||||
from aios_openclaw import HTTPRuntimeTransport, OpenClawAdapter
|
||||
from aios_openclaw.callback import AuditCallbackSink, EventPublisher
|
||||
|
||||
from .console import CONSOLE_HTML
|
||||
from .control_plane import AuditEvent, RequestContext
|
||||
from .execution_tracking import ExecutionTracker
|
||||
from .gateway import GatewayApplication, error_response
|
||||
from .gitops_status import collect_gitops_status
|
||||
from .orchestrator import ExecutionOrchestrator
|
||||
from .persistence import DurableControlPlaneService, SQLiteSnapshotRepository
|
||||
|
||||
|
||||
def _json_value(value: Any) -> Any:
|
||||
if dataclasses.is_dataclass(value):
|
||||
return {
|
||||
item.name: _json_value(getattr(value, item.name))
|
||||
for item in dataclasses.fields(value)
|
||||
}
|
||||
if isinstance(value, Enum):
|
||||
return value.value
|
||||
if isinstance(value, Mapping):
|
||||
return {str(key): _json_value(item) for key, item in value.items()}
|
||||
if isinstance(value, (list, tuple, set, frozenset)):
|
||||
return [_json_value(item) for item in value]
|
||||
return value
|
||||
|
||||
|
||||
class FileSigningMaterialResolver:
|
||||
"""Resolve runtime signing material from a root-owned file reference."""
|
||||
|
||||
def __init__(self, path: str | Path) -> None:
|
||||
self.path = Path(path).resolve()
|
||||
|
||||
def __call__(self, tenant_id: str, key_id: str) -> bytes:
|
||||
del tenant_id, key_id
|
||||
metadata = self.path.stat()
|
||||
if not stat.S_ISREG(metadata.st_mode):
|
||||
raise RuntimeError("signing material reference is not a regular file")
|
||||
if metadata.st_mode & 0o077:
|
||||
raise RuntimeError("signing material file permissions must be 0600 or stricter")
|
||||
value = self.path.read_bytes().strip()
|
||||
if len(value) < 32:
|
||||
raise RuntimeError("signing material is too short")
|
||||
return value
|
||||
|
||||
|
||||
class ControlPlaneEventPublisher(EventPublisher):
|
||||
"""Append OpenClaw runtime events to the tenant audit stream."""
|
||||
|
||||
def __init__(self, service: DurableControlPlaneService) -> None:
|
||||
self.service = service
|
||||
|
||||
def publish(self, event: Mapping[str, Any]) -> None:
|
||||
tenant_id = str(event.get("tenant_id") or "").strip()
|
||||
trace_id = str(event.get("trace_id") or "").strip()
|
||||
command_id = str(event.get("command_id") or "").strip()
|
||||
event_type = str(event.get("event_type") or "OPENCLAW_EVENT").strip()
|
||||
if not tenant_id or not trace_id or not command_id:
|
||||
raise ValueError("runtime audit event is missing tenant identity")
|
||||
state_value = str(event.get("state") or "unknown").lower()
|
||||
outcome = "allowed" if state_value == "completed" else state_value
|
||||
audit = AuditEvent(
|
||||
event_id=f"runtime_{command_id}_{len(self.service.store.audit_events) + 1}",
|
||||
tenant_id=tenant_id,
|
||||
principal_id="openclaw-runtime",
|
||||
trace_id=trace_id,
|
||||
action=event_type,
|
||||
resource_type="openclaw_command",
|
||||
resource_id=command_id,
|
||||
outcome=outcome,
|
||||
policy_version="runtime-v1",
|
||||
details=dict(event.get("details") or {}),
|
||||
)
|
||||
with self.service.store.lock:
|
||||
self.service.store.audit_events.append(audit)
|
||||
self.service.repository.save(self.service._snapshot())
|
||||
|
||||
|
||||
class CandidateApplication:
|
||||
"""Compose durable control plane, runtimes, evidence and console routes."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
gateway: GatewayApplication,
|
||||
orchestrator: ExecutionOrchestrator,
|
||||
tracker: ExecutionTracker,
|
||||
hermes: HermesAdapter,
|
||||
openclaw: OpenClawAdapter | None,
|
||||
) -> None:
|
||||
self.gateway = gateway
|
||||
self.orchestrator = orchestrator
|
||||
self.tracker = tracker
|
||||
self.hermes = hermes
|
||||
self.openclaw = openclaw
|
||||
|
||||
def context(self, headers: Mapping[str, str]) -> RequestContext:
|
||||
return self.gateway.context_from_headers(headers)
|
||||
|
||||
def handle_json(
|
||||
self,
|
||||
method: str,
|
||||
raw_path: str,
|
||||
headers: Mapping[str, str],
|
||||
body: Any,
|
||||
) -> tuple[int, Mapping[str, Any]]:
|
||||
parsed = urllib.parse.urlparse(raw_path)
|
||||
path = parsed.path.rstrip("/") or "/"
|
||||
method = method.upper()
|
||||
|
||||
if method == "GET" and path == "/api/v2/gitops":
|
||||
context = self.context(headers)
|
||||
return HTTPStatus.OK, {
|
||||
"tenant_id": context.tenant_id,
|
||||
"gitops": collect_gitops_status().as_dict(),
|
||||
}
|
||||
|
||||
if method == "GET" and path == "/api/v2/runtime":
|
||||
context = self.context(headers)
|
||||
runtimes: list[dict[str, Any]] = []
|
||||
if self.openclaw is None:
|
||||
runtimes.append(
|
||||
{
|
||||
"name": "openclaw",
|
||||
"status": "disabled",
|
||||
"details": {"reason": "runtime_not_configured"},
|
||||
}
|
||||
)
|
||||
else:
|
||||
try:
|
||||
health = self.openclaw.health()
|
||||
runtimes.append(
|
||||
{
|
||||
"name": "openclaw",
|
||||
"status": health.status,
|
||||
"details": _json_value(health),
|
||||
}
|
||||
)
|
||||
except Exception as exc:
|
||||
runtimes.append(
|
||||
{
|
||||
"name": "openclaw",
|
||||
"status": "unavailable",
|
||||
"details": {"message": str(exc)},
|
||||
}
|
||||
)
|
||||
hermes_health = self.hermes.health()
|
||||
runtimes.append(
|
||||
{
|
||||
"name": "hermes",
|
||||
"status": hermes_health.status,
|
||||
"details": _json_value(hermes_health),
|
||||
}
|
||||
)
|
||||
return HTTPStatus.OK, {
|
||||
"tenant_id": context.tenant_id,
|
||||
"runtimes": runtimes,
|
||||
}
|
||||
|
||||
if method == "GET" and path == "/api/v2/executions":
|
||||
context = self.context(headers)
|
||||
return HTTPStatus.OK, {
|
||||
"executions": _json_value(
|
||||
self.tracker.list(tenant_id=context.tenant_id)
|
||||
)
|
||||
}
|
||||
|
||||
dispatch_match = __import__("re").fullmatch(
|
||||
r"/api/v2/tasks/([^/]+)/dispatch",
|
||||
path,
|
||||
)
|
||||
if method == "POST" and dispatch_match:
|
||||
context = self.context(headers)
|
||||
task_id = urllib.parse.unquote(dispatch_match.group(1))
|
||||
payload = body if isinstance(body, Mapping) else {}
|
||||
timeout = payload.get("timeout_seconds")
|
||||
record = self.orchestrator.dispatch(
|
||||
context,
|
||||
task_id=task_id,
|
||||
timeout_seconds=float(timeout) if timeout is not None else None,
|
||||
)
|
||||
return HTTPStatus.ACCEPTED, {"dispatch": _json_value(record)}
|
||||
|
||||
if method == "POST" and path == "/api/v2/hermes/results/reconcile":
|
||||
context = self.context(headers)
|
||||
if not isinstance(body, Mapping):
|
||||
raise ValueError("JSON body must be an object")
|
||||
result = HermesResult(
|
||||
job_id=str(body["job_id"]),
|
||||
tenant_id=str(body["tenant_id"]),
|
||||
task_id=str(body["task_id"]),
|
||||
trace_id=str(body["trace_id"]),
|
||||
state=HermesJobState(str(body["state"])),
|
||||
worker_id=(
|
||||
str(body["worker_id"])
|
||||
if body.get("worker_id") is not None
|
||||
else None
|
||||
),
|
||||
output=body.get("output"),
|
||||
error=body.get("error"),
|
||||
started_at=body.get("started_at"),
|
||||
finished_at=body.get("finished_at"),
|
||||
)
|
||||
task = self.orchestrator.reconcile_hermes(context, result)
|
||||
return HTTPStatus.OK, {"task": _json_value(task)}
|
||||
|
||||
return self.gateway.handle(method, raw_path, headers, body)
|
||||
|
||||
|
||||
class CandidateRequestHandler(BaseHTTPRequestHandler):
|
||||
server_version = "AIOSV2Candidate/1.0"
|
||||
max_body_bytes = 1_048_576
|
||||
|
||||
@property
|
||||
def application(self) -> CandidateApplication:
|
||||
return self.server.application # type: ignore[attr-defined]
|
||||
|
||||
def _read_body(self) -> Any:
|
||||
length = int(self.headers.get("Content-Length", "0") or "0")
|
||||
if length < 0 or length > self.max_body_bytes:
|
||||
raise ValueError("request body is too large")
|
||||
if length == 0:
|
||||
return {}
|
||||
content_type = self.headers.get("Content-Type", "").split(";", 1)[0].strip()
|
||||
if content_type != "application/json":
|
||||
raise ValueError("Content-Type must be application/json")
|
||||
return json.loads(self.rfile.read(length).decode("utf-8"))
|
||||
|
||||
def _security_headers(self) -> None:
|
||||
self.send_header("Cache-Control", "no-store")
|
||||
self.send_header("X-Content-Type-Options", "nosniff")
|
||||
self.send_header("X-Frame-Options", "DENY")
|
||||
self.send_header("Referrer-Policy", "no-referrer")
|
||||
self.send_header(
|
||||
"Content-Security-Policy",
|
||||
"default-src 'self'; style-src 'unsafe-inline'; script-src 'unsafe-inline'; connect-src 'self'",
|
||||
)
|
||||
|
||||
def _send_json(self, status_code: int, payload: Mapping[str, Any]) -> None:
|
||||
encoded = json.dumps(
|
||||
_json_value(payload),
|
||||
ensure_ascii=False,
|
||||
separators=(",", ":"),
|
||||
).encode("utf-8")
|
||||
self.send_response(int(status_code))
|
||||
self.send_header("Content-Type", "application/json; charset=utf-8")
|
||||
self.send_header("Content-Length", str(len(encoded)))
|
||||
self._security_headers()
|
||||
self.end_headers()
|
||||
self.wfile.write(encoded)
|
||||
|
||||
def _send_console(self) -> None:
|
||||
encoded = CONSOLE_HTML.encode("utf-8")
|
||||
self.send_response(HTTPStatus.OK)
|
||||
self.send_header("Content-Type", "text/html; charset=utf-8")
|
||||
self.send_header("Content-Length", str(len(encoded)))
|
||||
self._security_headers()
|
||||
self.end_headers()
|
||||
self.wfile.write(encoded)
|
||||
|
||||
def _dispatch(self, method: str) -> None:
|
||||
path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
|
||||
if method == "GET" and path in {"/", "/console"}:
|
||||
self._send_console()
|
||||
return
|
||||
try:
|
||||
body = self._read_body() if method == "POST" else {}
|
||||
status_code, payload = self.application.handle_json(
|
||||
method,
|
||||
self.path,
|
||||
{key: value for key, value in self.headers.items()},
|
||||
body,
|
||||
)
|
||||
except BaseException as exc:
|
||||
status_code, payload = error_response(exc)
|
||||
self._send_json(status_code, payload)
|
||||
|
||||
def do_GET(self) -> None: # noqa: N802
|
||||
self._dispatch("GET")
|
||||
|
||||
def do_POST(self) -> None: # noqa: N802
|
||||
self._dispatch("POST")
|
||||
|
||||
|
||||
class CandidateHTTPServer(ThreadingHTTPServer):
|
||||
daemon_threads = True
|
||||
allow_reuse_address = True
|
||||
|
||||
def __init__(self, address: tuple[str, int], application: CandidateApplication) -> None:
|
||||
self.application = application
|
||||
super().__init__(address, CandidateRequestHandler)
|
||||
|
||||
|
||||
def build_application() -> CandidateApplication:
|
||||
state_dir = Path(os.getenv("AIOS_V2_STATE_DIR", "/var/lib/aios-v2")).resolve()
|
||||
state_dir.mkdir(parents=True, exist_ok=True)
|
||||
repository = SQLiteSnapshotRepository(state_dir / "control-plane.sqlite3")
|
||||
service = DurableControlPlaneService(repository)
|
||||
tracker = ExecutionTracker(state_dir / "executions")
|
||||
hermes = HermesAdapter()
|
||||
|
||||
openclaw_url = os.getenv("AIOS_OPENCLAW_URL", "").strip()
|
||||
material_file = os.getenv("AIOS_OPENCLAW_SIGNING_MATERIAL_FILE", "").strip()
|
||||
openclaw = None
|
||||
resolver = None
|
||||
if openclaw_url and material_file:
|
||||
resolver = FileSigningMaterialResolver(material_file)
|
||||
transport = HTTPRuntimeTransport(
|
||||
openclaw_url,
|
||||
credential_ref=os.getenv("AIOS_OPENCLAW_CREDENTIAL_REF") or None,
|
||||
)
|
||||
openclaw = OpenClawAdapter(
|
||||
transport,
|
||||
resolver,
|
||||
callback_sink=AuditCallbackSink(ControlPlaneEventPublisher(service)),
|
||||
)
|
||||
|
||||
gateway = GatewayApplication(service)
|
||||
orchestrator = ExecutionOrchestrator(
|
||||
service,
|
||||
openclaw=openclaw,
|
||||
hermes=hermes,
|
||||
openclaw_signing_material_resolver=resolver,
|
||||
)
|
||||
return CandidateApplication(gateway, orchestrator, tracker, hermes, openclaw)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
host = os.getenv("AIOS_V2_HOST", "127.0.0.1")
|
||||
port = int(os.getenv("AIOS_V2_PORT", "18920"))
|
||||
server = CandidateHTTPServer((host, port), build_application())
|
||||
print(f"AIOS V2 candidate listening on {host}:{port}")
|
||||
server.serve_forever()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,69 @@
|
||||
"""Dependency-free AIOS V2 web console assets.
|
||||
|
||||
Authentication and tenant identity are injected by the trusted reverse proxy.
|
||||
The browser never selects a tenant and never calls OpenClaw or Hermes directly.
|
||||
"""
|
||||
|
||||
CONSOLE_HTML = r'''<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||||
<meta name="referrer" content="no-referrer">
|
||||
<title>AIOS V2 Console</title>
|
||||
<style>
|
||||
:root{font-family:Inter,ui-sans-serif,system-ui,sans-serif;color:#152033;background:#f4f7fb}
|
||||
*{box-sizing:border-box} body{margin:0} header{background:#111827;color:white;padding:18px 24px;display:flex;justify-content:space-between;align-items:center}
|
||||
header h1{font-size:20px;margin:0} header span{font-size:12px;color:#cbd5e1}
|
||||
main{padding:24px;max-width:1440px;margin:auto}.notice{background:#fff7ed;border:1px solid #fed7aa;padding:12px 14px;border-radius:10px;margin-bottom:18px;font-size:13px}
|
||||
.cards{display:grid;grid-template-columns:repeat(auto-fit,minmax(190px,1fr));gap:14px;margin-bottom:20px}.card,.panel{background:white;border:1px solid #e5e7eb;border-radius:12px;box-shadow:0 1px 2px rgba(0,0,0,.04)}
|
||||
.card{padding:16px}.card b{display:block;font-size:28px;margin-top:8px}.card small{color:#64748b}.grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(420px,1fr));gap:16px}.panel{padding:16px;overflow:auto}.panel h2{font-size:16px;margin:0 0 12px}
|
||||
table{width:100%;border-collapse:collapse;font-size:12px}th,td{text-align:left;border-bottom:1px solid #edf0f4;padding:8px;vertical-align:top}th{color:#64748b;font-weight:600}.state{font-weight:700}.ok{color:#047857}.bad{color:#b91c1c}.muted{color:#64748b}
|
||||
button{border:0;border-radius:8px;padding:9px 13px;background:#2563eb;color:white;cursor:pointer}button:disabled{opacity:.5}.toolbar{display:flex;gap:8px;align-items:center;margin-bottom:16px}.error{color:#b91c1c;white-space:pre-wrap}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<header><h1>AIOS V2 Control Plane</h1><span id="updated">Not loaded</span></header>
|
||||
<main>
|
||||
<div class="notice">Read-only candidate console. Identity and tenant context must be injected by the trusted reverse proxy. The console never calls OpenClaw or Hermes directly.</div>
|
||||
<div class="toolbar"><button id="refresh">Refresh</button><span id="error" class="error"></span></div>
|
||||
<section class="cards">
|
||||
<div class="card"><small>Agents</small><b id="agentCount">—</b></div>
|
||||
<div class="card"><small>Tasks</small><b id="taskCount">—</b></div>
|
||||
<div class="card"><small>Audit events</small><b id="auditCount">—</b></div>
|
||||
<div class="card"><small>Executions</small><b id="executionCount">—</b></div>
|
||||
</section>
|
||||
<section class="grid">
|
||||
<div class="panel"><h2>Runtime health</h2><div id="runtime"></div></div>
|
||||
<div class="panel"><h2>Tasks</h2><div id="tasks"></div></div>
|
||||
<div class="panel"><h2>Agents</h2><div id="agents"></div></div>
|
||||
<div class="panel"><h2>Execution evidence</h2><div id="executions"></div></div>
|
||||
<div class="panel"><h2>Audit</h2><div id="audit"></div></div>
|
||||
</section>
|
||||
</main>
|
||||
<script>
|
||||
const esc=v=>String(v??'').replace(/[&<>"']/g,c=>({'&':'&','<':'<','>':'>','"':'"',"'":'''}[c]));
|
||||
const table=(cols,rows)=>`<table><thead><tr>${cols.map(c=>`<th>${esc(c[0])}</th>`).join('')}</tr></thead><tbody>${rows.map(r=>`<tr>${cols.map(c=>`<td>${esc(c[1](r))}</td>`).join('')}</tr>`).join('')}</tbody></table>`;
|
||||
async function api(path){const r=await fetch(path,{credentials:'same-origin',headers:{'Accept':'application/json'}});const data=await r.json().catch(()=>({}));if(!r.ok)throw new Error(`${r.status} ${data.message||data.error||path}`);return data}
|
||||
async function refresh(){
|
||||
const btn=document.getElementById('refresh');btn.disabled=true;document.getElementById('error').textContent='';
|
||||
try{
|
||||
const [agents,tasks,audit,runtime,executions]=await Promise.all([
|
||||
api('/api/v2/agents'),api('/api/v2/tasks'),api('/api/v2/audit'),api('/api/v2/runtime'),api('/api/v2/executions')
|
||||
]);
|
||||
document.getElementById('agentCount').textContent=agents.agents.length;
|
||||
document.getElementById('taskCount').textContent=tasks.tasks.length;
|
||||
document.getElementById('auditCount').textContent=audit.events.length;
|
||||
document.getElementById('executionCount').textContent=executions.executions.length;
|
||||
document.getElementById('agents').innerHTML=table([['ID',x=>x.agent_id],['Runtime',x=>x.runtime],['Enabled',x=>x.enabled]],agents.agents);
|
||||
document.getElementById('tasks').innerHTML=table([['Task',x=>x.task_id],['Action',x=>x.action],['State',x=>x.state],['Rev',x=>x.revision]],tasks.tasks);
|
||||
document.getElementById('audit').innerHTML=table([['Time',x=>x.created_at],['Action',x=>x.action],['Outcome',x=>x.outcome],['Resource',x=>x.resource_id]],audit.events.slice(-100).reverse());
|
||||
document.getElementById('executions').innerHTML=table([['Executor',x=>x.executor],['Task',x=>x.task_id],['Status',x=>x.status],['Progress',x=>`${x.progress_completed}/${x.progress_total}`],['Commit',x=>x.last_commit||'']],executions.executions);
|
||||
document.getElementById('runtime').innerHTML=table([['Runtime',x=>x.name],['Status',x=>x.status],['Details',x=>JSON.stringify(x.details)]],runtime.runtimes);
|
||||
document.getElementById('updated').textContent=`Updated ${new Date().toLocaleString()}`;
|
||||
}catch(e){document.getElementById('error').textContent=e.message||String(e)}finally{btn.disabled=false}
|
||||
}
|
||||
document.getElementById('refresh').addEventListener('click',refresh);refresh();setInterval(refresh,30000);
|
||||
</script>
|
||||
</body>
|
||||
</html>'''
|
||||
@@ -0,0 +1,627 @@
|
||||
"""Tenant-isolated AIOS V2 control-plane domain service.
|
||||
|
||||
This module deliberately contains no OpenClaw or Hermes implementation details.
|
||||
It owns authoritative tenant, agent, task-lifecycle, idempotency and audit state.
|
||||
Runtime adapters consume approved tasks through a separate boundary.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import hashlib
|
||||
import json
|
||||
import threading
|
||||
import uuid
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from enum import Enum
|
||||
from typing import Any, Callable, Iterable, Mapping
|
||||
|
||||
|
||||
def _utc_now() -> str:
|
||||
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||
|
||||
|
||||
def _new_id(prefix: str) -> str:
|
||||
return f"{prefix}_{uuid.uuid4().hex}"
|
||||
|
||||
|
||||
def _canonical_digest(value: Mapping[str, Any]) -> str:
|
||||
encoded = json.dumps(
|
||||
value,
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
default=str,
|
||||
).encode("utf-8")
|
||||
return hashlib.sha256(encoded).hexdigest()
|
||||
|
||||
|
||||
class ControlPlaneError(RuntimeError):
|
||||
"""Base error returned by the control-plane domain."""
|
||||
|
||||
|
||||
class Unauthorized(ControlPlaneError):
|
||||
"""The caller is not authenticated or lacks the required role."""
|
||||
|
||||
|
||||
class TenantIsolationError(ControlPlaneError):
|
||||
"""A request attempted to cross a tenant boundary."""
|
||||
|
||||
|
||||
class NotFound(ControlPlaneError):
|
||||
"""A tenant-scoped resource does not exist or is not visible."""
|
||||
|
||||
|
||||
class InvalidTransition(ControlPlaneError):
|
||||
"""A task state transition violates the lifecycle contract."""
|
||||
|
||||
|
||||
class IdempotencyConflict(ControlPlaneError):
|
||||
"""An idempotency key was reused for a different command."""
|
||||
|
||||
|
||||
class RevisionConflict(ControlPlaneError):
|
||||
"""Optimistic task revision does not match current state."""
|
||||
|
||||
|
||||
class PolicyDenied(ControlPlaneError):
|
||||
"""The policy gate denied a control-plane operation."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RequestContext:
|
||||
"""Authenticated request identity propagated across all AIOS layers."""
|
||||
|
||||
tenant_id: str
|
||||
principal_id: str
|
||||
trace_id: str
|
||||
roles: frozenset[str] = field(default_factory=frozenset)
|
||||
session_id: str | None = None
|
||||
policy_version: str = "v1"
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
for name in ("tenant_id", "principal_id", "trace_id"):
|
||||
if not str(getattr(self, name, "")).strip():
|
||||
raise ValueError(f"{name} is required")
|
||||
|
||||
@classmethod
|
||||
def create(
|
||||
cls,
|
||||
*,
|
||||
tenant_id: str,
|
||||
principal_id: str,
|
||||
roles: Iterable[str] = (),
|
||||
trace_id: str | None = None,
|
||||
session_id: str | None = None,
|
||||
policy_version: str = "v1",
|
||||
) -> "RequestContext":
|
||||
return cls(
|
||||
tenant_id=tenant_id.strip(),
|
||||
principal_id=principal_id.strip(),
|
||||
trace_id=trace_id or _new_id("trace"),
|
||||
roles=frozenset(str(role).strip() for role in roles if str(role).strip()),
|
||||
session_id=session_id,
|
||||
policy_version=policy_version,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TenantRecord:
|
||||
tenant_id: str
|
||||
display_name: str
|
||||
status: str = "active"
|
||||
metadata: Mapping[str, Any] = field(default_factory=dict)
|
||||
created_at: str = field(default_factory=_utc_now)
|
||||
updated_at: str = field(default_factory=_utc_now)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AgentRecord:
|
||||
agent_id: str
|
||||
tenant_id: str
|
||||
display_name: str
|
||||
runtime: str
|
||||
capabilities: tuple[str, ...] = field(default_factory=tuple)
|
||||
enabled: bool = True
|
||||
metadata: Mapping[str, Any] = field(default_factory=dict)
|
||||
created_at: str = field(default_factory=_utc_now)
|
||||
updated_at: str = field(default_factory=_utc_now)
|
||||
|
||||
|
||||
class TaskState(str, Enum):
|
||||
CREATED = "CREATED"
|
||||
APPROVED = "APPROVED"
|
||||
RUNNING = "RUNNING"
|
||||
COMPLETED = "COMPLETED"
|
||||
FAILED = "FAILED"
|
||||
CANCELLED = "CANCELLED"
|
||||
|
||||
|
||||
TERMINAL_TASK_STATES = frozenset(
|
||||
{TaskState.COMPLETED, TaskState.FAILED, TaskState.CANCELLED}
|
||||
)
|
||||
|
||||
ALLOWED_TASK_TRANSITIONS: Mapping[TaskState, frozenset[TaskState]] = {
|
||||
TaskState.CREATED: frozenset({TaskState.APPROVED, TaskState.CANCELLED}),
|
||||
TaskState.APPROVED: frozenset({TaskState.RUNNING, TaskState.CANCELLED}),
|
||||
TaskState.RUNNING: frozenset(
|
||||
{TaskState.COMPLETED, TaskState.FAILED, TaskState.CANCELLED}
|
||||
),
|
||||
TaskState.COMPLETED: frozenset(),
|
||||
TaskState.FAILED: frozenset(),
|
||||
TaskState.CANCELLED: frozenset(),
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TaskRecord:
|
||||
task_id: str
|
||||
tenant_id: str
|
||||
agent_id: str
|
||||
action: str
|
||||
payload: Mapping[str, Any]
|
||||
state: TaskState = TaskState.CREATED
|
||||
revision: int = 1
|
||||
idempotency_key: str | None = None
|
||||
request_digest: str = ""
|
||||
result: Mapping[str, Any] | None = None
|
||||
error: Mapping[str, Any] | None = None
|
||||
created_by: str = ""
|
||||
approved_by: str | None = None
|
||||
created_at: str = field(default_factory=_utc_now)
|
||||
updated_at: str = field(default_factory=_utc_now)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AuditEvent:
|
||||
event_id: str
|
||||
tenant_id: str
|
||||
principal_id: str
|
||||
trace_id: str
|
||||
action: str
|
||||
resource_type: str
|
||||
resource_id: str
|
||||
outcome: str
|
||||
policy_version: str
|
||||
details: Mapping[str, Any] = field(default_factory=dict)
|
||||
created_at: str = field(default_factory=_utc_now)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PolicyDecision:
|
||||
allowed: bool
|
||||
reason: str
|
||||
policy_version: str
|
||||
|
||||
|
||||
PolicyEvaluator = Callable[[RequestContext, str, Mapping[str, Any]], PolicyDecision]
|
||||
|
||||
|
||||
class InMemoryControlPlaneStore:
|
||||
"""Thread-safe reference store used by tests and the first gateway slice.
|
||||
|
||||
Production backends must preserve the same tenant-scoped interface and add
|
||||
database row-level security, migrations, backup and HA evidence.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.lock = threading.RLock()
|
||||
self.tenants: dict[str, TenantRecord] = {}
|
||||
self.agents: dict[str, AgentRecord] = {}
|
||||
self.tasks: dict[str, TaskRecord] = {}
|
||||
self.idempotency: dict[tuple[str, str], tuple[str, str]] = {}
|
||||
self.audit_events: list[AuditEvent] = []
|
||||
|
||||
|
||||
class ControlPlaneService:
|
||||
"""Authoritative tenant, agent and task lifecycle service."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
store: InMemoryControlPlaneStore | None = None,
|
||||
*,
|
||||
policy_evaluator: PolicyEvaluator | None = None,
|
||||
) -> None:
|
||||
self.store = store or InMemoryControlPlaneStore()
|
||||
self.policy_evaluator = policy_evaluator or self._default_policy
|
||||
|
||||
@staticmethod
|
||||
def _default_policy(
|
||||
context: RequestContext,
|
||||
action: str,
|
||||
resource: Mapping[str, Any],
|
||||
) -> PolicyDecision:
|
||||
del resource
|
||||
role_requirements = {
|
||||
"tenant.register": {"platform_admin"},
|
||||
"tenant.read": {"tenant_admin", "operator", "auditor", "platform_admin"},
|
||||
"agent.register": {"tenant_admin", "operator", "platform_admin"},
|
||||
"agent.read": {"tenant_admin", "operator", "auditor", "platform_admin"},
|
||||
"task.create": {"tenant_admin", "operator", "platform_admin"},
|
||||
"task.read": {"tenant_admin", "operator", "auditor", "platform_admin"},
|
||||
"task.approve": {"tenant_admin", "approver", "platform_admin"},
|
||||
"task.start": {"runtime", "operator", "platform_admin"},
|
||||
"task.complete": {"runtime", "operator", "platform_admin"},
|
||||
"task.fail": {"runtime", "operator", "platform_admin"},
|
||||
"task.cancel": {"tenant_admin", "operator", "approver", "platform_admin"},
|
||||
"audit.read": {"tenant_admin", "auditor", "platform_admin"},
|
||||
}
|
||||
required = role_requirements.get(action, {"platform_admin"})
|
||||
allowed = bool(context.roles.intersection(required))
|
||||
return PolicyDecision(
|
||||
allowed=allowed,
|
||||
reason="role_allowed" if allowed else "required_role_missing",
|
||||
policy_version=context.policy_version,
|
||||
)
|
||||
|
||||
def _require_context(self, context: RequestContext) -> None:
|
||||
if not context.principal_id or not context.tenant_id or not context.trace_id:
|
||||
raise Unauthorized("authenticated tenant context is required")
|
||||
|
||||
def _authorize(
|
||||
self,
|
||||
context: RequestContext,
|
||||
action: str,
|
||||
resource: Mapping[str, Any],
|
||||
) -> PolicyDecision:
|
||||
self._require_context(context)
|
||||
decision = self.policy_evaluator(context, action, resource)
|
||||
if not decision.allowed:
|
||||
self._audit(
|
||||
context,
|
||||
action=action,
|
||||
resource_type=str(resource.get("resource_type", "policy")),
|
||||
resource_id=str(resource.get("resource_id", "unknown")),
|
||||
outcome="denied",
|
||||
details={"reason": decision.reason},
|
||||
)
|
||||
raise PolicyDenied(f"{action} denied: {decision.reason}")
|
||||
return decision
|
||||
|
||||
@staticmethod
|
||||
def _assert_same_tenant(context: RequestContext, tenant_id: str) -> None:
|
||||
if context.tenant_id != tenant_id and "platform_admin" not in context.roles:
|
||||
raise TenantIsolationError("cross-tenant access denied")
|
||||
|
||||
def _audit(
|
||||
self,
|
||||
context: RequestContext,
|
||||
*,
|
||||
action: str,
|
||||
resource_type: str,
|
||||
resource_id: str,
|
||||
outcome: str,
|
||||
details: Mapping[str, Any] | None = None,
|
||||
) -> AuditEvent:
|
||||
event = AuditEvent(
|
||||
event_id=_new_id("audit"),
|
||||
tenant_id=context.tenant_id,
|
||||
principal_id=context.principal_id,
|
||||
trace_id=context.trace_id,
|
||||
action=action,
|
||||
resource_type=resource_type,
|
||||
resource_id=resource_id,
|
||||
outcome=outcome,
|
||||
policy_version=context.policy_version,
|
||||
details=copy.deepcopy(dict(details or {})),
|
||||
)
|
||||
self.store.audit_events.append(event)
|
||||
return event
|
||||
|
||||
def register_tenant(
|
||||
self,
|
||||
context: RequestContext,
|
||||
*,
|
||||
tenant_id: str,
|
||||
display_name: str,
|
||||
metadata: Mapping[str, Any] | None = None,
|
||||
) -> TenantRecord:
|
||||
tenant_id = tenant_id.strip()
|
||||
if not tenant_id or not display_name.strip():
|
||||
raise ValueError("tenant_id and display_name are required")
|
||||
self._authorize(
|
||||
context,
|
||||
"tenant.register",
|
||||
{"resource_type": "tenant", "resource_id": tenant_id},
|
||||
)
|
||||
self._assert_same_tenant(context, tenant_id)
|
||||
with self.store.lock:
|
||||
if tenant_id in self.store.tenants:
|
||||
raise ControlPlaneError(f"tenant already exists: {tenant_id}")
|
||||
tenant = TenantRecord(
|
||||
tenant_id=tenant_id,
|
||||
display_name=display_name.strip(),
|
||||
metadata=copy.deepcopy(dict(metadata or {})),
|
||||
)
|
||||
self.store.tenants[tenant_id] = tenant
|
||||
self._audit(
|
||||
context,
|
||||
action="tenant.register",
|
||||
resource_type="tenant",
|
||||
resource_id=tenant_id,
|
||||
outcome="allowed",
|
||||
)
|
||||
return tenant
|
||||
|
||||
def get_tenant(self, context: RequestContext, tenant_id: str | None = None) -> TenantRecord:
|
||||
tenant_id = tenant_id or context.tenant_id
|
||||
self._authorize(
|
||||
context,
|
||||
"tenant.read",
|
||||
{"resource_type": "tenant", "resource_id": tenant_id},
|
||||
)
|
||||
self._assert_same_tenant(context, tenant_id)
|
||||
with self.store.lock:
|
||||
tenant = self.store.tenants.get(tenant_id)
|
||||
if tenant is None:
|
||||
raise NotFound("tenant not found")
|
||||
return tenant
|
||||
|
||||
def register_agent(
|
||||
self,
|
||||
context: RequestContext,
|
||||
*,
|
||||
agent_id: str,
|
||||
display_name: str,
|
||||
runtime: str,
|
||||
capabilities: Iterable[str] = (),
|
||||
metadata: Mapping[str, Any] | None = None,
|
||||
) -> AgentRecord:
|
||||
self._authorize(
|
||||
context,
|
||||
"agent.register",
|
||||
{"resource_type": "agent", "resource_id": agent_id},
|
||||
)
|
||||
tenant = self.get_tenant(context)
|
||||
if tenant.status != "active":
|
||||
raise ControlPlaneError("tenant is not active")
|
||||
normalized_capabilities = tuple(
|
||||
sorted({str(item).strip() for item in capabilities if str(item).strip()})
|
||||
)
|
||||
if not agent_id.strip() or not display_name.strip() or not runtime.strip():
|
||||
raise ValueError("agent_id, display_name and runtime are required")
|
||||
with self.store.lock:
|
||||
existing = self.store.agents.get(agent_id)
|
||||
if existing is not None and existing.tenant_id != context.tenant_id:
|
||||
raise TenantIsolationError("agent id belongs to another tenant")
|
||||
if existing is not None:
|
||||
raise ControlPlaneError(f"agent already exists: {agent_id}")
|
||||
agent = AgentRecord(
|
||||
agent_id=agent_id.strip(),
|
||||
tenant_id=context.tenant_id,
|
||||
display_name=display_name.strip(),
|
||||
runtime=runtime.strip(),
|
||||
capabilities=normalized_capabilities,
|
||||
metadata=copy.deepcopy(dict(metadata or {})),
|
||||
)
|
||||
self.store.agents[agent.agent_id] = agent
|
||||
self._audit(
|
||||
context,
|
||||
action="agent.register",
|
||||
resource_type="agent",
|
||||
resource_id=agent.agent_id,
|
||||
outcome="allowed",
|
||||
details={"runtime": agent.runtime},
|
||||
)
|
||||
return agent
|
||||
|
||||
def get_agent(self, context: RequestContext, agent_id: str) -> AgentRecord:
|
||||
self._authorize(
|
||||
context,
|
||||
"agent.read",
|
||||
{"resource_type": "agent", "resource_id": agent_id},
|
||||
)
|
||||
with self.store.lock:
|
||||
agent = self.store.agents.get(agent_id)
|
||||
if agent is None:
|
||||
raise NotFound("agent not found")
|
||||
self._assert_same_tenant(context, agent.tenant_id)
|
||||
return agent
|
||||
|
||||
def list_agents(self, context: RequestContext) -> list[AgentRecord]:
|
||||
self._authorize(
|
||||
context,
|
||||
"agent.read",
|
||||
{"resource_type": "agent_collection", "resource_id": context.tenant_id},
|
||||
)
|
||||
with self.store.lock:
|
||||
return [
|
||||
agent
|
||||
for agent in self.store.agents.values()
|
||||
if agent.tenant_id == context.tenant_id
|
||||
]
|
||||
|
||||
def create_task(
|
||||
self,
|
||||
context: RequestContext,
|
||||
*,
|
||||
agent_id: str,
|
||||
action: str,
|
||||
payload: Mapping[str, Any] | None = None,
|
||||
idempotency_key: str | None = None,
|
||||
) -> TaskRecord:
|
||||
self._authorize(
|
||||
context,
|
||||
"task.create",
|
||||
{"resource_type": "task", "resource_id": idempotency_key or "new"},
|
||||
)
|
||||
agent = self.get_agent(context, agent_id)
|
||||
if not agent.enabled:
|
||||
raise ControlPlaneError("agent is disabled")
|
||||
if action not in agent.capabilities and "*" not in agent.capabilities:
|
||||
raise PolicyDenied(f"agent does not advertise capability: {action}")
|
||||
normalized_payload = copy.deepcopy(dict(payload or {}))
|
||||
request_digest = _canonical_digest(
|
||||
{"agent_id": agent_id, "action": action, "payload": normalized_payload}
|
||||
)
|
||||
normalized_key = idempotency_key.strip() if idempotency_key else None
|
||||
with self.store.lock:
|
||||
if normalized_key:
|
||||
idempotency_ref = (context.tenant_id, normalized_key)
|
||||
existing = self.store.idempotency.get(idempotency_ref)
|
||||
if existing is not None:
|
||||
existing_task_id, existing_digest = existing
|
||||
if existing_digest != request_digest:
|
||||
raise IdempotencyConflict(
|
||||
"idempotency key was reused with a different request"
|
||||
)
|
||||
return self.store.tasks[existing_task_id]
|
||||
task = TaskRecord(
|
||||
task_id=_new_id("task"),
|
||||
tenant_id=context.tenant_id,
|
||||
agent_id=agent_id,
|
||||
action=action,
|
||||
payload=normalized_payload,
|
||||
idempotency_key=normalized_key,
|
||||
request_digest=request_digest,
|
||||
created_by=context.principal_id,
|
||||
)
|
||||
self.store.tasks[task.task_id] = task
|
||||
if normalized_key:
|
||||
self.store.idempotency[(context.tenant_id, normalized_key)] = (
|
||||
task.task_id,
|
||||
request_digest,
|
||||
)
|
||||
self._audit(
|
||||
context,
|
||||
action="task.create",
|
||||
resource_type="task",
|
||||
resource_id=task.task_id,
|
||||
outcome="allowed",
|
||||
details={"agent_id": agent_id, "action": action},
|
||||
)
|
||||
return task
|
||||
|
||||
def get_task(self, context: RequestContext, task_id: str) -> TaskRecord:
|
||||
self._authorize(
|
||||
context,
|
||||
"task.read",
|
||||
{"resource_type": "task", "resource_id": task_id},
|
||||
)
|
||||
with self.store.lock:
|
||||
task = self.store.tasks.get(task_id)
|
||||
if task is None:
|
||||
raise NotFound("task not found")
|
||||
self._assert_same_tenant(context, task.tenant_id)
|
||||
return task
|
||||
|
||||
def list_tasks(
|
||||
self,
|
||||
context: RequestContext,
|
||||
*,
|
||||
states: Iterable[TaskState | str] | None = None,
|
||||
) -> list[TaskRecord]:
|
||||
self._authorize(
|
||||
context,
|
||||
"task.read",
|
||||
{"resource_type": "task_collection", "resource_id": context.tenant_id},
|
||||
)
|
||||
state_filter = (
|
||||
{TaskState(item) for item in states}
|
||||
if states is not None
|
||||
else None
|
||||
)
|
||||
with self.store.lock:
|
||||
return [
|
||||
task
|
||||
for task in self.store.tasks.values()
|
||||
if task.tenant_id == context.tenant_id
|
||||
and (state_filter is None or task.state in state_filter)
|
||||
]
|
||||
|
||||
def transition_task(
|
||||
self,
|
||||
context: RequestContext,
|
||||
*,
|
||||
task_id: str,
|
||||
target_state: TaskState | str,
|
||||
expected_revision: int | None = None,
|
||||
result: Mapping[str, Any] | None = None,
|
||||
error: Mapping[str, Any] | None = None,
|
||||
) -> TaskRecord:
|
||||
target = TaskState(target_state)
|
||||
action_name = {
|
||||
TaskState.APPROVED: "task.approve",
|
||||
TaskState.RUNNING: "task.start",
|
||||
TaskState.COMPLETED: "task.complete",
|
||||
TaskState.FAILED: "task.fail",
|
||||
TaskState.CANCELLED: "task.cancel",
|
||||
}.get(target)
|
||||
if action_name is None:
|
||||
raise InvalidTransition(f"cannot transition to {target.value}")
|
||||
self._authorize(
|
||||
context,
|
||||
action_name,
|
||||
{"resource_type": "task", "resource_id": task_id},
|
||||
)
|
||||
with self.store.lock:
|
||||
current = self.store.tasks.get(task_id)
|
||||
if current is None:
|
||||
raise NotFound("task not found")
|
||||
self._assert_same_tenant(context, current.tenant_id)
|
||||
if expected_revision is not None and current.revision != expected_revision:
|
||||
raise RevisionConflict(
|
||||
f"expected revision {expected_revision}, current {current.revision}"
|
||||
)
|
||||
if target not in ALLOWED_TASK_TRANSITIONS[current.state]:
|
||||
raise InvalidTransition(
|
||||
f"invalid transition {current.state.value} -> {target.value}"
|
||||
)
|
||||
if target == TaskState.COMPLETED and result is None:
|
||||
raise ValueError("completed task requires result")
|
||||
if target == TaskState.FAILED and error is None:
|
||||
raise ValueError("failed task requires error")
|
||||
updated = TaskRecord(
|
||||
task_id=current.task_id,
|
||||
tenant_id=current.tenant_id,
|
||||
agent_id=current.agent_id,
|
||||
action=current.action,
|
||||
payload=current.payload,
|
||||
state=target,
|
||||
revision=current.revision + 1,
|
||||
idempotency_key=current.idempotency_key,
|
||||
request_digest=current.request_digest,
|
||||
result=copy.deepcopy(dict(result)) if result is not None else current.result,
|
||||
error=copy.deepcopy(dict(error)) if error is not None else current.error,
|
||||
created_by=current.created_by,
|
||||
approved_by=(
|
||||
context.principal_id
|
||||
if target == TaskState.APPROVED
|
||||
else current.approved_by
|
||||
),
|
||||
created_at=current.created_at,
|
||||
updated_at=_utc_now(),
|
||||
)
|
||||
self.store.tasks[task_id] = updated
|
||||
self._audit(
|
||||
context,
|
||||
action=action_name,
|
||||
resource_type="task",
|
||||
resource_id=task_id,
|
||||
outcome="allowed",
|
||||
details={
|
||||
"from": current.state.value,
|
||||
"to": target.value,
|
||||
"revision": updated.revision,
|
||||
},
|
||||
)
|
||||
return updated
|
||||
|
||||
def list_audit_events(
|
||||
self,
|
||||
context: RequestContext,
|
||||
*,
|
||||
resource_id: str | None = None,
|
||||
) -> list[AuditEvent]:
|
||||
self._authorize(
|
||||
context,
|
||||
"audit.read",
|
||||
{"resource_type": "audit", "resource_id": resource_id or context.tenant_id},
|
||||
)
|
||||
with self.store.lock:
|
||||
return [
|
||||
event
|
||||
for event in self.store.audit_events
|
||||
if event.tenant_id == context.tenant_id
|
||||
and (resource_id is None or event.resource_id == resource_id)
|
||||
]
|
||||
@@ -0,0 +1,195 @@
|
||||
"""Git-backed agent execution evidence for AIOS V2.
|
||||
|
||||
Evidence is tenant-scoped, append-friendly, and contains no runtime credentials.
|
||||
It provides a deterministic source for claim, heartbeat, progress, result and
|
||||
verification state across Codex, OpenClaw and Hermes executors.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
from dataclasses import asdict, dataclass, field, replace
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Mapping
|
||||
|
||||
|
||||
def utc_now() -> str:
|
||||
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||
|
||||
|
||||
def _required(value: str, label: str) -> str:
|
||||
normalized = str(value or "").strip()
|
||||
if not normalized:
|
||||
raise ValueError(f"{label} is required")
|
||||
return normalized
|
||||
|
||||
|
||||
def _safe(value: Mapping[str, Any] | None) -> dict[str, Any]:
|
||||
return json.loads(json.dumps(dict(value or {}), ensure_ascii=False, default=str))
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ExecutionEvidence:
|
||||
execution_id: str
|
||||
tenant_id: str
|
||||
task_id: str
|
||||
agent_id: str
|
||||
executor: str
|
||||
branch: str
|
||||
status: str = "claimed"
|
||||
progress_completed: int = 0
|
||||
progress_total: int = 1
|
||||
current_step: str = "claim"
|
||||
last_commit: str | None = None
|
||||
test_status: str = "not_run"
|
||||
test_summary: Mapping[str, Any] = field(default_factory=dict)
|
||||
result_summary: Mapping[str, Any] = field(default_factory=dict)
|
||||
claimed_at: str = field(default_factory=utc_now)
|
||||
heartbeat_at: str = field(default_factory=utc_now)
|
||||
completed_at: str | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
for name in (
|
||||
"execution_id",
|
||||
"tenant_id",
|
||||
"task_id",
|
||||
"agent_id",
|
||||
"executor",
|
||||
"branch",
|
||||
"status",
|
||||
"current_step",
|
||||
"claimed_at",
|
||||
"heartbeat_at",
|
||||
):
|
||||
object.__setattr__(self, name, _required(getattr(self, name), name))
|
||||
if self.progress_completed < 0 or self.progress_total < 1:
|
||||
raise ValueError("invalid progress values")
|
||||
if self.progress_completed > self.progress_total:
|
||||
raise ValueError("progress_completed cannot exceed progress_total")
|
||||
object.__setattr__(self, "test_summary", _safe(self.test_summary))
|
||||
object.__setattr__(self, "result_summary", _safe(self.result_summary))
|
||||
|
||||
|
||||
class ExecutionTracker:
|
||||
"""Persist one JSON evidence record per tenant and execution."""
|
||||
|
||||
def __init__(self, root: str | Path) -> None:
|
||||
self.root = Path(root).resolve()
|
||||
self.root.mkdir(parents=True, exist_ok=True)
|
||||
self._lock = threading.RLock()
|
||||
|
||||
@staticmethod
|
||||
def _component(value: str) -> str:
|
||||
normalized = _required(value, "path component")
|
||||
if normalized in {".", ".."} or "/" in normalized or "\\" in normalized:
|
||||
raise ValueError("invalid evidence path component")
|
||||
return normalized
|
||||
|
||||
def _path(self, tenant_id: str, execution_id: str) -> Path:
|
||||
tenant = self._component(tenant_id)
|
||||
execution = self._component(execution_id)
|
||||
return self.root / tenant / f"{execution}.json"
|
||||
|
||||
def claim(self, evidence: ExecutionEvidence) -> ExecutionEvidence:
|
||||
path = self._path(evidence.tenant_id, evidence.execution_id)
|
||||
with self._lock:
|
||||
if path.exists():
|
||||
current = self._read(path)
|
||||
if current.task_id != evidence.task_id or current.agent_id != evidence.agent_id:
|
||||
raise RuntimeError("execution_id already belongs to another task")
|
||||
return current
|
||||
self._write(path, evidence)
|
||||
return evidence
|
||||
|
||||
def heartbeat(
|
||||
self,
|
||||
*,
|
||||
tenant_id: str,
|
||||
execution_id: str,
|
||||
status: str | None = None,
|
||||
current_step: str | None = None,
|
||||
progress_completed: int | None = None,
|
||||
progress_total: int | None = None,
|
||||
last_commit: str | None = None,
|
||||
) -> ExecutionEvidence:
|
||||
path = self._path(tenant_id, execution_id)
|
||||
with self._lock:
|
||||
current = self._read(path)
|
||||
updated = replace(
|
||||
current,
|
||||
status=status or current.status,
|
||||
current_step=current_step or current.current_step,
|
||||
progress_completed=(
|
||||
progress_completed
|
||||
if progress_completed is not None
|
||||
else current.progress_completed
|
||||
),
|
||||
progress_total=(
|
||||
progress_total if progress_total is not None else current.progress_total
|
||||
),
|
||||
last_commit=last_commit or current.last_commit,
|
||||
heartbeat_at=utc_now(),
|
||||
)
|
||||
self._write(path, updated)
|
||||
return updated
|
||||
|
||||
def complete(
|
||||
self,
|
||||
*,
|
||||
tenant_id: str,
|
||||
execution_id: str,
|
||||
status: str,
|
||||
test_status: str,
|
||||
test_summary: Mapping[str, Any] | None = None,
|
||||
result_summary: Mapping[str, Any] | None = None,
|
||||
last_commit: str | None = None,
|
||||
) -> ExecutionEvidence:
|
||||
path = self._path(tenant_id, execution_id)
|
||||
with self._lock:
|
||||
current = self._read(path)
|
||||
updated = replace(
|
||||
current,
|
||||
status=_required(status, "status"),
|
||||
current_step="complete",
|
||||
progress_completed=current.progress_total,
|
||||
last_commit=last_commit or current.last_commit,
|
||||
test_status=_required(test_status, "test_status"),
|
||||
test_summary=_safe(test_summary),
|
||||
result_summary=_safe(result_summary),
|
||||
heartbeat_at=utc_now(),
|
||||
completed_at=utc_now(),
|
||||
)
|
||||
self._write(path, updated)
|
||||
return updated
|
||||
|
||||
def get(self, *, tenant_id: str, execution_id: str) -> ExecutionEvidence:
|
||||
return self._read(self._path(tenant_id, execution_id))
|
||||
|
||||
def list(self, *, tenant_id: str) -> list[ExecutionEvidence]:
|
||||
tenant_dir = self.root / self._component(tenant_id)
|
||||
if not tenant_dir.exists():
|
||||
return []
|
||||
records = [self._read(path) for path in tenant_dir.glob("*.json")]
|
||||
return sorted(records, key=lambda item: (item.heartbeat_at, item.execution_id), reverse=True)
|
||||
|
||||
@staticmethod
|
||||
def _read(path: Path) -> ExecutionEvidence:
|
||||
try:
|
||||
value = json.loads(path.read_text(encoding="utf-8"))
|
||||
except FileNotFoundError as exc:
|
||||
raise KeyError("execution evidence not found") from exc
|
||||
if not isinstance(value, Mapping):
|
||||
raise RuntimeError("invalid execution evidence file")
|
||||
return ExecutionEvidence(**dict(value))
|
||||
|
||||
@staticmethod
|
||||
def _write(path: Path, evidence: ExecutionEvidence) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
temporary = path.with_suffix(f".tmp-{os.getpid()}")
|
||||
temporary.write_text(
|
||||
json.dumps(asdict(evidence), ensure_ascii=False, sort_keys=True, indent=2) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
os.replace(temporary, path)
|
||||
@@ -0,0 +1,346 @@
|
||||
"""HTTP/JSON gateway for the AIOS V2 control plane.
|
||||
|
||||
The gateway is intentionally runtime-neutral: browsers and external clients
|
||||
call this API, while OpenClaw and Hermes remain behind authenticated adapters.
|
||||
Identity headers are accepted only from a trusted reverse proxy in v1; direct
|
||||
public exposure is prohibited until signed identity middleware is enabled.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import urllib.parse
|
||||
from enum import Enum
|
||||
from http import HTTPStatus
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from typing import Any, Mapping
|
||||
|
||||
from .control_plane import (
|
||||
ControlPlaneError,
|
||||
ControlPlaneService,
|
||||
IdempotencyConflict,
|
||||
InvalidTransition,
|
||||
NotFound,
|
||||
PolicyDenied,
|
||||
RequestContext,
|
||||
RevisionConflict,
|
||||
TaskState,
|
||||
TenantIsolationError,
|
||||
Unauthorized,
|
||||
)
|
||||
|
||||
|
||||
TENANT_HEADER = "X-AIOS-Tenant-ID"
|
||||
PRINCIPAL_HEADER = "X-AIOS-Principal-ID"
|
||||
ROLES_HEADER = "X-AIOS-Roles"
|
||||
TRACE_HEADER = "X-AIOS-Trace-ID"
|
||||
SESSION_HEADER = "X-AIOS-Session-ID"
|
||||
POLICY_HEADER = "X-AIOS-Policy-Version"
|
||||
AUTHENTICATED_HEADER = "X-AIOS-Authenticated"
|
||||
|
||||
_RESOURCE_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{0,191}$")
|
||||
|
||||
|
||||
def _json_value(value: Any) -> Any:
|
||||
if dataclasses.is_dataclass(value):
|
||||
return {
|
||||
field.name: _json_value(getattr(value, field.name))
|
||||
for field in dataclasses.fields(value)
|
||||
}
|
||||
if isinstance(value, Enum):
|
||||
return value.value
|
||||
if isinstance(value, Mapping):
|
||||
return {str(key): _json_value(item) for key, item in value.items()}
|
||||
if isinstance(value, (list, tuple, set, frozenset)):
|
||||
return [_json_value(item) for item in value]
|
||||
return value
|
||||
|
||||
|
||||
def _require_mapping(value: Any, *, label: str = "JSON body") -> Mapping[str, Any]:
|
||||
if not isinstance(value, Mapping):
|
||||
raise ValueError(f"{label} must be an object")
|
||||
return value
|
||||
|
||||
|
||||
def _require_resource_id(value: Any, *, label: str) -> str:
|
||||
normalized = str(value or "").strip()
|
||||
if not _RESOURCE_ID_RE.fullmatch(normalized):
|
||||
raise ValueError(f"invalid {label}")
|
||||
return normalized
|
||||
|
||||
|
||||
class GatewayApplication:
|
||||
"""Pure request router that can be tested without a listening socket."""
|
||||
|
||||
def __init__(self, service: ControlPlaneService | None = None) -> None:
|
||||
self.service = service or ControlPlaneService()
|
||||
|
||||
@staticmethod
|
||||
def context_from_headers(headers: Mapping[str, str]) -> RequestContext:
|
||||
authenticated = str(headers.get(AUTHENTICATED_HEADER, "")).strip().lower()
|
||||
if authenticated not in {"1", "true", "yes"}:
|
||||
raise Unauthorized("trusted authentication assertion is required")
|
||||
roles = [
|
||||
role.strip()
|
||||
for role in str(headers.get(ROLES_HEADER, "")).split(",")
|
||||
if role.strip()
|
||||
]
|
||||
return RequestContext.create(
|
||||
tenant_id=str(headers.get(TENANT_HEADER, "")),
|
||||
principal_id=str(headers.get(PRINCIPAL_HEADER, "")),
|
||||
trace_id=str(headers.get(TRACE_HEADER, "")).strip() or None,
|
||||
session_id=str(headers.get(SESSION_HEADER, "")).strip() or None,
|
||||
policy_version=str(headers.get(POLICY_HEADER, "v1")).strip() or "v1",
|
||||
roles=roles,
|
||||
)
|
||||
|
||||
def handle(
|
||||
self,
|
||||
method: str,
|
||||
raw_path: str,
|
||||
headers: Mapping[str, str],
|
||||
body: Any = None,
|
||||
) -> tuple[int, Mapping[str, Any]]:
|
||||
parsed = urllib.parse.urlparse(raw_path)
|
||||
path = parsed.path.rstrip("/") or "/"
|
||||
query = urllib.parse.parse_qs(parsed.query, keep_blank_values=False)
|
||||
method = method.upper()
|
||||
|
||||
if method == "GET" and path in {"/healthz", "/api/v2/health"}:
|
||||
return HTTPStatus.OK, {
|
||||
"service": "aios-v2-control-plane",
|
||||
"status": "ok",
|
||||
"store": type(self.service.store).__name__,
|
||||
}
|
||||
|
||||
context = self.context_from_headers(headers)
|
||||
payload = _require_mapping(body or {})
|
||||
|
||||
if method == "POST" and path == "/api/v2/tenants":
|
||||
tenant = self.service.register_tenant(
|
||||
context,
|
||||
tenant_id=_require_resource_id(payload.get("tenant_id"), label="tenant_id"),
|
||||
display_name=str(payload.get("display_name", "")),
|
||||
metadata=_require_mapping(payload.get("metadata", {}), label="metadata"),
|
||||
)
|
||||
return HTTPStatus.CREATED, {"tenant": _json_value(tenant)}
|
||||
|
||||
if method == "GET" and path == "/api/v2/tenants/current":
|
||||
tenant = self.service.get_tenant(context)
|
||||
return HTTPStatus.OK, {"tenant": _json_value(tenant)}
|
||||
|
||||
if method == "POST" and path == "/api/v2/agents":
|
||||
agent = self.service.register_agent(
|
||||
context,
|
||||
agent_id=_require_resource_id(payload.get("agent_id"), label="agent_id"),
|
||||
display_name=str(payload.get("display_name", "")),
|
||||
runtime=str(payload.get("runtime", "")),
|
||||
capabilities=payload.get("capabilities", ()),
|
||||
metadata=_require_mapping(payload.get("metadata", {}), label="metadata"),
|
||||
)
|
||||
return HTTPStatus.CREATED, {"agent": _json_value(agent)}
|
||||
|
||||
if method == "GET" and path == "/api/v2/agents":
|
||||
return HTTPStatus.OK, {
|
||||
"agents": _json_value(self.service.list_agents(context))
|
||||
}
|
||||
|
||||
agent_match = re.fullmatch(r"/api/v2/agents/([^/]+)", path)
|
||||
if method == "GET" and agent_match:
|
||||
agent_id = _require_resource_id(
|
||||
urllib.parse.unquote(agent_match.group(1)), label="agent_id"
|
||||
)
|
||||
return HTTPStatus.OK, {
|
||||
"agent": _json_value(self.service.get_agent(context, agent_id))
|
||||
}
|
||||
|
||||
if method == "POST" and path == "/api/v2/tasks":
|
||||
task = self.service.create_task(
|
||||
context,
|
||||
agent_id=_require_resource_id(payload.get("agent_id"), label="agent_id"),
|
||||
action=str(payload.get("action", "")).strip(),
|
||||
payload=_require_mapping(payload.get("payload", {}), label="payload"),
|
||||
idempotency_key=(
|
||||
str(payload["idempotency_key"]).strip()
|
||||
if payload.get("idempotency_key") is not None
|
||||
else None
|
||||
),
|
||||
)
|
||||
return HTTPStatus.CREATED, {"task": _json_value(task)}
|
||||
|
||||
if method == "GET" and path == "/api/v2/tasks":
|
||||
requested_states = query.get("state")
|
||||
tasks = self.service.list_tasks(context, states=requested_states)
|
||||
return HTTPStatus.OK, {"tasks": _json_value(tasks)}
|
||||
|
||||
task_match = re.fullmatch(r"/api/v2/tasks/([^/]+)", path)
|
||||
if method == "GET" and task_match:
|
||||
task_id = _require_resource_id(
|
||||
urllib.parse.unquote(task_match.group(1)), label="task_id"
|
||||
)
|
||||
return HTTPStatus.OK, {
|
||||
"task": _json_value(self.service.get_task(context, task_id))
|
||||
}
|
||||
|
||||
transition_match = re.fullmatch(
|
||||
r"/api/v2/tasks/([^/]+)/(approve|start|complete|fail|cancel)", path
|
||||
)
|
||||
if method == "POST" and transition_match:
|
||||
task_id = _require_resource_id(
|
||||
urllib.parse.unquote(transition_match.group(1)), label="task_id"
|
||||
)
|
||||
transition = transition_match.group(2)
|
||||
target = {
|
||||
"approve": TaskState.APPROVED,
|
||||
"start": TaskState.RUNNING,
|
||||
"complete": TaskState.COMPLETED,
|
||||
"fail": TaskState.FAILED,
|
||||
"cancel": TaskState.CANCELLED,
|
||||
}[transition]
|
||||
expected_revision = payload.get("expected_revision")
|
||||
if expected_revision is not None:
|
||||
expected_revision = int(expected_revision)
|
||||
task = self.service.transition_task(
|
||||
context,
|
||||
task_id=task_id,
|
||||
target_state=target,
|
||||
expected_revision=expected_revision,
|
||||
result=(
|
||||
_require_mapping(payload.get("result"), label="result")
|
||||
if payload.get("result") is not None
|
||||
else None
|
||||
),
|
||||
error=(
|
||||
_require_mapping(payload.get("error"), label="error")
|
||||
if payload.get("error") is not None
|
||||
else None
|
||||
),
|
||||
)
|
||||
return HTTPStatus.OK, {"task": _json_value(task)}
|
||||
|
||||
if method == "GET" and path == "/api/v2/audit":
|
||||
resource_id = query.get("resource_id", [None])[0]
|
||||
events = self.service.list_audit_events(
|
||||
context,
|
||||
resource_id=(
|
||||
_require_resource_id(resource_id, label="resource_id")
|
||||
if resource_id
|
||||
else None
|
||||
),
|
||||
)
|
||||
return HTTPStatus.OK, {"events": _json_value(events)}
|
||||
|
||||
return HTTPStatus.NOT_FOUND, {
|
||||
"error": "not_found",
|
||||
"message": "route not found",
|
||||
}
|
||||
|
||||
|
||||
_ERROR_STATUS: tuple[tuple[type[BaseException], HTTPStatus], ...] = (
|
||||
(Unauthorized, HTTPStatus.UNAUTHORIZED),
|
||||
(PolicyDenied, HTTPStatus.FORBIDDEN),
|
||||
(TenantIsolationError, HTTPStatus.NOT_FOUND),
|
||||
(NotFound, HTTPStatus.NOT_FOUND),
|
||||
(IdempotencyConflict, HTTPStatus.CONFLICT),
|
||||
(RevisionConflict, HTTPStatus.CONFLICT),
|
||||
(InvalidTransition, HTTPStatus.CONFLICT),
|
||||
(ValueError, HTTPStatus.BAD_REQUEST),
|
||||
(ControlPlaneError, HTTPStatus.BAD_REQUEST),
|
||||
)
|
||||
|
||||
|
||||
def error_response(exc: BaseException) -> tuple[int, Mapping[str, Any]]:
|
||||
for error_type, status in _ERROR_STATUS:
|
||||
if isinstance(exc, error_type):
|
||||
return status, {
|
||||
"error": error_type.__name__,
|
||||
"message": str(exc),
|
||||
}
|
||||
return HTTPStatus.INTERNAL_SERVER_ERROR, {
|
||||
"error": "InternalServerError",
|
||||
"message": "internal server error",
|
||||
}
|
||||
|
||||
|
||||
class GatewayRequestHandler(BaseHTTPRequestHandler):
|
||||
server_version = "AIOSV2Gateway/1.0"
|
||||
max_body_bytes = 1_048_576
|
||||
|
||||
@property
|
||||
def application(self) -> GatewayApplication:
|
||||
return self.server.application # type: ignore[attr-defined]
|
||||
|
||||
def _read_body(self) -> Any:
|
||||
length = int(self.headers.get("Content-Length", "0") or "0")
|
||||
if length < 0 or length > self.max_body_bytes:
|
||||
raise ValueError("request body is too large")
|
||||
if length == 0:
|
||||
return {}
|
||||
content_type = self.headers.get("Content-Type", "").split(";", 1)[0].strip()
|
||||
if content_type != "application/json":
|
||||
raise ValueError("Content-Type must be application/json")
|
||||
raw = self.rfile.read(length)
|
||||
return json.loads(raw.decode("utf-8"))
|
||||
|
||||
def _send_json(self, status: int, payload: Mapping[str, Any]) -> None:
|
||||
encoded = json.dumps(
|
||||
_json_value(payload), ensure_ascii=False, separators=(",", ":")
|
||||
).encode("utf-8")
|
||||
self.send_response(int(status))
|
||||
self.send_header("Content-Type", "application/json; charset=utf-8")
|
||||
self.send_header("Content-Length", str(len(encoded)))
|
||||
self.send_header("Cache-Control", "no-store")
|
||||
self.send_header("X-Content-Type-Options", "nosniff")
|
||||
self.end_headers()
|
||||
self.wfile.write(encoded)
|
||||
|
||||
def _dispatch(self, method: str) -> None:
|
||||
try:
|
||||
body = self._read_body() if method in {"POST", "PUT", "PATCH"} else {}
|
||||
status, payload = self.application.handle(
|
||||
method,
|
||||
self.path,
|
||||
{key: value for key, value in self.headers.items()},
|
||||
body,
|
||||
)
|
||||
except BaseException as exc: # handler boundary; sanitized below
|
||||
status, payload = error_response(exc)
|
||||
self._send_json(status, payload)
|
||||
|
||||
def do_GET(self) -> None: # noqa: N802
|
||||
self._dispatch("GET")
|
||||
|
||||
def do_POST(self) -> None: # noqa: N802
|
||||
self._dispatch("POST")
|
||||
|
||||
def log_message(self, format: str, *args: Any) -> None:
|
||||
# Production deployment routes structured access logs through the
|
||||
# reverse proxy; do not log identity headers or request bodies here.
|
||||
super().log_message(format, *args)
|
||||
|
||||
|
||||
class GatewayHTTPServer(ThreadingHTTPServer):
|
||||
daemon_threads = True
|
||||
allow_reuse_address = True
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
server_address: tuple[str, int],
|
||||
application: GatewayApplication,
|
||||
) -> None:
|
||||
self.application = application
|
||||
super().__init__(server_address, GatewayRequestHandler)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
host = os.getenv("AIOS_V2_GATEWAY_HOST", "127.0.0.1")
|
||||
port = int(os.getenv("AIOS_V2_GATEWAY_PORT", "18920"))
|
||||
server = GatewayHTTPServer((host, port), GatewayApplication())
|
||||
print(f"AIOS V2 control-plane gateway listening on {host}:{port}")
|
||||
server.serve_forever()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,123 @@
|
||||
"""Sanitized GitOps runtime status for the AIOS Web control plane."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from dataclasses import dataclass, asdict
|
||||
from pathlib import Path
|
||||
from typing import Any, Mapping
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GitOpsStatus:
|
||||
available: bool
|
||||
service_status: str
|
||||
workspace_mode: str
|
||||
workspace_branch: str | None
|
||||
workspace_ref: str | None
|
||||
workspace_dirty: bool | None
|
||||
candidate_branch: str
|
||||
release_channel: str
|
||||
release_version: str
|
||||
sync_status: str
|
||||
details: Mapping[str, Any]
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
return asdict(self)
|
||||
|
||||
|
||||
def _run_git(worktree: Path, *args: str) -> tuple[int, str, str]:
|
||||
process = subprocess.run(
|
||||
["git", "-C", str(worktree), *args],
|
||||
text=True,
|
||||
capture_output=True,
|
||||
timeout=5,
|
||||
)
|
||||
return process.returncode, process.stdout.strip(), process.stderr.strip()
|
||||
|
||||
|
||||
def collect_gitops_status() -> GitOpsStatus:
|
||||
"""Return only non-sensitive Git/workspace metadata.
|
||||
|
||||
Remote URLs, environment values, credentials, filesystem contents and
|
||||
command stderr are deliberately excluded from the Web response.
|
||||
"""
|
||||
|
||||
candidate_branch = os.getenv(
|
||||
"AIOS_CANDIDATE_BRANCH",
|
||||
"integration/aios-v2r1-cto-production-candidate-20260720",
|
||||
)
|
||||
release_channel = os.getenv("AIOS_RELEASE_CHANNEL", "candidate")
|
||||
release_version = os.getenv("AIOS_RELEASE_VERSION", "V2R1C0-candidate")
|
||||
sync_status = os.getenv("AIOS_GITOPS_SYNC_STATUS", "unknown")
|
||||
worktree = Path(
|
||||
os.getenv(
|
||||
"AIOS_GITOPS_CONTROLLER_WORKTREE",
|
||||
"/data/aios-gitops/worktrees/controller",
|
||||
)
|
||||
)
|
||||
|
||||
if not worktree.is_dir():
|
||||
return GitOpsStatus(
|
||||
available=False,
|
||||
service_status="unknown",
|
||||
workspace_mode="unavailable",
|
||||
workspace_branch=None,
|
||||
workspace_ref=None,
|
||||
workspace_dirty=None,
|
||||
candidate_branch=candidate_branch,
|
||||
release_channel=release_channel,
|
||||
release_version=release_version,
|
||||
sync_status=sync_status,
|
||||
details={"reason": "controller_worktree_unavailable"},
|
||||
)
|
||||
|
||||
inside_code, inside, _ = _run_git(worktree, "rev-parse", "--is-inside-work-tree")
|
||||
if inside_code != 0 or inside != "true":
|
||||
return GitOpsStatus(
|
||||
available=False,
|
||||
service_status="unknown",
|
||||
workspace_mode="invalid",
|
||||
workspace_branch=None,
|
||||
workspace_ref=None,
|
||||
workspace_dirty=None,
|
||||
candidate_branch=candidate_branch,
|
||||
release_channel=release_channel,
|
||||
release_version=release_version,
|
||||
sync_status=sync_status,
|
||||
details={"reason": "controller_worktree_invalid"},
|
||||
)
|
||||
|
||||
branch_code, branch, _ = _run_git(worktree, "branch", "--show-current")
|
||||
ref_code, ref_value, _ = _run_git(worktree, "rev-parse", "HEAD")
|
||||
dirty_code, dirty_value, _ = _run_git(worktree, "status", "--porcelain=v1")
|
||||
|
||||
service_status = "unknown"
|
||||
try:
|
||||
process = subprocess.run(
|
||||
["systemctl", "is-active", "aios-gitops-model-api"],
|
||||
text=True,
|
||||
capture_output=True,
|
||||
timeout=3,
|
||||
)
|
||||
service_status = process.stdout.strip() or "inactive"
|
||||
except (FileNotFoundError, subprocess.TimeoutExpired):
|
||||
service_status = "unavailable"
|
||||
|
||||
branch_value = branch if branch_code == 0 and branch else None
|
||||
return GitOpsStatus(
|
||||
available=True,
|
||||
service_status=service_status,
|
||||
workspace_mode="branch" if branch_value else "detached",
|
||||
workspace_branch=branch_value,
|
||||
workspace_ref=ref_value if ref_code == 0 else None,
|
||||
workspace_dirty=bool(dirty_value) if dirty_code == 0 else None,
|
||||
candidate_branch=candidate_branch,
|
||||
release_channel=release_channel,
|
||||
release_version=release_version,
|
||||
sync_status=sync_status,
|
||||
details={
|
||||
"controller_clean": dirty_code == 0 and not dirty_value,
|
||||
"controller_detached": branch_code == 0 and not branch,
|
||||
},
|
||||
)
|
||||
@@ -0,0 +1,264 @@
|
||||
"""AIOS V2 execution orchestrator.
|
||||
|
||||
The orchestrator is the only component allowed to translate an approved AIOS
|
||||
task into an OpenClaw command or Hermes job. OpenClaw and Hermes remain pure
|
||||
execution runtimes and never own tenant policy or task lifecycle.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Mapping
|
||||
|
||||
from aios_hermes import HermesAdapter, HermesJob, HermesResult
|
||||
from aios_openclaw import (
|
||||
CommandState,
|
||||
OpenClawAdapter,
|
||||
OpenClawCommand,
|
||||
OpenClawResult,
|
||||
sign_command,
|
||||
)
|
||||
|
||||
from .control_plane import (
|
||||
AgentRecord,
|
||||
ControlPlaneService,
|
||||
RequestContext,
|
||||
TaskRecord,
|
||||
TaskState,
|
||||
)
|
||||
|
||||
|
||||
class OrchestrationError(RuntimeError):
|
||||
"""AIOS failed to dispatch or reconcile a runtime execution."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DispatchRecord:
|
||||
tenant_id: str
|
||||
task_id: str
|
||||
agent_id: str
|
||||
runtime: str
|
||||
runtime_execution_id: str
|
||||
state: str
|
||||
trace_id: str
|
||||
|
||||
|
||||
class ExecutionOrchestrator:
|
||||
"""Dispatch approved AIOS tasks to runtime adapters and reconcile results."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
control_plane: ControlPlaneService,
|
||||
*,
|
||||
openclaw: OpenClawAdapter | None = None,
|
||||
hermes: HermesAdapter | None = None,
|
||||
openclaw_signing_material_resolver=None,
|
||||
openclaw_key_id: str = "default",
|
||||
default_cell_id: str = "default",
|
||||
default_profile_id: str = "default",
|
||||
) -> None:
|
||||
self.control_plane = control_plane
|
||||
self.openclaw = openclaw
|
||||
self.hermes = hermes
|
||||
self.openclaw_signing_material_resolver = openclaw_signing_material_resolver
|
||||
self.openclaw_key_id = openclaw_key_id
|
||||
self.default_cell_id = default_cell_id
|
||||
self.default_profile_id = default_profile_id
|
||||
self._dispatches: dict[tuple[str, str], DispatchRecord] = {}
|
||||
|
||||
def dispatch(
|
||||
self,
|
||||
context: RequestContext,
|
||||
*,
|
||||
task_id: str,
|
||||
timeout_seconds: float | None = None,
|
||||
) -> DispatchRecord:
|
||||
task = self.control_plane.get_task(context, task_id)
|
||||
if task.state != TaskState.APPROVED:
|
||||
raise OrchestrationError("only APPROVED tasks may be dispatched")
|
||||
agent = self.control_plane.get_agent(context, task.agent_id)
|
||||
existing = self._dispatches.get((task.tenant_id, task.task_id))
|
||||
if existing is not None:
|
||||
return existing
|
||||
|
||||
running = self.control_plane.transition_task(
|
||||
context,
|
||||
task_id=task.task_id,
|
||||
target_state=TaskState.RUNNING,
|
||||
expected_revision=task.revision,
|
||||
)
|
||||
|
||||
runtime = agent.runtime.strip().lower()
|
||||
try:
|
||||
if runtime == "openclaw":
|
||||
record = self._dispatch_openclaw(
|
||||
context,
|
||||
agent,
|
||||
running,
|
||||
timeout_seconds=timeout_seconds,
|
||||
)
|
||||
elif runtime == "hermes":
|
||||
record = self._dispatch_hermes(context, agent, running)
|
||||
else:
|
||||
raise OrchestrationError(f"unsupported runtime: {agent.runtime}")
|
||||
except Exception as exc:
|
||||
current = self.control_plane.get_task(context, task.task_id)
|
||||
if current.state == TaskState.RUNNING:
|
||||
self.control_plane.transition_task(
|
||||
context,
|
||||
task_id=task.task_id,
|
||||
target_state=TaskState.FAILED,
|
||||
expected_revision=current.revision,
|
||||
error={"code": "dispatch_failed", "message": str(exc)},
|
||||
)
|
||||
raise
|
||||
|
||||
self._dispatches[(task.tenant_id, task.task_id)] = record
|
||||
return record
|
||||
|
||||
def _dispatch_openclaw(
|
||||
self,
|
||||
context: RequestContext,
|
||||
agent: AgentRecord,
|
||||
task: TaskRecord,
|
||||
*,
|
||||
timeout_seconds: float | None,
|
||||
) -> DispatchRecord:
|
||||
if self.openclaw is None or self.openclaw_signing_material_resolver is None:
|
||||
raise OrchestrationError("OpenClaw runtime is not configured")
|
||||
key_material = self.openclaw_signing_material_resolver(
|
||||
task.tenant_id,
|
||||
self.openclaw_key_id,
|
||||
)
|
||||
command = OpenClawCommand.create(
|
||||
tenant_id=task.tenant_id,
|
||||
task_id=task.task_id,
|
||||
trace_id=context.trace_id,
|
||||
action=task.action,
|
||||
payload=task.payload,
|
||||
idempotency_key=task.idempotency_key or task.task_id,
|
||||
key_id=self.openclaw_key_id,
|
||||
)
|
||||
command = sign_command(command, key_material)
|
||||
result = self.openclaw.execute(command, timeout_seconds=timeout_seconds)
|
||||
self.reconcile_openclaw(context, result)
|
||||
return DispatchRecord(
|
||||
tenant_id=task.tenant_id,
|
||||
task_id=task.task_id,
|
||||
agent_id=agent.agent_id,
|
||||
runtime="openclaw",
|
||||
runtime_execution_id=command.command_id,
|
||||
state=result.state.value,
|
||||
trace_id=context.trace_id,
|
||||
)
|
||||
|
||||
def _dispatch_hermes(
|
||||
self,
|
||||
context: RequestContext,
|
||||
agent: AgentRecord,
|
||||
task: TaskRecord,
|
||||
) -> DispatchRecord:
|
||||
if self.hermes is None:
|
||||
raise OrchestrationError("Hermes runtime is not configured")
|
||||
metadata = dict(agent.metadata)
|
||||
job = HermesJob.create(
|
||||
tenant_id=task.tenant_id,
|
||||
cell_id=str(metadata.get("cell_id") or self.default_cell_id),
|
||||
profile_id=str(metadata.get("profile_id") or self.default_profile_id),
|
||||
task_id=task.task_id,
|
||||
trace_id=context.trace_id,
|
||||
action=task.action,
|
||||
payload=task.payload,
|
||||
idempotency_key=task.idempotency_key or task.task_id,
|
||||
priority=int(metadata.get("priority", 100)),
|
||||
)
|
||||
queued = self.hermes.enqueue(job)
|
||||
return DispatchRecord(
|
||||
tenant_id=task.tenant_id,
|
||||
task_id=task.task_id,
|
||||
agent_id=agent.agent_id,
|
||||
runtime="hermes",
|
||||
runtime_execution_id=queued.job_id,
|
||||
state=queued.state.value,
|
||||
trace_id=context.trace_id,
|
||||
)
|
||||
|
||||
def reconcile_openclaw(
|
||||
self,
|
||||
context: RequestContext,
|
||||
result: OpenClawResult,
|
||||
) -> TaskRecord:
|
||||
task = self.control_plane.get_task(context, result.task_id)
|
||||
if result.tenant_id != task.tenant_id:
|
||||
raise OrchestrationError("OpenClaw result tenant mismatch")
|
||||
if task.state != TaskState.RUNNING:
|
||||
return task
|
||||
if result.state == CommandState.COMPLETED:
|
||||
return self.control_plane.transition_task(
|
||||
context,
|
||||
task_id=task.task_id,
|
||||
target_state=TaskState.COMPLETED,
|
||||
expected_revision=task.revision,
|
||||
result=result.output or {},
|
||||
)
|
||||
if result.state in {
|
||||
CommandState.FAILED,
|
||||
CommandState.TIMED_OUT,
|
||||
CommandState.REJECTED,
|
||||
}:
|
||||
return self.control_plane.transition_task(
|
||||
context,
|
||||
task_id=task.task_id,
|
||||
target_state=TaskState.FAILED,
|
||||
expected_revision=task.revision,
|
||||
error=result.error or {"code": result.state.value.lower()},
|
||||
)
|
||||
if result.state == CommandState.CANCELLED:
|
||||
return self.control_plane.transition_task(
|
||||
context,
|
||||
task_id=task.task_id,
|
||||
target_state=TaskState.CANCELLED,
|
||||
expected_revision=task.revision,
|
||||
)
|
||||
return task
|
||||
|
||||
def reconcile_hermes(
|
||||
self,
|
||||
context: RequestContext,
|
||||
result: HermesResult,
|
||||
) -> TaskRecord:
|
||||
task = self.control_plane.get_task(context, result.task_id)
|
||||
if result.tenant_id != task.tenant_id:
|
||||
raise OrchestrationError("Hermes result tenant mismatch")
|
||||
if task.state != TaskState.RUNNING:
|
||||
return task
|
||||
if result.state.value == "COMPLETED":
|
||||
return self.control_plane.transition_task(
|
||||
context,
|
||||
task_id=task.task_id,
|
||||
target_state=TaskState.COMPLETED,
|
||||
expected_revision=task.revision,
|
||||
result=result.output or {},
|
||||
)
|
||||
if result.state.value == "FAILED":
|
||||
return self.control_plane.transition_task(
|
||||
context,
|
||||
task_id=task.task_id,
|
||||
target_state=TaskState.FAILED,
|
||||
expected_revision=task.revision,
|
||||
error=result.error or {"code": "hermes_failed"},
|
||||
)
|
||||
if result.state.value == "CANCELLED":
|
||||
return self.control_plane.transition_task(
|
||||
context,
|
||||
task_id=task.task_id,
|
||||
target_state=TaskState.CANCELLED,
|
||||
expected_revision=task.revision,
|
||||
)
|
||||
return task
|
||||
|
||||
def dispatch_snapshot(self, *, tenant_id: str) -> Mapping[str, DispatchRecord]:
|
||||
return {
|
||||
task_id: record
|
||||
for (record_tenant, task_id), record in self._dispatches.items()
|
||||
if record_tenant == tenant_id
|
||||
}
|
||||
@@ -0,0 +1,181 @@
|
||||
"""Durable single-process persistence for the AIOS V2 control plane.
|
||||
|
||||
The candidate uses an atomic SQLite snapshot so a gateway restart does not lose
|
||||
tenants, agents, task state, idempotency records, or audit events. The domain
|
||||
service remains storage-agnostic; a later HA backend can replace this module
|
||||
without changing API or runtime-adapter contracts.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
import threading
|
||||
from dataclasses import asdict
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable
|
||||
|
||||
from .control_plane import (
|
||||
AgentRecord,
|
||||
AuditEvent,
|
||||
ControlPlaneService,
|
||||
InMemoryControlPlaneStore,
|
||||
TaskRecord,
|
||||
TaskState,
|
||||
TenantRecord,
|
||||
)
|
||||
|
||||
|
||||
class SQLiteSnapshotRepository:
|
||||
"""Store one versioned control-plane snapshot in SQLite."""
|
||||
|
||||
SCHEMA_VERSION = 1
|
||||
|
||||
def __init__(self, path: str | Path) -> None:
|
||||
self.path = Path(path).resolve()
|
||||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||
self._lock = threading.RLock()
|
||||
self._initialize()
|
||||
|
||||
def _connect(self) -> sqlite3.Connection:
|
||||
connection = sqlite3.connect(self.path, timeout=30.0)
|
||||
connection.execute("PRAGMA journal_mode=WAL")
|
||||
connection.execute("PRAGMA synchronous=FULL")
|
||||
connection.execute("PRAGMA foreign_keys=ON")
|
||||
return connection
|
||||
|
||||
def _initialize(self) -> None:
|
||||
with self._connect() as connection:
|
||||
connection.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS control_plane_snapshot (
|
||||
singleton_id INTEGER PRIMARY KEY CHECK (singleton_id = 1),
|
||||
schema_version INTEGER NOT NULL,
|
||||
payload_json TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
)
|
||||
"""
|
||||
)
|
||||
|
||||
def load(self) -> dict[str, Any] | None:
|
||||
with self._lock, self._connect() as connection:
|
||||
row = connection.execute(
|
||||
"SELECT schema_version, payload_json FROM control_plane_snapshot WHERE singleton_id = 1"
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
schema_version, payload_json = row
|
||||
if int(schema_version) != self.SCHEMA_VERSION:
|
||||
raise RuntimeError(
|
||||
f"unsupported control-plane snapshot version: {schema_version}"
|
||||
)
|
||||
decoded = json.loads(str(payload_json))
|
||||
if not isinstance(decoded, dict):
|
||||
raise RuntimeError("invalid control-plane snapshot")
|
||||
return decoded
|
||||
|
||||
def save(self, payload: dict[str, Any]) -> None:
|
||||
encoded = json.dumps(
|
||||
payload,
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
default=str,
|
||||
)
|
||||
with self._lock, self._connect() as connection:
|
||||
connection.execute("BEGIN IMMEDIATE")
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO control_plane_snapshot (
|
||||
singleton_id, schema_version, payload_json, updated_at
|
||||
) VALUES (1, ?, ?, CURRENT_TIMESTAMP)
|
||||
ON CONFLICT(singleton_id) DO UPDATE SET
|
||||
schema_version = excluded.schema_version,
|
||||
payload_json = excluded.payload_json,
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
""",
|
||||
(self.SCHEMA_VERSION, encoded),
|
||||
)
|
||||
connection.commit()
|
||||
|
||||
|
||||
class DurableControlPlaneService(ControlPlaneService):
|
||||
"""Control-plane service that persists after every successful mutation."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
repository: SQLiteSnapshotRepository,
|
||||
*,
|
||||
policy_evaluator=None,
|
||||
) -> None:
|
||||
self.repository = repository
|
||||
store = InMemoryControlPlaneStore()
|
||||
self._restore(store, repository.load())
|
||||
super().__init__(store, policy_evaluator=policy_evaluator)
|
||||
|
||||
@staticmethod
|
||||
def _restore(
|
||||
store: InMemoryControlPlaneStore,
|
||||
snapshot: dict[str, Any] | None,
|
||||
) -> None:
|
||||
if not snapshot:
|
||||
return
|
||||
for item in snapshot.get("tenants", []):
|
||||
record = TenantRecord(**item)
|
||||
store.tenants[record.tenant_id] = record
|
||||
for item in snapshot.get("agents", []):
|
||||
normalized = dict(item)
|
||||
normalized["capabilities"] = tuple(normalized.get("capabilities", ()))
|
||||
record = AgentRecord(**normalized)
|
||||
store.agents[record.agent_id] = record
|
||||
for item in snapshot.get("tasks", []):
|
||||
normalized = dict(item)
|
||||
normalized["state"] = TaskState(normalized["state"])
|
||||
record = TaskRecord(**normalized)
|
||||
store.tasks[record.task_id] = record
|
||||
store.audit_events.extend(AuditEvent(**item) for item in snapshot.get("audit", []))
|
||||
for item in snapshot.get("idempotency", []):
|
||||
store.idempotency[(item["tenant_id"], item["key"])] = (
|
||||
item["task_id"],
|
||||
item["digest"],
|
||||
)
|
||||
|
||||
def _snapshot(self) -> dict[str, Any]:
|
||||
with self.store.lock:
|
||||
return {
|
||||
"schema_version": self.repository.SCHEMA_VERSION,
|
||||
"tenants": [asdict(item) for item in self.store.tenants.values()],
|
||||
"agents": [asdict(item) for item in self.store.agents.values()],
|
||||
"tasks": [
|
||||
{**asdict(item), "state": item.state.value}
|
||||
for item in self.store.tasks.values()
|
||||
],
|
||||
"audit": [asdict(item) for item in self.store.audit_events],
|
||||
"idempotency": [
|
||||
{
|
||||
"tenant_id": tenant_id,
|
||||
"key": key,
|
||||
"task_id": task_id,
|
||||
"digest": digest,
|
||||
}
|
||||
for (tenant_id, key), (task_id, digest)
|
||||
in self.store.idempotency.items()
|
||||
],
|
||||
}
|
||||
|
||||
def _mutate(self, operation: Callable[[], Any]) -> Any:
|
||||
with self.store.lock:
|
||||
result = operation()
|
||||
self.repository.save(self._snapshot())
|
||||
return result
|
||||
|
||||
def register_tenant(self, *args, **kwargs):
|
||||
return self._mutate(lambda: super(DurableControlPlaneService, self).register_tenant(*args, **kwargs))
|
||||
|
||||
def register_agent(self, *args, **kwargs):
|
||||
return self._mutate(lambda: super(DurableControlPlaneService, self).register_agent(*args, **kwargs))
|
||||
|
||||
def create_task(self, *args, **kwargs):
|
||||
return self._mutate(lambda: super(DurableControlPlaneService, self).create_task(*args, **kwargs))
|
||||
|
||||
def transition_task(self, *args, **kwargs):
|
||||
return self._mutate(lambda: super(DurableControlPlaneService, self).transition_task(*args, **kwargs))
|
||||
@@ -1,355 +0,0 @@
|
||||
"""Fail-closed access to the authoritative bare Git repository."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import urllib.parse
|
||||
from pathlib import Path
|
||||
from typing import Iterable
|
||||
|
||||
_ALLOWED_COMMANDS = frozenset(
|
||||
{
|
||||
"cat-file",
|
||||
"config",
|
||||
"diff",
|
||||
"for-each-ref",
|
||||
"ls-remote",
|
||||
"merge-base",
|
||||
"push",
|
||||
"remote",
|
||||
"rev-parse",
|
||||
"show-ref",
|
||||
"tag",
|
||||
}
|
||||
)
|
||||
_OID_RE = re.compile(r"^[0-9a-f]{40}$")
|
||||
_MAX_OUTPUT = 1_048_576
|
||||
_TIMEOUT = 8.0
|
||||
|
||||
|
||||
class AuthoritativeGitError(RuntimeError):
|
||||
"""A fail-closed authoritative repository or ref query error."""
|
||||
|
||||
|
||||
class AuthoritativeGitNotFound(AuthoritativeGitError):
|
||||
"""A validated revision does not exist in the authoritative repository."""
|
||||
|
||||
|
||||
class AuthoritativeGitOutputTooLarge(AuthoritativeGitError):
|
||||
"""A controlled Git query exceeded the response-size boundary."""
|
||||
|
||||
|
||||
def _redact(text: str) -> str:
|
||||
value = str(text or "")
|
||||
value = re.sub(r"(?i)(authorization\s*[:=]\s*)(?:bearer|token)\s+\S+", r"\1REDACTED", value)
|
||||
value = re.sub(r"(?i)((?:access_)?token\s*[:=]\s*)\S+", r"\1REDACTED", value)
|
||||
value = re.sub(r"(?i)(https?://[^/\s:@]+:)[^@\s/]+@", r"\1REDACTED@", value)
|
||||
return value
|
||||
|
||||
|
||||
def _configured_path(git_dir: str | Path | None = None) -> Path:
|
||||
raw = str(git_dir or os.getenv("AIOS_GITOPS_AUTHORITATIVE_GIT_DIR", "")).strip()
|
||||
if not raw:
|
||||
raise AuthoritativeGitError("AUTHORITATIVE_REPOSITORY_UNAVAILABLE")
|
||||
path = Path(raw)
|
||||
if not path.is_absolute() or ".." in path.parts:
|
||||
raise AuthoritativeGitError("AUTHORITATIVE_REPOSITORY_UNAVAILABLE")
|
||||
try:
|
||||
resolved = path.resolve(strict=True)
|
||||
except OSError as exc:
|
||||
raise AuthoritativeGitError("AUTHORITATIVE_REPOSITORY_UNAVAILABLE") from exc
|
||||
if not resolved.is_dir():
|
||||
raise AuthoritativeGitError("AUTHORITATIVE_REPOSITORY_UNAVAILABLE")
|
||||
return resolved
|
||||
|
||||
|
||||
def _run(
|
||||
git_dir: Path,
|
||||
args: list[str],
|
||||
*,
|
||||
check: bool,
|
||||
timeout: float = _TIMEOUT,
|
||||
) -> subprocess.CompletedProcess[str]:
|
||||
if not args or args[0] not in _ALLOWED_COMMANDS:
|
||||
raise AuthoritativeGitError("GIT_REF_QUERY_FAILED")
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "--git-dir", str(git_dir), *args],
|
||||
text=True,
|
||||
capture_output=True,
|
||||
timeout=timeout,
|
||||
env={**os.environ, "GIT_OPTIONAL_LOCKS": "0"},
|
||||
)
|
||||
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||
raise AuthoritativeGitError("GIT_REF_QUERY_FAILED") from exc
|
||||
if len(result.stdout) + len(result.stderr) > _MAX_OUTPUT:
|
||||
raise AuthoritativeGitOutputTooLarge("GIT_OUTPUT_LIMIT_EXCEEDED")
|
||||
if check and result.returncode != 0:
|
||||
detail = _redact(result.stderr.strip() or result.stdout.strip())
|
||||
raise AuthoritativeGitError(
|
||||
"GIT_REF_QUERY_FAILED" + (f": {detail[:160]}" if detail else "")
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def _repository_identity_from_remote(url: str) -> str:
|
||||
value = str(url or "").strip().split("?", 1)[0].rstrip("/")
|
||||
if not value:
|
||||
return ""
|
||||
if "://" in value:
|
||||
value = urllib.parse.urlparse(value).path
|
||||
elif ":" in value:
|
||||
value = value.rsplit(":", 1)[1]
|
||||
value = value.rsplit("@", 1)[-1].strip("/")
|
||||
value = re.sub(r"\.git$", "", value, flags=re.IGNORECASE)
|
||||
parts = [part for part in value.split("/") if part]
|
||||
if len(parts) >= 2:
|
||||
value = "/".join(parts[-2:])
|
||||
return value.strip("/").lower()
|
||||
|
||||
|
||||
def verify_repository(
|
||||
git_dir: str | Path | None = None,
|
||||
repository_identity: str | None = None,
|
||||
) -> tuple[Path, str]:
|
||||
path = _configured_path(git_dir)
|
||||
expected = str(
|
||||
repository_identity
|
||||
or os.getenv("AIOS_GITOPS_REPOSITORY_IDENTITY", "")
|
||||
).strip().strip("/")
|
||||
if not expected or expected.count("/") != 1:
|
||||
raise AuthoritativeGitError("REPOSITORY_IDENTITY_MISMATCH")
|
||||
bare = _run(path, ["rev-parse", "--is-bare-repository"], check=True)
|
||||
if bare.stdout.strip().lower() != "true":
|
||||
raise AuthoritativeGitError("AUTHORITATIVE_REPOSITORY_UNAVAILABLE")
|
||||
remote = _run(path, ["config", "--get", "remote.gitea.url"], check=False)
|
||||
actual = _repository_identity_from_remote(remote.stdout)
|
||||
if actual != expected.lower():
|
||||
raise AuthoritativeGitError("REPOSITORY_IDENTITY_MISMATCH")
|
||||
return path, expected
|
||||
|
||||
|
||||
def run_authoritative_git(
|
||||
args: list[str],
|
||||
*,
|
||||
check: bool = True,
|
||||
git_dir: str | Path | None = None,
|
||||
timeout: float = _TIMEOUT,
|
||||
) -> subprocess.CompletedProcess[str]:
|
||||
path, _ = verify_repository(git_dir=git_dir)
|
||||
return _run(path, list(args), check=check, timeout=timeout)
|
||||
|
||||
|
||||
def authoritative_head_oid(
|
||||
*,
|
||||
git_dir: str | Path | None = None,
|
||||
) -> str:
|
||||
result = run_authoritative_git(["rev-parse", "--verify", "refs/heads/main"], git_dir=git_dir)
|
||||
oid = result.stdout.strip().lower()
|
||||
if not _OID_RE.fullmatch(oid):
|
||||
raise AuthoritativeGitError("GIT_REF_QUERY_FAILED")
|
||||
return oid
|
||||
|
||||
|
||||
def authoritative_tags(
|
||||
*,
|
||||
git_dir: str | Path | None = None,
|
||||
) -> list[dict[str, str | None]]:
|
||||
result = run_authoritative_git(
|
||||
[
|
||||
"for-each-ref",
|
||||
"refs/tags",
|
||||
"--format=%(refname:strip=2)\t%(objectname)\t%(objecttype)",
|
||||
],
|
||||
git_dir=git_dir,
|
||||
)
|
||||
tags: list[dict[str, str | None]] = []
|
||||
for line in result.stdout.splitlines():
|
||||
name, oid, object_type = (line.split("\t", 2) + ["", ""])[:3]
|
||||
if not name or not _OID_RE.fullmatch(oid.lower()):
|
||||
raise AuthoritativeGitError("GIT_REF_QUERY_FAILED")
|
||||
peeled_result = run_authoritative_git(
|
||||
["rev-parse", "--verify", f"refs/tags/{name}^{{commit}}"],
|
||||
git_dir=git_dir,
|
||||
)
|
||||
peeled = peeled_result.stdout.strip().lower()
|
||||
if not _OID_RE.fullmatch(peeled):
|
||||
raise AuthoritativeGitError("GIT_REF_QUERY_FAILED")
|
||||
tags.append(
|
||||
{
|
||||
"name": name,
|
||||
"tag_oid": oid.lower(),
|
||||
"object_type": object_type,
|
||||
"peeled_target_oid": peeled,
|
||||
}
|
||||
)
|
||||
return tags
|
||||
|
||||
|
||||
def authoritative_branches(
|
||||
*,
|
||||
git_dir: str | Path | None = None,
|
||||
) -> list[dict[str, str]]:
|
||||
result = run_authoritative_git(
|
||||
[
|
||||
"for-each-ref",
|
||||
"refs/heads",
|
||||
"--format=%(refname:strip=2)\t%(objectname)",
|
||||
],
|
||||
git_dir=git_dir,
|
||||
)
|
||||
branches: list[dict[str, str]] = []
|
||||
for line in result.stdout.splitlines():
|
||||
name, oid = (line.split("\t", 1) + [""])[:2]
|
||||
if not name or not _OID_RE.fullmatch(oid.lower()):
|
||||
raise AuthoritativeGitError("GIT_REF_QUERY_FAILED")
|
||||
branches.append({"name": name, "oid": oid.lower()})
|
||||
return branches
|
||||
|
||||
|
||||
def authoritative_commit(
|
||||
oid: str,
|
||||
*,
|
||||
git_dir: str | Path | None = None,
|
||||
) -> str:
|
||||
value = str(oid or "").strip().lower()
|
||||
if not _OID_RE.fullmatch(value):
|
||||
raise ValueError("commit OID must be a full 40-character value")
|
||||
result = run_authoritative_git(
|
||||
["rev-parse", "--verify", f"{value}^{{commit}}"],
|
||||
git_dir=git_dir,
|
||||
)
|
||||
resolved = result.stdout.strip().lower()
|
||||
if resolved != value:
|
||||
raise AuthoritativeGitError("GIT_REF_QUERY_FAILED")
|
||||
return resolved
|
||||
|
||||
|
||||
def authoritative_lineage(
|
||||
ancestor: str,
|
||||
descendant: str,
|
||||
*,
|
||||
git_dir: str | Path | None = None,
|
||||
) -> None:
|
||||
authoritative_commit(ancestor, git_dir=git_dir)
|
||||
authoritative_commit(descendant, git_dir=git_dir)
|
||||
result = run_authoritative_git(
|
||||
["merge-base", "--is-ancestor", ancestor, descendant],
|
||||
check=False,
|
||||
git_dir=git_dir,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
raise AuthoritativeGitError("evidence lineage mismatch")
|
||||
|
||||
|
||||
_SAFE_REF_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._/-]{0,239}$")
|
||||
|
||||
|
||||
def _safe_ref_name(value: str) -> bool:
|
||||
return bool(
|
||||
_SAFE_REF_RE.fullmatch(value)
|
||||
and ".." not in value
|
||||
and "//" not in value
|
||||
and "@{" not in value
|
||||
and not value.endswith(("/", ".", ".lock"))
|
||||
and not any(part in {"", ".", ".."} for part in value.split("/"))
|
||||
)
|
||||
|
||||
|
||||
def validate_git_revision(
|
||||
value: str,
|
||||
field: str,
|
||||
*,
|
||||
git_dir: str | Path | None = None,
|
||||
) -> str:
|
||||
revision = str(value or "").strip()
|
||||
if not revision or revision != str(value or "") or revision.startswith("-"):
|
||||
raise ValueError(f"invalid {field} revision")
|
||||
if _OID_RE.fullmatch(revision.lower()):
|
||||
try:
|
||||
return authoritative_commit(revision.lower(), git_dir=git_dir)
|
||||
except AuthoritativeGitError as exc:
|
||||
raise AuthoritativeGitNotFound(f"{field} revision not found") from exc
|
||||
if not _safe_ref_name(revision) or any(token in revision for token in ("^", "~", ":", "\\")):
|
||||
raise ValueError(f"invalid {field} revision")
|
||||
if revision.startswith("refs/"):
|
||||
if not revision.startswith(("refs/heads/", "refs/tags/")):
|
||||
raise ValueError(f"invalid {field} revision")
|
||||
candidates = [revision]
|
||||
else:
|
||||
candidates = [f"refs/heads/{revision}", f"refs/tags/{revision}"]
|
||||
matches: list[str] = []
|
||||
for candidate in candidates:
|
||||
result = run_authoritative_git(
|
||||
["rev-parse", "--verify", f"{candidate}^{{commit}}"],
|
||||
check=False,
|
||||
git_dir=git_dir,
|
||||
)
|
||||
oid = result.stdout.strip().lower()
|
||||
if result.returncode == 0 and _OID_RE.fullmatch(oid):
|
||||
matches.append(oid)
|
||||
if not matches:
|
||||
raise AuthoritativeGitNotFound(f"{field} revision not found")
|
||||
if len(set(matches)) != 1 or len(matches) != 1:
|
||||
raise ValueError(f"ambiguous {field} revision")
|
||||
return matches[0]
|
||||
|
||||
|
||||
def authoritative_diff(
|
||||
from_revision: str,
|
||||
to_revision: str,
|
||||
*,
|
||||
git_dir: str | Path | None = None,
|
||||
) -> dict[str, object]:
|
||||
path, _ = verify_repository(git_dir=git_dir)
|
||||
from_oid = validate_git_revision(from_revision, "from", git_dir=path)
|
||||
to_oid = validate_git_revision(to_revision, "to", git_dir=path)
|
||||
args = ["diff", "--no-ext-diff", "--no-textconv", from_oid, to_oid]
|
||||
result = run_authoritative_git(args, git_dir=path)
|
||||
return {
|
||||
"code": 0,
|
||||
"stdout": result.stdout,
|
||||
"stderr": "",
|
||||
"cmd": ["git", f"--git-dir={path}", *args],
|
||||
"from_oid": from_oid,
|
||||
"to_oid": to_oid,
|
||||
}
|
||||
|
||||
|
||||
def repository_snapshot(
|
||||
*,
|
||||
git_dir: str | Path | None = None,
|
||||
repository_identity: str | None = None,
|
||||
) -> dict[str, object]:
|
||||
configured = bool(
|
||||
str(git_dir or os.getenv("AIOS_GITOPS_AUTHORITATIVE_GIT_DIR", "")).strip()
|
||||
)
|
||||
try:
|
||||
path, identity = verify_repository(
|
||||
git_dir=git_dir,
|
||||
repository_identity=repository_identity,
|
||||
)
|
||||
head = authoritative_head_oid(git_dir=path)
|
||||
return {
|
||||
"repository_available": True,
|
||||
"authoritative_git_dir_configured": configured,
|
||||
"authoritative_git_dir_readable": True,
|
||||
"authoritative_git_dir": str(path),
|
||||
"authoritative_head_oid": head,
|
||||
"repository_identity": identity,
|
||||
"query_error": None,
|
||||
}
|
||||
except (AuthoritativeGitError, ValueError) as exc:
|
||||
return {
|
||||
"repository_available": False,
|
||||
"authoritative_git_dir_configured": configured,
|
||||
"authoritative_git_dir_readable": False,
|
||||
"authoritative_git_dir": None,
|
||||
"authoritative_head_oid": None,
|
||||
"repository_identity": str(
|
||||
repository_identity
|
||||
or os.getenv("AIOS_GITOPS_REPOSITORY_IDENTITY", "")
|
||||
).strip() or None,
|
||||
"query_error": _redact(str(exc)),
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,167 +0,0 @@
|
||||
"""Fail-closed operation authorization for an isolated Git writer."""
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from isolated_writer_token_registry import TokenRegistry
|
||||
|
||||
|
||||
ROUTE_OPERATIONS = {
|
||||
("GET", "/api/runtime-status"): "runtime.status.read",
|
||||
("GET", "/api/baseline"): "baseline.read",
|
||||
("GET", "/api/patches"): "patches.read",
|
||||
("POST", "/api/commit-file"): "commit_file.atomic",
|
||||
}
|
||||
OID_RE = re.compile(r"[0-9a-f]{40}")
|
||||
|
||||
|
||||
class AuthorizationError(PermissionError):
|
||||
def __init__(self, code: str, status: int = 403):
|
||||
super().__init__(code)
|
||||
self.code = code
|
||||
self.status = status
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AuthorizedGrant:
|
||||
record: dict[str, Any]
|
||||
operation: str
|
||||
use_count: int
|
||||
|
||||
@property
|
||||
def token_id(self) -> str:
|
||||
return str(self.record["token_id"])
|
||||
|
||||
@property
|
||||
def task_id(self) -> str:
|
||||
return str(self.record["task_id"])
|
||||
|
||||
|
||||
def operation_for(method: str, path: str) -> str:
|
||||
operation = ROUTE_OPERATIONS.get((method.upper(), path))
|
||||
if not operation:
|
||||
raise AuthorizationError("isolated_writer_operation_forbidden")
|
||||
return operation
|
||||
|
||||
|
||||
def _parse_time(value: Any) -> datetime:
|
||||
try:
|
||||
parsed = datetime.fromisoformat(str(value).replace("Z", "+00:00"))
|
||||
except (TypeError, ValueError):
|
||||
raise AuthorizationError("writer_token_time_invalid") from None
|
||||
if parsed.tzinfo is None:
|
||||
raise AuthorizationError("writer_token_time_invalid")
|
||||
return parsed.astimezone(timezone.utc)
|
||||
|
||||
|
||||
def _git_is_ancestor(workspace: Path, base: str, branch: str) -> bool:
|
||||
result = subprocess.run(
|
||||
["git", "merge-base", "--is-ancestor", base, branch],
|
||||
cwd=workspace,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
)
|
||||
return result.returncode == 0
|
||||
|
||||
|
||||
def _validate_grant(
|
||||
grant: dict[str, Any],
|
||||
*,
|
||||
audience: str,
|
||||
source_ip: str,
|
||||
operation: str,
|
||||
branch: str | None,
|
||||
path: str | None,
|
||||
workspace: Path,
|
||||
current_time: datetime,
|
||||
) -> None:
|
||||
if int(grant.get("schema_version", 0)) != 2:
|
||||
raise AuthorizationError("writer_token_policy_v2_required", 401)
|
||||
if str(grant.get("audience") or "") != audience:
|
||||
raise AuthorizationError("writer_token_audience_rejected")
|
||||
try:
|
||||
peer = ipaddress.ip_address(source_ip)
|
||||
networks = [ipaddress.ip_network(str(value), strict=True) for value in grant.get("source_cidrs", [])]
|
||||
except ValueError:
|
||||
raise AuthorizationError("writer_token_source_rejected") from None
|
||||
if not networks or not any(peer in network for network in networks):
|
||||
raise AuthorizationError("writer_token_source_rejected")
|
||||
created = _parse_time(grant.get("created_at"))
|
||||
not_before = _parse_time(grant.get("not_before"))
|
||||
expires = _parse_time(grant.get("expires_at"))
|
||||
if expires <= not_before or (expires - created).total_seconds() > 1800:
|
||||
raise AuthorizationError("writer_token_lifetime_rejected")
|
||||
if current_time < not_before:
|
||||
raise AuthorizationError("writer_token_not_yet_valid", 401)
|
||||
if current_time >= expires:
|
||||
raise AuthorizationError("writer_token_expired", 401)
|
||||
if bool(grant.get("revoked")):
|
||||
raise AuthorizationError("writer_token_revoked", 401)
|
||||
operations = grant.get("allowed_operations")
|
||||
if not isinstance(operations, list) or operation not in operations:
|
||||
raise AuthorizationError("writer_token_operation_rejected")
|
||||
expected_branch = str(grant.get("allowed_branch") or "")
|
||||
configured_branch = os.getenv("AIOS_GITOPS_ALLOWED_BRANCH", "")
|
||||
if expected_branch != configured_branch:
|
||||
raise AuthorizationError("writer_token_branch_rejected")
|
||||
if branch is not None and branch != expected_branch:
|
||||
raise AuthorizationError("writer_token_branch_rejected")
|
||||
if path is not None:
|
||||
paths = grant.get("allowed_paths")
|
||||
if not isinstance(paths, list) or path not in paths:
|
||||
raise AuthorizationError("writer_token_path_rejected")
|
||||
base = str(grant.get("approved_base_oid") or "").lower()
|
||||
if not OID_RE.fullmatch(base):
|
||||
raise AuthorizationError("writer_token_base_rejected")
|
||||
if not _git_is_ancestor(workspace, base, expected_branch):
|
||||
raise AuthorizationError("writer_token_base_not_ancestor")
|
||||
|
||||
|
||||
def authorize(
|
||||
registry: TokenRegistry,
|
||||
token: str,
|
||||
*,
|
||||
method: str,
|
||||
path: str,
|
||||
source_ip: str,
|
||||
workspace: Path,
|
||||
branch: str | None = None,
|
||||
target_path: str | None = None,
|
||||
current_time: datetime | None = None,
|
||||
) -> AuthorizedGrant:
|
||||
if os.getenv("AIOS_GITOPS_TOKEN_POLICY_VERSION_REQUIRED", "") != "2":
|
||||
raise AuthorizationError("writer_token_policy_v2_not_required")
|
||||
if not token:
|
||||
raise AuthorizationError("unauthorized", 401)
|
||||
grant = registry.find(token)
|
||||
if grant is None:
|
||||
raise AuthorizationError("unauthorized", 401)
|
||||
operation = operation_for(method, path)
|
||||
_validate_grant(
|
||||
grant,
|
||||
audience=os.getenv("AIOS_GITOPS_SERVICE_AUDIENCE", ""),
|
||||
source_ip=source_ip,
|
||||
operation=operation,
|
||||
branch=branch,
|
||||
path=target_path,
|
||||
workspace=workspace,
|
||||
current_time=current_time or datetime.now(timezone.utc),
|
||||
)
|
||||
limits = grant.get("max_uses")
|
||||
if not isinstance(limits, dict):
|
||||
raise AuthorizationError("writer_token_use_limit_missing")
|
||||
try:
|
||||
used = registry.consume(str(grant["token_id"]), operation, int(limits[operation]))
|
||||
except (KeyError, TypeError, ValueError):
|
||||
raise AuthorizationError("writer_token_use_limit_missing") from None
|
||||
except PermissionError as exc:
|
||||
code = str(exc)
|
||||
raise AuthorizationError(code, 401 if code == "writer_token_revoked" else 403) from None
|
||||
return AuthorizedGrant(grant, operation, used)
|
||||
@@ -1,403 +0,0 @@
|
||||
"""Atomic, idempotent commit-file Saga for the isolated writer."""
|
||||
from __future__ import annotations
|
||||
|
||||
import fcntl
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sqlite3
|
||||
import subprocess
|
||||
import tempfile
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import aios_isolated_writer_policy as policy
|
||||
|
||||
|
||||
OID_RE = re.compile(r"[0-9a-f]{40}")
|
||||
IDEMPOTENCY_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._:-]{7,127}")
|
||||
STATES = frozenset({
|
||||
"PENDING", "FILE_WRITTEN", "COMMITTED", "PUSHED", "VERIFIED", "COMPLETED",
|
||||
"FAILED_RECOVERABLE", "MANUAL_REVIEW_REQUIRED",
|
||||
})
|
||||
_LOCKS: dict[str, threading.Lock] = {}
|
||||
_LOCKS_GUARD = threading.Lock()
|
||||
_LEDGER_INIT_LOCK = threading.Lock()
|
||||
|
||||
|
||||
class CommitFileError(RuntimeError):
|
||||
def __init__(self, code: str, status: int = 400):
|
||||
super().__init__(code)
|
||||
self.code = code
|
||||
self.status = status
|
||||
|
||||
|
||||
class SimulatedCrash(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
def canonical_digest(data: dict[str, Any]) -> str:
|
||||
payload = {
|
||||
"branch": data["branch"],
|
||||
"path": data["path"],
|
||||
"content_sha256": hashlib.sha256(data["content"].encode("utf-8")).hexdigest(),
|
||||
"message": data["message"],
|
||||
"expected_head_oid": data["expected_head_oid"],
|
||||
}
|
||||
encoded = json.dumps(payload, sort_keys=True, separators=(",", ":")).encode("utf-8")
|
||||
return hashlib.sha256(encoded).hexdigest()
|
||||
|
||||
|
||||
class OperationLedger:
|
||||
def __init__(self, path: Path | str):
|
||||
self.path = Path(path)
|
||||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||
if self.path.parent.is_symlink() or self.path.is_symlink():
|
||||
raise PermissionError("writer_operation_ledger_symlink_rejected")
|
||||
with _LEDGER_INIT_LOCK:
|
||||
with self.connect() as db:
|
||||
db.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS writer_operations (
|
||||
operation_id TEXT PRIMARY KEY,
|
||||
token_id TEXT NOT NULL,
|
||||
task_id TEXT NOT NULL,
|
||||
idempotency_key TEXT NOT NULL UNIQUE,
|
||||
operation_digest TEXT NOT NULL,
|
||||
branch TEXT NOT NULL,
|
||||
path TEXT NOT NULL,
|
||||
content_sha256 TEXT NOT NULL,
|
||||
pre_head_oid TEXT NOT NULL,
|
||||
commit_oid TEXT,
|
||||
push_status TEXT,
|
||||
verification_status TEXT,
|
||||
state TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
error_code TEXT
|
||||
)
|
||||
"""
|
||||
)
|
||||
os.chmod(self.path, 0o600)
|
||||
|
||||
def connect(self) -> sqlite3.Connection:
|
||||
db = sqlite3.connect(self.path, timeout=30)
|
||||
db.row_factory = sqlite3.Row
|
||||
db.execute("PRAGMA journal_mode=WAL")
|
||||
db.execute("PRAGMA synchronous=FULL")
|
||||
return db
|
||||
|
||||
def get_or_create(self, grant: Any, data: dict[str, Any], digest: str) -> dict[str, Any]:
|
||||
key = data["idempotency_key"]
|
||||
with self.connect() as db:
|
||||
db.execute("BEGIN IMMEDIATE")
|
||||
row = db.execute(
|
||||
"SELECT * FROM writer_operations WHERE idempotency_key=?", (key,)
|
||||
).fetchone()
|
||||
if row:
|
||||
db.commit()
|
||||
record = dict(row)
|
||||
if record["operation_digest"] != digest:
|
||||
raise CommitFileError("writer_idempotency_conflict", 409)
|
||||
if (
|
||||
record["token_id"] != grant.token_id
|
||||
or record["task_id"] != grant.task_id
|
||||
):
|
||||
raise CommitFileError("writer_idempotency_owner_conflict", 409)
|
||||
return record
|
||||
operation_id = hashlib.sha256(
|
||||
f"{grant.task_id}\0{key}".encode("utf-8")
|
||||
).hexdigest()
|
||||
content_sha = hashlib.sha256(data["content"].encode("utf-8")).hexdigest()
|
||||
db.execute(
|
||||
"""
|
||||
INSERT INTO writer_operations(
|
||||
operation_id,token_id,task_id,idempotency_key,operation_digest,
|
||||
branch,path,content_sha256,pre_head_oid,state
|
||||
) VALUES(?,?,?,?,?,?,?,?,?,?)
|
||||
""",
|
||||
(
|
||||
operation_id, grant.token_id, grant.task_id, key, digest,
|
||||
data["branch"], data["path"], content_sha,
|
||||
data["expected_head_oid"], "PENDING",
|
||||
),
|
||||
)
|
||||
db.commit()
|
||||
return self.get(key)
|
||||
|
||||
def get(self, key: str) -> dict[str, Any]:
|
||||
with self.connect() as db:
|
||||
row = db.execute(
|
||||
"SELECT * FROM writer_operations WHERE idempotency_key=?", (key,)
|
||||
).fetchone()
|
||||
if row is None:
|
||||
raise KeyError(key)
|
||||
return dict(row)
|
||||
|
||||
def update(self, key: str, state: str, **fields: Any) -> dict[str, Any]:
|
||||
if state not in STATES:
|
||||
raise ValueError("invalid writer operation state")
|
||||
allowed = {"commit_oid", "push_status", "verification_status", "error_code"}
|
||||
assignments = ["state=?", "updated_at=CURRENT_TIMESTAMP"]
|
||||
values: list[Any] = [state]
|
||||
for name, value in fields.items():
|
||||
if name not in allowed:
|
||||
raise ValueError("invalid ledger field")
|
||||
assignments.append(f"{name}=?")
|
||||
values.append(value)
|
||||
values.append(key)
|
||||
with self.connect() as db:
|
||||
db.execute("BEGIN IMMEDIATE")
|
||||
db.execute(
|
||||
f"UPDATE writer_operations SET {','.join(assignments)} WHERE idempotency_key=?",
|
||||
values,
|
||||
)
|
||||
db.commit()
|
||||
return self.get(key)
|
||||
|
||||
|
||||
class AtomicCommitFile:
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
workspace: Path,
|
||||
bare_repo: Path,
|
||||
ledger: OperationLedger,
|
||||
remote: str,
|
||||
audit,
|
||||
crash_after_state: str | None = None,
|
||||
):
|
||||
self.workspace = workspace
|
||||
self.bare_repo = bare_repo
|
||||
self.ledger = ledger
|
||||
self.remote = remote
|
||||
self.audit = audit
|
||||
self.crash_after_state = crash_after_state
|
||||
|
||||
def _git(self, *args: str, check: bool = True) -> str:
|
||||
result = subprocess.run(
|
||||
["git", *args], cwd=self.workspace, text=True, capture_output=True
|
||||
)
|
||||
if check and result.returncode != 0:
|
||||
raise CommitFileError("writer_git_operation_failed", 500)
|
||||
return result.stdout.strip()
|
||||
|
||||
def _transition(self, key: str, state: str, **fields: Any) -> dict[str, Any]:
|
||||
record = self.ledger.update(key, state, **fields)
|
||||
if self.crash_after_state == state:
|
||||
raise SimulatedCrash(state)
|
||||
return record
|
||||
|
||||
def _validate(self, data: Any) -> dict[str, Any]:
|
||||
if not isinstance(data, dict):
|
||||
raise CommitFileError("writer_request_object_required")
|
||||
required = ("branch", "path", "content", "message", "expected_head_oid", "idempotency_key")
|
||||
if any(field not in data for field in required):
|
||||
raise CommitFileError("writer_request_incomplete")
|
||||
if data.get("force") or data.get("force_push"):
|
||||
raise CommitFileError("isolated_writer_force_rejected", 403)
|
||||
if set(data) != set(required):
|
||||
raise CommitFileError("writer_request_fields_rejected")
|
||||
item = {field: data[field] for field in required}
|
||||
if item["branch"] != policy.allowed_branch():
|
||||
raise CommitFileError("isolated_writer_branch_rejected", 403)
|
||||
try:
|
||||
item["path"] = policy.validate_path(item["path"], self.workspace)
|
||||
except PermissionError as exc:
|
||||
raise CommitFileError(str(exc), 403) from None
|
||||
if not isinstance(item["content"], str) or "\x00" in item["content"]:
|
||||
raise CommitFileError("writer_content_rejected")
|
||||
try:
|
||||
policy.validate_content_for_path(item["path"], item["content"])
|
||||
except PermissionError as exc:
|
||||
raise CommitFileError(str(exc), 403) from None
|
||||
if len(item["content"].encode("utf-8")) > 1_048_576:
|
||||
raise CommitFileError("writer_content_too_large", 413)
|
||||
message = item["message"]
|
||||
if (
|
||||
not isinstance(message, str)
|
||||
or not 1 <= len(message) <= 200
|
||||
or "\x00" in message
|
||||
or "\n" in message
|
||||
or "\r" in message
|
||||
):
|
||||
raise CommitFileError("writer_message_rejected")
|
||||
if not isinstance(item["expected_head_oid"], str) or not OID_RE.fullmatch(item["expected_head_oid"]):
|
||||
raise CommitFileError("writer_expected_head_rejected")
|
||||
if not isinstance(item["idempotency_key"], str) or not IDEMPOTENCY_RE.fullmatch(item["idempotency_key"]):
|
||||
raise CommitFileError("writer_idempotency_key_rejected")
|
||||
return item
|
||||
|
||||
def _atomic_write(self, relative: str, content: str) -> None:
|
||||
target = self.workspace / relative
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
fd, temporary = tempfile.mkstemp(prefix=".writer-", dir=target.parent)
|
||||
try:
|
||||
with os.fdopen(fd, "wb", closefd=True) as stream:
|
||||
stream.write(content.encode("utf-8"))
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
os.replace(temporary, target)
|
||||
parent_fd = os.open(target.parent, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
|
||||
try:
|
||||
os.fsync(parent_fd)
|
||||
finally:
|
||||
os.close(parent_fd)
|
||||
finally:
|
||||
if os.path.exists(temporary):
|
||||
os.unlink(temporary)
|
||||
|
||||
def _verify_remote(self, branch: str, oid: str) -> None:
|
||||
bare = subprocess.run(
|
||||
["git", "--git-dir", str(self.bare_repo), "rev-parse", f"refs/heads/{branch}"],
|
||||
text=True, capture_output=True,
|
||||
)
|
||||
if bare.returncode != 0 or bare.stdout.strip() != oid:
|
||||
raise CommitFileError("writer_authoritative_bare_oid_mismatch", 500)
|
||||
remote = subprocess.run(
|
||||
["git", "ls-remote", "--heads", self.remote, f"refs/heads/{branch}"],
|
||||
cwd=self.workspace, text=True, capture_output=True,
|
||||
)
|
||||
if remote.returncode != 0:
|
||||
raise CommitFileError("writer_gitea_verification_failed", 503)
|
||||
fields = remote.stdout.strip().split()
|
||||
if len(fields) != 2:
|
||||
raise CommitFileError("writer_gitea_ref_missing", 503)
|
||||
if fields[0] != oid:
|
||||
raise CommitFileError("writer_gitea_oid_conflict", 409)
|
||||
|
||||
def execute(self, grant: Any, raw_data: Any) -> dict[str, Any]:
|
||||
data = self._validate(raw_data)
|
||||
digest = canonical_digest(data)
|
||||
branch = data["branch"]
|
||||
with _LOCKS_GUARD:
|
||||
lock = _LOCKS.setdefault(branch, threading.Lock())
|
||||
lock_path = self.ledger.path.parent / f"{hashlib.sha256(branch.encode()).hexdigest()}.lock"
|
||||
with lock, open(lock_path, "a+b") as branch_lock:
|
||||
fcntl.flock(branch_lock.fileno(), fcntl.LOCK_EX)
|
||||
return self._execute_locked(grant, data, digest)
|
||||
|
||||
def _execute_locked(self, grant: Any, data: dict[str, Any], digest: str) -> dict[str, Any]:
|
||||
key = data["idempotency_key"]
|
||||
record = self.ledger.get_or_create(grant, data, digest)
|
||||
if record["state"] == "COMPLETED":
|
||||
return self._result(record, idempotent=True)
|
||||
if self._git("rev-parse", "--abbrev-ref", "HEAD") != data["branch"]:
|
||||
record = self._transition(
|
||||
key, "MANUAL_REVIEW_REQUIRED", error_code="writer_current_branch_conflict"
|
||||
)
|
||||
self._audit(grant, data, digest, record, "manual_review_required")
|
||||
raise CommitFileError("writer_current_branch_conflict", 409)
|
||||
head = self._git("rev-parse", "HEAD")
|
||||
commit_oid = record.get("commit_oid")
|
||||
if head != record["pre_head_oid"] and head != commit_oid:
|
||||
record = self._transition(
|
||||
key, "MANUAL_REVIEW_REQUIRED", error_code="writer_branch_head_conflict"
|
||||
)
|
||||
self._audit(grant, data, digest, record, "manual_review_required")
|
||||
raise CommitFileError("writer_branch_head_conflict", 409)
|
||||
try:
|
||||
state = record["state"]
|
||||
if state == "MANUAL_REVIEW_REQUIRED":
|
||||
raise CommitFileError("writer_manual_review_required", 409)
|
||||
if state == "FAILED_RECOVERABLE":
|
||||
if record.get("push_status") == "confirmed":
|
||||
state = "PUSHED"
|
||||
else:
|
||||
state = "COMMITTED" if commit_oid else "FILE_WRITTEN"
|
||||
if state == "PENDING":
|
||||
self._atomic_write(data["path"], data["content"])
|
||||
record = self._transition(key, "FILE_WRITTEN")
|
||||
state = record["state"]
|
||||
if state == "FILE_WRITTEN":
|
||||
actual_sha = hashlib.sha256(
|
||||
(self.workspace / data["path"]).read_bytes()
|
||||
).hexdigest()
|
||||
if actual_sha != record["content_sha256"]:
|
||||
raise CommitFileError("writer_file_content_conflict", 409)
|
||||
staged_before = self._git("diff", "--cached", "--name-only", check=False).splitlines()
|
||||
if staged_before:
|
||||
raise CommitFileError("writer_extra_staged_files_rejected", 409)
|
||||
self._git("add", "--", data["path"])
|
||||
staged = self._git("diff", "--cached", "--name-only").splitlines()
|
||||
if staged != [data["path"]]:
|
||||
raise CommitFileError("writer_extra_staged_files_rejected", 409)
|
||||
self._git("commit", "-m", data["message"])
|
||||
commit_oid = self._git("rev-parse", "HEAD")
|
||||
record = self._transition(key, "COMMITTED", commit_oid=commit_oid)
|
||||
state = record["state"]
|
||||
if state == "COMMITTED":
|
||||
commit_oid = str(record["commit_oid"])
|
||||
pushed = subprocess.run(
|
||||
["git", "push", "--porcelain", self.remote, f"HEAD:refs/heads/{data['branch']}"],
|
||||
cwd=self.workspace, text=True, capture_output=True,
|
||||
)
|
||||
if pushed.returncode != 0:
|
||||
self._transition(
|
||||
key, "FAILED_RECOVERABLE",
|
||||
push_status="failed", error_code="writer_gitea_push_failed",
|
||||
)
|
||||
raise CommitFileError("writer_gitea_push_failed", 503)
|
||||
record = self._transition(key, "PUSHED", push_status="confirmed")
|
||||
state = record["state"]
|
||||
if state == "PUSHED":
|
||||
commit_oid = str(record["commit_oid"])
|
||||
self._verify_remote(data["branch"], commit_oid)
|
||||
record = self._transition(
|
||||
key, "VERIFIED", verification_status="bare+gitea"
|
||||
)
|
||||
state = record["state"]
|
||||
if state == "VERIFIED":
|
||||
record = self._transition(key, "COMPLETED", error_code=None)
|
||||
self._audit(grant, data, digest, record, "completed")
|
||||
return self._result(record, idempotent=False)
|
||||
except SimulatedCrash:
|
||||
raise
|
||||
except CommitFileError as exc:
|
||||
latest = self.ledger.get(key)
|
||||
if latest["state"] not in {"FAILED_RECOVERABLE", "MANUAL_REVIEW_REQUIRED"}:
|
||||
failure_state = (
|
||||
"MANUAL_REVIEW_REQUIRED"
|
||||
if exc.code in {
|
||||
"writer_gitea_oid_conflict",
|
||||
"writer_file_content_conflict",
|
||||
"writer_current_branch_conflict",
|
||||
}
|
||||
else "FAILED_RECOVERABLE"
|
||||
)
|
||||
latest = self._transition(key, failure_state, error_code=exc.code)
|
||||
self._audit(grant, data, digest, latest, "failed")
|
||||
raise
|
||||
|
||||
def _audit(self, grant: Any, data: dict[str, Any], digest: str, record: dict[str, Any], result: str) -> None:
|
||||
self.audit({
|
||||
"token_id": grant.token_id,
|
||||
"task_id": grant.task_id,
|
||||
"audience": grant.record["audience"],
|
||||
"source_ip": grant.record.get("_source_ip", "socket-peer"),
|
||||
"operation": grant.operation,
|
||||
"branch": data["branch"],
|
||||
"path": data["path"],
|
||||
"idempotency_key": data["idempotency_key"],
|
||||
"operation_digest": digest,
|
||||
"path_policy_sha256": policy.configured_path_policy_sha256(),
|
||||
"pre_head_oid": record["pre_head_oid"],
|
||||
"commit_oid": record.get("commit_oid"),
|
||||
"result": result,
|
||||
})
|
||||
|
||||
@staticmethod
|
||||
def _result(record: dict[str, Any], *, idempotent: bool) -> dict[str, Any]:
|
||||
return {
|
||||
"status": "completed",
|
||||
"state": record["state"],
|
||||
"operation_id": record["operation_id"],
|
||||
"branch": record["branch"],
|
||||
"path": record["path"],
|
||||
"commit_oid": record.get("commit_oid"),
|
||||
"push_status": record.get("push_status"),
|
||||
"verification_status": record.get("verification_status"),
|
||||
"idempotent": idempotent,
|
||||
}
|
||||
@@ -1,422 +0,0 @@
|
||||
"""Strict service-level exact-path policy for isolated Git writers."""
|
||||
from __future__ import annotations
|
||||
|
||||
import grp
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import stat
|
||||
from dataclasses import dataclass
|
||||
from functools import lru_cache
|
||||
from pathlib import Path, PurePosixPath, PureWindowsPath
|
||||
from typing import Any
|
||||
from urllib.parse import unquote
|
||||
|
||||
|
||||
MAX_POLICY_BYTES = 65_536
|
||||
POLICY_FIELDS = frozenset(
|
||||
{
|
||||
"schema_version",
|
||||
"policy_id",
|
||||
"audience",
|
||||
"repository_identity",
|
||||
"allowed_branch",
|
||||
"allowed_operations",
|
||||
}
|
||||
)
|
||||
OPERATION_FIELDS = frozenset({"exact_paths", "max_content_bytes", "content_profile"})
|
||||
SUPPORTED_OPERATIONS = frozenset({"commit_file.atomic"})
|
||||
SUPPORTED_CONTENT_PROFILES = frozenset({"systemd-codex-review-worker-v1"})
|
||||
_SECRET_RE = re.compile(
|
||||
r"(?i)(authorization\s*:|bearer\s+|(?:token|password|secret)\s*[:=])"
|
||||
)
|
||||
_SYSTEMD_NAME_RE = re.compile(r"[A-Za-z][A-Za-z0-9]*\Z")
|
||||
_SYSTEMD_MAX_LINE_BYTES = 4096
|
||||
_SYSTEMD_ALLOWED_DIRECTIVES = {
|
||||
"Unit": frozenset({"Description", "After", "Wants"}),
|
||||
"Service": frozenset(
|
||||
{
|
||||
"Type",
|
||||
"User",
|
||||
"Group",
|
||||
"WorkingDirectory",
|
||||
"Environment",
|
||||
"ExecStart",
|
||||
"Restart",
|
||||
"RestartSec",
|
||||
"NoNewPrivileges",
|
||||
"PrivateTmp",
|
||||
"PrivateDevices",
|
||||
"ProtectSystem",
|
||||
"ProtectHome",
|
||||
"ProtectKernelTunables",
|
||||
"ProtectKernelModules",
|
||||
"ProtectControlGroups",
|
||||
"LockPersonality",
|
||||
"RestrictSUIDSGID",
|
||||
"RestrictRealtime",
|
||||
"MemoryDenyWriteExecute",
|
||||
"RuntimeDirectory",
|
||||
"StateDirectory",
|
||||
"CacheDirectory",
|
||||
"ReadOnlyPaths",
|
||||
"ReadWritePaths",
|
||||
"UMask",
|
||||
}
|
||||
),
|
||||
"Install": frozenset({"WantedBy"}),
|
||||
}
|
||||
_SYSTEMD_MULTI_DIRECTIVES = frozenset(
|
||||
{
|
||||
("Service", "Environment"),
|
||||
("Service", "ReadOnlyPaths"),
|
||||
("Service", "ReadWritePaths"),
|
||||
}
|
||||
)
|
||||
_SYSTEMD_EXACT_SINGLE_VALUES = {
|
||||
("Unit", "Description"): "AIOS Codex Review/Test Worker",
|
||||
("Unit", "After"): "network-online.target",
|
||||
("Unit", "Wants"): "network-online.target",
|
||||
("Service", "Type"): "simple",
|
||||
("Service", "User"): "aiman",
|
||||
("Service", "Group"): "gitops",
|
||||
(
|
||||
"Service",
|
||||
"WorkingDirectory",
|
||||
): "/usr/libexec/aios-codex-review-test-worker/current",
|
||||
(
|
||||
"Service",
|
||||
"ExecStart",
|
||||
): "/usr/bin/python3 -m task_plane.workers.codex_review_test",
|
||||
("Service", "Restart"): "on-failure",
|
||||
("Service", "RestartSec"): "2",
|
||||
("Service", "NoNewPrivileges"): "true",
|
||||
("Service", "PrivateTmp"): "true",
|
||||
("Service", "PrivateDevices"): "true",
|
||||
("Service", "ProtectSystem"): "strict",
|
||||
("Service", "ProtectHome"): "true",
|
||||
("Service", "ProtectKernelTunables"): "true",
|
||||
("Service", "ProtectKernelModules"): "true",
|
||||
("Service", "ProtectControlGroups"): "true",
|
||||
("Service", "LockPersonality"): "true",
|
||||
("Service", "RestrictSUIDSGID"): "true",
|
||||
("Service", "RestrictRealtime"): "true",
|
||||
("Service", "MemoryDenyWriteExecute"): "true",
|
||||
("Service", "RuntimeDirectory"): "aios-codex-review-test-worker",
|
||||
("Service", "StateDirectory"): "aios-codex-review-test-worker",
|
||||
("Service", "CacheDirectory"): "aios-codex-review-test-worker",
|
||||
("Service", "UMask"): "0077",
|
||||
("Install", "WantedBy"): "multi-user.target",
|
||||
}
|
||||
_SYSTEMD_EXACT_MULTI_VALUES = {
|
||||
("Service", "Environment"): (
|
||||
"AIOS_CODEX_WORKER_MODE=shadow",
|
||||
"AIOS_TASK_PLANE_EXECUTION_ENABLED=false",
|
||||
"PYTHONDONTWRITEBYTECODE=1",
|
||||
),
|
||||
("Service", "ReadOnlyPaths"): (
|
||||
"/data/aios-gitops/repo.git",
|
||||
"/usr/libexec/aios-codex-review-test-worker",
|
||||
),
|
||||
("Service", "ReadWritePaths"): (
|
||||
"/run/aios-codex-review-test-worker",
|
||||
"/var/lib/aios-codex-review-test-worker",
|
||||
"/var/cache/aios-codex-review-test-worker",
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
class PathPolicyError(PermissionError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class OperationPolicy:
|
||||
exact_paths: tuple[str, ...]
|
||||
max_content_bytes: int
|
||||
content_profile: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ServicePathPolicy:
|
||||
schema_version: int
|
||||
policy_id: str
|
||||
audience: str
|
||||
repository_identity: str
|
||||
allowed_branch: str
|
||||
allowed_operations: dict[str, OperationPolicy]
|
||||
sha256: str
|
||||
|
||||
def operation(self, name: str) -> OperationPolicy:
|
||||
try:
|
||||
return self.allowed_operations[name]
|
||||
except KeyError:
|
||||
raise PathPolicyError("isolated_writer_service_operation_rejected") from None
|
||||
|
||||
def require_exact_path(self, operation: str, path: str) -> OperationPolicy:
|
||||
selected = self.operation(operation)
|
||||
if path not in selected.exact_paths:
|
||||
raise PathPolicyError("isolated_writer_service_path_rejected")
|
||||
return selected
|
||||
|
||||
|
||||
def _canonical_json(value: Any) -> bytes:
|
||||
return (
|
||||
json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
|
||||
+ "\n"
|
||||
).encode("utf-8")
|
||||
|
||||
|
||||
def validate_relative_path_syntax(path: str) -> str:
|
||||
if not isinstance(path, str):
|
||||
raise PathPolicyError("isolated_writer_path_rejected")
|
||||
if not path or path != path.strip() or "\x00" in path or "\\" in path:
|
||||
raise PathPolicyError("isolated_writer_path_rejected")
|
||||
try:
|
||||
decoded = unquote(path, errors="strict")
|
||||
except (UnicodeDecodeError, ValueError):
|
||||
raise PathPolicyError("isolated_writer_path_rejected") from None
|
||||
if decoded != path:
|
||||
raise PathPolicyError("isolated_writer_path_rejected")
|
||||
posix = PurePosixPath(path)
|
||||
windows = PureWindowsPath(path)
|
||||
parts = path.split("/")
|
||||
if (
|
||||
posix.is_absolute()
|
||||
or windows.is_absolute()
|
||||
or bool(windows.drive)
|
||||
or any(part in {"", ".", ".."} for part in parts)
|
||||
):
|
||||
raise PathPolicyError("isolated_writer_path_rejected")
|
||||
return path
|
||||
|
||||
|
||||
def _require_fields(value: dict[str, Any], allowed: frozenset[str], code: str) -> None:
|
||||
if set(value) != allowed:
|
||||
raise PathPolicyError(code)
|
||||
|
||||
|
||||
def _parse_policy(document: Any, canonical: bytes) -> ServicePathPolicy:
|
||||
if not isinstance(document, dict):
|
||||
raise PathPolicyError("isolated_writer_path_policy_object_required")
|
||||
_require_fields(document, POLICY_FIELDS, "isolated_writer_path_policy_fields_rejected")
|
||||
if document["schema_version"] != 1:
|
||||
raise PathPolicyError("isolated_writer_path_policy_schema_rejected")
|
||||
text_fields = (
|
||||
"policy_id",
|
||||
"audience",
|
||||
"repository_identity",
|
||||
"allowed_branch",
|
||||
)
|
||||
if any(
|
||||
not isinstance(document[field], str) or not document[field].strip()
|
||||
for field in text_fields
|
||||
):
|
||||
raise PathPolicyError("isolated_writer_path_policy_value_rejected")
|
||||
raw_operations = document["allowed_operations"]
|
||||
if not isinstance(raw_operations, dict) or not raw_operations:
|
||||
raise PathPolicyError("isolated_writer_path_policy_operations_rejected")
|
||||
if not set(raw_operations).issubset(SUPPORTED_OPERATIONS):
|
||||
raise PathPolicyError("isolated_writer_path_policy_operations_rejected")
|
||||
operations: dict[str, OperationPolicy] = {}
|
||||
for name, raw in raw_operations.items():
|
||||
if not isinstance(raw, dict):
|
||||
raise PathPolicyError("isolated_writer_path_policy_operation_rejected")
|
||||
_require_fields(
|
||||
raw,
|
||||
OPERATION_FIELDS,
|
||||
"isolated_writer_path_policy_operation_fields_rejected",
|
||||
)
|
||||
paths = raw["exact_paths"]
|
||||
if (
|
||||
not isinstance(paths, list)
|
||||
or not paths
|
||||
or any(not isinstance(item, str) for item in paths)
|
||||
):
|
||||
raise PathPolicyError("isolated_writer_path_policy_paths_rejected")
|
||||
validated = tuple(validate_relative_path_syntax(item) for item in paths)
|
||||
if len(set(validated)) != len(validated):
|
||||
raise PathPolicyError("isolated_writer_path_policy_paths_rejected")
|
||||
maximum = raw["max_content_bytes"]
|
||||
if not isinstance(maximum, int) or isinstance(maximum, bool) or not 1 <= maximum <= 1_048_576:
|
||||
raise PathPolicyError("isolated_writer_path_policy_size_rejected")
|
||||
profile = raw["content_profile"]
|
||||
if profile not in SUPPORTED_CONTENT_PROFILES:
|
||||
raise PathPolicyError("isolated_writer_path_policy_profile_rejected")
|
||||
operations[name] = OperationPolicy(validated, maximum, profile)
|
||||
return ServicePathPolicy(
|
||||
schema_version=1,
|
||||
policy_id=document["policy_id"],
|
||||
audience=document["audience"],
|
||||
repository_identity=document["repository_identity"],
|
||||
allowed_branch=document["allowed_branch"],
|
||||
allowed_operations=operations,
|
||||
sha256=hashlib.sha256(canonical).hexdigest(),
|
||||
)
|
||||
|
||||
|
||||
def load_path_policy(
|
||||
path: Path,
|
||||
*,
|
||||
expected_owner_uid: int = 0,
|
||||
expected_group_gid: int | None = None,
|
||||
expected_parent_mode: int = 0o750,
|
||||
expected_file_mode: int = 0o640,
|
||||
) -> ServicePathPolicy:
|
||||
lexical = Path(os.path.abspath(str(path)))
|
||||
if lexical.is_symlink():
|
||||
raise PathPolicyError("isolated_writer_path_policy_symlink_rejected")
|
||||
try:
|
||||
parent_info = lexical.parent.stat(follow_symlinks=False)
|
||||
file_info = lexical.stat(follow_symlinks=False)
|
||||
except FileNotFoundError:
|
||||
raise PathPolicyError("isolated_writer_path_policy_missing") from None
|
||||
if stat.S_ISLNK(parent_info.st_mode) or not stat.S_ISDIR(parent_info.st_mode):
|
||||
raise PathPolicyError("isolated_writer_path_policy_parent_rejected")
|
||||
if stat.S_ISLNK(file_info.st_mode) or not stat.S_ISREG(file_info.st_mode):
|
||||
raise PathPolicyError("isolated_writer_path_policy_file_rejected")
|
||||
if lexical.resolve(strict=True) != lexical:
|
||||
raise PathPolicyError("isolated_writer_path_policy_symlink_rejected")
|
||||
if (
|
||||
parent_info.st_uid != expected_owner_uid
|
||||
or stat.S_IMODE(parent_info.st_mode) != expected_parent_mode
|
||||
):
|
||||
raise PathPolicyError("isolated_writer_path_policy_parent_permissions_rejected")
|
||||
if (
|
||||
file_info.st_uid != expected_owner_uid
|
||||
or stat.S_IMODE(file_info.st_mode) != expected_file_mode
|
||||
):
|
||||
raise PathPolicyError("isolated_writer_path_policy_permissions_rejected")
|
||||
if expected_group_gid is not None and (
|
||||
parent_info.st_gid != expected_group_gid or file_info.st_gid != expected_group_gid
|
||||
):
|
||||
raise PathPolicyError("isolated_writer_path_policy_group_rejected")
|
||||
if file_info.st_size <= 0 or file_info.st_size > MAX_POLICY_BYTES:
|
||||
raise PathPolicyError("isolated_writer_path_policy_file_size_rejected")
|
||||
raw = lexical.read_bytes()
|
||||
try:
|
||||
document = json.loads(raw.decode("utf-8"))
|
||||
except (UnicodeDecodeError, json.JSONDecodeError):
|
||||
raise PathPolicyError("isolated_writer_path_policy_json_rejected") from None
|
||||
canonical = _canonical_json(document)
|
||||
if raw != canonical:
|
||||
raise PathPolicyError("isolated_writer_path_policy_not_canonical")
|
||||
return _parse_policy(document, canonical)
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def configured_path_policy() -> ServicePathPolicy | None:
|
||||
configured = os.getenv("AIOS_GITOPS_PATH_POLICY_FILE", "").strip()
|
||||
if not configured:
|
||||
return None
|
||||
if not Path(configured).is_absolute():
|
||||
raise PathPolicyError("isolated_writer_path_policy_path_rejected")
|
||||
try:
|
||||
gitops_gid = grp.getgrnam("gitops").gr_gid
|
||||
except KeyError:
|
||||
raise PathPolicyError("isolated_writer_path_policy_group_rejected") from None
|
||||
loaded = load_path_policy(Path(configured), expected_group_gid=gitops_gid)
|
||||
if loaded.audience != os.getenv("AIOS_GITOPS_SERVICE_AUDIENCE", ""):
|
||||
raise PathPolicyError("isolated_writer_path_policy_audience_rejected")
|
||||
if loaded.repository_identity != os.getenv("AIOS_GITOPS_REPOSITORY_IDENTITY", ""):
|
||||
raise PathPolicyError("isolated_writer_path_policy_repository_rejected")
|
||||
if loaded.allowed_branch != os.getenv("AIOS_GITOPS_ALLOWED_BRANCH", ""):
|
||||
raise PathPolicyError("isolated_writer_path_policy_branch_rejected")
|
||||
return loaded
|
||||
|
||||
|
||||
def clear_path_policy_cache() -> None:
|
||||
configured_path_policy.cache_clear()
|
||||
|
||||
|
||||
def require_service_path(operation: str, path: str) -> OperationPolicy:
|
||||
loaded = configured_path_policy()
|
||||
if loaded is None:
|
||||
raise PathPolicyError("isolated_writer_service_path_policy_required")
|
||||
return loaded.require_exact_path(operation, path)
|
||||
|
||||
|
||||
def configured_path_policy_sha256() -> str | None:
|
||||
loaded = configured_path_policy()
|
||||
return None if loaded is None else loaded.sha256
|
||||
|
||||
|
||||
def validate_content(operation: str, path: str, content: str) -> None:
|
||||
selected = require_service_path(operation, path)
|
||||
try:
|
||||
encoded = content.encode("utf-8", errors="strict")
|
||||
except (AttributeError, UnicodeEncodeError):
|
||||
raise PathPolicyError("isolated_writer_systemd_profile_rejected") from None
|
||||
if len(encoded) > selected.max_content_bytes:
|
||||
raise PathPolicyError("isolated_writer_service_content_too_large")
|
||||
if selected.content_profile == "systemd-codex-review-worker-v1":
|
||||
_validate_systemd_codex_review_worker(content)
|
||||
return
|
||||
raise PathPolicyError("isolated_writer_path_policy_profile_rejected")
|
||||
|
||||
|
||||
def _validate_systemd_codex_review_worker(content: str) -> None:
|
||||
if not isinstance(content, str):
|
||||
raise PathPolicyError("isolated_writer_systemd_profile_rejected")
|
||||
try:
|
||||
content.encode("utf-8", errors="strict")
|
||||
except UnicodeEncodeError:
|
||||
raise PathPolicyError("isolated_writer_systemd_profile_rejected") from None
|
||||
if (
|
||||
"\x00" in content
|
||||
or "\r" in content
|
||||
or _SECRET_RE.search(content)
|
||||
or not content.endswith("\n")
|
||||
):
|
||||
raise PathPolicyError("isolated_writer_systemd_profile_rejected")
|
||||
|
||||
sections: set[str] = set()
|
||||
values: dict[tuple[str, str], list[str]] = {}
|
||||
section: str | None = None
|
||||
for line in content.splitlines():
|
||||
if len(line.encode("utf-8")) > _SYSTEMD_MAX_LINE_BYTES:
|
||||
raise PathPolicyError("isolated_writer_systemd_profile_rejected")
|
||||
if not line:
|
||||
continue
|
||||
if line[0].isspace() or line.startswith(("#", ";")):
|
||||
raise PathPolicyError("isolated_writer_systemd_profile_rejected")
|
||||
if line.startswith("[") and line.endswith("]"):
|
||||
candidate = line[1:-1]
|
||||
if (
|
||||
not _SYSTEMD_NAME_RE.fullmatch(candidate)
|
||||
or candidate not in _SYSTEMD_ALLOWED_DIRECTIVES
|
||||
or candidate in sections
|
||||
):
|
||||
raise PathPolicyError("isolated_writer_systemd_profile_rejected")
|
||||
sections.add(candidate)
|
||||
section = candidate
|
||||
continue
|
||||
if section is None or "=" not in line:
|
||||
raise PathPolicyError("isolated_writer_systemd_profile_rejected")
|
||||
directive, value = line.split("=", 1)
|
||||
if (
|
||||
not _SYSTEMD_NAME_RE.fullmatch(directive)
|
||||
or directive not in _SYSTEMD_ALLOWED_DIRECTIVES[section]
|
||||
or not value
|
||||
or value != value.strip()
|
||||
):
|
||||
raise PathPolicyError("isolated_writer_systemd_profile_rejected")
|
||||
key = (section, directive)
|
||||
if key in values and key not in _SYSTEMD_MULTI_DIRECTIVES:
|
||||
raise PathPolicyError("isolated_writer_systemd_profile_rejected")
|
||||
values.setdefault(key, []).append(value)
|
||||
|
||||
if sections != set(_SYSTEMD_ALLOWED_DIRECTIVES):
|
||||
raise PathPolicyError("isolated_writer_systemd_profile_rejected")
|
||||
if set(values) != set(_SYSTEMD_EXACT_SINGLE_VALUES) | set(
|
||||
_SYSTEMD_EXACT_MULTI_VALUES
|
||||
):
|
||||
raise PathPolicyError("isolated_writer_systemd_profile_rejected")
|
||||
for key, expected in _SYSTEMD_EXACT_SINGLE_VALUES.items():
|
||||
if values.get(key) != [expected]:
|
||||
raise PathPolicyError("isolated_writer_systemd_profile_rejected")
|
||||
for key, expected in _SYSTEMD_EXACT_MULTI_VALUES.items():
|
||||
observed = values.get(key, [])
|
||||
if len(observed) != len(set(observed)) or tuple(observed) != expected:
|
||||
raise PathPolicyError("isolated_writer_systemd_profile_rejected")
|
||||
@@ -1,241 +0,0 @@
|
||||
"""Digest-only, task-scoped token registry for isolated writers."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import os
|
||||
import grp
|
||||
import pwd
|
||||
import secrets
|
||||
import sqlite3
|
||||
import stat
|
||||
import threading
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
_REGISTRY_INIT_LOCK = threading.Lock()
|
||||
|
||||
|
||||
def utc_now() -> str:
|
||||
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
|
||||
def token_digest(token: str) -> str:
|
||||
return hashlib.sha256(token.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
class TokenRegistry:
|
||||
def __init__(self, path: Path | str):
|
||||
self.path = Path(path)
|
||||
self._prepare()
|
||||
|
||||
def _prepare(self) -> None:
|
||||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||
if self.path.parent.is_symlink() or self.path.is_symlink():
|
||||
raise PermissionError("writer_token_registry_symlink_rejected")
|
||||
with _REGISTRY_INIT_LOCK:
|
||||
with self._connect() as db:
|
||||
db.executescript(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS token_grants (
|
||||
token_id TEXT PRIMARY KEY,
|
||||
token_digest TEXT NOT NULL UNIQUE,
|
||||
grant_json TEXT NOT NULL,
|
||||
revoked INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TEXT NOT NULL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS token_uses (
|
||||
token_id TEXT NOT NULL,
|
||||
operation TEXT NOT NULL,
|
||||
use_count INTEGER NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY(token_id, operation),
|
||||
FOREIGN KEY(token_id) REFERENCES token_grants(token_id)
|
||||
);
|
||||
"""
|
||||
)
|
||||
os.chmod(self.path, 0o600)
|
||||
|
||||
def _connect(self) -> sqlite3.Connection:
|
||||
db = sqlite3.connect(self.path, timeout=30, isolation_level=None)
|
||||
db.row_factory = sqlite3.Row
|
||||
db.execute("PRAGMA journal_mode=WAL")
|
||||
db.execute("PRAGMA synchronous=FULL")
|
||||
db.execute("PRAGMA foreign_keys=ON")
|
||||
return db
|
||||
|
||||
def issue(self, grant: dict[str, Any], token: str | None = None) -> tuple[str, dict[str, Any]]:
|
||||
value = token or secrets.token_urlsafe(32)
|
||||
if len(value.encode("utf-8")) < 32:
|
||||
raise ValueError("writer_token_entropy_insufficient")
|
||||
record = dict(grant)
|
||||
record["schema_version"] = 2
|
||||
record["token_digest"] = token_digest(value)
|
||||
record["revoked"] = False
|
||||
record.setdefault("created_at", utc_now())
|
||||
required = (
|
||||
"token_id",
|
||||
"audience",
|
||||
"task_id",
|
||||
"source_cidrs",
|
||||
"allowed_branch",
|
||||
"approved_base_oid",
|
||||
"allowed_operations",
|
||||
"allowed_paths",
|
||||
"not_before",
|
||||
"expires_at",
|
||||
"max_uses",
|
||||
"issued_by",
|
||||
)
|
||||
if any(not record.get(field) for field in required):
|
||||
raise ValueError("writer_token_grant_incomplete")
|
||||
encoded = json.dumps(record, sort_keys=True, separators=(",", ":"))
|
||||
with self._connect() as db:
|
||||
db.execute("BEGIN IMMEDIATE")
|
||||
try:
|
||||
db.execute(
|
||||
"""
|
||||
INSERT INTO token_grants(token_id, token_digest, grant_json, revoked, created_at)
|
||||
VALUES(?,?,?,?,?)
|
||||
""",
|
||||
(
|
||||
str(record["token_id"]),
|
||||
str(record["token_digest"]),
|
||||
encoded,
|
||||
0,
|
||||
str(record["created_at"]),
|
||||
),
|
||||
)
|
||||
db.execute("COMMIT")
|
||||
except Exception:
|
||||
db.execute("ROLLBACK")
|
||||
raise
|
||||
return value, record
|
||||
|
||||
def find(self, token: str) -> dict[str, Any] | None:
|
||||
candidate = token_digest(token)
|
||||
with self._connect() as db:
|
||||
rows = db.execute(
|
||||
"SELECT token_digest, grant_json, revoked FROM token_grants"
|
||||
).fetchall()
|
||||
match = None
|
||||
for row in rows:
|
||||
if hmac.compare_digest(str(row["token_digest"]), candidate):
|
||||
match = dict(json.loads(str(row["grant_json"])))
|
||||
match["revoked"] = bool(row["revoked"])
|
||||
return match
|
||||
|
||||
def consume(self, token_id: str, operation: str, maximum: int) -> int:
|
||||
if maximum < 1:
|
||||
raise PermissionError("writer_token_operation_exhausted")
|
||||
with self._connect() as db:
|
||||
db.execute("BEGIN IMMEDIATE")
|
||||
try:
|
||||
row = db.execute(
|
||||
"SELECT revoked FROM token_grants WHERE token_id=?",
|
||||
(token_id,),
|
||||
).fetchone()
|
||||
if row is None or bool(row["revoked"]):
|
||||
raise PermissionError("writer_token_revoked")
|
||||
used_row = db.execute(
|
||||
"SELECT use_count FROM token_uses WHERE token_id=? AND operation=?",
|
||||
(token_id, operation),
|
||||
).fetchone()
|
||||
used = int(used_row["use_count"]) if used_row else 0
|
||||
if used >= maximum:
|
||||
raise PermissionError("writer_token_operation_exhausted")
|
||||
current = used + 1
|
||||
db.execute(
|
||||
"""
|
||||
INSERT INTO token_uses(token_id, operation, use_count)
|
||||
VALUES(?,?,?)
|
||||
ON CONFLICT(token_id, operation)
|
||||
DO UPDATE SET use_count=excluded.use_count
|
||||
""",
|
||||
(token_id, operation, current),
|
||||
)
|
||||
db.execute("COMMIT")
|
||||
return current
|
||||
except Exception:
|
||||
db.execute("ROLLBACK")
|
||||
raise
|
||||
|
||||
def revoke(self, token_id: str) -> bool:
|
||||
with self._connect() as db:
|
||||
db.execute("BEGIN IMMEDIATE")
|
||||
changed = db.execute(
|
||||
"UPDATE token_grants SET revoked=1 WHERE token_id=? AND revoked=0",
|
||||
(token_id,),
|
||||
).rowcount
|
||||
db.execute("COMMIT")
|
||||
return bool(changed)
|
||||
|
||||
def usage(self, token_id: str, operation: str) -> int:
|
||||
with self._connect() as db:
|
||||
row = db.execute(
|
||||
"SELECT use_count FROM token_uses WHERE token_id=? AND operation=?",
|
||||
(token_id, operation),
|
||||
).fetchone()
|
||||
return int(row["use_count"]) if row else 0
|
||||
|
||||
|
||||
def _write_secret_file(
|
||||
path: Path,
|
||||
token: str,
|
||||
*,
|
||||
owner: str | None = None,
|
||||
group: str | None = None,
|
||||
) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
if path.parent.is_symlink() or path.is_symlink():
|
||||
raise PermissionError("writer_token_file_symlink_rejected")
|
||||
flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL
|
||||
if hasattr(os, "O_NOFOLLOW"):
|
||||
flags |= os.O_NOFOLLOW
|
||||
fd = os.open(path, flags, 0o600)
|
||||
try:
|
||||
if not stat.S_ISREG(os.fstat(fd).st_mode):
|
||||
raise PermissionError("writer_token_file_not_regular")
|
||||
if owner is not None or group is not None:
|
||||
uid = pwd.getpwnam(owner).pw_uid if owner is not None else -1
|
||||
gid = grp.getgrnam(group).gr_gid if group is not None else -1
|
||||
os.fchown(fd, uid, gid)
|
||||
os.write(fd, token.encode("utf-8"))
|
||||
os.fsync(fd)
|
||||
finally:
|
||||
os.close(fd)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--registry", required=True)
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
issue = sub.add_parser("issue")
|
||||
issue.add_argument("--grant-file", required=True)
|
||||
issue.add_argument("--token-file", required=True)
|
||||
issue.add_argument("--owner")
|
||||
issue.add_argument("--group")
|
||||
revoke = sub.add_parser("revoke")
|
||||
revoke.add_argument("--token-id", required=True)
|
||||
args = parser.parse_args(argv)
|
||||
registry = TokenRegistry(args.registry)
|
||||
if args.command == "issue":
|
||||
grant = json.loads(Path(args.grant_file).read_text(encoding="utf-8"))
|
||||
token, record = registry.issue(grant)
|
||||
_write_secret_file(
|
||||
Path(args.token_file),
|
||||
token,
|
||||
owner=args.owner,
|
||||
group=args.group,
|
||||
)
|
||||
print(json.dumps({"token_id": record["token_id"], "created": True}, sort_keys=True))
|
||||
return 0
|
||||
changed = registry.revoke(args.token_id)
|
||||
print(json.dumps({"token_id": args.token_id, "revoked": changed}, sort_keys=True))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,564 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Execution-readiness adapters and the fail-closed Shadow controller service."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import base64
|
||||
import dataclasses
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import threading
|
||||
import time
|
||||
from http import HTTPStatus
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable, Mapping, Protocol
|
||||
from urllib.parse import parse_qs, urlparse
|
||||
|
||||
from cryptography.hazmat.primitives.asymmetric.ed25519 import (
|
||||
Ed25519PrivateKey,
|
||||
)
|
||||
|
||||
from silent_release_controller import _sanitize, utc_now
|
||||
from silent_release_hardening import (
|
||||
BuilderAttestation,
|
||||
CapabilityAuthorizationError,
|
||||
ReleaseRecoveryWorker,
|
||||
SignedApprovalVerifier,
|
||||
SignedReleasePlanApproval,
|
||||
SilentReleaseActionService,
|
||||
TrustedCandidateBundle,
|
||||
_canonical_json,
|
||||
_durable_json,
|
||||
_safe_child,
|
||||
)
|
||||
|
||||
|
||||
class BuilderSigner(Protocol):
|
||||
@property
|
||||
def key_id(self) -> str:
|
||||
...
|
||||
|
||||
def sign(self, payload: bytes) -> bytes:
|
||||
...
|
||||
|
||||
|
||||
class LocalTestBuilderSigner:
|
||||
"""Ephemeral in-memory signer. It is forbidden for production execution."""
|
||||
|
||||
test_only = True
|
||||
|
||||
def __init__(self, key_id: str, private_key: Ed25519PrivateKey):
|
||||
self._key_id = str(key_id)
|
||||
self._private_key = private_key
|
||||
|
||||
@property
|
||||
def key_id(self) -> str:
|
||||
return self._key_id
|
||||
|
||||
@classmethod
|
||||
def generate(
|
||||
cls, key_id: str = "shadow-builder-test-key"
|
||||
) -> "LocalTestBuilderSigner":
|
||||
return cls(key_id, Ed25519PrivateKey.generate())
|
||||
|
||||
@property
|
||||
def public_key(self):
|
||||
return self._private_key.public_key()
|
||||
|
||||
def sign(self, payload: bytes) -> bytes:
|
||||
return self._private_key.sign(payload)
|
||||
|
||||
|
||||
class ExternalKmsBuilderSigner:
|
||||
"""Signer facade whose private operation remains in KMS/HSM."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
key_id: str,
|
||||
credential_reference: str,
|
||||
sign_callback: Callable[[str, bytes], bytes],
|
||||
):
|
||||
if not str(credential_reference).startswith(
|
||||
("kms://", "hsm://", "secret-manager://")
|
||||
):
|
||||
raise ValueError("production builder requires a KMS/HSM reference")
|
||||
self._key_id = str(key_id)
|
||||
self.credential_reference = str(credential_reference)
|
||||
self._sign_callback = sign_callback
|
||||
|
||||
@property
|
||||
def key_id(self) -> str:
|
||||
return self._key_id
|
||||
|
||||
def sign(self, payload: bytes) -> bytes:
|
||||
return bytes(self._sign_callback(self.credential_reference, payload))
|
||||
|
||||
|
||||
class SignedTrustedBuilderAdapter:
|
||||
"""Signs evidence produced by a trusted collector, never caller evidence."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
collector: Callable[[Mapping[str, Any]], TrustedCandidateBundle],
|
||||
signer: BuilderSigner,
|
||||
):
|
||||
self.collector = collector
|
||||
self.signer = signer
|
||||
|
||||
def build(self, intent: Mapping[str, Any]) -> TrustedCandidateBundle:
|
||||
forbidden = {
|
||||
"artifact_sha256",
|
||||
"canary",
|
||||
"checks",
|
||||
"manifest_sha256",
|
||||
"tests",
|
||||
"tree_oid",
|
||||
}
|
||||
if forbidden.intersection(intent):
|
||||
raise CapabilityAuthorizationError(
|
||||
"trusted builder rejects caller-supplied evidence"
|
||||
)
|
||||
unsigned = self.collector(dict(intent))
|
||||
if unsigned.attestation is not None:
|
||||
raise CapabilityAuthorizationError(
|
||||
"collector cannot self-attest its evidence"
|
||||
)
|
||||
candidate = unsigned.candidate
|
||||
test_sha = hashlib.sha256(
|
||||
_canonical_json(_sanitize(candidate.tests))
|
||||
).hexdigest()
|
||||
canary_sha = hashlib.sha256(
|
||||
_canonical_json(_sanitize(candidate.canary))
|
||||
).hexdigest()
|
||||
attestation_id = "builder-attestation-" + hashlib.sha256(
|
||||
(
|
||||
candidate.candidate_id
|
||||
+ "\0"
|
||||
+ candidate.source_commit
|
||||
+ "\0"
|
||||
+ candidate.artifact_sha256
|
||||
).encode("utf-8")
|
||||
).hexdigest()[:24]
|
||||
attestation = BuilderAttestation(
|
||||
builder_attestation_id=attestation_id,
|
||||
builder_key_id=self.signer.key_id,
|
||||
source_commit=candidate.source_commit,
|
||||
tree_oid=candidate.tree_oid,
|
||||
artifact_sha256=candidate.artifact_sha256,
|
||||
manifest_sha256=candidate.manifest_sha256,
|
||||
package_closure_sha256=(
|
||||
unsigned.release_plan.deployment.package_closure_sha256
|
||||
),
|
||||
test_evidence_sha256=test_sha,
|
||||
canary_evidence_sha256=canary_sha,
|
||||
issued_at=utc_now(),
|
||||
signature="",
|
||||
)
|
||||
attestation = dataclasses.replace(
|
||||
attestation,
|
||||
signature=base64.b64encode(
|
||||
self.signer.sign(attestation.canonical_bytes())
|
||||
).decode("ascii"),
|
||||
)
|
||||
return dataclasses.replace(unsigned, attestation=attestation)
|
||||
|
||||
|
||||
class ApprovalBackendAdapter(Protocol):
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
...
|
||||
|
||||
def verify(
|
||||
self,
|
||||
approval: SignedReleasePlanApproval,
|
||||
*,
|
||||
candidate,
|
||||
release_plan,
|
||||
policy_sha256: str,
|
||||
) -> SignedReleasePlanApproval:
|
||||
...
|
||||
|
||||
|
||||
class LocalTestApprovalSigner:
|
||||
"""Test-only approval signer used solely for Shadow acceptance."""
|
||||
|
||||
test_only = True
|
||||
|
||||
def __init__(self, key_id: str, private_key: Ed25519PrivateKey):
|
||||
self.key_id = str(key_id)
|
||||
self.private_key = private_key
|
||||
|
||||
@classmethod
|
||||
def generate(
|
||||
cls, key_id: str = "shadow-owner-test-key"
|
||||
) -> "LocalTestApprovalSigner":
|
||||
return cls(key_id, Ed25519PrivateKey.generate())
|
||||
|
||||
def sign(
|
||||
self, approval: SignedReleasePlanApproval
|
||||
) -> SignedReleasePlanApproval:
|
||||
unsigned = dataclasses.replace(
|
||||
approval, key_id=self.key_id, signature=""
|
||||
)
|
||||
return dataclasses.replace(
|
||||
unsigned,
|
||||
signature=base64.b64encode(
|
||||
self.private_key.sign(unsigned.canonical_bytes())
|
||||
).decode("ascii"),
|
||||
)
|
||||
|
||||
|
||||
class ProductionApprovalBackendAdapter:
|
||||
"""Verification-only production adapter configured by a secret/KMS ref."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
verifier: SignedApprovalVerifier,
|
||||
credential_reference: str,
|
||||
):
|
||||
if not str(credential_reference).startswith(
|
||||
("kms://", "hsm://", "secret-manager://")
|
||||
):
|
||||
raise ValueError(
|
||||
"production approval backend requires a KMS/secret reference"
|
||||
)
|
||||
self.verifier = verifier
|
||||
self.credential_reference = str(credential_reference)
|
||||
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
return True
|
||||
|
||||
def verify(
|
||||
self,
|
||||
approval: SignedReleasePlanApproval,
|
||||
*,
|
||||
candidate,
|
||||
release_plan,
|
||||
policy_sha256: str,
|
||||
) -> SignedReleasePlanApproval:
|
||||
return self.verifier.verify_release_plan(
|
||||
approval, candidate, release_plan, policy_sha256
|
||||
)
|
||||
|
||||
|
||||
class UnavailableProductionApprovalBackend:
|
||||
credential_reference = None
|
||||
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
return False
|
||||
|
||||
def verify(self, *args, **kwargs):
|
||||
raise CapabilityAuthorizationError(
|
||||
"production approval backend is not configured"
|
||||
)
|
||||
|
||||
|
||||
class LeaderLock:
|
||||
def __init__(self, path: Path):
|
||||
self.path = Path(path)
|
||||
self.handle = None
|
||||
|
||||
def __enter__(self):
|
||||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||
os.chmod(self.path.parent, 0o700)
|
||||
self.handle = self.path.open("a+", encoding="utf-8")
|
||||
os.chmod(self.path, 0o600)
|
||||
try:
|
||||
import fcntl
|
||||
|
||||
fcntl.flock(
|
||||
self.handle.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB
|
||||
)
|
||||
except BlockingIOError as exc:
|
||||
self.handle.close()
|
||||
self.handle = None
|
||||
raise RuntimeError("silent release leader is already active") from exc
|
||||
return self
|
||||
|
||||
def __exit__(self, exc_type, exc, tb):
|
||||
if self.handle is not None:
|
||||
try:
|
||||
import fcntl
|
||||
|
||||
fcntl.flock(self.handle.fileno(), fcntl.LOCK_UN)
|
||||
finally:
|
||||
self.handle.close()
|
||||
self.handle = None
|
||||
|
||||
|
||||
class PersistentSilentReleaseController:
|
||||
"""Idempotent main-event controller with durable startup/recovery reports."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
root: Path,
|
||||
service: SilentReleaseActionService,
|
||||
*,
|
||||
main_probe: Callable[[], str],
|
||||
recovery_worker: ReleaseRecoveryWorker | None = None,
|
||||
):
|
||||
self.root = Path(root)
|
||||
self.service = service
|
||||
self.main_probe = main_probe
|
||||
self.recovery_worker = recovery_worker
|
||||
self.leader_path = self.root / "controller.leader.lock"
|
||||
self.status_path = self.root / "controller-status.json"
|
||||
|
||||
def startup_verify(self) -> dict[str, Any]:
|
||||
checked = 0
|
||||
failed = 0
|
||||
for journal in sorted(self.service.sagas.root.glob("*.events.jsonl")):
|
||||
candidate_id = journal.name[: -len(".events.jsonl")]
|
||||
saga = self.service.sagas.load(candidate_id)
|
||||
checked += 1
|
||||
if not saga.journal_integrity:
|
||||
failed += 1
|
||||
status = {
|
||||
"checked_sagas": checked,
|
||||
"execution_enabled": self.service.execution_enabled,
|
||||
"failed_closed": failed,
|
||||
"shadow_mode": not self.service.execution_enabled,
|
||||
"side_effects_executed": False,
|
||||
"status": "READY" if failed == 0 else "FAIL_CLOSED",
|
||||
}
|
||||
_durable_json(self.status_path, status)
|
||||
return status
|
||||
|
||||
def run_once(self) -> dict[str, Any]:
|
||||
with LeaderLock(self.leader_path):
|
||||
startup = self.startup_verify()
|
||||
if startup["status"] != "READY":
|
||||
result = {
|
||||
"execution_enabled": self.service.execution_enabled,
|
||||
"intent": None,
|
||||
"notification": None,
|
||||
"recovery": None,
|
||||
"shadow_mode": not self.service.execution_enabled,
|
||||
"side_effects_executed": False,
|
||||
"startup": startup,
|
||||
"status": "FAIL_CLOSED",
|
||||
}
|
||||
_durable_json(self.status_path, result)
|
||||
return _sanitize(result)
|
||||
source_commit = str(self.main_probe())
|
||||
event_id = f"main-{source_commit}"
|
||||
intent = self.service.create_release_intent(
|
||||
{"event_id": event_id, "source_commit": source_commit}
|
||||
)
|
||||
notification = None
|
||||
if intent.get("candidate_id"):
|
||||
notification = self.service.ensure_owner_notification(
|
||||
str(intent["candidate_id"])
|
||||
)
|
||||
recovery = (
|
||||
self.recovery_worker.run_once()
|
||||
if self.recovery_worker is not None
|
||||
else self.service.get_recovery_status()
|
||||
)
|
||||
result = {
|
||||
"execution_enabled": self.service.execution_enabled,
|
||||
"intent": intent,
|
||||
"notification": notification,
|
||||
"recovery": recovery,
|
||||
"shadow_mode": not self.service.execution_enabled,
|
||||
"side_effects_executed": False,
|
||||
"startup": startup,
|
||||
}
|
||||
_durable_json(self.status_path, result)
|
||||
return _sanitize(result)
|
||||
|
||||
|
||||
class ShadowActionHandler(BaseHTTPRequestHandler):
|
||||
service: SilentReleaseActionService
|
||||
controller: PersistentSilentReleaseController
|
||||
|
||||
def log_message(self, fmt, *args):
|
||||
return
|
||||
|
||||
def _json(self, payload: Mapping[str, Any], status=HTTPStatus.OK):
|
||||
body = json.dumps(_sanitize(dict(payload)), sort_keys=True).encode()
|
||||
self.send_response(int(status))
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def _body(self) -> dict[str, Any]:
|
||||
size = int(self.headers.get("Content-Length", "0"))
|
||||
if size > 1024 * 1024:
|
||||
raise ValueError("request body is too large")
|
||||
return json.loads(self.rfile.read(size) or b"{}")
|
||||
|
||||
def do_GET(self):
|
||||
parsed = urlparse(self.path)
|
||||
query = parse_qs(parsed.query)
|
||||
candidate_id = str((query.get("candidate_id") or [""])[0])
|
||||
try:
|
||||
if parsed.path == "/healthz":
|
||||
return self._json(
|
||||
{
|
||||
"execution_enabled": False,
|
||||
"shadow_mode": True,
|
||||
"side_effects_executed": False,
|
||||
"status": "ok",
|
||||
}
|
||||
)
|
||||
routes = {
|
||||
"/getReleaseCandidate": self.service.get_release_candidate,
|
||||
"/getReleasePlan": self.service.get_release_plan,
|
||||
"/getReleaseAcceptance": self.service.get_acceptance,
|
||||
"/getReleaseSaga": self.service.get_release_saga,
|
||||
"/getCapabilityStatus": self.service.get_capability_status,
|
||||
}
|
||||
if parsed.path in routes:
|
||||
return self._json(routes[parsed.path](candidate_id))
|
||||
if parsed.path == "/getRecoveryStatus":
|
||||
return self._json(self.service.get_recovery_status())
|
||||
self._json({"error": "not_found"}, HTTPStatus.NOT_FOUND)
|
||||
except (ValueError, FileNotFoundError) as exc:
|
||||
self._json(
|
||||
{"error": type(exc).__name__}, HTTPStatus.BAD_REQUEST
|
||||
)
|
||||
|
||||
def do_POST(self):
|
||||
parsed = urlparse(self.path)
|
||||
try:
|
||||
body = self._body()
|
||||
if parsed.path == "/createReleaseIntent":
|
||||
return self._json(self.service.create_release_intent(body))
|
||||
if parsed.path in {
|
||||
"/submitSignedApproval",
|
||||
"/submitOwnerApproval",
|
||||
}:
|
||||
return self._json(self.service.submit_owner_approval(body))
|
||||
if parsed.path == "/startDeployment":
|
||||
return self._json(
|
||||
self.service.start_deployment(
|
||||
str(body.get("candidate_id") or "")
|
||||
)
|
||||
)
|
||||
self._json({"error": "not_found"}, HTTPStatus.NOT_FOUND)
|
||||
except (ValueError, FileNotFoundError, CapabilityAuthorizationError) as exc:
|
||||
self._json(
|
||||
{"error": type(exc).__name__}, HTTPStatus.BAD_REQUEST
|
||||
)
|
||||
|
||||
|
||||
def git_main_probe(repository: Path) -> Callable[[], str]:
|
||||
repository = Path(repository)
|
||||
|
||||
def probe() -> str:
|
||||
completed = subprocess.run(
|
||||
[
|
||||
"git",
|
||||
f"--git-dir={repository}",
|
||||
"rev-parse",
|
||||
"refs/heads/main",
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=15,
|
||||
env={"PATH": os.environ.get("PATH", "")},
|
||||
)
|
||||
return completed.stdout.strip()
|
||||
|
||||
return probe
|
||||
|
||||
|
||||
def serve_shadow(
|
||||
*,
|
||||
state_root: Path,
|
||||
repository: Path,
|
||||
host: str,
|
||||
port: int,
|
||||
poll_seconds: float,
|
||||
) -> None:
|
||||
service = SilentReleaseActionService.shadow(state_root)
|
||||
controller = PersistentSilentReleaseController(
|
||||
state_root,
|
||||
service,
|
||||
main_probe=git_main_probe(repository),
|
||||
)
|
||||
controller.run_once()
|
||||
stop = threading.Event()
|
||||
|
||||
def controller_loop() -> None:
|
||||
while not stop.wait(poll_seconds):
|
||||
try:
|
||||
controller.run_once()
|
||||
except Exception as exc: # fail closed and keep the API available
|
||||
_durable_json(
|
||||
controller.status_path,
|
||||
{
|
||||
"error": type(exc).__name__,
|
||||
"execution_enabled": False,
|
||||
"shadow_mode": True,
|
||||
"side_effects_executed": False,
|
||||
"status": "FAIL_CLOSED",
|
||||
},
|
||||
)
|
||||
|
||||
handler = type(
|
||||
"ConfiguredShadowActionHandler",
|
||||
(ShadowActionHandler,),
|
||||
{"service": service, "controller": controller},
|
||||
)
|
||||
server = ThreadingHTTPServer((host, port), handler)
|
||||
api_thread = threading.Thread(
|
||||
target=server.serve_forever,
|
||||
name="silent-release-shadow-api",
|
||||
daemon=False,
|
||||
)
|
||||
controller_thread = threading.Thread(
|
||||
target=controller_loop,
|
||||
name="silent-release-shadow-controller",
|
||||
daemon=True,
|
||||
)
|
||||
api_thread.start()
|
||||
controller_thread.start()
|
||||
try:
|
||||
while api_thread.is_alive():
|
||||
api_thread.join(timeout=1)
|
||||
finally:
|
||||
stop.set()
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
controller_thread.join(timeout=max(1.0, poll_seconds + 1.0))
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--state-root", required=True, type=Path)
|
||||
parser.add_argument("--repository", required=True, type=Path)
|
||||
parser.add_argument("--host", default="127.0.0.1")
|
||||
parser.add_argument("--port", default=18891, type=int)
|
||||
parser.add_argument("--poll-seconds", default=30.0, type=float)
|
||||
arguments = parser.parse_args()
|
||||
if arguments.poll_seconds <= 0:
|
||||
raise SystemExit("poll seconds must be positive")
|
||||
if os.environ.get("AIOS_GITOPS_WRITES_ENABLED", "false").lower() != "false":
|
||||
raise SystemExit("GitOps writes must remain disabled")
|
||||
if (
|
||||
os.environ.get(
|
||||
"AIOS_SILENT_RELEASE_EXECUTION_ENABLED", "false"
|
||||
).lower()
|
||||
!= "false"
|
||||
):
|
||||
raise SystemExit("Silent Release execution must remain disabled")
|
||||
serve_shadow(
|
||||
state_root=arguments.state_root,
|
||||
repository=arguments.repository,
|
||||
host=arguments.host,
|
||||
port=arguments.port,
|
||||
poll_seconds=arguments.poll_seconds,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,21 +0,0 @@
|
||||
"""Task Plane runtime package."""
|
||||
|
||||
from .models import (
|
||||
EvidenceEnvelope,
|
||||
ProbeEvidence,
|
||||
ProbeOutcome,
|
||||
TaskPlan,
|
||||
TaskRecord,
|
||||
TaskStatus,
|
||||
TaskStateTransitionError,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"EvidenceEnvelope",
|
||||
"ProbeEvidence",
|
||||
"ProbeOutcome",
|
||||
"TaskPlan",
|
||||
"TaskRecord",
|
||||
"TaskStatus",
|
||||
"TaskStateTransitionError",
|
||||
]
|
||||
@@ -1,17 +0,0 @@
|
||||
"""Adapters for executing tasks via worker backends."""
|
||||
|
||||
from .base import WorkerAdapter
|
||||
from .openclaw import OpenClawAdapter
|
||||
from .codex import CodexAdapter
|
||||
from .hermes import HermesAdapter
|
||||
from .a2a import A2AAdapter
|
||||
from .systemd_worker import SystemdWorkerAdapter
|
||||
|
||||
__all__ = [
|
||||
"WorkerAdapter",
|
||||
"OpenClawAdapter",
|
||||
"CodexAdapter",
|
||||
"HermesAdapter",
|
||||
"A2AAdapter",
|
||||
"SystemdWorkerAdapter",
|
||||
]
|
||||
@@ -1,31 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from .base import WorkerAdapter
|
||||
|
||||
|
||||
class A2AAdapter(WorkerAdapter):
|
||||
worker_name = "a2a"
|
||||
|
||||
def capabilities(self) -> list[str]:
|
||||
return ["task.exchange", "agent.handoff"]
|
||||
|
||||
def claim(self, task):
|
||||
return {"worker": self.worker_name, "task_id": task.task_id, "status": "claimed"}
|
||||
|
||||
def execute(self, task, claim):
|
||||
return {"worker": self.worker_name, "task_id": task.task_id, "status": "SHADOW_VERIFIED", "claim_id": claim.claim_id, "execution_mode": "shadow", "production_side_effects": False}
|
||||
|
||||
def heartbeat(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": "alive"}
|
||||
|
||||
def report_progress(self, claim, progress):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "progress": progress}
|
||||
|
||||
def probe(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": claim.status}
|
||||
|
||||
def cancel(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": "cancelled"}
|
||||
|
||||
def collect_result(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "result": "not_collected"}
|
||||
@@ -1,41 +0,0 @@
|
||||
"""Base worker adapter contract."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import abc
|
||||
|
||||
|
||||
class WorkerAdapter(abc.ABC):
|
||||
worker_name = "base"
|
||||
|
||||
@abc.abstractmethod
|
||||
def capabilities(self) -> list[str]:
|
||||
raise NotImplementedError
|
||||
|
||||
@abc.abstractmethod
|
||||
def claim(self, task):
|
||||
raise NotImplementedError
|
||||
|
||||
@abc.abstractmethod
|
||||
def execute(self, task, claim):
|
||||
raise NotImplementedError
|
||||
|
||||
@abc.abstractmethod
|
||||
def heartbeat(self, claim):
|
||||
raise NotImplementedError
|
||||
|
||||
@abc.abstractmethod
|
||||
def report_progress(self, claim, progress):
|
||||
raise NotImplementedError
|
||||
|
||||
@abc.abstractmethod
|
||||
def probe(self, claim):
|
||||
raise NotImplementedError
|
||||
|
||||
@abc.abstractmethod
|
||||
def cancel(self, claim):
|
||||
raise NotImplementedError
|
||||
|
||||
@abc.abstractmethod
|
||||
def collect_result(self, claim):
|
||||
raise NotImplementedError
|
||||
@@ -1,31 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from .base import WorkerAdapter
|
||||
|
||||
|
||||
class CodexAdapter(WorkerAdapter):
|
||||
worker_name = "codex"
|
||||
|
||||
def capabilities(self) -> list[str]:
|
||||
return ["code.read", "code.write.patch", "test.python", "review.security"]
|
||||
|
||||
def claim(self, task):
|
||||
return {"worker": self.worker_name, "task_id": task.task_id, "status": "claimed"}
|
||||
|
||||
def execute(self, task, claim):
|
||||
return {"worker": self.worker_name, "task_id": task.task_id, "status": "SHADOW_VERIFIED", "claim_id": claim.claim_id, "execution_mode": "shadow", "production_side_effects": False}
|
||||
|
||||
def heartbeat(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": "alive"}
|
||||
|
||||
def report_progress(self, claim, progress):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "progress": progress}
|
||||
|
||||
def probe(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": claim.status}
|
||||
|
||||
def cancel(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": "cancelled"}
|
||||
|
||||
def collect_result(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "result": "not_collected"}
|
||||
@@ -1,31 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from .base import WorkerAdapter
|
||||
|
||||
|
||||
class HermesAdapter(WorkerAdapter):
|
||||
worker_name = "hermes"
|
||||
|
||||
def capabilities(self) -> list[str]:
|
||||
return ["architecture.review", "policy.review", "threat.model"]
|
||||
|
||||
def claim(self, task):
|
||||
return {"worker": self.worker_name, "task_id": task.task_id, "status": "claimed"}
|
||||
|
||||
def execute(self, task, claim):
|
||||
return {"worker": self.worker_name, "task_id": task.task_id, "status": "SHADOW_VERIFIED", "claim_id": claim.claim_id, "execution_mode": "shadow", "production_side_effects": False}
|
||||
|
||||
def heartbeat(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": "alive"}
|
||||
|
||||
def report_progress(self, claim, progress):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "progress": progress}
|
||||
|
||||
def probe(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": claim.status}
|
||||
|
||||
def cancel(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": "cancelled"}
|
||||
|
||||
def collect_result(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "result": "not_collected"}
|
||||
@@ -1,31 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from .base import WorkerAdapter
|
||||
|
||||
|
||||
class OpenClawAdapter(WorkerAdapter):
|
||||
worker_name = "openclaw"
|
||||
|
||||
def capabilities(self) -> list[str]:
|
||||
return ["code.read", "code.write.patch", "test.execute", "runtime.shadow"]
|
||||
|
||||
def claim(self, task):
|
||||
return {"worker": self.worker_name, "task_id": task.task_id, "status": "claimed"}
|
||||
|
||||
def execute(self, task, claim):
|
||||
return {"worker": self.worker_name, "task_id": task.task_id, "status": "SHADOW_VERIFIED", "claim_id": claim.claim_id, "execution_mode": "shadow", "production_side_effects": False}
|
||||
|
||||
def heartbeat(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": "alive"}
|
||||
|
||||
def report_progress(self, claim, progress):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "progress": progress}
|
||||
|
||||
def probe(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": claim.status}
|
||||
|
||||
def cancel(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": "cancelled"}
|
||||
|
||||
def collect_result(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "result": "not_collected"}
|
||||
@@ -1,31 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from .base import WorkerAdapter
|
||||
|
||||
|
||||
class SystemdWorkerAdapter(WorkerAdapter):
|
||||
worker_name = "systemd_worker"
|
||||
|
||||
def capabilities(self) -> list[str]:
|
||||
return ["systemd.exec", "runtime.shell"]
|
||||
|
||||
def claim(self, task):
|
||||
return {"worker": self.worker_name, "task_id": task.task_id, "status": "claimed"}
|
||||
|
||||
def execute(self, task, claim):
|
||||
return {"worker": self.worker_name, "task_id": task.task_id, "status": "SHADOW_VERIFIED", "claim_id": claim.claim_id, "execution_mode": "shadow", "production_side_effects": False}
|
||||
|
||||
def heartbeat(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": "alive"}
|
||||
|
||||
def report_progress(self, claim, progress):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "progress": progress}
|
||||
|
||||
def probe(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": claim.status}
|
||||
|
||||
def cancel(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "status": "cancelled"}
|
||||
|
||||
def collect_result(self, claim):
|
||||
return {"worker": self.worker_name, "claim_id": claim.claim_id, "result": "not_collected"}
|
||||
@@ -1,27 +0,0 @@
|
||||
"""Agent registry abstraction."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .errors import NotFoundError
|
||||
|
||||
|
||||
class AgentRegistry:
|
||||
def __init__(self):
|
||||
self._agents: dict[str, dict] = {}
|
||||
|
||||
def register(self, agent_id: str, metadata: dict) -> None:
|
||||
self._agents[agent_id] = {"agent_id": agent_id, **metadata}
|
||||
|
||||
def unregister(self, agent_id: str) -> None:
|
||||
if agent_id not in self._agents:
|
||||
raise NotFoundError("agent")
|
||||
del self._agents[agent_id]
|
||||
|
||||
def list(self) -> list[dict]:
|
||||
return list(self._agents.values())
|
||||
|
||||
def get(self, agent_id: str) -> dict:
|
||||
try:
|
||||
return self._agents[agent_id]
|
||||
except KeyError:
|
||||
raise NotFoundError("agent")
|
||||
@@ -1,557 +0,0 @@
|
||||
"""Task Plane HTTP API implementation (stdlib-only)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
import uuid
|
||||
import urllib.parse
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from typing import Any, Callable
|
||||
|
||||
from .delegation import DelegationEngine
|
||||
from .errors import (
|
||||
ConflictError,
|
||||
ForbiddenError,
|
||||
NotFoundError,
|
||||
PayloadTooLargeError,
|
||||
TaskPlaneHTTPError,
|
||||
UnauthorizedError,
|
||||
ValidationError,
|
||||
)
|
||||
from .evidence import EvidenceService
|
||||
from .journal import TaskJournal
|
||||
from .lease import LeaseManager
|
||||
from .models import (
|
||||
EvidenceEnvelope,
|
||||
ResultEnvelope,
|
||||
TaskPlan,
|
||||
TaskRecord,
|
||||
iso8601_now,
|
||||
contains_secret_text,
|
||||
validate_identifier,
|
||||
)
|
||||
from .policy import PolicyEngine
|
||||
from .store import TaskPlaneStore
|
||||
from .worker_registry import WorkerRegistry
|
||||
|
||||
|
||||
def _json_payload(data: dict[str, Any], status: int = 200) -> tuple[int, bytes]:
|
||||
body = json.dumps(data, ensure_ascii=False).encode("utf-8")
|
||||
return status, body
|
||||
|
||||
|
||||
def _bounded_int(value: Any, name: str, minimum: int, maximum: int) -> int:
|
||||
try:
|
||||
number = int(value)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValidationError(f"invalid {name}") from exc
|
||||
if not minimum <= number <= maximum:
|
||||
raise ValidationError(f"invalid {name}")
|
||||
return number
|
||||
|
||||
|
||||
class TaskPlaneService:
|
||||
def __init__(self, store: TaskPlaneStore, policy: PolicyEngine | None = None):
|
||||
self.store = store
|
||||
self.policy = policy or PolicyEngine(
|
||||
execution_enabled=False,
|
||||
identity_capabilities={
|
||||
"anon": {"task.creator", "task.read", "task.audit"}
|
||||
},
|
||||
)
|
||||
self.journal = TaskJournal(store)
|
||||
self.lease = LeaseManager(store)
|
||||
self.workers = WorkerRegistry(store)
|
||||
self.evidence = EvidenceService(store)
|
||||
self.delegation = DelegationEngine(store)
|
||||
self.store.probe_attestation_verifier = self.policy.probe_attestation_verifier
|
||||
|
||||
def create_task(self, body: dict[str, Any], idempotency_key: str | None = None) -> dict[str, Any]:
|
||||
if body.get("parent_task_id"):
|
||||
raise ForbiddenError("child tasks must use the delegate endpoint")
|
||||
issuer = body.get("issuer", {})
|
||||
actor = issuer.get("id", "anon") if isinstance(issuer, dict) else "anon"
|
||||
self.policy.assert_allowed("create", actor)
|
||||
try:
|
||||
plan = TaskPlan.from_dict(body)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValidationError("invalid task plan") from exc
|
||||
task = self.store.create_task_with_event(plan, idempotency_key=idempotency_key)
|
||||
return task.to_dict()
|
||||
|
||||
def list_tasks(self, actor: str | None = None) -> list[dict[str, Any]]:
|
||||
if actor is not None:
|
||||
self.policy.assert_allowed("read", actor)
|
||||
return [task.to_dict(include_task=False) for task in self.store.list_tasks()]
|
||||
|
||||
def get_task(self, task_id: str, actor: str | None = None) -> dict[str, Any]:
|
||||
if actor is not None:
|
||||
self.policy.assert_allowed("read", actor, task=self.store.get_task(task_id))
|
||||
return self.store.get_task(task_id).to_dict()
|
||||
|
||||
def get_worker(self, worker_id: str, actor: str | None = None) -> dict[str, Any]:
|
||||
if actor is not None:
|
||||
self.policy.assert_allowed("read", actor)
|
||||
return self.workers.get(worker_id).to_dict()
|
||||
|
||||
def list_workers(self, actor: str | None = None) -> list[dict[str, Any]]:
|
||||
if actor is not None:
|
||||
self.policy.assert_allowed("read", actor)
|
||||
return [w.to_dict() for w in self.workers.list()]
|
||||
|
||||
def register_worker(self, body: dict[str, Any]) -> dict[str, Any]:
|
||||
actor = str(body.get("actor_identity") or body.get("identity") or "")
|
||||
is_admin = actor in self.policy.admin_identities
|
||||
try:
|
||||
existing = self.store.get_worker(str(body.get("worker_id") or ""))
|
||||
except NotFoundError:
|
||||
existing = None
|
||||
if existing is not None and str(body.get("identity") or "") != existing.identity:
|
||||
raise ConflictError("worker identity takeover rejected")
|
||||
self.policy.assert_readonly_worker_registration(actor, body)
|
||||
self.policy.assert_allowed(
|
||||
"worker.admin" if is_admin else "worker.register", actor
|
||||
)
|
||||
try:
|
||||
worker = self.workers.register(
|
||||
body,
|
||||
actor_identity=actor,
|
||||
is_admin=is_admin,
|
||||
expected_revision=body.get("expected_revision"),
|
||||
)
|
||||
except (TypeError, ValueError, KeyError) as exc:
|
||||
raise ValidationError("invalid worker registration") from exc
|
||||
return worker.to_dict()
|
||||
|
||||
def worker_heartbeat(self, worker_id: str, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
interval = payload.get("heartbeat_interval_seconds")
|
||||
actor = str(payload.get("actor_identity") or "")
|
||||
self.policy.assert_readonly_worker_binding(actor, worker_id)
|
||||
self.policy.assert_allowed("worker.register", actor)
|
||||
return self.workers.heartbeat(
|
||||
worker_id,
|
||||
heartbeat_interval_seconds=interval,
|
||||
actor_identity=actor,
|
||||
expected_revision=payload.get("expected_revision"),
|
||||
).to_dict()
|
||||
|
||||
def disable_worker(self, worker_id: str, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
actor = str(payload.get("actor_identity") or "")
|
||||
self.policy.assert_allowed("worker.admin", actor)
|
||||
return self.store.disable_worker(
|
||||
worker_id,
|
||||
actor_identity=actor,
|
||||
expected_revision=int(payload.get("expected_revision")),
|
||||
reason=str(payload.get("reason") or "disabled by administrator"),
|
||||
).to_dict()
|
||||
|
||||
def restore_worker(self, worker_id: str, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
actor = str(payload.get("actor_identity") or "")
|
||||
self.policy.assert_allowed("worker.admin", actor)
|
||||
return self.store.restore_worker(
|
||||
worker_id,
|
||||
actor_identity=actor,
|
||||
expected_revision=int(payload.get("expected_revision")),
|
||||
).to_dict()
|
||||
|
||||
def claim(self, task_id: str, body: dict[str, Any]) -> dict[str, Any]:
|
||||
actor = body.get("actor_identity") or body.get("worker_identity") or "worker"
|
||||
self.policy.assert_allowed("claim", actor)
|
||||
worker_id = body.get("worker_id")
|
||||
if not worker_id:
|
||||
raise ValidationError("worker_id required")
|
||||
|
||||
task = self.store.get_task(task_id)
|
||||
self.policy.assert_readonly_worker_binding(actor, str(worker_id))
|
||||
self.policy.assert_allowed("claim", actor, task=task)
|
||||
lease_seconds = _bounded_int(
|
||||
body.get("lease_seconds", 30), "lease_seconds", 1, 3600
|
||||
)
|
||||
claim = self.lease.claim(
|
||||
task_id=task_id,
|
||||
worker_id=worker_id,
|
||||
worker_identity=actor,
|
||||
lease_seconds=lease_seconds,
|
||||
expected_revision=task.revision,
|
||||
operation_digest=task.task_plan.operation_digest(),
|
||||
)
|
||||
return {"claim_id": claim.claim_id, **claim.to_dict()}
|
||||
|
||||
def heartbeat(self, task_id: str, body: dict[str, Any]) -> dict[str, Any]:
|
||||
claim_id = body.get("claim_id")
|
||||
if not claim_id:
|
||||
raise ValidationError("claim_id required")
|
||||
actor = body.get("actor_identity") or body.get("worker_identity") or "worker"
|
||||
task = self.store.get_task(task_id)
|
||||
claim = self.store.get_claim(claim_id)
|
||||
self.policy.assert_readonly_worker_binding(actor, claim.worker_id)
|
||||
self.policy.assert_allowed("heartbeat", actor, task=task)
|
||||
if claim.worker_identity != actor:
|
||||
raise ForbiddenError("claim identity mismatch")
|
||||
lease_seconds = _bounded_int(
|
||||
body.get("lease_seconds", 30), "lease_seconds", 1, 3600
|
||||
)
|
||||
claim = self.lease.heartbeat(task_id, claim_id, lease_seconds)
|
||||
return {"claim_id": claim_id, "lease_until": claim.lease_until}
|
||||
|
||||
def progress(self, task_id: str, body: dict[str, Any]) -> dict[str, Any]:
|
||||
claim_id = body.get("claim_id")
|
||||
if not claim_id:
|
||||
raise ValidationError("claim_id required")
|
||||
actor = body.get("actor_identity") or body.get("worker_identity") or "worker"
|
||||
progress = body.get("progress")
|
||||
stage = body.get("stage", "RUNNING")
|
||||
message = body.get("message")
|
||||
if progress is None:
|
||||
raise ValidationError("progress required")
|
||||
task = self.store.get_task(task_id)
|
||||
claim = self.store.get_claim(claim_id)
|
||||
self.policy.assert_readonly_worker_binding(actor, claim.worker_id)
|
||||
self.policy.assert_allowed("progress", actor, task=task)
|
||||
if task.task_plan.data_classification == "credential-secret":
|
||||
if message not in {None, ""} or contains_secret_text(stage):
|
||||
raise ValidationError(
|
||||
"credential-secret progress must contain metadata only"
|
||||
)
|
||||
lease_seconds = _bounded_int(
|
||||
body.get("lease_seconds", 30), "lease_seconds", 1, 3600
|
||||
)
|
||||
claim = self.store.record_progress(
|
||||
task_id,
|
||||
claim_id,
|
||||
actor,
|
||||
progress=progress,
|
||||
stage=str(stage),
|
||||
message=None if message is None else str(message),
|
||||
lease_seconds=lease_seconds,
|
||||
)
|
||||
return {"claim_id": claim.claim_id, "status": claim.status, "progress": progress}
|
||||
|
||||
def complete(self, task_id: str, body: dict[str, Any]) -> dict[str, Any]:
|
||||
claim_id = body.get("claim_id")
|
||||
if not claim_id:
|
||||
raise ValidationError("claim_id required")
|
||||
actor = body.get("actor_identity") or body.get("worker_identity") or "worker"
|
||||
task = self.store.get_task(task_id)
|
||||
|
||||
status = body.get("status", "COMPLETED")
|
||||
outputs = body.get("outputs", [])
|
||||
evidences = body.get("evidence", [])
|
||||
if not isinstance(outputs, list):
|
||||
raise ValidationError("outputs must be list")
|
||||
if not isinstance(evidences, list):
|
||||
raise ValidationError("evidence must be list")
|
||||
if any(not isinstance(ev, dict) for ev in evidences):
|
||||
raise ValidationError("evidence items must be objects")
|
||||
claim = self.store.get_claim(claim_id)
|
||||
self.policy.assert_readonly_worker_binding(actor, claim.worker_id)
|
||||
self.policy.assert_allowed("result", actor, task=task)
|
||||
if claim.worker_identity != actor:
|
||||
raise ForbiddenError("claim identity mismatch")
|
||||
if task.task_plan.data_classification == "credential-secret":
|
||||
allowed_result_keys = {
|
||||
"kind",
|
||||
"hash",
|
||||
"sha256",
|
||||
"count",
|
||||
"status",
|
||||
"error_code",
|
||||
"signature",
|
||||
"attestation_ref",
|
||||
}
|
||||
for entry in [*outputs, *evidences]:
|
||||
if (
|
||||
not isinstance(entry, dict)
|
||||
or set(entry) - allowed_result_keys
|
||||
or contains_secret_text(entry)
|
||||
):
|
||||
raise ValidationError(
|
||||
"credential-secret result must contain hash/count/status only"
|
||||
)
|
||||
|
||||
result = ResultEnvelope(
|
||||
task_id=task_id,
|
||||
claim_id=claim_id,
|
||||
worker_identity=actor,
|
||||
status=status,
|
||||
outputs=outputs,
|
||||
evidence=evidences,
|
||||
started_at=str(body.get("started_at") or iso8601_now()),
|
||||
completed_at=str(body.get("completed_at") or iso8601_now()),
|
||||
result_digest=body.get("result_digest"),
|
||||
signature=body.get("signature"),
|
||||
attestation_ref=body.get("attestation_ref"),
|
||||
operation_digest=claim.operation_digest,
|
||||
)
|
||||
evidence_decision = self.policy.evidence_verifier.verify(task, evidences)
|
||||
if not evidence_decision.allowed:
|
||||
raise ForbiddenError(evidence_decision.reason)
|
||||
result_decision = self.policy.verify_result(result, task)
|
||||
if not result_decision.allowed:
|
||||
raise ForbiddenError(result_decision.reason)
|
||||
result.result_digest = result.computed_digest()
|
||||
previous = self.store._last_evidence_hash(task_id)
|
||||
evidence_entries = []
|
||||
for evidence in evidences:
|
||||
envelope = EvidenceEnvelope.create(
|
||||
task_id=task_id,
|
||||
claim_id=claim_id,
|
||||
kind=str(evidence.get("kind") or evidence.get("type") or "generic"),
|
||||
worker_identity=actor,
|
||||
payload=evidence,
|
||||
previous_evidence_hash=previous,
|
||||
signature=evidence.get("signature"),
|
||||
attestation_ref=evidence.get("attestation_ref"),
|
||||
)
|
||||
evidence_entries.append(envelope)
|
||||
previous = envelope.evidence_digest
|
||||
self.store.complete_claim_with_evidence_and_event(
|
||||
task_id,
|
||||
claim_id,
|
||||
status
|
||||
if status
|
||||
in {"COMPLETED", "FAILED_FINAL", "FAILED_RECOVERABLE", "CANCELLED"}
|
||||
else "COMPLETED",
|
||||
result.to_dict(),
|
||||
evidence_entries,
|
||||
)
|
||||
return result.to_dict()
|
||||
|
||||
def fail(self, task_id: str, body: dict[str, Any]) -> dict[str, Any]:
|
||||
claim_id = body.get("claim_id")
|
||||
if not claim_id:
|
||||
raise ValidationError("claim_id required")
|
||||
reason = body.get("reason", "failed")
|
||||
actor = body.get("actor_identity") or body.get("worker_identity") or "worker"
|
||||
self.policy.assert_allowed("fail", actor)
|
||||
task = self.store.get_task(task_id)
|
||||
self.policy.assert_allowed("fail", actor, task=task)
|
||||
claim = self.store.get_claim(claim_id)
|
||||
if claim.worker_identity != actor:
|
||||
raise ForbiddenError("claim identity mismatch")
|
||||
status = "FAILED_RECOVERABLE" if body.get("recoverable", False) else "FAILED_FINAL"
|
||||
self.store.fail_claim_with_event(task_id, claim_id, status, str(reason))
|
||||
return {"task_id": task_id, "status": status, "reason": reason}
|
||||
|
||||
def cancel(self, task_id: str, body: dict[str, Any]) -> dict[str, Any]:
|
||||
actor = body.get("actor_identity") or body.get("worker_identity") or "owner"
|
||||
self.policy.assert_allowed("cancel", actor)
|
||||
reason = body.get("reason", "cancelled")
|
||||
task = self.store.get_task(task_id)
|
||||
self.policy.assert_allowed("cancel", actor, task=task)
|
||||
updated = self.store.cancel_task_with_event(
|
||||
task_id, task.revision, reason=reason
|
||||
)
|
||||
return updated.to_dict()
|
||||
|
||||
def delegate(self, task_id: str, body: dict[str, Any]) -> dict[str, Any]:
|
||||
actor = body.get("actor_identity") or "owner"
|
||||
self.policy.assert_allowed("delegate", actor)
|
||||
parent = self.store.get_task(task_id)
|
||||
self.policy.assert_allowed("delegate", actor, task=parent)
|
||||
if "task" in body:
|
||||
try:
|
||||
child_plan = TaskPlan.from_dict(body["task"])
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValidationError("invalid child task plan") from exc
|
||||
else:
|
||||
plan_body = dict(body)
|
||||
plan_body.pop("actor_identity", None)
|
||||
try:
|
||||
child_plan = TaskPlan.from_dict(plan_body)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValidationError("invalid child task plan") from exc
|
||||
child_plan.parent_task_id = task_id
|
||||
child = self.delegation.delegate(task_id, child_plan, caller=actor)
|
||||
return child.to_dict()
|
||||
|
||||
def get_events(self, task_id: str, actor: str | None = None) -> list[dict[str, Any]]:
|
||||
if actor is not None:
|
||||
self.policy.assert_allowed("audit", actor, task=self.store.get_task(task_id))
|
||||
self.store.get_task(task_id)
|
||||
return self.store.list_events(task_id)
|
||||
|
||||
def get_evidence(self, task_id: str, actor: str | None = None) -> list[dict[str, Any]]:
|
||||
if actor is not None:
|
||||
self.policy.assert_allowed("audit", actor, task=self.store.get_task(task_id))
|
||||
self.store.get_task(task_id)
|
||||
return self.evidence.list(task_id)
|
||||
|
||||
def get_children(self, task_id: str, actor: str | None = None) -> list[str]:
|
||||
if actor is not None:
|
||||
self.policy.assert_allowed("read", actor, task=self.store.get_task(task_id))
|
||||
self.store.get_task(task_id)
|
||||
return self.store.list_children(task_id)
|
||||
|
||||
|
||||
class TaskPlaneRequestHandler(BaseHTTPRequestHandler):
|
||||
service: TaskPlaneService | None = None
|
||||
authenticator: Callable[[str], str | None] | None = None
|
||||
max_body_bytes = 1_000_000
|
||||
|
||||
def log_message(self, format: str, *args: Any) -> None:
|
||||
# BaseHTTPRequestHandler logs the raw request target; suppress it so
|
||||
# attacker-controlled paths or query strings cannot leak credentials.
|
||||
return
|
||||
|
||||
def _json(self) -> tuple[str | None, dict[str, Any]]:
|
||||
try:
|
||||
length = int(self.headers.get("Content-Length", "0"))
|
||||
except ValueError as exc:
|
||||
raise ValidationError("invalid content length") from exc
|
||||
if length < 0:
|
||||
raise ValidationError("invalid content length")
|
||||
if length > self.max_body_bytes:
|
||||
raise PayloadTooLargeError("payload exceeds limit")
|
||||
raw = self.rfile.read(length) if length > 0 else b""
|
||||
if not raw:
|
||||
return None, {}
|
||||
try:
|
||||
decoded = raw.decode("utf-8")
|
||||
payload = json.loads(decoded)
|
||||
if not isinstance(payload, dict):
|
||||
raise ValidationError("JSON payload must be an object")
|
||||
return decoded, payload
|
||||
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
|
||||
raise ValidationError("invalid JSON payload") from exc
|
||||
|
||||
def _response(self, status: int, payload: Any) -> None:
|
||||
encoded = json.dumps(payload, ensure_ascii=False).encode("utf-8")
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", "application/json; charset=utf-8")
|
||||
self.send_header("Content-Length", str(len(encoded)))
|
||||
self.end_headers()
|
||||
self.wfile.write(encoded)
|
||||
|
||||
def _authenticated_identity(self) -> str:
|
||||
authenticator = type(self).authenticator
|
||||
if authenticator is None:
|
||||
raise UnauthorizedError("authentication unavailable")
|
||||
header = self.headers.get("Authorization", "")
|
||||
if not header.startswith("Bearer "):
|
||||
raise UnauthorizedError("bearer authentication required")
|
||||
token = header[7:]
|
||||
if not token or any(ord(ch) < 33 for ch in token):
|
||||
raise UnauthorizedError("invalid bearer token")
|
||||
identity = authenticator(token)
|
||||
if not identity:
|
||||
raise UnauthorizedError("invalid credentials")
|
||||
return str(identity)
|
||||
|
||||
def _handle(self, handler):
|
||||
try:
|
||||
if self.service is None:
|
||||
raise TaskPlaneHTTPError("service not configured")
|
||||
result = handler()
|
||||
if isinstance(result, tuple):
|
||||
status, payload = result
|
||||
else:
|
||||
status, payload = 200, result
|
||||
self._response(status, payload)
|
||||
except TaskPlaneHTTPError as exc:
|
||||
self._response(exc.status_code, exc.to_payload())
|
||||
except Exception as exc: # defensive
|
||||
if isinstance(exc, TaskPlaneHTTPError):
|
||||
raise
|
||||
self._response(
|
||||
500,
|
||||
{"error": "internal_error", "message": "internal server error"},
|
||||
)
|
||||
|
||||
def do_GET(self): # noqa: N802
|
||||
def route():
|
||||
path = urllib.parse.urlparse(self.path).path
|
||||
parts = [p for p in path.split("/") if p]
|
||||
for part in parts[1:]:
|
||||
if part not in {"events", "evidence", "children", "heartbeat"}:
|
||||
try:
|
||||
validate_identifier(part, "resource identifier")
|
||||
except ValueError as exc:
|
||||
raise ValidationError("invalid resource identifier") from exc
|
||||
identity = self._authenticated_identity()
|
||||
|
||||
if len(parts) == 1 and parts[0] == "tasks":
|
||||
return 200, self.service.list_tasks(identity)
|
||||
if len(parts) == 2 and parts[0] == "tasks":
|
||||
task_id = parts[1]
|
||||
return 200, self.service.get_task(task_id, identity)
|
||||
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "events":
|
||||
return 200, self.service.get_events(parts[1], identity)
|
||||
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "evidence":
|
||||
return 200, self.service.get_evidence(parts[1], identity)
|
||||
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "children":
|
||||
return 200, self.service.get_children(parts[1], identity)
|
||||
if len(parts) == 1 and parts[0] == "workers":
|
||||
return 200, self.service.list_workers(identity)
|
||||
if len(parts) == 2 and parts[0] == "workers":
|
||||
return 200, self.service.get_worker(parts[1], identity)
|
||||
raise NotFoundError("unknown endpoint")
|
||||
|
||||
self._handle(route)
|
||||
|
||||
def do_POST(self): # noqa: N802
|
||||
def route():
|
||||
body_text, body = self._json()
|
||||
identity = self._authenticated_identity()
|
||||
path = urllib.parse.urlparse(self.path).path
|
||||
parts = [p for p in path.split("/") if p]
|
||||
headers = {k.lower(): v for k, v in self.headers.items()}
|
||||
|
||||
if len(parts) == 1 and parts[0] == "tasks":
|
||||
issuer = body.get("issuer")
|
||||
if issuer and (
|
||||
not isinstance(issuer, dict) or issuer.get("id") != identity
|
||||
):
|
||||
raise ForbiddenError("issuer identity mismatch")
|
||||
body["issuer"] = {"type": "authenticated", "id": identity}
|
||||
return 201, self.service.create_task(body or {}, headers.get("idempotency-key"))
|
||||
if body.get("actor_identity") not in {None, identity}:
|
||||
raise ForbiddenError("actor identity mismatch")
|
||||
body["actor_identity"] = identity
|
||||
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "claim":
|
||||
return 200, self.service.claim(parts[1], body or {})
|
||||
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "heartbeat":
|
||||
return 200, self.service.heartbeat(parts[1], body or {})
|
||||
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "progress":
|
||||
return 200, self.service.progress(parts[1], body or {})
|
||||
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "result":
|
||||
return 200, self.service.complete(parts[1], body or {})
|
||||
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "fail":
|
||||
return 200, self.service.fail(parts[1], body or {})
|
||||
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "cancel":
|
||||
return 200, self.service.cancel(parts[1], body or {})
|
||||
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "delegate":
|
||||
return 200, self.service.delegate(parts[1], body or {})
|
||||
|
||||
if len(parts) == 2 and parts[0] == "workers" and parts[1] == "register":
|
||||
if (
|
||||
body.get("identity") != identity
|
||||
and identity not in self.service.policy.admin_identities
|
||||
):
|
||||
raise ForbiddenError("worker identity mismatch")
|
||||
return 201, self.service.register_worker(body or {})
|
||||
if len(parts) == 3 and parts[0] == "workers" and parts[2] == "heartbeat":
|
||||
return 200, self.service.worker_heartbeat(parts[1], body or {})
|
||||
if len(parts) == 3 and parts[0] == "workers" and parts[2] == "disable":
|
||||
return 200, self.service.disable_worker(parts[1], body or {})
|
||||
if len(parts) == 3 and parts[0] == "workers" and parts[2] == "restore":
|
||||
return 200, self.service.restore_worker(parts[1], body or {})
|
||||
raise NotFoundError("unknown endpoint")
|
||||
|
||||
self._handle(route)
|
||||
|
||||
|
||||
def run_server(
|
||||
host: str,
|
||||
port: int,
|
||||
service: TaskPlaneService,
|
||||
*,
|
||||
authenticator: Callable[[str], str | None] | None = None,
|
||||
):
|
||||
handler = TaskPlaneRequestHandler
|
||||
handler.service = service
|
||||
handler.authenticator = authenticator
|
||||
server = ThreadingHTTPServer((host, port), handler)
|
||||
server.serve_forever()
|
||||
@@ -1,36 +0,0 @@
|
||||
"""Capability matcher helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from fnmatch import fnmatch
|
||||
|
||||
|
||||
def _normalize_capability(capability: str) -> str:
|
||||
return capability.strip().lower()
|
||||
|
||||
|
||||
def capability_matches(granted: str, required: str) -> bool:
|
||||
granted = _normalize_capability(granted)
|
||||
required = _normalize_capability(required)
|
||||
if granted == required:
|
||||
return True
|
||||
# A granted wildcard such as "test.*" should match required "test.unit"
|
||||
if "*" in granted:
|
||||
return fnmatch(required, granted)
|
||||
# A required wildcard is treated as a suffix/prefix requirement for callers.
|
||||
if "*" in required:
|
||||
return fnmatch(granted, required)
|
||||
return False
|
||||
|
||||
|
||||
def has_capabilities(granted_caps: list[str], required_caps: list[str]) -> bool:
|
||||
granted = [_normalize_capability(c) for c in granted_caps]
|
||||
for required in required_caps:
|
||||
required = _normalize_capability(required)
|
||||
if not any(capability_matches(g, required) for g in granted):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def normalize_capabilities(caps: list[str]) -> list[str]:
|
||||
return sorted({_normalize_capability(cap) for cap in caps})
|
||||
@@ -1,87 +0,0 @@
|
||||
"""Delegation guardrails."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .errors import ConflictError, ForbiddenError, NotFoundError
|
||||
from .models import Budget, TaskPlan, TaskRecord
|
||||
from .store import TaskPlaneStore
|
||||
|
||||
|
||||
class DelegationEngine:
|
||||
def __init__(self, store: TaskPlaneStore):
|
||||
self.store = store
|
||||
|
||||
def can_delegate(self, parent_id: str, child_plan: TaskPlan) -> bool:
|
||||
parent = self.store.get_task(parent_id)
|
||||
if parent.status == "COMPLETED":
|
||||
raise ConflictError("cannot delegate completed task")
|
||||
if parent.task_plan.max_delegation_depth <= 0:
|
||||
raise ConflictError("max delegation depth not configured")
|
||||
|
||||
depth = self._depth(parent_id)
|
||||
if depth >= parent.task_plan.max_delegation_depth:
|
||||
raise ConflictError("max delegation depth reached")
|
||||
|
||||
if child_plan.parent_task_id not in {None, parent_id}:
|
||||
raise ConflictError("delegation parent mismatch")
|
||||
current = parent_id
|
||||
while current:
|
||||
if child_plan.task_id and current == child_plan.task_id:
|
||||
raise ConflictError("delegation cycle")
|
||||
current = self.store.parent_of(current)
|
||||
return True
|
||||
|
||||
def delegate(self, parent_id: str, child_plan: TaskPlan, *, caller: str) -> TaskRecord:
|
||||
self.can_delegate(parent_id, child_plan)
|
||||
parent = self.store.get_task(parent_id)
|
||||
if child_plan.required_capabilities and not set(child_plan.required_capabilities).issubset(
|
||||
set(parent.task_plan.required_capabilities or [])
|
||||
):
|
||||
raise ForbiddenError("child capabilities exceed parent constraints")
|
||||
risk_rank = {"low": 1, "medium": 2, "high": 3}
|
||||
if risk_rank.get(child_plan.risk, 99) > risk_rank.get(parent.task_plan.risk, -1):
|
||||
raise ForbiddenError("child risk exceeds parent constraints")
|
||||
child_plan.parent_task_id = parent_id
|
||||
child_plan.correlation_id = parent.task_plan.correlation_id or parent.task_id
|
||||
child_plan.preferred_workers = parent.task_plan.preferred_workers
|
||||
child_plan.max_delegation_depth = min(
|
||||
child_plan.max_delegation_depth,
|
||||
parent.task_plan.max_delegation_depth,
|
||||
)
|
||||
child_plan.approval_policy = parent.task_plan.approval_policy
|
||||
|
||||
parent_budget = self.store.get_task_budget(parent_id)
|
||||
required = Budget()
|
||||
for name in ("time_seconds", "attempts", "tokens", "cost"):
|
||||
available = getattr(parent_budget, name)
|
||||
requested = getattr(child_plan.budgets, name)
|
||||
if name == "attempts" and requested is None:
|
||||
requested = 1
|
||||
if available is not None:
|
||||
requested = 0 if requested is None else requested
|
||||
if requested > available:
|
||||
raise ConflictError(f"budget exceeded for {name}")
|
||||
setattr(child_plan.budgets, name, requested)
|
||||
setattr(required, name, requested)
|
||||
|
||||
# A retried identical delegation is a read, not a second budget charge.
|
||||
if child_plan.task_id:
|
||||
try:
|
||||
existing = self.store.get_task(child_plan.task_id)
|
||||
except NotFoundError:
|
||||
existing = None
|
||||
if existing is not None:
|
||||
if (
|
||||
existing.task_plan.parent_task_id == parent_id
|
||||
and existing.task_plan.operation_digest()
|
||||
== child_plan.operation_digest()
|
||||
):
|
||||
return existing
|
||||
raise ConflictError("delegation task_id conflict")
|
||||
|
||||
return self.store.delegate_task_with_budget_and_events(
|
||||
parent_id, child_plan, caller=caller
|
||||
)
|
||||
|
||||
def _depth(self, task_id: str) -> int:
|
||||
return self.store._compute_depth(task_id)
|
||||
@@ -1,71 +0,0 @@
|
||||
"""Error types for task plane operations."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
class TaskPlaneError(Exception):
|
||||
"""Base exception for API-compatible failures."""
|
||||
|
||||
status_code = 500
|
||||
error_code = "task_plane_error"
|
||||
|
||||
def __init__(self, message: str, *, code: str | None = None, details: dict | None = None):
|
||||
super().__init__(message)
|
||||
self.message = message
|
||||
self.details = details or {}
|
||||
if code:
|
||||
self.error_code = code
|
||||
|
||||
def to_payload(self) -> dict:
|
||||
return {"error": self.error_code, "message": self.message, "details": self.details}
|
||||
|
||||
|
||||
class TaskPlaneHTTPError(TaskPlaneError):
|
||||
"""Base class with explicit HTTP status."""
|
||||
|
||||
status_code = 500
|
||||
|
||||
|
||||
class ValidationError(TaskPlaneHTTPError):
|
||||
status_code = 400
|
||||
error_code = "validation_error"
|
||||
|
||||
|
||||
class UnauthorizedError(TaskPlaneHTTPError):
|
||||
status_code = 401
|
||||
error_code = "unauthorized"
|
||||
|
||||
|
||||
class ForbiddenError(TaskPlaneHTTPError):
|
||||
status_code = 403
|
||||
error_code = "forbidden"
|
||||
|
||||
|
||||
class NotFoundError(TaskPlaneHTTPError):
|
||||
status_code = 404
|
||||
error_code = "not_found"
|
||||
|
||||
|
||||
class ConflictError(TaskPlaneHTTPError):
|
||||
status_code = 409
|
||||
error_code = "conflict"
|
||||
|
||||
|
||||
class PayloadTooLargeError(TaskPlaneHTTPError):
|
||||
status_code = 413
|
||||
error_code = "payload_too_large"
|
||||
|
||||
|
||||
class FailedDependencyError(TaskPlaneHTTPError):
|
||||
status_code = 424
|
||||
error_code = "failed_dependency"
|
||||
|
||||
|
||||
class LockedError(TaskPlaneHTTPError):
|
||||
status_code = 423
|
||||
error_code = "locked"
|
||||
|
||||
|
||||
class ServiceUnavailableError(TaskPlaneHTTPError):
|
||||
status_code = 503
|
||||
error_code = "service_unavailable"
|
||||
@@ -1,108 +0,0 @@
|
||||
"""CloudEvents-compatible event envelope and durable outbox helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from typing import Callable, Protocol
|
||||
|
||||
from .store import TaskPlaneStore
|
||||
|
||||
|
||||
@dataclass
|
||||
class CloudEvent:
|
||||
specversion: str = "1.0"
|
||||
id: str = ""
|
||||
type: str = ""
|
||||
source: str = "spiffe://aios/task-plane"
|
||||
subject: str = ""
|
||||
datacontenttype: str = "application/json"
|
||||
data: dict | None = None
|
||||
|
||||
@property
|
||||
def time(self) -> str:
|
||||
return datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
|
||||
|
||||
def asdict(self) -> dict:
|
||||
return {
|
||||
"specversion": self.specversion,
|
||||
"id": self.id,
|
||||
"type": self.type,
|
||||
"source": self.source,
|
||||
"subject": self.subject,
|
||||
"time": self.time,
|
||||
"datacontenttype": self.datacontenttype,
|
||||
"data": self.data or {},
|
||||
}
|
||||
|
||||
|
||||
NOT_DELIVERED = "NOT_DELIVERED"
|
||||
DELIVERED = "DELIVERED"
|
||||
FAILED_DELIVERY = "FAILED"
|
||||
|
||||
|
||||
class NotificationAdapter(Protocol):
|
||||
name: str
|
||||
|
||||
def deliver(self, envelope: dict) -> str | None:
|
||||
"""Return a provider receipt, or None when delivery is unconfirmed."""
|
||||
|
||||
|
||||
class CallbackNotificationAdapter:
|
||||
"""Adapter seam for webhook, NATS, GPT, dashboard, and WeChat transports."""
|
||||
|
||||
def __init__(self, name: str, callback: Callable[[dict], str | None]):
|
||||
self.name = name
|
||||
self._callback = callback
|
||||
|
||||
def deliver(self, envelope: dict) -> str | None:
|
||||
return self._callback(envelope)
|
||||
|
||||
|
||||
class WebhookAdapter(CallbackNotificationAdapter):
|
||||
def __init__(self, callback: Callable[[dict], str | None]):
|
||||
super().__init__("webhook", callback)
|
||||
|
||||
|
||||
class NatsJetStreamAdapter(CallbackNotificationAdapter):
|
||||
def __init__(self, callback: Callable[[dict], str | None]):
|
||||
super().__init__("nats-jetstream", callback)
|
||||
|
||||
|
||||
class GPTResultCallbackAdapter(CallbackNotificationAdapter):
|
||||
def __init__(self, callback: Callable[[dict], str | None]):
|
||||
super().__init__("gpt-result-callback", callback)
|
||||
|
||||
|
||||
class DashboardAdapter(CallbackNotificationAdapter):
|
||||
def __init__(self, callback: Callable[[dict], str | None]):
|
||||
super().__init__("dashboard", callback)
|
||||
|
||||
|
||||
class WeChatGatewayAdapter(CallbackNotificationAdapter):
|
||||
def __init__(self, callback: Callable[[dict], str | None]):
|
||||
super().__init__("wechat-gateway", callback)
|
||||
|
||||
|
||||
class DurableOutboxDispatcher:
|
||||
def __init__(self, store: TaskPlaneStore):
|
||||
self.store = store
|
||||
|
||||
def dispatch(self, message_id: str, adapter: NotificationAdapter) -> dict:
|
||||
row = self.store.get_outbox_message(message_id)
|
||||
import json
|
||||
|
||||
envelope = json.loads(row["payload"])
|
||||
try:
|
||||
receipt = adapter.deliver(envelope)
|
||||
except Exception:
|
||||
return self.store.record_outbox_delivery(
|
||||
message_id,
|
||||
receipt=None,
|
||||
error=f"{adapter.name} delivery failed",
|
||||
)
|
||||
return self.store.record_outbox_delivery(
|
||||
message_id,
|
||||
receipt=str(receipt) if receipt else None,
|
||||
error=None if receipt else f"{adapter.name} receipt unavailable",
|
||||
)
|
||||
@@ -1,31 +0,0 @@
|
||||
"""Evidence helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
|
||||
from .store import TaskPlaneStore
|
||||
|
||||
|
||||
def evidence_digest(payload: dict) -> str:
|
||||
body = json.dumps(payload, sort_keys=True, separators=(",", ":"))
|
||||
return hashlib.sha256(body.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
class EvidenceService:
|
||||
def __init__(self, store: TaskPlaneStore):
|
||||
self.store = store
|
||||
|
||||
def add(self, task_id: str, kind: str, payload: dict, worker_identity: str) -> str:
|
||||
return self.store.add_evidence(task_id, kind, payload, worker_identity)
|
||||
|
||||
def list(self, task_id: str) -> list[dict]:
|
||||
return self.store.list_evidence(task_id)
|
||||
|
||||
def verify(self, task_id: str, required_kinds: list[str]) -> bool:
|
||||
evidences = self.list(task_id)
|
||||
observed = {e["kind"] for e in evidences}
|
||||
return set(required_kinds).issubset(
|
||||
observed
|
||||
) and self.store.verify_evidence_integrity(task_id)
|
||||
@@ -1,22 +0,0 @@
|
||||
"""Event journaling helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .errors import ValidationError
|
||||
from .store import TaskPlaneStore
|
||||
|
||||
|
||||
class TaskJournal:
|
||||
def __init__(self, store: TaskPlaneStore):
|
||||
self.store = store
|
||||
|
||||
def append(self, task_id: str, event_type: str, data: dict, *, subject: str | None = None) -> None:
|
||||
if not event_type:
|
||||
raise ValidationError("event_type required")
|
||||
self.store.append_event(task_id, event_type, data, subject=subject)
|
||||
|
||||
def list(self, task_id: str) -> list[dict]:
|
||||
return self.store.list_events(task_id)
|
||||
|
||||
def check_chain(self, task_id: str) -> bool:
|
||||
return self.store.verify_event_chain(task_id)
|
||||
@@ -1,32 +0,0 @@
|
||||
"""Claim and lease management helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .errors import LockedError
|
||||
from .models import ProbeEvidence
|
||||
from .store import TaskPlaneStore
|
||||
|
||||
|
||||
class LeaseManager:
|
||||
def __init__(self, store: TaskPlaneStore):
|
||||
self.store = store
|
||||
|
||||
def claim(self, task_id: str, worker_id: str, worker_identity: str, lease_seconds: int, expected_revision: int, operation_digest: str):
|
||||
return self.store.claim_task(task_id, worker_id, worker_identity, lease_seconds, expected_revision, operation_digest)
|
||||
|
||||
def heartbeat(self, task_id: str, claim_id: str, lease_seconds: int):
|
||||
claim = self.store.heartbeat_claim(task_id, claim_id, lease_seconds)
|
||||
return claim
|
||||
|
||||
def probe(self, claim_id: str):
|
||||
claim = self.store.get_claim(claim_id)
|
||||
if claim.status != "ACTIVE":
|
||||
raise LockedError("claim not active")
|
||||
return claim
|
||||
|
||||
def recover(self, task_id: str, claim_id: str, probe: ProbeEvidence):
|
||||
return self.store.recover_expired_claim(
|
||||
task_id,
|
||||
claim_id,
|
||||
probe,
|
||||
)
|
||||
@@ -1,829 +0,0 @@
|
||||
"""Core data models for the task plane."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
import uuid
|
||||
import urllib.parse
|
||||
from enum import StrEnum
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any
|
||||
|
||||
|
||||
SENSITIVE_KEYS = {
|
||||
"secret",
|
||||
"password",
|
||||
"token",
|
||||
"api_key",
|
||||
"access_token",
|
||||
"refresh_token",
|
||||
"authorization",
|
||||
"private_key",
|
||||
"client_secret",
|
||||
"credential",
|
||||
}
|
||||
IDENTIFIER_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._:-]{0,127}")
|
||||
TYPE_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,127}")
|
||||
RISK_LEVELS = {"low", "medium", "high"}
|
||||
DATA_CLASSIFICATIONS = {
|
||||
"public",
|
||||
"internal",
|
||||
"confidential",
|
||||
"credential-secret",
|
||||
}
|
||||
SECRET_TEXT_PATTERNS = (
|
||||
re.compile(r"(?i)\bAuthorization\s*:\s*Bearer\s+\S+"),
|
||||
re.compile(r"(?i)\b(?:password|passwd|api[_-]?key|access[_-]?token|secret)\s*[:=]\s*\S+"),
|
||||
re.compile(r"\bsk-[A-Za-z0-9_-]{12,}\b"),
|
||||
re.compile(r"\bAKIA[0-9A-Z]{16}\b"),
|
||||
re.compile(r"-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----"),
|
||||
)
|
||||
TASK_PLAN_KEYS = {
|
||||
"schema_version",
|
||||
"task_id",
|
||||
"task_type",
|
||||
"issuer",
|
||||
"intent",
|
||||
"required_capabilities",
|
||||
"preferred_workers",
|
||||
"risk",
|
||||
"approval_policy",
|
||||
"dependencies",
|
||||
"max_delegation_depth",
|
||||
"budgets",
|
||||
"evidence_requirements",
|
||||
"parent_task_id",
|
||||
"correlation_id",
|
||||
"metadata",
|
||||
"data_classification",
|
||||
"execution_zone",
|
||||
"network_policy",
|
||||
"logging_policy",
|
||||
"allowed_input_paths",
|
||||
"allowed_output_paths",
|
||||
"output_policy",
|
||||
}
|
||||
|
||||
|
||||
def _now_ts() -> float:
|
||||
return time.time()
|
||||
|
||||
|
||||
def canonical_json(value: Any) -> str:
|
||||
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
|
||||
|
||||
|
||||
def compute_sha256_hex(value: Any) -> str:
|
||||
return hashlib.sha256(canonical_json(value).encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def contains_secret_text(value: Any) -> bool:
|
||||
if isinstance(value, str):
|
||||
return any(pattern.search(value) for pattern in SECRET_TEXT_PATTERNS)
|
||||
if isinstance(value, dict):
|
||||
return any(contains_secret_text(item) for item in value.values())
|
||||
if isinstance(value, (list, tuple)):
|
||||
return any(contains_secret_text(item) for item in value)
|
||||
return False
|
||||
|
||||
|
||||
def validate_relative_paths(values: Any, name: str) -> list[str]:
|
||||
paths = _normalize_list(values)
|
||||
for value in paths:
|
||||
decoded = urllib.parse.unquote(value)
|
||||
parts = value.replace("\\", "/").split("/")
|
||||
if (
|
||||
decoded != value
|
||||
or "\\" in value
|
||||
or value.startswith("/")
|
||||
or re.match(r"^[A-Za-z]:", value)
|
||||
or any(part in {"", ".", ".."} for part in parts)
|
||||
):
|
||||
raise ValueError(f"invalid {name}")
|
||||
return paths
|
||||
|
||||
|
||||
def validate_identifier(value: Any, name: str) -> str:
|
||||
text = str(value or "")
|
||||
if (
|
||||
not IDENTIFIER_RE.fullmatch(text)
|
||||
or urllib.parse.unquote(text) != text
|
||||
or any(ord(ch) < 32 or ord(ch) == 127 for ch in text)
|
||||
):
|
||||
raise ValueError(f"invalid {name}")
|
||||
return text
|
||||
|
||||
|
||||
def validate_type(value: Any, name: str) -> str:
|
||||
text = str(value or "")
|
||||
if not TYPE_RE.fullmatch(text):
|
||||
raise ValueError(f"invalid {name}")
|
||||
return text
|
||||
|
||||
|
||||
def redact_value(value: Any) -> Any:
|
||||
if isinstance(value, dict):
|
||||
redacted = {}
|
||||
for key, inner in value.items():
|
||||
lowered = str(key).lower()
|
||||
if lowered in SENSITIVE_KEYS or lowered.endswith(
|
||||
("_password", "_secret", "_access_token", "_refresh_token", "_private_key")
|
||||
):
|
||||
redacted[key] = "[REDACTED]"
|
||||
else:
|
||||
redacted[key] = redact_value(inner)
|
||||
return redacted
|
||||
if isinstance(value, list):
|
||||
return [redact_value(v) for v in value]
|
||||
if isinstance(value, tuple):
|
||||
return tuple(redact_value(v) for v in value)
|
||||
return value
|
||||
|
||||
|
||||
class TaskStateTransitionError(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
class TaskStatus(StrEnum):
|
||||
CREATED = "CREATED"
|
||||
QUEUED = "QUEUED"
|
||||
CLAIMED = "CLAIMED"
|
||||
RUNNING = "RUNNING"
|
||||
WAITING_CHILD = "WAITING_CHILD"
|
||||
VERIFYING = "VERIFYING"
|
||||
COMPLETED = "COMPLETED"
|
||||
FAILED_RECOVERABLE = "FAILED_RECOVERABLE"
|
||||
FAILED_FINAL = "FAILED_FINAL"
|
||||
CANCELLED = "CANCELLED"
|
||||
MANUAL_REVIEW_REQUIRED = "MANUAL_REVIEW_REQUIRED"
|
||||
|
||||
|
||||
TASK_STATES = {
|
||||
status.value for status in TaskStatus
|
||||
}
|
||||
|
||||
|
||||
TASK_NEXT_STATES = {
|
||||
"CREATED": {"QUEUED", "CLAIMED", "CANCELLED", "MANUAL_REVIEW_REQUIRED"},
|
||||
"QUEUED": {"CLAIMED", "CANCELLED", "FAILED_RECOVERABLE", "FAILED_FINAL"},
|
||||
"CLAIMED": {"RUNNING", "FAILED_RECOVERABLE", "FAILED_FINAL", "CANCELLED"},
|
||||
"RUNNING": {"VERIFYING", "WAITING_CHILD", "COMPLETED", "FAILED_RECOVERABLE", "FAILED_FINAL", "CANCELLED"},
|
||||
"WAITING_CHILD": {"RUNNING", "VERIFYING", "FAILED_RECOVERABLE", "FAILED_FINAL", "CANCELLED"},
|
||||
"VERIFYING": {"COMPLETED", "FAILED_RECOVERABLE", "FAILED_FINAL", "CANCELLED"},
|
||||
"COMPLETED": set(),
|
||||
"FAILED_RECOVERABLE": {"CLAIMED", "RUNNING", "CANCELLED"},
|
||||
"FAILED_FINAL": set(),
|
||||
"CANCELLED": set(),
|
||||
"MANUAL_REVIEW_REQUIRED": {"RUNNING", "CANCELLED"},
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class Budget:
|
||||
time_seconds: int | None = None
|
||||
attempts: int | None = None
|
||||
tokens: int | None = None
|
||||
cost: float | None = None
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: dict[str, Any] | None) -> "Budget":
|
||||
if not data:
|
||||
return cls()
|
||||
if not isinstance(data, dict) or set(data) - {
|
||||
"time_seconds",
|
||||
"attempts",
|
||||
"tokens",
|
||||
"cost",
|
||||
}:
|
||||
raise ValueError("invalid budget")
|
||||
for name in ("time_seconds", "attempts", "tokens", "cost"):
|
||||
value = data.get(name)
|
||||
if value is not None:
|
||||
try:
|
||||
if float(value) < 0:
|
||||
raise ValueError("budget values must be non-negative")
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError("invalid budget value") from exc
|
||||
return cls(
|
||||
time_seconds=_to_positive_int(data.get("time_seconds")),
|
||||
attempts=_to_positive_int(data.get("attempts")),
|
||||
tokens=_to_positive_int(data.get("tokens")),
|
||||
cost=_to_float(data.get("cost")),
|
||||
)
|
||||
|
||||
def to_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"time_seconds": self.time_seconds,
|
||||
"attempts": self.attempts,
|
||||
"tokens": self.tokens,
|
||||
"cost": self.cost,
|
||||
}
|
||||
|
||||
def merge(self, other: "Budget") -> "Budget":
|
||||
return Budget(
|
||||
time_seconds=_first_non_none(self.time_seconds, other.time_seconds),
|
||||
attempts=_first_non_none(self.attempts, other.attempts),
|
||||
tokens=_first_non_none(self.tokens, other.tokens),
|
||||
cost=_first_non_none(self.cost, other.cost),
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class TaskPlan:
|
||||
schema_version: int = 1
|
||||
task_id: str | None = None
|
||||
task_type: str = "code.change"
|
||||
issuer: dict[str, Any] = field(default_factory=dict)
|
||||
intent: str | None = None
|
||||
required_capabilities: list[str] = field(default_factory=list)
|
||||
preferred_workers: list[str] = field(default_factory=list)
|
||||
risk: str = "medium"
|
||||
approval_policy: str = "patch-only"
|
||||
dependencies: list[str] = field(default_factory=list)
|
||||
max_delegation_depth: int = 5
|
||||
budgets: Budget = field(default_factory=Budget)
|
||||
evidence_requirements: list[str] = field(default_factory=list)
|
||||
parent_task_id: str | None = None
|
||||
correlation_id: str | None = None
|
||||
metadata: dict[str, Any] = field(default_factory=dict)
|
||||
data_classification: str = "internal"
|
||||
execution_zone: str = "isolated-development"
|
||||
network_policy: str = "controlled"
|
||||
logging_policy: str = "redacted"
|
||||
allowed_input_paths: list[str] = field(default_factory=list)
|
||||
allowed_output_paths: list[str] = field(default_factory=list)
|
||||
output_policy: str = "standard"
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: dict[str, Any] | None) -> "TaskPlan":
|
||||
if not isinstance(data, dict):
|
||||
raise ValueError("task plan payload must be an object")
|
||||
unknown = set(data) - TASK_PLAN_KEYS
|
||||
if unknown:
|
||||
raise ValueError("unknown task plan fields")
|
||||
if "task_id" not in data or data["task_id"] is None:
|
||||
data = dict(data)
|
||||
data["task_id"] = f"task-{uuid.uuid4()}"
|
||||
risk = str(data.get("risk", "medium"))
|
||||
if risk not in RISK_LEVELS:
|
||||
raise ValueError("invalid risk")
|
||||
issuer = data.get("issuer", {})
|
||||
if not isinstance(issuer, dict):
|
||||
raise ValueError("issuer must be an object")
|
||||
schema_version = int(data.get("schema_version", 1))
|
||||
if schema_version != 1:
|
||||
raise ValueError("unsupported schema_version")
|
||||
max_depth = _to_positive_int(data.get("max_delegation_depth"), default=5)
|
||||
if max_depth is None or max_depth > 32:
|
||||
raise ValueError("invalid max_delegation_depth")
|
||||
classification = str(data.get("data_classification", "internal"))
|
||||
if classification not in DATA_CLASSIFICATIONS:
|
||||
raise ValueError("invalid data_classification")
|
||||
execution_zone = validate_type(
|
||||
data.get("execution_zone", "isolated-development"), "execution_zone"
|
||||
)
|
||||
network_policy = validate_type(
|
||||
data.get("network_policy", "controlled"), "network_policy"
|
||||
)
|
||||
logging_policy = validate_type(
|
||||
data.get("logging_policy", "redacted"), "logging_policy"
|
||||
)
|
||||
output_policy = validate_type(
|
||||
data.get("output_policy", "standard"), "output_policy"
|
||||
)
|
||||
if contains_secret_text(
|
||||
{
|
||||
"intent": data.get("intent"),
|
||||
"metadata": data.get("metadata", {}),
|
||||
}
|
||||
):
|
||||
raise ValueError("task content contains secret material")
|
||||
if classification == "credential-secret":
|
||||
if execution_zone != "secure-file-transformer":
|
||||
raise ValueError("credential-secret requires secure-file-transformer")
|
||||
if network_policy != "denied":
|
||||
raise ValueError("credential-secret requires network denied")
|
||||
if logging_policy != "metadata-only":
|
||||
raise ValueError("credential-secret requires metadata-only logging")
|
||||
if output_policy != "hash-count-status-only":
|
||||
raise ValueError("credential-secret requires restricted output")
|
||||
return cls(
|
||||
schema_version=schema_version,
|
||||
task_id=validate_identifier(data["task_id"], "task_id"),
|
||||
task_type=validate_type(data.get("task_type", "code.change"), "task_type"),
|
||||
issuer=issuer,
|
||||
intent=data.get("intent"),
|
||||
required_capabilities=_normalize_list(data.get("required_capabilities", [])),
|
||||
preferred_workers=_normalize_list(data.get("preferred_workers", [])),
|
||||
risk=risk,
|
||||
approval_policy=validate_type(
|
||||
data.get("approval_policy", "patch-only"), "approval_policy"
|
||||
),
|
||||
dependencies=_normalize_list(data.get("dependencies", [])),
|
||||
max_delegation_depth=max_depth,
|
||||
budgets=Budget.from_dict(data.get("budgets")),
|
||||
evidence_requirements=_normalize_list(data.get("evidence_requirements", [])),
|
||||
parent_task_id=data.get("parent_task_id"),
|
||||
correlation_id=data.get("correlation_id"),
|
||||
metadata=data.get("metadata", {}) if isinstance(data.get("metadata", {}), dict) else {},
|
||||
data_classification=classification,
|
||||
execution_zone=execution_zone,
|
||||
network_policy=network_policy,
|
||||
logging_policy=logging_policy,
|
||||
allowed_input_paths=validate_relative_paths(
|
||||
data.get("allowed_input_paths", []), "allowed_input_paths"
|
||||
),
|
||||
allowed_output_paths=validate_relative_paths(
|
||||
data.get("allowed_output_paths", []), "allowed_output_paths"
|
||||
),
|
||||
output_policy=output_policy,
|
||||
)
|
||||
|
||||
def to_dict(self, *, redact: bool = False) -> dict[str, Any]:
|
||||
payload = {
|
||||
"schema_version": self.schema_version,
|
||||
"task_id": self.task_id,
|
||||
"task_type": self.task_type,
|
||||
"issuer": self.issuer,
|
||||
"intent": self.intent,
|
||||
"required_capabilities": list(self.required_capabilities),
|
||||
"preferred_workers": list(self.preferred_workers),
|
||||
"risk": self.risk,
|
||||
"approval_policy": self.approval_policy,
|
||||
"dependencies": list(self.dependencies),
|
||||
"max_delegation_depth": self.max_delegation_depth,
|
||||
"budgets": self.budgets.to_dict(),
|
||||
"evidence_requirements": list(self.evidence_requirements),
|
||||
"parent_task_id": self.parent_task_id,
|
||||
"correlation_id": self.correlation_id,
|
||||
"metadata": dict(self.metadata),
|
||||
"data_classification": self.data_classification,
|
||||
"execution_zone": self.execution_zone,
|
||||
"network_policy": self.network_policy,
|
||||
"logging_policy": self.logging_policy,
|
||||
"allowed_input_paths": list(self.allowed_input_paths),
|
||||
"allowed_output_paths": list(self.allowed_output_paths),
|
||||
"output_policy": self.output_policy,
|
||||
}
|
||||
if redact:
|
||||
payload = redact_value(payload)
|
||||
return payload
|
||||
|
||||
def operation_digest(self) -> str:
|
||||
return compute_sha256_hex(self.to_dict())
|
||||
|
||||
|
||||
@dataclass
|
||||
class TaskRecord:
|
||||
task_id: str
|
||||
status: str
|
||||
revision: int
|
||||
task_plan: TaskPlan
|
||||
created_at: float
|
||||
updated_at: float
|
||||
root_task_id: str
|
||||
idempotency_key: str | None = None
|
||||
failure_reason: str | None = None
|
||||
|
||||
def with_status(self, status: str) -> "TaskRecord":
|
||||
allowed = TASK_NEXT_STATES.get(self.status, set())
|
||||
if status != self.status and (not allowed and self.status not in {"COMPLETED", "FAILED_FINAL", "CANCELLED"}):
|
||||
raise TaskStateTransitionError(f"Cannot transition {self.status} -> {status}")
|
||||
if status not in TASK_STATES:
|
||||
raise TaskStateTransitionError(f"Invalid status {status}")
|
||||
if status != self.status and self.status in TASK_NEXT_STATES and status not in TASK_NEXT_STATES[self.status]:
|
||||
raise TaskStateTransitionError(f"Cannot transition {self.status} -> {status}")
|
||||
return TaskRecord(
|
||||
task_id=self.task_id,
|
||||
status=status,
|
||||
revision=self.revision + 1,
|
||||
task_plan=self.task_plan,
|
||||
created_at=self.created_at,
|
||||
updated_at=_now_ts(),
|
||||
root_task_id=self.root_task_id,
|
||||
idempotency_key=self.idempotency_key,
|
||||
failure_reason=self.failure_reason,
|
||||
)
|
||||
|
||||
def to_dict(self, *, include_task=True, redact_payload: bool = True) -> dict[str, Any]:
|
||||
payload = {
|
||||
"task_id": self.task_id,
|
||||
"status": self.status,
|
||||
"revision": self.revision,
|
||||
"root_task_id": self.root_task_id,
|
||||
"parent_task_id": self.task_plan.parent_task_id,
|
||||
"created_at": _fmt_ts(self.created_at),
|
||||
"updated_at": _fmt_ts(self.updated_at),
|
||||
"failure_reason": self.failure_reason,
|
||||
}
|
||||
if include_task:
|
||||
payload["plan"] = self.task_plan.to_dict(redact=redact_payload)
|
||||
if self.idempotency_key:
|
||||
payload["idempotency_key"] = self.idempotency_key
|
||||
return payload
|
||||
|
||||
|
||||
@dataclass
|
||||
class WorkerRegistration:
|
||||
worker_id: str
|
||||
worker_type: str
|
||||
identity: str
|
||||
capabilities: list[str]
|
||||
trust_zone: str
|
||||
max_concurrency: int
|
||||
heartbeat_interval_seconds: int
|
||||
supported_protocols: list[str]
|
||||
registered_at: float
|
||||
last_heartbeat: float
|
||||
status: str = "active"
|
||||
disabled: bool = False
|
||||
owner_identity: str = ""
|
||||
registration_revision: int = 1
|
||||
registration_policy: str = "self"
|
||||
created_at: float = 0.0
|
||||
updated_at: float = 0.0
|
||||
disabled_reason: str | None = None
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: dict[str, Any]) -> "WorkerRegistration":
|
||||
if not isinstance(data, dict):
|
||||
raise ValueError("worker registration payload must be an object")
|
||||
now = _now_ts()
|
||||
worker_id = validate_identifier(data.get("worker_id"), "worker_id")
|
||||
worker_type = validate_type(data.get("worker_type"), "worker_type")
|
||||
identity = str(data.get("identity") or "").strip()
|
||||
if not identity or any(ord(ch) < 32 for ch in identity):
|
||||
raise ValueError("invalid identity")
|
||||
max_concurrency = _to_positive_int(data.get("max_concurrency"), default=1)
|
||||
heartbeat_interval = _to_positive_int(
|
||||
data.get("heartbeat_interval_seconds"), default=15
|
||||
)
|
||||
if not max_concurrency:
|
||||
raise ValueError("max_concurrency must be positive")
|
||||
if not heartbeat_interval:
|
||||
raise ValueError("heartbeat_interval_seconds must be positive")
|
||||
return cls(
|
||||
worker_id=worker_id,
|
||||
worker_type=worker_type,
|
||||
identity=identity,
|
||||
capabilities=_normalize_list(data.get("capabilities", [])),
|
||||
trust_zone=str(data.get("trust_zone") or "default"),
|
||||
max_concurrency=max_concurrency,
|
||||
heartbeat_interval_seconds=heartbeat_interval,
|
||||
supported_protocols=_normalize_list(
|
||||
data.get("supported_protocols", ["aios-task-v1"])
|
||||
),
|
||||
registered_at=float(data.get("registered_at") or now),
|
||||
last_heartbeat=float(data.get("last_heartbeat") or now),
|
||||
status=str(data.get("status") or "active"),
|
||||
disabled=bool(data.get("disabled", False)),
|
||||
owner_identity=str(data.get("owner_identity") or identity),
|
||||
registration_revision=int(data.get("registration_revision") or 1),
|
||||
registration_policy=str(data.get("registration_policy") or "self"),
|
||||
created_at=float(data.get("created_at") or data.get("registered_at") or now),
|
||||
updated_at=float(data.get("updated_at") or now),
|
||||
disabled_reason=data.get("disabled_reason"),
|
||||
)
|
||||
|
||||
def to_dict(self, redact_payload: bool = True) -> dict[str, Any]:
|
||||
payload = {
|
||||
"worker_id": self.worker_id,
|
||||
"worker_type": self.worker_type,
|
||||
"identity": self.identity,
|
||||
"capabilities": list(self.capabilities),
|
||||
"trust_zone": self.trust_zone,
|
||||
"max_concurrency": self.max_concurrency,
|
||||
"heartbeat_interval_seconds": self.heartbeat_interval_seconds,
|
||||
"supported_protocols": list(self.supported_protocols),
|
||||
"registered_at": _fmt_ts(self.registered_at),
|
||||
"last_heartbeat": _fmt_ts(self.last_heartbeat),
|
||||
"status": self.status,
|
||||
"disabled": self.disabled,
|
||||
"owner_identity": self.owner_identity,
|
||||
"registration_revision": self.registration_revision,
|
||||
"registration_policy": self.registration_policy,
|
||||
"created_at": _fmt_ts(self.created_at or self.registered_at),
|
||||
"updated_at": _fmt_ts(self.updated_at or self.last_heartbeat),
|
||||
"disabled_reason": self.disabled_reason,
|
||||
}
|
||||
if redact_payload:
|
||||
payload = redact_value(payload)
|
||||
return payload
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class EvidenceEnvelope:
|
||||
schema_version: int
|
||||
evidence_id: str
|
||||
task_id: str
|
||||
claim_id: str
|
||||
kind: str
|
||||
worker_identity: str
|
||||
payload: dict[str, Any]
|
||||
created_at: float
|
||||
previous_evidence_hash: str
|
||||
evidence_digest: str
|
||||
signature: str | None = None
|
||||
attestation_ref: str | None = None
|
||||
|
||||
@classmethod
|
||||
def create(
|
||||
cls,
|
||||
*,
|
||||
task_id: str,
|
||||
claim_id: str,
|
||||
kind: str,
|
||||
worker_identity: str,
|
||||
payload: dict[str, Any],
|
||||
previous_evidence_hash: str,
|
||||
signature: str | None = None,
|
||||
attestation_ref: str | None = None,
|
||||
evidence_id: str | None = None,
|
||||
created_at: float | None = None,
|
||||
) -> "EvidenceEnvelope":
|
||||
envelope = cls(
|
||||
schema_version=1,
|
||||
evidence_id=evidence_id or f"evd-{uuid.uuid4()}",
|
||||
task_id=validate_identifier(task_id, "task_id"),
|
||||
claim_id=validate_identifier(claim_id, "claim_id"),
|
||||
kind=validate_type(kind, "evidence kind"),
|
||||
worker_identity=str(worker_identity),
|
||||
payload=redact_value(payload),
|
||||
created_at=float(created_at or _now_ts()),
|
||||
previous_evidence_hash=str(previous_evidence_hash or ""),
|
||||
evidence_digest="",
|
||||
signature=signature,
|
||||
attestation_ref=attestation_ref,
|
||||
)
|
||||
return dataclass_replace(
|
||||
envelope, evidence_digest=compute_sha256_hex(envelope.digest_payload())
|
||||
)
|
||||
|
||||
def digest_payload(self) -> dict[str, Any]:
|
||||
return {
|
||||
"schema_version": self.schema_version,
|
||||
"evidence_id": self.evidence_id,
|
||||
"task_id": self.task_id,
|
||||
"claim_id": self.claim_id,
|
||||
"kind": self.kind,
|
||||
"worker_identity": self.worker_identity,
|
||||
"payload": self.payload,
|
||||
"created_at": self.created_at,
|
||||
"previous_evidence_hash": self.previous_evidence_hash,
|
||||
"attestation_ref": self.attestation_ref,
|
||||
}
|
||||
|
||||
def verify(
|
||||
self,
|
||||
*,
|
||||
expected_task_id: str,
|
||||
expected_claim_id: str,
|
||||
expected_previous_hash: str,
|
||||
attestation_verifier: Any = None,
|
||||
) -> bool:
|
||||
if (
|
||||
self.schema_version != 1
|
||||
or self.task_id != expected_task_id
|
||||
or self.claim_id != expected_claim_id
|
||||
or self.previous_evidence_hash != expected_previous_hash
|
||||
or self.evidence_digest != compute_sha256_hex(self.digest_payload())
|
||||
):
|
||||
return False
|
||||
if self.signature or self.attestation_ref:
|
||||
return True if attestation_verifier is None else bool(attestation_verifier(self))
|
||||
return True
|
||||
|
||||
def to_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
**self.digest_payload(),
|
||||
"evidence_digest": self.evidence_digest,
|
||||
"signature": self.signature,
|
||||
}
|
||||
|
||||
|
||||
class ProbeOutcome(StrEnum):
|
||||
NOT_APPLIED = "NOT_APPLIED"
|
||||
APPLIED_VERIFIED = "APPLIED_VERIFIED"
|
||||
INDETERMINATE = "INDETERMINATE"
|
||||
CONFLICT = "CONFLICT"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ProbeEvidence:
|
||||
schema_version: int
|
||||
probe_id: str
|
||||
task_id: str
|
||||
claim_id: str
|
||||
worker_id: str
|
||||
worker_identity: str
|
||||
operation_digest: str
|
||||
outcome: ProbeOutcome
|
||||
observed_at: float
|
||||
expires_at: float
|
||||
probe_executor_identity: str
|
||||
evidence_digest: str
|
||||
signature: str | None = None
|
||||
attestation_ref: str | None = None
|
||||
|
||||
@classmethod
|
||||
def create(cls, **data: Any) -> "ProbeEvidence":
|
||||
outcome = data["outcome"]
|
||||
if not isinstance(outcome, ProbeOutcome):
|
||||
outcome = ProbeOutcome(str(outcome))
|
||||
probe = cls(
|
||||
schema_version=1,
|
||||
probe_id=data.get("probe_id") or f"probe-{uuid.uuid4()}",
|
||||
task_id=validate_identifier(data["task_id"], "task_id"),
|
||||
claim_id=validate_identifier(data["claim_id"], "claim_id"),
|
||||
worker_id=validate_identifier(data["worker_id"], "worker_id"),
|
||||
worker_identity=str(data["worker_identity"]),
|
||||
operation_digest=str(data["operation_digest"]),
|
||||
outcome=outcome,
|
||||
observed_at=float(data["observed_at"]),
|
||||
expires_at=float(data["expires_at"]),
|
||||
probe_executor_identity=str(data["probe_executor_identity"]),
|
||||
evidence_digest="",
|
||||
signature=data.get("signature"),
|
||||
attestation_ref=data.get("attestation_ref"),
|
||||
)
|
||||
return dataclass_replace(
|
||||
probe, evidence_digest=compute_sha256_hex(probe.digest_payload())
|
||||
)
|
||||
|
||||
def digest_payload(self) -> dict[str, Any]:
|
||||
return {
|
||||
"schema_version": self.schema_version,
|
||||
"probe_id": self.probe_id,
|
||||
"task_id": self.task_id,
|
||||
"claim_id": self.claim_id,
|
||||
"worker_id": self.worker_id,
|
||||
"worker_identity": self.worker_identity,
|
||||
"operation_digest": self.operation_digest,
|
||||
"outcome": self.outcome.value,
|
||||
"observed_at": self.observed_at,
|
||||
"expires_at": self.expires_at,
|
||||
"probe_executor_identity": self.probe_executor_identity,
|
||||
"attestation_ref": self.attestation_ref,
|
||||
}
|
||||
|
||||
def verify(self, *, now: float, attestation_verifier: Any) -> bool:
|
||||
if (
|
||||
self.schema_version != 1
|
||||
or self.expires_at <= now
|
||||
or self.observed_at > now
|
||||
or self.evidence_digest != compute_sha256_hex(self.digest_payload())
|
||||
or not (self.signature or self.attestation_ref)
|
||||
):
|
||||
return False
|
||||
return bool(attestation_verifier and attestation_verifier(self))
|
||||
|
||||
def to_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
**self.digest_payload(),
|
||||
"evidence_digest": self.evidence_digest,
|
||||
"signature": self.signature,
|
||||
}
|
||||
|
||||
|
||||
def dataclass_replace(value: Any, **changes: Any) -> Any:
|
||||
from dataclasses import replace
|
||||
|
||||
return replace(value, **changes)
|
||||
|
||||
|
||||
@dataclass
|
||||
class WorkerClaim:
|
||||
claim_id: str
|
||||
task_id: str
|
||||
worker_id: str
|
||||
worker_identity: str
|
||||
claimed_at: float
|
||||
lease_until: float
|
||||
attempt: int
|
||||
task_revision: int
|
||||
operation_digest: str
|
||||
status: str = "ACTIVE"
|
||||
|
||||
def to_dict(self, redact_payload: bool = True) -> dict[str, Any]:
|
||||
payload = {
|
||||
"claim_id": self.claim_id,
|
||||
"task_id": self.task_id,
|
||||
"worker_id": self.worker_id,
|
||||
"worker_identity": self.worker_identity,
|
||||
"claimed_at": _fmt_ts(self.claimed_at),
|
||||
"lease_until": _fmt_ts(self.lease_until),
|
||||
"attempt": self.attempt,
|
||||
"task_revision": self.task_revision,
|
||||
"operation_digest": self.operation_digest,
|
||||
"status": self.status,
|
||||
}
|
||||
if redact_payload:
|
||||
payload = redact_value(payload)
|
||||
return payload
|
||||
|
||||
|
||||
@dataclass
|
||||
class ResultEnvelope:
|
||||
task_id: str
|
||||
claim_id: str
|
||||
worker_identity: str
|
||||
status: str
|
||||
outputs: list[dict[str, Any]]
|
||||
evidence: list[dict[str, Any]]
|
||||
started_at: str
|
||||
completed_at: str
|
||||
result_digest: str | None = None
|
||||
signature: str | None = None
|
||||
attestation_ref: str | None = None
|
||||
operation_digest: str | None = None
|
||||
trust_status: str = "untrusted_shadow_result"
|
||||
|
||||
def digest_payload(self) -> dict[str, Any]:
|
||||
return {
|
||||
"task_id": self.task_id,
|
||||
"claim_id": self.claim_id,
|
||||
"worker_identity": self.worker_identity,
|
||||
"status": self.status,
|
||||
"outputs": self.outputs,
|
||||
"evidence": self.evidence,
|
||||
"started_at": self.started_at,
|
||||
"completed_at": self.completed_at,
|
||||
"operation_digest": self.operation_digest,
|
||||
"attestation_ref": self.attestation_ref,
|
||||
}
|
||||
|
||||
def computed_digest(self) -> str:
|
||||
return compute_sha256_hex(self.digest_payload())
|
||||
|
||||
def to_dict(self, redact_payload: bool = True) -> dict[str, Any]:
|
||||
payload = {
|
||||
"task_id": self.task_id,
|
||||
"claim_id": self.claim_id,
|
||||
"worker_identity": self.worker_identity,
|
||||
"status": self.status,
|
||||
"outputs": self.outputs,
|
||||
"evidence": self.evidence,
|
||||
"started_at": self.started_at,
|
||||
"completed_at": self.completed_at,
|
||||
"result_digest": self.result_digest,
|
||||
"signature": self.signature,
|
||||
"attestation_ref": self.attestation_ref,
|
||||
"operation_digest": self.operation_digest,
|
||||
"trust_status": self.trust_status,
|
||||
}
|
||||
if redact_payload:
|
||||
payload = redact_value(payload)
|
||||
return payload
|
||||
|
||||
|
||||
def _normalize_list(value: Any) -> list[str]:
|
||||
if not isinstance(value, list):
|
||||
raise ValueError("expected list")
|
||||
if len(value) > 128 or any(
|
||||
not isinstance(v, str)
|
||||
or not v
|
||||
or len(v) > 256
|
||||
or any(ord(ch) < 32 for ch in v)
|
||||
for v in value
|
||||
):
|
||||
raise ValueError("invalid list item")
|
||||
return list(value)
|
||||
|
||||
|
||||
def _to_positive_int(value: Any, default: int | None = None) -> int | None:
|
||||
if value is None:
|
||||
return default
|
||||
try:
|
||||
number = int(value)
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
return number if number >= 0 else default
|
||||
|
||||
|
||||
def _to_float(value: Any) -> float | None:
|
||||
if value is None:
|
||||
return None
|
||||
try:
|
||||
return float(value)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _first_non_none(primary: Any, fallback: Any) -> Any:
|
||||
if primary is not None:
|
||||
return primary
|
||||
return fallback
|
||||
|
||||
|
||||
def iso8601_now() -> str:
|
||||
return datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
|
||||
|
||||
|
||||
def _fmt_ts(value: float | None) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
return datetime.fromtimestamp(value, tz=timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
|
||||
@@ -1,226 +0,0 @@
|
||||
"""Policy and verification logic (fail-closed)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Callable
|
||||
|
||||
from .errors import ForbiddenError, LockedError
|
||||
from .models import ProbeEvidence, ResultEnvelope, TaskRecord
|
||||
|
||||
|
||||
RISK_RANK = {"low": 1, "medium": 2, "high": 3}
|
||||
READONLY_WORKER_ACTIONS = frozenset(
|
||||
{"claim", "result", "fail", "progress", "heartbeat"}
|
||||
)
|
||||
ALL_ACTION_CAPABILITIES = frozenset(
|
||||
{
|
||||
"task.creator",
|
||||
"worker.self_register",
|
||||
"worker.admin",
|
||||
"task.claim",
|
||||
"task.cancel",
|
||||
"task.delegate",
|
||||
"task.read",
|
||||
"task.audit",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class PolicyDecision:
|
||||
allowed: bool
|
||||
reason: str = ""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ReadonlyWorkerProfile:
|
||||
"""A narrowly-scoped identity allowed to process safe Shadow tasks only."""
|
||||
|
||||
worker_id: str
|
||||
identity: str
|
||||
capabilities: frozenset[str]
|
||||
task_types: frozenset[str] = frozenset({"code.review", "test.execute"})
|
||||
|
||||
def permits_task(self, task: TaskRecord) -> bool:
|
||||
plan = task.task_plan
|
||||
if plan.task_type not in self.task_types or plan.risk != "low":
|
||||
return False
|
||||
if plan.data_classification not in {"public", "internal"}:
|
||||
return False
|
||||
if plan.metadata.get("production_side_effects") is not False:
|
||||
return False
|
||||
return set(plan.required_capabilities).issubset(self.capabilities)
|
||||
|
||||
|
||||
class IdentityVerifier:
|
||||
def __init__(self, trusted_identities: set[str] | None = None):
|
||||
self.trusted_identities = trusted_identities
|
||||
|
||||
def verify(self, identity: str) -> bool:
|
||||
if self.trusted_identities is None:
|
||||
return True
|
||||
return identity in self.trusted_identities
|
||||
|
||||
|
||||
class EvidenceVerifier:
|
||||
def __init__(self, mandatory: list[str] | None = None):
|
||||
self.mandatory = set(mandatory or [])
|
||||
|
||||
def verify(self, task: TaskRecord, evidences: list[dict]) -> PolicyDecision:
|
||||
required = set(task.task_plan.evidence_requirements or [])
|
||||
if not required and not self.mandatory:
|
||||
return PolicyDecision(True)
|
||||
observed = {e.get("kind") for e in evidences}
|
||||
missing = required.union(self.mandatory) - observed
|
||||
if missing:
|
||||
return PolicyDecision(False, f"missing evidence: {', '.join(sorted(missing))}")
|
||||
return PolicyDecision(True)
|
||||
|
||||
|
||||
class PolicyEngine:
|
||||
"""
|
||||
Fail-closed policy engine with minimal but explicit decisions.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
identity_verifier: IdentityVerifier | None = None,
|
||||
evidence_verifier: EvidenceVerifier | None = None,
|
||||
execution_enabled: bool = False,
|
||||
max_risk: str = "medium",
|
||||
result_signature_verifier: Callable[[ResultEnvelope], bool] | None = None,
|
||||
probe_attestation_verifier: Callable[[ProbeEvidence], bool] | None = None,
|
||||
identity_capabilities: dict[str, set[str]] | None = None,
|
||||
admin_identities: set[str] | None = None,
|
||||
readonly_worker_execution_enabled: bool = False,
|
||||
readonly_worker_profiles: list[ReadonlyWorkerProfile] | None = None,
|
||||
) -> None:
|
||||
if max_risk not in RISK_RANK:
|
||||
raise ValueError("invalid max_risk")
|
||||
self.identity_verifier = identity_verifier or IdentityVerifier()
|
||||
self.evidence_verifier = evidence_verifier or EvidenceVerifier()
|
||||
self.execution_enabled = execution_enabled
|
||||
self.max_risk = max_risk
|
||||
self.result_signature_verifier = result_signature_verifier
|
||||
self.probe_attestation_verifier = probe_attestation_verifier
|
||||
self.identity_capabilities = {
|
||||
identity: set(capabilities)
|
||||
for identity, capabilities in (identity_capabilities or {}).items()
|
||||
}
|
||||
self.admin_identities = set(admin_identities or ())
|
||||
self.readonly_worker_execution_enabled = readonly_worker_execution_enabled
|
||||
self.readonly_worker_profiles = {
|
||||
profile.identity: profile for profile in (readonly_worker_profiles or [])
|
||||
}
|
||||
|
||||
def _readonly_worker_allows(
|
||||
self, action: str, actor: str, task: TaskRecord | None
|
||||
) -> bool:
|
||||
profile = self.readonly_worker_profiles.get(actor)
|
||||
return bool(
|
||||
self.readonly_worker_execution_enabled
|
||||
and action in READONLY_WORKER_ACTIONS
|
||||
and profile is not None
|
||||
and (task is None or profile.permits_task(task))
|
||||
)
|
||||
|
||||
def assert_readonly_worker_binding(self, actor: str, worker_id: str) -> None:
|
||||
profile = self.readonly_worker_profiles.get(actor)
|
||||
if profile is not None and profile.worker_id != worker_id:
|
||||
raise ForbiddenError("worker identity binding mismatch")
|
||||
|
||||
def assert_readonly_worker_registration(self, actor: str, payload: dict) -> None:
|
||||
profile = self.readonly_worker_profiles.get(actor)
|
||||
if profile is None:
|
||||
return
|
||||
if (
|
||||
str(payload.get("worker_id") or "") != profile.worker_id
|
||||
or str(payload.get("identity") or "") != profile.identity
|
||||
or set(payload.get("capabilities") or []) != set(profile.capabilities)
|
||||
or str(payload.get("trust_zone") or "") != "isolated-review"
|
||||
or int(payload.get("max_concurrency") or 0) != 1
|
||||
or int(payload.get("heartbeat_interval_seconds") or 0) != 15
|
||||
):
|
||||
raise ForbiddenError("readonly worker registration mismatch")
|
||||
|
||||
def verify_result(
|
||||
self, result: ResultEnvelope, task: TaskRecord | None = None
|
||||
) -> PolicyDecision:
|
||||
computed = result.computed_digest()
|
||||
if result.result_digest and result.result_digest != computed:
|
||||
return PolicyDecision(False, "result digest mismatch")
|
||||
sensitive = bool(
|
||||
task
|
||||
and (
|
||||
task.task_plan.data_classification
|
||||
in {"confidential", "credential-secret"}
|
||||
or any(
|
||||
capability.startswith(
|
||||
("code.write", "deploy", "secret", "production")
|
||||
)
|
||||
for capability in task.task_plan.required_capabilities
|
||||
)
|
||||
)
|
||||
)
|
||||
if sensitive and not (result.signature or result.attestation_ref):
|
||||
return PolicyDecision(False, "trusted result proof required")
|
||||
if result.signature is not None or result.attestation_ref is not None:
|
||||
if self.result_signature_verifier is None:
|
||||
return PolicyDecision(False, "result signature verifier unavailable")
|
||||
if not self.result_signature_verifier(result):
|
||||
return PolicyDecision(False, "result signature invalid")
|
||||
result.trust_status = "trusted_result"
|
||||
else:
|
||||
result.trust_status = "untrusted_shadow_result"
|
||||
return PolicyDecision(True)
|
||||
|
||||
def authorize(self, action: str, actor: str, task: TaskRecord | None = None, context: dict | None = None) -> PolicyDecision:
|
||||
context = context or {}
|
||||
if not self.identity_verifier.verify(actor):
|
||||
return PolicyDecision(False, "identity untrusted")
|
||||
|
||||
action_capabilities = {
|
||||
"create": "task.creator",
|
||||
"read": "task.read",
|
||||
"audit": "task.audit",
|
||||
"claim": "task.claim",
|
||||
"result": "task.claim",
|
||||
"fail": "task.claim",
|
||||
"progress": "task.claim",
|
||||
"heartbeat": "task.claim",
|
||||
"cancel": "task.cancel",
|
||||
"delegate": "task.delegate",
|
||||
"worker.register": "worker.self_register",
|
||||
"worker.admin": "worker.admin",
|
||||
}
|
||||
required_capability = action_capabilities.get(action)
|
||||
if required_capability:
|
||||
granted = set(
|
||||
self.identity_capabilities.get(
|
||||
actor, self.identity_capabilities.get("*", set())
|
||||
)
|
||||
)
|
||||
if required_capability not in granted:
|
||||
return PolicyDecision(False, f"action {action} not permitted")
|
||||
|
||||
if action in {"claim", "result", "fail", "cancel", "progress", "delegate", "heartbeat"} and not self.execution_enabled:
|
||||
if not self._readonly_worker_allows(action, actor, task):
|
||||
return PolicyDecision(False, "execution disabled")
|
||||
|
||||
if action == "delegate" and task and task.status not in {"COMPLETED", "FAILED_FINAL", "CANCELLED"}:
|
||||
risk = str(context.get("risk", task.task_plan.risk))
|
||||
if RISK_RANK.get(risk, 2) > RISK_RANK.get(self.max_risk, 1):
|
||||
return PolicyDecision(False, f"risk {risk} exceeds policy")
|
||||
|
||||
return PolicyDecision(True)
|
||||
|
||||
def assert_allowed(self, action: str, actor: str, task: TaskRecord | None = None, context: dict | None = None):
|
||||
decision = self.authorize(action, actor, task, context=context)
|
||||
if not decision.allowed:
|
||||
if decision.reason == "identity untrusted":
|
||||
raise ForbiddenError(decision.reason)
|
||||
if decision.reason == "execution disabled":
|
||||
raise LockedError("execution disabled")
|
||||
raise ForbiddenError(decision.reason)
|
||||
@@ -1,76 +0,0 @@
|
||||
"""Task scheduler primitives."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from .errors import NotFoundError
|
||||
from .capabilities import has_capabilities
|
||||
from .models import WorkerClaim
|
||||
from .store import TaskPlaneStore
|
||||
|
||||
|
||||
class TaskScheduler:
|
||||
def __init__(self, store: TaskPlaneStore):
|
||||
self.store = store
|
||||
|
||||
def next_queued_task(self, worker_id: str):
|
||||
worker = self.store.get_worker(worker_id)
|
||||
tasks = sorted(
|
||||
self.store.list_tasks(),
|
||||
key=lambda task: (
|
||||
-int(task.task_plan.metadata.get("priority", 0)),
|
||||
task.created_at,
|
||||
task.task_id,
|
||||
),
|
||||
)
|
||||
for task in tasks:
|
||||
if task.status not in {"CREATED", "QUEUED", "FAILED_RECOVERABLE"}:
|
||||
continue
|
||||
if task.task_plan.task_type == "worker.registry.audit":
|
||||
continue
|
||||
if task.status == "FAILED_RECOVERABLE" and not self.store.task_has_retryable_probe(
|
||||
task.task_id
|
||||
):
|
||||
continue
|
||||
if not has_capabilities(
|
||||
worker.capabilities, task.task_plan.required_capabilities
|
||||
):
|
||||
continue
|
||||
if any(
|
||||
self.store.get_task(dependency).status != "COMPLETED"
|
||||
for dependency in task.task_plan.dependencies
|
||||
):
|
||||
continue
|
||||
if (
|
||||
task.task_plan.data_classification == "credential-secret"
|
||||
and worker.worker_type != "secure_file_transformer"
|
||||
and "secure_file_transformer" not in worker.capabilities
|
||||
):
|
||||
continue
|
||||
return task
|
||||
return None
|
||||
|
||||
def schedule_for_worker(self, worker_id: str, lease_seconds: int = 15) -> WorkerClaim:
|
||||
worker = self.store.get_worker(worker_id)
|
||||
if worker.disabled:
|
||||
from .errors import LockedError
|
||||
|
||||
raise LockedError("worker disabled")
|
||||
if time.time() > worker.last_heartbeat + (
|
||||
worker.heartbeat_interval_seconds * 3
|
||||
):
|
||||
from .errors import LockedError
|
||||
|
||||
raise LockedError("worker offline")
|
||||
task = self.next_queued_task(worker_id)
|
||||
if task is None:
|
||||
raise NotFoundError("no queued tasks available")
|
||||
claim = self.store.claim_task(
|
||||
task.task_id,
|
||||
worker_id,
|
||||
worker.identity,
|
||||
lease_seconds=lease_seconds,
|
||||
expected_revision=task.revision,
|
||||
operation_digest=task.task_plan.operation_digest(),
|
||||
)
|
||||
return claim
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,68 +0,0 @@
|
||||
"""Worker registry abstraction and read helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
|
||||
from .capabilities import has_capabilities
|
||||
from .errors import NotFoundError
|
||||
from .models import WorkerRegistration
|
||||
from .store import TaskPlaneStore
|
||||
|
||||
|
||||
class WorkerRegistry:
|
||||
def __init__(self, store: TaskPlaneStore):
|
||||
self.store = store
|
||||
|
||||
def register(
|
||||
self,
|
||||
payload: dict,
|
||||
*,
|
||||
actor_identity: str,
|
||||
is_admin: bool,
|
||||
expected_revision: int | None,
|
||||
) -> WorkerRegistration:
|
||||
reg = WorkerRegistration.from_dict(payload)
|
||||
return self.store.register_worker(
|
||||
reg,
|
||||
actor_identity=actor_identity,
|
||||
is_admin=is_admin,
|
||||
expected_revision=expected_revision,
|
||||
)
|
||||
|
||||
def heartbeat(
|
||||
self,
|
||||
worker_id: str,
|
||||
heartbeat_interval_seconds: int | None = None,
|
||||
*,
|
||||
actor_identity: str,
|
||||
expected_revision: int | None,
|
||||
) -> WorkerRegistration:
|
||||
return self.store.heartbeat_worker(
|
||||
worker_id,
|
||||
heartbeat_interval_seconds=heartbeat_interval_seconds,
|
||||
actor_identity=actor_identity,
|
||||
expected_revision=expected_revision,
|
||||
)
|
||||
|
||||
def list(self):
|
||||
return self.store.list_workers()
|
||||
|
||||
def get(self, worker_id: str) -> WorkerRegistration:
|
||||
try:
|
||||
return self.store.get_worker(worker_id)
|
||||
except NotFoundError:
|
||||
raise
|
||||
|
||||
def can_execute(self, worker_id: str, required_capabilities: list[str]) -> bool:
|
||||
worker = self.store.get_worker(worker_id)
|
||||
return has_capabilities(worker.capabilities, required_capabilities)
|
||||
|
||||
def is_online(self, worker_id: str, *, now: float | None = None) -> bool:
|
||||
worker = self.store.get_worker(worker_id)
|
||||
deadline = worker.last_heartbeat + (worker.heartbeat_interval_seconds * 3)
|
||||
return not worker.disabled and (time.time() if now is None else now) <= deadline
|
||||
|
||||
def stale_workers(self, *, now: float | None = None) -> list[WorkerRegistration]:
|
||||
current = time.time() if now is None else now
|
||||
return [worker for worker in self.list() if not self.is_online(worker.worker_id, now=current)]
|
||||
@@ -9,6 +9,7 @@ Complete autonomous cycle with notification integration.
|
||||
|
||||
import json, os, hashlib, time, sys, logging, subprocess, requests
|
||||
from pathlib import Path
|
||||
from urllib.parse import quote
|
||||
|
||||
WORKSPACE = Path(os.environ.get("AIOS_WORKSPACE", "/data/aios-gitops/workspace"))
|
||||
PENDING_DIR = WORKSPACE / "agents" / "patches" / "pending/"
|
||||
@@ -26,11 +27,48 @@ FAILED_DIR = WORKSPACE / "agents" / "commands" / "failed/"
|
||||
POLL_INTERVAL = int(os.environ.get("CONTROLLER_POLL_INTERVAL_SECONDS", "600"))
|
||||
GITEA_URL = os.environ.get("GITEA_URL", "http://127.0.0.1:3000")
|
||||
API_URL = os.environ.get("AIOS_API_URL", "http://127.0.0.1:18890")
|
||||
GITEA_TOKEN = os.environ.get("GITEA_TOKEN", "")
|
||||
|
||||
|
||||
def _load_gitea_token(environ=None):
|
||||
environ = os.environ if environ is None else environ
|
||||
token_file = environ.get("GITEA_TOKEN_FILE", "").strip()
|
||||
credentials_directory = environ.get("CREDENTIALS_DIRECTORY", "").strip()
|
||||
if not token_file and credentials_directory:
|
||||
token_file = str(Path(credentials_directory) / "gitea-token")
|
||||
if token_file:
|
||||
path = Path(token_file)
|
||||
try:
|
||||
if path.is_file():
|
||||
return path.read_text().strip()
|
||||
except OSError:
|
||||
return ""
|
||||
return environ.get("GITEA_TOKEN", "").strip()
|
||||
|
||||
|
||||
GITEA_TOKEN = _load_gitea_token()
|
||||
AUTHORITATIVE_REPO = Path(os.environ.get("AIOS_GITOPS_AUTHORITATIVE_REPO", "/data/aios-gitops/repo.git"))
|
||||
REMOTE_NAME = os.environ.get("AIOS_GITOPS_REMOTE_NAME", "gitea-http").strip()
|
||||
REPOSITORY_PATH = os.environ.get("AIOS_GITEA_REPOSITORY_PATH", "aios/aios").strip().strip("/")
|
||||
CANDIDATE_BRANCH = os.environ.get(
|
||||
"AIOS_V2_CANDIDATE_BRANCH",
|
||||
"integration/aios-v2r1-cto-production-candidate-20260720",
|
||||
).strip()
|
||||
TASK_BRANCHES = tuple(
|
||||
branch.strip()
|
||||
for branch in os.environ.get(
|
||||
"AIOS_V2_TASK_BRANCHES",
|
||||
"task/openclaw/AIOS-V2R1-PRODUCTION-DEPLOY-20260720,"
|
||||
"task/codex/AIOS-V2R1-PRODUCTION-SUPERVISOR-20260720",
|
||||
).split(",")
|
||||
if branch.strip()
|
||||
)
|
||||
CONTROLLER_CHECKOUT = Path(
|
||||
os.environ.get("AIOS_CONTROLLER_CHECKOUT", "/data/aios-gitops/worktrees/controller")
|
||||
)
|
||||
|
||||
logging.basicConfig(level=logging.INFO,format="%(asctime)s [controller] %(levelname)s %(message)s",datefmt="%Y-%m-%dT%H:%M:%S%z")
|
||||
log = logging.getLogger("autonomous-controller")
|
||||
CONTROLLER_VERSION = "2.0.0"
|
||||
CONTROLLER_VERSION = "2.1.0"
|
||||
|
||||
|
||||
def load_state():
|
||||
@@ -62,7 +100,8 @@ def notify(title, body, urgency="info"):
|
||||
|
||||
def h_gitea():
|
||||
try:
|
||||
r = requests.get(f"{GITEA_URL}/api/v1/version", timeout=5)
|
||||
headers = {"Authorization": f"token {GITEA_TOKEN}"} if GITEA_TOKEN else {}
|
||||
r = requests.get(f"{GITEA_URL}/api/v1/version", headers=headers, timeout=5)
|
||||
return {"ok": r.status_code == 200, "data": r.json() if r.status_code == 200 else None, "err": None if r.status_code == 200 else f"HTTP {r.status_code}"}
|
||||
except Exception as e: return {"ok": False, "err": str(e)}
|
||||
|
||||
@@ -75,23 +114,140 @@ def h_api():
|
||||
return {"ok": False, "err": f"HTTP {r.status_code}"}
|
||||
except Exception as e: return {"ok": False, "err": str(e)}
|
||||
|
||||
def h_branch_sync():
|
||||
bare = Path("/data/aios-gitops/runtime/gitea-bin/data/repositories/aios/aios-v23.git")
|
||||
def _configured_branches():
|
||||
return tuple(dict.fromkeys((CANDIDATE_BRANCH, *TASK_BRANCHES)))
|
||||
|
||||
|
||||
def _git(*args, timeout=10):
|
||||
return subprocess.run(
|
||||
["git", "--git-dir", str(AUTHORITATIVE_REPO), *args],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=timeout,
|
||||
)
|
||||
|
||||
|
||||
def _local_branch_state(branch):
|
||||
if not AUTHORITATIVE_REPO.is_dir():
|
||||
return {"status": "failed", "oid": None, "reason": "authoritative_repo_unavailable"}
|
||||
try:
|
||||
r = subprocess.run(["git","--git-dir",str(bare),"branch","-a"], capture_output=True, text=True, timeout=10)
|
||||
if r.returncode != 0: return {"ok": False, "err": r.stderr[:200]}
|
||||
bb = set(l.strip().replace("remotes/origin/","").replace("* ","") for l in r.stdout.strip().split("\n") if l.strip())
|
||||
gb = []; pg = 1
|
||||
while True:
|
||||
r2 = requests.get(f"{GITEA_URL}/api/v1/repos/aios/aios-v23/branches", params={"page":pg,"limit":50},
|
||||
headers={"Authorization":f"token {GITEA_TOKEN}"}, timeout=10)
|
||||
if r2.status_code != 200: break
|
||||
d = r2.json()
|
||||
if not d: break
|
||||
gb.extend(b["name"] for b in d); pg += 1
|
||||
gs = set(gb); miss = bb - gs
|
||||
return {"ok": len(miss)==0, "bare": len(bb), "gitea": len(gs), "missing": list(miss)[:10]}
|
||||
except Exception as e: return {"ok": False, "err": str(e)}
|
||||
result = _git("rev-parse", "--verify", f"refs/heads/{branch}")
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return {"status": "failed", "oid": None, "reason": "git_query_failed"}
|
||||
if result.returncode == 0:
|
||||
return {"status": "synced", "oid": result.stdout.strip()}
|
||||
return {"status": "missing", "oid": None, "reason": "branch_missing"}
|
||||
|
||||
|
||||
def _repository_api_path():
|
||||
parts = [part for part in REPOSITORY_PATH.split("/") if part]
|
||||
if len(parts) != 2:
|
||||
return None
|
||||
return "/".join(quote(part, safe="") for part in parts)
|
||||
|
||||
|
||||
def _gitea_branch_state(branch):
|
||||
repository_api_path = _repository_api_path()
|
||||
if not repository_api_path:
|
||||
return {"status": "failed", "oid": None, "reason": "repository_path_invalid"}
|
||||
headers = {"Authorization": f"token {GITEA_TOKEN}"} if GITEA_TOKEN else {}
|
||||
url = f"{GITEA_URL}/api/v1/repos/{repository_api_path}/branches/{quote(branch, safe='')}"
|
||||
try:
|
||||
response = requests.get(url, headers=headers, timeout=10)
|
||||
except requests.RequestException:
|
||||
return {"status": "failed", "oid": None, "reason": "remote_unreachable"}
|
||||
if response.status_code == 404:
|
||||
return {"status": "missing", "oid": None, "reason": "branch_missing"}
|
||||
if response.status_code in (401, 403):
|
||||
return {"status": "failed", "oid": None, "reason": "authentication_failed"}
|
||||
if response.status_code != 200:
|
||||
return {"status": "failed", "oid": None, "reason": f"http_{response.status_code}"}
|
||||
try:
|
||||
oid = str(response.json()["commit"]["id"]).strip()
|
||||
except (KeyError, TypeError, ValueError):
|
||||
return {"status": "failed", "oid": None, "reason": "invalid_branch_response"}
|
||||
if not oid:
|
||||
return {"status": "failed", "oid": None, "reason": "invalid_branch_response"}
|
||||
return {"status": "synced", "oid": oid}
|
||||
|
||||
|
||||
def _sync_status(branches):
|
||||
states = tuple(branches.values())
|
||||
if any(item["gitea_sync"]["status"] == "failed" for item in states):
|
||||
return "gitea_failed"
|
||||
if states and all(item["oid_match"] for item in states):
|
||||
return "fully_synced"
|
||||
if states and all(item["local_sync"]["status"] == "synced" for item in states) and all(
|
||||
item["gitea_sync"]["status"] == "missing" for item in states
|
||||
):
|
||||
return "local_only"
|
||||
return "gitea_pending"
|
||||
|
||||
|
||||
def h_branch_sync():
|
||||
try:
|
||||
remote = _git("config", "--get", f"remote.{REMOTE_NAME}.url")
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
remote = None
|
||||
remote_configured = bool(remote and remote.returncode == 0 and remote.stdout.strip())
|
||||
branches = {}
|
||||
for branch in _configured_branches():
|
||||
local = _local_branch_state(branch)
|
||||
gitea = _gitea_branch_state(branch)
|
||||
branches[branch] = {
|
||||
"local_sync": local,
|
||||
"gitea_sync": gitea,
|
||||
"oid_match": (
|
||||
local["status"] == "synced"
|
||||
and gitea["status"] == "synced"
|
||||
and local["oid"] == gitea["oid"]
|
||||
),
|
||||
}
|
||||
candidate = branches[CANDIDATE_BRANCH]
|
||||
status = _sync_status(branches)
|
||||
oid_match = bool(branches) and all(item["oid_match"] for item in branches.values())
|
||||
missing = [
|
||||
branch
|
||||
for branch, item in branches.items()
|
||||
if item["local_sync"]["status"] == "synced" and not item["oid_match"]
|
||||
]
|
||||
return {
|
||||
"ok": oid_match and remote_configured,
|
||||
"local_sync": candidate["local_sync"],
|
||||
"gitea_sync": candidate["gitea_sync"],
|
||||
"oid_match": oid_match,
|
||||
"branch_sync": oid_match,
|
||||
"sync_status": status if remote_configured else "gitea_failed",
|
||||
"remote_configured": remote_configured,
|
||||
"branches": branches,
|
||||
"missing": missing[:10],
|
||||
"config": {
|
||||
"authoritative_repo": str(AUTHORITATIVE_REPO),
|
||||
"remote_name": REMOTE_NAME,
|
||||
"repository_path": REPOSITORY_PATH,
|
||||
"candidate_branch": CANDIDATE_BRANCH,
|
||||
"task_branches": list(TASK_BRANCHES),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def h_controller_checkout():
|
||||
if not CONTROLLER_CHECKOUT.is_dir():
|
||||
return {"ok": False, "detached": None, "clean": None, "reason": "checkout_missing"}
|
||||
try:
|
||||
status = subprocess.run(
|
||||
["git", "-c", f"safe.directory={CONTROLLER_CHECKOUT}", "-C", str(CONTROLLER_CHECKOUT), "status", "--porcelain"],
|
||||
capture_output=True, text=True, timeout=10,
|
||||
)
|
||||
branch = subprocess.run(
|
||||
["git", "-c", f"safe.directory={CONTROLLER_CHECKOUT}", "-C", str(CONTROLLER_CHECKOUT), "symbolic-ref", "--quiet", "--short", "HEAD"],
|
||||
capture_output=True, text=True, timeout=10,
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return {"ok": False, "detached": None, "clean": None, "reason": "git_query_failed"}
|
||||
clean = status.returncode == 0 and not status.stdout.strip()
|
||||
detached = branch.returncode != 0
|
||||
return {"ok": clean, "detached": detached, "clean": clean}
|
||||
|
||||
def h_stuck():
|
||||
s = []
|
||||
@@ -157,6 +313,23 @@ def apply_approved_patches():
|
||||
f.rename(REJECTED_DIR / f.name)
|
||||
return applied
|
||||
|
||||
|
||||
def build_health_summary(results, plans_count, applied_count):
|
||||
branch_health = results["branch_sync"]
|
||||
return {
|
||||
"ts": time.strftime("%Y-%m-%dT%H:%M:%S%z"),
|
||||
"ok": all(result.get("ok", False) for result in results.values()),
|
||||
"checks": {name: result.get("ok") for name, result in results.items()},
|
||||
"local_sync": branch_health["local_sync"],
|
||||
"gitea_sync": branch_health["gitea_sync"],
|
||||
"oid_match": branch_health["oid_match"],
|
||||
"branch_sync": branch_health["branch_sync"],
|
||||
"sync_status": branch_health["sync_status"],
|
||||
"branches": branch_health["branches"],
|
||||
"plans": plans_count,
|
||||
"applied": applied_count,
|
||||
}
|
||||
|
||||
# === Main Cycle ===
|
||||
|
||||
def run_cycle():
|
||||
@@ -176,6 +349,14 @@ def run_cycle():
|
||||
s = "ok" if results["branch_sync"]["ok"] else "degraded"
|
||||
audit("branch_sync", s, results["branch_sync"])
|
||||
|
||||
log.info("Check: Controller checkout")
|
||||
results["controller_checkout"] = h_controller_checkout()
|
||||
audit(
|
||||
"controller_checkout",
|
||||
"ok" if results["controller_checkout"]["ok"] else "degraded",
|
||||
results["controller_checkout"],
|
||||
)
|
||||
|
||||
log.info("Check: Stuck commands")
|
||||
results["stuck"] = h_stuck()
|
||||
audit("stuck", "ok" if results["stuck"]["ok"] else "degraded", results["stuck"])
|
||||
@@ -191,15 +372,10 @@ def run_cycle():
|
||||
applied = apply_approved_patches()
|
||||
|
||||
# Update health file
|
||||
all_ok = all(r.get("ok",False) for r in results.values())
|
||||
summary = {
|
||||
"ts": time.strftime("%Y-%m-%dT%H:%M:%S%z"),
|
||||
"ok": all_ok, "checks": {k:v.get("ok") for k,v in results.items()},
|
||||
"plans": len(plans), "applied": len(applied)
|
||||
}
|
||||
summary = build_health_summary(results, len(plans), len(applied))
|
||||
HEALTH_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
HEALTH_FILE.write_text(json.dumps(summary, indent=2, ensure_ascii=False))
|
||||
log.info(f"Cycle done. All healthy: {all_ok}. Plans: {len(plans)}, Applied: {len(applied)}")
|
||||
log.info(f"Cycle done. All healthy: {summary['ok']}. Plans: {len(plans)}, Applied: {len(applied)}")
|
||||
return results, plans, applied
|
||||
|
||||
|
||||
|
||||
+12
-24
@@ -1,25 +1,13 @@
|
||||
{
|
||||
"current_baseline": "V2R1C2",
|
||||
"previous_baseline": "V2R1C1",
|
||||
"promoted_at": "2026-07-22T01:40:51Z",
|
||||
"git": "3dd622b5a515f1fa4ef72b9f8ae9ce48ac703f24",
|
||||
"promoted_by": "codex/owner-approved-gitops",
|
||||
"promotion_reason": "Owner-authorized V2R1C2 Unified Portal production release after exact tag binding, complete evidence verification, and production acceptance PASS.",
|
||||
"version": "V2R1C2",
|
||||
"release_oid": "3dd622b5a515f1fa4ef72b9f8ae9ce48ac703f24",
|
||||
"tag_oid": "a3bd84ebec9c705f7c9c5c03f7f4ec55f8854e36",
|
||||
"target_oid": "3dd622b5a515f1fa4ef72b9f8ae9ce48ac703f24",
|
||||
"release_run_id": "AIOS-V2R1C2-PRODUCTION-20260722T011836Z",
|
||||
"status": "PRODUCTION_DEPLOYED",
|
||||
"production_service": "aios-v2r1c2-production.service",
|
||||
"production_port": 18920,
|
||||
"rollback_version": "V2R1C1",
|
||||
"constraints": {
|
||||
"product_code_changed": true,
|
||||
"production_redeployed": true,
|
||||
"production_traffic_changed": true,
|
||||
"tag_rewritten": false,
|
||||
"force_push_allowed": false,
|
||||
"rollback_assets_preserved": true
|
||||
}
|
||||
}
|
||||
"current_baseline": "V1R4C6",
|
||||
"previous_baseline": "V1R4C5",
|
||||
"promoted_at": "2026-07-11T21:00:00+08:00",
|
||||
"git": "e204a90398bf646d46245b3088226749fa056edd",
|
||||
"promoted_by": "openclaw/admin",
|
||||
"promotion_reason": "V1R4C6 admin search agent integration, search gateway, agent platform, VERSION bump, merge tests deploy pipeline — not yet deployed to production",
|
||||
"constraints": {
|
||||
"memory_search_enabled": false,
|
||||
"force_push_allowed": false,
|
||||
"gitea_db_mutation_requires_owner_approval": true
|
||||
}
|
||||
}
|
||||
@@ -1,103 +0,0 @@
|
||||
{
|
||||
"schema_version": 4,
|
||||
"automatic": [
|
||||
"build",
|
||||
"test",
|
||||
"artifact",
|
||||
"canary",
|
||||
"evidence"
|
||||
],
|
||||
"owner": [
|
||||
"signed_release_plan"
|
||||
],
|
||||
"forbidden": [
|
||||
"force_push",
|
||||
"tag_rewrite",
|
||||
"direct_main_push",
|
||||
"baseline_mutation",
|
||||
"registry_mutation"
|
||||
],
|
||||
"runtime_deployment": {
|
||||
"approval_action": "signed_release_plan",
|
||||
"required_capability": "runtime_switch",
|
||||
"tag_operations_allowed": false,
|
||||
"baseline_mutation_allowed": false,
|
||||
"registry_mutation_allowed": false,
|
||||
"writer_default": false,
|
||||
"rollback_required": true
|
||||
},
|
||||
"tag_release": {
|
||||
"approval_action": "signed_release_plan",
|
||||
"required_capability": "tag_publish",
|
||||
"separate_controller": true,
|
||||
"duplicate_conflict": "fail_closed",
|
||||
"rewrite_allowed": false,
|
||||
"force_push_allowed": false
|
||||
},
|
||||
"baseline_registry_finalize": {
|
||||
"approval_action": "signed_release_plan",
|
||||
"required_capability": "baseline_registry_finalize",
|
||||
"separate_controller": true,
|
||||
"remote_tag_verification_required": true,
|
||||
"gitea_api_verification_required": true,
|
||||
"registry_baseline_consistency_required": true
|
||||
},
|
||||
"approval": {
|
||||
"algorithm": "Ed25519",
|
||||
"trust_domain": "aios.production",
|
||||
"maximum_lifetime_seconds": 3600,
|
||||
"release_plan_sha256_required": true,
|
||||
"not_before_required": true,
|
||||
"replay_protection": true,
|
||||
"revocation_fail_closed": true,
|
||||
"key_rotation_supported": true,
|
||||
"capabilities": [
|
||||
"runtime_switch",
|
||||
"tag_publish",
|
||||
"baseline_registry_finalize"
|
||||
]
|
||||
},
|
||||
"execution": {
|
||||
"environment_variable": "AIOS_SILENT_RELEASE_EXECUTION_ENABLED",
|
||||
"default_enabled": false,
|
||||
"shadow_side_effects_allowed": false,
|
||||
"durable_saga_required": true,
|
||||
"pre_switch_revalidation_required": true,
|
||||
"trusted_candidate_builder_required": true,
|
||||
"actual_plan_regeneration_required": true,
|
||||
"startup_fail_closed_on_missing_component": true,
|
||||
"recovery_worker_shadow_default": true,
|
||||
"production_approval_backend_required": true,
|
||||
"production_builder_signer": "kms-hsm-secret-manager-only",
|
||||
"production_execution_without_backend": "unavailable"
|
||||
},
|
||||
"durability": {
|
||||
"journal": "sha256-hash-chain",
|
||||
"probe_journal": "sha256-hash-chain",
|
||||
"exclusive_capability_claims": true,
|
||||
"side_effect_probe_before_lease_takeover": true,
|
||||
"side_effect_probe_before_failed_recoverable_retry": true,
|
||||
"probe_results": [
|
||||
"NOT_APPLIED",
|
||||
"APPLIED_VERIFIED",
|
||||
"INDETERMINATE",
|
||||
"CONFLICT"
|
||||
],
|
||||
"probe_operation_digest_required": true,
|
||||
"probe_freshness_required": true,
|
||||
"tamper_result": "MANUAL_OWNER_REVIEW_REQUIRED"
|
||||
},
|
||||
"tag_pending": {
|
||||
"required_observations": [
|
||||
"remote_git_ref",
|
||||
"annotated_tag_object",
|
||||
"peeled_target",
|
||||
"gitea_api",
|
||||
"gitea_listing",
|
||||
"repository_identity"
|
||||
],
|
||||
"indeterminate_result": "RECONCILIATION_REQUIRED",
|
||||
"conflict_result": "MANUAL_OWNER_REVIEW_REQUIRED",
|
||||
"automatic_reconciliation": false
|
||||
}
|
||||
}
|
||||
@@ -1,59 +0,0 @@
|
||||
version: 1
|
||||
policy_id: gitops-isolated-worktree-governance-20260714T232948UTC8
|
||||
|
||||
runtime_identity:
|
||||
user: aiman
|
||||
group: gitops
|
||||
umask: "0002"
|
||||
prohibit_root_workspace_writes: true
|
||||
|
||||
repository:
|
||||
source_worktree: /data/aios-gitops/workspace
|
||||
bare_repo: /data/aios-gitops/repo.git
|
||||
worktree_root: /data/aios-gitops/worktrees
|
||||
remote: origin
|
||||
|
||||
worktrees:
|
||||
controller: /data/aios-gitops/worktrees/controller
|
||||
task_pattern: /data/aios-gitops/worktrees/{agent}/{task_id}
|
||||
allowed_agents:
|
||||
- openclaw
|
||||
- chatgpt
|
||||
- controller
|
||||
require_clean_start: true
|
||||
remove_after_successful_push: true
|
||||
retain_on_failure: true
|
||||
|
||||
permissions:
|
||||
owner: aiman
|
||||
group: gitops
|
||||
directory_mode: "2775"
|
||||
file_mode: "0664"
|
||||
default_acl:
|
||||
user:aiman: rwx
|
||||
group:gitops: rwx
|
||||
apply_to:
|
||||
- /data/aios-gitops
|
||||
- /data/aios-gitops/worktrees
|
||||
|
||||
preflight:
|
||||
required_user: aiman
|
||||
required_group: gitops
|
||||
required_umask: "0002"
|
||||
require_writable_git_dir: true
|
||||
require_writable_task_dir: true
|
||||
require_writable_result_dir: true
|
||||
reject_shared_source_worktree: true
|
||||
reject_existing_git_lock: true
|
||||
|
||||
notifications:
|
||||
owner_on_received: required
|
||||
owner_on_terminal: required
|
||||
preferred_channel: weixin
|
||||
redact_secrets: true
|
||||
|
||||
rollback:
|
||||
preserve_source_worktree: true
|
||||
remove_created_worktrees: true
|
||||
restore_service_dropins: true
|
||||
restore_service_state: true
|
||||
@@ -1,14 +0,0 @@
|
||||
# Install as a systemd drop-in for GitOps/OpenClaw writer services.
|
||||
# Example: /etc/systemd/system/<service>.service.d/10-runtime-identity.conf
|
||||
|
||||
[Service]
|
||||
User=aiman
|
||||
Group=gitops
|
||||
SupplementaryGroups=gitops
|
||||
UMask=0002
|
||||
Environment=AIOS_GITOPS_ROOT=/data/aios-gitops
|
||||
Environment=AIOS_BARE_REPO=/data/aios-gitops/repo.git
|
||||
Environment=AIOS_WORKTREE_ROOT=/data/aios-gitops/worktrees
|
||||
Environment=AIOS_RUNTIME_USER=aiman
|
||||
Environment=AIOS_RUNTIME_GROUP=gitops
|
||||
Environment=AIOS_RUNTIME_UMASK=0002
|
||||
@@ -0,0 +1,183 @@
|
||||
#!/usr/bin/env bash
|
||||
set -Eeuo pipefail
|
||||
|
||||
REPO_ROOT="${AIOS_REPO_ROOT:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)}"
|
||||
BASE_REF="${AIOS_DOCS_BASE_REF:-}"
|
||||
RUN_LIVE_HEALTHCHECK="${AIOS_RUN_LIVE_HEALTHCHECK:-0}"
|
||||
RESULT_DIR="${AIOS_ACCEPTANCE_RESULT_DIR:-$REPO_ROOT/.aios-evidence/acceptance}"
|
||||
STAMP="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
RUN_DIR="$RESULT_DIR/$STAMP"
|
||||
SUMMARY="$RUN_DIR/summary.json"
|
||||
|
||||
install -d -m 0750 "$RUN_DIR"
|
||||
|
||||
pass=0
|
||||
failed=0
|
||||
warnings=0
|
||||
steps=()
|
||||
|
||||
record() {
|
||||
local name="$1" status="$2" code="$3" log="$4"
|
||||
steps+=("$name|$status|$code|$log")
|
||||
case "$status" in
|
||||
通过) pass=$((pass + 1)) ;;
|
||||
失败) failed=$((failed + 1)) ;;
|
||||
警告) warnings=$((warnings + 1)) ;;
|
||||
esac
|
||||
printf '%s:%s(退出码 %s)\n' "$status" "$name" "$code"
|
||||
}
|
||||
|
||||
run_step() {
|
||||
local name="$1"
|
||||
shift
|
||||
local safe_name
|
||||
safe_name="$(printf '%s' "$name" | tr ' /:' '___')"
|
||||
local log="$RUN_DIR/${safe_name}.log"
|
||||
set +e
|
||||
(cd "$REPO_ROOT" && "$@") >"$log" 2>&1
|
||||
local code=$?
|
||||
set -e
|
||||
if [[ "$code" -eq 0 ]]; then
|
||||
record "$name" "通过" "$code" "$log"
|
||||
else
|
||||
record "$name" "失败" "$code" "$log"
|
||||
fi
|
||||
}
|
||||
|
||||
run_optional_step() {
|
||||
local name="$1"
|
||||
shift
|
||||
local safe_name
|
||||
safe_name="$(printf '%s' "$name" | tr ' /:' '___')"
|
||||
local log="$RUN_DIR/${safe_name}.log"
|
||||
set +e
|
||||
(cd "$REPO_ROOT" && "$@") >"$log" 2>&1
|
||||
local code=$?
|
||||
set -e
|
||||
if [[ "$code" -eq 0 ]]; then
|
||||
record "$name" "通过" "$code" "$log"
|
||||
else
|
||||
record "$name" "警告" "$code" "$log"
|
||||
fi
|
||||
}
|
||||
|
||||
printf 'AIOS V2R1 Candidate 全量验收开始\n'
|
||||
printf '仓库:%s\n' "$REPO_ROOT"
|
||||
printf '结果目录:%s\n' "$RUN_DIR"
|
||||
|
||||
[[ -d "$REPO_ROOT" ]] || { echo "错误:仓库不存在" >&2; exit 2; }
|
||||
|
||||
DOCS_ARGS=(python3 tools/aios_v2_docs_code_check.py --repo "$REPO_ROOT")
|
||||
if [[ -n "$BASE_REF" ]]; then
|
||||
DOCS_ARGS+=(--base-ref "$BASE_REF")
|
||||
fi
|
||||
run_step "文档与代码匹配" "${DOCS_ARGS[@]}"
|
||||
|
||||
run_step "Python 语法编译" python3 -m compileall -q \
|
||||
active-runtime/app/aios_v2 \
|
||||
active-runtime/app/aios_openclaw \
|
||||
active-runtime/app/aios_hermes
|
||||
|
||||
run_step "候选单元与集成测试" python3 -m unittest discover \
|
||||
-s tests/aios_v2_candidate \
|
||||
-p 'test_*.py' \
|
||||
-v
|
||||
|
||||
run_step "Shell 脚本语法" bash -c '
|
||||
set -Eeuo pipefail
|
||||
while IFS= read -r -d "" path; do
|
||||
bash -n "$path"
|
||||
done < <(find deploy/aios-v2 -type f -name "*.sh" -print0)
|
||||
'
|
||||
|
||||
run_step "文档 Git 冲突标记" bash -c '
|
||||
set -Eeuo pipefail
|
||||
if grep -RInE "^(<<<<<<<|=======|>>>>>>>)" docs/aios-v2; then
|
||||
exit 1
|
||||
fi
|
||||
'
|
||||
|
||||
run_step "明文密钥基础扫描" python3 -c '
|
||||
from pathlib import Path
|
||||
import re
|
||||
import sys
|
||||
|
||||
roots = [Path("active-runtime/app"), Path("deploy/aios-v2"), Path("docs/aios-v2"), Path("tasks")]
|
||||
patterns = [
|
||||
re.compile(r"(?i)(api[_-]?key|password|private[_-]?key|access[_-]?token)\s*[:=]\s*[\"\x27][^\"\x27\n]{12,}[\"\x27]"),
|
||||
re.compile(r"-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----"),
|
||||
]
|
||||
violations = []
|
||||
for root in roots:
|
||||
if not root.exists():
|
||||
continue
|
||||
for path in root.rglob("*"):
|
||||
if not path.is_file() or path.suffix in {".pyc", ".db", ".sqlite3"}:
|
||||
continue
|
||||
text = path.read_text(encoding="utf-8", errors="ignore")
|
||||
if any(pattern.search(text) for pattern in patterns):
|
||||
violations.append(str(path))
|
||||
if violations:
|
||||
print("疑似明文密钥文件:", *violations, sep="\n", file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
'
|
||||
|
||||
if command -v systemd-analyze >/dev/null 2>&1; then
|
||||
run_optional_step "systemd 单元验证" systemd-analyze verify \
|
||||
deploy/aios-v2/systemd/aios-v2.service
|
||||
else
|
||||
record "systemd 单元验证" "警告" "127" "未安装 systemd-analyze"
|
||||
fi
|
||||
|
||||
if command -v docker >/dev/null 2>&1 && docker compose version >/dev/null 2>&1; then
|
||||
run_optional_step "Docker Compose 配置验证" docker compose \
|
||||
-f deploy/aios-v2/docker/compose.yaml config
|
||||
else
|
||||
record "Docker Compose 配置验证" "警告" "127" "未安装 Docker Compose"
|
||||
fi
|
||||
|
||||
if [[ "$RUN_LIVE_HEALTHCHECK" == "1" ]]; then
|
||||
run_step "在线健康检查" bash deploy/aios-v2/healthcheck/check.sh
|
||||
else
|
||||
record "在线健康检查" "警告" "0" "未启用 AIOS_RUN_LIVE_HEALTHCHECK=1"
|
||||
fi
|
||||
|
||||
branch="$(git -C "$REPO_ROOT" branch --show-current 2>/dev/null || true)"
|
||||
commit="$(git -C "$REPO_ROOT" rev-parse HEAD 2>/dev/null || true)"
|
||||
dirty="$(git -C "$REPO_ROOT" status --porcelain=v1 2>/dev/null || true)"
|
||||
|
||||
python3 - "$SUMMARY" "$branch" "$commit" "$dirty" "$pass" "$failed" "$warnings" "${steps[@]}" <<'PY'
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
|
||||
summary_path = pathlib.Path(sys.argv[1])
|
||||
branch, commit, dirty = sys.argv[2:5]
|
||||
passed, failed, warnings = map(int, sys.argv[5:8])
|
||||
steps = []
|
||||
for item in sys.argv[8:]:
|
||||
name, status, code, log = item.split("|", 3)
|
||||
steps.append({"name": name, "status": status, "exit_code": int(code), "log": log})
|
||||
payload = {
|
||||
"schema": "aios.v2.acceptance.v1",
|
||||
"generated_at": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"),
|
||||
"branch": branch or None,
|
||||
"commit": commit or None,
|
||||
"workspace_dirty": bool(dirty),
|
||||
"passed": passed,
|
||||
"failed": failed,
|
||||
"warnings": warnings,
|
||||
"steps": steps,
|
||||
"final_status": "passed" if failed == 0 else "failed",
|
||||
}
|
||||
summary_path.write_text(json.dumps(payload, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
print(json.dumps(payload, ensure_ascii=False, indent=2))
|
||||
PY
|
||||
|
||||
printf '验收结果:通过 %s,失败 %s,警告 %s\n' "$pass" "$failed" "$warnings"
|
||||
printf '证据:%s\n' "$SUMMARY"
|
||||
|
||||
if [[ "$failed" -ne 0 ]]; then
|
||||
exit 1
|
||||
fi
|
||||
@@ -0,0 +1,23 @@
|
||||
FROM python:3.12-slim
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
PYTHONPATH=/opt/aios/app \
|
||||
AIOS_V2_HOST=0.0.0.0 \
|
||||
AIOS_V2_PORT=18920 \
|
||||
AIOS_V2_STATE_DIR=/var/lib/aios-v2
|
||||
|
||||
RUN groupadd --system --gid 12001 aios \
|
||||
&& useradd --system --uid 12001 --gid aios --home-dir /nonexistent --shell /usr/sbin/nologin aios \
|
||||
&& install -d -o aios -g aios -m 0750 /opt/aios/app /var/lib/aios-v2
|
||||
|
||||
WORKDIR /opt/aios/app
|
||||
COPY --chown=aios:aios active-runtime/app/ /opt/aios/app/
|
||||
|
||||
USER aios:aios
|
||||
EXPOSE 18920
|
||||
|
||||
HEALTHCHECK --interval=15s --timeout=5s --start-period=10s --retries=4 \
|
||||
CMD python3 -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:18920/api/v2/health', timeout=3).read()"
|
||||
|
||||
CMD ["python3", "-m", "aios_v2.app"]
|
||||
@@ -0,0 +1,44 @@
|
||||
services:
|
||||
aios-v2:
|
||||
build:
|
||||
context: ../../..
|
||||
dockerfile: deploy/aios-v2/docker/Dockerfile
|
||||
image: aios-v2:V2R1C0-candidate
|
||||
init: true
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "127.0.0.1:${AIOS_V2_PORT:-18920}:18920"
|
||||
environment:
|
||||
AIOS_V2_HOST: 0.0.0.0
|
||||
AIOS_V2_PORT: 18920
|
||||
AIOS_V2_STATE_DIR: /var/lib/aios-v2
|
||||
AIOS_OPENCLAW_URL: ${AIOS_OPENCLAW_URL:-}
|
||||
AIOS_OPENCLAW_CREDENTIAL_REF: ${AIOS_OPENCLAW_CREDENTIAL_REF:-}
|
||||
AIOS_OPENCLAW_SIGNING_MATERIAL_FILE: ${AIOS_OPENCLAW_SIGNING_MATERIAL_FILE:-}
|
||||
AIOS_RELEASE_CHANNEL: candidate
|
||||
AIOS_RELEASE_VERSION: V2R1C0-candidate
|
||||
volumes:
|
||||
- aios-v2-state:/var/lib/aios-v2
|
||||
# Mount runtime signing material explicitly when OpenClaw is enabled.
|
||||
# The host file must be mode 0600; never store it in this repository.
|
||||
# - /etc/aios-v2/openclaw-signing-material:/run/secrets/openclaw-signing-material:ro
|
||||
read_only: true
|
||||
tmpfs:
|
||||
- /tmp:rw,noexec,nosuid,nodev,size=64m
|
||||
cap_drop:
|
||||
- ALL
|
||||
security_opt:
|
||||
- no-new-privileges:true
|
||||
healthcheck:
|
||||
test:
|
||||
- CMD
|
||||
- python3
|
||||
- -c
|
||||
- "import urllib.request; urllib.request.urlopen('http://127.0.0.1:18920/api/v2/health', timeout=3).read()"
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 4
|
||||
start_period: 10s
|
||||
|
||||
volumes:
|
||||
aios-v2-state:
|
||||
+21
@@ -0,0 +1,21 @@
|
||||
# AIOS V2 candidate runtime configuration.
|
||||
# Copy to /etc/aios-v2/aios-v2.env and keep ownership root:aios, mode 0640.
|
||||
|
||||
AIOS_V2_HOST=127.0.0.1
|
||||
AIOS_V2_PORT=18920
|
||||
AIOS_V2_STATE_DIR=/var/lib/aios-v2
|
||||
|
||||
# Optional OpenClaw runtime. Leave blank to run with OpenClaw disabled.
|
||||
AIOS_OPENCLAW_URL=
|
||||
AIOS_OPENCLAW_CREDENTIAL_REF=
|
||||
|
||||
# File path only. The referenced file must be mode 0600 or stricter and contain
|
||||
# at least 32 bytes. Never place signing material in this env file or Git.
|
||||
AIOS_OPENCLAW_SIGNING_MATERIAL_FILE=
|
||||
|
||||
# Reverse proxy must authenticate users and inject X-AIOS-* identity headers.
|
||||
AIOS_TRUST_PROXY_HEADERS=true
|
||||
|
||||
# Candidate deployment metadata exposed through health tooling.
|
||||
AIOS_RELEASE_CHANNEL=candidate
|
||||
AIOS_RELEASE_VERSION=V2R1C0-candidate
|
||||
@@ -0,0 +1,86 @@
|
||||
#!/usr/bin/env bash
|
||||
set -Eeuo pipefail
|
||||
|
||||
BASE_URL="${AIOS_V2_BASE_URL:-http://127.0.0.1:18920}"
|
||||
STATE_DIR="${AIOS_V2_STATE_DIR:-/var/lib/aios-v2}"
|
||||
CONTROLLER_WT="${AIOS_GITOPS_CONTROLLER_WORKTREE:-/data/aios-gitops/worktrees/controller}"
|
||||
SERVICE="${AIOS_V2_SERVICE:-aios-v2}"
|
||||
|
||||
fail() {
|
||||
echo "FAIL: $*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
pass() {
|
||||
echo "PASS: $*"
|
||||
}
|
||||
|
||||
command -v python3 >/dev/null || fail "python3 not installed"
|
||||
|
||||
if command -v systemctl >/dev/null && systemctl list-unit-files "$SERVICE.service" >/dev/null 2>&1; then
|
||||
systemctl is-active --quiet "$SERVICE" || fail "$SERVICE is not active"
|
||||
pass "$SERVICE is active"
|
||||
fi
|
||||
|
||||
python3 - "$BASE_URL" <<'PY'
|
||||
import json
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
base_url = sys.argv[1].rstrip("/")
|
||||
with urllib.request.urlopen(f"{base_url}/api/v2/health", timeout=5) as response:
|
||||
payload = json.loads(response.read().decode("utf-8"))
|
||||
if payload.get("status") not in {"ok", "healthy", "ready"}:
|
||||
raise SystemExit(f"unexpected health payload: {payload}")
|
||||
print(f"PASS: HTTP health status={payload.get('status')}")
|
||||
PY
|
||||
|
||||
[[ -d "$STATE_DIR" ]] || fail "state directory missing: $STATE_DIR"
|
||||
[[ -w "$STATE_DIR" ]] || fail "state directory not writable: $STATE_DIR"
|
||||
pass "state directory writable"
|
||||
|
||||
if [[ -e "$STATE_DIR/control-plane.sqlite3" ]]; then
|
||||
python3 - "$STATE_DIR/control-plane.sqlite3" <<'PY'
|
||||
import sqlite3
|
||||
import sys
|
||||
|
||||
path = sys.argv[1]
|
||||
connection = sqlite3.connect(f"file:{path}?mode=ro", uri=True, timeout=5)
|
||||
try:
|
||||
row = connection.execute("PRAGMA quick_check").fetchone()
|
||||
finally:
|
||||
connection.close()
|
||||
if not row or row[0] != "ok":
|
||||
raise SystemExit(f"SQLite quick_check failed: {row}")
|
||||
print("PASS: SQLite quick_check=ok")
|
||||
PY
|
||||
else
|
||||
echo "WARN: control-plane.sqlite3 not created yet"
|
||||
fi
|
||||
|
||||
if [[ -d "$CONTROLLER_WT" ]] && git -C "$CONTROLLER_WT" rev-parse --is-inside-work-tree >/dev/null 2>&1; then
|
||||
branch="$(git -C "$CONTROLLER_WT" branch --show-current)"
|
||||
dirty="$(git -C "$CONTROLLER_WT" status --porcelain=v1)"
|
||||
[[ -z "$branch" ]] || fail "GitOps controller attached to branch: $branch"
|
||||
[[ -z "$dirty" ]] || fail "GitOps controller workspace is dirty"
|
||||
pass "GitOps controller detached and clean"
|
||||
fi
|
||||
|
||||
python3 - "$STATE_DIR/executions" <<'PY'
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
root = pathlib.Path(sys.argv[1])
|
||||
if not root.exists():
|
||||
print("WARN: execution evidence directory not created yet")
|
||||
raise SystemExit(0)
|
||||
for path in root.glob("*/*.json"):
|
||||
value = json.loads(path.read_text(encoding="utf-8"))
|
||||
for key in ("execution_id", "tenant_id", "task_id", "executor", "status"):
|
||||
if not value.get(key):
|
||||
raise SystemExit(f"invalid execution evidence {path}: missing {key}")
|
||||
print("PASS: execution evidence files valid")
|
||||
PY
|
||||
|
||||
pass "AIOS V2 candidate healthcheck complete"
|
||||
@@ -0,0 +1,86 @@
|
||||
#!/usr/bin/env bash
|
||||
set -Eeuo pipefail
|
||||
|
||||
SERVICE="${AIOS_V2_SERVICE:-aios-v2}"
|
||||
RELEASE_ROOT="${AIOS_V2_RELEASE_ROOT:-/opt/aios/releases}"
|
||||
CURRENT_LINK="${AIOS_V2_CURRENT_LINK:-/opt/aios/current}"
|
||||
STATE_DIR="${AIOS_V2_STATE_DIR:-/var/lib/aios-v2}"
|
||||
BACKUP_ROOT="${AIOS_V2_BACKUP_ROOT:-/var/backups/aios-v2}"
|
||||
TARGET_RELEASE="${1:-}"
|
||||
RESTORE_STATE="${AIOS_V2_RESTORE_STATE:-0}"
|
||||
|
||||
fail() {
|
||||
echo "ERROR: $*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
[[ "${EUID}" -eq 0 ]] || fail "rollback must run as root"
|
||||
[[ -n "$TARGET_RELEASE" ]] || fail "usage: $0 <release-name-or-absolute-path>"
|
||||
|
||||
if [[ "$TARGET_RELEASE" = /* ]]; then
|
||||
TARGET_PATH="$(readlink -f "$TARGET_RELEASE")"
|
||||
else
|
||||
[[ "$TARGET_RELEASE" != *"/"* && "$TARGET_RELEASE" != "." && "$TARGET_RELEASE" != ".." ]] \
|
||||
|| fail "invalid release name"
|
||||
TARGET_PATH="$(readlink -f "$RELEASE_ROOT/$TARGET_RELEASE")"
|
||||
fi
|
||||
|
||||
[[ -d "$TARGET_PATH/active-runtime/app" ]] || fail "target release is invalid: $TARGET_PATH"
|
||||
[[ -f "$TARGET_PATH/deploy/aios-v2/healthcheck/check.sh" ]] \
|
||||
|| fail "target release has no healthcheck"
|
||||
|
||||
PREVIOUS_PATH=""
|
||||
if [[ -L "$CURRENT_LINK" ]]; then
|
||||
PREVIOUS_PATH="$(readlink -f "$CURRENT_LINK")"
|
||||
fi
|
||||
|
||||
STAMP="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
BACKUP_DIR="$BACKUP_ROOT/rollback-$STAMP"
|
||||
install -d -m 0750 "$BACKUP_DIR"
|
||||
|
||||
if [[ -d "$STATE_DIR" ]]; then
|
||||
tar --xattrs --acls -C "$(dirname "$STATE_DIR")" \
|
||||
-czf "$BACKUP_DIR/state-before-rollback.tar.gz" \
|
||||
"$(basename "$STATE_DIR")"
|
||||
fi
|
||||
|
||||
printf '%s\n' "$PREVIOUS_PATH" > "$BACKUP_DIR/previous-release.txt"
|
||||
printf '%s\n' "$TARGET_PATH" > "$BACKUP_DIR/target-release.txt"
|
||||
|
||||
systemctl stop "$SERVICE"
|
||||
ln -sfn "$TARGET_PATH" "${CURRENT_LINK}.next"
|
||||
mv -Tf "${CURRENT_LINK}.next" "$CURRENT_LINK"
|
||||
systemctl daemon-reload
|
||||
systemctl start "$SERVICE"
|
||||
|
||||
if AIOS_V2_SERVICE="$SERVICE" \
|
||||
AIOS_V2_STATE_DIR="$STATE_DIR" \
|
||||
"$TARGET_PATH/deploy/aios-v2/healthcheck/check.sh"; then
|
||||
echo "ROLLBACK_COMPLETE target=$TARGET_PATH backup=$BACKUP_DIR"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "Target release failed healthcheck; restoring previous release." >&2
|
||||
systemctl stop "$SERVICE" || true
|
||||
|
||||
if [[ -n "$PREVIOUS_PATH" && -d "$PREVIOUS_PATH" ]]; then
|
||||
ln -sfn "$PREVIOUS_PATH" "${CURRENT_LINK}.previous"
|
||||
mv -Tf "${CURRENT_LINK}.previous" "$CURRENT_LINK"
|
||||
fi
|
||||
|
||||
if [[ "$RESTORE_STATE" == "1" && -f "$BACKUP_DIR/state-before-rollback.tar.gz" ]]; then
|
||||
rm -rf "$STATE_DIR"
|
||||
tar --xattrs --acls -C "$(dirname "$STATE_DIR")" \
|
||||
-xzf "$BACKUP_DIR/state-before-rollback.tar.gz"
|
||||
fi
|
||||
|
||||
systemctl daemon-reload
|
||||
systemctl start "$SERVICE" || true
|
||||
|
||||
if [[ -n "$PREVIOUS_PATH" && -x "$PREVIOUS_PATH/deploy/aios-v2/healthcheck/check.sh" ]]; then
|
||||
AIOS_V2_SERVICE="$SERVICE" \
|
||||
AIOS_V2_STATE_DIR="$STATE_DIR" \
|
||||
"$PREVIOUS_PATH/deploy/aios-v2/healthcheck/check.sh" || true
|
||||
fi
|
||||
|
||||
fail "rollback target failed; previous release restored; evidence=$BACKUP_DIR"
|
||||
@@ -0,0 +1,37 @@
|
||||
[Unit]
|
||||
Description=AIOS V2 Control Plane Candidate
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=aios
|
||||
Group=aios
|
||||
WorkingDirectory=/opt/aios/current/active-runtime/app
|
||||
EnvironmentFile=-/etc/aios-v2/aios-v2.env
|
||||
Environment=PYTHONUNBUFFERED=1
|
||||
ExecStart=/usr/bin/python3 -m aios_v2.app
|
||||
Restart=on-failure
|
||||
RestartSec=3
|
||||
TimeoutStartSec=30
|
||||
TimeoutStopSec=30
|
||||
KillSignal=SIGTERM
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectControlGroups=true
|
||||
RestrictSUIDSGID=true
|
||||
RestrictRealtime=true
|
||||
LockPersonality=true
|
||||
MemoryDenyWriteExecute=true
|
||||
ReadWritePaths=/var/lib/aios-v2
|
||||
StateDirectory=aios-v2
|
||||
StateDirectoryMode=0750
|
||||
UMask=0027
|
||||
LimitNOFILE=65536
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,27 +0,0 @@
|
||||
# AIOS V2R1C2 Unified Portal production metadata.
|
||||
# No secret values belong in this file; runtime integrations use references.
|
||||
|
||||
AIOS_V2_HOST=127.0.0.1
|
||||
AIOS_V2_PORT=18920
|
||||
AIOS_V2_STATE_DIR=/var/lib/aios-v2r1c2-production
|
||||
AIOS_PROVENANCE_STORE_FILE=/var/lib/aios-v2r1c2-production/run-provenance.sqlite3
|
||||
AIOS_EXTERNAL_TASK_STORE_FILE=/var/lib/aios-v2r1c2-production/external-task-truth.sqlite3
|
||||
AIOS_EXECUTOR_CATALOG_FILE=/var/lib/aios-v2r1c2-production/executor-catalog.json
|
||||
AIOS_EXECUTION_PLAN_FILES=/opt/aios-v2r1c2/plans/AIOS-V2R1C2-UNIFIED-PORTAL-RELEASE.json:/opt/aios-v2r1c2/plans/AIOS-V2R1C2-UNIFIED-PORTAL-TAKEOVER.json
|
||||
AIOS_EXECUTION_PLAN_FILE=/opt/aios-v2r1c2/plans/AIOS-V2R1C2-UNIFIED-PORTAL-TAKEOVER.json
|
||||
|
||||
AIOS_RELEASE_VERSION=V2R1C2
|
||||
AIOS_TARGET_VERSION=V2R1C2
|
||||
AIOS_BASELINE_VERSION=V2R1C1
|
||||
AIOS_PREVIOUS_VERSION=V2R1C1
|
||||
AIOS_RELEASE_STATUS=PRODUCTION
|
||||
AIOS_RELEASE_SOURCE_OID=3dd622b5a515f1fa4ef72b9f8ae9ce48ac703f24
|
||||
AIOS_RELEASE_OID=3dd622b5a515f1fa4ef72b9f8ae9ce48ac703f24
|
||||
AIOS_RELEASE_TAG=V2R1C2
|
||||
AIOS_RELEASE_EVIDENCE_SHA256=2e27bb73b8e6297bd3077238f73a96d697fc3c3611c601c4dc9357e5fdcd8eee
|
||||
|
||||
AIOS_OPENCLAW_CONTRACT_VERSION=aios.openclaw.command.v2
|
||||
AIOS_OPENCLAW_SIGNATURE_ALGORITHM=Ed25519
|
||||
AIOS_OPENCLAW_PLUGIN_VERSION=2.1.0-rc.4
|
||||
AIOS_OPENCLAW_SIGNING_KEY_ID=openclaw-v2r1-candidate-ed25519
|
||||
AIOS_OWNER_WECHAT_TARGET_REF=AIOS_OWNER_WECHAT_TARGET_REF
|
||||
@@ -1,44 +0,0 @@
|
||||
[Unit]
|
||||
Description=AIOS V2R1C2 Unified Portal Production
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
Conflicts=aios-orchestrator-gateway.service
|
||||
ConditionPathExists=/opt/aios-v2r1c2/active-runtime/app/aios_web/release_application.py
|
||||
ConditionPathExists=/opt/aios-v2r1c2/plans/AIOS-V2R1C2-UNIFIED-PORTAL-TAKEOVER.json
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=aios
|
||||
Group=aios
|
||||
WorkingDirectory=/opt/aios-v2r1c2/active-runtime/app
|
||||
EnvironmentFile=/etc/aios-v2/aios-v2r1c2-production.env
|
||||
Environment=PYTHONUNBUFFERED=1
|
||||
Environment=PYTHONDONTWRITEBYTECODE=1
|
||||
ExecStartPre=/usr/bin/test "$AIOS_V2_PORT" = "18920"
|
||||
ExecStartPre=/usr/bin/test "$AIOS_RELEASE_SOURCE_OID" = "3dd622b5a515f1fa4ef72b9f8ae9ce48ac703f24"
|
||||
ExecStartPre=/usr/bin/test "$AIOS_RELEASE_TAG" = "V2R1C2"
|
||||
ExecStart=/usr/bin/python3 -m aios_web.release_application
|
||||
Restart=on-failure
|
||||
RestartSec=3
|
||||
TimeoutStartSec=60
|
||||
TimeoutStopSec=30
|
||||
KillSignal=SIGTERM
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectControlGroups=true
|
||||
RestrictSUIDSGID=true
|
||||
RestrictRealtime=true
|
||||
LockPersonality=true
|
||||
MemoryDenyWriteExecute=true
|
||||
ReadWritePaths=/var/lib/aios-v2r1c2-production
|
||||
StateDirectory=aios-v2r1c2-production
|
||||
StateDirectoryMode=0750
|
||||
UMask=0027
|
||||
LimitNOFILE=65536
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -11,8 +11,6 @@ Environment=AIOS_GITOPS_WORKSPACE=/data/aios-gitops/workspace
|
||||
Environment=AIOS_GITOPS_DB=/data/aios-gitops/runtime/gitops_model.sqlite3
|
||||
Environment=AIOS_GITOPS_TOKEN_FILE=/data/aios-gitops/runtime/secrets/gitops.tokens
|
||||
Environment=AIOS_GITOPS_BARE_REPO=/data/aios-gitops/runtime/gitea-bin/data/repositories/aios/aios-v23.git
|
||||
Environment=AIOS_GITOPS_AUTHORITATIVE_GIT_DIR=/data/aios-gitops/repo.git
|
||||
Environment=AIOS_GITOPS_REPOSITORY_IDENTITY=aios/aios
|
||||
Environment=AIOS_GITOPS_HOST=127.0.0.1
|
||||
Environment=AIOS_GITOPS_PORT=18890
|
||||
ExecStart=/usr/bin/python3 /data/aios-gitops/workspace/active-runtime/app/aios_chatgpt_actions_api.py
|
||||
|
||||
@@ -1,25 +0,0 @@
|
||||
[Unit]
|
||||
Description=AIOS Silent Release Shadow Controller
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=aiman
|
||||
Group=gitops
|
||||
UMask=0077
|
||||
Environment=AIOS_GITOPS_WRITES_ENABLED=false
|
||||
Environment=AIOS_SILENT_RELEASE_EXECUTION_ENABLED=false
|
||||
Environment=PYTHONDONTWRITEBYTECODE=1
|
||||
ExecStart=/usr/bin/python3 /usr/libexec/aios-silent-release-shadow/current/active-runtime/app/silent_release_execution.py --state-root /var/lib/aios-silent-release-shadow --repository /data/aios-gitops/repo.git --host 127.0.0.1 --port 18891 --poll-seconds 30
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ReadWritePaths=/var/lib/aios-silent-release-shadow
|
||||
ReadOnlyPaths=/data/aios-gitops/repo.git /usr/libexec/aios-silent-release-shadow
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,243 @@
|
||||
# AIOS V2 用户需求规格
|
||||
|
||||
- 文档版本:`V2R1C0-candidate`
|
||||
- 状态:候选版本需求基线
|
||||
- 适用范围:AIOS Control Plane、Agent Router、OpenClaw Adapter、Hermes Runtime、Web Console、GitOps、审计与部署
|
||||
|
||||
## 1. 背景与目标
|
||||
|
||||
现有系统已经具备 OpenClaw、Hermes、Agent Router、GitOps、局部 WebUI、审计和多租户候选能力。AIOS V2 的目标不是推倒重写,而是将这些分散能力收敛到统一、可审计、可 Web 化、可多租户隔离的控制面。
|
||||
|
||||
AIOS V2 必须成为唯一控制面,负责租户、身份、Agent、策略、任务生命周期、审批、审计、Runtime 目录和执行证据。OpenClaw 与 Hermes 只承担执行职责,不得持有第二套控制面状态。
|
||||
|
||||
## 2. 用户角色
|
||||
|
||||
| 角色 | 主要诉求 |
|
||||
|---|---|
|
||||
| 平台管理员 | 管理租户、运行时、发布、GitOps 状态和全局策略 |
|
||||
| 租户管理员 | 管理本租户 Agent、任务、审批、审计和成员权限 |
|
||||
| 操作员 | 创建任务、发起执行、取消任务、查看运行状态 |
|
||||
| 审批人 | 审批高风险任务,查看策略判定依据 |
|
||||
| 审计员 | 查询本租户完整审计链和执行证据 |
|
||||
| Runtime Worker | 领取已批准任务、心跳、返回执行结果 |
|
||||
| 开发与运维人员 | 通过 GitOps 提交、测试、部署、回滚和修复 Bug |
|
||||
|
||||
## 3. 核心用户需求
|
||||
|
||||
### UR-001 唯一控制面
|
||||
|
||||
所有任务创建、审批、取消、状态变化、策略判定和审计必须由 AIOS Control Plane 统一处理。
|
||||
|
||||
验收:
|
||||
|
||||
- Web Console、API、Codex、OpenClaw 和 Hermes 均不得绕过 AIOS 创建或修改任务生命周期。
|
||||
- OpenClaw/Hermes 的公开接口仅限执行、状态、取消、心跳和结果回传。
|
||||
|
||||
### UR-002 复用现有能力
|
||||
|
||||
系统不得重写已经稳定的 OpenClaw 执行能力、Hermes Runtime、Agent Router 或 GitOps 基础设施。
|
||||
|
||||
验收:
|
||||
|
||||
- 通过 Adapter/Boundary 接入现有能力。
|
||||
- 替换控制逻辑时保留原执行接口和可迁移数据。
|
||||
|
||||
### UR-003 OpenClaw 与 Hermes 解耦
|
||||
|
||||
OpenClaw 与 Hermes 必须从策略、租户、审批和任务生命周期中逐步解耦。
|
||||
|
||||
验收:
|
||||
|
||||
- OpenClaw 仅接收已签名、已批准、带租户上下文的命令。
|
||||
- Hermes 仅接收已批准的租户分区 Job。
|
||||
- 二者均不得直接写 Control Plane 权威状态。
|
||||
|
||||
### UR-004 全面 Web 化
|
||||
|
||||
所有面向用户和运维的能力必须通过 Web Console 或稳定 API 访问。
|
||||
|
||||
Web Console 至少展示:
|
||||
|
||||
- Tenant、Agent、Task、Approval、Runtime、Audit、Execution Evidence、GitOps Status。
|
||||
- 所有写操作由 AIOS API 完成,浏览器不得直接调用 OpenClaw/Hermes。
|
||||
|
||||
### UR-005 多租户隔离
|
||||
|
||||
所有模块必须支持租户隔离,且默认拒绝跨租户访问。
|
||||
|
||||
隔离范围:
|
||||
|
||||
- API 与会话
|
||||
- Tenant/Agent/Task 数据
|
||||
- OpenClaw 命令
|
||||
- Hermes Queue、Cell、Profile、Worker Lease
|
||||
- Memory、Cache、Artifact、SecretRef
|
||||
- Audit、Execution Evidence、日志和指标
|
||||
|
||||
验收:
|
||||
|
||||
- 资源记录的 `tenant_id` 不可为空。
|
||||
- 跨租户访问返回拒绝或“资源不存在”,不得泄露存在性。
|
||||
- 必须有跨租户负向测试。
|
||||
|
||||
### UR-006 权威任务生命周期
|
||||
|
||||
任务必须遵循统一状态机:
|
||||
|
||||
```text
|
||||
CREATED -> APPROVED -> RUNNING -> COMPLETED
|
||||
| | |
|
||||
+----------+----------+-> CANCELLED
|
||||
+-> FAILED
|
||||
```
|
||||
|
||||
验收:
|
||||
|
||||
- 禁止越级变化,例如 `CREATED -> COMPLETED`。
|
||||
- 每次变化包含 revision、principal、trace、时间和审计事件。
|
||||
|
||||
### UR-007 策略与审批
|
||||
|
||||
所有有副作用任务必须经过策略判定;高风险任务必须支持人工审批。
|
||||
|
||||
验收:
|
||||
|
||||
- Policy Decision 包含 allow/deny、reason 和 policy version。
|
||||
- 拒绝事件进入审计。
|
||||
- Runtime 不得自行放宽策略。
|
||||
|
||||
### UR-008 幂等与并发安全
|
||||
|
||||
任务和 Runtime 执行必须支持幂等与乐观并发控制。
|
||||
|
||||
验收:
|
||||
|
||||
- 相同租户与幂等键、相同内容返回同一任务/执行。
|
||||
- 相同幂等键不同内容返回冲突。
|
||||
- revision 或 branch lease 冲突不得静默覆盖。
|
||||
|
||||
### UR-009 可观测与执行证据
|
||||
|
||||
平台必须能够确认 Codex、OpenClaw、Hermes 等执行器是否领取任务、当前进度、最近提交、测试结果和完成状态。
|
||||
|
||||
验收:
|
||||
|
||||
- 存在 claim、heartbeat、progress、result 和 evidence 记录。
|
||||
- Web/API 可按租户查询。
|
||||
- 执行证据不得包含明文密钥。
|
||||
|
||||
### UR-010 GitOps 唯一变更流程
|
||||
|
||||
所有代码、配置、文档和部署资产修改必须进入非 `main` 分支,通过 commit、测试、审查和受控提升。
|
||||
|
||||
验收:
|
||||
|
||||
- Controller worktree 必须 detached 且 clean。
|
||||
- 本地 bare repo 与 Gitea 目标分支 OID 一致。
|
||||
- 不允许直接 push `main`、直接创建生产 Tag 或绕过审批部署。
|
||||
|
||||
### UR-011 持久化与恢复
|
||||
|
||||
租户、Agent、Task、Audit、Idempotency 和 Execution Evidence 必须在服务重启后恢复。
|
||||
|
||||
验收:
|
||||
|
||||
- 候选版支持 SQLite WAL 原子快照。
|
||||
- 有 restart recovery 测试。
|
||||
- 生产版需进一步提供迁移、备份、恢复、HA 和容量证据。
|
||||
|
||||
### UR-012 审计与合规
|
||||
|
||||
所有控制与执行动作必须生成不可混租户的审计事件。
|
||||
|
||||
至少记录:
|
||||
|
||||
- tenant、principal、trace、action、resource、outcome、policy version、timestamp。
|
||||
- OpenClaw/Hermes Runtime 生命周期。
|
||||
- GitOps、部署和回滚动作。
|
||||
|
||||
### UR-013 密钥安全
|
||||
|
||||
Git、日志、Web、任务 payload 和审计记录中不得出现明文 API Key、密码、Token 或私钥。
|
||||
|
||||
验收:
|
||||
|
||||
- Git 只存 `secret_ref`、`credential_ref`、`key_id` 或受保护文件路径。
|
||||
- Secret 文件权限为 `0600` 或更严格。
|
||||
- Secret Scanner 默认拒绝可疑提交。
|
||||
|
||||
### UR-014 部署与回滚
|
||||
|
||||
系统必须提供 systemd、Docker/Compose、健康检查和受保护回滚入口。
|
||||
|
||||
验收:
|
||||
|
||||
- 服务以非 root 权限运行。
|
||||
- 状态目录独立持久化。
|
||||
- 回滚前自动备份状态。
|
||||
- 默认只回滚代码,状态恢复需要显式授权。
|
||||
|
||||
### UR-015 文档与代码同步
|
||||
|
||||
任何影响用户行为、架构边界、API、数据模型、状态机、安全策略、部署或运维方式的变更,必须在同一分支同步更新 `docs/aios-v2/`。
|
||||
|
||||
验收:
|
||||
|
||||
- 架构/功能代码变化但无文档变化时,文档同步门禁失败。
|
||||
- Traceability Matrix 必须指向当前代码和测试。
|
||||
- 新架构决策必须增加或更新 ADR。
|
||||
|
||||
## 4. 非功能需求
|
||||
|
||||
### NFR-001 安全
|
||||
|
||||
- 默认拒绝、最小权限、租户隔离、身份可信链、SecretRef、审计。
|
||||
|
||||
### NFR-002 可靠性
|
||||
|
||||
- 幂等、revision、worker lease、heartbeat、重试边界、原子持久化、健康检查和回滚。
|
||||
|
||||
### NFR-003 可维护性
|
||||
|
||||
- 模块边界稳定,Control Plane 与 Runtime 解耦;代码、文档、测试和部署可追踪。
|
||||
|
||||
### NFR-004 可扩展性
|
||||
|
||||
- Store、Transport、Policy、EventPublisher、Runtime Adapter 均通过接口扩展。
|
||||
|
||||
### NFR-005 可运维性
|
||||
|
||||
- Web/API 状态、systemd/Docker、健康检查、日志、GitOps 状态、候选报告。
|
||||
|
||||
### NFR-006 性能候选指标
|
||||
|
||||
候选版上线前至少验证:
|
||||
|
||||
- API P95、任务调度延迟、队列等待、Worker 心跳、SQLite 锁等待、审计写入时间。
|
||||
- 具体 SLO 在预发布基准测试后固化,禁止无证据承诺。
|
||||
|
||||
## 5. 本候选版本范围
|
||||
|
||||
### 已纳入代码
|
||||
|
||||
- Control Plane、持久化、Gateway、Orchestrator、OpenClaw Adapter、Hermes Boundary、Execution Tracking、Web Console、GitOps Status、部署和回滚。
|
||||
|
||||
### 尚需环境验证
|
||||
|
||||
- 精确测试执行证据。
|
||||
- OpenClaw 真实 Runtime 联调。
|
||||
- 生产数据库/队列替换或容量验证。
|
||||
- 预发布性能、安全、重启和回滚演练。
|
||||
- Owner 对 merge、Tag 和生产部署的批准。
|
||||
|
||||
## 6. 用户验收场景
|
||||
|
||||
1. 平台管理员创建租户并注册 OpenClaw/Hermes Agent。
|
||||
2. 操作员创建任务,审批人批准。
|
||||
3. AIOS Orchestrator 选择 Runtime,并创建签名命令或租户分区 Job。
|
||||
4. Runtime 执行并返回结果。
|
||||
5. AIOS 更新 Task、Audit 和 Execution Evidence。
|
||||
6. 租户用户在 Web Console 查看完整链路。
|
||||
7. 跨租户访问被拒绝。
|
||||
8. 服务重启后状态恢复。
|
||||
9. 候选部署健康检查通过,回滚演练成功。
|
||||
@@ -0,0 +1,461 @@
|
||||
# AIOS V2 目标架构
|
||||
|
||||
- 文档版本:`V2R1C0-candidate`
|
||||
- 架构状态:目标架构已实现为候选代码,待预发布验证
|
||||
- 适用范围:Control Plane、Agent Router、OpenClaw、Hermes、Web Console、GitOps、审计、部署与运维
|
||||
|
||||
## 1. 架构目标
|
||||
|
||||
AIOS V2 的核心目标是把分散在 OpenClaw、Hermes、Agent Router、局部 WebUI 和 GitOps 中的控制能力收敛为一个权威控制面,同时保留现有执行能力,避免推倒重写。
|
||||
|
||||
目标状态:
|
||||
|
||||
```text
|
||||
用户 / Web Console / Public API
|
||||
|
|
||||
v
|
||||
AIOS Control Plane
|
||||
Tenant | Identity | Agent | Task | Policy | Approval | Audit
|
||||
|
|
||||
v
|
||||
Execution Orchestrator / Agent Router
|
||||
/ \
|
||||
v v
|
||||
OpenClaw Adapter Hermes Boundary
|
||||
| |
|
||||
v v
|
||||
OpenClaw Runtime Hermes Workers
|
||||
\ /
|
||||
v v
|
||||
Result Reconciliation + Execution Evidence
|
||||
```
|
||||
|
||||
设计结论:
|
||||
|
||||
- AIOS 是唯一控制面。
|
||||
- Agent Router 是调度平面,不是第二控制面。
|
||||
- OpenClaw 是执行适配与 Runtime。
|
||||
- Hermes 是队列、Worker、Profile 与执行 Runtime。
|
||||
- Web Console 只调用 AIOS API。
|
||||
- GitOps 是代码、配置、文档和发布变更的唯一入口。
|
||||
|
||||
## 2. 从现有架构到目标架构
|
||||
|
||||
### 2.1 现有能力
|
||||
|
||||
现有系统已经具备:
|
||||
|
||||
- OpenClaw 执行与 Fleet 能力。
|
||||
- Hermes Worker、Profile、队列和工作区能力。
|
||||
- Agent Router 路由经验和部分调度接口。
|
||||
- GitOps 分支、提交、审批、回滚和审计能力。
|
||||
- Tenant Context、Tenant Router、Memory Scope、SecretRef、Message Ledger 和审计候选实现。
|
||||
- 局部 WebUI、Gateway 和自动任务领取能力。
|
||||
|
||||
### 2.2 现有问题
|
||||
|
||||
- 控制逻辑分散,可能存在多个任务事实源。
|
||||
- OpenClaw/Hermes 内部仍可能保存租户、策略或任务状态。
|
||||
- 候选能力分散在多个分支,主线缺少权威实现。
|
||||
- WebUI 与 API 入口不统一。
|
||||
- 分支名称、代码、测试和部署证据之间缺少映射。
|
||||
- 多租户隔离多为逻辑约束,尚缺全链路生产证据。
|
||||
|
||||
### 2.3 演进原则
|
||||
|
||||
- 先加边界和适配器,再迁移控制权。
|
||||
- 先复用接口,再替换内部实现。
|
||||
- 先在候选分支验证,再受控提升。
|
||||
- 不允许同时存在两套权威任务生命周期。
|
||||
- 不以“大分支存在”代替验收证据。
|
||||
|
||||
## 3. 逻辑分层
|
||||
|
||||
### 3.1 Web 与 API 层
|
||||
|
||||
职责:
|
||||
|
||||
- 提供 Tenant、Agent、Task、Approval、Runtime、Audit、Execution Evidence 和 GitOps 页面/API。
|
||||
- 依赖可信身份代理注入 `X-AIOS-*` 身份头。
|
||||
- 不允许浏览器直接访问 OpenClaw 或 Hermes。
|
||||
- 不允许客户端自行选择或覆盖 `tenant_id`。
|
||||
|
||||
候选实现:
|
||||
|
||||
- `active-runtime/app/aios_v2/app.py`
|
||||
- `active-runtime/app/aios_v2/gateway.py`
|
||||
- `active-runtime/app/aios_v2/console.py`
|
||||
|
||||
### 3.2 AIOS Control Plane
|
||||
|
||||
权威职责:
|
||||
|
||||
- Tenant Registry
|
||||
- Agent Registry
|
||||
- Task Lifecycle
|
||||
- Policy Decision
|
||||
- Approval
|
||||
- Audit Event
|
||||
- Runtime Inventory
|
||||
- Execution Evidence
|
||||
- Deployment/GitOps Status
|
||||
|
||||
不负责:
|
||||
|
||||
- OpenClaw 内部执行细节。
|
||||
- Hermes Worker 内部执行细节。
|
||||
- 直接持有 Runtime 凭据明文。
|
||||
|
||||
候选实现:
|
||||
|
||||
- `active-runtime/app/aios_v2/control_plane.py`
|
||||
- `active-runtime/app/aios_v2/persistence.py`
|
||||
- `active-runtime/app/aios_v2/execution_tracking.py`
|
||||
- `active-runtime/app/aios_v2/gitops_status.py`
|
||||
|
||||
### 3.3 Execution Orchestrator / Agent Router
|
||||
|
||||
职责:
|
||||
|
||||
- 只接受 `APPROVED` 任务。
|
||||
- 根据 Agent Registry 和 Policy Result 选择 Runtime。
|
||||
- 生成 OpenClawCommand 或 HermesJob。
|
||||
- 传播 tenant、task、trace、idempotency 和 policy version。
|
||||
- 将 Runtime Result 回写 AIOS Task。
|
||||
|
||||
不负责:
|
||||
|
||||
- 创建租户。
|
||||
- 修改策略。
|
||||
- 绕过审批。
|
||||
- 保存第二套任务状态机。
|
||||
|
||||
候选实现:
|
||||
|
||||
- `active-runtime/app/aios_v2/orchestrator.py`
|
||||
|
||||
### 3.4 OpenClaw Adapter
|
||||
|
||||
职责:
|
||||
|
||||
- 验证签名命令。
|
||||
- 执行、查询、取消、健康检查和回调。
|
||||
- 支持幂等、超时和租户隔离。
|
||||
- 把 Runtime 生命周期转换为 AIOS 审计事件。
|
||||
|
||||
不负责:
|
||||
|
||||
- Tenant、Policy、Approval、Task Lifecycle。
|
||||
- 向浏览器暴露公网控制接口。
|
||||
|
||||
候选实现:
|
||||
|
||||
- `active-runtime/app/aios_openclaw/contract.py`
|
||||
- `active-runtime/app/aios_openclaw/adapter.py`
|
||||
- `active-runtime/app/aios_openclaw/transport.py`
|
||||
- `active-runtime/app/aios_openclaw/health.py`
|
||||
- `active-runtime/app/aios_openclaw/callback.py`
|
||||
|
||||
### 3.5 Hermes Runtime Boundary
|
||||
|
||||
职责:
|
||||
|
||||
- Tenant/Cell/Profile 分区队列。
|
||||
- Worker claim、start、heartbeat、complete、fail、cancel。
|
||||
- Lease 超时和自动回队列。
|
||||
- Result 回传和 Runtime Health。
|
||||
|
||||
不负责:
|
||||
|
||||
- Policy、Approval、Tenant Registry、权威 Task Lifecycle。
|
||||
|
||||
候选实现:
|
||||
|
||||
- `active-runtime/app/aios_hermes/contract.py`
|
||||
- `active-runtime/app/aios_hermes/adapter.py`
|
||||
- `active-runtime/app/aios_hermes/worker.py`
|
||||
|
||||
### 3.6 Persistence
|
||||
|
||||
候选实现使用 SQLite WAL 原子快照,持久化:
|
||||
|
||||
- Tenant
|
||||
- Agent
|
||||
- Task
|
||||
- Audit
|
||||
- Idempotency
|
||||
- Execution Evidence 文件
|
||||
|
||||
生产演进:
|
||||
|
||||
- Store 接口保持稳定。
|
||||
- 可替换为 PostgreSQL、对象存储或分布式事件存储。
|
||||
- 生产替换必须提供迁移、备份、恢复、HA、容量和租户隔离证据。
|
||||
|
||||
## 4. 多租户架构
|
||||
|
||||
### 4.1 权威租户上下文
|
||||
|
||||
每次请求和执行至少传播:
|
||||
|
||||
```text
|
||||
tenant_id
|
||||
principal_id
|
||||
session_id
|
||||
trace_id
|
||||
task_id
|
||||
policy_version
|
||||
```
|
||||
|
||||
Runtime 还传播:
|
||||
|
||||
```text
|
||||
command_id / job_id
|
||||
cell_id
|
||||
profile_id
|
||||
idempotency_key
|
||||
worker_id
|
||||
```
|
||||
|
||||
### 4.2 隔离层级
|
||||
|
||||
| 层级 | 隔离要求 |
|
||||
|---|---|
|
||||
| HTTP/API | 可信身份头、租户不可由浏览器覆盖 |
|
||||
| Control Plane | 所有资源带 `tenant_id`,跨租户默认拒绝 |
|
||||
| OpenClaw | 签名命令绑定 tenant/task/trace |
|
||||
| Hermes | tenant/cell/profile 三层分区 |
|
||||
| Storage | 候选为显式租户字段;生产需 RLS 或物理隔离策略 |
|
||||
| Evidence/Audit | 仅按当前租户查询 |
|
||||
| Cache/Memory | key 必须包含 tenant namespace |
|
||||
| Secret | 只保存引用,不保存明文 |
|
||||
| Logs/Metrics | 必须包含 tenant/task/trace 标签,并限制查询范围 |
|
||||
|
||||
### 4.3 失败行为
|
||||
|
||||
- 跨租户资源访问返回拒绝或“未找到”。
|
||||
- 不泄露其他租户资源存在性。
|
||||
- 缺少 tenant context 时 fail closed。
|
||||
- Runtime 返回的 tenant/task/trace 不匹配时拒绝回写。
|
||||
|
||||
## 5. 任务与执行数据流
|
||||
|
||||
### 5.1 任务创建与审批
|
||||
|
||||
```text
|
||||
User/API
|
||||
|
|
||||
v
|
||||
Trusted Identity
|
||||
|
|
||||
v
|
||||
AIOS Gateway
|
||||
|
|
||||
v
|
||||
Policy Gate
|
||||
|
|
||||
v
|
||||
Task CREATED
|
||||
|
|
||||
v
|
||||
Approval
|
||||
|
|
||||
v
|
||||
Task APPROVED
|
||||
```
|
||||
|
||||
### 5.2 OpenClaw 执行
|
||||
|
||||
```text
|
||||
APPROVED Task
|
||||
|
|
||||
v
|
||||
Execution Orchestrator
|
||||
|
|
||||
v
|
||||
Signed OpenClawCommand
|
||||
|
|
||||
v
|
||||
OpenClaw Adapter
|
||||
|
|
||||
v
|
||||
Runtime Result
|
||||
|
|
||||
v
|
||||
Task Reconciliation + Audit + Evidence
|
||||
```
|
||||
|
||||
### 5.3 Hermes 执行
|
||||
|
||||
```text
|
||||
APPROVED Task
|
||||
|
|
||||
v
|
||||
Execution Orchestrator
|
||||
|
|
||||
v
|
||||
Tenant/Cell/Profile HermesJob
|
||||
|
|
||||
v
|
||||
Worker Claim + Lease + Heartbeat
|
||||
|
|
||||
v
|
||||
HermesResult
|
||||
|
|
||||
v
|
||||
Task Reconciliation + Audit + Evidence
|
||||
```
|
||||
|
||||
## 6. 安全架构
|
||||
|
||||
### 6.1 身份边界
|
||||
|
||||
- Public API 位于可信反向代理之后。
|
||||
- 反向代理完成登录、会话、MFA 和身份头注入。
|
||||
- 候选应用拒绝未认证身份断言。
|
||||
- 生产不得允许客户端直接伪造 `X-AIOS-*`。
|
||||
|
||||
### 6.2 Runtime 命令完整性
|
||||
|
||||
OpenClawCommand 使用规范化序列化和 HMAC-SHA256:
|
||||
|
||||
- 签名覆盖 tenant、task、trace、action、payload、idempotency 和时间。
|
||||
- 支持 key ID、到期时间和时钟偏差。
|
||||
- Runtime 密钥通过受保护文件或外部 Secret Provider 解析。
|
||||
|
||||
### 6.3 Secret 边界
|
||||
|
||||
Git 允许:
|
||||
|
||||
```text
|
||||
secret_ref
|
||||
credential_ref
|
||||
key_id
|
||||
受保护文件路径
|
||||
```
|
||||
|
||||
Git 禁止:
|
||||
|
||||
```text
|
||||
API Key
|
||||
密码
|
||||
Token
|
||||
私钥
|
||||
签名材料
|
||||
```
|
||||
|
||||
### 6.4 Web 安全
|
||||
|
||||
候选应用设置:
|
||||
|
||||
- no-store
|
||||
- nosniff
|
||||
- DENY frame
|
||||
- no-referrer
|
||||
- CSP
|
||||
- 请求体大小限制
|
||||
|
||||
生产还需验证:
|
||||
|
||||
- CSRF/CORS
|
||||
- Rate Limit
|
||||
- Session Timeout
|
||||
- CSP Nonce/Hash
|
||||
- WAF 和反向代理访问策略
|
||||
|
||||
## 7. 可靠性架构
|
||||
|
||||
### 7.1 幂等
|
||||
|
||||
- Control Plane:`tenant_id + idempotency_key + request_digest`。
|
||||
- OpenClaw:`tenant_id + idempotency_key + signature`。
|
||||
- Hermes:`tenant_id + idempotency_key + job_digest`。
|
||||
|
||||
### 7.2 并发
|
||||
|
||||
- Task 使用 revision。
|
||||
- Hermes 使用 revision + worker lease。
|
||||
- GitOps 使用 detached controller + branch lease。
|
||||
- 持久化使用 SQLite `BEGIN IMMEDIATE` 和原子快照。
|
||||
|
||||
### 7.3 恢复
|
||||
|
||||
- Controller restart:detached guard。
|
||||
- AIOS restart:SQLite restore。
|
||||
- Worker crash:lease 到期后 Job 回到 QUEUED。
|
||||
- Release failure:健康检查失败自动恢复 previous release。
|
||||
|
||||
## 8. 可观测架构
|
||||
|
||||
统一关联键:
|
||||
|
||||
```text
|
||||
tenant_id
|
||||
principal_id
|
||||
trace_id
|
||||
task_id
|
||||
agent_id
|
||||
command_id / job_id
|
||||
worker_id
|
||||
commit
|
||||
```
|
||||
|
||||
Web Console 展示:
|
||||
|
||||
- Tasks
|
||||
- Agents
|
||||
- Runtime Health
|
||||
- Audit
|
||||
- Execution Evidence
|
||||
- GitOps Status
|
||||
|
||||
生产还需接入:
|
||||
|
||||
- 结构化日志
|
||||
- Metrics
|
||||
- Distributed Tracing
|
||||
- Alerting
|
||||
- SLO Dashboard
|
||||
|
||||
## 9. 部署架构
|
||||
|
||||
支持两种候选部署:
|
||||
|
||||
### systemd
|
||||
|
||||
- 非 root `aios` 用户。
|
||||
- `ProtectSystem=strict`、`NoNewPrivileges` 等安全选项。
|
||||
- 状态目录 `/var/lib/aios-v2`。
|
||||
|
||||
### Docker/Compose
|
||||
|
||||
- 非 root UID/GID。
|
||||
- Read-only root filesystem。
|
||||
- Drop all capabilities。
|
||||
- State volume。
|
||||
- HTTP healthcheck。
|
||||
|
||||
## 10. 架构约束
|
||||
|
||||
以下行为视为架构违规:
|
||||
|
||||
1. OpenClaw/Hermes 直接创建或批准 AIOS Task。
|
||||
2. Web Console 直接调用 Runtime。
|
||||
3. 新模块不携带 `tenant_id`。
|
||||
4. 新增第二套任务状态机或租户事实源。
|
||||
5. 在 Git 中提交明文凭据。
|
||||
6. 绕过 GitOps 修改生产代码/配置。
|
||||
7. 架构或功能变化未同步更新 `docs/aios-v2/`。
|
||||
8. 未通过测试和预发布门禁直接提升候选版本。
|
||||
|
||||
## 11. 目标架构完成判定
|
||||
|
||||
目标架构只有在以下证据齐全时才视为完成:
|
||||
|
||||
- 文档与代码映射完整。
|
||||
- Candidate tests 对精确 commit 执行通过。
|
||||
- OpenClaw/Hermes 实际 Runtime 联调通过。
|
||||
- 多租户负向测试通过。
|
||||
- 重启恢复和回滚演练通过。
|
||||
- GitOps 本地/Gitea OID 一致。
|
||||
- Owner 批准 merge、Tag 和生产部署。
|
||||
@@ -0,0 +1,321 @@
|
||||
# AIOS V2 迁移与交付策略
|
||||
|
||||
- 文档版本:`V2R1C0-candidate`
|
||||
- 交付原则:增量迁移、能力复用、控制权收敛、GitOps 唯一变更入口
|
||||
|
||||
## 1. 迁移目标
|
||||
|
||||
AIOS V2 不通过一次性重写替换现有系统,而是在保留 OpenClaw、Hermes、Agent Router、GitOps 和既有运行能力的前提下,将租户、策略、任务、审批、审计和发布治理收敛到 AIOS Control Plane。
|
||||
|
||||
迁移完成的标志不是“新代码存在”,而是:
|
||||
|
||||
- AIOS 成为唯一权威控制面;
|
||||
- OpenClaw/Hermes 不再保存第二套控制状态;
|
||||
- 所有控制功能 Web/API 化;
|
||||
- 所有模块通过租户隔离、测试和预发布门禁;
|
||||
- 本地 Git、Gitea、测试、部署和回滚证据可追踪。
|
||||
|
||||
## 2. 为什么采用增量迁移
|
||||
|
||||
### 2.1 避免能力回退
|
||||
|
||||
OpenClaw、Hermes 和 Agent Router 已经包含大量运行经验、边界处理和执行能力。整体重写会扩大风险面,并导致业务能力、操作流程和历史数据回退。
|
||||
|
||||
### 2.2 保持可回滚
|
||||
|
||||
每个迁移步骤必须:
|
||||
|
||||
- 独立分支;
|
||||
- 小范围提交;
|
||||
- 独立测试;
|
||||
- 可单独回滚;
|
||||
- 不要求立即关闭旧执行路径。
|
||||
|
||||
### 2.3 先收敛控制权,再优化内部实现
|
||||
|
||||
迁移优先级:
|
||||
|
||||
```text
|
||||
唯一入口
|
||||
-> 权威租户/任务/策略
|
||||
-> Runtime Adapter
|
||||
-> 数据层生产化
|
||||
-> Web 与可观测
|
||||
-> 性能与 HA
|
||||
```
|
||||
|
||||
## 3. 分阶段迁移路线
|
||||
|
||||
### Phase 0:GitOps 与架构事实源
|
||||
|
||||
目标:建立可靠变更链和权威设计基线。
|
||||
|
||||
已完成候选能力:
|
||||
|
||||
- Controller detached guard;
|
||||
- 避免 fetch 到已检出本地分支;
|
||||
- Gitea SSH 镜像;
|
||||
- `docs/aios-v2/` 架构事实源;
|
||||
- 候选分支、部署和回滚资产。
|
||||
|
||||
退出条件:
|
||||
|
||||
- Controller detached 且 clean;
|
||||
- 本地/Gitea OID 一致;
|
||||
- 架构与需求文档纳入版本。
|
||||
|
||||
### Phase 1:Control Plane 权威化
|
||||
|
||||
目标:将 Tenant、Agent、Task、Policy、Approval 和 Audit 收敛到 AIOS。
|
||||
|
||||
候选实现:
|
||||
|
||||
- `aios_v2/control_plane.py`
|
||||
- `aios_v2/persistence.py`
|
||||
- `aios_v2/gateway.py`
|
||||
- `aios_v2/app.py`
|
||||
|
||||
退出条件:
|
||||
|
||||
- 任务状态机统一;
|
||||
- 幂等、revision、审计和跨租户拒绝通过测试;
|
||||
- 服务重启后权威状态恢复。
|
||||
|
||||
### Phase 2:OpenClaw 解耦
|
||||
|
||||
目标:OpenClaw 从控制逻辑转为执行适配器。
|
||||
|
||||
步骤:
|
||||
|
||||
1. 定义签名命令合同;
|
||||
2. 引入 Adapter;
|
||||
3. 将直接调用迁移到 AIOS Orchestrator;
|
||||
4. Runtime 只返回执行结果和健康状态;
|
||||
5. 关闭或网络隔离旧公共控制入口。
|
||||
|
||||
退出条件:
|
||||
|
||||
- 未批准任务无法生成有效命令;
|
||||
- 命令绑定 tenant/task/trace/idempotency;
|
||||
- 旧控制接口冻结并记录替代路径。
|
||||
|
||||
### Phase 3:Hermes 解耦
|
||||
|
||||
目标:Hermes 只承担租户分区队列和 Worker Runtime。
|
||||
|
||||
步骤:
|
||||
|
||||
1. 定义 HermesJob/HermesResult;
|
||||
2. 强制 tenant/cell/profile 分区;
|
||||
3. 引入 claim/lease/heartbeat;
|
||||
4. 将结果统一回写 AIOS;
|
||||
5. 移除 Hermes 内部策略和任务事实源。
|
||||
|
||||
退出条件:
|
||||
|
||||
- Worker 无权创建 AIOS Task;
|
||||
- 租约过期后 Job 可恢复;
|
||||
- 跨租户 claim/read/result 被拒绝。
|
||||
|
||||
### Phase 4:Web 与运维统一
|
||||
|
||||
目标:所有用户与运维功能通过 AIOS Web/API。
|
||||
|
||||
范围:
|
||||
|
||||
- Tenant、Agent、Task、Runtime、Audit、Execution Evidence、GitOps;
|
||||
- systemd/Docker/Compose;
|
||||
- healthcheck、rollback、release report。
|
||||
|
||||
退出条件:
|
||||
|
||||
- 浏览器不直接调用 Runtime;
|
||||
- 生产身份代理和权限模型通过验证;
|
||||
- 部署、重启和回滚演练通过。
|
||||
|
||||
### Phase 5:生产数据层与 HA
|
||||
|
||||
目标:将候选 SQLite/内存运行状态替换或强化为生产级后端。
|
||||
|
||||
要求:
|
||||
|
||||
- 数据迁移;
|
||||
- RLS 或物理隔离;
|
||||
- 备份恢复;
|
||||
- HA 与容量;
|
||||
- 多实例并发;
|
||||
- 灾难恢复目标。
|
||||
|
||||
该阶段不得改变上层合同,避免 Runtime 和 Web 再次耦合存储实现。
|
||||
|
||||
## 4. 分支与提交策略
|
||||
|
||||
### 4.1 分支类型
|
||||
|
||||
| 类型 | 用途 |
|
||||
|---|---|
|
||||
| `task/*` | 可执行任务合同与主体实现 |
|
||||
| `patch/*` | 独立修复或窄范围增强 |
|
||||
| `integration/*` | 多模块候选收敛 |
|
||||
| `result/*` | 自动执行结果与证据 |
|
||||
| `release/*` | 受控发布候选 |
|
||||
|
||||
### 4.2 规则
|
||||
|
||||
- 禁止直接修改 `main`;
|
||||
- 一个提交应可独立审阅和回滚;
|
||||
- 大文件和大范围变更应拆分;
|
||||
- 所有行为变化必须同步文档和测试;
|
||||
- 生产 Tag、部署和回滚需要 Owner 批准。
|
||||
|
||||
## 5. GitOps 交付流程
|
||||
|
||||
```text
|
||||
需求/架构更新
|
||||
|
|
||||
v
|
||||
任务分支 + 文档 + 测试
|
||||
|
|
||||
v
|
||||
代码提交 + Secret Gate
|
||||
|
|
||||
v
|
||||
本地 bare repo 同步
|
||||
|
|
||||
v
|
||||
Gitea 镜像与 OID 校验
|
||||
|
|
||||
v
|
||||
Candidate 集成
|
||||
|
|
||||
v
|
||||
自动测试与预发布
|
||||
|
|
||||
v
|
||||
Owner 审批
|
||||
|
|
||||
v
|
||||
Merge / Tag / Production Deploy
|
||||
```
|
||||
|
||||
### 5.1 同步状态
|
||||
|
||||
GitOps 返回必须区分:
|
||||
|
||||
- local sync;
|
||||
- Gitea sync;
|
||||
- branch OID;
|
||||
- workspace mode;
|
||||
- dirty state。
|
||||
|
||||
仅返回 `common_git_dir synced` 不足以证明 Web 远端可见。
|
||||
|
||||
## 6. 发布门禁
|
||||
|
||||
### Gate A:设计完整性
|
||||
|
||||
- 用户需求、目标架构、模块契约、ADR 和追踪矩阵同步;
|
||||
- 无冲突标记、断链或过期路径。
|
||||
|
||||
### Gate B:代码质量
|
||||
|
||||
- 包可导入;
|
||||
- 静态编译通过;
|
||||
- Secret Scanner 通过;
|
||||
- 无已知越权旁路。
|
||||
|
||||
### Gate C:测试
|
||||
|
||||
- 单元、集成、多租户负向、重启恢复、Runtime 边界测试通过;
|
||||
- 记录 commit、命令、exit code、pass/fail 和 artifact digest。
|
||||
|
||||
### Gate D:预发布
|
||||
|
||||
- systemd 或 Compose 启动成功;
|
||||
- 健康检查通过;
|
||||
- OpenClaw/Hermes 真实联调通过;
|
||||
- 备份、重启和回滚演练通过;
|
||||
- 日志中无敏感信息。
|
||||
|
||||
### Gate E:生产批准
|
||||
|
||||
- Owner 批准 merge、Tag 和生产部署;
|
||||
- 发布窗口、回滚负责人和观察指标明确;
|
||||
- 生产变更通过 GitOps 记录。
|
||||
|
||||
## 7. 回滚策略
|
||||
|
||||
### 7.1 代码回滚
|
||||
|
||||
- 使用 `/opt/aios/current` 原子 symlink 切换;
|
||||
- 回滚前备份状态;
|
||||
- 健康检查失败自动恢复 previous release。
|
||||
|
||||
### 7.2 状态回滚
|
||||
|
||||
默认不自动回滚状态,避免覆盖候选运行期间新增的 Task、Audit 和 Evidence。
|
||||
|
||||
状态恢复只有在:
|
||||
|
||||
- 明确确认 schema 不兼容;
|
||||
- 数据备份可验证;
|
||||
- Owner 显式授权;
|
||||
|
||||
时启用。
|
||||
|
||||
### 7.3 Runtime 回滚
|
||||
|
||||
OpenClaw/Hermes Adapter 采用稳定合同。若候选执行层失败,可恢复旧 Runtime 版本,但不得恢复旧控制权旁路。
|
||||
|
||||
## 8. Bug 修复流程
|
||||
|
||||
生产或预发布 Bug:
|
||||
|
||||
1. 创建 `patch/bugfix/*`;
|
||||
2. 添加可复现测试;
|
||||
3. 修复代码;
|
||||
4. 更新相关需求/架构/契约/ADR;
|
||||
5. 运行候选验收;
|
||||
6. 进入新的 release candidate;
|
||||
7. Owner 批准部署。
|
||||
|
||||
禁止在生产目录直接编辑文件后不回写 Git。
|
||||
|
||||
## 9. 文档同步规则
|
||||
|
||||
以下变化必须更新文档:
|
||||
|
||||
- 用户行为或页面;
|
||||
- API 路径、Header、请求/响应;
|
||||
- 数据模型、状态机、策略;
|
||||
- OpenClaw/Hermes/Router 边界;
|
||||
- 多租户与安全模型;
|
||||
- 部署、健康检查、回滚;
|
||||
- 测试或发布门禁。
|
||||
|
||||
对应关系:
|
||||
|
||||
- 需求变化:更新 `01-USER-REQUIREMENTS.md`;
|
||||
- 架构变化:更新 `02-TARGET-ARCHITECTURE.md` 和 ADR;
|
||||
- 迁移/发布变化:更新本文件;
|
||||
- 合同变化:更新 `04-MODULE-CONTRACTS.md`;
|
||||
- 代码/测试路径变化:更新 `05-TRACEABILITY-MATRIX.md`。
|
||||
|
||||
## 10. 当前候选后续步骤
|
||||
|
||||
1. 完成文档同步门禁和 Acceptance Runner;
|
||||
2. 在精确 Candidate commit 执行测试;
|
||||
3. 推送 Gitea 并校验 OID;
|
||||
4. 部署隔离预发布环境;
|
||||
5. 联调真实 OpenClaw/Hermes;
|
||||
6. 执行重启、性能、安全和回滚演练;
|
||||
7. 生成最终 evidence;
|
||||
8. 请求 Owner 的 merge/Tag/production approval。
|
||||
|
||||
## 11. Controller GitOps 健康语义
|
||||
|
||||
Controller 必须从运行配置读取 `authoritative_repo`、`remote_name`、`repository_path`、`candidate_branch` 和 `task_branches`。健康检查只比较这些受控分支在权威 Bare Repo 与 Gitea 的实际 OID,并分别输出 `local_sync`、`gitea_sync`、`oid_match`、`branch_sync` 和 `sync_status`。
|
||||
|
||||
`sync_status=fully_synced` 仅表示所有受控分支的本地与 Gitea OID 一致;本地存在但 Gitea 缺失或滞后时必须报告 `local_only` 或 `gitea_pending`,认证或网络失败时报告 `gitea_failed`。不得把 Bare Repo 已更新解释为完全同步。
|
||||
|
||||
Controller 的 Gitea 只读凭据必须通过 systemd `LoadCredential` 或等价 Secret Provider 交付,禁止继续写入 world-readable `EnvironmentFile`。凭据仅授予 `read:repository`,状态和日志不得包含其值。
|
||||
@@ -0,0 +1,502 @@
|
||||
# AIOS V2 模块契约
|
||||
|
||||
- 文档版本:`V2R1C0-candidate`
|
||||
- 状态:候选实现契约基线
|
||||
- 原则:Control Plane 权威、Runtime 执行、全链路租户隔离、所有状态可审计
|
||||
|
||||
## 1. 通用上下文契约
|
||||
|
||||
所有控制与执行请求必须携带以下上下文:
|
||||
|
||||
| 字段 | 必填 | 说明 |
|
||||
|---|---:|---|
|
||||
| `tenant_id` | 是 | 权威租户标识 |
|
||||
| `principal_id` | 控制请求是 | 已认证主体 |
|
||||
| `trace_id` | 是 | 全链路追踪 |
|
||||
| `task_id` | 执行阶段是 | AIOS 权威任务 |
|
||||
| `session_id` | Web/交互请求建议 | 用户会话 |
|
||||
| `policy_version` | 控制请求是 | 策略版本 |
|
||||
| `idempotency_key` | 创建/执行是 | 幂等键 |
|
||||
|
||||
规则:
|
||||
|
||||
- 缺少必填字段时 fail closed。
|
||||
- 客户端不得覆盖由可信代理或 AIOS 生成的 `tenant_id`。
|
||||
- Runtime 返回的 tenant/task/trace 必须与请求完全匹配。
|
||||
- 所有持久化键、缓存键和队列分区必须包含租户命名空间。
|
||||
|
||||
## 2. Tenant Contract
|
||||
|
||||
### 2.1 TenantRecord
|
||||
|
||||
```text
|
||||
tenant_id
|
||||
display_name
|
||||
status
|
||||
metadata
|
||||
created_at
|
||||
updated_at
|
||||
```
|
||||
|
||||
约束:
|
||||
|
||||
- `tenant_id` 不可为空且不可变。
|
||||
- `status` 至少支持 `active` 和受控停用状态。
|
||||
- Tenant 只由 AIOS Control Plane 创建和维护。
|
||||
- Runtime 不得创建租户。
|
||||
|
||||
实现:`active-runtime/app/aios_v2/control_plane.py`
|
||||
|
||||
## 3. Agent Contract
|
||||
|
||||
### 3.1 AgentRecord
|
||||
|
||||
```text
|
||||
agent_id
|
||||
tenant_id
|
||||
display_name
|
||||
runtime
|
||||
capabilities
|
||||
enabled
|
||||
metadata
|
||||
created_at
|
||||
updated_at
|
||||
```
|
||||
|
||||
约束:
|
||||
|
||||
- Agent 属于唯一租户。
|
||||
- `runtime` 由 Registry 决定,可取 `openclaw`、`hermes` 或后续注册 Runtime。
|
||||
- `capabilities` 是允许调度的动作集合。
|
||||
- `metadata` 可存 `cell_id`、`profile_id`、priority 等非敏感引用。
|
||||
- 不得在 Agent metadata 中保存明文凭据。
|
||||
|
||||
## 4. Task Contract
|
||||
|
||||
### 4.1 TaskRecord
|
||||
|
||||
```text
|
||||
task_id
|
||||
tenant_id
|
||||
agent_id
|
||||
action
|
||||
payload
|
||||
state
|
||||
revision
|
||||
idempotency_key
|
||||
request_digest
|
||||
result
|
||||
error
|
||||
created_by
|
||||
approved_by
|
||||
created_at
|
||||
updated_at
|
||||
```
|
||||
|
||||
### 4.2 状态机
|
||||
|
||||
```text
|
||||
CREATED -> APPROVED -> RUNNING -> COMPLETED
|
||||
| | |
|
||||
+----------+----------+-> CANCELLED
|
||||
+-> FAILED
|
||||
```
|
||||
|
||||
允许转换:
|
||||
|
||||
| 当前 | 允许目标 |
|
||||
|---|---|
|
||||
| CREATED | APPROVED, CANCELLED |
|
||||
| APPROVED | RUNNING, CANCELLED |
|
||||
| RUNNING | COMPLETED, FAILED, CANCELLED |
|
||||
| COMPLETED | 无 |
|
||||
| FAILED | 无 |
|
||||
| CANCELLED | 无 |
|
||||
|
||||
约束:
|
||||
|
||||
- 每次转换必须校验 `revision`。
|
||||
- COMPLETED 必须带 `result`。
|
||||
- FAILED 必须带 `error`。
|
||||
- 每次转换生成 AuditEvent。
|
||||
- Runtime 不得直接写 Task Store,只能通过 Orchestrator reconcile。
|
||||
|
||||
## 5. Policy Contract
|
||||
|
||||
### 5.1 PolicyDecision
|
||||
|
||||
```text
|
||||
allowed
|
||||
reason
|
||||
policy_version
|
||||
```
|
||||
|
||||
约束:
|
||||
|
||||
- Policy 默认拒绝。
|
||||
- Runtime 不得覆盖 PolicyDecision。
|
||||
- 拒绝必须审计。
|
||||
- 高风险动作必须经过 Approval。
|
||||
|
||||
## 6. Audit Contract
|
||||
|
||||
### 6.1 AuditEvent
|
||||
|
||||
```text
|
||||
event_id
|
||||
tenant_id
|
||||
principal_id
|
||||
trace_id
|
||||
action
|
||||
resource_type
|
||||
resource_id
|
||||
outcome
|
||||
policy_version
|
||||
details
|
||||
created_at
|
||||
```
|
||||
|
||||
约束:
|
||||
|
||||
- 按租户查询。
|
||||
- `details` 不得包含明文凭据。
|
||||
- OpenClaw/Hermes 生命周期事件必须映射到统一 AuditEvent。
|
||||
- 生产版需增加不可变外部固化和完整性验证。
|
||||
|
||||
## 7. OpenClaw Contract
|
||||
|
||||
### 7.1 OpenClawCommand
|
||||
|
||||
```text
|
||||
contract_version
|
||||
command_id
|
||||
tenant_id
|
||||
task_id
|
||||
trace_id
|
||||
action
|
||||
payload
|
||||
idempotency_key
|
||||
created_at
|
||||
expires_at
|
||||
key_id
|
||||
signature
|
||||
```
|
||||
|
||||
签名覆盖:
|
||||
|
||||
- command_id
|
||||
- tenant_id
|
||||
- task_id
|
||||
- trace_id
|
||||
- action
|
||||
- payload
|
||||
- idempotency_key
|
||||
- created_at / expires_at
|
||||
- contract_version / key_id
|
||||
|
||||
约束:
|
||||
|
||||
- 使用规范化 JSON 和 HMAC-SHA256。
|
||||
- 只接受已批准任务生成的命令。
|
||||
- 过期、未来时间超差、签名错误命令必须拒绝。
|
||||
- `key_id` 只引用运行时材料,不包含材料本身。
|
||||
|
||||
### 7.2 OpenClawResult
|
||||
|
||||
```text
|
||||
command_id
|
||||
tenant_id
|
||||
task_id
|
||||
trace_id
|
||||
state
|
||||
output
|
||||
error
|
||||
started_at
|
||||
finished_at
|
||||
runtime_id
|
||||
attempts
|
||||
```
|
||||
|
||||
状态:
|
||||
|
||||
```text
|
||||
ACCEPTED
|
||||
RUNNING
|
||||
COMPLETED
|
||||
FAILED
|
||||
CANCELLED
|
||||
TIMED_OUT
|
||||
REJECTED
|
||||
```
|
||||
|
||||
约束:
|
||||
|
||||
- Adapter 对外只接受终态结果用于 reconcile。
|
||||
- 身份字段不匹配时拒绝回写。
|
||||
- 结果通过 Orchestrator 更新 AIOS Task。
|
||||
|
||||
### 7.3 RuntimeHealth
|
||||
|
||||
```text
|
||||
runtime
|
||||
runtime_id
|
||||
status
|
||||
version
|
||||
heartbeat_at
|
||||
active_tasks
|
||||
workspace_isolated
|
||||
credential_ref_configured
|
||||
details
|
||||
```
|
||||
|
||||
健康策略:
|
||||
|
||||
- heartbeat 不得超时。
|
||||
- workspace isolation 与 credential reference 必须符合策略。
|
||||
- 不安全 Runtime 应 quarantine,不得继续接收新任务。
|
||||
|
||||
实现:`active-runtime/app/aios_openclaw/`
|
||||
|
||||
## 8. Hermes Contract
|
||||
|
||||
### 8.1 HermesJob
|
||||
|
||||
```text
|
||||
job_id
|
||||
tenant_id
|
||||
cell_id
|
||||
profile_id
|
||||
task_id
|
||||
trace_id
|
||||
action
|
||||
payload
|
||||
idempotency_key
|
||||
priority
|
||||
state
|
||||
revision
|
||||
assigned_worker_id
|
||||
lease_expires_at
|
||||
created_at
|
||||
updated_at
|
||||
```
|
||||
|
||||
状态机:
|
||||
|
||||
```text
|
||||
QUEUED -> CLAIMED -> RUNNING -> COMPLETED
|
||||
| | |
|
||||
+---------+----------+-> CANCELLED
|
||||
+-> FAILED
|
||||
```
|
||||
|
||||
约束:
|
||||
|
||||
- `tenant_id + cell_id + profile_id` 形成执行分区。
|
||||
- claim 只能领取同分区 Job。
|
||||
- Worker ID 不得跨分区复用。
|
||||
- Lease 到期 Job 回到 QUEUED。
|
||||
- revision 冲突不得覆盖。
|
||||
|
||||
### 8.2 HermesResult
|
||||
|
||||
```text
|
||||
job_id
|
||||
tenant_id
|
||||
task_id
|
||||
trace_id
|
||||
state
|
||||
worker_id
|
||||
output
|
||||
error
|
||||
started_at
|
||||
finished_at
|
||||
```
|
||||
|
||||
约束:
|
||||
|
||||
- COMPLETED 必须有 output。
|
||||
- FAILED 必须有 error。
|
||||
- 只允许 AIOS Orchestrator reconcile。
|
||||
|
||||
### 8.3 HermesHealth
|
||||
|
||||
```text
|
||||
runtime_id
|
||||
status
|
||||
version
|
||||
heartbeat_at
|
||||
active_workers
|
||||
active_jobs
|
||||
queued_jobs
|
||||
tenant_partitions
|
||||
profile_isolation_enforced
|
||||
details
|
||||
```
|
||||
|
||||
实现:`active-runtime/app/aios_hermes/`
|
||||
|
||||
## 9. Execution Evidence Contract
|
||||
|
||||
### 9.1 ExecutionEvidence
|
||||
|
||||
```text
|
||||
execution_id
|
||||
tenant_id
|
||||
task_id
|
||||
agent_id
|
||||
executor
|
||||
branch
|
||||
status
|
||||
progress_completed
|
||||
progress_total
|
||||
current_step
|
||||
last_commit
|
||||
test_status
|
||||
test_summary
|
||||
result_summary
|
||||
claimed_at
|
||||
heartbeat_at
|
||||
completed_at
|
||||
```
|
||||
|
||||
状态建议:
|
||||
|
||||
```text
|
||||
claimed
|
||||
running
|
||||
blocked
|
||||
complete
|
||||
failed
|
||||
cancelled
|
||||
```
|
||||
|
||||
约束:
|
||||
|
||||
- 按租户存储和查询。
|
||||
- 每个阻塞必须记录原因、责任方和解除条件。
|
||||
- 每次结果必须记录 branch、commit、测试命令和状态。
|
||||
- 不得包含 Secret 或完整敏感 payload。
|
||||
|
||||
## 10. GitOps Status Contract
|
||||
|
||||
```text
|
||||
available
|
||||
service_status
|
||||
workspace_mode
|
||||
workspace_branch
|
||||
workspace_ref
|
||||
workspace_dirty
|
||||
candidate_branch
|
||||
release_channel
|
||||
release_version
|
||||
sync_status
|
||||
details
|
||||
```
|
||||
|
||||
约束:
|
||||
|
||||
- Web 响应不得包含 remote URL、凭据或 stderr。
|
||||
- Controller 必须 `detached + clean`。
|
||||
- `sync_status` 必须区分 local 和 Gitea 镜像状态。
|
||||
|
||||
## 11. Web/API Contract
|
||||
|
||||
### 11.1 身份头
|
||||
|
||||
可信代理注入:
|
||||
|
||||
```text
|
||||
X-AIOS-Authenticated
|
||||
X-AIOS-Tenant-ID
|
||||
X-AIOS-Principal-ID
|
||||
X-AIOS-Roles
|
||||
X-AIOS-Trace-ID
|
||||
X-AIOS-Session-ID
|
||||
X-AIOS-Policy-Version
|
||||
```
|
||||
|
||||
生产约束:
|
||||
|
||||
- 代理必须删除客户端同名头后重新注入。
|
||||
- 应用不得直接暴露公网端口。
|
||||
|
||||
### 11.2 核心 API
|
||||
|
||||
```text
|
||||
GET /api/v2/health
|
||||
POST /api/v2/tenants
|
||||
GET /api/v2/tenants/current
|
||||
POST /api/v2/agents
|
||||
GET /api/v2/agents
|
||||
POST /api/v2/tasks
|
||||
GET /api/v2/tasks
|
||||
GET /api/v2/tasks/{id}
|
||||
POST /api/v2/tasks/{id}/approve
|
||||
POST /api/v2/tasks/{id}/start
|
||||
POST /api/v2/tasks/{id}/complete
|
||||
POST /api/v2/tasks/{id}/fail
|
||||
POST /api/v2/tasks/{id}/cancel
|
||||
POST /api/v2/tasks/{id}/dispatch
|
||||
GET /api/v2/audit
|
||||
GET /api/v2/runtime
|
||||
GET /api/v2/executions
|
||||
GET /api/v2/gitops
|
||||
POST /api/v2/hermes/results/reconcile
|
||||
```
|
||||
|
||||
## 12. Persistence Contract
|
||||
|
||||
候选版:
|
||||
|
||||
- SQLite WAL
|
||||
- `BEGIN IMMEDIATE`
|
||||
- 原子 snapshot
|
||||
- schema version
|
||||
- restart restore
|
||||
|
||||
持久化范围:
|
||||
|
||||
- tenants
|
||||
- agents
|
||||
- tasks
|
||||
- audit
|
||||
- idempotency
|
||||
|
||||
生产替换要求:
|
||||
|
||||
- 不改变 Domain/API Contract。
|
||||
- 提供 migration、backup、restore、HA、RLS/隔离和容量测试。
|
||||
|
||||
## 13. Deployment Contract
|
||||
|
||||
### systemd
|
||||
|
||||
- 非 root `aios` 用户。
|
||||
- 状态目录 `/var/lib/aios-v2`。
|
||||
- 最小权限和文件系统保护。
|
||||
|
||||
### Docker
|
||||
|
||||
- 非 root UID/GID。
|
||||
- read-only root filesystem。
|
||||
- drop all capabilities。
|
||||
- 独立状态 volume。
|
||||
|
||||
### Rollback
|
||||
|
||||
- 回滚前备份状态。
|
||||
- 默认只回滚代码。
|
||||
- 状态恢复需显式授权。
|
||||
- 回滚后必须执行 healthcheck。
|
||||
|
||||
## 14. 契约变更规则
|
||||
|
||||
任何以下变化必须在同一提交或同一分支更新本文档:
|
||||
|
||||
- 字段、状态、Header、API、错误语义;
|
||||
- 租户或安全约束;
|
||||
- OpenClaw/Hermes 边界;
|
||||
- Persistence 或 Deployment Contract;
|
||||
- Evidence/Audit 格式。
|
||||
|
||||
不兼容变化必须新增 ADR 和版本号,不得静默修改。
|
||||
@@ -0,0 +1,122 @@
|
||||
# AIOS V2 需求—架构—代码—测试—部署追踪矩阵
|
||||
|
||||
- 文档版本:`V2R1C0-candidate`
|
||||
- 目的:确保每项用户需求都能追踪到架构设计、代码实现、自动测试、部署资产和验收命令
|
||||
|
||||
## 1. 追踪规则
|
||||
|
||||
每条需求至少具备:
|
||||
|
||||
1. 需求编号;
|
||||
2. 架构章节;
|
||||
3. 实现路径;
|
||||
4. 测试路径;
|
||||
5. 部署/运维资产;
|
||||
6. 验收状态。
|
||||
|
||||
状态定义:
|
||||
|
||||
- `Implemented`:代码已进入 Candidate 分支;
|
||||
- `Test-defined`:测试代码已进入 Candidate 分支;
|
||||
- `Test-verified`:精确 commit 的测试已运行并保存证据;
|
||||
- `Pre-production verified`:预发布部署、健康检查和回滚演练已通过;
|
||||
- `Production released`:Owner 批准并完成 merge/tag/deploy。
|
||||
|
||||
## 2. 核心追踪矩阵
|
||||
|
||||
| 需求 | 设计依据 | 代码实现 | 测试 | 部署/运维 | 当前状态 |
|
||||
|---|---|---|---|---|---|
|
||||
| UR-001 AIOS 唯一控制面 | `02-TARGET-ARCHITECTURE.md` §3.2/§3.3;ADR-001 | `aios_v2/control_plane.py`、`aios_v2/orchestrator.py`、`aios_v2/app.py` | `tests/aios_v2_candidate/test_candidate_flow.py` | `deploy/aios-v2/systemd/aios-v2.service` | Implemented / Test-defined |
|
||||
| UR-002 复用现有能力 | `02-TARGET-ARCHITECTURE.md` §2;`03-MIGRATION-AND-DELIVERY.md` §2 | `aios_openclaw/`、`aios_hermes/` | OpenClaw/Hermes flow tests | Candidate branch + adapter boundaries | Implemented / Test-defined |
|
||||
| UR-003 OpenClaw/Hermes 解耦 | `02-TARGET-ARCHITECTURE.md` §3.4/§3.5;ADR-002 | `aios_openclaw/adapter.py`、`aios_hermes/adapter.py` | runtime boundary tests in candidate flow suite | systemd/Docker runtime configuration | Implemented / Test-defined |
|
||||
| UR-004 全面 Web 化 | `02-TARGET-ARCHITECTURE.md` §3.1/§8 | `aios_v2/console.py`、`aios_v2/gateway.py`、`aios_v2/app.py` | API/console smoke test pending | Docker/systemd + reverse proxy requirement | Implemented / Test pending |
|
||||
| UR-005 多租户隔离 | `02-TARGET-ARCHITECTURE.md` §4;ADR-004 | `control_plane.py`、`orchestrator.py`、`aios_openclaw/*`、`aios_hermes/*`、`execution_tracking.py` | `test_cross_tenant_task_read_is_denied` | tenant-scoped state/evidence directories | Implemented / Test-defined |
|
||||
| UR-006 权威任务生命周期 | `04-MODULE-CONTRACTS.md` §4 | `control_plane.py` | OpenClaw/Hermes full-flow tests | persisted state + audit | Implemented / Test-defined |
|
||||
| UR-007 策略与审批 | `04-MODULE-CONTRACTS.md` §5 | `control_plane.py`、`gateway.py` | role/policy negative tests需补充 | reverse proxy identity + audit | Implemented / Partial test |
|
||||
| UR-008 幂等与并发安全 | `02-TARGET-ARCHITECTURE.md` §7 | `control_plane.py`、`aios_openclaw/adapter.py`、`aios_hermes/adapter.py`、`worker.py` | flow + restart tests;并发专项测试需补充 | GitOps detached guard + SQLite WAL | Implemented / Partial test |
|
||||
| UR-009 执行证据 | `04-MODULE-CONTRACTS.md` §9 | `execution_tracking.py` | `test_execution_tracking_persists_claim_heartbeat_and_result` | `/var/lib/aios-v2/executions` | Implemented / Test-defined |
|
||||
| UR-010 GitOps 唯一变更流程 | `03-MIGRATION-AND-DELIVERY.md` §4/§5;ADR-003 | GitOps Runtime、`gitops_status.py` | healthcheck 的 detached/clean 检查 | `deploy/aios-v2/healthcheck/check.sh` | Implemented / Runtime verified partially |
|
||||
| UR-011 持久化与恢复 | `02-TARGET-ARCHITECTURE.md` §3.6 | `persistence.py` | `test_durable_service_recovers_state_after_restart` | state dir + backup/rollback | Implemented / Test-defined |
|
||||
| UR-012 审计与合规 | `04-MODULE-CONTRACTS.md` §6 | `control_plane.py`、`aios_openclaw/callback.py`、`app.py` | flow test + audit assertions需补充 | Console audit view | Implemented / Partial test |
|
||||
| UR-013 密钥安全 | `02-TARGET-ARCHITECTURE.md` §6;ADR-006 | `FileSigningMaterialResolver`、credential refs、Secret Gate | permission/secret-scan test需补充 | env template + external protected file | Implemented / Partial test |
|
||||
| UR-014 部署与回滚 | `03-MIGRATION-AND-DELIVERY.md` §7 | deployment scripts/assets | acceptance runner + healthcheck | `deploy/aios-v2/` | Implemented / Pre-prod pending |
|
||||
| UR-015 文档与代码同步 | `03-MIGRATION-AND-DELIVERY.md` §9;ADR-005 | `tools/aios_v2_docs_code_check.py` | checker self-test / acceptance runner | CI/Gitea Actions 可调用 | Implemented after checker commit |
|
||||
|
||||
## 3. API 追踪
|
||||
|
||||
| API | 需求 | 实现 | 测试状态 |
|
||||
|---|---|---|---|
|
||||
| `GET /api/v2/health` | UR-004, UR-014 | `gateway.py` | healthcheck 定义完成 |
|
||||
| `POST /api/v2/tenants` | UR-001, UR-005 | `gateway.py` / `control_plane.py` | 需增加 HTTP 集成测试 |
|
||||
| `POST /api/v2/agents` | UR-001, UR-005 | `gateway.py` / `control_plane.py` | Domain flow 已覆盖 |
|
||||
| `POST /api/v2/tasks` | UR-006, UR-008 | `gateway.py` / `control_plane.py` | Domain flow 已覆盖 |
|
||||
| `POST /api/v2/tasks/{id}/approve` | UR-006, UR-007 | `gateway.py` | Domain flow 已覆盖 |
|
||||
| `POST /api/v2/tasks/{id}/dispatch` | UR-001, UR-003 | `app.py` / `orchestrator.py` | OpenClaw/Hermes flow 已覆盖 |
|
||||
| `GET /api/v2/runtime` | UR-004, UR-009 | `app.py` | HTTP smoke test待执行 |
|
||||
| `GET /api/v2/executions` | UR-009 | `app.py` / `execution_tracking.py` | Evidence persistence test已定义 |
|
||||
| `GET /api/v2/gitops` | UR-010 | `app.py` / `gitops_status.py` | Runtime/healthcheck验证待执行 |
|
||||
| `POST /api/v2/hermes/results/reconcile` | UR-003, UR-006 | `app.py` / `orchestrator.py` | Hermes flow已覆盖 Domain 路径 |
|
||||
|
||||
## 4. 运行时契约追踪
|
||||
|
||||
| 契约 | 设计 | 代码 | 测试 |
|
||||
|---|---|---|---|
|
||||
| OpenClawCommand | `04-MODULE-CONTRACTS.md` §7.1 | `aios_openclaw/contract.py` | OpenClaw full-flow;签名负向测试需补充 |
|
||||
| OpenClawResult | §7.2 | `aios_openclaw/contract.py` | OpenClaw full-flow |
|
||||
| RuntimeHealth | §7.3 | `aios_openclaw/contract.py`、`health.py` | health policy测试需补充 |
|
||||
| HermesJob | §8.1 | `aios_hermes/contract.py` | Hermes full-flow |
|
||||
| HermesResult | §8.2 | `aios_hermes/contract.py` | Hermes full-flow |
|
||||
| HermesHealth | §8.3 | `aios_hermes/contract.py`、`adapter.py` | healthcheck/worker test待执行 |
|
||||
| ExecutionEvidence | §9 | `execution_tracking.py` | evidence persistence test |
|
||||
| GitOpsStatus | §10 | `gitops_status.py` | healthcheck detached/clean检查 |
|
||||
|
||||
## 5. 部署追踪
|
||||
|
||||
| 能力 | 文件 | 验收命令 |
|
||||
|---|---|---|
|
||||
| systemd | `deploy/aios-v2/systemd/aios-v2.service` | `systemctl status aios-v2` |
|
||||
| 环境模板 | `deploy/aios-v2/env/aios-v2.env.example` | `systemd-analyze verify` + 文件权限检查 |
|
||||
| Docker | `deploy/aios-v2/docker/Dockerfile` | `docker build` |
|
||||
| Compose | `deploy/aios-v2/docker/compose.yaml` | `docker compose config` / `up` |
|
||||
| 健康检查 | `deploy/aios-v2/healthcheck/check.sh` | 直接执行 |
|
||||
| 回滚 | `deploy/aios-v2/rollback/rollback.sh` | 预发布回滚演练 |
|
||||
| 全量验收 | `deploy/aios-v2/acceptance/run-all.sh` | 直接执行 |
|
||||
|
||||
## 6. 文档覆盖追踪
|
||||
|
||||
| 文档 | 覆盖内容 | 代码/变更触发条件 |
|
||||
|---|---|---|
|
||||
| `README.md` | 事实源索引、状态语言、实现地图 | 新模块/目录/版本 |
|
||||
| `01-USER-REQUIREMENTS.md` | 用户需求、NFR、验收场景 | 用户行为、功能范围、验收变化 |
|
||||
| `02-TARGET-ARCHITECTURE.md` | 分层、边界、数据流、安全、租户 | 架构边界、数据流、运行时变化 |
|
||||
| `03-MIGRATION-AND-DELIVERY.md` | 迁移、GitOps、Gate、回滚 | 发布流程、迁移阶段、门禁变化 |
|
||||
| `04-MODULE-CONTRACTS.md` | 字段、API、状态机、错误语义 | Contract/API/模型变化 |
|
||||
| `05-TRACEABILITY-MATRIX.md` | 需求到代码测试部署映射 | 代码、测试、部署路径变化 |
|
||||
| `adr/*` | 关键不可逆或跨模块决策 | 新架构决策或旧决策替换 |
|
||||
|
||||
## 7. 当前未完成验证项
|
||||
|
||||
以下项目代码或测试定义已存在,但尚不能标记为 `Test-verified` 或 `Pre-production verified`:
|
||||
|
||||
1. 精确 Candidate commit 的测试执行结果;
|
||||
2. HTTP API 与 Web Console 冒烟测试;
|
||||
3. 真实 OpenClaw Runtime 签名、取消、超时和健康联调;
|
||||
4. Hermes 长任务心跳、Worker 崩溃和 Lease 恢复演练;
|
||||
5. 多实例 SQLite/并发容量边界;
|
||||
6. systemd、Docker、Compose、restart 和 rollback 演练;
|
||||
7. local/Gitea Candidate OID 一致证明;
|
||||
8. 生产身份代理、防伪 Header、CSRF/CORS/Rate Limit 验证;
|
||||
9. 文档—代码同步检查器在 CI/Acceptance 中执行通过。
|
||||
|
||||
## 8. 更新要求
|
||||
|
||||
任何代码、API、部署或测试路径变化,必须同步更新本矩阵。验收脚本会检查本文档引用的关键路径存在,并检查架构/功能代码变化是否同时包含 `docs/aios-v2/` 变化。
|
||||
|
||||
## 9. Controller 健康检查追踪
|
||||
|
||||
| 需求 | 实现 | 测试 |
|
||||
|---|---|---|
|
||||
| 配置化权威仓库、Remote、Gitea 仓库与分支集合 | `agents/scripts/autonomous_controller.py` | `tests/controller/test_autonomous_controller_health.py` |
|
||||
| 区分本地、Gitea 与 OID 一致性 | `h_branch_sync()` | OID 一致、本地较新、分支缺失、Remote 不可达、认证失败 |
|
||||
| 不在状态输出中暴露凭据 | `_gitea_branch_state()` 的固定错误码 | `test_secret_never_enters_status_output` |
|
||||
| 接受 detached 且 clean 的受控部署检出 | `h_controller_checkout()` | `test_controller_detached_and_clean` |
|
||||
@@ -0,0 +1,278 @@
|
||||
# AIOS V2 文档治理规范
|
||||
|
||||
- 文档版本:`V2R1C0-candidate`
|
||||
- 适用范围:`docs/aios-v2/`、架构决策记录、生产候选报告、在线任务合同、部署与验收说明
|
||||
- 状态:已采纳
|
||||
|
||||
## 1. 目的
|
||||
|
||||
本规范用于保证 AIOS V2 的需求、架构、代码、测试、部署和运行事实始终保持一致,避免出现以下问题:
|
||||
|
||||
- 代码已经变化,但架构文档仍描述旧边界;
|
||||
- API、字段或状态机变化后,测试与追踪矩阵未同步;
|
||||
- OpenClaw、Hermes 或 Agent Router 的控制职责发生漂移;
|
||||
- 候选分支存在实现,但版本无法解释为什么这样设计;
|
||||
- 部署、回滚或安全策略发生变化,却没有形成可审计文档;
|
||||
- 归档包只包含摘要或占位符,无法作为正式交付材料。
|
||||
|
||||
## 2. 文档语言要求
|
||||
|
||||
所有面向用户、产品、架构、运维、审计和发布的正式文档统一使用中文。
|
||||
|
||||
以下内容保留原始英文技术标识:
|
||||
|
||||
- 类名、函数名、字段名、环境变量;
|
||||
- API 路径、HTTP 方法和 Header;
|
||||
- Git 分支、Commit、Tag;
|
||||
- OpenClaw、Hermes、Agent Router、GitOps 等专有名称;
|
||||
- systemd、Docker、SQLite、HMAC、HTTP、JSON 等标准术语。
|
||||
|
||||
禁止在同一文档中无必要地切换中英文叙述。
|
||||
|
||||
## 3. 文档完整性要求
|
||||
|
||||
每份正式文档至少包含:
|
||||
|
||||
1. 文档目的;
|
||||
2. 适用范围;
|
||||
3. 当前版本和状态;
|
||||
4. 完整正文;
|
||||
5. 与代码、测试或部署的映射;
|
||||
6. 验收规则;
|
||||
7. 已知限制;
|
||||
8. 后续演进方向;
|
||||
9. 更新触发条件。
|
||||
|
||||
正式归档包禁止使用以下内容代替正文:
|
||||
|
||||
- “后续补充”;
|
||||
- “待完善”;
|
||||
- 只有标题没有说明;
|
||||
- 只包含目录摘要;
|
||||
- 与候选分支代码不一致的旧文档。
|
||||
|
||||
确实尚未验证的项目,应明确标记为“已实现但尚未验证”,并说明验证命令、责任方和进入下一状态的条件。
|
||||
|
||||
## 4. 文档分类与维护责任
|
||||
|
||||
| 文档 | 主要内容 | 触发更新的变化 |
|
||||
|---|---|---|
|
||||
| `README.md` | 文档索引、版本状态、实现地图 | 新增模块、目录、版本或交付物 |
|
||||
| `01-USER-REQUIREMENTS.md` | 用户目标、功能需求、非功能需求、验收场景 | 用户行为、产品范围、验收标准变化 |
|
||||
| `02-TARGET-ARCHITECTURE.md` | 分层、边界、数据流、安全、租户、部署架构 | 控制权、Runtime 边界、数据流或安全模型变化 |
|
||||
| `03-MIGRATION-AND-DELIVERY.md` | 迁移阶段、GitOps、发布门禁、回滚、Bug 修复 | 迁移路线、发布流程、门禁或回滚方式变化 |
|
||||
| `04-MODULE-CONTRACTS.md` | 数据模型、API、状态机、错误语义、Runtime Contract | 字段、接口、状态机、Header、错误行为变化 |
|
||||
| `05-TRACEABILITY-MATRIX.md` | 需求到代码、测试、部署、验收的映射 | 代码、测试、部署路径或状态变化 |
|
||||
| `adr/*` | 跨模块、长期或不可逆架构决策 | 关键决策新增、替换或废弃 |
|
||||
| `AIOS-V2R1-PRODUCTION-CANDIDATE.md` | 候选版本总览、测试、部署、风险和发布结论 | Candidate SHA、测试证据、部署或风险变化 |
|
||||
| `tasks/*` | OpenClaw、Codex 等在线执行任务 | 职责、边界、阻塞、验收或执行流程变化 |
|
||||
|
||||
## 5. 代码变化与文档同步规则
|
||||
|
||||
以下代码或配置路径发生变化时,必须同步更新 `docs/aios-v2/`:
|
||||
|
||||
### 5.1 Control Plane
|
||||
|
||||
触发路径:
|
||||
|
||||
```text
|
||||
active-runtime/app/aios_v2/control_plane.py
|
||||
active-runtime/app/aios_v2/persistence.py
|
||||
active-runtime/app/aios_v2/gateway.py
|
||||
active-runtime/app/aios_v2/app.py
|
||||
active-runtime/app/aios_v2/orchestrator.py
|
||||
active-runtime/app/aios_v2/execution_tracking.py
|
||||
active-runtime/app/aios_v2/gitops_status.py
|
||||
```
|
||||
|
||||
至少检查:
|
||||
|
||||
- 用户需求;
|
||||
- 目标架构;
|
||||
- 模块契约;
|
||||
- 追踪矩阵;
|
||||
- 必要时新增或更新 ADR。
|
||||
|
||||
### 5.2 OpenClaw
|
||||
|
||||
触发路径:
|
||||
|
||||
```text
|
||||
active-runtime/app/aios_openclaw/
|
||||
```
|
||||
|
||||
至少检查:
|
||||
|
||||
- OpenClawCommand、OpenClawResult、RuntimeHealth;
|
||||
- 签名、幂等、取消、超时和审计边界;
|
||||
- ADR-002;
|
||||
- 真实 Runtime 联调要求。
|
||||
|
||||
### 5.3 Hermes
|
||||
|
||||
触发路径:
|
||||
|
||||
```text
|
||||
active-runtime/app/aios_hermes/
|
||||
```
|
||||
|
||||
至少检查:
|
||||
|
||||
- HermesJob、HermesResult、HermesHealth;
|
||||
- tenant/cell/profile 分区;
|
||||
- Worker Lease、Heartbeat、Recovery;
|
||||
- ADR-002、ADR-004。
|
||||
|
||||
### 5.4 部署与运维
|
||||
|
||||
触发路径:
|
||||
|
||||
```text
|
||||
deploy/aios-v2/
|
||||
```
|
||||
|
||||
至少检查:
|
||||
|
||||
- 迁移与交付策略;
|
||||
- 部署契约;
|
||||
- 健康检查;
|
||||
- 回滚;
|
||||
- 候选报告。
|
||||
|
||||
### 5.5 API、Web 与用户行为
|
||||
|
||||
任何 API 路径、Header、请求/响应、页面或用户操作变化,必须同步:
|
||||
|
||||
- `01-USER-REQUIREMENTS.md`;
|
||||
- `02-TARGET-ARCHITECTURE.md`;
|
||||
- `04-MODULE-CONTRACTS.md`;
|
||||
- `05-TRACEABILITY-MATRIX.md`。
|
||||
|
||||
## 6. ADR 治理
|
||||
|
||||
必须新增或更新 ADR 的场景:
|
||||
|
||||
- AIOS、OpenClaw、Hermes、Agent Router 的控制边界变化;
|
||||
- 多租户隔离模型变化;
|
||||
- 持久化、消息、审计、密钥或部署出现跨模块变化;
|
||||
- 不兼容 API、字段或状态机变化;
|
||||
- GitOps、文档门禁或发布策略变化;
|
||||
- 替换核心基础设施或引入新的权威事实源。
|
||||
|
||||
ADR 状态:
|
||||
|
||||
- `已提出`:尚未正式采纳;
|
||||
- `已采纳`:当前有效;
|
||||
- `已替代`:被新 ADR 替代;
|
||||
- `已废弃`:保留历史但禁止新实现采用。
|
||||
|
||||
任何 ADR 替代必须说明:
|
||||
|
||||
- 被替代的 ADR;
|
||||
- 迁移影响;
|
||||
- 兼容性和回滚方案;
|
||||
- 受影响代码、测试和文档。
|
||||
|
||||
## 7. 文档—代码匹配门禁
|
||||
|
||||
验收必须执行:
|
||||
|
||||
```text
|
||||
python3 tools/aios_v2_docs_code_check.py
|
||||
```
|
||||
|
||||
门禁至少检查:
|
||||
|
||||
1. 必需文档存在且非空;
|
||||
2. 文档语言以中文为主;
|
||||
3. 文档引用的关键代码、测试和部署路径存在;
|
||||
4. 文档列出的核心 API 能在代码中找到;
|
||||
5. 文档列出的核心模型和字段能在代码中找到;
|
||||
6. Traceability Matrix 覆盖所有 `UR-*` 需求;
|
||||
7. ADR 索引引用的文件存在;
|
||||
8. 架构或功能代码变化时,当前交付范围包含 `docs/aios-v2/` 变化;
|
||||
9. 文档中不存在未解决 Git 冲突标记;
|
||||
10. 正式文档中不存在摘要占位符或空章节。
|
||||
|
||||
门禁失败时:
|
||||
|
||||
- Candidate 不得标记为 `Test-verified`;
|
||||
- OpenClaw 不得继续生产部署;
|
||||
- Codex 必须登记 blocker 并给出修复建议;
|
||||
- 修复必须进入 Git 分支并统一审视。
|
||||
|
||||
## 8. 在线任务文档要求
|
||||
|
||||
OpenClaw 和 Codex 任务必须使用中文说明,并保留可执行字段名。
|
||||
|
||||
每个任务至少包含:
|
||||
|
||||
- 任务目标;
|
||||
- 责任范围;
|
||||
- 禁止事项;
|
||||
- 输入和输出;
|
||||
- 分支;
|
||||
- 执行状态;
|
||||
- 验收命令;
|
||||
- claim、heartbeat、progress、blocker、commit、test_status、result 证据要求;
|
||||
- 阻塞升级路径。
|
||||
|
||||
## 9. 归档包要求
|
||||
|
||||
正式归档包必须包含完整正文:
|
||||
|
||||
```text
|
||||
docs/aios-v2/
|
||||
tasks/openclaw-production-deploy-task.yaml
|
||||
tasks/codex-production-supervisor-task.yaml
|
||||
deploy/aios-v2/
|
||||
AIOS-V2R1-PRODUCTION-CANDIDATE.md
|
||||
```
|
||||
|
||||
归档包必须同时生成清单:
|
||||
|
||||
- 文件路径;
|
||||
- 文件大小;
|
||||
- SHA-256;
|
||||
- Candidate branch;
|
||||
- Candidate commit;
|
||||
- 生成时间。
|
||||
|
||||
禁止只打包摘要副本。
|
||||
|
||||
## 10. 审查职责
|
||||
|
||||
### OpenClaw
|
||||
|
||||
- 部署和运行修复时同步相关部署、运维和已知问题文档;
|
||||
- 不得修改架构边界;
|
||||
- 发现架构影响时升级给 Codex。
|
||||
|
||||
### Codex
|
||||
|
||||
- 监督需求、架构、契约、追踪和 ADR 一致性;
|
||||
- 解决阻塞;
|
||||
- 拒绝缺少文档或证据的补丁;
|
||||
- 输出统一发布建议。
|
||||
|
||||
### AIOS Owner
|
||||
|
||||
- 审批合并、Tag 和生产部署;
|
||||
- 对重大架构决策和风险接受做最终确认。
|
||||
|
||||
## 11. 验收状态
|
||||
|
||||
文档完成状态只有在以下条件全部满足时才标记为“完整”:
|
||||
|
||||
- 所有必需文档和 ADR 已提交;
|
||||
- 文档—代码匹配检查通过;
|
||||
- 归档包包含仓库完整正文;
|
||||
- 任务合同为中文完整版本;
|
||||
- Candidate commit、清单和 SHA-256 已记录;
|
||||
- Codex 监督结论不存在未解决阻塞。
|
||||
|
||||
## 12. 健康状态证据治理
|
||||
|
||||
Controller 状态属于发布证据,不得只记录布尔 `gitea` 或 `branch_sync`。证据必须保留受控分支集合、`local_sync`、`gitea_sync`、`oid_match`、`branch_sync` 与 `sync_status`,但不得包含 Token、Authorization Header、SecretRef 解析值或带凭据的 Remote URL。配置变更、测试结果、部署备份和回滚点必须进入同一修复分支的审计链。
|
||||
|
||||
Controller 的 Gitea 凭据以 systemd Credential 形式运行时注入;Git 只记录凭据名称、权限合同和轮换事实,不记录值、Hash、长度或摘要。
|
||||
@@ -0,0 +1,232 @@
|
||||
# AIOS V2R1 监督状态与阻塞处置清单
|
||||
|
||||
- 文档版本:`V2R1C0-candidate`
|
||||
- 快照时间:`2026-07-20 17:03:53 CST`
|
||||
- 候选分支:`integration/aios-v2r1-cto-production-candidate-20260720`
|
||||
- 候选 OID:`634f6b164f5b20372ab9ec8144bbf76df97de335`
|
||||
- 当前决策:`STOP_PREPRODUCTION`
|
||||
- 当前状态:`BLOCKED`
|
||||
|
||||
## 1. 执行摘要
|
||||
|
||||
AIOS V2R1 的核心代码、完整中文文档、任务合同、部署资产和 GitOps 证据机制已经形成。OpenClaw 与 Codex 已经领取任务并向 Gitea 提交监督证据,三个关键分支的本地 OID 与 Gitea OID 已一致。
|
||||
|
||||
当前仍不允许进入预发布,原因不是主体功能缺失,而是安全、签名材料、Candidate 合并、执行来源证明和 Controller 健康检查五项高风险门禁未通过。
|
||||
|
||||
当前禁止:
|
||||
|
||||
- 进入 Gate A–D;
|
||||
- 执行在线 Runtime 联调;
|
||||
- 执行重启恢复和回滚演练;
|
||||
- 合并 `main`;
|
||||
- 创建 Tag;
|
||||
- 正式部署或切换流量。
|
||||
|
||||
## 2. 权威仓库与执行状态
|
||||
|
||||
| 项目 | 当前值 |
|
||||
|---|---|
|
||||
| 权威 Bare Repo | `/data/aios-gitops/repo.git` |
|
||||
| OpenClaw Worktree | 独立且 clean |
|
||||
| Codex Worktree | 独立且 clean |
|
||||
| Gitea Remote | `https://g.cnnetmkt.top/git/aios/aios.git` |
|
||||
| Candidate OID | `634f6b164f5b20372ab9ec8144bbf76df97de335` |
|
||||
| OpenClaw Task OID | `bd2784015d40c328d158cd52ebd8683b6e56b3ad` |
|
||||
| Codex Task OID | `e28245f411dd85ff9384ed595dacc53aab8a91ba` |
|
||||
| 三分支 Gitea 同步 | `local_oid == gitea_oid` |
|
||||
| OpenClaw 状态 | `blocked` |
|
||||
| OpenClaw Heartbeat | 有记录,但 `valid_for_running=false` |
|
||||
| Codex 状态 | `blocked` |
|
||||
| Codex Heartbeat | 新鲜 |
|
||||
| Codex 发布建议 | `STOP_PREPRODUCTION` |
|
||||
|
||||
## 3. 已完成的安全与运行修复
|
||||
|
||||
### 3.1 Gitea 凭据轮换
|
||||
|
||||
已完成:
|
||||
|
||||
- 旧 `owner/openclaw-deploy-cli`、`scope=all` 凭据 live count 为 `0`;
|
||||
- 新凭据:`gitops-bot/openclaw-deploy-cli-v2-20260720`;
|
||||
- 新凭据作用域:`write:repository`;
|
||||
- 凭据文件权限:`root:gitops 0640`;
|
||||
- 令牌值未进入 Git、日志、Evidence 或报告。
|
||||
|
||||
### 3.2 Gitea 与执行服务
|
||||
|
||||
已完成:
|
||||
|
||||
- 验证 Gitea SSH 指纹;
|
||||
- 备份并修复 `known_hosts`;
|
||||
- 恢复 `aios-agent-command-watcher`;
|
||||
- 恢复 `aios-autonomous-controller`;
|
||||
- 两项服务均为 `active`;
|
||||
- OpenClaw/One API 真实模型调用返回 HTTP 200;
|
||||
- OpenClaw/Codex claim、heartbeat、progress、blockers、result、status 已提交并同步 Gitea。
|
||||
|
||||
## 4. 当前开放阻塞
|
||||
|
||||
### 4.1 BLK-OPENCLAW-PLAINTEXT-SECRETS
|
||||
|
||||
- 严重级别:高
|
||||
- 状态:开放
|
||||
- 现象:OpenClaw 受管配置仍发现 3 处 `PLAINTEXT_FOUND`。
|
||||
- 风险:明文 Secret 可能进入配置、日志、命令参数或执行证据。
|
||||
- 解除条件:
|
||||
1. 三处明文内容全部迁移为 SecretRef、systemd Credential、只读 Secret Volume 或仓库外受保护文件;
|
||||
2. 旧值完成撤销和轮换;
|
||||
3. Secret Scanner 复验通过;
|
||||
4. Git 历史、日志和 Evidence 不包含明文。
|
||||
|
||||
### 4.2 BLK-AIOS-V2-SIGNING-MATERIAL-MISSING
|
||||
|
||||
- 严重级别:高
|
||||
- 状态:开放
|
||||
- 现象:`/etc/aios-v2/aios-v2.env` 已存在且权限合规,但 `/etc/aios-v2/openclaw-signing-material.bin` 不存在。
|
||||
- 风险:无法验证 OpenClaw 签名命令、Key ID 与真实 Runtime 身份链。
|
||||
- 外部受信输入:批准的签名材料。
|
||||
- 解除条件:
|
||||
1. 签名材料由批准的 Secret/Key 交付链安装到仓库外;
|
||||
2. 权限为 `0600` 或更严格;
|
||||
3. 运行用户可读;
|
||||
4. Key ID 与材料匹配;
|
||||
5. execute/status 路径完成真实签名验证;
|
||||
6. 不输出材料内容、Hash 或长度特征。
|
||||
|
||||
### 4.3 BLK-CANDIDATE-NOT-MERGED
|
||||
|
||||
- 严重级别:高
|
||||
- 状态:开放
|
||||
- 现象:OpenClaw Task 尚未包含当前 Candidate,`merge-base --is-ancestor` 未通过。
|
||||
- 风险:后续测试和部署不能证明当前 Candidate 可用。
|
||||
- 解除条件:
|
||||
1. 保护已有 `.agent/` Evidence;
|
||||
2. 普通 Merge 当前 Candidate 到 OpenClaw Task;
|
||||
3. 解决冲突并由 Codex 审查;
|
||||
4. 更新所有 Evidence 中的 `candidate_commit`;
|
||||
5. 本地与 Gitea OID 再次一致。
|
||||
|
||||
### 4.4 BLK-OPENCLAW-RUN-PROVENANCE-MISSING
|
||||
|
||||
- 严重级别:高
|
||||
- 状态:开放
|
||||
- 现象:OpenClaw 写工具被 `run_provenance_missing` 策略拒绝。
|
||||
- 风险:无法证明写操作来自已领取、已授权、可审计的执行会话。
|
||||
- 外部受信输入:允许七个 `.agent` JSON 写入的 OpenClaw Provenance 授权。
|
||||
- 解除条件:
|
||||
1. 创建带任务 ID、执行器实例、Claim OID、Candidate OID 和授权主体的 Run Provenance;
|
||||
2. Provenance 与七个 `.agent` JSON 写入范围绑定;
|
||||
3. 权限仅覆盖当前任务分支和 Evidence 路径;
|
||||
4. 由 Owner 或批准策略签发;
|
||||
5. 写入、Commit、Push 和审计链验证通过。
|
||||
|
||||
### 4.5 BLK-CONTROLLER-LEGACY-GITEA-CHECK
|
||||
|
||||
- 严重级别:高
|
||||
- 状态:开放
|
||||
- 现象:Controller 健康检查仍硬编码旧 `aios-v23`,并返回 `gitea=false`、`branch_sync=false`。
|
||||
- 风险:健康状态与当前 AIOS V2R1 仓库不一致,可能错误阻止或错误放行发布。
|
||||
- 解除条件:
|
||||
1. 移除旧仓库硬编码;
|
||||
2. 从配置读取权威仓库、Remote 和目标分支;
|
||||
3. 健康检查分别报告 `local_sync`、`gitea_sync` 和 `oid_match`;
|
||||
4. 不再把本地 Bare Repo 更新当作完全同步;
|
||||
5. 增加回归测试;
|
||||
6. 更新中文文档和追踪矩阵。
|
||||
|
||||
## 5. 模型审查策略修正
|
||||
|
||||
`gpt-5.3-codex-spark` 是子 Agent 审查的优先推荐模型,不是强制依赖。
|
||||
|
||||
执行规则:
|
||||
|
||||
1. Spark 可用时优先使用;
|
||||
2. Spark 不可用时,使用模型注册表中已批准、已启用且优先级最高的合适模型;
|
||||
3. 必须记录实际 `model_id`、模型名称和审查结果;
|
||||
4. 禁止冒充使用不存在的模型;
|
||||
5. 只有完全没有可用审查模型时,模型能力才构成阻塞。
|
||||
|
||||
当前五个 Spark 角色的独立结论一致为“禁止生产放行”,该结论作为已有审查证据保留;后续复验可使用任何满足上述规则的批准模型。
|
||||
|
||||
## 6. Gate 状态
|
||||
|
||||
| Gate | 当前状态 | 说明 |
|
||||
|---|---|---|
|
||||
| Gate A:文档与设计 | 未运行 | Candidate 未合并到任务,阻塞未解除 |
|
||||
| Gate B:代码与测试 | 未运行 | 不能基于不完整任务分支测试 |
|
||||
| Gate C:预发布 | 禁止 | Secret、签名、Provenance、Controller Health 未通过 |
|
||||
| Gate D:发布审查 | `STOP_PREPRODUCTION` | Codex 明确停止预发布 |
|
||||
| Gate E:生产发布 | 禁止 | 未满足任何生产 Gate |
|
||||
|
||||
## 7. 阻塞解除顺序
|
||||
|
||||
必须按以下顺序处理,避免重复测试和错误放行:
|
||||
|
||||
```text
|
||||
1. 清除 3 处明文 Secret
|
||||
↓
|
||||
2. 安装批准的签名材料
|
||||
↓
|
||||
3. 签发最小范围 Run Provenance
|
||||
↓
|
||||
4. 修复 Controller 旧仓库健康检查
|
||||
↓
|
||||
5. Merge 当前 Candidate 到 OpenClaw Task
|
||||
↓
|
||||
6. 校验 local_oid == gitea_oid
|
||||
↓
|
||||
7. 执行 Gate A:文档—代码检查
|
||||
↓
|
||||
8. 执行 Gate B:单元与集成测试
|
||||
↓
|
||||
9. 执行离线全量验收
|
||||
↓
|
||||
10. 执行在线 Runtime 联调
|
||||
↓
|
||||
11. 执行重启恢复与回滚
|
||||
↓
|
||||
12. Codex 重新审查并给出发布建议
|
||||
```
|
||||
|
||||
## 8. 外部受信输入
|
||||
|
||||
目前仅有两项确实需要外部受信输入:
|
||||
|
||||
1. **批准的签名材料**
|
||||
- 不得通过聊天、Git 或普通日志传递;
|
||||
- 必须通过批准的 Secret/Key 交付链安装。
|
||||
|
||||
2. **OpenClaw Run Provenance 授权**
|
||||
- 仅允许当前任务所需的七个 `.agent` JSON 写入;
|
||||
- 必须绑定任务、执行器实例、Claim、Candidate 和分支;
|
||||
- 不得授予通用生产写权限。
|
||||
|
||||
其余阻塞均可由 OpenClaw、Codex、Windows ChatGPT Owner 和 GitOps 运维流程处理。
|
||||
|
||||
## 9. 监督结论
|
||||
|
||||
当前状态保持:
|
||||
|
||||
```text
|
||||
BLOCKED
|
||||
STOP_PREPRODUCTION
|
||||
```
|
||||
|
||||
虽然三分支已经同步 Gitea、凭据已经最小权限轮换、执行服务已经恢复,但当前尚无足够证据允许执行 Gate A–D 或进入预发布。
|
||||
|
||||
任何后续状态更新必须同步:
|
||||
|
||||
- OpenClaw `.agent/status.json`、`blockers.json`、`progress.json`、`result.json`;
|
||||
- Codex 对应监督 Evidence;
|
||||
- 本文档;
|
||||
- `05-TRACEABILITY-MATRIX.md`;
|
||||
- `AIOS-V2R1-PRODUCTION-CANDIDATE.md`;
|
||||
- 如涉及架构或发布语义,更新 ADR。
|
||||
|
||||
## 9. 2026-07-20 Controller 修复进展
|
||||
|
||||
- `BLK-CONTROLLER-LEGACY-GITEA-CHECK` 已进入独立分支 `patch/bugfix/controller-gitea-health-v2r1-20260720` 修复;
|
||||
- 修复以 `/data/aios-gitops/repo.git`、配置的 Gitea 仓库路径和三条 V2R1 受控分支为输入;
|
||||
- 新状态合同区分本地状态、Gitea 状态、OID 一致性和聚合同步状态;
|
||||
- 分支测试、普通 Push、生产备份部署、重启与真实健康输出完成前,本 Blocker 仍保持未解决;
|
||||
- Gate A 仍不得启动,发布决策继续为 `STOP_PREPRODUCTION`。
|
||||
@@ -0,0 +1,129 @@
|
||||
# AIOS V2 架构与交付文档索引
|
||||
|
||||
- 版本线:`V2R1C0-candidate`
|
||||
- 候选分支:`integration/aios-v2r1-cto-production-candidate-20260720`
|
||||
- 架构负责人:AIOS V2 CTO
|
||||
- 当前状态:候选代码闭环已完成,运行时、预发布和生产验证尚待执行
|
||||
|
||||
## 1. 文档目的
|
||||
|
||||
本目录是 AIOS V2 用户需求、目标架构、模块边界、迁移策略、发布门禁和架构决策的唯一事实源。
|
||||
|
||||
所有后续架构、功能、接口、数据模型、安全策略、部署方式和验收流程变化,都必须在同一交付流程中同步更新本目录。
|
||||
|
||||
AIOS V2 遵循以下不可协商原则:
|
||||
|
||||
1. 不推倒重写现有平台。
|
||||
2. 优先复用 OpenClaw、Hermes 和 Agent Router 能力。
|
||||
3. OpenClaw 与 Hermes 从控制面职责中逐步解耦。
|
||||
4. AIOS 是唯一控制面。
|
||||
5. 所有面向用户和运维的能力必须通过 Web 或 API 访问。
|
||||
6. 所有模块必须强制多租户隔离。
|
||||
7. 所有修改必须通过 Git 分支、提交、测试、审查和受控提升完成。
|
||||
8. 代码变化必须同步更新需求、架构、契约、追踪矩阵或 ADR。
|
||||
|
||||
## 2. 文档地图
|
||||
|
||||
| 文档 | 用途 |
|
||||
|---|---|
|
||||
| [01-USER-REQUIREMENTS.md](01-USER-REQUIREMENTS.md) | 产品目标、用户角色、功能需求、非功能需求和验收标准 |
|
||||
| [02-TARGET-ARCHITECTURE.md](02-TARGET-ARCHITECTURE.md) | 目标逻辑架构、控制边界、数据流、安全模型和多租户模型 |
|
||||
| [03-MIGRATION-AND-DELIVERY.md](03-MIGRATION-AND-DELIVERY.md) | 增量迁移、能力复用、GitOps 流程、发布门禁和回滚策略 |
|
||||
| [04-MODULE-CONTRACTS.md](04-MODULE-CONTRACTS.md) | 租户上下文、任务生命周期、OpenClaw、Hermes、审计和执行证据契约 |
|
||||
| [05-TRACEABILITY-MATRIX.md](05-TRACEABILITY-MATRIX.md) | 需求、设计、代码、测试、部署和验收之间的追踪关系 |
|
||||
| [06-DOCUMENT-GOVERNANCE.md](06-DOCUMENT-GOVERNANCE.md) | 文档语言、完整性、同步门禁、版本与审查要求 |
|
||||
| [adr/README.md](adr/README.md) | 架构决策记录索引与治理规则 |
|
||||
|
||||
## 3. 当前实现映射
|
||||
|
||||
| 架构能力 | 候选实现 |
|
||||
|---|---|
|
||||
| Tenant、Agent、Task、Policy 和 Audit 控制面 | `active-runtime/app/aios_v2/control_plane.py` |
|
||||
| 控制面持久化 | `active-runtime/app/aios_v2/persistence.py` |
|
||||
| Web/API Gateway | `active-runtime/app/aios_v2/gateway.py` |
|
||||
| 统一应用入口 | `active-runtime/app/aios_v2/app.py` |
|
||||
| Runtime 调度与结果回写 | `active-runtime/app/aios_v2/orchestrator.py` |
|
||||
| Agent 执行证据 | `active-runtime/app/aios_v2/execution_tracking.py` |
|
||||
| GitOps 状态可视化 | `active-runtime/app/aios_v2/gitops_status.py` |
|
||||
| Web Console | `active-runtime/app/aios_v2/console.py` |
|
||||
| OpenClaw 执行边界 | `active-runtime/app/aios_openclaw/` |
|
||||
| Hermes Runtime 边界 | `active-runtime/app/aios_hermes/` |
|
||||
| 候选测试 | `tests/aios_v2_candidate/` |
|
||||
| 部署、健康检查与回滚 | `deploy/aios-v2/` |
|
||||
| OpenClaw 生产部署任务 | `tasks/openclaw-production-deploy-task.yaml` |
|
||||
| Codex 生产监督任务 | `tasks/codex-production-supervisor-task.yaml` |
|
||||
|
||||
## 4. 状态术语
|
||||
|
||||
项目必须严格区分“代码存在”和“可生产发布”:
|
||||
|
||||
- **已设计**:文档中存在明确边界和验收标准。
|
||||
- **已实现**:代码已经提交到候选分支。
|
||||
- **已定义测试**:Git 中存在自动化测试代码。
|
||||
- **已验证测试**:已对精确 commit 执行测试并保存证据。
|
||||
- **已验证预发布**:已在隔离环境完成部署、健康检查、重启和回滚演练。
|
||||
- **已生产发布**:已取得 Owner 批准并完成合并、Tag 和生产部署。
|
||||
|
||||
分支、文件、测试脚本或部署脚本存在,并不自动代表能力已通过生产验收。
|
||||
|
||||
## 5. 设计摘要
|
||||
|
||||
```text
|
||||
浏览器 / Web Console / Public API
|
||||
|
|
||||
v
|
||||
AIOS 唯一控制面
|
||||
Tenant | Agent | Task | Policy | Audit
|
||||
|
|
||||
v
|
||||
Execution Orchestrator / Agent Router
|
||||
/ \
|
||||
v v
|
||||
OpenClaw Adapter Hermes Boundary
|
||||
| |
|
||||
v v
|
||||
OpenClaw Runtime Hermes Workers
|
||||
\
|
||||
v
|
||||
结果回写 + 审计 + 执行证据
|
||||
```
|
||||
|
||||
控制面拥有权威身份、策略、审批、任务生命周期和审计状态。OpenClaw 与 Hermes 只接收带租户上下文、已经批准的执行工作,并返回运行结果。
|
||||
|
||||
## 6. 发布规则
|
||||
|
||||
候选版本在满足以下条件前不得提升:
|
||||
|
||||
- 需求到代码的追踪关系完整;
|
||||
- 文档—代码匹配检查通过;
|
||||
- 多租户负向测试通过;
|
||||
- GitOps Controller worktree 保持 detached 且 clean;
|
||||
- 本地 Git 与 Gitea 目标分支 OID 一致;
|
||||
- 持久化重启恢复测试通过;
|
||||
- 预发布健康检查和回滚演练通过;
|
||||
- Git 中不存在明文密钥;
|
||||
- OpenClaw 与 Hermes 真实 Runtime 联调通过;
|
||||
- Owner 对合并、Tag 和生产部署做出明确批准。
|
||||
|
||||
## 7. 文档完整性要求
|
||||
|
||||
每份正式文档必须包含:
|
||||
|
||||
- 文档目的与适用范围;
|
||||
- 当前版本和状态;
|
||||
- 关键设计或需求正文;
|
||||
- 代码、测试或部署映射;
|
||||
- 验收与更新规则;
|
||||
- 已知限制和后续演进方向。
|
||||
|
||||
禁止在正式归档包中以摘要、占位符或“后续补充”代替正文。
|
||||
|
||||
## 8. 中文输出要求
|
||||
|
||||
所有面向用户、产品、架构、运维和审计的正式文档统一使用中文。以下技术标识保留原始英文,避免与代码不一致:
|
||||
|
||||
- 类名、函数名、字段名;
|
||||
- API 路径和 HTTP Header;
|
||||
- Git 分支、Commit、Tag;
|
||||
- OpenClaw、Hermes、Agent Router、GitOps 等专有名称;
|
||||
- systemd、Docker、SQLite、HMAC、HTTP 等标准技术术语。
|
||||
@@ -0,0 +1,108 @@
|
||||
# ADR-001:AIOS 作为唯一控制面
|
||||
|
||||
- 状态:已采纳
|
||||
- 日期:2026-07-20
|
||||
- 影响范围:Control Plane、Agent Router、OpenClaw、Hermes、Web Console、GitOps、审计
|
||||
|
||||
## 1. 背景
|
||||
|
||||
现有系统已经包含多个执行、调度和治理能力。如果 OpenClaw、Hermes、Agent Router 或 Web 层分别保存自己的租户、策略、审批和任务状态,就会形成多个权威事实源,带来以下问题:
|
||||
|
||||
- 同一任务在不同模块中状态不一致;
|
||||
- 审计链不完整;
|
||||
- 多租户隔离难以统一验证;
|
||||
- Web、Runtime 和运维入口可能绕过策略;
|
||||
- Bug 修复和发布无法判断应该修改哪个事实源。
|
||||
|
||||
## 2. 决策
|
||||
|
||||
AIOS Control Plane 是唯一权威控制面,负责:
|
||||
|
||||
- Tenant Registry;
|
||||
- Identity 与角色上下文;
|
||||
- Agent Registry;
|
||||
- Task Lifecycle;
|
||||
- Policy Decision;
|
||||
- Approval;
|
||||
- Audit Event;
|
||||
- Runtime Inventory;
|
||||
- Execution Evidence;
|
||||
- GitOps 与发布状态可视化。
|
||||
|
||||
OpenClaw、Hermes 和 Agent Router 只能执行、路由或调度已经由 AIOS 批准的工作,不得拥有第二套租户、策略、审批或任务生命周期。
|
||||
|
||||
## 3. 目标边界
|
||||
|
||||
```text
|
||||
Web / API
|
||||
|
|
||||
v
|
||||
AIOS Control Plane
|
||||
|
|
||||
v
|
||||
Execution Orchestrator / Agent Router
|
||||
|
|
||||
+--> OpenClaw Adapter --> OpenClaw Runtime
|
||||
|
|
||||
+--> Hermes Boundary --> Hermes Workers
|
||||
```
|
||||
|
||||
## 4. 备选方案
|
||||
|
||||
### 方案 A:保持多个控制面
|
||||
|
||||
拒绝原因:状态会持续分裂,无法形成统一审计和租户隔离。
|
||||
|
||||
### 方案 B:由 OpenClaw 或 Hermes 成为主控制面
|
||||
|
||||
拒绝原因:二者主要价值是执行与 Runtime,不适合作为统一产品、租户、审批和发布控制面。
|
||||
|
||||
### 方案 C:推倒重写全部能力
|
||||
|
||||
拒绝原因:会丢失现有执行能力、运行经验和迁移可回滚性。
|
||||
|
||||
## 5. 正面影响
|
||||
|
||||
- 任务、策略和租户状态唯一;
|
||||
- Web/API、OpenClaw、Hermes 和 GitOps 使用同一权威模型;
|
||||
- 审计、执行证据和发布判断可以统一;
|
||||
- 新 Runtime 可以通过稳定 Adapter 接入;
|
||||
- 多租户和权限模型可集中验证。
|
||||
|
||||
## 6. 负面影响与风险
|
||||
|
||||
- 所有旧执行入口需要迁移到 Orchestrator;
|
||||
- Runtime 需要稳定 Contract;
|
||||
- Control Plane 可靠性要求提高;
|
||||
- 在迁移完成前可能短期存在旧旁路,需要明确冻结和监控。
|
||||
|
||||
## 7. 迁移与回滚
|
||||
|
||||
迁移顺序:
|
||||
|
||||
1. 建立 AIOS Control Plane 权威模型;
|
||||
2. 引入 Orchestrator;
|
||||
3. OpenClaw/Hermes 通过 Adapter 接入;
|
||||
4. 将 Web 和自动任务切换到 AIOS API;
|
||||
5. 关闭旧控制旁路。
|
||||
|
||||
回滚只允许回滚 Runtime 或应用版本,不允许恢复第二套权威控制状态。
|
||||
|
||||
## 8. 合规检查
|
||||
|
||||
任何新功能必须回答:
|
||||
|
||||
- 权威状态是否存放在 AIOS?
|
||||
- Runtime 是否只接收已批准工作?
|
||||
- Web 是否只调用 AIOS API?
|
||||
- 是否新增了第二套状态机或租户事实源?
|
||||
|
||||
若最后一项为“是”,变更必须拒绝或新增替代 ADR。
|
||||
|
||||
## 9. 实现与测试映射
|
||||
|
||||
- 实现:`active-runtime/app/aios_v2/control_plane.py`
|
||||
- 编排:`active-runtime/app/aios_v2/orchestrator.py`
|
||||
- 应用入口:`active-runtime/app/aios_v2/app.py`
|
||||
- 测试:`tests/aios_v2_candidate/test_candidate_flow.py`
|
||||
- 追踪:`docs/aios-v2/05-TRACEABILITY-MATRIX.md`
|
||||
@@ -0,0 +1,108 @@
|
||||
# ADR-002:OpenClaw 与 Hermes 从控制面职责中解耦
|
||||
|
||||
- 状态:已采纳
|
||||
- 日期:2026-07-20
|
||||
- 影响范围:OpenClaw、Hermes、Execution Orchestrator、Agent Router、Audit、Web Console
|
||||
|
||||
## 1. 背景
|
||||
|
||||
OpenClaw 和 Hermes 已经具备成熟的执行能力,但历史实现可能同时承担部分租户、策略、任务状态或运维控制职责。如果继续扩展这种模式,AIOS 将无法成为唯一控制面,也无法统一 Web、审计和多租户隔离。
|
||||
|
||||
## 2. 决策
|
||||
|
||||
OpenClaw 与 Hermes 通过稳定的 Adapter/Boundary 接入 AIOS:
|
||||
|
||||
```text
|
||||
AIOS Control Plane
|
||||
|
|
||||
v
|
||||
Execution Orchestrator
|
||||
|
|
||||
+--> OpenClaw Adapter --> OpenClaw Runtime
|
||||
|
|
||||
+--> Hermes Boundary --> Hermes Workers
|
||||
```
|
||||
|
||||
OpenClaw 只负责:
|
||||
|
||||
- 验证签名命令;
|
||||
- 执行;
|
||||
- 查询状态;
|
||||
- 取消;
|
||||
- 健康检查;
|
||||
- 返回结果和生命周期事件。
|
||||
|
||||
Hermes 只负责:
|
||||
|
||||
- tenant/cell/profile 分区队列;
|
||||
- Worker claim;
|
||||
- Lease 与 Heartbeat;
|
||||
- 执行;
|
||||
- 完成、失败、取消和结果回传。
|
||||
|
||||
二者均不得:
|
||||
|
||||
- 创建或批准 AIOS Task;
|
||||
- 修改租户或策略;
|
||||
- 维护第二套任务生命周期;
|
||||
- 绕过 AIOS Audit;
|
||||
- 向浏览器提供直接控制入口。
|
||||
|
||||
## 3. 备选方案
|
||||
|
||||
### 方案 A:让 OpenClaw 继续拥有控制职责
|
||||
|
||||
拒绝原因:会产生第二控制面,并使 Web、审批和审计持续分裂。
|
||||
|
||||
### 方案 B:让 Hermes 统一控制和执行
|
||||
|
||||
拒绝原因:Hermes 的核心价值是 Worker Runtime 与队列,不应承担产品、租户和策略治理。
|
||||
|
||||
### 方案 C:直接替换 OpenClaw/Hermes
|
||||
|
||||
拒绝原因:不符合能力复用原则,会扩大迁移风险和开发量。
|
||||
|
||||
## 4. 正面影响
|
||||
|
||||
- AIOS 保持唯一权威状态;
|
||||
- OpenClaw/Hermes 可以独立升级;
|
||||
- 新 Runtime 可使用相同接入模式;
|
||||
- Web 和审计不再依赖 Runtime 内部实现;
|
||||
- 可逐步关闭旧旁路,而不需要一次性重写执行层。
|
||||
|
||||
## 5. 负面影响与风险
|
||||
|
||||
- 需要维护稳定的命令、结果和健康 Contract;
|
||||
- 旧调用路径可能在迁移期并存;
|
||||
- Runtime 联调和错误语义需要统一;
|
||||
- Adapter 本身成为重要可靠性边界。
|
||||
|
||||
## 6. 迁移步骤
|
||||
|
||||
1. 定义 OpenClawCommand/OpenClawResult/RuntimeHealth;
|
||||
2. 定义 HermesJob/HermesResult/HermesHealth;
|
||||
3. 引入 AIOS Orchestrator;
|
||||
4. 将新任务先通过 Adapter 执行;
|
||||
5. 迁移 Web、Codex 和自动任务入口;
|
||||
6. 冻结旧 Runtime 控制接口;
|
||||
7. 通过网络策略或代码门禁关闭旁路。
|
||||
|
||||
## 7. 回滚原则
|
||||
|
||||
允许回滚 Runtime 或 Adapter 版本,但不允许回滚到 Runtime 重新拥有 Tenant、Policy、Approval 或 Task Lifecycle 的模式。
|
||||
|
||||
## 8. 合规检查
|
||||
|
||||
- OpenClaw 输入是否为 AIOS 生成的签名命令?
|
||||
- Hermes Job 是否包含 tenant/cell/profile?
|
||||
- Runtime 是否只返回结果,不直接写 AIOS Store?
|
||||
- Web 是否只调用 AIOS API?
|
||||
- Runtime Bug 修复是否同步更新 Contract、测试和文档?
|
||||
|
||||
## 9. 实现与测试映射
|
||||
|
||||
- OpenClaw:`active-runtime/app/aios_openclaw/`
|
||||
- Hermes:`active-runtime/app/aios_hermes/`
|
||||
- Orchestrator:`active-runtime/app/aios_v2/orchestrator.py`
|
||||
- 测试:`tests/aios_v2_candidate/test_candidate_flow.py`
|
||||
- 模块契约:`docs/aios-v2/04-MODULE-CONTRACTS.md`
|
||||
@@ -0,0 +1,127 @@
|
||||
# ADR-003:所有变更必须通过 GitOps
|
||||
|
||||
- 状态:已采纳
|
||||
- 日期:2026-07-20
|
||||
- 影响范围:代码、配置、文档、测试、部署、回滚、生产修复
|
||||
|
||||
## 1. 背景
|
||||
|
||||
直接在生产目录、长期 worktree、主分支或运行容器中修改代码,会造成以下问题:
|
||||
|
||||
- 变更不可追踪;
|
||||
- 无法稳定回滚;
|
||||
- 本地仓库与 Gitea 不一致;
|
||||
- Worktree 和 Branch Ref 冲突;
|
||||
- 紧急修复无法统一审视;
|
||||
- 文档、测试和部署资产无法同步。
|
||||
|
||||
## 2. 决策
|
||||
|
||||
所有代码、配置、文档、测试、部署和回滚资产必须经过以下流程:
|
||||
|
||||
```text
|
||||
非 main 分支
|
||||
-> Commit
|
||||
-> 本地仓库同步
|
||||
-> Gitea 镜像与 OID 校验
|
||||
-> 自动测试
|
||||
-> Codex 监督审查
|
||||
-> 候选集成
|
||||
-> Owner 批准
|
||||
-> Merge / Tag / Deploy
|
||||
```
|
||||
|
||||
GitOps Controller worktree 必须长期保持 detached 且 clean。相同分支写入必须串行,任务工作区必须可回收。
|
||||
|
||||
## 3. 必需证据
|
||||
|
||||
每个任务必须记录:
|
||||
|
||||
- 任务 ID;
|
||||
- 执行器;
|
||||
- 分支;
|
||||
- Commit;
|
||||
- 修改文件;
|
||||
- claim;
|
||||
- heartbeat;
|
||||
- progress;
|
||||
- blocker;
|
||||
- 测试命令、exit code 和结果;
|
||||
- 本地/Gitea 同步状态;
|
||||
- 回滚说明;
|
||||
- 最终结果和监督结论。
|
||||
|
||||
## 4. 禁止事项
|
||||
|
||||
- 直接 push `main`;
|
||||
- 未审批创建生产 Tag;
|
||||
- 未记录任务直接修改生产;
|
||||
- 在生产目录手工修复后不回写 Git;
|
||||
- fetch 到已被 worktree 检出的本地 branch ref;
|
||||
- 以新分支名反复规避 Worktree 生命周期缺陷;
|
||||
- 在 Git 中提交明文凭据;
|
||||
- 缺少文档和测试时标记为完成。
|
||||
|
||||
## 5. 备选方案
|
||||
|
||||
### 方案 A:允许紧急生产手工修改
|
||||
|
||||
拒绝原因:即使紧急修复也必须形成 Patch、Commit 和证据,否则无法统一审视和回滚。
|
||||
|
||||
### 方案 B:只同步本地 Bare Repo
|
||||
|
||||
拒绝原因:Web Gitea 无法看到真实任务进度,团队无法审查。
|
||||
|
||||
### 方案 C:Controller 长期检出任务分支
|
||||
|
||||
拒绝原因:会触发 `refusing to fetch into branch checked out at worktree`,并破坏并发任务。
|
||||
|
||||
## 6. 正面影响
|
||||
|
||||
- 所有变更可审计;
|
||||
- 本地和 Web 远端一致;
|
||||
- 紧急修复也可回滚;
|
||||
- OpenClaw/Codex 进度可统一查询;
|
||||
- 文档与代码可通过门禁同步。
|
||||
|
||||
## 7. 负面影响与风险
|
||||
|
||||
- 生产修复增加提交与证据步骤;
|
||||
- GitOps Runtime 成为关键基础设施;
|
||||
- 需要处理分支锁、任务恢复和镜像失败;
|
||||
- 自动执行器必须遵循统一任务格式。
|
||||
|
||||
## 8. 运行规则
|
||||
|
||||
- Controller:detached + clean;
|
||||
- Fetch:更新 remote-tracking ref,不写已检出本地 branch;
|
||||
- Write:独立任务工作区或受控写入;
|
||||
- Lock:同一分支串行;
|
||||
- Mirror:本地成功和 Gitea 成功分别记录;
|
||||
- Failure:状态标记为 degraded/blocked,不得伪报 synced。
|
||||
|
||||
## 9. 紧急 Bug 修复流程
|
||||
|
||||
1. 创建 `patch/bugfix/*`;
|
||||
2. 登记 blocker 和影响;
|
||||
3. 添加复现测试;
|
||||
4. 修复代码;
|
||||
5. 同步文档;
|
||||
6. 执行验收;
|
||||
7. Codex 审核;
|
||||
8. Owner 批准发布;
|
||||
9. 生产观察并记录结果。
|
||||
|
||||
## 10. 实现与验收映射
|
||||
|
||||
- GitOps 状态:`active-runtime/app/aios_v2/gitops_status.py`
|
||||
- 健康检查:`deploy/aios-v2/healthcheck/check.sh`
|
||||
- 回滚:`deploy/aios-v2/rollback/rollback.sh`
|
||||
- 任务:`tasks/openclaw-production-deploy-task.yaml`、`tasks/codex-production-supervisor-task.yaml`
|
||||
- 文档门禁:`tools/aios_v2_docs_code_check.py`
|
||||
|
||||
## 11. Controller 同步状态决定
|
||||
|
||||
Controller 不再依赖运行目录中的旧仓库名或“分支集合看似存在”的近似判断。所有受控分支必须以配置的权威 Bare Repo 为本地真源,以配置的 Gitea 仓库为远端真源,并逐分支比较 OID。
|
||||
|
||||
只有全部 OID 一致时才能输出 `fully_synced`。本地领先、远端缺失、远端不可达和认证失败必须保持可区分的失败语义。健康证据不得记录认证材料或带凭据的 URL;Controller 允许以 detached + clean 的部署检出运行,因为可追溯性由部署 Commit、备份和 Git 分支提供。
|
||||
@@ -0,0 +1,150 @@
|
||||
# ADR-004:多租户隔离采用默认拒绝模型
|
||||
|
||||
- 状态:已采纳
|
||||
- 日期:2026-07-20
|
||||
- 影响范围:Web/API、Control Plane、OpenClaw、Hermes、Persistence、Audit、Execution Evidence、Logs、Metrics、Secret
|
||||
|
||||
## 1. 背景
|
||||
|
||||
AIOS V2 同时承载多个租户的用户请求、控制面状态、OpenClaw 命令、Hermes 队列、执行证据、审计和 GitOps 状态。任何一层缺失租户边界,都可能造成数据泄露、跨租户任务执行或错误回写。
|
||||
|
||||
## 2. 决策
|
||||
|
||||
多租户隔离采用:
|
||||
|
||||
```text
|
||||
显式租户上下文
|
||||
+
|
||||
默认拒绝
|
||||
+
|
||||
全链路传播
|
||||
+
|
||||
负向测试
|
||||
```
|
||||
|
||||
所有资源、命令、Job、证据和审计记录都必须显式绑定 `tenant_id`。缺少上下文、上下文不匹配或无法确认所有权时,系统必须 fail closed。
|
||||
|
||||
## 3. 权威上下文
|
||||
|
||||
控制请求至少包含:
|
||||
|
||||
- `tenant_id`;
|
||||
- `principal_id`;
|
||||
- `trace_id`;
|
||||
- `policy_version`;
|
||||
- 可选 `session_id`。
|
||||
|
||||
执行请求增加:
|
||||
|
||||
- `task_id`;
|
||||
- `command_id` 或 `job_id`;
|
||||
- `idempotency_key`。
|
||||
|
||||
Hermes 额外包含:
|
||||
|
||||
- `cell_id`;
|
||||
- `profile_id`;
|
||||
- `worker_id`。
|
||||
|
||||
## 4. 各层执行规则
|
||||
|
||||
### Web/API
|
||||
|
||||
- 可信代理必须删除客户端同名 `X-AIOS-*` Header 后重新注入;
|
||||
- 浏览器不得自行选择或覆盖租户;
|
||||
- 未认证身份断言必须拒绝。
|
||||
|
||||
### Control Plane
|
||||
|
||||
- Tenant、Agent、Task、Audit 全部带不可为空的 `tenant_id`;
|
||||
- 跨租户读取返回拒绝或“未找到”;
|
||||
- 不泄露其他租户资源存在性。
|
||||
|
||||
### OpenClaw
|
||||
|
||||
- 签名覆盖 tenant/task/trace/action/payload/idempotency/time;
|
||||
- Result 身份字段必须与命令完全一致;
|
||||
- 跨租户 status/cancel 返回“未找到”。
|
||||
|
||||
### Hermes
|
||||
|
||||
- Job 按 tenant/cell/profile 分区;
|
||||
- Worker 只能 claim 自己的分区;
|
||||
- Worker ID 不得在未授权分区之间迁移;
|
||||
- Result 必须匹配 tenant/task/trace。
|
||||
|
||||
### Persistence
|
||||
|
||||
- 候选版使用显式租户字段;
|
||||
- 生产数据库必须提供 RLS 或物理隔离策略;
|
||||
- 备份、恢复和迁移必须保持租户边界。
|
||||
|
||||
### Audit 与 Evidence
|
||||
|
||||
- 仅按当前租户查询;
|
||||
- 不记录明文 Secret;
|
||||
- 统一包含 tenant/task/trace。
|
||||
|
||||
### Cache、Memory、Artifact、Logs、Metrics
|
||||
|
||||
- Key 或标签必须包含租户命名空间;
|
||||
- 查询和导出必须受租户权限限制;
|
||||
- 不允许使用无租户前缀的全局缓存保存业务数据。
|
||||
|
||||
## 5. 备选方案
|
||||
|
||||
### 方案 A:仅在 API 层检查租户
|
||||
|
||||
拒绝原因:后台任务、Runtime、缓存和结果回写仍可能跨租户。
|
||||
|
||||
### 方案 B:仅依赖数据库隔离
|
||||
|
||||
拒绝原因:命令、队列、审计和文件系统仍需要显式边界。
|
||||
|
||||
### 方案 C:发生冲突时自动推断租户
|
||||
|
||||
拒绝原因:推断可能产生跨租户副作用,必须默认拒绝。
|
||||
|
||||
## 6. 正面影响
|
||||
|
||||
- 安全模型一致;
|
||||
- Runtime 无法绕过租户边界;
|
||||
- 负向测试可自动化;
|
||||
- 审计和问题定位更明确。
|
||||
|
||||
## 7. 负面影响与风险
|
||||
|
||||
- 所有接口和模型都需要传播租户上下文;
|
||||
- 日志、指标和缓存设计更复杂;
|
||||
- 候选 SQLite 只有逻辑隔离,生产仍需更强存储证据;
|
||||
- 身份代理配置错误可能破坏入口隔离。
|
||||
|
||||
## 8. 测试要求
|
||||
|
||||
至少覆盖:
|
||||
|
||||
- Tenant B 读取 Tenant A Task;
|
||||
- Tenant B 查询或取消 Tenant A OpenClawCommand;
|
||||
- Worker 跨 tenant/cell/profile claim;
|
||||
- Runtime 返回错误 tenant/task/trace;
|
||||
- Evidence 和 Audit 跨租户查询;
|
||||
- 缺少 Tenant Context 的请求;
|
||||
- 客户端伪造身份 Header。
|
||||
|
||||
## 9. 合规检查
|
||||
|
||||
任何新模块没有以下内容时不得进入 Candidate:
|
||||
|
||||
- 显式 `tenant_id`;
|
||||
- 所有权检查;
|
||||
- 跨租户负向测试;
|
||||
- 审计映射;
|
||||
- 文档与追踪矩阵更新。
|
||||
|
||||
## 10. 实现与测试映射
|
||||
|
||||
- Control Plane:`active-runtime/app/aios_v2/control_plane.py`
|
||||
- OpenClaw:`active-runtime/app/aios_openclaw/`
|
||||
- Hermes:`active-runtime/app/aios_hermes/`
|
||||
- Evidence:`active-runtime/app/aios_v2/execution_tracking.py`
|
||||
- 测试:`tests/aios_v2_candidate/test_candidate_flow.py`
|
||||
@@ -0,0 +1,116 @@
|
||||
# ADR-005:文档与代码同步门禁
|
||||
|
||||
- 状态:已采纳
|
||||
- 日期:2026-07-20
|
||||
- 影响范围:需求、架构、代码、测试、部署、在线任务、发布和归档
|
||||
|
||||
## 1. 背景
|
||||
|
||||
架构、功能和部署变化如果不同步文档,会导致版本无法审计,后续 Agent 无法判断真实设计,发布人员也无法确认代码是否符合用户需求。历史上还出现过文档目录不存在、文档含冲突标记、分支名与代码内容不一致等问题。
|
||||
|
||||
## 2. 决策
|
||||
|
||||
所有影响以下内容的变更必须在同一交付流程中同步 `docs/aios-v2/`:
|
||||
|
||||
- 用户行为和功能范围;
|
||||
- API、Header、请求和响应;
|
||||
- 数据模型、字段和状态机;
|
||||
- Control Plane、Agent Router、OpenClaw、Hermes 的边界;
|
||||
- 多租户和安全策略;
|
||||
- 持久化、审计和执行证据;
|
||||
- 部署、健康检查和回滚;
|
||||
- 测试和发布门禁。
|
||||
|
||||
正式文档统一使用中文,技术标识保留代码中的原始英文名称。
|
||||
|
||||
## 3. 门禁要求
|
||||
|
||||
验收必须执行:
|
||||
|
||||
```text
|
||||
python3 tools/aios_v2_docs_code_check.py
|
||||
```
|
||||
|
||||
检查至少包括:
|
||||
|
||||
1. 必需文档和 ADR 存在且非空;
|
||||
2. 文档以中文叙述为主;
|
||||
3. 文档引用的代码、测试和部署路径存在;
|
||||
4. 核心 API 能在代码中找到;
|
||||
5. 核心模型和字段能在代码中找到;
|
||||
6. Traceability Matrix 覆盖所有 `UR-*`;
|
||||
7. ADR 索引引用的文件存在;
|
||||
8. 文档中不存在 Git 冲突标记;
|
||||
9. 不存在摘要占位符或空章节;
|
||||
10. 架构/功能代码变化时,同一交付范围包含文档变化。
|
||||
|
||||
## 4. 状态影响
|
||||
|
||||
门禁失败时:
|
||||
|
||||
- Candidate 不得标记为 `Test-verified`;
|
||||
- OpenClaw 不得继续生产部署;
|
||||
- Codex 必须登记 blocker;
|
||||
- 修复必须进入 Git 分支;
|
||||
- Owner 不得批准 Tag 或生产部署。
|
||||
|
||||
## 5. 备选方案
|
||||
|
||||
### 方案 A:依赖人工记忆更新文档
|
||||
|
||||
拒绝原因:长期必然产生文档漂移。
|
||||
|
||||
### 方案 B:发布前集中补文档
|
||||
|
||||
拒绝原因:无法确定每次代码变化对应的真实设计,也会增加错误归因。
|
||||
|
||||
### 方案 C:只检查文件存在
|
||||
|
||||
拒绝原因:文件存在不代表内容完整、路径有效或与代码一致。
|
||||
|
||||
## 6. 正面影响
|
||||
|
||||
- 架构和代码长期一致;
|
||||
- Agent 有明确执行上下文;
|
||||
- Bug 修复可以追踪需求和决策;
|
||||
- 归档包可以作为正式交付材料;
|
||||
- 发布风险和未验证项更透明。
|
||||
|
||||
## 7. 负面影响与风险
|
||||
|
||||
- 小功能也需要更新对应文档;
|
||||
- 检查器需要随代码结构维护;
|
||||
- 自动检查不能替代架构审查;
|
||||
- 文档完整性会增加提交内容,但这是可审计交付的必要成本。
|
||||
|
||||
## 8. 文档更新映射
|
||||
|
||||
- 用户需求变化:`01-USER-REQUIREMENTS.md`;
|
||||
- 架构边界变化:`02-TARGET-ARCHITECTURE.md` 和 ADR;
|
||||
- 迁移/发布变化:`03-MIGRATION-AND-DELIVERY.md`;
|
||||
- Contract 变化:`04-MODULE-CONTRACTS.md`;
|
||||
- 路径和状态变化:`05-TRACEABILITY-MATRIX.md`;
|
||||
- 文档规则变化:`06-DOCUMENT-GOVERNANCE.md`。
|
||||
|
||||
## 9. 在线任务要求
|
||||
|
||||
OpenClaw 和 Codex 的任何修改或阻塞必须回写:
|
||||
|
||||
- claim;
|
||||
- heartbeat;
|
||||
- progress;
|
||||
- blocker;
|
||||
- branch;
|
||||
- commit;
|
||||
- test_status;
|
||||
- result;
|
||||
- 文档同步状态。
|
||||
|
||||
OpenClaw 发现架构影响时必须停止扩大修复范围,并升级给 Codex。
|
||||
|
||||
## 10. 实现与验收映射
|
||||
|
||||
- 检查器:`tools/aios_v2_docs_code_check.py`
|
||||
- 验收入口:`deploy/aios-v2/acceptance/run-all.sh`
|
||||
- 治理规范:`docs/aios-v2/06-DOCUMENT-GOVERNANCE.md`
|
||||
- 追踪矩阵:`docs/aios-v2/05-TRACEABILITY-MATRIX.md`
|
||||
@@ -0,0 +1,142 @@
|
||||
# ADR-006:候选持久化采用可替换 SQLite Snapshot
|
||||
|
||||
- 状态:已采纳
|
||||
- 日期:2026-07-20
|
||||
- 影响范围:Control Plane、Persistence、Deployment、Backup、Recovery、HA 演进
|
||||
|
||||
## 1. 背景
|
||||
|
||||
AIOS V2 候选版本需要在服务重启后恢复 Tenant、Agent、Task、Audit 和 Idempotency 状态,同时又必须尽快进入预发布部署与 Bug 修复阶段。
|
||||
|
||||
候选阶段如果立即引入完整分布式数据库、消息系统和 HA,会显著扩大开发范围,推迟控制面、Runtime Boundary 和多租户模型的验证。
|
||||
|
||||
## 2. 决策
|
||||
|
||||
候选版本使用可替换的 SQLite WAL 原子 Snapshot:
|
||||
|
||||
- `SQLiteSnapshotRepository` 保存版本化 Control Plane Snapshot;
|
||||
- `DurableControlPlaneService` 在成功变更后持久化;
|
||||
- 使用 WAL、`synchronous=FULL` 和 `BEGIN IMMEDIATE`;
|
||||
- 状态目录独立于代码版本;
|
||||
- 上层 Domain、API、Orchestrator 和 Runtime Contract 不依赖 SQLite 实现。
|
||||
|
||||
执行证据使用租户分区 JSON 文件,由 `ExecutionTracker` 原子写入。
|
||||
|
||||
## 3. 持久化范围
|
||||
|
||||
候选 Snapshot 包含:
|
||||
|
||||
- Tenant;
|
||||
- Agent;
|
||||
- Task;
|
||||
- Audit;
|
||||
- Idempotency。
|
||||
|
||||
独立 Evidence 目录包含:
|
||||
|
||||
- claim;
|
||||
- heartbeat;
|
||||
- progress;
|
||||
- blocker;
|
||||
- commit;
|
||||
- test_status;
|
||||
- result。
|
||||
|
||||
## 4. 选择理由
|
||||
|
||||
- 无额外外部依赖;
|
||||
- 易于快速部署和回滚;
|
||||
- 支持事务、WAL 和完整性检查;
|
||||
- 可以验证上层控制面和 Runtime Contract;
|
||||
- 便于在预发布阶段复现和修复 Bug;
|
||||
- 可通过 Repository 边界替换为生产数据库。
|
||||
|
||||
## 5. 备选方案
|
||||
|
||||
### 方案 A:候选阶段直接使用 PostgreSQL 集群
|
||||
|
||||
暂不采用原因:会增加部署、迁移、HA、Secret、网络和容量范围,延迟核心架构验证。
|
||||
|
||||
### 方案 B:继续使用纯内存状态
|
||||
|
||||
拒绝原因:服务重启会丢失权威状态,无法进入预发布。
|
||||
|
||||
### 方案 C:每个模块各自使用 SQLite
|
||||
|
||||
拒绝原因:会形成多个事实源和跨模块事务问题。
|
||||
|
||||
### 方案 D:将完整状态写入 Git
|
||||
|
||||
拒绝原因:运行状态不应成为 Git 内容,可能暴露敏感业务数据并造成提交风暴。
|
||||
|
||||
## 6. 正面影响
|
||||
|
||||
- 候选版具备重启恢复能力;
|
||||
- 状态与代码版本解耦;
|
||||
- 回滚默认只切换代码,不覆盖新状态;
|
||||
- 可快速进入预发布测试;
|
||||
- 生产后端替换不改变上层 Contract。
|
||||
|
||||
## 7. 负面影响与风险
|
||||
|
||||
- SQLite 主要适用于单进程候选部署;
|
||||
- 尚未提供多实例写入和分布式锁;
|
||||
- 租户隔离是逻辑隔离,不是 RLS 或物理隔离;
|
||||
- 大规模任务、审计和并发容量尚未验证;
|
||||
- Evidence 文件数量增长需要归档策略;
|
||||
- Snapshot 粒度可能导致状态增长后的写放大。
|
||||
|
||||
## 8. 生产迁移要求
|
||||
|
||||
生产数据层替换必须满足:
|
||||
|
||||
1. 保持 Control Plane Domain/API Contract 不变;
|
||||
2. 提供 schema migration;
|
||||
3. 提供租户 RLS 或物理隔离;
|
||||
4. 提供备份、恢复、PITR 和灾难恢复;
|
||||
5. 提供 HA、多实例并发和故障转移;
|
||||
6. 提供容量、延迟和锁等待测试;
|
||||
7. 保持 Audit 与 Evidence 可追踪;
|
||||
8. 提供从 SQLite Candidate 到生产后端的数据迁移工具;
|
||||
9. 通过新的 ADR 说明生产持久化决策。
|
||||
|
||||
## 9. 回滚策略
|
||||
|
||||
- 回滚前备份 `/var/lib/aios-v2`;
|
||||
- 默认只回滚代码;
|
||||
- 状态恢复必须由 Owner 显式授权;
|
||||
- 恢复前必须验证 Snapshot schema 兼容性;
|
||||
- 回滚后执行 SQLite `PRAGMA quick_check` 和 AIOS Healthcheck。
|
||||
|
||||
## 10. 验收要求
|
||||
|
||||
候选版至少通过:
|
||||
|
||||
- 创建 Tenant、Agent、Task;
|
||||
- 服务重启;
|
||||
- 读取恢复后的 Task;
|
||||
- Idempotency 仍有效;
|
||||
- Audit 仍存在;
|
||||
- SQLite `quick_check=ok`;
|
||||
- 状态目录权限正确;
|
||||
- 回滚演练不丢失新增状态。
|
||||
|
||||
## 11. 实现与测试映射
|
||||
|
||||
- Repository:`active-runtime/app/aios_v2/persistence.py`
|
||||
- 应用入口:`active-runtime/app/aios_v2/app.py`
|
||||
- Evidence:`active-runtime/app/aios_v2/execution_tracking.py`
|
||||
- 测试:`tests/aios_v2_candidate/test_candidate_flow.py`
|
||||
- 健康检查:`deploy/aios-v2/healthcheck/check.sh`
|
||||
- 回滚:`deploy/aios-v2/rollback/rollback.sh`
|
||||
|
||||
## 12. 更新触发条件
|
||||
|
||||
以下变化必须更新或替代本 ADR:
|
||||
|
||||
- 引入生产数据库;
|
||||
- 支持多实例写入;
|
||||
- 修改 Snapshot 格式;
|
||||
- 改变 Evidence 存储;
|
||||
- 增加 RLS、HA、分片或事件溯源;
|
||||
- 修改状态回滚策略。
|
||||
@@ -0,0 +1,67 @@
|
||||
# AIOS V2 架构决策记录索引
|
||||
|
||||
- 文档版本:`V2R1C0-candidate`
|
||||
- 适用范围:影响多个模块、长期演进、兼容性、安全、数据、部署或交付流程的关键架构决策
|
||||
|
||||
## 1. 架构决策记录列表
|
||||
|
||||
| 编号 | 决策 | 状态 |
|
||||
|---|---|---|
|
||||
| [ADR-001](ADR-001-single-control-plane.md) | AIOS 作为唯一控制面 | 已采纳 |
|
||||
| [ADR-002](ADR-002-openclaw-hermes-decoupling.md) | OpenClaw 与 Hermes 从控制面职责中解耦 | 已采纳 |
|
||||
| [ADR-003](ADR-003-gitops-only-change-flow.md) | 所有变更必须通过 GitOps | 已采纳 |
|
||||
| [ADR-004](ADR-004-multi-tenant-isolation.md) | 多租户隔离采用默认拒绝模型 | 已采纳 |
|
||||
| [ADR-005](ADR-005-document-code-sync-gate.md) | 文档与代码同步门禁 | 已采纳 |
|
||||
| [ADR-006](ADR-006-candidate-persistence-strategy.md) | 候选持久化采用可替换 SQLite Snapshot | 已采纳 |
|
||||
|
||||
## 2. 状态定义
|
||||
|
||||
- **已提出**:决策已提出,但尚未被正式采纳。
|
||||
- **已采纳**:当前有效,所有新实现必须遵守。
|
||||
- **已替代**:已被新的 ADR 替代,保留用于历史追踪。
|
||||
- **已废弃**:保留历史,但禁止新实现继续采用。
|
||||
|
||||
## 3. 必须新增或更新 ADR 的场景
|
||||
|
||||
以下变化必须新增或更新 ADR:
|
||||
|
||||
1. AIOS、OpenClaw、Hermes、Agent Router 的控制边界变化;
|
||||
2. 多租户隔离模型变化;
|
||||
3. API、数据模型或状态机出现不兼容变化;
|
||||
4. 持久化、消息、密钥、审计或部署方案发生跨模块变化;
|
||||
5. GitOps、发布、回滚或文档门禁变化;
|
||||
6. 替换 OpenClaw、Hermes、Agent Router 或核心基础设施;
|
||||
7. 引入新的权威事实源、任务生命周期或租户状态;
|
||||
8. 生产安全模型、身份代理或 Secret 管理方式变化。
|
||||
|
||||
## 4. ADR 内容要求
|
||||
|
||||
每份 ADR 必须包含:
|
||||
|
||||
- 背景;
|
||||
- 决策;
|
||||
- 备选方案;
|
||||
- 采用理由;
|
||||
- 正面影响;
|
||||
- 负面影响和风险;
|
||||
- 迁移与回滚;
|
||||
- 合规检查;
|
||||
- 受影响的代码、测试和文档。
|
||||
|
||||
## 5. ADR 变更联动
|
||||
|
||||
ADR 新增、替代或废弃时,必须同步检查并更新:
|
||||
|
||||
- `02-TARGET-ARCHITECTURE.md`;
|
||||
- `03-MIGRATION-AND-DELIVERY.md`;
|
||||
- `04-MODULE-CONTRACTS.md`;
|
||||
- `05-TRACEABILITY-MATRIX.md`;
|
||||
- 相关代码、测试和部署资产;
|
||||
- `AIOS-V2R1-PRODUCTION-CANDIDATE.md` 中的风险与发布结论。
|
||||
|
||||
## 6. 审查规则
|
||||
|
||||
- OpenClaw 不得自行采纳改变控制面边界的 ADR;发现需求后必须升级给 Codex。
|
||||
- Codex 负责审查 ADR 与代码、测试和文档是否一致。
|
||||
- 涉及合并、Tag、生产部署或重大风险接受的 ADR,必须由 AIOS Owner 最终批准。
|
||||
- ADR 未完成审查前,相关变更不得标记为 `Test-verified` 或 `Production released`。
|
||||
@@ -1,268 +0,0 @@
|
||||
# AIOS Silent Release Controller Design Report
|
||||
|
||||
状态:隔离分支实现;生产 mutation 禁止
|
||||
目标:自动准备 → 自动验证 → 单次 Owner 签名授权 → 分控制器自动执行与验收
|
||||
|
||||
## 1. 当前痛点
|
||||
|
||||
现有流程需要多轮人工传递 commit、tree、artifact、manifest、测试、canary、drop-in
|
||||
和回滚证据。证据在不同消息里重复抄写,容易出现:
|
||||
|
||||
- 构建后 artifact、manifest 或 policy 已变化,但旧批准仍被使用;
|
||||
- Git、Gitea、systemd 与回滚结果由人工拼接,缺少同一一致性快照;
|
||||
- runtime switch、tag、Baseline、Registry 的授权边界不清;
|
||||
- 重启后只剩工作树内临时 JSON,无法可靠恢复;
|
||||
- approval replay、并发 writer、key revoke 和 secret inheritance 缺少统一防线。
|
||||
|
||||
## 2. 新架构
|
||||
|
||||
```text
|
||||
Build / Test / Canary
|
||||
|
|
||||
v
|
||||
ReleaseCandidate + DeploymentPlan
|
||||
|
|
||||
v
|
||||
ReleaseAcceptanceEngine
|
||||
|
|
||||
v
|
||||
READY_FOR_APPROVAL
|
||||
|
|
||||
v
|
||||
SignedReleasePlanApproval (Ed25519)
|
||||
|
|
||||
+--> runtime_switch capability
|
||||
| DeploymentController
|
||||
|
|
||||
+--> tag_publish capability
|
||||
| TagController
|
||||
|
|
||||
+--> baseline_registry_finalize capability
|
||||
BaselineRegistryFinalizeController
|
||||
|
||||
Durable ReleaseSagaStore + CapabilityLedger + Event Journal
|
||||
贯穿三个控制器;控制器互相不拥有对方的 mutation API。
|
||||
```
|
||||
|
||||
一次 Owner 批准是同一个不可变 release plan 的签名,不是一个万能执行入口。三个 capability
|
||||
分别消费、分别留痕、分别幂等恢复。`force_push`、`tag_rewrite`、`direct_main_push`
|
||||
永远不是可签发 capability。
|
||||
|
||||
## 3. 状态机
|
||||
|
||||
```text
|
||||
PREPARING
|
||||
-> VERIFYING
|
||||
-> READY_FOR_APPROVAL
|
||||
-> APPROVED
|
||||
-> PRE_SWITCH_REVALIDATING
|
||||
-> SWITCH_PENDING
|
||||
-> SWITCHED
|
||||
-> RUNTIME_VERIFYING
|
||||
-> TAG_PENDING
|
||||
-> TAG_VERIFIED
|
||||
-> BASELINE_REGISTRY_PENDING
|
||||
-> COMPLETED
|
||||
|
||||
任一可恢复故障 -> RECONCILIATION_REQUIRED
|
||||
对象冲突 -> MANUAL_OWNER_REVIEW_REQUIRED
|
||||
切换后失败 -> ROLLBACK_PENDING -> ROLLED_BACK
|
||||
```
|
||||
|
||||
状态快照使用 revision compare-and-set;事件同时写入 append-only journal。writer 崩溃后根据
|
||||
状态确定唯一恢复动作。并发控制器中只有一个 CAS winner。
|
||||
|
||||
## 4. 数据模型
|
||||
|
||||
### ReleaseCandidate
|
||||
|
||||
```text
|
||||
candidate_id
|
||||
source_commit
|
||||
tree_oid
|
||||
artifact_sha256
|
||||
manifest_sha256
|
||||
tests
|
||||
canary
|
||||
rollback_ready
|
||||
git_consistency
|
||||
gitea_consistency
|
||||
approval_state
|
||||
```
|
||||
|
||||
### DeploymentPlan
|
||||
|
||||
```text
|
||||
service
|
||||
release_path / release_root
|
||||
dropin_path / dropin_sha256
|
||||
rollback_path / rollback_sha256
|
||||
source_commit
|
||||
artifact_sha256
|
||||
manifest_sha256
|
||||
policy_sha256
|
||||
health_contract
|
||||
sha256
|
||||
```
|
||||
|
||||
### SignedReleasePlanApproval
|
||||
|
||||
```text
|
||||
release_plan_id
|
||||
candidate_id
|
||||
source_commit
|
||||
tree_oid
|
||||
artifact_sha256
|
||||
manifest_sha256
|
||||
deployment_plan_sha256
|
||||
policy_sha256
|
||||
approval_plan_id
|
||||
evidence_closure_oid
|
||||
owner_approval_record_oid
|
||||
capabilities
|
||||
expires_at
|
||||
nonce
|
||||
key_id
|
||||
signature
|
||||
fingerprint
|
||||
```
|
||||
|
||||
签名内容使用 canonical JSON,`signature` 不参与被签名字段。验证包括 Ed25519 签名、过期时间、
|
||||
公钥状态、approval revoke、candidate/plan/policy 逐字段绑定和 capability allowlist。
|
||||
|
||||
## 5. API
|
||||
|
||||
```text
|
||||
POST /createReleaseCandidate
|
||||
GET /getReleaseCandidate?candidate_id=...
|
||||
GET /getApprovalEnvelope?candidate_id=...
|
||||
POST /submitOwnerApproval
|
||||
POST /startDeployment
|
||||
GET /getReleaseSaga?candidate_id=...
|
||||
```
|
||||
|
||||
写接口要求 Bearer scope。`AIOS_SILENT_RELEASE_EXECUTION_ENABLED=false` 是默认值;此时可以
|
||||
持久化 candidate、生成 approval envelope、保存经验证的 Owner approval 和读取 Saga,
|
||||
但 `/startDeployment` 只返回 `SHADOW_MODE` 与空 `side_effects`。
|
||||
|
||||
## 6. PreSwitchValidator
|
||||
|
||||
切换前重新收集并比较:
|
||||
|
||||
- source commit 与 tree;
|
||||
- artifact、manifest、policy 和 deployment plan SHA-256;
|
||||
- rollback readiness;
|
||||
- local、authoritative bare 与 Gitea refs;
|
||||
- repository identity;
|
||||
- task writer/production writer 边界;
|
||||
- health contract。
|
||||
|
||||
任一变化都抛出:
|
||||
|
||||
```text
|
||||
APPROVAL_STALE_REAPPROVAL_REQUIRED
|
||||
```
|
||||
|
||||
旧批准不会被“最佳努力”继续使用。
|
||||
|
||||
## 7. systemd 防线
|
||||
|
||||
- service 与 drop-in 均为显式 allowlist;
|
||||
- release 与 drop-in resolved path 必须位于 release root 内;
|
||||
- 拒绝普通文件被替换为 symlink,拒绝 symlink escape;
|
||||
- deployment lock 覆盖 switch、restart 和首次 health check;
|
||||
- restart 前后记录 MainPID 与 ActiveEnterTimestampMonotonic;
|
||||
- artifact、drop-in、rollback 和 manifest 均按 plan hash 复验;
|
||||
- 子进程只接收固定 allowlist 环境,不继承 token、password、credential、askpass、
|
||||
auth socket、`LD_PRELOAD`、`PYTHONPATH` 或数据库连接串。
|
||||
|
||||
## 8. TagController 分离
|
||||
|
||||
TagController 必须独立完成 approval、execute、verify,并留下:
|
||||
|
||||
```text
|
||||
evidence_closure_oid
|
||||
approval_plan_id
|
||||
repository_identity
|
||||
remote_tag_oid
|
||||
remote_peeled_target_oid
|
||||
gitea_api_tag_oid
|
||||
gitea_listing_visible
|
||||
```
|
||||
|
||||
同名 tag 的 tag OID 或 peeled target 不同,进入
|
||||
`MANUAL_OWNER_REVIEW_REQUIRED`;自动流程不 push、不删除、不重写、不 force push。
|
||||
|
||||
## 9. Baseline / Registry 门禁
|
||||
|
||||
只有以下全部一致时,`baseline_registry_finalize` capability 才能完成:
|
||||
|
||||
```text
|
||||
Registry = Baseline = remote tag = Gitea API = expected target
|
||||
```
|
||||
|
||||
UNKNOWN、API 不可达、repository identity 不符或 listing 不可见均 fail closed。
|
||||
|
||||
## 10. Policy as Code
|
||||
|
||||
```text
|
||||
automatic:
|
||||
build, test, artifact, canary, evidence
|
||||
|
||||
owner:
|
||||
signed_release_plan
|
||||
|
||||
capabilities:
|
||||
runtime_switch
|
||||
tag_publish
|
||||
baseline_registry_finalize
|
||||
|
||||
forbidden:
|
||||
force_push
|
||||
tag_rewrite
|
||||
direct_main_push
|
||||
baseline_mutation
|
||||
registry_mutation
|
||||
```
|
||||
|
||||
最后两个 forbidden 项指“脱离受治理 finalize controller 的直接 mutation”。
|
||||
|
||||
## 11. Security model
|
||||
|
||||
- Ed25519 公钥 ring 支持 rotation 和 revoke,私钥不进入 runtime;
|
||||
- approval plan 可撤销,过期、未知 key、错误签名均 fail closed;
|
||||
- nonce 与 capability operation 持久化,重复相同请求幂等,不同 fingerprint 重放拒绝;
|
||||
- Saga snapshot 原子写、journal append-only、CAS 防并发 writer;
|
||||
- credential 只通过受控 reference 使用,不进入 Git、API 响应或异常;
|
||||
- 子进程环境从空集合构建,不复制 `os.environ`;
|
||||
- Shadow 模式默认开启,execution adapter 缺失时不能降级执行;
|
||||
- runtime、tag、Baseline/Registry 分属三个能力最小化控制器。
|
||||
|
||||
## 12. Migration plan
|
||||
|
||||
1. Shadow:生成 candidate、plan、acceptance、approval package,仅比较现有人工结果。
|
||||
2. Signed approval:接入 Owner 公钥 ring、撤销源和审计 record;仍不接生产 executor。
|
||||
3. Canary execution:使用非生产 service/release root 验证锁、PID、health、崩溃恢复和回滚。
|
||||
4. Controlled production switch:另行 Owner 授权启用 execution flag,只开放 runtime capability。
|
||||
5. Tag/finalize:分别完成 Gitea 与 Registry/Baseline 生产验收后再启用对应 capability。
|
||||
|
||||
每个阶段均可退回上一个阶段;不开启下一能力不会影响已经启用的控制器。
|
||||
|
||||
## 13. Rollback plan
|
||||
|
||||
- switch 前保存旧 symlink target、PID、启动时间、artifact 与 plan fingerprint;
|
||||
- restart 或 health 任一失败进入 `ROLLBACK_PENDING`;
|
||||
- 原子恢复旧 target,只 restart 指定 allowlist service;
|
||||
- 重新验证旧 runtime commit、writer=false 与 repository identity;
|
||||
- 成功进入 `ROLLED_BACK`,失败进入 `MANUAL_OWNER_REVIEW_REQUIRED`;
|
||||
- rollback 不修改 Git main、production tag、Baseline 或 Registry;
|
||||
- 保留脱敏 journal、approval fingerprint 和失败证据。
|
||||
|
||||
## 14. 测试与外部依赖边界
|
||||
|
||||
自动测试覆盖签名伪造、字段篡改、过期/revoke/key rotation、nonce replay、CAS race、
|
||||
崩溃恢复、artifact/policy/plan mutation、tag conflict、secret leakage、runtime without
|
||||
`.git`、rollback 和 shadow no-side-effects。
|
||||
|
||||
本隔离实现不创建生产 KMS/key、不安装 systemd unit、不部署 Action runtime、不创建 tag,
|
||||
也不修改 Baseline/Registry。上述动作必须在单独生产批准中执行。
|
||||
@@ -1,107 +0,0 @@
|
||||
# AIOS Silent Release Execution Readiness
|
||||
|
||||
## Scope and stop point
|
||||
|
||||
This change makes the controller execution-ready while deploying only a
|
||||
localhost-bound Shadow control plane. Both production gates remain false:
|
||||
|
||||
```text
|
||||
AIOS_GITOPS_WRITES_ENABLED=false
|
||||
AIOS_SILENT_RELEASE_EXECUTION_ENABLED=false
|
||||
```
|
||||
|
||||
Shadow completion is evidence preparation, not a production release. It cannot
|
||||
write a production tag, switch a business runtime, or mutate Baseline/Registry.
|
||||
|
||||
## Recovery state machine
|
||||
|
||||
Every expired or `FAILED_RECOVERABLE` capability requires fresh, operation-bound
|
||||
probe evidence before another adapter may run:
|
||||
|
||||
```text
|
||||
NOT_APPLIED -> one new exclusive claim may retry
|
||||
APPLIED_VERIFIED -> mark COMPLETED without adapter replay
|
||||
INDETERMINATE -> RECONCILIATION_REQUIRED; no automatic writes
|
||||
CONFLICT -> MANUAL_OWNER_REVIEW_REQUIRED; no automatic writes
|
||||
```
|
||||
|
||||
Probe evidence binds the immutable operation digest, is freshness checked, and
|
||||
is appended to a separate SHA-256 hash-chained journal. Completion still
|
||||
requires the active claim ID. File locks and leases ensure that concurrent
|
||||
recovery workers cannot execute the same adapter.
|
||||
|
||||
`TAG_PENDING` uses independent remote ref, annotated object, peeled target,
|
||||
Gitea API, Gitea listing, and repository identity evidence. Exact absence may
|
||||
roll back a switched runtime. Exact publication continues to finalization.
|
||||
Unavailable/divergent evidence stops at reconciliation; a different OID or
|
||||
target stops for Owner review. No branch can force, delete, or rewrite a tag.
|
||||
|
||||
## Trusted build and approval
|
||||
|
||||
Callers submit only an event ID and exact source commit. Test, canary, artifact,
|
||||
manifest, rollback, secret scan, Git and Gitea evidence are generated by the
|
||||
trusted builder. The builder emits an Ed25519 attestation binding:
|
||||
|
||||
- builder attestation/key IDs;
|
||||
- source commit and tree OID;
|
||||
- artifact, manifest and package-closure SHA-256;
|
||||
- test and canary evidence SHA-256;
|
||||
- issuance time and signature.
|
||||
|
||||
The controller contains only trusted public keys. Production signing is exposed
|
||||
through a KMS/HSM/secret-manager adapter; a local private-key adapter is marked
|
||||
test-only and is used only for ephemeral Shadow acceptance.
|
||||
|
||||
Owner approvals are immutable, time-bounded, revocable, trust-domain bound, and
|
||||
single-use per capability ledger. If the production approval backend reference
|
||||
is absent, production execution is unavailable while Shadow remains available.
|
||||
Caller-supplied `owner_id` is never a fallback.
|
||||
|
||||
## Persistent controller and API
|
||||
|
||||
`aios-silent-release-controller.service` runs independently from the business
|
||||
GitOps API. It uses a single leader lock, durable intents/candidates/Sagas,
|
||||
startup journal verification, idempotent main events, one notification record,
|
||||
and recovery reporting. The final deployed service is report-only Shadow mode.
|
||||
|
||||
The Action contract exposes:
|
||||
|
||||
- `createReleaseIntent`
|
||||
- `getReleaseCandidate`
|
||||
- `getReleasePlan`
|
||||
- `getReleaseAcceptance`
|
||||
- `submitSignedApproval`
|
||||
- `getReleaseSaga`
|
||||
- `getCapabilityStatus`
|
||||
- `startDeployment`
|
||||
- `getRecoveryStatus`
|
||||
|
||||
Every Shadow response declares `execution_enabled=false`, `shadow_mode=true`
|
||||
and `side_effects_executed=false`. `startDeployment` returns a simulated
|
||||
`SHADOW_COMPLETED` result and never claims production capabilities.
|
||||
|
||||
## Migration and rollback
|
||||
|
||||
1. Merge through a governed pull request.
|
||||
2. Build from a clean detached worktree at the exact merge commit.
|
||||
3. Verify artifact, manifest, package closure, tests, live fixture, no-.git
|
||||
canary, rollback archive and secret scan.
|
||||
4. Install a versioned Shadow release without changing the business runtime.
|
||||
5. Start the separate localhost-only controller with both gates false.
|
||||
6. Run Shadow candidates and freeze production PID/tag/Baseline/Registry hashes.
|
||||
7. Remove ephemeral test keys and temporary repositories.
|
||||
|
||||
Rollback stops and disables only the Shadow controller and restores/removes its
|
||||
versioned `current` link. The existing production API, tag set, Baseline and
|
||||
Registry are not involved.
|
||||
|
||||
## Future activation envelope
|
||||
|
||||
Activation requires a new one-time Owner approval binding the exact controller
|
||||
commit, artifact/manifest/policy/unit hashes, trusted key IDs, KMS/secret
|
||||
references, release and rollback roots, authoritative Git/repository identity,
|
||||
Shadow candidate/evidence IDs, test results, activation plan and rollback plan.
|
||||
That approval may only flip
|
||||
`AIOS_SILENT_RELEASE_EXECUTION_ENABLED=false` to `true`; each real release still
|
||||
requires its own signed ReleasePlan. This change does not request or consume
|
||||
that activation approval.
|
||||
@@ -1,84 +0,0 @@
|
||||
# Silent Release Controller Production Contract
|
||||
|
||||
Status: review-only implementation. Production execution remains disabled.
|
||||
|
||||
## Trust boundary
|
||||
|
||||
The caller submits only release intent. A configured trusted builder owns ref
|
||||
validation, build, tests, artifact and manifest hashing, package closure,
|
||||
canary, rollback readiness, Git/Gitea consistency, and secret scanning. Caller
|
||||
supplied success evidence is rejected.
|
||||
|
||||
The resulting immutable `ReleasePlan` contains deployment, tag, and
|
||||
finalization plans. One Ed25519 owner signature binds its SHA-256 digest,
|
||||
candidate evidence, policy digest, owner identity, trust domain, validity
|
||||
window, evidence lineage, and three narrow capabilities.
|
||||
|
||||
## Durable execution
|
||||
|
||||
The Saga journal is the source of truth. Every record is sequence-numbered and
|
||||
SHA-256 hash-chained from a fixed genesis hash. Snapshots are rebuildable
|
||||
caches. Journal corruption, truncation, reordering, missing records, or hash
|
||||
mismatch moves the release to `MANUAL_OWNER_REVIEW_REQUIRED`.
|
||||
|
||||
Capability execution uses an exclusive, leased claim. A second worker receives
|
||||
`BUSY`; an expired claim is not reassigned until a capability-specific
|
||||
side-effect probe reports that the operation was not applied. Runtime, tag,
|
||||
and finalization remain separately claimed and recoverable.
|
||||
|
||||
## Execution order
|
||||
|
||||
```text
|
||||
APPROVED
|
||||
-> PRE_SWITCH_REVALIDATING
|
||||
-> SWITCH_PENDING
|
||||
-> SWITCHED
|
||||
-> RUNTIME_VERIFYING
|
||||
-> RUNTIME_VERIFIED
|
||||
-> TAG_PENDING
|
||||
-> TAG_VERIFIED
|
||||
-> BASELINE_REGISTRY_PENDING
|
||||
-> COMPLETED
|
||||
```
|
||||
|
||||
The actual adapter configuration is regenerated before execution and must
|
||||
match the signed plan. Trusted allowlists come from controlled policy, not the
|
||||
request.
|
||||
|
||||
A failure before verified tag publication rolls the runtime back. A failure
|
||||
after the exact tag is verified enters `RECONCILIATION_REQUIRED`; the tag is
|
||||
never deleted or rewritten. A same-name/different-OID conflict enters
|
||||
`MANUAL_OWNER_REVIEW_REQUIRED`. Registry/Baseline finalization requires exact
|
||||
remote and Gitea tag proof, and Baseline is recorded last.
|
||||
|
||||
## Action and recovery contract
|
||||
|
||||
`submitOwnerApproval` verifies and durably stores the signed plan, then moves
|
||||
the Saga to `APPROVED`. `startDeployment` loads that durable approval and
|
||||
ignores caller-supplied approval content. Enabling execution without the
|
||||
trusted builder, verifier, validator, policy, and orchestrator fails during
|
||||
service construction.
|
||||
|
||||
The recovery worker scans incomplete journals and resumes through the same
|
||||
claims and verification gates. It never automatically resumes a manual-review
|
||||
Saga. In shadow mode it only reports state and performs no adapter side
|
||||
effects.
|
||||
|
||||
The deployed Action factory does not enable the legacy caller-evidence path.
|
||||
If the trusted builder is absent, candidate creation is rejected even while
|
||||
execution remains in shadow mode. The compatibility path exists only through
|
||||
the explicit test-only `SilentReleaseActionService.shadow()` constructor.
|
||||
|
||||
Finalization is composed from separate `RegistryController` and
|
||||
`BaselineController` claims. Registry is verified first and Baseline is the
|
||||
last mutation; a retry reuses completed Registry evidence and never moves
|
||||
Baseline ahead of tag verification.
|
||||
|
||||
## Governance
|
||||
|
||||
- `force_push`: forbidden
|
||||
- `tag_rewrite`: forbidden
|
||||
- `direct_main_push`: forbidden
|
||||
- production execution default: disabled
|
||||
- production tag operations in this change: not executed
|
||||
- Baseline/Registry mutations in this change: not executed
|
||||
@@ -1,68 +0,0 @@
|
||||
# Gitea Tag Publication Atomicity
|
||||
|
||||
Release publication is a fail-closed Saga bound to one immutable governance
|
||||
fingerprint:
|
||||
|
||||
- `version`
|
||||
- `target_oid`
|
||||
- `evidence_closure_oid`
|
||||
- `approval_plan_id`
|
||||
|
||||
The normal progression is:
|
||||
|
||||
`PENDING` → `LOCAL_TAG_CREATED` → `REMOTE_PUSH_CONFIRMED` →
|
||||
`GITEA_API_CONFIRMED` → `BASELINE_UPDATE_PENDING` → `BASELINE_UPDATED` →
|
||||
`PRODUCTION_DEPLOYED`
|
||||
|
||||
An unavailable remote uses `REMOTE_PUSH_PENDING` or
|
||||
`RECONCILIATION_REQUIRED`. A same-name tag object mismatch, peeled-target
|
||||
mismatch, changed governance fingerprint, or immutable deployment-evidence
|
||||
conflict enters `MANUAL_OWNER_REVIEW_REQUIRED`. That state blocks automatic
|
||||
push and reconciliation; recovery never deletes, rewrites, or force-pushes a
|
||||
tag.
|
||||
|
||||
## Legacy state migration
|
||||
|
||||
Readers normalize the historical `TAG_CREATED_LOCAL` value to
|
||||
`LOCAL_TAG_CREATED` in memory. Read-only status responses therefore expose only
|
||||
the canonical value. The next governed Saga write persists the canonical name;
|
||||
reading legacy state alone does not mutate the tag, Baseline, Registry, or Saga
|
||||
file. Any unrecognized persisted state fails closed as
|
||||
`RECONCILIATION_REQUIRED`.
|
||||
|
||||
`BASELINE_UPDATE_PENDING` is persisted before either `baseline/current.json` or
|
||||
`versions/<version>.json` changes. Recovery revalidates the governance
|
||||
fingerprint, evidence lineage, remote annotated tag, peeled target, Gitea API,
|
||||
listing, and repository identity before idempotently completing matching
|
||||
partial records. A different tag or target enters
|
||||
`MANUAL_OWNER_REVIEW_REQUIRED`.
|
||||
|
||||
## Durable state
|
||||
|
||||
Production writers set `AIOS_GITOPS_PUBLICATION_STATE_DIR` to storage outside
|
||||
their disposable worktree. Recreating a writer must reuse that directory.
|
||||
Worktree JSON is not the authoritative Saga store.
|
||||
|
||||
## Deployment gate
|
||||
|
||||
`mark_production_deployed` performs fresh Git remote, peeled-target, Gitea ref,
|
||||
Gitea tag-object, listing, repository-identity, evidence-lineage, Registry, and
|
||||
Baseline checks. Deployment is allowed only when:
|
||||
|
||||
`Registry = Baseline = remote tag = Gitea API = expected target`
|
||||
|
||||
Deployment evidence must be an immutable `sha256:<64 hex>` or `git:<40 hex>`
|
||||
identifier.
|
||||
|
||||
## Read-only acceptance
|
||||
|
||||
- `getTagPublicationStatus` reads durable Saga state.
|
||||
- `getReleaseConsistency` compares every release authority.
|
||||
- `runReleaseAcceptance` returns `accepted: true` only for `PASS`.
|
||||
|
||||
Unavailable evidence returns `UNKNOWN` with `fail_closed: true`; it is never
|
||||
treated as acceptance.
|
||||
|
||||
The CLI, HTTP Action, and release script all enter the same publisher core for
|
||||
evidence-lineage verification. The release script stages Registry and Baseline
|
||||
records only; durable Saga state remains outside Git.
|
||||
@@ -1,68 +0,0 @@
|
||||
# Isolated Writer Exact-Path Policy V1
|
||||
|
||||
This policy is an additional, service-owned authorization layer for
|
||||
`POST /api/commit-file`. It never expands a Token Policy V2 grant.
|
||||
|
||||
An atomic write is permitted only when all of the following agree:
|
||||
|
||||
1. socket peer, audience, operation, branch, base and exact path in the
|
||||
Token Policy V2 grant;
|
||||
2. service audience, repository identity, branch, operation and exact path
|
||||
in `AIOS_GITOPS_PATH_POLICY_FILE`;
|
||||
3. lexical path, resolved containment and symlink checks;
|
||||
4. the configured content-size and content-profile checks;
|
||||
5. operation digest, expected branch HEAD and durable operation ledger.
|
||||
|
||||
Any failed layer returns a fail-closed policy error. Coarse `/api/file` and
|
||||
`/api/commit` routes remain forbidden.
|
||||
|
||||
## File contract
|
||||
|
||||
The configured path must be absolute. The parent directory must be
|
||||
`root:gitops 0750`; the regular, non-symlink file must be
|
||||
`root:gitops 0640`. JSON must use UTF-8 canonical encoding with sorted keys,
|
||||
compact separators and one trailing newline. Unknown fields, unsupported
|
||||
operations or profiles, duplicate paths, oversized policies and
|
||||
group/world-writable files are rejected during Writer startup.
|
||||
|
||||
Example:
|
||||
|
||||
```json
|
||||
{"allowed_branch":"patch/codex-review-test-worker-20260724","allowed_operations":{"commit_file.atomic":{"content_profile":"systemd-codex-review-worker-v1","exact_paths":["deploy/systemd/aios-codex-review-test-worker.service"],"max_content_bytes":32768}},"audience":"aios-codex-review-worker-writer.service","policy_id":"codex-review-worker-path-policy-v1","repository_identity":"aios/aios","schema_version":1}
|
||||
```
|
||||
|
||||
The SHA-256 of the canonical policy is exposed in the isolated runtime
|
||||
status and written to the operation audit record. The request body cannot
|
||||
select or override the policy file.
|
||||
|
||||
## Content profile
|
||||
|
||||
`systemd-codex-review-worker-v1` is an exact, fail-closed unit grammar. The
|
||||
only sections are `[Unit]`, `[Service]` and `[Install]`. Each section has a
|
||||
directive allowlist; every single-value directive and every repeated
|
||||
`Environment`, `ReadOnlyPaths` and `ReadWritePaths` entry must match the
|
||||
governed Shadow unit contract exactly.
|
||||
|
||||
The profile fixes the Python module
|
||||
`task_plane.workers.codex_review_test`, `User=aiman`, `Group=gitops`,
|
||||
the controlled `/usr/libexec/aios-codex-review-test-worker/current`
|
||||
working directory, all hardening switches and the three permitted
|
||||
environment variables. It grants read access only to the authoritative
|
||||
bare repository and the versioned worker artifact, and write access only
|
||||
to the worker runtime, state and cache directories.
|
||||
|
||||
Unknown sections or directives are rejected, including lifecycle hooks,
|
||||
credential injection, environment imports, root images, bind mounts,
|
||||
capability expansion, devices, sockets and top-level writable paths. The
|
||||
parser also rejects duplicate sections, duplicate single-value directives,
|
||||
empty resets, continuation lines, CRLF, NUL, non-UTF-8 surrogate content,
|
||||
overlong lines and Unicode lookalike directive names. It does not use
|
||||
`ConfigParser(strict=False)` or silently merge repeated values.
|
||||
|
||||
Production credentials, if later authorized, must be supplied by a
|
||||
separately governed deployment drop-in. They are not permitted in the
|
||||
source unit or this content profile.
|
||||
|
||||
Policy changes require a separate maintenance branch and Owner-reviewed
|
||||
ordinary merge commit. A task-scoped Writer must not edit its own policy
|
||||
file.
|
||||
@@ -13,27 +13,6 @@ components:
|
||||
scheme: bearer
|
||||
bearerFormat: opaque-token
|
||||
schemas:
|
||||
PublicationState:
|
||||
type: string
|
||||
description: Canonical durable publication state. Legacy TAG_CREATED_LOCAL is normalized on read and is never emitted by new writes.
|
||||
enum:
|
||||
- PENDING
|
||||
- LOCAL_TAG_CREATED
|
||||
- REMOTE_PUSH_PENDING
|
||||
- REMOTE_PUSH_CONFIRMED
|
||||
- GITEA_API_CONFIRMED
|
||||
- BASELINE_UPDATE_PENDING
|
||||
- BASELINE_UPDATED
|
||||
- PRODUCTION_DEPLOYED
|
||||
- RECONCILIATION_REQUIRED
|
||||
- MANUAL_OWNER_REVIEW_REQUIRED
|
||||
PublicationStatus:
|
||||
type: object
|
||||
required: [version, state]
|
||||
properties:
|
||||
version: {type: string}
|
||||
state: {$ref: '#/components/schemas/PublicationState'}
|
||||
fail_closed: {type: boolean}
|
||||
BranchRequest:
|
||||
type: object
|
||||
required: [branch]
|
||||
@@ -51,31 +30,6 @@ components:
|
||||
path: {type: string, example: docs/gitops/chatgpt-change.md}
|
||||
content: {type: string}
|
||||
message: {type: string}
|
||||
AtomicFileCommitRequest:
|
||||
type: object
|
||||
additionalProperties: false
|
||||
required: [branch, path, content, message, expected_head_oid, idempotency_key]
|
||||
properties:
|
||||
branch:
|
||||
type: string
|
||||
pattern: '^patch/[A-Za-z0-9._/-]+$'
|
||||
path:
|
||||
type: string
|
||||
minLength: 1
|
||||
maxLength: 512
|
||||
content:
|
||||
type: string
|
||||
maxLength: 1048576
|
||||
message:
|
||||
type: string
|
||||
minLength: 1
|
||||
maxLength: 200
|
||||
expected_head_oid:
|
||||
type: string
|
||||
pattern: '^[0-9a-f]{40}$'
|
||||
idempotency_key:
|
||||
type: string
|
||||
pattern: '^[A-Za-z0-9][A-Za-z0-9._:-]{7,127}$'
|
||||
CommitRequest:
|
||||
type: object
|
||||
properties:
|
||||
@@ -96,20 +50,9 @@ components:
|
||||
message: {type: string}
|
||||
TagRequest:
|
||||
type: object
|
||||
required: [version, target_oid, evidence_closure_oid, approval_plan_id]
|
||||
required: [version]
|
||||
properties:
|
||||
version: {type: string, example: V1R1C1}
|
||||
target_oid:
|
||||
type: string
|
||||
pattern: '^[0-9a-fA-F]{40}$'
|
||||
description: Full approved commit OID. The tag is created explicitly at this commit, never implicitly at workspace HEAD.
|
||||
evidence_closure_oid:
|
||||
type: string
|
||||
pattern: '^[0-9a-fA-F]{40}$'
|
||||
description: Required evidence closure commit. target_oid must be its ancestor.
|
||||
approval_plan_id:
|
||||
type: string
|
||||
description: Audited owner-approved release or tag plan identifier.
|
||||
message: {type: string}
|
||||
execute:
|
||||
type: boolean
|
||||
@@ -160,216 +103,7 @@ components:
|
||||
properties:
|
||||
text: {type: string}
|
||||
branch: {type: string}
|
||||
DeploymentPlan:
|
||||
type: object
|
||||
required: [service, release_path, release_root, cutover_link, dropin_path, dropin_sha256, systemd_unit_sha256, rollback_path, rollback_sha256, source_commit, tree_oid, artifact_sha256, manifest_sha256, package_closure_sha256, authoritative_git_dir, repository_identity, writer_expected, baseline_before_sha256, registry_before_sha256, production_tag_refset_before_sha256, health_contract_version, timeout_policy, retry_policy, policy_sha256, health_contract]
|
||||
properties:
|
||||
service: {type: string}
|
||||
release_path: {type: string}
|
||||
release_root: {type: string}
|
||||
cutover_link: {type: string}
|
||||
dropin_path: {type: string}
|
||||
dropin_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
systemd_unit_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
rollback_path: {type: string}
|
||||
rollback_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
source_commit: {type: string, pattern: '^[0-9a-f]{40}$'}
|
||||
tree_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
|
||||
artifact_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
manifest_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
package_closure_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
authoritative_git_dir: {type: string}
|
||||
repository_identity: {type: string}
|
||||
writer_expected: {type: boolean, enum: [false]}
|
||||
baseline_before_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
registry_before_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
production_tag_refset_before_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
health_contract_version: {type: string}
|
||||
timeout_policy: {type: object}
|
||||
retry_policy: {type: object}
|
||||
policy_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
health_contract: {type: object, additionalProperties: true}
|
||||
ReleaseCandidateInput:
|
||||
type: object
|
||||
required: [source_commit, version]
|
||||
additionalProperties: false
|
||||
properties:
|
||||
candidate_id: {type: string, pattern: '^[A-Za-z0-9][A-Za-z0-9_.-]{2,127}$'}
|
||||
source_commit: {type: string, pattern: '^[0-9a-f]{40}$'}
|
||||
version: {type: string, pattern: '^V[0-9]+R[0-9]+C[0-9]+(?:[-.][A-Za-z0-9][A-Za-z0-9.-]*)?$'}
|
||||
ReleaseIntentInput:
|
||||
type: object
|
||||
required: [event_id, source_commit]
|
||||
additionalProperties: false
|
||||
properties:
|
||||
event_id: {type: string, pattern: '^[A-Za-z0-9][A-Za-z0-9_.:-]{2,255}$'}
|
||||
source_commit: {type: string, pattern: '^[0-9a-f]{40}$'}
|
||||
SignedReleasePlanApproval:
|
||||
type: object
|
||||
required: [release_plan_id, candidate_id, source_commit, tree_oid, artifact_sha256, manifest_sha256, deployment_plan_sha256, release_plan_sha256, policy_sha256, approval_plan_id, evidence_closure_oid, owner_approval_record_oid, capabilities, owner_identity, trust_domain, issued_at, not_before, expires_at, nonce, key_id, signature]
|
||||
properties:
|
||||
release_plan_id: {type: string}
|
||||
candidate_id: {type: string}
|
||||
source_commit: {type: string, pattern: '^[0-9a-f]{40}$'}
|
||||
tree_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
|
||||
artifact_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
manifest_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
deployment_plan_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
release_plan_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
policy_sha256: {type: string, pattern: '^[0-9a-f]{64}$'}
|
||||
approval_plan_id: {type: string}
|
||||
evidence_closure_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
|
||||
owner_approval_record_oid: {type: string, pattern: '^[0-9a-f]{40}$'}
|
||||
owner_identity: {type: string}
|
||||
trust_domain: {type: string}
|
||||
issued_at: {type: string, format: date-time}
|
||||
not_before: {type: string, format: date-time}
|
||||
capabilities:
|
||||
type: array
|
||||
uniqueItems: true
|
||||
items:
|
||||
type: string
|
||||
enum: [runtime_switch, tag_publish, baseline_registry_finalize]
|
||||
expires_at: {type: string, format: date-time}
|
||||
nonce: {type: string, minLength: 16}
|
||||
key_id: {type: string}
|
||||
signature: {type: string, format: byte, writeOnly: true}
|
||||
StartDeploymentRequest:
|
||||
type: object
|
||||
required: [candidate_id]
|
||||
additionalProperties: false
|
||||
properties:
|
||||
candidate_id: {type: string}
|
||||
paths:
|
||||
/createReleaseIntent:
|
||||
post:
|
||||
operationId: createReleaseIntent
|
||||
summary: Create an idempotent intent; all build evidence comes from the trusted builder
|
||||
security: [{bearerAuth: []}]
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema: {$ref: '#/components/schemas/ReleaseIntentInput'}
|
||||
responses:
|
||||
'200': {description: Intent created or idempotently returned}
|
||||
'400': {description: Caller evidence or invalid intent rejected}
|
||||
x-openai-isConsequential: true
|
||||
/createReleaseCandidate:
|
||||
post:
|
||||
operationId: createReleaseCandidate
|
||||
summary: Submit release intent to the trusted builder; caller evidence is rejected
|
||||
security: [{bearerAuth: []}]
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema: {$ref: '#/components/schemas/ReleaseCandidateInput'}
|
||||
responses:
|
||||
'200': {description: Candidate created or idempotently returned}
|
||||
'400': {description: Invalid or conflicting candidate evidence}
|
||||
'401': {description: Unauthorized}
|
||||
x-openai-isConsequential: true
|
||||
/getReleaseCandidate:
|
||||
get:
|
||||
operationId: getReleaseCandidate
|
||||
summary: Read a durable release candidate
|
||||
parameters:
|
||||
- {in: query, name: candidate_id, required: true, schema: {type: string}}
|
||||
responses:
|
||||
'200': {description: Candidate}
|
||||
'404': {description: Candidate not found}
|
||||
/getReleasePlan:
|
||||
get:
|
||||
operationId: getReleasePlan
|
||||
parameters:
|
||||
- {in: query, name: candidate_id, required: true, schema: {type: string}}
|
||||
responses:
|
||||
'200': {description: Immutable release plan}
|
||||
'404': {description: Candidate not found}
|
||||
/getReleaseAcceptance:
|
||||
get:
|
||||
operationId: getReleaseAcceptance
|
||||
parameters:
|
||||
- {in: query, name: candidate_id, required: true, schema: {type: string}}
|
||||
responses:
|
||||
'200': {description: Builder attestation and acceptance checks}
|
||||
'404': {description: Candidate not found}
|
||||
/getApprovalEnvelope:
|
||||
get:
|
||||
operationId: getApprovalEnvelope
|
||||
summary: Generate the immutable owner-signature payload
|
||||
parameters:
|
||||
- {in: query, name: candidate_id, required: true, schema: {type: string}}
|
||||
responses:
|
||||
'200': {description: Approval envelope fingerprint inputs}
|
||||
'404': {description: Candidate not found}
|
||||
/submitOwnerApproval:
|
||||
post:
|
||||
operationId: submitOwnerApproval
|
||||
summary: Verify and durably store a signed owner release plan
|
||||
security: [{bearerAuth: []}]
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema: {$ref: '#/components/schemas/SignedReleasePlanApproval'}
|
||||
responses:
|
||||
'200': {description: Signature verified and approval stored}
|
||||
'400': {description: Invalid approval}
|
||||
'403': {description: Signature, key, expiry, or revocation failure}
|
||||
'409': {description: Conflicting release plan}
|
||||
x-openai-isConsequential: true
|
||||
/submitSignedApproval:
|
||||
post:
|
||||
operationId: submitSignedApproval
|
||||
summary: Alias for immutable signed owner approval submission
|
||||
security: [{bearerAuth: []}]
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema: {$ref: '#/components/schemas/SignedReleasePlanApproval'}
|
||||
responses:
|
||||
'200': {description: Signature verified and approval stored}
|
||||
'403': {description: Production approval backend unavailable or signature invalid}
|
||||
x-openai-isConsequential: true
|
||||
/startDeployment:
|
||||
post:
|
||||
operationId: startDeployment
|
||||
summary: Resume the durable approved Saga; caller cannot replace approval
|
||||
security: [{bearerAuth: []}]
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema: {$ref: '#/components/schemas/StartDeploymentRequest'}
|
||||
responses:
|
||||
'200': {description: Shadow result or deployment Saga result}
|
||||
'403': {description: Execution disabled or approval invalid}
|
||||
'409': {description: Saga conflict}
|
||||
x-openai-isConsequential: true
|
||||
/getReleaseSaga:
|
||||
get:
|
||||
operationId: getReleaseSaga
|
||||
summary: Read durable Saga state and event journal
|
||||
parameters:
|
||||
- {in: query, name: candidate_id, required: true, schema: {type: string}}
|
||||
responses:
|
||||
'200': {description: Saga snapshot}
|
||||
'404': {description: Saga not found}
|
||||
/getCapabilityStatus:
|
||||
get:
|
||||
operationId: getCapabilityStatus
|
||||
parameters:
|
||||
- {in: query, name: candidate_id, required: true, schema: {type: string}}
|
||||
responses:
|
||||
'200': {description: Durable single-use capability records}
|
||||
/getRecoveryStatus:
|
||||
get:
|
||||
operationId: getRecoveryStatus
|
||||
responses:
|
||||
'200': {description: Fail-closed recovery classification}
|
||||
/api/base:
|
||||
get:
|
||||
operationId: getBaseline
|
||||
@@ -453,25 +187,6 @@ paths:
|
||||
schema: {$ref: '#/components/schemas/FileCommitRequest'}
|
||||
responses: {'200': {description: File committed}, '400': {description: Secret rejected or invalid path}, '401': {description: Unauthorized}}
|
||||
x-openai-isConsequential: true
|
||||
/api/commit-file:
|
||||
post:
|
||||
operationId: commitFileAtomic
|
||||
summary: Atomically write one allowlisted file, commit, push, and verify using Token Policy V2
|
||||
security: [{bearerAuth: []}]
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema: {$ref: '#/components/schemas/AtomicFileCommitRequest'}
|
||||
responses:
|
||||
'200': {description: Completed or idempotently replayed}
|
||||
'400': {description: Invalid canonical request}
|
||||
'401': {description: Missing, invalid, expired, or revoked Token Policy V2 grant}
|
||||
'403': {description: Audience, source, branch, operation, path, or policy denied}
|
||||
'409': {description: Idempotency conflict or manual review required}
|
||||
'413': {description: Request too large}
|
||||
'503': {description: Push or remote verification unavailable; recoverable}
|
||||
x-openai-isConsequential: true
|
||||
/api/commit:
|
||||
post:
|
||||
operationId: commitChanges
|
||||
@@ -484,39 +199,6 @@ paths:
|
||||
schema: {$ref: '#/components/schemas/CommitRequest'}
|
||||
responses: {'200': {description: Commit result}, '401': {description: Unauthorized}}
|
||||
x-openai-isConsequential: true
|
||||
/api/tag-publication-status:
|
||||
get:
|
||||
operationId: getTagPublicationStatus
|
||||
summary: Read durable publication Saga status without mutation
|
||||
parameters:
|
||||
- {in: query, name: version, required: true, schema: {type: string, pattern: '^V\d+R\d+C\d+(?:-[A-Za-z0-9_.-]+)?$'}}
|
||||
responses:
|
||||
'200':
|
||||
description: Canonical publication state or fail-closed UNKNOWN
|
||||
content:
|
||||
application/json:
|
||||
schema: {$ref: '#/components/schemas/PublicationStatus'}
|
||||
'400': {description: Version required}
|
||||
/api/release-consistency:
|
||||
get:
|
||||
operationId: getReleaseConsistency
|
||||
summary: Compare Registry, Baseline, remote tag, Gitea API, listing, repository identity, and expected target
|
||||
parameters:
|
||||
- {in: query, name: version, required: true, schema: {type: string}}
|
||||
- {in: query, name: target_oid, required: true, schema: {type: string, pattern: '^[0-9a-fA-F]{40}$'}}
|
||||
- {in: query, name: evidence_closure_oid, required: true, schema: {type: string, pattern: '^[0-9a-fA-F]{40}$'}}
|
||||
- {in: query, name: approval_plan_id, required: true, schema: {type: string}}
|
||||
responses: {'200': {description: PASS, FAIL, or fail-closed UNKNOWN}}
|
||||
/api/release-acceptance:
|
||||
get:
|
||||
operationId: runReleaseAcceptance
|
||||
summary: Run read-only release acceptance; UNKNOWN is not accepted
|
||||
parameters:
|
||||
- {in: query, name: version, required: true, schema: {type: string}}
|
||||
- {in: query, name: target_oid, required: true, schema: {type: string, pattern: '^[0-9a-fA-F]{40}$'}}
|
||||
- {in: query, name: evidence_closure_oid, required: true, schema: {type: string, pattern: '^[0-9a-fA-F]{40}$'}}
|
||||
- {in: query, name: approval_plan_id, required: true, schema: {type: string}}
|
||||
responses: {'200': {description: Acceptance result with accepted boolean}}
|
||||
/api/tag:
|
||||
post:
|
||||
operationId: requestOrCreateTag
|
||||
@@ -527,7 +209,7 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: {$ref: '#/components/schemas/TagRequest'}
|
||||
responses: {'200': {description: Gitea-confirmed tag and fail-closed baseline result}, '400': {description: Missing governance evidence or invalid OID}, '403': {description: Forbidden}, '409': {description: Existing tag target conflict or evidence lineage mismatch}, '502': {description: Gitea reconciliation required}, '503': {description: Remote push pending}}
|
||||
responses: {'200': {description: Tag request committed or tag created}, '403': {description: Forbidden}}
|
||||
x-openai-isConsequential: true
|
||||
/api/rollback:
|
||||
post:
|
||||
@@ -563,7 +245,4 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: {$ref: '#/components/schemas/NLRequest'}
|
||||
responses:
|
||||
'200': {description: Safe GitOps plan}
|
||||
'401': {description: Unauthorized}
|
||||
x-openai-isConsequential: true
|
||||
responses: {'
|
||||
@@ -1,56 +0,0 @@
|
||||
# V2R1C1 中央 Baseline 治理收口
|
||||
|
||||
## 收口范围
|
||||
|
||||
本次变更只登记已经完成生产验证并上线的 `V2R1C1`。它新增中央版本 Manifest、把中央 Baseline Pointer 从 `V1R4C6` 切换为 `V2R1C1`,并建立 Release、Companion、Artifact 与最终生产 Evidence 的追溯关系。
|
||||
|
||||
本次不修改产品实现,不移动或重建 `V2R1C1` Tag,不改变 Release OID,不重新部署生产,不切换生产流量,也不重写历史证据。
|
||||
|
||||
## 权威标识
|
||||
|
||||
- Release:`release/AIOS-V2R1C1-20260721`
|
||||
- Release OID:`4e0e6dcb90e6732aa9838fe9d0af0b34746829b0`
|
||||
- Tag / Tag OID:`V2R1C1` / `4e0e6dcb90e6732aa9838fe9d0af0b34746829b0`
|
||||
- SemVer:`2.1.0-rc.1`
|
||||
- OpenClaw Companion:`2.1.0-rc.4`
|
||||
- Companion Source OID:`83f8f878c7da5dd8263f2af2efa80750492e5cb3`
|
||||
- Companion OID:`12d1c13170e0c5a691d67a9408e739e7a17064a3`
|
||||
- Artifact SHA256:`aadd5f3cfa0d8e796a21f6368f5c95e8c60e9063d6e9fcd9c4947cfa0c013671`
|
||||
- Contract:`aios.openclaw.command.v2` / `Ed25519`
|
||||
- Evidence Root:`/var/lib/aios-gitops/acceptance/aios-v2r1c1-openclaw-v2-companion-20260721T094442Z`
|
||||
- Final Audit Manifest SHA256:`2cdb884abf195969de3cdaa44c76aaac52a9165fb4918436855291e46b2f8d55`
|
||||
- Production Status:`PRODUCTION_DEPLOYED`
|
||||
|
||||
## 治理路径
|
||||
|
||||
- Governance Branch:`patch/chatgpt/AIOS-V2R1C1-BASELINE-REGISTRY-20260721`
|
||||
- Base Ref:`refs/heads/main`
|
||||
- Base OID:`3cf9958cb94614449e093639cbf902216d73447f`
|
||||
- Manifest Commit:`a56caf46f6bc335ec2395b674ead4b2a427c5edd`
|
||||
- Baseline Commit:`6dbd5cbe07d2960ca653049fefbcf3a3e18f9ab1`
|
||||
- Reviewer:`gpt-5.3-codex-spark validator and docs_reporter`
|
||||
- Review:`PASS`(`2026-07-21T14:23:00Z`)
|
||||
|
||||
## 命令与退出码
|
||||
|
||||
| 阶段 | 命令 | 开始 UTC | 结束 UTC | Exit Code |
|
||||
| --- | --- | --- | --- | ---: |
|
||||
| 中央只读核对 | `GET /api/run; GET /api/base; GET /api/ver` | 2026-07-21T14:09:40Z | 2026-07-21T14:09:40Z | 0 |
|
||||
| 创建治理分支 | `bash /tmp/registry-create-branch.sh` | 2026-07-21T14:17:05Z | 2026-07-21T14:17:05Z | 0 |
|
||||
| 核心元数据验证 | `bash /tmp/registry-validate-core.sh` | 2026-07-21T14:19:56Z | 2026-07-21T14:19:56Z | 0 |
|
||||
|
||||
完整机读记录位于 `.agent/results/releases/V2R1C1-BASELINE-CLOSURE.json`,治理运行证据位于 `/var/lib/aios-gitops/acceptance/aios-v2r1c1-baseline-registry-20260721T141705Z`。
|
||||
|
||||
## 元数据回滚
|
||||
|
||||
若中央 Registry 解析或治理服务异常,保留 `versions/V2R1C1.json`,通过新的受审治理提交把 `baseline/current.json` 和 `VERSION` 临时恢复到 `V1R4C6`。该操作不得回滚生产系统、移动 Tag、修改 Release OID 或重写历史。
|
||||
|
||||
## 范围证明
|
||||
|
||||
```text
|
||||
product_code_changed=false
|
||||
tag_changed=false
|
||||
production_redeployed=false
|
||||
production_traffic_changed=false
|
||||
secret_material_recorded=false
|
||||
```
|
||||
@@ -1,35 +0,0 @@
|
||||
# V2R1C2 Production Release
|
||||
|
||||
- Release Run ID: `AIOS-V2R1C2-PRODUCTION-20260722T011836Z`
|
||||
- Previous production baseline: `V2R1C1`
|
||||
- Tag: `V2R1C2`
|
||||
- Tag object: `a3bd84ebec9c705f7c9c5c03f7f4ec55f8854e36`
|
||||
- Approved target: `3dd622b5a515f1fa4ef72b9f8ae9ce48ac703f24`
|
||||
- Evidence closure: `8cdca463aff9707edee00e20a3b56884130f6978`
|
||||
- Initial registry status: `TAG_CREATED_NOT_PROMOTED`
|
||||
- Final status: `PRODUCTION_DEPLOYED`
|
||||
- Production service: `aios-v2r1c2-production.service`
|
||||
- Production port: `18920`
|
||||
- Production acceptance SHA256: `a6c35e12cab7907ab6bcfd37d781b3d216f05a5ae002aa54a3112c70f43b991a`
|
||||
- Production online smoke SHA256: `fe9c6964e440f8871f2eb225f0132f70c634923389f7b8dfa85f615b100980a4`
|
||||
- Production health SHA256: `ee8084e8b57d188dc0039a276f4f3f1ed7025ff58ce91b111cc765627eb52d4c`
|
||||
- Live evidence directory: `/var/lib/aios-gitops/acceptance/AIOS-V2R1C2-PRODUCTION-20260722T011836Z`
|
||||
- Evidence hash index: `/var/lib/aios-gitops/acceptance/AIOS-V2R1C2-PRODUCTION-20260722T011836Z/production-evidence.sha256`
|
||||
|
||||
## Production gates
|
||||
|
||||
Promotion may proceed only when the 18921 candidate remains healthy, the committed evidence checksum file verifies without drift, the 18920 baseline is healthy, required secret references are available without exposing secret values, and the V2R1C1 rollback service/configuration remains intact.
|
||||
|
||||
The obsolete release-plan `DECLARED` rows are not authoritative for this run. The production gate must explicitly select `AIOS-V2R1C2-UNIFIED-PORTAL-TAKEOVER-20260722` and require its three required stages to be `SUCCEEDED`; an unqualified aggregate across stale plan rows fails closed.
|
||||
|
||||
## Synthetic WeChat scope
|
||||
|
||||
WeChat-facing acceptance for this release used synthetic/mock receipts and transcripts only. It did not impersonate a real user and does not claim a real-user WeChat session. The relevant committed evidence entries are `wechat_simulated_receipts` and `wechat_simulated_transcript`; the production online smoke also records `simulation: true` and `real_user_impersonated: false`.
|
||||
|
||||
## Rollback rule
|
||||
|
||||
If Health, API, UI, Role Matrix, Tenant Isolation, Runtime, or Audit verification fails after the 18920 switch, stop the V2R1C2 production unit and restart the preserved V2R1C1 18920 service. Do not delete or rewrite either release tag or rollback asset.
|
||||
|
||||
## Result
|
||||
|
||||
The production switch completed at `2026-07-22T01:40:51Z`. Full acceptance and the production online smoke passed, including Health, API, UI, Role Matrix, Tenant Isolation, Runtime, Audit and the selected takeover release gate. The previous `aios-orchestrator-gateway.service` unit, configuration and Release Run backup remain available for rollback.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user