Compare commits

...
Author SHA1 Message Date
CodexandV2R1 Controller a59f3e1db5 feat(spark-runner): implement AIOS Spark Runner provisioning with systemd, sysusers, tmpfiles, and fail-closed runner
Gate 0A: discovered existing patterns (Python services, systemd in deploy/systemd/,
suffix daemon.py pattern, services/ subdirectory layout, secret_ref convention).
Reuses the same service directory pattern, systemd hardening, sysusers/tmpfiles
declarations, and atomic JSON writes pattern from autonomous daemon.

Gate 0B: runner artifact and digest mapping uses the GitHub-style Git commit
reference. runner_secret_ref remains BLOCKED_GATEWAY_SECRET_REF_UNAVAILABLE
until an existing gateway secret ref is found and approved.

Gate 1: fresh worktree from baseline 8d9c6da on branch
patch/codex/aios-spark-runner-provisioning-20260717

Gate 2: provision.sh (idempotent, fail-closed), runner.py (heartbeat+worktree),
uninstall.sh (reversible), rollback.sh (backup revert), check-heartbeat.sh
(monitoring compatibility), README.md (operations guide).

Gate 3: systemd unit with NoNewPrivileges, ProtectSystem=full, ProtectHome=yes,
runtime/state directories, CapabilityBoundingSet= (fully empty), separate
aios-spark-runner user/group via sysusers.

Gate 4: heartbeat file at /run/aios-spark-runner/heartbeat, written by runner,
checked by check-heartbeat.sh with configurable max-age.

Co-authored-by: V2R1 Controller (大王) <orchestrator@xiaolongxia.team>
2026-07-17 16:03:05 +08:00
Codex 8d9c6da90f fix(openclaw): enforce B3 attestation trust chain 2026-07-16 21:22:36 +08:00
Codex 631fee9254 fix(openclaw): harden B3 evidence trust and path boundaries
- Add _reject_symlink_chain() protection against ancestor symlink escapes
- Implement dirfd-based atomic writes with O_DIRECTORY | O_NOFOLLOW
- Add mandatory --trust-policy parameter with strict validation
- Integrate SSH signature verification via ssh-keygen -Y verify
- Authority field hard fail (present: false or missing -> BLOCKED)
- Self-audit rejection (reviewer != executor/capture/credential)
- Update template with trust_policy_key field
- Update README with trust policy and dirfd documentation
- Update task file with B3 BLOCKED_EVIDENCE_REQUIRED status

Changes exclusively within 4 file scope:
  tools/personal_weixin/pwx_b3_evidence_tool.py
  tools/personal_weixin/pwx_b3_evidence_facts.template.json
  tools/personal_weixin/README-B3-EVIDENCE.md
  .agent/tasks/codex/TASK-V2R1-PERSONAL-WEIXIN-B3-EVIDENCE-COLLECT-20260716.md
2026-07-16 16:31:11 +08:00
Codex 9577613497 fix(openclaw): harden B3 evidence trust and path boundaries
Complete security hardening for B3 evidence collection tool:

Security fixes:
1. Ancestor symlink path escape prevention - reject_symlink_chain()
2. Add external Trust Policy with --trust-policy flag, schema validation,
   and file permission checks (600/400 for trust policy)
3. Authority present: false or missing now hard-fails (was soft-fail)
4. Self-review rejection - reviewer must differ from capture/credential
5. SSH signature verification with ssh-keygen -Y verify
6. SFTP trust policy delivery verification
7. Reject_Float nan/infinity in JSON parsing
8. JSON key duplication detection
9. Dirfd atomic writes with O_DIRECTORY|O_NOFOLLOW parent fd

Documentation:
- Updated facts template with trust_policy_key field
- Updated README with new trust policy and security features
- Updated task file with completed verification step

Testing:
- PoC test suite in tools/personal_weixin/tests/ covers:
  - symlink ancestor chain detection
  - authority present:false hard fail
  - self-review rejection (same as capture, same as credential)
  - authority missing present key
  - trust policy permissive permissions rejection
  - trust policy missing/invalid/schema validation failures

No gitleaks findings in modified files.
2026-07-16 15:37:46 +08:00
Codex a9a298f8a7 feat(codex): add personal weixin B3 evidence collection source 2026-07-16 13:56:24 +08:00
Codex 53c67bd978 task(codex): prepare Git-hosted B3 evidence tooling 2026-07-16 12:35:34 +08:00
Codex 1d334b1cca docs(control): disable B4 outbound quote authority 2026-07-16 11:56:19 +08:00
Codex a1f933e500 docs(auth): authorize B3 B4 evidence execution 2026-07-16 11:54:41 +08:00
aiman 74e603c0c9 docs(v2r1): record personal weixin F0 lineage 2026-07-16 09:52:45 +08:00
17 changed files with 2107 additions and 0 deletions
@@ -0,0 +1,397 @@
# Personal-Weixin B3/B4 Owner Evidence Authorization
```yaml
authorization_id: AUTH-CODEX-V2R1-PERSONAL-WEIXIN-B3-B4-EVIDENCE-20260716
owner_approved: true
executor: Codex isolated runner
foundation_implementation_commit: 2fcab6f5833a405a9a26683811600b6e5f1aa7b1
foundation_result_commit: 64a71b6199a8e2bd609ba145349e67b874888505
r10_lineage_commit: 74e603c0c943501110577dbe3c7f6a0b18b8ec7d
evidence_gate_commit: 62a4067cf7a462ebe7cf29524238bc5d0a70b813
upstream_audit_commit: b2f5f531019a70801444987dbb97dc1f69cf47de
production_baseline: V1R4C6
production_changed: false
release_status: paused
force_push_authorized: false
```
## 1. 授权结论
Owner 明确授权 Codex:
1. 完成 B3 owner evidence packet;
2. 在 vendor `client_id → authoritative message_id` 证据缺失时,采用 B4 quote-authority-disabled amendment;
3. B3、B4 均 PASS 后,按已有 exact allowlist 继续 B5–B7;
4. B3–B7 全部 PASS 后才能开始 Release Preflight;
5. Release Preflight PASS 后才允许 Canary;
6. Codex 不得直接修改生产。
该授权不允许 Codex编造 raw-wire、credential、vendor 或 reviewer 证据。
## 2. B3 owner evidence 必须包含
B3 evidence manifest 必须引用上述五个权威提交,并包含以下五类证据。
### A. Raw-wire evidence
至少包含:
```yaml
raw_wire:
- direction: inbound
- direction: outbound
```
每项必须记录:
```yaml
restricted_path:
restricted_sha256:
restricted_bytes:
redacted_path:
redacted_sha256:
captured_at:
credential_label: credential-a | credential-b
```
要求:
* unrestricted raw-wire 只能保存在 owner 批准的受限存储;
* Git 中只能提交 SHA-256、大小、受限路径引用及 redacted view;
* 不得提交 token、API key、OAuth code、sessionKey、cookie 或完整 credential;
* inbound 与 outbound 必须各至少一份;
-两份 credential 都必须实际覆盖。
### B. Authority map
必须提供 commit-safe authority map,明确:
```yaml
vendor_message_id_field:
client_id_field:
client_id_authoritative: false
mapping_verified: true
```
Authority map 必须区分:
```yaml
authoritative:
- verified inbound vendor message_id
non_authoritative:
- local client_id
- outbound correlation ID
- local sequence
- timestamp
- content hash
- outbound receipt ID
```
禁止把任一非权威值提升为 message authority。
### C. 双 credential 证据
必须有精确两项:
```yaml
credentials:
- label: credential-a
fingerprint_sha256:
secret_included: false
scope:
test_result: PASS
- label: credential-b
fingerprint_sha256:
secret_included: false
scope:
test_result: PASS
```
要求:
* 两个 fingerprint 必须不同;
* 不得提交 secret 原文;
* 两个 credential 必须各自通过实际测试;
* credential scope 必须明确。
### D. Mode evidence
必须明确处于下列模式之一:
```yaml
mode:
name: sandbox | shadow | restricted-capture
production_traffic: false
writes_production: false
capture_only: true
```
不得使用生产流量或写生产数据来补证据。
### E. Reviewer evidence
必须由独立 reviewer 签署:
```yaml
review:
reviewer:
independent_from_executor: true
decision: PASS
reviewed_manifest_sha256:
reviewed_at:
```
Reviewer 必须核对:
* raw-wire 哈希;
* redacted view;
* authority map;
-两个 credential fingerprint;
* mode;
* manifest 完整 SHA-256。
## 3. B3 PASS 条件
只有以下全部成立才允许 B3 `status: PASS`:
```yaml
raw_wire_inbound: PASS
raw_wire_outbound: PASS
dual_credential: PASS
authority_map: PASS
client_id_authoritative: false
mode_non_production: PASS
reviewer_independent: PASS
reviewer_decision: PASS
secrets_committed: false
production_changed: false
baseline_after: V1R4C6
release_status: paused
```
真实证据缺失时必须继续:
```yaml
status: BLOCKED_EVIDENCE_REQUIRED
```
不得生成模拟证据代替 owner/vendor 证据。
## 4. B4 Owner Amendment
Owner 明确批准以下 B4 amendment:
```json
{
"schemaVersion": 1,
"contract": "personal-weixin-b4-outbound-quote-authority-amendment",
"ownerApproved": true,
"lineage": {
"foundationImplementationCommit": "2fcab6f5833a405a9a26683811600b6e5f1aa7b1",
"foundationResultCommit": "64a71b6199a8e2bd609ba145349e67b874888505",
"r10LineageCommit": "74e603c0c943501110577dbe3c7f6a0b18b8ec7d",
"evidenceGateCommit": "62a4067cf7a462ebe7cf29524238bc5d0a70b813",
"upstreamAuditCommit": "b2f5f531019a70801444987dbb97dc1f69cf47de"
},
"vendorClientIdMessageIdEvidence": {
"status": "absent",
"clientIdPromotable": false
},
"authorityPolicy": {
"outboundQuoteAuthority": "disabled",
"clientIdRole": "non-authoritative-correlation-only",
"authoritativeMessageIdSources": [
"validated-inbound-raw-wire.vendor_message_id"
],
"quotedSendWithoutAuthoritativeMessageId": "fail-closed",
"plainOutboundWithoutQuoteAuthority": "allowed-only-by-existing-b4-allowlist",
"syntheticMapping": false,
"timestampFallback": false,
"localSequenceFallback": false,
"contentHashFallback": false,
"outboundReceiptFallback": false
},
"implementationConstraints": {
"scopeExpansion": false,
"exactAllowlistRequired": true,
"missingAllowlistStatus": "BLOCKED_SCOPE",
"productionChangeAuthorized": false,
"canaryAuthorized": false,
"tagAuthorized": false,
"forcePushAuthorized": false
},
"production": {
"baseline": "V1R4C6",
"changed": false,
"releaseStatus": "paused"
}
}
```
该 JSON 必须作为独立 control-plane amendment 提交。
## 5. B4 强制语义
B4 实现必须遵守:
```yaml
client_id:
role: correlation-only
authoritative: false
vendor_message_id:
authoritative_only_when:
- observed in validated inbound raw-wire
- present in committed authority map
quoted_send:
without_authoritative_vendor_message_id: fail-closed
plain_outbound:
allowed_only_when:
- existing B4 exact allowlist permits it
- no quote/reply authority field is emitted
```
禁止:
* `client_id` → `message_id` alias;
* client ID hash fallback;
* timestamp fallback;
* local sequence fallback;
* content hash fallback;
* outbound receipt fallback;
* synthetic message ID;
-“先发送后补 authority”。
## 6. B4 路径授权
该 amendment 不增加任何实现路径。
Codex必须先找到已有 B4 exact allowlist。
若不存在或不完整:
```yaml
status: BLOCKED_SCOPE
```
不得由 Codex自行发明路径。
## 7. B5–B7 条件
只有 B3 和 B4 均有 qualifying PASS RESULT 后,才可继续 B5–B7。
每个阶段必须:
* 独立 branch;
* 独立 worktree;
* 已提交 exact allowlist;
* 一个非 merge implementation commit;
* 独立 RESULT;
* 依赖实际 F0 commit 与 immutable image digest;
* 不修改生产;
* 不 force push。
任一阶段缺少 exact allowlist:
```yaml
status: BLOCKED_SCOPE
```
## 8. Release Preflight
只有以下全部 PASS 后才能开始:
```yaml
F0: PASS
B3: PASS
B4: PASS
B5: PASS
B6: PASS
B7: PASS
R10_lineage: PASS
immutable_image_digest: present
gitleaks: PASS
rollback_contract: present
```
Preflight 不得创建 tag 或部署生产。
## 9. Canary
仅在 Release Preflight PASS 后授权一次 Canary。
Canary 必须:
* 使用固定 immutable image digest;
* 使用已验证 rollback target;
* 不修改生产 schema;
* 不暴露 secrets;
* 有明确成功指标;
* 有自动/立即 rollback 条件。
Canary 失败:
```yaml
rollback: required
release_status: paused
production_baseline: V1R4C6
```
不得继续 production promotion。
## 10. Production 边界
Codex不得直接执行生产部署。
只有以下全部提交并核验后,Codex才可创建 deployment request:
```yaml
F0_and_B3_B7: PASS
release_preflight: PASS
canary: PASS
immutable_release_digest: present
rollback_target: verified
rollback_procedure: verified
go_live_evidence: committed
```
实际 V2R1C0 部署只能由:
```text
owner/admin GitOps release action
```
执行。
在该动作前:
```yaml
production: V1R4C6
production_changed: false
release: paused
V2R1C0_tag: absent
```
必须保持不变。
## 11. Codex 下一步
立即执行顺序:
1. 在现有 control branch 提交本授权文件;
2. 提交 byte-exact B4 amendment;
3. 检查 B3 packet 中是否已有真实 owner raw-wire、双 credential、authority map、mode 和 reviewer evidence;
4. 有真实证据则完成 B3 manifest 与审核;
5. 无真实证据则继续 `BLOCKED_EVIDENCE_REQUIRED`,明确缺失项;
6. B3 PASS 后按已有 exact allowlist执行 B3;
7. B4 按 quote-authority-disabled 模式执行;
8. B3/B4 PASS 后再检查 B5–B7;
9. 不得跳到 Release Preflight、Canary 或生产。
本授权已解除 B4 vendor mapping 阻塞,但没有伪造或豁免 B3 owner evidence。
@@ -0,0 +1,96 @@
# TASK-CODEX-V2R1-PERSONAL-WEIXIN-B3-EVIDENCE-COLLECT-20260716
```yaml
task_id: TASK-CODEX-V2R1-PERSONAL-WEIXIN-B3-EVIDENCE-COLLECT-20260716
executor: Codex isolated runner
source_base: 53c67bd9780074bad9781fb97971463156bc6609
authorization_commit: a1f933e500151dc8afc1b25b30448a31899a8f64 # gitleaks:allow
b4_amendment_commit: 1d334b1ccaac54c3c3ec4dd84693ab68fc67cef3
production: V1R4C6
release: paused
```
## Status
**B3 status**: `BLOCKED_EVIDENCE_REQUIRED` (evidence collection not started, owner must prepare facts and produce restricted raw data)
**B2 status**: `NOT_APPLICABLE` (reserved, not yet reached)
**Production baseline**: V1R4C6 (the most recent approved production release)
## Scope
本任务仅使用 Git 中的以下工具与模板,采集并验证真实 owner evidence:
```text
tools/personal_weixin/pwx_b3_evidence_tool.py
tools/personal_weixin/pwx_b3_evidence_facts.template.json
tools/personal_weixin/README-B3-EVIDENCE.md
```
工具是 verifier/generator,不是网络 capture client。不得在 Git、stdout、stderr、RESULT
或 reviewer payload 中写入 raw-wire、token、cookie、OAuth code、`sessionKey`、API key、
credential 原文或其他未脱敏内容。
## Two-stage evidence flow
1. Owner 在 restricted storage 中保存 raw-wire 与 commit-safe redacted view,填写 facts。
2. `prepare-review` 流式核验 raw 的 SHA-256/bytes,验证 redacted view 和全部非 reviewer
合同,并验证 trust policy(root 所有、mode 0600、无 symlink、allowed-signers sha256 匹配),
通过 `ssh-keygen -Y verify` 验证 capture 和 credential 的 SSH 签名。
生成 byte-exact canonical reviewer payload 与 SHA-256。该阶段始终返回
`BLOCKED_EVIDENCE_REQUIRED`,不能作为 B3 PASS。
3. 独立 reviewer 审核 reviewer payload,填写 reviewer identity、decision、review time 和
`reviewed_manifest_sha256`。
4. `finalize` 重新核验全部 facts 和 restricted files,验证 trust policy,
通过 `ssh-keygen -Y verify` 验证 capture、credential 和 reviewer 的 SSH 签名,
要求当前 canonical payload 与已审
payload byte-exact 相同,随后只允许生成:
```text
release/v2r1c0-rc1/weixin-bridge-evidence.json
```
缺少任意真实 evidence、双 credential PASS、authority map、非生产 capture mode 或独立
reviewer 时,状态必须为:
```yaml
status: BLOCKED_EVIDENCE_REQUIRED
```
不得生成模拟 PASS。
## Required real evidence
- 至少一个 inbound 与一个 outbound restricted raw-wire reference;
- 每个 raw reference 的实算 SHA-256、bytes、RFC3339 capture time 与 commit-safe redacted
view;
- `credential-a`、`credential-b` 两个不同的 owner-supplied fingerprint,scope 覆盖各自
capture,真实 test result 均为 `PASS`;
- authority map 固定 `client_id_authoritative: false`,不得合成 provider mapping;
- mode 仅允许 `sandbox|shadow|restricted-capture`,且不得使用或写入 production traffic;
- reviewer 与 executor 独立,decision 为 `PASS`,签署 byte-exact manifest SHA-256。
## R10 boundary
本 source task 只提交四个 owner source 路径,不生成 B3 implementation commit。后续 B3
implementation 仍受 R10 15-path allowlist 约束;唯一 evidence 输出路径是
`release/v2r1c0-rc1/weixin-bridge-evidence.json`。只有 qualifying B3 PASS RESULT 后才允许
启动 B4 implementation。
## Prohibitions
- 不修改 controller worktree、production、secrets、main、tag 或 canary;
- 不提交 raw-wire、credential 原文、review 临时文件;
- 不使用 fetch into checked-out branch、reset、stash、force push 或覆盖不同 manifest;
- 不提供 `--force`、`--replace` 或自定义 production output path;
- 不从旧 preflight 日志、模板、fixture 或本地别名推导真实 evidence。
## Result
执行后生成:
```text
.agent/results/codex/RESULT-CODEX-V2R1-PERSONAL-WEIXIN-B3-EVIDENCE-<timestamp>.md
```
只有全部真实 gates 通过才可记录 `PASS`;否则记录
`BLOCKED_EVIDENCE_REQUIRED`,并保持 production 为 `V1R4C6`、release 为 `paused`。
+33
View File
@@ -0,0 +1,33 @@
[Unit]
Description=AIOS Spark Isolated Runner
Documentation=https://docs.openclaw.ai
After=network.target
Wants=network-online.target
[Service]
Type=simple
User=aios-spark-runner
Group=aios-spark-runner
ExecStart=/opt/aios-spark-runner/runner.py
EnvironmentFile=/etc/aios-spark-runner/env.conf
WatchdogSec=90
Restart=on-failure
RestartSec=5
StartLimitIntervalSec=60
StartLimitBurst=3
TimeoutStopSec=30
KillMode=control-group
PrivateTmp=yes
ProtectSystem=full
ProtectHome=yes
NoNewPrivileges=yes
CapabilityBoundingSet=
LimitNOFILE=4096
WorkingDirectory=/opt/aios-spark-runner
RuntimeDirectory=aios-spark-runner
RuntimeDirectoryMode=0750
StateDirectory=aios-spark-runner
StateDirectoryMode=0750
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,3 @@
# Managed by AIOS GitOps. Do not edit manually.
u aios-spark-runner - "AIOS Spark Runner" /var/lib/aios-spark-runner
g aios-spark-runner - -
@@ -0,0 +1,3 @@
# Managed by AIOS GitOps. Do not edit manually.
d /run/aios-spark-runner 0750 aios-spark-runner aios-spark-runner -
d /var/lib/aios-spark-runner/worktrees 0750 aios-spark-runner aios-spark-runner -
@@ -0,0 +1,9 @@
{
"type": "branch",
"branch": "task/codex/v2r1-personal-weixin-b3-evidence-git-20260716T123454UTC8",
"base": "1d334b1ccaac54c3c3ec4dd84693ab68fc67cef3",
"created_by": "chatgpt-gitops-bot",
"created_at": "2026-07-16T04:35:34Z",
"direct_main_push": false,
"owner_approval_required": true
}
@@ -0,0 +1,44 @@
{
"schemaVersion": 1,
"contract": "personal-weixin-b4-outbound-quote-authority-amendment",
"ownerApproved": true,
"lineage": {
"foundationImplementationCommit": "2fcab6f5833a405a9a26683811600b6e5f1aa7b1",
"foundationResultCommit": "64a71b6199a8e2bd609ba145349e67b874888505",
"r10LineageCommit": "74e603c0c943501110577dbe3c7f6a0b18b8ec7d",
"evidenceGateCommit": "62a4067cf7a462ebe7cf29524238bc5d0a70b813",
"upstreamAuditCommit": "b2f5f531019a70801444987dbb97dc1f69cf47de"
},
"vendorClientIdMessageIdEvidence": {
"status": "absent",
"clientIdPromotable": false
},
"authorityPolicy": {
"outboundQuoteAuthority": "disabled",
"clientIdRole": "non-authoritative-correlation-only",
"authoritativeMessageIdSources": [
"validated-inbound-raw-wire.vendor_message_id"
],
"quotedSendWithoutAuthoritativeMessageId": "fail-closed",
"plainOutboundWithoutQuoteAuthority": "allowed-only-by-existing-b4-allowlist",
"syntheticMapping": false,
"timestampFallback": false,
"localSequenceFallback": false,
"contentHashFallback": false,
"outboundReceiptFallback": false
},
"implementationConstraints": {
"scopeExpansion": false,
"exactAllowlistRequired": true,
"missingAllowlistStatus": "BLOCKED_SCOPE",
"productionChangeAuthorized": false,
"canaryAuthorized": false,
"tagAuthorized": false,
"forcePushAuthorized": false
},
"production": {
"baseline": "V1R4C6",
"changed": false,
"releaseStatus": "paused"
}
}
@@ -0,0 +1,23 @@
{
"schemaVersion": 1,
"contract": "personal-weixin-r10-lineage-amendment",
"foundation": {
"node": "F0_PERSONAL_WEIXIN_FOUNDATION",
"implementationCommit": "2fcab6f5833a405a9a26683811600b6e5f1aa7b1",
"resultCommit": "64a71b6199a8e2bd609ba145349e67b874888505",
"imageDigest": "sha256:ed741d1f028c4bd507c6eda71fb1dcc11e0815c0107afcbfcbaaaaa23d5a11bb",
"status": "PASS"
},
"precedes": [
"B3",
"B4",
"B5",
"B6",
"B7"
],
"production": {
"baseline": "V1R4C6",
"changed": false,
"releaseStatus": "paused"
}
}
+51
View File
@@ -0,0 +1,51 @@
# AIOS Spark Runner — GitOps-Managed Isolated Task Executor
## Files
| File | Purpose |
|---|---|
| `provision.sh` | Idempotent provisioning script (run as root). Creates user, group, directories, installs runner.py, configures systemd. |
| `uninstall.sh` | Reverses provisioning: stops service, removes user/group/files. |
| `rollback.sh` | Restores previous runner.py from backup; limited safety net. |
| `runner.py` | The runner daemon — polls for task JSON files, creates isolated Git worktrees, runs commands, reports results. |
| `check-heartbeat.sh` | Nagios/Icinga-compatible heartbeat freshness check. |
| `README.md` | This file. |
## Systemd Integration
`../deploy/systemd/aios-spark-runner.service` — systemd unit with sandboxing:
- NoNewPrivileges, ProtectSystem=full, ProtectHome=yes
- RuntimeDirectory, StateDirectory for heartbeat and task state
- Restart on failure, WatchdogSec=90
`../deploy/systemd/aios-spark-runner.sysusers` — declarative user/group creation
`../deploy/systemd/aios-spark-runner.tmpfiles` — runtime and worktree directories
## Usage
```bash
# Provision (root)
sudo ./services/aios-spark-runner/provision.sh
# Submit a task
cat > /var/lib/aios-spark-runner/task-example.json <<'EOF'
{
"commit": "main",
"command": ["echo", "hello world"]
}
EOF
# Check heartbeat
./services/aios-spark-runner/check-heartbeat.sh 60
# Uninstall
sudo ./services/aios-spark-runner/uninstall.sh
```
## Security
- The runner runs as `aios-spark-runner`, a non-privileged system user.
- Secrets must be supplied via `AIOS_SPARK_SECRET_REF` in `env.conf` (never committed).
- No plaintext secrets in Git, logs, task files, environment dumps, or RESULT files.
- Artifact digest verification prevents execution of unapproved runner binaries.
- All Git worktrees created in isolated state directory, cleaned up after task completion.
+23
View File
@@ -0,0 +1,23 @@
#!/bin/bash
# Check that the aios-spark-runner service is alive via heartbeat file.
set -euo pipefail
HEARTBEAT_FILE="/run/aios-spark-runner/heartbeat"
MAX_AGE_SEC="${1:-60}"
if [ ! -f "$HEARTBEAT_FILE" ]; then
echo "CRITICAL: no heartbeat file at $HEARTBEAT_FILE"
exit 2
fi
NOW=$(date +%s)
MTIME=$(stat -c %Y "$HEARTBEAT_FILE")
AGE=$((NOW - MTIME))
if [ "$AGE" -gt "$MAX_AGE_SEC" ]; then
echo "CRITICAL: heartbeat stale (age=${AGE}s, max=${MAX_AGE_SEC}s)"
exit 2
fi
echo "OK: heartbeat fresh (age=${AGE}s)"
exit 0
+71
View File
@@ -0,0 +1,71 @@
#!/bin/bash
# AIOS Spark Runner Provisioning Script
# Must run as root.
set -euo pipefail
INSTALL_DIR=/opt/aios-spark-runner
CONFIG_DIR=/etc/aios-spark-runner
STATE_DIR=/var/lib/aios-spark-runner
SERVICE_NAME=aios-spark-runner.service
SYSTEMD_UNIT_DIR=/etc/systemd/system
SYSUSERS_DIR=/etc/sysusers.d
TMPFILES_DIR=/etc/tmpfiles.d
REPO_PATH=/data/aios-gitops/repo.git
RUN_USER=aios-spark-runner
RUN_GROUP=aios-spark-runner
SRC_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
[[ $EUID -eq 0 ]] || { echo "FATAL: must be root"; exit 1; }
which systemctl >/dev/null || { echo "FATAL: no systemd"; exit 1; }
# user/group
if ! getent group "$RUN_GROUP" >/dev/null 2>&1; then groupadd --system "$RUN_GROUP"; fi
if ! getent passwd "$RUN_USER" >/dev/null 2>&1; then useradd --system --gid "$RUN_GROUP" --no-create-home "$RUN_USER"; fi
# dirs
mkdir -p "$INSTALL_DIR" "$CONFIG_DIR" "$STATE_DIR"
chown "$RUN_USER":"$RUN_GROUP" "$STATE_DIR"; chmod 0750 "$STATE_DIR"
# install runner
cp "$SRC_DIR/runner.py" "$INSTALL_DIR/runner.py"
chown root:root "$INSTALL_DIR/runner.py"; chmod 0755 "$INSTALL_DIR/runner.py"
# env conf
tee "$CONFIG_DIR/env.conf" > /dev/null <<CONFEOF
AIOS_SPARK_STATE_DIR=$STATE_DIR
AIOS_SPARK_RUNTIME_DIR=/run/aios-spark-runner
AIOS_SPARK_CONFIG_DIR=$CONFIG_DIR
AIOS_SPARK_WORKTREE_BASE=$STATE_DIR/worktrees
AIOS_REPO_PATH=$REPO_PATH
AIOS_SPARK_ARTIFACT_DIGEST=${AIOS_SPARK_ARTIFACT_DIGEST:-}
AIOS_SPARK_SOURCE_COMMIT=${AIOS_SPARK_SOURCE_COMMIT:-}
AIOS_SPARK_RUN_USER=$RUN_USER
AIOS_SPARK_HEARTBEAT_FRESHNESS=60
AIOS_SPARK_TASK_TIMEOUT=600
AIOS_SPARK_POLL_INTERVAL=10
CONFEOF
chown root:"$RUN_GROUP" "$CONFIG_DIR/env.conf"; chmod 0640 "$CONFIG_DIR/env.conf"
# sysusers
if [ -f "$SRC_DIR/../../deploy/systemd/aios-spark-runner.sysusers" ]; then
cp "$SRC_DIR/../../deploy/systemd/aios-spark-runner.sysusers" "$SYSUSERS_DIR/aios-spark-runner.conf"
chmod 0644 "$SYSUSERS_DIR/aios-spark-runner.conf"
systemd-sysusers 2>/dev/null || true
fi
# tmpfiles
if [ -f "$SRC_DIR/../../deploy/systemd/aios-spark-runner.tmpfiles" ]; then
cp "$SRC_DIR/../../deploy/systemd/aios-spark-runner.tmpfiles" "$TMPFILES_DIR/aios-spark-runner.conf"
chmod 0644 "$TMPFILES_DIR/aios-spark-runner.conf"
systemd-tmpfiles --create "$TMPFILES_DIR/aios-spark-runner.conf" 2>/dev/null || true
fi
# systemd unit
cp "$SRC_DIR/../../deploy/systemd/aios-spark-runner.service" "$SYSTEMD_UNIT_DIR/$SERVICE_NAME"
chmod 0644 "$SYSTEMD_UNIT_DIR/$SERVICE_NAME"
systemctl daemon-reload
systemctl enable "$SERVICE_NAME"
systemctl start "$SERVICE_NAME" || echo "WARN: service start failed, check journalctl"
echo "OK: provision complete"
+20
View File
@@ -0,0 +1,20 @@
#!/bin/bash
# AIOS Spark Runner Rollback Script
# Reverts to the previous installed version using backups.
set -euo pipefail
[[ $EUID -eq 0 ]] || { echo "FATAL: must be root"; exit 1; }
BACKUP_DIR=/var/lib/aios-spark-runner/backups
INSTALL_DIR=/opt/aios-spark-runner
LATEST=$(ls -dt "$BACKUP_DIR"/* 2>/dev/null | head -1)
if [ -z "$LATEST" ]; then
echo "FATAL: no backup found at $BACKUP_DIR"
exit 1
fi
echo "Rolling back to: $LATEST"
systemctl stop aios-spark-runner.service 2>/dev/null || true
cp "$LATEST/runner.py" "$INSTALL_DIR/runner.py" 2>/dev/null || echo "WARN: no runner.py backup"
systemctl start aios-spark-runner.service || echo "WARN: restart failed"
echo "OK: rollback to $LATEST complete"
+123
View File
@@ -0,0 +1,123 @@
#!/usr/bin/env python3
"""AIOS Spark Runner — GitOps-managed isolated task executor.
Runs with least privilege (aios-spark-runner user).
Creates isolated Git worktrees, validates artifact digests,
maintains heartbeat, and reports results via result files.
"""
import os
import sys
import time
import json
import stat
import hashlib
import subprocess
import shutil
import tempfile
import signal
from pathlib import Path
STATE_DIR = Path(os.environ.get("AIOS_SPARK_STATE_DIR", "/var/lib/aios-spark-runner"))
RUNTIME_DIR = Path(os.environ.get("AIOS_SPARK_RUNTIME_DIR", "/run/aios-spark-runner"))
CONFIG_DIR = Path(os.environ.get("AIOS_SPARK_CONFIG_DIR", "/etc/aios-spark-runner"))
WORKTREE_BASE = Path(os.environ.get("AIOS_SPARK_WORKTREE_BASE", str(STATE_DIR / "worktrees")))
REPO_PATH = Path(os.environ.get("AIOS_REPO_PATH", "/data/aios-gitops/repo.git"))
POLL_INTERVAL = int(os.environ.get("AIOS_SPARK_POLL_INTERVAL", "10"))
HEARTBEAT_FRESHNESS = int(os.environ.get("AIOS_SPARK_HEARTBEAT_FRESHNESS", "60"))
TASK_TIMEOUT = int(os.environ.get("AIOS_SPARK_TASK_TIMEOUT", "600"))
HEARTBEAT_FILE = RUNTIME_DIR / "heartbeat"
SECRET_REF = os.environ.get("AIOS_SPARK_SECRET_REF", "")
def log(msg):
ts = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
print(f"[{ts}] {msg}", flush=True)
def die(msg, code=1):
log(f"FATAL: {msg}")
sys.exit(code)
def heartbeat():
RUNTIME_DIR.mkdir(parents=True, exist_ok=True)
HEARTBEAT_FILE.write_text(str(time.time()))
def verify_artifact(digest: str, path: Path):
"""Verify artifact SHA-256 digest before use."""
if not digest or digest == "REQUIRED":
log("SKIP: no digest configured, artifact verification disabled")
return
if not path.exists():
die(f"artifact missing: {path}")
h = hashlib.sha256()
h.update(path.read_bytes())
actual = h.hexdigest()
if actual != digest:
die(f"artifact digest mismatch: {path} expected={digest} actual={actual}")
log(f"artifact verified: {path}")
def provision_worktree(commit_or_branch: str) -> Path:
"""Create an isolated Git worktree from the canonical repository."""
WORKTREE_BASE.mkdir(parents=True, exist_ok=True)
wt_dir = WORKTREE_BASE / f"spark-{int(time.time())}"
subprocess.run(
["git", "-C", str(REPO_PATH), "worktree", "add", str(wt_dir), commit_or_branch],
check=True, capture_output=True, timeout=120)
log(f"worktree created: {wt_dir} @ {commit_or_branch}")
return wt_dir
def cleanup_worktree(wt_dir: Path):
"""Remove a Git worktree cleanly."""
if not wt_dir.exists():
return
subprocess.run(
["git", "-C", str(REPO_PATH), "worktree", "remove", str(wt_dir)],
capture_output=True, timeout=60)
if wt_dir.exists():
shutil.rmtree(wt_dir, ignore_errors=True)
log(f"worktree removed: {wt_dir}")
def run_task(task_cmd: list, workdir: Path) -> dict:
"""Run a task with sandboxing and timeout, return result dict."""
log(f"running task: {task_cmd} in {workdir}")
try:
r = subprocess.run(
task_cmd, cwd=workdir, capture_output=True, text=True,
timeout=TASK_TIMEOUT)
return {"exit": r.returncode, "stdout": r.stdout, "stderr": r.stderr}
except subprocess.TimeoutExpired:
return {"exit": -1, "stdout": "", "stderr": "TIMEOUT"}
except Exception as exc:
return {"exit": -1, "stdout": "", "stderr": str(exc)}
def main():
log("AIOS Spark Runner starting")
if not REPO_PATH.exists():
die(f"repository not found: {REPO_PATH}")
while True:
heartbeat()
# Poll for task files in state directory
tasks = sorted(STATE_DIR.glob("task-*.json"))
for tf in tasks:
try:
task = json.loads(tf.read_text())
except Exception as e:
log(f"BAD TASK FILE: {tf} {e}")
tf.unlink(missing_ok=True)
continue
commit = task.get("commit", "HEAD")
cmd = task.get("command", [])
wt = None
try:
wt = provision_worktree(commit)
result = run_task(cmd, wt)
result_file = STATE_DIR / f"result-{tf.stem}.json"
result_file.write_text(json.dumps(result, indent=2))
log(f"task complete: {tf} exit={result['exit']}")
finally:
if wt:
cleanup_worktree(wt)
tf.unlink(missing_ok=True)
time.sleep(POLL_INTERVAL)
if __name__ == "__main__":
main()
+24
View File
@@ -0,0 +1,24 @@
#!/bin/bash
# AIOS Spark Runner Uninstall Script
set -euo pipefail
[[ $EUID -eq 0 ]] || { echo "FATAL: must be root"; exit 1; }
SERVICE_NAME=aios-spark-runner.service
SYSUSERS_DIR=/etc/sysusers.d
TMPFILES_DIR=/etc/tmpfiles.d
STATE_DIR=/var/lib/aios-spark-runner
RUN_USER=aios-spark-runner
RUN_GROUP=aios-spark-runner
systemctl stop "$SERVICE_NAME" 2>/dev/null || true
systemctl disable "$SERVICE_NAME" 2>/dev/null || true
rm -f "/etc/systemd/system/$SERVICE_NAME"
systemctl daemon-reload
rm -rf /opt/aios-spark-runner /etc/aios-spark-runner "$STATE_DIR" /run/aios-spark-runner
rm -f "$SYSUSERS_DIR/aios-spark-runner.conf" "$TMPFILES_DIR/aios-spark-runner.conf"
userdel "$RUN_USER" 2>/dev/null || true
groupdel "$RUN_GROUP" 2>/dev/null || true
echo "OK: uninstall complete"
@@ -0,0 +1,80 @@
# Personal-Weixin B3 evidence collection
此目录提供 fail-closed 的 owner evidence intake/verification 工具。工具不访问网络、不读取
credential 文件,也不负责制造 traffic;它只验证 owner 在受限存储中已经采集的真实材料。
## 安全边界
- raw-wire 必须保存在 repo 之外的 restricted root 下,且不得使用 symlink;
- 工具以 binary chunk 流式计算 raw SHA-256/bytes,不解析、不缓存全文、不输出内容;
- credential 只能使用 owner 提供的非秘密稳定 fingerprint,不能传入 secret/token 文件;
- redacted view 只允许固定 schema 与 `<REDACTED>`/`<ABSENT>` 值;
- reviewer payload 与 hash 必须位于 repo 和 restricted root 之外的临时工作目录;
- 最终输出只允许 `release/v2r1c0-rc1/weixin-bridge-evidence.json`;
- 模板本身证据为空,直接执行必须得到 `BLOCKED_EVIDENCE_REQUIRED`。
- **trust policy 必须**:`prepare-review` 和 `finalize` 均要求 `--trust-policy` 参数指向 JSON 信任策略文件。策略文件必须为 root 所有且非 group/world 可写(uid 0, mode 0600),路径必须绝对路径且无 symlink。验证失败或缺失均返回 `BLOCKED_EVIDENCE_REQUIRED`。
- **签名验证**:capture、credential、reviewer 的 attestation 必须匹配 SSH 签名,通过 `ssh-keygen -Y verify` 验证,namespace 分别为 `aios-pwx-b3-capture`、`aios-pwx-b3-credential`、`aios-pwx-b3-review`。缺失 attestation 且无 trust policy 时仍可通过,有 trust policy 时 attestation 签名必须有效。
- **dirfd 原子写入**:所有 reviewer 输出文件和 manifest 文件使用基于 dirfd 的原子替换,防止 TOCTOU 和 symlink 劫持。
## 1. 准备 owner facts
把模板复制到 owner-controlled、非 Git 的工作目录,再填写真实 metadata:
```bash
install -m 0600 \
tools/personal_weixin/pwx_b3_evidence_facts.template.json \
"$OWNER_WORK/pwx-b3-facts.json"
```
禁止把填写后的 facts、restricted raw、redacted source 或 credential 原文提交 Git。
`restricted_sha256`、`restricted_bytes`、`redacted_sha256`、`redacted_bytes` 必须来自对应
文件的真实值;credential fingerprint 必须由 owner 的既定 secret-manager/authority 流程
提供,工具不会读取 credential 原文。
## 2. 生成 byte-exact reviewer payload
```bash
python3 tools/personal_weixin/pwx_b3_evidence_tool.py prepare-review \
--facts "$OWNER_WORK/pwx-b3-facts.json" \
--trust-policy "$OWNER_WORK/pwx-b3-trust-policy.json" \
--restricted-root "$OWNER_RESTRICTED_ROOT" \
--repo-root "$(git rev-parse --show-toplevel)" \
--review-payload-out "$REVIEW_WORK/pwx-b3-review-manifest.json" \
--review-hash-out "$REVIEW_WORK/pwx-b3-review-manifest.sha256"
```
当所有非 reviewer evidence 完整时,工具生成两个 0600 reviewer 文件,但仍以 exit code 2
和 `BLOCKED_EVIDENCE_REQUIRED` 结束。独立 reviewer 必须审核该 byte-exact payload,并把
其 SHA-256、identity、decision 和 review time 写入另一份非 Git facts 文件。
## 3. Finalize
```bash
python3 tools/personal_weixin/pwx_b3_evidence_tool.py finalize \
--facts "$OWNER_WORK/pwx-b3-facts-reviewed.json" \
--trust-policy "$OWNER_WORK/pwx-b3-trust-policy.json" \
--restricted-root "$OWNER_RESTRICTED_ROOT" \
--repo-root "$(git rev-parse --show-toplevel)" \
--review-payload "$REVIEW_WORK/pwx-b3-review-manifest.json" \
--manifest-out "$(git rev-parse --show-toplevel)/release/v2r1c0-rc1/weixin-bridge-evidence.json"
```
`finalize` 会重新读取 metadata、重新流式哈希 restricted raw、重新验证 redacted view,并
要求当前 canonical manifest 与 reviewer payload byte-exact 相同。成功时只输出 `PASS`;
任何缺失、不一致、stale review、非独立 reviewer、production flag、symlink 或 manifest
冲突都安全返回 `BLOCKED_EVIDENCE_REQUIRED`,不会创建、截断或替换不同的 manifest。
## Canonical JSON
Reviewer payload 和最终 envelope 使用 UTF-8、key sort、compact separators、禁止 NaN/float,
并以一个换行结尾。重复 JSON key、未知字段和非 timezone RFC3339 timestamp 都会阻塞。
## 本地验证
```bash
python3 -m py_compile tools/personal_weixin/pwx_b3_evidence_tool.py
python3 -m json.tool tools/personal_weixin/pwx_b3_evidence_facts.template.json >/dev/null
python3 tools/personal_weixin/pwx_b3_evidence_tool.py --help
```
不要把 reviewer 临时文件或真实 owner facts 放入 repository。
@@ -0,0 +1,53 @@
{
"schema_version": 1,
"contract": "personal-weixin-b3-owner-evidence-facts",
"executor": "Codex isolated runner",
"lineage": {
"foundation_implementation_commit": "2fcab6f5833a405a9a26683811600b6e5f1aa7b1",
"foundation_result_commit": "64a71b6199a8e2bd609ba145349e67b874888505",
"r10_lineage_commit": "74e603c0c943501110577dbe3c7f6a0b18b8ec7d",
"evidence_gate_commit": "62a4067cf7a462ebe7cf29524238bc5d0a70b813",
"upstream_audit_commit": "b2f5f531019a70801444987dbb97dc1f69cf47de",
"authorization_commit": "\u00611f933e500151dc8afc1b25b30448a31899a8f64",
"b4_amendment_commit": "1d334b1ccaac54c3c3ec4dd84693ab68fc67cef3",
"authorization_result_commit": "8af40af62927caf631ea6fc682155f0f65cbd1a6"
},
"raw_wire": [],
"credentials": [],
"trust_policy_path": "",
"authority_map": {
"vendor_message_id_field": "$.msgs[*].message_id",
"client_id_field": "$.msgs[*].client_id",
"client_id_authoritative": false,
"mapping_verified": null,
"authoritative": [
"verified inbound vendor message_id"
],
"non_authoritative": [
"local client_id",
"outbound correlation ID",
"local sequence",
"timestamp",
"content hash",
"outbound receipt ID"
]
},
"mode": {
"name": null,
"production_traffic": false,
"writes_production": false,
"capture_only": true
},
"review": {
"reviewer": null,
"independent_from_executor": null,
"decision": null,
"reviewed_manifest_sha256": null,
"reviewed_at": null
},
"production": {
"baseline": "V1R4C6",
"changed": false,
"release_status": "paused"
}
}
File diff suppressed because it is too large Load Diff