Compare commits

...
Author SHA1 Message Date
AIOS Governed Continuation 0e9c43c785 feat(task_plane): enumeration=cross-tenant skip+audit, direct claim remains fail-closed (G4)
Contract: enumeration of cross-tenant tasks must SKIP + record a structured
audit finding (never fail closed); direct claim by task_id stays 403
cross_tenant_denied.

- store: add tenant_skip_audit table + record_tenant_skip_audit(task_id,
  worker_id, task_tenant, worker_tenant, reason) persisting
  {source:tenant, kind:cross_tenant_skip, decision:skip+audit, task_tenant,
  worker_tenant} and appending TASK_TENANT_SKIPPED event; list_tenant_skip_audit
  read-back for testability.
- scheduler: TaskScheduler gains tenant_of_task/worker; next_queued_task skips +
  audits cross-tenant candidates (worker tenant != task tenant) instead of
  selecting them.
- schedule_service: SchedulingService plumbs tenant fns into scheduler;
  schedule/queued_count excludes cross-tenant with skip+audit; claim_governed
  adds fail-closed 403 cross_tenant_denied guard on direct claim (depth defense).
- tests: split test_a23_cross_tenant_denied into enumeration skip+audit,
  same-tenant preference, and direct-claim fail-closed.
- docs: task-plane README cross-tenant enumeration=skip+audit vs direct=fail-closed.

Verified: tests/task_plane + tests/baseline_v1_1/test_auto_claim_r2 -> 157 passed
/ 2 pre-existing failures; full task_plane+baseline_v1_1 -> 246 passed / 29
failed (failure set byte-identical to pre-existing G3 set; no new failures).
2026-09-14 19:12:06 +08:00
AIOS Governed Continuation 5e4d87dcf6 fix(task_plane): caller_scan default conservative UNKNOWN (spec-aligned, QA note)
- allow_unknown_lower default True->False: opaque dynamic import now returns
  UNKNOWN by default, never downgraded to orphan unless caller opts in (V1 spec
  red line). Update docstring; test updated + new allow_unknown_lower flag case.
- QA review note resolution. Tests: test_caller_scan 8/8 green.
2026-09-14 18:21:07 +08:00
AIOS Governed Continuation 66adfa9ec9 feat(task_plane): P1 light governors - caller_scan (WP0e), env_profile (WP1), toil_governor (WP2)
- caller_scan.py: read-only AST-based caller/live/orphan/UNKNOWN evidence (V1 spec); PROVEN_ORPHAN only on full coverage + zero live caller; UNKNOWN never downgraded.
- env_profile.py: read-only EnvironmentCapabilityReport (schema aios.environment-profile.v1); no privilege escalation, no secret collection.
- toil_governor.py: Retry/Toil Governor - suppress no-op retry on same blocker, exponential backoff, UNKNOWN -> reconcile not direct retry.
- Tests: test_caller_scan (7), test_env_profile (6), test_toil_governor (8) all green.

Base: agent/annex-2.3-platform-r2-20260914.
2026-09-14 18:07:26 +08:00
AIOS Governed Continuation 48eb5d2036 feat(task_plane): Runtime/Authority Catalog (P0-1) read-only resolver skeleton
- authority_catalog.py: AuthorityCatalog + CatalogResource (schema aios.catalog.v1).
  resolve_worker/task/repository/generic; no write path (red line); UNKNOWN fields null,
  never fabricate live evidence; summary() feeds runtime_source_unknown metric.
- Tests: test_authority.py (9) green incl. read-only invariant.

Base: agent/annex-2.3-platform-r2-20260914.
2026-09-14 18:07:11 +08:00
AIOS Governed Continuation 000d535019 feat(task_plane): Single Canonical Contract (P0-2) + Gate/Verdict (P0-5) skeleton
- contract.py: canonical single encoder wrapping models._canonical; sha256(); validate(); known_schema(); SchemaName for 6 PRD P0-2 objects. No second encoder (red line).
- verdict.py: VerdictStatus PASS/FAIL/NOT_VERIFIED/BLOCKED/DIVERGED; evaluate_verdict PASS only when required evidence all present; fail-closed on illegal scope.
- gate.py: GateKind; decide()/claim_gate()/evidence_gate() wrapping recheck + verdict.
- Tests: test_contract.py (15), test_verdict.py (13) all green.

Base: agent/annex-2.3-platform-r2-20260914 (from annex 6fb25c9).
2026-09-14 18:07:01 +08:00
AIOS Governed Continuation 6fb25c9b0a fix(governed): provisioner AIOS_GRANT_ROOT_CONFIG env injection under sudo (ANNEX-2.3 R2) 2026-09-14 16:06:09 +08:00
AIOS Governed Continuation 6045fbdf08 fix(governed): signer/validator sudo env injection + repo_path + result read (ANNEX-2.3 R2) 2026-09-14 15:56:33 +08:00
AIOS Governed Continuation f557f18620 feat(governed): ANNEX-2.3 R2 agent auto-claim chain (SchedulabilityProof→recheck→claim→pull→receipt)
Implement governed auto-claim loop on task_plane:
- SchedulabilityProof model (5-segment persisted diagnostic snapshot) + ProofBuilder
- authoritative_claim_recheck (A23: cross-tenant/task/generation/lease/capacity)
- SchedulingService wiring TaskScheduler to HTTP long_poll/poll/schedule
- governed_claim single-transaction (proof→recheck→claim→receipt→event)
- HttpRunnerTransport (real transport, renew with claim_id self-resolution)
- AgentPullRunner discover_once handles idle/no-task (REQ-03c)
- receipt/readback endpoints (/proofs, /receipts, /tasks/{id}/receipts)

Pure stdlib, no new deps; 25 tests pass (22 R2 + 2 heartbeat + 1 idle). Base: 3c320ae.
2026-09-14 14:37:42 +08:00
AIOS Governed Continuation 3c320aeb89 feat(governed): absorb control-plane integrity hardening into authority source line
Turn historical AIOS failure modes (dual-plane divergence, fake execution/
PASS-without-evidence, queued-without-claimant, exact-scope mismatch,
hash-only/placeholder artifacts, OpenAPI/backend asymmetry, capability
singular/plural asymmetry, terminal success without commit/readback) into
executable read-only invariants absorbed on the authoritative source line
(beabd05 = V2R2C2 authority main). This is a successor line; it does NOT
revive the old p0-chatgpt-bridge-v2 control-plane-convergence task line.

Adds:
- aios_control_plane_consistency: read-only derived SystemTruth (derive_truth)
  + artifact reality gate + OpenAPI/backend parity + capability alias checks.
- aios_control_plane_snapshot.attach_control_plane_truth: injects derived truth
  (state + findings + evidence_flow_confirmed) into the existing explain/status
  surface without mutating any persistence plane; base snapshot contract intact.
- config/gitops/system_truth_contract.json: state-ownership + completion/queue
  requirements + forbidden equivalences.
- tests: 13 original hardening tests + 5 snapshot-truth integration tests
  (18 passed locally).
- ADR + remediation work package docs; fault-injection + audit scripts.

Derivation is read-only; ADMITTED/QUEUED/service-active/drift_status=PASS are
not treated as evidence-complete. Evidence-required transitions and the final
E2E chain (through external projection readback / SCENARIO_ACCEPTED) remain the
release gate and are NOT claimed by this commit.

Signed-off-by: AIOS Governed Continuation <aios-governed-continuation@localhost>
2026-09-13 10:31:40 +08:00
owner beabd05396 test(governed): align reopen fake CAS with expected_revision contract (V2R2C2 hot-fix)
autonomy-smoke-tests pr82 PASS evidence_sha256=c16c68f86ee0c4e8cfd0af3f76078be45441ee151395fbde5800145c08e189b5
autonomy-unit-tests pr82 PASS evidence_sha256=168bb88573b44876966c64ec2dc2b9e030fc7635ca0b18538678f1c492c97ea2
campaign-approval-gate pr82 PASS evidence_sha256=0616d6aa11496391686dcbd658123e51d8d057e1f02661af7ef8512554e404d8
git-diff-check pr82 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
py-compile pr82 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
reconciliation-contract pr82 PASS evidence_sha256=8250e3e736db14432c421c46f995c201540e2b9c50958bdeb3ef7b057c4fd577
release-state-machine pr82 PASS evidence_sha256=b6c51b401d65a9c69435f8b86cecab752d65ded7188228aab3de3eb010990b60
secret-scan pr82 PASS evidence_sha256=f8ed9052d5ba0fbe8e52f171ea180fa353a89b4049cd3e1ed881736d0622c752
signer-validator-contract pr82 PASS evidence_sha256=775b7aa27d0c17f48b3feff20c836f14b6ed8ebcb90ec7e0b76d2a13c579268a
aios.owner-final-merge-authorization-v2 real v2 contract tests passed
2026-09-12 15:29:18 +08:00
owner 5a4e952b39 fix(governed): pin saga CAS to current revision in approval reopen (V2R2C2)
autonomy-smoke-tests pr82 PASS evidence_sha256=c16c68f86ee0c4e8cfd0af3f76078be45441ee151395fbde5800145c08e189b5
autonomy-unit-tests pr82 FAIL evidence_sha256=44994ff4cfe08df298fe59f790f3f98df7aa29e4d239ecd185b3143818c540cd
campaign-approval-gate pr82 PASS evidence_sha256=0616d6aa11496391686dcbd658123e51d8d057e1f02661af7ef8512554e404d8
git-diff-check pr82 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
py-compile pr82 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
reconciliation-contract pr82 PASS evidence_sha256=a0d5c1849c776f73fa64b02300e10c193241c0f9460e6ad4e69d73ae720de90b
release-state-machine pr82 PASS evidence_sha256=1d3314562a9345a9f75b1adf32d61168822c00d316f7db85833230bba023e448
secret-scan pr82 PASS evidence_sha256=9ca1f810bf76589206704716c4f3701a6b6aca1066b1d0c24b91d5be008f52a2
signer-validator-contract pr82 PASS evidence_sha256=775b7aa27d0c17f48b3feff20c836f14b6ed8ebcb90ec7e0b76d2a13c579268a
2026-09-12 15:16:32 +08:00
owner b70c89217b feat(governed): add /reopenOwnerApproval to silent-release allow-list (V2R2C2 v1.2)
autonomy-smoke-tests pr81 PASS evidence_sha256=e9b29f42e72804aedf5a269d4507aafa7270b39400c6923bbb014d8a335fe788
autonomy-unit-tests pr81 PASS evidence_sha256=068610c15e2716ad9dbda4df158f891693e53cb7da2c57aef3783e722fb6f354
campaign-approval-gate pr81 PASS evidence_sha256=0616d6aa11496391686dcbd658123e51d8d057e1f02661af7ef8512554e404d8
git-diff-check pr81 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
py-compile pr81 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
reconciliation-contract pr81 PASS evidence_sha256=f96a26b816cb58aec4639ca90e87da07ed8cf7d041d82fb67d755bbd5085dfca
release-state-machine pr81 PASS evidence_sha256=6cea88752b3e58ae5cee56ff3b1b13eab34117918d6bad9661194c031e2dc157
secret-scan pr81 PASS evidence_sha256=e85624b4ea35c4f7019bc9d9a3d7d44124afca891c3487fe9f550faf6c743245
signer-validator-contract pr81 PASS evidence_sha256=775b7aa27d0c17f48b3feff20c836f14b6ed8ebcb90ec7e0b76d2a13c579268a
aios.owner-final-merge-authorization-v2 real v2 contract tests passed
Narrow v1.2 delta: add the Owner approval-reopen POST path to
SILENT_RELEASE_POST_PATHS so reopen can proceed in Shadow without enabling
AIOS_GITOPS_WRITES_ENABLED. Route handler, release:approve guard, global
write check and all other allow-list paths are unchanged.

Adds tests/test_reopen_shadow_gate_contract.py covering:
- reopen is a narrow silent-release POST (route requires release:approve)
- global write gate is not enabled by this source patch
- all existing silent-release POST paths remain present
2026-09-12 14:29:33 +08:00
AIOS Governed Continuation dd2bd8fa08 feat(governed): add Owner approval reopen control-plane capability (V2R2C2)
autonomy-smoke-tests pr80 PASS evidence_sha256=c16c68f86ee0c4e8cfd0af3f76078be45441ee151395fbde5800145c08e189b5
autonomy-unit-tests pr80 PASS evidence_sha256=02e00edbe6415cf1d548ef7fb53b67e37c4abd29397b00075c6feb0d48b5f2c5
campaign-approval-gate pr80 PASS evidence_sha256=0616d6aa11496391686dcbd658123e51d8d057e1f02661af7ef8512554e404d8
git-diff-check pr80 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
py-compile pr80 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
reconciliation-contract pr80 PASS evidence_sha256=a98ae8e4b231cd5a106781a85b051ad0543e0481d604919bab8fc1047de342eb
release-state-machine pr80 PASS evidence_sha256=b6c51b401d65a9c69435f8b86cecab752d65ded7188228aab3de3eb010990b60
secret-scan pr80 PASS evidence_sha256=fb45191fc28616fa47b5c8ac1f5d6541f3e8d3163eb6653c9678ab84dff44e61
signer-validator-contract pr80 PASS evidence_sha256=775b7aa27d0c17f48b3feff20c836f14b6ed8ebcb90ec7e0b76d2a13c579268a
aios.owner-final-merge-authorization-v2 real v2 contract tests passed
Add a fail-closed control-plane operation POST /reopenOwnerApproval (scope
release:approve) that reopens one expired Owner approval cycle only after
signed_owner_approval_accepted is the final Saga event and every release byte
(candidate/source/tree/artifact/manifest/release-plan/policy) matches exactly.

- New module release_approval_reopen.py: archives the old approval byte-for-byte
  under approval-history/<candidate>/<fingerprint>.json, then drives
  ReleaseSagaStore.compare_and_set(APPROVED -> READY_FOR_APPROVAL) and removes
  only the old active approval slot (post-CAS). Idempotent per idempotency key.
- ReleaseCandidate v1 durable records do not persist side_effects_executed; this
  accepts an absent marker only when the accepted-approval event is the final
  Saga event, and rejects any explicit non-false value.
- Does not modify ReleaseSagaStore, never calls force_state_for_recovery_test,
  never performs direct DB or legacy-global writes. Publishes control-plane
  capability only; keeps 4642b74 on the mainline ancestry.
2026-09-12 12:13:33 +08:00
44 changed files with 6169 additions and 82 deletions
+2
View File
@@ -11,3 +11,5 @@ runtime/tmp/
runtime/cache/
logs/
config/test_write_perms
.pytmp/
.pytest_cache/
+12 -5
View File
@@ -129,9 +129,12 @@ class AgentPullRunner:
task = self.transport.poll(self.config.worker_id)
if not task:
return None
task_id = str(task.get("task_id", ""))
if not task_id:
raise RunnerError("task_id is required")
# A conforming transport may surface an idle/no-task response (e.g.
# {"status":"idle"}) without a task_id. REQ-03c: treat it as "no task"
# and return None instead of crashing the runner.
if not isinstance(task, dict) or not task.get("task_id"):
return None
task_id = str(task["task_id"])
if task_id in self.state.terminal:
return {"status": "duplicate_suppressed", "task_id": task_id}
claimed = dict(self.transport.claim(self.config.worker_id, task))
@@ -140,9 +143,13 @@ class AgentPullRunner:
return claimed
def renew(self, task_id: str) -> Mapping[str, Any]:
if task_id not in self.state.inflight:
inflight = self.state.inflight.get(task_id)
if inflight is None:
raise RunnerError("task is not inflight")
return self.transport.renew(self.config.worker_id, task_id)
claim_id = None
if isinstance(inflight, dict):
claim_id = inflight.get("claim_id")
return self.transport.renew(self.config.worker_id, task_id, claim_id=claim_id)
def report_progress(self, task_id: str, payload: Mapping[str, Any]) -> Mapping[str, Any]:
if task_id not in self.state.inflight:
+22 -3
View File
@@ -1,10 +1,16 @@
"""Small operational CLI for the Agent Pull Runner contract."""
"""Small operational CLI for the Agent Pull Runner contract.
Default mode talks to a live governed TaskPlane HTTP API via HttpRunnerTransport
(--base-url / --token). --self-test keeps an in-process stub transport for
offline smoke checks.
"""
from __future__ import annotations
import argparse
import json
from agent_pull_runner import AgentPullRunner, RunnerConfig
from task_plane.http_transport import HttpRunnerTransport
class _NoopTransport:
@@ -22,13 +28,26 @@ class _NoopTransport:
def main() -> int:
parser = argparse.ArgumentParser(description="AIOS Agent Pull Runner")
parser.add_argument("--worker-id", default="self-test-worker")
parser.add_argument("--base-url", default="http://127.0.0.1:8000")
parser.add_argument("--token", default="")
parser.add_argument("--actor-identity", default="")
parser.add_argument("--self-test", action="store_true")
args = parser.parse_args()
runner = AgentPullRunner(RunnerConfig(worker_id=args.worker_id, fallback_seconds=60), _NoopTransport())
if args.self_test:
transport = _NoopTransport()
else:
transport = HttpRunnerTransport(
args.base_url,
token=args.token,
actor_identity=args.actor_identity or None,
)
runner = AgentPullRunner(
RunnerConfig(worker_id=args.worker_id, fallback_seconds=60), transport
)
if args.self_test:
print(json.dumps({"status": "PASS", "registration": runner.register(), "fallback_seconds": runner.config.fallback_seconds}, sort_keys=True))
return 0
print(json.dumps({"status": "READY", "worker_id": args.worker_id}, sort_keys=True))
print(json.dumps({"status": "READY", "worker_id": args.worker_id, "base_url": args.base_url}, sort_keys=True))
return 0
@@ -0,0 +1,396 @@
#!/usr/bin/env python3
from __future__ import annotations
from dataclasses import dataclass, asdict, field
from enum import Enum
from pathlib import Path
from typing import Any, Iterable, Mapping, Sequence
import hashlib
import json
import re
import sqlite3
import subprocess
import time
PLACEHOLDER_PATTERNS = (
re.compile(r"(?im)^\s*(?:todo|fixme|tbd)\b"),
re.compile(r"(?im)^\s*pass\s*(?:#.*)?$"),
re.compile(r"(?im)\bnot\s+implemented\b"),
re.compile(r"(?im)\bplaceholder\b"),
re.compile(r"(?im)^\s*\.\.\.\s*$"),
)
class Severity(str, Enum):
BLOCKER = "BLOCKER"
MAJOR = "MAJOR"
MINOR = "MINOR"
INFO = "INFO"
class TruthState(str, Enum):
UNKNOWN = "UNKNOWN"
ADMITTED_NOT_PROJECTED = "ADMITTED_NOT_PROJECTED"
WAITING_CAPACITY = "WAITING_CAPACITY"
QUEUED = "QUEUED"
CLAIMED = "CLAIMED"
RUNNING = "RUNNING"
TERMINAL_UNPROVEN = "TERMINAL_UNPROVEN"
COMPLETE = "COMPLETE"
EXPIRED = "EXPIRED"
@dataclass(frozen=True)
class Finding:
code: str
severity: Severity
message: str
evidence: dict[str, Any] = field(default_factory=dict)
remediation: str | None = None
@dataclass
class AdmissionSnapshot:
task_id: str
envelope_id: str | None = None
admission_status: str | None = None
grant_id: str | None = None
grant_expires_at: int | None = None
one_time_use: bool | None = None
authority_base_oid: str | None = None
execution_digest: str | None = None
requested_capabilities: tuple[str, ...] = ()
file_allowlist: tuple[str, ...] = ()
admitted_at: int | None = None
@dataclass
class TaskSnapshot:
task_id: str
status: str | None = None
claimed_by: str | None = None
lease_expires_at: int | None = None
attempt: int | None = None
revision: int | None = None
created_at: str | None = None
updated_at: str | None = None
deadline: str | None = None
required_capabilities: tuple[str, ...] = ()
metadata: dict[str, Any] = field(default_factory=dict)
@dataclass
class WorkerSnapshot:
worker_id: str
status: str | None = None
last_seen_epoch: int | None = None
capabilities: tuple[str, ...] = ()
scope_task_id: str | None = None
scope_job_id: str | None = None
scope_branch: str | None = None
scope_execution_digest: str | None = None
@dataclass
class LeaseSnapshot:
task_id: str
holder: str | None = None
expires_at: int | None = None
state: str | None = None
@dataclass
class GitSnapshot:
branch: str
current_oid: str | None = None
expected_oid: str | None = None
commit_oid: str | None = None
post_oid: str | None = None
@dataclass
class EvidenceSnapshot:
commit_receipt: dict[str, Any] | None = None
runtime_readback: dict[str, Any] | None = None
external_projection_readback: dict[str, Any] | None = None
output_hashes: dict[str, str] = field(default_factory=dict)
def parse_iso_epoch(value: str | None) -> int | None:
if not value:
return None
from datetime import datetime
try:
return int(datetime.fromisoformat(value.replace("Z", "+00:00")).timestamp())
except Exception:
return None
def worker_is_healthy(worker: WorkerSnapshot, now: int, heartbeat_ttl: int = 180) -> bool:
return bool(
worker.status == "ACTIVE"
and worker.last_seen_epoch is not None
and now - worker.last_seen_epoch <= heartbeat_ttl
)
def worker_scope_matches(worker: WorkerSnapshot, task: TaskSnapshot) -> bool:
meta = task.metadata or {}
plan = meta.get("execution_plan") or meta.get("plan") or meta
job_id = plan.get("external_job_id") if isinstance(plan, dict) else None
branch = plan.get("target_branch") if isinstance(plan, dict) else None
digest = plan.get("execution_digest") if isinstance(plan, dict) else None
if worker.scope_task_id and worker.scope_task_id != task.task_id:
return False
if worker.scope_job_id and job_id and worker.scope_job_id != job_id:
return False
if worker.scope_branch and branch and worker.scope_branch != branch:
return False
if worker.scope_execution_digest and digest and worker.scope_execution_digest != digest:
return False
return True
def worker_has_capabilities(worker: WorkerSnapshot, required: Sequence[str]) -> bool:
have = set(worker.capabilities)
return set(required).issubset(have)
def eligible_workers(task: TaskSnapshot, workers: Sequence[WorkerSnapshot], now: int, heartbeat_ttl: int = 180) -> list[WorkerSnapshot]:
return [
w for w in workers
if worker_is_healthy(w, now, heartbeat_ttl)
and worker_has_capabilities(w, task.required_capabilities)
and worker_scope_matches(w, task)
]
def derive_truth(
admission: AdmissionSnapshot | None,
task: TaskSnapshot | None,
workers: Sequence[WorkerSnapshot],
lease: LeaseSnapshot | None,
git: GitSnapshot | None,
evidence: EvidenceSnapshot | None,
now: int | None = None,
queue_claim_threshold: int = 600,
heartbeat_ttl: int = 180,
) -> tuple[TruthState, list[Finding]]:
now = int(time.time()) if now is None else now
findings: list[Finding] = []
if admission and admission.admission_status == "ADMITTED" and not task:
findings.append(Finding(
"DUAL_PLANE_ADMITTED_TASK_MISSING",
Severity.BLOCKER,
"Admission plane says ADMITTED but TaskPlane has no task.",
{"task_id": admission.task_id, "envelope_id": admission.envelope_id},
"Use durable projection/outbox + exact readback; Admission must not imply execution.",
))
return TruthState.ADMITTED_NOT_PROJECTED, findings
if not task:
return TruthState.UNKNOWN, findings
if admission and admission.grant_expires_at and admission.grant_expires_at <= now and task.status in {"QUEUED", "CLAIMED", "RUNNING"}:
findings.append(Finding(
"GRANT_EXPIRED_BEFORE_EXECUTION",
Severity.BLOCKER,
"Execution state depends on an expired grant.",
{"expires_at": admission.grant_expires_at, "now": now, "task_state": task.status},
"Mint runtime TaskPrincipal just-in-time at claim; do not burn grant TTL while waiting in queue.",
))
if admission and admission.admission_status == "ADMITTED" and task.status in {"EXPIRED", "FAILED", "CANCELLED"}:
findings.append(Finding(
"DUAL_PLANE_TERMINAL_DRIFT",
Severity.MAJOR,
"Admission remains ADMITTED while TaskPlane is terminal.",
{"admission": admission.admission_status, "task": task.status},
"Expose admission_state and execution_state separately; never collapse both into one PASS/complete status.",
))
if task.status == "QUEUED":
elig = eligible_workers(task, workers, now, heartbeat_ttl)
if not elig:
findings.append(Finding(
"QUEUED_WITHOUT_ELIGIBLE_WORKER",
Severity.BLOCKER,
"Task is QUEUED but no healthy, capability- and scope-matching worker exists.",
{
"required_capabilities": list(task.required_capabilities),
"healthy_workers": [w.worker_id for w in workers if worker_is_healthy(w, now, heartbeat_ttl)],
},
"Gate TaskPlane projection on a SchedulabilityProof; keep Admission ADMITTED/WAITING_CAPACITY until claimable.",
))
return TruthState.WAITING_CAPACITY, findings
created = parse_iso_epoch(task.created_at)
if created is not None and now - created > queue_claim_threshold and not task.claimed_by:
findings.append(Finding(
"UNCLAIMED_TIMEOUT",
Severity.BLOCKER,
"QUEUED task exceeded claim SLA without a claim holder.",
{"queued_seconds": now - created, "threshold": queue_claim_threshold},
"Trigger scheduler diagnosis/takeover contract; do not extend TTL blindly.",
))
return TruthState.QUEUED, findings
if task.status == "CLAIMED":
if not task.claimed_by:
findings.append(Finding(
"FAKE_CLAIM_STATE",
Severity.BLOCKER,
"Task says CLAIMED but claimed_by is empty.",
{"task_id": task.task_id},
"Reject state transition unless claim holder + fencing generation are persisted atomically.",
))
if lease is None or lease.holder != task.claimed_by or (lease.expires_at and lease.expires_at <= now):
findings.append(Finding(
"CLAIM_WITHOUT_VALID_LEASE",
Severity.BLOCKER,
"CLAIMED task has no valid matching lease.",
{"claimed_by": task.claimed_by, "lease": asdict(lease) if lease else None},
"Require claim+lease readback before exposing CLAIMED.",
))
return TruthState.CLAIMED, findings
if task.status == "RUNNING":
if not task.claimed_by:
findings.append(Finding("FAKE_RUNNING_NO_CLAIM", Severity.BLOCKER, "RUNNING without claim holder."))
claimed_worker = next((w for w in workers if w.worker_id == task.claimed_by), None)
if not claimed_worker or not worker_is_healthy(claimed_worker, now, heartbeat_ttl):
findings.append(Finding(
"FAKE_RUNNING_NO_HEARTBEAT",
Severity.BLOCKER,
"RUNNING task has no fresh worker heartbeat.",
{"claimed_by": task.claimed_by},
"Transition to recovery only through lease/fencing rules; never report RUNNING from a stale projection.",
))
if lease is None or (lease.expires_at and lease.expires_at <= now):
findings.append(Finding("RUNNING_WITHOUT_VALID_LEASE", Severity.BLOCKER, "RUNNING without valid lease."))
return TruthState.RUNNING, findings
if task.status in {"SUCCEEDED", "COMPLETE"}:
ev = evidence or EvidenceSnapshot()
commit = ev.commit_receipt or {}
before = git.expected_oid if git else None
post = (git.post_oid if git else None) or commit.get("post_oid")
commit_oid = (git.commit_oid if git else None) or commit.get("commit_oid")
if not commit_oid or not post or (before and post == before):
findings.append(Finding(
"TERMINAL_SUCCESS_UNPROVEN",
Severity.BLOCKER,
"Task reports success without a proven branch mutation receipt.",
{"commit_oid": commit_oid, "post_oid": post, "pre_oid": before},
"SUCCEEDED requires lease id, commit receipt, pre/post OID change, output hashes, and runtime/projection readback.",
))
return TruthState.TERMINAL_UNPROVEN, findings
if not ev.runtime_readback:
findings.append(Finding("SUCCESS_WITHOUT_RUNTIME_READBACK", Severity.MAJOR, "Commit exists but runtime readback is absent."))
return TruthState.COMPLETE if not findings else TruthState.TERMINAL_UNPROVEN, findings
if task.status == "EXPIRED":
if task.attempt in (None, 0) and not task.claimed_by:
findings.append(Finding(
"EXPIRED_WITHOUT_ANY_CLAIM",
Severity.MAJOR,
"Task expired without a single execution attempt.",
{"attempt": task.attempt, "claimed_by": task.claimed_by},
"Treat as schedulability failure, not execution failure; fix worker/principal supply before recovery.",
))
return TruthState.EXPIRED, findings
return TruthState.UNKNOWN, findings
def inspect_artifact(path: Path, expected_sha256: str | None = None, min_bytes: int = 8) -> list[Finding]:
findings: list[Finding] = []
if not path.is_file():
return [Finding("ARTIFACT_MISSING", Severity.BLOCKER, f"Artifact missing: {path}")]
data = path.read_bytes()
if len(data) < min_bytes:
findings.append(Finding("ARTIFACT_TOO_SMALL", Severity.BLOCKER, f"Artifact is too small: {path}", {"bytes": len(data)}))
digest = hashlib.sha256(data).hexdigest()
if expected_sha256 and digest != expected_sha256:
findings.append(Finding("ARTIFACT_HASH_MISMATCH", Severity.BLOCKER, f"Artifact hash mismatch: {path}", {"expected": expected_sha256, "actual": digest}))
try:
text = data.decode("utf-8")
except UnicodeDecodeError:
text = ""
if text and any(p.search(text) for p in PLACEHOLDER_PATTERNS):
findings.append(Finding(
"PLACEHOLDER_ARTIFACT",
Severity.BLOCKER,
f"Artifact contains placeholder/stub markers: {path}",
remediation="Reject plan admission until actual bytes exist and match the declared digest.",
))
return findings
def operation_ids_from_openapi(path: Path) -> dict[str, str]:
try:
import yaml # type: ignore
except ImportError as exc:
raise RuntimeError("PyYAML is required for OpenAPI parity checks") from exc
doc = yaml.safe_load(path.read_text(encoding="utf-8-sig"))
result: dict[str, str] = {}
for route, methods in (doc.get("paths") or {}).items():
if not isinstance(methods, dict):
continue
for _, spec in methods.items():
if isinstance(spec, dict) and spec.get("operationId"):
result[str(spec["operationId"])] = str(route)
return result
ROUTE_PAIR_RE = re.compile(r'["\'](?P<route>/v\d+/[^"\']+)["\']\s*:\s*["\'](?P<op>[A-Za-z0-9_]+)["\']')
def operation_ids_from_server_source(path: Path) -> dict[str, str]:
text = path.read_text(encoding="utf-8")
out: dict[str, str] = {}
for m in ROUTE_PAIR_RE.finditer(text):
out[m.group("op")] = m.group("route")
return out
def check_openapi_backend_parity(openapi_path: Path, server_path: Path, required_ops: Iterable[str] = ()) -> list[Finding]:
oa = operation_ids_from_openapi(openapi_path)
be = operation_ids_from_server_source(server_path)
findings: list[Finding] = []
for op in sorted(set(required_ops) | set(oa)):
if op in oa and op not in be:
findings.append(Finding(
"OPENAPI_DEAD_ENDPOINT",
Severity.BLOCKER,
f"OpenAPI advertises {op} ({oa[op]}) but backend route mapping is absent.",
remediation="Deploy projection and backend from the same source commit; reject schema-only rollout.",
))
for op in sorted(set(required_ops) | set(be)):
if op in be and op not in oa:
findings.append(Finding(
"BACKEND_UNPROJECTED_OPERATION",
Severity.MAJOR,
f"Backend implements {op} ({be[op]}) but OpenAPI projection is absent.",
remediation="Generate/validate external projection from the canonical backend route manifest.",
))
return findings
def check_capability_aliases(required: Sequence[str], workers: Sequence[WorkerSnapshot]) -> list[Finding]:
findings: list[Finding] = []
req = set(required)
if "governed.write.files" in req:
for w in workers:
caps = set(w.capabilities)
if "governed.write.file" in caps and "governed.write.files" not in caps:
findings.append(Finding(
"CAPABILITY_SINGULAR_PLURAL_ASYMMETRY",
Severity.MAJOR,
f"Worker {w.worker_id} exposes governed.write.file but task requires governed.write.files.",
{"worker_capabilities": sorted(caps)},
"Do not treat names as aliases implicitly; define one canonical capability or an explicit server-side compatibility mapping with tests.",
))
return findings
def git_ref(git_dir: Path, ref: str) -> str | None:
cp = subprocess.run(
["git", f"--git-dir={git_dir}", "rev-parse", ref],
text=True, capture_output=True
)
if cp.returncode != 0:
return None
value = cp.stdout.strip()
return value if re.fullmatch(r"[0-9a-f]{40}", value) else None
def sqlite_rows(path: Path, query: str, params: Sequence[Any] = ()) -> list[sqlite3.Row]:
uri = f"file:{path}?mode=ro&immutable=1"
con = sqlite3.connect(uri, uri=True)
con.row_factory = sqlite3.Row
try:
return list(con.execute(query, params))
finally:
con.close()
def dump_report(path: Path, report: Mapping[str, Any]) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(report, ensure_ascii=False, indent=2, default=str) + "\n", encoding="utf-8")
@@ -1,7 +1,27 @@
"""Non-sensitive control-plane snapshot assembly."""
"""Non-sensitive control-plane snapshot assembly.
The base snapshot is a pure field-copy of each plane's reported state. It does
NOT itself assert whether that state is backed by evidence — a drift_status=PASS
or an ADMITTED status is a *claim*, not a proof alert/status change.
`attach_control_plane_truth` (see below) injects the read-only derived `SystemTruth`
(states + findings) from `aios_control_plane_consistency.derive_truth` into the
snapshot so that explain/status surfaces can expose *derived* truth alongside
the reported state, without ever mutating the persistence planes.
"""
from __future__ import annotations
from typing import Any, Mapping
from typing import Any, Mapping, Sequence
from aios_control_plane_consistency import (
AdmissionSnapshot,
EvidenceSnapshot,
GitSnapshot,
LeaseSnapshot,
TaskSnapshot,
WorkerSnapshot,
derive_truth,
)
def _copy(value: Mapping[str, Any] | None, keys: tuple[str, ...]) -> dict[str, Any]:
@@ -28,3 +48,48 @@ def build_control_plane_snapshot(*, target_version: str, runtime: Mapping[str, A
def getControlPlaneSnapshot(**kwargs: Any) -> dict[str, Any]:
return build_control_plane_snapshot(**kwargs)
def attach_control_plane_truth(
snapshot: Mapping[str, Any],
*,
admission: AdmissionSnapshot | None = None,
task: TaskSnapshot | None = None,
workers: Sequence[WorkerSnapshot] = (),
lease: LeaseSnapshot | None = None,
git: GitSnapshot | None = None,
evidence: EvidenceSnapshot | None = None,
now: int | None = None,
queue_claim_threshold: int = 600,
heartbeat_ttl: int = 180,
) -> dict[str, Any]:
"""Attach read-only derived SystemTruth to a control-plane snapshot.
This never writes to any persistence plane. It derives a ``system_truth``
block from the already-loaded plane snapshots and adds it under
``snapshot["derived"]``, leaving all existing keys intact.
The block includes the derived truth state, any findings, and a compact
"equivalence" guard (see config/gitops/system_truth_contract.json
``forbidden_equivalences``) so surfaces cannot silently present a local
plane claim as end-to-end evidence.
"""
from dataclasses import asdict
truth_state, findings = derive_truth(
admission, task, workers, lease, git, evidence, now,
queue_claim_threshold=queue_claim_threshold,
heartbeat_ttl=heartbeat_ttl,
)
out = dict(snapshot)
if "derived" not in out:
out["derived"] = {}
blockers = [f for f in findings if f.severity.value == "BLOCKER"]
out["derived"]["system_truth"] = {
"truth_state": truth_state.value,
"findings": [asdict(f) for f in findings],
"blocker_count": sum(1 for f in findings if f.severity.value == "BLOCKER"),
"major_count": sum(1 for f in findings if f.severity.value == "MAJOR"),
# Derivation is read-only; it never confirms completion on its own.
"evidence_flow_confirmed": truth_state.value == "COMPLETE" and not blockers,
}
return out
@@ -155,6 +155,7 @@ SILENT_RELEASE_POST_PATHS = frozenset({
'/submitSignedApproval',
'/submitOwnerApproval',
'/startDeployment',
'/reopenOwnerApproval',
})
def reject_non_read_http(handler, method: str, path: str) -> bool:
@@ -974,6 +975,17 @@ class H(BaseHTTPRequestHandler):
return self.sendj(
silent_release_service().submit_owner_approval(data)
)
if path == "/reopenOwnerApproval":
a = self.need("release:approve")
if not a:
return
from release_approval_reopen import ApprovalReopenError, reopen_owner_approval
try:
result = reopen_owner_approval(silent_release_service(), data)
except ApprovalReopenError as exc:
return self.sendj({"error": exc.code, "message": str(exc)}, 409)
return self.sendj(result)
if path == '/startDeployment':
a=self.need('release:deploy')
if not a: return
@@ -314,6 +314,27 @@ def _tool_argv(
return [str(path), *args]
def _provisioner_env_argv(
config: ProvisioningConfig,
root_config: Path,
path: Path,
*args: str,
) -> list[str]:
"""Build the provisioner argv with an explicit AIOS_GRANT_ROOT_CONFIG so it
survives sudo `Defaults env_reset` in production sudoers (same systemic fix
as the validator). Direct (non-sudo) runs keep the original form."""
if config.governed_use_sudo:
return [
str(config.sudo_path),
"-n",
"env",
f"AIOS_GRANT_ROOT_CONFIG={root_config}",
str(path),
*args,
]
return [str(path), *args]
def _verify_tool_contract(
path: Path,
name: str,
@@ -819,7 +840,7 @@ class AuthorizedTaskWriterProvisioner:
raise ProvisioningError(
"SIGNER_COMPATIBILITY_FAILED",
503,
details={"error": type(exc).__name__},
details={"error": type(exc).__name__, "message": str(exc)},
) from exc
signer_result = _read_json(signer_result_path)
signer = _verify_signer_result(
@@ -837,7 +858,7 @@ class AuthorizedTaskWriterProvisioner:
raise ProvisioningError(
"VALIDATOR_COMPATIBILITY_FAILED",
503,
details={"error": type(exc).__name__},
details={"error": type(exc).__name__, "message": str(exc)},
) from exc
validator_result = _read_json(validator_result_path)
_verify_validator_result(
@@ -851,8 +872,9 @@ class AuthorizedTaskWriterProvisioner:
validator_config_path
)
_run(
_tool_argv(
_provisioner_env_argv(
self.config,
validator_config_path,
self.config.provisioner,
"--grant-dir",
str(grant_dir),
@@ -0,0 +1,359 @@
#!/usr/bin/env python3
"""Governed, append-only reopen of one expired Owner approval cycle."""
from __future__ import annotations
import hashlib
import json
import os
import re
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Mapping
REQUEST_SCHEMA = "aios.owner-approval-reopen-request.v1"
EVENT_NAME = "owner_approval_reopen_authorized"
_IDEMPOTENCY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{7,127}$")
_HEX40 = re.compile(r"^[0-9a-f]{40}$")
_HEX64 = re.compile(r"^[0-9a-f]{64}$")
class ApprovalReopenError(RuntimeError):
def __init__(self, code: str, detail: str | None = None):
self.code = code
self.detail = detail
super().__init__(code if not detail else f"{code}:{detail}")
@dataclass(frozen=True)
class ReopenRequest:
candidate_id: str
expected_approval_fingerprint: str
expected_source_commit: str
expected_tree_oid: str
expected_artifact_sha256: str
expected_manifest_sha256: str
expected_release_plan_sha256: str
expected_policy_sha256: str
reason: str
idempotency_key: str
@classmethod
def from_mapping(cls, raw: Mapping[str, Any]) -> "ReopenRequest":
if raw.get("schema") != REQUEST_SCHEMA:
raise ApprovalReopenError("APPROVAL_REOPEN_SCHEMA_INVALID")
allowed = {
"schema", "candidate_id", "expected_approval_fingerprint",
"expected_source_commit", "expected_tree_oid",
"expected_artifact_sha256", "expected_manifest_sha256",
"expected_release_plan_sha256", "expected_policy_sha256",
"reason", "idempotency_key",
}
if set(raw) != allowed:
raise ApprovalReopenError("APPROVAL_REOPEN_REQUEST_FIELDS_INVALID")
req = cls(
candidate_id=str(raw["candidate_id"]),
expected_approval_fingerprint=str(raw["expected_approval_fingerprint"]),
expected_source_commit=str(raw["expected_source_commit"]),
expected_tree_oid=str(raw["expected_tree_oid"]),
expected_artifact_sha256=str(raw["expected_artifact_sha256"]),
expected_manifest_sha256=str(raw["expected_manifest_sha256"]),
expected_release_plan_sha256=str(raw["expected_release_plan_sha256"]),
expected_policy_sha256=str(raw["expected_policy_sha256"]),
reason=str(raw["reason"]),
idempotency_key=str(raw["idempotency_key"]),
)
if not req.candidate_id or "/" in req.candidate_id or ".." in req.candidate_id:
raise ApprovalReopenError("APPROVAL_REOPEN_CANDIDATE_ID_INVALID")
if not _HEX64.fullmatch(req.expected_approval_fingerprint):
raise ApprovalReopenError("APPROVAL_REOPEN_FINGERPRINT_INVALID")
if not _HEX40.fullmatch(req.expected_source_commit):
raise ApprovalReopenError("APPROVAL_REOPEN_SOURCE_INVALID")
if not _HEX40.fullmatch(req.expected_tree_oid):
raise ApprovalReopenError("APPROVAL_REOPEN_TREE_INVALID")
for value, code in (
(req.expected_artifact_sha256, "APPROVAL_REOPEN_ARTIFACT_INVALID"),
(req.expected_manifest_sha256, "APPROVAL_REOPEN_MANIFEST_INVALID"),
(req.expected_release_plan_sha256, "APPROVAL_REOPEN_RELEASE_PLAN_INVALID"),
(req.expected_policy_sha256, "APPROVAL_REOPEN_POLICY_INVALID"),
):
if not _HEX64.fullmatch(value):
raise ApprovalReopenError(code)
if not (1 <= len(req.reason.strip()) <= 512):
raise ApprovalReopenError("APPROVAL_REOPEN_REASON_INVALID")
if not _IDEMPOTENCY_RE.fullmatch(req.idempotency_key):
raise ApprovalReopenError("APPROVAL_REOPEN_IDEMPOTENCY_KEY_INVALID")
return req
def _canon(value: Any) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=True).encode()
def _sha(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def _time(value: Any, code: str) -> datetime:
if not isinstance(value, str) or not value:
raise ApprovalReopenError(code)
try:
dt = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError as exc:
raise ApprovalReopenError(code) from exc
if dt.tzinfo is None:
raise ApprovalReopenError(code)
return dt.astimezone(timezone.utc)
def _secure_json(path: Path) -> dict[str, Any]:
if path.is_symlink() or not path.is_file():
raise ApprovalReopenError("APPROVAL_REOPEN_STATE_FILE_INVALID")
st = path.stat()
if st.st_mode & 0o022:
raise ApprovalReopenError("APPROVAL_REOPEN_STATE_FILE_WRITABLE")
if st.st_uid not in {0, os.geteuid()}:
raise ApprovalReopenError("APPROVAL_REOPEN_STATE_FILE_OWNER_INVALID")
try:
obj = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
raise ApprovalReopenError("APPROVAL_REOPEN_STATE_FILE_INVALID") from exc
if not isinstance(obj, dict):
raise ApprovalReopenError("APPROVAL_REOPEN_STATE_FILE_INVALID")
return obj
def _fsync_dir(path: Path) -> None:
fd = os.open(path, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
try:
os.fsync(fd)
finally:
os.close(fd)
def _paths(root: Path, cid: str) -> dict[str, Path]:
return {
"candidate": root / "candidates" / f"{cid}.json",
"saga": root / "sagas" / f"{cid}.json",
"plan": root / "release-plans" / f"{cid}.json",
"acceptance": root / "acceptance" / f"{cid}.json",
"approval": root / "approvals" / f"{cid}.json",
}
def _history(root: Path, cid: str, fp: str) -> Path:
return root / "approval-history" / cid / f"{fp}.json"
def _archive(active: Path, history: Path) -> None:
history.parent.mkdir(parents=True, exist_ok=True, mode=0o750)
try:
os.chmod(history.parent, 0o750)
except PermissionError:
pass
original = active.read_bytes()
if history.exists():
if history.is_symlink() or not history.is_file() or history.read_bytes() != original:
raise ApprovalReopenError("APPROVAL_REOPEN_HISTORY_COLLISION")
return
try:
os.link(active, history)
except OSError:
fd = os.open(history, os.O_WRONLY | os.O_CREAT | os.O_EXCL,
active.stat().st_mode & 0o777)
try:
view = memoryview(original)
while view:
n = os.write(fd, view)
view = view[n:]
os.fsync(fd)
finally:
os.close(fd)
if history.read_bytes() != original:
raise ApprovalReopenError("APPROVAL_REOPEN_HISTORY_COPY_FAILED")
_fsync_dir(history.parent)
def _details(event: Mapping[str, Any]) -> Mapping[str, Any]:
val = event.get("details")
return val if isinstance(val, Mapping) else {}
def _cycle_event(saga: Mapping[str, Any], cycle_id: str) -> Mapping[str, Any] | None:
for event in saga.get("events") or []:
if isinstance(event, Mapping) and event.get("event") == EVENT_NAME \
and _details(event).get("approval_cycle_id") == cycle_id:
return event
return None
def _state(name: str) -> Any:
from silent_release_hardening import SagaState # lazy: avoids import cycle
return getattr(SagaState, name)
def _cycle_id(req: ReopenRequest) -> str:
return "approval-reopen-" + _sha(_canon({
"schema": REQUEST_SCHEMA,
"candidate_id": req.candidate_id,
"old_fingerprint": req.expected_approval_fingerprint,
"source_commit": req.expected_source_commit,
"tree_oid": req.expected_tree_oid,
"artifact_sha256": req.expected_artifact_sha256,
"manifest_sha256": req.expected_manifest_sha256,
"release_plan_sha256": req.expected_release_plan_sha256,
"policy_sha256": req.expected_policy_sha256,
"idempotency_key": req.idempotency_key,
}))[:24]
def _validate(req: ReopenRequest, candidate: Mapping[str, Any],
saga: Mapping[str, Any], plan: Mapping[str, Any],
acceptance: Mapping[str, Any], approval: Mapping[str, Any],
now: datetime) -> None:
if saga.get("journal_integrity") is False:
raise ApprovalReopenError("APPROVAL_REOPEN_JOURNAL_INTEGRITY_INVALID")
if saga.get("state") != "APPROVED":
raise ApprovalReopenError("APPROVAL_REOPEN_SAGA_STATE_INVALID")
if candidate.get("candidate_id") != req.candidate_id:
raise ApprovalReopenError("APPROVAL_REOPEN_CANDIDATE_MISMATCH")
if candidate.get("state") not in {"READY_FOR_APPROVAL", "APPROVED"}:
raise ApprovalReopenError("APPROVAL_REOPEN_CANDIDATE_STATE_INVALID")
if candidate.get("approval_state") not in {None, "PENDING"}:
raise ApprovalReopenError("APPROVAL_REOPEN_CANDIDATE_APPROVAL_STATE_INVALID")
# ReleaseCandidate v1 durable records do not persist side_effects_executed.
# Compatibility rule is intentionally fail-closed:
# - an absent key is accepted only together with the independent Saga
# invariant below that the accepted Owner-approval event is the final
# Saga event;
# - if a future durable schema persists the key, it must be exactly False;
# - explicit True/None/number/string values are rejected.
if "side_effects_executed" in candidate and candidate.get("side_effects_executed") is not False:
raise ApprovalReopenError("APPROVAL_REOPEN_SIDE_EFFECTS_ALREADY_EXECUTED")
checks = {
"source": candidate.get("source_commit") == req.expected_source_commit,
"tree": candidate.get("tree_oid") == req.expected_tree_oid,
"artifact": candidate.get("artifact_sha256") == req.expected_artifact_sha256,
"manifest": candidate.get("manifest_sha256") == req.expected_manifest_sha256,
"approval_fingerprint": approval.get("fingerprint") == req.expected_approval_fingerprint,
"approval_source": approval.get("source_commit") == req.expected_source_commit,
"approval_tree": approval.get("tree_oid") == req.expected_tree_oid,
"approval_artifact": approval.get("artifact_sha256") == req.expected_artifact_sha256,
"approval_manifest": approval.get("manifest_sha256") == req.expected_manifest_sha256,
"approval_release_plan": approval.get("release_plan_sha256") == req.expected_release_plan_sha256,
"approval_policy": approval.get("policy_sha256") == req.expected_policy_sha256,
"acceptance": acceptance.get("status") == "ACCEPTED",
"acceptance_release_plan": acceptance.get("release_plan_sha256") == req.expected_release_plan_sha256,
"plan_sha": (plan.get("sha256") or plan.get("release_plan_sha256")) == req.expected_release_plan_sha256,
"plan_policy": plan.get("policy_sha256") == req.expected_policy_sha256,
}
bad = sorted(k for k, ok in checks.items() if not ok)
if bad:
raise ApprovalReopenError("APPROVAL_REOPEN_BINDING_MISMATCH", ",".join(bad))
if now <= _time(approval.get("expires_at"), "APPROVAL_REOPEN_APPROVAL_EXPIRY_INVALID"):
raise ApprovalReopenError("APPROVAL_REOPEN_APPROVAL_NOT_EXPIRED")
events = [e for e in saga.get("events") or [] if isinstance(e, Mapping)]
accepted = None
for i, event in enumerate(events):
if event.get("event") == "signed_owner_approval_accepted" \
and _details(event).get("approval_fingerprint") == req.expected_approval_fingerprint:
accepted = i
if accepted is None:
raise ApprovalReopenError("APPROVAL_REOPEN_ACCEPTED_EVENT_MISSING")
if accepted != len(events) - 1:
raise ApprovalReopenError("APPROVAL_REOPEN_SAGA_ALREADY_ADVANCED")
def reopen_owner_approval(service: Any, raw: Mapping[str, Any], *,
state_root: Path | None = None,
now: datetime | None = None) -> dict[str, Any]:
req = ReopenRequest.from_mapping(raw)
raw_root = state_root or os.environ.get("AIOS_SILENT_RELEASE_STATE_ROOT", "")
if not str(raw_root):
raise ApprovalReopenError("APPROVAL_REOPEN_STATE_ROOT_MISSING")
root = Path(raw_root).expanduser().resolve()
paths = _paths(root, req.candidate_id)
cycle = _cycle_id(req)
history = _history(root, req.candidate_id, req.expected_approval_fingerprint)
saga = _secure_json(paths["saga"])
prior = _cycle_event(saga, cycle)
if prior is not None:
if saga.get("state") != "READY_FOR_APPROVAL":
raise ApprovalReopenError("APPROVAL_REOPEN_CYCLE_ALREADY_CONSUMED")
if not history.is_file():
raise ApprovalReopenError("APPROVAL_REOPEN_HISTORY_MISSING")
if paths["approval"].exists():
active = _secure_json(paths["approval"])
if active.get("fingerprint") != req.expected_approval_fingerprint:
raise ApprovalReopenError("APPROVAL_REOPEN_ACTIVE_APPROVAL_CHANGED")
paths["approval"].unlink()
_fsync_dir(paths["approval"].parent)
return {
"schema": "aios.owner-approval-reopen-result.v1",
"status": "IDEMPOTENT",
"candidate_id": req.candidate_id,
"approval_cycle_id": cycle,
"supersedes_fingerprint": req.expected_approval_fingerprint,
"saga_state": "READY_FOR_APPROVAL",
"history_ref": str(history.relative_to(root)),
}
candidate = _secure_json(paths["candidate"])
plan = _secure_json(paths["plan"])
acceptance = _secure_json(paths["acceptance"])
approval = _secure_json(paths["approval"])
_validate(req, candidate, saga, plan, acceptance, approval,
(now or datetime.now(timezone.utc)).astimezone(timezone.utc))
_archive(paths["approval"], history)
event_details = {
"schema": "aios.owner-approval-reopen-event.v1",
"approval_cycle_id": cycle,
"supersedes_approval_fingerprint": req.expected_approval_fingerprint,
"expired_at": approval.get("expires_at"),
"release_plan_sha256": req.expected_release_plan_sha256,
"source_commit": req.expected_source_commit,
"tree_oid": req.expected_tree_oid,
"artifact_sha256": req.expected_artifact_sha256,
"reason": req.reason.strip(),
"idempotency_digest": _sha(req.idempotency_key.encode()),
}
try:
service.sagas.compare_and_set(
req.candidate_id,
expected_revision=saga.get("revision"),
expected_states={_state("APPROVED")},
target=_state("READY_FOR_APPROVAL"),
event=EVENT_NAME,
details=event_details,
)
except Exception as exc:
raise ApprovalReopenError("APPROVAL_REOPEN_SAGA_CAS_FAILED") from exc
try:
active = _secure_json(paths["approval"])
if active.get("fingerprint") != req.expected_approval_fingerprint:
raise ApprovalReopenError("APPROVAL_REOPEN_ACTIVE_APPROVAL_CHANGED")
paths["approval"].unlink()
_fsync_dir(paths["approval"].parent)
except ApprovalReopenError:
raise
except OSError as exc:
raise ApprovalReopenError("APPROVAL_REOPEN_ACTIVE_APPROVAL_REMOVE_FAILED") from exc
return {
"schema": "aios.owner-approval-reopen-result.v1",
"status": "REOPENED",
"candidate_id": req.candidate_id,
"approval_cycle_id": cycle,
"supersedes_fingerprint": req.expected_approval_fingerprint,
"saga_state": "READY_FOR_APPROVAL",
"history_ref": str(history.relative_to(root)),
"release_plan_sha256": req.expected_release_plan_sha256,
"source_commit": req.expected_source_commit,
"tree_oid": req.expected_tree_oid,
}
@@ -77,6 +77,87 @@ def _read(path: Path) -> dict[str, Any]:
return value
def _read_json(path: Path) -> dict[str, Any]:
"""Read a JSON file that may be root-written (0600 root:root) by the
governed signer/validator. Mirrors `_read` but goes through
`_root_file_bytes` so root-only result files are read via sudo."""
if path.is_symlink() or not path.is_file():
raise SignerContractError("contract_input_missing")
try:
raw = _root_file_bytes(path)
except SignerContractError:
raise
try:
value = json.loads(raw.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise SignerContractError("contract_input_invalid") from exc
if not isinstance(value, dict):
raise SignerContractError("contract_object_required")
return value
def _root_is_readable(path: Path) -> bool:
try:
path.read_bytes()
return True
except (OSError, PermissionError):
return False
def _read_root_config(config_path: Path) -> dict[str, Any]:
"""Read the GrantRoot config. When the API service runs with governed sudo
(signer/validator/provisioner invoke via `sudo -n --`), this config is
root-only (0600); read it through sudo the same way instead of relaxing
permissions or breaking the private-key isolation boundary."""
if (
os.getenv("AIOS_GITOPS_GOVERNED_USE_SUDO", "").strip().lower()
in {"1", "true", "yes", "on"}
and not _root_is_readable(config_path)
):
sudo = os.getenv("AIOS_GITOPS_SUDO_PATH", "/usr/bin/sudo")
result = subprocess.run(
[sudo, "-n", "--", "cat", str(config_path)],
text=True,
capture_output=True,
check=False,
timeout=15,
)
if result.returncode != 0:
raise SignerContractError(f"root_config_unreadable:{result.returncode}")
try:
value = json.loads(result.stdout)
except (json.JSONDecodeError, UnicodeDecodeError) as exc:
raise SignerContractError("root_config_invalid") from exc
if not isinstance(value, dict):
raise SignerContractError("contract_object_required")
return value
return _read(config_path)
def _root_file_bytes(path: Path) -> bytes:
"""Read a raw file that may be root-only (0600 root:root). When the API
service runs with governed sudo, signer/validator/provisioner write
signature artifacts as root; read them through sudo the same way instead of
relaxing permissions or breaking the private-key isolation boundary."""
if (
os.getenv("AIOS_GITOPS_GOVERNED_USE_SUDO", "").strip().lower()
in {"1", "true", "yes", "on"}
and not _root_is_readable(path)
):
sudo = os.getenv("AIOS_GITOPS_SUDO_PATH", "/usr/bin/sudo")
result = subprocess.run(
[sudo, "-n", "--", "cat", str(path)],
text=False,
capture_output=True,
check=False,
timeout=15,
)
if result.returncode != 0:
raise SignerContractError(f"root_file_unreadable:{result.returncode}")
return result.stdout
return path.read_bytes()
def _write(path: Path, value: Any) -> None:
if path.is_symlink():
raise SignerContractError("contract_output_symlink")
@@ -119,8 +200,10 @@ def _run(
env=dict(env) if env is not None else None,
)
if result.returncode != 0:
detail = (result.stderr or result.stdout or "").strip()[-400:]
raise SignerContractError(
f"deployed_tool_failed:{Path(argv[0]).name}:{result.returncode}"
+ (f":{detail}" if detail else "")
)
return result
@@ -162,7 +245,7 @@ def _manifest(grant_dir: Path, names: Sequence[str]) -> None:
if not path.is_file() or path.is_symlink():
raise SignerContractError("manifest_input_missing")
rows.append(
f"{hashlib.sha256(path.read_bytes()).hexdigest()} {name}"
f"{hashlib.sha256(_root_file_bytes(path)).hexdigest()} {name}"
)
(grant_dir / "MANIFEST.sha256").write_text(
"\n".join(rows) + "\n",
@@ -321,7 +404,7 @@ def run_compatible_signer(
config_path = Path(
os.environ.get("AIOS_GRANT_ROOT_CONFIG", "/etc/aios/grant-root/config.json")
)
config = _read(config_path)
config = _read_root_config(config_path)
grant_namespace = str(config["grant_namespace"])
approval_namespace = str(config["approval_namespace"])
_sign(
@@ -339,8 +422,10 @@ def run_compatible_signer(
grant_dir / "signed-grant.key-id",
)
key_identity = (
grant_dir / "signed-grant.key-id"
).read_text(encoding="utf-8").strip()
_root_file_bytes(grant_dir / "signed-grant.key-id")
.decode("utf-8")
.strip()
)
if not key_identity:
raise SignerContractError("signer_identity_missing")
_manifest(
@@ -394,10 +479,9 @@ def _validator_config(
root_config = Path(
os.environ.get("AIOS_GRANT_ROOT_CONFIG", "/etc/aios/grant-root/config.json")
)
config = _read(root_config)
config = _read_root_config(root_config)
repo_path = (
os.environ.get("AIOS_GITOPS_VALIDATOR_REPO_PATH")
or os.environ.get("AIOS_GITOPS_WORKSPACE")
or config.get("repo_path")
)
config.update(
@@ -421,6 +505,42 @@ def _validator_config(
return path
def _validator_run_argv(
validator_command: Sequence[str],
config_path: Path,
grant_dir: Path,
result_path: Path,
) -> list[str]:
"""Build the validator argv. Production sudoers `Defaults env_reset` strips
AIOS_GRANT_ROOT_CONFIG from sudo, so when the command already carries a
sudo prefix we inject it explicitly via `sudo -n env VAR=... tool ...`.
A direct (non-sudo) invocation keeps its original form."""
validator_argv = list(validator_command)
if Path(validator_argv[0]).name in {"sudo", "sudo.exe"}:
rest = validator_argv[1:]
tool_at = 0
while tool_at < len(rest) and rest[tool_at].startswith("-"):
tool_at += 1
return [
validator_argv[0],
"-n",
"env",
f"AIOS_GRANT_ROOT_CONFIG={config_path}",
*rest[tool_at:],
"--grant-dir",
str(grant_dir),
"--result-json",
str(result_path),
]
return [
*validator_command,
"--grant-dir",
str(grant_dir),
"--result-json",
str(result_path),
]
def run_compatible_validator(
validator_command: Sequence[str],
grant_dir: Path,
@@ -438,7 +558,7 @@ def run_compatible_validator(
],
timeout=60,
)
legacy = _read(result_path)
legacy = _read_json(result_path)
if legacy.get("status") not in {"PASS", "VALID", "COMPLETED"}:
raise SignerContractError("legacy_validator_not_ready")
return Path(
@@ -460,19 +580,19 @@ def run_compatible_validator(
config_path = _validator_config(grant_dir, request)
env = os.environ.copy()
env["AIOS_GRANT_ROOT_CONFIG"] = str(config_path)
# sudo `Defaults env_reset` (production sudoers) strips AIOS_GRANT_ROOT_CONFIG
# from sudo, so the validator would read the default root config and hash/link
# the wrong approval-plan. Explicitly inject via `sudo -n env VAR=... tool ...`.
run_argv = _validator_run_argv(
validator_command, config_path, grant_dir, result_path
)
try:
_run(
[
*validator_command,
"--grant-dir",
str(grant_dir),
"--result-json",
str(result_path),
],
run_argv,
env=env,
timeout=60,
)
actual = _read(result_path)
actual = _read_json(result_path)
if actual.get("result") != "READY":
raise SignerContractError("validator_not_ready")
checks = {
+95
View File
@@ -33,6 +33,7 @@ from .models import (
validate_identifier,
)
from .policy import PolicyEngine
from .schedule_service import SchedulingService
from .store import TaskPlaneStore
from .worker_registry import WorkerRegistry
@@ -66,6 +67,7 @@ class TaskPlaneService:
self.workers = WorkerRegistry(store)
self.evidence = EvidenceService(store)
self.delegation = DelegationEngine(store)
self.scheduling = SchedulingService(store)
self.store.probe_attestation_verifier = self.policy.probe_attestation_verifier
def create_task(self, body: dict[str, Any], idempotency_key: str | None = None) -> dict[str, Any]:
@@ -385,6 +387,73 @@ class TaskPlaneService:
self.store.get_task(task_id)
return self.store.list_children(task_id)
# -- ANNEX-2.3 R2 governed claim / pull endpoints -------------------------
def governed_claim(self, task_id: str, body: dict[str, Any], actor: str) -> dict[str, Any]:
self.policy.assert_allowed("claim", actor)
task = self.store.get_task(task_id)
self.policy.assert_allowed("claim", actor, task=task)
worker_id = body.get("worker_id")
if not worker_id:
raise ValidationError("worker_id required")
worker = self.store.get_worker(worker_id)
if worker.identity != actor:
raise ForbiddenError("worker identity mismatch")
self.policy.assert_readonly_worker_binding(actor, worker_id)
lease_seconds = _bounded_int(
body.get("lease_seconds", 30), "lease_seconds", 1, 3600
)
return self.scheduling.claim_governed(
worker_id, task_id, lease_seconds=lease_seconds
)
def long_poll(self, worker_id: str, actor: str, timeout_seconds: int | None = None) -> dict[str, Any]:
self._assert_worker_pull(worker_id, actor)
return self.scheduling.long_poll(worker_id, timeout_seconds=timeout_seconds)
def poll(self, worker_id: str, actor: str) -> dict[str, Any]:
self._assert_worker_pull(worker_id, actor)
return self.scheduling.poll(worker_id)
def schedule(self, worker_id: str, actor: str) -> dict[str, Any]:
worker = self.store.get_worker(worker_id)
if worker.identity != actor:
raise ForbiddenError("worker identity mismatch")
return self.scheduling.schedule(worker_id)
def _assert_worker_pull(self, worker_id: str, actor: str) -> None:
self.policy.assert_allowed("claim", actor)
worker = self.store.get_worker(worker_id)
if worker.identity != actor:
raise ForbiddenError("worker identity mismatch")
self.policy.assert_readonly_worker_binding(actor, worker_id)
def get_proof(self, proof_id: str, actor: str | None = None) -> dict[str, Any]:
if actor is not None:
self.policy.assert_allowed("audit", actor)
return self.store.get_schedulability_proof(proof_id)
def get_receipt(self, receipt_id: str, actor: str | None = None) -> dict[str, Any]:
if actor is not None:
self.policy.assert_allowed("audit", actor)
return self.store.get_receipt(receipt_id)
def list_task_receipts(self, task_id: str, actor: str | None = None) -> list[dict[str, Any]]:
if actor is not None:
self.policy.assert_allowed("audit", actor, task=self.store.get_task(task_id))
self.store.get_task(task_id)
return self.store.list_receipts(task_id)
def get_active_claim(self, task_id: str, actor: str) -> dict[str, Any]:
"""Return the caller's active claim for a task (used by renew/readback)."""
self.policy.assert_allowed("read", actor, task=self.store.get_task(task_id))
claim = self.store.active_claim(task_id)
if claim is None:
return {"task_id": task_id, "active": False}
if claim.worker_identity != actor and actor not in self.policy.admin_identities:
raise ForbiddenError("claim identity mismatch")
return {"task_id": task_id, "active": True, "claim": claim.to_dict()}
class TaskPlaneRequestHandler(BaseHTTPRequestHandler):
service: TaskPlaneService | None = None
@@ -487,6 +556,30 @@ class TaskPlaneRequestHandler(BaseHTTPRequestHandler):
return 200, self.service.list_workers(identity)
if len(parts) == 2 and parts[0] == "workers":
return 200, self.service.get_worker(parts[1], identity)
if len(parts) == 3 and parts[0] == "workers" and parts[2] == "long_poll":
query = urllib.parse.parse_qs(
urllib.parse.urlparse(self.path).query
)
raw = query.get("timeout_seconds", ["15"])[0]
try:
timeout = int(raw)
except (TypeError, ValueError):
timeout = 15
return 200, self.service.long_poll(
parts[1], identity, timeout_seconds=timeout
)
if len(parts) == 3 and parts[0] == "workers" and parts[2] == "poll":
return 200, self.service.poll(parts[1], identity)
if len(parts) == 3 and parts[0] == "workers" and parts[2] == "schedule":
return 200, self.service.schedule(parts[1], identity)
if len(parts) == 2 and parts[0] == "proofs":
return 200, self.service.get_proof(parts[1], identity)
if len(parts) == 2 and parts[0] == "receipts":
return 200, self.service.get_receipt(parts[1], identity)
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "receipts":
return 200, self.service.list_task_receipts(parts[1], identity)
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "active-claim":
return 200, self.service.get_active_claim(parts[1], identity)
raise NotFoundError("unknown endpoint")
self._handle(route)
@@ -512,6 +605,8 @@ class TaskPlaneRequestHandler(BaseHTTPRequestHandler):
body["actor_identity"] = identity
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "claim":
return 200, self.service.claim(parts[1], body or {})
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "claim-governed":
return 200, self.service.governed_claim(parts[1], body or {}, identity)
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "heartbeat":
return 200, self.service.heartbeat(parts[1], body or {})
if len(parts) == 3 and parts[0] == "tasks" and parts[2] == "progress":
@@ -0,0 +1,286 @@
"""Read-only Runtime / Authority Catalog (Catalog V1).
Pure evidence-producer / resolve layer. It deliberately does NOT create an
authority store, does NOT write to any database, and does NOT mutate
worker / task / business state (red line: no parallel authority store
without an ADR).
Result shape follows the Catalog V1 contract (V4 section 4.1)::
{
"schema_version": "aios.catalog.v1",
"resource_type": ...,
"resource_id": ...,
"authority": {"kind", "endpoint_or_path", "revision"},
"runtime": {"service", "pid", "loaded_source_path", "source_commit",
"artifact_sha256", "state_root"},
"contracts": [...],
"conflicts": [],
"evidence": [],
}
Read-only principle: parse as much as is observable from the supplied
``TaskPlaneStore`` and the resolve arguments. Anything not observable is
left ``None`` / unknown -- we never fabricate live evidence. If no store
is supplied, every resolve returns a structurally complete record with all
observation fields set to ``None`` and an empty ``evidence`` list.
"""
from __future__ import annotations
import dataclasses
from dataclasses import dataclass, field
from typing import Any
SCHEMA_VERSION = "aios.catalog.v1"
# Runtime fields that describe the loaded source of a runtime instance.
# Their absence is what the ``runtime_source_unknown`` metric counts.
_RUNTIME_SOURCE_FIELDS = ("loaded_source_path", "source_commit", "artifact_sha256")
def _null_runtime() -> dict[str, Any]:
return {
"service": None,
"pid": None,
"loaded_source_path": None,
"source_commit": None,
"artifact_sha256": None,
"state_root": None,
}
def _null_authority(kind: str | None) -> dict[str, Any]:
return {
"kind": kind,
"endpoint_or_path": None,
"revision": None,
}
@dataclass(frozen=True)
class CatalogResource:
"""A single resolved catalog entry (immutable value object).
Every field is JSON-serialisable. Fields that could not be observed are
intentionally ``None`` rather than fabricated.
"""
schema_version: str = SCHEMA_VERSION
resource_type: str | None = None
resource_id: str | None = None
authority: dict = field(default_factory=lambda: _null_authority(None))
runtime: dict = field(default_factory=_null_runtime)
contracts: list = field(default_factory=list)
conflicts: list = field(default_factory=list)
evidence: list = field(default_factory=list)
def to_dict(self) -> dict[str, Any]:
return {
"schema_version": self.schema_version,
"resource_type": self.resource_type,
"resource_id": self.resource_id,
"authority": dict(self.authority),
"runtime": dict(self.runtime),
"contracts": list(self.contracts),
"conflicts": list(self.conflicts),
"evidence": list(self.evidence),
}
class AuthorityCatalog:
"""Read-only authority / runtime directory.
``store`` is optional. When ``None`` every resolve returns a complete
but field-empty record (no store -> nothing observable -> no fabrication).
Any read through the store is wrapped so a missing record degrades to the
same empty-but-complete shape instead of raising.
"""
def __init__(self, store: Any = None) -> None:
self.store = store
self._attempts = 0
self._resolved: dict[str, int] = {}
self._unknown: dict[str, int] = {}
self._missing = {
"authority_endpoint_or_path": 0,
"authority_revision": 0,
"loaded_source_path": 0,
"source_commit": 0,
"artifact_sha256": 0,
}
self._runtime_source_unknown = 0
# -- public resolve API ----------------------------------------------
def resolve_worker(self, worker_id: str) -> CatalogResource:
"""service/runtime/authority for a registered worker (or UNKNOWN)."""
resource = self._worker_or_unknown(str(worker_id))
self._record(resource, resolved=resource.authority.get("revision") is not None)
return resource
def resolve_task(self, task_id: str) -> CatalogResource:
"""task plan / revision authority (or UNKNOWN)."""
resource = self._task_or_unknown(str(task_id))
self._record(resource, resolved=resource.authority.get("revision") is not None)
return resource
def resolve_repository(
self, repo_id: str, authority_ref: str | None = None
) -> CatalogResource:
"""repo -> authority ref -> remote OID.
Remote OID is only reflected from the supplied ``authority_ref``; with
no live repository probe it reads as null/unknown rather than guessed.
"""
authority = {
"kind": "repository",
"endpoint_or_path": str(repo_id),
"revision": authority_ref,
}
resource = dataclasses.replace(
CatalogResource(resource_type="repository", resource_id=str(repo_id)),
authority=authority,
)
self._record(resource, resolved=authority.get("revision") is not None)
return resource
def resolve_generic(self, resource_type: str, resource_id: str) -> CatalogResource:
"""Dispatch known types; anything else -> complete-but-empty UNKNOWN."""
rtype = str(resource_type)
rid = str(resource_id)
if rtype == "worker":
return self.resolve_worker(rid)
if rtype == "task":
return self.resolve_task(rid)
if rtype == "repository":
return self.resolve_repository(rid)
# job / auth / release / schema / state_root / overlay / unknown:
# no live source -> never fabricated live state.
resource = CatalogResource(
resource_type=rtype,
resource_id=rid,
authority=_null_authority(rtype),
)
self._record(resource, resolved=False)
return resource
def summary(self) -> dict[str, Any]:
"""Resolve statistics, including ``runtime_source_unknown`` counters.
Consumed by the runtime-source-unknown metric: counts how many
resolved entries had no observable loaded source path / commit / SHA.
"""
return {
"schema_version": SCHEMA_VERSION,
"resolve_attempts": self._attempts,
"resolved": dict(self._resolved),
"unknown": dict(self._unknown),
"runtime_source_unknown": self._runtime_source_unknown,
"missing_fields": dict(self._missing),
}
# -- internal helpers ------------------------------------------------
def _worker_or_unknown(self, worker_id: str) -> CatalogResource:
base = CatalogResource(
resource_type="worker",
resource_id=worker_id,
authority=_null_authority("worker.registration"),
)
if self.store is None:
return base
try:
worker = self.store.get_worker(worker_id)
except Exception:
return base
return CatalogResource(
resource_type="worker",
resource_id=worker.worker_id,
authority={
"kind": "worker.registration",
"endpoint_or_path": worker.identity,
"revision": worker.registration_revision,
},
runtime={
"service": worker.worker_type,
"pid": None,
"loaded_source_path": None,
"source_commit": None,
"artifact_sha256": None,
"state_root": None,
},
contracts=[
{"field": "capabilities", "value": list(worker.capabilities)},
{"field": "supported_protocols", "value": list(worker.supported_protocols)},
{"field": "trust_zone", "value": worker.trust_zone},
{"field": "status", "value": worker.status},
],
conflicts=[],
evidence=[],
)
def _task_or_unknown(self, task_id: str) -> CatalogResource:
base = CatalogResource(
resource_type="task",
resource_id=task_id,
authority=_null_authority("task.plan"),
)
if self.store is None:
return base
try:
task = self.store.get_task(task_id)
except Exception:
return base
issuer = task.task_plan.issuer or {}
issuer_id = issuer.get("id") if isinstance(issuer, dict) else None
authorities = task.task_plan.authorities if hasattr(task.task_plan, "authorities") else None
return CatalogResource(
resource_type="task",
resource_id=task.task_id,
authority={
"kind": "task.plan",
"endpoint_or_path": issuer_id or None,
"revision": task.revision,
},
runtime={
"service": issuer.get("type") if isinstance(issuer, dict) else None,
"pid": None,
"loaded_source_path": authorities.get("loaded_source_path")
if isinstance(authorities, dict)
else None,
"source_commit": authorities.get("source_commit")
if isinstance(authorities, dict)
else None,
"artifact_sha256": authorities.get("artifact_sha256")
if isinstance(authorities, dict)
else None,
"state_root": task.task_plan.execution_zone,
},
contracts=[
{"field": "task_type", "value": task.task_plan.task_type},
{"field": "status", "value": task.status},
{"field": "risk", "value": task.task_plan.risk},
{"field": "data_classification", "value": task.task_plan.data_classification},
{"field": "execution_zone", "value": task.task_plan.execution_zone},
{"field": "network_policy", "value": task.task_plan.network_policy},
{"field": "required_capabilities", "value": list(task.task_plan.required_capabilities)},
],
conflicts=[],
evidence=[],
)
def _record(self, resource: CatalogResource, *, resolved: bool) -> None:
self._attempts += 1
rtype = resource.resource_type or "unknown"
bucket = self._resolved if resolved else self._unknown
bucket[rtype] = bucket.get(rtype, 0) + 1
if not resource.authority.get("endpoint_or_path"):
self._missing["authority_endpoint_or_path"] += 1
if resource.authority.get("revision") is None:
self._missing["authority_revision"] += 1
for name in _RUNTIME_SOURCE_FIELDS:
if resource.runtime.get(name) is None:
self._missing[name] += 1
if all(resource.runtime.get(name) is None for name in _RUNTIME_SOURCE_FIELDS):
self._runtime_source_unknown += 1
@@ -0,0 +1,320 @@
"""caller_scan: 只读的模块调用方扫描与孤儿判定骨架。
实现《AIOS_caller_scan_设计规格_V1》V1 骨架的只读部分:仅通过静态 AST 证据
识别哪些文件引用了目标模块(subject),并对"是否可能成为孤儿"给出评估。本模块
**绝不执行被扫描模块、不 import 任何业务模块、不读 secret、不做任何写操作**。
红线(本骨架强制):
- import 的唯一证据是 AST(标准库 ast),regex 仅用于 recall 并在 evidence 标注
evidence kind。
- 任一 probe 超时/无法解析 → coverage 非完整 → 禁止判 PROVEN_ORPHAN。
- UNKNOWN 禁止被误降为 orphan(除非显式 allow_unknown_lower)。
- 测试引用 (filename 含 tests_marker) 绝不能证明 live。
"""
from __future__ import annotations
import ast
import json
from dataclasses import dataclass
from enum import StrEnum
from pathlib import Path
from typing import Optional
class CallerKind(StrEnum):
STATIC_PRODUCTION = "STATIC_PRODUCTION"
STATIC_TEST_ONLY = "STATIC_TEST_ONLY"
SYSTEMD = "SYSTEMD"
API_ROUTE = "API_ROUTE"
SCHEDULER = "SCHEDULER"
RUNTIME_LOADED = "RUNTIME_LOADED"
WRAPPER = "WRAPPER"
UNKNOWN_DYNAMIC = "UNKNOWN_DYNAMIC"
class OrphanAssessment(StrEnum):
LIVE = "LIVE"
UNKNOWN = "UNKNOWN"
LIKELY_ORPHAN = "LIKELY_ORPHAN"
PROVEN_ORPHAN = "PROVEN_ORPHAN"
@dataclass(frozen=True)
class CallerReference:
"""一个静态调用方文件引用。"""
path: str
kind: CallerKind
line: int
evidence: str # 触发判定的语句片段
evidence_kind: str # "ast" 或 "regex-recall"
import_text: str = "" # 上一条 import 原文(recall / 说明用)
def to_dict(self) -> dict:
return {
"path": self.path,
"kind": str(self.kind),
"line": self.line,
"evidence": self.evidence,
"evidence_kind": self.evidence_kind,
"import_text": self.import_text,
}
@dataclass(frozen=True)
class CallerScanResult:
subject: str
callers: tuple[CallerReference, ...] = ()
runtime_loaded: tuple[str, ...] = ()
unknown_dynamic_refs: tuple[dict, ...] = ()
test_only_callers: tuple[CallerReference, ...] = ()
caller_count: int = 0
assessment: OrphanAssessment = OrphanAssessment.UNKNOWN
coverage: bool = False
evidence_refs: tuple[str, ...] = ()
def to_dict(self) -> dict:
return {
"subject": self.subject,
"callers": [c.to_dict() for c in self.callers],
"runtime_loaded": list(self.runtime_loaded),
"unknown_dynamic_refs": list(self.unknown_dynamic_refs),
"test_only_callers": [c.to_dict() for c in self.test_only_callers],
"caller_count": self.caller_count,
"assessment": str(self.assessment),
"coverage": self.coverage,
"evidence_refs": list(self.evidence_refs),
}
def as_json(self) -> str:
return json.dumps(self.to_dict(), ensure_ascii=False, sort_keys=True)
# ---------------------------------------------------------------- helpers
_IMPORTLIB_CALLS = {"import_module", "find_module", "__import__"}
_API_ROUTE_MARKERS = ("@app.", "add_url_rule", "APIRouter", "Blueprint", "ROUTES =")
_SCHEDULER_MARKERS = ("schedule.", "cron", "while True", "APScheduler", "every(")
_SYSTEMD_MARKERS = ("systemd", "Unit]", "ExecStart", ".service")
def _import_statement_text(source: str, node: ast.AST) -> str:
"""尽力还原一条 import 语句的原文(用于 evidence 说明,不判读逻辑依据)。"""
try:
lines = source.splitlines()
start = getattr(node, "lineno", 1)
end = getattr(node, "end_lineno", start)
return "\n".join(lines[start - 1 : end]).strip()
except Exception: # pragma: no cover - 尽力而为的展示
return ""
def _root(name: str) -> str:
return name.split(".")[0]
def _classify_file_kind(filename: str, text: str) -> CallerKind:
"""对非测试文件做启发式归类(骨架级,浅表判断)。"""
low = filename.lower()
if "wrapper" in low:
return CallerKind.WRAPPER
if any(m in text for m in _SCHEDULER_MARKERS):
return CallerKind.SCHEDULER
if any(m in text for m in _API_ROUTE_MARKERS):
return CallerKind.API_ROUTE
if any(m in text for m in _SYSTEMD_MARKERS):
return CallerKind.SYSTEMD
return CallerKind.STATIC_PRODUCTION
def _is_test_file(filename: str, tests_marker: str) -> bool:
base = Path(filename).name
low = base.lower()
norm = filename.replace("\\", "/")
return (
low.startswith(tests_marker)
or ("_" + tests_marker) in low
or ("/" + tests_marker + "/") in norm
or (tests_marker in low and "tests" in low)
)
# ---------------------------------------------------------------- main scan
def scan_subject(
subject: str,
repo_path: Optional[str] = None,
tests_marker: str = "test",
allow_unknown_lower: bool = False,
) -> CallerScanResult:
"""只读扫描 repo_path 目录下的 .py 文件,收集对 subject 的静态引用。
只引入标准库 ast,不执行/不 import 被扫描文件。
默认即保守(allow_unknown_lower=False):只要存在不可解析的 opaque 动态
import,assessment 一律返回 UNKNOWN,不把 UNKNOWN 误降为 orphan;仅当调用方
显式传入 allow_unknown_lower=True 时才允许降为 LIKELY_ORPHAN。
"""
if not subject:
raise ValueError("subject must not be empty")
root = Path(repo_path or Path.cwd())
callers: list[CallerReference] = []
test_only: list[CallerReference] = []
runtime_literals: set[str] = set()
unknown_dynamic_refs: list[dict] = []
evidence_refs: list[str] = []
scan_errors = 0
py_files = sorted(str(p) for p in root.rglob("*.py") if p.is_file())
for fpath in py_files:
try:
source = Path(fpath).read_text(encoding="utf-8", errors="replace")
tree = ast.parse(source, filename=fpath)
except (SyntaxError, OSError, UnicodeDecodeError):
scan_errors += 1
continue
is_test = _is_test_file(fpath, tests_marker)
lines = source.splitlines()
for node in ast.walk(tree):
# --- 静态 import / from ... import ---
if isinstance(node, ast.Import):
for alias in node.names:
if _root(alias.name) == _root(subject):
kind = (
CallerKind.STATIC_TEST_ONLY
if is_test
else _classify_file_kind(fpath, source)
)
ref = CallerReference(
path=fpath,
kind=kind,
line=node.lineno,
evidence=f"import {alias.name}",
evidence_kind="ast",
import_text=_import_statement_text(source, node),
)
(test_only if kind is CallerKind.STATIC_TEST_ONLY else callers).append(
ref
)
evidence_refs.append(
f"{fpath}:{node.lineno} (ast) import {alias.name}"
)
elif isinstance(node, ast.ImportFrom):
if node.module and _root(node.module) == _root(subject):
kind = (
CallerKind.STATIC_TEST_ONLY
if is_test
else _classify_file_kind(fpath, source)
)
ref = CallerReference(
path=fpath,
kind=kind,
line=node.lineno,
evidence=f"from {node.module} import",
evidence_kind="ast",
import_text=_import_statement_text(source, node),
)
(test_only if kind is CallerKind.STATIC_TEST_ONLY else callers).append(
ref
)
evidence_refs.append(
f"{fpath}:{node.lineno} (ast) from {node.module} import"
)
# --- 动态 import(importlib 调用 / __import__)---
elif _is_dynamic_import(node):
target = None
args = node.args
if args and isinstance(args[0], ast.Constant):
target = str(args[0].value)
known = target is not None
hits_subject = known and (
_root(target) == _root(subject)
or target.startswith(_root(subject) + ".")
)
if known and hits_subject:
runtime_literals.add(target)
# 无论是否点名 subject,动态 / 不可解析都计入 unknown_dynamic_refs
unknown_dynamic_refs.append(
{
"path": fpath,
"line": node.lineno,
"target": target,
"resolvable": known,
"evidence_kind": "ast" if known else "regex-recall",
}
)
has_production = any(
c.kind is not CallerKind.STATIC_TEST_ONLY for c in callers
)
has_runtime_literal = bool(runtime_literals)
has_unknown_dynamic = bool(unknown_dynamic_refs)
coverage = scan_errors == 0
assessment = _assess(
has_production=has_production,
has_runtime_literal=has_runtime_literal,
has_unknown_dynamic=has_unknown_dynamic,
scan_complete=coverage,
allow_unknown_lower=allow_unknown_lower,
)
caller_count = len(
[c for c in callers if c.kind is not CallerKind.STATIC_TEST_ONLY]
)
return CallerScanResult(
subject=subject,
callers=tuple(callers),
runtime_loaded=tuple(sorted(runtime_literals)),
unknown_dynamic_refs=tuple(unknown_dynamic_refs),
test_only_callers=tuple(test_only),
caller_count=caller_count,
assessment=assessment,
coverage=coverage,
evidence_refs=tuple(evidence_refs),
)
def _is_dynamic_import(node: ast.AST) -> bool:
"""判断是否为 importlib 动态加载调用(import_module / find_module / __import__)。"""
if not isinstance(node, ast.Call):
return False
if isinstance(node.func, ast.Name):
return node.func.id == "__import__"
if isinstance(node.func, ast.Attribute):
return (
getattr(node.func, "attr", "") in _IMPORTLIB_CALLS
and _root(getattr(node.func.value, "id", "")) == "importlib"
)
return False
def _assess(
has_production: bool,
has_runtime_literal: bool,
has_unknown_dynamic: bool,
scan_complete: bool,
allow_unknown_lower: bool,
) -> OrphanAssessment:
"""判定评估等级(只读、保守)。"""
if has_production:
return OrphanAssessment.LIVE
# 存在运行时字面量引用(importlib 显式点名 subject)→ 潜在 live,不降 orphan
if has_runtime_literal:
return OrphanAssessment.UNKNOWN
if has_unknown_dynamic:
# 无法解析的动态引用:coverage 不完整,禁止 PROVEN_ORPHAN
if allow_unknown_lower:
return OrphanAssessment.LIKELY_ORPHAN
return OrphanAssessment.UNKNOWN
if not scan_complete:
# coverage 不完整 → 禁止 PROVEN_ORPHAN
return OrphanAssessment.LIKELY_ORPHAN
# 全量扫描且无任何引用 → 可 PROVEN
return OrphanAssessment.PROVEN_ORPHAN
+210
View File
@@ -0,0 +1,210 @@
"""Single Canonical Contract (P0-2 / WP3).
Deterministic, replayable, cross-process encoding and hashing for every
digest / evidence / readback value in the AIOS platform.
ARCHITECTURAL RED LINE
----------------------
This module is the ONLY canonical serialization entry point. Do NOT hand-write
``json.dumps(sort_keys=True, ...)`` anywhere else for digest/evidence/readback
calculation. Everything must flow through :func:`encode` (and :func:`sha256`).
Design / cross-process replayability
------------------------------------
* ``encode`` reuses ``task_plane.store._canonical`` (which is
``task_plane.models.canonical_json`` = ``json.dumps(..., sort_keys=True,
separators=(",", ":"))``). We do NOT re-implement serialization here; we
normalize the object into JSON-safe plain data first, then delegate to that
single base implementation.
* Determinism is guaranteed by construction:
- object keys are sorted (``sort_keys=True``);
- ``separators=(",", ":")`` removes insignificant whitespace;
- floats are normalized by Python's ``repr(float)`` round-trip (so ``1.0``
and ``1.00`` encode identically) — this is exactly what ``json.dumps``
emits;
- booleans are distinguished from ints; ``True`` never collides with ``1``.
* Non-finite floats (``inf`` / ``nan``) are rejected loudly instead of being
silently serialized as ``Infinity`` / ``NaN``, so a digest can never be
produced from an undefined value.
* dataclass instances are flattened via :func:`dataclasses.asdict` before
serialization, so typed envelopes (``EvidenceEnvelope`` and friends) are
first-class citizens.
* Because the canonical bytes are a pure function of the logical object, two
processes on two hosts reproduce identical bytes without any shared state ->
digests are replayable and comparable across processes.
Only the standard library is used (no heavy framework).
"""
from __future__ import annotations
import dataclasses
import hashlib
import json
import math
from enum import Enum
from pathlib import Path
from typing import Any
from .store import _canonical as _canonical_base
# ---------------------------------------------------------------------------
# Schema names (the six share one canonical system-truth contract; see PRD P0-2)
# ---------------------------------------------------------------------------
class SchemaName(str, Enum):
EVIDENCE_ENVELOPE = "EVIDENCE_ENVELOPE"
RELEASE_PLAN = "RELEASE_PLAN"
TASK_EXECUTION_ENVELOPE = "TASK_EXECUTION_ENVELOPE"
AUTHORIZATION_ENVELOPE = "AUTHORIZATION_ENVELOPE"
RUNTIME_IDENTITY = "RUNTIME_IDENTITY"
RECOVERY_RECEIPT = "RECOVERY_RECEIPT"
SCHEMA_NAMES = frozenset(name.value for name in SchemaName)
# ---------------------------------------------------------------------------
# Contract schema source of truth
# ---------------------------------------------------------------------------
_CONFIG_RELATIVE = ("config", "gitops", "system_truth_contract.json")
# Overridable for exotic layouts; never required at import time.
CONTRACT_SCHEMA_PATH_ENV = "AIOS_SYSTEM_TRUTH_CONTRACT_PATH"
def _config_path() -> Path | None:
"""Locate ``system_truth_contract.json`` without importing config packages."""
import os
override = os.environ.get(CONTRACT_SCHEMA_PATH_ENV)
if override:
cand = Path(override)
if cand.is_file():
return cand
# repo_root == contract.py parents[3] (task_plane -> app -> active-runtime -> aios)
cand = Path(__file__).resolve().parents[3].joinpath(*_CONFIG_RELATIVE)
return cand if cand.is_file() else None
def _load_contract_version() -> str | None:
"""Return the ``schema`` version string from the system-truth contract.
Returns ``None`` if the file is missing or does not expose a ``schema``
key (caller decides whether that is acceptable).
"""
path = _config_path()
if path is None:
return None
try:
data = json.loads(path.read_text(encoding="utf-8"))
except (json.JSONDecodeError, OSError, UnicodeError):
return None
if not isinstance(data, dict):
return None
value = data.get("schema")
return str(value) if isinstance(value, str) else None
# ---------------------------------------------------------------------------
# Encoding primitives
# ---------------------------------------------------------------------------
class ContractError(ValueError):
"""Raised when an object cannot be canonically encoded."""
def _normalize(obj: Any) -> Any:
"""Reduce ``obj`` to JSON-safe plain data, rejecting what must not be
silently serialized (non-finite floats, sets, arbitrary objects)."""
if obj is None or isinstance(obj, (str, bool, int)):
return obj
if isinstance(obj, float):
if math.isinf(obj) or math.isnan(obj):
raise ContractError(
f"non-finite float ({obj!r}) cannot be canonically encoded"
)
return obj
if isinstance(obj, dict):
return {str(key): _normalize(value) for key, value in obj.items()}
if isinstance(obj, (list, tuple)):
return [_normalize(value) for value in obj]
if dataclasses.is_dataclass(obj) and not isinstance(obj, type):
return _normalize(dataclasses.asdict(obj))
raise ContractError(
f"unsupported type for canonical encoding: {type(obj).__name__}"
)
def encode(obj: Any) -> str:
"""Return the deterministic canonical byte-string for ``obj``.
This is the single canonical serialization implementation. It normalizes
the object into JSON-safe plain data and delegates to the shared base
encoder (``store._canonical`` / ``models.canonical_json``).
"""
return _canonical_base(_normalize(obj))
def sha256(obj: Any) -> str:
"""Return the lowercase hex sha256 of :func:`encode` of ``obj``."""
return hashlib.sha256(encode(obj).encode("utf-8")).hexdigest()
# ---------------------------------------------------------------------------
# Validation
# ---------------------------------------------------------------------------
@dataclasses.dataclass(frozen=True)
class ValidationResult:
valid: bool
reason: str | None = None
schema_name: str | None = None
version: str | None = None
def __bool__(self) -> bool:
return self.valid
def validate(obj: Any, schema_name: str | SchemaName | None = None) -> ValidationResult:
"""Validate that ``obj`` is canonically encodable and, when a schema name is
given, that the schema is a known member of the system-truth contract."""
try:
encode(obj)
except ContractError as exc:
return ValidationResult(valid=False, reason=str(exc))
version = None
if schema_name is not None:
name = _coerce_schema_name(schema_name)
if name not in SCHEMA_NAMES:
return ValidationResult(
valid=False,
reason=f"unknown schema name: {name}",
schema_name=str(name),
)
version = _load_contract_version()
if version is None:
return ValidationResult(
valid=False,
reason="canonical contract schema file unavailable",
schema_name=str(name),
)
return ValidationResult(
valid=True, schema_name=str(name), version=version
)
return ValidationResult(valid=True)
def known_schema(name: str | SchemaName) -> str | None:
"""Return the canonical contract version for a *known* schema name, or
``None`` for an unknown name / unavailable schema file."""
key = _coerce_schema_name(name)
if key not in SCHEMA_NAMES:
return None
return _load_contract_version()
def _coerce_schema_name(name: str | SchemaName) -> str:
if isinstance(name, SchemaName):
return name.value
if isinstance(name, str):
return name
return str(name)
@@ -0,0 +1,173 @@
"""env_profile: 只读的环境能力画像骨架。
实现 V4 §4.2 / V3 TrackA A1 的只读环境画像:探测操作系统、shell、posix、
常用工具、能力与凭证机制,**绝不自动扩权、绝不改 sudoers、绝不安装生产依赖、
绝不收集 secret**。全部探测只读并用 try/except 包裹,失败项置 None/'Unknown'
并记录到 missing_requirements。
"""
from __future__ import annotations
import json
import os
import platform
import shutil
import sys
from dataclasses import dataclass, field
from typing import Optional
SCHEMA_VERSION = "aios.environment-profile.v1"
def _probe(call):
"""只读探测;失败返回 None。"""
try:
return call()
except Exception:
return None
def _which(tool: str) -> Optional[str]:
try:
w = shutil.which(tool)
return w if w else None
except Exception:
return None
@dataclass(frozen=True)
class EnvironmentCapabilityReport:
schema_version: str = SCHEMA_VERSION
host_id: Optional[str] = None
os: Optional[str] = None
shell: Optional[str] = None
posix: Optional[bool] = None
tools: dict = field(default_factory=dict)
capabilities: dict = field(default_factory=dict)
credential_mechanisms: list = field(default_factory=list)
filesystem: dict = field(default_factory=dict)
missing_requirements: list = field(default_factory=list)
def to_dict(self) -> dict:
return {
"schema_version": self.schema_version,
"host_id": self.host_id,
"os": self.os,
"shell": self.shell,
"posix": self.posix,
"tools": dict(self.tools),
"capabilities": dict(self.capabilities),
"credential_mechanisms": list(self.credential_mechanisms),
"filesystem": dict(self.filesystem),
"missing_requirements": list(self.missing_requirements),
}
def as_json(self) -> str:
return json.dumps(self.to_dict(), ensure_ascii=False, sort_keys=True)
def build_report() -> EnvironmentCapabilityReport:
"""采集当前环境的只读画像。永不抛异常,字段可能为 None。"""
os_name = _probe(lambda: os.name)
sys_platform = _probe(lambda: sys.platform)
system = _probe(lambda: platform.system())
posix = True if os_name == "posix" else (False if os_name is not None else None)
hostname = None
if os_name == "posix":
hostname = _probe(lambda: os.uname().nodename)
if not hostname:
hostname = _probe(lambda: os.environ.get("COMPUTERNAME")) or _probe(
lambda: os.environ.get("HOSTNAME")
)
python_ver = _probe(lambda: ".".join(platform.python_version_tuple()[:2]))
tools = {}
for tool, label in (
("git", "git"),
("python", "python"),
("python3", "python3"),
("node", "node"),
("npm", "npm"),
("ssh", "ssh"),
("gpg", "gpg"),
("docker", "docker"),
):
tools[label] = _which(tool)
missing = []
if tools["git"] is None:
missing.append("git")
if tools["python"] is None and tools["python3"] is None:
missing.append("python")
if types := [t for t in ("git", "python") if tools[t] is None]:
missing.extend(types)
capabilities = {
"os_name": os_name,
"system": system,
"sys_platform": sys_platform,
"python": python_ver,
"posix": posix,
}
# 凭证机制:只做"存在性"探测(env var / 配置目录是否存在),绝不读取 secret 值本身
credential_mechanisms = []
for var, label in (
("GIT_SSH_COMMAND", "git_ssh"),
("AGENT_SSH_KEY_FILE", "agent_ssh_key_file"),
("SSH_AUTH_SOCK", "ssh_agent"),
):
present = _probe(lambda v=var: bool(os.environ.get(v)))
if present:
credential_mechanisms.append(label)
# 配置目录存在性(不读内容)
for path, label in (
("~/.ssh", "ssh_config_dir"),
("~/.gnupg", "gnupg_dir"),
):
try:
expanded = os.path.expanduser(path)
if os.path.isdir(expanded):
credential_mechanisms.append(label)
except Exception:
pass
filesystem = {"cwd": _probe(lambda: os.getcwd())}
return EnvironmentCapabilityReport(
schema_version=SCHEMA_VERSION,
host_id=hostname,
os=os_name,
shell=_probe(lambda: os.environ.get("SHELL")) or _probe(lambda: os.environ.get("COMSPEC")),
posix=posix,
tools=tools,
capabilities=capabilities,
credential_mechanisms=sorted(set(credential_mechanisms)),
filesystem=filesystem,
missing_requirements=sorted(set(missing)),
)
def check_requirements(required: list[str]) -> EnvironmentCapabilityReport:
"""在 build_report 基础上,按 required 校验缺失项并写入 missing_requirements。
只读,不安装任何东西。
"""
report = build_report()
missing = set(report.missing_requirements)
tool_requirements = ("git", "python", "node", "ssh", "gpg", "docker", "npm", "python3")
for req in required or []:
if req in tool_requirements:
if report.tools.get(req) is None:
missing.add(req)
else:
# 非工具类要求的可验证性有限;不做过度校验,仅当明确缺失时记录
val = report.capabilities.get(req)
if val is not None:
missing.discard(req)
return EnvironmentCapabilityReport(
**{**report.to_dict(), "missing_requirements": sorted(missing)}
)
+114
View File
@@ -0,0 +1,114 @@
"""Gate / Evidence Engine — gate decisions over Verdicts (WP5 / P0-5).
A GateDecision wraps a Verdict with the gate kind that produced it. The
harden rule is that ok == (verdict.status == PASS); a gate never "passes"
on partial evidence, service liveness, or historical proof.
"""
from __future__ import annotations
import time
from dataclasses import dataclass, field
from enum import StrEnum
from typing import Any
from .verdict import Verdict, VerdictStatus, evaluate_verdict
class GateKind(StrEnum):
CLAIM = "CLAIM"
ADMIT = "ADMIT"
EXECUTE = "EXECUTE"
EVIDENCE = "EVIDENCE"
AUTHORIZE = "AUTHORIZE"
READBACK = "READBACK"
@dataclass(frozen=True)
class GateDecision:
"""A gate's decision: the Verdict plus the gate kind it came from."""
gate: GateKind
ok: bool
verdict: Verdict
checks: tuple[dict[str, Any], ...] = ()
reason: str = ""
decided_at: float = field(default_factory=time.time)
def to_dict(self) -> dict[str, Any]:
return {
"gate": self.gate.value,
"ok": self.ok,
"verdict": self.verdict.to_dict(),
"checks": list(self.checks),
"reason": self.reason,
"decided_at": self.decided_at,
}
def decide(gate_kind: GateKind | str, verdict: Verdict) -> GateDecision:
"""Build a GateDecision. ok is True IFF the verdict is PASS."""
ok = verdict.status == VerdictStatus.PASS
return GateDecision(
gate=GateKind(gate_kind),
ok=ok,
verdict=verdict,
checks=verdict.checks,
reason=verdict.reason,
decided_at=verdict.decided_at,
)
def claim_gate(recheck_decision: Any) -> GateDecision:
"""Wrap an authoritative_claim_recheck result as a CLAIM gate.
The CLAIM gate only passes when the full recheck is accepted.
RecheckDecision exposes `allowed` (not a `verdict` field) and its checks;
there is no REJECT status on the recheck side, so a denial is mapped to
the fail-closed BLOCKED verdict here (documented mapping).
"""
decided_at = getattr(recheck_decision, "observed_at", time.time())
checks = [c.to_dict() for c in getattr(recheck_decision, "checks", [])]
allowed = bool(getattr(recheck_decision, "allowed", False))
subject = getattr(recheck_decision, "task_id", "claim")
raw_reason = getattr(recheck_decision, "reason", "") or ""
if allowed:
status = VerdictStatus.PASS
reason = raw_reason or "all claim recheck checks accepted"
else:
# NOTE: recheck has no REJECT status -> map a denied recheck to BLOCKED
# (fail-closed). RecheckDecision.reason carries the failing check detail.
status = VerdictStatus.BLOCKED
reason = raw_reason or "claim recheck rejected"
verdict = Verdict(
scope="contract",
status=status,
subject=subject,
checks=tuple(checks),
evidence_refs=(),
missing_evidence=(),
reason=reason,
decided_at=decided_at,
)
return decide(GateKind.CLAIM, verdict)
def evidence_gate(
required_evidence: Any,
available_evidence: Any,
*,
scope: str = "runtime",
subject: str = "evidence",
reason: str = "",
) -> GateDecision:
"""EVIDENCE gate: PASS only when the evidence closure is fully satisfied."""
verdict = evaluate_verdict(
scope=scope,
subject=subject,
required_evidence=required_evidence,
available_evidence=available_evidence,
reason=reason,
)
return decide(GateKind.EVIDENCE, verdict)
@@ -0,0 +1,141 @@
"""Real HTTP RunnerTransport for the Agent Pull Runner protocol.
Implements the RunnerTransport Protocol from agent_pull_runner.py over
stdlib-only urllib.request, talking to the governed TaskPlane HTTP API.
See ARCH-AUTO-CLAIM-R2-20260914 §2 / §6 T9.
"""
from __future__ import annotations
import json
import urllib.error
import urllib.parse
import urllib.request
from typing import Any, Mapping, Sequence
from agent_pull_runner import RunnerError
class HttpRunnerTransport:
def __init__(
self,
base_url: str,
*,
token: str = "",
actor_identity: str | None = None,
timeout: float = 10,
) -> None:
self.base_url = base_url.rstrip("/")
self.token = token
self.actor_identity = actor_identity
self.timeout = timeout
def _request(
self,
method: str,
path: str,
body: Mapping[str, Any] | None = None,
query: Mapping[str, Any] | None = None,
) -> Any:
url = self.base_url + path
if query:
url += "?" + urllib.parse.urlencode(query)
request = urllib.request.Request(url, method=method)
if self.token:
request.add_header("Authorization", "Bearer " + self.token)
data = None
if body is not None:
data = json.dumps(body).encode("utf-8")
request.add_header("Content-Type", "application/json")
request.add_header("Content-Length", str(len(data)))
try:
with urllib.request.urlopen(request, data=data, timeout=self.timeout) as response:
return json.loads(response.read().decode("utf-8"))
except urllib.error.HTTPError as exc:
raw = exc.read().decode("utf-8")
try:
payload = json.loads(raw)
except (ValueError, json.JSONDecodeError):
payload = {"error": "http_error", "status": exc.code}
raise RunnerError(payload) from None
# -- RunnerTransport protocol ------------------------------------------
def register_worker(self, worker_id: str, capabilities: Sequence[str]) -> Mapping[str, Any]:
return self._request(
"POST",
"/workers/register",
{
"worker_id": worker_id,
"worker_type": "code.change",
"identity": self.actor_identity or worker_id,
"capabilities": list(capabilities),
"trust_zone": "default",
"max_concurrency": 1,
"heartbeat_interval_seconds": 15,
"supported_protocols": ["aios-task-v1"],
},
)
def heartbeat(self, worker_id: str) -> Mapping[str, Any]:
return self._request("POST", f"/workers/{worker_id}/heartbeat", {})
def long_poll(self, worker_id: str, timeout_seconds: int) -> Mapping[str, Any] | None:
result = self._request(
"GET",
f"/workers/{worker_id}/long_poll",
query={"timeout_seconds": int(timeout_seconds)},
)
if isinstance(result, dict) and result.get("status") == "idle":
return None
return result
def poll(self, worker_id: str) -> Mapping[str, Any] | None:
result = self._request("GET", f"/workers/{worker_id}/poll")
if isinstance(result, dict) and result.get("status") == "idle":
return None
return result
def claim(self, worker_id: str, task: Mapping[str, Any]) -> Mapping[str, Any]:
if isinstance(task, dict) and task.get("claim_id"):
# In the governed flow the long_poll already claimed the task;
# remain idempotent and return the already-claimed task.
return task
task_id = task.get("task_id")
if not task_id:
raise RunnerError("task_id is required")
return self._request(
"POST",
f"/tasks/{task_id}/claim-governed",
{"worker_id": worker_id},
)
def renew(self, worker_id: str, task_id: str, claim_id: str | None = None) -> Mapping[str, Any]:
# Lease extension is delegated to the claim heartbeat endpoint, which
# requires the claim_id and matches the bearer identity to the claim.
# When the caller does not supply claim_id we resolve the caller's own
# active claim via the read endpoint first.
if not claim_id:
resolved = self._request("GET", f"/tasks/{task_id}/active-claim")
if isinstance(resolved, dict) and resolved.get("active"):
claim = resolved["claim"]
claim_id = claim.get("claim_id")
if not claim_id:
raise RunnerError("no active claim to renew")
body = {"lease_seconds": 30, "claim_id": claim_id}
return self._request("POST", f"/tasks/{task_id}/heartbeat", body)
def report_progress(
self, worker_id: str, task_id: str, payload: Mapping[str, Any]
) -> Mapping[str, Any]:
return self._request("POST", f"/tasks/{task_id}/progress", payload)
def report_evidence(
self, worker_id: str, task_id: str, payload: Mapping[str, Any]
) -> Mapping[str, Any]:
return self._request("POST", f"/tasks/{task_id}/result", payload)
def report_result(
self, worker_id: str, task_id: str, payload: Mapping[str, Any]
) -> Mapping[str, Any]:
return self._request("POST", f"/tasks/{task_id}/result", payload)
+78
View File
@@ -0,0 +1,78 @@
"""Claim/proof receipts for ANNEX-2.3 R2 governed claim audit read-back.
A receipt is a minimal, tamper-evident record generated alongside a governed
claim. Its digest is sha256(canonical_json(payload)) so the payload can be
re-verified on read-back, consistent with the evidence and event chain.
"""
from __future__ import annotations
import time
import uuid
from dataclasses import dataclass
from typing import Any
from .models import compute_sha256_hex
def receipt_digest(payload: dict[str, Any]) -> str:
return compute_sha256_hex(payload)
@dataclass
class ClaimReceipt:
receipt_id: str
kind: str
task_id: str
worker_id: str
claim_id: str | None
proof_id: str | None
digest: str
payload: dict[str, Any]
created_at: float
def to_dict(self) -> dict[str, Any]:
return {
"receipt_id": self.receipt_id,
"kind": self.kind,
"task_id": self.task_id,
"worker_id": self.worker_id,
"claim_id": self.claim_id,
"proof_id": self.proof_id,
"digest": self.digest,
"payload": self.payload,
"created_at": self.created_at,
}
def build_receipt(
*,
kind: str,
task_id: str,
worker_id: str,
claim_id: str | None = None,
proof_id: str | None = None,
extra: dict[str, Any] | None = None,
created_at: float | None = None,
) -> ClaimReceipt:
"""Build a ClaimReceipt whose digest covers the full digestable payload."""
extra = extra or {}
payload = {
"kind": kind,
"task_id": task_id,
"worker_id": worker_id,
"claim_id": claim_id,
"proof_id": proof_id,
**extra,
}
return ClaimReceipt(
receipt_id=f"rct-{uuid.uuid4()}",
kind=kind,
task_id=task_id,
worker_id=worker_id,
claim_id=claim_id,
proof_id=proof_id,
digest=receipt_digest(payload),
payload=payload,
created_at=created_at if created_at is not None else time.time(),
)
+240
View File
@@ -0,0 +1,240 @@
"""RecheckEngine producing auditable RecheckDecision for governed claims.
The authoritative_claim_recheck re-reads live state at claim time and evaluates
the full A23 check set (both positive evidence and the five A23 negative cases).
See ANNEX-2.3 R2 ARCH-AUTO-CLAIM-R2-20260914 §4.2.
"""
from __future__ import annotations
import time
import uuid
from dataclasses import dataclass, field
from typing import Any, Callable
from .capabilities import has_capabilities
from .errors import ConflictError, ForbiddenError, LockedError, TaskPlaneHTTPError
from .models import compute_sha256_hex
from .schedulability import default_tenant_of_task, default_tenant_of_worker
# check -> (error_code, exception_factory)
_A23_BY_CHECK = {
"tenant_match": ("cross_tenant_denied", ForbiddenError),
"principal_match": ("task_binding_mismatch", ConflictError),
"task_match": ("task_binding_mismatch", ConflictError),
"generation_ok": ("generation_stale", ConflictError),
"lease_valid": ("lease_expired_or_closed", LockedError),
"capacity_ok": ("capacity_exhausted", LockedError),
}
CLAIMABLE_STATUSES = {"CREATED", "QUEUED", "FAILED_RECOVERABLE"}
@dataclass
class Check:
check: str
ok: bool
detail: str = ""
def to_dict(self) -> dict[str, Any]:
return {"check": self.check, "ok": self.ok, "detail": self.detail}
@dataclass
class RecheckDecision:
recheck_id: str
proof_id: str
task_id: str
worker_id: str
claim_id: str | None
allowed: bool
reason: str | None
checks: list[Check]
observed_at: float
digest: str = ""
error_code: str | None = None
status_code: int = 400
def payload(self) -> dict[str, Any]:
return {
"recheck_id": self.recheck_id,
"proof_id": self.proof_id,
"task_id": self.task_id,
"worker_id": self.worker_id,
"claim_id": self.claim_id,
"allowed": self.allowed,
"reason": self.reason,
"checks": [c.to_dict() for c in self.checks],
"observed_at": self.observed_at,
}
def to_dict(self) -> dict[str, Any]:
return {
**self.payload(),
"digest": self.digest,
"error_code": self.error_code,
"status_code": self.status_code,
}
def failed_check(self) -> Check | None:
for check in self.checks:
if not check.ok:
return check
return None
class RecheckEngine:
def __init__(
self,
store: Any,
*,
tenant_of_task: Callable[[Any], str] | None = None,
tenant_of_worker: Callable[[Any], str] | None = None,
authorization_ok: Callable[[Any], bool] | None = None,
) -> None:
self.store = store
self.tenant_of_task = tenant_of_task or default_tenant_of_task
self.tenant_of_worker = tenant_of_worker or default_tenant_of_worker
self.authorization_ok = authorization_ok or (lambda worker: True)
def authoritative_claim_recheck(
self,
*,
proof: Any,
task: Any,
worker: Any,
now: float | None = None,
) -> RecheckDecision:
now = now if now is not None else time.time()
checks: list[Check] = []
tenant_task = self.tenant_of_task(task)
tenant_worker = self.tenant_of_worker(worker)
checks.append(
Check(
"tenant_match",
tenant_task == tenant_worker,
f"task_tenant={tenant_task} worker_tenant={tenant_worker}"
if tenant_task != tenant_worker
else "tenant matched",
)
)
principal_ok = proof.binding.get("worker_identity") == worker.identity
checks.append(
Check(
"principal_match",
principal_ok,
f"proof_identity={proof.binding.get('worker_identity')} actual={worker.identity}"
if not principal_ok
else "identity matched",
)
)
authz_ok = bool(self.authorization_ok(worker))
checks.append(
Check(
"authorization_ok",
authz_ok,
"worker not in governed authorization profile" if not authz_ok else "authorized",
)
)
task_match = task.task_id == proof.task_id and task.status in CLAIMABLE_STATUSES
checks.append(
Check(
"task_match",
task_match,
f"task_status={task.status}" if not task_match else "task matched",
)
)
gen_ok = task.revision == proof.runtime["generation"]["task_revision"]
checks.append(
Check(
"generation_ok",
gen_ok,
f"proof_revision={proof.runtime['generation']['task_revision']} actual={task.revision}"
if not gen_ok
else "generation current",
)
)
lease_ok = self.store.active_claim(task.task_id) is None
checks.append(
Check(
"lease_valid",
lease_ok,
"task already actively claimed" if not lease_ok else "lease free",
)
)
current_active = self.store.get_active_claim_count(worker.worker_id)
capacity_ok = current_active < worker.max_concurrency
checks.append(
Check(
"capacity_ok",
capacity_ok,
f"active={current_active} max={worker.max_concurrency}"
if not capacity_ok
else "capacity available",
)
)
online = (not worker.disabled) and (
now <= worker.last_heartbeat + (worker.heartbeat_interval_seconds * 3)
)
checks.append(
Check(
"worker_online",
online,
"worker offline or disabled" if not online else "online",
)
)
caps_ok = has_capabilities(worker.capabilities, task.task_plan.required_capabilities)
checks.append(
Check(
"capabilities_match",
caps_ok,
"worker lacks required capabilities" if not caps_ok else "capabilities match",
)
)
allowed = all(check.ok for check in checks)
failed = next((check for check in checks if not check.ok), None)
reason = failed.detail if failed else None
error_code, status_code = self._error_for(failed)
decision = RecheckDecision(
recheck_id=f"rc-{uuid.uuid4()}",
proof_id=proof.proof_id,
task_id=task.task_id,
worker_id=worker.worker_id,
claim_id=proof.claim_id,
allowed=allowed,
reason=reason,
checks=checks,
observed_at=now,
error_code=error_code,
status_code=status_code,
)
decision.digest = compute_sha256_hex(decision.payload())
return decision
def _error_for(self, failed: Check | None) -> tuple[str | None, int]:
if failed is None:
return None, 400
entry = _A23_BY_CHECK.get(failed.check)
if entry is None:
return "forbidden", 403
return entry[0], entry[1].status_code
def to_error(self, decision: RecheckDecision) -> TaskPlaneHTTPError:
if decision.allowed:
raise ValueError("cannot build an error from an allowed decision")
failed = decision.failed_check()
if failed is None:
return ForbiddenError(decision.reason or "claim rejected")
code, exc_factory = _A23_BY_CHECK.get(failed.check, ("forbidden", ForbiddenError))
return exc_factory(decision.reason or failed.detail, code=code)
@@ -0,0 +1,191 @@
"""SchedulabilityProof model and ProofBuilder for governed claim.
A SchedulabilityProof is a durable, server-side diagnostic snapshot that records
the state evaluated before an Agent may claim a task. It is NOT a bearer
credential; it is persisted for auditability and idempotent read-back. The five
sections captured are binding / worker / runtime / observations / status,
following ANNEX-2.3 R2 ARCH-AUTO-CLAIM-R2-20260914 §4.1.
"""
from __future__ import annotations
import time
import uuid
from dataclasses import dataclass, field
from typing import Any, Callable
from .models import compute_sha256_hex
class ProofStatus:
PENDING = "PENDING"
ACCEPTED = "ACCEPTED"
REJECTED = "REJECTED"
CLAIMED = "CLAIMED"
FAILED = "FAILED"
@dataclass
class Generation:
task_revision: int
attempt: int | None = None
lease_generation: int | None = None
def to_dict(self) -> dict[str, Any]:
return {
"task_revision": self.task_revision,
"attempt": self.attempt,
"lease_generation": self.lease_generation,
}
@classmethod
def from_dict(cls, data: dict[str, Any] | None) -> "Generation":
data = data or {}
return cls(
task_revision=int(data.get("task_revision") or 0),
attempt=data.get("attempt"),
lease_generation=data.get("lease_generation"),
)
@dataclass
class SchedulabilityProof:
proof_id: str
task_id: str
worker_id: str
generated_at: float
binding: dict[str, Any]
worker: dict[str, Any]
runtime: dict[str, Any]
observations: list[dict[str, Any]]
status: str = ProofStatus.PENDING
schema_version: int = 1
claim_id: str | None = None
digest: str = ""
def payload(self) -> dict[str, Any]:
"""The digest-stable payload (excludes the digest field itself)."""
return {
"proof_id": self.proof_id,
"schema_version": self.schema_version,
"task_id": self.task_id,
"worker_id": self.worker_id,
"claim_id": self.claim_id,
"generated_at": self.generated_at,
"binding": self.binding,
"worker": self.worker,
"runtime": self.runtime,
"observations": self.observations,
"status": self.status,
}
def to_dict(self) -> dict[str, Any]:
return {**self.payload(), "digest": self.digest}
def compute_proof_digest(proof_or_dict: Any) -> str:
"""sha256 over canonical_json of the digest-stable proof payload."""
if isinstance(proof_or_dict, SchedulabilityProof):
return compute_sha256_hex(proof_or_dict.payload())
payload = dict(proof_or_dict)
payload.pop("digest", None)
return compute_sha256_hex(payload)
def observation(source: str, field_: str, value: Any, ok: bool, at: float) -> dict[str, Any]:
return {"source": source, "field": field_, "value": value, "ok": ok, "at": at}
def default_tenant_of_task(task: Any) -> str:
return task.task_plan.metadata.get("tenant_id", "default")
def default_tenant_of_worker(worker: Any) -> str:
return worker.trust_zone or "default"
class ProofBuilder:
"""Builds a SchedulabilityProof from live store snapshots."""
def __init__(
self,
store: Any,
*,
tenant_of_task: Callable[[Any], str] | None = None,
tenant_of_worker: Callable[[Any], str] | None = None,
) -> None:
self.store = store
self.tenant_of_task = tenant_of_task or default_tenant_of_task
self.tenant_of_worker = tenant_of_worker or default_tenant_of_worker
def build(
self,
task: Any,
worker: Any,
*,
now: float | None = None,
claim_id: str | None = None,
) -> SchedulabilityProof:
now = now if now is not None else time.time()
active = self.store.get_active_claim_count(worker.worker_id)
capacity_ok = active < worker.max_concurrency
online = (not worker.disabled) and (
now <= worker.last_heartbeat + (worker.heartbeat_interval_seconds * 3)
)
tenant_worker = self.tenant_of_worker(worker)
tenant_task = self.tenant_of_task(task)
binding = {
"worker_identity": worker.identity,
"capabilities": list(worker.capabilities),
"trust_zone": worker.trust_zone,
"tenant_id": tenant_worker,
}
binding["binding_digest"] = compute_sha256_hex(binding)
worker_snapshot = {
"status": worker.status,
"disabled": worker.disabled,
"last_heartbeat": worker.last_heartbeat,
"heartbeat_interval_seconds": worker.heartbeat_interval_seconds,
"online": online,
"worker_type": worker.worker_type,
"max_concurrency": worker.max_concurrency,
"current_active": active,
}
runtime = {
"max_concurrency": worker.max_concurrency,
"current_active": active,
"capacity_ok": capacity_ok,
"generation": {
"task_revision": task.revision,
"attempt": None,
"lease_generation": None,
},
}
observations = [
observation("tenant", "tenant_match", {"task": tenant_task, "worker": tenant_worker}, tenant_task == tenant_worker, now),
observation("principal", "worker_identity", worker.identity, True, now),
observation("authorization", "authorization_ok", None, True, now),
observation("task", "task_id", task.task_id, task.status in {"CREATED", "QUEUED", "FAILED_RECOVERABLE"}, now),
observation("worker", "worker_status", worker.status, not worker.disabled, now),
observation("generation", "task_revision", task.revision, True, now),
observation("capacity", "capacity_ok", capacity_ok, capacity_ok, now),
]
proof = SchedulabilityProof(
proof_id=f"proof-{uuid.uuid4()}",
task_id=task.task_id,
worker_id=worker.worker_id,
generated_at=now,
binding=binding,
worker=worker_snapshot,
runtime=runtime,
observations=observations,
status=ProofStatus.PENDING,
claim_id=claim_id,
)
proof.digest = compute_sha256_hex(proof.payload())
return proof
@@ -0,0 +1,179 @@
"""SchedulingService wiring governed claim dispatch + long_poll/poll/schedule.
The service owns the proof -> recheck -> govern pipeline and the worker pull
endpoints (long_poll / poll / schedule) with a process-local semaphore to bound
blocking long-poll concurrency. See ARCH-AUTO-CLAIM-R2-20260914 §1.5/§5/§6 T6.
"""
from __future__ import annotations
import threading
import time
from typing import Any, Callable
from .errors import ForbiddenError, LockedError, NotFoundError
from .recheck import RecheckEngine
from .schedulability import ProofBuilder, default_tenant_of_task, default_tenant_of_worker
from .scheduler import TaskScheduler
class SchedulingService:
def __init__(
self,
store: Any,
*,
recheck: RecheckEngine | None = None,
tenant_of_task: Callable[[Any], str] | None = None,
tenant_of_worker: Callable[[Any], str] | None = None,
long_poll_seconds: int = 15,
long_poll_max: int = 30,
poll_interval_seconds: int = 1,
long_poll_slots: int = 8,
) -> None:
self.store = store
self.recheck = recheck or RecheckEngine(
store,
tenant_of_task=tenant_of_task,
tenant_of_worker=tenant_of_worker,
)
self.builder = ProofBuilder(
store,
tenant_of_task=tenant_of_task or default_tenant_of_task,
tenant_of_worker=tenant_of_worker or default_tenant_of_worker,
)
self.scheduler = TaskScheduler(
store,
tenant_of_task=tenant_of_task or default_tenant_of_task,
tenant_of_worker=tenant_of_worker or default_tenant_of_worker,
)
self.long_poll_seconds = long_poll_seconds
self.long_poll_max = long_poll_max
self.poll_interval_seconds = poll_interval_seconds
self._semaphore = threading.Semaphore(long_poll_slots)
# -- internal pipeline -------------------------------------------------
def _claim_for(self, worker: Any, task: Any, lease_seconds: int, now: float) -> dict[str, Any]:
proof = self.builder.build(task, worker, now=now)
decision = self.recheck.authoritative_claim_recheck(
proof=proof, task=task, worker=worker, now=now
)
if not decision.allowed:
self.store.record_proof_rejection(
proof=proof, decision=decision, task=task, now=now
)
raise self.recheck.to_error(decision)
claim, receipt_id = self.store.governed_claim(
task=task,
worker=worker,
lease_seconds=lease_seconds,
proof=proof,
decision=decision,
now=now,
)
return {
"task": task.to_dict(),
"task_id": task.task_id,
"claim_id": claim.claim_id,
"worker_id": worker.worker_id,
"proof_id": proof.proof_id,
"receipt_id": receipt_id,
"generation": {
"task_revision": claim.task_revision,
"attempt": claim.attempt,
"lease_generation": claim.attempt,
},
}
# -- public endpoint semantics -----------------------------------------
def poll(self, worker_id: str, *, lease_seconds: int = 30) -> dict[str, Any]:
worker = self._validate_online(worker_id)
task = self.scheduler.next_queued_task(worker_id)
if task is None:
return {"status": "idle", "reason": "no_task"}
return self._claim_for(worker, task, lease_seconds, time.time())
def claim_governed(
self, worker_id: str, task_id: str, *, lease_seconds: int = 30
) -> dict[str, Any]:
worker = self.store.get_worker(worker_id)
task = self.store.get_task(task_id)
# G4: direct claim of a cross-tenant task stays FAIL-CLOSED (403 cross_tenant_denied),
# in contrast to enumeration which skips + audits.
w_tenant = self.builder.tenant_of_worker(worker)
t_tenant = self.builder.tenant_of_task(task)
if w_tenant != t_tenant:
raise ForbiddenError(
f"cross-tenant claim denied: task_tenant={t_tenant} worker_tenant={w_tenant}",
code="cross_tenant_denied",
)
return self._claim_for(worker, task, lease_seconds, time.time())
def long_poll(
self,
worker_id: str,
*,
timeout_seconds: int | None = None,
lease_seconds: int = 30,
) -> dict[str, Any]:
timeout = max(1, min(int(timeout_seconds or self.long_poll_seconds), self.long_poll_max))
worker = self._validate_online(worker_id)
acquired = self._semaphore.acquire(blocking=False)
try:
deadline = time.monotonic() + timeout
while True:
task = self.scheduler.next_queued_task(worker_id)
if task is not None:
return self._claim_for(worker, task, lease_seconds, time.time())
remaining = deadline - time.monotonic()
if remaining <= 0:
return {"status": "idle", "reason": "timeout"}
time.sleep(max(0.0, min(self.poll_interval_seconds, remaining)))
finally:
if acquired:
self._semaphore.release()
def schedule(self, worker_id: str) -> dict[str, Any]:
worker = self.store.get_worker(worker_id)
current = self.store.get_active_claim_count(worker_id)
online = (not worker.disabled) and (
time.time() <= worker.last_heartbeat + (worker.heartbeat_interval_seconds * 3)
)
worker_tenant = self.builder.tenant_of_worker(worker)
# G4: queued_count must exclude cross-tenant tasks (skip + audit; NOT fail-closed).
available: list[Any] = []
for t in self.store.list_tasks():
if t.status not in {"CREATED", "QUEUED", "FAILED_RECOVERABLE"}:
continue
if t.task_plan.task_type == "worker.registry.audit":
continue
t_tenant = self.builder.tenant_of_task(t)
if t_tenant != worker_tenant:
self.store.record_tenant_skip_audit(
t.task_id,
worker_id,
task_tenant=t_tenant,
worker_tenant=worker_tenant,
reason="cross_tenant_skip",
)
continue
available.append(t)
return {
"worker_id": worker_id,
"online": online,
"disabled": worker.disabled,
"max_concurrency": worker.max_concurrency,
"current_active": current,
"capacity_ok": current < worker.max_concurrency,
"queued_count": len(available),
"status": "ready" if online else "offline",
}
def _validate_online(self, worker_id: str) -> Any:
worker = self.store.get_worker(worker_id)
if worker.disabled:
raise LockedError("worker disabled")
if time.time() > worker.last_heartbeat + (worker.heartbeat_interval_seconds * 3):
raise LockedError("worker offline")
return worker
+51 -12
View File
@@ -3,18 +3,37 @@
from __future__ import annotations
import time
from .errors import NotFoundError
from typing import Any, Callable
from .errors import LockedError, NotFoundError
from .capabilities import has_capabilities
from .models import WorkerClaim
from .recheck import RecheckEngine
from .schedulability import (
ProofBuilder,
default_tenant_of_task,
default_tenant_of_worker,
)
from .store import TaskPlaneStore
class TaskScheduler:
def __init__(self, store: TaskPlaneStore):
def __init__(
self,
store: TaskPlaneStore,
*,
recheck: RecheckEngine | None = None,
tenant_of_task: Callable[[Any], str] | None = None,
tenant_of_worker: Callable[[Any], str] | None = None,
):
self.store = store
self.recheck = recheck or RecheckEngine(store)
self.builder = ProofBuilder(store)
self._tenant_of_task = tenant_of_task or default_tenant_of_task
self._tenant_of_worker = tenant_of_worker or default_tenant_of_worker
def next_queued_task(self, worker_id: str):
worker = self.store.get_worker(worker_id)
worker_tenant = self._tenant_of_worker(worker)
tasks = sorted(
self.store.list_tasks(),
key=lambda task: (
@@ -28,6 +47,17 @@ class TaskScheduler:
continue
if task.task_plan.task_type == "worker.registry.audit":
continue
# G4: enumeration = skip + audit for cross-tenant tasks (NOT fail-closed).
task_tenant = self._tenant_of_task(task)
if task_tenant != worker_tenant:
self.store.record_tenant_skip_audit(
task.task_id,
worker_id,
task_tenant=task_tenant,
worker_tenant=worker_tenant,
reason="cross_tenant_skip",
)
continue
if task.status == "FAILED_RECOVERABLE" and not self.store.task_has_retryable_probe(
task.task_id
):
@@ -53,24 +83,33 @@ class TaskScheduler:
def schedule_for_worker(self, worker_id: str, lease_seconds: int = 15) -> WorkerClaim:
worker = self.store.get_worker(worker_id)
if worker.disabled:
from .errors import LockedError
raise LockedError("worker disabled")
if time.time() > worker.last_heartbeat + (
worker.heartbeat_interval_seconds * 3
):
from .errors import LockedError
raise LockedError("worker offline")
task = self.next_queued_task(worker_id)
if task is None:
raise NotFoundError("no queued tasks available")
claim = self.store.claim_task(
task.task_id,
worker_id,
worker.identity,
return self._governed_claim(worker, task, lease_seconds)
def _governed_claim(self, worker, task, lease_seconds: int) -> WorkerClaim:
now = time.time()
proof = self.builder.build(task, worker, now=now)
decision = self.recheck.authoritative_claim_recheck(
proof=proof, task=task, worker=worker, now=now
)
if not decision.allowed:
self.store.record_proof_rejection(
proof=proof, decision=decision, task=task, now=now
)
raise self.recheck.to_error(decision)
claim, _ = self.store.governed_claim(
task=task,
worker=worker,
lease_seconds=lease_seconds,
expected_revision=task.revision,
operation_digest=task.task_plan.operation_digest(),
proof=proof,
decision=decision,
now=now,
)
return claim
+436
View File
@@ -32,6 +32,7 @@ from .models import (
redact_value,
validate_identifier,
)
from .receipts import build_receipt
EventEnvelope = tuple[str, str, str, str, str, float]
@@ -272,6 +273,47 @@ class TaskPlaneStore:
)
"""
)
self.conn.execute(
"""
CREATE TABLE IF NOT EXISTS schedulability_proofs (
proof_id TEXT PRIMARY KEY,
task_id TEXT NOT NULL,
worker_id TEXT NOT NULL,
claim_id TEXT,
status TEXT NOT NULL,
envelope_json TEXT NOT NULL,
digest TEXT NOT NULL,
created_at REAL NOT NULL,
updated_at REAL NOT NULL
)
"""
)
self.conn.execute(
"""
CREATE TABLE IF NOT EXISTS receipts (
receipt_id TEXT PRIMARY KEY,
kind TEXT NOT NULL,
task_id TEXT NOT NULL,
claim_id TEXT,
proof_id TEXT,
worker_id TEXT NOT NULL,
digest TEXT NOT NULL,
payload_json TEXT NOT NULL,
created_at REAL NOT NULL
)
"""
)
self.conn.execute(
"""
CREATE TABLE IF NOT EXISTS tenant_skip_audit (
task_id TEXT NOT NULL,
worker_id TEXT NOT NULL,
finding_json TEXT NOT NULL,
created_at REAL NOT NULL,
PRIMARY KEY (task_id, worker_id)
)
"""
)
self._ensure_columns(
"workers",
{
@@ -305,6 +347,12 @@ class TaskPlaneStore:
"CREATE UNIQUE INDEX IF NOT EXISTS ux_events_task_seq ON events(task_id, seq)"
)
self.conn.execute("CREATE INDEX IF NOT EXISTS ix_outbox_status ON outbox(delivery_status)")
self.conn.execute(
"CREATE INDEX IF NOT EXISTS ix_proofs_task ON schedulability_proofs(task_id)"
)
self.conn.execute(
"CREATE INDEX IF NOT EXISTS ix_receipts_task ON receipts(task_id)"
)
def _ensure_columns(self, table: str, columns: dict[str, str]) -> None:
existing = {
@@ -1593,6 +1641,394 @@ class TaskPlaneStore:
).fetchone()
return row["parent_task_id"] if row else None
def get_active_claim_count(self, worker_id: str) -> int:
"""Number of currently active (non-expired) claims for a worker."""
with self._lock:
row = self.conn.execute(
"SELECT COUNT(1) FROM claims WHERE worker_id=? AND status='ACTIVE' AND lease_until>?",
(worker_id, _now()),
).fetchone()
return int(row[0])
# -- governed proof / receipt persistence --------------------------------
def save_schedulability_proof(self, proof) -> str:
"""Persist an upfront PENDING proof snapshot (idempotent upsert)."""
if hasattr(proof, "to_dict"):
proof_dict = proof.to_dict()
else:
proof_dict = dict(proof)
envelope = {k: v for k, v in proof_dict.items() if k != "digest"}
digest = _hash_hex(_canonical(envelope))
now = _now()
with self._transaction():
self.conn.execute(
"""
INSERT INTO schedulability_proofs
(proof_id, task_id, worker_id, claim_id, status, envelope_json, digest, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(proof_id) DO UPDATE SET
claim_id=excluded.claim_id,
status=excluded.status,
envelope_json=excluded.envelope_json,
digest=excluded.digest,
updated_at=excluded.updated_at
""",
(
proof_dict["proof_id"],
proof_dict["task_id"],
proof_dict["worker_id"],
proof_dict.get("claim_id"),
proof_dict.get("status", "PENDING"),
json.dumps(envelope, ensure_ascii=False),
digest,
now,
now,
),
)
return proof_dict["proof_id"]
def _insert_proof(
self,
proof,
status: str,
*,
claim_id: str | None = None,
generation: dict | None = None,
observations: list | None = None,
now: float | None = None,
) -> None:
now = now if now is not None else _now()
envelope = proof.payload() if hasattr(proof, "payload") else dict(proof)
envelope = {k: v for k, v in envelope.items() if k != "digest"}
if claim_id is not None:
envelope["claim_id"] = claim_id
envelope["status"] = status
runtime = envelope.setdefault("runtime", {})
if generation is not None:
runtime["generation"] = generation
if observations is not None:
envelope["observations"] = observations
digest = _hash_hex(_canonical(envelope))
self.conn.execute(
"""
INSERT INTO schedulability_proofs
(proof_id, task_id, worker_id, claim_id, status, envelope_json, digest, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(proof_id) DO UPDATE SET
claim_id=excluded.claim_id,
status=excluded.status,
envelope_json=excluded.envelope_json,
digest=excluded.digest,
updated_at=excluded.updated_at
""",
(
envelope["proof_id"],
envelope["task_id"],
envelope["worker_id"],
envelope.get("claim_id"),
status,
json.dumps(envelope, ensure_ascii=False),
digest,
now,
now,
),
)
def get_schedulability_proof(self, proof_id: str) -> dict:
proof_id = _validated_id(proof_id, "proof_id")
with self._lock:
row = self.conn.execute(
"SELECT * FROM schedulability_proofs WHERE proof_id=?", (proof_id,)
).fetchone()
if row is None:
raise NotFoundError(f"proof {proof_id} not found")
envelope = json.loads(row["envelope_json"])
envelope["digest"] = row["digest"]
return envelope
def _insert_receipt(self, receipt) -> str:
if hasattr(receipt, "to_dict"):
receipt_id = receipt.receipt_id
kind = receipt.kind
task_id = receipt.task_id
claim_id = receipt.claim_id
proof_id = receipt.proof_id
worker_id = receipt.worker_id
digest = receipt.digest
payload = receipt.payload
created_at = receipt.created_at
else:
data = dict(receipt)
receipt_id = data["receipt_id"]
kind = data["kind"]
task_id = data["task_id"]
claim_id = data.get("claim_id")
proof_id = data.get("proof_id")
worker_id = data["worker_id"]
digest = data["digest"]
payload = data.get("payload", {})
created_at = data.get("created_at", _now())
self.conn.execute(
"INSERT INTO receipts(receipt_id, kind, task_id, claim_id, proof_id, worker_id, digest, payload_json, created_at) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
(
receipt_id,
kind,
task_id,
claim_id,
proof_id,
worker_id,
digest,
json.dumps(payload, ensure_ascii=False),
created_at,
),
)
return receipt_id
def save_receipt(self, receipt) -> str:
with self._transaction():
return self._insert_receipt(receipt)
def get_receipt(self, receipt_id: str) -> dict:
receipt_id = _validated_id(receipt_id, "receipt_id")
with self._lock:
row = self.conn.execute(
"SELECT * FROM receipts WHERE receipt_id=?", (receipt_id,)
).fetchone()
if row is None:
raise NotFoundError(f"receipt {receipt_id} not found")
result = dict(row)
result["payload"] = json.loads(row["payload_json"])
return result
def list_receipts(self, task_id: str) -> list[dict]:
task_id = _validated_id(task_id, "task_id")
with self._lock:
rows = self.conn.execute(
"SELECT * FROM receipts WHERE task_id=? ORDER BY created_at ASC",
(task_id,),
).fetchall()
result = []
for row in rows:
item = dict(row)
item["payload"] = json.loads(row["payload_json"])
result.append(item)
return result
# -- atomic governed claim ----------------------------------------------
def governed_claim(
self,
*,
task,
worker,
lease_seconds: int,
proof,
decision,
now: float | None = None,
):
"""Atomically claim via the governed path.
Within a single reentrant transaction: reuse the claim_task kernel
(CAS/lease/fencing/concurrency), persist the proof as CLAIMED with the
re-read evidence, write claim+proof receipts geared to read-back, and
append the CLAIM_RECHECK_ACCEPTED / CLAIM_GOVERNED_ACCEPTED events.
"""
now = now if now is not None else _now()
with self._transaction():
claim = self.claim_task(
task.task_id,
worker.worker_id,
worker.identity,
lease_seconds=lease_seconds,
expected_revision=task.revision,
operation_digest=task.task_plan.operation_digest(),
)
generation = {
"task_revision": claim.task_revision,
"attempt": claim.attempt,
"lease_generation": claim.attempt,
}
self._insert_proof(
proof,
"CLAIMED",
claim_id=claim.claim_id,
generation=generation,
observations=[c.to_dict() for c in decision.checks],
now=now,
)
claim_receipt = build_receipt(
kind="claim",
task_id=task.task_id,
worker_id=worker.worker_id,
claim_id=claim.claim_id,
proof_id=proof.proof_id,
extra={
"status": "CLAIMED",
"attempt": claim.attempt,
"task_revision": claim.task_revision,
"lease_until": claim.lease_until,
},
created_at=now,
)
proof_receipt = build_receipt(
kind="proof",
task_id=task.task_id,
worker_id=worker.worker_id,
claim_id=claim.claim_id,
proof_id=proof.proof_id,
extra={"status": "CLAIMED", "allowed": True, "recheck_id": decision.recheck_id},
created_at=now,
)
self._insert_receipt(claim_receipt)
self._insert_receipt(proof_receipt)
self.append_event(
task.task_id,
"CLAIM_RECHECK_ACCEPTED",
{
"proof_id": proof.proof_id,
"worker_id": worker.worker_id,
"claim_id": claim.claim_id,
"recheck_id": decision.recheck_id,
"allowed": True,
"generation": {"task_revision": task.revision, "attempt": None, "lease_generation": None},
},
subject=f"task/{task.task_id}",
)
self.append_event(
task.task_id,
"CLAIM_GOVERNED_ACCEPTED",
{
"claim_id": claim.claim_id,
"proof_id": proof.proof_id,
"worker_id": worker.worker_id,
"generation": generation,
},
subject=f"task/{task.task_id}",
)
return claim, claim_receipt.receipt_id
def record_tenant_skip_audit(
self,
task_id: str,
worker_id: str,
*,
task_tenant: str,
worker_tenant: str,
reason: str = "cross_tenant_skip",
now: float | None = None,
) -> dict[str, Any]:
"""Persist a structured 'enumeration=skip+audit' finding for a
cross-tenant task that a worker's enumeration was NOT allowed to see.
Contract (G4): enumeration of cross-tenant tasks must SKIP and record a
structured audit finding -- it must NOT raise / fail closed. Direct
claim remains fail-closed (403 cross_tenant_denied) elsewhere.
"""
now = now if now is not None else _now()
finding = {
"source": "tenant",
"kind": "cross_tenant_skip",
"decision": "skip+audit",
"task_tenant": task_tenant,
"worker_tenant": worker_tenant,
"task_id": task_id,
"worker_id": worker_id,
"reason": reason,
}
with self._transaction():
self.conn.execute(
"""
INSERT INTO tenant_skip_audit(task_id, worker_id, finding_json, created_at)
VALUES (?, ?, ?, ?)
ON CONFLICT(task_id, worker_id) DO UPDATE SET
finding_json=excluded.finding_json,
created_at=excluded.created_at
""",
(task_id, worker_id, json.dumps(finding, ensure_ascii=False), now),
)
self.append_event(
task_id,
"TASK_TENANT_SKIPPED",
finding,
subject=f"task/{task_id}",
)
return finding
def list_tenant_skip_audit(self, task_id: str | None = None) -> list[dict[str, Any]]:
"""Read-back of recorded cross-tenant skip findings (testability)."""
if task_id is None:
with self._lock:
rows = self.conn.execute(
"SELECT task_id, worker_id, finding_json, created_at FROM tenant_skip_audit ORDER BY created_at ASC"
).fetchall()
else:
task_id = _validated_id(task_id, "task_id")
with self._lock:
rows = self.conn.execute(
"SELECT task_id, worker_id, finding_json, created_at FROM tenant_skip_audit WHERE task_id=? ORDER BY created_at ASC",
(task_id,),
).fetchall()
out: list[dict[str, Any]] = []
for r in rows:
finding = json.loads(r["finding_json"])
out.append(
{
"task_id": r["task_id"],
"worker_id": r["worker_id"],
"finding": finding,
"created_at": r["created_at"],
}
)
return out
def record_proof_rejection(
self,
*,
proof,
decision,
task,
now: float | None = None,
) -> str:
"""Atomically persist a rejected proof + rejection receipt + event."""
now = now if now is not None else _now()
with self._transaction():
self._insert_proof(
proof,
"REJECTED",
observations=[c.to_dict() for c in decision.checks],
now=now,
)
proof_receipt = build_receipt(
kind="proof",
task_id=task.task_id,
worker_id=proof.worker_id,
proof_id=proof.proof_id,
extra={
"status": "REJECTED",
"allowed": False,
"reason": decision.reason,
"error_code": decision.error_code,
"recheck_id": decision.recheck_id,
},
created_at=now,
)
self._insert_receipt(proof_receipt)
self.append_event(
task.task_id,
"CLAIM_RECHECK_REJECTED",
{
"proof_id": proof.proof_id,
"worker_id": proof.worker_id,
"reason": decision.reason,
"error_code": decision.error_code,
"generation": {"task_revision": task.revision, "attempt": None, "lease_generation": None},
},
subject=f"task/{task.task_id}",
)
return proof_receipt.receipt_id
def _compute_depth(self, task_id: str) -> int:
depth = 0
visited = set()
@@ -0,0 +1,168 @@
"""toil_governor: Retry / Toil Governor 骨架。
实现 V4 §4 / V3 TrackA A2:在同 operation + 同 blocker、且无新证据的前提下抑制
重复重试;blocker 变化或 record_success 后重置降频;长期外部 blocker 自动降频。
UNKNOWN blocker 不直接放行,置 reconcile_required 标记由调用方先 reconcile。
本模块内存态(进程内),无持久化、无写操作、无网络。
"""
from __future__ import annotations
import hashlib
import json
import time
from dataclasses import dataclass, field
from typing import Optional
@dataclass(frozen=True)
class RetryGovernorDecision:
allow: bool
reason: str
next_eligible_at: Optional[float] = None
suppressed_count: int = 0
def to_dict(self) -> dict:
return {
"allow": self.allow,
"reason": self.reason,
"next_eligible_at": self.next_eligible_at,
"suppressed_count": self.suppressed_count,
"reconcile_required": "reconcile" in self.reason,
}
_UNKNOWN_BLOCKER_FP = "UNKNOWN"
class ToilGovernor:
"""内存中的重试抑制器。同指纹退化,blocker 变化重置。"""
def __init__(self, min_backoff: float = 2.0):
self._min_backoff = min_backoff
# operation -> 状态
self._state: dict[str, dict] = {}
# blocker 指纹 -> 连续失败次数(用于自动降频)
self._blocker_failures: dict[str, int] = {}
# ------------------------------------------------------------ key
def operation_key(self, fingerprint: str) -> str:
"""规范化 operation 指纹为稳定 key。"""
if not fingerprint:
raise ValueError("operation fingerprint must not be empty")
fp = fingerprint.strip()
if len(fp) <= 64:
return fp
digest = hashlib.sha256(fp.encode("utf-8")).hexdigest()[:16]
return f"{fp[:40]}..{digest}"
def blocker_fingerprint(self, blocker) -> str:
"""把 blocker(str / dict / object)规范化为指纹字符串。"""
if blocker is None:
return _UNKNOWN_BLOCKER_FP
if isinstance(blocker, str):
return _UNKNOWN_BLOCKER_FP if blocker.strip() in ("", "unknown") else blocker.strip()
if isinstance(blocker, dict):
try:
return json.dumps(blocker, sort_keys=True, ensure_ascii=False, default=str)
except Exception:
return _UNKNOWN_BLOCKER_FP
try:
return f"{type(blocker).__name__}:{hash(blocker)}"
except Exception: # pragma: no cover - 不可哈希对象兜底
return _UNKNOWN_BLOCKER_FP
# ------------------------------------------------------------ decide
def decide(
self,
operation,
blocker=None,
now: Optional[float] = None,
backoff_base: float = 60.0,
) -> RetryGovernorDecision:
"""判断本次重试是否放行。
- 同 operation 同 blocker 连续调用且无成功记录 → 抑制(allow=False),
下次可重试时间按 backoff_base * 2^(连续次数-1) 退避。
- blocker 指纹与上上次不同 → 视为新证据,放行并重置连续计数。
- blocker 为 UNKNOWN → 不直接放行,置 reconcile_required,返回 UNKNOWN 语义。
"""
when = now if now is not None else time.time()
op = self.operation_key(operation)
bfp = self.blocker_fingerprint(blocker)
state = self._state.setdefault(
op, {"blocker": None, "failures": 0, "last_allowed": None}
)
# —— UNKNOWN blocker:不直接放行,需 reconcile ——
if bfp == _UNKNOWN_BLOCKER_FP:
cur = self._blocker_failures.get(bfp, 0) + 1
self._blocker_failures[bfp] = cur
delay = backoff_base * (2 ** max(0, cur - 1))
return RetryGovernorDecision(
allow=False,
reason="reconcile_required: unknown blocker, do not bypass",
next_eligible_at=when + delay,
suppressed_count=cur - 1,
)
# —— blocker 变化(新证据)→ 放行并重置 ——
prev_bfp = state["blocker"]
if prev_bfp is not None and prev_bfp != bfp:
state["blocker"] = bfp
state["failures"] = 0
state["last_allowed"] = when
self._blocker_failures[bfp] = 1
return RetryGovernorDecision(allow=True, reason="blocker_changed")
# —— 同 blocker 连续 ——
if state["blocker"] is not None:
state["failures"] += 1
else:
state["blocker"] = bfp
state["failures"] = 0
state["last_allowed"] = when
self._blocker_failures[bfp] = 1
return RetryGovernorDecision(allow=True, reason="first_attempt")
failures = state["failures"]
# 长期外部 blocker 自动降频:连续失败次数越多,退避越久
delay = backoff_base * (2 ** max(0, failures - 1))
return RetryGovernorDecision(
allow=False,
reason="suppressed: same blocker, no new evidence",
next_eligible_at=when + delay,
suppressed_count=failures,
)
# ------------------------------------------------------------ record
def record_success(self, operation, now: Optional[float] = None) -> None:
"""记录一次成功,重置该 operation 的降频与连续计数。"""
op = self.operation_key(operation)
state = self._state.get(op)
if state is not None:
state["blocker"] = None
state["failures"] = 0
state["last_allowed"] = now if now is not None else time.time()
else:
self._state[op] = {
"blocker": None,
"failures": 0,
"last_allowed": now if now is not None else time.time(),
}
# ------------------------------------------------------------ stats
def stats(self) -> dict:
return {
"operations": len(self._state),
"blocker_fingerprints": len(self._blocker_failures),
"suppressed_by_blocker": sum(
1 for s in self._state.values() if s["failures"] > 0
),
"blocker_failure_counts": dict(self._blocker_failures),
}
+201
View File
@@ -0,0 +1,201 @@
"""Gate / Evidence Engine — verdict model and evaluation (WP5 / P0-5).
The verdict is the auditable result of checking whether a scope's required
evidence set is fully satisfied. The single governing rule is:
PASS <=> every required evidence for the scope is present.
Otherwise => NOT_VERIFIED. An explicit blocker => BLOCKED.
Proof / evidence entries are audit snapshots, NOT bearer credentials. A
service being "active", a file "existing", or a historical evidence entry
does NOT imply a PASS for the runtime/business scope. See
ARCH-AUTO-CLAIM-R2-20260914 §4.3.
"""
from __future__ import annotations
import time
from dataclasses import dataclass, field
from enum import StrEnum
from typing import Any, Iterable, Mapping
VALID_SCOPES: frozenset[str] = frozenset(
{"unit", "contract", "staging", "runtime", "business"}
)
class VerdictStatus(StrEnum):
"""Ordered fail-closed verdict states."""
PASS = "PASS"
FAIL = "FAIL"
NOT_VERIFIED = "NOT_VERIFIED"
BLOCKED = "BLOCKED"
DIVERGED = "DIVERGED"
@dataclass(frozen=True)
class EvidenceRequirement:
"""A single expected evidence entry for a scope."""
name: str
present: bool = False
detail: str = ""
@dataclass(frozen=True)
class Verdict:
"""Immutable audit record produced by evaluate_verdict."""
schema_version = "aios.verdict.v1"
scope: str
status: VerdictStatus
subject: str
checks: tuple[dict[str, Any], ...] = ()
evidence_refs: tuple[str, ...] = ()
missing_evidence: tuple[str, ...] = ()
reason: str = ""
decided_at: float = field(default_factory=time.time)
def to_dict(self) -> dict[str, Any]:
return {
"schema_version": self.schema_version,
"scope": self.scope,
"status": str(self.status.value),
"subject": self.subject,
"checks": list(self.checks),
"evidence_refs": list(self.evidence_refs),
"missing_evidence": list(self.missing_evidence),
"reason": self.reason,
"decided_at": self.decided_at,
}
# Marker names carried by an EvidenceRequirement that explicitly flag a blocker
# when present (fail-closed on blocked conditions even inside the evidence set).
BLOCKER_NAME_MARKERS = ("blocked", "blocker", "forbidden", "denied")
def _present_names(reqs: list[EvidenceRequirement]) -> set[str]:
return {req.name for req in reqs if req.present}
def _detect_blocker(reqs: list[EvidenceRequirement]) -> EvidenceRequirement | None:
for req in reqs:
lowered = req.name.lower()
if any(marker in lowered for marker in BLOCKER_NAME_MARKERS):
return req
return None
def evaluate_verdict(
*,
scope: str,
subject: str,
required_evidence: Iterable[EvidenceRequirement]
| Mapping[str, EvidenceRequirement]
| Iterable[str]
| None,
available_evidence: Iterable[str] | None = None,
reason: str = "",
blocked: bool = False,
now: float | None = None,
) -> Verdict:
"""Evaluate a scope's evidence closure into a Verdict.
PASS only when every required evidence is present. Missing evidence ->
NOT_VERIFIED. An explicit blocker (either `blocked=True` or a present
evidence name matching a blocker marker) -> BLOCKED. An unknown scope ->
FAIL (fail-closed / parameter error).
"""
now = now if now is not None else time.time()
if scope not in VALID_SCOPES:
return Verdict(
scope=scope,
status=VerdictStatus.FAIL,
subject=subject,
reason=reason or f"invalid scope: {scope!r}",
decided_at=now,
)
reqs: list[EvidenceRequirement] = _normalize_requirements(required_evidence)
available: set[str] = _normalize_available(available_evidence)
checks: list[dict[str, Any]] = []
present: set[str] = set()
missing: list[str] = []
for req in reqs:
is_present = req.present or (req.name in available)
checks.append(
{
"check": f"evidence:{req.name}",
"ok": is_present,
"detail": req.detail,
}
)
if is_present:
present.add(req.name)
else:
missing.append(req.name)
evidence_refs = tuple(
name for name in (req.name for req in reqs) if name in present
)
blocker = _detect_blocker(reqs)
if blocked or blocker is not None:
status = VerdictStatus.BLOCKED
effective_reason = reason or (
f"explicit blocked condition: {blocker.name}" if blocker else "blocked"
)
elif missing:
status = VerdictStatus.NOT_VERIFIED
effective_reason = reason or (
f"missing evidence: {', '.join(missing)}"
)
else:
status = VerdictStatus.PASS
effective_reason = reason or "all required evidence present"
return Verdict(
scope=scope,
status=status,
subject=subject,
checks=tuple(checks),
evidence_refs=evidence_refs,
missing_evidence=tuple(missing),
reason=effective_reason,
decided_at=now,
)
def _normalize_requirements(
required: Iterable[EvidenceRequirement]
| Mapping[str, EvidenceRequirement]
| Iterable[str]
| None,
) -> list[EvidenceRequirement]:
if required is None:
return []
if isinstance(required, Mapping):
return [
value if isinstance(value, EvidenceRequirement)
else EvidenceRequirement(name=key, present=bool(value), detail="")
for key, value in required.items()
]
out: list[EvidenceRequirement] = []
for item in required:
if isinstance(item, EvidenceRequirement):
out.append(item)
else:
out.append(EvidenceRequirement(name=str(item)))
return out
def _normalize_available(available: Iterable[str] | None) -> set[str]:
if available is None:
return set()
return {str(name) for name in available}
+76
View File
@@ -0,0 +1,76 @@
{
"schema": "aios.system-truth-contract.v1",
"state_ownership": {
"TaskAdmission": {
"owns": [
"envelope",
"authorization",
"grant",
"admission_receipt"
],
"must_not_own": [
"claim",
"heartbeat",
"execution_status",
"commit_result"
]
},
"TaskPlane": {
"owns": [
"task_state",
"claim",
"lease",
"heartbeat",
"attempt",
"deadline"
],
"must_not_own": [
"owner_signature",
"git_commit_truth"
]
},
"GitAuthority": {
"owns": [
"branch_oid",
"commit_oid",
"file_bytes"
],
"must_not_own": [
"task_state"
]
},
"RuntimeProjection": {
"owns": [
"deployed_source_oid",
"served_routes",
"external_action_readback"
],
"must_not_own": [
"source_commit_status"
]
}
},
"completion_requires": [
"valid_claim_holder",
"fresh_worker_heartbeat",
"valid_writer_lease",
"commit_receipt",
"post_oid_differs_from_pre_oid",
"output_sha256_matches_plan",
"runtime_readback",
"external_projection_readback"
],
"queue_requires": [
"fresh_schedulability_proof",
"at_least_one_healthy_scope_matching_worker",
"grant_or_jit_principal_can_cover_claim_window"
],
"forbidden_equivalences": [
"ADMITTED == EXECUTED",
"QUEUED == CLAIMED",
"service_active == scenario_pass",
"drift_status_PASS == evidence_complete",
"hash_declared == artifact_exists",
"OpenAPI_operation == backend_route_exists"
]
}
@@ -0,0 +1,64 @@
# ADR: AIOS Control-Plane System Truth and Execution Integrity
## Status
Proposed implementation baseline for ANNEX-2.3 hardening.
## Problem
Historical failures show that multiple planes can each report locally valid state while the end-to-end goal is false: Admission can be ADMITTED while TaskPlane has no claim; a task can be QUEUED until its grant expires; a worker can have matching capability names but an exact scope for another job; OpenAPI can advertise a route absent from the deployed backend; and PASS labels can be emitted without production evidence.
## Decision
Do not create a third persistence plane. Keep one authority per state dimension and derive `SystemTruth` read-only.
| State dimension | Authority |
|---|---|
| envelope / grant / admission receipt | Task Admission |
| task lifecycle / claim / attempt / heartbeat / deadline | TaskPlane |
| writer lease | governed writer / lease authority |
| source bytes / branch and commit OIDs | Git authority |
| deployed source / served routes | runtime |
| external ChatGPT Action operations | external projection readback |
### 1. Admission is not execution
`ADMITTED` means authorization has been accepted. It must never be presented as running, claimed, committed, or complete.
### 2. Schedulability before QUEUED
Before TaskPlane projection to `QUEUED`, produce a fresh `SchedulabilityProof` containing:
- required capabilities
- exact task/job/branch/execution-digest scope
- at least one healthy eligible worker
- current grant/principal viability
- dependencies/freeze state
If no eligible worker exists, keep execution projection in `WAITING_CAPACITY` (Admission may remain ADMITTED). Do not burn a short one-time grant while the task is waiting for capacity.
### 3. Mint runtime authority just-in-time
Separate long-lived Owner authorization from short-lived runtime TaskPrincipal. Mint the execution principal at claim time or immediately before claim, not at admission time. This prevents 15-minute grants expiring in a queue with no eligible worker.
### 4. Exact-scope is mandatory
Capabilities alone never establish eligibility. A worker is eligible only when `task_id`, `external_job_id`, `target_branch`, `generation`, and `execution_digest` are within its signed scope.
### 5. No fake execution
Transitions require evidence:
- CLAIMED: claim holder + fencing generation + lease readback
- RUNNING: CLAIMED + fresh heartbeat + valid lease
- SUCCEEDED: commit receipt + pre/post OID change + planned file hashes + runtime readback
- SCENARIO_ACCEPTED: SUCCEEDED + external projection readback
`service active`, `drift_status=PASS`, focused tests, or an Admission receipt are not substitutes.
### 6. Real bytes, not placeholders
Execution plans must reference content-addressed bytes that exist before admission. Hash-only plans with missing bytes, zero-length files, TODO/pass placeholders, or mismatched hashes are rejected.
### 7. One source for route contracts
Backend route registry and external OpenAPI must derive from one canonical route manifest, or be parity-checked in CI. Schema-only deployment is blocked if the matching backend route is absent; backend-only deployment is blocked if the external projection is absent when required.
### 8. Dual-plane reconciliation
Projection is idempotent and read-back verified. Admission writes an append-only projection intent/outbox in the same transaction as the grant; a projector writes TaskPlane, reads back exact task identity/revision, and records a receipt. Divergence is a first-class failure, never silently masked.
## Required release gates
- no BLOCKER from control-plane consistency audit
- C4 regression: legacy task creation path remains TaskPrincipal-gated
- admission -> CLAIMED -> RUNNING -> WriterLease -> governed commit -> readback integration test
- real activation + real SSH signature host test
- external projection readback test
- no placeholder artifact findings
@@ -0,0 +1,33 @@
# Remediation Work Packages
## P0-A — Single System Truth (no new DB)
Implement the read-only SystemTruth derivation in the runtime explain/status surface. Remove or rename PASS labels that are configuration-only.
## P0-B — Schedulability Gate
Admission may authorize a task, but TaskPlane projection to QUEUED requires a fresh SchedulabilityProof. If no eligible worker exists, expose WAITING_CAPACITY and do not start the short runtime grant TTL.
## P0-C — Just-in-time TaskPrincipal
Keep Owner authorization durable; mint short-lived TaskPrincipal only when an eligible worker is ready to claim. Bind task/job/branch/generation/execution_digest exactly.
## P0-D — Evidence-required state transitions
Guard CLAIMED/RUNNING/SUCCEEDED with claim holder, heartbeat, lease, commit receipt, OID change, file hashes, runtime readback, external projection readback.
## P0-E — Artifact reality gate
Before admission, verify every planned file byte exists, is non-placeholder, and matches SHA256. Reject hash-only or placeholder plans.
## P0-F — Route/projection single source
Generate or verify ChatGPT Actions OpenAPI from the backend route manifest. Block dead endpoints and unprojected required operations.
## P0-G — Dual-plane durable projection
Use append-only projection intent + idempotent TaskPlane write + exact readback receipt. Admission status and execution status remain separate.
## P0-H — Worker scope/schedulability diagnostics
Eligibility = capability AND heartbeat AND exact signed scope. Never select a worker only because capability names match.
## P0-I — QA gaps
Add:
1. C4 legacy createGovernedTask/projectOnlineAgentExecution remains TaskPrincipal-gated.
2. `_verified_task_principal` body injection negative test.
3. bearer-only no-delegation negative test.
4. real CLAIMED/RUNNING/WriterLease/commit readback path.
5. real activation/SSH signature host test.
+44
View File
@@ -782,3 +782,47 @@ paths:
'200': {description: Safe GitOps plan}
'401': {description: Unauthorized}
x-openai-isConsequential: true
/reopenOwnerApproval:
post:
operationId: reopenOwnerApproval
summary: Reopen one expired Owner approval cycle without rebuilding release bytes
security:
- bearerAuth: []
x-openai-isConsequential: true
requestBody:
required: true
content:
application/json:
schema:
type: object
additionalProperties: false
required:
- schema
- candidate_id
- expected_approval_fingerprint
- expected_source_commit
- expected_tree_oid
- expected_artifact_sha256
- expected_manifest_sha256
- expected_release_plan_sha256
- expected_policy_sha256
- reason
- idempotency_key
properties:
schema:
type: string
enum: [aios.owner-approval-reopen-request.v1]
candidate_id: {type: string}
expected_approval_fingerprint: {type: string, pattern: "^[0-9a-f]{64}$"}
expected_source_commit: {type: string, pattern: "^[0-9a-f]{40}$"}
expected_tree_oid: {type: string, pattern: "^[0-9a-f]{40}$"}
expected_artifact_sha256: {type: string, pattern: "^[0-9a-f]{64}$"}
expected_manifest_sha256: {type: string, pattern: "^[0-9a-f]{64}$"}
expected_release_plan_sha256: {type: string, pattern: "^[0-9a-f]{64}$"}
expected_policy_sha256: {type: string, pattern: "^[0-9a-f]{64}$"}
reason: {type: string, minLength: 1, maxLength: 512}
idempotency_key: {type: string, minLength: 8, maxLength: 128}
responses:
"200": {description: Reopened or idempotent replay}
"403": {description: Missing release:approve capability}
"409": {description: Governed reopen gate rejected}
+57 -42
View File
@@ -1,42 +1,57 @@
# AIOS Task Plane
该模块是与任务发起方、Worker 类型和通知后端解耦的持久化任务控制平面。当前交付仅用于隔离开发与 Shadow 验收;默认 `execution_enabled=false`,不会修改生产、tag、Baseline 或 Registry。
## 架构
- `TaskPlaneService`:统一服务边界和 HTTP 路由。
- `TaskPlaneStore`:SQLite WAL、revision/CAS、串行事务、租约、hash-chained journal、evidence 和 durable outbox。
- `WorkerRegistry` / `LeaseManager`:身份绑定、capability、并发额度、心跳、过期探测后恢复。
- `DelegationEngine`:父子 DAG、全局深度、capability/risk/approval/budget 单调约束和幂等重放。
- `PolicyEngine`:默认关闭执行;身份、证据、结果摘要和可插拔签名验签均 fail-closed。
- `WorkerAdapter`:OpenClaw、Codex、Hermes、A2A 和 systemd Worker 的统一契约。默认实现只返回 `SHADOW_VERIFIED`,不执行外部命令。
- `DurableOutboxDispatcher`:Webhook、NATS JetStream、GPT callback、Dashboard 和 WeChat 的回执式通知接口。没有真实回执时始终保留 `NOT_DELIVERED`。
## 状态与恢复
正常路径为 `CREATED → QUEUED/CLAIMED → RUNNING → VERIFYING/COMPLETED`;失败路径使用 `FAILED_RECOVERABLE`、`FAILED_FINAL`、`CANCELLED` 或 `MANUAL_REVIEW_REQUIRED`。每次状态事件都进入同一事务中的 append-only hash chain。
租约过期不会自动重放副作用。控制平面必须先调用 Worker probe,只有明确确认原执行已停止,才可将 claim 标记为 `EXPIRED` 并把任务转为 `FAILED_RECOVERABLE`。结果必须匹配 task、claim、Worker identity、未过期 lease 和 operation digest;重复结果返回冲突。
## 安全模型
- task/worker/claim/message ID 使用 allowlist pattern,拒绝 URL 编码差异、路径分隔符和控制字符。
- HTTP body 最大 1 MB,非对象 JSON、异常 Content-Length 和非法参数返回 400/413。
- 内部异常响应不回显路径、payload 或凭据;默认 HTTP access log 不记录原始 request target。
- secret-like 字段在 durable task、event、evidence 和 outbox 写入前脱敏。
- 结果签名存在但没有配置验签器时拒绝;摘要、evidence 或 event 被篡改时验收失败。
- SQLite 文件创建为 0600;新建状态目录使用 0700。
## 验收
```bash
PYTHONPATH=active-runtime \
bash scripts/task-plane/run_task_plane_acceptance.sh
PYTHONPATH=active-runtime \
python3 scripts/task-plane/create_shadow_tasks.py \
--database reports/task-plane-shadow.sqlite3 \
--report reports/task-plane-shadow.json
```
上述脚本只使用本地 SQLite 和 Shadow adapter,不触发真实 OpenClaw、Codex、Hermes、微信或生产执行。
# AIOS Task Plane
该模块是与任务发起方、Worker 类型和通知后端解耦的持久化任务控制平面。当前交付仅用于隔离开发与 Shadow 验收;默认 `execution_enabled=false`,不会修改生产、tag、Baseline 或 Registry。
## 架构
- `TaskPlaneService`:统一服务边界和 HTTP 路由。
- `TaskPlaneStore`:SQLite WAL、revision/CAS、串行事务、租约、hash-chained journal、evidence 和 durable outbox。
- `WorkerRegistry` / `LeaseManager`:身份绑定、capability、并发额度、心跳、过期探测后恢复。
- `DelegationEngine`:父子 DAG、全局深度、capability/risk/approval/budget 单调约束和幂等重放。
- `PolicyEngine`:默认关闭执行;身份、证据、结果摘要和可插拔签名验签均 fail-closed。
- `WorkerAdapter`:OpenClaw、Codex、Hermes、A2A 和 systemd Worker 的统一契约。默认实现只返回 `SHADOW_VERIFIED`,不执行外部命令。
- `DurableOutboxDispatcher`:Webhook、NATS JetStream、GPT callback、Dashboard 和 WeChat 的回执式通知接口。没有真实回执时始终保留 `NOT_DELIVERED`。
## 状态与恢复
正常路径为 `CREATED → QUEUED/CLAIMED → RUNNING → VERIFYING/COMPLETED`;失败路径使用 `FAILED_RECOVERABLE`、`FAILED_FINAL`、`CANCELLED` 或 `MANUAL_REVIEW_REQUIRED`。每次状态事件都进入同一事务中的 append-only hash chain。
租约过期不会自动重放副作用。控制平面必须先调用 Worker probe,只有明确确认原执行已停止,才可将 claim 标记为 `EXPIRED` 并把任务转为 `FAILED_RECOVERABLE`。结果必须匹配 task、claim、Worker identity、未过期 lease 和 operation digest;重复结果返回冲突。
## 安全模型
- task/worker/claim/message ID 使用 allowlist pattern,拒绝 URL 编码差异、路径分隔符和控制字符。
- HTTP body 最大 1 MB,非对象 JSON、异常 Content-Length 和非法参数返回 400/413。
- 内部异常响应不回显路径、payload 或凭据;默认 HTTP access log 不记录原始 request target。
- secret-like 字段在 durable task、event、evidence 和 outbox 写入前脱敏。
- 结果签名存在但没有配置验签器时拒绝;摘要、evidence 或 event 被篡改时验收失败。
- SQLite 文件创建为 0600;新建状态目录使用 0700。
## 租户语义(cross-tenant)
任务与 Worker 各带租户标识:任务取自 `task_plan.metadata.tenant_id`(默认 `default`),
Worker 取自 `trust_zone`(默认 `default`)。跨租户(cross-tenant)处理语义为:
- **枚举 = skip + 审计**:`poll` / `long_poll` / `schedule`(`queued_count`)在遍历候选任务时,
对与 Worker 租户不一致的任务执行 SKIP,写入结构化审计 finding
`{source:"tenant", kind:"cross_tenant_skip", decision:"skip+audit", task_tenant, worker_tenant}`
(持久化于 `tenant_skip_audit` 表并在任务事件链追加 `TASK_TENANT_SKIPPED`),**不抛异常、不 fail-closed**。
仅当存在同租户候选时才会被选中。
- **直接读/认领 = fail-closed**:按 `task_id` 直接 `governed_claim` / `claim_governed` 跨租户任务时
保持拒绝,返回 `403 cross_tenant_denied`,不泄露任务内容。
即:Worker 只能“看到”并认领同租户任务;枚举路径让跨租户任务不可见且留有审计,直接认领路径则明确失败。
## 验收
```bash
PYTHONPATH=active-runtime \
bash scripts/task-plane/run_task_plane_acceptance.sh
PYTHONPATH=active-runtime \
python3 scripts/task-plane/create_shadow_tasks.py \
--database reports/task-plane-shadow.sqlite3 \
--report reports/task-plane-shadow.json
```
上述脚本只使用本地 SQLite 和 Shadow adapter,不触发真实 OpenClaw、Codex、Hermes、微信或生产执行。
@@ -0,0 +1,167 @@
#!/usr/bin/env python3
from __future__ import annotations
from pathlib import Path
import argparse, json, sys, time, re
from aios_control_plane_consistency import *
def jload(value):
try:
return json.loads(value or "{}")
except Exception:
return {}
def rows(path, query, params=()):
try:
return sqlite_rows(path, query, params)
except Exception as exc:
return [{"_error": str(exc)}]
def main() -> int:
ap = argparse.ArgumentParser(description="AIOS read-only control-plane consistency auditor")
ap.add_argument("--task-id", required=True)
ap.add_argument("--admission-db", type=Path, default=Path("/var/lib/aios-gitops/bridge-v2-staging/task-admission.sqlite3"))
ap.add_argument("--task-db", type=Path, default=Path("/var/lib/aios-agent-task/state.sqlite3"))
ap.add_argument("--git-dir", type=Path, default=Path("/data/aios-gitops/repo.git"))
ap.add_argument("--branch")
ap.add_argument("--openapi", type=Path)
ap.add_argument("--server-source", type=Path)
ap.add_argument("--artifact-root", type=Path)
ap.add_argument("--now", type=int)
ap.add_argument("--out", type=Path, default=Path("aios-control-plane-audit.json"))
ns = ap.parse_args()
now = ns.now or int(time.time())
findings = []
admission = None
task = None
workers = []
lease = None
git = None
evidence = EvidenceSnapshot()
# Admission plane.
try:
receipt = sqlite_rows(ns.admission_db,
"SELECT * FROM task_admission_receipts WHERE task_id=? ORDER BY rowid DESC LIMIT 1", (ns.task_id,))
if receipt:
r = dict(receipt[0])
envelope_id = r.get("envelope_id")
env = sqlite_rows(ns.admission_db,
"SELECT * FROM task_admission_envelopes WHERE envelope_id=? LIMIT 1", (envelope_id,))
grant = sqlite_rows(ns.admission_db,
"SELECT * FROM task_admission_grants WHERE envelope_id=? ORDER BY rowid DESC LIMIT 1", (envelope_id,))
ed = dict(env[0]) if env else {}
gd = dict(grant[0]) if grant else {}
gj = jload(gd.get("grant_json"))
ej = jload(ed.get("envelope_json"))
admission = AdmissionSnapshot(
task_id=ns.task_id,
envelope_id=envelope_id,
admission_status=ed.get("status"),
grant_id=gd.get("grant_id"),
grant_expires_at=gj.get("expires_at"),
one_time_use=gj.get("one_time_use"),
authority_base_oid=r.get("authority_base_oid") or ej.get("authority_base_oid"),
execution_digest=r.get("execution_digest") or ej.get("execution_digest"),
requested_capabilities=tuple(ej.get("requested_capabilities") or ()),
file_allowlist=tuple(ej.get("file_allowlist") or ()),
admitted_at=r.get("created_at") or r.get("admitted_at"),
)
except Exception as exc:
findings.append(Finding("ADMISSION_PLANE_UNREADABLE", Severity.BLOCKER, str(exc)))
# TaskPlane.
try:
trows = sqlite_rows(ns.task_db, "SELECT * FROM tasks WHERE task_id=? LIMIT 1", (ns.task_id,))
if trows:
td = dict(trows[0])
meta = jload(td.get("metadata_json"))
required = jload(td.get("required_capabilities_json"))
if isinstance(required, dict):
required = required.get("capabilities") or []
task = TaskSnapshot(
task_id=ns.task_id,
status=td.get("status"),
claimed_by=td.get("claimed_by"),
lease_expires_at=td.get("lease_expires_at"),
attempt=td.get("attempt"),
revision=td.get("revision"),
created_at=td.get("created_at"),
updated_at=td.get("updated_at"),
deadline=td.get("deadline"),
required_capabilities=tuple(required or ()),
metadata=meta if isinstance(meta, dict) else {},
)
arows = sqlite_rows(ns.task_db, "SELECT * FROM agents")
for row in arows:
d = dict(row)
caps = jload(d.get("capabilities_json"))
if isinstance(caps, dict):
caps = caps.get("capabilities") or []
last = parse_iso_epoch(d.get("last_seen_at"))
workers.append(WorkerSnapshot(
worker_id=d.get("agent_id") or d.get("worker_id") or "unknown",
status=d.get("status"),
last_seen_epoch=last,
capabilities=tuple(caps or ()),
))
except Exception as exc:
findings.append(Finding("TASK_PLANE_UNREADABLE", Severity.BLOCKER, str(exc)))
branch = ns.branch
if not branch and task and isinstance(task.metadata, dict):
plan = task.metadata.get("execution_plan") or task.metadata.get("plan") or task.metadata
if isinstance(plan, dict):
branch = plan.get("target_branch")
if branch:
current = git_ref(ns.git_dir, f"refs/heads/{branch}")
git = GitSnapshot(branch=branch, current_oid=current, expected_oid=admission.authority_base_oid if admission else None)
if admission and current and admission.authority_base_oid and current != admission.authority_base_oid:
findings.append(Finding(
"BRANCH_BINDING_DRIFT",
Severity.BLOCKER,
"Current branch OID differs from Admission authority_base_oid.",
{"current": current, "bound": admission.authority_base_oid},
"Replan/rebind formally; do not reuse old execution_digest.",
))
truth, derived = derive_truth(admission, task, workers, lease, git, evidence, now)
findings.extend(derived)
if task:
findings.extend(check_capability_aliases(task.required_capabilities, workers))
# Plan/artifact integrity if the immutable plan points to files.
if task and ns.artifact_root:
plan = task.metadata.get("execution_plan") or task.metadata.get("plan") or task.metadata
if isinstance(plan, dict):
hashes = plan.get("file_sha256") or {}
for rel in plan.get("file_allowlist") or []:
findings.extend(inspect_artifact(ns.artifact_root / rel, hashes.get(rel)))
if ns.openapi and ns.server_source:
findings.extend(check_openapi_backend_parity(
ns.openapi, ns.server_source,
required_ops=("getTaskAdmissionEnvelope","submitTaskAdmissionAuthorization","getTaskAdmissionStatus"),
))
report = {
"schema": "aios.control-plane-consistency-audit.v1",
"observed_at": now,
"task_id": ns.task_id,
"truth_state": truth.value,
"admission": asdict(admission) if admission else None,
"task": asdict(task) if task else None,
"eligible_workers": [asdict(w) for w in eligible_workers(task, workers, now)] if task else [],
"git": asdict(git) if git else None,
"findings": [asdict(f) for f in findings],
"summary": {
"blockers": sum(f.severity == Severity.BLOCKER for f in findings),
"majors": sum(f.severity == Severity.MAJOR for f in findings),
"minors": sum(f.severity == Severity.MINOR for f in findings),
},
}
dump_report(ns.out, report)
print(json.dumps(report["summary"], sort_keys=True))
return 2 if report["summary"]["blockers"] else 1 if report["summary"]["majors"] else 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,20 @@
#!/usr/bin/env python3
from aios_control_plane_consistency import *
NOW=2_000_000_000
scenarios = {}
a=AdmissionSnapshot(task_id="T",admission_status="ADMITTED",grant_expires_at=NOW+60)
scenarios["dual_plane_missing_task"]=derive_truth(a,None,[],None,None,None,NOW)
t=TaskSnapshot(task_id="T",status="QUEUED",required_capabilities=("writer.lease",),created_at="2033-05-18T03:23:20+00:00")
scenarios["queued_no_worker"]=derive_truth(a,t,[],None,None,None,NOW)
t2=TaskSnapshot(task_id="T",status="RUNNING",claimed_by="ghost")
scenarios["fake_running"]=derive_truth(a,t2,[],None,None,None,NOW)
t3=TaskSnapshot(task_id="T",status="SUCCEEDED")
scenarios["fake_success"]=derive_truth(a,t3,[],None,GitSnapshot("agent/x",expected_oid="a"*40),EvidenceSnapshot(),NOW)
for name,(state,findings) in scenarios.items():
print(name,state.value,[f.code for f in findings])
+522
View File
@@ -0,0 +1,522 @@
"""ANNEX-2.3 R2 governed auto-claim full-chain and A23 positive/negative tests.
Covers the acceptance line:
SchedulabilityProof -> authoritative_claim_recheck -> TaskPlane claim
-> Agent automatic pull -> receipt/readback
and the A23 negative checks with their error codes. Uses the governed
authorization profile (execution_enabled=True + {"*": ALL_ACTION_CAPABILITIES})
per ARCH-AUTO-CLAIM-R2-20260914 §8.
"""
from __future__ import annotations
import json
import threading
import time
import urllib.error
import urllib.request
from http.server import ThreadingHTTPServer
from unittest import mock
import pytest
from agent_pull_runner import AgentPullRunner, RunnerConfig
from task_plane.api import TaskPlaneRequestHandler, TaskPlaneService
from task_plane.errors import ConflictError, ForbiddenError, LockedError
from task_plane.http_transport import HttpRunnerTransport
from task_plane.policy import ALL_ACTION_CAPABILITIES, PolicyEngine
from task_plane.receipts import receipt_digest
from task_plane.recheck import Check, RecheckDecision, RecheckEngine
from task_plane.schedulability import ProofBuilder, compute_proof_digest
from task_plane.schedule_service import SchedulingService
from task_plane.store import TaskPlaneStore
IDENTITY = "spiffe://aios/agent/r2"
WORKER = "r2-worker"
REG = {
"worker_id": WORKER,
"worker_type": "test",
"identity": IDENTITY,
"capabilities": ["code.read"],
"trust_zone": "tenant-A",
"max_concurrency": 1,
"heartbeat_interval_seconds": 15,
"supported_protocols": ["aios-task-v1"],
}
def make_service() -> tuple[TaskPlaneStore, TaskPlaneService]:
store = TaskPlaneStore(":memory:")
policy = PolicyEngine(
execution_enabled=True,
identity_capabilities={"*": set(ALL_ACTION_CAPABILITIES)},
)
service = TaskPlaneService(store, policy=policy)
return store, service
def register_worker(service, **overrides) -> None:
payload = dict(REG)
payload.update(overrides)
service.register_worker(payload)
def _assert_proof_ok(service, identity, proof_id):
proof = service.get_proof(proof_id, identity)
assert proof["status"] == "CLAIMED"
payload = {k: v for k, v in proof.items() if k != "digest"}
assert compute_proof_digest(payload) == proof["digest"]
return proof
def _assert_receipt_ok(service, identity, receipt_id, kind=None):
receipt = service.get_receipt(receipt_id, identity)
assert receipt_digest(receipt["payload"]) == receipt["digest"]
if kind:
assert receipt["kind"] == kind
return receipt
# -- REQ: full chain ---------------------------------------------------------
def test_r2_full_chain_poll_claim_and_readback():
store, service = make_service()
register_worker(service)
task = service.create_task(
{
"task_id": "task-r2-a",
"required_capabilities": ["code.read"],
"metadata": {"tenant_id": "tenant-A"},
}
)
result = service.poll(WORKER, IDENTITY)
assert result["task"]["task_id"] == "task-r2-a"
claim_id = result["claim_id"]
proof_id = result["proof_id"]
receipt_id = result["receipt_id"]
claim = store.get_claim(claim_id)
assert claim.status == "ACTIVE"
assert result["generation"]["task_revision"] == claim.task_revision
assert result["generation"]["attempt"] == claim.attempt
proof = _assert_proof_ok(service, IDENTITY, proof_id)
assert proof["claim_id"] == claim_id
assert proof["binding"]["worker_identity"] == IDENTITY
_assert_receipt_ok(service, IDENTITY, receipt_id, kind="claim")
receipts = service.list_task_receipts("task-r2-a", IDENTITY)
kinds = {r["kind"] for r in receipts}
assert kinds == {"claim", "proof"}
event_types = {e["type"] for e in store.list_events("task-r2-a")}
assert "CLAIM_RECHECK_ACCEPTED" in event_types
assert "CLAIM_GOVERNED_ACCEPTED" in event_types
def test_r2_governed_claim_atomic_single_transaction_fault_free():
store, service = make_service()
register_worker(service)
task = service.create_task(
{"task_id": "task-r2-atomic", "metadata": {"tenant_id": "tenant-A"}}
)
with mock.patch("task_plane.store._now", return_value=time.time() + 0.01):
result = service.poll(WORKER, IDENTITY)
assert result["claim_id"]
# Proving atomicity: proof/receipt/event all present after one claim.
proof = service.get_proof(result["proof_id"], IDENTITY)
assert proof["status"] == "CLAIMED"
receipts = service.list_task_receipts("task-r2-atomic", IDENTITY)
assert len(receipts) == 2
assert store.verify_event_chain("task-r2-atomic")
# -- A23 negative cases ------------------------------------------------------
def test_a23_enumeration_skips_cross_tenant_with_audit():
# G4: enumeration of a cross-tenant task must SKIP + record a structured
# audit finding (NOT fail closed). A tenant-B worker polling with only a
# tenant-A task sees idle/no_task and records a skip+audit finding.
store, service = make_service()
register_worker(service, trust_zone="tenant-B") # worker in tenant-B
service.create_task(
{"task_id": "task-r2-x", "metadata": {"tenant_id": "tenant-A"}}
)
result = service.poll(WORKER, IDENTITY)
assert result["status"] == "idle"
assert result["reason"] == "no_task"
audit = store.list_tenant_skip_audit("task-r2-x")
assert any(
a["worker_id"] == WORKER
and a["finding"]["source"] == "tenant"
and a["finding"]["kind"] == "cross_tenant_skip"
and a["finding"]["decision"] == "skip+audit"
and a["finding"]["task_tenant"] == "tenant-A"
and a["finding"]["worker_tenant"] == "tenant-B"
for a in audit
)
def test_a23_enumeration_prefers_same_tenant_over_cross_tenant():
# G4: when a same-tenant task exists, enumeration picks it and skips the
# cross-tenant task (with audit), never selecting cross-tenant work.
store, service = make_service()
register_worker(service) # tenant-A worker (default trust_zone from REG is tenant-A)
service.create_task(
{"task_id": "task-r2-xcross", "metadata": {"tenant_id": "tenant-B"}}
)
service.create_task(
{"task_id": "task-r2-own", "metadata": {"tenant_id": "tenant-A"}}
)
result = service.poll(WORKER, IDENTITY)
assert result["task"]["task_id"] == "task-r2-own"
cross_audit = store.list_tenant_skip_audit("task-r2-xcross")
assert any(a["finding"]["decision"] == "skip+audit" for a in cross_audit)
def test_a23_direct_cross_tenant_claim_fail_closed():
# G4: a DIRECT claim (by task_id) of a cross-tenant task remains FAIL-CLOSED
# (403 cross_tenant_denied) -- in contrast to enumeration which skips+audits.
_, service = make_service()
register_worker(service, trust_zone="tenant-B") # worker in tenant-B
task = service.create_task(
{"task_id": "task-r2-x", "metadata": {"tenant_id": "tenant-A"}}
)
with pytest.raises(ForbiddenError) as exc:
service.governed_claim(task["task_id"], {"worker_id": WORKER}, IDENTITY)
assert exc.value.error_code == "cross_tenant_denied"
assert exc.value.status_code == 403
def test_a23_capacity_exhausted():
store, service = make_service()
register_worker(service) # max_concurrency=1
task_a = service.create_task(
{"task_id": "task-r2-cap-a", "metadata": {"tenant_id": "tenant-A"}}
)
# Worker holds one active (non-expired) claim already.
service.claim(
task_a["task_id"],
{"worker_id": WORKER, "actor_identity": IDENTITY, "lease_seconds": 60},
)
service.create_task(
{"task_id": "task-r2-cap-b", "metadata": {"tenant_id": "tenant-A"}}
)
with pytest.raises(LockedError) as exc:
service.poll(WORKER, IDENTITY)
assert exc.value.error_code == "capacity_exhausted"
assert exc.value.status_code == 423
assert store.get_active_claim_count(WORKER) == 1
def test_a23_generation_stale():
store, service = make_service()
register_worker(service)
task = service.create_task(
{"task_id": "task-r2-gen", "metadata": {"tenant_id": "tenant-A"}}
)
worker = store.get_worker(WORKER)
fresh_task = store.get_task(task["task_id"])
proof = ProofBuilder(store).build(fresh_task, worker)
# Task revision advances after the proof was built -> stale generation.
updated = store.update_task_status(task["task_id"], fresh_task.revision, "QUEUED")
assert updated.revision == fresh_task.revision + 1
engine = RecheckEngine(store)
decision = engine.authoritative_claim_recheck(
proof=proof, task=store.get_task(task["task_id"]), worker=worker
)
assert decision.allowed is False
matching = [c for c in decision.checks if c.check == "generation_ok"]
assert matching and matching[0].ok is False
assert decision.error_code == "generation_stale"
def test_a23_lease_valid_mapping_rejects_overlap():
# lease_valid (already actively claimed / closed lease) maps to
# lease_expired_or_closed -- verified deterministically via to_error.
_, service = make_service()
decision = _make_decision("lease_valid")
error = RecheckEngine(service.store).to_error(decision)
assert error.error_code == "lease_expired_or_closed"
assert error.status_code == 423
def test_a23_reclaim_flight_rejected():
_, service = make_service()
register_worker(service)
task = service.create_task(
{"task_id": "task-r2-lease", "metadata": {"tenant_id": "tenant-A"}}
)
service.claim(
task["task_id"],
{"worker_id": WORKER, "actor_identity": IDENTITY, "lease_seconds": 60},
)
# Re-claiming an in-flight (CLAIMED) task is rejected by the governed path.
with pytest.raises(ConflictError) as exc:
service.scheduling.claim_governed(WORKER, task["task_id"])
assert exc.value.error_code == "task_binding_mismatch"
# -- A23 error mapping (deterministic) ---------------------------------------
def _make_decision(failed_check: str) -> RecheckDecision:
all_checks = [
"tenant_match",
"principal_match",
"authorization_ok",
"task_match",
"generation_ok",
"lease_valid",
"capacity_ok",
"worker_online",
"capabilities_match",
]
checks = [Check(c, c != failed_check, "detail") for c in all_checks]
return RecheckDecision(
recheck_id="rc-x",
proof_id="proof-x",
task_id="task-x",
worker_id=WORKER,
claim_id=None,
allowed=False,
reason=next((c.detail for c in checks if not c.ok), None),
checks=checks,
observed_at=123.0,
)
@pytest.mark.parametrize(
"failed_check,error_code,status",
[
("tenant_match", "cross_tenant_denied", 403),
("principal_match", "task_binding_mismatch", 409),
("task_match", "task_binding_mismatch", 409),
("generation_ok", "generation_stale", 409),
("lease_valid", "lease_expired_or_closed", 423),
("capacity_ok", "capacity_exhausted", 423),
],
)
def test_a23_error_mapping(failed_check, error_code, status):
_, service = make_service()
decision = _make_decision(failed_check)
error = RecheckEngine(service.store).to_error(decision)
assert error.error_code == error_code
assert error.status_code == status
# -- long_poll / poll / schedule semantics -----------------------------------
def test_long_poll_times_out_to_idle():
store, service = make_service()
register_worker(service)
scheduling = SchedulingService(
store, long_poll_seconds=15, long_poll_max=30, poll_interval_seconds=0.05
)
result = scheduling.long_poll(WORKER, timeout_seconds=1)
assert result == {"status": "idle", "reason": "timeout"}
def test_long_poll_delivers_when_task_appears():
store, service = make_service()
register_worker(service)
scheduling = SchedulingService(
store, poll_interval_seconds=0.05
)
# Seed a task after the poll loop has started, out of a timer thread.
service.create_task(
{"task_id": "task-r2-lp", "metadata": {"tenant_id": "tenant-A"}}
)
result = scheduling.long_poll(WORKER, timeout_seconds=3)
assert result["task"]["task_id"] == "task-r2-lp"
assert result["claim_id"]
def test_schedule_endpoint_report():
_, service = make_service()
register_worker(service)
report = service.schedule(WORKER, IDENTITY)
assert report["online"] is True
assert report["capacity_ok"] is True
assert report["queued_count"] == 0
def test_poll_idle_when_no_task():
_, service = make_service()
register_worker(service)
result = service.poll(WORKER, IDENTITY)
assert result["status"] == "idle"
# -- receipt readback integrity ----------------------------------------------
def test_receipt_tamper_detected():
_, service = make_service()
register_worker(service)
service.create_task({"task_id": "task-r2-tamper", "metadata": {"tenant_id": "tenant-A"}})
result = service.poll(WORKER, IDENTITY)
receipt = service.get_receipt(result["receipt_id"], IDENTITY)
# Mutating payload breaks the digest.
mutated = dict(receipt)
mutated["payload"] = dict(receipt["payload"])
mutated["payload"]["attempt"] = 999
assert receipt_digest(mutated["payload"]) != mutated["digest"]
# -- HTTP handler routes -----------------------------------------------------
@pytest.fixture
def http_env():
store, service = make_service()
handler = type("H", (TaskPlaneRequestHandler,), {})
handler.service = service
def authenticator(token):
return IDENTITY if token == "secret-token" else None
handler.authenticator = authenticator
server = ThreadingHTTPServer(("127.0.0.1", 0), handler)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
base = f"http://127.0.0.1:{server.server_address[1]}"
yield service, base
server.shutdown()
server.server_close()
def _http(method, url, token, body=None):
request = urllib.request.Request(url, method=method)
if token:
request.add_header("Authorization", "Bearer " + token)
data = None
if body is not None:
data = json.dumps(body).encode("utf-8")
request.add_header("Content-Type", "application/json")
request.add_header("Content-Length", str(len(data)))
try:
with urllib.request.urlopen(request, data=data, timeout=10) as response:
return response.status, json.loads(response.read().decode("utf-8"))
except urllib.error.HTTPError as exc:
try:
payload = json.loads(exc.read().decode("utf-8"))
except Exception:
payload = {}
return exc.code, payload
def test_http_claim_governed_and_readback(http_env):
service, base = http_env
service.register_worker(REG)
service.create_task({"task_id": "task-r2-http", "metadata": {"tenant_id": "tenant-A"}})
status, result = _http(
"POST", f"{base}/tasks/task-r2-http/claim-governed", "secret-token",
{"worker_id": WORKER, "lease_seconds": 30},
)
assert status == 200
assert result["task"]["task_id"] == "task-r2-http"
status, proof = _http("GET", f"{base}/proofs/{result['proof_id']}", "secret-token")
assert status == 200
assert proof["status"] == "CLAIMED"
status, receipt = _http("GET", f"{base}/receipts/{result['receipt_id']}", "secret-token")
assert status == 200
assert receipt_digest(receipt["payload"]) == receipt["digest"]
status, receipts = _http("GET", f"{base}/tasks/task-r2-http/receipts", "secret-token")
assert status == 200
assert len(receipts) == 2
def test_http_poll_and_no_auth_rejected(http_env):
service, base = http_env
service.register_worker(REG)
service.create_task({"task_id": "task-r2-pollhttp", "metadata": {"tenant_id": "tenant-A"}})
status, result = _http("GET", f"{base}/workers/{WORKER}/poll", "secret-token")
assert status == 200
assert result["claim_id"]
assert result["proof_id"]
# No bearer -> 401 across governed endpoints.
status, body = _http("GET", f"{base}/workers/{WORKER}/poll", None)
assert status == 401
def test_http_transport_register_poll_chain(http_env):
service, base = http_env
service.register_worker(REG)
service.create_task({"task_id": "task-r2-tr", "metadata": {"tenant_id": "tenant-A"}})
transport = HttpRunnerTransport(
base, token="secret-token", actor_identity=IDENTITY
)
reg = transport.register_worker("r2-worker-b", ["code.read"])
assert reg["worker_id"] == "r2-worker-b"
# The primary registered worker can long_poll and obtain the claim.
claimed = transport.long_poll(WORKER, 1)
assert claimed and claimed["claim_id"]
assert transport.claim(WORKER, claimed)["claim_id"] == claimed["claim_id"]
# End-to-end renew (REQ-03b) succeeds with the claim_id from the pull.
renewed = transport.renew(WORKER, "task-r2-tr", claim_id=claimed["claim_id"])
assert renewed["claim_id"] == claimed["claim_id"]
assert renewed["lease_until"]
# renew without an explicit claim_id self-resolves the worker's active claim.
renewed2 = transport.renew(WORKER, "task-r2-tr")
assert renewed2["claim_id"] == claimed["claim_id"]
assert renewed2["lease_until"] > renewed["lease_until"]
def test_agent_pull_runner_driven_renew_chain(http_env):
service, base = http_env
service.register_worker(REG)
service.create_task(
{"task_id": "task-r2-runner", "metadata": {"tenant_id": "tenant-A"}}
)
transport = HttpRunnerTransport(
base, token="secret-token", actor_identity=IDENTITY
)
runner = AgentPullRunner(
RunnerConfig(worker_id=WORKER, long_poll_seconds=1), transport
)
claimed = runner.discover_once()
assert claimed and claimed["claim_id"]
# renew pulls claim_id from the runner's inflight state (no 400).
renewed = runner.renew("task-r2-runner")
assert renewed["claim_id"] == claimed["claim_id"]
assert renewed["lease_until"]
def test_agent_pull_runner_discover_idle_returns_none_without_crash():
"""REQ-03c: an idle/no-task response must not crash discover_once."""
class IdleTransport:
def long_poll(self, worker_id, timeout_seconds):
return {"status": "idle", "reason": "timeout"}
def poll(self, worker_id):
return {"status": "idle", "reason": "no_task"}
def claim(self, worker_id, task):
return task
runner = AgentPullRunner(
RunnerConfig(worker_id=WORKER, long_poll_seconds=1), IdleTransport()
)
# A transport returning idle mappings (no task_id) yields None, no exception.
assert runner.discover_once() is None
assert runner.discover_once() is None
+193
View File
@@ -0,0 +1,193 @@
"""Tests for the read-only Runtime / Authority Catalog (authority_catalog)."""
from __future__ import annotations
import json
import os
import sys
ROOT = os.getcwd()
if os.path.join(ROOT, "active-runtime") not in sys.path:
sys.path.append(os.path.join(ROOT, "active-runtime"))
from app.task_plane.authority_catalog import AuthorityCatalog, CatalogResource, SCHEMA_VERSION
from app.task_plane.models import WorkerRegistration
from app.task_plane.store import TaskPlaneStore
WORKER_DICT = {
"worker_id": "worker-cat-1",
"worker_type": "codex.readonly",
"identity": "alice@example.test",
"capabilities": ["code.change.apply", "shell.exec"],
"trust_zone": "isolated-development",
"max_concurrency": 1,
"heartbeat_interval_seconds": 15,
"supported_protocols": ["aios-task-v1"],
}
def test_no_store_returns_complete_null_shape_without_exception():
catalog = AuthorityCatalog(store=None)
resource = catalog.resolve_worker("worker-whatever")
assert isinstance(resource, CatalogResource)
assert resource.schema_version == SCHEMA_VERSION
assert resource.resource_type == "worker"
assert resource.resource_id == "worker-whatever"
# Structurally complete: all required blocks present.
for key in ("authority", "runtime", "contracts", "conflicts", "evidence"):
assert key in resource.to_dict()
# Nothing fabricated: observation fields are null, evidence empty.
assert resource.authority["endpoint_or_path"] is None
assert resource.authority["revision"] is None
assert resource.runtime["service"] is None
assert resource.runtime["pid"] is None
assert resource.runtime["loaded_source_path"] is None
assert resource.runtime["source_commit"] is None
assert resource.runtime["artifact_sha256"] is None
assert resource.evidence == []
# It is recorded as unknown, not resolved.
summary = catalog.summary()
assert summary["unknown"].get("worker") == 1
assert "worker" not in summary["resolved"]
def test_resolve_worker_returns_worker_id_and_capabilities():
store = TaskPlaneStore()
reg = WorkerRegistration.from_dict(WORKER_DICT)
store.register_worker(reg)
catalog = AuthorityCatalog(store)
resource = catalog.resolve_worker("worker-cat-1")
assert resource.resource_type == "worker"
assert resource.resource_id == "worker-cat-1"
assert resource.authority["endpoint_or_path"] == "alice@example.test"
assert resource.authority["revision"] == 1
assert resource.runtime["service"] == "codex.readonly"
caps = [c["value"] for c in resource.contracts if c["field"] == "capabilities"][0]
assert "shell.exec" in caps
# Not flagged as runtime source unknown because authority resolved:
# but runtime source fields are still null for a registration (honest).
summary = catalog.summary()
assert summary["resolved"].get("worker") == 1
assert summary["runtime_source_unknown"] >= 1
def test_resolve_worker_missing_record_is_unknown_not_live():
store = TaskPlaneStore()
catalog = AuthorityCatalog(store)
resource = catalog.resolve_worker("does-not-exist")
assert resource.resource_id == "does-not-exist"
assert resource.authority["endpoint_or_path"] is None
assert resource.authority["revision"] is None
assert resource.evidence == []
assert catalog.summary()["unknown"].get("worker") == 1
def test_resolve_generic_unknown_type_is_empty_unknown():
catalog = AuthorityCatalog(store=None)
resource = catalog.resolve_generic("release", "rel-9")
assert resource.resource_type == "release"
assert resource.resource_id == "rel-9"
assert resource.authority["kind"] == "release"
assert resource.authority["endpoint_or_path"] is None
assert resource.authority["revision"] is None
assert resource.runtime["loaded_source_path"] is None
assert resource.evidence == []
assert catalog.summary()["unknown"].get("release") == 1
# No live state fabricated: nothing was provisioned as a runtime service.
assert resource.runtime["service"] is None
assert resource.runtime["state_root"] is None
def test_to_dict_is_json_serializable():
store = TaskPlaneStore()
store.register_worker(WorkerRegistration.from_dict(WORKER_DICT))
catalog = AuthorityCatalog(store)
resource = catalog.resolve_worker("worker-cat-1")
payload = resource.to_dict()
# Round-trips through JSON without error and preserves values.
text = json.dumps(payload, ensure_ascii=False)
restored = json.loads(text)
assert restored == payload
assert restored["schema_version"] == SCHEMA_VERSION
assert restored["resource_id"] == "worker-cat-1"
assert restored["authority"]["revision"] == 1
def test_resolve_is_read_only_no_store_write():
store = TaskPlaneStore()
store.register_worker(WorkerRegistration.from_dict(WORKER_DICT))
before_tasks = [t.to_dict() for t in store.list_tasks()]
before_events = {
t.task_id: store.list_events(t.task_id) for t in store.list_tasks()
}
catalog = AuthorityCatalog(store)
catalog.resolve_worker("worker-cat-1")
catalog.resolve_worker("missing-worker")
catalog.resolve_repository("repo/main", authority_ref="oid-abcd" if False else None)
catalog.resolve_generic("schema", "s1")
after_tasks = [t.to_dict() for t in store.list_tasks()]
after_events = {
t.task_id: store.list_events(t.task_id) for t in store.list_tasks()
}
assert before_tasks == after_tasks
assert before_events == after_events
def test_catalog_exposes_no_write_path():
write_primitives = {
"register_worker",
"create_task",
"claim_task",
"append_event",
"add_evidence",
"save_receipt",
"heartbeat_worker",
"disable_worker",
"restore_worker",
"update_task_status",
"record_progress",
"complete_claim",
}
catalog = AuthorityCatalog(store=None)
public = {name for name in vars(type(catalog)) if not name.startswith("_")}
# The catalog only exposes resolve + summary; it must not carry any
# store-write entry point of its own.
assert not (public & write_primitives)
assert public <= {"resolve_worker", "resolve_task", "resolve_repository", "resolve_generic", "summary"}
def test_repository_without_ref_is_unknown_with_ref_reflected():
catalog = AuthorityCatalog(store=None)
empty = catalog.resolve_repository("repo/main")
assert empty.authority["revision"] is None
# Original caller still gets the repository's identifier as an endpoint
# hint, but the OID (revision) is unknown until a live ref is supplied.
assert empty.authority["revision"] is None
catalog2 = AuthorityCatalog(store=None)
with_ref = catalog2.resolve_repository("repo/main", authority_ref="de44a3f01beef")
assert with_ref.authority["revision"] == "de44a3f01beef"
assert with_ref.authority["endpoint_or_path"] == "repo/main"
assert catalog2.summary()["resolved"].get("repository") == 1
def test_summary_tracks_runtime_source_unknown():
store = TaskPlaneStore()
store.register_worker(WorkerRegistration.from_dict(WORKER_DICT))
catalog = AuthorityCatalog(store)
catalog.resolve_worker("worker-cat-1") # resolved, but source fields null
catalog.resolve_worker("nope") # unknown
catalog.resolve_generic("release", "r1") # unknown
summary = catalog.summary()
assert summary["resolve_attempts"] == 3
assert summary["resolved"].get("worker") == 1
assert summary["unknown"].get("worker") == 1
assert summary["unknown"].get("release") == 1
# All three resolved/unknown entries lack loaded source fields.
assert summary["runtime_source_unknown"] == 3
assert summary["missing_fields"]["loaded_source_path"] == 3
assert summary["missing_fields"]["source_commit"] == 3
assert summary["missing_fields"]["artifact_sha256"] == 3
+104
View File
@@ -0,0 +1,104 @@
"""caller_scan 只读能力的测试。
覆盖:静态 AST 证据、测试文件不计 production、动态 import 不可解析、to_dict JSON。
"""
from __future__ import annotations
import json
import os
import sys
ROOT = os.getcwd()
for _p in ("active-runtime", "active-runtime/app"):
_q = os.path.join(ROOT, _p)
if _q not in sys.path:
sys.path.append(_q)
from app.task_plane.caller_scan import ( # noqa: E402
CallerKind,
OrphanAssessment,
scan_subject,
)
def _write(tmp_path, name: str, content: str):
(tmp_path / name).write_text(content, encoding="utf-8")
def test_static_import_from_is_production(tmp_path):
_write(tmp_path, "foo.py", "x = 1\n")
_write(tmp_path, "consumer.py", "from foo import x\n")
res = scan_subject("foo", repo_path=str(tmp_path))
assert any(c.kind is CallerKind.STATIC_PRODUCTION for c in res.callers)
assert res.caller_count == 1
assert res.assessment is OrphanAssessment.LIVE
assert res.coverage is True
def test_plain_import_is_production(tmp_path):
_write(tmp_path, "foo.py", "x = 1\n")
_write(tmp_path, "service.py", "import foo\n")
res = scan_subject("foo", repo_path=str(tmp_path))
kinds = {c.kind for c in res.callers}
assert CallerKind.STATIC_PRODUCTION in kinds
assert res.caller_count == 1
assert res.assessment is OrphanAssessment.LIVE
def test_test_file_reference_is_test_only_not_live(tmp_path):
# 只有测试文件引用 foo,没有任何 production 引用 → 不得判定 LIVE
_write(tmp_path, "foo.py", "x = 1\n")
_write(tmp_path, "test_consumer.py", "import foo\n")
res = scan_subject("foo", repo_path=str(tmp_path))
assert res.test_only_callers, "应有 test 仅引用"
assert all(c.kind is CallerKind.STATIC_TEST_ONLY for c in res.callers)
assert res.caller_count == 0
assert res.assessment is not OrphanAssessment.LIVE
def test_dynamic_import_string_not_proven_orphan(tmp_path):
# importlib 动态加载 foo(字符串字面量)→ 不可判 PROVEN_ORPHAN,计入 unknown_dynamic_refs
_write(tmp_path, "foo.py", "x = 1\n")
_write(tmp_path, "loader.py", 'import importlib\nimportlib.import_module("foo")\n')
res = scan_subject("foo", repo_path=str(tmp_path))
assert res.unknown_dynamic_refs, "应有 unknown_dynamic_refs"
assert res.assessment is not OrphanAssessment.PROVEN_ORPHAN
assert "foo" in res.runtime_loaded
def test_dynamic_import_variable_default_unknown(tmp_path):
# 变量形式不可解析(opaque)→ 默认保守返回 UNKNOWN,不误降孤儿
_write(tmp_path, "foo.py", "x = 1\n")
_write(tmp_path, "loader.py", "import importlib\nn = 'foo'\nimportlib.import_module(n)\n")
res = scan_subject("foo", repo_path=str(tmp_path))
assert res.unknown_dynamic_refs, "应有 unknown_dynamic_refs"
assert res.assessment is OrphanAssessment.UNKNOWN
assert not res.runtime_loaded # 变量形式无法静态确认是 foo
assert res.assessment is not OrphanAssessment.PROVEN_ORPHAN
def test_dynamic_import_allow_unknown_lower_flag(tmp_path):
# 仅当显式 allow_unknown_lower=True 才允许降为 LIKELY_ORPHAN
_write(tmp_path, "foo.py", "x = 1\n")
_write(tmp_path, "loader.py", "import importlib\nn = 'foo'\nimportlib.import_module(n)\n")
res = scan_subject("foo", repo_path=str(tmp_path), allow_unknown_lower=True)
assert res.assessment is OrphanAssessment.LIKELY_ORPHAN
def test_unreferenced_complete_scan_proven_orphan(tmp_path):
_write(tmp_path, "foo.py", "x = 1\n")
_write(tmp_path, "other.py", "y = 2\n")
res = scan_subject("foo", repo_path=str(tmp_path))
assert res.coverage is True
assert res.assessment is OrphanAssessment.PROVEN_ORPHAN
def test_to_dict_is_json_serializable(tmp_path):
_write(tmp_path, "foo.py", "x = 1\n")
_write(tmp_path, "consumer.py", "from foo import x\n")
res = scan_subject("foo", repo_path=str(tmp_path))
payload = res.to_dict()
json.dumps(payload) # 不应抛异常
assert payload["subject"] == "foo"
assert "assessment" in payload
+150
View File
@@ -0,0 +1,150 @@
"""Tests for the Single Canonical Contract (P0-2 / WP3).
Covers: determinism, sha256 relationship, dataclass encoding, float
normalization, non-finite float rejection, validation, and schema lookup.
"""
from __future__ import annotations
import hashlib
import pytest
from task_plane.contract import (
ContractError,
SchemaName,
SCHEMA_NAMES,
encode,
known_schema,
sha256,
validate,
)
from task_plane.models import EvidenceEnvelope
# ---- 1. determinism -------------------------------------------------------
def test_encode_deterministic_same_object():
obj = {"task_id": "t1", "status": "COMPLETED", "revision": 3}
assert encode(obj) == encode(obj)
def test_encode_key_order_independent():
a = {"z": 1, "a": 2, "m": {"y": 1, "x": 2}}
b = {"m": {"x": 2, "y": 1}, "a": 2, "z": 1}
assert encode(a) == encode(b)
def test_encode_reproducible_across_calls():
obj = {"nested": {"list": [1, 2, 3], "flag": True}, "none": None}
assert encode(obj) == encode(obj)
assert encode(obj) == encode(dict(obj))
# ---- 2. sha256 / encode relationship --------------------------------------
def test_sha256_is_sha256_of_encode():
obj = {"task_id": "t1", "payload": {"k": "v"}}
expected = hashlib.sha256(encode(obj).encode("utf-8")).hexdigest()
assert sha256(obj) == expected
assert len(sha256(obj)) == 64
assert sha256(obj) == sha256(dict(obj)) # logical equality -> same digest
# ---- 3. dataclass ---------------------------------------------------------
def test_encode_dataclass_evidence_envelope():
envelope = EvidenceEnvelope.create(
task_id="task-1",
claim_id="claim-1",
kind="execution",
worker_identity="worker-a",
payload={"checks": ["readback"], "count": 1},
previous_evidence_hash="",
)
text = encode(envelope)
assert isinstance(text, str)
assert text == encode(envelope) # deterministic for a typed envelope too
assert "task-1" in text
# ---- 4. float normalization -----------------------------------------------
def test_float_normalization_1_0_equals_1_00():
assert encode({"x": 1.0}) == encode({"x": 1.00})
assert encode(1.0) == encode(1.00)
def test_float_vs_int_distinct():
# booleans/ints/floats must not silently collide
assert encode({"x": 1.0}) != encode({"x": 1})
assert encode({"x": True}) != encode({"x": 1})
# ---- 5. non-finite float rejection ----------------------------------------
def test_non_finite_float_rejected():
with pytest.raises(ContractError):
encode(float("inf"))
with pytest.raises(ContractError):
encode(float("-inf"))
with pytest.raises(ContractError):
encode(float("nan"))
with pytest.raises(ContractError):
encode({"payload": {"score": float("nan")}})
# ---- 6. validation --------------------------------------------------------
def test_validate_pass_for_encodable():
result = validate({"a": 1, "b": [1, 2, 3]})
assert result.valid is True
assert bool(result) is True
def test_validate_fail_for_obviously_bad_type():
# set is not JSON-serializable -> must fail, not silently pass
result = validate({1, 2, 3})
assert result.valid is False
assert result.reason
# arbitrary object with no dataclass/plain mapping
result2 = validate(object())
assert result2.valid is False
def test_validate_with_known_schema():
result = validate({"ok": 1}, schema_name=SchemaName.EVIDENCE_ENVELOPE)
assert result.valid is True
assert result.schema_name == "EVIDENCE_ENVELOPE"
assert result.version is not None
def test_validate_with_unknown_schema():
result = validate({"ok": 1}, schema_name="NOT_A_REAL_SCHEMA")
assert result.valid is False
# ---- 7. schema lookup -----------------------------------------------------
def test_known_schema_for_known_and_unknown():
# If the schema file structure does not match expectations, skip rather
# than fail — known_schema is best-effort.
try:
version = known_schema(SchemaName.EVIDENCE_ENVELOPE)
except Exception:
pytest.skip("system_truth_contract.json unreadable/unparsable")
# A known schema must yield some version string.
assert version is not None
assert isinstance(version, str)
# A schema name that exists in the file (any of the six) returns a version.
for name in SCHEMA_NAMES:
assert known_schema(name) == version
def test_known_schema_returns_none_for_unknown():
assert known_schema("not-a-schema") is None
def test_all_six_schema_names_declared():
expected = {
"EVIDENCE_ENVELOPE",
"RELEASE_PLAN",
"TASK_EXECUTION_ENVELOPE",
"AUTHORIZATION_ENVELOPE",
"RUNTIME_IDENTITY",
"RECOVERY_RECEIPT",
}
assert SCHEMA_NAMES == frozenset(expected)
@@ -0,0 +1,76 @@
from pathlib import Path
import time
from aios_control_plane_consistency import *
NOW=2_000_000_000
def task(status="QUEUED", caps=("governed.write.files","task.principal.issue","writer.lease"), **kw):
return TaskSnapshot(
task_id="T", status=status, required_capabilities=tuple(caps),
created_at="2033-05-18T03:23:20+00:00", metadata={
"external_job_id":"job-a",
"target_branch":"agent/x",
"execution_digest":"d"*64,
}, **kw
)
def admission(**kw):
base=dict(task_id="T", admission_status="ADMITTED", envelope_id="E", grant_expires_at=NOW+600)
base.update(kw)
return AdmissionSnapshot(**base)
def worker(worker_id="w", caps=("governed.write.files","task.principal.issue","writer.lease"), last_seen=NOW-10, **kw):
return WorkerSnapshot(worker_id=worker_id,status="ACTIVE",last_seen_epoch=last_seen,capabilities=tuple(caps),**kw)
def test_admitted_without_taskplane_is_not_execution():
state, fs=derive_truth(admission(),None,[],None,None,None,NOW)
assert state==TruthState.ADMITTED_NOT_PROJECTED
assert any(f.code=="DUAL_PLANE_ADMITTED_TASK_MISSING" for f in fs)
def test_queued_without_eligible_worker_is_waiting_capacity_not_real_execution():
state, fs=derive_truth(admission(),task(),[],None,None,None,NOW)
assert state==TruthState.WAITING_CAPACITY
assert any(f.code=="QUEUED_WITHOUT_ELIGIBLE_WORKER" for f in fs)
def test_scope_mismatched_worker_is_not_eligible():
w=worker(scope_task_id="OTHER")
assert eligible_workers(task(),[w],NOW)==[]
def test_stale_worker_is_not_eligible():
w=worker(last_seen=NOW-1000)
assert eligible_workers(task(),[w],NOW)==[]
def test_capability_singular_plural_is_not_silent_alias():
fs=check_capability_aliases(("governed.write.files",),[
worker(caps=("governed.write.file","task.principal.issue","writer.lease"))
])
assert any(f.code=="CAPABILITY_SINGULAR_PLURAL_ASYMMETRY" for f in fs)
def test_claimed_without_claim_holder_is_fake():
state, fs=derive_truth(admission(),task(status="CLAIMED"),[],None,None,None,NOW)
assert state==TruthState.CLAIMED
assert any(f.code=="FAKE_CLAIM_STATE" for f in fs)
def test_running_without_heartbeat_is_fake_execution():
t=task(status="RUNNING",claimed_by="w")
state, fs=derive_truth(admission(),t,[worker(last_seen=NOW-999)],LeaseSnapshot("T","w",NOW+60,"ACTIVE"),None,None,NOW)
assert any(f.code=="FAKE_RUNNING_NO_HEARTBEAT" for f in fs)
def test_succeeded_without_commit_receipt_is_unproven():
state, fs=derive_truth(admission(),task(status="SUCCEEDED"),[],None,GitSnapshot("agent/x",expected_oid="a"*40),EvidenceSnapshot(),NOW)
assert state==TruthState.TERMINAL_UNPROVEN
assert any(f.code=="TERMINAL_SUCCESS_UNPROVEN" for f in fs)
def test_expired_without_claim_is_schedulability_failure():
state, fs=derive_truth(admission(grant_expires_at=NOW-1),task(status="EXPIRED",attempt=0),[],None,None,None,NOW)
assert state==TruthState.EXPIRED
assert any(f.code=="EXPIRED_WITHOUT_ANY_CLAIM" for f in fs)
def test_placeholder_artifact_is_rejected(tmp_path):
p=tmp_path/"x.py"; p.write_text("TODO: real content\n",encoding="utf-8")
fs=inspect_artifact(p)
assert any(f.code=="PLACEHOLDER_ARTIFACT" for f in fs)
def test_hash_only_does_not_make_missing_artifact_real(tmp_path):
fs=inspect_artifact(tmp_path/"missing.py","0"*64)
assert any(f.code=="ARTIFACT_MISSING" for f in fs)
@@ -0,0 +1,89 @@
from __future__ import annotations
from aios_control_plane_consistency import (
AdmissionSnapshot,
EvidenceSnapshot,
GitSnapshot,
LeaseSnapshot,
TaskSnapshot,
WorkerSnapshot,
)
from aios_control_plane_snapshot import (
attach_control_plane_truth,
build_control_plane_snapshot,
)
NOW = 2_000_000_000
def _base():
return build_control_plane_snapshot(
target_version="V2R2C2",
runtime={"drift_status": "PASS"},
task={"task_id": "T", "reconciliation_state": "ok"},
)
def test_snapshot_base_has_no_derived_block_by_default():
assert "derived" not in _base()
def test_truth_attach_preserves_existing_keys():
out = attach_control_plane_truth(_base(), now=NOW)
assert set(_base().keys()) <= set(out.keys())
assert "derived" in out
def test_admitted_without_taskplane_is_admitted_not_projected():
out = attach_control_plane_truth(
_base(),
admission=AdmissionSnapshot(task_id="T", admission_status="ADMITTED", envelope_id="E"),
now=NOW,
)
truth = out["derived"]["system_truth"]
assert truth["truth_state"] == "ADMITTED_NOT_PROJECTED"
assert truth["blocker_count"] == 1
assert any(f["code"] == "DUAL_PLANE_ADMITTED_TASK_MISSING" for f in truth["findings"])
# A drift_status=PASS surface must NOT be reported as evidence-complete.
assert truth["evidence_flow_confirmed"] is False
def test_fake_success_is_terminal_unproven_even_under_drift_pass():
out = attach_control_plane_truth(
_base(),
admission=AdmissionSnapshot(task_id="T"),
task=TaskSnapshot(task_id="T", status="SUCCEEDED"),
git=GitSnapshot("agent/x", expected_oid="a" * 40),
evidence=EvidenceSnapshot(),
now=NOW,
)
truth = out["derived"]["system_truth"]
assert truth["truth_state"] == "TERMINAL_UNPROVEN"
assert truth["blocker_count"] == 1
assert truth["evidence_flow_confirmed"] is False
def test_complete_requires_full_evidence_chain():
worker = WorkerSnapshot(
worker_id="w", status="ACTIVE", last_seen_epoch=NOW - 10,
capabilities=("governed.write.files",),
)
task = TaskSnapshot(
task_id="T", status="SUCCEEDED", claimed_by="w",
required_capabilities=("governed.write.files",),
metadata={"external_job_id": "job-a", "target_branch": "agent/x",
"execution_digest": "d" * 64},
)
git = GitSnapshot("agent/x", expected_oid="a" * 40, commit_oid="b" * 40, post_oid="c" * 40)
evidence = EvidenceSnapshot(runtime_readback={"ok": True})
out = attach_control_plane_truth(
_base(),
admission=AdmissionSnapshot(task_id="T"),
task=task, workers=[worker],
lease=LeaseSnapshot("T", "w", NOW + 60, "ACTIVE"),
git=git, evidence=evidence, now=NOW,
)
truth = out["derived"]["system_truth"]
assert truth["truth_state"] == "COMPLETE"
assert truth["blocker_count"] == 0
assert truth["evidence_flow_confirmed"] is True
+72
View File
@@ -0,0 +1,72 @@
"""env_profile 只读环境画像的测试。"""
from __future__ import annotations
import json
import os
import sys
ROOT = os.getcwd()
for _p in ("active-runtime", "active-runtime/app"):
_q = os.path.join(ROOT, _p)
if _q not in sys.path:
sys.path.append(_q)
from app.task_plane.env_profile import ( # noqa: E402
SCHEMA_VERSION,
build_report,
check_requirements,
)
def test_build_report_returns_complete_structure_never_raises():
rep = build_report()
for field in (
"schema_version",
"host_id",
"os",
"shell",
"posix",
"tools",
"capabilities",
"credential_mechanisms",
"filesystem",
"missing_requirements",
):
assert hasattr(rep, field), f"missing field {field}"
def test_posix_consistent_with_os_name():
rep = build_report()
if os.name == "posix":
assert rep.posix is True
else:
assert rep.posix is False
def test_no_secret_collection_fields():
# 只读:报告里不应出现任何 secret 的明文值字段
rep = build_report()
blob = json.dumps(rep.to_dict())
for secret_marker in ("GIT_SSH_COMMAND=", "AGENT_SSH_KEY_FILE="):
assert secret_marker not in blob
# 凭证机制只记录机制名,不记录值
assert all(isinstance(m, str) for m in rep.credential_mechanisms)
def test_tools_schema_present():
rep = build_report()
for tool in ("git", "python", "python3", "node", "ssh", "docker"):
assert tool in rep.tools
def test_to_dict_json_serializable():
rep = build_report()
json.dumps(rep.to_dict())
def test_check_requirements_never_raises_and_lists_missing():
rep = check_requirements(["git", "python", "python3", "some_unknown_tool"])
assert isinstance(rep.missing_requirements, list)
# 本机大概率有 git;校验结构完整即可
assert "schema_version" in rep.to_dict()
@@ -0,0 +1,27 @@
from pathlib import Path
from aios_control_plane_consistency import check_openapi_backend_parity
def test_openapi_dead_endpoint_is_blocker(tmp_path):
oa=tmp_path/"o.yaml"
oa.write_text("""openapi: 3.1.0
info: {title: t, version: '1'}
paths:
/v2/task-admission/envelope:
post:
operationId: getTaskAdmissionEnvelope
""",encoding="utf-8")
srv=tmp_path/"s.py"
srv.write_text('ROUTES = {"/v2/other": "other"}\n',encoding="utf-8")
fs=check_openapi_backend_parity(oa,srv,("getTaskAdmissionEnvelope",))
assert any(f.code=="OPENAPI_DEAD_ENDPOINT" for f in fs)
def test_backend_unprojected_operation_is_reported(tmp_path):
oa=tmp_path/"o.yaml"
oa.write_text("""openapi: 3.1.0
info: {title: t, version: '1'}
paths: {}
""",encoding="utf-8")
srv=tmp_path/"s.py"
srv.write_text('ROUTES = {"/v2/task-admission/status": "getTaskAdmissionStatus"}\n',encoding="utf-8")
fs=check_openapi_backend_parity(oa,srv,("getTaskAdmissionStatus",))
assert any(f.code=="BACKEND_UNPROJECTED_OPERATION" for f in fs)
+87
View File
@@ -0,0 +1,87 @@
"""toil_governor 重试/辛劳治理骨架的测试。"""
from __future__ import annotations
import os
import sys
ROOT = os.getcwd()
for _p in ("active-runtime", "active-runtime/app"):
_q = os.path.join(ROOT, _p)
if _q not in sys.path:
sys.path.append(_q)
from app.task_plane.toil_governor import ToilGovernor # noqa: E402
def test_first_try_allowed():
g = ToilGovernor()
d = g.decide("op", blocker={"kind": "fs", "path": "/x"}, now=100.0)
assert d.allow is True
def test_same_blocker_second_call_suppressed():
g = ToilGovernor()
b = {"kind": "fs", "path": "/x"}
g.decide("op", blocker=b, now=100.0)
d = g.decide("op", blocker=b, now=100.0)
assert d.allow is False
assert d.next_eligible_at is not None and d.next_eligible_at > 100.0
assert d.suppressed_count >= 1
def test_blocker_change_resets_and_allows():
g = ToilGovernor()
g.decide("op", blocker={"a": 1}, now=100.0)
g.decide("op", blocker={"a": 1}, now=100.0) # 被抑制
d = g.decide("op", blocker={"a": 2}, now=200.0) # blocker 变化 = 新证据
assert d.allow is True
def test_record_success_resets_allowing_retry():
g = ToilGovernor()
b = {"kind": "net"}
g.decide("op", blocker=b, now=100.0)
g.decide("op", blocker=b, now=100.0) # 被抑制
g.record_success("op", now=300.0)
d = g.decide("op", blocker=b, now=300.0)
assert d.allow is True # 成功后同 blocker 重新放行
def test_unknown_blocker_requires_reconcile_not_direct_allow():
g = ToilGovernor()
d = g.decide("op", blocker="unknown", now=100.0)
assert d.allow is False
assert "reconcile" in d.reason
# None blocker 也按 UNKNOWN 处理
d2 = g.decide("op2", blocker=None, now=100.0)
assert d2.allow is False
def test_long_term_external_blocker_auto_backoff_increases():
g = ToilGovernor()
b = {"kind": "ext"}
g.decide("op", blocker=b, now=100.0)
d1 = g.decide("op", blocker=b, now=110.0)
d2 = g.decide("op", blocker=b, now=120.0)
assert d1.allow is False and d2.allow is False
# 连续次数越多下一次可重试时间越晚(自动降频)
assert d2.next_eligible_at is not None
assert d1.next_eligible_at is not None
assert d2.next_eligible_at > d1.next_eligible_at
def test_stats_reports_state():
g = ToilGovernor()
g.decide("op", blocker={"k": 1}, now=100.0)
g.decide("op", blocker={"k": 1}, now=100.0)
s = g.stats()
assert s["operations"] >= 1
assert s["suppressed_by_blocker"] >= 1
def test_operation_key_normalizes_long_fingerprint():
g = ToilGovernor()
long_fp = "x" * 200
k = g.operation_key(long_fp)
assert len(k) <= 64
+219
View File
@@ -0,0 +1,219 @@
"""Tests for the Gate / Evidence Engine (WP5 / P0-5).
Covers the verdict single-rule (PASS only on full evidence closure),
the echo/local-only anti-pattern, claim/evidence gate semantics, and
JSON serializability.
"""
import json
import pytest
from active_runtime.app.task_plane.verdict import (
EvidenceRequirement,
Verdict,
VerdictStatus,
evaluate_verdict,
)
from active_runtime.app.task_plane.gate import GateKind, decide, claim_gate, evidence_gate
from active_runtime.app.task_plane.recheck import Check, RecheckDecision
# --------------------------------------------------------------------------
# 1. evaluate_verdict: the single governing rule
# --------------------------------------------------------------------------
def test_all_required_present_is_pass():
reqs = [EvidenceRequirement("source_verified"), EvidenceRequirement("runtime_readback")]
v = evaluate_verdict(
scope="runtime",
subject="svc.prod",
required_evidence=reqs,
available_evidence=["source_verified", "runtime_readback"],
)
assert v.status == VerdictStatus.PASS
assert v.missing_evidence == ()
assert set(v.evidence_refs) == {"source_verified", "runtime_readback"}
def test_missing_one_is_not_verified():
reqs = [
EvidenceRequirement("source_verified"),
EvidenceRequirement("runtime_readback"),
]
v = evaluate_verdict(
scope="runtime",
subject="svc.prod",
required_evidence=reqs,
available_evidence=["source_verified"], # missing runtime_readback
)
assert v.status == VerdictStatus.NOT_VERIFIED
assert "runtime_readback" in v.missing_evidence
def test_blocked_input_yields_blocked():
reqs = [
EvidenceRequirement("source_verified"),
EvidenceRequirement("maintenance_window_open"),
]
# required present, but explicit blocked condition -> BLOCKED even w/o missing
v = evaluate_verdict(
scope="runtime",
subject="svc.prod",
required_evidence=reqs,
available_evidence=["source_verified", "maintenance_window_open"],
blocked=True,
)
assert v.status == VerdictStatus.BLOCKED
def test_invalid_scope_is_fail():
v = evaluate_verdict(
scope="not-a-scope",
subject="x",
required_evidence=[],
available_evidence=[],
)
assert v.status == VerdictStatus.FAIL
def test_evidence_requirement_present_flag_counts():
reqs = [
EvidenceRequirement("source_verified", present=True),
EvidenceRequirement("runtime_readback", present=False),
]
v = evaluate_verdict(scope="runtime", subject="svc", required_evidence=reqs)
assert v.status == VerdictStatus.NOT_VERIFIED
# --------------------------------------------------------------------------
# 2. echo / local-only must NEVER become an overall PASS
# --------------------------------------------------------------------------
def test_local_only_evidence_is_not_business_pass():
# echo/local-only: only "local tests passed" is available, but the business
# scope requires production-grade evidence closures.
v = evaluate_verdict(
scope="business",
subject="svc.prod",
required_evidence=[
EvidenceRequirement("source_verified"),
EvidenceRequirement("contract_verified"),
EvidenceRequirement("runtime_readback"),
EvidenceRequirement("service_active_confirmed"),
],
available_evidence=["local_tests_passed"],
)
assert v.status == VerdictStatus.NOT_VERIFIED
assert v.status != VerdictStatus.PASS
assert "runtime_readback" in v.missing_evidence
def test_service_active_alone_not_runtime_pass():
# service liveness != business/runtime acceptance
v = evaluate_verdict(
scope="runtime",
subject="svc.prod",
required_evidence=[EvidenceRequirement("runtime_readback")],
available_evidence=["service_active"],
)
assert v.status == VerdictStatus.NOT_VERIFIED
# --------------------------------------------------------------------------
# 3. claim_gate wrapping a RecheckDecision
# --------------------------------------------------------------------------
def _denied_recheck():
failed = Check("tenant_match", ok=False, detail="cross_tenant_denied")
ok_check = Check("capacity_ok", ok=True, detail="capacity available")
return RecheckDecision(
recheck_id="rc-1",
proof_id="proof-1",
task_id="task-1",
worker_id="worker-1",
claim_id=None,
allowed=False,
reason="cross_tenant_denied",
checks=[ok_check, failed],
observed_at=1.0,
error_code="cross_tenant_denied",
status_code=403,
)
def test_claim_gate_denied_recheck_is_not_ok():
decision = claim_gate(_denied_recheck())
assert decision.gate == GateKind.CLAIM
assert decision.ok is False
# recheck carries no REJECT; a denial is mapped to fail-closed BLOCKED
assert decision.verdict.status == VerdictStatus.BLOCKED
assert any(not c["ok"] for c in decision.checks)
def test_claim_gate_accepted_recheck_is_ok():
from active_runtime.app.task_plane.models import compute_sha256_hex
decision = claim_gate(
RecheckDecision(
recheck_id="rc-2",
proof_id="proof-2",
task_id="task-2",
worker_id="worker-2",
claim_id=None,
allowed=True,
reason=None,
checks=[Check("tenant_match", ok=True, detail="matched")],
observed_at=2.0,
)
)
assert decision.gate == GateKind.CLAIM
assert decision.ok is True
assert decision.verdict.status == VerdictStatus.PASS
# --------------------------------------------------------------------------
# 4. evidence_gate requires the full closure
# --------------------------------------------------------------------------
def test_evidence_gate_missing_readback_is_not_verified():
decision = evidence_gate(
["source_verified", "runtime_readback"],
["source_verified"], # missing runtime readback
scope="runtime",
subject="svc.prod",
)
assert decision.gate == GateKind.EVIDENCE
assert decision.ok is False
assert decision.verdict.status == VerdictStatus.NOT_VERIFIED
def test_evidence_gate_empty_closure_never_passes():
# with required evidence, "no evidence" must not become PASS
decision = evidence_gate(["deploy_proven"], [], scope="runtime", subject="svc")
assert decision.ok is False
assert decision.verdict.status == VerdictStatus.NOT_VERIFIED
# --------------------------------------------------------------------------
# 5. serializability
# --------------------------------------------------------------------------
def test_verdict_to_dict_is_json_serializable():
reqs = [
EvidenceRequirement("source_verified"),
EvidenceRequirement("runtime_readback"),
]
v = evaluate_verdict(
scope="runtime",
subject="svc.prod",
required_evidence=reqs,
available_evidence=["source_verified"],
)
dumped = json.dumps(v.to_dict())
assert "NOT_VERIFIED" in dumped
loaded = json.loads(dumped)
assert loaded["scope"] == "runtime"
assert loaded["status"] == "NOT_VERIFIED"
def test_gate_decision_to_dict_is_json_serializable():
gd = claim_gate(_denied_recheck())
dumped = json.dumps(gd.to_dict())
assert "CLAIM" in dumped
assert json.loads(dumped)["ok"] is False
+117
View File
@@ -0,0 +1,117 @@
from __future__ import annotations
import json
from datetime import datetime, timezone
from pathlib import Path
import sys
import pytest
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT/"active-runtime"/"app"))
import release_approval_reopen as reopen
CID="rc-v2r2c2-4642b74af62c"; SRC="4"*40; TREE="a"*40
ART="5"*64; MAN="8"*64; PLAN="f"*64; POL="2"*64; FP="9"*64
NOW=datetime(2026,9,12,3,10,tzinfo=timezone.utc)
class Sagas:
def __init__(self, root): self.root=root
def compare_and_set(self, candidate_id, *, expected_states, target, event, details, expected_revision):
p=self.root/"sagas"/f"{candidate_id}.json"; x=json.loads(p.read_text())
if x["state"] not in expected_states: raise RuntimeError("CAS")
if int(x.get("revision",0)) != expected_revision: raise RuntimeError("SagaConflict")
x["state"]=target; x["revision"]=int(x.get("revision",0))+1
x.setdefault("events",[]).append({"event":event,"details":details,"timestamp":"2026-09-12T03:10:00Z"})
p.write_text(json.dumps(x,sort_keys=True)); return x
class Service:
def __init__(self, root): self.sagas=Sagas(root)
def w(p,o):
p.parent.mkdir(parents=True,exist_ok=True); p.write_text(json.dumps(o,sort_keys=True)); p.chmod(0o640)
_MISSING = object()
def state(tmp, expired=True, side_marker=_MISSING, saga="APPROVED", acceptance="ACCEPTED", extra=False):
r=tmp/"silent-release"
for d in ("candidates","sagas","release-plans","acceptance","approvals"):
(r/d).mkdir(parents=True,exist_ok=True,mode=0o750)
candidate={"candidate_id":CID,"state":"READY_FOR_APPROVAL","approval_state":"PENDING",
"source_commit":SRC,"tree_oid":TREE,"artifact_sha256":ART,"manifest_sha256":MAN}
if side_marker is not _MISSING:
candidate["side_effects_executed"]=side_marker
w(r/"candidates"/f"{CID}.json", candidate)
ev=[{"event":"saga_created","details":{}},{"event":"trusted_candidate_ready_for_approval","details":{}},
{"event":"signed_owner_approval_accepted","details":{"approval_fingerprint":FP}}]
if extra: ev.append({"event":"deployment_started","details":{}})
w(r/"sagas"/f"{CID}.json",{"candidate_id":CID,"state":saga,"revision":2,"journal_integrity":True,"events":ev})
w(r/"release-plans"/f"{CID}.json",{"sha256":PLAN,"policy_sha256":POL})
w(r/"acceptance"/f"{CID}.json",{"status":acceptance,"release_plan_sha256":PLAN})
w(r/"approvals"/f"{CID}.json",{"candidate_id":CID,"fingerprint":FP,"source_commit":SRC,"tree_oid":TREE,
"artifact_sha256":ART,"manifest_sha256":MAN,"release_plan_sha256":PLAN,"policy_sha256":POL,
"expires_at":"2026-09-12T02:59:05Z" if expired else "2026-09-12T04:59:05Z"})
return r
def req(key="reopen:test:0001"):
return {"schema":reopen.REQUEST_SCHEMA,"candidate_id":CID,"expected_approval_fingerprint":FP,
"expected_source_commit":SRC,"expected_tree_oid":TREE,"expected_artifact_sha256":ART,
"expected_manifest_sha256":MAN,"expected_release_plan_sha256":PLAN,"expected_policy_sha256":POL,
"reason":"expired approval after pre-side-effect infrastructure recovery","idempotency_key":key}
@pytest.fixture(autouse=True)
def enum(monkeypatch): monkeypatch.setattr(reopen,"_state",lambda x:x)
def test_happy(tmp_path):
r=state(tmp_path); old=(r/"approvals"/f"{CID}.json").read_bytes()
out=reopen.reopen_owner_approval(Service(r),req(),state_root=r,now=NOW)
assert out["status"]=="REOPENED"; s=json.loads((r/"sagas"/f"{CID}.json").read_text())
assert s["state"]=="READY_FOR_APPROVAL" and s["events"][-1]["event"]==reopen.EVENT_NAME
assert not (r/"approvals"/f"{CID}.json").exists()
assert (r/out["history_ref"]).read_bytes()==old
def test_idempotent(tmp_path):
r=state(tmp_path); svc=Service(r); a=reopen.reopen_owner_approval(svc,req(),state_root=r,now=NOW)
b=reopen.reopen_owner_approval(svc,req(),state_root=r,now=NOW)
assert b["status"]=="IDEMPOTENT" and a["approval_cycle_id"]==b["approval_cycle_id"]
@pytest.mark.parametrize("kind",["unexpired","saga","acceptance","side","advanced"])
def test_rejections(tmp_path,kind):
kw={}
expected=""
if kind=="unexpired": kw["expired"]=False; expected="APPROVAL_REOPEN_APPROVAL_NOT_EXPIRED"
if kind=="saga": kw["saga"]="READY_FOR_APPROVAL"; expected="APPROVAL_REOPEN_SAGA_STATE_INVALID"
if kind=="acceptance": kw["acceptance"]="REJECTED"; expected="APPROVAL_REOPEN_BINDING_MISMATCH"
if kind=="side": kw["side_marker"]=True; expected="APPROVAL_REOPEN_SIDE_EFFECTS_ALREADY_EXECUTED"
if kind=="advanced": kw["extra"]=True; expected="APPROVAL_REOPEN_SAGA_ALREADY_ADVANCED"
r=state(tmp_path,**kw)
with pytest.raises(reopen.ApprovalReopenError,match=expected):
reopen.reopen_owner_approval(Service(r),req(),state_root=r,now=NOW)
def test_binding_drift(tmp_path):
r=state(tmp_path); q=req(); q["expected_artifact_sha256"]="6"*64
with pytest.raises(reopen.ApprovalReopenError,match="APPROVAL_REOPEN_BINDING_MISMATCH"):
reopen.reopen_owner_approval(Service(r),q,state_root=r,now=NOW)
def test_history_collision(tmp_path):
r=state(tmp_path); h=r/"approval-history"/CID/f"{FP}.json"; h.parent.mkdir(parents=True); h.write_text("{}")
with pytest.raises(reopen.ApprovalReopenError,match="APPROVAL_REOPEN_HISTORY_COLLISION"):
reopen.reopen_owner_approval(Service(r),req(),state_root=r,now=NOW)
def test_real_durable_schema_without_side_effect_field_is_supported(tmp_path):
r=state(tmp_path)
candidate=json.loads((r/"candidates"/f"{CID}.json").read_text())
assert "side_effects_executed" not in candidate
out=reopen.reopen_owner_approval(
Service(r), req("reopen:test:real-schema"), state_root=r, now=NOW
)
assert out["status"]=="REOPENED"
@pytest.mark.parametrize("value",[True,None,1,"false"])
def test_explicit_non_false_side_effect_marker_fails_closed(tmp_path,value):
r=state(tmp_path,side_marker=value)
with pytest.raises(
reopen.ApprovalReopenError,
match="APPROVAL_REOPEN_SIDE_EFFECTS_ALREADY_EXECUTED"
):
reopen.reopen_owner_approval(
Service(r), req("reopen:test:marker:bad"), state_root=r, now=NOW
)
@@ -0,0 +1,64 @@
"""
Contract: release_approval_reopen must perform an atomic saga CAS that is
pinned to the saga's current revision.
Guards against the defect where `service.sagas.compare_and_set(...)` was called
without the required keyword-only `expected_revision` argument, which raised
TypeError -> APPROVAL_REOPEN_SAGA_CAS_FAILED (HTTP 409) and blocked the reopen.
"""
import re
from pathlib import Path
APP = Path(__file__).resolve().parents[1] / "active-runtime" / "app" / "release_approval_reopen.py"
MODULE = APP.name
CAS_CALL = re.compile(
r"service\.sagas\.compare_and_set\("
r"[^)]*?"
r"expected_revision\s*=\s*[^\s,]+"
r"[^)]*?\)",
re.S,
)
REVISION_SRC = re.compile(r"expected_revision\s*=\s*(?:saga\.get\(\s*[\"']revision[\"']\s*\)|saga\s*\.\s*revision)")
STATE_CAS = re.compile(r"expected_states\s*=\s*\{_state\(\s*[\"']APPROVED[\"']\s*\)\}")
TARGET_RFA = re.compile(r"target\s*=\s*_state\(\s*[\"']READY_FOR_APPROVAL[\"']\s*\)")
def _read():
if not APP.exists():
raise AssertionError(f"{MODULE} not found at {APP}")
return APP.read_text(encoding="utf-8")
def test_compare_and_set_call_requires_expected_revision():
src = _read()
# Exactly one compare_and_set call site in reopen module
calls = CAS_CALL.findall(src)
assert len(calls) == 1, f"expected exactly 1 compare_and_set call in {MODULE}, found {len(calls)}"
# The call must pass expected_revision (required keyword-only arg)
assert "expected_revision=" in calls[0], (
f"{MODULE} calls compare_and_set without expected_revision -> "
"TypeError -> APPROVAL_REOPEN_SAGA_CAS_FAILED"
)
# The expected_revision must be derived from the saga's current revision
assert REVISION_SRC.search(calls[0]), (
"expected_revision must be pinned to the saga current revision "
"(e.g. saga.get('revision') or saga.revision) for a true atomic CAS"
)
def test_cas_semantics_not_weakened():
src = _read()
# Still transitions APPROVED -> READY_FOR_APPROVAL (no weakening of state CAS)
assert STATE_CAS.search(src), "expected_states={_state('APPROVED')} must be preserved"
assert TARGET_RFA.search(src), "target=_state('READY_FOR_APPROVAL') must be preserved"
def test_reopen_still_binds_all_fingerprints():
src = _read()
# The expected_* sha256 bindings must still be enforced by ReopenRequest
for field in ("expected_approval_fingerprint", "expected_source_commit",
"expected_tree_oid", "expected_artifact_sha256",
"expected_manifest_sha256", "expected_release_plan_sha256",
"expected_policy_sha256"):
assert field in src, f"{MODULE} no longer enforces {field}"
+75
View File
@@ -0,0 +1,75 @@
from __future__ import annotations
import ast
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parents[1]
STAGING = ROOT / "active-runtime" / "app" / "aios_model_api_staging.py"
if not STAGING.is_file():
pytest.skip(
"shadow-gate contract runs after package is applied to an AIOS checkout",
allow_module_level=True,
)
def _literal_string_set(node: ast.AST) -> set[str]:
"""Extract string literals from a set or frozenset({...}) literal.
The governed codebase represents allow-lists as ``frozenset({...})``; a
plain ``{...}`` set literal is also accepted. Only string literal members
are allowed in the silent-release allow-list.
"""
if isinstance(node, ast.Call) and isinstance(node.func, ast.Name) and node.func.id == "frozenset":
if len(node.args) == 1:
node = node.args[0]
if not isinstance(node, ast.Set):
raise AssertionError("SILENT_RELEASE_POST_PATHS must remain a set literal")
values: set[str] = set()
for elt in node.elts:
if not isinstance(elt, ast.Constant) or not isinstance(elt.value, str):
raise AssertionError("silent-release allow-list must contain only string literals")
values.add(elt.value)
return values
def test_reopen_is_narrow_silent_release_post():
tree = ast.parse(STAGING.read_text(encoding="utf-8"))
allow = None
for node in tree.body:
if not isinstance(node, ast.Assign):
continue
if any(isinstance(t, ast.Name) and t.id == "SILENT_RELEASE_POST_PATHS" for t in node.targets):
allow = _literal_string_set(node.value)
break
assert allow is not None
assert "/reopenOwnerApproval" in allow
# Existing governed silent-release paths must stay present.
assert {
"/createReleaseCandidate",
"/createReleaseIntent",
"/submitSignedApproval",
"/submitOwnerApproval",
"/startDeployment",
}.issubset(allow)
def test_reopen_route_still_requires_release_approve():
import re
source = STAGING.read_text(encoding="utf-8")
# Locate the route handler (the `== "/reopenOwnerApproval"` comparison),
# independent of single/double-quote style on the allow-list entry.
m = re.search(r'==\s*["\']/reopenOwnerApproval["\']', source)
assert m is not None, "reopen handler route comparison not found"
tail = source[m.start() : m.start() + 1200]
assert 'self.need("release:approve")' in tail
assert "reopen_owner_approval(silent_release_service(), data)" in tail
def test_global_write_gate_is_not_enabled_by_source_patch():
source = STAGING.read_text(encoding="utf-8")
# v1.2 must not hard-code or synthesize a global write enable.
assert 'AIOS_GITOPS_WRITES_ENABLED=true' not in source
assert 'AIOS_GITOPS_WRITES_ENABLED="true"' not in source