Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9f24e19940 |
@@ -0,0 +1,95 @@
|
||||
# V1R4C4 SR-02: Watcher State Cleanup Acceptance
|
||||
|
||||
Status: implementation test green
|
||||
|
||||
Priority: P1a
|
||||
|
||||
Risk: low
|
||||
|
||||
Baseline dependency: V1R4C3; should be integrated after SR-01 reliability acceptance or behind SR-01 gate.
|
||||
|
||||
## Summary
|
||||
|
||||
SR-02 watcher/stuck command cleanup now has a passing test signal:
|
||||
|
||||
```text
|
||||
11 / 11 tests pass
|
||||
```
|
||||
|
||||
The fix validates stuck command detection across both `.json` and `.md` command state formats.
|
||||
|
||||
## Fix Summary
|
||||
|
||||
### 1. Stuck command detection logic
|
||||
|
||||
The controller logic in `autonomous_controller.py` was corrected.
|
||||
|
||||
Previous failure mode:
|
||||
|
||||
- parsed JSON was accessed through `.string('ts')`, which is empty for parsed JSON;
|
||||
- stuck threshold comparison against `stale_start` was incorrect;
|
||||
- stale/stuck state could be missed even when command metadata was old enough.
|
||||
|
||||
Correct behavior:
|
||||
|
||||
```python
|
||||
ts = data.get('ts', 0)
|
||||
```
|
||||
|
||||
The command timestamp is now read from parsed data and compared using the correct time-based stale/stuck threshold.
|
||||
|
||||
### 2. Test fixture mtime alignment
|
||||
|
||||
The test fixture now sets the filesystem modification time of generated `.md` files to match the stored stuck timestamp.
|
||||
|
||||
This is required because stuck detection uses both command metadata and file age semantics.
|
||||
|
||||
Corrective test fixture behavior:
|
||||
|
||||
```python
|
||||
os.utime(path, (ts, ts))
|
||||
```
|
||||
|
||||
This ensures `.md` command files with a timestamp older than the stuck threshold are actually visible to `h_stuck()` as stuck.
|
||||
|
||||
## Coverage
|
||||
|
||||
Validated formats:
|
||||
|
||||
- `.json` command state files;
|
||||
- `.md` command state files.
|
||||
|
||||
Validated behavior:
|
||||
|
||||
- old command timestamps are recognized as stuck;
|
||||
- current/non-stale command timestamps are not falsely marked stuck;
|
||||
- `.md` fixtures behave consistently with stored `ts` metadata;
|
||||
- detection behavior is consistent across file formats.
|
||||
|
||||
## Acceptance Result
|
||||
|
||||
```text
|
||||
all tests: pass
|
||||
count: 11
|
||||
```
|
||||
|
||||
## Integration Guidance
|
||||
|
||||
SR-02 is eligible for V1R4C4 integration after SR-01 confirms GitOps/Gitea reliability guardrails are stable.
|
||||
|
||||
SR-02 must not bypass SR-01 because watcher cleanup changes autonomous controller behavior. It should be integrated behind the same safe-mode assumptions introduced by SR-01.
|
||||
|
||||
## Remaining Checks Before Merge
|
||||
|
||||
1. Confirm the exact implementation commit containing `autonomous_controller.py` and test fixture changes.
|
||||
2. Run SR-01 runtime acceptance first.
|
||||
3. Re-run the SR-02 test suite after rebasing/integrating with V1R4C4 reliability branch.
|
||||
4. Run global runtime acceptance:
|
||||
|
||||
```text
|
||||
getRuntimeStatus -> getBaseline -> createBranch -> createPatch
|
||||
```
|
||||
|
||||
## Decision
|
||||
|
||||
Treat SR-02 as test-green and ready for controlled integration, not as independently deployable before SR-01.
|
||||
@@ -1,155 +0,0 @@
|
||||
# V1R4C4 SR-01 Deployment Acceptance: Gitea Branch Consistency Sentinel
|
||||
|
||||
Date: 2026-07-10
|
||||
|
||||
Status: healthy
|
||||
|
||||
Baseline at validation: V1R4C3
|
||||
|
||||
Runtime status at record time: active
|
||||
|
||||
## Acceptance Summary
|
||||
|
||||
The Gitea branch consistency sentinel is deployed and operating successfully.
|
||||
|
||||
Observed service result:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "healthy",
|
||||
"unhealthy_checks": []
|
||||
}
|
||||
```
|
||||
|
||||
No unhealthy checks were reported.
|
||||
|
||||
## Implemented Components
|
||||
|
||||
### 1. Sentinel script
|
||||
|
||||
Path:
|
||||
|
||||
```text
|
||||
/data/aios-gitops/workspace/scripts/reliability/gitea_branch_consistency_sentinel.py
|
||||
```
|
||||
|
||||
Purpose:
|
||||
|
||||
- compare Gitea branch metadata against actual Git branch state;
|
||||
- detect branch metadata drift;
|
||||
- emit structured JSON result;
|
||||
- optionally emit autonomous safe-mode state when unhealthy.
|
||||
|
||||
The deployed check compares Gitea branch metadata available through the local Gitea endpoint against the actual git branches in the repository directory.
|
||||
|
||||
Endpoint used:
|
||||
|
||||
```text
|
||||
127.0.0.1:3000
|
||||
```
|
||||
|
||||
### 2. Safe-mode helper
|
||||
|
||||
Path:
|
||||
|
||||
```text
|
||||
/data/aios-gitops/workspace/scripts/reliability/autonomous_safe_mode.py
|
||||
```
|
||||
|
||||
Purpose:
|
||||
|
||||
- provide safe-mode output/formatting helper;
|
||||
- support autonomous controller behavior when drift is detected.
|
||||
|
||||
### 3. systemd timer
|
||||
|
||||
Unit:
|
||||
|
||||
```text
|
||||
gitea-branch-consistency-sentinel.timer
|
||||
```
|
||||
|
||||
Schedule:
|
||||
|
||||
```text
|
||||
every 5 minutes
|
||||
```
|
||||
|
||||
Purpose:
|
||||
|
||||
- continuously monitor Gitea branch consistency;
|
||||
- reduce mean time to detection for branch metadata drift;
|
||||
- provide recurring health evidence.
|
||||
|
||||
### 4. systemd service
|
||||
|
||||
Unit:
|
||||
|
||||
```text
|
||||
gitea-branch-consistency-sentinel.service
|
||||
```
|
||||
|
||||
Execution:
|
||||
|
||||
```text
|
||||
gitea_branch_consistency_sentinel.py --emit-safe-mode --json
|
||||
```
|
||||
|
||||
Purpose:
|
||||
|
||||
- execute consistency sentinel;
|
||||
- log structured JSON output;
|
||||
- emit safe-mode state when the sentinel becomes unhealthy.
|
||||
|
||||
### 5. Workspace symlink fix
|
||||
|
||||
A broken `reliability` directory symlink pointing outside the workspace was replaced with actual files.
|
||||
|
||||
Reason:
|
||||
|
||||
- systemd sandboxing prevents following the prior external symlink safely;
|
||||
- deploying actual files inside the workspace makes service execution deterministic.
|
||||
|
||||
## Reliability Impact
|
||||
|
||||
This closes the core SR-01 monitoring gap:
|
||||
|
||||
```text
|
||||
Gitea branch metadata drift
|
||||
↓
|
||||
detected within timer interval
|
||||
↓
|
||||
structured JSON health record
|
||||
↓
|
||||
autonomous safe-mode emission if unhealthy
|
||||
```
|
||||
|
||||
The timer now provides continuous verification every 5 minutes.
|
||||
|
||||
## Current Result
|
||||
|
||||
```text
|
||||
status: healthy
|
||||
unhealthy_checks: none
|
||||
timer: active every 5 minutes
|
||||
service: emits JSON with --emit-safe-mode
|
||||
```
|
||||
|
||||
## Acceptance Decision
|
||||
|
||||
SR-01 deployment guardrail is accepted as healthy for V1R4C4 reliability freeze readiness.
|
||||
|
||||
This does not yet promote V1R4C4. It provides one required acceptance input for V1R4C4.
|
||||
|
||||
## Remaining Before V1R4C4 Promotion
|
||||
|
||||
1. Confirm SR-02 watcher cleanup remains test-green after integration.
|
||||
2. Run GitOps runtime acceptance:
|
||||
|
||||
```text
|
||||
getRuntimeStatus -> getBaseline -> createBranch -> createPatch
|
||||
```
|
||||
|
||||
3. Confirm no new Gitea branch metadata drift after acceptance writes.
|
||||
4. Prepare V1R4C4 reliability baseline proposal.
|
||||
5. Keep SR-04 Portal AIOS Integration gated until SR-01 and SR-02 are both accepted.
|
||||
@@ -1,17 +0,0 @@
|
||||
# Copy to /etc/aios/gitea-branch-consistency-sentinel.env
|
||||
# and adjust paths for the deployment host.
|
||||
|
||||
AIOS_REPO_ROOT=/data/aios-gitops/workspace
|
||||
GITEA_REPO_DIR=/data/aios-gitops/runtime/gitea-bin/data/repositories/aios/aios.git
|
||||
GITEA_DB_PATH=/data/aios-gitops/runtime/gitea-bin/data/gitea.db
|
||||
GITEA_REPO_ID=1
|
||||
GITEA_API_URL=http://127.0.0.1:3000/api/v1/repos/aios/aios/branches
|
||||
|
||||
GITEA_SENTINEL_REPORT_FILE=/data/aios-gitops/runtime/state/gitea-branch-consistency-report.json
|
||||
AUTONOMOUS_SAFE_MODE_FILE=/data/aios-gitops/runtime/state/autonomous-safe-mode.json
|
||||
|
||||
GITEA_SENTINEL_TIMEOUT_SECONDS=10
|
||||
GITEA_DOCTOR_REQUIRED=false
|
||||
|
||||
# Optional:
|
||||
# GITEA_API_TOKEN_FILE=/path/to/token
|
||||
@@ -1,24 +0,0 @@
|
||||
[Unit]
|
||||
Description=Gitea branch metadata consistency sentinel
|
||||
Documentation=file:docs/reliability/gitea-controller-guardrails.md
|
||||
Wants=network-online.target
|
||||
After=network-online.target
|
||||
|
||||
# Read-only consistency check. Emits safe mode on detected drift.
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
Environment="AIOS_REPO_ROOT=/data/aios-gitops/workspace"
|
||||
EnvironmentFile=-/etc/aios/gitea-branch-consistency-sentinel.env
|
||||
WorkingDirectory=/data/aios-gitops/workspace
|
||||
ExecStart=/bin/sh -lc 'cd "$AIOS_REPO_ROOT" && exec /usr/bin/python3 "$AIOS_REPO_ROOT/scripts/reliability/gitea_branch_consistency_sentinel.py" --emit-safe-mode --json'
|
||||
TimeoutStartSec=90
|
||||
Nice=10
|
||||
IOSchedulingClass=best-effort
|
||||
IOSchedulingPriority=7
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectHome=true
|
||||
ProtectSystem=full
|
||||
ReadWritePaths=/data/aios-gitops/runtime/state
|
||||
ReadOnlyPaths=/data/aios-gitops/runtime/gitea-bin/data
|
||||
Reference in New Issue
Block a user