Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
12d09b1dea |
@@ -0,0 +1,131 @@
|
||||
# V1R4C4 SR-01: Gitea Branch Sync Fix
|
||||
|
||||
Status: started
|
||||
|
||||
Priority: P0
|
||||
|
||||
Risk: medium
|
||||
|
||||
Baseline dependency: V1R4C3
|
||||
|
||||
## Decision
|
||||
|
||||
Start V1R4C4 with SR-01 only.
|
||||
|
||||
SR-04 Portal AIOS Integration Merge remains deferred and is not part of this execution scope.
|
||||
|
||||
## Background
|
||||
|
||||
V1R4C3 completed the immediate branch_sync mitigation by changing branch synchronization to use `git ls-remote` instead of depending on the Gitea branch API pagination path.
|
||||
|
||||
That mitigation avoids the user-visible failure path, but it does not fully certify or eliminate the root reliability issue:
|
||||
|
||||
```text
|
||||
bare repository refs
|
||||
!=
|
||||
Gitea branch metadata / DB index
|
||||
!=
|
||||
Gitea branch API visible state
|
||||
```
|
||||
|
||||
SR-01 exists to validate and fix the root problem category: Gitea branch metadata/index drift from the bare repository state.
|
||||
|
||||
## Scope
|
||||
|
||||
SR-01 includes:
|
||||
|
||||
1. Confirm current branch_sync no longer depends on Gitea API pagination.
|
||||
2. Validate that Gitea DB branch metadata is consistent with bare repository refs.
|
||||
3. Add or activate consistency verification around branch metadata drift.
|
||||
4. Ensure autonomous operations do not continue mutating state when drift is detected.
|
||||
5. Produce a repeatable acceptance test for future incidents.
|
||||
|
||||
SR-01 excludes:
|
||||
|
||||
- Portal AIOS Integration Merge.
|
||||
- Search Gateway real provider binding.
|
||||
- Broad P1 capability merges.
|
||||
- Bulk patch merge or branch deletion.
|
||||
|
||||
## Required Checks
|
||||
|
||||
### Branch sync path
|
||||
|
||||
Verify branch_sync source of truth:
|
||||
|
||||
```text
|
||||
git ls-remote / bare repository refs
|
||||
```
|
||||
|
||||
The Gitea branch API must not be the mandatory source of truth for synchronization.
|
||||
|
||||
### Gitea consistency path
|
||||
|
||||
Verify equality between:
|
||||
|
||||
```text
|
||||
bare repository refs count
|
||||
Gitea branch live metadata count
|
||||
Gitea branch API paginated visible count
|
||||
```
|
||||
|
||||
Any mismatch must be treated as reliability drift.
|
||||
|
||||
### Safe-mode behavior
|
||||
|
||||
When drift is detected:
|
||||
|
||||
1. Stop autonomous mutations.
|
||||
2. Preserve read-only diagnostics.
|
||||
3. Emit safe-mode marker.
|
||||
4. Require controlled recovery before mutation resumes.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
SR-01 is complete when all of the following are true:
|
||||
|
||||
1. branch_sync continues to work without Gitea branch API pagination dependency.
|
||||
2. Gitea branch metadata drift can be detected deterministically.
|
||||
3. Drift causes autonomous safe mode instead of continued mutation.
|
||||
4. Recovery procedure includes minimal repair, not default restart.
|
||||
5. Runtime acceptance passes:
|
||||
|
||||
```text
|
||||
getRuntimeStatus
|
||||
->
|
||||
getBaseline
|
||||
->
|
||||
createBranch
|
||||
->
|
||||
createPatch
|
||||
```
|
||||
|
||||
Expected results:
|
||||
|
||||
- runtime status is active;
|
||||
- baseline reads V1R4C3 or later;
|
||||
- createBranch returns commit/rev/sync status;
|
||||
- createPatch returns commit/rev/sync status.
|
||||
|
||||
## Execution Order
|
||||
|
||||
1. Open SR-01 patch branch.
|
||||
2. Review V1R4C3 branch_sync behavior.
|
||||
3. Integrate Gitea branch consistency sentinel and safe-mode gate.
|
||||
4. Run runtime acceptance.
|
||||
5. If clean, prepare V1R4C4 reliability baseline proposal.
|
||||
|
||||
## Follow-on Gates
|
||||
|
||||
SR-02 Watcher State Cleanup may start after SR-01 acceptance is green.
|
||||
|
||||
SR-03 Search Gateway Real Provider Binding may start after SR-01 confirms GitOps/Gitea reliability is stable enough for non-P0 work.
|
||||
|
||||
SR-04 Portal AIOS Integration Merge remains deferred pending explicit approval.
|
||||
|
||||
## Safety Rules
|
||||
|
||||
- Do not bulk merge old patches.
|
||||
- Do not delete remote branches during SR-01 unless the branch consistency sentinel is already deployed and healthy.
|
||||
- Do not restart Gitea unless process-level failure is proven.
|
||||
- Do not modify Gitea DB without backup and owner/admin approval.
|
||||
@@ -1,155 +0,0 @@
|
||||
# V1R4C4 SR-01 Deployment Acceptance: Gitea Branch Consistency Sentinel
|
||||
|
||||
Date: 2026-07-10
|
||||
|
||||
Status: healthy
|
||||
|
||||
Baseline at validation: V1R4C3
|
||||
|
||||
Runtime status at record time: active
|
||||
|
||||
## Acceptance Summary
|
||||
|
||||
The Gitea branch consistency sentinel is deployed and operating successfully.
|
||||
|
||||
Observed service result:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "healthy",
|
||||
"unhealthy_checks": []
|
||||
}
|
||||
```
|
||||
|
||||
No unhealthy checks were reported.
|
||||
|
||||
## Implemented Components
|
||||
|
||||
### 1. Sentinel script
|
||||
|
||||
Path:
|
||||
|
||||
```text
|
||||
/data/aios-gitops/workspace/scripts/reliability/gitea_branch_consistency_sentinel.py
|
||||
```
|
||||
|
||||
Purpose:
|
||||
|
||||
- compare Gitea branch metadata against actual Git branch state;
|
||||
- detect branch metadata drift;
|
||||
- emit structured JSON result;
|
||||
- optionally emit autonomous safe-mode state when unhealthy.
|
||||
|
||||
The deployed check compares Gitea branch metadata available through the local Gitea endpoint against the actual git branches in the repository directory.
|
||||
|
||||
Endpoint used:
|
||||
|
||||
```text
|
||||
127.0.0.1:3000
|
||||
```
|
||||
|
||||
### 2. Safe-mode helper
|
||||
|
||||
Path:
|
||||
|
||||
```text
|
||||
/data/aios-gitops/workspace/scripts/reliability/autonomous_safe_mode.py
|
||||
```
|
||||
|
||||
Purpose:
|
||||
|
||||
- provide safe-mode output/formatting helper;
|
||||
- support autonomous controller behavior when drift is detected.
|
||||
|
||||
### 3. systemd timer
|
||||
|
||||
Unit:
|
||||
|
||||
```text
|
||||
gitea-branch-consistency-sentinel.timer
|
||||
```
|
||||
|
||||
Schedule:
|
||||
|
||||
```text
|
||||
every 5 minutes
|
||||
```
|
||||
|
||||
Purpose:
|
||||
|
||||
- continuously monitor Gitea branch consistency;
|
||||
- reduce mean time to detection for branch metadata drift;
|
||||
- provide recurring health evidence.
|
||||
|
||||
### 4. systemd service
|
||||
|
||||
Unit:
|
||||
|
||||
```text
|
||||
gitea-branch-consistency-sentinel.service
|
||||
```
|
||||
|
||||
Execution:
|
||||
|
||||
```text
|
||||
gitea_branch_consistency_sentinel.py --emit-safe-mode --json
|
||||
```
|
||||
|
||||
Purpose:
|
||||
|
||||
- execute consistency sentinel;
|
||||
- log structured JSON output;
|
||||
- emit safe-mode state when the sentinel becomes unhealthy.
|
||||
|
||||
### 5. Workspace symlink fix
|
||||
|
||||
A broken `reliability` directory symlink pointing outside the workspace was replaced with actual files.
|
||||
|
||||
Reason:
|
||||
|
||||
- systemd sandboxing prevents following the prior external symlink safely;
|
||||
- deploying actual files inside the workspace makes service execution deterministic.
|
||||
|
||||
## Reliability Impact
|
||||
|
||||
This closes the core SR-01 monitoring gap:
|
||||
|
||||
```text
|
||||
Gitea branch metadata drift
|
||||
↓
|
||||
detected within timer interval
|
||||
↓
|
||||
structured JSON health record
|
||||
↓
|
||||
autonomous safe-mode emission if unhealthy
|
||||
```
|
||||
|
||||
The timer now provides continuous verification every 5 minutes.
|
||||
|
||||
## Current Result
|
||||
|
||||
```text
|
||||
status: healthy
|
||||
unhealthy_checks: none
|
||||
timer: active every 5 minutes
|
||||
service: emits JSON with --emit-safe-mode
|
||||
```
|
||||
|
||||
## Acceptance Decision
|
||||
|
||||
SR-01 deployment guardrail is accepted as healthy for V1R4C4 reliability freeze readiness.
|
||||
|
||||
This does not yet promote V1R4C4. It provides one required acceptance input for V1R4C4.
|
||||
|
||||
## Remaining Before V1R4C4 Promotion
|
||||
|
||||
1. Confirm SR-02 watcher cleanup remains test-green after integration.
|
||||
2. Run GitOps runtime acceptance:
|
||||
|
||||
```text
|
||||
getRuntimeStatus -> getBaseline -> createBranch -> createPatch
|
||||
```
|
||||
|
||||
3. Confirm no new Gitea branch metadata drift after acceptance writes.
|
||||
4. Prepare V1R4C4 reliability baseline proposal.
|
||||
5. Keep SR-04 Portal AIOS Integration gated until SR-01 and SR-02 are both accepted.
|
||||
@@ -1,17 +0,0 @@
|
||||
# Copy to /etc/aios/gitea-branch-consistency-sentinel.env
|
||||
# and adjust paths for the deployment host.
|
||||
|
||||
AIOS_REPO_ROOT=/data/aios-gitops/workspace
|
||||
GITEA_REPO_DIR=/data/aios-gitops/runtime/gitea-bin/data/repositories/aios/aios.git
|
||||
GITEA_DB_PATH=/data/aios-gitops/runtime/gitea-bin/data/gitea.db
|
||||
GITEA_REPO_ID=1
|
||||
GITEA_API_URL=http://127.0.0.1:3000/api/v1/repos/aios/aios/branches
|
||||
|
||||
GITEA_SENTINEL_REPORT_FILE=/data/aios-gitops/runtime/state/gitea-branch-consistency-report.json
|
||||
AUTONOMOUS_SAFE_MODE_FILE=/data/aios-gitops/runtime/state/autonomous-safe-mode.json
|
||||
|
||||
GITEA_SENTINEL_TIMEOUT_SECONDS=10
|
||||
GITEA_DOCTOR_REQUIRED=false
|
||||
|
||||
# Optional:
|
||||
# GITEA_API_TOKEN_FILE=/path/to/token
|
||||
@@ -1,24 +0,0 @@
|
||||
[Unit]
|
||||
Description=Gitea branch metadata consistency sentinel
|
||||
Documentation=file:docs/reliability/gitea-controller-guardrails.md
|
||||
Wants=network-online.target
|
||||
After=network-online.target
|
||||
|
||||
# Read-only consistency check. Emits safe mode on detected drift.
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
Environment="AIOS_REPO_ROOT=/data/aios-gitops/workspace"
|
||||
EnvironmentFile=-/etc/aios/gitea-branch-consistency-sentinel.env
|
||||
WorkingDirectory=/data/aios-gitops/workspace
|
||||
ExecStart=/bin/sh -lc 'cd "$AIOS_REPO_ROOT" && exec /usr/bin/python3 "$AIOS_REPO_ROOT/scripts/reliability/gitea_branch_consistency_sentinel.py" --emit-safe-mode --json'
|
||||
TimeoutStartSec=90
|
||||
Nice=10
|
||||
IOSchedulingClass=best-effort
|
||||
IOSchedulingPriority=7
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectHome=true
|
||||
ProtectSystem=full
|
||||
ReadWritePaths=/data/aios-gitops/runtime/state
|
||||
ReadOnlyPaths=/data/aios-gitops/runtime/gitea-bin/data
|
||||
Reference in New Issue
Block a user