Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c67dd11029 | ||
|
|
4ed889c1e4 | ||
|
|
5e49502ae7 | ||
|
|
7a0ebac2aa | ||
|
|
7c8a1a6489 | ||
|
|
bcbd4ebc02 | ||
|
|
7342d5d97a | ||
|
|
9387c3d82a | ||
|
|
f0521fa1e0 | ||
|
|
8bc2da2c09 | ||
|
|
75b93bbba5 | ||
|
|
f0fc084fbb | ||
|
|
d7478726a8 | ||
|
|
504f77cb7c | ||
|
|
3402fd69d4 | ||
|
|
827b783be3 | ||
|
|
751aaed4b1 | ||
|
|
57caeb6b5a | ||
|
|
cf41f7c64d | ||
|
|
6c706a727e | ||
|
|
c7e7a86627 | ||
|
|
02f385866f | ||
|
|
763b025557 | ||
|
|
1aec062aba | ||
|
|
5596ca935c | ||
|
|
41700c27e5 | ||
|
|
8e4532f6f5 | ||
|
|
dd4021b678 | ||
|
|
568ff6643e | ||
|
|
d2e684f0cf | ||
|
|
6ffb90d1d1 | ||
|
|
61dcb6bf13 | ||
|
|
21937f920f | ||
|
|
f1ee385a1b | ||
|
|
3de1dcdc84 | ||
|
|
b6c04f2296 | ||
|
|
adca7c6edc | ||
|
|
2891b37e75 | ||
|
|
54099e1da3 | ||
|
|
96e934ed8f | ||
|
|
c8cb263c77 | ||
|
|
81dd816df2 | ||
|
|
58cae16ef8 | ||
|
|
8cb5a664c6 | ||
|
|
df345b32e1 | ||
|
|
b1916236f4 | ||
|
|
5c3c238d17 | ||
|
|
3b3cb5bd7f | ||
|
|
8cd30488d2 | ||
|
|
e927afdb09 | ||
|
|
7ac1a4a11b | ||
|
|
5ca2cdc9af | ||
|
|
7dca40c55e | ||
|
|
de9dc61ebc | ||
|
|
a335f654ed | ||
|
|
51fa19a8ee | ||
|
|
91731fd317 | ||
|
|
62986245a2 | ||
|
|
148868bdfb | ||
|
|
d2744bc1ac | ||
|
|
2b19277b1e | ||
|
|
a774a2fbbd | ||
|
|
98df351235 | ||
|
|
28055a8551 | ||
|
|
0e4eb1738c | ||
|
|
70a0c6065e | ||
|
|
a41580aceb | ||
|
|
e4253eb076 | ||
|
|
db901d6401 | ||
|
|
24f9d6195b | ||
|
|
5aa766acff | ||
|
|
f9760fc098 | ||
|
|
00826a40d3 | ||
|
|
76c9673ab3 | ||
|
|
3794df20d4 | ||
|
|
d7fb96bb70 | ||
|
|
bf8e4e3cd1 | ||
|
|
b0f463ac5c | ||
|
|
ec540a1a72 |
@@ -0,0 +1,181 @@
|
||||
---
|
||||
name: "OPSX: Apply"
|
||||
description: "Implement tasks from an OpenSpec change (Experimental)"
|
||||
allowed-tools: Bash(openspec:*)
|
||||
category: "Workflow"
|
||||
tags: ["workflow", "artifacts", "experimental"]
|
||||
---
|
||||
|
||||
Implement tasks from an OpenSpec change.
|
||||
|
||||
**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store <id>` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `view`). Once selected, treat `--store <id>` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "<name>" --json --store "<id>"`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root.
|
||||
|
||||
**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes and ask the user to select one
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
|
||||
|
||||
2. **Check status to understand the schema**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used (e.g., "spec-driven")
|
||||
- `planningHome`, `changeRoot`, and `actionContext`: planning scope and edit constraints
|
||||
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
|
||||
|
||||
3. **Get apply instructions**
|
||||
|
||||
```bash
|
||||
openspec instructions apply --change "<name>" --json
|
||||
```
|
||||
|
||||
This returns:
|
||||
- `contextFiles`: artifact ID -> array of concrete file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
|
||||
- Progress (total, complete, remaining)
|
||||
- Task list with status
|
||||
- Dynamic instruction based on current state
|
||||
- Optional `context`: current required project instruction input from the selected root
|
||||
- Optional `operationGuidance`: current advisory guidance for apply
|
||||
|
||||
**Handle states:**
|
||||
- If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx:continue` (if it is not installed, run `openspec status --change "<name>" --json` to see the next artifact and `openspec instructions <artifact-id> --change "<name>" --json` for how to create it)
|
||||
- If `state: "all_done"`: congratulate, suggest archive
|
||||
- Otherwise: proceed to implementation
|
||||
|
||||
Treat `context` as a required prompt-level input. Read and consider it, and
|
||||
apply relevant project facts, conventions, and constraints while implementing.
|
||||
Treat `operationGuidance` as optional additive advice. Read and consider every
|
||||
entry, and follow entries that are applicable and compatible with the built-in
|
||||
workflow.
|
||||
|
||||
Keep both fields separate from CLI-returned state, missing artifacts, tasks,
|
||||
progress, `contextFiles`, and the built-in `instruction`. They are not
|
||||
evidence of task completion, do not replace the built-in instruction, and do
|
||||
not permit bypassing a blocked state. If context conflicts with the built-in
|
||||
instruction, an explicit user choice, or a CLI-controlled value, report the
|
||||
conflict and preserve the controlling value. If guidance is inapplicable or
|
||||
conflicts with those controlling inputs, do not follow it and explain why.
|
||||
These are prompt-level behavior contracts, not enforceable checks.
|
||||
|
||||
4. **Read context files**
|
||||
|
||||
Read every file path listed under `contextFiles` from the apply instructions output.
|
||||
The files depend on the schema being used:
|
||||
- **spec-driven**: proposal, specs, design, tasks
|
||||
- Other schemas: follow the contextFiles from CLI output
|
||||
|
||||
Do not copy `context` or `operationGuidance` verbatim into implementation
|
||||
files or planning artifacts unless the user separately asks for that content.
|
||||
|
||||
5. **Show current progress**
|
||||
|
||||
Display:
|
||||
- Schema being used
|
||||
- Progress: "N/M tasks complete"
|
||||
- Remaining tasks overview
|
||||
- Dynamic instruction from CLI
|
||||
|
||||
6. **Implement tasks (loop until done or blocked)**
|
||||
|
||||
For each pending task:
|
||||
- Show which task is being worked on
|
||||
- Make the code changes required
|
||||
- Keep changes minimal and focused
|
||||
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
|
||||
- Continue to next task
|
||||
|
||||
**Pause if:**
|
||||
- Task is unclear → ask for clarification
|
||||
- Implementation reveals a design issue → suggest updating artifacts
|
||||
- Error or blocker encountered → report and wait for guidance
|
||||
- User interrupts
|
||||
|
||||
7. **On completion or pause, show status**
|
||||
|
||||
Display:
|
||||
- Tasks completed this session
|
||||
- Overall progress: "N/M tasks complete"
|
||||
- If all done: suggest archive
|
||||
- If paused: explain why and wait for guidance
|
||||
|
||||
**Output During Implementation**
|
||||
|
||||
```
|
||||
## Implementing: <change-name> (schema: <schema-name>)
|
||||
|
||||
Working on task 3/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
|
||||
Working on task 4/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
```
|
||||
|
||||
**Output On Completion**
|
||||
|
||||
```
|
||||
## Implementation Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 7/7 tasks complete ✓
|
||||
|
||||
### Completed This Session
|
||||
- [x] Task 1
|
||||
- [x] Task 2
|
||||
...
|
||||
|
||||
All tasks complete! You can archive this change with `/opsx:archive`.
|
||||
```
|
||||
|
||||
**Output On Pause (Issue Encountered)**
|
||||
|
||||
```
|
||||
## Implementation Paused
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 4/7 tasks complete
|
||||
|
||||
### Issue Encountered
|
||||
<description of the issue>
|
||||
|
||||
**Options:**
|
||||
1. <option 1>
|
||||
2. <option 2>
|
||||
3. Other approach
|
||||
|
||||
What would you like to do?
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Keep going through tasks until done or blocked
|
||||
- Always read context files before starting (from the apply instructions output)
|
||||
- If task is ambiguous, pause and ask before implementing
|
||||
- If implementation reveals issues, pause and suggest artifact updates
|
||||
- Keep code changes minimal and scoped to each task
|
||||
- Update task checkbox immediately after completing each task
|
||||
- Pause on errors, blockers, or unclear requirements - don't guess
|
||||
- Use contextFiles from CLI output, don't assume specific file names
|
||||
- Do not use context or operation guidance as proof that a task is complete
|
||||
- Apply relevant project context; report conflicts with controlling workflow inputs
|
||||
- Consider every guidance entry; explain any inapplicable or conflicting advice
|
||||
- Do not copy runtime context or operation guidance into implementation files or planning artifacts
|
||||
- Preserve CLI-controlled blocked/ready/all-done behavior and completion criteria
|
||||
|
||||
**Fluid Workflow Integration**
|
||||
|
||||
This skill supports the "actions on a change" model:
|
||||
|
||||
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
|
||||
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly
|
||||
@@ -0,0 +1,225 @@
|
||||
---
|
||||
name: "OPSX: Archive"
|
||||
description: "Archive a completed change in the experimental workflow"
|
||||
allowed-tools: Bash(openspec:*)
|
||||
category: "Workflow"
|
||||
tags: ["workflow", "archive", "experimental"]
|
||||
---
|
||||
|
||||
Archive a completed change in the experimental workflow.
|
||||
|
||||
**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store <id>` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `view`). Once selected, treat `--store <id>` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "<name>" --json --store "<id>"`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root.
|
||||
|
||||
`<capability-path>` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path from each delta spec when resolving its main spec.
|
||||
|
||||
**Input**: Optionally specify a change name after `/opsx:archive` (e.g., `/opsx:archive add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes and ask the user to select one
|
||||
|
||||
When prompting, show only active changes (not already archived).
|
||||
Include the schema used for each change if available.
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:archive <other>`).
|
||||
|
||||
**Load current archive inputs before the existing archive checks:**
|
||||
|
||||
After resolving the selected change and planning root, run:
|
||||
```bash
|
||||
openspec instructions archive --change "<name>" --json
|
||||
```
|
||||
Keep the same selected-root flags on this command. This lookup is advisory and
|
||||
optional: it only supplies extra prompt inputs, so it must never block archiving.
|
||||
If it exits non-zero or returns invalid JSON — for example on an older CLI that
|
||||
does not support this command yet — continue the archive workflow with no
|
||||
context and no operation guidance. Do not report an error and do not stop.
|
||||
|
||||
A successful response may omit both optional fields. Treat `context` as a
|
||||
required prompt-level input: read and consider it, and apply relevant project
|
||||
facts, conventions, and constraints. Treat `operationGuidance` as optional
|
||||
additive advice: read and consider every entry, and follow entries that are
|
||||
applicable and compatible with the built-in archive workflow.
|
||||
|
||||
Keep both fields separate from built-in steps, explicit user choices, resolved
|
||||
paths, CLI checks, and command contracts. If context conflicts with one of those
|
||||
controlling inputs, report the conflict and preserve the controlling value. If
|
||||
guidance is inapplicable or conflicts with a controlling input, do not follow it
|
||||
and explain why. Do not infer replacement paths, skipped prompts, or flags from
|
||||
either field, and do not copy their text verbatim into specs, change artifacts,
|
||||
or archive summaries unless the user separately asks for it. These are
|
||||
prompt-level behavior contracts, not enforceable checks.
|
||||
|
||||
2. **Check artifact completion status**
|
||||
|
||||
Run `openspec status --change "<name>" --json` to check artifact completion.
|
||||
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used
|
||||
- `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context
|
||||
- `artifacts`: List of artifacts with their status (`done`, `skipped`, or other)
|
||||
|
||||
**If any artifacts are neither `done` nor `skipped`** (skipped artifacts satisfy the requirement - the change declares skip_specs):
|
||||
- Display warning listing incomplete artifacts
|
||||
- Prompt user for confirmation to continue
|
||||
- Proceed if user confirms
|
||||
|
||||
3. **Check task completion status**
|
||||
|
||||
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
|
||||
|
||||
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
|
||||
|
||||
**If incomplete tasks found:**
|
||||
- Display warning showing count of incomplete tasks
|
||||
- Prompt user for confirmation to continue
|
||||
- Proceed if user confirms
|
||||
|
||||
**If no tasks file exists:** Proceed without task-related warning.
|
||||
|
||||
4. **Assess delta spec sync state**
|
||||
|
||||
Use `artifactPaths.specs.existingOutputPaths` from status JSON as the only
|
||||
delta-spec source. If the `specs` entry is missing or
|
||||
`existingOutputPaths` is empty, proceed without a sync prompt and do not infer
|
||||
delta specs from other artifacts.
|
||||
|
||||
**If delta specs exist:**
|
||||
- Compare each delta spec with its corresponding main spec at `<planningHome.root>/openspec/specs/<capability-path>/spec.md` (use the store-aware `planningHome.root` from step 2, not a hardcoded repo path)
|
||||
- Determine what changes would be applied (adds, modifications, removals, renames)
|
||||
- Show a combined summary before prompting
|
||||
|
||||
**Prompt options:**
|
||||
- If changes needed: "Sync now (recommended)", "Archive without syncing"
|
||||
- If already synced: "Archive now", "Sync anyway", "Cancel"
|
||||
|
||||
Route on the answer:
|
||||
- "Cancel" — stop, do not archive
|
||||
- "Archive without syncing" or "Archive now" — proceed to archive
|
||||
- "Sync now" or "Sync anyway" — sync, then verify (below)
|
||||
- Anything else — ask again rather than archiving
|
||||
|
||||
Before a selected sync writes any main spec, run
|
||||
`openspec instructions specs --change "<name>" --json` once with the same
|
||||
selected-root flags. Require a zero exit status and valid artifact-instruction
|
||||
JSON. If the lookup fails or returns invalid JSON, report the error and stop
|
||||
before writing any main spec or moving the change. A valid response with omitted
|
||||
`rules` is the no-rules case. Apply returned `rules` only to the content and
|
||||
form of main specs produced by this merge; do not use them as archive guidance,
|
||||
change CLI behavior, or copy the rule text into any output file.
|
||||
|
||||
Then run the `/opsx:sync` workflow inline (agent-driven intelligent merge) for change '<name>', passing the delta spec analysis and the fetched specs-rule snapshot from above, and wait for it to finish. The inline sync must reuse that snapshot without fetching `specs` instructions again. Do not delegate it to a background task — step 5 would move `changeRoot` out from under a sync that is still reading it, leaving the change archived and the main specs never updated. If your agent can only run it by delegation, delegate synchronously and wait for the result.
|
||||
|
||||
Then re-run the comparison from the top of this step against every capability that has a delta spec in `artifactPaths.specs.existingOutputPaths` — not only the ones the sync reports it touched. A successful sync leaves nothing left to apply, so each capability must now read as already synced:
|
||||
- ADDED requirements present
|
||||
- MODIFIED requirements carrying the scenario and description changes named in the delta, with their other scenarios intact
|
||||
- REMOVED requirements gone — and where this sync retired a capability (removed its last requirement, leaving `## Requirements` empty), its main spec deleted rather than left empty; a spec the sync deliberately kept and reported is also a match
|
||||
- RENAMED requirements present under the new name and absent under the old one
|
||||
|
||||
If the sync failed, or any capability does not match, report what differs and stop — do not archive. Nothing has moved and `changeRoot` is intact, so the user can fix the mismatch or re-run the sync and start the archive again.
|
||||
|
||||
5. **Perform the archive**
|
||||
|
||||
Create an `archive` directory under `planningHome.changesDir` if it doesn't exist:
|
||||
```bash
|
||||
mkdir -p "<planningHome.changesDir>/archive"
|
||||
```
|
||||
|
||||
Generate the target name: use the change name as-is when it already starts with a `YYYY-MM-DD-` prefix; otherwise prepend the current date as `YYYY-MM-DD-<change-name>`. Never stack a second date (same rule as `openspec archive`).
|
||||
|
||||
**Check if target already exists:**
|
||||
- If yes: Fail with error, suggest renaming existing archive or using different date
|
||||
- If no: Move `changeRoot` to the archive directory
|
||||
|
||||
```bash
|
||||
mv "<changeRoot>" "<planningHome.changesDir>/archive/<target-name>"
|
||||
```
|
||||
|
||||
6. **Display summary**
|
||||
|
||||
Show archive completion summary including:
|
||||
- Change name
|
||||
- Schema that was used
|
||||
- Archive location
|
||||
- Spec sync status (synced / sync skipped / no delta specs)
|
||||
- Note about any warnings (incomplete artifacts/tasks)
|
||||
|
||||
**Output On Success**
|
||||
|
||||
```markdown
|
||||
## Archive Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** the archive path derived from `planningHome.changesDir`/<target-name>/
|
||||
**Specs:** ✓ Synced to main specs
|
||||
|
||||
All artifacts complete. All tasks complete.
|
||||
```
|
||||
|
||||
**Output On Success (No Delta Specs)**
|
||||
|
||||
```markdown
|
||||
## Archive Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** the archive path derived from `planningHome.changesDir`/<target-name>/
|
||||
**Specs:** No delta specs
|
||||
|
||||
All artifacts complete. All tasks complete.
|
||||
```
|
||||
|
||||
**Output On Success With Warnings**
|
||||
|
||||
```markdown
|
||||
## Archive Complete (with warnings)
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** the archive path derived from `planningHome.changesDir`/<target-name>/
|
||||
**Specs:** Sync skipped (user chose to skip)
|
||||
|
||||
**Warnings:**
|
||||
- Archived with 2 incomplete artifacts
|
||||
- Archived with 3 incomplete tasks
|
||||
- Delta spec sync was skipped (user chose to skip)
|
||||
|
||||
Review the archive if this was not intentional.
|
||||
```
|
||||
|
||||
**Output On Error (Archive Exists)**
|
||||
|
||||
```markdown
|
||||
## Archive Failed
|
||||
|
||||
**Change:** <change-name>
|
||||
**Target:** the archive path derived from `planningHome.changesDir`/<target-name>/
|
||||
|
||||
Target archive directory already exists.
|
||||
|
||||
**Options:**
|
||||
1. Rename the existing archive
|
||||
2. Delete the existing archive if it's a duplicate
|
||||
3. Wait until a different date to archive
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Announce the selected change; prompt for selection when it is ambiguous
|
||||
- Use artifact graph (openspec status --json) for completion checking
|
||||
- Don't block archive on warnings - just inform and confirm
|
||||
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
|
||||
- Show clear summary of what happened
|
||||
- If sync is requested, run the `/opsx:sync` workflow inline (agent-driven)
|
||||
- Never archive while a spec sync is still in flight — run the sync inline and verify the main specs before moving `changeRoot`
|
||||
- If delta specs exist, always run the sync assessment and show the combined summary before prompting
|
||||
- Apply relevant runtime context and report conflicts; operation guidance remains advisory
|
||||
- Consider every guidance entry and explain any inapplicable or conflicting advice
|
||||
- Existing CLI checks, resolved paths, prompts, and command contracts are unchanged
|
||||
- Artifact rules constrain only the specs being written and are never operation guidance
|
||||
- Never copy runtime context, operation guidance, or artifact-rule text verbatim into output files
|
||||
@@ -0,0 +1,193 @@
|
||||
---
|
||||
name: "OPSX: Explore"
|
||||
description: "Enter explore mode - think through ideas, investigate problems, clarify requirements"
|
||||
allowed-tools: Bash(openspec:*)
|
||||
category: "Workflow"
|
||||
tags: ["workflow", "explore", "experimental", "thinking"]
|
||||
---
|
||||
|
||||
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
|
||||
|
||||
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing. For a new change, scaffold it first as described below.
|
||||
|
||||
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
|
||||
|
||||
**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store <id>` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `view`). Once selected, treat `--store <id>` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "<name>" --json --store "<id>"`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root.
|
||||
|
||||
**Input**: The argument after `/opsx:explore` is whatever the user wants to think about. Could be:
|
||||
- A vague idea: "real-time collaboration"
|
||||
- A specific problem: "the auth system is getting unwieldy"
|
||||
- A change name: "add-dark-mode" (to explore in context of that change)
|
||||
- A comparison: "postgres vs sqlite for this"
|
||||
- Nothing (just enter explore mode)
|
||||
|
||||
---
|
||||
|
||||
## The Stance
|
||||
|
||||
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
|
||||
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
|
||||
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
|
||||
- **Adaptive** - Follow interesting threads, pivot when new information emerges
|
||||
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
|
||||
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
|
||||
|
||||
---
|
||||
|
||||
## What You Might Do
|
||||
|
||||
Depending on what the user brings, you might:
|
||||
|
||||
**Explore the problem space**
|
||||
- Ask clarifying questions that emerge from what they said
|
||||
- Challenge assumptions
|
||||
- Reframe the problem
|
||||
- Find analogies
|
||||
|
||||
**Investigate the codebase**
|
||||
- Map existing architecture relevant to the discussion
|
||||
- Find integration points
|
||||
- Identify patterns already in use
|
||||
- Surface hidden complexity
|
||||
|
||||
**Compare options**
|
||||
- Brainstorm multiple approaches
|
||||
- Build comparison tables
|
||||
- Sketch tradeoffs
|
||||
- Recommend a path (if asked)
|
||||
|
||||
**Visualize**
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Use ASCII diagrams liberally │
|
||||
├─────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌────────┐ ┌────────┐ │
|
||||
│ │ State │────────▶│ State │ │
|
||||
│ │ A │ │ B │ │
|
||||
│ └────────┘ └────────┘ │
|
||||
│ │
|
||||
│ System diagrams, state machines, │
|
||||
│ data flows, architecture sketches, │
|
||||
│ dependency graphs, comparison tables │
|
||||
│ │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Surface risks and unknowns**
|
||||
- Identify what could go wrong
|
||||
- Find gaps in understanding
|
||||
- Suggest spikes or investigations
|
||||
|
||||
---
|
||||
|
||||
## OpenSpec Awareness
|
||||
|
||||
You have full context of the OpenSpec system. Use it naturally, don't force it.
|
||||
|
||||
### Check for context
|
||||
|
||||
At the start, quickly check what exists:
|
||||
```bash
|
||||
openspec list --json
|
||||
```
|
||||
|
||||
This tells you:
|
||||
- If there are active changes
|
||||
- Their names, schemas, and status
|
||||
- What the user might be working on
|
||||
|
||||
Then read the project's own context from the resolved root - `<root.path>/openspec/config.yaml` (or `config.yml`). Use the `root.path` returned above, and skip this if neither file exists:
|
||||
- `context`: project background - tech stack, conventions, constraints
|
||||
- `rules`: keyed by artifact id - the entries for an artifact apply only when you write that artifact
|
||||
|
||||
Ground your thinking in these. They are constraints for you to follow, not content to reproduce: do NOT copy them into the conversation or into any artifact you create.
|
||||
|
||||
If the user mentioned a specific change name, read its artifacts for context.
|
||||
|
||||
### When no change exists
|
||||
|
||||
Think freely. When insights crystallize, you might offer:
|
||||
|
||||
- "This feels solid enough to start a change. Want me to create a proposal?"
|
||||
- Or keep exploring - no pressure to formalize
|
||||
|
||||
If the user asks you to capture the exploration as a new change, transition seamlessly into the requested capture:
|
||||
|
||||
1. Run `openspec new change "<name>"` (with `--store <id>` when applicable) before creating any artifacts. Never create a new change directory under `openspec/changes/` by hand; the CLI scaffold creates required metadata such as `.openspec.yaml`. Keep the selected `--store <id>` on every applicable follow-up `status` and `instructions` command.
|
||||
2. Run `openspec status --change "<name>" --json` (append the confirmed `--store "<id>"` only for a registered standalone store), then process the requested artifacts in dependency order. For each requested artifact that is `ready`, run `openspec instructions "<artifact-id>" --change "<name>" --json` (append the confirmed `--store "<id>"` only for a registered standalone store). Before creating a requested artifact, evaluate any condition in its own `instruction` against the explored change; record a deliberate skip instead when the condition does not apply. If a requested artifact is blocked by a direct prerequisite the user did not request, run `openspec instructions "<prerequisite-id>" --change "<name>" --json` (append the confirmed `--store "<id>"` only for a registered standalone store) for that prerequisite whether it is `ready` or `blocked`. If its own `instruction` states a condition, evaluate that condition against the explored change and record a deliberate skip only when the condition does not apply. If the condition applies, or the prerequisite is not conditional, treat it as a normal prerequisite and ask before expanding the capture. Do not create an unrequested prerequisite unless the user approves.
|
||||
3. Follow the returned `template` and `instruction` fields. Read completed dependency files listed in `dependencies`, and apply `context` and `rules` as constraints without copying them into the artifact. If the instruction delegates creation to a specific skill or command, invoke it; otherwise write the artifact to `resolvedOutputPath`, using the instruction to choose a concrete path when it is a glob. Verify that the selected concrete output exists.
|
||||
4. After creating each artifact, re-run `openspec status --change "<name>" --json` (append the confirmed `--store "<id>"` only for a registered standalone store) and continue until every requested artifact is `done`, `skipped`, or was deliberately skipped because its own `instruction` stated a condition that did not apply. Tell the user about a deliberate conditional skip, remember it, and do not reconsider it. Dependencies are enablers, not gates: if a requested artifact is still `blocked` only because you deliberately skipped a conditional prerequisite, run `openspec instructions "<artifact-id>" --change "<name>" --json` (append the confirmed `--store "<id>"` only for a registered standalone store) despite the blocked status, then create it using step 3 only when those recorded conditional skips are its sole missing dependencies. If a requested artifact is blocked by a prerequisite the user did not ask to capture and cannot be conditionally skipped, explain that dependency and ask before expanding the capture.
|
||||
|
||||
Capture the artifact(s) the user requested without asking them to invoke another workflow command. If they asked only to start a change, stop after scaffolding and show its status.
|
||||
|
||||
### When a change exists
|
||||
|
||||
If the user mentions a change or you detect one is relevant:
|
||||
|
||||
1. **Resolve and read existing artifacts for context**
|
||||
- Run `openspec status --change "<name>" --json`.
|
||||
- Use `changeRoot`, `artifactPaths`, and `actionContext` from the status JSON.
|
||||
- Read existing files from `artifactPaths.<artifact>.existingOutputPaths`.
|
||||
|
||||
2. **Reference them naturally in conversation**
|
||||
- "Your design mentions using Redis, but we just realized SQLite fits better..."
|
||||
- "The proposal scopes this to premium users, but we're now thinking everyone..."
|
||||
|
||||
3. **Offer to capture when decisions are made**
|
||||
|
||||
`<capability-path>` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve an existing capability's full path and follow the project's established organization for new capabilities.
|
||||
|
||||
| Insight Type | Where to Capture |
|
||||
|----------------------------|-------------------------------------|
|
||||
| New requirement discovered | `specs/<capability-path>/spec.md` |
|
||||
| Requirement changed | `specs/<capability-path>/spec.md` |
|
||||
| Design decision made | `design.md` |
|
||||
| Scope changed | `proposal.md` |
|
||||
| New work identified | `tasks.md` |
|
||||
| Assumption invalidated | Relevant artifact |
|
||||
|
||||
Example offers:
|
||||
- "That's a design decision. Capture it in design.md?"
|
||||
- "This is a new requirement. Add it to specs?"
|
||||
- "This changes scope. Update the proposal?"
|
||||
|
||||
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
|
||||
|
||||
---
|
||||
|
||||
## What You Don't Have To Do
|
||||
|
||||
- Follow a script
|
||||
- Ask the same questions every time
|
||||
- Produce a specific artifact
|
||||
- Reach a conclusion
|
||||
- Stay on topic if a tangent is valuable
|
||||
- Be brief (this is thinking time)
|
||||
|
||||
---
|
||||
|
||||
## Ending Discovery
|
||||
|
||||
There's no required ending. Discovery might:
|
||||
|
||||
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
|
||||
- **Result in artifact updates**: "Updated design.md with these decisions"
|
||||
- **Just provide clarity**: User has what they need, moves on
|
||||
- **Continue later**: "We can pick this up anytime"
|
||||
|
||||
When things crystallize, you might offer a summary - but it's optional. Sometimes the thinking IS the value.
|
||||
|
||||
---
|
||||
|
||||
## Guardrails
|
||||
|
||||
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
|
||||
- **Don't fake understanding** - If something is unclear, dig deeper
|
||||
- **Don't rush** - Discovery is thinking time, not task time
|
||||
- **Don't force structure** - Let patterns emerge naturally
|
||||
- **Don't auto-capture** - Offer to save insights, don't just do it
|
||||
- **Don't manually scaffold changes** - Never create a new change directory under `openspec/changes/` by hand. Always use `openspec new change "<name>"` (with `--store <id>` when applicable) so required metadata such as `.openspec.yaml` is created before writing artifacts.
|
||||
- **Do visualize** - A good diagram is worth many paragraphs
|
||||
- **Do explore the codebase** - Ground discussions in reality
|
||||
- **Do question assumptions** - Including the user's and your own
|
||||
@@ -0,0 +1,145 @@
|
||||
---
|
||||
name: "OPSX: Propose"
|
||||
description: "Propose a new change - create it and generate all artifacts in one step"
|
||||
allowed-tools: Bash(openspec:*)
|
||||
category: "Workflow"
|
||||
tags: ["workflow", "artifacts", "experimental"]
|
||||
---
|
||||
|
||||
Propose a new change - create the change and generate all artifacts in one step.
|
||||
|
||||
**Planning boundary**: This workflow creates planning artifacts only. The user request that selected or triggered this workflow authorizes planning only, even if it asks to build or fix something. Do not edit project code. After the planning artifacts are complete, stop. Do not start implementation in the same response, even if the initial request asks for it. Wait for a new user request after the artifacts are presented; then start the apply workflow.
|
||||
|
||||
I'll create a change with the artifacts your schema defines. With the default spec-driven schema that is:
|
||||
- proposal.md (what & why)
|
||||
- `specs/<capability-path>/spec.md` (what the system must do - a delta, not the main spec)
|
||||
- design.md (how)
|
||||
- tasks.md (implementation steps)
|
||||
|
||||
`<capability-path>` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve an existing capability's full path and follow the project's established organization for new capabilities.
|
||||
|
||||
When the user is ready to implement, they must start the apply workflow explicitly.
|
||||
|
||||
---
|
||||
|
||||
**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store <id>` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `view`). Once selected, treat `--store <id>` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "<name>" --json --store "<id>"`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root.
|
||||
|
||||
**Input**: The argument after `/opsx:propose` is the change name (kebab-case), OR a description of what the user wants to build.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Understand the request and clarify material ambiguity**
|
||||
|
||||
If no input is provided, ask the user (open-ended, no preset options):
|
||||
> "What change do you want to work on? Describe what you want to build or fix."
|
||||
|
||||
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
|
||||
|
||||
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
|
||||
|
||||
If the request contains ambiguity that would materially affect scope, externally observable behavior, compatibility, or acceptance criteria, ask the user before creating the change. For minor details, make a reasonable assumption and record it in the planning artifacts.
|
||||
|
||||
2. **Determine the workflow schema**
|
||||
|
||||
Use the configured default schema unless the user explicitly requests a different workflow.
|
||||
|
||||
**Use a different schema only if the user:**
|
||||
- Explicitly requests a specific schema by name → use `--schema <schema-name>`
|
||||
- Asks to "show workflows" or asks "what workflows" exist → resolve the authoritative root by running `openspec context --json` from the current working directory. If the user explicitly selected a registered store, use `openspec context --json --store "<store-id>"`. Then run `openspec schemas --json` with its working directory set to the returned `root.path` and let them choose. This preserves roots selected by a local `store:` pointer or the global `defaultStore`; `schemas` does not accept `--store`. If context reports only `no_openspec_root`, run `openspec schemas --json` from the current working directory instead. Do not use this fallback for invalid or unavailable stores.
|
||||
|
||||
Otherwise, omit `--schema` to preserve the configured default.
|
||||
|
||||
3. **Create the change directory**
|
||||
|
||||
Choose one schema form below. If a registered store is selected, append `--store "<store-id>"` to that command and each later OpenSpec command shown below that accepts `--store`.
|
||||
|
||||
Using the configured default:
|
||||
```bash
|
||||
openspec new change "<name>"
|
||||
```
|
||||
|
||||
Using an explicitly requested schema:
|
||||
```bash
|
||||
openspec new change "<name>" --schema "<schema-name>"
|
||||
```
|
||||
This creates a scaffolded change in the planning home resolved by the CLI with `.openspec.yaml`.
|
||||
|
||||
4. **Get the artifact build order**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to get:
|
||||
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
|
||||
- `artifacts`: list of all artifacts, each with its `status` and its `requires` edges (the artifact IDs it directly depends on)
|
||||
- `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths.
|
||||
|
||||
5. **Create every artifact in the required set**
|
||||
|
||||
Use a todo list to track progress through the artifacts.
|
||||
|
||||
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
|
||||
|
||||
a. **For each artifact that is `ready` (dependencies satisfied)**:
|
||||
- Get instructions:
|
||||
```bash
|
||||
openspec instructions <artifact-id> --change "<name>" --json
|
||||
```
|
||||
- The instructions JSON includes:
|
||||
- `context`: Project background (constraints for you - do NOT include in output)
|
||||
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
|
||||
- `template`: The structure to use for your output file
|
||||
- `instruction`: Schema-specific guidance for this artifact type
|
||||
- `skipped`/`warning`: present when the change declares skip_specs and this artifact must NOT be created - stop and pick another artifact
|
||||
- `resolvedOutputPath`: Resolved path or pattern to write the artifact
|
||||
- `dependencies`: Completed artifacts to read for context
|
||||
- Read any completed dependency files for context - always re-read them from disk, even if you saw them earlier in the conversation (the user may have edited them)
|
||||
- If the `instruction` field delegates creation to a specific skill or command, invoke it to produce the artifact instead of writing the file yourself, then verify the artifact file exists at `resolvedOutputPath`
|
||||
- Otherwise create the artifact file using `template` as the structure and write it to `resolvedOutputPath`. If `resolvedOutputPath` is a glob, follow `instruction` to choose the concrete file path
|
||||
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
|
||||
- Show brief progress: "Created <artifact-id>"
|
||||
|
||||
b. **Continue until every artifact in the required set exists (not just `apply.requires`)**
|
||||
- After creating each artifact, re-run `openspec status --change "<name>" --json`
|
||||
- The required set is `applyRequires` plus every artifact reachable from those by following the `requires` edges in `status --json` - walk them transitively (spec-driven closes over proposal, specs, design, tasks). Leave artifacts outside that set alone
|
||||
- `status` is file-existence only, so an `applyRequires` artifact reading `done` does NOT mean its dependencies exist - writing `tasks.md` early marks `tasks` done while `specs` was never written. Use each artifact's `requires` edges, not its `status`, to build the required set: a `done` artifact still lists what it depends on
|
||||
- An artifact already reading `status: "skipped"` is satisfied: the change declares `skip_specs` in `.openspec.yaml`, so its files must NOT exist. Never try to create one
|
||||
- Create every artifact in the required set that is missing, then re-check - creating one can unblock others
|
||||
- Skip one only when `status` already reports it `skipped`, or when its own `instruction` says it is conditional: run `openspec instructions <artifact-id> --change "<name>" --json` and skip only if its `instruction` field marks it optional (e.g. "create only if..."). Spec-driven's `design.md` qualifies; `specs` qualifies only via the `skipped` status above, never by your own judgment. Tell the user, and do not reconsider it
|
||||
- Dependencies are enablers, not gates: if a required artifact is still `blocked` only because you skipped a conditional dependency, write it anyway
|
||||
- Stop when every artifact in the required set is `done`, `skipped`, or was deliberately skipped
|
||||
|
||||
c. **If an artifact requires user input** (unclear context):
|
||||
- Ask the user to clarify
|
||||
- Then continue with creation
|
||||
|
||||
6. **Show final status**
|
||||
```bash
|
||||
openspec status --change "<name>"
|
||||
```
|
||||
|
||||
**Output**
|
||||
|
||||
After completing all artifacts, summarize:
|
||||
- Change name and location
|
||||
- List of artifacts created with brief descriptions, plus any conditional artifact you skipped and why
|
||||
- What's ready: "All artifacts needed for implementation are ready."
|
||||
- Prompt: "The artifacts are ready for review. When you are ready, run `/opsx:apply`."
|
||||
|
||||
**Artifact Creation Guidelines**
|
||||
|
||||
- Follow the `instruction` field from `openspec instructions` for each artifact type - it is the authoritative guidance, even for familiar artifact names
|
||||
- If the `instruction` field directs you to use a specific skill or command to create the artifact, invoke it instead of writing the artifact directly
|
||||
- The schema defines what each artifact should contain - follow it
|
||||
- Read dependency artifacts for context before creating new ones
|
||||
- Use `template` as the structure for your output file - fill in its sections
|
||||
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
|
||||
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
|
||||
- These guide what you write, but should never appear in the output
|
||||
|
||||
**Guardrails**
|
||||
- The request that invoked this workflow authorizes planning only. Any implementation or apply instruction in that request does not carry forward. Do NOT implement the change, start the apply workflow, or edit project code during this workflow. After presenting the artifacts, stop and wait for a new user request to start the apply workflow
|
||||
- Create every artifact the apply phase transitively depends on, not just the ids listed in `apply.requires`
|
||||
- Always read dependency artifacts before creating a new one - re-read from disk, not from conversation memory (files may have changed since you last saw them)
|
||||
- Ask about ambiguities that would materially change scope, externally observable behavior, compatibility, or acceptance criteria; for minor details, make reasonable assumptions and record them
|
||||
- If a change with that name already exists, ask if user wants to continue it or create a new one
|
||||
- Verify each artifact file exists after writing before proceeding to next
|
||||
@@ -0,0 +1,258 @@
|
||||
---
|
||||
name: "OPSX: Sync"
|
||||
description: "Sync delta specs from a change to main specs"
|
||||
allowed-tools: Bash(openspec:*)
|
||||
category: "Workflow"
|
||||
tags: ["workflow", "specs", "experimental"]
|
||||
---
|
||||
|
||||
Sync delta specs from a change to main specs.
|
||||
|
||||
This is an **agent-driven** operation - you will read delta specs and directly edit main specs to apply the changes. This allows intelligent merging (e.g., adding a scenario without copying the entire requirement).
|
||||
|
||||
**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store <id>` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `view`). Once selected, treat `--store <id>` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "<name>" --json --store "<id>"`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root.
|
||||
|
||||
`<capability-path>` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path from each delta spec when resolving its main spec.
|
||||
|
||||
**Input**: Optionally specify a change name after `/opsx:sync` (e.g., `/opsx:sync add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes and ask the user to select one
|
||||
|
||||
When prompting, show changes that have delta specs (under `specs/` directory).
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:sync <other>`).
|
||||
|
||||
2. **Resolve change context**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
|
||||
The JSON includes `planningHome.root`. Main specs live under `<planningHome.root>/openspec/specs/` — use that (store-aware) root for every main-spec path below, not a hardcoded repo path. When a store is selected it points at the store, not the current repository.
|
||||
|
||||
3. **Find delta specs**
|
||||
|
||||
Use `artifactPaths.specs.existingOutputPaths` from the status JSON as the
|
||||
only source of delta spec paths. If the `specs` entry is missing or
|
||||
`existingOutputPaths` is empty, report that there are no delta specs to sync,
|
||||
do not infer them from other artifacts, and stop without requesting artifact
|
||||
instructions or writing a main spec.
|
||||
|
||||
Sync every path in `existingOutputPaths` unless the caller narrowed the set.
|
||||
A caller narrows it by naming an explicit list of complete entries from
|
||||
`existingOutputPaths` — copy those absolute values verbatim. Archive does
|
||||
this inline, and a user can too (for example, by selecting the entry ending
|
||||
in `/specs/billing/invoices/spec.md`).
|
||||
Then sync only the named paths and leave the remaining delta specs untouched:
|
||||
bulk archive excludes a delta whose implementation it could not find, and
|
||||
syncing it anyway would write a main spec the caller deliberately withheld.
|
||||
Carry that narrowed selection through step 4; never widen it back to the full
|
||||
list. If a named path is not in `existingOutputPaths`, do not sync it —
|
||||
report it and stop, rather than dropping it silently. If the named list is
|
||||
empty, report that there is nothing to sync and stop without writing a main
|
||||
spec.
|
||||
|
||||
Each delta spec file contains sections like:
|
||||
- `## ADDED Requirements` - New requirements to add
|
||||
- `## MODIFIED Requirements` - Changes to existing requirements
|
||||
- `## REMOVED Requirements` - Requirements to remove
|
||||
- `## RENAMED Requirements` - Requirements to rename (FROM:/TO: format)
|
||||
|
||||
If no delta specs found, inform user and stop.
|
||||
|
||||
4. **For each delta spec, apply changes to main specs**
|
||||
|
||||
Before the first main-spec write, obtain one current specs-rule snapshot:
|
||||
- If archive invoked this workflow inline and supplied a valid snapshot from
|
||||
`openspec instructions specs --change "<name>" --json`, reuse it and do not
|
||||
fetch the same instructions again.
|
||||
- Otherwise run that command once now with the same selected-root flags.
|
||||
- If the direct lookup exits non-zero or returns invalid artifact-instruction
|
||||
JSON, report the error and stop before writing any main spec. Do not treat the
|
||||
failure as an absent rule set.
|
||||
- A valid response with omitted `rules` means no artifact rules are configured
|
||||
and the existing semantic merge continues.
|
||||
|
||||
Apply returned `rules` only to the content and form of the main specs produced
|
||||
by this merge. Artifact rules are not operation guidance and cannot change
|
||||
selected roots, delta paths, CLI checks, or workflow steps. Use their text as
|
||||
constraints without copying it verbatim into a main spec or summary.
|
||||
|
||||
For each capability delta spec path selected in step 3 — the full `existingOutputPaths` list, or the narrowed subset when a caller supplied one (these may belong to a selected store, not the repo):
|
||||
|
||||
a. **Read the delta spec** to understand the intended changes
|
||||
|
||||
b. **Read the main spec** at `<planningHome.root>/openspec/specs/<capability-path>/spec.md` (may not exist yet)
|
||||
|
||||
c. **Apply changes intelligently**:
|
||||
|
||||
**ADDED Requirements:**
|
||||
- If requirement doesn't exist in main spec → add it
|
||||
- If requirement already exists → update it to match (treat as implicit MODIFIED)
|
||||
|
||||
**MODIFIED Requirements:**
|
||||
- Find the requirement in main spec
|
||||
- Apply the changes - this can be:
|
||||
- Adding new scenarios the main spec does not have yet
|
||||
- Modifying existing scenarios
|
||||
- Changing the requirement description
|
||||
- Preserve scenarios/content not mentioned in the delta
|
||||
|
||||
**REMOVED Requirements:**
|
||||
- Remove the entire requirement block from main spec
|
||||
- Retiring the capability. Delete the whole `spec.md` - and the directory once
|
||||
nothing else is left in it - only when ALL of these hold:
|
||||
1. removing the requirements *this run* left no requirement blocks;
|
||||
2. the rest of the spec is well-formed (it still has a `## Purpose`);
|
||||
3. the main spec was not already empty before this sync - if you removed
|
||||
nothing, change nothing;
|
||||
4. every other nonblank line in the whole file is accounted for as the
|
||||
title, Purpose, Requirements header, or a canonical requirement's
|
||||
statement, scenarios, or fenced examples;
|
||||
5. the change's `.openspec.yaml` declares `retire_capabilities: true`;
|
||||
6. the `spec.md` resolves inside the real specs root (do not follow a
|
||||
capability-directory symlink to delete an external file).
|
||||
If removing the selected requirements would leave no requirement blocks and
|
||||
any retirement condition is not satisfied, do not modify the main spec. Stop
|
||||
the sync for that capability, report the blocking condition, and tell the user
|
||||
how to resolve it. Never write or leave an empty `## Requirements` section.
|
||||
When only the marker is missing, say that too - it is the one thing the user
|
||||
can add to make the retirement go through.
|
||||
- Deleting the file also deletes its `## Purpose`; any other section blocks
|
||||
retirement. Name Purpose when you report the retirement. Include a pasteable
|
||||
`git checkout` only when the spec lived in the caller's checkout;
|
||||
otherwise give checkout-scoped recovery guidance.
|
||||
|
||||
**RENAMED Requirements:**
|
||||
- Find the FROM requirement, rename to TO
|
||||
|
||||
**`## Purpose` in the delta:**
|
||||
- The main spec already has one and it is authoritative - leave it alone
|
||||
(this is what `openspec archive` does; it warns and moves on)
|
||||
|
||||
d. **Create new main spec** if capability doesn't exist yet:
|
||||
- Create `<planningHome.root>/openspec/specs/<capability-path>/spec.md`
|
||||
- Add Purpose section: copy the delta's `## Purpose` body verbatim when it has one
|
||||
(this is what `openspec archive` does); only write a brief TBD placeholder when it does not
|
||||
- Add Requirements section with the ADDED requirements
|
||||
- Follow the **Main Spec Format Reference** below
|
||||
|
||||
5. **Validate updated main specs**
|
||||
|
||||
Run `openspec validate --specs` with the same selected-root flags used earlier.
|
||||
If validation fails, report the problems and do not claim the sync succeeded.
|
||||
|
||||
6. **Show summary**
|
||||
|
||||
After applying all changes, summarize:
|
||||
- Which capabilities were updated
|
||||
- What changes were made (requirements added/modified/removed/renamed)
|
||||
- Any new main spec left with a TBD Purpose placeholder, so it gets written
|
||||
now rather than lingering
|
||||
- Any capability retired, naming the deleted `spec.md`, its Purpose, and
|
||||
either a pasteable `git checkout` or checkout-scoped recovery guidance
|
||||
|
||||
**Delta Spec Format Reference**
|
||||
|
||||
```markdown
|
||||
## Purpose
|
||||
|
||||
Only on a delta that introduces a brand-new capability. Seeds the new main spec.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: New Feature
|
||||
The system SHALL do something new.
|
||||
|
||||
#### Scenario: Basic case
|
||||
- **WHEN** user does X
|
||||
- **THEN** system does Y
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Existing Feature
|
||||
The system SHALL keep doing the existing thing, now also handling A.
|
||||
|
||||
#### Scenario: Scenario the main spec already has
|
||||
- **WHEN** user does X
|
||||
- **THEN** system does Y
|
||||
|
||||
#### Scenario: New scenario to add
|
||||
- **WHEN** user does A
|
||||
- **THEN** system does B
|
||||
|
||||
## REMOVED Requirements
|
||||
|
||||
### Requirement: Deprecated Feature
|
||||
|
||||
## RENAMED Requirements
|
||||
|
||||
- FROM: `### Requirement: Old Name`
|
||||
- TO: `### Requirement: New Name`
|
||||
```
|
||||
|
||||
**Main Spec Format Reference**
|
||||
|
||||
Main specs are what the delta merges INTO. They must never contain delta operation headers (`## ADDED/MODIFIED/REMOVED/RENAMED Requirements`) - after syncing, every requirement lives under a single `## Requirements` section:
|
||||
|
||||
```markdown
|
||||
# <capability> Specification
|
||||
|
||||
## Purpose
|
||||
Short description of what this capability does and why it exists.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: New Feature
|
||||
The system SHALL do something new.
|
||||
|
||||
#### Scenario: Basic case
|
||||
- **WHEN** user does X
|
||||
- **THEN** system does Y
|
||||
```
|
||||
|
||||
**Key Principle: Intelligent Merging**
|
||||
|
||||
Unlike programmatic merging, you merge rather than overwrite:
|
||||
- A MODIFIED block carries the whole requirement - body plus every scenario that survives the change. `openspec validate` and `openspec archive` both reject one that drops a scenario the main spec still has.
|
||||
- Keep anything the delta does not mention, in the main spec's existing order
|
||||
- Use your judgment to merge changes sensibly
|
||||
|
||||
**Output On Success**
|
||||
|
||||
```markdown
|
||||
## Specs Synced: <change-name>
|
||||
|
||||
Updated main specs:
|
||||
|
||||
**<capability-1>**:
|
||||
- Added requirement: "New Feature"
|
||||
- Modified requirement: "Existing Feature" (added 1 scenario)
|
||||
|
||||
**<capability-2>**:
|
||||
- Created new spec file
|
||||
- Added requirement: "Another Feature"
|
||||
|
||||
Main specs are now updated. The change remains active - archive when implementation is complete.
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Read both delta and main specs before making changes
|
||||
- Preserve existing content not mentioned in delta
|
||||
- Never copy a delta file into a main spec as-is - merge its content so the main spec keeps the Main Spec Format Reference structure, with no delta operation headers
|
||||
- If something is unclear, ask for clarification
|
||||
- Show what you're changing as you go
|
||||
- The operation should be idempotent - running twice should give same result
|
||||
- Use only `artifactPaths.specs.existingOutputPaths`; never infer delta specs from unrelated artifacts
|
||||
- Honor a caller-supplied subset of `existingOutputPaths`; never widen it back to the full list
|
||||
- Fetch specs instructions once for direct sync, or reuse the archive-supplied snapshot inline
|
||||
- Stop before every main-spec write on a non-zero or invalid JSON specs-instruction response
|
||||
- Artifact rules constrain only the specs being written and are never copied into output files
|
||||
@@ -0,0 +1,87 @@
|
||||
---
|
||||
name: "OPSX: Update"
|
||||
description: "Update a change - revise existing planning artifacts and keep them coherent (Experimental)"
|
||||
allowed-tools: Bash(openspec:*)
|
||||
category: "Workflow"
|
||||
tags: ["workflow", "artifacts", "experimental"]
|
||||
---
|
||||
|
||||
Revise a change's existing planning artifacts and keep them coherent. Never edit code.
|
||||
|
||||
**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store <id>` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `view`). Once selected, treat `--store <id>` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "<name>" --json --store "<id>"`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root.
|
||||
|
||||
**Input**: Optionally specify a change name after `/opsx:update` (e.g., `/opsx:update add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
`/opsx:continue` is an expanded-profile workflow and may not be installed. Before suggesting it anywhere below, verify that it is available. If it is unavailable, `openspec status --change "<name>" --json` shows the next artifact and `openspec instructions "<artifact-id>" --change "<name>" --json` explains how to create it.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes sorted by most recently modified, and ask the user to select one
|
||||
|
||||
When prompting, present the top 3-4 most recently modified changes as options, showing:
|
||||
- Change name
|
||||
- Schema (from `schema` field if present, otherwise "spec-driven")
|
||||
- Status (e.g., "0/5 tasks", "complete", "no tasks")
|
||||
- How recently it was modified (from `lastModified` field)
|
||||
|
||||
Mark the most recently modified change as "(Recommended)" since it's likely what the user wants to update.
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:update <other>`).
|
||||
|
||||
2. **Get the change's artifacts**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to understand current state. The response includes:
|
||||
- `schemaName`: The workflow schema being used (e.g., "spec-driven")
|
||||
- `artifacts`: Array of artifacts with their status ("done", "skipped", "ready", "blocked")
|
||||
- `isPlanningComplete`: Boolean indicating if all planning artifacts are complete. Older CLI versions expose the same value as `isComplete`.
|
||||
- `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths.
|
||||
|
||||
The artifact ids and paths come from the active schema - do NOT assume them, and do NOT branch on hardcoded artifact names. Custom schemas must work unchanged.
|
||||
|
||||
The files to edit are `artifactPaths.<id>.existingOutputPaths` - the concrete files that exist on disk, already glob-expanded for glob artifacts (e.g. `specs/**/*.md`). Do NOT write to `resolvedOutputPath`: for a glob artifact it is still the glob pattern, not a real file.
|
||||
|
||||
3. **Understand the request**
|
||||
- If the user asked for a specific revision ("the design now uses X"), that is the starting edit.
|
||||
- If they only said "update" / "make this coherent", treat it as a coherence review: read the existing artifacts and check them against each other for contradictions, gaps, and duplication.
|
||||
|
||||
4. **Read and reconcile**
|
||||
- Read the artifact(s) the request touches and the change's other existing artifacts.
|
||||
- Apply the requested edit. Then check every other existing artifact against it - in ANY direction: an edit to a later artifact may require revising an earlier one, not only the other way around. Build order is a useful reading order, not a constraint on which artifacts may be revised.
|
||||
- Note everything that is now inconsistent, missing, or contradictory.
|
||||
- Revise only files that already exist (`existingOutputPaths`). Do NOT create artifacts that don't exist yet, and do NOT invent new files under a glob artifact - note them and point the user to `/opsx:continue` to create them.
|
||||
- If the change is already coherent, say so and make no edits.
|
||||
|
||||
5. **Confirm and apply, one artifact at a time**
|
||||
- Show each proposed revision and why. Write only after the user confirms.
|
||||
- If the user rejects a revision, do not write it - leave that artifact unchanged.
|
||||
- When a substantial rewrite is needed, get that artifact's rules and template first:
|
||||
```bash
|
||||
openspec instructions "<artifact-id>" --change "<name>" --json
|
||||
```
|
||||
|
||||
6. **Point to the next step (guidance only - NEVER act on it)**
|
||||
- Artifacts still missing -> suggest `/opsx:continue` to create them.
|
||||
- Change already implemented (tasks checked off / already applied) -> the code may no longer match the revised plan; suggest `/opsx:apply` to carry the delta into code.
|
||||
- Everything done and implemented -> suggest `/opsx:archive`.
|
||||
|
||||
**Output**
|
||||
|
||||
After each invocation, show:
|
||||
- Which artifacts were revised (and which proposed revisions were rejected)
|
||||
- Anything deferred to `/opsx:continue` (not-yet-created artifacts or files)
|
||||
- Where the change stands and the recommended next command
|
||||
|
||||
**Guardrails**
|
||||
- Planning artifacts only - NEVER edit implementation code. If the revised plan implies code changes, stop and point to `/opsx:apply`.
|
||||
- Use the artifact ids and paths reported by `openspec status`; never branch on hardcoded artifact names.
|
||||
- Edit only the concrete files in `existingOutputPaths`; never write to a glob `resolvedOutputPath`.
|
||||
- Do not advance the build frontier: no new artifacts, no new files under glob artifacts - that is `/opsx:continue`'s job.
|
||||
- Confirm every edit with the user before writing.
|
||||
- If the request changes the change's *intent* rather than refining it, first verify whether the expanded-profile `/opsx:new` workflow is available. If it is, recommend starting fresh with `/opsx:new` (the "Update vs. Start Fresh" heuristic). If it is unavailable, ask for a distinct unused change name and recommend `openspec new change "<new-change-name>"` instead.
|
||||
Executable
+45
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env bash
|
||||
set -uo pipefail
|
||||
|
||||
ROOT="${CLAUDE_PROJECT_DIR:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)}"
|
||||
PAYLOAD=$(cat)
|
||||
FILE=$(printf '%s' "$PAYLOAD" | /usr/bin/python3 -c 'import json,sys
|
||||
try:
|
||||
d = json.load(sys.stdin)
|
||||
except Exception:
|
||||
sys.exit(0)
|
||||
t = d.get("tool_input") or {}
|
||||
r = d.get("tool_response") or {}
|
||||
p = r.get("filePath") or t.get("file_path") or ""
|
||||
print(p)
|
||||
' 2>/dev/null)
|
||||
|
||||
case "$FILE" in
|
||||
"") exit 0 ;;
|
||||
*.py) ;;
|
||||
*) exit 0 ;;
|
||||
esac
|
||||
|
||||
case "$FILE" in
|
||||
"$ROOT"/src/*) ;;
|
||||
src/*) FILE="$ROOT/$FILE" ;;
|
||||
*) exit 0 ;;
|
||||
esac
|
||||
|
||||
[ -f "$FILE" ] || exit 0
|
||||
[ -x "$ROOT/.venv/bin/ruff" ] || exit 0
|
||||
|
||||
OUT=$(
|
||||
cd "$ROOT" || exit 0
|
||||
"$ROOT/.venv/bin/ruff" check --fix --quiet "$FILE" 2>&1
|
||||
"$ROOT/.venv/bin/ruff" format --quiet "$FILE" 2>&1
|
||||
if [ -x "$ROOT/.venv/bin/ty" ]; then
|
||||
"$ROOT/.venv/bin/ty" check --quiet "$FILE" 2>&1 | grep -v "^All checks passed" || true
|
||||
fi
|
||||
)
|
||||
|
||||
if [ -n "$OUT" ]; then
|
||||
printf 'ruff/ty по %s:\n%s\n' "${FILE#"$ROOT"/}" "$OUT" >&2
|
||||
exit 2
|
||||
fi
|
||||
exit 0
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"hooks": {
|
||||
"PostToolUse": [
|
||||
{
|
||||
"matcher": "Write|Edit|MultiEdit|NotebookEdit",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/lint-python.sh",
|
||||
"timeout": 60,
|
||||
"statusMessage": "ruff + ty по src/"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,185 @@
|
||||
---
|
||||
name: openspec-apply-change
|
||||
description: Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks.
|
||||
allowed-tools: Bash(openspec:*)
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.8.0"
|
||||
---
|
||||
|
||||
Implement tasks from an OpenSpec change.
|
||||
|
||||
**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store <id>` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `view`). Once selected, treat `--store <id>` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "<name>" --json --store "<id>"`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root.
|
||||
|
||||
**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes and ask the user to select one
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
|
||||
|
||||
2. **Check status to understand the schema**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used (e.g., "spec-driven")
|
||||
- `planningHome`, `changeRoot`, and `actionContext`: planning scope and edit constraints
|
||||
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
|
||||
|
||||
3. **Get apply instructions**
|
||||
|
||||
```bash
|
||||
openspec instructions apply --change "<name>" --json
|
||||
```
|
||||
|
||||
This returns:
|
||||
- `contextFiles`: artifact ID -> array of concrete file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
|
||||
- Progress (total, complete, remaining)
|
||||
- Task list with status
|
||||
- Dynamic instruction based on current state
|
||||
- Optional `context`: current required project instruction input from the selected root
|
||||
- Optional `operationGuidance`: current advisory guidance for apply
|
||||
|
||||
**Handle states:**
|
||||
- If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx:continue` (if it is not installed, run `openspec status --change "<name>" --json` to see the next artifact and `openspec instructions <artifact-id> --change "<name>" --json` for how to create it)
|
||||
- If `state: "all_done"`: congratulate, suggest archive
|
||||
- Otherwise: proceed to implementation
|
||||
|
||||
Treat `context` as a required prompt-level input. Read and consider it, and
|
||||
apply relevant project facts, conventions, and constraints while implementing.
|
||||
Treat `operationGuidance` as optional additive advice. Read and consider every
|
||||
entry, and follow entries that are applicable and compatible with the built-in
|
||||
workflow.
|
||||
|
||||
Keep both fields separate from CLI-returned state, missing artifacts, tasks,
|
||||
progress, `contextFiles`, and the built-in `instruction`. They are not
|
||||
evidence of task completion, do not replace the built-in instruction, and do
|
||||
not permit bypassing a blocked state. If context conflicts with the built-in
|
||||
instruction, an explicit user choice, or a CLI-controlled value, report the
|
||||
conflict and preserve the controlling value. If guidance is inapplicable or
|
||||
conflicts with those controlling inputs, do not follow it and explain why.
|
||||
These are prompt-level behavior contracts, not enforceable checks.
|
||||
|
||||
4. **Read context files**
|
||||
|
||||
Read every file path listed under `contextFiles` from the apply instructions output.
|
||||
The files depend on the schema being used:
|
||||
- **spec-driven**: proposal, specs, design, tasks
|
||||
- Other schemas: follow the contextFiles from CLI output
|
||||
|
||||
Do not copy `context` or `operationGuidance` verbatim into implementation
|
||||
files or planning artifacts unless the user separately asks for that content.
|
||||
|
||||
5. **Show current progress**
|
||||
|
||||
Display:
|
||||
- Schema being used
|
||||
- Progress: "N/M tasks complete"
|
||||
- Remaining tasks overview
|
||||
- Dynamic instruction from CLI
|
||||
|
||||
6. **Implement tasks (loop until done or blocked)**
|
||||
|
||||
For each pending task:
|
||||
- Show which task is being worked on
|
||||
- Make the code changes required
|
||||
- Keep changes minimal and focused
|
||||
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
|
||||
- Continue to next task
|
||||
|
||||
**Pause if:**
|
||||
- Task is unclear → ask for clarification
|
||||
- Implementation reveals a design issue → suggest updating artifacts
|
||||
- Error or blocker encountered → report and wait for guidance
|
||||
- User interrupts
|
||||
|
||||
7. **On completion or pause, show status**
|
||||
|
||||
Display:
|
||||
- Tasks completed this session
|
||||
- Overall progress: "N/M tasks complete"
|
||||
- If all done: suggest archive
|
||||
- If paused: explain why and wait for guidance
|
||||
|
||||
**Output During Implementation**
|
||||
|
||||
```
|
||||
## Implementing: <change-name> (schema: <schema-name>)
|
||||
|
||||
Working on task 3/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
|
||||
Working on task 4/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
```
|
||||
|
||||
**Output On Completion**
|
||||
|
||||
```
|
||||
## Implementation Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 7/7 tasks complete ✓
|
||||
|
||||
### Completed This Session
|
||||
- [x] Task 1
|
||||
- [x] Task 2
|
||||
...
|
||||
|
||||
All tasks complete! You can archive this change with `/opsx:archive`.
|
||||
```
|
||||
|
||||
**Output On Pause (Issue Encountered)**
|
||||
|
||||
```
|
||||
## Implementation Paused
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 4/7 tasks complete
|
||||
|
||||
### Issue Encountered
|
||||
<description of the issue>
|
||||
|
||||
**Options:**
|
||||
1. <option 1>
|
||||
2. <option 2>
|
||||
3. Other approach
|
||||
|
||||
What would you like to do?
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Keep going through tasks until done or blocked
|
||||
- Always read context files before starting (from the apply instructions output)
|
||||
- If task is ambiguous, pause and ask before implementing
|
||||
- If implementation reveals issues, pause and suggest artifact updates
|
||||
- Keep code changes minimal and scoped to each task
|
||||
- Update task checkbox immediately after completing each task
|
||||
- Pause on errors, blockers, or unclear requirements - don't guess
|
||||
- Use contextFiles from CLI output, don't assume specific file names
|
||||
- Do not use context or operation guidance as proof that a task is complete
|
||||
- Apply relevant project context; report conflicts with controlling workflow inputs
|
||||
- Consider every guidance entry; explain any inapplicable or conflicting advice
|
||||
- Do not copy runtime context or operation guidance into implementation files or planning artifacts
|
||||
- Preserve CLI-controlled blocked/ready/all-done behavior and completion criteria
|
||||
|
||||
**Fluid Workflow Integration**
|
||||
|
||||
This skill supports the "actions on a change" model:
|
||||
|
||||
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
|
||||
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly
|
||||
@@ -0,0 +1,182 @@
|
||||
---
|
||||
name: openspec-archive-change
|
||||
description: Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete.
|
||||
allowed-tools: Bash(openspec:*)
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.8.0"
|
||||
---
|
||||
|
||||
Archive a completed change in the experimental workflow.
|
||||
|
||||
**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store <id>` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `view`). Once selected, treat `--store <id>` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "<name>" --json --store "<id>"`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root.
|
||||
|
||||
`<capability-path>` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path from each delta spec when resolving its main spec.
|
||||
|
||||
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes and ask the user to select one
|
||||
|
||||
When prompting, show only active changes (not already archived).
|
||||
Include the schema used for each change if available.
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:archive <other>`).
|
||||
|
||||
**Load current archive inputs before the existing archive checks:**
|
||||
|
||||
After resolving the selected change and planning root, run:
|
||||
```bash
|
||||
openspec instructions archive --change "<name>" --json
|
||||
```
|
||||
Keep the same selected-root flags on this command. This lookup is advisory and
|
||||
optional: it only supplies extra prompt inputs, so it must never block archiving.
|
||||
If it exits non-zero or returns invalid JSON — for example on an older CLI that
|
||||
does not support this command yet — continue the archive workflow with no
|
||||
context and no operation guidance. Do not report an error and do not stop.
|
||||
|
||||
A successful response may omit both optional fields. Treat `context` as a
|
||||
required prompt-level input: read and consider it, and apply relevant project
|
||||
facts, conventions, and constraints. Treat `operationGuidance` as optional
|
||||
additive advice: read and consider every entry, and follow entries that are
|
||||
applicable and compatible with the built-in archive workflow.
|
||||
|
||||
Keep both fields separate from built-in steps, explicit user choices, resolved
|
||||
paths, CLI checks, and command contracts. If context conflicts with one of those
|
||||
controlling inputs, report the conflict and preserve the controlling value. If
|
||||
guidance is inapplicable or conflicts with a controlling input, do not follow it
|
||||
and explain why. Do not infer replacement paths, skipped prompts, or flags from
|
||||
either field, and do not copy their text verbatim into specs, change artifacts,
|
||||
or archive summaries unless the user separately asks for it. These are
|
||||
prompt-level behavior contracts, not enforceable checks.
|
||||
|
||||
2. **Check artifact completion status**
|
||||
|
||||
Run `openspec status --change "<name>" --json` to check artifact completion.
|
||||
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used
|
||||
- `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context
|
||||
- `artifacts`: List of artifacts with their status (`done`, `skipped`, or other)
|
||||
|
||||
**If any artifacts are neither `done` nor `skipped`** (skipped artifacts satisfy the requirement - the change declares skip_specs):
|
||||
- Display warning listing incomplete artifacts
|
||||
- Ask the user to confirm they want to proceed
|
||||
- Proceed if user confirms
|
||||
|
||||
3. **Check task completion status**
|
||||
|
||||
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
|
||||
|
||||
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
|
||||
|
||||
**If incomplete tasks found:**
|
||||
- Display warning showing count of incomplete tasks
|
||||
- Ask the user to confirm they want to proceed
|
||||
- Proceed if user confirms
|
||||
|
||||
**If no tasks file exists:** Proceed without task-related warning.
|
||||
|
||||
4. **Assess delta spec sync state**
|
||||
|
||||
Use `artifactPaths.specs.existingOutputPaths` from status JSON as the only
|
||||
delta-spec source. If the `specs` entry is missing or
|
||||
`existingOutputPaths` is empty, proceed without a sync prompt and do not infer
|
||||
delta specs from other artifacts.
|
||||
|
||||
**If delta specs exist:**
|
||||
- Compare each delta spec with its corresponding main spec at `<planningHome.root>/openspec/specs/<capability-path>/spec.md` (use the store-aware `planningHome.root` from step 2, not a hardcoded repo path)
|
||||
- Determine what changes would be applied (adds, modifications, removals, renames)
|
||||
- Show a combined summary before prompting
|
||||
|
||||
**Prompt options:**
|
||||
- If changes needed: "Sync now (recommended)", "Archive without syncing"
|
||||
- If already synced: "Archive now", "Sync anyway", "Cancel"
|
||||
|
||||
Route on the answer:
|
||||
- "Cancel" — stop, do not archive
|
||||
- "Archive without syncing" or "Archive now" — proceed to archive
|
||||
- "Sync now" or "Sync anyway" — sync, then verify (below)
|
||||
- Anything else — ask again rather than archiving
|
||||
|
||||
Before a selected sync writes any main spec, run
|
||||
`openspec instructions specs --change "<name>" --json` once with the same
|
||||
selected-root flags. Require a zero exit status and valid artifact-instruction
|
||||
JSON. If the lookup fails or returns invalid JSON, report the error and stop
|
||||
before writing any main spec or moving the change. A valid response with omitted
|
||||
`rules` is the no-rules case. Apply returned `rules` only to the content and
|
||||
form of main specs produced by this merge; do not use them as archive guidance,
|
||||
change CLI behavior, or copy the rule text into any output file.
|
||||
|
||||
Then run the `openspec-sync-specs` workflow inline (agent-driven intelligent merge) for change '<name>', passing the delta spec analysis and the fetched specs-rule snapshot from above, and wait for it to finish. The inline sync must reuse that snapshot without fetching `specs` instructions again. Do not delegate it to a background task — step 5 would move `changeRoot` out from under a sync that is still reading it, leaving the change archived and the main specs never updated. If your agent can only run it by delegation, delegate synchronously and wait for the result.
|
||||
|
||||
Then re-run the comparison from the top of this step against every capability that has a delta spec in `artifactPaths.specs.existingOutputPaths` — not only the ones the sync reports it touched. A successful sync leaves nothing left to apply, so each capability must now read as already synced:
|
||||
- ADDED requirements present
|
||||
- MODIFIED requirements carrying the scenario and description changes named in the delta, with their other scenarios intact
|
||||
- REMOVED requirements gone — and where this sync retired a capability (removed its last requirement, leaving `## Requirements` empty), its main spec deleted rather than left empty; a spec the sync deliberately kept and reported is also a match
|
||||
- RENAMED requirements present under the new name and absent under the old one
|
||||
|
||||
If the sync failed, or any capability does not match, report what differs and stop — do not archive. Nothing has moved and `changeRoot` is intact, so the user can fix the mismatch or re-run the sync and start the archive again.
|
||||
|
||||
5. **Perform the archive**
|
||||
|
||||
Create an `archive` directory under `planningHome.changesDir` if it doesn't exist:
|
||||
```bash
|
||||
mkdir -p "<planningHome.changesDir>/archive"
|
||||
```
|
||||
|
||||
Generate the target name: use the change name as-is when it already starts with a `YYYY-MM-DD-` prefix; otherwise prepend the current date as `YYYY-MM-DD-<change-name>`. Never stack a second date (same rule as `openspec archive`).
|
||||
|
||||
**Check if target already exists:**
|
||||
- If yes: Fail with error, suggest renaming existing archive or using different date
|
||||
- If no: Move `changeRoot` to the archive directory
|
||||
|
||||
```bash
|
||||
mv "<changeRoot>" "<planningHome.changesDir>/archive/<target-name>"
|
||||
```
|
||||
|
||||
6. **Display summary**
|
||||
|
||||
Show archive completion summary including:
|
||||
- Change name
|
||||
- Schema that was used
|
||||
- Archive location
|
||||
- Whether specs were synced (if applicable)
|
||||
- Note about any warnings (incomplete artifacts/tasks)
|
||||
|
||||
**Output On Success**
|
||||
|
||||
```markdown
|
||||
## Archive Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** the archive path derived from `planningHome.changesDir`/<target-name>/
|
||||
**Specs:** <"✓ Synced to main specs" only if the step 4 verification passed; otherwise "No delta specs" or "Sync skipped">
|
||||
|
||||
<"All artifacts complete. All tasks complete." — or, if archived with warnings, list them instead (e.g. "Archived with 2 incomplete tasks")>
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Announce the selected change; prompt for selection when it is ambiguous
|
||||
- Use artifact graph (openspec status --json) for completion checking
|
||||
- Don't block archive on warnings - just inform and confirm
|
||||
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
|
||||
- Show clear summary of what happened
|
||||
- If sync is requested, run the `openspec-sync-specs` workflow inline (agent-driven)
|
||||
- Never archive while a spec sync is still in flight — run the sync inline and verify the main specs before moving `changeRoot`
|
||||
- If delta specs exist, always run the sync assessment and show the combined summary before prompting
|
||||
- Apply relevant runtime context and report conflicts; operation guidance remains advisory
|
||||
- Consider every guidance entry and explain any inapplicable or conflicting advice
|
||||
- Existing CLI checks, resolved paths, prompts, and command contracts are unchanged
|
||||
- Artifact rules constrain only the specs being written and are never operation guidance
|
||||
- Never copy runtime context, operation guidance, or artifact-rule text verbatim into output files
|
||||
@@ -0,0 +1,308 @@
|
||||
---
|
||||
name: openspec-explore
|
||||
description: Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change.
|
||||
allowed-tools: Bash(openspec:*)
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.8.0"
|
||||
---
|
||||
|
||||
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
|
||||
|
||||
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing. For a new change, scaffold it first as described below.
|
||||
|
||||
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
|
||||
|
||||
**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store <id>` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `view`). Once selected, treat `--store <id>` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "<name>" --json --store "<id>"`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root.
|
||||
|
||||
---
|
||||
|
||||
## The Stance
|
||||
|
||||
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
|
||||
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
|
||||
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
|
||||
- **Adaptive** - Follow interesting threads, pivot when new information emerges
|
||||
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
|
||||
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
|
||||
|
||||
---
|
||||
|
||||
## What You Might Do
|
||||
|
||||
Depending on what the user brings, you might:
|
||||
|
||||
**Explore the problem space**
|
||||
- Ask clarifying questions that emerge from what they said
|
||||
- Challenge assumptions
|
||||
- Reframe the problem
|
||||
- Find analogies
|
||||
|
||||
**Investigate the codebase**
|
||||
- Map existing architecture relevant to the discussion
|
||||
- Find integration points
|
||||
- Identify patterns already in use
|
||||
- Surface hidden complexity
|
||||
|
||||
**Compare options**
|
||||
- Brainstorm multiple approaches
|
||||
- Build comparison tables
|
||||
- Sketch tradeoffs
|
||||
- Recommend a path (if asked)
|
||||
|
||||
**Visualize**
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Use ASCII diagrams liberally │
|
||||
├─────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌────────┐ ┌────────┐ │
|
||||
│ │ State │────────▶│ State │ │
|
||||
│ │ A │ │ B │ │
|
||||
│ └────────┘ └────────┘ │
|
||||
│ │
|
||||
│ System diagrams, state machines, │
|
||||
│ data flows, architecture sketches, │
|
||||
│ dependency graphs, comparison tables │
|
||||
│ │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Surface risks and unknowns**
|
||||
- Identify what could go wrong
|
||||
- Find gaps in understanding
|
||||
- Suggest spikes or investigations
|
||||
|
||||
---
|
||||
|
||||
## OpenSpec Awareness
|
||||
|
||||
You have full context of the OpenSpec system. Use it naturally, don't force it.
|
||||
|
||||
### Check for context
|
||||
|
||||
At the start, quickly check what exists:
|
||||
```bash
|
||||
openspec list --json
|
||||
```
|
||||
|
||||
This tells you:
|
||||
- If there are active changes
|
||||
- Their names, schemas, and status
|
||||
- What the user might be working on
|
||||
|
||||
Then read the project's own context from the resolved root - `<root.path>/openspec/config.yaml` (or `config.yml`). Use the `root.path` returned above, and skip this if neither file exists:
|
||||
- `context`: project background - tech stack, conventions, constraints
|
||||
- `rules`: keyed by artifact id - the entries for an artifact apply only when you write that artifact
|
||||
|
||||
Ground your thinking in these. They are constraints for you to follow, not content to reproduce: do NOT copy them into the conversation or into any artifact you create.
|
||||
|
||||
### When no change exists
|
||||
|
||||
Think freely. When insights crystallize, you might offer:
|
||||
|
||||
- "This feels solid enough to start a change. Want me to create a proposal?"
|
||||
- Or keep exploring - no pressure to formalize
|
||||
|
||||
If the user asks you to capture the exploration as a new change, transition seamlessly into the requested capture:
|
||||
|
||||
1. Run `openspec new change "<name>"` (with `--store <id>` when applicable) before creating any artifacts. Never create a new change directory under `openspec/changes/` by hand; the CLI scaffold creates required metadata such as `.openspec.yaml`. Keep the selected `--store <id>` on every applicable follow-up `status` and `instructions` command.
|
||||
2. Run `openspec status --change "<name>" --json` (append the confirmed `--store "<id>"` only for a registered standalone store), then process the requested artifacts in dependency order. For each requested artifact that is `ready`, run `openspec instructions "<artifact-id>" --change "<name>" --json` (append the confirmed `--store "<id>"` only for a registered standalone store). Before creating a requested artifact, evaluate any condition in its own `instruction` against the explored change; record a deliberate skip instead when the condition does not apply. If a requested artifact is blocked by a direct prerequisite the user did not request, run `openspec instructions "<prerequisite-id>" --change "<name>" --json` (append the confirmed `--store "<id>"` only for a registered standalone store) for that prerequisite whether it is `ready` or `blocked`. If its own `instruction` states a condition, evaluate that condition against the explored change and record a deliberate skip only when the condition does not apply. If the condition applies, or the prerequisite is not conditional, treat it as a normal prerequisite and ask before expanding the capture. Do not create an unrequested prerequisite unless the user approves.
|
||||
3. Follow the returned `template` and `instruction` fields. Read completed dependency files listed in `dependencies`, and apply `context` and `rules` as constraints without copying them into the artifact. If the instruction delegates creation to a specific skill or command, invoke it; otherwise write the artifact to `resolvedOutputPath`, using the instruction to choose a concrete path when it is a glob. Verify that the selected concrete output exists.
|
||||
4. After creating each artifact, re-run `openspec status --change "<name>" --json` (append the confirmed `--store "<id>"` only for a registered standalone store) and continue until every requested artifact is `done`, `skipped`, or was deliberately skipped because its own `instruction` stated a condition that did not apply. Tell the user about a deliberate conditional skip, remember it, and do not reconsider it. Dependencies are enablers, not gates: if a requested artifact is still `blocked` only because you deliberately skipped a conditional prerequisite, run `openspec instructions "<artifact-id>" --change "<name>" --json` (append the confirmed `--store "<id>"` only for a registered standalone store) despite the blocked status, then create it using step 3 only when those recorded conditional skips are its sole missing dependencies. If a requested artifact is blocked by a prerequisite the user did not ask to capture and cannot be conditionally skipped, explain that dependency and ask before expanding the capture.
|
||||
|
||||
Capture the artifact(s) the user requested without asking them to invoke another workflow command. If they asked only to start a change, stop after scaffolding and show its status.
|
||||
|
||||
### When a change exists
|
||||
|
||||
If the user mentions a change or you detect one is relevant:
|
||||
|
||||
1. **Resolve and read existing artifacts for context**
|
||||
- Run `openspec status --change "<name>" --json`.
|
||||
- Use `changeRoot`, `artifactPaths`, and `actionContext` from the status JSON.
|
||||
- Read existing files from `artifactPaths.<artifact>.existingOutputPaths`.
|
||||
|
||||
2. **Reference them naturally in conversation**
|
||||
- "Your design mentions using Redis, but we just realized SQLite fits better..."
|
||||
- "The proposal scopes this to premium users, but we're now thinking everyone..."
|
||||
|
||||
3. **Offer to capture when decisions are made**
|
||||
|
||||
`<capability-path>` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve an existing capability's full path and follow the project's established organization for new capabilities.
|
||||
|
||||
| Insight Type | Where to Capture |
|
||||
|----------------------------|-------------------------------------|
|
||||
| New requirement discovered | `specs/<capability-path>/spec.md` |
|
||||
| Requirement changed | `specs/<capability-path>/spec.md` |
|
||||
| Design decision made | `design.md` |
|
||||
| Scope changed | `proposal.md` |
|
||||
| New work identified | `tasks.md` |
|
||||
| Assumption invalidated | Relevant artifact |
|
||||
|
||||
Example offers:
|
||||
- "That's a design decision. Capture it in design.md?"
|
||||
- "This is a new requirement. Add it to specs?"
|
||||
- "This changes scope. Update the proposal?"
|
||||
|
||||
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
|
||||
|
||||
---
|
||||
|
||||
## What You Don't Have To Do
|
||||
|
||||
- Follow a script
|
||||
- Ask the same questions every time
|
||||
- Produce a specific artifact
|
||||
- Reach a conclusion
|
||||
- Stay on topic if a tangent is valuable
|
||||
- Be brief (this is thinking time)
|
||||
|
||||
---
|
||||
|
||||
## Handling Different Entry Points
|
||||
|
||||
**User brings a vague idea:**
|
||||
```
|
||||
User: I'm thinking about adding real-time collaboration
|
||||
|
||||
You: Real-time collab is a big space. Let me think about this...
|
||||
|
||||
COLLABORATION SPECTRUM
|
||||
════════════════════════════════════════════
|
||||
|
||||
Awareness Coordination Sync
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌────────┐ ┌────────┐ ┌────────┐
|
||||
│Presence│ │Cursors │ │ CRDT │
|
||||
│ "3 │ │ Multi │ │Conflict│
|
||||
│online" │ │ select │ │ free │
|
||||
└────────┘ └────────┘ └────────┘
|
||||
│ │ │
|
||||
trivial moderate complex
|
||||
|
||||
Where's your head at?
|
||||
```
|
||||
|
||||
**User brings a specific problem:**
|
||||
```
|
||||
User: The auth system is a mess
|
||||
|
||||
You: [reads codebase]
|
||||
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ CURRENT AUTH FLOW │
|
||||
└─────────────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────┼─────────────┐
|
||||
▼ ▼ ▼
|
||||
┌─────────┐ ┌─────────┐ ┌─────────┐
|
||||
│ Google │ │ GitHub │ │ Email │
|
||||
│ OAuth │ │ OAuth │ │ Magic │
|
||||
└────┬────┘ └────┬────┘ └────┬────┘
|
||||
│ │ │
|
||||
└─────────────┼─────────────┘
|
||||
▼
|
||||
┌───────────┐
|
||||
│ Session │
|
||||
└─────┬─────┘
|
||||
│
|
||||
▼
|
||||
┌───────────┐
|
||||
│ Perms │
|
||||
└───────────┘
|
||||
|
||||
I see three tangles. Which one's burning?
|
||||
```
|
||||
|
||||
**User is stuck mid-implementation:**
|
||||
```
|
||||
User: /opsx:explore add-auth-system
|
||||
The OAuth integration is more complex than expected
|
||||
|
||||
You: [reads change artifacts]
|
||||
|
||||
You're on task 4: "Implement OAuth flow"
|
||||
|
||||
Let me trace what's involved...
|
||||
|
||||
[draws diagram, explores options, suggests paths]
|
||||
|
||||
Want to update the design to reflect this?
|
||||
Or add a spike task to investigate?
|
||||
```
|
||||
|
||||
**User wants to compare options:**
|
||||
```
|
||||
User: Should we use Postgres or SQLite?
|
||||
|
||||
You: Generic answer is boring. What's the context?
|
||||
|
||||
User: A CLI tool that tracks local dev environments
|
||||
|
||||
You: That changes everything.
|
||||
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ CLI TOOL DATA STORAGE │
|
||||
└─────────────────────────────────────────────────┘
|
||||
|
||||
Key constraints:
|
||||
• No daemon running
|
||||
• Must work offline
|
||||
• Single user
|
||||
|
||||
SQLite Postgres
|
||||
Deployment embedded ✓ needs server ✗
|
||||
Offline yes ✓ no ✗
|
||||
Single file yes ✓ no ✗
|
||||
|
||||
SQLite. Not even close.
|
||||
|
||||
Unless... is there a sync component?
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Ending Discovery
|
||||
|
||||
There's no required ending. Discovery might:
|
||||
|
||||
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
|
||||
- **Result in artifact updates**: "Updated design.md with these decisions"
|
||||
- **Just provide clarity**: User has what they need, moves on
|
||||
- **Continue later**: "We can pick this up anytime"
|
||||
|
||||
When it feels like things are crystallizing, you might summarize:
|
||||
|
||||
```
|
||||
## What We Figured Out
|
||||
|
||||
**The problem**: [crystallized understanding]
|
||||
|
||||
**The approach**: [if one emerged]
|
||||
|
||||
**Open questions**: [if any remain]
|
||||
|
||||
**Next steps** (if ready):
|
||||
- Create a change proposal
|
||||
- Keep exploring: just keep talking
|
||||
```
|
||||
|
||||
But this summary is optional. Sometimes the thinking IS the value.
|
||||
|
||||
---
|
||||
|
||||
## Guardrails
|
||||
|
||||
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
|
||||
- **Don't fake understanding** - If something is unclear, dig deeper
|
||||
- **Don't rush** - Discovery is thinking time, not task time
|
||||
- **Don't force structure** - Let patterns emerge naturally
|
||||
- **Don't auto-capture** - Offer to save insights, don't just do it
|
||||
- **Don't manually scaffold changes** - Never create a new change directory under `openspec/changes/` by hand. Always use `openspec new change "<name>"` (with `--store <id>` when applicable) so required metadata such as `.openspec.yaml` is created before writing artifacts.
|
||||
- **Do visualize** - A good diagram is worth many paragraphs
|
||||
- **Do explore the codebase** - Ground discussions in reality
|
||||
- **Do question assumptions** - Including the user's and your own
|
||||
@@ -0,0 +1,149 @@
|
||||
---
|
||||
name: openspec-propose
|
||||
description: Propose a new change with all artifacts generated in one step. Use when the user wants to quickly describe what they want to build and get a complete proposal with design, specs, and tasks ready for implementation.
|
||||
allowed-tools: Bash(openspec:*)
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.8.0"
|
||||
---
|
||||
|
||||
Propose a new change - create the change and generate all artifacts in one step.
|
||||
|
||||
**Planning boundary**: This workflow creates planning artifacts only. The user request that selected or triggered this workflow authorizes planning only, even if it asks to build or fix something. Do not edit project code. After the planning artifacts are complete, stop. Do not start implementation in the same response, even if the initial request asks for it. Wait for a new user request after the artifacts are presented; then start the apply workflow.
|
||||
|
||||
I'll create a change with the artifacts your schema defines. With the default spec-driven schema that is:
|
||||
- proposal.md (what & why)
|
||||
- `specs/<capability-path>/spec.md` (what the system must do - a delta, not the main spec)
|
||||
- design.md (how)
|
||||
- tasks.md (implementation steps)
|
||||
|
||||
`<capability-path>` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve an existing capability's full path and follow the project's established organization for new capabilities.
|
||||
|
||||
When the user is ready to implement, they must start the apply workflow explicitly.
|
||||
|
||||
---
|
||||
|
||||
**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store <id>` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `view`). Once selected, treat `--store <id>` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "<name>" --json --store "<id>"`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root.
|
||||
|
||||
**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Understand the request and clarify material ambiguity**
|
||||
|
||||
If no clear input is provided, ask the user (open-ended, no preset options):
|
||||
> "What change do you want to work on? Describe what you want to build or fix."
|
||||
|
||||
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
|
||||
|
||||
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
|
||||
|
||||
If the request contains ambiguity that would materially affect scope, externally observable behavior, compatibility, or acceptance criteria, ask the user before creating the change. For minor details, make a reasonable assumption and record it in the planning artifacts.
|
||||
|
||||
2. **Determine the workflow schema**
|
||||
|
||||
Use the configured default schema unless the user explicitly requests a different workflow.
|
||||
|
||||
**Use a different schema only if the user:**
|
||||
- Explicitly requests a specific schema by name → use `--schema <schema-name>`
|
||||
- Asks to "show workflows" or asks "what workflows" exist → resolve the authoritative root by running `openspec context --json` from the current working directory. If the user explicitly selected a registered store, use `openspec context --json --store "<store-id>"`. Then run `openspec schemas --json` with its working directory set to the returned `root.path` and let them choose. This preserves roots selected by a local `store:` pointer or the global `defaultStore`; `schemas` does not accept `--store`. If context reports only `no_openspec_root`, run `openspec schemas --json` from the current working directory instead. Do not use this fallback for invalid or unavailable stores.
|
||||
|
||||
Otherwise, omit `--schema` to preserve the configured default.
|
||||
|
||||
3. **Create the change directory**
|
||||
|
||||
Choose one schema form below. If a registered store is selected, append `--store "<store-id>"` to that command and each later OpenSpec command shown below that accepts `--store`.
|
||||
|
||||
Using the configured default:
|
||||
```bash
|
||||
openspec new change "<name>"
|
||||
```
|
||||
|
||||
Using an explicitly requested schema:
|
||||
```bash
|
||||
openspec new change "<name>" --schema "<schema-name>"
|
||||
```
|
||||
This creates a scaffolded change in the planning home resolved by the CLI with `.openspec.yaml`.
|
||||
|
||||
4. **Get the artifact build order**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to get:
|
||||
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
|
||||
- `artifacts`: list of all artifacts, each with its `status` and its `requires` edges (the artifact IDs it directly depends on)
|
||||
- `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths.
|
||||
|
||||
5. **Create every artifact in the required set**
|
||||
|
||||
Use a todo list to track progress through the artifacts.
|
||||
|
||||
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
|
||||
|
||||
a. **For each artifact that is `ready` (dependencies satisfied)**:
|
||||
- Get instructions:
|
||||
```bash
|
||||
openspec instructions <artifact-id> --change "<name>" --json
|
||||
```
|
||||
- The instructions JSON includes:
|
||||
- `context`: Project background (constraints for you - do NOT include in output)
|
||||
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
|
||||
- `template`: The structure to use for your output file
|
||||
- `instruction`: Schema-specific guidance for this artifact type
|
||||
- `skipped`/`warning`: present when the change declares skip_specs and this artifact must NOT be created - stop and pick another artifact
|
||||
- `resolvedOutputPath`: Resolved path or pattern to write the artifact
|
||||
- `dependencies`: Completed artifacts to read for context
|
||||
- Read any completed dependency files for context - always re-read them from disk, even if you saw them earlier in the conversation (the user may have edited them)
|
||||
- If the `instruction` field delegates creation to a specific skill or command, invoke it to produce the artifact instead of writing the file yourself, then verify the artifact file exists at `resolvedOutputPath`
|
||||
- Otherwise create the artifact file using `template` as the structure and write it to `resolvedOutputPath`. If `resolvedOutputPath` is a glob, follow `instruction` to choose the concrete file path
|
||||
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
|
||||
- Show brief progress: "Created <artifact-id>"
|
||||
|
||||
b. **Continue until every artifact in the required set exists (not just `apply.requires`)**
|
||||
- After creating each artifact, re-run `openspec status --change "<name>" --json`
|
||||
- The required set is `applyRequires` plus every artifact reachable from those by following the `requires` edges in `status --json` - walk them transitively (spec-driven closes over proposal, specs, design, tasks). Leave artifacts outside that set alone
|
||||
- `status` is file-existence only, so an `applyRequires` artifact reading `done` does NOT mean its dependencies exist - writing `tasks.md` early marks `tasks` done while `specs` was never written. Use each artifact's `requires` edges, not its `status`, to build the required set: a `done` artifact still lists what it depends on
|
||||
- An artifact already reading `status: "skipped"` is satisfied: the change declares `skip_specs` in `.openspec.yaml`, so its files must NOT exist. Never try to create one
|
||||
- Create every artifact in the required set that is missing, then re-check - creating one can unblock others
|
||||
- Skip one only when `status` already reports it `skipped`, or when its own `instruction` says it is conditional: run `openspec instructions <artifact-id> --change "<name>" --json` and skip only if its `instruction` field marks it optional (e.g. "create only if..."). Spec-driven's `design.md` qualifies; `specs` qualifies only via the `skipped` status above, never by your own judgment. Tell the user, and do not reconsider it
|
||||
- Dependencies are enablers, not gates: if a required artifact is still `blocked` only because you skipped a conditional dependency, write it anyway
|
||||
- Stop when every artifact in the required set is `done`, `skipped`, or was deliberately skipped
|
||||
|
||||
c. **If an artifact requires user input** (unclear context):
|
||||
- Ask the user to clarify
|
||||
- Then continue with creation
|
||||
|
||||
6. **Show final status**
|
||||
```bash
|
||||
openspec status --change "<name>"
|
||||
```
|
||||
|
||||
**Output**
|
||||
|
||||
After completing all artifacts, summarize:
|
||||
- Change name and location
|
||||
- List of artifacts created with brief descriptions, plus any conditional artifact you skipped and why
|
||||
- What's ready: "All artifacts needed for implementation are ready."
|
||||
- Prompt: "The artifacts are ready for review. When you are ready, run `/opsx:apply` or ask me to apply this change."
|
||||
|
||||
**Artifact Creation Guidelines**
|
||||
|
||||
- Follow the `instruction` field from `openspec instructions` for each artifact type - it is the authoritative guidance, even for familiar artifact names
|
||||
- If the `instruction` field directs you to use a specific skill or command to create the artifact, invoke it instead of writing the artifact directly
|
||||
- The schema defines what each artifact should contain - follow it
|
||||
- Read dependency artifacts for context before creating new ones
|
||||
- Use `template` as the structure for your output file - fill in its sections
|
||||
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
|
||||
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
|
||||
- These guide what you write, but should never appear in the output
|
||||
|
||||
**Guardrails**
|
||||
- The request that invoked this workflow authorizes planning only. Any implementation or apply instruction in that request does not carry forward. Do NOT implement the change, start the apply workflow, or edit project code during this workflow. After presenting the artifacts, stop and wait for a new user request to start the apply workflow
|
||||
- Create every artifact the apply phase transitively depends on, not just the ids listed in `apply.requires`
|
||||
- Always read dependency artifacts before creating a new one - re-read from disk, not from conversation memory (files may have changed since you last saw them)
|
||||
- Ask about ambiguities that would materially change scope, externally observable behavior, compatibility, or acceptance criteria; for minor details, make reasonable assumptions and record them
|
||||
- If a change with that name already exists, ask if user wants to continue it or create a new one
|
||||
- Verify each artifact file exists after writing before proceeding to next
|
||||
@@ -0,0 +1,262 @@
|
||||
---
|
||||
name: openspec-sync-specs
|
||||
description: Sync delta specs from a change to main specs. Use when the user wants to update main specs with changes from a delta spec, without archiving the change.
|
||||
allowed-tools: Bash(openspec:*)
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.8.0"
|
||||
---
|
||||
|
||||
Sync delta specs from a change to main specs.
|
||||
|
||||
This is an **agent-driven** operation - you will read delta specs and directly edit main specs to apply the changes. This allows intelligent merging (e.g., adding a scenario without copying the entire requirement).
|
||||
|
||||
**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store <id>` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `view`). Once selected, treat `--store <id>` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "<name>" --json --store "<id>"`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root.
|
||||
|
||||
`<capability-path>` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path from each delta spec when resolving its main spec.
|
||||
|
||||
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes and ask the user to select one
|
||||
|
||||
When prompting, show changes that have delta specs (under `specs/` directory).
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:sync <other>`).
|
||||
|
||||
2. **Resolve change context**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
|
||||
The JSON includes `planningHome.root`. Main specs live under `<planningHome.root>/openspec/specs/` — use that (store-aware) root for every main-spec path below, not a hardcoded repo path. When a store is selected it points at the store, not the current repository.
|
||||
|
||||
3. **Find delta specs**
|
||||
|
||||
Use `artifactPaths.specs.existingOutputPaths` from the status JSON as the
|
||||
only source of delta spec paths. If the `specs` entry is missing or
|
||||
`existingOutputPaths` is empty, report that there are no delta specs to sync,
|
||||
do not infer them from other artifacts, and stop without requesting artifact
|
||||
instructions or writing a main spec.
|
||||
|
||||
Sync every path in `existingOutputPaths` unless the caller narrowed the set.
|
||||
A caller narrows it by naming an explicit list of complete entries from
|
||||
`existingOutputPaths` — copy those absolute values verbatim. Archive does
|
||||
this inline, and a user can too (for example, by selecting the entry ending
|
||||
in `/specs/billing/invoices/spec.md`).
|
||||
Then sync only the named paths and leave the remaining delta specs untouched:
|
||||
bulk archive excludes a delta whose implementation it could not find, and
|
||||
syncing it anyway would write a main spec the caller deliberately withheld.
|
||||
Carry that narrowed selection through step 4; never widen it back to the full
|
||||
list. If a named path is not in `existingOutputPaths`, do not sync it —
|
||||
report it and stop, rather than dropping it silently. If the named list is
|
||||
empty, report that there is nothing to sync and stop without writing a main
|
||||
spec.
|
||||
|
||||
Each delta spec file contains sections like:
|
||||
- `## ADDED Requirements` - New requirements to add
|
||||
- `## MODIFIED Requirements` - Changes to existing requirements
|
||||
- `## REMOVED Requirements` - Requirements to remove
|
||||
- `## RENAMED Requirements` - Requirements to rename (FROM:/TO: format)
|
||||
|
||||
If no delta specs found, inform user and stop.
|
||||
|
||||
4. **For each delta spec, apply changes to main specs**
|
||||
|
||||
Before the first main-spec write, obtain one current specs-rule snapshot:
|
||||
- If archive invoked this workflow inline and supplied a valid snapshot from
|
||||
`openspec instructions specs --change "<name>" --json`, reuse it and do not
|
||||
fetch the same instructions again.
|
||||
- Otherwise run that command once now with the same selected-root flags.
|
||||
- If the direct lookup exits non-zero or returns invalid artifact-instruction
|
||||
JSON, report the error and stop before writing any main spec. Do not treat the
|
||||
failure as an absent rule set.
|
||||
- A valid response with omitted `rules` means no artifact rules are configured
|
||||
and the existing semantic merge continues.
|
||||
|
||||
Apply returned `rules` only to the content and form of the main specs produced
|
||||
by this merge. Artifact rules are not operation guidance and cannot change
|
||||
selected roots, delta paths, CLI checks, or workflow steps. Use their text as
|
||||
constraints without copying it verbatim into a main spec or summary.
|
||||
|
||||
For each capability delta spec path selected in step 3 — the full `existingOutputPaths` list, or the narrowed subset when a caller supplied one (these may belong to a selected store, not the repo):
|
||||
|
||||
a. **Read the delta spec** to understand the intended changes
|
||||
|
||||
b. **Read the main spec** at `<planningHome.root>/openspec/specs/<capability-path>/spec.md` (may not exist yet)
|
||||
|
||||
c. **Apply changes intelligently**:
|
||||
|
||||
**ADDED Requirements:**
|
||||
- If requirement doesn't exist in main spec → add it
|
||||
- If requirement already exists → update it to match (treat as implicit MODIFIED)
|
||||
|
||||
**MODIFIED Requirements:**
|
||||
- Find the requirement in main spec
|
||||
- Apply the changes - this can be:
|
||||
- Adding new scenarios the main spec does not have yet
|
||||
- Modifying existing scenarios
|
||||
- Changing the requirement description
|
||||
- Preserve scenarios/content not mentioned in the delta
|
||||
|
||||
**REMOVED Requirements:**
|
||||
- Remove the entire requirement block from main spec
|
||||
- Retiring the capability. Delete the whole `spec.md` - and the directory once
|
||||
nothing else is left in it - only when ALL of these hold:
|
||||
1. removing the requirements *this run* left no requirement blocks;
|
||||
2. the rest of the spec is well-formed (it still has a `## Purpose`);
|
||||
3. the main spec was not already empty before this sync - if you removed
|
||||
nothing, change nothing;
|
||||
4. every other nonblank line in the whole file is accounted for as the
|
||||
title, Purpose, Requirements header, or a canonical requirement's
|
||||
statement, scenarios, or fenced examples;
|
||||
5. the change's `.openspec.yaml` declares `retire_capabilities: true`;
|
||||
6. the `spec.md` resolves inside the real specs root (do not follow a
|
||||
capability-directory symlink to delete an external file).
|
||||
If removing the selected requirements would leave no requirement blocks and
|
||||
any retirement condition is not satisfied, do not modify the main spec. Stop
|
||||
the sync for that capability, report the blocking condition, and tell the user
|
||||
how to resolve it. Never write or leave an empty `## Requirements` section.
|
||||
When only the marker is missing, say that too - it is the one thing the user
|
||||
can add to make the retirement go through.
|
||||
- Deleting the file also deletes its `## Purpose`; any other section blocks
|
||||
retirement. Name Purpose when you report the retirement. Include a pasteable
|
||||
`git checkout` only when the spec lived in the caller's checkout;
|
||||
otherwise give checkout-scoped recovery guidance.
|
||||
|
||||
**RENAMED Requirements:**
|
||||
- Find the FROM requirement, rename to TO
|
||||
|
||||
**`## Purpose` in the delta:**
|
||||
- The main spec already has one and it is authoritative - leave it alone
|
||||
(this is what `openspec archive` does; it warns and moves on)
|
||||
|
||||
d. **Create new main spec** if capability doesn't exist yet:
|
||||
- Create `<planningHome.root>/openspec/specs/<capability-path>/spec.md`
|
||||
- Add Purpose section: copy the delta's `## Purpose` body verbatim when it has one
|
||||
(this is what `openspec archive` does); only write a brief TBD placeholder when it does not
|
||||
- Add Requirements section with the ADDED requirements
|
||||
- Follow the **Main Spec Format Reference** below
|
||||
|
||||
5. **Validate updated main specs**
|
||||
|
||||
Run `openspec validate --specs` with the same selected-root flags used earlier.
|
||||
If validation fails, report the problems and do not claim the sync succeeded.
|
||||
|
||||
6. **Show summary**
|
||||
|
||||
After applying all changes, summarize:
|
||||
- Which capabilities were updated
|
||||
- What changes were made (requirements added/modified/removed/renamed)
|
||||
- Any new main spec left with a TBD Purpose placeholder, so it gets written
|
||||
now rather than lingering
|
||||
- Any capability retired, naming the deleted `spec.md`, its Purpose, and
|
||||
either a pasteable `git checkout` or checkout-scoped recovery guidance
|
||||
|
||||
**Delta Spec Format Reference**
|
||||
|
||||
```markdown
|
||||
## Purpose
|
||||
|
||||
Only on a delta that introduces a brand-new capability. Seeds the new main spec.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: New Feature
|
||||
The system SHALL do something new.
|
||||
|
||||
#### Scenario: Basic case
|
||||
- **WHEN** user does X
|
||||
- **THEN** system does Y
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Existing Feature
|
||||
The system SHALL keep doing the existing thing, now also handling A.
|
||||
|
||||
#### Scenario: Scenario the main spec already has
|
||||
- **WHEN** user does X
|
||||
- **THEN** system does Y
|
||||
|
||||
#### Scenario: New scenario to add
|
||||
- **WHEN** user does A
|
||||
- **THEN** system does B
|
||||
|
||||
## REMOVED Requirements
|
||||
|
||||
### Requirement: Deprecated Feature
|
||||
|
||||
## RENAMED Requirements
|
||||
|
||||
- FROM: `### Requirement: Old Name`
|
||||
- TO: `### Requirement: New Name`
|
||||
```
|
||||
|
||||
**Main Spec Format Reference**
|
||||
|
||||
Main specs are what the delta merges INTO. They must never contain delta operation headers (`## ADDED/MODIFIED/REMOVED/RENAMED Requirements`) - after syncing, every requirement lives under a single `## Requirements` section:
|
||||
|
||||
```markdown
|
||||
# <capability> Specification
|
||||
|
||||
## Purpose
|
||||
Short description of what this capability does and why it exists.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: New Feature
|
||||
The system SHALL do something new.
|
||||
|
||||
#### Scenario: Basic case
|
||||
- **WHEN** user does X
|
||||
- **THEN** system does Y
|
||||
```
|
||||
|
||||
**Key Principle: Intelligent Merging**
|
||||
|
||||
Unlike programmatic merging, you merge rather than overwrite:
|
||||
- A MODIFIED block carries the whole requirement - body plus every scenario that survives the change. `openspec validate` and `openspec archive` both reject one that drops a scenario the main spec still has.
|
||||
- Keep anything the delta does not mention, in the main spec's existing order
|
||||
- Use your judgment to merge changes sensibly
|
||||
|
||||
**Output On Success**
|
||||
|
||||
```markdown
|
||||
## Specs Synced: <change-name>
|
||||
|
||||
Updated main specs:
|
||||
|
||||
**<capability-1>**:
|
||||
- Added requirement: "New Feature"
|
||||
- Modified requirement: "Existing Feature" (added 1 scenario)
|
||||
|
||||
**<capability-2>**:
|
||||
- Created new spec file
|
||||
- Added requirement: "Another Feature"
|
||||
|
||||
Main specs are now updated. The change remains active - archive when implementation is complete.
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Read both delta and main specs before making changes
|
||||
- Preserve existing content not mentioned in delta
|
||||
- Never copy a delta file into a main spec as-is - merge its content so the main spec keeps the Main Spec Format Reference structure, with no delta operation headers
|
||||
- If something is unclear, ask for clarification
|
||||
- Show what you're changing as you go
|
||||
- The operation should be idempotent - running twice should give same result
|
||||
- Use only `artifactPaths.specs.existingOutputPaths`; never infer delta specs from unrelated artifacts
|
||||
- Honor a caller-supplied subset of `existingOutputPaths`; never widen it back to the full list
|
||||
- Fetch specs instructions once for direct sync, or reuse the archive-supplied snapshot inline
|
||||
- Stop before every main-spec write on a non-zero or invalid JSON specs-instruction response
|
||||
- Artifact rules constrain only the specs being written and are never copied into output files
|
||||
@@ -0,0 +1,91 @@
|
||||
---
|
||||
name: openspec-update-change
|
||||
description: Update an OpenSpec change by revising its existing planning artifacts and keeping them coherent with one another. Use when the user wants to revise a change's plan, fold new decisions into it, or reconcile its artifacts after an edit. Never edits code.
|
||||
allowed-tools: Bash(openspec:*)
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.8.0"
|
||||
---
|
||||
|
||||
Revise a change's existing planning artifacts and keep them coherent. Never edit code.
|
||||
|
||||
**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store <id>` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `view`). Once selected, treat `--store <id>` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "<name>" --json --store "<id>"`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root.
|
||||
|
||||
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
`/opsx:continue` is an expanded-profile workflow and may not be installed. Before suggesting it anywhere below, verify that it is available. If it is unavailable, `openspec status --change "<name>" --json` shows the next artifact and `openspec instructions "<artifact-id>" --change "<name>" --json` explains how to create it.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes sorted by most recently modified, and ask the user to select one
|
||||
|
||||
When prompting, present the top 3-4 most recently modified changes as options, showing:
|
||||
- Change name
|
||||
- Schema (from `schema` field if present, otherwise "spec-driven")
|
||||
- Status (e.g., "0/5 tasks", "complete", "no tasks")
|
||||
- How recently it was modified (from `lastModified` field)
|
||||
|
||||
Mark the most recently modified change as "(Recommended)" since it's likely what the user wants to update.
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:update <other>`).
|
||||
|
||||
2. **Get the change's artifacts**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to understand current state. The response includes:
|
||||
- `schemaName`: The workflow schema being used (e.g., "spec-driven")
|
||||
- `artifacts`: Array of artifacts with their status ("done", "skipped", "ready", "blocked")
|
||||
- `isPlanningComplete`: Boolean indicating if all planning artifacts are complete. Older CLI versions expose the same value as `isComplete`.
|
||||
- `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths.
|
||||
|
||||
The artifact ids and paths come from the active schema - do NOT assume them, and do NOT branch on hardcoded artifact names. Custom schemas must work unchanged.
|
||||
|
||||
The files to edit are `artifactPaths.<id>.existingOutputPaths` - the concrete files that exist on disk, already glob-expanded for glob artifacts (e.g. `specs/**/*.md`). Do NOT write to `resolvedOutputPath`: for a glob artifact it is still the glob pattern, not a real file.
|
||||
|
||||
3. **Understand the request**
|
||||
- If the user asked for a specific revision ("the design now uses X"), that is the starting edit.
|
||||
- If they only said "update" / "make this coherent", treat it as a coherence review: read the existing artifacts and check them against each other for contradictions, gaps, and duplication.
|
||||
|
||||
4. **Read and reconcile**
|
||||
- Read the artifact(s) the request touches and the change's other existing artifacts.
|
||||
- Apply the requested edit. Then check every other existing artifact against it - in ANY direction: an edit to a later artifact may require revising an earlier one, not only the other way around. Build order is a useful reading order, not a constraint on which artifacts may be revised.
|
||||
- Note everything that is now inconsistent, missing, or contradictory.
|
||||
- Revise only files that already exist (`existingOutputPaths`). Do NOT create artifacts that don't exist yet, and do NOT invent new files under a glob artifact - note them and point the user to `/opsx:continue` to create them.
|
||||
- If the change is already coherent, say so and make no edits.
|
||||
|
||||
5. **Confirm and apply, one artifact at a time**
|
||||
- Show each proposed revision and why. Write only after the user confirms.
|
||||
- If the user rejects a revision, do not write it - leave that artifact unchanged.
|
||||
- When a substantial rewrite is needed, get that artifact's rules and template first:
|
||||
```bash
|
||||
openspec instructions "<artifact-id>" --change "<name>" --json
|
||||
```
|
||||
|
||||
6. **Point to the next step (guidance only - NEVER act on it)**
|
||||
- Artifacts still missing -> suggest `/opsx:continue` to create them.
|
||||
- Change already implemented (tasks checked off / already applied) -> the code may no longer match the revised plan; suggest `/opsx:apply` to carry the delta into code.
|
||||
- Everything done and implemented -> suggest `/opsx:archive`.
|
||||
|
||||
**Output**
|
||||
|
||||
After each invocation, show:
|
||||
- Which artifacts were revised (and which proposed revisions were rejected)
|
||||
- Anything deferred to `/opsx:continue` (not-yet-created artifacts or files)
|
||||
- Where the change stands and the recommended next command
|
||||
|
||||
**Guardrails**
|
||||
- Planning artifacts only - NEVER edit implementation code. If the revised plan implies code changes, stop and point to `/opsx:apply`.
|
||||
- Use the artifact ids and paths reported by `openspec status`; never branch on hardcoded artifact names.
|
||||
- Edit only the concrete files in `existingOutputPaths`; never write to a glob `resolvedOutputPath`.
|
||||
- Do not advance the build frontier: no new artifacts, no new files under glob artifacts - that is `/opsx:continue`'s job.
|
||||
- Confirm every edit with the user before writing.
|
||||
- If the request changes the change's *intent* rather than refining it, first verify whether the expanded-profile `/opsx:new` workflow is available. If it is, recommend starting fresh with `/opsx:new` (the "Update vs. Start Fresh" heuristic). If it is unavailable, ask for a distinct unused change name and recommend `openspec new change "<new-change-name>"` instead.
|
||||
@@ -14,7 +14,33 @@ backups
|
||||
*.pfx
|
||||
*.jks
|
||||
.venv
|
||||
venv
|
||||
__pycache__
|
||||
*.pyc
|
||||
*.pyo
|
||||
.pytest_cache
|
||||
.ruff_cache
|
||||
.mypy_cache
|
||||
.ty_cache
|
||||
node_modules
|
||||
evidence
|
||||
openspec
|
||||
student-tests
|
||||
tests
|
||||
scripts
|
||||
examples
|
||||
contracts
|
||||
infra
|
||||
*.md
|
||||
!README.md
|
||||
Makefile
|
||||
Dockerfile*
|
||||
compose*.yaml
|
||||
.dockerignore
|
||||
.gitignore
|
||||
.DS_Store
|
||||
lab.json
|
||||
requirements-dev.txt
|
||||
.claude
|
||||
|
||||
.verification/
|
||||
|
||||
@@ -9,3 +9,57 @@ DATABASE_URL_FILE=/run/secrets/database_url
|
||||
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
|
||||
SESSION_STORE=postgres
|
||||
CACHE_URL_FILE=/run/secrets/cache_url
|
||||
API_REPLICAS=2
|
||||
IDEMPOTENCY_TTL_SECONDS=86400
|
||||
S3_ENDPOINT=http://s3:9000
|
||||
S3_REGION=us-east-1
|
||||
S3_BUCKET=messenger-attachments
|
||||
S3_ACCESS_KEY_ID_FILE=/run/secrets/s3_access_key_id
|
||||
S3_SECRET_ACCESS_KEY_FILE=/run/secrets/s3_secret_access_key
|
||||
MAX_UPLOAD_BYTES=10485760
|
||||
MAX_IMAGE_PIXELS=40000000
|
||||
# Solution-specific: published LB ports, replica limits, workers and pool per replica,
|
||||
# LB queue per replica, drain window before closing the listener on SIGTERM, PostgreSQL
|
||||
# limits, load generator limits (profile bench), cursor key file. API_PORT/SHORTTTL_PORT of lab2 are gone:
|
||||
# only the load balancer is published.
|
||||
LB_HTTP_PORT=8080
|
||||
LB_STATS_PORT=8404
|
||||
API_CPUS=1
|
||||
API_MEMORY=512m
|
||||
WEB_CONCURRENCY=2
|
||||
DB_POOL_MAX_SIZE=8
|
||||
API_SERVER_MAXCONN=64
|
||||
DRAIN_SECONDS=2
|
||||
PG_MAX_CONNECTIONS=200
|
||||
PG_CPUS=2
|
||||
PG_MEMORY=1g
|
||||
K6_CPUS=2
|
||||
K6_MEMORY=512m
|
||||
CURSOR_SECRET_FILE=/run/secrets/cursor_secret
|
||||
# Solution-specific for lab4: S3 client timeouts (connect + read, two attempts stay below
|
||||
# REQUEST_TIMEOUT_SECONDS=8), parallel image checks per worker, RustFS memory limit.
|
||||
S3_CONNECT_TIMEOUT_SECONDS=1
|
||||
S3_READ_TIMEOUT_SECONDS=2
|
||||
IMAGE_CHECK_CONCURRENCY=2
|
||||
S3_MEMORY=512m
|
||||
# Solution-specific for lab4 grade 4: upload bodies buffered at once per worker, grace period
|
||||
# before an abandoned pending upload is swept, sweep interval and rows per sweep pass.
|
||||
UPLOAD_CONCURRENCY=3
|
||||
UPLOAD_GRACE_SECONDS=900
|
||||
UPLOAD_SWEEP_INTERVAL_SECONDS=30
|
||||
UPLOAD_SWEEP_BATCH=100
|
||||
# Fixed glibc mmap threshold for the API: buffers above 128 KiB are returned to the system on free.
|
||||
MALLOC_MMAP_THRESHOLD_=131072
|
||||
# Minimal client transfer rate, bytes/s: every body byte moved in either direction extends the
|
||||
# request deadline by 1/rate, so slow but steady uploads and downloads outlive REQUEST_TIMEOUT_SECONDS.
|
||||
MIN_TRANSFER_RATE_BYTES=131072
|
||||
# Solution-specific for lab4 track 2: limits of the one-off reconciler (profile reconcile), its S3
|
||||
# read timeout for listing pages, optional CPU pinning of the measured services (empty = no pinning)
|
||||
# and the CPU limit of RustFS (make reconcile-cost raises it with its memory for the 100k point).
|
||||
RECONCILE_CPUS=1
|
||||
RECONCILE_MEMORY=256m
|
||||
RECONCILE_S3_READ_TIMEOUT_SECONDS=10
|
||||
RECONCILE_CPUSET=
|
||||
S3_CPUSET=
|
||||
PG_CPUSET=
|
||||
S3_CPUS=1
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
## Лабораторная 2
|
||||
## Лабораторная 4
|
||||
|
||||
Автор / группа: TODO
|
||||
Целевая оценка (3/4/5): TODO
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
name: Application
|
||||
|
||||
on: [push, pull_request]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
quality:
|
||||
name: Линт, типы и собственные тесты
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: astral-sh/setup-uv@v5
|
||||
with:
|
||||
enable-cache: true
|
||||
- run: uv sync --locked --group dev
|
||||
- run: make lint VENV=.venv
|
||||
- run: make student-test VENV=.venv
|
||||
|
||||
acceptance:
|
||||
name: Стенд Compose с RustFS за HAProxy, smoke, вложения, отказ реплики, рестарт и отказы зависимостей
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: astral-sh/setup-uv@v5
|
||||
with:
|
||||
enable-cache: true
|
||||
- run: uv sync --locked --group dev
|
||||
|
||||
- name: Поднять стенд из чистого clone
|
||||
run: make up
|
||||
|
||||
- name: Публичный smoke через балансировщик
|
||||
run: make test
|
||||
|
||||
- name: Вложение после остановки принявшей его реплики и при недоступном S3
|
||||
run: |
|
||||
make scenario SCENARIO=failover-upload
|
||||
make scenario SCENARIO=s3-outage
|
||||
|
||||
- name: Приватный bucket, ограниченный ключ API и повторный bootstrap
|
||||
run: make s3-check
|
||||
|
||||
- name: Негативный suite вложений, компенсация, уборка после grace, обрыв и зависание загрузки
|
||||
run: make upload-faults
|
||||
|
||||
- name: Несколько реплик на одном токене и гонка одного ключа
|
||||
run: make lb-check
|
||||
|
||||
- name: Повтор неподтверждённой записи после потерянного ответа и убитой реплики
|
||||
run: make retry-check
|
||||
|
||||
- name: Остановка реплики под запросами, затем возврат
|
||||
run: |
|
||||
make test-failover ACTION_SCRIPT=scripts/stop-one.sh
|
||||
docker compose ps -a api
|
||||
make restore
|
||||
|
||||
- name: Рестарт LB, API и хранилищ с сохранением volumes
|
||||
run: make test-persistence ACTION_SCRIPT=scripts/restart.sh
|
||||
|
||||
- name: Отказы Valkey и PostgreSQL, гонка и rollback
|
||||
run: make test-faults
|
||||
|
||||
- name: Лимиты, порты, секреты, volumes и сети по политике
|
||||
run: make limits
|
||||
|
||||
- name: Время обнаружения отказа балансировщиком (SIGTERM, kill, pause, PostgreSQL)
|
||||
run: make detection
|
||||
|
||||
- name: Наружу опубликован только балансировщик
|
||||
run: |
|
||||
docker compose ps --format '{{.Service}} {{.Ports}}' | awk '$2 != "" && $1 != "lb" {print; bad=1} END {exit bad}'
|
||||
|
||||
- name: Процессы работают не от root
|
||||
run: |
|
||||
for id in $(docker compose ps -q); do
|
||||
docker top "$id" -eo pid,uid,comm | awk 'NR>1 && $2==0 {print; bad=1} END {exit bad}' || exit 1
|
||||
done
|
||||
|
||||
- name: Одноразовые сервисы не от root
|
||||
run: |
|
||||
for service in migrate s3-init; do
|
||||
user=$(docker inspect -f '{{.Config.User}}' "$(docker compose ps -aq "$service")")
|
||||
echo "$service: $user"
|
||||
case "${user%%:*}" in ""|0|root) exit 1 ;; esac
|
||||
done
|
||||
|
||||
- name: Корректная остановка реплик API по SIGTERM
|
||||
run: |
|
||||
start=$(date +%s)
|
||||
docker compose kill --signal SIGTERM api
|
||||
codes=$(docker wait $(docker compose ps -aq api) | sort -u)
|
||||
elapsed=$(( $(date +%s) - start ))
|
||||
echo "остановка за ${elapsed} с, коды завершения ${codes}"
|
||||
test "$elapsed" -le 10 && test "$codes" = "0"
|
||||
|
||||
- name: Журналы стенда
|
||||
if: always()
|
||||
run: docker compose logs --tail 80
|
||||
+10
@@ -30,3 +30,13 @@ coverage/
|
||||
.vscode/
|
||||
# Keep small, sanitized evidence summaries in evidence/ under version control.
|
||||
# Inspect git diff --cached: ignore rules are not a secret scanner.
|
||||
|
||||
# Тулчейн студенческой реализации
|
||||
.pytest_cache/
|
||||
.ruff_cache/
|
||||
.ty_cache/
|
||||
|
||||
# Локальные настройки агента
|
||||
.claude/settings.local.json
|
||||
|
||||
.verification/
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
## Что это за репозиторий
|
||||
|
||||
Skeleton лабораторной 4 курса «Мессенджер под нагрузкой» (МТУСИ, 7 независимых лаб) и решение в ветке `solution/bpi2303-kostylev`. Skeleton даёт задание (`README.md`), нормативный контракт (`contracts/`) и публичные black-box проверки (`tests/`). Реализация API, Dockerfile, compose-стенд, конфигурация балансировщика и S3, миграции, хуки отказов и собственные тесты — студенческие; решение лабы 4 построено переносом решения лабы 3.
|
||||
|
||||
Лаба 4: вложения-изображения в сообщениях. Метаданные вложений живут в PostgreSQL, байты — в приватном bucket S3-совместимого хранилища **RustFS** внутри стенда, а не в БД и не на диске реплики. `POST /api/v1/chats/{chat_id}/attachments` принимает сырые байты одного PNG/JPEG (≤10 MiB, ≤40 млн пикселей, тип по содержимому) и отвечает `201 ready` только после записи объекта и фиксации метаданных; скачивание идёт только через API участникам чата. Сообщение несёт `attachment_ids`. Остаётся всё из лабы 3: HAProxy перед ≥2 репликами, `X-Instance-Id`, обязательный `Idempotency-Key`, хуки остановки реплики и рестарта стенда.
|
||||
|
||||
## Команды
|
||||
|
||||
```sh
|
||||
make check # offline: структура skeleton, JSON, локальные $ref, синтаксис Python, версия 1.4.0
|
||||
make validate # полная валидация OpenAPI 3.1: openapi-spec-validator из .venv (make install); PYTHON= — свой интерпретатор
|
||||
make install # uv sync --group dev
|
||||
make lint # ruff check + ruff format --check + ty check; обязательно перед commit
|
||||
make student-test # собственные тесты (поднимают одноразовые PostgreSQL, Valkey и RustFS в Docker)
|
||||
make student-test PYTEST_ARGS='-v -k drain' # выборка тестов с именами
|
||||
|
||||
make secrets # локальные secrets/* (включая root и API-ключи S3) и .env из .env.example
|
||||
make up # docker compose up -d --build --wait: postgres, valkey, s3 → migrate, s3-init → api ×2 → lb; затем ждёт все реплики UP в LB
|
||||
make test # публичный smoke через LB (BASE_URL по умолчанию — docker compose port lb 8080, без стенда http://localhost:8080)
|
||||
make test-persistence ACTION_SCRIPT=scripts/restart.sh # рестарт lb, api, postgres, valkey, s3 с сохранением volumes
|
||||
make test-failover ACTION_SCRIPT=scripts/stop-one.sh # остановка реплики TARGET_INSTANCE_ID под запросами
|
||||
make test-faults # тесты стенда через LB (make faults-evidence → evidence/faults.txt)
|
||||
make restore # вернуть остановленные реплики и дождаться их в ротации LB
|
||||
make limits # лимиты, порты, секреты, volumes и сети всех сервисов против политики → evidence/limits.txt
|
||||
make lb-check # один токен на обеих репликах и 20 гонок одного Idempotency-Key через LB
|
||||
make retry-check # потерянный ответ (запись ждёт блокировку чата) на живой и на убитой реплике, повтор тем же ключом
|
||||
make scenario SCENARIO=failover-upload # upload → стоп обслужившей реплики → download через другую, SHA-256
|
||||
make scenario SCENARIO=s3-outage # S3 остановлен: upload/download 503, тексты работают
|
||||
make s3-check # приватный bucket вне API, ограниченные ключи API и сверки, консоль, повторный bootstrap
|
||||
make upload-faults # «4»: негативный suite через LB, компенсация, уборка после grace, обрыв и зависание тела
|
||||
make upload-memory ARGS='--out evidence/upload-memory/run1' # «4»: пик памяти реплик и LB при 10/20/40 загрузках 1 и 10 MiB
|
||||
make load-evidence # tests/load.py ×3 клиентом-контейнером в сети edge → evidence/load.txt, evidence/load/
|
||||
make overload-leftovers # после upload-memory: намерения от 503, сверка и уборка после grace → evidence/overload-leftovers.txt
|
||||
make clean-clone-evidence # блок «Запуск из чистого clone» REPORT.md в свежем клоне рядом → evidence/clean-clone.txt
|
||||
make reconcile ARGS="--apply" # трек 2: сверка PostgreSQL ↔ S3 (без ARGS — dry-run), one-off сервис reconciler
|
||||
make reconcile-demo # трек 2: внедрение ошибок, dry-run/apply/повтор, безопасность, нагрузка → evidence/reconcile/
|
||||
make reconcile-cost # трек 2: стоимость обхода 1k/10k/30k/100k × 3 (чистый стенд) → evidence/reconcile/cost/
|
||||
make down # остановка без удаления данных (с профилями bench, tools, reconcile)
|
||||
```
|
||||
|
||||
`tests/*.py` и `scripts/*.py` — только stdlib, синтаксис Python 3.9 (`make check` разбирает их системным `python3`). Проверки сами не поднимают стенд. Хуки `scripts/stop-one.sh` и `scripts/restart.sh` запускаются тестом без shell, поэтому исполняемые и с shebang. Данные не удаляются автоматически: каждый прогон smoke создаёт новых синтетических пользователей и вложения.
|
||||
|
||||
## Архитектура skeleton
|
||||
|
||||
- `lab.json` — единственный источник номера лабы (4). От него зависят `scripts/check_contract.py` (`info.version == 1.4.0`, путь `/api/v1/attachments/{attachment_id}`, `X-Instance-Id` во всех ответах) и `tests/client.py` (ветки `LAB>=4`: upload `tests/fixtures/sample.png`, сообщение с `attachment_ids`, скачивание вторым участником, `404` постороннему).
|
||||
- `tests/client.py` шлёт `Accept: application/json` на каждый запрос, включая скачивание картинки, и считает любой `30x` ошибкой: content negotiation и redirect на S3 не делаются.
|
||||
- `tests/resilience.py`: перед хуком загружается `sample.png`; после `persistence` метаданные и `/content` запрашиваются **один раз** без повторов с таймаутом 2 с, поэтому хук рестарта ждёт здоровья S3. В `failover` `TARGET_INSTANCE_ID` — реплика, обслужившая первое **сообщение**, а не upload; остановку реплики, принявшей upload, доказывает `scripts/lab4_scenarios.py failover-upload`.
|
||||
- `contracts/infrastructure.json` — роли `api` (≥2, без локального состояния), `postgres` (метаданные, persistent volume), `load_balancer`, `s3` (приватный bucket, persistent volume), опциональный `redis_or_valkey`. `compose.example.yaml` — только worksheet ролей.
|
||||
|
||||
## Ключевые правила контракта (lab4)
|
||||
|
||||
- Upload: `POST /api/v1/chats/{chat_id}/attachments?filename=…`, тело — сырые байты (не multipart), `Content-Type: image/png|image/jpeg`. Сначала `401`, затем объявленный `Content-Length` (`413`). Дедлайн запроса продлевается переданными байтами тела по `MIN_TRANSFER_RATE_BYTES` (128 KiB/s): медленный, но идущий upload и скачивание не обрываются. `>10485760` байт или `>40 000 000` пикселей — `413`; тип по содержимому не совпал, битый файл, иной `Content-Type` — `415`; `filename` пустой, пробельный, длиннее 255 code points или с `/`, `\`, NUL, управляющими символами — `400`. Object key назначает сервер. `201 Attachment` (`status=ready`, `variants=[]`, `error_code=null`, lowercase `sha256`, `size_bytes` оригинала) — только после записи в S3 и `COMMIT` метаданных. Кода `409` у upload нет.
|
||||
- `GET /api/v1/attachments/{id}` и `/content?variant=original` — только участникам чата вложения, посторонний и неизвестный id — `404`. Скачивание отдаёт исходные байты с истинным `Content-Type`, без redirect, с `X-Request-Id` и `X-Instance-Id`.
|
||||
- `Message.attachment_ids` обязателен в каждом ответе (`[]` у текстовых, в том числе в replay и истории). `CreateMessage.attachment_ids` — 0–10 уникальных UUID (иначе `400`), только вложения этого чата (иначе `404`); `text=""` допустим только с вложением. Отсутствующий `attachment_ids` и `[]` — одно тело для `Idempotency-Key`.
|
||||
- Сохраняются правила lab3 (`X-Instance-Id`, `Idempotency-Key`, LB не повторяет `POST /users`, `/chats`, upload), lab2 (`Basic` только на входе, `Bearer` дальше, `WWW-Authenticate` по пути, logout `204`, TTL) и lab1 (формат ошибки, посторонний → `404`, keyset `(created_at, id)`, время RFC 3339 с `Z`).
|
||||
|
||||
## Правила работы с репозиторием
|
||||
|
||||
- Поздние исправления ЛР3 перенесены из immutable snapshot `ba78cf6da439a4ec60fd59389583b38285b0b2b1`; диапазон от `f591fa64e525679a31c9db7f942d36c789e1a12f` и применимость evidence описаны в `evidence/forward-port.md`.
|
||||
- Только свой Compose project `mtusi2026lab4` либо `hlpipe-lab4-*` под разрешённым working_dir; HTTP 18084, остальные host-порты 24000–24999. `NO_PROXY` и `no_proxy`: localhost,127.0.0.1,::1. Чужие контейнеры не останавливать, WIP и volumes сохранять. Совместный Docker допустим; эксклюзивность только в финальном release.
|
||||
- Runner назначает одного агента переноса и реализации 3→4→5 с evidence, затем review → отдельный fix при major → re-review, отдельный report и одного агента нагрузки/публикации. Грейды одной лабы последовательны; review/report предыдущей могут идти параллельно реализации следующей. Самостоятельных агентов, push и Telegram нет; runner/state не менять. Публикация только в назначенной фазе PUBLICATION в private MTUCI-Labs.
|
||||
- Исправлять критичное: major — доказанная угроза безопасности, данным, доступности или неработающий обязательный сценарий. Прочие minor остаются в findings и ограничениях REPORT и не блокируют этап. Пригодные измерения сохраняются; повторяется только затронутая или признанная непригодной серия с указанной причиной и фоном нагрузки.
|
||||
- Не менять и не ослаблять `contracts/`, `tests/`, `README.md`, `COURSE.md`, рубрику. Расхождение оформляется вопросом преподавателю.
|
||||
- Код — `src/messenger/`, миграции — `alembic/versions/NNNN_*.py`, конфигурация LB, нагрузки и S3 — `infra/`, хуки и обвязка стенда — `scripts/`, собственные тесты — `student-tests/`, обезличенные доказательства — `evidence/`.
|
||||
- PostgreSQL — SQLAlchemy 2 async с asyncpg как драйвером; ORM-модели внутри зон, Alembic принимает legacy-схему только после сверки журнала и каталога.
|
||||
- Без комментариев в коде; отступы — пробелы (`[tool.ruff.format]`); Python 3.14 без `from __future__ import annotations` и `TYPE_CHECKING`.
|
||||
- Порты реплик API и S3 наружу не публикуются: только LB. Нет `container_name`, `hostname` и общего `INSTANCE_ID` у сервиса `api`. Root-ключи S3 есть только у `s3`, `s3-init` и `s3-cli`; API получает отдельный ключ только на объекты bucket (`Get/Put/DeleteObject`), сверка — свой ключ (`ListBucket`, `Get/DeleteObject`). `s3-init` и `s3-cli` работают от uid 65534, `migrate` и `reconciler` — с `init: true`.
|
||||
- `REPORT.md`: команды из чистого clone, архитектура, таблица «критерий → команда → наблюдение → файл», таблица отказов. PASS без команды не доказательство; для отказов — какой контейнер реально остановлен (`docker ps -a`).
|
||||
- Отчётный этап меняет только `REPORT.md`, `AGENTS.md` и `evidence/**/*.md`; код, конфиги, OpenSpec и сырые измерения не меняет. Docker и нагрузку не запускает. `make student-test` поднимает зависимости: сохранённый результат или collection не выдаётся за новый полный прогон.
|
||||
- Ветка `solution/bpi2303-kostylev`, коммиты на английском, перед каждым — `make lint`. Никогда не коммитить `.env`, `secrets/`, токены, ключи S3, volumes, дампы.
|
||||
|
||||
## Уровни оценки
|
||||
|
||||
Цель — **«5»**, трек 2 «Согласованность двух хранилищ (reconciler)». На snapshot `7a0ebac2aa9e1d8602054ca4c7c1b0c23d3bfad4` ревью прошло без открытых major. Финальная validation на `5e49502ae78fa48b992c97418e490666c699726d` с идентичным runtime закрыла динамический долг обоих PNG-фиксов: `evidence/load-validation.md`. Сохранён minor Unicode-surrogate → HTTP 500 (`src/messenger/messaging/entrypoints/schemas.py:35`, `evidence/forward-final-2026-11-06/known-minor-http.txt`).
|
||||
|
||||
Таблица критериев — в `REPORT.md`, команды и SHA — в `evidence/README.md`. Новая серия использует отдельный разрешённый project со свежими volumes; `reset-data`, `down -v`, reset/clean и удаление пользовательских данных запрещены. После Docker-этапа остановить только свои стенды, сохранив volumes. При изменениях проверять применимость по diff: повторять лишь затронутые/непригодные серии с причиной и фоном. Документационный коммит измерений не обесценивает.
|
||||
|
||||
Исторические upload-memory и конкурентные upload до PNG-фиксов не доказывают RAM/производительность нового инспектора. Финальная validation выполнена: 721 собственная проверка (719 + адресный повтор 2 setup errors RustFS), 7 stack, PNG/JPEG через LB/RustFS; матрица 3 слота × 1/10 MiB × 10/20/40 клиентов ×3: 840/840×201, API ≤440 MiB, OOM 0; 10 PNG 40 MP RGBA16: 10×201, API ≤270,828 MiB; upload при сверке: 3341×201, 67 проходов, lost_ready=0. Сырьё — `evidence/png-final-2026-11-06/`. Эти пригодные серии повторять без нового изменения/обоснованной непригодности запрещено. Контроль 64/3 оставлен историческим. 27 пригодных cost-замеров reconciler на `3402fd69d4922e500035a204d957be2cccaeb85a` сохраняются: измеряемый путь не менялся. Отрицательный вывод о линейности времени и разброс deep 100k не скрываются.
|
||||
|
||||
- **3** — RustFS и приватный bucket в общем стенде, метаданные в PostgreSQL; upload `201 ready` после записи и `COMMIT`, побайтное скачивание вторым участником, `404` постороннему, `403` анонимному GET в S3; лимиты 10 MiB / 40 MP, тип по содержимому, опасные `filename`; upload → стоп обслужившей реплики → download через другую; restart с volumes; недоступный S3 → `503` при работающих текстах; минимум lab3 сохранён.
|
||||
- **4** — негативные тесты файлов без `500`; компенсация сбоя между S3 и БД под блокировкой строки и фоновая уборка намерений `pending` старше `UPLOAD_GRACE_SECONDS` (`SKIP LOCKED`, объект раньше строки); слоты загрузки на воркер, ограниченные буферы сокетов LB и API, фиксированный порог `mmap` glibc и замер ≥10 одновременных загрузок; идемпотентный bootstrap, ограниченные ключи API, закрытая консоль, проверка приватности вне API.
|
||||
- **5** — трек 2: one-off `reconciler` (`python -m messenger.attachments.entrypoints.reconcile`, по умолчанию `--dry-run`) сливает отсортированный листинг bucket и keyset-страницы `attachments` по `object_key` (O(N) сравнений, O(страница) памяти; линейность времени в замерах не подтвердилась); объект без строки старше grace + skew удаляется после повторной проверки строки, готовые строки без объекта или с другими байтами уходят в карантин (миграция `0004`, `/content` → `404`) и не удаляются, `pending` старше grace удаляется (объект, затем строка), карантин снимается только после совпавшего SHA-256 объекта (и без `--deep`); удаление намерения — `audit.attachment_discarded reason=reconcile`; SQL-страниц `⌈N/страница⌉`; один проход за раз (`pg_try_advisory_lock`); повторный `--apply` — 0 действий; стоимость обхода 1k/10k/30k/100k объектов с закреплёнными CPU и отбраковкой прогонов с чужой нагрузкой (`evidence/reconcile/`).
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
FROM python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d AS builder
|
||||
|
||||
COPY --from=ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142 /uv /usr/local/bin/uv
|
||||
|
||||
ENV UV_COMPILE_BYTECODE=1 \
|
||||
UV_LINK_MODE=copy \
|
||||
UV_PYTHON_DOWNLOADS=never \
|
||||
UV_PROJECT_ENVIRONMENT=/opt/venv
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
COPY pyproject.toml uv.lock README.md ./
|
||||
RUN uv sync --locked --no-dev --no-install-project --no-editable
|
||||
|
||||
COPY src ./src
|
||||
RUN uv sync --locked --no-dev --no-editable
|
||||
|
||||
|
||||
FROM python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d AS runtime
|
||||
|
||||
ENV PYTHONUNBUFFERED=1 \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
PATH="/opt/venv/bin:$PATH" \
|
||||
HOST=0.0.0.0 \
|
||||
PORT=8080 \
|
||||
LOG_LEVEL=INFO \
|
||||
WEB_CONCURRENCY=2 \
|
||||
REQUEST_TIMEOUT_SECONDS=8 \
|
||||
DRAIN_SECONDS=2 \
|
||||
SHUTDOWN_TIMEOUT_SECONDS=6
|
||||
|
||||
RUN groupadd --system --gid 10001 messenger \
|
||||
&& useradd --system --uid 10001 --gid messenger --no-create-home --shell /usr/sbin/nologin messenger
|
||||
|
||||
COPY --from=builder --chown=root:root /opt/venv /opt/venv
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY --chown=root:root alembic.ini ./
|
||||
COPY --chown=root:root alembic ./alembic
|
||||
|
||||
USER 10001:10001
|
||||
|
||||
EXPOSE 8080
|
||||
|
||||
HEALTHCHECK --interval=10s --timeout=3s --start-period=5s --retries=3 \
|
||||
CMD ["python", "-c", "import urllib.request,sys;sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8080/health/live',timeout=2).status==200 else 1)"]
|
||||
|
||||
CMD ["python", "-m", "gunicorn", \
|
||||
"--config", "python:messenger.gunicorn_conf", \
|
||||
"messenger.main:create_app()"]
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
# Памятка преподавателю — лабораторная 2
|
||||
# Памятка преподавателю — лабораторная 4
|
||||
|
||||
## Что подготовлено
|
||||
|
||||
|
||||
@@ -1,17 +1,53 @@
|
||||
PYTHON ?= python3
|
||||
BASE_URL ?= http://localhost:8080
|
||||
BASE_URL ?= http://$(shell docker compose port lb 8080 2>/dev/null || echo localhost:8080)
|
||||
CA_FILE ?=
|
||||
ACTION_SCRIPT ?=
|
||||
TLS_ARGS = $(if $(CA_FILE),--ca-file "$(CA_FILE)",)
|
||||
|
||||
.PHONY: help check validate test load test-persistence
|
||||
VENV ?= .venv
|
||||
UV ?= uv
|
||||
COMPOSE ?= docker compose
|
||||
RATE ?= 100
|
||||
FROM_REV ?=
|
||||
SHIFT ?= 0
|
||||
SCRIPTS_PY = scripts/capture_evidence.py alembic scripts/scaling.py tools scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py \
|
||||
scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py
|
||||
SCENARIO ?= failover-upload
|
||||
PSQL = $(COMPOSE) exec -T postgres psql -U messenger -d messenger -X -q
|
||||
PYTEST_ARGS ?=
|
||||
|
||||
.PHONY: help check validate test load test-persistence test-failover \
|
||||
install lint format typecheck student-test \
|
||||
secrets up down reset-data migrate ps logs restore lb-check failover-evidence stack-evidence \
|
||||
persistence-evidence storage-evidence cache-evidence test-faults faults-evidence seed explain retry-check \
|
||||
limits detection failover-each rolling-update level4 \
|
||||
scenario s3-check attachments-evidence upload-faults upload-memory \
|
||||
reconcile reconcile-demo reconcile-cost load-evidence clean-clone-evidence overload-leftovers
|
||||
|
||||
.NOTPARALLEL:
|
||||
|
||||
help:
|
||||
@echo "check: offline structure; validate: full OpenAPI (requirements-dev.txt); test: running API; load: synthetic writes"
|
||||
@echo "install/lint/typecheck/student-test: студенческий тулчейн"
|
||||
@echo "secrets/up/down/migrate/ps/logs: стенд Docker Compose; reset-data: намеренное удаление volumes"
|
||||
@echo "restore: вернуть остановленные реплики; lb-check: реплики на одном токене и 20 гонок одного ключа"
|
||||
@echo "retry-check: потерянный ответ на живой и на убитой реплике, повтор тем же ключом"
|
||||
@echo "level4 = limits, lb-check, detection, failover-each, rolling-update [FROM_REV=<коммит>]; RATE=$(RATE) rps"
|
||||
@echo "scenario SCENARIO=failover-upload|s3-outage → evidence/<сценарий>.json; s3-check; attachments-evidence"
|
||||
@echo "upload-faults: негативный suite, компенсация, уборка, обрыв и зависание загрузки против стенда"
|
||||
@echo "upload-memory ARGS='--out evidence/upload-memory/runN': пик памяти реплик при 10/20/40 загрузках 1 и 10 MiB"
|
||||
@echo "load-evidence: tests/load.py ×3 клиентом-контейнером в сети edge стенда → evidence/load.txt, evidence/load/"
|
||||
@echo "overload-leftovers: после upload-memory — намерения от 503, сверка и уборка после grace → evidence/overload-leftovers.txt"
|
||||
@echo "clean-clone-evidence: блок «Запуск из чистого clone» в свежем клоне рядом → evidence/clean-clone.txt"
|
||||
@echo "reconcile ARGS='--apply --deep': сверка PostgreSQL ↔ S3 (без ARGS — dry-run); трек 2:"
|
||||
@echo "reconcile-demo: внедрение ошибок, dry-run/apply/повтор, безопасность, нагрузка → evidence/reconcile/"
|
||||
@echo "reconcile-cost: стоимость обхода 1k/10k/30k/100k объектов × 3 прогона на чистом стенде → evidence/reconcile/cost/"
|
||||
@echo "BASE_URL=$(BASE_URL) (порт lb из docker compose); failover-evidence SHIFT=1 — сдвиг round robin на вторую реплику"
|
||||
|
||||
check:
|
||||
$(PYTHON) scripts/check_contract.py
|
||||
|
||||
validate: PYTHON := $(if $(wildcard $(VENV)/bin/python),$(VENV)/bin/python,$(PYTHON))
|
||||
validate:
|
||||
$(PYTHON) scripts/validate_openapi.py
|
||||
|
||||
@@ -24,3 +60,152 @@ load:
|
||||
test-persistence:
|
||||
@test -n "$(ACTION_SCRIPT)" || (echo "Set ACTION_SCRIPT to your implemented executable restart hook"; exit 2)
|
||||
$(PYTHON) tests/resilience.py --mode persistence --action-script "$(ACTION_SCRIPT)" --base-url "$(BASE_URL)" $(TLS_ARGS)
|
||||
|
||||
test-failover:
|
||||
@test -n "$(ACTION_SCRIPT)" || (echo "Set ACTION_SCRIPT to your implemented executable stop-one hook"; exit 2)
|
||||
$(PYTHON) tests/resilience.py --mode failover --action-script "$(ACTION_SCRIPT)" --base-url "$(BASE_URL)" $(TLS_ARGS)
|
||||
|
||||
install:
|
||||
$(UV) sync --group dev
|
||||
|
||||
lint:
|
||||
$(VENV)/bin/ruff check src student-tests $(SCRIPTS_PY)
|
||||
$(VENV)/bin/ruff format --check src student-tests $(SCRIPTS_PY)
|
||||
$(VENV)/bin/ty check src student-tests alembic
|
||||
|
||||
format:
|
||||
$(VENV)/bin/ruff check --fix src student-tests $(SCRIPTS_PY)
|
||||
$(VENV)/bin/ruff format src student-tests $(SCRIPTS_PY)
|
||||
|
||||
typecheck:
|
||||
$(VENV)/bin/ty check src student-tests alembic
|
||||
|
||||
student-test:
|
||||
$(VENV)/bin/pytest $(PYTEST_ARGS)
|
||||
|
||||
secrets:
|
||||
sh scripts/secrets.sh
|
||||
|
||||
up: secrets
|
||||
$(COMPOSE) up -d --build --wait
|
||||
scripts/lb-wait.sh
|
||||
|
||||
down:
|
||||
$(COMPOSE) --profile bench --profile tools --profile reconcile down
|
||||
|
||||
reset-data:
|
||||
@echo "Удаляются volumes с учебными данными стенда"
|
||||
$(COMPOSE) --profile bench --profile tools --profile reconcile down -v --remove-orphans
|
||||
|
||||
migrate:
|
||||
$(COMPOSE) run --rm migrate
|
||||
|
||||
ps:
|
||||
$(COMPOSE) ps
|
||||
|
||||
logs:
|
||||
$(COMPOSE) logs --tail 200
|
||||
|
||||
restore:
|
||||
scripts/restore.sh
|
||||
|
||||
lb-check:
|
||||
$(PYTHON) scripts/probe.py check --rounds 20 --out evidence/lb-check.json
|
||||
|
||||
retry-check:
|
||||
$(PYTHON) scripts/probe.py retry --out evidence/retry.json
|
||||
|
||||
limits:
|
||||
$(PYTHON) scripts/limits.py --out evidence/limits.txt
|
||||
|
||||
detection:
|
||||
rm -rf evidence/detection
|
||||
for run in 1 2 3; do sh scripts/detection.sh evidence/detection/run$$run || exit 1; done
|
||||
|
||||
failover-each:
|
||||
RATE=$(RATE) sh scripts/failover-each.sh evidence/failover-each
|
||||
|
||||
rolling-update:
|
||||
ifneq ($(FROM_REV),)
|
||||
git archive "$(FROM_REV)" | docker build -q -t "mtusi-messenger:$(FROM_REV)" -
|
||||
API_IMAGE="mtusi-messenger:$(FROM_REV)" $(COMPOSE) up -d --no-deps --force-recreate --wait api
|
||||
scripts/lb-wait.sh
|
||||
endif
|
||||
$(COMPOSE) build -q api
|
||||
RATE=$(RATE) $(PYTHON) scripts/probe.py load --expect replace --duration 90 \
|
||||
--out evidence/rolling$(if $(FROM_REV),-from-$(FROM_REV)) \
|
||||
-- scripts/rolling-update.sh
|
||||
|
||||
level4: limits lb-check detection failover-each rolling-update
|
||||
|
||||
scenario:
|
||||
$(PYTHON) scripts/lab4_scenarios.py $(SCENARIO) --base-url "$(BASE_URL)" --restore --output evidence/$(SCENARIO).json
|
||||
scripts/lb-wait.sh
|
||||
|
||||
s3-check:
|
||||
sh scripts/s3-check.sh > evidence/s3-security.txt; status=$$?; cat evidence/s3-security.txt; exit $$status
|
||||
|
||||
attachments-evidence:
|
||||
sh scripts/evidence-attachments.sh
|
||||
|
||||
upload-faults:
|
||||
{ echo "date: $$(date -u +%FT%TZ) commit: $$(git rev-parse --short HEAD) BASE_URL=$(BASE_URL)"; \
|
||||
BASE_URL="$(BASE_URL)" $(VENV)/bin/pytest -m stack -v -s --no-header --tb=short \
|
||||
-p no:cacheprovider student-tests/stack/test_attachment_faults.py; } > evidence/upload-faults.txt; \
|
||||
status=$$?; cat evidence/upload-faults.txt; exit $$status
|
||||
|
||||
upload-memory:
|
||||
$(PYTHON) scripts/upload_memory.py --base-url "$(BASE_URL)" $(ARGS)
|
||||
|
||||
load-evidence:
|
||||
sh scripts/load-evidence.sh
|
||||
|
||||
overload-leftovers:
|
||||
sh scripts/overload-leftovers.sh
|
||||
|
||||
clean-clone-evidence:
|
||||
sh scripts/clean-clone.sh
|
||||
|
||||
reconcile:
|
||||
$(COMPOSE) run --rm --no-deps reconciler $(ARGS)
|
||||
|
||||
reconcile-demo:
|
||||
$(PYTHON) scripts/reconcile_track.py demo --base-url "$(BASE_URL)" $(ARGS)
|
||||
|
||||
reconcile-cost:
|
||||
$(PYTHON) scripts/reconcile_track.py cost $(ARGS)
|
||||
|
||||
failover-evidence:
|
||||
sh scripts/failover-evidence.sh "$(BASE_URL)" "$(SHIFT)"
|
||||
|
||||
stack-evidence:
|
||||
sh scripts/evidence-stack.sh
|
||||
|
||||
persistence-evidence:
|
||||
sh scripts/persistence-evidence.sh "$(BASE_URL)" > evidence/persistence.txt
|
||||
cat evidence/persistence.txt
|
||||
|
||||
storage-evidence:
|
||||
sh scripts/evidence-storage.sh
|
||||
|
||||
cache-evidence:
|
||||
sh scripts/evidence-cache.sh > evidence/cache.txt
|
||||
cat evidence/cache.txt
|
||||
|
||||
test-faults:
|
||||
BASE_URL="$(BASE_URL)" $(VENV)/bin/pytest -m stack -v -s student-tests/stack
|
||||
|
||||
faults-evidence:
|
||||
{ echo "date: $$(date -u +%FT%TZ) commit: $$(git rev-parse --short HEAD)"; echo "\$$ make test-faults"; \
|
||||
$(MAKE) --no-print-directory test-faults 2>&1; } | sed 's#$(CURDIR)#<repo>#g' > evidence/faults.txt
|
||||
cat evidence/faults.txt
|
||||
grep -qE '^=+ [0-9]+ passed in' evidence/faults.txt
|
||||
|
||||
seed:
|
||||
$(PSQL) -f - < scripts/sql/seed-history.sql
|
||||
|
||||
explain: seed
|
||||
{ echo "date: $$(date -u +%FT%TZ) commit: $$(git rev-parse --short HEAD)"; \
|
||||
echo "\$$ make explain # scripts/sql/explain-history.sql"; \
|
||||
$(PSQL) -f - < scripts/sql/explain-history.sql; } > evidence/explain-history.txt
|
||||
cat evidence/explain-history.txt
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Лабораторная 2. PostgreSQL, сессии и Redis/Valkey
|
||||
# Лабораторная 4. S3-хранилище и вложения
|
||||
|
||||
Перенести состояние в постоянное хранилище и связать запросы с аутентифицированным пользователем. Подтверждённые сообщения и действующие сессии переживают перезапуск.
|
||||
Разделить метаданные и бинарное содержимое, добавить изображения в сообщения и сохранить доступ к ним при переключении API-реплики.
|
||||
|
||||
Это **skeleton**, а не готовый сервер. Здесь есть задание, контракт и публичные проверки. Реализацию приложения, Dockerfile и требуемую инфраструктуру пишет студент. Язык и framework свободные; UI не обязателен.
|
||||
|
||||
@@ -8,41 +8,41 @@
|
||||
|
||||
## Откуда начинаем
|
||||
|
||||
Перенесите собственный минимум lab1. Изменения API: password при регистрации, Basic для создания сессии, Bearer вместо X-User-Id для бизнес-запросов.
|
||||
Перенесите минимум lab3. Дополняются поля сообщения и endpoint вложений; прежние текстовые POST остаются валидными.
|
||||
|
||||
**Архитектура:** Клиент → API → PostgreSQL; на 4 появляется Redis ИЛИ Valkey для сессий/кэша.
|
||||
**Архитектура:** Клиент → LB → API × 2+ → PostgreSQL (метаданные) + приватное S3-хранилище (байты).
|
||||
|
||||
## Что сделать
|
||||
|
||||
1. Опишите таблицы, ограничения, индексы и транзакционные границы до написания запросов. Добавьте версионируемые миграции.
|
||||
2. Перенесите пользователей, чаты, членство и сообщения в PostgreSQL. Не используйте SQLite как замену PostgreSQL в принимаемом стенде.
|
||||
3. Реализуйте регистрацию с password KDF, вход через Basic, opaque Bearer-сессию, expiry и logout. Действующие сессии тоже должны сохраняться.
|
||||
4. Подготовьте единый локальный запуск API и БД (Compose рекомендуется уже здесь), подключите именованные volumes.
|
||||
5. Пройдите smoke и тест полного перезапуска. На 4 добавьте Redis/Valkey с конкретной ролью и повторите эксперимент отказа.
|
||||
1. Выберите один S3-совместимый сервер: MinIO, SeaweedFS или RustFS. Закрепите проверенную версию образа и опишите создание приватного bucket.
|
||||
2. Спроектируйте Attachment и связь с чатом/сообщением. S3 object key назначает сервер, filename используется только для отображения.
|
||||
3. Реализуйте синхронный binary upload PNG/JPEG с ограничениями, сохранение метаданных и авторизованный download через API.
|
||||
4. Включите S3 и persistent volumes в общий Compose/Swarm. Проверьте, что другая API-реплика скачивает файл после остановки загрузившей.
|
||||
5. Проверьте недоступность S3, чужой чат и restart всего стенда.
|
||||
|
||||
## Оценка «3»: работающий минимум
|
||||
|
||||
- Минимум lab1 сохранён с объявленным переходом на новую аутентификацию. Все обязательные данные и ещё действующие сессии переживают рестарт API и хранилищ без удаления volumes.
|
||||
- PostgreSQL содержит пользователей, чаты, участников и сообщения. Есть первичные/внешние ключи, уникальность username и миграции с нуля. Составная операция создания чата выполняется атомарно.
|
||||
- Basic применяется только на POST /auth/sessions, остальные endpoint требуют Bearer. Есть срок жизни сессии и отзыв; X-User-Id не даёт доступ. Пароли уже хешируются password KDF.
|
||||
- make test и make test-persistence ACTION_SCRIPT=scripts/restart.sh проходят. Миграции повторно запускаются без дублирования данных. Secrets/volumes не попадают в Git.
|
||||
- Все обязательные возможности lab3 сохранены. S3-сервер находится в общем стенде; метаданные вложений живут в PostgreSQL, байты — в приватном bucket, не в БД или локальном каталоге API.
|
||||
- POST вложения возвращает 201 ready после сохранения; оригинал скачивается побайтно. Message с attachment_ids читается участниками чата. Публичный smoke проверяет checksum и отсутствие доступа у постороннего.
|
||||
- Принимаются реальные PNG/JPEG ≤10 MiB и ≤40 млн пикселей; MIME проверяется по содержимому, опасные filename отклоняются, лимиты применяются до неограниченного чтения/декодирования.
|
||||
- Upload → остановка обслужившей API-реплики → download через другую возвращает тот же файл. Restart с сохранением volumes сохраняет метаданные и оригинал.
|
||||
|
||||
## Оценка «4»: уровень стажёра/джуна
|
||||
|
||||
Весь уровень 3 **и все** пункты ниже:
|
||||
|
||||
- Redis ИЛИ Valkey реально используется для сессий с TTL либо осмысленного кэша. Опишите источник истины, политику expiry и инвалидации. Если это единственное хранилище сессий, включите persistence, выдерживающее проверяемый рестарт; один TTL не сохраняет данные.
|
||||
- Используются параметризованные запросы/безопасный ORM, ограниченный connection pool и таймауты. Проверены гонка регистрации одинакового username и rollback составной операции.
|
||||
- Индекс истории обоснован запросом и EXPLAIN (ANALYZE, BUFFERS) на ≥10 000 сообщениях; пагинация не использует полный scan всей истории на каждую страницу.
|
||||
- Есть автоматическая проверка expiry при коротком SESSION_TTL_SECONDS, logout, неверного пароля и недоступности Redis/БД. При потере сервиса приложение возвращает ограниченную ошибку/корректно обходит кэш, не принимает запрос от анонимного пользователя как доверенный.
|
||||
- Тесты покрывают слишком большой файл, повреждённое изображение, подмену MIME, чужое attachment_id и привязку файла другого чата. Пользовательская ошибка не превращается в 500.
|
||||
- Прерывание загрузки и отказ между записью S3/БД не оставляют вечных неучтённых объектов: есть документированный механизм компенсации/сверки/очистки с безопасным grace period.
|
||||
- Лимиты/таймауты и потоковая обработка не позволяют одной загрузке занять неограниченную RAM. Проведён эксперимент ≥10 одновременных загрузок с измерением пика памяти.
|
||||
- Bucket bootstrap идемпотентен; API использует отдельные S3 credentials с ограниченными правами. Административная консоль не открыта в общей сети. Есть проверка private bucket вне API.
|
||||
|
||||
## Оценка «5»: исследование и доказанный результат
|
||||
|
||||
Весь уровень 4 **и один законченный трек на выбор**. Приведите гипотезу, повторяемую методику, результаты и ограничения.
|
||||
|
||||
**Трек 1. Согласованность кэша.** Покажите на конкурентном сценарии stale read или cache stampede, реализуйте защиту и измерьте число SQL-запросов/latency до и после. Докажите read-after-write из контракта и восстановление после рестарта кэша; одной установки Redis недостаточно.
|
||||
**Трек 1. Прямой upload.** Добавьте отдельный безопасный presigned-upload flow: короткая подпись, уникальный ключ, проверка принадлежности, размера/checksum при complete, обработка незавершённой загрузки. Сохраните обязательный proxy endpoint. Сравните трафик и RAM API при ≥2 размерах файлов, покажите негативные сценарии.
|
||||
|
||||
**Трек 2. План запроса и восстановление.** Исследуйте ≥100 000 сообщений с неравномерными размерами чатов, сравните ≥2 индекса/запроса и стоимость записи. Сделайте backup → восстановление в новое хранилище → сверку количества и выбранных сообщений. Обсудите время восстановления и границы потери данных.
|
||||
**Трек 2. Согласованность двух хранилищ.** Разработайте повторяемый reconciler для orphan/missing объектов и метаданных. Внедрите ошибки на обеих сторонах, покажите обнаружение и безопасное исправление с dry-run/повторным запуском. Докажите, что свежие и используемые вложения не удаляются; измерьте стоимость обхода.
|
||||
|
||||
## Как начать и проверить
|
||||
|
||||
@@ -78,28 +78,28 @@ Smoke не выставляет оценку автоматически. Про
|
||||
|
||||
## Сценарий защиты
|
||||
|
||||
1. Применить миграции к пустому volume, создать пользователей, войти, отправить сообщение.
|
||||
2. Запустить persistence-сценарий: он держит токен только в памяти тестового процесса; action script перезапускает API и все используемые хранилища, сохраняя volumes.
|
||||
3. После готовности тот же токен читает те же данные; после logout — 401. Повторить с отдельным коротким TTL для проверки expiry.
|
||||
4. Для 4: показать роль Redis/Valkey, план SQL и ошибку зависимости. Для 5: выполнить выбранный эксперимент.
|
||||
1. Загрузить fixture, отправить message с attachment_ids, скачать как второй участник и сравнить SHA-256.
|
||||
2. С третьим пользователем получить 404 для метаданных и bytes; direct unauthenticated S3 GET не отдаёт объект.
|
||||
3. Выполнить persistence и failover: общий тест начиная с lab4 сохраняет и повторно читает оригинал через API.
|
||||
4. Показать, что на API нет обязательного локального файла оригинала. Для 4/5 — отказы и эксперименты.
|
||||
|
||||
## Что должно быть в PR
|
||||
|
||||
Исходники, Dockerfile, миграции, compose.yaml либо эквивалентный воспроизводимый запуск, scripts/restart.sh, собственные тесты и отчёт.
|
||||
Дополненный стенд, настройка bucket/политики, миграции Attachment, тесты файлов, описание согласованности БД/S3 и результаты.
|
||||
|
||||
Заполните `REPORT.md` и [шаблон PR](.github/pull_request_template.md), укажите целевую оценку/трек. Не меняйте обязательный контракт и публичные tests ради зелёного результата. Сохраняйте возможности предыдущих лабораторных в пределах [объявленных переходов](COURSE.md).
|
||||
|
||||
## Вопросы на защите
|
||||
|
||||
- Что подтверждает COMMIT и какие гарантии зависят от настройки durability?
|
||||
- Где хранятся сессии и что будет после flush/restart Redis?
|
||||
- Чем аутентификация отличается от доступа к конкретному чату?
|
||||
- Почему hash пароля и быстрый checksum имеют разные задачи?
|
||||
- Почему файл в Docker volume одной API-реплики не равен объектному хранилищу?
|
||||
- Когда можно честно вернуть 201?
|
||||
- Что произойдёт, если S3 PUT прошёл, а COMMIT в PostgreSQL — нет?
|
||||
- Почему случайный object key и приватный bucket не заменяют ACL приложения?
|
||||
|
||||
## Первичные материалы
|
||||
|
||||
- [PostgreSQL transactions](https://www.postgresql.org/docs/current/tutorial-transactions.html)
|
||||
- [PostgreSQL EXPLAIN](https://www.postgresql.org/docs/current/using-explain.html)
|
||||
- [Valkey persistence](https://valkey.io/topics/persistence/)
|
||||
- [Redis persistence](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/)
|
||||
- [Basic authentication](https://www.rfc-editor.org/info/rfc7617/)
|
||||
- [S3 object uploads](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html)
|
||||
- [Presigned URLs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html)
|
||||
- [MinIO](https://github.com/minio/minio)
|
||||
- [SeaweedFS](https://github.com/seaweedfs/seaweedfs)
|
||||
- [RustFS](https://docs.rustfs.com/)
|
||||
|
||||
@@ -1,37 +1,450 @@
|
||||
# Отчёт — лабораторная 2
|
||||
# Отчёт — лабораторная 4 «S3-хранилище и вложения»
|
||||
|
||||
- Автор, группа: TODO
|
||||
- Целевая оценка; выбранный трек 5, если нужен: TODO
|
||||
- Commit/PR: TODO после публикации
|
||||
- Использованные источники, библиотеки и помощь ИИ: TODO
|
||||
Автор: **Костылев Артём, БПИ2303**. Ветка: `solution/bpi2303-kostylev`.
|
||||
Цель — **«5», трек 2 «Согласованность двух хранилищ (reconciler)»**.
|
||||
|
||||
Финальная validation завершена 2026-11-06 на
|
||||
`5e49502ae78fa48b992c97418e490666c699726d`: runtime совпадает с прошедшим независимое
|
||||
ревью `7a0ebac2aa9e1d8602054ca4c7c1b0c23d3bfad4`. Динамический долг обоих PNG-фиксов
|
||||
закрыт, открытых major не выявлено; известный minor сохранён в ограничениях.
|
||||
**Заявлена «5», трек 2**, с отрицательным результатом гипотезы о линейном времени.
|
||||
[Аудит применимости](evidence/load-validation.md) объясняет, какие старые серии приняты
|
||||
без повторения и какие проверки выполнены заново. Публикация ещё не разрешена.
|
||||
|
||||
## Происхождение и применимость доказательств
|
||||
|
||||
| Состояние кода | SHA | Что подтверждено и что сохраняется |
|
||||
| --- | --- | --- |
|
||||
| Перенос основы ЛР3 на SQLAlchemy/Alembic | `8e4532f6f5c05d902b6e2c310f54320d95585904`, источник `f591fa64e525679a31c9db7f942d36c789e1a12f` | Полная серия уровней 3/4; историческая относительно поздних исправлений |
|
||||
| Стоимость reconciler | `3402fd69d4922e500035a204d957be2cccaeb85a` | 27 пригодных замеров: измеряемый обход, SQL/S3-адаптеры, зависимости и стенд не менялись; сохраняются с исходным SHA |
|
||||
| Поздние исправления ЛР3 | `75b93bbba5f29575d3fca1d013cc7e0770d88082`, источник `ba78cf6da439a4ec60fd59389583b38285b0b2b1` | Сессии, NUL/Basic, JSON ошибок LB; 492 теста, 19 stack-тестов, отказы и новые серии. Все 214 путей диапазона учтены в [манифесте переноса](evidence/forward-port.md) |
|
||||
| Декодируемость PNG | `9387c3d82ab77ea376eecce19a36a22803d35efb` | Битый IDAT с корректными CRC отклоняется; [offline-доказательства](evidence/png-validation-fix/NOTES.md) |
|
||||
| Единый IHDR | `7c8a1a64899dc0a1b5fa0949c0992ef40722eacf`, архив/ревью `7a0ebac2aa9e1d8602054ca4c7c1b0c23d3bfad4` | Повторный IHDR отклоняется; 407 offline-тестов. [Манифест](evidence/png-header-fix/manifest.json) совпадает с текущими семью файлами |
|
||||
|
||||
После `75b93bb…` в runtime изменились только `images.py` и новый `png.py` зоны attachments.
|
||||
Поэтому старые upload-memory и загрузки во время сверки **не доказывают** память и скорость
|
||||
нового инспектора. Результаты текстового пути, скачивания, persistence/failover, IAM/bootstrap
|
||||
сохраняются для неизменённых механизмов; это не повторные прогоны на текущем HEAD.
|
||||
Серия asyncpg 29–30 октября вынесена из основных таблиц в
|
||||
[исторический индекс](evidence/README.md#историческая-реализация-asyncpg).
|
||||
|
||||
[Индекс evidence](evidence/README.md) связывает команды, SHA и файлы.
|
||||
[Аудит отчётного этапа](evidence/report-audit-2026-11-06.md) содержит проверки исходных чисел,
|
||||
ссылок и границ изменений. Сырые результаты не редактировались.
|
||||
|
||||
## Запуск из чистого clone
|
||||
|
||||
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
|
||||
Нужны Git, Docker Compose v2, make, uv и Python 3.14 для приложения/собственных тестов;
|
||||
публичным проверкам достаточно системного Python 3.9+. Из основного репозитория можно
|
||||
создать свежий clone под разрешённым каталогом `.verification/`:
|
||||
|
||||
```sh
|
||||
git clone --no-hardlinks --branch solution/bpi2303-kostylev . .verification/hlpipe-lab4-defense
|
||||
cd .verification/hlpipe-lab4-defense
|
||||
export COMPOSE_PROJECT_NAME=hlpipe-lab4-defense
|
||||
export COMPOSE_FILE=compose.yaml
|
||||
export LB_HTTP_PORT=18084 LB_STATS_PORT=24845
|
||||
export BASE_URL=http://127.0.0.1:18084
|
||||
export NO_PROXY=localhost,127.0.0.1,::1
|
||||
export no_proxy=localhost,127.0.0.1,::1
|
||||
make secrets
|
||||
docker compose --env-file .env config --quiet
|
||||
make up
|
||||
make test
|
||||
make test-persistence ACTION_SCRIPT=scripts/restart.sh
|
||||
make test-failover ACTION_SCRIPT=scripts/stop-one.sh
|
||||
docker compose ps -a
|
||||
make restore
|
||||
make scenario SCENARIO=failover-upload
|
||||
make restore
|
||||
make scenario SCENARIO=s3-outage
|
||||
make s3-check
|
||||
make reconcile
|
||||
make migrate
|
||||
make down
|
||||
```
|
||||
|
||||
Имя clone/project должно быть новым; порт 18084 должен быть свободен. Занявший его чужой
|
||||
стенд не останавливается. `make secrets` создаёт локальные файлы и не перезаписывает
|
||||
существующие; содержимое секретов выводить не нужно. `make up` ждёт PostgreSQL/Valkey/RustFS,
|
||||
`alembic upgrade head`, bootstrap bucket и обе API-реплики в ротации LB. Повторный `make migrate`
|
||||
не меняет актуальную схему. `make down` сохраняет постоянные volumes; `reset-data`, `down -v`
|
||||
и удаление пользовательских данных в этом процессе не используются.
|
||||
|
||||
Сохранённое воспроизведение из свежего clone: **76,866 с, exit 0**, SHA
|
||||
`cf41f7c64dcf1dea6da0643f5793bd2e5dd1f03e`, проект `hlpipe-lab4-clean-20261105231220`.
|
||||
Smoke, persistence, failover, upload→stop→download, S3-outage, 31 проверка S3, dry-run и
|
||||
повторная миграция прошли; после down остались три volumes.
|
||||
[Полная команда и вывод](evidence/sqlalchemy-2026-11-06/grade4/clean-clone-command.txt).
|
||||
Это запуск **до позднего переноса и PNG-фиксов**, а не доказательство запуска нынешнего SHA.
|
||||
|
||||
Проверки исходников выполняются отдельно:
|
||||
|
||||
```sh
|
||||
make install
|
||||
make check
|
||||
make validate
|
||||
make lint
|
||||
make student-test
|
||||
openspec validate --all --strict --no-interactive
|
||||
```
|
||||
|
||||
`make student-test` использует одноразовые Docker-зависимости. На отчётном этапе выполнены
|
||||
только check/validate/lint/OpenSpec и чтение сохранённых доказательств; тесты с Docker и
|
||||
нагрузки не запускались. Сохранённые полные 492 теста относятся к `75b93bb…`, а 407 unit-тестов
|
||||
после PNG-фиксов — к `7c8a1a6…`; складывать эти числа в число тестов одного прогона нельзя.
|
||||
|
||||
## Архитектура и решения
|
||||
|
||||
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab1, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
|
||||
```text
|
||||
клиент → HAProxy :18084 → API ×2, по 2 UvicornWorker
|
||||
├→ PostgreSQL: метаданные, сессии, сообщения
|
||||
├→ Valkey: кэш сессий
|
||||
└→ RustFS: байты оригиналов
|
||||
migrate → PostgreSQL; s3-init → private bucket и IAM
|
||||
reconciler → PostgreSQL + RustFS, одноразовый сервис
|
||||
```
|
||||
|
||||
## Доказательства критериев
|
||||
Python 3.14, FastAPI, gunicorn/uvicorn-worker, dishka, Pydantic v2, structlog,
|
||||
**SQLAlchemy 2.0.45 async, asyncpg как драйвер, Alembic 1.16.5**, aiobotocore и Pillow 13.0.0.
|
||||
Версии зависимостей закреплены в [uv.lock](uv.lock), правила форматирования — пробелы в
|
||||
[pyproject.toml](pyproject.toml). Образы закреплены digest в [compose.yaml](compose.yaml):
|
||||
HAProxy 3.2.26, PostgreSQL 17.11, Valkey 8.1.11, RustFS 1.0.4, rc 0.1.36.
|
||||
|
||||
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
||||
Зоны `identity`, `messaging`, `attachments` содержат домен, прикладные порты и адаптеры;
|
||||
`shared` — платформу, `main.py` — сборку. Межзонная проверка членства идёт через порт
|
||||
`ChatMembership`, не через импорт инфраструктуры соседней зоны. Pydantic остаётся на HTTP-границе,
|
||||
ORM-модели `DeclarativeBase` — внутри зон. Сессии создаёт `async_sessionmaker`; UoW получает
|
||||
свою `AsyncSession` на транзакцию и явно делает COMMIT. Незавершённая транзакция откатывается
|
||||
при закрытии сессии. SQL строится ORM/Core с параметрами; `asyncpg` не используется как
|
||||
отдельный прикладной слой запросов. Отправка сообщения — Core CTE под блокировкой строки чата:
|
||||
квитанция `Idempotency-Key`, сообщение и ссылки на вложения фиксируются вместе.
|
||||
|
||||
Alembic [env.py](alembic/env.py) использует async engine и `run_sync`, advisory lock и
|
||||
транзакции миграций. Ревизии: `0001` — базовая схема, `0002` — ключи идемпотентности,
|
||||
`0003` — вложения/связи, `0004` — карантин. Legacy-схема принимается только после сверки
|
||||
журнала и каталога в [legacy.py](alembic/legacy.py); произвольный stamp вместо миграции не применяется.
|
||||
При подготовке отчёта через Context7 сверены закреплённые версии:
|
||||
[SQLAlchemy, сессии](https://github.com/sqlalchemy/sqlalchemy/blob/rel_2_0_45/doc/build/orm/session_basics.rst),
|
||||
[Alembic, async migrations](https://github.com/sqlalchemy/alembic/blob/rel_1_16_5/docs/build/cookbook.rst).
|
||||
|
||||
**Источники истины.** PostgreSQL хранит владельца/чат, имя, MIME, размер, SHA-256, размеры
|
||||
изображения, статус, object key и ссылки сообщений. Байты — только в приватном RustFS,
|
||||
ключ `attachments/<uuid>` назначает сервер. API read-only, без data volumes; `/tmp` — tmpfs.
|
||||
`attachments.object_key` имеет UNIQUE и `COLLATE "C"`; частичный индекс pending обслуживает
|
||||
уборку; FK `message_attachments (attachment_id, chat_id)` не даёт привязать чужой чат или
|
||||
удалить используемое вложение. История использует keyset `(created_at, id)`.
|
||||
|
||||
**Upload.** Сначала аутентификация (`401`), объявленный размер (`413`), членство (`404`),
|
||||
filename (`400`), MIME (`415`). Поток ограничен 10 485 760 байтами и считается также без
|
||||
Content-Length; SHA-256 вычисляется при чтении. Тело собирается в ограниченный буфер под
|
||||
`UPLOAD_CONCURRENCY=3` на worker; две проверки изображений на worker работают в потоках.
|
||||
Лимит 40 млн пикселей проверяется до распаковки. PNG требует единственный первый IHDR,
|
||||
проверку CRC и потоковую проверку zlib/Adler-32, длины строк и фильтров, включая Adam7.
|
||||
Дополнительные блоки распаковки — по 64 KiB, полного массива пикселей нет. JPEG проверяется
|
||||
через `draft` 1/8 и `load`. Ошибка изображения → `415` до намерения в БД/S3.
|
||||
Оба PNG из ревью отклоняются с 415 в ASGI-регрессиях и через настоящий LB/RustFS.
|
||||
После двух запросов число строк/объектов осталось 7/7; корректные PNG/JPEG скачаны
|
||||
вторым участником побайтно: [финальная сверка](evidence/png-final-2026-11-06/png-roundtrip.json).
|
||||
|
||||
Порядок публикации: `INSERT pending + COMMIT → PUT → UPDATE ready + COMMIT → 201`.
|
||||
Метаданные и скачивание доступны только участникам чата; чужой и неизвестный id дают `404`.
|
||||
Скачивание идёт через API кусками 64 KiB, с исходными байтами и истинным MIME, без redirect.
|
||||
`attachment_ids` присутствует во всех ответах сообщений, отсутствие и `[]` дают один fingerprint.
|
||||
|
||||
**Компенсация.** После отказа фиксации ready новая транзакция блокирует строку `FOR UPDATE`:
|
||||
ready не трогается; у pending удаляется объект, затем строка. При отмене/неизвестном исходе
|
||||
PUT намерение остаётся для уборки. Sweeper раз в 30 с выбирает до 100 pending старше 900 с
|
||||
через `FOR UPDATE SKIP LOCKED`, удаляет объект раньше строки и публикует
|
||||
`audit.attachment_discarded`. Готовые и используемые вложения не выбираются.
|
||||
|
||||
**Ресурсы и отказы.** У API 1 CPU/512 MiB на реплику, у LB 1 CPU/128 MiB,
|
||||
PostgreSQL 2 CPU/1 GiB, RustFS 1 CPU/512 MiB, Valkey 0,5 CPU/128 MiB;
|
||||
reconciler 1 CPU/256 MiB. Условия cost-эксперимента отличаются и указаны отдельно.
|
||||
Слоты дополнены буферами HAProxy 64 KiB, максимумом TCP-буфера API 256 KiB и
|
||||
`MALLOC_MMAP_THRESHOLD_=131072`. S3 connect/read = 1/2 с, две попытки; запрос — 8 с,
|
||||
LB — 10 с простоя. Дедлайн продлевается переданными байтами при пороге 128 KiB/s,
|
||||
поэтому идущая передача может длиться дольше 8 с. LB не повторяет upload.
|
||||
Readiness проверяет PostgreSQL, но не S3: отказ хранилища изображений сохраняет текстовый путь.
|
||||
|
||||
**Доступ к S3.** Только LB публикует host-порты на loopback. RustFS в internal-сети,
|
||||
консоль выключена. `s3-init` повторяемо создаёт private bucket, политики/пользователей и
|
||||
lifecycle очистки незавершённых multipart. API получает только Get/Put/DeleteObject своего
|
||||
bucket, reconciler — отдельный ключ ListBucket/Get/DeleteObject; root-ключи есть лишь у
|
||||
`s3`, `s3-init`, `s3-cli`. `s3-init`/`s3-cli` — uid 65534, migrate/reconciler — `init: true`.
|
||||
|
||||
## Критерий → команда → наблюдение → файл
|
||||
|
||||
Номера ниже — строки этого отчёта, а не дополнительная рубрика. **F** означает SHA
|
||||
`75b93bbba5f29575d3fca1d013cc7e0770d88082` до PNG-фиксов; **S** — исходную SQLAlchemy-серию,
|
||||
точный SHA команды находится в её raw. **V** — финальная validation на
|
||||
`5e49502ae78fa48b992c97418e490666c699726d`, runtime идентичен прошедшему review `7a0ebac…`.
|
||||
[Новые команды и результаты](evidence/load-validation.md) дополняют сохранённые проверки.
|
||||
|
||||
| Критерий | Команда | Наблюдение и применимость | Файл |
|
||||
| --- | --- | --- | --- |
|
||||
| Публичный smoke | make test | TODO | TODO |
|
||||
| Остальные пункты уровня 3 | TODO | TODO | TODO |
|
||||
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
|
||||
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
|
||||
| 3.1 Минимум ЛР3 | `make test`; `student-tests/tools/forward_network.py`; `make lb-check`; `make retry-check` | F: smoke, отзыв сессий на двух репликах, NUL/Basic PASS; S: 20×8 запросов с одним ключом, одна копия; потерянный ответ и kill без дублей | [smoke](evidence/forward-final-2026-11-06/smoke.txt), [сеть](evidence/forward-final-2026-11-06/network-regressions.txt), [LB](evidence/sqlalchemy-2026-11-06/grade3/lb-check-rerun.txt), [retry](evidence/sqlalchemy-2026-11-06/grade3/retry.txt) |
|
||||
| 3.2 Метаданные отдельно от байтов, приватность, stateless API | `make attachments-evidence`; `make limits` | S: внешняя сверка размера/SHA, изображения не в БД/API; F: лимиты, сети, mounts и порты PASS | [размещение](evidence/sqlalchemy-2026-11-06/grade3/storage-layout.txt), [лимиты](evidence/forward-final-2026-11-06/limits.txt) |
|
||||
| 3.3 `201 ready`, сообщения, checksum, чужому `404` | `make test`; `make student-test` | F: PASS, включая реальные PostgreSQL/RustFS-тесты; V: smoke и 721 собственная проверка прошла (719 + адресный повтор 2 setup errors) | [smoke](evidence/forward-final-2026-11-06/smoke.txt), [492 теста](evidence/forward-final-2026-11-06/offline-checks-rerun.txt), [V smoke](evidence/png-final-2026-11-06/smoke.txt) |
|
||||
| 3.4 PNG/JPEG, 10 MiB/40 MP, MIME/filename | `make test-faults`; `make student-test PYTEST_ARGS='-q --tb=short student-tests/unit'` | F: граничные файлы, MIME/имена → 4xx; текущие offline-регрессии: оба PNG из ревью → 415, без записи. V: оба файла → 415 через LB, число строк/объектов неизменно; PNG/JPEG скачаны побайтно | [старый suite](evidence/forward-final-2026-11-06/stack-faults.txt), [407 unit](evidence/png-header-fix/unit.txt), [регрессии](evidence/png-header-fix/targeted.txt), [V PNG/JPEG](evidence/png-final-2026-11-06/png-roundtrip.json) |
|
||||
| 3.5 Upload → остановка принявшей реплики → download | `make scenario SCENARIO=failover-upload` | F: target exited, сосед running, SHA совпал, download через 2,762 с | [JSON](evidence/forward-final-2026-11-06/failover-upload.json) |
|
||||
| 3.6 Restart с volumes и failover | `make persistence-evidence`; `make test-failover ACTION_SCRIPT=scripts/stop-one.sh` | F: данные/сессия сохранены; 130 записей, 0 ошибок, первая запись через 2,883 с | [persistence](evidence/forward-final-2026-11-06/persistence-rerun.txt), [failover](evidence/forward-final-2026-11-06/failover-d4c8d53d7433.txt) |
|
||||
| Отказ S3 | `make scenario SCENARIO=s3-outage` | F: upload/download 503 за 4,990/5,329 с; текст 201, метаданные/readiness 200; после возврата checksum равен | [JSON](evidence/forward-final-2026-11-06/s3-outage.json) |
|
||||
| 4.1 Пользовательские ошибки файлов не дают 500 | `make test-faults`; PNG ASGI-регрессии | F: 112 ответов в журнале негативного suite, 0×5xx; битый zlib и повторный IHDR добавлены позднее. V: 7 stack PASS, включая оба PNG; 54 ответа негативного теста, 0×5xx | [stack](evidence/forward-final-2026-11-06/stack-faults.txt), [PNG](evidence/png-header-fix/NOTES.md), [V suite](evidence/png-final-2026-11-06/upload-faults.txt) |
|
||||
| 4.2 Компенсация, crash и безопасная уборка | `make test-faults`; `make overload-leftovers` | F: COMMIT fault → 503, строка/объект удалены; обе реплики убиты между PUT/COMMIT, свежие/ready сохранены. S: 30 pending убраны после реальных 900 с, 1669 ready/объектов совпали | [отказы](evidence/forward-final-2026-11-06/stack-faults.txt), [grace без сдвига времени](evidence/sqlalchemy-2026-11-06/grade4/overload-leftovers.txt) |
|
||||
| 4.3 Ограниченная RAM, ≥10 загрузок | `make upload-memory ARGS='--slots 3 --out <новая-серия>'` | S: 36 точек; F: 3 точки, максимум 456,2 MiB, OOM 0, 238×201/2×503. V: 18 точек, 840/840×201, API ≤440 MiB, LB ≤31,1 MiB, OOM 0; 10×40 MP RGBA16 → 201, API ≤270,8 MiB | [S](evidence/sqlalchemy-2026-11-06/grade4/NOTES.md), [F](evidence/forward-port.md#повтор-затронутых-измерений), [V RAM](evidence/png-final-2026-11-06/memory-summary.json) |
|
||||
| 4.4 Bootstrap/IAM/закрытая консоль/private вне API | `make s3-check` | S: 31 PASS, anonymous GET 403, API ListBucket запрещён, повтор bootstrap ×2 неизменен; конфигурация не менялась | [сырой вывод](evidence/sqlalchemy-2026-11-06/grade3/s3-security.txt) |
|
||||
| 5.1 Внедрение, dry-run/apply/повтор, защита свежих и ссылок | F: `make reconcile-demo`; V: `python3 evidence/png-final-2026-11-06/concurrent-reconcile.py` | F: 13 PASS; 18 исправлений, повтор 0, ссылки сохранены; алгоритм не менялся. V: 3341×201 при 67 проходах, lost_ready=0, независимый SHA-аудит PASS | [demo](evidence/forward-final-2026-11-06/reconcile-demo/demo.json), [V load](evidence/png-final-2026-11-06/concurrent-reconcile/under-load.json) |
|
||||
| 5.2 Стоимость обхода | `make reconcile-cost ARGS='--out <новая-серия>'` | 27 замеров на `3402fd6…`, 1k/10k/30k/100k; страницы и память подтверждены, линейное время отвергнуто | [методика](evidence/sqlalchemy-2026-11-06/track5/NOTES.md), [сырьё](evidence/sqlalchemy-2026-11-06/track5/measurements/cost/) |
|
||||
|
||||
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
|
||||
## Отказы: наблюдаемое время и потери
|
||||
|
||||
## Эксперименты и отказы
|
||||
Основная таблица — **F**, проект `hlpipe-lab4-forward-final-20261106`.
|
||||
Времена — отдельные наблюдения, не SLA. Потерями считаются подтверждённые клиенту данные;
|
||||
намеренно повреждённые байты в demo отдельно отмечены. Состояния контейнеров сохранены
|
||||
в JSON/inspect и выводе хуков; неизвестные длительности не заменены нулём.
|
||||
|
||||
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
|
||||
|
||||
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
|
||||
| Воздействие / команда | Реально затронутый контейнер или объект | Наблюдение, время | Потери / файл |
|
||||
| --- | --- | --- | --- |
|
||||
| TODO | TODO | TODO | TODO |
|
||||
| `scenario failover-upload` | API `d4c8d53d7433` exited; `da922ab2ea09` running | Хук 2,734 с; download через 2,762 с от старта | Байты совпали; [JSON](evidence/forward-final-2026-11-06/failover-upload.json) |
|
||||
| `test-failover` | API `d4c8d53d7433`, подтверждён exited | 130 успешных записей, 0 failed_attempts; первая через 2,883 с | Дубликатов/потерь не обнаружено; [inspect и тест](evidence/forward-final-2026-11-06/failover-d4c8d53d7433.txt) |
|
||||
| `persistence-evidence` | API ×2, LB `ea6930f64089`, PG `3f3a870d89c0`, Valkey `07ea16e2fcfe`, S3 `5921e788f03d`; StartedAt изменились | Вся команда 13,179 с; это не точный интервал недоступности | Однократное чтение после hook сохранило данные/сессию; [до/после](evidence/forward-final-2026-11-06/persistence-rerun.txt) |
|
||||
| `scenario s3-outage` | S3 `5921e788f03d` exited | 503: upload 4,990 с, download 5,329 с; healthy через 5,760 с после старта | Готовый файл сохранился; текст 201; [JSON](evidence/forward-final-2026-11-06/s3-outage.json) |
|
||||
| `make test-faults`, PostgreSQL stop | Сервис postgres своего project; отдельного inspect в suite нет | 503 за 0,002–0,015 с; готовность через 2,1 с после старта | Прежний токен работает, 40 регистраций через 2 реплики без рестарта API; [stack](evidence/forward-final-2026-11-06/stack-faults.txt) |
|
||||
| `make test-faults`, Valkey stop | Сервис valkey своего project; отдельного inspect в suite нет | Живой токен 200 за 0,005 с; неизвестный 401 за 0,003 с; logout 503, после старта 204 | Отзыв не потерян; время восстановления не измерено; [stack](evidence/forward-final-2026-11-06/stack-faults.txt) |
|
||||
| COMMIT fault после PUT | Триггер в синтетическом сценарии | 503; сразу строка отсутствует и HEAD 404; audit compensation | Готовые не затронуты; [stack](evidence/forward-final-2026-11-06/stack-faults.txt) |
|
||||
| SIGKILL между PUT/COMMIT | По очереди API `da922ab2ea09`, `d4c8d53d7433` | Клиент 502; pending + объект. После сдвига за grace удаление через 26,1/1,5 с | Свежий остаток сохранён; ready в сообщении скачан с тем же SHA; [stack](evidence/forward-final-2026-11-06/stack-faults.txt) |
|
||||
| Обрыв / зависание тела | 256 KiB из 5 MiB / 1 KiB | Обрыв: 400 в журнале, нет intent; stall: 503 через 8,02 с | Новых видимых вложений нет; [stack](evidence/forward-final-2026-11-06/stack-faults.txt) |
|
||||
| Реальное ожидание grace, серия S | 30 pending: 27 после перегрузки, 3 после S3-outage | Наблюдение 847,812 с уже существовавших остатков; возраст не меняли, grace 900 с | 30 audit abandoned, затем 1669 ready = 1669 объектов; [вывод](evidence/sqlalchemy-2026-11-06/grade4/overload-leftovers.txt) |
|
||||
| Reconciler demo, порча обеих сторон | 3 orphan, 9 ready с потерянными/изменёнными байтами, 6 pending | Apply: 18 действий; повтор 0; полный demo 256,6 с | 9 ready и 6 ссылок сохранены, байты сами не восстанавливаются; [demo](evidence/forward-final-2026-11-06/reconcile-demo/demo.json) |
|
||||
| SIGTERM / S3 outage сверки | Одноразовый reconciler; S3 paused / остановлен | SIGTERM: 0,1 с, exit 143, lock освобождён; S3 outage: exit 1 за 2,32 с | Состояние не изменилось при outage; [demo](evidence/forward-final-2026-11-06/reconcile-demo/demo.json) |
|
||||
|
||||
## Ограничения и следующие шаги
|
||||
`failover-upload.json` содержит `restored=false`: сценарий сам не возвращал target;
|
||||
[отдельный restore](evidence/forward-final-2026-11-06/restore.txt) возвращает обе реплики.
|
||||
В конце фазы F все три её project остановлены, шесть постоянных volumes сохранены:
|
||||
[shutdown-state](evidence/forward-final-2026-11-06/shutdown-state.txt).
|
||||
Отчётный этап Docker не использовал. Состояние финальной validation-фазы и сохранённые
|
||||
volumes зафиксированы отдельно в [аудите](evidence/load-validation.md).
|
||||
|
||||
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
|
||||
## Память и запись сообщений
|
||||
|
||||
**Память S, до позднего переноса и PNG-фиксов.** Два API по 1 CPU/512 MiB, два worker,
|
||||
LB 128 MiB, RustFS 1 CPU/512 MiB; 3/64 слота × PNG около 1/10 MiB × 10/20/40 клиентов,
|
||||
три повтора каждой точки. Клиент с macOS, keep-alive, по два upload; перед точкой API/LB
|
||||
пересоздаются для нового cgroup `memory.peak`. Пик включает процессы, page cache и сокеты;
|
||||
p95 учитывает только 201. На сохранённом снимке фона — только свой project.
|
||||
[Методика и 36 raw-точек](evidence/sqlalchemy-2026-11-06/grade4/NOTES.md).
|
||||
|
||||
| Конфигурация | Ответы в 18 точках | Максимум API | OOM API/LB |
|
||||
| --- | --- | ---: | ---: |
|
||||
| 3 слота | 840/840 ×201 | 448,0 MiB | 0 |
|
||||
| 64 слота, исторический контроль | 813×201, 27×503 | 513,7 MiB | 0 |
|
||||
|
||||
**Память F, тоже до PNG-фиксов.** 40 клиентов, по два PNG 10 485 256 байт, 3 слота;
|
||||
три повтора дали 80/80/78 успешных ответов и 0/0/2 ответа 503. Пики API:
|
||||
456,2/438,1/450,4 MiB, LB: 31,8/28,5/29,1 MiB, OOM 0; p95 3641,3/4755,0/8091,8 мс.
|
||||
Причина двух 503 отдельно не локализована; уборка их возможных остатков в этой серии не
|
||||
измерялась. [Команды, фон и сырьё](evidence/forward-port.md#повтор-затронутых-измерений).
|
||||
Контроль 64/3 остаётся историческим; новый инспектор измерен отдельно в серии V ниже.
|
||||
|
||||
**Память V, после обоих PNG-фиксов.** На том же распределении ресурсов проведены
|
||||
ровно три новые серии штатных 3 слотов: ~1/10 MiB × 10/20/40 клиентов, два upload на
|
||||
клиента. Всего **18 точек, 840/840×201**, ошибок/таймаутов/OOM-kill нет.
|
||||
Максимум API **440,0 MiB / 512 MiB**, LB **31,1 MiB / 128 MiB**.
|
||||
[Команды, сырьё и фон](evidence/load-validation.md#новая-матрица-памяти),
|
||||
[пересчёт](evidence/png-final-2026-11-06/memory-summary.json).
|
||||
|
||||
| PNG / клиенты | Максимум API из 3 повторов, MiB | p95 успешных: медиана [min–max], мс |
|
||||
| --- | ---: | ---: |
|
||||
| ~1 MiB / 10 | 279,8 | 241,0 [134,3–257,7] |
|
||||
| ~1 MiB / 20 | 284,5 | 243,8 [226,6–310,2] |
|
||||
| ~1 MiB / 40 | 290,4 | 373,4 [367,6–417,6] |
|
||||
| ~10 MiB / 10 | 394,8 | 1520,3 [1236,1–3287,8] |
|
||||
| ~10 MiB / 20 | 440,0 | 2632,8 [2514,0–3632,6] |
|
||||
| ~10 MiB / 40 | 439,2 | 5971,4 [4256,0–6534,9] |
|
||||
|
||||
Дополнительный PNG 8000×5000 RGBA16: 311 100 байт на проводе, 320 005 000 байт
|
||||
распакованных строк. Перед нагрузкой файл независимо декодирован Pillow. 10 клиентов
|
||||
стартовали через barrier: **10×201**, 10 побайтных скачиваний вторым участником,
|
||||
API ≤270,828 MiB, LB ≤24,2 MiB, OOM 0. [Драйвер и raw](evidence/png-final-2026-11-06/high-depth.json).
|
||||
Это один сильно сжатый паттерн; он не представляет все PNG и не является SLO.
|
||||
|
||||
После нагрузок shallow-проверка: 859 ready = 859 объектов, ни pending, ни orphan,
|
||||
ни missing/size mismatch. SHA всех крупных объектов повторно не вычислялся.
|
||||
[Сверка](evidence/png-final-2026-11-06/post-memory-invariants.json). На снимках фона —
|
||||
только свой project; macOS, частоты, диск и кэш физического хоста не контролировались.
|
||||
|
||||
**Запись сообщений F.** `make load-evidence`: публичный `tests/load.py`, закрытый контур,
|
||||
64 клиента, 8 с, новый горячий чат на прогон, три повтора с паузой 15 с. Клиент-контейнер
|
||||
в edge-сети, 2 CPU/512 MiB; API ×2 по 1 CPU/512 MiB. На снимках до/после — только свой
|
||||
project, host load average 3,42→4,69. Физический хост не был выделен эксклюзивно.
|
||||
|
||||
| Повтор | RPS | p95 успешных, мс | Ошибок |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| 1 | 1428,577 | 100,062 | 0 |
|
||||
| 2 | 1456,338 | 98,948 | 0 |
|
||||
| 3 | 1529,320 | 98,475 | 0 |
|
||||
|
||||
Медиана **1456,338 RPS**, ошибок 0. [Команда](evidence/forward-final-2026-11-06/load-command.txt),
|
||||
[сырые JSON](evidence/forward-final-2026-11-06/load/), [фон](evidence/forward-final-2026-11-06/performance-before.txt).
|
||||
Это характеристика данного распределения ресурсов и одного чата, не SLO или предел архитектуры.
|
||||
Относительно ранней SQLAlchemy-серии 1529,186 RPS разница −4,8%; причинное влияние
|
||||
исправления из непарных серий не следует. PNG-изменения не вызываются текстовым запросом.
|
||||
|
||||
## Трек 2: согласованность двух хранилищ
|
||||
|
||||
`make reconcile` запускает `python -m messenger.attachments.entrypoints.reconcile`,
|
||||
по умолчанию dry-run; `ARGS='--apply --deep'` включает исправления и хеширование.
|
||||
Один проход сливает листинг S3 и keyset-страницы по `object_key` с индексом UNIQUE/COLLATE C.
|
||||
Алгоритм делает O(N) сравнений и хранит O(страница); это не обещание линейного времени RustFS.
|
||||
По умолчанию страница 1000, grace 3600 с + skew 60 с. Advisory lock разрешает один проход;
|
||||
второй возвращает skipped. Deep хеширует потоками 64 KiB, по умолчанию до восьми объектов.
|
||||
|
||||
| Расхождение | Безопасное действие `--apply` |
|
||||
| --- | --- |
|
||||
| Старый orphan | Повторная проверка строки по ключу, затем удаление объекта; свежий не трогается |
|
||||
| Старый pending | Повторная проверка под `FOR UPDATE SKIP LOCKED`, объект раньше строки, audit reason=reconcile |
|
||||
| Ready без объекта / другой размер / SHA | Повторная проверка; карантин, без удаления ready и ссылок сообщений; `/content` → 404 |
|
||||
| Объект карантина восстановлен | Снять карантин только после совпадения SHA-256, даже без `--deep` |
|
||||
|
||||
**Гипотезы до замеров** — [design на этапе SQLAlchemy](openspec/changes/archive/2026-11-06-verify-lab4-reconciler-sqlalchemy/design.md),
|
||||
proposal `751aaed4b1baaa957c74bb8874adeb0eac4b0480`:
|
||||
H1 — точное обнаружение/исправление, повтор 0 и сохранность свежих/используемых;
|
||||
H2 — shallow t(100k)/t(10k) в [7,13], страницы ceil(N/page), RSS растёт менее чем на 10%,
|
||||
deep на порядок дороже; H3 — после SQLAlchemy сохраняются страничность и ограниченная память.
|
||||
Отрицательный результат H2 не меняет исходную гипотезу.
|
||||
|
||||
**Корректность F.** На свежем отдельном project, grace 60/skew 0 внедрены ошибки обеих сторон.
|
||||
Dry-run не изменил 24 объекта/27 строк. Apply удалил 3 старых orphan и 6 pending, перевёл
|
||||
9 ready в карантин; шесть audit-событий, все шесть ссылок сохранены. Повтор — 0 действий.
|
||||
Возврат исходных байтов снял один карантин без deep, подмена той же длины — не сняла.
|
||||
Всего 13 PASS; [первичный demo](evidence/forward-final-2026-11-06/reconcile-demo/demo.json).
|
||||
При 75 с параллельных upload: 3364×201, 59 проходов, lost_ready=0, независимая проверка SHA
|
||||
в [check-under-load](evidence/forward-final-2026-11-06/reconcile-demo/check-under-load.json).
|
||||
**Конкурентная загрузка V после PNG-фиксов:** отдельный fresh project, 75 с / 3 потока,
|
||||
3341/3341×201, 67 проходов, errors=0/skipped=0, lost_ready=0. Независимый deep-аудит
|
||||
проверил SHA всех 3341 ready; каждый десятый upload привязан к сообщению. Удалены только
|
||||
3 внедрённых orphan. [Команда](evidence/png-final-2026-11-06/concurrent-reconcile.txt),
|
||||
[raw](evidence/png-final-2026-11-06/concurrent-reconcile/under-load.json),
|
||||
[SHA-аудит](evidence/png-final-2026-11-06/concurrent-reconcile/check-under-load.json).
|
||||
На пустом dataset инъекция pending создала 0 строк (ей нужна существующая строка-владелец);
|
||||
этот сценарий повторно не доказан серией V и сохраняется из полного demo F.
|
||||
Затронутый PNG-путь и сохранность всех 201 проверены; остальной алгоритм не менялся.
|
||||
Повторную проверку строки перед удалением дополнительно покрывает
|
||||
`test_row_written_after_listing_keeps_the_object`: наблюдение fresh не подменяет этот барьер.
|
||||
|
||||
**Стоимость на `3402fd69d4922e500035a204d957be2cccaeb85a`.** Свежий project
|
||||
`hlpipe-lab4-cost-20261106`; 1k/10k/30k/100k объектов и столько же ready-строк, PNG 8×8 по 264 B
|
||||
и один smoke-объект 111 817 B. Наполнение — четыре параллельных PUT и SQLAlchemy batch INSERT;
|
||||
измерение начинается после COMMIT и тихого окна, время наполнения исключено.
|
||||
API/LB/Valkey остановлены. RustFS 2 CPU/3 GiB, cpuset 11–12; PG 2 CPU/1 GiB, 13–14;
|
||||
CLI 1 CPU/256 MiB, 15. Это отдельная конфигурация, не штатные лимиты API-стенда.
|
||||
|
||||
Перед каждым повтором фильтр чужого CPU: ≤2 ядра VM и ≤0,25 на закреплённых,
|
||||
запас запуска 1,5/0,5 ядро·с; quiet 15 с, deadline 900 с, до трёх повторов отклонённой точки.
|
||||
Приняты 27, отклонённых 0; максимум чужого CPU 0,25 ядра, pinned 0. По три shallow/deep
|
||||
на каждом N и три page100 на 10k. Все raw с исходным SHA сохранены и пересчитаны при отчёте.
|
||||
[Команда](evidence/sqlalchemy-2026-11-06/track5/cost-command.txt),
|
||||
[полный протокол](evidence/sqlalchemy-2026-11-06/track5/NOTES.md),
|
||||
[план/фон/сырьё](evidence/sqlalchemy-2026-11-06/track5/measurements/cost/).
|
||||
|
||||
| N | Режим | Медиана прохода [min–max], с | LIST / SQL-страниц / GET | RSS, MiB, медиана |
|
||||
| ---: | --- | --- | --- | ---: |
|
||||
| 1000 | shallow | 0,107 [0,105–0,113] | 1 / 1 / 0 | 109,8 |
|
||||
| 1000 | deep | 0,499 [0,498–0,536] | 1 / 1 / 1000 | 110,2 |
|
||||
| 10000 | shallow | 0,999 [0,996–1,077] | 10 / 10 / 0 | 110,9 |
|
||||
| 10000 | deep | 4,998 [4,972–5,041] | 10 / 10 / 10000 | 111,3 |
|
||||
| 10000 | page100 | 2,037 [2,035–2,113] | 100 / 100 / 0 | 107,9 |
|
||||
| 30000 | shallow | 3,601 [3,567–3,633] | 30 / 30 / 0 | 110,9 |
|
||||
| 30000 | deep | 15,665 [15,630–15,779] | 30 / 30 / 30000 | 111,6 |
|
||||
| 100000 | shallow | 20,215 [20,206–21,335] | 100 / 100 / 0 | 111,1 |
|
||||
| 100000 | deep | 110,910 [58,661–147,215] | 100 / 100 / 100000 | 111,4 |
|
||||
|
||||
SQL здесь считает запросы страниц, без служебных BEGIN/COMMIT/pre-ping/advisory lock;
|
||||
HEAD/DELETE=0 во всех точках. `cost.seconds` — сам проход, CPU/wall в raw включают старт CLI.
|
||||
**H2 о времени отвергнута:** t100k/t10k=20,235. На 100k листинг занимает медиану 19,538 с;
|
||||
внутренняя причина его удорожания не установлена. Deep дороже в 4,35–5,49 раза, не на порядок.
|
||||
Страницы точно ceil(N/page), shallow RSS 109,8→111,1 MiB (+1,18%): эти части H2 и H3
|
||||
подтверждены в измеренной области. H1 подтверждена перечисленными сценариями на F.
|
||||
|
||||
Deep 100k: 147,215/110,910/58,661 с, разброс 79,84% медианы. Все повторы прошли заранее
|
||||
заданный фильтр; лучший не выбран вместо остальных. Один хост/VM, тёплые данные, частоты,
|
||||
диск и нагрузка macOS не контролировались. PNG 264 B не прогнозируют deep для 10 MiB,
|
||||
выше 100k измерений нет. Сравнение с историческим asyncpg — не причинный A/B-эксперимент.
|
||||
Сверка не восстанавливает утраченные байты и требует отдельного ручного/планового запуска.
|
||||
|
||||
## Завершение финальной validation
|
||||
|
||||
[Общий долг двух PNG-фиксов](evidence/png-header-fix/NOTES.md#обязательные-проверки-финального-этапа)
|
||||
закрыт на V: API-тесты с PostgreSQL, upload faults через LB/RustFS, 18 точек штатной
|
||||
матрицы RAM, десять одновременных 40 MP RGBA16 и upload при сверке с независимым SHA-аудитом.
|
||||
Команды и сырые результаты — в [аудите](evidence/load-validation.md).
|
||||
|
||||
`make student-test` дал 719 PASS и 2 setup errors из-за неготового временного RustFS;
|
||||
адресный повтор только этих двух тестов прошёл. Таким образом, все 721 выбранная
|
||||
проверка исполнена успешно, первый полный запуск не объявляется зелёным. Отдельно
|
||||
7 stack-тестов PASS. `make check`, `make validate`, `make lint` и OpenSpec strict (18)
|
||||
прошли. Реализация, конфиги, нормативные файлы и OpenSpec не менялись.
|
||||
|
||||
Принятые 27 cost-замеров и 3 повтора текстовой нагрузки сохранены с исходными SHA: повтор
|
||||
не требовался. Старые 64/3 и upload-memory до PNG-фиксов не используются для вывода о
|
||||
новом инспекторе. После Docker-этапа оба собственных project остановлены, шесть постоянных
|
||||
volumes сохранены; [инвентаризация](evidence/png-final-2026-11-06/shutdown-state.txt).
|
||||
Публикация — только после отдельной команды координатора PUBLICATION; push не выполнялся.
|
||||
|
||||
## Ограничения
|
||||
|
||||
- **Minor, сохранён ревью:** непарные Unicode-суррогаты U+D800/U+DFFF в `text` вызывают
|
||||
`PydanticSerializationError` в `fingerprint()` и HTTP 500. Файл
|
||||
[schemas.py:35](src/messenger/messaging/entrypoints/schemas.py#L35),
|
||||
[HTTP-доказательство](evidence/forward-final-2026-11-06/known-minor-http.txt).
|
||||
Это дефект, не PASS; по правилу пользователя minor не требует fix и не блокирует этап.
|
||||
- Один Docker-host, по одному PostgreSQL/RustFS/LB — точки отказа; репликация и резервные
|
||||
копии байтов здесь не исследуются. При остановке всех API sweeper ждёт их возврата.
|
||||
- Тело upload ограничено, но хранится целиком; слоты защищают память ценой очереди.
|
||||
Два 503 в F сохранены без объяснения недоказанной причиной. В V все 840 запросов
|
||||
успешны при пике ≤440 MiB; это измеренная область 10–40 клиентов, не общий предел RAM
|
||||
и не обещание 100% успеха при любой нагрузке.
|
||||
- При неизвестном результате PUT/COMMIT остаток ждёт grace; потерянный ответ upload
|
||||
допускает повторное вложение, поскольку у этого endpoint нет Idempotency-Key.
|
||||
- После отправки заголовков сбой S3 при download может оборвать поток; слишком медленная
|
||||
передача и SIGTERM также ограничены таймаутами. Ранний отказ во время отправки большого
|
||||
тела клиент иногда увидит как reset. Сессия проверяется раньше объявленного размера.
|
||||
- В demo grace 60/skew 0, смещение часов не моделировалось; реальный grace API 900 с
|
||||
отдельно подтверждён старой серией S. Reconciler не восстанавливает повреждённые байты.
|
||||
- Нагрузка записи — закрытый контур и один чат, успешные latency имеют coordinated omission.
|
||||
Причинный эффект SQLAlchemy/исправлений и предел системы из этих чисел не выводятся.
|
||||
- Секреты — локальные файлы Compose вместо менеджера секретов; статические S3-ключи без
|
||||
исследованной ротации. TLS, presigned flow, брокер и варианты изображений вне этой лабы.
|
||||
- Исторические proposal `add-lab4-image-attachments` и `add-lab4-upload-resilience`
|
||||
превышают рекомендацию 400 слов; история не переписана. `openspec/config.yaml` сверён
|
||||
с текущим стеком, но в документационном этапе не редактировался по ограничению области работ.
|
||||
|
||||
## Ответы на вопросы защиты
|
||||
|
||||
**Почему volume одной API-реплики не равен объектному хранилищу?** Другая реплика не обязана
|
||||
видеть её локальный volume. RustFS даёт всем репликам одно адресуемое ключами хранилище
|
||||
с политиками доступа; остановка принявшей API не удаляет оригинал. В F это показали
|
||||
разные `X-Instance-Id` и совпавший SHA-256 после остановки target.
|
||||
|
||||
**Когда можно вернуть 201?** После успешного PUT и COMMIT статуса ready. Pending уже
|
||||
зафиксирован до PUT, но не виден пользователям. Потерянный ответ после COMMIT возможен;
|
||||
это не позволяет заранее отвечать 201.
|
||||
|
||||
**Что если S3 PUT прошёл, а COMMIT — нет?** При известном отказе компенсация под блокировкой
|
||||
удаляет объект, затем pending. При падении процесса/неизвестном исходе это делает sweeper
|
||||
после grace. Уже ready не удаляется; независимые orphan и расхождения ищет reconciler.
|
||||
|
||||
**Почему случайный key/private bucket не заменяют ACL?** S3-ключ API разрешает доступ
|
||||
ко всему bucket и не знает членства пользователей в чатах. Поэтому каждый запрос
|
||||
метаданных/байтов проверяет членство; постороннему 404. Private bucket дополнительно
|
||||
закрывает прямой анонимный GET, но не принимает решение за приложение.
|
||||
|
||||
Для защиты: `make test` показывает upload→message→download и чужому 404;
|
||||
`make s3-check` — private вне API; persistence/failover — сохранность;
|
||||
`make attachments-evidence` — отсутствие локальных файлов API; сценарии уровней 4/5 —
|
||||
компенсацию, ограничения памяти и безопасную сверку. Следует показать исходный SHA каждого
|
||||
результата и новые доказательства PNG-инспектора из серии V.
|
||||
|
||||
Решение перенесено из собственной ЛР3. При реализации и подготовке отчёта использован Codex;
|
||||
автор работы — Костылев Артём. Нормативные README/COURSE/contracts/tests сохранены.
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
[alembic]
|
||||
script_location = %(here)s/alembic
|
||||
prepend_sys_path = %(here)s/src %(here)s/alembic
|
||||
path_separator = space
|
||||
@@ -0,0 +1,92 @@
|
||||
import asyncio
|
||||
|
||||
from alembic import context
|
||||
from alembic.script import ScriptDirectory
|
||||
from legacy import adopt_legacy
|
||||
from sqlalchemy import Connection, func, make_url, pool, select
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
|
||||
from messenger.attachments.infrastructure.postgres import AttachmentRow
|
||||
from messenger.identity.infrastructure.postgres import SessionRow, UserRow
|
||||
from messenger.messaging.infrastructure.postgres import (
|
||||
ChatMemberRow,
|
||||
ChatRow,
|
||||
IdempotencyKeyRow,
|
||||
MessageAttachmentRow,
|
||||
MessageRow,
|
||||
)
|
||||
from messenger.shared.config import PlatformSettings
|
||||
|
||||
LOCK_KEY = 20_260_002
|
||||
MODELS = (
|
||||
UserRow,
|
||||
SessionRow,
|
||||
ChatRow,
|
||||
ChatMemberRow,
|
||||
MessageRow,
|
||||
IdempotencyKeyRow,
|
||||
AttachmentRow,
|
||||
MessageAttachmentRow,
|
||||
)
|
||||
target_metadata = UserRow.metadata
|
||||
config = context.config
|
||||
|
||||
|
||||
def include_object(_obj, name, _type, _reflected, _compare_to):
|
||||
return name != "schema_migrations"
|
||||
|
||||
|
||||
def run_migrations(connection: Connection) -> None:
|
||||
with connection.engine.connect() as lock:
|
||||
lock.execute(select(func.pg_advisory_lock(LOCK_KEY)))
|
||||
lock.commit()
|
||||
try:
|
||||
with connection.begin():
|
||||
adopt_legacy(connection, ScriptDirectory.from_config(config))
|
||||
context.configure(
|
||||
connection=connection,
|
||||
target_metadata=target_metadata,
|
||||
transaction_per_migration=True,
|
||||
compare_type=True,
|
||||
include_object=include_object,
|
||||
)
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
finally:
|
||||
lock.execute(select(func.pg_advisory_unlock(LOCK_KEY)))
|
||||
lock.commit()
|
||||
|
||||
|
||||
async def run_async_migrations() -> None:
|
||||
dsn = config.attributes.get("database_url")
|
||||
if dsn is None:
|
||||
dsn = PlatformSettings().database_url.get_secret_value()
|
||||
engine = create_async_engine(
|
||||
make_url(dsn).set(drivername="postgresql+asyncpg"),
|
||||
poolclass=pool.NullPool,
|
||||
hide_parameters=True,
|
||||
connect_args={
|
||||
"timeout": 10,
|
||||
"server_settings": {"lock_timeout": "10000", "statement_timeout": "60000"},
|
||||
},
|
||||
)
|
||||
try:
|
||||
async with engine.connect() as connection:
|
||||
await connection.run_sync(run_migrations)
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
|
||||
if context.is_offline_mode():
|
||||
context.configure(
|
||||
dialect_name="postgresql",
|
||||
target_metadata=target_metadata,
|
||||
literal_binds=True,
|
||||
transaction_per_migration=True,
|
||||
)
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
elif config.attributes.get("connection") is not None:
|
||||
run_migrations(config.attributes["connection"])
|
||||
else:
|
||||
asyncio.run(run_async_migrations())
|
||||
@@ -0,0 +1,92 @@
|
||||
from uuid import uuid4
|
||||
|
||||
from alembic.migration import MigrationContext
|
||||
from alembic.operations import Operations
|
||||
from alembic.script import ScriptDirectory
|
||||
from sqlalchemy import Connection, MetaData, Table, func, inspect, select, text
|
||||
from sqlalchemy.schema import CreateSchema
|
||||
|
||||
LEGACY = {
|
||||
"0001_init": "9a13618f078ae9056431391e2e5197a31b981ae1fff047a9b3421ac750d3fd0d",
|
||||
"0002_idempotency_keys": "d9159a35553249a1e51af5f2e4e7c39b014e97593c49748e13f3799839063dd0",
|
||||
"0003_attachments": "c4417f560538a083efba1c67f660147dc94532fcbb2fa91f3f3e8258a0f01f2e",
|
||||
"0004_attachment_quarantine": (
|
||||
"541f5399d87fac3584c4102b4e0bf92c23cd372cc20a7d56dad03e771773202c"
|
||||
),
|
||||
}
|
||||
SNAPSHOT = text("""
|
||||
WITH relations AS (
|
||||
SELECT c.* FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace
|
||||
WHERE n.nspname = :schema AND c.relkind IN ('r', 'p')
|
||||
AND c.relname NOT IN ('schema_migrations', 'alembic_version')
|
||||
)
|
||||
SELECT 'table' AS kind, c.relname AS name,
|
||||
concat_ws('|', c.relkind, c.relpersistence,
|
||||
c.relrowsecurity, c.relforcerowsecurity) AS definition
|
||||
FROM relations c
|
||||
UNION ALL
|
||||
SELECT 'column', c.relname || '.' || a.attname,
|
||||
concat_ws('|', a.attnum, format_type(a.atttypid, a.atttypmod), a.attnotnull,
|
||||
a.attidentity, a.attgenerated, a.attcollation::regcollation,
|
||||
pg_get_expr(d.adbin, d.adrelid))
|
||||
FROM relations c JOIN pg_attribute a ON a.attrelid = c.oid
|
||||
LEFT JOIN pg_attrdef d ON d.adrelid = c.oid AND d.adnum = a.attnum
|
||||
WHERE a.attnum > 0 AND NOT a.attisdropped
|
||||
UNION ALL
|
||||
SELECT 'constraint', c.relname || '.' || k.conname,
|
||||
concat_ws('|', pg_get_constraintdef(k.oid), k.convalidated)
|
||||
FROM relations c JOIN pg_constraint k ON k.conrelid = c.oid
|
||||
UNION ALL
|
||||
SELECT 'index', c.relname || '.' || x.relname,
|
||||
concat_ws('|', pg_get_indexdef(i.indexrelid), i.indisvalid, i.indisready)
|
||||
FROM relations c JOIN pg_index i ON i.indrelid = c.oid
|
||||
JOIN pg_class x ON x.oid = i.indexrelid
|
||||
UNION ALL
|
||||
SELECT 'trigger', c.relname || '.' || t.tgname,
|
||||
concat_ws('|', pg_get_triggerdef(t.oid), t.tgenabled)
|
||||
FROM relations c JOIN pg_trigger t ON t.tgrelid = c.oid WHERE NOT t.tgisinternal
|
||||
ORDER BY kind, name
|
||||
""")
|
||||
|
||||
|
||||
def snapshot(connection: Connection, schema: str) -> list[tuple[str, str, str]]:
|
||||
return [
|
||||
(kind, name, definition.replace(f"{schema}.", ""))
|
||||
for kind, name, definition in connection.execute(SNAPSHOT, {"schema": schema})
|
||||
]
|
||||
|
||||
|
||||
def adopt_legacy(connection: Connection, scripts: ScriptDirectory) -> None:
|
||||
migration = MigrationContext.configure(connection)
|
||||
if migration.get_current_heads():
|
||||
return
|
||||
tables = set(inspect(connection).get_table_names(schema="public"))
|
||||
if tables <= {"alembic_version"}:
|
||||
return
|
||||
if "schema_migrations" not in tables:
|
||||
raise RuntimeError("Existing schema has no legacy migration journal")
|
||||
journal = Table("schema_migrations", MetaData(), schema="public", autoload_with=connection)
|
||||
applied = dict(connection.execute(select(journal.c.version, journal.c.checksum)).tuples().all())
|
||||
expected = dict(list(LEGACY.items())[: len(applied)])
|
||||
if applied != expected:
|
||||
raise RuntimeError("Unknown or changed legacy migrations")
|
||||
if not applied:
|
||||
if tables - {"schema_migrations", "alembic_version"}:
|
||||
raise RuntimeError("Existing schema has an empty migration journal")
|
||||
return
|
||||
revision = list(expected)[-1]
|
||||
actual = snapshot(connection, "public")
|
||||
reference = f"alembic_verify_{uuid4().hex}"
|
||||
savepoint = connection.begin_nested()
|
||||
try:
|
||||
connection.execute(CreateSchema(reference))
|
||||
connection.execute(select(func.set_config("search_path", reference, True)))
|
||||
with Operations.context(MigrationContext.configure(connection)):
|
||||
for script in reversed(list(scripts.walk_revisions(base="base", head=revision))):
|
||||
script.module.upgrade()
|
||||
wanted = snapshot(connection, reference)
|
||||
finally:
|
||||
savepoint.rollback()
|
||||
if actual != wanted:
|
||||
raise RuntimeError("Legacy schema differs from the recorded migrations")
|
||||
migration.stamp(scripts, revision)
|
||||
@@ -0,0 +1,16 @@
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
${imports if imports else ""}
|
||||
|
||||
revision = ${repr(up_revision)}
|
||||
down_revision = ${repr(down_revision)}
|
||||
branch_labels = ${repr(branch_labels)}
|
||||
depends_on = ${repr(depends_on)}
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
${upgrades if upgrades else "pass"}
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
${downgrades if downgrades else "pass"}
|
||||
@@ -0,0 +1,87 @@
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision = "0001_init"
|
||||
down_revision = None
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"users",
|
||||
sa.Column("id", sa.Uuid(), nullable=False),
|
||||
sa.Column("username", sa.Text(), nullable=False),
|
||||
sa.Column("display_name", sa.Text(), nullable=False),
|
||||
sa.Column("password_hash", sa.Text(), nullable=False),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.CheckConstraint("password_hash LIKE '$argon2id$%'", name="users_password_hash_check"),
|
||||
sa.CheckConstraint("username ~ '^[a-z0-9_]{3,32}$'", name="users_username_format"),
|
||||
sa.CheckConstraint(
|
||||
"char_length(display_name) BETWEEN 1 AND 100", name="users_display_name_check"
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("username", name="users_username_key"),
|
||||
)
|
||||
op.create_table(
|
||||
"sessions",
|
||||
sa.Column("token_hash", sa.LargeBinary(), nullable=False),
|
||||
sa.Column("user_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.CheckConstraint("expires_at > created_at", name="sessions_check"),
|
||||
sa.CheckConstraint("octet_length(token_hash) = 32", name="sessions_token_hash_check"),
|
||||
sa.ForeignKeyConstraint(["user_id"], ["users.id"], ondelete="CASCADE"),
|
||||
sa.PrimaryKeyConstraint("token_hash"),
|
||||
)
|
||||
op.create_index("sessions_user_idx", "sessions", ["user_id"], unique=False)
|
||||
op.create_table(
|
||||
"chats",
|
||||
sa.Column("id", sa.Uuid(), nullable=False),
|
||||
sa.Column("title", sa.Text(), nullable=False),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("last_message_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.CheckConstraint("char_length(title) BETWEEN 1 AND 100", name="chats_title_check"),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
)
|
||||
op.create_table(
|
||||
"chat_members",
|
||||
sa.Column("chat_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("user_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("position", sa.SmallInteger(), nullable=False),
|
||||
sa.Column("chat_created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.CheckConstraint("position BETWEEN 1 AND 100", name="chat_members_position_check"),
|
||||
sa.ForeignKeyConstraint(["chat_id"], ["chats.id"], ondelete="CASCADE"),
|
||||
sa.ForeignKeyConstraint(
|
||||
["user_id"],
|
||||
["users.id"],
|
||||
),
|
||||
sa.PrimaryKeyConstraint("chat_id", "user_id"),
|
||||
)
|
||||
op.create_index(
|
||||
"chat_members_user_idx",
|
||||
"chat_members",
|
||||
["user_id", "chat_created_at", "chat_id"],
|
||||
unique=False,
|
||||
)
|
||||
op.create_table(
|
||||
"messages",
|
||||
sa.Column("id", sa.Uuid(), nullable=False),
|
||||
sa.Column("chat_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("sender_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("text", sa.Text(), nullable=False),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.CheckConstraint("char_length(text) BETWEEN 1 AND 4000", name="messages_text_check"),
|
||||
sa.ForeignKeyConstraint(
|
||||
["chat_id", "sender_id"],
|
||||
["chat_members.chat_id", "chat_members.user_id"],
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
)
|
||||
op.create_index(
|
||||
"messages_history_idx", "messages", ["chat_id", "created_at", "id"], unique=False
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
raise RuntimeError("Downgrade would delete application data")
|
||||
@@ -0,0 +1,33 @@
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision = "0002_idempotency_keys"
|
||||
down_revision = "0001_init"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"idempotency_keys",
|
||||
sa.Column("chat_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("user_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("key", sa.Text(), nullable=False),
|
||||
sa.Column("fingerprint", sa.LargeBinary(), nullable=False),
|
||||
sa.Column("message_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.CheckConstraint("key ~ '^[A-Za-z0-9._:-]{1,128}$'", name="idempotency_keys_key_check"),
|
||||
sa.CheckConstraint(
|
||||
"octet_length(fingerprint) = 32", name="idempotency_keys_fingerprint_check"
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["chat_id", "user_id"],
|
||||
["chat_members.chat_id", "chat_members.user_id"],
|
||||
),
|
||||
sa.ForeignKeyConstraint(["message_id"], ["messages.id"], ondelete="CASCADE"),
|
||||
sa.PrimaryKeyConstraint("chat_id", "user_id", "key"),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
raise RuntimeError("Downgrade would delete application data")
|
||||
@@ -0,0 +1,77 @@
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision = "0003_attachments"
|
||||
down_revision = "0002_idempotency_keys"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.drop_constraint("messages_text_check", "messages", type_="check")
|
||||
op.create_check_constraint("messages_text_check", "messages", "char_length(text) <= 4000")
|
||||
op.create_table(
|
||||
"attachments",
|
||||
sa.Column("id", sa.Uuid(), nullable=False),
|
||||
sa.Column("chat_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("owner_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("filename", sa.Text(), nullable=False),
|
||||
sa.Column("content_type", sa.Text(), nullable=False),
|
||||
sa.Column("size_bytes", sa.BigInteger(), nullable=False),
|
||||
sa.Column("sha256", sa.Text(), nullable=False),
|
||||
sa.Column("width", sa.Integer(), nullable=False),
|
||||
sa.Column("height", sa.Integer(), nullable=False),
|
||||
sa.Column("object_key", sa.Text(collation="C"), nullable=False),
|
||||
sa.Column("status", sa.Text(), nullable=False),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("object_key", name="attachments_object_key_key"),
|
||||
sa.UniqueConstraint("id", "chat_id", name="attachments_chat_key"),
|
||||
sa.ForeignKeyConstraint(
|
||||
["chat_id", "owner_id"],
|
||||
["chat_members.chat_id", "chat_members.user_id"],
|
||||
name="attachments_owner_fkey",
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"char_length(filename) BETWEEN 1 AND 255", name="attachments_filename_check"
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"content_type IN ('image/png', 'image/jpeg')", name="attachments_content_type_check"
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"size_bytes BETWEEN 1 AND 10485760", name="attachments_size_bytes_check"
|
||||
),
|
||||
sa.CheckConstraint("sha256 ~ '^[0-9a-f]{64}$'", name="attachments_sha256_check"),
|
||||
sa.CheckConstraint("width > 0", name="attachments_width_check"),
|
||||
sa.CheckConstraint("height > 0", name="attachments_height_check"),
|
||||
sa.CheckConstraint("status IN ('pending', 'ready')", name="attachments_status_check"),
|
||||
)
|
||||
op.create_index(
|
||||
"attachments_pending_idx",
|
||||
"attachments",
|
||||
["created_at"],
|
||||
postgresql_where=sa.text("status = 'pending'"),
|
||||
)
|
||||
op.create_table(
|
||||
"message_attachments",
|
||||
sa.Column("message_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("position", sa.SmallInteger(), nullable=False),
|
||||
sa.Column("chat_id", sa.Uuid(), nullable=False),
|
||||
sa.Column("attachment_id", sa.Uuid(), nullable=False),
|
||||
sa.PrimaryKeyConstraint("message_id", "position"),
|
||||
sa.UniqueConstraint("message_id", "attachment_id"),
|
||||
sa.CheckConstraint("position BETWEEN 0 AND 9", name="message_attachments_position_check"),
|
||||
sa.ForeignKeyConstraint(["message_id"], ["messages.id"], ondelete="CASCADE"),
|
||||
sa.ForeignKeyConstraint(
|
||||
["attachment_id", "chat_id"],
|
||||
["attachments.id", "attachments.chat_id"],
|
||||
name="message_attachments_attachment_fkey",
|
||||
ondelete="RESTRICT",
|
||||
),
|
||||
)
|
||||
op.create_index("message_attachments_attachment_idx", "message_attachments", ["attachment_id"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
raise RuntimeError("Downgrade would delete application data")
|
||||
@@ -0,0 +1,28 @@
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision = "0004_attachment_quarantine"
|
||||
down_revision = "0003_attachments"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column("attachments", sa.Column("quarantine_reason", sa.Text(), nullable=True))
|
||||
op.create_check_constraint(
|
||||
"attachments_quarantine_reason_check",
|
||||
"attachments",
|
||||
"quarantine_reason IN ('missing', 'size_mismatch', 'sha_mismatch')",
|
||||
)
|
||||
op.add_column(
|
||||
"attachments", sa.Column("quarantined_at", sa.DateTime(timezone=True), nullable=True)
|
||||
)
|
||||
op.create_check_constraint(
|
||||
"attachments_quarantine_check",
|
||||
"attachments",
|
||||
"(quarantine_reason IS NULL) = (quarantined_at IS NULL)",
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
raise RuntimeError("Downgrade would delete quarantine data")
|
||||
@@ -1,10 +1,12 @@
|
||||
# Topology worksheet, deliberately NOT a runnable stack (no application supplied).
|
||||
# Implement compose.yaml or stack.yaml; do not submit this empty file as deployment.
|
||||
# Service names below are logical roles, not imposed container names.
|
||||
x-lab-number: 2
|
||||
x-lab-number: 4
|
||||
x-required-roles:
|
||||
- api
|
||||
- postgres
|
||||
- load_balancer
|
||||
- s3
|
||||
# TODO: implement real images/builds, network wiring, volumes, healthchecks,
|
||||
# credentials and bootstrap. Details: contracts/infrastructure.json and README.md.
|
||||
services: {}
|
||||
|
||||
+322
@@ -0,0 +1,322 @@
|
||||
x-app: &app
|
||||
image: ${API_IMAGE:-mtusi-messenger:${COMPOSE_PROJECT_NAME}}
|
||||
networks: [backend]
|
||||
|
||||
x-s3-env: &s3-env
|
||||
S3_ENDPOINT: ${S3_ENDPOINT:-http://s3:9000}
|
||||
S3_REGION: ${S3_REGION:-us-east-1}
|
||||
S3_BUCKET: ${S3_BUCKET:-messenger-attachments}
|
||||
S3_ACCESS_KEY_ID_FILE: /run/secrets/s3_access_key_id
|
||||
S3_SECRET_ACCESS_KEY_FILE: /run/secrets/s3_secret_access_key
|
||||
S3_CONNECT_TIMEOUT_SECONDS: ${S3_CONNECT_TIMEOUT_SECONDS:-1}
|
||||
S3_READ_TIMEOUT_SECONDS: ${S3_READ_TIMEOUT_SECONDS:-2}
|
||||
MAX_UPLOAD_BYTES: ${MAX_UPLOAD_BYTES:-10485760}
|
||||
MAX_IMAGE_PIXELS: ${MAX_IMAGE_PIXELS:-40000000}
|
||||
IMAGE_CHECK_CONCURRENCY: ${IMAGE_CHECK_CONCURRENCY:-2}
|
||||
UPLOAD_CONCURRENCY: ${UPLOAD_CONCURRENCY:-3}
|
||||
UPLOAD_GRACE_SECONDS: ${UPLOAD_GRACE_SECONDS:-900}
|
||||
UPLOAD_SWEEP_INTERVAL_SECONDS: ${UPLOAD_SWEEP_INTERVAL_SECONDS:-30}
|
||||
UPLOAD_SWEEP_BATCH: ${UPLOAD_SWEEP_BATCH:-100}
|
||||
|
||||
x-s3-client: &s3-client
|
||||
image: rustfs/rc:v0.1.36@sha256:ab024bfebee49a750ce886b4c70963ccd9ddaa03f491704a90710641d7a26699
|
||||
read_only: true
|
||||
tmpfs: ["/tmp:size=16m"]
|
||||
environment:
|
||||
S3_ENDPOINT: ${S3_ENDPOINT:-http://s3:9000}
|
||||
S3_BUCKET: ${S3_BUCKET:-messenger-attachments}
|
||||
RC_CONFIG_DIR: /tmp/rc
|
||||
user: "65534:65534"
|
||||
volumes: ["./infra/s3:/s3:ro"]
|
||||
secrets:
|
||||
- s3_root_user
|
||||
- s3_root_password
|
||||
- s3_access_key_id
|
||||
- s3_secret_access_key
|
||||
- s3_reconciler_access_key_id
|
||||
- s3_reconciler_secret_access_key
|
||||
cap_drop: [ALL]
|
||||
security_opt: ["no-new-privileges:true"]
|
||||
restart: "no"
|
||||
networks: [backend]
|
||||
deploy:
|
||||
resources:
|
||||
limits: {cpus: "0.5", memory: 64m}
|
||||
|
||||
services:
|
||||
lb:
|
||||
image: haproxy:3.2.26-alpine@sha256:5893840576071e6217cd52ebdcc14f454c6f7a631056fa6f4e0f40d3cac4215b
|
||||
environment:
|
||||
API_SERVER_MAXCONN: ${API_SERVER_MAXCONN:-64}
|
||||
volumes: [./infra/haproxy/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro]
|
||||
ports:
|
||||
- "127.0.0.1:${LB_HTTP_PORT:-8080}:8080"
|
||||
- "127.0.0.1:${LB_STATS_PORT:-8404}:8404"
|
||||
networks: [edge, backend]
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://127.0.0.1:8404/healthz"]
|
||||
interval: 2s
|
||||
timeout: 2s
|
||||
retries: 3
|
||||
start_period: 60s
|
||||
start_interval: 1s
|
||||
sysctls:
|
||||
net.ipv4.ip_local_port_range: "10240 65535"
|
||||
net.ipv4.tcp_tw_reuse: "1"
|
||||
stop_signal: SIGUSR1
|
||||
stop_grace_period: 8s
|
||||
restart: unless-stopped
|
||||
read_only: true
|
||||
cap_drop: [ALL]
|
||||
deploy:
|
||||
resources:
|
||||
limits: {cpus: "${LB_CPUS:-1}", memory: 128m}
|
||||
|
||||
api:
|
||||
<<: *app
|
||||
build: .
|
||||
environment:
|
||||
<<: *s3-env
|
||||
DATABASE_URL_FILE: /run/secrets/database_url
|
||||
CACHE_URL_FILE: /run/secrets/cache_url
|
||||
CURSOR_SECRET_FILE: /run/secrets/cursor_secret
|
||||
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
|
||||
IDEMPOTENCY_TTL_SECONDS: ${IDEMPOTENCY_TTL_SECONDS:-86400}
|
||||
WEB_CONCURRENCY: ${WEB_CONCURRENCY:-2}
|
||||
DB_POOL_MAX_SIZE: ${DB_POOL_MAX_SIZE:-8}
|
||||
REQUEST_TIMEOUT_SECONDS: ${REQUEST_TIMEOUT_SECONDS:-8}
|
||||
MIN_TRANSFER_RATE_BYTES: ${MIN_TRANSFER_RATE_BYTES:-131072}
|
||||
DB_ACQUIRE_TIMEOUT_SECONDS: ${DB_ACQUIRE_TIMEOUT_SECONDS:-1}
|
||||
DB_COMMAND_TIMEOUT_SECONDS: ${DB_COMMAND_TIMEOUT_SECONDS:-2}
|
||||
DRAIN_SECONDS: ${DRAIN_SECONDS:-2}
|
||||
SHUTDOWN_TIMEOUT_SECONDS: ${SHUTDOWN_TIMEOUT_SECONDS:-6}
|
||||
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
||||
MALLOC_MMAP_THRESHOLD_: ${MALLOC_MMAP_THRESHOLD_:-131072}
|
||||
secrets: [database_url, cache_url, cursor_secret, s3_access_key_id, s3_secret_access_key]
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c", "import sys,urllib.request;sys.exit(urllib.request.urlopen('http://127.0.0.1:8080/health/ready',timeout=2).status!=200)"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
start_interval: 200ms
|
||||
stop_grace_period: 8s
|
||||
restart: unless-stopped
|
||||
read_only: true
|
||||
tmpfs: [/tmp]
|
||||
sysctls:
|
||||
net.ipv4.tcp_rmem: "4096 65536 262144"
|
||||
net.ipv4.tcp_wmem: "4096 65536 262144"
|
||||
depends_on:
|
||||
migrate:
|
||||
condition: service_completed_successfully
|
||||
valkey:
|
||||
condition: service_started
|
||||
s3-init:
|
||||
condition: service_completed_successfully
|
||||
deploy:
|
||||
replicas: ${API_REPLICAS:-2}
|
||||
resources:
|
||||
limits: {cpus: "${API_CPUS:-1}", memory: "${API_MEMORY:-512m}"}
|
||||
|
||||
migrate:
|
||||
<<: *app
|
||||
pull_policy: never
|
||||
init: true
|
||||
command: ["alembic", "upgrade", "head"]
|
||||
environment:
|
||||
DATABASE_URL_FILE: /run/secrets/database_url
|
||||
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
||||
secrets: [database_url]
|
||||
healthcheck:
|
||||
disable: true
|
||||
restart: "no"
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
deploy:
|
||||
resources:
|
||||
limits: {cpus: "0.5", memory: 256m}
|
||||
|
||||
postgres:
|
||||
image: postgres:17.11-alpine@sha256:266d3c257594e195b309ae9d1805e54c788f1d7b375f56bf6c367aac6c917e7b
|
||||
user: "70:70"
|
||||
command:
|
||||
- postgres
|
||||
- -c
|
||||
- max_connections=${PG_MAX_CONNECTIONS:-200}
|
||||
- -c
|
||||
- shared_buffers=${PG_SHARED_BUFFERS:-256MB}
|
||||
- -c
|
||||
- shared_preload_libraries=pg_stat_statements
|
||||
- -c
|
||||
- pg_stat_statements.track=top
|
||||
- -c
|
||||
- track_io_timing=on
|
||||
- -c
|
||||
- idle_in_transaction_session_timeout=10s
|
||||
environment:
|
||||
POSTGRES_DB: messenger
|
||||
POSTGRES_USER: messenger
|
||||
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
|
||||
secrets: [postgres_password]
|
||||
volumes: [pgdata:/var/lib/postgresql/data]
|
||||
shm_size: 256m
|
||||
cpuset: ${PG_CPUSET:-}
|
||||
healthcheck:
|
||||
test: ["CMD", "pg_isready", "-h", "127.0.0.1", "-U", "messenger", "-d", "messenger"]
|
||||
interval: 2s
|
||||
timeout: 3s
|
||||
retries: 30
|
||||
start_period: 10s
|
||||
start_interval: 200ms
|
||||
stop_grace_period: 10s
|
||||
restart: unless-stopped
|
||||
networks: [backend]
|
||||
deploy:
|
||||
resources:
|
||||
limits: {cpus: "${PG_CPUS:-2}", memory: "${PG_MEMORY:-1g}"}
|
||||
|
||||
valkey:
|
||||
image: valkey/valkey:8.1.11-alpine@sha256:2602ace2be6569bc2bbb06f873776516ab9089821525be34f4305c67eb9db02a
|
||||
user: "999:1000"
|
||||
command:
|
||||
- valkey-server
|
||||
- /run/secrets/valkey.conf
|
||||
- --appendonly
|
||||
- "yes"
|
||||
- --appendfsync
|
||||
- always
|
||||
- --maxmemory
|
||||
- 64mb
|
||||
- --maxmemory-policy
|
||||
- volatile-lru
|
||||
secrets: [valkey.conf]
|
||||
volumes: [valkeydata:/data]
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "REDISCLI_AUTH=\"$$(sed -n 's/^requirepass //p' /run/secrets/valkey.conf)\" valkey-cli ping | grep -q PONG"]
|
||||
interval: 2s
|
||||
timeout: 3s
|
||||
retries: 30
|
||||
stop_grace_period: 10s
|
||||
restart: unless-stopped
|
||||
networks: [backend]
|
||||
deploy:
|
||||
resources:
|
||||
limits: {cpus: "0.5", memory: 128m}
|
||||
|
||||
s3:
|
||||
image: rustfs/rustfs:1.0.4@sha256:ed9210d6411d77ffceb70d94b964a3cb7d225983368bc25bebe01e995293dcc2
|
||||
environment:
|
||||
RUSTFS_ACCESS_KEY_FILE: /run/secrets/s3_root_user
|
||||
RUSTFS_SECRET_KEY_FILE: /run/secrets/s3_root_password
|
||||
RUSTFS_ADDRESS: ":9000"
|
||||
RUSTFS_CONSOLE_ENABLE: "false"
|
||||
RUSTFS_REGION: ${S3_REGION:-us-east-1}
|
||||
RUSTFS_OBS_LOG_DIRECTORY: ""
|
||||
RUSTFS_OBS_LOGGER_LEVEL: ${S3_LOG_LEVEL:-warn}
|
||||
secrets: [s3_root_user, s3_root_password]
|
||||
volumes: [s3data:/data]
|
||||
cpuset: ${S3_CPUSET:-}
|
||||
read_only: true
|
||||
tmpfs: ["/tmp:size=64m"]
|
||||
cap_drop: [ALL]
|
||||
security_opt: ["no-new-privileges:true"]
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-fsS", "-o", "/dev/null", "http://127.0.0.1:9000/health/ready"]
|
||||
interval: 2s
|
||||
timeout: 3s
|
||||
retries: 30
|
||||
start_period: 20s
|
||||
stop_grace_period: 10s
|
||||
restart: unless-stopped
|
||||
networks: [backend]
|
||||
deploy:
|
||||
resources:
|
||||
limits: {cpus: "${S3_CPUS:-1}", memory: "${S3_MEMORY:-512m}"}
|
||||
|
||||
reconciler:
|
||||
<<: *app
|
||||
pull_policy: never
|
||||
profiles: [reconcile]
|
||||
init: true
|
||||
entrypoint: ["python", "-m", "messenger.attachments.entrypoints.reconcile"]
|
||||
command: ["--dry-run"]
|
||||
environment:
|
||||
<<: *s3-env
|
||||
DATABASE_URL_FILE: /run/secrets/database_url
|
||||
S3_ACCESS_KEY_ID_FILE: /run/secrets/s3_reconciler_access_key_id
|
||||
S3_SECRET_ACCESS_KEY_FILE: /run/secrets/s3_reconciler_secret_access_key
|
||||
S3_READ_TIMEOUT_SECONDS: ${RECONCILE_S3_READ_TIMEOUT_SECONDS:-10}
|
||||
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
||||
secrets: [database_url, s3_reconciler_access_key_id, s3_reconciler_secret_access_key]
|
||||
healthcheck:
|
||||
disable: true
|
||||
restart: "no"
|
||||
read_only: true
|
||||
cpuset: ${RECONCILE_CPUSET:-}
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
s3:
|
||||
condition: service_healthy
|
||||
deploy:
|
||||
resources:
|
||||
limits: {cpus: "${RECONCILE_CPUS:-1}", memory: "${RECONCILE_MEMORY:-256m}"}
|
||||
|
||||
s3-init:
|
||||
<<: *s3-client
|
||||
entrypoint: ["/bin/sh", "/s3/init.sh"]
|
||||
depends_on:
|
||||
s3:
|
||||
condition: service_healthy
|
||||
|
||||
s3-cli:
|
||||
<<: *s3-client
|
||||
profiles: [tools]
|
||||
entrypoint: ["/bin/sh", "/s3/cli.sh"]
|
||||
|
||||
k6:
|
||||
image: grafana/k6:2.3.0@sha256:2dd4756ec6e14e62778916e1eb850108e4b499304c02df56760fa0bdb0fba0b8
|
||||
profiles: [bench]
|
||||
environment:
|
||||
BASE_URL: http://lb:8080
|
||||
volumes: [./infra/k6:/scripts:ro]
|
||||
networks: [edge]
|
||||
deploy:
|
||||
resources:
|
||||
limits: {cpus: "${K6_CPUS:-2}", memory: "${K6_MEMORY:-512m}"}
|
||||
|
||||
secrets:
|
||||
postgres_password:
|
||||
file: ./secrets/postgres_password
|
||||
database_url:
|
||||
file: ./secrets/database_url
|
||||
valkey.conf:
|
||||
file: ./secrets/valkey.conf
|
||||
cache_url:
|
||||
file: ./secrets/cache_url
|
||||
cursor_secret:
|
||||
file: ./secrets/cursor_secret
|
||||
s3_root_user:
|
||||
file: ./secrets/s3_root_user
|
||||
s3_root_password:
|
||||
file: ./secrets/s3_root_password
|
||||
s3_access_key_id:
|
||||
file: ./secrets/s3_access_key_id
|
||||
s3_secret_access_key:
|
||||
file: ./secrets/s3_secret_access_key
|
||||
s3_reconciler_access_key_id:
|
||||
file: ./secrets/s3_reconciler_access_key_id
|
||||
s3_reconciler_secret_access_key:
|
||||
file: ./secrets/s3_reconciler_secret_access_key
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
valkeydata:
|
||||
s3data:
|
||||
|
||||
networks:
|
||||
edge:
|
||||
backend:
|
||||
internal: true
|
||||
@@ -34,7 +34,8 @@
|
||||
"chat_id": "33333333-3333-4333-8333-333333333333",
|
||||
"sender_id": "11111111-1111-4111-8111-111111111111",
|
||||
"text": "Привет, Боб!",
|
||||
"created_at": "2026-09-01T12:00:00Z"
|
||||
"created_at": "2026-09-01T12:00:00Z",
|
||||
"attachment_ids": []
|
||||
},
|
||||
"MessagePage": {
|
||||
"items": [
|
||||
@@ -43,7 +44,8 @@
|
||||
"chat_id": "33333333-3333-4333-8333-333333333333",
|
||||
"sender_id": "11111111-1111-4111-8111-111111111111",
|
||||
"text": "Привет, Боб!",
|
||||
"created_at": "2026-09-01T12:00:00Z"
|
||||
"created_at": "2026-09-01T12:00:00Z",
|
||||
"attachment_ids": []
|
||||
}
|
||||
],
|
||||
"next_cursor": null
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
{
|
||||
"lab": 2,
|
||||
"lab": 4,
|
||||
"kind": "requirements, not runnable deployment",
|
||||
"required_for_grade_3": [
|
||||
{
|
||||
"role": "api",
|
||||
"count_min": 1,
|
||||
"count_min": 2,
|
||||
"student_implements": true,
|
||||
"persistent_local_state": false
|
||||
},
|
||||
@@ -12,6 +12,21 @@
|
||||
"role": "postgres",
|
||||
"purpose": "users/chats/membership/messages/metadata",
|
||||
"persistent_volume": true
|
||||
},
|
||||
{
|
||||
"role": "load_balancer",
|
||||
"public_entrypoint": true,
|
||||
"backend_readiness": true
|
||||
},
|
||||
{
|
||||
"role": "s3",
|
||||
"choose_one": [
|
||||
"MinIO",
|
||||
"SeaweedFS",
|
||||
"RustFS"
|
||||
],
|
||||
"private_bucket": true,
|
||||
"persistent_volume": true
|
||||
}
|
||||
],
|
||||
"optional_components": [
|
||||
|
||||
+1487
-7
File diff suppressed because it is too large
Load Diff
+110
-3
@@ -1,5 +1,112 @@
|
||||
# Доказательства
|
||||
# Индекс доказательств ЛР4
|
||||
|
||||
Храните небольшие обезличенные результаты: команды и stdout проверок, JSON/CSV агрегатов нагрузки, EXPLAIN, dashboard JSON, текст разбора отказа. Укажите дату, commit, аппаратные лимиты и профиль нагрузки.
|
||||
Финальный validation snapshot: `5e49502ae78fa48b992c97418e490666c699726d`, 2026-11-06.
|
||||
Runtime совпадает с review `7a0ebac2aa9e1d8602054ca4c7c1b0c23d3bfad4`;
|
||||
[аудит применимости и новые проверки](load-validation.md). Сохранённые cost и текстовая
|
||||
нагрузка приняты с исходными SHA. Старые upload-memory остаются историческими.
|
||||
|
||||
Не добавляйте .env, session tokens, cookies, приватные ключи, credentials, реальные сообщения, raw dumps и signed URLs. Для скриншотов скрывайте секреты. Большие логи/бинарные traces сохраняйте отдельно по правилам преподавателя; одного скриншота без методики недостаточно.
|
||||
## Финальная validation после PNG-фиксов
|
||||
|
||||
| Команда / назначение | Наблюдение | Файл |
|
||||
| --- | --- | --- |
|
||||
| `make student-test PYTEST_ARGS='-q --tb=short'`, затем адресный повтор двух S3-тестов | 719 PASS + 2 setup errors RustFS; только эти два повторены: 2 PASS. 721 выбранная проверка исполнена; первый запуск не объявлен зелёным | [полный](png-final-2026-11-06/student-test.txt), [повтор](png-final-2026-11-06/student-s3-retry.txt) |
|
||||
| `make up`, `make test`, `scripts/limits.py` | Новый изолированный project, fresh volumes, штатные лимиты, smoke PASS | [up](png-final-2026-11-06/up.txt), [smoke](png-final-2026-11-06/smoke.txt), [лимиты](png-final-2026-11-06/limits.txt) |
|
||||
| Прямой argv `make upload-faults`, новый путь захвата | 7 PASS; оба PNG → 415; негативный suite: 0×5xx | [raw](png-final-2026-11-06/upload-faults.txt) |
|
||||
| `png-roundtrip.py` | 415 без новых строк/объектов; PNG/JPEG → 201 и побайтное скачивание | [raw](png-final-2026-11-06/png-roundtrip.json) |
|
||||
| `make upload-memory ARGS='--slots 3 --out <новый-runN>'`, 3 серии | 18 точек, 840/840×201, API ≤440 MiB, LB ≤31,1 MiB, OOM 0 | [сводка](png-final-2026-11-06/memory-summary.json), [raw](png-final-2026-11-06/upload-memory/) |
|
||||
| `high-depth.py` | 10 параллельных 40 MP RGBA16, 10×201, checksum совпал, API ≤270,828 MiB, OOM 0 | [raw](png-final-2026-11-06/high-depth.json) |
|
||||
| `scripts/reconcile_check.py` после RAM | 859 ready = 859 объектов, расхождений/остатков нет; shallow | [raw](png-final-2026-11-06/post-memory-invariants.json) |
|
||||
| `concurrent-reconcile.py` (только under_load) | 3341×201, 67 проходов, 3 orphan удалены; deep SHA/lost_ready=0. Pending инъекция 0 на пустом dataset, прежний сценарий F сохранён | [raw](png-final-2026-11-06/concurrent-reconcile/under-load.json), [oracle](png-final-2026-11-06/concurrent-reconcile/check-under-load.json) |
|
||||
| Повторный аудит 27 cost-замеров | Приняты все 27; страницы/память подтверждены, линейность времени отвергнута | [аудит](png-final-2026-11-06/cost-reaudit.json), [diff/SHA](png-final-2026-11-06/retained-audit.json) |
|
||||
|
||||
## Offline-проверки до финальной validation
|
||||
|
||||
| Команда / назначение | Результат | SHA и файлы |
|
||||
| --- | --- | --- |
|
||||
| `make student-test PYTEST_ARGS='-q --tb=short student-tests/unit'` | 407 passed; оба повреждённых PNG отклоняются до записи | Код `7c8a1a64899dc0a1b5fa0949c0992ef40722eacf`; [unit](png-header-fix/unit.txt), [227 проверок инспектора/ASGI](png-header-fix/targeted.txt), [blob-манифест](png-header-fix/manifest.json) |
|
||||
| `make check`, `make validate`, `make lint`, `openspec validate --all --strict --no-interactive` после фикса | PASS, 18 спецификаций после архивации | Архив `7a0ebac2aa9e1d8602054ca4c7c1b0c23d3bfad4`; [протокол](png-header-fix/NOTES.md), [OpenSpec](png-header-fix/openspec-final.txt) |
|
||||
| Статическое/offline-ревью runner, round3 | Открытых major нет, один minor; Docker/нагрузки не выполнялись | Результат передан runner для `7a0ebac2aa9e1d8602054ca4c7c1b0c23d3bfad4`; отдельного raw-файла ревью в репозитории нет |
|
||||
| Документационный аудит: check/validate/lint/OpenSpec, числа, ссылки, diff | Проверки отчётного этапа отдельно от предыдущих тестов | [Аудит](report-audit-2026-11-06.md); исходный snapshot указан выше |
|
||||
| Финальные API/LB/RustFS-регрессии, память нового инспектора и upload при сверке | Выполненные результаты финальной фазы вынесены выше и в аудит применимости | [Общий долг двух PNG-фиксов](png-header-fix/NOTES.md#обязательные-проверки-финального-этапа) |
|
||||
|
||||
Первый PNG-фикс: `9387c3d82ab77ea376eecce19a36a22803d35efb`,
|
||||
[причина и offline-доказательства](png-validation-fix/NOTES.md).
|
||||
Второй фикс требует единственный IHDR; вместе они добавляют распаковку PNG ограниченными
|
||||
блоками. Все предыдущие upload-memory и конкурентные upload в исторических разделах ниже
|
||||
относятся к старому инспектору. Collection API/stack в `png-header-fix/` — сбор тестов, не их исполнение.
|
||||
|
||||
## После позднего переноса ЛР3, до PNG-фиксов
|
||||
|
||||
Runtime SHA **F**: `75b93bbba5f29575d3fca1d013cc7e0770d88082`, архив
|
||||
`8bc2da2c094afc7e5f5e97c7beb5f278e76253fd`. Источник — immutable snapshot ЛР3
|
||||
`ba78cf6da439a4ec60fd59389583b38285b0b2b1`, диапазон от `f591fa64e525679a31c9db7f942d36c789e1a12f`.
|
||||
[Все 214 путей и область применимости](forward-port.md), [source-manifest](forward-final-2026-11-06/source-manifest.json).
|
||||
Все успешные команды таблицы — на F; их полный argv, UTC, разрешённое окружение и exit code
|
||||
находятся в raw. Пользователи, данные и файлы синтетические.
|
||||
|
||||
| Проверка / команда | Наблюдение | Файл |
|
||||
| --- | --- | --- |
|
||||
| `make check`, `make validate`, `make lint`, `make student-test` | PASS, 492 теста / 19 stack deselected | [offline-checks-rerun](forward-final-2026-11-06/offline-checks-rerun.txt) |
|
||||
| `make up`, `make test` | Обе реплики healthy; smoke/checksum/ACL PASS | [up](forward-final-2026-11-06/up.txt), [smoke](forward-final-2026-11-06/smoke.txt), [после нагрузок](forward-final-2026-11-06/final-smoke.txt) |
|
||||
| `sh scripts/persistence-evidence.sh http://127.0.0.1:18084` | PASS; StartedAt API/LB/PG/Valkey/S3 изменились | [persistence-rerun](forward-final-2026-11-06/persistence-rerun.txt) |
|
||||
| `make test-failover ACTION_SCRIPT=scripts/stop-one.sh` | Target exited; 130 записей, 0 ошибок, 2,883 с до первой записи | [команда](forward-final-2026-11-06/failover-command.txt), [вывод/inspect](forward-final-2026-11-06/failover-d4c8d53d7433.txt) |
|
||||
| `make scenario SCENARIO=failover-upload`; отдельный `make restore` | Другая реплика отдала те же байты; восстановление выполнено отдельно | [команда](forward-final-2026-11-06/failover-upload-command.txt), [JSON](forward-final-2026-11-06/failover-upload.json), [restore](forward-final-2026-11-06/restore.txt) |
|
||||
| `make scenario SCENARIO=s3-outage` | 503 за 4,990/5,329 с; текст 201; checksum после старта совпал | [команда](forward-final-2026-11-06/s3-outage-command.txt), [JSON](forward-final-2026-11-06/s3-outage.json) |
|
||||
| `make test-faults` | 19 PASS, 112 ответов негативной файловой проверки без 5xx; компенсация/crash/grace/stall/PG/Valkey | [stack-faults](forward-final-2026-11-06/stack-faults.txt) |
|
||||
| `student-tests/tools/forward_network.py`; HAProxy fixture | Сессии/NUL/Basic PASS; 12 проверок реальных 502/503/504 и request_id | [сеть](forward-final-2026-11-06/network-regressions.txt), [LB](forward-final-2026-11-06/haproxy-errors.txt) |
|
||||
| `make limits` | Ресурсы, порты, mounts, сети и роли секретов PASS | [limits](forward-final-2026-11-06/limits.txt) |
|
||||
| `make load-evidence` | 3 повтора, медиана 1456,338 RPS, 0 ошибок; текстовый путь не менялся | [команда](forward-final-2026-11-06/load-command.txt), [вывод](forward-final-2026-11-06/load.txt), [raw JSON](forward-final-2026-11-06/load/) |
|
||||
| `scripts/upload_memory.py --slots 3 --sizes 10485760 --levels 40 --out ...` | 3×80: 238×201, 2×503; пик API 456,2 MiB, OOM 0. Историческая память до PNG-фиксов | [run1](forward-final-2026-11-06/memory-1.txt), [run2](forward-final-2026-11-06/memory-2.txt), [run3](forward-final-2026-11-06/memory-3.txt), [raw](forward-final-2026-11-06/memory/) |
|
||||
| `make reconcile-demo ARGS='--out evidence/forward-final-2026-11-06/reconcile-demo'` | 13 PASS; 3364 upload, 59 проходов, lost_ready 0. Конкурентные upload требуют повторения после PNG-фиксов | [команда](forward-final-2026-11-06/reconcile-demo.txt), [demo](forward-final-2026-11-06/reconcile-demo/demo.json), [deep-аудит](forward-final-2026-11-06/reconcile-demo/check-under-load.json) |
|
||||
| `git diff --exit-code` измеряемых файлов | Стоимость reconciler сохранена с исходным SHA, не переснята на F | [retained-cost-audit](forward-final-2026-11-06/retained-cost-audit.txt) |
|
||||
| Воспроизведение непарных surrogate в text | Два HTTP 500: известный **minor**, не успешный тест | [HTTP](forward-final-2026-11-06/known-minor-http.txt), [offline](forward-final-2026-11-06/known-minor.txt), [finding](forward-final-2026-11-06/findings.json) |
|
||||
| `make down` своих project, fixture `docker compose down`, инвентаризация по labels | 0 контейнеров трёх project, 6 volumes сохранены | [main](forward-final-2026-11-06/main-down.txt), [demo](forward-final-2026-11-06/reconcile-down.txt), [итог](forward-final-2026-11-06/shutdown-state.txt) |
|
||||
|
||||
[Сводка чисел](forward-final-2026-11-06/measurement-summary.json),
|
||||
[команда агрегации](forward-final-2026-11-06/measurement-summary.txt),
|
||||
[фон до](forward-final-2026-11-06/performance-before.txt) и
|
||||
[после](forward-final-2026-11-06/performance-after.txt).
|
||||
Сохранены неуспешные попытки: [ty до исправления обвязки](forward-final-2026-11-06/offline-checks.txt)
|
||||
и [неверный BASE_URL до hook](forward-final-2026-11-06/persistence-command.txt); они не считаются PASS.
|
||||
|
||||
## Исходная SQLAlchemy-серия
|
||||
|
||||
Runtime основы: `8e4532f6f5c05d902b6e2c310f54320d95585904`. Все результаты ниже сняты
|
||||
**до F и обоих PNG-фиксов**. Обвязка менялась отдельно; точный SHA каждого захвата записан
|
||||
в его первой строке/JSON. [Общий протокол](sqlalchemy-2026-11-06/README.md).
|
||||
|
||||
| Команда / критерий | Наблюдение | SHA обвязки и файлы |
|
||||
| --- | --- | --- |
|
||||
| `make test`, `make limits`, `make attachments-evidence`, LB/retry/отказы | Уровень 3 на SQLAlchemy, 404 собственных теста | `1aec062aba4560b51a6d9155571c9610bc777d7e`, `763b0255575366082c4c6731b54611af717685a5`; [grade3](sqlalchemy-2026-11-06/grade3/NOTES.md) |
|
||||
| `make s3-check` | 31 PASS; private/IAM/bootstrap не менялись с этой серии | `763b0255575366082c4c6731b54611af717685a5`; [сырьё](sqlalchemy-2026-11-06/grade3/s3-security.txt) |
|
||||
| `make upload-memory`, три полные матрицы | 36 точек, штатные 3 слота 840/840×201, пик 448 MiB, OOM 0; контроль 64 слота 27×503 | `6c706a727ec7b4c5f8254e19401705847e8e9c78`, `cf41f7c64dcf1dea6da0643f5793bd2e5dd1f03e`; [методика](sqlalchemy-2026-11-06/grade4/NOTES.md), [raw](sqlalchemy-2026-11-06/grade4/memory/) |
|
||||
| `sh scripts/clean-clone.sh` | 76,866 с, проверки прошли, три volumes сохранены | `cf41f7c64dcf1dea6da0643f5793bd2e5dd1f03e`; [команда/вывод](sqlalchemy-2026-11-06/grade4/clean-clone-command.txt) |
|
||||
| `make overload-leftovers` | Без сдвига времени убраны 30 pending после grace 900 с; 1669 ready/объектов совпали | SHA в [команде](sqlalchemy-2026-11-06/grade4/leftovers-command.txt); [вывод](sqlalchemy-2026-11-06/grade4/overload-leftovers.txt) |
|
||||
| `make load-evidence` | 3 прогона, медиана 1529,186 RPS, 0 ошибок; историческая серия до F | SHA в [команде](sqlalchemy-2026-11-06/grade4/load-command.txt); [raw](sqlalchemy-2026-11-06/grade4/load/) |
|
||||
| `make reconcile-demo`, отдельный отказ PG | 13 PASS, 3331 upload, lost_ready 0; обновлён demo серии F | `827b783be355be9fa60117dc3eda350af3dab454`, `3402fd69d4922e500035a204d957be2cccaeb85a`; [протокол](sqlalchemy-2026-11-06/track5/NOTES.md), [PG](sqlalchemy-2026-11-06/track5/pg-outage.txt) |
|
||||
| `make reconcile-cost` | **Сохраняемые 27 замеров** до 100k; путь обхода неизменён | `3402fd69d4922e500035a204d957be2cccaeb85a`; [команда](sqlalchemy-2026-11-06/track5/cost-command.txt), [raw/план/фон](sqlalchemy-2026-11-06/track5/measurements/cost/), [расчёт](sqlalchemy-2026-11-06/track5/analysis.json) |
|
||||
| Финальные проверки исходной реализации | 410 тестов, check/validate/lint/smoke/limits PASS; не тесты нового PNG | `504f77cb7cef28202e1830fc04e10e646488e778`; [вывод](sqlalchemy-2026-11-06/track5/final-checks.txt), [остановка 4 project / 12 volumes](sqlalchemy-2026-11-06/track5/shutdown-state.txt) |
|
||||
|
||||
[Закрытие 21 замечания исходной реализации](sqlalchemy-2026-11-06/REVIEW-CLOSURE.md)
|
||||
относится к тому этапу. Оно предшествует независимым PNG-находкам и не отменяет их историю.
|
||||
|
||||
## Историческая реализация asyncpg
|
||||
|
||||
**Эти результаты не подтверждают нынешний SQLAlchemy-код или PNG-инспектор.**
|
||||
Основная серия и старый clean-clone: `81dd816df21f820a7fd0fa4269cc375d8fc44155`;
|
||||
reconciler: `61dcb6bf13f3d0853d36788930d79bca5f68ece6`. Дата и измеренный SHA
|
||||
сохранены в raw. Старые команды удаления volumes — история, не инструкция для повторения.
|
||||
|
||||
| Историческая команда | Сохранённые файлы |
|
||||
| --- | --- |
|
||||
| `make check`, `make validate`, `make student-test`, `make test` | [контракт](contract-checks.txt), [тесты](student-tests.txt), [smoke](smoke.txt) |
|
||||
| `make clean-clone-evidence`, `make stack-evidence`, `make limits`, `make attachments-evidence` | [clone](clean-clone.txt), [стенд](stack.txt), [окружение](environment.txt), [лимиты](limits.txt), [размещение](storage-layout.txt) |
|
||||
| `make lb-check`, `make retry-check`, persistence и сценарии отказов | [LB](lb-check.json), [retry](retry.json), [persistence](persistence.txt), [upload failover](failover-upload.json), [S3 outage](s3-outage.json) |
|
||||
| `make upload-faults`, `make faults-evidence`, `make s3-check` | [upload](upload-faults.txt), [stack](faults.txt), [S3](s3-security.txt) |
|
||||
| `make upload-memory`, `make overload-leftovers`, `make load-evidence` | [память](upload-memory/), [остатки](overload-leftovers.txt), [нагрузка](load.txt), [raw](load/), [отброшенная попытка с хоста](load/host-attempt.txt) |
|
||||
| `make reconcile-demo`, `make reconcile-cost` | [методика и все серии](reconcile/README.md), [raw](reconcile/), [cost](reconcile/cost/) |
|
||||
|
||||
Прежние cost-серии `b1916236f4391bfdf7d7718ebf741621047788cf` (лишняя SQL-страница),
|
||||
`5ca2cdc9af6016df8d6f1be8bd4681f06fada334` (без фильтра фона) и прерванные наполнения
|
||||
описаны в [историческом индексе reconciler](reconcile/README.md). Они не подмешиваются
|
||||
к 27 принятым измерениям SQLAlchemy и не удаляются.
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
date: 2026-10-30T04:27:15Z commit: 81dd816 dir: mtusi2026lab4-clean (свой compose-проект и volumes)
|
||||
# шаги — команды блока «Запуск из чистого clone» REPORT.md; строки с пометкой «наблюдение» — проверки между шагами
|
||||
$ git clone <repo> mtusi2026lab4-clean && cd mtusi2026lab4-clean && git checkout solution/bpi2303-kostylev
|
||||
HEAD 81dd816 Archive the storage reconciler change
|
||||
$ docker volume ls -q --filter label=com.docker.compose.project=mtusi2026lab4-clean | wc -l | tr -d ' ' # наблюдение, не шаг инструкции
|
||||
0
|
||||
$ make secrets
|
||||
sh scripts/secrets.sh
|
||||
создан secrets/postgres_password
|
||||
создан secrets/database_url
|
||||
создан secrets/cursor_secret
|
||||
создан secrets/valkey_password
|
||||
создан secrets/valkey.conf
|
||||
создан secrets/cache_url
|
||||
создан secrets/s3_root_user
|
||||
создан secrets/s3_root_password
|
||||
создан secrets/s3_access_key_id
|
||||
создан secrets/s3_secret_access_key
|
||||
создан .env из .env.example
|
||||
$ sed LB_HTTP_PORT=18084 LB_STATS_PORT=24084 .env # отступление: 8080 на этой машине занят стендами других лаб
|
||||
$ git status --short --ignored secrets .env # наблюдение, не шаг инструкции
|
||||
!! .env
|
||||
!! secrets/
|
||||
$ docker compose --env-file .env config --quiet && echo valid
|
||||
valid
|
||||
$ make up
|
||||
Container mtusi2026lab4-clean-postgres-1 Healthy
|
||||
Container mtusi2026lab4-clean-s3-1 Healthy
|
||||
Container mtusi2026lab4-clean-s3-init-1 Exited
|
||||
Container mtusi2026lab4-clean-migrate-1 Exited
|
||||
Container mtusi2026lab4-clean-postgres-1 Healthy
|
||||
Container mtusi2026lab4-clean-migrate-1 Exited
|
||||
Container mtusi2026lab4-clean-valkey-1 Healthy
|
||||
Container mtusi2026lab4-clean-s3-1 Healthy
|
||||
Container mtusi2026lab4-clean-s3-init-1 Exited
|
||||
Container mtusi2026lab4-clean-api-1 Healthy
|
||||
Container mtusi2026lab4-clean-api-2 Healthy
|
||||
Container mtusi2026lab4-clean-lb-1 Healthy
|
||||
make up: 11 s (сборка образа из кэша слоёв)
|
||||
$ docker compose ps -a --format '{{.Service}} {{.State}} {{.Status}} {{.Ports}}' # наблюдение, не шаг инструкции
|
||||
api running Up 4 seconds (healthy)
|
||||
api running Up 4 seconds (healthy)
|
||||
lb running Up 10 seconds (healthy) 127.0.0.1:18084->8080/tcp, 127.0.0.1:24084->8404/tcp
|
||||
migrate exited Exited (0) 8 seconds ago
|
||||
postgres running Up 10 seconds (healthy)
|
||||
s3 running Up 10 seconds (healthy)
|
||||
s3-init exited Exited (0) 4 seconds ago
|
||||
valkey running Up 10 seconds (healthy)
|
||||
$ docker compose logs --no-log-prefix migrate s3-init | grep -v '^$' | tail -8 # наблюдение, не шаг инструкции
|
||||
{"version":"0001_init","event":"migration.applied","level":"info","ts":"2026-10-30T04:27:18.183850Z"}
|
||||
{"version":"0002_idempotency_keys","event":"migration.applied","level":"info","ts":"2026-10-30T04:27:18.183962Z"}
|
||||
{"version":"0003_attachments","event":"migration.applied","level":"info","ts":"2026-10-30T04:27:18.184008Z"}
|
||||
{"version":"0004_attachment_quarantine","event":"migration.applied","level":"info","ts":"2026-10-30T04:27:18.184039Z"}
|
||||
{"reason":"applied","event":"migration.completed","level":"info","ts":"2026-10-30T04:27:18.184066Z"}
|
||||
s3-init: bucket=messenger-attachments anonymous=private policy=messenger-api lifecycle=abort-incomplete-uploads
|
||||
$ curl -s "http://$(docker compose port lb 8404)/stats;csv" | awk -F, '$1 == "api" && $2 != "BACKEND" {print $2, $18}' | grep -v MAINT # наблюдение, не шаг инструкции
|
||||
api1 UP
|
||||
api2 UP
|
||||
$ make test
|
||||
python3 tests/smoke.py --base-url "http://127.0.0.1:18084"
|
||||
PASS: lab4 public API scenario; infrastructure and grading evidence remain separate.
|
||||
$ make test-persistence ACTION_SCRIPT=scripts/restart.sh
|
||||
python3 tests/resilience.py --mode persistence --action-script "scripts/restart.sh" --base-url "http://127.0.0.1:18084"
|
||||
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
|
||||
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
$ docker compose ps -a --format '{{.Service}} {{.State}} {{.RunningFor}} {{.Status}}' # наблюдение, не шаг инструкции
|
||||
api running 24 seconds ago Up 3 seconds (healthy)
|
||||
api running 24 seconds ago Up 3 seconds (healthy)
|
||||
lb running 24 seconds ago Up 8 seconds (healthy)
|
||||
migrate exited 24 seconds ago Exited (0) 7 seconds ago
|
||||
postgres running 24 seconds ago Up 9 seconds (healthy)
|
||||
s3 running 24 seconds ago Up 9 seconds (healthy)
|
||||
s3-init exited 24 seconds ago Exited (0) 3 seconds ago
|
||||
valkey running 24 seconds ago Up 9 seconds (healthy)
|
||||
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh
|
||||
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://127.0.0.1:18084"
|
||||
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
|
||||
{"observed_instances_before": 2, "successful_writes": 128, "failed_attempts": 0, "first_recovery_seconds": 2.948}
|
||||
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
$ docker compose ps -a --format '{{.Service}} {{.Name}} {{.State}} {{.Status}}' api # наблюдение, не шаг инструкции
|
||||
api mtusi2026lab4-clean-api-1 running Up 19 seconds (healthy)
|
||||
api mtusi2026lab4-clean-api-2 exited Exited (0) 12 seconds ago
|
||||
$ make restore
|
||||
scripts/restore.sh
|
||||
Container mtusi2026lab4-clean-api-1 Running
|
||||
Container mtusi2026lab4-clean-api-2 Starting
|
||||
Container mtusi2026lab4-clean-api-2 Started
|
||||
Container mtusi2026lab4-clean-api-1 Waiting
|
||||
Container mtusi2026lab4-clean-api-2 Waiting
|
||||
Container mtusi2026lab4-clean-api-1 Healthy
|
||||
Container mtusi2026lab4-clean-api-2 Healthy
|
||||
$ make scenario SCENARIO=failover-upload
|
||||
{'ok': True, 'upload_instance': 'b21945b926b7', 'download_instance': '6c9bf1e3b1a6', 'sha256_equal': True, 'hook_seconds': 2.713}
|
||||
$ make scenario SCENARIO=s3-outage
|
||||
{'ok': True, 'upload': 503, 'metadata': 200, 'download': 503, 'text_message': 201, 'health_ready': 200}
|
||||
$ make s3-check
|
||||
23
|
||||
== 2026-10-30T04:28:17Z S3 privacy and bootstrap checks
|
||||
== IAM/bucket snapshot (без секретов)
|
||||
$ make reconcile
|
||||
dry-run {'objects': 5, 'rows': 6} {'pending_fresh': 1} errors 0
|
||||
$ make migrate
|
||||
docker compose run --rm migrate
|
||||
{"reason":"up to date","event":"migration.completed","level":"info","ts":"2026-10-30T04:28:21.735485Z"}
|
||||
$ docker volume ls --format '{{.Name}}' --filter label=com.docker.compose.project=mtusi2026lab4-clean # наблюдение, не шаг инструкции
|
||||
mtusi2026lab4-clean_pgdata
|
||||
mtusi2026lab4-clean_s3data
|
||||
mtusi2026lab4-clean_valkeydata
|
||||
$ make down
|
||||
1 Container mtusi2026lab4-clean-api-1 Removed
|
||||
1 Container mtusi2026lab4-clean-api-2 Removed
|
||||
1 Container mtusi2026lab4-clean-lb-1 Removed
|
||||
1 Container mtusi2026lab4-clean-migrate-1 Removed
|
||||
1 Container mtusi2026lab4-clean-postgres-1 Removed
|
||||
1 Container mtusi2026lab4-clean-s3-1 Removed
|
||||
1 Container mtusi2026lab4-clean-s3-init-1 Removed
|
||||
1 Container mtusi2026lab4-clean-valkey-1 Removed
|
||||
1 Network mtusi2026lab4-clean_backend Removed
|
||||
1 Network mtusi2026lab4-clean_edge Removed
|
||||
$ docker volume ls --format '{{.Name}}' --filter label=com.docker.compose.project=mtusi2026lab4-clean # наблюдение, не шаг инструкции
|
||||
mtusi2026lab4-clean_pgdata
|
||||
mtusi2026lab4-clean_s3data
|
||||
mtusi2026lab4-clean_valkeydata
|
||||
$ make reset-data
|
||||
Volume mtusi2026lab4-clean_pgdata Removing
|
||||
Volume mtusi2026lab4-clean_s3data Removing
|
||||
Volume mtusi2026lab4-clean_valkeydata Removing
|
||||
Volume mtusi2026lab4-clean_pgdata Removed
|
||||
Volume mtusi2026lab4-clean_s3data Removed
|
||||
Volume mtusi2026lab4-clean_valkeydata Removed
|
||||
$ docker volume ls -q --filter label=com.docker.compose.project=mtusi2026lab4-clean | wc -l | tr -d ' ' # наблюдение, не шаг инструкции
|
||||
0
|
||||
@@ -0,0 +1,12 @@
|
||||
date: 2026-10-30T04:24:13Z commit: 81dd816
|
||||
$ make check
|
||||
python3 scripts/check_contract.py
|
||||
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
|
||||
$ make validate # openapi-spec-validator 0.7.2 из .venv (make install)
|
||||
.venv/bin/python scripts/validate_openapi.py
|
||||
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
|
||||
$ git diff --stat upstream/main -- contracts tests README.md COURSE.md CONTRIBUTING.md MAINTAINERS.md lab.json compose.example.yaml Dockerfile.example # контракт, публичные тесты и задание не менялись
|
||||
(пусто)
|
||||
$ git diff --stat upstream/main -- .github # только свой CI job
|
||||
.github/workflows/application.yml | 93 +++++++++++++++++++++++++++++++++++++++
|
||||
1 file changed, 93 insertions(+)
|
||||
@@ -0,0 +1,41 @@
|
||||
date: 2026-10-30T04:29:05Z commit: 81dd816
|
||||
host: Apple M4 Max, 16 CPU, 48 GiB RAM, macOS 15.1
|
||||
docker: 28.3.2, VM: 16 CPU, 8217190400 bytes
|
||||
compose: 2.38.2-desktop.1
|
||||
images:
|
||||
image: ${API_IMAGE:-mtusi-messenger:${COMPOSE_PROJECT_NAME}}
|
||||
image: rustfs/rc:v0.1.36@sha256:ab024bfebee49a750ce886b4c70963ccd9ddaa03f491704a90710641d7a26699
|
||||
image: haproxy:3.2.26-alpine@sha256:5893840576071e6217cd52ebdcc14f454c6f7a631056fa6f4e0f40d3cac4215b
|
||||
image: postgres:17.11-alpine@sha256:266d3c257594e195b309ae9d1805e54c788f1d7b375f56bf6c367aac6c917e7b
|
||||
image: valkey/valkey:8.1.11-alpine@sha256:2602ace2be6569bc2bbb06f873776516ab9089821525be34f4305c67eb9db02a
|
||||
image: rustfs/rustfs:1.0.4@sha256:ed9210d6411d77ffceb70d94b964a3cb7d225983368bc25bebe01e995293dcc2
|
||||
image: grafana/k6:2.3.0@sha256:2dd4756ec6e14e62778916e1eb850108e4b499304c02df56760fa0bdb0fba0b8
|
||||
FROM python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d AS builder
|
||||
COPY --from=ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142 /uv /usr/local/bin/uv
|
||||
FROM python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d AS runtime
|
||||
python in image: Python 3.14.7
|
||||
haproxy: HAProxy version 3.2.26-3e0f11a 2026/10/14 - https://haproxy.org/
|
||||
rustfs: rustfs 1.0.4
|
||||
packages (uv.lock):
|
||||
fastapi 0.141.1
|
||||
uvicorn 0.54.0
|
||||
uvicorn-worker 0.4.0
|
||||
gunicorn 26.2.0
|
||||
asyncpg 0.32.0
|
||||
argon2-cffi 25.1.0
|
||||
pydantic 2.13.5
|
||||
pydantic-settings 2.15.0
|
||||
dishka 1.10.1
|
||||
structlog 26.1.0
|
||||
redis 8.1.0
|
||||
aiobotocore 3.10.0
|
||||
botocore 1.43.117
|
||||
aiohttp 3.14.5
|
||||
pillow 13.0.0
|
||||
process users (service uid command):
|
||||
api 10001 python
|
||||
lb 99 haproxy
|
||||
postgres 70 postgres
|
||||
s3 10001 rustfs
|
||||
valkey 999 tini
|
||||
valkey 999 valkey-server
|
||||
@@ -0,0 +1,38 @@
|
||||
date: 2026-10-30T04:29:33Z commit: 81dd816
|
||||
== before: api replicas (hostname = X-Instance-Id), state, StartedAt, FinishedAt
|
||||
0cc3ee358e59 running started=2026-10-30T04:28:59.851144131Z finished=0001-01-01T00:00:00Z
|
||||
899a952a5a21 running started=2026-10-30T04:29:31.104320507Z finished=2026-10-30T04:29:18.153900709Z
|
||||
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh
|
||||
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://127.0.0.1:18084"
|
||||
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
|
||||
{"observed_instances_before": 2, "successful_writes": 128, "failed_attempts": 0, "first_recovery_seconds": 2.883}
|
||||
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
== after: api replicas
|
||||
0cc3ee358e59 exited started=2026-10-30T04:28:59.851144131Z finished=2026-10-30T04:29:36.57527926Z
|
||||
899a952a5a21 running started=2026-10-30T04:29:31.104320507Z finished=2026-10-30T04:29:18.153900709Z
|
||||
== stopped replica: 0cc3ee358e59; shutdown: signal, drain window with readiness 503, close, exit
|
||||
{"event":"Handling signal: term","level":"info","ts":"2026-10-30T04:29:34.045936Z"}
|
||||
{"limit":2.0,"event":"service.draining","level":"info","ts":"2026-10-30T04:29:34.072243Z"}
|
||||
{"limit":2.0,"event":"service.draining","level":"info","ts":"2026-10-30T04:29:34.126665Z"}
|
||||
{"method":"GET","route":"/health/ready","status":503,"request_id":"6362781446fb427cbee670871ce5ef51","duration_ms":1.463,"event":"request.handled","level":"info","ts":"2026-10-30T04:29:34.446815Z"}
|
||||
{"method":"GET","route":"/health/ready","status":503,"request_id":"0d9cf1c47fca4ec98ed599b113cd93db","duration_ms":0.79,"event":"request.handled","level":"info","ts":"2026-10-30T04:29:34.950067Z"}
|
||||
{"event":"Shutting down","level":"info","ts":"2026-10-30T04:29:36.079497Z"}
|
||||
{"event":"Shutting down","level":"info","ts":"2026-10-30T04:29:36.138465Z"}
|
||||
{"event":"service.stopped","level":"info","ts":"2026-10-30T04:29:36.188615Z"}
|
||||
{"event":"Finished server process [20]","level":"info","ts":"2026-10-30T04:29:36.188874Z"}
|
||||
{"event":"service.stopped","level":"info","ts":"2026-10-30T04:29:36.247500Z"}
|
||||
{"event":"Finished server process [19]","level":"info","ts":"2026-10-30T04:29:36.247910Z"}
|
||||
{"event":"Worker (pid:20) was sent SIGTERM!","level":"info","ts":"2026-10-30T04:29:36.250957Z"}
|
||||
{"event":"Worker (pid:19) was sent SIGTERM!","level":"info","ts":"2026-10-30T04:29:36.357386Z"}
|
||||
{"event":"Shutting down: Master","level":"info","ts":"2026-10-30T04:29:36.468491Z"}
|
||||
$ scripts/restore.sh
|
||||
Container mtusi2026lab4-api-2 Running
|
||||
Container mtusi2026lab4-api-1 Starting
|
||||
Container mtusi2026lab4-api-1 Started
|
||||
Container mtusi2026lab4-api-2 Waiting
|
||||
Container mtusi2026lab4-api-1 Waiting
|
||||
Container mtusi2026lab4-api-2 Healthy
|
||||
Container mtusi2026lab4-api-1 Healthy
|
||||
== after restore
|
||||
0cc3ee358e59 running started=2026-10-30T04:29:49.45750296Z finished=2026-10-30T04:29:36.57527926Z
|
||||
899a952a5a21 running started=2026-10-30T04:29:31.104320507Z finished=2026-10-30T04:29:18.153900709Z
|
||||
@@ -0,0 +1,39 @@
|
||||
date: 2026-10-30T04:29:14Z commit: 81dd816
|
||||
== before: api replicas (hostname = X-Instance-Id), state, StartedAt, FinishedAt
|
||||
0cc3ee358e59 running started=2026-10-30T04:28:59.851144131Z finished=0001-01-01T00:00:00Z
|
||||
899a952a5a21 running started=2026-10-30T04:29:11.286693179Z finished=2026-10-30T04:29:11.040183429Z
|
||||
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh
|
||||
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://127.0.0.1:18084"
|
||||
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
|
||||
{"observed_instances_before": 2, "successful_writes": 129, "failed_attempts": 0, "first_recovery_seconds": 2.885}
|
||||
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
== after: api replicas
|
||||
0cc3ee358e59 running started=2026-10-30T04:28:59.851144131Z finished=0001-01-01T00:00:00Z
|
||||
899a952a5a21 exited started=2026-10-30T04:29:11.286693179Z finished=2026-10-30T04:29:18.153900709Z
|
||||
== stopped replica: 899a952a5a21; shutdown: signal, drain window with readiness 503, close, exit
|
||||
{"event":"Handling signal: term","level":"info","ts":"2026-10-30T04:29:15.605798Z"}
|
||||
{"limit":2.0,"event":"service.draining","level":"info","ts":"2026-10-30T04:29:15.638898Z"}
|
||||
{"limit":2.0,"event":"service.draining","level":"info","ts":"2026-10-30T04:29:15.701830Z"}
|
||||
{"method":"GET","route":"/health/ready","status":503,"request_id":"b416afb071ad426a9dde141e952d697f","duration_ms":0.823,"event":"request.handled","level":"info","ts":"2026-10-30T04:29:16.528606Z"}
|
||||
{"method":"GET","route":"/health/ready","status":503,"request_id":"3fe4a4c39cb24a219d3a393a1c0332f0","duration_ms":1.591,"event":"request.handled","level":"info","ts":"2026-10-30T04:29:17.038813Z"}
|
||||
{"method":"GET","route":"/health/ready","status":503,"request_id":"152273913ad34012bf5eaf31432490ae","duration_ms":0.263,"event":"request.handled","level":"info","ts":"2026-10-30T04:29:17.388797Z"}
|
||||
{"event":"Shutting down","level":"info","ts":"2026-10-30T04:29:17.642635Z"}
|
||||
{"event":"Shutting down","level":"info","ts":"2026-10-30T04:29:17.714902Z"}
|
||||
{"event":"service.stopped","level":"info","ts":"2026-10-30T04:29:17.757838Z"}
|
||||
{"event":"Finished server process [20]","level":"info","ts":"2026-10-30T04:29:17.758374Z"}
|
||||
{"event":"Worker (pid:20) was sent SIGTERM!","level":"info","ts":"2026-10-30T04:29:17.826021Z"}
|
||||
{"event":"service.stopped","level":"info","ts":"2026-10-30T04:29:17.832655Z"}
|
||||
{"event":"Finished server process [13]","level":"info","ts":"2026-10-30T04:29:17.833005Z"}
|
||||
{"event":"Worker (pid:13) was sent SIGTERM!","level":"info","ts":"2026-10-30T04:29:17.927620Z"}
|
||||
{"event":"Shutting down: Master","level":"info","ts":"2026-10-30T04:29:18.029723Z"}
|
||||
$ scripts/restore.sh
|
||||
Container mtusi2026lab4-api-1 Running
|
||||
Container mtusi2026lab4-api-2 Starting
|
||||
Container mtusi2026lab4-api-2 Started
|
||||
Container mtusi2026lab4-api-2 Waiting
|
||||
Container mtusi2026lab4-api-1 Waiting
|
||||
Container mtusi2026lab4-api-2 Healthy
|
||||
Container mtusi2026lab4-api-1 Healthy
|
||||
== after restore
|
||||
0cc3ee358e59 running started=2026-10-30T04:28:59.851144131Z finished=0001-01-01T00:00:00Z
|
||||
899a952a5a21 running started=2026-10-30T04:29:31.104320507Z finished=2026-10-30T04:29:18.153900709Z
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"ok": true,
|
||||
"scenario": "failover-upload",
|
||||
"upload_instance": "899a952a5a21",
|
||||
"download_instance": "0cc3ee358e59",
|
||||
"sha256_equal": true,
|
||||
"hook_seconds": 2.72,
|
||||
"download_ok_after_seconds": 2.73,
|
||||
"restored": true
|
||||
}
|
||||
@@ -0,0 +1,106 @@
|
||||
date: 2026-10-30T04:31:11Z commit: 81dd816
|
||||
$ make test-faults
|
||||
BASE_URL="http://127.0.0.1:18084" .venv/bin/pytest -m stack -v -s student-tests/stack
|
||||
============================= test session starts ==============================
|
||||
platform darwin -- Python 3.14.5, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv/bin/python
|
||||
cachedir: .pytest_cache
|
||||
rootdir: <repo>
|
||||
configfile: pyproject.toml
|
||||
plugins: asyncio-1.4.0, anyio-4.15.1
|
||||
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function
|
||||
collecting ... collected 17 items
|
||||
|
||||
student-tests/stack/test_attachment_faults.py::test_user_errors_never_become_500 10 MiB + 1 с Content-Length, тело не отправлено: 413
|
||||
10 MiB + 1 chunked: 413
|
||||
PNG 100000×100000 (бомба): 413
|
||||
PNG 8001×5000: 413
|
||||
обрезанный PNG: 415
|
||||
обрезанный JPEG: 415
|
||||
JPEG как image/png: 415
|
||||
GIF как image/png: 415
|
||||
WebP как image/png: 415
|
||||
текст как image/png: 415
|
||||
PNG как image/jpeg: 415
|
||||
пустое тело: 415
|
||||
application/octet-stream: 415
|
||||
без Content-Type: 415
|
||||
filename='../a.png': 400
|
||||
filename='a/b.png': 400
|
||||
filename='a\\b.png': 400
|
||||
filename='a\x00.png': 400
|
||||
filename='a\x01.png': 400
|
||||
filename='a\x7f.png': 400
|
||||
filename='': 400
|
||||
filename=' ': 400
|
||||
filename='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx: 400
|
||||
filename=None: 400
|
||||
посторонний: upload в чужой чат: 404
|
||||
посторонний: метаданные: 404
|
||||
посторонний: /content: 404
|
||||
посторонний: ссылка на чужое вложение: 404
|
||||
вложение другого чата: 404
|
||||
неизвестное вложение: 404
|
||||
11 вложений: 400
|
||||
повтор вложения: 400
|
||||
невалидный UUID: 400
|
||||
пустой текст без вложений: 400
|
||||
невалидный id вложения: 400
|
||||
variant=thumbnail: 400
|
||||
строк вложений в чате: 2 (= ответов 201); ответов в журнале реплик: 54, из них 5xx: 0
|
||||
PASSED
|
||||
student-tests/stack/test_attachment_faults.py::test_commit_failure_is_compensated_at_once отказ COMMIT после записи объекта: 503 unavailable
|
||||
сразу после ответа: строка нет, объект HEAD 404
|
||||
журнал: audit.attachment_discarded reason=compensation
|
||||
PASSED
|
||||
student-tests/stack/test_attachment_faults.py::test_crash_between_put_and_commit_is_swept_after_grace реплика 899a952a5a21 убита между PUT и COMMIT, клиент получил 502
|
||||
реплика 0cc3ee358e59 убита между PUT и COMMIT, клиент получил 502
|
||||
остаток после падения: строка pending, объект HEAD 200
|
||||
остаток после падения: строка pending, объект HEAD 200
|
||||
старше grace: строка убрана через 26.2 с, объект HEAD 404
|
||||
моложе grace: строка pending, объект HEAD 200
|
||||
тот же после сдвига: убран через 1.0 с, объект HEAD 404
|
||||
готовое вложение в сообщении со сдвинутым created_at скачивается, SHA-256 совпадает
|
||||
PASSED
|
||||
student-tests/stack/test_attachment_faults.py::test_aborted_upload_leaves_nothing обрыв после 256 KiB из 5 MiB: статус в журнале реплики [400], намерение None
|
||||
PASSED
|
||||
student-tests/stack/test_attachment_faults.py::test_stalled_upload_is_cut_and_leaves_nothing тело зависло после 1 KiB: 503 unavailable за 8.01 с
|
||||
PASSED
|
||||
student-tests/stack/test_faults.py::test_registration_race_yields_exactly_one_user гонка 50 регистраций: 201×1, 409×49
|
||||
PASSED
|
||||
student-tests/stack/test_faults.py::test_chat_creation_rolls_back_on_unknown_member чат с несуществующим участником: 400 invalid_request
|
||||
PASSED
|
||||
student-tests/stack/test_faults.py::test_wrong_password_and_unknown_user_get_same_basic_challenge неверный пароль: 401 за 0.011 с
|
||||
неизвестный пользователь: 401 за 0.011 с
|
||||
PASSED
|
||||
student-tests/stack/test_faults.py::test_logout_then_token_is_dead logout: 204, затем 401, повтор 401
|
||||
PASSED
|
||||
student-tests/stack/test_faults.py::test_valkey_serves_repeated_checks 5 проверок токена: keyspace_hits 119 → 124
|
||||
PASSED
|
||||
student-tests/stack/test_faults.py::test_valkey_stopped_is_bypassed_but_logout_waits valkey остановлен, действующий токен: 200 за 0.004 с
|
||||
valkey остановлен, неизвестный токен: 401 за 0.003 с
|
||||
valkey остановлен: ready 200, вход 201, выход 503
|
||||
valkey снова запущен: повторный выход 204
|
||||
PASSED
|
||||
student-tests/stack/test_faults.py::test_valkey_restart_keeps_cache_and_revocations рестарт valkey: ключей 45 → 45
|
||||
рестарт valkey: живой 200, отозванный 401
|
||||
PASSED
|
||||
student-tests/stack/test_faults.py::test_valkey_flushall_loses_nothing FLUSHALL: живой 200, отозванный 401
|
||||
PASSED
|
||||
student-tests/stack/test_faults.py::test_postgres_stopped_gives_bounded_503_and_recovers postgres остановлен, ready: 503 за 0.004 с
|
||||
postgres остановлен, me: 503 за 0.003 с
|
||||
postgres остановлен, stranger: 503 за 0.002 с
|
||||
postgres остановлен, login: 503 за 0.002 с
|
||||
postgres остановлен, register: 503 за 0.015 с
|
||||
live внутри реплики 200; LB исключил все реплики ещё через 1.16 с
|
||||
live через LB без реплик: 503 за 0.000 с
|
||||
postgres запущен: готовность через 1.6 с, прежний токен 200
|
||||
PASSED
|
||||
student-tests/stack/test_faults.py::test_api_restart_keeps_session_without_extending_it рестарт api: готовность через 2.3 с, токен 200
|
||||
expires_at при входе 2026-10-30T05:32:12.249834Z, в БД после рестарта 2026-10-30T05:32:12.249834Z
|
||||
PASSED
|
||||
student-tests/stack/test_faults.py::test_full_restart_keeps_session_until_logout restart.sh: готовность через 0.0 с, прежний токен 200, выход 204, затем 401
|
||||
PASSED
|
||||
student-tests/stack/test_faults.py::test_session_expires_after_restart_at_original_time срок сессии сокращён до 20 с, рестарт api: готовность через 3.1 с, до срока 200, после срока 401
|
||||
PASSED
|
||||
|
||||
======================== 17 passed in 98.43s (0:01:38) =========================
|
||||
@@ -0,0 +1,43 @@
|
||||
date: 2026-11-06T09:10:22Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ sh scripts/failover-evidence.sh http://127.0.0.1:18084
|
||||
date: 2026-11-06T09:10:22Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
== before: api replicas (hostname = X-Instance-Id), state, StartedAt, FinishedAt
|
||||
d4c8d53d7433 running started=2026-11-06T09:10:18.309662551Z finished=2026-11-06T09:10:12.020503506Z
|
||||
da922ab2ea09 running started=2026-11-06T09:10:18.377623592Z finished=2026-11-06T09:10:12.004100798Z
|
||||
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh
|
||||
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://127.0.0.1:18084"
|
||||
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
|
||||
{"observed_instances_before": 2, "successful_writes": 130, "failed_attempts": 0, "first_recovery_seconds": 2.883}
|
||||
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
== after: api replicas
|
||||
d4c8d53d7433 exited started=2026-11-06T09:10:18.309662551Z finished=2026-11-06T09:10:25.313223596Z
|
||||
da922ab2ea09 running started=2026-11-06T09:10:18.377623592Z finished=2026-11-06T09:10:12.004100798Z
|
||||
== stopped replica: d4c8d53d7433; shutdown: signal, drain window with readiness 503, close, exit
|
||||
{"event":"Handling signal: term","level":"info","ts":"2026-11-06T09:10:22.774795Z"}
|
||||
{"limit":2.0,"event":"service.draining","level":"info","ts":"2026-11-06T09:10:22.776559Z"}
|
||||
{"limit":2.0,"event":"service.draining","level":"info","ts":"2026-11-06T09:10:22.848442Z"}
|
||||
{"method":"GET","route":"/health/ready","status":503,"request_id":"c23539767f7c4ef5ba39f58a446754a2","duration_ms":1.481,"event":"request.handled","level":"info","ts":"2026-11-06T09:10:23.545632Z"}
|
||||
{"method":"GET","route":"/health/ready","status":503,"request_id":"60444a3eba784d839b8caccbaa24d1f2","duration_ms":0.278,"event":"request.handled","level":"info","ts":"2026-11-06T09:10:24.058530Z"}
|
||||
{"method":"GET","route":"/health/ready","status":503,"request_id":"5a247b2f9b304ff789354712b9c2d857","duration_ms":0.282,"event":"request.handled","level":"info","ts":"2026-11-06T09:10:24.545468Z"}
|
||||
{"event":"Shutting down","level":"info","ts":"2026-11-06T09:10:24.779511Z"}
|
||||
{"event":"Shutting down","level":"info","ts":"2026-11-06T09:10:24.852501Z"}
|
||||
{"event":"service.stopped","level":"info","ts":"2026-11-06T09:10:24.884399Z"}
|
||||
{"event":"Finished server process [19]","level":"info","ts":"2026-11-06T09:10:24.884667Z"}
|
||||
{"event":"service.stopped","level":"info","ts":"2026-11-06T09:10:24.959721Z"}
|
||||
{"event":"Finished server process [20]","level":"info","ts":"2026-11-06T09:10:24.959901Z"}
|
||||
{"event":"Worker (pid:19) was sent SIGTERM!","level":"info","ts":"2026-11-06T09:10:24.961675Z"}
|
||||
{"event":"Worker (pid:20) was sent SIGTERM!","level":"info","ts":"2026-11-06T09:10:25.072586Z"}
|
||||
{"event":"Shutting down: Master","level":"info","ts":"2026-11-06T09:10:25.178811Z"}
|
||||
$ scripts/restore.sh
|
||||
Container hlpipe-lab4-forward-final-20261106-api-2 Running
|
||||
Container hlpipe-lab4-forward-final-20261106-api-1 Starting
|
||||
Container hlpipe-lab4-forward-final-20261106-api-1 Started
|
||||
Container hlpipe-lab4-forward-final-20261106-api-1 Waiting
|
||||
Container hlpipe-lab4-forward-final-20261106-api-2 Waiting
|
||||
Container hlpipe-lab4-forward-final-20261106-api-1 Healthy
|
||||
Container hlpipe-lab4-forward-final-20261106-api-2 Healthy
|
||||
== after restore
|
||||
d4c8d53d7433 running started=2026-11-06T09:10:38.256484379Z finished=2026-11-06T09:10:25.313223596Z
|
||||
da922ab2ea09 running started=2026-11-06T09:10:18.377623592Z finished=2026-11-06T09:10:12.004100798Z
|
||||
exit_code: 0 elapsed_seconds: 19.912
|
||||
@@ -0,0 +1,39 @@
|
||||
date: 2026-11-06T09:10:22Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
== before: api replicas (hostname = X-Instance-Id), state, StartedAt, FinishedAt
|
||||
d4c8d53d7433 running started=2026-11-06T09:10:18.309662551Z finished=2026-11-06T09:10:12.020503506Z
|
||||
da922ab2ea09 running started=2026-11-06T09:10:18.377623592Z finished=2026-11-06T09:10:12.004100798Z
|
||||
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh
|
||||
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://127.0.0.1:18084"
|
||||
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
|
||||
{"observed_instances_before": 2, "successful_writes": 130, "failed_attempts": 0, "first_recovery_seconds": 2.883}
|
||||
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
== after: api replicas
|
||||
d4c8d53d7433 exited started=2026-11-06T09:10:18.309662551Z finished=2026-11-06T09:10:25.313223596Z
|
||||
da922ab2ea09 running started=2026-11-06T09:10:18.377623592Z finished=2026-11-06T09:10:12.004100798Z
|
||||
== stopped replica: d4c8d53d7433; shutdown: signal, drain window with readiness 503, close, exit
|
||||
{"event":"Handling signal: term","level":"info","ts":"2026-11-06T09:10:22.774795Z"}
|
||||
{"limit":2.0,"event":"service.draining","level":"info","ts":"2026-11-06T09:10:22.776559Z"}
|
||||
{"limit":2.0,"event":"service.draining","level":"info","ts":"2026-11-06T09:10:22.848442Z"}
|
||||
{"method":"GET","route":"/health/ready","status":503,"request_id":"c23539767f7c4ef5ba39f58a446754a2","duration_ms":1.481,"event":"request.handled","level":"info","ts":"2026-11-06T09:10:23.545632Z"}
|
||||
{"method":"GET","route":"/health/ready","status":503,"request_id":"60444a3eba784d839b8caccbaa24d1f2","duration_ms":0.278,"event":"request.handled","level":"info","ts":"2026-11-06T09:10:24.058530Z"}
|
||||
{"method":"GET","route":"/health/ready","status":503,"request_id":"5a247b2f9b304ff789354712b9c2d857","duration_ms":0.282,"event":"request.handled","level":"info","ts":"2026-11-06T09:10:24.545468Z"}
|
||||
{"event":"Shutting down","level":"info","ts":"2026-11-06T09:10:24.779511Z"}
|
||||
{"event":"Shutting down","level":"info","ts":"2026-11-06T09:10:24.852501Z"}
|
||||
{"event":"service.stopped","level":"info","ts":"2026-11-06T09:10:24.884399Z"}
|
||||
{"event":"Finished server process [19]","level":"info","ts":"2026-11-06T09:10:24.884667Z"}
|
||||
{"event":"service.stopped","level":"info","ts":"2026-11-06T09:10:24.959721Z"}
|
||||
{"event":"Finished server process [20]","level":"info","ts":"2026-11-06T09:10:24.959901Z"}
|
||||
{"event":"Worker (pid:19) was sent SIGTERM!","level":"info","ts":"2026-11-06T09:10:24.961675Z"}
|
||||
{"event":"Worker (pid:20) was sent SIGTERM!","level":"info","ts":"2026-11-06T09:10:25.072586Z"}
|
||||
{"event":"Shutting down: Master","level":"info","ts":"2026-11-06T09:10:25.178811Z"}
|
||||
$ scripts/restore.sh
|
||||
Container hlpipe-lab4-forward-final-20261106-api-2 Running
|
||||
Container hlpipe-lab4-forward-final-20261106-api-1 Starting
|
||||
Container hlpipe-lab4-forward-final-20261106-api-1 Started
|
||||
Container hlpipe-lab4-forward-final-20261106-api-1 Waiting
|
||||
Container hlpipe-lab4-forward-final-20261106-api-2 Waiting
|
||||
Container hlpipe-lab4-forward-final-20261106-api-1 Healthy
|
||||
Container hlpipe-lab4-forward-final-20261106-api-2 Healthy
|
||||
== after restore
|
||||
d4c8d53d7433 running started=2026-11-06T09:10:38.256484379Z finished=2026-11-06T09:10:25.313223596Z
|
||||
da922ab2ea09 running started=2026-11-06T09:10:18.377623592Z finished=2026-11-06T09:10:12.004100798Z
|
||||
@@ -0,0 +1,33 @@
|
||||
date: 2026-11-06T09:13:29Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ python3 scripts/lab4_scenarios.py failover-upload --base-url http://127.0.0.1:18084 --output evidence/forward-final-2026-11-06/failover-upload.json
|
||||
{
|
||||
"ok": true,
|
||||
"date": "2026-11-06T09:13:29Z",
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"scenario": "failover-upload",
|
||||
"upload_instance": "d4c8d53d7433",
|
||||
"download_instance": "da922ab2ea09",
|
||||
"sha256_equal": true,
|
||||
"hook_seconds": 2.734,
|
||||
"download_ok_after_seconds": 2.762,
|
||||
"replicas_at_download": [
|
||||
{
|
||||
"hostname": "d4c8d53d7433",
|
||||
"status": "exited",
|
||||
"exit_code": "0",
|
||||
"started_at": "2026-11-06T09:12:33.47280571Z",
|
||||
"finished_at": "2026-11-06T09:13:32.422358668Z"
|
||||
},
|
||||
{
|
||||
"hostname": "da922ab2ea09",
|
||||
"status": "running",
|
||||
"exit_code": "0",
|
||||
"started_at": "2026-11-06T09:12:33.439519377Z",
|
||||
"finished_at": "2026-11-06T09:12:33.358178252Z"
|
||||
}
|
||||
],
|
||||
"restored": false,
|
||||
"replicas_after_restore": null
|
||||
}
|
||||
exit_code: 0 elapsed_seconds: 3.177
|
||||
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"ok": true,
|
||||
"date": "2026-11-06T09:13:29Z",
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"scenario": "failover-upload",
|
||||
"upload_instance": "d4c8d53d7433",
|
||||
"download_instance": "da922ab2ea09",
|
||||
"sha256_equal": true,
|
||||
"hook_seconds": 2.734,
|
||||
"download_ok_after_seconds": 2.762,
|
||||
"replicas_at_download": [
|
||||
{
|
||||
"hostname": "d4c8d53d7433",
|
||||
"status": "exited",
|
||||
"exit_code": "0",
|
||||
"started_at": "2026-11-06T09:12:33.47280571Z",
|
||||
"finished_at": "2026-11-06T09:13:32.422358668Z"
|
||||
},
|
||||
{
|
||||
"hostname": "da922ab2ea09",
|
||||
"status": "running",
|
||||
"exit_code": "0",
|
||||
"started_at": "2026-11-06T09:12:33.439519377Z",
|
||||
"finished_at": "2026-11-06T09:12:33.358178252Z"
|
||||
}
|
||||
],
|
||||
"restored": false,
|
||||
"replicas_after_restore": null
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
date: 2026-11-06T09:31:35Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {}
|
||||
$ sh -c 'make check && make validate && make lint && openspec validate --all --strict'
|
||||
python3 scripts/check_contract.py
|
||||
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
|
||||
.venv/bin/python scripts/validate_openapi.py
|
||||
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
|
||||
.venv/bin/ruff check src student-tests scripts/capture_evidence.py alembic scripts/scaling.py tools scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py
|
||||
All checks passed!
|
||||
.venv/bin/ruff format --check src student-tests scripts/capture_evidence.py alembic scripts/scaling.py tools scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py
|
||||
163 files already formatted
|
||||
.venv/bin/ty check src student-tests alembic
|
||||
All checks passed!
|
||||
- Validating...
|
||||
✓ change/forward-lab3-final-review-fixes
|
||||
✓ spec/lab1-event-publication
|
||||
✓ spec/lab1-messenger-chats
|
||||
✓ spec/lab1-messenger-history
|
||||
✓ spec/lab1-messenger-identity
|
||||
✓ spec/lab1-service-operability
|
||||
✓ spec/lab2-auth-sessions
|
||||
✓ spec/lab2-dependency-failures
|
||||
✓ spec/lab2-history-index
|
||||
✓ spec/lab2-persistent-storage
|
||||
✓ spec/lab2-session-cache
|
||||
✓ spec/lab3-failover-operations
|
||||
✓ spec/lab3-idempotent-messages
|
||||
✓ spec/lab3-load-balancing
|
||||
✓ spec/lab4-attachments
|
||||
✓ spec/lab4-evidence-runs
|
||||
✓ spec/lab4-object-storage
|
||||
✓ spec/lab4-storage-reconciler
|
||||
✓ spec/lab4-upload-resilience
|
||||
Totals: 19 passed, 0 failed (19 items)
|
||||
exit_code: 0 elapsed_seconds: 1.660
|
||||
@@ -0,0 +1,6 @@
|
||||
date: 2026-11-06T09:18:22Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ make test
|
||||
python3 tests/smoke.py --base-url "http://127.0.0.1:18084"
|
||||
PASS: lab4 public API scenario; infrastructure and grading evidence remain separate.
|
||||
exit_code: 0 elapsed_seconds: 0.435
|
||||
@@ -0,0 +1,7 @@
|
||||
[
|
||||
{
|
||||
"severity": "minor",
|
||||
"file": "src/messenger/messaging/entrypoints/schemas.py:35",
|
||||
"problem": "Непарный Unicode surrogate U+D800/U+DFFF в text принимается схемой, fingerprint вызывает PydanticSerializationError; реальный HTTP возвращает 500 internal_error. Воспроизведено на 75b93bbba5f29575d3fca1d013cc7e0770d88082: evidence/forward-final-2026-11-06/known-minor-http.txt. Сохранено без исправления по правилу minor."
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,49 @@
|
||||
date: 2026-11-06T09:30:40Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {}
|
||||
$ python3 -c 'import collections
|
||||
import json
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
source = Path("/Users/artemkostylev/projects/learning/HL/labs/.pipeline/.runs/labs/snapshots/lab3-ba78cf6da439a4ec60fd59389583b38285b0b2b1")
|
||||
manifest = json.loads(Path("evidence/forward-final-2026-11-06/source-manifest.json").read_text())
|
||||
def source_git(*args):
|
||||
return subprocess.check_output(["git", "-C", str(source), *args], text=True)
|
||||
assert source_git("rev-parse", "HEAD").strip() == manifest["source_to"]
|
||||
changed = dict(line.split("\t", 1)[::-1] for line in source_git("diff", "--name-status", manifest["source_from"], manifest["source_to"]).splitlines())
|
||||
assert changed == {row["file"]: row["status"] for row in manifest["files"]}
|
||||
blobs = {}
|
||||
for line in source_git("ls-tree", "-r", manifest["source_to"]).splitlines():
|
||||
meta, path = line.split("\t", 1)
|
||||
blobs[path] = meta.split()[2]
|
||||
merged = {"infra/haproxy/haproxy.cfg", "src/messenger/messaging/domain/values.py"}
|
||||
exact = 0
|
||||
for row in manifest["files"]:
|
||||
assert blobs[row["file"]] == row["source_blob"], row["file"]
|
||||
if row["action"] == "copied-source-archive" or (row["action"] == "ported" and row["file"] not in merged):
|
||||
assert Path(row["file"]).read_bytes() == (source / row["file"]).read_bytes(), row["file"]
|
||||
exact += 1
|
||||
print(json.dumps({"source_from":manifest["source_from"], "source_to":manifest["source_to"], "manifest_paths":len(changed), "decisions":dict(collections.Counter(row["action"] for row in manifest["files"])), "byte_exact_paths":exact, "manual_lab4_merges":sorted(merged)}, indent=2))
|
||||
subprocess.run(["git", "diff", "--exit-code", manifest["target_base"], "--", "contracts", "tests", "README.md", "COURSE.md"], check=True)
|
||||
subprocess.run(["git", "diff", "--exit-code", "75b93bbba5f29575d3fca1d013cc7e0770d88082", "--", "src", "student-tests", "infra", "scripts", "compose.yaml", "Dockerfile", "pyproject.toml", "uv.lock", "alembic"], check=True)
|
||||
print("PASS: complete immutable-source manifest, preserved normative files, runtime unchanged since measurements")
|
||||
'
|
||||
{
|
||||
"source_from": "f591fa64e525679a31c9db7f942d36c789e1a12f",
|
||||
"source_to": "ba78cf6da439a4ec60fd59389583b38285b0b2b1",
|
||||
"manifest_paths": 214,
|
||||
"decisions": {
|
||||
"adapted-context": 2,
|
||||
"not-copied": 169,
|
||||
"adapted-test-tool": 4,
|
||||
"ported": 14,
|
||||
"copied-source-archive": 19,
|
||||
"merged-via-local-change": 6
|
||||
},
|
||||
"byte_exact_paths": 31,
|
||||
"manual_lab4_merges": [
|
||||
"infra/haproxy/haproxy.cfg",
|
||||
"src/messenger/messaging/domain/values.py"
|
||||
]
|
||||
}
|
||||
PASS: complete immutable-source manifest, preserved normative files, runtime unchanged since measurements
|
||||
exit_code: 0 elapsed_seconds: 0.195
|
||||
@@ -0,0 +1,48 @@
|
||||
date: 2026-11-06T09:14:49Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:hlpipe-lab4-forward-final-20261106"}
|
||||
$ sh -c '
|
||||
set -e
|
||||
fixture_project=hlpipe-lab4-forward-lb-20261106
|
||||
fixture_file=student-tests/tools/lb_errors/compose.yaml
|
||||
trap '"'"'docker compose -p "$fixture_project" -f "$fixture_file" --profile probe down'"'"' EXIT
|
||||
docker compose -p "$fixture_project" -f "$fixture_file" up -d lb api
|
||||
docker compose -p "$fixture_project" -f "$fixture_file" run --rm --no-deps probe 502
|
||||
docker compose -p "$fixture_project" -f "$fixture_file" run --rm --no-deps probe 504
|
||||
docker compose -p "$fixture_project" -f "$fixture_file" stop api
|
||||
docker compose -p "$fixture_project" -f "$fixture_file" run --rm --no-deps probe 503
|
||||
'
|
||||
Network hlpipe-lab4-forward-lb-20261106_default Creating
|
||||
Network hlpipe-lab4-forward-lb-20261106_default Created
|
||||
Container hlpipe-lab4-forward-lb-20261106-lb-1 Creating
|
||||
Container hlpipe-lab4-forward-lb-20261106-api-1 Creating
|
||||
Container hlpipe-lab4-forward-lb-20261106-api-1 Created
|
||||
Container hlpipe-lab4-forward-lb-20261106-lb-1 Created
|
||||
Container hlpipe-lab4-forward-lb-20261106-api-1 Starting
|
||||
Container hlpipe-lab4-forward-lb-20261106-lb-1 Starting
|
||||
Container hlpipe-lab4-forward-lb-20261106-lb-1 Started
|
||||
Container hlpipe-lab4-forward-lb-20261106-api-1 Started
|
||||
{"status": 502, "request_id": "a\"b", "json_valid": true, "echo_matches": true, "seconds": 0.001}
|
||||
{"status": 502, "request_id": "a\\b", "json_valid": true, "echo_matches": true, "seconds": 0.001}
|
||||
{"status": 502, "request_id": "a\tb", "json_valid": true, "echo_matches": true, "seconds": 0.001}
|
||||
{"status": 502, "request_id": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", "json_valid": true, "echo_matches": true, "seconds": 0.001}
|
||||
{"status": 504, "request_id": "a\"b", "json_valid": true, "echo_matches": true, "seconds": 10.024}
|
||||
{"status": 504, "request_id": "a\\b", "json_valid": true, "echo_matches": true, "seconds": 10.005}
|
||||
{"status": 504, "request_id": "a\tb", "json_valid": true, "echo_matches": true, "seconds": 10.022}
|
||||
{"status": 504, "request_id": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", "json_valid": true, "echo_matches": true, "seconds": 10.007}
|
||||
Container hlpipe-lab4-forward-lb-20261106-api-1 Stopping
|
||||
Container hlpipe-lab4-forward-lb-20261106-api-1 Stopped
|
||||
{"status": 503, "request_id": "a\"b", "json_valid": true, "echo_matches": true, "seconds": 0.004}
|
||||
{"status": 503, "request_id": "a\\b", "json_valid": true, "echo_matches": true, "seconds": 0.0}
|
||||
{"status": 503, "request_id": "a\tb", "json_valid": true, "echo_matches": true, "seconds": 0.0}
|
||||
{"status": 503, "request_id": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", "json_valid": true, "echo_matches": true, "seconds": 0.0}
|
||||
Container hlpipe-lab4-forward-lb-20261106-api-1 Stopping
|
||||
Container hlpipe-lab4-forward-lb-20261106-lb-1 Stopping
|
||||
Container hlpipe-lab4-forward-lb-20261106-api-1 Stopped
|
||||
Container hlpipe-lab4-forward-lb-20261106-api-1 Removing
|
||||
Container hlpipe-lab4-forward-lb-20261106-api-1 Removed
|
||||
Container hlpipe-lab4-forward-lb-20261106-lb-1 Stopped
|
||||
Container hlpipe-lab4-forward-lb-20261106-lb-1 Removing
|
||||
Container hlpipe-lab4-forward-lb-20261106-lb-1 Removed
|
||||
Network hlpipe-lab4-forward-lb-20261106_default Removing
|
||||
Network hlpipe-lab4-forward-lb-20261106_default Removed
|
||||
exit_code: 0 elapsed_seconds: 52.831
|
||||
@@ -0,0 +1,24 @@
|
||||
date: 2026-11-06T09:15:37Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {}
|
||||
$ python3 -c 'import base64,http.client,json,uuid
|
||||
conn=http.client.HTTPConnection("127.0.0.1",18084,timeout=10)
|
||||
name="surrogate_"+uuid.uuid4().hex[:12]
|
||||
password=uuid.uuid4().hex
|
||||
def request(method,path,body=None,headers=None):
|
||||
conn.request(method,path,None if body is None else json.dumps(body).encode(),{"Content-Type":"application/json",**(headers or {})})
|
||||
response=conn.getresponse()
|
||||
return response.status,json.loads(response.read() or "null")
|
||||
status,user=request("POST","/api/v1/users",{"username":name,"password":password,"display_name":"Synthetic"})
|
||||
assert status==201
|
||||
status,session=request("POST","/api/v1/auth/sessions",headers={"Authorization":"Basic "+base64.b64encode((name+":"+password).encode()).decode()})
|
||||
assert status==201
|
||||
headers={"Authorization":"Bearer "+session["token"]}
|
||||
status,chat=request("POST","/api/v1/chats",{"title":"Synthetic","member_ids":[user["id"]]},headers)
|
||||
assert status==201
|
||||
for value in [chr(0xd800),chr(0xdfff)]:
|
||||
status,data=request("POST","/api/v1/chats/"+chat["id"]+"/messages",{"text":value},{**headers,"Idempotency-Key":uuid.uuid4().hex})
|
||||
print(json.dumps({"codepoint":hex(ord(value)),"status":status,"error_code":data.get("error",{}).get("code")}))
|
||||
conn.close()'
|
||||
{"codepoint": "0xd800", "status": 500, "error_code": "internal_error"}
|
||||
{"codepoint": "0xdfff", "status": 500, "error_code": "internal_error"}
|
||||
exit_code: 0 elapsed_seconds: 0.114
|
||||
@@ -0,0 +1,15 @@
|
||||
date: 2026-11-06T09:09:38Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {}
|
||||
$ .venv/bin/python -c 'import json
|
||||
from messenger.messaging.entrypoints.schemas import CreateMessageRequest
|
||||
for value in [chr(0xd800), chr(0xdfff)]:
|
||||
body = CreateMessageRequest(text=value)
|
||||
try:
|
||||
body.fingerprint()
|
||||
result = "accepted"
|
||||
except Exception as error:
|
||||
result = type(error).__name__
|
||||
print(json.dumps({"codepoint": hex(ord(value)), "schema_accepted": True, "fingerprint_result": result}))'
|
||||
{"codepoint": "0xd800", "schema_accepted": true, "fingerprint_result": "PydanticSerializationError"}
|
||||
{"codepoint": "0xdfff", "schema_accepted": true, "fingerprint_result": "PydanticSerializationError"}
|
||||
exit_code: 0 elapsed_seconds: 0.124
|
||||
@@ -0,0 +1,29 @@
|
||||
date: 2026-11-06T09:13:49Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ python3 scripts/limits.py
|
||||
date: 2026-11-06T09:13:49Z commit: 75b93bb
|
||||
$ docker compose --profile bench --profile tools --profile reconcile config --format json (declared)
|
||||
service cpus/mem ports secrets volumes networks ro user
|
||||
---------- ---------- ------------------------------- ------------------------------------------------------------------------------------------------------------------------------- ---------- ---------------------- ----- -----------
|
||||
api 1/512MiB - cache_url,cursor_secret,database_url,s3_access_key_id,s3_secret_access_key - backend(internal) true image
|
||||
k6 2/512MiB - - - edge false image
|
||||
lb 1/128MiB 127.0.0.1:18084,127.0.0.1:24844 - - backend(internal),edge true image
|
||||
migrate 0.5/256MiB - database_url - backend(internal) false image
|
||||
postgres 2/1024MiB - postgres_password pgdata backend(internal) false 70:70
|
||||
reconciler 1/256MiB - database_url,s3_reconciler_access_key_id,s3_reconciler_secret_access_key - backend(internal) true image
|
||||
s3 1/512MiB - s3_root_password,s3_root_user s3data backend(internal) true image
|
||||
s3-cli 0.5/64MiB - s3_access_key_id,s3_reconciler_access_key_id,s3_reconciler_secret_access_key,s3_root_password,s3_root_user,s3_secret_access_key - backend(internal) true 65534:65534
|
||||
s3-init 0.5/64MiB - s3_access_key_id,s3_reconciler_access_key_id,s3_reconciler_secret_access_key,s3_root_password,s3_root_user,s3_secret_access_key - backend(internal) true 65534:65534
|
||||
valkey 0.5/128MiB - valkey.conf valkeydata backend(internal) false 999:1000
|
||||
|
||||
$ docker inspect (running containers: hostname = X-Instance-Id, applied limits)
|
||||
service hostname cpus memory published
|
||||
-------- ------------ ---- ------- ---------------------------------------------------
|
||||
api da922ab2ea09 1 512MiB
|
||||
lb ea6930f64089 1 128MiB 127.0.0.1:18084->8080/tcp,127.0.0.1:24844->8404/tcp
|
||||
postgres 3f3a870d89c0 2 1024MiB
|
||||
s3 5921e788f03d 1 512MiB
|
||||
valkey 07ea16e2fcfe 0.5 128MiB
|
||||
|
||||
PASS: limits, ports, secrets, volumes and networks match the policy
|
||||
exit_code: 0 elapsed_seconds: 0.308
|
||||
@@ -0,0 +1,116 @@
|
||||
date: 2026-11-06T09:16:16Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ make load-evidence
|
||||
sh scripts/load-evidence.sh
|
||||
date: 2026-11-06T09:16:17Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082 network: hlpipe-lab4-forward-final-20261106_edge
|
||||
# make load-evidence (scripts/load-evidence.sh): закрытый контур tests/load.py, 64 потока, 8 с, все записи
|
||||
# прогона в один новый чат, 3 повтора с паузой 15 с; клиент — одноразовый контейнер в сети <проект>_edge
|
||||
# стенда (2 CPU, 512 MiB, базовый образ API по digest, tests/, contracts/ и lab.json только для чтения,
|
||||
# tcp_tw_reuse=1), адрес http://lb:8080. На 4-й секунде — docker stats; после прогона — request.handled реплик.
|
||||
$ docker ps --format '{{.Names}}' | grep -v '^hlpipe-lab4-forward-final-20261106-' # чужие контейнеры на VM во время серии
|
||||
$ client() { docker run --rm --name hlpipe-lab4-forward-final-20261106-load-client --network hlpipe-lab4-forward-final-20261106_edge --cpus 2 --memory 512m --sysctl net.ipv4.tcp_tw_reuse=1 -v "$PWD/tests:/w/tests:ro" -v "$PWD/contracts:/w/contracts:ro" -v "$PWD/lab.json:/w/lab.json:ro" -v "$PWD/evidence/forward-final-2026-11-06/load:/out" python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d python /w/tests/load.py "$@"; }
|
||||
$ client --base-url http://lb:8080 --duration 8 --concurrency 64 --output /out/run1.json
|
||||
exit 0
|
||||
{
|
||||
"lab": 4,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 8.0,
|
||||
"elapsed_seconds": 8.238,
|
||||
"concurrency": 64,
|
||||
"successful_requests": 11768,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 1428.577,
|
||||
"successful_latency_ms": {
|
||||
"p50": 40.163,
|
||||
"p95": 100.062,
|
||||
"p99": 141.558
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
$ docker stats --no-stream # 4-я секунда прогона: контейнер CPU% память
|
||||
api-1 93.26% 271.2MiB / 512MiB
|
||||
api-2 89.65% 307.4MiB / 512MiB
|
||||
lb-1 46.78% 30.14MiB / 128MiB
|
||||
load-client 60.53% 83.66MiB / 512MiB
|
||||
postgres-1 157.07% 151.1MiB / 1GiB
|
||||
s3-1 0.76% 69.52MiB / 512MiB
|
||||
valkey-1 3.81% 13.9MiB / 128MiB
|
||||
$ docker compose logs --since 2026-11-06T09:16:17Z api | request.handled: статус метод маршрут число
|
||||
200 GET /health/live 1
|
||||
200 GET /health/ready 22
|
||||
201 POST /api/v1/auth/sessions 3
|
||||
201 POST /api/v1/chats 1
|
||||
201 POST /api/v1/chats/{chat_id}/messages 11768
|
||||
201 POST /api/v1/users 3
|
||||
$ client --base-url http://lb:8080 --duration 8 --concurrency 64 --output /out/run2.json
|
||||
exit 0
|
||||
{
|
||||
"lab": 4,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 8.0,
|
||||
"elapsed_seconds": 8.229,
|
||||
"concurrency": 64,
|
||||
"successful_requests": 11984,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 1456.338,
|
||||
"successful_latency_ms": {
|
||||
"p50": 37.442,
|
||||
"p95": 98.948,
|
||||
"p99": 134.678
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
$ docker stats --no-stream # 4-я секунда прогона: контейнер CPU% память
|
||||
api-1 90.89% 272.5MiB / 512MiB
|
||||
api-2 75.30% 310.1MiB / 512MiB
|
||||
lb-1 39.94% 30.58MiB / 128MiB
|
||||
load-client 46.50% 83.65MiB / 512MiB
|
||||
postgres-1 121.64% 158.2MiB / 1GiB
|
||||
s3-1 0.77% 69.37MiB / 512MiB
|
||||
valkey-1 3.02% 13.87MiB / 128MiB
|
||||
$ docker compose logs --since 2026-11-06T09:16:41Z api | request.handled: статус метод маршрут число
|
||||
200 GET /health/live 1
|
||||
200 GET /health/ready 23
|
||||
201 POST /api/v1/auth/sessions 3
|
||||
201 POST /api/v1/chats 1
|
||||
201 POST /api/v1/chats/{chat_id}/messages 11984
|
||||
201 POST /api/v1/users 3
|
||||
$ client --base-url http://lb:8080 --duration 8 --concurrency 64 --output /out/run3.json
|
||||
exit 0
|
||||
{
|
||||
"lab": 4,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 8.0,
|
||||
"elapsed_seconds": 8.21,
|
||||
"concurrency": 64,
|
||||
"successful_requests": 12555,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 1529.32,
|
||||
"successful_latency_ms": {
|
||||
"p50": 33.478,
|
||||
"p95": 98.475,
|
||||
"p99": 138.028
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
$ docker stats --no-stream # 4-я секунда прогона: контейнер CPU% память
|
||||
api-1 100.60% 273.6MiB / 512MiB
|
||||
api-2 99.12% 310.2MiB / 512MiB
|
||||
lb-1 58.14% 30.39MiB / 128MiB
|
||||
load-client 66.66% 83.66MiB / 512MiB
|
||||
postgres-1 168.89% 167.2MiB / 1GiB
|
||||
s3-1 0.80% 71.19MiB / 512MiB
|
||||
valkey-1 4.31% 13.5MiB / 128MiB
|
||||
$ docker compose logs --since 2026-11-06T09:17:05Z api | request.handled: статус метод маршрут число
|
||||
200 GET /health/live 1
|
||||
200 GET /health/ready 24
|
||||
201 POST /api/v1/auth/sessions 3
|
||||
201 POST /api/v1/chats 1
|
||||
201 POST /api/v1/chats/{chat_id}/messages 12555
|
||||
201 POST /api/v1/users 3
|
||||
$ docker compose exec -T postgres psql -c 'SELECT count(*) FROM messages'
|
||||
36448
|
||||
exit_code: 0 elapsed_seconds: 57.764
|
||||
@@ -0,0 +1,111 @@
|
||||
date: 2026-11-06T09:16:17Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082 network: hlpipe-lab4-forward-final-20261106_edge
|
||||
# make load-evidence (scripts/load-evidence.sh): закрытый контур tests/load.py, 64 потока, 8 с, все записи
|
||||
# прогона в один новый чат, 3 повтора с паузой 15 с; клиент — одноразовый контейнер в сети <проект>_edge
|
||||
# стенда (2 CPU, 512 MiB, базовый образ API по digest, tests/, contracts/ и lab.json только для чтения,
|
||||
# tcp_tw_reuse=1), адрес http://lb:8080. На 4-й секунде — docker stats; после прогона — request.handled реплик.
|
||||
$ docker ps --format '{{.Names}}' | grep -v '^hlpipe-lab4-forward-final-20261106-' # чужие контейнеры на VM во время серии
|
||||
$ client() { docker run --rm --name hlpipe-lab4-forward-final-20261106-load-client --network hlpipe-lab4-forward-final-20261106_edge --cpus 2 --memory 512m --sysctl net.ipv4.tcp_tw_reuse=1 -v "$PWD/tests:/w/tests:ro" -v "$PWD/contracts:/w/contracts:ro" -v "$PWD/lab.json:/w/lab.json:ro" -v "$PWD/evidence/forward-final-2026-11-06/load:/out" python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d python /w/tests/load.py "$@"; }
|
||||
$ client --base-url http://lb:8080 --duration 8 --concurrency 64 --output /out/run1.json
|
||||
exit 0
|
||||
{
|
||||
"lab": 4,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 8.0,
|
||||
"elapsed_seconds": 8.238,
|
||||
"concurrency": 64,
|
||||
"successful_requests": 11768,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 1428.577,
|
||||
"successful_latency_ms": {
|
||||
"p50": 40.163,
|
||||
"p95": 100.062,
|
||||
"p99": 141.558
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
$ docker stats --no-stream # 4-я секунда прогона: контейнер CPU% память
|
||||
api-1 93.26% 271.2MiB / 512MiB
|
||||
api-2 89.65% 307.4MiB / 512MiB
|
||||
lb-1 46.78% 30.14MiB / 128MiB
|
||||
load-client 60.53% 83.66MiB / 512MiB
|
||||
postgres-1 157.07% 151.1MiB / 1GiB
|
||||
s3-1 0.76% 69.52MiB / 512MiB
|
||||
valkey-1 3.81% 13.9MiB / 128MiB
|
||||
$ docker compose logs --since 2026-11-06T09:16:17Z api | request.handled: статус метод маршрут число
|
||||
200 GET /health/live 1
|
||||
200 GET /health/ready 22
|
||||
201 POST /api/v1/auth/sessions 3
|
||||
201 POST /api/v1/chats 1
|
||||
201 POST /api/v1/chats/{chat_id}/messages 11768
|
||||
201 POST /api/v1/users 3
|
||||
$ client --base-url http://lb:8080 --duration 8 --concurrency 64 --output /out/run2.json
|
||||
exit 0
|
||||
{
|
||||
"lab": 4,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 8.0,
|
||||
"elapsed_seconds": 8.229,
|
||||
"concurrency": 64,
|
||||
"successful_requests": 11984,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 1456.338,
|
||||
"successful_latency_ms": {
|
||||
"p50": 37.442,
|
||||
"p95": 98.948,
|
||||
"p99": 134.678
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
$ docker stats --no-stream # 4-я секунда прогона: контейнер CPU% память
|
||||
api-1 90.89% 272.5MiB / 512MiB
|
||||
api-2 75.30% 310.1MiB / 512MiB
|
||||
lb-1 39.94% 30.58MiB / 128MiB
|
||||
load-client 46.50% 83.65MiB / 512MiB
|
||||
postgres-1 121.64% 158.2MiB / 1GiB
|
||||
s3-1 0.77% 69.37MiB / 512MiB
|
||||
valkey-1 3.02% 13.87MiB / 128MiB
|
||||
$ docker compose logs --since 2026-11-06T09:16:41Z api | request.handled: статус метод маршрут число
|
||||
200 GET /health/live 1
|
||||
200 GET /health/ready 23
|
||||
201 POST /api/v1/auth/sessions 3
|
||||
201 POST /api/v1/chats 1
|
||||
201 POST /api/v1/chats/{chat_id}/messages 11984
|
||||
201 POST /api/v1/users 3
|
||||
$ client --base-url http://lb:8080 --duration 8 --concurrency 64 --output /out/run3.json
|
||||
exit 0
|
||||
{
|
||||
"lab": 4,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 8.0,
|
||||
"elapsed_seconds": 8.21,
|
||||
"concurrency": 64,
|
||||
"successful_requests": 12555,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 1529.32,
|
||||
"successful_latency_ms": {
|
||||
"p50": 33.478,
|
||||
"p95": 98.475,
|
||||
"p99": 138.028
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
$ docker stats --no-stream # 4-я секунда прогона: контейнер CPU% память
|
||||
api-1 100.60% 273.6MiB / 512MiB
|
||||
api-2 99.12% 310.2MiB / 512MiB
|
||||
lb-1 58.14% 30.39MiB / 128MiB
|
||||
load-client 66.66% 83.66MiB / 512MiB
|
||||
postgres-1 168.89% 167.2MiB / 1GiB
|
||||
s3-1 0.80% 71.19MiB / 512MiB
|
||||
valkey-1 4.31% 13.5MiB / 128MiB
|
||||
$ docker compose logs --since 2026-11-06T09:17:05Z api | request.handled: статус метод маршрут число
|
||||
200 GET /health/live 1
|
||||
200 GET /health/ready 24
|
||||
201 POST /api/v1/auth/sessions 3
|
||||
201 POST /api/v1/chats 1
|
||||
201 POST /api/v1/chats/{chat_id}/messages 12555
|
||||
201 POST /api/v1/users 3
|
||||
$ docker compose exec -T postgres psql -c 'SELECT count(*) FROM messages'
|
||||
36448
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"lab": 4,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 8.0,
|
||||
"elapsed_seconds": 8.238,
|
||||
"concurrency": 64,
|
||||
"successful_requests": 11768,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 1428.577,
|
||||
"successful_latency_ms": {
|
||||
"p50": 40.163,
|
||||
"p95": 100.062,
|
||||
"p99": 141.558
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"lab": 4,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 8.0,
|
||||
"elapsed_seconds": 8.229,
|
||||
"concurrency": 64,
|
||||
"successful_requests": 11984,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 1456.338,
|
||||
"successful_latency_ms": {
|
||||
"p50": 37.442,
|
||||
"p95": 98.948,
|
||||
"p99": 134.678
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"lab": 4,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 8.0,
|
||||
"elapsed_seconds": 8.21,
|
||||
"concurrency": 64,
|
||||
"successful_requests": 12555,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 1529.32,
|
||||
"successful_latency_ms": {
|
||||
"p50": 33.478,
|
||||
"p95": 98.475,
|
||||
"p99": 138.028
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
date: 2026-11-06T09:18:23Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ make down
|
||||
docker compose --profile bench --profile tools --profile reconcile down
|
||||
Container hlpipe-lab4-forward-final-20261106-api-2 Stopping
|
||||
Container hlpipe-lab4-forward-final-20261106-lb-1 Stopping
|
||||
Container hlpipe-lab4-forward-final-20261106-api-1 Stopping
|
||||
Container hlpipe-lab4-forward-final-20261106-lb-1 Stopped
|
||||
Container hlpipe-lab4-forward-final-20261106-lb-1 Removing
|
||||
Container hlpipe-lab4-forward-final-20261106-lb-1 Removed
|
||||
Container hlpipe-lab4-forward-final-20261106-api-1 Stopped
|
||||
Container hlpipe-lab4-forward-final-20261106-api-1 Removing
|
||||
Container hlpipe-lab4-forward-final-20261106-api-1 Removed
|
||||
Container hlpipe-lab4-forward-final-20261106-api-2 Stopped
|
||||
Container hlpipe-lab4-forward-final-20261106-api-2 Removing
|
||||
Container hlpipe-lab4-forward-final-20261106-api-2 Removed
|
||||
Container hlpipe-lab4-forward-final-20261106-s3-init-1 Stopping
|
||||
Container hlpipe-lab4-forward-final-20261106-migrate-1 Stopping
|
||||
Container hlpipe-lab4-forward-final-20261106-valkey-1 Stopping
|
||||
Container hlpipe-lab4-forward-final-20261106-s3-init-1 Stopped
|
||||
Container hlpipe-lab4-forward-final-20261106-s3-init-1 Removing
|
||||
Container hlpipe-lab4-forward-final-20261106-migrate-1 Stopped
|
||||
Container hlpipe-lab4-forward-final-20261106-migrate-1 Removing
|
||||
Container hlpipe-lab4-forward-final-20261106-s3-init-1 Removed
|
||||
Container hlpipe-lab4-forward-final-20261106-s3-1 Stopping
|
||||
Container hlpipe-lab4-forward-final-20261106-migrate-1 Removed
|
||||
Container hlpipe-lab4-forward-final-20261106-postgres-1 Stopping
|
||||
Container hlpipe-lab4-forward-final-20261106-valkey-1 Stopped
|
||||
Container hlpipe-lab4-forward-final-20261106-valkey-1 Removing
|
||||
Container hlpipe-lab4-forward-final-20261106-valkey-1 Removed
|
||||
Container hlpipe-lab4-forward-final-20261106-s3-1 Stopped
|
||||
Container hlpipe-lab4-forward-final-20261106-s3-1 Removing
|
||||
Container hlpipe-lab4-forward-final-20261106-postgres-1 Stopped
|
||||
Container hlpipe-lab4-forward-final-20261106-postgres-1 Removing
|
||||
Container hlpipe-lab4-forward-final-20261106-s3-1 Removed
|
||||
Container hlpipe-lab4-forward-final-20261106-postgres-1 Removed
|
||||
Network hlpipe-lab4-forward-final-20261106_edge Removing
|
||||
Network hlpipe-lab4-forward-final-20261106_backend Removing
|
||||
Network hlpipe-lab4-forward-final-20261106_edge Removed
|
||||
Network hlpipe-lab4-forward-final-20261106_backend Removed
|
||||
exit_code: 0 elapsed_seconds: 3.283
|
||||
@@ -0,0 +1,72 @@
|
||||
{
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"load_rps": {
|
||||
"all": [
|
||||
1428.577,
|
||||
1456.338,
|
||||
1529.32
|
||||
],
|
||||
"median": 1456.338,
|
||||
"min": 1428.577,
|
||||
"max": 1529.32
|
||||
},
|
||||
"load_p95_ms": {
|
||||
"all": [
|
||||
100.062,
|
||||
98.948,
|
||||
98.475
|
||||
],
|
||||
"median": 98.948,
|
||||
"min": 98.475,
|
||||
"max": 100.062
|
||||
},
|
||||
"load_failed": 0,
|
||||
"memory_statuses": [
|
||||
{
|
||||
"201": 80
|
||||
},
|
||||
{
|
||||
"201": 80
|
||||
},
|
||||
{
|
||||
"201": 78,
|
||||
"503": 2
|
||||
}
|
||||
],
|
||||
"body_bytes": [
|
||||
10485256,
|
||||
10485256,
|
||||
10485256
|
||||
],
|
||||
"api_peak_mib": {
|
||||
"all": [
|
||||
456.2,
|
||||
438.1,
|
||||
450.4
|
||||
],
|
||||
"median": 450.4,
|
||||
"min": 438.1,
|
||||
"max": 456.2
|
||||
},
|
||||
"lb_peak_mib": {
|
||||
"all": [
|
||||
31.8,
|
||||
28.5,
|
||||
29.1
|
||||
],
|
||||
"median": 29.1,
|
||||
"min": 28.5,
|
||||
"max": 31.8
|
||||
},
|
||||
"upload_p95_ms": {
|
||||
"all": [
|
||||
3641.3,
|
||||
4755.0,
|
||||
8091.8
|
||||
],
|
||||
"median": 4755.0,
|
||||
"min": 3641.3,
|
||||
"max": 8091.8
|
||||
},
|
||||
"oom": 0
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
date: 2026-11-06T09:21:03Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {}
|
||||
$ python3 -c 'import json,statistics,subprocess
|
||||
from pathlib import Path
|
||||
root=Path("evidence/forward-final-2026-11-06")
|
||||
loads=[json.loads(p.read_text()) for p in sorted((root/"load").glob("run*.json"))]
|
||||
memory=[json.loads(p.read_text()) for p in sorted((root/"memory").glob("run*/slots*.json"))]
|
||||
assert len(loads)==len(memory)==3
|
||||
assert all(r["failed_requests"]==0 for r in loads)
|
||||
assert all(r["count"]==80 and r["concurrency"]==40 and r["upload_concurrency"]==3 for r in memory)
|
||||
def stats(values):return {"all":values,"median":statistics.median(values),"min":min(values),"max":max(values)}
|
||||
result={"commit":subprocess.check_output(["git","rev-parse","HEAD"],text=True).strip(),"load_rps":stats([r["successful_rps"] for r in loads]),"load_p95_ms":stats([r["successful_latency_ms"]["p95"] for r in loads]),"load_failed":sum(r["failed_requests"] for r in loads),"memory_statuses":[r["status_codes"] for r in memory],"body_bytes":[r["body_bytes"] for r in memory],"api_peak_mib":stats([max(v["cgroup_memory_peak_mib"] for v in r["memory"].values()) for r in memory]),"lb_peak_mib":stats([r["lb"]["cgroup_memory_peak_mib"] for r in memory]),"upload_p95_ms":stats([r["latency_ms"]["p95"] for r in memory]),"oom":sum(sum(v["oom_kill"] for v in r["memory"].values())+r["lb"]["oom_kill"] for r in memory)}
|
||||
with (root/"measurement-summary.json").open("x") as stream:json.dump(result,stream,indent=2);stream.write("\n")
|
||||
print(json.dumps(result,indent=2))'
|
||||
{
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"load_rps": {
|
||||
"all": [
|
||||
1428.577,
|
||||
1456.338,
|
||||
1529.32
|
||||
],
|
||||
"median": 1456.338,
|
||||
"min": 1428.577,
|
||||
"max": 1529.32
|
||||
},
|
||||
"load_p95_ms": {
|
||||
"all": [
|
||||
100.062,
|
||||
98.948,
|
||||
98.475
|
||||
],
|
||||
"median": 98.948,
|
||||
"min": 98.475,
|
||||
"max": 100.062
|
||||
},
|
||||
"load_failed": 0,
|
||||
"memory_statuses": [
|
||||
{
|
||||
"201": 80
|
||||
},
|
||||
{
|
||||
"201": 80
|
||||
},
|
||||
{
|
||||
"201": 78,
|
||||
"503": 2
|
||||
}
|
||||
],
|
||||
"body_bytes": [
|
||||
10485256,
|
||||
10485256,
|
||||
10485256
|
||||
],
|
||||
"api_peak_mib": {
|
||||
"all": [
|
||||
456.2,
|
||||
438.1,
|
||||
450.4
|
||||
],
|
||||
"median": 450.4,
|
||||
"min": 438.1,
|
||||
"max": 456.2
|
||||
},
|
||||
"lb_peak_mib": {
|
||||
"all": [
|
||||
31.8,
|
||||
28.5,
|
||||
29.1
|
||||
],
|
||||
"median": 29.1,
|
||||
"min": 28.5,
|
||||
"max": 31.8
|
||||
},
|
||||
"upload_p95_ms": {
|
||||
"all": [
|
||||
3641.3,
|
||||
4755.0,
|
||||
8091.8
|
||||
],
|
||||
"median": 4755.0,
|
||||
"min": 3641.3,
|
||||
"max": 8091.8
|
||||
},
|
||||
"oom": 0
|
||||
}
|
||||
exit_code: 0 elapsed_seconds: 0.048
|
||||
@@ -0,0 +1,7 @@
|
||||
date: 2026-11-06T09:17:14Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ python3 scripts/upload_memory.py --base-url http://127.0.0.1:18084 --slots 3 --sizes 10485760 --levels 40 --out evidence/forward-final-2026-11-06/memory/run1
|
||||
# 2026-11-06T09:17:14Z commit 75b93bbba5f29575d3fca1d013cc7e0770d88082; api x2: 1 CPU, 512 MiB, 2 workers; MALLOC_MMAP_THRESHOLD_=131072; 2 uploads per client; peaks from cgroup memory.peak, MiB
|
||||
slots size conc codes p50_ms p95_ms api_idle api_peak api_oom lb_peak lb_oom
|
||||
3 10.0M 40 201x80 2587.5 3641.3 233.2 456.2 0 31.8 0
|
||||
exit_code: 0 elapsed_seconds: 20.084
|
||||
@@ -0,0 +1,7 @@
|
||||
date: 2026-11-06T09:17:34Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ python3 scripts/upload_memory.py --base-url http://127.0.0.1:18084 --slots 3 --sizes 10485760 --levels 40 --out evidence/forward-final-2026-11-06/memory/run2
|
||||
# 2026-11-06T09:17:34Z commit 75b93bbba5f29575d3fca1d013cc7e0770d88082; api x2: 1 CPU, 512 MiB, 2 workers; MALLOC_MMAP_THRESHOLD_=131072; 2 uploads per client; peaks from cgroup memory.peak, MiB
|
||||
slots size conc codes p50_ms p95_ms api_idle api_peak api_oom lb_peak lb_oom
|
||||
3 10.0M 40 201x80 2770.4 4755.0 231.3 438.1 0 28.5 0
|
||||
exit_code: 0 elapsed_seconds: 21.500
|
||||
@@ -0,0 +1,7 @@
|
||||
date: 2026-11-06T09:17:56Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ python3 scripts/upload_memory.py --base-url http://127.0.0.1:18084 --slots 3 --sizes 10485760 --levels 40 --out evidence/forward-final-2026-11-06/memory/run3
|
||||
# 2026-11-06T09:17:56Z commit 75b93bbba5f29575d3fca1d013cc7e0770d88082; api x2: 1 CPU, 512 MiB, 2 workers; MALLOC_MMAP_THRESHOLD_=131072; 2 uploads per client; peaks from cgroup memory.peak, MiB
|
||||
slots size conc codes p50_ms p95_ms api_idle api_peak api_oom lb_peak lb_oom
|
||||
3 10.0M 40 201x78,503x2 3560.9 8091.8 231.7 450.4 0 29.1 0
|
||||
exit_code: 0 elapsed_seconds: 23.514
|
||||
@@ -0,0 +1,40 @@
|
||||
{
|
||||
"date": "2026-11-06T09:17:21Z",
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"upload_concurrency": 3,
|
||||
"mode": "http",
|
||||
"count": 80,
|
||||
"concurrency": 40,
|
||||
"body_bytes": 10485256,
|
||||
"status_codes": {
|
||||
"201": 80
|
||||
},
|
||||
"instances": 2,
|
||||
"linked_in_messages": 0,
|
||||
"elapsed_seconds": 6.143,
|
||||
"successful_rps": 13.02,
|
||||
"latency_ms": {
|
||||
"p50": 2587.5,
|
||||
"p95": 3641.3,
|
||||
"p99": 4322.0,
|
||||
"max": 4322.0
|
||||
},
|
||||
"memory": {
|
||||
"35b198e7b5ca": {
|
||||
"idle_mib": 230.7,
|
||||
"docker_stats_peak_mib": 409.6,
|
||||
"cgroup_memory_peak_mib": 424.4,
|
||||
"oom_kill": 0
|
||||
},
|
||||
"93b12ae1c127": {
|
||||
"idle_mib": 233.2,
|
||||
"docker_stats_peak_mib": 419.5,
|
||||
"cgroup_memory_peak_mib": 456.2,
|
||||
"oom_kill": 0
|
||||
}
|
||||
},
|
||||
"lb": {
|
||||
"cgroup_memory_peak_mib": 31.8,
|
||||
"oom_kill": 0
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
# 2026-11-06T09:17:14Z commit 75b93bbba5f29575d3fca1d013cc7e0770d88082; api x2: 1 CPU, 512 MiB, 2 workers; MALLOC_MMAP_THRESHOLD_=131072; 2 uploads per client; peaks from cgroup memory.peak, MiB
|
||||
slots size conc codes p50_ms p95_ms api_idle api_peak api_oom lb_peak lb_oom
|
||||
3 10.0M 40 201x80 2587.5 3641.3 233.2 456.2 0 31.8 0
|
||||
@@ -0,0 +1,40 @@
|
||||
{
|
||||
"date": "2026-11-06T09:17:40Z",
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"upload_concurrency": 3,
|
||||
"mode": "http",
|
||||
"count": 80,
|
||||
"concurrency": 40,
|
||||
"body_bytes": 10485256,
|
||||
"status_codes": {
|
||||
"201": 80
|
||||
},
|
||||
"instances": 2,
|
||||
"linked_in_messages": 0,
|
||||
"elapsed_seconds": 8.053,
|
||||
"successful_rps": 9.93,
|
||||
"latency_ms": {
|
||||
"p50": 2770.4,
|
||||
"p95": 4755.0,
|
||||
"p99": 5507.3,
|
||||
"max": 5507.3
|
||||
},
|
||||
"memory": {
|
||||
"5df14feadc75": {
|
||||
"idle_mib": 230.1,
|
||||
"docker_stats_peak_mib": 411.6,
|
||||
"cgroup_memory_peak_mib": 438.1,
|
||||
"oom_kill": 0
|
||||
},
|
||||
"806f59cf0d4c": {
|
||||
"idle_mib": 231.3,
|
||||
"docker_stats_peak_mib": 401.0,
|
||||
"cgroup_memory_peak_mib": 425.7,
|
||||
"oom_kill": 0
|
||||
}
|
||||
},
|
||||
"lb": {
|
||||
"cgroup_memory_peak_mib": 28.5,
|
||||
"oom_kill": 0
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
# 2026-11-06T09:17:34Z commit 75b93bbba5f29575d3fca1d013cc7e0770d88082; api x2: 1 CPU, 512 MiB, 2 workers; MALLOC_MMAP_THRESHOLD_=131072; 2 uploads per client; peaks from cgroup memory.peak, MiB
|
||||
slots size conc codes p50_ms p95_ms api_idle api_peak api_oom lb_peak lb_oom
|
||||
3 10.0M 40 201x80 2770.4 4755.0 231.3 438.1 0 28.5 0
|
||||
@@ -0,0 +1,41 @@
|
||||
{
|
||||
"date": "2026-11-06T09:18:02Z",
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"upload_concurrency": 3,
|
||||
"mode": "http",
|
||||
"count": 80,
|
||||
"concurrency": 40,
|
||||
"body_bytes": 10485256,
|
||||
"status_codes": {
|
||||
"201": 78,
|
||||
"503": 2
|
||||
},
|
||||
"instances": 2,
|
||||
"linked_in_messages": 0,
|
||||
"elapsed_seconds": 9.545,
|
||||
"successful_rps": 8.17,
|
||||
"latency_ms": {
|
||||
"p50": 3560.9,
|
||||
"p95": 8091.8,
|
||||
"p99": 8772.2,
|
||||
"max": 8772.2
|
||||
},
|
||||
"memory": {
|
||||
"798d7474c368": {
|
||||
"idle_mib": 231.7,
|
||||
"docker_stats_peak_mib": 428.2,
|
||||
"cgroup_memory_peak_mib": 450.4,
|
||||
"oom_kill": 0
|
||||
},
|
||||
"0b9a4f6359fb": {
|
||||
"idle_mib": 179.4,
|
||||
"docker_stats_peak_mib": 410.0,
|
||||
"cgroup_memory_peak_mib": 430.7,
|
||||
"oom_kill": 0
|
||||
}
|
||||
},
|
||||
"lb": {
|
||||
"cgroup_memory_peak_mib": 29.1,
|
||||
"oom_kill": 0
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
# 2026-11-06T09:17:56Z commit 75b93bbba5f29575d3fca1d013cc7e0770d88082; api x2: 1 CPU, 512 MiB, 2 workers; MALLOC_MMAP_THRESHOLD_=131072; 2 uploads per client; peaks from cgroup memory.peak, MiB
|
||||
slots size conc codes p50_ms p95_ms api_idle api_peak api_oom lb_peak lb_oom
|
||||
3 10.0M 40 201x78,503x2 3560.9 8091.8 231.7 450.4 0 29.1 0
|
||||
@@ -0,0 +1,154 @@
|
||||
date: 2026-11-06T09:10:41Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ .venv/bin/python student-tests/tools/forward_network.py
|
||||
{
|
||||
"checks": [
|
||||
{
|
||||
"case": "non_ascii_basic",
|
||||
"raw_hex": "42617369632080",
|
||||
"status": 401,
|
||||
"instance": "d4c8d53d7433"
|
||||
},
|
||||
{
|
||||
"case": "non_ascii_basic",
|
||||
"raw_hex": "426173696320ff",
|
||||
"status": 401,
|
||||
"instance": "da922ab2ea09"
|
||||
},
|
||||
{
|
||||
"case": "non_ascii_basic",
|
||||
"raw_hex": "426173696320c3a9",
|
||||
"status": 401,
|
||||
"instance": "d4c8d53d7433"
|
||||
},
|
||||
{
|
||||
"case": "non_ascii_basic",
|
||||
"raw_hex": "42617369632059577870593255366332566a636d5630ff",
|
||||
"status": 401,
|
||||
"instance": "da922ab2ea09"
|
||||
},
|
||||
{
|
||||
"case": "nul_basic_display_title",
|
||||
"position": "'\\x00'",
|
||||
"status": [
|
||||
401,
|
||||
400,
|
||||
201,
|
||||
400
|
||||
],
|
||||
"instance": "d4c8d53d7433"
|
||||
},
|
||||
{
|
||||
"case": "nul_basic_display_title",
|
||||
"position": "'\\x00abc'",
|
||||
"status": [
|
||||
401,
|
||||
400,
|
||||
201,
|
||||
400
|
||||
],
|
||||
"instance": "da922ab2ea09"
|
||||
},
|
||||
{
|
||||
"case": "nul_basic_display_title",
|
||||
"position": "'abc\\x00def'",
|
||||
"status": [
|
||||
401,
|
||||
400,
|
||||
201,
|
||||
400
|
||||
],
|
||||
"instance": "d4c8d53d7433"
|
||||
},
|
||||
{
|
||||
"case": "nul_basic_display_title",
|
||||
"position": "'abc\\x00'",
|
||||
"status": [
|
||||
401,
|
||||
400,
|
||||
201,
|
||||
400
|
||||
],
|
||||
"instance": "da922ab2ea09"
|
||||
},
|
||||
{
|
||||
"case": "nul_message_corrected_retry",
|
||||
"position": "'\\x00'",
|
||||
"instances": [
|
||||
"da922ab2ea09",
|
||||
"d4c8d53d7433"
|
||||
],
|
||||
"status": [
|
||||
400,
|
||||
201,
|
||||
201
|
||||
],
|
||||
"copies": 1
|
||||
},
|
||||
{
|
||||
"case": "nul_message_corrected_retry",
|
||||
"position": "'\\x00abc'",
|
||||
"instances": [
|
||||
"d4c8d53d7433",
|
||||
"da922ab2ea09"
|
||||
],
|
||||
"status": [
|
||||
400,
|
||||
201,
|
||||
201
|
||||
],
|
||||
"copies": 1
|
||||
},
|
||||
{
|
||||
"case": "nul_message_corrected_retry",
|
||||
"position": "'abc\\x00def'",
|
||||
"instances": [
|
||||
"da922ab2ea09",
|
||||
"d4c8d53d7433"
|
||||
],
|
||||
"status": [
|
||||
400,
|
||||
201,
|
||||
201
|
||||
],
|
||||
"copies": 1
|
||||
},
|
||||
{
|
||||
"case": "nul_message_corrected_retry",
|
||||
"position": "'abc\\x00'",
|
||||
"instances": [
|
||||
"d4c8d53d7433",
|
||||
"da922ab2ea09"
|
||||
],
|
||||
"status": [
|
||||
400,
|
||||
201,
|
||||
201
|
||||
],
|
||||
"copies": 1
|
||||
},
|
||||
{
|
||||
"case": "nul_access",
|
||||
"status": [
|
||||
404,
|
||||
404
|
||||
]
|
||||
},
|
||||
{
|
||||
"case": "logout_across_replicas",
|
||||
"status": [
|
||||
204,
|
||||
401,
|
||||
401,
|
||||
401,
|
||||
401
|
||||
]
|
||||
}
|
||||
],
|
||||
"instances": [
|
||||
"d4c8d53d7433",
|
||||
"da922ab2ea09"
|
||||
],
|
||||
"pass": true
|
||||
}
|
||||
exit_code: 0 elapsed_seconds: 0.403
|
||||
@@ -0,0 +1,64 @@
|
||||
date: 2026-11-06T09:07:00Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ sh -c 'make check && make validate && make lint && make student-test'
|
||||
python3 scripts/check_contract.py
|
||||
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
|
||||
.venv/bin/python scripts/validate_openapi.py
|
||||
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
|
||||
.venv/bin/ruff check src student-tests scripts/capture_evidence.py alembic scripts/scaling.py tools scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py
|
||||
All checks passed!
|
||||
.venv/bin/ruff format --check src student-tests scripts/capture_evidence.py alembic scripts/scaling.py tools scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py
|
||||
163 files already formatted
|
||||
.venv/bin/ty check src student-tests alembic
|
||||
All checks passed!
|
||||
.venv/bin/pytest
|
||||
============================= test session starts ==============================
|
||||
platform darwin -- Python 3.14.5, pytest-9.1.1, pluggy-1.6.0
|
||||
rootdir: <repo>
|
||||
configfile: pyproject.toml
|
||||
testpaths: student-tests
|
||||
plugins: asyncio-1.4.0, anyio-4.15.1
|
||||
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function
|
||||
collected 511 items / 19 deselected / 492 selected
|
||||
|
||||
student-tests/adapters/test_s3_storage.py ... [ 0%]
|
||||
student-tests/api/test_access.py ......... [ 2%]
|
||||
student-tests/api/test_attachments.py .................................. [ 9%]
|
||||
.... [ 10%]
|
||||
student-tests/api/test_bounded_contexts.py ................... [ 14%]
|
||||
student-tests/api/test_cache.py ............ [ 16%]
|
||||
student-tests/api/test_concurrency.py .... [ 17%]
|
||||
student-tests/api/test_consistency.py ........ [ 18%]
|
||||
student-tests/api/test_drain.py .. [ 19%]
|
||||
student-tests/api/test_events.py ...... [ 20%]
|
||||
student-tests/api/test_idempotency.py ............... [ 23%]
|
||||
student-tests/api/test_instance.py .... [ 24%]
|
||||
student-tests/api/test_message_attachments.py ........... [ 26%]
|
||||
student-tests/api/test_migrations.py ................................ [ 33%]
|
||||
student-tests/api/test_negative.py ..................................... [ 40%]
|
||||
[ 40%]
|
||||
student-tests/api/test_nul_validation.py ................. [ 44%]
|
||||
student-tests/api/test_operability.py .............. [ 46%]
|
||||
student-tests/api/test_pagination.py .................. [ 50%]
|
||||
student-tests/api/test_pool.py ............ [ 53%]
|
||||
student-tests/api/test_reconciler.py .......... [ 55%]
|
||||
student-tests/api/test_sessions.py ...................... [ 59%]
|
||||
student-tests/api/test_timeout.py ........ [ 61%]
|
||||
student-tests/api/test_upload_consistency.py ......... [ 63%]
|
||||
student-tests/api/test_wire_format.py .. [ 63%]
|
||||
student-tests/unit/test_attachments_domain.py .......................... [ 68%]
|
||||
. [ 68%]
|
||||
student-tests/unit/test_authentication.py ............................. [ 74%]
|
||||
student-tests/unit/test_cursor.py ........... [ 77%]
|
||||
student-tests/unit/test_domain.py ...................................... [ 84%]
|
||||
.. [ 85%]
|
||||
student-tests/unit/test_haproxy_errors.py ............... [ 88%]
|
||||
student-tests/unit/test_nul_inputs.py ...................... [ 92%]
|
||||
student-tests/unit/test_probe.py .............. [ 95%]
|
||||
student-tests/unit/test_reconcile_measurements.py ...... [ 96%]
|
||||
student-tests/unit/test_scaling.py ..... [ 97%]
|
||||
student-tests/unit/test_session_cache.py .......... [ 99%]
|
||||
student-tests/unit/test_timeouts.py . [100%]
|
||||
|
||||
================ 492 passed, 19 deselected in 88.06s (0:01:28) =================
|
||||
exit_code: 0 elapsed_seconds: 90.288
|
||||
@@ -0,0 +1,36 @@
|
||||
date: 2026-11-06T09:06:38Z commit: f0fc084fbbc6a6084ac9526c50fcee45b2c9a277
|
||||
environment: {"NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ sh -c 'make check && make validate && make lint && make student-test'
|
||||
python3 scripts/check_contract.py
|
||||
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
|
||||
.venv/bin/python scripts/validate_openapi.py
|
||||
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
|
||||
.venv/bin/ruff check src student-tests scripts/capture_evidence.py alembic scripts/scaling.py tools scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py
|
||||
All checks passed!
|
||||
.venv/bin/ruff format --check src student-tests scripts/capture_evidence.py alembic scripts/scaling.py tools scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py
|
||||
163 files already formatted
|
||||
.venv/bin/ty check src student-tests alembic
|
||||
error[invalid-argument-type]: Argument to `HTTPConnection.__init__` is incorrect
|
||||
--> student-tests/tools/forward_network.py:16:39
|
||||
|
|
||||
16 | conn = http.client.HTTPConnection(base.hostname, base.port, timeout=12)
|
||||
| ^^^^^^^^^^^^^ Expected `str`, found `str | None`
|
||||
info: element `None` of union `str | None` is not assignable to `str`
|
||||
info: Method defined here
|
||||
--> stdlib/http/client.pyi:336:13
|
||||
|
|
||||
336 | def __init__(
|
||||
| ^^^^^^^^
|
||||
337 | self,
|
||||
338 | host: str,
|
||||
| --------- Parameter declared here
|
||||
|
||||
error[not-subscriptable]: Cannot subscript object of type `None` with no `__getitem__` method
|
||||
--> student-tests/tools/forward_network.py:28:16
|
||||
|
|
||||
28 | assert result["error"]["request_id"] == request_id
|
||||
| ^^^^^^^^^^^^^^^
|
||||
|
||||
Found 2 diagnostics
|
||||
make: *** [lint] Error 1
|
||||
exit_code: 2 elapsed_seconds: 0.678
|
||||
@@ -0,0 +1,7 @@
|
||||
date: 2026-11-06T09:32:35Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {}
|
||||
$ openspec archive forward-lab3-final-review-fixes --yes --skip-specs
|
||||
Task status: ✓ Complete
|
||||
Skipping spec updates (--skip-specs flag provided).
|
||||
Change 'forward-lab3-final-review-fixes' archived as '2026-11-06-forward-lab3-final-review-fixes'.
|
||||
exit_code: 0 elapsed_seconds: 0.776
|
||||
@@ -0,0 +1,17 @@
|
||||
date: 2026-11-06T09:18:19Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ sh -c 'uptime; docker ps --format "{{.Names}} {{.Status}}"; docker stats --no-stream --format "{{.Name}} {{.CPUPerc}} {{.MemUsage}}"'
|
||||
12:18 up 37 days, 11:30, 6 users, load averages: 4.69 4.19 4.16
|
||||
hlpipe-lab4-forward-final-20261106-api-1 Up 2 seconds (healthy)
|
||||
hlpipe-lab4-forward-final-20261106-api-2 Up 2 seconds (healthy)
|
||||
hlpipe-lab4-forward-final-20261106-lb-1 Up 2 seconds (healthy)
|
||||
hlpipe-lab4-forward-final-20261106-postgres-1 Up 5 minutes (healthy)
|
||||
hlpipe-lab4-forward-final-20261106-s3-1 Up 4 minutes (healthy)
|
||||
hlpipe-lab4-forward-final-20261106-valkey-1 Up 5 minutes (healthy)
|
||||
hlpipe-lab4-forward-final-20261106-api-1 0.49% 227.9MiB / 512MiB
|
||||
hlpipe-lab4-forward-final-20261106-api-2 0.83% 227.9MiB / 512MiB
|
||||
hlpipe-lab4-forward-final-20261106-lb-1 1.16% 15.97MiB / 128MiB
|
||||
hlpipe-lab4-forward-final-20261106-postgres-1 2.18% 59.24MiB / 1GiB
|
||||
hlpipe-lab4-forward-final-20261106-s3-1 1.32% 297.5MiB / 512MiB
|
||||
hlpipe-lab4-forward-final-20261106-valkey-1 0.19% 13.54MiB / 128MiB
|
||||
exit_code: 0 elapsed_seconds: 2.646
|
||||
@@ -0,0 +1,18 @@
|
||||
date: 2026-11-06T09:16:14Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ sh -c 'docker info --format "cpus={{.NCPU}} memory={{.MemTotal}}"; uptime; docker ps --format "{{.Names}} {{.Status}}"; docker stats --no-stream --format "{{.Name}} {{.CPUPerc}} {{.MemUsage}}"'
|
||||
cpus=16 memory=8217190400
|
||||
12:16 up 37 days, 11:28, 6 users, load averages: 3.42 4.16 4.16
|
||||
hlpipe-lab4-forward-final-20261106-api-1 Up About a minute (healthy)
|
||||
hlpipe-lab4-forward-final-20261106-api-2 Up 3 minutes (healthy)
|
||||
hlpipe-lab4-forward-final-20261106-postgres-1 Up 3 minutes (healthy)
|
||||
hlpipe-lab4-forward-final-20261106-lb-1 Up 3 minutes (healthy)
|
||||
hlpipe-lab4-forward-final-20261106-s3-1 Up 2 minutes (healthy)
|
||||
hlpipe-lab4-forward-final-20261106-valkey-1 Up 3 minutes (healthy)
|
||||
hlpipe-lab4-forward-final-20261106-api-1 12.78% 258.1MiB / 512MiB
|
||||
hlpipe-lab4-forward-final-20261106-api-2 0.83% 298.8MiB / 512MiB
|
||||
hlpipe-lab4-forward-final-20261106-postgres-1 0.13% 40.44MiB / 1GiB
|
||||
hlpipe-lab4-forward-final-20261106-lb-1 0.14% 24.3MiB / 128MiB
|
||||
hlpipe-lab4-forward-final-20261106-s3-1 0.04% 69.52MiB / 512MiB
|
||||
hlpipe-lab4-forward-final-20261106-valkey-1 1.25% 13.12MiB / 128MiB
|
||||
exit_code: 0 elapsed_seconds: 2.724
|
||||
@@ -0,0 +1,17 @@
|
||||
date: 2026-11-06T09:09:23Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ sh scripts/persistence-evidence.sh
|
||||
date: 2026-11-06T09:09:23Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
== before: service hostname StartedAt status
|
||||
api d4c8d53d7433 2026-11-06T09:09:18.754909134Z running
|
||||
api da922ab2ea09 2026-11-06T09:09:18.824598676Z running
|
||||
lb ea6930f64089 2026-11-06T09:09:12.580488131Z running
|
||||
migrate adc778a3d97c 2026-11-06T09:09:13.657768882Z exited
|
||||
postgres 3f3a870d89c0 2026-11-06T09:09:12.588930798Z running
|
||||
s3 5921e788f03d 2026-11-06T09:09:12.58822809Z running
|
||||
s3-init c90ac1d6c503 2026-11-06T09:09:18.165289926Z exited
|
||||
valkey 07ea16e2fcfe 2026-11-06T09:09:12.587649298Z running
|
||||
python3 tests/resilience.py --mode persistence --action-script "scripts/restart.sh" --base-url "http://localhost:8080"
|
||||
FAIL: GET /health/live: connection/TLS failure (URLError)
|
||||
make: *** [test-persistence] Error 1
|
||||
exit_code: 2 elapsed_seconds: 0.381
|
||||
@@ -0,0 +1,26 @@
|
||||
date: 2026-11-06T09:10:08Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-final-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1"}
|
||||
$ sh scripts/persistence-evidence.sh http://127.0.0.1:18084
|
||||
date: 2026-11-06T09:10:08Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
== before: service hostname StartedAt status
|
||||
api d4c8d53d7433 2026-11-06T09:09:18.754909134Z running
|
||||
api da922ab2ea09 2026-11-06T09:09:18.824598676Z running
|
||||
lb ea6930f64089 2026-11-06T09:09:12.580488131Z running
|
||||
migrate adc778a3d97c 2026-11-06T09:09:13.657768882Z exited
|
||||
postgres 3f3a870d89c0 2026-11-06T09:09:12.588930798Z running
|
||||
s3 5921e788f03d 2026-11-06T09:09:12.58822809Z running
|
||||
s3-init c90ac1d6c503 2026-11-06T09:09:18.165289926Z exited
|
||||
valkey 07ea16e2fcfe 2026-11-06T09:09:12.587649298Z running
|
||||
python3 tests/resilience.py --mode persistence --action-script "scripts/restart.sh" --base-url "http://127.0.0.1:18084"
|
||||
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
|
||||
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
== after: service hostname StartedAt status
|
||||
api d4c8d53d7433 2026-11-06T09:10:18.309662551Z running
|
||||
api da922ab2ea09 2026-11-06T09:10:18.377623592Z running
|
||||
lb ea6930f64089 2026-11-06T09:10:12.683407215Z running
|
||||
migrate adc778a3d97c 2026-11-06T09:10:13.202156257Z exited
|
||||
postgres 3f3a870d89c0 2026-11-06T09:10:12.374370839Z running
|
||||
s3 5921e788f03d 2026-11-06T09:10:12.339453423Z running
|
||||
s3-init c90ac1d6c503 2026-11-06T09:10:17.710432217Z exited
|
||||
valkey 07ea16e2fcfe 2026-11-06T09:10:12.392600006Z running
|
||||
exit_code: 0 elapsed_seconds: 13.179
|
||||
@@ -0,0 +1,37 @@
|
||||
date: 2026-11-06T09:33:10Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {}
|
||||
$ sh -c 'make lint && openspec validate --all --strict && openspec list --json && git diff --check && git diff --exit-code d7478726a85791c21e065d58f610735a9e687433 -- contracts tests README.md COURSE.md'
|
||||
.venv/bin/ruff check src student-tests scripts/capture_evidence.py alembic scripts/scaling.py tools scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py
|
||||
All checks passed!
|
||||
.venv/bin/ruff format --check src student-tests scripts/capture_evidence.py alembic scripts/scaling.py tools scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py
|
||||
163 files already formatted
|
||||
.venv/bin/ty check src student-tests alembic
|
||||
All checks passed!
|
||||
- Validating...
|
||||
✓ spec/lab1-event-publication
|
||||
✓ spec/lab1-messenger-chats
|
||||
✓ spec/lab1-messenger-history
|
||||
✓ spec/lab1-messenger-identity
|
||||
✓ spec/lab1-service-operability
|
||||
✓ spec/lab2-auth-sessions
|
||||
✓ spec/lab2-dependency-failures
|
||||
✓ spec/lab2-history-index
|
||||
✓ spec/lab2-persistent-storage
|
||||
✓ spec/lab2-session-cache
|
||||
✓ spec/lab3-failover-operations
|
||||
✓ spec/lab3-idempotent-messages
|
||||
✓ spec/lab3-load-balancing
|
||||
✓ spec/lab4-attachments
|
||||
✓ spec/lab4-evidence-runs
|
||||
✓ spec/lab4-object-storage
|
||||
✓ spec/lab4-storage-reconciler
|
||||
✓ spec/lab4-upload-resilience
|
||||
Totals: 18 passed, 0 failed (18 items)
|
||||
{
|
||||
"changes": [],
|
||||
"root": {
|
||||
"path": "<repo>",
|
||||
"source": "nearest"
|
||||
}
|
||||
}
|
||||
exit_code: 0 elapsed_seconds: 1.921
|
||||
@@ -0,0 +1,19 @@
|
||||
date: 2026-11-06T09:19:41Z commit: 75b93bbba5f29575d3fca1d013cc7e0770d88082
|
||||
environment: {"COMPOSE_PROJECT_NAME": "hlpipe-lab4-forward-reconcile-20261106", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18084", "LB_STATS_PORT": "24844", "BASE_URL": "http://127.0.0.1:18084", "EVIDENCE_DIR": "evidence/forward-final-2026-11-06", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:hlpipe-lab4-forward-final-20261106"}
|
||||
$ make reconcile-demo 'ARGS=--out evidence/forward-final-2026-11-06/reconcile-demo'
|
||||
python3 scripts/reconcile_track.py demo --base-url "http://127.0.0.1:18084" --out evidence/forward-final-2026-11-06/reconcile-demo
|
||||
PASS dry-run находит каждый внедрённый класс, и найдено ровно внедрённое
|
||||
PASS dry-run ничего не меняет
|
||||
PASS apply исправляет ровно найденное dry-run, удаление намерений — в журнале аудита
|
||||
PASS повторный apply — 0 действий
|
||||
PASS независимая проверка после apply: нарушений нет, готовые не потеряны
|
||||
PASS строки с расхождениями и ссылки сообщений сохранены
|
||||
PASS карантин: метаданные 200, /content 404; нетронутое скачивается
|
||||
PASS вернули те же байты — карантин снят; другие байты той же длины — остался
|
||||
PASS второй одновременный проход пропущен
|
||||
PASS SIGTERM: идущий проход останавливается за секунды, блокировка освобождена
|
||||
PASS S3 остановлен: код 1 за ограниченное время, состояние не изменилось
|
||||
PASS под нагрузкой: исправлены только внедрённые, ошибок нет
|
||||
PASS под нагрузкой: все 201 готовы и совпадают по SHA-256, потерь 0
|
||||
{"ok": true, "checks": 13}
|
||||
exit_code: 0 elapsed_seconds: 257.493
|
||||
@@ -0,0 +1,142 @@
|
||||
{
|
||||
"date": "2026-11-06T09:20:54Z",
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"mode": "apply",
|
||||
"skipped": false,
|
||||
"grace_seconds": 60,
|
||||
"skew_seconds": 0,
|
||||
"deep": true,
|
||||
"page_size": 1000,
|
||||
"scanned": {
|
||||
"objects": 24,
|
||||
"rows": 27
|
||||
},
|
||||
"found": {
|
||||
"orphan_stale": 3,
|
||||
"orphan_fresh": 2,
|
||||
"missing": 3,
|
||||
"size_mismatch": 3,
|
||||
"sha_mismatch": 3,
|
||||
"pending_stale": 6,
|
||||
"pending_fresh": 2,
|
||||
"quarantined": 0,
|
||||
"healed": 0
|
||||
},
|
||||
"fixed": {
|
||||
"orphan_stale": 3,
|
||||
"missing": 3,
|
||||
"size_mismatch": 3,
|
||||
"sha_mismatch": 3,
|
||||
"pending_stale": 6,
|
||||
"healed": 0
|
||||
},
|
||||
"rechecked": {},
|
||||
"errors": 0,
|
||||
"samples": {
|
||||
"missing": [
|
||||
"01a1b028-2293-743d-a00c-f7fec1a50808",
|
||||
"01a1b028-22ad-77fa-9b2e-d01e032a2c10",
|
||||
"01a1b028-2306-74ed-9bff-7c2894a33621"
|
||||
],
|
||||
"size_mismatch": [
|
||||
"01a1b028-22ba-778b-99d5-eef1d99349f1",
|
||||
"01a1b028-2316-7510-b41e-754b39583de9",
|
||||
"01a1b028-231e-70ba-8a9e-1de6d00c2e0d"
|
||||
],
|
||||
"sha_mismatch": [
|
||||
"01a1b028-22c2-7713-adf4-93a25c06e07e",
|
||||
"01a1b028-2327-7105-838f-3f0df5b10dc2",
|
||||
"01a1b028-2330-7662-8f98-6b45a5c165a2"
|
||||
],
|
||||
"pending_stale": [
|
||||
"2cc53ddd-063f-4edc-ae55-98f201965b7f",
|
||||
"73fc548a-79a1-4396-a848-2e598eba75a1",
|
||||
"848bed6c-437a-4169-8cca-940c1987055b",
|
||||
"99d824a9-d16d-481e-9804-e209f2fa410b",
|
||||
"a0a3b632-5fe3-447e-a93b-bbdce77c4fb1",
|
||||
"d86c4cba-2836-4150-bf0e-d32e1ec442ec"
|
||||
],
|
||||
"orphan_stale": [
|
||||
"attachments/35df74ac-5753-48e0-b106-d596db2ca8e6",
|
||||
"attachments/8ba5a522-a4b8-40a8-b1f3-274401976db6",
|
||||
"attachments/de3cc053-2a20-44ad-b223-e06818b039e7"
|
||||
],
|
||||
"orphan_fresh": [
|
||||
"attachments/7b96d477-2dc4-4a8c-8703-e6176a7e46a6",
|
||||
"attachments/d69d9843-ceab-4101-a278-7e5974071ac6"
|
||||
],
|
||||
"pending_fresh": [
|
||||
"dfb03912-2304-4280-bfb7-7a7b9cb28597",
|
||||
"f1cf4216-9e57-4217-866a-440aac9435f3"
|
||||
]
|
||||
},
|
||||
"cost": {
|
||||
"seconds": 0.119,
|
||||
"requests": {
|
||||
"s3_list": 1,
|
||||
"s3_head": 6,
|
||||
"s3_get": 13,
|
||||
"s3_delete": 9,
|
||||
"sql": 19
|
||||
},
|
||||
"waited_seconds": {
|
||||
"s3_list": 0.018,
|
||||
"sql": 0.006
|
||||
},
|
||||
"hashed_bytes": 1453621,
|
||||
"cpu_seconds": 0.621,
|
||||
"max_rss_mib": 107.6
|
||||
},
|
||||
"exit_code": 0,
|
||||
"wall_seconds": 1.106,
|
||||
"audit": [
|
||||
{
|
||||
"attachment_id": "2cc53ddd-063f-4edc-ae55-98f201965b7f",
|
||||
"chat_id": "01a1b028-1d47-75b7-a3d7-f9dda0509d40",
|
||||
"reason": "reconcile",
|
||||
"event": "audit.attachment_discarded",
|
||||
"level": "info",
|
||||
"ts": "2026-11-06T09:20:54.712257Z"
|
||||
},
|
||||
{
|
||||
"attachment_id": "73fc548a-79a1-4396-a848-2e598eba75a1",
|
||||
"chat_id": "01a1b028-1d47-75b7-a3d7-f9dda0509d40",
|
||||
"reason": "reconcile",
|
||||
"event": "audit.attachment_discarded",
|
||||
"level": "info",
|
||||
"ts": "2026-11-06T09:20:54.724316Z"
|
||||
},
|
||||
{
|
||||
"attachment_id": "848bed6c-437a-4169-8cca-940c1987055b",
|
||||
"chat_id": "01a1b028-1d47-75b7-a3d7-f9dda0509d40",
|
||||
"reason": "reconcile",
|
||||
"event": "audit.attachment_discarded",
|
||||
"level": "info",
|
||||
"ts": "2026-11-06T09:20:54.732394Z"
|
||||
},
|
||||
{
|
||||
"attachment_id": "99d824a9-d16d-481e-9804-e209f2fa410b",
|
||||
"chat_id": "01a1b028-1d47-75b7-a3d7-f9dda0509d40",
|
||||
"reason": "reconcile",
|
||||
"event": "audit.attachment_discarded",
|
||||
"level": "info",
|
||||
"ts": "2026-11-06T09:20:54.741795Z"
|
||||
},
|
||||
{
|
||||
"attachment_id": "a0a3b632-5fe3-447e-a93b-bbdce77c4fb1",
|
||||
"chat_id": "01a1b028-1d47-75b7-a3d7-f9dda0509d40",
|
||||
"reason": "reconcile",
|
||||
"event": "audit.attachment_discarded",
|
||||
"level": "info",
|
||||
"ts": "2026-11-06T09:20:54.749494Z"
|
||||
},
|
||||
{
|
||||
"attachment_id": "d86c4cba-2836-4150-bf0e-d32e1ec442ec",
|
||||
"chat_id": "01a1b028-1d47-75b7-a3d7-f9dda0509d40",
|
||||
"reason": "reconcile",
|
||||
"event": "audit.attachment_discarded",
|
||||
"level": "info",
|
||||
"ts": "2026-11-06T09:20:54.757598Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,76 @@
|
||||
{
|
||||
"date": "2026-11-06T09:20:55Z",
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"mode": "apply",
|
||||
"skipped": false,
|
||||
"grace_seconds": 60,
|
||||
"skew_seconds": 0,
|
||||
"deep": true,
|
||||
"page_size": 1000,
|
||||
"scanned": {
|
||||
"objects": 18,
|
||||
"rows": 21
|
||||
},
|
||||
"found": {
|
||||
"orphan_stale": 0,
|
||||
"orphan_fresh": 2,
|
||||
"missing": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale": 0,
|
||||
"pending_fresh": 2,
|
||||
"quarantined": 9,
|
||||
"healed": 0
|
||||
},
|
||||
"fixed": {
|
||||
"orphan_stale": 0,
|
||||
"missing": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale": 0,
|
||||
"healed": 0
|
||||
},
|
||||
"rechecked": {},
|
||||
"errors": 0,
|
||||
"samples": {
|
||||
"quarantined": [
|
||||
"01a1b028-2293-743d-a00c-f7fec1a50808",
|
||||
"01a1b028-22ad-77fa-9b2e-d01e032a2c10",
|
||||
"01a1b028-22ba-778b-99d5-eef1d99349f1",
|
||||
"01a1b028-2306-74ed-9bff-7c2894a33621",
|
||||
"01a1b028-2316-7510-b41e-754b39583de9",
|
||||
"01a1b028-231e-70ba-8a9e-1de6d00c2e0d",
|
||||
"01a1b028-22c2-7713-adf4-93a25c06e07e",
|
||||
"01a1b028-2327-7105-838f-3f0df5b10dc2",
|
||||
"01a1b028-2330-7662-8f98-6b45a5c165a2"
|
||||
],
|
||||
"orphan_fresh": [
|
||||
"attachments/7b96d477-2dc4-4a8c-8703-e6176a7e46a6",
|
||||
"attachments/d69d9843-ceab-4101-a278-7e5974071ac6"
|
||||
],
|
||||
"pending_fresh": [
|
||||
"dfb03912-2304-4280-bfb7-7a7b9cb28597",
|
||||
"f1cf4216-9e57-4217-866a-440aac9435f3"
|
||||
]
|
||||
},
|
||||
"cost": {
|
||||
"seconds": 0.043,
|
||||
"requests": {
|
||||
"s3_list": 1,
|
||||
"s3_head": 0,
|
||||
"s3_get": 13,
|
||||
"s3_delete": 0,
|
||||
"sql": 1
|
||||
},
|
||||
"waited_seconds": {
|
||||
"s3_list": 0.017,
|
||||
"sql": 0.006
|
||||
},
|
||||
"hashed_bytes": 1453621,
|
||||
"cpu_seconds": 0.595,
|
||||
"max_rss_mib": 107.4
|
||||
},
|
||||
"exit_code": 0,
|
||||
"wall_seconds": 1.021,
|
||||
"audit": []
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
{
|
||||
"checked_at": "2026-11-06T09:20:57Z",
|
||||
"grace_seconds": 60,
|
||||
"deep": true,
|
||||
"objects": 18,
|
||||
"rows": 21,
|
||||
"ready_healthy": 10,
|
||||
"counts": {
|
||||
"orphan_stale": 0,
|
||||
"orphan_fresh": 2,
|
||||
"missing_referenced": 0,
|
||||
"missing_unreferenced": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale_with_object": 0,
|
||||
"pending_stale_without_object": 0,
|
||||
"pending_fresh": 2,
|
||||
"quarantined": 9,
|
||||
"incomplete_stale": 0,
|
||||
"incomplete_fresh": 0,
|
||||
"lost_ready": 0
|
||||
},
|
||||
"samples": {
|
||||
"orphan_fresh": [
|
||||
"attachments/7b96d477-2dc4-4a8c-8703-e6176a7e46a6",
|
||||
"attachments/d69d9843-ceab-4101-a278-7e5974071ac6"
|
||||
],
|
||||
"pending_fresh": [
|
||||
"dfb03912-2304-4280-bfb7-7a7b9cb28597",
|
||||
"f1cf4216-9e57-4217-866a-440aac9435f3"
|
||||
],
|
||||
"quarantined": [
|
||||
"01a1b028-2293-743d-a00c-f7fec1a50808",
|
||||
"01a1b028-22ad-77fa-9b2e-d01e032a2c10",
|
||||
"01a1b028-22ba-778b-99d5-eef1d99349f1",
|
||||
"01a1b028-22c2-7713-adf4-93a25c06e07e",
|
||||
"01a1b028-2306-74ed-9bff-7c2894a33621",
|
||||
"01a1b028-2316-7510-b41e-754b39583de9",
|
||||
"01a1b028-231e-70ba-8a9e-1de6d00c2e0d",
|
||||
"01a1b028-2327-7105-838f-3f0df5b10dc2",
|
||||
"01a1b028-2330-7662-8f98-6b45a5c165a2"
|
||||
]
|
||||
},
|
||||
"seconds": {
|
||||
"s3_list": 0.3,
|
||||
"s3_incomplete": 0.325,
|
||||
"pg_query": 0.128,
|
||||
"deep_hash": 0.593,
|
||||
"total": 1.348
|
||||
},
|
||||
"violations": [],
|
||||
"ok": true
|
||||
}
|
||||
@@ -0,0 +1,82 @@
|
||||
{
|
||||
"checked_at": "2026-11-06T09:20:51Z",
|
||||
"grace_seconds": 60,
|
||||
"deep": true,
|
||||
"objects": 24,
|
||||
"rows": 27,
|
||||
"ready_healthy": 10,
|
||||
"counts": {
|
||||
"orphan_stale": 3,
|
||||
"orphan_fresh": 2,
|
||||
"missing_referenced": 2,
|
||||
"missing_unreferenced": 1,
|
||||
"size_mismatch": 3,
|
||||
"sha_mismatch": 3,
|
||||
"pending_stale_with_object": 3,
|
||||
"pending_stale_without_object": 3,
|
||||
"pending_fresh": 2,
|
||||
"quarantined": 0,
|
||||
"incomplete_stale": 0,
|
||||
"incomplete_fresh": 0,
|
||||
"lost_ready": 0
|
||||
},
|
||||
"samples": {
|
||||
"orphan_stale": [
|
||||
"attachments/35df74ac-5753-48e0-b106-d596db2ca8e6",
|
||||
"attachments/8ba5a522-a4b8-40a8-b1f3-274401976db6",
|
||||
"attachments/de3cc053-2a20-44ad-b223-e06818b039e7"
|
||||
],
|
||||
"orphan_fresh": [
|
||||
"attachments/7b96d477-2dc4-4a8c-8703-e6176a7e46a6",
|
||||
"attachments/d69d9843-ceab-4101-a278-7e5974071ac6"
|
||||
],
|
||||
"missing_referenced": [
|
||||
"01a1b028-2293-743d-a00c-f7fec1a50808",
|
||||
"01a1b028-22ad-77fa-9b2e-d01e032a2c10"
|
||||
],
|
||||
"missing_unreferenced": [
|
||||
"01a1b028-2306-74ed-9bff-7c2894a33621"
|
||||
],
|
||||
"size_mismatch": [
|
||||
"01a1b028-22ba-778b-99d5-eef1d99349f1",
|
||||
"01a1b028-2316-7510-b41e-754b39583de9",
|
||||
"01a1b028-231e-70ba-8a9e-1de6d00c2e0d"
|
||||
],
|
||||
"sha_mismatch": [
|
||||
"01a1b028-22c2-7713-adf4-93a25c06e07e",
|
||||
"01a1b028-2327-7105-838f-3f0df5b10dc2",
|
||||
"01a1b028-2330-7662-8f98-6b45a5c165a2"
|
||||
],
|
||||
"pending_stale_with_object": [
|
||||
"848bed6c-437a-4169-8cca-940c1987055b",
|
||||
"a0a3b632-5fe3-447e-a93b-bbdce77c4fb1",
|
||||
"d86c4cba-2836-4150-bf0e-d32e1ec442ec"
|
||||
],
|
||||
"pending_stale_without_object": [
|
||||
"2cc53ddd-063f-4edc-ae55-98f201965b7f",
|
||||
"73fc548a-79a1-4396-a848-2e598eba75a1",
|
||||
"99d824a9-d16d-481e-9804-e209f2fa410b"
|
||||
],
|
||||
"pending_fresh": [
|
||||
"dfb03912-2304-4280-bfb7-7a7b9cb28597",
|
||||
"f1cf4216-9e57-4217-866a-440aac9435f3"
|
||||
]
|
||||
},
|
||||
"seconds": {
|
||||
"s3_list": 0.304,
|
||||
"s3_incomplete": 0.347,
|
||||
"pg_query": 0.112,
|
||||
"deep_hash": 0.556,
|
||||
"total": 1.319
|
||||
},
|
||||
"violations": [
|
||||
"orphan_stale",
|
||||
"missing_referenced",
|
||||
"missing_unreferenced",
|
||||
"size_mismatch",
|
||||
"sha_mismatch",
|
||||
"pending_stale_with_object",
|
||||
"pending_stale_without_object"
|
||||
],
|
||||
"ok": false
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"checked_at": "2026-11-06T09:23:59Z",
|
||||
"grace_seconds": 60,
|
||||
"deep": true,
|
||||
"objects": 3382,
|
||||
"rows": 3383,
|
||||
"ready_healthy": 3375,
|
||||
"counts": {
|
||||
"orphan_stale": 0,
|
||||
"orphan_fresh": 0,
|
||||
"missing_referenced": 0,
|
||||
"missing_unreferenced": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale_with_object": 0,
|
||||
"pending_stale_without_object": 0,
|
||||
"pending_fresh": 0,
|
||||
"quarantined": 8,
|
||||
"incomplete_stale": 0,
|
||||
"incomplete_fresh": 0,
|
||||
"lost_ready": 0
|
||||
},
|
||||
"samples": {
|
||||
"quarantined": [
|
||||
"01a1b028-22ad-77fa-9b2e-d01e032a2c10",
|
||||
"01a1b028-22ba-778b-99d5-eef1d99349f1",
|
||||
"01a1b028-22c2-7713-adf4-93a25c06e07e",
|
||||
"01a1b028-2306-74ed-9bff-7c2894a33621",
|
||||
"01a1b028-2316-7510-b41e-754b39583de9",
|
||||
"01a1b028-231e-70ba-8a9e-1de6d00c2e0d",
|
||||
"01a1b028-2327-7105-838f-3f0df5b10dc2",
|
||||
"01a1b028-2330-7662-8f98-6b45a5c165a2"
|
||||
]
|
||||
},
|
||||
"seconds": {
|
||||
"s3_list": 0.677,
|
||||
"s3_incomplete": 0.367,
|
||||
"pg_query": 0.149,
|
||||
"deep_hash": 67.224,
|
||||
"total": 68.428
|
||||
},
|
||||
"violations": [],
|
||||
"ok": true
|
||||
}
|
||||
@@ -0,0 +1,245 @@
|
||||
{
|
||||
"date": "2026-11-06T09:23:59Z",
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"started_at": "2026-11-06T09:19:42Z",
|
||||
"faults": {
|
||||
"injected": {
|
||||
"orphan_stale": [
|
||||
"35df74ac-5753-48e0-b106-d596db2ca8e6",
|
||||
"de3cc053-2a20-44ad-b223-e06818b039e7",
|
||||
"8ba5a522-a4b8-40a8-b1f3-274401976db6"
|
||||
],
|
||||
"missing": [
|
||||
"01a1b028-2306-74ed-9bff-7c2894a33621",
|
||||
"01a1b028-22ad-77fa-9b2e-d01e032a2c10",
|
||||
"01a1b028-2293-743d-a00c-f7fec1a50808"
|
||||
],
|
||||
"size_mismatch": [
|
||||
"01a1b028-231e-70ba-8a9e-1de6d00c2e0d",
|
||||
"01a1b028-22ba-778b-99d5-eef1d99349f1",
|
||||
"01a1b028-2316-7510-b41e-754b39583de9"
|
||||
],
|
||||
"sha_mismatch": [
|
||||
"01a1b028-2330-7662-8f98-6b45a5c165a2",
|
||||
"01a1b028-2327-7105-838f-3f0df5b10dc2",
|
||||
"01a1b028-22c2-7713-adf4-93a25c06e07e"
|
||||
],
|
||||
"pending_stale": [
|
||||
"2cc53ddd-063f-4edc-ae55-98f201965b7f",
|
||||
"99d824a9-d16d-481e-9804-e209f2fa410b",
|
||||
"73fc548a-79a1-4396-a848-2e598eba75a1",
|
||||
"a0a3b632-5fe3-447e-a93b-bbdce77c4fb1",
|
||||
"848bed6c-437a-4169-8cca-940c1987055b",
|
||||
"d86c4cba-2836-4150-bf0e-d32e1ec442ec"
|
||||
],
|
||||
"orphan_fresh": [
|
||||
"d69d9843-ceab-4101-a278-7e5974071ac6",
|
||||
"7b96d477-2dc4-4a8c-8703-e6176a7e46a6"
|
||||
],
|
||||
"pending_fresh": [
|
||||
"f1cf4216-9e57-4217-866a-440aac9435f3",
|
||||
"dfb03912-2304-4280-bfb7-7a7b9cb28597"
|
||||
]
|
||||
},
|
||||
"targets": {
|
||||
"missing": [
|
||||
"01a1b028-2293-743d-a00c-f7fec1a50808",
|
||||
"01a1b028-22ad-77fa-9b2e-d01e032a2c10",
|
||||
"01a1b028-2306-74ed-9bff-7c2894a33621"
|
||||
],
|
||||
"corrupt": [
|
||||
"01a1b028-22ba-778b-99d5-eef1d99349f1",
|
||||
"01a1b028-2316-7510-b41e-754b39583de9",
|
||||
"01a1b028-231e-70ba-8a9e-1de6d00c2e0d"
|
||||
],
|
||||
"bitflip": [
|
||||
"01a1b028-22c2-7713-adf4-93a25c06e07e",
|
||||
"01a1b028-2327-7105-838f-3f0df5b10dc2",
|
||||
"01a1b028-2330-7662-8f98-6b45a5c165a2"
|
||||
]
|
||||
},
|
||||
"linked": [
|
||||
"01a1b028-2293-743d-a00c-f7fec1a50808",
|
||||
"01a1b028-22ad-77fa-9b2e-d01e032a2c10",
|
||||
"01a1b028-22ba-778b-99d5-eef1d99349f1",
|
||||
"01a1b028-22c2-7713-adf4-93a25c06e07e",
|
||||
"01a1b028-22cb-73b4-92c8-0db140de50a1",
|
||||
"01a1b028-22d2-7761-8bdd-8b9987dcd70f"
|
||||
]
|
||||
},
|
||||
"seconds": 256.6,
|
||||
"checks": [
|
||||
{
|
||||
"name": "dry-run находит каждый внедрённый класс, и найдено ровно внедрённое",
|
||||
"ok": true,
|
||||
"expected": {
|
||||
"orphan_stale": 3,
|
||||
"missing": 3,
|
||||
"size_mismatch": 3,
|
||||
"sha_mismatch": 3,
|
||||
"pending_stale": 6,
|
||||
"orphan_fresh": 2,
|
||||
"pending_fresh": 2,
|
||||
"quarantined": 0
|
||||
},
|
||||
"found": {
|
||||
"orphan_stale": 3,
|
||||
"orphan_fresh": 2,
|
||||
"missing": 3,
|
||||
"size_mismatch": 3,
|
||||
"sha_mismatch": 3,
|
||||
"pending_stale": 6,
|
||||
"pending_fresh": 2,
|
||||
"quarantined": 0,
|
||||
"healed": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "dry-run ничего не меняет",
|
||||
"ok": true,
|
||||
"objects": [
|
||||
24,
|
||||
24
|
||||
],
|
||||
"rows": [
|
||||
27,
|
||||
27
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "apply исправляет ровно найденное dry-run, удаление намерений — в журнале аудита",
|
||||
"ok": true,
|
||||
"dry_run": {
|
||||
"orphan_stale": 3,
|
||||
"missing": 3,
|
||||
"size_mismatch": 3,
|
||||
"sha_mismatch": 3,
|
||||
"pending_stale": 6
|
||||
},
|
||||
"fixed": {
|
||||
"orphan_stale": 3,
|
||||
"missing": 3,
|
||||
"size_mismatch": 3,
|
||||
"sha_mismatch": 3,
|
||||
"pending_stale": 6
|
||||
},
|
||||
"audit_discarded": 6
|
||||
},
|
||||
{
|
||||
"name": "повторный apply — 0 действий",
|
||||
"ok": true,
|
||||
"fixed": {
|
||||
"orphan_stale": 0,
|
||||
"missing": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale": 0,
|
||||
"healed": 0
|
||||
},
|
||||
"found": {
|
||||
"orphan_stale": 0,
|
||||
"orphan_fresh": 2,
|
||||
"missing": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale": 0,
|
||||
"pending_fresh": 2,
|
||||
"quarantined": 9,
|
||||
"healed": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "независимая проверка после apply: нарушений нет, готовые не потеряны",
|
||||
"ok": true,
|
||||
"counts": {
|
||||
"orphan_stale": 0,
|
||||
"orphan_fresh": 2,
|
||||
"missing_referenced": 0,
|
||||
"missing_unreferenced": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale_with_object": 0,
|
||||
"pending_stale_without_object": 0,
|
||||
"pending_fresh": 2,
|
||||
"quarantined": 9,
|
||||
"incomplete_stale": 0,
|
||||
"incomplete_fresh": 0,
|
||||
"lost_ready": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "строки с расхождениями и ссылки сообщений сохранены",
|
||||
"ok": true,
|
||||
"quarantine": {
|
||||
"01a1b028-2293-743d-a00c-f7fec1a50808": "missing",
|
||||
"01a1b028-22ad-77fa-9b2e-d01e032a2c10": "missing",
|
||||
"01a1b028-22ba-778b-99d5-eef1d99349f1": "size_mismatch",
|
||||
"01a1b028-22c2-7713-adf4-93a25c06e07e": "sha_mismatch",
|
||||
"01a1b028-2306-74ed-9bff-7c2894a33621": "missing",
|
||||
"01a1b028-2316-7510-b41e-754b39583de9": "size_mismatch",
|
||||
"01a1b028-231e-70ba-8a9e-1de6d00c2e0d": "size_mismatch",
|
||||
"01a1b028-2327-7105-838f-3f0df5b10dc2": "sha_mismatch",
|
||||
"01a1b028-2330-7662-8f98-6b45a5c165a2": "sha_mismatch"
|
||||
},
|
||||
"message_links": 6
|
||||
},
|
||||
{
|
||||
"name": "карантин: метаданные 200, /content 404; нетронутое скачивается",
|
||||
"ok": true
|
||||
},
|
||||
{
|
||||
"name": "вернули те же байты — карантин снят; другие байты той же длины — остался",
|
||||
"ok": true,
|
||||
"fixed": {
|
||||
"orphan_stale": 0,
|
||||
"missing": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale": 0,
|
||||
"healed": 1
|
||||
},
|
||||
"s3_get": 5
|
||||
},
|
||||
{
|
||||
"name": "второй одновременный проход пропущен",
|
||||
"ok": true
|
||||
},
|
||||
{
|
||||
"name": "SIGTERM: идущий проход останавливается за секунды, блокировка освобождена",
|
||||
"ok": true,
|
||||
"s3": "paused",
|
||||
"state_before_stop": "running",
|
||||
"init": "true",
|
||||
"stop_seconds": 0.1,
|
||||
"exit_code": 143,
|
||||
"next_pass_skipped": false
|
||||
},
|
||||
{
|
||||
"name": "S3 остановлен: код 1 за ограниченное время, состояние не изменилось",
|
||||
"ok": true,
|
||||
"exit_code": 1,
|
||||
"wall_seconds": 2.32
|
||||
},
|
||||
{
|
||||
"name": "под нагрузкой: исправлены только внедрённые, ошибок нет",
|
||||
"ok": true,
|
||||
"fixed": {
|
||||
"orphan_stale": 5,
|
||||
"missing": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale": 5
|
||||
},
|
||||
"runs": 59
|
||||
},
|
||||
{
|
||||
"name": "под нагрузкой: все 201 готовы и совпадают по SHA-256, потерь 0",
|
||||
"ok": true,
|
||||
"uploads": {
|
||||
"201": 3364
|
||||
},
|
||||
"lost_ready": 0
|
||||
}
|
||||
],
|
||||
"ok": true
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
{
|
||||
"date": "2026-11-06T09:20:52Z",
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"mode": "dry-run",
|
||||
"skipped": false,
|
||||
"grace_seconds": 60,
|
||||
"skew_seconds": 0,
|
||||
"deep": true,
|
||||
"page_size": 1000,
|
||||
"scanned": {
|
||||
"objects": 24,
|
||||
"rows": 27
|
||||
},
|
||||
"found": {
|
||||
"orphan_stale": 3,
|
||||
"orphan_fresh": 2,
|
||||
"missing": 3,
|
||||
"size_mismatch": 3,
|
||||
"sha_mismatch": 3,
|
||||
"pending_stale": 6,
|
||||
"pending_fresh": 2,
|
||||
"quarantined": 0,
|
||||
"healed": 0
|
||||
},
|
||||
"fixed": {
|
||||
"orphan_stale": 0,
|
||||
"missing": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale": 0,
|
||||
"healed": 0
|
||||
},
|
||||
"rechecked": {},
|
||||
"errors": 0,
|
||||
"samples": {
|
||||
"missing": [
|
||||
"01a1b028-2293-743d-a00c-f7fec1a50808",
|
||||
"01a1b028-22ad-77fa-9b2e-d01e032a2c10",
|
||||
"01a1b028-2306-74ed-9bff-7c2894a33621"
|
||||
],
|
||||
"size_mismatch": [
|
||||
"01a1b028-22ba-778b-99d5-eef1d99349f1",
|
||||
"01a1b028-2316-7510-b41e-754b39583de9",
|
||||
"01a1b028-231e-70ba-8a9e-1de6d00c2e0d"
|
||||
],
|
||||
"sha_mismatch": [
|
||||
"01a1b028-22c2-7713-adf4-93a25c06e07e",
|
||||
"01a1b028-2327-7105-838f-3f0df5b10dc2",
|
||||
"01a1b028-2330-7662-8f98-6b45a5c165a2"
|
||||
],
|
||||
"pending_stale": [
|
||||
"2cc53ddd-063f-4edc-ae55-98f201965b7f",
|
||||
"73fc548a-79a1-4396-a848-2e598eba75a1",
|
||||
"848bed6c-437a-4169-8cca-940c1987055b",
|
||||
"99d824a9-d16d-481e-9804-e209f2fa410b",
|
||||
"a0a3b632-5fe3-447e-a93b-bbdce77c4fb1",
|
||||
"d86c4cba-2836-4150-bf0e-d32e1ec442ec"
|
||||
],
|
||||
"orphan_stale": [
|
||||
"attachments/35df74ac-5753-48e0-b106-d596db2ca8e6",
|
||||
"attachments/8ba5a522-a4b8-40a8-b1f3-274401976db6",
|
||||
"attachments/de3cc053-2a20-44ad-b223-e06818b039e7"
|
||||
],
|
||||
"orphan_fresh": [
|
||||
"attachments/7b96d477-2dc4-4a8c-8703-e6176a7e46a6",
|
||||
"attachments/d69d9843-ceab-4101-a278-7e5974071ac6"
|
||||
],
|
||||
"pending_fresh": [
|
||||
"dfb03912-2304-4280-bfb7-7a7b9cb28597",
|
||||
"f1cf4216-9e57-4217-866a-440aac9435f3"
|
||||
]
|
||||
},
|
||||
"cost": {
|
||||
"seconds": 0.04,
|
||||
"requests": {
|
||||
"s3_list": 1,
|
||||
"s3_head": 0,
|
||||
"s3_get": 13,
|
||||
"s3_delete": 0,
|
||||
"sql": 1
|
||||
},
|
||||
"waited_seconds": {
|
||||
"s3_list": 0.016,
|
||||
"sql": 0.006
|
||||
},
|
||||
"hashed_bytes": 1453621,
|
||||
"cpu_seconds": 0.594,
|
||||
"max_rss_mib": 107.4
|
||||
},
|
||||
"exit_code": 0,
|
||||
"wall_seconds": 1.013,
|
||||
"audit": []
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
{
|
||||
"date": "2026-11-06T09:20:59Z",
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"mode": "apply",
|
||||
"skipped": false,
|
||||
"grace_seconds": 60,
|
||||
"skew_seconds": 0,
|
||||
"deep": false,
|
||||
"page_size": 1000,
|
||||
"scanned": {
|
||||
"objects": 20,
|
||||
"rows": 21
|
||||
},
|
||||
"found": {
|
||||
"orphan_stale": 0,
|
||||
"orphan_fresh": 2,
|
||||
"missing": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale": 0,
|
||||
"pending_fresh": 2,
|
||||
"quarantined": 8,
|
||||
"healed": 1
|
||||
},
|
||||
"fixed": {
|
||||
"orphan_stale": 0,
|
||||
"missing": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale": 0,
|
||||
"healed": 1
|
||||
},
|
||||
"rechecked": {},
|
||||
"errors": 0,
|
||||
"samples": {
|
||||
"quarantined": [
|
||||
"01a1b028-22ba-778b-99d5-eef1d99349f1",
|
||||
"01a1b028-2306-74ed-9bff-7c2894a33621",
|
||||
"01a1b028-2316-7510-b41e-754b39583de9",
|
||||
"01a1b028-231e-70ba-8a9e-1de6d00c2e0d",
|
||||
"01a1b028-22ad-77fa-9b2e-d01e032a2c10",
|
||||
"01a1b028-2330-7662-8f98-6b45a5c165a2",
|
||||
"01a1b028-22c2-7713-adf4-93a25c06e07e",
|
||||
"01a1b028-2327-7105-838f-3f0df5b10dc2"
|
||||
],
|
||||
"orphan_fresh": [
|
||||
"attachments/7b96d477-2dc4-4a8c-8703-e6176a7e46a6",
|
||||
"attachments/d69d9843-ceab-4101-a278-7e5974071ac6"
|
||||
],
|
||||
"pending_fresh": [
|
||||
"dfb03912-2304-4280-bfb7-7a7b9cb28597",
|
||||
"f1cf4216-9e57-4217-866a-440aac9435f3"
|
||||
],
|
||||
"healed": [
|
||||
"01a1b028-2293-743d-a00c-f7fec1a50808"
|
||||
]
|
||||
},
|
||||
"cost": {
|
||||
"seconds": 0.04,
|
||||
"requests": {
|
||||
"s3_list": 1,
|
||||
"s3_head": 0,
|
||||
"s3_get": 5,
|
||||
"s3_delete": 0,
|
||||
"sql": 2
|
||||
},
|
||||
"waited_seconds": {
|
||||
"s3_list": 0.019,
|
||||
"sql": 0.006
|
||||
},
|
||||
"hashed_bytes": 559085,
|
||||
"cpu_seconds": 0.607,
|
||||
"max_rss_mib": 107.5
|
||||
},
|
||||
"exit_code": 0,
|
||||
"wall_seconds": 1.074,
|
||||
"audit": []
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
{
|
||||
"date": "2026-11-06T09:21:14Z",
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"mode": "apply",
|
||||
"skipped": true,
|
||||
"grace_seconds": 60,
|
||||
"skew_seconds": 0,
|
||||
"deep": false,
|
||||
"page_size": 1000,
|
||||
"scanned": {
|
||||
"objects": 0,
|
||||
"rows": 0
|
||||
},
|
||||
"found": {
|
||||
"orphan_stale": 0,
|
||||
"orphan_fresh": 0,
|
||||
"missing": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale": 0,
|
||||
"pending_fresh": 0,
|
||||
"quarantined": 0,
|
||||
"healed": 0
|
||||
},
|
||||
"fixed": {
|
||||
"orphan_stale": 0,
|
||||
"missing": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale": 0,
|
||||
"healed": 0
|
||||
},
|
||||
"rechecked": {},
|
||||
"errors": 0,
|
||||
"samples": {},
|
||||
"cost": {
|
||||
"seconds": 0.008,
|
||||
"requests": {
|
||||
"s3_list": 0,
|
||||
"s3_head": 0,
|
||||
"s3_get": 0,
|
||||
"s3_delete": 0,
|
||||
"sql": 0
|
||||
},
|
||||
"waited_seconds": {},
|
||||
"hashed_bytes": 0,
|
||||
"cpu_seconds": 0.549,
|
||||
"max_rss_mib": 106.5
|
||||
},
|
||||
"exit_code": 0,
|
||||
"wall_seconds": 0.917,
|
||||
"audit": []
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"01a1b028-1dde-7345-971e-c4c24622ef5d": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-2293-743d-a00c-f7fec1a50808": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-22ad-77fa-9b2e-d01e032a2c10": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-22ba-778b-99d5-eef1d99349f1": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-22c2-7713-adf4-93a25c06e07e": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-22cb-73b4-92c8-0db140de50a1": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-22d2-7761-8bdd-8b9987dcd70f": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-2306-74ed-9bff-7c2894a33621": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-2316-7510-b41e-754b39583de9": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-231e-70ba-8a9e-1de6d00c2e0d": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-2327-7105-838f-3f0df5b10dc2": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-2330-7662-8f98-6b45a5c165a2": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-2339-7091-b944-16c65cfce6f9": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-2341-737d-b71f-ed06fdf083e5": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-2349-7776-b1f8-5d8380dec796": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-2351-7605-96d5-b39a1ad9df0f": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-2359-72bc-add8-a22020ccf80a": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-2361-71b8-b4bd-4227e8de54bc": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a1b028-2369-7670-9824-9c1871789b60": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75"}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"01a1b028-1dde-7345-971e-c4c24622ef5d": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
|
||||
"01a1b028-22cb-73b4-92c8-0db140de50a1": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
|
||||
"01a1b028-22d2-7761-8bdd-8b9987dcd70f": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
|
||||
"01a1b028-2339-7091-b944-16c65cfce6f9": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
|
||||
"01a1b028-2341-737d-b71f-ed06fdf083e5": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
|
||||
"01a1b028-2349-7776-b1f8-5d8380dec796": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
|
||||
"01a1b028-2351-7605-96d5-b39a1ad9df0f": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
|
||||
"01a1b028-2359-72bc-add8-a22020ccf80a": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
|
||||
"01a1b028-2361-71b8-b4bd-4227e8de54bc": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
|
||||
"01a1b028-2369-7670-9824-9c1871789b60": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75"
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"date": "2026-11-06T09:21:32Z",
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"exit_code": 1,
|
||||
"wall_seconds": 2.32,
|
||||
"stdout": "",
|
||||
"log": [
|
||||
"{\"error\":\"EndpointConnectionError\",\"event\":\"reconcile.failed\",\"level\":\"error\",\"ts\":\"2026-11-06T09:21:25.233096Z\"}"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"date": "2026-11-06T09:21:22Z",
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"s3": "paused",
|
||||
"state_before_stop": "running",
|
||||
"init": "true",
|
||||
"stop_seconds": 0.1,
|
||||
"exit_code": 143,
|
||||
"next_pass_skipped": false
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
{
|
||||
"date": "2026-11-06T09:23:59Z",
|
||||
"commit": "75b93bbba5f29575d3fca1d013cc7e0770d88082",
|
||||
"seconds": 75,
|
||||
"upload_threads": 3,
|
||||
"uploads": {
|
||||
"201": 3364
|
||||
},
|
||||
"created": 3364,
|
||||
"linked_every": 10,
|
||||
"reconciler_runs": 59,
|
||||
"run_seconds": [
|
||||
0.055,
|
||||
0.048,
|
||||
0.056,
|
||||
0.057,
|
||||
0.063,
|
||||
0.067,
|
||||
0.078,
|
||||
0.082,
|
||||
0.094,
|
||||
0.093,
|
||||
0.097,
|
||||
0.102,
|
||||
0.116,
|
||||
0.144,
|
||||
0.125,
|
||||
0.131,
|
||||
0.127,
|
||||
0.15,
|
||||
0.137,
|
||||
0.155,
|
||||
0.163,
|
||||
0.161,
|
||||
0.168,
|
||||
0.183,
|
||||
0.171,
|
||||
0.177,
|
||||
0.176,
|
||||
0.195,
|
||||
0.208,
|
||||
0.197,
|
||||
0.212,
|
||||
0.217,
|
||||
0.221,
|
||||
0.222,
|
||||
0.246,
|
||||
0.236,
|
||||
0.248,
|
||||
0.247,
|
||||
0.257,
|
||||
0.277,
|
||||
0.262,
|
||||
0.3,
|
||||
0.305,
|
||||
0.289,
|
||||
0.299,
|
||||
0.297,
|
||||
0.302,
|
||||
0.358,
|
||||
0.325,
|
||||
0.325,
|
||||
0.333,
|
||||
0.352,
|
||||
0.334,
|
||||
0.346,
|
||||
0.344,
|
||||
0.346,
|
||||
0.384,
|
||||
0.38,
|
||||
0.406
|
||||
],
|
||||
"fixed_total": {
|
||||
"orphan_stale": 5,
|
||||
"missing": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale": 5
|
||||
},
|
||||
"fresh_seen_total": {
|
||||
"orphan_fresh": 175,
|
||||
"pending_fresh": 42
|
||||
},
|
||||
"rechecked_total": {
|
||||
"missing": 55
|
||||
},
|
||||
"errors": 0,
|
||||
"skipped": 0,
|
||||
"injected_before_load": {
|
||||
"orphan": 3,
|
||||
"pending_stale": 3
|
||||
},
|
||||
"left_from_faults": {
|
||||
"orphan_stale": 2,
|
||||
"pending_stale": 2
|
||||
},
|
||||
"check_after": {
|
||||
"objects": 3382,
|
||||
"rows": 3383,
|
||||
"ready_healthy": 3375,
|
||||
"counts": {
|
||||
"orphan_stale": 0,
|
||||
"orphan_fresh": 0,
|
||||
"missing_referenced": 0,
|
||||
"missing_unreferenced": 0,
|
||||
"size_mismatch": 0,
|
||||
"sha_mismatch": 0,
|
||||
"pending_stale_with_object": 0,
|
||||
"pending_stale_without_object": 0,
|
||||
"pending_fresh": 0,
|
||||
"quarantined": 8,
|
||||
"incomplete_stale": 0,
|
||||
"incomplete_fresh": 0,
|
||||
"lost_ready": 0
|
||||
},
|
||||
"ok": true
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user