Files
mtusi2026lab3/evidence/tests/test-isolated.md
T
ricoromaandClaude Opus 5.5 c5840bfaa8 docs: add idempotency and X-Instance-Id evidence to the report
- evidence for replay, conflict, replay through a new API process and
  after restarting API and storages, keys table, key race,
  X-Instance-Id in responses and logs, explicit and invalid
  instance_id and key TTL, test runs
- stand evidence: make wipe_data leaves the lab 2 volume intact, build
  from a fresh git clone passes smoke, bench restore stage sends with
  Idempotency-Key
- REPORT.md: X-Instance-Id, idempotency design, IDEMPOTENCY_TTL_SECONDS
  mapping, limitations and evidence rows
- tasks.md: mark the change tasks done, add follow-ups from verify

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:08:47 +03:00

89 lines
5.5 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# `make test_isolated`: smoke lab3 и свой набор против контейнера
| | |
|---|---|
| Спеки | `student-tests` «Прогон на временной базе» — «Успешный прогон»; «Покрытие сценариев основных спеков» — «Сверка покрытия» |
| Change | `instance-id-idempotency`, задача 9.1 |
| Дата | 2026-10-08 16:46:03 UTC, 21 с |
| Commit | `af4bca0` |
| Стенд | образ `mtusi-messenger:lab3` из рабочего дерева на `af4bca0`; временная база `test_messenger` в Postgres стенда `mtusi-lab3`, Valkey стенда; отдельный контейнер `mtusi-lab3-test-api` на `127.0.0.1:18088`; один процесс API, без балансировщика |
| Рабочий стенд | `make run_docker` запущен (API на `127.0.0.1:8080`), во время прогона не трогался |
| Хост | Docker Desktop 29.8.1, macOS arm64; ВМ Docker: 12 CPU, 7,7 GiB RAM; лимиты контейнерам не задавались |
## Прогон
```text
$ make test_isolated # exit 0, 21 с
[test_isolated] database=test_messenger container=mtusi-lab3-test-api base_url=http://127.0.0.1:18088
[test_isolated] build image
[test_isolated] create database test_messenger
[test_isolated] migrations
Running upgrade -> 16c279725564, added messenger tables
Running upgrade 16c279725564 -> 42bc3b9be38f, added sessions and password hashes
Running upgrade 42bc3b9be38f -> 7652184b7016, added idempotency keys
[test_isolated] start api container
[test_isolated] ready
[test_isolated] make test
PASS: lab3 public API scenario; infrastructure and grading evidence remain separate.
[test_isolated] make student_test
collecting ... collected 142 items / 15 deselected / 127 selected
...
coverage: 133 scenarios in 11 specs, 121 tested (16 by make failure_checks), 12 excluded
race: 10 clients x 10 messages, statuses {201: 100}, max in flight 10; history: 100 messages, 100 unique ids, 4 pages by 30
username race: 10 clients, statuses {409: 9, 201: 1}, max in flight 10
key race: 10 clients, statuses {201: 10}, max in flight 10; history: 1 messages
===================== 127 passed, 15 deselected in 13.25s ======================
[test_isolated] cleanup
[test_isolated] results: {'test': 0, 'student_test': 0}
```
Публичный smoke лабы 3 (`tests/smoke.py`, профиль lab3) проверяет
`X-Instance-Id` у каждого ответа, повтор с тем же ключом (тот же JSON),
409 на другое тело и 400 без ключа. 15 тестов `fault` собраны, но
выполняет их только `make failure_checks` (`failure-checks.md`).
## Тесты лабы 3 в наборе
```text
student-tests/test_api_conventions.py::test_instance_id_on_success_and_error PASSED
student-tests/test_api_conventions.py::test_instance_id_on_service_responses PASSED
student-tests/test_api_conventions.py::test_one_instance_id_per_process PASSED
student-tests/test_contract.py::test_bad_instance_id_fails[None] PASSED
student-tests/test_contract.py::test_bad_instance_id_fails[] PASSED
student-tests/test_contract.py::test_bad_instance_id_fails[api 1] PASSED
student-tests/test_contract.py::test_bad_instance_id_fails[aaaa…(65)] PASSED
student-tests/test_idempotency.py::test_no_key PASSED
student-tests/test_idempotency.py::test_bad_key PASSED
student-tests/test_idempotency.py::test_longest_key PASSED
student-tests/test_idempotency.py::test_key_on_other_requests PASSED
student-tests/test_idempotency.py::test_replay_same_body PASSED
student-tests/test_idempotency.py::test_replay_other_json_spelling PASSED
student-tests/test_idempotency.py::test_other_body_conflict PASSED
student-tests/test_idempotency.py::test_same_key_other_user PASSED
student-tests/test_idempotency.py::test_same_key_other_chat PASSED
student-tests/test_idempotency.py::test_replay_with_revoked_token PASSED
student-tests/test_idempotency.py::test_replay_with_new_session PASSED
student-tests/test_idempotency.py::test_outsider_with_members_key PASSED
student-tests/test_idempotency.py::test_no_session_and_no_key PASSED
student-tests/test_idempotency.py::test_outsider_without_key PASSED
student-tests/test_idempotency.py::test_invalid_body_with_used_key PASSED
student-tests/test_idempotency.py::test_invalid_request_keeps_key_free PASSED
student-tests/test_race.py::test_same_key_at_once PASSED
```
Кроме них в наборе 103 теста лаб 1–2: все отправки сообщений в них идут
с ключом (`Api.post_message`), каждый ответ через `contract.py`
проверяется и на `X-Instance-Id`.
Ожидания тестов идемпотентности проверены и на ошибочном коде (задача
7.2, против консольного сервиса, мутации откачены и в коммит не
попали):
| Мутация сервиса | Упали |
|---|---|
| ключ всегда считается свободным | `test_replay_same_body`, `test_replay_other_json_spelling`, `test_other_body_conflict`, `test_replay_with_new_session`, `test_outsider_with_members_key` |
| хэш тела не сравнивается | `test_other_body_conflict` |
Тест «Одно значение на процесс» верен только против одного процесса,
как здесь; стенд с балансировщиком — change `replicas-lb`.