Files
mtusi2026lab4/REPORT.md
T

15 KiB

????? ? ???????????? 4

  • ?????: ???????? ???? ????????, ???2303.
  • ??????? ??????: ?3?. ?????? 4/5 ? ????????????????? ????? ?? ??????????.
  • ?????: solution/BPI2303-Scherbakov; ??????: ??? 4 ? ???2303 ? ???????? ? ?? 3?. Hash: git log -1 --oneline. ????? ??? PR ? SUBMISSION.md; PR ????????? ????????.
  • ?????? ??: OpenAI Codex ? ???????? lab1?3, ?????????? lab4, ???????? ? ?????.
  • ??????: ??????????? ?????????? ??????; ????? ???????????? ??????? ?? ??????????????.

?????? ?? ??????? clone

????? Docker ? Linux-????????????, Compose v2 ? Python 3.12 ??? ????????. ??? ??????????? ??????????? ?? ????? ????????; lab1?3 ??? ??????? ?? ?????.

python scripts/bootstrap.py
docker compose --env-file .env config --quiet
docker compose --env-file .env up -d --build --wait
python scripts/check_contract.py
python tests/smoke.py --base-url http://localhost:8080

Bootstrap ??????? .env ? ????????? ??????? credentials ? secrets/, ?? ??????? ? ?? ?????????????? ???????????? ?????. ??? ????????? ?? Git ? Docker build context. postgres_password ? database_url ?????? PostgreSQL; s3_access_key_id, s3_secret_access_key ? s3_config.json ? ????????????? S3 identity. ? JSON ??? anonymous identity. ?? ?????????? secrets ?? lab3.

?? ??????????? Windows-????? ? .env ?????????? APP_PORT=18084; ????? ? http://127.0.0.1:18084. ??? ??????????????? ?????????? ???? ???? ????? ????????. ?? ????????? .env.example ?????? 8080. ??? ??????? HTTP proxy ???????? localhost ? 127.0.0.1 ? NO_PROXY. ?????? ???????? ???? ?????? NO_PROXY.

???????? ????????? ??????????? ?????? migrate ????? ?????????? PostgreSQL. ?????? s3-init ???? S3, ??????? bucket, ???? ??? ???, ? ????????????? private ACL. API ??????????? ????? ????????? ?????????? ????? jobs; Exited (0) ? ??? ? ?????. ????????? bootstrap bucket ?? ??????? ???????. ?? ?????? ???????????? credentials ??? ????????????? ????????????? ????????? ? API.

python scripts/verify.py --base-url http://127.0.0.1:18084 --output evidence/grade3-checks.txt

??? ??????? ????????? ????????? smoke, ??????????? ?????, ?????? ????????, ???????? restart ???? ????????, ????????? ??????????? ???? API (?? ??????? ?????), ????????? S3 GET, ????? S3 ? ????????? failover-???????? ????? API. ??? ??????? ????????????? ?????? ? ???????? ????????????? ??????? ?????? ??????? mtusi2026lab4; volumes ?? ?????????. ????????? ?????? ? Docker.

????????? ???????? ? PowerShell:

$env:BASE_URL = 'http://127.0.0.1:18084'
python -m unittest discover -s student-tests -v
python tests/resilience.py --mode persistence --action-script scripts/restart.cmd --base-url $env:BASE_URL
python tests/resilience.py --mode failover --action-script scripts/stop-one.cmd --base-url $env:BASE_URL
docker compose --env-file .env up -d --wait api1 api2

Linux/macOS: chmod +x scripts/restart.sh scripts/stop-one.sh ? ???????????? .sh ?????? .cmd. ? GNU Make: make check test test-student PYTHON=python BASE_URL=http://127.0.0.1:18084; ??? persistence/failover ???????? ??????????????? ACTION_SCRIPT ???????? tests/README.md. ?? Windows ???????? ????????? ??????? Python-?????????, ?????????????? Make targets.

??????? ?????????: docker compose --env-file .env down ? volumes ???????????. ?????????? ????? ???? ??????? ??????: docker compose --env-file .env down -v. ??? ??????? ? hooks ? ????????? ?? ????????????.

??????????? ? ??????

flowchart LR
    C[???????] --> LB[HAProxy / localhost:APP_PORT]
    LB --> A1[api1]
    LB --> A2[api2]
    A1 --> PG[(PostgreSQL: metadata / sessions / message keys)]
    A2 --> PG
    A1 --> S3[(SeaweedFS: private originals)]
    A2 --> S3
    PG --> PV[postgres_data]
    S3 --> SV[s3_data]
???? / service ?????? Readiness / bootstrap ???????? / ????
api1, api2 Python 3.12.12, ??????????? Dockerfile /health/ready: PostgreSQL + schema 003 + S3 HeadBucket read-only root, ??? data volume ? ?????????????? ??????
postgres postgres:16.10-bookworm pg_isready; ????????? migrate postgres_data, ?????? ?????????? ????
s3 chrislusf/seaweedfs:3.85 s3-init ???? ??????????? ListBuckets; API ????????? HeadBucket s3_data ?? /data, S3 :8333, ????? ?????? ?? ???????????
lb haproxy:3.0.11-alpine HTTP checks API ?????? ???????, fall=2/rise=1 ?????? 127.0.0.1:APP_PORT ? :8080

API: 1 CPU / 512 MiB ?? ???????; PostgreSQL: 1 CPU / 256 MiB; S3: 1 CPU / 512 MiB; LB: 0.5 CPU / 128 MiB. ???????? ? bucket job: 0.5 CPU / 128 MiB. ?????? API-????? ?? ????????????. Docker secrets ??????????? ? /run/secrets; ???????? ?? ?????? ? ????? ? HTTP-??????.

Runtime-??????????? ?????????? ? requirements.txt: psycopg 3.2.6, psycopg-pool 3.2.6, typing_extensions 4.16.0, boto3 1.40.61, Pillow 12.0.0. ??????? ??????????? ???????????? ?????? ??????????? ? ?????????/??????????; ??????? lockfile ? hashes ???. ?????? MinIO ? ??????????? ???????? ????????? ?????????? ??? ??????????; ?????? ??????????? ???????? SeaweedFS.

SeaweedFS server -filer -s3 ????????? master, volume server, filer ? S3 gateway. ??????? ??????? ?????? ? /data; ?? ??????? ? persistent volume, ??????? ?????????? filer ? ?????? volumes. ?????? volume ????????? 100 MiB, ???????? 8 volumes ? ??????? ????????? ???????, ??? ?????????? ?? ?????? host.

??? ?????????? ? ????????

?? lab1 ???????????? HTTP transport, ??????? ?????????, ??????, keyset-????????? ? ?????????? ?? SIGTERM. Lab2 ???????? ????????? ? PostgreSQL, ???????? scrypt ? ????? Bearer-??????. ?? lab3 ?????????? ??? ?????????, ???????????? ???????? 001/002, ??????????? ?????, ??? API ?? HAProxy ? message_keys. ?????????? ???????? ?? ??????????. README ???????, COURSE, contracts ? ????????? tests ??????? lab4 ?????????; ???????? skeleton ?????? ??? ?? ????????????.

????? ???????? 003 ??????? Attachment ? FK ?? ??? ? ?????????-?????????, ????????, MIME, SHA-256, ????????? object_key ? UTC timestamps. ???????? ???????? ? PostgreSQL ???. messages.attachment_ids ? JSONB-?????? ?? ?????? ?????????? UUID; ??? ?????? ??????????? ? ?????????? ????????, ????????? ???????????. ??????? ?????????? attachment_ids, ? ??? ????? [] ??? ?????? ????????? ?????????.

????? ? ????????????? attachment_ids ? ? [] ???????????? ?????. ??????????????? ?????????? text ? ????????????? ?????? ??????????????? UUID. ?????????? ?????? ???? ? PostgreSQL ??????????? ????????????? ???????? ???? API; ?????? ????????? ? ?????????? ????? ?????????? ????????. ???? ????????? 24 ????. ?????? tokens, HMAC cursor key ? ????????? ????????? ???????? ?????????? restart.

?????????? ????????:

  1. Bearer-?????? ? ???????? ? ???? ??????????? ?? ?????? ????.
  2. filename ? 1?255 Unicode code points; slash, backslash, NUL, ??????????? ???????, surrogate, ??????/?????????? ??? ???????????. ??? ?????? ???????????? ???.
  3. ????????? Content-Length, Transfer-Encoding ?? ???????????. ?? ?????? ??????????? ???????? 10 MiB; ???????? ????? ??????????? ????? ??????.
  4. Pillow ????????? ?????? PNG/JPEG, ?????????? ????????? MIME ? Content-Type, ????????? ?????? ?? ????????? ?? load(), ????? verify() ? ?????? ????????????? ??? ???????????? ???????????? ??????. ?????? ? 40 ??? ???????? ????????????.
  5. S3 PUT ?? ?????????? originals/<uuid> ? INSERT ?????????? ? COMMIT ? 201 ready. variants=[], error_code=null. ???????? ?? ??????????????.

?? API ??? ?????????? ????? ?????????. ???? ????????????? decode ?? ??????? ????????? semaphore, ????????? ???????? ???????? 503 ????? 1 ??????? ????????. ???? ???????? 10 MiB ???????? ? RAM; ??? ???????????? ???????????, ?? ????????? ????????? ?????? 4. ????? ???????: 10 s, S3 connect/read: 1/3 s, ??? retry SDK; LB client/server: 30/20 s. ??????????? byte/pixel ????? ?????? ????????? env. ?? ?????????? ???? ?????? ???????????? uploads ? ?? ?????????? ????? ?????? ?? DoS.

Download: Bearer ? membership query ? S3 GET ? ???????????? ?????? ? ???????? size/SHA-256 ? bytes ? ???????? Content-Type, X-Request-Id ? X-Instance-Id. ????????? ? ?????????? S3 URL ???. ??????????? ???????? ?????????? 404 ??? metadata ? content. ??????????? ??????? ???? 404. Metadata ?? ?????????? object_key.

/health/live ?? ?????????? ? ??????????. /health/ready ??????? PostgreSQL, ???????? 003 ? S3, ????????? ???????? ? ???????????? ?????????? ????. ??? ?????? S3 LB ??????? ??? ??????? ?? ????????????; ????????? ???????? ? metadata ???? ?? ???? ????? ?? ???????? ? PostgreSQL, ?? ???????? ????? readiness ????????? ??????.

????????? .env.example ???????????? Compose: APP_PORT ? ??????? ???? LB, APP_ENV/SESSION_TTL_SECONDS ? ?????/??????, S3_ENDPOINT/REGION/BUCKET ? S3 client ? bootstrap, MAX_UPLOAD_BYTES/MAX_IMAGE_PIXELS ? ??????????? API. ?????????? APP_HOST=0.0.0.0, APP_PORT=8080 ? ???? ? secret-?????? ?????? ??????????????? ? compose.yaml. API_REPLICAS ?? ?????????: ???? ????????? api1/api2.

?????????????? ?????????

?????? ???????? ????? ???????: evidence/grade3-checks.txt. ????????? ??????, smoke, 23 ??????????? ????? ? ?????? ????????????: evidence/final-checks.txt. ?????? ?????? ?????? ???????? 21 ????; ????? ????????? ??? ????????? ????????, ??? 23 ????? ????????? ?? ????????? ??????. ???????? ? ????????? tests ??????? ? Git. ?????? ?????????????? OpenAPI-????????? ?? ???????????: ?? host ??????????? openapi-spec-validator. ??? ???: python -m pip install -r requirements-dev.txt, ????? python scripts/validate_openapi.py. ???????????? check_contract ??????. ???? ?????????: 13 ?????? 2026. Windows-????, Docker Desktop 4.77.0, Engine 29.5.3; Docker VM: 8 CPU, 2 967 707 648 ???? RAM. ??? ??????? VM, ?????? ??????????? CPU ? ??????? ?????????????????? ?? ?????????????. ?????? Docker, ??????? VM, instance aliases, ??????????, StartedAt, mounts ? ?????????? ???????????? ? ???????? ?????????????.

???????? ?????? 3 ??????? / ???????? ??????????
?????????? ??????? ? ????????? smoke scripts/check_contract.py; tests/smoke.py; student-tests ??????????? ???????? ? ??????????? ???????? API/??????/?????????/???????????????
S3 ? ????? ??????, ?????????? ???????? compose config; state() ? verify.py; ???????? 003 PostgreSQL ? SeaweedFS ? ?????????? volumes; API ??? data volume
201 ready, ???????? ???????? smoke; test_original_and_message_shared_with_member; test_jpeg_original_and_truncation PNG/JPEG, ?????? ? SHA-256 ?????????; attachment-only ????????? ?????? ?????? ????????
????????? bucket ? ACL ???? verify_upload_failover; test_acl_before_image_validation ?????? unsigned GET ????????????? S3-??????? ? 403; ??????????? ? API ? 404
Byte/pixel/MIME/filename limits test_byte_limit_before_reading_body; test_image_validation; test_filename 413 ?? ???????? oversized ????; 415 ??? ???????/?????; 400 ??? ???????? ?????
????????? ?????????? ??????? test_maximum_byte_size_accepted; test_exactly_40_million_pixels_accepted ??????????? 10 MiB ???????????? ? ???????? PNG 8000?5000
????????? ??????????? API verify_upload_failover, ??? API ?? ??????? ??????????????? alias ?? ?????? upload; download ?????? ???????? ???????? ?????????
Restart ???? stores ? API ????????? persistence ? restart.cmd/.sh + inspect ??/????? StartedAt ???????? ? s3/postgres/api1/api2/lb, volumes ???????????; ??????? token ? ???? ????????
??????? failover lab3 ????????? failover ??? ????? API, inspect exited ?????????????? ????????? ? ???? ?????????, retry ??? ??????, ?????? ????????????????? ?? ?????? ?????
????????????? S3 verify_s3_failure ????? ? API: live=200, ready=503, metadata=200, upload/download=503; ????? metadata ?? ???????????

???????? ??????????? ???????: ????????? persistence ? failover ????? API, ????????? S3 GET=403, ?????????????? ????? S3, 23 ??????????? ?????. ? ?????? ???????: ????????? api1 ? 97 ???????? ???????, 0 ????????? POST, first_recovery=3.844 s; ????????? api2 ? 96 ???????, 0 ????????? POST, first_recovery=4.750 s. ???????????? ??????: 148/145 ????????, ?? ????? ?????????? ??????. first_recovery ???????? ?????? Docker hook, ??? ?? ???????????? ??????? ???????. ?????????? ????????? ? ????? ?????????? ????????????.

??????, ??????????????? ? ???????????

??????? migration jobs ????????? ?? checksum ??? advisory lock; ????????? ?????? ?? ?????? ?????????? ?????. ??? restart ????????? ???????? ????????? ??????? ??????, ????????? ? ????; ?????????????? ???????? ????????? cursor ? ????????? ?????????????? POST. ???????? inspect ???????????? ???????? ?????????? ?????????.

??????????? ????????? ????????? ??????????? ??????????
?????????? ??????????? API ???? ???????? ?????? ??????? ??? UPLOAD_FAILOVER ? ??????? stopped/download aliases ? ?????????? SHA-256
?????????? ???? API ?? ????? POST ?????? ?????????? ??????; retry ??? ?????? ????????? failover: successes, failures, first_recovery_seconds ? ?????????
????????????? ???? ????? ????????? volumes, ?????? ? ?????? BEFORE_RESTART/AFTER_RESTART, persistence PASS
?????????? S3 ?????????????? 503, ??? ??????? 201 S3 outage PASS, ?????????? ????? attachments; ????? start ???????? ?????????
GET ?????? ????? ?????? ????????????? ??????? 404 ??? metadata/content, 401 ??? ??????

? PostgreSQL ? S3 ??? ????? ??????????. ???? PUT ???????, ? INSERT/COMMIT ?? ???????????, ?????? ?? ???????? 201; ???????? orphan object. ??? ?????????????? ?????????? COMMIT ?????? ?? ?????????, ????? ?? ????????? ??? ??????????????? ??????. ?????????????? reconciler/??????????? ? grace period ?? ??????????? ? ????????? ? ?????? 4/5. ??? ?????? S3 ?? ????? GET/PUT API ?????????? 503; ??????? ?????? ?????? ?? ??????????? ??? ???????? ????????? ???????????????.

??????? ?????????: ???? S3 identity ? ????????????????? ??????? ???????????? API ? bootstrap. ??????? ?? ???????????, anonymous identity ???????????; ????????? ???????????? credentials API ????????? ? ?????? 4. ???? Compose ????????? ??????????, ?????????? HTTP-?????????? SeaweedFS ???????? ???????? ???????????. ?????? ??? ?? ????????????. ??? ?? ?????? ?? ????????????? ?????????? ?????? ????.

???? Docker-host, LB, PostgreSQL ? SeaweedFS ???????? ??????? ??????? ??????. ??? ?????????? S3/??, backup, TLS, workers, presigned upload, thumbnail, ?????? ?? ?????? volumes/host ? ???????? ??????????? ??? ?????? stores. ????????? ????????? ????????? API ? ??????????? ??? ?????? restart; ??? ?? ?????????? ???????????? HA. ??????????? ????????????, RPS ? p95/p99 ??? ?????? 3 ?? ??????????. ?????/???????????? ? ????????? ?????????????.

?????????