ЛР 4 - БПИ2303 - Кимпаев А.Р. - на 3 #103

Closed
kimpaev wants to merge 1 commits from kimpaev/kimpaev-mtusi2026lab4:solution/bpi2303-kimpaev into main
44 changed files with 1333 additions and 23 deletions
+10
View File
@@ -0,0 +1,10 @@
FROM python:3.12.9-slim
WORKDIR /srv
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY app ./app
COPY migrations ./migrations
EXPOSE 8080
STOPSIGNAL SIGTERM
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080", "--timeout-graceful-shutdown", "10"]
+39 -23
View File
@@ -1,37 +1,53 @@
# Отчёт — лабораторная 4
# Отчёт — лабораторная работа №4. S3-хранилище и вложения
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
**Студент:** Кимпаев А. Р.
**Группа:** БПИ2303
**Целевая оценка:** 3 (полный минимум; дополнительные негативные тесты не являются заявкой на 4)
**Ветка:** `solution/bpi2303-kimpaev`
**Источники:** README.md, COURSE.md, contracts/README.md, contracts/openapi.json, tests/README.md из skeleton ЛР4; исходники собственной ЛР3. Использована помощь ИИ для переноса кода, реализации, документации и проведения проверок. Журналы фактических запусков приложены.
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
ОС: Windows 11 с Docker Desktop (Linux containers) либо Linux/WSL2. Требуются Docker Engine + Compose v2, Python 3.9+ для тестов. В Windows запустите `.\start-windows.cmd` из PowerShell. Ниже показан эквивалент для Linux / Git Bash. Приложение — Python 3.12.9, FastAPI 0.115.12, psycopg 3.2.6, boto3 1.37.38, Pillow 11.2.1. Образы: postgres:16.8-alpine, nginx:1.27.5-alpine, MinIO RELEASE.2026-11-09T00-10-40Z и mc DEVELOPMENT.GOGET из Chainguard. Оба образа S3 закреплены по digest в `compose.yaml`.
## Архитектура и решения
```bash
sh scripts/setup.sh
make check # или python3 scripts/check_contract.py
# Проверить конфигурацию до запуска
docker compose --env-file .env config --quiet
docker compose --env-file .env up -d --build
docker compose --env-file .env ps -a
make test # или python3 tests/smoke.py --base-url http://localhost:8080
python3 student-tests/attachments_negative.py --base-url http://localhost:8080
```
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab3, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
`setup.sh` создаёт `.env` и индивидуальные DB/S3 credentials в игнорируемом Git каталоге `secrets/`. Секреты не передаются через командную строку API. `migrate` выполняет SQL-миграции один раз, под PostgreSQL advisory lock. `s3-bootstrap` создаёт закрытый bucket и пользователя S3 с ограниченной политикой. Ожидаем завершения обоих job с кодом 0 и доступности `/health/ready` через LB. `.env.example` описывает логические параметры; фактически используются `SESSION_TTL_SECONDS`, `DATABASE_URL_FILE`, `S3_ENDPOINT`, `S3_REGION`, `S3_BUCKET`, `S3_ACCESS_KEY_ID_FILE`, `S3_SECRET_ACCESS_KEY_FILE`. Остальные указанные в `.env.example` — справочные интерфейсные переменные skeleton; лимиты задаются в коде (`MAX_BYTES`, `MAX_PIXELS`).
## Доказательства критериев
Безопасно остановить: `docker compose --env-file .env down` — данные сохранены. Намеренно удалить тестовую базу и файлы: `docker compose --env-file .env down -v` — **необратимо** для локальных volumes.
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
## Архитектура
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
```text
HTTP client → Nginx :8080 → FastAPI api1 / api2
├── PostgreSQL (users, sessions, chats, messages,
│ idempotency, attachment metadata/link)
└── private MinIO S3 bucket (originals/<random UUID>)
```
## Эксперименты и отказы
Из ЛР3 перенесены все endpoints пользователей, Basic→Bearer session, чаты, история с keyset cursor, X-Request-Id/X-Instance-Id, 24h+ идемпотентность отправки сообщения, PostgreSQL, Nginx и две API-реплики, сценарии restart/failover. Сохранены прежние текстовые POST; `attachment_ids` необязателен. В `messages` разрешён `text=""` **только при наличии вложения** (проверка Pydantic + запись ссылок в одной транзакции).
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
Новая миграция `003_attachments.sql` создаёт `attachments` и `message_attachments`. Сообщения, ссылки и ключ идемпотентности фиксируются атомарно в PostgreSQL. Метаданные `Attachment` включают ID, chat/owner, filename, MIME, размер, SHA-256, status=`ready`, timestamps. S3 key сервер генерирует случайно и никогда не возвращает в публичном API. Политика S3 разрешает приложению только bucket list и операции с префиксом `originals/*`; административный доступ остаётся только у bootstrap. S3/DB используют named volumes. Порты S3 и административной консоли не опубликованы на хосте.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
Upload: авторизация участника → проверка имени/MIME/Content-Length → инкрементальное чтение ≤10 MiB в disk-backed spooled tempfile → Pillow проверяет реальный формат, декодируемость и ≤40 млн пикселей → S3 PUT → COMMIT метаданных → ответ 201. При исключении после PUT выполняется best-effort DELETE из S3. Нет отдельных временно опубликованных локальных файлов, оригинал в API хранится только в spool на время HTTP-запроса. Download: проверка доступа к чату по PostgreSQL → GET из S3 → ответ StreamingResponse с оригинальными bytes, без redirect; чужие ID маскируются 404. Все API-реплики используют общий bucket.
## Ограничения и следующие шаги
**Транзакционная граница:** S3 PUT и COMMIT БД не являются общей ACID-транзакцией. При неудачном DB COMMIT запрашивается компенсационное удаление; если в этот момент S3 также недоступен, возможен orphan. Для оценки 4 необходим отдельный reconciler с grace period; здесь он не заявлен. `ready` возвращается только после успешного commit.
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
## Ограничения и дальнейшие шаги
- Стенд учебный: HTTP привязан к localhost, для публичного развёртывания нужен HTTPS/TLS; single-node PostgreSQL, S3 и LB не являются полностью HA.
- Проверка повреждённых изображений декодирует уже ограниченные по bytes/pixels данные, но для оценки 4 следует замерить фактический пик RAM под 10+ конкурентными upload.
- Очистка orphan после неуспешной компенсации S3 и расширенные fault-injection тесты не реализованы; это задачи уровня 4.
- БД enforce-ит ссылочную целостность вложений; условие «пустой текст только с вложениями» проверяется на уровне API, а не SQL CHECK.
- Повторение проверок на Windows и Linux описано в `STUDENT_RUN.md`.
+77
View File
@@ -0,0 +1,77 @@
# ЛР4 — команды запуска (Windows / Linux)
Автор: **Кимпаев А. Р.**, группа **БПИ2303**, fork: **kimpaev**, ветка: **solution/bpi2303-kimpaev**.
## Windows 11
Запустите Docker Desktop в режиме Linux containers. Распакуйте архив в новый каталог. Для чистого запуска достаточно Docker Desktop и штатного PowerShell. Из корня проекта выполните в PowerShell:
```powershell
.\start-windows.cmd
```
Сценарий создаёт локальные секреты, собирает и запускает стенд, ждёт готовности и запрашивает `/health/ready`. Ожидается `{"status":"ok"}`, обе API-реплики в состоянии `healthy`, а `migrate` и `s3-bootstrap` завершены с кодом 0. При ошибке сценарий выводит состояние Compose и логи сервисов. Для интеграционных тестов установите Python 3.9+:
```powershell
py -3 -X utf8 scripts/check_contract.py
py -3 tests/smoke.py --base-url http://127.0.0.1:8080
py -3 student-tests/attachments_negative.py --base-url http://127.0.0.1:8080
```
Если у вас есть прежние данные ЛР4, сохраните прежние `.env` и каталог `secrets`. Compose связывает тома с именем проекта. Для чистого стенда при наличии прежнего проекта с тем же именем задайте перед запуском `$env:COMPOSE_PROJECT_NAME = 'mtusi2026lab4_fixed'`. Старые контейнеры, занимающие порт 8080, нужно остановить в Docker Desktop.
## Git Bash / Linux
```bash
sh scripts/setup.sh
python3 scripts/check_contract.py
docker compose --env-file .env up -d --build --wait --wait-timeout 180
python3 tests/smoke.py --base-url http://127.0.0.1:8080
python3 student-tests/attachments_negative.py --base-url http://127.0.0.1:8080
```
MinIO и mc берутся из публичного реестра Chainguard (`cgr.dev`).
## Отказы и сохранность
В Windows запустите полный прогон из корня проекта:
```powershell
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\verify-windows.ps1
```
Сценарий запускает стенд, выполняет smoke, persistence и failover, после failover возвращает обе реплики в состояние healthy. Журнал сохраняется в `evidence/windows-run-*.txt`. Для отдельных проверок в Linux / WSL:
```bash
make test
make test-persistence ACTION_SCRIPT=scripts/restart.sh
make test-failover ACTION_SCRIPT=scripts/stop-one.sh
docker compose --env-file .env up -d --no-deps --wait api1 api2
```
В persistence сначала перезапускаются PostgreSQL, S3 и обе реплики, затем после их готовности перезапускается Nginx. Тест failover оставляет выбранную реплику остановленной, поэтому последняя команда возвращает её в работу.
Проверить непубличность S3: сервис `s3` не публикует порт 9000 на хост. Из контейнера временного клиента выполнить анонимный GET к известному объекту и убедиться, что доступ запрещён. Не используйте административные credentials при этой проверке. Для демонстрации ограниченной политики проверьте отдельным сервисным пользователем чтение objects из назначенного bucket и отказ в административных операциях.
Сохранность данных: `docker compose --env-file .env down` не удаляет volumes. **Безвозвратное удаление только тестовых данных**: `docker compose --env-file .env down -v` — удаляет базу и S3-объекты.
## PR по CONTRIBUTING.md
1. Fork **репозитория ЛР4**, clone своего fork (не ЛР3) и переключитесь в новую ветку от skeleton:
```bash
git checkout -b solution/bpi2303-kimpaev
```
2. Скопируйте содержимое этого решения **поверх своего клона ЛР4**, без `.git`, `.env`, `secrets/`, volumes. Не заменяйте публичные `tests/`, `contracts/`, `README.md`, `COURSE.md`.
3. После реальных проверок занесите их вывод в `REPORT.md` (не выдавайте непроверенные тесты за выполненные).
4. Создайте коммит и отправьте ветку:
```bash
git add app migrations student-tests infra scripts Dockerfile requirements.txt compose.yaml REPORT.md STUDENT_RUN.md WINDOWS11_FIX.md start-windows.cmd verify-windows.ps1 .gitattributes .github/pull_request_template.md evidence/offline-check.txt evidence/windows-verified.txt evidence/additional-checks.txt
git commit -m "Implement lab4 S3 attachments, preserve lab3 API"
git push -u origin solution/bpi2303-kimpaev
```
5. Откройте PR: **`ЛР 4 — БПИ2303 — Кимпаев — на 3`**. Base: ветка задания ЛР4 преподавателя; compare: `kimpaev:solution/bpi2303-kimpaev`. Текст PR из `.github/pull_request_template.md`; заполните фактические результаты.
Если вы собираете проект из готового архива, он уже содержит **skeleton ЛР4** и корректно перенесённые файлы ЛР3. Не создавайте PR в репозиторий ЛР3.
+40
View File
@@ -0,0 +1,40 @@
name: Lab4 application
on: [push, pull_request]
permissions:
contents: read
jobs:
integration:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: '3.12'
- name: Prepare test credentials
run: sh scripts/setup.sh
- name: Build and start the stack
run: docker compose --env-file .env up -d --build --wait --wait-timeout 180
- name: Check API readiness
run: curl --fail --show-error http://127.0.0.1:8080/health/ready
- name: Run smoke test
run: make test PYTHON=python BASE_URL=http://127.0.0.1:8080
- name: Print diagnostics on failure
if: failure()
run: |
docker compose --env-file .env ps -a
docker compose --env-file .env logs --tail=80
- name: Stop the CI stack
if: always()
run: docker compose --env-file .env down
View File
+26
View File
@@ -0,0 +1,26 @@
import hashlib
from fastapi import Header, HTTPException
from app.storage import db
BEARER_CHALLENGE = {"WWW-Authenticate": 'Bearer realm="messenger"'}
BASIC_CHALLENGE = {"WWW-Authenticate": 'Basic realm="messenger", charset="UTF-8"'}
def unauthorized(basic=False):
raise HTTPException(status_code=401, detail="Unauthorized", headers=BASIC_CHALLENGE if basic else BEARER_CHALLENGE)
def current_user(authorization: str | None = Header(default=None)):
if not authorization or not authorization.startswith("Bearer "):
unauthorized()
token = authorization[7:]
if not token or " " in token:
unauthorized()
with db() as conn:
user = conn.execute("""SELECT u.id,u.username,u.display_name,u.created_at FROM sessions s
JOIN users u ON u.id=s.user_id
WHERE s.token_hash=%s AND s.expires_at > now()""",
(hashlib.sha256(token.encode()).hexdigest(),)).fetchone()
if user is None:
unauthorized()
return user
+69
View File
@@ -0,0 +1,69 @@
"""Replicated API. Schema migration is a separate one-shot Compose service."""
import os
from uuid import uuid4
from fastapi import FastAPI, Request, HTTPException
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
from psycopg import OperationalError
from app.routers import users, chats, messages, attachments
from app.s3 import s3_client,bucket
from botocore.exceptions import BotoCoreError, ClientError
from app.storage import db
INSTANCE_ID = os.environ.get('API_INSTANCE_ID') or str(uuid4())
app = FastAPI()
def error_code(status):
return {400:'invalid_request',401:'unauthorized',404:'not_found',409:'conflict',413:'payload_too_large',415:'unsupported_media_type',429:'rate_limited',503:'unavailable'}.get(status,'internal_error')
def error_response(request,status,message,headers=None):
result=JSONResponse(status_code=status,content={'error':{
'code':error_code(status),'message':message,'request_id':request.state.request_id}},headers=headers)
result.headers['X-Request-Id']=request.state.request_id
result.headers['X-Instance-Id']=INSTANCE_ID
return result
@app.middleware('http')
async def add_request_id(request:Request,call_next):
request.state.request_id=str(uuid4())
try:
response=await call_next(request)
except OperationalError:
response=error_response(request,503,'Database unavailable')
except Exception:
response=error_response(request,500,'Internal server error')
response.headers['X-Request-Id']=request.state.request_id
response.headers['X-Instance-Id']=INSTANCE_ID
return response
@app.exception_handler(HTTPException)
async def http_error(request,exc):
return error_response(request,exc.status_code,str(exc.detail),exc.headers)
@app.exception_handler(RequestValidationError)
async def validation_error(request,exc):
return error_response(request,400,'Invalid request')
app.include_router(users.router)
app.include_router(chats.router)
app.include_router(messages.router)
app.include_router(attachments.router)
@app.get('/health/live')
def live():
return {'status':'ok'}
@app.get('/health/ready')
def ready():
with db() as conn:
conn.execute('SELECT 1')
try:
s3_client().head_bucket(Bucket=bucket())
except (ClientError,BotoCoreError,OSError):
raise HTTPException(503,'Storage unavailable')
return {'status':'ok'}
+5
View File
@@ -0,0 +1,5 @@
"""One-shot schema migration (never run from individual API replicas)."""
from app.storage import migrate
if __name__ == '__main__':
migrate()
print('Migrations completed')
View File
+39
View File
@@ -0,0 +1,39 @@
from datetime import datetime
from app.models.time_format import utc_z
from uuid import UUID
from pydantic import BaseModel, ConfigDict, Field, field_validator, field_serializer
class CreateChat(BaseModel):
model_config = ConfigDict(extra="forbid")
title: str = Field(min_length=1, max_length=100)
member_ids: list[UUID] = Field(min_length=1, max_length=100)
@field_validator("title")
@classmethod
def title_must_not_be_blank(cls, value: str) -> str:
if not value.strip():
raise ValueError("title must not be blank")
return value
@field_validator("member_ids")
@classmethod
def member_ids_must_be_unique(cls, value: list[UUID]) -> list[UUID]:
if len(value) != len(set(value)):
raise ValueError("member_ids must be unique")
return value
class Chat(BaseModel):
model_config = ConfigDict(extra="forbid")
id: UUID
title: str
member_ids: list[UUID]
created_at: datetime
@field_serializer("created_at", when_used="json")
def serialize_created_at(self, value: datetime) -> str:
return utc_z(value)
class ChatPage(BaseModel):
model_config = ConfigDict(extra="forbid")
items: list[Chat]
next_cursor: str | None
+35
View File
@@ -0,0 +1,35 @@
from datetime import datetime
from uuid import UUID
from pydantic import BaseModel, ConfigDict, Field, field_validator, field_serializer, model_validator
from app.models.time_format import utc_z
class CreateMessage(BaseModel):
model_config = ConfigDict(extra='forbid')
text: str = Field(max_length=4000)
attachment_ids: list[UUID] = Field(default_factory=list,max_length=10)
@model_validator(mode='after')
def valid_content(self):
if len(set(self.attachment_ids)) != len(self.attachment_ids):
raise ValueError('duplicate attachments')
if not self.text.strip() and not self.attachment_ids:
raise ValueError('message must contain text or attachments')
return self
class Message(BaseModel):
model_config = ConfigDict(extra='forbid')
id: UUID
chat_id: UUID
sender_id: UUID
text: str
created_at: datetime
attachment_ids: list[UUID]
@field_serializer('created_at', when_used='json')
def serialize_created_at(self, value: datetime) -> str:
return utc_z(value)
class MessagePage(BaseModel):
model_config = ConfigDict(extra='forbid')
items: list[Message]
next_cursor: str | None
+9
View File
@@ -0,0 +1,9 @@
"""Contract requires RFC3339 UTC timestamps with a literal Z suffix."""
from datetime import datetime, timezone
def utc_z(value: datetime) -> str:
if value.tzinfo is None:
# Database columns are TIMESTAMPTZ, but keep the serializer defensive.
value = value.replace(tzinfo=timezone.utc)
return value.astimezone(timezone.utc).isoformat().replace('+00:00', 'Z')
+28
View File
@@ -0,0 +1,28 @@
from datetime import datetime
from app.models.time_format import utc_z
from uuid import UUID
from pydantic import BaseModel, ConfigDict, Field, field_validator, field_serializer
class CreateUser(BaseModel):
model_config = ConfigDict(extra="forbid")
username: str = Field(pattern=r"^[a-z0-9_]{3,32}$")
display_name: str = Field(min_length=1, max_length=100)
password: str = Field(min_length=12, max_length=128)
@field_validator("display_name")
@classmethod
def nonempty(cls, value):
if not value.strip():
raise ValueError("Blank display name")
return value
class User(BaseModel):
model_config = ConfigDict(extra="forbid")
id: UUID
username: str
display_name: str
created_at: datetime
@field_serializer("created_at", when_used="json")
def serialize_created_at(self, value: datetime) -> str:
return utc_z(value)
View File
+166
View File
@@ -0,0 +1,166 @@
"""Bounded image uploads, authorized streaming downloads and private object storage."""
import hashlib
import os
import tempfile
import unicodedata
from uuid import UUID, uuid4
from io import BytesIO
from fastapi import APIRouter, Depends, HTTPException, Request, Query
from fastapi.responses import StreamingResponse
from starlette.concurrency import run_in_threadpool
from PIL import Image, UnidentifiedImageError, ImageFile
from botocore.exceptions import BotoCoreError, ClientError
from app.dependencies import current_user
from app.routers.chats import get_chat_row
from app.storage import db
from app.s3 import s3_client, bucket
from app.models.time_format import utc_z
from pydantic import BaseModel, ConfigDict, field_serializer
from datetime import datetime
router=APIRouter()
MAX_BYTES=10*1024*1024
MAX_PIXELS=40_000_000
CHUNK=64*1024
Image.MAX_IMAGE_PIXELS=MAX_PIXELS
ImageFile.LOAD_TRUNCATED_IMAGES=False
class Attachment(BaseModel):
model_config=ConfigDict(extra='forbid')
id:UUID
chat_id:UUID
owner_id:UUID
filename:str
content_type:str
size_bytes:int
sha256:str
status:str
variants:list=[]
error_code:str|None=None
created_at:datetime
updated_at:datetime
@field_serializer('created_at','updated_at',when_used='json')
def as_utc(self,value:datetime)->str:
return utc_z(value)
def public(row):
return {k:row[k] for k in ('id','chat_id','owner_id','filename','content_type','size_bytes',
'sha256','status','created_at','updated_at')} | {'variants':[],'error_code':None}
def check_name(name):
if not 1<=len(name)<=255 or any(ch in '/\\' or ord(ch)<32 or 127<=ord(ch)<160 or unicodedata.category(ch)=='Cc' for ch in name):
raise HTTPException(400,'Invalid filename')
def inspect_image(file, declared):
file.seek(0)
try:
with Image.open(file) as img:
fmt=img.format
w,h=img.size
if w<=0 or h<=0 or w*h>MAX_PIXELS:
raise HTTPException(413,'Image pixel count exceeded')
expected={'PNG':'image/png','JPEG':'image/jpeg'}.get(fmt)
if not expected or expected!=declared:
raise HTTPException(415,'Image content type mismatch')
img.verify()
file.seek(0)
with Image.open(file) as img:
img.load() # reject truncated PNG/JPEG and decompression errors
except HTTPException:
raise
except (UnidentifiedImageError, ValueError, OSError, SyntaxError, Image.DecompressionBombError, Image.DecompressionBombWarning):
raise HTTPException(415,'Invalid image')
finally:
file.seek(0)
def put_original(file, key, content_type):
client=s3_client()
file.seek(0)
client.upload_fileobj(file,bucket(),key,ExtraArgs={'ContentType':content_type},
Config=__import__('boto3').s3.transfer.TransferConfig(
multipart_threshold=16*1024*1024,use_threads=False))
def delete_original(key):
s3_client().delete_object(Bucket=bucket(),Key=key)
@router.post('/api/v1/chats/{chat_id}/attachments',response_model=Attachment,status_code=201)
async def upload(chat_id:UUID,request:Request, filename:str=Query(...),user=Depends(current_user)):
check_name(filename)
declared=request.headers.get('content-type','').lower()
if declared not in ('image/png','image/jpeg'):
raise HTTPException(415,'Only PNG and JPEG are supported')
with db() as conn:
if get_chat_row(conn,chat_id,user['id']) is None:
raise HTTPException(404,'Chat not found')
length=request.headers.get('content-length')
if length is not None:
try:
if int(length)>MAX_BYTES:raise HTTPException(413,'Upload too large')
except ValueError:
raise HTTPException(400,'Invalid content length')
size=0
digest=hashlib.sha256()
# Disk-backed temporary spool; no API replica retains permanent content.
with tempfile.SpooledTemporaryFile(max_size=1024*1024,mode='w+b') as file:
async for part in request.stream():
size+=len(part)
if size>MAX_BYTES:
raise HTTPException(413,'Upload too large')
digest.update(part)
file.write(part)
if not size:raise HTTPException(415,'Empty image')
await run_in_threadpool(inspect_image,file,declared)
ident=uuid4()
key='originals/'+str(uuid4())
try:
await run_in_threadpool(put_original,file,key,declared)
except (ClientError,BotoCoreError,OSError):
# A partially written object is possible after a transport error.
try:await run_in_threadpool(delete_original,key)
except Exception:pass # periodic reconciler can remove an orphan after grace
raise HTTPException(503,'Storage unavailable')
try:
with db() as conn:
if get_chat_row(conn,chat_id,user['id']) is None:
raise HTTPException(404,'Chat not found')
row=conn.execute('''INSERT INTO attachments
(id,chat_id,owner_id,filename,content_type,size_bytes,sha256,object_key)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s) RETURNING *''',
(ident,chat_id,user['id'],filename,declared,size,digest.hexdigest(),key)).fetchone()
except Exception:
try:await run_in_threadpool(delete_original,key)
except Exception:pass
raise
return public(row)
def authorized(ident,user):
with db() as conn:
row=conn.execute('''SELECT a.* FROM attachments a
JOIN chat_members cm ON cm.chat_id=a.chat_id AND cm.user_id=%s
WHERE a.id=%s''',(user['id'],ident)).fetchone()
if row is None:raise HTTPException(404,'Attachment not found')
return row
@router.get('/api/v1/attachments/{attachment_id}',response_model=Attachment)
def get_attachment(attachment_id:UUID,user=Depends(current_user)):
return public(authorized(attachment_id,user))
@router.get('/api/v1/attachments/{attachment_id}/content')
def download(attachment_id:UUID,variant:str='original',user=Depends(current_user)):
row=authorized(attachment_id,user)
if variant!='original':raise HTTPException(404,'Variant not found')
try:
obj=s3_client().get_object(Bucket=bucket(),Key=row['object_key'])
except (ClientError,BotoCoreError,OSError):
raise HTTPException(503,'Storage unavailable')
body=obj['Body']
def chunks():
try:
for chunk in body.iter_chunks(chunk_size=CHUNK):
if chunk:yield chunk
finally:
body.close()
return StreamingResponse(chunks(),media_type=row['content_type'],headers={
'Content-Length':str(row['size_bytes']),
'Content-Disposition':'attachment',
'X-Content-Type-Options':'nosniff'})
+52
View File
@@ -0,0 +1,52 @@
from uuid import UUID, uuid4
from fastapi import APIRouter, Depends, HTTPException, Query
from app.dependencies import current_user
from app.models.chats import Chat, ChatPage, CreateChat
from app.storage import db
router = APIRouter()
def get_chat_row(conn, chat_id, user_id):
return conn.execute("""SELECT c.id,c.title,c.created_at FROM chats c
JOIN chat_members cm ON cm.chat_id=c.id WHERE c.id=%s AND cm.user_id=%s""",
(chat_id,user_id)).fetchone()
def chat_output(conn,row):
members=conn.execute('SELECT user_id FROM chat_members WHERE chat_id=%s ORDER BY user_id',(row['id'],)).fetchall()
return {**row, 'member_ids':[m['user_id'] for m in members]}
@router.post('/api/v1/chats', response_model=Chat, status_code=201)
def create_chat(payload:CreateChat,user=Depends(current_user)):
if user['id'] not in payload.member_ids:
raise HTTPException(status_code=400,detail='Creator must be a chat member')
with db() as conn:
rows=conn.execute('SELECT id FROM users WHERE id = ANY(%s)',(payload.member_ids,)).fetchall()
if len(rows)!=len(payload.member_ids):
raise HTTPException(status_code=400,detail='Unknown member')
chat=conn.execute('INSERT INTO chats(id,title) VALUES (%s,%s) RETURNING id,title,created_at',
(uuid4(),payload.title)).fetchone()
for member in payload.member_ids:
conn.execute('INSERT INTO chat_members(chat_id,user_id) VALUES (%s,%s)',(chat['id'],member))
return chat_output(conn,chat)
@router.get('/api/v1/chats',response_model=ChatPage)
def list_chats(limit:int=Query(default=50,ge=1,le=100),user=Depends(current_user)):
with db() as conn:
rows=conn.execute("""SELECT c.id,c.title,c.created_at FROM chats c
JOIN chat_members m ON m.chat_id=c.id WHERE m.user_id=%s
ORDER BY c.created_at,c.id LIMIT %s""",(user['id'],limit+1)).fetchall()
items=[chat_output(conn,row) for row in rows[:limit]]
return {'items':items,'next_cursor':None}
@router.get('/api/v1/chats/{chat_id}',response_model=Chat)
def get_chat(chat_id:UUID,user=Depends(current_user)):
with db() as conn:
row=get_chat_row(conn,chat_id,user['id'])
if row is None:
raise HTTPException(status_code=404,detail='Chat not found')
return chat_output(conn,row)
+113
View File
@@ -0,0 +1,113 @@
import base64
import binascii
import json
import hashlib
import re
from datetime import datetime
from uuid import UUID,uuid4
from fastapi import APIRouter,Depends,HTTPException,Query,Header
from app.dependencies import current_user
from app.models.messages import CreateMessage,Message,MessagePage
from app.routers.chats import get_chat_row
from app.storage import db
router=APIRouter()
MESSAGE_SELECT = """SELECT m.id,m.chat_id,m.sender_id,m.text,m.created_at,
COALESCE((SELECT array_agg(ma.attachment_id ORDER BY ma.position)
FROM message_attachments ma WHERE ma.message_id=m.id), ARRAY[]::uuid[]) AS attachment_ids
FROM messages m"""
def encode_cursor(chat_id,row):
raw=json.dumps({'chat_id':str(chat_id),'created_at':row['created_at'].isoformat(),'id':str(row['id'])}).encode()
return base64.urlsafe_b64encode(raw).decode().rstrip('=')
def decode_cursor(chat_id,cursor):
try:
raw=base64.b64decode(cursor+'='*(-len(cursor)%4),altchars=b'-_',validate=True)
data=json.loads(raw)
ts=datetime.fromisoformat(data['created_at'])
ident=UUID(data['id'])
if data['chat_id']!=str(chat_id) or ts.tzinfo is None:
raise ValueError('Wrong cursor')
return ts,ident
except (ValueError,KeyError,TypeError,UnicodeDecodeError,binascii.Error):
raise HTTPException(status_code=400,detail='Invalid cursor')
IDEMPOTENCY_KEY_RE = re.compile(r'^[A-Za-z0-9._:-]{1,128}$', re.ASCII)
def canonical_message_hash(payload):
# Canonical semantics, not raw JSON bytes or field order.
data = payload.model_dump(mode='json', exclude_none=True)
data.setdefault('attachment_ids', [])
return hashlib.sha256(json.dumps(data, sort_keys=True, separators=(',', ':'),
ensure_ascii=False).encode('utf-8')).hexdigest()
def advisory_lock_id(user_id,chat_id,key):
scope = f'{user_id}:{chat_id}:{key}'.encode('utf-8')
return int.from_bytes(hashlib.sha256(scope).digest()[:8], 'big', signed=True)
@router.post('/api/v1/chats/{chat_id}/messages',response_model=Message,status_code=201)
def create_message(chat_id:UUID,payload:CreateMessage,user=Depends(current_user),
idempotency_key:str|None=Header(default=None,alias='Idempotency-Key')):
if idempotency_key is None or not IDEMPOTENCY_KEY_RE.fullmatch(idempotency_key):
raise HTTPException(status_code=400,detail='Invalid or missing Idempotency-Key')
request_hash=canonical_message_hash(payload)
with db() as conn:
# Explicit authorization is checked on EVERY replay, not just initial POST.
if get_chat_row(conn,chat_id,user['id']) is None:
raise HTTPException(status_code=404,detail='Chat not found')
# Same key on different processes serializes in PostgreSQL. The message and
# key record commit together, so a network timeout cannot create duplicates.
conn.execute('SELECT pg_advisory_xact_lock(%s)',
(advisory_lock_id(user['id'],chat_id,idempotency_key),))
saved=conn.execute("""SELECT request_hash,message_id,created_at FROM message_idempotency
WHERE user_id=%s AND chat_id=%s AND idem_key=%s FOR UPDATE""",
(user['id'],chat_id,idempotency_key)).fetchone()
if saved is not None:
# Contract promises a replay window of at least 24 hours; retaining
# keys longer is safe and prevents late duplicate deliveries.
if saved['request_hash'] != request_hash:
raise HTTPException(status_code=409,detail='Idempotency-Key used with another body')
return conn.execute(MESSAGE_SELECT + """ WHERE m.id=%s""",(saved['message_id'],)).fetchone()
if payload.attachment_ids:
authorized=conn.execute("""SELECT id FROM attachments
WHERE chat_id=%s AND id=ANY(%s) AND status='ready' FOR SHARE""",
(chat_id,payload.attachment_ids)).fetchall()
if len(authorized)!=len(payload.attachment_ids):
raise HTTPException(status_code=404,detail='Attachment not found')
row=conn.execute("""INSERT INTO messages(id,chat_id,sender_id,text)
VALUES (%s,%s,%s,%s) RETURNING id,chat_id,sender_id,text,created_at""",
(uuid4(),chat_id,user['id'],payload.text)).fetchone()
for pos,attachment_id in enumerate(payload.attachment_ids):
conn.execute("""INSERT INTO message_attachments(message_id,attachment_id,position)
VALUES (%s,%s,%s)""",(row['id'],attachment_id,pos))
row['attachment_ids']=payload.attachment_ids
conn.execute("""INSERT INTO message_idempotency(user_id,chat_id,idem_key,request_hash,message_id)
VALUES (%s,%s,%s,%s,%s)""",
(user['id'],chat_id,idempotency_key,request_hash,row['id']))
return row
@router.get('/api/v1/chats/{chat_id}/messages',response_model=MessagePage)
def list_messages(chat_id:UUID,limit:int=Query(default=50,ge=1,le=100),
cursor:str|None=Query(default=None,min_length=1),user=Depends(current_user)):
with db() as conn:
if get_chat_row(conn,chat_id,user['id']) is None:
raise HTTPException(status_code=404,detail='Chat not found')
if cursor is not None:
ts,ident=decode_cursor(chat_id,cursor)
rows=conn.execute(MESSAGE_SELECT + """ WHERE m.chat_id=%s AND (m.created_at,m.id)>(%s,%s)
ORDER BY m.created_at,m.id LIMIT %s""",(chat_id,ts,ident,limit+1)).fetchall()
else:
rows=conn.execute(MESSAGE_SELECT + """ WHERE m.chat_id=%s ORDER BY m.created_at,m.id LIMIT %s""",(chat_id,limit+1)).fetchall()
items=rows[:limit]
next_cursor=encode_cursor(chat_id,items[-1]) if len(rows)>limit else None
return {'items':items,'next_cursor':next_cursor}
+78
View File
@@ -0,0 +1,78 @@
import base64
import binascii
import hashlib
import hmac
import os
import secrets
from datetime import datetime, timedelta, timezone
from uuid import uuid4
from fastapi import APIRouter, Depends, Header, HTTPException, Response
from psycopg.errors import UniqueViolation
from app.dependencies import current_user, unauthorized
from app.models.users import CreateUser, User
from app.models.time_format import utc_z
from app.storage import db
router = APIRouter()
def password_hash(password):
salt = os.urandom(16)
derived = hashlib.scrypt(password.encode('utf-8'), salt=salt, n=2**14, r=8, p=1, dklen=32)
return 'scrypt$' + salt.hex() + '$' + derived.hex()
def verify_password(password, encoded):
try:
_, salt_hex, expected_hex = encoded.split('$')
actual = hashlib.scrypt(password.encode('utf-8'),salt=bytes.fromhex(salt_hex),n=2**14,r=8,p=1,dklen=32)
return hmac.compare_digest(actual, bytes.fromhex(expected_hex))
except (ValueError, TypeError):
return False
@router.post('/api/v1/users', response_model=User, status_code=201)
def register_user(payload: CreateUser):
with db() as conn:
try:
user = conn.execute("""INSERT INTO users(id,username,display_name,password_hash)
VALUES (%s,%s,%s,%s) RETURNING id,username,display_name,created_at""",
(uuid4(),payload.username,payload.display_name,password_hash(payload.password))).fetchone()
except UniqueViolation:
raise HTTPException(status_code=409,detail='Username already exists')
return user
@router.get('/api/v1/users/me', response_model=User)
def get_current_user(user=Depends(current_user)):
return user
@router.post('/api/v1/auth/sessions', status_code=201)
def create_session(authorization: str | None = Header(default=None)):
if not authorization or not authorization.startswith('Basic '):
unauthorized(basic=True)
try:
raw = base64.b64decode(authorization[6:],validate=True).decode('utf-8')
username, password = raw.split(':',1)
except (ValueError, UnicodeDecodeError, binascii.Error):
unauthorized(basic=True)
with db() as conn:
row = conn.execute('SELECT * FROM users WHERE username=%s',(username,)).fetchone()
if row is None or not verify_password(password,row['password_hash']):
unauthorized(basic=True)
token = secrets.token_urlsafe(48)
ttl = int(os.getenv('SESSION_TTL_SECONDS','3600'))
expires = datetime.now(timezone.utc)+timedelta(seconds=ttl)
conn.execute('INSERT INTO sessions(token_hash,user_id,expires_at) VALUES (%s,%s,%s)',
(hashlib.sha256(token.encode()).hexdigest(),row['id'],expires))
return {'token':token,'token_type':'Bearer','expires_at':utc_z(expires),
'user':User.model_validate({key:row[key] for key in ('id','username','display_name','created_at')})}
@router.delete('/api/v1/auth/sessions/current',status_code=204)
def logout(authorization: str | None = Header(default=None),user=Depends(current_user)):
token=authorization[7:]
with db() as conn:
conn.execute('DELETE FROM sessions WHERE token_hash=%s',(hashlib.sha256(token.encode()).hexdigest(),))
return Response(status_code=204)
+19
View File
@@ -0,0 +1,19 @@
"""S3 access with least-privilege application credentials (never root credentials)."""
import os
from pathlib import Path
import boto3
from botocore.config import Config
def s3_client():
return boto3.client(
's3', endpoint_url=os.environ['S3_ENDPOINT'],
region_name=os.getenv('S3_REGION','us-east-1'),
aws_access_key_id=Path(os.environ['S3_ACCESS_KEY_ID_FILE']).read_text().strip(),
aws_secret_access_key=Path(os.environ['S3_SECRET_ACCESS_KEY_FILE']).read_text().strip(),
config=Config(connect_timeout=2,read_timeout=15,retries={'max_attempts':2},
s3={'addressing_style':'path'}))
def bucket():
return os.getenv('S3_BUCKET','messenger-attachments')
+31
View File
@@ -0,0 +1,31 @@
"""PostgreSQL is the source of truth, including bearer sessions."""
import os
from contextlib import contextmanager
from pathlib import Path
import psycopg
from psycopg.rows import dict_row
def database_url():
secret_file = os.getenv("DATABASE_URL_FILE")
if secret_file:
return Path(secret_file).read_text().strip()
return os.environ["DATABASE_URL"]
@contextmanager
def db():
with psycopg.connect(database_url(), row_factory=dict_row, connect_timeout=5) as conn:
yield conn
def migrate():
with db() as conn:
# Serialize migration execution even if two operators race to bootstrap.
conn.execute("SELECT pg_advisory_xact_lock(83475391)")
conn.execute("CREATE TABLE IF NOT EXISTS schema_migrations (version TEXT PRIMARY KEY)")
for file in sorted((Path(__file__).parent.parent / "migrations").glob("*.sql")):
if conn.execute("SELECT 1 FROM schema_migrations WHERE version=%s", (file.name,)).fetchone():
continue
conn.execute(file.read_text())
conn.execute("INSERT INTO schema_migrations(version) VALUES (%s)", (file.name,))
+40
View File
@@ -0,0 +1,40 @@
2026-11-11T17:03:17.0735085+03:00
Negative attachment checks
PASS: invalid size/image/MIME/filename, outsider ACL and cross-chat bindings
Direct S3 access control
PASS: authenticated S3 GET succeeded and anonymous GET of the same object returned HTTP 403
Upload replica shutdown and S3 outage
Container mtusi2026lab4_verified-api2-1 Stopping
Container mtusi2026lab4_verified-api2-1 Stopped
PASS: upload via api2, stop api2, download via api1, SHA-256 99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75
Container mtusi2026lab4_verified-api1-1 Running
Container mtusi2026lab4_verified-api2-1 Starting
Container mtusi2026lab4_verified-api2-1 Started
Container mtusi2026lab4_verified-api2-1 Waiting
Container mtusi2026lab4_verified-api1-1 Waiting
Container mtusi2026lab4_verified-api1-1 Healthy
Container mtusi2026lab4_verified-api2-1 Healthy
Container mtusi2026lab4_verified-s3-1 Stopping
Container mtusi2026lab4_verified-s3-1 Stopped
PASS: stopped S3 returned HTTP 503 for readiness and valid PNG upload
Container mtusi2026lab4_verified-s3-1 Starting
Container mtusi2026lab4_verified-s3-1 Started
PASS: readiness recovered after S3 restart
Final readiness
{"status":"ok"}
Final container state
api1 Up 3 minutes (unhealthy)
api2 Up About a minute (unhealthy)
db Up 3 minutes (healthy)
lb Up 3 minutes
migrate Exited (0) 3 minutes ago
s3 Up 1 second
s3-bootstrap Exited (0) 3 minutes ago
State after healthchecks converged
api1 Up 3 minutes (healthy)
api2 Up About a minute (healthy)
db Up 3 minutes (healthy)
lb Up 3 minutes
migrate Exited (0) 3 minutes ago
s3 Up 8 seconds
s3-bootstrap Exited (0) 3 minutes ago
+8
View File
@@ -0,0 +1,8 @@
2026-11-11
PASS py -3 -X utf8 scripts/check_contract.py
PASS Python AST parsing (25 files)
PASS Windows PowerShell setup (six files, matching PostgreSQL password, no CR/LF)
PASS Windows PowerShell setup rerun (all six hashes unchanged)
PASS docker compose --env-file .env config --quiet (Compose v5.6.1)
See windows-verified.txt and additional-checks.txt for Docker integration results.
PASS POSIX shell syntax and CRLF normalization (Git Bash)
Binary file not shown.
Binary file not shown.
+59
View File
@@ -0,0 +1,59 @@
2026-11-11T17:00:35.1451077+03:00
--- Docker Engine ---
linux
Docker Engine exit code = 0
--- Python ---
Python 3.15.0
Python exit code = 0
--- Startup and readiness ---
{"status":"ok"}
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
mtusi2026lab4_verified-api1-1 sha256:46c6fa875a96226ff6edbf016ac743907c36a0dae253f3a101661e7d877ef142 "uvicorn app.main:ap…" api1 2 minutes ago Up 20 seconds (healthy) 8080/tcp
mtusi2026lab4_verified-api2-1 sha256:ab5b93ca1b43b1b4ca873d209996c6dcc432f35df57eb261ba454b10f9d9b923 "uvicorn app.main:ap…" api2 2 minutes ago Up About a minute (healthy) 8080/tcp
mtusi2026lab4_verified-db-1 postgres:16.8-alpine "docker-entrypoint.s…" db 2 minutes ago Up About a minute (healthy) 5432/tcp
mtusi2026lab4_verified-lb-1 nginx:1.27.5-alpine "/docker-entrypoint.…" lb 2 minutes ago Up About a minute 127.0.0.1:8080->8080/tcp
mtusi2026lab4_verified-migrate-1 sha256:372898c88c95ea3b004c908e0a3e45b6ea4a34378676e8801db33fa03ff487c3 "python -m app.migra…" migrate 2 minutes ago Exited (0) 1 second ago
mtusi2026lab4_verified-s3-1 cgr.dev/chainguard/minio@sha256:fb893e36fc369d41e92f678b17ec27378fa5bf0576aeef7ac15460c13629ab5a "/usr/bin/minio serv…" s3 About a minute ago Up About a minute
mtusi2026lab4_verified-s3-bootstrap-1 cgr.dev/chainguard/minio-client@sha256:fe649f71872c299489deb3cb9e057e5d2f76a45c85b7599f7c21616ac80b5e63 "/bin/sh /etc/bootst…" s3-bootstrap About a minute ago Exited (0) 2 seconds ago
Startup and readiness exit code = 0
--- Smoke ---
PASS: lab4 public API scenario; infrastructure and grading evidence remain separate.
Smoke exit code = 0
--- Persistence ---
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
Persistence exit code = 0
--- Failover ---
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 67, "failed_attempts": 0, "first_recovery_seconds": 1.079}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
Failover exit code = 0
--- State after failover ---
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
mtusi2026lab4_verified-api1-1 sha256:46c6fa875a96226ff6edbf016ac743907c36a0dae253f3a101661e7d877ef142 "uvicorn app.main:ap…" api1 2 minutes ago Up 26 seconds (healthy) 8080/tcp
mtusi2026lab4_verified-api2-1 sha256:ab5b93ca1b43b1b4ca873d209996c6dcc432f35df57eb261ba454b10f9d9b923 "uvicorn app.main:ap…" api2 2 minutes ago Exited (0) 16 seconds ago
mtusi2026lab4_verified-db-1 postgres:16.8-alpine "docker-entrypoint.s…" db 2 minutes ago Up 26 seconds (healthy) 5432/tcp
mtusi2026lab4_verified-lb-1 nginx:1.27.5-alpine "/docker-entrypoint.…" lb 2 minutes ago Up 19 seconds 127.0.0.1:8080->8080/tcp
mtusi2026lab4_verified-migrate-1 sha256:372898c88c95ea3b004c908e0a3e45b6ea4a34378676e8801db33fa03ff487c3 "python -m app.migra…" migrate 2 minutes ago Exited (0) 34 seconds ago
mtusi2026lab4_verified-s3-1 cgr.dev/chainguard/minio@sha256:fb893e36fc369d41e92f678b17ec27378fa5bf0576aeef7ac15460c13629ab5a "/usr/bin/minio serv…" s3 2 minutes ago Up 26 seconds
mtusi2026lab4_verified-s3-bootstrap-1 cgr.dev/chainguard/minio-client@sha256:fe649f71872c299489deb3cb9e057e5d2f76a45c85b7599f7c21616ac80b5e63 "/bin/sh /etc/bootst…" s3-bootstrap 2 minutes ago Exited (0) 35 seconds ago
State after failover exit code = 0
--- Restore replicas ---
Container mtusi2026lab4_verified-api1-1 Running
Container mtusi2026lab4_verified-api2-1 Starting
Container mtusi2026lab4_verified-api2-1 Started
Container mtusi2026lab4_verified-api1-1 Waiting
Container mtusi2026lab4_verified-api2-1 Waiting
Container mtusi2026lab4_verified-api1-1 Healthy
Container mtusi2026lab4_verified-api2-1 Healthy
Restore replicas exit code = 0
--- Final state ---
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
mtusi2026lab4_verified-api1-1 sha256:46c6fa875a96226ff6edbf016ac743907c36a0dae253f3a101661e7d877ef142 "uvicorn app.main:ap…" api1 2 minutes ago Up 33 seconds (healthy) 8080/tcp
mtusi2026lab4_verified-api2-1 sha256:ab5b93ca1b43b1b4ca873d209996c6dcc432f35df57eb261ba454b10f9d9b923 "uvicorn app.main:ap…" api2 2 minutes ago Up 6 seconds (healthy) 8080/tcp
mtusi2026lab4_verified-db-1 postgres:16.8-alpine "docker-entrypoint.s…" db 2 minutes ago Up 33 seconds (healthy) 5432/tcp
mtusi2026lab4_verified-lb-1 nginx:1.27.5-alpine "/docker-entrypoint.…" lb 2 minutes ago Up 26 seconds 127.0.0.1:8080->8080/tcp
mtusi2026lab4_verified-migrate-1 sha256:372898c88c95ea3b004c908e0a3e45b6ea4a34378676e8801db33fa03ff487c3 "python -m app.migra…" migrate 2 minutes ago Exited (0) 41 seconds ago
mtusi2026lab4_verified-s3-1 cgr.dev/chainguard/minio@sha256:fb893e36fc369d41e92f678b17ec27378fa5bf0576aeef7ac15460c13629ab5a "/usr/bin/minio serv…" s3 2 minutes ago Up 33 seconds
mtusi2026lab4_verified-s3-bootstrap-1 cgr.dev/chainguard/minio-client@sha256:fe649f71872c299489deb3cb9e057e5d2f76a45c85b7599f7c21616ac80b5e63 "/bin/sh /etc/bootst…" s3-bootstrap 2 minutes ago Exited (0) 42 seconds ago
Final state exit code = 0
+35
View File
@@ -0,0 +1,35 @@
#!/bin/sh
set -eu
read_secret() {
tr -d '\r\n' < "/run/secrets/$1"
}
ROOT_ACCESS=$(read_secret s3_root_access)
ROOT_SECRET=$(read_secret s3_root_secret)
API_ACCESS=$(read_secret s3_access_key_id)
API_SECRET=$(read_secret s3_secret_access_key)
# Bound bootstrap wait: fail with diagnostic rather than stall forever.
count=0
until output=$(timeout 4 mc alias set local http://s3:9000 "$ROOT_ACCESS" "$ROOT_SECRET" 2>&1) && output=$(timeout 4 mc admin info local 2>&1); do
count=$((count + 1))
if [ "$count" -ge 12 ]; then
echo "ERROR: MinIO did not become reachable after 12 attempts; inspect docker compose logs s3" >&2
if [ -n "$output" ]; then printf '%s\n' "$output" >&2; fi
exit 1
fi
sleep 2
done
mc mb --ignore-existing local/messenger-attachments
mc anonymous set none local/messenger-attachments
# Repeatable bootstrap: policy may exist already after restarts.
mc admin policy create local messenger-app /etc/bootstrap/s3-policy.json
mc admin user add local "$API_ACCESS" "$API_SECRET"
if ! mc admin policy attach local messenger-app --user "$API_ACCESS"; then
mc admin policy entities local messenger-app --user "$API_ACCESS" --json | grep -Fq "\"$API_ACCESS\""
fi
mc anonymous get local/messenger-attachments
mc alias set api http://s3:9000 "$API_ACCESS" "$API_SECRET" >/dev/null
mc ls api/messenger-attachments >/dev/null
printf 'S3 bootstrap completed (private bucket, limited API identity).\n'
+26
View File
@@ -0,0 +1,26 @@
worker_processes auto;
events { worker_connections 1024; }
http {
resolver 127.0.0.11 valid=1s ipv6=off;
upstream messenger_api {
zone messenger_api 64k;
server api1:8080 resolve max_fails=1 fail_timeout=2s;
server api2:8080 resolve max_fails=1 fail_timeout=2s;
}
server {
listen 8080;
client_max_body_size 11m;
location / {
proxy_pass http://messenger_api;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 1s;
proxy_read_timeout 35s;
proxy_send_timeout 35s;
proxy_next_upstream error timeout http_502 http_503;
proxy_next_upstream_tries 2;
}
}
}
+1
View File
@@ -0,0 +1 @@
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["s3:ListBucket"],"Resource":["arn:aws:s3:::messenger-attachments"]},{"Effect":"Allow","Action":["s3:GetObject","s3:PutObject","s3:DeleteObject"],"Resource":["arn:aws:s3:::messenger-attachments/originals/*"]}]}
+35
View File
@@ -0,0 +1,35 @@
CREATE TABLE IF NOT EXISTS users (
id UUID PRIMARY KEY,
username VARCHAR(32) NOT NULL UNIQUE,
display_name VARCHAR(100) NOT NULL,
password_hash TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT username_format CHECK (username ~ '^[a-z0-9_]{3,32}$')
);
CREATE TABLE IF NOT EXISTS chats (
id UUID PRIMARY KEY,
title VARCHAR(100) NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS chat_members (
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id),
PRIMARY KEY (chat_id, user_id)
);
CREATE INDEX IF NOT EXISTS chat_members_user ON chat_members(user_id, chat_id);
CREATE TABLE IF NOT EXISTS messages (
id UUID PRIMARY KEY,
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
sender_id UUID NOT NULL REFERENCES users(id),
text VARCHAR(4000) NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT nonblank_message CHECK (length(btrim(text)) > 0)
);
CREATE INDEX IF NOT EXISTS messages_chat_cursor ON messages(chat_id, created_at, id);
CREATE TABLE IF NOT EXISTS sessions (
token_hash CHAR(64) PRIMARY KEY,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
expires_at TIMESTAMPTZ NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS sessions_expiry ON sessions(expires_at);
+11
View File
@@ -0,0 +1,11 @@
-- Lab3: one atomic idempotency result per user, chat and key.
CREATE TABLE IF NOT EXISTS message_idempotency (
user_id UUID NOT NULL REFERENCES users(id),
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
idem_key VARCHAR(128) NOT NULL,
request_hash CHAR(64) NOT NULL,
message_id UUID NOT NULL UNIQUE REFERENCES messages(id) ON DELETE CASCADE,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (user_id, chat_id, idem_key)
);
CREATE INDEX IF NOT EXISTS message_idempotency_expiry ON message_idempotency(created_at);
+25
View File
@@ -0,0 +1,25 @@
-- A file is owned by exactly one chat. Binary bytes live only in private S3.
CREATE TABLE attachments (
id UUID PRIMARY KEY,
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
owner_id UUID NOT NULL REFERENCES users(id),
filename VARCHAR(255) NOT NULL,
content_type VARCHAR(32) NOT NULL CHECK (content_type IN ('image/png','image/jpeg')),
size_bytes INTEGER NOT NULL CHECK (size_bytes BETWEEN 1 AND 10485760),
sha256 CHAR(64) NOT NULL,
object_key VARCHAR(160) NOT NULL UNIQUE,
status VARCHAR(20) NOT NULL DEFAULT 'ready' CHECK (status='ready'),
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX attachments_chat_idx ON attachments(chat_id);
ALTER TABLE messages DROP CONSTRAINT nonblank_message;
-- Blank text is permitted only with attachment IDs; enforced by application transaction.
CREATE TABLE message_attachments (
message_id UUID NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
attachment_id UUID NOT NULL REFERENCES attachments(id),
position SMALLINT NOT NULL CHECK (position BETWEEN 0 AND 9),
PRIMARY KEY (message_id, attachment_id),
UNIQUE (message_id, position)
);
CREATE INDEX message_attachments_attachment_idx ON message_attachments(attachment_id);
+5
View File
@@ -0,0 +1,5 @@
fastapi==0.115.12
uvicorn==0.34.2
psycopg[binary]==3.2.6
boto3==1.37.38
Pillow==11.2.1
+10
View File
@@ -0,0 +1,10 @@
@echo off
setlocal
cd /d "%~dp0.."
rem Preserve PostgreSQL named volume, tokens and secrets.
docker compose --env-file .env restart -t 5 db s3 api1 api2
if errorlevel 1 exit /b 1
docker compose --env-file .env up -d --no-deps --wait --wait-timeout 55 api1 api2
if errorlevel 1 exit /b 1
docker compose --env-file .env restart -t 5 lb
exit /b %ERRORLEVEL%
+8
View File
@@ -0,0 +1,8 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
# Keep PostgreSQL named volume and the generated DB credentials unchanged.
# Restart shared DB and both API replicas; restart proxy to refresh backends.
docker compose --env-file .env restart -t 5 db s3 api1 api2
docker compose --env-file .env up -d --no-deps --wait --wait-timeout 55 api1 api2
docker compose --env-file .env restart -t 5 lb
+5
View File
@@ -0,0 +1,5 @@
@echo off
setlocal
cd /d "%~dp0.."
docker compose --env-file .env up -d --no-deps --wait --wait-timeout 55 api1 api2
exit /b %ERRORLEVEL%
+4
View File
@@ -0,0 +1,4 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
docker compose --env-file .env start api1 api2
+36
View File
@@ -0,0 +1,36 @@
$ErrorActionPreference = 'Stop'
Set-Location (Join-Path $PSScriptRoot '..')
if (!(Test-Path -LiteralPath '.env')) {
Copy-Item -LiteralPath '.env.example' -Destination '.env'
}
New-Item -ItemType Directory -Path 'secrets' -Force | Out-Null
function New-SecretFile([string]$Name, [int]$Length) {
$path = Join-Path (Get-Location) "secrets/$Name"
if (!(Test-Path -LiteralPath $path)) {
$bytes = [byte[]]::new($Length)
$rng = [System.Security.Cryptography.RandomNumberGenerator]::Create()
try {
$rng.GetBytes($bytes)
} finally {
$rng.Dispose()
}
$value = [BitConverter]::ToString($bytes).Replace('-', '').ToLowerInvariant()
[IO.File]::WriteAllText($path, $value, [Text.Encoding]::ASCII)
}
}
New-SecretFile 'db_password' 24
New-SecretFile 's3_root_access' 10
New-SecretFile 's3_root_secret' 20
New-SecretFile 's3_access_key_id' 10
New-SecretFile 's3_secret_access_key' 20
if (!(Test-Path -LiteralPath 'secrets/database_url')) {
$password = (Get-Content -LiteralPath 'secrets/db_password' -Raw).TrimEnd([char]10)
$encoded = [Uri]::EscapeDataString($password)
$url = "postgresql://messenger:${encoded}@db:5432/messenger"
[IO.File]::WriteAllText((Join-Path (Get-Location) 'secrets/database_url'), $url, [Text.Encoding]::ASCII)
}
Write-Host 'Local secrets are ready.'
+22
View File
@@ -0,0 +1,22 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
if [ ! -e .env ]; then cp .env.example .env; fi
mkdir -p secrets
if [ ! -e secrets/db_password ]; then
python3 -c 'import secrets;print(secrets.token_urlsafe(36),end="")' > secrets/db_password
fi
if [ ! -e secrets/database_url ]; then
python3 -c 'from pathlib import Path;from urllib.parse import quote; p=Path("secrets/db_password").read_text().strip();print("postgresql://messenger:"+quote(p,safe="")+"@db:5432/messenger",end="")' > secrets/database_url
fi
python3 - <<'PYGEN'
from pathlib import Path
import secrets
root=Path('secrets')
for name,length in [('s3_root_access',10),('s3_root_secret',20),('s3_access_key_id',10),('s3_secret_access_key',20)]:
file=root/name
if not file.exists():
file.write_bytes(secrets.token_hex(length).encode('ascii'))
PYGEN
chmod 600 secrets/* .env
printf 'Local secrets prepared (values not displayed).\n'
+10
View File
@@ -0,0 +1,10 @@
@echo off
setlocal
cd /d "%~dp0.."
if /I "%TARGET_INSTANCE_ID%"=="api1" goto stop
if /I "%TARGET_INSTANCE_ID%"=="api2" goto stop
echo Invalid TARGET_INSTANCE_ID; refusing to stop an arbitrary service 1>&2
exit /b 2
:stop
docker compose --env-file .env stop -t 2 %TARGET_INSTANCE_ID%
exit /b %ERRORLEVEL%
+7
View File
@@ -0,0 +1,7 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
case "${TARGET_INSTANCE_ID:-}" in
api1|api2) exec docker compose --env-file .env stop -t 2 "$TARGET_INSTANCE_ID" ;;
*) echo 'Unknown TARGET_INSTANCE_ID; refusing to stop an arbitrary service' >&2; exit 2 ;;
esac
+30
View File
@@ -0,0 +1,30 @@
@echo off
setlocal
cd /d "%~dp0"
where docker >nul 2>&1
if errorlevel 1 (
echo Docker CLI was not found. Start Docker Desktop with Linux containers.
exit /b 1
)
docker info --format "{{.OSType}}"
if errorlevel 1 exit /b 1
powershell.exe -NoProfile -ExecutionPolicy Bypass -File "%~dp0scripts\setup.ps1"
if errorlevel 1 exit /b 1
docker compose --env-file .env config --quiet
if errorlevel 1 exit /b 1
docker compose --env-file .env up -d --build --wait --wait-timeout 180
if errorlevel 1 goto diagnostics
curl.exe --fail --silent --show-error --retry 10 --retry-connrefused --retry-delay 1 --max-time 5 http://127.0.0.1:8080/health/ready
if errorlevel 1 goto diagnostics
echo.
docker compose --env-file .env ps -a
exit /b %errorlevel%
:diagnostics
docker compose --env-file .env ps -a
docker compose --env-file .env logs --tail=60 db s3 s3-bootstrap migrate api1 api2 lb
exit /b 1
+35
View File
@@ -0,0 +1,35 @@
#!/usr/bin/env python3
"""Additional negative integration checks against a running lab4 stack."""
import sys
from pathlib import Path
sys.path.insert(0,str(Path(__file__).resolve().parents[1]/'tests'))
from client import Client, arguments, run
from uuid import uuid4
def main():
args=arguments('Negative attachment checks')
c=Client(args.base_url,args.ca_file)
alice,bob,eve,chat=c.bootstrap()
prefix='/api/v1/chats/'+chat['id']+'/attachments?filename='
fixture=(Path(__file__).resolve().parents[1]/'tests/fixtures/sample.png').read_bytes()
hdr={'Content-Type':'image/png'}
c.call('POST',prefix+'../bad.png',400,actor=alice,raw=fixture,headers=hdr)
c.call('POST',prefix+'bad%5Cname.png',400,actor=alice,raw=fixture,headers=hdr)
c.call('POST',prefix+'too-large.png',413,actor=alice,raw=fixture+b'\0'*(10*1024*1024+1),headers=hdr)
c.call('POST',prefix+'corrupt.png',415,actor=alice,raw=b'not an image',headers=hdr)
c.call('POST',prefix+'spoof.png',415,actor=alice,raw=fixture,headers={'Content-Type':'image/jpeg'})
att,_=c.attachment(alice,chat)
for suffix in ('','/content'):
c.call('GET','/api/v1/attachments/'+att['id']+suffix,404,actor=eve)
other=c.call('POST','/api/v1/chats',201,actor=alice,body={
'title':'Other chat','member_ids':[alice['id'],eve['id']]},schema='Chat')['json']
c.message(alice,chat,'',attachments=[att['id']])
c.call('POST','/api/v1/chats/'+other['id']+'/messages',404,actor=alice,
body={'text':'','attachment_ids':[att['id']]},headers={'Idempotency-Key':uuid4().hex})
c.call('POST','/api/v1/chats/'+chat['id']+'/messages',404,actor=alice,
body={'text':'','attachment_ids':[str(uuid4())]},headers={'Idempotency-Key':uuid4().hex})
print('PASS: invalid size/image/MIME/filename, outsider ACL and cross-chat bindings')
if __name__=='__main__':
run(main)
+38
View File
@@ -0,0 +1,38 @@
#!/usr/bin/env python3
"""Integration check: concurrent same-key POSTs via LB, no custom test dependencies."""
import concurrent.futures
import json
import sys
import uuid
from pathlib import Path
sys.path.insert(0,str(Path(__file__).resolve().parents[1] / 'tests'))
from client import Client, run, arguments, check
def main():
args=arguments('Concurrent idempotency and LB check')
c=Client(args.base_url,args.ca_file,timeout=8)
alice,bob,eve,chat=c.bootstrap()
key=uuid.uuid4().hex
body={'text':'Конкурентное сообщение '+key}
path='/api/v1/chats/'+chat['id']+'/messages'
def send(_):
# Each worker has its own client, simulating independent processes.
worker=Client(args.base_url,args.ca_file,timeout=8)
return worker.call('POST',path,201,actor=alice,body=body,
headers={'Idempotency-Key':key},schema='Message')
with concurrent.futures.ThreadPoolExecutor(max_workers=16) as pool:
results=list(pool.map(send,range(32)))
ids={r['json']['id'] for r in results}
instances={r['headers'].get('X-Instance-Id') for r in results}
check(len(ids)==1,'Concurrent messages were duplicated')
check(len(instances)>=2,'Only one instance was observed; check LB')
c.call('POST',path,409,actor=alice,body={'text':'different text'},headers={'Idempotency-Key':key})
stored=c.call('GET',path,actor=bob,schema='MessagePage')['json']['items']
check(sum(x['id'] in ids for x in stored)==1,'History contains duplicate or no record')
print(json.dumps({'concurrent_requests':len(results),'unique_messages':len(ids),
'observed_replicas':sorted(instances)},ensure_ascii=False))
print('PASS: shared idempotency is atomic across replicas')
if __name__=='__main__':
run(main)
+47
View File
@@ -0,0 +1,47 @@
$ErrorActionPreference = 'Continue'
Set-Location $PSScriptRoot
if (!(Test-Path -LiteralPath 'start-windows.cmd')) {
throw 'Place this script in the extracted mtusi2026lab4 directory.'
}
New-Item -ItemType Directory -Force -Path evidence | Out-Null
$log = Join-Path $PSScriptRoot ("evidence/windows-run-{0:yyyyMMdd-HHmmss-fff}.txt" -f (Get-Date))
$script:failed = $false
function Write-Log([string]$Message) {
$Message | Tee-Object -FilePath $log -Append
}
function Invoke-Check([string]$Name, [string]$Program, [string[]]$Arguments) {
Write-Log ("--- {0} ---" -f $Name)
$global:LASTEXITCODE = 1
& $Program @Arguments 2>&1 | ForEach-Object { "$_" } | Tee-Object -FilePath $log -Append
$code = $LASTEXITCODE
Write-Log ("{0} exit code = {1}" -f $Name, $code)
$script:stepOK = $code -eq 0
if (!$script:stepOK) { $script:failed = $true }
}
$baseUrl = 'http://127.0.0.1:8080'
Write-Log (Get-Date -Format o)
try {
Invoke-Check 'Docker Engine' docker @('info', '--format', '{{.OSType}}')
if (!$script:stepOK) { exit 1 }
Invoke-Check 'Python' py @('-3', '--version')
if (!$script:stepOK) { exit 1 }
Invoke-Check 'Startup and readiness' cmd.exe @('/d', '/c', 'start-windows.cmd')
if (!$script:stepOK) { exit 1 }
Invoke-Check 'Smoke' py @('-3', '-X', 'utf8', 'tests/smoke.py', '--base-url', $baseUrl)
Invoke-Check 'Persistence' py @('-3', '-X', 'utf8', 'tests/resilience.py', '--mode', 'persistence', '--action-script', 'scripts/restart.cmd', '--base-url', $baseUrl)
try {
Invoke-Check 'Failover' py @('-3', '-X', 'utf8', 'tests/resilience.py', '--mode', 'failover', '--action-script', 'scripts/stop-one.cmd', '--base-url', $baseUrl)
Invoke-Check 'State after failover' docker @('compose', '--env-file', '.env', 'ps', '-a')
} finally {
Invoke-Check 'Restore replicas' cmd.exe @('/d', '/c', 'scripts\restore-replicas.cmd')
Invoke-Check 'Final state' docker @('compose', '--env-file', '.env', 'ps', '-a')
}
} finally {
Write-Host "Evidence file: $log"
}
if ($script:failed) { exit 1 }
exit 0