ЛР 4 - БПИ2303 - Кимпаев А.Р. - на 3 #103
+10
@@ -0,0 +1,10 @@
|
||||
FROM python:3.12.9-slim
|
||||
WORKDIR /srv
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
|
||||
COPY requirements.txt ./
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
COPY app ./app
|
||||
COPY migrations ./migrations
|
||||
EXPOSE 8080
|
||||
STOPSIGNAL SIGTERM
|
||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080", "--timeout-graceful-shutdown", "10"]
|
||||
@@ -1,37 +1,53 @@
|
||||
# Отчёт — лабораторная 4
|
||||
# Отчёт — лабораторная работа №4. S3-хранилище и вложения
|
||||
|
||||
- Автор, группа: TODO
|
||||
- Целевая оценка; выбранный трек 5, если нужен: TODO
|
||||
- Commit/PR: TODO после публикации
|
||||
- Использованные источники, библиотеки и помощь ИИ: TODO
|
||||
**Студент:** Кимпаев А. Р.
|
||||
**Группа:** БПИ2303
|
||||
**Целевая оценка:** 3 (полный минимум; дополнительные негативные тесты не являются заявкой на 4)
|
||||
**Ветка:** `solution/bpi2303-kimpaev`
|
||||
**Источники:** README.md, COURSE.md, contracts/README.md, contracts/openapi.json, tests/README.md из skeleton ЛР4; исходники собственной ЛР3. Использована помощь ИИ для переноса кода, реализации, документации и проведения проверок. Журналы фактических запусков приложены.
|
||||
|
||||
## Запуск из чистого clone
|
||||
|
||||
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
|
||||
ОС: Windows 11 с Docker Desktop (Linux containers) либо Linux/WSL2. Требуются Docker Engine + Compose v2, Python 3.9+ для тестов. В Windows запустите `.\start-windows.cmd` из PowerShell. Ниже показан эквивалент для Linux / Git Bash. Приложение — Python 3.12.9, FastAPI 0.115.12, psycopg 3.2.6, boto3 1.37.38, Pillow 11.2.1. Образы: postgres:16.8-alpine, nginx:1.27.5-alpine, MinIO RELEASE.2026-11-09T00-10-40Z и mc DEVELOPMENT.GOGET из Chainguard. Оба образа S3 закреплены по digest в `compose.yaml`.
|
||||
|
||||
## Архитектура и решения
|
||||
```bash
|
||||
sh scripts/setup.sh
|
||||
make check # или python3 scripts/check_contract.py
|
||||
# Проверить конфигурацию до запуска
|
||||
docker compose --env-file .env config --quiet
|
||||
docker compose --env-file .env up -d --build
|
||||
docker compose --env-file .env ps -a
|
||||
make test # или python3 tests/smoke.py --base-url http://localhost:8080
|
||||
python3 student-tests/attachments_negative.py --base-url http://localhost:8080
|
||||
```
|
||||
|
||||
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab3, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
|
||||
`setup.sh` создаёт `.env` и индивидуальные DB/S3 credentials в игнорируемом Git каталоге `secrets/`. Секреты не передаются через командную строку API. `migrate` выполняет SQL-миграции один раз, под PostgreSQL advisory lock. `s3-bootstrap` создаёт закрытый bucket и пользователя S3 с ограниченной политикой. Ожидаем завершения обоих job с кодом 0 и доступности `/health/ready` через LB. `.env.example` описывает логические параметры; фактически используются `SESSION_TTL_SECONDS`, `DATABASE_URL_FILE`, `S3_ENDPOINT`, `S3_REGION`, `S3_BUCKET`, `S3_ACCESS_KEY_ID_FILE`, `S3_SECRET_ACCESS_KEY_FILE`. Остальные указанные в `.env.example` — справочные интерфейсные переменные skeleton; лимиты задаются в коде (`MAX_BYTES`, `MAX_PIXELS`).
|
||||
|
||||
## Доказательства критериев
|
||||
Безопасно остановить: `docker compose --env-file .env down` — данные сохранены. Намеренно удалить тестовую базу и файлы: `docker compose --env-file .env down -v` — **необратимо** для локальных volumes.
|
||||
|
||||
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
||||
| --- | --- | --- | --- |
|
||||
| Публичный smoke | make test | TODO | TODO |
|
||||
| Остальные пункты уровня 3 | TODO | TODO | TODO |
|
||||
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
|
||||
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
|
||||
## Архитектура
|
||||
|
||||
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
|
||||
```text
|
||||
HTTP client → Nginx :8080 → FastAPI api1 / api2
|
||||
├── PostgreSQL (users, sessions, chats, messages,
|
||||
│ idempotency, attachment metadata/link)
|
||||
└── private MinIO S3 bucket (originals/<random UUID>)
|
||||
```
|
||||
|
||||
## Эксперименты и отказы
|
||||
Из ЛР3 перенесены все endpoints пользователей, Basic→Bearer session, чаты, история с keyset cursor, X-Request-Id/X-Instance-Id, 24h+ идемпотентность отправки сообщения, PostgreSQL, Nginx и две API-реплики, сценарии restart/failover. Сохранены прежние текстовые POST; `attachment_ids` необязателен. В `messages` разрешён `text=""` **только при наличии вложения** (проверка Pydantic + запись ссылок в одной транзакции).
|
||||
|
||||
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
|
||||
Новая миграция `003_attachments.sql` создаёт `attachments` и `message_attachments`. Сообщения, ссылки и ключ идемпотентности фиксируются атомарно в PostgreSQL. Метаданные `Attachment` включают ID, chat/owner, filename, MIME, размер, SHA-256, status=`ready`, timestamps. S3 key сервер генерирует случайно и никогда не возвращает в публичном API. Политика S3 разрешает приложению только bucket list и операции с префиксом `originals/*`; административный доступ остаётся только у bootstrap. S3/DB используют named volumes. Порты S3 и административной консоли не опубликованы на хосте.
|
||||
|
||||
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
|
||||
| --- | --- | --- | --- |
|
||||
| TODO | TODO | TODO | TODO |
|
||||
Upload: авторизация участника → проверка имени/MIME/Content-Length → инкрементальное чтение ≤10 MiB в disk-backed spooled tempfile → Pillow проверяет реальный формат, декодируемость и ≤40 млн пикселей → S3 PUT → COMMIT метаданных → ответ 201. При исключении после PUT выполняется best-effort DELETE из S3. Нет отдельных временно опубликованных локальных файлов, оригинал в API хранится только в spool на время HTTP-запроса. Download: проверка доступа к чату по PostgreSQL → GET из S3 → ответ StreamingResponse с оригинальными bytes, без redirect; чужие ID маскируются 404. Все API-реплики используют общий bucket.
|
||||
|
||||
## Ограничения и следующие шаги
|
||||
**Транзакционная граница:** S3 PUT и COMMIT БД не являются общей ACID-транзакцией. При неудачном DB COMMIT запрашивается компенсационное удаление; если в этот момент S3 также недоступен, возможен orphan. Для оценки 4 необходим отдельный reconciler с grace period; здесь он не заявлен. `ready` возвращается только после успешного commit.
|
||||
|
||||
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
|
||||
|
||||
|
||||
## Ограничения и дальнейшие шаги
|
||||
|
||||
- Стенд учебный: HTTP привязан к localhost, для публичного развёртывания нужен HTTPS/TLS; single-node PostgreSQL, S3 и LB не являются полностью HA.
|
||||
- Проверка повреждённых изображений декодирует уже ограниченные по bytes/pixels данные, но для оценки 4 следует замерить фактический пик RAM под 10+ конкурентными upload.
|
||||
- Очистка orphan после неуспешной компенсации S3 и расширенные fault-injection тесты не реализованы; это задачи уровня 4.
|
||||
- БД enforce-ит ссылочную целостность вложений; условие «пустой текст только с вложениями» проверяется на уровне API, а не SQL CHECK.
|
||||
- Повторение проверок на Windows и Linux описано в `STUDENT_RUN.md`.
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
# ЛР4 — команды запуска (Windows / Linux)
|
||||
|
||||
Автор: **Кимпаев А. Р.**, группа **БПИ2303**, fork: **kimpaev**, ветка: **solution/bpi2303-kimpaev**.
|
||||
|
||||
## Windows 11
|
||||
|
||||
Запустите Docker Desktop в режиме Linux containers. Распакуйте архив в новый каталог. Для чистого запуска достаточно Docker Desktop и штатного PowerShell. Из корня проекта выполните в PowerShell:
|
||||
|
||||
```powershell
|
||||
.\start-windows.cmd
|
||||
```
|
||||
|
||||
Сценарий создаёт локальные секреты, собирает и запускает стенд, ждёт готовности и запрашивает `/health/ready`. Ожидается `{"status":"ok"}`, обе API-реплики в состоянии `healthy`, а `migrate` и `s3-bootstrap` завершены с кодом 0. При ошибке сценарий выводит состояние Compose и логи сервисов. Для интеграционных тестов установите Python 3.9+:
|
||||
|
||||
```powershell
|
||||
py -3 -X utf8 scripts/check_contract.py
|
||||
py -3 tests/smoke.py --base-url http://127.0.0.1:8080
|
||||
py -3 student-tests/attachments_negative.py --base-url http://127.0.0.1:8080
|
||||
```
|
||||
|
||||
Если у вас есть прежние данные ЛР4, сохраните прежние `.env` и каталог `secrets`. Compose связывает тома с именем проекта. Для чистого стенда при наличии прежнего проекта с тем же именем задайте перед запуском `$env:COMPOSE_PROJECT_NAME = 'mtusi2026lab4_fixed'`. Старые контейнеры, занимающие порт 8080, нужно остановить в Docker Desktop.
|
||||
|
||||
## Git Bash / Linux
|
||||
|
||||
```bash
|
||||
sh scripts/setup.sh
|
||||
python3 scripts/check_contract.py
|
||||
docker compose --env-file .env up -d --build --wait --wait-timeout 180
|
||||
python3 tests/smoke.py --base-url http://127.0.0.1:8080
|
||||
python3 student-tests/attachments_negative.py --base-url http://127.0.0.1:8080
|
||||
```
|
||||
|
||||
MinIO и mc берутся из публичного реестра Chainguard (`cgr.dev`).
|
||||
|
||||
## Отказы и сохранность
|
||||
|
||||
В Windows запустите полный прогон из корня проекта:
|
||||
|
||||
```powershell
|
||||
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\verify-windows.ps1
|
||||
```
|
||||
|
||||
Сценарий запускает стенд, выполняет smoke, persistence и failover, после failover возвращает обе реплики в состояние healthy. Журнал сохраняется в `evidence/windows-run-*.txt`. Для отдельных проверок в Linux / WSL:
|
||||
|
||||
```bash
|
||||
make test
|
||||
make test-persistence ACTION_SCRIPT=scripts/restart.sh
|
||||
make test-failover ACTION_SCRIPT=scripts/stop-one.sh
|
||||
docker compose --env-file .env up -d --no-deps --wait api1 api2
|
||||
```
|
||||
|
||||
В persistence сначала перезапускаются PostgreSQL, S3 и обе реплики, затем после их готовности перезапускается Nginx. Тест failover оставляет выбранную реплику остановленной, поэтому последняя команда возвращает её в работу.
|
||||
|
||||
Проверить непубличность S3: сервис `s3` не публикует порт 9000 на хост. Из контейнера временного клиента выполнить анонимный GET к известному объекту и убедиться, что доступ запрещён. Не используйте административные credentials при этой проверке. Для демонстрации ограниченной политики проверьте отдельным сервисным пользователем чтение objects из назначенного bucket и отказ в административных операциях.
|
||||
|
||||
Сохранность данных: `docker compose --env-file .env down` не удаляет volumes. **Безвозвратное удаление только тестовых данных**: `docker compose --env-file .env down -v` — удаляет базу и S3-объекты.
|
||||
|
||||
## PR по CONTRIBUTING.md
|
||||
|
||||
1. Fork **репозитория ЛР4**, clone своего fork (не ЛР3) и переключитесь в новую ветку от skeleton:
|
||||
```bash
|
||||
git checkout -b solution/bpi2303-kimpaev
|
||||
```
|
||||
2. Скопируйте содержимое этого решения **поверх своего клона ЛР4**, без `.git`, `.env`, `secrets/`, volumes. Не заменяйте публичные `tests/`, `contracts/`, `README.md`, `COURSE.md`.
|
||||
3. После реальных проверок занесите их вывод в `REPORT.md` (не выдавайте непроверенные тесты за выполненные).
|
||||
4. Создайте коммит и отправьте ветку:
|
||||
```bash
|
||||
git add app migrations student-tests infra scripts Dockerfile requirements.txt compose.yaml REPORT.md STUDENT_RUN.md WINDOWS11_FIX.md start-windows.cmd verify-windows.ps1 .gitattributes .github/pull_request_template.md evidence/offline-check.txt evidence/windows-verified.txt evidence/additional-checks.txt
|
||||
git commit -m "Implement lab4 S3 attachments, preserve lab3 API"
|
||||
git push -u origin solution/bpi2303-kimpaev
|
||||
```
|
||||
5. Откройте PR: **`ЛР 4 — БПИ2303 — Кимпаев — на 3`**. Base: ветка задания ЛР4 преподавателя; compare: `kimpaev:solution/bpi2303-kimpaev`. Текст PR из `.github/pull_request_template.md`; заполните фактические результаты.
|
||||
|
||||
Если вы собираете проект из готового архива, он уже содержит **skeleton ЛР4** и корректно перенесённые файлы ЛР3. Не создавайте PR в репозиторий ЛР3.
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
name: Lab4 application
|
||||
|
||||
on: [push, pull_request]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
integration:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: '3.12'
|
||||
|
||||
- name: Prepare test credentials
|
||||
run: sh scripts/setup.sh
|
||||
|
||||
- name: Build and start the stack
|
||||
run: docker compose --env-file .env up -d --build --wait --wait-timeout 180
|
||||
|
||||
- name: Check API readiness
|
||||
run: curl --fail --show-error http://127.0.0.1:8080/health/ready
|
||||
|
||||
- name: Run smoke test
|
||||
run: make test PYTHON=python BASE_URL=http://127.0.0.1:8080
|
||||
|
||||
- name: Print diagnostics on failure
|
||||
if: failure()
|
||||
run: |
|
||||
docker compose --env-file .env ps -a
|
||||
docker compose --env-file .env logs --tail=80
|
||||
|
||||
- name: Stop the CI stack
|
||||
if: always()
|
||||
run: docker compose --env-file .env down
|
||||
@@ -0,0 +1,26 @@
|
||||
import hashlib
|
||||
from fastapi import Header, HTTPException
|
||||
from app.storage import db
|
||||
|
||||
BEARER_CHALLENGE = {"WWW-Authenticate": 'Bearer realm="messenger"'}
|
||||
BASIC_CHALLENGE = {"WWW-Authenticate": 'Basic realm="messenger", charset="UTF-8"'}
|
||||
|
||||
|
||||
def unauthorized(basic=False):
|
||||
raise HTTPException(status_code=401, detail="Unauthorized", headers=BASIC_CHALLENGE if basic else BEARER_CHALLENGE)
|
||||
|
||||
|
||||
def current_user(authorization: str | None = Header(default=None)):
|
||||
if not authorization or not authorization.startswith("Bearer "):
|
||||
unauthorized()
|
||||
token = authorization[7:]
|
||||
if not token or " " in token:
|
||||
unauthorized()
|
||||
with db() as conn:
|
||||
user = conn.execute("""SELECT u.id,u.username,u.display_name,u.created_at FROM sessions s
|
||||
JOIN users u ON u.id=s.user_id
|
||||
WHERE s.token_hash=%s AND s.expires_at > now()""",
|
||||
(hashlib.sha256(token.encode()).hexdigest(),)).fetchone()
|
||||
if user is None:
|
||||
unauthorized()
|
||||
return user
|
||||
+69
@@ -0,0 +1,69 @@
|
||||
"""Replicated API. Schema migration is a separate one-shot Compose service."""
|
||||
import os
|
||||
from uuid import uuid4
|
||||
from fastapi import FastAPI, Request, HTTPException
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.responses import JSONResponse
|
||||
from psycopg import OperationalError
|
||||
from app.routers import users, chats, messages, attachments
|
||||
from app.s3 import s3_client,bucket
|
||||
from botocore.exceptions import BotoCoreError, ClientError
|
||||
from app.storage import db
|
||||
|
||||
INSTANCE_ID = os.environ.get('API_INSTANCE_ID') or str(uuid4())
|
||||
app = FastAPI()
|
||||
|
||||
|
||||
def error_code(status):
|
||||
return {400:'invalid_request',401:'unauthorized',404:'not_found',409:'conflict',413:'payload_too_large',415:'unsupported_media_type',429:'rate_limited',503:'unavailable'}.get(status,'internal_error')
|
||||
|
||||
|
||||
def error_response(request,status,message,headers=None):
|
||||
result=JSONResponse(status_code=status,content={'error':{
|
||||
'code':error_code(status),'message':message,'request_id':request.state.request_id}},headers=headers)
|
||||
result.headers['X-Request-Id']=request.state.request_id
|
||||
result.headers['X-Instance-Id']=INSTANCE_ID
|
||||
return result
|
||||
|
||||
|
||||
@app.middleware('http')
|
||||
async def add_request_id(request:Request,call_next):
|
||||
request.state.request_id=str(uuid4())
|
||||
try:
|
||||
response=await call_next(request)
|
||||
except OperationalError:
|
||||
response=error_response(request,503,'Database unavailable')
|
||||
except Exception:
|
||||
response=error_response(request,500,'Internal server error')
|
||||
response.headers['X-Request-Id']=request.state.request_id
|
||||
response.headers['X-Instance-Id']=INSTANCE_ID
|
||||
return response
|
||||
|
||||
|
||||
@app.exception_handler(HTTPException)
|
||||
async def http_error(request,exc):
|
||||
return error_response(request,exc.status_code,str(exc.detail),exc.headers)
|
||||
|
||||
|
||||
@app.exception_handler(RequestValidationError)
|
||||
async def validation_error(request,exc):
|
||||
return error_response(request,400,'Invalid request')
|
||||
|
||||
app.include_router(users.router)
|
||||
app.include_router(chats.router)
|
||||
app.include_router(messages.router)
|
||||
app.include_router(attachments.router)
|
||||
|
||||
@app.get('/health/live')
|
||||
def live():
|
||||
return {'status':'ok'}
|
||||
|
||||
@app.get('/health/ready')
|
||||
def ready():
|
||||
with db() as conn:
|
||||
conn.execute('SELECT 1')
|
||||
try:
|
||||
s3_client().head_bucket(Bucket=bucket())
|
||||
except (ClientError,BotoCoreError,OSError):
|
||||
raise HTTPException(503,'Storage unavailable')
|
||||
return {'status':'ok'}
|
||||
@@ -0,0 +1,5 @@
|
||||
"""One-shot schema migration (never run from individual API replicas)."""
|
||||
from app.storage import migrate
|
||||
if __name__ == '__main__':
|
||||
migrate()
|
||||
print('Migrations completed')
|
||||
@@ -0,0 +1,39 @@
|
||||
from datetime import datetime
|
||||
from app.models.time_format import utc_z
|
||||
from uuid import UUID
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator, field_serializer
|
||||
|
||||
class CreateChat(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
title: str = Field(min_length=1, max_length=100)
|
||||
member_ids: list[UUID] = Field(min_length=1, max_length=100)
|
||||
|
||||
@field_validator("title")
|
||||
@classmethod
|
||||
def title_must_not_be_blank(cls, value: str) -> str:
|
||||
if not value.strip():
|
||||
raise ValueError("title must not be blank")
|
||||
return value
|
||||
|
||||
@field_validator("member_ids")
|
||||
@classmethod
|
||||
def member_ids_must_be_unique(cls, value: list[UUID]) -> list[UUID]:
|
||||
if len(value) != len(set(value)):
|
||||
raise ValueError("member_ids must be unique")
|
||||
return value
|
||||
|
||||
class Chat(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
id: UUID
|
||||
title: str
|
||||
member_ids: list[UUID]
|
||||
created_at: datetime
|
||||
|
||||
@field_serializer("created_at", when_used="json")
|
||||
def serialize_created_at(self, value: datetime) -> str:
|
||||
return utc_z(value)
|
||||
|
||||
class ChatPage(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
items: list[Chat]
|
||||
next_cursor: str | None
|
||||
@@ -0,0 +1,35 @@
|
||||
from datetime import datetime
|
||||
from uuid import UUID
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator, field_serializer, model_validator
|
||||
from app.models.time_format import utc_z
|
||||
|
||||
class CreateMessage(BaseModel):
|
||||
model_config = ConfigDict(extra='forbid')
|
||||
text: str = Field(max_length=4000)
|
||||
attachment_ids: list[UUID] = Field(default_factory=list,max_length=10)
|
||||
|
||||
@model_validator(mode='after')
|
||||
def valid_content(self):
|
||||
if len(set(self.attachment_ids)) != len(self.attachment_ids):
|
||||
raise ValueError('duplicate attachments')
|
||||
if not self.text.strip() and not self.attachment_ids:
|
||||
raise ValueError('message must contain text or attachments')
|
||||
return self
|
||||
|
||||
class Message(BaseModel):
|
||||
model_config = ConfigDict(extra='forbid')
|
||||
id: UUID
|
||||
chat_id: UUID
|
||||
sender_id: UUID
|
||||
text: str
|
||||
created_at: datetime
|
||||
attachment_ids: list[UUID]
|
||||
|
||||
@field_serializer('created_at', when_used='json')
|
||||
def serialize_created_at(self, value: datetime) -> str:
|
||||
return utc_z(value)
|
||||
|
||||
class MessagePage(BaseModel):
|
||||
model_config = ConfigDict(extra='forbid')
|
||||
items: list[Message]
|
||||
next_cursor: str | None
|
||||
@@ -0,0 +1,9 @@
|
||||
"""Contract requires RFC3339 UTC timestamps with a literal Z suffix."""
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
def utc_z(value: datetime) -> str:
|
||||
if value.tzinfo is None:
|
||||
# Database columns are TIMESTAMPTZ, but keep the serializer defensive.
|
||||
value = value.replace(tzinfo=timezone.utc)
|
||||
return value.astimezone(timezone.utc).isoformat().replace('+00:00', 'Z')
|
||||
@@ -0,0 +1,28 @@
|
||||
from datetime import datetime
|
||||
from app.models.time_format import utc_z
|
||||
from uuid import UUID
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator, field_serializer
|
||||
|
||||
class CreateUser(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
username: str = Field(pattern=r"^[a-z0-9_]{3,32}$")
|
||||
display_name: str = Field(min_length=1, max_length=100)
|
||||
password: str = Field(min_length=12, max_length=128)
|
||||
|
||||
@field_validator("display_name")
|
||||
@classmethod
|
||||
def nonempty(cls, value):
|
||||
if not value.strip():
|
||||
raise ValueError("Blank display name")
|
||||
return value
|
||||
|
||||
class User(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
id: UUID
|
||||
username: str
|
||||
display_name: str
|
||||
created_at: datetime
|
||||
|
||||
@field_serializer("created_at", when_used="json")
|
||||
def serialize_created_at(self, value: datetime) -> str:
|
||||
return utc_z(value)
|
||||
@@ -0,0 +1,166 @@
|
||||
"""Bounded image uploads, authorized streaming downloads and private object storage."""
|
||||
import hashlib
|
||||
import os
|
||||
import tempfile
|
||||
import unicodedata
|
||||
from uuid import UUID, uuid4
|
||||
from io import BytesIO
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request, Query
|
||||
from fastapi.responses import StreamingResponse
|
||||
from starlette.concurrency import run_in_threadpool
|
||||
from PIL import Image, UnidentifiedImageError, ImageFile
|
||||
from botocore.exceptions import BotoCoreError, ClientError
|
||||
from app.dependencies import current_user
|
||||
from app.routers.chats import get_chat_row
|
||||
from app.storage import db
|
||||
from app.s3 import s3_client, bucket
|
||||
from app.models.time_format import utc_z
|
||||
from pydantic import BaseModel, ConfigDict, field_serializer
|
||||
from datetime import datetime
|
||||
|
||||
router=APIRouter()
|
||||
MAX_BYTES=10*1024*1024
|
||||
MAX_PIXELS=40_000_000
|
||||
CHUNK=64*1024
|
||||
Image.MAX_IMAGE_PIXELS=MAX_PIXELS
|
||||
ImageFile.LOAD_TRUNCATED_IMAGES=False
|
||||
|
||||
class Attachment(BaseModel):
|
||||
model_config=ConfigDict(extra='forbid')
|
||||
id:UUID
|
||||
chat_id:UUID
|
||||
owner_id:UUID
|
||||
filename:str
|
||||
content_type:str
|
||||
size_bytes:int
|
||||
sha256:str
|
||||
status:str
|
||||
variants:list=[]
|
||||
error_code:str|None=None
|
||||
created_at:datetime
|
||||
updated_at:datetime
|
||||
@field_serializer('created_at','updated_at',when_used='json')
|
||||
def as_utc(self,value:datetime)->str:
|
||||
return utc_z(value)
|
||||
|
||||
def public(row):
|
||||
return {k:row[k] for k in ('id','chat_id','owner_id','filename','content_type','size_bytes',
|
||||
'sha256','status','created_at','updated_at')} | {'variants':[],'error_code':None}
|
||||
|
||||
def check_name(name):
|
||||
if not 1<=len(name)<=255 or any(ch in '/\\' or ord(ch)<32 or 127<=ord(ch)<160 or unicodedata.category(ch)=='Cc' for ch in name):
|
||||
raise HTTPException(400,'Invalid filename')
|
||||
|
||||
def inspect_image(file, declared):
|
||||
file.seek(0)
|
||||
try:
|
||||
with Image.open(file) as img:
|
||||
fmt=img.format
|
||||
w,h=img.size
|
||||
if w<=0 or h<=0 or w*h>MAX_PIXELS:
|
||||
raise HTTPException(413,'Image pixel count exceeded')
|
||||
expected={'PNG':'image/png','JPEG':'image/jpeg'}.get(fmt)
|
||||
if not expected or expected!=declared:
|
||||
raise HTTPException(415,'Image content type mismatch')
|
||||
img.verify()
|
||||
file.seek(0)
|
||||
with Image.open(file) as img:
|
||||
img.load() # reject truncated PNG/JPEG and decompression errors
|
||||
except HTTPException:
|
||||
raise
|
||||
except (UnidentifiedImageError, ValueError, OSError, SyntaxError, Image.DecompressionBombError, Image.DecompressionBombWarning):
|
||||
raise HTTPException(415,'Invalid image')
|
||||
finally:
|
||||
file.seek(0)
|
||||
|
||||
def put_original(file, key, content_type):
|
||||
client=s3_client()
|
||||
file.seek(0)
|
||||
client.upload_fileobj(file,bucket(),key,ExtraArgs={'ContentType':content_type},
|
||||
Config=__import__('boto3').s3.transfer.TransferConfig(
|
||||
multipart_threshold=16*1024*1024,use_threads=False))
|
||||
|
||||
def delete_original(key):
|
||||
s3_client().delete_object(Bucket=bucket(),Key=key)
|
||||
|
||||
@router.post('/api/v1/chats/{chat_id}/attachments',response_model=Attachment,status_code=201)
|
||||
async def upload(chat_id:UUID,request:Request, filename:str=Query(...),user=Depends(current_user)):
|
||||
check_name(filename)
|
||||
declared=request.headers.get('content-type','').lower()
|
||||
if declared not in ('image/png','image/jpeg'):
|
||||
raise HTTPException(415,'Only PNG and JPEG are supported')
|
||||
with db() as conn:
|
||||
if get_chat_row(conn,chat_id,user['id']) is None:
|
||||
raise HTTPException(404,'Chat not found')
|
||||
length=request.headers.get('content-length')
|
||||
if length is not None:
|
||||
try:
|
||||
if int(length)>MAX_BYTES:raise HTTPException(413,'Upload too large')
|
||||
except ValueError:
|
||||
raise HTTPException(400,'Invalid content length')
|
||||
size=0
|
||||
digest=hashlib.sha256()
|
||||
# Disk-backed temporary spool; no API replica retains permanent content.
|
||||
with tempfile.SpooledTemporaryFile(max_size=1024*1024,mode='w+b') as file:
|
||||
async for part in request.stream():
|
||||
size+=len(part)
|
||||
if size>MAX_BYTES:
|
||||
raise HTTPException(413,'Upload too large')
|
||||
digest.update(part)
|
||||
file.write(part)
|
||||
if not size:raise HTTPException(415,'Empty image')
|
||||
await run_in_threadpool(inspect_image,file,declared)
|
||||
ident=uuid4()
|
||||
key='originals/'+str(uuid4())
|
||||
try:
|
||||
await run_in_threadpool(put_original,file,key,declared)
|
||||
except (ClientError,BotoCoreError,OSError):
|
||||
# A partially written object is possible after a transport error.
|
||||
try:await run_in_threadpool(delete_original,key)
|
||||
except Exception:pass # periodic reconciler can remove an orphan after grace
|
||||
raise HTTPException(503,'Storage unavailable')
|
||||
try:
|
||||
with db() as conn:
|
||||
if get_chat_row(conn,chat_id,user['id']) is None:
|
||||
raise HTTPException(404,'Chat not found')
|
||||
row=conn.execute('''INSERT INTO attachments
|
||||
(id,chat_id,owner_id,filename,content_type,size_bytes,sha256,object_key)
|
||||
VALUES (%s,%s,%s,%s,%s,%s,%s,%s) RETURNING *''',
|
||||
(ident,chat_id,user['id'],filename,declared,size,digest.hexdigest(),key)).fetchone()
|
||||
except Exception:
|
||||
try:await run_in_threadpool(delete_original,key)
|
||||
except Exception:pass
|
||||
raise
|
||||
return public(row)
|
||||
|
||||
def authorized(ident,user):
|
||||
with db() as conn:
|
||||
row=conn.execute('''SELECT a.* FROM attachments a
|
||||
JOIN chat_members cm ON cm.chat_id=a.chat_id AND cm.user_id=%s
|
||||
WHERE a.id=%s''',(user['id'],ident)).fetchone()
|
||||
if row is None:raise HTTPException(404,'Attachment not found')
|
||||
return row
|
||||
|
||||
@router.get('/api/v1/attachments/{attachment_id}',response_model=Attachment)
|
||||
def get_attachment(attachment_id:UUID,user=Depends(current_user)):
|
||||
return public(authorized(attachment_id,user))
|
||||
|
||||
@router.get('/api/v1/attachments/{attachment_id}/content')
|
||||
def download(attachment_id:UUID,variant:str='original',user=Depends(current_user)):
|
||||
row=authorized(attachment_id,user)
|
||||
if variant!='original':raise HTTPException(404,'Variant not found')
|
||||
try:
|
||||
obj=s3_client().get_object(Bucket=bucket(),Key=row['object_key'])
|
||||
except (ClientError,BotoCoreError,OSError):
|
||||
raise HTTPException(503,'Storage unavailable')
|
||||
body=obj['Body']
|
||||
def chunks():
|
||||
try:
|
||||
for chunk in body.iter_chunks(chunk_size=CHUNK):
|
||||
if chunk:yield chunk
|
||||
finally:
|
||||
body.close()
|
||||
return StreamingResponse(chunks(),media_type=row['content_type'],headers={
|
||||
'Content-Length':str(row['size_bytes']),
|
||||
'Content-Disposition':'attachment',
|
||||
'X-Content-Type-Options':'nosniff'})
|
||||
@@ -0,0 +1,52 @@
|
||||
from uuid import UUID, uuid4
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query
|
||||
from app.dependencies import current_user
|
||||
from app.models.chats import Chat, ChatPage, CreateChat
|
||||
from app.storage import db
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def get_chat_row(conn, chat_id, user_id):
|
||||
return conn.execute("""SELECT c.id,c.title,c.created_at FROM chats c
|
||||
JOIN chat_members cm ON cm.chat_id=c.id WHERE c.id=%s AND cm.user_id=%s""",
|
||||
(chat_id,user_id)).fetchone()
|
||||
|
||||
|
||||
def chat_output(conn,row):
|
||||
members=conn.execute('SELECT user_id FROM chat_members WHERE chat_id=%s ORDER BY user_id',(row['id'],)).fetchall()
|
||||
return {**row, 'member_ids':[m['user_id'] for m in members]}
|
||||
|
||||
|
||||
@router.post('/api/v1/chats', response_model=Chat, status_code=201)
|
||||
def create_chat(payload:CreateChat,user=Depends(current_user)):
|
||||
if user['id'] not in payload.member_ids:
|
||||
raise HTTPException(status_code=400,detail='Creator must be a chat member')
|
||||
with db() as conn:
|
||||
rows=conn.execute('SELECT id FROM users WHERE id = ANY(%s)',(payload.member_ids,)).fetchall()
|
||||
if len(rows)!=len(payload.member_ids):
|
||||
raise HTTPException(status_code=400,detail='Unknown member')
|
||||
chat=conn.execute('INSERT INTO chats(id,title) VALUES (%s,%s) RETURNING id,title,created_at',
|
||||
(uuid4(),payload.title)).fetchone()
|
||||
for member in payload.member_ids:
|
||||
conn.execute('INSERT INTO chat_members(chat_id,user_id) VALUES (%s,%s)',(chat['id'],member))
|
||||
return chat_output(conn,chat)
|
||||
|
||||
|
||||
@router.get('/api/v1/chats',response_model=ChatPage)
|
||||
def list_chats(limit:int=Query(default=50,ge=1,le=100),user=Depends(current_user)):
|
||||
with db() as conn:
|
||||
rows=conn.execute("""SELECT c.id,c.title,c.created_at FROM chats c
|
||||
JOIN chat_members m ON m.chat_id=c.id WHERE m.user_id=%s
|
||||
ORDER BY c.created_at,c.id LIMIT %s""",(user['id'],limit+1)).fetchall()
|
||||
items=[chat_output(conn,row) for row in rows[:limit]]
|
||||
return {'items':items,'next_cursor':None}
|
||||
|
||||
|
||||
@router.get('/api/v1/chats/{chat_id}',response_model=Chat)
|
||||
def get_chat(chat_id:UUID,user=Depends(current_user)):
|
||||
with db() as conn:
|
||||
row=get_chat_row(conn,chat_id,user['id'])
|
||||
if row is None:
|
||||
raise HTTPException(status_code=404,detail='Chat not found')
|
||||
return chat_output(conn,row)
|
||||
@@ -0,0 +1,113 @@
|
||||
import base64
|
||||
import binascii
|
||||
import json
|
||||
import hashlib
|
||||
import re
|
||||
from datetime import datetime
|
||||
from uuid import UUID,uuid4
|
||||
from fastapi import APIRouter,Depends,HTTPException,Query,Header
|
||||
from app.dependencies import current_user
|
||||
from app.models.messages import CreateMessage,Message,MessagePage
|
||||
from app.routers.chats import get_chat_row
|
||||
from app.storage import db
|
||||
|
||||
router=APIRouter()
|
||||
|
||||
MESSAGE_SELECT = """SELECT m.id,m.chat_id,m.sender_id,m.text,m.created_at,
|
||||
COALESCE((SELECT array_agg(ma.attachment_id ORDER BY ma.position)
|
||||
FROM message_attachments ma WHERE ma.message_id=m.id), ARRAY[]::uuid[]) AS attachment_ids
|
||||
FROM messages m"""
|
||||
|
||||
|
||||
|
||||
def encode_cursor(chat_id,row):
|
||||
raw=json.dumps({'chat_id':str(chat_id),'created_at':row['created_at'].isoformat(),'id':str(row['id'])}).encode()
|
||||
return base64.urlsafe_b64encode(raw).decode().rstrip('=')
|
||||
|
||||
|
||||
def decode_cursor(chat_id,cursor):
|
||||
try:
|
||||
raw=base64.b64decode(cursor+'='*(-len(cursor)%4),altchars=b'-_',validate=True)
|
||||
data=json.loads(raw)
|
||||
ts=datetime.fromisoformat(data['created_at'])
|
||||
ident=UUID(data['id'])
|
||||
if data['chat_id']!=str(chat_id) or ts.tzinfo is None:
|
||||
raise ValueError('Wrong cursor')
|
||||
return ts,ident
|
||||
except (ValueError,KeyError,TypeError,UnicodeDecodeError,binascii.Error):
|
||||
raise HTTPException(status_code=400,detail='Invalid cursor')
|
||||
|
||||
|
||||
IDEMPOTENCY_KEY_RE = re.compile(r'^[A-Za-z0-9._:-]{1,128}$', re.ASCII)
|
||||
|
||||
|
||||
def canonical_message_hash(payload):
|
||||
# Canonical semantics, not raw JSON bytes or field order.
|
||||
data = payload.model_dump(mode='json', exclude_none=True)
|
||||
data.setdefault('attachment_ids', [])
|
||||
return hashlib.sha256(json.dumps(data, sort_keys=True, separators=(',', ':'),
|
||||
ensure_ascii=False).encode('utf-8')).hexdigest()
|
||||
|
||||
|
||||
def advisory_lock_id(user_id,chat_id,key):
|
||||
scope = f'{user_id}:{chat_id}:{key}'.encode('utf-8')
|
||||
return int.from_bytes(hashlib.sha256(scope).digest()[:8], 'big', signed=True)
|
||||
|
||||
|
||||
@router.post('/api/v1/chats/{chat_id}/messages',response_model=Message,status_code=201)
|
||||
def create_message(chat_id:UUID,payload:CreateMessage,user=Depends(current_user),
|
||||
idempotency_key:str|None=Header(default=None,alias='Idempotency-Key')):
|
||||
if idempotency_key is None or not IDEMPOTENCY_KEY_RE.fullmatch(idempotency_key):
|
||||
raise HTTPException(status_code=400,detail='Invalid or missing Idempotency-Key')
|
||||
request_hash=canonical_message_hash(payload)
|
||||
with db() as conn:
|
||||
# Explicit authorization is checked on EVERY replay, not just initial POST.
|
||||
if get_chat_row(conn,chat_id,user['id']) is None:
|
||||
raise HTTPException(status_code=404,detail='Chat not found')
|
||||
# Same key on different processes serializes in PostgreSQL. The message and
|
||||
# key record commit together, so a network timeout cannot create duplicates.
|
||||
conn.execute('SELECT pg_advisory_xact_lock(%s)',
|
||||
(advisory_lock_id(user['id'],chat_id,idempotency_key),))
|
||||
saved=conn.execute("""SELECT request_hash,message_id,created_at FROM message_idempotency
|
||||
WHERE user_id=%s AND chat_id=%s AND idem_key=%s FOR UPDATE""",
|
||||
(user['id'],chat_id,idempotency_key)).fetchone()
|
||||
if saved is not None:
|
||||
# Contract promises a replay window of at least 24 hours; retaining
|
||||
# keys longer is safe and prevents late duplicate deliveries.
|
||||
if saved['request_hash'] != request_hash:
|
||||
raise HTTPException(status_code=409,detail='Idempotency-Key used with another body')
|
||||
return conn.execute(MESSAGE_SELECT + """ WHERE m.id=%s""",(saved['message_id'],)).fetchone()
|
||||
if payload.attachment_ids:
|
||||
authorized=conn.execute("""SELECT id FROM attachments
|
||||
WHERE chat_id=%s AND id=ANY(%s) AND status='ready' FOR SHARE""",
|
||||
(chat_id,payload.attachment_ids)).fetchall()
|
||||
if len(authorized)!=len(payload.attachment_ids):
|
||||
raise HTTPException(status_code=404,detail='Attachment not found')
|
||||
row=conn.execute("""INSERT INTO messages(id,chat_id,sender_id,text)
|
||||
VALUES (%s,%s,%s,%s) RETURNING id,chat_id,sender_id,text,created_at""",
|
||||
(uuid4(),chat_id,user['id'],payload.text)).fetchone()
|
||||
for pos,attachment_id in enumerate(payload.attachment_ids):
|
||||
conn.execute("""INSERT INTO message_attachments(message_id,attachment_id,position)
|
||||
VALUES (%s,%s,%s)""",(row['id'],attachment_id,pos))
|
||||
row['attachment_ids']=payload.attachment_ids
|
||||
conn.execute("""INSERT INTO message_idempotency(user_id,chat_id,idem_key,request_hash,message_id)
|
||||
VALUES (%s,%s,%s,%s,%s)""",
|
||||
(user['id'],chat_id,idempotency_key,request_hash,row['id']))
|
||||
return row
|
||||
|
||||
|
||||
@router.get('/api/v1/chats/{chat_id}/messages',response_model=MessagePage)
|
||||
def list_messages(chat_id:UUID,limit:int=Query(default=50,ge=1,le=100),
|
||||
cursor:str|None=Query(default=None,min_length=1),user=Depends(current_user)):
|
||||
with db() as conn:
|
||||
if get_chat_row(conn,chat_id,user['id']) is None:
|
||||
raise HTTPException(status_code=404,detail='Chat not found')
|
||||
if cursor is not None:
|
||||
ts,ident=decode_cursor(chat_id,cursor)
|
||||
rows=conn.execute(MESSAGE_SELECT + """ WHERE m.chat_id=%s AND (m.created_at,m.id)>(%s,%s)
|
||||
ORDER BY m.created_at,m.id LIMIT %s""",(chat_id,ts,ident,limit+1)).fetchall()
|
||||
else:
|
||||
rows=conn.execute(MESSAGE_SELECT + """ WHERE m.chat_id=%s ORDER BY m.created_at,m.id LIMIT %s""",(chat_id,limit+1)).fetchall()
|
||||
items=rows[:limit]
|
||||
next_cursor=encode_cursor(chat_id,items[-1]) if len(rows)>limit else None
|
||||
return {'items':items,'next_cursor':next_cursor}
|
||||
@@ -0,0 +1,78 @@
|
||||
import base64
|
||||
import binascii
|
||||
import hashlib
|
||||
import hmac
|
||||
import os
|
||||
import secrets
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from uuid import uuid4
|
||||
from fastapi import APIRouter, Depends, Header, HTTPException, Response
|
||||
from psycopg.errors import UniqueViolation
|
||||
from app.dependencies import current_user, unauthorized
|
||||
from app.models.users import CreateUser, User
|
||||
from app.models.time_format import utc_z
|
||||
from app.storage import db
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def password_hash(password):
|
||||
salt = os.urandom(16)
|
||||
derived = hashlib.scrypt(password.encode('utf-8'), salt=salt, n=2**14, r=8, p=1, dklen=32)
|
||||
return 'scrypt$' + salt.hex() + '$' + derived.hex()
|
||||
|
||||
|
||||
def verify_password(password, encoded):
|
||||
try:
|
||||
_, salt_hex, expected_hex = encoded.split('$')
|
||||
actual = hashlib.scrypt(password.encode('utf-8'),salt=bytes.fromhex(salt_hex),n=2**14,r=8,p=1,dklen=32)
|
||||
return hmac.compare_digest(actual, bytes.fromhex(expected_hex))
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
|
||||
|
||||
@router.post('/api/v1/users', response_model=User, status_code=201)
|
||||
def register_user(payload: CreateUser):
|
||||
with db() as conn:
|
||||
try:
|
||||
user = conn.execute("""INSERT INTO users(id,username,display_name,password_hash)
|
||||
VALUES (%s,%s,%s,%s) RETURNING id,username,display_name,created_at""",
|
||||
(uuid4(),payload.username,payload.display_name,password_hash(payload.password))).fetchone()
|
||||
except UniqueViolation:
|
||||
raise HTTPException(status_code=409,detail='Username already exists')
|
||||
return user
|
||||
|
||||
|
||||
@router.get('/api/v1/users/me', response_model=User)
|
||||
def get_current_user(user=Depends(current_user)):
|
||||
return user
|
||||
|
||||
|
||||
@router.post('/api/v1/auth/sessions', status_code=201)
|
||||
def create_session(authorization: str | None = Header(default=None)):
|
||||
if not authorization or not authorization.startswith('Basic '):
|
||||
unauthorized(basic=True)
|
||||
try:
|
||||
raw = base64.b64decode(authorization[6:],validate=True).decode('utf-8')
|
||||
username, password = raw.split(':',1)
|
||||
except (ValueError, UnicodeDecodeError, binascii.Error):
|
||||
unauthorized(basic=True)
|
||||
with db() as conn:
|
||||
row = conn.execute('SELECT * FROM users WHERE username=%s',(username,)).fetchone()
|
||||
if row is None or not verify_password(password,row['password_hash']):
|
||||
unauthorized(basic=True)
|
||||
token = secrets.token_urlsafe(48)
|
||||
ttl = int(os.getenv('SESSION_TTL_SECONDS','3600'))
|
||||
expires = datetime.now(timezone.utc)+timedelta(seconds=ttl)
|
||||
conn.execute('INSERT INTO sessions(token_hash,user_id,expires_at) VALUES (%s,%s,%s)',
|
||||
(hashlib.sha256(token.encode()).hexdigest(),row['id'],expires))
|
||||
return {'token':token,'token_type':'Bearer','expires_at':utc_z(expires),
|
||||
'user':User.model_validate({key:row[key] for key in ('id','username','display_name','created_at')})}
|
||||
|
||||
|
||||
@router.delete('/api/v1/auth/sessions/current',status_code=204)
|
||||
def logout(authorization: str | None = Header(default=None),user=Depends(current_user)):
|
||||
token=authorization[7:]
|
||||
with db() as conn:
|
||||
conn.execute('DELETE FROM sessions WHERE token_hash=%s',(hashlib.sha256(token.encode()).hexdigest(),))
|
||||
return Response(status_code=204)
|
||||
@@ -0,0 +1,19 @@
|
||||
"""S3 access with least-privilege application credentials (never root credentials)."""
|
||||
import os
|
||||
from pathlib import Path
|
||||
import boto3
|
||||
from botocore.config import Config
|
||||
|
||||
|
||||
def s3_client():
|
||||
return boto3.client(
|
||||
's3', endpoint_url=os.environ['S3_ENDPOINT'],
|
||||
region_name=os.getenv('S3_REGION','us-east-1'),
|
||||
aws_access_key_id=Path(os.environ['S3_ACCESS_KEY_ID_FILE']).read_text().strip(),
|
||||
aws_secret_access_key=Path(os.environ['S3_SECRET_ACCESS_KEY_FILE']).read_text().strip(),
|
||||
config=Config(connect_timeout=2,read_timeout=15,retries={'max_attempts':2},
|
||||
s3={'addressing_style':'path'}))
|
||||
|
||||
|
||||
def bucket():
|
||||
return os.getenv('S3_BUCKET','messenger-attachments')
|
||||
@@ -0,0 +1,31 @@
|
||||
"""PostgreSQL is the source of truth, including bearer sessions."""
|
||||
import os
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
import psycopg
|
||||
from psycopg.rows import dict_row
|
||||
|
||||
|
||||
def database_url():
|
||||
secret_file = os.getenv("DATABASE_URL_FILE")
|
||||
if secret_file:
|
||||
return Path(secret_file).read_text().strip()
|
||||
return os.environ["DATABASE_URL"]
|
||||
|
||||
|
||||
@contextmanager
|
||||
def db():
|
||||
with psycopg.connect(database_url(), row_factory=dict_row, connect_timeout=5) as conn:
|
||||
yield conn
|
||||
|
||||
|
||||
def migrate():
|
||||
with db() as conn:
|
||||
# Serialize migration execution even if two operators race to bootstrap.
|
||||
conn.execute("SELECT pg_advisory_xact_lock(83475391)")
|
||||
conn.execute("CREATE TABLE IF NOT EXISTS schema_migrations (version TEXT PRIMARY KEY)")
|
||||
for file in sorted((Path(__file__).parent.parent / "migrations").glob("*.sql")):
|
||||
if conn.execute("SELECT 1 FROM schema_migrations WHERE version=%s", (file.name,)).fetchone():
|
||||
continue
|
||||
conn.execute(file.read_text())
|
||||
conn.execute("INSERT INTO schema_migrations(version) VALUES (%s)", (file.name,))
|
||||
@@ -0,0 +1,40 @@
|
||||
2026-11-11T17:03:17.0735085+03:00
|
||||
Negative attachment checks
|
||||
PASS: invalid size/image/MIME/filename, outsider ACL and cross-chat bindings
|
||||
Direct S3 access control
|
||||
PASS: authenticated S3 GET succeeded and anonymous GET of the same object returned HTTP 403
|
||||
Upload replica shutdown and S3 outage
|
||||
Container mtusi2026lab4_verified-api2-1 Stopping
|
||||
Container mtusi2026lab4_verified-api2-1 Stopped
|
||||
PASS: upload via api2, stop api2, download via api1, SHA-256 99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75
|
||||
Container mtusi2026lab4_verified-api1-1 Running
|
||||
Container mtusi2026lab4_verified-api2-1 Starting
|
||||
Container mtusi2026lab4_verified-api2-1 Started
|
||||
Container mtusi2026lab4_verified-api2-1 Waiting
|
||||
Container mtusi2026lab4_verified-api1-1 Waiting
|
||||
Container mtusi2026lab4_verified-api1-1 Healthy
|
||||
Container mtusi2026lab4_verified-api2-1 Healthy
|
||||
Container mtusi2026lab4_verified-s3-1 Stopping
|
||||
Container mtusi2026lab4_verified-s3-1 Stopped
|
||||
PASS: stopped S3 returned HTTP 503 for readiness and valid PNG upload
|
||||
Container mtusi2026lab4_verified-s3-1 Starting
|
||||
Container mtusi2026lab4_verified-s3-1 Started
|
||||
PASS: readiness recovered after S3 restart
|
||||
Final readiness
|
||||
{"status":"ok"}
|
||||
Final container state
|
||||
api1 Up 3 minutes (unhealthy)
|
||||
api2 Up About a minute (unhealthy)
|
||||
db Up 3 minutes (healthy)
|
||||
lb Up 3 minutes
|
||||
migrate Exited (0) 3 minutes ago
|
||||
s3 Up 1 second
|
||||
s3-bootstrap Exited (0) 3 minutes ago
|
||||
State after healthchecks converged
|
||||
api1 Up 3 minutes (healthy)
|
||||
api2 Up About a minute (healthy)
|
||||
db Up 3 minutes (healthy)
|
||||
lb Up 3 minutes
|
||||
migrate Exited (0) 3 minutes ago
|
||||
s3 Up 8 seconds
|
||||
s3-bootstrap Exited (0) 3 minutes ago
|
||||
@@ -0,0 +1,8 @@
|
||||
2026-11-11
|
||||
PASS py -3 -X utf8 scripts/check_contract.py
|
||||
PASS Python AST parsing (25 files)
|
||||
PASS Windows PowerShell setup (six files, matching PostgreSQL password, no CR/LF)
|
||||
PASS Windows PowerShell setup rerun (all six hashes unchanged)
|
||||
PASS docker compose --env-file .env config --quiet (Compose v5.6.1)
|
||||
See windows-verified.txt and additional-checks.txt for Docker integration results.
|
||||
PASS POSIX shell syntax and CRLF normalization (Git Bash)
|
||||
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,59 @@
|
||||
2026-11-11T17:00:35.1451077+03:00
|
||||
--- Docker Engine ---
|
||||
linux
|
||||
Docker Engine exit code = 0
|
||||
--- Python ---
|
||||
Python 3.15.0
|
||||
Python exit code = 0
|
||||
--- Startup and readiness ---
|
||||
{"status":"ok"}
|
||||
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
|
||||
mtusi2026lab4_verified-api1-1 sha256:46c6fa875a96226ff6edbf016ac743907c36a0dae253f3a101661e7d877ef142 "uvicorn app.main:ap…" api1 2 minutes ago Up 20 seconds (healthy) 8080/tcp
|
||||
mtusi2026lab4_verified-api2-1 sha256:ab5b93ca1b43b1b4ca873d209996c6dcc432f35df57eb261ba454b10f9d9b923 "uvicorn app.main:ap…" api2 2 minutes ago Up About a minute (healthy) 8080/tcp
|
||||
mtusi2026lab4_verified-db-1 postgres:16.8-alpine "docker-entrypoint.s…" db 2 minutes ago Up About a minute (healthy) 5432/tcp
|
||||
mtusi2026lab4_verified-lb-1 nginx:1.27.5-alpine "/docker-entrypoint.…" lb 2 minutes ago Up About a minute 127.0.0.1:8080->8080/tcp
|
||||
mtusi2026lab4_verified-migrate-1 sha256:372898c88c95ea3b004c908e0a3e45b6ea4a34378676e8801db33fa03ff487c3 "python -m app.migra…" migrate 2 minutes ago Exited (0) 1 second ago
|
||||
mtusi2026lab4_verified-s3-1 cgr.dev/chainguard/minio@sha256:fb893e36fc369d41e92f678b17ec27378fa5bf0576aeef7ac15460c13629ab5a "/usr/bin/minio serv…" s3 About a minute ago Up About a minute
|
||||
mtusi2026lab4_verified-s3-bootstrap-1 cgr.dev/chainguard/minio-client@sha256:fe649f71872c299489deb3cb9e057e5d2f76a45c85b7599f7c21616ac80b5e63 "/bin/sh /etc/bootst…" s3-bootstrap About a minute ago Exited (0) 2 seconds ago
|
||||
Startup and readiness exit code = 0
|
||||
--- Smoke ---
|
||||
PASS: lab4 public API scenario; infrastructure and grading evidence remain separate.
|
||||
Smoke exit code = 0
|
||||
--- Persistence ---
|
||||
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
|
||||
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
Persistence exit code = 0
|
||||
--- Failover ---
|
||||
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
|
||||
{"observed_instances_before": 2, "successful_writes": 67, "failed_attempts": 0, "first_recovery_seconds": 1.079}
|
||||
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
Failover exit code = 0
|
||||
--- State after failover ---
|
||||
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
|
||||
mtusi2026lab4_verified-api1-1 sha256:46c6fa875a96226ff6edbf016ac743907c36a0dae253f3a101661e7d877ef142 "uvicorn app.main:ap…" api1 2 minutes ago Up 26 seconds (healthy) 8080/tcp
|
||||
mtusi2026lab4_verified-api2-1 sha256:ab5b93ca1b43b1b4ca873d209996c6dcc432f35df57eb261ba454b10f9d9b923 "uvicorn app.main:ap…" api2 2 minutes ago Exited (0) 16 seconds ago
|
||||
mtusi2026lab4_verified-db-1 postgres:16.8-alpine "docker-entrypoint.s…" db 2 minutes ago Up 26 seconds (healthy) 5432/tcp
|
||||
mtusi2026lab4_verified-lb-1 nginx:1.27.5-alpine "/docker-entrypoint.…" lb 2 minutes ago Up 19 seconds 127.0.0.1:8080->8080/tcp
|
||||
mtusi2026lab4_verified-migrate-1 sha256:372898c88c95ea3b004c908e0a3e45b6ea4a34378676e8801db33fa03ff487c3 "python -m app.migra…" migrate 2 minutes ago Exited (0) 34 seconds ago
|
||||
mtusi2026lab4_verified-s3-1 cgr.dev/chainguard/minio@sha256:fb893e36fc369d41e92f678b17ec27378fa5bf0576aeef7ac15460c13629ab5a "/usr/bin/minio serv…" s3 2 minutes ago Up 26 seconds
|
||||
mtusi2026lab4_verified-s3-bootstrap-1 cgr.dev/chainguard/minio-client@sha256:fe649f71872c299489deb3cb9e057e5d2f76a45c85b7599f7c21616ac80b5e63 "/bin/sh /etc/bootst…" s3-bootstrap 2 minutes ago Exited (0) 35 seconds ago
|
||||
State after failover exit code = 0
|
||||
--- Restore replicas ---
|
||||
Container mtusi2026lab4_verified-api1-1 Running
|
||||
Container mtusi2026lab4_verified-api2-1 Starting
|
||||
Container mtusi2026lab4_verified-api2-1 Started
|
||||
Container mtusi2026lab4_verified-api1-1 Waiting
|
||||
Container mtusi2026lab4_verified-api2-1 Waiting
|
||||
Container mtusi2026lab4_verified-api1-1 Healthy
|
||||
Container mtusi2026lab4_verified-api2-1 Healthy
|
||||
Restore replicas exit code = 0
|
||||
--- Final state ---
|
||||
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
|
||||
mtusi2026lab4_verified-api1-1 sha256:46c6fa875a96226ff6edbf016ac743907c36a0dae253f3a101661e7d877ef142 "uvicorn app.main:ap…" api1 2 minutes ago Up 33 seconds (healthy) 8080/tcp
|
||||
mtusi2026lab4_verified-api2-1 sha256:ab5b93ca1b43b1b4ca873d209996c6dcc432f35df57eb261ba454b10f9d9b923 "uvicorn app.main:ap…" api2 2 minutes ago Up 6 seconds (healthy) 8080/tcp
|
||||
mtusi2026lab4_verified-db-1 postgres:16.8-alpine "docker-entrypoint.s…" db 2 minutes ago Up 33 seconds (healthy) 5432/tcp
|
||||
mtusi2026lab4_verified-lb-1 nginx:1.27.5-alpine "/docker-entrypoint.…" lb 2 minutes ago Up 26 seconds 127.0.0.1:8080->8080/tcp
|
||||
mtusi2026lab4_verified-migrate-1 sha256:372898c88c95ea3b004c908e0a3e45b6ea4a34378676e8801db33fa03ff487c3 "python -m app.migra…" migrate 2 minutes ago Exited (0) 41 seconds ago
|
||||
mtusi2026lab4_verified-s3-1 cgr.dev/chainguard/minio@sha256:fb893e36fc369d41e92f678b17ec27378fa5bf0576aeef7ac15460c13629ab5a "/usr/bin/minio serv…" s3 2 minutes ago Up 33 seconds
|
||||
mtusi2026lab4_verified-s3-bootstrap-1 cgr.dev/chainguard/minio-client@sha256:fe649f71872c299489deb3cb9e057e5d2f76a45c85b7599f7c21616ac80b5e63 "/bin/sh /etc/bootst…" s3-bootstrap 2 minutes ago Exited (0) 42 seconds ago
|
||||
Final state exit code = 0
|
||||
@@ -0,0 +1,35 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
read_secret() {
|
||||
tr -d '\r\n' < "/run/secrets/$1"
|
||||
}
|
||||
|
||||
ROOT_ACCESS=$(read_secret s3_root_access)
|
||||
ROOT_SECRET=$(read_secret s3_root_secret)
|
||||
API_ACCESS=$(read_secret s3_access_key_id)
|
||||
API_SECRET=$(read_secret s3_secret_access_key)
|
||||
|
||||
# Bound bootstrap wait: fail with diagnostic rather than stall forever.
|
||||
count=0
|
||||
until output=$(timeout 4 mc alias set local http://s3:9000 "$ROOT_ACCESS" "$ROOT_SECRET" 2>&1) && output=$(timeout 4 mc admin info local 2>&1); do
|
||||
count=$((count + 1))
|
||||
if [ "$count" -ge 12 ]; then
|
||||
echo "ERROR: MinIO did not become reachable after 12 attempts; inspect docker compose logs s3" >&2
|
||||
if [ -n "$output" ]; then printf '%s\n' "$output" >&2; fi
|
||||
exit 1
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
|
||||
mc mb --ignore-existing local/messenger-attachments
|
||||
mc anonymous set none local/messenger-attachments
|
||||
# Repeatable bootstrap: policy may exist already after restarts.
|
||||
mc admin policy create local messenger-app /etc/bootstrap/s3-policy.json
|
||||
mc admin user add local "$API_ACCESS" "$API_SECRET"
|
||||
if ! mc admin policy attach local messenger-app --user "$API_ACCESS"; then
|
||||
mc admin policy entities local messenger-app --user "$API_ACCESS" --json | grep -Fq "\"$API_ACCESS\""
|
||||
fi
|
||||
mc anonymous get local/messenger-attachments
|
||||
mc alias set api http://s3:9000 "$API_ACCESS" "$API_SECRET" >/dev/null
|
||||
mc ls api/messenger-attachments >/dev/null
|
||||
printf 'S3 bootstrap completed (private bucket, limited API identity).\n'
|
||||
@@ -0,0 +1,26 @@
|
||||
worker_processes auto;
|
||||
events { worker_connections 1024; }
|
||||
http {
|
||||
resolver 127.0.0.11 valid=1s ipv6=off;
|
||||
upstream messenger_api {
|
||||
zone messenger_api 64k;
|
||||
server api1:8080 resolve max_fails=1 fail_timeout=2s;
|
||||
server api2:8080 resolve max_fails=1 fail_timeout=2s;
|
||||
}
|
||||
server {
|
||||
listen 8080;
|
||||
client_max_body_size 11m;
|
||||
location / {
|
||||
proxy_pass http://messenger_api;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_connect_timeout 1s;
|
||||
proxy_read_timeout 35s;
|
||||
proxy_send_timeout 35s;
|
||||
proxy_next_upstream error timeout http_502 http_503;
|
||||
proxy_next_upstream_tries 2;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["s3:ListBucket"],"Resource":["arn:aws:s3:::messenger-attachments"]},{"Effect":"Allow","Action":["s3:GetObject","s3:PutObject","s3:DeleteObject"],"Resource":["arn:aws:s3:::messenger-attachments/originals/*"]}]}
|
||||
@@ -0,0 +1,35 @@
|
||||
CREATE TABLE IF NOT EXISTS users (
|
||||
id UUID PRIMARY KEY,
|
||||
username VARCHAR(32) NOT NULL UNIQUE,
|
||||
display_name VARCHAR(100) NOT NULL,
|
||||
password_hash TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT username_format CHECK (username ~ '^[a-z0-9_]{3,32}$')
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS chats (
|
||||
id UUID PRIMARY KEY,
|
||||
title VARCHAR(100) NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS chat_members (
|
||||
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
user_id UUID NOT NULL REFERENCES users(id),
|
||||
PRIMARY KEY (chat_id, user_id)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS chat_members_user ON chat_members(user_id, chat_id);
|
||||
CREATE TABLE IF NOT EXISTS messages (
|
||||
id UUID PRIMARY KEY,
|
||||
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
sender_id UUID NOT NULL REFERENCES users(id),
|
||||
text VARCHAR(4000) NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT nonblank_message CHECK (length(btrim(text)) > 0)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS messages_chat_cursor ON messages(chat_id, created_at, id);
|
||||
CREATE TABLE IF NOT EXISTS sessions (
|
||||
token_hash CHAR(64) PRIMARY KEY,
|
||||
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
expires_at TIMESTAMPTZ NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS sessions_expiry ON sessions(expires_at);
|
||||
@@ -0,0 +1,11 @@
|
||||
-- Lab3: one atomic idempotency result per user, chat and key.
|
||||
CREATE TABLE IF NOT EXISTS message_idempotency (
|
||||
user_id UUID NOT NULL REFERENCES users(id),
|
||||
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
idem_key VARCHAR(128) NOT NULL,
|
||||
request_hash CHAR(64) NOT NULL,
|
||||
message_id UUID NOT NULL UNIQUE REFERENCES messages(id) ON DELETE CASCADE,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
PRIMARY KEY (user_id, chat_id, idem_key)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS message_idempotency_expiry ON message_idempotency(created_at);
|
||||
@@ -0,0 +1,25 @@
|
||||
-- A file is owned by exactly one chat. Binary bytes live only in private S3.
|
||||
CREATE TABLE attachments (
|
||||
id UUID PRIMARY KEY,
|
||||
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
owner_id UUID NOT NULL REFERENCES users(id),
|
||||
filename VARCHAR(255) NOT NULL,
|
||||
content_type VARCHAR(32) NOT NULL CHECK (content_type IN ('image/png','image/jpeg')),
|
||||
size_bytes INTEGER NOT NULL CHECK (size_bytes BETWEEN 1 AND 10485760),
|
||||
sha256 CHAR(64) NOT NULL,
|
||||
object_key VARCHAR(160) NOT NULL UNIQUE,
|
||||
status VARCHAR(20) NOT NULL DEFAULT 'ready' CHECK (status='ready'),
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX attachments_chat_idx ON attachments(chat_id);
|
||||
ALTER TABLE messages DROP CONSTRAINT nonblank_message;
|
||||
-- Blank text is permitted only with attachment IDs; enforced by application transaction.
|
||||
CREATE TABLE message_attachments (
|
||||
message_id UUID NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
|
||||
attachment_id UUID NOT NULL REFERENCES attachments(id),
|
||||
position SMALLINT NOT NULL CHECK (position BETWEEN 0 AND 9),
|
||||
PRIMARY KEY (message_id, attachment_id),
|
||||
UNIQUE (message_id, position)
|
||||
);
|
||||
CREATE INDEX message_attachments_attachment_idx ON message_attachments(attachment_id);
|
||||
@@ -0,0 +1,5 @@
|
||||
fastapi==0.115.12
|
||||
uvicorn==0.34.2
|
||||
psycopg[binary]==3.2.6
|
||||
boto3==1.37.38
|
||||
Pillow==11.2.1
|
||||
@@ -0,0 +1,10 @@
|
||||
@echo off
|
||||
setlocal
|
||||
cd /d "%~dp0.."
|
||||
rem Preserve PostgreSQL named volume, tokens and secrets.
|
||||
docker compose --env-file .env restart -t 5 db s3 api1 api2
|
||||
if errorlevel 1 exit /b 1
|
||||
docker compose --env-file .env up -d --no-deps --wait --wait-timeout 55 api1 api2
|
||||
if errorlevel 1 exit /b 1
|
||||
docker compose --env-file .env restart -t 5 lb
|
||||
exit /b %ERRORLEVEL%
|
||||
@@ -0,0 +1,8 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
# Keep PostgreSQL named volume and the generated DB credentials unchanged.
|
||||
# Restart shared DB and both API replicas; restart proxy to refresh backends.
|
||||
docker compose --env-file .env restart -t 5 db s3 api1 api2
|
||||
docker compose --env-file .env up -d --no-deps --wait --wait-timeout 55 api1 api2
|
||||
docker compose --env-file .env restart -t 5 lb
|
||||
@@ -0,0 +1,5 @@
|
||||
@echo off
|
||||
setlocal
|
||||
cd /d "%~dp0.."
|
||||
docker compose --env-file .env up -d --no-deps --wait --wait-timeout 55 api1 api2
|
||||
exit /b %ERRORLEVEL%
|
||||
@@ -0,0 +1,4 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
docker compose --env-file .env start api1 api2
|
||||
@@ -0,0 +1,36 @@
|
||||
$ErrorActionPreference = 'Stop'
|
||||
Set-Location (Join-Path $PSScriptRoot '..')
|
||||
|
||||
if (!(Test-Path -LiteralPath '.env')) {
|
||||
Copy-Item -LiteralPath '.env.example' -Destination '.env'
|
||||
}
|
||||
New-Item -ItemType Directory -Path 'secrets' -Force | Out-Null
|
||||
|
||||
function New-SecretFile([string]$Name, [int]$Length) {
|
||||
$path = Join-Path (Get-Location) "secrets/$Name"
|
||||
if (!(Test-Path -LiteralPath $path)) {
|
||||
$bytes = [byte[]]::new($Length)
|
||||
$rng = [System.Security.Cryptography.RandomNumberGenerator]::Create()
|
||||
try {
|
||||
$rng.GetBytes($bytes)
|
||||
} finally {
|
||||
$rng.Dispose()
|
||||
}
|
||||
$value = [BitConverter]::ToString($bytes).Replace('-', '').ToLowerInvariant()
|
||||
[IO.File]::WriteAllText($path, $value, [Text.Encoding]::ASCII)
|
||||
}
|
||||
}
|
||||
|
||||
New-SecretFile 'db_password' 24
|
||||
New-SecretFile 's3_root_access' 10
|
||||
New-SecretFile 's3_root_secret' 20
|
||||
New-SecretFile 's3_access_key_id' 10
|
||||
New-SecretFile 's3_secret_access_key' 20
|
||||
|
||||
if (!(Test-Path -LiteralPath 'secrets/database_url')) {
|
||||
$password = (Get-Content -LiteralPath 'secrets/db_password' -Raw).TrimEnd([char]10)
|
||||
$encoded = [Uri]::EscapeDataString($password)
|
||||
$url = "postgresql://messenger:${encoded}@db:5432/messenger"
|
||||
[IO.File]::WriteAllText((Join-Path (Get-Location) 'secrets/database_url'), $url, [Text.Encoding]::ASCII)
|
||||
}
|
||||
Write-Host 'Local secrets are ready.'
|
||||
@@ -0,0 +1,22 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
if [ ! -e .env ]; then cp .env.example .env; fi
|
||||
mkdir -p secrets
|
||||
if [ ! -e secrets/db_password ]; then
|
||||
python3 -c 'import secrets;print(secrets.token_urlsafe(36),end="")' > secrets/db_password
|
||||
fi
|
||||
if [ ! -e secrets/database_url ]; then
|
||||
python3 -c 'from pathlib import Path;from urllib.parse import quote; p=Path("secrets/db_password").read_text().strip();print("postgresql://messenger:"+quote(p,safe="")+"@db:5432/messenger",end="")' > secrets/database_url
|
||||
fi
|
||||
python3 - <<'PYGEN'
|
||||
from pathlib import Path
|
||||
import secrets
|
||||
root=Path('secrets')
|
||||
for name,length in [('s3_root_access',10),('s3_root_secret',20),('s3_access_key_id',10),('s3_secret_access_key',20)]:
|
||||
file=root/name
|
||||
if not file.exists():
|
||||
file.write_bytes(secrets.token_hex(length).encode('ascii'))
|
||||
PYGEN
|
||||
chmod 600 secrets/* .env
|
||||
printf 'Local secrets prepared (values not displayed).\n'
|
||||
@@ -0,0 +1,10 @@
|
||||
@echo off
|
||||
setlocal
|
||||
cd /d "%~dp0.."
|
||||
if /I "%TARGET_INSTANCE_ID%"=="api1" goto stop
|
||||
if /I "%TARGET_INSTANCE_ID%"=="api2" goto stop
|
||||
echo Invalid TARGET_INSTANCE_ID; refusing to stop an arbitrary service 1>&2
|
||||
exit /b 2
|
||||
:stop
|
||||
docker compose --env-file .env stop -t 2 %TARGET_INSTANCE_ID%
|
||||
exit /b %ERRORLEVEL%
|
||||
@@ -0,0 +1,7 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
case "${TARGET_INSTANCE_ID:-}" in
|
||||
api1|api2) exec docker compose --env-file .env stop -t 2 "$TARGET_INSTANCE_ID" ;;
|
||||
*) echo 'Unknown TARGET_INSTANCE_ID; refusing to stop an arbitrary service' >&2; exit 2 ;;
|
||||
esac
|
||||
@@ -0,0 +1,30 @@
|
||||
@echo off
|
||||
setlocal
|
||||
cd /d "%~dp0"
|
||||
where docker >nul 2>&1
|
||||
if errorlevel 1 (
|
||||
echo Docker CLI was not found. Start Docker Desktop with Linux containers.
|
||||
exit /b 1
|
||||
)
|
||||
docker info --format "{{.OSType}}"
|
||||
if errorlevel 1 exit /b 1
|
||||
|
||||
powershell.exe -NoProfile -ExecutionPolicy Bypass -File "%~dp0scripts\setup.ps1"
|
||||
if errorlevel 1 exit /b 1
|
||||
|
||||
docker compose --env-file .env config --quiet
|
||||
if errorlevel 1 exit /b 1
|
||||
|
||||
docker compose --env-file .env up -d --build --wait --wait-timeout 180
|
||||
if errorlevel 1 goto diagnostics
|
||||
|
||||
curl.exe --fail --silent --show-error --retry 10 --retry-connrefused --retry-delay 1 --max-time 5 http://127.0.0.1:8080/health/ready
|
||||
if errorlevel 1 goto diagnostics
|
||||
echo.
|
||||
docker compose --env-file .env ps -a
|
||||
exit /b %errorlevel%
|
||||
|
||||
:diagnostics
|
||||
docker compose --env-file .env ps -a
|
||||
docker compose --env-file .env logs --tail=60 db s3 s3-bootstrap migrate api1 api2 lb
|
||||
exit /b 1
|
||||
@@ -0,0 +1,35 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Additional negative integration checks against a running lab4 stack."""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
sys.path.insert(0,str(Path(__file__).resolve().parents[1]/'tests'))
|
||||
from client import Client, arguments, run
|
||||
from uuid import uuid4
|
||||
|
||||
|
||||
def main():
|
||||
args=arguments('Negative attachment checks')
|
||||
c=Client(args.base_url,args.ca_file)
|
||||
alice,bob,eve,chat=c.bootstrap()
|
||||
prefix='/api/v1/chats/'+chat['id']+'/attachments?filename='
|
||||
fixture=(Path(__file__).resolve().parents[1]/'tests/fixtures/sample.png').read_bytes()
|
||||
hdr={'Content-Type':'image/png'}
|
||||
c.call('POST',prefix+'../bad.png',400,actor=alice,raw=fixture,headers=hdr)
|
||||
c.call('POST',prefix+'bad%5Cname.png',400,actor=alice,raw=fixture,headers=hdr)
|
||||
c.call('POST',prefix+'too-large.png',413,actor=alice,raw=fixture+b'\0'*(10*1024*1024+1),headers=hdr)
|
||||
c.call('POST',prefix+'corrupt.png',415,actor=alice,raw=b'not an image',headers=hdr)
|
||||
c.call('POST',prefix+'spoof.png',415,actor=alice,raw=fixture,headers={'Content-Type':'image/jpeg'})
|
||||
att,_=c.attachment(alice,chat)
|
||||
for suffix in ('','/content'):
|
||||
c.call('GET','/api/v1/attachments/'+att['id']+suffix,404,actor=eve)
|
||||
other=c.call('POST','/api/v1/chats',201,actor=alice,body={
|
||||
'title':'Other chat','member_ids':[alice['id'],eve['id']]},schema='Chat')['json']
|
||||
c.message(alice,chat,'',attachments=[att['id']])
|
||||
c.call('POST','/api/v1/chats/'+other['id']+'/messages',404,actor=alice,
|
||||
body={'text':'','attachment_ids':[att['id']]},headers={'Idempotency-Key':uuid4().hex})
|
||||
c.call('POST','/api/v1/chats/'+chat['id']+'/messages',404,actor=alice,
|
||||
body={'text':'','attachment_ids':[str(uuid4())]},headers={'Idempotency-Key':uuid4().hex})
|
||||
print('PASS: invalid size/image/MIME/filename, outsider ACL and cross-chat bindings')
|
||||
|
||||
if __name__=='__main__':
|
||||
run(main)
|
||||
@@ -0,0 +1,38 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Integration check: concurrent same-key POSTs via LB, no custom test dependencies."""
|
||||
import concurrent.futures
|
||||
import json
|
||||
import sys
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
sys.path.insert(0,str(Path(__file__).resolve().parents[1] / 'tests'))
|
||||
from client import Client, run, arguments, check
|
||||
|
||||
|
||||
def main():
|
||||
args=arguments('Concurrent idempotency and LB check')
|
||||
c=Client(args.base_url,args.ca_file,timeout=8)
|
||||
alice,bob,eve,chat=c.bootstrap()
|
||||
key=uuid.uuid4().hex
|
||||
body={'text':'Конкурентное сообщение '+key}
|
||||
path='/api/v1/chats/'+chat['id']+'/messages'
|
||||
def send(_):
|
||||
# Each worker has its own client, simulating independent processes.
|
||||
worker=Client(args.base_url,args.ca_file,timeout=8)
|
||||
return worker.call('POST',path,201,actor=alice,body=body,
|
||||
headers={'Idempotency-Key':key},schema='Message')
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=16) as pool:
|
||||
results=list(pool.map(send,range(32)))
|
||||
ids={r['json']['id'] for r in results}
|
||||
instances={r['headers'].get('X-Instance-Id') for r in results}
|
||||
check(len(ids)==1,'Concurrent messages were duplicated')
|
||||
check(len(instances)>=2,'Only one instance was observed; check LB')
|
||||
c.call('POST',path,409,actor=alice,body={'text':'different text'},headers={'Idempotency-Key':key})
|
||||
stored=c.call('GET',path,actor=bob,schema='MessagePage')['json']['items']
|
||||
check(sum(x['id'] in ids for x in stored)==1,'History contains duplicate or no record')
|
||||
print(json.dumps({'concurrent_requests':len(results),'unique_messages':len(ids),
|
||||
'observed_replicas':sorted(instances)},ensure_ascii=False))
|
||||
print('PASS: shared idempotency is atomic across replicas')
|
||||
|
||||
if __name__=='__main__':
|
||||
run(main)
|
||||
@@ -0,0 +1,47 @@
|
||||
$ErrorActionPreference = 'Continue'
|
||||
Set-Location $PSScriptRoot
|
||||
if (!(Test-Path -LiteralPath 'start-windows.cmd')) {
|
||||
throw 'Place this script in the extracted mtusi2026lab4 directory.'
|
||||
}
|
||||
|
||||
New-Item -ItemType Directory -Force -Path evidence | Out-Null
|
||||
$log = Join-Path $PSScriptRoot ("evidence/windows-run-{0:yyyyMMdd-HHmmss-fff}.txt" -f (Get-Date))
|
||||
$script:failed = $false
|
||||
|
||||
function Write-Log([string]$Message) {
|
||||
$Message | Tee-Object -FilePath $log -Append
|
||||
}
|
||||
|
||||
function Invoke-Check([string]$Name, [string]$Program, [string[]]$Arguments) {
|
||||
Write-Log ("--- {0} ---" -f $Name)
|
||||
$global:LASTEXITCODE = 1
|
||||
& $Program @Arguments 2>&1 | ForEach-Object { "$_" } | Tee-Object -FilePath $log -Append
|
||||
$code = $LASTEXITCODE
|
||||
Write-Log ("{0} exit code = {1}" -f $Name, $code)
|
||||
$script:stepOK = $code -eq 0
|
||||
if (!$script:stepOK) { $script:failed = $true }
|
||||
}
|
||||
|
||||
$baseUrl = 'http://127.0.0.1:8080'
|
||||
Write-Log (Get-Date -Format o)
|
||||
try {
|
||||
Invoke-Check 'Docker Engine' docker @('info', '--format', '{{.OSType}}')
|
||||
if (!$script:stepOK) { exit 1 }
|
||||
Invoke-Check 'Python' py @('-3', '--version')
|
||||
if (!$script:stepOK) { exit 1 }
|
||||
Invoke-Check 'Startup and readiness' cmd.exe @('/d', '/c', 'start-windows.cmd')
|
||||
if (!$script:stepOK) { exit 1 }
|
||||
Invoke-Check 'Smoke' py @('-3', '-X', 'utf8', 'tests/smoke.py', '--base-url', $baseUrl)
|
||||
Invoke-Check 'Persistence' py @('-3', '-X', 'utf8', 'tests/resilience.py', '--mode', 'persistence', '--action-script', 'scripts/restart.cmd', '--base-url', $baseUrl)
|
||||
try {
|
||||
Invoke-Check 'Failover' py @('-3', '-X', 'utf8', 'tests/resilience.py', '--mode', 'failover', '--action-script', 'scripts/stop-one.cmd', '--base-url', $baseUrl)
|
||||
Invoke-Check 'State after failover' docker @('compose', '--env-file', '.env', 'ps', '-a')
|
||||
} finally {
|
||||
Invoke-Check 'Restore replicas' cmd.exe @('/d', '/c', 'scripts\restore-replicas.cmd')
|
||||
Invoke-Check 'Final state' docker @('compose', '--env-file', '.env', 'ps', '-a')
|
||||
}
|
||||
} finally {
|
||||
Write-Host "Evidence file: $log"
|
||||
}
|
||||
if ($script:failed) { exit 1 }
|
||||
exit 0
|
||||
Reference in New Issue
Block a user