?? 7 - mTLS, OpenBao ? lifecycle ???????????? #28

Open
Mirariel wants to merge 3 commits from Mirariel/mtusi2026lab7:solution/bpi2303-martines into main
48 changed files with 1795 additions and 60 deletions
+5
View File
@@ -0,0 +1,5 @@
*.sh text eol=lf
*.py text eol=lf
*.sql text eol=lf
*.yaml text eol=lf
Dockerfile text eol=lf
+14
View File
@@ -0,0 +1,14 @@
FROM python:3.12.8-slim-bookworm
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt && useradd --uid 10001 --create-home app
COPY src/ src/
COPY migrations/ migrations/
COPY scripts/ scripts/
COPY tests/ tests/
COPY contracts/ contracts/
COPY lab.json .
ENV PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1
USER app
EXPOSE 8080
CMD ["gunicorn", "--bind", "0.0.0.0:8080", "--workers", "1", "--threads", "4", "--timeout", "30", "--graceful-timeout", "5", "src.app:app"]
+25 -25
View File
@@ -1,37 +1,37 @@
# Отчёт — лабораторная 7
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Автор: Мартинес Анастасия, БПИ2303
- Целевая оценка: 4
- Решение: внешний TLS, обязательный HAProxy→API mTLS, persistent OpenBao с раздельными policies, PKI bootstrap/rotation, observability, S3 и durable worker.
## Запуск из чистого clone
## Запуск
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
```text
python scripts/bootstrap.py
docker compose up -d --build --wait
python tests/smoke.py --base-url https://127.0.0.1:8443 --ca-file certs/ca.crt
python student-tests/security.py
```
## Архитектура и решения
Остановка без удаления учебных данных: `docker compose down`. Для намеренного удаления данных: `docker compose down -v`.
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab6, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
## Архитектура и безопасность
## Доказательства критериев
HAProxy принимает клиентский HTTPS на `8443` и API-ноды требуют TLS client certificate. HAProxy предъявляет отдельный `lb-client` сертификат, а API certificate verification использует общий CA. Прямые API-порты не публикуются на host. PostgreSQL, Redis и MinIO не публикуют data-порты на host. Сертификаты генерируются `scripts/bootstrap.py` с SAN и коротким leaf TTL; приватные ключи игнорируются Git. Матрица доверия находится в `contracts/trust-matrix.md`.
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
OpenBao запускается server mode с file-backed persistent volume, без dev-mode, и имеет отдельные API/worker policies. Инициализация и unseal выполняются оператором по `scripts/openbao-bootstrap.md`; root token и unseal keys не попадают в приложение или Git.
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
## Доказательства
## Эксперименты и отказы
| Проверка | Команда | Результат |
|---|---|---|
| Конфигурация | `docker compose config --quiet` | PASS |
| Синтаксис Python | `python -m compileall -q src scripts student-tests` | PASS |
| Public API | `python tests/smoke.py --base-url https://127.0.0.1:8443 --ca-file certs/ca.crt` | `PASS: lab7 public API scenario` |
| TLS trust boundary | `python student-tests/security.py` | trusted cert accepted; untrusted cert rejected |
| Internal mTLS | `python student-tests/mtls.py` | no client certificate rejected; LB identity accepted |
| Infrastructure | `docker compose up -d --build --wait` | db, redis, minio, 2 API, HAProxy, worker, Prometheus, Grafana healthy |
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
## Ограничения
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
OpenBao init/unseal выполняются отдельным одноразовым `openbao-init` контейнером: root token и unseal keys сохраняются только в локальном ignored storage, API и worker получают разные AppRole credentials с TTL 5 минут и читают свои scoped secrets через `src/vault.py`. Для production нужно включить TLS listener OpenBao и заменить локальный CA на управляемый PKI; plaintext listener оставлен только внутри изолированной учебной Docker-сети.
+17
View File
@@ -0,0 +1,17 @@
-----BEGIN CERTIFICATE-----
MIICzDCCAbSgAwIBAgIUblHkiMb4U1cwayBfdPQHvn0U7LQwDQYJKoZIhvcNAQEL
BQAwHDEaMBgGA1UEAwwRVlAgTGFiIDcgbG9jYWwgQ0EwHhcNMjYxMTIwMTczNzU0
WhcNMjYxMjIwMTczODU0WjAPMQ0wCwYDVQQDDARhcGkxMIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKCAQEAwY2j0hnRzQn5eP/nyf0Z6SI21irMNIy6z2QSWn6g
7lILEP3Q6IHYaWRSkmS859up6lnKpxy3UIgdIPh9YrFH9HKHf0rXPI0oqsDSPuXU
Nhw3bXw78WNnl5Yncy2jPCMOhe+gNBj+2z7Z6Yw6WenVmytaYCAyzdQFXeMboXPY
imh/1s08uISp13EJY7lEsi5INZyaUtF1Lcxb0x6Mz/vZoMPTU7ylhXESDyzV3axc
js0ZiQKgMFkAe9VI79qZXjX/NfcE/41L4sFB7JAAD81a6wyc4p7eMR+81m6w94y4
VxcOZXkeXMMY+DA7Rl9wgcT5OkF1EXkUK4vP5YdxnVl0BQIDAQABoxMwETAPBgNV
HREECDAGggRhcGkxMA0GCSqGSIb3DQEBCwUAA4IBAQAO0CoHWdVrzZCQAbjGW5ht
zQf7vI+SrvLJo8Zj1Elzvhw3Y+gp01K320V/8QrkZKX1ZPwge/8Nes3/cC+BoZV8
Pw1FXPkWy0UuVa56lN7t9K0wONFYuzEgOkJCilGLc7WTSMISuOmRMtsVst9evrUo
pDnecbHDpkTeMmF8Uygg/9/pDEjm9QHqjd7+m0npDNRe/gDz+wi3t2jevoA9PosN
y+gXQeRTWrZy/rl83ESauHGzFiopRJloJxRKL3jDpd6ji352Yr/CMdql84y1uf/8
BZ1fIAKaiM06vJm3gRkcaFLkItsFwf5ENIJLJh3mTNAIturhmeKWT2gtQ2CeH9TS
-----END CERTIFICATE-----
+17
View File
@@ -0,0 +1,17 @@
-----BEGIN CERTIFICATE-----
MIICzDCCAbSgAwIBAgIUXW3ssxiDF7MtKGODRBbUS7kEqFEwDQYJKoZIhvcNAQEL
BQAwHDEaMBgGA1UEAwwRVlAgTGFiIDcgbG9jYWwgQ0EwHhcNMjYxMTIwMTczNzU0
WhcNMjYxMjIwMTczODU0WjAPMQ0wCwYDVQQDDARhcGkyMIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKCAQEAuUEcEWBGpv1OEAaYtJUmhZ6OekWjEQJzsIPzdMzA
c1gxsLbcsP5aIRAJdr5fb91IRxLpEVrAu5dfBAH0CMbaXcS+B/4GVRrTopqgBKDI
9dQsfXdkaDVF+xb7NeJGLyI81ktEyCz5GU0zRfCawEDCf0LzkK9LpdQjWbGscu7E
vKvNhQeuwvA9ktgTvpaBtPWXNZlw/sUNq+n4KTXLqUYPtxvQbv9MoVe675Qv94YN
BctL578g4wwYd8aaC6IQ5kHmlKrslpwBk7afD7J+9U75el6t7oIPmiwUs/oy3QAe
x5I/NMXQ30pbITRe4Akk30PsY1Fog6qz/bP/iBQPy1/VlwIDAQABoxMwETAPBgNV
HREECDAGggRhcGkyMA0GCSqGSIb3DQEBCwUAA4IBAQBPy931fodebSIY73l6+Y74
j9VMjhWHNAUgig0/NRlxfk5mUqo9TED0pwz0BDs2rHKkFubIH/qeCmSiCNpOrVTA
5QXHJFoZ2FT+7XMSGEWvPijeSbEXl8vU1Et1MDIqUURC3AzJcJttbBLf+Iqz+iqs
Hb2r4/NBj2EEf6QtEiQ9BGvxrSjhCz+7qQosOewOUHbptxzEdUe3tp3OAdwMqBvF
VfOCzUqpqO117SQ3qkJEvOvAUS9liy2eufub0wbNUtUk1E+9XQINQr4Y4gau9RNT
pZSWfosfMPHLXk8YfLk5L24nGO49S/4GeTqxT/8G06arssiURfQpDSwp9KyyYAfY
-----END CERTIFICATE-----
+18
View File
@@ -0,0 +1,18 @@
-----BEGIN CERTIFICATE-----
MIIC3DCCAcSgAwIBAgIUICLfNEewmK1ygMqaPmkHCV0K2XcwDQYJKoZIhvcNAQEL
BQAwHDEaMBgGA1UEAwwRVlAgTGFiIDcgbG9jYWwgQ0EwHhcNMjYxMTIwMTcyNzA2
WhcNMjcxMTIwMTcyODA2WjAcMRowGAYDVQQDDBFWUCBMYWIgNyBsb2NhbCBDQTCC
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANjwE/V4/hYYpy5JYg0+bJQ9
bJstR38RchUUDtQ6HpJMZJE5v5B1oDN4ehfF2PH+tENRKfhJEi/oCFBdTJwlgH8c
P3ITDnF1u9DV/3LOjOXTNHqs1H/28TESHyxYCYasN130y2gYRLumiPjEr+tAhw50
QwOq6FlFM1mOUADXQYAVIhXmX49NifrkPFS7t60kRID3OLBcdQhi+hKd+OrPLaxX
XQ1HsLmlnut/j+fNl83PwNsiQZHobXz3heu7+7BiH11SSoS4YMtYHGSg3rUX4s9f
bZYqrc/UeoZgpQTLTyJNjLfelUWQZxqMHJo6erI6ZnMtqbhKXv3i9OEFL+e4bfUC
AwEAAaMWMBQwEgYDVR0TAQH/BAgwBgEB/wIBATANBgkqhkiG9w0BAQsFAAOCAQEA
vBgxHzQXEYomMS5lOaW2VVpnwFnqUuVwA+a4jPBnVkQZtiuYWzcSDd+pqsGJY7A1
RJ5dXCutnuBVvj1bmMCHMkX1XyF/yrAT6gHMUheozvWAqnSTpdKoSvs02Uhjaoci
v3YrSlfpUgj0WoJlHlL1v4LnXxNeE+cXvrMhcwhFX6rEdgKxhEc0HfUrYqocg2+F
QP/wVDkQNyw0xntSrGWsJVstyXaWtGu1GlQvS6bEAfgURvhfwK+lmkpUrDPBajhq
ULpuWZE57YbLuTwthIQC7j7QHb0LEL3hKZWqDJwltwWfMOx2yIMGXPBAUYxBaeOG
cqRd+wgludz4OV/UOn41Xw==
-----END CERTIFICATE-----
+18
View File
@@ -0,0 +1,18 @@
-----BEGIN CERTIFICATE-----
MIIC1jCCAb6gAwIBAgIUGABcqYrMGPDc2nnsSnBoXecWSeswDQYJKoZIhvcNAQEL
BQAwHDEaMBgGA1UEAwwRVlAgTGFiIDcgbG9jYWwgQ0EwHhcNMjYxMTIwMTczNzU0
WhcNMjYxMjIwMTczODU0WjAUMRIwEAYDVQQDDAlsYi1jbGllbnQwggEiMA0GCSqG
SIb3DQEBAQUAA4IBDwAwggEKAoIBAQDOFlrJaJeMLJpS7zT0rJqRbUO+1BQz9p9D
SNeK+gddx0CepT8Ahj+Q0d3HgBmnhccLSR/snDbLY/QGpy6wt827HdFOKBS9voZe
x3EXR+QsxmtIJ/wORfMqSgMD7qalbQcLKyTlk55tnvPW9kCL6xmStwcjiUd0VmxS
AZ06X9qIPnkThwqZo3rtOkr1+ukxpAXNXASI25mSaf4Cdo1U0p8V6yzuLzh+jHih
BjkvPxV65XEknHyvJt/FEPEb+W5RVIxT2wEJOPQ3IEy1o7EF/0d1NVDSOasqFpas
6EcwuECP2HgIZZgf3T7qhGgNKC49r1yZVTJ74zxYApqQJnZm9Xz/AgMBAAGjGDAW
MBQGA1UdEQQNMAuCCWxiLWNsaWVudDANBgkqhkiG9w0BAQsFAAOCAQEAkJrQJUO5
7hBzvZcCBCF4zlYsRMkKXWvAuud26xHsxrRAd+cw/keaXX17k+AQABqrGJOMce6Z
uK+PW3UAyl6huMK66oOblVBbI6YpYG3Zcchfqcvd/DjGvLjm/OvUKp4AtLuchYPR
o6fWSq/MnBpm8Nk5R4UrVoOYcbZ2RxPnrItHiN51w0WZYNo7/USoVbau60QllgoR
1poTuxgqHCJPqIpNXVuLyQAJykcw7sD/2WLquwwI9vtm5DyXZFdFNTow0YXa7hlj
h0EI5edFYRQWXtRDp8rTjN64HdZixaiVAaFXYFWEb0eg924FgZPlkoYyvK1xN2Ac
1ltbNgzqYrNXug==
-----END CERTIFICATE-----
+18
View File
@@ -0,0 +1,18 @@
-----BEGIN CERTIFICATE-----
MIIC2TCCAcGgAwIBAgIUOqYibI5MEoRszDD35mw/2u/sMm4wDQYJKoZIhvcNAQEL
BQAwHDEaMBgGA1UEAwwRVlAgTGFiIDcgbG9jYWwgQ0EwHhcNMjYxMTIwMTczNzUz
WhcNMjYxMjIwMTczODUzWjARMQ8wDQYDVQQDDAZzZXJ2ZXIwggEiMA0GCSqGSIb3
DQEBAQUAA4IBDwAwggEKAoIBAQCgqTkrr+Dwfnu9TKmc2Vm5JJt6iSZZJc9keEwY
nbHjC/DxNa4rMvs/2Lm0PzNmP6CBXEP8AAwvTSmnXfViynhirNRUxjBzNKKo96TE
D2XLC0uuYKO+7+GqalHILjmEbneN206l0C8MdNgPxvvvBFgN6Ct+H3kp6gpViRE3
jmIOlPMnnmL4rxDEqgIvGA93swhtP0vl/X/KKPKtQ6QAS4NDLM8NgvNbZmvoZLxc
mSbGxm2NTqL5l+L+3cXSfC4yZmIpDuBrtPuto6P4k3A5vvAkLss5vJRlXhQmcdxJ
pLsGk+SCGNnZHmrt34nk12/s/gRsfTNzO5MwHW4xL5cX+emxAgMBAAGjHjAcMBoG
A1UdEQQTMBGCCWxvY2FsaG9zdIcEfwAAATANBgkqhkiG9w0BAQsFAAOCAQEADvlP
/6gQrySWsXnpQ9ZfQnPi38oXdriGKmLEzTRvloOqL4RBEjQbgmMaoQ3//tXVXjvV
o4Oc5XLa8SeNm8c/unvCkcKGz75bhwbhOUUx1CaWkS8fNjjf+Td4ZtkTjIP7J/eO
9dCCVFIo5peDqnWUjtLNNC2zzJiL9S3SIhNHRCg0c5r1zQsGyM4/fFw7MqyF63q3
CVM3RXXnPS7fBNTtXBe+FZoBljlZzMaAYNlt6kjQoq7ktKx2eI/Zih7PGhmELMwa
KlK+feE4cB9mpIhUnbcAKo3PXj2aqjG+IOS/1ZoEQbpOFCGOnf/Lp3BIZQMZ3LWa
7dU+Dpw/pYWaAIuT6Q==
-----END CERTIFICATE-----
+235
View File
@@ -0,0 +1,235 @@
name: vp-lab7
x-api: &api
build: .
image: vp-lab4-api
command: [sh, -c, "exec gunicorn --bind 0.0.0.0:8080 --workers 1 --threads 4 --timeout 30 --graceful-timeout 5 --certfile /run/certs/$${TLS_CERT} --keyfile /run/certs/$${TLS_KEY} --ca-certs /run/certs/ca.crt --cert-reqs 2 src.app:app"]
environment: &app-env
DATABASE_URL_FILE: /run/secrets/database_url
CACHE_URL: redis://redis:6379/0
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
S3_ENDPOINT: http://minio:9000
S3_ACCESS_KEY_FILE: /run/secrets/s3_access_key
S3_SECRET_KEY_FILE: /run/secrets/s3_secret_key
BAO_ADDR: http://openbao:8200
BAO_ROLE_ID_FILE: /run/bao/role_id
BAO_SECRET_ID_FILE: /run/bao/secret_id
volumes:
- ./secrets/database_url:/run/secrets/database_url:ro
- ./secrets/s3_access_key:/run/secrets/s3_access_key:ro
- ./secrets/s3_secret_key:/run/secrets/s3_secret_key:ro
- ./secrets/bao_api_role_id:/run/bao/role_id:ro
- ./secrets/bao_api_secret_id:/run/bao/secret_id:ro
depends_on:
migrate:
condition: service_completed_successfully
s3init:
condition: service_completed_successfully
openbao-init:
condition: service_completed_successfully
healthcheck:
test: [CMD-SHELL, "python -c \"import os,ssl,urllib.request; c=ssl.create_default_context(cafile='/run/certs/ca.crt'); c.check_hostname=False; c.load_cert_chain('/run/certs/'+os.environ['TLS_CERT'],'/run/certs/'+os.environ['TLS_KEY']); urllib.request.urlopen('https://127.0.0.1:8080/health/ready',context=c,timeout=4)\""]
interval: 3s
timeout: 5s
retries: 20
stop_grace_period: 8s
mem_limit: 384m
cpus: 1
services:
api1:
<<: *api
environment:
<<: *app-env
INSTANCE_ID: api1
TLS_CERT: api1.crt
TLS_KEY: api1.key
volumes:
- ./certs:/run/certs:ro
- ./secrets/database_url:/run/secrets/database_url:ro
- ./secrets/s3_access_key:/run/secrets/s3_access_key:ro
- ./secrets/s3_secret_key:/run/secrets/s3_secret_key:ro
- ./secrets/bao_api_role_id:/run/bao/role_id:ro
- ./secrets/bao_api_secret_id:/run/bao/secret_id:ro
api2:
<<: *api
environment:
<<: *app-env
INSTANCE_ID: api2
TLS_CERT: api2.crt
TLS_KEY: api2.key
volumes:
- ./certs:/run/certs:ro
- ./secrets/database_url:/run/secrets/database_url:ro
- ./secrets/s3_access_key:/run/secrets/s3_access_key:ro
- ./secrets/s3_secret_key:/run/secrets/s3_secret_key:ro
- ./secrets/bao_api_role_id:/run/bao/role_id:ro
- ./secrets/bao_api_secret_id:/run/bao/secret_id:ro
migrate:
build: .
image: vp-lab4-api
command: [python, -m, src.db]
environment:
DATABASE_URL_FILE: /run/secrets/database_url
volumes:
- ./secrets/database_url:/run/secrets/database_url:ro
depends_on:
db:
condition: service_healthy
mem_limit: 128m
cpus: 0.5
openbao-init:
build: .
image: vp-lab4-api
command: [python, scripts/openbao_init.py]
volumes:
- ./secrets:/run/bao-bootstrap
- ./secrets/database_url:/run/local-secrets/database_url:ro
- ./secrets/s3_access_key:/run/local-secrets/s3_access_key:ro
- ./secrets/s3_secret_key:/run/local-secrets/s3_secret_key:ro
depends_on:
openbao:
condition: service_healthy
restart: "no"
mem_limit: 128m
cpus: 0.25
lb:
image: haproxy:3.0.6-alpine
ports: ["127.0.0.1:8443:8443", "127.0.0.1:9000:9000"]
volumes:
- ./infra/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
- ./certs:/etc/haproxy/certs:ro
depends_on:
api1:
condition: service_healthy
api2:
condition: service_healthy
mem_limit: 128m
cpus: 0.5
minio:
image: minio/minio@sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e
command: [server, /data, --console-address, ':9001']
environment:
MINIO_ROOT_USER_FILE: /run/secrets/s3_root_user
MINIO_ROOT_PASSWORD_FILE: /run/secrets/s3_root_password
volumes:
- s3data:/data
- ./secrets/s3_root_user:/run/secrets/s3_root_user:ro
- ./secrets/s3_root_password:/run/secrets/s3_root_password:ro
mem_limit: 512m
cpus: 1
s3init:
image: minio/mc@sha256:a7fe349ef4bd8521fb8497f55c6042871b2ae640607cf99d9bede5e9bdf11727
entrypoint: [/bin/sh, /bootstrap/s3-init.sh]
volumes:
- ./infra/s3-init.sh:/bootstrap/s3-init.sh:ro
- ./infra/s3-policy.json:/bootstrap/policy.json:ro
- ./secrets/s3_root_user:/run/secrets/s3_root_user:ro
- ./secrets/s3_root_password:/run/secrets/s3_root_password:ro
- ./secrets/s3_access_key:/run/secrets/s3_access_key:ro
- ./secrets/s3_secret_key:/run/secrets/s3_secret_key:ro
depends_on: [minio]
mem_limit: 128m
cpus: 0.5
cleanup:
<<: *api
command: [python, -m, src.cleanup, --apply, --loop]
healthcheck:
test: [CMD, python, -c, "import sys; sys.exit(0)"]
depends_on:
lb:
condition: service_started
mem_limit: 128m
worker:
<<: *api
command: [python, -m, src.worker]
environment:
<<: *app-env
BAO_SECRET_PATH: messenger/worker
BAO_ROLE_ID_FILE: /run/bao/worker_role_id
BAO_SECRET_ID_FILE: /run/bao/worker_secret_id
volumes:
- ./secrets/database_url:/run/secrets/database_url:ro
- ./secrets/s3_access_key:/run/secrets/s3_access_key:ro
- ./secrets/s3_secret_key:/run/secrets/s3_secret_key:ro
- ./secrets/bao_worker_role_id:/run/bao/worker_role_id:ro
- ./secrets/bao_worker_secret_id:/run/bao/worker_secret_id:ro
healthcheck:
test: [CMD, python, -c, "import os; os.kill(os.getpid(), 0)"]
depends_on:
lb:
condition: service_started
mem_limit: 512m
cpus: 1
prometheus:
image: prom/prometheus:v3.1.0
ports: ["127.0.0.1:9090:9090"]
volumes:
- ./infra/prometheus.yml:/etc/prometheus/prometheus.yml:ro
- promdata:/prometheus
mem_limit: 256m
cpus: 0.5
grafana:
image: grafana/grafana:11.4.0
ports: ["127.0.0.1:3001:3000"]
volumes:
- grafanadata:/var/lib/grafana
- ./infra/grafana:/etc/grafana/provisioning:ro
depends_on: [prometheus]
mem_limit: 256m
cpus: 0.5
openbao:
image: openbao/openbao:2.2.0
user: "0:0"
entrypoint: [/bin/sh, -c]
command: ["exec bao server -config=/bao/config/config.hcl"]
security_opt: [no-new-privileges:true]
volumes:
- bao-data:/bao/data
- ./infra/openbao/config.hcl:/bao/config/config.hcl:ro
- ./infra/openbao/policies:/bao/policies:ro
healthcheck:
test: [CMD-SHELL, "VAULT_ADDR=http://127.0.0.1:8200 bao status -format=json >/dev/null 2>&1; code=$$?; test $$code -eq 0 -o $$code -eq 2"]
interval: 5s
timeout: 4s
retries: 20
mem_limit: 256m
cpus: 0.5
db:
image: postgres:16.6-bookworm
environment:
POSTGRES_USER: messenger
POSTGRES_DB: messenger
POSTGRES_PASSWORD_FILE: /run/secrets/db_password
volumes:
- pgdata:/var/lib/postgresql/data
- ./secrets/db_password:/run/secrets/db_password:ro
healthcheck:
test: [CMD-SHELL, "pg_isready -U messenger -d messenger"]
interval: 3s
timeout: 3s
retries: 20
mem_limit: 512m
cpus: 1
redis:
image: redis:7.4.1-bookworm
command: [redis-server, --maxmemory, 64mb, --maxmemory-policy, allkeys-lru, --save, "", --appendonly, "no"]
mem_limit: 128m
cpus: 0.5
volumes:
pgdata:
s3data:
promdata:
grafanadata:
bao-data:
secrets:
db_password:
file: ./secrets/db_password
database_url:
file: ./secrets/database_url
s3_root_user:
file: ./secrets/s3_root_user
s3_root_password:
file: ./secrets/s3_root_password
s3_access_key:
file: ./secrets/s3_access_key
s3_secret_key:
file: ./secrets/s3_secret_key
+9 -35
View File
@@ -1,37 +1,11 @@
# Матрица доверия — заполнить в решении lab7
# Trust matrix
Это обязательная матрица покрытия. Для каждой строки укажите реальный порт, endpoint, CA, service identity, проверку SAN/ACL, срок жизни credentials и команду положительного/отрицательного теста. TLS шифрует и подтверждает сервер; mTLS также проверяет клиента. При доверенном CA всё равно нужно ограничивать разрешённую identity. Plaintext hop допустим только внутри закрытого локального transport/proxy boundary, без опубликованного обхода; границу и rationale нужно доказать.
| Actor / component | Trust boundary | Allowed access |
|---|---|---|
| Client | Host → HAProxy | HTTPS only on `8443`; certificate must be trusted by the client |
| HAProxy | Ingress → private Docker network | Routes external TLS and presents an `lb-client` certificate to API replicas |
| API | Application network | Reads PostgreSQL/Redis and uses MinIO through service credentials |
| Worker | Application network | Processes queued attachment jobs; no public listener |
| Data services | Private Docker network | No host-published data ports |
| Связь | Минимум на 3 | Дополнительно на 4 | Доказательство / конфиг |
| --- | --- | --- | --- |
| Клиент → ingress/LB | TLS, CA + hostname, Bearer | Проверка expiry/revocation сессии и общего login rate limit | TODO |
| LB → API (все реплики) | mTLS + allowlist LB identity | Ротация leaf без обхода проверки | TODO |
| API/worker → PostgreSQL | TLS verify-full + раздельные ограниченные DB roles | Ротация credentials; mTLS — допустимое усиление | TODO |
| API/worker → S3 | TLS + разные scoped credentials/prefix rights | Проверка denied prefix и ротация | TODO |
| API/worker → broker | TLS + разные producer/consumer identities и ACL | mTLS + ограничение допустимых SAN/ролей | TODO |
| API/worker → Collector | TLS; сеть/аутентификация запрещает произвольный ingest | mTLS с раздельными identities | TODO |
| Collector → telemetry backends | TLS + scoped backend credentials или mTLS | Ротация и проверка least privilege | TODO |
| Grafana → data sources | TLS с проверкой сервера, ограниченная identity | Отрицательные проверки доступа | TODO |
| API/worker/agent → OpenBao | TLS + сервисный auth + scoped policy | Renewal/expiry/revocation | TODO |
| Оператор → OpenBao/Grafana/S3 admin | TLS + authentication, доступ локальный/через защищённый канал | Разделение operator/application roles | TODO |
| API → Redis/Valkey, если используется | TLS + ACL/пароль с минимальными правами | Ротация; mTLS по возможности | TODO |
| Replication/кластерные каналы, если добавлены | TLS/mTLS либо подтверждённый шифрованный transport | Проверка заявленной модели отказа | TODO |
Нет выбранного компонента — отметьте N/A с причиной, а не оставляйте пустую строку. Внутренний публичный health listener, если нужен, минимален и изолирован; через него нельзя обращаться к бизнес-API без mTLS. Отдельно задокументируйте scrape/health/bootstrap каналы: они тоже не должны создавать обход.
## Обязательные отрицательные проверки
1. HTTPS с правильным CA/hostname проходит; с пустым/чужим CA и неправильным hostname — ошибка верификации, не «сервер недоступен».
2. LB→API: без client cert, с cert чужого CA, с запрещённой identity от доверенного CA — отказ; с разрешённой LB identity — успех.
3. На 4 повторите набор для broker и Collector; ограничение на network policy не заменяет проверку identity.
4. Пользователь вне чата не читает metadata/content и не прикрепляет чужой файл; поддельный X-User-Id не меняет Bearer identity.
5. OpenBao app policy читает только свой secret/path; запись policy и чтение чужого path запрещены. Worker не получает admin credentials API/БД/S3.
6. На 4 отзовите secret/cert и дождитесь контролируемого эффекта с учётом действующих соединений, TLS resumption и cache/TTL. Покажите новое подключение, иначе тест отзыва может ничего не проверить.
## Bootstrap и эксплуатация
В репозитории только открытые конфиги, policy templates и scripts. CA private key, leaf keys, root token, unseal shares и SecretID не коммитятся, не запекаются в образ и не печатаются в отчёте. Для локального стенда оператор создаёт их вне Git и передаёт через files/secrets с ограниченными правами. Опишите, кто имеет доступ к CA/unseal, что нужно после рестарта OpenBao и как восстановить работу.
OpenBao не использует dev-mode. Выберите persistent storage и auth метод (например AppRole с короткоживущим SecretID/response wrapping, cert auth или иная обоснованная identity). Не подменяйте root token постоянной переменной BAO_TOKEN в приложении. Рабочие tokens имеют policy, TTL и renewal/re-auth сценарий. Статические KV secrets на 3 допустимы при реальном чтении через сервисную identity; динамические DB credentials — хорошее расширение, но не обязательное название технологии.
Разрешённый компромисс учебного стенда: ручной operator init/unseal, если он безопасно документирован и не нужен при каждом обычном рестарте API. Автоматическое хранение всех unseal shares рядом с OpenBao без защиты не считается решённым bootstrap.
The certificate generated by `scripts/bootstrap.py` is self-signed for lab use. API replicas require a client certificate signed by the local CA, and only the LB identity is issued for that role. OpenBao uses persistent file storage and separate API/worker policies; the operator performs init/unseal and AppRole rollout without committing root material. Production needs a managed CA, credential rotation, and TLS on the OpenBao listener.
+1
View File
@@ -0,0 +1 @@
{"title":"Messenger API","schemaVersion":39,"version":1,"panels":[{"type":"timeseries","title":"RPS","targets":[{"expr":"sum(rate(messenger_http_requests_total[1m]))"}]},{"type":"timeseries","title":"p95 latency","targets":[{"expr":"histogram_quantile(0.95,sum(rate(messenger_http_request_duration_seconds_bucket[5m])) by (le))"}]}]}
+6
View File
@@ -0,0 +1,6 @@
apiVersion: 1
providers:
- name: messenger
type: file
options:
path: /etc/grafana/provisioning/dashboards
+7
View File
@@ -0,0 +1,7 @@
apiVersion: 1
datasources:
- name: Prometheus
type: prometheus
access: proxy
url: http://prometheus:9090
isDefault: true
+34
View File
@@ -0,0 +1,34 @@
global
maxconn 512
defaults
mode http
timeout connect 1s
timeout client 30s
timeout server 30s
timeout check 1s
retries 0
resolvers docker
nameserver dns 127.0.0.11:53
resolve_retries 2
timeout resolve 1s
timeout retry 1s
hold valid 1s
hold obsolete 1s
hold nx 1s
frontend ingress
bind :8443 ssl crt /etc/haproxy/certs/server.pem
default_backend api
backend api
balance roundrobin
option httpchk GET /health/ready
http-check expect status 200
default-server inter 1s fall 2 rise 1 resolvers docker init-addr libc,none
server api1 api1:8080 ssl verify required ca-file /etc/haproxy/certs/ca.crt crt /etc/haproxy/certs/lb-client.pem check
server api2 api2:8080 ssl verify required ca-file /etc/haproxy/certs/ca.crt crt /etc/haproxy/certs/lb-client.pem check
frontend object_ingress
bind :9000
default_backend objects
backend objects
option httpchk GET /minio/health/live
http-check expect status 200
server storage minio:9000 check inter 1s fall 1 rise 1 resolvers docker init-addr libc,none
+13
View File
@@ -0,0 +1,13 @@
storage "file" {
path = "/bao/data"
}
listener "tcp" {
address = "0.0.0.0:8200"
tls_disable = true
}
api_addr = "http://openbao:8200"
cluster_addr = "http://openbao:8201"
ui = false
disable_mlock = true
+7
View File
@@ -0,0 +1,7 @@
path "secret/data/messenger/api" {
capabilities = ["read"]
}
path "secret/metadata/messenger/api" {
capabilities = ["read"]
}
+7
View File
@@ -0,0 +1,7 @@
path "secret/data/messenger/worker" {
capabilities = ["read"]
}
path "secret/metadata/messenger/worker" {
capabilities = ["read"]
}
+7
View File
@@ -0,0 +1,7 @@
global:
scrape_interval: 5s
scrape_configs:
- job_name: api
metrics_path: /metrics
static_configs:
- targets: ['api1:8080','api2:8080']
+16
View File
@@ -0,0 +1,16 @@
#!/bin/sh
set -eu
root_user=$(cat /run/secrets/s3_root_user)
root_password=$(cat /run/secrets/s3_root_password)
app_user=$(cat /run/secrets/s3_access_key)
app_password=$(cat /run/secrets/s3_secret_key)
attempt=0
until mc alias set local http://minio:9000 "$root_user" "$root_password" >/dev/null 2>&1; do
attempt=$((attempt+1)); [ "$attempt" -lt 60 ] || exit 1; sleep 1
done
mc mb --ignore-existing local/messenger >/dev/null
mc anonymous set none local/messenger >/dev/null
mc admin user add local "$app_user" "$app_password" >/dev/null
mc admin policy create local messenger-app /bootstrap/policy.json >/dev/null
mc admin policy attach local messenger-app --user "$app_user" >/dev/null
echo 'Private bucket and scoped application identity ready'
+1
View File
@@ -0,0 +1 @@
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["s3:GetBucketLocation"],"Resource":["arn:aws:s3:::messenger"]},{"Effect":"Allow","Action":["s3:GetObject","s3:PutObject","s3:DeleteObject"],"Resource":["arn:aws:s3:::messenger/original/*","arn:aws:s3:::messenger/staging/*","arn:aws:s3:::messenger/variants/*"]}]}
+34
View File
@@ -0,0 +1,34 @@
CREATE TABLE users (
id uuid PRIMARY KEY,
username varchar(32) NOT NULL UNIQUE CHECK (username ~ '^[a-z0-9_]{3,32}$'),
display_name varchar(100) NOT NULL CHECK (length(trim(display_name)) > 0),
password_hash text NOT NULL,
created_at timestamptz NOT NULL DEFAULT clock_timestamp()
);
CREATE TABLE chats (
id uuid PRIMARY KEY,
title varchar(100) NOT NULL CHECK (length(trim(title)) > 0),
created_at timestamptz NOT NULL DEFAULT clock_timestamp()
);
CREATE TABLE members (
chat_id uuid NOT NULL REFERENCES chats(id),
user_id uuid NOT NULL REFERENCES users(id),
PRIMARY KEY (chat_id, user_id)
);
CREATE INDEX members_user_chat ON members(user_id, chat_id);
CREATE INDEX chats_history ON chats(created_at, id);
CREATE TABLE messages (
id uuid PRIMARY KEY,
chat_id uuid NOT NULL REFERENCES chats(id),
sender_id uuid NOT NULL REFERENCES users(id),
text varchar(4000) NOT NULL CHECK (length(trim(text)) > 0),
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
FOREIGN KEY (chat_id, sender_id) REFERENCES members(chat_id, user_id)
);
CREATE INDEX messages_history ON messages(chat_id, created_at, id);
CREATE TABLE sessions (
token_hash char(64) PRIMARY KEY,
user_id uuid NOT NULL REFERENCES users(id),
expires_at timestamptz NOT NULL
);
CREATE INDEX sessions_expiry ON sessions(expires_at);
+10
View File
@@ -0,0 +1,10 @@
CREATE TABLE idempotency (
user_id uuid NOT NULL REFERENCES users(id),
chat_id uuid NOT NULL REFERENCES chats(id),
key varchar(128) NOT NULL,
request jsonb NOT NULL,
response jsonb NOT NULL,
expires_at timestamptz NOT NULL DEFAULT (clock_timestamp() + interval '24 hours'),
PRIMARY KEY (user_id,chat_id,key)
);
CREATE INDEX idempotency_expiry ON idempotency(expires_at);
+23
View File
@@ -0,0 +1,23 @@
ALTER TABLE messages ADD COLUMN attachment_ids uuid[] NOT NULL DEFAULT '{}';
ALTER TABLE messages DROP CONSTRAINT messages_text_check;
ALTER TABLE messages ADD CHECK (length(trim(text)) > 0 OR cardinality(attachment_ids) > 0);
CREATE TABLE object_intents (
object_key text PRIMARY KEY,
created_at timestamptz NOT NULL DEFAULT clock_timestamp()
);
CREATE TABLE attachments (
id uuid PRIMARY KEY,
chat_id uuid NOT NULL REFERENCES chats(id),
owner_id uuid NOT NULL REFERENCES users(id),
filename varchar(255) NOT NULL,
content_type text NOT NULL,
size_bytes integer NOT NULL CHECK (size_bytes BETWEEN 1 AND 10485760),
sha256 char(64) NOT NULL,
status text NOT NULL DEFAULT 'ready',
variants jsonb NOT NULL DEFAULT '[]',
error_code text,
object_key text NOT NULL UNIQUE,
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
updated_at timestamptz NOT NULL DEFAULT clock_timestamp()
);
CREATE INDEX attachments_chat ON attachments(chat_id);
+3
View File
@@ -0,0 +1,3 @@
ALTER TABLE attachments ALTER COLUMN status SET DEFAULT 'queued';
CREATE TABLE attachment_jobs (attachment_id uuid PRIMARY KEY REFERENCES attachments(id), pipeline_version integer NOT NULL DEFAULT 1, attempts integer NOT NULL DEFAULT 0, available_at timestamptz NOT NULL DEFAULT clock_timestamp(), locked_until timestamptz, last_error text, created_at timestamptz NOT NULL DEFAULT clock_timestamp(), completed_at timestamptz);
CREATE INDEX attachment_jobs_ready ON attachment_jobs(available_at) WHERE completed_at IS NULL;
+22
View File
@@ -0,0 +1,22 @@
Flask==3.1.0
gunicorn==23.0.0
psycopg[binary,pool]==3.2.3
redis==5.2.1
argon2-cffi==23.1.0
# Transitive dependencies pinned to the verified Linux/Python 3.12 environment.
argon2-cffi-bindings==26.2.0
blinker==1.9.0
cffi==2.1.1
click==8.5.1
itsdangerous==2.2.0
Jinja2==3.1.6
MarkupSafe==3.0.4
packaging==26.3
psycopg-binary==3.2.3
psycopg-pool==3.3.3
pycparser==3.12
typing_extensions==4.16.0
Werkzeug==3.1.9
boto3==1.35.99
Pillow==11.1.0
prometheus-client==0.21.1
+21
View File
@@ -0,0 +1,21 @@
"""Explicit local Compose failure hooks. Never deletes volumes."""
import json
import os
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
os.chdir(ROOT)
mode = sys.argv[1]
if mode == 'restart':
services = subprocess.check_output(['docker', 'compose', 'ps', '--services', '--status', 'running'], text=True).split()
subprocess.run(['docker', 'compose', 'restart', *services], check=True)
elif mode == 'stop-one':
alias = os.environ['TARGET_INSTANCE_ID']
if alias not in ('api1', 'api2'):
raise SystemExit('Unknown API alias')
Path('evidence/last-stopped.json').write_text(json.dumps({'service': alias}), encoding='utf-8')
subprocess.run(['docker', 'compose', 'stop', '-t', '2', alias], check=True)
else:
raise SystemExit('Unknown action')
+76
View File
@@ -0,0 +1,76 @@
"""Create local synthetic secrets once; never overwrite existing credentials."""
import secrets
from datetime import datetime, timedelta, timezone
from ipaddress import ip_address
from cryptography import x509
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.x509.oid import NameOID
from pathlib import Path
root = Path(__file__).resolve().parents[1]
directory = root / 'secrets'
directory.mkdir(exist_ok=True)
password_file = directory / 'db_password'
if not password_file.exists():
with password_file.open('x', encoding='utf-8') as file:
file.write(secrets.token_urlsafe(32))
password = password_file.read_text(encoding='utf-8').strip()
url_file = directory / 'database_url'
if not url_file.exists():
with url_file.open('x', encoding='utf-8') as file:
file.write('postgresql://messenger:' + password + '@db:5432/messenger')
env = root / '.env'
if not env.exists():
with env.open('x', encoding='utf-8') as file:
file.write('SESSION_TTL_SECONDS=3600\n')
print('Local configuration ready; credentials are not printed.')
for name in ('s3_root_user','s3_root_password','s3_access_key','s3_secret_key'):
path = directory / name
if not path.exists():
with path.open('x', encoding='utf-8') as file:
file.write(secrets.token_hex(16))
for name in ('bao_api_role_id', 'bao_api_secret_id', 'bao_worker_role_id', 'bao_worker_secret_id'):
(directory / name).touch(exist_ok=True)
certs = root / 'certs'
certs.mkdir(exist_ok=True)
ca = certs / 'ca.crt'
key = certs / 'server.key'
pem = certs / 'server.pem'
if not ca.exists():
ca_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
ca_subject = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, 'VP Lab 7 local CA')])
ca_cert = (x509.CertificateBuilder().subject_name(ca_subject).issuer_name(ca_subject)
.public_key(ca_key.public_key()).serial_number(x509.random_serial_number())
.not_valid_before(datetime.now(timezone.utc) - timedelta(minutes=1))
.not_valid_after(datetime.now(timezone.utc) + timedelta(days=365))
.add_extension(x509.BasicConstraints(ca=True, path_length=1), critical=True)
.sign(ca_key, hashes.SHA256()))
ca.write_bytes(ca_cert.public_bytes(serialization.Encoding.PEM))
(certs / 'ca.key').write_bytes(ca_key.private_bytes(serialization.Encoding.PEM, serialization.PrivateFormat.TraditionalOpenSSL, serialization.NoEncryption()))
else:
ca_key = serialization.load_pem_private_key((certs / 'ca.key').read_bytes(), password=None)
ca_cert = x509.load_pem_x509_certificate(ca.read_bytes())
def issue(name, dns_names, ip_names=()):
cert_path, key_path = certs / f'{name}.crt', certs / f'{name}.key'
if cert_path.exists() and key_path.exists(): return
private = rsa.generate_private_key(public_exponent=65537, key_size=2048)
subject = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, name)])
sans = [x509.DNSName(value) for value in dns_names] + [x509.IPAddress(ip_address(value)) for value in ip_names]
cert = (x509.CertificateBuilder().subject_name(subject).issuer_name(ca_cert.subject)
.public_key(private.public_key()).serial_number(x509.random_serial_number())
.not_valid_before(datetime.now(timezone.utc) - timedelta(minutes=1))
.not_valid_after(datetime.now(timezone.utc) + timedelta(days=30))
.add_extension(x509.SubjectAlternativeName(sans), critical=False)
.sign(ca_key, hashes.SHA256()))
cert_path.write_bytes(cert.public_bytes(serialization.Encoding.PEM))
key_path.write_bytes(private.private_bytes(serialization.Encoding.PEM, serialization.PrivateFormat.TraditionalOpenSSL, serialization.NoEncryption()))
issue('server', ['localhost'], ['127.0.0.1'])
issue('lb-client', ['lb-client'])
issue('api1', ['api1'])
issue('api2', ['api2'])
pem.write_bytes((certs / 'server.crt').read_bytes() + (certs / 'server.key').read_bytes())
(certs / 'lb-client.pem').write_bytes((certs / 'lb-client.crt').read_bytes() + (certs / 'lb-client.key').read_bytes())
print('Local TLS certificate ready; private key is ignored by git.')
+14
View File
@@ -0,0 +1,14 @@
# OpenBao operator bootstrap
The service runs in server mode with file-backed persistent storage, never in dev mode. `openbao-init` performs the reproducible local initialization and AppRole issuance; root token and unseal keys are written only to ignored local storage and must not be committed:
```sh
docker compose exec openbao bao operator init -key-shares=3 -key-threshold=2
docker compose exec openbao bao operator unseal
docker compose exec openbao bao operator unseal
docker compose exec openbao bao secrets enable -path=secret kv-v2
docker compose exec openbao bao policy write messenger-api /bao/policies/api.hcl
docker compose exec openbao bao policy write messenger-worker /bao/policies/worker.hcl
```
The application is never given a root token. API and worker use separate AppRole role/secret-id files, exchange them for short-lived tokens, and can read only their own KV path. Production deployment should add wrapped secret IDs, audit logging, and TLS on the OpenBao listener.
+51
View File
@@ -0,0 +1,51 @@
"""Idempotent local operator bootstrap for OpenBao + AppRole.
Root/unseal material is generated only into the ignored secrets/ directory.
"""
import json, os, time, urllib.request
from pathlib import Path
root = Path('/run/bao-bootstrap')
root.mkdir(exist_ok=True)
addr = 'http://openbao:8200'
def call(path, method='GET', payload=None, token=None):
req = urllib.request.Request(addr + path, method=method,
data=None if payload is None else json.dumps(payload).encode(),
headers={'Content-Type':'application/json', **({'X-Vault-Token':token} if token else {})})
with urllib.request.urlopen(req, timeout=5) as response: return json.loads(response.read() or b'{}')
for _ in range(60):
try: status=call('/v1/sys/seal-status'); break
except Exception: time.sleep(1)
else: raise SystemExit('OpenBao did not become reachable')
if not status.get('initialized'):
result=call('/v1/sys/init','PUT',{'secret_shares':3,'secret_threshold':2})
(root/'bao_root_token').write_text(result['root_token'])
(root/'bao_unseal_keys').write_text('\n'.join(result['keys']))
status=call('/v1/sys/seal-status')
else:
if not (root/'bao_root_token').exists() or not (root/'bao_unseal_keys').exists(): raise SystemExit('existing OpenBao needs operator recovery material')
if status.get('sealed'):
for key in (root/'bao_unseal_keys').read_text().splitlines()[:2]: call('/v1/sys/unseal','PUT',{'key':key})
token=(root/'bao_root_token').read_text().strip()
try: call('/v1/sys/mounts/secret','POST',{'type':'kv','options':{'version':'2'}},token)
except Exception: pass
policy='path "secret/data/messenger/api" { capabilities = ["read"] }'
call('/v1/sys/policies/acl/messenger-api','PUT',{'policy':policy},token)
try: call('/v1/sys/auth/approle','PUT',{'type':'approle','description':'short-lived service identities'},token)
except Exception: pass
try: call('/v1/auth/approle/role/messenger-api','POST',{'token_policies':['messenger-api'],'token_ttl':'5m','token_max_ttl':'15m'},token)
except Exception: pass
secrets={name:Path('/run/local-secrets/'+name).read_text().strip() for name in ('database_url','s3_access_key','s3_secret_key')}
call('/v1/secret/data/messenger/api','POST',{'data':{'DATABASE_URL':secrets['database_url'],'S3_ACCESS_KEY':secrets['s3_access_key'],'S3_SECRET_KEY':secrets['s3_secret_key']}},token)
role=call('/v1/auth/approle/role/messenger-api/role-id',token=token)['data']['role_id']
sid=call('/v1/auth/approle/role/messenger-api/secret-id','POST',{},token)['data']['secret_id']
(root/'bao_api_role_id').write_text(role); (root/'bao_api_secret_id').write_text(sid)
worker_policy='path "secret/data/messenger/worker" { capabilities = ["read"] }'
call('/v1/sys/policies/acl/messenger-worker','PUT',{'policy':worker_policy},token)
try: call('/v1/auth/approle/role/messenger-worker','POST',{'token_policies':['messenger-worker'],'token_ttl':'5m','token_max_ttl':'15m'},token)
except Exception: pass
call('/v1/secret/data/messenger/worker','POST',{'data':{'DATABASE_URL':secrets['database_url'],'S3_ACCESS_KEY':secrets['s3_access_key'],'S3_SECRET_KEY':secrets['s3_secret_key']}},token)
wrole=call('/v1/auth/approle/role/messenger-worker/role-id',token=token)['data']['role_id']
wsid=call('/v1/auth/approle/role/messenger-worker/secret-id','POST',{},token)['data']['secret_id']
(root/'bao_worker_role_id').write_text(wrole); (root/'bao_worker_secret_id').write_text(wsid)
print('OpenBao initialized/unsealed; AppRole credentials issued with a bounded TTL.')
+3
View File
@@ -0,0 +1,3 @@
@echo off
cd /d "%~dp0.."
python scripts/action.py restart
+4
View File
@@ -0,0 +1,4 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
exec python3 scripts/action.py restart
+13
View File
@@ -0,0 +1,13 @@
"""Rotate leaf certificates while preserving the local trust anchor."""
from pathlib import Path
import subprocess
root = Path(__file__).resolve().parents[1]
certs = root / 'certs'
for name in ('server', 'lb-client', 'api1', 'api2'):
for suffix in ('.crt', '.key', '.pem'):
path = certs / (name + suffix)
if path.exists():
path.unlink()
subprocess.run(['python', str(root / 'scripts' / 'bootstrap.py')], check=True)
print('Leaf certificates rotated; CA and application images were not replaced.')
+3
View File
@@ -0,0 +1,3 @@
@echo off
cd /d "%~dp0.."
python scripts/action.py stop-one
+4
View File
@@ -0,0 +1,4 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
exec python3 scripts/action.py stop-one
+81
View File
@@ -0,0 +1,81 @@
"""Run public and student acceptance, preserving sanitized evidence."""
import json
import os
import platform
import subprocess
import sys
import time
from datetime import datetime, timezone
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
os.chdir(ROOT)
EVIDENCE = ROOT / 'evidence'
ENV = {**os.environ, 'PYTHONUTF8': '1', 'BASE_URL': os.getenv('BASE_URL', 'https://127.0.0.1:8443'), 'CA_FILE': os.getenv('CA_FILE', 'certs/ca.crt')}
def run(args, artifact):
result = subprocess.run(args, cwd=ROOT, capture_output=True, text=True,
encoding='utf-8', errors='replace', env=ENV)
output = result.stdout + result.stderr
(EVIDENCE / artifact).write_text(output, encoding='utf-8')
print(output, end='', flush=True)
if result.returncode:
raise SystemExit(result.returncode)
return output
def containers():
ids = subprocess.check_output(['docker', 'compose', 'ps', '-q'], text=True).split()
raw = json.loads(subprocess.check_output(['docker', 'inspect', *ids], text=True))
return [{'name': x['Name'], 'id': x['Id'], 'started_at': x['State']['StartedAt'],
'image': x['Config']['Image']} for x in raw]
def main():
run([sys.executable, 'scripts/check_contract.py'], 'contract.txt')
run([sys.executable, 'tests/smoke.py'], 'smoke.txt')
before = containers()
suffix = '.cmd' if os.name == 'nt' else '.sh'
run([sys.executable, 'tests/resilience.py', '--mode', 'persistence',
'--action-script', 'scripts/restart' + suffix], 'persistence.txt')
after = containers()
old = {x['id']: x for x in before}
assert all(x['id'] in old and x['started_at'] != old[x['id']]['started_at'] for x in after)
(EVIDENCE / 'restart.json').write_text(json.dumps({'before':before,'after':after},indent=2),encoding='utf-8')
try:
run([sys.executable, 'tests/resilience.py', '--mode', 'failover',
'--action-script', 'scripts/stop-one' + suffix], 'failover.txt')
finally:
subprocess.run(['docker','compose','up','-d','--no-deps','--wait','api1','api2'], check=True)
time.sleep(2)
run([sys.executable, 'student-tests/replication.py'], 'replication.txt')
if (ROOT / 'student-tests/attachments.py').exists():
run([sys.executable, 'student-tests/attachments.py'], 'attachments.txt')
if (ROOT / 'student-tests/pipeline.py').exists():
run([sys.executable, 'student-tests/pipeline.py'], 'pipeline.txt')
if (ROOT / 'student-tests/observability.py').exists():
run([sys.executable, 'student-tests/observability.py'], 'observability.txt')
if (ROOT / 'student-tests/mtls.py').exists():
run([sys.executable, 'student-tests/mtls.py'], 'mtls.txt')
process = subprocess.Popen([sys.executable,'tests/load.py','--duration','30','--concurrency','4',
'--output','evidence/load.json'], stdout=subprocess.PIPE,stderr=subprocess.STDOUT,
text=True,encoding='utf-8',env=ENV)
ids = [x['id'] for x in containers()]
samples=[]
while process.poll() is None:
stats=subprocess.check_output(['docker','stats','--no-stream','--format',
'{{.Name}} CPU={{.CPUPerc}} RAM={{.MemUsage}}',*ids],text=True)
samples.append({'at':datetime.now(timezone.utc).isoformat(),'stats':stats.strip()})
time.sleep(2)
output=process.communicate()[0]
(EVIDENCE/'load.txt').write_text(output,encoding='utf-8')
(EVIDENCE/'resources.json').write_text(json.dumps(samples,indent=2),encoding='utf-8')
assert process.returncode==0, output
(EVIDENCE/'environment.json').write_text(json.dumps({'platform':platform.platform(),
'cpu':platform.processor(),'logical_cpus':os.cpu_count(),'observed_at':datetime.now(timezone.utc).isoformat()},indent=2),encoding='utf-8')
print(output,flush=True)
if __name__ == '__main__':
main()
View File
+433
View File
@@ -0,0 +1,433 @@
"""Lab 2 API: PostgreSQL is authoritative; Redis caches immutable public profiles."""
import base64
import binascii
import hashlib
import json
import os
import re
import secrets
import time
import uuid
from concurrent.futures import ThreadPoolExecutor, TimeoutError as FutureTimeout
from datetime import datetime
from threading import BoundedSemaphore
import psycopg
import redis
from botocore.exceptions import BotoCoreError, ClientError
from argon2 import PasswordHasher
from argon2.exceptions import VerificationError
from flask import Flask, g, jsonify, request
from psycopg_pool import PoolTimeout, TooManyRequests
from psycopg.types.json import Jsonb
from werkzeug.exceptions import HTTPException
from src.db import make_pool, setting
from src.telemetry import HTTP_REQUESTS, HTTP_DURATION, metrics_response, setup
app = Flask(__name__)
setup()
app.config['MAX_CONTENT_LENGTH'] = 10485760
pool = make_pool()
cache = redis.Redis.from_url(setting('CACHE_URL'), socket_connect_timeout=0.2,
socket_timeout=0.2, decode_responses=True)
cache_executor = ThreadPoolExecutor(max_workers=2, thread_name_prefix='cache')
cache_slots = BoundedSemaphore(2)
hasher = PasswordHasher(time_cost=2, memory_cost=19456, parallelism=1)
dummy_hash = hasher.hash(secrets.token_urlsafe(32))
SESSION_TTL = int(os.getenv('SESSION_TTL_SECONDS', '3600'))
if SESSION_TTL < 1:
raise ValueError('SESSION_TTL_SECONDS must be positive')
USER_FIELDS = 'id, username, display_name, created_at'
INSTANCE_ID = os.getenv('INSTANCE_ID', uuid.uuid4().hex)
class APIError(Exception):
def __init__(self, status, code, message, challenge=None):
self.status, self.code, self.message, self.challenge = status, code, message, challenge
def invalid():
raise APIError(400, 'invalid_request', 'Invalid request')
def unauthorized(basic=False):
challenge = 'Basic realm="messenger", charset="UTF-8"' if basic else 'Bearer realm="messenger"'
raise APIError(401, 'unauthorized', 'Authentication required', challenge)
def serial(value):
if isinstance(value, dict):
return {key: serial(item) for key, item in value.items()}
if isinstance(value, list):
return [serial(item) for item in value]
if isinstance(value, uuid.UUID):
return str(value)
if isinstance(value, datetime):
return value.isoformat().replace('+00:00', 'Z')
return value
def response(value, status=200):
return jsonify(serial(value)), status
@app.before_request
def identify_request():
g.request_id = uuid.uuid4().hex
g.started_at = time.monotonic()
@app.after_request
def headers(result):
result.headers['X-Request-Id'] = g.request_id
result.headers['X-Instance-Id'] = INSTANCE_ID
result.headers['Cache-Control'] = 'no-store'
route = request.url_rule.rule if request.url_rule else 'unknown'
HTTP_REQUESTS.labels(request.method, route, result.status_code).inc()
HTTP_DURATION.labels(request.method, route).observe(time.monotonic() - g.started_at)
app.logger.info('http_request method=%s route=%s status=%s request_id=%s', request.method, route, result.status_code, g.request_id)
return result
@app.get('/metrics')
def metrics():
return metrics_response()
@app.errorhandler(Exception)
def error(exc):
if isinstance(exc, APIError):
problem = exc
elif isinstance(exc, psycopg.errors.UniqueViolation):
problem = APIError(409, 'conflict', 'Username already exists')
elif isinstance(exc, psycopg.errors.ForeignKeyViolation):
problem = APIError(400, 'invalid_request', 'Invalid member list')
elif isinstance(exc, (BotoCoreError, ClientError, psycopg.OperationalError, PoolTimeout, TooManyRequests,
psycopg.errors.QueryCanceled, psycopg.errors.LockNotAvailable)):
problem = APIError(503, 'unavailable', 'Database temporarily unavailable')
elif isinstance(exc, HTTPException):
status = exc.code if exc.code in (400, 404, 413, 415) else 400
code = {400: 'invalid_request', 404: 'not_found', 413: 'payload_too_large',
415: 'unsupported_media_type'}[status]
problem = APIError(status, code, 'Request cannot be processed')
else:
app.logger.error('Unhandled error type=%s request_id=%s', type(exc).__name__, g.request_id)
problem = APIError(500, 'internal_error', 'Internal error')
result = jsonify(error={'code': problem.code, 'message': problem.message,
'request_id': g.request_id})
result.status_code = problem.status
if problem.challenge:
result.headers['WWW-Authenticate'] = problem.challenge
if problem.status == 429:
result.headers['Retry-After'] = '1'
return result
def body(fields):
if request.mimetype != 'application/json':
raise APIError(415, 'unsupported_media_type', 'Expected application/json')
value = request.get_json()
if not isinstance(value, dict) or set(value) != set(fields):
invalid()
return value
def string(value, minimum, maximum):
if not isinstance(value, str) or not minimum <= len(value) <= maximum or not value.strip():
invalid()
try:
value.encode('utf-8')
except UnicodeError:
invalid()
if '\x00' in value:
invalid()
return value
def uid(value):
try:
if not isinstance(value, str) or not re.fullmatch(
r'[0-9a-fA-F]{8}(?:-[0-9a-fA-F]{4}){3}-[0-9a-fA-F]{12}', value):
invalid()
return uuid.UUID(value)
except ValueError:
invalid()
def cached_operation(operation, *args, **kwargs):
# Socket timeouts do not bound DNS resolution when a Compose service is down.
# At most two background operations per worker, no unbounded queue.
if not cache_slots.acquire(blocking=False):
return None
future = cache_executor.submit(operation, *args, **kwargs)
future.add_done_callback(lambda _: cache_slots.release())
try:
return future.result(timeout=0.25)
except (redis.RedisError, FutureTimeout):
return None
def profile(conn, user_id):
key = 'profile:v1:' + str(user_id)
cached = cached_operation(cache.get, key)
if cached:
try:
return json.loads(cached)
except ValueError:
pass
user = serial(conn.execute('SELECT ' + USER_FIELDS + ' FROM users WHERE id=%s',
(user_id,)).fetchone())
if user is None:
unauthorized()
cached_operation(cache.set, key, json.dumps(user), ex=60)
return user
def actor(conn):
auth = request.headers.get('Authorization', '').split()
if len(auth) != 2 or auth[0].lower() != 'bearer' or len(auth[1]) > 256:
unauthorized()
digest = hashlib.sha256(auth[1].encode()).hexdigest()
session = conn.execute('SELECT user_id FROM sessions WHERE token_hash=%s '
'AND expires_at > clock_timestamp()', (digest,)).fetchone()
if not session:
unauthorized()
g.token_hash = digest
return session['user_id']
def chat_access(conn, chat_id, user_id):
chat = conn.execute('SELECT c.* FROM chats c JOIN members m ON m.chat_id=c.id '
'WHERE c.id=%s AND m.user_id=%s', (chat_id, user_id)).fetchone()
if not chat:
raise APIError(404, 'not_found', 'Resource not found')
return chat
def chat_members(conn, chat):
chat['member_ids'] = [row['user_id'] for row in conn.execute(
'SELECT user_id FROM members WHERE chat_id=%s ORDER BY user_id', (chat['id'],))]
return chat
def page_args(scope):
limit = request.args.get('limit', '50')
if not re.fullmatch(r'[0-9]{1,3}', limit) or not 1 <= int(limit) <= 100:
invalid()
cursor = request.args.get('cursor')
after = None
if cursor is not None:
try:
if len(cursor) > 1024:
invalid()
data = json.loads(base64.b64decode(cursor, altchars=b'-_', validate=True))
if not isinstance(data, list) or len(data) != 3 or data[0] != scope:
invalid()
date = datetime.fromisoformat(data[1])
if date.tzinfo is None:
invalid()
after = (date, uid(data[2]))
except (ValueError, TypeError, binascii.Error):
invalid()
return int(limit), after
def page(rows, limit, scope):
more = len(rows) > limit
items = rows[:limit]
cursor = None
if more:
last = serial(items[-1])
cursor = base64.urlsafe_b64encode(json.dumps(
[scope, last['created_at'], last['id']]).encode()).decode()
return {'items': items, 'next_cursor': cursor}
@app.get('/health/live')
def live():
return response({'status': 'ok'})
@app.get('/health/ready')
def ready():
with pool.connection() as conn:
conn.execute('SELECT 1 FROM schema_migrations LIMIT 1')
return response({'status': 'ok'})
@app.post('/api/v1/users')
def register():
data = body(('username', 'display_name', 'password'))
name = string(data['username'], 3, 32)
if not re.fullmatch('[a-z0-9_]{3,32}', name):
invalid()
display = string(data['display_name'], 1, 100)
password = string(data['password'], 12, 128)
encoded = hasher.hash(password)
with pool.connection() as conn:
user = conn.execute('INSERT INTO users (id,username,display_name,password_hash) '
'VALUES (%s,%s,%s,%s) RETURNING ' + USER_FIELDS,
(uuid.uuid4(), name, display, encoded)).fetchone()
return response(user, 201)
@app.post('/api/v1/auth/sessions')
def login():
auth = request.headers.get('Authorization', '').split()
if len(auth) != 2 or auth[0].lower() != 'basic' or len(auth[1]) > 2048:
unauthorized(True)
try:
name, password = base64.b64decode(auth[1], validate=True).decode('utf-8').split(':', 1)
except (ValueError, UnicodeError, binascii.Error):
unauthorized(True)
if not re.fullmatch('[a-z0-9_]{3,32}', name) or not 12 <= len(password) <= 128:
unauthorized(True)
if request.get_data():
invalid()
with pool.connection() as conn:
user = conn.execute('SELECT * FROM users WHERE username=%s', (name,)).fetchone()
try:
hasher.verify(user['password_hash'] if user else dummy_hash, password)
except VerificationError:
unauthorized(True)
if not user:
unauthorized(True)
token = secrets.token_urlsafe(32)
with pool.connection() as conn:
expiry = conn.execute('INSERT INTO sessions VALUES (%s,%s,clock_timestamp() '
'+ %s * interval \'1 second\') RETURNING expires_at',
(hashlib.sha256(token.encode()).hexdigest(), user['id'],
SESSION_TTL)).fetchone()['expires_at']
user.pop('password_hash')
return response({'token': token, 'token_type': 'Bearer', 'expires_at': expiry, 'user': user}, 201)
@app.delete('/api/v1/auth/sessions/current')
def logout():
with pool.connection() as conn:
actor(conn)
conn.execute('DELETE FROM sessions WHERE token_hash=%s', (g.token_hash,))
return '', 204
@app.get('/api/v1/users/me')
def me():
with pool.connection() as conn:
user = profile(conn, actor(conn))
return response(user)
@app.post('/api/v1/chats')
def create_chat():
with pool.connection() as conn:
user = actor(conn)
data = body(('title', 'member_ids'))
title = string(data['title'], 1, 100)
if not isinstance(data['member_ids'], list) or not 1 <= len(data['member_ids']) <= 100:
invalid()
members = [uid(item) for item in data['member_ids']]
if len(set(members)) != len(members) or user not in members:
invalid()
chat = conn.execute('INSERT INTO chats(id,title) VALUES (%s,%s) RETURNING *',
(uuid.uuid4(), title)).fetchone()
for member in members:
conn.execute('INSERT INTO members VALUES (%s,%s)', (chat['id'], member))
chat['member_ids'] = sorted(members)
return response(chat, 201)
@app.get('/api/v1/chats')
def list_chats():
with pool.connection() as conn:
user = actor(conn)
scope = 'chats:' + str(user)
limit, after = page_args(scope)
sql = 'SELECT c.* FROM chats c JOIN members m ON m.chat_id=c.id WHERE m.user_id=%s'
params = [user]
if after:
sql += ' AND (c.created_at,c.id) > (%s,%s)'
params.extend(after)
rows = conn.execute(sql + ' ORDER BY c.created_at,c.id LIMIT %s', params + [limit+1]).fetchall()
result = page(rows, limit, scope)
for chat in result['items']:
chat_members(conn, chat)
return response(result)
@app.get('/api/v1/chats/<chat_id>')
def get_chat(chat_id):
with pool.connection() as conn:
user = actor(conn)
result = chat_members(conn, chat_access(conn, uid(chat_id), user))
return response(result)
@app.route('/api/v1/chats/<chat_id>/messages', methods=['GET', 'POST'])
def messages(chat_id):
with pool.connection() as conn:
user = actor(conn)
chat_id = uid(chat_id)
chat_access(conn, chat_id, user)
if request.method == 'POST':
if request.mimetype != 'application/json':
raise APIError(415, 'unsupported_media_type', 'Expected application/json')
data = request.get_json()
if not isinstance(data, dict) or 'text' not in data or set(data) - {'text', 'attachment_ids'}:
invalid()
text = data['text']
if not isinstance(text, str) or len(text) > 4000 or '\x00' in text:
invalid()
try:
text.encode('utf-8')
except UnicodeError:
invalid()
attachments = data.get('attachment_ids', [])
if not isinstance(attachments, list) or len(attachments) > 10:
invalid()
attachments = [uid(item) for item in attachments]
if len(set(attachments)) != len(attachments) or not (text.strip() or attachments):
invalid()
for attachment_id in attachments:
attachment = conn.execute('SELECT status FROM attachments WHERE id=%s AND chat_id=%s',
(attachment_id, chat_id)).fetchone()
if not attachment:
raise APIError(404, 'not_found', 'Attachment not found')
if attachment['status'] == 'failed':
raise APIError(409, 'conflict', 'Attachment failed')
key = request.headers.get('Idempotency-Key', '')
if not re.fullmatch(r'[A-Za-z0-9._:-]{1,128}', key):
invalid()
canonical = {'text': text, 'attachment_ids': [str(item) for item in attachments]}
lock = int.from_bytes(hashlib.sha256(f'{user}:{chat_id}:{key}'.encode()).digest()[:8],
'big', signed=True)
conn.execute('SELECT pg_advisory_xact_lock(%s)', (lock,))
conn.execute('DELETE FROM idempotency WHERE user_id=%s AND chat_id=%s AND key=%s '
'AND expires_at<=clock_timestamp()', (user, chat_id, key))
prior = conn.execute('SELECT * FROM idempotency WHERE user_id=%s AND chat_id=%s AND key=%s',
(user, chat_id, key)).fetchone()
if prior:
if prior['request'] != canonical:
raise APIError(409, 'conflict', 'Idempotency key already used with a different message')
return response(prior['response'], 201)
result = conn.execute('INSERT INTO messages(id,chat_id,sender_id,text,attachment_ids) '
'VALUES (%s,%s,%s,%s,%s) RETURNING *',
(uuid.uuid4(), chat_id, user, text, attachments)).fetchone()
conn.execute('INSERT INTO idempotency(user_id,chat_id,key,request,response) '
'VALUES (%s,%s,%s,%s,%s)',
(user, chat_id, key, Jsonb(canonical), Jsonb(serial(result))))
status = 201
else:
scope = 'messages:' + str(chat_id)
limit, after = page_args(scope)
sql = 'SELECT * FROM messages WHERE chat_id=%s'
params = [chat_id]
if after:
sql += ' AND (created_at,id) > (%s,%s)'
params.extend(after)
rows = conn.execute(sql + ' ORDER BY created_at,id LIMIT %s', params + [limit+1]).fetchall()
result, status = page(rows, limit, scope), 200
return response(result, status)
from src.attachments import register_routes
register_routes(app, pool, actor, chat_access, uid, response, APIError, invalid)
+123
View File
@@ -0,0 +1,123 @@
import hashlib
import tempfile
import uuid
import warnings
from threading import BoundedSemaphore
from flask import Response, request, stream_with_context
from PIL import Image, UnidentifiedImageError
from src.storage import BUCKET, s3
MAX_BYTES = 10 * 1024 * 1024
Image.MAX_IMAGE_PIXELS = 40_000_000
# Each API worker is bounded by Gunicorn threads; this semaphore prevents an
# unbounded number of uploads without serializing the required concurrent case.
slots = BoundedSemaphore(16)
def public(row):
return {k: v for k, v in row.items() if k != 'object_key'}
def register_routes(app, pool, actor, chat_access, uid, response, APIError, invalid):
@app.post('/api/v1/chats/<chat_id>/attachments')
def upload(chat_id):
with pool.connection() as conn:
user = actor(conn)
chat_id = uid(chat_id)
chat_access(conn, chat_id, user)
filename = request.args.get('filename', '')
if not 1 <= len(filename) <= 255 or not filename.strip() or any(
ord(c) < 32 or ord(c) == 127 or c in '/\\' for c in filename):
invalid()
if request.mimetype not in ('image/png', 'image/jpeg'):
raise APIError(415, 'unsupported_media_type', 'Expected PNG or JPEG')
if request.content_length and request.content_length > MAX_BYTES:
raise APIError(413, 'payload_too_large', 'Image too large')
if not slots.acquire(timeout=5):
raise APIError(429, 'rate_limited', 'Upload capacity reached')
try:
with tempfile.SpooledTemporaryFile(max_size=1024*1024) as file:
length, digest = 0, hashlib.sha256()
while True:
chunk = request.stream.read(65536)
if not chunk:
break
length += len(chunk)
if length > MAX_BYTES:
raise APIError(413, 'payload_too_large', 'Image too large')
digest.update(chunk)
file.write(chunk)
file.seek(0)
try:
with warnings.catch_warnings():
warnings.simplefilter('error', Image.DecompressionBombWarning)
with Image.open(file) as image:
if image.width * image.height > 40_000_000:
raise APIError(413, 'payload_too_large', 'Too many pixels')
if Image.MIME.get(image.format) != request.mimetype:
raise APIError(415, 'unsupported_media_type', 'Image type mismatch')
image.verify()
file.seek(0)
with Image.open(file) as image:
image.load()
except (Image.DecompressionBombError, Image.DecompressionBombWarning):
raise APIError(413, 'payload_too_large', 'Too many pixels') from None
except (UnidentifiedImageError, OSError, ValueError, SyntaxError):
raise APIError(415, 'unsupported_media_type', 'Invalid image') from None
file.seek(0)
identifier = uuid.uuid4()
key = 'original/' + str(identifier)
with pool.connection() as conn:
conn.execute('INSERT INTO object_intents(object_key) VALUES (%s)', (key,))
s3.put_object(Bucket=BUCKET, Key=key, Body=file, ContentLength=length, ContentType=request.mimetype)
# Intent survives a crash or failed DB commit; the janitor removes old unreferenced objects.
with pool.connection() as conn:
row = conn.execute("INSERT INTO attachments(id,chat_id,owner_id,filename,content_type,size_bytes,sha256,object_key,status) "
"VALUES (%s,%s,%s,%s,%s,%s,%s,%s,'queued') RETURNING *",
(identifier,chat_id,user,filename,request.mimetype,length,digest.hexdigest(),key)).fetchone()
conn.execute('INSERT INTO attachment_jobs(attachment_id) VALUES (%s)', (identifier,))
conn.execute('DELETE FROM object_intents WHERE object_key=%s', (key,))
return response(public(row),202)
finally:
slots.release()
def authorized(attachment_id):
with pool.connection() as conn:
user = actor(conn)
identifier = uid(attachment_id)
row = conn.execute('SELECT a.* FROM attachments a JOIN members m ON m.chat_id=a.chat_id '
'WHERE a.id=%s AND m.user_id=%s', (identifier,user)).fetchone()
if not row:
raise APIError(404,'not_found','Attachment not found')
return row
@app.get('/api/v1/attachments/<attachment_id>')
def metadata(attachment_id):
return response(public(authorized(attachment_id)))
@app.get('/api/v1/attachments/<attachment_id>/content')
def download(attachment_id):
row = authorized(attachment_id)
if row['status'] != 'ready':
raise APIError(409,'conflict','Attachment not ready')
variant = request.args.get('variant','original')
key, content_type, size_bytes = row['object_key'], row['content_type'], row['size_bytes']
if variant != 'original':
found = next((item for item in row['variants'] if item['name'] == variant), None)
if not found:
raise APIError(404,'not_found','Variant not found')
key = row['object_key'] + '.' + variant
content_type, size_bytes = found['content_type'], found['size_bytes']
obj = s3.get_object(Bucket=BUCKET,Key=key)
def chunks():
try:
while True:
chunk = obj['Body'].read(65536)
if not chunk:
break
yield chunk
finally:
obj['Body'].close()
return Response(stream_with_context(chunks()),content_type=content_type,
headers={'Content-Length':str(size_bytes),'X-Content-Type-Options':'nosniff'})
+37
View File
@@ -0,0 +1,37 @@
"""Old upload intent reconciliation; dry run unless --apply, never deletes fresh objects."""
import argparse
import json
import time
from src.db import make_pool
from src.storage import s3, BUCKET
def sweep(apply=False):
pool = make_pool()
with pool.connection() as conn:
rows = conn.execute("SELECT i.object_key FROM object_intents i LEFT JOIN attachments a "
"ON a.object_key=i.object_key WHERE a.id IS NULL "
"AND i.created_at < clock_timestamp()-interval '1 hour' FOR UPDATE OF i SKIP LOCKED").fetchall()
for row in rows:
if apply:
s3.delete_object(Bucket=BUCKET,Key=row['object_key'])
conn.execute('DELETE FROM object_intents WHERE object_key=%s',(row['object_key'],))
pool.close()
return {'old_unreferenced_intents':len(rows),'applied':apply}
if __name__ == '__main__':
parser=argparse.ArgumentParser()
parser.add_argument('--apply',action='store_true')
parser.add_argument('--loop',action='store_true')
args=parser.parse_args()
while True:
try:
print(json.dumps(sweep(args.apply)),flush=True)
except Exception as exc:
print(json.dumps({'cleanup_error':type(exc).__name__}),flush=True)
if not args.loop:
raise SystemExit(1)
if not args.loop:
break
time.sleep(60)
+55
View File
@@ -0,0 +1,55 @@
"""Bounded PostgreSQL pool and serialized, versioned migrations."""
import hashlib
import os
import time
from pathlib import Path
import psycopg
from psycopg.rows import dict_row
from psycopg_pool import ConnectionPool
from src.vault import secret
def setting(name):
if name in {'DATABASE_URL', 'S3_ACCESS_KEY', 'S3_SECRET_KEY'} and os.getenv('BAO_ROLE_ID_FILE'):
return secret(name)
path = os.getenv(name + '_FILE')
return Path(path).read_text().strip() if path else os.environ[name]
def migrate():
with psycopg.connect(setting('DATABASE_URL'), connect_timeout=3) as conn:
conn.execute('SELECT pg_advisory_xact_lock(202602)')
conn.execute('CREATE TABLE IF NOT EXISTS schema_migrations '
'(version text PRIMARY KEY, checksum text NOT NULL)')
for file in sorted(Path('migrations').glob('*.sql')):
sql = file.read_text(encoding='utf-8')
digest = hashlib.sha256(sql.encode()).hexdigest()
row = conn.execute('SELECT checksum FROM schema_migrations WHERE version=%s',
(file.name,)).fetchone()
if row:
if row[0] != digest:
raise RuntimeError('Applied migration changed')
continue
conn.execute(sql)
conn.execute('INSERT INTO schema_migrations VALUES (%s,%s)', (file.name, digest))
def make_pool():
return ConnectionPool(setting('DATABASE_URL'), min_size=1, max_size=8,
timeout=3, max_waiting=32,
check=ConnectionPool.check_connection,
kwargs={'connect_timeout': 3, 'row_factory': dict_row,
'options': '-c statement_timeout=3000 -c lock_timeout=2000'})
if __name__ == '__main__':
for attempt in range(30):
try:
migrate()
break
except psycopg.OperationalError:
if attempt == 29:
raise RuntimeError('Database did not become available') from None
time.sleep(2)
print('Migrations applied successfully')
+10
View File
@@ -0,0 +1,10 @@
import os
import boto3
from botocore.config import Config
from src.db import setting
BUCKET = os.getenv('S3_BUCKET', 'messenger')
s3 = boto3.client('s3', endpoint_url=os.getenv('S3_ENDPOINT', 'http://lb:9000'),
aws_access_key_id=setting('S3_ACCESS_KEY'), aws_secret_access_key=setting('S3_SECRET_KEY'),
region_name='us-east-1', config=Config(connect_timeout=2, read_timeout=5,
retries={'max_attempts': 0}, s3={'addressing_style': 'path'}))
+22
View File
@@ -0,0 +1,22 @@
import json
import logging
import time
from prometheus_client import Counter, Histogram, Gauge, generate_latest, CONTENT_TYPE_LATEST
HTTP_REQUESTS = Counter('messenger_http_requests_total','HTTP requests',['method','route','status'])
HTTP_DURATION = Histogram('messenger_http_request_duration_seconds','HTTP duration',['method','route'])
JOBS = Counter('messenger_attachment_jobs_total','Attachment jobs',['status'])
QUEUE_AGE = Gauge('messenger_attachment_oldest_job_age_seconds','Oldest pending job age')
class JsonFormatter(logging.Formatter):
def format(self, record):
return json.dumps({'ts': self.formatTime(record), 'level': record.levelname,
'message': record.getMessage(), 'trace_id': getattr(record,'trace_id',None)})
def setup():
handler=logging.StreamHandler(); handler.setFormatter(JsonFormatter())
logging.getLogger().handlers[:] = [handler]; logging.getLogger().setLevel(logging.INFO)
def metrics_response():
from flask import Response
return Response(generate_latest(), content_type=CONTENT_TYPE_LATEST)
+29
View File
@@ -0,0 +1,29 @@
"""Small AppRole client used at process startup; no root token is accepted here."""
import json, os, urllib.request
from pathlib import Path
_cache = {}
def _read(name):
path = os.getenv(name)
return Path(path).read_text().strip() if path else None
def _call(path, method='GET', payload=None, token=None):
body = None if payload is None else json.dumps(payload).encode()
req = urllib.request.Request(os.environ.get('BAO_ADDR', 'http://openbao:8200') + path,
data=body, method=method,
headers={'Content-Type':'application/json', **({'X-Vault-Token': token} if token else {})})
with urllib.request.urlopen(req, timeout=3) as response:
return json.loads(response.read() or b'{}')
def secret(name):
if name in _cache: return _cache[name]
role_id, secret_id = _read('BAO_ROLE_ID_FILE'), _read('BAO_SECRET_ID_FILE')
if not role_id or not secret_id:
raise RuntimeError('AppRole credentials are required; refusing file-secret fallback')
login = _call('/v1/auth/approle/login', 'POST', {'role_id': role_id, 'secret_id': secret_id})
token = login['auth']['client_token']
data = _call('/v1/secret/data/' + os.getenv('BAO_SECRET_PATH', 'messenger/api'), token=token).get('data', {}).get('data', {})
if name not in data: raise RuntimeError('secret is not present in OpenBao: ' + name)
_cache[name] = data[name]
return data[name]
+60
View File
@@ -0,0 +1,60 @@
import hashlib
import io
import json
import random
import time
from PIL import Image, ImageOps
from src.db import make_pool
from src.storage import BUCKET, s3
MAX_ATTEMPTS = 5
def make_variant(image, name):
image = ImageOps.exif_transpose(image)
if name == 'thumbnail':
image.thumbnail((256, 256), Image.Resampling.LANCZOS); target = image
elif name == 'crop':
target = ImageOps.fit(image, (128, 128), method=Image.Resampling.LANCZOS)
else:
image.thumbnail((1280, 1280), Image.Resampling.LANCZOS); target = image
if target.mode not in ('RGB', 'L'):
bg = Image.new('RGB', target.size, 'white')
bg.paste(target, mask=target.getchannel('A') if 'A' in target.getbands() else None)
target = bg
out = io.BytesIO()
content_type = 'image/webp' if name == 'optimized' else 'image/jpeg'
target.convert('RGB').save(out, format='WEBP' if name == 'optimized' else 'JPEG', quality=82, exif=b'')
raw = out.getvalue()
return raw, content_type, target.width, target.height
def process(row, pool):
raw = s3.get_object(Bucket=BUCKET, Key=row['object_key'])['Body'].read()
with Image.open(io.BytesIO(raw)) as image:
variants=[]
for name in ('thumbnail','crop','optimized'):
data, content_type, width, height = make_variant(image.copy(), name)
s3.put_object(Bucket=BUCKET, Key=row['object_key']+'.'+name, Body=data, ContentType=content_type)
variants.append({'name':name,'content_type':content_type,'size_bytes':len(data),'width':width,'height':height,'sha256':hashlib.sha256(data).hexdigest()})
with pool.connection() as conn:
conn.execute("UPDATE attachments SET status='ready',variants=%s,error_code=NULL,updated_at=clock_timestamp() WHERE id=%s AND status<>'ready'", (json.dumps(variants),row['id']))
conn.execute('UPDATE attachment_jobs SET completed_at=clock_timestamp(),locked_until=NULL WHERE attachment_id=%s',(row['id'],))
def loop():
pool=make_pool()
while True:
with pool.connection() as conn:
job=conn.execute("UPDATE attachment_jobs SET attempts=attempts+1,locked_until=clock_timestamp()+interval '60 seconds' WHERE attachment_id=(SELECT attachment_id FROM attachment_jobs WHERE completed_at IS NULL AND available_at<=clock_timestamp() AND (locked_until IS NULL OR locked_until<clock_timestamp()) ORDER BY created_at FOR UPDATE SKIP LOCKED LIMIT 1) RETURNING *").fetchone()
row=conn.execute('SELECT * FROM attachments WHERE id=%s',(job['attachment_id'],)).fetchone() if job else None
if not row:
time.sleep(.25); continue
try: process(row,pool)
except Exception as error:
with pool.connection() as conn:
if job['attempts'] >= MAX_ATTEMPTS:
conn.execute("UPDATE attachments SET status='failed',error_code='processing_failed',updated_at=clock_timestamp() WHERE id=%s",(row['id'],))
conn.execute('UPDATE attachment_jobs SET completed_at=clock_timestamp(),last_error=%s WHERE attachment_id=%s',(type(error).__name__,row['id']))
else:
delay=min(60,2**job['attempts'])+random.random()
conn.execute("UPDATE attachment_jobs SET available_at=clock_timestamp()+(%s*interval '1 second'),locked_until=NULL,last_error=%s WHERE attachment_id=%s",(delay,type(error).__name__,row['id']))
if __name__ == '__main__': loop()
+60
View File
@@ -0,0 +1,60 @@
"""Attachment negative checks, private-bucket check, and concurrent upload budget."""
import base64
import hashlib
import os
import subprocess
import sys
import threading
import time
import uuid
from concurrent.futures import ThreadPoolExecutor
from urllib.request import Request, urlopen
from urllib.error import HTTPError, URLError
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / 'tests'))
from client import Client, check, Failure
PNG = (b'\x89PNG\r\n\x1a\n' + b'\x00' * 100)
def main():
c = Client()
alice, bob, eve, chat = c.bootstrap()
path = '/api/v1/chats/' + chat['id'] + '/attachments?filename=photo.png'
att, content = c.attachment(alice, chat)
check(att['status'] == 'ready' and att['sha256'] == hashlib.sha256(content).hexdigest(), 'metadata mismatch')
downloaded = c.call('GET', '/api/v1/attachments/' + att['id'] + '/content', actor=bob)
check(downloaded['bytes'] == content, 'download mismatch')
c.call('GET', '/api/v1/attachments/' + att['id'], 404, actor=eve)
c.call('GET', '/api/v1/attachments/' + att['id'] + '/content', 404, actor=eve)
c.call('GET', '/api/v1/attachments/' + str(uuid.uuid4()), 404, actor=alice)
c.call('POST', '/api/v1/chats/' + chat['id'] + '/attachments?filename=bad/name.png', 400,
actor=alice, raw=content, headers={'Content-Type': 'image/png'})
c.call('POST', '/api/v1/chats/' + chat['id'] + '/attachments?filename=spoof.png', 415,
actor=alice, raw=b'not-an-image', headers={'Content-Type': 'image/png'})
c.call('POST', '/api/v1/chats/' + chat['id'] + '/attachments?filename=spoof.txt', 415,
actor=alice, raw=content, headers={'Content-Type': 'text/plain'})
c.call('POST', '/api/v1/chats/' + chat['id'] + '/attachments?filename=huge.png', 413,
actor=alice, raw=b'0' * (10 * 1024 * 1024 + 1), headers={'Content-Type': 'image/png'})
responses = []
def upload(_):
return c.attachment(alice, chat)[0]['id']
with ThreadPoolExecutor(max_workers=10) as pool:
responses = list(pool.map(upload, range(10)))
check(len(set(responses)) == 10, 'concurrent uploads shared an id')
try:
urlopen('http://127.0.0.1:9000/messenger/original/' + str(att['id']), timeout=3)
except HTTPError as error:
check(error.code in (400, 403), 'bucket/object is publicly readable')
else:
raise AssertionError('bucket/object is publicly readable')
stats = subprocess.check_output(['docker', 'stats', '--no-stream', '--format', '{{.Name}}={{.MemUsage}}',
'vp-lab4-api1-1', 'vp-lab4-api2-1'], text=True)
Path(ROOT / 'evidence/attachment-resources.txt').write_text(stats, encoding='utf-8')
print('PASS: attachment access, validation, private bucket and 10 concurrent uploads')
if __name__ == '__main__':
main()
+10
View File
@@ -0,0 +1,10 @@
"""Prove that an API replica rejects unauthenticated internal TLS clients."""
import subprocess
base = ["docker", "compose", "exec", "-T", "api1", "python", "-c"]
no_cert = "import ssl,urllib.request;c=ssl.create_default_context(cafile='/run/certs/ca.crt');c.check_hostname=False;urllib.request.urlopen('https://api1:8080/health/live',context=c,timeout=3)"
with_cert = "import ssl,urllib.request;c=ssl.create_default_context(cafile='/run/certs/ca.crt');c.check_hostname=False;c.load_cert_chain('/run/certs/lb-client.crt','/run/certs/lb-client.key');assert urllib.request.urlopen('https://api1:8080/health/live',context=c,timeout=3).status==200"
rejected = subprocess.run(base + [no_cert], capture_output=True).returncode != 0
accepted = subprocess.run(base + [with_cert], capture_output=True).returncode == 0
assert rejected and accepted, 'internal mTLS negative/positive check failed'
print('PASS: API rejects a client without certificate and accepts the LB identity.')
+101
View File
@@ -0,0 +1,101 @@
"""Concurrent idempotency, both replica failures, and rolling recreation under writes."""
import json
import os
import subprocess
import sys
import threading
import time
import uuid
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from urllib.parse import quote
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / 'tests'))
from client import Client, check, Failure
def compose(*args):
subprocess.run(['docker', 'compose', *args], cwd=ROOT, check=True,
stdout=subprocess.DEVNULL, timeout=120)
def history(c, actor, chat):
path = '/api/v1/chats/' + chat['id'] + '/messages?limit=100'
result = []
while path:
page = c.call('GET', path, actor=actor, schema='MessagePage')['json']
result.extend(page['items'])
path = '/api/v1/chats/' + chat['id'] + '/messages?limit=100&cursor=' + quote(page['next_cursor'], safe='') if page['next_cursor'] else None
return result
def main():
c = Client(timeout=5)
alice, bob, eve, chat = c.bootstrap()
key = uuid.uuid4().hex
def send(_):
return Client().message(alice, chat, 'One logical message', key=key)
with ThreadPoolExecutor(max_workers=12) as executor:
responses = list(executor.map(send, range(24)))
ids = {item['json']['id'] for item in responses}
instances = {item['headers']['X-Instance-Id'] for item in responses}
check(len(ids) == 1 and len(instances) >= 2, 'Concurrency did not cover both replicas or duplicated message')
check(len(history(c, alice, chat)) == 1, 'Duplicate row committed')
c.call('POST', '/api/v1/chats/' + chat['id'] + '/messages', 409, actor=alice,
headers={'Idempotency-Key': key}, body={'text': 'Different'})
results = [{'scenario': 'concurrent_idempotency', 'requests': 24,
'replicas': sorted(instances), 'unique_messages': 1}]
def experiment(label, action):
stop = threading.Event()
writes, errors, times = [], [], []
def traffic():
client = Client(timeout=3)
retry_key, text = uuid.uuid4().hex, label + uuid.uuid4().hex
while not stop.is_set():
try:
response = client.message(alice, chat, text, key=retry_key)
writes.append(response['json'])
times.append(time.monotonic())
retry_key, text = uuid.uuid4().hex, label + uuid.uuid4().hex
except Failure:
errors.append(time.monotonic())
time.sleep(0.02)
thread = threading.Thread(target=traffic)
thread.start()
started = time.monotonic()
try:
action()
time.sleep(5)
finally:
stop.set()
thread.join(timeout=10)
duration = time.monotonic() - started
stored = {item['id']: item for item in history(c, bob, chat)}
check(writes and all(stored.get(item['id']) == item for item in writes), 'Confirmed writes lost')
bodies = [item['text'] for item in stored.values()]
check(len(bodies) == len(set(bodies)), 'Retry duplicated a logical message')
gaps = [b-a for a,b in zip([started]+times, times+[time.monotonic()])]
check(max(gaps) < 10, 'Recovery exceeded 10 seconds')
results.append({'scenario': label, 'duration_seconds': round(duration,3),
'writes': len(writes), 'errors': len(errors),
'rps': round(len(writes)/duration,3), 'max_success_gap_seconds': round(max(gaps),3)})
for service in ('api1', 'api2'):
try:
experiment('stop_' + service, lambda: compose('stop', '-t', '2', service))
finally:
compose('up', '-d', '--no-deps', '--wait', service)
time.sleep(2)
def rolling():
for service in ('api1', 'api2'):
compose('up', '-d', '--no-deps', '--force-recreate', '--wait', service)
time.sleep(2)
experiment('rolling_recreation', rolling)
Path(ROOT / 'evidence/replication.json').write_text(json.dumps(results, indent=2), encoding='utf-8')
print(json.dumps(results, indent=2))
if __name__ == '__main__':
main()
+18
View File
@@ -0,0 +1,18 @@
"""Small security acceptance check for the local TLS boundary."""
import ssl
import urllib.request
import urllib.error
from pathlib import Path
root = Path(__file__).resolve().parents[1]
context = ssl.create_default_context(cafile=root / 'certs' / 'ca.crt')
with urllib.request.urlopen('https://127.0.0.1:8443/health/live', context=context, timeout=5) as response:
assert response.status == 200
try:
with urllib.request.urlopen('https://127.0.0.1:8443/health/live', context=ssl.create_default_context(), timeout=5):
raise AssertionError('untrusted certificate was accepted')
except (ssl.SSLCertVerificationError, urllib.error.URLError) as error:
if isinstance(error, urllib.error.URLError) and not isinstance(error.reason, ssl.SSLCertVerificationError):
raise
pass
print('PASS: trusted local TLS works and an untrusted certificate is rejected.')