feat(research) grade 5 assignment done
This commit is contained in:
@@ -1,9 +1,9 @@
|
||||
# Отчёт — лабораторная 2
|
||||
|
||||
- Автор, группа: Булла Максим, БПИ2301
|
||||
- Целевая оценка: **4**
|
||||
- Commit/PR: локальный тег `lab2-green`
|
||||
- Использованные источники, библиотеки и помощь ИИ: Go 1.27.1, Fiber v3.5.0, pgx v5.11.0, go-redis v9.23.0, Argon2id из `golang.org/x/crypto`; PostgreSQL 16.10, Redis 7.4.7. DeepSeek V4.1 Flash + Opencode Harness для генерации решений; все заявленные результаты получены командами из таблицы ниже и сохранены в `evidence/`.
|
||||
- Автор, группа: ``TODO``
|
||||
- Целевая оценка: 5, трек 2
|
||||
- Commit/PR: локальный тег ``TODO``
|
||||
- Использованные источники, библиотеки и помощь ИИ: Go 1.27.1, Fiber v3.5.0, pgx v5.11.0, go-redis v9.23.0, Argon2id из `golang.org/x/crypto`; PostgreSQL 16.10, Redis 7.4.7, `pg_dump`/`pg_restore`/`pgbench`. DeepSeek V4.1 Flash + Opencode Harness для генерации решений; все заявленные результаты получены командами из таблицы ниже и сохранены в `evidence/` и `research/results/`.
|
||||
|
||||
## Запуск из чистого clone
|
||||
|
||||
@@ -105,6 +105,8 @@ docker compose --env-file .env down -v # осознанно удалить
|
||||
| Гонка регистрации username | `grade4_checks.py` | 12 одновременных запросов → один `201`, одиннадцать `409` | `evidence/07-grade4-checks.txt` |
|
||||
| Rollback составной операции | `grade4_checks.py` | после ошибки чат с тем же title отсутствует в списке | `evidence/07-grade4-checks.txt` |
|
||||
| Keyset-пагинация и индекс, доказательство на ≥10 000 сообщений | `./scripts/seed_and_explain.sh` | `Index Scan using messages_chat_created_id_idx`, 5–6 shared buffers, ~0,09–0,10 мс | `evidence/08-explain.txt` |
|
||||
| Трек 2: выбор индекса при ограниченной памяти | `./research/bench_indexes.sh` | 300k сообщений; без индекса истории `chat_messages` 57,6 мс вместо 0,43 мс; без индекса членства `user_chats` 7,4 мс вместо 1,4 мс | `research/results/indexes.md` |
|
||||
| Трек 2: backup → restore и проверка | `./research/bench_recovery.sh` + `python3 research/analyze.py` | 100k/200k/300k: dump 1,02/1,92/2,66 с, restore 1,44/2,36/3,28 с; counts и MD5-отпечаток сообщений совпали | `research/results/recovery.csv`, `research/results/recovery.md` |
|
||||
| Автопроверки expiry/logout/wrong password/Redis/PostgreSQL outage | `grade4_checks.py` | все пункты `PASS` | `evidence/07-grade4-checks.txt` |
|
||||
| Redis недоступен: live 200, ready 503, protected 503 | `grade4_checks.py` (stop/start Redis) | live=200, ready=503, protected=503; после старта тот же токен и сообщение целы | `evidence/07-grade4-checks.txt` |
|
||||
| PostgreSQL недоступен: live 200, ready 503, protected 503 | `grade4_checks.py` (stop/start PostgreSQL) | live=200, ready=503, protected=503; сессия и данные пережили отказ | `evidence/07-grade4-checks.txt` |
|
||||
@@ -137,6 +139,30 @@ docker compose --env-file .env down -v # осознанно удалить
|
||||
|
||||
Сырые метрики — `evidence/load.json`, команда и вывод — `evidence/09-load.txt`.
|
||||
|
||||
### Трек 2: индекс и восстановление
|
||||
|
||||
Гипотеза: при 300k сообщений индекс истории полезнее индекса членства, поэтому при жёстком бюджете памяти первым стоит убрать `chat_members_user_idx`. Время восстановления должно расти линейно с числом сообщений.
|
||||
|
||||
Стенд: PostgreSQL 16.10, 5 000 чатов, 2 000 пользователей, 72 300 членств и 300k сообщений с перекошенными размерами чатов (крупнейший чат — около 6% истории). Схема создана миграцией `0001_init.up.sql`. В двух сравниваемых состояниях ровно один из двух индексов; состояние проверяется скриптом, в конце оба индекса возвращаются. Запросы: список чатов пользователя, keyset-страница сообщений одного чата и keyset-лента сообщений по всем чатам пользователя. В таблице — медиана пяти `EXPLAIN (ANALYZE, BUFFERS)`.
|
||||
|
||||
| Состояние | `user_chats`, мс | `chat_messages`, мс | `user_feed`, мс |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| оба индекса | 1,38 | 0,43 | 35,40 |
|
||||
| только `messages_chat_created_id_idx` | 7,36 | 0,47 | 35,22 |
|
||||
| только `chat_members_user_idx` | 1,40 | 57,63 | 80,40 |
|
||||
|
||||
Размеры: `chat_members_user_idx` — около 3,5 МБ, `messages_chat_created_id_idx` — около 17 МБ. `pgbench` на клонированной таблице (4 клиента, 10 с) дал 2 719 tps с индексом истории и 2 640 tps без него; разница в пределах шума. Без индекса истории страница чата замедляется примерно в 135 раз, а лента — в 2,3 раза; без индекса членства список чатов замедляется примерно в 5 раз. **Вывод: сохранять `messages_chat_created_id_idx`; при необходимости удалять `chat_members_user_idx`.**
|
||||
|
||||
Backup → restore выполнялся с обоими индексами на 100k, 200k и 300k сообщениях; после восстановления сверялись количество сообщений/членств, MD5-отпечаток всех сообщений и наличие обоих индексов.
|
||||
|
||||
| Сообщений | dump, с | restore, с | dump, МиБ | БД, МиБ | потеря при 878 RPS |
|
||||
| ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 100 000 | 1,02 | 1,44 | 4,9 | 45,2 | ~896 |
|
||||
| 200 000 | 1,92 | 2,36 | 9,3 | 70,8 | ~1 686 |
|
||||
| 300 000 | 2,66 | 3,28 | 13,8 | 95,0 | ~2 335 |
|
||||
|
||||
Линейные зависимости (`N` — миллионы сообщений): dump `t = 0,23 + 8,20·N` с, restore `t = 0,52 + 9,20·N` с, dump+restore `t = 0,75 + 17,4·N` с (R² ≥ 0,997). Потеря в таблице — это `dump_time × 878 RPS`; `pg_dump` не блокирует записи, поэтому оценка означает «записи после снимка», а не остановку API. При полном офлайн-восстановлении тот же расчёт по `dump + restore` даёт около 2,2k / 3,7k / 5,2k сообщений. Подробная методика и команды — `research/README.md`.
|
||||
|
||||
### Отказы зависимостей
|
||||
|
||||
| Воздействие | Что ожидали | Что наблюдали | Восстановление / потери |
|
||||
@@ -152,4 +178,5 @@ docker compose --env-file .env down -v # осознанно удалить
|
||||
- AOF с `appendfsync everysec`: при аварийном убийстве процесса возможна потеря до ~1 секунды последних операций с сессиями. Graceful-перезапуск, который использует `scripts/restart.sh`, проходит без потерь.
|
||||
- Список `member_ids` в ответе чата упорядочен по UUID, а не по порядку запроса; состав участников от этого не меняется. Порядок чатов и сообщений, который важен контракту, соблюдается.
|
||||
- Нагрузочный отчёт ограничен closed-loop клиентом и не является надёжным измерением ёмкости.
|
||||
- Трек 2: синтетические перекошенные данные, один узел PostgreSQL, тёплый кэш и отсутствие конкурентной записи во время индексных замеров; dump/restore измерен по одному разу на размер. Оценка потерь использует 878 RPS из closed-loop замера, а не максимум пропускной способности.
|
||||
- В отчёте не раскрываются токены, пароли и содержимое `.env`; в `evidence/` сохранены только агрегаты и коды ответов.
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
# Track 2: query plan and recovery
|
||||
|
||||
Hypothesis: at 300k messages the history index is worth keeping even under a
|
||||
tight memory budget, while the membership index is the cheaper one to remove.
|
||||
Recovery time should grow linearly with message count, so the dump-loss window
|
||||
can be estimated from a few points.
|
||||
|
||||
## Dataset
|
||||
|
||||
One PostgreSQL 16.10 database, `messenger_research`:
|
||||
|
||||
- 5,000 chats, 2,000 users, 72,300 memberships.
|
||||
- 100k / 200k / 300k messages, timestamps spread over 90 days.
|
||||
- Chat sizes are deliberately skewed (`random()^3`): the largest chat holds
|
||||
about 6% of all messages, most chats stay small.
|
||||
- Probe user 1 belongs to the ten largest chats and every 100th chat, so the
|
||||
chat-list and cross-chat feed queries always return rows.
|
||||
- The app database `messenger` is not touched.
|
||||
|
||||
Schema is applied from `app/internal/database/migrations/0001_init.up.sql`.
|
||||
The two compared indexes are:
|
||||
|
||||
- `chat_members_user_idx (user_id, chat_id)`
|
||||
- `messages_chat_created_id_idx (chat_id, created_at, id)`
|
||||
|
||||
## Index experiment
|
||||
|
||||
`bench_indexes.sh` rebuilds the dataset at 300k, warms each query, then reports
|
||||
the median of five `EXPLAIN (ANALYZE, BUFFERS)` runs. The comparison states have
|
||||
exactly one of the two indexes; the script asserts that state and restores both
|
||||
indexes at the end.
|
||||
|
||||
| state | user_chats ms | chat_messages ms | user_feed ms |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| both indexes | 1.38 | 0.43 | 35.40 |
|
||||
| only `messages_chat_created_id_idx` | 7.36 | 0.47 | 35.22 |
|
||||
| only `chat_members_user_idx` | 1.40 | 57.63 | 80.40 |
|
||||
|
||||
Query shapes:
|
||||
|
||||
- `user_chats`: `chats JOIN chat_members WHERE user_id = $1 ORDER BY created_at, id LIMIT 50`
|
||||
- `chat_messages`: keyset page of one chat on `(created_at, id)`
|
||||
- `user_feed`: `messages JOIN chat_members WHERE user_id = $1` plus a keyset
|
||||
cursor, ordered globally by `(created_at, id)`
|
||||
|
||||
Index sizes after `REINDEX` (300k messages, both present):
|
||||
|
||||
- `chat_members_user_idx`: about 3.5 MB
|
||||
- `messages_chat_created_id_idx`: about 17 MB
|
||||
- `messages` table including indexes: about 75 MB
|
||||
|
||||
Write cost (pgbench, 4 clients, 10 s, message-like inserts on a clone table):
|
||||
2,719 tps with the messages index, 2,640 tps without it. The difference is
|
||||
small and within run noise; the read benefit below is much larger.
|
||||
|
||||
## Recovery experiment
|
||||
|
||||
`bench_recovery.sh` recreates the database at each size from the migration,
|
||||
seeds it, checks that both indexes exist, then runs `pg_dump -Fc` and
|
||||
`pg_restore` into a fresh database. It verifies message and membership counts,
|
||||
an MD5 fingerprint of every message row, and both indexes in the restored copy.
|
||||
|
||||
| messages | dump s | restore s | dump MiB | DB MiB | lost msgs at 878 RPS |
|
||||
| ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 100,000 | 1.02 | 1.44 | 4.9 | 45.2 | 896 |
|
||||
| 200,000 | 1.92 | 2.36 | 9.3 | 70.8 | 1,686 |
|
||||
| 300,000 | 2.66 | 3.28 | 13.8 | 95.0 | 2,335 |
|
||||
|
||||
Linear fits (`N` = messages in millions):
|
||||
|
||||
- dump: `t = 0.23 + 8.20 * N` s, R² = 1.00
|
||||
- restore: `t = 0.52 + 9.20 * N` s, R² = 1.00
|
||||
- dump + restore: `t = 0.75 + 17.4 * N` s, R² = 1.00
|
||||
|
||||
The loss column uses the closed-loop 878 RPS already measured in
|
||||
`REPORT.md`. `pg_dump` takes a consistent snapshot and does not block writers;
|
||||
the estimate means "writes committed after that snapshot are lost if the dump
|
||||
is the recovery point". Total outage including an offline restore is
|
||||
`dump + restore` times the write rate (about 2.2k, 3.7k and 5.2k messages at
|
||||
the three sizes).
|
||||
|
||||
## Decision
|
||||
|
||||
Keep `messages_chat_created_id_idx`. Without it a chat page goes from 0.43 ms
|
||||
to 57.6 ms and the cross-chat feed from 35.4 ms to 80.4 ms. If exactly one of
|
||||
the two indexes must be removed, remove `chat_members_user_idx`: it costs about
|
||||
3.5 MB and saves about 6 ms on the chat list, while the history index costs
|
||||
about 17 MB and prevents a two-order-of-magnitude regression on history reads.
|
||||
If the budget allows 21 MB for both, keep both.
|
||||
|
||||
## Reproduce
|
||||
|
||||
Requires Docker Compose and the repository `.env` (`cp .env.example .env`,
|
||||
then change `POSTGRES_PASSWORD`). Only the `postgres` service is needed.
|
||||
|
||||
```sh
|
||||
docker compose --env-file .env up -d --wait postgres
|
||||
./research/bench_indexes.sh
|
||||
./research/bench_recovery.sh
|
||||
python3 research/analyze.py | tee research/results/recovery.md
|
||||
```
|
||||
|
||||
Outputs: `research/results/indexes.md`, `research/results/recovery.csv`,
|
||||
`research/results/recovery.md`.
|
||||
|
||||
## Limits
|
||||
|
||||
- Synthetic skewed data, not a production trace.
|
||||
- Single PostgreSQL node, warm cache, no concurrent writes during the query
|
||||
benchmark.
|
||||
- One dump/restore run per size; timings are indicative, not a capacity plan.
|
||||
- The 878 RPS figure comes from the closed-loop client in `REPORT.md`; it is
|
||||
not a maximum throughput claim.
|
||||
Executable
+46
@@ -0,0 +1,46 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fit recovery time against message count and estimate the dump-loss window."""
|
||||
import csv
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
RPS = float(os.environ.get("RPS", "878")) # closed-loop POST RPS from REPORT.md
|
||||
rows = list(csv.DictReader((ROOT / "results/recovery.csv").open()))
|
||||
|
||||
|
||||
def fit(ys):
|
||||
# AGENT: three points are enough for the linear trend this lab asks for;
|
||||
# R^2 is printed so a bad fit is visible instead of silently accepted.
|
||||
xs = [int(row["messages"]) for row in rows]
|
||||
n = len(xs)
|
||||
mx = sum(xs) / n
|
||||
my = sum(ys) / n
|
||||
slope = sum((x - mx) * (y - my) for x, y in zip(xs, ys)) / sum((x - mx) ** 2 for x in xs)
|
||||
intercept = my - slope * mx
|
||||
total = sum((y - my) ** 2 for y in ys)
|
||||
residual = sum((y - (intercept + slope * x)) ** 2 for x, y in zip(xs, ys))
|
||||
r2 = 1 - residual / total if total else 1.0
|
||||
return intercept, slope, r2
|
||||
|
||||
|
||||
dump = fit([float(row["dump_s"]) for row in rows])
|
||||
restore = fit([float(row["restore_s"]) for row in rows])
|
||||
total = fit([float(row["dump_s"]) + float(row["restore_s"]) for row in rows])
|
||||
|
||||
print("| messages | dump s | restore s | dump MiB | DB MiB | lost msgs at %.0f RPS |" % RPS)
|
||||
print("| ---: | ---: | ---: | ---: | ---: | ---: |")
|
||||
for row in rows:
|
||||
dump_s = float(row["dump_s"])
|
||||
print("| {messages} | {dump_s} | {restore_s} | {dump_mib:.1f} | {db_mib:.1f} | {loss:.0f} |".format(
|
||||
messages=row["messages"],
|
||||
dump_s=row["dump_s"],
|
||||
restore_s=row["restore_s"],
|
||||
dump_mib=int(row["dump_bytes"]) / 2**20,
|
||||
db_mib=int(row["db_bytes"]) / 2**20,
|
||||
loss=dump_s * RPS,
|
||||
))
|
||||
|
||||
print()
|
||||
for name, (a, b, r2) in (("dump", dump), ("restore", restore), ("dump+restore", total)):
|
||||
print(f"{name}: t = {a:.3f} + {b * 1_000_000:.3f} * (messages / 1M) s, R^2={r2:.3f}")
|
||||
Executable
+191
@@ -0,0 +1,191 @@
|
||||
#!/usr/bin/env bash
|
||||
# Track 2: compare the two supporting indexes on one skewed dataset.
|
||||
# Exactly one of chat_members_user_idx / messages_chat_created_id_idx is
|
||||
# present in the two comparison states; both are restored at the end.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
set -a
|
||||
. ./.env
|
||||
set +a
|
||||
|
||||
DB=messenger_research
|
||||
SIZE=${SIZE:-300000}
|
||||
CHAT_COUNT=${CHAT_COUNT:-5000}
|
||||
USER_COUNT=${USER_COUNT:-2000}
|
||||
RESULTS=research/results/indexes.md
|
||||
COMPOSE=(docker compose --env-file .env)
|
||||
ZERO=00000000-0000-0000-0000-000000000000
|
||||
mkdir -p research/results
|
||||
|
||||
psqlq() {
|
||||
"${COMPOSE[@]}" exec -T postgres psql -X -q -A -t \
|
||||
-U "$POSTGRES_USER" -d "$DB" -c "$1"
|
||||
}
|
||||
|
||||
psqlf() {
|
||||
"${COMPOSE[@]}" exec -T postgres psql -X -q \
|
||||
-U "$POSTGRES_USER" -d "$DB" -v ON_ERROR_STOP=1 "$@"
|
||||
}
|
||||
|
||||
echo "Rebuilding $DB at $SIZE messages..."
|
||||
"${COMPOSE[@]}" exec -T postgres psql -X -q -U "$POSTGRES_USER" -d postgres \
|
||||
-v ON_ERROR_STOP=1 \
|
||||
-c "DROP DATABASE IF EXISTS $DB WITH (FORCE);" \
|
||||
-c "CREATE DATABASE $DB;" >/dev/null
|
||||
psqlf -f - < app/internal/database/migrations/0001_init.up.sql >/dev/null
|
||||
psqlq "CREATE TABLE IF NOT EXISTS schema_migrations (version bigint NOT NULL PRIMARY KEY, dirty boolean NOT NULL);
|
||||
INSERT INTO schema_migrations VALUES (1, false) ON CONFLICT DO NOTHING;" >/dev/null
|
||||
psqlf -v target_messages="$SIZE" -v chat_count="$CHAT_COUNT" \
|
||||
-v user_count="$USER_COUNT" -f - < research/seed.sql >/dev/null
|
||||
|
||||
# All three queries are built from the API's access patterns:
|
||||
# user_chats - list the chats of one user
|
||||
# chat_messages - keyset page of one chat
|
||||
# user_feed - messages from all of one user's chats, keyset page
|
||||
# The feed cursor is recent so this is a realistic page, not a full-history
|
||||
# sort; the query still joins every message to the user's memberships.
|
||||
Q_USER_CHATS="SELECT c.id, c.title, c.created_at
|
||||
FROM chats c
|
||||
JOIN chat_members cm ON cm.chat_id = c.id
|
||||
WHERE cm.user_id = research_user_uuid(1)
|
||||
ORDER BY c.created_at, c.id
|
||||
LIMIT 50;"
|
||||
|
||||
Q_CHAT_MESSAGES="SELECT id, chat_id, sender_id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = research_chat_uuid(1)
|
||||
AND (created_at, id) > ('2026-02-15T00:00:00Z'::timestamptz, '$ZERO'::uuid)
|
||||
ORDER BY created_at, id
|
||||
LIMIT 51;"
|
||||
|
||||
Q_USER_FEED="SELECT m.id, m.chat_id, m.sender_id, m.text, m.created_at
|
||||
FROM messages m
|
||||
JOIN chat_members cm ON cm.chat_id = m.chat_id
|
||||
WHERE cm.user_id = research_user_uuid(1)
|
||||
AND (m.created_at, m.id) > ('2026-03-01T00:00:00Z'::timestamptz, '$ZERO'::uuid)
|
||||
ORDER BY m.created_at, m.id
|
||||
LIMIT 50;"
|
||||
|
||||
ensure_both() {
|
||||
psqlq "CREATE INDEX IF NOT EXISTS chat_members_user_idx ON chat_members (user_id, chat_id);
|
||||
CREATE INDEX IF NOT EXISTS messages_chat_created_id_idx ON messages (chat_id, created_at, id);" >/dev/null
|
||||
}
|
||||
|
||||
messages_only() {
|
||||
psqlq "CREATE INDEX IF NOT EXISTS messages_chat_created_id_idx ON messages (chat_id, created_at, id);
|
||||
DROP INDEX IF EXISTS chat_members_user_idx;" >/dev/null
|
||||
}
|
||||
|
||||
membership_only() {
|
||||
psqlq "CREATE INDEX IF NOT EXISTS chat_members_user_idx ON chat_members (user_id, chat_id);
|
||||
DROP INDEX IF EXISTS messages_chat_created_id_idx;" >/dev/null
|
||||
}
|
||||
|
||||
# AGENT: the comparison is only meaningful if exactly one index is present;
|
||||
# a leftover second index would hide the regression this experiment measures.
|
||||
assert_single() {
|
||||
local want_membership=$1 want_messages=$2
|
||||
local count has_membership has_messages
|
||||
count=$(psqlq "SELECT count(*) FROM pg_indexes
|
||||
WHERE indexname IN ('chat_members_user_idx', 'messages_chat_created_id_idx');")
|
||||
has_membership=$(psqlq "SELECT to_regclass('chat_members_user_idx') IS NOT NULL;")
|
||||
has_messages=$(psqlq "SELECT to_regclass('messages_chat_created_id_idx') IS NOT NULL;")
|
||||
if [ "$count" != "1" ] || [ "$has_membership" != "$want_membership" ] || [ "$has_messages" != "$want_messages" ]; then
|
||||
echo "index state is not controlled: count=$count membership=$has_membership messages=$has_messages" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
assert_both() {
|
||||
local count
|
||||
count=$(psqlq "SELECT count(*) FROM pg_indexes
|
||||
WHERE indexname IN ('chat_members_user_idx', 'messages_chat_created_id_idx');")
|
||||
[ "$count" = "2" ] || { echo "both indexes were not restored" >&2; exit 1; }
|
||||
}
|
||||
|
||||
bench_query() {
|
||||
local label=$1 sql=$2
|
||||
psqlq "EXPLAIN (ANALYZE, BUFFERS) $sql" >/dev/null
|
||||
local times=()
|
||||
local i t
|
||||
for i in 1 2 3 4 5; do
|
||||
t=$(psqlq "EXPLAIN (ANALYZE, BUFFERS) $sql" |
|
||||
sed -n 's/.*Execution Time: \([0-9.]*\) ms/\1/p')
|
||||
times+=("$t")
|
||||
done
|
||||
local median plan
|
||||
median=$(printf '%s\n' "${times[@]}" | sort -n |
|
||||
awk '{a[NR]=$1} END {print a[int((NR + 1) / 2)]}')
|
||||
plan=$(psqlq "EXPLAIN (ANALYZE, BUFFERS) $sql" |
|
||||
grep -oE '(Index Only Scan|Index Scan|Seq Scan|Bitmap Heap Scan|Sort|Hash Join|Nested Loop|Merge Join)' |
|
||||
awk '{if ($0 != prev) print; prev = $0}' |
|
||||
paste -sd '>' -)
|
||||
printf '| %s | %s | %s |\n' "$label" "$median" "$plan"
|
||||
}
|
||||
|
||||
bench_write() {
|
||||
psqlq "DROP TABLE IF EXISTS research_write_bench;
|
||||
CREATE TABLE research_write_bench (
|
||||
id uuid PRIMARY KEY,
|
||||
chat_id uuid NOT NULL,
|
||||
sender_id uuid NOT NULL,
|
||||
text text NOT NULL,
|
||||
created_at timestamptz NOT NULL
|
||||
);" >/dev/null
|
||||
if [ "$1" = "yes" ]; then
|
||||
psqlq "CREATE INDEX research_write_bench_idx
|
||||
ON research_write_bench (chat_id, created_at, id);" >/dev/null
|
||||
fi
|
||||
"${COMPOSE[@]}" cp research/write.sql postgres:/tmp/research_write.sql >/dev/null
|
||||
local tps
|
||||
tps=$("${COMPOSE[@]}" exec -T postgres pgbench -n -c 4 -j 2 -T 10 \
|
||||
-U "$POSTGRES_USER" -d "$DB" -f /tmp/research_write.sql 2>&1 |
|
||||
sed -n 's/^tps = \([0-9.]*\).*/\1/p')
|
||||
psqlq "DROP TABLE research_write_bench;" >/dev/null
|
||||
echo "$tps"
|
||||
}
|
||||
|
||||
{
|
||||
echo "# Index experiment"
|
||||
echo
|
||||
echo "Dataset: $SIZE messages, $CHAT_COUNT chats, $USER_COUNT users."
|
||||
echo "Median of 5 warm-cache runs; plan is the sequence of access nodes."
|
||||
echo
|
||||
echo "| state | query | median ms | plan |"
|
||||
echo "| --- | --- | ---: | --- |"
|
||||
|
||||
ensure_both
|
||||
assert_both
|
||||
bench_query "both | user_chats" "$Q_USER_CHATS"
|
||||
bench_query "both | chat_messages" "$Q_CHAT_MESSAGES"
|
||||
bench_query "both | user_feed" "$Q_USER_FEED"
|
||||
write_with=$(bench_write yes)
|
||||
|
||||
messages_only
|
||||
assert_single f t
|
||||
bench_query "messages_only | user_chats" "$Q_USER_CHATS"
|
||||
bench_query "messages_only | chat_messages" "$Q_CHAT_MESSAGES"
|
||||
bench_query "messages_only | user_feed" "$Q_USER_FEED"
|
||||
|
||||
membership_only
|
||||
assert_single t f
|
||||
bench_query "membership_only | user_chats" "$Q_USER_CHATS"
|
||||
bench_query "membership_only | chat_messages" "$Q_CHAT_MESSAGES"
|
||||
bench_query "membership_only | user_feed" "$Q_USER_FEED"
|
||||
write_without=$(bench_write no)
|
||||
|
||||
ensure_both
|
||||
assert_both
|
||||
|
||||
echo
|
||||
echo "Index sizes after restoring both:"
|
||||
psqlq "SELECT 'chat_members_user_idx=' || pg_size_pretty(pg_relation_size('chat_members_user_idx')) ||
|
||||
', messages_chat_created_id_idx=' || pg_size_pretty(pg_relation_size('messages_chat_created_id_idx')) ||
|
||||
', messages_table=' || pg_size_pretty(pg_total_relation_size('messages'));"
|
||||
echo
|
||||
echo "pgbench message-like inserts, 4 clients, 10 s: with messages index ${write_with} tps; without ${write_without} tps."
|
||||
echo "The research DB is left with both indexes."
|
||||
} | tee "$RESULTS"
|
||||
|
||||
echo "Wrote $RESULTS"
|
||||
Executable
+112
@@ -0,0 +1,112 @@
|
||||
#!/usr/bin/env bash
|
||||
# Track 2: dump/restore timing at 100k, 200k and 300k messages.
|
||||
# The schema comes from the repository migration; both indexes are present
|
||||
# for every dump. The research DB is left at 300k with both indexes.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
set -a
|
||||
. ./.env
|
||||
set +a
|
||||
|
||||
DB=messenger_research
|
||||
RESTORE_DB=messenger_research_restore
|
||||
DUMP=/tmp/research_recovery.dump
|
||||
RESULTS=research/results/recovery.csv
|
||||
COMPOSE=(docker compose --env-file .env)
|
||||
mkdir -p research/results
|
||||
|
||||
psql_db() {
|
||||
local db=$1 sql=$2
|
||||
"${COMPOSE[@]}" exec -T postgres psql -X -q -A -t \
|
||||
-U "$POSTGRES_USER" -d "$db" -c "$sql"
|
||||
}
|
||||
|
||||
psql_admin() {
|
||||
"${COMPOSE[@]}" exec -T postgres psql -X -q -A -t \
|
||||
-U "$POSTGRES_USER" -d postgres -v ON_ERROR_STOP=1 -c "$1"
|
||||
}
|
||||
|
||||
measure() {
|
||||
local command=$1
|
||||
# AGENT: /usr/bin/time runs inside the container so the client-side
|
||||
# docker exec overhead is not counted as backup or restore time.
|
||||
if ! "${COMPOSE[@]}" exec -T postgres sh -c \
|
||||
"/usr/bin/time -f '%e' -o /tmp/research_time $command >/dev/null 2>&1"; then
|
||||
echo "timed command failed: $command" >&2
|
||||
exit 1
|
||||
fi
|
||||
"${COMPOSE[@]}" exec -T postgres cat /tmp/research_time
|
||||
}
|
||||
|
||||
ensure_both_indexes() {
|
||||
psql_db "$DB" "CREATE INDEX IF NOT EXISTS chat_members_user_idx
|
||||
ON chat_members (user_id, chat_id);
|
||||
CREATE INDEX IF NOT EXISTS messages_chat_created_id_idx
|
||||
ON messages (chat_id, created_at, id);" >/dev/null
|
||||
}
|
||||
|
||||
printf 'messages,dump_s,restore_s,dump_bytes,db_bytes,table_bytes,index_bytes\n' > "$RESULTS"
|
||||
|
||||
for n in 100000 200000 300000; do
|
||||
echo "Rebuilding $DB at $n messages..."
|
||||
psql_admin "DROP DATABASE IF EXISTS $RESTORE_DB WITH (FORCE);" >/dev/null
|
||||
psql_admin "DROP DATABASE IF EXISTS $DB WITH (FORCE);" >/dev/null
|
||||
psql_admin "CREATE DATABASE $DB;" >/dev/null
|
||||
"${COMPOSE[@]}" exec -T postgres psql -X -q -U "$POSTGRES_USER" -d "$DB" \
|
||||
-v ON_ERROR_STOP=1 -f - < app/internal/database/migrations/0001_init.up.sql >/dev/null
|
||||
psql_db "$DB" "CREATE TABLE IF NOT EXISTS schema_migrations
|
||||
(version bigint NOT NULL PRIMARY KEY, dirty boolean NOT NULL);
|
||||
INSERT INTO schema_migrations VALUES (1, false)
|
||||
ON CONFLICT DO NOTHING;" >/dev/null
|
||||
"${COMPOSE[@]}" exec -T postgres psql -X -q -U "$POSTGRES_USER" -d "$DB" \
|
||||
-v ON_ERROR_STOP=1 -v target_messages="$n" -v chat_count=5000 \
|
||||
-v user_count=2000 -f - < research/seed.sql >/dev/null
|
||||
ensure_both_indexes
|
||||
|
||||
db_bytes=$(psql_db "$DB" "SELECT pg_database_size(current_database());")
|
||||
table_bytes=$(psql_db "$DB" "SELECT pg_total_relation_size('messages');")
|
||||
index_bytes=$(psql_db "$DB" "SELECT pg_relation_size('chat_members_user_idx') +
|
||||
pg_relation_size('messages_chat_created_id_idx');")
|
||||
|
||||
dump_s=$(measure "pg_dump -U $POSTGRES_USER -d $DB -Fc -f $DUMP")
|
||||
dump_bytes=$("${COMPOSE[@]}" exec -T postgres stat -c %s "$DUMP")
|
||||
psql_admin "CREATE DATABASE $RESTORE_DB;" >/dev/null
|
||||
restore_s=$(measure "pg_restore -U $POSTGRES_USER -d $RESTORE_DB --no-owner --no-privileges $DUMP")
|
||||
|
||||
src_count=$(psql_db "$DB" "SELECT count(*) FROM messages;")
|
||||
dst_count=$(psql_db "$RESTORE_DB" "SELECT count(*) FROM messages;")
|
||||
src_members=$(psql_db "$DB" "SELECT count(*) FROM chat_members;")
|
||||
dst_members=$(psql_db "$RESTORE_DB" "SELECT count(*) FROM chat_members;")
|
||||
# AGENT: one aggregate fingerprint covers every message row, so the check
|
||||
# is not limited to whichever row happened to be newest.
|
||||
src_fingerprint=$(psql_db "$DB" "SELECT md5(string_agg(
|
||||
id::text || '|' || chat_id::text || '|' || sender_id::text || '|' ||
|
||||
text || '|' || created_at::text, E'\n' ORDER BY id)) FROM messages;")
|
||||
dst_fingerprint=$(psql_db "$RESTORE_DB" "SELECT md5(string_agg(
|
||||
id::text || '|' || chat_id::text || '|' || sender_id::text || '|' ||
|
||||
text || '|' || created_at::text, E'\n' ORDER BY id)) FROM messages;")
|
||||
dst_indexes=$(psql_db "$RESTORE_DB" "SELECT count(*) FROM pg_indexes
|
||||
WHERE indexname IN ('chat_members_user_idx',
|
||||
'messages_chat_created_id_idx');")
|
||||
|
||||
if [ "$src_count" != "$dst_count" ] || [ "$src_members" != "$dst_members" ] ||
|
||||
[ "$src_fingerprint" != "$dst_fingerprint" ] || [ "$dst_indexes" != "2" ]; then
|
||||
echo "restore verification failed at $n rows" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
printf '%s,%s,%s,%s,%s,%s,%s\n' \
|
||||
"$n" "$dump_s" "$restore_s" "$dump_bytes" "$db_bytes" "$table_bytes" "$index_bytes" \
|
||||
>> "$RESULTS"
|
||||
echo " dump ${dump_s}s, restore ${restore_s}s, ${dump_bytes} bytes; verified"
|
||||
psql_admin "DROP DATABASE $RESTORE_DB WITH (FORCE);" >/dev/null
|
||||
"${COMPOSE[@]}" exec -T postgres rm -f "$DUMP"
|
||||
done
|
||||
|
||||
ensure_both_indexes
|
||||
echo
|
||||
echo "Recovery results (CSV: messages,dump_s,restore_s,dump_bytes,db_bytes,table_bytes,index_bytes):"
|
||||
cat "$RESULTS"
|
||||
echo
|
||||
echo "Final research DB: $(psql_db "$DB" "SELECT count(*) FROM messages;") messages, both indexes present."
|
||||
@@ -0,0 +1,22 @@
|
||||
# Index experiment
|
||||
|
||||
Dataset: 300000 messages, 5000 chats, 2000 users.
|
||||
Median of 5 warm-cache runs; plan is the sequence of access nodes.
|
||||
|
||||
| state | query | median ms | plan |
|
||||
| --- | --- | ---: | --- |
|
||||
| both | user_chats | 1.379 | Sort>Hash Join>Seq Scan>Index Only Scan |
|
||||
| both | chat_messages | 0.428 | Index Scan |
|
||||
| both | user_feed | 35.396 | Sort>Nested Loop>Index Only Scan>Index Scan |
|
||||
| messages_only | user_chats | 7.360 | Sort>Hash Join>Seq Scan |
|
||||
| messages_only | chat_messages | 0.469 | Index Scan |
|
||||
| messages_only | user_feed | 35.218 | Sort>Nested Loop>Seq Scan>Index Scan |
|
||||
| membership_only | user_chats | 1.402 | Sort>Hash Join>Seq Scan>Index Only Scan |
|
||||
| membership_only | chat_messages | 57.634 | Sort>Seq Scan |
|
||||
| membership_only | user_feed | 80.404 | Sort>Hash Join>Seq Scan>Index Only Scan |
|
||||
|
||||
Index sizes after restoring both:
|
||||
chat_members_user_idx=3536 kB, messages_chat_created_id_idx=17 MB, messages_table=75 MB
|
||||
|
||||
pgbench message-like inserts, 4 clients, 10 s: with messages index 2718.889742 tps; without 2640.028063 tps.
|
||||
The research DB is left with both indexes.
|
||||
@@ -0,0 +1,4 @@
|
||||
messages,dump_s,restore_s,dump_bytes,db_bytes,table_bytes,index_bytes
|
||||
100000,1.02,1.44,5109980,47411683,26402816,9535488
|
||||
200000,1.92,2.36,9803565,74240483,53231616,15417344
|
||||
300000,2.66,3.28,14500027,99627491,78618624,21307392
|
||||
|
@@ -0,0 +1,9 @@
|
||||
| messages | dump s | restore s | dump MiB | DB MiB | lost msgs at 878 RPS |
|
||||
| ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 100000 | 1.02 | 1.44 | 4.9 | 45.2 | 896 |
|
||||
| 200000 | 1.92 | 2.36 | 9.3 | 70.8 | 1686 |
|
||||
| 300000 | 2.66 | 3.28 | 13.8 | 95.0 | 2335 |
|
||||
|
||||
dump: t = 0.227 + 8.200 * (messages / 1M) s, R^2=0.997
|
||||
restore: t = 0.520 + 9.200 * (messages / 1M) s, R^2=1.000
|
||||
dump+restore: t = 0.747 + 17.400 * (messages / 1M) s, R^2=0.999
|
||||
@@ -0,0 +1,114 @@
|
||||
-- Synthetic, repeatable dataset for the Track 2 experiments.
|
||||
-- Usage:
|
||||
-- psql -v target_messages=300000 -v chat_count=5000 -v user_count=2000 -f seed.sql
|
||||
-- The distribution is deliberately skewed: a few chats hold most messages,
|
||||
-- which is closer to a real messenger than one evenly filled chat.
|
||||
\set ON_ERROR_STOP on
|
||||
\if :{?target_messages}
|
||||
\else
|
||||
\set target_messages 300000
|
||||
\endif
|
||||
\if :{?chat_count}
|
||||
\else
|
||||
\set chat_count 5000
|
||||
\endif
|
||||
\if :{?user_count}
|
||||
\else
|
||||
\set user_count 2000
|
||||
\endif
|
||||
|
||||
-- Deterministic, readable UUIDs. The version/variant nibbles are valid.
|
||||
CREATE OR REPLACE FUNCTION research_user_uuid(i integer)
|
||||
RETURNS uuid LANGUAGE sql IMMUTABLE AS $$
|
||||
SELECT ('00000000-0000-4000-8000-' || lpad(to_hex(i), 12, '0'))::uuid
|
||||
$$;
|
||||
|
||||
CREATE OR REPLACE FUNCTION research_chat_uuid(i integer)
|
||||
RETURNS uuid LANGUAGE sql IMMUTABLE AS $$
|
||||
SELECT ('10000000-0000-4000-8000-' || lpad(to_hex(i), 12, '0'))::uuid
|
||||
$$;
|
||||
|
||||
DROP TABLE IF EXISTS research_chat_sender;
|
||||
TRUNCATE messages, chat_members, chats, users CASCADE;
|
||||
|
||||
-- Reproducible pseudo-random layout for a given PostgreSQL build.
|
||||
SELECT setseed(0.4242);
|
||||
|
||||
INSERT INTO users (id, username, display_name, password_hash, created_at)
|
||||
SELECT research_user_uuid(g),
|
||||
'u' || lpad(g::text, 4, '0'),
|
||||
'User ' || g,
|
||||
'not-a-login-hash',
|
||||
'2026-01-01T00:00:00Z'::timestamptz
|
||||
FROM generate_series(1, :user_count) AS g;
|
||||
|
||||
INSERT INTO chats (id, title, created_at)
|
||||
SELECT research_chat_uuid(g),
|
||||
'Research chat ' || g,
|
||||
'2026-01-01T00:00:00Z'::timestamptz + (g || ' seconds')::interval
|
||||
FROM generate_series(1, :chat_count) AS g;
|
||||
|
||||
-- Each chat gets a small random membership set (5..24 distinct users).
|
||||
INSERT INTO chat_members (chat_id, user_id)
|
||||
SELECT research_chat_uuid(c),
|
||||
research_user_uuid(1 + floor(random() * :user_count)::int)
|
||||
FROM generate_series(1, :chat_count) AS c
|
||||
CROSS JOIN generate_series(1, 5 + (c % 20)) AS m
|
||||
ON CONFLICT DO NOTHING;
|
||||
|
||||
-- Probe user 1 is a member of the ten largest chats and every 100th chat.
|
||||
-- user_chats and user_feed queries therefore always have rows to return.
|
||||
INSERT INTO chat_members (chat_id, user_id)
|
||||
SELECT research_chat_uuid(c), research_user_uuid(1)
|
||||
FROM generate_series(1, :chat_count) AS c
|
||||
WHERE c <= 10 OR c % 100 = 0
|
||||
ON CONFLICT DO NOTHING;
|
||||
|
||||
-- One valid sender per chat, so every synthetic message has a member sender.
|
||||
CREATE TABLE research_chat_sender AS
|
||||
SELECT chat_id, (array_agg(user_id ORDER BY random()))[1] AS sender_id
|
||||
FROM chat_members
|
||||
GROUP BY chat_id;
|
||||
ALTER TABLE research_chat_sender ADD PRIMARY KEY (chat_id);
|
||||
|
||||
-- AGENT: a uniform chat size would make every query look cheap, so sizes are
|
||||
-- skewed. The probe user is deliberately in the largest chats to exercise the
|
||||
-- worst realistic case for the cross-chat feed.
|
||||
-- floor(random()^3 * chat_count) makes chat 1 hold roughly 6% of all
|
||||
-- messages at 300k, while most chats stay small. Timestamps span 90 days.
|
||||
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||
SELECT gen_random_uuid(),
|
||||
s.chat_id,
|
||||
cs.sender_id,
|
||||
'research message ' || s.g || ' ' || repeat('x', 20 + (s.g % 60)),
|
||||
'2026-01-01T00:00:00Z'::timestamptz + interval '90 days' * s.r
|
||||
FROM (
|
||||
SELECT g,
|
||||
research_chat_uuid(floor(power(random(), 3) * :chat_count)::int + 1) AS chat_id,
|
||||
random() AS r
|
||||
FROM generate_series(1, :target_messages) AS g
|
||||
) AS s
|
||||
JOIN research_chat_sender cs USING (chat_id);
|
||||
|
||||
-- Rebuild the two indexes so their reported sizes are not inflated by the
|
||||
-- random insertion order. This keeps the index comparison and the dump size
|
||||
-- on the same, compact baseline.
|
||||
REINDEX INDEX chat_members_user_idx;
|
||||
REINDEX INDEX messages_chat_created_id_idx;
|
||||
VACUUM (ANALYZE) messages, chat_members, chats, users, research_chat_sender;
|
||||
|
||||
SELECT 'messages' AS item, count(*)::text AS value FROM messages
|
||||
UNION ALL SELECT 'chat_members', count(*)::text FROM chat_members
|
||||
UNION ALL SELECT 'chats', count(*)::text FROM chats
|
||||
UNION ALL SELECT 'users', count(*)::text FROM users
|
||||
UNION ALL SELECT 'messages_table_bytes', pg_total_relation_size('messages')::text
|
||||
UNION ALL SELECT 'chat_members_user_idx_bytes', pg_relation_size('chat_members_user_idx')::text
|
||||
UNION ALL SELECT 'messages_chat_created_id_idx_bytes', pg_relation_size('messages_chat_created_id_idx')::text;
|
||||
|
||||
-- Shape check: largest chats should dominate the message count.
|
||||
SELECT c.id, c.title, count(m.id) AS messages
|
||||
FROM chats c
|
||||
LEFT JOIN messages m ON m.chat_id = c.id
|
||||
GROUP BY c.id, c.title
|
||||
ORDER BY messages DESC
|
||||
LIMIT 5;
|
||||
@@ -0,0 +1,9 @@
|
||||
\set g random(1, 1000000)
|
||||
INSERT INTO research_write_bench (id, chat_id, sender_id, text, created_at)
|
||||
VALUES (
|
||||
gen_random_uuid(),
|
||||
research_chat_uuid(1 + (:g % 5000)),
|
||||
research_user_uuid(1 + (:g % 2000)),
|
||||
'bench ' || :g,
|
||||
'2026-01-01T00:00:00Z'::timestamptz + (:g || ' seconds')::interval
|
||||
);
|
||||
Reference in New Issue
Block a user