forked from mtusi2026/mtusi2026lab2
feat(research) grade 5 assignment done
This commit is contained in:
@@ -1,9 +1,9 @@
|
|||||||
# Отчёт — лабораторная 2
|
# Отчёт — лабораторная 2
|
||||||
|
|
||||||
- Автор, группа: Булла Максим, БПИ2301
|
- Автор, группа: ``TODO``
|
||||||
- Целевая оценка: **4**
|
- Целевая оценка: 5, трек 2
|
||||||
- Commit/PR: локальный тег `lab2-green`
|
- Commit/PR: локальный тег ``TODO``
|
||||||
- Использованные источники, библиотеки и помощь ИИ: Go 1.27.1, Fiber v3.5.0, pgx v5.11.0, go-redis v9.23.0, Argon2id из `golang.org/x/crypto`; PostgreSQL 16.10, Redis 7.4.7. DeepSeek V4.1 Flash + Opencode Harness для генерации решений; все заявленные результаты получены командами из таблицы ниже и сохранены в `evidence/`.
|
- Использованные источники, библиотеки и помощь ИИ: Go 1.27.1, Fiber v3.5.0, pgx v5.11.0, go-redis v9.23.0, Argon2id из `golang.org/x/crypto`; PostgreSQL 16.10, Redis 7.4.7, `pg_dump`/`pg_restore`/`pgbench`. DeepSeek V4.1 Flash + Opencode Harness для генерации решений; все заявленные результаты получены командами из таблицы ниже и сохранены в `evidence/` и `research/results/`.
|
||||||
|
|
||||||
## Запуск из чистого clone
|
## Запуск из чистого clone
|
||||||
|
|
||||||
@@ -105,6 +105,8 @@ docker compose --env-file .env down -v # осознанно удалить
|
|||||||
| Гонка регистрации username | `grade4_checks.py` | 12 одновременных запросов → один `201`, одиннадцать `409` | `evidence/07-grade4-checks.txt` |
|
| Гонка регистрации username | `grade4_checks.py` | 12 одновременных запросов → один `201`, одиннадцать `409` | `evidence/07-grade4-checks.txt` |
|
||||||
| Rollback составной операции | `grade4_checks.py` | после ошибки чат с тем же title отсутствует в списке | `evidence/07-grade4-checks.txt` |
|
| Rollback составной операции | `grade4_checks.py` | после ошибки чат с тем же title отсутствует в списке | `evidence/07-grade4-checks.txt` |
|
||||||
| Keyset-пагинация и индекс, доказательство на ≥10 000 сообщений | `./scripts/seed_and_explain.sh` | `Index Scan using messages_chat_created_id_idx`, 5–6 shared buffers, ~0,09–0,10 мс | `evidence/08-explain.txt` |
|
| Keyset-пагинация и индекс, доказательство на ≥10 000 сообщений | `./scripts/seed_and_explain.sh` | `Index Scan using messages_chat_created_id_idx`, 5–6 shared buffers, ~0,09–0,10 мс | `evidence/08-explain.txt` |
|
||||||
|
| Трек 2: выбор индекса при ограниченной памяти | `./research/bench_indexes.sh` | 300k сообщений; без индекса истории `chat_messages` 57,6 мс вместо 0,43 мс; без индекса членства `user_chats` 7,4 мс вместо 1,4 мс | `research/results/indexes.md` |
|
||||||
|
| Трек 2: backup → restore и проверка | `./research/bench_recovery.sh` + `python3 research/analyze.py` | 100k/200k/300k: dump 1,02/1,92/2,66 с, restore 1,44/2,36/3,28 с; counts и MD5-отпечаток сообщений совпали | `research/results/recovery.csv`, `research/results/recovery.md` |
|
||||||
| Автопроверки expiry/logout/wrong password/Redis/PostgreSQL outage | `grade4_checks.py` | все пункты `PASS` | `evidence/07-grade4-checks.txt` |
|
| Автопроверки expiry/logout/wrong password/Redis/PostgreSQL outage | `grade4_checks.py` | все пункты `PASS` | `evidence/07-grade4-checks.txt` |
|
||||||
| Redis недоступен: live 200, ready 503, protected 503 | `grade4_checks.py` (stop/start Redis) | live=200, ready=503, protected=503; после старта тот же токен и сообщение целы | `evidence/07-grade4-checks.txt` |
|
| Redis недоступен: live 200, ready 503, protected 503 | `grade4_checks.py` (stop/start Redis) | live=200, ready=503, protected=503; после старта тот же токен и сообщение целы | `evidence/07-grade4-checks.txt` |
|
||||||
| PostgreSQL недоступен: live 200, ready 503, protected 503 | `grade4_checks.py` (stop/start PostgreSQL) | live=200, ready=503, protected=503; сессия и данные пережили отказ | `evidence/07-grade4-checks.txt` |
|
| PostgreSQL недоступен: live 200, ready 503, protected 503 | `grade4_checks.py` (stop/start PostgreSQL) | live=200, ready=503, protected=503; сессия и данные пережили отказ | `evidence/07-grade4-checks.txt` |
|
||||||
@@ -137,6 +139,30 @@ docker compose --env-file .env down -v # осознанно удалить
|
|||||||
|
|
||||||
Сырые метрики — `evidence/load.json`, команда и вывод — `evidence/09-load.txt`.
|
Сырые метрики — `evidence/load.json`, команда и вывод — `evidence/09-load.txt`.
|
||||||
|
|
||||||
|
### Трек 2: индекс и восстановление
|
||||||
|
|
||||||
|
Гипотеза: при 300k сообщений индекс истории полезнее индекса членства, поэтому при жёстком бюджете памяти первым стоит убрать `chat_members_user_idx`. Время восстановления должно расти линейно с числом сообщений.
|
||||||
|
|
||||||
|
Стенд: PostgreSQL 16.10, 5 000 чатов, 2 000 пользователей, 72 300 членств и 300k сообщений с перекошенными размерами чатов (крупнейший чат — около 6% истории). Схема создана миграцией `0001_init.up.sql`. В двух сравниваемых состояниях ровно один из двух индексов; состояние проверяется скриптом, в конце оба индекса возвращаются. Запросы: список чатов пользователя, keyset-страница сообщений одного чата и keyset-лента сообщений по всем чатам пользователя. В таблице — медиана пяти `EXPLAIN (ANALYZE, BUFFERS)`.
|
||||||
|
|
||||||
|
| Состояние | `user_chats`, мс | `chat_messages`, мс | `user_feed`, мс |
|
||||||
|
| --- | ---: | ---: | ---: |
|
||||||
|
| оба индекса | 1,38 | 0,43 | 35,40 |
|
||||||
|
| только `messages_chat_created_id_idx` | 7,36 | 0,47 | 35,22 |
|
||||||
|
| только `chat_members_user_idx` | 1,40 | 57,63 | 80,40 |
|
||||||
|
|
||||||
|
Размеры: `chat_members_user_idx` — около 3,5 МБ, `messages_chat_created_id_idx` — около 17 МБ. `pgbench` на клонированной таблице (4 клиента, 10 с) дал 2 719 tps с индексом истории и 2 640 tps без него; разница в пределах шума. Без индекса истории страница чата замедляется примерно в 135 раз, а лента — в 2,3 раза; без индекса членства список чатов замедляется примерно в 5 раз. **Вывод: сохранять `messages_chat_created_id_idx`; при необходимости удалять `chat_members_user_idx`.**
|
||||||
|
|
||||||
|
Backup → restore выполнялся с обоими индексами на 100k, 200k и 300k сообщениях; после восстановления сверялись количество сообщений/членств, MD5-отпечаток всех сообщений и наличие обоих индексов.
|
||||||
|
|
||||||
|
| Сообщений | dump, с | restore, с | dump, МиБ | БД, МиБ | потеря при 878 RPS |
|
||||||
|
| ---: | ---: | ---: | ---: | ---: | ---: |
|
||||||
|
| 100 000 | 1,02 | 1,44 | 4,9 | 45,2 | ~896 |
|
||||||
|
| 200 000 | 1,92 | 2,36 | 9,3 | 70,8 | ~1 686 |
|
||||||
|
| 300 000 | 2,66 | 3,28 | 13,8 | 95,0 | ~2 335 |
|
||||||
|
|
||||||
|
Линейные зависимости (`N` — миллионы сообщений): dump `t = 0,23 + 8,20·N` с, restore `t = 0,52 + 9,20·N` с, dump+restore `t = 0,75 + 17,4·N` с (R² ≥ 0,997). Потеря в таблице — это `dump_time × 878 RPS`; `pg_dump` не блокирует записи, поэтому оценка означает «записи после снимка», а не остановку API. При полном офлайн-восстановлении тот же расчёт по `dump + restore` даёт около 2,2k / 3,7k / 5,2k сообщений. Подробная методика и команды — `research/README.md`.
|
||||||
|
|
||||||
### Отказы зависимостей
|
### Отказы зависимостей
|
||||||
|
|
||||||
| Воздействие | Что ожидали | Что наблюдали | Восстановление / потери |
|
| Воздействие | Что ожидали | Что наблюдали | Восстановление / потери |
|
||||||
@@ -152,4 +178,5 @@ docker compose --env-file .env down -v # осознанно удалить
|
|||||||
- AOF с `appendfsync everysec`: при аварийном убийстве процесса возможна потеря до ~1 секунды последних операций с сессиями. Graceful-перезапуск, который использует `scripts/restart.sh`, проходит без потерь.
|
- AOF с `appendfsync everysec`: при аварийном убийстве процесса возможна потеря до ~1 секунды последних операций с сессиями. Graceful-перезапуск, который использует `scripts/restart.sh`, проходит без потерь.
|
||||||
- Список `member_ids` в ответе чата упорядочен по UUID, а не по порядку запроса; состав участников от этого не меняется. Порядок чатов и сообщений, который важен контракту, соблюдается.
|
- Список `member_ids` в ответе чата упорядочен по UUID, а не по порядку запроса; состав участников от этого не меняется. Порядок чатов и сообщений, который важен контракту, соблюдается.
|
||||||
- Нагрузочный отчёт ограничен closed-loop клиентом и не является надёжным измерением ёмкости.
|
- Нагрузочный отчёт ограничен closed-loop клиентом и не является надёжным измерением ёмкости.
|
||||||
|
- Трек 2: синтетические перекошенные данные, один узел PostgreSQL, тёплый кэш и отсутствие конкурентной записи во время индексных замеров; dump/restore измерен по одному разу на размер. Оценка потерь использует 878 RPS из closed-loop замера, а не максимум пропускной способности.
|
||||||
- В отчёте не раскрываются токены, пароли и содержимое `.env`; в `evidence/` сохранены только агрегаты и коды ответов.
|
- В отчёте не раскрываются токены, пароли и содержимое `.env`; в `evidence/` сохранены только агрегаты и коды ответов.
|
||||||
|
|||||||
@@ -0,0 +1,113 @@
|
|||||||
|
# Track 2: query plan and recovery
|
||||||
|
|
||||||
|
Hypothesis: at 300k messages the history index is worth keeping even under a
|
||||||
|
tight memory budget, while the membership index is the cheaper one to remove.
|
||||||
|
Recovery time should grow linearly with message count, so the dump-loss window
|
||||||
|
can be estimated from a few points.
|
||||||
|
|
||||||
|
## Dataset
|
||||||
|
|
||||||
|
One PostgreSQL 16.10 database, `messenger_research`:
|
||||||
|
|
||||||
|
- 5,000 chats, 2,000 users, 72,300 memberships.
|
||||||
|
- 100k / 200k / 300k messages, timestamps spread over 90 days.
|
||||||
|
- Chat sizes are deliberately skewed (`random()^3`): the largest chat holds
|
||||||
|
about 6% of all messages, most chats stay small.
|
||||||
|
- Probe user 1 belongs to the ten largest chats and every 100th chat, so the
|
||||||
|
chat-list and cross-chat feed queries always return rows.
|
||||||
|
- The app database `messenger` is not touched.
|
||||||
|
|
||||||
|
Schema is applied from `app/internal/database/migrations/0001_init.up.sql`.
|
||||||
|
The two compared indexes are:
|
||||||
|
|
||||||
|
- `chat_members_user_idx (user_id, chat_id)`
|
||||||
|
- `messages_chat_created_id_idx (chat_id, created_at, id)`
|
||||||
|
|
||||||
|
## Index experiment
|
||||||
|
|
||||||
|
`bench_indexes.sh` rebuilds the dataset at 300k, warms each query, then reports
|
||||||
|
the median of five `EXPLAIN (ANALYZE, BUFFERS)` runs. The comparison states have
|
||||||
|
exactly one of the two indexes; the script asserts that state and restores both
|
||||||
|
indexes at the end.
|
||||||
|
|
||||||
|
| state | user_chats ms | chat_messages ms | user_feed ms |
|
||||||
|
| --- | ---: | ---: | ---: |
|
||||||
|
| both indexes | 1.38 | 0.43 | 35.40 |
|
||||||
|
| only `messages_chat_created_id_idx` | 7.36 | 0.47 | 35.22 |
|
||||||
|
| only `chat_members_user_idx` | 1.40 | 57.63 | 80.40 |
|
||||||
|
|
||||||
|
Query shapes:
|
||||||
|
|
||||||
|
- `user_chats`: `chats JOIN chat_members WHERE user_id = $1 ORDER BY created_at, id LIMIT 50`
|
||||||
|
- `chat_messages`: keyset page of one chat on `(created_at, id)`
|
||||||
|
- `user_feed`: `messages JOIN chat_members WHERE user_id = $1` plus a keyset
|
||||||
|
cursor, ordered globally by `(created_at, id)`
|
||||||
|
|
||||||
|
Index sizes after `REINDEX` (300k messages, both present):
|
||||||
|
|
||||||
|
- `chat_members_user_idx`: about 3.5 MB
|
||||||
|
- `messages_chat_created_id_idx`: about 17 MB
|
||||||
|
- `messages` table including indexes: about 75 MB
|
||||||
|
|
||||||
|
Write cost (pgbench, 4 clients, 10 s, message-like inserts on a clone table):
|
||||||
|
2,719 tps with the messages index, 2,640 tps without it. The difference is
|
||||||
|
small and within run noise; the read benefit below is much larger.
|
||||||
|
|
||||||
|
## Recovery experiment
|
||||||
|
|
||||||
|
`bench_recovery.sh` recreates the database at each size from the migration,
|
||||||
|
seeds it, checks that both indexes exist, then runs `pg_dump -Fc` and
|
||||||
|
`pg_restore` into a fresh database. It verifies message and membership counts,
|
||||||
|
an MD5 fingerprint of every message row, and both indexes in the restored copy.
|
||||||
|
|
||||||
|
| messages | dump s | restore s | dump MiB | DB MiB | lost msgs at 878 RPS |
|
||||||
|
| ---: | ---: | ---: | ---: | ---: | ---: |
|
||||||
|
| 100,000 | 1.02 | 1.44 | 4.9 | 45.2 | 896 |
|
||||||
|
| 200,000 | 1.92 | 2.36 | 9.3 | 70.8 | 1,686 |
|
||||||
|
| 300,000 | 2.66 | 3.28 | 13.8 | 95.0 | 2,335 |
|
||||||
|
|
||||||
|
Linear fits (`N` = messages in millions):
|
||||||
|
|
||||||
|
- dump: `t = 0.23 + 8.20 * N` s, R² = 1.00
|
||||||
|
- restore: `t = 0.52 + 9.20 * N` s, R² = 1.00
|
||||||
|
- dump + restore: `t = 0.75 + 17.4 * N` s, R² = 1.00
|
||||||
|
|
||||||
|
The loss column uses the closed-loop 878 RPS already measured in
|
||||||
|
`REPORT.md`. `pg_dump` takes a consistent snapshot and does not block writers;
|
||||||
|
the estimate means "writes committed after that snapshot are lost if the dump
|
||||||
|
is the recovery point". Total outage including an offline restore is
|
||||||
|
`dump + restore` times the write rate (about 2.2k, 3.7k and 5.2k messages at
|
||||||
|
the three sizes).
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Keep `messages_chat_created_id_idx`. Without it a chat page goes from 0.43 ms
|
||||||
|
to 57.6 ms and the cross-chat feed from 35.4 ms to 80.4 ms. If exactly one of
|
||||||
|
the two indexes must be removed, remove `chat_members_user_idx`: it costs about
|
||||||
|
3.5 MB and saves about 6 ms on the chat list, while the history index costs
|
||||||
|
about 17 MB and prevents a two-order-of-magnitude regression on history reads.
|
||||||
|
If the budget allows 21 MB for both, keep both.
|
||||||
|
|
||||||
|
## Reproduce
|
||||||
|
|
||||||
|
Requires Docker Compose and the repository `.env` (`cp .env.example .env`,
|
||||||
|
then change `POSTGRES_PASSWORD`). Only the `postgres` service is needed.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker compose --env-file .env up -d --wait postgres
|
||||||
|
./research/bench_indexes.sh
|
||||||
|
./research/bench_recovery.sh
|
||||||
|
python3 research/analyze.py | tee research/results/recovery.md
|
||||||
|
```
|
||||||
|
|
||||||
|
Outputs: `research/results/indexes.md`, `research/results/recovery.csv`,
|
||||||
|
`research/results/recovery.md`.
|
||||||
|
|
||||||
|
## Limits
|
||||||
|
|
||||||
|
- Synthetic skewed data, not a production trace.
|
||||||
|
- Single PostgreSQL node, warm cache, no concurrent writes during the query
|
||||||
|
benchmark.
|
||||||
|
- One dump/restore run per size; timings are indicative, not a capacity plan.
|
||||||
|
- The 878 RPS figure comes from the closed-loop client in `REPORT.md`; it is
|
||||||
|
not a maximum throughput claim.
|
||||||
Executable
+46
@@ -0,0 +1,46 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Fit recovery time against message count and estimate the dump-loss window."""
|
||||||
|
import csv
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parent
|
||||||
|
RPS = float(os.environ.get("RPS", "878")) # closed-loop POST RPS from REPORT.md
|
||||||
|
rows = list(csv.DictReader((ROOT / "results/recovery.csv").open()))
|
||||||
|
|
||||||
|
|
||||||
|
def fit(ys):
|
||||||
|
# AGENT: three points are enough for the linear trend this lab asks for;
|
||||||
|
# R^2 is printed so a bad fit is visible instead of silently accepted.
|
||||||
|
xs = [int(row["messages"]) for row in rows]
|
||||||
|
n = len(xs)
|
||||||
|
mx = sum(xs) / n
|
||||||
|
my = sum(ys) / n
|
||||||
|
slope = sum((x - mx) * (y - my) for x, y in zip(xs, ys)) / sum((x - mx) ** 2 for x in xs)
|
||||||
|
intercept = my - slope * mx
|
||||||
|
total = sum((y - my) ** 2 for y in ys)
|
||||||
|
residual = sum((y - (intercept + slope * x)) ** 2 for x, y in zip(xs, ys))
|
||||||
|
r2 = 1 - residual / total if total else 1.0
|
||||||
|
return intercept, slope, r2
|
||||||
|
|
||||||
|
|
||||||
|
dump = fit([float(row["dump_s"]) for row in rows])
|
||||||
|
restore = fit([float(row["restore_s"]) for row in rows])
|
||||||
|
total = fit([float(row["dump_s"]) + float(row["restore_s"]) for row in rows])
|
||||||
|
|
||||||
|
print("| messages | dump s | restore s | dump MiB | DB MiB | lost msgs at %.0f RPS |" % RPS)
|
||||||
|
print("| ---: | ---: | ---: | ---: | ---: | ---: |")
|
||||||
|
for row in rows:
|
||||||
|
dump_s = float(row["dump_s"])
|
||||||
|
print("| {messages} | {dump_s} | {restore_s} | {dump_mib:.1f} | {db_mib:.1f} | {loss:.0f} |".format(
|
||||||
|
messages=row["messages"],
|
||||||
|
dump_s=row["dump_s"],
|
||||||
|
restore_s=row["restore_s"],
|
||||||
|
dump_mib=int(row["dump_bytes"]) / 2**20,
|
||||||
|
db_mib=int(row["db_bytes"]) / 2**20,
|
||||||
|
loss=dump_s * RPS,
|
||||||
|
))
|
||||||
|
|
||||||
|
print()
|
||||||
|
for name, (a, b, r2) in (("dump", dump), ("restore", restore), ("dump+restore", total)):
|
||||||
|
print(f"{name}: t = {a:.3f} + {b * 1_000_000:.3f} * (messages / 1M) s, R^2={r2:.3f}")
|
||||||
Executable
+191
@@ -0,0 +1,191 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Track 2: compare the two supporting indexes on one skewed dataset.
|
||||||
|
# Exactly one of chat_members_user_idx / messages_chat_created_id_idx is
|
||||||
|
# present in the two comparison states; both are restored at the end.
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
|
||||||
|
set -a
|
||||||
|
. ./.env
|
||||||
|
set +a
|
||||||
|
|
||||||
|
DB=messenger_research
|
||||||
|
SIZE=${SIZE:-300000}
|
||||||
|
CHAT_COUNT=${CHAT_COUNT:-5000}
|
||||||
|
USER_COUNT=${USER_COUNT:-2000}
|
||||||
|
RESULTS=research/results/indexes.md
|
||||||
|
COMPOSE=(docker compose --env-file .env)
|
||||||
|
ZERO=00000000-0000-0000-0000-000000000000
|
||||||
|
mkdir -p research/results
|
||||||
|
|
||||||
|
psqlq() {
|
||||||
|
"${COMPOSE[@]}" exec -T postgres psql -X -q -A -t \
|
||||||
|
-U "$POSTGRES_USER" -d "$DB" -c "$1"
|
||||||
|
}
|
||||||
|
|
||||||
|
psqlf() {
|
||||||
|
"${COMPOSE[@]}" exec -T postgres psql -X -q \
|
||||||
|
-U "$POSTGRES_USER" -d "$DB" -v ON_ERROR_STOP=1 "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
echo "Rebuilding $DB at $SIZE messages..."
|
||||||
|
"${COMPOSE[@]}" exec -T postgres psql -X -q -U "$POSTGRES_USER" -d postgres \
|
||||||
|
-v ON_ERROR_STOP=1 \
|
||||||
|
-c "DROP DATABASE IF EXISTS $DB WITH (FORCE);" \
|
||||||
|
-c "CREATE DATABASE $DB;" >/dev/null
|
||||||
|
psqlf -f - < app/internal/database/migrations/0001_init.up.sql >/dev/null
|
||||||
|
psqlq "CREATE TABLE IF NOT EXISTS schema_migrations (version bigint NOT NULL PRIMARY KEY, dirty boolean NOT NULL);
|
||||||
|
INSERT INTO schema_migrations VALUES (1, false) ON CONFLICT DO NOTHING;" >/dev/null
|
||||||
|
psqlf -v target_messages="$SIZE" -v chat_count="$CHAT_COUNT" \
|
||||||
|
-v user_count="$USER_COUNT" -f - < research/seed.sql >/dev/null
|
||||||
|
|
||||||
|
# All three queries are built from the API's access patterns:
|
||||||
|
# user_chats - list the chats of one user
|
||||||
|
# chat_messages - keyset page of one chat
|
||||||
|
# user_feed - messages from all of one user's chats, keyset page
|
||||||
|
# The feed cursor is recent so this is a realistic page, not a full-history
|
||||||
|
# sort; the query still joins every message to the user's memberships.
|
||||||
|
Q_USER_CHATS="SELECT c.id, c.title, c.created_at
|
||||||
|
FROM chats c
|
||||||
|
JOIN chat_members cm ON cm.chat_id = c.id
|
||||||
|
WHERE cm.user_id = research_user_uuid(1)
|
||||||
|
ORDER BY c.created_at, c.id
|
||||||
|
LIMIT 50;"
|
||||||
|
|
||||||
|
Q_CHAT_MESSAGES="SELECT id, chat_id, sender_id, text, created_at
|
||||||
|
FROM messages
|
||||||
|
WHERE chat_id = research_chat_uuid(1)
|
||||||
|
AND (created_at, id) > ('2026-02-15T00:00:00Z'::timestamptz, '$ZERO'::uuid)
|
||||||
|
ORDER BY created_at, id
|
||||||
|
LIMIT 51;"
|
||||||
|
|
||||||
|
Q_USER_FEED="SELECT m.id, m.chat_id, m.sender_id, m.text, m.created_at
|
||||||
|
FROM messages m
|
||||||
|
JOIN chat_members cm ON cm.chat_id = m.chat_id
|
||||||
|
WHERE cm.user_id = research_user_uuid(1)
|
||||||
|
AND (m.created_at, m.id) > ('2026-03-01T00:00:00Z'::timestamptz, '$ZERO'::uuid)
|
||||||
|
ORDER BY m.created_at, m.id
|
||||||
|
LIMIT 50;"
|
||||||
|
|
||||||
|
ensure_both() {
|
||||||
|
psqlq "CREATE INDEX IF NOT EXISTS chat_members_user_idx ON chat_members (user_id, chat_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS messages_chat_created_id_idx ON messages (chat_id, created_at, id);" >/dev/null
|
||||||
|
}
|
||||||
|
|
||||||
|
messages_only() {
|
||||||
|
psqlq "CREATE INDEX IF NOT EXISTS messages_chat_created_id_idx ON messages (chat_id, created_at, id);
|
||||||
|
DROP INDEX IF EXISTS chat_members_user_idx;" >/dev/null
|
||||||
|
}
|
||||||
|
|
||||||
|
membership_only() {
|
||||||
|
psqlq "CREATE INDEX IF NOT EXISTS chat_members_user_idx ON chat_members (user_id, chat_id);
|
||||||
|
DROP INDEX IF EXISTS messages_chat_created_id_idx;" >/dev/null
|
||||||
|
}
|
||||||
|
|
||||||
|
# AGENT: the comparison is only meaningful if exactly one index is present;
|
||||||
|
# a leftover second index would hide the regression this experiment measures.
|
||||||
|
assert_single() {
|
||||||
|
local want_membership=$1 want_messages=$2
|
||||||
|
local count has_membership has_messages
|
||||||
|
count=$(psqlq "SELECT count(*) FROM pg_indexes
|
||||||
|
WHERE indexname IN ('chat_members_user_idx', 'messages_chat_created_id_idx');")
|
||||||
|
has_membership=$(psqlq "SELECT to_regclass('chat_members_user_idx') IS NOT NULL;")
|
||||||
|
has_messages=$(psqlq "SELECT to_regclass('messages_chat_created_id_idx') IS NOT NULL;")
|
||||||
|
if [ "$count" != "1" ] || [ "$has_membership" != "$want_membership" ] || [ "$has_messages" != "$want_messages" ]; then
|
||||||
|
echo "index state is not controlled: count=$count membership=$has_membership messages=$has_messages" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_both() {
|
||||||
|
local count
|
||||||
|
count=$(psqlq "SELECT count(*) FROM pg_indexes
|
||||||
|
WHERE indexname IN ('chat_members_user_idx', 'messages_chat_created_id_idx');")
|
||||||
|
[ "$count" = "2" ] || { echo "both indexes were not restored" >&2; exit 1; }
|
||||||
|
}
|
||||||
|
|
||||||
|
bench_query() {
|
||||||
|
local label=$1 sql=$2
|
||||||
|
psqlq "EXPLAIN (ANALYZE, BUFFERS) $sql" >/dev/null
|
||||||
|
local times=()
|
||||||
|
local i t
|
||||||
|
for i in 1 2 3 4 5; do
|
||||||
|
t=$(psqlq "EXPLAIN (ANALYZE, BUFFERS) $sql" |
|
||||||
|
sed -n 's/.*Execution Time: \([0-9.]*\) ms/\1/p')
|
||||||
|
times+=("$t")
|
||||||
|
done
|
||||||
|
local median plan
|
||||||
|
median=$(printf '%s\n' "${times[@]}" | sort -n |
|
||||||
|
awk '{a[NR]=$1} END {print a[int((NR + 1) / 2)]}')
|
||||||
|
plan=$(psqlq "EXPLAIN (ANALYZE, BUFFERS) $sql" |
|
||||||
|
grep -oE '(Index Only Scan|Index Scan|Seq Scan|Bitmap Heap Scan|Sort|Hash Join|Nested Loop|Merge Join)' |
|
||||||
|
awk '{if ($0 != prev) print; prev = $0}' |
|
||||||
|
paste -sd '>' -)
|
||||||
|
printf '| %s | %s | %s |\n' "$label" "$median" "$plan"
|
||||||
|
}
|
||||||
|
|
||||||
|
bench_write() {
|
||||||
|
psqlq "DROP TABLE IF EXISTS research_write_bench;
|
||||||
|
CREATE TABLE research_write_bench (
|
||||||
|
id uuid PRIMARY KEY,
|
||||||
|
chat_id uuid NOT NULL,
|
||||||
|
sender_id uuid NOT NULL,
|
||||||
|
text text NOT NULL,
|
||||||
|
created_at timestamptz NOT NULL
|
||||||
|
);" >/dev/null
|
||||||
|
if [ "$1" = "yes" ]; then
|
||||||
|
psqlq "CREATE INDEX research_write_bench_idx
|
||||||
|
ON research_write_bench (chat_id, created_at, id);" >/dev/null
|
||||||
|
fi
|
||||||
|
"${COMPOSE[@]}" cp research/write.sql postgres:/tmp/research_write.sql >/dev/null
|
||||||
|
local tps
|
||||||
|
tps=$("${COMPOSE[@]}" exec -T postgres pgbench -n -c 4 -j 2 -T 10 \
|
||||||
|
-U "$POSTGRES_USER" -d "$DB" -f /tmp/research_write.sql 2>&1 |
|
||||||
|
sed -n 's/^tps = \([0-9.]*\).*/\1/p')
|
||||||
|
psqlq "DROP TABLE research_write_bench;" >/dev/null
|
||||||
|
echo "$tps"
|
||||||
|
}
|
||||||
|
|
||||||
|
{
|
||||||
|
echo "# Index experiment"
|
||||||
|
echo
|
||||||
|
echo "Dataset: $SIZE messages, $CHAT_COUNT chats, $USER_COUNT users."
|
||||||
|
echo "Median of 5 warm-cache runs; plan is the sequence of access nodes."
|
||||||
|
echo
|
||||||
|
echo "| state | query | median ms | plan |"
|
||||||
|
echo "| --- | --- | ---: | --- |"
|
||||||
|
|
||||||
|
ensure_both
|
||||||
|
assert_both
|
||||||
|
bench_query "both | user_chats" "$Q_USER_CHATS"
|
||||||
|
bench_query "both | chat_messages" "$Q_CHAT_MESSAGES"
|
||||||
|
bench_query "both | user_feed" "$Q_USER_FEED"
|
||||||
|
write_with=$(bench_write yes)
|
||||||
|
|
||||||
|
messages_only
|
||||||
|
assert_single f t
|
||||||
|
bench_query "messages_only | user_chats" "$Q_USER_CHATS"
|
||||||
|
bench_query "messages_only | chat_messages" "$Q_CHAT_MESSAGES"
|
||||||
|
bench_query "messages_only | user_feed" "$Q_USER_FEED"
|
||||||
|
|
||||||
|
membership_only
|
||||||
|
assert_single t f
|
||||||
|
bench_query "membership_only | user_chats" "$Q_USER_CHATS"
|
||||||
|
bench_query "membership_only | chat_messages" "$Q_CHAT_MESSAGES"
|
||||||
|
bench_query "membership_only | user_feed" "$Q_USER_FEED"
|
||||||
|
write_without=$(bench_write no)
|
||||||
|
|
||||||
|
ensure_both
|
||||||
|
assert_both
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "Index sizes after restoring both:"
|
||||||
|
psqlq "SELECT 'chat_members_user_idx=' || pg_size_pretty(pg_relation_size('chat_members_user_idx')) ||
|
||||||
|
', messages_chat_created_id_idx=' || pg_size_pretty(pg_relation_size('messages_chat_created_id_idx')) ||
|
||||||
|
', messages_table=' || pg_size_pretty(pg_total_relation_size('messages'));"
|
||||||
|
echo
|
||||||
|
echo "pgbench message-like inserts, 4 clients, 10 s: with messages index ${write_with} tps; without ${write_without} tps."
|
||||||
|
echo "The research DB is left with both indexes."
|
||||||
|
} | tee "$RESULTS"
|
||||||
|
|
||||||
|
echo "Wrote $RESULTS"
|
||||||
Executable
+112
@@ -0,0 +1,112 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Track 2: dump/restore timing at 100k, 200k and 300k messages.
|
||||||
|
# The schema comes from the repository migration; both indexes are present
|
||||||
|
# for every dump. The research DB is left at 300k with both indexes.
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
|
||||||
|
set -a
|
||||||
|
. ./.env
|
||||||
|
set +a
|
||||||
|
|
||||||
|
DB=messenger_research
|
||||||
|
RESTORE_DB=messenger_research_restore
|
||||||
|
DUMP=/tmp/research_recovery.dump
|
||||||
|
RESULTS=research/results/recovery.csv
|
||||||
|
COMPOSE=(docker compose --env-file .env)
|
||||||
|
mkdir -p research/results
|
||||||
|
|
||||||
|
psql_db() {
|
||||||
|
local db=$1 sql=$2
|
||||||
|
"${COMPOSE[@]}" exec -T postgres psql -X -q -A -t \
|
||||||
|
-U "$POSTGRES_USER" -d "$db" -c "$sql"
|
||||||
|
}
|
||||||
|
|
||||||
|
psql_admin() {
|
||||||
|
"${COMPOSE[@]}" exec -T postgres psql -X -q -A -t \
|
||||||
|
-U "$POSTGRES_USER" -d postgres -v ON_ERROR_STOP=1 -c "$1"
|
||||||
|
}
|
||||||
|
|
||||||
|
measure() {
|
||||||
|
local command=$1
|
||||||
|
# AGENT: /usr/bin/time runs inside the container so the client-side
|
||||||
|
# docker exec overhead is not counted as backup or restore time.
|
||||||
|
if ! "${COMPOSE[@]}" exec -T postgres sh -c \
|
||||||
|
"/usr/bin/time -f '%e' -o /tmp/research_time $command >/dev/null 2>&1"; then
|
||||||
|
echo "timed command failed: $command" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
"${COMPOSE[@]}" exec -T postgres cat /tmp/research_time
|
||||||
|
}
|
||||||
|
|
||||||
|
ensure_both_indexes() {
|
||||||
|
psql_db "$DB" "CREATE INDEX IF NOT EXISTS chat_members_user_idx
|
||||||
|
ON chat_members (user_id, chat_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS messages_chat_created_id_idx
|
||||||
|
ON messages (chat_id, created_at, id);" >/dev/null
|
||||||
|
}
|
||||||
|
|
||||||
|
printf 'messages,dump_s,restore_s,dump_bytes,db_bytes,table_bytes,index_bytes\n' > "$RESULTS"
|
||||||
|
|
||||||
|
for n in 100000 200000 300000; do
|
||||||
|
echo "Rebuilding $DB at $n messages..."
|
||||||
|
psql_admin "DROP DATABASE IF EXISTS $RESTORE_DB WITH (FORCE);" >/dev/null
|
||||||
|
psql_admin "DROP DATABASE IF EXISTS $DB WITH (FORCE);" >/dev/null
|
||||||
|
psql_admin "CREATE DATABASE $DB;" >/dev/null
|
||||||
|
"${COMPOSE[@]}" exec -T postgres psql -X -q -U "$POSTGRES_USER" -d "$DB" \
|
||||||
|
-v ON_ERROR_STOP=1 -f - < app/internal/database/migrations/0001_init.up.sql >/dev/null
|
||||||
|
psql_db "$DB" "CREATE TABLE IF NOT EXISTS schema_migrations
|
||||||
|
(version bigint NOT NULL PRIMARY KEY, dirty boolean NOT NULL);
|
||||||
|
INSERT INTO schema_migrations VALUES (1, false)
|
||||||
|
ON CONFLICT DO NOTHING;" >/dev/null
|
||||||
|
"${COMPOSE[@]}" exec -T postgres psql -X -q -U "$POSTGRES_USER" -d "$DB" \
|
||||||
|
-v ON_ERROR_STOP=1 -v target_messages="$n" -v chat_count=5000 \
|
||||||
|
-v user_count=2000 -f - < research/seed.sql >/dev/null
|
||||||
|
ensure_both_indexes
|
||||||
|
|
||||||
|
db_bytes=$(psql_db "$DB" "SELECT pg_database_size(current_database());")
|
||||||
|
table_bytes=$(psql_db "$DB" "SELECT pg_total_relation_size('messages');")
|
||||||
|
index_bytes=$(psql_db "$DB" "SELECT pg_relation_size('chat_members_user_idx') +
|
||||||
|
pg_relation_size('messages_chat_created_id_idx');")
|
||||||
|
|
||||||
|
dump_s=$(measure "pg_dump -U $POSTGRES_USER -d $DB -Fc -f $DUMP")
|
||||||
|
dump_bytes=$("${COMPOSE[@]}" exec -T postgres stat -c %s "$DUMP")
|
||||||
|
psql_admin "CREATE DATABASE $RESTORE_DB;" >/dev/null
|
||||||
|
restore_s=$(measure "pg_restore -U $POSTGRES_USER -d $RESTORE_DB --no-owner --no-privileges $DUMP")
|
||||||
|
|
||||||
|
src_count=$(psql_db "$DB" "SELECT count(*) FROM messages;")
|
||||||
|
dst_count=$(psql_db "$RESTORE_DB" "SELECT count(*) FROM messages;")
|
||||||
|
src_members=$(psql_db "$DB" "SELECT count(*) FROM chat_members;")
|
||||||
|
dst_members=$(psql_db "$RESTORE_DB" "SELECT count(*) FROM chat_members;")
|
||||||
|
# AGENT: one aggregate fingerprint covers every message row, so the check
|
||||||
|
# is not limited to whichever row happened to be newest.
|
||||||
|
src_fingerprint=$(psql_db "$DB" "SELECT md5(string_agg(
|
||||||
|
id::text || '|' || chat_id::text || '|' || sender_id::text || '|' ||
|
||||||
|
text || '|' || created_at::text, E'\n' ORDER BY id)) FROM messages;")
|
||||||
|
dst_fingerprint=$(psql_db "$RESTORE_DB" "SELECT md5(string_agg(
|
||||||
|
id::text || '|' || chat_id::text || '|' || sender_id::text || '|' ||
|
||||||
|
text || '|' || created_at::text, E'\n' ORDER BY id)) FROM messages;")
|
||||||
|
dst_indexes=$(psql_db "$RESTORE_DB" "SELECT count(*) FROM pg_indexes
|
||||||
|
WHERE indexname IN ('chat_members_user_idx',
|
||||||
|
'messages_chat_created_id_idx');")
|
||||||
|
|
||||||
|
if [ "$src_count" != "$dst_count" ] || [ "$src_members" != "$dst_members" ] ||
|
||||||
|
[ "$src_fingerprint" != "$dst_fingerprint" ] || [ "$dst_indexes" != "2" ]; then
|
||||||
|
echo "restore verification failed at $n rows" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
printf '%s,%s,%s,%s,%s,%s,%s\n' \
|
||||||
|
"$n" "$dump_s" "$restore_s" "$dump_bytes" "$db_bytes" "$table_bytes" "$index_bytes" \
|
||||||
|
>> "$RESULTS"
|
||||||
|
echo " dump ${dump_s}s, restore ${restore_s}s, ${dump_bytes} bytes; verified"
|
||||||
|
psql_admin "DROP DATABASE $RESTORE_DB WITH (FORCE);" >/dev/null
|
||||||
|
"${COMPOSE[@]}" exec -T postgres rm -f "$DUMP"
|
||||||
|
done
|
||||||
|
|
||||||
|
ensure_both_indexes
|
||||||
|
echo
|
||||||
|
echo "Recovery results (CSV: messages,dump_s,restore_s,dump_bytes,db_bytes,table_bytes,index_bytes):"
|
||||||
|
cat "$RESULTS"
|
||||||
|
echo
|
||||||
|
echo "Final research DB: $(psql_db "$DB" "SELECT count(*) FROM messages;") messages, both indexes present."
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
# Index experiment
|
||||||
|
|
||||||
|
Dataset: 300000 messages, 5000 chats, 2000 users.
|
||||||
|
Median of 5 warm-cache runs; plan is the sequence of access nodes.
|
||||||
|
|
||||||
|
| state | query | median ms | plan |
|
||||||
|
| --- | --- | ---: | --- |
|
||||||
|
| both | user_chats | 1.379 | Sort>Hash Join>Seq Scan>Index Only Scan |
|
||||||
|
| both | chat_messages | 0.428 | Index Scan |
|
||||||
|
| both | user_feed | 35.396 | Sort>Nested Loop>Index Only Scan>Index Scan |
|
||||||
|
| messages_only | user_chats | 7.360 | Sort>Hash Join>Seq Scan |
|
||||||
|
| messages_only | chat_messages | 0.469 | Index Scan |
|
||||||
|
| messages_only | user_feed | 35.218 | Sort>Nested Loop>Seq Scan>Index Scan |
|
||||||
|
| membership_only | user_chats | 1.402 | Sort>Hash Join>Seq Scan>Index Only Scan |
|
||||||
|
| membership_only | chat_messages | 57.634 | Sort>Seq Scan |
|
||||||
|
| membership_only | user_feed | 80.404 | Sort>Hash Join>Seq Scan>Index Only Scan |
|
||||||
|
|
||||||
|
Index sizes after restoring both:
|
||||||
|
chat_members_user_idx=3536 kB, messages_chat_created_id_idx=17 MB, messages_table=75 MB
|
||||||
|
|
||||||
|
pgbench message-like inserts, 4 clients, 10 s: with messages index 2718.889742 tps; without 2640.028063 tps.
|
||||||
|
The research DB is left with both indexes.
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
messages,dump_s,restore_s,dump_bytes,db_bytes,table_bytes,index_bytes
|
||||||
|
100000,1.02,1.44,5109980,47411683,26402816,9535488
|
||||||
|
200000,1.92,2.36,9803565,74240483,53231616,15417344
|
||||||
|
300000,2.66,3.28,14500027,99627491,78618624,21307392
|
||||||
|
@@ -0,0 +1,9 @@
|
|||||||
|
| messages | dump s | restore s | dump MiB | DB MiB | lost msgs at 878 RPS |
|
||||||
|
| ---: | ---: | ---: | ---: | ---: | ---: |
|
||||||
|
| 100000 | 1.02 | 1.44 | 4.9 | 45.2 | 896 |
|
||||||
|
| 200000 | 1.92 | 2.36 | 9.3 | 70.8 | 1686 |
|
||||||
|
| 300000 | 2.66 | 3.28 | 13.8 | 95.0 | 2335 |
|
||||||
|
|
||||||
|
dump: t = 0.227 + 8.200 * (messages / 1M) s, R^2=0.997
|
||||||
|
restore: t = 0.520 + 9.200 * (messages / 1M) s, R^2=1.000
|
||||||
|
dump+restore: t = 0.747 + 17.400 * (messages / 1M) s, R^2=0.999
|
||||||
@@ -0,0 +1,114 @@
|
|||||||
|
-- Synthetic, repeatable dataset for the Track 2 experiments.
|
||||||
|
-- Usage:
|
||||||
|
-- psql -v target_messages=300000 -v chat_count=5000 -v user_count=2000 -f seed.sql
|
||||||
|
-- The distribution is deliberately skewed: a few chats hold most messages,
|
||||||
|
-- which is closer to a real messenger than one evenly filled chat.
|
||||||
|
\set ON_ERROR_STOP on
|
||||||
|
\if :{?target_messages}
|
||||||
|
\else
|
||||||
|
\set target_messages 300000
|
||||||
|
\endif
|
||||||
|
\if :{?chat_count}
|
||||||
|
\else
|
||||||
|
\set chat_count 5000
|
||||||
|
\endif
|
||||||
|
\if :{?user_count}
|
||||||
|
\else
|
||||||
|
\set user_count 2000
|
||||||
|
\endif
|
||||||
|
|
||||||
|
-- Deterministic, readable UUIDs. The version/variant nibbles are valid.
|
||||||
|
CREATE OR REPLACE FUNCTION research_user_uuid(i integer)
|
||||||
|
RETURNS uuid LANGUAGE sql IMMUTABLE AS $$
|
||||||
|
SELECT ('00000000-0000-4000-8000-' || lpad(to_hex(i), 12, '0'))::uuid
|
||||||
|
$$;
|
||||||
|
|
||||||
|
CREATE OR REPLACE FUNCTION research_chat_uuid(i integer)
|
||||||
|
RETURNS uuid LANGUAGE sql IMMUTABLE AS $$
|
||||||
|
SELECT ('10000000-0000-4000-8000-' || lpad(to_hex(i), 12, '0'))::uuid
|
||||||
|
$$;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS research_chat_sender;
|
||||||
|
TRUNCATE messages, chat_members, chats, users CASCADE;
|
||||||
|
|
||||||
|
-- Reproducible pseudo-random layout for a given PostgreSQL build.
|
||||||
|
SELECT setseed(0.4242);
|
||||||
|
|
||||||
|
INSERT INTO users (id, username, display_name, password_hash, created_at)
|
||||||
|
SELECT research_user_uuid(g),
|
||||||
|
'u' || lpad(g::text, 4, '0'),
|
||||||
|
'User ' || g,
|
||||||
|
'not-a-login-hash',
|
||||||
|
'2026-01-01T00:00:00Z'::timestamptz
|
||||||
|
FROM generate_series(1, :user_count) AS g;
|
||||||
|
|
||||||
|
INSERT INTO chats (id, title, created_at)
|
||||||
|
SELECT research_chat_uuid(g),
|
||||||
|
'Research chat ' || g,
|
||||||
|
'2026-01-01T00:00:00Z'::timestamptz + (g || ' seconds')::interval
|
||||||
|
FROM generate_series(1, :chat_count) AS g;
|
||||||
|
|
||||||
|
-- Each chat gets a small random membership set (5..24 distinct users).
|
||||||
|
INSERT INTO chat_members (chat_id, user_id)
|
||||||
|
SELECT research_chat_uuid(c),
|
||||||
|
research_user_uuid(1 + floor(random() * :user_count)::int)
|
||||||
|
FROM generate_series(1, :chat_count) AS c
|
||||||
|
CROSS JOIN generate_series(1, 5 + (c % 20)) AS m
|
||||||
|
ON CONFLICT DO NOTHING;
|
||||||
|
|
||||||
|
-- Probe user 1 is a member of the ten largest chats and every 100th chat.
|
||||||
|
-- user_chats and user_feed queries therefore always have rows to return.
|
||||||
|
INSERT INTO chat_members (chat_id, user_id)
|
||||||
|
SELECT research_chat_uuid(c), research_user_uuid(1)
|
||||||
|
FROM generate_series(1, :chat_count) AS c
|
||||||
|
WHERE c <= 10 OR c % 100 = 0
|
||||||
|
ON CONFLICT DO NOTHING;
|
||||||
|
|
||||||
|
-- One valid sender per chat, so every synthetic message has a member sender.
|
||||||
|
CREATE TABLE research_chat_sender AS
|
||||||
|
SELECT chat_id, (array_agg(user_id ORDER BY random()))[1] AS sender_id
|
||||||
|
FROM chat_members
|
||||||
|
GROUP BY chat_id;
|
||||||
|
ALTER TABLE research_chat_sender ADD PRIMARY KEY (chat_id);
|
||||||
|
|
||||||
|
-- AGENT: a uniform chat size would make every query look cheap, so sizes are
|
||||||
|
-- skewed. The probe user is deliberately in the largest chats to exercise the
|
||||||
|
-- worst realistic case for the cross-chat feed.
|
||||||
|
-- floor(random()^3 * chat_count) makes chat 1 hold roughly 6% of all
|
||||||
|
-- messages at 300k, while most chats stay small. Timestamps span 90 days.
|
||||||
|
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||||
|
SELECT gen_random_uuid(),
|
||||||
|
s.chat_id,
|
||||||
|
cs.sender_id,
|
||||||
|
'research message ' || s.g || ' ' || repeat('x', 20 + (s.g % 60)),
|
||||||
|
'2026-01-01T00:00:00Z'::timestamptz + interval '90 days' * s.r
|
||||||
|
FROM (
|
||||||
|
SELECT g,
|
||||||
|
research_chat_uuid(floor(power(random(), 3) * :chat_count)::int + 1) AS chat_id,
|
||||||
|
random() AS r
|
||||||
|
FROM generate_series(1, :target_messages) AS g
|
||||||
|
) AS s
|
||||||
|
JOIN research_chat_sender cs USING (chat_id);
|
||||||
|
|
||||||
|
-- Rebuild the two indexes so their reported sizes are not inflated by the
|
||||||
|
-- random insertion order. This keeps the index comparison and the dump size
|
||||||
|
-- on the same, compact baseline.
|
||||||
|
REINDEX INDEX chat_members_user_idx;
|
||||||
|
REINDEX INDEX messages_chat_created_id_idx;
|
||||||
|
VACUUM (ANALYZE) messages, chat_members, chats, users, research_chat_sender;
|
||||||
|
|
||||||
|
SELECT 'messages' AS item, count(*)::text AS value FROM messages
|
||||||
|
UNION ALL SELECT 'chat_members', count(*)::text FROM chat_members
|
||||||
|
UNION ALL SELECT 'chats', count(*)::text FROM chats
|
||||||
|
UNION ALL SELECT 'users', count(*)::text FROM users
|
||||||
|
UNION ALL SELECT 'messages_table_bytes', pg_total_relation_size('messages')::text
|
||||||
|
UNION ALL SELECT 'chat_members_user_idx_bytes', pg_relation_size('chat_members_user_idx')::text
|
||||||
|
UNION ALL SELECT 'messages_chat_created_id_idx_bytes', pg_relation_size('messages_chat_created_id_idx')::text;
|
||||||
|
|
||||||
|
-- Shape check: largest chats should dominate the message count.
|
||||||
|
SELECT c.id, c.title, count(m.id) AS messages
|
||||||
|
FROM chats c
|
||||||
|
LEFT JOIN messages m ON m.chat_id = c.id
|
||||||
|
GROUP BY c.id, c.title
|
||||||
|
ORDER BY messages DESC
|
||||||
|
LIMIT 5;
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
\set g random(1, 1000000)
|
||||||
|
INSERT INTO research_write_bench (id, chat_id, sender_id, text, created_at)
|
||||||
|
VALUES (
|
||||||
|
gen_random_uuid(),
|
||||||
|
research_chat_uuid(1 + (:g % 5000)),
|
||||||
|
research_user_uuid(1 + (:g % 2000)),
|
||||||
|
'bench ' || :g,
|
||||||
|
'2026-01-01T00:00:00Z'::timestamptz + (:g || ' seconds')::interval
|
||||||
|
);
|
||||||
Reference in New Issue
Block a user