forked from mtusi2026/mtusi2026lab3
Lab 3 grade-5 scaling experiment
Generated: 2026-10-23T15:23:12.734339+00:00.
Hypothesis
API replicas stop improving throughput when shared PostgreSQL CPU saturates; adding PostgreSQL CPU should improve the unchanged four-replica profile.
Reproduction
- workload: closed-loop POST messages through the public HAProxy gateway; unique idempotency keys; no client retries;
- phase duration: 10 s;
- comparison concurrency: 32;
- saturation sweep: [1, 2, 4, 8, 16, 32];
- fixed budget: 1.00 API CPU, 512 MiB API RAM, 16 total DB pool connections;
- growth series: 0.50 API CPU, 256 MiB RAM and 8 DB pool connections per replica;
- saturation rule: RPS gain below 10% and p95 growth at least 1.5x versus the previous step;
- host: 8 logical CPU, 3813.6 MiB RAM;
- Docker 29.7.2, Compose 5.5.0;
- command:
make grade5-experiment; warmup runs before every changed profile.
Measurements
| Phase | Replicas | Concurrency | RPS | Errors | p50 ms | p95 ms | p99 ms | PG CPU avg | API CPU avg |
|---|---|---|---|---|---|---|---|---|---|
| saturation-c1 | 2 | 1 | 230.369 | 0 (0.000%) | 3.839 | 5.065 | 6.282 | 23.9 | 26.85 |
| saturation-c2 | 2 | 2 | 320.19 | 0 (0.000%) | 5.192 | 7.196 | 15.11 | 54.987 | 48.3 |
| saturation-c4 | 2 | 4 | 435.822 | 0 (0.000%) | 6.171 | 29.304 | 47.665 | 63.093 | 54.453 |
| saturation-c8 | 2 | 8 | 470.828 | 0 (0.000%) | 9.159 | 56.23 | 70.71 | 65.945 | 48.505 |
| saturation-c16 | 2 | 16 | 472.892 | 0 (0.000%) | 19.862 | 73.628 | 98.773 | 65.08 | 38.028 |
| saturation-c32 | 2 | 32 | 564.012 | 0 (0.000%) | 51.089 | 94.785 | 109.138 | 83.825 | 51.965 |
| fixed-r1 | 1 | 32 | 578.242 | 0 (0.000%) | 48.579 | 95.757 | 115.549 | 79.058 | 36.575 |
| fixed-r2 | 2 | 32 | 529.499 | 0 (0.000%) | 53.316 | 100.145 | 129.953 | 81.065 | 44.7 |
| fixed-r4 | 4 | 32 | 353.955 | 0 (0.000%) | 69.354 | 139.408 | 291.312 | 74.677 | 46.25 |
| growth-r1 | 1 | 32 | 533.656 | 0 (0.000%) | 53.378 | 93.541 | 115.58 | 77.252 | 38.273 |
| growth-r2 | 2 | 32 | 563.46 | 0 (0.000%) | 50.516 | 96.428 | 129.46 | 85.597 | 46.653 |
| growth-r4 | 4 | 32 | 438.868 | 0 (0.000%) | 68.331 | 109.446 | 199.616 | 84.847 | 35.69 |
| improved-db-cpu-r4 | 4 | 32 | 587.574 | 0 (0.000%) | 49.466 | 93.991 | 112.002 | 95.573 | 55.2 |
Conclusion
The first saturation signal appears at concurrency 8: RPS changes by +8.03% while p95 changes by 1.92x versus concurrency 4.
Observed bottleneck classification: postgres_cpu.
The PostgreSQL CPU change produced +33.88% RPS and -14.12% p95; confirmed improvement: true.
After the improvement the remaining bottleneck classification is load_balancer_cpu.
Graphs
Raw data and limits
Raw aggregates are in measurements.csv; one-second request buckets are in timeseries.csv; every resource sample is in resources.csv; complete metadata and applied limits are in results.json.
- single Docker host: replicas do not prove host-level availability
- closed-loop workload has coordinated omission
- successful-response latency percentiles are reported separately from failures
- one measured run per profile; host scheduling noise is possible