Считать в отчёте экспериментов CPU на запрос по компонентам и потолок базы при такой цене
This commit is contained in:
+48
-2
@@ -10,8 +10,11 @@ output directory:
|
||||
control.csv the noise checks of every run, the host speed probe and the
|
||||
loadgen CPU per request at the first step, each flagged when
|
||||
more than 25 % off its median;
|
||||
cost.csv CPU milliseconds per request of each component at the
|
||||
COST_RATES steps (valid, error-free, unflagged runs only), and
|
||||
the rate one database CPU would serve at that cost;
|
||||
<series>-throughput.png, <series>-p99.png, capacity.png,
|
||||
cpu-at-capacity.png, pool-waits.png.
|
||||
cpu-at-capacity.png, pool-waits.png, cost-per-request.png.
|
||||
Capacity is the highest step that passed the criterion loadgen applies (valid,
|
||||
under 1 % errors, p99 of successful requests within 500 ms). Charts show the
|
||||
median of the repeats; a hollow marker means the generator fell behind.
|
||||
@@ -30,6 +33,9 @@ import matplotlib.pyplot as plt # noqa: E402
|
||||
|
||||
SLO_P99_MS = 500.0
|
||||
CONTROL_TOLERANCE = 0.25
|
||||
# Below saturation in every configuration, so the cost is not inflated by
|
||||
# queueing.
|
||||
COST_RATES = (800, 1000)
|
||||
COMPONENTS = ("api", "lb", "postgres", "valkey", "loadgen")
|
||||
|
||||
|
||||
@@ -185,11 +191,51 @@ def main(root, out):
|
||||
w = csv.writer(f)
|
||||
w.writerow(["series", "config", "rep", "host_probe_s", "probe_deviation",
|
||||
"loadgen_cpu_ms_per_1000_requests", "cpu_deviation", "flagged"])
|
||||
flagged = set()
|
||||
for series, config, rep, probe, cpu in control:
|
||||
probe_dev = probe / probe_median - 1 if probe is not None else 0.0
|
||||
cpu_dev = cpu / cpu_median - 1
|
||||
noisy = max(abs(probe_dev), abs(cpu_dev)) > CONTROL_TOLERANCE
|
||||
if noisy:
|
||||
flagged.add((series, config, rep))
|
||||
w.writerow([series, config, rep, probe if probe is not None else "", f"{probe_dev:+.2f}", cpu,
|
||||
f"{cpu_dev:+.2f}", max(abs(probe_dev), abs(cpu_dev)) > CONTROL_TOLERANCE])
|
||||
f"{cpu_dev:+.2f}", noisy])
|
||||
|
||||
cost = {}
|
||||
for r in rows:
|
||||
if (r["series"], r["config"], r["rep"]) in flagged or r["offered_rps"] not in COST_RATES:
|
||||
continue
|
||||
if not r["valid"] or r["error_share"] > 0:
|
||||
continue
|
||||
per = cost.setdefault((r["series"], r["config"]), {"runs": set()})
|
||||
per["runs"].add(r["rep"])
|
||||
for c in ("api", "postgres", "lb"):
|
||||
if r[f"cpu_{c}_pct"] != "":
|
||||
per.setdefault(c, []).append(float(r[f"cpu_{c}_pct"]) * 10 / r["offered_rps"])
|
||||
with open(out / "cost.csv", "w", newline="", encoding="utf-8") as f:
|
||||
w = csv.writer(f)
|
||||
w.writerow(["series", "config", "runs", "api_cpu_ms_per_request", "postgres_cpu_ms_per_request",
|
||||
"lb_cpu_ms_per_request", "rps_per_database_cpu"])
|
||||
for (series, config), per in cost.items():
|
||||
med = {c: statistics.median(per[c]) for c in ("api", "postgres", "lb") if per.get(c)}
|
||||
w.writerow([series, config, " ".join(sorted(per["runs"])),
|
||||
*(round(med[c], 2) if c in med else "" for c in ("api", "postgres", "lb")),
|
||||
round(1000 / med["postgres"]) if med.get("postgres") else ""])
|
||||
if cost:
|
||||
keys = list(cost)
|
||||
fig, ax = plt.subplots(figsize=(max(6, len(keys) * 0.9), 4.5))
|
||||
width = 0.27
|
||||
for i, c in enumerate(("api", "postgres", "lb")):
|
||||
ax.bar([j + i * width for j in range(len(keys))],
|
||||
[statistics.median(cost[k][c]) if cost[k].get(c) else 0 for k in keys], width, label=c)
|
||||
ax.set_xticks([j + width for j in range(len(keys))], [f"{s_}/{c_}" for s_, c_ in keys], rotation=30)
|
||||
ax.set_ylabel("CPU ms per request")
|
||||
ax.set_title(f"Cost per request at {'/'.join(map(str, COST_RATES))} rps (clean runs)")
|
||||
ax.grid(axis="y", alpha=0.3)
|
||||
ax.legend()
|
||||
fig.tight_layout()
|
||||
fig.savefig(out / "cost-per-request.png", dpi=120)
|
||||
plt.close(fig)
|
||||
|
||||
def median_by_offered(series, config, field):
|
||||
points, valid = {}, {}
|
||||
|
||||
Reference in New Issue
Block a user