Repetition changes an experiment
Monitoring repeats fixed prompts for comparability. Repetition can also interact with provider caching, so a stable prompt does not automatically represent a fresh task. Cache policy and observed usage categories belong in the record. Document whether a profile represents repeated or uncached work.
Keep profiles distinct
A short health check asks whether a request completes. A speed profile examines timing under a defined task. An optional coding profile adds another kind of work. Combining their distributions can make a provider appear to change speed simply because the mix of requests changed.
Use explicit versions
Version prompt text, output limits, reasoning settings and measurement code. A change creates a new comparison identity or documented migration, not a silent alteration of an established baseline. Record exact response model identity where available, because an alias can change without its visible catalog name changing.
Explain what is unknown
Caching is only one contributor to latency. Do not infer a provider internal cache state from timing alone. Use documented response fields where available and mark unknowns as unknown. A fair comparison explains what was controlled, what was observed and what could not be determined, so readers can assess the experiment.