Experiments

Before execution, an experiment records its hypothesis, baseline, changed component, model and workload limits, plan digests, environment, metrics, quality requirements, assurance work, stopping rule, and expected artifacts.

Report throughput, latency, memory, CPU, online and offline network traffic, disk, preparation work, quality, and assurance separately. Keep failed and negative results. A publication claim must isolate the contribution from its baseline and identify the method unambiguously.

See benchmarks, metrics, and publication assessment.