What does a repeated microbenchmark leave out?
Compare the costs visible in an isolated hash-table operation with the effect of workload and cache behavior. Microbenchmarks help answer a narrower question than end-to-end measurements.
Read on through a shared source
When does an implementation detail become a promise? →Callers can depend on observable behavior that was never documented. Changing an interpreter or a library then becomes a compatibility decision.When does an implementation detail become a promise? →Interpreter behavior, hash tables and public APIs illustrate how callers come to rely on what they can observe. Observable behavior need not have been formally guaranteed.When is a typo a valid program? →Features can interact so that an unintended expression still parses. The language accepts it, leaving the programmer to discover the changed meaning.
Compare the passages
But without running the data once, how do you know what
dimensions are inconsequential?when you go to time it you're timing the feature of interest which puts it in cash anyway