Why benchmarks?
Growing consensus that we need to focus on these problems in the research community
- HotOS-7: “No-Futz Computing”
- Hennessy at FCRC:
- “other qualities [than performance] will become crucial: availability, maintainability, scalability”
- “if access to services on servers is the killer app--availability is the key metric”
- Lampson at SOSP ’99:
- big challenge for systems research is building “systems that work: meet specifications, are always available, evolve while they run, grow without practical limits”
Benchmarks shape a field!
- they define what we can measure