Measurement trumps analysis

John Haman

2020/05/20

Categories: Statistics Tags: Statistics

Along with good design, precise measurement beats analysis.

Statisticians can do a lot of good by convincing researchers to measure the extremity of an event, rather than its occurrence. This is hard: it’s much easier to measure that a component failed than the time of failure. It easier to say a block is broken than to (possible subjectively) measure its degree of broken-ness.

How do we enable researchers and testers to make the best measurements? I don’t have the answers. I just think we can have some big wins by collecting precise and informative data instead of advocating for our favorite methods.

Methodological development is slow, and the gains are logarithmic. If every program focused time/energy on better measurements, the gains would be huge, immediate.