I’ve thought about this a lot too and I’ve come to the conclusion that testing is a data analysis problem…something we are all familiar with! To that end, I like to think about instrumentation, storage and analysis as discrete problems with testing. Something gets the data, the data is stored and then you figure out what it means. Keeping it modular let’s us evolve each piece as we need to. So that noise you referenced earlier…over time our analysis should learn to deal with that