Very well put. The analyst's job is to figure whether it matters or not. With limited data that can be *very* challenging and if the sample size is small may be impossible.
Good experiment design is absolutely critical. In a seismic survey one has to make sure it is adequately sampled in X, Y, Z & T. On a $10 million 3D survey the technical specifications in the contract run for many pages and there is a company rep on board who is constantly checking that everything is within spec. With the tremendous increase in low cost compute power they will process the data on board to check the results. No one would use that for anything but field QC. But because the SNR is so much less than 1, unless the data is run through the full processing sequence there is no way to tell if you have usable data at 10 seconds or if it is all noise.
As I understand the terms (I dealt with the data, not the acquisition), if the data does not meet spec, all ship and crew time are on the seismic company's nickel until the data meets spec. There are pages of stuff in the contract about how near other vessels can be, marine life, sea state and more strange seeming, but critical specs. I have only seen one contract and that only for a few minutes. But I was stunned by how strict the requirements were. And even more stunned when I realized that the bad data I often saw had met the spec, but still needed a *lot* of work to clean up.
Reg