A lot of the time, experiments look like very precise recipes. You have specified amounts of ingredients, a set procedure to follow, and at the very least an expectation of what is going to come out of the oven when you’re done.
There’s play, for sure. There’s tweaking and substituting, forgetting ingredients and adding new ones, or carefully written amendments in the margins reminding your future self to do it differently. But it follows the same line of reasoning as particle physics or artificial enzyme synthesis or computer programming: with knowledge of all the properties of the inputs and a strictly-followed procedure that makes use of these properties, you can reasonably assume that you’re going to get the same results every time you do it. And if it fails, you simply have to look at what you’ve done differently, perhaps in your lab manual or perhaps in your memory, and correct it for next time.
I work for Measurement Canada, the federal agency of metrology. The federal lab of measuring things accurately and making sure the things that are measuring things are even more accurate. Almost entirely for industrial purposes, be it for electricity meters or gas pumps or grocery scales. What you might consider to be the epitome of the scientific method, because our process is tested and tweaked constantly and all our variables are locked down, from the temperature and humidity of the room to the equipment drift rate to the standard of gravity at the height of the desks we work at. We get many of those standards from the NRC (National Research Council of Canada), which is the most highly accredited scientific body of research in all of Canada. If Measurement Canada were the epitome of the scientific method, one would think that the NRC would be the very definition.
And yet, as I learned recently while picking up a piston from the NRC from a scientist in the mass lab, it is not always necessary to know every single measurable property about a system in order to get meaningful data from it. In fact, they use a measurement called “covariance” all the time.
Covariance, without getting into the more commonly used algebraic definition, is the measurement of changes in properties that arise out of the symbiosis of variables without necessarily caring about the measurements of the properties of the variables individually. As an example, you can go back to what we know about recipes and think about making icing for a cake. To make icing, you need icing sugar, cream, and butter, and you need to make enough to cover a cake. You know very roughly that you’ll need about a tablespoon full of butter, maybe a cup of icing sugar and maybe half a cup of cream, but anyone who’s made icing can tell you that the amounts listed in a recipe are never really followed all that closely; you’ll always end up adding more of something and tweaking until you have enough icing. In this instance, as you are adding ingredients, you are not necessarily keeping track of how much of each thing you are putting in, but you are keeping track of how much icing you have in total because you need exactly enough to cover a cake. This is the same as if you were measuring the optical path length of a laser; you are constantly tweaking the lenses and mirrors that are directing the laser, but you don’t care about how much Mirror A is moving nearly as much as you care about what phase is being projected on a screen.
This is also relevant in the pressure lab that I work in. Some equipment deals with gauge pressure, which is solely measuring the difference in pressure between two environments without really caring about how much pressure is in each environment compared to a vacuum, which would be absolute pressure. In fact this idea is prevalent in all disciplines, especially in quantum mechanics, where the uncertainty principle make it impossible to know some of the individual properties of particles anyway.
To me, this says something about the way we think about the scientific method and how difficult it is to define it. “A process or conclusion cannot be considered scientific”, they say in school with the same hushed reverence as a monk has for the divine, “unless there is empirically measured evidence and all variables are known.” But in some cases we can get meaningful information from an experiment when we are only measuring the outcome of relevant changes and not the changes themselves. Does this make it unscientific? Of course not. But if we are including the certainty of all relevant variables in our definition of a scientific process, there seems to be something important that we have not quite incorporated into our definition of the scientific method.