Tuesday, June 19, 2012

The Scientific Method and Epistemology

photo by widdowquinn via PhotoRee

Listening to a past Intelligence Squared discussion on whether "Science will have all the answers", or as interpreted by one of the panelist "science is the only route to knowledge." This question lies in the purview of epistemology, the study of the nature of knowledge and the limits of that knowledge. Curiously enough, all this question is ultimately wrapped up in mathematical logic. Taking a step back, consider for a moment when you were first introduced to the scientific method. Hypothesize, predict, and test. It sounds all nice and intuitively rigorous, but where does it come from? Why does it supposed to intrinsically make sense? It turns out that there is nothing really fundamental about this formulation of investigation. My sense of the matter is that people still treat science very superstitiously as if it were some kind of magic or dogma. From grade school on, we were taught the one true answer as to how the natural world works. We revere Galileo and Darwin, and their respective scientific discoveries as if they received the discoveries from the heavens and did not require experimentation, hotly debated peer review, and eventual validation by other work. Moreover, we were taught at least implicitly that these great scientists were never wrong. Their theories never had to be revised or be subject to independent validation. Moreover, it seems as if we think that their theories are and were always the only reasonable answer. There is a kind of survivor's bias in science education. We laugh off Lamarck and others as being so obviously wrong and even as absurd forgetting that they were real scientists too despite not having directly produced the prevailing theory of our time. In fact, I dare say that it would be difficult to say what our understanding of the world would look like today if not for these other scientists upon whose work still contributed to the development of our current understanding of the world. Even the scientific method itself is treated as received knowledge and not as an amalgam of multiple reasonable routes of scientific investigation. Science itself is presented as a winner-takes-all activity.

Don't get me wrong. As one obviously with strong vested interest in science research, I know very well that the scientific method serves a role. Empiricism and experimentalism both have strong appeals. However, I would argue that they do not provide the whole picture. Many rigorous fields of knowledge do not rely on the scientific method as commonly formulated. I think mathematical logic, the associated philosophy, and other fields rooted in mathematical logic such as Computability Theory, Complexity Theory, and Programming Languages Theory are great examples of this.

One of the panelists explains the methodology of science as getting the view from no where, a completely objective view. Her claim is that in many cases, this is not practical.

A paper by MARK WINDSCHITL, JESSICA THOMPSON, MELISSA BRAATEN of UWash entitled "Beyond the Scientific Method: Model-Based Inquiry as a New Paradigm of Preference for School Science Investigations" (2007) discusses the epistemic aspects of the scientific method in science education. They begin with a stunner:

One needs only to open popular science texts, to examine students’ laboratory notebooks, or to listen to science fair presentations at your local middle school to recognize that TSM remains a durable icon that actively shapes how teachers and learners think about scientific practice. Most teachers and many of their students can recite from memory the steps of this process. With only minor variations: observe, develop a question, develop a hypothesis, conduct an experiment, analyze data, state conclusions, generate new questions. We assert, however, that TSM is not scientific at all when considered from an epistemic perspective, and that it subverts young learners’ understandings of both the practices and the content of the discipline.

There is actually a rich literature is in this in terms of a critique of the scientific method (see Windschitl et al's bibliography). Being from a background of programming languages research and mathematical logic, my primary mode of investigation is one of those that does not fit nicely into the mold of the scientific method. My advisor once only half-jokingly said that in programming languages research, we were studying a natural phenomenon, that of how people programmed. Yet, programming languages researchers generally do not follow the steps of the scientific method, and certainly not in that explicit order. Instead, researchers observe, develop a formal framework of that phenomenon we are observing (the model), formulate lemmas and theorems, attempt to prove the propositions, and then possibly refine the propositions enough so that we can prove it. Of course, there are a number of epistemic problems in this process, as happens to be the case with all formulaic processes. The fact remains that mathematical logicians follow a similar process. They are also truth-seekers. In fact, I would consider them to be more fundamental truth seekers than many applied scientists, yet a mathematical logician's mode of investigation does not fit into the framework of the scientific method.

An interesting aspect of all this is that those studying the epistemology of the scientific method must ultimately resort to formal logic. Why is this the case? The scientific method is essentially an exercise in some kind of induction. It is also statistical. The idea is that more independent runs of the experiment can establish results with greater confidence. In the claims of rigor of testability and replication, however, lies an implicit assumption of induction, that after establishing one case and then a few others, one can make a generalized claim. Most people are familiar with a simple form of induction from middle or high school, that for a proposition P to prove P(k) holds for all k >e; 0, it is sufficient to prove P(0) and P(k) =< P(k+1). That is a special case of a more general form of induction called well-founded (Noetherian) induction which is derived from set theory. Well-founded induction is the basis of much of programming languages theory. In PL theory, one uses a special form called structural induction. In any case, precise mathematical induction only works under certain assumptions. In particular, the universe one is inducting over must be a well-founded partially ordered set, meaning that there must be a well-defined least element in the set and that other elements must be comparable. The scientific method as put forth in textbooks is making the implicit claim that for a hypothesis P, if an experiment establishes P(0), P(1), and so forth, then P(k) holds for all k. One interesting way to explain this is with the so-called Raven Paradox. To establish the hypothesis that "All ravens are black", we can show this by observing ravens. Each raven we observe to be black adds to the evidence. Moreover, the contrapositive of the hypothesis must also hold, namely, "All non-black things are non-ravens." Thus, for each non-black thing we observe to be a non-raven, that also adds to our evidence. The observation about the contrapositive is the paradox part of things because it seems very counter-intuitive to be establishing hypotheses about ravens by observing non-ravens. However, the resolution of the paradox lies in a Bayesian argument that different kinds of observations add to our confidence in the conclusion differently.

My other reservation about the scientific method as it is popularly understood is that it is very much an ideal. Scientists are human after all. Specifically, the major tenant of the scientific method is replication. Supposedly claims and results will be validated by independent parties following a well documented experimental method to replicate the said results. Unfortunately, this ideal is very far from reality. The reality is that science research is also a slave to economics. To varying degrees, what research gets done depends on what is funded. It is difficult to get funding for merely replicating results. Moreover, a scientist does not advance his or her career by replicating because replication does not normally produce publishable results. Simply put, scientific results are seldom replicated to the extent that they should be. Consequently, many of the results can still be subject to a lot of interpretation.

What we need is a new kind of epistemology, a kind of behavioral epistemology to do for science and the philosophy of science what behavioral economics did for economics. In the very least, the history of science should be more widely taught. We take a lot of scientific principles and models for granted today, but scientific theories don't become popular and printed in all the textbooks overnight.

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.