Archive for May, 2013

What, Exactly, Is Probability?

“Probability is the bane of the age,” said Moreland, now warming up. “Every Tom, Dick, and Harry thinks he knows what is probable. The fact is most people have not the smallest idea what is going on round them. Their conclusions about life are based on utterly irrelevant – and usually inaccurate – premises.”

Anthony Powell, “Casanova’s Chinese Restaurant” in 2nd Movement in A Dance to the Music of Time, University of Chicago Press, 1995

Because many events can’t be predicted with total certainty, often the best we can do is say what the probability is that an event will occur – that is, how likely it is to happen. The probability that a particular event (or set of events) will occur is expressed on a linear scale from 0 (impossibility) to 1 (certainty), or as a percentage between 0 and 100%.

The analysis of events governed by probability is called statistics, a branch of mathematics that studies the possible outcomes of given events together with their relative likelihoods and distributions. It is one of the last major areas of mathematics to be developed, with its beginnings usually dated to correspondence between the mathematicians Blaise Pascal and Pierre de Fermat in the 1650′s concerning certain problems that arose from gambling.

Chevalier de Méré, a French nobleman with an interest in gaming and gambling questions, called Pascal’s attention to an apparent contradiction concerning a popular dice game that consisted in throwing a pair of dice 24 times. The problem was to decide whether or not to bet even money on the occurrence of at least one “double six” during the 24 throws. A seemingly well-established gambling rule led de Méré to believe that betting on a double six in 24 throws would be profitable, but his own calculations indicated just the opposite. This problem (as well as others posed by de Méré) led to the correspondence in which the fundamental principles of probability theory were formulated for the first time.

Statistics is routinely used in in every social and natural science. It is making inroads in law and in the humanities. It has been so successful as a discipline that most research is not regarded as legitimate without it. It’s also used in a wide variety of practical tasks. Physicians rely on computer programs that use probabilistic methods to interpret the results of some medical tests. Construction workers use a chart based on probability theory when mixing the concrete for the foundation of buildings, and tax assessors use a statistical package to decide how much the house is worth.

While there a number of forms of statistical analysis, the two dominant forms are Frequentist and Bayesian.

Bayesian analysis is the older form, and focuses on P(H|D) – the probability (P) of the hypothesis (H), given the data (D). This approach treats the data as fixed (these are the only data you have) and hypotheses as random (the hypothesis might be true or false, with some probability between 0 and 1). This approach is called Bayesian because it uses Bayes’ Theorem to calculate P(H|D).

The conceptual framework for Bayes’ Theorem was developed by the Reverend Thomas Bayes), and published posthumously in 1764. It was perfected and advanced by French physicist Pierre Simon Laplace, who gave it its modern mathematical form and scientific application. Bayes’ theorem has a 250-year history, and the method of inverse probability that was developed from it dominated statistical thinking into the twentieth century.

For the Bayesian:
• Probability is subjective – a measurement of the degree of belief that an event will occur – and can be applied to single events based on degree of confidence or beliefs. For example, Bayesian can refer to tomorrow’s weather as having a 50% chance of rain.
• Parameters are random variables that have a given distribution, and other probability statements can be made about them.
• Probability has a distribution over the parameters, and point estimates are usually done by either taking the mode or the mean of the distribution.

A Bayesian basically says, “I don’t know how the world is. All I have to go on is finite data. So I’ll use statistics to infer something from those data about how probable different possible states of the world are.”

Frequentist (sometimes called “a posteriori”, “empirical”, or “classical”) analysis focuses on P(D|H), the probability (P) of the data (D), given the hypothesis (H). That is, this approach treats data as random (if you repeated the study, the data might come out differently), and hypotheses as fixed (the hypothesis is either true or false, and so has a probability of either 1 or 0, you just don’t know for sure which it is). This approach is called frequentist because it’s concerned with the frequency with which one expects to observe the data, given some hypothesis about the world.

Frequentist statistical analysis is associated with Sir Ronald Fisher (who created the null hypothesis and p-values as evidence against the null), Jerzy Neyman (who was the first to introduce the modern concept of a confidence interval in hypothesis testing) and Egon Pearson (who with Neyman developed the concept of Type I and II errors, power, alternative hypotheses, and deciding to reject or not reject based on an alpha level). They use the relative frequency concept – you must perform one experiment lots of times and measure the proportion where you get a positive result.

For the Frequentist:
• Probability is objective and refers to the limit of an event’s relative frequency in a large number of trials. For example, a coin with a 50% probability of heads will turn up heads 50% of the time.
• Parameters are all fixed and unknown constants.
• Any statistical process only has interpretations based on limited frequencies. For example, a 95% confidence interval of a given parameter will contain the true value of the parameter 95% of the time.
• Referring to tomorrow’s weather as having a 50% chance of rain would not make sense to a Frequentist because tomorrow is just one unique event, and cannot be referred to as a relative frequency in a large number of trials. But they could say that 70% of days in April are rainy in Seattle.

A Frequentist basically says, “The world is a certain way, but I don’t know how it is. Further, I can’t necessarily tell how the world is just by collecting data, because data are always finite and noisy. So I’ll use statistics to line up the alternative possibilities, and see which ones the data more or less rule out.”

Frequentist and Bayesian approaches represent deeply conflicting approaches with deeply conflicting goals. Perhaps the most important conflict has to do with alternative interpretations of what “probability” means. These alternative interpretations arise because it often doesn’t make sense to talk about possible states of the world. For instance, there’s either life on Mars, or there’s not.

We don’t know for sure which it is, but we can say with certainty that it’s one or the other. So if you insist on putting a number on the probability of life on Mars (i.e. the probability that the hypothesis “There is life on Mars” is true), you are forced to drop the Frequentist interpretation of probability. A Frequentist interprets the word “probability” as meaning “the frequency with which something would happen in a lengthy series of trials”.

The Bayesian interprets the word “probability” as “subjective degree of belief” – the probability that you (personally) attach to a hypothesis is a measure of how strongly you (personally) believe that hypothesis. So a Frequentist would never say “There’s probably not life on Mars”, unless they were speaking loosely and using that phrase as shorthand for “The data are inconsistent with the hypothesis of life on Mars”. But the Bayesian would say “There’s probably not life on Mars”, not as a loose way of speaking about Mars, but as a very literal and precise way of speaking about their beliefs about Mars. A lot of the choice between Frequentist and Bayesian statistics comes down to whether you think science should comprise statements about the world, or statements about our beliefs.

Let’s look at the simple task of flipping a coin. The flip of a fair coin has no memory, or as mathematicians would say, each flip is independent. Even if by chance the coin comes up heads ten times in a row, the probability of getting heads or tails on the next flip is precisely equal. You may believe that a coin that, because a flipped coin has come up heads ten times in a row, that “tails is way overdue”, but the coin doesn’t know and doesn’t care about the last ten flips; the next flip is just as likely to be the eleventh head in a row as the tail that breaks the streak. The probability that the flip of a fair coin will come up heads or tails, then, is 50%.

But what, exactly, do we mean when we say that the probability is 50%? A Frequentist would say that if the probability of landing or either side is 50%, this means that if we were to repeat the experiment of flipping the coin a large number of times, we would expect to see approximately the same number of heads as tails. That is, the ratio of heads to tails will approach 1:1 as we flip the coin more and more times.

In contrast, a Bayesian would say that probability is a very personal opinion. What probability of 50% means to you is different from what it might mean to me. If pressed to place a bet on the outcome of flipping a single coin, you would just as well guess heads or tails. More generally, if you were to bet on the flip of a coin and was told that the probability of either side coming up was 50%, and the rewards for guessing correctly on any outcome are equal, then it would make no difference to you what side of the coin you bet on.

Both approaches are addressing the same fundamental problem (what are the odds that flipping a coin will result in it landing heads up), but attack the problem in reverse orders (the probability of getting data, given a model, versus probability of a model, given some data). It’s quite common to get the same basic result out of both methods, but many will argue that the Bayesian approach more closely relates to the fundamental problem in science (we have some data, and we want to infer the most likely truth.)

So, which approach is best? The Frequentist position would seem to be the answer. In our coin-flipping example, the probability of a fair coin landing heads is 50% because it lands heads half the time. Defining probability in terms of frequency seems to be the empirical thing to do. After all, frequency is “real”. It isn’t metaphysical, like “degree of certainty,” or “degree of warranted belief.” You can go out and observe it.

However, the Frequentist position also has some significant problems. First, it requires the long run relative frequency interpretation of probability – that is, the limiting frequency with which that outcome appears in a long series of similar events. Dice, coins and shuffled playing cards can be used to generate random variables; therefore, they have a frequency distribution, and the frequency definition of probability theory can be used. Unfortunately, the frequency interpretation can only be used in cases such as these. Another problem is that almost all prior information is ignored, and it doesn’t allow you to incorporate what you already know. Even more seriously, a hypothesis that may be true may be rejected because it hasn’t predicted observable results that have not occurred.

But the Bayesian position has its own set of problems. Bayesian calculations almost invariably require integrations over uncertain parameters, making them computationally difficult. Second, Bayesian methods require specifying prior probability distributions, which are often themselves unknown. Bayesian analyses generally assume so-called “uninformative” (often uniform) priors in such cases. But such assumptions may or may not be valid, and more importantly, it may not be possible to determine their validity with any degree of certainty.

Finally, though Bayes’ theorem is trivially true for random variables X and Y, it’s not clear that parameters or hypotheses should be treated as random variables. It’s accepted that you can talk about the probability of observed data given a model – the frequency with which you would obtain those data in the limit of infinite trials. But if you talk about the “probability”’ of a one-time, non-repeatable event that is either true or false, there is no frequency interpretation.

While both approaches have their (often rabid) proponents, I would argue that the approach you take depends on the question (or questions) you’re asking. Let’s take the hypothetical case of a patient you want to perform a test on.

You know the patient is either healthy (H) or sick (S). Once you perform the test, the result will either be Positive (+) or Negative (-). Now, let’s assume that if the patient is sick, they will always get a Positive result. We’ll call this the correct (C) result and say that if the patient is healthy, the test will be negative 95% of the time, but there will be some false positives. In other words, the probability of the test being Correct, for healthy people, is 95%. So the test is either 100% accurate or 95% accurate, depending on whether the patient is healthy or sick. Taken together, this means the test is at least 95% accurate.

These are the statements that would be made by a Frequentist. The statements are simple to understand and are demonstrably true. But what if we ask a more difficult, and arguably a more useful question – given the test result, what can you learn about the health of the patient?

If you get a negative test result, the patient is obviously healthy, as there are no false negatives. But what if the test is positive? Was the test positive because the patient was actually sick, or was it a false positive? This is where the frequentist and Bayesian diverge. Everybody will agree that this cannot be answered at the moment. The frequentist will refuse to answer. The Bayesian will be prepared to give you an answer, but you’ll have to give the Bayesian a prior first – i.e. tell it what proportion of the patients are sick.

If you are satisfied with statements such as “for healthy patients, the test is very accurate” and “for sick patients, the test is very accurate”, the Frequentist approach is best. But for the question “for those patients that got a positive test result, how accurate is the test?”, a Bayesian approach is required.


Ambaum, Maarten H. P., 2012. Frequentist vs Bayesian statistics—a non-statisticians view.

Bayarri, M.J. and Berge, J.O. The Interplay of Bayesian and Frequentist Analysis. Statist. Sci. Volume 19, Number 1 (2004), 58-80.

Fienberg, Stephen E., 2006. When Did Bayesian Inference Become Bayesian? Bayesian Analysis Volume 1, Number 1, pp. 1-40.

Gustafson, Paul and Greenland, Sander, 2009. Interval Estimation for Messy Observational Data. Statist. Sci. Volume 24, Number 3, 28–342.

Hald, Anders, 2003. A History of Probability and Statistics and Their Applications before 1750. Hoboken, NJ: Wiley-Interscience

Hampel, Frank, 1998. On the foundations of statistics: A frequentist approach, Research Report No. 85. Zurich, Switzerland: Seminar fur Statistik, Eidgenossische Technische Hochschule (ETH)

Samaniego, Francisco J., 2010. A Comparison of the Bayesian and Frequentist Approaches to Estimation. New York, NY: Springer

Shafer, Glenn, 1990. The Unity and Diversity of Probability. Statist. Sci. Volume 5, Number 4, 435-444.

Zabell , Sandy , 1989. R. A. Fisher on the History of Inverse Probability. Statist. Sci. Volume 4, Number 3, 247-256.

Follow the Rats Out

Last weekend I got lost twice, once going to and then coming from a local bookstore that I’ve been to several times. I only travel out of town in cases of rare necessity. My ability to get lost defies the assistance of MapQuest and the like.  The timing of my adventure couldn’t be better. It was the event that tipped the scales in favor of rats vs. the promised glowing fish from my last article.

I knew as soon as I saw the link to “A Sense of Where You Are” on Jay Lake ‘s Link Salad, that I had to explore the topic here. The article discusses two doctors, May-Britt and Evard Moser. The husband and wife team direct the Kavli Institute for Systems Neuroscience and Centre for the Biology of Memory  which is a neuroscience research center at the Norwegian University of Science and Technology. Under their direction, the center has become known for the discovery of grid cells. The cells help rats know where they are, remember where they’ve been, and understand where they are going.

“The scientific goal of the Kavli Institute for Systems Neuroscience is to advance our understanding of neural circuits and systems. By focusing on spatial representation and memory, the investigators hope to uncover general principles of neural network computation in the mammalian cortex.” (Wikipedia Kavli stub article)

My husband is very good at finding his way around our city and reaching destinations without getting lost. My parents are too. In fact, I couldn’t remember a single time that we got lost on road trips as a kid. I decided to call my mom and fact check my memory against her’s.

Mom told me that Carol is more like me. She has difficulty placing the city, county, and state on the map in her mind without one in front of her. I just hadn’t known Mom was her co-pilot for our road trips back when we were kids. Mom then reminded me of Kittery, Maine.

We’d been house hunting in the great state of Maine, and Kittery was one of few towns that a motel that allowed you to take dogs inside. We always stayed there a few times while going up and down the state. When Carol and I would go out foraging for local take out, we got lost. Often. I remember crossing a bridge that meant you were leaving Maine and going into New Hampshire. The roads were confusing to us, and somehow we kept making the same mistakes. It was scary at first but funny after a while. it turned into Groundhog Day – the you aren’t from around here version. I had a good laugh when Mom brought up the movie. That is exactly how it felt.

It got me thinking about how Bill Murray’s character learned the layout of town and the timing of the events of the day. Rats in these experiments become familiar with their mazes and will often return to where they found or stored food even with their sense of smell taken out of the equation. (Researchers apparently put food under the maze to mask where it might be in the maze.) It isn’t just rats; squirrels and birds also remember where they put their food. I have no link for the squirrels. I have a friend that could attest to this: something about flower pots and peanut shells. I am not sure I recommend that you ask her, though.

Rats also know how to get out of a flood. I’ve seen it in movies and read it in books countless times. In times of disaster, you follow the rats out. There is a test that demonstrates this, Morris water navigation task.

In reading about Spatial memory I began to understand some of the reasons behind my uncanny ability to get lost anywhere. The added layers of sensory input, echoes of previous trips that went to nearby locations (which blur landmarks, boundaries  and my  association to them), the distracting traffic and receiving imput from a passenger who is a little challenged herself with this at times (the overlay of routes she’s taken with others does this to her) all work together to confuse the Grid cells that create my Cognitive map. At least, this is the theory I am going with after the research I’ve been doing for this article.

Maybe with my brain sorting through data in unorganized, rapid succession I get gridlocked. No, that isn’t a scientific term, but maybe it is coming. Speaking of gridlock, some scientists have tested taxi cab drivers in virtual settings. Hello, The Matrix! They found out that these spacial memory experts are better at recall of this nature and worked to understand why. In another study, a virtual taxi driving game was used to further analyze this sense of direction while testing  to see if stimulation of this memory area in the hippocampus and the entorhinal cortex (EC), while learning, makes the memory stick. The entorhinal cortex is one of the first areas affected Alzheimer’s Disease causing patients to suffer the loss of spacial memory very early in the progression of the disease. This research, Memory Enhancement and Deep-Brain Stimulation of the Entorhinal Area, might tie into ways to counteract those effects.

Here are some random thoughts I had regarding this topic:

I wonder if my inability to accurately translate dance moves I see to ones produced by my body has any relation to my frequent adventures in getting lost. Spacial memory is also in charge of movement. Does this explain why I can’t dance?

Is the memorizing technique, Method of loci, more difficult to use for those of us with a poor sense of direction?

Are we more or less likely to notice when things are rearranged at home or work because it upsets our landmarks and boundaries that we rely so heavily upon even in these familiar places? I know it leaves me feeling off kilter.

Do grid cells, cognitive maps, and spacial memory tie into animal migration and navigation? Some animals are born with an understanding of where they should be headed when they get older. Elephants remember where their ancestors are buried. So are there genetic markers that lead the hippocampus to develop this understanding? How does this factor into their social learning?

Consider what might happen if scientist found a way to tinker with this in animals.  Imagine being able to create natural psychological boundaries for packs of wolves, the selling point being they would be kept off the farmland or ranches and kept in their sanctuaries. We have bears that keep coming into Greensboro. It is a little exciting and unexpected. Local wildlife authorities just tranquilized a bear that has been here twice now. He is tagged and they know this for certain. What if there was a way to steer him clear of this danger before someone decides he is just going to keep coming back and should be shot? Say a nanobot-sized thing could be put in the drinking water of a herd, and the nanobots could wind up in the brains of a herd of moose and steer them clear of say the busier highways or cities so that as they travelled they would face less peril.

In Stephen King’s Cell the not-zombies seem to flock and seem to migrate. I wonder if the heart of that fictionalized action could also be based in the hippocampus as the non-zombies still possess their motor skills.

We have GPS systems at our finger tips. Will this memory exercise become stale for us? Five generations out from now, will we only be able to travel with the aid of technology. Will we lose our sense of direction?

And just for fun because I enjoy this series and it is remotely related: Tattoo Typos, Senses, Posters and Bad Movies –  A Vlogbrothers YouTube Post


Into space, alone?

There’s no way that humans can head into space all by themselves. Even leaving aside the bacteria, fungi and arthropods that inhabit our bodies, and without which we won’t stay healthy for long, once we go in for large-scale space travel and exploration it’s going to be incredibly hard to keep insect pests and even small mammals from hitching a ride. (I don’t think any space programs so far have reported roaches or mice, but do you think they would?)

What about animals we intend to spend into space? Decades of animal research have resulted in dogs, cats, monkeys, chimps, and many, many lab mice in orbit. Also guinea pigs, frogs, fish, many species of insects… yes, even cockroaches, but only on purpose.

Those have all been for science, but science fiction at least has posited that people will want to have their familiar animal companions (or their science fictional equivalents).

We think that our pets are cute (and we might even be hard-wired to do so), but pets could also be good for us. There’s some evidence that animal companions can lower blood pressure, reduce anxiety, and improve immune system function. Those sound like good things, even in space (likely a stressful environment, don’t you think?).

fluffy kitten

Cats, ferrets and terriers would also be good at controlling the accidental animals, the rats and mice of our habitats. They’ve been bred for just that sort of work for centuries if not longer, as well as to live well in the company of humans. Or we could genetically engineer something new to fit our new lifestyles, instead of working through centuries of conventional breeding.

So what do you think? Would you take your pet into space, and why or why not? Will future creatures be just as cute as today’s pets? Will they have to work for a living, or is companionship enough?

So much science…

… so little time.

There’s so much nifty science out there that our small group of writers is utterly overwhelmed, and that’s why we need you.

That’s right, you too could be part of the SiMF crew. We’re looking for people who love science and fiction both, regardless of their formal qualifications in either, love to write, and want to share those enthusiasms with the world. SiMF is an all-volunteer outfit: we do it for love, not money.

We’re most interested in short essays about how current science topics are relevant to speculative fiction, and we’re not particularly interested in reviews. An ideal writer will be able to contribute something every couple of months, but we’re also willing to consider one-off guest posts.

To apply, please email sarah.goslee at gmail dot com with a brief description of your qualifications and why you’re interested in SiMF, and either a link to relevant online articles you’ve written or a sample of your work that would be appropriate for SiMF.