|
Why it pays to understand probability
posted by GJ on September 30, 2009 @ 7:14AM
So, you get a letter from the company you applied for insurance from, telling you that you are not eligible for the insurance you've applied for because you failed a blood test. They don't get into specifics, of course, so you hop over to you doctor and he recommends an HIV test, since that's the most likely thing an otherwise healthy middle-aged man might be failing in terms of a blood test. A couple of days later, your doctor calls. "I'm sorry," he says, "the test came back positive. This test is almost never wrong--only a 1 in 1,000 show an error. I'm really sorry." You're crushed--and confused. You're not promiscuous, happily married (and heterosexual, too) for 20+ years. Maybe your wife was cheating on you? You don't abuse intravenous drugs, and didn't have any blood transfusions back in the early 1980s, either. Any way you cut it, your life is in shambles, because even with medication an HIV diagnosis is a ten year death sentence, all but guaranteed. Sure, getting a diagnosis like Patrick Swayze's pancreatic cancer is far worse, but this isn't something most folks will be jumping for joy over. But wait. The doctor is schooled in medicine, not probability analysis. Maybe he's wrong? Let's create the sample space--that is, the realm of all possible conditions. There are four: - True positive: test was positive for HIV and you actually have HIV
- False positive: test was positive but you don't have HIV
- True negative: test was negative and you don't have HIV
- False negative: test was negative yet you do actually have HIV
Assume that the failure rate is the false positive rate--and that the false negative rate is very near zero. This is the case for most tests like this. The failure rate quoted is almost always the false positive incident rate. So, in a population of 10,000 adults like you, that is, male, hetero, single parter, non-drug users, the HIV infection rate is 1 in 10,000. In our test population, then, 1 person will be a true positive. However, with a test failure rate of 1 in 1,000, there will be 10 others in this population that turn out to be false positives, while the far greater majority remain true negatives. Time to whittle down the sample space, since you're in the group where the test came back positive. What are the odds that you really have HIV? Not very good, as it turns out--only 1 in 11. But your doctor told you that the chances were 999/1,000! That's because he mistakenly applied the false positive rate to the problem as a whole--something we humans do far too often. Now, the probability changes if you are in fact homosexual. See, the rate for infection among homosexuals is more like 1 in 100. So, back to our population: instead of one 1 true positive, we have 100 true positives to go with our 10 false positives. So, if you're a gay male, your chances of being a true positive are instead 10 in 11, even though the test has the same failure rate no matter who takes it. Even so, those odds that your still not HIV are an order of magnitude better that 99.9% likely, so your doctor still would have been wrong, just less wrong. Pretty cool, huh? The author of the book I'm reading had this happen to him. This is a fascinating book, and I strongly recommend you pick it up if you found my parable the least bit interesting. The book is The Drunkard's Walk, by Leonard Mlodinow. I'll bring it with me to the wedding, if someone wants to borrow my copy.
| Tags: math, books
|
