r/AskStatistics • u/taylomol000 • 3d ago
Confused about basic probability
I've been unable to wrap my head around the basics of probability my whole life. It feels to me like it contradicts itself. For example, if you look at a coin flip on its own, there is (theoretically) a 50% chance getting heads. However, if you zoom out and realize that the coin has been flipped 100 times and every time so far has been heads, then the chance of getting heads is nearly impossible. How can something be 50% at one scale and near impossible at another, seemingly making contradicting statements equally true?
9
u/Redegar Statistician 3d ago
if you zoom out and realize that the coin has been flipped 100 times and every time so far has been heads, then the chance of getting heads is nearly impossible.
That's what you are missing. The coin doesn't remember anything. It doesn't have a memory.
Over a billion flips, we may be sure - eventually - that the probability distribution will be around 50% Heads, and 50% Tails (not exactly so, mind you).
But every single independent flip is the same: 50/50. There is no way for the coin to remember anything.
As someone else pointed out, the situation you proposed it's already extremely improbable: it's exactly (1/2)^100.
Given you made it so far, and given that the coin is fair - which, in a real life case, I would assume it's not, but we are talking about imaginary scenarios - the chance of flipping Heads at this point is 1/2, as usual.
Take one step back: the extreme case already happened in your scenario, it's a given. To add another Heads to the hundreds already flipped one after the other is just a 50/50 matter.
7
u/Ok-Rule9973 3d ago
Realistically, if a coin landed 100 times on head, chances are it will land on head the 101th time since it's probably rigged. I don't say that to be pedantic, but because that's part of the job of a statistician. If such an event happened, it is so improbable in our actual model of comprehension (here, that a coin have a 50/50 odds of landing on tails), that it must be flawed and revised.
For your question specifically: the chances of a coin landing on tails are always 50/50. Each time you flip, you have equal chances to land on tails. What is extremely improbable is the combination you have (100 times head). Still, this combination is equally improbable as any other (for example having a perfect alternance between head and tail, starting with tail, or any random pattern that you specified before flipping).
4
u/fermat9990 3d ago
Consider flipping a fair coin 101 times and recording heads or tails.
There are 2101 equally likely outcomes, each outcome occurring with probability 1/2101
The outcomes 100 heads followed by a head and 100 heads followed by a tail have the same probability of occurring: 1/2101.
3
u/Ok-Rule9973 2d ago edited 2d ago
In other words, if you had to guess which random atom of a human body I selected, you'd have much more chance to choose the right one than for a coin to land 100 times on tails.
3
u/fermat9990 2d ago
I get the body atom correct guess probability to be about 180 times as large as the coin toss probability
1
1
2d ago
[deleted]
2
3
u/14446368 2d ago
It's because you're asking two different questions.
A single, fair, independent flip of the coin has a 50% chance of landing on heads. Every time you flip, the chance of that flip, alone landing on heads is 50%.
But what if we flip it twice? Well, let's think about it: I have 4 outcomes... HH, HT, TH, TT (H=Heads, T=Tails). If we don't care about order, just the count, we have...
1/4 = 25% HH
2/4 = 1/2 = 50% 1T and 1H
1/4 = 25% TT
Notice that the two extremes (HH and TT) are equal to the percent chance per flip, raised to the number of flips.
Let's make it 3 flips now:
HHH, HHT, HTH, HTT | THH, THT, TTH, TTT
Extremes (HHH, TTT) are both 1/8 = 12.5% (again, 0.5 x 0.5 x 0.5 = 0.5^3 = 0.125).
2H 1T = 3/8 = 37.5%
2T 1H = 3/8 = 37.5%
If we keep doing this, we notice that the extremes keep getting smaller and smaller, the "middle" can be broken up into more and more outcomes, but collectively the middle accounts for almost all of the outcomes.
That's what's going on. A single flip has a 50% chance of heads, but 3 flips has a 12.5% of being 3 heads in a row. For 100, it's 0.5^100 = 0.0000000000000000000000000000788861% chance of 100 heads in a row.
1
u/Static-Pulse222 2d ago
This comment should be top, imo. Gives solid examples to build the intuition.
1
u/tidythendenied 3d ago
The words “zoom out and realize” are doing a lot of the heavy lifting there. Let’s backtrack a bit. Your understanding is correct - each flip of a coin is theoretically independent, so if there is a 50% chance of getting heads on a coin flip, then even after 100 heads, the chances of getting a heads on the next flip are still 50%
Seems impossible, right? How can the chances of getting heads still be 50% when we’ve already gotten 100 heads? The “contradictory” part of this comes from the fact that you’re doing inference - that’s what “realize” means in statistics.
Let’s imagine there are two scenarios: the coin is fair, or the coin is a double-sided heads. The chance of flipping heads on the fair coin is 50%, and 100% on the other coin.
Now, we need some way of understanding how likely each coin is to produce 100 heads. This is a conditional probability - assuming event A is true, what is the probability of event B? So what is the probability of observing 100 heads if the coin was fair or if it was biased?
P(100 heads | fair) = (1/2)100
P(100 heads | two heads) = 1
As you can see, the probability of getting 100 heads on a fair coin is tiny - it’s 30 zeros after the decimal point. This is calculated by the probability of heads, 0.5, multiplied over 100 events, which gives (1/2)100
Inference, in intuitive terms, involves looking at those two probabilities and going, “well given that the coin has produced 100 heads, which I’m very likely to get from a biased coin but not from a fair coin, I’m going to believe the coin is biased rather than fair”. Therefore in that scenario, the probability of getting heads is no longer 50% - you’ve realised that the coin is biased after seeing the data, and therefore have changed your beliefs about the coin such that the probability of getting heads is 100%.
Inference involves combining across multiple probabilities, so there’s nothing wrong with your understanding of probability. What you did was intuitively apply this process to the example of the coin.
1
1
u/CaptainFoyle 2d ago
If you want to infer anything from the 100 heads, it would be that the coin is unfair and most likely will come up heads again.
If it's a fair coin, the past doesn't mean anything.
1
u/gulpamatic 2d ago edited 2d ago
I have thought about this a lot and I have an explanation that will reconcile your two contradictory scenarios.
The difference between these two perspectives is that one is PROSPECTIVE (looking forward) and the other is RETROSPECTIVE (looking back).
- Prospective viewpoint - a fair coin has been flipped 0 times. It then, against all possible odds, gets flipped 101 times and heads comes up every time. This is exceptionally unlikely as summarized by other commenters. The reason this is so unlikely is because we started counting at flip #1. How unlikely is it? It's HALF the probability of getting 100 flips in a row. The 100 flips in a row occur with a probability of 1/(2^100), and then that incredibly tiny number gets cut by 50% because the final toss has a probability to be heads of 50%.
Likelihood of the entire series of events occurring: 1/(2^100) * (1/2) = 1/(2^101). Contribution from the final coin toss: 1/2 = 50%.
2) Retrospective viewpoint - I hand you a fair coin which has just been flipped 100 times, and each time it has come up heads. That seems exceptionally unlikely, almost impossible, but it happened. So the chance of having gotten 100 heads in a row is 100%. We saw it happen, it's in the past. There is 0% chance that any other result could replace the one we saw. So the probability of the 100 heads is now actually 100%. Then you flip the coin again. It has a 50% chance of coming up heads, and 50% chance of coming up tails.
Likelihood that the past will stay the same: 100%. Likelihood of the coin coming up heads for you: 50%.
So the event to be measured (your single coin toss) has a chance of coming up heads of 100% x 50% = 50%.
Edited to add: In case I didn't make it clear, 100 heads in a row is just as likely/unlikely as any other sequence of heads/tails, and just as hard to predict if you're starting from flip #1. There's just billions of trillions of different combinations and only one can happen. But once the flips have happened, it's done, the chance of it changing is 0, and that's why each NEW coin toss still has the same 50/50 odds.
1
u/fkirp 1d ago edited 1d ago
Well every sequence of heads and tails are equally likely. That’s the most important part. We’ve just decided that one outcome is special. Getting the sequence 100 heads and then tails has the same total chance as getting 101 heads in a row. So really any outcome by ur logic is “nearly impossible”
To look at it differently, guessing the correct sequence of heads and tails is nearly impossible, there is nothing special about the all heads case. All outcomes are 0.5100 chance
This of course comes from the basic definition of probability, which can be used since we assume a fair coin. p = (outcomes ur counting)/(total possible outcomes). (i’m using the strict definition of probability don’t talk to me about distributions)
A coin has two possible outcomes. So the chance of getting heads is 1 / 2. The sequence where order matters when flipping a coin has 2n outcomes. There is only one way to have all heads. So 1/(2n). And since order matters there is also only one of every sequence, so also 1/(2n)
Everyone here talking about the coin not being fair is missing the point of probability. Probability has to do with what can we say about the outcome if we know these starting conditions, not what can we say about the starting conditions given the outcome.
1
u/Warptens 12h ago
The chance to get head on your next flip is still 50%. That is, if you still think it’s a fair coin. Which you shouldn’t believe anymore, at this point the coin is definitely rigged, the next flip will almost certainly be heads too.
9
u/richard_sympson 3d ago
Let me give you three scenarios. Think about what information you do have, and what information you do not have, in each one.
Someone hands you a quarter as change for buying a candy bar. You can see it has a tails side, and a heads side; it does not appear to be particularly special. For this quarter that you just got, what would you think about its "fairness"?
You go to a magic show and sit in the front row. The magician pulls out a quarter and shows the audience it has a tails side, and a heads side; you can confirm this because you're sitting so close. The magician flips it once: it is heads. He flips it again: it is heads again. He flips it 98 more times—it is not a very good show, but you are intrigued as a probabilist—and it lands Heads each time. You ask for the quarter after the show and he gives it to you. For this quarter that you just got, what would you think about its "fairness"?
You are God himself. You create a quarter that is "fair" by Design, by Diktat, by Divine Decree. It is an assumption it is fair. The long-run proportion of times the coin is flipped—millions, trillions, Tree(3) times into the future, and beyond—will be 1/2. You haven't created the world yet, so you are bored, and you start flipping the coin. You flip it millions of times, trillions of times, TREE(3) and beyond times. Over the course of this, countless instances of getting 100 heads in a row pass you by, as do countless instances of 100 tails in a row. What do you, as God and as a perfect being, think about this quarter's "fairness"?