The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare continue to
offer high quality educational resources for free.
To make a donation or view additional materials from
hundreds of MIT courses, visit MIT OpenCourseWare at
ocw.mit.edu.
PROFESSOR: OK, so welcome to 6.041/6.431, the class on
probability models and the like.
I'm John Tsitsiklis.
I will be teaching this class, and I'm looking forward to
this being an enjoyable and also useful experience.
We have a fair amount of staff involved in this course, your
recitation instructors and also a bunch of TAs, but I
want to single out our head TA, Uzoma, who is the key
person in this class.
Everything has to go through him.
If he doesn't know in which recitation section you are,
then simply you do not exist, so keep that in mind.
All right.
So we want to jump right into the subject, but I'm going to
take just a few minutes to talk about a few
administrative details and how the course is run.
So we're going to have lectures twice a week and I'm
going to use old fashioned transparencies.
Now, you get copies of these slides with plenty of space
for you to keep notes on them.
A useful way of making good use of the slides is to use
them as a sort of mnemonic summary of
what happens in lecture.
Not everything that I'm going to say is, of course, on the
slides, but by looking them you get the sense of what's
happening right now.
And it may be a good idea to review them before you go to
recitation.
So what happens in recitation?
In recitation, your recitation instructor is going to maybe
review some of the theory and then solve some
problems for you.
And then you have tutorials where you meet in very small
groups together with your TA.
And what happens in tutorials is that you actually do the
problem solving with the help of your TA and the help of
your classmates in your tutorial section.
Now probability is a tricky subject.
You may be reading the text, listening to lectures,
everything makes perfect sense, and so on, but until
you actually sit down and try to solve problems, you don't
quite appreciate the subtleties and the
difficulties that are involved.
So problem solving is a key part of this class.
And tutorials are extremely useful just for this reason
because that's where you actually get the practice of
solving problems on your own, as opposed to seeing someone
else who's solving them for you.
OK but, mechanics, a key part of what's going to happen
today is that you will turn in your schedule forms that are
at the end of the handout that you have in your hands.
Then, the TAs will be working frantically through the night,
and they're going to be producing a list of who goes
into what section.
And when that happens, any person in this class, with
probability 90%, is going to be happy with their assignment
and, with probability 10%, they're going to be unhappy.
Now, unhappy people have an option, though.
You can resubmit your form together with your full
schedule and constraints, give it back to the head TA, who
will then do some further juggling and reassign people,
and after that happens, 90% of those unhappy people will
become happy.
And 10% of them will be less unhappy.
OK.
So what's the probability that a random person is going to be
unhappy at the end of this process?
It's 1%.
Excellent.
Good.
Maybe you don't need this class.
OK, so 1%.
We have about 100 people in this class, so there's going
to be about one unhappy person.
I mean, anywhere you look in life, in any group you look
at, there's always one unhappy person, right?
So, what can we do about it?
All right.
Another important part about mechanics is to read carefully
the statement that we have about collaboration, academic
honesty, and all that.
You're encouraged, it's a very good idea to
work with other students.
You can consult sources that are out there, but when you
sit down and write your solutions you have to do that
by setting things aside and just write them on your own.
You cannot copy something that somebody else
has given to you.
One reason is that we're not going to like it when it
happens, and then another reason is that you're not
going to do yourself any favor.
Really the only way to do well in this class is to get a lot
of practice by solving problems yourselves.
So if you don't do that on your own, then when quiz and
exam time comes, things are going to be difficult.
So, as I mentioned here, we're going to have recitation
sections, that some of them are for 6.041 students, some
are for 6.431 students, the graduate section of the class.
Now undergraduates can sit in the
graduate recitation sections.
What's going to happen there is that things may be just a
little faster and you may be covering a problem that's a
little more advanced and is not covered in
the undergrad sections.
But if you sit in the graduate section, and you're an
undergraduate, you're still just responsible for the
undergraduate material.
That is, you can just do the undergraduate work in the
class, but maybe be exposed at the different section.
OK.
A few words about the style of this class.
We want to focus on basic ideas and concepts.
There's going to be lots of formulas, but what we try to
do in this class is to actually have you understand
what those formulas mean.
And, in a year from now when almost all of the formulas
have been wiped out from your memory, you still have the
basic concepts.
You can understand them, so when you look things up again,
they will still make sense.
It's not the plug and chug kind of class where you're
given a list of formulas, you're given numbers, and you
plug in and you get answers.
The really hard part is usually to choose which
formulas you're going to use.
You need judgment, you need intuition.
Lots of probability problems, at least the interesting ones,
often have lots of different solutions.
Some are extremely long, some are extremely short.
The extremely short ones usually involve some kind of
deeper understanding of what's going on so that you can pick
a shortcut and use it.
And hopefully you are going to develop this
skill during this class.
Now, I could spend a lot of time in this lecture talking
about why the subject is important.
I'll keep it short because I think it's almost obvious.
Anything that happens in life is uncertain.
There's uncertainty anywhere, so whatever you try to do, you
need to have some way of dealing or thinking about this
uncertainty.
And the way to do that in a systematic way is by using the
models that are given to us by probability theory.
So if you're an engineer and you're dealing with a
communication system or signal processing, basically you're
facing a fight against noise.
Noise is random, is uncertain.
How do you model it?
How do you deal with it?
If you're a manager, I guess you're dealing with customer
demand, which is, of course, random.
Or you're dealing with the stock market, which is
definitely random.
Or you play the casino, which is, again, random, and so on.
And the same goes for pretty much any other field that you
can think of.
But, independent of which field you're coming from, the
basic concepts and tools are really all the same.
So you may see in bookstores that there are books,
probability for scientists, probability for engineers,
probability for social scientists, probability for
astrologists.
Well, what all those books have inside them is exactly
the same models, the same equations, the same problems.
They just make them somewhat different word problems.
The basic concepts are just one and the same, and we'll
take this as an excuse for not going too much into specific
domain applications.
We will have problems and examples that are motivated,
in some loose sense, from real world situations.
But we're not really trying in this class to develop the
skills for domain-specific problems.
Rather, we're going to try to stick to general understanding
of the subject.
OK.
So the next slide, of which you do have in your handout,
gives you a few more details about the class.
Maybe one thing to comment here is that you do need to
read the text.
And with calculus books, perhaps you can live with a
just a two page summary of all of the interesting formulas in
calculus, and you can get by just with those formulas.
But here, because we want to develop concepts and
intuition, actually reading words, as opposed to just
browsing through equations, does make a difference.
In the beginning, the class is kind of easy.
When we deal with discrete probability, that's the
material until our first quiz, and some of you may get by
without being too systematic about following the material.
But it does get substantially harder afterwards.
And I would keep restating that you do have to read the
text to really understand the material.
OK.
So now we can start with the real part of the lecture.
Let us set the goals for today.
So probability, or probability theory, is a framework for
dealing with uncertainty, for dealing with situations in
which we have some kind of randomness.
So what we want to do is, by the end of today's lecture, to
give you anything that you need to know how to set up
what does it take to set up a probabilistic model.
And what are the basic rules of the game for dealing with
probabilistic models?
So, by the end of this lecture, you will have
essentially recovered half of this
semester's tuition, right?
So we're going to talk about probabilistic
models in more detail--
the sample space, which is basically a description of all
the things that may happen during a random experiment,
and the probability law, which describes our beliefs about
which outcomes are more likely to occur
compared to other outcomes.
Probability laws have to obey certain properties that we
call the axioms of probability.
So the main part of today's lecture is to describe those
axioms, which are the rules of the game, and consider a few
really trivial examples.
OK, so let's start with our agenda.
The first piece in a probabilistic model is a
description of the sample space of an experiment.
So we do an experiment, and by experiment we just mean that
just something happens out there.
And that something that happens, it could be flipping
a coin, or it could be rolling a dice, or it could be doing
something in a card game.
So we fix a particular experiment.
And we come up with a list of all the possible things that
may happen during this experiment.
So we write down a list of all the possible outcomes.
So here's a list of all the possible outcomes of the
experiment.
I use the word "list," but, if you want to be a little more
formal, it's better to think of that list as a set.
So we have a set.
That set is our sample space.
And it's a set whose elements are the possible outcomes of
the experiment.
So, for example, if you're dealing with flipping a coin,
your sample space would be heads, this is one outcome,
tails is one outcome.
And this set, which has two elements, is the sample space
of the experiment.
OK.
What do we need to think about when we're setting up the
sample space?
First, the list should be mutually exclusive,
collectively exhaustive.
What does that mean?
Collectively exhaustive means that, no matter what happens
in the experiment, you're going to get one of the
outcomes inside here.
So you have not forgotten any of the possibilities of what
may happen in the experiment.
Mutually exclusive means that if this happens, then that
cannot happen.
So at the end of the experiment, you should be able
to point out to me just one, exactly one, of these outcomes
and say, this is the outcome that happened.
OK.
So these are sort of basic requirements.
There's another requirement which is a little more loose.
When you set up your sample space, sometimes you do have
some freedom about the details of how you're going to
describe it.
And the question is, how much detail are
you going to include?
So let's take this coin flipping experiment and think
of the following sample space.
One possible outcome is heads, a second possible outcome is
tails and it's raining, and the third possible outcome is
tails and it's not raining.
So this is another possible sample space for the
experiment where I flip a coin just once.
It's a legitimate one.
These three possibilities are mutually exclusive and
collectively exhaustive.
Which one is the right sample space?
Is it this one or that one?
Well, if you think that my coin flipping inside this room
is completely unrelated to the weather outside, then you're
going to stick with this sample space.
If, on the other hand, you have some superstitious belief
that maybe rain has an effect on my coins, you might work
with the sample space of this kind.
So you probably wouldn't do that, but it's a legitimate
option, strictly speaking.
Now this example is a little bit on the frivolous side, but
the issue that comes up here is a basic one that shows up
anywhere in science and engineering.
Whenever you're dealing with a model or with a situation,
there are zillions of details in that situation.
And when you come up with a model, you choose some of
those details that you keep in your model, and some that you
say, well, these are irrelevant.
Or maybe there are small effects, I can neglect them,
and you keep them outside your model.
So when you go to the real world, there's definitely an
element of art and some judgment that you need to do
in order to set up an appropriate sample space.
So, an easy example now.
So of course, the elementary examples are
coins, cards, and dice.
So let's deal with dice.
But to keep the diagram small, instead of a six-sided die,
we're going to think about the die that only has four faces.
So you can do that with a tetrahedron,
doesn't really matter.
Basically, it's a die that when you roll it, you get a
result which is one, two, three or four.
However, the experiment that I'm going to think about will
consist of two rolls of a dice.
A crucial point here--
I'm rolling the die twice, but I'm thinking of this as just
one experiment, not two different experiments, not a
repetition twice of the same experiment.
So it's one big experiment.
During that big experiment various things could happen,
such as I'm rolling the die once, and then I'm
rolling the die twice.
OK.
So what's the sample space for that experiment?
Well, the sample space consists of
the possible outcomes.
One possible outcome is that your first roll resulted in
two and the second roll resulted in three.
In which case, the outcome that you get is this one, a
two followed by three.
This is one possible outcome.
The way I'm describing things, this outcome is to be
distinguished from this outcome here, where a three is
followed by two.
If you're playing backgammon, it doesn't matter which one of
the two happened.
But if you're dealing with a probabilistic model that you
want to keep track of everything that happens in
this composite experiment, there are good reasons for
distinguishing between these two outcomes.
I mean, when this happens, it's definitely something
different from that happening.
A two followed by a three is different from a three
followed by a two.
So this is the correct sample space for this experiment
where we roll the die twice.
It has a total of 16 elements and it's, of
course, a finite set.
Sometimes, instead of describing sample spaces in
terms of lists, or sets, or diagrams of this kind, it's
useful to describe the experiment in
some sequential way.
Whenever you have an experiment that consists of
multiple stages, it might be useful, at least visually, to
give a diagram that shows you how those stages evolve.
And that's what we do by using a sequential description or a
tree-based description by drawing a tree of the possible
evolutions during our experiment.
So in this tree, I'm thinking of a first stage in which I
roll the first die, and there are four possible results,
one, two, three and four.and 4.
And, given what happened, let's say in the first roll,
suppose I got a one.
Then I'm rolling the second dice, and there are four
possibilities for what may happen to the second die.
And the possible results are one, tow,
three and four again.
So what's the relation between the two diagrams?
Well, for example, the outcome two followed by three
corresponds to this path on the tree.
So this path corresponds to two followed by a three.
Any path is associated to a particular outcome, any
outcome is associated to a particular path.
And, instead of paths, you may want to think in terms of the
leaves of this diagram.
Same thing, think of each one of the leaves as being one
possible outcome.
And of course we have 16 outcomes here, we have 16
outcomes here.
Maybe you noticed the subtlety that I used in my language.
I said I rolled the first dice and the result
that I get is a two.
I didn't use the word "outcome." I want to reserve
the word "outcome" to mean the overall outcome at the end of
the overall experiment.
So "2, 3" is the outcome of the experiment.
The experiment consisted of stages.
Two was the result in the first stage, three was the
result in the second stage.
You put all those results together, and
you get your outcome.
OK, perhaps we are splitting hairs here, but it's useful to
keep the concepts right.
What's special about this example is that, besides being
trivial, it has a sample space which is finite.
There's 16 possible total outcomes.
Not every experiment has a finite sample space.
Here's an experiment in which the sample space is infinite.
So you are playing darts and the target is this square.
And you're perfect at that game, so you're sure that your
darts will always fall inside the square.
So, but where exactly your dart would fall inside that
square, that itself is random.
We don't know what it's going to be.
It's uncertain.
So all the possible points inside the square are possible
outcomes of the experiment.
So a typical outcome of the experiment is going to a pair
of numbers, x,y, where x and y are real numbers
between zero and one.
Now there's infinitely many real numbers, there's
infinitely many points in the square, so this is an example
in which our sample space is an infinite set.
OK, so we're going to revisit this example a little later.
So these are two examples of what the sample space might be
in simple experiments.
Now, the more important order of business is now to look at
those possible outcomes and to make some statements about
their relative likelihoods.
Which outcome is more likely to occur
compared to the others?
And the way we do this is by assigning
probabilities to the outcomes.
Well, not exactly.
Suppose that all you were to do was to assign probabilities
to individual outcomes.
If you go back to this example, and you consider one
particular outcome-- let's say this point--
what would be the probability that you hit exactly this
point to infinite precision?
Intuitively, that probability would be zero.
So any individual point in this diagram in any reasonable
model should have zero probability.
So if you just tell me that any individual outcome has
zero probability, you're not really telling me
much to work with.
For that reason, what instead we're going to do is to assign
probabilities to subsets of the sample space, as opposed
to assigning probabilities to individual outcomes.
So here's the picture.
We have our sample space, which is omega, and we
consider some subset of the sample space.
Call it A. And I want to assign a number, a numerical
probability, to this particular subset which
represents my belief about how likely this set is to occur.
OK.
What do we mean "to occur?" And I'm introducing here a
language that's being used in probability theory.
When we talk about subsets of the sample space, we usually
call them events, as opposed to subsets.
And the reason is because it works nicely with the language
that describes what's going on.
So the outcome is a point.
The outcome is random.
The outcome may be inside this set, in which case we say that
event A occurred, if we get an outcome inside here.
Or the outcome may fall outside the set, in which case
we say that event A did not occur.
So we're going to assign probabilities to events.
And now, how should we do this assignment?
Well, probabilities are meant to describe your beliefs about
which sets are more likely to occur versus other sets.
So there's many ways that you can assign those
probabilities.
But there are some ground rules for this game.
First, we want probabilities to be numbers between zero and
one because that's the usual convention.
So a probability of zero means we're certain that something
is not going to happen.
Probability of one means that we're essentially certain that
something's going to happen.
So we want numbers between zero and one.
We also want a few other things.
And those few other things are going to be encapsulated in a
set of axioms.
What "axioms" means in this context, it's the ground rules
that any legitimate probabilistic
model should obey.
You have a choice of what kind of probabilities you use.
But, no matter what you use, they should obey certain
consistency properties because if they obey those properties,
then you can go ahead and do useful calculations and do
some useful reasoning.
So what are these properties?
First, probabilities should be non-negative.
OK?
That's our convention.
We want probabilities to be numbers between zero and one.
So they should certainly be non-negative.
The probability that event A occurs should be a
non-negative number.
What's the second axiom?
The probability of the entire sample space is equal to one.
Why does this make sense?
Well, the outcome is certain to be an element of the sample
space because we set up a sample space, which is
collectively exhaustive.
No matter what the outcome is, it's going to be an element of
the sample space.
We're certain that event omega is going to occur.
Therefore, we represent this certainty by saying that the
probability of omega is equal to one.
Pretty straightforward so far.
The more interesting axiom is the third rule.
Before getting into it, just a quick reminder.
If you have two sets, A and B, the intersection of A and B
consists of those elements that belong both to A and B.
And we denote it this way.
When you think probabilistically, the way to
think of intersection is by using the word "and." This
event, this intersection, is the event that A occurred and
B occurred.
If I get an outcome inside here, A has occurred and B has
occurred at the same time.
So you may find the word "and" to be a little more convenient
than the word "intersection."
And similarly, we have some notation for the union of two
events, which we write this way.
The union of two sets, or two events, is the collection of
all the elements that belong either to the first set, or to
the second, or to both.
When you talk about events, you can use the word "or." So
this is the event that A occurred or B occurred.
And this "or" means that it could also be that both of
them occurred.
OK.
So now that we have this notation, what does
the third axiom say?
The third axiom says that if we have two events, A and B,
that have no common elements--
so here's A, here's B, and perhaps this is
our big sample space.
The two events have no common elements.
So the intersection of the two events is the empty set.
There's nothing in their intersection.
Then, the total probability of A together with B has to be
equal to the sum of the individual probabilities.
So the probability that A occurs or B occurs is equal to
the probability that A occurs plus the
probability that B occurs.
So think of probability as being cream cheese.
You have one pound of cream cheese, the total probability
assigned to the entire sample space.
And that cream cheese is spread out over this set.
The probability of A is how much cream cheese sits on top
of A. Probability of B is how much sits on top of B. The
probability of A union B is the total amount of cream
cheese sitting on top of this and that, which is obviously
the sum of how much is sitting here and how
much is sitting there.
So probabilities behave like cream cheese, or
they behave like mass.
For example, if you think of some material object, the mass
of this set consisting of two pieces is obviously the sum of
the two masses.
So this property is a very intuitive one.
It's a pretty natural one to have.
OK.
Are these axioms enough for what we want to do?
I mentioned a while ago that we want probabilities to be
numbers between zero and one.
Here's an axiom that tells you that probabilities are
non-negative.
Should we have another axiom that tells us that
probabilities are less than or equal to one?
It's a desirable property.
We would like to have it in our hands.
OK, why is it not in that list?
Well, the people who are in the axiom making business are
mathematicians and mathematicians tend to be
pretty laconic.
You don't say something if you don't have to say it.
And this is the case here.
We don't need that extra axiom because we can derive it from
the existing axioms.
Here's how it goes.
One is the probability over the entire sample space.
Here we're using the second axiom.
Now the sample space consists of A together with the
complement of A. OK?
When I write the complement of A, I mean the complement of A
inside of the set omega.
So we have omega, here's A, here's the complement of A,
and the overall set is omega.
OK.
Now, what's the next step?
What should I do next?
Which axiom should I use?
We use axiom three because a set and the complement of that
set are disjoint.
They don't have any common elements.
So axiom three applies and tells me that this is the
probability of A plus the probability of A complement.
In particular, the probability of A is equal to one minus the
probability of A complement, and this is less
than or equal to one.
Why?
Because probabilities are non-negative,
by the first axiom.
OK.
So we got the conclusion that we wanted.
Probabilities are always less than or equal to one, and this
is a simple consequence of the three axioms that we have.
This is a really nice argument because it actually uses each
one of those axioms.
The argument is simple, but you have to use all of these
three properties to get the conclusion that you want.
OK.
So we can get interesting things out of our axioms.
Can we get some more interesting ones?
How about the union of three sets?
What kind of probability should it have?
So here's an event consisting of three pieces.
And I want to say something about the probability of A
union B union C. What I would like to say is that this
probability is equal to the sum of the three individual
probabilities.
How can I do it?
I have an axiom that tells me that I can
do it for two events.
I don't have an axiom for three events.
Well, maybe I can manage things and still be able to
use that axiom.
And here's the trick.
The union of three sets, you can think of it as forming the
union of the first two sets and then taking the union with
the third set.
OK?
So taking unions, you can take the unions in any
order that you want.
So here we have the union of two sets.
Now, ABC are disjoint, by assumption or
that's how I drew it.
So if A, B, and C are disjoint, then A union B is
disjoint from C. So here we have the union of
two disjoint sets.
So by the additivity axiom, the probability of that the
union is going to be the probability of the first set
plus the probability of the second set.
And now I can use the additivity axiom once more to
write that this is probability of A plus probability of B
plus probability of C. So by using this axiom which was
stated for two sets, we can actually derive a similar
property for the union of three disjoint sets.
And then you can repeat this argument as many
times as you want.
It's valid for the union of ten disjoint sets, for the
union of a hundred disjoint sets, for the union of any
finite number of sets.
So if A1 up to An are disjoint, then the probability
of A1 union An is equal to the sum of the probabilities of
the individual sets.
OK.
Special case of this is when we're
dealing with finite sets.
Suppose I have just a finite set of outcomes.
I put them together in a set and I'm interested in the
probability of that set.
So here's our sample space.
There's lots of outcomes, but I'm taking a few of these and
I form a set out of them.
This is a set consisting of, in this
picture, three elements.
In general, it consists of k elements.
Now, a finite set, I can write it as a union of single
element sets.
So this set here is the union of this one element set,
together with this one element set together with that one
element set.
So the total probability of this set is going to be the
sum of the probabilities of the one element sets.
Now, probability of a one element set, you need to use
the brackets here because probabilities
are assigned to sets.
But this gets kind of tedious, so here one abuses notation a
little bit and we get rid of those brackets and just write
probability of this single, individual outcome.
In any case, conclusion from this exercise is that the
total probability of a finite collection of possible
outcomes, the total probability is equal to the
sum of the probabilities of individual elements.
So these are basically the axioms of probability theory.
Or, well, they're almost the axioms.
There are some subtleties that are involved here.
One subtlety is that this axiom here doesn't quite do
the job for everything we would like to do.
And we're going to come back to this at
the end of the lecture.
A second subtlety has to do with weird sets.
We said that an event is a subset of the sample space and
we assign probabilities to events.
Does this mean that we are going to assign probability to
every possible subset of the sample space?
Ideally, we would wish to do that.
Unfortunately, this is not always possible.
If you take a sample space, such as the square, the square
has nice subsets, those that you can describe by cutting it
with lines and so on.
But it does have some very ugly subsets, as well, that
are impossible to visualize, impossible to imagine, but
they do exist.
And those very weird sets are such that there's no way to
assign probabilities to them in a way that's consistent
with the axioms of probability.
OK.
So this is a very, very fine point that you can immediately
forget for the rest of this class.
You will only encounter these sets if you end up doing
doctoral work on the theoretical aspects of
probability theory.
So it's just a mathematical subtlety that some very weird
sets do not have probabilities assigned to them.
But we're not going to encounter these sets and they
do not show up in any applications.
OK.
So now let's revisit our examples.
Let's go back to the die example.
We have our sample space.
Now we need to assign a probability law.
There's lots of possible probability laws
that you can assign.
I'm picking one here, arbitrarily, in which I say
that every possible outcome has the same
probability of 1/16.
OK.
Why do I make this model?
Well, empirically, if you have well-manufactured dice, they
tend to behave that way.
We will be coming back to this kind of story
later in this class.
But I'm not saying that this is the only probability law
that there can be.
You might have weird dice in which certain outcomes are
more likely than others.
But to keep things simple, let's take every outcome to
have the same probability of 1/16.
OK.
Now that we have in our hands a sample space and the
probability law, we can actually solve any
problem there is.
We can answer any question that could be posed to us.
For example, what's the probability that the outcome,
which is this pair, is either 1,1 or 1,2.
We're talking here about this particular event, 1,1 or 1,2.
So it's an event consisting of these two items.
According to what we were just discussing, the probability of
a finite collection of outcomes is the sum of their
individual probabilities.
Each one of them has probability of 1/16, so the
probability of this is 2/16.
How about the probability of the event that x is equal to
one. x is the first roll, so that's the probability that
the first roll is equal to one.
Notice the syntax that's being used here.
Probabilities are assigned to subsets, to sets, so we think
of this as meaning the set of all outcomes such that x is
equal to one.
How do you answer this question?
You go back to the picture and you try to visualize or
identify this event of interest.
x is equal to one corresponds to this event here.
These are all the outcomes at which x is equal to one.
There's four outcomes.
Each one has probability 1/16, so the answer is 4/16.
OK.
How about the probability that x plus y is odd?
OK.
That will take a little bit more work.
But you go to the sample space and you identify all the
outcomes at which the sum is an odd number.
So that's a place where the sum is odd, these are other
places, and I guess that exhausts all the possible
outcomes at which we have an odd sum.
We count them.
How many are there?
There's a total of eight of them.
Each one has probability 1/16, total probability is 8/16.
And harder question.
What is the probability that the minimum of the two rolls
is equal to 2?
This is something that you probably couldn't do in your
head without the help of a diagram.
But once you have a diagram, things are simple.
You ask the question.
OK, this is an event, that the minimum of the two rolls is
equal to two.
This can happen in several ways.
What are the several ways that it can happen?
Go to the diagram and try to identify them.
So the minimum is equal to two if both of them are two's.
Or it could be that x is two and y is bigger, or y is two
and x is bigger.
OK.
I guess we rediscover that yellow and blue make green, so
we see here that there's a total of
five possible outcomes.
The probability of this event is 5/16.
Simple example, but the procedure that we followed in
this example actually applies to any probability model you
might ever encounter.
You set up your sample space, you make a statement that
describes the probability law over that sample space, then
somebody asks you questions about various events.
You go to your pictures, identify those events, pin
them down, and then start kind of counting and calculating
the total probability for those outcomes that you're
considering.
This example is a special case of what is called the discrete
uniform law.
The model obeys the discrete uniform law if all outcomes
are equally likely.
It doesn't have to be that way.
That's just one example of a probability law.
But when things are that way, if all outcomes are equally
likely and we have N of them, and you have a set A that has
little n elements, then each one of those elements has
probability one over capital N since all
outcomes are equally likely.
And for our probabilities to add up to one, each one must
have this much probability, and there's little n elements.
That gives you the probability of the event of interest.
So problems like the one in the previous slide and more
generally of the type described here under discrete
uniform law, these problems reduce to just counting.
How many elements are there in my sample space?
How many elements are there inside the event of interest?
Counting is generally simple, but for some problems it gets
pretty complicated.
And in a couple of weeks, we're going to have to spend
the whole lecture just on the subject of how to count
systematically.
Now the procedure we followed in the previous example is the
same as the procedure you would follow in continuous
probability problems.
So, going back to our dart problem, we get the random
point inside the square.
That's our sample space.
We need to assign a probability law.
For lack of imagination, I'm taking the probability law to
be the area of a subset.
So if we have two subsets of the sample space that have
equal areas, then I'm postulating that they are
equally likely to occur.
The probably that they fall here is the same as the
probability that they fall there.
The model doesn't have to be that way.
But if I have sort of complete ignorance of which points are
more likely than others, that might be the
reasonable model to use.
So equal areas mean equal probabilities.
If the area is twice as large, the probability is going to be
twice as big.
So this is our model.
We can now answer questions.
Let's answer the easy one.
What's the probability that the outcome is
exactly this point?
That of course is zero because a single point has zero area.
And since this probability is equal to area, that's zero
probability.
How about the probability that the sum of the coordinates of
the point that we got is less than or equal to 1/2?
How do you deal with it?
Well, you look at the picture again, at your sample space,
and try to describe the event that you're talking about.
The sum being less than 1/2 corresponds to getting an
outcome that's below this line, where this line is the
line where x plus y equals to 1/2.
So the intercepts of that line with the axis are 1/2 and 1/2.
So you describe the event visually and then you use your
probability law.
The probability law that we have is that the probability
of a set is equal to the area of that set.
So all we need to find is the area of this triangle, which
is 1/2 times 1/2 times 1/2, half, equals to 1/8.
OK.
Moral from these two examples is that it's always useful to
have a picture and work with a picture to visualize the
events that you're talking about.
And once you have a probability law in your hands,
then it's a matter of calculation to find the
probabilities of an event of interest.
The calculations we did in these two examples, of course,
were very simple.
Sometimes calculations may be a lot harder, but it's a
different business.
It's a business of calculus, for example, or being good in
algebra and so on.
As far as probability is concerned, it's clear what you
will be doing, and then maybe you're faced with a harder
algebraic part to actually carry out the calculations.
The area of a triangle is easy to compute.
If I had put down a very complicated shape, then you
might need to solve a hard integration problem to find
the area of that shape, but that's stuff that belongs to
another class that you have presumably mastered by now.
Good, OK.
So now let me spend just a couple of minutes to return to
a point that I raised before.
I was saying that the axiom that we had about additivity
might not quite be enough.
Let's illustrate what I mean by the following example.
Think of the experiment where you keep flipping a coin and
you wait until you obtain heads for the first time.
What's the sample space of this experiment?
It might happen the first flip, it might happen in the
tenth flip.
Heads for the first time might occur in the millionth flip.
So the outcome of this experiment is going to be an
integer and there's no bound to that integer.
You might have to wait very much until that happens.
So the natural sample space is the set of
all possible integers.
Somebody tells you some information about the
probability law.
The probability that you have to wait for n flips is equal
to two to the minus n.
Where did this come from?
That's a separate story.
Where did it come from?
Somebody tells this to us, and those probabilities are
plotted here as a function of n.
And you're asked to find the probability that the outcome
is an even number.
How do you go about calculating that probability?
So the probability of being an even number is the probability
of the subset that consists of just the even numbers.
So it would be a subset of this kind, that includes two,
four, and so on.
So any reasonable person would say, well the probability of
obtaining an outcome that's either two or four or six and
so on is equal to the probability of obtaining a
two, plus the probability of obtaining a four, plus the
probability of obtaining a six, and so on.
These probabilities are given to us.
So here I have to do my algebra.
I add this geometric series and I get an answer of 1/3.
That's what any reasonable person would do.
But the person who only knows the axioms that they posted
just a little earlier may get stuck.
They would get stuck at this point.
How do we justify this?
We had this property for the union of disjoint sets and the
corresponding property that tells us that the total
probability of finitely many things, outcomes, is the sum
of their individual probabilities.
But here we're using it on an infinite collection.
The probability of infinitely many points is equal to the
sum of the probabilities of each one of these.
To justify this step we need to introduce one additional
rule, an additional axiom, that tells us that this step
is actually legitimate.
And this is the countable additivity axiom, which is a
little stronger, or quite a bit stronger, than the
additivity axiom we had before.
It tells us that if we have a sequence of sets that are
disjoint and we want to find their total probability, then
we are allowed to add their individual probabilities.
So the picture might be such as follows.
We have a sequence of sets, A1, A2, A3, and so on.
I guess in order to fit them inside the sample space, the
sets need to get smaller and smaller perhaps.
They are disjoint.
We have a sequence of such sets.
The total probability of falling anywhere inside one of
those sets is the sum of their individual probabilities.
A key subtlety that's involved here is that we're talking
about a sequence of events.
By "sequence" we mean that these events can
be arranged in order.
I can tell you the first event, the second event, the
third event, and so on.
So if you have such a collection of events that can
be ordered as first, second, third, and so on, then you can
add their probabilities to find the
probability of their union.
So this point is actually a little more subtle than you
might appreciate at this point, and I'm going to return
to it at the beginning of the next lecture.
For now, enjoy the first week of classes
and have a good weekend.
Thank you.