Terminology Recap: Random Variable / Distribution / PMF / PDF / Independence / Marginal Distribution / Joint Distribution / Conditional Random Variable
主要参考:
- NOTES ON PROBABILITY - Greg Lawler
- Measure theory and probability - Alexander Grigoryan
- Lebesgue Measure on
- John K. Hunter - Conditional random variables - Lawrence Pettit
Prerequisite #1 : -algebraPermalink
非常蛋疼的一个事实:
Definition: In mathematical analysis and in probability theory, a
- 因为
同时它是 closed under complement,所以 -algebra, -ring 和 -field 都是有关系的,但这里不表
Prerequisite #2 : Borel Set / Borel -algebraPermalink
In mathematics, a Borel set is any set in a topological space that can be formed from open sets (or, equivalently, from closed sets) through the operations of countable union, countable intersection, and relative complement.
- relative complement of
in 就是 - relative complement of
in 就是
For a topological space
, the collection of all Borel sets on forms a -algebra , known as the Borel algebra or Borel -algebra. The Borel -algebra on is the smallest -algebra containing all open sets (or, equivalently, all closed sets).
关于可数性:
- A set
is said to be countable if it’s finite or (Cantor Diagonal Argument)- If
is a Borel algebra in , then- 结论:
不可数
- 结论:
Prerequisite #3 : Measurable Function / Measurable SpacePermalink
Definition: A measurable space is a tuple of
- measurable space 又称 Borel space
Definition: Let
是 inverse function- 扩展一下
的定义: - 这个定义相当于:
使得- 这个
即
- 这个
- 为了强调
是一个 measurable function,我们也可以把它写作
Prerequisite #4 : Measure / Measure SpacePermalink
Definition: Let
- Non-negativity:
- 注:不满足这个条件的 measure 是存在的,比如 signed measure
- Null empty set:
- Countable additivity (or
-additivity): where and :
Definition: A measure space is such a triple of
Prerequisite #5 : Probability Measure / Probability SpacePermalink
Definition: Measure
指全集
Definition: A probability space is a measure space with a probability measure, denoted by
is called an outcome is called an event is a probability measure is the probability of
Prerequisite #3/#4/#5 SummaryPermalink
- measurable function
定义在 measurable space 上 - measurable function
有潜力构成一个 measure - measure
+ measurable space = measure space- probability measure
是特殊的 measure - 装备 probability measure 的 measure space 是 probability space
- probability measure
我们可以把 measurable function
- 比如我们可以定义
然后根据 -additivity 有:
- 注意我这里的意思是:我们可以这样做,但没有规定说一定要这样做;
也不一定要通过 定义, 也不一定满足进化成 的要求
1. Random VariablePermalink
Definition: A random variable
- 准确来说应该是
而不仅仅是 是 上的 Borel -algebra
若
- 我们称
是 -measurable. We define to be the smallest -algebra on for which is measurable. - 比较一下
和 :- 首先注意定义域:
(random variable 接收 outcome) (probability measure 接收 event)
是 measurable function, 是 probability measure,我们可以像上面 一样定义一个 使它可以 ,但是!没有必要。后面 distribution 的部分会阐述。
- 首先注意定义域:
以投骰子为例 (一个骰子,仅投一次):
包括但不限于 、 、 、 、 、 、- 假设有
- 注意 event
表示 “roll 出 1 或者 3”,而不是 “roll 两次,一次是 1 一次是 3”- “roll 两次,一次是 1 一次是 3” 的 event 应该是
- “roll 两次,一次是 1 一次是 3” 的 event 应该是
- 所以
,同理有 - “roll 出 1 且 3” 是不可能事件,即
,由 measure 的定义得到
- 注意 event
2. Distribution of a Random VariablePermalink
假定有 probability space
Definition: The push-forward measure of
- 注意根据 random variable 的定义,
,所以 在 的定义域内 一定是一个 probability measure,使得 构成一个 probability space- 若
,即 ,可得
我们称
我们这里重点考察一下
- 按理来说,
应该是 ,但是我们通过 的定义把它扩展成了 - 于是
就成了一个 的函数 - 所以
就可以看作一个 “event encoder“,它把每一个 event 映射到一个 Borel set - 同理
就可以看成一个 “event decoder“,它把每一个 Borel set 又映射回原来的 event - Event encoding 的作用在于:可以把各种不同的、具体的
转化为统一的、抽象的- 比如 “投骰子” 和 “黑盒子里 6 个不同颜色的球,抓一个出来” 这两个实验,它们的 event 是不一样的,但我们明显可以看出它们的本质是一样的,这个本质体现在它们通过
encoding 以后,得到的 Borel set 是一样的 (或者说得到的 函数是一样的)
- 比如 “投骰子” 和 “黑盒子里 6 个不同颜色的球,抓一个出来” 这两个实验,它们的 event 是不一样的,但我们明显可以看出它们的本质是一样的,这个本质体现在它们通过
- Event decoding 的作用在于计算,因为
需要借助 才能算出具体的值 - 我们平时根本就没有注意到这个 event encoding/decoding 的过程是因为:它太顺理成章了。比如上面 “投骰子” 的例子,我们直接就写出了
,所以可以有 ,亦即 ,等于没有做 event encoding/decoding,于是我们也没有区分 和 ,因为 - 但是我也可以定义说
,那你可能需要 encode 一下,得到:- 所以
- 当然,你的
的定义可以不用与 event 的语义对应,比如我定义 ,也是可以的
题外话:
- 先说结论:这是个有点过分的简写
- 首先
应该是 ( 接收 event) - 二来
应该理解为 - 这么一来,令
,套公式可得:
- 所以
整体是一个 event (informal);而 是一个 Borel set - 若
,则 , ,从而
- If
gives measure one to a countable set of reals, then is called a discrete random variable. , 然后 不可数- 但
的 domain 可能只是 的一个可数子集
- If
gives zero measure to every singleton set, and hence to every countable set, is called a continuous random variable.- Every random variable can be written as a sum of a discrete random variable and a continuous random variable.
- All random variables defined on a discrete probability space are discrete
Definition: 对任意的 (locally finite) measure
那既然
严格来说,
3. Probability Mass Functions (for the discrete), and Probability Density Functions (for the continuous)Permalink
Definition: Probability mass function for discrete random variable
其实就是把
Definition: Probability density function for continuous random variable
- 严格来说,
应该叫做 “the density or Radon–Nikodym derivative with respect to Lebesgue measure of random variable ”
若
- 我们可以写
- If
is continuous at
4. Tilde / i.i.d.Permalink
根据 Ben O’Neill: Why are probability distributions denoted with a tilde?,
- 所以
它不是 distribution,而是一个 random variable - 如果
,那么 - 如果
不确定, 可以看做一个 parametric random variable- 注意如果有
,那么这里 一定是表示一个具体的 random variable (once 确定下来),而不能理解为是一个 family of random variables
- 注意如果有
那么问题来了:”has the same distribution as” 这个 distribution 指的是
- 若
都是 discrete random variable,那么明显 更直接,所以一般我们用 这个结论- 进而有
- 进而有
- 若
都是 continuous random variable,那么明显 才有意义,所以一般我们用 这个结论- 进而有
- 进而有
我们直接研究 random variable
另外还有一个常见的概念是 i.i.d. (independent and identically distributed),它是用来形容一组 random variables 的。简单说,如果
(我觉得诡异的是这么多年我就没见过哪本教材用这个式子来描述 i.i.d.) 互相是 independent 的
5. Independence / Marginal Distribution / Join DistributionPermalink
我们先从
Definition: (1) Two events
(2) A collection of events
(3) A collection of events
(4) A finite collection of
(5) An infinite collection of
If
- Let
denote the standard topology on consisting of all open sets is the -algebra generated by all the open set, i.e.
假设原有 probability spaces
is the Cartesian product of the two sets is the -algebra on , generated by subsets of the form where and- A product measure
is defined to be a measure on the measurable space satisfying ,
假设
- distribution:
- distribution function of distribution:
- PMF:
- PDF:
Definition: For random variable
Definition: For joint distribution
Definition: For random variable
Definition: For random variable
Definition: Random variables
(1) Joint distribution is the product of all marginal distributions:
- This is equivalent of saying “joint distribution is the product measure of all marginal distributions”:
- Marginal distribution of
其实就是 ’s individual distribution,它只在 joint distribution 这个 context 下有意义。语出二维的 discrete joint distribution table,比如:
(2) Joint distribution function is the product of all marginal distribution functions:
(3) Joint PMF is the product of all individual PMFs:
(4) Joint PDF (if exists) is the product of all individual PDFs:
(5) The
6. Conditional Random VariablePermalink
Suppse
Definition: The discrete conditional random variable
Similarly, we can have
Definition: The continuous conditional random variable
Comments