摘要訊息 : 進一步完善大數法則和由 Chebyshav 不等式得到的估計.

0. 前言

在《【機率論】初等機率論——Bernoulli 概型 : 大數法則》中, 我們介紹了 Chebyshav 不等式, 並且由它導出了大數法則和最小觀測次數. 然而, 我們在導出最小觀測次數的時候, 已經指出最小觀測次數 \left \lceil \frac {1}{4\varepsilon\alpha} \right \rceil 是粗略的. 在本篇文章中, 我們將繼續討論, 得到盡可能精確的結論, 並且形成新的定理.

更新紀錄 :

• 2022 年 6 月 12 日進行第一次更新和修正.

1. De Moivre-Laplace 局部極限定理

考慮隨機變數 \xi_{1}, \xi_{2}, ..., \xi_{n}, 記 S_{n} = \xi_{1} + \xi_{2} + ... + \xi_{n}, 有 \mathop {\mathrm {E}} \left ( \frac {S_{n}}{n} \right ) = p. 由於 \text {Var}(S_{n}) = npq, 而 \displaystyle {\begin {aligned} \text {Var}(S_{n}) &= \mathop {\mathrm {E}} \left ( (S_{n} - \mathop {\mathrm {E}}(S_{n}))^{2} \right ) = \mathop {\mathrm {E}} \left ( (S_{n} - np)^{2} \right ) \\ &= \mathop {\mathrm {E}} \left ( \left ( \frac {S_{n} - np}{n} \cdot n \right )^{2} \right ) = \mathop {\mathrm {E}} \left ( n^{2} \left ( \frac {S_{n} - np}{n} \right )^{2} \right ) \\ &= n^{2}\mathop {\mathrm {E}} \left ( \left ( \frac {S_{n}}{n} - p \right )^{2} \right ) = npq. \end {aligned}} 故有 \mathop {\mathrm {E}} \left ( \left ( \frac {S_{n}}{n} - p \right )^{2} \right ) = \frac {pq}{n}.

我們已經在大數法則中得到了 \frac {Sn}{n} 與 p 之間的機率意義. 那麼, 我們自然希望得到 \left ( \frac {S_{n}}{n} - p \right )^{2} 與 \frac {pq}{n} 的機率意義. 例如, 考慮下面這個機率 \displaystyle {\mathop {\mathbf {P}} \left \{ \left | \frac {S_{n}}{n} - p \right | \leq x\sqrt {\frac {pq}{n}} \right \}, x \in R} 如何?

又由於 \mathop {\mathrm {E}}(S_{n}) = np 且 \text {Var}(S_{n}) = npq, 而 \displaystyle {\frac {\left | \frac {S_{n}}{n} - p \right |}{\sqrt {\frac {pq}{n}}} = \frac {\left | \frac {S_{n}}{n} - p \right |}{\sqrt {\frac {pq}{n} \cdot \frac {n}{n}}} = \frac {\left | \sqrt {n^{2}}\frac {S_{n} - np}{n} \right |}{\sqrt {\frac {pq}{n}}} = \frac {\left | S_{n} - np \right |}{\sqrt {npq}} = \frac {\left | S_{n} - \mathop {\mathrm {E}}(S_{n}) \right |}{\sqrt {\text {Var}(S_{n})}}}. 我們又可以考慮 \displaystyle {\mathop {\mathbf {P}} \left \{ \left | \frac {S_{n} - \mathop {\mathrm {E}}(S_{n})}{\sqrt {\text {Var}(S_{n})}} \right | \leq x \right \}}, 如果對於 n \geq 1, 記 P_{n}(k) = \binom {k}{n}p^{k}q^{n - k} (k = 0, 1, 2, …, n), 則有 \displaystyle {\mathop {\mathbf {P}} \left \{ \left | \frac {S_{n} - \mathop {\mathrm {E}}(S_{n})}{\sqrt {\text {Var}(S_{n})}} \right | \leq x \right \} = \sum \limits_{\left \{ k : \frac {\left |k - np \right |}{\sqrt {npq}} \leq x \right \}}P_{n}(k)}. 那麼當 n \to \infty 時, 機率 P_{n}(k) 如何?

定理 1. (局部極限定理) 記 P_{n}(k) = \binom {k}{n}p^{k}q^{n - k}\ (k = 0, 1, 2, ..., n), 設 0 < p < 1, 則對滿足 \left | k - np \right | = o \left ( (npq)^{\frac {2}{3}} \right ) 的所有 k, 一致有 \displaystyle {P_{n}(k) \to \frac {1}{\sqrt {2\pi npq}}\mathrm {e}^{-\frac {(k - np)^{2}}{2npq}} \ (n \to \infty)} 即當 n \to \infty 時, \displaystyle {\sup \limits_{\left \{ k : \left | k - np \right | \leq \varphi(n) \right \}} \left \{ \left | \frac {P_{n}(k)}{\frac {1}{\sqrt {2\pi npq}}\mathrm {e}^{-\frac {(k - np)^{2}}{2npq}}} - 1 \right | \right \} \to 0}. 其中, \varphi(n) 是任意滿足 \varphi(n) = o \left ( (npq)^{\frac {2}{3}} \right ) 的非負函數.

證明 :

根據 Stering 公式 (《複雜度下界》引理 1), 有 \displaystyle {n! = \sqrt {2\pi n}\frac {n^{n}}{\mathrm {e}^{n}}(1 + R(n))}. 其中, 當 n \to \infty 時, R(n) \to 0. 若 n \to \infty 時, k \to \infty, n - k \to \infty, 則 \displaystyle {\begin {aligned} \binom {k}{n} &= \frac {n!}{k!(n - k)!} \\ &= \frac {\sqrt {2\pi n}\left ( \frac {n}{e} \right )^{n}(1 + R(n))}{\sqrt {2\pi k \times 2\pi (n - k)}\left ( \frac {k}{e} \right )^{k}\left ( \frac {n - k}{e} \right )^{n - k}(1 + R(k))(1 + R(n - k))} \\ &= \frac {\frac {n^{n}}{\mathrm {e}^{n}}(1 + \varepsilon(n, k, n - k))}{\frac {1}{\sqrt {n}}\sqrt {2\pi k(n - k)}\frac {k^{k}}{\mathrm {e}^{k}}\frac {(n - k)^{n - k}}{\mathrm {e}^{n - k}}} \\ &= \frac {n^{n}(1 + \varepsilon(n, k, n - k))}{\sqrt {2\pi\frac {k}{n}(n - k)}k^{k}(n - k)^{n - k}} \\ &= \frac {n^{n}(1 + \varepsilon(n, k, n - k))}{\sqrt {2\pi\frac {k}{n}(n - k)}n^{k} \left ( \frac {k}{n} \right )^{k}n^{n - k} \left ( 1 - \frac {k}{n} \right )^{n - k}} \\ &= \frac {1}{\sqrt {2\pi n\frac {k}{n}\left ( 1 - \frac {k}{n} \right )}} \times \frac {1 + \varepsilon(n, k, n - k)}{\left ( \frac {k}{n} \right )^{k}\left ( 1 - \frac {k}{n} \right )^{n - k}}. \end {aligned}} 其中, 當 n \to \infty, k \to \infty, n - k \to \infty 時, \varepsilon(n, k, n - k) \to 0. 故有 \displaystyle {P_{n}(k) = \binom {k}{n}p^{k}q^{n - k} = \frac {p^{k}(1 - p)^{n - k} \left ( 1 + \varepsilon(n, k, n - k) \right )}{\sqrt {2\pi n\frac {k}{n} \left ( 1 - \frac {k}{n} \right ) \left ( \frac {k}{n} \right )^{k} \left ( 1 - \frac {k}{n} \right )^{n - k}}}}. 記 \hat {p} = \frac {k}{n}, 則 \displaystyle {\begin {aligned} P_{n}(k) &= \binom {k}{n}p^{k}q^{n - k} = \frac {p^{k}(1 - p)^{n - k} \left ( 1 + \varepsilon(n, k, n - k) \right )}{\sqrt {2\pi n\hat {p} \left ( 1 - \hat {p} \right ) \hat {p}^{k} \left ( 1 - \hat {p} \right )^{n - k}}} \\ &= \frac {1 + \varepsilon(n, k, n - k)}{\sqrt {2\pi n\hat {p}\left ( 1 - \hat {p} \right )}}\left ( \frac {p}{\hat {p}} \right )^{k}\left ( \frac {1 - p}{1 - \hat {p}} \right )^{n - k} \\ &= \frac {1 + \varepsilon(n, k, n - k)}{\sqrt {2\pi n\hat {p}\left ( 1 - \hat {p} \right )}} \exp \left \{ k\ln {\frac {p}{\hat {p}} + (n - k)\ln {\frac {1 - p}{1 - \hat {p}}}} \right \} \\ &= \frac {1 + \varepsilon(n, k, n - k)}{\sqrt {2\pi n\hat {p}\left ( 1 - \hat {p} \right )}} \exp \left \{n \left (\frac {k}{n}\ln {\frac {p}{\hat {p}}} + \left ( 1- \frac {k}{n} \right )\ln {\frac {1 - p}{1 - \hat {p}}} \right )\right \} \\&= \frac {1 + \varepsilon(n, k, n - k)}{\sqrt {2\pi n\hat {p}\left ( 1 - \hat {p} \right )}}\exp \left \{-nH(\hat {p}) \right \} \\ &= \frac {1 + \varepsilon(n, k, n - k)}{\sqrt {2\pi n\hat {p}\left ( 1 - \hat {p} \right )}} \cdot \frac {1}{\mathrm {e}^{nH(\hat {p})}}. \end {aligned}} 其中, H(x) = x\ln {\frac {x}{p}} + (1 - x)\ln {1 - x}{1 - p}.

由於 k 滿足 \left | k - np \right | = o \left ( (npq)^{\frac {2}{3}} \right ), 故當 k \to \infty 時, p - \hat {p} \to 0. 另外, 對於 0 < x < 1, 有 \displaystyle {H'(x) = \ln {\frac {x}{p}} - \ln {1 - x}{1 - p}, H''(x) = \frac {1}{x} + \frac {1}{1 - x}, H'''(x) = -\frac {1}{x^{2}} + \frac {1}{(1 - x)^{2}}}. 將 H(\hat {p}) 表示為 H(p + (\hat {p} - p)), 根據 Taylor 公式, 當 n 充分大的時候, 有 \displaystyle {\begin {aligned} H(\hat {p}) &= H(p) + H'(p)(\hat {p} - p) + \frac {1}{2}H''(p)(\hat {p} - p)^{2} + o(\left | \hat {p} - p \right |^{3}) \\ &= \frac {1}{2}\left ( \frac {1}{p} + \frac {1}{q} \right )\left ( \hat {p} - p \right )^{2} + o(\left | \hat {p} - p \right |^{3}). \end {aligned}} 從而有 \displaystyle {P_{n}(k) = \frac {1 + \varepsilon(n, k, n - k)}{\sqrt {2\pi n\hat {p}(1 - \hat {p})}}\exp \left \{ -\frac {n}{2pq}\left ( \hat {p} - p \right )^{2} + n \cdot o \left ( \left | \hat {p} - p \right |^{3} \right ) \right \}}. 注意到 \displaystyle {\frac {n}{2pq}\left ( \hat {p} - p \right )^{2} = \frac {n}{2pq}\left ( \frac {k}{n} - p \right )^{2} = \frac {(n - kp)^{2}}{2npq}}, 故有 \displaystyle {P_{n}(k) \to \frac {1}{\sqrt {2\pi npq}}\mathrm {e}^{-\frac {(k - np)^{2}}{2npq}}\left ( 1 + \varepsilon'(n, k, n - k) \right )}. 其中, 1 + \varepsilon'(n, k, n - k) = 1 + \varepsilon(n, k, n - k)\mathrm {e}^{n \cdot o(\left | \hat {p} - p \right |^{3})}\sqrt {\frac {p(1 - p)}{\hat {p}(1 - \hat {p})}}. 顯然, 當 n \to \infty 時, \sup \left | \varepsilon'(n, k, n - k) \right | \to 0. 因此, 有 \displaystyle {P_{n}(k) \to \frac {1}{\sqrt {2\pi npq}}\mathrm {e}^{-\frac {(k - np)^{2}}{2npq}} \ (n \to \infty)}. 其中, \left | k - np \right | \leq \varphi(n) 且 \varphi(n) = o \left ( (npq)^{\frac {2}{3}} \right ).

\blacksquare

推論 1. 對於一切 x \in R, 若 x = o \left ( (npq)^{\frac {1}{6}} \right ), 而 np + x\sqrt {npq} 的結果是集合 \left \{ 0, 1, 2, ..., n \right \} 中的整數, 則 \displaystyle {P_{n}(np + x\sqrt {npq}) \to \frac {1}{\sqrt {2\pi npq}}\mathrm {e}^{-\frac {x^{2}}{2}} \ (n \to \infty)}. 即當 n \to \infty 時, 有 \displaystyle {\sup \limits_{\left \{ x : \left | x \right | \leq \psi(n) \right \}} \left \{ \left | \frac {P_{n}(np + x\sqrt {npq})}{\frac {1}{\sqrt {2\pi npq}}\mathrm {e}^{-\frac {x^{2}}{2}}} - 1 \right | \right \} \to 0}. 其中, \psi(x) = o \left ((npq)^{\frac {1}{6}} \right ).

在大數法則中, 我們曾給出 \mathop {\mathbf {P}} \left \{ \left | \frac {S_{n}}{n} - p \right | \geq \varepsilon \right \} \to 0 \ (n \to \infty) 以質點游動的解釋. 引入上述理論之後, 我們可以將之前的解釋描述為 \displaystyle {\mathop {\mathbf {P}} \left \{ S_{n} = k \right \} \to \frac {1}{\sqrt {2\pi npq}}\mathrm {e}^{-\frac {(k - np)^{2}}{2npq}}, \left | k - np \right | = o \left ( (npq)^{\frac {2}{3}} \right )} 和 \displaystyle {\mathop {\mathbf {P}} \left \{ \frac {S_{n} - np}{\sqrt {npq}} = x \right \} \to \frac {1}{\sqrt {2\pi npq}\mathrm {e}^{-\frac {x^{2}}{2}}}, x = o \left ( (npq)^{\frac {1}{6}} \right )\ \ \ \ \ \ \ \ \ \ (\mathrm {I})}. 其中, \mathop {\mathbf {P}} \left \{ \frac {S_{n} - np}{\sqrt {npq}} = x \right \} 中的 np + x\sqrt {npq} \in \left \{ 0, 1, 2, ..., n \right \}.

若設 t_{k} = \frac {k - np}{\sqrt {npq}}, \Delta t_{k} = t_{k + 1} - t_{k} = \frac {1}{\sqrt {npq}},, 那麼通過置換可以將 (\mathrm {I}) 式改寫為, \displaystyle {\mathop {\mathbf {P}} \left \{ \frac {S_{n} - \mathop {\mathrm {E}}(S_{n})}{\sqrt {\text {Var}(S_{n})}} \right \} = t_{k} \to \frac {\Delta t_{k}}{\sqrt {2\pi}}\mathrm {e}^{-\frac {t_{k}^{2}}{2}}, t_{k} = o \left ( (npq)^{\frac {1}{6}} \right )}. 顯然, 當 n \to \infty 時, \displaystyle {\Delta t_{k} = \frac {1}{\sqrt {npq}} \to 0}. 根據上述描述, 我們發現 \mathrm {e}^{-\frac {t_{k}^{2}}{2}} \cdot \Delta t_{k} 與定積分的定義類似, 於是我們自然想到 \displaystyle {\mathop {\mathbf {P}} \left \{ a < \frac {S_{n} - \mathop {\mathrm {E}}(S_{n})}{\sqrt {\text {Var}(S_{n})}} \leq b \right \} = t_{k} \to \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x}, -\infty < a \leq b < +\infty}. 接下來, 我們將圍繞該式進行表述.

2. De Moivre-Laplace 極限定理

對於 -\infty < a \leq b < +\infty, 設 \displaystyle {P_{n}(a, b] = \sum \limits_{a < x \leq b}P_{n}(np + x\sqrt {npq})}, 其中求和是針對一切使得 np + x\sqrt {npq} 為整數的 x 求和. 由局部極限定理可見, 對於 k = np + t_{k}\sqrt {npq} 決定且滿足 \left | t_{k} \right | \leq T < +\infty 的 t_{k}, 由推論 1 有 \displaystyle {P_{n}(np + t_{k}\sqrt {npq}) \to \frac {t_{k}}{\sqrt {2\pi}}\mathrm {e}^{-\frac {t_{k}^{2}}{2}}\big ( 1 + \varepsilon(t_{k}, n) \big )}. 其中, \sup \limits_{\left | t_{k} \right | \leq T}\left | \varepsilon(t_{k}, n) \right | \to 0 \ (n \to \infty). 從而對於固定的 a 和 b, 其中 -\infty < -T \leq a \leq b \leq T < +\infty, 有 \displaystyle {\begin {aligned} \sum \limits_{a < t_{k} \leq b}P_{n}(np + t_{k}\sqrt {npq}) &= \sum \limits_{a < t_{k} \leq b}\frac {t_{k}}{\sqrt {2\pi}}\mathrm {e}^{-\frac {t_{k}^{2}}{2}} + \sum \limits_{a < t_{k} \leq b}\varepsilon(t_{k}, n)\frac {\Delta t_{k}}{\sqrt {2\pi}}\mathrm {e}^{-\frac {t_{k}^{2}}{2}} \\ &= \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} + R_{n}^{(1)}(a, b) + R_{n}^{(2)}(a, b). \end {aligned}} 其中, R_{1}^{(n)} = \sum \limits_{a < t_{k} \leq b}\frac {\Delta t_{k}}{\sqrt {2\pi}}\mathrm {e}^{-\frac {t_{k}^{2}}{2}} - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x}, R_{n}^{(2)} = \sum \limits_{a < t_{k} \leq b}\varepsilon(t_{k}, n)\frac {\Delta t_{k}}{\sqrt {2\pi}}\mathrm {e}^{-\frac {t_{k}^{2}}{2}}. 對於 R_{n}^{(1)}(a, b), 顯然地, 當 n \to \infty 時, \sup \limits_{-T \leq a \leq b \leq T}\left | R_{n}^{(1)}(a, b) \right | \to 0. 為了繼續導出結論, 我們引入數學分析中的一個事實.

引理 1. \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{+\infty}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} = 1.

這個引理我們暫時不進行證明.

由於 f(x) = \mathrm {e}^{-\frac {x^{2}}{2}} 是連續函數且 f(x) > 0, 故必有 \displaystyle {\frac {1}{\sqrt {2\pi}}\int_{-T}^{T}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \leq \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{+\infty}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \leq 1}. 因此, \displaystyle {\begin {aligned} &\sup \limits_{-T \leq a \leq b \leq T}\left | R_{n}^{(2)}(a, b) \right | \\ &= \sup \limits_{-T \leq a \leq b \leq T}\left | \sum \limits_{a < t_{k} \leq b}\varepsilon(t_{k}, n)\frac {\Delta t_{k}}{\sqrt {2\pi}}\mathrm {e}^{-\frac {t_{k}^{2}}{2}} \right | \\ &= \sup \limits_{-T \leq a \leq b \leq T}\left | \sum \limits_{a < t_{k} \leq b}\varepsilon(t_{k}, n) \right |\frac {\Delta t_{k}}{\sqrt {2\pi}}\mathrm {e}^{-\frac {t_{k}^{2}}{2}} \\ &\leq \sup \limits_{\left | t_{k} \right | \leq T}\left | \varepsilon(t_{k}, n) \right | \sum \limits_{\left | t_{k} \right | \leq T}\frac {\Delta t_{k}}{\sqrt {2\pi}}\mathrm {e}^{-\frac {t_{k}^{2}}{2}} \\ &= \sup \limits_{-T \leq a \leq b \leq T}\left |\varepsilon(t_{k}, n) \right | \left ( R_{1}^{(n)}(a, b) + \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \right ) \\ &= \sup \limits_{-T \leq a \leq b \leq T}\left |\varepsilon(t_{k}, n) \right | \sup \limits_{-T \leq a \leq b \leq T}R_{1}^{(n)}(a, b) + \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \\ &= \sup \limits_{-T \leq a \leq b \leq T} \left | \varepsilon(t_{k}, n) \right |. \end {aligned}} 所以, 當 n \to \infty 時, \sup \limits_{-T \leq a \leq b \leq T}\left |R_{n}^{(2)}(a, b) \right | \to 0.

記 \Phi(x) = \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{x}\mathrm {e}^{-\frac {t^{2}}{2}}dt, 由上述過程可見 : \displaystyle {\sup \limits_{-T \leq a \leq b \leq T}\left | P_{n}(a, b] - (\Phi(b) - \Phi(a)) \right | \to 0 \ (n \to \infty)}. 我們已經證明, 對於有限的 T, 上式成立. 那麼對於 T \to \infty, 上式是否也成立呢?

顯然, 對於任意給定的 \varepsilon > 0, 存在有限的 T = T(\varepsilon) 使得 \displaystyle {\frac {1}{\sqrt {2\pi}}\int_{-T}^{T}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} > 1 - \frac {\varepsilon}{4}}. 此時, 對於任意 \varepsilon > 0, 存在正整數 N, 使得對於一切 n > N 和 T = T(\varepsilon), 有 \displaystyle {\sup \limits_{-T \leq a \leq b \leq T}\left | P_{n}(a, b] - (\Phi(b) - \Phi(a)) \right | < \frac {\varepsilon}{4}}. 結合 P(a, b] 的定義與 \frac {1}{2}\int_{-T}^{T}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} > 1 - \frac {\varepsilon}{4}, 必定有 \displaystyle {P_{n}(-T, T] > 1 - \frac {\varepsilon}{2}}. 而 P_{n}(-\infty, -T] + P_{n}(-T, T] + P_{n}(T, +\infty) = 1. 因此, \displaystyle {P_{n}(-\infty, -T] + P_{n}(T, +\infty) \leq \frac {\varepsilon}{2}}. 其中, P_{n}(-\infty, T] = \lim \limits_{S \to -\infty}P_{n}(S, T], P_{n}(T, +\infty) = \lim \limits_{S \to +\infty}P_{n}(T, S]. 這樣, 對於任意 -\infty < a \leq -T \leq T \leq b < +\infty, 根據絕對值不等式 (《【數學分析】實數——實數的四則運算》定理 3), 有 \displaystyle {\begin {aligned} \left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \right | &= \Bigg | \left ( P_{n}(a, -T) - \frac {1}{\sqrt {2\pi}}\int_{a}^{-T} \mathrm {e}^{\frac {x^{2}}{2}}\mathrm {d}{x} \right ) + \\ &\ \ \ \ \ \ \ \left ( P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T} \mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \right ) + \\ &\ \ \ \ \ \ \ \left ( P_{n}(T, b] - \frac {1}{\sqrt {2\pi}}\int_{T}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \right ) \Bigg | \\ &\leq \left | P_{n}(a, -T] - \frac {1}{\sqrt {2\pi}}\int_{a}^{-T}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \right | + \\ &\ \ \ \ \ \left | P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \right | + \\ &\ \ \ \ \ \left | P_{n}(T, b] - \frac {1}{\sqrt {2\pi}}\int_{T}^{b}\mathrm {e}^{\frac {x^{2}}{2}}\mathrm {d}{x} \right |. \end {aligned}} 顯然地, 我們有 \displaystyle {P_{n}(a, -T] \leq P_{n}(-\infty, -T], P_{n}(T, b] \leq P_{n}(T, +\infty)}. 除此之外, 根據絕對值不等式, 我們又得 \displaystyle {\left | P_{n}(a, -T] - \frac {1}{\sqrt {2\pi}}\int_{a}^{-T}\mathrm {e}^{\frac {x^{2}}{2}}\mathrm {d}{x} \right |} \leq \left | P_{n}(a, -T] \right | + \left | \frac {1}{\sqrt {2\pi}\int_{a}^{-T}\mathrm {e}^{-\frac {x^{2}}{2}}}\mathrm {d}{x} \right | 和 \displaystyle {\left | P_{n}(T, b] - \frac {1}{\sqrt {2\pi}}\int_{T}^{b}\mathrm {e}^{\frac {x^{2}}{2}}\mathrm {d}{x} \right |} \leq \left | P_{n}(a, -T] \right | + \left | \frac {1}{\sqrt {2\pi}\int_{a}^{-T}\mathrm {e}^{-\frac {x^{2}}{2}}}\mathrm {d}{x} \right |. 因此, \displaystyle {\begin {aligned} \left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \right | &\leq \left | P_{n}(-\infty, -T] \right | + \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{-T}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} + \\ &\ \ \ \ \ \left | P_{n}(T, +\infty) \right | + \frac {1}{\sqrt {2\pi}}\int_{T}^{+\infty}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} + \\ &\ \ \ \ \ \left | P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \right |. \end {aligned}} 因為 P_{n}(-\infty, -T] \geq 0, P_{n}(T, +\infty) \geq 0, 於是 \displaystyle {\begin {aligned} \left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \right | &\leq P_{n}(-\infty, -T] + \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{-T}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} + \\ &\ \ \ \ \ P_{n}(T, +\infty) + \frac {1}{\sqrt {2\pi}}\int_{T}^{+\infty}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} + \\ &\ \ \ \ \ \left | P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \right |. \end {aligned}} 由於 \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{-T}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} + \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} + \frac {1}{\sqrt {2\pi}}\int_{T}^{+\infty}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} = 1 且 \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} > 1 - \frac {\varepsilon}{4}, 則 \displaystyle {\frac {1}{\sqrt {2\pi}}\int_{-\infty}^{-T}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} + \frac {1}{\sqrt {2\pi}}\int_{T}^{+\infty}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \leq \frac {\varepsilon}{4}}. 結合 P_{n}(-\infty, -T] + P(T, +\infty) \leq \frac {\varepsilon}{2}, 那麼 \displaystyle {\begin {aligned} \left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{\frac {x^{2}}{2}}\mathrm {d}{x} \right | &\leq \frac {\varepsilon}{2} + \frac {\varepsilon}{4} + \left |P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}\mathrm {e}^{\frac {x^{2}}{2}}\mathrm {d}{x} \right | \\ &= \frac {3}{4}\varepsilon + \left | P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}\mathrm {e}^{\frac {x^{2}}{2}}\mathrm {d}{x} \right |. \end {aligned}} 對於 \left | P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}\mathrm {e}^{\frac {x^{2}}{2}}\mathrm {d}{x} \right |, 我們已經知道 P_{n}(-T, T] \geq 1 - \frac {\varepsilon}{2}, \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \geq 1 - \frac {\varepsilon}{4}, 故 \displaystyle {P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}\mathrm {e}^{\frac {x^{2}}{2}}\mathrm {d}{x} \geq -\frac {\varepsilon}{4}}. 不等式兩側取負, 不等式變號, 得到 \displaystyle {- \left (P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}\mathrm {e}^{\frac {x^{2}}{2}}\mathrm {d}{x} \right ) \leq \frac {\varepsilon}{4}}. 不等式兩側取絕對值, 有 \displaystyle {\left | P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}\mathrm {e}^{\frac {x^{2}}{2}}\mathrm {d}{x} \right | \leq \frac {\varepsilon}{4}}. 綜上所述, \displaystyle {\left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \right | \leq \frac {3}{4}\varepsilon + \frac {\varepsilon}{4} = \varepsilon}. 對於等號, 我們只需要適當放大 \varepsilon, 即可得到對於任意 \varepsilon > 0, 存在正整數 N, 當 n > N 時, 有 \displaystyle {\left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \right | < \varepsilon}. 根據極限的定義, 有 \displaystyle {\lim \limits_{n \to \infty}P_{n}(a, b] = \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x}}. 我們總結一下上面的討論.

定理 2. (De Moivre - Laplace 積分定理) 設 0 < p < 1, \displaystyle {P_{n}(k) = \binom {k}{n}p^{k}q^{n - k}, P_{n}(a, b] = \sum \limits_{a < x \leq b}P_{n}(np + x\sqrt {npq})} 那麼 \displaystyle {\sup \limits_{-\infty < a \leq b < +\infty} \left \{ \left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{\frac {x^{2}}{2}}\mathrm {d}{x} \right | \right \} \to 0 \ (n \to \infty)}.

在大數法則中, 我們得到的 \mathop {\mathbf {P}} \left \{ \left | \frac {S_{n}}{n} - p \right | \geq \varepsilon \right \} \to 0 \ (n \to \infty) 可以更精確地表示為 \displaystyle {\sup \limits_{-\infty < a < b < +\infty} \left \{ \left | \mathop {\mathbf {P}} \left \{ a < \frac {S_{n} - \mathop {\mathrm {E}}(S_{n})}{\sqrt {\text {Var}(S_{n})}} \leq b \right \}- \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \right | \right \} \to 0 \ (n \to \infty)}. 由此可見, 對於任意 -\infty < A < B < +\infty, 有 \displaystyle {\mathop {\mathbf {P}} \left \{ A < S_{n} \leq B \right \} - \left ( \Phi \left( \frac {B - np}{\sqrt {npq}} \right ) - \Phi \left( \frac {A - np}{\sqrt {npq}} \right ) \right ) \to 0 \ (n \to \infty)}.

例題 1. 將規則的骰子擲 12000 次. 問 : "6 點" 出現在次數區間 (1800, 2100] 次的機率如何?

解 :

根據前面的討論, 我們可以寫出 \displaystyle {\begin {aligned} P_{12000}(k) &= \sum \limits_{1800 < x \leq 2100}\binom {k}{12000} \left ( \frac {1}{6} \right )^{k}\left ( \frac {5}{6} \right )^{k} \\ &\doteq \Phi \left ( \frac {B - np}{\sqrt {npq}} \right ) - \Phi\left ( \frac {A - np}{\sqrt {npq}} \right ) \doteq 0.992. \end {aligned}} 其中, A = 1800, B = 2100, n = 12000, p = \frac {1}{6}, q = \frac {5}{6}, \Phi(x) = \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{x}\mathrm {e}^{-\frac {t^{2}}{2}}\mathrm {d}{t}.

\blacksquare

把二項機率 P_{n}(np + x\sqrt {npq}) (僅考慮能夠使得 np + x\sqrt {npq} 取非負整數的 x) 標在圖上 :

那麼局部定理表明, 對於 x = o \left ( (npq)^{\frac {1}{6}} \right ), 機率 P_{n}(np + x\sqrt {npq}) 的值比較好地位於函數 f(x) = \frac {1}{\sqrt {2\pi npq}}\mathrm {e}^{-\frac {x^{2}}{2}} 的曲線上. 由積分定理可知, 機率 P_{n}(a, b] = \mathop {\mathbf {P}} \left \{ a\sqrt {npq} < S_{n} - np \leq b \sqrt {npq} \right \} = \mathop {\mathbf {P}} \left \{ np + a\sqrt {npq} < S_{n} \leq np + b\sqrt {npq} \right \} 的值可以較好地由積分 \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} 逼近.

記 F_{n}(x) = P_{n}(-\infty, x] = \mathop {\mathbf {P}} \left \{ \frac {S_{n} - np}{\sqrt {npq}} \leq x \right \}. 由 \displaystyle {\sup \limits_{-\infty < a < b < +\infty}\left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \right | \to 0 \ (n \to \infty)} 可見 : \displaystyle {\sup \limits_{-\infty < x < +\infty}\left | F_{n}(x) - \Phi(x) \right | \to 0 \ (n \to \infty)}. 那麼隨著 n 的增長, \sup \limits_{-\infty < a \leq b < +\infty}\left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}\mathrm {e}^{\frac {x^{2}}{2}}\mathrm {d}{x} \right | 和 \sup \limits_{-\infty < x < +\infty}\left | F_{n}(x) - \Phi(x) \right | 兩式趨向於 0 的速度如何? 這裡直接引用 A.C. Berry - C.G. Esseen 定理的特殊情形 : \displaystyle {\sup \limits_{-\infty < x < +\infty} \left | F_{n}(x) - \Phi(x) \right | \leq \frac {p^{2} + q^{2}}{\sqrt {npq}}}. 要特別強調的是, 估計 \frac {1}{\sqrt {npq}} 的量級不能再提高了. 這裡指的是當 p 的值接近於 0 或者 1 的時候, 甚至對於充分大的 n, 用函數 \Phi(x) 來逼近 F_{n}(x) 的效果也可能不太好. 因此, 產生了一個問題 : 當 p 或者 q 比較小的時候, 能否找到一個比局部定理和積分定理給出的更好的逼近呢? 為此, 我們指出, 當 p = \frac {1}{2} 時, 二項分佈 \left \{ P_{n}(k) \right \} 具有對稱的形狀. 不過當 p 比較小的時候, 二項分佈的形狀不是對稱的, 不能指望用常態逼近有好的結果.

3. Poisson 定理

定理 3. (Poisson 定理) 設 \displaystyle {P_{n}(k) = \begin {cases} \binom {k}{n}p^{k}q^{n - k} & {k = 0, 1, 2, …, n} \\ 0 & {k = n + 1, n + 2, …}. \end {cases}} 其中, p 是關於 n 的函數, 即 p = p(n). 當 n \to \infty 時, p(n) \to 0 且 np(n) \to \lambda. 其中, \lambda > 0. 那麼, 對於任意 k = 0, 1, 2, ..., 有 \displaystyle {P_{n}(k) \to \pi_{k} = \frac {\lambda^{k}\mathrm {e}^{-\lambda}}{k!}}.

證明 :

根據條件, 我們構造 \displaystyle {p(n) = \frac {\lambda}{n} + o \left ( \frac {1}{n} \right )}. 因此, 對於固定的 k = 0, 1, 2, ... 和充分大的 n, \displaystyle {\begin {aligned} P_{n}(k) &= \binom {k}{n}p^{k}q^{n - k} \\ &= \frac {n!}{k!(n - k)!}p^{k}q^{n - k} \\ &= \frac {n(n - 1)...(n - k - 1)}{k!}\left ( \frac {\lambda}{n} + o \left ( \frac {1}{n} \right ) \right )^{k}\left ( 1 - \left (\frac {\lambda}{n} + o \left ( \frac {1}{n} \right ) \right ) \right )^{n - k} \\ &= \frac {1}{k!} \cdot \frac {n(n - 1)...(n - k - 1)}{n^{k}}\left ( \lambda + o(1) \right )^{k}\left ( 1 - \left ( \frac {\lambda}{n} + o \left ( \frac {1}{n} \right ) \right ) \right )^{n - k}. \end {aligned}} 對於 \frac {n(n - 1)...(n - k - 1)}{n^{k}}, 有 \displaystyle {\frac {\overbrace {(n - k - 1) \cdot (n - k - 1) \cdot ... \cdot (n - k - 1)}^{k \text { 個}}}{n^{k}} \leq \frac {n(n - 1)...(n - k - 1)}{n^{k}} \leq \frac {\overbrace {n \cdot n \cdot ... \cdot n}^{k \text { 個}}}{n^{k}}}. 由於 \lim \limits_{n \to \infty}\frac {(n - k - 1)^{k}}{n^{k}} = \lim \limits_{n \to \infty}\frac {n^{k} + \sum \limits_{i = 1}^{k}\binom {i}{k}n^{i}(1 - k)^{k - i}}{n^{k}} = 1 且 \lim \limits_{n \to \infty}\frac {n^{k}}{n^{k}} = 1, 由夾擠定理可知 \displaystyle {\lim \limits_{n \to \infty}\frac {n(n - 1)...(n - k - 1)}{n^{k}} = 1}. 那麼當 n \to \infty 時, \displaystyle {\frac {1}{k!} \cdot \frac {n(n - 1)...(n - k - 1)}{n^{k}}\left ( \lambda^{k} + o(1) \right )^{k} \to \lambda^{k}}. 而\displaystyle {\begin {aligned} \lim \limits_{n \to \infty} \left ( 1 - \left (\frac {\lambda}{n} + o \left ( \frac {1}{n} \right ) \right ) \right )^{n - k} &= \lim \limits_{n \to \infty}\left (1 - \frac {\lambda}{n} + o \left ( \frac {1}{n} \right ) \right )^{n - k} \\ &= \lim \limits_{n \to \infty}\left ( 1 + \left (o\left ( \frac {1}{n} \right ) - \frac {\lambda}{n} \right ) \right )^{n - k} \\ &= \lim \limits_{n \to \infty}\left ( 1 + \left ( o \left ( \frac {1}{n} \right ) - \frac {\lambda}{n} \right ) \right )^{(n - k)\frac {o \left ( \frac {1}{n} \right ) - \frac {\lambda}{n}}{o \left ( \frac {1}{n} \right ) - \frac {\lambda}{n}}} \\ &= \mathrm {e}^{\lim \limits_{n \to \infty}(n - k)\left ( o \left ( \frac {1}{n} \right ) - \frac {\lambda}{n} \right )} \\ &= \mathrm {e}^{\lim \limits_{n \to \infty}(n - k) \cdot o \left ( \frac {1}{n} \right ) - (n - k)\frac {\lambda}{n}} \\ &= \mathrm {e}^{\lim \limits_{n \to \infty}-(n - k)\frac {\lambda}{n}} \\ &= \mathrm {e}^{\lim \limits_{n \to \infty}-\lambda + \frac {k}{n}\lambda} \\ &= \mathrm {e}^{-\lambda}. \end {aligned}} 故當 n \to \infty 時, P_{n}(k) \to \frac {\lambda^{k}\mathrm {e}^{-\lambda}}{k!}.

綜上所述, 當 n \to \infty 時, p(n) \to 0 且 np(n) \to \lambda. 其中, \lambda > 0. 那麼, 對於任意 k = 0, 1, 2, ..., 有 \displaystyle {P_{n}(k) \to \pi_{k} = \frac {\lambda^{k}\mathrm {e}^{-\lambda}}{k!}}.

\blacksquare

序列 \left \{ \pi_{k}, k = 0, 1, 2, ... \right \} 滿足 \displaystyle {\pi_{k} > 0 \text { 且 } \sum \limits_{k = 0}^{\infty}\pi_{k} = 1}, 因此可以作為機率分佈, 我們稱之為 Poisson 分佈 (Posson distribution). 之前所討論的所有分佈都是集中在有限個點上, Poisson 分佈時目前遇到的第一個集中在無限個點上的機率分佈.

和之前一樣, 我們仍然考慮當 n \to \infty 時, 機率 P_{n}(k) 收斂於 \pi_{k} 的速度如何? 這裡直接給出一個不經證明的結論 : 若 np(n) = \lambda, 則 \displaystyle {\sum \limits_{k = 0}^{\infty} \left | P_{n}(k) - \pi_{k} \right | \leq \frac {2\lambda}{n}\min \left \{ 2, \lambda \right \}}.

4. De Moivre - Laplace 極限定理和大數法則

De Moivre - Laplace 極限定理和大數法則都是建立在 Bernoulli 概型上的, 那麼是否可以由 De Moivre - Laplace 極限定理導出大數法則呢?

因為 \displaystyle {\begin {aligned} \mathop {\mathbf {P}} \left \{ \left | \frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} &= \mathop {\mathbf {P}} \left \{ \left | \frac {S_{n} - np}{n} \right | \leq \varepsilon \right \} \\ &= \mathop {\mathbf {P}} \left \{ \frac {\left | S_{n} - np \right |}{\sqrt {n} \cdot \sqrt {n}} \leq \varepsilon \right \} \\ &= \mathop {\mathbf {P}} \left \{ \frac {\left | S_{n} - np \right |}{\sqrt {n} \cdot \sqrt {n} \cdot \sqrt {pq}} \leq \frac {\varepsilon}{\sqrt {pq}} \right \} \\ &= \mathop {\mathbf {P}} \left \{ \frac {\left | S_{n} - np \right |}{\sqrt {npq}} \leq \sqrt {\frac {n}{pq}}\varepsilon \right \} \\ &= \mathop {\mathbf {P}} \left \{ \left | \frac {S_{n} - np}{\sqrt {npq}} \right | \leq \sqrt {\frac {n}{pq}}\varepsilon \right \} \\ &= \mathop {\mathbf {P}} \left \{-\varepsilon\sqrt {\frac {n}{pq}} \leq \frac {S_{n} - np}{\sqrt {npq}} \leq \varepsilon\sqrt {\frac {n}{pq}} \right \}, \end {aligned}} 根據 De Moivre - Laplace 極限定理, 當 n \to \infty 時, 有 \displaystyle {\mathop {\mathbf {P}} \left \{ \left | \frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \to \frac {1}{\sqrt {2\pi}}\int_{-\varepsilon\sqrt {\frac {n}{pq}}}^{\varepsilon\sqrt {\frac {n}{pq}}}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \to \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{+\infty}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \to 1}. 即 \mathop {\mathbf {P}} \left \{ \left | \frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \to 1. 這便是大數法則.

由上述的討論, 我們可以得到 \displaystyle {\mathop {\mathbf {P}} \left \{ \left | \frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \to \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{+\infty}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} \ (n \to \infty)}. 然而, Chebyshev 不等式只能給出下面的估計 \displaystyle {\mathop {\mathbf {P}} \left \{ \left | \frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \geq 1 - \frac {pq}{n\varepsilon^{2}}}. 在大數法則中, 為了得到使得 \displaystyle {\mathop {\mathbf {P}} \left \{ \left | \frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \geq 1 - \alpha} 成立所需要的觀測次數, 我們由 Chebyshev 不等式得到了下面的估計 : \displaystyle {n \geq \left \lfloor \frac {1}{4\varepsilon^{2}\alpha} \right \rfloor = n_{1}(\alpha)}. 現在, 我們可以通過 De Moivre - Laplace 極限定理得到更精確的估計. 我們由方程式 \displaystyle {\frac {1}{\sqrt {2\pi}}\int_{-k(\alpha)}^{k(\alpha)}\mathrm {e}^{-\frac {x^{2}}{2}}\mathrm {d}{x} = 1 - \alpha} 求得 k(\alpha). 由於 \displaystyle {\varepsilon\sqrt {\frac {n}{pq}} \geq 2\varepsilon\sqrt {n}} 且根據不等式 \displaystyle {2\varepsilon\sqrt {n} \geq k(\alpha)} 求出最小整數, 得到 \displaystyle {\mathop {\mathbf {P}} \left \{ \left | \frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \geq 1 - \alpha}. 由 2\varepsilon\sqrt {n} \geq k(\alpha) 可見 \displaystyle {n = n_{2}(\alpha) = \left \lfloor \frac {k^{2}(\alpha)}{4\varepsilon^{2}} \right \rfloor} 可以保證 \mathop {\mathbf {P}} \left \{ \left | \frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \geq 1 - \alpha 成立, 其逼近精度可以由 \displaystyle {\sup \limits_{-\infty < x < +\infty}\left | F_{n}(x) - \Phi(x) \right | \leq \frac {p^{2} + q^{2}}{\sqrt {npq}}} 得到.

例題 2. 設 \varepsilon = 0.02, \alpha = 0.05. 則基於 Chebyshev 不等式的估計觀測次數為 12500 次, 而基於極限定理的估計觀測次數為 2500 次.

5. 常態分佈

在 De Moivre - Laplace 積分定理中引入的函數 \displaystyle {\Phi(x) = \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{x}\mathrm {e}^{-\frac {t^{2}}{2}}\mathrm {d}{t}} 在機率輪中有很大的作用, 稱為常態分佈函數 (normal distribution function) 或者 Gauss 分佈函數 (Gauss distribution function). 其導數 \displaystyle {\varphi(x) = \frac {1}{\sqrt {2\pi}}\mathrm {e}^{-\frac {x^{2}}{2}}, x \in R} 稱為常態密度函數 (normal density function) 或者 Gauss 密度函數 (Gauss density function).

常態分佈屬於機率輪中另一種類型的分佈——連續分佈, 而我們之前討論的都是離散型分佈. 在相當一般的條件下, 大量獨立隨機變數 (未必是 Bernoulli 隨機變數) 之和的分佈可以很好地使用常態分佈來逼近.

函數 \varphi(x) = \frac {1}{\sqrt {2\pi}}\mathrm {e}^{-\frac {x^{2}}{2}} 的圖形關於縱軸對稱, 且隨著 \left | x \right | \to \infty 而趨於 0. 而函數 \Phi(x) = \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{x}\mathrm {e}^{-\frac {t^{2}}{2}}\mathrm {d}{t} 隨著 x \to +\infty 而趨於 1, 隨著 x \to -\infty 而趨於 0.

除了 \Phi(x) 之外, 我們還經常使用誤差函數 : \displaystyle {\mathop {\mathrm {erf}}(x) = \frac {2}{\sqrt {\pi}}\int_{0}^{x}\mathrm {e}^{-t^{2}}\mathrm {d}{t}, x > 0}. 顯然, 對於 x > 0, 有 \displaystyle {\Phi(x) = \frac {1}{2}\left ( 1 + \mathop {\mathrm {erf}} \left ( \frac {x}{\sqrt {2}} \right ) \right )} 和 \displaystyle {\mathop {\mathrm {erf}}(x) = 2\Phi \left ( \sqrt {2}x \right ) - 1}.

6. 成功頻率對機率的偏差滿足一定要求的試驗次數

在大數法則中, 我們曾指出, 由 Chebyshev 不等式給出的事件 \left \{ \omega : \left | \frac {S_{n}}{n} - p \right | \geq \varepsilon \right \} 機率的估計是相當粗略的. 這一估計對於非負隨機變數 X, 是由 Chebyshev 不等式 \displaystyle {\mathop {\mathbf {P}} \left \{ X \geq \varepsilon \right \} \leq \frac {\mathop {\mathrm {E}} \left (X^{2} \right )}{\varepsilon^{2}}} 得到的. 不過可以利用 Chebyshev 不等式的另一種形式 : \displaystyle {\mathop {\mathbf {P}} \left \{ X \geq \varepsilon \right \} = \mathop {\mathbf {P}} \left \{ X^{2k} \geq \varepsilon^{2k} \right \} \leq \frac {\mathop {\mathrm {E}} \left ( X^{2k} \right )}{\varepsilon^{2k}}}. 然而, 還可以更進一步, 利用 Chebyshev 不等式的指數形式 : 對於 X \geq 0 和 \lambda > 0, \displaystyle {\mathop {\mathbf {P}} \left \{ X \geq \varepsilon \right \} = \mathop {\mathbf {P}} \left \{ \mathrm {e}^{\lambda x} \geq \varepsilon^{\lambda x} \right \} \leq \frac {\mathop {\mathrm {E}} \left ( \mathrm {e}^{\lambda x} \right )}{\mathrm {e}^{\lambda\varepsilon}} = \frac {\mathop {\mathrm {E}} \left ( \mathrm {e}^{\lambda x} \right )}{\mathop {\mathrm {E}} \left ( \mathrm {e}^{\lambda\varepsilon} \right )} = \mathop {\mathrm {E}} \left ( \frac {\mathrm {e}^{\lambda x}}{\mathrm {e}^{\lambda\varepsilon}} \right ) = \mathop {\mathrm {E}} \left ( \mathrm {e}^{\lambda(X - \varepsilon)} \right )}. 由於 \lambda > 0 的任意性, 可見 \displaystyle {\mathop {\mathbf {P}} \left \{ X \geq \varepsilon \right \} \leq \inf \limits_{\lambda > 0}\mathop {\mathrm {E}} \left ( \mathrm {e}^{\lambda(X - \varepsilon)} \right )}.

接下來, 我們討論當 X = \frac {S_{n}}{n}, S_{n} = \xi_{1} + \xi_{2} + ... + \xi_{n}, \mathop {\mathbf {P}} \left \{ \xi_{i} = 1 \right \} = p, \mathop {\mathbf {P}} \left \{ \xi_{i} = 0 \right \} = q, i = 1, 2, ..., n 時, 沿此路徑將會導致的結果.

記 \varphi(\lambda) = \mathop {\mathrm {E}} \left ( \mathrm {e}^{\lambda \xi_{1}} \right ), 則 \displaystyle {\begin {aligned} \varphi(\lambda) &= \left . \mathrm {e}^{\lambda \xi_{1}} \right |_{\xi_{1} = 0}\mathop {\mathbf {P}} \left \{ \xi_{1} = 0 \right \} + \left . \mathrm {e}^{\lambda \xi_{1}} \right |_{\xi_{1} = 1}\mathop {\mathbf {P}} \left \{ \xi_{1} = 1 \right \} \\ &= 1 \times q + \mathrm {e}^{\lambda} \times p \\ &= 1 - p + p\mathrm {e}^{\lambda}. \end {aligned}} 對於兩兩獨立的隨機變數 \xi_{1}, \xi_{2}, ..., \xi_{n}, 則有 \displaystyle {\begin {aligned} \mathop {\mathrm {E}} \left ( \mathrm {e}^{\lambda S_{n}} \right ) &= \mathop {\mathrm {E}} \left ( \mathrm {e}^{\lambda(\xi_{1} + \xi_{2} + ... + \xi_{n})} \right ) \\ &= \mathop {\mathrm {E}} \left ( \mathrm {e}^{\lambda\xi_{1} + \lambda\xi_{2} + ... + \lambda\xi_{n}} \right ) \\ &= \mathop {\mathrm {E}} \left ( \mathrm {e}^{\lambda\xi_{1}}\mathrm {e}^{\lambda\xi_{2}}...\mathrm {e}^{\lambda\xi_{n}} \right ) \\ &= \mathop {\mathrm {E}} \left ( \mathrm {e}^{\lambda\xi_{1}} \right )\mathop {\mathrm {E}} \left ( \mathrm {e}^{\lambda\xi_{2}} \right )...\mathop {\mathrm {E}} \left ( \mathrm {e}^{\lambda\xi_{n}} \right ) \\ &= \mathrm {e}^{n} \left ( \mathrm {e}^{\lambda\xi_{1}} \right ) \\ &= \varphi^{n}(\lambda). \end {aligned}} 因此, 對於 0 < a < 1, 根據 Chebyshev 不等式的指數形式, 有 \displaystyle {\begin {aligned} \mathop {\mathbf {P}} \left \{ \frac {S_{n}}{n} \geq a \right \} &\leq \inf \limits_{\lambda > 0}\mathop {\mathrm {E}} \left ( \mathrm {e}^{\lambda \left ( \frac {S_{n}}{n} - a \right )} \right ) \\ &= \inf \limits_{\lambda > 0}\mathop {\mathrm {E}} \left ( \mathrm {e}^{\left ( \frac {\lambda}{n}S_{n} - \lambda a \right )} \right ) = \inf \limits_{\lambda > 0}\mathop {\mathrm {E}} \left ( \frac {\mathrm {e}^{\frac {\lambda}{n}S_{n}}}{\mathrm {e}^{\lambda a}} \right ) = \inf \limits_{\lambda > 0}\frac {\mathop {\mathrm {E}} \left ( \mathrm {e}^{\frac {\lambda}{n}S_{n}} \right )}{\mathrm {e}^{\lambda a}} \\ &= \inf \limits_{\lambda > 0}\frac {\varphi^{n} \left ( \frac {\lambda}{n} \right )}{\mathrm {e}^{\lambda a}} = \inf \limits_{\lambda > 0}\mathrm {e}^{-\lambda a} \cdot \varphi^{n} \left ( \frac {\lambda}{n} \right ) = \inf \limits_{\lambda > 0} \mathrm {e}^{-\lambda a}\mathrm {e}^{\ln {\varphi^{n} \left ( \frac {\lambda}{n} \right )}} \\ &= \inf \limits_{\lambda > 0}\mathrm {e}^{-\lambda a + n \ln {\varphi \left ( \frac {\lambda}{n} \right )}} = \inf \limits_{\lambda > 0}\mathrm {e}^{-n \left ( \frac {\lambda}{n}a - \ln {\varphi \left ( \frac {\lambda}{n} \right )} \right )} = \inf \limits_{\lambda > 0}\mathrm {e}^{-n \left (a\mu - \ln {\varphi(\mu)} \right )} \\ &= \mathrm {e}^{-n\sup \limits_{\mu > 0}a\mu - \ln {\varphi(\mu)}}. \end {aligned}} 其中, \mu = \frac {\lambda}{n}. 類似地, 也有 \displaystyle {\mathop {\mathbf {P}} \left \{ \frac {S_{n}}{n} \leq a \right \} \leq \mathrm {e}^{-n\sup \limits_{\mu < 0}a\mu - \ln {\varphi(\mu)}}}. 當 p \leq a \leq 1 時, 記函數 f(x) = ax - \ln {\left ( 1 - p + p\mathrm {e}^{x} \right )}, 則 f'(x) = a - \frac {p\mathrm {e}^{x}}{1 - p + p\mathrm {e}^{2}}. 令 f'(x) = 0, 得 \displaystyle {\mathrm {e}^{x} = \frac {a(1 - p)}{p(1 - a)}}. 若 x = x_{0} 使得 f(x) 取得最大值, 則點 x_{0} 滿足 \mathrm {e}^{x_{0}} = \frac {a(1 - p)}{p(1 - a)}. 因此, \displaystyle {\sup \limits_{\mu > 0}f(\mu) = H(a)}. 其中, H(a) = a\ln {\frac {a}{p}} + (1 - a)\ln {\frac {1 - a}{1 - p}}. 這樣, 當 p \leq a \leq 1 時, \displaystyle {\mathop {\mathbf {P}} \left \{ \frac {S_{n}}{n} \geq a \right \} \leq \mathrm {e}^{-n\sup \limits_{\mu > 0}f(\mu)} = \mathrm {e}^{-nH(a)}.\ \ \ \ \ \ \ \ \ \ (\mathrm {II})} 而由於 H(p + x) \geq 2x^{2}, 0 \leq p + x \leq 1, 那麼對於任何 \varepsilon > 0 和 0 \leq p \leq 1, 有 \displaystyle {\mathop {\mathbf {P}} \left \{ \frac {S_{n}}{n} - p \geq \varepsilon \right \} \leq \mathrm {e}^{-2n\varepsilon^{2}}.\ \ \ \ \ \ \ \ \ \ (\mathrm {III})} 類似可得, 當 a \leq p \leq 1 時, \displaystyle {\mathop {\mathbf {P}} \left \{ \frac {S_{n}}{n} \leq a \right \} \leq \mathrm {e}^{-nH(a)}}.\ \ \ \ \ \ \ \ \ \ (\mathrm {IV}) 從而對於任意 \varepsilon > 0 和 0 \leq p \leq 1, 有 \displaystyle {\mathop {\mathbf {P}} \left \{ \frac {S_{n}}{n} - p \leq -\varepsilon \right \} \leq \mathrm {e}^{-2n\varepsilon^{2}}.\ \ \ \ \ \ \ \ \ \ (\mathrm {V})} 於是, \mathop {\mathbf {P}} \left \{ \left | \frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \leq 2\mathrm {e}^{-2n\varepsilon^{2}}. 由此可見, 對於任意 0 \leq p \leq 1, 保證不等式 \displaystyle {\mathop {\mathbf {P}} \left \{ \left | \frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \geq 1 - \alpha} 成立的估計觀測次數 n_{3}(\alpha) 決定於 \displaystyle {n_{3}(\alpha) = \left \lfloor \frac {\ln {\frac {2}{\alpha}}}{2\varepsilon^{2}} \right \rfloor}. 若不取整數部分, 直接將 n_{3}(\alpha) 於 n_{1}(\alpha) = \left \lfloor \frac {1}{4\alpha\varepsilon^{2}} \right \rfloor 比較可見 : \displaystyle {\frac {n_{1}(\alpha)}{n_{3}(\alpha)} = \frac {\frac {1}{4\alpha\varepsilon^{2}}}{\frac {\ln {\frac {2}{\alpha}}}{2\varepsilon^{2}}}} = \frac {1}{2\alpha\ln {\frac {2}{\alpha}}} \to +\infty \ (\alpha \to 0^{+}). 由此可見, 當 \alpha \to 0^{+} 時, 由 Chebyshev 不等式的指數形式估計最小必須要觀測的次數, 比用一般的 Chebyshev 不等式估計的次數更加精確, 特別是對於比較小的 \alpha.

利用數學分析中的結論 \displaystyle {\lim \limits_{x \to -\infty}\frac {1}{\sqrt {2\pi}}\int_{x}^{+\infty}\mathrm {e}^{-\frac {t^{2}}{2}}\mathrm {d}{t} = \frac {1}{\sqrt {2\pi}x}\mathrm {e}^{-\frac {x^{2}}{2}}}, 可以證明當 \alpha \to o^{+} 時, k^{2}(\alpha) \to 2 \ln {\frac {2}{\alpha}}. 於是有 \displaystyle {\lim \limits_{\alpha \to 0^{+}}\frac {n_{2}(\alpha)}{n_{3}(\alpha)} = 1}. 式 (\mathrm {II}), (\mathrm {III}), (\mathrm {IV}), (\mathrm {V}) 和式 \displaystyle {\mathop {\mathbf {P}} \left \{ \left | \frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \leq 2\mathrm {e}^{-2n\varepsilon^{2}}}\ \ \ \ \ \ \ \ \ \ (\mathrm {VI}) 合稱為大偏差機率的不等式 (inequalities for the probability of large deviations). 利用 De Moivre - Laplace 定理, 可以簡單地對事件 \left \{ \left | S_{n} - np \right | \leq x\sqrt {n} \right \} 的機率進行估計. 此事件表示 S_{n} 對 np (數量級為 \sqrt {n}) 的標準離差 (standard deviation). 而不等式 (\mathrm {III}), (\mathrm {V}) 和 (\mathrm {VI}) 對 \displaystyle {\left \{ \left | S_{n} - np \right | \leq x\sqrt {n} \right \}} 給出的估計描繪量級大於 \sqrt {n} 的離差, 其數量級為 n.

7. 練習題

自主習題 1. 設 p = \frac {1}{2}, 而 Z_{n} = 2S_{n} - n (n 組試驗中的 1 比 0 多出的數量). 證明 : \displaystyle {\sup \limits_{j}\left | \sqrt {\pi n}\mathop {\mathbf {P}} \left \{ Z_{2n} = j - \mathrm {e}^{-\frac {j^{2}}{4^{n}}} \right \} \right | \to 0 \ (n \to \infty)}.

自主習題 2. 證明 : Poisson 定理對於 p = \frac {\lambda}{n} 的收斂速度為 \displaystyle {\sup \limits_{k}\left | P_{n}(k) - \frac {\lambda^{k}\mathrm {e}^{-\lambda}}{k!} \right | \leq \frac {\lambda^{2}}{n}}.