De Moivre-Laplace 局部極限定理

考慮隨機變數 \xi_{1}, \xi_{2}, ..., \xi_{n}, 記 S_{n} = \xi_{1} + \xi_{2} + ... + \xi_{n}, 有 E(\frac {S_{n}}{n}) = p. 由於 \text {Var}(S_{n}) = npq, 而

\displaystyle {\begin {aligned} \text {Var}(S_{n}) &= E \left ((S_{n} - E(S_{n}))^{2} \right ) = E \left ( (S_{n} - np)^{2} \right ) \\ &= E \left ( \left (\frac {S_{n} - np}{n} \cdot n \right )^{2} \right ) = E \left ( n^{2} \left (\frac {S_{n} - np}{n} \right )^{2} \right ) \\ &= n^{2}E \left ( \left (\frac {S_{n}}{n} - p \right )^{2} \right ) = npq \end {aligned}}

故有 E \left ( \left (\frac {S_{n}}{n} - p \right )^{2} \right ) = \frac {pq}{n}

我們已經在大數法則中得到了 \frac {Sn}{n}p 之間的機率意義. 那麼, 我們自然希望得到 \left (\frac {S_{n}}{n} - p \right )^{2}\frac {pq}{n} 的機率意義. 例如, 考慮下面這個機率如何 :

\displaystyle {P \left \{ \left |\frac {S_{n}}{n} - p \right | \leq x\sqrt {\frac {pq}{n}} \right \}, x \in R}

又由於 E(S_{n}) = np, \text {Var}(S_{n}) = npq, 而

\displaystyle {\frac {\left | \frac {S_{n}}{n} - p \right |}{\sqrt {\frac {pq}{n}}} = \frac {\left | \frac {S_{n}}{n} - p \right |}{\sqrt {\frac {pq}{n} \cdot \frac {n}{n}}} = \frac {\left |\sqrt {n^{2}}\frac {S_{n} - np}{n} \right |}{\sqrt {\frac {pq}{n}}} = \frac {\left |S_{n} - np \right |}{\sqrt {npq}} = \frac {\left |S_{n} - E(S_{n}) \right|}{\sqrt {\text {Var}(S_{n})}}}

我們又可以考慮

\displaystyle {P \left \{ \left |\frac {S_{n} - E(S_{n})}{\sqrt {\text {Var}(S_{n})}} \right | \leq x \right \}}

如果對於 n \geq 1, 記 P_{n}(k) = \binom {k}{n}p^{k}q^{n - k}\ (k = 0, 1, 2, ..., n), 則有

P \left \{ \left |\frac {S_{n} - E(S_{n})}{\sqrt {\text {Var}(S_{n})}} \right | \leq x \right \} = \sum \limits_{\left \{ k : \frac {\left |k - np \right |}{\sqrt {npq}} \leq x \right \}}P_{n}(k)

那麼當 n \to \infty 時, 機率 P_{n}(k) 如何?

定理 1. (局部極限定理) 記 P_{n}(k) = \binom {k}{n}p^{k}q^{n - k}\ (k = 0, 1, 2, ..., n), 設 0 < p < 1, 則對滿足 \left |k - np \right | = o \left ((npq)^{\frac {2}{3}} \right ) 的所有 k, 一致有

\displaystyle {P_{n}(k) \to \frac {1}{\sqrt {2\pi npq}}e^{-\frac {(k - np)^{2}}{2npq}}\ (n \to \infty)}

即當 n \to \infty 時,

\displaystyle {\sup \limits_{\left \{ k : \left |k - np \right | \leq \varphi(n) \right \}} \left \{ \left |\frac {P_{n}(k)}{\frac {1}{\sqrt {2\pi npq}}e^{-\frac {(k - np)^{2}}{2npq}}} - 1 \right | \right \} \to 0}

其中, \varphi(n) 是任意滿足 \varphi(n) = o \left ((npq)^{\frac {2}{3}} \right ) 的非負函數

:

根據 Stering 公式, 有

\displaystyle {n! = \sqrt {2\pi n}\frac {n^{n}}{e^{n}}(1 + R(n))}

其中, 當 n \to \infty 時, R(n) \to 0. 若 n \to \infty 時, k \to \infty, n - k \to \infty, 則

\displaystyle {\begin {aligned} \binom {k}{n} &= \frac {n!}{k!(n - k)!} \\ &= \frac {\sqrt {2\pi n}\left (\frac {n}{e} \right )^{n}(1 + R(n))}{\sqrt {2\pi k \times 2\pi (n - k)}\left ( \frac {k}{e} \right )^{k}\left ( \frac {n - k}{e} \right )^{n - k}(1 + R(k))(1 + R(n - k))} \\ &= \frac {\frac {n^{n}}{e^{n}}(1 + \varepsilon(n, k, n - k))}{\frac {1}{\sqrt {n}}\sqrt {2\pi k(n - k)}\frac {k^{k}}{e^{k}}\frac {(n - k)^{n - k}}{e^{n - k}}} \\ &= \frac {n^{n}(1 + \varepsilon(n, k, n - k))}{\sqrt {2\pi\frac {k}{n}(n - k)}k^{k}(n - k)^{n - k}} \\ &= \frac {n^{n}(1 + \varepsilon(n, k, n - k))}{\sqrt {2\pi\frac {k}{n}(n - k)}n^{k} \left (\frac {k}{n} \right )^{k}n^{n - k} \left ( 1 - \frac {k}{n} \right )^{n - k}} \\ &= \frac {1}{\sqrt {2\pi n\frac {k}{n}\left (1 - \frac {k}{n} \right )}} \times \frac {1 + \varepsilon(n, k, n - k)}{\left ( \frac {k}{n} \right )^{k}\left (1 - \frac {k}{n} \right )^{n - k}} \end {aligned}}

其中, 當 n \to \infty, k \to \infty, n - k \to \infty 時, \varepsilon(n, k, n - k) \to 0. 故有

\displaystyle {P_{n}(k) = \binom {k}{n}p^{k}q^{n - k} = \frac {p^{k}(1 - p)^{n - k} \left (1 + \varepsilon(n, k, n - k) \right )}{\sqrt {2\pi n\frac {k}{n} \left (1 - \frac {k}{n} \right ) \left (\frac {k}{n} \right )^{k} \left (1 - \frac {k}{n} \right )^{n - k}}}}

\hat {p} = \frac {k}{n}, 則

\displaystyle {\begin {aligned} P_{n}(k) &= \binom {k}{n}p^{k}q^{n - k} = \frac {p^{k}(1 - p)^{n - k} \left (1 + \varepsilon(n, k, n - k) \right )}{\sqrt {2\pi n\hat {p} \left (1 - \hat {p} \right ) \hat {p}^{k} \left (1 - \hat {p} \right )^{n - k}}} \\ &= \frac {1 + \varepsilon(n, k, n - k)}{\sqrt {2\pi n\hat {p}\left (1 - \hat {p} \right )}}\left ( \frac {p}{\hat {p}} \right )^{k}\left (\frac {1 - p}{1 - \hat {p}} \right )^{n - k} \\ &= \frac {1 + \varepsilon(n, k, n - k)}{\sqrt {2\pi n\hat {p}\left (1 - \hat {p} \right )}} \exp \left \{ k\ln {\frac {p}{\hat {p}} + (n - k)\ln {\frac {1 - p}{1 - \hat {p}}}} \right \} \\ &= \frac {1 + \varepsilon(n, k, n - k)}{\sqrt {2\pi n\hat {p}\left (1 - \hat {p} \right )}} \exp \left \{n \left (\frac {k}{n}\ln {\frac {p}{\hat {p}}} + \left ( 1- \frac {k}{n} \right )\ln {\frac {1 - p}{1 - \hat {p}}} \right )\right \} \\&= \frac {1 + \varepsilon(n, k, n - k)}{\sqrt {2\pi n\hat {p}\left (1 - \hat {p} \right )}}\exp \left \{-nH(\hat {p}) \right \} \\ &= \frac {1 + \varepsilon(n, k, n - k)}{\sqrt {2\pi n\hat {p}\left (1 - \hat {p} \right )}} \cdot \frac {1}{e^{nH(\hat {p})}} \end {aligned}}

其中, H(x) = x\ln {\frac {x}{p}} + (1 - x)\ln {1 - x}{1 - p}

由於 k 滿足 |k - np| = o \left ((npq)^{\frac {2}{3}} \right ), 故當 k \to \infty 時, p - \hat {p} \to 0. 另外, 對於 0 < x < 1, 有

\displaystyle {H'(x) = \ln {\frac {x}{p}} - \ln {1 - x}{1 - p}, H''(x) = \frac {1}{x} + \frac {1}{1 - x}, H'''(x) = -\frac {1}{x^{2}} + \frac {1}{(1 - x)^{2}}}

H(\hat {p}) 表示為 H(p + (\hat {p} - p)), 根據 Taylor 公式, 當 n 充分大的時候, 有

\displaystyle {\begin {aligned} H(\hat {p}) &= H(p) + H'(p)(\hat {p} - p) + \frac {1}{2}H''(p)(\hat {p} - p)^{2} + o(\left |\hat {p} - p \right |^{3}) \\ &= \frac {1}{2}\left (\frac {1}{p} + \frac {1}{q} \right )\left (\hat {p} - p \right )^{2} + o(\left |\hat {p} - p \right |^{3}) \end {aligned}}

從而有

\displaystyle {P_{n}(k) = \frac {1 + \varepsilon(n, k, n - k)}{\sqrt {2\pi n\hat {p}(1 - \hat {p})}}\exp \left \{ -\frac {n}{2pq}\left (\hat {p} - p \right )^{2} + n \cdot o \left (\left |\hat {p} - p \right |^{3} \right ) \right \}}

注意到

\displaystyle {\frac {n}{2pq}\left (\hat {p} - p \right )^{2} = \frac {n}{2pq}\left (\frac {k}{n} - p \right )^{2} = \frac {(n - kp)^{2}}{2npq}}

故有

\displaystyle {P_{n}(k) \to \frac {1}{\sqrt {2\pi npq}}e^{-\frac {(k - np)^{2}}{2npq}}(1 + \varepsilon'(n, k, n - k))}

其中, 1 + \varepsilon'(n, k, n - k) = 1 + \varepsilon(n, k, n - k)e^{n \cdot o(\left |\hat {p} - p \right |^{3})}\sqrt {\frac {p(1 - p)}{\hat {p}(1 - \hat {p})}}

顯然, 當 n \to \infty 時, \sup \left |\varepsilon'(n, k, n - k) \right | \to 0. 因此, 有

\displaystyle {P_{n}(k) \to \frac {1}{\sqrt {2\pi npq}}e^{-\frac {(k - np)^{2}}{2npq}}\ (n \to \infty)}

其中, |k - np| \leq \varphi(n), \varphi(n) = o \left ((npq)^{\frac {2}{3}} \right )

\blacksquare

推論 1. 對於一切 x \in R, 若 x = o \left ((npq)^{\frac {1}{6}} \right ), 而 np + x\sqrt {npq} 的結果是集合 \left \{ 0, 1, 2, ..., n \right \} 中的整數, 則

\displaystyle {P_{n}(np + x\sqrt {npq}) \to \frac {1}{\sqrt {2\pi npq}}e^{-\frac {x^{2}}{2}}\ (n \to \infty)}

即當 n \to \infty 時, 有

\displaystyle {\sup \limits_{\left \{ x : |x| \leq \psi(n) \right \}} \left \{ \left |\frac {P_{n}(np + x\sqrt {npq})}{\frac {1}{\sqrt {2\pi npq}}e^{-\frac {x^{2}}{2}}} - 1 \right | \right \} \to 0}

其中, \psi(x) = o \left ((npq)^{\frac {1}{6}} \right )

在大數法則中, 我們曾給出 P \left \{ \left |\frac {S_{n}}{n} - p \right | \geq \varepsilon \right \} \to 0\ (n \to \infty) 以質點游動的解釋. 引入上述理論之後, 我們可以將之前的解釋描述為

\displaystyle {P \left \{ S_{n} = k \right \} \to \frac {1}{\sqrt {2\pi npq}}e^{-\frac {(k - np)^{2}}{2npq}}, |k - np| = o \left ((npq)^{\frac {2}{3}} \right )}

\displaystyle {P \left \{ \frac {S_{n} - np}{\sqrt {npq}} = x \right \} \to \frac {1}{\sqrt {2\pi npq}e^{-\frac {x^{2}}{2}}}, x = o \left ((npq)^{\frac {1}{6}} \right )\ \ \ \ \ \ \ \ \ \ (I)}

其中, P \left \{ \frac {S_{n} - np}{\sqrt {npq}} = x \right \} 中的 np + x\sqrt {npq} \in \left \{ 0, 1, 2, ..., n \right \}

若設

\displaystyle {t_{k} = \frac {k - np}{\sqrt {npq}}, \Delta t_{k} = t_{k + 1} - t_{k} = \frac {1}{\sqrt {npq}}}

那麼通過置換可以將 (I) 式改寫為,

\displaystyle {P \left \{ \frac {S_{n} - E(S_{n})}{\sqrt {\text {Var}(S_{n})}} \right \} = t_{k} \to \frac {\Delta t_{k}}{\sqrt {2\pi}}e^{-\frac {t_{k}^{2}}{2}}, t_{k} = o \left ((npq)^{\frac {1}{6}} \right )}

顯然, 當 n \to \infty 時,

\displaystyle {\Delta t_{k} = \frac {1}{\sqrt {npq}} \to 0}

根據上述描述, 我們發現 e^{-\frac {t_{k}^{2}}{2}} \cdot \Delta t_{k} 與定積分的定義類似, 於是我們自然想到

\displaystyle {P \left \{ a < \frac {S_{n} - E(S_{n})}{\sqrt {\text {Var}(S_{n})}} \leq b \right \} = t_{k} \to \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x, -\infty < a \leq b < +\infty}

接下來, 我們將給出確切的表述

De Moivre-Laplace 極限定理

對於 -\infty < a \leq b < +\infty, 設

\displaystyle {P_{n}(a, b] = \sum \limits_{a < x \leq b}P_{n}(np + x\sqrt {npq})}

其中, 求和是針對一切使得 np + x\sqrt {npq} 為整數的 x 求和. 由局部極限定理可見, 對於 k = np + t_{k}\sqrt {npq} 決定且滿足 |t_{k}| \leq T < +\inftyt_{k}, 由推論 1

\displaystyle {P_{n}(np + t_{k}\sqrt {npq}) \to \frac {t_{k}}{\sqrt {2\pi}}e^{-\frac {t_{k}^{2}}{2}}(1 + \varepsilon(t_{k}, n))}

其中, \sup \limits_{|t_{k}| \leq T} \left \{ \left |\varepsilon(t_{k}, n) \right | \right \} \to 0\ (n \to \infty). 從而對於固定的 ab, 其中 -\infty < -T \leq a \leq b \leq T < +\infty, 有

\displaystyle {\begin {aligned} \sum \limits_{a < t_{k} \leq b}P_{n}(np + t_{k}\sqrt {npq}) &= \sum \limits_{a < t_{k} \leq b}\frac {t_{k}}{\sqrt {2\pi}}e^{-\frac {t_{k}^{2}}{2}} + \sum \limits_{a < t_{k} \leq b}\varepsilon(t_{k}, n)\frac {\Delta t_{k}}{\sqrt {2\pi}}e^{-\frac {t_{k}^{2}}{2}} \\ &= \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x + R_{n}^{(1)}(a, b) + R_{n}^{(2)}(a, b) \end {aligned}}

其中, R_{1}^{(n)} = \sum \limits_{a < t_{k} \leq b}\frac {\Delta t_{k}}{\sqrt {2\pi}}e^{-\frac {t_{k}^{2}}{2}} - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x, R_{n}^{(2)} = \sum \limits_{a < t_{k} \leq b}\varepsilon(t_{k}, n)\frac {\Delta t_{k}}{\sqrt {2\pi}}e^{-\frac {t_{k}^{2}}{2}}

對於 R_{n}^{(1)}(a, b), 顯然地, 當 n \to \infty 時, \sup \limits_{-T \leq a \leq b \leq T} \left \{ \left |R_{n}^{(1)}(a, b) \right | \right \} \to 0

為了繼續導出結論, 我們引入數學分析中的事實 :

引理 1. \displaystyle {\frac {1}{\sqrt {2\pi}}\int_{-\infty}^{+\infty}e^{-\frac {x^{2}}{2}}\mathrm {d}x = 1}

這個引理我們暫時不進行證明, 詳見之後發布的《分析》系列的文章

由於 f(x) = e^{-\frac {x^{2}}{2}} 是連續函數且 f(x) > 0, 故必有

\displaystyle {\frac {1}{\sqrt {2\pi}}\int_{-T}^{T}e^{-\frac {x^{2}}{2}}\mathrm {d}x \leq \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{+\infty}e^{-\frac {x^{2}}{2}}\mathrm {d}x \leq 1}

因此,

\displaystyle {\begin {aligned} &\sup \limits_{-T \leq a \leq b \leq T}\left \{ \left |R_{n}^{(2)}(a, b) \right | \right \} \\ &= \sup \limits_{-T \leq a \leq b \leq T}\left \{ \left | \sum \limits_{a < t_{k} \leq b}\varepsilon(t_{k}, n)\frac {\Delta t_{k}}{\sqrt {2\pi}}e^{-\frac {t_{k}^{2}}{2}} \right | \right \} \\ &= \sup \limits_{-T \leq a \leq b \leq T}\left \{ \left | \sum \limits_{a < t_{k} \leq b}\varepsilon(t_{k}, n) \right |\frac {\Delta t_{k}}{\sqrt {2\pi}}e^{-\frac {t_{k}^{2}}{2}} \right \} \\ &\leq \sup \limits_{|t_{k}| \leq T} \left \{ \left |\varepsilon(t_{k}, n) \right | \sum \limits_{|t_{k}| \leq T}\frac {\Delta t_{k}}{\sqrt {2\pi}}e^{-\frac {t_{k}^{2}}{2}} \right \} \\ &= \sup \limits_{-T \leq a \leq b \leq T} \left \{ \left |\varepsilon(t_{k}, n) \right | \left ( R_{1}^{(n)}(a, b) + \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x \right ) \right \} \\ &= \sup \limits_{-T \leq a \leq b \leq T} \left \{ \left |\varepsilon(t_{k}, n) \right | \right \}\sup \limits_{-T \leq a \leq b \leq T} \left \{ R_{1}^{(n)}(a, b) + \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x \right \} \\ &= \sup \limits_{-T \leq a \leq b \leq T} \left \{ \left |\varepsilon(t_{k}, n) \right | \right \} \end {aligned}}

所以, 當 n \to \infty 時, \sup \limits_{-T \leq a \leq b \leq T} \left \{ \left |R_{n}^{(2)}(a, b) \right | \right \} \to 0

\Phi(x) = \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{x}e^{-\frac {t^{2}}{2}}dt, 由上述過程可見 :

\displaystyle {\sup \limits_{-T \leq a \leq b \leq T} \left \{ \left |P_{n}(a, b] - (\Phi(b) - \Phi(a)) \right | \right \} \to 0\ (n \to \infty)}

我們已經證明, 對於有限的 T, 上式成立. 那麼對於 T \to \infty, 上式是否也成立呢?

顯然, 對於任意給定的 \varepsilon > 0, 存在有限的 T = T(\varepsilon) 使得

\displaystyle {\frac {1}{\sqrt {2\pi}}\int_{-T}^{T}e^{-\frac {x^{2}}{2}}\mathrm {d}x > 1 - \frac {\varepsilon}{4}}

此時, 對於任意 \varepsilon > 0, 存在正整數 N, 使得對於一切 n > NT = T(\varepsilon), 有

\displaystyle {\sup \limits_{-T \leq a \leq b \leq T}\left \{ \left |P_{n}(a, b] - (\Phi(b) - \Phi(a)) \right | \right \} < \frac {\varepsilon}{4}}

結合 P(a, b] 的定義與 \frac {1}{2}\int_{-T}^{T}e^{-\frac {x^{2}}{2}}\mathrm {d}x > 1 - \frac {\varepsilon}{4}, 必定有

\displaystyle {P_{n}(-T, T] > 1 - \frac {\varepsilon}{2}}

P_{n}(-\infty, -T] + P_{n}(-T, T] + P_{n}(T, +\infty) = 1. 因此,

\displaystyle {P_{n}(-\infty, -T] + P_{n}(T, +\infty) \leq \frac {\varepsilon}{2}}

其中, P_{n}(-\infty, T] = \lim \limits_{S \to -\infty}P_{n}(S, T], P_{n}(T, +\infty) = \lim \limits_{S \to +\infty}P_{n}(T, S]. 這樣, 對於任意 -\infty < a \leq -T \leq T \leq b < +\infty, 根據絕對值不等式, 有

\displaystyle {\begin {aligned} \left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x \right | &= \Bigg | \left (P_{n}(a, -T) - \frac {1}{\sqrt {2\pi}}\int_{a}^{-T} e^{\frac {x^{2}}{2}}\mathrm {d}x \right ) + \\ &\ \ \ \ \ \ \ \left (P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T} e^{-\frac {x^{2}}{2}}\mathrm {d}x \right ) + \\ &\ \ \ \ \ \ \ \left ( P_{n}(T, b] - \frac {1}{\sqrt {2\pi}}\int_{T}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x  \right ) \Bigg | \\ &\leq \left | P_{n}(a, -T] - \frac {1}{\sqrt {2\pi}}\int_{a}^{-T}e^{-\frac {x^{2}}{2}}\mathrm {d}x \right | + \\ &\ \ \ \ \ \left |P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}e^{-\frac {x^{2}}{2}}\mathrm {d}x \right | + \\ &\ \ \ \ \ \left | P_{n}(T, b] - \frac {1}{\sqrt {2\pi}}\int_{T}^{b}e^{\frac {x^{2}}{2}}\mathrm {d}x \right | \end {aligned}}

顯然地, 我們有

\displaystyle {P_{n}(a, -T] \leq P_{n}(-\infty, -T], P_{n}(T, b] \leq P_{n}(T, +\infty)}

除此之外, 根據絕對值不等式, 我們又得

\displaystyle {\left | P_{n}(a, -T] - \frac {1}{\sqrt {2\pi}}\int_{a}^{-T}e^{\frac {x^{2}}{2}}\mathrm {d}x \right |} \leq \left | P_{n}(a, -T] \right | + \left | \frac {1}{\sqrt {2\pi}\int_{a}^{-T}e^{-\frac {x^{2}}{2}}}\mathrm {d}x \right |

\displaystyle {\left | P_{n}(T, b] - \frac {1}{\sqrt {2\pi}}\int_{T}^{b}e^{\frac {x^{2}}{2}}\mathrm {d}x \right |} \leq \left | P_{n}(a, -T] \right | + \left | \frac {1}{\sqrt {2\pi}\int_{a}^{-T}e^{-\frac {x^{2}}{2}}}\mathrm {d}x \right |

因此,

\displaystyle {\begin {aligned} \left |P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x \right | &\leq \left |P_{n}(-\infty, -T] \right | + \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{-T}e^{-\frac {x^{2}}{2}}\mathrm {d}x + \\ &\ \ \ \ \ \left |P_{n}(T, +\infty) \right | + \frac {1}{\sqrt {2\pi}}\int_{T}^{+\infty}e^{-\frac {x^{2}}{2}}\mathrm {d}x + \\ &\ \ \ \ \ \left | P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}e^{-\frac {x^{2}}{2}}\mathrm {d}x \right | \end {aligned}}

因為 P_{n}(-\infty, -T] \geq 0, P_{n}(T, +\infty) \geq 0, 於是

\displaystyle {\begin {aligned} \left |P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x \right | &\leq P_{n}(-\infty, -T] + \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{-T}e^{-\frac {x^{2}}{2}}\mathrm {d}x + \\ &\ \ \ \ \ P_{n}(T, +\infty) + \frac {1}{\sqrt {2\pi}}\int_{T}^{+\infty}e^{-\frac {x^{2}}{2}}\mathrm {d}x + \\ &\ \ \ \ \ \left | P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}e^{-\frac {x^{2}}{2}}\mathrm {d}x \right | \end {aligned}}

由於 \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{-T}e^{-\frac {x^{2}}{2}}\mathrm {d}x + \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}e^{-\frac {x^{2}}{2}}\mathrm {d}x + \frac {1}{\sqrt {2\pi}}\int_{T}^{+\infty}e^{-\frac {x^{2}}{2}}\mathrm {d}x = 1\frac {1}{\sqrt {2\pi}}\int_{-T}^{T}e^{-\frac {x^{2}}{2}}\mathrm {d}x > 1 - \frac {\varepsilon}{4}, 則

\displaystyle {\frac {1}{\sqrt {2\pi}}\int_{-\infty}^{-T}e^{-\frac {x^{2}}{2}}\mathrm {d}x + \frac {1}{\sqrt {2\pi}}\int_{T}^{+\infty}e^{-\frac {x^{2}}{2}}\mathrm {d}x \leq \frac {\varepsilon}{4}}

結合 P_{n}(-\infty, -T] + P(T, +\infty) \leq \frac {\varepsilon}{2}, 那麼

\displaystyle {\begin {aligned} \left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{\frac {x^{2}}{2}}\mathrm {d}x \right | &\leq \frac {\varepsilon}{2} + \frac {\varepsilon}{4} + \left |P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}e^{\frac {x^{2}}{2}}\mathrm {d}x \right | \\ &= \frac {3}{4}\varepsilon + \left |P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}e^{\frac {x^{2}}{2}}\mathrm {d}x \right | \end {aligned}}

對於 \left |P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}e^{\frac {x^{2}}{2}}\mathrm {d}x \right |, 我們已經知道 P_{n}(-T, T] \geq 1 - \frac {\varepsilon}{2}, \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}e^{-\frac {x^{2}}{2}}\mathrm {d}x \geq 1 - \frac {\varepsilon}{4}, 故

\displaystyle {P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}e^{\frac {x^{2}}{2}}\mathrm {d}x \geq -\frac {\varepsilon}{4}}

不等式兩側取負, 不等式變號, 得

\displaystyle {- \left (P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}e^{\frac {x^{2}}{2}}\mathrm {d}x \right ) \leq \frac {\varepsilon}{4}}

不等式兩側取絕對值, 有

\displaystyle {\left |P_{n}(-T, T] - \frac {1}{\sqrt {2\pi}}\int_{-T}^{T}e^{\frac {x^{2}}{2}}\mathrm {d}x \right | \leq \frac {\varepsilon}{4}}

綜上所述,

\displaystyle {\left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x \right | \leq \frac {3}{4}\varepsilon + \frac {\varepsilon}{4} = \varepsilon}

對於等號, 我們只需要適當放大 \varepsilon, 即可得到對於任意 \varepsilon > 0, 存在正整數 N, 當 n > N 時, 有

\displaystyle {\left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x \right | < \varepsilon}

根據極限的定義, 有

\displaystyle {\lim \limits_{n \to \infty}P_{n}(a, b] = \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x}

上面的討論導出了 :

定理 2. (De Moivre - Laplace 積分定理) 設 0 < p < 1,

\displaystyle {P_{n}(k) = \binom {k}{n}p^{k}q^{n - k}, P_{n}(a, b] = \sum \limits_{a < x \leq b}P_{n}(np + x\sqrt {npq})}

那麼

\displaystyle {\sup \limits_{-\infty < a \leq b < +\infty} \left \{ \left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{\frac {x^{2}}{2}}\mathrm {d}x \right | \right \} \to 0\ (n \to \infty)}

在大數法則中, 我們得到的 P \left \{ \left |\frac {S_{n}}{n} - p \right | \geq \varepsilon \right \} \to 0\ (n \to \infty) 可以更精確地表示為

\displaystyle {\sup \limits_{-\infty < a < b < +\infty} \left \{ \left | P \left \{ a < \frac {S_{n} - E(S_{n})}{\sqrt {\text {Var}(S_{n})}} \leq b \right \}- \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x \right | \right \} \to 0\ (n \to \infty)}

由此可見, 對於任意 -\infty < A < B < +\infty, 有

\displaystyle {P \left \{ A < S_{n} \leq B \right \} - \left ( \Phi \left( \frac {B - np}{\sqrt {npq}} \right ) - \Phi \left( \frac {A - np}{\sqrt {npq}} \right ) \right ) \to 0\ (n \to \infty)}

例題 1. 將規則的骰子擲 12000 次. 問 : "6 點" 出現在次數區間 (1800, 2100] 次的機率如何?

:

\displaystyle {\begin {aligned} P_{12000}(k) &= \sum \limits_{1800 < x \leq 2100}\binom {k}{12000} \left ( \frac {1}{6} \right )^{k}\left ( \frac {5}{6} \right )^{k} \\ &\doteq \Phi \left ( \frac {B - np}{\sqrt {npq}} \right ) - \Phi\left ( \frac {A - np}{\sqrt {npq}} \right ) \doteq 0.992 \end {aligned}}

其中, A = 1800, B = 2100, n = 12000, p = \frac {1}{6}, q = \frac {5}{6}, \Phi(x) = \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{x}e^{-\frac {t^{2}}{2}}\mathrm {d}t

\blacksquare

把二項機率 P_{n}(np + x\sqrt {npq}) (僅考慮能夠使得 np + x\sqrt {npq} 取非負整數的 x) 標在圖上 :

那麼, 局部定理表明, 對於 x = o \left ( (npq)^{\frac {1}{6}} \right ), 機率 P_{n}(np + x\sqrt {npq}) 的值比較好地位於函數 f(x) = \frac {1}{\sqrt {2\pi npq}}e^{-\frac {x^{2}}{2}} 的曲線上

由積分定理可知, 機率 P_{n}(a, b] = P \left \{ a\sqrt {npq} < S_{n} - np \leq b \sqrt {npq} \right \} = P \left \{ np + a\sqrt {npq} < S_{n} \leq np + b\sqrt {npq} \right \} 的值可以較好地由積分 \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x 逼近

F_{n}(x) = P_{n}(-\infty, x] = P \left \{ \frac {S_{n} - np}{\sqrt {npq}} \leq x \right \}. 由 \sup \limits_{-\infty < a < b < +\infty} \left \{ \left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{-\frac {x^{2}}{2}}\mathrm {d}x \right | \right \} \to 0\ (n \to \infty) 可見 :

\displaystyle {\sup \limits_{-\infty < x < +\infty} \left \{ \left |F_{n}(x) - \Phi(x) \right | \right \} \to 0\ (n \to \infty)}

那麼, 隨著 n 的增長, \sup \limits_{-\infty < a \leq b < +\infty} \left \{ \left | P_{n}(a, b] - \frac {1}{\sqrt {2\pi}}\int_{a}^{b}e^{\frac {x^{2}}{2}}\mathrm {d}x \right | \right \} 和 \sup \limits_{-\infty < x < +\infty} \left \{ \left |F_{n}(x) - \Phi(x) \right | \right \} 兩式趨向於 0 的速度如何? 這裡直接引用 A.C. Berry - C.G. Esseen 定理的特殊情形 :

\displaystyle {\sup \limits_{-\infty < x < +\infty} \left \{ \left | F_{n}(x) - \Phi(x) \right | \right \} \leq \frac {p^{2} + q^{2}}{\sqrt {npq}}}

要特別強調的是, 估計 \frac {1}{\sqrt {npq}} 的量級不能再提高了. 這裡指的是當 p 的值接近於 0 或者 1 的時候, 甚至對於充分大的 n, 用函數 \Phi(x) 來逼近 F_{n}(x) 的效果也可能不太好. 因此, 產生了一個問題 : 當 p 或者 q 比較小的時候, 能否找到一個比局部定理和積分定理給出的更好的逼近呢? 為此, 我們指出, 當 p = \frac {1}{2} 時, 二項分佈 \left \{ P_{n}(k) \right \} 具有對稱的形狀. 不過當 p 比較小的時候, 二項分佈的形狀不是對稱的, 不能指望用常態逼近有好的結果

Poisson 定理

\displaystyle {P_{n}(k) = \begin {cases} \binom {k}{n}p^{k}q^{n - k} & {k = 0, 1, 2, ..., n} \\ 0 & {k = n + 1, n + 2, ...} \end {cases}}

其中, p 是關於 n 的函數, 即 p = p(n)

定理 3. (Poisson 定理) 設當 n \to \infty 時, p(n) \to 0np(n) \to \lambda. 其中, \lambda > 0. 那麼, 對於任意 k = 0, 1, 2, ..., 有

\displaystyle {P_{n}(k) \to \pi_{k} = \frac {\lambda^{k}e^{-\lambda}}{k!}}

:

根據條件, 我們構造

\displaystyle {p(n) = \frac {\lambda}{n} + o \left ( \frac {1}{n} \right )}

因此, 對於固定的 k = 0, 1, 2, ... 和充分大的 n,

\displaystyle {\begin {aligned}P_{n}(k) &= \binom {k}{n}p^{k}q^{n - k} \\ &= \frac {n!}{k!(n - k)!}p^{k}q^{n - k} \\ &= \frac {n(n - 1)...(n - k - 1)}{k!}\left (\frac {\lambda}{n} + o\left ( \frac {1}{n} \right ) \right )^{k}\left (1 - \left (\frac {\lambda}{n} + o\left ( \frac {1}{n} \right ) \right ) \right )^{n - k} \\ &= \frac {1}{k!} \cdot \frac {n(n - 1)...(n - k - 1)}{n^{k}} \left (\lambda + o(1) \right )^{k}\left (1 - \left (\frac {\lambda}{n} + o\left ( \frac {1}{n} \right ) \right ) \right )^{n - k} \end {aligned}}

對於 \frac {n(n - 1)...(n - k - 1)}{n^{k}}, 有

\displaystyle {\frac {\overbrace {(n - k - 1) \cdot (n - k - 1) \cdot ... \cdot (n - k - 1)}^{k\ \text {個}}}{n^{k}} \leq \frac {n(n - 1)...(n - k - 1)}{n^{k}} \leq \frac {\overbrace {n \cdot n \cdot ... \cdot n}^{k\ \text {個}}}{n^{k}}}

由於 \lim \limits_{n \to \infty}\frac {(n - k - 1)^{k}}{n^{k}} = \lim \limits_{n \to \infty}\frac {n^{k} + \sum \limits_{i = 1}^{k}\binom {i}{k}n^{i}(1 - k)^{k - i}}{n^{k}} = 1\lim \limits_{n \to \infty}\frac {n^{k}}{n^{k}} = 1, 由夾擠定理可知

\displaystyle {\lim \limits_{n \to \infty}\frac {n(n - 1)...(n - k - 1)}{n^{k}} = 1}

那麼當 n \to \infty 時,

\displaystyle {\frac {1}{k!} \cdot \frac {n(n - 1)...(n - k - 1)}{n^{k}}\left ( \lambda^{k} + o(1) \right )^{k} \to \lambda^{k}}

\displaystyle {\begin {aligned} \lim \limits_{n \to \infty} \left (1 - \left (\frac {\lambda}{n} + o\left ( \frac {1}{n} \right ) \right ) \right )^{n - k} &= \lim \limits_{n \to \infty}\left (1 - \frac {\lambda}{n} + o\left ( \frac {1}{n} \right ) \right )^{n - k} \\ &= \lim \limits_{n \to \infty}\left (1 + \left (o\left ( \frac {1}{n} \right ) - \frac {\lambda}{n} \right ) \right )^{n - k} \\ &= \lim \limits_{n \to \infty}\left (1 + \left (o\left ( \frac {1}{n} \right ) - \frac {\lambda}{n} \right ) \right )^{(n - k)\frac {o \left ( \frac {1}{n} \right ) - \frac {\lambda}{n}}{o \left ( \frac {1}{n} \right ) - \frac {\lambda}{n}}} \\ &=  e^{\lim \limits_{n \to \infty}(n - k)\left ( o\left ( \frac {1}{n} \right ) - \frac {\lambda}{n} \right )} \\ &=  e^{\lim \limits_{n \to \infty}(n - k) \cdot o\left ( \frac {1}{n} \right ) - (n - k)\frac {\lambda}{n}} \\ &= e^{\lim \limits_{n \to \infty}-(n - k)\frac {\lambda}{n}} \\ &= e^{\lim \limits_{n \to \infty}-\lambda + \frac {k}{n}\lambda} \\ &= e^{-\lambda} \end {aligned}}

故當 n \to \infty 時, P_{n}(k) \to \frac {\lambda^{k}e^{-\lambda}}{k!}

綜上所述, 當 n \to \infty 時, p(n) \to 0np(n) \to \lambda. 其中, \lambda > 0. 那麼, 對於任意 k = 0, 1, 2, ..., 有

\displaystyle {P_{n}(k) \to \pi_{k} = \frac {\lambda^{k}e^{-\lambda}}{k!}}

\blacksquare

序列 \left \{ \pi_{k}, k = 0, 1, 2, ... \right \} 滿足

\pi_{k} > 0\sum \limits_{k = 0}^{\infty}\pi_{k} = 1

因此可以作為機率分佈, 我們稱之為 Poisson 分佈. 之前所討論的所有分佈都是集中在有限個點上, Poisson 分佈時目前遇到的第一個集中在無限個點上的機率分佈

和之前一樣, 我們仍然考慮當 n \to \infty 時, 機率 P_{n}(k) 收斂於 \pi_{k} 的速度如何? 這裡直接給出一個不經證明的結論 : 若 np(n) = \lambda, 則

\displaystyle {\sum \limits_{k = 0}^{\infty} \left | P_{n}(k) - \pi_{k} \right | \leq \frac {2\lambda}{n}\min \left \{ 2, \lambda \right \}}

De Moivre - Laplace 極限定理和大數法則

De Moivre - Laplace 極限定理和大數法則都是建立在 Bernoulli 概型上的, 那麼是否可以由 De Moivre - Laplace 極限定理導出大數法則呢?

因為

\displaystyle {\begin {aligned} P \left \{ \left |\frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} &= P \left \{ \left | \frac {S_{n} - np}{n} \right | \leq \varepsilon \right \} \\ &= P \left \{ \frac {\left |S_{n} - np \right |}{\sqrt {n} \cdot \sqrt {n}} \leq \varepsilon \right \} \\ &= P \left \{ \frac {\left |S_{n} - np \right |}{\sqrt {n} \cdot \sqrt {n} \cdot \sqrt {pq}} \leq \frac {\varepsilon}{\sqrt {pq}} \right \} \\ &= P \left \{ \frac {\left |S_{n} - np \right |}{\sqrt {npq}} \leq \sqrt {\frac {n}{pq}}\varepsilon \right \} \\ &= P \left \{ \left |\frac {S_{n} - np}{\sqrt {npq}} \right | \leq \sqrt {\frac {n}{pq}}\varepsilon \right \} \\ &= P \left \{-\varepsilon\sqrt {\frac {n}{pq}} \leq \frac {S_{n} - np}{\sqrt {npq}} \leq \varepsilon\sqrt {\frac {n}{pq}} \right \} \end {aligned}}

根據 De Moivre - Laplace 極限定理, 當 n \to \infty 時, 有

\displaystyle {P \left \{ \left |\frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \to \frac {1}{\sqrt {2\pi}}\int_{-\varepsilon\sqrt {\frac {n}{pq}}}^{\varepsilon\sqrt {\frac {n}{pq}}}e^{-\frac {x^{2}}{2}}\mathrm {d}x \to \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{+\infty}e^{-\frac {x^{2}}{2}}\mathrm {d}x \to 1}

P \left \{ \left |\frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \to 1. 這便是大數法則

由上述推導, 我們可以得到

\displaystyle {P \left \{ \left |\frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \to \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{+\infty}e^{-\frac {x^{2}}{2}}\mathrm {d}x\ (n \to \infty)}

然而, Chebyshev 不等式只能給出下面的估計

\displaystyle {P \left \{ \left |\frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \geq 1 - \frac {pq}{n\varepsilon^{2}}}

在大數法則中, 為了得到使得

\displaystyle {P \left \{ \left |\frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \geq 1 - \alpha}

成立所需要的觀測次數, 我們由 Chebyshev 不等式得到了下面的估計 :

\displaystyle {n \geq \left [ \frac {1}{4\varepsilon^{2}\alpha} \right ] = n_{1}(\alpha)}

其中, [x]x 的整數部分. 現在, 我們可以通過 De Moivre - Laplace 極限定理得到更精確的估計. 我們由方程式

\displaystyle {\frac {1}{\sqrt {2\pi}}\int_{-k(\alpha)}^{k(\alpha)}e^{-\frac {x^{2}}{2}}\mathrm {d}x = 1 - \alpha}

求得 k(\alpha). 由於

\displaystyle {\varepsilon\sqrt {\frac {n}{pq}} \geq 2\varepsilon\sqrt {n}}

且根據不等式

\displaystyle {2\varepsilon\sqrt {n} \geq k(\alpha)}

求出最小整數, 得

\displaystyle {P \left \{ \left |\frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \geq 1 - \alpha}

由 2\varepsilon\sqrt {n} \geq k(\alpha) 可見

\displaystyle {n = n_{2}(\alpha) = \left [ \frac {k^{2}(\alpha)}{4\varepsilon^{2}} \right ]}

可以保證 P \left \{ \left |\frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \geq 1 - \alpha 成立, 其逼近精度可以由

\displaystyle {\sup \limits_{-\infty < x < +\infty} \left \{ \left | F_{n}(x) - \Phi(x) \right | \right \} \leq \frac {p^{2} + q^{2}}{\sqrt {npq}}}

得到

例題 2. 設 \varepsilon = 0.02, \alpha = 0.05. 則基於 Chebyshev 不等式的估計觀測次數為 12500 次, 而基於極限定理的估計觀測次數為 2500

\blacksquare

常態分佈

在 De Moivre - Laplace 積分定理中引入的函數

\displaystyle {\Phi(x) = \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{x}e^{-\frac {t^{2}}{2}}\mathrm {d}t}

在機率輪中有很大的作用, 稱為常態分佈函數或者 Gauss 分佈函數. 其導數

\displaystyle {\varphi(x) = \frac {1}{\sqrt {2\pi}}e^{-\frac {x^{2}}{2}}, x \in R}

稱為常態密度函數或者 Gauss 密度函數

常態分佈屬於機率輪中另一種類型的分佈 —— 連續分佈, 而我們之前討論的都是離散型分佈. 在相當一般的條件下, 大量獨立隨機變數 (未必是 Bernoulli 隨機變數) 之和的分佈可以很好地使用常態分佈來逼近

函數 \varphi(x) = \frac {1}{\sqrt {2\pi}}e^{-\frac {x^{2}}{2}} 的圖形關於縱軸對稱, 且隨著 |x| \to \infty 而趨於 0. 而函數 \Phi(x) = \frac {1}{\sqrt {2\pi}}\int_{-\infty}^{x}e^{-\frac {t^{2}}{2}}\mathrm {d}t 隨著 x \to +\infty 而趨於 1, 隨著 x \to -\infty 而趨於 0

除了 \Phi(x) 之外, 我們還經常使用誤差函數 :

\displaystyle {\text {erf}(x) = \frac {2}{\sqrt {\pi}}\int_{0}^{x}e^{-t^{2}}\mathrm {d}t, x > 0}

顯然, 對於 x > 0, 有

\displaystyle {\Phi(x) = \frac {1}{2}\left ( 1 + \text {erf}(\frac {x}{\sqrt {2}}) \right ), \text {erf}(x) = 2\Phi(\sqrt {2}x) - 1}

成功頻率對機率的偏差滿足一定要求的試驗次數

在大數法則中, 我們曾指出, 由 Chebyshev 不等式給出的事件 \left \{ \omega : \left |\frac {S_{n}}{n} - p \right | \geq \varepsilon \right \} 機率的估計是相當粗略的. 這一估計對於非負隨機變數 X, 是由 Chebyshev 不等式

\displaystyle {P \left \{ X \geq \varepsilon \right \} \leq \frac {E \left (X^{2} \right )}{\varepsilon^{2}}}

得到的. 不過可以利用 Chebyshev 不等式的另一種形式 :

\displaystyle {P \left \{ X \geq \varepsilon \right \} = P \left \{ X^{2k} \geq \varepsilon^{2k} \right \} \leq \frac {E \left (X^{2k} \right )}{\varepsilon^{2k}}}

然而, 還可以更進一步, 利用 Chebyshev 不等式的指數形式 : 對於 X \geq 0\lambda > 0,

\displaystyle {P \left \{ X \geq \varepsilon \right \} = P \left \{ e^{\lambda x} \geq \varepsilon^{\lambda x} \right \} \leq \frac {E \left (e^{\lambda x} \right )}{e^{\lambda\varepsilon}} = \frac {E \left ( e^{\lambda x} \right )}{E \left (e^{\lambda\varepsilon} \right )} = E \left ( \frac {e^{\lambda x}}{e^{\lambda\varepsilon}} \right ) = E \left ( e^{\lambda(X - \varepsilon)} \right )}

由於 \lambda > 0 的任意性, 可見

\displaystyle {P \left \{ X \geq \varepsilon \right \} \leq \inf \limits_{\lambda > 0} \left \{ E \left (e^{\lambda(X - \varepsilon)} \right ) \right \}}

接下來, 我們討論當 X = \frac {S_{n}}{n}, S_{n} = \xi_{1} + \xi_{2} + ... + \xi_{n}, P \left \{ \xi_{i} = 1 \right \} = p, P \left \{ \xi_{i} = 0 \right \} = q, i = 1, 2, ..., n 時, 沿此路徑將會導致的結果

\varphi(\lambda) = E \left (e^{\lambda \xi_{1}} \right ), 則

\displaystyle {\begin {aligned} \varphi(\lambda) &= \left . e^{\lambda \xi_{1}} \right |_{\xi_{1} = 0}P \left \{\xi_{1} = 0 \right \} + \left . e^{\lambda \xi_{1}} \right |_{\xi_{1} = 1}P \left \{\xi_{1} = 1 \right \} \\ &= 1 \times q + e^{\lambda} \times p \\ &= 1 - p + pe^{\lambda} \end {aligned}}

對於兩兩獨立的隨機變數 \xi_{1}, \xi_{2}, ..., \xi_{n}, 則有

\displaystyle {\begin {aligned} E \left (e^{\lambda S_{n}} \right ) &= E \left (e^{\lambda(\xi_{1} + \xi_{2} + ... + \xi_{n})} \right ) \\ &= E \left (e^{\lambda\xi_{1} + \lambda\xi_{2} + ... + \lambda\xi_{n}} \right ) \\ &= E \left ( e^{\lambda\xi_{1}}e^{\lambda\xi_{2}}...e^{\lambda\xi_{n}} \right ) \\ &= E \left (e^{\lambda\xi_{1}} \right )E \left (e^{\lambda\xi_{2}} \right )...E \left (e^{\lambda\xi_{n}} \right ) \\ &= E^{n} \left (e^{\lambda\xi_{1}} \right ) \\ &= \varphi^{n}(\lambda) \end {aligned}}

因此, 對於 0 < a < 1, 根據 Chebyshev 不等式的指數形式, 有

\displaystyle {\begin {aligned} P \left \{ \frac {S_{n}}{n} \geq a \right \} &\leq \inf \limits_{\lambda > 0} \left \{ E \left ( e^{\lambda \left ( \frac {S_{n}}{n} - a \right )} \right ) \right \} = \inf \limits_{\lambda > 0}E \left ( e^{\left ( \frac {\lambda}{n}S_{n} - \lambda a \right )} \right ) = \inf \limits_{\lambda > 0} \left \{ E \left ( \frac {e^{\frac {\lambda}{n}S_{n}}}{e^{\lambda a}} \right ) \right \} \\ &= \inf \limits_{\lambda > 0} \left \{ \frac {E \left ( e^{\frac {\lambda}{n}S_{n}} \right )}{e^{\lambda a}} \right \} = \inf \limits_{\lambda > 0} \left \{ \frac {\varphi^{n} \left (\frac {\lambda}{n} \right )}{e^{\lambda a}} \right \} = \inf \limits_{\lambda > 0} \left \{ e^{-\lambda a} \cdot \varphi^{n} \left (\frac {\lambda}{n} \right ) \right \} \\ &= \inf \limits_{\lambda > 0} \left \{ e^{-\lambda a}e^{\ln {\varphi^{n} \left ( \frac {\lambda}{n} \right )}} \right \} = \inf \limits_{\lambda > 0} \left \{ e^{-\lambda a + n \ln {\varphi \left ( \frac {\lambda}{n} \right )}} \right \} = \inf \limits_{\lambda > 0} \left \{ e^{-n \left ( \frac {\lambda}{n}a - \ln {\varphi \left ( \frac {\lambda}{n} \right )} \right )} \right \} \\ &= \inf \limits_{\lambda > 0} \left \{ e^{-n \left (a\mu - \ln {\varphi(\mu)} \right )} \right \} \\ &= e^{-n\sup \limits_{\mu > 0} \left \{ a\mu - \ln {\varphi(\mu)} \right \}} \end {aligned}}

其中, \mu = \frac {\lambda}{n}. 類似地, 也有

\displaystyle {P \left \{ \frac {S_{n}}{n} \leq a \right \} \leq e^{-n\sup \limits_{\mu < 0}a\mu - \ln {\varphi(\mu)}}}

p \leq a \leq 1 時, 記函數 f(x) = ax - \ln {\left (1 - p + pe^{x} \right )}, 則 f'(x) = a - \frac {pe^{x}}{1 - p + pe^{2}}. 令 f'(x) = 0, 得

\displaystyle {e^{x} = \frac {a(1 - p)}{p(1 - a)}}

x = x_{0} 使得 f(x) 取得最大值, 則點 x_{0} 滿足 e^{x_{0}} = \frac {a(1 - p)}{p(1 - a)}. 因此,

\displaystyle {\sup \limits_{\mu > 0} \left \{ f(\mu) \right \} = H(a)}

其中, H(a) = a\ln {\frac {a}{p}} + (1 - a)\ln {\frac {1 - a}{1 - p}}. 這樣, 當 p \leq a \leq 1 時,

\displaystyle {P \left \{ \frac {S_{n}}{n} \geq a \right \} \leq e^{-n\sup \limits_{\mu > 0} \left \{ f(\mu) \right \}} = e^{-nH(a)}\ \ \ \ \ \ \ \ \ \ (II)}

而由於 H(p + x) \geq 2x^{2}, 0 \leq p + x \leq 1, 那麼對於任何 \varepsilon > 0, 0 \leq p \leq 1, 有

\displaystyle {P \left \{ \frac {S_{n}}{n} - p \geq \varepsilon \right \} \leq e^{-2n\varepsilon^{2}}\ \ \ \ \ \ \ \ \ \ (III)}

類似可得, 當 a \leq p \leq 1 時,

\displaystyle {P \left \{ \frac {S_{n}}{n} \leq a \right \} \leq e^{-nH(a)}}\ \ \ \ \ \ \ \ \ \ (IV)

從而對於任意 \varepsilon > 0, 0 \leq p \leq 1, 有

\displaystyle {P \left \{ \frac {S_{n}}{n} - p \leq -\varepsilon \right \} \leq e^{-2n\varepsilon^{2}}\ \ \ \ \ \ \ \ \ \ (V)}

於是, P \left \{ \left | \frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \leq 2e^{-2n\varepsilon^{2}}. 由此可見, 對於任意 0 \leq p \leq 1, 保證不等式

\displaystyle {P \left \{ \left | \frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \geq 1 - \alpha}

成立的估計觀測次數 n_{3}(\alpha) 決定於

\displaystyle {n_{3}(\alpha) = \left [ \frac {\ln {\frac {2}{\alpha}}}{2\varepsilon^{2}} \right ]}

其中, [x] 表示 x 得整數部分. 若不取整數部分, 直接將 n_{3}(\alpha)n_{1}(\alpha) = \left [\frac {1}{4\alpha\varepsilon^{2}} \right ] 比較可見 :

\displaystyle {\frac {n_{1}(\alpha)}{n_{3}(\alpha)} = \frac {\frac {1}{4\alpha\varepsilon^{2}}}{\frac {\ln {\frac {2}{\alpha}}}{2\varepsilon^{2}}}} = \frac {1}{2\alpha\ln {\frac {2}{\alpha}} \to +\infty\ (\alpha \to 0^{+})}

由此可見, 當 \alpha \to 0^{+} 時, 由 Chebyshev 不等式的指數形式估計最小必須要觀測的次數, 比用一般的 Chebyshev 不等式估計的次數更加精確, 特別是對於比較小的 \alpha

利用數學分析中的結論 :

\displaystyle {\lim \limits_{x \to -\infty}\frac {1}{\sqrt {2\pi}}\int_{x}^{+\infty}e^{-\frac {t^{2}}{2}}\mathrm {d}t = \frac {1}{\sqrt {2\pi}x}e^{-\frac {x^{2}}{2}}}

可以證明當 \alpha \to o^{+} 時, k^{2}(\alpha) \to 2 \ln {\frac {2}{\alpha}}. 於是有

\displaystyle {\lim \limits_{\alpha \to 0^{+}}\frac {n_{2}(\alpha)}{n_{3}(\alpha)} = 1}

(II), (III), (IV), (V) 和式

\displaystyle {P \left \{ \left | \frac {S_{n}}{n} - p \right | \leq \varepsilon \right \} \leq 2e^{-2n\varepsilon^{2}}}\ \ \ \ \ \ \ \ \ \ (VI)

合稱為大偏差機率的不等式. 利用 De Moivre - Laplace 定理, 可以簡單地對事件 \left \{ \left |S_{n} - np \right | \leq x\sqrt {n} \right \} 的機率進行估計. 此事件表示 S_{n}np (數量級為 \sqrt {n}) 的標準離差. 而不等式 (III), (V)(VI)

\displaystyle {\left \{ \left |S_{n} - np \right | \leq x\sqrt {n} \right \}}

給出的估計描繪量級大於 \sqrt {n} 的離差, 其數量級為 n

練習題

自主習題 1. 設 p = \frac {1}{2}, 而 Z_{n} = 2S_{n} - n (n 組試驗中的 10 多出的數量). 證明 :

\displaystyle {\sup \limits_{j} \left \{ \left | \sqrt {\pi n}P \left \{ Z_{2n} = j - e^{-\frac {j^{2}}{4^{n}}} \right \} \right | \right \} \to 0\ (n \to \infty)}

自主習題 2. 證明 : Poisson 定理對於 p = \frac {\lambda}{n} 的收斂速度為

\displaystyle {\sup \limits_{k} \left \{ \left | P_{n}(k) - \frac {\lambda^{k}e^{-\lambda}}{k!} \right | \right \} \leq \frac {\lambda^{2}}{n}}