$$ \newcommand{\floor}[1]{\left\lfloor #1 \right\rfloor} \newcommand{\ceil}[1]{\left\lceil #1 \right\rceil} \newcommand{\N}{\mathbb{N}} \newcommand{\R}{\mathbb{R}} \newcommand{\Z}{\mathbb{Z}} \newcommand{\Q}{\mathbb{Q}} \newcommand{\C}{\mathbb{C}} \renewcommand{\L}{\mathcal{L}} \newcommand{\x}{\times} \newcommand{\contra}{\scalebox{1.5}{$\lightning$}} \newcommand{\inner}[2]{\left\langle #1 , #2 \right\rangle} \newcommand{\st}{\text{ such that }} \newcommand{\for}{\text{ for }} \newcommand{\Setcond}[2]{ \left\{\, #1 \mid #2 \, \right\}} \newcommand{\setcond}[2]{\Setcond{#1}{#2}} \newcommand{\seq}[1]{ \left\langle #1 \right\rangle} \newcommand{\Set}[1]{ \left\{ #1 \right\}} \newcommand{\set}[1]{ \set{#1} } \newcommand{\sgn}{\text{sign}} \newcommand{\halfline}{\vspace{0.5em}} \newcommand{\diag}{\text{diag}} \newcommand{\legn}[2]{\left(\frac{#1}{#2}\right)} \newcommand{\ord}{\text{ord}} \newcommand{\di}{\mathrel{|}} \newcommand{\gen}[1] \newcommand{\irr}{\mathrm{irr }} \renewcommand{\deg}{\mathrm{deg }} \newcommand{\nsgeq}{\trianglelefteq} \newcommand{\nsg}{\triangleleft} \newcommand{\argmin}{\mathrm{argmin}} \newcommand{\argmax}{\mathrm{argmax}} \newcommand{\minimize}{\mathrm{minimize}} \newcommand{\maximize}{\mathrm{maximize}} \newcommand{\subto}{\mathrm{subject\ to}} \newcommand{\DKL}[2]{D_{\mathrm{KL}}\left(#1 \di\di #2\right)} \newcommand{\ReLU}{\mathrm{ReLU}} \newcommand{\E}{\mathsf{E}} \newcommand{\V}{\mathsf{Var}} \newcommand{\Corr}{\mathsf{Corr}} \newcommand{\Cov}{\mathsf{Cov}} \newcommand{\covariance}[1]{\Cov\left(#1\right)} \newcommand{\variance}[1]{\V\left[#1\right]} \newcommand{\variancewith}[1]{\V\left[#1\right]} \newcommand{\expect}[1]{\E\left[#1\right]} \newcommand{\expectwith}[2]{\E_{#1}\left[#2\right]} \renewcommand{\P}{\mathsf{P}} \newcommand{\uniform}[2]{\mathrm{Uniform}\left(#1 \dots #2\right)} \newcommand{\gdist}[2]{\mathcal{N}\left(#1, #2\right)} \DeclarePairedDelimiter{\norm}{\lVert}{\rVert} $$ \everymath{\displaystyle}

Probability and Statistics

Mathematical statistics, Probability theory, etc

ML/DL과 알고리즘을 공부하면서, 확률론과 통계학에 대한 베이스가 부족함을 느낀 적이 꽤 많았습니다. 그래서 언젠가 한번 방학때 이런 부분들을 공부하는 기회를 삼으려고 하는데, 완전 독학을 하다보니 motivation이 너무 떨어져서 여기에 정리하면서 공부하려 합니다.

Mathematical Statistics

공부하는 책은 서울대학교 통계학과 수리통계 과목의 교과서인 “수리통계학(김우철 교수님 저)” 입니다.

Topic	Book Chapter
Definition of Probability, Conditional Probability	1.1 - 1.2
Random variables, Expectations	1.3 - 1.5
Useful Inequalities	1.6
Bivariate random variables	2.1 - 2.2
Bivariate Conditional distributions	2.3 - 2.4
Multivariate distributions	2.5
Discrete probability distributions	3.1 - 3.3
Poisson distribution	3.4
Continuous probability distributions	3.5
Normal distribution	3.6, 4.4
Sample distributions	4.1 - 4.2
Order statistics	4.3
Central Limit Theorem	5.1
Limit distribution	5.2 - 5.3
Method of Moments Estimation	6.1
Maximum Likelihood Estimation	6.2 - 6.3
Maximum Likelihood Estimation : Properties	6.5
Least Square Estimation	6.5