Back to : ml-study
Contents

Normal Equation

• Iteration์ ํตํด ๊ทน์์ ์ ์๋ ดํ๋ ๊ฒ์ด ์๋๋ผ, Analytically ์ต์ ํด $\theta$๋ฅผ ๊ตฌํ๋ ๋ฐฉ๋ฒ.
• ex) $J(\theta) = a\theta^2 + b\theta + c$ ($a > 0$) ๋ฅผ ์ต์ํํ๋ $\theta$ ๋ $-\frac{b}{2a}$ ์์ ์ฝ๊ฒ ์ ์ ์๋ค.
• How to do for vector parameter $J$?
• => Vector Calculus. $\pdv{}{\theta_i} J(\theta)$ ๊ฐ ๋ชจ๋ 0์ด ๋๋ $\theta$ ๋ฅผ ์ฐพ์ผ๋ฉด ๋๋ค.
• Parameter๋ค์ ํ๋ ฌ $X$๋ก ๋ง๋ค๊ณ , ์ด์ ๋์ํ๋ ๊ฐ๋ค์ $y$๋ก ๋ง๋ค์.
• $\theta = (X^T X)^{-1} X^T y$ ๊ฐ ์ฐ๋ฆฌ์ Linear Regression์ ๋์ํจ์ด ์๋ ค์ ธ ์๋ค.
• Feature scaling ๊ฐ์ ํํฌ๋ ๋ถํ์.
• ์ฅ์  : $\alpha$๋ฅผ ์๊ฐํ์ง ์์๋ ๋๊ณ , ๋ฐ๋ณต์ ์ผ๋ก ์ ์ ํ $\alpha$๋ฅผ ์ฐพ์ ํ์๊ฐ ์๋ค.
• ๋จ์  : ํ๋ ฌ๊ณฑ์ ๋ฐ inverse๋ ๊ต์ฅํ ๋๋ฆผ. ํนํ $n$์ด ํฌ๋ฉด ํ๋ ฌ๊ณฑ์์ ์ฐ๊ธฐ ์ด๋ ต๋ค.

Noninvertible Case

• $(X^T X)$๊ฐ invertibleํ์ง ์์ผ๋ฉด??
• Pseudoinverse (octave pinv ํจ์)
• ํฌ๊ฒ ๋ ๊ฐ์ง ๊ฒฝ์ฐ
• ๋ feature๊ฐ ์ฌ์ค linear ๊ด๊ณ์ ์๋ ๊ฒฝ์ฐ.
• ex) size in feet^2 ์ size in m^2
• Design matrix $X$๊ฐ dependent column ๊ฐ์ง๋ค.
• Redundant features -> Throw away.
• Too many features.
• Data๋ ์ ์๋ฐ feature๋ ๋ง์ ๊ฒฝ์ฐ.
• Feature ๋ช๊ฐ ๋ฒ๋ฆฌ๊ธฐ / ๋๋ Regularization.