Back to : ml-study
Contents

### Motivation

• Complex, nonlinear hypothesis
• ๋ง์ ์์ polynomial feature์ ์ธ ์๋ ์๊ฒ ์ง๋งโฆ ์ฌ๋ฌ ๊ฐ์ feature๋ฅผ ๊ฐ์ง ๋ฌธ์ ์ ์ ์ฉํ๊ธฐ๋ ์ด๋ ต๋ค.
• 100๊ฐ์ feature๊ฐ ์๋ค๋ฉด? ๊ทธ ์ด์์ด๋ผ๋ฉด? ์ ์ ํ ๊ณ ์ฐจํญ์ ์ฐ๊ธฐ๋ ๋งค์ฐ ์ด๋ ค์ด ์ผ.
• ex) Computer Vision. ์ด ์ด๋ฏธ์ง๋ ์ฐจ๋์ธ๊ฐ?
• Pixel intensity matrix๋ฅผ ๋ณด๊ณ  ์๋์ ์ด๋ฏธ์ง๋ฅผ ์ธ์ํ  ์ ์๋๊ฐ?
• Classification problem.
• Feature size = ํฝ์์ ์ (x3 if RGB)
• ์ด๋ ๊ฒ ๋ง์ feature๋ก๋ logistic regression๊ฐ์ ์๊ณ ๋ฆฌ์ฆ์ ์ธ ์๊ฐ ์๋ค.
• Goal : Algorithm that mimics brain.

### Background

• 80๋๋-90๋๋ ์ด์ ํฌ๊ฒ ์ ํํ์ผ๋, 90๋๋ ๋ง์๋ ๋ณ๋ก..
• Computationally expensive.
• ํ๋์๋ ์ด์ ๋ ์์์ ์ฌ์ฉํ  ๋ง ํ๋ค => STATE OF THE ART
• ๋๋ ์๋ง์ ๊ธฐ๋ฅ์ ์ฒ๋ฆฌํ๋ค -> ์ด ๋ง์ ๊ธฐ๋ฅ (์ธ์ด, ์๊ฐ, ์ฒญ๊ฐโฆ) ์ ๊ฐ๊ฐ ๊ตฌํํด์ผ ํ๋๊ฐ?
• NO. ONE LEARNING ALGORITHM HYPOTHESIS์ ์ํ๋ฉด, ์ค์ ๋ก ๋์์ ์๋ํ๋ ๊ฒ์ ๋จ ํ๋์ ํ์ต ์๊ณ ๋ฆฌ์ฆ.
• ์ฒญ๊ฐ ๊ด๋ จ ๊ธฐ๋ฅ์ ๋๊ณ  ์๊ฐ ๊ด๋ จ ๋ถ๋ถ์ ์ด๋ฅผ ์ฐ๊ฒฐํ๋ฉด, ๋๊ฐ ์์์ ์ ๋งคํํด์ ์๋ํ๋๋ผ.
• Brain rewiring experiment
• ๋์ ๋ค๋ฅธ ์ผ์๋ ์ ์ฐ๊ฒฐํ๋ฉด (direction ๋ฑ) ๋๋ต ์ ์๋ํ๋๋ผ.
• ์๋ง๋ ๊ฐ๊ฐ์ ๊ธฐ๋ฅ์ ๋ณ๊ฐ์ sw๊ฐ ์๋๊ฒ์ด๋ค.
• Neuron : ์ ๊ฒฝ๊ณ๋ฅผ ๊ตฌ์ฑํ๋ ๊ธฐ๋ณธ ์ธํฌ.
• Inputs (Dendrites)
• Outputs (Axons)
• I/O๋ฅผ ๊ฐ์ง ๊ธฐ๋ณธ์ ์ธ ๊ณ์ฐ ๋จ์์ฒ๋ผ ์๊ฐํ  ์ ์๋ค.

### Neuron Model

• Logistic Unit : $x_1, x_2, x_3$ ์ ์๋ ฅ๋ฐ์์ $h_\theta(x)$๋ฅผ computeํ๋ neuron์ ์๊ฐ.
• Layer structure (Neural Network) : Neuron๋ค์ output์ ๋ค์ ๋ฐ์์ ์๋ก์ด ๊ฐ์ ๊ณ์ฐํ๋ neuron๊ฐ์ layer๋ฅผ ์๋ ๋๋.
• Layer 1 (Input Layer) - ๋งจ ๋ (Output Layer) ์ฌ์ด์ Hidden layer๋ค์ด ์์นํ๋ ๊ตฌ์กฐ.
• Bias unit ๊ฐ์ ์ถ๊ฐ ํํฌ๋๋ค ์ฌ์ฉ.
• $a_i^{(j)}$ : โActivationโ of unit $i$ in layer $j$
• $\Theta^(j)$ : Matrix of weights, ๋ค์ layer๋ก ๋์ด๊ฐ๋ ๊ฐ๋ค.
• Forward Propagation์ Vectorize๋ฅผ ํตํด ๋น๊ต์  ํจ์จ์ ์ผ๋ก ์ฐ์ฐ ๊ฐ๋ฅํ๋ค.
• ์ด ๋ฐฉ๋ฒ์ด ์ ์ข์๊ฐ?
• ๋งจ ๋ Layer (Output Layer) ๋ ์ผ์ข์ Logistic regression
• ๊ทธ ์ด์ ์ Hidden layer๋ ๊ทธ ์์ฒด๊ฐ Learning๋ ๊ฒฐ๊ณผ๋ฌผ. ์ฆ, feature ์์ฒด๊ฐ ํ์ต์ ํตํด ๋ฐ์ ํ๋ค.
• Flexibleํ ๋ฐฉ๋ฒ.