[DS141] 點做 linear regression?

做緊data scientist
但睇唔明你堆野

你可以先 review 咗 linear algebra + optimization 然後再望返呢個 post. 因為 householder transform 對 undergrad 嚟講都算 advanced, 小妹有諗住 skip 埋呢 part, 但 skip 咗就好似 black box 咗 QR factorization 嘅原理

你只係講緊用咩方法計(X'X)^{-1}
我詳細解吓錯係邊: 計(X'X)^{-1} 即係先計呢 inverse matrix 再用 inverse matrix 乘返 t(X)*y, 我提嘅方法並唔係計 inverse matrix 嘅 alternative,
所以解 normal equation 並唔需要計 (X'X)^{-1}
唔好再誤人子弟啦

另外呢位巴打 sdv 乜鳩柒 confused 咗 quasi-newton method 同埋 newton-raphson method 佢段自制 code 用 BFGS 係 quasi-newton 同佢口入面講嘅 newton-raphson 唔同. 另外我講緊 gradient descent 係 first order method, 點解 confuse 又 confuse? 其實 sdv 乜柒都唔知自己講緊啲乜, 不過先等我譯去英文同同事 share 花生先

你有閱讀障礙?
"gradient descent, newton method, quasi-newton method (e.g. BFGS)"
我咪話BFGS is an example of quasi-newton method

我係話有closed form solution 做咩要用numerical method which include first order (gradient descent) 同second order (newton, quasi-newton) method

我見你誤人子弟先出聲

同埋我幾時有講newton-raphson

你理解下咩叫closed form solution先啦

勸你唔好睇太多

我理解用pinv 應該主要都係因為佢比QR numerically stable?

純數撚剩係識證條normal equation點嚟

點計個H[j]係乜野?

仲有
係唔知error 係咩distribution下
用gauss markov theorem 證明ols estimator of linear model 係best linear unbiased estimator (blue)

知道error 係normal 既情況下
用cramer rao lower bound 證明ols/mle estimator of linear model 係 best unbiased (including linear and non-linear) estimator

呢d先係linaer regression有趣之處

留名學習

SVD 比 QR 更加 stable, 最直接一個原因係 SVD 可以 handle rank deficiency 而 QR 唔得. 但問題 pinv (即係用 SVD) 係比 QR 慢好多

點都唔夠 sdv 乜鳩柒咁無知以為先計 inv(Gram(X)) 再乘 X'*y 咁蠢, 唔識數學又喺度出風頭又唔敢自己開 post

Householder transform matrix, 係一個 rotation matrix
呢個H[j]乘落 X 度會進行動態清零

比 sdv 唔識串騎劫咗啦, 咁多嘢講唔自己開 post 嘅廢物, 我懷疑佢連 svd 同埋計 inverse 要用幾多 flops 都唔知

根據 sdvsvsdav 嘅神邏輯: 用 QR decomposition = 計 inv(gram(X)) 再乘返埋. 然後小妹就 prove 俾你睇有咩唔同:
X = QR
Gram(X) = t(X)*X = t(QR)*(QR) = t(R)*R
inv(Gram(X)) = (t(R)*(R))^{-1}
事實我地係冇計 inv(Gram(X)) 呢個 matrix
反而直接 plug 落去
Gram(X)*b = t(X)*y
t(R)*(R)*b = t(R)*t(Q)*y
R*b = t(Q)*y
你係唔需要計 R 嘅 inverse, 而係 forward substitution.
sdvsvsdav 係咪真係以為要計 inverse matrix 咁 on9? 真係 undergrad 冇讀 numerical methods? 不如你教下我 inverse matrix 計咗係邊啦

sdvsvsdav 呢個空白 account 上連登 然後專門 mon 住 econ/stat post 嚟 show-off, 點解唔敢自己開 post 呢? 因為佢連 linear algebra 都搞唔掂

屌我想學野呀

留名跟 sdv...連名都打唔好嘅人學嘢, 佢好似連 general 嘅 neural network 同埋 feedforward network 都未分清, 以為 neural network = multilayer 嘅 logistic regression.


統計學會唔會俾machine learning收皮？
- 分享自 LIHKG 討論區
https://lih.kg/bcsvBLV

先唔講佢連 neural 同 neutral 呢個簡單英文水平啦 究竟喺一個 Boltzmann machine 點樣 "seems to be a nested generalized linear model" ? 小妹就係真係唔明啦, 不過好明顯佢唔係 stat/ml 嘅 practitioner

小妹懷疑 sdv...連名都打唔好嘅人唔知咩係 generalized linear model (GLM), 其實用 multilayer 嘅 feedforward neural network 可以做到 beta regression, 但 beta regression 並唔係 GLM, 只係類似 GLM, 請問 neural network (限定喺 regression 上) 又點會係 nested GLM? 連 GLM 都學唔好都夠膽 mon 住連登 stat post 喺度泥漿摔角?

佢自稱data scientist，印象中佢睇嘅econ書係工具書居多

如果正經討論冇乜所謂, 佢係嗰種孔乙己 type
#25 幾有趣, 呢條 sdv 柒頭講到買 index fund 好似先要讀過 MPT 知道 diversification 係咩咁. 完全誤解咗上一個 comment 意思.

之後下一個 comment 巴打講解幾詳細, 呢條 sdv 粉皮又彈出嚟 show-off 指正話最新點點點好似啲亞氏保加症兒童咁


大學想揀Econ嘅同學，有咩想問可以入嚟問 (2)
- 分享自 LIHKG 討論區
https://lih.kg/aRqgRQV

以前post過

工唔工具睇你點define
mwg 都可以話係工具書