[DS141] 點做 linear regression?

哦, 相對論更加出名, 我唔係喺學術界都知
你有冇讀過呢?

J痕巴打幾時出本econ theory 書比我學下野?
點解 J痕巴打需要出本 theory 書俾你學嘢呢?

我想問買 index fund 點解要讀過 MPT 呢? 你係咪可以 assert 現實冇 investor 會冇讀 MPT 而去買 index fund? 咁 portfolio diversification 嘅行為點解歷史上會比 MPT 出現先呢?

你係咪唔識用 matrix/vector operation? 垃圾 code 唔好 post 出嚟獻醜啦屌你老母. 小妹啲 code 係 source code 到 copy 出嚟, 唔會好似你寫到核突.

我呢度講 least squares (OLS) 嘅 numerical algorithm 喎, 你講 MLE for t-distributed noise 係離題喎, show-off 自己 programming 能力係冇問題, 但你應該開個 github repo 俾大家睇下有幾多粒星啦

既然有closed form solution 點解要用gradient descent 呢d numerical method?

呢個唔係有冇 closed form solution 嘅問題, 係 numerical stability 同埋 computational speed
同埋原文係 "我間唔中見到有人問點解唔用 gradient descent 做 linear regression." 我反而叫人唔好咁做喎, 請問錯咗係邊呢?
另外有 closed form solution 就代表所有 numerical method 都會俾到你一樣嘅答案, 我呢個 post 就係 share 我喺呢方面嘅舊 notes, 你指控我之前有冇睇吓我到底寫乜呢? 你鍾意 show-off 拋書包, 點解唔好似我咁另外開 post 要喺呢度 distract 呢?

推 唔識

正如我之前 reply, 係唔需要計 (X'X)^{-1} 呢個 matrix, numerical algorithm 唔係咁運作. 另外基本教科書係唔會 cover 其他 distribution, 而呢個 post 都唔係講他 distribution. 你離題唔會顯得自己有料, 反而顯得你妒忌我識得多過你啫. 呢堆嘢都係我以前 undergrad notes, 你唔會 ug 冇 major/minor maths 吓話?

如果 sdvsvsdav 有咩 comment 我係冇覆, 歡迎通知小妹. 佢一入嚟話我錯, 但錯嘅嘢係冇講佢 off-topic 離題嘅嘢. 呢個 ds141 series 會繼續

唔好睇, sdvsvsvsvsv 話有 closed form solution 所以唔洗知電腦上點計, 只需要用 mle 喎. 但 R 同 scipy/numpy 都似乎唔係佢講咁, 唔知邊個先誤人子弟

R 嘅 lm() class 唔係你段 code 咁講用 mle(), 我上面有 source code 講咗用 qr factorization. 你好似冇研究過自己 call 緊乜 function. 小妹呢個 post 就分享事實係點, 睇多啲對你有益

R 入面 lm() 唔係你段 code 咁寫, 唔好混淆 thanks.

https://github.com/SurajGupta/r-source/blob/master/src/library/stats/R/lm.R

搵唔到你段 code, 有冇 reference? 我地呢度講點做 linear regression (OLS), 你俾嘅 code 應該要係 lm 到搵到

https://github.com/SurajGupta/r-source/blob/master/src/library/stats/R/lm.R
但 R 嘅運作係同我講嘅一樣, 點解你冇提呢?

我係話有closed form solution 做咩要用numerical method which include...

This statement is wrong, QR factorization is numerical method but used in solving least squares problem. The reason is finding the inverse explicitly is numerically unstable and slow. 原來你呢樣都唔知.
唔知唔緊要, 唔好喺亂咁屈人錯.

#3

(呢個 post 並唔係教書，基礎相關教科書都有提 MLE)

文盲?

因為 J痕巴打冇出本 theory 書俾佢學下嘢 = 唔識嘢 冇睇過

沉迷學術致使心理扭曲

但你出咗本書, 佢又會用你冇提過嘅嘢話你誤人子弟

我覺得好無辜, 我原意係書本上嘅 closed form solution 放落 software 度都會俾你 numerical solution. 但好似打爛咗某人嘅學術玻璃心咁

就算係四庫全書咁厚提晒所有嘢冇誤人子弟喇，就會係citations未到100萬，J痕巴打唔出名，係廢

sqrt(2) 係 x^2=2 其中一個 closed form solution, but it is computed by numerical method
https://en.wikipedia.org/wiki/Methods_of_computing_square_roots
你有冇真係讀過一個 computer science / numerical method course 架? 有 closed form 同唔洗 numerical method 係兩回事嚟傻閪.

你有冇真係讀過我講嘅嘢先? 直接計 closed form solution 會有 numerical instability 唔係我發明架喎:

I should mention that there are also numerical accuracy issues that can make the use of the closed form solution to the least squares problem unadvisable. However, this would require a discussion of ill-conditioning that seems likely to be beyond the current understanding of the original poster

(1) https://stats.stackexchange.com/questions/23128/solving-for-regression-parameters-in-closed-form-vs-gradient-descent

computing a matrix inverse takes O(n^3 ) operations!

(2) https://www.kth.se/social/upload/529cc6a2f276544283a53ddc/Lecture%2010.pdf

KTH 數學排名咁高都會 notes 誤人子弟? 我理解完 closed-form solution 啦, 你嘅回答呢? QR factorization 係冇計 inverse 我 quote 埋唔同 source 同其他例子嚟論證, 錯嗰位係你呀

Newton and Newton-Raphson are just different names for the same method. Sometimes Newton-Raphson is prefered for the scalar/univariate case.
https://math.stackexchange.com/questions/1139288/difference-newtons-method-newton-rhapson-method-gauss-newton-method
找數, 你有講 newton-raphson 呀

其實唔答你係唔想教你
In general,
Ax = b
If A is invertible,
x = A^{-1}b (closed form solution)
x必然係A^{-1}b
你用咩method e.g. QR decomposition 去solve Ax = b 都係計緊A^{-1}b

Ref (p.9, p.12)
https://www.stat.cmu.edu/~ryantibs/convexopt/lectures/num-lin-alg.pdf

屌又應左你