Video series on trying to reverse engineer neural networks https://www.youtube.com/playlist?list=PLoyGOS2WIonajhAVqKUgEMNmeq3nEeM51 Eric Jang (@ericjang11) points out a new and possibly very important machine learning phenomenon called "grokking". In summary, if you just keep training neural networks well past the point of overfitting, a sudden "phase change" can occur improving generalization performance from random chance level to perfect generalization. More data accelerates the speed at which this phase change occurs.
Links for 2022-01-12
Links for 2022-01-12
Links for 2022-01-12
Video series on trying to reverse engineer neural networks https://www.youtube.com/playlist?list=PLoyGOS2WIonajhAVqKUgEMNmeq3nEeM51 Eric Jang (@ericjang11) points out a new and possibly very important machine learning phenomenon called "grokking". In summary, if you just keep training neural networks well past the point of overfitting, a sudden "phase change" can occur improving generalization performance from random chance level to perfect generalization. More data accelerates the speed at which this phase change occurs.