Links for 2023-12-25
Progress on the deep computational basis of high-level visual functional organization https://www.biorxiv.org/content/10.1101/2023.12.19.572460v1
Compact deep neural network models of visual cortex https://www.biorxiv.org/content/10.1101/2023.11.22.568315v1.abstract
“Our paper suggests that the information flow inside Transformers can be decomposed cleanly at a macroscopic level. This gives hope that we could design safety applications to know what models are thinking or intervene on their mechanisms without the need to fully understand their internal computations.” https://www.lesswrong.com/posts/uCuvFKnvzwh34GuX3/a-universal-emergent-decomposition-of-retrieval-tasks-in
“Our hope was to understand a circuit in superposition at the parameter level, but we failed at this. We carefully falsify several naive hypotheses, but fact recall seems pretty cursed. We can black box the lookup part, so this doesn't sink the mech interp agenda, but it's a blow.” https://www.alignmentforum.org/s/hpWHhjvjn67LJ4xXX/p/iGuwZTHWb6DFY3sKB
How To Speed Up Pandas in Python By 150x https://www.youtube.com/watch?v=Yl3xCV2bN6E
Introduction to LLM Agents https://developer.nvidia.com/blog/introduction-to-llm-agents/
Will the new Mamba architecture replace Transformers? https://twitter.com/labenz/status/1738214611168395398
“This paper has received significantly less attention than it deserves, so let me shed a bit more light on it and describe why it's so good...honestly, I would even expect it to be incorporated in some GPT-5 in the future.” https://twitter.com/isskoro/status/1738661307455316236
What The Prisoner's Dilemma Reveals About Life, The Universe, and Everything https://www.youtube.com/watch?v=mScpHTIi-kM
Political links:
Parenting practices seem to have little or no impact on children's personalities, contrary to some of the best-known theories in psychology. (Longitudinal study; N = 3,880) https://online.ucpress.edu/collabra/article/7/1/29766/118998/Longitudinal-Associations-Between-Parenting-and
“Does poverty cause crime? A methodological challenge is that many other qualities are correlated with poverty, which may explain any correlation between the two. This study analyzes lottery earnings and finds no evidence that it reduces crime.” https://www.nber.org/papers/w31962
“This study of the black-white IQ gap estimates that the portion of that gap explained by genetic differences between the groups is about 60%, which is roughly consistent with the results of an experts survey published in the journal Intelligence in 2020.” https://www.mdpi.com/2624-8611/1/1/34/htm
Stopping another attempt by Russians to attack Krynky, Kherson region with a single FPV drone. https://twitter.com/sternenko/status/1738872592390344947
Views of the Russian failure north of Vodiane, #Donetsk Oblast, #Ukraine https://twitter.com/naalsio26/status/1738721314406379964