Links for 2023-06-14
I-JEPA: The first AI model based on Yann LeCun’s vision for more human-like AI
[Meta AI:] Today we're releasing our work on I-JEPA — self-supervised computer vision that learns to understand the world by predicting it. It's the first model based on a component of Yann LeCun's vision to make AI systems learn and reason like animals and humans.
Achieves SOTA by training 632M parameter visual transformer model using 16 A100 GPUs in under <72 hrs. Other methods take 10 times more GPU-hours.
I-JEPA captures common-sense knowledge through self-supervised learning from unlabeled data.
The model predicts missing information at a high level of abstraction, avoiding generative model limitations.
Read more: https://ai.facebook.com/blog/yann-lecun-ai-model-i-jepa/
A high-fidelity universal neural audio compression algorithm that achieves ~90x compression of 44.1 KHz audio into tokens at just 8kbps bandwidth. https://arxiv.org/abs/2306.06546
Understanding DeepMind's Sorting Algorithm https://justine.lol/sorting/
Augmenting Language Models with Long-Term Memory https://arxiv.org/abs/2306.07174
The past few years have seen breakthroughs in language modeling. Will these continue? Can we predict the performance of future language models? https://epochai.org/blog/extrapolating-performance-in-language-modelling-benchmarks
The Entire Field of Actuarial Psychology Summarized into a List of Findings https://www.reddit.com/r/slatestarcodex/comments/pecop0/the_entire_field_of_actuarial_psychology/
Gently Jolting the Brain With Electrical Currents Could Boost Cognitive Function https://singularityhub.com/2023/06/05/gently-jolting-the-brain-with-electrical-currents-could-boost-cognitive-function/
Viable offspring derived from single unfertilized mammalian oocytes https://www.pnas.org/doi/full/10.1073/pnas.2115248119
A “Twelve Step Program” to revive birthrates around the world https://twitter.com/MoreBirths/status/1663202168516628481
Scott Aaronson's lecture notes on Quantum Information Science https://www.scottaaronson.com/qclec.pdf
People conform to the moral judgments of the group in an online Asch-style conformity experiment: “[W]e showed that some individuals might have problems standing against a group even if that group says something that goes against their moral values.“ https://link.springer.com/article/10.1007/s12144-023-04765-0
Leading population geneticist David Reich discussing advances in our understanding of population diversity. Reich says the claim that "there isn't space for substantial differences" between population groups "simply is not true". (via Paolo Shirasi) https://www.youtube.com/watch?v=SJ1npkBQGV8&t=4061s