Links for 2024-05-27
AI:
Meta presents Automatic Data Curation for Self-Supervised Learning: Features trained on their automatically curated datasets outperform ones trained on manually curated data https://arxiv.org/abs/2405.15613
Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization — “…we demonstrate that for a challenging reasoning task with a large search space, GPT-4-Turbo and Gemini-1.5-Pro based on non-parametric memory fail badly regardless of prompting styles or retrieval augmentation, while a fully grokked transformer can achieve near-perfect accuracy, showcasing the power of parametric memory for complex reasoning.” https://arxiv.org/abs/2405.15071
AutoCoder, the first Large Language Model to surpass GPT-4 Turbo (April 2024) and GPT-4o in pass@1 on the Human Eval benchmark test (90.9% vs. 90.2%). https://arxiv.org/abs/2405.14906
Luban: Building Open-Ended Creative Agents via Autonomous Embodied Verification https://arxiv.org/abs/2405.15414
iVideoGPT: Interactive VideoGPTs are Scalable World Models https://arxiv.org/abs/2405.15223
Look Once to Hear: Target Speech Hearing with Noisy Examples. Look once to hear is an intelligent hearable system where users choose to hear a target speaker by just looking at them for a few seconds. https://github.com/vb000/LookOnceToHear
Top VC Kai-Fu Lee says his prediction that AI will displace 50% of jobs by 2027 is ‘uncannily accurate’ https://fortune.com/2024/05/25/ai-job-displacement-forecast-50-percent-2027-kai-fu-lee-chatgpt-openai/ [Unlikely]
How long until AGI? Elon Musk says next year. https://x.com/elonmusk/status/1794240517875920935 [Very unlikely]
Musk AI startup raises $6B https://x.ai/blog/series-b
Bird Intelligence:
"Among others, bird cognition encompasses abilities such as delay of gratification, mental time travel, reasoning, metacognition, mirror self-recognition, theory of mind, and third-party intervention." http://cell.com/trends/cognitive-sciences/abstract/S1364-6613(16)00042-5
African grey can perform some cognitive tasks at levels beyond that of 5-year-old humans. https://news.harvard.edu/gazette/story/2019/02/harvard-study-shows-parrots-can-pass-classic-test-of-intelligence/
Cognitive performance of four-months-old ravens may parallel adult apes https://phys.org/news/2020-12-cognitive-four-months-old-ravens-parallel-adult.html
New Caledonian crows plan for specific future tool use https://royalsocietypublishing.org/doi/10.1098/rspb.2020.1490
Crows keep special tools extra safe. Study suggest that crows have some concept of the relative 'value' of different tool types. https://www.mpg.de/18061418/1217-ornr-crows-keep-special-tools-extra-safe-987453-x
These crows have counting skills previously only seen in people https://www.nature.com/articles/d41586-024-01482-x
For the first time, research reveals crows use statistical logic https://arstechnica.com/science/2023/09/for-the-first-time-research-reveals-crows-use-statistical-logic/
New Zealand birds are capable of statistical inference. https://www.science.org/content/article/new-zealand-birds-show-humanlike-ability-make-predictions
Study Finds That Crows Are So Intelligent They Understand the Concept of Zero https://mymodernmet.com/crows-understand-zero/
Crows and magpies using anti-bird spikes to build nests, researchers find https://www.theguardian.com/science/2023/jul/11/crows-and-magpies-show-their-metal-by-using-anti-bird-spikes-to-build-nests
Brainiacs, not birdbrains: Crows possess higher intelligence and conscious processes long thought a primarily human attribute — "Research unveiled in Science finds that crows know what they know and can ponder the content of their own minds, a manifestation of higher intelligence and analytical thought long believed the sole province of humans and a few other higher mammals." https://www.statnews.com/2020/09/24/crows-possess-higher-intelligence-long-thought-primarily-human/
How did birds get so smart? Part of the answer is that they've got primate-like numbers of neurons squeezed into their tiny bird brains. Another part is that avian neurons consume three times less glucose than mammalian neurons. https://doi.org/10.1016/j.cub.2022.07.070
“Corvids and parrots with brains of 5–20 g show cognitive properties similar to those of great apes like chimpanzees with brains of about 400 g…many associative pallial neurons, a pallial area that overtakes functions of the mammalian prefrontal cortex, dense dopaminergic innervation of associative pallial areas, and flexible neural fundaments of working memory…” https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(23)00281-4
Songbird species with more-complex vocal learning have larger brains and greater intelligence, consistent with "a long-standing assumption of a link between human spoken language, advanced cognition, and larger brain size." https://www.science.org/doi/10.1126/science.adh3428
Evolution:
Bats and dolphins evolved echolocation in the same way (down to the molecular level). https://www.science.org/content/article/bats-and-dolphins-evolved-echolocation-same-way [no paywall: https://archive.is/0nmcB]
Unlike humans, octopuses have evolved eyes without a blind spot. https://www.pbs.org/wgbh/evolution/change/grand/page05.html
Depression, schizophrenia and bipolar disorder linked with ancient viral DNA in our genome – new research https://theconversation.com/depression-schizophrenia-and-bipolar-disorder-linked-with-ancient-viral-dna-in-our-genome-new-research-230490
Miscellaneous:
The [central limit theorem] would have been personified by the Greeks and deified, if they had known of it. It reigns with serenity... amidst the wildest confusion... the greater the anarchy, the more perfect its sway. It is the supreme law of Unreason.
— Galton
A tiny raycasting engine and city generator that fits in a standalone 256 byte html file. https://frankforce.com/city-in-a-bottle-a-256-byte-raycasting-system/
“So here's a story of, by far, the weirdest bug I've encountered in my CS career.” https://x.com/CupiaBart/status/1793930355617259811
