Links for 2024-10-11
AI:
Google DeepMind leaders share Nobel Prize in Chemistry for protein prediction AI https://www.nature.com/articles/d41586-024-03214-7
How close is DeepMind to achieving AGI? Nobel Laureate Demis Hassabis says: We're on track and DeepMind is targeting completion of AGI by 2030, but I wouldn't be surprised if it's in the next decade. https://youtu.be/pZybROKrj2Q?si=ovoUbPic9SUpG4n0&t=2962
Semantic Training Signals Promote Hierarchical Syntactic Generalization in Transformers https://adityayedetore.github.io/assets/pdf/emnlp_2024_semantic_cues_to_hierarchy.pdf
Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning. Achieves 5 − 6× gain in sample efficiency, 1.5 − 5× more compute-efficiency, and > 6% gain in accuracy, over ORMs on test-time search. https://arxiv.org/abs/2410.08146
LLMs are in-context RL learners, but not great because they can’t explore well. How do we teach LLMs to explore better? Solution: Supervised fine-tuning on full exploration trajectories. https://arxiv.org/abs/2410.06238
Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification https://arxiv.org/abs/2410.05318
Differential Transformer outperforms Transformer when scaling up model size and training tokens. https://arxiv.org/abs/2410.05258
Can we build more capable AI agents by learning from cognitive science? Cognitive Architectures for Language Agents (CoALA) introduces a structured approach to design AI Agents by integrating cognitive architecture principles with modern LLMs. https://arxiv.org/abs/2309.02427
LLMs have original, research-worthy ideas https://learnandburn.ai/p/llms-have-original-research-worthy
The spontaneous emergence of “a sense of beauty” in untrained deep neural networks. https://psycnet.apa.org/record/2025-32757-001
Complexity exposure drives intelligence in LLMs, with optimal performance at the "edge of chaos." https://www.arxiv.org/abs/2410.02536
Math transformers learn better when trained from repeated examples. https://arxiv.org/html/2410.07041v1
LLMs Can In-context Learn Multiple Tasks in Superposition https://arxiv.org/abs/2410.05603
“We collected an additional set of ~20 newly released models, including the most capable open models to date, like Llama 3.1-405B. Our preregistered predictions accurately extrapolate to these models.” https://x.com/YangjunR/status/1844462548353155135
“I think there is a good chance that normalizing flow-based variational inference will displace MCMC as the go-to method for Bayesian posterior inference as soon as everyone gets access to good GPUs.” https://statmodeling.stat.columbia.edu/2024/10/08/defining-statistical-models-in-jax/
AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs https://arxiv.org/abs/2410.05295
Ancestor simulations: Large Language Models based on historical text could offer informative tools for behavioral science https://www.pnas.org/doi/10.1073/pnas.2407639121
Can AI Outpredict Humans? Results From Metaculus's Q3 AI Forecasting Benchmark https://www.lesswrong.com/posts/LHdNtJCm93pxNHJKb/can-ai-outpredict-humans-results-from-metaculus-s-q3-ai
We might be closer to a fast takeoff than many people expected:
OpenAI on automated ML research capabilities:
...agents capable of performing open-ended ML research tasks, at the level of improving their own training code, could improve the capabilities of frontier models significantly faster than human researchers. If innovations are produced faster than our ability to understand their impacts, we risk developing models capable of catastrophic harm or misuse without parallel developments in securing, aligning, and controlling such models.
Read more: https://openai.com/index/mle-bench/
See also this comment by Bogdan Ionut Cirstea: https://www.lesswrong.com/posts/wr2SxQuRvcXeDBbNZ/bogdan-ionut-cirstea-s-shortform?commentId=dyndDEn9qqdt9Mhx2
How much AI compute is out there, and who owns it?
NVIDIA has likely sold the equivalent of around three million H100 GPUs in computing power since early 2022.
Google has millions of TPUs, or the equivalent to at least ~600,000 NVIDIA H100s in computing power.
Note: 1 million H100 equivalents is roughly 1 million kW = 1 gigawatt of compute. Times two months that is a 10^28 FLOP, 1 megaton of TNT AI model.
Learn more: https://epochai.org/data/notable-ai-models#computing-capacity
Training GPT-4 reportedly cost around $100 million. In 2024, OpenAI is expected to invest $3 billion in training new models. The advancements ahead are set to be monumental!
Technology:
Expansion microscopy seems to be able to expand proteins to the extent that their structure is viewable by optical microscopy. But not literally. They anchor the protein to gel, then light it up like a Christmas tree with NHS-fluorescein, then expand the gel, which breaks apart the protein but (with great effort) keeps the fluorescein particles at the same relative angles. https://www.nature.com/articles/s41587-024-02431-9
Google says its research shows the existence of a "stable computationally complex phase" is reachable with current quantum processors. Even with noise, these quantum computers can perform calculations that are beyond the capabilities of classical supercomputers https://research.google/blog/validating-random-circuit-sampling-as-a-benchmark-for-measuring-quantum-progress/
Holographic 3D printing has the potential to revolutionize multiple industries, say Concordia researchers https://www.concordia.ca/news/stories/2024/10/08/holographic-3d-printing-has-the-potential-to-revolutionize-multiple-industries-say-concordia-researchers.html
Miscellaneous:
A new study adds evidence that consciousness requires communication between sensory and cognitive regions of the brain’s cortex. https://news.mit.edu/2024/how-sensory-prediction-changes-under-anesthesia-tells-us-how-conscious-cognition-works-1010
Values Are Real Like Harry Potter https://www.lesswrong.com/posts/a5hpPfABQnrkfGGxb/values-are-real-like-harry-potter
Additive, multiplicative, and exponential economics — “A simple, and hardly unique economic observation: when you are poor, money is additive. As you get more, it becomes multiplicative. And eventually exponential.” https://aleph.se/andart2/uncategorized/additive-multiplicative-and-exponential-economics/
Ukraine:
Ukrainian drones attacked an ammunition depot in Karachev, Bryansk region, Russia. The 67th GRAU arsenal (~3.5 km²) storing ammo, including from North Korea, was hit. Detonations have begun, and despite claims of 12 drones being shot down, the situation seems out of control. https://x.com/Osinttechnical/status/1843894863198335376
Ukraine has destroyed a warehouse believed to be housing some 400 Iranian drones. Videos show major secondary explosions. https://x.com/bayraktar_1love/status/1844090412689977509 (satellite image: https://x.com/bayraktar_1love/status/1844714265355165764)
Ukrainian drones attacked the military airfield "Khansk" near the Russian city of Maykop. Local residents observe large clouds of smoke rising from the airport in the morning. Residents are currently being evacuated from Rodnikovo, near Maykop. https://x.com/bayraktar_1love/status/1844366888450785512 (satellite image: https://x.com/NOELreports/status/1844735292038647871)
Russian marine oil terminal in Feodosia, Crimea. Four days after it was attacked by Ukrainian drones. https://x.com/Osinttechnical/status/1844472439281164563
Ukraine also suffered a major loss today when Russia is believed to have hit a Patriot radar along with its mission control station. https://x.com/WarVehicle/status/1844048895841878173
Ukraine is moving to lift its wartime ban on drone exports to boost production and match Russia's capabilities. Drone companies aim to sell abroad, generating up to $20bn in revenue to fund more military supplies. https://www.ft.com/content/aec4c3b3-56ab-4774-b342-250d5445ba6e [no paywall: https://archive.is/rctfL]
Very nice airburst fragmentation warhead for drones. Useful against soft targets, primarily infantry, but I would also love seeing a swarm of these packed onto a larger (aerial or sea) drone and realeased against airbases or logistics hubs. https://x.com/AndrewPerpetua/status/1843674086377173220
“The 🇺🇦Ukrainian 4th Battalion "Syla Svobody" shares a video of a repelling Russian attack in the Donetsk region on 24.05.2024. This video is not the work of several months. This is a record of a single powerful attack on our positions.” https://x.com/GloOouD/status/1844448972573933703
Finnish President Alexander Stubb in an interview with Fox News https://x.com/Gerashchenko_en/status/1843645835143192744
“Barack Obama visited Ukraine only ONE time, in 2005 as senator. He was involved in destroying crucial weapons in Donetsk that Ukraine later needed to defend when Russia attacked. They promised to protect Ukraine, but little help was provided in 2014.” https://x.com/eurovanya/status/1843695127447150741
“Despite repeated warnings, Ukrainian President Volodymyr Zelensky dismissed the idea that Putin would actually invade, even after Vice President Kamala Harris told him during a February 2022 meeting at the Munich Security Conference that an invasion was imminent.” https://edition.cnn.com/2024/10/08/politics/bob-woodward-book-war-joe-biden-putin-netanyahu-trump/index.html
Former Prime Minister of Australia Malcolm Turnbull: “When you see Trump with Putin, as I have on a few occasions, he’s like the 12-year-old boy that goes to high school and meets the captain of the football team. ‘My hero!’ It’s really creepy…the creepiness was palpable” https://x.com/RpsAgainstTrump/status/1843832767974707214
Michael Weiss explains Russia's destabilization, espionage and assasination efforts across the western world. https://x.com/JayinKyiv/status/1843903462016463301
“I just finished my basic training course in the Ukrainian Army! That was an adventure, however it prepared me for the real fight against r*ssian invasion. This is how Ukraine trains its soldiers” https://x.com/dim0kq/status/1844035699751919757