Links for 2025-01-13
AI:
MIDAS speeds up language model training by up to 40%. This method is a type of "gradual stacking," where a smaller model is trained first, and its middle layers are reused to initialize a deeper model. This process is repeated, gradually increasing the model's depth. While MIDAS-trained models may have similar or slightly worse perplexity (a measure of how well a model predicts the next word in a sequence) compared to traditional training methods, they perform significantly better on downstream reasoning tasks. MIDAS shares some conceptual similarities with Universal Transformers but focuses on static depth expansion during training rather than dynamic depth adjustment during inference. https://arxiv.org/abs/2409.19044
Building AI Research Fleets https://www.lesswrong.com/posts/WJ7y8S9WdKRvrzJmR/building-ai-research-fleets
Superhuman forecaster seems reachable in 2025 https://arxiv.org/abs/2412.18544
Training Transformers for simple next token prediction on videos leads to competitive performance across all benchmarks. https://arxiv.org/abs/2501.05453
Optimizing LLM Test-Time Compute Involves Solving a Meta-RL Problem https://blog.ml.cmu.edu/2025/01/08/optimizing-llm-test-time-compute-involves-solving-a-meta-rl-problem/
Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning https://arxiv.org/abs/2412.15797
Creating a LLM-as-a-Judge That Drives Business Results https://hamel.dev/blog/posts/llm-judge/
Grokking at the Edge of Numerical Stability https://arxiv.org/abs/2501.04697
Ethan Mollick suggests that recent predictions about the imminent arrival of superintelligent AI by industry insiders could signal that something unprecedented is happening https://www.oneusefulthing.org/p/prophecies-of-the-flood
Can transformers be scaled up to AGI? Ilya Sutskever: Obviously, yes https://youtu.be/Ft0gTO2K85A?si=ab3ADAzLoUr4n5Ns&t=1680
Samsung announced a $181M investment into Rainbow Robotics, expanding its presence in humanoid robotics https://news.samsung.com/global/samsung-electronics-to-become-largest-shareholder-in-rainbow-robotics-accelerating-future-robot-development
Web UI for interacting with Qwen (Alibaba) models including their reasoning model https://chat.qwenlm.ai/
For the people who say AGI is always 20 years away, here is Google DeepMind co-founder and chief AGI scientist Shane Legg, who has maintained a remarkable consistency since I last asked him about it in 2011:
Harvard professor who also works at OpenAI:
OpenAI safety researcher:
https://x.com/McaleerStephen/status/1878555949662666895
AI politics:
What would happen if remote work were fully automated? Matthew Barnett argues the economic impact would be massive—with the economy doubling in size even in the most conservative scenario. https://epoch.ai/gradient-updates/consequences-of-automating-remote-work
Once robots can do physical jobs, how quickly could they scale up? Converting car factories might produce 1 billion robots annually in under 5 years. Here are some maths for rapid robot deployment. https://www.lesswrong.com/posts/6Jo4oCzPuXYgmB45q/how-quickly-could-robots-scale-up
David Dalrymple on Safeguarded, Transformative AI https://www.youtube.com/watch?v=MPrU69sFQiE
Human takeover might be worse than AI takeover https://www.lesswrong.com/posts/FEcw6JQ8surwxvRfr/human-takeover-might-be-worse-than-ai-takeover
“Chips, data, energy and talent are the keys to winning on AI—and this is a race America can and must win.” https://openai.com/global-affairs/openais-economic-blueprint/
NVIDIA CEO Jensen Huang: "the critical technologies necessary to build general humanoid robotics is just around the corner" and an aging population and declining birthrate makes this imperative as the world needs more workers https://youtu.be/Z_DR1_zhmCU?si=3-yePRXlqzQtTHeX&t=65
Wall Street Job Losses May Top 200,000 as AI Replaces Roles https://www.bloomberg.com/news/articles/2025-01-09/wall-street-expected-to-shed-200-000-jobs-as-ai-erodes-roles [no paywall: https://archive.is/RfiRH]
41% of companies worldwide plan to reduce workforces by 2030 due to AI https://edition.cnn.com/2025/01/08/business/ai-job-losses-by-2030-intl/index.html
A lawyer for Elon Musk has called on the California and Delaware attorneys-general to force OpenAI to auction off a large stake in its business, intensifying a bitter fight with the company's chief executive Sam Altman. https://www.ft.com/content/596dddb3-0607-44b9-b565-9c0d62e49b9f [no paywall: https://archive.is/LO337]
T1 Blue - unrestricted buying and data center building
T2 Yellow - maximum national purchase between now and 2027 capped at fifty thousand GPUs
T3 Red - restricted
See also: FACT SHEET: Ensuring U.S. Security and Economic Strength in the Age of Artificial Intelligence https://www.whitehouse.gov/briefing-room/statements-releases/2025/01/13/fact-sheet-ensuring-u-s-security-and-economic-strength-in-the-age-of-artificial-intelligence/
Brains:
“Yet more evidence that Alzheimer's is caused by human herpesvirus variants. The HHV family of viruses is almost certainly responsible for a very wide variety of horrifying human illnesses. (EBV, for example, is the root cause of Multiple Sclerosis.)” (via Perry E. Metzger) https://www.science.org/doi/10.1126/scisignal.ado6430
The Lateral Prefrontal Cortex appears de novo in primates. Its role: a mental workspace where sensory signals and memories are combined and multiplexed creating complex spatio-temporal patterns that guide intelligent behaviours. https://www.cell.com/cell-reports/fulltext/S2211-1247%2824%2901475-X
The human reward system encodes the subjective value of ideas during creative thinking https://www.nature.com/articles/s42003-024-07427-4
Psychology:
Are the average genetic scores for intelligence decreasing between birth cohorts? https://www.emilkirkegaard.com/p/dysgenics-within-and-between
New study finds enhanced creativity in autistic adults is linked to co-occurring ADHD rather than autism itself (N=352). https://psycnet.apa.org/fulltext/2025-66159-001.html
Computer science:
“Above my pay grade: Jensen Huang and the quantum computing stock market crash” https://scottaaronson.blog/?p=8567
“The single axiom ((a•b)•c)•(a•((a•c)•a))=c is a complete axiom system for Boolean algebra (and is the simplest possible)” https://writings.stephenwolfram.com/2025/01/who-can-understand-the-proof-a-window-on-formalized-mathematics/
The purposeful drunkard https://www.lesswrong.com/posts/s39XbvtzzmusHxgky/the-purposeful-drunkard
Efficient CPA Attack on Hardware Implementation of ML-DSA in Post-Quantum Root of Trust — The Dilithium implementation in Google and Microsoft's Caliptra root of trust just got hacked. By measuring the switching power consumption of internal pipeline registers, attackers extracted keys with just 10,000 power traces. [PDF] https://eprint.iacr.org/2025/009.pdf
Biology:
Heritable polygenic editing: the next frontier in genomic medicine? Very large potential gains in long-term health from completely removing certain bad alleles present in our collective gene pool. https://www.nature.com/articles/s41586-024-08300-4
Variant effects depend on polygenic background: experimental, clinical, and evolutionary implications https://www.biorxiv.org/content/10.1101/2025.01.07.631805v1
Obelisks are viroid-like things that probably live in your mouth. https://www.biorxiv.org/content/10.1101/2024.01.20.576352v1
Politics:
Good immigrants, bad immigrants: Dutch edition https://www.emilkirkegaard.com/p/good-immigrants-bad-immigrants-dutch
How did the American Civil War Actually Happen? https://www.youtube.com/watch?v=bYaYCltLsdk
Danish intelligence accused Russia of forging a 2019 letter to Senator Tom Cotton, claiming to be from Greenland's foreign minister and alleging there'd be an independence referendum. https://www.reuters.com/world/europe/denmark-accuses-china-russia-iran-espionage-threat-2022-01-13/
Afgantsy Redux: How Russian military intelligence used the Taliban to bleed U.S. forces at the end of America’s longest war https://theins.ru/en/politics/277723