Links for 2024-10-15
AI:
Dario Amodei (CEO of Anthropic) says AGI - what he calls powerful AI - could come as early as 2026 and it is possible that 1000 years of progress could happen in the 5-10 years following https://www.lesswrong.com/posts/DxvzLngnWHhvdCGLf/dario-amodei-machines-of-loving-grace
Nuclear energy is making a comeback due to the belief that AI will transform the world. Following Microsoft, Google is also turning to nuclear power for its data centers. They are truly putting their money where their mouth is. https://blog.google/outreach-initiatives/sustainability/google-kairos-power-nuclear-energy-agreement/
US Weighs Capping Exports of AI Chips From Nvidia and AMD to Some Countries https://www.bloomberg.com/news/articles/2024-10-15/us-weighs-capping-exports-of-ai-chips-from-nvidia-and-amd-to-some-countries [no paywall: https://archive.is/TWzIq]
Meta presents Thinking LLMs: General Instruction Following with Thought Generation — Superior performance on AlpacaEval and Arena-Hard. Gains from thinking on even non-reasoning categories such as marketing, health, and general knowledge. https://arxiv.org/abs/2410.10630
LeanAgent: Lifelong Learning for Formal Theorem Proving — “It performs up to 11× better than the static LLM baseline, proving challenging theorems in domains like abstract algebra and algebraic topology while showcasing a clear progression of learning from basic concepts to advanced topics.” https://arxiv.org/abs/2410.06209
Unleashing System 2 Thinking? AlphaCodium Outperforms Direct Prompting of OpenAI o1 https://www.qodo.ai/blog/system-2-thinking-alphacodium-outperforms-direct-prompting-of-openai-o1/
AI Agents Could Collaborate on Far Grander Scales Than Humans, Study Says https://singularityhub.com/2024/10/11/ai-agents-could-collaborate-on-far-grander-scales-than-humans-study-says/
Circuits in Superposition: Compressing many small neural networks into one https://www.lesswrong.com/posts/roE7SHjFWEoMcGZKd/circuits-in-superposition-compressing-many-small-neural
An industry that communicates with unstructured documents turns to generative AI to clean up its processes. And it's delivering real returns. https://www.bigtechnology.com/p/wheres-the-generative-ai-roi-start
VCs Can't Get Enough of AI for Lawyers https://www.newcomer.co/p/vcs-cant-get-enough-of-ai-for-lawyers
Tamay Besiroglu on AI in 2030: Scaling, Automation, and AI Agents https://www.youtube.com/watch?v=HyCFtWx5rX0
Features are fate: a theory of transfer learning in high-dimensional regression https://arxiv.org/abs/2410.08194v1
INTELLECT–1: Launching the First Decentralized Training of a 10B Parameter Model https://www.primeintellect.ai/blog/intellect-1
AMD is launching an AI chip to rival Nvidia’s Blackwell GPUs. https://www.cnbc.com/2024/10/10/amd-launches-mi325x-ai-chip-to-rival-nvidias-blackwell-.html
Apple Paper:
If you look at the results of the new Apple paper, which supposedly shows that AI models cannot reason, you find three things:
1. Larger models perform notably better than smaller ones.
2. Scaling inference compute improves model performance even further.
3. They never assessed how much human performance was impacted by the same tricks they used to challenge AI models.
One way they tried to trick these models was to change the names and numbers that appear in problems. This reduces their performance. But is that surprising?
Imagine telling a person that pressing a green button opens a door and pressing a red button closes it. If they perform this action thousands of times, their brain creates an efficient, ingrained routine. Now, if you reverse the button functions, it takes time and effort for the person to adapt. They'll likely continue pressing the green button to open the door until the old habit fades.
Ironically, the paper's results show that models like OpenAI's o1, which allocate more effort to breaking down learned information, experience much smaller performance declines. In other words, models designed for reasoning do reason.
In summary:
1. We wouldn't conclude that humans can't reason just because of cognitive hiccups like the Stroop effect. Why, then, should similar challenges in AI be taken as evidence of an absence of reasoning? In fact, these similarities suggest we're on the right path.
2. The evidence clearly shows that scaling works, and there's no reason to believe it won't continue improving future models.
Satire:
Can humans truly reason? We explore this key question by changing the context of abstract philosophical thought experiments to partisan hot-button political issues. We observe a LARGE drop in performance, but also an increase in variance, making humans increasingly unreliable.
This begs the question: Do humans truly understand rational decision-making? Introducing #Party_Switch! We reversed the position of a political party on two major and salient issues. Check what happens next! Human political opinions changed immediately and substantially when their party switched its policy position – even when the new position went against humans previously held views. While it'll be interesting to see how AI models perform in similar tests, we doubt the drop-off would be as severe.
Can scaling human IQ and better education solve this? We find that smarter humans tend to perform significantly better on abstract thought experiments, but only become more sophisticated at self-deception and hallucinating seemingly plausible but flawed arguments when dealing with real-world political issues.
Overall, we found no evidence for independent reasoning in humans. Human behavior is better explained by sophisticated groupthink - so fragile, in fact, that changing the names of political parties can alter policy evaluations by ~10%!
Miscellaneous:
Supercritical geothermal looking more and more like it's going to work! https://www.nature.com/articles/s41467-024-52092-0
One-shot entorhinal maps enable flexible navigation in novel environments https://www.nature.com/articles/s41586-024-08034-3
MIT team takes a major step toward fully 3D-printed active electronics https://news.mit.edu/2024/mit-team-takes-major-step-toward-fully-3d-printed-active-electronics-1015
‘Phenomenal’ tool sequences DNA and tracks proteins — without cracking cells open https://www.nature.com/articles/d41586-024-03276-7
Dark Food: Feeding People In Space Without Photosynthesis https://essopenarchive.org/users/536288/articles/599019-dark-food-feeding-people-in-space-without-photosynthesis
Big Advance on Simple-Sounding Math Problem Was a Century in the Making https://www.quantamagazine.org/big-advance-on-simple-sounding-math-problem-was-a-century-in-the-making-20241014/
Politics:
On China's and Europe's failures to innovate. https://www.cremieux.xyz/p/chinas-upside-down-meritocracy
A threat of quantum computing to Bitcoin? “To any of you who are worried about post-quantum cryptography—by now I’m so used to delivering a message of, maybe, eventually, someone will need to start thinking about migrating from RSA and Diffie-Hellman and elliptic curve crypto to lattice-based crypto, or other systems that could plausibly withstand quantum attack. I think today that message needs to change. I think today the message needs to be: yes, unequivocally, worry about this now. Have a plan.” https://scottaaronson.blog/?p=8329
Did a top NIH official manipulate Alzheimer's and Parkinson’s studies for decades? https://www.science.org/content/article/research-misconduct-finding-neuroscientist-eliezer-masliah-papers-under-suspicion
"A combination of new gene editing techniques... sequencing methods & DNA synthesis tools is opening a new world of possibilities in synthetic biology for greater precision in genetic manipulation &..a new world of risks from...powerful bioweapons" https://www.cnas.org/publications/reports/ai-and-the-evolution-of-biological-national-security-risks
“U.S. officials don’t know who is behind the drones that have flown unhindered over sensitive national-security sites—or how to stop them” https://www.wsj.com/politics/national-security/drones-military-pentagon-defense-331871f4 [no paywall: https://archive.is/qUmPm]
Deadly drone attack by Hezbollah exposes Israel’s weaknesses https://edition.cnn.com/2024/10/14/middleeast/hezbollah-attack-israel-weaknesses-drones-intl-cmd/index.html
Ukraine:
"If we want Ukraine to be able to stand against Russia effectively, it must have competitive fighter jets at its disposal," Zagorodnyuk said. "We cannot develop any strategy or campaign plan to fight against Russia without airpower." https://www.newsweek.com/ukraine-f16-russia-fighter-jets-1968602
“A very interesting graphic is being discussed on Russian channels this evening, including by the well-known Zakhar Prilepin. It reflects the results of a survey on attitudes toward the "Military Actions" in Russia, broken down by profession. As we can see, the overwhelming majority of science workers, that is, the most educated segment of the population, are against military actions. A similar thing can be said about those who are obtaining this education—almost half of the students do not support the war.” https://x.com/wartranslated/status/1845557144793608473
Assessment of the situation in Kursk direction https://x.com/emilkastehelmi/status/1845858146901782574
Zelensky says North Korea has sent troops to support Russian military https://www.france24.com/en/europe/20241013-zelensky-says-north-korea-has-sent-troops-to-support-russian-military