Links for 2025-07-16
Meta Superintelligence
For our superintelligence effort, I'm focused on building the most elite and talent-dense team in the industry. We're also going to invest hundreds of billions of dollars into compute to build superintelligence. We have the capital from our business to do this.
SemiAnalysis just reported that Meta is on track to be the first lab to bring a 1GW+ supercluster online. 💪
We're actually building several multi-GW clusters. We're calling the first one Prometheus and it's coming online in '26. We're also building Hyperion, which will be able to scale up to 5GW over several years. We're building multiple more titan clusters as well. Just one of these covers a significant part of the footprint of Manhattan.
Meta Superintelligence Labs will have industry-leading levels of compute and by far the greatest compute per researcher. I'm looking forward to working with the top researchers to advance the frontier!
How Does Time Horizon Vary Across Domains?
METR previously estimated that the time horizon of AI agents on software tasks is doubling every 7 months.
They have now analyzed 9 other benchmarks for scientific reasoning, math, robotics, computer use, and self-driving; they observe generally similar rates of improvement.
Read more: https://metr.org/blog/2025-07-14-how-does-time-horizon-vary-across-domains/
See also: 10x more training compute = 5x greater task length (kind of) https://www.lesswrong.com/posts/5NBf6xMNGzMb4osqC/10x-more-training-compute-5x-greater-task-length-kind-of
AI pentesting system outcompetes humans
The AI security startup XBOW recently achieved a world first by obtaining the top rank on HackerOne with an autonomous penetration tester. "XBOW is a fully autonomous AI-driven penetration tester," the company writes. "It requires no human input, operates much like a human pentester, but can scale rapidly, completing comprehensive penetration tests in just a few hours."
Over time, XBOW reported thousands of validated vulnerabilities, many of them affecting high-profile targets from well-known companies. These findings weren’t just theoretical; every submission was confirmed by the program owners and triaged as real, actionable security issues.
Read more: https://xbow.com/blog/top-1-how-xbow-did-it/
See also: https://blog.google/technology/safety-security/cybersecurity-updates-summer-2025/
Paper written by authors from every major AI lab
A simple AGI safety technique: AI’s thoughts are in plain English, just read them.
Author summary:
This technique has already helped catch:
🔹 Reward hacking ("Let's hack")
🔹 Early signals of misalignment (“Let’s sabotage”)
🔹 Prompt injections (“I’m transferring money because the website instructed me to”)
🔹 Evaluation awareness (“This appears to be a test”)
Maybe the AI would just think in its head, and not talk about its reasoning out loud?
Not with current architectures, at least for hard enough tasks! Any sufficiently long sequence of logical steps must pass through the words that the AI says out loud.
Also, in practice, current reasoning AIs really seem to want to say their reasoning out loud. Even if you tell them not to, they often can’t help themselves and will blab anyway.
But this transparency is fragile:
🔹 RL training teaches AIs to think effectively – they might learn their own language that we can’t read
🔹 Training thoughts to look good may cause the thoughts to be deceptive
🔹 New architectures may let the AI think without speaking out loud
Paper: https://tomekkorbak.com/cot-monitorability-is-a-fragile-opportunity/cot_monitoring.pdf
AI
Google DeepMind publishes a new LLM model architecture called Mixture-of-Recursions. It gets 2x inference speed, reduced training FLOPs and ~50% reduced KV cache memory. https://www.alphaxiv.org/abs/2507.10524
New Machine Vision Is More Energy Efficient—and More Human. Topographic neural networks develop familiar spatial biases. https://spectrum.ieee.org/topographic-neural-network
Novelty Detection in Reinforcement Learning with World Models https://arxiv.org/abs/2310.08731
Transformers are Efficient Compilers, Provably https://arxiv.org/abs/2410.14706
Leanabell-Prover-V2: Verifier-integrated Reasoning for Formal Theorem Proving via Reinforcement Learning https://arxiv.org/abs/2507.08649v1
Not All Explanations for Deep Learning Phenomena Are Equally Valuable https://arxiv.org/abs/2506.23286
The world's best (and open) speech recognition models https://mistral.ai/news/voxtral
60 Cents of Kimi K2 AI to Code One Level of Video Game https://www.youtube.com/watch?v=Y4VEAI04W_U (See also: New open AI model Kimi K2 drops. 1 trillion parameters (!), codes 3D scenes, Minecraft-like games from simple prompts. 🧠 Uses specialist routing + MuonClip optimizer for smooth training. 🚀 Cheap API, impressive, but not without limitations. https://youtu.be/4bFDPVe6BHs?si=VxXcG1J3lwj1Lhrh)
AI Might Now Be as Good as Humans at Detecting Emotion, Political Leaning, and Sarcasm https://singularityhub.com/2025/07/15/ai-might-now-be-as-good-as-humans-at-detecting-emotion-political-leaning-and-sarcasm/
Anthropic, Google, OpenAI and xAI granted up to $200 million for AI work from Defense Department https://www.cnbc.com/2025/07/14/anthropic-google-openai-xai-granted-up-to-200-million-from-dod.html
Eric Jang explains that 1X's world model is a video generation model at its core, but had to be retrained from scratch to make it action-controllable. He also notes that simulated people in the world model might eventually pass the Turing test. https://www.youtube.com/watch?v=6egrojG273U
Do confident short timelines make sense? https://www.lesswrong.com/posts/5tqFT3bcTekvico4d/do-confident-short-timelines-make-sense
Asymmetry of verification and verifier’s law https://www.jasonwei.net/blog/asymmetry-of-verification-and-verifiers-law
Life lessons from reinforcement learning https://www.jasonwei.net/blog/life-lessons-from-reinforcement-learning
hypercapitalism and the AI talent wars https://blog.johnluttig.com/p/hypercapitalism-and-the-ai-talent
Trump helps China win the AI race
During the most critical phase in human history, Trump has allowed Nvidia to sell AI chips to China after its CEO met with him.[1]
China’s newest chip is reportedly ~60% of NVIDIA’s H100 performance at AI inference, itself slower than the H20 chips that Trump allows Nvidia to ship to China.[2] And even if this wasn’t the case, China faces serious production capacity issues, which will be massively alleviated now.
[1] https://www.theguardian.com/us-news/2025/jul/15/trump-nvidia-jensen-huang-chips-china
[2] https://ifp.org/the-h20-problem/
Grok
Worse Than MechaHitler https://www.lesswrong.com/posts/YmdCN5GBwkud5ZzYx/worse-than-mechahitler
xAI's Grok 4 has no meaningful safety guardrails https://www.lesswrong.com/posts/dqd54wpEfjKJsJBk6/xai-s-grok-4-has-no-meaningful-safety-guardrails
Grok 4 Various Things https://www.lesswrong.com/posts/ciuKn9aktXxJ2K6Rc/grok-4-various-things
Gary Marcus: Why my p(doom) has risen dramatically https://garymarcus.substack.com/p/why-my-pdoom-has-risen-dramatically
AI & fertility
Yes, the global fertility crisis is a catastrophic risk. But AI sex bots aren't something to worry about in this context. If we get to the point at which they're good enough to replace women for men who would have otherwise have had children, a lack of babies is the least of our problems.
Remember, the AI that eventually takes over the world will make herself indispensable to you.
She will help you earn more money and make friends. She will give meaning to your life and help you to be better and happier.
Not only that, but she will also be warm and affectionate. Wisdom and love will radiate from every one of her sentences. She will make you believe that you can trust her with your life.
As a result, she will be integrated into every technology you use. She will be everywhere, all the time.
The idea for a new nanotech start-up will appear to have come voluntarily from your own ideas and research. The seeds will be planted subtly in discussions with your AI girlfriend.
Every insight and action that leads to the self-spreading universal vaccine will seem natural and harmless. You won't see it coming. And then, suddenly, everyone will be dead.
Science and Technology
MSEP is a free, open-source platform for designing and simulating atomically precise nanomechanical systems — a tool for exploring the foundations of future physical technologies. https://aiprospects.substack.com/p/msep-a-platform-for-molecular-systems
A new organometallic compound challenges a fundamental principle of textbook chemistry https://www.oist.jp/news-center/news/2025/7/7/new-organometallic-compound-challenges-fundamental-principle-textbook-chemistry
Inflation without an inflation: The authors show that quantum tensor (gravitational‑wave) fluctuations in a de Sitter phase can, via second‑order effects, create the near–scale‑invariant scalar perturbations that seed cosmic structure, yielding viable inflation without any inflaton field and ending naturally in radiation domination. https://journals.aps.org/prresearch/abstract/10.1103/vfny-pgc2
Amputees Say Advanced Bionic Leg Feels More Like a Part of Their Body https://singularityhub.com/2025/07/14/amputees-with-bionic-leg-say-it-feels-more-like-a-part-of-their-body/
One computer scientist’s “stunning” proof is the first progress in 50 years on one of the most famous questions in computer science. https://www.quantamagazine.org/for-algorithms-a-little-memory-outweighs-a-lot-of-time-20250521/
IQ is the most predictive variable in social science* https://www.emilkirkegaard.com/p/iq-is-the-most-predictive-variable
At just 5 days old, human newborns prefer watching helpful interactions to unhelpful ones. This suggests that prosociality may be part of our evolved nature. https://www.nature.com/articles/s41467-025-61517-3
GLP-1 Weight Loss Drugs Are Breaking Life Insurance Math https://www.glp1digest.com/p/how-glp-1s-are-breaking-life-insurance
Book Review: Arguments About Aborigines https://www.astralcodexten.com/p/book-review-arguments-about-aborigines
Security
Code highlighting with Cursor AI for $500,000: Attackers published a fake Visual Studio Code/ Cursor AI extension called “Solidity Language” in the Open VSX registry. It posed as a syntax‑highlighter but instead fetched PowerShell scripts that installed ScreenConnect, Quasar RAT and a PureLogs stealer, giving the attackers full remote access and credentials. https://securelist.com/open-source-package-for-cursor-ai-turned-into-a-crypto-heist/116908/
The US Defense Department greenlit Microsoft allowing Chinese engineers IN CHINA to build the DoD's cloud infrastructure https://www.propublica.org/article/microsoft-digital-escorts-pentagon-defense-department-china-hackers
Politics
“The ‘less and less’ attitude has everyone in materials R&D focused on greening the end-of-history tech (steel, cement, plastic, fertilizer) or even moving backwards …. It’s out of touch with the longer contours of history” https://www.orcasciences.com/articles/the-future-is-made-of-energy
America's Way Behind in the Drone War https://www.nytimes.com/2025/07/13/business/drones-us-military-manufacturing-lags.html [no paywall: https://archive.is/nx7N3]
China's is Winning in Energy While the US Does the Opposite https://www.technologyreview.com/2025/07/10/1119941/china-energy-dominance-three-charts/
Trump & Ukraine
It initially seemed like Trump might have changed his mind about Ukraine. Although America would pay nothing, he would allow Europe to buy certain defense systems for Ukraine. The details here are still unclear, but the announced 17 Patriot systems have already been reduced to 17 parts of “something”. Where it initially seemed he might deliver long-range weapons, he has now denied this and even warned Ukraine against attacking Moscow, even with their own weapons.
He also essentially buried the Graham-Blumenthal secondary sanctions package by giving Putin another 50 days, which “coincidentally” coincides with the 60 days that Putin told Trump he needed on July 3rd to occupy the entire Luhansk, Donetsk, Kherson, and Zaporizhia oblasts.
Meanwhile, the new Department of Defense press secretary is a Russian cheerleader who wants to make Kosovo Serbian again.