Links for 2024-12-23
AI:
What o3 Becomes by 2028 — “We haven't seen AIs made from compute optimal LLMs pretrained on these systems yet, but the systems were around for 6+ months, so the AIs should start getting deployed imminently, and will become ubiquitous in 2025.” https://www.lesswrong.com/posts/NXTkEiaLA4JdS5vSZ/what-o3-becomes-by-2028
Orienting to 3 year AGI timelines https://www.lesswrong.com/posts/jb4bBdeEEeypNkqzj/orienting-to-3-year-agi-timelines
“AI that exceeds human performance in nearly every cognitive domain is almost certain to be built and deployed in the next few years. We need to act accordingly now.” https://milesbrundage.substack.com/p/times-up-for-ai-policy
Thread on how impressive o3's math performance is: “For context, FrontierMath is a brutally difficult benchmark with problems that would stump many mathematicians. The easier problems are as hard as IMO/Putnam; the hardest ones approach research-level complexity. With earlier models like o1-preview, Pass@1 performance (solving on first attempt) was only around 2%. When allowing 8 attempts per problem (Pass@8) and counting problems solved at least once, we saw ~6% performance. o3's 25.2% at Pass@1 is substantially more impressive...This is notable because our earlier tests showed only a few percentage points performance gained per OOM. o3's increase to 25% suggests both improved per-token reasoning and better scaling behavior.” https://x.com/tamaybes/status/1870333137374544077
“Genesis is a nice simulator with lots of cool features, and simulation indeed is something I strongly believe in when it comes to scaling up synthetic data for training and evaluating embodied AI / robotics...However, the main takeaway is that Genesis is not as fast as reported (it is slower by >100x than claimed), and compared to an existing GPU sim Genesis is slower by 3-10x on environments with slightly more collisions / complex dynamics.” https://stoneztao.substack.com/p/the-new-hyped-genesis-simulator-is
Stanford researchers introduced a new system that can generate physically plausible human-object interactions from natural language https://hoifhli.github.io/
ExBody2 an advanced whole-body controller for humanoid robots https://exbody2.github.io/
“Can neuroscience localizers uncover brain-like functional specializations in LLMs? Yes! We analyzed 18 LLMs and found units mirroring the brain's language, theory of mind, and multiple demand networks!” https://arxiv.org/abs/2411.02280
A biologically-inspired hierarchical convolutional energy model predicts V4 responses to natural videos https://www.biorxiv.org/content/10.1101/2024.12.16.628781v1
LlamaFusion: Adapting Pretrained Language Models for Multimodal Generation https://arxiv.org/abs/2412.15188
Memory Layers at Scale https://arxiv.org/abs/2412.09764
The Surprising Effectiveness of Test-Time Training for Abstract Reasoning https://arxiv.org/abs/2411.07279
Streaming LifeLong Learning With Any-Time Inference [published in 2023] https://arxiv.org/abs/2301.11892
Large Language Models as Tool Makers [published in 2023] https://arxiv.org/abs/2305.17126
Deliberative alignment: reasoning enables safer language models https://openai.com/index/deliberative-alignment/
Learning to Reason, Insights from Language Modeling https://www.youtube.com/watch?v=YR9EztOF0R8
It never ceases to amaze me how much progress has been made in the last few years: Hannah Fry shows what Google DeepMind's Project Astra - an AI model that can understand video - can do, such an identifying objects and situational context, and analyzing what it sees https://www.youtube.com/watch?v=ctWfv4WUp2I
Anthropic co-founder Jack Clark says when he and Dario Amodei went to the White House in 2023 to meet with Gina Raimondo and Kamala Harris they were told, "We've got our eye on you guys. AI is going to be a really big deal and we're now actually paying attention" https://youtu.be/om2lIWXLLN4?si=ySlUNtsBzZVe7Tc-&t=705
UN Secretary-General Antonio Guterres tells the UN Security Council that "those that feel like technology is moving very fast must understand the simple fact: Technology will never move in the future as slowly as today," saying that AI is revolutionizing the world and tasks that required years of human expertise are now completed in a heartbeat, which also poses huge risks https://x.com/tsarnick/status/1871016318247604624
Yann LeCun addressing the UN Security Council says AI will profoundly transform the world in the coming years, amplifying human intelligence, accelerating progress in science, solving aging and decreasing populations, surpassing human intellectual capabilities to become superintelligent and leading to a new Renaissance and a period of enlightenment for humanity https://x.com/tsarnick/status/1870962281250701381
The Heist. Every shot was done via text-to video with Google Veo 2. https://www.youtube.com/watch?v=lFc1jxLHhyM
Circa 2040:
So the AI cured cancer, I get it. But it feels like brute force to me. Not really reasoning. They had to spend tens of millions of dollars on compute to get that result.
Some people are making us believe that we're really close to AGI. We're actually very far from it. I mean, when I say very far, it's… several years.
— Yann LeCun (Source: https://youtu.be/UmxlgLEscBs?si=kvyxHTNPPyMTtX0J&t=1654)
ARC benchmark:
Five years of OpenAI models vs. the ARC-AGI benchmark:
Website of the benchmark: https://arcprize.org/
Note also: Sometimes o3’s answers were more correct than the expected output “solution” https://imgur.com/a/QxdrwsL
Technology:
Aqueous Homogeneous Miniature Reactors Could Supply U.S. Bases With Unlimited Fuel https://www.forbes.com/sites/davidhambling/2024/12/05/miniature-reactors-could-supply-us-bases-with-unlimited-fuel/
Are Amazon’s Drones Finally Ready for Prime Time? https://www.nytimes.com/2024/12/20/technology/amazon-prime-air-drone-delivery.html [no paywall: https://archive.is/Hufi3]
Cancer:
Game-Changing Dual Cancer Therapy Completely Eradicates Tumors Without Harsh Side Effects https://news.mit.edu/2024/implantable-microparticles-can-deliver-two-cancer-therapies-1028
Cancer cells can pierce the immune cells send after them with intracellular nanotubes, and pull out their mitochondria for their own use! https://www.nature.com/articles/s41565-021-01000-4
Miscellaneous:
Are most of senescent cells immune cells? https://www.nature.com/articles/s12276-024-01354-4
When Is Insurance Worth It? https://www.lesswrong.com/posts/wf4jkt4vRH7kC2jCy/when-is-insurance-worth-it
Politics:
Ukraine:
North Korea may have started supplying kamikaze drones to Russian forces in Ukraine and is preparing to increase its military presence in Russia, South Korean intelligence reports. https://m-en.yna.co.kr/view/AEN20241223002700315?section=nk/nk
North Korea is ramping up its weapons deliveries to Russia. 60% of the artillery and mortar shells used by Russia in Ukraine now come from Pyongyang. https://www.wsj.com/world/russia-north-korea-weapons-shipment-676d7f52 [no paywall: https://archive.is/OffDs]
North Korea Likely Transferred KN-15 Missile Systems to Russia https://mil.in.ua/en/news/north-korea-likely-transferred-kn-15-missile-systems-to-russia/
Ukrainian drones blew up an ammunition depot in Novocherkassk, Rostov region of Russia. The video shows the moment of the explosion with a beautiful mushroom cloud. https://x.com/NOELreports/status/1870834612450300179
Kupyansky direction. The 14th Brigade repels a powerful enemy assault using armored vehicles. https://x.com/olddog100ua/status/1870744944438124622
A massive Russian assault with the use of atypical vehicles captured on video! https://www.youtube.com/watch?v=5VkQ6R1lEMg
Multiple targets in Rylsk, Kursk region, were hit by HIMARS. A video shows the aftermath of the strike near the Culture Palace. https://x.com/NOELreports/status/1870234745046483143
“Kazan, Russia – This morning, the city was attacked by drones, with several hitting high-rise buildings. A total of 6 to 8 explosions were reported across the city. One possible target was the Kazan Helicopter Plant.” https://x.com/wartranslated/status/1870389379081904255
“Video of a Russian armored column on the Kupiansk front repelled with FPV strikes by the Achilles strike UAV battalion, 116th Mechanized Brigade, 1st National Guard Brigade, and 114th TDF Brigade.” https://x.com/RALee85/status/1870242632095408475
“Russia just intentionally bombed the regional cancer hospital in Kherson, claiming without evidence that the top floors were being used to launch UAVs. This is in the same area that drones have been indiscriminately targeting civilians who dare to venture outside of their homes.” https://x.com/KyleJGlen/status/1870239842950377529
Russia is executing more and more Ukrainian prisoners of war https://www.bbc.com/news/articles/c7ve11lr247o
“Russian-occupied Mariinka, Ukriane. The hellscape of the “Russian world.” For the umpteenth time, this is not that different from the aftermath of a nuclear strike.” https://x.com/IAPonomarenko/status/1870602221177414126
“Ukrainian soldier of the 60th Mechanised Brigade destroys Russian stormtrooper in small arms combat in Lyman direction, Donetsk region” https://x.com/GloOouD/status/1870585402735927718
Electric scooters may not be the best form of transport for frontline attacks. https://x.com/bayraktar_1love/status/1870879891731677429