Links for 2024-08-27

AI:

1. What if you could use all the computing power in the world to train a shared, open source AI model? DisTrO (Distributed Training Over-the-Internet) is a family of architecture-agnostic and network-agnostic distributed optimizers that reduces the inter-GPU communication requirements by 1000x to 10,000x. Enables low-latency training of large neural networks on slow internet bandwidths with heterogeneous networking hardware. Preliminary report: https://github.com/NousResearch/DisTrO/blob/main/A_Preliminary_Report_on_DisTrO.pdf

2. “Perplexity is the first generalized AI chatbot I’ve found useful enough to integrate into any part of my daily workflow, much less across multiple domains.” https://www.lesswrong.com/posts/CcJ2YWPqcMsWg8vGt/perplexity-wins-my-ai-race-1

3. AI-powered coding pulls in almost $1bn of funding to claim ‘killer app’ status https://www.ft.com/content/4868bd38-613c-4fa9-ba9d-1ed8fa8a40c8 [no paywall: https://archive.is/SY7PJ]

4. "Chips or Not, Chinese AI Pushes Ahead" (chip starvation driving Chinese DL flight to cheap/specialized/edge uses, away from SOTA leading-edge LLMs) https://www.wsj.com/tech/ai/chips-or-not-chinese-ai-pushes-ahead-31034e3d [no paywall: https://archive.is/lhdhG]

5. "Is Xi Jinping an AI doomer? China’s elite is split over artificial intelligence" https://www.economist.com/china/2024/08/25/is-xi-jinping-an-ai-doomer [no paywall: https://archive.is/7E6Ea]

6. AlphaFold2 knows some protein folding principles https://www.biorxiv.org/content/10.1101/2024.08.25.609581v1

7. Learning to Move Like Professional Counter-Strike Players https://arxiv.org/abs/2408.13934

8. Training-free Long Video Generation with Chain of Diffusion Model Experts https://arxiv.org/abs/2408.13423

9. Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences https://arxiv.org/abs/2407.09499

10. Training independent subnetworks for robust prediction [published in 2020] https://arxiv.org/abs/2010.06610

11. AI Spots Cancer and Viral Infections at Nanoscale Precision https://www.crg.eu/en/news/ai-spots-cancer-and-viral-infections-nanoscale-precision

12. Machine learning algorithm improves 3D printing efficiency https://www.engineering.com/machine-learning-algorithm-improves-3d-printing-efficiency/

13. Astrophysicists Use AI to Precisely Calculate Universe’s ‘Settings’ https://www.simonsfoundation.org/2024/08/26/astrophysicists-use-ai-to-precisely-calculate-universes-settings/

14. Building a nuclear fusor in your bedroom with Claude Sonnet 3.5 + Projects? https://x.com/hud_zah/status/1827057785995141558

15. Stuart Russel on "AI: What if we succeed?" (25 April 2024) https://www.youtube.com/watch?v=UvvdFZkhhqE

16. What’s Really Going On in Machine Learning? Some Minimal Models https://writings.stephenwolfram.com/2024/08/whats-really-going-on-in-machine-learning-some-minimal-models/

“Cost of 1M tokens has dropped from $180 to $0.75 in ~18 months (240X cheaper!)” https://x.com/eladgil/status/1827521805755806107

https://x.com/elonmusk/status/1828205685386936567

---

Few realize that an increasing number of large companies are now either primarily AI companies or have at least established their own AI research departments.

Disney, for example, is no longer just about movies. The company is building humanoid robots that can move like humans: https://www.youtube.com/watch?v=Q2I7u0tjlJs

At Tesla, a lot of research is going into autonomous driving and humanoid robots: https://x.com/elonmusk/status/1827981493924155796

X is now primarily focused on competing with OpenAI.

The same goes for Meta, Amazon, Apple, etc.

We estimate [our GenAI assistant] has saved us the equivalent of 4,500 developer-years of work.

— Amazon (https://x.com/ajassy/status/1826608791741493281)

But analysts are still saying that AI is a bubble. As if all these companies are delusional and sinking billions of dollars into AI for nothing.

Neuroscience:

1. In language-processing areas of the brain, some cell populations respond to one word, while others respond to strings of words. https://news.mit.edu/2024/scientists-find-neurons-process-language-on-different-timescales-0826

2. Why do varied DNN designs yield equally good models of human vision? Paper shows that diverse DNNs represent images with a shared set of latent dimensions, and these shared dimensions turn out to also be the most brain-aligned. https://arxiv.org/abs/2408.12804

3. Possible evidence for the Law of General Intelligence in honeybees [PDF] https://gwern.net/doc/iq/animal/2024-penaherreraaguirre.pdf

4. Do Cleaner Fish (Labroides dimidiatus) Have General Cognitive Ability? A Reanalysis of Individual Differences Data and Consideration of Phylogenetic Context [PDF] https://gwern.net/doc/iq/animal/2023-woodley.pdf

5. New study may have cracked the code for a type of working memory in C. elegans https://www.biorxiv.org/content/10.1101/2024.08.11.607402v1

Technology:

1. Compute Moonshots: Harnessing New Physics https://www.mackenziemorehead.com/compute-moonshots/

2. Ever wanted to use Python and recreate the first Monte Carlo simulations that were made on an IBM 701 in the 50s? That's just 1 of the goal of "Learn molecular simulations with Python". https://mdcourse.github.io/

3. A low-cost, cross-platform teleoperation system. Fully open-source. https://ace-teleop.github.io/

Politics:

“Not a single country in the entire world has a higher fertility rate than in 1950.” https://x.com/StatisticUrban/status/1828171338760196549

1. “Elevating the Status of Motherhood Solves Low Birthrates: The Extraordinary Case of Mongolia...For 68 years, Mongolian leaders have given the Order of Maternal Glory to mothers. This raised the status of motherhood and helped forge a remarkably pronatal culture...In 🇲🇳, the president himself gives an award to every mother of four!” https://x.com/MoreBirths/status/1827418468813017441

2. “I will argue that the root of this coordination problem lies in the very mechanisms that shaped our species: natural selection. This evolutionary process, operating as a trial-and-error optimization algorithm, prioritizes immediate survival and reproduction over long-term, global outcomes. As a result, our innate tendencies often favor short-term gains and localized benefits, even when they conflict with the greater good of our species and planet.” https://www.lesswrong.com/posts/q3YmKemEzyrcphAeP/darwinian-traps-and-existential-risks

3. The staggering death toll of scientific lies https://www.vox.com/future-perfect/368350/scientific-research-fraud-crime-jail-time

4. Thoughts on seed oil https://www.lesswrong.com/posts/DHkkL2GxhxoceLzua/thoughts-on-seed-oil

5. Is Telegram really an encrypted messaging app? https://blog.cryptographyengineering.com/2024/08/25/telegram-is-not-really-an-encrypted-messaging-app/

6. In a First, a Chinese Military Plane Breaches Japan’s Territorial Airspace https://www.nytimes.com/2024/08/26/world/asia/china-surveillance-japan-airspace.html [no paywall: https://archive.is/VdBou]

Ukraine:

1. “Footage of the Russians launching Kalibr cruise missiles from ships of the Black Sea Fleet this morning. 28 Kalibr missiles were launched by Russians. And in total 236 missiles and drones of different types were launched during today's Russian attack on Ukraine.” https://x.com/bayraktar_1love/status/1828108011459788837

2. “The majority of units previously fighting in Kharkiv direction have sent their whole unit or part of their units to Kursk to stop the Ukrainian advances. We are additionally seeing a lot of Naval Infantry and VDV. I think the will try to push Ukraine out.” https://x.com/DefMon3/status/1828108166854533626

3. Ukraine's commander-in-chief, Col. Gen. Oleksandr Syrskiy, says Russia has re-deployed around 30,000 troops to Kursk, away from other frontline positions, to defend against the Ukrainian incursion there. The situation near Pokrovsk remains tough, he says. https://x.com/Mike_Eckel/status/1828400761966842265

4. “It happened again: Ukrainian missiles strike training base of Russian armed formations with numerous militants getting hit.” https://x.com/wartranslated/status/1828054614224392479

5. Ukraine counts on new long-range weapon to bypass Western restrictions and hit deep into Russia https://apnews.com/article/ukraine-russia-war-long-range-missile-palianytsia-1f0a1eaf560bb2c1fd70ddd56a39968d

6. Ukraine has successfully tested a new domestically produced ballistic missile, per Ukrainian president Zelensky. https://x.com/NOELreports/status/1828404648908120416

7. “Around noon today, a large explosion rocked the Russian Gazpromneft-ONPZ Refinery in Omsk.” https://x.com/Osinttechnical/status/1828042609803010445

8. "Took 15 years to build, and one week to waste it" - says a Russian employee of the Proletarsk oil depot which was ruined in a fire that lasted over a week after it was hit by Ukrainian drones. https://x.com/wartranslated/status/1828341950279278931

9. Ukrainian Fight Drone Simulator - Announcement Trailer https://www.youtube.com/watch?v=adjyRud79uY

10. More Americans want the US to stay the course in Ukraine as long as it takes https://www.brookings.edu/articles/more-americans-want-the-us-to-stay-the-course-in-ukraine-as-long-as-it-takes/

11. Russia’s Eliminationist Rhetoric Against Ukraine: A Collection https://www.justsecurity.org/81789/russias-eliminationist-rhetoric-against-ukraine-a-collection/

12. “A fascinating story by Russian soldier Platon Mamatov about the early stage of the "special operation". The Russian troops were in chaos, they lost a lot of people in extremely idiotic situations.” https://x.com/wartranslated/status/1828344052841894238