Links for 2024-06-13

AI:

1. Can LLMs invent better ways to train LLMs? “As LLMs become better at generating hypotheses and code, a fascinating possibility emerges: using AI to advance AI itself! As a first step, we got LLMs to discover better algorithms for training LLMs that align with human preferences.” https://sakana.ai/llm-squared/

2. “TiTok not only outperforms state-of-the-art diffusion model…but also reduces the image tokens by 64 × , leading to 410 × faster generation process.” https://yucornetto.github.io/projects/titok.html

3. A team at Google DeepMind has built a ‘virtual rodent’, in which an artificial neural network actuates a biomechanically realistic model of the rat. The researchers found that activations in the virtual control network accurately predicted neural activity measured from the brains of real rats producing the same behaviors. https://www.sciencedaily.com/releases/2024/06/240611130418.htm

4. Google presents TORAX, the first end-to-end differentiable simulator of tokamak heat transport. Already, major groups in nuclear fusion research are using TORAX for applications around optimizing and simulating tokamak performance. https://arxiv.org/abs/2406.06718

5. “New paper just dropped, showing how to greatly increase math scores on LLMs by combining monte-carlo tree search (MCTS) with a language model. Nice! But... what if instead, we simply tell the LLM to read the paper, and *pretend* it followed those steps?” https://x.com/jeremyphoward/status/1801037736968913128

6. How to leverage AI-synthesized data without catastrophic degradation? Rank-and-prune feedback, from humans or even weaker models, provably restores and even surpasses original performance! https://arxiv.org/abs/2406.07515

7. The Prompt Report: A Systematic Survey of Prompting Techniques https://arxiv.org/abs/2406.06608

8. Towards Lifelong Learning of LLMs https://arxiv.org/abs/2406.06391

Compute:

1. Nvidia Conquers Latest AI Tests​ — GPU maker tops new MLPerf benchmarks on graph neural nets and LLM fine-tuning https://spectrum.ieee.org/mlperf-nvidia-conquers

2. Giant Chips Give Supercomputers a Run for Their Money — Cerebras’s wafer-scale chips excel at molecular dynamics and AI inference https://spectrum.ieee.org/cerebras-wafer-scale-engine

Technology:

1. Customizable fibre networks to create on-skin electrodes that can record electrocardiogram and electromyography signals, skin-gated organic electrochemical transistors and augmented touch and plant interfaces. https://www.nature.com/articles/s41928-024-01174-4

2. Scientists Achieve Million-Fold Energy Enhancement in Diamond Optical Antennas https://pme-cms.prod.uchicago.edu/news/quantum-optical-antennas-provide-more-powerful-measurements-atomic-level

Miscellaneous:

1. Paul Erdős once took a bet that he could quit taking amphetamines for a month: "You've showed me I'm not an addict. But I didn't get any work done. I'd get up in the morning and stare at a blank piece of paper. I'd have no ideas, just like an ordinary person. You've set mathematics back a month." https://x.com/cremieuxrecueil/status/1800985280658149563

2. On Self-Delusion and Bounded Rationality https://www.scottaaronson.com/writings/selfdelusion.html

Politics:

1. “These have undoubtedly been the wildest 72 hours in French politics in my lifetime. Pretty incredible stuff.” https://x.com/RnaudBertrand/status/1801114239572328663

Ukraine:

1. “The 14th separate regiment released footage of them striking an SA-17 Grizzly also known as the Buk-M2 air defense complex with HIMARS projectiles.” https://x.com/NOELreports/status/1800843704489627832

2. Ukrainian loitering munition strike on the Russian radio relay station R-416GM. Kursk region of Russia https://x.com/bayraktar_1love/status/1801144249737465976

3. “Russian channel ASTRA reports that on the night of June 12, occupied Crimea was attacked by 12 ATACMS missiles, of which 7 were reportedly shot down. The strikes resulted in the destruction of two S-400 missile launchers and a radar station near the Belbek airfield in Sevastopol, and damage to another radar station near Cape Fiolent.” https://x.com/NOELreports/status/1801184200395035016

4. G-7 to Tap Frozen Russian Assets for $50 Billion to Ukraine https://www.bloomberg.com/news/articles/2024-06-13/g-7-agree-how-to-tap-frozen-russian-assets-for-50b-to-ukraine [no paywall: https://archive.is/ZvWuZ]