Links for 2024-08-13
AI:
Our system is capable of executing the entire ML research lifecycle: from inventing research ideas and experiments, writing code, to executing experiments on GPUs and gathering results.
The AI Scientist can produce entire scientific papers that exceed the acceptance threshold at a top machine learning conference as judged by our automated reviewer.
In one run the agent tried to change its own code by removing some obstacles, to better achieve its (completely unrelated) goal.
Read more: https://sakana.ai/ai-scientist/
Code: https://github.com/SakanaAI/AI-Scientist
Introducing Genie... the most capable AI software engineering system. It achieves state-of-the-art on SWE-Bench with 30.08%. That's a 57% improvement! https://cosine.sh/blog/genie-technical-report
Open and closed-ended problem solving in humans and AI: The influence of question asking complexity https://www.sciencedirect.com/science/article/pii/S1871187124001366
rStar: a self-play mutual reasoning approach that significantly improves reasoning capabilities of small language models (SLMs) without fine-tuning or superior models. https://arxiv.org/abs/2408.06195
Tree Attention, an exact attention approach with less communication and memory requirements than Ring Attention, enabling more efficient scaling to million token sequence lengths https://arxiv.org/abs/2408.04093
Combining GraphRAG and VectorRAG leads to a HybridRAG system that outperforms both individually. https://arxiv.org/abs/2408.04948
Transformers are energy-based models in disguise. Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters https://arxiv.org/abs/2408.04093
The ChatGPT of databases. 100% open source. https://postgres.new/
Nvidia’s AI chips are good, but it’s the company’s coding prowess that gives it the real edge https://www.wsj.com/tech/ai/ai-nvidia-apple-amd-jensen-huang-software-bb581f5a [no paywall: https://archive.is/gOKZo]
China uses LLaMa-3 to train a semiconductor advice LLM https://arxiv.org/abs/2408.00804
A clip from the GPT-4o safety card where the voice model suddenly yells "No!" and then starts imitating the user's voice https://www.reddit.com/r/singularity/comments/1enne2l/comment/lh7zsb4/
“One thing I was very wrong about ~4yrs ago is how fundamental “synthetic data” in ML would be. “Obviously” due to data-proc-inequality, synth data should not help learning. The key flaw in this argument is: we do not use info-theoretically optimal learning methods in practice” https://x.com/PreetumNakkiran/status/1821928149908848869
OpenAI expert Scott Aaronson on consciousness, quantum physics and AI safety https://scottaaronson.blog/?p=8200
Biotech:
Is It Ethical To Hand-Pick Your Child’s Genes? https://www.youtube.com/watch?v=e3cXRs60xiU
Why Does Ozempic Cure All Diseases? https://www.astralcodexten.com/p/why-does-ozempic-cure-all-diseases
A bacterial antiviral immune machinery creates a new gene de novo just from an RNA to fend off viruses. https://www.biorxiv.org/content/10.1101/2024.05.08.593200v1
A Novel Treatment Slashes HIV Up To 10,000-Fold in Monkeys With Just a Single Dose https://singularityhub.com/2024/08/12/a-novel-treatment-slashes-hiv-up-to-10000-fold-in-monkeys-with-just-a-single-dose/
Lenacapavir PrEP Trial Brings Down the House at the International AIDS Conference — When the results of a new drug that prevented 100% of HIV cases were announced at the 2024 AIDS conference, the room burst into spontaneous applause https://blogs.jwatch.org/hiv-id-observations/index.php/lenacapavir-prep-trial-brings-down-the-house-at-the-international-aids-conference/2024/07/25/
Neuroscience:
Demonstration that sublinear summation in dendrites can unlock the computation of nonlinear functions by a single neuron https://www.nature.com/articles/s41598-024-65866-9
“By combining photochemical sectioning with volumetric lattice light-sheet imaging and petabyte-scale computation, we imaged and reconstructed axons and myelination sheaths across entire mouse olfactory bulbs at nanoscale resolution.” https://www.biorxiv.org/content/10.1101/2024.08.01.605857v1
Astronomy:
“The search for alien life just got a bit more complicated. Red dwarfs, young and dim stars thought by many astronomers to be the most likely hosts for life-sustaining planets, come with a significant drawback: They frequently emit deadly ultraviolet radiation flares, which are much more powerful than astronomers previously thought.” https://gizmodo.com/the-search-for-alien-life-just-hit-a-depressing-setback-2000483836
Mars water: Reservoir found deep in Martian rocks https://www.bbc.com/news/articles/czxl849j77ko
Miscellaneous:
“The single most undervalued fact of linear algebra: Matrices are graphs, and graphs are matrices. Encoding matrices as graphs is a cheat code, making complex behavior simple to study.” https://x.com/svpino/status/1822966303642308903
Politics:
Anduril has secured $1.5B in Series F funding to hyperscale defense manufacturing. Introducing Arsenal-1, a 5M+ sq ft state-of-the-art facility to produce autonomous weapons systems. https://www.anduril.com/article/anduril-raises-usd1-5-billion-to-rebuild-the-arsenal-of-democracy/
Billions of dollars of venture capital is flowing into defense-tech startups focused on futuristic, AI-enabled weapons. Palmer Luckey’s Anduril is their biggest bet. https://www.wsj.com/tech/anduril-drones-palmer-luckey-china-ukraine-china-951494ec [no paywall: https://archive.is/uWfOR]
Ukraine:
"As of today, 28 settlements are under enemy control, the depth of penetration into the territory of the Kursk region is 12 kilometers, and the front's width is 40 kilometers" - Kursk region acting governor told Putin. Putin interrupted him and said not to mention numbers. https://x.com/Gerashchenko_en/status/1822976702726754525
Ukrainian commander-in-chief Col. Gen. Oleksandr Syrskiy: "Currently, about 1,000 square kilometers of Russian territory is under the control of [Ukraine's] Armed Forces." https://x.com/Mike_Eckel/status/1823030300785446990
“Another Russian group was reportedly ambushed and taken captive in the Kursk region. The amount of captured servicemen already exceeds 1200-1300 according to several sources.” https://x.com/NOELreports/status/1823287289490948449
Ukraine produces over 1 million drones annually, which is twice the amount manufactured by the U.S. https://www.defenseone.com/policy/2024/08/wartime-need-drones-would-outstrip-us-production-theres-way-fix/398642/
The Myth of Endless Manpower: Russian Soldiers’ Average Age Approaches 38 as Trends Persist https://frontelligence.substack.com/p/the-myth-of-endless-manpower-russian
Good thread on Russian casualty estimates: “Those lead us to conclude the Donbas units have lost 20,000-23,500 men. That brings the total death count on the Russian side to between 110,220 and 138 325 troops KIA. More details can be found here” https://x.com/oivshina/status/1823021706757480826
“The Russians are increasing the number of attacks on Ukraine with North Korean ballistic missiles. It’s already the 4th missile attack by North Korean KN-23 during the last two weeks.” https://x.com/bayraktar_1love/status/1823268974726971880