Links for 2024-07-28
AI:
Recursive Introspection: Teaching Language Model Agents How to Self-Improve https://arxiv.org/abs/2407.18219
How do we leverage AI-synthesized data without catastrophic degradation? Rank-and-prune feedback, from humans or even weaker models, provably restores and even surpasses original performance! https://arxiv.org/abs/2406.07515 (related: Model collapse is not a significant threat under current best practices https://x.com/RylanSchaeffer/status/1816881533795422404)
Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk https://arxiv.org/abs/2401.05033
The space of human goals is infinitely vast -- yet, people spontaneously infer plausible motivations for others from just a few actions. How? Infinite Ends from Finite Samples: Open-Ended Goal Inference as Top-Down Bayesian Filtering of Bottom-Up Proposals https://arxiv.org/abs/2407.16770
How to make your robot handle diverse visual scenarios? Maniwhere: A Visual Generalizable Framework For Reinforcement Learning https://gemcollector.github.io/maniwhere/
“Do neural networks dream of internal goals? We confirm RNNs trained to play Sokoban with RL learn to plan. Our black-box analysis reveals novel behaviors such as agents “pacing” to gain thinking time. We open-source the RNNs as model organisms for interpretability research.” https://far.ai/post/2024-07-learned-planners/
Discovery of Crystallizable Organic Semiconductors with Machine LearningClick to copy article link https://pubs.acs.org/doi/10.1021/jacs.4c05245
Advanced Hardware Device Slashes AI Energy Consumption by 1000x https://cse.umn.edu/college/news/researchers-develop-state-art-device-make-artificial-intelligence-more-energy
“it turns out that the reason GPT-2 couldn't multiply four-digit numbers wasn't too few layers, too small of a hidden dimension, or bad training data. all that was fine. the learning algorithm itself was the issue” https://x.com/jxmnop/status/1816958426385383753
“Why the failure? If you analyze multiplication into a graph of steps, you find models failed as if they approximately matched parts of the problem (subgraphs) based on surface similarity to training items. This can be formalized information theoretically.” https://arxiv.org/abs/2305.18654
DiVA — Distilled Voice Assistant: DiVA Llama 3 outperforms existing Speech LMs on QA, Emotion Recognition, and Translation with a speech encoder trained using only weak supervision. https://diva-audio.github.io/
Biden to receive AI national security memo outlining forbidden uses, opportunities for innovation https://www.nextgov.com/artificial-intelligence/2024/07/biden-receive-ai-national-security-memo-outlining-forbidden-uses-areas-innovation/398353/
Comments on DeepMind's artificial IMO problem-solving system
Terence Tao: “As per the "AI effect", once explained, it does not "feel" like an exhibition of human-like intelligence; but it is still an expansion of the capability of our suite of AI-assisted problem solving tools.” https://mathstodon.xyz/@tao/112850716240504978
Timothy Gowers: “…what it does is way beyond what a pure brute-force search would be capable of, so there is clearly something interesting going on when it operates.” https://x.com/wtgowers/status/1816509803407040909
Michael Nielsen: “I can't imagine betting against Gold Medal performance, with many fewer caveats, in 5 years. And 2-3 years seems likely...” https://x.com/michael_nielsen/status/1816530386681470976
Science:
Dual-action antibiotic could make bacterial resistance nearly impossible https://today.uic.edu/dual-action-antibiotic-could-make-bacterial-resistance-nearly-impossible/
The brain does not have a miraculous ability to 'rewire' itself or repurpose neurons to overcome injury or disability, despite the popular explanation. Instead it just enhances or emphasizes different parts of pre-existing structures by simple repetition https://elifesciences.org/articles/84716
Remember the supermodel granny drug that extended lifespan by 25% in mice by IL-11 last week? It only took 8 days for a new intervention to come out increasing lifespan by 35%!! https://www.biorxiv.org/content/10.1101/2024.07.25.605097v1
Physics:
Big Bang Gears: With the fastest gear turning once every 1.5 seconds and a gear ratio of 4:1 the last gear completes a full turn once every (you guessed it) 13.7 billion years!! The outer teeth on the last gear move by a proton's diameter every 25 minutes… https://x.com/TamasGorbe/status/1816761031743471847
Within the flash of a brilliant gamma-ray burst, astronomers have found a remarkable signal: Matter and antimatter destroying each other. While fleeing a newborn black hole. Moving at 99.9% the speed of light. https://science.nasa.gov/science-research/astrophysics/gamma-ray-bursts/nasas-fermi-finds-new-feature-in-brightest-gamma-ray-burst-yet-seen/
Contrary to what general relativity says, black holes can't be made solely from light - because quantum effects prevent it. https://physics.aps.org/articles/v17/119
US economy:
“The US economy has been expanding for 15 years, only interrupted by 2 months of contraction in early 2020″…and…”The US economy has now been in an expansion for 51 months with annualized real GDP growth of 4.5% over that time.” https://x.com/jmhorp/status/1816469415786918337
US economic growth ramps up, growing 2.8% in latest quarter https://www.theguardian.com/business/article/2024/jul/25/june-inflation-interest-rates
In January 2023, 99% of US CEOs surveyed expected a recession. https://x.com/jdcmedlock/status/1816522025391022198
Politics:
Russia got defeated badly in Mali. Footage of an assault group of dozens of Russian soldiers destroyed by the rebels. As claimed by Russian sources, total Russian losses in Mali are close to 100 people. For comparison: France lost around 40 KIA in Mali over 8 years (2014-2022). It was a big deal when U.S. lost 4 in Niger in 2017. https://x.com/NOELreports/status/1817297023831593248
“We now have fertility data for 2023 from Denmark, and it looks like every major group—from nonwestern migrants to native Danes—is still below replacement, and the situation is largely progressing in the wrong direction…Yes, even Muslims, Africans, and other globally high-fertility groups have low fertility after migration. It really speaks to the nature of the fertility crisis.” https://x.com/cremieuxrecueil/status/1816878701549617245
1% of people are responsible for 24% of the health spending in America and 5% of people are responsible for just over half. https://x.com/cremieuxrecueil/status/1817249178017321257
Ukraine:
“The Russians changed tactics to fight smarter, using intel to attack weak units rather than weak fortifications. They began the offensive by bringing in highly 100s of trained FPV drone operators using a new frequency Ukrainian ECM was not designed to jam and hitting logistics.” https://x.com/MaxRTucker/status/1817121404585656812
“Russian military reporter Alexander Kharchenko reports that logistics on the Russian side are getting worse over time due to the abundance of drones on the Ukrainian side. He says that even a year ago in Robotyne they were dismounted 1 kilometer from the positions, but now dismounting from armour takes place 8-10 km from the front. However, there are no available solutions to the problem yet.” https://x.com/wartranslated/status/1817122729536962935
“Ukrainian forces coordinated an attack on an advancing Russian column of armored vehicles, resulting in the destruction of tanks and BMPs. The Russian advance was halted.” https://x.com/Tendar/status/1817531805341262302