Links for 2024-04-12
AI:
Microsoft presents Rho-1: RHO-1-1B and 7B achieves SotA results of 40.6% and 51.8% on MATH dataset, respectively — matching DeepSeekMath with only 3% of the pretraining tokens. https://github.com/microsoft/rho
From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examples — “Our results demonstrate that large language models are capable of doing regression when given in-context examples of (input, output) pairs, despite not being explicitly trained to do so.” https://github.com/robertvacareanu/llm4regression
THOUGHTSCULPT: Reasoning with Intermediate Revision and Search — “…outperforms state-of-the-art reasoning methods across three challenging tasks: Story Outline Improvement (up to +30% interestingness), Mini-Crosswords Solving (up to +16% word success rate), and Constrained Generation (up to +10% concept coverage).” https://arxiv.org/abs/2404.05966
RoT: Enhancing Large Language Models with Reflection on Search Trees https://arxiv.org/abs/2404.05449
A reminder that fully autonomous self-driving cars already exist and are being rolled out: “Tech startup Waymo said Tuesday that it would begin offering paid robotaxi rides in Los Angeles beginning Wednesday, as the nation’s experiment with self-driving car technology picks up steam.” https://www.nbcnews.com/tech/innovation/waymo-will-launch-paid-robotaxi-service-los-angeles-wednesday-rcna147101
Ophiology (or, how the Mamba architecture works) https://www.lesswrong.com/posts/TYLQ8gAMAmpeFcwXN/ophiology-or-how-the-mamba-architecture-works
Finding high quality data lets you use both less data and compute to train a foundation model. https://www.youtube.com/watch?v=fmI_OciHV_8
Miscellaneous:
Inside The Hard Tech Startups Turning Sci-Fi Into Reality: Supersonic jets. 3D printed rockets. Asteroid mining. Satellite launches into space. Mobile carbon capture. Engineering organisms. Fusion energy. https://www.youtube.com/watch?v=erDE2e69dlc
Quasar J059-4351 has the mass of 17 billion Suns, devours 2x10^27 tons of gas per day and shines 40,000 times brighter than all the stars in the Milky Way... combined. https://phys.org/news/2024-02-brightest-fastest-astronomers-quasar.html
A Gentle Introduction to the Art of Mathematics (a free, open-source textbook) https://giam.southernct.edu/GIAM/
Politics:
‘A U.S. official with knowledge of the matter said Thursday that American intelligence reports indicate an Iranian retaliatory strike within days, “possibly on Israeli soil” as opposed to Israeli interests elsewhere.’ https://www.cbsnews.com/news/israel-iran-hamas-war-us-travel-warning-middle-east-conflict/
“schooling years turn out to be almost linearly predictive of low fertility, whether within a country over time, or across countries.” https://medium.com/@lymanstone/fertility-and-income-some-notes-581e1a6db3c7
Ukraine:
List Of Naval Losses During The Russian Invasion Of Ukraine https://www.oryxspioenkop.com/2022/03/list-of-naval-losses-during-2022.html