Links for 2024-02-21
AI:
Using an LLM to rephrase text documents to be "in high quality English language as in sentences on Wikipedia" can achieve ~3x faster LLM pertaining https://arxiv.org/abs/2401.16380
Reformatted Alignment: Merely by reformatting the response, LLM performance improves significantly (e.g. LLaMA-2-13B on GSM8K: 46% -> 56%) https://arxiv.org/abs/2402.12219
Learning to Learn Faster from Human Feedback with Language Model Predictive Control https://robot-teaching.github.io/
“How do you design a network that can optimize (edit, transform, ...) the weights of another neural network? Our latest answer to that question: *Universal* Neural Functionals (UNFs) that can process the weights of *any* deep architecture.” https://arxiv.org/abs/2402.05232
“Over the past few months, Google DeepMind, Microsoft, Amazon, Meta, Apple, OpenAI, and various academic labs have all published research that uses an AI model to improve another AI model, or even itself, in many cases leading to notable improvements. Numerous tech executives have heralded this approach as the technology’s future.” https://www.theatlantic.com/technology/archive/2024/02/artificial-intelligence-self-learning/677484/ [no paywall: https://archive.is/zRCku]
Google presents How to Train Data-Efficient LLMs: Models trained on ASK-LLM data consistently outperform full-data training—even when we reject 90% of the original dataset, while converging up to 70% faster https://arxiv.org/abs/2402.09668
Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation https://arxiv.org/abs/2402.10210
Miscellaneous:
Was the high-profile recent Nature result about science becoming less disruptive and innovative due to a software bug? https://zenodo.org/records/10656941/preview/Dataset_Artefacts_are_the_Hidden_Drivers_of_the_Declining_Disruptiveness_in_Science.pdf
“We find that standard population growth projections imply larger reductions in [per capita] income than even the most extreme widely-adopted climate change scenario.” https://marginalrevolution.com/marginalrevolution/2024/02/sentences-to-ponder-126.html
New time crystal stable for more than 40 minutes: Nobel Prize on the way? https://www.youtube.com/watch?v=sTbYKPP7q3E
Is Eric Turkheimer a Scientist? https://cremieux.medium.com/is-eric-turkheimer-a-scientist-ed5850b028d1
Politics:
Indicted ex-FBI informant told investigators he got Hunter Biden dirt from Russian intelligence officials https://edition.cnn.com/2024/02/20/politics/biden-former-fbi-informant-russian-intelligence/index.html
Ukraine:
“Reported as the first images from the military training ground that was visited by Ukrainian HIMARS. Lots of Russians were killed.” https://x.com/NOELreports/status/1760067464065360103
“A couple of days ago they’ve been attacked by Russian marines (40th Marine brigade)...Attacks were repelled. Let’s take a look” https://x.com/OSINTua/status/1760014876049551529
Russian 'war correspondent' Andrei 'Murz' Morozov has committed suicide after saying that Russia had lost 16,000 men in the battle for Avdiivka https://x.com/francis_scarr/status/1760217486018920705
“Not only does Balitsky openly admit to forced deportations (which are war crimes) but he also hints at executions of the pro-Ukrainian residents of Melitopol.” https://x.com/NatalkaKyiv/status/1760154124958449971