Links for 2024-03-30
AI:
Microsoft and OpenAI are drawing up plans for a $100 billion AI supercomputer. The supercomputer, codenamed "Stargate," would contain *millions* of GPUs and require several gigawatts of power. https://www.reuters.com/technology/microsoft-openai-planning-100-billion-data-center-project-information-reports-2024-03-29/
“When prompting language models to complete a task, users often leave important things unsaid. Can language models teach themselves to ask clarifying questions? In STaR-GATE, we explore LMs' ability to self-improve by rewarding the model for generating useful questions!” https://arxiv.org/abs/2403.19154
The future of education: personal AI tutors — "What if I told you there could be a TA that never got tired, knew the syllabus and what a student knew, and could help any student learn any subject? That's Shepherd (YC W24). Here it is solving a math problem ChatGPT can't." https://x.com/garrytan/status/1773491917256577203
“Robust Conformal Prediction under Distribution Shift via Physics-Informed Structural Causal Model” uses physics informed causal model PI-SCM to improve models’ domain generalization by capturing physical causality underneath available data. https://arxiv.org/abs/2403.15025
FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions https://arxiv.org/abs/2403.15246
Nvidia is partnering with Hippocratic AI to introduce AI "nurses" for virtual patient care tasks. They’re AI avatars you actually talk to, they’ll cost $9 an hour, and they’re being tested in over 40 healthcare providers. https://www.morningbrew.com/daily/stories/2024/03/22/nvidia-powered-ai-nurses-scrubbing-in
Read more about algorithmic progress:
With today's algorithms, computers would have beat the world chess champion already in 1994 on a contemporary desk computer https://www.lesswrong.com/posts/75dnjiD8kv2khe9eQ/measuring-hardware-overhang
Algorithmic progress has yielded more gains than classical hardware efficiency https://openai.com/blog/ai-and-efficiency/
While improvements in hardware accounted for an approximate 1,000-fold increase in calculation speed over a 15-year time span, improvements in algorithms accounted for an over 43,000-fold increase https://www.johndcook.com/blog/2015/12/08/algorithms-vs-moores-law/
Related:
Nvidia predicts AI models one million times more powerful than ChatGPT within 10 years https://www.pcgamer.com/nvidia-predicts-ai-models-one-million-times-more-powerful-than-chatgpt-within-10-years/
“In 2022, a model with 70%+ MMLU score, would cost 20 dollars per 1M tokens (instructGPT 3.5). Today it costs less than $1! It is perfectly reasonable to expect that in say five years, you will be able to use a model with 90%+ MMLU score for just a few cents per 1M tokens.” https://x.com/_jasonwei/status/1773825589440602418
Encryption:
An extremely practical explanation of zero-knowledge proofs, i.e. proving that you know a value x without conveying any information other than the fact that you know the value x. https://youtu.be/3NL0ThdvWMU?si=J0It935H8s3jwpnE&t=307
Here is another video that explains zero-knowledge proofs in more detail. https://youtu.be/5ovdoxnfFVc
Cryptographers Unveil Breakthrough in Achieving Indistinguishability Obfuscation -- Encrypting a computer program so the code is unintelligible but all the functionality is retained—can be achieved. https://forbes.com/sites/tonybradley/2020/12/15/cryptographers-unveil-breakthrough-in-achieving-indistinguishability-obfuscation/ [no paywall: https://archive.is/8lLu6]
Miscellaneous:
Logical probability and the strength of mathematical conjectures: What sense can be made of the notion of the probability that an unproved mathematical conjecture is true? https://philsci-archive.pitt.edu/16562/
Geothermal is the hottest thing in clean energy. Here’s why https://www.canarymedia.com/articles/geothermal/gothermal-is-the-hottest-thing-in-clean-energy-heres-why
A New Treatment Rejuvenates Aging Immune Systems in Elderly Mice https://singularityhub.com/2024/03/29/a-new-treatment-rejuvenates-aging-immune-systems-in-elderly-mice/
Mental health:
The hot new book saying smartphones and social media are warping our kids' minds will sell like hot cakes because it speaks to parents' worries - but it's bad science says this review "telling stories that are unsupported by research". https://www.nature.com/articles/d41586-024-00902-2 [archived version: https://archive.is/I7vcX]
“Long COVID brain fog might stem from damaged blood vessels letting unwanted substances into the brain. MRI scans showed that in patients with brain fog, a dye indicating blood vessel damage leaked into brain areas crucial for language and memory.” https://www.sciencenews.org/article/long-covid-brain-fog-blood-brain-barrier-damage
Ukraine:
"Denying Russia’s Only Strategy for Success" https://understandingwar.org/backgrounder/denying-russia%E2%80%99s-only-strategy-success
Commander-in-Chief Oleksandr Syrskyi published an interview with Ukrinform https://x.com/wartranslated/status/1773634829965299826
Fully unmanned drone combat in Donetsk Oblast. https://x.com/Osinttechnical/status/1774062457746604463