Links for 2023-09-03
The Singularity Is Near š
Inflection CEO and Google DeepMind founder: https://80000hours.org/podcast/episodes/mustafa-suleyman-getting-washington-and-silicon-valley-to-tame-ai/
Iām talking about models that are two or three or maybe four orders of magnitude on from where we are. And weāre not far away from that. Weāre going to be training models that are 1,000x larger than they currently are in the next three years. Even at Inflection, with the compute that we have, will be 100x larger than the current frontier models in the next 18 months.
See also:
Nvidia predicts AI models one million times more powerful than ChatGPT within 10 years https://www.pcgamer.com/nvidia-predicts-ai-models-one-million-times-more-powerful-than-chatgpt-within-10-years/
With today's algorithms, computers would have beat the world chess champion already in 1994 on a contemporary desk computer https://www.lesswrong.com/posts/75dnjiD8kv2khe9eQ/measuring-hardware-overhang
Algorithmic progress has yielded more gains than classical hardware efficiency: https://openai.com/blog/ai-and-efficiency/
While improvements in hardware accounted for an approximate 1,000 fold increase in calculation speed over a 15-year time-span, improvements in algorithms accounted for an over 43,000 fold increase https://www.johndcook.com/blog/2015/12/08/algorithms-vs-moores-law/
GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fields https://yanjieze.com/GNFactor/
SayTap: Language to quadrupedal locomotion https://blog.research.google/2023/08/saytap-language-to-quadrupedal.html
āMost experts were surprised by progress in language models in 2022 and 2023. There may be more surprises ahead, so experts should register their forecasts now about 2024 and 2025.ā https://www.planned-obsolescence.org/language-models-surprised-us/
OpenAI API base models are not sycophantic, at any size (the tendency to agree with a user's stated view when asked a question) https://www.lesswrong.com/posts/3ou8DayvDXxufkjHD/openai-api-base-models-are-not-sycophantic-at-any-size
A brain-inspired algorithm that mitigates catastrophic forgetting of artificial and spiking neural networks with low computational cost https://science.org/doi/full/10.1126/sciadv.adi2947
Automorphic: Infuse knowledge into language models with just 10 samples ā āAutomorphic lets you fine tune LLMs incrementally, and that means you can evolve them much faster. It's like programming in a repl.ā https://www.ycombinator.com/launches/JOw-automorphic-infuse-knowledge-into-language-models-with-just-10-samples
New models suggest a smaller PlanetX, orbiting closer in. This world, frozen and dark, would be no greater than 3 times the mass of Earth, and orbiting at high inclination no farther than 500 au, near the Sunās gravity-lens zone. https://www.sciencealert.com/scientists-think-an-earth-like-planet-may-be-hiding-in-our-solar-system
āThe organisation responsible for Japanās national defences against cyber attacks has itself been infiltrated by hackers, who may have gained access to sensitive data for as much as nine monthsā https://www.ft.com/content/de0042f8-a7ce-4db5-bf7b-aed8ad3a4cfd [https://archive.is/1Hhs2]
"On Thursday, Japanās Defense Ministry requested nearly $53 billion for its next defense budget. The 13 percent increase is the largest request in the agencyās history and the 12th consecutive raise." https://foreignpolicy.com/2023/09/01/japan-defense-budget-request-military-indo-pacific-china-north-korea/
Code provided by ChatGPT: https://chat.openai.com/share/a022e1d6-dddc-4817-8bbd-944a3e742d9f
Mastodon link: https://mathstodon.xyz/@tao/110991566012143311
Ukraine links:
āEverything is ahead of usā: Ukraine breaks Russian strongholdās first line of defence https://www.theguardian.com/world/2023/sep/02/everything-is-ahead-of-us-ukraine-breaks-russias-first-line-of-defence-in-stronghold
āSelect Russian sources claimed that Russian officers of the 58th Combined Arms Army (CAA) defending in Zaporizhia Oblast contacted former 58th CAA commander Major General Ivan Popov due to the worsening situation at the Russian frontline.ā https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-september-2-2023
āThe US plans to supply Ukraine with AMRAAM air-to-air missiles with a range of 160-180 kilometers, which are capable of hitting Russian aircraft without entering the air defense zone,ā said Yuri Ignat, spokesman of the Ukrainian Air Force said. https://twitter.com/NOELreports/status/1698003170285203923
āI was in Kupyansk for two days this week. Russians are not advancing. They are located about seven kilometers from the center - for a while now, without gaining territory. Soldiers on the spot: āthey will never cross our linesā. Occasional bombing of houses and coffee shops. Yes.ā https://twitter.com/arnauddedecker/status/1698030291267309773
Want to see Russiaās fortifications in occupied Ukraine for yourself? Check out this interactive map that links each point to satellite imagery. https://read.bradyafrick.com/p/russian-field-fortifications-in-ukraine