Links for 2024-04-19
AI:
Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing — “AlphaLLM for the self-improvements of LLMs, which integrates Monte Carlo Tree Search (MCTS) with LLMs to establish a self-improving loop, thereby enhancing the capabilities of LLMs without additional annotations.” https://arxiv.org/abs/2404.12253
Can Language Models Solve Olympiad Programming? Uses self-reflection and retrieval over episodic knowledge to boost the performance of GPT-4 on a USA Computing Olympiad benchmark from 8.7% pass@1 to 20.2%. https://arxiv.org/abs/2404.10952
Fewer Truncations Improve Language Modeling — “…achieves superior performance (e.g., relatively +4.7% on reading comprehension; +16.8% in context following; and +9.2% on program synthesis), and reduces closed-domain hallucination effectively by up to 58.3%.” https://arxiv.org/abs/2404.10830
VASA, a framework for generating lifelike talking faces of virtual charactors with appealing visual affective skills, given a single static image and a speech audio clip. https://www.microsoft.com/en-us/research/project/vasa-1/
“Microsoft on Tuesday said it would make a $1.5 billion investment in G42, an artificial intelligence giant in the United Arab Emirates, in a deal largely orchestrated by the Biden administration to box out China” https://www.nytimes.com/2024/04/16/technology/microsoft-g42-uae-ai.html [no paywall: https://archive.is/tPfN0]
Race for AI Supremacy in Middle East Is Measured in Data Centers https://www.bloomberg.com/news/articles/2024-04-11/race-for-ai-supremacy-in-middle-east-is-measured-in-data-centers [no paywall: https://archive.is/xDjgu]
Gemini API Cookbook: “This is a collection of guides and examples for the Gemini API, including quickstart tutorials for writing prompts and using different features of the API, and examples of things you can build.” https://github.com/google-gemini/cookbook
Meta Llama 3:
Meta is currently training a 400B+ parameter model which is already on par with Claude 3 Opus and it will be fully open source https://ai.meta.com/blog/meta-llama-3/
Scaling laws and Meta’s new 8B model: “Very notably, 15T is a very very large dataset to train with for a model as "small" as 8B parameters, and this is not normally done and is new and very welcome. The Chinchilla "compute optimal" point for an 8B model would be train it for ~200B tokens. (if you were only interested to get the most "bang-for-the-buck" w.r.t. model performance at that size). So this is training ~75X beyond that point, which is unusual but personally, I think extremely welcome. Because we all get a very capable model that is very small, easy to work with and inference. Meta mentions that even at this point, the model doesn't seem to be "converging" in a standard sense. In other words, the LLMs we work with all the time are significantly undertrained by a factor of maybe 100-1000X or more, nowhere near their point of convergence.” https://x.com/karpathy/status/1781028605709234613
“LLaMA-3 is a prime example of why training a good LLM is almost entirely about data quality…” https://x.com/cwolferesearch/status/1781009242989703216
Mark Zuckerberg on: Llama 3, open sourcing towards AGI, custom silicon, synthetic data, & energy constraints on scaling, Caesar Augustus, intelligence explosion, bioweapons, $10b models, & much more https://www.dwarkeshpatel.com/p/mark-zuckerberg
Politics:
Allegedly, Israel took military action against Iran tonight. Iran claims that the attack was launched from inside Iran using small drones. A senior Iranian official reportedly told Reuters that Iran has no plans to immediately respond to the Israeli strike, which was described differently in Iranian state media. https://www.foxnews.com/world/israel-strikes-site-iran-retaliation-weekend-assault (See also: 'A regional intelligence source with knowledge of Iran's potential reaction to Friday's strike, which was carried out by Israel according to a US official, said that direct state-to-state strikes between the two enemies were "over."' https://edition.cnn.com/middleeast/live-news/israel-iran-gaza-conflict-news-04-19-24/h_b3be04c2f747ec45fc2a338bbc768cf6)
Immigrant selection and crime in Britain: Do less-selected groups have higher incarceration rates? https://www.aporiamagazine.com/p/immigrant-selection-and-crime-in
Ukraine:
“A Russian Tu-22M3 downed in the Stavropol Krai, Russia. According to Russian information due to 'technical failures'. The Ukrainian GUR claim responsibility for the downing.” https://x.com/RALee85/status/1781182577925124138
Losses for yesterday include two Ukrainian MIG-29 https://x.com/AndrewPerpetua/status/1781224212750819676
“Colonel Кропотов Павел Александрович (Kropotov Pavel Alexandrovich), commander of the 59th Guards Communications Brigade, was eliminated in Ukraineon 13 April ’24 in a Storm Shadow strike on the military headquarters in Luhansk.” https://x.com/KilledInUkraine/status/1781258948869300465
There is no doubt that using Russia's $340 billion in frozen assets for Ukraine's military defense could be the difference between defeat and victory. It's now been 26 months and the assets are still not seized however... https://www.standard.co.uk/news/uk/russia-ukraine-war-frozen-assets-london-bill-browder-b1152050.html
Prioritising technology and investing hundreds of millions of dollars into long-range drones, Ukraine has now developed new drone models with a range of 3000 kilometers, able to reach even Siberia. https://www.economist.com/europe/2024/04/18/ukraine-is-ignoring-us-warnings-to-end-drone-operations-inside-russia [no paywall: https://archive.is/sekqN]
Ukraine can possibly receive six more Patriot systems from partners: "We have heard about seven additional systems. One of them is ours. We do hope to find six more in the context of NATO, and I have again used this opportunity in many negotiations to convey this opinion," he emphasized. https://audiovisual.ec.europa.eu/en/video/I-256220?lg=EN
“Video of Russian Su-25 attack aircraft operating close to Chasiv Yar. A clear sign of a lack of Ukrainian air defense ammunition.” https://x.com/RALee85/status/1781071897419014195
CIA Director Burns sounds a more dire warning about Ukraine if they don't get more aid: “There is a very real risk that the Ukrainians could lose on the battlefield by the end of 2024, or at least put Putin in a position where he could dictate the terms.." https://edition.cnn.com/2024/04/18/politics/cia-director-ukraine-russia-warning/index.html
“Mr Macron, says a French military source, no longer harbours any doubts about Moscow’s expansionist ambitions.” https://www.economist.com/europe/2024/04/18/how-russia-targeted-france-and-radicalised-emmanuel-macron [no paywall: https://archive.is/3bydH]