Links for 2023-08-05
Learning to Model the World with Language
“Our key idea is that language helps agents predict the future: what will be observed, how the world will behave, and which situations will be rewarded. This perspective unifies language understanding with future prediction as a powerful self-supervised learning objective.”
“We propose Dynalang, an agent that grounds language to visual experience via future prediction.”
“We demonstrate that Dynalang learns to understand diverse kinds of language to solve a broad range of tasks, often outperforming state-of-the-art RL algorithms and task-specific architectures.”
“We show that the Dynalang formulation enables additional capabilities: language generation can be unified in the same model, as well as text-only pretraining without actions or task rewards.”
Project page: https://dynalang.github.io/
4 Charts That Show Why AI Progress Is Unlikely to Slow Down https://time.com/6300942/ai-progress-charts/ [https://archive.is/6DPo5]
Multimodal medical AI https://ai.googleblog.com/2023/08/multimodal-medical-ai.html
“Back in April, I set out to write an explainer on how large language models work. First I had to learn how they worked, and that was harder than I expected. This article is the culmination of 2+ months of in-depth research. I hope people find it useful.” https://www.understandingai.org/p/large-language-models-explained-with
PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedback https://arxiv.org/abs/2307.14936
Breakthrough in Monte Carlo computer simulations: Researchers develop new algorithm to effectively investigate long-range interacting systems https://www.uni-leipzig.de/en/newsdetail/artikel/breakthrough-in-monte-carlo-computer-simulations-2023-07-26
Apache Spark now has an English SDK. It goes a step beyond tools like CoPilot, allowing you to use English directly when writing code. https://www.databricks.com/blog/introducing-english-new-programming-language-apache-spark
“Cutting off welfare after a certain number of children to discourage people from having more kids than they can afford is popular. In 2017 the UK implemented the "2 child limit". It hasn't had much of an effect on birth rates” https://twitter.com/sentientist/status/1685052914308689922
Flipping a Switch and Making Cancers Self-Destruct https://www.nytimes.com/2023/07/26/health/cancer-self-destruct.html [https://archive.is/0fiT5]
Students from China's rice-growing regions: 1. contribute more to public goods & 2. heavily punish free-riders https://academic.oup.com/ej/article-abstract/133/654/2504/7120030
Will we end up with an 'evolutionary future' that nobody necessarily wants, but everyone is forced to accept due to competitive pressures — or something we specifically choose? https://hearthisidea.com/episodes/assadi
Scientific productivity by country: The relatively subpar performance by USA and China https://www.emilkirkegaard.com/p/scientific-productivity-by-country
Ukraine links:
Friday night, Ukraine attacked several Russian ports with drones and took out a Russian warship. Tonight, a Russian oil tanker, a prolific sanctions violator and a vital lifeline for the Russian war effort in Syria, was attacked. The engine room is flooded and the ship cannot move. This could be the beginning of an attack on Russia's oil trade in general.
Onboard footage of the Ukrainian USV charging at the Russian Ropucha landing ship and clearly hitting midship. https://twitter.com/Tendar/status/1687360590535348224
Onboard footage of the Ukrainian drone hitting the Russian oil tanker https://twitter.com/JimmySecUK/status/1687719113043439616
“This Russian Ropucha Class landing ship looks to not be having a great time after this mornings Ukrainian USV attack. Pretty significant listing to the port side.” https://twitter.com/OAlexanderDK/status/1687348634256384000
“A busy night in the Black Sea as Ukrainian drones hit a Russian fuel depot in Feodosia in Russian-occupied Crimea (video) and unmanned surface vehicles targeted Novorossiysk, a Russian Black Sea naval base. Both targets are hundreds of kilometers from Ukrainian-controlled areas.” https://twitter.com/StateOfUkraine/status/1687300170541613056
Ukraine strikes Russian naval vessel in first attack on oil exports https://www.ft.com/content/fbd3c163-6e7b-40ab-a693-dba500c82d9e [https://archive.is/959Bw]
More information about the oil tank that was hit https://twitter.com/Osinttechnical/status/1687720467400916992
Photos from inside the Russian oil tanker https://twitter.com/Osinttechnical/status/1687624844291006466
More:
“Ukrainian FPV loitering munitions continue to swarm Staromlynivka, Donetsk Oblast, seen here hitting a Russian Z-STS Akhmat MRAP at least twice and a BMD-2 AIFV once.” https://twitter.com/Osinttechnical/status/1687328807995236352
Ukraine's homegrown cruise missile Trembita ready to challenge Russian defenses https://www.armyrecognition.com/ukraine_-_russia_conflict_war_2022/ukraine_s_homegrown_cruise_missile_trembita_ready_to_challenge_russian_defenses.html