Links for 2024-04-27
AI “safety”:
“This is the kind of flavor a technical program to solve ASI in a mature, sane civilization might have.” — Connor Leahy https://vimeo.com/932838403/7e7bfe96c4 (More here: https://www.aria.org.uk/programme-safeguarded-ai/)
“Do models need to reason in words to benefit from chain-of-thought tokens? In our experiments, the answer is no! Models can perform on par with CoT using repeated '...' filler tokens. This raises alignment concerns: Using filler, LMs can do hidden reasoning not visible in CoT” https://arxiv.org/abs/2404.15758
AI:
Claims about Chinese LLMs: Chinese firm SenseTime launched a new LLM. It can beat GPT-4 Turbo across nearly all key benchmarks. The ~600 billion parameter model features a 200k context window and was trained on over 10TB of largely synthetic data. https://x.com/rowancheung/status/1783531708307882146 (Careful: “Their MMLU of GPT 4 turbo is not accurate. Their development pace is on par with open source eco system.” https://x.com/technofrontiers/status/1783547864649465922)
“Distributed training seems close enough to being a solved problem that a project costing north of a billion dollars might get it working on schedule. It's easier to stay within a single datacenter, and so far it wasn't necessary to do more than that, so distributed training not being routinely used yet is hardly evidence that it's very hard to implement.” https://www.lesswrong.com/posts/xyL5kb8RBGLiupGLf/scaling-of-ai-training-runs-will-slow-down-after-gpt-5?commentId=bzgA4n68D97ek7ipq
Eric Schmidt-backed Augment, a GitHub Copilot rival, launches out of stealth with $252M https://techcrunch.com/2024/04/24/eric-schmidt-backed-augment-a-github-copilot-rival-launches-out-of-stealth-with-252m/
Apple presents OpenELM: More accurate LLMs with fewer parameters and fewer pre-training tokens. https://arxiv.org/abs/2404.14619
Snowflake Arctic: An open-source LLM (Apache 2.0 license.) that uses a unique Dense-MoE Hybrid transformer architecture. It claims to use 17x less compute budget than Llama 3 70B. The training compute is roughly under $2 million (less than 3K GPU weeks). https://www.snowflake.com/blog/arctic-open-efficient-foundation-language-models-snowflake/
Graph Machine Learning in the Era of Large Language Models (LLMs) https://arxiv.org/abs/2404.14928
Neuroscience:
In the brain, bursts of beta rhythms implement cognitive control https://picower.mit.edu/news/brain-bursts-beta-rhythms-implement-cognitive-control
Organ Neuroprosthetics: Connecting Transplanted and Artificial Organs with the Nervous System https://onlinelibrary.wiley.com/doi/10.1002/adhm.202302896
Miscellaneous:
A Win–Win Approach: Maximizing Wi-Fi Performance Using Game Theory https://www.shibaura-it.ac.jp/en/headline/detail/20240416_7070_001.html
Nandan Nilekani’s vision for the “Finternet” is to enable individuals to transfer any financial asset, in any amount, at any time, to anyone, anywhere in the world—cheaply, securely, and near-instantaneously. https://drive.google.com/file/d/1cNYxT6yWvWurrXCEeXlWPpJ9AXFv1QaS/view
The "Cyclone-Rotor" UFO Drone: Parts and print files to build an extremely efficient hovering platform. https://hackaday.io/project/186410-the-cyclone-rotor-ufo-drone
Politics:
“Fertility intentions suggest China's fertility may have a lot further to fall. Given China's TFR was just 1.02 in 2023, this seems surprising, but South Korea shows there is no natural floor to fertility.” https://x.com/MoreBirths/status/1784045382340653512
“Germans with no migrant background will definitely fall below 70% amongst all ages in 2024 and amongst children, they'll fall below 55% by 2027, if not before.” https://x.com/Shifty_Coffee/status/1783138836341506253
“Does diversity make teams work better? Apparently not! A new, comprehensive preregistered meta-analysis found that, whether the diversity was demographic, cognitive, or occupational, its relationship with performance was near-zero.” https://x.com/cremieuxrecueil/status/1782856671091380497
Ukraine:
“Overnight, Ukrainian forces successfully struck Russia's Kushchyovskaya airbase in Krasnodar Krai. At least one drone hit the vicinity of the flight line. Russian aircraft have reportedly been damaged, and footage shows a number of destroyed glide bomb kits.” https://x.com/Osinttechnical/status/1784120241179513327
“Several oil refineries were attacked this night in the Krasnodar Territory, Russian media report. The Slovyansk-ECO refinery, the Bitumen plant, the Ilya refinery and a number of other objects were hit. Among them a military airfield in Kushchevsk.” https://x.com/NOELreports/status/1784064764605305036
“Russian channels are dissatisfied with the official coverage of successful attacks on the occupiers’ energy facilities. They're asking, what are these “fragments” that are taking out our refineries one by one?” https://x.com/wartranslated/status/1784185358575337478
“Footage of multiple Ukrainian drones from the GUR's Gusy-9 unit attacking a modern Russian 48Ya6-K1 “Podlet K1” radar installation, seen here successfully hitting the radar antenna and the installation's generator.” https://x.com/Osinttechnical/status/1784105888099258767
“This video depicts why I believe dive bomber drones are superior to FPV Drones.” https://x.com/AndrewPerpetua/status/1783951519454212202
“The situation on the Avdiivka axis continues to deteriorate further for us.” https://x.com/Teoyaomiquu/status/1783903914552709314
“…if Ukraine had stuck to the defensive game plan from day 1, they would be in a much better position today. So now they have to drag themselves out of the mess they created. Which is really unfortunate.” https://x.com/AndrewPerpetua/status/1784068547171246320
“Without this manpower advantage, Russia's artillery and airpower advantage would not be sufficient for Russia to make gains on the battlefield. The relative manpower situation is likely the most important factor that will determine the war's trajectory, particularly if Russia can sustain recruiting 20-30k a month.” https://x.com/RALee85/status/1783972910333407356
After talks with Xi Jinping, Blinken commented on China's support for the Russian Federation: "If Beijing does not take measures to solve this problem, Washington will." https://x.com/NOELreports/status/1783873765295759736
“You talk to any soldier in Ukraine: you don't go anywhere without a drone flying, either in support of you or against you" - At Army’s special-ops school, the biggest changes in a generation. https://www.defenseone.com/policy/2024/04/armys-special-ops-school-biggest-changes-generation/396100/
“Ukrainian forces have started using Yak-52 propeller planes to hunt down Russian drones and obviously they caught their first prey. The Russian drone is emergency-parachuting to the ground. According to the source it happened somewhere near an Odesa airfield.” https://x.com/Tendar/status/1784197479979774311
Belgium has decided to transfer F-16 fighter jets to Ukraine already in 2024. Previously, deliveries were planned for 2025. The planes are delivered as soon as possible while the exact amount is still undisclosed. https://brf.be/national/1824951/