Links for 2024-04-16
AI:
Meta announces Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length https://arxiv.org/abs/2404.08801
Google presents TransformerFAM: A novel Transformer architecture that leverages a feedback loop to enable the network to attend to its own latent representations. This design fosters the emergence of working memory within the Transformer, allowing it to process indefinitely long sequences. https://arxiv.org/abs/2404.09173
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models https://arxiv.org/abs/2404.07839
AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees https://arxiv.org/abs/2404.08417
GPT-4 rivals doctors in many medical exams - and beats them in psychiatry https://www.stevestewartwilliams.com/p/a-new-milestone-for-chatgpt
Claude 3 Opus can operate as a Turing machine https://x.com/ctjlewis/status/1779740038852690393
MathGPT: Leveraging Llama 2 to create a platform for highly personalized learning https://ai.meta.com/blog/llama-2-mathgpt-mathpresso-qanda-upstage-open-source-llm/
A list of resources for AI in Mathematics https://docs.google.com/document/d/1kD7H4E28656ua8jOGZ934nbH2HcBLyxcRgFDduH5iQ0/
AI Consciousness is Inevitable: A Theoretical Computer Science Perspective https://arxiv.org/abs/2403.17101
Compute:
AI Chip Trims Energy Budget Back by 99+ Percent — Photonic tech uses both diffraction and interference to boost performance https://spectrum.ieee.org/optical-neural-network
"Our computer hardware exchange data using a standard called PCI Express. Your disk, your network and your GPU are limited by what PCI Express can do. Currently, it means that you are limited to a few gigabytes per second of bandwidth. PCI Express is about to receive a big performance boost in 2025 when PCI Express 7 comes out: it will support bandwidth of up to 512 GB/s. That is really, really fast. It does not follow that your disks and graphics are going to improve very soon, but it provides the foundation for future breakthroughs." https://www.extremetech.com/computing/blazing-fast-pci-express-70-gets-updated-to-version-05-is-launching-in
Miscellaneous:
RNA that doesn’t age. Neuroscientists at FAU discover building blocks in nerve cells that last a life time. https://www.fau.eu/2024/04/05/news/research/rna-that-doesnt-age/
“Orchid's whole genome embryo reports read 100x more data than traditional tests. But what does that extra coverage get you? Watch this video to learn more!” https://x.com/OrchidInc/status/1779872924041695690
The US aims to ‘crack the code’ on scaling up geothermal energy production https://www.theguardian.com/environment/2024/apr/02/geothermal-energy-electricity
“The Bureau of Land Management is giving geothermal energy exploration a categorical exclusion from environmental review under NEPA.” https://x.com/AlecStapp/status/1780048314768969885
Israel:
Russia wants to transfer air defense systems and Su-35 fighter jets to Iran to bolster Tehran's defenses against airstrikes by Israel or the United States. Russia has also pledged to provide technical assistance with Iranian spy satellites, as well as help building rockets to put more satellites into space. https://www.washingtonpost.com/national-security/2024/04/15/iran-israel-russia-drones-missiles/ [no paywall: https://archive.is/b61vX]
Ukraine:
“Majority Speaker of the House, Mike Johnson, announced that he will introduce 3 separate bills for funding Ukraine, Israel and Taiwan to the Floor.” https://x.com/NOELreports/status/1780120730740342961
Macron and others keep talking about how Ukraine must win, even hinting at the possibility of sending Western troops. But in practice, they won't even fund the available artillery shells that are desperately needed to hold the line. What the hell is going on? https://x.com/deaidua/status/1779957799536992434
On Russia/Ukraine peace talks in early 2022: “…the Russians attempted to subvert this crucial article by insisting that such action would occur only “on the basis of a decision agreed to by all guarantor states”—giving the likely invader, Russia, a veto…the Russians attempted to subvert this crucial article by insisting that such action would occur only “on the basis of a decision agreed to by all guarantor states”—giving the likely invader, Russia, a veto.” https://www.foreignaffairs.com/ukraine/talks-could-have-ended-war-ukraine [no paywall: https://archive.is/ftEhB]
“Ukraine’s new national security chief spent five formative years in Moscow as a cryptology cadet at the elite KGB Academy” https://www.economist.com/europe/2024/04/15/as-russias-attacks-step-up-ukraine-fears-waning-western-support [no paywall: https://archive.is/aot9T]
“Overall, unfortunately, the situation with Russian air domination is only getting worse.” https://x.com/wartranslated/status/1780146233165722062
This is what Chasiv Yar looks like. https://x.com/AndrewPerpetua/status/1779962641412239517