Links for 2024-03-06
AI:
[This link was broken yesterday] Sergey Brin 3/2 (AGI House) https://www.youtube.com/watch?v=idOD-2D7nxs
When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method https://arxiv.org/abs/2402.17193
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers https://arxiv.org/abs/2402.19479
MOSAIC: A Modular System for Assistive and Interactive Cooking https://portal-cornell.github.io/MOSAIC/
Waymo got the OK to expand to highways in Los Angeles and the Bay Area, and it allows their cars to go up to 65mph. https://www.theverge.com/2024/3/2/24088454/waymo-california-highway-expansion-los-angeles-bay-area
Humanoid Locomotion as Next Token Prediction https://huggingface.co/papers/2402.19469
Learning to walk in confined spaces using 3D representation https://www.youtube.com/watch?v=QAwBoN55p9I
It's very easy to build an AI-powered suicide drone https://x.com/luiswenus/status/1763978511092478221
OpenAI vs. Musk:
OpenAI: “We are dedicated to the OpenAI mission and have pursued it every step of the way. We’re sharing some facts about our relationship with Elon, and we intend to move to dismiss all of his claims.” https://openai.com/blog/openai-elon-musk
gwern: “One of OA's most important challenges is infosec and detecting things like side-channels. So it's unfortunate that they used a totally broken redaction tool which lets you trivially figure out that Elon is forwarding emails from Hassabis & Karpathy. (The HTML isn't even valid!) / The redaction tool makes the second-worst mistake all redaction tools do by doing length-based redaction: it's a 2:1 character:em ratio. So you can see that the 2.5em redaction is 'Andrej', and the 5 and 5,9 redactions are 'Demis' and 'Demis Hassabis'. / You can also see that in Musk's emails, it's doing redaction *per word*. If you leak the exact length in characters of every word in a paragraph with this much context & known authors, you can probably use a LLM to infer the entire paragraph!” https://twitter.com/gwern/status/1765225703736213546
Claude 3:
Claude-3 learns translation on a language that has little presence on the Internet https://x.com/hahahahohohe/status/1765088860592394250
“People are reading way too much into Claude-3's uncanny "awareness". Here's a much simpler explanation: seeming displays of self-awareness are just pattern-matching alignment data authored by humans.” https://x.com/DrJimFan/status/1765076396404363435
“Claude 3 was trained on synthetic data (“data we generate internally”). Fairly clear that compute is the bottleneck given that parameter count and data can be scaled.” https://x.com/Justin_Halford_/status/1764677260555034844
“Other notable results included the model setting up the open source LM, sampling from it, and fine-tuning a smaller model on a relevant synthetic dataset the agent constructed” https://x.com/lawhsw/status/1764664887744045463
“Claude 3 Opus just reinvented this quantum algorithm from scratch in just 2 prompts.” https://x.com/GillVerd/status/1764901418664882327
Claude 3 can handle various encodings and simple encryptions out of the box. https://x.com/AdjectiveAlli/status/1764787911890014688
“Claude-3 by @AnthropicAI blows the other AIs out of the water in terms of passing an IQ test.” https://x.com/maximlott/status/1764910211901370758
“I really love how Claude 3 models are really good at d3. Asked Claude 3 Opus to draw a self-portrait.” https://x.com/karinanguyen_/status/1764789887071580657
Regarding Google
Lots of people argue Google is dead because of the recent Gemini image generation scandal. But that ignores the huge human capital they still have. Until those people leave, Google will continue to be first class.
Remember that James Damore was fired in 2017. Google was already acidly woke then. Yet many revolutionary AI results like MuZero, AlphaFold, Gato, Robotics Transformers, AlphaCode, AlphaDev, FunSearch, AlphaGeometry, Genie, etc., etc., were released after 2017.
They could indeed benefit enormously from firing a lot of woke people. But they are far from irrelevant. If you think that, the next few years will surprise you.
And anyway, Google is no longer a search engine company. Meta is no longer a social networking company... They're now all in the business of creating the first sand god.
P.S. Don't forget that Google owns by far the largest video platform which they can use to train god and that they build their own AI processors.
Miscellaneous:
Two mathematicians have shown that origami can, in principle, be used to perform any possible computation. https://www.quantamagazine.org/how-to-build-an-origami-computer-20240130/
“Rice University scientists have discovered a first-of-its-kind material, a 3D crystalline metal in which quantum correlations and the geometry of the crystal structure combine to frustrate the movement of electrons and lock them in place.” https://news.rice.edu/news/2024/rice-scientists-pull-quantum-coup
Contrary to widespread consensus among scientists, "psi" effects have not become smaller or vanished in ongoing research. https://osf.io/preprints/psyarxiv/wd26f
“…most blind mathematicians work in geometry and topology. It is argued that the spatial intuition of sighted people is degraded by the triviality of retinal perception.” https://x.com/teortaxesTex/status/1751789887194308984
The 30 most surprising things to be learned from reading Our World in Data. https://x.com/BoyanSlat/status/1746211702968906049
Ukraine:
For the first time, Russia has managed to destroy a HIMARS. https://x.com/GloOouD/status/1765060493088276769
“THREE 🇷🇺Russian 220mm BM-27 «Uragan» MLRS were destroyed by Ukrainian soldiers with M142 «HIMARS» MLRS in Zaporizhzhia region” https://x.com/GloOouD/status/1765078611370058012
“Russian losses in the Avdiivka direction. On a small stretch of land, the corpses of numerous Russian soldiers are visible alongside destroyed heavy equipment.” https://x.com/NOELreports/status/1765094338491478124
“A failed Russian mechanized attack in the direction of Novomykhailivka consisting of tanks and armored carriers, even protected by wooden logs.” https://x.com/NOELreports/status/1765120006625362303
“Footage of the effective use of FPV drones by the 3rd Assault Brigade's UAV Battalion near Avdiivka: enemy attack repelled, resulting in 15 enemy deaths. The enemy platoon was completely destroyed from a distance. Annotation provided.” https://x.com/wartranslated/status/1765120707011149938
“One of the largest plants producing raw iron ore materials in Russia in Zheleznogorsk, Kursk Oblast, was attacked by unidentified flying objects. Reportedly, the drone struck right into the fuel tank on the territory of the plant.” https://twitter.com/Tendar/status/1765281709484839388
Macron: "Who launched the war in Ukraine? Vladimir Putin. Who threatens us, whatever we do whatever we say, with nuclear weapons? President Putin. If every day we explain what our limits are in the face of someone who has none and launched this war, I can already tell you that the spirit of defeat is there lurking. Not amongst us." https://x.com/EmmanuelMacron/status/1765125007791759663
Macron: “I say this at a serious moment in our common history (...) Europe has been cut in two by cowardice, by the desire of one part of Europe not to see the difficulties of the other, to abandon its destiny to totalitarianism.” https://x.com/aidefranceukr/status/1764970297462079604
'It is unclear to what extent Russia can sustain these losses in the long term. RUSI analysts said this month that Russian aircrew losses amounted to nearly 160 personnel, which they described as “a serious loss of capability.”' https://nytimes.com/2024/03/05/world/europe/ukraine-war-russia-planes.html [no paywall: https://archive.is/UtuWq]
Large Russian channel practically admits that the Black Sea Fleet has been defeated by Ukraine. https://x.com/wartranslated/status/1764938008170574244