Links for 2024-03-19
AI:
Sam Altman talks about GPT-5 in South Korea: https://m.sedaily.com/NewsView/2D6O83AF81
There are many questions about whether there are any limits to GPT, but I can confidently say 'no'. We are confident that there are no limits to the GPT model and that if sufficient computational resources are invested, it will not be difficult to build AGI that surpasses humans. Many startups are happy assuming that GPT-5 will only make slight progress rather than significant advancements, but I think this is a big mistake. If you overlook the pace of improvement, you'll be 'steamrolled'...
Sam Altman is interviewed by Lex Fridman: https://lexfridman.com/sam-altman-2-transcript
Here are some interesting tidbits I found while scanning the transcript:
Ilya has not seen AGI. OpenAI has not built AGI.
Pertaining Q* Altman says they haven’t cracked the code yet.
Altman considers releasing even more iteratively since their goal is not to have shock updates to the world.
He doesn’t know when GPT-5 is coming out. That’s his honest answer.
They will release an amazing new model this year. They’ll release many different things in the coming months.
Altman thinks compute is going to be the currency of the future; the most precious commodity in the world, and he thinks we should be investing heavily to make a lot more compute.
He confirms that people, state actors, are trying to infiltrate OpenAI.
Altman is excited that the models are getting better across the board, not just in a few areas.
In 5, to 10 years, programming will be in a very different shape. Maybe some people will program entirely in natural language.
When will we build AGI? He expects that “by the end of this decade and possibly somewhat sooner than that, we will have quite capable systems that we look at and say, “Wow, that’s really remarkable.” If we could look at it now.”
More AI links:
Nvidia Unveils Blackwell, Its Next GPU: It is capable of delivering four times the training performance, up to 30 times the inference performance, and up to 25 times better energy efficiency, compared to its predecessor. https://spectrum.ieee.org/nvidia-blackwell
An AI-Designed Drug Is Moving Toward Approval at an Impressive Clip https://singularityhub.com/2024/03/18/an-ai-designed-drug-is-moving-toward-approval-at-an-impressive-clip/
Amazon can see $1 billion training runs on the horizon (see slide 21) https://perspectives.mvdirona.com/2024/01/cidr-2024/
“Using a cross-modal training technique enhanced by LLMs, we set a new state-of-the-art for silent speech (12.2% word error rate, open vocabulary) and brain-to-text (8.9% WER; Rank 1 on Brain-to-Text Benchmark '24)” https://arxiv.org/abs/2403.05583
Amazon presents Chronos: Learning the Language of Time Series https://arxiv.org/abs/2403.07815
“Weight-sharing is an integral part of ML models, e.g., CNNs and transformers. But instead of hard-coding it into the model, let's learn it!” https://arxiv.org/abs/2311.03096
“…a limited number of in-context samples may lead to the retrieval of an incorrect skill, thereby increasing the risk, which will eventually diminish as task learning takes effect with more in-context samples.” https://arxiv.org/abs/2402.18819
Robotics
Nvidia Announces GR00T, a Foundation Model For Humanoids https://spectrum.ieee.org/nvidia-gr00t-ros
Mercedes Hires Humanoid Robots to Work at Its Factories https://www.pcmag.com/news/mercedes-hires-humanoid-robots-work-at-its-factories
A new algorithm reduces travel time by identifying shortcuts a robot could take on the way to its destination. https://news.mit.edu/2024/researchers-help-robots-navigate-efficiently-uncertain-environments-0314
Learning Generalizable Feature Fields for Mobile Manipulation https://geff-b1.github.io/
Miscellaneous:
Thread about the practical implications of low intelligence and health issues. https://x.com/nathan_culley/status/1521927810809569281
Scientists get closer to solving chemical puzzle of the origin of life https://www.washingtonpost.com/climate-environment/2024/02/29/life-earth-origin-chemistry/ [no paywall: https://archive.is/ow12G]
Ukraine:
“During battle Mykola lost his arm, but not his resilience. After a tough recovery and training with a bionic prosthesis by @EsperBionics he’s back to his former occupation – crafting. But instead of appliances, he builds reconnaissance FPV drones – with incredible precision!” https://x.com/FedorovMykhailo/status/1769342207121588318
The European Union has prepared legislation that would see Ukraine start to receive profits generated by frozen Russian sovereign assets as early as July. https://www.bloomberg.com/news/articles/2024-03-18/eu-drafts-law-to-send-russian-central-bank-profits-to-ukraine [no paywall: https://archive.is/F35g3]
"In the near future, European companies will be able to produce the necessary amount of ammunition to replenish stockpiles of European countries, as well as to provide Ukraine with ammunition if it needs it in the near future," said Petr Pavel, President of the Czech Republic. https://x.com/NOELreports/status/1769981630263762982
Ukrainian drone strikes have significantly impacted Russia's oil-refining capacity, with about 600,000 barrels per day offline according to Gunvor Group Ltd.'s CEO Torbjörn Törnqvist. The strikes, hitting plants deep inside Russia's borders over the weekend, have led to diesel and gasoline futures climbing for multiple consecutive sessions. https://www.bloomberg.com/news/articles/2024-03-18/gunvor-says-drones-shut-down-600-000-barrels-of-russia-refining [no paywall: https://archive.is/T9ZKl]