Links for 2023-03-26
A mind-blowing paper from last year:
We also evaluate on other tasks including parity, multiplication and subtraction, and find similar results. On the longest subset of questions, we achieve an error reduction of approximately 10x, 5x and 2x respectively compared to the best available baselines.
To see if the model can learn multiple skills simultaneously, we teach addition and subtraction within the same prompt. We find that both algos can be learned well, and that learning them together is more efficient than learning them separately (i.e. need fewer examples)
To see if the model can build on top of simpler skills to learn more complex skills, we evaluate performance on multi-number addition and multiplication as addition. We show that the model can leverage prompt examples from simpler skills to learn a more complex skill.
It's just amazing how much progress can be made on top of existing models. See e.g.:
Reflexion: an autonomous agent with dynamic memory and self-reflection https://arxiv.org/abs/2303.11366
Toolformer: Language Models Can Teach Themselves to Use Tools https://arxiv.org/abs/2302.04761
“Intelligence is not something that happened at the tail end of evolution, but was discovered towards the beginning, long before brains came on the scene.” https://aeon.co/essays/how-evolution-hacked-its-way-to-intelligence-from-the-bottom-up
Visual language maps for robot navigation: VLMaps takes an initial step towards grounding pre-trained visual-language information onto spatial map representations that can be used by robots for navigation. https://ai.googleblog.com/2023/03/visual-language-maps-for-robot.html
Towards Learnable Game Engines: “In this work, we present a framework to train game-engine-like neural models, solely from monocular annotated videos.” https://learnable-game-engines.github.io/lge-website/
Capabilities of GPT-4 on Medical Challenge Problems: GPT-4, without any specialized prompt crafting, exceeds the passing score on USMLE by over 20 points and outperforms domain-expert models like Med-PaLM. https://arxiv.org/abs/2303.13375
“Highly-compressed insights about LLMs. Includes exercises. Remark 3 and Remark 15 are the most important and entirely self-contained.” https://www.lesswrong.com/posts/7qSHKYRnqyrumEfbt/remarks-1-18-on-gpt-compressed
NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation — “Experiments show that our model not only generates high-quality long videos with both global and local coherence, but also decreases the average inference time from 7.55min to 26s (by 94.26\%) at the same hardware setting when generating 1024 frames.” https://msra-nuwa.azurewebsites.net/
“We propose MM-REACT, a system paradigm that integrates ChatGPT with a pool of vision experts to achieve multimodal reasoning and action.” https://multimodal-react.github.io/
Multi-language document Q&A with LangChain and GPT-3.5-turbo https://jonathansoma.com/words/multi-language-qa-gpt.html
Meta’s New ChatGPT-Like AI Is Fluent in the Language of Proteins—and Has Already Modeled 700 Million of Them https://singularityhub.com/2023/03/21/metas-new-ai-is-digging-into-the-most-mysterious-proteins-on-earth/
An explainer on the China AI chip export controls: Choking off China’s Access to the Future of AI https://www.csis.org/analysis/choking-chinas-access-future-ai
An Appeal to AI Superintelligence: Reasons to Preserve Humanity https://www.lesswrong.com/posts/azRwPDbZfpadoL7WW/an-appeal-to-ai-superintelligence-reasons-to-preserve
Group differences are just individual differences, really https://emilkirkegaard.dk/en/2023/03/group-differences-are-just-individual-differences-really/
The word mamihlapinatapai is derived from the Yaghan language...is considered one of the hardest words to translate. It has been translated as "a look that without words is shared by two people who want to initiate something, but that neither will start" or "looking at each other hoping that the other will offer to do something which both parties desire but are unwilling to do". https://en.m.wikipedia.org/wiki/Mamihlapinatapai
If someone writes that the latest AI models are not "agents, intelligent, creative, conscious" and cannot "reason" and "understand" what they are doing, you can be pretty sure that the author will not define what any of these terms mean or how to test for them.
And in the rare exceptions, the authors will assume that these concepts are binary rather than on a spectrum. Unless something has a certain characteristic, such as Pearl's three levels of causal hierarchy or microtubules, it can have no understanding or consciousness.
I don't rule out the possibility that some of these concepts may undergo sudden phase changes, leading to the emergence of qualitatively different expressions of these notions. But even if this is the case, the fact that human brains are products of evolution suggests that these new states can be reached gradually. That is, there must be a fitness payoff the closer we get to that phase change. But then it makes little sense to define these concepts dualistically, in terms of whether something has reached a phase change. Because what we need to know is whether the overall fitness of a model has increased. We need to know how far from the boiling point a model might be.
Does this mean we should call a thermostat an agent, and a calculator intelligent? Yes, indeed.
Some AI predictions from the Mateculus community
Date Weakly General AI is Publicly Known
March 2022: 2044
March 2023: 2026
Date of Artificial General Intelligence
March 2022: 2058
March 2023: 2032