Links for 2023-04-03
DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents — “It provides a simple, interpretable forum for models to communicate feedback and iteratively improve output. We frame our dialog as a discussion between two agent types - a Researcher, who processes information and identifies crucial problem components, and a Decider, who has the autonomy to integrate the Researcher's information and makes judgments on the final output…shows significant improvement over the base GPT-4 performance in both human expert preference evaluations and quantitative metrics.” https://arxiv.org/abs/2303.17071
GPT 4 Can Improve Itself - (ft. Reflexion, HuggingGPT, Bard Upgrade and much more) https://youtu.be/5SgJKZLBrmg /
Auto-GPT can recursively debug code. Andrej Karpathy: “Next frontier of prompt engineering imo: "AutoGPTs" . 1 GPT call is just like 1 instruction on a computer. They can be strung together into programs. Use prompt to define I/O device and tool specs, define the cognitive loop, page data in and out of context window, .run().” https://twitter.com/karpathy/status/1642598890573819905
“…we find interesting evidence that simple sequence prediction can lead to the formation of a world model…Our experiment provides evidence supporting that these language models are developing world models and relying on the world model to generate sequences.” https://thegradient.pub/othello/
“Actually, Othello-GPT Has A Linear Emergent World Representation ... "predict the next token" transformer models are capable of learning a model of the world.” https://www.lesswrong.com/posts/nmxzr2zsjNtjaHh7x/actually-othello-gpt-has-a-linear-emergent-world
Modern language models refute Chomsky’s approach to language https://lingbuzz.net/lingbuzz/007180
Prediction market: Conditional on an okay outcome with AGI, how did that happen? https://www.lesswrong.com/posts/uNepkB5EqETC8b9C2/manifold-if-okay-agi-why
The most comprehensive database on DARPA neurotech projects, program managers, and outcomes (probably). Plus, a case study. https://cell.substack.com/p/darpa-neurotech
Size matters: Bigger brains equal more complex hand movements in primates. Big brains come at a big cost, however. https://bigthink.com/life/primate-brain-size/
Modern neuroscience confirms race differences in brain size and functioning https://kirkegaard.substack.com/p/modern-neuroscience-confirms-race (James Thompson: Racial differences are brain deep https://thompsonj.substack.com/p/not-unreasonable)
Meta-analysis of quasi-experimental studies on how child abuse impacts mental health finds that about 45% of the correlation normally observed is non-causal. The remaining effect is not large and is similar for physical, sexual, and emotional abuse. (via @Sean__Last) https://gwern.net/doc/psychiatry/2023-baldwin.pdf
60 Minutes explores advancements in artificial prosthetics technology that can now restore a sense of touch. https://www.cbsnews.com/news/advancements-in-prosthetics-limb-technology-allow-feeling-control-60-minutes-transcript-2023-03-26/
Thread about Richard III: “And whether Richard III was a true heir or fraud, the big picture's the same: descended from Anatolians who became Romans, who in turn entered the barbarian elite and climbed to its top, 1000 miles away, in 1000 years.” https://twitter.com/paul_hundred/status/1639824849899069441