Links for 2022-12-18
Must-read thread: “As winter approaches, here's a story about why hardware is hard. ❄️🥶 About a year ago, we started getting reports from the field about undesirable behavior when our robots were turned on. They would behave unpredictably.” https://threadreaderapp.com/thread/1604018884662951938.html
Self-Play and Self-Describe: “When deploying the trained policy to a new task or a new environment, we first let the policy self-play with randomly generated instructions to record the demonstrations. While the execution could be wrong, we can use the pre-trained foundation models to accurately self-describe (i.e., re-label or classify) the demonstrations. This automatically provides new pairs of demonstration-instruction data for policy fine-tuning. We evaluate our method on a broad range of experiments with the focus on generalization on unseen objects, unseen tasks, unseen environments, and sim-to-real transfer. We show SPLAYD improves baselines by a large margin in all cases.” https://geyuying.github.io/SPLAYD/
“CRFM announces PubMedGPT, a new 2.7B language model that achieves a new SOTA on the US medical licensing exam. The recipe is simple: a standard Transformer trained from scratch on PubMed...then fine-tuned for the QA task.” https://threadreaderapp.com/thread/1603469265583353856.html
China bans AI-generated media without watermarks: China regulates generative AI tech with rules that aim to spur growth and ban deception https://arstechnica.com/information-technology/2022/12/china-bans-ai-generated-media-without-watermarks/
How does GPT Obtain its Ability? Tracing Emergent Abilities of Language Models to their Sources https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1
University of Georgia (UGA) scientists have created an ear recognition system that can authenticate individuals with up to 97.25% accuracy. https://news.uga.edu/new-facial-recognition-technology-scans-your-ear/
“Current Language Models predict tokens with a softmax which makes it difficult to predict rare patterns. We introduce NPM: the first nonparametric masked LM that replaces this softmax with a nonparametric distribution over a text corpus.” https://arxiv.org/abs/2212.01349
Introducing Notion AI: “Leverage the limitless power of AI in any Notion page. Write faster, think bigger, and augment your creativity. Like magic!” https://www.notion.so/product/ai
Say Hello to the Toughest Material on Earth https://newscenter.lbl.gov/2022/12/08/say-hello-to-the-toughest-material-on-earth/
Breakthroughs of the year according to The Atlantic https://archive.vn/N6xPr
"The terrible things that were perpetrated under communism should be part of everyone's core knowledge just like the Holocaust." [The Spectator] https://archive.vn/XtPFw
Is dark matter’s “nightmare scenario” true? https://bigthink.com/starts-with-a-bang/dark-matter-nightmare-scenario/
Left-wing authoritarians share key psychological traits with far right https://phys.org/news/2021-09-left-wing-authoritarians-key-psychological-traits.html
The dress can be interpreted in two ways: black and blue under a yellow-tinted illumination (left figure) or gold and white under a blue-tinted illumination (right figure).
Image credit: Figure design by Kasuga~jawiki; vectorization by Editor at Large; "The dress" modification by Jahobr. Creative Commons license
Source: https://commons.wikimedia.org/wiki/File:Wikipe-tan_wearing_The_Dress_reduced.svg