Links for 2023-04-23
“...the core building block of Transformers, the attention operator, exhibits quadratic cost in sequence length, limiting the amount of context accessible...we propose Hyena...reaching Transformer quality with a 20% reduction in training compute required at sequence length 2K. Hyena operators are twice as fast as highly optimized attention at sequence length 8K, and 100x faster at sequence length 64K.” https://arxiv.org/abs/2302.10866
The A.I. Dilemma (recorded before the release of GPT-4): "Tristan Harris and Aza Raskin discuss how existing A.I. capabilities already pose catastrophic risks to a functional society, how A.I. companies are caught in a race to deploy as quickly as possible without adequate safety measures, and what it would mean to upgrade our institutions to a post-A.I. world." https://www.youtube.com/watch?v=xoVJKj8lcNQ
“The basic reasons I expect AGI ruin” https://www.lesswrong.com/posts/eaDCgdkbsfGqpWazi/the-basic-reasons-i-expect-agi-ruin
AI Poised to Transform Video Compression Landscape — Apple’s WaveOne purchase heralds new era in smart-streaming of AR and video https://spectrum.ieee.org/ai-video-codecs-waveone
AI is already taking video game illustrators’ jobs in China https://restofworld.org/2023/ai-image-china-video-game-layoffs/
InstructRL for Human-AI Coordination: "...enables humans to specify what kind of strategies they expect from their AI partners through natural language instructions. We use pretrained large language models to generate a prior policy conditioned on the human instruction and use the prior to regularize the RL objective. This leads to the RL agent converging to equilibria that are aligned with human preferences." https://arxiv.org/abs/2304.07297
“…gisting enables up to 26x compression of prompts, resulting in up to 40% FLOPs reductions, 4.2% wall time speedups, storage savings, and minimal loss in output quality.” https://arxiv.org/abs/2304.08467
ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation https://arxiv.org/abs/2304.05977
Emergence of belief-like representations through reinforcement learning https://www.biorxiv.org/content/10.1101/2023.04.04.535512v1
StabilityAI released their own LLM, called StableLM: "The Alpha version of the model is available in 3 billion and 7 billion parameters, with 15 billion to 65 billion parameter models to follow." https://stability.ai/blog/stability-ai-launches-the-first-of-its-stablelm-suite-of-language-models
How to train your own Large Language Models https://blog.replit.com/llm-training
"Five years later, with double the demographic data, naked mole-rat mortality rates continue to defy Gompertzian laws by not increasing with age", Ruby et al 2023 https://www.biorxiv.org/content/10.1101/2023.03.27.534424v1.full
Programmed versus non-programmed evolution of aging. What is the evidence? Pamplona et al. (2023) argue that animal aging is the result of a genetic program. https://www.sciencedirect.com/science/article/pii/S0531556523000839
“Remarkable” Brain Boosting Peptide: MIT Neuroscientists Discover Way To Reverse Alzheimer’s Disease https://news.mit.edu/2023/new-peptide-may-hold-potential-alzheimers-treatment-0413
Hours of daylight as a function of day of the year and latitude: https://www.reddit.com/r/dataisbeautiful/comments/duax05/oc_hours_of_daylight_as_a_function_of_day_of_the/
Mars CO2 clouds captured about an hour before sunrise: https://www.flickr.com/photos/semeion/52788862763/
The NASA Perseverance Mars Rover captured these high altitude clouds with its left navigation camera on Sol 738 before Sunrise. These high altitude clouds, likely made up of CO2, were already lit by the sun.