Links for 2022-12-31
Cramming: Training a Language Model on a Single GPU in One Day — “We provide evidence that even in this constrained setting, performance closely follows scaling laws observed in large-compute settings.” https://arxiv.org/abs/2212.14034 [“Many assume AI training requires big resources and hence will be fairly centralized and hence in principle controllable. This kind of finding shows the limits of that assumption. About 2 orders of magnitude less compute produced comparable results.” — Anders Sandberg]
The progress that has been made this year is really mind-blowing. I forgot that Gato only came out in May. Feels like that happened several years ago. Check out this summary by Zhengdong Wang. Kurzweil was right. The singularity is near. https://zhengdongwang.com/2022/12/28/2022-letter.html
For the first time, researchers have demonstrated that human brain organoids implanted in mice have established functional connectivity to the animals’ cortex and responded to external sensory stimuli. https://today.ucsd.edu/story/human-brain-organoids-implanted-into-mouse-cortex-respond-to-visual-stimuli-for-first-time
A literature review of notable cognitive abilities of honeybees https://wiki.aiimpacts.org/doku.php?id=uncategorized:bugs_cognitive_capabilities
"Predictive validity in drug discovery: what it is, why it matters and how to improve it" https://www.gwern.net/docs/statistics/order/2022-scannell.pdf
New population-based study using VA's Million Veteran Program by Dr. Mark Logue and his colleagues reveals dementia risks unique to people with African ancestry -- different from those seen in people of European ancestry. https://medicalxpress.com/news/2022-12-reveals-dementia-unique-people-african.html
Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation https://tuneavideo.github.io/
Schooling enhances IQ, not intelligence https://humanvarieties.org/2022/12/22/schooling-enhances-iq-not-intelligence/
“The dual triumphs of the Greeks at Himera in Sicily and Salamis in Greece in 480 BC checked Carthage & Persia. The rest of the century saw the Greeks of Syracuse breaking Etruscan sea power while the Etruscans lost land in tandem to their neighbors.” https://threadreaderapp.com/thread/1604613969775194112.html
"One lesson is that “big war is back” and with that the need for countries to have the industrial capacity and massive weapons stocks to sustain high-intensity fighting." [Financial Times] https://archive.ph/KHuqg
A collection of various questions I have been asking https://axisofordinary.substack.com/p/i-have-questions
Thread of maps depicting key moments of the war in Ukraine:
https://threadreaderapp.com/thread/1608836642063933441.html