Links for 2023-01-10
“RETRO models are a giant capability unlock for LLM tech, and they're shockingly under the radar. The first ones should come out this year. They might be even more significant than GPT-4.” https://threadreaderapp.com/thread/1612224161916780544.html
The Taipei 101 stabilizing ball during the 6.8 earthquake in Taiwan (September 18, 2022) https://www.youtube.com/watch?v=k51nFin7Wzc (for more see: https://en.wikipedia.org/wiki/Tuned_mass_damper)
A nice empirical demonstration of Galileo's principle of relativity https://www.youtube.com/watch?v=GIMNS0YiHzY [Terence Tao: “If one replaces the person on the trampoline with a light beam bouncing up and down between two horizontal mirrors, and assumes that the speed of light is c in every inertial frame, one obtains (from Pythagoras) the well-known 'light clock" thought experiment of Einstein that derives the fact that such clocks slow down by a factor of √(1−𝑣²/𝑐²) when moving at a velocity v.”
Couldn't Care Less. Cormac McCarthy in conversation with David Krakauer — “The jagged mountains were pure blue in the dawn and everywhere birds twittered and the sun when it rose caught the moon in the west so that they lay opposed to each other across the earth, the sun white hot and the moon a pale replica, as if they were the ends of a common bore beyond whose terminals burned worlds past all reckoning” https://www.youtube.com/watch?v=HrUy1Vn2KdI
A wide-ranging discussion with Holden Karnofsky, the co-CEO of Open Philanthropy. Does transformative AI make this the most important century? Does he regret OpenPhil’s $30m grant to OpenAI? Digital people, progress, forecasting, and ethics. https://www.youtube.com/watch?v=UckqpcOu5SY
The French Revolutionaries invented a new Calendar: "By adopting calendrical rhythms alien to the rest of the world. The French created artificial barriers to communication, understanding, and ultimately trade." https://threadreaderapp.com/thread/1609202630760488962.html
"the Royal Astronomical Society in Britain waded in ... instructed that no astronomer who submits a paper to its journals should type the words “James Webb.” They must use the abbreviation JWST." [The New York Times] https://archive.ph/RpOby
Study looked at the long-term outcomes of universal preschool in Boston and found that it had no impact on standardized test scores. https://blueprintlabs.mit.edu/research/the-long-term-effects-of-universal-preschool-in-boston/
Which European Nations Are Overrepresented in the History of Science? https://inquisitivebird.substack.com/p/which-european-nations-are-overrepresented
“The Ottoman empire did as the Romans do and debased it's currency as it ran into trouble. The golden age was gone and due to rapid inflation so was the age of silver. Similar patterns exists in many empires.” https://threadreaderapp.com/thread/1611452139066347521.html
Many people thought they knew what the end would look like. Some imagined humanoid robots with laser guns or sketched plots involving molecular nanotechnology. Others outlined some intricate deception. But nobody expected the machines to be so funny that humanity would literally die of laughter.
I pretty much made my prediction in 2020:
Depending on how much of a difference sheer scaling made, I will update toward shorter or longer timelines. If a bunch of important capabilities emerged, lots of players will rush to create an even larger GPT-5.
I expect “bigger is better” to continue beyond even GPT-5. Scale will outshine all hacks and fine-tuning. After all, it works for biological neural networks. Primates with bigger brains are smarter. Humans with bigger brains are smarter.
Evidence for the scaling hypothesis isn't hard to find: https://www.gwern.net/Scaling-hypothesis
1. A recent meta-analysis showed that bigger brain size is robustly related to higher intelligence: https://royalsocietypublishing.org/doi/10.1098/rsos.211621
2. African-descended people (Blacks) average cranial capacities of 1267 cm^3, European-descended people (Whites) 1347 cm^3, and East Asian-descended people (East Asians) 1364 cm^3. Blacks average an IQ of 85, Whites 100, and East Asians 106: https://www.sciencedirect.com/science/article/abs/pii/S016028960200137X
3. Low SES people tend to have smaller overall GMV (gray matter volume) as well as somewhat different volumes within brain regions: https://www.science.org/doi/10.1126/sciadv.abm2923
4. On average, males have larger total brain volumes than females. Accordingly, there are more men at the right tail of the intelligence distribution: https://pubmed.ncbi.nlm.nih.gov/16248939/ (see also: https://largescaleassessmentsineducation.springeropen.com/articles/10.1186/s40536-019-0070-9)
5. Across studies, the correlation coefficient of height and intelligence was generally found to be around 0.2, indicating a positive association between height and intelligence: https://en.wikipedia.org/wiki/Height_and_intelligence
P.S. Birds have twice as many neurons per unit mass as humans. This is why the cognitive performance of even four-month-old ravens parallels that of adult apes despite their smaller brains.
If we could genetically engineer humans to have the same neuron density, how might this affect their cognitive performance?