Links for 2022-12-13
“As the models scale up, the performance generally improves for both GPT-3 and PaLM model series on all languages. Neither model achieves a substantial solve rate until a certain scale...hence multilingual reasoning can be considered an emergent ability of large language models...It is worth noting that the amount of training data per language is constant across language model scales for PaLM—the fact that scale facilitates reasoning implies that further scaling may continue to improve the multilingual reasoning ability of large language models.” https://arxiv.org/pdf/2210.03057.pdf#page=6
A list of predictions for 2023 for the field of LLMs by Stanislas Polu https://threadreaderapp.com/thread/1602283271001350146.html
49-tweet thread summarizing the main ideas from all 15 of NeurIPS' dedicated Outstanding Papers. https://threadreaderapp.com/thread/1596911064251187201.html
Scott Aaronson: “My main project so far has been a tool for statistically watermarking the outputs of a text model like GPT...instead of selecting the next token randomly, the idea will be to select it pseudorandomly, using a cryptographic pseudorandom function, whose key is known only to OpenAI. That won’t make any detectable difference to the end user...” https://scottaaronson.blog/?p=6823
“Finite factored sets are a new paradigm for talking about causality. You can use them to do some cool things you can’t do with Pearl’s causal graphs, for example inferring a causal arrow between two binary variables.” https://www.lesswrong.com/posts/PfcQguFpT8CDHcozj/finite-factored-sets-in-pictures-6
Why Integrated Information Theory (IIT) is unsound (both mathematically and epistemologically). https://jakerhanson.weebly.com/blog/my-graduate-experience-with-integrated-information-theory-iit
Fitness levels accurately predicted using wearable devices – no exercise required https://www.cam.ac.uk/research/news/fitness-levels-can-be-accurately-predicted-using-wearable-devices-no-exercise-required
AI war startup Anduril raises $1.48 billion https://blog.anduril.com/anduril-raises-1-48-billion-in-series-e-funding-ac8c7299d182
“Why I’m optimistic about [OpenAI's] alignment approach” https://aligned.substack.com/p/alignment-optimism
You don't need a perfectly random sample for useful data, jfc https://aella.substack.com/p/you-dont-need-a-perfectly-random
Expert Opinion On Race, IQ, Their Validity, & Their Connection: What do the experts actually believe? How have the experts' views changed? How do the views of elite experts differ from the views of less-elite experts? https://werkat.substack.com/p/expert-opinion-on-race-iq-their-validity