Links for 2024-05-28
AI:
Transformers Can Do Arithmetic with the Right Embeddings — Achieves up to 99% accuracy on 100 digit addition problems by training on only 20 digit numbers with a single GPU for one day. https://github.com/mcleish7/arithmetic
Machine Learning and Information Theory Concepts towards an AI Mathematician. ~ Yoshua Bengio and Nikolay Malkin. [PDF] https://www.ams.org/journals/bull/2024-61-03/S0273-0979-2024-01839-4/S0273-0979-2024-01839-4.pdf
Mathematics as a Translation Task - the Importance of Training Distributions https://www.youtube.com/watch?v=k9xLg-3WfG8
HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models https://arxiv.org/abs/2405.14831
NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models https://arxiv.org/abs/2405.17428
MoEUT: Mixture-of-Experts Universal Transformers https://arxiv.org/abs/2405.16039
IDEA Research unveiled Grounding DINO 1.5. It's a new set of AI models that can accurately detect and ID objects in images and videos in real time. https://github.com/IDEA-Research/Grounding-DINO-1.5-API
Artificial intelligence is far more efficient at producing content than human beings, as far as carbon emissions go. https://www.nature.com/articles/s41598-024-54271-x
The Zhousidun dataset: A Chinese-originated dataset that exhaustively labels critical components of American and Allied destroyers. https://arxiv.org/abs/2405.12167v1
Some falsifiable predictions about AI from an economist: “What about the share of tasks that will be affected by AI and related technologies? Using numbers from recent studies, I estimate this to be around 4.6%, implying that AI will increase TFP by only 0.66% over ten years, or by 0.06% annually. Of course, since AI will also drive an investment boom, the increase in GDP growth could be a little larger, perhaps in the 1-1.5% range.” https://www.project-syndicate.org/commentary/ai-productivity-boom-forecasts-countered-by-theory-and-data-by-daron-acemoglu-2024-05 [no paywall: https://archive.is/1fYkJ]
Stanford discusses the dark arts of GPU programming https://hazyresearch.stanford.edu/blog/2024-05-12-tk
Miscellaneous:
Truthseeking is the ground in which other principles grow https://www.lesswrong.com/posts/kbnJHpapusMJZb6Gs/truthseeking-is-the-ground-in-which-other-principles
Paper Tiger? Chinese Science and Home Bias in Citations https://www.nber.org/papers/w32468
Local exposure to poor individuals reduces support for redistribution among the well-off. https://www.cambridge.org/core/journals/british-journal-of-political-science/article/unequal-and-unsupportive-exposure-to-poor-people-weakens-support-for-redistribution-among-the-rich/FC32FA59B3C5525A178C7012859F95D8
Females mount stronger immune responses to many pathogens, they awaken more frequently at night, they express greater concern about physically dangerous stimuli, they exert more effort to avoid social conflicts, they exhibit a personality style more focused on life’s dangers, they react to threats with greater fear, disgust and sadness and they develop more threat-based clinical conditions than males. https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/abs/selfprotection-as-an-adaptive-female-strategy/596ABFD521F358F1E8797C542084679B
Exclusive Interview: Ukrainian Sniper's Arsenal Revealed https://www.youtube.com/watch?v=KEgLrrrs5ZQ
On Miscellaneous 3: These findings are not surprising.
If you have regular contact with the impoverished, you gain a sense of whether their poverty is due to personal factors like (a) impulsivity, (b) sensation seeking, (c) low intelligence, and (d) the consequent idiotic choices they make. And if you decide from observation that the impoverished both have bad character AND are rock stupid, you will not believe that redistributing wealth to them will make any difference to their poverty. They will just be poor again till the next welfare cheque comes in.
I am supportive of some redistribution, but on the general, moral principle that no wealthy society should let even its dumbest, most offensive members starve to death, die of exposure, and have no possibility of accessing education. Anything more than that should be done through private charity.
I don't have a negative or positive opinions of Jeff Bezos, Elon Musk, or any other billionaire, but if the state is going to take an increased amount of their income, I put that to the same standard I would ask for taking more of mine: it should be spent on things that give long term benefit, widespread benefit like infrastructure or basic science research. Redistributionist policies are most likely driven more by envy and resentment than genuine concern for the worst off - if people genuinely have that concern, they can be charitable with their own funds.