Links for 2023-10-07
Some people are very excited about this result claiming the"hard problem" of mechanistic interpretability has been solved: “The fact that most individual neurons are uninterpretable presents a serious roadblock to a mechanistic understanding of language models. We demonstrate a method for decomposing groups of neurons into interpretable features with the potential to move past that roadblock.” https://transformer-circuits.pub/2023/monosemantic-features/index.html
“AI models are not just black boxes or giant inscrutable matrices. We discover they have interpretable internal representations, and we control these to influence hallucinations, bias, harmfulness, and whether a LLM lies.” https://www.ai-transparency.org/
“We found a pattern in how NNs extrapolate: as inputs become more OOD, model outputs tend to go towards some “average”-like prediction.” https://arxiv.org/abs/2310.00873
Training 100000 small networks, studying their statistics and interpreting them. Leads to an intriguing mechanism for neural scaling laws, which happens to explain the 0.34 exponent observed in Chinchill. https://arxiv.org/abs/2310.02258
Large Language Models as Analogical Reasoners https://arxiv.org/abs/2310.01714
LLM-grounded Video Diffusion Models https://arxiv.org/abs/2309.17444
“Researchers are learning just what’s possible using the gene-editing technology CRISPR, which advocates say can prevent disease, improve food security and reduce methane emissions.” https://modernfarmer.com/2023/09/the-future-is-crispr/
Why is inflation bad? https://mru.org/courses/principles-economics-macroeconomics/costs-of-inflation-price-confusion-money-illusion
“There were many brothels in ancient Rome, but standard Roman coins, Serters and Denars, were not used and were even banned from being there. Store-bought Tokens were used for Denar and Serters and images on them were often obscene. Reason is understandable, because on all Roman coins there was a head image of emperor of that period. Even picture of emperor entering the brothel was synonymous with humiliating and cursing him. Of course, punishment was severe.” https://twitter.com/archeohistories/status/1708099551952036039
Finland has 54,000 nuclear bunkers. Finnish civil defence shelter tour guide: “Please, take photos. Share on social media. We want to signal to Russia that we are prepared.” https://www.youtube.com/watch?v=922Db-kztC0
The first half of a new open-access online textbook in human population genetics https://web.stanford.edu/group/pritchardlab/HGbook.html
A sci-fi short story about a sapient mitotic species. https://etirabys.tumblr.com/post/730007640274567169/the-dreamtime
The James Webb Space Telescope has spotted Jupiter-sized "planets" free-floating in space. These objects appear to be moving in pairs. Astronomers are currently struggling to explain them.
I asked Bing to create images of these objects and their inhabitants:
The filaments are advanced nanotech harvesting elements from interstellar space.
Read more: https://www.bbc.com/news/science-environment-66974738
Ukraine links:
Accumulations of destroyed/damaged Russian tanks, including two T-90M. Plus one Grad MLRS. Luhansk region. As the source says, "More than 10 destroyed tanks per 2 square kilometers, which were destroyed literally in two days". https://twitter.com/bayraktar_1love/status/1710251092238303680
A Russian assault group is caught by drones in the Bakhmut direction and immediately gets supressed by grenades dropped on top of them. https://twitter.com/NOELreports/status/1710362406885458371
“Escadrone released footage of what can only be described as a Ukrainian GUR FPV drone rampage through the Russian front and rear areas around Bakhmut” https://twitter.com/Osinttechnical/status/1710481979576140260
“Recently, I've tried to refrain from making negative statements about the overall situation. However, over the past two months, I've come across a continuous stream of statements that I find not only incorrect but also potentially harmful.”
https://twitter.com/Tatarigami_UA/status/1710382234803290154
Israel links:
Hamas began a military operation against Israel launching 5000 rockets within 20 minutes; fighters entering Israeli territory https://twitter.com/JoeTruzman/status/1710505139998789862
Sderot, Israel: “There are images of piles of dead civilians in bus stops. People on their way to work, dragged from their cars and shot right there in the streets.” https://twitter.com/tobiaschneider/status/1710555037007491150 (Video: https://t.me/supernova_plus/24346)
A Merkava Mark IV Main Battle Tank used by the Israeli Defense Force has been Destroyed by Palestinians along the Border with the Gaza Strip. https://twitter.com/sentdefender/status/1710529957024104617
Footage of a hamas infiltrator using a motorized hang glider to get into Israel. https://twitter.com/Global_Mil_Info/status/1710511661533028629
Captured Israeli military jeep https://twitter.com/michaelh992/status/1710546756633530873