“Want generalization? Train your LM for longer! Here, we find that vanilla transformer LMs (no pre-training) generalize to structurally novel inputs (and acquire syntax), if you train them for much longer than needed. We call this behavior “Structural Grokking" ...” https://arxiv.org/abs/2305.18741
Models generating training data: huge win or fake win? https://dblalock.substack.com/p/models-generating-training-data-huge
The performance of Language Models is highest when relevant information appear at the beginning or end of the input context, and significantly lower otherwise. https://arxiv.org/abs/2307.03172
“Toyota claimed it had made a ‘technological breakthrough’… that would allow an electric vehicle powered by a solid-state battery to have a range of 1,200km and charging time of 10 minutes or less.” https://www.ft.com/content/87cb8e92-8e82-4755-8fc3-2943f8f63e1d [https://archive.is/aZVks]
At Last, Single-Photon Cameras Could Peer Into Your Brain https://spectrum.ieee.org/single-photon-camera
Risk-taking in chimpanzees mirrors risk-taking in humans: It’s a stable individual trait; it’s more common in males than females; and it peaks in early adulthood. https://journals.sagepub.com/doi/10.1177/09567976221140326
Sappers risk their lives to win Ukraine back, inch by inch: ““We don’t have air cover or plentiful Western equipment, so we risk our people. Small group tactics, like the Finns used in the winter war [of 1939-40].” Surprise tactics were the only way to breach an obstacle course like no other built since the second world war, he said.” https://www.economist.com/europe/2023/07/09/sappers-risk-their-lives-to-win-ukraine-back-inch-by-inch [https://archive.is/Xwm13]
Rheinmetall kamikaze drone carrier https://www.youtube.com/watch?v=Cfi7sdGnJPY
Rheinmetall Mission Master CXT – Fire Support, an autonomous counter-UAS solution https://www.youtube.com/watch?v=uYhlNBfMZl4
I cannot overemphasize how little there was to do before we all had smartphones. A barren expanse of empty time would stretch out before you: waiting for the bus, or for someone to come home, or for the next scheduled event to start. Someone might be late or take longer than expected, but no notice of such delay would arrive, so you’d stare out the window, hoping to see some sign of activity down the block. You’d pace, or sulk, or stew.
https://www.theatlantic.com/technology/archive/2023/07/before-smartphones-boredom/674631/ [https://archive.is/pUmLx]
Keep up the good work, Alex. Your Substack is the first thing I read every morning.