Links for 2023-11-13
Google seems to have experimented with a 50,000+ TPU training run: "To give a sense of scale, this cluster of Cloud TPU v5e chips has more AI accelerators than the TOP1 Supercomputer Frontier at Oak Ridge National Laboratory, which featured 37,888 AMD M1250X GPUs" https://cloud.google.com/blog/products/compute/the-worlds-largest-distributed-llm-training-job-on-tpu-v5e
NVIDIA's Eos supercomputer just broke its own AI training benchmark record: The system can train a 175 billion parameter GPT-3 model in under four minutes. It needs just 7.2 seconds for BERT. https://www.engadget.com/nvidias-eos-supercomputer-just-broke-its-own-ai-training-benchmark-record-170042546.html
Try out the Blazing Fast LCM Lora SD 1.5 in your browser. It can generate images as fast as you can type. https://huggingface.co/spaces/latent-consistency/super-fast-lcm-lora-sd1.5
Curiosity-Driven Learning of Joint Locomotion and Manipulation Tasks https://www.youtube.com/watch?v=Qob2k_ldLuw
Medical forms are littered with jargon that nobody understands. Can ChatGPT help? https://www.bostonglobe.com/2023/08/23/metro/can-chatgpt-help-with-medical-forms/ [https://archive.ph/1u3MK]
“Compared to state-of-the-art libraries such as HuggingFace PEFT and vLLM (with naive support of LoRA serving), S-LoRA can improve the throughput by up to 4 times and increase the number of served adapters by several orders of magnitude.” https://arxiv.org/abs/2311.03285
Integration of 3D-printed cerebral cortical tissue into an ex vivo lesioned brain slice https://www.nature.com/articles/s41467-023-41356-w
"The fact that such a large object can exist only half a billion years after the Big Bang … strongly suggesting that supermassive black holes formed without ever having gone through an intermediate step involving a star." https://arstechnica.com/science/2023/11/half-of-the-mass-of-an-early-galaxy-is-in-its-central-black-hole/
“So, if you know computer science history, this is kind of amazing. Augustus de Morgan (he of "de Morgan's Laws" in logic) wrote this in the middle of the 19th century. I would say he was quite right about the future fame of his good friend Boole, who had recently passed away.” https://twitter.com/ZachWeiner/status/1722997405988118819
The fact that this dumb trick works well enough that the model even has a physical model of reality to interpret “a ball balancing on top of a door” and produce a perfectly legible image to us feels very spooky. The universe shouldn’t be that simple to encode!
Political links:
More "Game Changers" (and Failures) in Ukraine - From Starlink & Electronic warfare to Hypersonics https://www.youtube.com/watch?v=jaWVrphbHXI
The Ukrainian 47th Mechanized released additional footage of smashed Russian mechanized formations outside of Avdiivka. https://twitter.com/Osinttechnical/status/1723780085809914361
Avdiivka Frontline: Logistics in Focus https://frontelligence.substack.com/p/avdiivka-frontline-logistics-in-focus
"U.S. military forces conducted precision strikes today on facilities in eastern Syria used by Iran's IRGC and Iran-affiliated groups in response to continued attacks against U.S. personnel in Iraq and Syria." https://www.defense.gov/News/Releases/Release/Article/3586509/statement-from-secretary-of-defense-lloyd-j-austin-iii-on-additional-us-militar/