Links for 2024-03-20
Are we one step closer to having AI generate code better than humans?
There's a new open-source, state-of-the-art code generation tool. It's a new approach that improves the performance of Large Language Models generating code.
The paper's authors call the process "AlphaCodium" and tested it on the CodeContests dataset, which contains around 10,000 competitive programming problems.
The results put AlphaCodium as the best approach to generate code we've seen. It beats DeepMind's AlphaCode and their new AlphaCode2 without needing to fine-tune a model!
Paper: https://arxiv.org/abs/2401.08500
Blog: https://codium.ai/blog/alphacodium-state-of-the-art-code-generation-for-code-contests/
Code: https://github.com/Codium-ai/AlphaCodium
In fact, none of our chips are possible today without AI. Literally. The H100s we're shipping today was designed with the assistance of a whole lot of AIs.
— Jen-Hsun Huang, CEO of Nvidia
See also:
MuZero, AlphaZero, and AlphaDev: Optimizing computer systems https://deepmind.google/impact/optimizing-computer-systems-with-more-generalized-ai-tools/
In Race for AI Chips, Google DeepMind Uses AI to Design Specialized Semiconductors https://www.wsj.com/articles/in-race-for-ai-chips-google-deepmind-uses-ai-to-design-specialized-semiconductors-dcd78967
More AI links:
IBM presents Larimar: Large Language Models with Episodic Memory Control -- Attains accuracy comparable to most competitive baselines while yielding speed-ups of 4-10x https://arxiv.org/abs/2403.11901
VideoAgent🦾, a 🧠memory-augmented (RAG), 🔧tool-use agent for general video understanding. http://videoagent.github.io
1X robotics company released a new update: “1. all capabilities merged into a single set of goal-conditioned network weights 2. contact-rich manipulation skills like table wiping and t-shirt folding 3. general picking of diverse objects. Everything in the video is the same autonomous neural network checkpoint running on multiple robots, across many tasks. No teleop or scripted motion plans.” https://x.com/1x_tech/status/1770106870105358770
Apple presents MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training https://arxiv.org/abs/2403.09611
“Language models can imitate patterns in prompts. But this can lead them to reproduce inaccurate information if present in the context. Our work shows that when given incorrect demonstrations for classification tasks, models first compute the correct answer internally, but then "overthink" and override it with the incorrect answer. We trace this behavior to specific "false induction heads”, where ablating these heads helps mitigate incorrect-context following.” https://arxiv.org/abs/2307.09476
GiT: Towards Generalist Vision Transformer through Universal Language Interface https://arxiv.org/abs/2403.09394
“What I learned from looking at 900 most popular open source AI tools” https://huyenchip.com/2024/03/14/ai-oss.html
Miscellaneous:
Making Cells Young (On Aging and Cellular Reprogramming) https://press.asimov.com/resources/making-cells-young
A probabilistic, chemical computer. https://www.nature.com/articles/s41467-024-45896-7
Intra and interindividual Connectivity Analyses (IICA) A neuropsychological method to improve creativity in teams https://osf.io/preprints/psyarxiv/m6hqn
A bee is taught by humans to solve a complicated puzzle. That bee shows other bees how to solve the puzzle. Yet without this 'demonstrator bee', no bees can figure out the puzzle. https://www.nature.com/articles/s41586-024-07126-4
"a small percentage of workers is responsible for the bulk of the work...the top 10% of the most prolific elite can be credited with 50% of all contributions, whereas the bottom 50% of least productive workers can claim only 15%...the most productive contributor is about 100 times more prolific than the least" https://psycnet.apa.org/record/1989-01127-001
“Robin Hanson: if fertility trends continue, humans could be extinct in as few as 1600 years” https://x.com/tsarnick/status/1769592649764470921
Ukraine:
“In Engels, Russia, explosions rang out at night after the city was attacked by drones. UAVs also flew in the Bilhorod and Saratov regions. A strategic aviation base is located in Engels, where Tu-160 and Tu-95MS bombers are located, which are used in mass missile attacks on Ukraine.” https://x.com/NOELreports/status/1770344562491293934
“Some brief thoughts following a recent field study trip to Ukraine. The current situation is difficult, but UA is working to stabilize the front. Much depends on whether the US provides support, and UA effectively addresses its manpower issues in the coming months.” https://x.com/KofmanMichael/status/1770140961059831916