Links for 2023-12-15
Can LLMs uncover new knowledge - the scientific equivalent of AlphaGo’s move 37? Google DeepMind unveils FunSearch, their new LLM-based approach for program search. It has uncovered new results in Maths and Computing. It pairs the creativity of an LLM with an automated evaluator to guard against hallucinations and incorrect ideas. https://deepmind.google/discover/blog/funsearch-making-new-discoveries-in-mathematical-sciences-using-large-language-models/
“Introducing Octo 🐙, a generalist robot policy, trained on 800k robot trajectories, stronger than RT-1X, flexible observation + action spaces, fully open source!” https://octo-models.github.io/
This new system can teach a robot a simple household task within 20 minutes https://www.technologyreview.com/2023/12/14/1085231/new-system-teach-robot-household-task/ [https://archive.is/rHChO]
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision — “In the future, humans will need to supervise AI systems much smarter than them. We study an analogy: small models supervising large models. GPT-4 supervised by ~GPT-2 recovers performance close to GPT-3.5 supervised by humans—generalizing to solve even hard problems where the weak supervisor failed!” https://openai.com/research/weak-to-strong-generalization
Deep neural networks show promise as models of human hearing https://news.mit.edu/2023/deep-neural-nets-show-promise-models-of-human-hearing-1213
Portable, non-invasive, mind-reading AI turns thoughts into text https://www.uts.edu.au/news/tech-design/portable-non-invasive-mind-reading-ai-turns-thoughts-text
“Noise-Reuse Evolution Strategies, an unbiased, online, memory efficient, variance-reduced gradient estimator that can outperform many other methods (including Backprop) on some particularly challenging unrolled computation graph problems.” https://arxiv.org/abs/2304.12180
“Why has Autonomous Driving (AD) failed? We believe that the key to unlocking AD may be in 1) collecting rich out-of-distribution data from emerging economies such as #Peru; and 2) developing brain-aligned computer vision models #NeuroAI” https://www.artificio.org/blog/why-has-autonomous-driving-failed-perspectives-from-peru-and-insights-from-neuroai
New Device Leads to “Dendrocentric Learning” -- Stanford researchers mimic brain structure with ferroelectric material https://spectrum.ieee.org/dendrites
Tesla Releases A New Humanoid Robot "Optimus-Gen 2" https://twitter.com/DrJimFan/status/1734960212786696407
On being wrong about AI https://scottaaronson.blog/?p=7672
Related: The Test That Terence Tao Aced at Age 7
Quote:
Terence got 60/60 on the Operations Test...Although I had given the test to many very bright primary-school-age children before, none of them had ever got more than 57/60 - and Terence was probably the youngest person I had ever asked to do the test.
Not only did he have an astounding grasp of algebraic definitions, for someone who was still seven years old, but I was amazed at how he used sophisticated mathematical language freely.
...Terence had taught himself BASIC language (by reading a book) and had written many programs on mathematics problems...Also, it is fascinating to observe that Terence wrote many of his programs...when he was 6 years old.
[It's also amazing how much progress he made over the next five weeks.]
...Terence, at age 6 years and 4 months, is said to have gained maximum or near maximum scores on the Wechsler Intelligence Scale for children...His overall Mental Age was 14 years...
Video: https://www.youtube.com/watch?v=I_IFTN2Toak
Full report: https://web.archive.org/web/20210521044614/https://math.fau.edu/yiu/Oldwebsites/MPS2010/TerenceTao1984.pdf