Capabilities of large generative models can emerge abruptly and unpredictably: "For example, the percentage of generated synthetic programs that solve a given programming problem jumps substantially from 6% to 13% when the model size increases by ∼ 2x from 68B to 138B parameters, despite very small increases over the previous two orders of magnitude."
Links for 2022-02-18
Links for 2022-02-18
Links for 2022-02-18
Capabilities of large generative models can emerge abruptly and unpredictably: "For example, the percentage of generated synthetic programs that solve a given programming problem jumps substantially from 6% to 13% when the model size increases by ∼ 2x from 68B to 138B parameters, despite very small increases over the previous two orders of magnitude."