“I think: a NN will use its weights to memorize answers because that's easy, until it has to memorize so many that it's easier to instead start encoding the algorithm that generates the answers. Neural nets are lazy, so you have to give them a hard enough job (enough, and diverse enough data) that they can't take the lazy way out.” —
Links for 2022-06-18
Links for 2022-06-18
Links for 2022-06-18
“I think: a NN will use its weights to memorize answers because that's easy, until it has to memorize so many that it's easier to instead start encoding the algorithm that generates the answers. Neural nets are lazy, so you have to give them a hard enough job (enough, and diverse enough data) that they can't take the lazy way out.” —