Responding to your chess question. Starting with random weights? No, not even close. For a talented(i.e having some innate ability to learn the game) child, it's actually hard for me to describe how fast they can improve even over the course of a single game. I've seen it happen in real time several times.
It's very hard for me to ignore the intuition that there's some higher level cognitive process going on that can pick out abstract concepts and use them to control and focus the lower level "training" that might look more similar to what we're doing in ML these days.
How do you know that isn’t emergent given enough embodied experience of the world? This is the big thing with LLMs for me, showing that yes, if you scale up, all sorts of emergent abilities pop up. And LLMs aren’t even embodied, it’s still a very primitive approach and we’re already seeing this level of competence. Truly makes me wonder if I’m not just a stochastic parrot underneath it all.
It's very hard for me to ignore the intuition that there's some higher level cognitive process going on that can pick out abstract concepts and use them to control and focus the lower level "training" that might look more similar to what we're doing in ML these days.