Read Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges, and is this really the path to open-endedness? I think it’s cool that models can learn from a small set of training examples and then generalize to harder and harder examples. Also really cool to see that pre-trained models improve exponentially faster than cold-start! I think there’s something there to be learned of how rather than trying to evolve intelligent systems starting from 0, it might make sense to start with LLMs or other “intelligent” systems.

I think my main gripe with this paper was the fact that you had to have some human-designed heuristic for how hard a problem was. We don’t really get that in real life. I’m much more in the camp of Jeff Clune’s AI-GA where the problems also have to be set by AI so that this recursive self-improvement is truly open-ended.

This also reminded me a bit of Robust Autonomy Emerges from Self-Play in that interesting behavior emerges from self-play. The environment is just the same agent with other params. Could we do something like this but with LLMs?