Status: rough first draft on trying to write down everything I know

what is the open-endedness end game?

Looking at the field, I’m wondering where we see it in 10 years. Will we have fully autonomous AI agents roaming the world? Will there be cheap little robots roaming around everywhere ensuring we don’t run out of training data? Will we learn how to better create and harness synthetic data?

While people point to AlphaGo/AlphaZero as to what open-endedness should look like within language, I’m not sure that notion is entirely correct. While it’s true that those models learned to play their respective games at superhuman levels, they never broke the rules. I’m not sure the same constraints apply to language/science. If we take a Kuhnian view of the world, then whatever a “superhuman” knowledge of Aristotelian physics means is still… quite useless. True open-endedness means rewriting the rules of the game yet somehow in a way that is more true?

the basic tenets of evolution are not enough

The basic definition of evolution is descent with modification. Usually this is paired with natural selection where things that reproduce better tend to survive more and are what persist. This alone isn’t enough to create open-endedness in computational systems. Why is that?

We have made systems where the three things for evolution are all there, but we don’t see open-ended growth. I think part of that might be the scale of the systems that we’ve made? After all, would the world be open-ended if we weren’t such tiny specks within the universe? Is it sheer scale that creates the near infinite space of novelty? Would the world look open-ended without humans? what if evolution stopped before humans? Is open-endedness a property of the environment as a whole and its ability to even bring about humans or a feature of humans?

how do we measure it?

This is the other founding pillar of open-endedness. How in the world do we tell whether one system is more or less open-ended than another? Is this even a question that is worth asking? I don’t have many answers here.

Rough outline: MODES toolbox as one way where you measure ecology, complexity, and some other benchmarks. The key problem when we were doing this is measuring complexity. Given an “organism” (which is already a tough line to draw), how can you tell how complex it is? Right now, the complexity measure has to be changed per environment. This is kind of fruit that there is a lot of work that could be done here to clear up this confusion.

Technically, the best thing to do would be Kolmogorov Complexity but that is unmeasurable. Is there a good way to approximate it? LLMs/compressors? (I should read the LLMs are general compressors paper)

If an “organism” arises at a new reproductive level, then does that lower the complexity at the lower levels? For instance, if we go from single to multicellular organisms, then the unit we were previously observing would no longer be the primary unit of reproduction. In that case, we might actually see complexity lower on an individual level as the higher organism seeks to control it? Would there be more diversity? I’m not sure. Also want to think about these questions. Intuitively, I think there being higher orders imposes some order on the lower levels and limits the amount of entropy there is. In that case, it might seem like complexity is decreasing when it has actually moved to another level. How can one deal with that?

do we need mutation?

There was the recent paper Arcas et al. 2024 (TODO: fill in link) where they found that even without mutation, you could eventually get self-replicating programs just from random recombination. That seems kind of wild to me. Of course, the building blocks that the programs were made of were a lot higher level than the chemical building blocks, but it does make me wonder. In their paper, subleq wasn’t able to create any self-replicators because the minimal program needed was too long.

Do we have an idea of what the minimal replicator is within this universe? There’s “something something’s monster” (lookup the guy from Illinois), but even before RNA, what is the smallest replicator that is possible within the universe or that exists? I think this is an important question for how evolution was able to bootstrap itself.

If you want to reduce the scale of operations like mentioned above, then perhaps you need to enable the lowest level to be replicators. I would’ve loved to see what would’ve come from if Arcas had run their system for 10x as long. It seems that it mostly died out, and there is a hard-cap on how large organisms can be in their system. Is there a way to change that? How could we extend their work to allow for arbitrarily large programs? Some set of rules that could create open-endedness.

required for AGI

TODO! There was this position piece from DeepMind about this topic that I had a chat to one of the people about. I think this is largely agreed upon though. How can something be a general intelligence if it is fit for a static distribution? Perhaps you could argue that humans are also kind of like that, but I think we adapt much faster than LLMs could. It isn’t that we’re the ultimate intelligences, just that LLMs clearly aren’t there yet and a big part of that is how they learn a static distribution. I think a key development in the next 10/20 years will be trying to find how to get them to keep learning (lifelong learning), but not just distributions that humans feed in but that they make themselves. On the other hand, why can we not have millions of tiny robots feeding video and what not into a LLM and then it predicts how well something will do.

Tangent: I wonder if we could use LLMs as an evolution operator. I know it has been done before (I don’t quite remember the name), but it has inspired me that it has been used in RL where you can just prompt it for what an expert level player would do and it should do that. What happens if you prompt it for something out of training data? I think it probably falls apart? Maybe? Can you actually perform better than the data it was trained on? Curious. I’ll have to look into this.

simple rules

I’m of the belief that there are extremely simple rules for creating open-ended evolution. In much the same way that physics is just one equation now that describes the cosmos, I see no reason we cannot have something similar for infinite creativity. In a sense, my inspiration here is something like AlphaGo/AlphaZero where they took a field riddled with hacks and heuristics and proved a simple solution scaled up could work.

In open-endedness, find where the heuristics are. When people are researching how to patch up holes and new methods, that’s where there is progress to be made since something here can be clarified.

papers

Promptbreeder

Rough idea here is that rather than doing manual CoT, we should instead have an algorithm that is able to generate these prompts automatically! Key thing here is that while we have a set of mutation operators (“change this prompt to be more fun”, “elaborate on all your thinking”, etc.), we also have a hypergraph where we’re also able to adapt these mutation operators!

Something here that erme