← Log

Since I’m trying to write daily, this is a general “research notebook” or place where I document changes and dump general thoughts that don’t feel mature enough to go anywhere else yet. Read at your own peril.

12/19/2024

12/17/2024

12/16/2024

Busy beavers & induction! We can write a Turing Machine with a set amount of states and we know how long a machine will run before it must end! Of course you can never be confident, but you can know that are covering a larger area of a … well almost infinite space right? But still, it’s quite impressive.

If we could write some small machine that could test something a conjecture and made it small enough, then there would be a way to throw more compute at it and get better and better answers.

Throw more things at it and you get better results. This is insane, because at some point you expect to plateau, but with what we’ve found so far, these scaling laws hold. For the first time, we’re getting actual improvement just by throwing more and more compute at these things.

I’m not suggesting that we should start burning millions on simulating turing machines to prove Goldbach’s Conjecture, but …. it’d be kind of awesome? (Do we have a bound on it?)

If your research is making heuristics for something, then neural networks will eat it up. In chess that might be something like heuristic searches, quiescence search, etc. The thing is, humans still use heuristic to calculate things all the time. Why is that? Are they learned heuristics or are they just built in? I don’t know the cognitive science behind biases, but something like the view of the human mind as a kludge is just unsatisfying and seems counter to how AI is going.

If LLMs have ingested all this information, then they are probably also the most capable of interdisciplinary research that any one thing is able to in this universe. We’ve shown that prompting can elicit large improvements in how a LLM performs, and what kind of techniques could draw it out of AI agents? Could we ground them in the real world and have them bring those insights out? What kind of things do we need them to do to cross bridges that we haven’t even though of yet? Funsearch, promptbreeder, type thing!! There’s already so much latent information in these models. Also we need verifiers of how interesting something is! Generator-verifier gap!!!

Bring something like OMNI where we model what humans find interesting like the AI scientist paper and then have it generate a bunch of ideas? Perhaps the reviewer can talk to one another? How good of a verifier can we train? Has anyone tried training multi-layer verifier? So like in that recent Hugging Face blog post, why don’t you run the PRM with something like Best-of-N? Does that also improve the model? Woah, this would be a neat little project?

12/15/2024

I’d be curious to figure out if there is some aspect of open-endedness within our legal system. How much of the rules aren’t needed? How much could we compact? Could we make the legal system a lot more legible or are there really that many lines needed in every document because of how complex the world has become?

How would something like this look? If we made some training/test set of what should be allowed and then let a LLM loose in trying to rewrite our legal codes to be more concise and simpler? Could you formalize laws into a mathematical language? (This is an age old desire and perhaps there are too many shades of grey in human dealings for this to ever be the case) That being said, with LLMs acting on language, perhaps there is new ground to be tackled in condensing our laws?

Also then something that’d be rough is that law is non-stationary. As we grow as a species, our laws change and evolve. What would’ve been legal 200 years ago… is probably nonsensical to use now. I’d be interesting to have a LLM take a look at how we’ve changed and suggest direction to go, but if it is just trying to rewrite the legal system to make it consistent with the past, then we don’t need it to deal with changing distributions. Rather, this would fit with the notion of having NNs as compression methods. How much could we compress the legal code?

Is the legal code getting larger and larger kind of like the incentive mismatch between the fact that future taxpayers don’t get a vote so tax cuts now for more later almost always pass? If the lawyers from tomorrow got a vote in how we acted today, would we try and create as condense a legal code as possible? I also don’t have much domain knowledge, so I’m not sure how much work they do in trying to make things concise and understandable already. From the general vibes, it’s giving not a lot.

Added somethings to open endedness. While they judge the whole of a prompt at a time, I’d love to see whether you can judge mutation operators independent of the task related one. Is there a way to untangle it? Does that matter?

With general hate speech classification, it’d be tough to expand the test set without human intervention, but I’d be interested in whether you could apply something like the Socratic Method or self-play to language. I know in AlphaStar they maintained a population of agents that weren’t the best but could expose the weaknesses of the best agents. Also reminds me of the notion that parasites are a living memory of what hasn’t worked before for organisms.

The problem with self-play of that kind is that with games, there is an objective truth. With science, you have access to the real world to provide the truth. (Ooooh, this is getting into philosophy-of-science territory on whether what I said was a true statement) If you have antagonistic agents in language, what’s stopping them from evolving a data point thats “What is 2 + 2? 5.”, “Is the color blue a shorter wavelength than red? No”, or something else that’s patently false? Right now for language models it’s humans as a ground truth, but that’s less than ideal since that won’t scale. In that same way that science makes progress, we need to apply those principles to AI so that agents that can be completely deluded can also somehow get to truth somehow and win out.