Status: This is very much a working draft where I might completely go back on what is written. If you have any thoughts or comments, please reach out to me on Twitter.

In his essay The Bitter Lesson, Richard Sutton argues that while hybrid models of neural networks infused with human knowledge dominate in subdomains to begin with, networks with no information and more computation eventually supersede them. This pattern can be seen in the development from Deep Blue, the neural network which beat the Chess World Champion using search and expert heuristics, to the modern day MuZero which acquired superhuman skills with no knowledge of even the rules.

While human knowledge being completely removed from artificial intelligence is still on the horizon, I believe that there are lessons from AI that can be applied to human cognition.

Note: This is NOT related to DL frameworks for neuroscience, but merely thoughts on applying concepts from DL to everyday life. I just wanted to have a bit of fun reflecting on how principles in ML could be useful in everyday life.

I. Data

No matter how good an algorithm is, our AI today requires two requirements for the data: a metric ton and variety. This has gotten me thinking about what I’m willing to allow into my life and the importance of the media and thoughts I intake.

I’m currently reading Guns, Germs, and Steel, and although I’m only at the beginning, the argument for environment dictating differences in humans seems not only compelling, but applicable to both AI and human intelligence. More complex environments where people had to contend with competing Homo Sapiens led to advances such as writing and naval technology.

How is this principle something that can be harnessed to enhance human intelligence?

While the platitude of “surround yourself with good people” needs no introduction, how strongly do your peers dictate your own success? I’m sure there’s literature on this that I haven’t gotten around to reading.

In a similar vein, is there a way to bootstrap your way up? Could you cold-email and harness social media to craft a virtual community of great thinkers to enhance your own abilities?

Since I’ve first drafted this essay, I’ve attempted to do so through Twitter and cold emails, and here are my conclusions as of now (April 10, 2022): sort of.

You can build a Twitter timeline with awe-inspiring people that push you to do better, but I discounted the importance of the people you interact with on a day to day basis. These are the people who help build your ideas and challenge your beliefs. Passive consumption isn’t enough.

A community isn’t a body of literature you interact with through blog posts or 280 Unicode glyph messages, it’s a collection of individuals. It’s web of connections and friendships.

You find someone interesting, send a message into the ethereal realm, and hope they find you as interesting as you find them. One message at a time, you slowly find your people.

Maybe they don’t respond, so you follow up, follow up, and then follow up.

It was never about “crafting a virtual community” or “bootstrapping your way up”, it was about making amazing friends.

II. Big Data

Sometimes I feel bad for our little networks. They’re stuck there ad infinitum, churning away in their same distribution.

We aren’t like that. We have the power to change the world around us. It’s easy to stay in the same distribution, as they offer a comfort in the same manner as “better the devil you know than the devil you don’t” – they’re predictable. But being predictable isn’t optimal. Comfort in knowing how your actions will affect the world aren’t as satisfying as your actions having a more positive impact on those around you.

TODO: Finish writing how we can change distributions! We’re a bit like unaligned neural networks when we stay safe in the distribution we know for “high reward” rather than the one we don’t. Resistance to change!

III. Calculus

AI, at least the models we use today, are dressed up calculus. Back-propagation calculates how much each weight or bias affects the end result and tweaks them to achieve a smaller loss.

{Weekly, Monthly, Yearly} Reviews can be found in any productivity blog you come across, but I wonder if we’re really optimizing for the correct targets. Would you be better off optimizing for self-efficacy, focus, or something else entirely? While the loss function for a model is the same throughout its life-cycle, our goals and to a lesser extent, values, are constantly changing.

• What are improvements to core beliefs that we could reliably “backpropagate” each week which wouldn’t change over a lifetime?
• What models do we have for dealing with such an uncertain world and shifting objectives (Classic explore versus exploit problem)?
• Much more philosophical, do there even exist fundamental beliefs?

Thank you to Jason Benn for originally inspiring the idea of back-propagation applied to daily life and Kanjun for helping me refine some ideas.