I expect that at some point within the next few years someone (hopefully anthropic) will "solve" software engineering.
By this I mean that we'll have an AI system that is
more effective without human assitance than with
.
What do I do at that point?
I'm pretty good at softwere engineering; I span a pretty wide chunk of the stack, and one of my greatest strengths is learning and operating within a novel system quickly.
Frequently I end up as a "cursed systems specialist" who can make sense of things that an LLM would have a hard time with.
At the point where I don't have a comparative advantage as a SWE, what bottlenecks remain to be solved?
Macro Engineering
Play the
RTS game
managing a swarm of agents at a higher abstraction level.
I dunno, I think this is at most a temporary stopping point along the journey.
It's possible that this lasts us up until the point of singularity; that this is the last bottleneck, and at the point where one semi-technical uber-manager can achieve perfect utilization of physical resources, everything else is solved.
I think this is unlikely -- engineering is a bounded problem. It's straightforward to imagine perfect utilization of a compute fleet (given a certain level of research understanding), perfect reliability, etc etc.
Versus proving optimality and/or safety properties of a given training algorithm sounds much harder.
But maybe perfected research tools make those problems tractable?
Expecting to move through this phase, but not to dwell here in the limit.
Pivot to Research
Focus on the math and science behind the algorithms that we run, and try to improve those more effectively than Claude can.
Again, I'm skeptical -- I think there are already lots of humans better at this than I am.
Pivot to Interp
There's a possible safety argument for having humans that can understand and make sense of the god-AI's machinations, even if they are less effective than the AI at doing so.
I actually do think this is a likely path.
Even if humans are unlikely to adequately understand the decision process, this is likely a realm where every little bit helps.
Robots
Software is liable to go fast because you can give the AI a quick iteration loop, but operating in the real world might be much harder and slower.
I am theoretically an electrical engineer, I could go push on robots for a bit.
The outside view is that this is better left to the specialists, but I've learned that with a little bit of effort I can probably overcome this.
This is why I'm starting Sparrow Workshop -- it's a long shot, but on the off chance that it's useful I'd regret to have not tried.
Sit By a Lake
Enjoy the singularity, or else the last moments of pre-singularity humanity, comfortable that I've done my part.
The goal is to end up here eventually, and I'd do well to remember that, even if I'm blasting stimulants trying to keep up with the frentic pace of the SF cyberpunk opera right up until the moment before.