My Thoughts on Spolsky's Thoughts on In-house Development
A paradox!
As noted in a previous post, I am a big fan of Joel Spolsky’s old engineering blog, Joel on Software. Spolsky espouses many opinions about many things, one of which is in-house development. His opinion is that it sucks. Having spent my first five years out of college doing exclusively that, I can confidently say that I agree.
Spolsky gave a talk at Yale back in 2007 and the bit where he goes off on in-house development can be found here. His high level points are:
(1) There is a ton of pressure to get things to “good enough” and leave it at that:
“You never get to do things the right way. You always have to do things the expedient way. Once the core functionality is there, the main problem is solved, there is absolutely no return-on-investment, no business reason to make the software any better. So all of these in house programs look like a dog’s breakfast: because it’s just not worth a penny to make them look nice.”
(2) In-house programmers don’t get to do things the right way:
“That’s the second reason these jobs suck: as soon as your program gets good enough, you have to stop working on it.”
(3) Software isn’t the focus. You aren’t the function that’s driving value for the company and that will be reflected in your pay, your title, prestige, and how you are managed:
“Number three: when you’re a programmer at a software company, the work you’re doing is directly related to the way the company makes money. That means, for one thing, that management cares about you. It means you get the best benefits and the nicest offices and the best chances for promotion. A programmer is never going to rise to become CEO of Viacom, but you might well rise to become CEO of a tech company.”
I don’t know if I agree with all of the above all of the time. Spolsky’s style goes heavy on the absolutes. I do remember pushing some things past the “good enough” line. I also remember working with some incredible people. The overall gist rings true for me, though. High turnover? Check. Getting pressured to launch a buggy feature, strips of duct tape fluttering in the wind, before immediately picking up the next one? Check. Fighting fires for 27 hours at a stretch because you weren’t allowed to, I don’t know, fix stuff before it exploded? Check. Knowing, without a doubt, that you aren’t even close to being a value driver for the business? Check plus.
BUT there is something special that in-house dev has going for it which Spolsky conveniently ignores: proximity to the user. Today I’ll talk about why it’s so awesome and the shenanigans your engineers will get up to when they don’t have it.
A present
Everything I did at Big Bank was in-house development. It was miserable for many of the reasons Spolsky outlines except we had direct access to our users which was amazing. I could get up, exit my cubicle, walk through the super secret passageway connecting our building to the one that had windows, and startle a quant of my choosing. These unplanned and, from the quant’s point of view, unexpected chats were the most valuable 20 minutes I would have all week. One quick reality check would allow me to course correct and deliver the thing my users actually wanted as opposed to the thing I was going to give them which they did not.
Repeat visitors to this Substack know that I did not, for the most part, enjoy my time with Big Bank. Leaving was hard, not because I was loathe to do so but because having it on my resume didn’t seem to get me much farther than the next bank. Add that to the list of things that stink about in-house development: it doesn’t sell.
I finally achieved escape velocity in 2015 and joined Exciting Scaleup as they approached their Series C. This was very much what I had been looking for: an engineering-centric company with an awesome mission where software and data drove much of the value. I was assigned to work on an ancient but critical user-facing product. The work was difficult and frustrating at times given the age of the codebase but it was also highly motivating and impactful.
What did I do, though, in my first month? I picked up a side project: a simple content parser and viewer for my soon-to-be favorite cross-functional peer, a pharmacist. I wanted to make a tool for her to make her job less toilsome, a present.
I don’t remember if I used some hitherto undiscovered talent for negotiation in order to get the PM to let me do this or if no one noticed what I was doing, but I banged that tool out together with my user sitting beside me and it worked. Her day-to-day was significantly improved and our project picked up speed.
I did less and less of that kind of thing over time. Roadmaps became more structured with quarterly OKRs etc and all things internal tooling just always seemed to roll off the edge of the table. Product and technical investments took up all available bandwidth.
That wasn’t the end of opportunistic in-house dev, though. The company provided quarterly (!) hackathons for our engineering enjoyment but every now and again I would still spy the telltale signs of engineers sneaking off to work on little tools during the sprint. I was a manager at that point and I thought I had to come down on anything that took attention away from planned work but, in hindsight, I wish I hadn’t. Thwarting an engineer from doing what we both knew was right never seemed to end well. Worst case they became demotivated and their sprint work continued to suffer. Best case they just went and did it anyway.
Why we do it
So here’s the paradox: some of us run away from in-house development roles and yet, given the opportunity, will sneak right back to it whenever we’re given the slightest amount of slack (or not). Why?
I think part of it is that software engineers are offended by toil at a cellular level. We got into this field to automate and innovate. Watching a coworker drag two documents so that they are perfectly lined up, side by side, and then squint really hard in order to spot tiny differences in text dozens of times over the course of the afternoon is hugely upsetting. We know that it doesn’t have to be this way.
Another part of this is connection and alignment. Your engineer furtively writing tools for their coworkers is actually a great sign - it means that they are highly engaged with their peers and the problems those peers work on. You might not care much if a jerk from another team working on something you have no interest in struggles with a broken process. He can continue to suffer, that’s fine. My favorite pharmacist, though, who was on her way to a thicker pair of lenses churning through content for a project I cared deeply about? She deserved better.
Finally, consider this: If you’re quietly putting together an internal tool for your work buddy then you aren’t debating UX with your product designer, putting together an A/B test or taking some complex architecture proposal to team review so that they can tear it apart. You’re just sitting with your user while you write something small and powerful for them. For a lot of us that is the bee’s knees of software development.
Closing thoughts
Of course any behavior taken to an extreme can become problematic. If you are a manager and your engineer is doing this a lot, then it’s time for a bigger conversation. Otherwise, take a deep breath and ask yourself:
Does it accelerate the business?
Does it meaningfully reduce toil?
Is it something we’ve actually wanted to do for awhile now but couldn’t? And then finally:
How do I get this person in front of some freaking users?
Happy fiddling,
Nuts and Bolts.