That time it was obvious that I didn't know what I was doing and my CTO had to give me The Talk
What my crummy first system design project taught me about decision making, insecurity, and assuming best intent.
I stink at systems design. I came to it late and have a gaping hole in my skillset that I never quite stitched up. Today’s story is about my first, belated systems design experience and how I almost allowed my gap and all the feelings it entailed to keep me from Getting Stuff Done (GSD) as a senior engineer up for promotion to staff.
Questionable
I remember my first performance review as a senior engineer at Exciting Scaleup. I had done well in my role but wasn’t quite ready for promotion. I had one glaring blank space in my review holding me back: proof that I had owned the technical design and execution for a system from start to finish. This wasn’t an accident of course - that blank space was present because there was nothing to put there. I had never designed anything bigger than a collection of classes, an API route, or a new table in an existing application.
I was a senior engineer on the verge of staff with precisely zero systems design experience.
How was I hired into this role without it, you ask?
I don’t know.
This made me feel pretty inadequate. A lot of things made me feel inadequate at Exciting Scaleup. Some of them were in my head but the others were real enough. I came from the banking industry which wasn’t exactly known as a hot bed of top engineering talent back then. I was also hired specifically to work on an absolute beast of a legacy application which had come over as part of an acquisition. It was made quite clear to me during the hiring process that I would not be considered for other roles.
More on that below:
I took the role because it was a great company. When my new coworkers discovered that I was working on this thing instead of the modern shiny stuff, though, the tone of conversation shifted. They didn’t like the beast one little bit. Being associated with it was bad news even when they knew that I hadn’t actually written any of it and was here to help it get better.
All of this was on my mind going into a challenge that I, according to the engineering ladder, should have tackled long ago.
Stalled Out
My project was immediately put on the schedule for our architecture wonks, engineers with design spikes who I found intimidating AF, to review. My manager at the time was one of those wonks, though, and that was great. He closely mentored me through the brainstorming and drafting phases. I didn’t absorb half of what he taught me but we landed on what felt like a solid plan. Then he went on parental leave and I had to defend that design to the other wonks.
The tippy top wonk didn’t like it and so we argued. By “we argued” I mean he argued and I floundered. I understood the technical constraints well and why we had gone with our chosen approach vs the alternative he was most interested in. Every time he pushed back, though, I would go all deer in the headlights. The second order questions whizzed past me at top speed.
It took a while for me to gain the deep context I needed in order to counter his points. This was probably very good for me - it forced me to learn a bunch of stuff and dig into details that I otherwise would have missed.
It didn’t work, though. He held on to his objections despite my now well-informed counterarguments.
The project was stalled. What could I do?
The Talk(s)
I ended up venting to my kind and extremely understanding CTO. I vaguely remember saying things like “I can’t do this”, “I’m not supposed to be here” and “you only hired me because I was willing to do stuff with Visual Basic”. There was probably also a fair bit of “why is he doing this to me” because that is certainly what I was feeling at the time.
This was the point at which I was given The Talk. Well, two talks:
The Imposter Syndrome talk. Didn’t work. Full points for effort.
The “It is your project, you own the technical decisions” talk. My reviewers would provide their input and feedback, but it was ultimately up to me to select a final proposal to put forward.
I huffed my way through the imposter syndrome bit. The second point landed, though. I took a deep breath and set up a final chat with Mr. Architecture Wonk.
I told him what I was going to do. I repeated the concerns he’d voiced and why I opted to go with my original proposal all the same. I braced myself and put on my Serious Engineer face.
Do you know what this guy did?
He smiled! He nodded! He requested that I please keep in mind one particular risk when implementing my design! He offered to help me if I got stuck while my manager was out!
The nerve!
Takeaways
Anywho.
These are some of the things I learned from my first, very uncomfortable system design challenge. You’ll note that none of them have anything to do with system design in the slightest:
It was my decision. I needed input from various folks who knew more than I did in their respective realms. I was the decision maker, though, and the project finally moved forward once I internalized that fact.
Decision makers do better when their orgs make roles clear. It’s hard to own the decision if everyone you talk to thinks that they get a veto. This is something Exciting Scaleup learned a little later and it made a big difference in getting things done.
Tell, don’t ask. Have you made your decision? Is it time to communicate that decision to the other person? Put together your argument, understand their concerns, think through how you will address those concerns, and then tell them what you are going to do. It will be ok.
Assume best intent. No one wanted me to be stuck or feel like crap! Mr. Architecture Wonk was actually invested in my success all along.
Finally, my insecurity around systems design tripped me up big time. I’d like to say that I figured that one out at the end of this story but I didn’t. I’d also like to say that I resolved to close my gap and caught up to where I thought I should have been.
This is what happened instead:
Very big thank you to Anton Zaides for providing feedback on a draft for this post. Please check out https://newsletter.manager.dev/ and consider subscribing if you haven't already!