This post was written by Mando Escamilla, Engineering Manager at Gradient Works, and the guy who runs our DevOps. In the post, Mando talks about what he's learned standing up the technology stack for a brand new company.
AUSTIN, SIX MONTHS INTO A GLOBAL PANDEMIC
Hayes:
Me:
The next thing I knew I was leaving a good job in the city to be the engineering manager for a 5-person startup (also in the city, I suppose).
Day 1
I didn't really know what I was walking into. I knew that Hayes and Chloe (co-founder and first engineer, respectively) already had some product and infrastructure in place but I didn't know what or how much. I log into Slack, do some HR-y stuff, dive into a product tour, then get my first task: go over our AWS spend and make sure we're spending money on the right things.
That's when I learned the first lesson:
1. Ain't no such thing as from scratch
Unless it's just you working on something, there's always gonna be something already in place. And that's great - staring at that blank screen is intimidating as heck, which is probably why it's taken me so long to write this post. In our case, we had a smattering of AWS infrastructure up and running and that was great too - I've got some pretty decent AWS experience under my belt. That's a corollary to this lesson: build with stuff you know.
There's a lot that's been said about choosing boring technologies and it's nuanced but there's a nugget of truth and wisdom at the core. If you are spending more time wrestling with unfamiliar tech than building your product or infrastructure, maybe it's time to reconsider your path, especially because of lesson #2:
2. You've got less than you think
Especially if you're coming from a mid-to-large company, you're gonna realize how little technical scaffolding you actually have (and maybe how much you took for granted at your last place).
You need to stand up your java service? How do you intend to do that - there's no kubernetes cluster for you, no config management system to copy from to get your EC2 instance running, no nothing. And once you decide on how you're gonna stand that service up, there's no reporting service that alerts you when your users hit an error and no monitoring service to warn you if the service is down. Are your customers actually using your service? There's no metrics service to tell you!
You gotta build all that yourself, friend.
That's lesson #3:
3. You're the expert
There's no architectural review board, no group of engineers or engineering leaders that you can take a proposal to and get their opinions on. You gotta make a decision and execute it.
I'm really REALLY lucky - I've got this amazing group of friends that are absolutely brilliant engineers and they still answer my texts when I ask them yet another question about kubernetes vs fargate or whatever. I hope you have something like that. If you don't, send me an email and we can be friends (the answer is kubernetes but sometimes fargate).
Being the expert doesn't mean you have to know everything. Usually it means that you're pretty sure that your decision is the best wrong decision at the time and that it probably won't make you hate yourself for a while. It also means that you're always thinking about lesson #4:
4. Time and cost are your constant companions
Every decision you make comes down to time and cost.
Do you need Sentry and Datadog and Pagerduty and a kube cluster and a log aggregation service and on and on and on? Probably?
Do you need them right now? Maybe not. Maybe a couple of cloudwatch alarms that ping you in Slack is good enough for now. It's tough to know for sure what the right approach is, but if you can say "yes!" to the following then you're in good enough shape:
- Will I get alerted if our customers can't use the product?
- Can I see that our customers are using the product?
- Can I see how much our product costs to run?
After that, you can focus on lesson #5:
5. Invest in things that help you go fast
There's a whole class of stuff that at first glance might seem like it's not worth the time or cost but will, in my experience, pay off big.
This is stuff like CI/CD. At the very least, get the CI part down cold and automate your deployments. If deploying means running a script, you're right around the corner from CD. We use Github Actions for our CI/CD and it's been great. No offense, but I don't miss running a Jenkins cluster.
This is also stuff like self-documenting utility scripts that automate common things. make test may not seem like a big improvement over gradlew test, but when every project has a standard Makefile things can move faster.
The easier you can make things on yourself and the team is crucial because of lesson #6:
6. Work less than you think you should
Early startup life can be rough, even when you're not in the middle of a global pandemic or a crucially important election cycle. There's a lot riding on you and the decisions that you make. It can feel like you're the only person that can do X or build Y or finish Z.
That's a lot of pressure.
When that pressure builds, it can be tempting to do things like work through lunch or dinner. It can be tempting to open up that laptop on a lazy Saturday or Sunday. It's especially easy to fall into this trap now since we're all still trying to stay home and get over the vaccine finish line.
"I'm just gonna finish this one little thing", you'll say. That'll quickly turn into "Oh I gotta finish up this security review" and then it's "Aw beans I forgot about this JIRA" and the next thing you know your kids have built a fort out of all the leftover pizza boxes and when you try to tell them to get rid of it they call you "The Usurper" and now you're afraid to go to sleep.
Early startup life is rough but don't make it even rougher.
In conclusion, America is a land of contrasts
It's been a bit over 4 months since I started at Gradient Works. I'm sitting here, looking at my notes, trying to piece together a story of the work we've done and, well, it's A LOT.
Just believe me. I don't want to have to kill you.
It's been great and it's been hard and I love it. I get to work with people that I love and adore and who inspire me every day to do my best, even on the days when my best maybe isn't so great. I'm getting better at things that I thought I already knew and I'm learning new stuff every day. I'm excited to open up that laptop every morning and that's something I haven't always been able to say.
We're trying very hard to make Gradient Works a startup for grownups and I think we're doing a pretty ok job.
May we all be so lucky.
