SD/TL: Scalability
Will a burst be celebrated or dreaded?
This is a post in my “system design applied to tech leadership” series.
In this series I compare elements of tech leadership with elements of system design.
*
Is it scalable?
In both system design and team leadership, a leader benefits from understanding scalability needs ahead of consequences.
The classic system design example is a system scaled for N orders experiences N^2 orders on Black Friday in the US.
The equivalent in team leadership is the question of having support ready for when a new major feature goes live or for when a major customer is added to the platform: all the leadup work will mean little if your customers aren’t supported adequately during the change— and that means having scalable teams to match the new demands.
In system design, we have tools for addressing scalability:
serverless compute [though be prepared to scale your credit card!]
load balancers
autoscaling for resources
queues
observability and notifications
even caches, such as CDNs, can assist with scaling across countries and within data domains themselves
SLAs referee whether a system delay or failure is within or outside of business expectations
But teams? What are the equivalents?
A good ticketing system and dashboards with key metrics allows for load balancing, queuing, and observability
Having excess capacity, or the ability to pull in capacity, with people who can support the system represents the ability to autoscale, even serverless compute. These are supported by playbooks, SOPs, and support tiers.
Knowledgebases handle caching.
SLAs referee whether the delays in response are within or are outside of contractual obligations
Is scalability needed?
In system design and in team leadership, designing for scale which is not needed is an anti-pattern. Part of owning a system or of leading a team means being able to anticipate the amount of traffic that system or that team will encounter. Based on my experience, this is even more difficult challenge than scaling the system or the team.
And part of the difficulty of the challenge is deciding to focus on it. I’ve seen seasoned architects fail when they spend weeks on a design for which the demand can be overturned in a two-minute question: “How many users, max, will use the system?”
I’ve also seen cases where team leads and architects have done the opposite: an elegant org chart or architectural diagram which works brilliantly for the current scope, but is blind to known scope changes months from now.
An answer to this is to include scale and scope statements in every team structure document and in every architecture diagram. Not being able to produce this means you have a good challenge ahead of you, and one worth taking on.
When does it need to scale?
But scaling today for demand two years from now is not necessarily the answer either. The team lead or architect benefits from developing a phased scaling plan, not one based on date-based milestones but one triggered by events.
Example: Not “we’ll bring on the extra capacity in six months”, but “we’ll start to add capacity when the feature reaches system-integration-testing” or “we’ll add capacity when we get a signed contract from the new customer.”
Scalability assessments
I recommend answering these 3 questions whenever changes to features, customers, or types of demand are being pitched relating to the team or to the system.
They make a quick, high-level playbook for making sure your teams and systems keep up with demand over time.