.svg)
How the VP of Engineering at Glitch plans and prioritizes work for a 40 person software team without stifling creativity
As an engineering organization grows, one of the biggest challenges is managing and prioritizing the backlog. Planning is essential, as you need to agree on deadlines to align with internal and external stakeholders and to have a way to ensure that everyone is pulling in the right direction. But you don’t want to spend too much time planning, or you risk losing the insights that the development team can bring to proposing or prioritizing work.
Last month, I sat down with James Turnbull, the VP Engineering at Glitch. James has been running engineering teams for a while. Some of his previous roles include CTO at Microsoft for Startups, VP Services at Docker, CTO Kickstarter, VP Engineering at Venmo, and VP Technology at Puppet Labs!
At Glitch, he runs a team of around 40 product, engineering, and design professionals and reports directly to the COO. If you’ve not come across it, Glitch is both ‘a simple tool for creating web apps’ and ‘a friendly, creative community’ that allows everyone to create the web.
Planning the Work
Peter: At Glitch, the foundation of all planning is the company vision and values. I asked James to elaborate.
James: “We care very deeply about things like privacy and doing no harm. We’re a very community-oriented site, and we care very deeply about the community experience and the health and safety of users on the platform.”
Peter: At Glitch, the vision and values provide a shared understanding of building and prioritizing the backlog. There are multi-day quarterly roadmap planning sessions. But instead of focusing on specific features, the focus is much more thematic. Based on the company's strategic goals for the quarter, they map out the problems they’re trying to solve and the metrics that will indicate success - most often focused on either acquisition or engagement.
James: “We typically limit the team to 3-4 metrics a quarter so they can really move the needle as a relatively small team”.
Peter: From there, they come up with a list of high-level Stories, with cross-functional teams brainstorming the best ways to move the needle on the metrics that will support the company’s quarterly goals. Sometimes a new feature or experiment will be proposed by the CEO or an external stakeholder like Microsoft, Google, or Mozilla, but other times they’ll bubble up from the team based on their experience of building and operating the platform.
The power of focusing on metrics rather than Stories as a starting point is that it can provide alignment without stifling creativity. Instead of having the CEO tell everyone exactly what to code, they can identify the metrics that the company should focus on. Then the product and engineering teams can brainstorm potential experiments that they think will help to achieve those goals.
The entire team is involved in the quarterly planning process. They take some time to discuss the vision, values, and metrics, and then they propose a series of Epics that they think will best help the company to achieve its goals. After that, they decompose the Epics into 8-15 Stories with a short paragraph describing what each story is about.
James: “Let's say we're building a checkout feature. The Epic would be payments, and the Stories underneath it might be things like the back end integration with a payments provider, reporting, invoicing interface settings, a checkout page, and maybe some notifications, emails, and things like that.”
Working the plan
Peter: The teams run a “small a” agile process. From the Stories identified during the quarterly planning, the teams defer detailed planning until their sprint planning meetings just before each two-week sprint when they start to decompose the Stories into sub-Stories and then assign those out to members of the team, so they get increasing granularity as they get closer to work being done, rather than presuming that they have an understanding of what things should look like upfront.
James: “I'm a big believer in Minimum Viable Products. So, as fast as possible, I want to get to something that we can put in front of a customer and ask,’ Is this what you thought?’ And if it isn't, then we can iterate quickly or decide that we need to build something else.”
Breaking the Sprint
Peter: A common challenge for teams running fixed-length sprints is how to handle Stories that “break the sprint.” At Glitch, James has a few approaches to solving that problem.
James: “One approach is ‘can we draw a line underneath this?’ Can we say that this achieves 80% of our objectives and just move on, perhaps putting some new Stories in the backlog to remediate edge cases later? Another is just admitting that we got the estimate wrong and spilling over to the next sprint. It’s OK to keep working on something if it still has the potential to provide a good Return On Investment. The third (and least common) is to pull the plug if the story is way bigger than expected and doesn’t provide a good return based on the new estimate of the scope”.
On Estimates
Peter: Different companies have very different approaches to estimation - from avoiding it altogether to considering a project a failure if it doesn’t come in exactly on time and on budget. At Glitch, James takes a middle path. I asked him to explain the value of the practice for a developer that ‘doesn’t want to waste time on estimation.’
James: “So first of all, I think that's not realistic. Any engineer that thinks like that has two problems. One is that leadership can’t have any confidence in what they’re going to achieve. The second thing is that you can’t do any planning. If you say everything is entirely fluid and it'll happen when it happens, you can't run a business like that. What engineering teams need to remember - and this is sometimes a hard lesson for some teams - is that the reason we have a job is the business. The business folks are our customers. This is how we're going to grow. If the CFO said to the CEO, look, I'm not really sure about the budget; it's going to be as big as it's going to be - we’ll just wing it. Well, that’s not an acceptable answer. This is exactly the same thing. This is a quantifiable thing that the company is investing in. You have an obligation both as a responsible leader and as a good business person to treat it like a professional investment.”
Peter: He is, however, realistic about the limitations of estimates:
James: “I think the problem that you get with estimates is when people are dogmatic and forceful about them. When it stops being an estimate and starts being a deadline, it becomes a problem. If you say to somebody ‘you only have two weeks to finish this’ and they respond ‘well, it's not going to take two weeks. I think it's gonna take three’. If you still require them to finish in two weeks, you’re either asking them to work at 150%, or you’re being delusional about the amount of time that the work is going to take. Maybe it does take two weeks, but if it takes three and you have no ability to be flexible there, I don't think that's a very responsible way to run a team.
James: “I think if you accept that estimates can be off and you hopefully get better at estimates as you get more experienced, you can see that it’s an evolutionary process. If you say the first time we estimate we might be as much as 50% off, that would be perfectly okay as long as we learnt something from it, and the next time around, maybe it's only 25% off. I think that if you approach it like that, then the engineers know that you trust them, they know they have some agency in how they plan and do their work, and you can have an actual discussion. ‘Why do you think this is three weeks? Let's break this down; let's understand what's actually happening here. Oh, okay. Well, we actually don't need to do that piece. So if we don't do that, then maybe it is only two weeks.’ That is far better than creating some artificial constraint.”
Peter: Just as there are many opinions on estimation, there are many approaches to implementing it; Fibonacci story points, t-shirt sizes, a simple story count. At Glitch, the team uses t-shirt sizes.
James: “We generally use t-shirt sizes. We broadly classify Stories as ‘small,’ ‘medium,’ or ‘large,’ and we have a rough feel for how many of those Stories fit into a sprint depending on the number of people we have available and various other constraints.”
Managing the Backlog
Peter: I then asked James to give us a sense of how they keep track of the backlog and progress and how they manage it.
James: ”Glitch was part of Fog Creek, so originally we used a mixture of spreadsheets, Trello and ZenHub. Different teams did their own things, and it wasn't hugely successful - largely because it was very hard to work out what was happening across the organization.”
Often, one of the reasons people move to Shortcut is when they outgrow tools that aren’t optimized for rolling up metrics and providing visibility across multiple teams. Glitch was no exception.
James: “We were using Trello, and it’s an amazing tool. I still use it a lot to manage a bunch of things in my life. But Trello is a very broad church - It's a lot of things for a lot of people. A tool that you can plan your wedding or your apartment move with inherently has to be very flexible. Even though we're not particularly dogmatic about process, in agile, you still need guardrails. You still need some direction, and you also need some ability to track what's happening. For example, you really need the ability to audit the movement of cards. Also, you want to be able to define workflows. I want to be able to define how things progress through our process, and Trello is really not designed to do that. It's really designed to be a bit more free form, so we looked around at tools that actually allowed us to model our workflow.”
Peter: A lot of orgs move to JIRA, so why they didn’t just do that?
James: “Well, I didn’t want to inflict JIRA onto the team - I’m not a monster! Seriously, I'm not a huge fan of JIRA. I think JIRA is an amazingly powerful tool. But I think it has the opposite problem of Trello in the sense that it’s trying to be everything for everybody in the product management, product engineering, and process management space. It has to cater to teams of 10 people and teams of 10,000 people. As a result, it has every bell and whistle you can possibly imagine. Because of that, it’s a non-trivial exercise to set it up and to manage it. At Glitch, we just don't need that level of complexity. We're not a multi-divisional organization. I don't roll up thousands of engineers’ work. I don't need the ‘single pane of glass.’ So for me, it was a combination of massive overkill as well as the complexity that we didn’t need. We wanted something that provides guide rails, but not something that locks us in.”
James: ”Not long after I joined, we moved all of the teams to Shortcut. They published a couple of blog posts about how they internally use Shortcut, and we modeled our workflow in a similar way. We have a series of projects, and each team and each Epic is in Shortcut. We assign teams to Stories and have various states for the Stories - icebox, backlog, started/in development, ready for review, and shipped.”
James: "This is all really closely tied into GitHub using the built-in integrations, so a story gets moved into development when someone opens a PR, and a story is automatically pushed into shipped when they merge the PR, and the code deploys. We tie Shortcut into all of our other tooling as well. For example, the support team can create Stories in the backlog based on bugs, and we have an integration with Sentry for error reporting, so you can take an error report and automatically create a story from it. We also tie it into some other bits and pieces that allow me to take the reporting from Shortcut and turn it into a narrative about what happened in this sprint or this quarter.”
James: “At Glitch, we use Shortcut across the whole product and engineering org, so the product team uses it to track their progress, the solutions engineering team uses it to track their integration engineering projects, and we’ll probably end up pulling in the marketing team at some point as they’re a key stakeholder for us. They have lots of asks, and we do a lot of cross-functional work with them, so it’ll make a lot of sense to move them on to Shortcut eventually.”
On Reporting
Peter: I asked James to elaborate on what kind of reports he likes to see.
James: “I like the ability to use burn-down charts to see the overall progress of the organization against goals. I also like to be able to look at all of the Epics on a single screen to see their percentage completion. I typically then compare those reports sprint-on-sprint to see that we’re continuing to make good progress.
James: “I also look at deploy centric metrics. Number of deployments, number of failed deployments, things like that. I see them as a tangible representation of whether we’re successful in shipping. The goal is to deploy multiple times a day, and if we don’t ship anything for a day, I know something is probably very wrong. We also plan to rely more heavily on feature flags for bigger changes and for things like A/B testing so we can still integrate and deploy code regularly.”
Plug: Shortcut, a fast and enjoyable software project management tool. Now free forever up to 10 users.
Shortcut is the first project management platform for software development that breaks down silos and brings teams together to build better products. Shortcut is simple enough that anyone can use it and flexible enough so software teams can keep their processes lightweight and productive. Shortcut is fast to load, quick to configure and incredibly quick to navigate, all so we can get out of your way.
Zoom-in to pinpoint status updates on a single project, or easily zoom-out and filter across multiple teams so you have all of the information at your fingertips. Shortcut ties into your existing tools, services, and workflow. Get notifications or create a Story in Slack, attach commits, branches and PRs with Stories, build your own integration with our API, and more.