Why the Agile Coach Is Not a Luxury in a Software Team

When a company has to reduce its costs, the agile coach is very often one of the first positions that is removed, and the reasoning usually sounds sensible: we already have a tech lead, we have a CTO, and the developers know what a sprint is, so why do we pay someone whose job is to run meetings? I have worked as a developer in teams for many years, and in my view this decision is one of the most expensive "savings" a software company can make. In this article I want to explain why the work of an agile coach can't simply be handed to the tech lead, why agile is about results and not about comfort, and why the ceremonies that many developers see as a waste of time are actually part of what keeps a team together.

The tech lead can't be the coach of their own team

The first argument I hear is that a good tech lead or a good CTO can do this job as well, because they know the team, they know the product, and they are already in every meeting. Although this is partly true, but it misses the most important point, which is that the tech lead is part of the system that the coach has to look at.

I'm not the first person to say that even very capable leaders need someone outside their own role to look at them. Eric Schmidt has told many times that when he was CEO of Google, board member John Doerr told him he needed a coach, and when Schmidt answered that he didn't, Doerr's reply was "No, no, everybody needs a coach." Schmidt's explanation of why it worked is, in my view, the best short description of the coach's job: "The one thing that people are never good at is seeing themselves as others see them." The coach Doerr brought in was Bill Campbell, who also worked with Steve Jobs and Larry Page, and Schmidt, Jonathan Rosenberg and Alan Eagle later wrote a whole book about his way of working, Trillion Dollar Coach [1][2]. If the CEO of Google needed someone outside himself to see what he couldn't, it's difficult to argue that a tech lead with a full sprint of work doesn't.

A tech lead is responsible for technical decisions, for the architecture, for code review and very often for delivery dates. That's a lot of responsibility, and it shapes how they see the team. When a sprint goes badly, the tech lead naturally looks for the technical cause: the estimate was wrong, the API was not ready, the tests were missing. Sometimes that is the real cause, but sometimes the real cause is how the team is being led, and it's very difficult for anyone to see that clearly when they are the one who is leading.

Additionally, there is the question of who people talk to. A developer who feels that every decision is taken above their head, or that the workload is not fair, will usually not say this openly to the same person who reviews their pull requests and has an opinion in their performance review. They will say it to someone who has no technical authority over them and whose job is specifically to listen. That's because honest feedback needs a safe channel, and the tech lead, whatever good intentions they have, is not a neutral channel.

The Scrum Guide itself is very clear about this. It doesn't describe the Scrum Master as a meeting organiser but says that "the Scrum Master is accountable for the Scrum Team's effectiveness", and it lists coaching the team members, causing the removal of impediments and "leading, training, and coaching the organization in its Scrum adoption" among the responsibilities of that role [3]. That is a full job, and it's a different job from designing the system and reviewing the code.

Two views that a technical leader doesn't have by default

In my view the agile coach brings two ways of looking at the team which are very different from the technical view.

The first one is the agile view: how the work flows through the team, where it gets stuck, whether the team really plans together or only receives a list of tasks, whether the sprint goal means something or is just a label on the board. A coach notices when the process has slowly turned into something that has the names of Scrum but none of its purpose.

The second one is close to the human resources view: how people feel in the team, who has stopped speaking in meetings, where there is a conflict that nobody wants to touch, and whether the way the team is managed is building trust or slowly destroying it. Google's research on team effectiveness, known as Project Aristotle, found that psychological safety was the most important of the five dynamics they identified in effective teams, ahead of dependability, structure and clarity, meaning and impact [4]. The idea itself comes from Amy Edmondson at Harvard, whose 1999 study of 51 work teams defined psychological safety as a shared belief that the team is safe for interpersonal risk taking, and found that it was connected to how much a team actually learns [5]. Psychological safety is exactly the thing that a busy technical leader has the least time to look after.

This is where the coach becomes really valuable. When the tech lead starts to manage the team in a wrong direction, for example by taking every decision alone, by overloading the sprint to satisfy a stakeholder, or by solving every problem personally instead of letting the team grow, the coach is the person who can see it early and help the lead to correct it. This matters more than it sounds, because the way a team is led has a very big effect on it. Gallup, analysing engagement data from millions of employees, found that the manager accounts for about 70% of the variance in team engagement [6]. When the leadership of a team goes in a wrong direction, the whole team follows it, and someone should be watching that direction. This is not a fight between the coach and the lead. A good coach works with the tech lead, and in many cases the tech lead is the person who benefits most from having one. Google's own research on managers, Project Oxygen, put "is a good coach" first in its list of behaviours of good managers [7], and I think that is exactly the point: coaching is so important that it needs someone who has time for it, and a tech lead who also owns the architecture and the delivery dates rarely has that time.

Agile is about performance, not about less pressure

There is a common idea, also among managers, that agile is mainly about making developers comfortable: less pressure, fewer deadlines, more freedom. With this idea in mind, the agile coach looks like a nice-to-have position, something a company can afford when things are good and remove when things are hard.

I think this idea is wrong. Agile methodology came from a very practical problem, which was that long plans made at the start of a project were wrong by the time the software was delivered. Short iterations, regular planning and regular inspection exist so that the team finds its mistakes early, while they are still cheap to fix. That is about results. A team that plans honestly, knows its real capacity and fixes its own problems every few weeks delivers more and delivers more predictably than a team that is simply pushed harder.

It's also worth looking at which companies have treated this role as a standard part of the team and not as a decoration. When Henrik Kniberg and Anders Ivarsson described how Spotify organised its engineering in 2012, every squad, which was the small autonomous team at the centre of that model, had access to an agile coach who helped it to improve the way it worked [8]. That model was later studied and copied by many large organisations, and it came from a company whose main problem was moving fast, not keeping developers comfortable.

So when we talk about the agile coach, we are talking about someone whose work is directly connected to the performance of the team and the results the team produces. Whether we like it or not, the cost of a team that works badly together is much higher than the salary of the person who helps it to work well, and that cost is mostly invisible because it appears as delays, rework and people leaving, not as a line in the budget.

Remote teams need it even more

All of this becomes more important when the team works remotely. In an office, a lot of the coordination happens without anyone planning it: people hear each other's conversations, they notice when a colleague is frustrated, and small problems are solved at the coffee machine before they become big ones.

In a remote team almost none of that happens by itself. A developer can be blocked for two days and nobody notices, a disagreement can stay in a private chat and grow, and a new team member can feel completely alone without anyone realising it. The only moments in which the whole team really meets are the ones that are planned, which means the quality of those moments decides a large part of the quality of the collaboration. Someone has to take care of them deliberately, and that is a very big part of what an agile coach does in a distributed team.

This is not only a feeling. Researchers at Microsoft studied the emails, calendars, chats and calls of 61,182 Microsoft employees in the United States during the first half of 2020, when the whole company moved to remote work. They found that the collaboration network became more static and siloed, that the share of time people spent with colleagues in other groups dropped by about a quarter, and that communication moved from synchronous to asynchronous [9]. In other words, when nobody takes care of the connections inside and around a team, they get weaker.

The ceremonies are the glue of the team

This brings me to the meetings themselves. Sprint planning, the daily stand-up, the review and the retrospective are often described by developers as time that is taken away from "real work", and I understand this feeling, because a badly run meeting really is a waste of time. But the answer to a badly run meeting is to run it better, not to remove it.

Each of these events has a specific job. Sprint planning is where the team agrees together what it is going to do and why, so that the work belongs to the team and not only to the person who wrote the tickets. The daily stand-up, which the Scrum Guide limits to fifteen minutes, is where blockers become visible while they are still small. The review is where the team and the stakeholders look at real software instead of status reports. And the retrospective is the moment in which the team looks at itself and decides what to change. The Scrum Guide says its purpose is "to plan ways to increase quality and effectiveness" [3], and this idea is older than Scrum's popularity: one of the twelve principles behind the Agile Manifesto is that "at regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly" [10].

The retrospective is usually the first one to go, because it doesn't produce anything that can be shown on a board and because it's honestly not easy to run well. Removing it feels like a small saving in the first weeks, and nothing seems to break. In my experience, over a longer period it has very negative effects on the environment of the team, on collaboration and on teamwork, because the team loses the one place where problems are allowed to be said out loud. Norm Kerth, who wrote one of the first books about retrospectives, suggested opening every one of them with what he called the Prime Directive: "Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand." [11] A sentence like that only works when someone is responsible for making it true in the room, and that is a facilitation skill, not a technical one. The problems don't disappear when the meeting disappears; they just move into private conversations, into frustration, and eventually into resignations.

In short, these ceremonies are part of how a team is built and maintained. They are the glue that holds a group of individual developers together as a team, and keeping that glue in good condition is a real job, which needs time, skill and a person who is responsible for it.

What I would suggest to a company that is thinking about this role

If a company has decided that it can't afford a full time agile coach for every team, I understand that, and there are reasonable middle ways: one coach shared between two or three teams, or a coach who works intensively with a new or struggling team for a few months. What I would not suggest is to give the role to the tech lead "on top of" their current job and expect the same result, because, as I explained above, the tech lead is inside the problem that the coach has to see from outside.

And if your team has already dropped the retrospective because it felt like a waste of time, it may be worth bringing it back for a few sprints, run by someone who is not the tech lead, and seeing what the team says when it finally has a place to say it.

References

1. Eric Schmidt on the best advice he received, quoted in "Everybody needs a coach", Daily FT: https://www.ft.lk/Management/Everybody-needs-a-coach/53-706873
2. Eric Schmidt, Jonathan Rosenberg and Alan Eagle, Trillion Dollar Coach: The Leadership Playbook of Silicon Valley's Bill Campbell, Harper Business, 2019.
3. Ken Schwaber and Jeff Sutherland, The Scrum Guide, November 2020: https://scrumguides.org/scrum-guide.html
4. Google re:Work, "Guide: Understand team effectiveness" (Project Aristotle): https://rework.withgoogle.com/intl/en/guides/understand-team-effectiveness
5. Amy Edmondson, "Psychological Safety and Learning Behavior in Work Teams", Administrative Science Quarterly 44(2), 1999, pp. 350 to 383.
6. Gallup, "Managers Account for 70% of Variance in Employee Engagement": https://news.gallup.com/businessjournal/182792/managers-account-variance-engagement.aspx
7. Google re:Work, Project Oxygen, the behaviours of great managers.
8. Henrik Kniberg and Anders Ivarsson, "Scaling Agile @ Spotify with Tribes, Squads, Chapters and Guilds", 2012.
9. Longqi Yang, David Holtz et al., "The effects of remote work on collaboration among information workers", Nature Human Behaviour, 2021: https://www.microsoft.com/en-us/research/?p=772330
10. "Principles behind the Agile Manifesto": https://agilemanifesto.org/principles.html
11. Norm Kerth, Project Retrospectives: A Handbook for Team Reviews, Dorset House, 2001.