What is technical debt?

This article is my own synthesis of ideas presented at Lean Agile Scotland in September 2014 by Dan North, Jabe Bloom and Torbjörn Gyllebring, with some of my own embellishments. Any misunderstanding of their points is mine! I’ve linked to their original talks at the foot of this post.

Conway’s Law says that the design of our system will be rooted in the design of our organisation.

So, we can never design a system from scratch: some of the design will have already started before we start designing.

This isn’t necessarily a problem, however: indeed, we should embrace it, because organisations suck if they don’t have the right APIs and other architectural borders in place between where teams are (that’s when teams start to get bureaucratic layers preventing their work from proceeding as their pace.) We should take time to get this right, and this is why having micro-services to do communication between teams is a good idea.

However much we work on this, however, we will face problems in the code we write, many of which will relate to the architecture. We can either chose to fix the symptom (via a quick fix) or to fix the underlying problem (which is generally more time-consuming). Quick fixes aren’t necessarily bad when applied in conjunction with a longer-term fix but remember they are akin to painkillers, and you can’t take painkillers indefinitely.

Quick fixes often take the form of a mandated structure: we might add too much or too little process, a particular build system, an architectural design ‘recommendation’, a decision on a particular source control system or IDE, or similar. As managers, we should be especially careful with such quick fixes. We are aiming to apply a ‘Goldilocks’ amount of constraint on our teams: too constrained and development will stop because the people can’t do their job; too little constraint and there will be too many options to do the right thing. The right amount is a strategy that’s clear enough to give us a task for today and a constrained set of options for the future. That strategy is actually a conversation – top-down constraints from management, and bottom-up information and designs from the development teams.

Partly because of the difficulty of identifying the ‘Goldilocks amount’ without information from the team (which presupposes time taken to do investigation), quick fixes generally tend to exhibit more unintended consequences than fixes that correct the underlying problem.

In a Cynefin sense, rather than taking time to analyse a problem as demanded by Complicated scenarios, our quick fix has made matters worse by creating a Complex relationship. The side effects can’t be foreseen, but become apparent in retrospect. Worse still, the quick fix may fail to address the root cause of the problem, and so we may get a positive feedback loop causing problems to increase as a result of our solutions.

This is technical debt, because the system starts constraining what we can do.

Cynefin tells us we should monitor for this positive feedback and, when we sense it, take time to respond to it, possibly by rolling back the change and trying a different approach.

To check for technical debt, therefore, we should periodically take time to ensure that our system isn’t constraining us. We can do this, for example, by ensuring our tools haven’t become our masters by checking that we can still build our products using a single command line (without complex CI systems) and that refactoring can be done in Notepad (without Resharper).

Original videos:

Understanding a feature through 3-amigos

The last project I ran here at Red Gate was on SQL Compare / Data Compare 11 project. SQL Compare is a superficially simple product designed, as it’s name suggests, to compare Microsoft SQL database schemas. Its cousin, SQL Data Compare compares the data in the tables.

Like the product itself, the Compare 11 project had a superficially simple goal: add support for the new features in SQL Server 2014. Easy, right? It turns out, not so much…

The largest new feature is Memory-Optimized Tables (MOTs): the ability to keep data in memory so you get significant performance improvements through reduced disk-access. But as you dig into what MOTs can and can’t do, you realise that you’ll need a lot of special-cases in the code to deal with the limitations. The details of those limitations is well outside the scope of this post, but one of my colleagues Michelle Taylor has written a useful post on our company blog for those who want to know more.

When the team came to start work on this story, which was before the release of SQL Server, some of the limitations weren’t immediately apparent. This led me (as project manager) to a false sense of security, and so I was far too lax on getting the user stories and acceptance criteria defined in detail. The lack of knowledge this created in the team caused the feature to become massively under-estimated: just 50 task-points (at their velocity, each task-point turned out to be just over an hour of real work).

The first tasks related to this story went on the Scrum board early in the project: ‘Add property to CreateOrAlterTable’, ‘Add property to DatabaseObjectsTable’, ‘Update DatabaseObjectsTable.Equals()’. You see what’s missing? We had dived straight into implementation with no form of planning first. That first sprint was estimated as 15 points. By the time we got to the end of it, we had uncovered quite how unexpectedly complex this feature was likely to become. We were nowhere near 30% (15 out of 50 points) complete on the feature. We carried some of the tasks forward to the following Sprint, however, trying to convince ourselves that we’d crack it soon. Before I knew it, we’d spent nearly a month on this feature, continually uncovering more and more complexity, and morale on the team was starting to become impacted.

Even then, my monthly project A3 report excessively optimistically promised that we’re complete both Inline Indexes (another 2014 feature) and Memory-Optimised Tables in the next four weeks.

It was time to pull the Andon cord.

Luckily, this also coincided with a slight change to business focus. The team was being asked to ship SQL Compare 10.7, giving support for connecting to SQL Server 2014 (but not the new features) to ship with the final release of SQL Server 2014.

Whilst this was a massive disruption to the team, it did give us some breathing space to find out more about MOTs before we had to work on them again.

By the time we were ready to start work on MOTs again, the Product Owner had defined the Acceptance Criteria in more detail. There were a series of tasks, themselves totalling 42 task-points in estimates, which centred on reviewing these ACs and making sure they made sense. This worked OK, and put us in a much better place, but with a large team, the meetings didn’t really work.

At this point, one of my colleagues pointed me to the idea of a 3-amigos meeting. I’d actually tried this once before on a different project, and I found myself wondering why I hadn’t tried this earlier on the MOTs work. The idea is to have three people (specifically the Product Owner, a Developer, and a Tester) in a room together, defining how the feature should work for users, in the code, and in test cases respectively. There’s a good write up of the meeting on the Scrum Alliance website; the only difference for us being that the Product Owner also doubled in the BA role for us. Although it felt slightly more waterfall than we were used to, the output from the meeting was a set of really well defined, clear Acceptance Criteria and a good common understanding of what MOTs involved, that was ready for sharing across the team.

The team re-estimated the work, and came up with an estimate of 250 points. In the end, it took 337 – still rather more than 250, but also far more than the original 50 points planned!

This process worked so well that we adopted the same meeting for other features in SQL Server 2014, rotating the role of the amigos through the dev team. As long as the communication about what was discussed is good immediately after the meeting, everyone became brought into the idea of 3-amigos, and ensured that the chaotic start to MOTs wasn’t repeated during the rest of the project.