Category Archives: Red Gate

Agile, Project Management, Red Gate

Direction, momentum and oil: Why software development teams are like bikes

Leave a comment

I haven’t posted for a while on what I’ve been up to at work and there’s a good reason for that: the team I’ve been working with was pretty dysfunctional twelve months ago and I’ve been spending the last year trying to turn them around. In this post, I’ll describe that journey and why I’ve concluded that software development teams are like bicycles.

Context

I was first asked to spend a couple weeks in April 2015 working with Redgate’s SQL Monitor team as an external observer, finding out why their Project Manager was struggling and why the team was perceived as delivering slowly. In this environment, some of the team were even starting to leave the company…

A bit of background: the SQL Monitor project had been stopped for just over a year after the release of version 4 in January 2014. In the intervening time, no work had happened on the product, and the original team had disbanded. Many had gone off to do other things. In the meantime, revenue had continued to increase and so, in early 2015, the company decided to form a new team to continue that growth. The new ‘team’ (a mix of different people that had been cobbled together, most of whom had no experience on SQL Monitor) would work on one feature in order to get used to the codebase, while the company decided on what the further roadmap looked like. That didn’t sound like a team set up for success.

In my favour was the fact that the team knew they had a problem and so were prepared to work with me to address it. We didn’t have much time, though: the following week, two newly-hired developers were set to join them and we needed to make their introduction as good as possible. So, the usual advice that a Project Manager / Agile Coach should just start by observing the team went out the window. I had to act fast.

I started by organising a retrospective with the team to investigate their various problems. This was a fairly painful session, which I facilitated, using a fishbone diagram technique:

Fishbone diagram

As expected, there were a number of really fundamental issues here, mainly with the state of the code, but feedback was that it was quite cathartic for the team to see them written down in one place.

To summarize, this was a pretty bad situation: the team had no goals, no deadlines and were struggling to do valuable work. In other words, they had no direction or momentum and the bicycle chain wasn’t even allowing the different parts to work in concert.

Giving a direction

During the same week as that retrospective, I sat down with the Product Manager and got his opinion on commercial direction. It turned out he had a presentation, that had already been presented to other managers, with his idea of a roadmap. It was a well-thought through plan; it’s just that the dev team hadn’t yet seen it. That was another easy fix – I got the Product Manager to give the presentation to the developers and let them challenge the rationale. With this approved, he could then go away to create a 5-year roadmap – rather unusual at Redgate but vital for giving the developers a sense of confidence that the project wasn’t about to be stopped for another year again.

The outcome of the Product Manager’s work was that there were two obvious big features to be done soon – support for Active Directory and support for SQL Server Availability Groups. Both of these were features which competitors had implemented already, and were actively costing us sales. The work felt likely to take a few months, it was mid-April 2015 and there was a big tradeshow, SQL PASS 2015, coming up in October. The way forward seemed clear. I’d re-focus the whole team on those two features and temporarily shut down the other distractions the small team was simultaneously trying to work on (mainly support escalations and code quality improvements). Stopping distractions, and especially support escalations, isn’t easy, and can’t be a long-term solution, but in some cases I’ve found it to be a very useful tool. If you have the right conversations, even the support personnel in the firing line from customers can be brought into the idea.

Some people may question the idea of stopping the work to address code quality but the fact is that random pieces of improvement work were being picked up by developers with little planning or sense of what the most important work to do was. Instead, I asked the team to just write down the known technical debt on Post-Its as they were discovered, and stick them to the wall. Together with the actions from the fishbone retrospective, that became a visible reminder of how much debt needed paying back and, in due course, would form a backlog of improvement work to be addressed.

The tech debt wall

Of course, things never are straight-forward. Just as I was starting to get this running, a critical security vulnerability was found in SQL Monitor 4.1. The plan had to be put on hold while SQL Monitor 4.2 was released, and the fix was also backported to previous versions of SQL Monitor.

Explicitly showing what work is prioritised and what is to be done later on

 There – my job was done. The team had a road to follow and, a month or so after joining them, I could leave them in the hands of their normal Project Manager to continue the work…

Speeding up

At the end of July, I returned from a holiday to discover that the existing Project Manager had decided that he no longer felt confident in managing the project. It was a very brave decision to make, and meant that I was catapulted back into the SQL Monitor team again – this time as full-time Project Manager.

My job, though, was fairly obvious by now: keeping the team focused on delivering in time for SQL PASS 2015. This mainly consisted in keeping a fairly tight reign on the team’s approach to sprint planning and capacity planning, at least at first. This ensured also that there was no time available for some of the quality improvement work the team wanted to do, a deliberate choice by me to keep the sense of purpose. Over time, we found a way to handle support escalations once again, and I eventually released the brakes I’d artificially applied to guard against scope creep/changes.

To bring the team back on track, we enforced user stories, estimation and acceptance criteria.

We did it – SQL Monitor 5.0 was released just before the tradeshow, on time and without significant amounts of cutting things from the planned scope.

Through the work on version 5.0 we’d deliberately not released frequently: releasing was hard and required extensive manual testing because the automated tests were unreliable or covered the wrong things. Given the length of time since the last major release, and the team’s lack of knowledge, trying to release regularly could have caused too many derailments.

So, having reached SQL PASS 2015 intact, it was time to set the next challenge for the team: work out how to release frequently. For the major release, the testers had done an extensive risk-based analysis on the major areas of the product they’d need to test. They now used that as a first step to working out what minimal testing would be required. Meanwhile, the developers set to work getting rid of those unreliable automated tests, replacing them where possible, and changing our branching strategy so we’d always be in a position to release.

And that backlog of improvements to the code we needed to make? I got buy-in from the stakeholders so that we could keep our promise. We gave the team two months, without pressure to deliver features, to start working their way through the backlog.

By the start of 2016, we could release every sprint and some of the most egregious code problems were gone. The bike had gained its own momentum.

Providing the oil

Naturally, it hasn’t all been completely plain-sailing. The release notes from the start of 2016 show that there were several bugs and regressions that we introduced as we moved to the new model. For the customers who were affected, we’re sorry. But the fact that the SQL Monitor code, while still far from perfect, is improved, better understood by the team and able to be released quickly means that we’re in a much better position to do some great work in 2016 than we were last year.

By now it should be clear what my role in all this was. The team is pretty much unchanged. The way they work is only slightly different. The developers, testers and designers are still the gears, chain and wheels of the bike. Without them, we wouldn’t have a bike at all. My job was one of lubrication: providing the oil and a bit of direction and momentum to ensure they could go on their journey. It was done through adding brakes and stabilisers where necessary to prevent them from going too far off-course in the early days, then empowering the individuals by giving them the time and space to address the problems they wanted to fix and by protecting them from the mud of management that might have interfered with the way they were working.

Postscript

The team’s now working on some really exciting stuff ahead of SQL PASS 2016, including a very impressive new dashboard which is nearing completion.

As for myself, my mentor, Simon Cromarty once told me that the aim of any project manager should be to make themselves redundant: You succeed when the team doesn’t need you any more. So, in a rather perverse way, it should be good news that I stand to be made redundant from Redgate in the coming weeks. If you need someone to help oil your bicycle,  let me know…

Agile, Project Management, Red Gate

Understanding a feature through 3-amigos

Leave a comment

The last project I ran here at Red Gate was on SQL Compare / Data Compare 11 project. SQL Compare is a superficially simple product designed, as it’s name suggests, to compare Microsoft SQL database schemas. Its cousin, SQL Data Compare compares the data in the tables.

Like the product itself, the Compare 11 project had a superficially simple goal: add support for the new features in SQL Server 2014. Easy, right? It turns out, not so much…

The largest new feature is Memory-Optimized Tables (MOTs): the ability to keep data in memory so you get significant performance improvements through reduced disk-access. But as you dig into what MOTs can and can’t do, you realise that you’ll need a lot of special-cases in the code to deal with the limitations. The details of those limitations is well outside the scope of this post, but one of my colleagues Michelle Taylor has written a useful post on our company blog for those who want to know more.

When the team came to start work on this story, which was before the release of SQL Server, some of the limitations weren’t immediately apparent. This led me (as project manager) to a false sense of security, and so I was far too lax on getting the user stories and acceptance criteria defined in detail. The lack of knowledge this created in the team caused the feature to become massively under-estimated: just 50 task-points (at their velocity, each task-point turned out to be just over an hour of real work).

The first tasks related to this story went on the Scrum board early in the project: ‘Add property to CreateOrAlterTable’, ‘Add property to DatabaseObjectsTable’, ‘Update DatabaseObjectsTable.Equals()’. You see what’s missing? We had dived straight into implementation with no form of planning first. That first sprint was estimated as 15 points. By the time we got to the end of it, we had uncovered quite how unexpectedly complex this feature was likely to become. We were nowhere near 30% (15 out of 50 points) complete on the feature. We carried some of the tasks forward to the following Sprint, however, trying to convince ourselves that we’d crack it soon. Before I knew it, we’d spent nearly a month on this feature, continually uncovering more and more complexity, and morale on the team was starting to become impacted.

Even then, my monthly project A3 report excessively optimistically promised that we’re complete both Inline Indexes (another 2014 feature) and Memory-Optimised Tables in the next four weeks.

It was time to pull the Andon cord.

Luckily, this also coincided with a slight change to business focus. The team was being asked to ship SQL Compare 10.7, giving support for connecting to SQL Server 2014 (but not the new features) to ship with the final release of SQL Server 2014.

Whilst this was a massive disruption to the team, it did give us some breathing space to find out more about MOTs before we had to work on them again.

By the time we were ready to start work on MOTs again, the Product Owner had defined the Acceptance Criteria in more detail. There were a series of tasks, themselves totalling 42 task-points in estimates, which centred on reviewing these ACs and making sure they made sense. This worked OK, and put us in a much better place, but with a large team, the meetings didn’t really work.

At this point, one of my colleagues pointed me to the idea of a 3-amigos meeting. I’d actually tried this once before on a different project, and I found myself wondering why I hadn’t tried this earlier on the MOTs work. The idea is to have three people (specifically the Product Owner, a Developer, and a Tester) in a room together, defining how the feature should work for users, in the code, and in test cases respectively. There’s a good write up of the meeting on the Scrum Alliance website; the only difference for us being that the Product Owner also doubled in the BA role for us. Although it felt slightly more waterfall than we were used to, the output from the meeting was a set of really well defined, clear Acceptance Criteria and a good common understanding of what MOTs involved, that was ready for sharing across the team.

The team re-estimated the work, and came up with an estimate of 250 points. In the end, it took 337 – still rather more than 250, but also far more than the original 50 points planned!

This process worked so well that we adopted the same meeting for other features in SQL Server 2014, rotating the role of the amigos through the dev team. As long as the communication about what was discussed is good immediately after the meeting, everyone became brought into the idea of 3-amigos, and ensured that the chaotic start to MOTs wasn’t repeated during the rest of the project.

Red Gate

Hacking the office

1 Comment

(or what happens if you mash-up a Software-Defined Radio, a radio microphone, a Belkin Wemo, a fridge, Yammer and the Microsoft Exchange Server API)

Every few months at work (Red Gate), we have a ‘Down Tools Week’. This is a chance to divert ourselves from normal product ideas and to look instead at other things we could do and problems we could address. Our Down Tools Weeks generally have (optional) themes, like creating internal tools or coming up with product ideas. In this way, the Down Tools Week in March 2014 has been focused on ‘The Internet of Things’ (IoT). Specifically, we were interested in finding the problems with existing toolchains and ways of working, rather than coming up with something immediately productizable, which of course gives a greater amount of freedom for hacking things together.

If the IoT is really about the Internet of Everything (as it was put in the keynote at the recent Microsoft TechDays Paris), where better to start than looking at writing code for some daily problems in the office?

One such problem is that, as a company of around 250 people in the Cambridge office, we lack space for whole company meetings, although we still regard them as important. As a result, we often use our canteen area for larger events, which are also streamed on the intranet. But this has a problem: canteens have noisy fridges which can disturb the event. If you turn off the fridges, tomorrow’s lunch could be spoilt, and the canteen staff get upset.

Can the Internet of Things solve this?

Identifying when a large meeting is happening

The first issue is to work out when a larger meeting is happening. Our canteen is a bookable resource in the Exchange server, so on the face of it it’s easy, but ‘Lunch’ is also a recurring appointment in the room, as are some ad-hoc meetings between individuals.

One feature larger meetings share is that they are streamed as I noted above, and this also means they use radio mics. Radio mics are, of course, nothing more than a simple FM radio transmitter and in Europe at least, they tend to be at around 860MHz. Getting a cheap receiver that can listen to multiple radio channels simultaneously is also remarkably easy: the RTL-SDR dongle is ideal for this sort of application.

Once we have this receiver, we can use the GPL’d RTLSDR Scanner software to monitor for the presence of the radio microphone. This is a nice little Python script that can output a CSV file with frequency and strength – just what we need.

Writing a bit of code lets us see when the radio mic is on. While we’re at it, we also connect to the internal Microsoft Exchange Server’s API to find out what the current appointment is. We write the appointment’s ID into a SQL Server database table to ensure we don’t query the Exchange Server more than once.

If there’s an appointment that’s just starting, we can now post to Yammer to remind everyone, giving the link to the intranet stream. The Yammer post can also contain a still from the webcam that monitors the queues in the canteen.

yammer

Turning the fridge off

The last part of the task is to turn the fridge off. I’m using a Belkin Wemo switch for this purpose, which uses a very simple (un-authenticated!) SOAP protocol to query it. The program I’ve written just fires the ‘off’ command to the switch when the data comes back from the Exchange Server API.

Turning the fridge back on

Just as important as turning the fridge off is turning it back on again. We could do this when the Wireless Mic signal disappears but we frequently have multiple speakers swapping between different mics. It wouldn’t be great for the fridge to keep being switched on for a few seconds as they swap over. So, we record the time the mic went off in the database table too. A few minutes later, if the signal has not been re-acquired, we turn the fridge on again.

So, there you have it – the result of hacking together the unlikely combination of a Software-Defined Radio, a radio microphone, a Belkin Wemo, a fridge, Yammer and the Microsoft Exchange Server API!