All posts by dnas2

Amateur Radio, chile, DXpeditions

Radio activity from Chile

1 Comment

As some of you might be aware, I’m going travelling in Chile for most of November and December. This isn’t a radio DXpedition as I can only take one 23kg suitcase to last 6 weeks and clothes and camping stuff have to take priority!

That said, I do have a Chilean licence, and I am taking some equipment, so this is the plan at time of writing:

Callsign

As I am travelling and don’t have a particular base in the country, the Radio Club de Chile have kindly suggested I use their HQ address in Santiago (CE3) as my licence address. This means that my Chilean licence has been issued as CE3/M0BLF

As I understand the Chilean rules, when I am operating from other parts of Chile, I have to use the local area number as a suffix to the Santiago callsign, eg. CE3/M0BLF/6

HF

I’m taking my IC-706 and a 15m dipole. There won’t be any linear and power will be from a Gel Cell (Sealed Lead Acid battery), which I’ll buy out there to carry on the part of my journey through Patagonia from Temuco – Tierra del Fuego (19th November – 11th December, if all goes to plan). Due to the fact that I can’t take such batteries on planes, I won’t be using the HF gear from other places in the country.

I expect most activity to be CW / PSK31 to reduce power demands. I may also send back a couple photos via SSTV, just because I can. When I’m further south, away from where most people are beaming, I’ll aim to appear on WSPR for a little while before I switch to CW. WSPR allows me to announce I’m there without requiring a full-power CQ call that would be needed to reach the RBN.

Activity will only be occasional, from campsites, and I may not be able to get the antenna up high as I don’t have luggage space for poles, etc.

CQ WW CW

The biggest CW contest of the year takes place while I’m in Chile. I plan to operate as much as the batteries will permit as CE3/M0BLF/6 from the Puyehue region in central Chile. This will be Single Band on 15m. I’ll decide out there whether to enter Low Power or QRP.

IOTA Activations

Assuming I stick to my itinerary, I should also be active from:

  • Chiloe (SA-018) on 30th November and 1st December as CE3/M0BLF/7
  • Tierra del Fuego (SA-008) on 10th December as CE3/M0BLF/8

Again, this will be low-power, battery-operated, single-band, so please don’t expect too much! For the same reason, I will not arrange skeds.

Yes, I will be passing lots of rare island groups as I sail down through Patagonia between Puerto Montt and Puerto Natales, but I’ll be on a ferry and I won’t be able to disembark. Another time, perhaps.

APRS/Echolink

  • I’ll be active on APRS as M0BLF-7 in areas with APRS coverage, but be aware that coverage in the country isn’t very great
  • I’ll be taking an Iridium tracker (thanks M0VFC) which cross-posts positions to APRS as M0BLF-12
  • I hope to use some of the local 2m Echolink nodes to contact people at home

Updated 5-Nov-15: Switched band to 15m in the light of the latest propagation predictions; mentioned the WSPR beacon.

Amateur Radio, DXpeditions

That’s all folks…

Leave a comment

That’s it. At 01:30 UTC on 17th July, we finished our DXpedition to Svalbard. So, how did we do?

The raw numbers

On the face of it, this wasn’t our most successful DXpedition in terms of QSO count. During the six full days we were QRV, we only made 6305 contacts between the six of us.

That hides a significant fact, however: this far north we never really expected to make a huge number of contacts. I’ve written more about propagation below, but to summarise only 30m, 20m and 17m were really open.

We had decided before we came to split into two groups, therefore: On Sunday, M0VFC, M1ACB and myself went on an excursion to Barentsberg allowing G3ZAY, M0HSW and M0TJH to stay in the shack. We did the opposite on Monday. Then, on Tuesday and Wednesday, we adopted the same pattern for our excursion to Pyramiden. This means that for most of the time, there were only three people in the shack.

Perhaps more surprising for us was the 44:56% SSB:CW (2858:3447) split. As it happened G3ZAY, M0VFC and I made almost exclusively CW QSOs, leaving M1ACB, M0HSW and M0TJH to do almost all the voice operation. That wasn’t planned as such, but worked well from our point-of-view.

Propagation

Propagation in Svalbard in the summer is weird. First of all, there is no nighttime during the months of Midnight Sun, so nothing below 30m ever really works. We made just 29 QSOs on 40m, and none on anything lower.

30m had some good openings during morning and evening but was pretty dead during the main part of the day. Throughout what would be the ‘night’, there was often good propagation to the US, which we could see from the 30m APRS beacon we operated.   965 (15%) contacts were on 30m.

20m was the most reliable band and was often open to JA, EU and NA at the same time. Even it went through a quiet period from mid-morning until mid-afternoon however. 3534 (56%) contacts were on 20m.

17m was often open during the afternoon to JA and NA, and then EU in the early evening. There was almost nothing here during the mornings. 1772 (28%) contacts were on 17m.

15m and higher were pretty much constantly dead. We made just 5 CW QSOs on 15m.

Overall, however, the main thing we noticed was that propagation was hugely susceptible to the K-Index. At a K of 5, most bands were pretty unusable. On days when K went down to 2, we made many more contacts, except for our final day which seemed to buck that trend. It might be that flux collapsing to 100 was responsible for that.

Logging

We were logging with Win-Test. With six operators using their own callsigns, you might have assumed we had six laptops but with only three stations we didn’t do that. We instead adopted the approach we’d used successfully in VP9: all copies of Win-Test were networked together and we all logged into a single log. We ‘logged in’ to Win-Test for each of our shifts, however, so we could then separate the logs based on the operator callsign.

During the latter part of the trip, we were uploading logs automatically every 30 minutes to Clublog and LotW. We’ll blog more about how this was done later (and I think a presentation for the RSGB Convention is planned), but basically we had a program listening to Win-Test’s network, capturing the QSOs, compressing them, and then batching them up.

APRS

We always like to innovate in some way on our DXpeditions, and this year (in addition to the live uploads), we also experimented with running an APRS beacon using MFSK16 on 30m. This was less successful than we had hoped given propagation on that band, but it was useful in that I could check how open 30m was to the States when I woke up. I could have used WSPR for the same result, though, assuming internet connection.

That’s kind of the point, though: had we not had internet access (which we weren’t expecting to have), APRS could have given us a route to do basic messaging if required. I think it’s something we will continue playing with to see whether the technology can bring anything extra to DXpeditions.

Thanks to members of the Cross Country Wireless email reflector for running IGates for our packets, and especially to Chris G4HYG for writing APRS Messenger, which we used for the beacon.

And finally…

Photos

My photos from the trip are in a set on Flickr.

Aerial photography

Hugo, M0HSW, brought a drone with him as his hand-luggage. We might have ridiculed him at the airport a little bit, but the film of aerial shots of Svalbard (Longyearbyen and Pyramiden) that he has produced is spectacular. Viewing it is highly recommended:

 

Amateur Radio, DXpeditions

Staying at JW5E

Leave a comment

While we’re in Svalbard, we’re operating from the JW5E shack, the local radio club cabin. I’m also sleeping here, so I thought I’d give a quick introduction to what’ it’s like to stay at the station, which can be rented by other radio amateurs. For official information on the station, see jw5e.com

Before going any further, I should stress that this post is intended only as a factual description of the shack at time of writing, in an attempt to help future visitors with some of the stuff we hadn’t known. This isn’t to criticise – the local amateurs have been wonderfully welcoming, and their willingness to open the station to others is very much appreciated.

Location

The shack is located on the shore of the fjord, just east of the main centre of Longyearbyen, but within easy walking distance of the main shopping street and the hotels in town.

Polar bears have been known to come up the fjord, so there is a slight risk, but the shack is within the ‘no polar bear protection needed’ area of town. There’s a can of CS gas by the door in case of an unexpected visitor, but we’re told it would be technically illegal to use this (on the safety grounds that you’d be likely to spray yourself while panicking!).

There’s good mobile phone coverage in Longyearbyen from both Telenor and NetCom. Although our mobile operators in the UK claimed they didn’t have roaming agreements with Svalbard, in practice it seems that if your operator has an agreement with the Norwegian operators listed above, roaming works. For Three UK customers, it looks as though this means Svalbard comes under the Feel At Home deal for Norway; I’ll update this information if we get an unexpectedly large phone bill on our return!

Credit cards are widely accepted. I’d always carry some cash but we haven’t needed very much. On the whole, goods are quite expensive because of the cost of importing them to such a remote location.

JW5E

Facilities

Living space

The cabin has kitchen with fridge, kettle, coffee machine and microwave, 2 bunk beds, sofas and a toilet. There’s heating (and in fact the shack has been at times too warm at night, at about 20c) but there’s no running water here. You can buy bottles of water in the amply-stocked supermarket, and we’ve also been filling bottles at the hotel where some of the other operators are staying.

The toilet is a combustion toilet ‘for solid matter only’ (liquid deposits are left on the beach outside), which is easy to operate. It seems you have to guess whether you need a 40 minute or 70 minute burn for the size of the job, however! I’d recommend bringing some alcohol hand gel as there’s no water to wash your hands.

The beds are comfortable, although right now someone using the upper bunk would have to share it with a Butternut vertical! Just bring a sleeping bag and you’re good to go.

Shack

The shack currently comes equipped with a FT-1000MP Mark V and TS-870, with an interlocked six-pack antenna switch at each of the two operating positions. We’ve actually crammed four operating positions into the space and put the TS-870 to one side so we can fit the two K3s and IC-706Mk2G we brought with us. The FT-1000MP Mark V station has a MicroHam unit and computer. Each operating position has a 20A power supply.

The shack used to have an amplifer but this has now been removed due to repeated damage by visiting groups, which is a pity but understandable. We’ve got a KPA500 with us. On the other hand, the shack now has Wifi, which was a nice surprise. This is supplied from Martin JW5ZLA’s home connection, via a Ubiquiti link on the tower.

The shack also has a range of useful accessories, patch cables, etc neatly arranged on shelves. There’s also a limited supply of tools but as always, it’s probably safest to bring your own accessories! Power, by the way, is 220V 50Hz with European plugs.

If you’ve seen W2NAF’s video of the shack, there has been some renovation work since it was filmed, so the orientation is now the other way around:

JW5E shack

Antennas

The state of antennas at the shack is a bit ropey, so you may want to bring plenty of wire and fibreglass poles. A climbing harness for the tower may also come in useful. We’ve installed a vertical dipole for 20m on the beach outside the shack for our operation.

As of today, JW5E has:

  • 10m / 12m beam
    We haven’t used this as the bands haven’t been open. It’s very close vertically to the 15/17/20 Optibeam, however, so it may not be in tune.
  • 15 / 17 / 20m Optibeam
    This is missing a director and is rather close to the 10/12m beam. SWR is OK on 17m but it is high on 15m (about 2:1) and very high on 20m (about 3:1 at the CW end and 2:1 at the SSB end). We’re using it as a 17m single band beam.
  • 30m dipole
    This is working OK.
  • 40m sloper
    Needs an ATU on 40m but works fine with a good SWR on 15m.
  • 80m dipole
    We haven’t used this as the band isn’t open during the 24h sunlight in the summer.
  • 6m beam (fixed pointing South)
    The reflector is badly bent and the coax doesn’t enter the shack.
    We’ve now bent the reflector back since the photo below.
  • 6 – 40m vertical
    We’ve used this a bit on 40m but generally the sloper seems better. We haven’t used it on other bands as we noticed that RF gets into the shack (it’s literally just outside the window).
  • 2m and 70cms verticals
    We haven’t tested these.

The HF beams are on a ProSisTel rotator in the shack.

JW5E antennas

Amateur radio on Spitsbergen

There is a 2m repeater, JW9SRR, on a mountain out of town, which can be reached from a handheld outside in Longyearbyen. We’re told it’s not completely reliable at the moment but it seemed to open when we tested it. It’s on 145.600 with a -600Hz shift, 91.5Hz CTCSS.

There’s also an APRS node on the European APRS frequency (144.8MHz) and there are simplex frequencies on the VHF bands used by the locals, which are listed in the shack.

Amateur Radio, DXpeditions

An excursion to Pyramiden

1 Comment

After our trip to Barentsberg a couple days previously, yesterday we went to Pyramiden, named after the Pyramid-shaped mountain above the town. Like Barentsberg, this is a Russian coal-mining settlement. Unlike Barentsberg, however, Pyramiden is a ghost town.

Or almost a ghost town. The mine closed in 1997. Under the Svalbard Treaty, if a settlement is abandoned for 10 years, the country responsible has to remove all trace of the town and return the area to its natural state. In early 2007 – 9 years and 10 months after the town had been abandoned – the Russian government decided it was cheaper to pay for 2 people to stay there, repairing buildings, than removing all trace of a town for over 1000 people and the associated mine. Today, the population has increased to 8. They get all their supplies from Barentsberg and, apart from one square metre on the harbour where there’s mobile telephone coverage from Longyearbyen, the only communication is by letter twice a month or satellite phone for emergencies.

Nevertheless, to all intents and purposes, this is a ghost town, and a fascinating relic of a remote outpost of the Soviet Union. Recent polar bear visits to the town meant that we weren’t free to explore; we had to travel as a group with our guide, rifle in hand, just in case.

Even the 'r' of the town's name 'Pyramiden' is missing on this sign.
Even the ‘r’ of the town’s name ‘Pyramiden’ is missing on this sign.

Pyramiden town

A child's painting in Pyramiden cultural centre
A child’s painting in Pyramiden cultural centre
Film abandoned on the floor in the Cultural Centre, which was also the town's cinema.
Film abandoned on the floor in the Cultural Centre, which was also the town’s cinema.
The main street on Pyramiden, called '60th Anniversary of the October Revolution Street', looking towards the Cultural Centre and statue of Lenin.
The main street in Pyramiden, called ’60th Anniversary of the October Revolution Street’, looking towards the Cultural Centre and statue of Lenin.
Our boat's route to Pyramiden. Again, the track that appears to be on land is because the glacier has receded, so the map is out-of-date.
Our boat’s route to Pyramiden. Again, the track that appears to be on land is because the glacier has receded, so the map is out-of-date.

For anyone wishing to do this trip themselves, this excursion was organised by Spitsbergen Travel and comes highly recommended. The guides are excellent. If you’re wondering whether to do both Barentsberg and Pyramiden, I’d say emphatically, yes. The towns might both be Russian, but that’s where the similarities end. I found Pyramiden felt more comfortable as its ghost town nature is explicable with the collapse of mining after the fall of the USSR, unlike Barentsberg which just feels out of place.

Amateur Radio, DXpeditions

An excursion to Barentsberg

2 Comments

Yesterday, Rob M0VFC, Steve M1ACB and I left the rest of the group in the shack while we took a trip to the Russian settlement of Barentsberg here on Svalbard.

The excursion was organised by Svalbard Travel and is moderately expensive, but well worth the cost for a long day on a boat with an excellent guide. The only disappointment was the fog that plagued us most of the way to Barentsberg.

The trip starts with a detour to the Esmark glacier, where we were given barbecued whale on the deck for lunch. I’m not insensitive to the controversy surrounding eating whale but when in Rome… I’d never had it before, and it’s unlikely I will have it again as the meat was rather tasteless other than the chilli marinade it had been kept in.

Esmark GlacierAfter this spectacular detour (where we even got to see some of the ice breaking off), the ship turned South across the fjord to Barentsberg, where — after a minor issue in which our boat got entangled in another boat’s mooring lines — our guide handed over to a Russian guide for the tour of the town.

Like most places on Svalbard, Barentsberg was established as a coal mining town and was Russian-controlled for almost all of the 20th century (and it still is). There’s something distinctly not quite right about the place, however. Most of the town has seen better days. The buildings are rotting and collapsing and little effort is being made to preserve them as nature takes over. A broken child’s swing gave me echoes of the photos you see of Pripyat near Chernobyl.

Barentsberg

In the main square, the guide was keen to point out the statue of Lenin and the sign proclaiming ‘Our goal: Communism’, saying that of course their goal no longer is Communism and that the banner today only reflects an important part in Russia’s recent history. She then went on to explain how the workers are provided housing in large accommodation blocks, there’s a subsidized canteen, and only two shops in the town – one for food and the other for everything else. They operate a cashless society, with credit being stored on cards with which everyone is issued. Contracts for the mining workers are for a fixed 2 years, after which they are allowed to go home for 2 months. I’ll leave the reader to consider the implications of the preceding sentences, but suffice it to say, the place felt like a timewarp. We were also told about the Russian consulate in Barentsberg being the most northerly in the world, with the great acclaim from our guide ‘It’s Russian, and it’s here’.

Lenin outside the worker's accommodation
Statue of Lenin outside the worker’s accommodation. The banner reads ‘Our goal: Communism’

Perhaps the strangest thing about the town, however, is the size of some of the facilities. The current population is just 400 and yet the school (for 40 pupils) and newly-built hospital are clearly designed for many, many more than just the current number of miners and their families.

The school for 40 pupils. The artwork was done a few years ago by students at the St Petersberg School of Arts.
The school for 40 pupils. The artwork was done a few years ago by students at the St Petersberg School of Arts.
Our tour's route from Longyearbyen to Esford to Barentsberg and back to Longyearbyen. The position that suggests we were on land near Grumant is incorrect - it's the map that's wrong not the position!
Our tour’s route from Longyearbyen to Esford to Barentsberg and back to Longyearbyen. The position that suggests we were on land near Grumant is incorrect – it’s the map that’s wrong not the position!

 

 

 

 

Amateur Radio, DXpeditions

Hello from Svalbard

Leave a comment

After two flights (London – Oslo, then Oslo – Longyearbyen), we’ve arrived in Svalbard. This is the view from the shack, taken just after we arrived (about 00:30 local time). The midnight sun is very real!

View from the shack on Svalbard

The first surprise, we have mobile phone signals and the shack has internet. So, our fears about lack of connectivity were slightly misplaced. Nevertheless, we’re still going to be uploading logs via the Iridium satellite link, and we’ll be doing APRS over HF, because we can.

The JW5E shack has been slightly reconfigured since the video W2NAF took last year but some of the antennas no longer work efficiently. As I type, this is the current situation (which may change over the week):

  • We’ve installed a new 30m dipole. Rob M0VFC is on 30m CW with the shack’s FT-1000MP Mark 5.
  • Steve M1ACB is using the 40-6m vertical to operate 17m SSB with his Elecraft K3.
  • Hugo M0HSW is on 20m using the 15/17/20m beam using Rob’s K3 and a KPA500.
  • My IC-706 is set up as the data station for later on. It will use any antennas that are free.

We’ll be on during the IARU contest – we’ll be giving reports but we aren’t entering: with so few bands open this far North and multiple callsigns between us, we wouldn’t be able to be competitive.

M0HSW, M1ACB and M0VFC operating in Svalbard
M0HSW, M1ACB and M0VFC operating in Svalbard
Amateur Radio, DXpeditions

DXpedition Announcement: Svalbard, July 2015

Leave a comment

From 10th-17th July 2015, I will be one of six operators activating the JW5E shack on Svalbard.

We’ll be operating on 40m-10m (and maybe some 6m if conditions allow), mainly on SSB and CW, using JW/{homecall}.

We won’t arrive in Longyearbyen until late in the evening of Friday 10th, and we depart very early in the morning on Friday 17th, so in practice we’ll be QRV for about six days.

Please remember that Svalbard has midnight sun for about four months, so it won’t get dark while we’re there. That means that we’re expecting 40m and 15m+ to be very difficult.

QSL will be via Clublog OQRS (please don’t send paper cards). We will be swapping operator at times, so please ensure you log the correct home call – don’t rely on an old spot from the cluster.

For more information, see my main DXpedition page at http://www.domsmith.co.uk/amateur-radio/dxpeditions/2015/jw

The full operator list is:

  • Myself
  • Martin G3ZAY
  • Hugo M0HSW
  • Tom M0TJH
  • Rob M0VFC
  • Gavin M1BXF
Cambridge 105

How to send a text message to the 1970s

Leave a comment

As some of you may know, I’ve been involved in Cambridge 105’s ‘Pirate Cambridge 105’ special event – a 12-hour live broadcast from a riverboat moored on the River Cam, intended to celebrate and reminisce about the golden era of Pirate Radio.

In keeping with the theme, we limited ourselves to only using equipment from the 1960s and 1970s. For mixing desks and audio processing, that’s an easy deal as long as you can get hold of the gear. (It’s amazing what people have hidden in the back of their garage.) But this left a crucial question: How do we get text messages and emails using only 1970s gear?

The easy option would have been to say that we don’t take listener input during the day. But we’d already started making arrangements to broadcast on the Channel 292 frequency, 6070kHz shortwave. This meant that we’d potentially have many more listeners across Europe, so hearing from them would be great.

One bit of technology that did exist in that era, of course, is radio teletype (RTTY).

What’s RTTY?

For those who aren’t familiar with RTTY, essentially you have two audio tones which represent 0 and 1 respectively. Encoding characters in the 5-bit Baudot code, you can set up a radio transmitter to rapidly switch which audio tone it is sending, which lets you transmit characters.

By modern standards, RTTY is incredibly slow: about 50 baud (50 bits per second, or 10 characters per second), but it works.

Originally, the receiving end would have been a teleprinter, which was essentially identical to a mechanical typewriter, but connected to a radio driving it.

So how did we do it?

Well, we didn’t have a genuine teleprinter available, and besides it would have been too noisy next to a radio microphone even if we did. Instead, we got a couple Raspberry Pis (running Fldigi, which allows you to transmit and receive RTTY in software).

At the transmit end, a script on the Pi grabbed incoming texts, tweets and text messages to the studio and sent them to a radio every 2 minutes. Messages were truncated at 200 characters (to avoid using all the receipt roll at the other end!) and word wraps had to be inserted every 40 characters, which is the width of the receipt printer font we were using.

At the receive end, the other Pi did the decoding – yes, we broke the 1970’s equipment rule, but the technology was still of the era – and sent the result every minute to a dot matrix receipt printer.

Messages that were OK to read on-air could then be given to the presenter by hand, on tiny pieces of receipt roll.

Any improvements?

I don’t know whether we’ll ever have cause to use this system again but, if we do, we need to detect the end of messages rather than just doing the print every minute. We hadn’t expected the volume of messages we received (we used most of roll despite the truncation!) and this meant a significant number of messages got cut as they were printed mid-transmission.

Pirate Receipt Roll
Photo by Jonathan Watts

Agile, Project Management

What is technical debt?

Leave a comment

This article is my own synthesis of ideas presented at Lean Agile Scotland in September 2014 by Dan North, Jabe Bloom and Torbjörn Gyllebring, with some of my own embellishments. Any misunderstanding of their points is mine! I’ve linked to their original talks at the foot of this post.

Conway’s Law says that the design of our system will be rooted in the design of our organisation.

So, we can never design a system from scratch: some of the design will have already started before we start designing.

This isn’t necessarily a problem, however: indeed, we should embrace it, because organisations suck if they don’t have the right APIs and other architectural borders in place between where teams are (that’s when teams start to get bureaucratic layers preventing their work from proceeding as their pace.) We should take time to get this right, and this is why having micro-services to do communication between teams is a good idea.

However much we work on this, however, we will face problems in the code we write, many of which will relate to the architecture. We can either chose to fix the symptom (via a quick fix) or to fix the underlying problem (which is generally more time-consuming). Quick fixes aren’t necessarily bad when applied in conjunction with a longer-term fix but remember they are akin to painkillers, and you can’t take painkillers indefinitely.

Quick fixes often take the form of a mandated structure: we might add too much or too little process, a particular build system, an architectural design ‘recommendation’, a decision on a particular source control system or IDE, or similar. As managers, we should be especially careful with such quick fixes. We are aiming to apply a ‘Goldilocks’ amount of constraint on our teams: too constrained and development will stop because the people can’t do their job; too little constraint and there will be too many options to do the right thing. The right amount is a strategy that’s clear enough to give us a task for today and a constrained set of options for the future. That strategy is actually a conversation – top-down constraints from management, and bottom-up information and designs from the development teams.

Partly because of the difficulty of identifying the ‘Goldilocks amount’ without information from the team (which presupposes time taken to do investigation), quick fixes generally tend to exhibit more unintended consequences than fixes that correct the underlying problem.

In a Cynefin sense, rather than taking time to analyse a problem as demanded by Complicated scenarios, our quick fix has made matters worse by creating a Complex relationship. The side effects can’t be foreseen, but become apparent in retrospect. Worse still, the quick fix may fail to address the root cause of the problem, and so we may get a positive feedback loop causing problems to increase as a result of our solutions.

This is technical debt, because the system starts constraining what we can do.

Cynefin tells us we should monitor for this positive feedback and, when we sense it, take time to respond to it, possibly by rolling back the change and trying a different approach.

To check for technical debt, therefore, we should periodically take time to ensure that our system isn’t constraining us. We can do this, for example, by ensuring our tools haven’t become our masters by checking that we can still build our products using a single command line (without complex CI systems) and that refactoring can be done in Notepad (without Resharper).

Original videos:

Agile, Project Management, Red Gate

Understanding a feature through 3-amigos

Leave a comment

The last project I ran here at Red Gate was on SQL Compare / Data Compare 11 project. SQL Compare is a superficially simple product designed, as it’s name suggests, to compare Microsoft SQL database schemas. Its cousin, SQL Data Compare compares the data in the tables.

Like the product itself, the Compare 11 project had a superficially simple goal: add support for the new features in SQL Server 2014. Easy, right? It turns out, not so much…

The largest new feature is Memory-Optimized Tables (MOTs): the ability to keep data in memory so you get significant performance improvements through reduced disk-access. But as you dig into what MOTs can and can’t do, you realise that you’ll need a lot of special-cases in the code to deal with the limitations. The details of those limitations is well outside the scope of this post, but one of my colleagues Michelle Taylor has written a useful post on our company blog for those who want to know more.

When the team came to start work on this story, which was before the release of SQL Server, some of the limitations weren’t immediately apparent. This led me (as project manager) to a false sense of security, and so I was far too lax on getting the user stories and acceptance criteria defined in detail. The lack of knowledge this created in the team caused the feature to become massively under-estimated: just 50 task-points (at their velocity, each task-point turned out to be just over an hour of real work).

The first tasks related to this story went on the Scrum board early in the project: ‘Add property to CreateOrAlterTable’, ‘Add property to DatabaseObjectsTable’, ‘Update DatabaseObjectsTable.Equals()’. You see what’s missing? We had dived straight into implementation with no form of planning first. That first sprint was estimated as 15 points. By the time we got to the end of it, we had uncovered quite how unexpectedly complex this feature was likely to become. We were nowhere near 30% (15 out of 50 points) complete on the feature. We carried some of the tasks forward to the following Sprint, however, trying to convince ourselves that we’d crack it soon. Before I knew it, we’d spent nearly a month on this feature, continually uncovering more and more complexity, and morale on the team was starting to become impacted.

Even then, my monthly project A3 report excessively optimistically promised that we’re complete both Inline Indexes (another 2014 feature) and Memory-Optimised Tables in the next four weeks.

It was time to pull the Andon cord.

Luckily, this also coincided with a slight change to business focus. The team was being asked to ship SQL Compare 10.7, giving support for connecting to SQL Server 2014 (but not the new features) to ship with the final release of SQL Server 2014.

Whilst this was a massive disruption to the team, it did give us some breathing space to find out more about MOTs before we had to work on them again.

By the time we were ready to start work on MOTs again, the Product Owner had defined the Acceptance Criteria in more detail. There were a series of tasks, themselves totalling 42 task-points in estimates, which centred on reviewing these ACs and making sure they made sense. This worked OK, and put us in a much better place, but with a large team, the meetings didn’t really work.

At this point, one of my colleagues pointed me to the idea of a 3-amigos meeting. I’d actually tried this once before on a different project, and I found myself wondering why I hadn’t tried this earlier on the MOTs work. The idea is to have three people (specifically the Product Owner, a Developer, and a Tester) in a room together, defining how the feature should work for users, in the code, and in test cases respectively. There’s a good write up of the meeting on the Scrum Alliance website; the only difference for us being that the Product Owner also doubled in the BA role for us. Although it felt slightly more waterfall than we were used to, the output from the meeting was a set of really well defined, clear Acceptance Criteria and a good common understanding of what MOTs involved, that was ready for sharing across the team.

The team re-estimated the work, and came up with an estimate of 250 points. In the end, it took 337 – still rather more than 250, but also far more than the original 50 points planned!

This process worked so well that we adopted the same meeting for other features in SQL Server 2014, rotating the role of the amigos through the dev team. As long as the communication about what was discussed is good immediately after the meeting, everyone became brought into the idea of 3-amigos, and ensured that the chaotic start to MOTs wasn’t repeated during the rest of the project.