A few months ago an article was published in Computer Weekly where corporate IT Lawyer Alistair Maughan, argues the case that Agile will not work in Government projects.  I didn’t see this article when it was first published, but it was brought to my attention when a colleague noticed that one of the comments arguing against the article, referenced a presentation I gave.

Many thanks to Francis Fish for referencing my presentation where I discussed introducing Scrum on a UK government project and used Agile methods to rescue a failing waterfall project, again within a UK government programme of work.  I’m really glad my message got across despite the ropey presentation, although in my defence it was my very first attempt at public speaking.

I’ve been working on UK government projects for over 10 years. During my initial 5 years on government accounts I was part of Valtech teams using RUP and aspects of XP, the later 5 years I was fortunate enough to implement Scrum and Kanban on my projects.  With my Agile UK government experience I feel I am justified in challenging many of the misconceptions mentioned in the article.

Firstly, mismanaged projects will go spectacularly wrong independent of the project management framework.  Projects fail for many reasons including ill-conceived requirements, poorly skilled developers, inept management as well as inappropriate SDLC.

So to address some of the points in this article:

“But Agile simply won’t work in the real world of government ICT.“  This is not true, Valtech have implemented a number of successful projects using an Agile SDLC on different government accounts.  Often Agile methods have been used to “rescue” otherwise failing projects but increasingly the merits of running a project using an Agile approach from the outset have been accepted.

“We need a Richard Dawkins to bust the myth of the Agile gospel.”  We need to bust the myths of Agile, that Alastair has heard and believed.  Agile does all of the same value adding activities found in a waterfall project, just at an appropriate time, for example User Acceptance Testing when the real requirement pops out during development rather than right at the very end.  This approach increases the effectiveness of risk management enabling us to make better choices over the life of the project ensuring on-time delivery if (and only if) this is the most critical measure of success.

“Under Agile projects, you pay a given amount of money for a set amount of effort. But you can’t guarantee a specified outcome for a specific price.” You can guarantee an outcome by controlling the richness of the implementation.  A key differentiator of Agile methods when compared with more traditional approaches is the focus on outcomes in preference to outputs.

“Departmental budgets are managed very tightly” Agile enables much more detailed and accurate planning at the appropriate time, using prioritisation and identification of the richness of the solution that is acceptable to the stakeholders. With the added benefit of being able to more accurately account for the overspend as soon as it happens and justify the reduction in richness when appropriate.

“Agile can’t give you a clear specification of outputs up-front.” Neither can Waterfall, the chances of change is too high while the procurement proceeds, this says more about government procurement than Agile.  A well executed Agile project will provide sufficient specification at the right time to satisfy stakeholders.  It is not an issue of Agile projects rejecting documentation (or specification), rather that, when done well, they defer detailed specification to reduce exposure to late breaking changes or lessons learnt resulting in rework to the specifications.

“As if that isn’t problem enough, Agile offers insufficient means of remedy if things go wrong.” It’s a contractual responsibility not the SDLC to offer remedy.  Further, an Agile project should provide evidence of progress in a far more tangible manner. Valtech (and many others) have been working hard to develop contracts that deliver the flexibility in scope that customers deserve while acknowledging the suppliers responsibility for the management of technical risks and the intrinsic qualities of the product.

“You can have an ICT project with a watertight contract, clear deliverables, openly and legally procured, with a fixed price and appropriate remedies if you don’t get what you want.” Waterfalls might deliver what the customer wanted at the time of specification but rarely what the customer needs; as has been often quoted, 40% of features delivered by waterfall projects are never used   (Gartner). The act of insisting on a full specification of everything to deliver will actually make projects bigger. This is a natural consequence of the penalties traditionally applied when new scope is added late. By contrast an Agile project will be accepting changes in scope while encouraging challenge to “gold plated” solutions preferring “good enough” (but no less!).

Please find links to my presentation slides on implementing Agile on Government projects.

http://www.slideshare.net/valtechuk/agile-adoption-in-a-waterfall-environment

http://www.slideshare.net/valtechuk/agile-project-rescue-in-a-waterfall-environment

I feel enthused to blog about my current project, which is the first project I’ve ran where we have a dedicated UX expert on the team. For those of you who are not aware of the term UX, it stands for User eXperience. The UX expert will work closely with the customer and business analysts, defining user journeys throughout the site/system, producing the wireframes that feed into the designs and ultimately the UI development.

Consciously or even sub consciously we all appreciate a website with a good UX and can get really frustrated by poor UX.

While this is the first time I have had a specific UX expert in my delivery team, I have been aware of other Valtech projects delivering UX, over the last couple of years, particularly in the the Agency space for our large retail clients.

Now I wasn’t exactly sceptical, but didn’t fully appreciate just what the UX role entailed and what it could bring to the project. I will be honest and admit to being slightly derogatory about what a UX consultant could offer, and I may have possibly labelled it as ‘colouring in’. My bad

My ignorance was however, strongly influenced by the types of projects I had previously delivered. A lot of my previous experience has been delivering large government systems, where the kudos was always focused on the back-end technologies and not the front-end. Also a good user experience was never a non functional requirement.

The UI design and UX for these large government CMS systems is on the whole quite poor, overly complex and unintuitive. In some instances we may have worked with graphic designers, but most of the time the UI was being designed the customers Business Analysts, where only basic principles of a quickly outdated style guide are to be adhered to. On more than one instance, I was running a dev team which inherited reams of up front, waterfall, external design documents. These had been created using Visio and MSPaint, which led to the designing of near impossible components and cross-field validation. Chaos would generally ensue in System test and UAT when the final screens didn’t look exactly like the signed off Visio and MS Paint designs.

On my current project, the UX expert has made an instant impact. Working with the lead BA, it was obvious that the business owner was engaged and bought in to this way of working. High quality interactive wireframes, are produced in Photoshop with key animation sequences overlaid in PowerPoint. Great care is taken understanding different personas and minimum marketable features. The customer gets near instant feedback of how they want the site to be used.

I sort of expected the benefits that the UX expert would bring to the analysis and design, what I didn’t predict was how important the UX expert has become to the developers. Collaboration between the UX and devs was really visible from the outset and was recognised as one of the most significant positives at our recent retrospective. Key to this success was the way the UX expert researches the technology options, ensuring the design can be achieved using CSS or identifying the necessary JQuery plugin. Non standard design choices are always reviewed with the development team prior to sprit planning to ensure no near impossible designs are agreed with the customer.

Our Agile processes enable this collaborative way of working. We are all collocated provide our updates at a single, large but efficient daily stand up meeting. The devs and UX expert are seeing each others updates, progress and blockers. The BA / UX team are working one or two sprints ahead of the developers, so no design artefacts are going stale in the backlog. We have also ensured that a UX peer review is part of the definition of done policy for the front end developers.

This way of working culminated in a really positive first demo, with great feedback from the business owner and sales director.

The case study describes how agile practices were applied to turn around a failing waterfall project.

Specifically by living the agile principles of respecting the team, increasing visibility and effective prioritisation we combatted the classic trappings of waterfall and the team successfully delivered to a satisfied customer.

How the introduction of basic Agile and Scrum principles helped a failing waterfall project to succeed.

As the 14th project manager in two years to take on this task, I was recently seconded into a client’s troublesome project in an attempt to bring it back under control and to a successful conclusion.  The project had all the hallmarks of failure: spiralling costs, difficult relationship with the business customer, team moral at rock bottom with little visibility by the management of the challenges the project faced.  With an unrealistic go-live date fast approaching there was little belief that the project would deliver a viable solution in the demanding timescales.

Immediate actions:   To bring the project back under control the following 8 KEY ACTIONS were taken:

1. Baseline the requirements:   One of the biggest problems on the project was a very inexperienced user group, who were poorly led and had not defined the requirements properly at the beginning of the project, recall that this project set out to be waterfall.  To counter the lack of definitive requirements, the business customer was in daily communication with the two most experienced developers, who would convince the project manager of the importance of any new requests.  This was happening on an almost daily basis.   I escalated this to my client’s senior management team and their customer’s senior management team as the main issue the project was failing.  We worked together with the customer to agree a new base lined scope, plan and price.  Each subsequent change would now go through a formal change request process and have an associated cost and/or an impact on the new Go Live date. 

While baselining the requirements may not sound very agile, this approach served to make clear to the customer where a change was occurring and set for the effective prioritisation, as well as the option for outright replacement of requirements by feature.

This was successful in helping the customer understand the true priority of each potential change.  It was also necessary to reiterate to the customer’s business team and the development team that all new change requests, no matter how trivial, should come through the new change control process.  Not everyone on the customer’s team understood the problem that they were causing, but the developers really appreciated this new stance as they had sometimes felt bullied into taking changes forward, which ensured the new process was enforced.

2. Introduce product owner role:  The concept of the product owner was also introduced.  This role was taken on by the customer’s senior project manager who would become responsible for owning the product backlog and would act as the single point of contact for customer’s business team.   This was necessary to make certain that ad hoc requests would no longer come from the business customer direct to any developers.   It was agreed with all stakeholders that any future change would have its impact assessed properly and be introduced at a time appropriate to the developers.  The main benefit here was that clarity of scope was ensured, not just for the developers but also for the customer themselves.

3. Become a team member:   This may sound obvious, but it was not always the case on this project.  Previous PMs would sit away from the team, communicating to the team by phone and email, despite being in the same building.  The business analysts and developers did have the advantage of being in the same room, with the system testers also being close by, but they were completely cut off from any daily management presence.  One of the first things I did was to sit right in amongst the team.  This had an immediate positive impact on the team, who were previously feeling detached, leaderless and under too much direct pressure from the business customer.  Once in situ, I was on hand to answer or take away any of the teams problems and I was able to keep the team as informed and up to date as possible.  However, this wasn’t a free for all, conventional daily stand up meetings were introduced, where I took on the Scrum Master role.  It was also necessary to work the same hours as the team, as long anti-social hours were still required, ensuring I was available to progress any issues when they occurred. 

4. Introduce daily stand up meetings:   Daily 15 minute stand ups replaced a daily 2 hour long customer facing meeting, which had only been attended by the customer, the business analysts and the PM.  Initially this did not go down too well with everyone.  There was one particular BA who enjoyed the kudos and procrastination of a customer facing role and there were a couple of developers who were just averse to any type of meeting.  Introducing the three questions: ‘what have you done since the last stand up?’, ‘what will you be doing between now and the next meeting?’ and ‘what is getting in the way of doing your work?’  got everyone involved, enhanced communication and gave everyone better visibility of the outstanding tasks. 

5. Introduce the task board:  The daily stand up meetings took place in front of a large, previously unused white board, which was converted into the task board with basic ‘To Do’, ‘In Progress’ and ‘Complete’ columns, with the tasks information on post it notes.  The task board made a significant difference to the team, who had always heard of, but never seen, ‘a project plan’, so were not always aware of the overall project progress.   The task board gave the team a view of progress, which really helped moral, but when coupled with the daily stand up meeting, we were able to identify and deal with any impediments in a timely manner.

6. Empower the whole team:  I insisted that the whole team got to contribute to the prioritisation of tasks and the provision of new estimates.  Previously this type of responsibility was only deemed suitable for the team leader, which meant that estimates were not always accurate, tasks were not suitably delegated and priorities were always based on the business customer’s mandate.  This did not have a negative effect on the team leader, who previously owned this responsibility.  To the contrary getting input from the whole team helped relieve some of their burden and motivated the rest of the team who had not previously been involved.

7. Introduce retrospectives:  While we never had the opportunity to introduce true post sprint retrospectives, we did arrange a number of retrospectives, which occurred: (1) after a particular wave of change requests had been implemented, (2) after UAT and (3) after Go Live.  Despite there being a blame culture within the organisation, we were able to foster feedback to all teams which ensured that past mistakes were not repeated. 

8. Do food:  This project didn’t do the waste line much good, but it did help team bonding.  Tuesday was pizza day (usually my shout), however any other particularly fraught day also qualified to become a pizza day.  Quite often the client’s senior management team would also appear for free pizza, informally mixing with the team, which had never happened before.  The free plastic coffee was always by-passed for proper Italian coffee from the local deli around the corner, with everyone taking their turn to go, and a few late nights and weekends were aided by Wagamamas or Dominos. 

THE OUTCOME:  The project was successfully delivered 5 months after I joined. This was only three weeks later than had initially been agreed in the revised baseline of the project plan, but it was an acceptable delay, which had been agreed with the customer to allow the introduction of additional scope.   The final 5 months of the project was on budget, due to the quality of the estimates and the change control process.  This led to a renewed level of confidence from the customer in my client and further releases have been commissioned. 
 More importantly the project experience is now seen as a positive one for the team, who had the satisfaction of a successful delivery after such a long period of time.
 Personally, there was initial trepidation about becoming project manager number 14.  Number 14 had started out as a derogatory label, implying there would be a number 15 soon.  Later it became a positive motivation and a compliment of sorts.