When to release

My colleague was working on an engagement – doing some “fire-fighting” on a project that is drastically slipping with some severely pressurised managers.

One of the Project Managers pulled him aside with the following:

PM – “We’re looking to release tomorrow morning, can we?”

EQE — “I don’t know whether you can or not? Have you the ability?”

PM – “Eh? What do you mean? Can we release or not?”

EQE — “Again I don’t know you well enough to comment on whether you have the ability or not to make a release.”

PM – “Ok! Are you happy for us to make a release tomorrow morning?”

EQE — “You making a release has no bearing upon my happiness.”

PM  -  ”Oh stop dancing around!! What shape is the application in!!!?”

EQE — “Ah, now – there’s a question – I started testing 3 days ago. I have tested the administration section, the reporting section and the projects section. I have found 6 show stoppers which developers are chasing. I am unable to access the financial section. I have been talking to the developers and they believe that they will be able to have fixes in this evening so I can look into the financial section tomorrow. I aim to finish my testing at end of play Friday.”

PM – “Oh. So its not good to release then? – What will I tell my boss? He’ll go mad when he hears this!? – So you’re not releasing the system then.”

EQE — “I didn’t say that. Listen, it is not my system, I didn’t make the requirements, I didn’t design it, or code it, nor  ”

PM – (Interrupting) “Yes – but you’re testing it and YOU’RE not finished – what are you going to do?”

EQE — (Paused before continuing slowly) “… and nor did I put defects into the system. Its not my money being spent on the system and I couldn’t make the decision on whether to release or not. But what I can do is provide a daily summary of the tests performed, the defects found and a list of risk of failures if the system ships at that point. You can discuss the business risks with your boss. That way your boss can make the decision to ship based upon objective information that I collect.”

EQE — “Let me give you an example, today there is an issue of failure if we were to ship as we cannot access the financial sub section. We aim to be able to access this and start testing it tomorrow. Now if your boss wants to release then that will be based upon information that I have collected and on other business information that I will not have. But it is his business and it is his decision.”

PM – “OK. Thanks,… but he’s not going to be happy.”

EQE – “Maybe not – but at least he’ll have enough information to start making decisions and getting a feel for what he can and cannot do or what he is willing to do. He’ll prefer that than being told to release a system based upon the fear of reaction. Who knows, maybe he will have a business need to release today that out strips any of the connotations from the data presented. “

By the way. The release didn’t go ahead that day. The boss was not happy but then he got more involved on the daily updates and he made the decision to release on the Thursday (and not on the Friday) with an additional release the following Tuesday.

This issue might be a sympton and not the cause of the problem

I recently implemented a Fault Tolerant File Server for a client and had been monitoring the system for a few weeks before handing the “keys” over to the client. This solution was chosen via a stringent tool selection process and I was very happy with the choice.

About 5 weeks later I started to receive calls from the client stating that they cannot access their files. “This is compromising our business – what is going on!!” Initial investigation showed that the File System access was fine. The client stated that this was an intermittent problem – “It never happened before your solution was implemented, so that meant that the problem is with the File System“.

Further investigation showed that:

• the hardware was up and running,
• the operational lights were running as normal,
• the IP cable was secure and the network port light was flashing as normal.

So it might be that the software on the File Server is faulty,  maybe it’s the configuration or perhaps there are thresholds being reached and not being managed correctly? How could this have gone wrong? These were things that I was saying out loud to myself whilst trouble shooting. One of the features of the File Server solution was its advanced management system. The error logs were complaining that the unit itself could not access the network. So I glanced at the switch and routers and all looked fine.

I asked the users if anything had changed within the office since I was last there. They replied that there was no new software installations and the hardware had not been changed. Everyone had been performing their work as before – until my “solution messed things up!” The only thing that happened was that there was a power shortage for about 5 minutes the day before the issue started.

The client also gave the same replies to the questions. Except that after the power shortage, he decided to attach the file server to an Uninterrupted Power Supply (UPS) unit. This unit sits under and behind one of the office switches. So I checked each and every cable one by one. Removing the cable and then testing the connectivity within the office. One cable had a missing secure tab and by gently moving the cable it dislodged itself from the port. A quick test confirmed lost connectivity between the majority of computers (and the File Server). By changing the cable to a more secure one, there had been no connectivity issues (and there are still none for over 2 weeks).

So what is the moral of this story then?
It looked like the problem of no connectivity to the Fault Tolerant File Server was originated in the File Server itself. This was the main focal point for the users and its lack of availability meant that they rightly saw this as the cause of their ills. However, in this instance, the cause of the problem resided in a relatively unrelated area, entirely. By accessing the UPS unit, the faulty cable became slightly dislodged and vibrations within the office would dislodge and re-lodge the cable. This would prevent access to the File System (which was the focal point of the office), but it also prevented access between the majority of computers within that particular network topology and this was not visible to the users.

This reminded me of software – in that in the majority of high-end integration testing, the initial problem found is normally only a symptom of the problem and not the cause of the problem. Testers and users may need to investigate issues further and allow time to do so.

The only question now is…. was the faulty cable the root cause or a symton of another problem?

IQ Trainwrecks

This is a great website: http://www.iqtrainwrecks.com
It provides details of information and data quality disasters from around the world and there are some interesting stories on it.

There is a fascinating article regarding an ‘near-fatal’ miss of an aircraft in Australia in 2009. 75% of the action plan put forward by the operator involves corrective and preventative actions surrounding information quality problems.

This site is full of examples of real life problems arising from poor information quality. These are excellent for helping build up a case for additional software/hardware quality assurance and testing activities. They are also great for getting the mind focused on test designs.

Have look for yourself.

Reviews and Inspections

Anyone who has questioned why they need to have a formal review and inspection process should take a look at Barry Boehm’s “Software Defect Reduction Top Ten List”.  http://www.cs.umd.edu/projects/SoftEng/ESEG/papers/82.78.pdf

In particular, look at the following items from the list.

• Item 6 states “Peer reviews catch 60 percent of the defects.”
• Item 7 states “Perspective-based reviews catch 35 percent more defects than nondirected reviews.”

…. all I can say is, you should have seen some of the content that was going to be published on the web-site. Content that would appear coherent and sane from an author who has been suffering from 2 hours of ‘snow-blindness’ is quickly (and kindly) established as anything during a formal review.

(“Snow-blindness” – where a person is spending so much focus on a task that the person gradually fails to see the wider picture until the only thing apparent is the problem or solution created. The person is blinded by the problem/solution – hence snow blindness. A colleague (with a fresh perspective) can quickly identify issues/solutions very quickly).

Ever get that when writing code or designing an algorithm?

To people who say that they don’t have time to perform a review – read Boehm’s article and question if you can afford not to.

A new web site – what’s involved?

I’m going to use the live example of building EQE’s website to highlight the ‘challenges’ in delivering software to a customer and to spark off thoughts and discussions in the area of creating quality software.

EQE’s previous web site was created by using some of the basic templates from our ISP. While it provided a presence on the web very quickly and provided a reference point for customers, it always felt lethargic and old. So the decision was made to create a fresh new web site and a new logo. Colin acted as the customer on this task and my role was to deliver a tangible product.

Two things were apparent right away. One was that we didn’t have the capability to create a logo in house. The other was that we felt that by engaging a web development company we would be paying someone to ask us to to spend time writing the web narrative. Given that we have the technical capability in house, we decided to ‘look after’ this ourselves.

For the logo, a designer was called in and a design was established. The 3 arrows on the logo was to identify the ideals of EQE -

1. people are the key to any organisation,
2. processes, (when utilised correctly), can be a massive benefit to an organisation and
3. the accumulation and sharing of knowledge allows an organisation to grow and be more effective.

For the website, it was decided to create this in-house for a number of reasons:

1. the real work is in the narrative and that EQE’s time will have to be allocated it regardless of the technical creation,
2. the technical side was easily within our remit,
3. to optimise our 1 day “web test” offering,
4. to ensure that we get what we wanted and
5. to provide a topic for our first blog.

From a management side, it was decided that an incremental approach would be utilised to build the site. The rationale for this was two fold: to ensure content is released as soon as possible for the benefit of EQE and the realisation that the web-site will always be a “live” project  and never be finished. We used time-boxes to identify and track our immediate requirements per incremental release.

The first white board session was to examine the basic requirement – “build a fresh website”. The outcome of this initial white-board session was: to develop a web site that was professional looking, easy to maintain, W3C compliant, relevant to our target audience, adhere to EQE’s web standards, establish the level and scope of testing involved, identify the risks, establish the non functional requirements, identify the configuration management approach, score a high percentage in EQE’s 1 day web test (greater than 80%), implement search engine optimisation, develop a secure test environment, allow for audience feedback, allow for blogging and so on……

So, (like most software projects), we quickly went from an easily understood and simple customer request of building a fresh website to several high level and diverse requirements in a very short space of time…… and the customers of software projects question why the development of software takes so long???

The increase in requirements listed above is an example of the need for contiuous communication. All of these requirements, plus more, came from white board sessions between just 2 people in the same company, (who understand and adopt appropriate techniques for requirements elicitation and management). Furthermore, even before the start of the first white board session, the question about the use of language in requirements is raised – the use of subjective terminology and the typical problems associated with ambiguous requirements. Simply – first question is – define ‘fresh’….?

The first blog

Hi, this is my first attempt at a blog and I want to talk about the website that is a work in progress for EQE.

First thing first, my name is Barry McManus and I am a consultancy partner and MD of Empowerment Quality Engineering Ltd (EQE) – www.empowermentqe.com. This is the blog for EQE. Colin Dinan is the other consultancy partner in the organisation and between us, we will be ‘blogging’ away about topics that are affecting our working lives.

I am not sure how to write a blog but here it goes….. hopefully we will generate some dialogue and thoughts…