Bloggo back to the blog
One Week – Moving to Sign-Off-->
The fourth of five weekly reports from Peter Morgan. Stakeholders sign-off is the aim for the week, as well as reducing the defects outstanding, and increasing the tests passed. The figures show that this was partially successful – but we achieved sign-off, so next week is the week of deployment.
Sometimes all we have are figures, and they don’t tell the whole story. Looking at the tests completed / test outstanding figures, it looks great. Everything has run. The defects raised and defects open, however, is not as good. Twelve raised in the week, and still eleven open. What is going on? Well neither is as good / bad as it appears. But the one residual System Test item has been closed – hurray!
The solution to be deployed next week is building upon and depends upon that which has gone hitherto. That is a solution that the current team developed and implemented. If any of us were to start again, with the knowledge we have now, it would not be the same. Take pushed reports from Business Objects. These are sent out by e-mail, giving the total records loaded, those that failed and feeds processed over the last 10 days (as an example). The recipients of this report are different for UAT and Production, and e-mail addresses are hard-coded into the Business Objects solution for each environment. The current development will now (as of this week) use e-mail distribution lists, so that any additions to or removals from the e-mail list need only an amendment to a distribution list (owned by business users), not a ‘code change’. Reports currently produced in Production are being changed, so some of the ‘defects’ are to cover this.
As of one week to go, there have been 29 UAT defects raised, and all but four are closed. One of these is where there is a difference between the UAT and System Test environments – something that works in System Test does not perform i the same way in UAT. Strangely enough, thus is another success/failure e-mail – the e-mail is not produced in UAT. None of the UAT defects outstanding is something that will prevent a Production deployment – and some others on the ‘open’ list will either be resolved by the forthcoming deployment, or are on the ‘futures’ list. It is not as bad as it looks.
On the test passed side, all data feeds successfully load into the data warehouse. There are one or two ‘data issues’, where customer number format needs some manipulation before it conforms to the company-wide unified structure – a new one of which occurred on the afternoon of the last day of the week. Just after the sign-off meeting, of which more below. The business users are in discussions about some presentation aspects of the reporting layer, but these are presentation aspects only – there is no indication that data is incorrectly entered into the warehouse, it is just the presentation in the reporting layer. Testing of the reporting layer always was firmly with the business, and formal UAT tests were very generic in nature. The generic tests have ‘passed’, but the details are still being explored! “If it passes, it passes; carry over the detail in sign-off caveats” – Brett Gonzales.
In the run-up to an implementation, it is only right and proper that there should be business sign-off, to formally approve the deployment, and the stakeholders agree that on the evidence of UAT, the software solution is fit for purpose. This was scheduled as a “Go/No Go’ meeting on Friday lunchtime. Let me make it plain. Any formal such meeting should not result in a decision that is a surprise. For two weeks, the joint message from the project team and stakeholders has been that the software solution gives business value. Even if some parts are not used in the first weeks (one or more of the (two) quarterly / (four) monthly data feeds are not exercised initially), sufficient value will be added. Reporting presentation aspects are being iteratively refined, but from the business perspective, are “there or here-abouts”. So the decision – “go” – was not a surprise. A simple sign-off document was available (summarising the business caveats around reporting presentation aspects), in place of the formal UAT test completion report – the latter is one of my tasks for next week.
Just after the decision, a few final tests were being executed in UAT. The UAT environment is as production-like as possible, but “automated user requested schedule initiation” is only available in Production, and the impending implementation has resulted in small schedule changes to accommodate this facility. Data files for April for three of the monthly feeds have just become available – these seem good to use as this final testing. BANG! There was a surprise in the data, one that was not expected, and would need some “manipulative amendment” within the ETL layer to accommodate. So even before the ink was dry on the sign-off document, a change is required before one of the monthly feeds for April can be loaded. It is prudent that we have scheduled a release for the end of the month (incidentally, to start preparation for future project phases). An amendment will have to be included for this aspect – but at least other data feeds can be loaded in Production, everything except this one feed for April.
The project team is small; two full-time and one part-time developers, business users, one tester (test analyst, teat lead, test co-ordinator, test manager; hold on – that is at least four. No it is just me!) Other people supply services and develop solutions as needed. In such a small team, everyone can seem like a single point of failure, and two things happened this week that were unwelcome. It is not good to have a Bank Holiday in the week before an implementation – OK, so we knew about it a long time ago, but it still seemed like a day ‘lost’. The second was that my nearest and dearest hurt her hip in an unexplained way. So on the morning of the Go/No Go meeting, I was in casualty until 02:30. Fortunately rest is the main treatment, there is nothing seriously wrong, but not going to work that day was not really an option. It could really have put the brakes on the deployment, which is now next week!
Peter Morgan is a testing professional who has been involved in the ICT industry for more than 30 years, and worked in the freelance marketplace for much of that time. His time has sometimes moved from testing to ‘development’, but he would add “always using the mindset of a tester”. He is passionate about testing and a firm advocate of testing qualifications. An enthusiastic speaker and author, Peter tries to base his output on hands-on experience, attempting to relate fine sounding ideas back to how it will affect Joe or Jane Tester in their everyday working lives in the war of attrition that we call software testing.
Peter is also one of the 2013 EuroSTAR Community Leaders.