Bloggo back to the blog
Status Report – The call is closed, reviews to finish this week-->
The call for submissions closed on 14 February and the wider review team, the programme committee and myself have been reading and scoring the submissions in readiness for the programme committee meeting on Monday 10 March. This year there have been 458 submissions which I believe is the most ever received. They are all stored in an online system which everyone has used to read, review and score the entries.
The volunteer reviewers are being asked to read and score around 20-30 submissions each. The goal is to get every submission reviewed at least once by someone outside the programme committee and myself. As far as I know, this has been achieved. The programme committee, with some trepidation are aiming to review all the submissions, paying most attention to the track sessions then looking at the tutorials. I am ploughing through the whole lot. Right now, I have 120 left to do. We have to finish scoring submissions by the end of the week.
I’ve reviewed lots of informal abstracts for the events that we run through Gerrard Consulting and maybe I see thirty or forty spread through the year. Reading 450 in less than four weeks is an altogether different challenge. I had believed it would be easy to identify the stand-out best entries and it would be a straightforward task to select the forty-four track sessions to pull together the programme in March. In my dreams.
I have to say, it IS easy to spot the poor ones. Some entries miss the point, sound boring, had little to say or were just too difficult to understand. But the duds represent less than ten percent of the whole. So that leaves the other 400+ to deal with. Now, I have seen a number of exceptional entries that really have stood out beyond the rest. But mostly, the rest of the entries are of a consistently interesting and high standard. I’m not just saying that – it is proving to be truly difficult for us to pick the winners.
What seems to happen is that a typical submission has an interesting title and description and the take-aways look OK and the speaker doesn’t seem to pushing a commercial agenda and so on. It could make a reasonably good track session. Time after time after time, the entries have reached this ‘acceptable standard’. But acceptable is not a high enough standard because this would leave us with two to three hundred entries on our short list. That would be just unworkable and defeat the purpose of scoring in the first place. So, inevitably, we have to look more closely for reasons not to score a submission highly. We have to look for faults.
Now, I share the feelings of the other members of the programme committee that picking faults in a short document that presents the story of possibly months of endeavour and experience is not very comfortable. We all feel that in some way, we are being a little unfair by being pedantic and nit-picking. But then again, it’s no different from trying to break some software that a developer has spent weeks or months trying to perfect. Even so, we have all felt a little uneasy marking down people’s best efforts in a cold-blooded way.
The review process is quite subjective for all of us, so the hope is that the gaps in our knowledge, our prejudices and biases are all averaged out. One fear is that this could leave us with a rather bland, averaged-out selection of sessions. But I think the risk of that happening is extremely low.
Next Monday, the 10th of March is when the programme committee of Alexandra, Pusser, Henk, Mike and myself get together with the EuroSTAR team in Galway. The goal is to select the top sessions and draft the programme from the top-scoring submissions – in one day. I fully expect there to be some debate and dispute and we will all have our own favoured sessions that don’t quite make it into the programme. Lorraine and Siobhan will keep us under control and on-task, I have no doubt. Then follows a week or two tying up the loose ends, shuffling the schedule around and finalising the programme.
If you submitted a session proposal for inclusion in the programme I am sure you are keen to know the outcome. So am I and the team. In my next EuroSTAR Blog, I hope it will all be settled and I’ll be telling you how great the EuroSTAR 2014 event in Dublin will be.
– Paul Gerrard, 4 March 2014.
Paul Gerrard is a consultant, teacher, author, webmaster, developer, tester, conference speaker, rowing coach and a publisher. He has conducted consulting assignments in all aspects of software testing and quality assurance, specialising in test assurance. He has presented keynote talks and tutorials at testing conferences across Europe, the USA, Australia, South Africa and occasionally won awards for them.
Educated at the universities of Oxford and Imperial College London, in 2010, Paul won the EuroSTAR European Testing excellence Award. In 2012, with Susan Windsor, Paul recently co-authored “The Business Story Pocketbook”.
He is Principal of Gerrard Consulting Limited and is the host of the UK Test Management Forum and the UK Business Analysis Forum.