go back to the blog

The Joy of Reviewing

  • 01/03/2017
  • no comments
  • Posted by Ard Kramer

Over the past week I have been reviewing the abstracts for EuroSTAR. It is my second year as a reviewer and I have found it very it interesting and fun to do. The procedure is clear and simple.

First of all, you must review anonymous papers i.e you don’t know who is on the other side. This also means that the writer of the abstract must be aware that they, as a speaker, don’t just get accepted onto the programme because they you have proven to be a great speaker in the past. Therefore, every applicant must make a good impression. The quality of submissions differ immensely: from very short (too short!) papers who give only two word abstracts to great big abstracts which gave me a great indication of what the speaker will be presented. (Just to be clear – EuroSTAR Conferences also made sure that reviewers are not responsible for reviewing their own submissions, or those of people in their organisation).

Secondly, it is up to the program chair to give the reviewers instructions. These instructions by the chair, Iris Pinkster – O’Riordainhelp the reviewers to identify what is important for the 2017 conference in Copenhagen. These instructions reflect the categories we can score, such as: is the paper on or off topic? But also, is it a new idea? We have five categories in total to score the submissions on and in the end, we submit an an overall score which the programme committee can use as input when picking the upcoming EuroSTAR programme.

Of course, I can not tell you anything about the very interesting and not so interesting content I read, but I can you give you some general feedback:

  • A good paper is well written and to the point (if it is not ‘smoothly’ written the reviewer is distracted, so always ask somebody at home to read it before you submit it to EuroSTAR)
  • Make sure you highlight a good combination of points you want to address in your talk and add a few (short) examples. This gives a good impression to the reviewer of what you’re going to present. Also, be aware that in a presentation has time restrictions so be realistic about the amount of points you plan to cover.
  • There were papers that had great practical content so I was able to make a visualisation about what is going to be presented: these are great papers and fun to read!
  • There were a few short papers that gave me the impression that you applicant started, but forgot to finish. This doesn’t mean that a short paper can not be interesting, but it has to have a head and a tale.
  • I had a few papers where testing was not a part of the content. I’m sorry,  but as this is for a testing conference, i’d expect there to some link (any link can do) with testing.
  • I have had the luxury of attending quite a few conferences over the past few years. This also means that I have seen a lot of testing presentations. Therefore, if an applicant puts forward an idea that is new or unique, it certainly gives them a big plus when I am reviewing their paper.
  • After every review, I add some feedback to explain the score. The feedback can be passed on to the applicant if they are unsuccessful in their bid to get on the programme and hopefully it will help them shape their application for future EuroSTAR Conferences. 
  • I must say, reviewing is fun to do! It is inspiring to see so many people trying to get their message across, and although I can be very critical on the content, I appreciate the effort of each applicant. I know from my own experience how hard it is to write and it is even harder to make the program. Not because your paper isn’t great, but because of the sheer volume of applicants (over 550 applications in 2016 for 60 speaking slots!)


Of course, others will review the same applications that I do because as testers, we know for certain that we have biases and that we make assumptions.  All these reviews & scores can help the programme committee to make decisions regarding the EuroSTAR programme 2017. After reviewing some interesting papers, I wish the committee the best of luck in picking the programme of course, I hope that I’ll we also on the right pile of papers ;-)

Blog post by

go back to the blog

Ard Kramer

Ard is a principal Test consultant at Alten IT since 2008 and started as a tester in 1997. He is focused on new developments in testing and looks how innovations can be used in testing. Therefore he was a speaker at different conferences, such as EuroStar, Sigist, Testnet, Expo:QA, CAST and the Belgium Testing Days. He likes to have a broad view on testing and therefore he uses the knowledge and experiences form other fields of expertise. For example: sports. Ard has many years of experience as a sport coach (volleyball). As a sport coach, he must keep up with the latest developments and therefore he followed a course “sport psychology”. With this knowledge he saw a connection how to improve as a tester and therefore he wants to present his insights to the testing community.

Leave your blog link in the comments below.

EuroSTAR In Pictures

View image gallery