EuroSTAR Scoring Instructions

Scoring Instructions for Review of Conference 2019 Submissions

Please be aware of the following instructions before reviewing allocated submissions. We ask that you provide your reactions to the proposal in the comment box and score the proposals (on a score of 1 – 5, 5 being best) as per the criteria outlined below:

Comments come first!

Describe your reactions or feelings to the proposal based on:

  • First impressions
  • Reflection
  • Final assessment

Please also comment on the suitable audience for this session (see general notes at the end):

  • suitable for beginner, intermediate, or advanced audiences

Scoring Criteria

Here are the scoring criteria by which we would like you to evaluate the proposals, followed by general notes to consider when scoring. You’ll provide scores for the submission:

  1. engaging
  2. new ideas
  3. scope
  4. relevant to theme
  5. overall feeling

1. Engaging

The greatest weight will be given to this section. Give the proposal a higher score to the degree that it does one or more of these things:

  • Raises an interesting, open question that will prompt discussion
  • Tells a compelling story about experience
  • Presents a compelling theoretical insight
  • Engages the participants
  • Has exercises or interaction
  • Challenges existing thinking
  • Provide inspiration or motivation to practitioners

2. New Ideas 

Score the proposal more highly if it:

  • Contains original or new ideas
  • Presents experience of new technological challenges or techniques
  • Introduces useful ideas from other disciplines and different perspectives
  • Presents an interesting new technique or application of tools

3. Scope

Consider the proposed scope versus presentation time. For example, a typical session will be either 30-40 minutes with 5-15-minute period for questions or 15 with 30 minutes discussion, score the proposal more highly if it:

  • Offers clear and limited set (1-3) of strong ideas
  • Does not risk overwhelming the audience with too many ideas
  • Can be delivered within the time that’s requested for it

If you think the submission would be better considered as a different type of session, then put that point in your comments.

4. Relevant to the theme

The theme of the conference is “Working Well”. This can mean many things, and as a reviewer you can read the call for submissions and get a sense of what it means to you. Be aware as you review that you are not looking for submissions that claim, “I have the perfect solution that can be used in any project” or for submissions you agree with. Instead, as there are many interpretations of “working well” and many ways to work well, we want to see a wide range of ideas and approaches incorporated in the conference. We are also interested in ideas of working well from outside testing – lessons we can learn for other disciplines and industries.

Score by your own interpretation or more highly if it involves any of these aspects:

  • is based on experience
  • examples of learning
  • has a subjective and unique angle
  • is based on academic research or experiments
  • shows learnings from other disciplines and industries
  • interprets “working well” in a new and interesting way
  • includes things you want to know more about to help you work well

5. Overall Feeling

How much do YOU want to see this talk? Score the proposal more highly if it compels you or grabs you, even if you can’t quite put your finger on why. Would you circle it in your program to attend it over other sessions?

 

General points about scoring

Points Against

Your comments and scoring should also expose the problems you identify in the proposal. Score the submission lower if, for example, it:

  • Represents an attempt by a marketer to sell a technique, method, product or tool that makes too-good-to-be-true, unsupportable claims
  • Is off-topic, not relevant or has no value to testing practitioners
  • Contains tired material or shopworn ideas; for example, ‘what is scrum’ may be a topic that is already covered into other more advanced talks
  • Seems incoherent or unfocused – ‘all over the dartboard’
  • Attempts to present too large a range of ideas or solutions to problems
  • Presents a polemic view against or for something without offering actual proof of experience, or at least a reasoned argument or viable alternative

Different types of session may require different criteria:

Type of sessionSpecial points when reviewing submissions
Tutorials (full day or half day)Score higher if:

–          Evidence of a track record for presenting tutorials

–          Topic you / your manager would pay to go to yourself

–          Topic you / your manager would pay for your team to attend

–          Hot topic

–          Essential skill

–          Materials will fit in the time for the tutorial

–          Hands-on/ exercises/ discussions/ experiential

Score lower if:

–          Too much material/content for the time

–          Too little material/content for the time

–          No evidence of previous presenting/teaching

–          Topic is not one you would pay to go to…

 

If you think the session would be better as a workshop during the conference, tell us

If you think this is a tutorial, but longer/shorter than applied for, tell us

KeynotesScore higher if:

–          Evidence of track record as an excellent speaker

–          Visionary/ forward looking / controversial/ will provoke discussion

Workshops / discussions / special sessionsScore higher if:

–          Topic is new to the conference / forward looking / controversial/ will provoke discussion

–          Author does not offer a solution but an opportunity to solve problems/ discuss issues/ do something hands on in a group

–          Offers fun and learning

Score lower if:

–          Just a presentation asking for a longer time slot

If you think a workshop would make a good tutorial, let us know

Track sessionsIf you think a track submission is worthy of consideration as a keynote, let us know.  Make a note of brand-new speakers; if their story is potentially a good one, is it worth working with them to provide mentoring and/or improvements to their abstract?
New Generation KeynotesScore higher if:

–          Visionary idea

–          Great story

–          Speaker has never keynoted and has great scores from a previous EuroSTAR/UKSTAR

–          Speaker has never keynoted and has provided links to videos of their presentations which you really enjoyed watching

–          You really want to hear this story/idea

Score lower / disqualify if:

–          You know this speaker has previously presented a keynote at a conference (disqualify)

–          The submission story is mundane

–          The speaker scores offered in evidence are under 8/10 or equivalent

Tell us if you think this should be a track if it does not make NGK

 

Type of audience

Make sure the submission fits the selected audience type

Type of audienceSpecial points to note
IntroductoryTalks marked as introductory assume no previous experience of the subject matter. If this is marked as an introductory talk, ensure the content is introductory. If it is a topic that is your expertise area, comment on the proposed content, without needing to agree with it. If it is not your expertise area, would you go to it?
IntermediateShould be focused on suitability for most conference attendees
AdvancedTalks marked as expert should move the discourse on from current perceptions. If you are an expert or practitioner in this subject area, will you have learnt something new as a result of this talk? Will it have stretched your brain and challenged you?

 

 

Download PDF Copy of EuroSTAR Scoring Instructions 2019

 

Review ZoneBack to Review Zone