Back home

Phase 3: Test

Estimated time – one to three months

The objective of the Test phase is to validate the challenge design elements developed during the Design phase in order to arrive at a challenge design that is likely to deliver on the objectives.

A crucial element of this phase is seeking feedback from stakeholders and experts in an iterative way to refine the design of the challenge. The challenge design should be tested with a range of stakeholders and experts including those involved in the Understand phase consultations (i.e. subject matter experts, innovators, in particular those that you anticipate would participate in a challenge, the people/groups who would benefit from solutions to the problem) and those that can or will assess proposed solutions (i.e. jurors).

Activities that can help test the validity of a challenge design may include interviews, workshops, events, or targeted online surveys. These efforts should be supported by research and data analysis conducted by the challenge team to calibrate the information and feedback being gathered from experts and stakeholders

The challenge statement

(where applicable in the challenge design)

  • Does the challenge statement clearly address the original problem?
  • Is the challenge statement defined in a clear and accurate way?
  • Will the statement incentivize innovators to participate in the challenge? • Does the statement provide an indication of the assessment criteria?

Challenge structure and incentives

  • Is the challenge timeline appropriate to allow for the development and testing of solutions?
  • Will the prize incentive(s) motivate innovators to participate?

Assessment Criteria

  • Do the assessment criteria set the appropriate barrier to entry for innovators?
  • Are the assessment criteria clear and effective?
  • Are the assessment criteria objective?
  • Are the assessment criteria measurable and comparable?
  • Are there criteria that create perverse incentives or are easy to “game”?
  • Are there criteria that are not feasible to achieve (especially given budget and timeline)?

Communications

  • Will the challenge structure, prize incentives, eligibility criteria and assessment criteria be easy to communicate to a variety of stakeholders?