PA Awards - Evaluation Guidelines
Important Aspects for Assessors and Jurors
Your engagement as an assessor or juror, and the decisions that you make in the process, will have an impact on the awards and its organisers, the individuals whose work you evaluate, and your colleagues who join you as assessors or jurors.
- Before accepting the invitation to serve as assessor or juror, please familiarise yourself with the application rules and the assessment criteria established by the award organiser.
- Your agreement to serve as an assessor or juror establishes a bond of trust between you and the award organiser. The consequences, should that trust be broken, are outlined in this document.
- The assessment and judging process for the competition should be clearly understood and carefully observed.
- As assessor or juror, you are contributing to the success and further development of the award and its improvement for future editions. You will be encouraged to share observations and recommendations about the methodology and process between the pool of colleague assessors and jurors and the organisers of the Award. By discussing the methodology and process and agreeing on the criteria and procedures, you are significantly contributing to the success, transparency and fairness of the award.
- Assessors and jurors need to state a potential conflict of interest with any of the submitted applications and will not be able to evaluate, vote on or comment on these applications. These applications will be assigned to another assessor or juror.
A conflict of interest is possible if the application is submitted by an organisation from the same country as the assessor or juror, or an application where the assessor or juror has been directly involved in any project phase, or when there are close relationships (family or work related) with the people involved in the PA Award.
- Assessors and jurors must treat all information about the products, applicants, process and results as strictly confidential.
- The evaluation process and voting will be conducted via the Public Administration Awards in the Western Balkans platform.
Evaluation Process - Objectives
- Consensus based
Persons (Jurors, Assessors):
- Gender balanced
- Professional expertise
- Balanced in terms of national and regional background
Excellent, inspiring, practical solutions (tools, systems, approaches) from the region, demonstrating the diversity within countries, creating a benchmark and learning opportunity for the region.
Phase 1: Eligibility CheckAward secretariat will assess all submitted initiatives confirming that:
- The initiative has been submitted by a public sector institution from Albania, Bosnia and Herzegovina, North Macedonia, Montenegro, Serbia or Kosovo*.
- The application form is filled out completely and adequate information has been submitted to permit evaluation of the initiative.
Clarification: “False applications” and applications that are not relevant to any of the categories will be disqualified.
- Applicants whose submission is incomplete will be contacted by telephone or email by the award secretariat in order to provide further information.
- Notification of ineligible applications will be given by the secretariat.
Phase 2: Screening and Interview Phase for Assessors
Phase 2a – Assessment Phase: initial screening, overview of all applications and adjustment of methodology
Introductory video call with all assessors and award secretariat:
- Explaining the methodology, process and platform functionality to the assessors.
Depending on the number and quality of submissions, the assessors may adapt the process in agreement with the award secretariat.
- Forming the assessor teams, depending on the number of applications (each submission will be evaluated by two assessors).
- Teams may change if an assessor states a conflict of interest for an assigned application.
- Agreeing on a schedule for video calls, discussing the process, criteria, methodology, answering questions.
- Introducing the awards platform and its technical functions.
Assessors start screening all submitted applications that meet the technical awards criteria and start to:
- Note questions and comments for each submitted initiative.
- Note comments about unclear or missing information in the platform.
- Assessors submit their individual votes using the platform.
- Assessor teams discuss their votes, rank applications and comments and agree on:
- applications that need a certain clarification, more information, etc – the award secretariat will collect this additional information from the applicants.
- the best applications that they have evaluated and that should be considered for the interview and data collection phase.
- The number of applications considered in phase 2b may vary between the teams of assessors, depending on the quality of the randomly-assigned applications.
Phase 2b - Interview Phase: telephone calls with the most promising applicants
- The award secretariat will schedule calls between applicants and assessors to find out further information, answer open questions and get a feeling for the team behind the initiative.
- The award secretariat will provide a questionnaire for these calls and schedule the calls between assessors and applicants, supported by an online tool.
- Assessors take minutes during the calls and upload them on the contest platform.
- After the calls, assessors review their votes through the platform.
Phase 2c - Shortlisting Phase: award secretariat and assessors
- The award secretariat reviews the final votes from the assessors in a combined list of rankings and recommends a list of shortlisted projects (reflecting the quality and diversity of projects).
- Assessors will have the opportunity to speak for/against specific applications and agree on the suggested shortlist to be presented to the jury.
- The award secretariat informs all shortlisted projects and those not selected and announces the finalists/shortlist on the awards webpage.
- Assessors prepare the list of rankings of projects with brief comments and explanations and present the suggested shortlist to the secretariat, including comments on why an application was or was not selected for the shortlist.
Phase 3: Jury Meeting
In the case of a small number of applications (up to 20), the entire process will be conducted by the International Experts Jury as explained in the description of the Phase 2 (2a, 2b and 2c).
The awards secretariat will provide support with telephone calls, requesting information, etc.
Jury meeting with international experts (on-site), resulting in the decision on the winning initiatives of the first Public Administration Awards in the Western Balkans (depending on the coronavirus (COVID-19) measures, this meeting might take place online):
- The award secretariat presents the methodology and process to the jury members.
- The award secretariat presents the complete list of rankings to the jury members and presents the shortlisted solutions.
The jury of experts can request the removal applications from the shortlist, or their inclusion, if there is agreement by the majority of the jury members and sufficient justification.
- All agreed shortlisted applications will be presented by the assessors to the jury for evaluation, including information on:
- why the initiative could be a winner
- why is it better/more effective than others
- what was the concrete challenge and solution
- any numbers, impact achieved, further information
- the local context.
- The Jury is responsible for discussing all selected applications in terms of award criteria and carefully considering all of the parameters of the initiatives.
- The jury can access each application through the platform and review all submitted material, including the comments of assessors, the information gathered and the insights provided through the telephone calls.
- Following the presentations and discussions, jury members will vote on each project through the system.
- The secretariat will present the voting results to the jury, who make the final decision on award winners and provide the reasons for their decision.
Evaluation Criteria for Assessors and Jury Members
Evidence shows that the initiative would also improve the level of adaptability of a government in possible future situations of crisis.
The initiative must provide regulations, procedures, mechanisms and means to ensure its sustainability and resilience in the future.
The initiative must introduce new ways of working or delivering services in the public sector and an innovative approach in addressing problems occurring in extraordinary circumstances.
Effective & Impactful
Participatory & Inclusiveness
Transferable & Replicable
The possibility of replicating the project, partially or fully, in a different setting or context in the Western Balkan Countries during and after the crisis context.
How to Nominate Guidelines EVALUATION GUIDELINES DOWNLOAD
Sign in with the username and password you received from the contest organisers
Login link: https://db.icnm.net/Signup/PAA/2020/
Please note: For technical reasons, assessors and jurors will be both called “juror/jury” in the database.
Complete your profile before you start your evaluation
Evaluation overview page
Evaluation of an application
Overview for assessors/jurors – you now see that the evaluated project has been moved to “Evaluated projects”.
If you have any questions related to the rules, contest, etc, please contact:
If you have any difficulties with the awards database, please contact