Quality Assurance & Labelling

Traditions are hard to change. The ORPHEUS Labelling and Evaluation Certificate procedures provide graduate schools with tools to persuade stakeholders of the benefits of improving doctoral training.

ORPHEUS Self-Evaluation / Labelling

Purpose

The purpose of these tools is to assist institutions to reflect on their doctoral training programmes, by providing institutions with a comprehensive and flexible procedure to assist this process of self-reflection. This procedure will lead to award of an ‘ORPHEUS Evaluation Certificate’ and possibly also an ‘ORPHEUS Label’.

The procedure is based on the ORPHEUS/ AMSE “Best Practices for PhD Training”, 2016. This represents a broad consensus, developed over many years, between around 100 biomedical and health science institutions all across Europe concerning the aims for the outcomes and content of PhD programmes.

The procedure has three stages. It will be the decision of the individual institution as to how many stages they wish to use. The procedure is organized by the ORPHEUS Labelling Board that consists of persons who have practical experience as heads of graduate schools. Whether all stages of the procedure are conducted or not, it is the experience of ORPHEUS that institutions find all aspects of the evaluation of substantial benefit.

Download the ORPHEUS Self-Evaluation Form

 

Testimonials

With this token of motivation, we will work with ORPHEUS for a constant improvement of graduate education in our institution. 

Mannheim, 2016

We take note of the recommendations for further improvement of the program. 

Bergen, 2016

The process of evaluation, particularly the site visit was considered extremely useful by all participants (students, supervisors, administration). The suggestions have been very useful and actions are already in place to follow the advice provided. 

Minho, 2019

I would like to thank ORPHEUS, for your valuable guidance to us in achieving our goals. 

Dokuz Eylül, Izmir, 2015

Labelling Board

The Labelling Board is assisted by many Special Consultants who also have experience of running graduate schools. They are listed further down this page.

Prof Michael Mulvany

Prof Michael Mulvany

Labelling Board Chair

Aarhus University, Denmark

Prof Gül Güner-Akdogan

Prof Gül Güner-Akdogan

Labelling Board Member

Izmir University of Economics, Turkey

Dr Janet Carton

Dr Janet Carton

Labelling Board Member

University College Dublin, Ireland      

Prof John Creemers

Prof John Creemers

Labelling Board Member

KU Leuven, Belgium                             

Prof Roland Jonsson

Prof Roland Jonsson

Labelling Board Member

University of Bergen, Norway              

Prof Zdravko Lackovic

Prof Zdravko Lackovic

Labelling Board Member

University of Zagreb, Croatia             

The Evaluation Procedure

Stage 1

Completion of an ORPHEUS self-evaluation questionnaire and submission of the completed questionnaire to ORPHEUS. It should be clarified which stakeholders have been involved in completing the questionnaire.

Stage 2

Evaluation of the questionnaire by two members of the ORPHEUS Labelling Board, or other persons appointed by the Board, who will provide the institution with feedback about compliance/non-compliance with the ORPHEUS recommendations and suggestions about how the programme could be improved. 

 

 

Stage 3

To obtain a full picture of the institution’s PhD programme, ORPHEUS can organize a 2-day site-visit to facilitate the self-reflection process. The facilitators will be persons with practical experience as heads of graduate schools appointed by the Board. During the site-visit the facilitators meet with relevant stakeholders, e.g. head of graduate school, dean, head of graduate school administration, programme directors, supervisors, PhD candidates, etc.

The ORPHEUS facilitators will provide a list of recommendations regarding the extent to which the institution’s PhD programme complies with the ORPHEUS recommendations and suggestions for the institution to consider regarding measures to possibly improve the programme.

Institutions completing Stage 2 may be awarded an ORPHEUS Evaluation Certificate. Institutions that on the basis of stage 3 are found to comply with the Basic ORPHEUS recommendations may be awarded an ORPHEUS label.

The procedure may be iterative. For example, if an institution does not fully comply with the recommendations under stage 3, then it may make the requisite changes and re- apply for consideration under stage 2 or 3.

These ORPHEUS services are provided free of charge, but institutions are asked to cover the costs of the site-visit. Institutions with limited budgets may apply to ORPHEUS for waiving all or part of these costs. 

 

Benefits of Using ORPHEUS Tools

Implementation of these changes to allow PhD training to meet the needs of modern society often requires changes in tradition, and traditions are hard to change. The ORPHEUS Self-evaluation, Evaluation Certificate and Labelling tools assist graduate schools to persuade stakeholders of the benefits of improving graduate training.

This is appreciated as indicated by the many testimonials received from recipients of Labels or Evaluation Certificates, for example:

  • The process of evaluation, particularly the site-visit was considered extremely useful by all participants (students, supervisors, administration). The suggestions have been very useful and actions are already in place to follow the advice provided. Minho, 2019
  • With this token of motivation, we will work with ORPHEUS for a constant improvement of graduate education in our institution. Mannheim, 2016
  • We take note of the recommendations for further improvement of the program. Bergen, 2016
  • I would like to thank ORPHEUS, for your valuable guidance to us in achieving our goals. Dokuz Eylül, Izmir, 2015
Best Practices and Recommendations Repository

The Best Practices document includes

  • Basic Recommendations: Recommendations that are thought to be particularly important.
  • Quality Development: Further recommendations that are in accordance with international consensus about good practice.
  • Annotations that are used to clarify, amplify or exemplify expressions in the recommendations, and also to indicate flexibility

The recommendations relate to all aspects of graduate education – including research environment, outcomes, admission policy and criteria, PhD training programmes, supervision, content of PhD theses, thesis assessment, and graduate school structure. The document has been developed in collaboration with around 100 graduate schools from all over Europe and found to be relevant not only for those with large resources but also for those in the process of developing their PhD training.

ORPHEUS Self-Evaluation Form

This is a tool to allow institutions to assess the extent to which they comply with the ORPHEUS Best Practices document. This gives institutions a basis to discuss and reflect on their PhD programmes and decide if improvements might be made. The effectiveness of the process will likely be enhanced if relevant stakeholders (e.g. PhD candidates, supervisors, research directors, graduate school administration, graduate school leadership) are involved in completing the form.

ORPHEUS Evaluation Certificate

Institutions can apply for an ORPHEUS Evaluation Certificate by submitting a completed self-evaluation form. The completed questionnaire will be evaluated by two experts appointed by the ORPHEUS Labelling Board. These experts will provide the institution with suggestions about how the programme could be improved. 

Institutions that have satisfactorily completed this process receive an ORPHEUS Evaluation Certificate.

This ORPHEUS service is provided free of charge to members of ORPHEUS.

ORPHEUS Label

Graduate schools that believe that they comply with the recommendations of the ORPHEUS Best Practices can apply for an ORPHEUS Label. To do this they should submit a completed Self-evaluation form. If this is generally satisfactory, the Labelling Board will then appoint two evaluators who will make a 2-day site visit to meet with relevant stakeholders, e.g. head of graduate school, dean, head of graduate school administration, programme directors, supervisors, PhD candidates, etc. The evaluators will then make a report to the Board, either indicating changes that need to be made or else confirming that the recommendations of the Best Practices are complied with allowing the award of a Label.

This ORPHEUS service is provided free of charge to members, but institutions are asked to cover the costs of the site visit. Institutions with limited budgets may apply to ORPHEUS for waiving all or part of these costs.

Institutions that have Received the ORPHEUS Label:

  • Belgrade, Serbia
  • Bergen, Norway
  • Copenhagen, Denmark
  • Dokuz Eylul (Izmir), Turkey
  • Manitoba, Canada
  • Graz, Austria
  • Hacettepe (Ankara), Turkey
  • Iceland
  • Kocaeli, Turkey
  • Lund, Sweden
  • Minho, Portugal
  • Karolinska, Sweden

Institutions that have Received the ORPHEUS Evaluation Certificates

  • Amsterdam Medical Centre, NL
  • Cardiff, UK
  • Gazi, Turkey
  • Gothenburg, Sweden
  • Leuven, Belgium
  • Liverpool, UK
  • Manitoba, Canada
  • Manheim, Germany
  • Oslo, Norway
  • Sumy, Ukraine
  • Vanderbilt, USA

Further Information

To initiate the process or for further information please contact us.

Robert Harris, Robert.Harris@ki.se

Michael Mulvany, mjm@biomed.au.dk

Gül Güner Akdogan, akdogan.gul@ieu.edu.tr

Examples of Substantive Changes Resulting from Labelling

  • Increasing time for research project from 2 years to 2½ years.
  • Making course-work more research oriented and reducing extent.
  • Introduction of supervisor training courses.
  • Introduction of clear independence of the assessment committee and removal of supervisor as voting member.
  • Increase in number of paper equivalents required for thesis.
  • Introduction of a mandatory course in ethics.
  • Introduction of a clear limit to the length of the training.
  • Master’s or equivalent level becoming requirement for admission to PhD training
  • Increased central control over the individual programmes to ensure more uniformity

 

Special Consultants

Ana Borovecki, Zagreb; André Nieoullon, Paris; Hakan Orer, Istanbul; Helga Ogmundsdottir, Reykjavik; Joana Palho, Braga; Miroslav Cervinka, Prague; Nebojsa Lalic, Belgrade; Oksana Sulaieva, Kiev; Peter Holzer, Graz; Robert Harris, Stockholm; Rudolph Schubert, Augsburg; Seppo Meri, Helsinki

 

Contact Us

Šalata 3b
10000 Zagreb
Croatia

Become a Member

Join our network of over 119 organisations