The method compared the interventions based on their cost-effectiveness, impact on health inequalities, and reach. These characteristics were estimated as follows:

  • Cost-effectiveness: A literature review was undertaken to identify high quality evidence of the effectiveness of the intervention. Decision models were constructed to extrapolate estimates of the effectiveness into estimates of QALY gain and health cost savings associated with the intervention. These were compared with the cost of the intervention to estimate its cost per QALY gained.
  • Health inequalities: Data on the prevalence of health problems and eligibility criteria were employed to estimate the proportion of disadvantaged groups who would be reached by the intervention and the proportion of the whole population who would be reached by the intervention. Health inequality impact was defined as the ratio of these two estimates.
  • Reach: Data on the prevalence of the health problem being targeted was combined with estimates of the proportion of the target population likely to receive the intervention.

A survey was undertaken to determine how decision makers chose between interventions with different levels of cost-effectiveness, health inequality impact, and reach. The results of this survey were used to combine the characteristics of the intervention into a priority score.


A Multi Criteria Decision Analysis (MCDA) approach was employed to develop a prioritisation method. MCDA is a method for comparing interventions across a range of attributes, such as cost-effectiveness, and impact on health inequalities.

MCDA approaches vary according to the source and nature of information used to inform decision making. The following steps, however, are common to all MCDAs:

  1. Identifying interventions to evaluate
  2. Identifying criteria against which to evaluate the interventions
  3. Measuring the interventions against the criteria
  4. Combining the criteria scores to produce a ranking of each intervention

By breaking decision making down into these four steps, MCDA approaches provide an open and explicit basis for decision making and a framework for combining a decision maker’s objectives and values with expert measurement of performance (CLG, 2009).

A key characteristic of MCDA approaches is what is referred to as ‘the socio-technical system’, or the balance between decision maker input and expert measurement. The different components of the MCDA be classified as either deliberative or data-driven. Deliberation refers to the process of negotiation between various stakeholders, based on factors such as their own knowledge of the field, existing policy commitments, ethical values and so on. Data-driven components are those which are primarily based on research evidence, such as assessments of the clinical effectiveness or cost-effectiveness of particular interventions. The balance between deliberative and data-driven components adopted by the prioritisation method are summarised below.

The sources of data employed in the prioritisation method

Identifying interventions The interventions had to be of interest to decision makers. and have been the subject of reviews of effectiveness and cost-effectiveness.
Identifying criteria Criteria had to be identified by decision makers and be measured quantitatively.
Measuring criteria   The criteria included in the analysis were measured through a combination of evidence reviews and decision modelling.
Combining criteria Criteria were weighted based on the results of a discrete choice experiment undertaken with decision makers.

The balance between deliberative and data-driven components in the MCDA was chosen based on the following principles:

  1. The interventions included should be of current interest to decision makers.
  2. Decision makers’ opinions are a legitimate form of value or preference.
  3. The analysis should draw on data where decision makers’ knowledge is limited.


CLG (Department for Communities and Local Government) (2009) Multi-Criteria Analysis: A manual. London: Department for Communities and Local Government.

 Back to previous page

All content is copyrighted. Please do not duplicate or redistribute without our permission.

14 December 2018 11:06 Health England Leading Prioritisation vaspiraHub 2018.7760