35-370 / Improvement of health and social services organizations combining consensus processes for indicator development with methods of external assessment

GOP Information
Organisation sharing the GOP
Related practices from PaSQ database
Instituto Universitario Avedis Donabedian


Quality indicators
Audit system

GOP Description
Implementation level
Clinical settings
We applied our methodology in 25 projects covering seven different health/social care sectors: Assisted living facilities, elderly health care (nursing homes, convalescence and palliative care), care for people with drug abuse problems, mental health, abused woman, mental incapacity and physical handicapped people.
Total of 684 centres from 3 regions in Spain.
Improvement the quality of care provided by health and social services organizations using consensus indicators for the external assessment of health and social services.
All patients that receive services in the organizations involved in the project.
Our methodology follows a four-step approach: 1. stakeholder involvement and creating an enabling environment, 2. using standardized consensus methods for indicator development, 3. using rigorous external evaluation methods to assess results and 4. developing and implementing quality improvement initiatives.
Timeframe implementation
The first phases of the project (development of indicators, external assessment and development of improvement plans) can take around 1 and a half and 2 years. The implementation of the improvement plan varies, but it usually covers a 2 – 3 year period.
Implementation tools available
– Description of the methodology
– Several indicator sets (for all the clinical settings previously named)
– Training materials for external evaluators
– Software to introduce results of external evaluations
Please note that all these materials are in Spanish.
Implementation cost
Development of indicators:
– 15 – 20 working days of QI expert
– 6 – 8 days of 5 to 10 experts from the field.External assessment:
– 3 days per organizationImplementing QI initiatives.
– 5 to 10 working days of QI expert
– Field staff: varies depending on the goals
Method used to measure the results
The set of indicators developed by consensus is used in order to measure the results in two different steps of the project: 1) at the beginning, to establish the baseline and to identify opportunities for improvement and 2) after a 2- 3 year period working on the implementation of results. Data is collected by external evaluators, specifically trained for this project.
For the overall performance assessment we observe in all four sectors major improvements ranging from 8.9% to 65.6% (difference in average compliance on the set of indicators between baseline assessment and the reassessment after 2 -3 years)
Analysis of the results
Results show improvement in all organizations involved in the project, but the improvement is higher for organizations with a lower baseline score. This is positive because it means that the project promotes both improvement and homogeneization of services.
Implementation barriers
Did you find implementation barriers?
Please describe implementation barriers
– Some experiences only included the baseline assessment, due to lack of funding for the reassessment
– Political changes that imply a change in priorities could be a challenge for this kind of 2 – 3 years long projects
Describe the strategies used to overcome the barriers (If needed)
The project involves developing a set of indicators that is also useful internally as a basis for quality improvement, in case the second reassessment was not possible due to lack of funding or strategic / political changes.
Other information
Other information about the GOP that you would like to add (Link or attached document)
The methodology for this GOP has been published in a peer review international journal in English, what could help sharing the information with other MS: Hilarion P, Sunol R, Groene O, Vallejo P, Herrera E, Saura RM. Making performance indicators work: the experience of using consensus indicators for external assessment of health and social services at regional level in Spain. Health Policy 2009 Apr;90(1):94-103

The method is not limited to the settings where it has already been implanted, it could also be implemented in any healthcare and social care setting, both at local, regional and national level.