Measuring Democracy

Printer-friendly versionSend by email

Measuring Democracy

Organizer:
Network of Democracy Research Institutes (NDRI)

Moderator:
Larry Diamond – U.S.

Rapporteur:
Sabine Donner – Germany

Presenters:
Asher Arian – Israel
Ghia Nodia – Georgia
Hauke Hartmann – Germany
Christopher Walker – U.S.

The workshop presenters provided an overview of two global, cross-national indices on democracy based on experts' analyses (Freedom of the World by Freedom House and the Bertelsmann Transformation Index by the Bertelsmann Foundation), and a national democracy audit conducted by gathering empirical data and conducting public opinion surveys (Israel Democracy Index by the Israel Democracy Institute). The goals of the three indices are comparisons with other countries and monitoring the development of democracy within a certain country over time.

Key questions:

  • Is cross-country comparison legitimate?
  • Subjective versus objective measures: How to ensure the reliability of measurement?
  • Is there a conceptual validity to the definition of democracy?
  • Should the findings be aggregated in a single measure?
  • Do the different indices complement each other?
  • Do the results bear comparison with the real situation in a country? Observations:
  • Participants agreed that serious efforts to measure democracy are indispensable as complex and problematic as they might be.
  • There is a rising demand for and a growing importance of democracy measurement. Reform actors from countries in transition and the increasing amount of money spent on democracy promotion trigger this demand to a certain extent. The indices are used more and more to justify the spending. The level of responsibility for the producers of indices rises with the attention paid to them. Therefore, questions of data reliability and validity have to be taken very seriously.
  • Many participants shared a concern for the impact that measurement efforts have on policy makers. While some stated that measuring democracy in a country can be used as a tool for shaming or encouraging political actors, others emphasized the danger that the findings could be instrumentalized by decision makers for their own ends.
  • Rankings and indices support the view that building democracy is a long-term commitment and therefore needs continuous monitoring. The pluralism of democracy measurement can help foster the ongoing debate on different concepts and definitions of democracy.
  • Some participants suggested that the indices should make a clearer distinction between established and new democracies or countries in transition. In fact, distinguishing between transformation and democracy consolidation, the Bertelsmann Foundation will publish a separate index on established democracies in the fall of 2007.
  • A stronger comparison of the results and concepts of existing indices was suggested.
  • Aggregating the findings in a single score was seen as a means of attracting attention and facilitating comparison, although it poses several questions about the weighting of different dimensions. Disaggregating the scores is inevitable in order to explain the various aspects of democracy in detail. While the Israel Democracy Index does not aggregate its data in the first place, the Bertelsmann Transformation Index offers access to all disaggregated scores. In addition, Freedom House will publish its sub-data for the first time in its next Freedom in the World survey.