The report on the evaluation of the judicial systems of the Council of Europe's 47 member States is based on an evaluation scheme devised to ensure maximum homogeneity and comparability of the data at European level. The scheme's 170 questions: quantitative, qualitative, matrix-based questions, open questions – generate a database providing a wealth of information and opening the way for a multitude of analyses.
The data were collected by means of an electronic questionnaire developed by the CEPEJ. In each Member country, a national correspondent responsible for coordinating the replies served as the main point of contact throughout the evaluation process. On the CEPEJ side, a scientific expert was appointed to supervise the coherency of data and the analysis and processing of statistics. This person dealt with the national correspondents and monitored and updated the database on an ongoing basis. The vast majority of national correspondents were asked to supply additional information or explanations with regard to the replies submitted. Any amendments to replies were approved by the national correspondent concerned and recorded by the expert.
Before undertaking statistical analysis, substantial data-checking work was carried out. This vital phase, which is part and parcel of any survey-based study, made it possible to eliminate all kinds of errors: material errors, typing errors, omitted replies, situations that were not applicable, misinterpretation of questions. Each reply in the electronic questionnaire was checked for each country. In addition, the replies were compared with the replies from the 2004-2006 evaluation cycle. This comparison helped to identify changed situations or modified procedures, laws and structures and to ensure that replies were accurate. Further explanations were requested to clarify these changes.
At this stage of the evaluation process it was not possible to exploit the trends-over-time aspect as fully as it could be. Comparison with the 2004-2006 data made it possible to report variations but in principle served to ensure the quality of data with a view to creating uniform time-sets that could later be used for longitudinal analyses. This comparison was of great importance as it highlighted situations, variations or modifications which national correspondents were unable to isolate or grasp. Thanks to numerous exchanges with national correspondents, methodological problems were resolved and major variations in figures were explained.
While work to check data coherency was necessary, it delayed statistical analysis and the production of tables and graphs. Data stability was definitively attained only by July, leaving little time for the more complex analyses. This third report presents judicial systems chiefly through multiple rates, ratios and tables and graphs grouping the replies. Owing to the highly individualised nature of data the approach adopted tended to be descriptive. A few attempts were made at factorial analyses, leading to the identification of clusters (groups of countries) presenting major similarities or differences. New indicators such as "disposition time" were devised and integrated in the report.
Ultimately, thanks to the combined efforts of all the national correspondents and the team of experts within the CEPEJ, all the questions were exploited and presented in the report.