No. 9

November 2011

Council of Europe
F-67075 Strasbourg Cedex
cepej@coe.int

The aim of the CEPEJ is to contribute to improving the quality of justice and the efficiency of its functioning in the 47 Member States of the Council of Europe.

Editorial

Stéphane Leyenberger, Secretary of the CEPEJ

The work accomplished by CEPEJ on the evaluation of European judicial systems enjoys wide recognition today and is used in an increasing number of Council of Europe member states as a guide to public justice policies. We hope that with the invaluable support of the national correspondents, the members of the Working Group on the Evaluation of Judicial Systems and the scientific experts, the 2012 edition of the report, due in September 2012, will meet these expectations.

In parallel to this process of analysing the operation of judicial systems, CEPEJ is also interested in how their operation is perceived by the beneficiaries: the users (professionals and citizens seeking justice) and, ultimately, the community. This information, which draws on more subjective elements, is necessary to supplement the factual information and ensure that due account is taken of user’s expectations in the organisation and reform of judicial systems. A handbook has accordingly been produced, as part of the work of the Working Group on the Quality of Justice, to enable courts to conduct user satisfaction surveys, together with a training programme in which CEPEJ experts are at the disposal of interested courts to train their staff and help them to organise satisfaction surveys and analyse the results.

In setting up this training programme, CEPEJ members and experts have chosen to leave the meeting rooms of the Palais de l’Europe and reach out to judicial practitioners, who are the main beneficiaries of our Commission’s work. This project is actually not the only one falling within this new area of work. First, there is the CEPEJ “Roadshow”, which provides an opportunity for member states to organise information and exchange sessions for a wide audience of professionals based on the activities of CEPEJ. Secondly, a training programme in the use of CEPEJ’s SATURN tools for improving judicial time management is available to interested courts.

CEPEJ intends in this way to provide a better understanding and clearer explanation of its tools for improving the efficiency and quality of justice and to ascertain the views of professionals on the relevance of those tools, in order to build on them and make them more directly relevant in the day-to-day work of the courts. True to its line of action, it intends to remain an innovative body anchored in the day-to-day reality of judicial systems.

Stéphane Leyenberger
Secretary of the CEPEJ

File: Measuring the quality of justice: ascertainment of users’ expectations

Judicial systems have certain privileged audiences, such as their regular users in a professional capacity, or stakeholders, the academics studying their functioning or commenting on the decisions delivered, or their agents in person when reflecting on their own practices.

Parties to proceedings are occasional users, and set foot inside the law courts on rare occasions in their lives. Capturing the essence of their experience through opinion polls is a chancy undertaking, even if the sampling is representative of a given society. Rather, the representations of justice in a country are revealed by such surveys.

In fact it is not easy to record the opinions of parties who have effective experience to disclose. To do so, the Working group on Quality of Justice (GT-QUAL) has developed a standard instrument for measuring the satisfaction of real court users, in the form of a questionnaire administered “on leaving the hearing”, designed in such a way as to be usable in the 47 Council of Europe member states: the Handbook for conducting satisfaction surveys aimed at court users (CEPEJ Studies No. 14), adopted in September 2010.

In a few months, it has already undergone the ordeal by fire four times in three different countries: the judicial districts of Angoulême (France), Turin and Catania (Italy) have conducted their own satisfaction surveys on the basis of the questionnaire prepared by the CEPEJ. In the Netherlands, it has been conducted at national level. In the first two cases, it was a matter of questioning the users of a court of first instance, in the last those of a court of appeal.

While the complete results are still being analysed, the first inferences to be drawn are these:

- The tool works, and its practical use has proved its suitability;
- The survey proceeds all the more satisfactorily when users are supported in their effort of participation: if staff members of the court registries, judicial service cadets (trainee magistrates) or law, sociology or political science students are present to complete the questionnaire with justice users, participation is greater and the replies gain in quality;
- One original feature of this tool, the distinction between users’ level of satisfaction for a given criterion and the importance attached to that criterion, is well understood by the respondents.
- This distinction allows rapid determination of the fields where action is necessary, ie those for which the level of satisfaction is low whereas the degree of importance is high.
- Use of the questionnaire uncovers the shortcomings of the services offered to persons before the courts: a given service, believed to meet expectations, actually receives poor marks and should be improved.

These first successful experiments enable the CEPEJ to make available to the member states a tool which has the approval of its experts, besides being tested in practice. For greater convenience in its use, the CEPEJ offers practical help to interested courts, for example by offering them the assistance of an expert during the conduct of the survey, whether for implementation or for the subsequent analysis.

François Paychère
Judge at the Court of Justice (Geneva/Switzerland)
President of the CEPEJ working group on quality of Justice

(GT-QUAL)

File: Court users satisfaction surveys; first experiences in France, Italy and in the Netherlands

France: Angoulème Tribunal

Implementation of the CEPEJ questionnaire on the quality of justice at the Angoulème Tribunal de Grande Instance in June 2010

The Angoulème Tribunal de Grande Instance was the first court in Europe to implement the questionnaire drawn up by the CEPEJ’s experts.

In the minds of the judges and officials working at this French court with jurisdiction over the entire département of Charentes, which has a surface area of 5 956 sq. km and a population of nearly 340 000, the aim was not just to test the questionnaire but also to make it a full and integral part of a process of assessing and improving the quality of justice that has already been going on for several years now.

Accordingly, at the formal opening session of the court in January 2010, the heads of the court publicly announced their intention to implement the CEPEJ questionnaire in the form of a court project.

In March 2010, a working group made up of volunteer officials and judicial assistants (assistants de justice) was appointed to run the project.

In June 2010, two-member teams made up of one court official and an assistant or trainee went to meet all the users of the Angoulème law courts.

245 court users agreed to share their feelings about each of the matters raised in the questionnaire.

In their replies on their general perception of the justice system, most of the respondents said that the system was still slow and expensive and that communication was poor. Their trust in the justice system lay just above the halfway point (3.3 out of 6) as did the importance they attached to this (3.7).

The main expectations of Angoulème’s court users related mostly to security (5.3 out of 6 in the importance ranking) and the attitude of judges and clerks (4.6 and 4.85).

Therefore, even in a rural département, the court users had very high demands in terms of security when they came to our courts (5.3/6) and expressed considerable satisfaction with the security arrangements at the courthouse (5.3/6), stating that they appreciated the attitude and politeness of the security officers (5.1).

The expectations concerning an aspect that lies at the very heart of a judge’s functions, namely impartiality and politeness, were unsurprisingly high. We were reassured by the users’ overall satisfaction level in this area, which was 4.3 for the politeness of judges and officials, 4.17 for the clarity of the language used and 4.1 for the impartiality of judges when conducting the oral proceedings.

The most surprising thing about the replies to the questionnaire was the fact that users were least satisfied about the punctuality of hearings (3.61) and support for victims (2.4!).

On the latter issue, it was clear that the system for advising and supporting victims, which is highly developed and available on the spot, thanks in particular to an association with an office at the courthouse, was still not sufficiently user-friendly.

Similarly, there are clear inadequacies with the signposting at the courthouse and the way in which victims called to appear in court are actually dealt with.

User dissatisfaction concerning the punctuality of hearings seems to derive from users' failure to understand why they should be summoned at the beginning of hearings and then forced to wait for hours without any information on how it is likely to progress.

The court has promised to conduct an investigation into these two sources of dissatisfaction, which were mentioned at the formal opening session of the court in January 2011, and draw up proposals which should be applied in the course of 2011.

For the Angoulème Court, the questionnaire has been an invaluable tool and an integral part of the process of improving the quality of justice in Charentes.

It has borne out two of our key ideas:

– the first is that it is impossible to conduct a quality control procedure at a court without involving the public as it is clear that the professionals’ view of their organisation and practices can be far removed from the feelings of those for whom these systems have been set up;

– the second is that it is not always necessary to make use of an external auditing firm to conduct the assessment if motivated and willing staff can be called on, who are able to accept criticism and whose main concern is to improve procedures continually to meet their fellow citizens’ expectations.

Nicolas JACQUET
Public prosecutor
at the Tribunal de Grande Instance d'Angoulême

The Netherlands

A national court user satisfaction survey held in the Dutch courts

In the Netherlands there is a history of ten years of court user satisfaction surveys. In 2001 the courts started to conduct these surveys and in 2011 almost all courts executed this at least two times. In 2009 the assembly of court presidents chose one national survey for all the (19) district courts and the (5) courts of appeal, held simultaneously, under the condition of a minimal work load for the courts.

In 2010 a project group presented a plan in the assembly of court presidents. The plan enhanced a description of the products that would be delivered. Among those products were an individual report for each court and a general report including all the courts. A major advantage of the national survey is the ability of benchmarking: for example division civil-trade from court x with all divisions civil-trade in The Netherlands. Instead of a local survey which compares the different divisions within a court.

Tender
The project group published a tender on internet for the execution of the survey. The tender dealt with specific criteria for the reports and the methodology. The demands included the use of digital questionnaires for professionals that would participate and live interviews of litigants after their court session.

In preparation of the actual survey, the project group made two arrangements. They developed the questionnaire and approached general organisations for lawyers, public prosecutors and representatives of governmental bodies to hand in files with emails from the professionals (for sending the digital questionnaire).

Questionnaire
As a base for the questionnaire mostly previous questions were used, making it possible to compare data with previous surveys. Questions which delivered less response or from which the answered hardly differed from related questions were deleted. The new questionnaire is ordered along five themes: performance of organisation, reception in the courts, functioning of judges, general satisfaction and background questions. All questions can be answered on a five point scale, except for one open question (What could be improved?). One new question was added: satisfaction on digital accessibility.

Execution
The company which won the tender produced brochures for the courts, for the citizens in the courts and for the organisations which participated. This information was distributed early 2011. Subsequently the interviewers visited the courts to introduce themselves. The interviews were held between February and May. In this period the professionals received an email with the questionnaire. Some time after this they received a reminder. At the end of the research investigation period the company started drafting the reports. In the general report names of courts are mentioned, but only when there’s a significant difference to the average satisfaction on a certain aspect.

Results
The reports show high satisfaction with the functioning of the judges, but improvement is deemed necessary in the management of timeframes. The general report of all the courts is published at the end of September. In October the assembly of presidents will discuss the results and agree on initiatives for improvement. The presidents will also discuss if and when another national user satisfaction survey will be held.

Court user satisfaction survey The Netherlands: general results 2011

The survey was held in all the (19) district courts and (5) courts of appeal simultaneously.

Satisfaction litigants	(Very) Satisfied	Neutral	(Very) Unsatisfied	Don’t know/n.a.
Overall satisfaction	81%	13%	6%	-
Accessibility	67%	30%	3 %	39%
- Telephonic accessibility	72%	25%	3%	51%
- Rechtspraak.nl (website)	64%	32%	5%	60%
- Digital accessibility	56%	38%	7%	67%
Reception	56%	40%	4%	-
- Punctuality of the hearings	78%	9%	13%	-
- Waiting conditions	74%	16%	10%	1%
- Privacy of waiting areas	55%	21%	24%	1%
- Information upfront	71%	16%	12%	7%
Conduct of hearings	87%	12%	2%	-
- The judge allows litigants to have their say	87%	7%	6%	1%
- Preparation by the judge	82%	12%	6%	4%
- The judge listens to the various points of view	91%	5%	4%	1%
- The judge imagines himself in the situation of the litigant	76%	17%	7%	3%
- Perceived expertise	82%	15%	3%	5%
- Perceived impartiality	79%	15%	5%	2%
- Explaining the case	91%	6%	3%	-
- Treatment of the parties	86%	12%	2%	21%
Decision	77%	19%	5%	1%
- Founding of the decision	79%	12%	9%	1%
- Comprehensibility of the decision	84%	9%	7%	1%
Timeframes	55%	17%	29%	1%
Explaining how the case will be dealt with further	90%	6%	3%	3%

Satisfaction professional parties (lawyers)	(Very) Satisfied	Neutral	(Very) Unsatisfied	Don’t know/n.a.
Overall satisfaction	73%	17%	10%	-
Administrative contacts	63%	34%	3%	1%
- Telephonic accessibility	75%	19%	6%	8%
- Information upfront	74%	19%	7%	2%
- Keeping appointments	71%	22%	7%	10%
- Flexibility in making appointments	54%	33%	12%	15%
Reception	40%	47%	13%	3%
- Punctuality of the hearings	63%	20%	16%	4%
- Waiting conditions	47%	35%	18%	5%
- Privacy of waiting areas	31%	38%	31%	6%
Conduct of hearings	78%	19%	4%	2%
- The judge allows litigants to have their say	82%	12%	6%	3%
- The judge listens to the various points of view	81%	12%	7%	3%
- The judge imagines himself in the situation of the litigant	73%	19%	8%	4%
- Perceived expertise	79%	14%	7%	3%
- Perceived impartiality	78%	15%	7%	3%
- Treatment of the parties	84%	12%	5%	3%
Decision	65%	24%	10%	7%
- Founding of the decision	67%	19%	15%	8%
- Comprehensibility of the decision	71%	18%	11%	8%
- Perceived unity of law	56%	32%	11%	20%
Timeframes	46%	27%	27%	3%
Rechtspraak.nl (website)	65%	30%	5%	14%
Digital accessibility	40%	38%	21%	27%

Ezra van Duuren
Quality program secretary, the Dutch Council for the Judiciary

Secretary of the survey project group

&
Yinka Tempelman
Quality program leader, the Dutch Council for the Judiciary

Member of the CEPEJ-GT-QUAL

Italy: The experience of Turin and Catania

Users’ customer satisfaction analysis: the Italian experience (January-March) 2011

Taking the opportunity offered by the CEPEJ, in September 2010 the Statistics department within the Italian Ministry of Justice invited the Tribunal of Turin and the Court of Appeal of Catania (pilot courts within the network of CEPEJ) to conduct a Customer Satisfaction Survey among the users of justice. Also the Court of Appeal of Turin and the Tribunal of Catania immediately joint the initiative.

A key success factor towards the success of such kind of activity is to involve as much as possible all the stakeholders of the world-justice. Therefore, we decided to create a working group with the participation of representatives from the courts (judges and staff), from the Ministry of Justice (mainly statisticians with experience in conducting such surveys), from the local bar of lawyers and from the university.

The working group decided that the objectives of the survey were:

• To evaluate the overall satisfaction level of Justice amongst the final users
• To establish Key Performance Indicators (KPIs) to be tracked throughout time
• To pinpoint the areas of improvement and (propose to) take action accordingly
• To analyse the satisfaction amongst specific sub-samples (e.g. males/ females, age classes, …)
• To plot Performance vs Importance diagram.

Italian members of the CEPEJ were responsible for managing and coordinating the initiative in both cities.

Since the Satisfaction Surveys had no budget, we decided to maximize the use of internal resources and free external resources. The experience of the statisticians within the ministry’s department guaranteed the quality of the whole project. Thanks to a cooperation between the Courts and the University, a group of around 25 students in each city have been selected to conduct the on-field interviews. Prior to the fieldwork, a number of motivational sessions were carried out. Moreover, in order to ensure good quality standards, interviewers were carefully briefed on the approach to the interview, on the behaviour and good conduct, and, obviously, on the questionnaire content.

Based on the population of users visiting the courts under investigation, we have calculated that a sample of 600 interviews in each city were necessary to get a reliable result, within an acceptable standard error (lower than 4%).

The target of people to be interviewed was defined as parties, witnesses, interpreters, experts, relatives of the parties, of witnesses, etc. People working for justice like judges and lawyers were excluded from the target.

Before the interviews started, many posters promoting the initiative were stuck to the walls of the Courts premises. This simple action had a great impact on the respondents’ willingness to take part into the survey.

The questionnaire included 20 questions. A full report of results from Catania and Turin is available for consultation.

With regards to the key questions about the overall judgement given by the users to the service received in the two cities, we have that in Turin the 50% of users gave a “satisfied” and “very satisfied” mark, and only 32% of interviewed gave a negative feedback. In Catania, as expected, the result was slightly different as the positive feedback was given by 25% of respondents and negative feedback by 54%.

Very similar the responses were given by citizens to the key important factors of justice, that were ranked as follows:

1. Competence of judges
2. Quality of judgment
3. Length of proceeding
4. Courtesy of personnel
5. Quality of premises

We are very proud of the overall experience done in Italy and the feedback received by the different entities involved is very positive as well. We have been informed that specific actions have been undertaken by the courts in order to address some of the issues identified by the survey.

Fabio Bartolomeo
Director general of statistics at the Italian Ministry of Justice

CEPEJ Expert

Ongoing activities: what's new?

Evaluation of judicial systems

The working group on evaluation of judicial systems (CEPEJ-GT-EVAL) is implementing the new evaluation process of judicial systems, which should lead to the publication of a new report in September 2012. The national correspondents have until 31 December 2011 to complete the Scheme.

Work is also underway, in co-operation with the CEPEJ-GT-QUAL and with the Steering group of the SATURN Centre, to define judicial quality indicators.

More information

Timeframes of proceedings

The Sterring group of the SATURN Centre for judicial time management (CEPEJ-SATURN), in co-operation with the Network of Pilot courts, pursues the setting up of a European Observatory of judicial timeframes, with a qualitative and quantitative double-approach. A training programme for the courts has been set up for the courts wishing to be supported in the implementation of the measures developed by the CEPEJ to improve judicial time management; experts from the CEPEJ are at the disposal of the voluntary courts to work with them in this direction, on the basis of a handbook of methodology set up by the SATURN Centre. The interested courts are invited to contact the CEPEJ Secretariat: Stephane.Leyenberger@coe.int.

Furthermore, the SATURN Centre expects to refine the number of civil, criminal and administrative cases which could be subject to a regular follow-up in order to measure and understand timeframes of proceedings for these cases, considered in an homogeneous manner in the different member States.

More information

Quality of Justice

The working group on quality of justice (CEPEJ-GT-QUAL) proposes to voluntary courts a training programme for the organisation of court users satisfaction surveys, on the basis of a methodological handbook adopted last year. CEPEJ experts may accompany voluntary courts for this purpose. The interested courts are invited to contact the CEPEJ Secretariat: Stephane.Leyenberger@coe.int.

Work is also underway, in co-operation with the CEPEJ-GT-EVAL and the Groupe de pilotage of the SATURN Centre, to define justice quality indicators.

Furthermore, the working group is preparing guidelines on the main principles which should preside the reorganisation of judicial maps.

More information...

Network of Pilot courts

The plenary meeting of the Network of pilot courts took place in Strasbourg, on 22 September 2011. The Group provided its expertise in the field of the CEPEJ's work already underway for the organisation of satisfaction surveys in courts, the definition of common case categories which might be subject to a specific study in the field of length of procedure and identification of quality indicators of the functioning of the courts.

More information...

European Day of Justice

The main event of the 9th edition of the European Day of Justice took place in the Tribunal de Grande Instance of Toulouse (France) in presence of representatives of the Council of Europe and of the European Commission. About fifteen countries participated to the celebration of the day through the organisation of events aimed at improving the knowledge of citizens of the functioning of their judicial systems. The events which do not figure on the CEPEJ Website may be transmitted to Annette.Sattel@coe.int. Furthermore, the member States willing to introduce their application for the celebration of the main event in 2012, should send their draft application to this address before 31 January 2012.

More information...

Crystal Scales of Justice

The call for applications to participate in the 6th edition of the Crystal Scales of Justice will be launched in March 2012.

More information...

2010 CEPEJ activity report

The CEPEJ Vice-President, Georg Stawa, presented the 2010 CEPEJ activity report to the Committee of Ministers. He has underlined working prospects for the coming months. The Committee of Ministers widely congratulated the CEPEJ for its works which are oriented towards concrete results in the fields of independence and efficiency of justice.

2010 CEPEJ activity report

Publications

Study on the situation of the contractualisation and judicial process in Europe - CEPEJ Studies No. 16 / Julien Lhuillier, Institute of Criminology and Criminal Law, University of Lausanne, Scientific expert of the European Commission for the Efficiency of Justice

Study on Council of Europe Member States on Appeal and Supreme Courts' Lengths of Proceedings - CEPEJ Studies No. 17 / Report prepared by Marco Velicogna IRSIG-CNR, Expert of the European Commission for the Efficiency of Justice

Readers corner

You wish to react to an article published in this issue or suggest topics to be addressed in future editions ? Please send us your suggestions by e-mail to the following address: cepej@coe.int.

Forthcoming events

- 18th CEPEJ plenary meeting, Strasbourg

07-08 December 2011