The European Commission for the Efficiency of Justice

    6 December 2012 - Celebration of the 10th anniversary of the CEPEJ
    “The CEPEJ: 10 years of concrete action for improving
    the efficiency and quality of the functioning of courts, for the sake of users”

        The CEPEJ Handbook for conducting satisfaction surveys

aimed at court users:
a tool for improving the perception of the users’ trust in their judicial system

      Carmen DANTIS, Spokesperson at the Court of Vrancea, Romania

        Dear Ladies and Gentlemen,

    Allow me to relay onto you how the Court of Vrancea, Romania, organized a survey relating to the court users’ satisfaction. The survey gathered and analyzed data on the activities of the courts, having as technical support “The Handbook for conducting satisfaction surveys aimed at court users in Council of Europe’s member states.” We also benefitted from the direct assistance of the CEPEJ, after a coaching session that was directed by Judge Giacomo Oberto – as an expert.

    Our court tried, during 2011, to organize a satisfaction survey, on the basis of the self-administered questionnaires within the courts, which it had as the aim the evaluation of the activities of the Archive and Registration Departments. The motive of this activity was the analysis of the object of the petitions transmitted to the court and the observations of the citizens who had visited the Public Relations Department. But this survey didn’t offer us eloquent outcomes, mainly because of the distribution and collecting methods chosen, which turned out to be inefficient. Handing of the questionnaires by the Archive staff to the persons who requested the services of this department was the main distribution method. We collected the questionnaires through the placing of an urn in the main hall of the court.

    In this context, we learned that CEPEJ could offer assistance to the courts that want to organize surveys, and thus our decision to contact the representatives of the Secretariat of the CEPEJ was a natural follow-up of our efforts in this direction. After all organizational details had been established, on 8th December 2011, Judge Oberto came to our court to present and explain the methods and techniques recommended by the CEPEJ for a well-organized survey –methods that are presented in the Handbook, too. The coaching session also gave us the opportunity to be briefly informed about others projects carried by the CEPEJ (such as SATURN), or about other programs implemented by the Court of Turin, where Judge Oberto presides.

    We should mention the very important role that the financial assistance of the CEPEJ had had, in consideration of the fact that our court budget didn’t provide finances for the organization of such coaching session, or for the translation of the Handbook.

    At the beginning of 2012, The Court of Vrancea started the effective organization of the survey pertaining to the court users’ satisfaction in regards to the activities of the courts. At first, we set up a steering committee, under the co-ordination of the President of the Court - Judge Costica Dita. In view of the fact that this activity implies some specialized knowledge in the sociological field, we decided to contact and to collaborate with University professor George Neamtu, who offered us his expertise in this field.

    The main objectives of our survey were the evaluation of the court users ‘satisfaction level in regards to the activities of the Court of Vrancea and of the Court of Focsani, and the quality improvement of the offered services. Depending on these, we established the indicators of the quantitative survey which we proposed to be realized.

    Among the methods analyzed and presented by the CEPEJ in the Handbook, we decided to resort to the administration of the questionnaires within the courts, since we had had minimum human, technology and budgetary resources. At the same time because - from our previous experience - we observed that the returning rate of the self-administered postal questionnaires – as a secondary collecting method chosen in that case – was very low.

    In the first organizational meetings we decided that the survey should be applied for the first time among actual users of the concerned court – as direct beneficiaries of this public service. In the future, we will try to differentiate between specific categories of users (such as lawyers, members of the prosecution service or other legal professionals). At that moment of our activity, the first difficulty we faced was the setting of the representative sample dimension. As a result of the analysis realized by our collaborator University Professor Neamtu, and taking into account some parameters of the concerned courts, it was recommended to us that the lowest number of the answering persons to be about 200.

We don’t know exactly the methodology that was used to determine this number. Since the Handbook doesn’t report on this matter either, we appreciate that the CEPEJ could analyze the possibility to complete this methodological guide with a calculation algorithm of the representative sample, that it could be followed by all courts that want to organize a survey.

    Finally, the total number of the questionnaires applied during the survey organized by the Court of Vrancea was 512, which we’d say gave us a superior level of precision.

    Regarding the timing of the survey, the Court of Vrancea took into account the judicial schedule. We noticed that the number of the justice seekers is greater in the courthouse especially in the middle of the week (on Tuesdays, Wednesdays and Thursdays), and between 9 a.m. and 1 p.m. Concretely, this stage of the survey carried on between 5th June and 5th July 2012.

    The questionnaires administrated within the courts were submitted by the volunteers – as operators who were in the courthouse. They directed the users to give answers to the proposed questions. We resorted to human personnel since the collection of data for the previous survey didn’t prove to be an efficient method. Urns were set up for questionnaire drop off, and we had a low returned rate.

    Concerning the selection of operators, we came up against some difficulties because the city where we carry on isn’t a university centre. But we found that high school students were interested in such an activity. We also found volunteers at the Probation Service. The operators played a very important role in the economy of the whole survey. The manner they acted could have affected the perception of the justice seekers about the steps of the Court of Vrancea and the accuracy of the collected data, too. Because of that we paid special attention to the preparation and informing sessions, within the framework they were informed about the rules that should be followed. One of these rules refers to the fact that an operator only reads the questionnaire, without making suggestions, offering personal explanations or making clear different words or expressions. From this point of view, the qualification of the operators is not important at all. But we believe that minimum law knowledge and legal terminology are necessary and we will take this into account in case we will resort to this method of data collection in a future survey.

    After we applied the questionnaires we found, both in a direct manner through the quantitative evaluation, and according to the operators’ opinions, that the court users manifested a larger availability to answer the survey questions in the days when were settled preponderate civil cases, comparative with the days with criminal cases. Percentage-wise, only 15% of the justice seekers declared that they took part in criminal cases. This aspect should be taken into account at a next survey, too.

    Regarding the recommendation from the Handbook about the necessity that the court concerned should inform users beforehand, in order that they feel involved in the survey procedure, we didn’t give course to this suggestion to avoid the situation in which the users would premeditate the filling in of the questionnaire, offering previous prepared answers, induced by certain interests or influenced by special events. This beforehand informing stage was reduced to a brief explanation of the carry-on activity that was realized by the operators, when they addressed the users. In some situations, a determining role in the agreement to participate in the survey had the view of the Council of Europe’s logo on the questionnaire paper and the information about the technical support given by this international organization.

    The drawing up of the questionnaire was a challenging stage for which the Handbook was a very important fulcrum. At the same time, we analyzed the questionnaires applied at the Court of Turin and that one used in Geneva, that Mr. Oberto gave us. From this point of view, the coaching session turned out to be a really exchange of experiences, an occasion for sharing good practices and successful transfer of knowledge.

    The version of the questionnaire applied by the Court of Vrancea included, in majority, closed questions that could only be answered in the pre-established versions. But it also contained open-ended questions, to allow court users to elaborate on issues they cared about or on issues not included in the survey. As the Handbook recommends, too, the number of this kind of questions was reduced – two – because of the difficulty in the processing of data.

    We appreciate that in the composition of a survey an important role must have the checking questions which control an expressed opinion from the veracity and consistency viewpoint. We used in our questionnaire this kind of questions too, which permitted us, for example, to know how the court users had formed their opinion about justice, in general, or about the act of judging and the services offered by the Court of Vrancea and the Court of Focsani, in particular, and to find if the given answers were correlated.

    As response scales, we used various types, like scales that ask the user to choose an item from many pre-established ones or scales with more detailed user preferences through a longer rating. We consider as being very interesting and with eloquent outcomes the type of questionnaire used in the Canton of Geneva (Switzerland), that make it possible to measure the gap between user expectations and user satisfaction for each item using a dual assessment (importance and satisfaction). However, we decided not to utilize this method, because it requires a superior capacity of discrimination and an increased attention in the completion of the questionnaire from the respondents. We believe there would be no availability for that, mostly because of the lack of time.

    Regarding the analysis of the results, we didn’t have the possibility to resort to outside bodies for this stage of the survey. The analysis was prepared by the court staff, using a common program – Excel – but which offered limited possibilities of interpretation and correlation. We hoped to buy specialized software for statistical analysis of data, which failed because of the lack of the budgetary funds.

From this point of view we believe that CEPEJ could analyze the possibility to purchase the software for statistical studies that offer rich statistical capabilities paired with features that make it easier to access and manage data, select and perform analyses and share results. We think that the licenses of such programs could be temporarily assigned to the courts that organize surveys, during the analyses data stage.

    At this moment, the Court of Vrancea is in process of reporting results. This stage takes place in-house first. Following the communication of the outcomes, we will identify concrete solutions for the optimization of the activity, and, as far as possible, for the qualitative increase of those proceedings that finally lead to the growth of the court users’ satisfaction in regard to the activities of the courts.

    We also must to inform justice seekers about the results of the survey, by uploading them to our webpage or by calling the media support for the popularization. The communication of the survey results to the court users is essential because they will have their opinions validated. We hope that they will manifest availability to regularly take part in this kind of activity – which will permit us to measure the evolution of satisfaction, on base of the services evaluation.

    As the Handbook proposes, the Court of Vrancea will analyze the possibility to supplement the survey with other methods, but this will depends on the available resources.

    Concluding, in today’s society, majority entities, either public or private institutions, as well as the systems in the whole, pattern their behavior on the satisfaction of the personal users. Starting from the analysis of their expectations and opinions, no matter the mode of expression, the entities set out their directions of action to obtain performances. The systems of justice in the whole, as integral parts of the society, should have in view and analyze the feedback given by justice seekers in order to increase the quality of their activities. An easy manner to obtain the court users’ feedback is to organize surveys, and the Handbook is a very useful instrument for that, since it gives methods and techniques that can be used in the realization of the survey. If the methodologies proposed in this guideline will be put in practice by more courts, it can be obtain an increase of the users’ trust in the judicial systems, as a result of the changes that can be made on bases of the outcomes of such surveys. For these reason we believe that this instrument developed by the CEPEJ should continue to be improved and it must be promoted more and more among the judicial systems of the Council of Europe's member states.

        Thank you and happy anniversary CEPEJ!