Participative Monitoring and Evaluation seminar for the Council of Europe


INTRAC Training report

Participatory Monitoring and Evaluation seminar for the Council of Europe
2-3-4 June 2004

MEETING REPORT
 

Seminar organised by

The Council of Europe, Roma and Travellers Division

 

Introduction

The Council of Europe requested INTRAC to provide a short training seminar on Monitoring and Evaluation on the JP “Roma under the Stability Pact” II, for five of their staff and two expert advisors. The training Sessions were held at the European Youth Centre (Strasbourg). A list of the participants is attached at Annexe 1.

The participants were approached pre training and provided with an outline of the topics that could be covered, the content was agreed and the final outline prepared, attached at Annexe 2.

Background

The training was part of a more substantial initiative that the Council of Europe and INTRAC were working on together. This initiative is aimed at promoting the participatory monitoring and evaluation of Roma Strategies in CoE member States. The Council of Europe will hold a policy Seminar in September, in partnership with INTRAC, to identify the current situation within these countries, to identify challenges in monitoring and evaluations that require support, and to develop a set of draft recommendations on the monitoring and evaluation of Roma Strategies. Depending on the needs identified, a longer- term joint initiative may well develop between the Council of Europe and INTRAC.


Training Sessions

Session 1
The participants and the INTRAC consultants introduced themselves to each other and examined what they would be covering over the next three days. Looking at how the training session would help them with monitoring the work they are doing with and on behalf of Roma communities. The group spent the time getting to know each other, setting ground rules and establishing the participant’s needs.

Session 2
This session was spent examining the theory behind monitoring and evaluation, looking at definitions, why the need to monitor and evaluate, key questions related to M&E and finally examining the terminology used in M&E processes. The session was mainly a presentation of the theory using the facilitator’s experiences to demonstrate how theory links to the practical application of M&E. Also getting participants to link the theory to any previous work they have done on M&E. It was a lively session with many questions being raised that would be answered as the training developed. The second half of the session was spent looking at stakeholder analysis examining issues related to who the stakeholders were in any work linked to Roma programmes who owned the work, who was responsible for completing the work and who maintained quality control. Differentiating between the different stakeholders needs, power levels and negotiating base.

Session 3
During this session the participants explored the underlying approaches to development and by implication monitoring and evaluation. Examining the two major approaches to evaluations, technocratic and interpretative related to the four generations of evaluation. The second half of the session was spent helping participants to define the differences between Projects and Programmes, and management by objectives, examining the principles behind the hierarchy of objectives and where that fits into the project cycle. Including the theory behind ‘framework analysis’ in planning and monitoring a programme or project. Again this was a more informative session where the facilitator provided the theory with experiences.

Session 4, 5, 6,
The participants selected a project that they could all relate to and using the framework analysis developed a logical framework with support from the facilitator. The project selected was to assist with changing employment legislation for Roma in an Eastern European Country. The participants initially prepared a hierarchy of objectives based on the logical framework model of other Council of Europe projects. (Overall objective, Project Purpose, expected results and activities) We then moved on to looking at risks and how to lower risks in the planning phase. The groups looked particularly at involving stakeholders at all levels and the importance of getting the different perspectives of groups at the planning phase.

The exercise took most of the day and at the end the participants were satisfied they had the skills to develop the outline of a participatory planning logical framework.

Session 7
During this session the participants looked at the theory of developing non tangible indicators and adapted the learning to the practical exercise started the day before. This session raised some major constraints the Council of Europe staff are facing when attempting to measure their work. The main one being the lack of interest in or by respective Governments to include Roma issues into both legislation and non discriminatory practices. This certainly highlighted the issue of how important it is for CoE staff to include Government and Roma people themselves in developing indicators of achievement that they are all willing to be measured against. Interestingly enough the discussions also raised issues around whether the Roma NGOs had the same perceived needs of the Roma population as the population itself.

Session 8
The participants were introduced to the theory of institutional learning looking at what is a learning organisation, how institutions learn, problems and solutions for learning organisations. This was followed by them all doing an exercise developed by Bruce Britton for INTRAC to examine if the Council of Europe was a learning organisation.

Session 9
Participants reflected on the work they had done over the previous three days and completed an action plan they would use to ensure they incorporated the lessons learned from this workshop into their work. The action plan was for on immediate return to their workplace, three months and 6 months later. The participants retained their action plans as aid-memoirs post training.

Evaluation
During the final session the participants completed evaluation forms on the content of the training course:

The general findings were very positive; all participants enjoyed the methodology and found the training important to their work. On a scale of one to ten participants were asked to rank how they found the course the findings were positive 2x10 2x9 1x8 and 1x7. The ranking for having fulfilled the aims of the course were 2x10 2x9 2x8 again very positive. Each participant found different parts of the training useful and for one participant the training did raise frustrations on the level of work the CoE is working on and how to get stakeholders involved at such a high level. Five of the participants mentioned that they would now try to revise their present projects by including their stakeholders in developing new logical frameworks.

Anne Garbutt
INTRAC
June 2004

 

Annexe 1

Participants list

Council of Europe Migration and Roma staff.

Managers:
Mrs Eleni Tsetsekou
Mr Michael Guet
Ms Elena Piscopo

Assistants:
Ms Nelly Tasnadi
Mrs Brigitte Thomas

Consultants (tbc):
Mrs Ina Zoon
Mrs Judith Kiers

INTRAC Consultants

Allan Phillips ( 2 June only)
Anne Garbutt
 

Annexe 2

Monitoring and Evaluation using the project-planning framework approach


AIM: Participants will explore monitoring and evaluation using different approaches, including how to develop a participatory planning framework and how to use that framework to help monitor the life of the project.


FOR: Council of Europe staff


PROGRAMME OUTLINE:

• Participants can select a case study that will be the basis for the full two days work
• Explore the purposes of Monitoring and Evaluation (fourth Generation)
• Examining the information requirements of different stakeholders and how to involve them at all stages of the project.
• Reflecting on the strengths and weaknesses in using the a hierarchy of objectives in both planning and monitoring of social development programmes and projects
• Identifying indicators that can measure outputs and outcomes
• Looking at Impact and how we measure change
• Explore ways of ensuring institutional learning

Methods:
Presentation on the theory of project planning, including management techniques and monitoring & evaluation
The two days will be based on the selected case study of the participants own work, including group work, group discussions and plenary feedback sessions.

Resource Persons:

Anne Garbutt: INTRAC
Alan Phillips: INTRAC

Duration: 2 days 2-4 June 2004

 

T