Strategy for a Learning Society - Appendix - Project Monitoring and Evaluation

Appendix:

Considerations for Project Monitoring and Evaluation

Monitoring and impact assessment of the projects and initiatives which make up the set of first steps toward building a learning society is of the highest importance, particularly given the innovative nature of the proposed approach. The goal common to all the proposals is the generation of new learning dynamics in El Salvador, which implies the constant correction, broadening, and replication of early experiences. Without an adequate evaluation of preliminary results, and timely corrections on the basis of problems identified, the prospects of creating these dynamics will be unnecessarily limited.

The design and execution of the monitoring and evaluation component should be integrated with the overall strategy, instead of project-by-project, in order to generate consistent results, create a core group of Salvadorans capable of measuring the impact of information and knowledge on development, and facilitate the possibility of measuring overall progress in building a learning society.

The evaluation of results, based on the monitoring carried out throughout project activities, should be conducted by a group that is not directly involved in the success of the project, in order to ensure greater objectivity. However, project members should work together with the evaluation team to define the methods and variables to be used in evaluating results, so that they may play an active role in monitoring and collecting the quantitative and qualitative data necessary to achieve a worthwhile evaluation.

Objectives

Record the main features of the process of implementing the projects so that the lessons learned can be thoroughly understood, especially by those who have no direct connection with the case.
Assemble evidence of the achievements or errors of the projects.
Enable corrective actions to be taken in the course of the project’s implementation.

General principles governing monitoring and impact assessment

Identify in detail the starting situation of each area and group involved in the project (initial snapshot).
Where possible, set up control groups who do not participate in the project, in order to compare results; these must be monitored just like the project participants, including the initial snapshot.
The monitoring should be continuous throughout the project.
The process should be as simple and unobtrusive as possible.
The critical success factors should be identified beforehand and in concert with participating groups.
The monitoring and evaluation variables should concentrate on a few key business or social needs, as specific as possible, defined by the stakeholders.
The monitoring and evaluation should cover both factual evidence and the stakeholders’ appreciation of the process and results.
The focus should be on the process of social participation in the creation, use, and application of knowledge resources, more than on the availability, use of, and satisfaction with these resources.
Solid evidence and guidance should be obtained regarding the best ways to consolidate, expand, or replicate the experiment to other areas, sectors or groups.
The indicators should be easy to calculate and self-explaining for the participating groups.

Summary of aspects to consider during monitoring and impact assessment – preliminary list:

Appraisal of the environment (local, national, international)
Knowledge of the initial situation (directly related to the project)
Understanding of the knowledge cycle of the target community analyzed
Management control (adapt to the particular case)
Use of the facilities and programs, participation in dynamics created
User satisfaction
Apparent achievements of the users
Changes in the initial situation (impact)
Innovation in the services on the basis of monitoring and evaluation
Sustainability
Potential for replication
Lessons learned, both positive and negative.

Requirements for personnel in charge of impact assessment

Good understanding of the concept of the impact of information and knowledge on development goals.
Understanding of the knowledge cycle, ranging from the creation of knowledge to the systematization of new knowledge as a way to enhance the stock of knowledge of a group or individual.
Appropriate methods for collecting relevant data and evaluating it in the context of learning society goals.
Knowledge and understanding of the socio-economic and cultural attributes of the project target groups.

Feasibility

Given the novelty of this strategy, it is highly unlikely that El Salvador currently has the capacity to guarantee the success of this component. However, with a focused effort involving a group of champions (approximately one month’s work), a core group of evaluators could be successfully trained. This group would work with the different projects and adapt methodologies to individual project requirements; it would also train project participants in the continuous monitoring techniques necessary for each project.

Therefore, a permanent team should be set up to conduct monitoring and impact assessment activities; its members should come from the organizations and institutions involved in different aspects of learning society activities. The team members should be supported by outside specialists in social impact assessment; they receive intensive training at the start, with backstopping throughout the first several months of work, and should function as a team –to share experiences and lessons learned– as well as individually, in their respective assignments, to assess the impact of the array of projects in this strategy.

Conectándonos al Futuro, San Salvador, El Salvador

Updated 25-06-1999

[Table of Contents] [Executive Summary] [Introduction] [Chapter I Context] [Chapter II Building] [Education] [Migration] [Local Development] [Rural Development] [Micro-Small Enterprise] [Learning Organizations] [Infocenters] [Government] [Democracy & Citizenship] [Business Competitiveness] [Culture & Identity] [Quality of Life] [Conclusions]

Appendices: [Methodology] [Project Implementation] [Monitoring & Evaluation] [Local Development] [Migration] [Rural Technology Transfer] [Water Resources] [Electronic Commerce] [Public Information] [Government ICTs] [Evaluation of 3 Organizations] [Electronic Conference] [Participants] [Bibliography] [Glossary of Acronyms]

All rights reserved. Permission is granted to reproduce all or part of this publication as long as the complete source is cited: Conectándonos al Futuro de El Salvador, "Strategy for Building a Learning Society", San Salvador, 1999, http://www.conectando.org.sv/English/Strategy/