[Facilitator’s Note: Additional contribution was received from Diego Recalde, Programme Specialist, RASC Panama. Please, review the full contribution.]

MDGNET Consolidated Reply - REVISED
FOR COMMENTS: MDG Monitoring Approach - Tracking UNDP Efforts

Cross-posted on MDGNet and MPN (Management Practice Network)

27 July 2006, prepared by Andrea Cuzyova

Original Query

We would like to present for your review the attached Note from BDP and BOM, on an MDG Monitoring Approach – Tracking UNDP Efforts, supporting an initiative to allow UNDP to track our efforts and resources at the country level by each country’s national MDG goals and targets.

A key area of UNDP focus today is to support our partner countries in achieving their national MDG objectives. Yet, if one were to ask how much of our effort and resources are managed and expended to doing such, and even more specifically towards specific MDGs, we could hardly provide an answer.  In fact, although significant effort and analysis is done across the aid industry to monitor indicators and progress towards achievement of the MDGs, very little is done in tracking organisational effort and effectiveness against the same.  

What is the value of tracking effort against MDGs? Firstly, if MDG goals and targets are part of our country programming efforts, we would be able to focus our outcomes and outputs on MDGs at the start of the planning cycle. Secondly, understanding how we budget and expend by national MDG goals and targets allows us to effectively plan and resource ourselves to support our national counterparts objectives. Thirdly, understanding and analysing our effort through national progress towards the MDGs can give us some idea of whether our interventions support such achievement, or at least tell us where we might want to accelerate or reallocate our efforts. Finally, tracking this effort is increasingly viewed by many of our partners and our Board as an important indicator of organisational effectiveness, and UNDP’s contribution to development effectiveness.

How would we track our efforts by MDG? At a macro level, we can use our current results structures found in Atlas “project trees” to link MDG targets to our UNDP Service Lines. To be more precise, linking country-level outcomes to country-level MDG targets would provide a more granular level of reporting. As you would expect, there are challenges with doing this, including the fact that our work does not easily map to MDG targets, and the overlapping nature of some MDG goals and targets. Issues of attribution and all else that plagues the attempts to look at any organisations’ contributions to development goals, will hold. Nonetheless, some level of logical mapping can be done. Other options have also been suggested.

We look forward to your views on the values of this tracking, the approach recommended, and suggestions for improvement.

Thanks,

John Patterson

Kanni Wignaraja

Jens Wandel

Contributions received with thanks from:

§                     Jacob Massuanganhe, UNCDF/UNDP Mozambique

§                     Sudarshan, UNDP Bangkok Regional Center, Thailand

§                     Abla Amawi, Sub-Regional Resource Facility-Arab States, Lebanon

§                     Erdal Esin, UNDP OHR, HQ

§                     Dorothy Rosenberg, Poverty Group, HQ

§                     Heather Bryant, UNDP Nepal

§                     Tim Scott, UNDP HDRO, HQ – comments provided offline

§                     Mari Matsumoto, UNDP Capacity Development Group, HQ– comments provided offline

§                     Bharati Silawal, UNDP Gender Team, HQ– comments provided offline

§                     Diego Recalde, RASC-Panama

Summary of Comments: 

‘MDG Monitoring Approach: MDG Monitoring Framework for tracking UNDP Efforts’ outlines options and approaches for UNDP to track, monitor and report on UNDP expenses by MDGs i.e. what is being planned and what has been done/expended according to the MDGs. The scope of the proposal considers all of UNDP development efforts - global, regional, and country-specific. The approach does not concern tracking national progress against MDGs.

The current draft will benefit from the feedback received and present a good starting point for further refinement of the methodology.

Comments on: ‘Monitoring Approach Recommendations’

MDG-based monitoring and evaluation of UNDP programmes and projects can offer an alternative way of measuring UNDP performance that is both quantitative and qualitative.

Several contributors pointed out note 9 (p. 5) that states ‘UNDP’s work is not directly related to specific MDG goals and targets e.g. capacity development, democratic governance … This focus on creating an ‘enabling environment’ for achievement of the MDGs means that UNDP efforts in these areas must be considered in some way relevant to achieving the MDGs.’. Some contributors proposed to design a monitoring and evaluation framework for the ‘enabling environment’ that would include key aspects of governance, accountability, capacity, etc. as well as means to measure them. Others pointed out a more fundamental issue related the proposed monitoring approach (incl. note 9): ‘…the approach – to monitor and report on UNDP efforts by MDG Goal, Target or indicator - appears to be based on a literal reading of the MDGs as a development agenda, rather than in their function as cumulative outcome indicators – that is outcomes which reflect the combined effects of a broad range of different factors, including the inputs and actions of a variety of actors.’

Another essential issue raised in the feedback related to the inter-relationship of all of the MDG goals and targets. For example, progress on Goals 2 and 3 and 7 have both direct and indirect effects on the outcomes measured under Goals 4, 5, and 6, making it extremely difficult to define the precise relationship between inputs and outcomes. I.e. example from the Country Office Nepal mentions: ‘Governance programmes at the outcome level are focussed on the “enabling environment” or the Millennium Declaration, however at the ground level, they contribute to many different MDGs in very concrete ways. In other cases the clearest available indicators relate to an MDG the project doesn’t seem otherwise connected to – for example, many (governance) projects support construction of small infrastructure including drinking water, which maps directly to MDG 7, Target 10.’

It was proposed to measure UNDP efforts against the priority UN activities at country level – monitoring, campaigning, analysis and operational activities – outlined in the UN Core strategy (www.undg.org). 

Other comments (in addition or in support of the points raised above) proposed:

• Embedding MDG monitoring into the country programming process;
• Project results be measured according to MDG targets and indicators to assess accomplishments against short and long term objectives, on the national and local levels and against the global MDGs framework;
• Detailed level monitoring information, as well as ability to aggregate to the national level are needed;
• Mapping at the indicator level;
• Considering local level indicators and targets in the exercise.

Experience from Nepal supports the suggestion to track efforts and resources against indicators (in addition to MDG goals and targets). There is a need to track results (through indicators) through national MDG reports and monitoring and evaluating own programmes and projects to assess the impact. During the deliberations on linking projects to MDGs, the Country Office developed a matrix listing the MDGs (or main MD themes) in the first column, with relevant service lines in the second column, and the contributing projects in a third column. In re to indicators, it was chosen to illustrate results chains with the MDGs as the “impact”. Country Office experience in relation to mapping projects to MDGs through service lines or directly to the MDGs emphasized the need for consistency in tagging or linking to MDGs.

It was suggested that DevInfo or a similar system should host collected data and display them in variety of ways i.e. aggregates or disaggregated, based on selected criteria, in charts, tables, or a map format.

Comments on: ‘Initial Mapping Options & Recommendations’

The mapping will eventually need to be done at several levels of granularity. If the mapping is conducted at the core result, outcome, or project level, it will have to be done at the service line level as well to provide a framework for tagging projects and outcomes at lower levels.

In terms of reactions to Options 1,2,3,4 in regards to tagging MDG goals or targets, the contributors presented various views.  Some viewed the proposal of starting with option 1 and moving on to more detail, if necessary, as reasonable. Others viewed Option 3 and 4 as more feasible and related to the situation on the ground.

Regardless of decision made in re to tagging MDG goals and targets to country outcomes, projects, etc., a dedicated monitoring capacity would be required at the country level to ensure systematic tracking. The systematic monitoring approach proposed in the paper also calls for a capacity development package for the country offices for related types of tracking and monitoring.

Previous relevant CRs:

DEVINFO:  DevInfo is a database system (data depository and display system) which allows for storage, presentation and dissemination of data in a very appealing fashion i.e. through maps.  etc. DevInfo contributes to setting universal standards for data storage, access, presentation and dissemination of indicators, and allows for monitoring of global as well as national (sub-national) indicators. More on DevInfo can be found at www.devinfo.org

Uzbekistan / Web-based expanded MDG monitoring tool

Introduced Kazakhstan InfoBase, a web-based expanded MDG monitoring database, that provides an integrated overview of national human development including statistics on economic growth, national accounts, trade and finance. Background on the system was shared, including development process, documents, and relation to DevInfo.

Namibia/Assessing the need for DevInfo or other information systems

Requested comparative experience on determining data base and information systems needs with emphasis on simplification, efficiency and maximum ownership and impact on country level. Responses highlight the relative advantages of DevInfo and suggestions and considerations when selecting to adopt this system.

Responses in full:

Jacob Massuanganhe wrote:

I would like first to congratulate the BDP and the BOM for this effort. I found the note very important to address the MDGs at the national level (the National perspective). It is clear that the approach focus on tracking UNDP efforts by MDGs, and not tracking national progress against MDGs.

Second, my comment is in terms of targets and indicators setup. Here, I would like to draw your attention to the need of considering in the MDG Monitoring Framework, the local level indicators and targets in the programming exercise.  This because the concept of planning or monitoring the MDGs, has mostly stopped at the national level and rarely does the concept sufficiently includes the sub-national level targets, provinces, districts and villages, where needs are felt and service delivery/capacity building are needed (the field perspective).

It will lead to enhance effectiveness and efficiency of efforts and resources at the country level, since in many cases it is pointed out that those national interventions/frameworks are limited and cannot respond to local needs and demands, in line with the MDGs concept.  UNCDF's experience in Mozambique shows that the MDG’s implies strong commitment and long term development vision adressed to local demands, and that the MDGs will not be achieved if the interventions do not strengthen institutions and build capacity at local level.

In this regard, it is important that the paper considers (i) the Participatory Approach, involving local actors and (ii) to idetify key indicators/targets to be considered or at least, be mainstreamed at the CO level (the CO perspective). For reference it is attached the Working Paper Number 3: Participatory Monitoring and evaluation systems at district level. 

Resources: 

·         Participatory Monitoring & Evaluation at district and local levels: Mechanisms, Evidences and Practices (April 2005) [ pdf ]

·         Modeling PRSP II & Poverty Reduction in Mozambique Local Development: Econometric Analysis of factors determining Millennium Development Goals by 2015 [ pdf ]

Regards,

Israel Jacob Massuanganhe,

UNCDF/UNDP Programme Specialist

Decentralization and Local Development

Email: Jacob.massuanganhe@undp.org  

Phone: +258 21 481475

Mobile: +258 82 3045610

UNDP Mozambique

Kenneth Kaunda Ave. 931

Maputo

Sudarshan wrote:

The paper is both useful and timely. The latter because the two Asian Regional Centres are in the throes of rolling out the Integrated Packed of Services for MDGs in the priority countries in the region. The reasons for foregrounding MDGs and tracking UNDP's own efforts are sound. Moreover, it will help us from unwittingly inviting the "double-standards" criticism that one hears so much about in the discourse on human rights and democracy promotion and the United States of America.

Sudarshan

Legal Reform and Justice Policy Advisor

UNDP Bangkok Regional Center

Abla Amawi and CAP2015 Team in Arab States wrote:

First of all, this is an excellent concept to use the MDGs as Results-Based performance measures. For sure, this will improve UNDP's support to MDG-based national development planning and localizing the MDGs if we can make our management MDG-based as well. In addition, MDG-based monitoring and evaluation of UNDP programmes and projects can offer another way of measuring UNDP performance for HQ and for donors that is both quantitative and qualitative, instead of being based on financial measures like "service delivery," which show input measures, rather than measures for outputs and outcomes.

One consideration that should be kept in mind however is that this MDG tracking system must focus on the level of outputs and outcomes, and not stop at simply making links to MDGs. The fact that an environmental project happens to be related to MDG 7, and thus MDG-based, even though it was not designed to be so, is worth mentioning. But to really measure progress, the project should be measured according to MDG targets and indicators to see what it has really accomplished in terms of short and long term objectives, both for the national and local level, and according to the global MDGs framework.

Monitoring Considerations

1. It is important to aggregate as well as disaggregate, because there is a need to have national data to track progress as well.
2. Mapping at the level of goals and targets is surely logical, but perhaps mapping at the level of indicators could be useful as well, especially at the local level.
3. The ideal, in fact, would be to put all possible data into a DEVinfo like system which can display the data in many different ways, as aggregates, or disaggregated, according to several criteria, in chart, table, or map format.
4. We agree 100% with point #9 about the enabling environment. But there must be some ways of monitoring and evaluating progress in initiatives aimed at developing the "enabling environment," and especially of linking UNDP's work to certain outcomes where relevant. This is often easier to describe than to quantify, but some quantity measures are relevant. Perhaps a framework could be proposed for the enabling environment to support MDG-based development that would include key factors on governance, accountability, capacity, etc. and ways to measure them.

With regard to the mapping and tagging options presented:

1.                   It would seem that mapping will eventually need to be done at several levels of granularity. If mapping is to be brought down to the core result, outcome, or project level, for example, it will also need to be done at the service line level, in order to provide guidelines and a framework for those who are tagging projects and outcomes at lower levels.

2.                   Of course, ideally, the more specific project and outcomes levels should be mapped. But a balance must be struck between pragmatism and idealism. The approach's suggestion of starting with option 1 and moving on to more detail if necessary seems a reasonable proposal for the current time.

3.                   Another suggestion is that the approach should further study ways to integrate MDG monitoring and evaluation of UNDP projects with UNDAFs, CCAs and PRSPs, as well as with MYFF service lines.