Most donor support programmes use a range of monitoring, evaluation and learning (MEL) approaches to keep track of their progress and learn from, and communicate, their outcomes. These approaches are typically based on the implementation of technical solutions delivered by a range of external experts.
Carrying out MEL in the context of developing an energy market – a complex system of the actions, interactions and interdependencies of a vast number of diverse actors, and that cannot be controlled or predicted with certainty – requires a different approach. This includes monitoring the interaction of the market actors to prioritise their resources, and to evaluate the tangible and intangible changes, and learn from the outcomes.
M&E is a popular abbreviation of the two terms monitoring and evaluation, but, although they are complementary, it is important they are treated differently.
Monitoring is defined as assessing an action as it happens, including how it happens
Evaluation is defined as assessing the results of an action including its relative value
It is important to note that the divide between monitoring and evaluation is often blurred, as every time an intervention is carried out, the facilitators often carry out some form of evaluation, even informally or subjectively. The difference becomes clearer as the evaluation becomes more formalised, codified and standardised, particularly when comparing impacts between programmes.
Table 1 can be used to assess the important aspects of monitoring and evaluation, and to identify the differences between them.
Table 1: Important Aspects of Monitoring and Evaluation
|Who||Normally carried out by the implementation teams and partners||Normally carried out by the programme managers and/or independent evaluators|
|When||Frequently - as the facilitators interact with the stakeholders or every month or quarter of the intervention||Less frequently - every six months or annually during the intervention, as well as at the end and after a period after the end|
|Why||To understand how the market system is reacting to the facilitator’s actions and to adjust as required to maximise the influence and minimise costs||To assess the extent the programme is, or has, achieved its goals and at what cost|
The facilitators and market actors can use the collected information as follows:
As this roadmap process focuses on facilitating the processes of joint learning, visioning, consensus and compromise from a range of actors within a selected energy system, learning from the successful implementation its 3 principles - systemic thinking, participation and facilitation – is of critical importance, as summarised in Table 2.
Table 2: Learning within Market System Development
|Facilitation||Creating the right conditions for public and private market actors to build joint visions, strategies and action plans that connect with their interests, capacities and intentions to drive positive change within their energy system.||The appropriate conditions can only be created if the facilitators are constantly learning what makes sense, and what drives, the market actors. This is particularly important for ensuring the involvement of the marginalised actors, and gender equality.|
|Participation||Creating the conditions for “co-creation” of solutions that make sense to all the market system actors including the marginalised actors.||Co-creation cannot happen without a constant flow of clear and transparent information and constant learning, between the market actors, external experts and facilitators.|
|Systemic thinking||Assessing markets as complex systems composed of many actors playing a range of functions and learning and reacting to each other, often in unpredictable ways.||Learning is at the heart of understanding how market systems function, in particular to how marginalised actors can better integrate into them, through accessing knowledge and working with others.|
Constant Learning: Facilitators need to constantly learn from what they are doing and to convert these lessons and insights into codified knowledge that can add value to their organisation and others. This is often very complicated and is not done well in practice. It takes constant effort as well as a good understanding of the structures and incentives of the actors, and many other aspects fundamental for navigating complex and uncertain energy systems.
Collaboration: Facilitators need to work together to make sure that lessons, insights, questions and even intuitions of individual staff are openly discussed and then codified (putting knowledge into words, images or sounds) to circulate to various audiences. The discussions must be collective and honest, with the knowledge easily understandable, appropriate and easily accessible to the different end-users.
Hard Work: Facilitators need to work hard to ensure monitoring and evaluation does not become a self-contained, bureaucratic exercise to satisfy managers and donors but instead, helps all market actors gain a deeper understanding of a selected energy market system and how facilitated interventions can help marginalised market actors work better for the poor, including women.