The CommonSensing project is funded by the United Kingdom Space Agency (UKSA) and aims to  strengthen disaster risk reduction and climate change resilience in Fiji, Solomon Islands and Vanuatu through 1) increasing national resource capacities in the use of Earth Observation (EO) solutions to address disaster risk reduction and climate change resilience by 2020, and 2) enhancing evidence-based decision making by using CS solutions for disaster risk reduction and climate change adaptation (CCA).

The evaluation’s specific objectives are to 1) track the progress against targets, 2) identify the main problems and challenges that undermine project implementation and the achievement of results, and 3) provide recommendations for corrective actions. The evaluation focuses on appraising the situation of the project, as well as identifying enabling and preventing factors of project performance, including the assessment of output results. This is done by applying the six evaluation criteria of the Organization for Economic Co-operation and Development’s Development Assistance Committee (OECD DAC).

Using a mixed methods approach, the evaluation included the review of existing project documents; interviews with key staff from project partners, partner countries and development agencies in partner countries; and a survey deployed to beneficiaries, carried out jointly with the project’s Monitoring and Evaluation (M&E) expert, using statistical sampling. A field mission for on-site observation and interviews was not possible due to the global emergency situation caused by the COVID-19 pandemic.

The evaluation had several limitations and challenges such as the cancellation of a field mission due to COVID-19 which resulted in the need to redefine data collection tools. Data was collected by the UNITAR/UNOSAT local focal points in Fiji and Vanuatu only under the supervision of the evaluation expert. No stakeholders were interviewed from Solomon Islands, however. Additionally, cyclone Harold further delayed data collection.

Despite the lack of outcome-level results, the project remains relevant for most of the project’s stakeholders. Most of the participants found the content of the training interesting and relevant to their jobs. Backstopping activities are also highly appreciated for their capacity to respond quickly and effectively to beneficiaries’ demands.

In terms of coherence, the project  generally aligns with sector policies and national strategies. However, the evaluation found evidence of overlap and duplication, or the risk arising, with projects of other development partners or regional institutions.

The project includes a Theory of Change and logframe that complement each other. Indicators, baselines and targets are provided at the output, outcome and impact level. Most of them are relevant and pertinent, but there is room for improvement.

Despite the project including gender-based monitoring with sex-disaggregated data collection, the project cannot be considered to be gender-sensitive or one in which gender is effectively mainstreamed. A gender analysis at the beginning of the project is missing, and a deepening in the gendered data collected would be desirable and necessary to address existing gender gaps.

Given the scope and magnitude of CommonSensing, project management  has proved to be challenging with multiple stakeholders and implementing agencies.

The preliminary cost-effectiveness analysis finds that the CS project is cost-effective compared to other projects that offer non-space alternative solutions, such as unmanned aerial vehicles and helicopters. In fact, the satellite-supported method would have a lower cost-effectiveness ratio than the UAV and aircraft solution during both the project implementation period (2018–2021) and beyond.

Although it is difficult to determine the level of impact of the project at this stage, some signs of impact are traced to capacity development resulting from training and backstopping activities. Notably, 80 to 90 per cent of participants considered having achieved ‘high or moderate competency in utilising EO for DRR and CCA’ as per participant feedback provided after the trainings.

Finally, it was difficult at this time to determine the sustainability of the project, not only because the CS Platform has not been completed, climate finance advisors are not yet in the field, and business models and sustainability plans are still under development.

Based on the above findings, the evaluation issued nine recommendations:

On coherence

Recommendation 1: UNITAR and Catapult should establish a mechanism to ensure the overall complementarity and coherence of activities and outputs so the results chain can deliver the expected intermediate and final outcomes.

On internal communication

Recommendation 2: UNITAR and Catapult should strengthen information-sharing, including information about what each partner is doing and how this is done.

On stakeholder engagement

Recommendation 3: UNITAR and Catapult should further strengthen relationships, communication and visibility with beneficiary institutions and the most relevant development partners or at least with those who show interest.

On external communication

Recommendation 4: UNITAR and Catapult should strengthen external communication and visibility of the project’s results.

On gender

Recommendation 5: UNITAR and Catapult should elaborate case studies to deepen information on gender issues and the potential of women of becoming drivers of change in the sector.

On the log frame

Recommendation 6: UNITAR and Catapult should review the outcome indicators 5 to 10. This may include:

1. Merging or deleting indicators that might be tracking the same or similar information, which will help to better monitor project performance and map specific outcomes; and
2. Including intermediate outcome indicators in order to fill the current gap between outputs and more general outcomes, and include qualitative indicators complementary to the current ones as suggested by the baseline evaluation.

Recommendation 7: UNITAR and Catapult should address the inconsistencies between the ToC and the log frame with particular attention to alignment with the outcome and impact of the log frame.

On capacity development through learning

Recommendation 8: UNITAR and Catapult should develop a process or system where capacity development planning and assessment systems are integrated to promote individual and organizational learning and improvement strategies (e.g. developing learning paths, close beneficiary tracing up and follow up coaching/mentoring).

On sustainability

Recommendation 9: The TA in climate financing will not automatically make the project sustainable over time. Sustainability requires ownership, and ownership is built through engagement at the policy and political levels. UNITAR should improve engagement with government officials from line ministries as well as from prime ministers’ office and/or cabinet.

The evaluation identified the following lessons:

1. Integrating flexible mechanisms such as backstopping activities that can react to local demands, particularly in rapidly changing environments such as the Pacific Islands, is supportive of successful project implementation.
2. Coherence and alignment between the log frame and ToC is key for a clear pathway to impact.
3. Both internal and external communication is key for project success.
4. A governance system based on co-leadership is complex and requires frequent interactions and exchanges.
5. With country projects it is important to involve local partners and engage with a wider community as much as possible to ensure local ownership and participation.