The CommonSensing project is funded by the United Kingdom Space Agency and aims to strengthen disaster risk reduction and climate change resilience in Fiji, Solomon Islands and Vanuatu through 1) increasing national resource capacities in the use of Earth Observation (EO) solutions to address disaster risk reduction and climate change resilience by 2020, and 2) enhancing evidence-based decision making by using CS solutions for disaster risk reduction and climate change adaptation (CCA).

The endline evaluation assesses the project’s effectiveness, efficiency and likelihood of impact and sustainability. This includes the assessment of project performance at the output, outcome and impact levels in accordance with the log frame. The evaluation’s terms of reference also requested the identification of enabling and disabling factors, and the issuance of recommendations and lessons learned.

The evaluation team comprised an international senior expert as the evaluation team leader and three local experts, with one based in each of the three project countries. Using a mixed methods approach, the evaluation included the review of existing project documents; interviews with key staff from project partners and partner countries; and a survey deployed to beneficiaries, carried out jointly with the project’s monitoring and evaluation (M&E) expert, using statistical sampling; a focus group to deal with gender-related issues; and an outcome mapping exercise. A field mission for on-site observation and interviews by the team leader were not possible due to the COVID-19 pandemic. 

The evaluation had several limitations and challenges such as restrictions on mobility at the international level due to COVID-19 limited data collection and natural disasters affecting mainly Fiji and Solomon Islands limiting the availability of stakeholders to be interviewed. Moreover, the evaluation was performed while the project was still being implemented, with some key activities remaining to be implemented, including the completion of the data cube setting and its use (e.g. delivery of user trainings) as well as the provision of climate finance technical assistance. Furthermore, some targets of the log frame were modified by project management, and new indicators added in the last three months of the project when the present evaluation was underway. Data collection in Samoa to allow for comparison with a counterfactual for a possible legacy evaluation proved difficult due to the absence of a climate finance database in Samoa.

At the time of the evaluation’s data collection and analysis, most project partners had completed all work packages, and some 83 per cent of the project budget had been spent. Most activities delivered relate to capacity development and creation of data cube and tools. Two key project components, the installation and functioning of the CommonSensing Platform (CS Platform) and technical advisory assistance for accessing climate funds were not completed by the project’s initially scheduled completion date (31 March 2021).

Regarding capacity development in the form of training and technical backstopping activities, the project remains relevant for most project stakeholders throughout implementation. On project impact, some signs of impact could be traced to capacity development from training and backstopping activities, with evidence of skills and knowledge acquired being used in policy making and planning emergency preparedness. 

The evaluation found limited evidence that the CS Platform and backstopping activities have been used to apply for climate funds, however. At the time of data collection and analysis, at least two applications for climate-related funding were prepared and were likely finalised and submitted to donors. Reasons that could explain this limited use of CS project outputs for climate financing are attributed to the CS Platform not being completed and handed over to the stakeholders in time to build relevant capacity of government officials working on climate finance. The late joining of climate finance advisors also led to the delay of publication of CF manuals and workshop delivery.

Following the recommendations of the midline evaluation, project management introduced modifications that led to the improvement of communication among project partners and coordination at the output level. Additional support was hired to enhance stakeholders’ engagement at the country level.

Project implementation was initially heavily affected by travel and mobility restrictions following the declaration of the COVID-19 pandemic. Due to much uncertainty, project management took approximately three months to reorganize delivery timelines and methods, including the reallocation of budget. The finalization of the CS Platform and the provision of climate finance training and on-the-job technical assistance accumulated significant delays. Nevertheless, the project partners found effective solutions to continue the delivery of project activities that involved converting in-person training into blended learning activities or creating systems that could allow the development of the CS Platform for Vanuatu and Solomon Islands remotely.

During the final months of project implementation, partners made much effort to give more visibility to the role of women in the sector, including the publication of case studies and setting participation quotas in training, with some degree of success. As in the midline evaluation, there was evidence observed that women showed less confidence in improving their knowledge.

Concerning the achievement of project target results, the attainment of expected outcomes remained somewhat linked to performance (or underperformance) at the output level. Consequently, outcomes related to climate finance, including the use of the CS Platform for climate finance were not fully achieved. At the impact level, it was difficult to assess performance due to difficulties in collecting data as well as attribution issues. Nonetheless, the contribution of CS to the achievement of impact targets lacks sufficient evidence. 

It was difficult at the time of the evaluation to assess project sustainability as the ultimate end date of the project was not clear. As mentioned, the CS Platform was not fully functioning in the three targeted countries, and climate finance advisors had only recently started working. Independent of project extension, the evaluation found consortium partners to be committed to seeking alternative funding to complete climate finance-related activities and to ensure the installation of the CS Platform in Vanuatu and Solomon Islands after the project’s planned end date of March 2021.

Based on the above findings, the evaluation issues the following four recommendations, with the assumption that an additional no-cost extension will be granted through March 2022. 

Recommendation 1: UNITAR-UNOSAT and Catapult should complete the delivery of all project activities in the next nine to 12 months. In particular, it is recommended that UNITAR-UNOSAT continue to deliver some key technical trainings using the existing online and distance learning platforms to ensure complementarity with the use of the CS Platform and, in turn, ensure coordination and complementarity of output delivery during the last months of the project. This is also important in terms of sustainability as it could serve as guidance to partner countries on how to use and ensure the sustainability of the results once the project is completed.

Recommendation 2: Based on the information and experience gathering data to inform project indicators, UNITAR-UNOSAT and Catapult should delete the log frame indicators that are not measurable and review the collection of data and data collection methods where needed.

Recommendation 3: Recommendations provided in the midline evaluation are applicable to the no-cost extension. It is strongly recommended that project partners focus on ensuring project sustainability and place special attention to strengthening the capacity of partner countries in climate financing and climate funding. Therefore, it is important that climate finance advisors:

• Narrow the scope of institutions (e.g. Ministry of Finance, Ministry of Environment, National Disaster Management Office) for participating to capacity development activities by targeting staff and institutions involved in climate finance applications only.
• Follow up on policy and budget processes so that governments allocate the necessary human and financial resources to sustain project results in the medium/long term as well as ensure the protection of data.
• Provide support to enhance data collection in terms of climate funding, as the three countries seem to experience challenges in collecting and tracking climate finance information as indicated by project performance results.
• It is recommended that UNITAR-UNOSAT and Catapult continue to place effort on engaging stakeholders and seize the extension period to increase outreach by involving civil society organizations and other development partners beyond those in the region.

Recommendation 4: UNITAR-UNOSAT and Catapult should continue to benefit from project results and experiences by drafting and publishing articles and case studies related to the use of EO for combating climate change and enhancing DRR and continue to make the gender-related issues in the sector more visible. 

The evaluation identified the following lessons:

Lesson 1: Importance of defining realistic, measurable results. While project log frames are dynamic instruments and may be subject to review and modification, it is important that project metrics have means of verification and can be measured within the project’s resource constraints. 

Lesson 2: Importance of gender analysis to ensure gender mainstreaming. A thorough gender assessment is important to undertake as part of the needs assessment and analysis to ensure that gender mainstreaming in project design is relevant and adequate and can be realistically delivered throughout project implementation. 

Lesson 3: Importance of local staff and partners in consortia. Engaging local staff and institutional partners is instrumental to support effective project delivery and ensure ownership and sustainability of results. This is particularly important for projects implemented in geographic regions distant from the location of the main project partners. 

Lesson 4: Measures to ensure sustainability need to be front-loaded. The more measures to promote sustainability of results are front-loaded, the more the likelihood that such measures will become part of the process of delivering outputs and ensure sustainability of outcomes.  

Lesson 5: Uncertainty about  the end date of a project leads to planning insecurity, implementation and spending imbalances.

Lesson 6: High transaction costs are associated with turnover in personnel of project delivery and beneficiary partners. Turnover of staff of delivery and beneficiary partners can produce delays in implementation and reverberate and create inefficiencies by delaying output delivery and compromising achievement of outcomes. 

Lesson 7: Unintended outcomes can be highly relevant, appreciated and rewarding. Adaptive management is crucial to address important niche areas for capacity support, such as the case for responding to demand-driven and tailored backstopping support and addressing digital divide by supporting online and blended learning solutions in the wake of the COVID-19 pandemic. 

Lesson 8: The project’s financial business model can present risks for efficient delivery of results. The absence of advance-funding creates challenges in financial management that may affect and present risks related to project planning, spending and ultimately delivery of results.