The Strategic Framework Fund (SFF) was established by the UNITAR Board of Trustees in November 2018 as a flexible, pooled funding instrument to help UNITAR deliver on its mandate and achieve the objectives of the 2018-2021 strategic framework.

The purpose of the evaluation is to assess the relevance, coherence, effectiveness, efficiency, impact, and sustainability of SFF-related programming; to identify any problems or challenges that the SFF has encountered; to issue recommendations, and to identify lessons to be learned on the SFF’s design, implementation, and management.

The evaluation employed a mixed methods approach which incorporated a desk review of project documentation, online survey and key informant interviews (including UNITAR directors, managers, the Executive Director, project beneficiaries, a donor and an implementing partner). In total, 47 allocations formed part of the evaluation scope based on the availability of documentation and the implementation phase of the projects.

The evaluation's main limitation is the small-scale nature of the projects and the cluster approach which did not allow the evaluation to engage in a project-specific focus. Additionally, only 8 projects allocations were analysed in-depth. Thirdly, the data collected from the surveys was of subjective nature and is based on participants’ perceptions. Moreover, the use of colour codes to rate the evaluation criteria imposes a degree of misrepresentation given the variation in the borderline aggregate scores. Finally, the COVID-19 pandemic limited the evaluator’s ability to undertake field visits.

Using a four-level scale for assessment (poor, unsatisfactory, satisfactory and strong achievement), the evaluator found a varying degree of achievement across the evaluation criteria.

The project’s relevance to meet Member States’ and beneficiaries’ needs was assessed as satisfactory. However, the low level of outreach to countries in special situations and vulnerable groups, the large number of small-scale allocations and the relative low level of SFF resources attenuate the relevance of the SFF to the strategic framework’s emphasis on reaching the further behind first and helping Member States achieve the SDGs.

Coherence was assessed as poor. Only few projects are implemented jointly with other divisions. Nevertheless, the complementarity of the SFF-funded projects is increasing. Due to the small budget size and short timeframes for SFF-funded projects, the SFF funding modality is less likely to forge many long-term partnerships.

On effectiveness, the evaluation found that the SFF’s contribution to UNITAR’s achievement of its objectives was satisfactory. With UNITAR’s long-standing delivery on online training courses, the pandemic only affected the SFF delivery to a low degree. 

Moreover, with a timely response to Member State’s needs, a fast project approval process, a sound COVID-19 mitigation strategy and increased integration of gender considerations, the SFF obtained a strong achievement rating for efficiency.

The project’s likelihood of impact was assessed as satisfactory thanks to evidence on changes in individual knowledge and behavior. However, the lack of a measurable results structure hinders the evaluability of the degree of progress of impact.   

Despite the likelihood of the beneficiaries’ benefits to be sustainable, the institutionalization and the financial sustainability of the SFF remain uncertain. The likelihood of sustainability of the SFF thus attained poor achievement.

Based on the above findings, the evaluation issued seven recommendations:

On relevance

R1: While carefully balancing needs and/or requests from Member States and SFF allocation requirements, a clear “Leave no one behind” component should be required for each SFF-funded project.

On coherence and effectiveness

R2: Decisions on allocation awards should take a blended approach of small and short-term seed funding-oriented projects (based on clear criteria, demonstrated needs and clearly identified results which are sustainable) with a limited number of larger and longer-term projects with cross-divisional cooperation, where possible.

R3: Emphasize should be placed on the development of country-level partnerships, including UN Country Team engagement, in SFF project design and allocation decisions.

On effectiveness

R4: UNITAR programme units should ensure that allocation requests include a clearly defined results framework with specified Level 3 performance needs in project design and in post-training, including, for example, cost neutral communities of practice among alumni, and develop a budget/time criterion to ensure that larger scale and term future SFF projects include a compulsory post-training follow-up action to measure and assess performance components.

R5: UNITAR programme units should at least consider partial virtual training delivery or alternative delivery mechanisms such as radio broadcasting to leverage increased reach, reduce costs and reduce the environmental footprint. This could become additional SFF project allocation criteria.

On efficiency

See recommendations 1,5, and 7

On likelihood of impact

R6: Programme units should document impact stories using available guidance to analyse personal and institutional changes and the reach of training in Member States through alumni.

On sustainability

See recommendations 1 and 3

R7: The Executive Director’s Office, in cooperation with the Board of Trustees, should strengthen efforts to widen the SFF donor base and overall SFF resources, complementing the engagement of ambassadors in Geneva with a targeted outreach to donor capitals.

The evaluation identified four lessons learned:

L1: Moving training online requires stable internet connectivity and reliable access to electricity, which is not necessarily given in all Member States. The evaluation shows, based on other evaluative evidence, that the use of other training approaches such as broadcasting radio programs can significantly enhance the reach of UNITAR.

L2: The systematic use of the criteria of leaving no one behind and programming focusing on gender equality is required to have allocations support programming benefiting these groups.

L3: The SFF evaluation shows that cross-programmatic collaboration can help overcome silo cultures, but needs to be unambiguously communicated to all staff. It also requires a long-term focus and at sufficient scale for meaningful collaboration.

L4: To help ensure sustainability of results and scalability, it is important to include exit strategies right at the project design. Besides, linkages to complementary programming either in UNITAR or partners agencies need to be established as early as during the project design.