UNITAR is guided by a four-year Strategic Framework which articulates the Institute’s vision, mission, core values and strategic objectives and enablers. The present Framework, covering the period from 2018 to 2021, was approved by the Institute’s Board of Trustees in November 2017.

The Framework structures the Institute’s objectives, programming and activities under the peace, people, planet and prosperity pillars of the 2030 Agenda for Sustainable, in addition to cross-cutting programme pillars on accelerating the implementation of the 2030 Agenda, multilateral diplomacy and optimizing the use of technologies for evidence-based decision-making. The evaluation covered the period of the first two years of the four-year Framework and focused on progress achieving the five strategic objectives. The exercise was also forward-looking to help inform the subsequent strategy from 2022.

The evaluation used both qualitative and quantitative approaches, commencing with an initial desk review and stakeholder analysis. The key evaluation tools used included  surveys, stakeholder interviews, focus group discussions, and a document review. The evaluation further used a theory of change (ToC) approach to assess implementation effectiveness against the expected causal pathway. The evaluation further adopted a participatory approach by seeking guidance from a reference group of the UNITAR Board of Trustees and included consultation with key stakeholder groups including UNITAR staff, partners, donors and participants.

Limitations include the following:

• The evaluation did not seek to assess individual project performance.
• It relied on remote data collection.
• There was uneven survey response rate and stakeholder engagement across different groups, particularly with reliance of electronic means of engagement.

Relevance. The Strategic Framework and its implementation are highly relevant in relation to responding to global challenges. UNITAR is responding well to the demand of donors, partners and participants. The structure of the Framework around the “P”s of the 2030 Agenda is straightforward for staff, partners, donors and participants to understand.

Coherence. As an autonomous UN agency, UNITAR is seen by donors and partners as having a clear point of difference in the training sector. At present, UNITAR does not have a field network but does engage well in some countries, particularly where there are strong partnerships with other institutions. Engagement with the United Nations Sustainable Development Cooperation Framework (UNSDCF) is not well developed yet would provide an opportunity for UNITAR to increase connection at the country level and promote UNITAR’s role and capacity.

Effectiveness. The feedback on UNITAR’s effectiveness in delivery of results is uniformly high. Programmes are well-designed, in line with agreed objectives. The achievement of outputs at the project level across the divisions is generally positive.

There has been a shift towards larger, multi-year programmes. To date, the results of these are promising and well-recognized by partners. Yet, the organization still relies heavily on smaller scale events as its mainstay of programming. Small-scale events are generally successful, the results are wider and more evident where the event supports efforts that are related to institutional strengthening and/or training of trainers. Similarly, small scale events with limited duration and funding are difficult to monitor changes beyond outputs and immediate outcomes so the achievement of results and the contribution towards strategic outcomes is difficult to assess. A stronger focus on training of trainers to facilitate wider reach with training material is a potential area for improvement.

There has been an effect of COVID-19 on programming, with many programmes and events being delayed, but recent feedback from partners indicate that UNITAR has been extremely flexible and responsive to required changes. Many events are now proceeding in the digital domain, in some cases with even wider reach than pre-COVID, whilst others have been unable to proceed.

Given that the expertise of staff is a strength, the substantial amount of time that key personnel spend in resource mobilization activities detracts from time that could be invested in effective programming. There are opportunities for more integrated programming between divisions, but only in specific, carefully targeted interventions where the potential of added value from integration is clear. There could also be greater inclusion of cross-cutting issues in programme design such as focus on human rights, gender equality, disability and support to vulnerable groups.

Efficiency. UNITAR is seen by partners and beneficiaries as providing good value for money. Furthermore, project management is considered efficient because results are generally delivered on time and within budget. Where adjustments have been required, UNITAR is respected for being in close contact with partners to discuss requirements. However, there are challenges to efficiency. UNITAR’s business model is based on a programme-driven fundraising approach and the average project size is small. The average project value with an average duration of 11 months is $223,833 and if the 8 large projects (>$3 million) are counted separately, the average value per project drops to $174,080. For a small organization with few administrative staff, this places a heavy burden on resource mobilization and contract management functions.

The Strategic Framework Fund (SFF), established in November 2018, has potential to be a cost-effective vehicle for pooling loosely earmarked funds, but as it is a recently established instrument its performance is yet to be demonstrated to meet the needs of countries in special situations and other vulnerable groups. Another potential avenue for increased leveraging of funds is increased integration into the UN system at the country level through mechanisms such as Common Country Analyses (CCA) and the UNSDCF.

A further issue is the loss of revenue resulting from deviations to full cost recovery rates.

Sustainability. Sustainability was assessed in two ways, the sustainability of programme results, where applicable, and UNITAR’s broader contribution to sustainable development through the Strategic Framework approach. Both staff and external stakeholders were clear that UNITAR is committed to sustainability and that programming is explicitly aimed to achieve more sustainable outcomes. This occurs through the sustainable change in knowledge and capacity of the individual participants, and also through institutional change in some cases where UNITAR’s support is designed to support institutional strengthening, particularly with longer-term engagement. A clear intervention and exit strategy can assist in defining expected pathways to sustainability. The alignment of the Framework to the 2030 Agenda is a definitive step towards contributing towards broader sustainable development.

Impact. The assessment of impact is challenging because in general, there is little tracking of impact in UNITAR apart from follow-up evaluations to assess the extent to which training has been applied or has given rise to higher level outcomes. Given the limited project level and corporate resources available for evaluations, the level of impact data is sparse. Staff, stakeholders and participants all expressed interest in greater follow-up to check on and build on results to attain greater impact. Nonetheless, the feedback directly from participants during this evaluation indicates that the impact of UNITAR’s support may already be more far-reaching than expected or presently reported.

Reports from participants and partners suggest that learning, particularly in training-of-trainer events, is being passed on to a wider audience. This is occurring particularly where participants are themselves teachers, trainers or in a position of influence within a country or community. Yet, even direct participants in most cases during the evaluation focus group discussions mentioned that they had transferred knowledge either formally or informally to other people, engaged in activities where training content was then included in changing work practice, was embedded in organisational training or was shared with colleagues and friends, building their knowledge and capacity. However, these results and the actual impact achieved within each country supported could be better documented and showcased to demonstrate wider impact. The impact stories launched in 2018 are a good step in this direction and if developed more systematically could provide useful learning on what impact is being achieved, and how wider benefits could be achieved.

Crosscutting Issues. During the period of the Framework there is evidence of increasing attention to issues of countries in special situations and vulnerable communities. UNITAR’s main partners are already committed to improving outcomes for those left behind and programming for these audiences is integral. There has been some feedback that more can be done to raise the profile of strategic and transformational approaches to gender equality.

The feedback from beneficiaries was that UNITAR strongly promotes and strengthens knowledge on environment, gender equality and human rights but there is less attention to young people and Indigenous Peoples, particularly disability. In addition, given the predominant focus on English language courses, there are some barriers for non-English speakers.

Based on the above findings, the evaluation issued a set of five recommendations to inform the on-going implementation of the Framework and future strategic direction:

• R1. Ensure responsiveness to the needs and priorities of intended users, donors and strategic partners, and increase linkages to UN system particularly UNSDCF.
• R2. Maintain clear alignment with the present development agenda and continue the Framework for next 4-year cycle and place more effort on aligning activities to objectives.
• R3. Focus programming on core functions for delivering maximum, longer-term results.
• R4. Deliver more and wider results through targeted fund-raising and cost-recovery.
• R5: Seek to further understand and expand UNITAR’s contribution to impact

• L1: The Strategic Framework aligned to the 2030 Agenda is powerful on the global stage and is also recognised at the local level so is an effective framework to adopt and continue.
• L2: UNITAR, as a small agency, cannot easily directly reach the most vulnerable populations in its countries of operation. Yet, UNITAR’s partnerships with relevant institutions that support the most vulnerable enable UNITAR to influence sustainable solutions without direct intervention. Working at the institutional level does lead to positive benefits for vulnerable people.
• L3: UNITAR’s work on frontier technologies is highly valued and has a large potential to influence all areas of UNITAR’s work. UNITAR has a natural inclination towards innovation and technology which should be nurtured but not forced where it is likely to be resource intensive without sufficient return.
• L4: While UNITAR is more expensive compared to some other capacity building actors the quality of their activities is high enough to warrant the extra pricing and strategic partners appreciate the quality of activities.
• L5: UNITAR’s impact is evident through informal and unexpected channels such as the sharing of knowledge by participants through personal networks.