An independent evaluation of the Afghanistan Fellowship Programme (AFP) was published in July 2018, assessing the AFP’s performance over the 2014 – 2017 Fellowship cycles. The programme aims to contribute to Afghanistan’s reconstruction by building the capacities of its civil service, with a view to institutional capacity building. The evaluation was informed by a document and literature review, key informant interviews, an online survey and observed training sessions.

Several limitations of the evaluation were noted, namely the security situation in Afghanistan preventing a field visit, and the reasonably limited temporal scope of the evaluation. The absence of a results framework, a Theory of Change (ToC) and baseline data also hindered the measurement of the results. The assessment ultimately is based on contribution analysis.

Relevance

The evaluation found that the AFP was aligned with the Afghan National Priorities, particularly the Governments National Priority Programme No. 3 on effective government. The AFP was also found to be well aligned with the UNITAR 2018-2021 Strategic Objective 1, as well as SDG’s 4, 5.5 and 16. The potential to contribute to SDG 5.5, on increasing the proportion of women in managerial positions, largely remains to be achieved however.

There was an appreciation for the innovative, blended learning approach of the programme design.

The AFPs multi-year approach also follows best practice for individual and institutional capacity development in post-conflict countries, something appreciated by graduates and other national stakeholders.

A few survey respondents felt that the process for the selection of fellows, coaches and mentors should be more robust and transparent.

There is a need to update aspects of the programme to reflect many of the changes in the country since the AFP was originally formulated.

40 per cent of respondents rated the Programme as providing a great deal or a lot of value for money to their jobs, and 36 per cent of respondents found the Programme to offer little or no value to their jobs. More than 70 per cent felt that the programme would have more value for money if it were offered as a degree programme. 

Effectiveness

Significant increase in skills in relation to the Programme’s learning objectives was expressed. Eighty-eight per cent also reported applying what was learned from the Programme to improve job performance.

Evidence of application of knowledge/skills (level 3 in the Kirkpatrick model) in terms of continuing to work on collective and/or individual projects following cycle completion was collected. Nineteen per cent of survey respondents indicated that projects were fully implemented, and 58 per cent reported them being partly implemented.

Evidence against Kirkpatrick levels 1 and 2 are highly satisfactory. The Programme does not have any metric for assessing post-cycle performance or other intended development changes.

No objective assessments of knowledge or skills are administered, and virtually all fellows receive a certificate of completion.

Moderate success in leveraging partnerships, with some national and UN partnerships being very limited, while partnerships with the private sector and academia have been more successful.

Efficiency

The evaluation found that delivery of the AFP has been efficient and cost-effective.

M&E results are used to create narrative completion reports, which identify recommendations and lessons learned.

The Programme’s costing and mixed funding arrangement, the annual cycle delivery format and the lack of a robust results framework limit the extent to which the Hiroshima Office can monitor and assess AFP results at higher outcome levels and determine the extent to which the Programme is achieving its stated goals.

Sustainability

The AFP does not have any explicit handover or exit strategy, and planned alumni networks are weak when they exist at all.

There is some evidence on the institutionalisation of some of the Fellow’s projects.

46 of 54 survey respondents confirmed that they applied what they had learned from the AFP to improve their job performance, perhaps the best sign of sustainability.

“UNITAR participation for me has been a great personal and career development opportunity”

1. Establish a light AFP Advisory Group.
2. Review, in consultation with the AFP’s stakeholders, the Programme’s competencies, content and methods.
3. Review current eligibility requirements of the AFP and ensure that selection and promotion process is standardised and transparent.
4. Articulate a women’s empowerment strategy in the framework of the Programme.
5. Articulate a clear theory of change and results framework with relevant metrics to assess the AFP’s medium to long-term results.
6. Conduct a tracer study to identify which AFP group projects, as well as individual projects which have been developed out of group projects, have been institutionalised.
7. Develop a platform to promote a more purposeful community of AFP alumni.
8. UNITAR should request membership as a Non-Resident Agency in the Afghanistan UNCT.

• Identifying, building and maintaining partnerships requires time and perseverance, as well as an alignment of potential partners’ expectations.
• Capacity Development requires ongoing awareness-raising, accompaniment and tracking. This tracking is essential for learning-related programming which aims to bring about institutional transformation.
• Women empowerment will require a more strategic approach, beyond requesting national partners to ensure that they select more female Fellowship candidates.
• For the AFP to be truly nationally-owned, the content of its curriculum should be driven by national priorities and guided by strategic national partners.