Consultancy for project ex-post evaluation

CONSULTANCY NOTICE - EXTENSION

(Open to national and international candidates)

Commissioned by: The International Organization for Migration (IOM) Mission in Viet Nam  

Managed by: National Programme Officer (LMI)   

1.    Evaluation context 

??Viet Nam's labour productivity reportedly lags behind that of other ASEAN nations, primarily due to a scarcity of skilled workers which can be attributed to deficiencies in vocational training and human resource quality. Shortcomings include a lack of technical and digital skills, leading to a mismatch in core work qualifications. Digital skills have become imperative, aligning with the demands of the fourth industrial revolution and the challenges posed by COVID-19. Acknowledging that there is an inadequacy in the digital preparedness of the country's workforce, the Government of Viet Nam (GoVN) is actively promoting digital transformation across all sectors, particularly in technical and vocational education and training (TVET). Government policies are focused on harnessing digital capabilities to enhance the employability and productivity of the working-age population, with specific attention to empowering vulnerable and disadvantaged groups. 

In alignment with national digital transformation initiatives and with the objective of empowering prospective and returning migrants, IOM Viet Nam in partnership with Microsoft launched an online open-source digital learning platform in 2021. The platform is specifically tailored to enhance the digital literacy of migrant workers, thereby amplifying their prospects for employment; it has been successfully introduced across various vocational colleges in Viet Nam through multiple IOM projects, strategically supported by diverse government partners, showcasing a widespread and positive implementation.  

Building on the ongoing public-private partnership with Microsoft and the strong interest of the GoVN in leveraging this platform, IOM Viet Nam secured seed funding from the IOM Development Fund (IOMDF) to implement the project “Supporting the Government of Viet Nam in the Digital Transformation of Labour through Enhancing Technical and Vocational Education and Training” from September 2021 to August 2023. The project’s main objective was to contribute to Viet Nam’s digital transformation of labour through enhancing digital skills of Vietnamese youth labour force including migrant workers and strengthening capacity of TVET authorities to implement digital transformation in the sub-sector. 

The project involved two outcomes and four outputs: 

Outcome 1: Vietnamese youth and migrant workers improve their digital skills through the updated Microsoft-IOM Learning Platform. 

• Output 1.1: Updated platform is available for use by TVET authorities and other concerned stakeholders. 
• Output 1.2: Vietnamese youth and migrant workers are sensitized on and have access to the updated platform. 

Outcome 2: TVET authorities and managers implement the process of digital transformation in the TVET sub-sector. 

• Output 2.1: “Digital Transformation Plan: How-to Guide” is developed and available for TVET institutions. 
• Output 2.2: TVET authorities have strengthened capacity in supporting digital transformation in the sub-sector.  

The project was aligned with the GoVN’s priorities, including Resolution No.52-NQ/TW in 2019 of the Politburo on some directions to proactively participate in the IR 4.0 and Decision No.749/QD-TTg in 2020 by the Prime Minister on “the National Strategy for Digital Transformation” with a stress on digital skills for workers. Additionally, it contributed to IOM’s MiGOF Objectives 1 “Advance the socioeconomic well-being of migrants and society,” former Strategic Vision Pillar 2, and Asia Pacific Regional Strategic Priorities 4.2 on Mobility. It also showed the relevance of digitalization in TVET for Sustainable Development Goals specifically SDG 4, 8, 9, and 17, as well as Objective 2 and 18 of the Global Compact for Migration. Although the project was developed and implemented prior to the roll out of IOM’s new Strategic Results Framework, it corresponds well to long-term outcome 3A “Channels for regular migration are sustainable and responsive to current and emerging trends.” 

2. Evaluation purpose and objective 

The evaluation is an IOM ex-post evaluation to be conducted through an external consultant or consultant team for accountability and learning, intending to inform IOM Viet Nam and the IOMDF about the immediate and medium-term outcomes of the project. The specific objective of the evaluation is to assess the relevance of project design, coherence of the interventions vis-à-vis other interventions, the effectiveness and performance of the project, the efficiency of project management and implementation, the impact and sustainability of the project.  

Additionally, the evaluation will draw out lessons learned and good practices that will inform the development of future projects. Cross-cutting themes such as gender and rights-based approach will be analysed where relevant. The IOMDF will use the findings of the evaluation in its decision-making on the use of the Fund as seed funding, and IOM Viet Nam will use the recommendations at a strategic level to improve future interventions.  

3. Evaluation scope 

The evaluation will cover the period of project implementation from 1 September 2021 to 31 August 2023, focusing on all outcomes and outputs. It will primarily involve key stakeholders and beneficiaries of the project, including the General Directorate of Vocational Education and Training (DVET), Viet Nam General Confederation of Labour, capacitated TVET trainers, and youth who received skills training under the project. For Outcome 1, the evaluation will focus on the project target locations of Dong Nai and Binh Duong, to be decided upon further consultation and agreement with DVET. 

4. Evaluation criteria  

Given the overall purpose, the evaluation will cover the criteria of relevance, coherence, effectiveness, efficiency, impact and sustainability. In total, the evaluation criteria form the basis and guidance for the evaluation suggested questions as presented below. The consultant will analyse and present findings per each criteria according to the IOM Development Fund’s scoring matrix which will be provided during the inception discussion. 

5. Evaluation questions 

Relevance: 

• To what extent was the project aligned with national priorities and policies?  
• To what extent were the needs of beneficiaries and stakeholders considered during project design? Was the project designed based on a solid rationale/needs assessment? 
• Were the project activities and outputs consistent with the intended outcomes and objective?  
• How is the quality of the intervention logic design? Was the project designed with a logical connection between its objective, outcomes, outputs and indicators based on a solid rationale/needs assessment? 

Coherence: 

• Were project activities coordinated with other actors? If yes, to what extent? 
• Was there any synergy and interlinkage between project activities and other interventions carried out by the local institutions/government agencies as well as IOM Viet Nam? 

Effectiveness: 

• Have the project’s outputs, outcomes, and objective been achieved across all gender groups and those most left behind in accordance with the stated plans and Result Matrix? Is there any gap between the desired and achieved results? How can it be interpreted and explained? 
• What does each beneficiary and stakeholder group think of the quality, usefulness, and scalability of the updated learning platform?  

Efficiency: 

• Were the activities implemented according to the schedule of the project’s work plan? If not, what were the reasons and the steps taken to complete them? 

1. Were challenges in project implementation identified and addressed quickly and appropriately? 

• How efficiently were the resources used to achieve the intended results in line with the implementation context?  

Impact: 

• What intended or unintended positive effects were produced by the project? 
• What intended or unintended negative effects were produced by the project, if any? 

Sustainability: 

• What were the major factors affecting sustainability? 
• Have structures, resources, and processes been established to ensure that the benefits generated by the project persist after external support has concluded? 
• To what extent has the project been integrated into institutional structures that are expected to endure beyond the project's lifespan? 

Cross-cutting issues 

• To what extent were gender mainstreaming and the inclusion of vulnerable and disadvantaged groups integrated into the project design and implementation? 
• Was the project designed, planned and implemented with consideration of a Rights-Based Approach (HRBA) and adherence to available guidelines? 

6. Evaluation methodology 

The consultant is ultimately responsible for the development of the overall methodological approach to achieve the aims of this evaluation. The final methodology and proposed approach will be discussed and agreed with the Evaluation Manager. 

Efforts shall be exerted to ensure the inclusivity and engagement of relevant stakeholders to bring out their voices on how they perceived the implementation and results of the project. Focus group discussions with each beneficiary group should be held. Similarly, interviews with a selected sample from the beneficiaries should be conducted to assess how the project has responded according to their expectations, objectives and priorities. The sampling method will be purposive, and sample size will be determined during the inception phase.  

7. Ethics, norms, and standards for evaluation  

The evaluation processes shall be in line with IOM Data Protection Manual, do no harm principles, and the norms and standards of UNEG as well as the ethical conduct guidelines of UNEG and with the UNEG codes of conduct. IOM Viet Nam will provide technical support and guidance during the process. The M&E Officer at IOM's Regional Office in Bangkok will be consulted when deemed necessary.  

8. Evaluation deliverables  

The consultant should: 

1. Produce an Inception Report using the IOM template and including a stakeholder mapping, an evaluation matrix, data collection tools and workplan.  
2. Share all data collected from the different sources.  
3. Draft the Evaluation Report for IOM’s review and comments. 
4. Facilitate briefing and debriefing meetings, in addition to the routine meetings and discussions, with the Evaluation Manager, project team, and Reference Group.  
5. Produce a Final Evaluation Report (incorporating comments and technical inputs from the Reference Group) with a summary of the evaluation (two-page Evaluation Brief). 
6. Deliver the partially completed Management Response Matrix (MRM). 

9. Specification of roles 

The Evaluation Manager will oversee the evaluation and act as the main contact for the selected consultant. The Evaluation Manager will arrange an initial management meeting to ensure a common understanding of the ToR and evaluation process, confirm final logistics, and review and give inputs into data collection plans as outlined in the Inception Report. The project team at IOM Viet Nam will provide support in arranging interviews with project stakeholders and beneficiaries.  

Representatives from IOM Viet Nam (Evaluation Manager, MEAL Officer, Reporting and Research Officer), IOM Regional Office for Asia and Pacific (ROMERO), IOMDF, and DVET will be closely engaged to offer technical guidance to the consultant during the evaluation. This group will be dedicated to reviewing deliverables and providing constructive feedback throughout the process. 

The consultant will be accountable for executing the evaluation in accordance with the current TOR and the specified deliverables. He/she will facilitate briefing and debriefing meetings in addition to the routine meetings and discussions with the Evaluation Manager and the relevant stakeholders mentioned above. Approval from IOM is a requisite for all deliverables, and coordination with the Evaluation Manager is essential throughout the process. 

Once approved by the Evaluation Manager, the report, brief, and management response will be shared first with the ROMERO for endorsement, and then with the IOMDF for final approval. 

10. Time schedule 

The assignment is expected to commence as soon as the contract is signed with the final report expected by December 2024. 

11. Budget 

The total budget, covering all expenses related to the evaluation process, shall be proposed by the consultant and approved by the Evaluation Manager. The payment terms shall be issued per the terms and condition of the Purchase Order (PO) based on the disbursement schedule below: 

• Satisfactory inception report submission – 30% 
• Submission and approval of first draft of the Evaluation Report together with raw data – 30% 
• Satisfactory final Evaluation Report, Evaluation Brief and MRM – 40% 

The final payment shall be issued not less than 30 days upon (1) the completion of the work, (2) receipt of the final original invoice and (3) receipt of the final Evaluation Report, Evaluation Brief and MRM following the incorporation of all feedback received.  

12. Evaluation requirements 

The consultant/consultant team should possess the following minimum qualifications: 

• Master's degree in advanced applied research/evaluation methods, or any related academic discipline or an affiliation with a research institution or a university, holding a PhD, or being in pursuit of a PhD in a relevant field is an advantage. 
• Experience in evaluating development programmes or at least two evaluation contracts of similar value, nature and complexity implemented over the last three years or more. 
• Familiarity with Viet Nam’s socio-economic development context, especially regarding labour migration, skills development, digitalization, or related subject areas. 
• Strong background in monitoring and evaluation techniques. 
• Experience in developing and implementing evaluations with the UN, international NGOs, or donors. 
• Experience and expertise in mainstreaming gender equality and women empowerment and human rights principles in evaluations. 
• Familiarity with the OECD/DAC and UNEG evaluation framework. 
• Excellent knowledge and experience in survey design, implementation of surveys and statistical data analysis. 
• Excellent analytical, communication, writing and presentation skills in English. 
• Ability to analyse complex intervention. 

13. Submission of application 

The interested consultant/consultant team should submit a technical proposal (detailed evaluation methodology, indicative work plan, and the overall approach to the evaluation) and an all-inclusive financial proposal (cost breakdown, including travel, interpretation, and accommodation costs if required).  The submission of proposals (technical and financial) and/or related questions should be directed to [email protected] no later than 30 June 2024 

The submission should also include the followings: 

• A cover letter; 
• CV and biographies of the lead consultant and evaluation team members (if any); 
• References for each evaluation team member; 
• Examples of evaluations conducted. 

Important note: When evaluating the competing applicants, IOM will consider the written qualifications/capability, financial offer, the information provided by the applicants, and any other information obtained by IOM through its research.  

Applications are non-returnable and only short-listed candidates will be contacted.

Female consultants are encouraged to bid for this required service. Preference will be given to equally technically qualified female consultants.