GOAL Syria: Evaluation of Humanitarian Action Terms of Reference 1. Introduction 1.1. Background 2011 witnessed the start of nationwide protests in Syria against the government which eventually escalated into a full blown civil war that is now in its third year. According to OCHA estimates, the conflict has left 4.25 million people internally displaced and 2.2 displaced to neighbouring countries where they have refugee status (Dec 2013). Furthermore, the Syrian economy has collapsed resulting in a break in access to basic services including water, electricity, health etc. Based on OCHA’s possible scenarios of the conflict, continued fragmentation of political powers and military is expected in the next six months which will further reduce access to basic services for Syrians. OCHA also estimates that humanitarian support will be required even after the conflict ends. A Joint Rapid Assessment conducted in Northern Syria (J-RANS) earlier this year revealed food security to be the number one priority area in the region (Joint Rapid Assessment of Northern Syria, 2013). The recent Syria Integrated Needs Assessment (SINA) estimates a total of between 5.5m and 7.5 people in need in the 8 Governorates of northern Syria alone. 1.2. GOAL’s Programme In response GOAL launched a humanitarian programme in Northern Syria from October 2012 with a sub-office established in Harem in February. The overall goal of the programme is to provide emergency support to families displaced or affected within Syria by the conflict, including through distribution of food, non-food items and vouchers. This evaluation is intended for current USAID funded projects: Emergency support to internally displaced persons in Idleb Governorate, Northern Syria that has been funded by USAID through FFP (Food for Peace) and OFDA (Office of US Foreign Disaster Assistance). Through FFP funding, upwards of 22,300 households will receive Family Food Rations (FFR) on a recurring basis. Additionally, some of these households will also receive bread through a bakery programme where flour has been subsidized for bakeries. Through OFDA, NFI and hygiene kits will be distributed to 24,000 households (through both direct distributions and vouchers) with a focus on winterization items. The main goal of this project is to protect and prevent deterioration in the health and welfare of highly vulnerable, conflict affected IDP and host communities 2. Objectives The main purpose of this evaluation to assess the implementation of the USAID funded projects. This evaluation should build on the baseline, monitoring data and lessons learnt through the project cycle with a specific focus on the following: a) To assess programme appropriateness and effectiveness and capture lessons learned in relation to modality of assistance, cross-sector synergies and integration, outcomes, community perception, preference and participation. b) To review GOAL Syria’s operational systems and structure, assess their appropriateness for the context and capture lessons learned related to the country programme set-up and delivery. c) Formulate specific and realistic recommendations for: ? Actions that can be implemented immediately to improve the impact of GOAL’s activities on the affected population and the systems used to deliver assistance. ? Longer-term programming which is best-suited to needs, context and organisational capacity which can be incorporated in the 2014 strategy 3. Stakeholder Involvement The Commissioning Manager for this evaluation is the M&E Coordinator in the country programme who will liaise with key members of the team (in Turkey and Syria) to ensure that the necessary stakeholders participate in the research, reporting and dissemination activities. 4. Evaluation Criteria/Benchmarks and Questions 4.1. OECD-DAC Criteria The evaluation should use the OECD-DAC criteria as a framework for achieving the objectives: ? Relevance/Appropriateness ? Effectiveness ? Coverage ? Efficiency ? Connectedness and Sustainability ? Coordination 4.2. Evaluation Questions Relevance/Appropriateness ? Is the project based on a sound understanding of the crisis? ? How well is the project analysing and responding to the humanitarian needs presented by the conflict? ? Is the focus of the response still relevant as the conflict evolves? ? How appropriate are the programme interventions to the context? ? How well does the project respond to the specific needs of men, women, boys and girls, the elderly and people living with disabilities? ? How well does the project address protection and reflect the principle of ‘do no harm’? ? To what extent has the integration of programmes been achieved and what outcome has this had on target communities? ? How well suited are the different modalities of implementation (vouchers vs direct) in our operational context and how do the outcomes of these modalities vary on our targeted population? How well suited are vouchers to local markets? ? How well is GOAL communicating with affected communities? Is information shared with beneficiaries systematically? ? How well is GOAL accepted by key stakeholders? ? How well do staff understand their obligations under humanitarian codes (i.e. Red Cross/Red Crescent Code of Conduct and GOAL Code of Conduct)? Effectiveness ? How well has GOAL responded to the internal and external factors which impact on the speed of our response at country level? ? How effective has the team been at remote management of the response on ground inside Syria? ? How effectively is monitoring data being used to inform decision making at all levels? ? How well has the response worked with partners in order to increase the effectiveness of activities and build local capacity? ? To what extent has our work with partners increased the effectiveness of the response? Connectedness and sustainability ? How appropriate were decisions made regarding selection of geographical locations and respective operational needs/resource allocation? ? What are the wider, indirect outcomes on local markets (especially of vouchers) in terms of income generation and availability? ? How has the project worked with local partners to increase their capacity in a sustainable way? ? To what extent has contingency planning taken changes in the security environment into consideration in terms of sustainability of the programme? Coverage ? How well are we making decisions regarding targeting in order to reach the most vulnerable, in the most affected locations? How effectively are we identifying and reaching those groups? ? What is GOAL’s coverage as compared to need in the area? ? What are the factors (if any) which might limit our ability to reach our targets in the future? ? Is GOAL effective in coordination between different actors in reducing the risk of duplication? Coordination ? What have been the key challenges (timeliness and coordination) in the remote management structure in place? ? How well is GOAL engaging and maintaining relationships with key stakeholders (at field level/national level)? ? How well is GOAL engaging in the humanitarian coordination structure/architecture (e.g. sector working groups)? 5. Methodology A recommended methodology is outlined below, but the final methodology and tools to be used is to be determined by the evaluation team. The evaluation team should consist of one international Team Leader and one Syrian Team Member. Access to Syria is likely to be impossible for the international consultant but feasible for the Syrian consultant. 5.1. Planning Before arriving in country the evaluation team will do the following: ? Review key documents ? Prepare a list of key informants/respondents required for interviews and share with the M&E Coordinator ? Draft standard interview questions, focus group discussion points, inclusive methodologies, and other data collection tools as required. They will also prepare data entry formats if required. On arrival in country the evaluation team will: ? Hold a short planning meeting with all members of the evaluation team including the commissioning manager, to review and, as needed, amend the questions, methods, any data collection tools, the stakeholders to be interviewed, logistic arrangements and the daily schedule for each member of the team. ? Hold a short meeting/workshop with the SMT to ensure that all are clear on objectives and outcomes. This will include a short description of the evaluation questions and methods they are using, including the schedule and stakeholders they will be meeting with. 5.2. Data Collection Data collection will take place in a minimum of 5 sub-districts inside Syria (if the security situation permits) and in Turkey. If the security situation does not allow access to Syria, staff and stakeholders can be interviewed over skype/phone and/or in Turkey. A fair representation of following stakeholders must be consulted: ? Cross-section of Beneficiaries (including men, women, the elderly, disabled and children when possible children, when possible and in the presence of parents / guardians) ? Indirect beneficiaries (shopkeepers, contractors etc.) ? Non-beneficiary community members (as possible) ? GOAL staff at country level and at field level ? Partner staff ? Donors, UN agencies and other INGOs (including sector working group leads) ? Local Council and Relief Committee representatives Data can be collected through a selection of the following methodologies: ? Desk review of relevant existing literature (situation reports from other agencies, previous assessments conducted by GOAL etc.) ? Desk review of existing assessment, baseline and endline findings, monitoring data (including post-distribution monitoring, price monitoring, distribution monitoring, satisfaction surveys) and beneficiary complaints/feedback ? Semi-structured interviews ? Focus group discussions (using participatory methodologies) ? Observation and informal discussion ? Participatory reflection workshops (e.g. with GOAL staff at country and field level) Consultants are requested to include a timeframe (which GOAL will then input with regard to travel days) within their proposal document in the following format: Activity Number of days Person responsible Deliverables e.g. Focus Group Discussion 2 days Lead consultant Focus Group Discussion analysis report 6. Ethical Considerations Evaluator will make clear to all participating stakeholders that they are under no obligation to participate in the evaluation study. All participants will be assured that there will be no negative consequences if they choose not to participate. Evaluator/s will obtain informed consent from the participants. Evaluation team will have to receive prior permission for taking and use of visual still/ moving images for specific purposes, i.e., ‘for evaluation report and presentations. Evaluator/s will assure the participants’ anonymity and confidentiality and will ensure the visual data is protected and used for agreed purpose only. 7. Presentation and documentation of findings and recommendations The findings of the evaluation must be shared with GOAL in the following formats: ? Closing workshop in Antakya and Harem with GOAL staff to present findings and get feedback ? For agreed immediate action points: formulation of action plan with deadlines and responsibilities ? Agreed recommendations that will inform GOAL strategy ? Draft Evaluation Report submitted to M&E Coordinator for feedback and comments ? Final Evaluation Report- The report must be clear and concise and the following sections must be included: Executive Summary, Methodology, Analysis of findings, Recommendations; Annexes: TORs, a timeline of the response, a list of individuals interviewed, a bibliography, a description of methods employed, a summary of survey results (if appropriate) and any other relevant materials. ? A short learning paper / external report to be shared externally Deliverable 1: Presentation of findings Deliverable 2: Action Plan with deadlines Deliverable 3: List of agreed recommendations Deliverable 4: Final Evaluation Report Deliverable 5: Short paper for external sharing 8. Dissemination of Findings Results and recommendations will be made available externally to all interested stakeholders (including beneficiary communities), both in hard copy and electronically 9. Assumptions and Requirements ? Evaluators will have access to all documentation and can take part in relevant meetings and field trips. ? Evaluators will have access to key staff in the responding GOAL offices in Syria and Turkey and partner offices for conducting interviews. ? The evaluation team will have access to members of the affected population for conducting interviews. ? Evaluators will take confidentiality and objectivity into consideration during the process. ? Security concerns could impact the timing and the scope of the evaluation. It is important for the team to remain flexible. They must be open to making changes to the schedule and itinerary such as visiting alternate sites, conducting remote reviews and interviews, etc. ? GOAL will provide all transport within Turkey and Syria 10. Consultant Profile For the purpose of this evaluation, 2 consultants are required that will work together: 1. Local Consultant (will conduct trips to GOAL’s office and programmes inside Syria) ? Master’s degree either in development/humanitarian studies, social sciences or economics ? At least 10 years of experience in development programming ideally in conflict situations including food security, livelihoods, nutrition, NFI and voucher programming ? Experience of conducting evaluations (USAID desirable) that includes an in-depth knowledge of different quantitative and qualitative research tools and methods ? Capacity to work collaboratively with multiple stakeholders ? Fluency in Arabic and English is essential 2. International Consultant (will work through Antakya Office) ? Master’s degree either in development/humanitarian studies, social sciences or economics ? Experience of working in humanitarian contexts and good understanding of humanitarian response work – both in programmes and operations ? At least 10 years of experience in humanitarian programming ideally in conflict situations including food security, livelihoods, nutrition, NFI and voucher programming ? Experience of conducting evaluations (USAID desirable) ideally leading an evaluation team and experience of designing evaluation methodology / tools, data analysis etc. ? In-depth knowledge of quantitative and qualitative research methods ? Good presentation and writing skills ? Capacity to work collaboratively with multiple stakeholders ? Excellent analytical and writing in English and Arabic (preferred) 11. Proposal details and submission The deadline for submission of the technical and financial proposal and accompanying documents is 18-04-2014. All applications should be submitted by email to syriajobs@sy.goal.ie. The application should include: • Technical proposal including detailed methodology and proposed schedule, your relevant experience, how you meet the profile required and details of time required • Up to three relevant examples of past assessments completed • CVs of key personnel • Detailed, itemised cost proposal, including daily fee and any other associated costs (GOAL will provide accommodation) • Details of referees Proposals will be assessed against the following selection criteria: • Correct relevant documents submitted with application • Relevant skills and previous experience • Appropriateness of proposal methodology and timeframe, to the tasks specified in the ToR • Availability • Value for money