Draft Terms of Reference for post project Evaluation of ECHO supported ‘Provision of Effective and Integrated Nutrition Care for Children Under Five Years’ project in Aweil North and Aweil West Counties in Northern Bahr el Ghazal, South Sudan Introduction: Concern Worldwide is a non-governmental, international, humanitarian organisation dedicated to the reduction of suffering and working towards the ultimate elimination of extreme poverty in the world’s poorest countries. Concern works in partnership with the very poorest people in these countries directly enabling them to improve their lives, as well as using Concern Worldwide’s knowledge and experience to influence decisions made at local, national and international level that can significantly reduce extreme poverty. Concern implements both emergency responses and longer term development interventions in 25 countries in Asia, Africa and the Caribbean. Globally, Concern Worldwide is considered one of the lead actors in the area of nutrition and nutritional responses, having pioneered the Community Therapeutic Care (CTC) and CMAM approach to nutrition programming and it currently implements nutrition programmes in 13 of its 25 country programmes and is supported by a team of nutrition advisors based out of its head office in Dublin and its US office in New York. Concern Worldwide is part of the Global as well as National Nutrition Clusters and an active member of the Capacity Development Working Group of the Global Nutrition Cluster. Concern is already engaged in the nutrition sector in NBeG in collaboration with the nutrition cluster at country and state levels. Background: Since 1998, Concern Worldwide has been working in South Sudan responding to emergency and rehabilitation needs of the conflict affected communities following the linking relief, rehabilitation and development (LRRD) continuum. Concern Worldwide currently operates in two counties of Northern Bahr el Ghazal (NBeG) state i.e. Aweil West and Aweil North with the Country office based in Juba providing the programme support and representation function. The focus of the current interventions has been on addressing the emergency and transitional food, non-food, nutritional and health needs of the returnees, IDPs and host communities. The Health and Nutrition Programme is currently implementing a basic package of health and nutrition services in Aweil West through supporting 22 health facilities and an emergency nutrition response in Aweil North through supporting 7 health facilities for treatment of malnutrition. In Aweil North an additional 9 health facilities receive nutritional support only. In both counties this is complemented by a community health and nutrition education component focusing on improved health and nutrition practices in the communities. The Food Security & Livelihoods programme is being implemented in Aweil West focusing on improving the food security situation for vulnerable food insecure households and returnees and capacity building of local government authorities to provide agricultural services. Both the programmes are implemented in the same geographies to improve synergy and outcomes. NBeG is the poorest state in South Sudan having being on the proximity to the civil conflict with the effect of chronic underdevelopment with infrastructure, human resources and household and community assets in a severely dilapidated state with 75.6% living below the poverty line. It is the most food insecure state with 62% moderately food insecure and 9% severely food insecure . This is attributed to most of the population lacking sufficient assets to engage in food production and have poor agronomic practices. This is reflected by some of the worst health indicators in South Sudan with infant mortality and under five mortality rates of 130/1,000 live births and 164/1.000 live births respectively along with a maternal mortality rate of 2,182/100,000 live births . With the border between Sudan and South Sudan un-demarcated, there are tensions with Sudan over the disputed border areas with bombardments leading to displacement in April-May 2012 and November-December 2012. NBeG also has the highest number of returnees with 456,860 returning till now and in 2012 the state saw the fourth highest arrivals with 13,907 returnees . While the rates of malnutrition for returnees screened upon arrival are initially low, as they have little means to re-establish themselves, engage in food production or income generation activities, the rates of malnutrition quickly increase. These factors have contributed to seasonal cycles of chronically high malnutrition rates. In 2011 a Pre Harvest Nutrition Survey indicated Global Malnutrition (GAM) rates of 24.6% and Severe Acute Malnutrition Rates (SAM) of 3.8% in Aweil West and Aweil North had GAM and SAM rates of 24.3% and 6.3% respectively. In 2012 the Pre Harvest Nutrition Survey indicated a GAM rate of 25.8% and a SAM rate of 4.1% in Aweil West. Due to insecurity no survey could be conducted in Aweil North. These high rates of malnutrition are also attributed to poor health and nutrition practices among the population including poor Infant and Young Child Feeding (IYCF) practices with low numbers of infants exclusively breastfed, insufficient meals provided to children and poor dietary diversity. In addition, the County Health Departments (CHD) has limited human, financial or material resources to respond. Further, the supply chain rupture and low beneficiary coverage (from WFP and UNICEF) also adds to the problem both in terms of failing to respond to the needs as well as reducing participants’ confidence in the programme. Concern Worldwide has undertaken two surveys i.e. Knowledge, Practices and Coverage and Health Facility Assessment (attached) in 2012 to establish reliable baseline for both nutrition and health interventions in Aweil West and North Counties. These two reports will provide valuable insights and information about the status as well as in designing future interventions. With malnutrition rates continually way above emergency thresholds, Concern Worldwide intended to scale up the nutrition response in 2012 in Aweil West and Aweil North Counties. This was supported by Irish Aid, UN Common Humanitarian Fund, ECHO, Concern’s own funds, while the BPHS delivery in Aweil West was supported by the DfID funded Basic Services Fund. Concern Worldwide also received nutrition food commodities as in-kind donations from UNICEF and WFP. The nutrition project aimed to achieve the following: Specific Objective: Nutritional status of malnourished children under five years of age and PLW are managed according to Ministry of Health Guidelines and Sphere Standards Result 1: 10,855 children under the age of 5 years and 2,500 PLW with SAM and MAM managed, through 34 SCs, OTP and SFP sites in Aweil North and Aweil West. Result 2: 129,511 individuals will benefit from community based preventive nutrition intervention. The response focused on the treatment of children under 5 with SAM through Outpatient Therapeutic Programme (OTP) in 34 health facilities and those with complications were treated at two Stabilisation Centres (SC) and children under 5 and pregnant and lactating women (PLW) with MAM through Targeted Supplementary Feeding Programme at 34 health facilities. To reduce the risk of malnutrition the community component of the response involved community sensitisation to improved nutrition practices and conducting IYCF awareness sessions. The key implementation approach during the response was through Community based Management of Acute Malnutrition (CMAM). CMAM focused on community mobilisation, community screening and community referral to supplementary, and outpatient therapeutic feeding programmes as well as support to the SCs for acute cases with medical complications. This latest phase of the nutrition programme was designed to integrate the management of acute malnutrition in these areas with the provision of basic medical services through health clinics. This is in line with the South Sudan Ministry of Health Guidelines on the Integrated Management of Acute Malnutrition (IMAM). Concern Worldwide would like to undertake an external evaluation of the project with the aim of assessing the performance of nutrition interventions up to March 2012 and more importantly, to guide future direction of the programme and transitional nutrition responses from emergency to more preventive initiatives, taking into account the chronic factors influencing high malnutrition rates in NBeG. Purpose of the Evaluation The purpose of the end of project evaluation is to assess the extent to which the Nutrition project’s principal and specific objectives and results have been achieved as set forth in the project proposal using all programme indicators. In addition the evaluation should assess the extent to which the nutrition services were integrated in the general health service provision system at point of service delivery. The evaluation will also offer strategic and operational recommendations to improve the direction, effectiveness and relevance of similar programmes in future. Specific objectives Specifically the evaluation should focus on the following four areas: 1. Determine to which extent the project’s specific objective have been met: • Nutritional status of malnourished children under five years of age and PLW is managed according to Ministry of Health Guidelines and Sphere Standards • Nutritional status of malnourished children under five years of age and PLW is improved. 2. Verify the extent to which the results outlined in the project proposal have been achieved as a result of the activities funded by the project. Specifically • Improved provision of CMAM services to treat and rehabilitate children and PLWs with acute malnutrition through 2 SCs, 38 OTP sites and 34 TSFP centres in Aweil West and North counties. Table 1: CMAM Program Sites by County Aweil West Aweil North Total Old OTP/SFP sites 18 16 34 New OTP sites 4 0 4 Total 22 16 38 Total Population per County 125409 155715 281124 • Improved IYCF practices in Aweil North and West Counties 3. Assess the project performance in terms of the following: • Relevance – did the interventions meet the conditions and situation of the target beneficiaries? Did the conditions and situation change during the project period? Were project activities modified to maintain or improve their relevance? • Appropriateness – were the interventions provided in the most appropriate manner given the local current social, environmental and cultural context? • Timeliness – were the results achieved within the timeline set out in the proposal? Was the timeline to achieve the stated results over or under ambitious? • Effectiveness – how effective was the project in achieving the stated project objectives and results? If project activities were modified to maintain or improve their relevance to meet changing conditions and situations of the beneficiaries, were these effective? Are there more effective ways and methods that could be used to reach these objectives and results? What would these recommendations be taking into account the context and levels of funding? Is the level of the staff hired sufficient for the program and activities proposed? Are capacity building and trainings level proposed to them sufficient and adapted? • Efficiency – how cost effective was the project in the context? Does it represent good value for money compared with other similar programmes? Is the collaboration with the relevant stakeholders efficient? How to improve it? What would you recommend to increase the efficiency of similar programmes? • Sustainability/LRRD – how sustainable are the outcomes of the project? Has the MoH personnel benefitted from a series of measures to improve their skills? Is the integrated methodology suitable for the geographical area? Was the assistance provided in a way that took into account the evolution of the emergency context? Did an exit strategy exist in any form; was this utilized and adhered to? How successful were the activities to strengthen the capacity of communities and individuals? • Programme quality- were the required national and international protocols, standards and procedures duly and consistently applied during the project implementation? How was this done? If not, why not? • Community participation-was the target community properly involved in the project activities as intended? How was it achieved? If not, why not? • Government involvement- what roles, responsibilities and functions did both local and central governments undertake in facilitating implementation of project activities? How was this done? If not, why not? • Integration- Was Integration in context of SS done in efficient way, how successful was the integration of nutrition services into the service delivery system, If not, why not? How successful was the project in terms of integrating other complementary health and nutrition related activities? What factors made this integration process successful? If not, why not? • Mainstreaming: did the project succeed in mainstreaming key cross-cutting issues including gender, HIV/AIDS, environment and others identified in the project application? How was this achieved and if not, why not? • Monitoring and Evaluation: was M&E plan designed and used as expected? How was this achieved? If not, why not? How is the monitoring process formalized? Is it adapted? Were the distribution and information flow, data collection and analyse satisfactory among the team? How could it be improved? Is the information delivered in time (monitoring data / financial follow up) in order to facilitate decision-making? If not what are the constraints and how could they be released? Was there a mechanism which dealt with complaints / dissatisfaction in relation to the program? • Comment on SWOT of CTC/CMAM integration 4. Document the lessons learned related to the programme and policy, if any and provide practical recommendations for improving the execution and replication of similar interventions in NBeG State or in any other high malnutrition prevalent environments. Key issues that are likely to be included: • What other interventions were being undertaken in the project area at the same time by Concern Worldwide or other stakeholders that improved or undermined the achievement of project results? Was there any steps taken by Concern Worldwide to integrate the project component with the different activities that were being undertaken by other stakeholders? • Are there any external factors that unduly affected the project’s impacts in any way? Could the project have been better designed to address these? How did the project coordinated with other national/international agencies and bodies? • Relevant lessons learned when assessing the project’s performance (including questions from #3 above) Evaluation Process and Methodology Prospective consultants are encouraged to submit their Proposals for the Evaluation and to ensure the evaluation objectives are fully met, however at minimum it is expected the following will be undertaken: • In country review of TOR with Country team • Submission of work plan within three days of signing the contract including the tools to be used in the evaluation for review and feedback. • Review of all supporting documents relating to the project (see box 1 below. • Meet with Concern Worldwide technical and programme staff (both in Juba and field level) to discuss programme implementation, challenges & issues and lessons learned. • Meet Concern Worldwide finance staff to assess budget expenditure as per the proposal. Also including an assessment of cost efficiency drawing from comparable projects elsewhere • Meet DG ECHO field staff to discuss about the outcomes of the project monitoring • If needed, meet with other project implementers in the area to document their interventions and how these might have affected our results. • Develop a checklist questionnaire(s) to how far all planned project activities were completed – documenting the reasons for any under or over achievement • Review the monitoring plans, tools and reports used before and during implementation and assess whether they were sufficiently robust to measure progress and impact. • Review relevant qualitative and quantitative data, as well as the methodology of data collection and utilisation for and by the project staff and partners. Concern Worldwide will provide further support or re-analyse the data for the consultant as required. • Undertake a field visit to the project locations, with a minimum of 40% of supported health/nutrition facilities to be visited, and undertake focus groups discussion with all beneficiary groups i.e. - PLWs and other mothers - Children 15 – 18 years – where possible - Members of Health Management Teams (facility and county level) - County Health and Nutrition staff – including nutrition supervisors, nutrition officers, nurses ,midwives and community health workers • Preparation of a PowerPoint presentation on draft findings for presentation to key staff in Concern Worldwide at the field level. • Presentation of preliminary results at Juba level, first with CWW and then with donors. • Submission of a draft report for comment and review by Concern Worldwide • Final report incorporating feedbacks. • Concern expect a participatory approach including concern staff, beneficiary (male/female), local MoH at all levels and other Stakeholders • Separate meetings will be organize with UN, MOH, Selected community members, and other INGO implementing CTC/CMAM in areas (if any) Box 1: Documents to be provided by CWW for Review ECHO proposal including the logframe IAPF proposal Health and Nutrition M&E framework 5 years Review document Health Facility Assessment Report KPC Report HSU Advisors Visit Report ECHO Visit Report UNCHF Visit Report BSF Visit Report Key deliverables: • Checklist questionnaire(s) to how far all planned project activities were completed – documenting the reasons for any under or over achievement • Focus Group Discussion information including but not limited to questionnaires, specific numbers and types of FGDs conducted, results of FGDs • Draft presentation of all findings; possibly in PowerPoint format • Final report that incorporates all components of this TOR and contractual agreement as well as comments and feedback from Concern South Sudan and Concern HQ staff Format of The report • Executive summary 4-5pages • Summary recommendation • Table of content • Introduction • Context/background • Main findings • Lessons learnt • Conclusion and recommendation • Appendices Accountabilities and Responsibilities: The consultant shall report directly to the Country Director of Concern Worldwide South Sudan and work closely with the Nutrition Coordinator and other technical staff. Concern Worldwide Assistant Country Director Programmes or his designate will represent the agency during the evaluation as overall evaluation focal person. The consultant will be bound by Concern Worldwide’s Programme Participant Protection (P4) Policy which ensures maximum protection of programme participants especially beneficiaries form all forms of abuse and exploitation. Concern Worldwide is responsible for: • Overall responsibility and accountability for the evaluation. • Facilitate travel, accommodation and administrative requirements on the ground • Facilitating communication with relevant actors for the purpose of this evaluation The consultant will be responsible for: • Conducting the evaluation. • Regular progress reporting to Concern’s evaluation focal point • Production of deliverables in accordance with contractual requirements Location and Duration of the Assignment: The assignment begins as soon as the contract is signed and should be completed within 20 days (including report writing) starting in early April 2013. The evaluation should take place in the Aweil West and Aweil North Counties, NBeG state. This may involve travel to areas of moderate security risk. Concern Worldwide takes all feasible precautions to ensure safety of its staff, partners and other stakeholders, and this of course would extend to personnel on a field trip. Parties interested in undertaking this assignment should send a short proposal of no more than one page outlining how they would undertake the assignment and a proposed timeline for the activities listed above. A latest CV and report shall be submitted with the proposal and advise on daily fee. Concern will provide Finalised Itinerary Reference documents to read prior to visit For further information please contact: Mireille Ndikumagenge Desk Officer South Sudan, Organisation ECHO Focal Point Concern Worldwide Email: mireille.ndikumagenge@concern.net Tel: +353.1.417 77 00/417 80 19