External Evaluations of Our Work
TechnoServe has a long-standing commitment to pursuing rigorous results measurement.
Whenever possible, we leverage external evaluators to conduct independent assessments and evaluate our implementation model using experimental design methods. We’ve highlighted some of our most rigorous results here.
Our review of the external and internal evidence for TechnoServe’s impact highlights several key insights:
- Market-oriented interventions that combine capacity building and linkages to resources and markets can have a positive impact.
- Shifting farmers from subsistence agriculture to a market-oriented focus, particularly on globally traded export crops, can be transformative for their livelihoods and welfare.
- Quality of delivery is a key feature of an impactful project, yet impact evaluations using rigorous research methods are often unable to disarticulate quality from other factors affecting project impact.
We’ve incorporated these insights into project design, and continue to invest in research that will help us hone our approach to creating more inclusive market systems.
Auditing Our Impact
In 2017, TechnoServe engaged ImpactMatters, an impact audit firm founded with support from Innovations for Poverty Action (IPA), to review several projects within our portfolio. TechnoServe is the first large organization to undergo an impact audit with ImpactMatters, which assesses the quality and cost effectiveness of an organization’s impact. We believe that value for money and the sustainability of development interventions are crucial factors to consider when assessing the success of development efforts.
The impact audit for four of our projects – Coffee Initiative II, Impulsa Tu Empresa, Mejoramiento Agrícola Sostenible, and Women Mean Business – showed strong results, finding participants earned up to $34 in net revenue for every dollar TechnoServe spent on projects. Most projects also received a good rating for quality of evidence.
ImpactMatters’ audit, published in March 2018, confirms the powerful impact that TechnoServe projects can have and demonstrate the importance of a rigorous, transparent accounting of impact. Read more about the audit on our blog.
Randomized Controlled Trials and Quasi-Experimental Impact Evaluations
Randomized controlled trials, or RCTs, are often referred to as the gold standard in impact evaluation. In an RCT, evaluators randomly assign a set of community members to receive project assistance (known as the treatment group) or to receive no additional assistance (known as the control group). The evaluators then capture and statistically analyze data across the treatment and control groups to quantify the project’s impact. In projects that use quasi-experimental design, evaluators use different techniques to modify random selection into the program when complete randomization is unfeasible.
We enthusiastically support any opportunity to evaluate our projects to this degree of rigor, while acknowledging the difficulty of running RCTs in development interventions. As we continue to seek opportunities to implement RCTs and quasi-experimental impact evaluations, we also encourage projects to implement creative and resourceful ways of measuring impact where measurement resources are limited. View this video to learn more about RCTs.
- Women Mean Business (2017, IPA) – Training and mentorship programs aimed to help SMEs develop business expansion strategies in Uganda helped women-led businesses increase monthly revenues by 22 percent and take home income from business by 17 percent. Increased share of take-home earnings indicate greater financial security for households of businesswomen. Results forthcoming.
- TechnoServe Coalition for Smallholder Sourcing (2016, IDInsight) – The pilot program which was launched to assist farmers adopt improved agricultural technology had some very positive impacts. Assistance to integrate farmers into contract farming increased the proportion of farmers planting reliable hybrid maize by 40 percent.
- Coffee Initiative II (2013, Laterite) – Participation in the agronomy training program resulted in increase in yields of up to 75.5 percent for farmers in 2011. Estimates at the cooperative level also show positive changes, with one cooperative experiencing an increase in yield as high as 148.1 percent after one year of training.
- Coffee Initiative II (2012, JPAL/IPA) – Trainings in good agronomic practices resulted in farmers being more knowledgeable about best practices. Trained farmers were significantly more likely to have adopted good agronomic practices on their coffee plantations.
- Andean Business Plan Competition (2007, Harvard) – Entrepreneurs that receive training are more likely to establish a business or grow their existing business. Trained entrepreneurs were more likely to open a new business (9-11 percentage point increase) or expand their business (23- 26 percentage point increase) as a result of the program.
Other Evaluative Work
We are also at the forefront of emerging evaluation topics including cost-effectiveness and the sustainability of impact.
- Coffee Initiative II (2016, TripleLine) – Farmers continue to apply improved agronomic practices five years after the end of TechnoServe’s intervention. (Read a summary of the findings here.)
- Mejoramiento Agrícola Sostenible (MAS) (est. December 2017, ANED) – Measures the impact of a coffee market intervention on smallholder farmers’ revenues.
- Uganda Coffee Project (est. 2020, IFPRI) – Measures the effect of training on smallholder coffee farmer yields.
- Women’s Advancement in Rural Development & Agriculture, Bihar (est. 2020, 3ie) – Measures the effect of business training on the incomes of smallholder women farmers.
- MAS 2.0 (est. 2022, IFPRI) – Measures the impact of a coffee market intervention on smallholder farmers’ revenues.
- Business Women Connect (2018, World Bank) – Measures the effect of business training and increased access to a mobile savings platform on Tanzanian women microentrepreneurs’ ability to save. Read about preliminary findings from this research by Center for Global Development.
- STRYDE (est. 2019, 3ie) – Measures the effect of employability training on Tanzanian youth’s incomes.
- Impulsa Tu Empresa (est. 2022, Miami University) – Measures the effect of business training and advisory services on SMEs’ revenues.
RCTs are not always feasible or, given their high cost, the most appropriate evaluation design choice. When we are unable to undertake an externally evaluated impact evaluation, we use the most rigorous method available to the project, for instance, by using quasi-experimental methods. The results we report in our annual impact report undergoes similar assessments to determine the impact of our work as rigorously as possible.
External research supports TechnoServe’s theory of change
There is increasing published evidence that supports our theory of change. For instance, Stewart et al. find evidence that participatory approaches to farmer training, coupled with other market interventions, can substantially increase incomes. Piza et al. find that business skills development improves the performance of small and growing businesses, while acknowledging that the policy environment and training quality strongly affect outcomes. We are encouraged by these findings but also believe there should be greater investment made in researching the evidence for impact of development efforts. We are confident that more research would bolster our own findings.