Outcome Evaluation: Sustainability Differentials and Sustainability Investments (SD/SI)
External valuation of SD/SI (2022-2025)
The contribution of “Sustainability Differentials” (SD) and “Sustainability Investments” (SI) to sustainable production and shared responsibility in RA-certified value chains (2022-2025)
- Deadline for written questions about the RfP: 14 of July, to firstname.lastname@example.org
- Deadline for proposals: 14 of August 2022. Please upload your proposal to this webpage under “cv/resume” (scroll down) in a single file.
- Budget: undisclosed.
SD/SI in the 2020 Rainforest Alliance Sustainable Agriculture Standard
The Rainforest Alliance recognizes that sustainable production, trade, and consumption of agricultural commodities is a long-term journey and that all supply chain actors share a responsibility to make this happen. An inclusive supply chain is one where value and risks are shared between producers and buyers. This requires a shift to a system where the costs of sustainable production are shared equitably between producers and buyers, more so than is currently the case.
The 2020 Sustainable Agriculture Standard (RASAS) introduced two requirements for buyers of Rainforest Alliance Certified agricultural products: the payment of a “Sustainability Differential” and of “Sustainability Investments” to producers, on top of the market price.
The Sustainability Differential is a mandatory payment to producers on top of the market price for certified crops (previously called “premium”). The amount of SD and the way it is established or negotiated differs per sector (see annex 1).
Sustainability Investments (monetary or in-kind) are additional contributions of buyers to enable producers to make investments to implement the RA standard. Producers have to elaborate an Investments Plan to solicit and allocate SI payments.
Theory of Change
The theory of change behind SD/SI is that this mechanism and the ensuing payments will incentivize producers to adopt sustainable practices codified by the RA Sustainable Agriculture Standard. These payments should be additional to the market price, so as to enable certified producers to cover the costs associated with certification / sustainable production. These include the costs to comply with the standard, the costs of self-selected improvements (e.g. paying living wages). Management systems to implement certification and third-party audits also add to the costs of certification .
Previous evaluations by RA and third-party studies have signaled that the costs of implementing voluntary sustainability standards are mostly shouldered by producers . Although the adoption of standards has also been associated with efficiency gains and cost reductions, many producers say that the economic benefits of certification are outweighed by the costs. Meanwhile, at sector level the growing volumes and proportion of certified production have not fundamentally changed market dynamics, the purchasing practices of buyers or the power imbalances in value chains. Producers of agricultural export commodities remain price takers. This market imbalance also places producers at the “receiving end’ of sustainability standards. Producers who wish to gain or keep access to (certified) export markets must accept sustainability standards in demand in destination markets, bearing the commercial risk that not all certified supply may be sold as certified, in which case they forego premium payments. Whereas the leverage that buyers hold over their suppliers on one hand drives the adoption of sustainability standards, at the same time it seems to be a limiting factor when it comes to internalizing the cost of sustainable production in the market price of certified products. SD/SI payments are RA’s way to partially address this market failure.
The theory of change behind SD/SI assumes that a market-driven approach will generate a race to the top, with increasing amounts of SD/SI being channeled to producers, enabling them to make further sustainability improvements on their farms. The logic is that where certified products are in demand and enable buyers (brands, manufacturers, retailers) to add value and differentiate their products (or to manage certain risks), buyers will compete and make efforts to secure certified supply. In theory this should create virtuous cycles of higher SD/SI payments leading to more sustainability improvements.
The evidence base on voluntary standards shows that such virtuous cycles and a “race to the top” may occur under certain conditions, but that this is not a necessary outcome. In early stages of sector transformation , early adopters of standards have a window to differentiate certified products on the market before sustainable standards become mainstream. As certified products become mainstream, premiums tend to erode .
Figure 1: Theory of Change of SD/SI in the RA 2020 Sustainable Agriculture Standard (in blue font: critical assumptions)
RA expects that the introduction of SD/SI will trigger a new wave of sustainability payments of buyers to reward producers for implementing the RA standard, which is more demanding than the preceding RA and UTZ standards. SD/SI has been designed as mechanism to enable buyers to meet their sustainability commitments, make additional financial contributions to sustainable producers and be recognized for these contributions (claims). The difference between SD and SI is that whereas both are mandatory and negotiable, SD is a premium above market price, to be entirely transferred to producers, whereas SI payments requires that producers formulate a “sustainability investment plan” that targets specific categories of improvements. Earmarking enables buyers to link their contributions to specific categories of improvements, and to claim these contributions on the marketplace.
An important element of SD/SI is that the certification program provides transparency to stakeholders as to the origin, amount and use of SD/SI payments. The standard has specific requirements, tooling, instructions and guidance to steer producers and buyers in planning, budgeting, prioritizing, monitoring, paying, reporting and eventually claiming the payments delivered and received.
The implementation of SD/SI requirements takes place at a different pace in different sectors (see annex).
Rationale and purpose of the evaluation
RA is commissioning this evaluation of its SD/SI policies because:
a) SD/SI is a pivotal element of the RA Standard to operationalize the principle of “shared responsibility”
b) RA needs to know to what extent SD/SI is an effective policy instrument to mobilize and channel additional payments / investments from buyers to certified producers in different sectors, in orders of magnitude that are commensurate with the costs of sustainable production.
c) RA wants to be accountable to all stakeholders in terms of delivering an effective program that makes a difference to producers, workers, and the environment.
In accordance with the ISEAL Code of Good Practice for Assessing the Impacts of Social and Environmental Systems , this evaluation report will be published, and its findings used to inform and improve the certification program.
1) How are SD/SI requirements being implemented by different stakeholders?
2) What are key obstacles and enablers?
3) What amounts of SD/SI have been paid to producers? Can the amounts be confirmed from different sources? Are there significant discrepancies between the reported and received payments?
4) How are SD/SI amounts decided upon / negotiated? Which factors influence these decisions?
5) What portion of SD/SI is transferred in cash to group members and/or workers?
6) Do workers have a say on SD/SI, as prescribed by the standard?
7) To which categories of sustainability costs / investments is SI being allocated?
8) Are external audits sufficiently verifying SD/SI payments and investments?
As RA monitors the implementation of the SD/SI requirements, the data to answer implementation questions is collected by RA and will be shared with the evaluators. The evaluators can assess, validate and interpret these data with a view of answering the questions on outcomes and impact.
Outcomes and impact
- Who benefits from SD/SI, in what way(s)? How substantive are these benefits?
- To which sustainability outcomes does SD/SI contribute?
- What are the transaction costs of SD/SI? How cost-effective is the system?
- Are there any unintended or unexpected (positive or negative) effects of SD/SI?
- What external factors have influenced implementation and outcomes?
- Is SD/SI associated with changes in buying practices and terms of trade?
- Has SD/SI contributed to shared responsibility and more equitable value chains?
- How valid are the assumptions and the causal logic of the theory of change?
- Did SD/SI contribute to a “race to the top” in terms sustainability investments?
Design and Methodology
- This evaluation will be carried out in phases over a 4-year period (2022-2025), taking into account different implementation timetables of SD/SI for the coffee, tea, cocoa, and banana sectors. RA wants to learn from the implementation in sectors where SD/SI has started to be implemented (coffee, cocoa), to use insights and finetune the approach in sectors where SD/Si enters into effect in 2023 (banana and tea).
- RA intends to use the (emerging) insights of this evaluation to improve the program. The certification program undergoes periodic revisions, based on monitoring data and feedback from members. The findings from this evaluation will also be taken into account. The evaluation design should reflect and enable this learning approach.
- Proposals should include a) an overall evaluation design for the 4-year period b) a detailed proposal for year 1 (baseline, early implementation). We aim to establish a 5-year MoU followed up by yearly contracts.
Year 1 (2022 Q3+Q4) Inception and baseline
- Design, conceptualization, evaluation framework, desk study, operationalization of evaluation questions, indicators, data points, risk analysis and validity threats, outline of yearly reports
- Develop indicators for evaluating changes in “shared responsibility”, and “equitability in value chains”.
- Inception report / Baseline report
Years 2, 3 and 4 (2023-2025) monitoring reports
- To answer the evaluation questions in year 5 the evaluators will use the longitudinal dataset built on the basis of monitoring data collected mainly through RA’s certification, traceability and member platforms. Available data includes transactions of certified products and agreed SD/SI amounts. Certification data from audits should provide insights in the type of sustainability investments to which SD/SI payments are being allocated. Audits will verify the transparency of SD/SI payments in terms of amounts received / transferred. RA’s monitoring data may initially be somewhat fuzzy and contain ‘noise’ due to data quality, but we expect data quality will become more consistent and reliable from year 2.
- The evaluation team will use RA’s monitoring data to analyze how SD/SI is being implemented by certificate holders, identifying trends, patterns and challenges. This SD/SI monitoring data is mainly self-reported and may need to be triangulated and validated using other sources, on the basis of a methodology and protocol to be developed by the evaluators in the inception phase.
- The evaluation team may want to collect additional data from a sample of producers and buyers to a) validate key monitoring indicators, b) monitor how “shared responsibility” and “equitability in value chains” are evolving, using indicators developed for this evaluation, c) gain insights about influencing factors and unexpected outcomes.
- Deliverables: 4 monitoring reports over 2023, 2024, 2025 (for RA’s internal use) including recommendations to RA.
Year 5 (2026) Evaluation report
- Based on the data and analysis of 4 years of implementation, a final evaluation report will be presented in 2026. This final report will be published.
Qualifications of the evaluation organization / team
- Research Institute or consultancy organization. (Individual researchers/consultants or ad hoc teams are not eligible).
- Strong record track with evaluation design and research
- Affinity with a learning evaluation approach
- Expertise in business economics, value chains, accounting, sustainability standards
- Experience with consulting / researching in the coffee, tea, cocoa, and/or banana value chains
Evaluation governance / RACI
- The relation with the Research/consulting firm will be managed by a senior member of RA’s DMERL team (Design, Monitoring, Evaluation and Research and Learning), under supervision of a steering committee integrated by RA’s Markets, Themes and DMERL teams. The final responsibility for this evaluation lies with the director of the DMERL department (Design, Monitoring, Evaluation, Research and Learning).
RA is commissioning this work as a competitive bid.
Proposals will be assessed and scored independently by 3 RA staff looking at:
- Qualifications / expertise of the organization and team
- Proposed approach, design and plan
- Budget / value for money
Please provide a detailed budget (excluding VAT) for year 1 and an estimated figure for the each of the subsequent years. Budgets should include at least the # of consultancy days per task / component of the plan for year 1, and the daily fee of each member of the evaluation team.
Timeline and procedure
- RfP published: 1 of July, 2022
- Optional: Submission of questions about the RfP (in writing only to email@example.com): deadline July 14, 2022). The Q&A will be posted on July 18th, 2022.
- Deadline for proposals: 14 of August, 2022
- Interviews: week from 22-26 of August
- Contracting: 9 of September, 2022
- Inception report: 16 December
- Baseline report: 27 Feb 2023
Annex: overview of SD/SI provisions per sector
 The costs of certification for producers broadly speaking consist of the costs to reach compliance (making necessary improvements, implementing internal management systems) and the costs of external audits. The costs of certification for buyers broadly speaking consist of: a) the support and (additional) services they provide to producers to reach certification (the level of support differs per buyer, and may also include paying for external audits of producers, operating internal management systems on behalf of producer groups, etc); b) SD/SI payments to producers; c) membership fees to the RA program/organization d) audit costs for chain of custody certification if applicable.
 Oya et al 2018, The effectiveness of agricultural certification in developing countries: a systematic review, in World Development Volume 112, December 2018, Pages 282-312.
 Ponte, Stefano 2019, Business, Power and Sustainability in a World of Global Value Chains, Zed Books: London and New York.
 LeBaron, G. 2020, Combatting Modern Slavery: Why Labour Governance is Failing And What We Can Do About It, Cambridge, Polity Press.
 IIED, AidEnvironment and New Foresight, 2015, Sustainable Sector Transformation: How to drive sustainability performance in smallholder-dominated agricultural sectors?
 Grabs, J. 2020, Selling Sustainability Short? The private governance of labor and the environment in the coffee sector, Cambridge University Press.