Accounting for complexity – Intervention design in the context of studying social accountability for reproductive health

Heather McMullen; Victoria Boydell; Joanna Paula Cordero; Petrus S. Steyn; James Kiarie; Patrick Kinemo; Alice Monyo; Mary Awelana Addah; Jacob Tetteh Ahuno; Osei-Bonsu Gyamfi

doi:10.12688/gatesopenres.13260.1

Home Browse Accounting for complexity – Intervention design in the context of...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Open Letter

Accounting for complexity – Intervention design in the context of studying social accountability for reproductive health

[version 1; peer review: 3 approved with reservations]

Heather McMullen¹, Victoria Boydell², Joanna Paula Cordero³, [...] Petrus S. Steyn ³, James Kiarie³, Patrick Kinemo⁴, Alice Monyo⁴, Mary Awelana Addah⁵, Jacob Tetteh Ahuno⁵, Osei-Bonsu Gyamfi⁵

Heather McMullen¹, Victoria Boydell², [...] Joanna Paula Cordero³, Petrus S. Steyn ³, James Kiarie³, Patrick Kinemo⁴, Alice Monyo⁴, Mary Awelana Addah⁵, Jacob Tetteh Ahuno⁵, Osei-Bonsu Gyamfi⁵

PUBLISHED 22 Jul 2021

Author details Author details

¹ Centre for Global Health, Institute of Population Health Sciences, Queen Mary, University of London, London, E1 4NS, UK
² Global Health Centre, Geneva Institute of International and Development Studies, Geneva, 1211, Switzerland
³ UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP Research), Department of Sexual and Reproductive Health and Research, World Health Organization, Geneva, 1202, Switzerland
⁴ Sikika, Dar es Salaam, Tanzania
⁵ Ghana Integrity Initiative, Accra, Ghana

Heather McMullen
Roles: Conceptualization, Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing

Victoria Boydell
Roles: Conceptualization, Methodology, Resources, Writing – Review & Editing

Joanna Paula Cordero
Roles: Conceptualization, Methodology, Project Administration, Writing – Review & Editing

Petrus S. Steyn
Roles: Conceptualization, Funding Acquisition, Methodology, Project Administration, Supervision, Writing – Review & Editing

James Kiarie
Roles: Conceptualization, Methodology, Supervision, Writing – Review & Editing

Patrick Kinemo
Roles: Resources, Writing – Review & Editing

Alice Monyo
Roles: Resources, Writing – Review & Editing

Mary Awelana Addah
Roles: Resources, Writing – Review & Editing

Jacob Tetteh Ahuno
Roles: Resources, Writing – Review & Editing

Osei-Bonsu Gyamfi
Roles: Resources, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background: Social accountability interventions aim to propel change by raising community voices and holding duty bearers accountable for delivering on rights and entitlements. Evidence on the role of such interventions for improving community health outcomes is steadily emerging, including for sexual and reproductive health and rights (SRHR). However, these interventions are complex social processes with numerous actors, multiple components, and a highly influential local context. Unsurprisingly, determining the mechanisms of change and what outcomes may be transferable to other similar settings can be a challenge. We report our methodological considerations to account for complexity in a social accountability intervention exploring contraceptive uptake and use in Ghana and Tanzania.
Main Body: The Community and Provider driven Social Accountability Intervention (CaPSAI) study explores the relationship between a health facility-focused social accountability intervention and contraceptive service provision in two countries. This 24-month mixed-method quasi-experimental study, using an interrupted time series with a parallel control group, is being undertaken in 16 sites across Ghana and Tanzania in collaboration with local research and implementation partners. The primary outcomes include changes in contraceptive uptake and use. We also measure outcomes related to current social accountability theories of change and undertake a process evaluation.
We present three design features: co-design, ‘conceptual’ fidelity, and how we aim to track the intervention as ‘intended vs. implemented’ to explore how the intervention could be responsive to the embedded routines, local contextual realities, and the processual nature of the social accountability intervention.
Conclusions: Through a discussion of these design features and their rationale, we conclude by suggesting approaches to intervention design that may go some way in responding to recent challenges in accounting for social accountability interventions, bearing relevance for evaluating health system interventions.

Keywords

Social Accountability, Reproductive Health, Complex Interventions

Corresponding author: Petrus S. Steyn

Competing interests: No competing interests were disclosed.

Grant information: The author(s) declared that no grants were involved in supporting this work.

Copyright: © 2021 McMullen H et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: McMullen H, Boydell V, Cordero JP et al. Accounting for complexity – Intervention design in the context of studying social accountability for reproductive health [version 1; peer review: 3 approved with reservations]. Gates Open Res 2021, 5:107 (https://doi.org/10.12688/gatesopenres.13260.1) First published: 22 Jul 2021, 5:107 (https://doi.org/10.12688/gatesopenres.13260.1) Latest published: 23 May 2022, 5:107 (https://doi.org/10.12688/gatesopenres.13260.2)

Abbreviations

CaPSAI – Community and Provider driven Social Accountability Intervention

HRP - UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP Research)

SRHR – Sexual and reproductive health and research

WHO – World Health Organization

Disclaimer

The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views, decisions or policies of the institutions with which they are affiliated.

Introduction

The importance of taking a complexity approach to evaluating ‘real world’ health interventions has now been well established (Craig et al., 2013; Greenhalgh & Papoutsi, 2018; Moore et al., 2015; Portela et al., 2019). Social accountability interventions, which have gained recognition as a part of health systems strengthening and raising community voice, are associated with an increasing range of health benefits (Kruk et al., 2018; Schaaf et al., 2017; Van Belle et al., 2018). As a result, there is growing interest in how to understand, assess, and scale successful results. We explore the design of a social accountability intervention evaluated in a two-country study aiming to improve quality of care in order to increase contraceptive uptake and use and present three design features that responded to and aimed to account for complexity.

Social accountability interventions aim to propel community-driven change and empower citizens and communities to hold duty bearers accountable for promised rights and entitlements (Joshi, 2017). These interventions aim to be community-owned and led and improve life for local citizens by raising their voices, representing their interests, and increasing their capabilities, ultimately transforming power relations. Accountability in the context of sexual and reproductive health and rights has been described as ‘the appropriate prioritisation of sexual and reproductive health and rights (SRHR) and its implementation throughout the health system and ensuring access to SRHR services, with attention to high-quality and respectful care.’ (Boydell et al., 2019). Social accountability is conceptualised as able to bring about change through a series of activities over time. These may include community education and empowerment, increasing the understanding of rights and entitlements, community mobilisation and data collection, a process of evaluation and measurement against standards and priorities, and a process of interfacing duty bearers and rights-holders or service users working to hold them to account. The process of interfacing can be in the form of meetings, public hearings, or other forums where community members can interface with power holders and each other to share concerns, apply pressure, and track change. Social accountability is an ongoing contingent and often political process, and it operates both within and outside of formal structures and processes. It is, amongst other things, evidently complex. However, how social accountability interventions work, whether the theories of change are accurate, and what the key ingredients for success are across contexts and health topics require more empirical insight, particularly as such interventions are taken on by mainstream health actors and implementers across a range of settings.

A recent World Health Organization (WHO) convened supplement on complexity approaches, and public health guidance describes the multiple component nature, nonlinear causal pathways, role of local context, and general under examination of outcomes as particular challenges for health interventions with multiple priorities and limited resources (Portela et al., 2019). Updated guidance on implementation studies from the Medical Research Council was described by Greenhalgh and Papoutsi as emphasising ‘‘‘non-linearity and iterative local tailoring’ and placed substantially more emphasis on the need for non-experimental, mixed methods and process-based approaches for studying such phenomena’ (Greenhalgh & Papoutsi, 2018; Moore et al., 2015). A WHO department of sexual and reproductive health and research convened community of practice on measuring and evaluating social accountability interventions for reproductive, maternal, and child health reported similar considerations with specific emphasis on power relations and the political nature of accountability interventions (Boydell et al., 2019).

Here we explore the Community and Provider driven Social Accountability Intervention (CaPSAI), a study that explores the impact of a social accountability process on contraceptive uptake and use (Steyn et al., 2020). It is one of the first studies to evaluate the effectiveness of social accountability interventions on behaviour related to family planning. It aims to build the evidence base on the potential for such interventions to improve SRHR. Literature and evidence on accountability strategies to improve SRHR is steadily emerging (Boydell et al., 2019; Gullo et al., 2017; Van Belle et al., 2018) though challenges remain in understanding how best to evaluate these programmes and determine best practices for scale-up. Considering the social accountability process as a complex intervention, with numerous actors, a highly influential local context, and a number of interacting components, as well as being a process over time, it was considered essential to move beyond simple intervention thinking and call on complexity approaches to design and evaluate the CaPSAI study. This paper explores how the CaPSAI intervention was designed to respond to real-life conditions and how recent thinking in implementation science and complex intervention evaluation were considered in developing the CaPSAI intervention. We consider three design features and how they aimed to account for complexity; intervention fidelity, co-design, and the intervention as ‘intended vs. implemented.’

The Community and Provider driven Social Accountability Intervention (CaPSAI)

The Community and Provider driven Social Accountability Intervention (CaPSAI) study aims to make a robust addition to the literature and evidence base on participatory and social accountability processes for health. The CaPSAI Project has been registered at Australian New Zealand Clinical Trials Registry (ACTRN12619000378123, 11/03/2019). CaPSAI is a quasi-experimental, mixed methods evaluation implemented across 16 sites in Ghana and Tanzania. The intervention was delivered and evaluated by local civil society partners and research organisations over a period of 24 months. Specifically, it explores the role of such interventions for aspects of SRHR by evaluating the impact and process of implementation of an eight-step social accountability intervention on contraceptive uptake and use in low resource settings with low modern contraceptive uptake (Moore et al., 2015). The CaPSAI study aims to describe and examine how social accountability processes are implemented and operationalised, focusing on behaviours, decision-making processes, and the barriers and facilitators of change. The findings aim to be generalisable to other like settings. It also aims to develop more responsive quantitative measures for social accountability interventions and demonstrate the relationship between social accountability processes and the uptake and use of contraceptives and other family planning behaviours (See Steyn et al., 2020 for details on the research protocol).

Contraceptive uptake is evaluated through an interrupted time series design with a control group (ITS-CG). A cohort of women who are new users of contraception is tracked using standardised interview questions across both intervention and control facilities to measure changes in behaviours around contraceptive use over one year. To capture social accountability intermediate outcomes, such as empowerment of women and health providers and expansion of negotiated spaces, a cross-sectional survey using accountability-related psychometric scales is conducted at pre- and post-intervention phases. The effects of the social accountability intervention and the implementation process are measured through a process evaluation comprising context mapping, qualitative interviews, document review, and implementation tracking. Case studies of change are also collected. A process evaluation was seen as essential due to the complex and processual nature of the social accountability intervention and the challenges in determining causal chains and clearly attributing outcomes to intervention inputs (Moore et al., 2015; Palmer et al., 2016).

Figure 1 outlines the CaPSAI theory of change, and Table 1 describes the eight identified steps in more detail. While these are referred to as ‘steps,’ they may be better conceptualised as phases and may contain a number of activities or ranges of activities within each step. We also acknowledge that social accountability interventions are best considered as a process and not as linear discrete steps or tools that will necessarily combine to create social change; however, these steps present a structure for the intervention.

Figure 1. CaPSAI Theory of Change (This figure has been reproduced with permission from Steyn et al., 2020).

Table 1. Standard steps in the CaPSAI social accountability process.

Step	Description
1.Introduction of the intervention to the community	The implementation partner (usually a civil society organization) meets with local leaders, identifies stakeholders and sets up the infrastructure to deliver the community scorecard.
2.Mobilization of participants for the intervention	The implementing partner will gather community partners, service providers and the user groups of the health facility/ services to describe the intervention. The implementing civil society organisation should also identify a respected community mobilizer who will jointly facilitate the roll out of the intervention and join the implementation team.
3.Health, rights and civic education with community participants	The implementation partner shares information on existing service standards and provides training on rights, good governance and accountability. The group begins to rate existing services against rights-based standards and generate discussion about local priorities.
4.Prioritization meeting with community	The implementation partner distills themes and priorities raised by the community. The community groups then collectively score the issues and indicators and set priority areas for action.
5.Prioritization meeting with duty bearers	The implementation partner distills themes and priorities raised by the service providers. The providers then collectively score the issues and indicators and set priority areas for action.
6.Interface meeting and joint action planning	The implementation partner then holds a joint meeting between the community, the service providers and other duty bearers. Following the presentation of results from the prioritization meetings the community groups and the service providers will aim to reach consensus on the ranking of the priority items and the actions required to address them. An action plan with assigned roles and responsibilities will be developed for the following 6–12 month period.
7.First follow-up meeting with community and duty bearers at three months	Priority areas and action items will be followed up with both the community and service providers. For any unresolved issues these meetings present an opportunity to involve high-level duty bearers of third party pressure (media/ politicians) to increase the pressure to act.
8.Second follow-up meeting with community duty bearers at six months	A second follow up meeting will enable the monitoring of longer range outcomes and on the remedy of unresolved issues raised in the first follow up meeting. The community and service providers will continue to monitor the action plan for changes in relation to agreed priority areas.

Key design features in enabling and accounting for local adaptation

The social accountability process is deeply situated in, and contingent upon, the local context and involves multiple actors and factors that can be difficult to account for, inevitably presenting challenges for evaluation. Each process will be and should be different as it responds to locally determined concerns and power relations. As social accountability interventions are not singular discrete interventions, obtaining mainstream achievements and measurements of fidelity, dose, and reach and ensuring knowledge of the ‘active ingredients that allowed the outcomes to take hold can be particularly challenging (Boydell et al., 2019; Craig et al., 2013; Moore et al., 2015) Thus, in designing the intervention and its evaluation the team considered emerging research and best practice on complex intervention design and evaluation. Table 2 describes dimensions of complexity present in the CaPSAI intervention. We now present three broad considerations: co-design, considering ‘conceptual fidelity’ versus standardisation, and accounting for the intervention as intended versus implemented.

Table 2. Accounting for complexity in the Community and Provider driven Social Accountability Intervention (CaPSAI) study design (This table has been reproduced with permission from Steyn et al., 2020).

Dimension of complexity	CaPSAI dimension	MRC recommended design features	CaPSAI study design features
A large number of interactions between components within the intervention (Craig et al., 2008).	CaPSAI intervention requires separate and joint activities with community organizations, health providers, and duty bearers to produce an effective space for collective action and change.	A theoretical understanding of how the intervention causes change (Craig et al., 2008).	CaPSAI developed a theory of change (Figure 1)
A number of behaviors required by those delivering or receiving the intervention (Craig et al., 2008).	Behavior change in varying degrees on the part of community members, health providers, and duty bearers is required for effects to take hold.	A process evaluation design to study the implementation process to address ‘how’ the intervention worked in practice and better understand the ‘active ingredients’ (Craig et al., 2008; Moore et al., 2015).	Process evaluation is a main component of the study design
A number of groups or organizational levels targeted by the intervention (Craig et al., 2008).	CaPSAI intervention targets community members, health care providers and duty-bearers at the community and facility level.	A larger sample size and cluster rather than individual level designs (Craig et al., 2008).	Evaluation and sampling at both the community (individual) and health facility level (cluster).
Numerous and variable outcomes (Craig et al., 2008).	The primary outcomes include an increase in contraceptive uptake and indicators of contraceptive use alongside intermediary outcomes such as increases in social capital, collective efficacy, and empowerment.	Use of a range and mix of measures and methods to capture complexity and unintended consequences (Craig et al., 2008; Moore et al., 2015).	A range of methods and instruments aim to capture the primary and intermediary outcomes as well as the implementation process itself.
A degree of flexibility or tailoring of the intervention is permitted (Craig et al., 2008).	Implementation should maintain conceptual fidelity to the eight standard steps but allows for local adaptation to produce the effects.	Fidelity should be considered ‘functionally rather than Compositionally (Fox, 2015) to allow interventions to be responsive to context while still being meaningfully evaluated (Craig et al., 2008; McMullen et al., 2015; Moore et al., 2015).	The process evaluation and combination of research instruments have been designed to capture functional fidelity in the implementation

Co-design

Alongside the growth of implementation science has been calls for greater uptake of co-design approaches, particularly in the context of complex interventions (Craig et al., 2013; Moore et al., 2015). Co-design stresses equal participation, particularly of ‘end users’, and recognises that interventions will be more responsive and likely to deliver meaningful results if all those with a stake in their delivery and outcomes are involved in all aspects of intervention design, delivery, and evaluation (Donetto et al., 2015; Goodyear Smith et al., 2015; Slattery et al., 2020). For the CaPSAI study, local civil society organisations with experience in delivering health-related social accountability interventions were selected as implementing partners. Implementing partners then selected community representatives to become members of the implementation and research teams. Community members were also tasked with the facilitation and implementation of the intervention across the 16 sites. In some cases, these community members had already worked with the civil society organisations; in others, they were newly recruited. In order to recognise the wealth of experience and established routines and practices of implementing partners as well as respond to the local context, the intervention was co-designed by the study implementation leads and civil society implementing partners from the local community.

Stages in the intervention design

Stage 1: The first stage in the design of the intervention was a review of existing literature and programmes related to social accountability and health. Programme descriptions, evidence, and programme reports for health-related social accountability interventions such as community scorecards, report cards, citizen voice, accountability, and citizen hearings were gathered to define the key phases in the social accountability process (See Table 3). These data were brought together with the findings from the formative phase study UPTAKE Project (Cordero et al., 2019; Steyn et al., 2016) and emerging findings from the Evidence Project studies on social accountability in the context of family planning (Boydell et al., 2018) to set the groundwork and proof of concept for CaPSAI and were used to develop the overarching theory of change. From a review, a composite of components (‘steps’) were determined that are typical of social accountability interventions and theories of change (see Table 3. These ‘steps’ structured the second phase of the intervention design process.

Table 3. Toolkits reviewed to develop the intervention.

Document Title	Author	Date of publication	Source
Information and collective action in community-based monitoring of schools: Field and lab experimental evidence from Uganda	Abigail Barr, F. Mugisha, P. Serneels, A. Zeitlin	Aug.12	https://pdfs.semanticscholar.org/99f5/806ab361f3308d652b9549b390e4f183b672.pdf
Enhancing governance and health system accountability for people centered healthcare: an exploratory study of community scorecards in Afghanistan	Edward Anbrasi, Kojo Osei-Bonsu, Casey Branchini, Temor Shah Targhal, Said Habib, Arwal A.	Jul.15	http://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-015-0946-5
When is community based monitoring effective? Evidence from a randomized experiment in primary health in Uganda	Bjorkman, M. and Svensson, J.	Apr.10	www.jstor.org.ezproxy.library.qmul.ac.uk/stable/pdf/40601246.pdf
Scorecards and social accountability for improved maternal and newborn health services: A pilot in the Ashanti and Volta regions of Ghana	Blake, C., Annorbah- Sarpei, N., Ismaila, Y., Clark, S.	Sep.16	http://ac.els-cdn.com/S0020729216304428/1-s2.0-S0020729216304428-main. pdf?_tid=cce78452-faaa-11e6-aa3c-00000aacb361&acdnat=1487952397_ af8e3ccf5825efa4ea70fa08c3318cee
Citizen Voice and Action	Jeff Hall	Jul.14	https://www.escr-net.org/node/366893
East African Community Regional Reproductive Maternal Newborn and Child Health Scorecard	Elizabeth Muiruri	Nov.14	https://everyone.savethechildren.net/sites/everyone.savethechildren.net/files/ EAC%20RMNCH%20Scorecard.pdf
Tanzania Community Score Card Transparency and Accountability Project	Sandra Michaelson	Nov.13	http://saatlas.org/uploads/files/Results_for_Development_SC_Tanzania_CSC_Interim_ Report.pdf
Scorecard Toolkit: CARE Malawi	Sarah Gullo	May.13	https://resourcecentre.savethechildren.net/sites/default/files/documents/6800.pdf
Maternal Health Alliance Malawi	Sarah Gullo	Nov.14	http://familyplanning.care2share.wikispaces.net/file/view/MHAP_November_Update.pdf
Role of Social Accountability in Improving Health Outcomes: Overview and Analysis of Selected INGO Experiences	K.D. Hoffman	Jun.14	http://www.coregroup.org/storage/documents/Resources/Tools/Social_ Accountability_Final_online.pdf
Citizen Voice and Action (as reviewed by "Role of Social Accountabilty" paper	K.D. Hoffman	Jun.14	http://www.coregroup.org/storage/documents/Resources/Tools/Social_ Accountability_Final_online.pdf
Partnership Defined Quality Framework	K.D. Hoffman	Jun.14	http://www.coregroup.org/storage/documents/Resources/Tools/Social_ Accountability_Final_online.pdf
CARE's Community Scorecard	K.D. Hoffman	Jun.14	http://www.coregroup.org/storage/documents/Resources/Tools/Social_ Accountability_Final_online.pdf
White Ribbon Alliance Social Watch Approach	K.D. Hoffman	Jun.14	http://www.coregroup.org/storage/documents/Resources/Tools/Social_ Accountability_Final_online.pdf
White Ribbon Alliance Uganda Participatory Health Facility Assessment	K.D. Hoffman	Jun.14	http://www.coregroup.org/storage/documents/Resources/Tools/Social_ Accountability_Final_online.pdf
GOAL Accountability can Transform Health	K.D. Hoffman	Jun.14	http://www.coregroup.org/storage/documents/Resources/Tools/Social_ Accountability_Final_online.pdf
The community scorecard process: methodology, use, successes, challenges, and opportunities	Jephter Mwanza and Nina Ghambi	Dec.11	http://pubs.iied.org/pdfs/G03207.pdf
Effects of CARE’s Community Score Card on reproductive health-related outcomes	Sara Gullo, Christine Galavotti, Anne Sebert Kuhlmann, Thumbiko Msiska, Phil Hastings, C. Nathan Marti	Feb.17	http://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0171316&type= printable
Social Accountability Services: Case Study 1: Andhra Pradesh, India: Improving Health Services through Community Score Cards	Vivek Mistra, P. Ramasankar, Lakshmi Durga, Sanjay Agarwal, J.V.R. Murty	Aug.07	http://unpan1.un.org/intradoc/groups/public/documents/un-dpadm/unpan044598.pdf
Governance and Transparency Fund, Environmental Governance, Ghana, the Community Scorecard Approach	Governance and Transparency Programme	2014	https://washmatters.wateraid.org
Evaluation and Design of Social Accountability Component of the Protection of Basic Services Project, Ethiopia	Samuel Taddesse, Biraj Swain, Merga Afeta, Gadissa Bultosa	Jun.10	http://siteresources.worldbank.org/INTPCENG/Resources/CSC+Gambia.pdf
Transparency for development baseline report, Tanzania	Jean Arkedis, Jessica Creighton, Archon Fung, Stephen Kosack, Dan Levy, Rohit Naimpally, Lindsey Roots and Courtney Tolmie	Sep.16	http://t4d.ash.harvard.edu/sites/default/files/file-uploads/Full%20T4D%20Baseline%2 0Report%20FINAL.pdf
Accountability in Local Service Delivery: The Tuungane Community Scorecard Approach	Guillaume Labrecque and Isatou Batonon	May.15	https://www.rescue.org/sites/default/files/document/660/ accountabilityinlocalservicedeliveryenglishfinal.pdf

Stage 2: For each of the eight social accountability ‘steps’ a set of questions was developed that aimed to elicit the routine practices, knowledge, experiences, and concerns of the two national non-governmental organisations implementing the activities. If routines are highly contextual and specific practices that are ‘recurrent, collective and interactive behaviour patterns’ help implementers structure and make sense of their roles and worlds, it was essential to understand existing routines and roles and the ‘normal practice’ of implementation (Greenhalgh, 2008, p1269). The key questions aimed to ascertain the ways in which the implementing partners are already addressing the core aspects of intervention fidelity (as considered functionally (Hawe et al., 2004)) in their regular practice and existing implementation strategies for existing social accountability programmes. Understanding these practices in advance allows for designing an intervention that better reflects what may take place in ‘actual’ implementation by not suggesting new or changed practices where they are not necessary. Findings from the first and second design stages were synthesised into a guide for design stages three and four.

Stage 3: In the third stage of the intervention design, multiple meetings were held over a period of months with the implementation teams prior to the start of implementation. Initial meetings introduced the study and the objectives. Subsequent meetings engaged a discussion structured around the findings of the review and the key questions. The tentative theory of change and identified ‘steps’ were used as prompts to discuss and explain previous experiences of the implementing partners in delivering social accountability interventions. Discussions were recorded and loosely transcribed, and notes were fed back to the team and used to ‘build out’ the intervention. Through an iterative process, an intervention manual to use during the study was put together.

The intervention manual is written in a ‘workbook’ style and does not set out an overly prescriptive or standardised intervention to be delivered. The manual sets out the aims and objectives of the study, along with essential information for study conduct and implementation as part of a research project (which was new for the implementation teams). For each of the eight social accountability ‘steps,’ the manual describes how the step is conceptualised within the theory of change and lists key questions and considerations for implementers with examples that emerged from the co-design process. Workbook pages are included for pre and post-implementation. While the study intervention implementation manual was primarily designed by the implementation team, comprised of WHO team members and civil society partners, the site-specific implementation plans involve the local facilitators and community implementers across the 16 sites. The pre-implementation plan is where implementers set out how their plans adhere to the core tenets of the intervention step and respond to key criteria and concerns, essentially how ‘fidelity’ is composed. This plan is then reviewed by the implementing teams with the implementation leads to discuss how it meets the requirements of the study and achieves fidelity. The post-implementation report allows implementers to account for implementation ‘on the day’ and note any deviations from the plan or to remark on exceptional events.

Stage 4: In the final stages, feedback on the draft manual was gathered, and further refinements were made, and the ‘workbook’ aspect of the manual was further considered. Finally, the teams worked to finalise the planned intervention and agree on a final manual. Training then took place with the local implementing teams over a period of days.

Tracking the intervention was developed to enable and account for local adaptation. As reflected in the design of the implementation manual, fidelity to the theory of change and core aspects of the intervention is key, and how this fidelity is composed is expected to vary. The (re)consideration of fidelity is one of the intervention design features that aims to account for the complex and contextual nature of social accountability interventions and their success.

Intervention ‘fidelity’

Recent literature acknowledges that adaptive intervention strategies yield more responsive and localised interventions that may respond better to community needs (Greenhalgh & Papoutsi, 2018). Some thinking in implementation science and complex interventions indicates the value of considering intervention integrity and fidelity functionally rather than compositionally (Greenhalgh & Papoutsi, 2018; Hawe et al., 2004; McMullen et al., 2015; Perez et al., 2016). In measuring complex interventions alongside quantitative study designs, this conceptualisation of fidelity has been suggested as a potentially more responsive approach to assessing the integrity of interventions. This requires distilling the essential criteria required to have fidelity to the overarching tenets of the intervention but moves away from an overly prescribed and standardised pathway of implementation. Hawe et al. (2004) ask what standardisation is in a complex intervention and suggest ‘rather than defining the components of the intervention as standard … what should be defined as standard are the steps in the change process that the elements are purporting to facilitate or the key functions that they are meant to have’ (pg.1561). The ‘workbook’ style of the implementation plan for each step develops a form of ‘mini’ site-specific protocol for each intervention step, that adheres to the core aspects that support the theory of change while accounting for local context and adaptability (McMullen et al., 2015). The implementation plans and design also respond to the design of the process evaluation. As described by Palmer et al. (2016), ‘A key challenge is how to find a balance between the fluidity that complexity and process so obviously warrant and the development of process evaluation aims, questions and procedures in advance (pg 2).’

A complex social intervention such as a social accountability process, implemented by organisations with previous experience and established routines and practices indicates that the intervention as envisaged prior to implementation will differ slightly from actual implementation ‘on the day.’ To account for this reality, a ‘pre-implementation plan and post-implementation report ‘is included as a part of the intervention design, to be completed before and after each step of the social accountability process.

The intervention ‘as intended’ versus ‘as implemented’

The pre-implementation plans and post-implementation reports form a part of the document review for the process evaluation research team, alongside the qualitative interviews, context mapping, and case studies of change. The reports support an understanding of the dose, reach, and fidelity of the intervention while accounting for and enabling adaptability.

This feature allows the intervention ‘as intended’ and the intervention ‘as implemented’ to be tracked and to account for divergences at the reporting stage. This may allow for a better understanding of the ‘active ingredients’ in the implementation process and for a better description of what took place over the course of the study. This will assist evaluators in assessing whether the theory of change was accurate, what the causal pathways for intervention outcomes may be, and what may be essential for scalability and generalisability. Alongside the pre-implementation plans and post-intervention reports, the process evaluation contains a method where small case studies of change are gathered. Through in-depth qualitative interviews and document review, reported instances of change thought to be attributed to the social accountability intervention are explored. Here researchers can trace the instances of change and gather the accounts of local actors as to how these changes took place, gathering relevant documentation and triangulating interviews as needed. Drawing on ethnographic methods, these combined accounts help develop the picture represented by the quantitative findings with rich accounts of the intervention in action and local perceptions and descriptions of impact.

Conclusion

As described by Hawe and Shiell back in 2004, ‘reducing a complex system to its component parts amounts to an “irretrievable loss of what makes it a system”’ (Hawe et al., 2004; p. 1562). ‘Real world’ interventions also experience real-world pressures such as budget constraints, tight timelines, international teams, and so on. Trying to incorporate emerging best practice that challenges the status quo can present challenges. What is described in this paper reflects efforts to incorporate some of the relevant guidance to support a complex and political intervention on what is often a controversial and contested set of rights and health behaviours. This is a reflection on the design process, and the results of the study will be reported, upon completion, elsewhere. Co-design of the intervention, a conceptualisation of fidelity that is as open and adaptive as possible, and using ethnographic approaches to track the intervention steps as intended versus as implemented as part of a process evaluation have been how we considered complexity in the intervention design for the CaPSAI study.

Data availability

No underlying data are associated with this article.

Acknowledgments

The authors acknowledge the CaPSAI Project team members, specifically: Dela Nai and Kamil Fuseini, Population Council Ghana; Donat Shamba and Sigilbert Mrema, Ifakara Health Institute, Tanzania; Vernon Mochache, independent; Cecilia Milford, MaTCH Research Unit.

Faculty Opinions recommended

References

Boydell V, Neema S, Wright K, et al.: Closing the Gap between People and Programs: Lessons from Implementation of Social Accountability for Family Planning and Reproductive Health in Uganda. Afr J Reprod Health. 2018; 22(1): 73–84. PubMed Abstract
Boydell V, Schaaf M, George A, et al.: Building a transformative agenda for accountability in SRHR: lessons learned from SRHR and accountability literatures. Sex Reprod Health Matters. 2019; 27(2): 1622357. PubMed Abstract | Publisher Full Text | Free Full Text
Cordero JP, Steyn PS, Gichangi P, et al.: Community and provider perspectives on addressing unmet need for contraception: key findings from a formative phase research in Kenya, South Africa and Zambia (2015-2016). Afr J Reprod Health. 2019; 23(3): 106–19. https://www.ajol.info/index.php/ajrh/article/view/191220. PubMed Abstract
Craig P, Dieppe P, Macintyre S, et al.: Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008; 337: a1655. PubMed Abstract | Publisher Full Text
Craig P, Dieppe P, Macintyre S, et al.: Developing and evaluating complex interventions: the new Medical Research Council guidance. Int J Nurs Stud. 2013; 50(5): 587–92. PubMed Abstract | Publisher Full Text
Donetto S, Pierri P, Tsianakas V, et al.: Experience-based Co-design and Healthcare Improvement: Realizing Participatory Design in the Public Sector. Des J. 2015; 18(2): 227–248. Publisher Full Text
Fox JA: Social Accountability: What Does the Evidence Really Say? World Dev. 2015; 72: 346–361. Publisher Full Text
Goodyear-Smith F, Jackson C, Greenhalgh T: Co-design and implementation research: challenges and solutions for ethics committees. BMC Med Ethics. 2015; 16: 78. PubMed Abstract | Publisher Full Text | Free Full Text
Greenhalgh T: Role of routines in collaborative work in healthcare organisations. BMJ. 2008; 337: a2448. (Published 17 November 2008). PubMed Abstract | Publisher Full Text
Greenhalgh T, Papoutsi C: Studying complexity in health services research: desperately seeking an overdue paradigm shift. BMC Med. 2018; 16(1): 95. PubMed Abstract | Publisher Full Text | Free Full Text
Gullo S, Galavotti C, Kuhlmann AS, et al.: Effects of a social accountability approach, CARE's Community Score Card, on reproductive health-related outcomes in Malawi: A cluster-randomized controlled evaluation. PLoS One. 2017; 12(2): e0171316. PubMed Abstract | Publisher Full Text | Free Full Text
Hawe P, Shiell A, Riley T: Complex interventions: how “out of control” can a randomised controlled trial be? BMJ. 2004; 328(7455): 1561–3. PubMed Abstract | Publisher Full Text | Free Full Text
Joshi A: Legal Empowerment and Social Accountability: Complementary Strategies Toward Rights-based Development in Health? World Dev. 2017; 99: 160–72. Publisher Full Text
Kruk ME, Gage AD, Arsenault C, et al.: High-quality health systems in the Sustainable Development Goals era: time for a revolution. Lancet Glob Health. 2018; 6(11): e1196–e1252. PubMed Abstract | Publisher Full Text | Free Full Text
McMullen H, Griffiths C, Leber W, et al.: Explaining high and low performers in complex intervention trials: a new model based on diffusion of innovations theory. Trials. 2015; 16: 242. PubMed Abstract | Publisher Full Text | Free Full Text
Moore GF, Audrey S, Barker M, et al.: Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015; 350: h1258. PubMed Abstract | Publisher Full Text
Palmer VJ, Piper D, Richard L, et al.: Balancing Opposing Forces—A Nested Process Evaluation Study Protocol for a Stepped Wedge Designed Cluster Randomized Controlled Trial of an Experience Based Codesign Intervention: The CORE Study. Int J Qual Methods. 2016; 15(1). Publisher Full Text
Pérez D, van der Stuyft P, Zabala MC, et al.: A modified theoretical framework to assess implementation fidelity of adaptive public health interventions. Implement Sci. 2016; 11(1): 91. PubMed Abstract | Publisher Full Text | Free Full Text
Portela A, Tunçalp Ö, Norris SL: Taking a complexity perspective when developing public health guidelines. Bull World Health Organ. World Health Organization. 2019; 97(‎4)‎: 247–247A. PubMed Abstract | Publisher Full Text | Free Full Text
Schaaf M, Topp SM, Ngulube M: From favours to entitlements: community voice and action and health service quality in Zambia. Health policy and planning. 2017; 32(6): 847–859. PubMed Abstract | Publisher Full Text | Free Full Text
Slattery P, Saeri AK, Bragge P: Research co-design in health: a rapid overview of reviews. Health Res Policy Syst. 2020; 18(1): 17. PubMed Abstract | Publisher Full Text | Free Full Text
Steyn PS, Boydell V, Cordero JP, et al.: Rationale and design of a complex intervention measuring the impact and processes of social accountability applied to contraceptive programming: CaPSAI Project [version 2; peer review: 2 approved]. Gates Open Res. 2020; 4: 26. PubMed Abstract | Publisher Full Text | Free Full Text
Steyn PS, Cordero JP, Gichangi P, et al.: Participatory approaches involving community and healthcare providers in family planning/contraceptive information and service provision: a scoping review. Reprod Health. 2016; 13(1): 88. PubMed Abstract | Publisher Full Text | Free Full Text
van Belle S, Boydell V, George AS, et al.: Broadening understanding of accountability ecosystems in sexual and reproductive health and rights: A systematic review. PLoS One. 2018; 13(5): e0196788. PubMed Abstract | Publisher Full Text | Free Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 22 Jul 2021

Author details Author details

Heather McMullen
Roles: Conceptualization, Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing

Victoria Boydell
Roles: Conceptualization, Methodology, Resources, Writing – Review & Editing

Joanna Paula Cordero
Roles: Conceptualization, Methodology, Project Administration, Writing – Review & Editing

Petrus S. Steyn
Roles: Conceptualization, Funding Acquisition, Methodology, Project Administration, Supervision, Writing – Review & Editing

James Kiarie
Roles: Conceptualization, Methodology, Supervision, Writing – Review & Editing

Patrick Kinemo
Roles: Resources, Writing – Review & Editing

Alice Monyo
Roles: Resources, Writing – Review & Editing

Mary Awelana Addah
Roles: Resources, Writing – Review & Editing

Jacob Tetteh Ahuno
Roles: Resources, Writing – Review & Editing

Osei-Bonsu Gyamfi
Roles: Resources, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

The author(s) declared that no grants were involved in supporting this work.

Article Versions (2)

version 2

Revised

Published: 23 May 2022, 5:107

https://doi.org/10.12688/gatesopenres.13260.2

version 1

Published: 22 Jul 2021, 5:107

https://doi.org/10.12688/gatesopenres.13260.1

© 2021 McMullen H et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
Gates Open Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

McMullen H, Boydell V, Cordero JP et al. Accounting for complexity – Intervention design in the context of studying social accountability for reproductive health [version 1; peer review: 3 approved with reservations]. Gates Open Res 2021, 5:107 (https://doi.org/10.12688/gatesopenres.13260.1)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?

Key to Reviewer Statuses VIEW HIDE

ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Version 1

VERSION 1

PUBLISHED 22 Jul 2021

Views

Reviewer Report 08 Oct 2021

Deborah D. Diliberto, Global Health, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada

Approved with Reservations

https://doi.org/10.21956/gatesopenres.14495.r31154

This open letter reports the methodological considerations for designing and evaluating CaPSAI, a social accountability intervention exploring contraceptive uptake and use in Ghana and Tanzania. While the letter focuses specifically on a social accountability intervention addressing reproductive health and rights, it makes important contributions to the design of complex interventions more generally, and therefore, will be of interest to readers working in complex health and social interventions research.

The letter addresses an important issue in complex interventions research – how to account for complexity as part of the contingent and emergent links between the intervention, individuals, and society. Given the challenges with representing the intersecting and intertwining processes of designing a complex intervention, I commend the authors for their work on this letter.

Throughout the letter, the authors emphasize the notion of complexity as contingent. What is missing, however, is an exploration of why accounting for complexity in this way is important. The letter would be strengthened with a short explanation of what gets lost or omitted when taking the mainstream notion of intervention as a linear process and a static collection of materials – or alternatively, what is gained when taking a more contingent and context-specific approach to complexity and design.This could be included in the introduction – for example, revising paragraph 4 to provide more explanation of their view/approach to complexity, rather than references that describe acknowledgements of complexity.

The letter would also be strengthened with a short description of the intervention itself. There is a heading with a few paragraphs describing the aims of the CaPSAI and evaluation; however, there is no description of the actual intervention. What is the social accountability mechanism? Table 1 describes a community scorecard, but no further explanation is given. It is also unclear what aspects of the intervention were decided/set in advance, and what aspects were co-designed. For example, was the community scorecard decided by the implementation team, then the approach to implementation (i.e. the “intervention manual”) co-designed with the implementing partners?

Additionally, the letter would be strengthened with more methodological detail about the stages in the intervention design process to understand how, and to what extent, the aims of co-design were achieved. At present, it is not clear who was involved in each intervention design stage and what methods were used. For example, who conducted the reviews in Stage 1? What (participatory?) methods were used to ‘elicit the routine practices…’ (page 6) in Stage 2 and who synthesized findings from Stage 1 and 2? Who facilitated the meetings in Stage 3 and which team ‘built out’ the intervention (page 6)? This information would also help readers looking towards similar approaches for intervention design.

Finally, in the section ‘The intervention ‘as intended’ versus ‘as implemented’, the authors suggest their approach “…may allow for a better understanding of the ‘active ingredients’ in the implementation process…” (page 10) – my emphasis. This is interesting because it is a departure from the mainstream thinking that focuses on distilling intervention components and disregards how interventions are implemented in situ. This distinction between implementation versus intervention is also touched on somewhat in the section on ‘Intervention ‘fidelity’’, but overall, the nuance of this argument gets lost in the text. The letter would be improved with more clearly articulated discussion of this distinction and what a focus on implementation offers before going into describing how this was achieved.

Is the rationale for the Open Letter provided in sufficient detail?

Yes
Does the article adequately reference differing views and opinions?

Yes
Are all factual statements correct, and are statements and arguments made adequately supported by citations?

Yes
Is the Open Letter written in accessible language?

Yes
Where applicable, are recommendations and next steps explained clearly for others to follow?

Partly

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Design and evaluation of complex health and social interventions in low resource settings; interdisciplinary and mixed methods approaches.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 31 May 2022

Joanna Cordero, UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP Research), Department of Sexual and Reproductive Health and Research, World Health Organization, Geneva, 1202, Switzerland

31 May 2022

Author Response

Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

REVIEWER 3:

... Continue reading Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

REVIEWER 3:

EDIT 10:

Throughout the letter, the authors emphasize the notion of complexity as contingent. What is missing, however, is an exploration of why accounting for complexity in this way is important. The letter would be strengthened with a short explanation of what gets lost or omitted when taking the mainstream notion of intervention as a linear process and a static collection of materials – or alternatively, what is gained when taking a more contingent and context-specific approach to complexity and design.This could be included in the introduction – for example, revising paragraph 4 to provide more explanation of their view/approach to complexity, rather than references that describe acknowledgements of complexity.

Response: We have added a paragraph on the CaPSAI approach to social accountability and why this is important under the section describing the project.

EDIT 11:
The letter would also be strengthened with a short description of the intervention itself. There is a heading with a few paragraphs describing the aims of the CaPSAI and evaluation; however, there is no description of the actual intervention. What is the social accountability mechanism? Table 1 describes a community scorecard, but no further explanation is given. It is also unclear what aspects of the intervention were decided/set in advance, and what aspects were co-designed. For example, was the community scorecard decided by the implementation team, then the approach to implementation (i.e. the “intervention manual”) co-designed with the implementing partners?

Response: Thank you for Reviewers 1 and 3 on their suggestions for fleshing out the stages of intervention design and intervention description. We now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI. Detail was added on this in the description of Stage 3 of the intervention design including some short examples of how the intervention manual reflects the co-design as well as how the description of the intervention was changed by this. We have also put in a reference to other study resources with more detail on the intervention and the process.

EDIT 12:
Additionally, the letter would be strengthened with more methodological detail about the stages in the intervention design process to understand how, and to what extent, the aims of co-design were achieved. At present, it is not clear who was involved in each intervention design stage and what methods were used. For example, who conducted the reviews in Stage 1? What (participatory?) methods were used to ‘elicit the routine practices…’ (page 6) in Stage 2 and who synthesized findings from Stage 1 and 2? Who facilitated the meetings in Stage 3 and which team ‘built out’ the intervention (page 6)? This information would also help readers looking towards similar approaches for intervention design.

Response: As per our response to reviewer 1, EDIT 4, we now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI.

EDIT 13:
Finally, in the section ‘The intervention ‘as intended’ versus ‘as implemented’, the authors suggest their approach “…may allow for a better understanding of the ‘active ingredients’ in the implementation process…” (page 10) – my emphasis. This is interesting because it is a departure from the mainstream thinking that focuses on distilling intervention components and disregards how interventions are implemented in situ. This distinction between implementation versus intervention is also touched on somewhat in the section on ‘Intervention ‘fidelity’’, but overall, the nuance of this argument gets lost in the text. The letter would be improved with more clearly articulated discussion of this distinction and what a focus on implementation offers before going into describing how this was achieved.

Response: We address this in the paragraph under the description of CaPSAI on our approach to social accountability.
Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

REVIEWER 3:

EDIT 10:

Throughout the letter, the authors emphasize the notion of complexity as contingent. What is missing, however, is an exploration of why accounting for complexity in this way is important. The letter would be strengthened with a short explanation of what gets lost or omitted when taking the mainstream notion of intervention as a linear process and a static collection of materials – or alternatively, what is gained when taking a more contingent and context-specific approach to complexity and design.This could be included in the introduction – for example, revising paragraph 4 to provide more explanation of their view/approach to complexity, rather than references that describe acknowledgements of complexity.

Response: We have added a paragraph on the CaPSAI approach to social accountability and why this is important under the section describing the project.

EDIT 11:
The letter would also be strengthened with a short description of the intervention itself. There is a heading with a few paragraphs describing the aims of the CaPSAI and evaluation; however, there is no description of the actual intervention. What is the social accountability mechanism? Table 1 describes a community scorecard, but no further explanation is given. It is also unclear what aspects of the intervention were decided/set in advance, and what aspects were co-designed. For example, was the community scorecard decided by the implementation team, then the approach to implementation (i.e. the “intervention manual”) co-designed with the implementing partners?

Response: Thank you for Reviewers 1 and 3 on their suggestions for fleshing out the stages of intervention design and intervention description. We now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI. Detail was added on this in the description of Stage 3 of the intervention design including some short examples of how the intervention manual reflects the co-design as well as how the description of the intervention was changed by this. We have also put in a reference to other study resources with more detail on the intervention and the process.

EDIT 12:
Additionally, the letter would be strengthened with more methodological detail about the stages in the intervention design process to understand how, and to what extent, the aims of co-design were achieved. At present, it is not clear who was involved in each intervention design stage and what methods were used. For example, who conducted the reviews in Stage 1? What (participatory?) methods were used to ‘elicit the routine practices…’ (page 6) in Stage 2 and who synthesized findings from Stage 1 and 2? Who facilitated the meetings in Stage 3 and which team ‘built out’ the intervention (page 6)? This information would also help readers looking towards similar approaches for intervention design.

Response: As per our response to reviewer 1, EDIT 4, we now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI.

EDIT 13:
Finally, in the section ‘The intervention ‘as intended’ versus ‘as implemented’, the authors suggest their approach “…may allow for a better understanding of the ‘active ingredients’ in the implementation process…” (page 10) – my emphasis. This is interesting because it is a departure from the mainstream thinking that focuses on distilling intervention components and disregards how interventions are implemented in situ. This distinction between implementation versus intervention is also touched on somewhat in the section on ‘Intervention ‘fidelity’’, but overall, the nuance of this argument gets lost in the text. The letter would be improved with more clearly articulated discussion of this distinction and what a focus on implementation offers before going into describing how this was achieved.

Response: We address this in the paragraph under the description of CaPSAI on our approach to social accountability.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 31 May 2022

Joanna Cordero, UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP Research), Department of Sexual and Reproductive Health and Research, World Health Organization, Geneva, 1202, Switzerland

31 May 2022

Author Response

Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

REVIEWER 3:

... Continue reading Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

REVIEWER 3:

EDIT 10:

Throughout the letter, the authors emphasize the notion of complexity as contingent. What is missing, however, is an exploration of why accounting for complexity in this way is important. The letter would be strengthened with a short explanation of what gets lost or omitted when taking the mainstream notion of intervention as a linear process and a static collection of materials – or alternatively, what is gained when taking a more contingent and context-specific approach to complexity and design.This could be included in the introduction – for example, revising paragraph 4 to provide more explanation of their view/approach to complexity, rather than references that describe acknowledgements of complexity.

Response: We have added a paragraph on the CaPSAI approach to social accountability and why this is important under the section describing the project.

EDIT 11:
The letter would also be strengthened with a short description of the intervention itself. There is a heading with a few paragraphs describing the aims of the CaPSAI and evaluation; however, there is no description of the actual intervention. What is the social accountability mechanism? Table 1 describes a community scorecard, but no further explanation is given. It is also unclear what aspects of the intervention were decided/set in advance, and what aspects were co-designed. For example, was the community scorecard decided by the implementation team, then the approach to implementation (i.e. the “intervention manual”) co-designed with the implementing partners?

Response: Thank you for Reviewers 1 and 3 on their suggestions for fleshing out the stages of intervention design and intervention description. We now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI. Detail was added on this in the description of Stage 3 of the intervention design including some short examples of how the intervention manual reflects the co-design as well as how the description of the intervention was changed by this. We have also put in a reference to other study resources with more detail on the intervention and the process.

EDIT 12:
Additionally, the letter would be strengthened with more methodological detail about the stages in the intervention design process to understand how, and to what extent, the aims of co-design were achieved. At present, it is not clear who was involved in each intervention design stage and what methods were used. For example, who conducted the reviews in Stage 1? What (participatory?) methods were used to ‘elicit the routine practices…’ (page 6) in Stage 2 and who synthesized findings from Stage 1 and 2? Who facilitated the meetings in Stage 3 and which team ‘built out’ the intervention (page 6)? This information would also help readers looking towards similar approaches for intervention design.

Response: As per our response to reviewer 1, EDIT 4, we now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI.

EDIT 13:
Finally, in the section ‘The intervention ‘as intended’ versus ‘as implemented’, the authors suggest their approach “…may allow for a better understanding of the ‘active ingredients’ in the implementation process…” (page 10) – my emphasis. This is interesting because it is a departure from the mainstream thinking that focuses on distilling intervention components and disregards how interventions are implemented in situ. This distinction between implementation versus intervention is also touched on somewhat in the section on ‘Intervention ‘fidelity’’, but overall, the nuance of this argument gets lost in the text. The letter would be improved with more clearly articulated discussion of this distinction and what a focus on implementation offers before going into describing how this was achieved.

Response: We address this in the paragraph under the description of CaPSAI on our approach to social accountability.
Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

REVIEWER 3:

EDIT 10:

Throughout the letter, the authors emphasize the notion of complexity as contingent. What is missing, however, is an exploration of why accounting for complexity in this way is important. The letter would be strengthened with a short explanation of what gets lost or omitted when taking the mainstream notion of intervention as a linear process and a static collection of materials – or alternatively, what is gained when taking a more contingent and context-specific approach to complexity and design.This could be included in the introduction – for example, revising paragraph 4 to provide more explanation of their view/approach to complexity, rather than references that describe acknowledgements of complexity.

Response: We have added a paragraph on the CaPSAI approach to social accountability and why this is important under the section describing the project.

EDIT 11:
The letter would also be strengthened with a short description of the intervention itself. There is a heading with a few paragraphs describing the aims of the CaPSAI and evaluation; however, there is no description of the actual intervention. What is the social accountability mechanism? Table 1 describes a community scorecard, but no further explanation is given. It is also unclear what aspects of the intervention were decided/set in advance, and what aspects were co-designed. For example, was the community scorecard decided by the implementation team, then the approach to implementation (i.e. the “intervention manual”) co-designed with the implementing partners?

Response: Thank you for Reviewers 1 and 3 on their suggestions for fleshing out the stages of intervention design and intervention description. We now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI. Detail was added on this in the description of Stage 3 of the intervention design including some short examples of how the intervention manual reflects the co-design as well as how the description of the intervention was changed by this. We have also put in a reference to other study resources with more detail on the intervention and the process.

EDIT 12:
Additionally, the letter would be strengthened with more methodological detail about the stages in the intervention design process to understand how, and to what extent, the aims of co-design were achieved. At present, it is not clear who was involved in each intervention design stage and what methods were used. For example, who conducted the reviews in Stage 1? What (participatory?) methods were used to ‘elicit the routine practices…’ (page 6) in Stage 2 and who synthesized findings from Stage 1 and 2? Who facilitated the meetings in Stage 3 and which team ‘built out’ the intervention (page 6)? This information would also help readers looking towards similar approaches for intervention design.

Response: As per our response to reviewer 1, EDIT 4, we now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI.

EDIT 13:
Finally, in the section ‘The intervention ‘as intended’ versus ‘as implemented’, the authors suggest their approach “…may allow for a better understanding of the ‘active ingredients’ in the implementation process…” (page 10) – my emphasis. This is interesting because it is a departure from the mainstream thinking that focuses on distilling intervention components and disregards how interventions are implemented in situ. This distinction between implementation versus intervention is also touched on somewhat in the section on ‘Intervention ‘fidelity’’, but overall, the nuance of this argument gets lost in the text. The letter would be improved with more clearly articulated discussion of this distinction and what a focus on implementation offers before going into describing how this was achieved.

Response: We address this in the paragraph under the description of CaPSAI on our approach to social accountability.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 06 Oct 2021

Jashodhara Dasgupta, SAHAYOG, Lucknow, Uttar Pradesh, India; Feminist Policy Collective, Lucknow, Uttar Pradesh, India

Approved with Reservations

https://doi.org/10.21956/gatesopenres.14495.r31151

Summary of the article - it describes a way of designing the implementation and evaluation of the CaPSAI intervention that used Social Accountability methods to improve service uptake for contraceptives. The way it was designed quite deliberately uses complex intervention evaluation as well as implementation science. The three key aspects of the designing that are highlighted in the paper include co-designing, intervention fidelity and tracking the 'intended vs implemented' processes throughout the eight steps of the programme. It is indicated that a separate article will bring out the outcomes and findings of the CaPSAI programme.

With regard to my comment about inadequate 'reference to differing views and opinions', my suggestions to improve the paper are as below:

Table 1 sets out the steps of the intervention, based on the various programme interventions studied as per Table 3. However the list as indicated in Table 3 is limited to short-duration interventions by international actors/donors/ researchers who set out to make an intervention in an under-resourced setting in the global South. The objective is to improve contraceptive usage, a 'civilizing objective' based on the assumptions that citizen voice and facilitated negotiations with duty bearers will overcome health system challenges, commodity scarcity and cultural/gender norms.

Joshi and Houtzager (2012¹) have usefully made the distinction between 'widgets and watchdogs' and define Social Accountability as 'long-term ongoing political engagement of social actors with the state' as opposed to short term project interventions initiated by researchers or donors that seek to bring about linear processes of change based on the formulaic application of 'steps' of social accountability with insufficient analysis of local context.

The paper would have been improved by stating clearly these limitations and referring to this literature that refers to the profoundly political nature of social accountability processes that seek to confront power asymmetries.

The paper would be strengthened by discussing some of the characteristics of the specific local context: for instance by analyzing whether absence of citizen voice and provider accountability are the only barriers to contraceptive use or whether health system challenges, commodity scarcity and cultural/gender norms may be at the root of low contraceptive uptake in the area.

The paper does not sufficiently refer to literature that acknowledges the non-linear and iterative nature of change processes that work within complex socio-political contexts of public-service provision in most under-resourced health systems of LMICs (refer to Schaaf and Dasgupta, 2019², Balestra et al., 2018³, Dasgupta, 2011⁴ and so forth).

Is the rationale for the Open Letter provided in sufficient detail?

Yes
Does the article adequately reference differing views and opinions?

Partly
Are all factual statements correct, and are statements and arguments made adequately supported by citations?

Yes
Is the Open Letter written in accessible language?

Yes
Where applicable, are recommendations and next steps explained clearly for others to follow?

Not applicable

References

1. Joshi A, Houtzager P: Widgets or Watchdogs?. Public Management Review. 2012; 14 (2): 145-162 Publisher Full Text
2. Schaaf M, Dasgupta J: "Our fear is finished," but nothing changes: efforts of marginalized women to foment state accountability for maternal health care in a context of low state capacity.BMC Public Health. 2019; 19 (1): 732 PubMed Abstract | Publisher Full Text
3. Balestra GL, Dasgupta J, Sandhya YK, Mannell J: Developing political capabilities with Community-Based Monitoring for health accountability: The case of the Mahila Swasthya Adhikar Manch.Glob Public Health. 13 (12): 1853-1864 PubMed Abstract | Publisher Full Text
4. Dasgupta J: Ten years of negotiating rights around maternal health in Uttar Pradesh, India.BMC Int Health Hum Rights. 2011; 11 Suppl 3: S4 PubMed Abstract | Publisher Full Text

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Gender and equity analysis, rights-based approaches, social accountability, maternal health, sexual and reproductive health and rights, choice and adolescent issues, social and political determinants of health, public health systems governance

CITE

Report a concern

Author Response 31 May 2022

Joanna Cordero, UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP Research), Department of Sexual and Reproductive Health and Research, World Health Organization, Geneva, 1202, Switzerland

31 May 2022

Author Response

Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

EDIT 6:
Table 1 ... Continue reading Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

EDIT 6:
Table 1 sets out the steps of the intervention, based on the various programme interventions studied as per Table 3. However the list as indicated in Table 3 is limited to short-duration interventions by international actors/donors/ researchers who set out to make an intervention in an under-resourced setting in the global South. The objective is to improve contraceptive usage, a 'civilizing objective' based on the assumptions that citizen voice and facilitated negotiations with duty bearers will overcome health system challenges, commodity scarcity and cultural/gender norms.

EDIT 7:
Joshi and Houtzager (2012) have usefully made the distinction between 'widgets and watchdogs' and define Social Accountability as 'long-term ongoing political engagement of social actors with the state' as opposed to short term project interventions initiated by researchers or donors that seek to bring about linear processes of change based on the formulaic application of 'steps' of social accountability with insufficient analysis of local context.

The paper would have been improved by stating clearly these limitations and referring to this literature that refers to the profoundly political nature of social accountability processes that seek to confront power asymmetries.

Response: As per our response to Review 1, EDIT 5, we have added a section under the description of CaPSAI to describe our approach to social accountability.

EDIT 8:
The paper would be strengthened by discussing some of the characteristics of the specific local context: for instance by analyzing whether absence of citizen voice and provider accountability are the only barriers to contraceptive use or whether health system challenges, commodity scarcity and cultural/gender norms may be at the root of low contraceptive uptake in the area.

Response: Thank you for this comment. The focus of this paper is describing the methodology of co-desiging the intervention. Other findings from the study, including the context mapping will be reported elsewhere.

EDIT 9:
The paper does not sufficiently refer to literature that acknowledges the non-linear and iterative nature of change processes that work within complex socio-political contexts of public-service provision in most under-resourced health systems of LMICs (refer to Schaaf and Dasgupta, 2019; Balestra et al., 2018, Dasgupta, 2011 and so forth).

Response: Thank you for Reviewers 2 for their suggestions on addressing the issue of nonlinearity of social accountability interventions and for directing us to literature. We have added the paragraph under the description of the CaPSAI Project to describe our approach to social accountability and how this fits within the literature on social accountability and included additional references.
Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

EDIT 6:
Table 1 sets out the steps of the intervention, based on the various programme interventions studied as per Table 3. However the list as indicated in Table 3 is limited to short-duration interventions by international actors/donors/ researchers who set out to make an intervention in an under-resourced setting in the global South. The objective is to improve contraceptive usage, a 'civilizing objective' based on the assumptions that citizen voice and facilitated negotiations with duty bearers will overcome health system challenges, commodity scarcity and cultural/gender norms.

EDIT 7:
Joshi and Houtzager (2012) have usefully made the distinction between 'widgets and watchdogs' and define Social Accountability as 'long-term ongoing political engagement of social actors with the state' as opposed to short term project interventions initiated by researchers or donors that seek to bring about linear processes of change based on the formulaic application of 'steps' of social accountability with insufficient analysis of local context.

The paper would have been improved by stating clearly these limitations and referring to this literature that refers to the profoundly political nature of social accountability processes that seek to confront power asymmetries.

Response: As per our response to Review 1, EDIT 5, we have added a section under the description of CaPSAI to describe our approach to social accountability.

EDIT 8:
The paper would be strengthened by discussing some of the characteristics of the specific local context: for instance by analyzing whether absence of citizen voice and provider accountability are the only barriers to contraceptive use or whether health system challenges, commodity scarcity and cultural/gender norms may be at the root of low contraceptive uptake in the area.

Response: Thank you for this comment. The focus of this paper is describing the methodology of co-desiging the intervention. Other findings from the study, including the context mapping will be reported elsewhere.

EDIT 9:
The paper does not sufficiently refer to literature that acknowledges the non-linear and iterative nature of change processes that work within complex socio-political contexts of public-service provision in most under-resourced health systems of LMICs (refer to Schaaf and Dasgupta, 2019; Balestra et al., 2018, Dasgupta, 2011 and so forth).

Response: Thank you for Reviewers 2 for their suggestions on addressing the issue of nonlinearity of social accountability interventions and for directing us to literature. We have added the paragraph under the description of the CaPSAI Project to describe our approach to social accountability and how this fits within the literature on social accountability and included additional references.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 31 May 2022

Joanna Cordero, UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP Research), Department of Sexual and Reproductive Health and Research, World Health Organization, Geneva, 1202, Switzerland

31 May 2022

Author Response

Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

EDIT 6:
Table 1 ... Continue reading Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

EDIT 6:
Table 1 sets out the steps of the intervention, based on the various programme interventions studied as per Table 3. However the list as indicated in Table 3 is limited to short-duration interventions by international actors/donors/ researchers who set out to make an intervention in an under-resourced setting in the global South. The objective is to improve contraceptive usage, a 'civilizing objective' based on the assumptions that citizen voice and facilitated negotiations with duty bearers will overcome health system challenges, commodity scarcity and cultural/gender norms.

EDIT 7:
Joshi and Houtzager (2012) have usefully made the distinction between 'widgets and watchdogs' and define Social Accountability as 'long-term ongoing political engagement of social actors with the state' as opposed to short term project interventions initiated by researchers or donors that seek to bring about linear processes of change based on the formulaic application of 'steps' of social accountability with insufficient analysis of local context.

The paper would have been improved by stating clearly these limitations and referring to this literature that refers to the profoundly political nature of social accountability processes that seek to confront power asymmetries.

Response: As per our response to Review 1, EDIT 5, we have added a section under the description of CaPSAI to describe our approach to social accountability.

EDIT 8:
The paper would be strengthened by discussing some of the characteristics of the specific local context: for instance by analyzing whether absence of citizen voice and provider accountability are the only barriers to contraceptive use or whether health system challenges, commodity scarcity and cultural/gender norms may be at the root of low contraceptive uptake in the area.

Response: Thank you for this comment. The focus of this paper is describing the methodology of co-desiging the intervention. Other findings from the study, including the context mapping will be reported elsewhere.

EDIT 9:
The paper does not sufficiently refer to literature that acknowledges the non-linear and iterative nature of change processes that work within complex socio-political contexts of public-service provision in most under-resourced health systems of LMICs (refer to Schaaf and Dasgupta, 2019; Balestra et al., 2018, Dasgupta, 2011 and so forth).

Response: Thank you for Reviewers 2 for their suggestions on addressing the issue of nonlinearity of social accountability interventions and for directing us to literature. We have added the paragraph under the description of the CaPSAI Project to describe our approach to social accountability and how this fits within the literature on social accountability and included additional references.
Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

EDIT 6:
Table 1 sets out the steps of the intervention, based on the various programme interventions studied as per Table 3. However the list as indicated in Table 3 is limited to short-duration interventions by international actors/donors/ researchers who set out to make an intervention in an under-resourced setting in the global South. The objective is to improve contraceptive usage, a 'civilizing objective' based on the assumptions that citizen voice and facilitated negotiations with duty bearers will overcome health system challenges, commodity scarcity and cultural/gender norms.

EDIT 7:
Joshi and Houtzager (2012) have usefully made the distinction between 'widgets and watchdogs' and define Social Accountability as 'long-term ongoing political engagement of social actors with the state' as opposed to short term project interventions initiated by researchers or donors that seek to bring about linear processes of change based on the formulaic application of 'steps' of social accountability with insufficient analysis of local context.

The paper would have been improved by stating clearly these limitations and referring to this literature that refers to the profoundly political nature of social accountability processes that seek to confront power asymmetries.

Response: As per our response to Review 1, EDIT 5, we have added a section under the description of CaPSAI to describe our approach to social accountability.

EDIT 8:
The paper would be strengthened by discussing some of the characteristics of the specific local context: for instance by analyzing whether absence of citizen voice and provider accountability are the only barriers to contraceptive use or whether health system challenges, commodity scarcity and cultural/gender norms may be at the root of low contraceptive uptake in the area.

Response: Thank you for this comment. The focus of this paper is describing the methodology of co-desiging the intervention. Other findings from the study, including the context mapping will be reported elsewhere.

EDIT 9:
The paper does not sufficiently refer to literature that acknowledges the non-linear and iterative nature of change processes that work within complex socio-political contexts of public-service provision in most under-resourced health systems of LMICs (refer to Schaaf and Dasgupta, 2019; Balestra et al., 2018, Dasgupta, 2011 and so forth).

Response: Thank you for Reviewers 2 for their suggestions on addressing the issue of nonlinearity of social accountability interventions and for directing us to literature. We have added the paragraph under the description of the CaPSAI Project to describe our approach to social accountability and how this fits within the literature on social accountability and included additional references.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 17 Sep 2021

Stephanie M. Topp, College of Public Health, Medical and Veterinary Sciences, James Cook University, Townsville, Qld, Australia

Approved with Reservations

https://doi.org/10.21956/gatesopenres.14495.r31153

This open letter introduces the CaPSAI (Community and Provider driven Social Accountability Intervention) study which explores the impact of a social accountability process on contraceptive uptake and use in two countries. More specifically, the letter reflects on the design process, which, recognizing that the intervention is a ‘complex social change process’ required specific methodological considerations.

The letter will be of most interest to readers with closely aligned interests in the design of complex interventions tackling social change processes, and seeking to improve the methodological robustness of such studies. Despite being relatively short it is dense with both conceptual and process-related detail.

Allowing for that target audience, I found the letter to contain some important and interesting considerations, in particular, the way ‘conceptual fidelity’ was operationalized as a design feature within the study. This is the first time I personally have seen/read about this being done (at least in such explicit terms) and it provides an excellent example of a methodological approach that can clearly help account for complexity within a quasi-experimental study design. The potential for applying this approach to other areas of research where ‘controlling for context’ is inappropriate seem substantial.

The (preceding) description of *co-design* and later description of *intended vs actual* process tracking felt less novel. Particularly co-design since this is a long-advocated approach to building meaningful and context-specific research (or other endeavors). As the authors note there is now a substantial literature on ‘co-design’ specifically; and there is a long history of participatory approaches including PAR that have incorporated related approaches to research or interventions. Nonetheless, and bearing in mind the likely readership for this letter is one seeking detailed examples of study design, I do see the value in the descriptions provided.

There are several areas in which the letter could be strengthened. First, careful editing of the front two sections (Intro / CaPSAI section) to reduce the duplicative mentions of the study’s aims and approach would be worthwhile. A single, clear and precise articulation of the study’s aims (potentially sub-titled for ease of reference) in the Introduction or early in the following CaPSAI sub-section would suffice. There are currently a number of slightly different descriptions (e.g. Intro, para 1, final sentence / Intro para 4, first sentence / CaPSAI, para 1, ‘Specifically…’ ) which affect the clarity of writing and comprehension.

With regards to the 3 main ‘features’ – consistency of language around the way they are described would also be helpful. Are co-design, conceptual fidelity and accounting for intended vs implemented activities, ‘features’ or ‘considerations’ or ‘approaches’?

The 4^th paragraph of the Introduction could be much clearer; rather than long introductions to the sources of information, a focus on the nature of the knowledge/methodological developments in the field of complexity research with reference to the source only via citations would help the reader focus on the key information and not get lost in the names of institutions and journals.

The Co-design section: I think this would be elevated by having a tighter summary of the 'stages of design' process, in favour of more explicit reflection on how the co-design process was influenced by, and adapted to manage, power dynamics among the many stakeholders, which is an obvious feature of complexity that co-design is, in part, there to address. This might be prompted by responding to the question: how this co-design process differed from a more straightforward, albeit extensive consultation? Notwithstanding that there is the *intention* to recognise local stakeholders' experience as expertise for example, it wasn't clear to me how the co-design stages mitigated the tendency for external experts to dominate decision points. It would be helpful to illustrate examples of adaptations to the study design (away from the original theory of change for example) arising from the co-design process.

Finally, throughout the letter, there remains a not insubstantial tension between the repeated assertion of the way complexity design must account for context specificity and non-linear pathways on the one hand, and the idea that knowledge arising from these studies will be scalable and generalizable on the other hand. One of the reasons I find the description of ‘conceptual fidelity’ compelling, is precisely because it provides an example of how to plot a methodological path through the epistemological tension created by applying a quasi-experimental study design to a research problem which is contextually contingent. But at a broader level throughout this letter, that tension between describing a study design that produces findings which account for complexity, and the expectation that those findings will be scalable or generalizable does not receive much attention; given the earlier observation that the key readership for this letter is likely to be those with a keen methodological interest, closer reflection on that tension by the authors is recommended.

Is the rationale for the Open Letter provided in sufficient detail?

Yes
Does the article adequately reference differing views and opinions?

Yes
Are all factual statements correct, and are statements and arguments made adequately supported by citations?

Yes
Is the Open Letter written in accessible language?

Yes
Where applicable, are recommendations and next steps explained clearly for others to follow?

Partly

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Health systems research with a focus on governance, accountability and trust; qualitative methodologies.

CITE

Report a concern

Author Response 31 May 2022

Joanna Cordero, UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP Research), Department of Sexual and Reproductive Health and Research, World Health Organization, Geneva, 1202, Switzerland

31 May 2022

Author Response

Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

REVIEWER 1:

... Continue reading Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

REVIEWER 1:

EDIT 1:
First, careful editing of the front two sections (Intro / CaPSAI section) to reduce the duplicative mentions of the study’s aims and approach would be worthwhile. A single, clear and precise articulation of the study’s aims (potentially sub-titled for ease of reference) in the Introduction or early in the following CaPSAI sub-section would suffice. There are currently a number of slightly different descriptions (e.g. Intro, para 1, final sentence / Intro para 4, first sentence / CaPSAI, para 1, ‘Specifically…’ ) which affect the clarity of writing and comprehension.

Response: We have removed some of the duplication regarding the aims of the CaPSAI intervention from the introduction.

EDIT 2:
With regards to the 3 main ‘features’ – consistency of language around the way they are described would also be helpful. Are co-design, conceptual fidelity and accounting for intended vs implemented activities, ‘features’ or ‘considerations’ or ‘approaches’?

Response: We have standardised language across the text to refer to the three design components versus features, considerations or approaches.

EDIT 3:
The 4th paragraph of the Introduction could be much clearer; rather than long introductions to the sources of information, a focus on the nature of the knowledge/methodological developments in the field of complexity research with reference to the source only via citations would help the reader focus on the key information and not get lost in the names of institutions and journals.

Response: We have removed references to institutions in the Introduction paragraph as suggested, to improve readability.

EDIT 4:
The Co-design section: I think this would be elevated by having a tighter summary of the 'stages of design' process, in favour of more explicit reflection on how the co-design process was influenced by, and adapted to manage, power dynamics among the many stakeholders, which is an obvious feature of complexity that co-design is, in part, there to address. This might be prompted by responding to the question: how this co-design process differed from a more straightforward, albeit extensive consultation? Notwithstanding that there is the *intention* to recognise local stakeholders' experience as expertise for example, it wasn't clear to me how the co-design stages mitigated the tendency for external experts to dominate decision points. It would be helpful to illustrate examples of adaptations to the study design (away from the original theory of change for example) arising from the co-design process.

Response: Thank you for Reviewers 1 and 3 on their suggestions for fleshing out the stages of intervention design and intervention description. We now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI. Detail was added on this in the description of Stage 3 of the intervention design including some short examples of how the study intervention manual reflects the co-design as well as how the description of the intervention was changed by this. We have also put in a reference to other study resources with more detail on the intervention and the process.

EDIT 5:
Finally, throughout the letter, there remains a not insubstantial tension between the repeated assertion of the way complexity design must account for context specificity and non-linear pathways on the one hand, and the idea that knowledge arising from these studies will be scalable and generalizable on the other hand. One of the reasons I find the description of ‘conceptual fidelity’ compelling, is precisely because it provides an example of how to plot a methodological path through the epistemological tension created by applying a quasi-experimental study design to a research problem which is contextually contingent. But at a broader level throughout this letter, that tension between describing a study design that produces findings which account for complexity, and the expectation that those findings will be scalable or generalizable does not receive much attention; given the earlier observation that the key readership for this letter is likely to be those with a keen methodological interest, closer reflection on that tension by the authors is recommended.

Response: Thank you for Reviewers 1 and 2 for their suggestions on addressing the issue of nonlinearity of social accountability interventions and for directing us to literature. We have added a paragraph under the CaPSAI study description to describe our approach to social accountability. We acknowledge that there are many approaches to social accountability. On one end, it is viewed as a short-term, bounded tactical intervention and at the opposite end of the continuum are approaches that understand social accountability as a process whereby service users, as citizens, demand their legal rights and contribute to social transformation. CaPSAI aims to bridge the two ends of the continuum by codifying the approach of two national groups that have been at the forefront of transparency and accountability for years into short-term interventions.

We also added a few lines addressing generalizability in the conclusion.
Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

REVIEWER 1:

EDIT 1:
First, careful editing of the front two sections (Intro / CaPSAI section) to reduce the duplicative mentions of the study’s aims and approach would be worthwhile. A single, clear and precise articulation of the study’s aims (potentially sub-titled for ease of reference) in the Introduction or early in the following CaPSAI sub-section would suffice. There are currently a number of slightly different descriptions (e.g. Intro, para 1, final sentence / Intro para 4, first sentence / CaPSAI, para 1, ‘Specifically…’ ) which affect the clarity of writing and comprehension.

Response: We have removed some of the duplication regarding the aims of the CaPSAI intervention from the introduction.

EDIT 2:
With regards to the 3 main ‘features’ – consistency of language around the way they are described would also be helpful. Are co-design, conceptual fidelity and accounting for intended vs implemented activities, ‘features’ or ‘considerations’ or ‘approaches’?

Response: We have standardised language across the text to refer to the three design components versus features, considerations or approaches.

EDIT 3:
The 4th paragraph of the Introduction could be much clearer; rather than long introductions to the sources of information, a focus on the nature of the knowledge/methodological developments in the field of complexity research with reference to the source only via citations would help the reader focus on the key information and not get lost in the names of institutions and journals.

Response: We have removed references to institutions in the Introduction paragraph as suggested, to improve readability.

EDIT 4:
The Co-design section: I think this would be elevated by having a tighter summary of the 'stages of design' process, in favour of more explicit reflection on how the co-design process was influenced by, and adapted to manage, power dynamics among the many stakeholders, which is an obvious feature of complexity that co-design is, in part, there to address. This might be prompted by responding to the question: how this co-design process differed from a more straightforward, albeit extensive consultation? Notwithstanding that there is the *intention* to recognise local stakeholders' experience as expertise for example, it wasn't clear to me how the co-design stages mitigated the tendency for external experts to dominate decision points. It would be helpful to illustrate examples of adaptations to the study design (away from the original theory of change for example) arising from the co-design process.

Response: Thank you for Reviewers 1 and 3 on their suggestions for fleshing out the stages of intervention design and intervention description. We now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI. Detail was added on this in the description of Stage 3 of the intervention design including some short examples of how the study intervention manual reflects the co-design as well as how the description of the intervention was changed by this. We have also put in a reference to other study resources with more detail on the intervention and the process.

EDIT 5:
Finally, throughout the letter, there remains a not insubstantial tension between the repeated assertion of the way complexity design must account for context specificity and non-linear pathways on the one hand, and the idea that knowledge arising from these studies will be scalable and generalizable on the other hand. One of the reasons I find the description of ‘conceptual fidelity’ compelling, is precisely because it provides an example of how to plot a methodological path through the epistemological tension created by applying a quasi-experimental study design to a research problem which is contextually contingent. But at a broader level throughout this letter, that tension between describing a study design that produces findings which account for complexity, and the expectation that those findings will be scalable or generalizable does not receive much attention; given the earlier observation that the key readership for this letter is likely to be those with a keen methodological interest, closer reflection on that tension by the authors is recommended.

Response: Thank you for Reviewers 1 and 2 for their suggestions on addressing the issue of nonlinearity of social accountability interventions and for directing us to literature. We have added a paragraph under the CaPSAI study description to describe our approach to social accountability. We acknowledge that there are many approaches to social accountability. On one end, it is viewed as a short-term, bounded tactical intervention and at the opposite end of the continuum are approaches that understand social accountability as a process whereby service users, as citizens, demand their legal rights and contribute to social transformation. CaPSAI aims to bridge the two ends of the continuum by codifying the approach of two national groups that have been at the forefront of transparency and accountability for years into short-term interventions.

We also added a few lines addressing generalizability in the conclusion.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 31 May 2022

Joanna Cordero, UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP Research), Department of Sexual and Reproductive Health and Research, World Health Organization, Geneva, 1202, Switzerland

31 May 2022

Author Response

Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

REVIEWER 1:

... Continue reading Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

REVIEWER 1:

EDIT 1:
First, careful editing of the front two sections (Intro / CaPSAI section) to reduce the duplicative mentions of the study’s aims and approach would be worthwhile. A single, clear and precise articulation of the study’s aims (potentially sub-titled for ease of reference) in the Introduction or early in the following CaPSAI sub-section would suffice. There are currently a number of slightly different descriptions (e.g. Intro, para 1, final sentence / Intro para 4, first sentence / CaPSAI, para 1, ‘Specifically…’ ) which affect the clarity of writing and comprehension.

Response: We have removed some of the duplication regarding the aims of the CaPSAI intervention from the introduction.

EDIT 2:
With regards to the 3 main ‘features’ – consistency of language around the way they are described would also be helpful. Are co-design, conceptual fidelity and accounting for intended vs implemented activities, ‘features’ or ‘considerations’ or ‘approaches’?

Response: We have standardised language across the text to refer to the three design components versus features, considerations or approaches.

EDIT 3:
The 4th paragraph of the Introduction could be much clearer; rather than long introductions to the sources of information, a focus on the nature of the knowledge/methodological developments in the field of complexity research with reference to the source only via citations would help the reader focus on the key information and not get lost in the names of institutions and journals.

Response: We have removed references to institutions in the Introduction paragraph as suggested, to improve readability.

EDIT 4:
The Co-design section: I think this would be elevated by having a tighter summary of the 'stages of design' process, in favour of more explicit reflection on how the co-design process was influenced by, and adapted to manage, power dynamics among the many stakeholders, which is an obvious feature of complexity that co-design is, in part, there to address. This might be prompted by responding to the question: how this co-design process differed from a more straightforward, albeit extensive consultation? Notwithstanding that there is the *intention* to recognise local stakeholders' experience as expertise for example, it wasn't clear to me how the co-design stages mitigated the tendency for external experts to dominate decision points. It would be helpful to illustrate examples of adaptations to the study design (away from the original theory of change for example) arising from the co-design process.

Response: Thank you for Reviewers 1 and 3 on their suggestions for fleshing out the stages of intervention design and intervention description. We now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI. Detail was added on this in the description of Stage 3 of the intervention design including some short examples of how the study intervention manual reflects the co-design as well as how the description of the intervention was changed by this. We have also put in a reference to other study resources with more detail on the intervention and the process.

EDIT 5:
Finally, throughout the letter, there remains a not insubstantial tension between the repeated assertion of the way complexity design must account for context specificity and non-linear pathways on the one hand, and the idea that knowledge arising from these studies will be scalable and generalizable on the other hand. One of the reasons I find the description of ‘conceptual fidelity’ compelling, is precisely because it provides an example of how to plot a methodological path through the epistemological tension created by applying a quasi-experimental study design to a research problem which is contextually contingent. But at a broader level throughout this letter, that tension between describing a study design that produces findings which account for complexity, and the expectation that those findings will be scalable or generalizable does not receive much attention; given the earlier observation that the key readership for this letter is likely to be those with a keen methodological interest, closer reflection on that tension by the authors is recommended.

Response: Thank you for Reviewers 1 and 2 for their suggestions on addressing the issue of nonlinearity of social accountability interventions and for directing us to literature. We have added a paragraph under the CaPSAI study description to describe our approach to social accountability. We acknowledge that there are many approaches to social accountability. On one end, it is viewed as a short-term, bounded tactical intervention and at the opposite end of the continuum are approaches that understand social accountability as a process whereby service users, as citizens, demand their legal rights and contribute to social transformation. CaPSAI aims to bridge the two ends of the continuum by codifying the approach of two national groups that have been at the forefront of transparency and accountability for years into short-term interventions.

We also added a few lines addressing generalizability in the conclusion.
Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

REVIEWER 1:

EDIT 1:
First, careful editing of the front two sections (Intro / CaPSAI section) to reduce the duplicative mentions of the study’s aims and approach would be worthwhile. A single, clear and precise articulation of the study’s aims (potentially sub-titled for ease of reference) in the Introduction or early in the following CaPSAI sub-section would suffice. There are currently a number of slightly different descriptions (e.g. Intro, para 1, final sentence / Intro para 4, first sentence / CaPSAI, para 1, ‘Specifically…’ ) which affect the clarity of writing and comprehension.

Response: We have removed some of the duplication regarding the aims of the CaPSAI intervention from the introduction.

EDIT 2:
With regards to the 3 main ‘features’ – consistency of language around the way they are described would also be helpful. Are co-design, conceptual fidelity and accounting for intended vs implemented activities, ‘features’ or ‘considerations’ or ‘approaches’?

Response: We have standardised language across the text to refer to the three design components versus features, considerations or approaches.

EDIT 3:
The 4th paragraph of the Introduction could be much clearer; rather than long introductions to the sources of information, a focus on the nature of the knowledge/methodological developments in the field of complexity research with reference to the source only via citations would help the reader focus on the key information and not get lost in the names of institutions and journals.

Response: We have removed references to institutions in the Introduction paragraph as suggested, to improve readability.

EDIT 4:
The Co-design section: I think this would be elevated by having a tighter summary of the 'stages of design' process, in favour of more explicit reflection on how the co-design process was influenced by, and adapted to manage, power dynamics among the many stakeholders, which is an obvious feature of complexity that co-design is, in part, there to address. This might be prompted by responding to the question: how this co-design process differed from a more straightforward, albeit extensive consultation? Notwithstanding that there is the *intention* to recognise local stakeholders' experience as expertise for example, it wasn't clear to me how the co-design stages mitigated the tendency for external experts to dominate decision points. It would be helpful to illustrate examples of adaptations to the study design (away from the original theory of change for example) arising from the co-design process.

Response: Thank you for Reviewers 1 and 3 on their suggestions for fleshing out the stages of intervention design and intervention description. We now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI. Detail was added on this in the description of Stage 3 of the intervention design including some short examples of how the study intervention manual reflects the co-design as well as how the description of the intervention was changed by this. We have also put in a reference to other study resources with more detail on the intervention and the process.

EDIT 5:
Finally, throughout the letter, there remains a not insubstantial tension between the repeated assertion of the way complexity design must account for context specificity and non-linear pathways on the one hand, and the idea that knowledge arising from these studies will be scalable and generalizable on the other hand. One of the reasons I find the description of ‘conceptual fidelity’ compelling, is precisely because it provides an example of how to plot a methodological path through the epistemological tension created by applying a quasi-experimental study design to a research problem which is contextually contingent. But at a broader level throughout this letter, that tension between describing a study design that produces findings which account for complexity, and the expectation that those findings will be scalable or generalizable does not receive much attention; given the earlier observation that the key readership for this letter is likely to be those with a keen methodological interest, closer reflection on that tension by the authors is recommended.

Response: Thank you for Reviewers 1 and 2 for their suggestions on addressing the issue of nonlinearity of social accountability interventions and for directing us to literature. We have added a paragraph under the CaPSAI study description to describe our approach to social accountability. We acknowledge that there are many approaches to social accountability. On one end, it is viewed as a short-term, bounded tactical intervention and at the opposite end of the continuum are approaches that understand social accountability as a process whereby service users, as citizens, demand their legal rights and contribute to social transformation. CaPSAI aims to bridge the two ends of the continuum by codifying the approach of two national groups that have been at the forefront of transparency and accountability for years into short-term interventions.

We also added a few lines addressing generalizability in the conclusion.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 22 Jul 2021

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2	3
Version 2 (revision) 23 May 22		read	read
Version 1 22 Jul 21	read	read	read

Stephanie M. Topp, James Cook University, Townsville, Australia
Jashodhara Dasgupta, SAHAYOG, Lucknow, India; Feminist Policy Collective, Lucknow, India
Deborah D. Diliberto, McMaster University, Hamilton, Canada

Comments on this article

All Comments(0)

Add a comment

Back to all reports

Reviewer Report

8 Views

04 Aug 2022 | for Version 2

Deborah D. Diliberto, Global Health, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada

8 Views Cite this report Responses(0)

Approved

I have read the authors' response and I am satisfied with their revisions. I have no further comments to make.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Design and evaluation of complex health and social interventions in low resource settings; interdisciplinary and mixed methods approaches.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

31 Views

25 May 2022 | for Version 2

Jashodhara Dasgupta, SAHAYOG, Lucknow, Uttar Pradesh, India; Feminist Policy Collective, Lucknow, Uttar Pradesh, India

31 Views Cite this report Responses(1)

Approved With Reservations

I appreciate the attention paid to the earlier comments and the added materials and revisions in the text of the paper. Nonetheless, I regret that a point I have made in the earlier review has apparently not been acknowledged.

The revised paper still does not mention how or when the context analysis was undertaken even while it mentions that the process of ‘co-designing’ recognized the ‘wealth of experience’ and tried to ‘respond to the local context’.
Without analysis of the context as a preliminary step in designing any SRHR intervention, there remains a risk that the strategy adopted may not be the optimal one to address the issue.

For instance, it is not mentioned whether the current barriers to low usage of modern contraceptives were analyzed at any stage by the team: were they cultural/gender norms, or supply-side constraints/community scarcity or maybe there were some health system challenges in terms of low accountability to the end-users of SRHR services.
Without analysis of the barriers, or examination of possible alternative strategies to address the issue and its root causes, there is an implicit decision made that social accountability will best serve the purpose towards attaining the intended SRHR outcomes. Yet the paper does not clarify the reasons why this decision was made, or at what stage it was assumed that the social accountability process would be the most effective answer to low usage of modern contraceptives.

This fundamental limitation is not acknowledged anywhere, which leads this paper to promote the rather risky approach that context analysis, understanding the root causes and considering various alternative/complementary strategies are not really needed. The paper may tacitly indicate that merely following the eight steps (with attention to co-design, fidelity and intention-implementation comparisons) is sufficient to ensure outcomes.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Gender and equity analysis, rights-based approaches, social accountability, maternal health, sexual and reproductive health and rights, choice and adolescent issues, social and political determinants of health, public health systems governance

Respond to this report

Responses (1)

Author Response

26 Jul 2022

Joanna Cordero, UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP Research), Department of Sexual and Reproductive Health and Research, World Health Organization, Geneva, 1202, Switzerland

Response: Apologies for not responding to your comments in the earlier response to comments. The CaPSAI Project was developed in line with the World Health Organization guideline: “Ensuring human rights in the provision of contraceptive information and services”, which identified nine key health and human rights standards that need to be strengthened: non-discrimination, availability, accessibility, acceptability, Quality of Care (QoC), participation, informed decision-making and choice, privacy/confidentiality and accountability (WHO 2014). The project implemented tested social accountability processes to promote community monitoring and social accountability of contraceptive programs. To do this, we identified national civil society organization using social accountability and were already working or wanting to work on sexual and reproductive health.

We want to clarify that two types of contextual analysis were undertaken as part of the larger study and project. The first was led by the research team at the start of each project year to better understand the current social accountability and SRHR programmes that were underway in the locations where the project was taking place. The second was undertaken by the national CSOs facilitating the social accountability activities with the community, they undertook an analysis of the SRHR context locally as part of the project preparation work and then this was discussed at the start of the community engagement process. It was felt that the national partners supporting the social accountability processes were best placed to undertake the analysis of the local and wider barriers to SRHR as the processes have to be responsive to local contexts. Though many of these barriers are beyond the reach of social accountability processes (which can often be locally bounded if they are not linked up to wider mobilization efforts) and as you point out, this can limit accountability efforts, it was the national partners in conjunction with their community partners who decided what was the most appropriate barriers to tackle based on their analysis.

Thank you for your careful consideration of our manuscript.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

13 Views

08 Oct 2021 | for Version 1

Deborah D. Diliberto, Global Health, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada

13 Views Cite this report Responses(1)

Approved With Reservations

Is the rationale for the Open Letter provided in sufficient detail?

Yes
Does the article adequately reference differing views and opinions?

Yes
Are all factual statements correct, and are statements and arguments made adequately supported by citations?

Yes
Is the Open Letter written in accessible language?

Yes
Where applicable, are recommendations and next steps explained clearly for others to follow?

Partly

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Design and evaluation of complex health and social interventions in low resource settings; interdisciplinary and mixed methods approaches.

Respond to this report

Responses (1)

Author Response

31 May 2022

Joanna Cordero, UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP Research), Department of Sexual and Reproductive Health and Research, World Health Organization, Geneva, 1202, Switzerland

Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

REVIEWER 3:

EDIT 10:

Throughout the letter, the authors emphasize the notion of complexity as contingent. What is missing, however, is an exploration of why accounting for complexity in this way is important. The letter would be strengthened with a short explanation of what gets lost or omitted when taking the mainstream notion of intervention as a linear process and a static collection of materials – or alternatively, what is gained when taking a more contingent and context-specific approach to complexity and design.This could be included in the introduction – for example, revising paragraph 4 to provide more explanation of their view/approach to complexity, rather than references that describe acknowledgements of complexity.

Response: We have added a paragraph on the CaPSAI approach to social accountability and why this is important under the section describing the project.

EDIT 11:
The letter would also be strengthened with a short description of the intervention itself. There is a heading with a few paragraphs describing the aims of the CaPSAI and evaluation; however, there is no description of the actual intervention. What is the social accountability mechanism? Table 1 describes a community scorecard, but no further explanation is given. It is also unclear what aspects of the intervention were decided/set in advance, and what aspects were co-designed. For example, was the community scorecard decided by the implementation team, then the approach to implementation (i.e. the “intervention manual”) co-designed with the implementing partners?

Response: Thank you for Reviewers 1 and 3 on their suggestions for fleshing out the stages of intervention design and intervention description. We now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI. Detail was added on this in the description of Stage 3 of the intervention design including some short examples of how the intervention manual reflects the co-design as well as how the description of the intervention was changed by this. We have also put in a reference to other study resources with more detail on the intervention and the process.

EDIT 12:
Additionally, the letter would be strengthened with more methodological detail about the stages in the intervention design process to understand how, and to what extent, the aims of co-design were achieved. At present, it is not clear who was involved in each intervention design stage and what methods were used. For example, who conducted the reviews in Stage 1? What (participatory?) methods were used to ‘elicit the routine practices…’ (page 6) in Stage 2 and who synthesized findings from Stage 1 and 2? Who facilitated the meetings in Stage 3 and which team ‘built out’ the intervention (page 6)? This information would also help readers looking towards similar approaches for intervention design.

Response: As per our response to reviewer 1, EDIT 4, we now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI.

EDIT 13:
Finally, in the section ‘The intervention ‘as intended’ versus ‘as implemented’, the authors suggest their approach “…may allow for a better understanding of the ‘active ingredients’ in the implementation process…” (page 10) – my emphasis. This is interesting because it is a departure from the mainstream thinking that focuses on distilling intervention components and disregards how interventions are implemented in situ. This distinction between implementation versus intervention is also touched on somewhat in the section on ‘Intervention ‘fidelity’’, but overall, the nuance of this argument gets lost in the text. The letter would be improved with more clearly articulated discussion of this distinction and what a focus on implementation offers before going into describing how this was achieved.

Response: We address this in the paragraph under the description of CaPSAI on our approach to social accountability.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

17 Views

06 Oct 2021 | for Version 1

Jashodhara Dasgupta, SAHAYOG, Lucknow, Uttar Pradesh, India; Feminist Policy Collective, Lucknow, Uttar Pradesh, India

17 Views Cite this report Responses(1)

Approved With Reservations

Is the rationale for the Open Letter provided in sufficient detail?

Yes
Does the article adequately reference differing views and opinions?

Partly
Are all factual statements correct, and are statements and arguments made adequately supported by citations?

Yes
Is the Open Letter written in accessible language?

Yes
Where applicable, are recommendations and next steps explained clearly for others to follow?

Not applicable

References

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Respond to this report

Responses (1)

Author Response

31 May 2022

Joanna Cordero, UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP Research), Department of Sexual and Reproductive Health and Research, World Health Organization, Geneva, 1202, Switzerland

Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

EDIT 6:
Table 1 sets out the steps of the intervention, based on the various programme interventions studied as per Table 3. However the list as indicated in Table 3 is limited to short-duration interventions by international actors/donors/ researchers who set out to make an intervention in an under-resourced setting in the global South. The objective is to improve contraceptive usage, a 'civilizing objective' based on the assumptions that citizen voice and facilitated negotiations with duty bearers will overcome health system challenges, commodity scarcity and cultural/gender norms.

EDIT 7:
Joshi and Houtzager (2012) have usefully made the distinction between 'widgets and watchdogs' and define Social Accountability as 'long-term ongoing political engagement of social actors with the state' as opposed to short term project interventions initiated by researchers or donors that seek to bring about linear processes of change based on the formulaic application of 'steps' of social accountability with insufficient analysis of local context.

The paper would have been improved by stating clearly these limitations and referring to this literature that refers to the profoundly political nature of social accountability processes that seek to confront power asymmetries.

Response: As per our response to Review 1, EDIT 5, we have added a section under the description of CaPSAI to describe our approach to social accountability.

EDIT 8:
The paper would be strengthened by discussing some of the characteristics of the specific local context: for instance by analyzing whether absence of citizen voice and provider accountability are the only barriers to contraceptive use or whether health system challenges, commodity scarcity and cultural/gender norms may be at the root of low contraceptive uptake in the area.

Response: Thank you for this comment. The focus of this paper is describing the methodology of co-desiging the intervention. Other findings from the study, including the context mapping will be reported elsewhere.

EDIT 9:
The paper does not sufficiently refer to literature that acknowledges the non-linear and iterative nature of change processes that work within complex socio-political contexts of public-service provision in most under-resourced health systems of LMICs (refer to Schaaf and Dasgupta, 2019; Balestra et al., 2018, Dasgupta, 2011 and so forth).

Response: Thank you for Reviewers 2 for their suggestions on addressing the issue of nonlinearity of social accountability interventions and for directing us to literature. We have added the paragraph under the description of the CaPSAI Project to describe our approach to social accountability and how this fits within the literature on social accountability and included additional references.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

24 Views

17 Sep 2021 | for Version 1

Stephanie M. Topp, College of Public Health, Medical and Veterinary Sciences, James Cook University, Townsville, Qld, Australia

24 Views Cite this report Responses(1)

Approved With Reservations

Is the rationale for the Open Letter provided in sufficient detail?

Yes
Does the article adequately reference differing views and opinions?

Yes
Are all factual statements correct, and are statements and arguments made adequately supported by citations?

Yes
Is the Open Letter written in accessible language?

Yes
Where applicable, are recommendations and next steps explained clearly for others to follow?

Partly

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Health systems research with a focus on governance, accountability and trust; qualitative methodologies.

Respond to this report

Responses (1)

Author Response

31 May 2022

Joanna Cordero, UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP Research), Department of Sexual and Reproductive Health and Research, World Health Organization, Geneva, 1202, Switzerland

Thank you for your comments. We found them helpful for improving our paper and have made a number of changes. These changes are listed below.

REVIEWER 1:

EDIT 1:
First, careful editing of the front two sections (Intro / CaPSAI section) to reduce the duplicative mentions of the study’s aims and approach would be worthwhile. A single, clear and precise articulation of the study’s aims (potentially sub-titled for ease of reference) in the Introduction or early in the following CaPSAI sub-section would suffice. There are currently a number of slightly different descriptions (e.g. Intro, para 1, final sentence / Intro para 4, first sentence / CaPSAI, para 1, ‘Specifically…’ ) which affect the clarity of writing and comprehension.

Response: We have removed some of the duplication regarding the aims of the CaPSAI intervention from the introduction.

EDIT 2:
With regards to the 3 main ‘features’ – consistency of language around the way they are described would also be helpful. Are co-design, conceptual fidelity and accounting for intended vs implemented activities, ‘features’ or ‘considerations’ or ‘approaches’?

Response: We have standardised language across the text to refer to the three design components versus features, considerations or approaches.

EDIT 3:
The 4th paragraph of the Introduction could be much clearer; rather than long introductions to the sources of information, a focus on the nature of the knowledge/methodological developments in the field of complexity research with reference to the source only via citations would help the reader focus on the key information and not get lost in the names of institutions and journals.

Response: We have removed references to institutions in the Introduction paragraph as suggested, to improve readability.

EDIT 4:
The Co-design section: I think this would be elevated by having a tighter summary of the 'stages of design' process, in favour of more explicit reflection on how the co-design process was influenced by, and adapted to manage, power dynamics among the many stakeholders, which is an obvious feature of complexity that co-design is, in part, there to address. This might be prompted by responding to the question: how this co-design process differed from a more straightforward, albeit extensive consultation? Notwithstanding that there is the *intention* to recognise local stakeholders' experience as expertise for example, it wasn't clear to me how the co-design stages mitigated the tendency for external experts to dominate decision points. It would be helpful to illustrate examples of adaptations to the study design (away from the original theory of change for example) arising from the co-design process.

Response: Thank you for Reviewers 1 and 3 on their suggestions for fleshing out the stages of intervention design and intervention description. We now refer to the section on Co-design as ‘aspects of co-design’ and have provided more detail where possible on how these aspects of co-design were applied in CaPSAI. Detail was added on this in the description of Stage 3 of the intervention design including some short examples of how the study intervention manual reflects the co-design as well as how the description of the intervention was changed by this. We have also put in a reference to other study resources with more detail on the intervention and the process.

EDIT 5:
Finally, throughout the letter, there remains a not insubstantial tension between the repeated assertion of the way complexity design must account for context specificity and non-linear pathways on the one hand, and the idea that knowledge arising from these studies will be scalable and generalizable on the other hand. One of the reasons I find the description of ‘conceptual fidelity’ compelling, is precisely because it provides an example of how to plot a methodological path through the epistemological tension created by applying a quasi-experimental study design to a research problem which is contextually contingent. But at a broader level throughout this letter, that tension between describing a study design that produces findings which account for complexity, and the expectation that those findings will be scalable or generalizable does not receive much attention; given the earlier observation that the key readership for this letter is likely to be those with a keen methodological interest, closer reflection on that tension by the authors is recommended.

Response: Thank you for Reviewers 1 and 2 for their suggestions on addressing the issue of nonlinearity of social accountability interventions and for directing us to literature. We have added a paragraph under the CaPSAI study description to describe our approach to social accountability. We acknowledge that there are many approaches to social accountability. On one end, it is viewed as a short-term, bounded tactical intervention and at the opposite end of the continuum are approaches that understand social accountability as a process whereby service users, as citizens, demand their legal rights and contribute to social transformation. CaPSAI aims to bridge the two ends of the continuum by codifying the approach of two national groups that have been at the forefront of transparency and accountability for years into short-term interventions.

We also added a few lines addressing generalizability in the conclusion.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] Boydell V, Neema S, Wright K, et al.: Closing the Gap between People and Programs: Lessons from Implementation of Social Accountability for Family Planning and Reproductive Health in Uganda. Afr J Reprod Health. 2018; 22(1): 73–84. PubMed Abstract

[2] Boydell V, Schaaf M, George A, et al.: Building a transformative agenda for accountability in SRHR: lessons learned from SRHR and accountability literatures. Sex Reprod Health Matters. 2019; 27(2): 1622357. PubMed Abstract | Publisher Full Text | Free Full Text

[3] Cordero JP, Steyn PS, Gichangi P, et al.: Community and provider perspectives on addressing unmet need for contraception: key findings from a formative phase research in Kenya, South Africa and Zambia (2015-2016). Afr J Reprod Health. 2019; 23(3): 106–19. https://www.ajol.info/index.php/ajrh/article/view/191220. PubMed Abstract

[4] Craig P, Dieppe P, Macintyre S, et al.: Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008; 337: a1655. PubMed Abstract | Publisher Full Text

[5] Craig P, Dieppe P, Macintyre S, et al.: Developing and evaluating complex interventions: the new Medical Research Council guidance. Int J Nurs Stud. 2013; 50(5): 587–92. PubMed Abstract | Publisher Full Text

[6] Donetto S, Pierri P, Tsianakas V, et al.: Experience-based Co-design and Healthcare Improvement: Realizing Participatory Design in the Public Sector. Des J. 2015; 18(2): 227–248. Publisher Full Text

[7] Fox JA: Social Accountability: What Does the Evidence Really Say? World Dev. 2015; 72: 346–361. Publisher Full Text

[8] Goodyear-Smith F, Jackson C, Greenhalgh T: Co-design and implementation research: challenges and solutions for ethics committees. BMC Med Ethics. 2015; 16: 78. PubMed Abstract | Publisher Full Text | Free Full Text

[9] Greenhalgh T: Role of routines in collaborative work in healthcare organisations. BMJ. 2008; 337: a2448. (Published 17 November 2008). PubMed Abstract | Publisher Full Text

[10] Greenhalgh T, Papoutsi C: Studying complexity in health services research: desperately seeking an overdue paradigm shift. BMC Med. 2018; 16(1): 95. PubMed Abstract | Publisher Full Text | Free Full Text

[11] Gullo S, Galavotti C, Kuhlmann AS, et al.: Effects of a social accountability approach, CARE's Community Score Card, on reproductive health-related outcomes in Malawi: A cluster-randomized controlled evaluation. PLoS One. 2017; 12(2): e0171316. PubMed Abstract | Publisher Full Text | Free Full Text

[12] Hawe P, Shiell A, Riley T: Complex interventions: how “out of control” can a randomised controlled trial be? BMJ. 2004; 328(7455): 1561–3. PubMed Abstract | Publisher Full Text | Free Full Text

[13] Joshi A: Legal Empowerment and Social Accountability: Complementary Strategies Toward Rights-based Development in Health? World Dev. 2017; 99: 160–72. Publisher Full Text

[14] Kruk ME, Gage AD, Arsenault C, et al.: High-quality health systems in the Sustainable Development Goals era: time for a revolution. Lancet Glob Health. 2018; 6(11): e1196–e1252. PubMed Abstract | Publisher Full Text | Free Full Text

[15] McMullen H, Griffiths C, Leber W, et al.: Explaining high and low performers in complex intervention trials: a new model based on diffusion of innovations theory. Trials. 2015; 16: 242. PubMed Abstract | Publisher Full Text | Free Full Text

[16] Moore GF, Audrey S, Barker M, et al.: Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015; 350: h1258. PubMed Abstract | Publisher Full Text

[17] Palmer VJ, Piper D, Richard L, et al.: Balancing Opposing Forces—A Nested Process Evaluation Study Protocol for a Stepped Wedge Designed Cluster Randomized Controlled Trial of an Experience Based Codesign Intervention: The CORE Study. Int J Qual Methods. 2016; 15(1). Publisher Full Text

[18] Pérez D, van der Stuyft P, Zabala MC, et al.: A modified theoretical framework to assess implementation fidelity of adaptive public health interventions. Implement Sci. 2016; 11(1): 91. PubMed Abstract | Publisher Full Text | Free Full Text

[19] Portela A, Tunçalp Ö, Norris SL: Taking a complexity perspective when developing public health guidelines. Bull World Health Organ. World Health Organization. 2019; 97(‎4)‎: 247–247A. PubMed Abstract | Publisher Full Text | Free Full Text

[20] Schaaf M, Topp SM, Ngulube M: From favours to entitlements: community voice and action and health service quality in Zambia. Health policy and planning. 2017; 32(6): 847–859. PubMed Abstract | Publisher Full Text | Free Full Text

[21] Slattery P, Saeri AK, Bragge P: Research co-design in health: a rapid overview of reviews. Health Res Policy Syst. 2020; 18(1): 17. PubMed Abstract | Publisher Full Text | Free Full Text

[22] Steyn PS, Boydell V, Cordero JP, et al.: Rationale and design of a complex intervention measuring the impact and processes of social accountability applied to contraceptive programming: CaPSAI Project [version 2; peer review: 2 approved]. Gates Open Res. 2020; 4: 26. PubMed Abstract | Publisher Full Text | Free Full Text

[23] Steyn PS, Cordero JP, Gichangi P, et al.: Participatory approaches involving community and healthcare providers in family planning/contraceptive information and service provision: a scoping review. Reprod Health. 2016; 13(1): 88. PubMed Abstract | Publisher Full Text | Free Full Text

[24] van Belle S, Boydell V, George AS, et al.: Broadening understanding of accountability ecosystems in sexual and reproductive health and rights: A systematic review. PLoS One. 2018; 13(5): e0196788. PubMed Abstract | Publisher Full Text | Free Full Text

Accounting for complexity – Intervention design in the context of studying social accountability for reproductive health

Abstract

Keywords

Abbreviations

Disclaimer

Introduction

The Community and Provider driven Social Accountability Intervention (CaPSAI)

Figure 1. CaPSAI Theory of Change (This figure has been reproduced with permission from Steyn et al., 2020).

Table 1. Standard steps in the CaPSAI social accountability process.

Key design features in enabling and accounting for local adaptation

Table 2. Accounting for complexity in the Community and Provider driven Social Accountability Intervention (CaPSAI) study design (This table has been reproduced with permission from Steyn et al., 2020).

Co-design

Stages in the intervention design

Table 3. Toolkits reviewed to develop the intervention.

Intervention ‘fidelity’

The intervention ‘as intended’ versus ‘as implemented’

Conclusion

Data availability

Acknowledgments

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Are you a Gates-funded researcher?

Thank you!

Accounting for complexity – Intervention design in the context of studying social accountability for reproductive health

Abstract

Keywords

Abbreviations

Disclaimer

Introduction

The Community and Provider driven Social Accountability Intervention (CaPSAI)

Figure 1. CaPSAI Theory of Change (This figure has been reproduced with permission from Steyn et al., 2020).

Table 1. Standard steps in the CaPSAI social accountability process.

Key design features in enabling and accounting for local adaptation

Table 2. Accounting for complexity in the Community and Provider driven Social Accountability Intervention (CaPSAI) study design (This table has been reproduced with permission from Steyn et al., 2020).

Co-design

Stages in the intervention design

Table 3. Toolkits reviewed to develop the intervention.

Intervention ‘fidelity’

The intervention ‘as intended’ versus ‘as implemented’

Conclusion

Data availability

Acknowledgments

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Competing Interests Policy

Stay Updated

Are you a Gates-funded researcher?

Thank you!