Ripple effects of research capacity strengthening: a study of the effects of a project to support test facilities in three African countries towards Good Laboratory Practice certification

Ethical statement

Ethical approval to conduct this research study was obtained from the Liverpool School of Tropical Medicine Research Ethics Committee (approval number 18-041), the National Institute for Medical Research Tanzania (ref NIMR/HQ/R.8c/Vol./I/554), and the Centre Suisse de Recherches Scientifiques en Côte d'Ivoire Institute Review Board (ref 19-549). Institutions taking part remotely (i.e., interviews with members of research staff via Skype/email) provided an institutional approval document in lieu of in-country REC approval, as per point 3c of the LSTM’s Approval Processes for Network and Capacity Strengthening Studies.

Participants were informed about the research using participant information sheets¹⁸. Written consent was obtained from each participant prior to undertaking an interview.

Setting

Seven insecticide test facilities engaged in the testing of novel vector control products for the purpose of supporting malaria control programmes have received investment and support from IVCC to achieve GLP certification. Of these seven facilities, five have been included in this study, encompassing test facilities in Tanzania, Côte D’Ivoire and Burkina Faso. These five test facilities encompass a diverse array of contexts. PAMVERC-KCMUCo, Tanzania, provides crucial information on how GLP certification can be achieved, being the first insecticide testing facility in Africa to do so. Comparison between East and West African contexts was facilitated through inclusion of Centre Suisse de Recherches Scientifques en Côte D’Ivoire (CSRS) and Institut de Recherche en Sciences de la Santé (IRSS), Burkina Faso. Comparison between government and non-government test facilities was facilitated through inclusion of National institute For Medical Research (NIMR), Amani Centre, Tanzania and Ifakara Health Institute (IHI), Tanzania. These contrasting test facilities enhanced our ability to identify both direct and indirect effects of investments in developing a QMS. Generalisability of findings was assessed through using these facilities to compare effects of investment in QMS in a diverse range of contexts, including different national policy contexts and government/non-government supported test facilities.

Sampling

To capture the views of individuals who had exposure to the GLP certification process at all levels of these test facilities, a maximum-variation purposive sampling strategy was used¹⁹. Sampling included those who hold key roles within a test facility, as determined by a case-study conducted on the first test facility to achieve GLP certification, KCMUCo-PAMVERC²⁰, as well as multiple representatives at each organisational level of the facility. This allowed triangulation between different data sources to determine the trustworthiness of findings. Test facility organograms were used to identify relevant participants, with guidance from stakeholders at IVCC and GLP project managers.

Data collection and analysis

Semi-structured interviews were conducted with individual staff members involved in the GLP process in three test facilities; KCMUCo-PAMVERC, NIMR Amani Centre, and CSRS. The interview topic guide¹⁸ was developed based on previous studies of laboratory capacity strengthening⁸, with additional questions derived from findings from a case study of the GLP certification process at PAMVERC-KCMUCo²⁰. One overarching question was specifically related to perceived effects of the project. However, due to the semi-structured nature of the interview, interview participants reflected on the effect of the project throughout the interview. Specific questions asked from the topic guide were matched to the roles and responsibilities of the interviewee. Interviews were audio-recorded and transcribed in full. All interviews were conducted in person, in a private room or office, by two researchers, one of whom had a technical understanding of GLP requirements in insecticide testing facilities and the other having systems evaluation experience. Whilst the lead researcher spoke basic French and Swahili, for interview participants who preferred to undertake the interview in a language other than English, a trusted colleague or research student sat in on the interview to aid with translation.

A combination of email and remote video-call interviews were conducted with individual staff members involved in the GLP process at two other test facilities, IRSS and IHI. This was necessitated by restrictions on travel and reduced working hours following the COVID-19 pandemic, which resulted in significant disruption from March 2019. The overarching questions asked in these interviews were retained from the semi-structured interview guide used for in-person interviews. Follow-up questions, where relevant, were conducted via video-call or email.

A framework analysis²¹ was used to identify themes emerging from the interview transcripts following the five-step process of familiarization, identification of thematic framework, indexing, charting and mapping/interpretation. The framework identified was the Research Capacity Strengthening evaluation framework developed by Khisa et al., from African Population and Health Research Center, Nairobi, Kenya and Centre for Capacity Research, Liverpool School of Tropical Medicine, UK²². This framework delineates the identified and envisioned effect of research capacity strengthening initiatives at the individual, institutional, and national/international level, developed from a review of the research capacity strengthening literature and refined in consultation with research capacity strengthening funders, implementers, managers and evaluators (Table 1). Following familiarisation with the interview data, further themes were identified and incorporated into the framework, while retaining the individual, institution, and societal level structure. All interview transcripts were indexed using NVivo software version 11 (QSR International).

Institutional level effects

At the institutional level, the GLP quality management system, infrastructural improvements of laboratories and offices, development of clearer and more effective organisational structures, more staff employed, and the transfer of GLP-standard practices to other studies were all identified as research capacity strengthening effects resulting from the GLP project.

The development of a GLP-compliant quality management system and, at some test facilities, the achievement of GLP certification following inspection by the GLP monitoring authority SANAS, is a clear outcome of the work undertaken through the IVCC project. Of the seven test facilities included in the wider project, two have achieved GLP certification to date, and four have submitted their application for GLP certification to SANAS. As a result of GLP certification, these two test facilities were able to deliver national/internationally competitive research, with data meeting international standards. This effect extended also to non-GLP studies conducted at these test facilities, as best practice from GLP studies was applied also to non-GLP studies by both scientists involved in the GLP project and other scientists within the institution, particularly with respect to study documentation and use of Standard Operating Procedures (SOPs). Thus, the overall quality of data generated at these test facilities was enhanced. (Quotes: INS3) Test facilities also identified broader effects on working practices, resulting from the implementation of GLP standards. In particular, increased structure in working practices resulting in a range of benefits including cost savings on reagents, more effective problem solving, and better organisation of work throughout the test facility. (Quotes: INS5)

Career pathways were enhanced by strengthening the processes, policies, and documentation that surrounded organisational structure and human resources. Development of clear organisational structures facilitated communication between individuals in different departments and at different levels within the test facility. This was supported through development and implementation of key SOPs for regular, documented human resource support including appraisals and Curriculum Vitae review. Together, these had an additional effect on individuals’ sense of place and therefore, sense of value within the test facility. In some test facilities, new structures were put in place for requesting training for career development, and staff were adequately empowered to take up these opportunities. Across test facilities, but particularly in those that had achieved GLP certification, there were more job opportunities at the institution, with more studies an investment attracted to the test facility. (Quotes: INS1)

In-house training programmes were developed and delivered across test facilities including general training in GLP awareness, Quality Assurance, training in SOPs, Health and Safety/Fire training, archiving training, leadership training, and computer system validation and usage. Training programmes were often developed by test facility staff following attendance at externally delivered training courses. Implementation of training was overseen by staff in a range of roles, as a result of the additional responsibilities being taken on by staff at all levels. Test facility management noted that MSc and PhD students from institutions attached to their test facility had had the opportunity to train in a GLP environment as a result of the developed quality management systems. (Quotes: INS2)

Infrastructural improvements at test facilities enhanced the research environment including laboratory, office and shared spaces. Areas of test facilities that were built from scratch or were refurbished included: insecticide testing laboratories, molecular laboratories, insecticide spray rooms, bed net washing areas, insectaries and animal houses. Enhancements included installation of new equipment, improved separation between resistant and non-resistant mosquito strains in insectaries, construction of new facilities to allow new test types (for example, net washing facilities to allow testing of insecticide-treated nets), increased space within existing laboratories, and enhancements to working conditions (e.g. new benching, stools, and wipe-clean tiled surfaces). Installation of new equipment, such as PCR machines, facilitated establishment of new assays and meant that testing of samples could be conducted in-house, reducing the time to obtaining results. Non-laboratory facilities built or refurbished included office spaces, communal break and training areas, facility archives and computer server rooms. For both laboratory and non-laboratory facilities, this enhanced the working environment linked to individuals’ motivation, job satisfaction and pride in their jobs. (Quotes: INS4a)

The research environment was also strengthened through improvements in the procurement processes in some test facilities, and to IT infrastructure across all test facilities. Streamlined procurement processes included the implementation of quality management system practices initiated by the GLP project, in particular in the widespread use of SOPs. This simplified processes and made transfer of work responsibilities when colleagues were absent more seamless. IT infrastructure improvements were relevant across GLP and non-GLP studies, improving processes for accessing and storing study data, managing results in preparation for scientific reports and publications, and improving communication between staff within the test facility through, for example, more widespread use of email and installation of internal telephone systems. (Quotes: INS4b)

Individual level effects

Whilst the project was focused at the institution level, secondary effects were identified at the individual level. These effects included extensive training, strengthening of career prospects, furtherment of careers, structured working practices and enhanced work motivation.

While areas covered by training programmes varied between test facilities, there was a substantial increase in both breadth and depth in all training programmes. Training examples cited included 24 topics or areas, encompassing training related to QMSs, science specific training, training relating to safety, and business, leadership and life skills training. The training programmes reached staff at all levels of the facility, including non-technical staff such as administrators, drivers, office attendants and gardeners. Training was often specifically tailored to the needs of the test facility staff. (Quotes: IND1)

This training, combined with the practical experience of working in a GLP-compliant laboratory, was highly valued in enhancing career prospects. In all test facilities, staff took on additional responsibilities through, for example, leading on fire safety and organising fire drills or chairing training committees.

Individuals felt an enhanced sense of professionalism and prestige associated with developing and working in a GLP-compliant test facility and seeing the effect of work they had been involved with on changes in vector control policies and practices. This enhanced motivation amongst test facility staff at all levels and technicians and non-scientific staff in particular felt that their work was more structured, meaningful and purposeful following the project (Quotes: IND4). This motivation was enhanced further by an improved working environment following infrastructure improvements, including more working space, air conditioning, and better-quality workstations. (Quotes: IND2)

Examples of career progressions and internal promotions within test facilities were cited across several locations, including promotion of laboratory technicians to laboratory supervisors, and laboratory supervisors to senior management positions. (Quotes: IND3)

National/international level effects

At the national and international level, identified secondary effects included sharing of best practices within consortia and linked institutions, and the development of regional expertise related to data management and quality assurance.

Test facilities saw increased support from national level institutions, including increased investment in infrastructure. Alongside this, test facilities’ expertise in GLP was recognised at a national level, with the expectation that they would now act as national centres of excellence, both as a model of best practice and as a provider of training in entomology and relevant SOPs. Increased engagement with research outputs at the national decision-making level was anticipated as the next stage of this enhanced relationship with national level institutions, alongside a belief that this would raise policy-makers’ expectations of the test facilities’ performance. (Quotes: NAT1 and NAT2)

At a national and international level, the opportunity to meet and share experiences with the seven collaborating test facilities allowed best practice to be shared throughout the network, although this was not always fully realised as test facilities sought to strike a balance between collaboration and retaining a competitive advantage as a provider of product testing services. For construction and renovation of infrastructure, best practice was shared between test facilities that were geographically close together, because the requirements for buildings were the same and because travelling to these test facilities to see the buildings in person was easier. Data management and quality assurance expertise that was developed in test facilities further along the path to GLP certification, and by individuals associated with these test facilities, was also disseminated through the network. This was done formally through the project network, via training workshops and shared resources such as SOPs, and informally as these individuals acted in consultancy roles both within and outside of the institutions collaborating in the programme. Involvement in this network also raised the profile of individual test facilities, allowing these facilities to attract new studies and collaborators – including both GLP and non-GLP studies. (Quotes: NAT3)

Non-research capacity strengthening “ripple” effects

Ripple effects of the project beyond research capacity strengthening were widely reported for both individuals and the community surrounding the institutions. At the individual level, these were particularly focused on the transfer of skills developed through training and new practices associated with GLP to home lives. This was particularly true in test facilities that had broad and inclusive training programmes, encompassing topics such as how the test facility was budgeting for GLP and including staff in roles across the test facility such as drivers/cleaners. Here, individuals noted how they had applied time management, organisation, and budgeting skills developed through the GLP project to managing their personal lives and households. (Quotes: IND4).

Effects on communities, which could be described as national level effects, were rooted in often locally sourced solutions to challenges and, in particular, procurement and infrastructure development. By being locally based and finding local solutions, communities around the test facility saw investment in local businesses for consumables, construction materials and construction teams. Also reported was an increase in local employment as new studies were attracted, creating roles such as mosquito collection for experimental hut studies, and improvements in shared infrastructure such as roads. Test facility staff who recognised these effects in the community both took pride in these effects and valued them highly. (Quotes: NAT4).

Strengths and limitations

The strengths of this study are in the diversity of participants involved, capturing the views of staff filling a wide range of roles in five test facilities across three African countries. This approach ensured that effects meaningful to staff in diverse roles were reflected in the findings and offered a voice to staff less often heard within research teams, such as those of technicians and administrators. Furthermore, by using a qualitative approach, this study was able to richly describe the perceived effects of the GLP project and reveal and explain interactions between these effects.

This study is, however, limited by several factors. With a grounding in a specific laboratory capacity strengthening project, caution should be exercised on generalising these findings to all research capacity strengthening projects. Test facilities were at different stages towards GLP certification, with two test facilities having been granted GLP certification to date and this study is unlikely, therefore, to have captured all of the effects of the GLP project. Further effects will likely be identified by staff as the test facilities progress through certification and begin to attract GLP studies from multinational company sponsors. In addition, given the relatively small amount of time specifically dedicated to this question within the interviews, it is likely that additional effects may have been identified given more interview time. Finally, changes had to be made to data collection methods due to the COVID-19 pandemic: the responses at the two test facilities that participated via email and video-call are likely to be more superficial due to reduced opportunities to ask follow-up questions on observations.

Underlying data

Transcriptions of interviews with facility staff are available from the research group on request (please email ccr@lstmed.ac.uk to request access), on a case by case basis for the purpose of informing further research and on the condition that it will not be published in part or in entirety. They have not been made available as a dataset because they cannot be de-identified without compromising anonymity and the ethical approval conditions for the project stated that only the research team would have access to the data.

Extended data

Harvard Dataverse: Interview Guide and Information Sheets for: Developing laboratory capacity for Good Laboratory Practice certification: lessons from a Tanzanian insecticide testing facility. https://doi.org/10.7910/DVN/NADZPS¹⁸.

This project contains the following extended data:

Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).