WhatsApp as a medium to collect qualitative data among adolescents: lessons learned and considerations for future use [version 2; peer review: 2 approved with reservations, 1 not approved]

To understand dynamics surrounding adolescents’ access to and use of contraception, perspectives of youth themselves must be considered. Methods of data collection leveraging technological solutions may provide avenues for increased anonymity for adolescents to share their experiences with contraception. With the use of WhatsApp and other smartphone-based messaging applications as platforms for qualitative methodologies, public health researchers may find increased access to adolescents through utilizing mediums that adolescents already engage with in their daily lives. This article describes the methodology of using WhatsApp for conducting focus group discussions with adolescents in Malawi, including lessons learned around data collection, data use, and data quality. In June 2018, VillageReach used WhatsApp to conduct focus group discussions (FGD) to collect qualitative data on contraceptive use from Malawian youth. WhatsApp FGD participants were male and female adolescents, aged 15-19 representing rural and urban populations across three geographic regions of Malawi (northern, central and southern). Youth researchers (YR), aged 22-25, were trained and facilitated fifteen WhatsApp FGDs, as in-person (12) and remote (3) sessions. WhatsApp FGDs consisted of a YR sharing fictitious but contextually realistic ASRH scenarios, and managing the virtual discussion with questions and probing. Youth indicated comfort using WhatsApp to express their opinions and experiences related to contraceptive use and preferences. Based on the data collected, the researchers felt that use of the technology


Introduction
In order to understand the dynamics surrounding adolescents' access to and use of contraception, perspectives of youth themselves must be considered. In Malawi, where 22% of married and 52% of unmarried sexually active women between the ages of 15-19 report an unmet need for family planning (National Statistical Office/Malawi and ICF, 2017), more detailed data on adolescents' contraceptive barriers, needs and preferences can help bridge the gap between adolescents who want to prevent pregnancies but do not use contraception. Technology may provide an avenue for collecting this ASRH data and with added anonymity. Although anonymity can be compromised when using online methods, technology platforms do present an opportunity for youth to engage with other participants and facilitators on sensitive subjects such as ASRH, without having to make personal statements in face-toface discussions.
WhatsApp is a mobile messaging technology widely used in many areas of the world, including Africa, which allows users to send images, text, video and voice messages; and conduct individual or group conversations 1 (Kumar & Sharma, 2017). As WhatsApp has become a popular medium for communication (Rosenfeld et al., 2018), examples have emerged of the platform being used in health care and public health 2 . WhatsApp has been employed to conduct supportive supervision with community health care workers, reduce rural health worker feelings of isolation and recent nursing graduates' fears about transitioning to professional life, improve communication between consulting and emergency physicians, support delivery of out-of-hours pharmacy services, and bolster medical education programs ( . The use of WhatsApp focus group discussions (FGDs) for a 2019 waste management study in Singapore indicated the potential for using the platform for health research, while underscoring the need to further develop mobile messaging as a qualitative research method (Chen & Neo, 2019). Another study published in 2019 utilized WhatsApp to conduct remote interviews exploring health workers' perceptions in a conflict zone. The study commented on issues such as internet connectivity and confidentiality concerns as factors that impacted the ability to conduct interviews via WhatsApp (Fardousi et al., 2019).
In June 2018, VillageReach used WhatsApp to conduct FGDs with Malawian youth on the topic of contraceptive needs and preferences, as part of a larger study funded by the Bill & Melinda Gates Foundation entitled Youth-led Contraceptive Knowledge Generation and Design. WhatsApp was selected as a tool for eliciting information from adolescents because the mobile messaging application is popular among sub-populations of Malawian youth 4 (Matto & Kazungu, 2018), thereby offering researchers a chance to test whether or not the platform could offer an innovative approach to obtaining sensitive information from young people who may be uncomfortable sharing such information in traditional face-to-face FGDs.

Approach
Our approach was guided by principles of community based participatory research (CBPR), working with youth to design methods and instruments to explore the research questions (Leung et al., 2004). Six Malawian youth researchers (YRs), aged 22-to 25 and from the target districts, were recruited to facilitate the WhatsApp FGDs. The YR position was posted on local job boards and newspapers and resulted in over 500 applications, of which 23 applicants were interviewed. The interviews covered past experience and how applicants would handle hypothetical data quality and research ethics scenarios. A scoring rubric was completed by the interview panel and scores were used to select the six YRs for hire. Facilitation training for the YRs lasted four days and included study background, research ethics, FGD logistics, and role as moderator. Just prior to WhatsApp data collection we hosted a fourhour refresher training devoted to the WhatsApp technology and how to moderate in that context. Following the training, YR's worked with VillageReach staff to finalize eight scenarios depicting context-relevant ASRH situations to generate discussion in the FGD sessions.
Scenarios were derived through brainstorms with YRs, who drew upon insights gathered from methods used in the larger study design that preceded the WhatsApp FGDs (full design not described here but can be requested from authors). Scenarios included situations for the participants to react to and expound on, such as a scenario in which a married teen couple has young children and one partner would like to start using contraception, while another scenario described two teens who decide to have sex for the first time and use contraception, and how that discussion might unfold.
To recruit WhatsApp FGD participants the study team met with youth network community leaders and district youth officers to describe the research and target group. These community contacts then reached out to local youth groups, purposively selecting potential participants who fit the age and geographic sampling criteria. In-person WhatsApp FGD participants (n=96) were loaned a WhatsApp-enabled phone for one hour to allow for their participation in the FGD. In-person focus groups were mixed gender, stratified by marital status, urban/rural locality, and geographic region (or district); there were eight participants per FGD.
Remote WhatsApp FGD participants (n=38) did not gather at a central location; rather they used their own WhatsApp-enabled smartphones to respond to the weekly facilitated scenarios as they continued with their daily activities. Remote groups were mixed with regard to participant gender, marital status, and urban/rural locality, and were only stratified by geographic region. Northern, central and southern region remote sessions had 9, 11, and 16 participants, respectively.
Participants provided written informed consent. Those participating in the in-person FGDs with study phones were in the same room and therefore knew who was participating in the discussion overall, but their specific responses were fully anonymous given that the phone numbers associated with their contributions were linked to the study phones and not personal devices. Participants in remote sessions used their own phones and thus responses were linked to their phone numbers. The consent form made this clear and encouraged participants to remove identifying features from their WhatsApp profile during the participation in the sessions. Participants were also asked not to share their name, address, and other personal information in the chats, and were instructed not to contact any of the participants outside the context of the group discussion. Potential participants were informed that researchers could not prevent this from happening, but that researchers would remove anyone from the chat who was not following the guidelines, and that participants could block messages from such participants.
WhatsApp FGDs consisted of a YR sharing an ASRH scenario, and then managing the virtual discussion with posed questions and probing. Participants were able to message directly with the YR if they did not want to share something with the full group. A VillageReach staff member was present In-person participants were given transport reimbursement, while those in the remote FGDs and using personal phones were compensated with 500MB of airtime data. Participants in remote sessions were given an additional 500MB of data as needed during the course of their participation.

Lessons learned
Applying WhatsApp mobile messaging technology to facilitate FGDs yielded important learnings about the strengths and limitations of this methodology; these have been organized into learnings related to data collection, data quality and data analysis. These learnings may be useful to other researchers attempting to use WhatsApp as a means of data collection.

Data collection
Using an application reliant on mobile data or internet connectivity led to challenges in data collection. For example, in some areas with limited mobile network coverage there were delays in the 'chat' causing messages to appear long after they were written. This resulted in disorganized conversations which were challenging for YRs and participants to follow. For some of the rural in-person sessions, unavailable or unstable electricity led to phones losing battery charge during sessions, resulting in participants dropping in and out of the group while YRs made alternate power sources available. In some cases, participants had to move from one area to another in search of a location with strong enough mobile data network coverage, causing delays in data collection. Despite these challenges, using WhatsApp technology enhanced the quantity of data collected in several ways. For example, with remote FGDs, the use of WhatsApp allowed for ongoing discussion over longer periods of time and distance. Although participation did taper off toward the end of the four-week period, in general the elongated timeframe allowed for increased quantity of data. The difference is attributable to remote groups responding to more scenarios than the in-person groups, as well as having increased time to reflect and share their thoughts. Additionally, remote participants shared that as they could access improved internet connectivity throughout the month, and this allowed them the opportunity to further comment on scenarios. This option offered increased time and flexibility for discussion in comparison to the inperson FGDs but also required increased effort to effectively moderate and supervise.

Data quality
Participants in the WhatsApp FGDs shared detailed examples and engaged in extended discussions about their experiences and preferences. In some cases, this led to language or discussions that the YRs felt were too explicit, and they redirected the discussion per their training. Due to the nature of the remote-FGDs, participants were required to have access to their own smartphone to participate, thus limiting transferability of results to adolescents who do not have access to smartphones. In-person session participants were loaned smartphones; those who had never used one before were not able to use the technology as quickly or as effectively to communicate and thus contributed sparsely to written conversation, although they were able to utilize the voice memo function to clearly express themselves.

Data analysis
Challenges encountered during data collection affected our ability to analyze WhatsApp FGD data. Choppy discussion threads due to limited network coverage in some areas led to difficulty trying to piece together narratives during transcription. One unanticipated issue was that participants favored using the voice memo feature over typing. Despite an intensive search, no efficient way to export the incoming text and voice memos into one file compatible with both text and audio, in chronological order, was identified. Thus, both audio recording and text messages had to be manually woven together to construct a complete transcript. In the end, this additional work resulted in use of discussion summary rather than verbatim transcription for analysis, and delayed the overall analysis process.

Considerations for future use
While there are benefits to the WhatsApp FGD methodology, there are considerations that may complicate data collection in comparison to other qualitative methods. The approach requires certain logistics such as access to power sources to keep phones charged as well as WIFI or stable mobile data network connection to ensure FGDs run smoothly, and ensuring these pre-conditions exist requires planning. Additionally, the methodology may not be conducive to every demographic and as elaborated elsewhere, requires target populations to have some level of smartphone literacy, which should be considered when sampling populations with low rates of smart phone ownership (Chen & Neo, 2019).
Costs associated with WhatsApp FGDs (providing devices, data) may prohibit use of the platform for research in health. However, with a projected increase in smartphone ownership 5 paired with introduction of low-cost WhatsApp bundles by some sub-Saharan mobile network operators, more people have access to this platform and other social media than ever before 6 . WhatsApp may also be a more cost-effective option in some circumstances, as it allows researchers to collect data on an ongoing basis without traditional costs such as hall rentals, transportation reimbursement to participants, etc.
Regardless of cost, this methodology may be more convenient for researchers and participants in some cases, particularly if aiming to engage the same cohort over time. The method may also be better positioned to reach the adolescent audience due to the technology's flexibility, convenience and portability, and expanding familiarity of the shared. Additionally, researchers should consider whether participants will be required to have access to their own smartphones to participate or if devices will be provided, and the resulting trade-offs around cost, representativeness of the sample, etc.

Conclusions/discussion
WhatsApp provides an alternative platform for researchers when eliciting information from adolescents on sensitive topics. The tool has both advantages and disadvantages associated with data collection, data quality and analysis, which should be considered prior to using WhatsApp for FGDs. The potential of this methodology is to allow researchers to collect in-depth and granular qualitative data and is a strength of the approach when working with challenging topics or stigmatized populations, who may be particularly concerned participating in more visible forms of data collection such as in person FGDs or in-depth interviews. Adolescents indicated feeling comfort using WhatsApp to share their perspectives on sensitive topics. Additionally, the opportunity to collect qualitative data over a longer period of time allows researchers to collect a great quantity of data as well as offers the potential for in-depth nuanced data as participants have more time to gain confidence and trust with the researchers and fellow participants.
By utilizing mobile messaging technologies, such as WhatsApp, health researchers may be better able to connect to adolescent experiences in ways reflecting contemporary youth culture and communication preferences. This in turn, could enhance the visibility of youth perspectives in health research and help improve the effectiveness of youth-targeted public health interventions.
As the use of WhatsApp grows to support health sector projects, there is a need to elevate more examples of lessons learned to drive technology to address the challenges faced using the tool. It will also be important to promote successes so researchers can consider whether it is a suitable platform for their work. As technology is applied in new ways in research, it is critical that the research community establishes guidelines for addressing ethical considerations, such as recruitment of diverse perspectives given a growing digital divide (Afrobarometer, 2022), and considerations around data privacy to be able to fully inform research participants of the risks and benefits to prior to participating in research through platforms such as WhatsApp.

Data availability
All data underlying the results are available as part of the article and no additional source data are required. The authors describe clear and in a well manner the procedures followed to develop the study. However, being a methodological article, it would be necessary to include other information about the process, results and interpretation of the application of focus groups through a digital platform as whatsapp. Particularly, it would be useful to specify the reasons for certain decisions, and their methodological implications. In specific, there is no clarification in the document about how and why the researcher decided to do more groups in person, than remote. And in the remote groups, how did they decide follow the discussion during a month? Also, it is necessary to specify what was the objective of exploring the application in person and remotely. Was part of the objective to observe differences between these two approaches?

Open Peer Review
In Lessons Learned section, I believe the analysis of using the WhatsApp application to collect information through focus groups could be more exhaustive. General issues are mentioned and the situations that occurred throughout the application of the methodology at the different levels of stratification are not delved into, for example, the same difficulties arose regarding the categories of analysis of data collection?, data quality?, and data analysis?, between the three geographic districts, the rural and urban area, and in the groups according to marital status?. Likewise, the differences between having applied it in person or remotely are not very in-depth analyzed in the text. For example, about the advantage that arises from a greater amount of information obtained in remote focus groups. It is more information, but diminishes the quality?, or the quality is same between in person and remote?
There are questions that arise during reading that should be addressed by the authors.

Are sufficient details provided to allow replication of the method development and its use by others? Partly
If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Melissa Gilliam
Center for Interdisciplinary Inquiry and Innovation in Sexual and Reproductive Health, Section of Family Planning and Contraceptive Research, University of Chicago, Chicago, IL, USA In this article, the authors consider WhatsApp for collecting qualitative data from adolescents. There are interesting things to learn and the paper would be greatly enhanced if more information was provided up front regarding the purpose of the paper. Specifically, the authors should clarify that it is a methods paper in which the research team and facilitators describe their experience.

Introduction:
In the introduction, there is a bit of overstatement. The following are some examples but please review the manuscript to consider this concern, as follows: In the first paragraph of the intro, please change declarative sentences such as "candid youth perspectives are necessary…" perhaps instead "can help in understanding…" -are candid perspectives necessary?

Approach:
In-person participants were mixed gender-correct this to say that the sessions were mixed gender not the participants.

○
In this section, start with the design rather than the results of the FGD. The number of participants and demographics are part of the study findings and the selection of YRs is part of the design.

○
The approach section should be reordered chronologically. Also, it is not clear until the end of the approach study that the paper will not include results from the FGD. Thus, the focus of the paper needs to be made clear up front. That this is about the research team's experience using WhatsApp.

○
The use of 3 -10 hour discussion seems unusual why were boundaries not up in place to gain greater uniformity of data collection?

Conclusions:
There is no mention of IRB approval.

If any results are presented, are all the source data underlying the results available to ensure full reproducibility? No
Are the conclusions about the method and its performance adequately supported by the findings presented in the article? Partly Competing Interests: No competing interests were disclosed. There are interesting things to learn and the paper would be greatly enhanced if more information was provided up front regarding the purpose of the paper. Specifically, the authors should clarify that it is a methods paper in which the research team and facilitators describe their experience. Clarified in abstract that paper is about the methods used and includes overall lessons learned.

Introduction:
In the introduction, there is a bit of overstatement. The following are some examples but please review the manuscript to consider this concern, as follows: In the first paragraph of the intro, please change declarative sentences such as "candid youth perspectives are necessary…" perhaps instead "can help in understanding…" -are candid perspectives necessary? Revised sentence to better convey our intention and removed "candid"

Conclusions:
There is no mention of IRB approval. We had included this in the approach section and have made it more apparent.

Competing Interests
No competing interests were disclosed.
The paper describes a methodological approach that is valuable for: (1) generating additional insights to improve access to ASRH services; and (2) supporting and advancing the production of knowledge by Malawian youth, for Malawian youth, using the digital tools and media that they prefer. This emphasis on youth participation aligns well with recent calls to decolonize global public health research and practice (see Abraham  . To further consider these emancipatory possibilities, you might refer to this special issue (from the education sector) on voice and representation in digital media production (Dussel & Dahya, 2017 4 ), which I think resonates strongly with the research described this paper.
As you observe, WhatsApp has been deployed in global health operations widely but more research is needed to understand how to integrate social media platforms into qualitative research methods. This paper helps to fill that gap and once additional 'detail provided to enable others to improve the method' is provided', it will make an important contribution to the field.

Recommended Revisions:
My major concern with the paper is that concepts of privacy, anonymity, and confidentiality are not adequately explained/operationalized (see Hunter et al., 2018 5 ) for a helpful review). Informed consent was secured and the study protocol was approved, so the issue is not so much whether these issues were addressed, but rather how they were addressed. The reader needs to understand this in order to replicate and build on your work.
I think my reservations along this line can be addressed by making the following changes: In the first paragraph of the Introduction section, you write "Technology may provide an ○ avenue for collecting this ASRH data and with added privacy." Explain how you are using the word "privacy" here -is it referring to the privacy of study participants in relation to researchers? In relation to other focus group participants? Family members? Other peers? Other research methodologies? Clarify whether this claim only applies to focus groups, or whether it also pertains to other qualitative methods such as interviews and photoelicitation. Be more specific about the ways in which we might expect technology to enhance "privacy".
In the Approach section, provide more detail about how your study protocol protected the "privacy" of your study participants. You mention the institutional review board in this section, and I think privacy in this respect means something slightly different than what was described in the Introduction. How was the privacy of study subjects protected? Was it possible to assure participants anonymity or confidentiality? are not adequately explained/operationalized (see Hunter et al., 2018 5 ) for a helpful review). Informed consent was secured and the study protocol was approved, so the issue is not so much whether these issues were addressed, but rather how they were addressed. The reader needs to understand this in order to replicate and build on your work.
I think my reservations along this line can be addressed by making the following changes: In the first paragraph of the Introduction section, you write "Technology may provide an avenue for collecting this ASRH data and with added privacy." Explain how you are using the word "privacy" here -is it referring to the privacy of study participants in relation to researchers? In relation to other focus group participants? Family members? Other peers? Other research methodologies? Clarify whether this claim only applies to focus groups, or whether it also pertains to other qualitative methods such as interviews and photo-elicitation. Be more specific about the ways in which we might expect technology to enhance "privacy". Great suggestion to reference the Hunter article, as we were using "privacy" too loosely. We in fact mean "anonymity" when interfacing with fellow participants and researchers, and we have revised accordingly.

○
In the Approach section, provide more detail about how your study protocol protected the "privacy" of your study participants. You mention the institutional review board in this section, and I think privacy in this respect means something slightly different than what was described in the Introduction. How was the privacy of study subjects protected? Was it possible to assure participants anonymity or confidentiality? If so, what measures were adopted to achieve this? Did those measures differ according to whether FGDs were in-person or remote? Yes, we meant "anonymity" in the introduction and in approach section we mean "anonymity" and "confidentiality." We have revised to clarify and have shared what measures were in place to safeguard both anonymity and confidentiality. We provided additional context on safeguards for and potential shortcomings related to anonymity and confidentiality and in-person and remote forums.

○
In last sentence of the section Consideration for future: (1) use observations from your debriefing summaries and the Lessons learned section to further develop and support the claim about anonymity and privacy that is made here, and (2)