Gates Open Res Gates Open Research 2572-4754 F1000 Research Limited London, UK 10.12688/gatesopenres.13169.2 Method Article Articles WhatsApp as a medium to collect qualitative data among adolescents: lessons learned and considerations for future use

[version 2; peer review: 3 approved with reservations, 1 not approved]

 Singer Barbara Conceptualization Funding Acquisition Writing – Original Draft Preparation 1 Walsh Caitlin M. Formal Analysis Investigation Validation Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0002-3508-2224 1 Gondwe Lucky Conceptualization Formal Analysis Investigation Methodology Project Administration Validation Writing – Review & Editing https://orcid.org/0000-0002-6476-4742 2 Reynolds Katie Conceptualization Formal Analysis Investigation Project Administration Validation Writing – Review & Editing https://orcid.org/0000-0002-5222-8950 a 1 Lawrence Emily Formal Analysis Investigation Methodology Project Administration Validation Writing – Review & Editing https://orcid.org/0000-0002-4572-1800 1 Kasiya Alinafe Conceptualization Funding Acquisition Supervision 2 VillageReach, Seattle, WA, 98102, USA VillageReach, Lilongwe, Malawi katie.reynolds@villagereach.org

No competing interests were disclosed.

 19 4 2023 2020 4 130 17 4 2023 Copyright: © 2023 Singer B et al. 2023 This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

In order to understand the dynamics surrounding adolescents’ access to and use of contraception, perspectives of youth themselves must be considered. Methods of data collection leveraging technological solutions may provide avenues for increased anonymity for adolescents to share their experiences with contraception. With the use of WhatsApp and other smartphone-based messaging applications as platforms for qualitative methodologies, public health researchers may find increased access to adolescents through utilizing mediums that adolescents already engage with in their daily lives. This article describes the methodology of using WhatsApp for conducting focus group discussions with adolescents in Malawi, including lessons learned around data collection, data use, and data quality.

In June 2018, VillageReach used WhatsApp to conduct focus group discussions (FGD) to collect qualitative data on contraceptive use from Malawian youth. WhatsApp FGD participants were male and female adolescents, aged 15–19 representing rural and urban populations across three geographic regions of Malawi (northern, central and southern). Youth researchers (YR), aged 22–25, were trained and facilitated fifteen WhatsApp FGDs, as in-person (12) and remote (3) sessions. WhatsApp FGDs consisted of a YR sharing fictitious but contextually realistic ASRH scenarios, and managing the virtual discussion with questions and probing. Youth indicated comfort using WhatsApp to express their opinions and experiences related to contraceptive use and preferences. Based on the data collected, the researchers felt that use of the technology may increase willingness to provide additional levels of information during anonymized WhatsApp FGDs vis à vis traditional, in-person FGDs.

Using WhatsApp as a methodological tool to facilitate FGDs has practical implications for data collection, data quality and data analysis. This paper reports lessons learned as well as the advantages and disadvantages of using WhatsApp to conduct FGDs in a low-resource setting. It also calls for developing guidelines and an ethical code of conduct for the future use of mobile applications for conducting qualitative research with vulnerable populations.

 adolescents sexual health WhatsApp Malawi contraception qualitative methods focus groups Gates Foundation INV-006863 This work was supported by the Gates Foundation [INV-006863]. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Amendments from Version 1

This updated version addresses reviewer feedback.  Specifically, we have clarified the focus of the paper on the process itself and lessons learned, as opposed to the specific results of the research. We have also included additional detail to allow readers to better understand our approach and how to replicate it.

 Introduction

In order to understand the dynamics surrounding adolescents’ access to and use of contraception, perspectives of youth themselves must be considered. In Malawi, where 22% of married and 52% of unmarried sexually active women between the ages of 15–19 report an unmet need for family planning ( National Statistical Office/Malawi and ICF, 2017), more detailed data on adolescents’ contraceptive barriers, needs and preferences can help bridge the gap between adolescents who want to prevent pregnancies but do not use contraception. Technology may provide an avenue for collecting this ASRH data and with added anonymity. Although anonymity can be compromised when using online methods, technology platforms do present an opportunity for youth to engage with other participants and facilitators on sensitive subjects such as ASRH, without having to make personal statements in face-to-face discussions.

WhatsApp is a mobile messaging technology widely used in many areas of the world, including Africa, which allows users to send images, text, video and voice messages; and conduct individual or group conversations 1 ( Kumar & Sharma, 2017). As WhatsApp has become a popular medium for communication ( Rosenfeld et al., 2018), examples have emerged of the platform being used in health care and public health 2 . WhatsApp has been employed to conduct supportive supervision with community health care workers, reduce rural health worker feelings of isolation and recent nursing graduates’ fears about transitioning to professional life, improve communication between consulting and emergency physicians, support delivery of out-of-hours pharmacy services, and bolster medical education programs ( Ajuwon et al., 2018; Copestake et al., 2020; Gulacti et al., 2016; Henry et al., 2016; Pimmer et al., 2017; Pimmer et al., 2019; Pimmer, 2015; Raiman et al., 2017; Rathbone et al., 2020; Willemse, 2015; Willemse et al., 2019). WhatsApp has also been utilized to help deliver public health interventions, including a bed net campaign in Mozambique, HIV counseling for men who have sex with men in Peru, and for sending health information to new parents in South Africa 3 ( Arroz et al., 2019; Bayona et al., 2017). However, there are few recorded examples of WhatsApp being used for research in public health ( Chen & Neo, 2019; Fardousi et al., 2019). The use of WhatsApp focus group discussions (FGDs) for a 2019 waste management study in Singapore indicated the potential for using the platform for health research, while underscoring the need to further develop mobile messaging as a qualitative research method ( Chen & Neo, 2019). Another study published in 2019 utilized WhatsApp to conduct remote interviews exploring health workers’ perceptions in a conflict zone. The study commented on issues such as internet connectivity and confidentiality concerns as factors that impacted the ability to conduct interviews via WhatsApp ( Fardousi et al., 2019).

In June 2018, VillageReach used WhatsApp to conduct FGDs with Malawian youth on the topic of contraceptive needs and preferences, as part of a larger study funded by the Bill & Melinda Gates Foundation entitled Youth-led Contraceptive Knowledge Generation and Design. WhatsApp was selected as a tool for eliciting information from adolescents because the mobile messaging application is popular among sub-populations of Malawian youth 4 ( Matto & Kazungu, 2018), thereby offering researchers a chance to test whether or not the platform could offer an innovative approach to obtaining sensitive information from young people who may be uncomfortable sharing such information in traditional face-to-face FGDs.

 Approach

Our approach was guided by principles of community based participatory research (CBPR), working with youth to design methods and instruments to explore the research questions ( Leung et al., 2004). Six Malawian youth researchers (YRs), aged 22-to 25 and from the target districts, were recruited to facilitate the WhatsApp FGDs. The YR position was posted on local job boards and newspapers and resulted in over 500 applications, of which 23 applicants were interviewed. The interviews covered past experience and how applicants would handle hypothetical data quality and research ethics scenarios. A scoring rubric was completed by the interview panel and scores were used to select the six YRs for hire. Facilitation training for the YRs lasted four days and included study background, research ethics, FGD logistics, and role as moderator. Just prior to WhatsApp data collection we hosted a four-hour refresher training devoted to the WhatsApp technology and how to moderate in that context. Following the training, YR’s worked with VillageReach staff to finalize eight scenarios depicting context-relevant ASRH situations to generate discussion in the FGD sessions.

Scenarios were derived through brainstorms with YRs, who drew upon insights gathered from methods used in the larger study design that preceded the WhatsApp FGDs (full design not described here but can be requested from authors). Scenarios included situations for the participants to react to and expound on, such as a scenario in which a married teen couple has young children and one partner would like to start using contraception, while another scenario described two teens who decide to have sex for the first time and use contraception, and how that discussion might unfold.

To recruit WhatsApp FGD participants the study team met with youth network community leaders and district youth officers to describe the research and target group. These community contacts then reached out to local youth groups, purposively selecting potential participants who fit the age and geographic sampling criteria. In-person WhatsApp FGD participants (n=96) were loaned a WhatsApp-enabled phone for one hour to allow for their participation in the FGD. In-person focus groups were mixed gender, stratified by marital status, urban/rural locality, and geographic region (or district); there were eight participants per FGD.

Remote WhatsApp FGD participants (n=38) did not gather at a central location; rather they used their own WhatsApp-enabled smartphones to respond to the weekly facilitated scenarios as they continued with their daily activities. Remote groups were mixed with regard to participant gender, marital status, and urban/rural locality, and were only stratified by geographic region. Northern, central and southern region remote sessions had 9, 11, and 16 participants, respectively.

Participants provided written informed consent. Those participating in the in-person FGDs with study phones were in the same room and therefore knew who was participating in the discussion overall, but their specific responses were fully anonymous given that the phone numbers associated with their contributions were linked to the study phones and not personal devices. Participants in remote sessions used their own phones and thus responses were linked to their phone numbers. The consent form made this clear and encouraged participants to remove identifying features from their WhatsApp profile during the participation in the sessions. Participants were also asked not to share their name, address, and other personal information in the chats, and were instructed not to contact any of the participants outside the context of the group discussion. Potential participants were informed that researchers could not prevent this from happening, but that researchers would remove anyone from the chat who was not following the guidelines, and that participants could block messages from such participants.

WhatsApp FGDs consisted of a YR sharing an ASRH scenario, and then managing the virtual discussion with posed questions and probing. Participants were able to message directly with the YR if they did not want to share something with the full group. A VillageReach staff member was present at each in-person session and was included in each remote group to provide support to the YRs if needed.

Debriefing sessions between VillageReach staff and YRs were held either in-person or by phone following each FGD in which the team iteratively discussed their observations and experiences and built consensus on what worked, what did not work and lessons learned for future use of WhatsApp FGDs. Specifically, YRs were asked to reflect on: “what were the key takeaways from the group’s discussion? (including debates, powerful statements)”, “what feedback did participants have? (either on the content of the discussion or more generally around process),” “what worked well in facilitating that should be repeated?,” “what should be changed or improved about how the session was facilitated?” and “other.” Findings from these prompts were documented as debriefing summaries and approved by all debriefing session participants.

Fifteen WhatsApp FGDs were conducted, either as in-person (12) or remote (3) sessions. In-person WhatsApp FGDs were conducted at a specific time and location, such as youth clubs, sports facilities, schools or health centers, for approximately one hour. Remote FGDs convened several times each week over the period of one month, ranging from discussions totaling three hours to 10 hours each week. The amount of time dedicated each week depended on participant and facilitator availability and level of engagement throughout the week. Participants were male and female Malawian adolescents, aged 15–19 (n=134). Sessions occurred in three target geographic regions across Malawi (northern, central and southern). The study was reviewed and received approval from the Malawi National Health Sciences Research Committee (Protocol Number: 17/05/1813). In-person participants were given transport reimbursement, while those in the remote FGDs and using personal phones were compensated with 500MB of airtime data. Participants in remote sessions were given an additional 500MB of data as needed during the course of their participation.

Summary of FGDs.
Session ID In-person or remote Geographic district Urban or Rural Marital status
1 In-person Nkhatabay Urban Unmarried
2 In-person Nkhatabay Urban Married
3 In-person Nkhatabay Rural Unmarried
4 In-person Nkhatabay Rural Married
5 In-person Machinga Urban Unmarried
6 In-person Machinga Urban Married
7 In-person Machinga Rural Unmarried
8 In-person Machinga Rural Married
9 In-person Lilongwe Urban Unmarried
10 In-person Lilongwe Urban Married
11 In-person Lilongwe Rural Unmarried
12 In-person Lilongwe Rural Married
13 Remote Nkhatabay (remote) Both; participants from urban and rural areas Both unmarried and married participants
14 Remote Machinga (remote) Both; participants from urban and rural areas Both unmarried and married participants
15 Remote Lilongwe (remote) Both; participants from urban and rural areas Both unmarried and married participants
Lessons learned

Applying WhatsApp mobile messaging technology to facilitate FGDs yielded important learnings about the strengths and limitations of this methodology; these have been organized into learnings related to data collection, data quality and data analysis. These learnings may be useful to other researchers attempting to use WhatsApp as a means of data collection.

Data collection

Using an application reliant on mobile data or internet connectivity led to challenges in data collection. For example, in some areas with limited mobile network coverage there were delays in the ‘chat’ causing messages to appear long after they were written. This resulted in disorganized conversations which were challenging for YRs and participants to follow. For some of the rural in-person sessions, unavailable or unstable electricity led to phones losing battery charge during sessions, resulting in participants dropping in and out of the group while YRs made alternate power sources available. In some cases, participants had to move from one area to another in search of a location with strong enough mobile data network coverage, causing delays in data collection. Despite these challenges, using WhatsApp technology enhanced the quantity of data collected in several ways. For example, with remote FGDs, the use of WhatsApp allowed for ongoing discussion over longer periods of time and distance. Although participation did taper off toward the end of the four-week period, in general the elongated timeframe allowed for increased quantity of data. The difference is attributable to remote groups responding to more scenarios than the in-person groups, as well as having increased time to reflect and share their thoughts. Additionally, remote participants shared that as they could access improved internet connectivity throughout the month, and this allowed them the opportunity to further comment on scenarios. This option offered increased time and flexibility for discussion in comparison to the in-person FGDs but also required increased effort to effectively moderate and supervise.

 Data quality

Participants in the WhatsApp FGDs shared detailed examples and engaged in extended discussions about their experiences and preferences. In some cases, this led to language or discussions that the YRs felt were too explicit, and they redirected the discussion per their training. Due to the nature of the remote-FGDs, participants were required to have access to their own smartphone to participate, thus limiting transferability of results to adolescents who do not have access to smartphones. In-person session participants were loaned smartphones; those who had never used one before were not able to use the technology as quickly or as effectively to communicate and thus contributed sparsely to written conversation, although they were able to utilize the voice memo function to clearly express themselves.

 Data analysis

Challenges encountered during data collection affected our ability to analyze WhatsApp FGD data. Choppy discussion threads due to limited network coverage in some areas led to difficulty trying to piece together narratives during transcription. One unanticipated issue was that participants favored using the voice memo feature over typing. Despite an intensive search, no efficient way to export the incoming text and voice memos into one file compatible with both text and audio, in chronological order, was identified. Thus, both audio recording and text messages had to be manually woven together to construct a complete transcript. In the end, this additional work resulted in use of discussion summary rather than verbatim transcription for analysis, and delayed the overall analysis process.

 Considerations for future use

While there are benefits to the WhatsApp FGD methodology, there are considerations that may complicate data collection in comparison to other qualitative methods. The approach requires certain logistics such as access to power sources to keep phones charged as well as WIFI or stable mobile data network connection to ensure FGDs run smoothly, and ensuring these pre-conditions exist requires planning. Additionally, the methodology may not be conducive to every demographic and as elaborated elsewhere, requires target populations to have some level of smartphone literacy, which should be considered when sampling populations with low rates of smart phone ownership ( Chen & Neo, 2019).

Costs associated with WhatsApp FGDs (providing devices, data) may prohibit use of the platform for research in health. However, with a projected increase in smartphone ownership 5 paired with introduction of low-cost WhatsApp bundles by some sub-Saharan mobile network operators, more people have access to this platform and other social media than ever before 6 . WhatsApp may also be a more cost-effective option in some circumstances, as it allows researchers to collect data on an ongoing basis without traditional costs such as hall rentals, transportation reimbursement to participants, etc.

Regardless of cost, this methodology may be more convenient for researchers and participants in some cases, particularly if aiming to engage the same cohort over time. The method may also be better positioned to reach the adolescent audience due to the technology’s flexibility, convenience and portability, and expanding familiarity of the WhatsApp platform (and of smartphones, generally) in adolescent sub-populations across Africa 7 . Further, the amount of detail and insights youth contributed in discussions indicates that the anonymity offered by WhatsApp FGDs may be an important factor in youth feeling comfortable to share sensitive information, especially in social environments like Malawi where adolescents place a high value on keeping sexual behavior hidden from elders, parents and peers ( Limaye et al., 2012; Munthali et al., 2006).

Researchers who plan to use this technology should carefully consider the study design and any implications. For instance, researchers may want to clearly articulate to participants the ways the platform could be used in terms of whether voice recordings, photos/emoticons, text are all allowed, or if participants should only use the text feature, for example. Such guidelines may have implications for the resulting data format and ease of analysis, as well as how comfortable participants are sharing and the amount and nuance of information shared. Additionally, researchers should consider whether participants will be required to have access to their own smartphones to participate or if devices will be provided, and the resulting trade-offs around cost, representativeness of the sample, etc.

 Conclusions/discussion

WhatsApp provides an alternative platform for researchers when eliciting information from adolescents on sensitive topics. The tool has both advantages and disadvantages associated with data collection, data quality and analysis, which should be considered prior to using WhatsApp for FGDs. The potential of this methodology is to allow researchers to collect in-depth and granular qualitative data and is a strength of the approach when working with challenging topics or stigmatized populations, who may be particularly concerned participating in more visible forms of data collection such as in person FGDs or in-depth interviews. Adolescents indicated feeling comfort using WhatsApp to share their perspectives on sensitive topics. Additionally, the opportunity to collect qualitative data over a longer period of time allows researchers to collect a great quantity of data as well as offers the potential for in-depth nuanced data as participants have more time to gain confidence and trust with the researchers and fellow participants.

By utilizing mobile messaging technologies, such as WhatsApp, health researchers may be better able to connect to adolescent experiences in ways reflecting contemporary youth culture and communication preferences. This in turn, could enhance the visibility of youth perspectives in health research and help improve the effectiveness of youth-targeted public health interventions.

As the use of WhatsApp grows to support health sector projects, there is a need to elevate more examples of lessons learned to drive technology to address the challenges faced using the tool. It will also be important to promote successes so researchers can consider whether it is a suitable platform for their work. As technology is applied in new ways in research, it is critical that the research community establishes guidelines for addressing ethical considerations, such as recruitment of diverse perspectives given a growing digital divide ( Afrobarometer, 2022), and considerations around data privacy to be able to fully inform research participants of the risks and benefits to prior to participating in research through platforms such as WhatsApp.

 Data availability

All data underlying the results are available as part of the article and no additional source data are required.

 Acknowledgements

Youth researchers: Isaac Chawinga, Ullanda Gondwe, Quipo Theu, Nookota Kumadzulo, Jessy Uchindami Gondwe, Mustarff Komanje, Justin Mpalabwazi.

1 https://www.messengerpeople.com/global-messenger-usage-statistics/; https://qz.com/africa/1206935/whatsapp-is-the-most-popular-messaging-app-in-africa/; https://www.businessofapps.com/data/whatsapp-statistics/; https//www.bizcommunity.com/Article/196/793/180448.html

2 https://www.whatsapp.com/business/api

3 https://www.forbes.com/sites/tobyshapshak/2017/12/04/african-messaging-wonder-momconnect-launches-on-whatsapp/#7d0e35c57c3b

4 https://dai-global-digital.com/digital-insights-malawi-communication-among-rural-communities.html

5 https://data.gsmaintelligence.com

6 https://www.messengerpeople.com/global-messenger-usage-statistics/; https://www.nyasatimes.com/airtel-introduces-new-social-media-bundle/; https://techweez.com/2020/01/21/airtel-amazing-data-bundles-offer-double-data/; https://www.itnewsafrica.com/2020/03/mtn-announces-massive-data-price-cuts/; https://iharare.com/save-money-cheapest-data-bundles-to-use-during-lockdown/; https://mybroadband.co.za/news/cellular/317857-whatsapp-is-killing-sms-in-south-africa.html; https://mybroadband.co.za/news/cellular/314105-how-to-save-mobile-data-with-whatsapp-bundles.html; https://www.slideshare.net/wearesocial/digital-in-2018-in-eastern-africa-part-2-south-86865801

7 https://dai-global-digital.com/digital-insights-malawi-communication-among-rural-communities.html; https://qz.com/africa/1206935/whatsapp-is-the-most-popular-messaging-app-in-africa/

Afrobarometer: Digital divide: Who in Africa is connected and who is not. Afrobarometer Dispatch No. 582,2022. Reference Source Ajuwon A Pimmer C Odetola T : Mobile instant messaging (MIM) to support teaching practice: insights from a nurse tutor program in Nigeria. Malawi Med J. 2018;30(2):120–126. 30627340 10.4314/mmj.v30i2.12 6307071 Arroz JAH Candrinho BN Mussambala F : WhatsApp: A supplementary tool for improving bed nets universal coverage campaign in Mozambique. BMC Health Serv Res. BioMed Central Ltd.,2019;19(1):86. 30709376 10.1186/s12913-019-3929-0 6359840 Bayona E Menacho L Segura ER : The Experiences of Newly Diagnosed Men Who Have Sex with Men Entering the HIV Care Cascade in Lima, Peru, 2015-2016: A Qualitative Analysis of Counselor-Participant Text Message Exchanges. Cyberpsychol Behav Soc Netw. Mary Ann Liebert Inc.2017;20(6):389–396. 28622034 10.1089/cyber.2016.0435 5510042 Chen J Neo P : Texting the waters: An assessment of focus groups conducted via the WhatsApp smartphone messaging application. Method Innov. SAGE Publications.2019;12(3). 10.1177/2059799119884276 Copestake J Theuss M Brownie S : Recently graduated midwives in Uganda: Self-perceived achievement, wellbeing and work prospects. Midwifery. Churchill Livingstone. 2020;82:102596. 31887471 10.1016/j.midw.2019.102596 Fardousi N Douedari Y Howard N : Healthcare under siege: A qualitative study of health-worker responses to targeting and besiegement in Syria. BMJ Open. BMJ Publishing Group,2019;9(9):e029651. 31488482 10.1136/bmjopen-2019-029651 6731863 Gulacti U Lok U Hatipoglu S : An Analysis of WhatsApp Usage for Communication Between Consulting and Emergency Physicians. J Med Syst. Springer New York LLC.2016;40(6):130. 27083574 10.1007/s10916-016-0483-8 Henry JV Winters N Lakati A : Enhancing the supervision of community health workers with WhatsApp mobile messaging: Qualitative findings from 2 low-resource settings in Kenya. Glob Health Sci Pract. Johns Hopkins University Press.2016;4(2):311–325. 27353623 10.9745/GHSP-D-15-00386 4982254 Kumar N Sharma S : Survey Analysis on the usage and Impact of Whatsapp Messenger. Global Journal of Enterprise Information System. 2017;8(3). (Accessed: 20 May 2020). 10.18311/GJEIS/2016/15741 Leung MW Yen IH Minkler M : Community based participatory research: a promising approach for increasing epidemiology's relevance in the 21st century. Int J Epidemiol. 2004;33(3):499–506. 15155709 10.1093/ije/dyh010 Limaye RJ Rimal RN Mkandawire G : Talking about sex in Malawi: toward a better understanding of interpersonal communication for HIV prevention. J Public Health Res. PAGEPress Publication.2012;1(2):117–25. 25170453 10.4081/jphr.2012.e17 4140368 Matto G Kazungu I : Mobile Phone and Chat Apps Usage Among Malawian University Students: LUANAR’s Experience. Research Report Series. 2018;2(1). Reference Source Munthali AC Moore AM Konyani S : Qualitative Evidence of Adolescents’ Sexual and Reproductive Health Experiences in Selected Districts of Malawi.New York. 2006. Reference Source National Statistical Office/Malawi and ICF : Malawi Demographic and Health Survey 2015-16.Zomba, Malawi.2017; (Accessed: 28 April 2020). Reference Source Pimmer C : Using WhatsApp Groups to support Community Health Workers. 2015. 10.13140/RG.2.1.1450.5048 Pimmer C Abiodun R Daniels F : "I felt a sense of belonging somewhere". Supporting graduates' job transitions with WhatsApp groups. Nurse Educ Today. Churchill Livingstone. 2019;81:57–63. 31330403 10.1016/j.nedt.2019.06.010 Pimmer C Mhango S Mzumara A : Mobile instant messaging for rural community health workers: A case from Malawi. Glob Health Action. Taylor and Francis Ltd.2017;10(1):1368236. 28914165 10.1080/16549716.2017.1368236 5645652 Raiman L Antbring R Mahmood A : WhatsApp messenger as a tool to supplement medical education for medical students on clinical attachment. BMC Med Educ. BioMed Central Ltd.2017;17(1):7. 28061777 10.1186/s12909-017-0855-x 5219809 Rathbone AP Norris R Parker P : Exploring the use of WhatsApp in out-of-hours pharmacy services: A multi-site qualitative study. Res Social Adm Pharm. Elsevier Inc.2020;16(4):503–510. 31278013 10.1016/j.sapharm.2019.06.019 Rosenfeld A Sina S Sarne D : WhatsApp usage patterns and prediction of demographic characteristics without access to message content. Demogr Res. 2018;39(22): 647–670. 10.4054/DemRes.2018.39.22 Willemse JJ : Undergraduate nurses reflections on Whatsapp use in improving primary health care education. Curationis. AOSIS.2015;38(2):1512. 26304053 10.4102/curationis.v38i2.1512 6091559 Willemse JJ Jooste K Bozalek V : Experiences of undergraduate nursing students on an authentic mobile learning enactment at a higher education institution in South Africa. Nurse Educ Today. Churchill Livingstone.2019;74: 69–75. 30594902 10.1016/j.nedt.2018.11.021 10.21956/gatesopenres.15873.r33229 Reviewer response for version 2 Guthrie Brandon 1 Referee Global Health, University of Washington, Seattle, Washington, USA

Competing interests: No competing interests were disclosed.

 11 9 2023 Copyright: © 2023 Guthrie B 2023 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

Overall comments:

This methodological manuscript describes the implementation of focus group discussions (FDGs) among youth in Malawi using the WhatsApp messaging platform. The manuscript briefly describes the approach used to implement WhatsApp FGDs and effectively summarizes the advantages and disadvantages of this approach. This manuscript provides a valuable addition to the scientific literature regarding this innovative approach that has the potential to increase participation an engagement in FGDs, particularly on sensitive topics. My primary recommendation would be to provide some additional description of how the FGDs were implemented so that others could replicate or improve on this approach.

Additional comments and recommendations are listed below:

Abstract:

It would be helpful to have a brief summary of some of the key takeaways regarding “lessons learned and advantages and disadvantages of using WhatsApp to conduct FGDs” rather than just saying that the paper includes this information. Ideally, the abstract should contain a summary of the findings.

Approach:

Additional details about how the in-person and remote FDGs were actually conducted would be very helpful. This would include things like: For the in-person FGDs, were the prompts spoke out loud or only in text over WhatsApp?

Is the rationale for developing the new method (or application) clearly explained?

Yes

Is the description of the method technically sound?

Partly

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Yes

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes

Are sufficient details provided to allow replication of the method development and its use by others?

Partly

Reviewer Expertise:

Epidemiology, HIV/AIDS, eHealth

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

 10.21956/gatesopenres.15873.r33512 Reviewer response for version 2 Estrada Fatima 1 Referee National Council of Humanities Science and Technology, National Institute of Public Health of Mexico, Cuernavaca, Mexico

Competing interests: No competing interests were disclosed.

 18 7 2023 Copyright: © 2023 Estrada F 2023 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

The main theme of the document about the inclusion of digital communication in the area of ​​health research is very relevant. Describe the advantages and disadvantages of the use of an application so widely used worldwide, such as whatsapp, it is still an original topic in the area of ​​qualitative methodology.

The introduction is clear and concise, although I consider that it is necessary to include more information about the analysis that other authors have shown about the use of whatsapp in qualitative health research. I share some references that can be consulted:

 

Anderson E, Koss M, Castro Luque AL, Garcia D, Lopez E, Ernst K (2021). WhatsApp-Based Focus Groups Among Mexican-Origin Women in Zika Risk Area: Feasibility, Acceptability, and Data Quality.  JMIR Formative Research, 5(10), e20970. https://doi.org/10.2196/20970 1 .

Colom A. (2021). Using WhatsApp for focus group discussions: Ecological validity, inclusion and deliberation.  Qualitative Research. https://doi.org/10.1177/1468794120986074 2 .

Gibson K. (2020). Bridging the digital divide: Reflections on using WhatsApp instant messenger interviews in youth research.  Qualitative Research in Psychology. https://doi.org/10.1080/14780887.2020.1751902 3 .

Neo, P. H., Lim, J. M., Tan, R. K., & Ong, S. E. (2022). Using WhatsApp Focus Group Discussions for Qualitative Data Collection: Exploring Knowledge, Attitudes, and Perceptions of COVID-19 in Singapore. International Journal of Qualitative Methods, 21. https://doi.org/10.1177/16094069221090355 4 .

The authors describe clear and in a well manner the procedures followed to develop the study. However, being a methodological article, it would be necessary to include other information about the process, results and interpretation of the application of focus groups through a digital platform as whatsapp. Particularly, it would be useful to specify the reasons for certain decisions, and their methodological implications. In specific, there is no clarification in the document about how and why the researcher decided to do more groups in person, than remote. And in the remote groups, how did they decide follow the discussion during a month? Also, it is necessary to specify what was the objective of exploring the application in person and remotely. Was part of the objective to observe differences between these two approaches?

In Lessons Learned section, I believe the analysis of using the WhatsApp application to collect information through focus groups could be more exhaustive. General issues are mentioned and the situations that occurred throughout the application of the methodology at the different levels of stratification are not delved into, for example, the same difficulties arose regarding the categories of analysis of data collection?, data quality?, and data analysis?, between the three geographic districts, the rural and urban area, and in the groups according to marital status?. Likewise, the differences between having applied it in person or remotely are not very in-depth analyzed in the text. For example, about the advantage that arises from a greater amount of information obtained in remote focus groups. It is more information, but diminishes the quality?, or the quality is same between in person and remote?

There are questions that arise during reading that should be addressed by the authors.

Is the rationale for developing the new method (or application) clearly explained?

Yes

Is the description of the method technically sound?

Yes

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Partly

Are sufficient details provided to allow replication of the method development and its use by others?

Partly

Reviewer Expertise:

Adolescent sexual reproductive health

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

References : WhatsApp-Based Focus Groups Among Mexican-Origin Women in Zika Risk Area: Feasibility, Acceptability, and Data Quality. JMIR Form Res .2021;5(10) : 10.2196/20970 e20970 34709185 10.2196/20970 : Using WhatsApp for focus group discussions: ecological validity, inclusion and deliberation. Qualitative Research .2022;22(3) : 10.1177/1468794120986074 452-467 10.1177/1468794120986074 : Bridging the digital divide: Reflections on using WhatsApp instant messenger interviews in youth research. Qualitative Research in Psychology .2022;19(3) : 10.1080/14780887.2020.1751902 611-631 10.1080/14780887.2020.1751902 : Using WhatsApp Focus Group Discussions to Collect Qualitative Data Collection During a Pandemic: Exploring Knowledge, Attitudes, and Perceptions of COVID-19 in Singapore. Int J Qual Methods .2022;21: 10.1177/16094069221090355 16094069221090355 35437433 10.1177/16094069221090355 10.21956/gatesopenres.14368.r29954 Reviewer response for version 1 Gilliam Melissa 1 Referee Center for Interdisciplinary Inquiry and Innovation in Sexual and Reproductive Health, Section of Family Planning and Contraceptive Research, University of Chicago, Chicago, IL, USA

Competing interests: No competing interests were disclosed.

 30 11 2020 Copyright: © 2020 Gilliam M 2020 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation reject

In this article, the authors consider WhatsApp for collecting qualitative data from adolescents.

There are interesting things to learn and the paper would be greatly enhanced if more information was provided up front regarding the purpose of the paper. Specifically, the authors should clarify that it is a methods paper in which the research team and facilitators describe their experience.

Introduction:

In the introduction, there is a bit of overstatement. The following are some examples but please review the manuscript to consider this concern, as follows:

In the first paragraph of the intro, please change declarative sentences such as “candid youth perspectives are necessary…” perhaps instead “can help in understanding…” - are candid perspectives necessary?

Similarly, why candid?

Similarly, “more precise data” - is precision really the issue? It is not clear how one would know that.

Is WhatsApp popular across the entire world? Perhaps popular in many countries.

 Approach:

In-person participants were mixed gender—correct this to say that the sessions were mixed gender not the participants.

In this section, start with the design rather than the results of the FGD. The number of participants and demographics are part of the study findings and the selection of YRs is part of the design.

The approach section should be reordered chronologically. Also, it is not clear until the end of the approach study that the paper will not include results from the FGD. Thus, the focus of the paper needs to be made clear up front. That this is about the research team’s experience using WhatsApp.

The use of 3 -10 hour discussion seems unusual why were boundaries not up in place to gain greater uniformity of data collection?

The word “scenarios” is not made clear, what types of scenarios were being commented upon?

Please describe the debrief process in more detail. It will be important to know that there was a systematic approach.

 Lessons Learned:

This section should be organized thematically

There are some new ideas introduced such as it is not clear what a WhatsApp group compared to the traditional FGDs earlier in the study. What are the traditional groups?

Was data collected on income so we can understand who owns phones?

Not all of the concerns mentioned are inherent to the platform, some are due to study design. For example, economic status could be controlled for through recruitment inclusion criteria. People could have been instructed regarding whether recording was allowed.

 Conclusions:

There is no mention of IRB approval.

Is the rationale for developing the new method (or application) clearly explained?

Partly

Is the description of the method technically sound?

No

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

No

Are sufficient details provided to allow replication of the method development and its use by others?

No

Reviewer Expertise:

Youth reproductive health

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

Reynolds Katie

Competing interests: No competing interests were disclosed.

 3 3 2023

Dear Dr. Gillam, many thanks for your thorough and thoughtful review. Your feedback pushed us to clarify our writing and to strengthen the “considerations for future use.” We value and appreciate your suggestions and the time you invested in reviewing.  Please see our responses (in italics) and let us know if you have any questions or concerns about this next iteration that we are uploading and imagine will be published shortly. Sincerely,

Katie Reynolds

Melissa Gilliam, Center for Interdisciplinary Inquiry and Innovation in Sexual and Reproductive Health, Section of Family Planning and Contraceptive Research, University of Chicago, Chicago, IL, USA 

Melissa Gillam

Center for Interdisciplinary Inquiry and Innovation in Sexual and Reproductive Health, Section of Family Planning and Contraceptive Research, University of Chicago, Chicago, IL, USA

 NOT APPROVED

info_outline

In this article, the authors consider WhatsApp for collecting qualitative data from adolescents.

There are interesting things to learn and the paper would be greatly enhanced if more information was provided up front regarding the purpose of the paper. Specifically, the authors should clarify that it is a methods paper in which the research team and facilitators describe their experience. Clarified in abstract that paper is about the methods used and includes overall lessons learned.

Introduction:

In the introduction, there is a bit of overstatement. The following are some examples but please review the manuscript to consider this concern, as follows:

In the first paragraph of the intro, please change declarative sentences such as “candid youth perspectives are necessary…” perhaps instead “can help in understanding…” - are candid perspectives necessary?   Revised sentence to better convey our intention and removed “candid”

Similarly, why candid? Removed

Similarly, “more precise data” - is precision really the issue? It is not clear how one would know that. Revised to “detailed”

Is WhatsApp popular across the entire world? Perhaps popular in many countries. Revised to say many areas of the world, including Africa

Approach:

In-person participants were mixed gender—correct this to say that the sessions were mixed gender not the participants. Revised

In this section, start with the design rather than the results of the FGD. The number of participants and demographics are part of the study findings and the selection of YRs is part of the design. Re-ordered the entire section such that we begin with recruiting YRs, training YRs, writing scenarios, recruiting and consenting participants, debrief procedures, number of participants and demographics.  We decided to keep participants and demographics in approach section since results is focused on the general lessons learned and not the findings from the FGDs themselves.

The approach section should be reordered chronologically. Also, it is not clear until the end of the approach study that the paper will not include results from the FGD. Thus, the focus of the paper needs to be made clear up front. That this is about the research team’s experience using WhatsApp. Yes, great feedback that we incorporated. Re-ordered the entire section such that we begin with recruiting YRs, training YRs, writing scenarios, recruiting and consenting participants, debrief procedures, number of participants and demographics.  We decided to keep participants and demographics in approach section since results is focused on the general lessons learned and not the findings from the FGDs themselves. Added clarification at beginning of the paper that the paper focuses on general lessons learned.

The use of 3 -10 hour discussion seems unusual why were boundaries not up in place to gain greater uniformity of data collection? Added context that length varied based on participant and facilitator availability, as well as the group’s level of engagement throughout the week

The word “scenarios” is not made clear, what types of scenarios were being commented upon? Added context and examples to explain the scenarios.

Please describe the debrief process in more detail. It will be important to know that there was a systematic approach. Added what the specific prompts were for each debrief discussion.

Lessons Learned:

This section should be organized thematically We kept this section organized by “data collection, data use, and data quality” and revised the introduction to this section to make it clear that is how we are organizing the section.

There are some new ideas introduced such as it is not clear what a WhatsApp group compared to the traditional FGDs earlier in the study. We also did face to face FGDs without the use of technology through the course of this project, but I think it is confusing to introduce at this stage and the comparison of data quality between the two was not a rigorous process. We have revised to state that detail in WhatsApp groups was strong and sufficient, rather than comparing to other groups.     What are the traditional groups? Revised to add “in-person” to more clearly convey what we meant by use of “traditional.”

Was data collected on income so we can understand who owns phones? We did not collect income data from the youth participants.

Not all of the concerns mentioned are inherent to the platform, some are due to study design. For example, economic status could be controlled for through recruitment inclusion criteria.  We revised this section to state that the nature of the platform requires that people have a smartphone instead of making assumptions about income and smart phone ownership. We provided study phones during the in-person WhatsApp FGDs, and could have decided to provide smartphones for the remote groups but opted not to for logistics and cost.  We added a sentence to “considerations for future use” to encourage others to think about whether or not they would supply smartphones and if not, to consider what implications this has on who their resulting participants were.  People could have been instructed regarding whether recording was allowed. Agree that we could have mitigated this; the lesson learned is about how to do it differently and not necessarily only what’s inherent to the platform but unforeseen (to us) challenges about how to utilize it.  We added a sentence in “considerations for future use” that states researchers will want to consider guidelines around this.

Conclusions:

There is no mention of IRB approval. We had included this in the approach section and have made it more apparent.

Is the rationale for developing the new method (or application) clearly explained?

Partly

Is the description of the method technically sound?

No

Are sufficient details provided to allow replication of the method development and its use by others?

No

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

No

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Youth reproductive health

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

10.21956/gatesopenres.14368.r29578 Reviewer response for version 1 Vu Henry Jade 1 Referee https://orcid.org/0000-0001-7026-1351 Centre for Invention and Social Process, Goldsmiths, University of London, London, UK

Competing interests: No competing interests were disclosed.

 12 10 2020 Copyright: © 2020 Vu Henry J 2020 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

The paper describes a methodological approach that is valuable for: (1) generating additional insights to improve access to ASRH services; and (2) supporting and advancing the production of knowledge by Malawian youth, for Malawian youth, using the digital tools and media that they prefer. This emphasis on youth participation aligns well with recent calls to decolonize global public health research and practice (see Abraham et al., 2020 1 ; Bertram et al., 2020 2 ; Büyüm et al., 2020 3 ). To further consider these emancipatory possibilities, you might refer to this special issue (from the education sector) on voice and representation in digital media production (Dussel & Dahya, 2017 4 ), which I think resonates strongly with the research described this paper.

As you observe, WhatsApp has been deployed in global health operations widely but more research is needed to understand how to integrate social media platforms into qualitative research methods. This paper helps to fill that gap and once additional ‘ detail provided to enable others to improve the method’ is provided', it will make an important contribution to the field.

 Recommended Revisions:

My major concern with the paper is that concepts of privacy, anonymity, and confidentiality are not adequately explained/operationalized (see Hunter et al., 2018 5 ) for a helpful review). Informed consent was secured and the study protocol was approved, so the issue is not so much whether these issues were addressed, but rather how they were addressed. The reader needs to understand this in order to replicate and build on your work.

I think my reservations along this line can be addressed by making the following changes: 

 In the first paragraph of the Introduction section, you write “Technology may provide an avenue for collecting this ASRH data and with added privacy.” Explain how you are using the word "privacy" here – is it referring to the privacy of study participants in relation to researchers? In relation to other focus group participants? Family members? Other peers? Other research methodologies? Clarify whether this claim only applies to focus groups, or whether it also pertains to other qualitative methods such as interviews and photo-elicitation. Be more specific about the ways in which we might expect technology to enhance "privacy".

In the Approach section, provide more detail about how your study protocol protected the "privacy" of your study participants. You mention the institutional review board in this section, and I think privacy in this respect means something slightly different than what was described in the Introduction. How was the privacy of study subjects protected? Was it possible to assure participants anonymity or confidentiality? If so, what measures were adopted to achieve this? Did those measures differ according to whether FGDs were in-person or remote?

 In last sentence of the section Consideration for future: (1) use observations from your debriefing summaries and the Lessons learned section to further develop and support the claim about anonymity and privacy that is made here, and (2) move the references from Limaye et al. and Munthali et al. to the Introduction, in support of your explanation of privacy there.

 In the last sentence of the Conclusion/discussion section, explain what is meant by ‘data security and privacy’. Here I believe you are referring to privacy in relation to the issue of hacking and platform capitalism? This use of the word ‘privacy’ needs to be further elaborated and distinguished from the other ways that the term is used in the paper.

To give the reader a better understanding of how your method was developed and deployed, provide references to the qualitative methodological scholarship that underpins your focus group approach.

I think that the Approach section provides all of the individual elements that are necessary to replicate your method, but I had trouble getting an overall picture of what was done. If the format of this publication allows, I think adding the following 15x9 table with the following column headings would be very helpful:

session id #  (1-15)

in-person or remote?

geographic region

urban or rural?

location

# participants

% female

% married

% with prior WhatsApp experience

  

You’ve described how your study participants occasionally used the voice recording features of WhatsApp rather than relying only on its text-based functions, and you’ve also written about the difficulties of transcribing and analyzing that data. I think these are important findings and I would like to see more discussion about how your study participants appropriated the different capabilities of the WhatsApp technology. More specifically, I’d like to know the extent to which participants posted photos, videos, and emoticons, and whether you were able to include such material in your data analysis. Research in the fields of education and media studies suggest that these kinds of images are frequently used by youth to convey affect and address sensitive topics (Miltner & Highfield, 2017 6 ; Sarwatay, 2020 7 ; Wargo, 2017 8 ) and there are approaches to visual data analysis (Hand, 2016 9 ) that might be useful in the setting of global health programs.

Is the rationale for developing the new method (or application) clearly explained?

Yes

Is the description of the method technically sound?

Partly

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

No source data required

Are sufficient details provided to allow replication of the method development and its use by others?

Partly

Reviewer Expertise:

education, digital technology, public health evaluation,  science and technology studies, feminist theory

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

References : What IsGlobal Health: Science and Practice Doing to Address Power Imbalances in Publishing?. Global Health: Science and Practice .2020;8(3) : 10.9745/GHSP-D-20-00453 325-326 10.9745/GHSP-D-20-00453 : Silenced Voices in Global Health. ThinkGlobalHealth .2020; Reference source : Decolonising global health: if not now, when?. BMJ Glob Health .2020;5(8) : 10.1136/bmjgh-2020-003394 32759186 10.1136/bmjgh-2020-003394 : Introduction: problematizing voice and representation in youth media production. Learning, Media and Technology .2017;42(1) : 10.1080/17439884.2016.1205602 1-7 10.1080/17439884.2016.1205602 : Ethical Issues in Social Media Research for Public Health. Am J Public Health .108(3) : 10.2105/AJPH.2017.304249 343-348 29346005 10.2105/AJPH.2017.304249 : Never Gonna GIF You Up: Analyzing the Cultural Significance of the Animated GIF. Social Media + Society .2017;3(3) : 10.1177/2056305117725223 10.1177/2056305117725223 : “We Don't Do That Here” and “Isme Tera Ghata, Mera Kuch Nahi Jata”.2020; 10.1002/9781119166900.ch6 85-95 10.1002/9781119166900.ch6 : “Every selfie tells a story …”: LGBTQ youth lifestreams and new media narratives as connective identity texts. New Media & Society .2017;19(4) : 10.1177/1461444815612447 560-578 10.1177/1461444815612447 : Visuality in Social Media: Researching Images, Circulations and Practices.2016; 10.4135/9781473983847.n14 215-231 10.4135/9781473983847.n14 Reynolds Katie

Competing interests: No competing interests were disclosed.

 3 3 2023

Dear Dr. Vu Henry, many thanks for your thorough and thoughtful review. Your specific feedback helped us articulate how others might replicate the process, and what they might learn from it.  We appreciate your insights and time.  Please see our responses (in italics) and let us know if you have any questions or concerns about this next iteration that we are uploading and imagine will be published shortly.  Sincerely,

Katie Reynolds

Jade Vu Henry

Centre for Invention and Social Process, Goldsmiths, University of London, London, UK

 APPROVED WITH RESERVATIONS

info_outline

The paper describes a methodological approach that is valuable for: (1) generating additional insights to improve access to ASRH services; and (2) supporting and advancing the production of knowledge by Malawian youth, for Malawian youth, using the digital tools and media that they prefer. This emphasis on youth participation aligns well with recent calls to decolonize global public health research and practice (see Abraham et al., 2020 1 ; Bertram et al., 2020 2 ; Büyüm et al., 2020 3 ). To further consider these emancipatory possibilities, you might refer to this special issue (from the education sector) on voice and representation in digital media production (Dussel & Dahya, 2017 4 ), which I think resonates strongly with the research described this paper.

As you observe, WhatsApp has been deployed in global health operations widely but more research is needed to understand how to integrate social media platforms into qualitative research methods. This paper helps to fill that gap and once additional ‘ detail provided to enable others to improve the method’ is provided', it will make an important contribution to the field.

 Recommended Revisions:

My major concern with the paper is that  concepts of privacy, anonymity, and confidentiality are not adequately explained/operationalized (see Hunter et al., 2018 5 ) for a helpful review). Informed consent was secured and the study protocol was approved, so the issue is not so much  whether these issues were addressed, but rather  how they were addressed. The reader needs to understand this in order to replicate and build on your work.

I think my reservations along this line can be addressed by making the following changes: 

 In the first paragraph of the  Introduction section, you write “Technology may provide an avenue for collecting this ASRH data and with added privacy.” Explain how you are using the word "privacy" here – is it referring to the privacy of study participants in relation to researchers? In relation to other focus group participants? Family members? Other peers? Other research methodologies? Clarify whether this claim only applies to focus groups, or whether it also pertains to other qualitative methods such as interviews and photo-elicitation. Be more specific about the ways in which we might expect technology to enhance "privacy".  Great suggestion to reference the Hunter article, as we were using “privacy” too loosely.  We in fact mean “anonymity” when interfacing with fellow participants and researchers, and we have revised accordingly.

In the  Approach section, provide more detail about how your study protocol protected the "privacy" of your study participants. You mention the institutional review board in this section, and I think privacy in this respect means something slightly different than what was described in the Introduction. How was the privacy of study subjects protected? Was it possible to assure participants anonymity or confidentiality? If so, what measures were adopted to achieve this? Did those measures differ according to whether FGDs were in-person or remote? Yes, we meant “anonymity” in the introduction and in approach section we mean “anonymity” and “confidentiality.” We have revised to clarify and have shared what measures were in place to safeguard both anonymity and confidentiality. We provided additional context on safeguards for and potential shortcomings related to anonymity and confidentiality and in-person and remote forums.

 In last sentence of the section  Consideration for future: (1) use observations from your debriefing summaries and the Lessons learned section to further develop and support the claim about anonymity and privacy that is made here, and (2) move the references from Limaye et al. and Munthali et al. to the Introduction, in support of your explanation of privacy there.  Removed the mention of “privacy” here because what we really meant was anonymity. Added context on why we think what we observed indicates that anonymity may be an important factor in youth sharing sensitive information.

 In the last sentence of the Conclusion/discussion section, explain what is meant by ‘data security and privacy’. Here I believe you are referring to privacy in relation to the issue of hacking and platform capitalism? This use of the word ‘privacy’ needs to be further elaborated and distinguished from the other ways that the term is used in the paper. We fixed “privacy” earlier in the paper so that it can stand as “privacy” here. Added context to share that we need to understand privacy considerations to be able to fully inform participants during consenting process.

To give the reader a better understanding of how your method was developed and deployed, provide references to the qualitative methodological scholarship that underpins your focus group approach. Added to methods section that community based participatory research informed how we approached the work overall and designing FGDs with the Youth Researchers.

I think that the Approach section provides all of the individual elements that are necessary to replicate your method, but I had trouble getting an overall picture of what was done. If the format of this publication allows, I think adding the following 15x9 table with the following column headings would be very helpful:  Great idea. We’ve added a table that shares the design. We don’t have current access to the specific demographic breakdown of participants so limited it to a 15x5 table.

session id #  (1-15)

in-person or remote?

geographic region

urban or rural?

location

# participants

% female

% married

% with prior WhatsApp experience

  

You’ve described how your study participants occasionally used the voice recording features of WhatsApp rather than relying only on its text-based functions, and you’ve also written about the difficulties of transcribing and analyzing that data. I think these are important findings and I would like to see more discussion about how your study participants appropriated the different capabilities of the WhatsApp technology. More specifically, I’d like to know the extent to which participants posted photos, videos, and emoticons, and whether you were able to include such material in your data analysis. Research in the fields of education and media studies suggest that these kinds of images are frequently used by youth to convey affect and address sensitive topics (Miltner & Highfield, 2017 6 ; Sarwatay, 2020 7 ; Wargo, 2017 8 ) and there are approaches to visual data analysis (Hand, 2016 9 ) that might be useful in the setting of global health programs.  Our participants did not incorporate videos, photos, etc. and relied on the text and voice memos features. I included a comment on this in the “Considerations for future use” section.

Is the rationale for developing the new method (or application) clearly explained?

Yes

Is the description of the method technically sound?

Partly

Are sufficient details provided to allow replication of the method development and its use by others?

Partly

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

No source data required

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

References

1. Abraham S, Hodgins S, Saad A, Fabic M: What IsGlobal Health: Science and Practice Doing to Address Power Imbalances in Publishing?.  Global Health: Science and Practice. 2020;  8 (3): 325-326  Publisher Full Text

2. Bertram K, Erondu N, Pai M: Silenced Voices in Global Health.  ThinkGlobalHealth. 2020.  Reference Source

3. Büyüm AM, Kenney C, Koris A, Mkumba L, et al.: Decolonising global health: if not now, when?.  BMJ Glob Health. 2020;  5 (8).  PubMed Abstract |  Publisher Full Text

4. Dussel I, Dahya N: Introduction: problematizing voice and representation in youth media production.  Learning, Media and Technology. 2017;  42 (1): 1-7  Publisher Full Text

5. Hunter RF, Gough A, O'Kane N, McKeown G, et al.: Ethical Issues in Social Media Research for Public Health. Am J Public Health.  108 (3): 343-348  PubMed Abstract |  Publisher Full Text

6. Miltner K, Highfield T: Never Gonna GIF You Up: Analyzing the Cultural Significance of the Animated GIF.  Social Media + Society. 2017;  3 (3).  Publisher Full Text

7. Sarwatay D: “We Don't Do That Here” and “Isme Tera Ghata, Mera Kuch Nahi Jata”. 2020. 85-95  Publisher Full Text

8. Wargo J: “Every selfie tells a story …”: LGBTQ youth lifestreams and new media narratives as connective identity texts.  New Media & Society. 2017;  19 (4): 560-578  Publisher Full Text

9. Hand M: Visuality in Social Media: Researching Images, Circulations and Practices. 2016. 215-231  Publisher Full Text

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

education, digital technology, public health evaluation,  science and technology studies, feminist theory

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.