Using the HIV Prevention Self-Assessment Tools (PSAT) to assess and monitor sex workers HIV programmes in selected countries in Africa

Introduction

The 2016 United Nations Political Declaration on HIV and AIDS proposed a fast-track strategy to end the AIDS epidemic by 2030¹. UN Member States committed to a 75% reduction in the number of new HIV diagnoses by 2020 compared to 2010, by reducing the annual number of new HIV diagnoses to 500,000 by 2020². Despite significant scale-up of antiretroviral therapy (ART) and a substantial achievement towards reaching the global 90-90-90 targets, the rate of new HIV diagnoses stayed higher than what was desired to successfully reach these prevention targets³. The United Nations 2021 Political Declaration on HIV and AIDS further reiterated the global commitment to end all inequalities faced by people living with, vulnerable to and affected by HIV. The declaration also recognised that key populations are more likely to be exposed to HIV or are living with HIV and provided cross-cutting commitments and targets based on strategic priorities of the Global AIDS Strategy 2021–2026⁴. To support global efforts by accelerating HIV prevention, the Global HIV Prevention Coalition (GPC) was formed, which was co-convened by UNAIDS and UNFPA⁵. The GPC seeks to generate commitment, speed, investment and accountability across different stakeholders while systemising a technical support function to intensify progress on HIV prevention. Initially focusing on 25 countries with the highest numbers of new HIV diagnoses, the GPC expanded to 28 countries in 2021⁶. The GPC endorsed a HIV Prevention 2020 Road Map to achieve the UN political declaration targets, rallying the scaling up of combination prevention programmes for key population groups². A South to South Learning Network (SSLN) was established in 2020 with the aim to support the GPC to focus on countries with the highest HIV rates in the global south and strengthen country HIV prevention programmes through shared learning and networking⁷.

The HIV Prevention Self-Assessment Tools (PSATs) were developed by the GPC as an easy-to-use tool for country-led review of national HIV programmes against a global standardised set of programmatic components⁸. The PSATs were developed through an iterative and collaborative process that involved an extensive literature review to identify guidance on best practices, key informant interviews with content experts, piloting of the tool with countries, and final verification with multiple key stakeholders. Separate PSATs were developed for each of the five thematic pillars of HIV prevention highlighted by the HIV Prevention 2020 Road Map⁹. The SSLN countries used the PSATs as a first step to self-assess their progress on condom use and key population HIV prevention programming⁹. This paper presents a comprehensive overview of the process and an analysis of the PSAT conducted for the sex worker (SW) programmes in 10 focused GPC countries in Africa.

Methods

As part of the SSLN, country-level data on HIV prevention programmes for sex workers were collected through the GPC’s SW-PSAT tool⁸. There are PSATs focused on other key populations, however, the SSLN focused on sex workers in its first phase of operation. The PSAT is structured as a rubric that is used to score the three “domains” of the HIV prevention programme. The domains include Programme Management, Programme Implementation and Programme Outcomes and Sustainability, each of which comprises essential “functions” required for a successful HIV prevention programme. Each function is comprised of a set of “elements” i.e., activities and services provided by the programme. The PSAT provided country HIV prevention stakeholders with an opportunity for an in-depth discussion on their progress in each domain and reaching a consensus on a score that best reflects the country’s performance.

In addition to data collected through PSATs, in-depth interviews were conducted with key informants from SSLN member countries.

The SSLN is an initiative of the GPC that aims to support countries to strengthen their HIV prevention programmes through shared learning and networking. It was conceptualized to test the idea of using peer learning in a network structure to improve capacity building efforts for HIV prevention across 10 countries during its Phase I (March 2020 – February 2022), but now has scaled-up to implement this concept across 15 African countries in its second phase (March 2022 – February 2025).

Countries opt-into being part of the SSLN and nominate country champions who represent various HIV preventions stakeholders within the country to directly engage with the SSLN and represent their constituencies in the network. As a routine SSLN programme activity, countries complete a HIV PSAT to reflect and assess their Key Populations HIV prevention programme at a national-level. The PSATs were developed by UNAIDS and the process that was followed to complete the PSAT within the SSLN is a process that is recommended by UNAIDS, who are also the co-chairs of the SSLN advisory board. The PSATs are widely shared on the UNAIDS website.

PSATs are meant for country-led review of national programmes in each of the five thematic pillars of HIV prevention and provide a comprehensive overview on the status and quality of programming and implementation in a thematic area. It is expected that such country-led reviews will be done on a regular basis to routinely monitor the prevention programmes. Seeing that the PSAT completion process is not based on any personal data and is a self-assessment of a national programme not requiring any primary data collection, ethical approval was not obtained. Completing the PSAT is a routine programme activity that countries who have opted into the SSLN complete upon joining the programme. The data in this manuscript are from a reflection exercise countries have done of their HIV prevention programme and does not represent any individuals but is seen and analysed at a national level.

Similarly, the key informant interviews that were conducted were part of a routine monitoring and evaluation activity of the SSLN programme to assess the programme’s ability to meet country needs and gauge the experience of countries within the network. We aim to conduct these interviews annually throughout the duration of the programme to remain responsive and adapt to the evolving country needs. Participation in the interview was voluntary. Country champions were contacted via email and were invited to participate in a virtual interview. Any country champion in the network was eligible to participate after confirming their participation via email. An interview guide was developed and used for all interviews. Data were anonymous and de-identified. Ethical approval was not obtained in this case as the country champions reflected on the SSLN programme and their country HIV prevention programmes and personal information was not collected.

Results

In total, 10 PSATs were included in the analysis and Figure 1 shows self-rated country PSAT scores along with domain scores, out of a possible score of five^12,13. The bars represent the total PSAT country score, while each coloured dot represents the various domain functions i.e., Programme Management, Programme Implementation and Programme Outcomes and Sustainability. The PSAT scores ranged from the highest score of 4.1 (Kenya) to the lowest score of 2.3 in Zambia. Five of the selected countries (Kenya, Ghana, Zimbabwe, South Africa and Mozambique) scored above 3, while the remaining five scored below 3. Of the three domains, Programme Management achieved the highest scores, with four countries (Kenya, Ghana, Zimbabwe, and South Africa) scoring more than 4. Kenya and Ghana had a high score in Programme Implementation as well, while five countries (Zimbabwe, South Africa, Mozambique, Malawi and Uganda) scored between 3 to 4. Tanzania, Nigeria and Zambia scored below 3. All countries except Kenya scored fairly poorly in the third domain of Programme Outcomes and Sustainability. It should be noted that within this domain, Kenya scored themselves only on the sustainability elements. As such their domain score is not directly comparable to other countries. The scores ranged between 3.8 in Kenya to 1.5 in Zambia, with nine out of the 10 countries scoring below 3.

Figure 1. Self-rated PSAT scores for sex worker programmes in selected countries in Africa.

PSAT, Prevention Self-Assessment Tools.

As mentioned, each “function” is made up of a set of “elements” i.e., activities and services provided by the programme. Figure 2 presents details on each of the programme elements scored between 1 to 5, with 5 being the highest score. Within the domain of Programme Management, all countries indicated having strong elements of leadership and coordination of their SW programmes. High levels of accountability and national KP strategies and policies were reported by most countries while costing and financial planning were unanimously reported as a challenge. Within Programme Implementation, various sub-domains made up the domain scores. For most countries, behavioural interventions were reported to have performed well, and the availability of a comprehensive clinical service package including HIV testing, treatment, counselling and support. Programme linkages with a hepatitis programme, substance use disorder services and mental health services were components most commonly lacking. The domain scored low in elements of structural interventions i.e., support for enabling and supportive environment, violence, stigma and discrimination etc. The Programme Outcomes and the Sustainability domain, on average, scored lower than the Programme Management and Programme Implementation domains in all countries other than Kenya, which did not provide complete scores in this section. These lower scores can be attributed to countries not having a sustainability plan and transition roadmap in place and some of the elements not being monitored such as hepatitis B (HBV) and C (HCV) testing, the proportion of people with concomitant HIV and HBV receiving combined treatment, and the proportion of people with concomitant HIV and HCV starting HCV treatment.

Figure 2. Key element scores within each PSAT domain for SW programmes in selected countries in Africa.

PSAT, Prevention Self-Assessment Tools; SW, sex worker.

Figures 3a and 3b below show Quadrant plots for PSAT scores with the percentage of SWs who used at least two HIV prevention interventions in the last two months and the percentage of SWs who reported the use of a condom with their most recent client. Our analysis shows that countries with the highest PSAT scores such as Kenya, Ghana, Zimbabwe, and South Africa have the highest proportion of SWs who used condoms and are seen in the right upper quadrant of the plot. Ghana and South Africa also had a high proportion of SWs, who used at least two HIV prevention interventions. The high PSAT scores that were calculated for Kenya and Zimbabwe, however, did not correlate with high programme coverage as shown by the proportion of SWs who used HIV prevention interventions.

Figure 3. Quadrant plots for SW programme-related PSAT scores with key Programme Outcomes and HIV Epidemic metrics for SSLN partner countries.

PSAT, Prevention Self-Assessment Tools; SW, sex worker; SSLN, South to South Learning Network.

Figures 3c and 3d show two of the key UNAIDS-suggested Epidemic Metrics plotted against PSAT scores. Countries with high PSAT scores such as Kenya, Zimbabwe and South Africa showed the highest reduction in HIV incidence from 2010 to 2019. All three countries also showed a lower Incidence Prevalence Ratio (IPRs) in 2019. Ghana however is one of the exceptions, that despite having a high PSAT score, apparently does not have good epidemic control shown by a poor IPR and a smaller reduction in HIV incidence.

Results of qualitative interviews with country champions were analysed in two thematic areas. Within the thematic area of “engagement and involvement in the SSLN activities and completing PSAT” respondents reflected on how useful the PSATs have been in better understanding their country programmes and having good oversight of their progress. Although the exercise was considered time-consuming and rigorous, the benefits of having done the PSAT outweighed the time cost of completing it.

“The most important success is to be able to examine ourselves as a country and looking at the situation of our HIV response as a team in an easy way. I call that a success because I was able to see … “ohh, this is how we are doing it and this is what is lacking”. So in reality it is assessing ourselves; you are not judged by anyone, you are judging yourself” (Key informant - Tanzania).

Within the thematic area of “relevance of the SSLN activities” respondents reported an increased awareness of in-country and peer programmes as the PSAT provided an opportunity to have a better overview of their progress and challenges and also to learn from peer countries. Countries reflected that they were able to better identify the gaps in their programme through the self-reflection process of completing the PSAT and by learning and hearing about how other countries are implementing their programmes. By identifying these gaps, they were able to set priorities and better manage their programmes going forward. However, to further ensure that the learning activities are relevant, it was advised by countries that the SSLN considers how the learnings can be actioned and applied after the learning events occur.

“We can’t over-emphasise the relevance of this in our country's context. Very eye-opening. Er, the PSAT gave us a kind of contextual view of, not just our KP, but as well HIV prevention, situation, and background priority-setting. So, first, as a programme manager of the county where I work, it gives me a lot of advantages to viewing the HIV response in totality. It kind of lifts me above, and look at HIV response with a different understanding” (Key informant - Kenya).

The key informants also recommended that the SSLN should a) provide the options for countries to opt-in to other key and behaviourally vulnerable population groups as countries highlighted a desire and need to complete PSATs for other key and behaviourally vulnerable groups i.e., adolescent girls and young women, people who inject drugs, transgender people and persons in prison and b) align the completion of PSATs with planning for new National Strategic Plans, Global Fund applications, annual performance/work plans or mid-term reviews for the greatest value.

Discussion

For an effective HIV prevention response, service delivery programmes need to be regularly assessed and evaluated to track the progress of the response. While several guidelines and evaluation tools already exist^14,15, PSAT provides an excellent opportunity for programme planners and implementors to self-assess the alignment and consistency of the programme with global guidance. The PSAT also helps the countries to identify their needs and strengths towards achieving global programme indicators and targets. As part of the SSLN, these countries became the first in the world to use the PSATs. Results of our study have shown that the tool is an easy-to-use method for countries to assess and monitor their progress toward comprehensive prevention programming and assists in identifying programme areas that need attention. It brought together all relevant stakeholders and quickly produced a snapshot of the SW programme at a national level while allowing for comparisons between programme elements and countries.

This study identified some of the key programmatic gaps and areas of concern among SW programmes. This is consistent with UNAIDS and GPC’s previous assessments that prevention services are not being provided on an adequate scale, with sufficient intensity, nor reaching those most in need¹⁶. While we were able to highlight a few country-specific prevention issues, several key challenges were identified unanimously across most countries. Although most countries have population size estimates available, these estimates may be lower than the actual population size, especially in countries where sex work is still criminalised by the State or where there are very high levels of stigma and discrimination against SW. For HIV prevention efforts to have an impact, increased emphasis on robust programme management systems, evidence‐driven programme design and implementation, strategic and iterative programme monitoring, and rigorous evaluation of outcomes are required¹⁷. Combinations of community-led behavioural and biomedical interventions can significantly reduce HIV transmission amongst SW¹⁸, but this requires an enabling and supportive environment¹⁹. Our analysis has shown that countries with higher PSAT scores e.g., Kenya, South Africa, Ghana and Zimbabwe not only have programmes based on a comprehensive “combination prevention approach” but have also included various active outreach, engagement and mobilisation practices of SWs as an integral part of the programme to reduce vulnerability^20–23. On the other hand, various structural barriers including violence, stigma and discrimination, which can discourage SW from accessing services whilst also holding back improvements to service coverage and quality were noticed by most countries. Likewise, prevention, screening and treatment of viral hepatitis, as well as issues of substance use and mental health are components lacking in most programmes. Programmes need to focus on the integration of the HIV prevention programme services with other health services, which tends to improve service uptake and programme outcomes significantly²⁴.

Generally, while most countries scored satisfactorily within the domains of Programme Management and Programme Implementation, the scores within the domain of Programme Outcomes and Sustainability were poor, highlighting that although there are good strategies, plans and programmes, several of these programmes are not implemented on a scale large enough to significantly impact the outcomes. This also highlights significant challenges faced by nearly all countries in collecting programme outcome-related quality data regularly, using it for decision-making, routinely monitoring programme coverage and conducting data quality assessments. These gaps in both the availability as well as the use of key populations data to inform HIV prevention and treatment programmes have already been highlighted by previous research^25,26. We also have noticed that most countries do not have a clear understanding of key indicators to measure the SW programme, and lack the technical capacity to estimate population size, report routinely, and the use of data for decision-making²⁷. These are critical areas for strengthening as countries attempt to scale up SW programmes.

Finally, the comparison of PSAT scores with UNAIDS suggested epidemic metrices and programme outcome indicators showed interesting results. We noticed a strong alignment between overall PSAT scores and epidemic metrics and programme outcomes in a few countries such as Kenya, South Africa and Zimbabwe, which reassured the notion that PSATs can be reliable tools for self-evaluating programmes. Countries such as Mozambique and Ghana, that showed a disconnect of PSAT scores with outcomes in the quadrant plots, might be indicative of the constraints and challenges experienced with leadership, management and coordination although the PSAT has scored high.

A few limitations of the approach need to be mentioned. Since this is a self-assessment tool and is based on countries’ own perceptions of their programmes, the potential of overrating or underrating the PSATs cannot be overruled. It is therefore important to obtain stakeholder consensus through a Technical Working Group or broader stakeholders' in-country review of the tool, to minimise this limitation. Furthermore, since this is an extremely comprehensive tool, its completion is a lengthy and time-consuming process and countries might find it challenging to repeat the process to track programmes’ progress regularly. Despite these limitations, all countries who participated in this study vetted the process and found it very useful to understand the larger HIV prevention landscape of their SW programmes.

To conclude, undertaking this exercise provided the countries with a chance to self-reflect on their programmes, and increased their level of ownership and confidence in those programmes. It allowed a holistic self-reflection of HIV prevention programmes not often found in traditional HIV prevention programme management, which can often be driven by results and donor requirements. We highly recommend that countries should use the PSATs to assess their prevention programmes to identify programmatic and knowledge gaps and strategize their response based on these assessments. When paired with thoughtful facilitation and peer learning, PSATs can unlock powerful new programmatic insights and have the potential to make HIV prevention programmes more responsive and community-focused.