Learner experiences of safety at public high schools in three South African townships: Baseline findings from the National School Safety Framework learner surveys

Design and setting

A cross sectional descriptive design was employed using the NSSF 152-question learner survey, designed by the Centre for Justice and Crime Prevention with the DBE and UNICEF South Africa (a copy of the survey instrument is available in Extended data²⁷). NSSF implementation and monitoring is part of routine DBE programming and is therefore not a research study. In line with this, the methodology and tools were predefined. Recruitment and data collection for the baseline survey was conducted in the Khayelitsha, Soweto and Thembisa townships, SA between March 2018 – April 2019. Khayelitsha is situated in the Cape Town metropole, in the Western Cape²⁸. Much of the population are Black African and 37.2% are under 19 years of age²⁸. Soweto, located to the south west of Johannesburg in Gauteng Province, has a population with 32.6% under 19 years²⁹. It comprises 84.2% formal dwellings and 6357 persons per km²²⁹. Thembisa, also located in Gauteng province, has a population with 29.2% under 19 years, 72.5% formal dwellings and 10 820 persons per km²³⁰. These sites and schools (26) were invited to participate as they were part of the ongoing GAP Year trial. The GAP Year school selection inclusion criteria were as follows: mixed sex government funded high schools in Khayelitsha, Soweto and Thembisa; in quintiles 1–3¹, which had not been exposed to any asset building interventions in the past six months. Supported by the School Safety Directorate in the DBE as well as the district DBE stakeholders and circuit managers, the team had various engagements with school principals of all 26 GAP Year schools to explain the NSSF survey, the benefits of the data collected and the whole NSSF implementation process. Following these engagements, only 15 of the 26 schools agreed to participate, whilst others had competing priorities at the time of the study.

Population and sampling technique

Grade 9–11 learners, ranging from 13 – 19 years, from 15 government-funded high schools participating in the GAP Year trial were surveyed, using the predefined NSSF methodology²⁴ (page 65); specifically, classes were randomly selected from all the classes in the participating grades, ensuring that at least 1 class per grade were selected, per school. All participants in that class were given the opportunity to participate, irrespective of sex, race or age. As per DBE directive²⁴, the survey excluded Grade 8 and grade 12 learners; grade 8 learners were new and less familiar to the school environment while grade 12 learners would have left the school when endline assessments were conducted in the following year.

Instrument and data collection

An existing DBE-approved learner survey²⁴ (pages 70 – 77), consisting of 152 questions, was used to collect data on experiences and perceptions of violence and safety within the ongoing GAP Year trial. The tool had Likert scale responses, and comprised of 8 sections, covering the following themes: (1) Dangerous objects; (2) Drugs and Alcohol; (3) Bullying; (4) Verbal abuse; (5) Physical violence; (6) Discrimination; (7) Sexual violence and (8) the journey to and from school. This survey sought to elicit learners’ experiences and perceptions of school safety and violence. The GAP Year research team led the recruitment of participants and went from classroom to classroom, clearly explaining the survey purpose to learners and invited them to participate. Learners self-completed the survey in their classroom, during school time, as agreed with the principal and class educator²⁴. The survey took approximately 30 minutes for learners to complete and no personally identifying information was collected on the survey tool, potentially reducing bias. While the survey was being completed, the research team sought to ensure that learners were not looking at each other’s answers. The team also circulated around the class to ensure that the questions were correctly understood.

Ethics statement

The NSSF is a nationally mandated program of DBE, and these data were collected on behalf of the DBE. The learner survey is part of routine DBE programming and not a research study, and under DBE’s jurisdiction does not require ethical approval. In line with this, the NSSF methodology was carefully applied to ensure learner safety.

The GAP Year trial did receive ethical approval from the Human Research Ethics Committee (HREC) based at the University of the Witwatersrand (#M160940).

Parental consent was not sought from learners participating in the NSSF learner survey for the following reasons: the schools wherein learners are enrolled serve in loco parentis and therefore provide guardianship for learners when participating in a myriad of DBE organised programmes with its partners. This responsibility, known as the ‘in loco parentis’ principle, tasks educators to act in the place of a parent by carrying out legal responsibilities and functions in line with the Fundamental Rights of Children in the Constitution of the Republic of South Africa (CRSA) of 1996. Educators have a duty-of-care and supervision to learners equal to the task as expected of parent(s); taking responsibility for the emotional, psychological and physical well-being of the learners to ensure there is no foreseeable risk of injury to the child.

No written informed consent was sought from learners, however the nature and purpose of the completion of the learner surveys were clearly explained to learners before participation and their assent was provided. Learners were provided with the aims of the study, guaranteed confidentiality, provided details on who will have access to the learners’ information, indicated that the survey is voluntary, provided information on the storage of data, and the dissemination of the findings. We did not collect the number who refused to participate but this could impact the study results as it may not be representative. Willing learners completed the survey at their desk in the classroom and the surveys were conducted anonymously: learner names and other personal information was not collected.

Due to the sensitive nature of the data collection tool, the GAP Year distress protocol was implemented with a social worker available to provide support. Learners who indicated distress during survey completion, were noted by the team and referred to the GAP Year social worker for further support and referrals. Willing learners were informed of their right to withdraw from the survey at any time if they felt the need to do so. It was explained to learners that they cannot pass or fail the survey, as there are no ‘’correct’’ answers. Withdrawal or non-participation from the survey did not affect the learners in any way.

Data analysis

The main outcome variables were experiences of violence and safety. A four-stepped process was employed in the analysis of data.

Firstly, given the presence of multi-dimensional data, Principal Component Analysis (PCA) was employed as a dimensionality reduction method, reducing large datasets into smaller uncorrelated variables known as “principal components”, whilst maintaining the variance explained by each component³¹. PCA assisted in reducing 152 questions that were highly correlated, into fewer questions, capturing the strong data patterns that represents the data well. To achieve this, eigenvectors accompanied by their associated eigenvalues (number that indicates how much variance is explained in the data) were first computed. PCA criteria is that components with an eigenvalue >1 should be retained for further analysis, providing more information on the questions to be kept, creating the final composite variables³².

Secondly, all the components obtained in each of the themes were analysed using a scree plot to select the number of relevant components to be considered in further PCA analysis. These were then further determined by identifying variables with the highest correlation and then calculating eigenvalues based on the correlation matrix.

Thirdly, PCA was then performed again limited now to components that had a minimum eigenvalue of 1, using the Orthogonal (Varimax) Rotation method which assisted in clarifying the relationship between the variables by placing them under the relevant components. The Varimax Rotation Method helped minimise the complexity of the factor loadings by isolating factors that had eigenvalues >1.0 and loading them into relevant items with accompanying total variances explained.

Lastly, to test for internal reliability, the Cronbach Alpha’s test was performed. A reliability measure of 0.6-0.7 or above is considered acceptable as it has been largely deemed that a higher alpha value translates into higher reliability^33–36. Based on this, item loadings that had a Cronbach Alpha score of 0.7 were retained in the PCA and used in creating the final 11 composite variables, measuring “violence and safety”. The 11 composite variables were as follows and included in the final analysis:

Appendix 1 outlines the set of questions combined to create outcomes based on Cronbach Alpha score (available in Extended data). The responses to each experience of violence were coded as (1) “Yes” for learners who had experienced any form of violence and (0) “No” for learners who had not experienced any form of violence. Descriptive analysis was then used to show the frequency distributions of each of the composite variables. A chi-square test of association (χ²) was then used to test for an association between the 11 composite variables and sex. A p-value <0.05 was considered statistically significant. Stata version 15³⁷ was used for all data management and analysis undertaken. Missing values were excluded to yield results only for those who had responded to all the key variables.

Experiences of violence and safety, by sex

Table 2 describes experiences of violence and safety in public high schools in South Africa, by sex and the 11 composite variables. Overall, almost 30% (29.1%, n=301) have experienced peer provocation and relational aggression: 7.2% (n=75) who reported that they experienced peer violence perpetration and victimisation once in the school term while 83.6% (n=864) had not experienced this type of violence at all. Fourteen percent (n=147) have experienced peer perpetrated sexual harassment and almost 50% (48.2%, n=498) who reported that they had a perception of feeling unsafe to and from school. More than 5% (n=56) reported that they had experienced identity-based bullying and 16.6% (n=172) experienced verbal abuse from a peer or educator once in the school term. While almost 90% of the participants (89.2%, n=922) were exposed to illicit drugs on the school property, 76.8% (n=794) reported that discipline was enforced in their schools. Over half of participants (58.3%, n=603) reported that they were able to comfortably report any form of violence to educators, and 78.2% (n=809) indicated that there was a code of conduct that pertained to various forms of violence at school. Half (51.6%, n=534) reported being exposed to life skills lessons at school.

The results show that there is a statistically significant association between 4 of the 11 composite variables in relation to sex: experience of peer provocation and relational aggression, perception of feeling unsafe to and from school, effectiveness of school management on learner safety (particular focus on enforcement of discipline at school) and ability to comfortably report any form of violence to educators and sex (p-value<0.05).

When looking at the differences in violence between females and males, over half (55%) of male learners have experienced peer provocation and relational aggression compared to 45% of female learners (p<0.001). Almost 60% of female learners (58%) reported feeling unsafe on their way to and from school, compared to 42% of male learners (p<0.003). Over half (55%) of male learners reported that their school was not effective in enforcing discipline relating to any form of violence: this contrasts with 55% of females noting the opposite (p<0.002). Most females (58%) noted the ability to comfortably report any form of violence to educators, compared to 44% of males (p<0.001).

Whilst the following do not show statistically significant sex differences, these findings are important to note. Female learners represent the largest proportion of those that experience peer violence perpetration and victimisation most days a term: 59% females vs 41% males. Over half (55%) of females and 44% of males have experienced peer perpetrated sexual harassment. Fifty-three percent (53%) of female learners and 47% of males had exposure to illicit drugs on the school property. Only half (54%) of female learners noted the presence of a code of conduct pertaining to various forms of violence at school: this was only 46% for male learners. Less than half of male learners (45%) reported exposure to life skills lessons at school: this rose to 55% for female learners.

Limitations

Data collection relied upon learner recall of experiences which may result in an under-reporting of violence⁶³. There was a larger sample size of females than males which may have skewed the data. We only gathered data from 15 schools across the 3 sites and therefore survey findings are not generalisable to the wider areas of Soweto, Thembisa and Khayelitsha. The 15 schools who did participate were perhaps not reflective of the sites we surveyed as there is a possible participation bias and an over representation in one of the sites. There was also a number of participants with missing data for specific variables: given that the PCA technique was employed to create composite outcomes for each variable of interest and the main objective was to make statistical inferences using a complete dataset and standardized the sample size for our observations of interest, missing data were excluded as it reduced the statistical power of the study resulting in biased results and invalid conclusions. Due to the nature of the study design, response rates were not available: this should be acknowledged as a limitation. Also, given the number of missing values that the dataset entailed, the missing data would have reduced the true representation of the sampled population which could have threatened the validity of the results obtained in the study, and thus result in invalid conclusions. Despite these limitations, the findings provide important insight into NSSF implementation and areas which require strengthening.

Opportunities to strengthen the administration of the NSSF survey tool and implemenation and recommendations for the NSSF learner tool

Following the use of the NSSF learner tool, we have noted a few limitations and recommendations for adaptation. Question amendments: currently, there is only ‘female’ and ‘male’ as sex options in the tool: we suggest this is amended to include ‘other’ as a sex category to ensure the tool is gender inclusive. When asking about violence, the tool does not ask the sex of the perpetrator and therefore cannot provide transformative interventions to address violence in schools: we suggest adding a field to gather the sex of the perpetrator of violence. Length of the survey tool: The reductive process undertaken in this analysis has allowed us to focus on fewer but relevant questions pertaining to safety and violence. As such we recommend shortening the survey tool, to only include a few questions for each key variable, capturing the most important summary scores for the outcomes of interest. This may also potentially increase the response rate and improve data quality by reducing multicollinearity errors. Content of the tool: The presentation of many variables (similar variables) may provide the same information which may appear to be significant but ultimately provide information that cannot be used to inform future interventions. The use of such a statistical technique adopted in this manuscript may thus assist in summarising the variance explained by each key variable that measures the main construct e.g., physical violence and also helps merge similar constructs together. COVID-19 adaptations: since the outbreak of the COVID-19 pandemic, we suggest the inclusion of various infection control protocols to be included in the NSSF tool. This will support in creating an enabling school environment.