This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on http://publichealth.jmir.org, as well as this copyright and license information must be included.
Population size estimates (PSEs) for hidden populations at increased risk of HIV, including female sex workers (FSWs), are important to inform public health policy and resource allocation. The service multiplier method (SMM) is commonly used to estimate the sizes of hidden populations. We used this method to obtain PSEs for FSWs at 9 sites in Zimbabwe and explored methods for assessing potential biases that could arise in using this approach.
This study aimed to guide the assessment of biases that arise when estimating the population sizes of hidden populations using the SMM combined with respondent-driven sampling (RDS) surveys.
We conducted RDS surveys at 9 sites in late 2013, where the Sisters with a Voice program (the program), which collects program visit data of FSWs, was also present. Using the SMM, we obtained PSEs for FSWs at each site by dividing the number of FSWs who attended the program, based on program records, by the RDS-II weighted proportion of FSWs who reported attending this program in the previous 6 months in the RDS surveys. Both the RDS weighting and SMM make a number of assumptions, potentially leading to biases if the assumptions are not met. To test these assumptions, we used convergence and bottleneck plots to assess seed dependence of RDS-II proportion estimates, chi-square tests to assess if there was an association between the characteristics of FSWs and their knowledge of program existence, and logistic regression to compare the characteristics of FSWs attending the program with those recruited to RDS surveys.
The PSEs ranged from 194 (95% CI 62-325) to 805 (95% CI 456-1142) across 9 sites from May to November 2013. The 95% CIs for the majority of sites were wide. In some sites, the RDS-II proportion of women who reported program use in the RDS surveys may have been influenced by the characteristics of selected seeds, and we also observed bottlenecks in some sites. There was no evidence of association between characteristics of FSWs and knowledge of program existence, and in the majority of sites, there was no evidence that the characteristics of the populations differed between RDS and program data.
We used a series of rigorous methods to explore potential biases in our PSEs. We were able to identify the biases and their potential direction, but we could not determine the ultimate direction of these biases in our PSEs. We have evidence that the PSEs in most sites may be biased and a suggestion that the bias is toward underestimation, and this should be considered if the PSEs are to be used. These tests for bias should be included when undertaking population size estimation using the SMM combined with RDS surveys.
In sub-Saharan Africa, female sex workers (FSWs) are at increased risk of HIV acquisition compared with the general population [
The service multiplier method (SMM) is a commonly used method to estimate the size of key populations. The method uses 2 data sources [
In recent applications, respondent-driven sampling (RDS) surveys have been used to obtain a probability-based estimate of the proportion of the target population who are service users [
In addition to the SMM, various approaches for population size estimation have been used, including the enumeration method [
In this paper, we build on existing guidance for implementing the SMM with RDS data [
We first describe the data sources used, our application of the SMM, and then our approach to investigating the degree to which our study met the methodological assumptions and the potential resulting biases.
Service data come from the
The probability-based sample comes from a baseline RDS survey of the Sisters Antiretroviral therapy Program for Prevention of HIV—an Integrated Response (SAPPH-IRe) trial, a cluster randomized controlled trial that was conducted among FSWs at 14 different sites across Zimbabwe in November and December 2013 (PACTR201312000722390) [
To initiate RDS recruitment, we purposively sampled 6 to 8 participants (
To determine
We applied the formula for the SMM,
The RDS-II estimator was used to estimate P [
As recommended, we used the delta method to estimate the variance of
The SMM makes at least four assumptions, including (1) all members of the population being counted should have a chance of being included in both sources [
For RDS-II estimates to be considered unbiased, assumptions including reciprocity, sampling with replacement, a completely connected networked population at each site, accurate report of personal network size, final sample independent of the original seeds, and random recruitment have to be satisfied [
Reciprocity is an assumption of the Markov process, which states that if individual A recruited individual B, then in principle, B could have recruited A [
Other potential biases in
We, therefore, investigated some of the RDS and SMM assumptions listed in
Respondent-driven sampling and service multiplier method assumptions.
Assumption | Criteria | Expected outcome | ||
|
||||
|
|
|||
|
|
Reciprocity (N/Ab) | Ask participants’ relationship to the person who gave them a study coupon and if they say |
Participants more likely to be recruited by friends and acquaintances. |
|
|
Sampling with replacement (N/A) | Always violated in real-life RDSc studies, when the RDS successive sampling estimator is not used. | —d |
|
|
Accurate report of personal network size (N/A) | Sensitivity analysis of different network size questions. | RDS estimates should agree with each other regardless of different network size questions used. |
|
|
Final sample independent of the original seeds | Assess whether seed dependence was removed using convergence plots. | Overall estimate of |
|
|
Completely connected networked population at each site | Assess whether the FSWe population is networked using bottleneck plots. | Estimate of |
|
|
Random recruitment | Assess whether there is an indication of nonrandom recruitment by measuring recruitment homophily. | Recruitment homophily should be approximately 1. |
|
Two data sources combined are drawn from the same population, with the RDS data being representative of the target population | Compare sociodemographic and other characteristics of RDS surveys participants reporting program attendance with records of program attenders for the same time reference using logistic regression. | No evidence of difference in characteristics of RDS surveys participants who report program attendance within the reference period and the characteristics of program attenders in the program dataset during the reference period. | |
All members of the population being counted should have a chance of being included in both sources | Assess if all RDS surveys participants are familiar with the existence of the program by using chi-square tests to compare characteristics of individuals who had ever heard of the program with those who had not across sites. | No evidence of difference between individuals who had ever heard of the program with those who had not. | ||
Data sources should have the same and clear time references, age ranges, geographic areas and individuals should not be counted more than once in each data source. | Assess if time references, age ranges and geographic areas of RDS and program data are similar or not; deduplicate program data if participants visited the program several times during the reference period. | Report if time references, age ranges and geographic areas are similar or not. |
||
The 2 data sources should be independent of each other, that is inclusion of individuals in 1 source should not be related to the inclusion of individuals in the other source. | Do not identify seeds and participants in general through the program; given that seed participants might also be more likely to be program attenders, even if they are not selected on this basis, assess convergence of |
Report how RDS participants were identified and recruited; overall estimate of |
aRDS-II: RDS Volz-Heckathorn estimator.
bN/A: denotes the assumptions that could not be investigated with the data available in this study.
cRDS: respondent-driven sampling.
dAssumption always violated when other RDS estimators (not the RDS successive sampling estimator) are used.
eFSWs: female sex workers.
In the RDS framework, seeds are selected purposively with the assumption that if recruitment is done with a sufficiently large number of waves, then the final sample would be independent of the seed characteristics [
We assessed whether the RDS-II weighted cumulative estimates of
The indication of nonrandom recruitment was investigated by measuring recruitment homophily on
The SMM requires that all members of the target population have a nonzero probability of being included in both the RDS survey and the program data [
We also assessed the SMM assumption that the 2 data sources to be combined should be drawn from the same population, with the RDS data being representative of this population [
Unweighted descriptive analyses of program data and RDS-II weighted descriptive analyses of RDS data as well as comparison of the 2 data sources were performed using Stata version 14.2 (StataCorp LLC), and all the other RDS diagnostics were performed using RDS Analyst version 0.5.1, which is based on the RDS package for R [
We recruited a total of 1739 FSWs from 8 seeds at site 1 and 6 seeds from each of the other 8 sites. Of these seeds at each site, only 1 seed had attended the program at site 1, 3 at sites 7 and 9, 5 at sites 2, 3, 5, 6, and 8, and all 6 at site 4.
The PSEs and 95% CIs calculated using the SMM are shown in
Population size estimates of female sex workers and 95% CI.
Site | RDSa sample size | Number of FSWsb who attended the program within the last 6 months (M) | SE for Mc | Percentd reporting visit ( |
SE for P | Population size estimate | SE for the population size estimatee | 95% CI | Percent of FSWs among all women aged 15 to 49 years |
1 | 220 | 57 | 7.4 | 20.3 (11.6-29.1) | 4.5 | 281 | 70.1 | 133-407 | 0.8 |
2 | 196 | 100 | 10.0 | 25.0 (15.3-34.7) | 4.9 | 400 | 87.2 | 225-566 | 4.8 |
3 | 153 | 111 | 10.5 | 46.1 (35.1-57.1) | 5.7 | 241 | 37.2 | 166-311 | 2.8 |
4 | 202 | 372 | 19.2 | 68.7 (60.8-76.5) | 4 | 541 | 42.0 | 455-619 | 3.5 |
5 | 197 | 84 | 9.2 | 20.6 (5.4-35.8) | 7.8 | 408 | 160.4 | 93-722 | 3.9 |
6 | 200 | 28 | 5.3 | 14.3 (5.6-22.4) | 4.2 | 194 | 67.0 | 62-325 | 2.6 |
7 | 165 | 34 | 5.8 | 11.0 (7.2-14.8) | 1.9 | 310 | 75.4 | 162-458 | 1.2 |
8 | 198 | 46 | 6.8 | 16.7 (7.4-26.1) | 4.8 | 275 | 88.7 | 101-449 | 3.0 |
9 | 208 | 165 | 12.8 | 20.5 (12.4-28.7) | 4.2 | 805 | 175.1 | 456-1142 | 2.6 |
aRDS: respondent-driven sampling.
bFSWs: female sex workers.
cCalculated using the normal approximation to Poisson distribution.
dRDS-II adjusted percentages.
eCalculated using the delta method.
The number of women who attended program sites in the previous 6 months before the survey ranged from 28 at a site where the program was relatively new to 372 at a site where the clinic had been established for 2 years. The proportion of FSWs reporting program attendance varied from 11% to 69%. The highest PSE was 805 FSWs (95% CI 456-1142) and the lowest was 194 FSWs (95% CI 62-325). The 95% CIs for the majority of sites were wide (
At sites 1 and 6, the estimate of
Site convergence plots. RDS-II: respondent-driven sampling Volz-Heckathorn estimator.
The bottleneck plots (
Site bottleneck plots. RDS-II: respondent-driven sampling Volz-Heckathorn estimator.
There was little evidence of recruitment homophily, ranging from 0.9 to 1.1 at sites 2 to 9, suggesting a weak tendency for women to recruit others like themselves with respect to reporting program attendance in the past 6 months. However, at site 1, recruitment homophily was moderate (1.4;
Recruitment homophily in P.
Site | Recruitment homophily in |
1 | 1.39 |
2 | 1.14 |
3 | 1.04 |
4 | 0.96 |
5 | 1.05 |
6 | 0.97 |
7 | 1.00 |
8 | 0.92 |
9 | 1.21 |
There was little evidence of an association between the majority of sociodemographic characteristics and knowledge of program existence. Evidence of association was seen for education, where a higher proportion of women who reported secondary school or higher had heard about the program compared with those who reported primary school or none (44% vs 36%;
Association between sociodemographic characteristics and knowledge of program existence among respondent-driven sampling survey participants by site.
Characteristics | Total individuals (N=1739), n | Individuals who have ever heard about the program (N=803), n (%) | Comparison |
Interaction |
|||||
|
.40 | .40 | |||||||
|
18-24 | 418 | 174 (36.8) |
|
|
||||
|
25-29 | 424 | 202 (40.1) |
|
|
||||
|
30-39 | 597 | 284 (43.1) |
|
|
||||
|
40+ | 299 | 143 (44.6) |
|
|
||||
|
.06 | .10 | |||||||
|
Never married | 356 | 170 (42.1) |
|
|
||||
|
Married or widowed | 335 | 139 (33.3) |
|
|
||||
|
Divorced or separated | 1047 | 494 (43.31) |
|
|
||||
|
.02 | .47 | |||||||
|
Primary or none | 531 | 209 (35.7) |
|
|
||||
|
Secondary or higher | 1192 | 590 (44.13) |
|
|
||||
|
.87 | .23 | |||||||
|
<18 | 343 | 157 (41.5) |
|
|
||||
|
18-24 | 630 | 284 (39.3) |
|
|
||||
|
25-29 | 398 | 195 (42.9) |
|
|
||||
|
>30 | 367 | 167 (41.2) |
|
|
||||
|
.32 | .52 | |||||||
|
0-1 | 186 | 86 (36.8) |
|
|
||||
|
2-5 | 587 | 245 (39.3) |
|
|
||||
|
>5 | 956 | 468 (43.7) |
|
|
||||
|
.13 | .02 | |||||||
|
0 | 79 | 43 (49) |
|
|
||||
|
1 | 372 | 179 (40.6) |
|
|
||||
|
2-4 | 1031 | 457 (38.83) |
|
|
||||
|
>5 | 256 | 124 (50.4) |
|
|
||||
|
.24 | .32 | |||||||
|
0 | 132 | 59 (36.6) |
|
|
||||
|
1-4 | 705 | 312 (38.2) |
|
|
||||
|
5-9 | 415 | 205 (45.2) |
|
|
||||
|
>10 | 486 | 227 (44.7) |
|
|
||||
|
.24 | .01 | |||||||
|
0 | 360 | 167 (37.5) |
|
|
||||
|
1-2 | 912 | 425 (43.8) |
|
|
||||
|
>3 | 466 | 211 (38.7) |
|
|
||||
|
|
|
|||||||
|
No | 110 | 36 (27.0) |
|
|
||||
|
Yes | 1628 | 767 (42.02) |
|
|
||||
|
.50 | .89 | |||||||
|
1 | 292 | 124 (38.3) |
|
|
||||
|
2-4 | 910 | 431 (42.2) |
|
|
||||
|
>5 | 417 | 209 (44.9) |
|
|
||||
|
.36 | .93 | |||||||
|
Negative | 898 | 413 (40.7) |
|
|
||||
|
Positive | 720 | 349 (44.0) |
|
|
||||
|
.91 | .32 | |||||||
|
Consistent | 1180 | 540 (40.79) |
|
|
||||
|
Nonconsistent | 369 | 171 (40.3) |
|
|
aChi-square
b
cFSWs: female sex workers.
dAmong those ever tested for HIV.
There was little evidence of differences in the distribution of most sociodemographic characteristics between women who attended the program and those who reported program use in RDS data (
Comparison of sociodemographic characteristics of individuals who attended the program and individuals who reported program use in respondent-despondent sampling surveys.
Characteristic | Individuals who reported program use in RDSa data (N=535), n (%a) | Individuals who actually attended the program (N=997), n (%) | Comparison |
Interaction |
|
|
.88 | .67 | |||
|
18-24 | 108 (22.4) | 187 (19.2) |
|
|
|
25-29 | 137 (22.7) | 246 (25.2) |
|
|
|
30-39 | 192 (35.1) | 370 (38.0) |
|
|
|
>40 | 98 (19.8) | 171 (17.6) |
|
|
|
.61 | .52 | |||
|
Never married | 110 (19.4) | 194 (19.8) |
|
|
|
Married or widowed | 93 (15.3) | 192 (19.6) |
|
|
|
Divorced or separated | 332 (65.3) | 594 (60.6) |
|
|
|
.47 | .16 | |||
|
Primary or none | 146 (31.7) | 243 (28.0) |
|
|
|
Secondary or higher | 386 (68.3) | 625 (72.0) |
|
|
|
.01 | .22 | |||
|
0-1 | 64 (16.1) | 225 (25.3) |
|
|
|
>2 | 467 (83.9) | 666 (74.7) |
|
|
|
.42 | .17 | |||
|
0 | 108 (23.0) | 238 (24.0) |
|
|
|
1-2 | 288 (56.6) | 593 (59.8) |
|
|
|
>3 | 139 (20.4) | 161 (16.2) |
|
|
|
.18 | .75 | |||
|
No | 26 (4.9) | 64 (6.6) |
|
|
|
Yes | 509 (95.1) | 911 (93.4) |
|
|
|
.42 | .48 | |||
|
Negative | 262 (53.4) | 442 (49.7) |
|
|
|
Positive | 242 (46.6) | 447 (50.3) |
|
|
aRDS-II (respondent-driven sampling) weighted percentages.
bWald
c
We combined data on the proportion of FSWs recruited to RDS surveys in 9 Zimbabwean sites and who reported attending the program (
We employed existing RDS diagnostics [
We found little evidence that women with particular characteristics were likely excluded from the program, suggesting that the SMM assumption that all members of the target population should have a nonzero probability of being included in both the RDS survey and the program was met. Characteristics of program attendees were similar to RDS participants, suggesting that the data sources were likely from the same population with the RDS surveys representative of the population, therefore partly satisfying the requirements of the SMM.
The major strength of the SMM is that it can be implemented using data collected for other purposes [
This study has several strengths. Our simple and straightforward diagnostics were able to identify potential biases and suggest the potential direction of bias in the PSEs. The RDS survey data were carefully collected with an in-house coupon manager software to track coupons, verify them, and check that they were redeemed only once [
Study limitations include the inability to investigate all assumptions made by RDS and SMM. The SAPPH-IRe trial baseline was not set up to be used to estimate PSEs, and as such could not investigate all assumptions made by RDS and SMM. We were not able to assess the RDS assumption of accurate reporting of personal network size by participants. We also could not assess the SMM assumption that the 2 data sources should be independent of each other. We do not have data about every sex worker that a woman knows and all of their characteristics to assess whether the ones she recruits for the survey are a random sample or not (though this would be challenging to collect in practice). The assessment of convergence and bottleneck plots is rather qualitative and relies upon visually assessing graphics, which might result in making subjective conclusions.
Although we checked the design effect for the primary outcome of the trial for which these data were collected, which confirmed that the target sample sizes of 200 were adequate, we did not check the design effect for
Although there is guidance on RDS sample size calculations [
We used a single multiplier for illustrative purposes, but in line with other groups, we recommend the use of more than one as multipliers are prone to unmeasurable bias [
When incorporating the SMM in RDS surveys for population size estimation, it is important to understand the context in each site, which can be achieved through detailed mapping [
Double counting of participants in program data needs to be minimized, as this could potentially result in overestimation of the PSEs. The program to be used in population size estimation should be accessible to all members of the target population, and members need to be given unique identifiers coupled with collection of additional information such that if they forget their program unique identifiers, they can easily be reminded. This will reduce the problem of duplication in the counting of individuals who attend the program on several occasions and partly contribute to the accurate calculation of PSEs. When estimating key population sizes, the SMM will ideally be triangulated with other population size estimation methods (capture-recapture, census, network scale-up, and SS-PSE). The size estimates obtained from each of these methods can be quite variable [
The SMM can be used to incorporate RDS proportion estimates [
female sex worker
population size estimate
respondent-driven sampling
Sisters Antiretroviral therapy Program for Prevention of HIV—an Integrated Response
service multiplier method
This work was supported by the Measurement and Surveillance of HIV Epidemics Consortium, which is funded by the Bill & Melinda Gates Foundation. Data collection was funded by the United Nations Population Fund (through Zimbabwe’s Integrated Support Fund funded by the UK Department for International Development, Irish Aid, and Swedish International Development Cooperation Agency). Analyses were made possible by the European & Developing Countries Clinical Trials Partnership through project MF.2013.40205.014.
None declared.