Ethical Review Process
Obtaining institutional review board (IRB) approval from all participating groups raises certain challenges [28]. For all research funded by US federal agencies and institutions, participating institutions in the United States and each participating country must have a Federal Wide Assurance (FWA) and approval from an Office for Human Research Protections (OHRP) registered IRB. The FWA provides a commitment by the institution to follow standard ethical guidelines for human subjects research (eg, the Belmont Report, the Declaration of Helsinki, or similar codes). To obtain an FWA, institutions must have a designated IRB registered with OHRP. A database of institutions with FWAs and IRBs that meet US human subjects’ protections guidelines for research is maintained by OHRP [36]. Each participating country outside of the United States may have its own required IRB for in-country research, which may not be approved by the US OHRP [28]. Thus, reviews by multiple IRBs may be required. Coordination of timing of reviews is critical to ensure that all are completed prior to study initiation. In addition, some IRBs may require approval by IRBs in the other participating countries prior to providing local approval, so it is critical for investigators to be aware of such local regulations and communicate documentation of approvals to each IRB within the required time frame [22,24,37,38].
Finally, investigators need to be aware of culturally-specific variations in what are acceptable research practices to produce a final protocol that meets regulatory requirements of each country. For example, in our prior work in the Dominican Republic, paying subjects for completing surveys, a practice that is acceptable by US IRB standards was considered coercive by Dominican Republic IRB standards, though provision of a small thank you gift at the end of the survey was acceptable with the caveat that participants not be informed of the gift until after completing the survey. In addition, in partner Dominican Republic communities, providing written consent for participation was culturally unacceptable as it raised concerns about contractual requirements. Thus, investigators worked with all IRBs to develop an acceptable verbal consent procedure with documentation of consent provided by data collectors [22,24,27,28]. Understanding IRB requirements of all participating sites is critical to successful global partnerships for research [37].
The IRB process at the University of Puerto Rico (UPR/MSC) was another source of lessons learned. The initial request for IRB approval was submitted for the online/offline versions of the survey to be administered by means of iPads and facilitated by a member of the research team. Once the IRB approval was ready (see below), the team proceeded to collect data. Once data collection began, the team faced the challenge of implementing offline collection, as the study sites were in remote areas in Puerto Rico with limited access to an Internet connection. Not being able to use the printed, hard copy as an alternative to the electronic version was a setback that required an amendment to the original protocol and changes to the data collection schedule. The lessons learned here – to gain initial IRB approval for both electronic and paper versions of survey data collection - can help to avoid further delays in the study plan.
Participants
Survey 1
Participants in Survey 1 consisted of a convenience sample of 32 Puerto Rican residents who were engaging with their medical health care system in one of three Federally Qualified Health Centers (FQHCs) in rural areas of Northern Puerto Rico over a 2-week period. Potential participants were approached by project data collectors (see below) and, if agreeable, were provided an information letter about the purpose of the survey. Implementing the survey consisted of a face-to-face interview, using a digital tablet to enter responses.
Survey 2
In the development phase of the study prior to an actual pilot implementation of Survey 2, qualitatively pre-testing all the sections and items of the developing survey was accomplished with a convenience sample of 16 trainees who were participating in an National Institute of Health (NIH) funded eHealth training initiative, and who will later be data collectors using Survey 2 and offline data collection. This sample was mostly composed of highly educated persons (eg, health providers, technology experts) who will, later in the parent study, be responsible for administering the final version of Survey 2 in their own countries (Costa Rica, Honduras, and Dominican Republic). Feedback was obtained on content, wording, and device navigation (ie, using the REDCap mobile survey on a digital tablet).
Data Collectors
Survey 1
For the field work with Survey 1 in Puerto Rico, data collectors were six Puerto Rican-based staff who were participating in the Puerto Rico Testsite for Exploring Contamination Threats (PROTECT), a research project that follows a cohort of 1800 pregnant women in the northern Karst area of Puerto Rico and their exposure to environmental agents [39,40]. Biological samples for PROTECT, an IRB-approved study, include examination of the DNA for the PAX gene and for future genetic testing.
The PROTECT staff that participated in the field work with Survey 1 included one registered nurse and fieldwork coordinator with ample experience in research, one master’s degree nutritionist, a doctoral student with a master's in public health (MPH) degree and experience in field research, and the co-principal director for the PROTECT Community Engagement Core with a doctorate of philosophy (PhD) in policy analysis and research. The staff were all previously familiar with REDCap, although they had not yet used the mobile version.
Training activities included observation and a 3-hour workshop to become familiar with both the content of the survey as well as its online and offline implementation via the mobile REDCap protocol. Each staff was given a tablet and assigned a unique username. All staff members participating in the project completed human subject research protection training from the Collaborative Institutional Training Initiative (CITI) prior to the collection of any data.
Survey 2
Qualitative data collection for the pre-testing phase of Survey 2 was conducted by a senior US research project staff (behavioral scientist), who interactively presented a draft of the draft survey as part of a didactic module during a week-long short course in Costa Rica (see below) on online/offline digital data collection. During the module and in subsequent meetings, senior project staff documented qualitative feedback via field notes and confirmed common themes and item improvement ideas. At the same time, another senior project team member provided live support and feedback from the United States, including real-time problem-solving that led to lessons learned (eg, problems with offline/online synchronization, log-in credentials, and other functionality).
Instruments
Survey 1: From Mechanical Turk to Offline
The present fieldwork began with the development of a Spanish-language version of a survey initially available online at Mechanical Turk (mTurk), Amazon's crowdsourcing website [41-43]. The initial survey was assembled primarily from items validated in previous surveys. Reliability and validity have not yet been established, but it is currently in use in several of our initiatives and will be examined for reliability metrics in subsequent phases of project development. The Internet as a medium and, more recently, the use of “crowdsourcing” as a strategy, has greatly expanded the potential for low-cost timely survey research initiatives. Crowdsourcing is the paid (or often unpaid) “recruitment of an independent global workforce for the objective of working on a specifically defined task or set of tasks” [44]. Crowdsourcing is increasingly recognized as a legitimate strategy to engage with research subjects [45-47].
Spanish Back Translation Process
In order to create an equivalent, as well as culturally appropriate Spanish language version of the initial English version, it was first translated into Spanish, and then back translated into English using the Brislin method [48]. The team then compared the back-translated version with the original to identify problematic translations. These problematic translations were then examined by two native Puerto Rican speakers and edits were provided. Finally, a native Spanish speaker did a final edit of the entire survey before it was pre-tested. The Spanish version was pre-tested during the back translation process by six bilingual investigators from the United States and Puerto Rico for readability, skip patterns, formatting, and content. Committed partners and an organized and systematic approach facilitated the translation process. A barrier to the process was time to complete translation given that people were volunteering their time.
Research Electronic Data Capture (REDCap)
For the present field work, surveys were developed and implemented using the mobile (offline) application of REDCap, a software toolset and workflow methodology for electronic collection and management of research and clinical trial data [21]. The secure, Web-based application (whether online or offline) provides an intuitive interface for users to enter data and real-time validation rules (with automated data type and range checks) at the time of data entry. REDCap offers easy data manipulation with audit trails and functionality for reporting, monitoring and querying patient records, as well as an automated export mechanism to common statistical packages (SPSS, SAS, Stata, R/S-Plus).
The NIH-funded Clinical and Translational Science Institute’s (CTSI’s) informatics core, a unit of the University of Rochester's School of Medicine & Dentistry Academic Information Technology (AIT) Group, serves as a central facilitator for data processing and management. REDCap data collection projects rely on a thorough study-specific data dictionary defined in an iterative self-documenting process by all members of the research team, with planning assistance from the AIT-CTSI informatics core. The iterative development and testing process result in a well-planned data collection strategy for individual studies.
REDCap servers are housed in a local data center at the University of Rochester and all Web-based information transmission is encrypted. REDCap was developed in a manner consistent with Health Insurance Portability and Accountability Act (HIPAA) security requirements and is recommended to University of Rochester researchers by the University of Rochester Medical Center (URMC) research privacy officer and office for Human Subject Protection.
REDCap Mobile for Offline Data Collection
For the purpose of this fieldwork the REDCap application for offline data collection was used, because many surveillance sites will be remote and will have limited Internet access. Mobile offline data collection allows for real-time data collection and storage. Data are later “synced” or uploaded to an online server and added to the primary database.
Although software programs and hardware devices were not prospectively compared and tested with our LMIC project partners, we employed lessons learned from our previous work regarding reach and engagement considerations of eHealth strategies [49,50], and from the formative evaluation phase of one of our randomized controlled trials (RCTs) [51]. In this trial, surveys were developed and pretested for implementation with US English speakers with low health literacy across a large geographic area. This included formative evaluation (key informant interviews and focus groups, exploring offline data capture) and, later, process evaluation steps (tracking participant engagement) to determine how and under what circumstances various mHealth strategies increased project engagement. Platforms that were considered for survey implementation included: (1) use of a vendor to manage all online surveys with sophisticated programming and support, (2) use of a commercial product such as Survey Monkey, and (3) use of REDCap. REDCap was chosen as the mHealth platform because (1) it is HIPAA protected; (2) there is no cost to the project; (3) programming support is available; and (4)respondents who are located anywhere geographically can access it using any common device such as personal computers, laptops, tablets, or mobile phones with app capabilities.
The process of enabling the system began with survey instrument development on the REDCap website platform, and then offline module approval was obtained from the US institution’s REDCap administrator. This approval is in the form of a Quick Response Code (QR Code), which contains a unique identifier for each specific requester and is necessary for installation of the survey into the offline application. This QR Code is provided for each staff member collecting data in the field, and each QR code matches with each username. The ability to scan this QR code with the mobile device enhances the project’s ability to collect secure data in the field, including the facilitation of later data collection when offline. Failure to match in this way, however, results in a substantial data storage complication (a “metadata conflict”), whereby the username and the QR code do not match and data collection from that username cannot occur and/or data may be compromised. This code is scanned to each mobile device where the username is used (eg, mobile phones and tablets).
Where online connections are possible, the mobile application provides direct data entry into the REDCap environment on the institution’s server. In LMICs or other low-resource settings where there are limited wireless (WiFi) or cellular connections to ensure predictable online access, the mobile application provides the flexibility for offline data collection. This expanded ability to use digital devices to collect data also limits the need for paper surveys (except as a backup) and the risk of losing data. The portability is also maximized by the ability to be used on iOS and Android platforms (including tablets and mobile phones). With both online and offline options, there is capability for direct data entry, which reduces the time needed for further data management and analysis. Finally, although matching each portable device to unique users requires advanced planning and start-up time, registering each username individually allows for superior tracking of data collection by each data collector.
Survey 2: Maternal Health Information Communication Technology
A second survey development and implementation project continued to provide lessons learned for overall project methodologies (ie, formative evaluation for Survey 1 as well as Survey 2) regarding the refinement of the development of both the online and offline survey procedures. The Survey 2 goals in this second project (“MundoComm” based in Costa Rica and the Dominican Republic) include conducting qualitative and quantitative assessments of multi-level determinants of maternal health behaviors that can potentially be addressed through technological innovation, and identifying electronic readiness (e-readiness), defined as use and acceptability of current information communication technology resources (such as social media, texting, as well as earlier technologies such as radio) to guide project development.
During a week-long “short course” training in Costa Rica with teams from the Dominican Republic, Honduras, and Costa Rica, experiences with the offline digital tablet (iPad) demonstrations in vivo revealed new challenges, primarily that systematic upgrades to the REDCap platform at our US-based institution caused immediate problems in the current version accessible on the digital tablet. It was determined that additional or back-up strategies to save all data collected since the last upload/sync will need to be developed. Also, explaining the multi-step processes to gain initial online access at the US-based institution, the steps needed to ensure current access, survey administration, data syncing and trouble-shooting, were all challenging for both instructors and trainees.