The Real-World Insights of People Living with HIV Shared Through Electronic Devices (RISE) Study


Name Affiliation
Tammeka Evans
Independent Researcher, London, UK
Katelyn Cutts
Evidera, Bethesda, MD, USA
Konstantinos Lykopoulos
Independent Researcher, London, UK
Pedro Eitz Ferrer
Independent Researcher, Raleigh, NC
Paul Swinburn
Independent Researcher, London, UK
contributed: 2021-01-25
final review: 2021-04-07
published: 2021-05-03

As antiretroviral treatment (ART) has made viral suppression and survival less of a concern for individuals with the human immunodeficiency virus (HIV), the focus of research on treatment and care for people living with HIV (PLHIV) has shifted to normalizing living with HIV infection through improving their overall well-being and health-related quality of life (HRQoL). The RISE study aimed to implement an innovative method for gathering real-world data (RWD) on these topics through mobile application (app) surveys and subsequently, compare the outcomes to a normative population sample

For this study, targeted recruiting methods through social media and other online sources were utilized. Participants were at least 21 years old with a self-reported diagnosis of HIV, living in the United States (US). Participants took mobile app surveys, detailing medical information on their personal mobile devices. From this data collection, it was concluded that White, non-Hispanic southern males who had private insurance, as well as other ethnicities and those who reported a greater general satisfaction, were significantly more likely to be adherent to their treatment and experienced less internalized stigma over HIV. Participants who were lower income or on Medicare or Medicaid were associated with a decrease HRQoL, as well as a higher internalized stigma over HIV.

Keywords: people living with human immunodeficiency virus; health-related quality of life; real-world data; internalized stigma


The advent of effective antiretroviral treatment (ART) for human immunodeficiency virus (HIV) infections, which improved greatly around 1996 with the development and widespread use of highly active ART, has reduced the survival concerns for people living with HIV (PLHIV) and transformed HIV into a chronic condition requiring long-term care and prolonged adherence to treatment [1]. Consequently, treatment and care for PLHIV is no longer focused solely on viral suppression and extending their life span, but gradually is also shifting to normalizing living with HIV infection through improving well-being and health-related quality of life (HRQoL) [2].

In a recent opinion paper responding to the World Health Organization’s (WHO) 2016–2021 target for HIV of “90–90–90” — which aims to achieve a 90% diagnosed, 90% on treatment, and 90% virally suppressed patient population — Lazarus and colleagues [3] postulated adding a “fourth 90” to ensure 90% of virally suppressed PLHIV had a good HRQoL. The authors further highlighted that to meet the needs of virally suppressed PLHIV, healthcare systems should become more integrated and patient-centric, attending to the patients’ comorbidities and self-perceived quality of life (QoL) as primary domains of HRQoL. In line with this, current public health recommendations for the development of new therapeutic strategies for chronic conditions indicate that the impact of drugs on HRQoL and patient-reported outcomes (PROs) in general should be carefully evaluated [4].

While HRQoL for PLHIV has been evaluated in clinical trials that study the effects of ART, there are few real-world studies that assess HRQoL. In recent years, there has been an increased interest in the applications of real-world data (RWD). Patient registries, healthcare databases, and pharmacy/insurance databases have been used as valuable sources of data to inform research on patient populations. Technological developments have now made it possible to directly access large populations in a simple, cost-effective fashion, particularly through proliferation of mobile devices.

To address the gaps highlighted in the literature, the authors decided to explore the potential of a novel data-collection method using the electronic administration of a survey via a mobile phone application (app). This study aimed to obtain a current understanding of the impact HIV has on patients’ lives in the United States (US) and evaluate the HRQoL, internalized stigma, and general satisfaction in PLHIV who have achieved viral suppression.



The “Real-word Insights of PLHIV Shared through Electronic devices” (RISE) study was a cross-sectional electronic mobile app survey recruiting from January to July 2018. The study was designed to run until 3,000 PLHIV were recruited or the study had been running for 6 months. Participants were at least 21 years old with a self-reported diagnosis of HIV, living in the United States (US), who possessed a smartphone with internet access, and who were able to read English or Spanish.

Study Procedures

Participants were recruited through several channels, including social media, a google search campaign, and advertisements. Specific targeted recruitment included: participants living in southern states, Hispanic/Latinx individuals, Gay/bisexual men, and African American individuals. Participants who were interested in the RISE study and found eligible via the online screener would then complete an electronic consent form. Participants were then instructed to download the RISE mobile app from the Apple or Android app store. The various questionnaires and targeted questions could be completed within seven days of starting the survey. Since there was no single PRO tool that captured data on all important domains of HRQoL, emotional distress, and health concerns, a combination of questionnaires were selected. The instruments were either validated HIV specific questionnaires or general PROs deemed suitable for use in HIV. The questionnaires were administered in the following order: 1. the Functional Assessment of HIV Infection (FAHI); 2. the Patient Satisfaction Questionnaire Short-form survey (PSQ-18); 3. the Symptom Distress Module (SDM); 4. the Patient Health Questionnaire-2 (PHQ-2); 5. the Internalized Stigma of Acquired Immunodeficiency Syndrome (AIDS) Tool (ISAT); 6. the adherence visual analogue scale (VAS); and 7. the PozQoL. During the seven-day enrollment period, participants received daily reminders to complete the questions. Prior to initiation, the study protocol was approved by an US institutional review board (IRB), including ethical review (Quorum Study Number: 32869; approval date: January 10, 2018).



The FAHI is a 47-item questionnaire evaluating the HRQoL of PLHIV. The FAHI was developed from the Functional Assessment of Cancer Therapy – General (FACT-G). The items in the FACT-G (the G items) are included in the FAHI, in addition to the more HIV-specific items. T-scores for the G items in the FAHI were calculated and compared with the reference data for the US general population for the total and domain scores [5]. The T-scores are conversions of the raw scores to a linear scale; they have a mean of 50 and a standard deviation of 10.


The PSQ-18 is a short-form version of the 50-item PSQ III and assesses a patient’s general satisfaction with medical care [6]. The questionnaire yields separate scores for each of the seven difference subscales: general satisfaction, technical quality, interpersonal manner, communication, financial aspects, time spent with a doctor, and accessibility and convenience. Subscale scores are calculated by taking the average of all items that were answered.


The PHQ-2 evaluates depression and anhedonia (inability to experience pleasure), and it includes the first two items of the PHQ-9. A total summary score is calculated from the two items with scores ranging from zero to six [7].


The ISAT consists of 10-items related to negative self-perceptions related to HIV [8]. The items are rated on a five-point Likert scale from one (strongly disagree) to five (strongly agree). A total score is computed by summing the scores of all items in the scale, with higher scores representing higher levels of internalized stigma.

Medication adherence VAS

The medication adherence VAS assesses the percent of prescribed anti-HIV medication a patient has taken in the last 30 days, with the scale ranging from 0% (no medication taken) to 100% (every single dose taken).


Descriptive statistics were summarized for all sociodemographic and clinical characteristics for the overall sample, as well as by viral suppression status. Results reported the n, mean, standard deviation, median, 25th and 75th percentiles, and range for continuous scores, as well as frequency for the categorical variables. To evaluate the impact of HIV in virally suppressed PLHIV, the FAHI domain and total scores were compared to normative scores from the FACT-G based on a US general population sample [5]. Raw score means and standard deviations of the FAHI scores were compared to the raw score means and standard deviations in the US normative sample. FACT-G scores were prorated to place them on the same metric (i.e., same item range) as the FAHI scores. A two-point difference on the domain scores and a five-point difference on the total score was considered clinically meaningful [5]. In addition, multiple regression models were conducted to evaluate the relationship between predictors and the HRQOL. Full models were conducted, as well as reduced models, where specific variables were removed from the model due to non-significance and/or collinearity. Results from the reduced models were reported.


Participant Characteristics

Participants were recruited through several social media platforms, including Facebook, with targeted recruitment for select subgroups of interest and non-responders. Less than a quarter of the participants who passed screening (n=11,687) also consented to participating through signing the form (n=2,648). Of the eligible and consented participants, less than half (n=1,226, 46.3%) completed all items of the sociodemographic and biomedical information questionnaire. The remaining PROs were completed in full by 44.7%–45.5% of the eligible and consented sample. Participants were not required to complete all PROs in the survey, resulting in differing completion rates.

Most of the overall sample were male (81.0%), less than 50 years old (58.1%), White, non-Hispanic (53.3%), lived in the South (44.0%), and had private health insurance (41.5%). The majority reported they are taking a single treatment regimen (63.2%) for their HIV and are very satisfied with their treatment (76.3%). Most participants reported visiting their healthcare providers every three (48.9%) or six (41.8%) months. Participants averaged 3.8 comorbidities, with many participants reporting anxiety (84.1%), other sexually transmitted infections (STI [50.4%]), and depression (47.0%). A small subset of participants (n=92, 7.5%) self-reported not being virally suppressed (Table 1).

Table 1. Patient Sociodemographic and Self-reported Clinical Characteristics

1Not mutually exclusive

2Other includes other government programs, no coverage, and don’t know coverage

Abbreviations: HIV = human immunodeficiency virus; MTR = multiple treatment; SD = standard deviation; STI = sexual transmitted infection; STR = single treatment; VA = Veteran’s Affairs; US= United States


FAHI Scores in PLHIV vs. General US and Cancer Populations

Virally suppressed patients who completed the FAHI (n=1,116, mean=105.9, SE=30.2) reported a lower QoL in comparison to the US (n=1,075, mean=121.8) and cancer normative samples (n=2,236, mean=120.9). The domain scores were also higher in the US normative samples relative to the RISE virally suppressed patients, with mean differences ranging from 1.2 to 7.4 (Table 2).

Table 2.    FAHI Scores in Virally Suppressed PLHIV vs General Population and Cancer Population1

1FACT-G scores were prorated to place them on the same metric (i.e. same item range) as the FAHI scores.

2 This analysis was based on virally suppressed PLHIV who completed the FAHI (N=1,116).

3The cognitive domain score is not evaluated in the FACT-G; therefore, it is removed from the FAHI total score.

Abbreviations: FAHI = Functional Assessment of HIV Infection; NR = not reported; RISE = Real-World Insights of People Living with HIV Shared Through Electronic Devices; SD = standard deviation; US = United States

Predicting QoL

To explore the relationship between HRQoL and key predictors further, a model was run with the full sample to determine the effect of viral suppression, age, and comorbidities on HRQoL. Older patients (≥50 years old) reported a higher HRQoL, while patients who did not have private insurance, were lower income (<$49,999), and had more comorbidities reported a lower HRQoL (R2 = .167; F-test =19.93; p-value<0.0001). In a model assessing predictors in only virally suppressed PLHIV, being lower income and on Medicare or Medicaid were associated with a decreased HRQoL, as well as a higher internalized stigma, lower general satisfaction, and increased level of depression (R2 = 0.771; F-test =278.3; p-value<0.0001).

Stigma and Medication Adherence

Additional models were conducted to determine the key predictors of internalized stigma and medication adherence for virally suppressed PLHIV. Patients who were <50 years old and lower income reported higher stigma relative to other patients (R2 = .156; F-test =24.6; p-value<0.0001). White, Non-Hispanic, and other ethnicities, as well as individuals who reported greater general satisfaction, were significantly more likely to be at least 90% adherent relative to other patients (Max-rescaled R2 = .081; Chi-square= 53.19; p-value<0.0001).


The objective of this study was to assess the HRQoL, internalized stigma, and general satisfaction among virally suppressed PLHIV using a mobile app survey. The potential of RWD to revolutionize the patient care landscape is significant. Nevertheless, RWD opportunities are often challenging to realize as researchers must contend with issues like a lack of data standardization and quality assessment. However, several studies have shown data equivalence between mobile apps and other modes of data collection [9, 10]. Electronic data capture can also be beneficial in large-scale studies by providing the ability to capture a wide range of participants with minimal cost implications [10].

As was expected, virally suppressed PLHIV reported significantly better HRQoL, general satisfaction, and medication adherence, as well as a lower internalized stigma and being less likely to indicate depressive tendencies. While achieving viral suppression significantly improves the HRQoL of PLHIV, this remains meaningfully worse when compared with the US and cancer normative populations. A two-point difference on the FAHI domain scores and five-point difference on the total score were considered clinically meaningful [5], with a mean difference of 15.9 for virally suppressed PLHIV and US normative, and 15.0 for virally suppressed PLHIV and cancer normative. Physical well-being, functional, and global well-being scores were comparable for the virally suppressed PLHIV and cancer normative sample (e.g., <2-point difference), with all other domains being meaningfully different across the virally suppressed PLHIV, US normative, and cancer normative samples.

Several limitations were identified, and these pertained largely to the recruitment and data collection methods employed in the study. Participants were recruited via the use of social media and required to complete the survey via a mobile application, which may have contributed to the sociodemographic bias observed in the sample. Participants were generally male,younger, better educated and more affluent than the US normative sample. There was also evidence of reduced ethnic diversity in the study population with under-representation of Black and Hispanic individuals when compared to the US HIV population data [11]. Given that the study population participants were established members of online HIV communities, it is possible the sample represented people who were more actively engaged with their condition and, thus, potentially less impacted by social stigma. It is also conceivable that this engagement may have resulted in improved knowledge regarding their condition and potentially better health outcomes.

In addition, there was a substantial survey non-completion rate (22.7%), such that potential participants completed eligibility questions but did not complete informed consent. Only 46.3% of participants who consented then downloaded the app and completed at least the sociodemographic and medical information section. This reduction in consent and mobile app downloading could also have contributed to the sociodemographic bias found in the sample, with certain participants being more likely to consent and download the app than others. Previous studies have indicated a significant difference in key characteristics of responders and non-responders in surveys [12].

Study participants were required to self-report sociodemographic and biomedical characteristics, including both their diagnostic and viral suppression statuses. While this is common practice for this type of cross-sectional survey study, some consideration should be given to the potential for unreliable or inaccurate reporting.

This study demonstrated the feasibility of implementing a novel data collection method to capture an up-to-date understanding of PLHIV and their HRQoL. While virally suppressed patients show a better HRQoL than those who have failed to achieve viral suppression, the study suggests that it is still significantly lower than the US normative population. Further research should aim to provide additional insights into the factors influencing HRQoL for PLHIV and could extend to understanding the social and employment conditions of individuals, as well as the implications for healthcare resource utilization.

Conflicts of interest

The research was funded by ViiV Healthcare and the Evidera team was a consultant for this work.


1. Engler K, Lessard D, Lebouche B. A Review of HIV-Specific Patient-Reported Outcome Measures. Patient. 2017;10(2):187-202. doi:10.1007/s40271-016-0195-7.

2. Drewes J, Gusy B, Rüden Uv. More Than 20 Years of Research into the Quality of Life of People with HIV and AIDS—A Descriptive Review of Study Characteristics and Methodological Approaches of Published Empirical Studies. Journal of the International Association of Providers of AIDS Care (JIAPAC). 2013;12(1):18-22. doi:10.1177/1545109712456429.

3. Lazarus JV, Safreed-Harmon K, Barton SE, Costagliola D, Dedes N, del Amo Valero J et al. Beyond viral suppression of HIV – the new quality of life frontier. BMC Medicine. 2016;14(1):94. doi:10.1186/s12916-016-0640-4.

4. Duracinsky M, Herrmann S, Berzins B, Armstrong AR, Kohli R, Le Coeur S et al. The development of PROQOL-HIV: an international instrument to assess the health-related quality of life of persons living with HIV/AIDS. J Acquir Immune Defic Syndr. 2012;59(5):498-505. doi:10.1097/QAI.0b013e318245cafe.

5. Brucker PS, Yost K, Cashy J, Webster K, Cella D. General population and cancer patient norms for the Functional Assessment of Cancer Therapy-General (FACT-G). Eval Health Prof. 2005;28(2):192-211. doi:10.1177/0163278705275341.

6. Marshall GN, Hays RD. The Patient Satisfaction Questionnaire Short-Form (PSQ-18). 1994. Accessed November 19, 2018.

7. Kroenke K, Spitzer RL, Williams JB. The Patient Health Questionnaire-2: validity of a two-item depression screener. Med Care. 2003;41(11):1284-92. doi:10.1097/01.MLR.0000093487.78664.3C.

8. Phillips KD, Moneyham L, Tavakoli A. Development of an Instrument to Measure Internalized Stigma in Those with HIV/AIDS. Issues in Mental Health Nursing. 2011;32(6):359-66. doi:10.3109/01612840.2011.575533.

9. Brodey BB, Gonzalez NL, Elkin KA, Sasiela WJ, Brodey IS. Assessing the Equivalence of Paper, Mobile Phone, and Tablet Survey Responses at a Community Mental Health Center Using Equivalent Halves of a 'Gold-Standard' Depression Item Bank. JMIR Ment Health. 2017;4(3):e36. doi:10.2196/mental.6805.

10. Marcano Belisario JS, Jamsek J, Huckvale K, O'Donoghue J, Morrison CP, Car J. Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods. Cochrane Database Syst Rev. 2015(7):MR000042. doi:10.1002/14651858.MR000042.pub2.

11. Centers for Disease Control and Prevention. HIV Surveillabce Report: Diagnoses of HIV Infection in the United States and Dependent Areas, 2016. Centers for Disease Control and Prevention. 2016.

12. Hill A, Roberts J, Ewings P, Gunnell D. Non-response bias in a lifestyle survey. J Public Health Med. 1997;19(2):203-7. doi:10.1093/oxfordjournals.pubmed.a024610.

About Us

Journal of Health Policy & Outcomes Research (JHPOR) is a peer-reviewed, international scientific journal, covering health policy, pharmacoeconomics and outcomes research in Poland and worldwide. The journal is issued under the auspices of the Polish Society of Pharmacoeconomics.

Subscribe to our newsletter:

Latest Articles

Our Contacts

Śliska 3 lok. 55
00-127 Warszawa
NIP 5252390463
REGON 140936540
KRS 0000277843

2017 © Pro Medicina Foundation