All of Us Research Program to host ELSI workshop

June 10, 2019

All of Us has released a white paper building on the learnings and insights from the June 2019 ELSI workshop.

The All of Us Research Program’s Policy Office will be hosting its Ethical, Legal, and Social Implications (ELSI) Priorities Workshop on June 24-25, 2019. All of Us is committed to advancing efforts to identify and assess the ethical, legal, and social issues in research using the All of Us data resources to benefit both the program and the broader research community. The workshop is a two-day endeavor to help All of Us refine its ELSI goals.

The agenda will include a series of speakers, including a keynote by Stephanie Devaney, PhD, the Deputy Director. Additionally, the workshop will feature talks from key All of Us staff and consortia members to provide a deeper overview of the program’s many parts.  For the rest of the workshop, attendees will break out into working groups dedicated to three topic areas: genomics, social determinants of health, and regulatory and legal policy. These working group sessions will address crucial ELSI research questions pertinent to All of Us and the information and tools needed to answer them.

Workshop attendees will include ELSI experts, consortia members, federal staff from the National Institutes of Health and Health and Human Services, citizen and community scientists, participant ambassadors, and early career ELSI researchers. Attendees will explore the All of Us data resources in detail, including the data architecture and data types, highlighting any additional data collection, data structural changes, and research tools needed to advance ELSI research. The workshop’s open sessions will be streamed via NIH Videocast and archived for on-demand viewing. We will interact with our viewers through Twitter using the hashtag #JoinAllofUs. We urge you join the conversation, leave us your questions, and provide us your feedback.

View ELSI Workshop Agenda.

View ELSI Workshop Summary PDF | 341.21 KB.

For more information, please contact

Resources for Workshop Attendees

Please explore these sites to learn more about the All of Us Research Hub, data browser, data sources, and overview of participant characteristics and types of data collected from participants.

General (All participants)

Genomics Workstream

Social Determinants Workstream

Policy/Legal Workstream

Genomics Workstream

Social Determinants Workstream

Policy/Legal Workstream


Genomics Workstream

Social Determinants Workstream

Policy/Legal Workstream

Responses to Questions from ELSI Workshop Attendees

ELSI workshop attendees submitted several questions seeking information on the program’s current approach or plans for addressing specific issues across several ELSI-related themes. These questions were compiled from written attendee feedback provided during the breakout sessions (days 1 and 2 of the workshop) as well as from emails submitted following the workshop.

Will All of Us capture the change in medicine use for a participant who has received pharmacogenetic results back from the program, discussed/validated the results with their physician, and made changes to their prescriptions as a return of those results?

The program plans to perform reassessments through surveys every two to three years. The reassessments will include updating personal medical information, including any changes in current medications. However, we currently do not have a question specifically asking participants whether they changed their prescriptions based on the All of Us return of pharmacogenomics results. We expect that some of this information will be captured in Electronic Health Records (EHR) for those participants who have shared their EHRs and opted to receive genomic results. We anticipate that EHR data available on the Research Hub will be updated on a quarterly basis. However, we recognize that this may not allow for confirmation that changes in prescriptions resulted directly from health provider actions taken in response to pharmacogenetic results returned by All of Us. The program may also explore collecting these data in the future as part of pilot programs for returning pharmacogenetic results to participants.

Social scientists need multi-level data to work on social determinants of health, such as community factors, health system–level factors, and broader data that can be linked to geocodes. What is the program’s plan for getting those data?

Data elements available through the registered tier currently include EHRs, survey responses, and physical measurements. All of Us is also exploring ways to enhance the data in the future by linking to external data sources such as the National Death Index, Census tract data, and other publicly available sources and by implementation of geocoding. All of Us will continue to consider other sources of community and health system data, based on researcher, community, and participant feedback.

At this time, All of Us is also working to create policies about the types of data and tools that researchers can safely upload into their workspaces. This may include software packages for statistical analyses, phenotype algorithms, and information in public knowledge bases, among other categories. Any external data or software uploaded into the workspaces must be compliant with the All of Us Data Use Agreement (DUA), which prohibits reidentification of All of Us Research Program participants, stigmatizing research, or any other malicious use.

Will phone use and health literacy information be collected from participants?

All of Us surveys include questions on health literacy. However, the program does not currently ask about landline or mobile phone use. More data on these topics may be collected from participants in other ways in the future for research. All of Us will continue to consider other types of data for collection based on researcher, community, and participant feedback. There will be a feedback form through the researcher workbench that allows for structured feedback on scientific suggestions and data types that we should capture in the future. This form will be sent directly to NIH staff.

Will ELSI researchers be allowed to recontact participants to do qualitative and other types of data collection?

All of Us is planning to allow participant recontact for approved ancillary studies. The program is in the process of developing the requisite policies and technical infrastructure to support this aspect of the program. However, at this time, external researchers cannot recontact All of Us participants. We will notify prospective and current users about these and other All of Us developments through the Research Hub.

Can family members (twins, siblings) choose to have their data linked together (rather than looking at genetic similarity)?

Currently, the ability to link familial relations through the participant portal is not available, due to the potential risk of reidentification. Familial data linkage is a functionality that the program will consider for the future, but if it is added, the appropriate data protection and participant notification and controls will be proactively implemented.

How does All of Us address lower data quality concerns—for example, EHR data for some outcomes or self-reported stigmatized behaviors (e.g., drug /alcohol abuse)? 

The curation team in the All of Us Data and Research Center works actively on data quality assessments and metrics and performs quality control on data elements. The curation team also works closely with the All of Us Science Committee on data quality requirements. However, there are limitations to the reliability of self-reported and EHR data, especially for specific behaviors or conditions (e.g., drug and alcohol use) that are considered stigmatizing. Data users should carefully consider these data quality limitations in their study designs and analyses.

How valuable are the data from wearable devices? Should we just allocate resources for return of value? 

Data from wearable devices, such as activity trackers, are one of many data types collected by All of Us. Such data can provide a unique set of information about participant lifestyles. A recent study notes the potential for research using data from wearables because of the amount of data available at a relatively low cost and the ability to collect data from individuals in their natural environment (rather than from individuals being monitored in a research study). The paper cites studies that have demonstrated this potential and outlines best practices for using such data. Another recent study, however, shows that further research is needed to assess reliability of wearables data (e.g., heart rate data) from diverse populations and to develop evidence on health utility of wearables for different population subgroups. All of Us plans to conduct pilots to learn about these issues before implementing collection of wearables data across the entire cohort. All of Us is already allocating resources for return of value and to accomplish these goals.

How does All of Us assess sufficiency of sample sizes for within-group analyses?

At present, the program does not plan to preemptively monitor user workspaces at that level of specificity. However, proper implementation of methods and statistical tools may be assessed as parts of publicly prompted and program-initiated workspace audits by the Resource Access Board (RAB). The RAB, composed of scientists, participant ambassadors, and ELSI subject matter experts who are part of All of Us, is the entity in the program’s governance structure that oversees the use of All of Us data resources and that monitors and reviews workspaces for potential violations of the DUA. The RAB is authorized to use whatever resources it deems necessary to make its assessments, including consultation with subject matter experts either within or outside the program.

What happens when self-reported race/ethnicity does not match EHR data? How will All of Us resolve it?

The EHR data that the program collects will be made available in the participant’s portal. To protect the integrity of the data, when the self-reported race/ethnicity is different from what is reported in the EHR, participants will not be asked to correct the EHR data already sent to All of Us. Participants will need to contact their provider(s) to rectify the inconsistencies; subsequent releases of the data may reflect the changes. All of Us will not notify participants of discrepancies in their data. However, if a participant notices a discrepancy in their EHR data and asks to change it, All of Us will advise them on how they can get the EHR changed through their health provider. All of Us will not make those changes.

How will researchers get updated data about their cohorts? 

The All of Us Data and Research Center will periodically push newer versions of the curated data into the Researcher Workbench. All of Us is still deciding the cadence of the updates. When using the data in the Researcher Workbench, researchers can choose to use the newer version of the database to access updated data on their cohorts (as well as new participants who may have enrolled and qualify to be part of their cohorts).

How are pharma and biotech companies considering using All of Us data if at all?

Individuals in the pharmaceutical and biotechnology industries will be able to access and use All of Us data in accordance with the All of Us Data Use Agreement (DUA). As authorized data users, they will have access to individual-level data and samples to make discoveries. All data users must provide their affiliation (if they have one) and descriptions of their research for public posting, which will allow anyone to see how pharma and biotech companies use the data. As stipulated in the DUA, All of Us data cannot be used for marketing purposes. The findings from their research projects will be added to the All of Us database.

How will citizen scientists be held accountable? Do journalists count as citizen scientists?

All individuals who wish to access the non-public data tiers must verify their identity, undergo ethics training, and sign and comply with the DUA stipulating the terms of access and use. Penalties for violations of the DUA may include revoking access to data, among other yet-to-be-determined penalties that the program is currently considering.

How is the Resource Access Board (RAB) adjudicating stigmatizing research?

The Resource Access Board (RAB) will conduct reviews of project workspaces instigated by the workspace owner; the RAB itself periodically; or anyone in the public who flags the workspaces for review. Reviews of potentially stigmatizing research will include consideration of a project’s intent and the effect(s) that a study or its findings may have on individuals, groups, or communities, including the burden of stigma and potential outcomes. The RAB may inspect whatever workspace components or related information is deemed necessary to make its determination, including contents of the user’s workspace, linked Jupyter notebooks, the publication history of the workspace owner(s)/contributor(s), and/or professional or public track record. The RAB also has latitude to request input from subject matter experts from within or outside the consortium, such as participant ambassadors, community partners, or individuals with meaningful ties to and/or the authority to speak for relevant communities or groups. Furthermore, to ensure that reviews are undertaken with an appropriate level of cultural awareness, the RAB can engage ad hoc members to round out its existing scope of expertise.

How do you determine if a project description does not match what researchers are actually doing?

Data users are required to submit a description of their proposed research upon establishment of a workspace and if/when their project changes. Additionally, users must review these workspace descriptions for accuracy on an annual basis. The Resource Access Board (RAB) will conduct manual audits of workspaces based on a variety of factors, including, but not limited to, keywords, data types queried, or populations studied, as well as scrutiny of the code within the workspace. In addition to random audits, the RAB will also conduct reviews of workspaces in response to requests from members of the public and researchers themselves, which could potentially arise from concerns about data use. Furthermore, the project descriptions will be available for review by the public, which includes fellow researchers as well as participants. Members of the public can flag research projects that will trigger a RAB review.

Will genomic data be made available to participants for personal sharing? Will platforms like blockchain-based applications be integrated?

Participants who choose to receive their information will receive genomic reports through the participant portal and will be able to download and share these reports. All of Us participants will have the option to receive raw genomic data in the future. Currently, there are no plans for integration of blockchain-based applications.

Will polygenic risk information be returned to participants and, if so, to whom?

All of Us would like to return a variety of types of genomic information to participants over time, including polygenic risk scores. However, planning for how and under what circumstances polygenic risk information would be returned has not yet begun.

How does All of Us ensure participants are not identifiable? How is confidentiality protected?

All of Us uses the most up-to-date industry standards and practices to prevent security breaches. Absolute security of data can never be guaranteed; however, the program has brought in teams of experts to conduct rigorous ongoing security testing and establish safeguards. In addition, the program has implemented strict internal policies and procedures to prevent misuse of data.

Here are a few of the steps All of Us takes to protect participant privacy:

  • Information we collect will be stored on protected servers. We will limit and keep track of who sees the information.
  • We will remove participants’ names and other direct identifiers (such as dates of birth) from their accounts and replace those data with codes. There will be a master list linking the codes to names, but it will be kept separate and secure.
  • In order to work with participants’ health information, researchers must promise not to try to find out who they are.
  • We will inform participants if there is a data breach.

In addition, every part of the All of Us Research Program follows the Precision Medicine Initiative (PMI) Privacy and Trust Principles and the PMI Data Security Policy Principles and Framework. The Privacy and Trust Principles are basic rules on governance, transparency, respect for participant preferences, participant access to information, and data sharing, access, use, and quality. The Data Security Policy Principles and Framework are a way of keeping data safe.

Data are provided through a website application portal and stored in a secure back-end database. The database is accessible only to a handful of vetted administrators and developers. Front-end applications provide an interface through which developers, portal users, and researchers can use the data. A series of data flows provided by application programming interfaces (APIs) and database interconnections facilitates all data movement between application components. Furthermore, the program conducts privacy risk assessments, based on identifiability and sensitivity, for triaging data elements into different access tiers. These assessments are conducted initially and periodically to determine whether data elements are assigned to appropriate tiers.

Does investigation of intersectionality (multiple aspects or features of a person’s social or demographic identity) flag research as too “identifiable"?

The program is considering implementing flags whenever a user queries particularly sensitive data types, including categories that delineate populations that are underrepresented in biomedical research. However, those flags are not in place now. There is no preclusion of combining queries to study intersectionality, only to using analytical techniques to purposefully reidentify participants. Categories with small sample sizes are obscured to lower the risk of reidentification.

Will persons with language barriers (other than Spanish) be included in All of Us?

At this time, the program can enroll only individuals who understand English or Spanish. However, the program is working to develop a language access plan to incorporate other languages at the national and local levels. In the meantime, the All of Us Support Center can answer questions related to the program in a wide variety of languages.

How do disability communities access the All of Us Research Program?

To ensure that participation is open to persons with physical disabilities, the protocol allows the trained site staff to offer a range of accommodations to individuals needing assistance. Trained site staff may provide assistance and accommodations to prospective participants so long as the assistance is an augmentation of the consent process rather than a requisite for the acquisition of consent. For instance, staff may offer a prospective participant adaptive technology, such as screen readers to self-navigate the consent workflow, or may read them the full consent text. Family members or friends may also provide such assistance to participants during the consent process. The program also works to ensure that all electronic materials that are provided for obtaining consent and available on the All of Us website are accessible to people with disabilities, in compliance with federal regulations (Section 508). The program is working to expand consent capacities and hopes to offer accommodations to serve an even wider range of needs of prospective participants in the future.

Will people with physical and cognitive disabilities be included as participants?

Participation is open to eligible individuals intellectually capable of providing informed consent. Individuals who require or prefer assistance with the consent process due to physical, educational, or other limitations may be offered assistance by an All of Us staff member trained in facilitating consent. The trained staff member will engage the prospective participant in the informed consent discussion and will co-sign the informed consent document. Family members or friends may also provide such assistance to participants during the consent process. The program is working to expand consent capacities to serve an even wider range of needs of prospective participants. A separate protocol amendment is being developed to allow enrollment of individuals who require a legally authorized representative to sign on their behalf for reasons of decisional impairment.

What are the processes for including vulnerable populations (incarcerated persons, children, people with developmental disabilities)?

The inclusion of vulnerable populations will be phased in over time. The staggered approach aims to ensure that adequate consenting procedures and other program support measures are in place so that participants’ rights, safety, and welfare are not compromised. This approach will also ensure that we have the appropriate systems in place to support participants’ long-term engagement.

How will broad access to products resulting from All of Us research be assured?

All of Us supports commercial products and services emerging from secondary research using All of Us data and biospecimens and recommends that those products and services be accessible broadly and equitably. The program does not claim intellectual property rights to data and biospecimens made available for research. Although All of Us does not have mechanisms to enforce policies that would ensure broad access to such products and services, the program will emphasize the need to share information and findings broadly and equitably in order not to exacerbate health disparities.