Books by Emily Zimmerman
Papers by Emily Zimmerman
PubMed, Jun 1, 2016
The research community faces a growing need to deliver useful data and actionable evidence to sup... more The research community faces a growing need to deliver useful data and actionable evidence to support health systems and policymakers on ways to optimize the health of populations. Translating science into policy has not been the traditional strong suit of investigators, who typically view a journal publication as the endpoint of their work. They are less accustomed to seeing their data as an input to the work of communities and policymakers to improve population health. This article offers four suggestions as potential solutions: (1) shaping a research portfolio around user needs, (2) understanding the decision-making environment, (3) engaging stakeholders, and (4) strategic communication.
Oxford University Press eBooks, 2016
JAMA, May 4, 2021
The funders had no role in the design and conduct of the study; collection, management, analysis,... more The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Disclaimer: The analyses described are the responsibility of the authors and do not necessarily reflect the views or policies of the US Department of Health and Human Services. The mention of trade names, commercial products, or organizations does not imply endorsement by the US government.
SAGE Publications, Inc. eBooks, 2021
SAGE Publications, Inc. eBooks, 2021
Qualitative Research, Jun 10, 2021
Online research methods have risen in popularity over recent decades, particularly in the wake of... more Online research methods have risen in popularity over recent decades, particularly in the wake of COVID-19. We conducted five online workshops capturing the experiences of participatory health researchers in relation to power, as part of a collaborative project to develop global knowledge systems on power in participatory health research. These workshops included predominantly academic researchers working in 24 countries across Africa, Asia, Europe, and the Americas. Here, we reflect on the opportunities, limitations, and key considerations of using online workshops for knowledge generation and shared learning. The online workshop approach offers the potential for cross-continental knowledge exchange and for the amplification of global South voices. However, this study highlights the need for deeper exploration of power dynamics exposed by online platform use, particularly the 'digital divide' between academic partners and community co-researchers. Further research is needed to better understand the role of online platforms in generating more inclusive knowledge systems.
American Journal of Preventive Medicine, Jul 1, 2017
Introduction: A demonstration project in Richmond, Virginia involved patients and other stakehold... more Introduction: A demonstration project in Richmond, Virginia involved patients and other stakeholders in the creation of a research agenda on dietary and behavioral management of diabetes and hypertension. Given the impact of these diseases on morbidity and mortality, considerable research has been directed at the challenges patients face in chronic disease management. The continuing need to understand disparities and find evidence-based interventions to improve outcomes has been fruitful, but disparities and unmet needs persist. Methods: The Stakeholder Engagement in Question Development (SEED) method is a stakeholder engagement methodology that combines engagement with a review of available evidence to generate research questions that address current research gaps and are important to patients and other stakeholders. Using the SEED method, patients and other stakeholders participated in research question development through a combination of collaborative, participatory, and consultative engagement. Steps in the process included: (1) identifying the topic and recruiting participants; (2) conducting focus groups and interviews; (3) developing conceptual models; (4) developing research questions; and (5) prioritizing research questions. Results: Stakeholders were involved in the SEED process from February to August 2015. Eighteen questions were prioritized for inclusion in the research agenda, covering diverse domains, from healthcare provision to social and environmental factors. Data analysis took place September to May 2016. During this time, researchers conducted a literature review to target research gaps. Conclusions: The stakeholder-prioritized, novel research questions developed through the SEED process can directly inform future research and guide the development of evidence that translates more directly to clinical practice.
Background: Deep learning offers great benefits in classification tasks such as medical imaging d... more Background: Deep learning offers great benefits in classification tasks such as medical imaging diagnostics or stock trading, especially when compared with human-level performances, and can be a viable option for classifying distinct levels within community-engaged research (CEnR). CEnR is a collaborative approach between academics and community partners with the aim of conducting research that is relevant to community needs while incorporating diverse forms of expertise. In the field of deep learning and artificial intelligence (AI), training multiple models to obtain the highest validation accuracy is common practice; however, it can overfit toward that specific data set and not generalize well to a real-world population, which creates issues of bias and potentially dangerous algorithmic decisions. Consequently, if we plan on automating human decision-making, there is a need for creating techniques and exhaustive evaluative processes for these powerful unexplainable models to ensure that we do not incorporate and blindly trust poor AI models to make real-world decisions. Objective: We aimed to conduct an evaluation study to see whether our most accurate transformer-based models derived from previous studies could emulate our own classification spectrum for tracking CEnR studies as well as whether the use of calibrated confidence scores was meaningful. Methods: We compared the results from 3 domain experts, who classified a sample of 45 studies derived from our university's institutional review board database, with those from 3 previously trained transformer-based models, as well as investigated whether calibrated confidence scores can be a viable technique for using AI in a support role for complex decision-making systems. Results: Our findings reveal that certain models exhibit an overestimation of their performance through high confidence scores, despite not achieving the highest validation accuracy. Conclusions: Future studies should be conducted with larger sample sizes to generalize the results more effectively. Although our study addresses the concerns of bias and overfitting in deep learning models, there is a need to further explore methods that allow domain experts to trust our models more. The use of a calibrated confidence score can be a misleading metric when determining our AI model's level of competency.
NAM perspectives, Jun 5, 2014
It is now widely recognized that health outcomes are deeply influenced by a variety of social fac... more It is now widely recognized that health outcomes are deeply influenced by a variety of social factors outside of health care. The dramatic differences in morbidity, mortality, and risk factors that researchers have documented within and between countries are patterned after classic social determinants of health, such as education and income (Link and Phelan, 1995; CSDH, 2008), as well as placed-based characteristics of the physical and social environment in which people live-and the macrostructural policies that shape them. A 2013 report from the National Research Council and the Institute of Medicine cited these socioecological factors, along with unhealthy behaviors and deficiencies in the health care system, as leading explanations for the "health disadvantage" of the United States. In a comparison of 17 high-income countries, age-adjusted all-cause mortality rates for 2008 ranged from 378.0 per 100,000 in Australia to 504.9 in the United States. The report found a pervasive pattern of health disadvantages across diverse categories of illness and injury that existed across age groups, sexes, racial and ethnic groups, and social classes (NRC and IOM, 2013). Recent attention has focused on the substantial health disparities that exist within the United States, where life expectancy varies at the state level by 7.0 years for males and 6.7 years for females (NRC and IOM, 2013) but mortality and life expectancy vary even more substantially across smaller geographic areas such as counties (University of Wisconsin Population Health Institute, 2013; Kulkarni et al., 2011) and census tracts. In many U.S. cities, life expectancy can vary by as much as 25 years across neighborhoods (Evans et al., 2012). The same dramatic geographic disparities can be seen for other outcomes, such as infant mortality, obesity, and the prevalence of diabetes and other chronic diseases. Of the various social determinants of health that explain health disparities by geography or demographic characteristics (e.g., age, gender, race-ethnicity), the literature has always pointed prominently to education. Research based on decades of experience in the developing world has identified educational status (especially of the mother) as a major predictor of health outcomes, and economic trends in the industrialized world have intensified the relationship between education and health. In the United States, the gradient in health outcomes by educational attainment has steepened over the last four decades (Goldman and Smith, 2011; Olshansky et al., 2012) in all regions of the United States (Montez and Berkman, 2014), producing a larger gap in health status between Americans with high and low education. Among white Americans without a high school diploma, especially women, life expectancy has decreased since the 1990s, 1 The authors are participants in the activities of the IOM Roundtable on Population Health Improvement.
Rural and Remote Health, Jun 1, 2022
Introduction: Opioid use disorder is a leading public health issue in the USA, with complex drive... more Introduction: Opioid use disorder is a leading public health issue in the USA, with complex drivers requiring a multi-level response. Rural communities are particularly affected by opioid misuse. Due to variability in local conditions and resources, they require community-specific responses. The aim of this study was to gain insight into the perceptions, knowledge, and experiences of Rural and Remote Health rrh.org.au
<sec> <title>BACKGROUND</title> <p>Community-Engaged Research (CEnR) is a... more <sec> <title>BACKGROUND</title> <p>Community-Engaged Research (CEnR) is a research approach in which scholars partner with community organizations or individuals with whom they share an interest in the study topic, typically with the goal of supporting that community's wellbeing. CEnR is well-established in numerous disciplines including the clinical and social sciences. However, universities experience challenges reporting comprehensive CEnR metrics, limiting development of appropriate CEnR infrastructure and advancement of relationships with communities, funders, and stakeholders.</p> </sec> <sec> <title>OBJECTIVE</title> <p>n/a</p> </sec> <sec> <title>METHODS</title> <p>We propose a novel approach to identifying and categorizing community-engaged studies by applying attention-based deep learning models to human subjects protocols that have been submitted to the university's Institutional Review Board (IRB). We manually classified a sample of protocols submitted to the IRB using a 3 and 6-level CEnR heuristic. We then trained an attention-based Bidirectional-LSTM on the classified protocols and compared it to transformer models such as BERT, Bio+ClinicalBERT, and XLM-RoBERTa. We applied the best performing models to the full sample of unlabeled IRB protocols submitted in the years 2013-2019 (n &gt; 6000).</p> </sec> <sec> <title>RESULTS</title> <p>Transfer learning appears to be superior, receiving a .9952 testing F1 Score for all transformer models implemented compared to the attention-based Bi-LSTM model. This finding is consistent across several methodological adjustments: an augmented dataset with and without cross-validation, an unaugmented dataset with and without cross-validation, a 6 class CEnR spectrum, and a 3 class one. BERT and the transformer models showed an understanding of our data unlike the attention-based model, promising usability for real-world application.</p> </sec> <sec> <title>CONCLUSIONS</title> <p>Transfer learning is a viable method for differentiating small datasets characterized by the idiosyncrasies and errors of CEnR descriptions used by principal investigators in research protocols.</p> </sec>
JMIR formative research, Sep 6, 2022
Background: Community-engaged research (CEnR) is a research approach in which scholars partner wi... more Background: Community-engaged research (CEnR) is a research approach in which scholars partner with community organizations or individuals with whom they share an interest in the study topic, typically with the goal of supporting that community's well-being. CEnR is well-established in numerous disciplines including the clinical and social sciences. However, universities experience challenges reporting comprehensive CEnR metrics, limiting the development of appropriate CEnR infrastructure and the advancement of relationships with communities, funders, and stakeholders. Objective: We propose a novel approach to identifying and categorizing community-engaged studies by applying attention-based deep learning models to human participants protocols that have been submitted to the university's institutional review board (IRB). Methods: We manually classified a sample of 280 protocols submitted to the IRB using a 3-and 6-level CEnR heuristic. We then trained an attention-based bidirectional long short-term memory unit (Bi-LSTM) on the classified protocols and compared it to transformer models such as Bidirectional Encoder Representations From Transformers (BERT), Bio + Clinical BERT, and Cross-lingual Language Model-Robustly Optimized BERT Pre-training Approach (XLM-RoBERTa). We applied the best-performing models to the full sample of unlabeled IRB protocols submitted in the years 2013-2019 (n>6000). Results: Although transfer learning is superior, receiving a 0.9952 evaluation F1 score for all transformer models implemented compared to the attention-based Bi-LSTM (between 48%-80%), there were key issues with overfitting. This finding is consistent across several methodological adjustments: an augmented data set with and without cross-validation, an unaugmented data set with and without cross-validation, a 6-class CEnR spectrum, and a 3-class one. Conclusions: Transfer learning is a more viable method than the attention-based bidirectional-LSTM for differentiating small data sets characterized by the idiosyncrasies and variability of CEnR descriptions used by principal investigators in research protocols. Despite these issues involving overfitting, BERT and the other transformer models remarkably showed an understanding of our data unlike the attention-based Bi-LSTM model, promising a more realistic path toward solving this real-world application.
Journal of clinical and translational science, Nov 22, 2021
Community-engaged research (CEnR) is now an established research approach. The current research s... more Community-engaged research (CEnR) is now an established research approach. The current research seeks to pilot the systematic and automated identification and categorization of CEnR to facilitate longitudinal tracking using administrative data. We inductively analyzed and manually coded a sample of Institutional Review Board (IRB) protocols. Comparing the variety of partnered relationships in practice with established conceptual classification systems, we developed five categories of partnership: Non-CEnR, Instrumental, Academicled, Cooperative, and Reciprocal. The coded protocols were used to train a deep-learning algorithm using natural language processing to categorize research. We compared the results to data from three questions added to the IRB application to identify whether studies had a community partner and the type of engagement planned. The preliminary results show that the algorithm is potentially more likely to categorize studies as CEnR compared to investigator-recorded data and to categorize studies at a higher level of engagement. With this approach, universities could use administrative data to inform strategic planning, address progress in meeting community needs, and coordinate efforts across programs and departments. As scholars and technical experts improve the algorithm's accuracy, universities and research institutions could implement standardized reporting features to track broader trends and accomplishments.
Research Involvement and Engagement, Jan 11, 2019
There is a need for methods that engage lay people and other stakeholders, such as patients and h... more There is a need for methods that engage lay people and other stakeholders, such as patients and healthcare providers, in developing research questions about health issues important to them and their communities. Involving stakeholders helps ensure that funding goes to research that addresses their concerns. The SEED Method engages stakeholders in a systematic process to explore health issues and develop research questions. Diverse groups of stakeholders participate at three levels: as collaborators that lead the process throughout, as participants who use their expertise to develop the questions, and as consultants who provide additional perspectives about the health topic. We used the SEED Method to engage 61 stakeholders from different socioeconomic and professional backgrounds to create research questions on lung cancer outcomes. Participants included cancer patients and caregivers, healthcare providers and administrators, and policymakers from a rural Virginia community. They developed causal models that diagrammed factors that influence lung cancer outcomes and the relationships between them. They used these models to develop priority research questions. The questions reflect the participants' diverse perspectives and address different areas of inquiry related to lung cancer outcomes, including access to care, support systems, social determinants of health, and quality of care. Participants felt well prepared to perform the project tasks because they had the opportunity to review lung cancer information, receive causal model and research question development training, and participate in facilitated group activities. The SEED Method can be used in a variety of settings and applied to any health topic of interest to stakeholders.
Children and Youth Services Review, Jul 1, 2005
Although research that focuses on sibling placements in foster care has increased in recent years... more Although research that focuses on sibling placements in foster care has increased in recent years, for the most part this research has focused on single samples from a point-in-time perspective. In this paper, we approach the matter of sibling placements with longitudinal data, differentiating between the notions of togetherness and intactness in order to describe the placement experiences of sibling groups. We generally found that, although siblings often enter care on the same day, they make up less than half the groups entering care. We also found that small sibling groups are more likely to be placed intact. So, too, are siblings placed with relatives. We also studied intactness over time. All told, when the movement between statuses is accounted for fully, more sibling groups were intact at 6 months as a percentage of children still in care than at the time of placement. Moreover, there is evidence that separated siblings who remain in care are sometimes brought together over time, sibling group size and placement type affect the likelihood that siblings are brought together, and children who follow their siblings into care are much less likely to be placed with a sibling compared to siblings that enter foster care on the same day.
JMIR formative research, Mar 20, 2023
Background Deep learning offers great benefits in classification tasks such as medical imaging di... more Background Deep learning offers great benefits in classification tasks such as medical imaging diagnostics or stock trading, especially when compared with human-level performances, and can be a viable option for classifying distinct levels within community-engaged research (CEnR). CEnR is a collaborative approach between academics and community partners with the aim of conducting research that is relevant to community needs while incorporating diverse forms of expertise. In the field of deep learning and artificial intelligence (AI), training multiple models to obtain the highest validation accuracy is common practice; however, it can overfit toward that specific data set and not generalize well to a real-world population, which creates issues of bias and potentially dangerous algorithmic decisions. Consequently, if we plan on automating human decision-making, there is a need for creating techniques and exhaustive evaluative processes for these powerful unexplainable models to ensure that we do not incorporate and blindly trust poor AI models to make real-world decisions. Objective We aimed to conduct an evaluation study to see whether our most accurate transformer-based models derived from previous studies could emulate our own classification spectrum for tracking CEnR studies as well as whether the use of calibrated confidence scores was meaningful. Methods We compared the results from 3 domain experts, who classified a sample of 45 studies derived from our university’s institutional review board database, with those from 3 previously trained transformer-based models, as well as investigated whether calibrated confidence scores can be a viable technique for using AI in a support role for complex decision-making systems. Results Our findings reveal that certain models exhibit an overestimation of their performance through high confidence scores, despite not achieving the highest validation accuracy. Conclusions Future studies should be conducted with larger sample sizes to generalize the results more effectively. Although our study addresses the concerns of bias and overfitting in deep learning models, there is a need to further explore methods that allow domain experts to trust our models more. The use of a calibrated confidence score can be a misleading metric when determining our AI model’s level of competency.
Teaching Sociology, May 18, 2015
materials accompanying the book for the instructor or the student, the book’s appendix does inclu... more materials accompanying the book for the instructor or the student, the book’s appendix does include specific tips and examples as well as a listing of references and other resources. Making this book available through the library system would, however, be a beneficial resource for students who are especially interested in qualitative analysis or who perhaps respond better to the book’s creative format than more traditionally structured texts. The Good, the Bad, and the Data could be used in a variety of courses. For example, it could be distributed as a supplementary text for research methods material in an introductory course. Students may be given an assignment to interview some peers on a current topic and then have the class analyze the data using the book as a reference guide. Actually, it could be used in any subjectspecific course where an instructor may want to have the class experience qualitative data collection and analysis, as Galman’s approach does not assume significant statistical expertise. Last, for an undergraduate student who chooses to use a content analysis or interview methodology in a capstone course, this book would be an excellent starting point, as the information provided about qualitative analysis is presented at a level that is accessible to an undergraduate audience with a wide spectrum of research skills.
Michigan Journal of Community Service-Learning, Dec 12, 2018
Each community-based participatory research (CBPR) partnership may incur "ripple effects"-impacts... more Each community-based participatory research (CBPR) partnership may incur "ripple effects"-impacts that happen outside the scope of planned projects. We used brainstorming and interviewing to create a roadmap that incorporated input from nine CBPR participants and five community/academic partners to retrospectively assess the ripple effects observed after five years of participatory research in one urban community. The resulting roadmap reflected a range of community impacts which we then divided into four key areas: impacts in the community (i.e., strategies, programs, and policies implemented by community partners), impacts on the CBPR team, impacts on individuals (participants and community members), and contributions to the field and the university. Our approach focused on observing what happened in the community that was directly or indirectly related to our partnership, process, products, and relationships. Much of the impact we observed reflected the synergy of sharing our research and community voice with responsive partners and stakeholders.
Urban Education, Dec 28, 2016
Awareness of the impact of education on health remains relatively low among the public, professio... more Awareness of the impact of education on health remains relatively low among the public, professionals, and policy makers. Virginia Commonwealth University's Center on Society and Health sought to raise awareness among key decision makers about the impact of education on health outcomes through its Education and Health Initiative (EHI). EHI utilized four key strategies to raise awareness: user-oriented research, strategic communication, local and national stakeholder engagement, and policy outreach to decision makers. We review the research highlighted in four stages of EHI product releases, as well as the development, process, products, and key outcomes associated with this initiative.
Uploads
Books by Emily Zimmerman
Papers by Emily Zimmerman