Institutional Effectiveness by Aaron S Horn
Journal of Education Finance, 2023
This study estimates the effect of state appropriations on the graduation rates of freshman cohor... more This study estimates the effect of state appropriations on the graduation rates of freshman cohorts by race/ethnicity. Data were obtained for public four-year institutions (n = 415) representing six freshman cohorts between 2007 and 2012. Hybrid regression models indicated that a ten percent increase in appropriations would yield a percentage point increase in graduation rates of .59 for all students, .99 for Black students, .84 for Latinx students, and .59 for White students. However, the effect of state appropriations on graduation rates varied across institutions (-1.03 to 2.99 percentage point change) and was frequently larger at institutions with medium or high subsidy reliance (.70 to 1.39 percentage point
change). Also, the effect of state appropriations on Black student graduation rates was 2.48 times larger at HBCUs. This study suggests that state appropriations can be an effective instrument for raising the graduation rates of diverse students to help meet state attainment goals.
Midwestern Higher Education Compact, 2021
This report illustrates a method for developing state-level projections of postsecondary enrollme... more This report illustrates a method for developing state-level projections of postsecondary enrollment, credential production, and the college-educated population. Projections are developed for several underrepresented, non-traditional, and historically disadvantaged groups, including Black, Latinx, low-income, and older adult populations. Two types of projections are provided throughout the report. First, a “status quo” projection assumes that current rates of college enrollment, completion, and attainment will largely remain constant through 2035. These status quo projections show that the state would fail to make significant progress in bridging attainment gaps. Second, “equity scenario” projections show an alternative future created by gradually closing gaps in enrollment, completion, and attainment over the same projection period.
STUDIES IN HIGHER EDUCATION, 2017
Researchers in higher education frequently evaluate institutional effectiveness as the difference... more Researchers in higher education frequently evaluate institutional effectiveness as the difference between an actual and predicted graduation rate, but little is known about whether such a method is reliable or valid. This study examines the measurement properties of effectiveness scores derived from regression residuals for community colleges in the United States (n = 875). The test-retest reliability of the resulting effectiveness measure over two consecutive years was acceptable to good (r = .63 to .72). A test of convergent validity indicated that effectiveness scores were positively associated with students' perceptions of a supportive campus environment (r = .26). A test of discriminant validity revealed relatively small correlations between effectiveness scores and institutional attributes, such as the percent of low-income students (r = −.01). The results support the use of effectiveness scores as a complementary performance indicator for community colleges to better articulate the value added to the student experience.
Research in Higher Education, 2016
A relatively simple way of measuring institutional effectiveness in relation to degree completion... more A relatively simple way of measuring institutional effectiveness in relation to degree completion is to estimate the difference between an actual and predicted graduation rate, but the reliability and validity of this method have not been thoroughly examined. Longitudinal data were obtained from IPEDS for both public and private not-for-profit 4-year institutions (n = 1496). Hierarchical panel regression was used to predict 4- and 6-year graduation rates based on structural, demographic, financial, and contextual attributes. A direct effects model yielded effectiveness scores that were highly correlated between consecutive data years (r = 0.65–0.80), which indicated acceptable to good test–retest reliability. A test of convergent validity indicated that effectiveness scores were positively associated with students’ perceptions of a supportive campus environment (r = 0.32–0.45). A test of discriminant validity revealed relatively small correlations between effectiveness scores and institutional attributes, such as educational expenditures (r = 0.07–0.16). The modeling of interaction effects in relation to institutional type marginally improved the validity of effectiveness scores among public but not private institutions. The results suggest that correct model specification can yield residual scores that reliably and validly measure institutional effectiveness in promoting timely degree completion.
Assessing Quality in Postsecondary Education: International Perspectives, 2018
KEDI Journal of Educational Policy, 2019
Institutional efficiency is frequently measured as a ratio between input and output (e.g., expend... more Institutional efficiency is frequently measured as a ratio between input and output (e.g., expenditures per credential), but differences in cost structures preclude attempts to accurately compare and benchmark performance across institutions. This study thus compares actual and predicted educational expenditures while accounting for variation in institutional mission, degree production profiles, faculty employment, and the cost of living. Longitudinal data were obtained from IPEDS for both public and private not-for-profit four-year institutions (n=1,496). The Regional Price Parities index was used to adjust expenditures for differences in the cost of living. Panel regression was used to predict educational expenditures from the number of credentials by award level and discipline, the proportion of full-time faculty, the student-faculty ratio, and average professor salary. An educational expenditures index was then computed as the standardized, three-year average difference between actual and predicted expenditures. Cross-sector comparisons revealed that the prevalence of public four-year institutions with higher-than-expected expenditures ranged from 19 to 25 percent, compared to 36 to 40 percent of private institutions.
Educational Policy, 2019
This study illustrates a method for evaluating the accuracy of performance metrics and identifies... more This study illustrates a method for evaluating the accuracy of performance metrics and identifies potential institutional and student attributes that may increase the likelihood of classification errors. Longitudinal data were obtained from the Integrated Postsecondary Eduation Data System for public 4-year institutions (N = 558) to evaluate various performance metrics based on graduation rates, total credentials conferred, and total credentials per 100 full-time equivalent students. A value-added measure defined as the difference between an institution's actual and predicted graduation rate served as the reference standard. The results revealed that 18% to 56% of institutions were misclassified as effective or ineffective when using unadjusted performance metrics. Mean comparisons indicated that institutions misclassified as ineffective frequently had input disadvantages (e.g., low admissions selectivity). Some performance metrics may thus yield unacceptably high error rates in classifying institutions as effective or ineffective, which may hinder efforts to identify best practices and reward institutions that promote state and federal attainment goals.
Journal of Studies in International Education, 2007
This study presents an analysis of the relative internationalization of 77 research universities ... more This study presents an analysis of the relative internationalization of 77 research universities in the United States. Institutions enrolling undergraduate students were selected from the 2003 national report, The Top American Research Universities. Data were collected from publicly available sources for 19 indicators of internationalization pertaining to student characteristics, scholar characteristics, research orientation, curricular content, and organizational support. Data were standardized, weighted by a panel of experts, and summed to yield an overall internationalization index score for each institution. Index scores were then used to rank the 77 institutions. A sensitivity analysis yielded a significant positive correlation (.97, p < .001) between the ranking based on the weighted indicators and a ranking derived from unweighted indicators.
Civic and Moral Development by Aaron S Horn
Academic Medicine, 2007
Purpose The authors examine training in the responsible conduct of research and mentoring in rela... more Purpose The authors examine training in the responsible conduct of research and mentoring in relation to behaviors that may compromise the integrity of science. Method The analysis is based on data from the authors' 2002 national survey of 4,160 early-career and 3,600 midcareer biomedical and social science researchers who received research support from the U.S. National Institutes of Health. The authors used logistic regression analysis to examine associations between receipt of separate or integrated training in research ethics, mentoring related to ethics and in general, and eight categories of ethically problematic behavior. Analyses controlled for gender, type of doctoral degree, international degree, and disciplinary field. Results Responses were received from 1,479 early-career and 1,768 midcareer scientists, yielding adjusted response rates of 43% and 52%, respectively. Results for early-career researchers: Training in research ethics was positively associated with problematic behavior in the data category. Mentoring related to ethics and research, as well as personal mentoring, decreased the odds of researchers' engaging in problematic behaviors, but mentoring on financial issues and professional survival increased these odds. Results for midcareer researchers: Combined separate and integrated training in research ethics was associated with decreased odds of problematic behavior in the categories of policy, use of funds, and cutting corners. Ethics mentoring was associated with lowered odds of problematic behavior in the policy category. Conclusions The effectiveness of training in obviating problematic behavior is called into question. Mentoring has the potential to influence behavior in ways that both increase and decrease the likelihood of problematic behaviors.
Journal of Youth and Adolescence, 2012
Community service is widely regarded as a fundamental experience in preparation for good citizens... more Community service is widely regarded as a fundamental experience in preparation for good citizenship, but it remains unclear whether common variants of service are consequential for civic outcomes. This study examines changes in the relative importance assigned to prosocial and egoistic values associated with service through different types of organizations, service prompted by external contingencies, and service that spans a narrow or wide frame of time. Data were drawn from the survey responses of 16,749 secondary school students (50 percent female, 28 percent ethnic minority, modal age = 15) who participated in the National Educational Longitudinal Study during their sophomore year in 1990 and completed a follow-up survey during their senior year in 1992. Results from a propensity score analysis indicated that service through humanitarian organizations but not other types of organizations was positively associated with the adoption of a prosocial value orientation. Service prompted by an institutional mandate or social pressure was negatively associated with prosociality relative to service characterized as strictly voluntary, although all students except those with an initially egoistic value orientation benefited from mandatory service relative to no service. While short-term service during the sophomore year had no enduring effect, long-term service predicted gains in prosociality above and beyond the effect of concurrent service. The findings suggest that the relationship between community service and prosocial value development can be optimized to the extent that service through a humanitarian organization is prompted by autonomy-supportive conditions over the course of 2 years.
Higher Education, 2012
While considerable consensus exists regarding the importance of internationalizing postsecondary ... more While considerable consensus exists regarding the importance of internationalizing postsecondary education, little is known about whether distinct forms of internationalization influence civic engagement. This study estimates the degree to which study abroad and internationalization at home (IaH) are associated with international volunteerism among undergraduate alumni at research universities in the United States (n = 93). Results from a multiple regression analysis revealed that extensive IaH—reflected by both the presence of internationalized general education and a high concentration of international students—and the institutional study abroad rate were both positively associated with international volunteerism, operationalized as participation in the Peace Corps. Limited forms of IaH were not predictive of international volunteerism. Regression models explained between 29 and 38% of the variance in international volunteerism. The findings suggest that institutions can better prepare students for the challenges of global civic engagement by increasing participation in study abroad and implementing an extensive and systematic IaH strategy.
VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 2013
Study abroad is cited commonly as a critical element of education for global civic engagement, bu... more Study abroad is cited commonly as a critical element of education for global civic engagement, but the significance of various programmatic features has not been well established. This study examines the relationships among three facets of study abroad programs—destination, type, and duration—and subsequent volunteerism through international development organizations. Survey responses were analyzed from 2,250 college alumni (71 % female, 14 % ethnic minority, M age = 27) who studied abroad between 1995 and 2005. The results of logistic regression indicated that studying abroad in a developing country and engaging in international service-learning were positively associated with the odds of development volunteerism. The number of months spent abroad also predicted development volunteerism, though duration did not moderate the effects of program destination and type. An interaction test demonstrated that program effects were not dependent on gender. The findings suggest that differences in the nature of study abroad programs influence the extent to which participants become actively engaged in global civil society.
University of Minnesota Digital Conservancy, 2013
Several facets of the undergraduate experience have been linked with the importance students assi... more Several facets of the undergraduate experience have been linked with the importance students assign to prosocial values (e.g., helping people in need), including academic major, exposure to social and cultural diversity, community service, and a prosocial institutional ethos. However, these factors have not been examined within a single framework that links prosocial value development to educative conditions and critical psychological assets, including empathy, existential engagement, prosocial expectancies, and social problem understanding. Longitudinal data were derived from the responses of 7,709 undergraduate students (59 percent female, 15 percent non-White ethnicity, modal age= 18) who completed the CIRP Freshman Survey during 1991 and the College Senior Survey as graduating seniors during 1995. Responses were examined with multiple regression and path analysis. The multiple regression results demonstrated that gains in empathy, existential engagement, and prosocial expectancies, as well as perceived change in social problem understanding, predicted greater prioritization of helping people in need than of being financially well off during the senior year (β= .08-.14). Majoring in any field, relative to majoring in business or economics, was positively associated with prosocial value prioritization. Particularly large associations with prosocial value prioritization were observed among students who majored in the humanities, psychology, social sciences, caring-oriented fields (e.g., education), and biology (β= .37-.50), relative to majoring in business. Path model results indicated that majoring in the social sciences was indirectly associated with the value measure via perceived change in social problem understanding, and majoring in the humanities predicted value orientation via existential engagement. Participation in diversity workshops, the completion of diversity coursework, and frequent cross-racial interactions predicted greater prosocial value development (β= .06-.16). The relationship between diversity and value orientation was at least partly mediated by empathy. Participation in volunteerism only (β= .20), or a combination of service-learning and volunteerism (β= .33) but not service-learning alone, was positively associated with the development of a prosocial value orientation. Moreover, an analysis of the organizational context of service revealed that service through educational, healthcare, community relief, and social services organizations (but not through public safety, political, recreational, or environmental organizations) was uniquely associated with senior-year value orientation (β= .08 -.15). Although the duration of service was positively associated with prosocial value prioritization, direct contrasts of service duration categories indicated that participation in service for less than one month did not predict gains in prosocial value development relative to no service. Furthermore, experiences of 7-12 months were not associated with greater prosocial value development relative to service duration of more than 12 months. Each service variable was also indirectly associated with value orientation: the combination of service-learning and volunteerism via social problem understanding; social services organization via each psychological asset; community relief organization via existential engagement and social problem understanding; and service duration via empathy, prosocial outcome expectancies, and social problem understanding. Finally, exposure to a prosocial ethos was also positively associated with prosocial value prioritization (β= .14), an association that was partly mediated by existential engagement. The findings suggest that prosocial value development can be promoted by (a) ensuring that major curricula explicitly endorse the prioritization of prosocial ideals, incite a reasoned examination of ethical dilemmas, and connect specialized knowledge with social problems and solutions; (b) increasing exposure to a general civic curriculum, including diversity coursework and high-quality service-learning courses; (c) facilitating interactions among students of diverse backgrounds; (d) promoting participation in a co-curriculum that includes diversity workshops and volunteerism through humanitarian organizations; and (e) cultivating a campus community that expects and recognizes the pursuit of prosocial ideals. Among the future directions for research, a longitudinal analysis of multiple time lags is needed to understand the dynamic changes in the reflective evaluation, affirmation, and enactment of personal values during college.
Mental Health by Aaron S Horn
Higher Education Policy volume, 2017
The purpose of this article is to provide an overview of mental health issues and counseling serv... more The purpose of this article is to provide an overview of mental health issues and counseling services on college campuses in the USA. The findings from several national surveys are reviewed to estimate the prevalence of anxiety and depression, suicide and suicidal ideation, and violence among college students. Common prevention and treatment programs are then described with particular attention to innovative campus-wide programs. Student outcomes research is examined to determine whether receiving counseling services is associated with academic performance and the likelihood of graduation. The article concludes with a set of recommended practices to improve the effectiveness of counseling services on campus.
Journal of Human Behavior in the Social Environment, 2008
Many assertions have been made that the changing landscape of postsecondary education has resul... more Many assertions have been made that the changing landscape of postsecondary education has resulted in a higher level of occupational stress, thereby presenting a potential problem for the recruitment and retention of individuals in academe. However, it has yet to be empirically demonstrated that stressors for faculty have in fact changed by nature or intensity during the last decade of the 20th century. The present study compares the dimensions and correlates of faculty stress using national survey data collected in 1989 and 2001. A factor analysis of 16 stressor items was conducted to construct five stress scales, comprising work overload, role conflict, faculty interaction, academic advancement, and aging considerations. Multiple regression, using four blocks of variables (i.e., demographics, faculty status, work life, and cohort), explained between 6 and 21 percent of the variance in the stress scales. Results indicated that sources of stress have remained relatively stable between 1989 and 2001.
Uploads
Institutional Effectiveness by Aaron S Horn
change). Also, the effect of state appropriations on Black student graduation rates was 2.48 times larger at HBCUs. This study suggests that state appropriations can be an effective instrument for raising the graduation rates of diverse students to help meet state attainment goals.
Civic and Moral Development by Aaron S Horn
Mental Health by Aaron S Horn
change). Also, the effect of state appropriations on Black student graduation rates was 2.48 times larger at HBCUs. This study suggests that state appropriations can be an effective instrument for raising the graduation rates of diverse students to help meet state attainment goals.