This report focuses on how situational characteristics affect feedback seeking. It also examines ... more This report focuses on how situational characteristics affect feedback seeking. It also examines how individual differences in feedback propensities affect feedback seeking. Proposed antecedents of feedback eliciting (overt feedback seeking) were examined utilizing a computer-based technique that permitted objective measurement of the behavior. A 2 X 2 research design was used, with two levels of social presence (an observer present or absent) and two different task rationales (evaluating the task evaluating the participant). Feedback eliciting was operationalized in two complementary ways-as the number of times the participant elicited feedback and as the number of seconds he or she spent examining feedback information. Two types of feedback were examined: (1) outcome feedback or information about level of performance, and (2) process feedback or information about how to improve performance. Separate hypotheses were formulated for each, in addition, to social presence and task rationale, the following variables were hypothesized to affect feedback eliciting: performance, arousal (measured as state anxiety), external feedback propensity, task-specific internal feedback ability, task familiarity, internal feedback propensity, self-esteem, locus of control, tolerance for ambiguity, and need for achievement.
Many statistical techniques were originally invented for the purpose of making comparisons betwee... more Many statistical techniques were originally invented for the purpose of making comparisons between and among groups of entities. Not surprisingly, difficulties arise when the entities are human beings. Comparisons among groups of people are straightforward when the variables have objective operationalizations, such as age, income, and education level. They are less straightforward when the variables are attitudes, such as self-esteem and job satisfaction. Attitudes are usually operationalized as scale scores, and before such scores can be meaningfully compared, it must be shown that the scale items "mean the same thing" to members of all the groups being studied. This is the so-called measurement invariance problem. In recent years, structural equation modeling (SEM) has been used to test the measurement invariance of survey-type items across two groups; e.g., two national cultures. Unfortunately the currently recommended procedures are computationally intensive and not well suited for testing item invari...
This study of 137 helicopter pilot trainees investigated individual strategies used to obtain per... more This study of 137 helicopter pilot trainees investigated individual strategies used to obtain performance feedback during two consecutive phases of their training. Individual and situational factors cited in previous research were investigated as predictors of two feedback seeking behaviors: eliciting (asking directly for feedback) and monitoring (using indirect techniques, such as observing, to gain additional feedback). Both individual and situational factors were significant predictors of feedback seeking behaviors. Feedback seeking costs and the student pilots' external propensity (an individual difference measure assessing their desire for external feedback) were found to be the most consistent predictors of feedback eliciting and monitoring, both within and across the two training phases. In addition, the results point to higher feedback eliciting when performance was rated as low. The implications of this research are discussed, especially with respect to training. Within the performance feedback area, the topic of feedback seeking has been receiving increased attention (Miller & Jablin, 1991; Van Dyne, Earley, & Cummings, 1990). One reason for this interest is that the investigation of feedback seeking behaviors (FSBs) has changed the direction of feedback research. Prior to Ashford and Cummings (1983), most research treated the feedback recipient as relatively passive (11gen, Fisher, & Taylor, 1979); feedback was seen as being given to recipients and responses occurred based on individual differences (e.g., self-esteem) and situational variables (e.g., the superior-subordinate relationship). This viewpoint was overly simplistic in that feedback recipients exist in information rich environments (Hanser & Muchinsky, 1978; The authors are indebted to the following personnel of the U.S. Army Research Institute Aviation R & D Activity, Fort Rucker, Alabama: Charles A. Gainer (Chief), Gabriel €? Intano (Team Leader), William R. Howse (Psychologist), and Larry J. Murdock (ADP Manager). Special thanks are due to our project coordinator, Gabe Intano; without his interest and support over a period of 9 months, the research reported in this paper could not have been performed. The authors would also like to thank Charles K. Parson, Dennis H. Nagao, Robert C. Liden, and three anonymous reviewers. Their insightful comments were indispensable.
Many cross-cultural researchers are concerned with factorial invariance; that is, with whether or... more Many cross-cultural researchers are concerned with factorial invariance; that is, with whether or not members of different cultures associate survey items, or similar measures, with similar constructs. Researchers usually test items for factorial invariance using confirmatory factor analysis (CFA). CFA, however, poses certain problems that must be dealt with. Primary among them is standardization, the process that assigns units of measurement to the constructs (latent variables). Two standardization procedures and several minor variants have been reported in the literature, but using these procedures when testing for factorial invariance can lead to inaccurate results. In this paper we review basic theory, and propose an extension of Byrne, Shavelson, and Muthgn’s (1989) procedure for identifying non-invariant items. The extended procedure solves the standardization problem by performing a systematic comparison of all pairs of factor loadings across groups. A numerical example based upon a large published data set is presented to illustrate the utility of the new procedure, particularly with regard to partial factorial invariance.
Measurement invariance is usually tested using Multigroup Confirmatory Factor Analysis, which exa... more Measurement invariance is usually tested using Multigroup Confirmatory Factor Analysis, which examines the change in the goodness-of-fit index (GFI) when cross-group constraints are imposed on a measurement model. Although many studies have examined the ...
Educational and Psychological Measurement, Dec 1, 1998
Comparing different groups (e.g., cultures, age cohorts) using survey-type instruments raises the... more Comparing different groups (e.g., cultures, age cohorts) using survey-type instruments raises the question of factorial invariance, that is, whether or not members of different groups ascribe the same meanings to survey items. This article attempts to advance multi-group research by (a) providing a concise summary of the factorial invariance problem, (b) proposing a simplified notation intended to facilitate discussion of the problem, and (c) suggesting a structured approach for testing large models. This procedure is illustrated using an extended example. Two computer programs designed to make the recommended procedures less laborious are offered.
The fit between a structural equation model and a data set is operationalized as the value of goo... more The fit between a structural equation model and a data set is operationalized as the value of goodness-of-fit indices. The discrepancy between the estimated value and the value indicating perfect fit has three sources: misspecification, error arising from theoretical parsimony in the description of the model (parsimony error), and sampling error. Misspecification, which represents a disparity between “realworld” relationships and relationships in the model, is the most important source of error for researchers. It cannot be accurately assessed, however, unless parsimony error and sampling error are taken into account. Parsimony error occurs in measurement models when secondary relationships are excluded. Secondary relationships are defined here as secondary factor loadings and error term correlations that have small values, no theoretical bases, and no substantive meaning. A simulation was conducted to examine the effects of parsimony error on perfect measurement models and to establish appropriate criteria for model fit when parsimony error is present.
Extreme response styles (ERS) and acquiescence response styles (ARS) may constitute important sou... more Extreme response styles (ERS) and acquiescence response styles (ARS) may constitute important sources of cross-cultural differences on survey-type instruments. Differences in ERS and ARS, if undetected, may give rise to spurious results that do not reflect genuine differences in attitudes or perceptions. Multiple-group confirmatory factor analysis is recommended as the most effective method of testing for ERS and ARS and determining whether cultural groups can be meaningfully compared on the basis of factor (latent) means. A detailed numerical example is provided.
International Journal of Human Resource Management, May 1, 2006
This study examines the influence of individual needs and task characteristics on a multifaceted ... more This study examines the influence of individual needs and task characteristics on a multifaceted definition of perceived empowerment using an interactional perspective. Respondents were Hong Kong Chinese employees and most of them lower-level or nonmanagerial level workers in business organizations. We used regression analyses to test a number of hypotheses informed by social psychology theories. The significant findings indicate that
Educational and Psychological Measurement, Feb 1, 1996
This article describes the development of three domain-specific measures of individual difference... more This article describes the development of three domain-specific measures of individual differences in feedback propensities. In a series of studies, the authors identify the primary dimensions, psychometric characteristics, and construct validation evidence for internal ability, internal propensity, and external propensity for feedback. Confirmatory factor analysis supports the three dimensional representation. Correlations between the new scales and existing measures of personality are consistent with theoretical predictions. Theoretical and practical extensions of the current work are discussed.
Many cross-cultural researchers are concerned with factorial invariance; that is, with whether or... more Many cross-cultural researchers are concerned with factorial invariance; that is, with whether or not members of different cultures associate survey items, or similar measures, with similar constructs. Researchers usually test items for factorial invariance using confirma-tory factor analysis (CFA). CFA, however, poses certain problems that must be dealt with. Primary among them is standardization, the process that assigns units of measurement to the constructs (latent variables). Two standardization procedures and several minor variants have been reported in the literature, but using these procedures when testing for factorial invariance can lead to inaccurate results. In this paper we review basic theory, and propose an extension of Byrne, Shavelson, and Muthgn's (1989) procedure for identifying non-invariant items. The extended procedure solves the standardization problem by performing a systematic comparison of all pairs of factor loadings across groups. A numerical example...
Abstract Researchers who use survey-type instruments to perform cross-cultural studies are freque... more Abstract Researchers who use survey-type instruments to perform cross-cultural studies are frequently confronted with factorial non-invariance. This situation arises when the same set of items is used to measure a construct in different cultures, but the relationships between the items and the construct differ significantly across cultures. These (non-invariant) items are usually eliminated or ignored when calculating an index value representing the level of the construct. This paper proposes that under certain conditions non-invariant items ought to be retained, since they may constitute the only sources of information concerning cross-cultural differences. A new procedure is described that decomposes a cross-cultural difference into a component due to item responses, and a second component due to unequal item-construct relationships. A numerical example based upon a large published data set is presented, and the results of the new procedure are compared with those obtained using simple summated scores and regression-weighted factor scores.
Structural Equation Modeling: A Multidisciplinary Journal, 2002
Measurement invariance is usually tested using Multigroup Confirmatory Factor Analysis, which exa... more Measurement invariance is usually tested using Multigroup Confirmatory Factor Analysis, which examines the change in the goodness-of-fit index (GFI) when cross-group constraints are imposed on a measurement model. Although many studies have examined the ...
Relatively little attention has been given to detecting influential cases (ICs) when estimating s... more Relatively little attention has been given to detecting influential cases (ICs) when estimating structural equation models (SEMs). Most techniques examine individual cases using covariance-based techniques such as the Mahalanobis distance, which examine the distributional characteristics of the cases but ignore the model. Cases identified using such model-free techniques are usually referred to as out-liers. In SEM, however, the model is of central importance. The characteristics of the model (number of latent variables, etc.) have an effect on which cases are influential. The authors propose applying the well-known jackknife procedure to detect model-based ICs, which may be influential with respect to overall fit, particular model parameters, or both. The procedure is illustrated by two studies—one using simulated data, the other empirical data.
The fit between a structural equation model and a data set is operationalized as the value of goo... more The fit between a structural equation model and a data set is operationalized as the value of goodness-of-fit indices. The discrepancy between the estimated value and the value indicating perfect fit has three sources: misspecification, error arising from theoretical parsimony in the description of the model (parsimony error), and sampling error. Misspecification, which represents a disparity between “realworld” relationships and relationships in the model, is the most important source of error for researchers. It cannot be accurately assessed, however, unless parsimony error and sampling error are taken into account. Parsimony error occurs in measurement models when secondary relationships are excluded. Secondary relationships are defined here as secondary factor loadings and error term correlations that have small values, no theoretical bases, and no substantive meaning. A simulation was conducted to examine the effects of parsimony error on perfect measurement models and to estab...
Many cross-cultural researchers are concerned with factorial invariance; that is, with whether or... more Many cross-cultural researchers are concerned with factorial invariance; that is, with whether or not members of different cultures associate survey items, or similar measures, with similar constructs. Researchers usually test items for factorial invariance using confirmatory factor analysis (CFA). CFA, however, poses certain problems that must be dealt with. Primary among them is standardization, the process that assigns units of measurement to the constructs (latent variables). Two standardization procedures and several minor variants have been reported in the literature, but using these procedures when testing for factorial invariance can lead to inaccurate results. In this paper we review basic theory, and propose an extension of Byrne, Shavelson, and Muthgn’s (1989) procedure for identifying non-invariant items. The extended procedure solves the standardization problem by performing a systematic comparison of all pairs of factor loadings across groups. A numerical example based upon a large published data set is presented to illustrate the utility of the new procedure, particularly with regard to partial factorial invariance.
This report focuses on how situational characteristics affect feedback seeking. It also examines ... more This report focuses on how situational characteristics affect feedback seeking. It also examines how individual differences in feedback propensities affect feedback seeking. Proposed antecedents of feedback eliciting (overt feedback seeking) were examined utilizing a computer-based technique that permitted objective measurement of the behavior. A 2 X 2 research design was used, with two levels of social presence (an observer present or absent) and two different task rationales (evaluating the task evaluating the participant). Feedback eliciting was operationalized in two complementary ways-as the number of times the participant elicited feedback and as the number of seconds he or she spent examining feedback information. Two types of feedback were examined: (1) outcome feedback or information about level of performance, and (2) process feedback or information about how to improve performance. Separate hypotheses were formulated for each, in addition, to social presence and task rationale, the following variables were hypothesized to affect feedback eliciting: performance, arousal (measured as state anxiety), external feedback propensity, task-specific internal feedback ability, task familiarity, internal feedback propensity, self-esteem, locus of control, tolerance for ambiguity, and need for achievement.
Many statistical techniques were originally invented for the purpose of making comparisons betwee... more Many statistical techniques were originally invented for the purpose of making comparisons between and among groups of entities. Not surprisingly, difficulties arise when the entities are human beings. Comparisons among groups of people are straightforward when the variables have objective operationalizations, such as age, income, and education level. They are less straightforward when the variables are attitudes, such as self-esteem and job satisfaction. Attitudes are usually operationalized as scale scores, and before such scores can be meaningfully compared, it must be shown that the scale items "mean the same thing" to members of all the groups being studied. This is the so-called measurement invariance problem. In recent years, structural equation modeling (SEM) has been used to test the measurement invariance of survey-type items across two groups; e.g., two national cultures. Unfortunately the currently recommended procedures are computationally intensive and not well suited for testing item invari...
This study of 137 helicopter pilot trainees investigated individual strategies used to obtain per... more This study of 137 helicopter pilot trainees investigated individual strategies used to obtain performance feedback during two consecutive phases of their training. Individual and situational factors cited in previous research were investigated as predictors of two feedback seeking behaviors: eliciting (asking directly for feedback) and monitoring (using indirect techniques, such as observing, to gain additional feedback). Both individual and situational factors were significant predictors of feedback seeking behaviors. Feedback seeking costs and the student pilots' external propensity (an individual difference measure assessing their desire for external feedback) were found to be the most consistent predictors of feedback eliciting and monitoring, both within and across the two training phases. In addition, the results point to higher feedback eliciting when performance was rated as low. The implications of this research are discussed, especially with respect to training. Within the performance feedback area, the topic of feedback seeking has been receiving increased attention (Miller & Jablin, 1991; Van Dyne, Earley, & Cummings, 1990). One reason for this interest is that the investigation of feedback seeking behaviors (FSBs) has changed the direction of feedback research. Prior to Ashford and Cummings (1983), most research treated the feedback recipient as relatively passive (11gen, Fisher, & Taylor, 1979); feedback was seen as being given to recipients and responses occurred based on individual differences (e.g., self-esteem) and situational variables (e.g., the superior-subordinate relationship). This viewpoint was overly simplistic in that feedback recipients exist in information rich environments (Hanser & Muchinsky, 1978; The authors are indebted to the following personnel of the U.S. Army Research Institute Aviation R & D Activity, Fort Rucker, Alabama: Charles A. Gainer (Chief), Gabriel €? Intano (Team Leader), William R. Howse (Psychologist), and Larry J. Murdock (ADP Manager). Special thanks are due to our project coordinator, Gabe Intano; without his interest and support over a period of 9 months, the research reported in this paper could not have been performed. The authors would also like to thank Charles K. Parson, Dennis H. Nagao, Robert C. Liden, and three anonymous reviewers. Their insightful comments were indispensable.
Many cross-cultural researchers are concerned with factorial invariance; that is, with whether or... more Many cross-cultural researchers are concerned with factorial invariance; that is, with whether or not members of different cultures associate survey items, or similar measures, with similar constructs. Researchers usually test items for factorial invariance using confirmatory factor analysis (CFA). CFA, however, poses certain problems that must be dealt with. Primary among them is standardization, the process that assigns units of measurement to the constructs (latent variables). Two standardization procedures and several minor variants have been reported in the literature, but using these procedures when testing for factorial invariance can lead to inaccurate results. In this paper we review basic theory, and propose an extension of Byrne, Shavelson, and Muthgn’s (1989) procedure for identifying non-invariant items. The extended procedure solves the standardization problem by performing a systematic comparison of all pairs of factor loadings across groups. A numerical example based upon a large published data set is presented to illustrate the utility of the new procedure, particularly with regard to partial factorial invariance.
Measurement invariance is usually tested using Multigroup Confirmatory Factor Analysis, which exa... more Measurement invariance is usually tested using Multigroup Confirmatory Factor Analysis, which examines the change in the goodness-of-fit index (GFI) when cross-group constraints are imposed on a measurement model. Although many studies have examined the ...
Educational and Psychological Measurement, Dec 1, 1998
Comparing different groups (e.g., cultures, age cohorts) using survey-type instruments raises the... more Comparing different groups (e.g., cultures, age cohorts) using survey-type instruments raises the question of factorial invariance, that is, whether or not members of different groups ascribe the same meanings to survey items. This article attempts to advance multi-group research by (a) providing a concise summary of the factorial invariance problem, (b) proposing a simplified notation intended to facilitate discussion of the problem, and (c) suggesting a structured approach for testing large models. This procedure is illustrated using an extended example. Two computer programs designed to make the recommended procedures less laborious are offered.
The fit between a structural equation model and a data set is operationalized as the value of goo... more The fit between a structural equation model and a data set is operationalized as the value of goodness-of-fit indices. The discrepancy between the estimated value and the value indicating perfect fit has three sources: misspecification, error arising from theoretical parsimony in the description of the model (parsimony error), and sampling error. Misspecification, which represents a disparity between “realworld” relationships and relationships in the model, is the most important source of error for researchers. It cannot be accurately assessed, however, unless parsimony error and sampling error are taken into account. Parsimony error occurs in measurement models when secondary relationships are excluded. Secondary relationships are defined here as secondary factor loadings and error term correlations that have small values, no theoretical bases, and no substantive meaning. A simulation was conducted to examine the effects of parsimony error on perfect measurement models and to establish appropriate criteria for model fit when parsimony error is present.
Extreme response styles (ERS) and acquiescence response styles (ARS) may constitute important sou... more Extreme response styles (ERS) and acquiescence response styles (ARS) may constitute important sources of cross-cultural differences on survey-type instruments. Differences in ERS and ARS, if undetected, may give rise to spurious results that do not reflect genuine differences in attitudes or perceptions. Multiple-group confirmatory factor analysis is recommended as the most effective method of testing for ERS and ARS and determining whether cultural groups can be meaningfully compared on the basis of factor (latent) means. A detailed numerical example is provided.
International Journal of Human Resource Management, May 1, 2006
This study examines the influence of individual needs and task characteristics on a multifaceted ... more This study examines the influence of individual needs and task characteristics on a multifaceted definition of perceived empowerment using an interactional perspective. Respondents were Hong Kong Chinese employees and most of them lower-level or nonmanagerial level workers in business organizations. We used regression analyses to test a number of hypotheses informed by social psychology theories. The significant findings indicate that
Educational and Psychological Measurement, Feb 1, 1996
This article describes the development of three domain-specific measures of individual difference... more This article describes the development of three domain-specific measures of individual differences in feedback propensities. In a series of studies, the authors identify the primary dimensions, psychometric characteristics, and construct validation evidence for internal ability, internal propensity, and external propensity for feedback. Confirmatory factor analysis supports the three dimensional representation. Correlations between the new scales and existing measures of personality are consistent with theoretical predictions. Theoretical and practical extensions of the current work are discussed.
Many cross-cultural researchers are concerned with factorial invariance; that is, with whether or... more Many cross-cultural researchers are concerned with factorial invariance; that is, with whether or not members of different cultures associate survey items, or similar measures, with similar constructs. Researchers usually test items for factorial invariance using confirma-tory factor analysis (CFA). CFA, however, poses certain problems that must be dealt with. Primary among them is standardization, the process that assigns units of measurement to the constructs (latent variables). Two standardization procedures and several minor variants have been reported in the literature, but using these procedures when testing for factorial invariance can lead to inaccurate results. In this paper we review basic theory, and propose an extension of Byrne, Shavelson, and Muthgn's (1989) procedure for identifying non-invariant items. The extended procedure solves the standardization problem by performing a systematic comparison of all pairs of factor loadings across groups. A numerical example...
Abstract Researchers who use survey-type instruments to perform cross-cultural studies are freque... more Abstract Researchers who use survey-type instruments to perform cross-cultural studies are frequently confronted with factorial non-invariance. This situation arises when the same set of items is used to measure a construct in different cultures, but the relationships between the items and the construct differ significantly across cultures. These (non-invariant) items are usually eliminated or ignored when calculating an index value representing the level of the construct. This paper proposes that under certain conditions non-invariant items ought to be retained, since they may constitute the only sources of information concerning cross-cultural differences. A new procedure is described that decomposes a cross-cultural difference into a component due to item responses, and a second component due to unequal item-construct relationships. A numerical example based upon a large published data set is presented, and the results of the new procedure are compared with those obtained using simple summated scores and regression-weighted factor scores.
Structural Equation Modeling: A Multidisciplinary Journal, 2002
Measurement invariance is usually tested using Multigroup Confirmatory Factor Analysis, which exa... more Measurement invariance is usually tested using Multigroup Confirmatory Factor Analysis, which examines the change in the goodness-of-fit index (GFI) when cross-group constraints are imposed on a measurement model. Although many studies have examined the ...
Relatively little attention has been given to detecting influential cases (ICs) when estimating s... more Relatively little attention has been given to detecting influential cases (ICs) when estimating structural equation models (SEMs). Most techniques examine individual cases using covariance-based techniques such as the Mahalanobis distance, which examine the distributional characteristics of the cases but ignore the model. Cases identified using such model-free techniques are usually referred to as out-liers. In SEM, however, the model is of central importance. The characteristics of the model (number of latent variables, etc.) have an effect on which cases are influential. The authors propose applying the well-known jackknife procedure to detect model-based ICs, which may be influential with respect to overall fit, particular model parameters, or both. The procedure is illustrated by two studies—one using simulated data, the other empirical data.
The fit between a structural equation model and a data set is operationalized as the value of goo... more The fit between a structural equation model and a data set is operationalized as the value of goodness-of-fit indices. The discrepancy between the estimated value and the value indicating perfect fit has three sources: misspecification, error arising from theoretical parsimony in the description of the model (parsimony error), and sampling error. Misspecification, which represents a disparity between “realworld” relationships and relationships in the model, is the most important source of error for researchers. It cannot be accurately assessed, however, unless parsimony error and sampling error are taken into account. Parsimony error occurs in measurement models when secondary relationships are excluded. Secondary relationships are defined here as secondary factor loadings and error term correlations that have small values, no theoretical bases, and no substantive meaning. A simulation was conducted to examine the effects of parsimony error on perfect measurement models and to estab...
Many cross-cultural researchers are concerned with factorial invariance; that is, with whether or... more Many cross-cultural researchers are concerned with factorial invariance; that is, with whether or not members of different cultures associate survey items, or similar measures, with similar constructs. Researchers usually test items for factorial invariance using confirmatory factor analysis (CFA). CFA, however, poses certain problems that must be dealt with. Primary among them is standardization, the process that assigns units of measurement to the constructs (latent variables). Two standardization procedures and several minor variants have been reported in the literature, but using these procedures when testing for factorial invariance can lead to inaccurate results. In this paper we review basic theory, and propose an extension of Byrne, Shavelson, and Muthgn’s (1989) procedure for identifying non-invariant items. The extended procedure solves the standardization problem by performing a systematic comparison of all pairs of factor loadings across groups. A numerical example based upon a large published data set is presented to illustrate the utility of the new procedure, particularly with regard to partial factorial invariance.
Uploads
Papers by Roger Rensvold