The overall aim was to examine the equated values when using different linkage plans and differen... more The overall aim was to examine the equated values when using different linkage plans and different observed-score equipercentile equating methods with the equivalent groups (EG) design and the nonequivalent groups with anchor test (NEAT) design. Both real data from a college admissions test and simulated data were used with frequency estimation, chained equating, and kernel equating methods. The overall results were that different linkage plans gave different equated values and standard errors (SEs) both in the EG design and in the NEAT design, and there were some differences between the equating methods. The simulation study confirmed the empirical results and suggested that the kernel equating methods gave lower SEs in the examined conditions and had fewer differences that mattered compared with traditional equating methods when different linking plans were used. Finally, when one of the test forms was easier or if the ability of the test groups differed, the choice of linkage pla...
Springer Proceedings in Mathematics & Statistics, 2021
Mixed-format tests contain items with different formats such as dichotomously scored and polytomo... more Mixed-format tests contain items with different formats such as dichotomously scored and polytomously scored items. The aim of this study was to examine the impact of item discrimination, sample size, and proportion of polytomously scored items on item response theory (IRT) kernel equating of mixed-format tests under the equivalent groups design. A simulation study was performed to examine the aim. The results from the simulation study showed that the percent relative errors were low and stable for all conditions, whereas differences in standard errors and equated values where found for the conditions with different sample sizes and item discriminations. Also, the standard errors were higher when the proportion of polytomously scored items in the test where higher.
Nonparametric procedures are used to add flexibility to models. Three nonparametric item response... more Nonparametric procedures are used to add flexibility to models. Three nonparametric item response models have been proposed, but not directly compared: the Kernel smoothing (KS-IRT); the Davidian-Curve (DC-IRT); and the Bayesian semiparametric Rasch model (SP-Rasch). The main aim of the present study is to compare the performance of these procedures in recovering simulated true scores, using sum scores as benchmarks. The secondary aim is to compare their performances in terms of practical equivalence with real data. Overall, the results show that, apart from the DC-IRT, which is the model that performs the worse, all the other models give results quite similar to those when sum scores are used. These results are followed by a discussion with practical implications and recommendations for future studies.
Springer Proceedings in Mathematics & Statistics, 2019
This proceedings volume highlights the latest research and developments in psychometrics and stat... more This proceedings volume highlights the latest research and developments in psychometrics and statistics. It compiles and expands on selected and peer-reviewed presentations given at the 83rd Annual International Meeting of the Psychometric Society (IMPS), organized by Columbia University and held in New York, USA, July 9th to 13th, 2018. The IMPS is one of the largest international meetings on quantitative measurement in education, psychology, and the social sciences. The last couple of years it has attracted more than 500 participants and more than 250 paper presentations from researchers around the world. Leading experts and promising young researchers from around the world have written the 38 chapters. The chapters address a large variety of topics including but not limited to item response theory, multistage adaptive testing, and cognitive diagnostic models. This volume is the 7th in a series of recent volumes to cover research presented at the IMPS
Springer Proceedings in Mathematics & Statistics, 2021
Equating test scores from nonequivalent groups requires to separate the effect of differences in ... more Equating test scores from nonequivalent groups requires to separate the effect of differences in ability from differences in test form difficulty between the test forms. The gold standard for this purpose is to use common items, however, not all testing programs have such items available. There might be other useful information available, and this paper studies the use of covariates in equating as a substitute for common items. To gain further insight about how to best incorporate covariates in equating this paper studies the propensity score, a scalar function of the covariates. The sensitivity of the propensity score as a proxy for ability is studied to investigate if bias is introduced in the estimated equated scores if the propensity score estimation model is misspecified. By considering an incorrect link function, leaving out a covariate and leaving out a second-order term from the estimation model, common misspecifications are studied. The results show that equating with respect to the propensity score is relatively robust to model misspecification, especially modeling nonlinearity with a linear model and leaving out second-order terms.
Computerized tests have typically been analysed with either classical test theory or item respons... more Computerized tests have typically been analysed with either classical test theory or item response theory (IRT). The probability of answering an item correctly has typically been modelled with IRT. There are however at least two problems with using IRT to model test items in achievement tests. First, not all items have the properties required for using parametric modelling and does it might be problematic to obtain a good fit to an IRT model. Second, the probability of answering an item correctly in IRT is modelled over an infinite defined ability interval. This arbitrary defined ability might be difficult to explain to test takers which are unfamiliar with statistics and test theory. An advantage, with using the examined nonparametric estimation when modelling test items, is that test takers’ ability is defined over the same distribution as the sum scores. The aim is to present the advantage of using nonparametric estimation of item characteristic curves by comparing this approach ...
When using kernel equating to equate two test forms, a bandwidth needs to be selected. The bandwi... more When using kernel equating to equate two test forms, a bandwidth needs to be selected. The bandwidth parameter determines the smoothness of the continuized score distributions and has been shown to have a large effect on the kernel density estimate. There are a number of suggested criteria for selecting the bandwidth, and currently four of them have been implemented in kernel equating. In this paper, all four of the existing bandwidth selectors suggested for kernel equating are evaluated and compared against each other using real test data together with a new criterion that implements leave-one-out cross-validation. Although the bandwidth methods generally were similar in terms of equated scores, there were potentially important differences in the upper part of the score scale where critical admission decisions are typically made.
APPENDIX Development and construct validation of the Swedish Zimbardo Time Perspective Inventory ... more APPENDIX Development and construct validation of the Swedish Zimbardo Time Perspective Inventory (S-ZTPI) in European Journal of Psychological Assessment, 2011, 27(4), 220-227. Abstract. In this study, we developed and evaluated a Swedish version of the Zimbardo Time Perspective Inventory (ZTPI; Zimbardo & Boyd, 1999). The original version of the ZTPI was extended by including a Future Negative scale, and the psychometric properties of both versions were examined in a sample of 419 adults aged between 18 and 80 years. Confirmatory factor analysis (CFA) provided support both for the original five-factor solution proposed by Zimbardo and Boyd (1999) in a Swedish sample and for a six-factor solution with the Future Negative scale as an independent factor. These findings extend the original ZTPI and suggest that negative feelings about the future constitute a central dimension of the temporal perspective. The Swedish Zimbardo Time Perspective Inventory (S-ZTPI) provides a reliable and v...
Abstract. In this study, we developed and evaluated a Swedish version of the Zimbardo Time Perspe... more Abstract. In this study, we developed and evaluated a Swedish version of the Zimbardo Time Perspective Inventory (ZTPI; Zimbardo & Boyd, 1999). The original version of the ZTPI was extended by including a Future Negative scale, and the psychometric properties of both versions were examined in a sample of 419 adults aged between 18 and 80 years. Confirmatory factor analysis (CFA) provided support both for the original five-factor solution proposed by Zimbardo and Boyd (1999) in a Swedish sample and for a six-factor solution with the Future Negative scale as an independent factor. These findings extend the original ZTPI and suggest that negative feelings about the future constitute a central dimension of the temporal perspective. The Swedish Zimbardo Time Perspective Inventory (S-ZTPI) provides a reliable and valid instrument for measuring time perspective in the context of Swedish research and to be beneficial in its application in multiple areas of psychology and related disciplines...
Springer Proceedings in Mathematics & Statistics
Test equating methods are widely used in order to make comparable different test forms administer... more Test equating methods are widely used in order to make comparable different test forms administered at different occasions to different test takers. Although software for test equating is currently available, in this paper we focus the attention on four different R packages which can facilitate test equating for researchers and test developers. This paper list the different R packages which are available at the moment. Examples are provided for the equate, equateIRT, kequate, and the SNSequate packages. Additional features of these packages are discussed as well.
Reduction of graphs is a class of procedures used to decrease the dimensionality of a given graph... more Reduction of graphs is a class of procedures used to decrease the dimensionality of a given graph in which the properties of the reduced graph are to be induced from the properties of the larger original graph. This paper introduces both a new method for reducing chain graphs to simpler directed acyclic graphs (DAGs), that we call power chain graphs (PCG), as well as a procedure for structure learning of this new type of graph from correlational data of a Gaussian Graphical model (GGM). A definition for PCGs is given, directly followed by the reduction method. The structure learning procedure is a two-step approach: first, the correlation matrix is used to cluster the variables; and then, the averaged correlation matrix is used to discover the DAGs using the PC-stable algorithm. The results of simulations are provided to illustrate the theoretical proposal, which demonstrate initial evidence for the validity of our procedure to recover the structure of power chain graphs. The paper ends with a discussion regarding suggestions for future studies as well as some practical implications.
In a Nordic perspective, the finish students overall achieve the highest score on PISA (Programme... more In a Nordic perspective, the finish students overall achieve the highest score on PISA (Programme for International Student Assessment), while the Swedish students exhibit declining results. The results of the Swedish students have drawn attention to the quality of education and the role of the educational professionals and the efficiency of the school. It is therefore of vital importance to investigate whether these results can be related to school level factors in a Nordic perspective. However, TIMSS (Trends in International Mathematics and Science Study) and PISA exhibit similarities as well as differences as they target different subjects. In addition, the results on TIMSS and PISA differ between countries. The aim of this study is to investigate whether school level factors can contribute to the explanation of the results for the Nordic countries participating in PISA 2009 and, if so, identify factors that can be influenced in order to enhance students ’ achievement. We focus o...
In the kernel method of test score equating, the first step of the procedure is to presmooth the ... more In the kernel method of test score equating, the first step of the procedure is to presmooth the score distributions. The most common way of doing so is by fitting a log-linear model to the observed-score distributions. In this way, irregularities in the score distributions are smoothed, yielding a more stable estimated equating transformation. Within the kernel equating framework, an alternative way of presmoothing by using item response theory models has recently been suggested. There are furthermore several model selection criteria available for both of these classes of models. Here the model selection criteria are studied for both log-linear and item response theory models. Specifically, the likelihood ratio, AIC, and BIC measures are compared using real admissions data. Results show that the different model selection criteria result in equated scores that have real impact differences.
The aim of this paper is to describe, analyze, and discuss how Swedish schools and the national t... more The aim of this paper is to describe, analyze, and discuss how Swedish schools and the national tests in schools, university teaching and examination, and the college admissions test, Swedish Scholastic Aptitude Test (SweSAT), have been affected by the COVID-19 situation. A further aim is to discuss the challenges in schools, universities and in the admissions test process in Sweden which are due to the COVID-19 situation. Contrary to many other countries, Swedish schools remained open, except for upper secondary school and universities where teaching went online. However, the spring administrations of the national tests and the high-stake college admission test, SweSAT, were cancelled, which had impact on admissions to universities in the fall. By using documentation from the news, school, and university authorities, as well as governmental reports of the events and a student survey, challenges are discussed. The novelty of this study includes a discussion of the events and their u...
The overall aim was to examine the equated values when using different linkage plans and differen... more The overall aim was to examine the equated values when using different linkage plans and different observed-score equipercentile equating methods with the equivalent groups (EG) design and the nonequivalent groups with anchor test (NEAT) design. Both real data from a college admissions test and simulated data were used with frequency estimation, chained equating, and kernel equating methods. The overall results were that different linkage plans gave different equated values and standard errors (SEs) both in the EG design and in the NEAT design, and there were some differences between the equating methods. The simulation study confirmed the empirical results and suggested that the kernel equating methods gave lower SEs in the examined conditions and had fewer differences that mattered compared with traditional equating methods when different linking plans were used. Finally, when one of the test forms was easier or if the ability of the test groups differed, the choice of linkage pla...
Springer Proceedings in Mathematics & Statistics, 2021
Mixed-format tests contain items with different formats such as dichotomously scored and polytomo... more Mixed-format tests contain items with different formats such as dichotomously scored and polytomously scored items. The aim of this study was to examine the impact of item discrimination, sample size, and proportion of polytomously scored items on item response theory (IRT) kernel equating of mixed-format tests under the equivalent groups design. A simulation study was performed to examine the aim. The results from the simulation study showed that the percent relative errors were low and stable for all conditions, whereas differences in standard errors and equated values where found for the conditions with different sample sizes and item discriminations. Also, the standard errors were higher when the proportion of polytomously scored items in the test where higher.
Nonparametric procedures are used to add flexibility to models. Three nonparametric item response... more Nonparametric procedures are used to add flexibility to models. Three nonparametric item response models have been proposed, but not directly compared: the Kernel smoothing (KS-IRT); the Davidian-Curve (DC-IRT); and the Bayesian semiparametric Rasch model (SP-Rasch). The main aim of the present study is to compare the performance of these procedures in recovering simulated true scores, using sum scores as benchmarks. The secondary aim is to compare their performances in terms of practical equivalence with real data. Overall, the results show that, apart from the DC-IRT, which is the model that performs the worse, all the other models give results quite similar to those when sum scores are used. These results are followed by a discussion with practical implications and recommendations for future studies.
Springer Proceedings in Mathematics & Statistics, 2019
This proceedings volume highlights the latest research and developments in psychometrics and stat... more This proceedings volume highlights the latest research and developments in psychometrics and statistics. It compiles and expands on selected and peer-reviewed presentations given at the 83rd Annual International Meeting of the Psychometric Society (IMPS), organized by Columbia University and held in New York, USA, July 9th to 13th, 2018. The IMPS is one of the largest international meetings on quantitative measurement in education, psychology, and the social sciences. The last couple of years it has attracted more than 500 participants and more than 250 paper presentations from researchers around the world. Leading experts and promising young researchers from around the world have written the 38 chapters. The chapters address a large variety of topics including but not limited to item response theory, multistage adaptive testing, and cognitive diagnostic models. This volume is the 7th in a series of recent volumes to cover research presented at the IMPS
Springer Proceedings in Mathematics & Statistics, 2021
Equating test scores from nonequivalent groups requires to separate the effect of differences in ... more Equating test scores from nonequivalent groups requires to separate the effect of differences in ability from differences in test form difficulty between the test forms. The gold standard for this purpose is to use common items, however, not all testing programs have such items available. There might be other useful information available, and this paper studies the use of covariates in equating as a substitute for common items. To gain further insight about how to best incorporate covariates in equating this paper studies the propensity score, a scalar function of the covariates. The sensitivity of the propensity score as a proxy for ability is studied to investigate if bias is introduced in the estimated equated scores if the propensity score estimation model is misspecified. By considering an incorrect link function, leaving out a covariate and leaving out a second-order term from the estimation model, common misspecifications are studied. The results show that equating with respect to the propensity score is relatively robust to model misspecification, especially modeling nonlinearity with a linear model and leaving out second-order terms.
Computerized tests have typically been analysed with either classical test theory or item respons... more Computerized tests have typically been analysed with either classical test theory or item response theory (IRT). The probability of answering an item correctly has typically been modelled with IRT. There are however at least two problems with using IRT to model test items in achievement tests. First, not all items have the properties required for using parametric modelling and does it might be problematic to obtain a good fit to an IRT model. Second, the probability of answering an item correctly in IRT is modelled over an infinite defined ability interval. This arbitrary defined ability might be difficult to explain to test takers which are unfamiliar with statistics and test theory. An advantage, with using the examined nonparametric estimation when modelling test items, is that test takers’ ability is defined over the same distribution as the sum scores. The aim is to present the advantage of using nonparametric estimation of item characteristic curves by comparing this approach ...
When using kernel equating to equate two test forms, a bandwidth needs to be selected. The bandwi... more When using kernel equating to equate two test forms, a bandwidth needs to be selected. The bandwidth parameter determines the smoothness of the continuized score distributions and has been shown to have a large effect on the kernel density estimate. There are a number of suggested criteria for selecting the bandwidth, and currently four of them have been implemented in kernel equating. In this paper, all four of the existing bandwidth selectors suggested for kernel equating are evaluated and compared against each other using real test data together with a new criterion that implements leave-one-out cross-validation. Although the bandwidth methods generally were similar in terms of equated scores, there were potentially important differences in the upper part of the score scale where critical admission decisions are typically made.
APPENDIX Development and construct validation of the Swedish Zimbardo Time Perspective Inventory ... more APPENDIX Development and construct validation of the Swedish Zimbardo Time Perspective Inventory (S-ZTPI) in European Journal of Psychological Assessment, 2011, 27(4), 220-227. Abstract. In this study, we developed and evaluated a Swedish version of the Zimbardo Time Perspective Inventory (ZTPI; Zimbardo & Boyd, 1999). The original version of the ZTPI was extended by including a Future Negative scale, and the psychometric properties of both versions were examined in a sample of 419 adults aged between 18 and 80 years. Confirmatory factor analysis (CFA) provided support both for the original five-factor solution proposed by Zimbardo and Boyd (1999) in a Swedish sample and for a six-factor solution with the Future Negative scale as an independent factor. These findings extend the original ZTPI and suggest that negative feelings about the future constitute a central dimension of the temporal perspective. The Swedish Zimbardo Time Perspective Inventory (S-ZTPI) provides a reliable and v...
Abstract. In this study, we developed and evaluated a Swedish version of the Zimbardo Time Perspe... more Abstract. In this study, we developed and evaluated a Swedish version of the Zimbardo Time Perspective Inventory (ZTPI; Zimbardo & Boyd, 1999). The original version of the ZTPI was extended by including a Future Negative scale, and the psychometric properties of both versions were examined in a sample of 419 adults aged between 18 and 80 years. Confirmatory factor analysis (CFA) provided support both for the original five-factor solution proposed by Zimbardo and Boyd (1999) in a Swedish sample and for a six-factor solution with the Future Negative scale as an independent factor. These findings extend the original ZTPI and suggest that negative feelings about the future constitute a central dimension of the temporal perspective. The Swedish Zimbardo Time Perspective Inventory (S-ZTPI) provides a reliable and valid instrument for measuring time perspective in the context of Swedish research and to be beneficial in its application in multiple areas of psychology and related disciplines...
Springer Proceedings in Mathematics & Statistics
Test equating methods are widely used in order to make comparable different test forms administer... more Test equating methods are widely used in order to make comparable different test forms administered at different occasions to different test takers. Although software for test equating is currently available, in this paper we focus the attention on four different R packages which can facilitate test equating for researchers and test developers. This paper list the different R packages which are available at the moment. Examples are provided for the equate, equateIRT, kequate, and the SNSequate packages. Additional features of these packages are discussed as well.
Reduction of graphs is a class of procedures used to decrease the dimensionality of a given graph... more Reduction of graphs is a class of procedures used to decrease the dimensionality of a given graph in which the properties of the reduced graph are to be induced from the properties of the larger original graph. This paper introduces both a new method for reducing chain graphs to simpler directed acyclic graphs (DAGs), that we call power chain graphs (PCG), as well as a procedure for structure learning of this new type of graph from correlational data of a Gaussian Graphical model (GGM). A definition for PCGs is given, directly followed by the reduction method. The structure learning procedure is a two-step approach: first, the correlation matrix is used to cluster the variables; and then, the averaged correlation matrix is used to discover the DAGs using the PC-stable algorithm. The results of simulations are provided to illustrate the theoretical proposal, which demonstrate initial evidence for the validity of our procedure to recover the structure of power chain graphs. The paper ends with a discussion regarding suggestions for future studies as well as some practical implications.
In a Nordic perspective, the finish students overall achieve the highest score on PISA (Programme... more In a Nordic perspective, the finish students overall achieve the highest score on PISA (Programme for International Student Assessment), while the Swedish students exhibit declining results. The results of the Swedish students have drawn attention to the quality of education and the role of the educational professionals and the efficiency of the school. It is therefore of vital importance to investigate whether these results can be related to school level factors in a Nordic perspective. However, TIMSS (Trends in International Mathematics and Science Study) and PISA exhibit similarities as well as differences as they target different subjects. In addition, the results on TIMSS and PISA differ between countries. The aim of this study is to investigate whether school level factors can contribute to the explanation of the results for the Nordic countries participating in PISA 2009 and, if so, identify factors that can be influenced in order to enhance students ’ achievement. We focus o...
In the kernel method of test score equating, the first step of the procedure is to presmooth the ... more In the kernel method of test score equating, the first step of the procedure is to presmooth the score distributions. The most common way of doing so is by fitting a log-linear model to the observed-score distributions. In this way, irregularities in the score distributions are smoothed, yielding a more stable estimated equating transformation. Within the kernel equating framework, an alternative way of presmoothing by using item response theory models has recently been suggested. There are furthermore several model selection criteria available for both of these classes of models. Here the model selection criteria are studied for both log-linear and item response theory models. Specifically, the likelihood ratio, AIC, and BIC measures are compared using real admissions data. Results show that the different model selection criteria result in equated scores that have real impact differences.
The aim of this paper is to describe, analyze, and discuss how Swedish schools and the national t... more The aim of this paper is to describe, analyze, and discuss how Swedish schools and the national tests in schools, university teaching and examination, and the college admissions test, Swedish Scholastic Aptitude Test (SweSAT), have been affected by the COVID-19 situation. A further aim is to discuss the challenges in schools, universities and in the admissions test process in Sweden which are due to the COVID-19 situation. Contrary to many other countries, Swedish schools remained open, except for upper secondary school and universities where teaching went online. However, the spring administrations of the national tests and the high-stake college admission test, SweSAT, were cancelled, which had impact on admissions to universities in the fall. By using documentation from the news, school, and university authorities, as well as governmental reports of the events and a student survey, challenges are discussed. The novelty of this study includes a discussion of the events and their u...
Uploads
Papers by Marie Wiberg