This article reports on a NISO Plus 2022 session that addressed what can be done to safeguard the... more This article reports on a NISO Plus 2022 session that addressed what can be done to safeguard the integrity of the scholarly content being created, disseminated, and used. How much responsibility does the information community have in ensuring that the content we provide is authoritative? Preprints are a great way to make early research results available, but it is not always clear that those results are not yet thoroughly vetted. Peer review-a key element of scholarly publication-can help, but is far from foolproof. Retractions are another important tool, but most retracted research is still all too readily available. What can and should we be doing to safeguard the integrity of the content being created, disseminated, and used?
No formal investigations have been conducted into the efficacy or potential influence of reviewer... more No formal investigations have been conducted into the efficacy or potential influence of reviewer recommendations on editorial decisions, and the impact of this on the expectations and behaviour of authors, reviewers and journal editors. This article addresses key questions about this critical aspect of the peer review submission process. We suggest several future steps which could be taken towards improving the review process and make it more transparent, better understood, and fairer for all parties.
The importance of nanotechnology standardization is to reach the main topic in developing standar... more The importance of nanotechnology standardization is to reach the main topic in developing standards, which is “uniformity in manufacturing and facilitating the commercialization of nano-products.” According to this goal, activities on standardization in nanotechnology have been started in Iran, where development of nanotechnology is assigned to National Nanotechnology Committee of Iran. This committee is working under direct supervision of presidency office. As written in the committee’s official website [http://www.nano.ir], one of the long term goals of this movement, according to the “fourth development program of Iran” is to reach appropriate share of world trade based on nanotechnology. For this purpose standard developing and quality management system is needed for facilitating industrial and technological cooperation and decreasing costs raised from quality unawareness. So our workgroup has selected nanotechnology standardization as one of its research topics. We have studied...
Reviewers do not only help to screen manuscripts for publication in academic journals; they also ... more Reviewers do not only help to screen manuscripts for publication in academic journals; they also serve to increase the rigor and value of manuscripts by constructive feedback. However, measuring this developmental function of peer review is difficult as it requires fine-grained data on reports and journals without any optimal benchmark. To fill this gap, we adapted a recently proposed quality assessment tool and tested it on a sample of 1.3 million reports submitted to 740 Elsevier journals in 2018-2020. Results showed that the developmental standards of peer review are shared across areas of research, yet with remarkable differences. Reports submitted to social science and economics journals show the highest developmental standards. Reports from youngest reviewers, women and reviewers from Western Europe are generally more developmental than those from senior, men and reviewers working in academic institutions outside Western regions. Our findings suggest that increasing the standards of peer review at journals requires effort to assess interventions and measure practices with context-specific and multi-dimensional frameworks.
During the early months of the COVID-19 pandemic, there was an unusually high submission rate of ... more During the early months of the COVID-19 pandemic, there was an unusually high submission rate of scholarly articles. Given that most academics were forced to work from home, the competing demands for familial duties may have penalized the scientific productivity of women. To test this hypothesis, we looked at submitted manuscripts and peer review activities for all Elsevier journals between February and May 2018-2020, including data on over 5 million authors and referees. Results showed that during the first wave of the pandemic, women submitted proportionally fewer manuscripts than men. This deficit was especially pronounced among more junior cohorts of women academics. The rate of the peer-review invitation acceptance showed a less pronounced gender pattern with women taking on a greater service responsibility for journals, except for health & medicine, the field where the impact of COVID-19 research has been more prominent. Our findings suggest that the first wave of the pandemic...
During the early months of the COVID-19 pandemic, the submission rate to scholarly journals incre... more During the early months of the COVID-19 pandemic, the submission rate to scholarly journals increased abnormally (e.g., more than 90% in health & medicine Elsevier journals). Given that most academics have been forced to work from home, the competing demands for home-schooling, child and other care duties might have penalised the scientific productivity of women. To test this hypothesis, we looked at submitted manuscripts and peer review activities for all Elsevier journals between February and May 2018-2020, including data on almost 6 million academics. Results showed that women submitted proportionally fewer manuscripts than men during the COVID-19 lockdown months. This deficit was especially pronounced among women in more advanced stages of their career. The rate of the peer-review invitation acceptance showed a less pronounced gender pattern, with the exception of health & medicine, where women were also less keen to accept reviewing. Our findings indicate that the pandemic has already created cumulative advantages for men.
No evidence that women are penalized in peer review and editorial processes, finds study on 145 j... more No evidence that women are penalized in peer review and editorial processes, finds study on 145 journals in multiple disciplines.
Perception of review quality by authors and editors may play a vital role in helping to keep the ... more Perception of review quality by authors and editors may play a vital role in helping to keep the peer review process constructive. Comprehensive studies examining author and editor perceptions of reviews of manuscripts from different disciplines are rare. We assessed satisfaction of corresponding authors and opinions of editors with reviewer-generated reports and reviewers' recommendations and checked whether there was association between authors' and editors' perceptions and recommendations in 12 Elsevier journals across four disciplines. We used a modified Review Quality Instrument (RQI) to measure review quality for 809 unique manuscripts from which we accessed 1,313 reviews and recommendations, 331 authors' perception of the review's helpfulness before editor's decision, and 541 editor's opinions regarding both review timeliness and impact on decision. Authors were most satisfied with reviews that recommended acceptance compared to revision or rejection. Reviews that recommended revisions had highest quality as reflected by the RQI. Authors highly rated their satisfaction with review constructiveness from natural sciences, and editors for the same subject also highly rated timeliness and reviews' influence on publication. Editors' opinion regarding the impact of review on their publication decision and RQI were associated. Our findings suggest that more constructive reviews may better guide the editorial decision-making process.
This article examines gender bias in peer review with complete data on 145 journals in various fi... more This article examines gender bias in peer review with complete data on 145 journals in various fields of research, including about 1.7 million authors and 740,000 referees. We reconstructed three possible sources of bias, i.e., the editorial selection of referees, referee recommendations, and editorial decisions, and examined all their possible relationships. In line with previous research, we found that editors were sensitive to gender homophily in that they tended to match authors and referee by gender systematically. Results showed that in general manuscripts written by women as solo authors or co-authored by women are treated even more favorably by referees and editors. This is especially so in biomedicine and health journals, whereas women were treated relatively less favorably in social science & humanities journals, i.e., the field in which the ratio of female authors was the highest in our sample. Although with some caveat, our findings suggest that peer review and editorial...
To increase transparency in science, some scholarly journals are publishing peer review reports. ... more To increase transparency in science, some scholarly journals are publishing peer review reports. But it is unclear how this practice affects the peer review process. Here, we examine the effect of publishing peer review reports on referee behavior in five scholarly journals involved in a pilot study at Elsevier. By considering 9,220 submissions and 18,525 reviews from 2010 to 2017, we measured changes both before and during the pilot and found that publishing reports did not significantly compromise referees’ willingness to review, recommendations, or turn-around times. Younger and non-academic scholars were more willing to accept to review and provided more positive and objective recommendations. Male referees tended to write more constructive reports during the pilot. Only 8.1% of referees agreed to reveal their identity in the published report. These findings suggest that open peer review does not compromise the process, at least when referees are able to protect their anonymity.
A presentation of a taxonomy for peer review that will be used to describe the peer review proces... more A presentation of a taxonomy for peer review that will be used to describe the peer review process to authors and readers in a consistent and transparent way.
eer review is the defining feature of scholarly communication. In a 2018 survey of more than 11,0... more eer review is the defining feature of scholarly communication. In a 2018 survey of more than 11,000 researchers, 98% said that they considered peer review important or extremely important for ensuring the quality and integrity of scholarly communication 1. Indeed, now that the Internet and social media have assumed journals' original role of dissemination, a journal's main function is curation. Both the public and the scientific community trust peer review to uphold shared values of rigour, ethics, originality and analysis by improving publications and filtering out weak or errant ones. Scholarly communities rely on peer review to establish common knowledge and credit. Despite decades of calls for study, research on peer review is scarce 2. Current investigations are fragmented, with few connections and limited knowledge-sharing, as manifested by how sparsely these researchers cite each other's papers 3. The most rigorous work is generally restricted to one or a few journals per study, often in the same field. There is a lack of systematic research on how journals manage the process (such as selecting, instructing and rewarding reviewers, managing conflicting reviews, or publishing reviewers' reports); on how to define the quality and utility of individual reviews; and on how to assess peer review (such as who
We illustrate the correspondence between the quantum Interaction Picture-evolution of the state o... more We illustrate the correspondence between the quantum Interaction Picture-evolution of the state of a quantum system in Hilbert space and a combination of local and global transformations of its Wigner function in phase space. To this aim, we consider the time-evolution of a quantized harmonic oscillator driven by both a linear and a quadratic (in terms of bosonic creation and annihilation operators) potentials and employ the Magnus series to derive the exact form of the time-evolution operator. In this case, the Interaction Picture corresponds to a local transformation of phase space-reference frame into the one that is co-moving with the Wigner function.
This article reports on a NISO Plus 2022 session that addressed what can be done to safeguard the... more This article reports on a NISO Plus 2022 session that addressed what can be done to safeguard the integrity of the scholarly content being created, disseminated, and used. How much responsibility does the information community have in ensuring that the content we provide is authoritative? Preprints are a great way to make early research results available, but it is not always clear that those results are not yet thoroughly vetted. Peer review-a key element of scholarly publication-can help, but is far from foolproof. Retractions are another important tool, but most retracted research is still all too readily available. What can and should we be doing to safeguard the integrity of the content being created, disseminated, and used?
No formal investigations have been conducted into the efficacy or potential influence of reviewer... more No formal investigations have been conducted into the efficacy or potential influence of reviewer recommendations on editorial decisions, and the impact of this on the expectations and behaviour of authors, reviewers and journal editors. This article addresses key questions about this critical aspect of the peer review submission process. We suggest several future steps which could be taken towards improving the review process and make it more transparent, better understood, and fairer for all parties.
The importance of nanotechnology standardization is to reach the main topic in developing standar... more The importance of nanotechnology standardization is to reach the main topic in developing standards, which is “uniformity in manufacturing and facilitating the commercialization of nano-products.” According to this goal, activities on standardization in nanotechnology have been started in Iran, where development of nanotechnology is assigned to National Nanotechnology Committee of Iran. This committee is working under direct supervision of presidency office. As written in the committee’s official website [http://www.nano.ir], one of the long term goals of this movement, according to the “fourth development program of Iran” is to reach appropriate share of world trade based on nanotechnology. For this purpose standard developing and quality management system is needed for facilitating industrial and technological cooperation and decreasing costs raised from quality unawareness. So our workgroup has selected nanotechnology standardization as one of its research topics. We have studied...
Reviewers do not only help to screen manuscripts for publication in academic journals; they also ... more Reviewers do not only help to screen manuscripts for publication in academic journals; they also serve to increase the rigor and value of manuscripts by constructive feedback. However, measuring this developmental function of peer review is difficult as it requires fine-grained data on reports and journals without any optimal benchmark. To fill this gap, we adapted a recently proposed quality assessment tool and tested it on a sample of 1.3 million reports submitted to 740 Elsevier journals in 2018-2020. Results showed that the developmental standards of peer review are shared across areas of research, yet with remarkable differences. Reports submitted to social science and economics journals show the highest developmental standards. Reports from youngest reviewers, women and reviewers from Western Europe are generally more developmental than those from senior, men and reviewers working in academic institutions outside Western regions. Our findings suggest that increasing the standards of peer review at journals requires effort to assess interventions and measure practices with context-specific and multi-dimensional frameworks.
During the early months of the COVID-19 pandemic, there was an unusually high submission rate of ... more During the early months of the COVID-19 pandemic, there was an unusually high submission rate of scholarly articles. Given that most academics were forced to work from home, the competing demands for familial duties may have penalized the scientific productivity of women. To test this hypothesis, we looked at submitted manuscripts and peer review activities for all Elsevier journals between February and May 2018-2020, including data on over 5 million authors and referees. Results showed that during the first wave of the pandemic, women submitted proportionally fewer manuscripts than men. This deficit was especially pronounced among more junior cohorts of women academics. The rate of the peer-review invitation acceptance showed a less pronounced gender pattern with women taking on a greater service responsibility for journals, except for health & medicine, the field where the impact of COVID-19 research has been more prominent. Our findings suggest that the first wave of the pandemic...
During the early months of the COVID-19 pandemic, the submission rate to scholarly journals incre... more During the early months of the COVID-19 pandemic, the submission rate to scholarly journals increased abnormally (e.g., more than 90% in health & medicine Elsevier journals). Given that most academics have been forced to work from home, the competing demands for home-schooling, child and other care duties might have penalised the scientific productivity of women. To test this hypothesis, we looked at submitted manuscripts and peer review activities for all Elsevier journals between February and May 2018-2020, including data on almost 6 million academics. Results showed that women submitted proportionally fewer manuscripts than men during the COVID-19 lockdown months. This deficit was especially pronounced among women in more advanced stages of their career. The rate of the peer-review invitation acceptance showed a less pronounced gender pattern, with the exception of health & medicine, where women were also less keen to accept reviewing. Our findings indicate that the pandemic has already created cumulative advantages for men.
No evidence that women are penalized in peer review and editorial processes, finds study on 145 j... more No evidence that women are penalized in peer review and editorial processes, finds study on 145 journals in multiple disciplines.
Perception of review quality by authors and editors may play a vital role in helping to keep the ... more Perception of review quality by authors and editors may play a vital role in helping to keep the peer review process constructive. Comprehensive studies examining author and editor perceptions of reviews of manuscripts from different disciplines are rare. We assessed satisfaction of corresponding authors and opinions of editors with reviewer-generated reports and reviewers' recommendations and checked whether there was association between authors' and editors' perceptions and recommendations in 12 Elsevier journals across four disciplines. We used a modified Review Quality Instrument (RQI) to measure review quality for 809 unique manuscripts from which we accessed 1,313 reviews and recommendations, 331 authors' perception of the review's helpfulness before editor's decision, and 541 editor's opinions regarding both review timeliness and impact on decision. Authors were most satisfied with reviews that recommended acceptance compared to revision or rejection. Reviews that recommended revisions had highest quality as reflected by the RQI. Authors highly rated their satisfaction with review constructiveness from natural sciences, and editors for the same subject also highly rated timeliness and reviews' influence on publication. Editors' opinion regarding the impact of review on their publication decision and RQI were associated. Our findings suggest that more constructive reviews may better guide the editorial decision-making process.
This article examines gender bias in peer review with complete data on 145 journals in various fi... more This article examines gender bias in peer review with complete data on 145 journals in various fields of research, including about 1.7 million authors and 740,000 referees. We reconstructed three possible sources of bias, i.e., the editorial selection of referees, referee recommendations, and editorial decisions, and examined all their possible relationships. In line with previous research, we found that editors were sensitive to gender homophily in that they tended to match authors and referee by gender systematically. Results showed that in general manuscripts written by women as solo authors or co-authored by women are treated even more favorably by referees and editors. This is especially so in biomedicine and health journals, whereas women were treated relatively less favorably in social science & humanities journals, i.e., the field in which the ratio of female authors was the highest in our sample. Although with some caveat, our findings suggest that peer review and editorial...
To increase transparency in science, some scholarly journals are publishing peer review reports. ... more To increase transparency in science, some scholarly journals are publishing peer review reports. But it is unclear how this practice affects the peer review process. Here, we examine the effect of publishing peer review reports on referee behavior in five scholarly journals involved in a pilot study at Elsevier. By considering 9,220 submissions and 18,525 reviews from 2010 to 2017, we measured changes both before and during the pilot and found that publishing reports did not significantly compromise referees’ willingness to review, recommendations, or turn-around times. Younger and non-academic scholars were more willing to accept to review and provided more positive and objective recommendations. Male referees tended to write more constructive reports during the pilot. Only 8.1% of referees agreed to reveal their identity in the published report. These findings suggest that open peer review does not compromise the process, at least when referees are able to protect their anonymity.
A presentation of a taxonomy for peer review that will be used to describe the peer review proces... more A presentation of a taxonomy for peer review that will be used to describe the peer review process to authors and readers in a consistent and transparent way.
eer review is the defining feature of scholarly communication. In a 2018 survey of more than 11,0... more eer review is the defining feature of scholarly communication. In a 2018 survey of more than 11,000 researchers, 98% said that they considered peer review important or extremely important for ensuring the quality and integrity of scholarly communication 1. Indeed, now that the Internet and social media have assumed journals' original role of dissemination, a journal's main function is curation. Both the public and the scientific community trust peer review to uphold shared values of rigour, ethics, originality and analysis by improving publications and filtering out weak or errant ones. Scholarly communities rely on peer review to establish common knowledge and credit. Despite decades of calls for study, research on peer review is scarce 2. Current investigations are fragmented, with few connections and limited knowledge-sharing, as manifested by how sparsely these researchers cite each other's papers 3. The most rigorous work is generally restricted to one or a few journals per study, often in the same field. There is a lack of systematic research on how journals manage the process (such as selecting, instructing and rewarding reviewers, managing conflicting reviews, or publishing reviewers' reports); on how to define the quality and utility of individual reviews; and on how to assess peer review (such as who
We illustrate the correspondence between the quantum Interaction Picture-evolution of the state o... more We illustrate the correspondence between the quantum Interaction Picture-evolution of the state of a quantum system in Hilbert space and a combination of local and global transformations of its Wigner function in phase space. To this aim, we consider the time-evolution of a quantized harmonic oscillator driven by both a linear and a quadratic (in terms of bosonic creation and annihilation operators) potentials and employ the Magnus series to derive the exact form of the time-evolution operator. In this case, the Interaction Picture corresponds to a local transformation of phase space-reference frame into the one that is co-moving with the Wigner function.
Uploads
Papers by Bahar Mehmani