Journal Impact Factor Suppression
A Critical Examination of Academic Censorship and Evaluation Metrics
Executive Summary: This page examines the controversial practice of Journal Impact Factor suppression, where academic institutions or organizations actively limit access to or discussion of Impact Factor data. We analyze the motivations behind such suppression, its consequences for academic discourse, and explore alternative approaches to scholarly evaluation.
Understanding Journal Impact Factor
The Journal Impact Factor (JIF), developed by Eugene Garfield and first published in the 1960s, has become one of the most contentious metrics in scientific publishing. Calculated annually by Clarivate Analytics (formerly Thomson Reuters), JIF represents the average number of citations to recent articles published in a journal. It has evolved from a bibliometric tool for librarians into a dominant metric for evaluating both journals and researchers.
JIF is calculated by dividing the number of citations in the current year to items published in the previous two years by the number of items published in those two years. For example, a journal with a 2020 Impact Factor of 3.5 means that on average, papers published in 2018 and 2019 received 3.5 citations in 2020.
What is Journal Impact Factor Suppression?
Journal Impact Factor suppression refers to the deliberate restriction of access to, discussion of, or consideration of Impact Factor data within academic contexts. This can take several forms:
Forms of Suppression:
- Structural suppression: University systems or policies that explicitly prohibit the use of Impact Factors in evaluations
- Financial suppression: Budget decisions that prevent departments from acquiring access to JCR (Journal Citation Reports) data
- Normative suppression: Departmental cultures that stigmatize discussion or consideration of Impact Factors
- Methodological suppression: Mandating alternative evaluation approaches that effectively exclude JIF consideration
In some cases, these measures are part of thoughtful initiatives to reduce metric misuse. In others, they may represent restrictions on academic freedom or transparency in evaluation processes.
Motivations Behind Impact Factor Suppression
Criticisms of the Impact Factor:
The push toward Impact Factor suppression has emerged in response to well-documented flaws and misapplications of the metric:
- Inapplicability to individuals: Impact Factors measure journal performance, not individual researcher contribution, yet they are frequently misused to evaluate scientists
- Discipline bias: Citation practices vary dramatically across fields, with mathematics and engineering typically showing lower average Impact Factors than biomedical sciences
- Time lag: The two-year citation window disadvantages fields with slower knowledge development and citation accumulation
- Manipulation potential: Some journals engage in "citation stacking," excessive self-citation, or other practices to artificially boost Impact Factors
- Narrow scope: Impact Factors capture only citation impact, ignoring societal impact, educational value, or methodological contributions
Case Study: The San Francisco Declaration on Research Assessment (DORA)
Launched in 2012 at the Annual Meeting of the American Society for Cell Biology, DORA has been signed by over 15,000 individuals and organizations worldwide. It explicitly recommends eliminating journal Impact Factors as a primary criterion in evaluating researchers' work. However, DORA calls for better assessment, not necessarily complete suppression of Impact Factor information, advocating instead for more considered application of metrics.
Institutional Approaches to Impact Factor Suppression
Universities worldwide have adopted various strategies addressing Impact Factor usage in faculty evaluation, promotion, and hiring decisions:
- Impact Factor-free evaluations: Institutions like Utrecht University have adopted evaluation frameworks that explicitly prohibit mentioning Impact Factors
- Narrative-focused assessment: Medical schools like those implementing the UK's REF framework emphasize structured narrative assessments over numerical metrics
- Diversified metrics: Universities like Harvard Medical School advocate consideration of multiple indicators including alternative metrics (altmetrics) and qualitative assessment
- Transparency mandates: Some institutions require justification when any metric is used, preventing unthinking application of Impact Factors
These approaches range from suppression to transparency initiatives, reflecting different philosophies about academic freedom, accountability, and the role of metrics in research evaluation.
The Controversy: Suppression vs. Reform
The academic community remains divided on the appropriate approach to addressing Impact Factor misuse:
Arguments for Suppression:
- Complete prohibition prevents backsliding into metric-dependent evaluations
- Mandatory alternatives force development of more thoughtful assessment approaches
- Removing Impact Factor consideration reduces perverse incentives to publish in specific journals regardless of field-appropriateness
- Suppression prevents the "halo effect" where journal standing influences perception of individual work quality
Arguments Against Suppression:
- Restricting access to information limits academic freedom and transparency
- Impact Factors remain valuable for certain purposes like library collection development
- Suppression doesn't address underlying evaluation challenges but simply removes one tool
- Complete prohibition may drive consideration of Impact Factors underground rather than promoting transparency
Alternative Metrics and Evaluation Approaches
As institutions move away from Impact Factors, several alternatives have gained prominence:
Alternative Metrics:
- h-index: A researcher-level metric incorporating both productivity and citation impact
- Altmetrics: Measures capturing broader impact through mentions in policy documents, media coverage, social media shares, and download statistics
- Field-normalized metrics: Citation indicators that account for discipline-specific citation patterns
- Narrative CVs: Qualitative descriptions of research contributions rather than quantified metrics
Emerging Approaches:
- Peer Review-based assessment: Emphasizing comprehensive peer review of portfolio submissions
- Contribution-focused evaluation: Detailed assessment of individuals' specific roles in research projects
- Open science contributions: Valuing data sharing, protocol registration, and replication studies
- Adoption and implementation: Evaluating research based on its translation into practice, policy, or further research
Consequences of Impact Factor Suppression
Evaluating the effects of Impact Factor suppression remains challenging, but evidence suggests several outcomes:
- Shifting publication patterns: Some evidence suggests faculty in suppression environments publish more in field-specific journals rather than general-interest high-Impact Factor venues
- Reduced anxiety: Many early-career researchers report less pressure when Impact Factors are not emphasized
- Potential bias introduction: Subjective evaluation methods may introduce new forms of bias when replacing metrics
- Increased assessment burden: Narrative and qualitative evaluation approaches require significantly more committee time and expertise
- Career portability concerns: Researchers sometimes worry about how Impact Factor-free evaluations translate when moving to metric-heavy institutions
Future Directions and Recommendations
The conversation around Impact Factor suppression continues to evolve. Several potential future directions bear consideration:
Emerging Solutions:
- Responsible metrics movement: Emphasizing transparency, justification, and appropriate application of any metrics
- Composite indicators: Developing multi-dimensional approaches that acknowledge various aspects of research value
- Contextual assessment: Creating evaluation frameworks that consider discipline-specific publication landscapes
- Open evaluation systems: Developing transparent processes where both metrics and qualitative judgments are openly documented
Practical Recommendations:
- Institutions considering Impact Factor restrictions should implement comprehensive faculty development on alternative assessment approaches
- Evaluation committees should receive training in mitigating bias when moving to more qualitative assessment approaches
- Departments should consider pilot programs testing new evaluation frameworks before institutional-wide implementation
- Regular assessment of the effects of evaluation policy changes should be conducted and shared with the broader academic community
Conclusion
Journal Impact Factor suppression represents both a reaction to metric misuse and a movement toward more thoughtful approaches to research evaluation. While the Impact Factor has clear limitations and is frequently misapplied, whether complete suppression represents the optimal approach remains contested. The most promising solutions may lie not in eliminating access to information but in creating more sophisticated, transparent, and context-appropriate evaluation frameworks that acknowledge both the benefits and limitations of quantitative metrics.
As higher education continues this important conversation, maintaining academic freedom while ensuring fair and meaningful evaluation of scholarly work remains the central challenge. The path forward likely requires balancing appropriate skepticism of metrics with recognition that fully qualitative approaches have their own limitations and subjectivities.
Selected References
- Garfield, E. (1955). Citation indexes for science. Science, 122(3159), 108-111.
- San Francisco Declaration on Research Assessment (DORA). (2012). Available at: https://sfdora.org/
- Wouters, P., et al. (2019). The metric tide: bibliometrics and the evolution of journal impact factor. Journal of Documentation, 75(4), 823-839.
- Moed, H. F. (2017). Applied evaluative informetrics. Springer.
- Haunschild, R., Schier, H., & Bornmann, L. (2016). Proposal for a standardized format of the citation statement in references (CRIS) with journal impact factors stored on the crossref database. Publications, 4(3), 28.
- Hicks, D., et al. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429-431.
- Brembs, B., et al. (2018). The impact factor curse and its benign alternatives. F1000Research, 7, 878.
2021 Journal Impact Factor Without Self Citations and Reference File Download Link
Journal Impact Factor And Quartile Ranking and Reference File Download Link