The current medical practice relies on evidence to produce a value-based healthcare environment [1,2,3]. Medical research is aimed at improving clinical practice by producing more accurate diagnosis and safer and more efficient treatments. Results are published through peer-reviewed journals, which also serve to filter the most meaningful ideas and guarantee a certain level of scientific methodology. The impact a research publication has on the scientific community is a measure of success which may be quantified by the number of citations. In addition, there is also academic pressure to publish, with many scholars being evaluated based on the number and impact of their research publications [4,5,6].

The Clarivate analytics Web of Science (WoS) is regarded as the best and most prestigious journal ranking system and scientific publication citation aggregator. The global publication output is growing year by year. With respect to orthopedics, the United States of America (USA) has maintained its dominance, but the percentage is decreasing. Countries such as China and South Korea are increasing production at a steady pace [7,8,9]. However, there is limited information regarding orthopedic publication outputs from countries in eastern Europe [7,8,10,11].

We therefore aimed to analyze the publication output from Romania under the WoS category of orthopedics.

In the WoS Core Collection Advanced Search we used field tags, Boolean operators and parentheses to create the following queries:

(SU = Orthopedics OR WC = Orthopedics) AND CU = Romania;

These were first limited to English language, document type article, custom year range 2009–2018 and SCIE—Science Citation Index Expanded, and then with all document types and all WoS Core Collection citation indexes (including Emerging Sources Citation Index—ESCI) (where SU = Research Area, WC = Web of Science Category, CU = Country/Region).

Separate searches were also performed for: (SU = Orthopedics OR WC = Orthopedics); CU = Romania; CU = USA; limited to English language, document type—article, and SCI-Expanded index timespan 2009–2018 and all years (1975–2008).

According to WoS, research areas are a subject categorization system common to all WoS databases. In addition, journals covered by the WoS Core Collection are assigned to at least one category and each WoS category is mapped to one research area [12]. Highly Cited in Field (HCF) means that the article has received enough citations to place it in the top 1% of the academic field of Clinical Medicine, based on a highly cited threshold for the field and publication year [12].

Descriptive statistics were used to assess differences against main comparators: WoS orthopedics category output USA, WoS orthopedics category output world, and WoS Romania total research output. To determine journal impact factor (IF) quartile, we used the Journal Citation Reports for 2018 (76), although in 2009 this only contained 56 titles [13]. Ethics approval was not applicable for this type of research.

This is the first critical analysis of orthopedic publication output from Romania based on WoS category and indexation. Although the total number of articles was low compared to world category and national level, over the last five years, the number of publications per year doubled, a much greater increase than all comparators. The distribution by document types was similar, with a preponderance of original research. Compared to the United States, in the top five WoS categories/research areas, ‘Sports sciences’ were less represented. At the national level, Victor Babes University of Medicine and Pharmacy from Timisoara shared the top place with Carol Davila University of Medicine and Pharmacy from Bucharest, an above-average result based on national medical university rankings [12]. International Orthopedics was by far the leading journal for Romanian authors, publishing 30.5% of titles. Open access notably increased when all document types and all WoS core collection citation indexes were included. The top five articles by number of citations were the same in both search modes, and ranged from 30 to 91. There were no publications included in the HCF category. At the national level, all top five HCF WoS categories/research areas were from clinical medicine, even though the main outputs from Romania were from the fields of chemistry, physics, engineering and mathematics. Both national and the top orthopedic publication collaborations were with countries from western Europe, England and the United States.

A bibliometric analysis of Romania’s research output between 2005 and 2014 was commissioned by the Executive Unit for Financing Higher Education, Research, Development and Innovation (UEFISCDI) in partnership with Thompson Reuters [Lau]. This allowed full data access including to Thomson Reuters InCites: Essential Science Indicators field of ‘Clinical medicine’. The top five categories by number of publications were Surgery, Gastroenterology & Hepatology, Medicine, Research & Experimental and Endocrinology & Metabolism; however, only Oncology ranked much higher than the world domain average in terms of citations. A UEFISCDI report chose the Czech Republic, Poland, Slovenia, Turkey and Hungary as direct comparators. On an aggregate level, Turkey published the most research papers, but Romania had the greatest increase in publication output. Citation trends were similar and mostly below the world average citation impact, and only in 2014 had Romania’s normalized citation impact climbed above that threshold [10].

The IF system is a historic authority on research impact and measure of prestige in the field. Some of its criticism has been that it includes self-citations, it favors review articles, there is wide disparity among disciplines, it is not a measure of each article but a cumulative journal effort, and it makes no distinction regarding the quality of citing sources. In top-ranking journals, randomized controlled trials and metanalyses receive more citations overall and are more likely to be cited outside the field, in the general medical literature [14]. Altmetric scores had a significant positive association with the citation rates of articles in five high-impact orthopedic journals [15]. Citations in the WoS system were the most restrictive. Scopus and Google Scholar had higher citation counts than WoS, and the difference was larger between Google Scholar and WoS [16].

The IF of orthopedic journals falls well behind other specialties with broader addressability, such as internal medicine [17]. By using only the WoS category, we did not include orthopedic publications and publications authored by orthopedic surgeons in other journals [18]. This is particularly important because the pressure to publish may push academic orthopedics to look for open access, multidisciplinary mega-journals instead of the classic orthopedic titles. Additionally, the effects may be multiple: faster publication, easier acceptance, higher impact factor and even increased citations. Mavrogenis et al. explored the validity of assessing orthopedic surgeons by their number of publications. Practicing clinicians from the academic sector are often evaluated solely by their research performance. Future reforms should try to also include surgical proficiency in the ranking of orthopedists [19].

Ultimately, the goal of orthopedic research should be to increase translation into practice. To achieve this target, the orthopedic literature has to overcome several hurdles. The clinician should be able to understand the technical soundness of a paper [20]. On the other hand, researchers should avoid academic misconduct and fraud [21]. Everyone in the health care system has to limit potential conflicts of interest [22]. An analysis found that the majority of the recommended readings for residency training curricula stem from higher impact general orthopedic and major subspecialty journals, albeit with a preponderance of low level of evidence (Level IV) research [23]. Independent and peer-reviewed sources of information are preferred by surgeons when choosing between treatments and implants. However, there is also bias stemming from extrinsic factors such as investigator reputation and perceived journal quality [24].

Orthopedic publications from Romania have increased in the last decade, but are still low compared to category averages; however, the bibliometric qualitative distribution and patterns mostly resemble those of comparators.