INTRODUCTION

With increasing emphasis on the development of evidence-based clinical decision-making,1–4 more focus is being placed on the availability of high quality evidence.5,6 The reason for this is that access to computer-based communication networks and online, critically appraised medical information can potentially improve clinical decision making by increasing information availability.7 Evidence-based clinical decision-making, however, requires the presence of, and access to, a large volume of high quality clinical information.

To identify and verify the availability of clinical orthodontic information for evidence-based clinical decision-making, we conducted a benchmarking study of MEDLINE. We employed bibliometric methods—the analysis of a literature using statistical methods to reveal the historical development of subjects and patterns of authorship, use, and publications.8 Similar methods have been successfully employed for similar purposes in medicine. Bibliometric analysis methods are currently being used by the United States National Academy of Sciences for evaluating research programs9 and have been used for evaluating clinical progress in cardiology,10 audiology,11 mental health,12 epilepsy,13 emergency medicine,14 family practice,15 allied health,16 arthritis,17 and dentistry.4

The objectives of the current study were to develop and implement MEDLINE search strategies that would access the orthodontic literature, and then to estimate the availability of literature that one can potentially use to make clinical decisions and examine trends. In particular, we were interested in the availability of articles on etiology, diagnosis, therapy, and prognosis. This is a first step toward developing computer-based clinical knowledge systems to assist in clinical decision analysis. We hypothesize that there is a substantial orthodontic literature upon which to make evidence-based decisions.

MATERIALS AND METHODS

Literature search

A comprehensive manual review of the Tree Hierarchy of MEDLINE was performed online (http://gateway.ovid.com/re1410/server2/ovidweb.cgi) to identify the Medical Subject Heading (MeSH) terms and their related subheading describing orthodontics. MeSH terms “orthodontics,” “cephalometry,” and “malocclusion” were deemed most relevant and selected to capture orthodontic journal articles. From this search a vocabulary in orthodontics was developed (Table 1). A literature search was then developed and applied using these terms (Table 2). The search strategy queried MEDLINE from year 1966 to September 1999, week 2. The search was performed using the Ovid Web Gateway (Ovid Technologies Inc, NY, NY) Internet interface for MEDLINE (http://gateway.ovid.com). Several search commands were applied to these subject headings. “Explode” (abbreviated “exp”) was applied to topics “orthodontics” and “malocclusion” for the most comprehensive search, thus all conceptually related subtopics were included in the search. “Explode” was not applied to cephalometry because doing so would retrieve irrelevant subtopics such as “craniometry.” “Focus (abbreviated “*”) was applied to “cephalometery” instead. Table 2 alphabetically lists all 3 orthodontic topics and their subtopics covered by the literature search. To limit the topics only to orthodontics, “Temporomandibular joint dysfunction syndrome” was eliminated, as it is an overlapping topic spanning many dental specialties. This was accomplished using the Boolean operator “not,” as shown in step 6 of the search. The identified literature was then limited to humans, to articles written in English, and from the year 1990 to the year 1998. Validated sensitive and specific methodologic filters were then used to identify 4 clinical categories of information: etiology, diagnosis, therapy, and prognosis (Table 3).18 As used here, a sensitive search indicates a search strategy that retrieves the largest number of relevant articles but also includes some irrelevant ones. A specific search indicates a search strategy that identifies a small number of the most relevant articles, but also excludes some relevant articles and most irrelevant articles. The results of the search were then subdivided by year from 1990 to 1998 to identify trends.

TABLE 1. Orthodontic Terms Included in MeSH Headings

View Fullscreen

TABLE 2. Orthodontic Search Strategy Based on MeSH Headings (1966 to September 1999 week 2)

View Fullscreen

TABLE 3. Methodological Filters for Searching

View Fullscreen

The search results were also stratified to identify dental journals that published the greatest number of articles per year in all 4 clinical categories during the past 5 years (1994–1998). First, for a given year, citations in each clinical category and each search type (for example, 1994 diagnosis and sensitive search) were tallied according to their journal sources. Then, for each clinical category and each search type, the mean number of articles published/year (±SD) were calculated. The journals that published these articles were then stratified from those that published most to least.

RESULTS

The results of the implemented search strategy quantify the availability of orthodontic literature. The data (Table 2) indicate that, over the period between 1966 and September 1999, week 2, there were 34,450 articles published on orthodontics. Of these, approximately 20% or 6938 articles were published between 1990 and 1998 (inclusive) in English and addressed human orthodontics.

To determine the number of articles per year and the distribution of these articles, sensitive and specific methodologic filters were applied (Table 3) to identify articles addressing etiology, diagnosis, therapy, and prognosis. Figure 1 and Table 4 present summary data for sensitive and specific searches for each of the 4 clinical categories over the 9-year period. The mean number of articles published per year in all 4 categories combined ranged from 42 (±26) for specific searches to 313 (±124) for sensitive searches. We further examined the difference between sensitive search (a strategy that retrieves the largest number of relevant articles but also includes some irrelevant ones) and specific search (a strategy that identifies a small number of the most relevant articles, but also excludes some relevant articles and most irrelevant articles). As for individual category, the number of articles per year for specific and sensitive searches ranged between 19 ± 15 and 91 ± 37 for etiology, between 11 ± 5 and 80 ± 35 for diagnosis, between 3 ± 1 and 50 ± 23 for therapy, and between 10 ± 7 and 93 ± 33 for prognosis. Within each category, sensitive searches identified more articles per year than specific searches (all P < .0002, paired t-test). Comparison between the 2 means indicates that, as expected, sensitive search consistently identified more articles than specific search (P < .0001, paired t-test).

View Figure

• Download as Powerpoint
• Download Figure

TABLE 4. Mean Number of Articles (± SD) Per Year 1990–1998^A. Percentage Indicates the Average Percent Increase of Articles Per Year

View Fullscreen

Comparing publication quantity reveals the relative emphasis placed on different aspects of clinical information in orthodontic literature. Data in Figure 1 and Table 4 were further examined to determine the relative publication quantity of 4 clinical categories. For sensitive searches, the data suggest a publication quantity of (in the decreasing order): prognosis, etiology, diagnosis, and therapy. Statistical analysis indicated that search results of prognosis, etiology, and diagnosis were greater than that of therapy, with no significant difference among the first 3 categories (all P < .001, ANOVA). For specific searches, the data suggest a publication quantity of (in decreasing order): etiology followed by diagnosis, prognosis, and therapy. Statistical analysis indicated that etiology was greater than those of prognosis and therapy (both P < .005, ANOVA), but there was no significant difference between diagnosis, prognosis, and therapy (all P < .05, ANOVA). Overall, the clinical articles on 4 categories are comparable in publication quantity, with no particular order of ranking in quantity. This finding suggests that the body of orthodontic literature is balanced in clinical content.

To better understand the dynamics of information evolution in the 4 clinical categories, the searches were stratified by publication year. Figures 2–5 illustrate the time course of diagnosis, therapy, etiology, and prognosis publications, respectively. The figures confirm the consistency with which sensitive searches identified more articles than specific searches. The figures also suggest that for all clinical categories the number of articles generally increased each year for sensitive and specific searches. Linear regression substantiates this impression in that the slopes were all positive in sensitive and specific searches respectively: 12.1 and 4.8 for etiology, 12.4 and 1.4 for diagnosis, 7.9 and 0.4 for therapy, and 10.4 and 2.4 for prognosis. These slopes were significantly different from 0 for all 4 categories in both sensitive and specific searches (all P < .02, linear regression). Spearman rank correlation of publication number with year (r2), which approximated slopes of liner regression, substantiated the impression that all 4 categories increased significantly each year for both sensitive and specific searches.

View Figure

• Download as Powerpoint
• Download Figure

From the above results, calculations were made to determine the average percent increase of articles over the 9-year period. The average percent increases by category for sensitive and specific searches were: 13% and 25% for etiology, 16% and 13% for diagnosis, 16% and 13% for therapy, and 11% and 24% for prognosis. Therefore, the composite average percent increase of articles per year (sum of slopes divided by sum of percent increase) was 14% for sensitive searches and 21% for specific searches over the past 9 years.

From a clinical viewpoint, the ultimate goal of clinical activities is to improve the prognosis of a patient's condition(s). In a body of balanced orthodontic literature, publications on prognosis would be expected to constitute (1) approximately one-fourth of all publications, or (2) increase over time as the evidence base of other clinical information grows to support more research on prognosis. To test this hypothesis, we examined the ratio of the number of prognosis articles to the total number of articles in all 4 clinical categories. This was done for the 9-year period. The mean percentage of prognosis articles ranged from 22% (±7%) for specific searches to 30% (±4%) for sensitive searches. Figure 6 illustrates that the number of prognosis articles relative to those of other categories has remained steady. These results fulfill ones expectation.

View Figure

• Download as Powerpoint
• Download Figure

To examine the source of information in the 4 clinical categories, journals that published the articles were identified. Table 5 shows the 5 journals that yielded the greatest mean number of articles per year in all 4 categories combined over the past 5 years in the respective sensitive and specific searches: American Journal of Orthodontics & Dentofacial Orthopedics (81 ± 25 and 14 ± 7), European Journal of Orthodontics (27 ± 8 and 5 ± 2), Angle Orthodontist (25 ± 7 and 4 ± 2), British Journal of Orthodontics (20 ± 3 and 4 ± 2), and International Journal of Adult Orthodontics & Orthognathic Surgery (14 ± 3 and 2 ± 3). These 5 journals totaled 167 (±33) and 30 (±13) articles per year in sensitive and specific searches respectively, which correspond to 41% (±8%) and 48% (±21%) of the mean total number of articles per year; approximately 66 other dental journals published the remainder of all articles.

TABLE 5. The 5 Journals Publishing the Most Number of Orthodontic Articles Per Year (1994–1998)

View Fullscreen

DISCUSSION

The current study was conducted to identify and quantify the availability of orthodontic literature upon which one can potentially make clinical decisions. This is a first step toward developing online knowledge systems for clinical decision analysis. The results indicated that over the last 9 years there is a significant body of clinically relevant articles published in orthodontics.

The results, while informative, have several important implications. First, there are, on average, between 42 (specific search) and 314 (sensitive search) articles published per year addressing the etiology, diagnosis, prognosis, and therapy of orthodontics. Second, if all of these publications are of high clinical applicability, these results also suggest that one would need to read, digest, and implement into clinical practice between 1 and 6 articles per week, 52 weeks per year to keep current. Third, all 4 categories of clinical information significantly increased over the 9-year period. The average increase of articles per year is 14% for sensitive search and 21% for specific search. One can expect this trend to continue. Fourth, there was no strong evidence of relative emphasis given to any particular category of clinical information, pointing to a balanced body of orthodontic literature. Fifth, the evolution of an increasing body of literature predicts a subsequent development of additional diagnostic and therapeutic procedures and potentially the need for additional codes. Just as conceivable will be the parallel development of a combined literature that will enhance prognostics and advanced understanding of disease etiology. Finally, while libraries (personal or public) may subscribe to the “top 5” journals and cover approximately 45% of the clinical information, it is sobering to note that this misses 55% of the literature. This can lead to wide variation in “standards” of care.

This study has several limitations. First, certain relevant articles may have been omitted, while other irrelevant ones may have been included. It was with this issue in mind that the search strategies attempted to “bound” the available literature by using sensitive and specific searches. As expected, a sensitive search, in all 4 categories, retrieved more articles than a specific search. Second, the classification of “orthodontic” articles and their isolation from other specialties was made based on the authors' judgement. For example, “temporomandibular joint dysfunction syndrome” was an overlapping topic in multiple dental specialties. To isolate a potential overlap from articles of other disciplines, articles on temporomandibular joint dysfunction syndrome were not included in this analysis. This should be the focus of another study. Third, the key words used in the MEDLINE search were limited to MeSH vocabulary. The word selection was meant to be inclusive, but it may have excluded some relevant articles. Fourth, the data presented here are probably upper estimates of number of the articles that provide valid, clinically important, and clinically applicable information at a high level of evidence. Further assessments are needed to critically appraise the identified articles. Fifth, the level of evidence needs to be assessed. For example, the Agency Health Care Policy16 and the Centre for Evidence-based Medicine (http://cebm.jr2.ox.ac.uk/docs/levels.html) categorize evidence by quality levels.

In conclusion, while the search strategies, methodological filters, and results demonstrate a substantial and increasing clinical orthodontic literature, this approach to bibliometric assessment may also be useful for information retrieval and stratification. Thus, the next steps are to sample, critically appraise, stratify, and electronically catalog the identified literature to provide an accessible and ongoing electronic database. Such a database will be useful for multiple audiences. These audiences include: patients and clinicians making decisions about clinical care; academics concerned about the evidence base for curricular decisions; researchers interested in identifying gaps in the available knowledge base; corporate entities interested in developing new products; policy makers who fund clinical research; health care purchasers who make decisions about care compensation; and finally for professional societies that seek to provide guidance for their membership.