INTRODUCTION

The Internet is a worldwide provider of health-related information that is increasingly accessed by both health professionals and nonexperts.1 A large-scale telephone poll in the United States revealed that 74% of adult users searched online in 2011 for medical topics.2 A similar tendency has been also traced for dental care consultation, as estimated by the Online Health Search 2006,3 which reports that 17 million Americans have occasionally sought relevant information.

The existing literature implicates a number of therapeutic procedures (ie, separator placement, archwire placement and activations, application of orthopedic forces, and debonding) to instigate a pain response in patients.4 Incidence of pain symptoms among orthodontically treated samples has been reported5–9 to vary from 70% to 95%. Furthermore, behavioral research5 has demonstrated that fear of anticipated pain prevented some patients from seeking orthodontic care. Based on this perspective, it is not surprising that every 10th orthodontic patient may discontinue therapy as a result of pain disturbances during the early stages of treatment.10

Given the growing influence of the Internet, it can be expected that persons undergoing or interested in orthodontic treatment will attempt to acquire data online on iatrogenic pain experience. Nevertheless, as a result of the unregulated and diverse nature of the Web, and given the lack of navigation knowledge of laypersons on valid electronic databases and peer-reviewed portals, there is a high likelihood that searchers will come across Web information that is of doubtful credibility and accuracy. Hence, the aim of this study was to assess the qualitative characteristics of publicly available e-information on pain associated with orthodontic intervention.

MATERIALS AND METHODS

Search Methods

A search of the Internet was carried out in February 2012 using five popular search engines: Google™ (www.google.com), bing™ (www.bing.com), YAHOO!® (www.yahoo.com), Ask® (www.ask.com), and AOL. (www.aol.com). The terms ‘orthodontic pain’ and ‘braces pain’ were entered individually to simulate the typical online search of a common layperson. The top 25 results for each search alternative were compiled for content appraisal. Exclusion criteria included the following: promotional product sites, discussion groups, video feeds, and links to scientific articles (Figure 1). Author name, profession, and type of information of all eligible Internet portals were extracted to contribute to the evaluation process.

View Figure

• Download as Powerpoint
• Download Figure

Quality Assessment

Accuracy

Accuracy can be defined as the degree of concordance of the information disseminated with the best evidence or with generally accepted clinical practice. The use of textbooks, evidence-based guidelines, and primary literature as reference sources have been methodologically adopted in studies assessing the accuracy of e-health information.11 In agreement with similar studies,12,13 we screened Web pages for providing accurate instructions using textbook material14 as the standard criterion. In particular, two of the authors investigated concurrently whether information regarding duration, etiology, and management of pain was given and, accordingly, scored the sites on a consensus basis, as follows: 0 points  =  no pain topic was discussed; 1 point  =  there was a discussion of one topic; 2 points  =  there was a discussion of two topics; 3 points  =  all three topics were discussed with major omissions; 4 points  =  all three topics were discussed with minor omissions; and 5 points  =  there was a complete discussion. The final ratings were converted to percentage scales to facilitate combined presentation of results.

Readability

Readability of a written text is an objective measure of the reading skills an individual must possess to understand that material.15 The Flesch Reading Ease Score (FRES)16 was used to determine the readability of reviewed sources. FRE, average sentence length (ASL), and average number of syllables per word (ASW) are interrelated through the following equation: FRE  =  206.835 − (1.015 × ASL) − (84.6 × ASW). The outcome is a number ranging from 0 to 100, with higher scores indicating easier text reading. Scores between 90 and 100 are considered easily understandable by an average fifth-grade student. A score range between 60 and 70 can be easily comprehended by eighth- and ninth-grade students. Finally, understanding of up to 30 score texts is considered representative of college- or university-graduate–level reading. For the purposes of the current study, an abstract from each site consisting of 200–500 words was copied and pasted into an online FRES calculator program (http://www.readabilityformulas.com/free-readability-formula-tests.php). Prior comparison12 of automated and manual calculation has confirmed the accuracy of the online method.

Further qualitative analysis was performed by means of the LIDA instrument (The LIDA Instrument, Version 1.2, Minervation Ltd, Oxford, UK). This validation tool was developed to assess the design and content of health care Web sites. There are three distinct areas measured: accessibility, usability, and reliability. The accessibility score is computed by filling in the Web address of the site on a customized Web platform (http://www.minervation.com/mod product/LIDA). A nine-item questionnaire is applied to evaluate usability and reliability (Table 1). The response alternatives are graded from 0 to 3 (0: never; 1: sometimes; 2: mostly; 3: always). Depending upon the input, this software produces final percentile scores that correspond to high, medium, or low quality. The LIDA score that is the average value of the three subscores indicates the overall rating of the design and structure of the site.

Table 1. LIDA Instrument Questions Intended to Evaluate Usability (1–4) and Reliability (5–9) of Web Sites

View Fullscreen

RESULTS

Authorship and Type of Information

In 56% of the sites, the text was not signed by the author (Table 2). A higher proportion of Web platforms, 17 out of 25, did not cite professional details of the contributor. The remainder of the articles were written by two orthodontists, two dentists, and four media professionals. Two of the authors, one from the media group and one of unknown occupation, mentioned personal treatment experience as the motivational factor to initiate writing about orthodontics. With reference to the information specificity, 13 pages claimed to provide exclusive information about aspects of orthodontic therapy. One Web page was found to cover topics of multidisciplinary oral rehabilitation. Tooth and general pain guidance were given in one and two Internet addresses, respectively. Of note, 32% of the reviewed pages had a broader scope of interest, posting information on fields such as nutrition, fitness, entertainment, and lifestyle.

Table 2. Author and Information Details of the Web Sites Analyzed in the Study^a

View Fullscreen

Quality Assessment

The evaluation outcomes and the description statistics of percentile scores are summarized in Tables 3 and 4. A box-and-whisker diagram was constructed to enable schematic representation of the distribution of the data set (Figure 2). It appears that accuracy, usability, and reliability were scattered over a wide range, implying, therefore, the variable quality of these characteristics in the available Internet sources.

View Figure

• Download as Powerpoint
• Download Figure

Table 3. Evaluation Scores of the Web Sites in Terms of Accuracy, Flesch Reading Ease Score (FRES), Accessibility, Usability, Reliability, and LIDA (in Brackets: the Maximum Achievable Scores; in Parentheses: the Corresponding Percentile Scores)

View Fullscreen

Table 4. Mean and Standard Deviation (SD) Values of Evaluated Categories—Percentile Scores (n  =  25)^a

View Fullscreen

DISCUSSION

To date studies dealing with orthodontic Web-based information have been rare, with the exception of those that deal directly with orthodontic issues such as extraction treatment17 or indirectly with the issues of cleft lip and palate12 and orthognathic surgery.18 All of these authors reached the conclusion that the quality of related information published on the Internet is variable. Lately, Knösel and Jung1 perceived a wide variety of orthodontic clips on a video-sharing Internet platform, with the highest proportion of clips uploaded by orthodontic patients.

Our intention was to apply a multievaluative approach to rating ordinary information sources visited by active or prospective orthodontic patients concerned about iatrogenic pain, a clinical issue that affects patient cooperation and acceptance of appliances.19 As indicated by the last Harris Poll results,2 more than two-thirds (69%) of health information queries start at a general search engine. Likewise, we operated five prevalent search programs, namely Google™, bing™, YAHOO!®, Ask®, and AOL, which, according to updated statistical figures,20 represent 65.9%, 15.1%, 14.5%, 2.9%, and 1.6% of the US search engine market share, respectively. To expand the pool of search results, we explored far beyond the first page of result links, which accounts for the standard retrieval technique in e-health information searches.21,22

Dental pain information, including actions taken to relieve pain and the impact on the quality of life, is shared extensively among Twitter users.23 Therefore, we decided to look into the accuracy of statements about legitimate patient concerns such as cause of pain, duration of pain, and remedies. To measure the data completeness we modified the scoring system used by Rajagopalan and colleagues13 in a comparative study of electronic scientific databases. A rather poor agreement was found between the broadcast data and reference textbook. Questions on the accuracy and theoretical quality of e-health information have been raised by recent studies24–26 in multiple medical branches. Several Web pages seemingly focused on orthodontic thematology present incomplete instructions. Another interesting observation was that non-orthodontists were in charge of the sites with the relatively highest scores. Differently stated, a Web site offered by a health professional or health service may not necessarily comprise quality information.27

Health domains are frequently produced at a level that is inappropriate for a public of mixed socioeconomic and educational background.28 Under this condition, readers may fail to broaden their understanding of disease processes or may even misinterpret crucial information on health care decisions. It is imperative for patient education materials to be written at a comprehensive level. Easily understood information can enhance patient's self-esteem and optimize delivery care at home as well as disease self-management.29 By extracting FRES, one of the oldest textual difficulty measures, we found that the bulk of the Web information was written in understandable language. Health domain administrators should be encouraged to improve reading ease to communicate a broad range of messages to a wide variety of audiences. On the other hand, oversimplifying text may yield the opposite result and render it difficult to convey accurate information.15

A wealth of quality rating instruments have emerged over recent years, aimed at reviewing health informative Web sites. Disappointingly, many of them have been identified as incompletely developed and of doubtful utility.30 Our evaluation tool of choice was the LIDA instrument, a validated set of criteria broadly used in assessment of health care–related material on the Internet.17,28,31–33 The mean total LIDA score fell within the range described by the aforementioned studies, approximating the outcome of 65% of the orthodontic study of Patel and Cobourne.17 Looking at the individual categories, the Web pages were rated with medium accessibility, even though they scored higher than all of the previous LIDA studies. Despite the ongoing efforts of Web administrators to secure availability for people with disabilities or with low-end technology,11 further improvement in this area should be pursued. Usability results exhibited a higher variability than did accessibility results, with the domains supported by specialists displaying the greatest scores. Efficient presentation of the information data is as essential as the information, per se. From this point of view, Internet users may continue the search even after accessing a page containing the desired answer, although the page is poorly organized.21 Lastly, we discerned a considerably low reliability of the sites under investigation, with direct implications for site transparency, expert authorship, review procedures, and update frequency. Therefore, the average, non–scientifically qualified person may be at risk of being misinformed or manipulated by secondary profit motives.

Access to controlled, user-friendly, and reliable information relating to orthodontic pain may offer numerous benefits to the public. By gaining knowledge over underlying causality, impact on daily activities, and alleviation strategies, patients can become more compliant and at the same time can be more active in a shared consumer-specialist decision-making model, keeping pace with the international trend in health care provision. Continuous development of orthodontic information e-resources and communities will contribute to improvement of quality health care in the near future. In the meantime, orthodontists should temper the existing discrepancy by referring patients to evidence-based education materials on the Internet.

CONCLUSIONS

• In this multiple evaluative study, we realized a variable quality of Internet information about pain experience encountered by orthodontic patients. The Web sites were regarded as disseminating incomplete data that require standard reading skills. LIDA scores for accessibility, usability, and reliability ranged from medium to low, with the last category being rated the least.

• No inferences could be made with regard to the relationship between authorship and quality of informational content. Internet users should be aware of the limitations when searching for online orthodontic advice and should allow themselves to be guided by specialists to valid Web databases.