Millions of people now use the Internet to gather medical information.[1] This can be seen as both an advantage and a concern. It is an advantage because it has become a rather easy tool for healthcare professionals and patients to acquire and share medical information. However, according to Silberg,[2] it is also a cause for concern, as when it comes to medical information, the Internet too often resembles a hodgepodge of information than a tool for effective health care communication and decision-making. The problem is not because of limited information, but rather, because it is too much, and that most of it may be inaccurate, misleading, and dangerous.[3] It has become increasingly difficult to discern which information is accurate and appropriate for users. This could potentially result in detrimental effects on those who do not use it appropriately.
There are several studies that have shown that some of the consumer health information on the Internet is inaccurate. Impicciatore and colleagues[4] looked at the accuracy of information on the treatment of fever in children and concluded that only a few of their reviewed web pages provided complete and accurate information for such a common and widely discussed condition.
Several published guidelines for evaluating the quality of health information on the Internet are available, which can help Internet users to avoid inaccurate information.[2]-[3],[5] These guidelines usually include a list of markers, indicators or criteria that are intended to help Internet users evaluate the overall quality, and, in effect, assume the accuracy of the content of the websites. Some of the more common indicators are the following:
- Authorship. Authors and contributors, their affiliations, and relevant credentials should be provided.
- Attribution. References and sources for all content should be listed clearly and all relevant copyright information noted.
- Disclosure. Website “ownership” should be prominent and disclosed, as should any sponsorship, advertising, commercial funding arrangements or support or potential conflict of interest.
- Currency. Dates that content was posted and updated should be indicated.
- Contact address or feedback mechanism. Contact information or address should be provided.
- Links. The links to other sources should be included.
- Design, aesthetics, grammar. It should characterize quality layout, interactivity, presentation, appeal, graphics, and use of media.
In a study by Fallis et al.,[6] they tried to find out if the proposed indicators of quality from the guidelines were indeed correlated with accuracy. Using the same topic by Impicciatore and colleagues[4] on the treatment of fever in children, they found that only some of the proposed indicators were correlated with accuracy, and that many of them were not. The presence of the HONcode logo (Health on the Net code), for example, turns out to be a fairly good indicator of accurate information about the treatment of fever in children. Several proposed indicators from prior published guidelines, such as authority and currency, do not appear to be good indicators of accurate health information.
One of the crucial health information readily available on the Internet pertains to diabetes mellitus. This particular health condition continues to be an important public concern, causing substantial morbidity and mortality and long-term complications. With the increasing rates of childhood and adult obesity, diabetes is expected to become even more prevalent in the coming decades. Despite advances in diagnosis and therapy, many people with diabetes are using complementary and alternative medicine (CAM).
In the United States, CAM is frequently used by adults, with 40% reporting use in the past 12 months[7] and an estimated 34% of adults with diabetes use some type of CAM therapy[8] despite the fact that most of them have not been proven to be beneficial. In the Philippines, a study done by Dahilig and Salenga[9] showed that 68.4% of the respondents in the rural area and 51.5% in the urban areas they surveyed in selected communities in Batangas, Caloocan, and Paranaque admitted to the use of CAM for the treatment of various diseases, including diabetes. Also from that study, it was observed that bitter melon (Momordica charantia), or ampalaya in the vernacular, was one of the more commonly used herbs, amounting to up to 43.5% usage by the respondents.
Bitter melon (Momordica charantia), also known as bitter gourd, has been traditionally used for diabetes. Several preparations of this plant are locally available and are widely advertised. However, the clinical efficacy of bitter melon is unproven. According to the American Diabetes Association guideline,[10] there is insufficient evidence to support the use of herbs or supplements for the treatment of diabetes. Safety issues are also a concern, as adverse effects are not well documented in some studies. Because patients with diabetes often take multiple prescription medications, there exists the potential for herb-drug and herb-dietary supplement interactions, leading to adverse events.[11]-[12] This is particularly troublesome for the majority (63%) of the general population does not disclose the use of CAM therapies to their physicians.[13]
It is, therefore, important for the health and well-being of the consumers of health information on the Internet, and for the societies to which they belong, that they are able to distinguish accurate from inaccurate information. Reliable quality indicators can be of help in evaluating data on the Internet. In this study, the objectives were to assess the accuracy of websites that contain information on the use of bitter melon in diabetes mellitus by comparing it to an accuracy tool that was developed, to seek the presence of the proposed quality indicators, and to determine whether these indicators indeed correlate with accuracy.
METHODOLOGYEthical Consideration
The study was submitted to the University of the Philippines Manila Research Ethics Board (UPMREB) Panel for ethics review and approval. The study was conducted only upon approval from UPMREB Panel. There are no conflicts of interest for the conduct of this study.
I. Internet Search ProtocolWebsites containing information on the use and role of bitter melon in diabetes mellitus were searched and noted. Popular internet search engines, specifically, the local versions of Google (google.com.ph) and Yahoo (ph.search.yahoo.com) were used, using the keywords “ampalaya” or “bitter gourd” or “bitter melon” and “diabetes.” The goal was to search for health information on the Internet in a manner that might be used by a layperson who needed information on bitter melon use in diabetes.
Included websites were those that specifically provided information on the role of bitter melon in diabetes mellitus. Websites that were simply selling bitter melon products in any form and not detailing information on its role in diabetes were not included. Websites with information found in peer-reviewed medical articles or electronic article databases were excluded. Websites were limited to English and Filipino languages. An attempt was made to find as many websites on this topic as possible, but, as there can be more than 400,000 results in a single search, only the first 20 search pages of Google and Yahoo, with each search page containing 10 websites, were included. The Internet search was done on a pre-specified time period, specifically, December 22-29, 2014.
II. Measure of AccuracyIn order to determine the accuracy of the content of the websites, the information contained on the websites was compared with the recommendations of authoritative sources, mainly published guidelines, and review articles. In particular, the following resources were used:
- Nutrition Therapy Recommendations for the Management of Adults With Diabetes: A Position Statement of the American Diabetes Association.[14]
- Complementary and Alternative Medicine Therapies for Diabetes: A Clinical Review by Birdee and Yeh.[15]
- Standards of Medical Care in Diabetes (2014) by the American Diabetes Association.[10]
- Systematic Review of Herbs and Dietary Supplements for Glycemic Control in Diabetes, Yeh.[16]
- Anti-diabetic and hypoglycaemic effects of Momordica charantia (bitter melon): a mini review by Leung et al.[17]
An accuracy measure tool was developed which consisted of several statements covering the following topics:
- The insufficiency of evidence to support the use Momordica charantia in the treatment of diabetes OR that it has not been approved for therapeutic use for patients with diabetes
- Potential adverse effect/s and contraindication/s to use (short and long term effects)
- The potential for drug interactions
Among the 3 topics, the first or the statement on the evidence of bitter melon and its recommendation for use in diabetics is deemed the most important.
Two observers (DPH, RJH) independently applied this accuracy measure tool to each website and recorded the results in a spreadsheet. For each of the topics, a website received 1 point for a completely correct statement, 0.5 points if it did not mention that specific topic and 0 points for an incorrect statement on the topic. In cases of disagreement, the two observers reassessed the information on the website to reach a consensus.
An overall accuracy score (between 0 and 3) was computed for each website by adding the scores on the individual statements or questions. This overall accuracy score takes into account both the correctness and the completeness of the information. Websites with the perfect score of 3 are deemed the “most accurate.” As mentioned above, the topic on the evidence/recommendation for use of bitter melon in diabetes is deemed the most important among the three topics, and as such, those websites that have at least a correct statement on that specific topic even though they did not get all three statements correct are deemed “somewhat accurate.” Those websites that did not give a correct statement on the first topic were deemed “least accurate.”
Accuracy, which is commonly used in diagnostics testing, is defined as the extent to which a measurement reflects the true value. For this study, this particular technique for assigning an overall accuracy score to a website was based on a previous study by Fallis et al.[6] and was modified to reflect the importance of the recommendation for use of bitter melon in diabetes over the other topics.
III. Measure of the Proposed Indicators of QualityIn addition to measuring the accuracy of the websites, the presence of the proposed quality indicators was determined. Several proposed indicators of quality were taken from published guidelines for evaluating the quality of health information on the internet.[2]-[2],[5] For each website, each proposed indicator below would be determined if it is present or absent.
- Whether the website had a commercial domain (e.g., webmd.com), an organization domain (e.g., bitter-gourd.org), an education domain (e.g., med.nyu.edu), and a government or country domain (e.g., ampalaya.ph)
- Whether the website was up to date
- Whether the website displayed the HON code logo – a code of ethical conduct for medical and health-related information available on the internet established by the Health On the Net (HON) Foundation
- Whether the website carried any advertising
- Whether the author was identified (and if so, whether the author was identified as a healthcare professional – a licensed physician, nurse, or a dietitian)
- Whether copyright was claimed or acknowledged
- Whether contact information or contact page was given
- Whether spelling errors appeared on the page (and, if so, how many)
- Whether peer-reviewed medical literature was cited
Data analysis was done using Stata SE Version 13. Quantitative variables were summarized and presented as the mean and standard deviation, while qualitative variables were tabulated and presented as frequency and percent distribution. Indicators associated with the “most accurate,” “somewhat accurate,” and “least accurate” websites were determined using logistic regression analysis. The level of significance was set at 5%.
Out of the first 20 search pages for both Google and Yahoo search, 158 websites were included in the study (see Figure 1). Table 1 describes the profile of the websites based on the presence of the proposed quality indicators.
Figure 1. Diagram showing how the websites were identified and screened for inclusion.
Table 1. Profile of websites based on the presence of the proposed quality indicators
Most of the current internet-based information on bitter melon’s role in diabetes is inaccurate as only 10 websites (6.33%) got an overall perfect score of 3, and thus, were deemed the “most accurate” and complete (Table 2). Logistic regression analysis showed that only the indicator HONcode logo (OR 12.1, p = 0.01) was significantly correlated with having a perfect score of 3.
Table 2. Indicators of websites that were deemed as “most accurate” (Score of 3/3)
Out of the 158 websites, only 21 (13.3%) at least gave a correct statement on the evidence/recommendation for use of bitter melon, (the most important among the 3 topics) and were deemed the “somewhat accurate” websites. Indicators, namely the author identified as a healthcare professional (OR 6.11, p = 0.008) and the cited peer-reviewed medical literature (OR 2.92, p = 0.029), were significantly correlated with at least giving the correct statement on the evidence/recommendation for use of bitter melon (Table 3). In contrast, the indicator no date given (OR 0.28, p 0.031) had a low probability of being accurate with regard to the above statement (Table 3).
Table 3. Indicators of websites that has at least an accurate statement on the evidence/recommendation for use of bitter melon (“somewhat accurate” websites)
Out of the 158 websites, 103 (65.2%) gave an incorrect statement on the evidence/recommendation for use of bitter melon and were deemed the “least accurate” websites. No indicator was noted to have the likelihood that the website would give an incorrect statement on the evidence/recommendation for use of bitter melon (Table 4).
Table 4. Indicators of websites that had an inaccurate statement on evidence/recommendation for use of bitter melon (“least accurate” websites).
Most of the information on bitter melon use in diabetes mellitus in the internet appeared to be inaccurate based on the accuracy measure that was developed, as only 21 (13.3%) websites stated the correct evidence or recommendation for its use, and fewer still, only 10 (6.33%) gave a more complete and accurate information. This was consistent with the findings of Molassiotis and Xu[18] about the quality of web-based information about herbal medicines, but this time, in the treatment of cancer, which showed that most sites were of low quality in several factors, including the accuracy of information.
The only indicator that was identified which correlated with the “most accurate” and complete websites was the presence of the HONcode logo (Table 2). It seemed that the quality-control checks required for certification by the Health On the Net (HON) Foundation’s Code of Conduct tend to result in giving out more accurate and complete health information. This is in accord with the conclusion of a study done by Fallis6 using a different health topic. In a study on the health information on hip resurfacing, it was noted that websites with the HONcode logo scored twice the total scores of those websites without it. The authors advocated that the patients should look for the presence of an “independent credibility check” such as the HONcode when searching for information on hip resurfacing.19 Thus, the presence of the HONcode logo seemed to be a proven indicator of accurate health information. The Health on the Net (HON) Foundation has also made it easier for the public to look for HON-accredited websites by creating a search engine, the HONcodeHunt (http://www.hon.ch/HONsearch/Patients/hunt.html) to search for HONcode-accredited websites, and the HON toolbar that users can install in their internet browsers (IE and Firefox) to automatically check the certification of the website being viewed. One foremost limitation of the HONcode is that it is not commonly seen in the websites that were included in this study, as it is only present in 5 (3.2%) of the 158 websites. Also, only about 5% of consumers would recognize the HONcode logo and know what it means,20, which may greatly limit its value.
Focusing only on the websites that at least gave the correct statement on the recommendation for use of bitter melon in diabetes and that were deemed “somewhat accurate,” two indicators – the author identified as a healthcare professional, and the presence of citation of a peer-reviewed medical literature – were correlated with being accurate (Table 3). Because the HONcode logo is infrequently seen, these two indicators may be the next best option for the public to use. It should be noted these two indicators were not seen to be consistent predictors of accuracy in a previous study6 unlike the HONcode logo, which was correlated with accuracy in previous studies, and thus, may be seen as a less robust indicator of accuracy. It seems that some indicators that were previously not correlated with accuracy do not guarantee that it will always remain that way. Also, identifying that a citation correctly comes from a peer-reviewed medical literature database may be challenging for the general public to do so to be truly useful.
The above-mentioned indicators of accuracy can be helpful in evaluating health information about bitter melon’s role in diabetes on the Internet, especially for the lay people. However, it should be noted that the presence of these indicators on a website does not guarantee that it contains accurate information because the relationship between these indicators and accuracy is probabilistic. Although the HONcode logo was significantly correlated with the “most accurate” websites, its presence on a website does not guarantee that it would have accurate information all the time, as was explicitly mentioned on its homepage[21] – it just would have a higher probability that its content would be accurate.
A major limitation of this study is that only the websites from the first 20 search pages from both Yahoo and Google were included out of the possible hundreds of thousands of websites. An automated Internet tool is ideal and should be developed to facilitate reviewing this large number of websites faster.
With increased use of the Internet for health information nowadays, it is important that lay people be able to distinguish accurate information from inaccurate ones. Most of the information on the use of bitter melon in diabetes mellitus on the Internet is inaccurate and misleading. This can be potentially hazardous to the general public. Reliable indicators of accuracy can be of help to them. This study has identified that the presence of the HONcode logo is a good indicator of the “most accurate” websites. Other indicators that can be used, at least to predict the accurate statement on the evidence or recommendation for use of bitter melon, include that the author is identified as a healthcare professional and that there is the presence of citation of a peer-reviewed medical literature. The absence of the date when the article was posted or updated seems to give a low probability of being accurate.
Future research can be done to include a wider range of health topics to see if these indicators of accuracy hold true to them and not just on this specific topic on bitter melon use on diabetics. It will also serve to monitor if a specific indicator continues to be correlated with accuracy and thus a relevant tool that the public can use.
AcknowledgmentsThe authors thank Jundelle Romulo Jalique for his inputs in the statistical analysis.
Statement of AuthorshipAll authors certified fulfillment of ICMJE authorship criteria.
Author DisclosureAll authors have declared no conflict of interest.
Funding SourceNone.
[1] Atkinson NL, Saperstein SL, Pleis J. Using the Internet for Health-Related Activities: Findings From a National Probability Sample. J Med Internet Res 2009;11(1):e4. PubMed PMCID DOI.