TikTok and Attention-Deficit/Hyperactivity Disorder: A Cross-Sectional Study of Social Media Content Quality

1Social media platforms are a popular means of sharing medical information online. TikTok in particular is a relatively new social media platform that has seen rapid adoption by adolescents and young adults, becoming the most downloaded social media application in 2020 with more than 1 billion monthly active users., In particular, the popularity of the platform appears to have contributed to increased awareness of attention-deficit/hyperactivity disorder (ADHD), with some individuals seeking a diagnosis after watching videos about ADHD on the platform., The hashtag “#adhd” is currently the seventh most popular health-related hashtag on the platform. Although social media can reduce mental health stigma and improve health literacy, there is also concern about misinformation and the potential for illness/health anxiety (“cyberchondria”) due to the volume of unmoderated, user-generated content online.,, For example, TikTok videos have been implicated in a recent rise of tic-like behaviours in adolescents, and it is thought that exposure to tic-related videos is responsible for this phenomenon.-

2Additionally, social media platforms use proprietary algorithms that focus on increasing user engagement and may promote videos that do not necessarily reflect accurate health information. A recent systematic review found that the prevalence of health misinformation was high across almost all social media platforms. With TikTok being a relatively new social media platform, there is little research on the quality and accuracy of its content. As a result, there have been increasing calls for more research to understand the potential implications of TikTok on health information dissemination. Recent studies examining TikTok videos have found that misinformation was common for a variety of medical topics including those about acne, diabetes, and cosmetic surgery.- To our knowledge, no study has examined the quality of mental health information disseminated on TikTok. Thus, our objective was to assess the quality of medical information in the most popular TikTok videos related to ADHD.

3The TikTok mobile phone application was queried with a search for the hashtag “#adhd” on July 18, 2021 (Figure 1). The proprietary search algorithm returns the most popular videos based on the number of views and likes per video. Our inclusion criteria were videos that specifically described or educated viewers about: ADHD symptoms/diagnosis, lived experience with ADHD, or ADHD management. Exclusion criteria were videos with no audio or text, non-English videos, videos unrelated to ADHD, or duplicate videos. Videos were sorted in descending order by view count, and the inclusion criteria were applied until 100 videos were identified. The sample size of 100 videos was chosen as a feasible and representative sample size to capture the most viewed videos on the platform. Previous studies have used similar sample sizes to analyze health-related videos on social media platforms such as TikTok and YouTube.,- Videos meeting the inclusion criteria were then included for extraction of video characteristics and further analysis.

4One psychiatrist and one psychiatry resident (AY, EN) with clinical experience in the diagnosis and management of ADHD independently assessed the videos using a categorical classification often used to assess the content quality of social media videos., Videos were viewed, analyzed, and classified as: (1) useful; (2) personal experience; or (3) misleading. Videos were classified as useful if the video contained scientifically correct information about any aspect of ADHD: prevention, symptoms, diagnosis, treatments, or other information (e.g., etiology, psychopathology, epidemiology, prognosis). Videos were classified as personal experiences if it described a user's own personal or anecdotal experience of ADHD symptoms or treatment. Videos were classified as misleading if it contained information lacking scientific evidence (e.g., unsubstantiated claims about treating ADHD). If a personal experience video additionally contained any generalized misleading statements, it was classified as misleading rather than personal experience.

5Where applicable, raters referenced the Canadian ADHD Resource Alliance (CADDRA) Canadian ADHD Practice Guidelines, Fourth Edition, and American Academy of Pediatrics ADHD Clinical Practice Guidelines., In cases where there was disagreement between the first two raters, a third psychiatrist rater (EAJ) independently rated the video to decide on the final rating.

6All videos were also assessed using the Patient Education Materials Assessment Tool for Audiovisual Materials (PEMAT-A/V), a validated instrument to assess the understandability and actionability of audiovisual patient education videos. Finally, videos were also rated on the Journal of American Medical Association (JAMA) benchmark criteria, a four-point scale which assesses medical information quality based on: (1) authorship (all authors and contributors, their affiliations, and relevant credentials should be provided), (2) attribution (references and sources for all content should be listed clearly, and all relevant copyright information noted), (3) disclosure (any sponsorship, advertising, underwriting, commercial funding arrangements or support, or potential conflicts of interest disclosed), and (4) currency (dates that content was posted and updated are indicated).

7Video metadata was extracted automatically using an open-source web scraper using TypeScript programming language (TikTok-scraper). Descriptive characteristics for each video included: user type (individual, organization, or healthcare provider [HCP]), number of views, number of likes, and number of shares. The kappa statistic was calculated to assess inter-rater reliability for the initial video classification between the two raters. Student's t test was used to compare video characteristics, PEMAT scores, and JAMA scores between HCPs and non-HCPs video uploaders. Pearson's chi-square test was used to compare frequencies of the video classifications between HCPs and non-HCPs. One-way analysis of variance was used to compare video characteristics, PEMAT scores, and JAMA scores between the three video classifications. A P-value of less than 0.05 was considered significant for all analyses. All statistical analyses were performed using Stata 17.0.

8Prior to applying the exclusion criteria, there were a total of 4.3 billion views on videos tagged with “#adhd” at the time of the query. The 100 videos meeting inclusion criteria had a total of 283,459,400 views; 89 videos were uploaded by non-HCPs, while 11 were uploaded by individuals identifying as HCPs (i.e., licensed therapists, nurses, psychologists, or physicians). None of the videos in this sample were uploaded by corporations, health organizations, for-profit entities, or non-profit entities. There was an average of 2.8 million views (range: 892,100–14,800,000) and 31,175 shares (range: 1,249–245,200) per video. The average video length was 36.7 s (range: 8.0–74). Across all videos, the average PEMAT-A/V understandability score was 94.3% (range: 66.7–100%), and the average PEMAT-A/V actionability score was 14.3% (range: 0–100%). The average JAMA benchmark score was 1.3 (range: 1–3).

9The full video characteristics by the classification are described in Table 1. Of the 100 videos analyzed, 52% were classified as misleading, 27% as personal experience, and 21% as useful. Agreement between the raters on the classification was 86%, with a kappa statistic of 0.7766 (P < 0.001). Personal experience videos had the highest PEMAT-A/V understandability score, with an average score of 98.3% (P < 0.001), and were the most popular (based on the average number of likes, views, and shares).

10Table 2 provides illustrative examples of useful, personal experience, and misleading videos that were included for analysis. Of the 52 misleading videos, 37 videos (71%) misattributed transdiagnostic psychiatric symptoms as being specific only to ADHD, including anxiety, depression, anger, relationship conflicts, dissociation, and mood swings. None of the misleading videos recommended viewers to seek out a medical, psychiatric, or psychological assessment before attributing these symptoms to ADHD. Eight videos (15%) misrepresented the pathophysiology of ADHD, including oversimplifying the disorder as a purely dopamine-deficient state. Four videos (7%) provided incorrect information about the approach to diagnosing ADHD, such as including an audio quiz to determine whether an individual has ADHD. Two videos (4%) misrepresented the association between ADHD and developmental theories, such as stating that individuals with ADHD lack object permanence. Finally, one video (2%) suggested non-validated coping strategies for ADHD with no evidence base.

11Of the 21 useful videos, 17 videos (81%) described signs and symptoms of ADHD specific to the disorder, or informed viewers that transdiagnostic symptoms may be due to other medical conditions or comorbid psychiatric disorders. Three videos (14%) provided information on treatments or strategies for coping with ADHD that have been validated. One video (5%) accurately described the prognosis and risk factors for ADHD. Of the 27 personal experience videos, all videos described individual experiences of living with ADHD symptoms and/or its impact on day-to-day life.

12The full descriptive characteristics of the videos by creator type (HCP vs. non-HCP) are described in Table 3. Non-HCPs had significantly more misleading (55.1% vs. 27.3%, P < 0.001) and personal experience videos (29.2% vs. 9.1%, P < 0.001) than HCPs. HCPs had significantly more useful videos than non-HCPs (63.6% vs. 15.7%, P < 0.001). Non-HCP videos were more popular, though this was not statistically significant.

13In this analysis of popular TikTok videos about ADHD, there were over 2.8 million views per video and each video was shared on average 31,000 times. Approximately half of the videos analyzed (52%) were misleading, with non-HCPs uploading most of these videos (49 out of 52 videos). There was substantial agreement among raters on the classification of the videos as useful, misleading, or personal experience. Our study is the first to show that misleading videos about ADHD are being widely disseminated and viewed on TikTok.

14These findings are in line with previous studies of TikTok videos that have found high rates of misinformation for medical conditions, including acne and diabetes., Our results are also similar to a study by Thapa et al. that analyzed YouTube videos about ADHD and found that 38% of analyzed videos were misleading and only 5% were useful. The study also found that first-person or personal experience videos received the highest engagement (i.e., likes and views), while misleading videos were the most common. Our study replicated these findings on TikTok: personal experience videos received the highest engagement (i.e., views, likes, and shares), while misleading videos were the most common. These consistent findings across different social media platforms suggest that viewers are most drawn to videos made by individuals with lived experience, and less so towards institutional or HCP-created videos. Increasingly, social media influencers and independent video creators have been found to have an outsized influence on the dissemination of health information online.- Indeed, in our study, none of the top 100 videos were uploaded by corporations, health organizations, for-profit entities, or non-profit entities, and only 11% of videos were uploaded by HCPs. This limited presence of HCPs on TikTok has been observed in other studies as well. Overall, HCPs did score significantly higher on the JAMA benchmark criteria and made significantly more useful videos and less misleading videos. Although this suggests that healthcare providers do upload higher quality content on the platform, 27% of HCP videos (3 out of 11) were still rated as misleading.

15We additionally found that all videos about ADHD were highly understandable, scoring over 90% on the PEMAT-A/V understandability score. The understandability score does not reflect accuracy, but merely that the information is presented in an understandable manner. Thus, individuals may be seeing videos about ADHD on the platform that are highly understandable and yet misleading. This was seen in our qualitative review of misleading videos, which often had an oversimplified or reductionist explanation of ADHD.

16The findings from our study provide several important insights into the dissemination of medical information about ADHD on TikTok. Approximately half of all videos analyzed were misleading, and the misinformation they contain has the potential to contribute to health anxiety or lead to increased healthcare utilization. The proprietary TikTok algorithm has been found to have a propensity to show users similar videos over time, which may further propagate misleading videos., Additionally, viral trends on TikTok, “echo chambers” in social media, and romanticization of mental health symptoms, may perpetuate the dissemination of misleading information on TikTok. Increased healthcare use may occur for individuals who seek clinical attention after viewing TikTok videos about ADHD, a phenomenon that has been widely documented by major news organizations., Since self-report of ADHD symptoms may be over-endorsed, there may be an increased risk for overdiagnosis or misdiagnosis in these individuals as well. In general, there has also been renewed interest over rising rates of ADHD diagnoses,- and debate about the overdiagnosis of ADHD.

17TikTok has become particularly popular over the course of the COVID-19 pandemic. The pandemic context raises the question of whether individuals may be misattributing difficulties from pandemic public health measures (e.g., having difficulty with maintaining attention when viewing a screen in isolation for prolonged periods) to ADHD symptoms., One related example is the rapid increase in tic-like behaviours that have been observed over the course of the COVID-19 pandemic., Tic disorder specialists have seen a dramatic increase in referrals for tic-like behaviours in primarily adolescent females that are inconsistent with usual presentations of primary tic disorders. These presentations are thought to be most consistent with a diagnosis of functional neurological disorder, related to stressors over the course of the pandemic. The sociogenic propagation of tic-like behaviours across TikTok is a reflection of the impact of social media platforms on health behaviours.,

18Limitations of our study include TikTok's proprietary search algorithm results, which exclude video ads and do not allow for systematic searching of all videos or deleted videos. Not all videos contained detailed information to identify whether an uploader was a health professional or not. Additionally, the current study only screened for the top 100 most popular videos. Less popular videos may have different content quality and characteristics not captured in this study. Also, the conceptualization of what is considered as a misleading, useful, or personal experience video was done from the perspective of raters who are HCPs. This perspective may not necessarily be in line with healthcare consumers or patients. For instance, while some videos may not be useful from the perspective of an HCP, others may find it useful for other reasons. Additionally, although our inter-rater agreement was high, videos classified as useful could potentially be classified as personal experience videos. Videos on the platform may also be recorded to be humorous or spontaneous, and not necessarily recorded with the intent to disseminate medical information. However, even videos made without intent to disseminate medical information may describe non-specific symptoms, overgeneralizations, and characterizations about ADHD that could be misleading to viewers. Finally, there have also been criticisms of the use of tools such as the JAMA benchmark criteria as a method to evaluate online videos. However, there are currently no other widely used or accepted methods of assessing medical information videos online.

19In summary, this study is the first to show that misleading videos about ADHD are being widely disseminated on TikTok. Although the platform is the most popular social media application of 2020 and 2021, it is currently the least studied of the major social media platforms. It is thus important for clinicians to be aware of the dissemination of misleading videos on TikTok and the potential impact on clinical care. Future areas of research include a better understanding of the prevalence and nature of misinformation on TikTok for other mental health topics such as depression, anxiety, suicide, and self-harm. Our finding that HCPs generally upload higher quality and more useful videos deserves replication and further study to see whether HCP engagement on the platform could help correct misinformation regarding ADHD and other mental health disorders.