Abstract
Purpose: Telemedicine approaches to autism (ASD) assessment have become increasingly common, yet few validated tools exist for this purpose. This study presents results from a clinical trial investigating two approaches to tele-assessment for ASD in toddlers. Methods: 144 children (29% female) between 17 and 36 months of age (mean = 2.5 years, SD = 0.33 years) completed tele-assessment using either the TELE-ASD-PEDS (TAP) or an experimental remote administration of the Screening Tool for Autism in Toddlers (STAT). All children then completed traditional in-person assessment with a blinded clinician, using the Mullen Scales of Early Learning (MSEL), Vineland Adaptive Behavior Scales, 3rd Edition (VABS-3), and Autism Diagnostic Observation Schedule (ADOS-2). Both tele-assessment and in-person assessment included a clinical interview with caregivers. Results: Results indicated diagnostic agreement for 92% of participants. Children diagnosed with ASD following in-person assessment who were missed by tele-assessment (n = 8) had lower scores on tele- and in-person ASD assessment tools. Children inaccurately identified as having ASD by tele-assessment (n = 3) were younger than other children and had higher developmental and adaptive behavior scores than children accurately diagnosed with ASD by tele-assessment. Diagnostic certainty was highest for children correctly identified as having ASD via tele-assessment. Clinicians and caregivers reported satisfaction with tele-assessment procedures. Conclusion: This work provides additional support for the use of tele-assessment for identification of ASD in toddlers, with both clinicians and families reporting broad acceptability. Continued development and refinement of tele-assessment procedures is recommended to optimize this approach for the needs of varying clinicians, families, and circumstances.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Berger, N. I., Wainer, A. L., Kuhn, J., Bearss, K., Attar, S., Carter, A. S., Ibanez, L. V., Ingersoll, B. R., Neiderman, H., Scott, S., & Stone, W. L. (2021). Characterizing available tools for synchronous virtual Assessment of Toddlers with suspected autism spectrum disorder: a brief report. Journal of Autism and Developmental Disorders. https://doi.org/10.1007/s10803-021-04911-2.
Charter, R. A., & Feldt, L. S. (1996). Testing the Equality of two alpha coefficients. Perceptual and Motor Skills, 82(3), 763–768. https://doi.org/10.2466/pms.1996.82.3.763.
Corona, L., Hine, J., Nicholson, A., Stone, C., Swanson, A., Wade, J., Wagner, L., Weitlauf, A., Warren, Z. TELE-ASD-PEDS: A Telemedicine-based ASD Evaluation Tool for Toddlers and, & Children, Y. (2020). https://vkc.vumc.org/vkc/triad/tele-asd-peds (Vanderbilt University Medical Center)
Corona, L. L., Wagner, L., Wade, J., Weitlauf, A. S., Hine, J., Nicholson, A., Stone, C., Vehorn, A., & Warren, Z. (2021). Toward Novel Tools for Autism Identification: fusing computational and clinical expertise. Journal of Autism and Developmental Disorders, 51(11), 4003–4012. https://doi.org/10.1007/s10803-020-04857-x.
Corona, L. L., Weitlauf, A. S., Hine, J., Berman, A., Miceli, A., Nicholson, A., Stone, C., Broderick, N., Francis, S., Juárez, A. P., Vehorn, A., Wagner, L., & Warren, Z. (2021). Parent perceptions of caregiver-mediated Telemedicine Tools for assessing autism risk in toddlers. Journal of Autism and Developmental Disorders, 51(2), 476–486. https://doi.org/10.1007/s10803-020-04554-9.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/bf02310555.
Dahiya, A. V., Delucia, E., McDonnell, C. G., & Scarpa, A. (2021). A systematic review of technological approaches for autism spectrum disorder assessment in children: implications for the COVID-19 pandemic. Research in Developmental Disabilities, 109, 103852. https://doi.org/10.1016/j.ridd.2021.103852.
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the Areas under two or more correlated receiver operating characteristic curves: a Nonparametric Approach. Biometrics, 44(3), https://doi.org/10.2307/2531595.
Diedenhofen, B. (2016). Cocron: Statistical Comparisons of Two or more Alpha Coefficients (Version 1.0–1). Available from http://comparingcronbachalphas.org
Dow, D., Holbrook, A., Toolan, C., McDonald, N., Sterrett, K., Rosen, N., Kim, S. H., & Lord, C. (2021). The brief Observation of symptoms of Autism (BOSA): development of a New adapted Assessment measure for remote Telehealth Administration through COVID-19 and Beyond. Journal of Autism and Developmental Disorders. https://doi.org/10.1007/s10803-021-05395-w.
Feldt, L. S. (1969). A test of the hypothesis that cronbach’s alpha or kuder-richardson coefficent twenty is the same for two tests. Psychometrika, 34(3), 363–373. https://doi.org/10.1007/bf02289364.
Feldt, L. S. (1980). A test of the hypothesis that Cronbach’s alpha reliability coefficient is the same for two tests administered to the same sample. Psychometrika, 45(1), 99–105. https://doi.org/10.1007/bf02293600.
Finn, J. A., Ben-Porath, Y. S., & Tellegen, A. (2015). Dichotomous versus polytomous response options in psychopathology assessment: method or meaningful variance? Psychological Assessment, 27(1), https://doi.org/10.1037/pas0000044.
Jones, M. K., Zellner, M. A., Hobson, A. N., Levin, A., & Roberts, M. Y. (2022). Understanding caregiver satisfaction with a telediagnostic assessment of autism spectrum disorder. American Journal of Speech-Language Pathology, 31(2), 982–990. https://doi.org/10.1044/2021_AJSLP-21-00139.
Juárez, A. P., Weitlauf, A. S., Nicholson, A., Pasternak, A., Broderick, N., Hine, J., Stainbrook, J. A., & Warren, Z. (2018). Early Identification of ASD through Telemedicine: potential value for Underserved populations. Journal of Autism and Developmental Disorders, 48(8), 2601–2610. https://doi.org/10.1007/s10803-018-3524-y.
Lord, C., Risi, S., Lambrecht, L., Cook, E. H. Jr., Leventhal, B. L., Dilavore, P. C., Pickles, A., & Rutter, M. (2000). The Autism Diagnostic Observation schedule - generic: a standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30, 205–223. doi:https://doi.org/10.1023/a:1005592401947.
Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S. (2012). Autism diagnostic observation schedule, second edition (ADOS-2) manual (part 1): modules 1–4. Western Psychological Services.
Luyster, R., Gotham, K., Guthrie, W., Coffing, M., Petrak, R., Pierce, K., Bishop, S., Esler, A., Hus, V., Oti, R., Richler, J., Risi, S., & Lord, C. (2009). The Autism Diagnostic Observation Schedule—Toddler Module: a New Module of a standardized diagnostic measure for Autism Spectrum Disorders. Journal of Autism and Developmental Disorders, 39(9), 1305–1320. https://doi.org/10.1007/s10803-009-0746-z.
Maxwell, S. E., & Delaney, H. D. (2003). Designing Experiments and Analyzing Data. In Designing Experiments and Analyzing Data. https://doi.org/10.4324/9781410609243
Moskowitz, C. S., & Pepe, M. S. (2006). Comparing the predictive values of diagnostic tests: sample size and analysis for paired study designs. Clinical Trials, 3(3), 272–279. https://doi.org/10.1191/1740774506cn147oa.
Mullen, E. M. (1995). Mullen Scales of early learning. American Guidance Service.
R Core Team (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.
Reese, R. M., Jamison, R., Wendland, M., Fleming, K., Braun, M. J., Schuttler, J. O., & Turek, J. (2013). Evaluating interactive videoconferencing for assessing symptoms of Autism. Telemedicine and e-Health, 19(9), 671–677. https://doi.org/10.1089/tmj.2012.0312.
Reese, R. M., Jamison, T. R., Braun, M., Wendland, M., Black, W., Hadorn, M., Nelson, E. L., & Prather, C. (2015). Brief report: use of interactive television in identifying autism in Young Children: Methodology and Preliminary Data. Journal of Autism and Developmental Disorders, 45(5), 1474–1482. https://doi.org/10.1007/s10803-014-2269-5.
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J. C., & Müller, M. (2011). pROC: An open-source package for R and S + to analyze and compare ROC curves. BMC Bioinformatics, 12. https://doi.org/10.1186/1471-2105-12-77
Sparrow, S. D., Cicchetti, D. V., & Saulnier, C. A. (2016). Vineland Adaptive Behavior Scales, Third Edition. Pearson.
Stainbrook, J. A., Weitlauf, A. S., Juárez, A. P., Taylor, J. L., Hine, J., Broderick, N., Nicholson, A., & Warren, Z. (2019). Measuring the service system impact of a novel telediagnostic service program for young children with autism spectrum disorder. Autism, 23(4), 1051–1056. https://doi.org/10.1177/1362361318787797.
Stavropoulos, K. K. M., Bolourian, Y., & Blacher, J. (2022). A scoping review of telehealth diagnosis of autism spectrum disorder. Plos One, 17(2), e0263062. https://doi.org/10.1371/journal.pone.0263062.
Stavropoulos, K. K. M., Heyman, M., Salinas, G., Baker, E., & Blacher, J. (2022). Exploring telehealth during COVID for assessing autism spectrum disorder in a diverse sample. Psychology in the Schools. https://doi.org/10.1002/pits.22672.
Stock, C., & Hielscher, T. (2014). DTComPair: comparison of binary diagnostic tests in a paired study design. R package version 1.0.3. URL: http://CRAN.R-project.org/package=DTComPair
Stone, W. L., Coonrod, E. E., & Ousley, O. Y. (2000, Dec). Brief report: screening tool for autism in two-year-olds (STAT): development and preliminary data. Journal Of Autism And Developmental Disorders, 30(6), 607–612. https://www.ncbi.nlm.nih.gov/pubmed/11261472.
Stone, W. L., Coonrod, E. E., & Ousley, O. Y. (2000). Brief report: Screening Tool for Autism in Two-Year-Olds(STAT): Development and Preliminary Data. Journal of Autism and Developmental Disorders, 30(6), 607–612. https://doi.org/10.1023/a:1005647629002.
Stone, W. L., Coonrod, E. E., Turner, L. M., & Pozdol, S. L. (2004). Psychometric Properties of the STAT for early autism screening. Journal of Autism and Developmental Disorders, 34(6), 691–701. https://doi.org/10.1007/s10803-004-5289-8.
Stone, W. L., McMahon, C. R., & Henderson, L. M. (2008). Use of the Screening Tool for Autism in Two-Year-Olds (STAT) for children under 24 months. Autism, 12(5), 557–573. https://doi.org/10.1177/1362361308096403.
Thiele, C., & Hirschfeld, G. (2021). Cutpointr: improved estimation and validation of optimal cutpoints in r. Journal of Statistical Software. https://doi.org/10.18637/jss.v098.i11. 98.
Tomarken, A. J., & Serlin, R. C. (1986). Comparison of anova Alternatives under Variance Heterogeneity and specific noncentrality structures. Psychological Bulletin, 99(1), https://doi.org/10.1037/0033-2909.99.1.90.
Wagner, L., Corona, L. L., Weitlauf, A. S., Marsh, K. L., Berman, A. F., Broderick, N. A., Francis, S., Hine, J., Nicholson, A., Stone, C., & Warren, Z. (2020). Use of the TELE-ASD-PEDS for autism evaluations in response to COVID-19: preliminary outcomes and clinician acceptability. Journal of Autism and Developmental Disorders. https://doi.org/10.1007/s10803-020-04767-y.
Wagner, L., Stone, C., Wade, J., Corona, L., Hine, J., Nicholson, A., Swanson, A., Vehorn, A., Weitlauf, A., & Warren, J. (2021). TELE-ASD-PEDS user’s Manual. Vanderbilt Kennedy Center Treatment and Research Institute for Autism Spectrum Disorders.
Wagner, L., Weitlauf, A. S., Hine, J., Corona, L. L., Berman, A. F., Nicholson, A., Allen, W., Black, M., & Warren, Z. (2022). Transitioning to Telemedicine during COVID-19: impact on perceptions and use of Telemedicine Procedures for the diagnosis of Autism in Toddlers. Journal of Autism and Developmental Disorders, 52(5), 2247–2257. https://doi.org/10.1007/s10803-021-05112-7.
Zhou, X. H., Obuchowski, N. A., & McClish, D. K. (2011). Measures of diagnostic accuracy. In Statistical Methods in Diagnostic Medicine (pp. 13–55). Wiley. https://doi.org/10.1002/9780470906514
Zwaigenbaum, L., & Warren, Z. (2020). Commentary: embracing innovation is necessary to improve assessment and care for individuals with ASD: a reflection on Kanne and Bishop (2020). Journal of Child Psychology and Psychiatry. https://doi.org/10.1111/jcpp.13271.
Funding
The study was supported by funding from NIH/NIMH (R21MH118539), the Eunice Kennedy Shriver National Institute of Child Health and Human Development (U54 HD08321), and the Vanderbilt Institute for Clinical and Translational Research. The Vanderbilt Institute for Clinical and Translational Research (VICTR) is funded by the National Center for Advancing Translational Sciences (NCATS) Clinical Translational Science Award (CTSA) Program, Award Number 5UL1TR002243-03.
Author information
Authors and Affiliations
Contributions
The first draft of the manuscript was written by Laura Corona. Madison Hooper completed substantial data analysis and reviewed multiple drafts of the manuscript. Liliana Wagner and Amy Weitlauf reviewed and revised the manuscript. All authors contributed to study design and/or data collection. All authors reviewed the final manuscript.
Data included in this manuscript can be accessed in the NIMH Data Archive. Dataset identifiers: https://doi.org/10.15154/1528433.
Corresponding author
Ethics declarations
Conflict of Interest
Authors Corona, Wagner, Weitlauf, Hine, Nicholson, Stone, and Warren are authors of the TAP. They do not receive compensation related to use of this tool.
Ethics Approval
All procedures performed in this study were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This research was approved by the Institutional Review Board at Vanderbilt University Medical Center.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Corona, L.L., Wagner, L., Hooper, M. et al. A Randomized Trial of the Accuracy of Novel Telehealth Instruments for the Assessment of Autism in Toddlers. J Autism Dev Disord 54, 2069–2080 (2024). https://doi.org/10.1007/s10803-023-05908-9
Accepted:
Published:
Version of record:
Issue date:
DOI: https://doi.org/10.1007/s10803-023-05908-9