There has been increasing evidence of White Matter (WM) microstructural disintegrity and connectome disruption in Autism Spectrum Disorder (ASD). We evaluated the effects of age on WM microstructure by examining Diffusion Tensor Imaging (DTI) metrics and connectome Edge Density (ED) in a large dataset of ASD and control patients from different age cohorts. N = 583 subjects from four studies from the National Database of Autism Research were included, representing four different age groups: (1) A Longitudinal MRI Study of Infants at Risk of Autism [infants, median age: 7 (interquartile range 1) months, n = 155], (2) Biomarkers of Autism at 12 months [toddlers, 32 (11)m, n = 102], (3) Multimodal Developmental Neurogenetics of Females with ASD [adolescents, 13.1 (5.3) years, n = 230], (4) Atypical Late Neurodevelopment in Autism [young adults, 19.1 (10.7)y, n = 96]. For each subject, we created Fractional Anisotropy (FA), Mean- (MD), Radial- (RD), and Axial Diffusivity (AD) maps as well as ED maps. We performed voxel-wise and tract-based analyses to assess the effects of age, ASD diagnosis and sex on DTI metrics and connectome ED. We also optimized, trained, tested, and validated different combinations of machine learning classifiers and dimensionality reduction algorithms for prediction of ASD diagnoses based on tract-based DTI and ED metrics. There is an age-dependent increase in FA and a decline in MD and RD across WM tracts in all four age cohorts, as well as an ED increase in toddlers and adolescents. After correction for age and sex, we found an ASD-related decrease in FA and ED only in adolescents and young adults, but not in infants or toddlers. While DTI abnormalities were mostly limited to the corpus callosum, connectomes showed a more widespread ASD-related decrease in ED. Finally, the best performing machine-leaning classification model achieved an area under the receiver operating curve of 0.70 in an independent validation cohort. Our results suggest that ASD-related WM microstructural disintegrity becomes evident in adolescents and young adults—but not in infants and toddlers. The ASD-related decrease in ED demonstrates a more widespread involvement of the connectome than DTI metrics, with the most striking differences being localized in the corpus callosum.