Tau pathology and brain atrophy are the closest correlate of cognitive decline in Alzheimer’s disease (AD). Understanding heterogeneity and longitudinal progression of atrophy during the disease course will play a key role in understanding AD pathogenesis. We propose a framework for longitudinal clustering that simultaneously: 1) incorporates whole brain data, 2) leverages unequal visits per individual, 3) compares clusters with a control group, 4) allows for study confounding effects, 5) provides cluster visualization, 6) measures clustering uncertainty. We used amyloid-β positive AD and negative healthy subjects, three longitudinal structural magnetic resonance imaging scans (cortical thickness and subcortical volume) over two years. We found three distinct longitudinal AD brain atrophy patterns: one typical diffuse pattern (n=34, 47.2%), and two atypical patterns: minimal atrophy (n=23 31.9%) and hippocampal sparing (n=9, 12.5%). We also identified outliers (n=3, 4.2%) and observations with uncertain classification (n=3, 4.2%). The clusters differed not only in regional distributions of atrophy at baseline, but also longitudinal atrophy progression, age at AD onset, and cognitive decline. A framework for the longitudinal assessment of variability in cohorts with several neuroimaging measures was successfully developed. We believe this framework may aid in disentangling distinct subtypes of AD from disease staging.