The first measurement of longitudinal decorrelations of harmonic flow amplitudes vn for n=2-4 in Xe+Xe collisions at sNN=5.44 TeV is obtained using 3 μb-1 of data with the ATLAS detector at the LHC. The decorrelation signal for v3 and v4 is found to be nearly independent of collision centrality and transverse momentum (pT) requirements on final-state particles, but for v2 a strong centrality and pT dependence is seen. When compared with the results from Pb+Pb collisions at sNN=5.02 TeV, the longitudinal decorrelation signal in midcentral Xe+Xe collisions is found to be larger for v2, but smaller for v3. Current hydrodynamic models reproduce the ratios of the vn measured in Xe+Xe collisions to those in Pb+Pb collisions but fail to describe the magnitudes and trends of the ratios of longitudinal flow decorrelations between Xe+Xe and Pb+Pb. The results on the system-size dependence provide new insights and an important lever arm to separate effects of the longitudinal structure of the initial state from other early and late time effects in heavy-ion collisions.