Prenatal androgen exposure has been suggested to be one of the factors influencing handedness, making the androgen receptor gene (AR) a likely candidate gene for individual differences in handedness. Here, we examined the relationship between the length of the CAG-repeat in AR and different handedness phenotypes in a sample of healthy adults of both sexes (n = 1057). Since AR is located on the X chromosome, statistical analyses in women heterozygous for CAG-repeat lengths are complicated by X chromosome inactivation. We thus analyzed a sample of women that were homozygous for the CAG-repeat length (n = 77). Mixed-handedness in men was significantly associated with longer CAG-repeat blocks and women homozygous for longer CAG-repeats showed a tendency for stronger left-handedness. These results suggest that handedness in both sexes is associated with the AR CAG-repeat length, with longer repeats being related to a higher incidence of non-right-handedness. Since longer CAG-repeat blocks have been linked to less efficient AR function, these results implicate that differences in AR signaling in the developing brain might be one of the factors that determine individual differences in brain lateralization.