Via mechanisms not accessible at equilibrium, self-propelled particles can form phases with positional order, such as crystals, and with orientational order, such as polar flocks. However, the interplay between these two types of order remains relatively unexplored. Here, we address this point by studying crystals of active particles that turn either towards or away from each other, which can be experimentally realised with phoretic or Janus colloids or with elastically-coupled walker robots. We show that, depending on how these interactions vary with interparticle distance, the particles align along directions determined by the underlying crystalline lattice. To explain the results, we map the orientational dynamics of the active crystal onto a lattice of spins that interact via (anti-)ferromagnetic alignment with each other plus nematic alignment with the lattice directions. Our findings indicate that orientational and positional order can be strongly coupled in active crystals, thus suggesting strategies to control orientational order by engineering the underlying crystalline lattice.