Antiferromagnetic FePS₃ has recently gained significant interest in its potential applications in spin-related devices. Here, we show that in-plane structural anisotropy has a major impact in shaping the optical responses of FePS₃ single-crystals from the bulk form down to the monolayer limit. X-ray diffraction on a bulk FePS₃ crystal confirms a distorted FeS₆ octahedron causing inequivalent Fe-Fe distances and consequently resulting in a higher a/b lattice parameter ratio. Micro-photoluminescence observations on bulk and monolayer FePS₃ reveal four emissions: one intra-atomic d-d transition (band A, centered at ∼1.24 eV) and three p-d charge transfer transitions (bands B, C, and D, centered around ∼1.79, ∼2.3, and ∼2.56 eV, respectively). These bands exhibit different polarization behaviors, which persist down to the monolayer limit. Density functional theory calculations from bulk to monolayer FePS₃ reveal the underlying electronic structure, assign the observed emissions, and indicate why these peaks have contrasting linear and circular polarization responses. These results establish a direct structure-optics relation in FePS₃, highlighting the strong coupling between lattice anisotropy, electronic transitions, and symmetry-selective optical selection rules.