Nearly magnetic metals often have layered lattice structures consisting of coupled planes. In such a situation, physical properties will display, upon decreasing temperature or energy, a dimensional crossover from two-dimensional (2D) to three-dimensional (3D) behavior, which is particularly interesting near quantum criticality. Here we study this crossover in thermodynamics using a suitably generalized Landau-Ginzburg-Wilson approach to the critical behavior, combined with renormalization group techniques. We focus on two experimentally relevant cases: the crossover from a 2D to a 3D antiferromagnet, and the crossover from a 2D ferromagnet to a 3D antiferromagnet. As naive scaling does not apply at and above the upper critical dimension, two crossover scales arise which can be associated with separate dimensional crossovers of classical and quantum fluctuations, respectively. In particular, we find an intermediate regime with distinct power laws where the quantum fluctuations still have 2D and the classical fluctuations already have a 3D character. For the ferromagnet-to-antiferromagnet crossover, the mismatch of the dynamical exponents between the 2D and 3D regimes leads to an even richer crossover structure, with an interesting 2D noncritical regime sandwiched between two critical regimes. For all cases, we find that thermal expansion and compressibility are particularly sensitive probes of the dimensional crossover. Finally, we relate our results to experiments on the quantum critical heavy-fermion metals CeCu6-x Aux, YbRh2 Si2, and CeCoIn5.