The effective theories for many quantum phase transitions can be mapped onto those of classical transitions. Here we show that the naive mapping fails for the sub-Ohmic spin-boson model which describes a two-level system coupled to a bosonic bath with power-law spectral density, J(ω) α ω^s. Using an ∈ expansion we prove that this model has a quantum transition controlled by an interacting fixed point at small s, and support this by numerical calculations. In contrast, the corresponding classical long-range Ising model is known to display mean-field transition behavior for 0<s<1/2, controlled by a noninteracting fixed point. The failure of the quantum-classical mapping is argued to arise from the long-ranged interaction in imaginary time in the quantum model.