Organic pi-conjugated polymers are deemed to be soft materials with strong electron-phonon coupling, which results in the formation of polarons, i.e., charge carriers dressed by self-localized distortion of the nuclei. Universal signatures for polarons are optical resonances below the band gap and intense vibrational modes (IVMs), both found in the infrared (IR) spectral region. Here, we study p-doped conjugated homo- and copolymers by combining first-principles modelling and optical spectroscopy from the far-IR to the visible. Polaronic IVMs are found to feature absorption intensities comparable to purely electronic transitions and, most remarkably, show only loose resemblance to the Raman or IR-active modes of the neutral polymer. The IVM frequency is dramatically scaled down (up to 50%) compared to the backbone carbon-stretching modes in the pristine polymers. The very large intensity of IVMs is associated with displacement of the excess positive charge along the backbone driven by specific vibrational modes. We propose a quantitative picture for the identification of these polaron shifting modes that solely based on structural information, directly correlates with their IR intensity. This finding finally discloses the elusive microscopic mechanism behind the huge IR intensity of IVMs in doped polymeric semiconductors.