The sensory-action theory proposes that the neural substrates underlying action representations are related to a visuomotor action system encompassing the left ventral premotor cortex, the anterior intraparietal (AIP) and left posterior middle temporal gyrus (LPMT). Using fMRI, we demonstrate that semantic decisions on action, relative to non-action words, increased activation in the left AIP and LPMT irrespective of whether the words were presented in a written or spoken form. Left AIP and LPMT might thus play the role of amodal semantic regions that can be activated via auditory as well as visual input. Left AIP and LPMT did not distinguish between different types of actions such as hand actions and whole body movements, although a right STS region responded selectively to whole body movements.