Verbal rating scale polarity and verbalizations of the middle category that do not match the polarity in agreement rating scales were investigated. Two randomized web survey experiments were conducted using a probability panel of German Internet users. The classical bipolar "disagree/agree" verbalization was compared with the unipolar "do not agree/agree" alternative. In both experiments, attitudes on gender roles were measured using a different number of rating scale categories (seven vs. five) in each. After controlling for variables related to satisficing, biases were lowest with the unipolar "do not agree/agree" rating scale in which the verbalization of the middle category was also unipolar. Reliability decreased when the verbalization of the middle category did not suit the rating scale polarity. Researchers should pay attention to the polarity of rating scales and use verbalization of the middle category consistent with the rating scale polarity.