Osteoarthritis (OA) is the most common degenerative joint disease-causing pain and functional impairment. Adequate pain management using non-pharmacological therapy and/or pharmacological treatment may delay or even eliminate the need for total joint replacement (TJR) in some OA patients. To gain insights about the switching behavior between different types of pharmacological treatment for osteoarthritic-related pain over time, we analyzed long-term pharmacological treatment patterns for pain in patients with hip/knee OA before hip/knee TJR. Our study was conducted in a retrospective cohort design using anonymized German health claims data of N = 4,197,260 individuals. Starting from the date of a TJR in 2018, patients with OA were followed retrospectively over a period of five years and assigned to distinct, potency-related drug classes on a half-yearly basis. Within the main cohort, we identified n = 13,557 patients with OA and TJR in 2018 (mean age: 70.1 [SD = 10.4] years; women: 58.8%). The therapeutic potency of prescriptions for analgesic drugs and the switching behavior between drug classes increased with proximity to the TJR in 2018. 66.9% of all patients never had prescriptions for opioids. Moreover, 5.4% of all patients had no prescriptions for analgesic drugs at all. In particular, 30.6% of all patients did not have prescriptions for analgesic drugs in the ultimate half-year period before TJR in 2018. The results of our study suggest that existing pharmacological treatment options for pain may be inadequate for a relevant number of OA patients in need of higher-potent pain management.