Background: Longitudinal studies examining mental health trajectories in refugees and asylum seekers are scarce. Objectives: To investigate trajectories of psychological symptoms and wellbeing in refugees and asylum seekers, and identify factors associated with these trajectories. Method: 912 asylum seekers and refugees from the control arm of three trials in Europe (n = 229), Turkey (n = 320), and Uganda (n = 363) were included. We described trajectories of psychological symptoms and wellbeing, and used trauma exposure, age, marital status, education, and individual trial as predictors. Then, we assessed the bidirectional interactions between wellbeing and psychological symptoms, and the effect of each predictor on each outcome controlling for baseline values. Results: Symptom improvement was identified in all trials, and for wellbeing in 64.7% of participants in Europe and Turkey, versus 31.5% in Uganda. In Europe and Turkey domestic violence predicted increased symptoms at post-intervention (ß = 1.36, 95% CI 0.17–2.56), whilst murder of family members at 6-month follow-up (ß = 1.23, 95% CI 0.27–2.19). Lower wellbeing was predicted by murder of family member (ß = −1.69, 95% CI −3.06 to −0.32), having been kidnapped (ß = −1.67, 95% CI −3.19 to −0.15), close to death (ß = −1.38, 95% CI −2.70 to −0.06), and being in the host country ≥2 years (ß = −1.60, 95% CI −3.05 to −0.14). In Uganda at post-intervention, having been kidnapped predicted increased symptoms (ß = 2.11, 95% CI 0.58–3.65), and lack of shelter (ß = −2.51, 95% CI −4.44 to −0.58) and domestic violence predicted lower wellbeing (ß = −1.36, 95% CI −2.67 to −0.05). Conclusion: Many participants adapt to adversity, but contextual factors play a critical role in determining mental health trajectories.