The noncentrosymmetric ferromagnetic Weyl semimetal CeAlSi with simultaneous space-inversion and time-reversal symmetry breaking provides a unique platform for exploring novel topological states. Here, by employing multiple experimental techniques, we demonstrate that ferromagnetism and pressure can serve as efficient parameters to tune the positions of Weyl nodes in CeAlSi. At ambient pressure, a magnetism-facilitated anomalous Hall/Nernst effect (AHE/ANE) is uncovered. Angle-resolved photoemission spectroscopy (ARPES) measurements demonstrated that the Weyl nodes with opposite chirality are moving away from each other upon entering the ferromagnetic phase. Under pressure, by tracing the pressure evolution of AHE and band structure, we demonstrate that pressure could also serve as a pivotal knob to tune the positions of Weyl nodes. Moreover, multiple pressure-induced phase transitions are also revealed. These findings indicate that CeAlSi provides a unique and tunable platform for exploring exotic topological physics and electron correlations, as well as catering to potential applications, such as spintronics.