The ongoing transition to renewable energy supply comes with a restructuring of power grids, changing their effective interaction topologies, more and more strongly decentralizing them and substantially modifying their input, output, and response characteristics. All of these changes imply that power grids become increasingly affected by collective, nonlinear dynamic phenomena, structurally and dynamically more distributed and less predictable in space and time, more heterogeneous in its building blocks, and as a consequence less centrally controllable. Here cornerstone aspects of data-driven and mathematical modeling of collective dynamical phenomena emerging in real and model power grid networks by combining theories from nonlinear dynamics, stochastic processes and statistical physics, anomalous statistics, optimization, and graph theory are reviewed. The mathematical background required for adequate modeling and analysis approaches is introduced, an overview of power system models is given, and a range of collective dynamical phenomena are focused on, including synchronization and phase locking, flow (re)routing, Braess's paradox, geometric frustration, and spreading and localization of perturbations and cascading failures, as well as the nonequilibrium dynamics of power grids, where fluctuations play a pivotal role.