A promising application of neural-network quantum states is to describe the time dynamics of many-body quantum systems. To realize this idea, we employ neural-network quantum states to approximate the implicit midpoint rule method, which preserves the symplectic form of Hamiltonian dynamics. We ensure that our complex-valued neural networks are holomorphic functions, and exploit this property to efficiently compute gradients. Application to the transverse-field Ising model on a one- and two-dimensional lattice exhibits an accuracy comparable to the stochastic configuration method proposed in [Carleo and Troyer, Science 355, 602-606 (2017)], but does not require computing the (pseudo-)inverse of a matrix.