Correct nervous system development depends on the timely differentiation of progenitor cells into neurons. While the output of progenitor differentiation is well investigated at the population and clonal level, how stereotypic or variable fate decisions are during development is still more elusive. To fill this gap, we here follow the fate outcome of single neurogenic progenitors in the zebrafish retina over time using live imaging. We find that neurogenic progenitor divisions produce two daughter cells, one of deterministic and one of probabilistic fate. Interference with the deterministic branch of the lineage affects lineage progression. In contrast, interference with fate probabilities of the probabilistic branch results in a broader range of fate possibilities than in wild-type and involves the production of any neuronal cell type even at non-canonical developmental stages. Combining the interference data with stochastic modelling of fate probabilities revealed that a simple gene regulatory network is able to predict the observed fate decision probabilities during wild-type development. These findings unveil unexpected lineage flexibility that could ensure robust development of the retina and other tissues.