This review paper summarizes and critically discusses the morphology and structure formation of carbon fibers (CFs) from polymer precursor systems. Throughout this review, we focus on the key mutual interactions between the polymeric precursor systems, their physically determined processability into filaments, the thermally initiated crystal conversion mechanisms, as well as the morphological and physical properties of the resulting CFs and graphite fibers (GFs). Understanding the behavior of crystal conversion mechanisms from a polymeric semi-crystalline structure into a turbostratic, glass-like or even a graphite-like carbonaceous crystalline structure is essential to carbon and graphite fiber formation. The nature of the crystal conversion and thermal processing largely determine the recovery degree and behavior of the carbonaceous crystal orientation. Over the last three decades, CFs and GFs have earned a significant reputation as lightweight fibrous reinforcement materials, and considerable advances have been achieved in understanding the structure of CFs and GFs and in tailoring their performance towards specific applications. The utilization of CFs and GFs in different thermoplastics and thermosets, as well as in concrete as reinforcements, is well known thanks to the abundant number of reports and reviews available. Nevertheless, large-scale utilization of CFs in high-technology sectors, such as the aerospace industry, is mainly driven by the required performance of the CFs. For civilian applications such as general engineering and the automotive industry, however, the large-scale production of CFs is immensely limited by production costs. Numerous reports and reviews are available in the field of CFs and its precursors. Therefore, we focus on reviewing the less-discussed structure–property relationship and the influence of the different manufacturing processes on this relationship. Throughout this review, we identify areas that require future research and development regarding the morphology and structure formation of CFs from emerging precursor systems, e.g., lignin.