Colorectal cancer (CRC) is a common and lethal disease with a high therapeutic need. For most patients with metastatic CRC, chemotherapy is the only viable option. Currently, immunotherapy is restricted to the particular genetic subgroup of mismatch-repair deficient (MMRd)/microsatellite instable (MSI) CRC. Anti-PD1 therapy was recently FDA-approved as a second-line treatment in this subgroup. However, in a metastatic setting, these MMRd/MSI tumors are vastly outnumbered by mismatch-repair proficient (MMRp)/microsatellite stable (MSS) tumors. These MMRp/MSS tumors do not meaningfully respond to any traditional immunotherapy approach including checkpoint blockade, adoptive cell transfer and vaccination. This resistance to immunotherapy is due to a complex tumor microenvironment that counteracts antitumor immunity through a combination of poorly antigenic tumor cells and an immunosuppressive tumor microenvironment. To find ways of overcoming immunotherapy resistance in the majority of CRC patients, it is necessary to analyze the immunological makeup in an in-depth and personalized way and in the context of their tumor genetic makeup. Flexible, biomarker-guided early-phase immunotherapy trials are needed to optimize this workflow. In this review, we detail key mechanisms for immune evasion and emerging immune biomarkers for personalized immunotherapy in CRC. Also, we present a template for biomarker-guided clinical trials that are needed to move new immunotherapy approaches closer to clinical application.