Hematopoietic stem cells (HSCs) maintain life-long production of immune cells and can directly respond to infection, but sustained effects on the immune response remain unclear. We show that acute immune stimulation with lipopolysaccharide (LPS) induced only transient changes in HSC abundance, composition, progeny, and gene expression, but persistent alterations in accessibility of specific myeloid lineage enhancers occurred, which increased responsiveness of associated immune genes to secondary stimulation. Functionally, this was associated with increased myelopoiesis of pre-exposed HSCs and improved innate immunity against the gram-negative bacterium P. aeruginosa. The accessible myeloid enhancers were enriched for C/EBPβ targets, and C/EBPβ deletion erased the long-term inscription of LPS-induced epigenetic marks and gene expression. Thus, short-term immune signaling can induce C/EBPβ-dependent chromatin accessibility, resulting in HSC-trained immunity, during secondary infection. This establishes a mechanism for how infection history can be epigenetically inscribed in HSCs as an integral memory function of innate immunity.