The market for nonvolatile memory devices is growing rapidly. Today, the vast majority of nonvolatile memory devices are based on the floating gate device which is facing serious scaling limitations. Material innovations currently under investigation to extend the scalability of floating gate devices are discussed. An alternative path is to replace the floating gate by a charge trapping material. The combination of charge trapping and localized channel hot electron injection allows storing two physically separated bits in one memory cell. The current status and prospects of charge trapping devices are reviewed, demonstrating their superior scalability. Floating gate as well as charge trapping memory cells suffer from severe performance limitations with respect to write and erase speed and endurance driving system overhead. A memory that works like random access memory and is nonvolatile would simplify system design. This, however, calls for new switching effects that are based on integrating new materials into the memory cell. An outlook to memory concepts that use ferroelectric switching, magnetic switching, phase change, or other resistive switching effects is given, illustrating how the integration of new materials may solve the limitations of today's semiconductor memory concepts.