State-of-the-art implementations of Trusted Execution Environments (TEEs) present system designers and users with several problems: First, it is not possible to choose a TEE implementation independently from the instruction set architecture. Second, the security-critical functionality of such TEEs is deeply integrated into the micro-architecture of complex processor cores, making programs running in such TEEs vulnerable to side-channel attacks. And third, the interface and execution model of certain types of TEEs make it hard to integrate these TEEs with the system software. To address these issues, we propose a modular TEE design. We apply this modular design to the M3 hardware/software co-design platform and demonstrate how TEE support can be made a first-class feature at the system-architecture level.