As an unfavorable factor of machining process, chatter threatens the machined quality of workpiece, which determines the assembly and fatigue performance of the workpiece. During the interaction between machine tool and thin-walled workpiece, the process damping effect, multiple modes response and dynamic changes caused by the material removal of the in-process workpiece (IPW) will make the machining process more complicated and introduce great difficulties to dynamic modeling and performance prediction. In this paper, we considered the process damping determined by the indentation volume between flank face of milling tool and machined surface, and used multi-mode model to describe this behavior. In order to establish the assembled material removal model of the IPW dynamics with multiple modes, the structure dynamic modification (SDM) and finite element method (FEM) were combined together. The updated third-order full discretization method was applied to solve the dynamic equation in modal space. Then, the three-dimensional stability lobe diagrams (SLDs) with and without material removal along the tool path were obtained respectively by enveloping multiple modes of the IPW and milling tool together. Finally, the cutting tests were carried out. The experiments showed that the assembled model could predict the dynamics of IPW accurately, and the proposed stability analysis model was relatively close to experimental results. Besides, the modes of thin-walled workpiece with weak rigidity do not always play a dominant role in the process of machining. Although the material removal rate is limited by considering the multiple modes of the system, the processing quality can be ensured.