Abstract: A new human 3D model-based 2D action recognition algorithm is presented in this paper for adapting to the actor's variable orientates. The actor is represented by 3D occupancy grids and the human joints extracted from the 3D actor are used as the feature of action. We learn separate exemplar-based hidden Markov models (EHMM) for each action class. In addition, some frames of the 3D actors' samples are chosen as the 3D key pose set. This set is a bridge linking 2D observation action with 3D human joints feature. The 3D reconstruction is not required during the 2D action recognition phase. 2D observation sequences are collected from single/multiple un-calibrated cameras. Firstly, a 3D key pose sequence, which is the most similar to the 2D observation sequence, is selected from the 3D key pose set. Then, the 3D key pose sequence is classified using the action classifier. The effectiveness of the proposed algorithm is demonstrated with experiments on three action datasets. The results prove robustness of the proposed method with respect to the actor's orientates and camera configurations.