Authentication schemes using tokens or biometric modalities have been proposed to ameliorate the security strength on mobile devices. However, the existing approaches are obtrusive since the user is required to perform explicit gestures in order to be authenticated. While the gait signal captured by inertial sensors is understood to be a reliable profile for effective implicit authentication, recent studies have been conducted in ideal conditions and might therefore be inapplicable in the real mobile context. Particularly, the acquiring sensor is always fixed to a specific position and orientation. This paper mainly focuses on addressing the instability of sensor’s orientation which mostly happens in the reality. A flexible solution taking advantages of available sensors on mobile devices which can help to handle this problem is presented. Moreover, a novel gait recognition method utilizes statistical analysis and supervised learning to adapt itself to the instability of the biometric gait under various circumstances is also proposed. By adopting PCA+SVM to construct the gait model, the proposed method outperformed other state-of-the-art studies, with an equal error rate of 2.45% and accuracy rate of 99.14% in terms of the verification and identification aspects being achieved, respectively.