The track alignment detection is a key technology to ensure the safety of train operation. Due to the cumulative error of the gyroscope and the accelerometer, the track linear detection method based on the conventional inertial measurement has low accuracy under the low-speed. In order to solve this problem, a new detection method for track linear based on multi-sensors fusion of machine vision and inertial measurement is proposed. To achieve multi-sensor fusion calibration, the conversion matrixes of inertial measurement unit and machine vision, inclinometer and inertia measurement unit and translation relationship between inertial measurement unit and machine vision are established, respectively. Machine vision and inertia information fusion is given by extended Kalman filter. Afterwards, the detection performance of the proposed method is investigated by six degrees of freedom platform and experimental line. And the results demonstrate that the measurement accuracy of the new method is less than 0.5mm. Compared with the conventional inertial measurement method, it is found that the measurement accuracy efficiency of the proposed method has improved nearly 10 times.