Aiming at the problem that the biaxial geomagnetic sensor is easy to be interfered by the external error factors in the system of biaxial geomagnetic sensor and satellite combined measurement of projectile attitude， the paper analyzes the error factors that affect the measurement accuracy of the geomagnetic sensor， and proposes a parameter calibration and error compensation algorithm based on elliptical rotation. Firstly， the virtual x-axis is assumed to exist in the biaxial geomagnetic sensor， and the error factor of the sensor is modeled to obtain the error compensation model. Next the parameters in the output expression of the sensor are calibrated. Then the ellipse formed by the measured values of the biaxial is rotated. Finally， six parameters in the error compensation model are calculated. Through the analysis of the experimental data of the three-axis non-magnetic turntable， this method can control the error accuracy of the roll angle of the biaxial geomagnetic sensor within ±2° and basically meet the requirements of the projectile attitude measurement.