While the response properties of the piezoelectric force sensor in a high frequency band were good, its response properties in a low frequency band were poor. Therefore, in a low frequency band, the testing result of a piezoelectric micro-vibration testing platform consisting of piezoelectric force sensors was unreliable. A compensation method was proposed to solve this problem. This method was completed in a frequency domain, which was the post-processing of the output signal of the piezoelectric force sensor. The output signal of the sensor after compensation should be the product of the output signal of the sensor before compensation and the compensation function, which was the function of the discharge time constant of the piezoelectric force sensor. Taking a P5H piezoelectric ceramic piece as a piezoelectric force sensor, the method was verified by comparing the known forces with the testing results before and after compensation under the action of two different kinds of known forces, which were step force and harmonic force in a low frequency band. The results showed that this method can effectively compensate the response properties of a piezoelectric force sensor in low frequency band.