The fractional derivative model can effectively represent the mechanical behavior of the viscoelastic material in the time and frequency domain. The identification of fractional derivative model is of great importance for viscoelastic composite structure modeling as well as for damping reduction design. A method for identifying the fractional derivative model of viscoelastic materials based on sweeping response of composite structure was proposed. On the basis of considering both the viscoelastic material damping and the remaining equivalent viscous damping, the dynamic equation of viscoelastic composite plate under basement excitation is established and the method for solving the vibration response is also presented. Furthermore, a matching calculation method based on sensitivity is developed to identify the parameters of the fractional derivative model. The cantilevered titanium plate attached with ZN-1 viscoelastic material is chosen to display the proposed method and the parameters of fractional derivative model of this material are obtained. The identified parameters are substituted into the analysis model of the viscoelastic damping plate and the rationality of the identified results is verified by comparing the vibration responses obtained by theoretical calculation and experiments.