The under-determined blind source separation(BSS) was employed in identifying modal parameters when there were fewer sensors than modal responses. The separation process was divided into two steps: first, estimation of the mixing matrix, and second, recovery of sources. A new method for estimating the mixing matrix was proposed that utilizes the scatter of the local energy maximum value in the frequency domain. Then the L1-norm minimum was used for recovering sources. Finally, frequency and damping ratios could be obtained from the modal responses using each single degree of freedom mode. The effectiveness of the proposed method was verified by theoretical analysis and simulation and experimental data, and the background noise was greatly suppressed.