We propose a linear-combination-of-pseudo-atomic-orbitals scheme for a finite electric field method based on the modern theory of polarization. We derive the matrix elements of the effective potential for the static and homogeneous field and the corresponding terms of the forces on atoms. In addition, we successfully evaluated the static and electronic dielectric constants and Born effective charges of typical semiconducting and insulating materials. Our formalism will aid in the study of materials under electric fields.