Abstract: Based on the plastic theory with internal variables, the homogenization technique has been applied into the rock engineering to present the deformation and failure of rock, The BEM is developed to simulate the progressive failure of rock engineering. The method can estimate the effects of the progressive failure, no-tension and dilation of rock mass. Some examples are solved to illustrate the efficiency of the boundary element method and its application to rock engineering.