Abstract: The hull plates inevitably have different holes, which reduce the continuity of the structure and have an important influence on the strength and stability of the structure. It is important to study the buckling performance of plates with holes to ensure the safety of ships. Under in-plane loads, the shear buckling tests are carried out on two types of thin plates with different holes using a picture frame fixture and a 3D full-field deformation measurement-analysis system. The critical buckling load, full-field displacement/strain information and load-elongation curves for the plate with circular hole and the plate with square hole are obtained. Considering the influence of test fixture, numerical simulation of thin plates with different holes is carried out based on Abaqus. And the eigenvalue buckling analysis and nonlinear buckling analysis are carried out to obtain the mechanical response information of ship plates under buckling and post-buckling. The effectiveness and accuracy of the numerical method are verified by comparing the numerical results with the experimental results. Then the load-elongation relationship, full-field displacement, critical buckling load and stress distribution characteristics at the edge of the hole are analyzed, and the influence of circular hole and filleted square hole on the shear stability of the square plate under the condition of equivalent area is identified. The findings would provide reference for the experiment and simulation of in-plane shear stability of thin plates as well as design optimization and mechanical performance evaluation of large ship structures.