Abstract: Based on the contact and non-contact crack monitoring technology and driven by the health monitoring data of the masonry structure of ancient buildings, it is an important measure for preventive protection of ancient buildings to dig the effective information of monitoring data and analyze the crack characteristics. A pre-processing method was proposed for the health monitoring data of masonry structures of ancient buildings, and the proposed prediction model was used to identify anomalies in the real-time monitoring data and realize structural safety early warning. The periodic characteristics and influencing factors of the monitoring data of masonry structure crack opening and closing were explored by taking the monitoring data of masonry structure crack opening and closing and the monitoring data of field environment temperature and humidity inside the structure as the research objects. Considering that mural crack imaging features have strong interference characteristics compared with a single background crack, a monitoring technology for the growth and deformation of mural wall cracks in ancient Tibetan buildings was studied. Based on U-Net semantic segmentation model, an intelligent mural crack segmentation detection model was constructed which is robust to environmental interference. For large-size mural cracks, the image segmentation algorithm based on component tree SSR was further developed, and then the environmental factors affecting the monitoring system in the process of image acquisition were analyzed and tested. The performance of the monitoring system in practical application was tested and analyzed, and the feasibility of applying it to the long-term growth deformation monitoring of mural cracks in ancient buildings was verified.