Abstract: Obtaining health monitoring data for ancient masonry structures through both contact and non-contact crack monitoring techniques, and deeply extracting valuable information from these data to analyze crack characteristics, holds significant importance for the structural health monitoring of ancient buildings. This paper proposes an anomaly detection method for health monitoring data of ancient masonry structures, investigates predictive models, and assesses the anomaly probabilities of real-time monitoring data based on mathematical methods. The periodic characteristics and influencing factors of the monitoring data of masonry structure crack opening and closing were explored by taking the monitoring data of masonry structure crack opening and closing and the monitoring data of field environment temperature and humidity inside the structure as the research objects. Considering that mural crack imaging features have strong interference characteristics compared with a single background crack, a monitoring technology for the growth and deformation of mural wall cracks in ancient Tibetan buildings was studied. Based on U-Net semantic segmentation model, an intelligent mural crack segmentation detection model was constructed which is robust to environmental interference. For large-size mural cracks, the image segmentation algorithm based on component tree SSR was further developed, and then the environmental factors affecting the monitoring system in the process of image acquisition were analyzed and tested. The performance of the monitoring system in practical application was tested and analyzed. The feasibility of the application of the monitoring system in the static and expanding state of the fracture is verified by simulating the fracture development.