The flowback behavior of hydraulic fractured horizontal well in shale gas reservoir is relatively different from that of conventional reservoirs. Therefore, it is necessary to investigate the relationship between the potential influencing factors and the flowback behavior in shale gas reservoirs. This study is based on experimental observations and numerical simulations. In the experiments, the flowback process was simulated through a gas displacement experiment, and the cores were scanned simultaneously to obtain the water distribution. Then, the water migration and retention mechanisms were investigated to determine the flowback behavior. For the numerical simulations, a multi-porosity model was established. The mathematical model accounted for the capillary pressure term. By matching the fluid saturation-front curves of the experimental and simulation results, a fitted capillary pressure curve, which reflects the multiple mechanisms controlling flowback, was obtained. Based on the established model and fitted capillary pressure, the flowback behavior and relevant influencing factors of the shale gas were investigated. The results show that the flowback ratio is inversely proportional to the clay content of the shale. A high salinity fracturing fluid or a surfactant solution can increase the flowback ratio. In addition, the injection pressure is proportional to the flowback ratio, while the matrix permeability and the flowback ratio have an inverse relationship. The adsorption–desorption process of gas has no significant effect on the flowback ratio. This study aims to provide a new method for analyzing the flowback performance of shale gas using a combination of experimental and numerical simulation methods.