Rural infrastructure has rather fixed users compared to urban infrastructure. This study evaluated the effect of rural infrastructure construction from the perspective of farmers. First, this study revised the American Customer Satisfaction Index (ACSI) model and selected a performance appraisal index for rural infrastructure based on this revised model. Then, the study adopted an interpretive structural model (ISM), analyzed the influence of each index factor, and developed a hierarchical directed graph. Finally, based on the mutual-influence relationships among the index factors in the hierarchical directed graph, a performance appraisal analytic network process (ANP) model was established. Based on discussions with rural college students and rural households in Sichuan, China, 246 questionnaires were obtained pertaining to rural infrastructure, and an empirical analysis was conducted. The results indicated that the performance of rural infrastructure construction is not very good. In particular, the full use of infrastructure and its role in improving the environment were found to be the worst. Meanwhile, the possibility of building information transparency and the longitudinal comparison of perceived performance appraisal results were the best. The performance of rural infrastructure construction was evaluated based on the perceptions of the direct users of rural infrastructure, and the relationship between the factors and the weight was measured reasonably. The proposed method was found to be workable and the analysis results reliable and effective.