Compared to terrestrial plants whose diversity is more directly influenced by climate, aquatic plant diversity is considered to be more dependent on water environments. Therefore, it could be predicted that the phylogenetic relatedness of terrestrial plants is more susceptible to climate filtering than that of aquatic plants. We compiled a comprehensive distribution dataset of herbaceous angiosperms in China, including both terrestrial and aquatic species. We compared the phylogenetic relatedness and its environmental correlation of the two groups, using the standardized effect size of phylogenetic diversity (PD_ses) and the standardized effect size of mean phylogenetic distance (MPD_ses), which reflect shallow and deep evolutionary histories, respectively. We also use the deviation of PD_ses (ΔPD_ses) and MPD_ses (ΔMPD_ses) between terrestrial and aquatic plants to reflect differences in the phylogenetic relatedness between terrestrial and aquatic plants. Our results showed that the geographical patterns of PD_ses and MPD_ses between aquatic and terrestrial plants are roughly consistent. ΔPD_ses and ΔMPD_ses between terrestrial and aquatic plants vary across the geographical scale and environmental gradient. Environmental variables (current climate, historical climate change and topography) explained more of the variation in PD_ses and MPD_ses of terrestrial plants than that of aquatic plants, with the current climate explaining more of ΔPD_ses and ΔMPD_ses between terrestrial and aquatic plants. Our results reveal the differential impacts of large-scale environmental factors on the phylogenetic relatedness of terrestrial versus aquatic plant communities, providing a new perspective for understanding the ecological and evolutionary dynamics of these two distinct plant assemblages.