Exploring the impact of climate factors on vegetation phenology is crucial to understanding climate–vegetation interactions as well as carbon and water cycles in ecosystems in the context of climate change. In this article, we extracted the vegetation phenology data from 2002 to 2021 based on the dynamic threshold method in the source region of the Yangtze and Yellow Rivers. Trend and correlation analyses were used to investigate the relationship between vegetation phenology and temperature, precipitation and their spatial evolution characteristics. The results showed that: (i) From 2002 to 2021, the multi-year average start of growing season (SOS), end of growing season (EOS) and length of growing season (LOS) for plants were concentrated in May, October and 4–6 months, with a trend of 4.9 days (earlier), 1.5 days (later), 6.3 days/10 a (longer), respectively. (ii) For every 100 m increase in elevation, SOS, EOS and LOS were correspondingly delayed by 1.8 days, advanced by 0.8 days and shortened by 2.6 days, respectively. (iii) The impacts of temperature and precipitation on vegetation phenology varied at different stages of vegetation growth. Influencing factors of spring phenology experienced a shift from temperature to precipitation, while autumn phenology experienced precipitation followed by temperature. (iv) The climate factors in the previous period significantly affected the vegetation phenology in the study area and the spatial variability was obvious. Specifically, the temperature in April significantly affected the spring phenology and precipitation in August widely affected the autumn phenology.