Ecosystem models often perform poorly in reproducing interannual variability in carbon and water fluxes, resulting in considerable uncertainty when estimating the land-carbon sink. While many aggregated variables (growing season length, seasonal precipitation, or temperature) have been suggested as predictors for interannual variability in carbon fluxes, their explanatory power is limited and uncertainties remain as to their relative contributions. Recent results show that the annual count of hours where evapotranspiration (ET) is larger than its 95th percentile is strongly correlated with the annual variability of ET and gross primary production (GPP) in an ecosystem model. This suggests that the occurrence of favorable conditions has a strong influence on the annual carbon budget. Here we analyzed data from eight forest sites of the AmeriFlux network with at least 7 years of continuous measurements. We show that for ET and the carbon fluxes GPP, ecosystem respiration (RE), and net ecosystem production, counting the “most active hours/days” (i.e., hours/days when the flux exceeds a high percentile) correlates well with the respective annual sums, with correlation coefficients generally larger than 0.8. Phenological transitions have much weaker explanatory power. By exploiting the relationship between most active hours and interannual variability, we classify hours as most active or less active and largely explain interannual variability in ecosystem fluxes, particularly for GPP and RE. Our results suggest that a better understanding and modeling of the occurrence of large values in high-frequency ecosystem fluxes will result in a better understanding of interannual variability of these fluxes.