Human epidermal stem cells transit from a slow cycling to an actively proliferating state to contribute to homeostasis. Both stem cell states differ in their cell cycle profiles but must remain guarded from differentiation and senescence. Here we show that Cbx4, a Polycomb Repressive Complex 1 (PRC1)-associated protein, maintains human epidermal stem cells as slow-cycling and undifferentiated, while protecting them from senescence. Interestingly, abrogating the polycomb activity of Cbx4 impairs its antisenescent function without affecting stem cell differentiation, indicating that differentiation and senescence are independent processes in human epidermis. Conversely, Cbx4 inhibits stem cell activation and differentiation through its SUMO ligase activity. Global transcriptome and chromatin occupancy analyses indicate that Cbx4 regulates modulators of epidermal homeostasis and represses factors such as Ezh2, Dnmt1, and Bmi1 to prevent the active stem cell state. Our results suggest that distinct Polycomb complexes balance epidermal stem cell dormancy and activation, while continually preventing senescence and differentiation.