Extensive changes in posttranslational histone modifications accompany the rewiring of the transcriptional program during stem cell differentiation. However, the mechanisms controlling the changes in specific chromatin modifications and their function during differentiation remain only poorly understood. We show that histone H2B monoubiquitination (H2Bub1) significantly increases during differentiation of human mesenchymal stem cells (hMSCs) and various lineage-committed precursor cells and in diverse organisms. Furthermore, the H2B ubiquitin ligase RNF40 is required for the induction of differentiation markers and transcriptional reprogramming of hMSCs. This function is dependent upon CDK9 and the WAC adaptor protein, which are required for H2B monoubiquitination. Finally, we show that RNF40 is required for the resolution of the H3K4me3/H3K27me3 bivalent poised state on lineage-specific genes during the transition from an inactive to an active chromatin conformation. Thus, these data indicate that H2Bub1 is required for maintaining multipotency of hMSCs and plays a central role in controlling stem cell differentiation.