Embryonic stem cell (ESC) pluripotency requires bivalent epigenetic modifications of key developmental genes regulated by various transcription factors and chromatin-modifying enzymes. How these factors coordinate with one another to maintain the bivalent chromatin state so that ESCs can undergo rapid self-renewal while retaining pluripotency is poorly understood. We report that Utf1, a target of Oct4 and Sox2, is a bivalent chromatin component that buffers poised states of bivalent genes. By limiting PRC2 loading and histone 3 lysine-27 trimethylation, Utf1 sets proper activation thresholds for bivalent genes. It also promotes nuclear tagging of messenger RNAs (mRNAs) transcribed from insufficiently silenced bivalent genes for cytoplasmic degradation through mRNA decapping. These opposing functions of Utf1 promote coordinated differentiation. The mRNA degradation function also ensures rapid cell proliferation by blocking the Myc-Arf feedback control. Thus, Utf1 couples the core pluripotency factors with Myc and PRC2 networks to promote the pluripotency and proliferation of ESCs.