Complex genome organizations participate in various nuclear processes including transcription, DNA replication, and repair. However, the mechanisms that generate and regulate these functional genome structures remain largely unknown. Here, we describe how the Ku heterodimer complex, which functions in nonhomologous end joining, mediates clustering of long terminal repeat retrotransposons at centromeres in fission yeast. We demonstrate that the CENP-B subunit, Abp1, functions as a recruiter of the Ku complex, which in turn loads the genome-organizing machinery condensin to retrotransposons. Intriguingly, histone H3 lysine 56 (H3K56) acetylation, which functions in DNA replication and repair, interferes with Ku localization at retrotransposons without disrupting Abp1 localization and, as a consequence, dissociates condensin from retrotransposons. This dissociation releases condensin-mediated genomic associations during S phase and upon DNA damage. ATR (ATM- and Rad3-related) kinase mediates the DNA damage response of condensin-mediated genome organization. Our study describes a function of H3K56 acetylation that neutralizes condensin-mediated genome organization.