The CRISPR-Cas9 systems are bacterial encoded adaptive immune systems to defend against phages infection, through RNA-guided endonuclease activity of Cas9 to degrade double-stranded DNA bearing a protospacer adjacent motif (PAM) and complementary sequences to the guide RNA^1,2,3,4,5. Recently, two anti-CRISPR proteins AcrIIA2 and AcrIIA4 from Listeria monocytogenes(Lmo) prophages have been identified, both of which potently inhibit Streptococcus pyogenes Cas9 (SpyCas9) and LmoCas9 activity in bacteria and human cells⁶. However, the mechanism of AcrIIA2- or AcrIIA4-mediated Cas9 inhibition remains unknown. Here we report a crystal structure of SpyCas9 in complex with a single guide RNA (sgRNA) and AcrIIA4. Our data show that AcrIIA2 and AcrIIA4 interact with SpyCas9 in a sgRNA-dependent manner. The structure reveals that AcrIIA4 inhibits SpyCas9 activity through structurally mimicking PAM to occupy the PAM-interacting site in PI domain, thereby blocking SpyCas9 recognition of dsDNA substrates. AcrIIA4 further inhibits the endonuclease activity of SpyCas9 by shielding its RuvC active site. Structural comparison reveals that formation of the AcrIIA4 binding site of SpyCas9 is induced by sgRNA binding. Our study reveals the mechanism of SpyCas9 inhibition by AcrIIA4, laying a structural basis for developing "off-switch" tools of SpyCas9 to avoid unwanted genome edits within cells and tissues.