I tried deploying the latest state of the plugin to a dev environment for blog.amp.dev, and something surprising resulted:

This is due to the site's theme having this style:

.page-content>*, .entry-content>*, .entry-summary>* {
    display: block;
    margin-right: auto;
    margin-left: auto;
    padding-right: 1.5rem;
    padding-left: 1.5rem;
    max-width: calc(45 * 1rem);
}

To help mitigate this, I think we perhaps need to inject the link and style elements into the pre tag as opposed to preceding that tag.