@maximearmstrong this is mostly ready for review and I'd love your feedback if you have cycles.

It isn't fully ready to review/commit because I haven't updated the comment_generator.py script yet. That said, I wanted to propose a secondary refactor there as well. The outputcomparator is already producing a block of text to be logged by the output comparator that seems pretty similar to what's being generated by the comment_generator.py script. Maybe we could have outputcomparator just save its log lines to a file and use that in the PR comment, scraping the script entirely?

Maybe we could have output comparator just save its log lines to a file and use that in the PR comment, scraping the script entirely?

I agree. I think it would also be better from a design standpoint if the output comparator took on that responsibility.

I've updated the comment generation code, as discussed. This is ready for final review.

Thanks for working on this @bdferris-v2 !
I have added a comment above. We might need to update ACCEPTANCE_TESTS.md to reflect the changes in this PR.

What is going to be the schema of the new report with these changes?

I've approved the PR. Thanks @bdferris-v2!

We might need to update ACCEPTANCE_TESTS.md to reflect the changes in this PR.

What is going to be the schema of the new report with these changes?

@isabelle-dr do we need clarity on these items before merging?

Thanks Maxime, I think it is in the scope of this PR to update ACCEPTANCE_TESTS.md with the changes (addition of dropped errors, report schema change).

@isabelle-dr I updated the ACCEPTANCE_TESTS.md to reflect the changes in this PR.