Skip to content

Support copy-paste in rich-text fields and fine-tune editing window#3446

Merged
ajrbyers merged 26 commits into
masterfrom
1481-bleach
Sep 18, 2023
Merged

Support copy-paste in rich-text fields and fine-tune editing window#3446
ajrbyers merged 26 commits into
masterfrom
1481-bleach

Conversation

@joemull

@joemull joemull commented Mar 22, 2023

Copy link
Copy Markdown
Member

Implementation notes

I opted to use django-bleach selectively on a range of rich-text fields. I created a default set of allowed tags and attributes in Janeway global settings, prioritising semantic markup.

I implemented it in several different ways, sometimes in the model, sometimes in the form, and sometimes with an option to turn it off:

  • In fields such as abstract, I just changed the model field because we do not want custom HTML in there
  • With fields like press-level journal description, I used it as a form field because there might be a case when someone wants to use custom markup via the admin interface
  • With CMS and news item content, I added a supports_copy_paste field that controls whether the content is bleached. Existing pages are set not to be bleached via a custom migration function, so that current styling data isn't lost. New pages are bleached by default but users can still turn it off.

I also took this opportunity to fine-tune the Summernote toolbar (so it matches the allowed tags) and add syntax highlighting and soft line wrapping for the codemirror window.

Drawbacks

I could not make this work for all rich-text fields with the time available, because some of them are governed by JQTE and we would need to standardize these to Summernote before fixing them. These include the custom HTML block homepage element, most notably.

Alternatives considered

The django-bleach project depends on bleach, which depends on html5lib, which has recently been deprecated. A possible replacement is under construction in ammonia, its Python binding nh3, and Mark Walker's Django helper django-nh3. The Django helper is too new to use immediately but it is written by the same author as django-bleach, so moving to it in a year or two might be a good idea.

Closes #1481
Closes #2282

@joemull joemull requested review from ajrbyers and mauromsl March 22, 2023 16:29
@joemull joemull linked an issue Mar 22, 2023 that may be closed by this pull request

@mauromsl mauromsl left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @joemull, this will be a very welcome new feature.
I added a few of comments inline.

Comment thread src/cms/forms.py Outdated


class PageForm(JanewayTranslationModelForm):
class PageForm(BleachableForm, JanewayTranslationModelForm):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The order of the inheritance would be correct in most cases, but this is an exception:

Since the MRO in python3 relies on C3 Linearization, this order will cause the methods of django.forms.Form to run before django.forms.ModelForm. Since this is a ModelForm, not only is this order wrong, we don't want to bring the execution path of django.forms.Form over at all.

There are two solutions:

  • Base the BleachableForm on the common ancestor of forms.Form and forms.ModelForm which is django.forms.BaseForm. You can then construct your forms as class Foo(BleachableForm, ModelForm) and class Bar(BleachableForm, forms.Form)
  • Use a Mixin that self contains the behaviour of bleach, and then create two differently branching form classes:
classs BleachableFormMixin(forms.BaseForm):
    def save(self, *args, **kwargs):
        [..]

class BleachableForm(BleachFormMixin, forms.Form): pass
class BleachableModelForm(BleachFormMixin, forms.ModelForm): pass

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this and explaining the solution--commit incoming

Comment thread src/cms/models.py Outdated
)
is_markdown = models.BooleanField(default=True)
edited = models.DateTimeField(auto_now=timezone.now)
support_copy_paste = models.BooleanField(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add this to a BleachModelMixin instead? Not only makes the implementation DRY, it also self-contains the logic into a model that self-documents the existance of the support_copy_paste attribute.

name='support_copy_paste',
field=models.BooleanField(default=True, help_text='Turn this on if copy-pasting content into rich-text fields from a word processor or using the toolbar to format text. Turn it off only if you are editing HTML and CSS using the code view.'),
),
migrations.RunPython(preserve_old_formatting, reverse_code=migrations.RunPython.noop),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One neat way of doing this without the custom logic is to set default=False on the AddField instruction above
followed by an AlterField instruction here that sets default=True.

your implementation is valid too though

]

# Which HTML attributes are allowed
BLEACH_ALLOWED_ATTRIBUTES = ['href', 'title', 'src', 'target']

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to support class as well to make use of custom classes provided by each theme's CSS

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I excluded class in order to remove cruft from Microsoft Word when bleaching. There is no way to do that unless we disallow class. As for site managers using custom styling, we've provided the "Support copy paste" boolean for them to turn off bleaching on a case-by-case basis. Is that OK with you?

Comment thread src/submission/models.py
Comment on lines +582 to +645
abstract = BleachField(
blank=True,
null=True,
help_text=_('Please avoid pasting content from word processors as they can add '
'unwanted styling to the abstract. You can retype the abstract '
'here or copy and paste it into notepad/a plain text editor before '
'pasting here.')
help_text=_('Copying and pasting from word processors is supported.'),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting this comment here since I can't point at untouched lines: recently it surfaced that we need to bleach the article title field, since while not a rich-text field, we still render it as safe. This turns the article title into a vector for JS injection

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I've changed the title field to a BleachField

@joemull

joemull commented May 23, 2023

Copy link
Copy Markdown
Member Author

We discussed user error prevention for this feature this morning, and here is some additional work we are going to do to CMS pages to make sure press managers can undo saves that implemented unwanted changes.

User Story

As a tech support person, I might implement custom CSS or advanced HTML for a press website, but I might be the only one who knows that code is saved in the CMS.

Then, as a press manager who does not know about HTML and CSS, I may not know that the bleach boolean could wipe out the custom styling my tech support person put in. I might edit a page by pasting in a sentence or two, and then check “Support copy paste” because I copy-pasted. This would result in the previous custom styling work being wiped out.

We should give users a way to undo the save that caused the unwanted result.

Solution: page versions for CMS

  1. Add field backup_for, ForeignKey to self
  2. Modify save method to create new backup for self, and delete old backups. Logic: keep last 5 today’s saves, keep last 5 saves from previous days, and delete all others. Use field edited as timestamp.
  3. Add method backups, which fetches all the CMS pages that point to self
  4. Add method restore_backup, which returns error if it is not a backup, or otherwise fetches all the sister backups and the main version and points their backup_for field at self
  5. Some template text or buttons showing the timestamps of backups that exist for a given page, with anchor links view them. Some template text that warns if the current page is a backup (“You are viewing a previous version of [link to current version]”), and disables the Save button, replacing it with a Restore button. On clicking restore, you’d get “Are you sure you wish to restore this version?” If you click yes, the page is restored and you are taken to the page for that version as the main version.
  6. Make sure the edited field is updated on save
  7. Add an active_objects manager to filter out pages with something in backup_for
  8. Test

@joemull

joemull commented May 23, 2023

Copy link
Copy Markdown
Member Author

@joemull

joemull commented May 24, 2023

Copy link
Copy Markdown
Member Author

This also looks good: https://django-simple-history.readthedocs.io/en/latest/ Thanks for suggesting it @mauromsl.

For future reference, I've just tried implementing django-simple-history, and I encountered an error once I had configured it. It has to do with compatibility with django-modeltranslation: https://django-simple-history.readthedocs.io/en/latest/common_issues.html?highlight=translation#usage-with-django-modeltranslation

UPDATE: I got around the translation issue, sort of, but the resulting migrations are quite messy and redundant. You can see what I mean on this work-in-progress branch: 1481-bleach...1481-bleach-django-simple-history

Which way should we implement this for the 1.5.1 release?

@joemull

joemull commented May 25, 2023

Copy link
Copy Markdown
Member Author

By using django-admin-history, we give users a pathway to revert unwanted changes.

A new button appears beside the Save button on the forms for Page, NewsItem, and Repository, since those are the three that are now optionally bleachable.
Screenshot from 2023-05-25 14-41-48

Staff members can follow direct links to the Django admin page for a version, where they have the option to revert it:
Screenshot from 2023-05-25 14-38-34

Editors are not given admin links but are instead prompted to ask staff for help reverting a past version:
Screenshot from 2023-05-25 14-39-40

@joemull joemull requested a review from mauromsl May 25, 2023 15:15
@joemull

joemull commented Aug 11, 2023

Copy link
Copy Markdown
Member Author

This one has got a really long comment history, but FYI @ajrbyers and @mauromsl, it is ready to review. The changes you requested @mauromsl have been made.

@joemull

joemull commented Sep 18, 2023

Copy link
Copy Markdown
Member Author

@ajrbyers I've rebased this onto 954e4e8 to resolve conflicts, so it should be ready to merge as soon as Jenkins approves it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Junk code in abstract field breaks metadata modal Bleach Auto-remove erroneous formatting for abstracts in metadata

3 participants