Have you considered following tmux's implementation of this feature? It has the benefit of being well-known, and it won't step on tmux's own implementation as long as the key combination to activate it doesn't coincide with tmux's

Here's a little guide that explains how that works:
http://www.rushiagr.com/blog/2016/06/16/everything-you-need-to-know-about-tmux-copy-pasting-ubuntu/

Originally posted by @fpqc in #2840

The main idea here (I think) is rather than holding the shift key to switch anchors, you just toggle which selection anchor you're moving.

Off the top of my head, I'm not sure how to make it not conflict with the existing implementation of mark mode, but I'm interested 😊