Process render events with RenderQueue. by StendProg · Pull Request #813 · ScottPlot/ScottPlot

StendProg · 2021-02-21T19:21:11Z

Purpose:
Process all interactive events using RenderQueue. #761
Base concept:
There is 9 types of events:

BenchmarkToggle
ApplyZoomRectangle
MouseAutoAxis
MouseMovedToZoomRectangle
MousePan
MouseScroll
MouseupClearRender
MouseZoom
PlottableDrag

For each of these types, there is a corresponding class that contains the Action with "math" logic and the RenderType.

After an interactive event occurs, the corresponding class is added to the queue. The logic handler is guaranteed to be called for it, but rendering will happen only when the queue is cleared, i.e. after processing the last event.

The behavior of the Render after the last event is specified using the RenderType and can be unique for each event type.
There is 5 RenderTypes:

LQ only - Low quality Render will be called if this event is last in queue
HQ only - High quality Render` will be called if this event is last in queue
HQAfterLQImmediately - Low quality Render will be called if this event is last in queue, HQ after, if new events not added during LQ Render
HQAfterLQDelayed - the same as 3 but HQ called after delay, and if new event not added during this delay
None - Run only logic handler, if this event is last, use previous not None event behavior.

ControlBackend contain 2 fields:

EventProcessor eventProcessor - process events
IUIEventFactory eventFactory - create EventClasses to process.

RenderType for classes can be customise using UIEventFactoryDecorators.
There is 5 types build in, but their number can be easily expanded.

ClassicDecorator - tweaks the behavior of the events to match the previous rendering behavior.
ModernDecorator - The main difference is that delayed high-quality rendering starts when the interaction stops, even without releasing the mouse button.
LQOnlyDecorator - all rendering goes with low quality.
HQOnlyDecorator - all rendering goes with high quality (Performs very poorly, render events do not occur until interactions have ceased, because i's strongly out of the basic concept. But do you need to achieve it to work?).
AllDelayed - every event get HQ render after delay.

TODO:

Calling various Control, Plot events used to be tightly tied to Render, so now many Interactions will be skipped if rendering is not called for them. We need to call these events directly in the logic. Or is it not necessary?
Unit tests. It looks like the code turned out to be easy testable, but it takes time, I'm not sure when it will be.
Determine the desired behavior, and throw out unnecessary decorators. Or make it customizable through the control parameters.
Throw out debug output in EventProcessor. At the very last moment, it is very helpful in understanding what is going on.

New Functionality:
An example of setting different behaviors, in theory, the behavior can be changed on the fly (Not tested)

eventFactory = new ModernDecorator(new UIEventFactory(Configuration, Settings, Plot));
//eventFactory = new ClassicDecorator(new UIEventFactory(Configuration, Settings, Plot));
//eventFactory = new AllDelayedDecorator(new UIEventFactory(Configuration, Settings, Plot));

swharden · 2021-03-07T17:24:13Z

Hi @StendProg, thank you for this fantastic contribution!

There is a lot of code/architecture for me to study with this PR, so I waited until I had a lot of time to review it thoroughly. I am excited to work on this today, and I will start by adding XML documentation and renaming things if it would make them more easily understood. I'll keep my modifications limited to today so we don't get merge conflicts in case you want to keep working on this PR.

I'll follow-up shortly with comments addressing the TODOs/questions in your original message. Thanks again for this wonderful PR!

swharden · 2021-03-08T01:21:52Z

Hi @StendProg,

I noticed the Task.Delay() inside the queue processor loop negatively impacts performance when using the mouse to pan and zoom. I understand it must be there, but I am curious what you think about the best value to use. In your original PR it was 10, but if you set it to 100 you can see the slow behavior I am referring to. When I set it to 1 it runs much better on my computer, but I can't tell if that is because my computer is fast and perhaps this would be a problem for others. What do you think about this delay and what value it should be?

Thanks for your input!

ScottPlot/src/ScottPlot/Control/EventProcess/EventsProcessor.cs

Lines 129 to 171 in 07cc64e

    
           /// <summary> 
        
           /// Process every event in the queue. 
        
           /// A render will be executed after each event is processed. 
        
           /// A slight delay will be added between queue checks. 
        
           /// </summary> 
        
           private async Task RunQueueProcessor() 
        
           { 
        
               RenderType lastEventRenderType = RenderType.None; 
        
               while (true) 
        
               { 
        
                   QueueProcessorIsRunning = true; 
        
                   bool eventRenderRequested = false; 
        
                   while (Queue.Count > 0) 
        
                   { 
        
                       var uiEvent = Queue.Dequeue(); 
        
                       uiEvent.ProcessEvent(); 
        
                       if (uiEvent.RenderOrder == RenderType.HQAfterLQDelayed) 
        
                           RenderDelayTimer.Restart(); 
        
                       if (uiEvent.RenderOrder != RenderType.None) 
        
                       { 
        
                           lastEventRenderType = uiEvent.RenderOrder; 
        
                           eventRenderRequested = true; 
        
                       } 
        
                   } 
        
                   if (eventRenderRequested) 
        
                       RenderPreview(lastEventRenderType); 
        
                   await Task.Delay(1); 
        
                   // if all events were processed, consider performing a final render and exiting 
        
                   if (Queue.Count == 0) 
        
                   { 
        
                       bool finalRenderExecuted = RenderFinal(lastEventRenderType); 
        
                       if (finalRenderExecuted) 
        
                           break; 
        
                   } 
        
               }; 
        
               QueueProcessorIsRunning = false; 
        
           }

StendProg · 2021-03-09T16:14:55Z

Hi @swharden,
Sorry that the work is not finished yet, I just can't find free time, there is clearly more than a couple of hours...

I noticed the Task.Delay() inside the queue processor loop negatively impacts performance when using the mouse to pan and zoom. I understand it must be there, but I am curious what you think about the best value to use. In your original PR it was 10, but if you set it to 100 you can see the slow behavior I am referring to. When I set it to 1 it runs much better on my computer, but I can't tell if that is because my computer is fast and perhaps this would be a problem for others. What do you think about this delay and what value it should be?

This is a simple analogue of DoEvents, but for asynchronous methods. It simply transfers control to the main thread, which will update the UI and fill the queue as needed.
I think you can safely put 1 ms. A large delay is needed only in order to reduce the load on the processor once again when the queue is empty and a high-quality render timer is waiting. In addition, if the queue is empty, this thread will end completely after the delayed rendering, and this delay will not have any effect during idle.
Also, delays of less than 30ms are not guaranteed for Windows. so 1ms one way or another can turn into 30 automatically, and maybe more everything depends on the workload of processes.

This commit simplifies render quality configuration by moving render quality state out of individual events and placing it in the control Configuration class. This commit replaces the decorated event factory, removes the interface from the factory, and also changes most class fields to be readonly.

swharden · 2021-03-13T18:02:46Z

Sorry that the work is not finished yet, I just can't find free time

No problem! I appreciate your contribution so much, and this is already an excellent improvement.

I want to keep moving forward with regular releases and avoid long-running PRs, so I'm going to work on this a lot today and merge it in today or tomorrow. I'll continue to refactor it to reduce complexity, refine language, and add documentation.

RenderLowThenHighQuality() has been added to controls. Resizing now uses the render queue to trigger LQ/HQ renders. Resizing produces flickering, but no longer blocks the UI thread. XML documentation has been added to public methods/fields in controls and the control back-end.

This commit fixes a bug where mouse events were getting sent to the queue for processing too early, and the host control did not have time to wire-up the event handler before processing them. This new strategy uses a flag to hold the event processor in an infinite loop until the host control calls Backend.StartProcessingEvents()

Previously bitmaps were displayed in the control when new bitmaps were created. This causes a problem where unfinished bitmaps are rendered to the screen. This problem wasn't observed previously because render events were UI blocking, but now that they're decoupled resizing causes flickering. This commit fixes this problem by moving the UpdateBitmap() call to inside the render loop, so it's only called after the new bitmap is rendered upon. This commit also improves behavior such that resize events trigger low-quality initial renders followed by a delayed high-quality render.

prevent duplicate low quality render

bring into consistency with FormsPlot

This lets developers enable/disable the experimental queue-based rendering system. This is off by default and the traditional (UI-blocking) render system is used. Delayed high quality render following the mouse wheel is no longer supported with the traditional rendering system.

swharden · 2021-03-14T22:51:44Z

@StendProg, thank you again for this excellent PR! I worked on it extensively and will merge it in and release a NuGet package today. Here are some important points:

Summary of the New Render System

When GUI events occur (mouse clicks, mouse drags to pan/zoom/drag, mouse moves which may require a cursor change, or window resize events) the user controls call a function in the backend module named according to that interaction. The backend module then constructs an event (using a factory for each event type) and decides how to process it.

If Configuration.UseRenderQueue is false, the event is processed immediately and a render is performed. This a thread-blocking process, so intensive rendering tasks can make the UI feel slow.

If Configuration.UseRenderQueue is true, the event is added to the queue of an event processor (stored in the control itself). Adding the event to the queue is a non-blocking task, so even if rendering is intensive the UI will remain interactive. Adding the event will start a thread for a queue processor which is responsible for executing the event, rendering the result, and possibly remain in a loop to follow-up with a high quality re-render after a period of inactivity.

Right now the render queue is disabled by default.

Compare Queue vs. Traditional Rendering

The sandbox folder contains WinFormsFrameworkApp and WpfApp that make it easy to compare plots using the traditional vs. queue-based rendering method. I find the best test is a scatter plot with 1000 random points or more because this rendering task is slow.

I find it especially interesting to compare these methods paying close attention to click-drag pan/zoom responsiveness and also program responsiveness when resizing the window.

The queue method is faster when interacting with complex plots, but slower when interacting with simple plots.

WinForms	WPF

Next Steps / Room for Improvement?

Responsiveness: The render queue method is slightly (but noticeably) less responsive while panning/zooming simple plots than the original method. This is the main reason I don't want to enable it by default. Considering how threading is handled, I'm not sure if it is possible to improve this. I welcome any ideas for improvement! For complex/intensive plots, it's great to have this render queue system in place.

Documentation/API: I'll keep thinking about the best API to use here. It would be nice if the user had automatic and manual control of rendering. For example, when adding a complex dataset to a plot programmatically it would be nice if the user could trigger non-blocking renders (perhaps with an initial low quality render and an automatic high quality render afterword).

I may continue to refactor the queue system over the next few weeks to improve simplicity, features, and documentation. @StendProg if you're interested in working on this system more let me know and I'll stop working on it for a while to avoid merge conflicts. Thanks again for all your support here! This is a great improvement for this library!

StendProg · 2021-03-15T15:50:19Z

Hi @swharden,

Thank you for finishing this work, I was a little ashamed for leaving the unfinished for weeks, but after looking at what you did, I realized that only you could do it (added your video and greatly simplified it). I have several non-critical remarks, I will express them since I have already read your code, but all of them are essentially insignificant, as advice or reason to take note of for the future.

It's common to name public properties with a capital letter and private fields with a lowercase letter (even with an underscore, but I just can't get used to it). In any case, looking at the code, it is convenient to understand what area of responsibility this or that variable has, and this is easy to understand if you follow the rules described above. capitalized private fields are a little confusing, but this is a minor detail.

A readonly modifier protects only the object itself from changes, while the fields of this object may well be changeable.

public RenderType RenderType => Configuration.QualityConfiguration.MouseInteractiveDragged;

A property of this kind may look convenient, but it creates unnecessary dependencies in the code, and potentially there may be problems with logic and multithreading. The immutable class is preferable, there will definitely not be any problems with it. If you want to preserve the convenience of configuration, then you can copy the RenderType when creating the event object, and only then let it remain unchanged. (This is just a description of my initial motivation, in this particular case, I think there will be no problems with your option).

            while (Enable == false)
            {
                await Task.Delay(1);
            }

Looks like a big crutch. At first glance, you can unravel all this and run the initialization in the correct order, but I have not gotten to know this problem in detail, there may be some complications.

StendProg · 2021-03-15T16:03:46Z

The queue method is faster when interacting with complex plots, but slower when interacting with simple plots.

I had a guess why this is, and it's not about the additional overhead, but about a different drawing order. After running the demo application that you provided, I caught myself thinking that I could not catch the difference, no matter how I tried. I cannot understand what my problem is.

In any case, I will express my thoughts. Blocking approach after the arrival of the first event immediately starts Render, which can additionally block multiple events of changing the mouse cursor, instead of them one event with a large but correct coordinates delta will come.
RenderQueue, waits for events to finish arriving or are processed, and only then starts the first Render. As a result, the first render, by definition, will be later than in the blocking approach.
It would be possible to make this behavior customizable inside RenderQueue. Start render after the very first event, without waiting for the queue to clear. And to see what it will give, but I really cannot grasp the initial difference, which means I cannot check the result of such changes.

StendProg added 2 commits February 21, 2021 21:16

Process render events with RenderQueue.

72f1ed6

Fix formating

c824c20

swharden mentioned this pull request Mar 1, 2021

Triaged Tasks and Features (Feb 2021) #716

Closed

48 tasks

swharden added 2 commits March 7, 2021 12:11

XML docs: RenderType

3126ca6

XML docs: Control Events

7b2b5fe

swharden added 4 commits March 7, 2021 14:46

move EventType up one namespace level

cba06ba

refactor Control.EventsProcessor

c6bdee5

refactor Control.EventsProcessor

a695dc3

simplify Queue length checking

07cc64e

swharden added 13 commits March 13, 2021 13:46

improve control back-end documentation

af84227

set default MouseUp quality to delayed HQ

12ff08a

update sandbox app to test fast and slow plots

c03f84f

use .NET 5 autoformatter

f1e0bb2

improve Backend.RenderDelayedHighQuality()

67f82ed

prevent duplicate low quality render

WpfPlot.RenderLowThenImmediateHighQuality()

ea78b08

bring into consistency with FormsPlot

UseRenderQueue sandbox demo

83faf3d

improve configuration XML docs

b80c53e

UseRenderQueue WPF demo

aa19927

swharden merged commit bb622c0 into ScottPlot:master Mar 14, 2021

swharden mentioned this pull request Mar 15, 2021

Dot, Lollipop and Cleveland Dot Plots #842

Merged

swharden mentioned this pull request May 17, 2021

Triaged Tasks and Features #1028

Closed

82 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Process render events with RenderQueue.#813

Process render events with RenderQueue.#813
swharden merged 22 commits intoScottPlot:masterfrom
StendProg:ControlQueue

StendProg commented Feb 21, 2021 •

edited

Loading

Uh oh!

swharden commented Mar 7, 2021

Uh oh!

swharden commented Mar 8, 2021

Uh oh!

StendProg commented Mar 9, 2021 •

edited

Loading

Uh oh!

swharden commented Mar 13, 2021

Uh oh!

swharden commented Mar 14, 2021

Uh oh!

StendProg commented Mar 15, 2021

Uh oh!

StendProg commented Mar 15, 2021 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

StendProg commented Feb 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

swharden commented Mar 7, 2021

Uh oh!

swharden commented Mar 8, 2021

Uh oh!

StendProg commented Mar 9, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

swharden commented Mar 13, 2021

Uh oh!

swharden commented Mar 14, 2021

Summary of the New Render System

Compare Queue vs. Traditional Rendering

Next Steps / Room for Improvement?

Uh oh!

StendProg commented Mar 15, 2021

Uh oh!

StendProg commented Mar 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

StendProg commented Feb 21, 2021 •

edited

Loading

StendProg commented Mar 9, 2021 •

edited

Loading

StendProg commented Mar 15, 2021 •

edited

Loading