Performance with Large Numbers of Signal Plots

I need to plot a large set of signals, up to some hundreds of each several thousands samples

To make this conversation easier let's come-up with some hard numbers to use as a test case to assess performance:

• I'll test 100 signal plots each with 10,000 points.
• Every signal plot will "grow" with time, up to a maximum limit of 100,000 total points per signal plot.
• Data will be updated 10 times per second (every 100ms) using a timer adding 100 points per tick
• I'll use WinForms where renders block the GUI thread

The technique I'm using is described here: https://scottplot.net/faq/live-data/#growing-data-with-partial-array-rendering

This feels surprisingly responsive to me. Performance could be improved by updating the plot once per second (forcing fewer required redraws)

public partial class Form1 : Form
{
    readonly Random Rand = new();
    readonly List<ScottPlot.Plottable.SignalPlot> SignalPlots = new();

    public Form1()
    {
        InitializeComponent();

        int signalCount = 100;
        int maxPointsPerSignal = 100_000;
        int startingPointCount = 10_000;

        for (int i = 0; i < signalCount; i++)
        {
            System.Diagnostics.Debug.WriteLine($"Pre-populating data for signal {i + 1} of {signalCount}...");
            double[] ys = ScottPlot.DataGen.RandomWalk(Rand, maxPointsPerSignal, offset: Rand.NextDouble() * 1000);
            ScottPlot.Plottable.SignalPlot sig = formsPlot1.Plot.AddSignal(ys);
            sig.MaxRenderIndex = startingPointCount;
            SignalPlots.Add(sig);
        }

        formsPlot1.Plot.Benchmark(true);
        formsPlot1.Refresh();
    }

    private void timer1_Tick(object sender, EventArgs e)
    {
        foreach (var sig in SignalPlots)
        {
            // In this example we pre-populated the sig.Ys[] array with random data, but in practice
            // the array would be all zeros and updated by this method as new data comes in.
            sig.MaxRenderIndex += 100;
        }

        formsPlot1.Refresh();
    }
}

WARNING: since the maximum size of the data was set to 100k points, and this starts at 10k and grows by 1000 points a second (gorinw by 100 ten times a second), it reaches the end of the array in 90 seconds. The maximum array size can be adjusted as needed depending on your application.

As my signals are not changed, contrarily to the overlays, I'd like to re-render only the overlays when a new data analysis is performed, not the underlying data.

Because of the way the rendering system works, this isn't really possible with the current system. ScottPlot actually has a "console first" design, and by that I mean it only generates full-size flat-file bitmaps as its primary rendering function. The mouse-interactive user controls are actually a bit of a hack on to of this which looks at mouse actions (click/drag) and quickly re-renders a whole new bitmap and puts it on a picturebox. They're not really smart enough to do partial image re-renders, and its unlikely the library will be restructured toward this design. Good idea though! If ScottPlot were only a WinForms or WPF control, this design would make a lot of sense.

Legend with Large Numbers of Signal Plots

I am very much thinking about Matlab figures as an example.

I'm only somewhat familiar with Matlab. A screenshot or animation demonstrating your suggestion may be helpful.

[is it possible] to know which plottable is under the mouse ?
Technically yes, but this means checking if each data point is within X pixels of the cursor, which is extremely non-performant because (a) it has to iterate every one of thousands or millions of data points and (b) it has to ask this question every time the mouse moves a single pixel

[is it possible] to interact with the legend to show or hide plots ?

This suggestion came-up recently #1482 - I triaged it so I could focus on some other work, but it seems possible to implement. It may not be useful if you have 100s of plots, but for a smaller number it seems totally doable.

[is it possible] to plot the legend in a separate area, detached from the plot

Indeed! I should add that to the https://scottplot.net/faq (If you're inspired to make it a page, you can add to the website by making a PR here https://github.com/ScottPlot/ScottPlot.NET)

Basically Plot.GetLegendBitmap() returns a Bitmap of the legend you can display in a PictureBox. One caveat is that you must Render() the plot or Refresh() the control at least once before the legend is available.

[is it possible] to have a scrollable legend so that it does not cover a large portion of the plot if numerous signals are plotted

Another good question! I probably won't implement this in ScottPlot itself (scrollable doesn't really make sense in the "console-first" design), but now that you can get a legend as a Bitmap you could place it in a panel that has a scrollbar. Perhaps that would be a better place to put your custom "click to show/hide" logic anyhow? That picture box could respond to OnClick events, determine the coordinates of what was clicked, figure out which plottable that is, and show/hide it.

Plot.Legend() actually returns the legend, and there's no method like this but if you'd find it helpful you're welcome to add it and open a PR for it to let people do something like this:

// in the init method
var myLegend = formsPlot1.Plot.Legend();

// in the OnClick method
ScottPlot.Plottable.IPlottable clickedPlottable = myLegend.GetPlottableUnderMouse(x, y);
clickedPlottable.IsVisible = !clickedPlottable.IsVisible;
formsPlot1.Refresh()

That's a lot of info in a big answer, but I hope it helps! 🚀