Skip to content

Experimental DataLogger2 using a CircularBuffer<T>#3946

Merged
swharden merged 8 commits intoScottPlot:mainfrom
drolevar:new_datalogger2
Jun 24, 2024
Merged

Experimental DataLogger2 using a CircularBuffer<T>#3946
swharden merged 8 commits intoScottPlot:mainfrom
drolevar:new_datalogger2

Conversation

@drolevar
Copy link
Contributor

There are two main goals behind this Datalogger:

  • Improve performance by bringing in binary search.
  • Support circular buffers, which is crucial when logging for an extended period of time.

@drolevar drolevar marked this pull request as draft June 14, 2024 16:07
@drolevar drolevar changed the title Implement a new DataLogger Implement a new DataLogger (WIP) Jun 14, 2024
@drolevar
Copy link
Contributor Author

@swharden Will be very nice to get your thoughts on that. The main reason for making this change for me personally was to limit memory consumption when running for very long periods. That's why I went on with adding CircularBuffer.

@swharden
Copy link
Member

Hi @drolevar, thanks for this PR! I'm starting to run out of time today and may have limited time tomorrow 😅 but this PR is definitely next up for me to review 👍

I took a quick look in my browser and this is looking great! I like how you created a dedicated CircularBuffer class too. Two quick thoughts before I give it a more thorough review...

  1. Where did you source CircularBuffer.cs from? It seems copy/pasted from something Microsofty but I'd love to include a URL to the exact source

  2. Perhaps we should incorporate this behavior into the DataStreamer class (which already has a hacky circular buffer) instead of creating a duplicate DataLogger for this purpose? I haven't studied this PR in detail yet so there may be an obvious reason this wasn't done, but I thought I'd mention it just in case

@swharden
Copy link
Member

Hi @drolevar, I haven't heard anything in a little while and this PR is still marked "draft", however it's looking pretty good so I'm going to take a few minutes now to modify it slightly and merge it in. If you had additional work you were planning to do to these classes we can start a new PR and take a closer look.

Thanks again for this new datalogger! This is looking fantastic 🚀

@swharden
Copy link
Member

Where did you source CircularBuffer.cs from? It seems copy/pasted from something Microsofty but I'd love to include a URL to the exact source

This looks close, but not identical
https://github.com/dotnet/aspire/blob/main/src/Shared/CircularBuffer.cs

@drolevar
Copy link
Contributor Author

@swharden

  • as for the CircularBuffer, it is where I took it, but I made certain modifications. I don't think we need insert functionality, and implementing it would require pulling System.Memory. If you think it's OK, it can be done.
  • I was a bit distracted by different tasks and couldn't answer properly, sorry. The reason I didn't use DataStreamer is that it didn't have the functionality I needed, however at the moment I don't remember what that was.
  • The only thing which is missing, is the support for the rotated data source. Currently the property is there, but it won't function properly.

@swharden swharden changed the title Implement a new DataLogger (WIP) Experimental DataLogger2 using a CircularBuffer<T> Jun 24, 2024
@swharden swharden marked this pull request as ready for review June 24, 2024 12:30
@swharden swharden enabled auto-merge (squash) June 24, 2024 12:30
@swharden swharden disabled auto-merge June 24, 2024 12:30
@swharden
Copy link
Member

Hi @drolevar, thanks for the follow up! I'll throw an exception when rotation is assigned to and we can add that functionality if it's desired in the future 👍

@swharden swharden enabled auto-merge (squash) June 24, 2024 12:34
@swharden swharden merged commit 4f3a402 into ScottPlot:main Jun 24, 2024
@drolevar
Copy link
Contributor Author

@swharden Do you have an idea of what has to be done do take it out of the experimental?

@swharden
Copy link
Member

@swharden Do you have an idea of what has to be done do take it out of the experimental?

Probably update the following sections:

To clarify how this new class is different from the existing ones. To be honest, I'm still uncertain about this myself.

If the new class can simply replace the existing DataStreamer, that's an even better option, as no changes to documentation would be required!

A few messages up you said "The reason I didn't use DataStreamer is that it didn't have the functionality I needed, however at the moment I don't remember what that was.", so maybe just clarifying that point will let us deprecate the old one and replace it with the new one

@jpgarza93
Copy link
Contributor

jpgarza93 commented Jun 24, 2024

This would be amazing on a DataLogger along with the nice ViewFull() and ViewSlide() functionality.

I also need this functionality. I currently achieve a circular buffer by calling

if (IsDataCountLargerThanBuffer && IsCircularBuffer)
{
    DataLogger.Data.Coordinates.RemoveAt(0);
}

but this is not the best approach and comes with some issues: #3969

KroMignon added a commit to KroMignon/ScottPlot that referenced this pull request Jun 26, 2024
* upstream/main:
  Fix interaction of axis panels when scale factor is more than 1 (ScottPlot#3994)
  Added ResetMinAndMaxValues() to DataLoggerSource.cs (ScottPlot#3993)
  CoordinateLine: add constructor overloads (ScottPlot#3987)
  Colormap.GetColors() (ScottPlot#3983)
  Added a constructor overload that accepts List<Coordinates> (ScottPlot#3982)
  Signal: improve support for IReadOnlyList<T> (ScottPlot#3978)
  Axes: improve sharpness of axis lines, tick marks, and grid lines (ScottPlot#3976)
  adding console write file name function (ScottPlot#3965)
  Color.ToColor()
  Sandbox: extend minimal API
  Sandbox: Create .NET API project
  SVG XML Updates (ScottPlot#3957)
  Repeat render if changes are made in invoked events (ScottPlot#3952)
  CI: autoformat
  Experimental DataLogger2 using a `CircularBuffer<T>` (ScottPlot#3946)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants