Optimize the drift-over-time API for V3IO TSDB. The main issue was that we were iterating over the data multiple times after retrieving it, even though this can be done in a single pass. We also added some optimization in the code logic (see more details below).

🛠️ Changes Made

• Use integers representing nanoseconds instead of datetime objects for bucketing. This also means working in nanoseconds rather than microseconds for better performance during this phase.
• Filter out invalid data early, before any processing. Previously, we processed everything first and filtered later. With the new design, we avoid doing unnecessary work.
• Reduce multiple separate iterations over the data into a single pass that aggregates directly while iterating.
• Perform datetime conversions only at the very end, after all filtering and processing is done.
• Create fewer objects, reducing memory allocations.

✅ Checklist

• I updated the documentation (if applicable)
• I have tested the changes in this PR
• I confirmed whether my changes are covered by system tests
  • If yes, I ran all relevant system tests and ensured they passed before submitting this PR
  • I updated existing system tests and/or added new ones if needed to cover my changes
• If I introduced a deprecation:
  • I followed the Deprecation Guidelines
  • I updated the relevant Jira ticket for documentation

🧪 Testing

• TestMonitoringAppFlow passed (includes testing for drift over time values).

🔗 References

• Ticket link: https://iguazio.atlassian.net/browse/ML-11820
• Design docs links:
• External links:

🚨 Breaking Changes?

• Yes (explain below)
• No

🔍️ Additional Notes