Skip to content

Add external timestamp support to endpointer to prevent clock drift#421

Merged
dhdaines merged 4 commits intomainfrom
kal-endpointer-timestamp-fix
Jul 29, 2025
Merged

Add external timestamp support to endpointer to prevent clock drift#421
dhdaines merged 4 commits intomainfrom
kal-endpointer-timestamp-fix

Conversation

@lenzo-ka
Copy link
Contributor

@lenzo-ka lenzo-ka commented Jul 16, 2025

Summary

This PR adds support for external timestamp sources to ps_endpointer_t to prevent timestamp drift between audio and system clocks, addressing issue #352.

Problem

The endpointer currently calculates timestamps based solely on audio sample counts:

ep->timestamp += (double)nsamp / ps_endpointer_sample_rate(ep);

This causes drift when the audio clock is not synchronized with the system clock, making it difficult to correlate speech detection events with system time.

Solution

This implementation allows applications to provide their own timestamp source (e.g., system time, monotonic clock) while maintaining full backward compatibility:

New Features

  1. Timestamp Callback Type: ps_endpointer_timestamp_cb_t

    • Allows applications to provide custom timestamp sources

  2. New API Functions:

    • ps_endpointer_set_timestamp_func() - Set external timestamp source
    • ps_endpointer_timestamp() - Get current timestamp

  3. Example Program: examples/endpointer_timestamp_example.c

    • Demonstrates how to use system time as timestamp source

Implementation Details

  • External timestamp callback is completely optional
  • Default behavior uses audio-based timestamps (unchanged)
  • Internal audio timestamp tracking is maintained for queue management
  • Speech start/end timestamps use external source when provided
  • Smooth transitions between timestamp sources

Testing

  • Comprehensive test suite added: test/unit/test_endpointer_timestamp.c
  • Tests include:
    • Default audio-based timestamps
    • External timestamp callback
    • Reverting between timestamp sources
    • Clock drift simulation
    • Full backward compatibility verification
    • All existing APIs tested without callback
  • All existing tests pass unchanged

Backward Compatibility

✅ Fully backward compatible - Existing code continues to work without any changes:

  • Default behavior unchanged
  • All existing APIs function identically
  • New features are opt-in only

Usage Example

/* System time callback */
static double get_system_time(void *user_data) {
    struct timespec ts;
    clock_gettime(CLOCK_MONOTONIC, &ts);
    return ts.tv_sec + ts.tv_nsec / 1000000000.0;
}

/* Set external timestamp source */
ps_endpointer_set_timestamp_func(ep, get_system_time, NULL);

/* Speech timestamps now use system time */
double start = ps_endpointer_speech_start(ep);  // System time
double end = ps_endpointer_speech_end(ep);      // System time

Fixes #352

lenzo-ka added 2 commits July 16, 2025 15:21
@lenzo-ka
Add external timestamp support to endpointer to prevent clock drift
6918597 
- Added ps_endpointer_timestamp_cb_t callback type for external timestamps
- Added ps_endpointer_set_timestamp_func() to set external timestamp source
- Added ps_endpointer_timestamp() to get current timestamp
- Modified internal timestamp handling to use callback when available
- Maintains full backward compatibility - existing code works unchanged
- Added comprehensive test suite (test_endpointer_timestamp)
- Added example program demonstrating the new functionality

This addresses issue #352 by allowing applications to provide their own
timestamp source (e.g., system time, monotonic clock) to avoid drift
between audio clock and system clock. The implementation maintains the
audio-based timestamp tracking internally for queue management while
using the external source for speech start/end timestamps when provided.
@lenzo-ka
This lets CI work
813ad97
@lenzo-ka lenzo-ka requested a review from dhdaines July 16, 2025 19:24
@lenzo-ka lenzo-ka mentioned this pull request Jul 17, 2025
Closed
dhdaines
dhdaines requested changes Jul 26, 2025
View reviewed changes
Copy link
Contributor

@dhdaines dhdaines left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! I'm not and probably never will be an expert in real-time audio processing but I definitely understand why this is necessary.

There's one comment in the code to verify. Also I have one question: is this mechanism sufficiently flexible to be used with common audio APIs e.g. CoreAudio, OpenAL, etc? (there are like thousands of these and I understand basically zero of them)

@dhdaines dhdaines self-requested a review July 29, 2025 13:59
@lenzo-ka
Copy link
Contributor Author

lenzo-ka commented Jul 29, 2025

it should be pretty generic, the callback just needs to return a double with the current time:
typedef double (*ps_endpointer_timestamp_cb_t)(void *user_data);

Let me see if i can make some examples

CoreAudio

 typedef struct {
    AudioQueueRef queue;
} coreaudio_data_t;

double coreaudio_timestamp(void *user_data) {
    // Use host time and convert to seconds
    uint64_t hostTime = AudioGetCurrentHostTime();
    return AudioConvertHostTimeToSeconds(hostTime);
}

PortAudio

typedef struct {
    PaStream *stream;
} portaudio_data_t;

double portaudio_timestamp(void *user_data) {
    portaudio_data_t *data = (portaudio_data_t *)user_data;
    return Pa_GetStreamTime(data->stream);  // Already in seconds!
}

ALSA

typedef struct {
    snd_pcm_t *pcm;
    struct timespec start_time;
} alsa_data_t;

double alsa_timestamp(void *user_data) {
    struct timespec now;
    clock_gettime(CLOCK_MONOTONIC, &now);
    return now.tv_sec + now.tv_nsec / 1e9;
}

WebAudio

double webaudio_timestamp(void *user_data) {
    // Could call into JavaScript to get AudioContext.currentTime
    return EM_ASM_DOUBLE({
        return audioContext.currentTime;
    });
}

Windows WASAPI

typedef struct {
    IAudioClock *clock;
    UINT64 freq;
} wasapi_data_t;

double wasapi_timestamp(void *user_data) {
    wasapi_data_t *data = (wasapi_data_t *)user_data;
    UINT64 position;
    data->clock->GetPosition(&position, NULL);
    return (double)position / data->freq;
}

dhdaines
dhdaines approved these changes Jul 29, 2025
View reviewed changes
Copy link
Contributor

@dhdaines dhdaines left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good now! Thank you so much!

This will go into a PocketSphinx 5.1.0 release in the next few days once I iron out some build issues.

@dhdaines dhdaines merged commit 2c5db7e into main Jul 29, 2025
21 checks passed
@dhdaines
Copy link
Contributor

dhdaines commented Jul 29, 2025

it should be pretty generic, the callback just needs to return a double with the current time: typedef double (*ps_endpointer_timestamp_cb_t)(void *user_data);

Let me see if i can make some examples

Oh, thank you for the examples. Probably this should go in some documentation somewhere but just to have a record of it here already helps.

@dhdaines dhdaines added this to the 5.1.0 milestone Aug 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dhdaines dhdaines dhdaines approved these changes

Assignees

No one assigned

Labels

None yet

Milestone

5.1.0

Development

Successfully merging this pull request may close these issues.

Timestamp drifting.

2 participants

@lenzo-ka @dhdaines