[C++] StackOverflow in merge generator causes segmentation fault in scan

XMLWordPrintableJSON

      I'm still working to make a minimal reproducer for this, though I can reliably reproduce it below (though that means needing to download a bunch of data first...). I've cleaned out much of the unnecessary code (so this query below is a bit silly, and not what I'm actually trying to do), but haven't been able to make a constructed dataset that reproduces this.

      Working on some example with the new | more cleaned taxi dataset at s3://ursa-labs-taxi-data-v2, I've run into a segfault:

      library(arrow)
      library(dplyr)
      
      ds <- open_dataset("path/to/new_taxi/")
      
      ds %>%
        filter(!is.na(pickup_location_id)) %>%
        summarise(n = n()) %>% collect()
      

      Most of the time ends in a segfault (though I have gotten it to work on occasion). I've tried with smaller files | constructed datasets and haven't been able to replicate it yet. One thing that might be important is: pickup_location_id is all NAs | nulls in the first 8 years of the data or so.

      I've attached a backtrace in case that's enough to see what's going on here.

        1. backtrace.txt
          1014 kB
          Jonathan Keane

            Assignee:
            Weston Pace
            Reporter:
            Jonathan Keane
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2h 10m
                2h 10m