Skip to content

Conversation

@zeroshade
Copy link
Member

@zeroshade zeroshade commented Nov 10, 2025

Allow loading a driver via the driver manager using only a URI

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're going to do this, the logic to detect URI/driver from the driver arg needs to be either removed or made the same (as it doesn't split on colon). (Also I think that logic should probably validate that the "scheme" is alphanumeric+a few other symbols instead of accepting anything)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really need to validate the scheme here rather than just let the driver handle validating the URI itself?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not saying to validate the scheme; just avoid a false positive if there's something that looks like a colon slash later in the "uri" but the part before it wouldn't have been a valid scheme in the first place

@zeroshade zeroshade marked this pull request as ready for review November 14, 2025 18:51
@zeroshade zeroshade requested a review from wjones127 as a code owner November 14, 2025 18:51
@github-actions github-actions bot added this to the ADBC Libraries 22 milestone Nov 14, 2025
@zeroshade
Copy link
Member Author

I have no idea why python is failing to find the bigquery driver shared lib on windows in the CI. Any ideas @lidavidm?

@zeroshade
Copy link
Member Author

@kou @eitsupi any ideas on why the bigquery driver is failing to be found by the driver manager on windows in python? I can't see how the change for handling the driver uris would affect this only on windows.

@kou
Copy link
Member

kou commented Nov 18, 2025

Could you share the CI failure log?

@kou
Copy link
Member

kou commented Nov 18, 2025

https://github.com/apache/arrow-adbc/actions/runs/19443805171/job/55633450295#step:12:181

______________________________ test_load_driver _______________________________

    def test_load_driver():
        # Fails, but in environments where we don't have access to any BigQuery
        # projects, this checks that we can at least *load* the driver
        with pytest.raises(
            adbc_driver_manager.ProgrammingError, match="ProjectID is empty"
        ):
>           with adbc_driver_bigquery.connect("") as db:

python\adbc_driver_bigquery\tests\test_lowlevel.py:47: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
venv\lib\site-packages\adbc_driver_bigquery\__init__.py:227: in connect
    return adbc_driver_manager.AdbcDatabase(driver=_driver_path(), **kwargs)
adbc_driver_manager/_lib.pyx:605: in adbc_driver_manager._lib.AdbcDatabase.__init__
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   adbc_driver_manager.ProgrammingError: NOT_FOUND: [Driver Manager] D: LoadLibraryExW() failed: (126) The specified module could not be found.

E   
E   D.dll: LoadLibraryExW() failed: (126) The specified module could not be found.

E   
E   Also searched these paths for manifests:
E   	not set: ADBC_DRIVER_PATH
E   	additional search path: D:\a\arrow-adbc\arrow-adbc\adbc\venv\etc/adbc/drivers
E   	not enabled at build time: Conda prefix
E   	Registry: HKEY_CURRENT_USER\[Driver Manager] SOFTWARE\ADBC\DRIVERS not found
E   	does not exist: C:\Users\runneradmin\AppData\Local\ADBC/Drivers
E   	Registry: HKEY_LOCAL_MACHINE\[Driver Manager] SOFTWARE\ADBC\DRIVERS not found

adbc_driver_manager/_lib.pyx:262: ProgrammingError

During handling of the above exception, another exception occurred:

    def test_load_driver():
        # Fails, but in environments where we don't have access to any BigQuery
        # projects, this checks that we can at least *load* the driver
>       with pytest.raises(
            adbc_driver_manager.ProgrammingError, match="ProjectID is empty"
        ):
E       AssertionError: Regex pattern did not match.
E         Expected regex: 'ProjectID is empty'
E         Actual message: 'NOT_FOUND: [Driver Manager] D: LoadLibraryExW() failed: (126) The specified module could not be found.\r\n\nD.dll: LoadLibraryExW() failed: (126) The specified module could not be found.\r\n\nAlso searched these paths for manifests:\n\tnot set: ADBC_DRIVER_PATH\n\tadditional search path: D:\\a\\arrow-adbc\\arrow-adbc\\adbc\\venv\\etc/adbc/drivers\n\tnot enabled at build time: Conda prefix\n\tRegistry: HKEY_CURRENT_USER\\[Driver Manager] SOFTWARE\\ADBC\\DRIVERS not found\n\tdoes not exist: C:\\Users\\runneradmin\\AppData\\Local\\ADBC/Drivers\n\tRegistry: HKEY_LOCAL_MACHINE\\[Driver Manager] SOFTWARE\\ADBC\\DRIVERS not found'

python\adbc_driver_bigquery\tests\test_lowlevel.py:44: AssertionError

@zeroshade
Copy link
Member Author

@kou
Copy link
Member

kou commented Nov 18, 2025

Great!

I'll take a look at them!

AlmaLinux 9: Installing old Apache Arrow from EPEL isn't expected. I'll try why it happens on local.

@zeroshade
Copy link
Member Author

Thanks!!

@kou
Copy link
Member

kou commented Nov 18, 2025

AlmaLinux 9: I think that https://lists.apache.org/thread/rqsvtdj2jxks03z8wzymk46br71dqd4h is the same problem.
Anyway, this is not related to this PR.

Windows: This has been failed since 2964c54 but the change will not be related. Some CI environments may be changed (e.g. package versions in Conda):

The first failure log: https://github.com/apache/arrow-adbc/actions/runs/18672884467/job/53237442892
The last success log: https://github.com/apache/arrow-adbc/actions/runs/18590135696/job/53003407218

Anyway, this is also not related to this PR.

@zeroshade
Copy link
Member Author

@kou Thanks for looking into this, do you think we could create a separate PR to fix the almalinux issue at least?

@zeroshade zeroshade requested a review from lidavidm November 18, 2025 20:41
@kou
Copy link
Member

kou commented Nov 18, 2025

Yes. But I don't have a correct fix for the AlmaLinux 9 failure for now. (I think that the patch in the mailing list post is a workaround.) So could you create a separated issue for now?

@zeroshade
Copy link
Member Author

Filed #3732 and assigned you to it @kou

@zeroshade zeroshade merged commit 2519191 into apache:main Nov 19, 2025
80 of 84 checks passed
@kou
Copy link
Member

kou commented Nov 20, 2025

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants