rust: Every ADBC operation acquires a lock on the driver preventing parallel statement execution #2715
Description
What happened?
Statement execution is unnecessarily serialized when issuing statements from multiple threads even when each thread has their own dedicated Connection.
This happens because a lock on the driver is acquired for every operation, so there is no chance of parallelization when using an ADBC driver from Rust.
fn execute(&mut self) -> Result<impl RecordBatchReader> {
let driver = &self.inner.connection.database.driver.driver.lock().unwrap();
let mut statement = self.inner.statement.lock().unwrap();
let mut error = ffi::FFI_AdbcError::with_driver(driver);
let method = driver_method!(driver, StatementExecuteQuery);
let mut stream = FFI_ArrowArrayStream::empty();
let status = unsafe { method(statement.deref_mut(), &mut stream, null_mut(), &mut error) };
check_status(status, error)?;
let reader = ArrowArrayStreamReader::try_new(stream)?;
Ok(reader)
}Creating connections can also prevent the execution of statements, because connection creation also needs a lock on the driver.
/// Returns a new connection using the loaded driver.
///
/// This uses `&mut self` to prevent a deadlock.
fn connection_new(&mut self) -> Result<ffi::FFI_AdbcConnection> {
let driver = &self.inner.driver.driver.lock().unwrap();
let mut connection = ffi::FFI_AdbcConnection::default();
// ConnectionNew
let mut error = ffi::FFI_AdbcError::with_driver(driver);
let method = driver_method!(driver, ConnectionNew);
let status = unsafe { method(&mut connection, &mut error) };
check_status(status, error)?;
Ok(connection)
}Stack Trace
No response
How can we reproduce the bug?
No response
Environment/Setup
No response