Repository navigation
[Java] jdbc:arrow-flight-sql executeQuery behaves incorrectly #38785
Description
Describe the bug, including details regarding any error messages, version, and platform.
Arrow Version 13.0.0
I use the following code to connect to flight sql server to execute query, two unexpected behaviors occur:
acceptPutPreparedStatementUpdateof the flight server is called, andflightStream.getRoot().getRowCount() == 0is true. but query is not an Update statement.
- Shouldn't
acceptPutPreparedStatementUpdatebe called?
I'm using Python DB-API 2.0 cursor.execute(sql) and cursor.fetchallarrow() which does not call acceptPutPreparedStatementUpdate after calling createPreparedStatement.
- The FlightEndpoint returned by
getFlightInfoPreparedStatementis another flight server, but not fetch result from other flight server. Instead,getStreamStatementof the current flight server is called.
- Why is there no fetch result from another flight server?
- Why is
getStreamStatementcalled instead ofgetStreamPreparedStatement?
Actual calling sequence:
createPreparedStatementacceptPutPreparedStatementUpdategetFlightInfoPreparedStatementgetStreamStatementclosePreparedStatement
I'm looking for help, thanks!
Connection conn = null;
Statement stmt = null;
Class.forName("org.apache.arrow.driver.jdbc.ArrowFlightJdbcDriver");
try {
conn = DriverManager.getConnection("jdbc:arrow-flight-sql://xx:xx?useServerPrepStmts=false&cachePrepStmts=true&useSSL=false&useEncryption=false", "root", "");
String sql = "select * from clickbench.hits limit 50";
stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery(sql);
int rowNum = rs.getRow();
System.out.println(rowNum);
rs.close();
stmt.close();
conn.close();
} catch (SQLException e) {
throw new RuntimeException(e);
}
Component(s)
FlightRPC
Activity
[-][Java] jdbc:arrow-flight-sql behaves incorrectly[/-][+][Java] jdbc:arrow-flight-sql executeQuery behaves incorrectly[/+]xinyiZzz commentedon Nov 20, 2023
FlightInfo Endpoint in
ArrowFlightSqlClientHandler.getStreamsis correctxinyiZzz commentedon Nov 20, 2023
Upgraded to the latest 14.0.1, problem still exists
It seems that Location is not used when initializing FlightStream?
jduo commentedon Nov 20, 2023
Hi @xinyiZzz ,
It'd be great to create two separate Issues for each problem.
For the problem with the Location not being used, this was reported in #34532 and recently fixed in #38521 . This fix hasn't been released into a new artifact yet though.
For problem 1, I'm not sure if I understand the issue being raised. The acceptPutPreparedStatementUpdate RPC is used when the query supplied is an update-type statement (and if that's the case, you would use either the execute() or executeUpdate() JDBC functions on Statement/PreparedStatement instead of executeQuery()). Or are you saying you are running a SELECT statement with executeQuery() and the driver is issuing the acceptPutPreparedStatementUpdate message instead?
aiguofer commentedon Nov 20, 2023
One thing I noticed while working on this code is that the way we're identifying
UPDATEstatements is REALLY naive. If the server returns anullschema for thecreatePreparedStatementcall, the driver thinks this is anUPDATEcall. This could be your problem. Is the server correctly setting the schema?xinyiZzz commentedon Nov 21, 2023
Thanks! , ok, I'll separate later~
That's great! When will this fix be released into a new artifact? , or is there a nightly build that can be used? I want to use it as soon as possible.
Yes, I running a SELECT statement with executeQuery() and the driver is issuing the acceptPutPreparedStatementUpdate message instead. I think as @aiguofer said.
xinyiZzz commentedon Nov 21, 2023
Great! I reversed the order of
DatasetSchemaandParameterSchemaas parameters, and myParameterSchemawas null.Thanks for your attention @aiguofer , ask a question again, it seems that the
DatasetSchemareturned bycreatePreparedStatementis not really used? I return nullDatasetSchema, python adbc queries normally.xinyiZzz commentedon Nov 21, 2023
I found nightly builds and will try it:
https://arrow.apache.org/docs/developers/java/building.html#installing-nightly-packages
xinyiZzz commentedon Nov 21, 2023
Thanks! @jduo , I used nightlies 15.0.0-SNAPSHOT and the problem disappeared, getStream requested other endpoints.
aiguofer commentedon Nov 22, 2023
The
DatasetSchemais exposed to clients to do whatever they wish with it. They might need to know the schema ahead of having the results, for example.As you noticed, It's also used to determine the type of query, which might affect how to execute the query. For example, the JDBC driver doesn't differentiate betwee
executeQueryandexecuteUpdatewhenexecuteis called, so the driver itself must do it.This does mean that in order for your server to meet the spec you need to include a
DatasetSchema.xinyiZzz commentedon Nov 22, 2023
Thanks!, solved all my problems.
jduo commentedon Nov 22, 2023
Thanks for the update @xinyiZzz and thanks @aiguofer .
Closing as all questions have been resolved.
[fix](arrow-flight) Modify FE Arrow version to 14.0.1 #28093
[fix](arrow-flight) Modify FE Arrow version to 14.0.1 apache#28093