Skip to content

Improvements in Database Metadata to prevent Statement leaks and enhance Statement caching#806

Merged
cheenamalhotra merged 9 commits intomicrosoft:devfrom
cheenamalhotra:fixDatabaseMetadata
Oct 27, 2018
Merged

Improvements in Database Metadata to prevent Statement leaks and enhance Statement caching#806
cheenamalhotra merged 9 commits intomicrosoft:devfrom
cheenamalhotra:fixDatabaseMetadata

Conversation

@cheenamalhotra
Copy link
Copy Markdown
Member

This PR introduces below changes to SQLServerDatabaseMetadata:

  • Improved Statement caching for different catalogs
  • Enables Statement caching for 'null' catalog
  • Enables closing un-cached Statements on completion of ResultSet (Prevents Statement Leaks)

@codecov-io
Copy link
Copy Markdown

codecov-io commented Sep 20, 2018

Codecov Report

Merging #806 into dev will decrease coverage by <.01%.
The diff coverage is 82.97%.

Impacted file tree graph

@@             Coverage Diff              @@
##                dev     #806      +/-   ##
============================================
- Coverage     48.58%   48.57%   -0.01%     
+ Complexity     2794     2791       -3     
============================================
  Files           116      116              
  Lines         27879    27890      +11     
  Branches       4651     4652       +1     
============================================
+ Hits          13545    13548       +3     
- Misses        12124    12141      +17     
+ Partials       2210     2201       -9
Flag Coverage Δ Complexity Δ
#JDBC42 48.12% <82.97%> (+0.13%) 2750 <0> (+4) ⬆️
#JDBC43 48.49% <82.97%> (-0.06%) 2786 <0> (-7)
Impacted Files Coverage Δ Complexity Δ
...om/microsoft/sqlserver/jdbc/SQLServerBulkCopy.java 52.79% <69.23%> (+0.33%) 264 <0> (+1) ⬆️
...soft/sqlserver/jdbc/SQLServerDatabaseMetaData.java 34.91% <88.23%> (+1.42%) 66 <0> (+3) ⬆️
...rc/main/java/com/microsoft/sqlserver/jdbc/DDC.java 44.2% <0%> (-1.54%) 106% <0%> (-2%)
...ncurrentlinkedhashmap/ConcurrentLinkedHashMap.java 38.14% <0%> (-0.87%) 42% <0%> (-1%)
...m/microsoft/sqlserver/jdbc/SQLServerResultSet.java 32.38% <0%> (-0.39%) 252% <0%> (-4%)
...rc/main/java/com/microsoft/sqlserver/jdbc/dtv.java 62.28% <0%> (-0.18%) 0% <0%> (ø)
...in/java/com/microsoft/sqlserver/jdbc/IOBuffer.java 56.35% <0%> (+0.13%) 0% <0%> (ø) ⬇️
...n/java/com/microsoft/sqlserver/jdbc/Parameter.java 63.72% <0%> (+0.2%) 64% <0%> (ø) ⬇️
...om/microsoft/sqlserver/jdbc/ReaderInputStream.java 45.05% <0%> (+1.09%) 15% <0%> (ø) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e521780...f3a844a. Read the comment docs.

Comment thread src/main/java/com/microsoft/sqlserver/jdbc/SQLServerBulkCopy.java
@cheenamalhotra
Copy link
Copy Markdown
Member Author

Jars for testing : mssql-jdbc-PR806.zip

@cheenamalhotra cheenamalhotra added this to the 7.1.2 milestone Sep 28, 2018
SQLServerResultSet rsMoreMetaData = (!connection
.getServerSupportsColumnEncryption() ? statementMoreMetadata
.executeQueryInternal("select collation_name from sys.columns where "
+ "object_id=OBJECT_ID('" + Util.escapeSingleQuotes(destinationTableName) + "') "
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is less readable to me. Maybe you could only "if else" the query string?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we can't place it in try with resources block. Also will need another try block, and finally block to manually close it.
I don't see any issues - its a simple ternary operator.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
+ "object_id=OBJECT_ID('" + Util.escapeSingleQuotes(destinationTableName) + "') "
/**
* Returns the column metadata for the destination table (and saves it for later)
*/
private void getDestinationMetadata() throws SQLServerException {
if (null == destinationTableName) {
SQLServerException.makeFromDriverError(null, null,
SQLServerException.getErrString("R_invalidDestinationTable"), null, false);
}
SQLServerResultSet rs = null;
SQLServerStatement stmt = null;
String metaDataQuery = null;
String bulkCopyEncrytionType = null;
String escapedTableName = Util.escapeSingleQuotes(destinationTableName);
try {
if (null != destinationTableMetadata) {
rs = (SQLServerResultSet) destinationTableMetadata;
} else {
stmt = (SQLServerStatement) connection.createStatement(ResultSet.TYPE_FORWARD_ONLY,
ResultSet.CONCUR_READ_ONLY, connection.getHoldability(), stmtColumnEncriptionSetting);
// Get destination metadata
rs = stmt.executeQueryInternal(
"sp_executesql N'SET FMTONLY ON SELECT * FROM " + escapedTableName + " '");
}
destColumnCount = rs.getMetaData().getColumnCount();
destColumnMetadata = new HashMap<>();
destCekTable = rs.getCekTable();
if (connection.getServerSupportsColumnEncryption()) {
metaDataQuery = "select collation_name, encryption_type from sys.columns where "
+ "object_id=OBJECT_ID('" + escapedTableName + "') " + "order by column_id ASC";
} else {
metaDataQuery = "select collation_name from sys.columns where " + "object_id=OBJECT_ID('"
+ escapedTableName + "') " + "order by column_id ASC";
}
try (SQLServerStatement statementMoreMetadata = (SQLServerStatement) connection.createStatement();
SQLServerResultSet rsMoreMetaData = statementMoreMetadata.executeQueryInternal(metaDataQuery)) {
for (int i = 1; i <= destColumnCount; ++i) {
if (rsMoreMetaData.next()) {
if (connection.getServerSupportsColumnEncryption()) {
bulkCopyEncrytionType = rsMoreMetaData.getString("encryption_type");
}
destColumnMetadata.put(i, new BulkColumnMetaData(rs.getColumn(i), bulkCopyEncrytionType, null));
} else {
destColumnMetadata.put(i, new BulkColumnMetaData(rs.getColumn(i)));
}
}
}
} catch (SQLException e) {
// Unable to retrieve metadata for destination
throw new SQLServerException(SQLServerException.getErrString("R_unableRetrieveColMeta"), e);
} finally {
if (null != rs)
rs.close();
if (null != stmt)
stmt.close();
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider the suggestion above.

@cheenamalhotra cheenamalhotra merged commit cd3891c into microsoft:dev Oct 27, 2018
@cheenamalhotra cheenamalhotra changed the title Improvements in Database metadata to prevent Statement leaks and enhance Statement caching Improvements in Database Metadata to prevent Statement leaks and enhance Statement caching Oct 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants