Fixed in https://issues.apache.org/jira/browse/SPARK-23534 (note that this issue is still open because there's no official builds with hadoop 3 but we build it ourselves)