[Spark] Keep the writer's two Spark type dispatchers in sync, and clean up if construction fails #13

Since PR #7, the writer dispatches on Spark type in two places:

• SparkToArrowSchema, which builds the Arrow schema;
• ArrowFieldWriter.create, which builds the per-column writers.

Nothing keeps the two in sync.

Why it matters

The constructor only catches IOException, and ArrowFieldWriter.create runs after the native writer has opened the output file.

Suppose a type is added to SparkToArrowSchema but not to ArrowFieldWriter, or the vector classes don't line up. Then create throws IllegalArgumentException or ClassCastException out of the constructor. Spark never gets a DataWriter to abort, so the native writer, its open file, the RootAllocator and the VectorSchemaRoot all leak.

Impact

Latent. Both dispatchers cover the same types today, and an unsupported type throws in SparkToArrowSchema.convert, before the native writer is opened.

Where

• Constructor in PR #7
• ArrowFieldWriter.create

Fix

• Catch RuntimeException in the constructor and release whatever was opened before rethrowing.
• Add a test that every type SparkToArrowSchema accepts also gets an ArrowFieldWriter.