Failed response over API looks like success #4215
Description
Repro is with Zed commit 0066c8a.
I was tinkering in a Brim branch https://github.com/brimdata/brim/tree/parquet-export to add Parquet as an Export format. It works fine for the happy path, but the backend currently won't create Parquet output when the query response has one or more of the following:
- Multiple shapes
- A
uniontype
With the Zed CLI tooling, this is well-reported to the user. The following examples use test data sample.zng from the Brim repo.
$ zq -f parquet sample.zng > /dev/null
Parquet output requires uniform records but multiple types encountered (consider 'fuse')
$ echo $?
1
$ zq -f parquet 'fuse' sample.zng > /dev/null
union type unsupported
$ echo $?
1
$ curl http://localhost:9867/version
{"version":"v1.2.0-132-g0066c8aa"}
$ zed create foo
pool created: foo 2HeFzcYxjPT6hCQ7oZ54rUu7MPa
$ zed -use foo load sample.zng
(2/1) 4910B/4910B 4910B/s 100.00%
2HeG09yuVGPz3gQg5d0c4yUC8ru committed
$ zed query -f parquet 'from foo' > /dev/null
Parquet output requires uniform records but multiple types encountered (consider 'fuse')
$ echo $?
1
$ zed query -f parquet 'from foo | fuse' > /dev/null
union type unsupported
$ echo $?
1
However, If I attempt those queries over the API (as the Brim app would need to do), the errors seem to be masked by the fact that the response comes back as HTTP 200.
$ curl -v --fail-with-body -X POST --output /dev/null -H 'Accept: application/x-parquet' -H 'Content-Type: application/json' http://localhost:9867/query -d '{"query":"from foo"}'
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 127.0.0.1:9867...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to localhost (127.0.0.1) port 9867 (#0)
> POST /query HTTP/1.1
> Host: localhost:9867
> User-Agent: curl/7.84.0
> Accept: application/x-parquet
> Content-Type: application/json
> Content-Length: 20
>
} [20 bytes data]
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: application/x-parquet
< Vary: Origin
< X-Request-Id: 2HeGbXwUEzZLarpK8Cj4N6CyTgT
< Date: Wed, 16 Nov 2022 22:36:47 GMT
< Content-Length: 1111
<
{ [1111 bytes data]
100 1131 100 1111 100 20 134k 2484 --:--:-- --:--:-- --:--:-- 552k
* Connection #0 to host localhost left intact
$ echo $?
0
$ curl -v --fail-with-body -X POST --output /dev/null -H 'Accept: application/x-parquet' -H 'Content-Type: application/json' http://localhost:9867/query -d '{"query":"from foo | fuse"}'
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 127.0.0.1:9867...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to localhost (127.0.0.1) port 9867 (#0)
> POST /query HTTP/1.1
> Host: localhost:9867
> User-Agent: curl/7.84.0
> Accept: application/x-parquet
> Content-Type: application/json
> Content-Length: 27
>
} [27 bytes data]
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Vary: Origin
< X-Request-Id: 2HeGg9a4VuJ1QpVjYkHMAbRMK5h
< Date: Wed, 16 Nov 2022 22:37:23 GMT
< Content-Length: 0
<
100 27 0 0 100 27 0 3580 --:--:-- --:--:-- --:--:-- 13500
* Connection #0 to host localhost left intact
$ echo $?
0
I showed this to @nwt and his first thought was that the client would maybe need to get smarter here, which makes sense since the Zed CLI tools were somehow able to deduce there was an error. However, I then also remembered that the zed query pulls the response over the API as ZNG and does the Parquet conversion client-side, so that explains it.