bug: Writing to firebase-tools GCS emulator fails if content-type or cache-control is set #5624
Description
Describe the bug
When I try to write to the official GCS emulator (firebase-tools storage emulator) with any of content-type, content-encoding or cache-control set, writing will fail.
For example, let's say I initialize the operator like this
let bucket = "opendal-test.appspot.com";
let gcs_emulator = Operator::new(
Gcs::default()
.endpoint("http://127.0.0.1:9199")
.bucket(&bucket)
.disable_config_load()
.disable_vm_metadata()
.allow_anonymous(),
)
.unwrap()
.finish();now this code works
let write_with_header = gcs_emulator
.write_with("test", "Without headers")
.await;while this code does not
let write_with_header = gcs_emulator
.write_with("test", "With headers")
.cache_control("no-store")
.await;but leads to the following error instead
thread 'main' panicked at src/main.rs:27:17:
Error writing *with* header: Unexpected (permanent) at Writer::close => GcsErrorResponse {
error: GcsError {
code: 400,
message: "Bad content type. multipart/form-data; boundary=opendal-14161605-73cd-42bf-aec1-f5a7a8f3e7a4",
errors: []
}
}
Context:
uri: http://127.0.0.1:9199/upload/storage/v1/b/opendal-test.appspot.com/o?uploadType=multipart&name=test
response: Parts {
status: 400,
version: HTTP/1.1,
headers: {
"x-powered-by": "Express",
"vary": "Origin",
"access-control-expose-headers": "content-type,x-firebase-storage-version,X-Goog-Upload-Size-Received,x-goog-upload-url,x-goog-upload-command,x-gupload-uploadid,x-goog-upload-header-content-length,x-goog-upload-header-content-type,x-goog-upload-protocol,x-goog-upload-status,x-goog-upload-chunk-granularity,x-goog-upload-control-url",
"content-type": "application/json; charset=utf-8",
"content-length": "127",
"etag": "W/\"7f-/tlh15o1qtnhW0JhdjwzbgUJ2E4\"",
"date": "Wed, 12 Feb 2025 20:09:16 GMT",
"connection": "keep-alive",
"keep-alive": "timeout=5"
}
}
service: gcs
path: test
written: 11
Steps to Reproduce
Clone this repo, and follow the steps in README
Expected Behavior
Writing should work, and the headers should be set. The Emulator suites Storage emulator supports Content-Type at least, but that doesn't work when using OpenDAL.
Additional Context
This emulator is the official emulator for Google Cloud Storage. OpenDAL not working with it is very inconvenient.
Are you willing to submit a PR to fix this bug?
- Yes, I would like to submit a PR.