Skip to content

[storage emulator] Written 3 bytes but got 5 bytes #3556

Closed

Description

@gebv

[REQUIRED] Environment info

firebase-tools: 9.14.0

Platform: macOS

For use storage emulator in local uses hack from googleapis/google-cloud-go#2476 (comment)

[REQUIRED] Test case

package main

import (
	"context"
	"io/ioutil"
	"log"
	"net/http"
	"net/url"

	"cloud.google.com/go/storage"
	"google.golang.org/api/option"
)

type roundTripper url.URL

func (rt roundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
	req.Host = rt.Host
	req.URL.Host = rt.Host
	req.URL.Scheme = rt.Scheme
	return http.DefaultTransport.RoundTrip(req)
}

func main() {
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	u, _ := url.Parse("http://localhost:9199")
	hClient := &http.Client{Transport: roundTripper(*u)}
	cs, err := storage.NewClient(context.Background(), option.WithHTTPClient(hClient))

	if err != nil {
		log.Fatalln("Failed firebase app to storage.", err)
	}

	b := cs.Bucket("default-bucket")

	{
		w := b.Object("f1").NewWriter(ctx)
		dat := "123"
		n, _ := w.Write([]byte(dat))
		log.Printf("Bytes written %q (size %d)\n", dat, n)
		if err := w.Close(); err != nil {
			log.Fatalln("Failed write file.", err)
		}
	}

	{
		r, err := b.Object("f1").NewReader(ctx)
		if err != nil {
			log.Fatalln("Failed open file.", err)
		}
		dat, err := ioutil.ReadAll(r)
		if err != nil {
			log.Fatalln("Failed read file.", err)
		}
		if err := r.Close(); err != nil {
			log.Fatalln("Failed write file.", err)
		}
		log.Printf("Got data %q (size %d)\n", string(dat), r.Size())
	}

}
2021/07/07 15:30:19 Bytes written "123" (size 3)
2021/07/07 15:30:19 Got data "123\r\n" (size 5)

[REQUIRED] Steps to reproduce

$cat firebase.json
{
  "emulators": {
    "storage": {
      "port": 9199
    },
    "ui": {
      "enabled": true,
      "port": 4000
    }
  },
  "storage": {
    "rules": "storage.rules"
  }
}

$ cat storage.rules
rules_version = '2';
service firebase.storage {
  match /b/{bucket}/o {
    match /{allPaths=**} {
      allow read, write: if true;
    }
  }
}

$ firebase emulators:start

$ STORAGE_EMULATOR_HOST=localhost:9199 PROJECT=myproject go run ./main.go

[REQUIRED] Expected behavior

I want got what was send

[REQUIRED] Actual behavior

I get extra 2 bytes of line wrap in content

PS storage emulator not ready for work yet?

Activity

  1. changed the title [-]Written 3 bytes but got 5 bytes[/-] [+][cloud storage emulator] Written 3 bytes but got 5 bytes[/+] on Jul 7, 2021
  2. changed the title [-][cloud storage emulator] Written 3 bytes but got 5 bytes[/-] [+][storage emulator] Written 3 bytes but got 5 bytes[/+] on Jul 7, 2021
  3. abeisgoat commented on Jul 7, 2021

    @abeisgoat
    Contributor

    Our emulator doesn't currently guarantee compatibility with any Google Cloud SDKs / Firebase Admin SDKs except Node.js, so you're correct that this is a bug, but it's also not something currently have in scope. I'll leave this open because it might be an easy fix, but we currently don't officially support use of the Go SDK with the Cloud Storage for Firebase emulator.

  4. tonyjhuang commented on Jul 29, 2022

    @tonyjhuang
    Contributor

    Filed internal bug to track, b/240637644

  5. removed their assignment
    on Nov 21, 2024
  6. joehan commented on Jun 4, 2025

    @joehan
    Member

    Closing as out of scope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions