Skip to content

Loading deep proto files having "imports" #368

@rektide

Description

@rektide

Hello,

Other examples of protobuf software seem to have some concept of a base directory where protobufs will be imported from, but I'm having trouble loading proto files in protobuf.js. Whenever I have an import, it seems like protobuf.js expects that proto to be in that directory or a subdirectory.

Specifically, I've been trying to get a Node project up that can talk to Etcd's new v3 protobuf/grpc interfaces. I'm having trouble getting the .loadProto method to load this proto file and it's imports without fairly brutal hackery.

The proto files seem to assume there's a base directory. Rpc.proto is in this etcdserver/etcdserverpb/rpc.proto folder, yet it has these imports:
import "gogoproto/gogo.proto"; import "etcd/storage/storagepb/kv.proto";

So when I try to load the rpc.proto file, it joins the dir of rpc.proto with the specified, expecting for example a etcdserver/etcdserver/gogoproto/gogo.proto file, whereas Etcd/Go-protobuf-tooling seems cognizant of that gogoproto/ is a top level directory for the project.

I've had to add a bunch of symlinks to my project, such that I have rpc.proto symlinked into a "top-level directory", and a symlink for etcd/storage/storagepb/gogoproto to satisfy etcd/storage/storagepb/kv.proto's duplicate ask for gogoproto/gogo.proto.

Ideally, I'd like to be able to do loadProto on each dependency in order, and I had a go at that that I thought ought work. I had to add the above mentioned symlinks, and worse, it indicates that it's not me loading the imports ahead of time: now loadProto is searching for imports on the fs, rather than me providing it to it ahead of time. So I'd have to add etcd/storage/storagepb/gogodeps/gogo.proto as another explicit, ahead of time import on line 14 in my code, where I enumerate deps.

Looking at the data-structures protobuf.js generates, it seems to use full paths for dependencies. It seems like other protobuf tooling has some notion of a top level directory, that it can also use to search for files from. That concept could be added, or trivially/hackierly, perhaps protobuf.js could try recursing up the directory tree, looking for imports until satisfied.

Please let me know if I can help clarify or inform further. I'd really like to get these proto's loaded in protobuf.js, and get on with my little app thing. Thanks for the amazing project dcodeIO &al, quite a fantastic work here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions