Background

We have a single repository with a Go server and a Typescript client. We use protobufs to communicate between the two. We try to maintain forwards and backwards compatibility between the two, i.e. the client can send an extra field which the server ignores, or the server accepts an extra field that the client does not send yet. Protobuf, in principle, is excellent for this.

The issue

Currently, for proto files shared between Typescript and Go, there is a semantics difference which makes it difficult to add new fields. Suppose we have a message Foo with an int64 id = 1; and we add the additional field string name = 2; to the proto definition. Now, if we don't set a value for this field explicitly in the Go code, it is simply interpreted as having the default value for that field (e.g. "" for strings). For Typescript code, an error is generated for all instantiations of the message indicating a type mismatch between {id: number} and Foo, you're missing {name: string}.

Current options

The options I'm aware of:

• The useOptionals=true ts-proto parameter allows message fields to be optional, but not scalars or repeated fields.
• Type wrappers could be used, e.g. StringValue instead of string, making the scalar field a message field, allowing it to be optional; in combination with the option above, it would allow message fields and scalars to be optional, but not repeated fields.
• In protoc 3.15+, scalar fields can be marked optional, but not repeated fields. Also, proto3's use case is different: it indicates "field presence" semantics, allowing to see the difference between a default value and an unset value; proto3 already allows all fields to be missing from all protobufs.
• Issue Repeated fields can be undefined #225 addresses this for repeated fields, with a PR in feat: implemented forceOptionalRepeated=true flag #234, but no action since May and it doesn't change scalars.
• If one uses the solution from feat: implemented forceOptionalRepeated=true flag #234 and enable useOptionals=true and use type wrappers, the issue would arguably be resolved for scalars, message fields and repeated fields? Is this the preferred way to go?
• Every time you update the proto, you can update all instantiations of the message. If the type is used a lot, this is tedious.
• One can use Foo.fromPartial(), but that means using fromPartial everywhere, which is also tedious.

It seems to me none of these options truly resolve the issue, they are all workarounds.

The proposal

I would like to add an additional parameter that enables ts-proto to follow Go semantics and make all fields optional. With the option enabled, I would expect the following proto:

message Foo {
  int64 id1 = 1;
  optional int64 id2 = 2;
  repeated int64 id3 = 3;

  Bar bar1 = 4;
  optional Bar bar2 = 5;
  repeated Bar bar3 = 6;
}

message Bar {}

to generate the following Typescript (or something similar):

export interface Foo {
  id1?: number; /* after conversion from empty protobuf to JS, id1 is 0 */
  id2?: number; /* after conversion from empty protobuf to JS, id2 is undefined */
  id3?: number[]; /* after conversion from empty protobuf to JS, id3 is [] */

  bar1?: Bar;
  bar2?: Bar;
  bar3?: Bar[];
}

export interface Bar {}

I am aware of the downside that this allows typo's, i.e. I can use {idd1: 5} as a Foo without errors since all fields are optional and Typescript allows setting additional fields (until Exact Types are implemented). Of course, in Go this risk does not exist since you are not allowed to set any fields that don't exist, so you would get an error here. I accept this risk and would like to cover it with better testing.

Do you think this is possible, and do you think it is useful to add?