Another title of this post would be: for protocol buffers in web clients, which TypeScript protocol buffer libraries are well-documented, and which ones are badly documented?

In this post, I’ll talk about the mess that’s the open source TypeScript protocol buffer ecosystem. For historical reasons, based on various libraries that were released when others didn’t exist, there are some libraries that work with each other, some that are outdated, etc. In general, this happens quite commonly in the open source ecosystem, and I hope this post helps others.

The TLDR, if you want to use protocol buffers with the browser:

• Use protobuf-ts and grpc/grpc-web
• Do NOT use ts-proto or improbable-eng/grpc-web

My recommendations don’t even have to do with the functionality necessarily; they’re about the documentation. Note that documentation doesn’t just mean READMEs but also code examples, which are notably missing from the ones that I suggest not to use.

One disclaimer: perhaps for other side use cases those libraries are well-documented. For the particular use case of web RPCs, they are lacking.

The rest of the post is mostly an explanation of my findings of the current state of TypeScript protobuf tools. If you just want to know which tools to use, trust me on the TLDR, and skip the rest of the post.

Context

I wanted to make a simple demo of a TypeScript React client connecting to a TypeScript Node backend.

I used to work at Google for four years, and like everyone who’s worked at Google (there’s even a derisive term “proto pusher” to sometimes the mindless work of protos), I’ve used protocol buffers extensively. I’m not claiming to be the world’s foremost expert on protocol buffers or that I should be able to understand anything about protos instantly, but I just want to make the point: I’ve used them to set up web flows, data pipelines, clients and servers from scratch, etc. With a foundational knowledge of protobufs and how they’re transmitted, I should, in theory, be able to find some documentation and set up a basic “Hello World” RPC quickly.

Of course, documentation is different inside and outside of Google, but that’s the point of this post. Given the wild environment of various solutions created at various times, I hope to provide an explanation of what works (until this post becomes outdated, of course).

In short: after spending five hours desperately searching for documentation to use the combination I said not to use (which have higher number of stars on GitHub), I set up a working end-to-end connection in 20 minutes using the combination that I said to use (grpc/grpc-web and protobuf-ts) because of good documentation.

Points of confusion

Type generation

You have protocol buffers, which define a contract for entities that you want to work with. Now you want to generate Typescript types from those protos.

As far as I can tell, there exist two main open source options for type generation:

• ts-proto (has more stars)
• protobuf-ts

I think there’s also now TypeScript generation built into the official protoc, but again, the documentation in this world is so obfuscated that I only figured that out from reading grpc/grpc-web example code five directories deep in the repository.

protobuf-ts is the clear winner here, simply because of MANUAL.md. The MANUAL.md , while quite long, has stellar examples for use cases on both the client- and server- sides. On top of that, there is a whole folder of actual code examples, while ts-proto, from what I’ve seen in some responses to the issues, asks you read their integration tests (possible, but more difficult than parsing real code examples).

The author of protobuf-ts has written a summary of the differences between these two libraries, although to be honest, my currently visualized use case is basic enough that none of these will likely matter to me. As someone newly figuring out the integrations in the TypeScript protobuf world, I can recommend protobuf-ts solely due to its documentation.

Client-side GRPC Client

There are two identically named repositories: grpc/grpc-web and improbable-eng/grpc-web. The former is created by Google, and the second by another company. However, the second one is in maintenance mode with the recommendation to use the official one, which wasn’t even posted on the repo until I made a PR (the maintainer had been warning people on the repo’s issues).

For technical reasons (Google it if you’re curious), using GRPC in the browser requires a proxy. The improbable-eng/grpc-web library requires their own Go binary as a proxy, which, upon realizing this, I immediately surmised that it would be a nightmare to figure out how to deploy some random Go proxy if I wanted to combine it with other things like AWS Lambda, etc. As expected, the documentation for the proxy is scant. I didn’t think that the documentation for deploying would be promising either, and as expected, there’s little guidance there, except for a few GitHub issues.

In contrast, grpc/grpc-web suggests using Envoy as a proxy, which is quite well-known, well-maintained, and well-documented. It has Docker support, and is, in general, more well-vetted.

I initially tried the following combination: ts-proto with improbable-eng/grpc-web. Both seemed popular. After getting a semi-working example using these two libraries, I then wanted to try the ts-proto and official grpc/grpc-web combinations (moving off the improbable-eng due to the aforementioned proxy). From the supported options of ts-proto, it supports improbable-eng/grpc-web, but not the official Google grpc/grpc-web.

It seems strange that ts-proto wouldn’t integrate with the official Google library. Or maybe it does? But that’s the point — perhaps it does support it, but the documentation isn’t there. I couldn’t find any code examples, any flag documentation, any references, and I couldn’t figure out how to get the types to work on my own by digging into the generated types (admittedly due to my own lack of knowledge but hey, that’s the point of Hello World examples).

And if it is really true that ts-proto doesn’t work with the official, Google-released version of grpc/grpc-web, then it suggests that the tool may be outdated. Perhaps that version grpc/grpc-web didn’t exist then, but in any case, the documentation should clarify this. The only reference I found to this was this answer from the maintainer about how the library predates gRPC, if I understood the post correctly.

Server-side

For writing a TypeScript Node server, there does exist decent documentation for vanilla JavaScript. I just Googled some tutorials and tried to write a TypeScript version of the official tutorial. Using both ts-proto and protobuf-ts here worked fine. Ultimately, I only ran into problems with client-side web RPC typing. Nonetheless, you would want to use the same generator for both the server and client side.

Conclusion

In conclusion, documentation. But also, if you are in a similar boat to me and want to get started with a basic hello world, TypeScript gRPC web client and server, I suggest using the two libraries I talked about.

Finally, I want to say that this post does not mean to detract from the work of these open source contributors. Each of these libraries, at some point in time, had their crucial role in the ecosystem.