top | item 44338966

(no title)

CactusRocket | 8 months ago

I imagine that one of the points of a solid protocol buffers library would be to align the types even across programming languages. E.g. explicitly force a 64-bit integer rather than "int" relying on the platform. And to have some custom "string" type which is always UTF-8 encoded in memory rather than depending on the platform-specific encoding.

(I have no idea if that is the case with protobuf, I don't have enough experience with it.)

discuss

order

chubot|8 months ago

Why would you "imagine" that?

Again, the problem has more to do with the programming languages themselves, rather than with protobufs or parsing.

Protobuf has both signed and unsigned integers - the initial use case was C++ <-> C++ communication

Java doesn't have unsigned integers

Python has arbitrary precision integers

JavaScript traditionally only had doubles, which means it can represent integers up to 53 bit exactly. It has since added arbitrary size integers -- but that doesn't mean that the protobuf libraries actually use them

---

These aren't the only possibilities -- every language is fundamentally different

OCaml has 31- or 63-bit integers IIRC

https://protobuf.dev/programming-guides/encoding/#int-types

And again, strings also differ between all these languages -- there are three main choices, which are basically 8-bit, 16-bit, or 32-bit code units

Go and Rust favor 8-bit units; Java and JavaScript favor 16-bit units; and Python/C/C++ favors 32-bit units (which are code points)

CactusRocket|8 months ago

I think I didn't explain myself well.

As long as a language has bytes and arrays, you can implement anything on top of them, like unsigned integers, 8-bit strings, UTF-8 strings, UCS-2, whatever you want. Sure it won't be native types, so it will probably be slower and could have an awkward memory layout, but it's possible

Granted, if a language is so gimped that it doesn't even have integers (as you mentioned JavaScript), then that language will not be able to fully support it indeed.