[dottrap]

As another extremely simple idea, in my personal experience, just using regular Lua table syntax for serialization and then pre-compiling it with luac so it could be loaded quickly via dofile(), produced results just as fast for loading it in as using lua-protobuf. (I don't remember the difference between writing out Lua tables vs. lua-protobuf because my use case needed to read the data more often than generate it, but it must have not been big enough, if any, otherwise I would probably remember it.) I was loading gigabytes of data for large batch processing.

[lifthrasiir]

We exactly did the first half in my previous job, because JSON was really painful to use as is (and none of JSON5 or HOCON or whatever were popular enough at that time). The second half was substantially different due to our idiosyncratic requirements though.

[binary132]

For what it’s worth, luac has been deprecated now.

[EDIT] sorry folks! Must have misunderstood something I read a while back. Trying to dig it up now. But I could’ve sworn I’d read somewhere that this at least wasn’t suggested, and I thought it was also removed from the build in a 5.4 patch. Will circle back if I find what I’m looking for.

[dottrap]

Where did you see that? I'm skeptical of that claim because I know some embedded uses of Lua strongly utilize pre-compiling to Lua bytecode so they can keep their hardware as slim as possible. I know some will even strip out the Lua compiler from the binary they ship to shrink the profile even more. Also, many video games like to pre-compile their shipped Lua scripts to reduce their load times.

I know that the Lua team has internal private repositories, and that luac is developed in a separate repo for them. I've seen occasional reports on the Lua mailing list that luac.c is forgotten or not updated in non-official releases. That is because those source drops didn't go through the full official release process which includes merging in from their luac repo. Maybe you are confusing these intermediate source drops with deprecation? If there is deprecation, I would like to see the details on that. I presume they would be introducing some kind of replacement that addresses all the real world use cases that rely on the abilities of pre-compiling to Lua bytecode.

[pansa2]

> But I could’ve sworn I’d read somewhere that this at least wasn’t suggested

Maybe you’re thinking about the use of untrusted bytecode? Loading it is strongly discouraged because that’s insecure - Lua has nothing like a bytecode verifier.

[josephcsible]

Are you sure? It doesn't look like it has to me.

[csears]

Would that be safe even with untrusted data? (Assuming you were the one who serialized and compiled the table)

[pansa2]

Lua should be robust in the face of untrusted source code. However untrusted bytecode is unsafe.

[Dylan16807]

Are you asking if there are special numbers or strings or some other kind of plain data that would glitch out the interpreter when they're read back in? That would be an impressive failure of a programming language.

Yes it's safe as long as you're serializing correctly (which isn't very hard).