[swisniewski]

There's a much simpler way to do this:

If you want your library to operate on bytes, then rather than taking in an io.Reader and trying to figure out how to get bytes out of it the most efficient way, why not just have the library taken in []byte rather than io.Reader?

If someone has a complex reader and needs to extract to a temporary buffer, they can do that. But if like in the author's case you already have []byte, then just pass that it rather than trying to wrap it.

I think the issue here is that the author is adding more complexity to the interface than needed.

If you need a []byte, take in a []byte. Your callers should be able to figure out how to get you that when they need to.

With go, the answer is usually "just do the simple thing and you will have a good time".

[TheDong]

The author is trying to integrate with the Go stdlib, which requires you produce images from an 'io.Reader". See https://pkg.go.dev/image#RegisterFormat

Isn't using the stdlib simpler than not for your callers?

I also often hear gophers say to take inspiration from the go stdlib. The 'net/http' package's 'http.Request.Body' also has this same UX. Should there be `Body` and `BodyBytes` for the case when your http request wants to refer to a reader, vs wants to refer to bytes you already have?

[jchw]

The BodyBytes hypothetical isn't particularly convincing because you usually don't actually have the bytes before reading them, they're queued up on a socket.

In most cases I'd argue it really is idiomatic Go to offer a []byte API if that can be done more efficiently. The Go stdlib does sometimes offer both a []byte and Reader API for input to encoding/json, for example. Internally, I don't think it actually streams incrementally.

That said I do see why this doesn't actually apply here. IMO the big problem here is that you can't just rip out the Bytes() method with an upcast and use that due to the wrapper in the way. If Go had a way to do somehow transparent wrapper types this would possilby not be an issue. Maybe it should have some way to do that.

[tptacek]

It is, but one of the virtues of the Go ecosystem is that it's also often very easy to fork the standard library; people do it with the whole TLS stack all the time.

The tension Ted is raising at the end of the article --- either this is an illustration of how useful casting is, or a showcase of design slipups in the standard library --- well, :why-not-both:. Go is very careful about both the stability of its standard library and the coherency of its interfaces (no popen, popen2, subprocess). Something has to be traded off to get that; this is one of the things. OK!

[throwaway894345]

How does using the stdlib internally simplify things for callers? And what does that have to do with tanking inspiration from the stdlib?

On the second point, passing a []byte to something that really does not want a streaming interface is perfectly idiomatic per the stdlib.

I don’t think it complicates things for the caller if the author used a third party deciding function unless it produced a different type besides image.Image (and even then only a very minor inconvenience).

I also don’t think it’s the fault of the stdlib that it doesn’t provide high performance implementations of every function with every conceivable interface.

I do think there’s some reasonable critique to be made about the stdlib’s use of reflection to detect unofficial interfaces, but it’s also a perfectly pragmatic solution for maintaining compatibility while also not have the perfect future knowledge to build the best possible interface from day 0. :shrug:

[int_19h]

Because it forces the reader to read data into a temporary buffer in its entirety. If the thing this function is trying to do doesn't actually require it to do its job, that introduces unnecessary overhead.

[mbrumlow]

What? Where else would it be?

It’s either in the socket(and likely not fully arrived) or … in a buffer.

Peak is not some magic, it is well a temporary buffer.

Beyond that, I keep seeing people ask for a byte interface. Has anybody looked at the IO.reader interface ???

type Reader interface { Read(p []byte) (n int, err error) }

You can read as little or as much as you would like and you can do this at any stage of a chain if readers.

[vjerancrnjak]

That's how leaky abstraction of many file std implementations starts.

Reading into a byte buffer, pass in a buffer to read values, pass in a buffer to write values. Then OS does the same thing, has its own buffer that accepts your buffer, then the underlying storage volume has its own buffer.

Buffers all the way down to inefficiency.

[woah]

Seems pretty crazy to force a bunch of data to be saved into memory all the time just for programming language aesthetic reasons

[0points]

When you are working with streaming data, you really should be passing around io.Readers if you want any sort of performance out of it.

A []byte require you to read ALL data in advance.

And if you still end up with []byte and need to use a interface taking io.Reader, then you wrap []byte in a bytes.Buffer which implements io.Reader.

[Seb-C]

100% this, that is the easiest and less error prone way to do it.

Even if the author still insisted on using a single interface, he could also do what he wants by relying on bytes.Buffer rather than bytes.Reader.

[silverwind]

A good API should just accept either,e.g. the union of []byte and io.Reader.

Both have pros and cons and those should be for the user to decide.

[throwaway894345]

Nah, a good API doesn’t push the conditionals down. You don’t need to pass a union to let the user decide, you just need to present an API for each (including a generic implementation that monomorphizes into multiple concrete implementations) https://matklad.github.io/2023/11/15/push-ifs-up-and-fors-do...

[thayne]

Ah, but go doesn't have union types.

[dgb23]

Personally I rarely use or even implement interfaces except some other part needs them. My brain thinks in terms of plain data by default.

I appreciate how they compose, for example when I call io.Copy and how things are handled for me. But when I structure my code that way, it’s extra effort that doesn’t come naturally at all.

[lanstin]

I use them for testing, where I can have a client that is called by the code under test and can either just run a test CB, send a REST call to a remote server, send a gRPC call to a remote server, or make a function call to an in-process gRPC server object.