top | item 43373587

(no title)

joevandyk | 11 months ago

From https://rubyllm.com/#have-great-conversations

    # Stream responses in real-time
    chat.ask "Tell me a story about a Ruby programmer" do |chunk|
      print chunk.content
    end

discuss

order

jupp0r|11 months ago

This will synchronously block until ‘chat.ask’ returns though. Be prepared to be paying for the memory of your whole app tens/low hundreds of MB of memory being held alive doing nothing (other than handling new chunks) until whatever streaming API this is using under the hood is finished streaming.

andrewmutz|11 months ago

Threads?

kyledrake|11 months ago

That looks good, I didn't see that earlier.