[calebtv]

Thanks! The process function runs as a Ray Actor (https://docs.ray.io/en/latest/ray-core/actors.html). So we have the same serialization requirements as Ray (https://docs.ray.io/en/latest/ray-core/objects/serialization...)

I think the most common limitation will be ensure that your output is serializable. Typically returning python dictionaries or dataclasses is fine.

But if you had a specific limitation in mind let me know happy to dive into it!

[calebtv]

One other thing I should mention that's relevent, we do also have a class abstraction instead of a decorator: https://github.com/launchflow/buildflow/blob/main/buildflow/...

This can help with things like setting up RPC clients. But it all boils down to the same runner whether you're using the class or decorator.

[fhenrywells]

Do you see this as a direct competitor to Ray's built-in workflow abstraction https://docs.ray.io/en/latest/workflows/management.html

Exciting to see more libraries built on Ray in any case!

[calebtv]

Great question! We actually looked at using the workflow abstraction for batch processing in our runner, but ultimately didn't because it was still in alpha (we use the dataset API for batch flows).

I think one area where we differ is our focus on streaming processing which I don't think is well supported with the workflow abstraction, and also having more resource management / use case driven IO.