top | item 34757412

(no title)

slewis | 3 years ago

The response here makes me think most commenters don’t have experience with this particular footgun.

To clarify: Python can gc your task before it starts, or during its execution if you don’t hold a ref to it yourself.

I can’t think of any scenario in which you’d want that behavior, and it is very surprising as a user.

Python should hold a ref to tasks until they are complete to prevent this. This also “feels right”, in that if I submit a task to a loop, I’d think the loop would have a ref to the task!

It’d be interesting to dig up the internal dev discussions to see why the “fix this” camp hasn’t won.

discuss

order

kevincox|3 years ago

I can see this behaviour being useful if you are no longer interested in the result of a "pure" task. For example imagine fetching some data via HTTP. If you no longer need to response canceling the request could make sense.

But I agree that this is unexpected and most code probably isn't ready for being cancelled at random points. (Although I guess in Python your code should be exception safe which is often similar)

account42|3 years ago

If you want to avoid an expensive operation in cases where the result is no longer needed then surely you'd want to cancel the task as soon as you know that and not at some indeterminate time when the GC runs so I don't think this behavior makes sense even for that scenario.

hgomersall|3 years ago

That would make tidying up rogue tasks impossible. Of course we all like to think we do cancellation perfectly, but it's nice to know that the task scheduler has your back.

Edit: I don't quite understand why a user would expect a task to remain live _after_ the last reference to it has been dropped...

int_19h|3 years ago

Because they have expressed the intent to run it by scheduling it on the event loop?

I don't follow the argument wrt tidying up rogue tasks. What does it mean for the task to be "rogue"? If there was some state change that made the task redundant - because it clearly wasn't when it was submitted! - then the code that makes that change, or some other code that observes it, should cancel it. If it isn't cancelled, the fact that nobody is able to observe the value that the task will yield is not sufficient to auto-cancel, as there may still be a dependency on side effects.

And, speaking of tidying up, what if the scheduled task is the one that performs some kind of cleanup?

BerislavLopac|3 years ago

A possible solution might be a built-in dunder flag to explicitly tell the interpreter not to get rid of a specific object. Something like __keep_alive__ or similar.