[porridgeraisin]

This is the key. Personally I think you just have to something similar to an auditor or whatever. Demand that if a self driving taxi operates in your city, they assign one legally responsible person per $major-division-of-city. All accidents in that region are due to that guy.

Naturally, this will incentivise them to improve the system that deals with edge cases in their ML model, and better yet you'll have the legally responsible guy shit himself and directly manage remote drivers for his location himself. Adds another layer of accountability.