[lurker456]

You should, because anyone that can compromise your phone will be able to get into your email.

From there it's a small step to reset passwords (SMS 2FA won't help here, as they also have your phone) to all online services you signed up for with that email.

[kartickv]

Yes, but you need to weigh that risk against the risk of not having 2FA.

Taking a step back, and responding to the other comment in response to mine as well, I was just speculating. I don't work in abuse, and I'm inclined to trust the Google abuse engineers over myself or random HN commentators to keep my Google account safe.