[brandmeyer]

Exactly what order and degree were you using to evaluate the model? Variations in drag and solar pressure are more significant than the uncertainty in the gravity field for objects in LEO somewhere much less than 127th order (40 microseconds on my machine, your smileage may vary), so you can safely truncate the model for simulations. GRACE worked by making many passes such that they could average out those perturbations to make their measurement. But for practical applications, those tiny terms are irrelevant.

IERS Technical Note 36 section 6.1 gives recommendations for model truncation if you are looking for justification. https://iers-conventions.obspm.fr/content/tn36.pdf