[ocfnash]

I think it is worth comparing this problem with the question of the behaviour of a particle placed at the apex of a cone. I claim it is clear that in this case, the problem is clearly not well-posed because the apex is a singular point: the slope at the apex is undefined. The singular nature of the slope (first derivative) is the issue.

This "dome" is essentially the same issue just with the singularity buried one level deeper: you need to take second derivatives to see it. Indeed a planar cross section containing the vertical axis through its center is a graph of the equation $y^2 = |x|^3$ (up to constants) and this is not twice differentiable at $x = y = 0$. Newtonian mechanics is governed by a second order differential equation, so we need a C^2 regularity assumption to get uniqueness.

So for me there is not really any more philosophically interesting than the question about a particle balancing at the apex of a cone.