[heads]

Put wings on it and go up sideways. It solves the thrust/weight issue the same way a hill climber does: instead of going directly up they zig zag.

You need the lateral velocity anyway, for orbit.

[sjducb]

Now your rocket needs to be strong horizontally and vertically, rather than just vertically. You have a much heavier rocket.

[NikkiA]

Why would you choose to lose fuel/acceleration to changing direction?