[maketheman]

1) Find a few brands with more options than just "S/M/L". You want fit variants, like Brooks Brothers' "Milano/Regent/Madison" (and some others) or Jcrew's "slim/extra slim". If button-ups, consider going with neck & arm sizing plus the above cut-variants, instead of "S/M/L" sizing at all. Also, look into Japanese brands (Kamakura?), they might fit you better even if they only have one or two variants of cut. These brands will tend to have pretty consistent sizing.

2) You're gonna want measurements of your own actual chest, neck, and natural waist. Easy to DIY close enough for this, no need for help.

3) Also, measurements of some clothes that fit you well. Shoulder, seam-to-seam; arm length; pit-to-pit width.

4) The brands from #1 will have charts that provide measurements for their clothes, or else suggested body measurements at a given size. Use the info from 2 & 3 to get good guesses at which sizes and fit variants will work.

5) Thrift or ebay some options. You can resell the ones that don't work out for close to what you bought them for, if it's worth it to you, or just dump them at goodwill. You're paying a few tens or a little over a hundred dollars in failures to find your sizing in a few brands. This is basically the only totally-lost cost to this process (and again, you can actually recover most of this if it's worth it to you)

6) Now you know your size in a few good consistent-sized brands. Ebay with your exact size in each brand (use saved searches, like "(jcrew,j.crew,j crew) 'extra slim' medium shirt") or sale-shop directly from the brands' sites. No, or very few, returns needed. You can also thrift-shop much more efficiently, at least at places that size-sort. You're just skimming for brand tags you recognize, and totally ignoring everything else.

[EDIT] And actually with the measurements from #3 you can ebay lots of shirts from brands you've never tried, with reasonable success rates—any listing worth bothering with will provide pit-to-pit and (when relevant) sleeve measurements, at least.