[knute]

/*/tree is only for directory listings. File contents will be under a /blob/ path, e.g. https://github.com/facebook/react/blob/main/AUTHORS, and should be, AFAIK, indexable.

(mandatory disclaimer: I'm a GitHub employee, not speaking on behalf of the company)

[simonw]

I asked about this on the support forum a while ago and never got a satisfactory response: https://github.com/community/community/discussions/20958

[staplung]

If they can't hit `/*/tree` is there a way to know the URLs of the files?

[pancrufty]

Direct links from crawlable pages

[kadoban]

Sure, clone the git repo.