their own docs also specify that the robots.txt does not stop indexing or showing up in search, they even bolded it "it is not a mechanism for keeping a web page out of Google"
The only way for links to appear in a Google search would be to host a public resource, that is linked from another public resource.
If you have specified in your robots.txt that you do not want the page(s) or directories ingested then only the url is indexed (if it is linked from another page). It does prevent the public display of the content of a page and creation description/summary.
threeseed|1 year ago
“While Google won't crawl or index the content blocked by a robots.txt file”
They will show the URL if someone else has linked to it. But the content itself is not indexed.
alphan0n|1 year ago
If you have specified in your robots.txt that you do not want the page(s) or directories ingested then only the url is indexed (if it is linked from another page). It does prevent the public display of the content of a page and creation description/summary.
https://support.google.com/webmasters/answer/7489871?hl=en