WingNews logo WingNews
top | item 46039721

(no title)

ggirelli | 3 months ago

Out of curiosity, I see my website not being well indexed and I wondered whether it is because I include ``` User-agent: * Disallow: / ``` in my robots.txt. What should one add to allow Marginalia to crawl their website?

discuss

order

marginalia_nu|3 months ago

This should work

  User-agent: search.marginalia.nu
  Allow: /

ggirelli|3 months ago

Thank you! I just added it :)