User-Agent: * Disallow: /noidx/ Disallow: /files/noidx/ Disallow: /drafts/ Disallow: /private/ Disallow: /placeholder.html # redundant indexes, but meta tagged #Disallow: /tags.html #Disallow: /tag/ #Disallow: /authors.html #Disallow: /author/ #Disallow: /categories.html #Disallow: /cat/ Sitemap: https://rya.nc/sitemap.xml User-Agent: Twitterbot Allow: /private/ User-Agent: Googlebot Allow: /private/