# # robots.txt # # This file is to prevent the crawling and indexing of certain parts # of your site by web crawlers and spiders run by sites like Yahoo! # and Google. By telling these "robots" where not to go on your site, # you save bandwidth and server resources. # # This file will be ignored unless it is at the root of your host: # Used: http://example.com/robots.txt # Ignored: http://example.com/site/robots.txt # # For more information about the robots.txt standard, see: # http://www.robotstxt.org/robotstxt.html # # For syntax checking, see: # http://www.frobee.com/robots-txt-check User-agent: * Disallow: /tags-0/ Disallow: /tags-1/ Disallow: /tags-10/ Disallow: /tags-11/ Disallow: /tags-12/ Disallow: /tags-13/ Disallow: /tags-14/ Disallow: /tags-15/ Disallow: /tags-16/ Disallow: /tags-17/ Disallow: /tags-18/ Disallow: /tags-19/ Disallow: /tags-2/ Disallow: /tags-20/ Disallow: /tags-21/ Disallow: /tags-22/ Disallow: /tags-23/ Disallow: /tags-24/ Disallow: /tags-25/ Disallow: /tags-26/ Disallow: /tags-27/ Disallow: /tags-28/ Disallow: /tags-29/ Disallow: /tags-3/ Disallow: /tags-30/ Disallow: /tags-31/ Disallow: /tags-32/ Disallow: /tags-33/ Disallow: /tags-34/ Disallow: /tags-35/ Disallow: /tags-36/ Disallow: /tags-37/ Disallow: /tags-38/ Disallow: /tags-39/ Disallow: /tags-4/ Disallow: /tags-40/ Disallow: /tags-41/ Disallow: /tags-42/ Disallow: /tags-43/ Disallow: /tags-44/ Disallow: /tags-45/ Disallow: /tags-46/ Disallow: /tags-47/ Disallow: /tags-5/ Disallow: /tags-7/ Disallow: /tags-8/ Disallow: /tags-9/ Disallow: /categories-0/ Disallow: /bringoutthebold/ Disallow: /*bvstate* Disallow: /*bvrrp* Disallow: /where-to-buy/* Disallow: /janrain/forgot?* Disallow: /file/ Disallow: /?q=file/ # Assets # JS/CSS Allow: /misc/*.js Allow: /misc/*.css Allow: /modules/*.js Allow: /modules/*.css Allow: /profiles/*.js Allow: /profiles/*.css Allow: /themes/*.js Allow: /themes/*.css # Images Allow: /misc/*.jpg Allow: /misc/*.gif Allow: /misc/*.png Allow: /misc/*.jpeg Allow: /modules/*.jpg Allow: /modules/*.gif Allow: /modules/*.png Allow: /modules/*.jpeg Allow: /profiles/*.jpg Allow: /profiles/*.gif Allow: /profiles/*.png Allow: /profiles/*.jpeg Allow: /themes/*.jpg Allow: /themes/*.gif Allow: /themes/*.png Allow: /themes/*.jpeg Disallow: /*bvstate* Disallow: /*bvrrp* Disallow: /where-to-buy/* Sitemap: https://www.listerine.co.uk/sitemap.xml Disallow: /janrain/forgot?* Disallow: /file/ Disallow: /?q=file/ Disallow: /quizzes/results/*