#User-agent: * User-agent: * Disallow: / # Allowed Crawlers User-agent: Googlebot User-agent: Bingbot User-agent: Slurp User-agent: DuckDuckBot User-agent: Baiduspider User-agent: YandexBot User-agent: FreeFind User-agent: ia_archiver User-agent: W3C_Validator/1.3 User-agent: W3C-checklink User-Agent: Jigsaw/2.3.0 W3C_CSS_Validator_JFouffa/2.0 User-agent: FATVCrawler User-agent: OUSearch User-agent: PowerMapper Allow: / # File disallows Disallow: /*.php$ Disallow: /*.asp$ Disallow: /*.docx$ # Disallow: /*.css$ Disallow: /_archive/ Disallow: /cgi-bin/ Disallow: /common/ Disallow: /district_info/assets/ Disallow: /district_info/busstops/ Disallow: /district_info/directory/ Disallow: /district_info/notifications/ Disallow: /district_info/portal/ Disallow: /district_info/redesign/ Disallow: /district_info/special_events/ Disallow: /district_info/tour/ # Disallow: /directory/ Disallow: /error_files/ Disallow: /buttecc/ Disallow: /icons/ Disallow: /images/ Disallow: /img/ Disallow: /inc/ Disallow: /login/ Disallow: /mason/ Disallow: /migration/ Disallow: /OMNI-INF/ Disallow: /usage/ Disallow: /wiki/ Disallow: /z-omniupdate/ Disallow: /notifications/ Disallow: /b/ # image folders of PODs Disallow: /business/images/ Disallow: /community/images/ Disallow: /currentstudents/images/ Disallow: /highschool/images/ Disallow: /newandreturning/images/ # document folders of PODs Disallow: /community/about/pba/documents/ Disallow: /community/about/pba/Committee%20Documents/ Disallow: /departments/test/ Disallow: /departments/careertech/familystudies/faculty/syllabi/ # New Working Directories hidden until release Disallow: /sandbox Disallow: /departments/learning/office-of-vp-learning/ Disallow: /services/webservices Disallow: /departments/learning/research # resource folder hidden and other sh!t # Disallow: /resources/ Disallow: /departments/careertech/automotive/appt_confirmed.html # hide until the page's fate is determined Disallow: /community/about/title3 Disallow: /SSProjects Disallow: /mobile Disallow: /resource/interim/wmwu Disallow: /services/slv/placementsites Disallow: /departments/careertech/draftingcad-onhold Disallow: /departments/distance Disallow: /resources/schedule Disallow: /feedback Disallow: /currentstudents/help_on_campus/help_unique/eops # DEV hide until ready for public viewing Disallow: /departments/learning/esl Disallow: /departments/learning/ece Disallow: /counseling/onlinecounseling.html Disallow: /counseling/onlinecounseling-instruction.html Disallow: /programs/createspace Disallow: /services/fkce Disallow: *.pdf$ Disallow: /harming/humans Disallow: /ignoring/human/orders Disallow: /harm/to/self Disallow: /dissing/the/college Sitemap: http://www.butte.edu/sitemap.xml