Allow ,Disallow of specific files in websites use robots.txt file for this ,you will learned alot follow this link :http://www.robotstxt.org/orig.html