User-agent: * Disallow: /DataStore Disallow: /Application Disallow: /SisterClone/ Disallow: /index Disallow: /cgi-bin Disallow: /*.htm # The following command will disable just the directory "DataStore" and all its contents: # Disallow: /DataStore/ # # ..while the following command will disable both the "DataStore" directory and any file or # directory path starting with the text "/DataStore" (so "/DataStore.cfm", "/DataStore/*.cfm") # Disallow: /DataStore # # The "*" wildchar in file names (*.htm) is not supported by (all) the user-agents addressed by # this block of code. You should use the wildchar "*" in a block of code exclusively addressed # to spiders that support the wildchar (Eg. Googlebot).