Update robots.txt comments

clarify comments and update permanently moved URL

public/robots.txt

@@ -1,6 +1,10 @@
-# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
+# See http://www.robotstxt.org/orig.html for documentation on how to use the robots.txt file
 #
 # To ban all spiders from the entire site uncomment the next two lines:
+# User-Agent: *
+# Disallow: /
+
+# To ban all spiders from only specific directories such as /people /u or /tag etc.  
 User-Agent: *
 Disallow: /people/
 Disallow: /u/