Recap of Last Lesson: We covered explored the files within Natas2 through viewing indexed directories.
Find the password to log into level 4.
- URL: http://natas3.natas.labs.overthewire.org/
Same as always, we get very little info on the page. Using our tactics from earlier exercises, we view the source.
<!-- No more information leaks!! Not even Google will find it this time... -->
Not even Google, huh? How would a webmaster go against having Google indexing their website? Lets go ask Google.
Google describes this process for us:
“Site owners have many choices about how Google crawls and indexes their sites through Webmaster Tools and a file called “robots.txt”. With the robots.txt file, site owners can choose not to be crawled by Googlebot, or they can provide more specific instructions about how to process pages on their sites. ”
Lets dig a little deeper. The /robots.txt is a de-facto standard, which means it is not published by any governing body but it is universally accepted. To learn more about this file we can go to http://www.robotstxt.org/robotstxt.html as they describe how to use this file. They suggest putting this file in the top level of the directory, so lets go look there.
Aha. Looks like we found our “hidden” directory. Inside the directory, we find exactly what we were looking for. Easy peasy. *WARNING* keep in mind that this file will stop honest crawlers (like google) from indexing your website. It will not stop hackers, and they make look for this to crawl specifically.