Google SEO Training How Googlebot Finds Your Site

October 21, 2008 by Victoria Stankard  
Filed under Google SEO Training

Once you’ve finished building an amazing website and are now ready to have it indexed in Google, you’ll want to make sure that Google knows about your site’s pages. Chances are good that Google may already know of your pages if at least one other quality website has included a relevant link to your your site on their site. Of course, acquiring such links is a great way for Google to learn about your website and pages.

XML Sitemaps

So that people know about all of your pages, you can submit what’s called an XML sitemap. Whether your site is brand new or has been around for awhile, it’s highly recommended that you create an XML sitemap, which helps Google and the other search engines to better understand the pages on your site and find them.

These can be very useful for sites that are:

  • Dynamically driven
  • Have a large amount of new and updated pages
  • Have few incoming links

You can create a general XML sitemap in just minutes as well as XML sitemaps for other uses such as videos. Keep in mind that this isn’t the same as an HTML site map or one that is user visible. HTML sitemaps can compliment XML site maps and help people quickly find and easily navigate to the content that’s deep within the pages of your site.

Pages You Don’t Want Indexed

More than likely, there will be some pages that you don’t want Google to be able to access. For example, you may not want Google’s automated page fetching robot called Googlebot to access documents that contain private information or pages that you are not ready to show to the world yet.

In these types of situations, you can use use one of two reliable methods for blocking Google’s access from being able to obtain specific content. You can use a disallow line in the robots.text file or use a no index meta tag on each of the pages that you don’t want indexed.

If you're new here, you may want to subscribe to my RSS feed. Thanks for visiting!