|
|
|
|
|
Google (sitemap.xml) and Yahoo! (urlist.txt) Sitemap Generator for Windows

Easy to use sitemap generator for windows that is completely free.
This software GSiteCrawler will take a listing of your websites URLs, let you edit the settings and generate Google
and Yahoo Sitemap files. However, the GSiteCrawler is very flexible and allows you to do a whole lot more than "just" that!
Capture URLs for your site using
- a normal website crawl - emulating a Googlebot, looking for all links and pages within your website
- an import of an existing Google Sitemap file
- an import of a server log file
- an import of any text file with URLs in it
The Crawler
- does a text-based crawl of each page, even finding URLs in javascript
- respects your robots.txt file
- respects robots meta tags for index / follow
- can run up to 15 times in parallel
- can be throttled with a user defined wait-time between URLs
- can be controlled with filters, bans, automatic URL modifications
With each page, it
- checks date (from the server of using a date meta-tag) and size of the page
- checks title, description and keyword tags
- keeps track of the time required to download and crawl the page
Once the pages are in the database, you can
- modify Google Sitemap settings like "priority" and "change frequency"
- search for pages by URL parts, title, description or keywords tags
- filter pages based on custom criteria - adjust their settings globally
- edit, add and delete pages manually
And you have everything the way you want it, you can export it as
- a Google Sitemap file in XML format sitemap.xml with or without the optional attributes like "change date", "priority" or "change frequency"
- a text URL listing for other programs (or for use as a UrlList for Yahoo!
urlist.txt
- a simple RSS feed
- Excel / CSV files with URLs, settings and attributes like title, description, keywords
- a ROR (Resources of Resources) XML file
- a static HTML sitemap file (with relative or absolute paths)
- a new robots.txt file based on your chosen filters
- ... or almost any type of file you want - the export function uses a user-adjustable text-based template-system
For more information, it also generates
- a general site overview with the number of URLs (total, crawlable, still in queue), oldest URLs, etc
- a listing of all broken URLs linked in your site (or otherwise not-accessable URLs from the crawl)
- an overview of your sites speed with the largest pages, slowest pages by total download time or download speed (unusually server-intensive pages), and those with the most processing time (many links)
- an overview of URLs leading to "duplicate content" - with the option of automatically disabling those pages for the Google Sitemap file
|
|
|
|