
16-05-2008, 11:41 AM
|
 |
Senior Webmaster
|
|
Join Date: Mar 2008
Location: pd
Posts: 200
Rep Power: 11
|
|
Noticed an improvement with sitemap?
I see Google reading the sitemap on one of my sites more than daily, but still almost all the URLs it reads (~400 a day) are extracted from pages. I generate my sitemaps using a script I wrote myself to produce working URLs that have minimal arguments in them (no language=, no currency=, etc), but the robot is still crawling duplicate pages that have different URL args.
Anybody else noticed that?
|