Help with Short Links

[agarvey]

I am using Bill's toolkit for short links on this site: www.bedwettingstore.com. From the front end everything is working fine. Google Webmaster tools seems to be giving me issues however.

Here is my current struggle:

When I am on this page: http://bedwettingstore.com/bedwetting-alarms.html and use the alarm chooser, the site will take me to correct pages with correct urls, for example http://bedwettingstore.com/sleep-dry...for-girls.html.

BUT

When I submitted these pages to the sitemap, google is having issues and says that they are restricted by robots.txt. Strange because they look like the right url http://bedwettingstore.com/sleep-dry...for-girls.html, but when I copy and paste the url into the browser, I get this url: http://bedwettingstore.com/mm5/merch...tion-for-girls which would be restricted by our robots.txt file because of this statement:

Disallow: /mm5/merchant.mvc
So, to give you a summary of my htaccess, we have a bunch of 301 redirects because we just moved our site and the url structure is completely different. These are at the top:

...
Redirect 301 /watch-timex.htm http://bedwettingstore.com/timex-mid...le-watch.html?
Redirect 301 /bedwetting-malemwireless.htm http://bedwettingstore.com/malem-wir...m-system.html?
Redirect 301 /Blog http://bedwettingstore.com/blog/

#######################

DirectoryIndex index.php /mm5/merchant.mvc

RewriteRule ^mm5/admin.mvc? - [QSA,L]

#from Bill's toolkit module
RewriteCond %{REQUEST_FILENAME} !-s
RewriteRule ^([^/]+)-p-([^/.]+).html$ /mm5/merchant.mvc?Screen=PROD&Product_Code=$1&Category_ Code=$2&%{QUERY_STRING}

RewriteCond %{REQUEST_FILENAME} !-s
RewriteRule ^([^/.]+).html$ /mm5/merchant.mvc?Screen=TKSL&Code=$1&%{QUERY_STRING} [L]

RewriteCond %{REQUEST_FILENAME} !-s
RewriteRule ^product/([^/]+)/([^/.]+).html$ /mm5/merchant.mvc?Screen=PROD&Category_code=$1&Product_ code=$2 [QSA,L]

RewriteCond %{REQUEST_FILENAME} !-s
RewriteRule ^([^/.]+).html$ /mm5/merchant.mvc?Screen=$1 [QSA,L]

#For removing the www
RewriteCond %{HTTP_HOST} !^bedwettingstore.com$ [NC]
RewriteRule ^(.*)$ http://bedwettingstore.com/$1 [L,R=301]

#Options +FollowSymLinks
#RewriteEngine On
#RewriteRule ^(index-5\.html|mobilesurvey\.html|google-exempt\.html|whats_new\.html|order-status\.html|resize\.html|review\.html|specials\.h tml) - [L]
#RewriteRule ^([A-Za-z0-9_-]+)\.html$ /mm5/merchant.mvc?Screen=USL&Code=$1 [T=application/x-httpd-mv]

##Rules for blog
<IfModule mod_rewrite.c>
RewriteBase /blog
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} ^/blog
RewriteRule . /blog/index.php [L]
</IfModule>

## Rule for css3 PIE inserted by clark 10/14/11
AddType text/x-component .htc

##Rule for redirecting anything with rate_item in url to home page
RewriteCond %{REQUEST_URI} /Customer_Service/rate_item.htm
RewriteRule (.*) /? [R,L]

##Rule for redirecting anything with index in url to home page
RewriteCond %{REQUEST_URI} /index-5.html
RewriteRule (.*) /? [R,L]

##Rule for eliminating sub-directory Bedwetting_Alarms from url for reviews
RewriteCond %{REQUEST_URI} /Bedwetting_Alarms/review.html
RewriteRule (.*) /review.html [R,L]

##Rule for redirecting anything with zoom_query
RewriteCond %{REQUEST_URI} /search/search.php?
RewriteRule (.*) /? [R,L]

Why would google be finding the TKSL pages to begin with?

Thanks

[Siamese-Dream.Com]

Re: Help with Short Links

Have you tried using fetch with google bot to try out both the short links version and the long query string version of the URLs???

That should be the first thing you try.

also, it looks like your short links are indexed in the google search results pages (SERPs) so it is probably working fine and you just need to clear your cache.

I hope this helps.

[agarvey]

Re: Help with Short Links

Thanks for the response. I think I solved the problem.
1) I had submitted www.bedwettingstore.com in my sitemap when I should have just submitted bedwettingstore.com
2) This statement came after our short links code.

#For removing the www
RewriteCond %{HTTP_HOST} !^bedwettingstore.com$ [NC]
RewriteRule ^(.*)$ http://bedwettingstore.com/$1 [L,R=301]

So I believe it was interpreting the short link rule and then stopping--never getting to the www rule. Not 100% sure why this helps, but it seems to. I will have to keep an eye on webmaster tools to see if that solves the problem.

Also, you were right about the cache being an issue. Even after I made the change, I still needed to clear the cache.