Announcement

Collapse
No announcement yet.

How do you prevent templates from being indexed/listed?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    How do you prevent templates from being indexed/listed?

    I'm not sure if this has come up before but I've noticed that Page Templates such as CTGY are being picked up by the search engines as https://www.domain.com/CTGY.html. If you put "noindex,nofollow" in the template it will prevent every category from being listed. I suspect this started becoming an issue when we were able to start viewing the Page Templates. This particular site is not using the Marketing Feeds Sitemap, it uses an external sitemap app.The link is not in it nor is it in the SMAP page (thank goodness). It would seem Google is finding it all on it's own. So how do we stop it?

    Leslie Kirk
    Miva Certified Developer
    Miva Merchant Specialist since 1997
    Previously of Webs Your Way
    (aka Leslie Nord leslienord)

    Email me: [email protected]
    www.lesliekirk.com

    Follow me: Twitter | Facebook | FourSquare | Pinterest | Flickr

    #2
    Originally posted by lesliekirk View Post
    I'm not sure if this has come up before but I've noticed that Page Templates such as CTGY are being picked up by the search engines as https://www.domain.com/CTGY.html. If you put "noindex,nofollow" in the template it will prevent every category from being listed. I suspect this started becoming an issue when we were able to start viewing the Page Templates. This particular site is not using the Marketing Feeds Sitemap, it uses an external sitemap app.The link is not in it nor is it in the SMAP page (thank goodness). It would seem Google is finding it all on it's own. So how do we stop it?
    How about robots.txt disallow statement for the category in question file?
    Thank you, Bill Davis

    Comment


      #3
      Originally posted by William Davis View Post

      How about robots.txt disallow statement for the category in question file?
      It's not a category - it's the Page Template (code CTGY name Category Display). This issue seems to have started when MM10 started letting you view a Page Template just by clicking the View icon. Maybe it's been a problem all along but this time I have encountered the Template Page being listed in Google.
      Leslie Kirk
      Miva Certified Developer
      Miva Merchant Specialist since 1997
      Previously of Webs Your Way
      (aka Leslie Nord leslienord)

      Email me: [email protected]
      www.lesliekirk.com

      Follow me: Twitter | Facebook | FourSquare | Pinterest | Flickr

      Comment


        #4
        I think william means something like this:

        disallow: https://www.domain.com/CTGY.html

        in the robots.txt file.

        i don't think its an issue but just to be safe, you can use Web Master Tools to verify that a category USING that page template can still be followed and indexed.
        Bruce Golub
        Phosphor Media - "Your Success is our Business"

        Improve Your Customer Service | Get MORE Customers | Edit CSS/Javascript/HTML Easily | Make Your Site Faster | Get Indexed by Google | Free Modules | Follow Us on Facebook
        phosphormedia.com

        Comment


          #5
          Originally posted by Bruce - PhosphorMedia View Post
          I think william means something like this:

          disallow: https://www.domain.com/CTGY.html

          in the robots.txt file.

          i don't think its an issue but just to be safe, you can use Web Master Tools to verify that a category USING that page template can still be followed and indexed.
          Correct!
          Thank you, Bill Davis

          Comment


            #6
            Originally posted by Bruce - PhosphorMedia View Post
            I think william means something like this:

            disallow: https://www.domain.com/CTGY.html

            in the robots.txt file.

            i don't think its an issue but just to be safe, you can use Web Master Tools to verify that a category USING that page template can still be followed and indexed.
            Well that's a little scary - so I could potentially block all of the category pages by doing this?
            Leslie Kirk
            Miva Certified Developer
            Miva Merchant Specialist since 1997
            Previously of Webs Your Way
            (aka Leslie Nord leslienord)

            Email me: [email protected]
            www.lesliekirk.com

            Follow me: Twitter | Facebook | FourSquare | Pinterest | Flickr

            Comment


              #7
              No, you only want to block one category:

              Example:

              Disallow: /cat-abc.html

              https://www.robotstxt.org/robotstxt.html
              Thank you, Bill Davis

              Comment


                #8
                Maybe I'm missing something, but this seems weird to me. I don't see how a URL like https://www.domain.com/CTGY.html can even display a page. Since there is no category code, it ought to either generate a not-found error, or redirect to the storefront.

                Maybe there is a form on a page that's submitting this URL and passing the Category_Code in a hidden field? If so, you can just change the form to use the store's main URL, and put the "CTGY" in a hidden field with the name "Screen."

                Or maybe there's a bug in their feed, and they are submitting invalid URLs to the search engines?

                I guess what I'm saying is, I don't understand why a store would use such a URL. There may be a bug somewhere, and fiddling with things like robots.txt is not a real solution. Sorry if I'm off base; no offense intended to anyone.
                Kent Multer
                Magic Metal Productions
                http://TheMagicM.com
                * Web developer/designer
                * E-commerce and Miva
                * Author, The Official Miva Web Scripting Book -- available on-line:
                http://www.amazon.com/exec/obidos/IS...icmetalproducA

                Comment


                  #9
                  Originally posted by Kent Multer View Post
                  Maybe I'm missing something, but this seems weird to me. I don't see how a URL like https://www.domain.com/CTGY.html can even display a page. Since there is no category code, it ought to either generate a not-found error, or redirect to the storefront.

                  Maybe there is a form on a page that's submitting this URL and passing the Category_Code in a hidden field? If so, you can just change the form to use the store's main URL, and put the "CTGY" in a hidden field with the name "Screen."

                  Or maybe there's a bug in their feed, and they are submitting invalid URLs to the search engines?

                  I guess what I'm saying is, I don't understand why a store would use such a URL. There may be a bug somewhere, and fiddling with things like robots.txt is not a real solution. Sorry if I'm off base; no offense intended to anyone.
                  No offense - I'm just trying to figure out how Google picked up the link. It is not in the sitemap for the store. If you go in an MM10 admin, go to any of the of the Page Templates, then click the "View" icon. You'll be able to see the page.

                  Leslie Kirk
                  Miva Certified Developer
                  Miva Merchant Specialist since 1997
                  Previously of Webs Your Way
                  (aka Leslie Nord leslienord)

                  Email me: [email protected]
                  www.lesliekirk.com

                  Follow me: Twitter | Facebook | FourSquare | Pinterest | Flickr

                  Comment


                    #10
                    Just a few ideas:

                    Have you looked at the URI Management settings and see if there is a rule that is creating a default page template URI's?

                    Or have you checked the URI tab of the CTGY page to see that there are no URI entries?

                    Or try a Google search with the link: operator



                    http://www.alphabetsigns.com/

                    Comment


                      #11
                      the /CTGY.html page is a legit page on most miva sites due to either htaccess rules or URI settings for example, default shadows has

                      /category-display.html

                      as a loadable page (kent: a call to the category page without a category code should load all to level cats.)

                      as for 'messing' with the robots file--that's what its there for. for example, we'll often block all urls starting with /mm5/ to force bots to index the friendly URIs. Obvoiously, that requires what we "allow" *.jpg, *.css, *.js files to be indexed.

                      there are some great online robot file builders and testors out there (sorry don't have one currently bookmarked) and no harm would come from a 20 minute test.
                      Bruce Golub
                      Phosphor Media - "Your Success is our Business"

                      Improve Your Customer Service | Get MORE Customers | Edit CSS/Javascript/HTML Easily | Make Your Site Faster | Get Indexed by Google | Free Modules | Follow Us on Facebook
                      phosphormedia.com

                      Comment


                        #12
                        Originally posted by alphabet View Post
                        Just a few ideas:

                        Have you looked at the URI Management settings and see if there is a rule that is creating a default page template URI's?

                        Or have you checked the URI tab of the CTGY page to see that there are no URI entries?

                        Or try a Google search with the link: operator
                        if URI Management is turned on, by default it creates a URI when a page is created.
                        Bruce Golub
                        Phosphor Media - "Your Success is our Business"

                        Improve Your Customer Service | Get MORE Customers | Edit CSS/Javascript/HTML Easily | Make Your Site Faster | Get Indexed by Google | Free Modules | Follow Us on Facebook
                        phosphormedia.com

                        Comment


                          #13
                          Originally posted by alphabet View Post
                          Just a few ideas:

                          Have you looked at the URI Management settings and see if there is a rule that is creating a default page template URI's?

                          Or have you checked the URI tab of the CTGY page to see that there are no URI entries?

                          Or try a Google search with the link: operator


                          There is an entry /CTGY.html - should it be removed?
                          Leslie Kirk
                          Miva Certified Developer
                          Miva Merchant Specialist since 1997
                          Previously of Webs Your Way
                          (aka Leslie Nord leslienord)

                          Email me: [email protected]
                          www.lesliekirk.com

                          Follow me: Twitter | Facebook | FourSquare | Pinterest | Flickr

                          Comment


                            #14
                            you can't leave the URI blank once its been populated.
                            Bruce Golub
                            Phosphor Media - "Your Success is our Business"

                            Improve Your Customer Service | Get MORE Customers | Edit CSS/Javascript/HTML Easily | Make Your Site Faster | Get Indexed by Google | Free Modules | Follow Us on Facebook
                            phosphormedia.com

                            Comment


                              #15
                              Originally posted by lesliekirk View Post

                              There is an entry /CTGY.html - should it be removed?
                              I would think so, yes. The category page should not have a URI, because it makes no sense to view it except when a specific category is requested. So the categories have URIs, not the page template.

                              When you enter that generic URL into your browser, what do you see?

                              When I go to the CTGY page in admin and click the View button, I get an empty category page. It has a header and footer but no content: no products or subcategories.
                              Kent Multer
                              Magic Metal Productions
                              http://TheMagicM.com
                              * Web developer/designer
                              * E-commerce and Miva
                              * Author, The Official Miva Web Scripting Book -- available on-line:
                              http://www.amazon.com/exec/obidos/IS...icmetalproducA

                              Comment

                              Working...
                              X