How do you prevent templates from being indexed/listed?

[lesliekirk]

Do you use the marketing feed to generate a sitemap? If so, is "Include pages in the sitemap" checked?

As previously mentioned - no.

[colin]

Do you use the marketing feed to generate a sitemap? If so, is "Include pages in the sitemap" checked?

[alphabet]

I agree with Kent but just want to add to first double check with the domain's Google Search Console to see if there are any internal or external links pointing to the /CTGY.html page.

If so, you should fix the internal links pointing to /CTGY.html and if the external links have any SEO value (doubtful) then create a redirect uri.

Otherwise just delete the uri.

[lesliekirk]

I would think so, yes. The category page should not have a URI, because it makes no sense to view it except when a specific category is requested. So the categories have URIs, not the page template.

When you enter that generic URL into your browser, what do you see?

When I go to the CTGY page in admin and click the View button, I get an empty category page. It has a header and footer but no content: no products or subcategories.

I see the page that Google has indexed. It looks very much like any other page on the website, Global Header, Global Footer, cattree and the sub-category display. But I went to see what canonical link it has and it shows this:

<link rel="canonical" href="" itemprop="url">

All removing the Canonical URI from the template URI and then click the Preview Icon just pulls the template up with the long style link https://www.domain.com/Merchant2/mer...vc?Screen=CTGY and the canonical href is still empty. You would think it would have something in it:

<mvt:if expr="'CTGY' CIN l.settings:page:code">
<link rel="canonical" href="&mvte:category:link;" itemprop="url">
<mvt:elseif expr="'PROD' CIN l.settings:page:code OR l.settings:page:code EQ 'PATR'">
<link rel="canonical" href="&mvt:product:link;" itemprop="url">
<mvt:elseif expr="'blog' CIN l.settings:page:code">
<mvt:else>
<link rel="canonical" href="&mvte:urls:_self:auto;" itemprop="url">
</mvt:if>

[Bruce - PhosphorMedia]

I would think so, yes. The category page should not have a URI, because it makes no sense to view it except when a specific category is requested. So the categories have URIs, not the page template.

When you enter that generic URL into your browser, what do you see?

When I go to the CTGY page in admin and click the View button, I get an empty category page. It has a header and footer but no content: no products or subcategories.

Not sure about all sites but everyone of 6 show top level cats with /mm5/merchant.mvc>Screen=CTGY
Also tested, those with URIs populated CANNOT BE removed. You have to block bot access in some other way...

[Kent Multer]

There is an entry /CTGY.html - should it be removed?

I would think so, yes. The category page should not have a URI, because it makes no sense to view it except when a specific category is requested. So the categories have URIs, not the page template.

When you enter that generic URL into your browser, what do you see?

When I go to the CTGY page in admin and click the View button, I get an empty category page. It has a header and footer but no content: no products or subcategories.

[Bruce - PhosphorMedia]

you can't leave the URI blank once its been populated.

[lesliekirk]

Just a few ideas:

Have you looked at the URI Management settings and see if there is a rule that is creating a default page template URI's?

Or have you checked the URI tab of the CTGY page to see that there are no URI entries?

Or try a Google search with the link: operator

There is an entry /CTGY.html - should it be removed?

[Bruce - PhosphorMedia]

Just a few ideas:

Have you looked at the URI Management settings and see if there is a rule that is creating a default page template URI's?

Or have you checked the URI tab of the CTGY page to see that there are no URI entries?

Or try a Google search with the link: operator

if URI Management is turned on, by default it creates a URI when a page is created.

[Bruce - PhosphorMedia]

the /CTGY.html page is a legit page on most miva sites due to either htaccess rules or URI settings for example, default shadows has

/category-display.html

as a loadable page (kent: a call to the category page without a category code should load all to level cats.)

as for 'messing' with the robots file--that's what its there for. for example, we'll often block all urls starting with /mm5/ to force bots to index the friendly URIs. Obvoiously, that requires what we "allow" *.jpg, *.css, *.js files to be indexed.

there are some great online robot file builders and testors out there (sorry don't have one currently bookmarked) and no harm would come from a 20 minute test.

[alphabet]

Just a few ideas:

Have you looked at the URI Management settings and see if there is a rule that is creating a default page template URI's?

Or have you checked the URI tab of the CTGY page to see that there are no URI entries?

Or try a Google search with the link: operator

[lesliekirk]

Maybe I'm missing something, but this seems weird to me. I don't see how a URL like https://www.domain.com/CTGY.html can even display a page. Since there is no category code, it ought to either generate a not-found error, or redirect to the storefront.

Maybe there is a form on a page that's submitting this URL and passing the Category_Code in a hidden field? If so, you can just change the form to use the store's main URL, and put the "CTGY" in a hidden field with the name "Screen."

Or maybe there's a bug in their feed, and they are submitting invalid URLs to the search engines?

I guess what I'm saying is, I don't understand why a store would use such a URL. There may be a bug somewhere, and fiddling with things like robots.txt is not a real solution. Sorry if I'm off base; no offense intended to anyone.

No offense - I'm just trying to figure out how Google picked up the link. It is not in the sitemap for the store. If you go in an MM10 admin, go to any of the of the Page Templates, then click the "View" icon. You'll be able to see the page.

[Kent Multer]

Maybe I'm missing something, but this seems weird to me. I don't see how a URL like https://www.domain.com/CTGY.html can even display a page. Since there is no category code, it ought to either generate a not-found error, or redirect to the storefront.

Maybe there is a form on a page that's submitting this URL and passing the Category_Code in a hidden field? If so, you can just change the form to use the store's main URL, and put the "CTGY" in a hidden field with the name "Screen."

Or maybe there's a bug in their feed, and they are submitting invalid URLs to the search engines?

I guess what I'm saying is, I don't understand why a store would use such a URL. There may be a bug somewhere, and fiddling with things like robots.txt is not a real solution. Sorry if I'm off base; no offense intended to anyone.

[William Davis]

No, you only want to block one category:

Example:

Disallow: /cat-abc.html

https://www.robotstxt.org/robotstxt.html

[lesliekirk]

I think william means something like this:

disallow: https://www.domain.com/CTGY.html

in the robots.txt file.

i don't think its an issue but just to be safe, you can use Web Master Tools to verify that a category USING that page template can still be followed and indexed.

Well that's a little scary - so I could potentially block all of the category pages by doing this?