Benchmarking Miva category tree vs file_read

[Mike521w]

Every now and then I want to use a file_read to include a static .html file on my site, and I always wonder if that might slow things down, especially if I'm doing it globally. Today I started tinkering with using a static .html instead of the category tree. There's a minor difference in page load time between the two, with static .html being faster, and I'm wondering if my test could be flawed in some way. If it's accurate, then I'm wondering why. Anyone have any ideas?

In general, using the category tree seemed to take around 800 microseconds, and using a static .html took around 100.

Here's how I tested:

top of head tag content (saves the page start time and current time, and creates a results variable which will be appended to and eventually displayed in the footer)

<mvt:if expr="NOT ISNULL g.mivaBenchmark">
    <mvt:assign name="benchmarkResults" value="''" />
    <mvt:assign name="benchmarkPageStart" value="s.dyn_tm_usec" />
    <mvt:assign name="benchmarkLast" value="g.benchmarkPageStart" />
    <mvt:assign name="benchmarkResults" value="g.benchmarkResults $ '<br>benchmarkPageStart = ' $ g.benchmarkPageStart" />
</mvt:if>

global header (decides between category tree or static include based on query string flag, marks the moment before and after)

<mvt:if expr="NOT ISNULL g.mivaBenchmark">
    <mvt:assign name="benchmarkCurrentTime" value="s.dyn_tm_usec" />
    <mvt:assign name="benchmarkTimeSinceLast" value="g.benchmarkCurrentTime - g.benchmarkLast" />
    <mvt:assign name="benchmarkLast" value="g.benchmarkCurrentTime" />
    <mvt:assign name="benchmarkResults" value="g.benchmarkResults $ '<br>before category tree:'" />
    <mvt:assign name="benchmarkResults" value="g.benchmarkResults $ '<br>--benchmarkTimeSinceLast = ' $ g.benchmarkTimeSinceLast" />
    <mvt:assign name="benchmarkResults" value="g.benchmarkResults $ '<br>--benchmarkCurrentTime = ' $ g.benchmarkCurrentTime" />
</mvt:if>

<mvt:if expr="NOT ISNULL g.mivaTestUseCatTree">
    <mvt:item name="category_tree" />
<mvt:else>
    <mvt:assign name="leftnavResult" value="file_read( '/path/to/my/leftnav.html', 'script', g.leftnav )" />
    &mvt:global:leftnav;
</mvt:if>

<mvt:if expr="NOT ISNULL g.mivaBenchmark">
    <mvt:assign name="benchmarkCurrentTime" value="s.dyn_tm_usec" />
    <mvt:assign name="benchmarkTimeSinceLast" value="g.benchmarkCurrentTime - g.benchmarkLast" />
    <mvt:assign name="benchmarkLast" value="g.benchmarkCurrentTime" />
    <mvt:assign name="benchmarkResults" value="g.benchmarkResults $ '<br>after leftnav:'" />
    <mvt:assign name="benchmarkResults" value="g.benchmarkResults $ '<br>--benchmarkTimeSinceLast = ' $ g.benchmarkTimeSinceLast" />
    <mvt:assign name="benchmarkResults" value="g.benchmarkResults $ '<br>--benchmarkCurrentTime = ' $ g.benchmarkCurrentTime" />
</mvt:if>

bottom of global footer (prints results)

<mvt:if expr="NOT ISNULL g.mivaBenchmark">
    <mvt:assign name="benchmarkCurrentTime" value="s.dyn_tm_usec" />
    <mvt:assign name="benchmarkTimeSinceLast" value="g.benchmarkCurrentTime - g.benchmarkLast" />
    <mvt:assign name="benchmarkTimeSinceStart" value="g.benchmarkCurrentTime - g.benchmarkPageStart" />
    <mvt:assign name="benchmarkLast" value="g.benchmarkCurrentTime" />
    <mvt:assign name="benchmarkResults" value="g.benchmarkResults $ '<br>page complete:'" />
    <mvt:assign name="benchmarkResults" value="g.benchmarkResults $ '<br>--benchmarkTimeSinceLast = ' $ g.benchmarkTimeSinceLast" />
    <mvt:assign name="benchmarkResults" value="g.benchmarkResults $ '<br>--benchmarkTimeSinceStart = ' $ g.benchmarkTimeSinceStart" />
    <mvt:assign name="benchmarkResults" value="g.benchmarkResults $ '<br>--benchmarkCurrentTime = ' $ g.benchmarkCurrentTime" />
    <div style="border: 1px solid #000; margin: 20px; padding: 20px; position: absolute; width: 80%; background-color: #fff; z-index: 999999999;">
        &mvt:global:benchmarkResults;
    </div>
</mvt:if>

Output when using category tree:

benchmarkPageStart = 772249
before leftnav:
--benchmarkTimeSinceLast = 16798
--benchmarkCurrentTime = 789047
after leftnav:
--benchmarkTimeSinceLast = 834
--benchmarkCurrentTime = 789881
page complete:
--benchmarkTimeSinceLast = 717
--benchmarkTimeSinceStart = 18349
--benchmarkCurrentTime = 790598

Output when using static .html:

benchmarkPageStart = 790250
before leftnav:
--benchmarkTimeSinceLast = 16828
--benchmarkCurrentTime = 807078
after leftnav:
--benchmarkTimeSinceLast = 131
--benchmarkCurrentTime = 807209
page complete:
--benchmarkTimeSinceLast = 725
--benchmarkTimeSinceStart = 17684
--benchmarkCurrentTime = 807934

[Brennan]

A static file will always be significantly faster than the default category tree because you're bypassing the loading of all the category data.

When you have the category tree item assigned to the page, it loads every category in your store, including name, code, cat tree image and category title image. If you have 1000's of categories you can see a significant load time for this.

When you have are using file_read to load a static file, it is bypasses the database load completely (assuming you unassign the category_tree item from the page).

[Mike521w]

Thanks Brennan,

Actually I did this with the category tree item still assigned to the page, but the <mvt:item name="category_tree" /> itself was "skipped" due to the if block. Not sure if that makes a difference?

I've seen this sort of thing in other situations too, not just the category tree. It's really not a major difference in terms of load time, but it's consistent so it got me wondering.

[RayYates]

Dynamic html does not exist until it is requested, loaded, formatted and output. Static html always wins because it is just the pre-rendered output.

You could use a conditional set to your IP address to render the category tree and save it to a file. There's a trick to this but it's not too difficult.

You would see the rendered category tree, everyone else would load the saved static file created when you viewed the site.

[alphabet]

Server benchmarks are useful but lately I've been additionally using console.time("benchmark-name") and console.timeEnd("benchmark-name") for RUM (Real User Monitoring/Metrics).

It writes the timestamps and automatically calculates the time to write to the DOM, with any inherent network latency, to the dev console.

This helps understand user experience.

For example, server benchmarks will calculate time to load a variable similarly but console.time will calculate them differently.

// Load very small text

console.time("some-text");

<p>&mvt:product:name;</p>

console.timeEnd("some-text");

// Load very large image

console.time("very-large-image");

<img src="&mvt:product:image;" alt="very-high-res-image" width="2000" height="2000" />

console.timeEnd("very-large-image");

[Mike521w]

Dynamic html does not exist until it is requested, loaded, formatted and output. Static html always wins because it is just the pre-rendered output.

What about when there's nothing dynamic in the content being included by mvt:item?

I'm getting the feeling that there's also just more overhead of some sort when using mvt:item. I set up a blank test page with a miscellaneous content mvt:item. I filled the content item with 25 paragraphs of lorem ipsum text. I also uploaded the same text in a .html include file. Last, I embedded the same text in the main page. Then I used an if block to decide between the three options, and printed the benchmark results.

Embedded in page = usually around 40 ms
Include File = usually around 143 ms
Content Item = usually around 375 ms

[Gordon - Phosphor Media]

What about when there's nothing dynamic in the content being included by mvt:item?

Static content is still faster. It has the least possible overhead. Anything that involves database access/CPU is always additive to the minimum overhead.

[Bruce - PhosphorMedia]

Dynamic html does not exist until it is requested, loaded, formatted and output. Static html always wins because it is just the pre-rendered output.

You could use a conditional set to your IP address to render the category tree and save it to a file. There's a trick to this but it's not too difficult.

You would see the rendered category tree, everyone else would load the saved static file created when you viewed the site.

We use this technique all the time, especially with new jQuery/CSS menus that render the complete category tree content with each pass and writing it out to the page. While beneficial to Search Engines and UX in general, with a large set of categories, it can quickly become a speed bump. On one site, with 5K categories, it's 17 to 20 seconds, while a file_read is, well basically not measurable. (In this case, we actually run a cookie so the category rewrite only happens the first time a listed IP is seen. )

[Rick Wilson]

"What about when there's nothing dynamic in the content being included by mvt:item?"

I think you're missing the point, that anytime you call a .mvc page to load, it's dynamic (even if the content doesn't "feel" dynamic, such as blank page). The (even blank) page doesn't exist before you call it, so it has to create it on the fly, which is the overhead.

If you were to simply save a blank html page, then it's already created (as a counterexample) and thus not dynamic in nature.

[RayYates]

What about when there's nothing dynamic in the content being included by mvt:item?

I'm getting the feeling that there's also just more overhead of some sort when using mvt:item. I set up a blank test page with a miscellaneous content mvt:item. I filled the content item with 25 paragraphs of lorem ipsum text. I also uploaded the same text in a .html include file. Last, I embedded the same text in the main page. Then I used an if block to decide between the three options, and printed the benchmark results.

Embedded in page = usually around 40 ms
Include File = usually around 143 ms
Content Item = usually around 375 ms

Embedded in page: This means the static content is saved as string data directly in the compiled page template. (e.g. nothing to load, nothing to render)

Include File: Not sure how you are doing this but the following would be about as fast as it gets. My tested loading a 1mb. external file took 2.5ms. (Note: does not including downloading the output)

<mvt:if expr="file_read('/robots.txt', 'script', g.include_file)">&mvt:global:include_file;</mvt:if>

Content Item: This would include Merchant component module overhead + your script overhead + file access or database access overhead + output. This would be the same for headers / footers / content item / or your own component module.

Bruce makes an excellent suggestion. He sets a cookie after running a script the first time to cache the categories. On the next page load the cached categories are loaded.

Rick's point about calling any .mvc file is well taken. There is a baseline overhead value to displaying anything in Miva. In my test above, I test the actual command in the page template and not the entire page display. This provides a more accurate assessment of various methods not skewed by web site, internet, and local network traffic.

<mvt:item name="ry_toolbelt" param="benchmark|read|START" />
<mvt:if expr="file_read('/mm5/graphics/00000001/74-402D-02.jpg', 'script', g.include_file)">OK</mvt:if>
<mvt:item name="ry_toolbelt" param="benchmark|read|END" />
<mvt:item name="ry_toolbelt" param="benchmark|read|SHOW" />

That said... your results do represent the relative strengths of various methods for handling static content. It's always a tradeoff between speed and simplicity for the end user.

[Mike521w]

"What about when there's nothing dynamic in the content being included by mvt:item?"

I think you're missing the point, that anytime you call a .mvc page to load, it's dynamic (even if the content doesn't "feel" dynamic, such as blank page). The (even blank) page doesn't exist before you call it, so it has to create it on the fly, which is the overhead.

If you were to simply save a blank html page, then it's already created (as a counterexample) and thus not dynamic in nature.

I didn't think of an mvt:item as a separate .mvc page, I wasn't really sure how it worked internally.

Anyway none of this is really a big issue, I mainly wanted to make sure my testing method was accurate enough to make decisions on. In reality, my category tree is not large, loads quickly enough anyway, and (for other reasons) I now need to use mvt code in it, so I'm loading it using mvt:item