Miva Merchant Empresa Bugs?

[John Stange]

Re: MIVA Empresa Bugs?

Found a trivially reproducible segfault in 5.13 in the config file parser library (distributed as 3x.so); tested on CentOS 4 with the i386 binary. Omitting a field in one of the config file entries will get you a crash; expected behavior would be a parse error and orderly exit.

Example config line to reproduce:

<DATABASE-LIB LIBRARY ="/home3/johnsons/upgrade/public_html/cgi-bin/lib/databases/mivasql.so">

(note that the above omits the METHOD field)

Resulting stack trace (using Valgrind) below, which suggests a simple NULL pointer dereference problem. Probably a quick fix.

==1234== Memcheck, a memory error detector.
==1234== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==1234== Using LibVEX rev 1575, a library for dynamic binary translation.
==1234== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==1234== Using valgrind-3.1.1, a dynamic binary instrumentation framework.
==1234== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==1234== For more details, rerun with: -v
==1234==
==1234== Invalid read of size 1
==1234== at 0x4005E7C: strlen (mac_replace_strmem.c:243)
==1234== by 0x80983CB: CommerceLibrary::CommerceLibrary(char const *, char const *) (in /someuser/webroot/cgi-bin/mivavm)
==1234== by 0x8098803: CommerceLibraryManager::RegisterDSO(char const *, char const *) (in /someuser/webroot/cgi-bin/mivavm)
==1234== by 0x80A6AA1: ExternalConfig::Register_Library_Commerce(char const *, char const *) (in /someuser/webroot/cgi-bin/mivavm)
==1234== by 0x8114E82: mvConfig_Register_Library_Commerce (in /someuser/webroot/cgi-bin/mivavm)
==1234== by 0x411B7A0: config3x_tag_end (in /someuser/webroot/cgi-bin/libmivaconfig.so)
==1234== by 0x411CEBE: config_parse (in /someuser/webroot/cgi-bin/libmivaconfig.so)
==1234== by 0x411A852: config3x_loadconfiguration (in /someuser/webroot/cgi-bin/libmivaconfig.so)
==1234== by 0x4119D3A: config3x_api_init (in /someuser/webroot/cgi-bin/libmivaconfig.so)
==1234== by 0x80A65AC: ExternalConfig::Load(char const *, int, void *, int) (in /someuser/webroot/cgi-bin/mivavm)
==1234== by 0x8078EE7: CGIApplication::LoadConfiguration(void) (in /someuser/webroot/cgi-bin/mivavm)
==1234== by 0x807870B: CGIApplication::Go(void) (in /someuser/webroot/cgi-bin/mivavm)
==1234== Address 0x0 is not stack'd, malloc'd or (recently) free'd
==1234==
==1234== Process terminating with default action of signal 11 (SIGSEGV)
==1234== Access not within mapped region at address 0x0
==1234== at 0x4005E7C: strlen (mac_replace_strmem.c:243)
==1234== by 0x80983CB: CommerceLibrary::CommerceLibrary(char const *, char const *) (in /someuser/webroot/cgi-bin/mivavm)
==1234== by 0x8098803: CommerceLibraryManager::RegisterDSO(char const *, char const *) (in /someuser/webroot/cgi-bin/mivavm)
==1234== by 0x80A6AA1: ExternalConfig::Register_Library_Commerce(char const *, char const *) (in /someuser/webroot/cgi-bin/mivavm)
==1234== by 0x8114E82: mvConfig_Register_Library_Commerce (in /someuser/webroot/cgi-bin/mivavm)
==1234== by 0x411B7A0: config3x_tag_end (in /someuser/webroot/cgi-bin/libmivaconfig.so)
==1234== by 0x411CEBE: config_parse (in /someuser/webroot/cgi-bin/libmivaconfig.so)
==1234== by 0x411A852: config3x_loadconfiguration (in /someuser/webroot/cgi-bin/libmivaconfig.so)
==1234== by 0x4119D3A: config3x_api_init (in /someuser/webroot/cgi-bin/libmivaconfig.so)
==1234== by 0x80A65AC: ExternalConfig::Load(char const *, int, void *, int) (in /someuser/webroot/cgi-bin/mivavm)
==1234== by 0x8078EE7: CGIApplication::LoadConfiguration(void) (in /someuser/webroot/cgi-bin/mivavm)
==1234== by 0x807870B: CGIApplication::Go(void) (in /someuser/webroot/cgi-bin/mivavm)
==1234==
==1234== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 33 from 2)
==1234== malloc/free: in use at exit: 284,989 bytes in 164 blocks.
==1234== malloc/free: 194 allocs, 30 frees, 285,916 bytes allocated.
==1234== For counts of detected errors, rerun with: -v
==1234== searching for pointers to 164 not-freed blocks.
==1234== checked 808,896 bytes.
==1234==
==1234== LEAK SUMMARY:
==1234== definitely lost: 80 bytes in 16 blocks.
==1234== possibly lost: 0 bytes in 0 blocks.
==1234== still reachable: 284,909 bytes in 148 blocks.
==1234== suppressed: 0 bytes in 0 blocks.
==1234== Use --leak-check=full to see details of leaked memory.
Segmentation fault

[burch]

Re: Miva Merchant Empresa Bugs?

Thank you for the bug report. We were already aware of the issue and it will be fixed in 5.14. I've added your information to our internal defect report.

[RayYates]

Re: Miva Merchant Empresa Bugs?

Miva Merchant 5.5
Production Release 8 Update 6
Miva Merchant Engine v5.13
Database API: mysql


This seems to be a Template Compiler bug, I did not test the with equivalent Mivascript code, but I think I remember Bruce G having the same problem a while back.

The conditional <mvt:if expr="( g.test[1]:code EQ l.settings:topcat:code )"> causes the null array g.test to stop being null.

Sampe code

<mvt:item name="ry_toolbelt" param="assign|g.max|miva_array_max(g.test)" />
~&mvt:global:max;~<br>

<mvt:if expr="( g.test[1]:code EQ l.settings:topcat:code )">

</mvt:if>

<mvt:item name="ry_toolbelt" param="assign|g.max|miva_array_max(g.test)" />
~&mvt:global:max;~<br>

Result:
~0~
~1~

[burch]

Re: Miva Merchant Empresa Bugs?

The conditional <mvt:if expr="( g.test[1]:code EQ l.settings:topcat:code )"> causes the null array g.test to stop being null.

This is actually intentional behavior for the following reason.

Prior to version 4.06, the VM displayed that exact behavior in all expressions. Reading a member or array element would cause the member/element to be created.

In 4.06, new instructions were added to make those accesses read only. However, if you try to run code that contains those instructions on an older VM, you'll end up with a fatal runtime error. The template compiler is intentionally generating the most compatible code it can.

At this point it's debatable if maintaining backwards compatibility with the 4.05 and older VM is necessary for generated template code, but that's the reasoning behind the current behavior.

I looked through the old compiler/VM release notes and didn't find any explicit mention of this change, FWIW.

[RayYates]

Re: Miva Merchant Empresa Bugs?

Sorry I'm not following.


4.05 and older: VM had this behavior.
4.06 Problem fixed in VM


4.05 compiled template contains instructions that crash on 4.06 VM
So something undefined was added to VM 4.06 to prevent this.

The template compiler is intentionally generating the most compatible code it can.

So If I'm following this correctly a 4.06 compiled template is creating code so it can run on VM 4.05


Really? How would a new compiled template get on a machine with an older VM?

[burch]

Re: Miva Merchant Empresa Bugs?

So If I'm following this correctly a 4.06 compiled template is creating code so it can run on VM 4.05

Really? How would a new compiled template get on a machine with an older VM?

Yes, really. The compiler can do the same thing, with the -C flag. The difference is that the template compiler defaults to the most compatible code and the regular compiler defaults to the least compatible code. If you compiled your non-template test with "-C 4.05" you should see the same behavior as you're getting from the template compiler.

There are plenty of ways a compiled template could get from a newer engine to an older one. Site migrations, engine downgrades, etc... It's unlikely that this would happen these days since there's so many versions between the releases, but the behavior you're encountering was defined when it was far more likely.

[RayYates]

Re: Miva Merchant Empresa Bugs?

Downgrades, didn't think of that. OK then, but if I had a vote it would be to Upgrade it :-)

Example for posterity:
<mvt:if expr="(l.settings:breadcrumbs[1]:code EQ g.product_code) OR (l.settings:breadcrumbs[2]:code EQ g.product_code)">


</mvt:if>

Implications:
If you use a mvt:foreach loop on what you expected to be an empty data, the code inside the loop will execute. i


<mvt:foreach iterator="bread" array="breadcrumbs">
If breadcrumbs is empty the code will always execute outputting unexpected data.
</mvt:foreach>


<mvt:if expr="l.settings:breadcrumbs">
If you test the value first the test will return true and again the code will always execute.
</mvt:if>

[Scott McCollough]

Re: Miva Merchant Empresa Bugs?

This is a script issue too, not just template.

<MvIF EXPR = "{ ISNULL g.test1 }">There is no g.test1 <MvELSE>There is a g.test1 </MvIF><br />
<MvIF EXPR = "{ ISNULL g.test1 }">NOW there is a g.test1 <MvELSE>g.test1 is an illusion </MvIF><br />


<MvIF EXPR = "{ ISNULL g.test2:one }">There is no g.test2:one <MvELSE>There is a g.test2:one </MvIF><br />
<MvIF EXPR = "{ ISNULL g.test2:one }">NOW there is a g.test2:one <MvELSE>g.test2:one is an illusion </MvIF><br />


<MvIF EXPR = "{ ISNULL g.test3[1] }">There is no g.test3[1] <MvELSE>There is a g.test3[1] </MvIF><br />
<MvIF EXPR = "{ ISNULL g.test3[1] }">NOW there is a g.test3[1] <MvELSE>g.test3[1] is an illusion </MvIF><br />

There is no g.test1
NOW there is a g.test1
There is no g.test2:one
NOW there is a g.test2:one
There is no g.test3[1]
NOW there is a g.test3[1]

If you gaze into the variables, the variables gazes also into you...

[burch]

Re: Miva Merchant Empresa Bugs?

Scott, your test script is flawed in two ways:

1. The second MvIF in each case is backwards
2. A variable created in this way WILL be NULL, because it will be empty

Here's an example:

Structure before probe: <MvEVAL EXPR = "{ miva_member_exists( g.test1, 'one' ) }"><br />
<MvIF EXPR = "{ g.test1:one }">This is impossible</MvIF>
Structure after probe: <MvEVAL EXPR = "{ miva_member_exists( g.test1, 'one' ) }"><br />


Array before probe: <MvEVAL EXPR = "{ miva_element_exists( g.test2, 1 ) }"><br />
<MvIF EXPR = "{ g.test2[ 1 ] }">This is impossible</MvIF>
Array after probe: <MvEVAL EXPR = "{ miva_element_exists( g.test2, 1 ) }"><br />

When compiled with a 4.06 or newer compiler and no specific compatibility flag, you will get this output:

Structure before probe: 0
Structure after probe: 0
Array before probe: 0
Array after probe: 0

But when compiled with -C 4.05, you'll get this output:

Structure before probe: 0
Structure after probe: 1
Array before probe: 0
Array after probe: 1

[Scott McCollough]

Re: Miva Merchant Empresa Bugs?

Found a reason to use the long-lost keyword_in() function and think I found a bug. Or if not bug, then idiosyncrasy and should be documented.

The keyword(s) entered need to be in lower case, even if the word in the string is upper case.

<MvASSIGN NAME = "l.string" VALUE = "The quick brown fox jumped over the Lazy dog. Meanwhile, grumpy wizards make toxic brew for the evil Queen and Jack.">

<MvASSIGN NAME = "l.keyword_array" INDEX = "1" VALUE = "{ 'quick' }">
<MvASSIGN NAME = "l.keyword_array" INDEX = "2" VALUE = "{ 'orange' }">
<MvASSIGN NAME = "l.keyword_array" INDEX = "3" VALUE = "{ 'queen' }">
<MvASSIGN NAME = "l.keyword_array" INDEX = "4" VALUE = "{ 'Queen' }">

<MvASSIGN NAME = "l.return" VALUE = "{ keyword_in( l.keyword_array, l.string ) }">

<MvFOREACH ITERATOR = "l.words" ARRAY = "l.keyword_array" INDEX = "l.pos">
<br />Did &quot;<MvEVAL EXPR = "{ l.words }">&quot; show up? <MvEVAL EXPR = "{ l.return[ l.pos ] }">
</MvFOREACH>

Output:
Did "quick" show up? 1
Did "orange" show up? 0
Did "queen" show up? 1
Did "Queen" show up? 0

[branden]

Re: Miva Merchant Empresa Bugs?

Sorry if this is out of place (I'm a lurker) but I've just spent a while tracking down a bug that was causing the compiler (both Windows and Linux) to hard exit without an error message and without generating a compiled mvc file. I did it by placing poorly-worded MvASSIGN statements that would generate compiler warnings throughout the code, and seeing which was the last warning before the compiler exited.

Anyway, I found that nesting the in-line MvDO-equivalent function calls was causing the compiler error. By which I mean a statement like this:

<MvASSIGN NAME="l.variable" VALUE="{[ './functions.mvc' ].write('Item #' $ [ './functions.mvc' ].cleanup(l.this_item) $ ' added')}">

Code like this used to compile, in earlier versions of the v5.0 compiler, but stopped working around version v5.09. Of course, now that I know the issue, it is easy to work around, but finding it took a bit of effort.

Thanks

[burch]

Re: Miva Merchant Empresa Bugs?

I was able to reproduce the issue with your sample code, so I'll file a bug and we'll try to get a fix in for 5.20. I'm not sure what we would have changed recently to introduce that behavior.

Edit: It looks like it was introduced in 5.14, BTW.

[wmgilligan]

Re: Miva Merchant Empresa Bugs? MvDo ?

Not sure if its me or Miva, but the docs say (and I have done this before):

<!--#include file="/external.html"-->

This can be replaced by:
<MvDO FILE="/external.html">

So I add this:

<MvDo file="news.html">

But this error occurs:

MvDO: Unable to open 'news.html': File is either corrupt or not a valid compiled Miva Script file

[Rick Wilson]

Re: Miva Merchant Empresa Bugs?

I would guess you won't get a reply on this from Burch until next Tuesday when he's back in the office.

[virtualkiwi]

Re: Miva Merchant Empresa Bugs? MvDo ?

Not sure if its me or Miva, but the docs say (and I have done this before):

<!--#include file="/external.html"-->

This can be replaced by:
<MvDO FILE="/external.html">

So I add this:

<MvDo file="news.html">

But this error occurs:

MvDO: Unable to open 'news.html': File is either corrupt or not a valid compiled Miva Script file

That worked back in the days of interpreted mivascript, but I don't think it's ever worked with compiled mivascript as the include has to be a compiled mivascript.
You can use file_read() function and and <mveval > to achieve the intended result.