Rich Text Editor cr/lf breaks JSON

[ids]

I don't know if this is a bug yet. Took a couple hours to track down where I finding a problem.

I have an integration where the helper module reads product list data on the fly (as the screen loads). The data is filtered and literally placed back into the data object (in a script tag) at the end of the page. The issue is the product "description" field.

The HTML tags were not a problem -- that was the first thing I checked/eliminated. But Cutting to the chase it turns out when I change to source code mode and use a carriage return (for making the code readable) and clicked update, the data string assigned to its json name-value breaks the plugin being used.

It appears that character(s) inserted aren't compatible. So the question is: when I <mveval to create the json data is there a way to process the string so these hidden incompatible characters can be stripped out or ignored? I'm not using MVT: EVAL to render the string, it's in the helper module using <mveval. I mention it because I'm not sure if a mvtj type of output encoding is available that might solve this issue.

Thanks,

Scott

[216justin]

If you need to replace carriage returns and newlines from a string, you can reference those special characters through Miva's built in asciichar() function and use glosub() to replace them. The ascii value of the carriage return character is 13, and for the new line character, 10.

So if I had a variable named l.dirty_json and I wanted to purge it of the carriage return + newline combinations found within, I'd do something like:

<MvASSIGN NAME="l.clean_json" VALUE="{glosub(l.dirty_json, (asciichar(13) $ asciichar(10)), '')}">

[rguisewite]

If you're encoding your output correctly, newlines should be interpreted correctly. It seems like you're not using json/javascript encoding when outputting the values in the JSON block.

Using this as an example:

<mvt:assign name="l.settings:test" value="'about us on
multiple lines
of text'" />

<script>
var test = '&mvtj:test;';
console.log( test );
</script>

you can see (if you put that in a template and run it), that there will be no JavaScript error, even though it is on multiple lines, because when we output the multiline text, instead of outputting the line breaks, it outputs the \r\n notation (encoded).

On the other hand, if you did this:

<mvt:assign name="l.settings:test" value="'about us on
multiple lines
of text'" />

<script>
var test = '&mvt:test;';
console.log( test );
</script>

You would get a js error, because the encoding is incorrect for the interpreter you're handing the data off to. Since you're coming straight from your module and using <MvEVAL EXPR = "{ ... }"> instead of <mvt:eval expr="...">, you can use the following:

<MvEVAL EXPR = "{ encodejavascriptstring( l.your_variable_or_string ) }">

to output the value in the value pair. Depending on preference and how you're outputting the data, you could also use our builtin JSON_Output( l.variable ) function:

<MvEVAL EXPR = "{ [ g.Module_JSON ].JSON_Output( l.whole_member_structure_to_output_or_individual_value ) }">

Just remember that you should always be encoding your data output into the format the intended interpreter will be able to handle. If outputting to javascript/JSON, use encodejavascriptstring/mvtj (if a string) - be sure to wrap in quotes otherwise it is useless, int(), ROUND, [ g.Module_JSON ].JSON_Boolean( value ), etc...

If you're outputting for HTML, use encodeentities/mvte
If you're outputting for URL name-value-pairs, use encodeattribute/mvta
etc etc. Not doing so will lead to broken code and security risks (XSS, etc). Anyway, the above should work for your scenario. New lines via carriage return and new line characters (\r\n | asciicar( 13 ) $ asciichar( 10 )) should not be a problem at all when properly encoded for JavaScript, nor should it be for HTML when parsed (unless placed into a <pre> tag, or a tag that renders the content with pre styling. So you shouldn't need to strip anything out of the descriptions.

[ids]

Thanks Ryan.

Not sure I knew about this function. I'll give it a whirl.

encodejavascriptstring( l.your_variable_or_string )

Scott

[rguisewite]

No problem :) Just remember that encodejavascriptstring is only useful inside a quoted string when output to javascript. If you use the [ g.Module_JSON ].JSON_Output function (you should be able to see what the function is doing in the LSK), it will determine the output type and encode "correctly" based on the variable type (int, array, structure, string, etc). I put correctly in quotes, because sometimes what the engines stores and what you expect are two different things (numbers represented as strings, as an example). Anyway, best of luck, let me know if something isn't working quite as you'd expect, or you still have troubles.

[ids]

Hey Ryan. The issue is solved with the encodejavascript(string) function.

There is some additional info that might help someone else. With it being JSON data I should have actually looked at the browser console. The error found explained that there were "unescaped characters." So, essentially, I am adding is some vocabulary to this question and solution by clarifying I was dealing with "unescaped" characters where these characters are inserted by the Rich Text Editor in "source code" mode.

I am thinking also that 216Justin's response would have probably worked too in this instance.

Scott

[rguisewite]

I am thinking also that 216Justin's response would have probably worked too in this instance.

Until a single quote or some other character that would be interpreted by the js/json was used ;) Proper encoding is always the best route to take, even if another option seems to work in a specific use case.

[ids]

In my "controlled" scenario, other characters would not have existed. But initially, not knowing the RTE was, assumingly, using these characters for the formatting in the RTE, encoding the data wasn't an obvious conclusion. Creating the same content in the RTE without using source mode didn't break the JSON block. I guess I isolated the issue to what happens in "source code mode" with the data. The inserted characters are benign to most everything else until that data needs to be, in a sense, literal. In which case, encoding was the answer to the question. Learned about two things with this: encodejavascript function and I don't know when that became available( and might be needed less for many devs) and the potential effects of the RTE.

Do you know of anywhere in the Miva documentation that explains the side effects of the data when using the RTE? I don't think I've seen anything like this referenced.

Scott