structured return values.

structured return values.



Hi aGorilla!

Interesting subject... There are a few things that come to mind:

1.) Regarding speed of variables by reference: A while ago, I ran some speed
comparisons, and the result is actually quite impressing, somewhere in the
range of 15-30% for x00000 loops in a compiled scripts (vars by reference
being faster). Of course this greatly depends on the size of the variables
and the computer/memory etc.

This said, I still prefer the way that you described, meaning passing
variables by value, simply for clarity sake. After all, even a 30%
difference is not very much - after all we are talking about a few usecs per
iteration at best.

But there are two more important aspects for not using variables by
reference too often, which has to do with what you wrote about error
handling, as well as code reuse.


2.) After trying lots of different approaches to find the silver bullet for
an integrated / semi-automatic error catching & handling, I found that the
following design pattern works quite well for me. To simplify implementing
this in a consistent manner I use a precompiler that builds all the wrapping
and writes the functions.

On a high level, my scripts are built in form of subsystems, each performing
some specific logical/programmatical tasks. This is similar to Merchant
modules, but less restricted. Subsystems are always nested. Each subsystem
has one interface, with a number of supporting functions / implementations.
The source code is stored in a database, together with error codes and pre
and postconditions, documentation as well as some profiling or debugging
instructions. A subsystem is automatically entered when an interface is
called.

Subsystem
InterfaceA(this,that)
Precondition //checks that the input is correct
ifError --->Errorhandling (correct error or exit)

w_interfaceA(this,that) // main functionbody or dispatcher
calls supporting functions or subsystems....
ifError --->Errorhandling()

Postcondition // ensures that the result is as expected
ifError --->Errorhandling

Both pre- and postconditions are optional. They are added at precompilation
time to the source code and compiled, for performance reasons. The
post-conditions might appear redundant, but they help when you look at the
documentation to understand what is really expected from this particular
program task.

Each interface is nothing but a wrapper to the original entry point to
perform task X. Ideally, this main function is a dispatcher that controls
the flow within the subsystem (which happens in the supporting functions).
There are a handful of validation functions in a library that are used to
validate the pre- & postconditions, like for positive integer, arrays,
structures, within a specific range, etc. I use a simple web-based IDE where
I can add such a pre-condition in form of "validation.check_email(l.email)",
the precompiler then generates the necessary MvDO to the validation routine.



Interfaces can be called dynamically at runtime through simple tags (for
example in templates or database values), supporting functions can't. This
ensures that a user cannot bypass the errorhandling. It is however possible
to bypass the wrappers if hardcoded in the source.



In case of an error, a g.__runtime_errors:structure gets a few values that
are defined in a database, so that the program knows what happened and what
to do. Since this is a global variable, the error can propagate through the
script until a function/interface is found that knows what to do with it to
reset it. The neat thing is that after returning from a subsystem, the lower
level interface can simply check for the presence of g.__runtime_errors to
see if something went wrong.

Since each wrapper stores the original input variables (locals), it can
continue after an error without having these values been changed by a
routine that went wrong. Obviously, changed database values and global
variables must be handled separately.


Since the subsystems are nested, the program/errorhandler can also calculate
the nesting level (it maintains a simple log of all interface calls), and in
case of an error that it can't fix automatically step back to an earlier
(lower) level, and try to pick up the program flow from there, discarting
what it unsuccessfully tried before. If instructed accordingly, a large
program task that for instance calls a dozen other subsystems can be
interrupted and jump back to the original lower level call, without passing
through a dozen indidivual errorhandlers or the previous functions on the
same program level.



The main purpose of this approach is to isolate program parts as much as
possible from each other, and to have a maximum of control through the
interfaces, and not inside the program body. In a packaged version, the
interfaces/wrappers are actually open-source, while the supporting functions
and implementations can remain closed. So clients have full control about
the program flow, errorhandling and thanks to the pre and post conditions
can see what is required/expected, but I can still keep some of my secrets
or protect program elements from changes. The wrapper also allows to switch
on/off very fine-grained profiling and debugging settings on a subsystem
level.



Isolating those subsystems also ensures that it is much easier to reuse them
in different programs, since they are pretty much independent and get their
data through the interfaces and not through global or referenced variables.
Referenced vars should really only be used within the same subsystem in
calls between the supporting functions.


One thing that's really difficult for me is that it is always very tempting
to write errorhandlers into the main program body (where the error initially
occurs ), instead of relying on the interfaces to handle them in a more
organised/structured/global way. I still tend to make functions too
intelligent, which always messes up the code and makes them hard to read and
debug. The tricky thing is find a healthy balance between making a program
too fine-grained or to let a function do too much. Oh well...


Markus














-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Bill Guindon
Sent: Montag, 16. Mai 2005 03:17
To: Miva Users
Subject: [meu] structured return values.

had a thought earlier today related to structures as return values...

When I run into situations where I want a function to modify multiple
values, I tend to add those values as parameters, and pass them by VAR then
let the function modify them.

This has a few side effects (some good, some bad)...

not intuitive, you can't tell that the assign is happening simply by looking
at the function call.

if the function doesn't return a value, you find yourself using MvEVAL's,
which just don't seem right.

the function can still return a value, which I tend to use as the 'l.ok'
success/fail value.

Now I've been using these approaches for some time. Part of the reason for
that, was a speed issue. It seemed that when you had very large structures,
the code ran faster (uncompiled) if you passed them by var instead of
passing them back as a return value. I'm not sure whether this is still an
issue or not, and should probably run some tests to find out.

Assuming speed is no longer an issue (compiled at least), I'm thinking of
changing it to structured return values. Dunno why it never crossed my
mind, and not sure how well it would work, but I'm thinking if I use some
standard naming convention, it could work.

Can still return an 'ok' value, just as a branch:
l.ret:ok

If it's ok, stuff any data you have on a data branch:
l.ret:data

If you have errors, you can stick them on independent branches -- which
allows you to track multiple errors, and/or respond to different errors
different ways.
l.ret:error:MvOpen
l.ret:error:EOF
l.ret:error:Fexists
l.ret:error:DataEntry

It may also pay to make it a structured array, due to miva's lack of
'reflection' (ie: not easy to ask a structure "what branches do you have?" -
at least not in 4.x and under).

The error branch is the beauty of it, as it's let's you seperate error
handling from the core functions. All they do is report the error, it's up
to the calling function to decide what to do - which could include ignoring
them. Of course, that does mean the burden shifts, but that can be handled
with some generic error handling (which can vary by app, or by function
call).

In Merchant's db.mv code, there are a lot of functions that can't be used
'flexibly' because they cause the script to error out.
Sometimes... "Category_Find_Code" is just a 'check', but if I need to check
to see if a category code exists, I can't use that function due to it's hard
coded error check.

anyway, just thinking out loud, thought I'd share it.

--
Bill Guindon (aka aGorilla)

structured return values.



On 5/16/05, Ivo Truxa <[email protected]> wrote:
> Passing references or pointers in function arguments is a common techniqu=
e
> in many languages. It is of course much more efficient, especially at mor=
e
> complex arguments like structures or arrays, because it avoids creating t=
he
> temporary local variables and copying the original content to the stack (=
or
> standard operating memory in some cases) and then back again. References =
not
> only save all those extra CPU instructions needed for the creation of the
> temporary variables, but it also avoids blocking additional memory.

That was my take on it, much faster to pass a memory address than to
recreate it on the fly. Seems that I got mixed results when testing
it, but that was a couple years ago. I'll try to run some new
benchmarks.
=20
> The disadvantage of breaking the intuitivity and readability is real, but
> can be partially reduced by consistently following "Hungarian" naming
> conventions of the functions: using a specific prefixes at those function=
s,
> so that the difference between a function with plain arguments and a
> function using (and possibly modifying) references is apparent on the fir=
st
> look.

Good point. Can take it a step further, and apply it to both, the
function name, and the parameter names. If the function name has a
prefix, it returns a structure (allowing for the error msg ideas I
mentioned), if the paramater has a prefix, it's a sign that it may be
modified in the function. Of course, you have to remember to use the
naming on the parameters as that can't be enforced.

> It certainly does not mean that references should be used everywhere inst=
ead
> of plain arguments. Understanding what happens on lower levels (OS / BIOS=
/
> Assembler / CPU ...) certainly helps very much when programming in high
> level languages like Miva Script too, but generally you can consider that
> passing arrays and structures as references is typically much more
> efficient.

One thing I'd like to compare is simple assign's (both left and right)
- I'd imagine that 'simple' variables may have a slight speed
advantage over structured variables. When I get the chance, I'll try
some more benchmarking.

> Ivo
> http://mivo.truxoft.com
>=20
>=20
> -----Original Message-----
> From: Claudiu
>=20
> Hi Bill,
>=20
> Thanks for sharing your ideas and your research on the list. I hope other
> ppl on that list ("community") will follow you example. There are just a =
few
> (but very valuable) who do that.. but in my opinion that's not enough..
>=20
> Thanks again
> Claudiu Bischoff
>=20
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]]On Behalf Of Bill Guindon
> Sent: lundi 16 mai 2005 03:17
> To: Miva Users
> Subject: [meu] structured return values.
>=20
> had a thought earlier today related to structures as return values...
>=20
> When I run into situations where I want a function to modify multiple
> values, I tend to add those values as parameters, and pass them by VAR
> then let the function modify them.
>=20
> This has a few side effects (some good, some bad)...
>=20
> not intuitive, you can't tell that the assign is happening simply by
> looking at the function call.
>=20
> if the function doesn't return a value, you find yourself using
> MvEVAL's, which just don't seem right.
>=20
> the function can still return a value, which I tend to use as the
> 'l.ok' success/fail value.
>=20
> Now I've been using these approaches for some time. Part of the
> reason for that, was a speed issue. It seemed that when you had very
> large structures, the code ran faster (uncompiled) if you passed them
> by var instead of passing them back as a return value. I'm not sure
> whether this is still an issue or not, and should probably run some
> tests to find out.
>=20
> Assuming speed is no longer an issue (compiled at least), I'm thinking
> of changing it to structured return values. Dunno why it never
> crossed my mind, and not sure how well it would work, but I'm thinking
> if I use some standard naming convention, it could work.
>=20
> Can still return an 'ok' value, just as a branch:
> l.ret:ok
>=20
> If it's ok, stuff any data you have on a data branch:
> l.ret:data
>=20
> If you have errors, you can stick them on independent branches --
> which allows you to track multiple errors, and/or respond to different
> errors different ways.
> l.ret:error:MvOpen
> l.ret:error:EOF
> l.ret:error:Fexists
> l.ret:error:DataEntry
>=20
> It may also pay to make it a structured array, due to miva's lack of
> 'reflection' (ie: not easy to ask a structure "what branches do you
> have?" - at least not in 4.x and under).
>=20
> The error branch is the beauty of it, as it's let's you seperate error
> handling from the core functions. All they do is report the error,
> it's up to the calling function to decide what to do - which could
> include ignoring them. Of course, that does mean the burden shifts,
> but that can be handled with some generic error handling (which can
> vary by app, or by function call).
>=20
> In Merchant's db.mv code, there are a lot of functions that can't be
> used 'flexibly' because they cause the script to error out.
> Sometimes... "Category_Find_Code" is just a 'check', but if I need to
> check to see if a category code exists, I can't use that function due
> to it's hard coded error check.
>=20
> anyway, just thinking out loud, thought I'd share it.
>=20
> --
> Bill Guindon (aka aGorilla)
>=20

structured return values.



On 5/16/05, Claudiu <[email protected]> wrote:
> Hi Bill,
>=20
> Thanks for sharing your ideas and your research on the list. I hope other
> ppl on that list ("community") will follow you example. There are just a =
few
> (but very valuable) who do that.. but in my opinion that's not enough..

Funny thing is, it started out as a private email to a few people,
then I decided to just throw it at the list for a wider audience.
=20
> Thanks again
> Claudiu Bischoff
>=20
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]]On Behalf Of Bill Guindon
> Sent: lundi 16 mai 2005 03:17
> To: Miva Users
> Subject: [meu] structured return values.
>=20
> had a thought earlier today related to structures as return values...
>=20

--=20
Bill Guindon (aka aGorilla)

structured return values.



Passing references or pointers in function arguments is a common technique
in many languages. It is of course much more efficient, especially at more
complex arguments like structures or arrays, because it avoids creating the
temporary local variables and copying the original content to the stack (or
standard operating memory in some cases) and then back again. References not
only save all those extra CPU instructions needed for the creation of the
temporary variables, but it also avoids blocking additional memory.

The disadvantage of breaking the intuitivity and readability is real, but
can be partially reduced by consistently following "Hungarian" naming
conventions of the functions: using a specific prefixes at those functions,
so that the difference between a function with plain arguments and a
function using (and possibly modifying) references is apparent on the first
look.

It certainly does not mean that references should be used everywhere instead
of plain arguments. Understanding what happens on lower levels (OS / BIOS /
Assembler / CPU ...) certainly helps very much when programming in high
level languages like Miva Script too, but generally you can consider that
passing arrays and structures as references is typically much more
efficient.

Ivo
http://mivo.truxoft.com




-----Original Message-----
From: Claudiu

Hi Bill,

Thanks for sharing your ideas and your research on the list. I hope other
ppl on that list ("community") will follow you example. There are just a few
(but very valuable) who do that.. but in my opinion that's not enough..

Thanks again
Claudiu Bischoff

-----Original Message-----
From: [email protected]
[mailto:[email protected]]On Behalf Of Bill Guindon
Sent: lundi 16 mai 2005 03:17
To: Miva Users
Subject: [meu] structured return values.


had a thought earlier today related to structures as return values...

When I run into situations where I want a function to modify multiple
values, I tend to add those values as parameters, and pass them by VAR
then let the function modify them.

This has a few side effects (some good, some bad)...

not intuitive, you can't tell that the assign is happening simply by
looking at the function call.

if the function doesn't return a value, you find yourself using
MvEVAL's, which just don't seem right.

the function can still return a value, which I tend to use as the
'l.ok' success/fail value.

Now I've been using these approaches for some time. Part of the
reason for that, was a speed issue. It seemed that when you had very
large structures, the code ran faster (uncompiled) if you passed them
by var instead of passing them back as a return value. I'm not sure
whether this is still an issue or not, and should probably run some
tests to find out.

Assuming speed is no longer an issue (compiled at least), I'm thinking
of changing it to structured return values. Dunno why it never
crossed my mind, and not sure how well it would work, but I'm thinking
if I use some standard naming convention, it could work.

Can still return an 'ok' value, just as a branch:
l.ret:ok

If it's ok, stuff any data you have on a data branch:
l.ret:data

If you have errors, you can stick them on independent branches --
which allows you to track multiple errors, and/or respond to different
errors different ways.
l.ret:error:MvOpen
l.ret:error:EOF
l.ret:error:Fexists
l.ret:error:DataEntry

It may also pay to make it a structured array, due to miva's lack of
'reflection' (ie: not easy to ask a structure "what branches do you
have?" - at least not in 4.x and under).

The error branch is the beauty of it, as it's let's you seperate error
handling from the core functions. All they do is report the error,
it's up to the calling function to decide what to do - which could
include ignoring them. Of course, that does mean the burden shifts,
but that can be handled with some generic error handling (which can
vary by app, or by function call).

In Merchant's db.mv code, there are a lot of functions that can't be
used 'flexibly' because they cause the script to error out.
Sometimes... "Category_Find_Code" is just a 'check', but if I need to
check to see if a category code exists, I can't use that function due
to it's hard coded error check.

anyway, just thinking out loud, thought I'd share it.

--
Bill Guindon (aka aGorilla)

structured return values.



Hi Bill,

Thanks for sharing your ideas and your research on the list. I hope other
ppl on that list ("community") will follow you example. There are just a few
(but very valuable) who do that.. but in my opinion that's not enough..

Thanks again
Claudiu Bischoff

-----Original Message-----
From: [email protected]
[mailto:[email protected]]On Behalf Of Bill Guindon
Sent: lundi 16 mai 2005 03:17
To: Miva Users
Subject: [meu] structured return values.


had a thought earlier today related to structures as return values...

When I run into situations where I want a function to modify multiple
values, I tend to add those values as parameters, and pass them by VAR
then let the function modify them.

This has a few side effects (some good, some bad)...

not intuitive, you can't tell that the assign is happening simply by
looking at the function call.

if the function doesn't return a value, you find yourself using
MvEVAL's, which just don't seem right.

the function can still return a value, which I tend to use as the
'l.ok' success/fail value.

Now I've been using these approaches for some time. Part of the
reason for that, was a speed issue. It seemed that when you had very
large structures, the code ran faster (uncompiled) if you passed them
by var instead of passing them back as a return value. I'm not sure
whether this is still an issue or not, and should probably run some
tests to find out.

Assuming speed is no longer an issue (compiled at least), I'm thinking
of changing it to structured return values. Dunno why it never
crossed my mind, and not sure how well it would work, but I'm thinking
if I use some standard naming convention, it could work.

Can still return an 'ok' value, just as a branch:
l.ret:ok

If it's ok, stuff any data you have on a data branch:
l.ret:data

If you have errors, you can stick them on independent branches --
which allows you to track multiple errors, and/or respond to different
errors different ways.
l.ret:error:MvOpen
l.ret:error:EOF
l.ret:error:Fexists
l.ret:error:DataEntry

It may also pay to make it a structured array, due to miva's lack of
'reflection' (ie: not easy to ask a structure "what branches do you
have?" - at least not in 4.x and under).

The error branch is the beauty of it, as it's let's you seperate error
handling from the core functions. All they do is report the error,
it's up to the calling function to decide what to do - which could
include ignoring them. Of course, that does mean the burden shifts,
but that can be handled with some generic error handling (which can
vary by app, or by function call).

In Merchant's db.mv code, there are a lot of functions that can't be
used 'flexibly' because they cause the script to error out.
Sometimes... "Category_Find_Code" is just a 'check', but if I need to
check to see if a category code exists, I can't use that function due
to it's hard coded error check.

anyway, just thinking out loud, thought I'd share it.

--
Bill Guindon (aka aGorilla)

had a thought earlier today related to structures as return values...

When I run into situations where I want a function to modify multiple
values, I tend to add those values as parameters, and pass them by VAR
then let the function modify them.

This has a few side effects (some good, some bad)...

not intuitive, you can't tell that the assign is happening simply by
looking at the function call.

if the function doesn't return a value, you find yourself using
MvEVAL's, which just don't seem right.

the function can still return a value, which I tend to use as the
'l.ok' success/fail value.

Now I've been using these approaches for some time. Part of the
reason for that, was a speed issue. It seemed that when you had very
large structures, the code ran faster (uncompiled) if you passed them
by var instead of passing them back as a return value. I'm not sure
whether this is still an issue or not, and should probably run some
tests to find out.

Assuming speed is no longer an issue (compiled at least), I'm thinking
of changing it to structured return values. Dunno why it never
crossed my mind, and not sure how well it would work, but I'm thinking
if I use some standard naming convention, it could work.

Can still return an 'ok' value, just as a branch:
l.ret:ok

If it's ok, stuff any data you have on a data branch:
l.ret:data

If you have errors, you can stick them on independent branches --
which allows you to track multiple errors, and/or respond to different
errors different ways.
l.ret:error:MvOpen
l.ret:error:EOF
l.ret:error:Fexists
l.ret:error:DataEntry

It may also pay to make it a structured array, due to miva's lack of
'reflection' (ie: not easy to ask a structure "what branches do you
have?" - at least not in 4.x and under).

The error branch is the beauty of it, as it's let's you seperate error
handling from the core functions. All they do is report the error,
it's up to the calling function to decide what to do - which could
include ignoring them. Of course, that does mean the burden shifts,
but that can be handled with some generic error handling (which can
vary by app, or by function call).

In Merchant's db.mv code, there are a lot of functions that can't be
used 'flexibly' because they cause the script to error out.=20
Sometimes... "Category_Find_Code" is just a 'check', but if I need to
check to see if a category code exists, I can't use that function due
to it's hard coded error check.

anyway, just thinking out loud, thought I'd share it.

--=20
Bill Guindon (aka aGorilla)