Code name "Wombat" API changes

[burch]

Hello all,

This thread will serve as a place to discuss API changes in the upcoming "Wombat" release of Miva Merchant.

We're making some major changes to the software in this release, and our goal is to minimize the impact on module developers as much as possible. Ideally, existing modules will continue to function without modification, and your only concern will be integrating the new functionality into your modules.

To accomplish this goal, we're being extremely careful about how we change the API footprint of the software--both functions that you call within our code (such as util.mvc and db.mvc) and functions we call within your code (the module API). There may still be issues for people that go directly to our database tables, but we are trying to minimize them where possible.

[burch]

Address Line 2

The first change I want to discuss is the addition of a second address line.

Here is a list of the database changes that have been made:

sNN_StandardFields:
add mode_addr2 CHAR(1)

sNN_Baskets:
add ship_addr2 CHAR( 100 )
add bill_addr2 CHAR( 100 )

sNN_Customers:
add ship_addr2 CHAR( 100 )
add bill_addr2 CHAR( 100 )

sNN_Orders:
add ship_addr2 CHAR( 100 )
add bill_addr2 CHAR( 100 )

All of the existing functions that operate on these tables now return the address in two forms. The existing ship_addr and bill_addr fields are returned with the contents of the two address fields concatenated together. In addition, new fields ship_addr1/ship_addr2 and bill_addr1/bill_addr2 are also returned.

When calling Customer_Insert, Order_Create, Order_Insert, and Basket_Insert, if [ship|bill]_addr is sent in, and BOTH [ship|bill]_addr1 AND [ship|bill]_addr2 are empty, the _addr value will be assumed to include the entire address.

When calling Customer_Update, Order_Update_Customer_Information, and Basket_Update_Customer_Information, if [ship|bill]_addr is present, it overrides any _addr1/_addr2 value. NOTE!!! This means that new callers that want to use these functions to update address line 1 and line 2 must be modified to explicitly set [ship|bill]_addr to an empty value, otherwise the DB layer will assume it is dealing with an older module and put the entire value into _addr, emptying _addr2.

There is an additional sanity check in the Update functions so that if an older module is calling these functions without changing the address, and there was a previous _addr2 value, we don't "break" the address and put everything into _addr.

Here's the code for Basket_Read() and Basket_Update_Customer_Information() to show exactly how the checks work:

<MvFUNCTION NAME = "Basket_Read" PARAMETERS = "basket var" STANDARDOUTPUTLEVEL = "">
    <MvASSIGN NAME = "l.basket:session_id"      VALUE = "{ Baskets.d.session_id }">
    <MvASSIGN NAME = "l.basket:cust_id"         VALUE = "{ Baskets.d.cust_id }">
    <MvASSIGN NAME = "l.basket:basket_id"       VALUE = "{ Baskets.d.basket_id }">
    <MvASSIGN NAME = "l.basket:order_id"        VALUE = "{ Baskets.d.order_id }">
    <MvASSIGN NAME = "l.basket:order_proc"      VALUE = "{ Baskets.d.order_proc }">
    <MvASSIGN NAME = "l.basket:lastupdate"      VALUE = "{ Baskets.d.lastupdate }">
    <MvASSIGN NAME = "l.basket:ship_fname"      VALUE = "{ Baskets.d.ship_fname }">
    <MvASSIGN NAME = "l.basket:ship_lname"      VALUE = "{ Baskets.d.ship_lname }">
    <MvASSIGN NAME = "l.basket:ship_email"      VALUE = "{ Baskets.d.ship_email }">
    <MvASSIGN NAME = "l.basket:ship_comp"       VALUE = "{ Baskets.d.ship_comp }">
    <MvASSIGN NAME = "l.basket:ship_phone"      VALUE = "{ Baskets.d.ship_phone }">
    <MvASSIGN NAME = "l.basket:ship_fax"        VALUE = "{ Baskets.d.ship_fax }">

    <MvIF EXPR = "{ len( Baskets.d.ship_addr2 ) EQ 0 }">
        <MvASSIGN NAME = "l.basket:ship_addr"   VALUE = "{ Baskets.d.ship_addr }">
    <MvELSE>
        <MvASSIGN NAME = "l.basket:ship_addr"   VALUE = "{ Baskets.d.ship_addr $ ' ' $ Baskets.d.ship_addr2 }">
    </MvIF>

    <MvASSIGN NAME = "l.basket:ship_addr1"      VALUE = "{ Baskets.d.ship_addr }">
    <MvASSIGN NAME = "l.basket:ship_addr2"      VALUE = "{ Baskets.d.ship_addr2 }">
    <MvASSIGN NAME = "l.basket:ship_city"       VALUE = "{ Baskets.d.ship_city }">
    <MvASSIGN NAME = "l.basket:ship_state"      VALUE = "{ Baskets.d.ship_state }">
    <MvASSIGN NAME = "l.basket:ship_zip"        VALUE = "{ Baskets.d.ship_zip }">
    <MvASSIGN NAME = "l.basket:ship_cntry"      VALUE = "{ Baskets.d.ship_cntry }">
    <MvASSIGN NAME = "l.basket:bill_fname"      VALUE = "{ Baskets.d.bill_fname }">
    <MvASSIGN NAME = "l.basket:bill_lname"      VALUE = "{ Baskets.d.bill_lname }">
    <MvASSIGN NAME = "l.basket:bill_email"      VALUE = "{ Baskets.d.bill_email }">
    <MvASSIGN NAME = "l.basket:bill_comp"       VALUE = "{ Baskets.d.bill_comp }">
    <MvASSIGN NAME = "l.basket:bill_phone"      VALUE = "{ Baskets.d.bill_phone }">
    <MvASSIGN NAME = "l.basket:bill_fax"        VALUE = "{ Baskets.d.bill_fax }">

    <MvIF EXPR = "{ len( Baskets.d.bill_addr2 ) EQ 0 }">
        <MvASSIGN NAME = "l.basket:bill_addr"   VALUE = "{ Baskets.d.bill_addr }">
    <MvELSE>
        <MvASSIGN NAME = "l.basket:bill_addr"   VALUE = "{ Baskets.d.bill_addr $ ' ' $ Baskets.d.bill_addr2 }">
    </MvIF>

    <MvASSIGN NAME = "l.basket:bill_addr1"      VALUE = "{ Baskets.d.bill_addr }">
    <MvASSIGN NAME = "l.basket:bill_addr2"      VALUE = "{ Baskets.d.bill_addr2 }">
    <MvASSIGN NAME = "l.basket:bill_city"       VALUE = "{ Baskets.d.bill_city }">
    <MvASSIGN NAME = "l.basket:bill_state"      VALUE = "{ Baskets.d.bill_state }">
    <MvASSIGN NAME = "l.basket:bill_zip"        VALUE = "{ Baskets.d.bill_zip }">
    <MvASSIGN NAME = "l.basket:bill_cntry"      VALUE = "{ Baskets.d.bill_cntry }">
    <MvASSIGN NAME = "l.basket:ship_id"         VALUE = "{ Baskets.d.ship_id }">
    <MvASSIGN NAME = "l.basket:ship_data"       VALUE = "{ Baskets.d.ship_data }">
</MvFUNCTION>

<MvFUNCTION NAME = "Basket_Update_Customer_Information" PARAMETERS = "basket var" STANDARDOUTPUTLEVEL = "">
    <MvIF EXPR = "{ len( l.basket:ship_addr ) AND
                    l.basket:ship_addr NE ( l.basket:ship_addr1 $ ' ' $ l.basket:ship_addr2 ) }">
        <MvASSIGN NAME = "l.basket:ship_addr1"  VALUE = "{ l.basket:ship_addr }">
        <MvASSIGN NAME = "l.basket:ship_addr2"  VALUE = "">
    </MvIF>

    <MvIF EXPR = "{ len( l.basket:bill_addr ) AND
                    l.basket:bill_addr NE ( l.basket:bill_addr1 $ ' ' $ l.basket:bill_addr2 ) }">
        <MvASSIGN NAME = "l.basket:bill_addr1"  VALUE = "{ l.basket:bill_addr }">
        <MvASSIGN NAME = "l.basket:bill_addr2"  VALUE = "">
    </MvIF>

    <MvQUERY NAME   = "Merchant"
             QUERY  = "{ 'UPDATE ' $ g.Store_Table_Prefix $ 'Baskets
                          SET
                            ship_fname  = ?,
                            ship_lname  = ?,
                            ship_email  = ?,
                            ship_comp   = ?,
                            ship_phone  = ?,
                            ship_fax    = ?,
                            ship_addr   = ?,
                            ship_addr2  = ?,
                            ship_city   = ?,
                            ship_state  = ?,
                            ship_zip    = ?,
                            ship_cntry  = ?,
                            bill_fname  = ?,
                            bill_lname  = ?,
                            bill_email  = ?,
                            bill_comp   = ?,
                            bill_phone  = ?,
                            bill_fax    = ?,
                            bill_addr   = ?,
                            bill_addr2  = ?,
                            bill_city   = ?,
                            bill_state  = ?,
                            bill_zip    = ?,
                            bill_cntry  = ?
                          WHERE
                            session_id  = ?' }"
             FIELDS = "l.basket:ship_fname, l.basket:ship_lname, l.basket:ship_email, l.basket:ship_comp,
                       l.basket:ship_phone, l.basket:ship_fax, l.basket:ship_addr1, l.basket:ship_addr2, l.basket:ship_city,
                       l.basket:ship_state, l.basket:ship_zip, l.basket:ship_cntry, l.basket:bill_fname,
                       l.basket:bill_lname, l.basket:bill_email, l.basket:bill_comp, l.basket:bill_phone,
                       l.basket:bill_fax, l.basket:bill_addr1, l.basket:bill_addr2, l.basket:bill_city, l.basket:bill_state,
                       l.basket:bill_zip, l.basket:bill_cntry,
                       l.basket:session_id">
    <MvIF EXPR = "{ g.MvQUERY_Error }">
        <MvFUNCTIONRETURN VALUE = "{ [ g.Library_Filename_Utilities ].Error( 'MER-DBP-BSK-00005', g.MvQUERY_Error ) }">
    </MvIF>

    <MvIF EXPR = "{ len( l.basket:ship_addr2 ) EQ 0 }">
        <MvASSIGN NAME = "l.basket:ship_addr"   VALUE = "{ l.basket:ship_addr1 }">
    <MvELSE>
        <MvASSIGN NAME = "l.basket:ship_addr"   VALUE = "{ l.basket:ship_addr1 $ ' ' $ l.basket:ship_addr2 }">
    </MvIF>

    <MvIF EXPR = "{ len( l.basket:bill_addr2 ) EQ 0 }">
        <MvASSIGN NAME = "l.basket:bill_addr"   VALUE = "{ l.basket:bill_addr1 }">
    <MvELSE>
        <MvASSIGN NAME = "l.basket:bill_addr"   VALUE = "{ l.basket:bill_addr1 $ ' ' $ l.basket:bill_addr2 }">
    </MvIF>

    <MvFUNCTIONRETURN VALUE = 1>
</MvFUNCTION>

[burch]

Additional Address Line 2 Compatibility

In addition to the database layer, the Miva Merchant actions and components also handle the existing g.ShipAddress/g.BillAddress variables, or the new style g.ShipAddress1, g.ShipAddress2/g.BillAddress1, g.BillAddress2 variables.

The handling of the old style variables is roughly the same as the way the database variables are handled above. We ensure that we create the old-name variables even if the runtime user interface sent in the new-name variables, so that system extension (or other) modules can still find the data in the old location.

[burch]

Re: Code name "Wombat" API changes

Before I begin documenting the rest of the Wombat API changes, I want to make the Wombat Beta 2 MMLSK available. I'm attaching it to this post.

This MMLSK includes all of the CSSUI source code, as well as the Order database API compatibility layers.

[burch]

Re: Code name "Wombat" API changes

Additional Database changes (besides those discussed above for address line 2):

New table: TrackingLinks (domain level):
----------------------------------------------
type CHAR( 50 )
url CHAR( 254 )

This table controls the format of shipment tracking links when displayed to the user. The token %tracknum% will be replaced with the actual tracking number. The initial data for this table is pulled from setup.xml on installation.

Order processing changes:
------------------------------
The database structure used to store orders has changed dramatically. There are new object types (OrderShipments, OrderReturns, OrderPayments), and multi-level status fields for each object.

OrderShipments and OrderReturns are a new concept in Wombat. OrderPayments store roughly the same data as in previous versions, but there may now be multiple payment records for a single order. Payment modules that implement the new API functions can create transaction records as required to record auth, auth/capture, capture, void, and refund operations. Our strategy in this regard has been to make the payment module mirror the state of transactions that a user would see in the gateway's user interface. For example, in CHASE Paymentech, certain operations change an authorization to an authorization + capture, while others create an entirely new transaction record.

There are many new functions in db.mvc to deal with these new objects. However, we have maintained all of the existing database API functions for orders, and the new compatibility layer (lib/dbeng/order_compat.mv) emulates the old database format where required.

sNN_Orders
-------------
The column "processed" is deleted entirely. It is replaced with status INTEGER, which may contain the following values:

0 Pending
100 Processing
200 Shipped
201 Partially Shipped
300 Cancelled
400 Backordered
500 RMA Issued
600 Returned

The status of an order is generally controlled by the status of items within the order. The db.mvc function Order_Update_Status determines the status of the order by looking at the disposition of the various items.

The following additional fields have been added. Some of these are redundant and may be removed before production:
pay_status NUMBER( 0, 0 )
stk_status NUMBER( 0, 0 )
dt_instock CHAR( 10 )
total_auth NUMBER( 10, 2 )
total_capt NUMBER( 10, 2 )
pend_count NUMBER( 0, 0 )
bord_count NUMBER( 0, 0 )

sNN_OrderItems:
--------------------
The following columns are added:
status NUMBER( 0, 0 )
shpmnt_id NUMBER( 0, 0 )
rma_id NUMBER( 0, 0 )
dt_instock CHAR( 10 )

Status is generally the same as for orders, but with the following values:
100 Picking
200 Shipped
300 Cancelled
400 Backordered (in which case dt_instock optionally contains the expected date in stock)
500 RMA Issued (In which case rma_id references a record in the OrderReturns table)
600 Returned

sNN_OrderShipments:
-------------------------
This new table records shipment information for one or more items in an order.
id NUMBER( 0, 0 )
code CHAR( 50 )
order_id NUMBER( 0, 0 )
status NUMBER( 0, 0 )
ship_date CHAR( 10 )
tracknum CHAR( 100 )
tracktype CHAR( 50 )
weight NUMBER( 10, 3 )
cost NUMBER( 10, 2 )
label_data MEMO
label_type CHAR( 100 )

Valid status values for a shipment are:
100 Picking
200 Shipped

sNN_OrderReturns:
----------------------
This new table records return authorizations for one or more items in an order.
id NUMBER( 0, 0 )
order_id NUMBER( 0, 0 )
code CHAR( 100 )
status NUMBER( 0, 0 )
dt_issued CHAR( 10 )
dt_recvd CHAR( 10 )

Valid status values for an OrderReturn are:
0 RMA Issued
600 Received

sNN_OrderPayments:
------------------------
This table contains all of the payment information that was recorded in the old database structure, as well as additional information.
id NUMBER( 0, 0 )
parent_id NUMBER( 0, 0 )
order_id NUMBER( 0, 0 )
type NUMBER( 0, 0 )
refnum CHAR( 254 )
amount NUMBER( 10, 2 )
available NUMBER( 10, 2 )
dtstamp CHAR( 10 )
expires CHAR( 10 )
pay_id NUMBER( 0, 0 )
pay_data MEMO
pay_secid NUMBER( 0, 0 )
pay_seckey MEMO
pay_secdat MEMO

type is one of:
1 Legacy Authorization (used by the compatibility layer for pre-wombat payment modules)
2 Legacy Capture
3 Authorization
4 Capture
5 Authorization + Capture
6 Refund
7 VOID

available indicates the portion of amount that is available for a subsequent operation. For example, for an authorization record, available indicates the amount that may be captured or voided. For a capture record, available indicates the amount that may be refunded. This value may be updated by the payment module as operations occur. For example, when doing a partial capture of an authorization, or a refund.

sNN_Encryption/sNN_PrivateKeys:
--------------------------------------
PA-DSS requires us to be able to store the RSA private key(s) used for order encryption separately from the order data. The column "prv_key" has been moved from sNN_Encryption into a new table, sNN_PrivateKeys, which has the following columns:

id NUMBER( 0, 0 )
prv_key MEMO

id references a record in sNN_Encryption, and prv_key is the same data as previously stored in the sNN_Encryption table.

merchdb.dat may now contain an additional set of connection fields (after the primary database fields) that will indicate where the sNN_PrivateKeys table exists. This may be a MivaSQL instance on the webserver, a separate MySQL database, or, if not present, indicates that the sNN_PrivateKeys table exists in the same database instance as the rest of the data. The default for upgraded installations is to leave the private keys in the same database.

[burch]

Re: Code name "Wombat" API changes

New module API features for order processing:

Feature "not_order":
Module_Notify_Order_StatusChange( module var, original_status, order var )
This function is called whenever the status of an order changes.

Feature "not_orderitem":
Module_Notify_OrderItem_StatusChange( module var, count, original_status var, orderitems var )
This function is called whenever the status of one or more order items changes. Whenever possible, changes are grouped together to avoid overhead. original_status and orderitems are arrays with "count" elements.

Feature "not_orderreturn":
Module_Notify_OrderReturn_StatusChange( module var, original_status, orderreturn var )
This function is called whenever the status of an OrderReturn changes.

Feature "not_ordershpmnt":
Module_Notify_OrderShipment_StatusChange( module var, original_status, ordershipment var )
This function is called whenever the status of an OrderShipment changes.

[burch]

Re: Code name "Wombat" API changes

Payment Module API changes:

If a payment module reports API version 5.60 or above, it must implement the following functions in addition to the existing PaymentModule_Authorize and PaymentModule_Process:

PaymentModule_Runtime_Authorize( module var, module_data, total )
Similar to the previous PaymentModule_Authorize function, except that the module is responsible for inserting an appropriate OrderPayment record after a successful authorization.

PaymentModule_Order_Authorize( module var, module_data, order var, amount )
This function is used for authorization from the administrative interface. It is also repsonsible for inserting an appropriate OrderPayment record after a successful authorization. All of the module-specific fields that would be present during a runtime authorization are also present.

PaymentModule_OrderPayment_Capture( module var, order var, auth_payment var, auth_pay_data var, auth_secure_data var, amount )
This function is called to do a complete or partial capture of a preexisting authorization. The module is responsible for updating or inserting any OrderPayment records required to reflect the new transaction state.

PaymentModule_OrderPayment_Refund( module var, module_data, order var, capture_payment var, capture_pay_data var, capture_secure_data var, amount )
This function performs a refund. The "module_data" parameter will probably be removed before production. Again, the module is responsible for updating or inserting any required OrderPayment records.

PaymentModule_OrderPayment_VOID( module var, data var, order var, auth_payment var, auth_pay_data var, auth_secure_data var, amount )
This function performs a VOID. The "data" parameter will probably be removed before production. Again, the module is responsible for updating or inserting any required OrderPayment records.

All functions return 1 on success or 0 on failure.

[wcw]

Re: Code name "Wombat" API changes

Jon,

In admin, when you click the New Order button you can create an order. What are the API functions involved. Specifically what are the functions for the fulfillment modules? Also would this be a new feature, eg we have vis_order for a regular order edit screen. The legacy order processing still works fine with Module_Order_Update. So either something different needs to be done in the code or a new API function is involved or the API version of the module has to be bumped up. Could you give me a hint so I don't spend a lot of time on the wrong assumption.

[wcw]

Re: Code name "Wombat" API changes

FWIW. After 4 hours of trying to figure this out blindly, it turns out it is a show stopper bug in Merchant. I'll contact Jon.

[wcw]

Re: Code name "Wombat" API changes

Rick,

I tried contacting Jon and he has not responded. Where do I report this show stopper bug? This should be fixed before RC1.

[Rick Wilson]

Re: Code name "Wombat" API changes

Email me as well, virtually everyone in the office was in an all day PA-DSS security training yesterday which is probably why you didn't hear from Jon.

[RayYates]

Re: Code name "Wombat" API changes

I documented all the files and functions in the Wombat Beta 2 LSK. The link below provides a complete searchable listing.

Total Files: 180
Total Functions: 3756

If you save the page to your local hard drive and save it in the folder where you unpacked the LSK you will be able to click on the file links to open the file in what ever program you have configured to open .mv files.

http://www.pcinet-llc.com/docs/MM5_L..._List_WB2.html

You can still find the Core-17 Beta LSK here.

http://www.pcinet-llc.com/docs/MM5_L...tion_List.html

Enjoy

[RayYates]

Re: Code name "Wombat" API changes

Good conference guys, congrats.

I was exploring the new mmlsk-cmp-cssui-links componenet.

First it did not get installed as a page item during installation. I had to install it manually.

In the LSK mmlsk-cmp-cssui-links.mv ComponentModule_Render_Start()
checkout is listed twice with two different ways to render it.

Also it seems like it needs a few more options:
<MvELSEIF EXPR = "{ l.param EQ 'cattree' }">
<MvELSEIF EXPR = "{ l.param EQ 'category-product' }">
<MvELSEIF EXPR = "{ l.param EQ 'product' }">

And while your at it, could someone make this help page actually match the Domain SEO Settings tab.

http://docs.mivamerchant.com/en-US/m...ttings.htm#seo

[RayYates]

Re: Code name &quot;Wombat&quot; API changes

Also for the mmlsk-cmp-cssui-links, their was some discussion about making the format a template. for the links. ON the Domain | SEO Settings tab you are already displaying a "preview" of the links.

What if you added a tab to mmlsk-cmp-cssui-links containing a "template" input box for each of these links (like the button templates)?

For example the category link template might look like this. If someone wanted to add a class, here is where they would do it.

<a class="catlink" href="http://&mvt:domain:name;/&mvt:seo_settings:cat_lit;&mvt:seo_settings:url_de lim;%catcode%&mvt:seo_settings:suffix;">

In this example the template would be initilized once and saved as a string (appended to l.settings:seo_settings) then only %catcode% would be replaced during page rendering. This way the entire string would not have to be re-rendered each time it's called.

[pbrynie]

Re: Code name &quot;Wombat&quot; API changes

As of Beta 6, I believe, we added the ability to pass additional parameters in the following formats...

<mvt:item name="cssui_links" param="product:CODE">Product Link</mvt:item>
<mvt:item name="cssui_links" param="category:CODE">Category Link</mvt:item>
<mvt:item name="cssui_links" param="screen:CODE">Screen Link</mvt:item>

Using these you are able to jump directly to one of these particular resources.

As for the ability to control additional attributes for these generated anchors, we've come up with the following format as a possibility...

<mvt:item name="cssui_links" param="product:CODE;class:bigLink;rel:nofollow;">W idget 1</mvt:item>
<mvt:item name="cssui_links" param="sitemap;class:smallLink;target:_blank">Site map</mvt:item>

This format would be a semi-colon delimited, with the individual name/value pairs separated with colons. This way, it would allow developers to insert any attribute into any link, on a link by link basis. The alternative, using templates, would control the attribute content for all links of each type.