Groovy MDM

Here is the video that shows how easy it is to build a simple GUI for an MDM Server transaction.

This video shows real, live web services making calls from the sMash environment to MDM Server through a simple http request.

I’ve recorded a few videos showing the integration between InfoSphere MDM Server and WebSphere sMash.  There are three parts:

1. Resource Modeling Tool -  How you define attributes and labels for your demo
2. Web Services Calling MDMS - Real transactions invoked by a simple http requests
3. Building the UI - A two minute GUI

The display quality of the you tube posts are not great, so I will eventually upload the high res recordings and post links here.  But for now, here is the you tube posting for part 1 of 3:

Apparently, you can’t designate any arbitrary attribute as the resource identifier.  (At least, the model editor GUI does not support it.)  Rather, sMash just assumes the identifier will be an attribute called, “id”.

This became as issue when I tried to implement the delete and update services.  The sMash runtime should pass the identifier value in the calling URL for the web service.  e.g. http://localhost:8080/resources/personModel/12358983622.  That tells the web service which instance to act on. But before I added “id” to my resource model, the URL did not have any identifier value tacked on the end.

I took a number of steps to correct this problem so I don’t know which ones actually made the difference.  First, I added an attribute called “id” to the resource model.  Second, I ran the console command to generate the tables for this resource.  Third, I replaced the resource handler code with ZRM.delegate() and tested the application using the default relational persistence.  That worked.  And when I rewired the app to the custom handler that hooks to MDM Server, it still worked.  I don’t know whether generating the tables was actually necessary.

Firebug and Poster are two very useful tools for web development.  Both are Firefox plug-ins.  (Gotta love open source.)

Poster is a simple http client that enabled me to test the RESTful web services before building the UI.  This is a great programming model.  Encapsulated, reusable services…but so much easier to build and use than the full blown, bulky web services we usually think of.  No WSDL, no SOAP messages, etc.

Firebug monitors the browser.  With it, I can watch all of the individual http calls when the application page loads.  Very useful when debugging.  But, Firebug does so much more.  Want to watch the javascript code execute?  Want to step through the javascript execution and set break points?

The MDMS Handler model maps the RESTful web services invoked by sMash to MDM Server services.  Here is an example of the handler for the “person” resource.

/public/handlers/personModel.json:

{
“model”: “personModel”,
“CREATE”: {
“request_type”: “addPerson”,
“is_new_style_doc”: false,
“service_type”: “Tx”,
“wrapper_input”: “mdmPerson”,
“wrapper_output”: “mdmPersonSearch”
},
“DELETE”: {
“request_type”: “deletePerson”,
“is_new_style_doc”: false,
“service_type”: “Tx”,
“wrapper_input”: “mdmPerson”,
“wrapper_output”: “mdmPersonSearch”
},
“LIST”: {
“request_type”: “searchPerson”,
“is_new_style_doc”: false,
“service_type”: “Tx”,
“wrapper_input”: “mdmPersonSearch”,
“wrapper_output”: “mdmPersonSearchResult”
},
“RETRIEVE”: {
“request_type”: “getPerson”,
“is_new_style_doc”: false,
“service_type”: “Tx”,
“wrapper_input”: “mdmPerson”,
“wrapper_output”: “mdmPersonSearch”
},
“UPDATE”: {
“request_type”: “updatePerson”,
“is_new_style_doc”: false,
“service_type”: “Tx”,
“wrapper_input”: “mdmPerson”,
“wrapper_output”: “mdmPersonSearch”
}
}

The “wrapper” binds the sMash resource model to the MDM Server BObj, matching model attribute names to BObj xpaths.  This is the wrapper (represented in json like the resource model) for the TCRMPersonObj.  The sMash file editor insures correct json syntax, but I created this by typing it in manually.  In addition to attribute mapping information included in “public names”, the wrapper identifies the child BObjs of interest and points to the XML prototype for the BObj.  I’m not fully explaining the full format here.

/public/wrappers/mdmPerson.json:

{
“public_names”: {
“Id”: { “path”: “PartyId” },
“First Name”: { “path”: “TCRMPersonNameBObj/GivenNameOne” },
“Last Name”: { “path”: “TCRMPersonNameBObj/LastName” },
“Contact Method”: { “path”: “TCRMPartyContactMethodBObj”, “child”: “party_contact_method” },
“Phone”: { “path”: “TCRMPartyContactMethodBObj”, “child”: “party_contact_method”, “type_filter”: “1″ },
“Email”: { “path”: “TCRMPartyContactMethodBObj”, “child”: “party_contact_method”, “type_filter”: “9″ },
“Birthdate”: { “path”: “BirthDate” },
“Gender”: { “path”: “GenderType” },
“Soc Sec No”: { “path”: “TCRMPartyIdentificationBObj/IdentificationNumber” },
“Address Line 1″: { “path”: “TCRMPartyAddressBObj/TCRMAddressBObj/AddressLineOne” },
“Address Line 2″: { “path”: “TCRMPartyAddressBObj/TCRMAddressBObj/AddressLineTwo” },
“City”: { “path”: “TCRMPartyAddressBObj/TCRMAddressBObj/City” },
“State ID”: { “path”: “TCRMPartyAddressBObj/TCRMAddressBObj/ProvinceStateType” },
“State”: { “path”: “TCRMPartyAddressBObj/TCRMAddressBObj/ProvinceStateValue” },
“Zip”: { “path”: “TCRMPartyAddressBObj/TCRMAddressBObj/ZipPostalCode” },
“Country ID”: { “path”: “TCRMPartyAddressBObj/TCRMAddressBObj/CountryType” },
“Country”: { “path”: “TCRMPartyAddressBObj/TCRMAddressBObj/CountryValue” },
“Faves”: { “path”: “TCRMPartyValueBObj”, “child”: “party_value” }
},
“children”: {
“party_contact_method”: “mdmPartyContactMethod”,
“party_value”: “mdmPartyValue”
},
“xml”: “TCRMPersonBObj”
}

This is what a sMash resource model looks like.  Models describe resource attributes and optionally define collections and filter conditions.  This is a model for a “person” which would correspond to an MDM Server TCRMPersonBObj.  sMash provides GUI tools to create these models.  For my MDM Server application in this environment, I’m creating one resource model for each enity of interest, e.g. person, contract,product, etc.

Models by intention present a simplified, flattened out view of the entity.  That is, they won’t include all of the attributes or all of the nested object relationshipships.  (MDM Server flattens out objects too, by the way.  Look at the difference between a TCRMPersonSearchResultBObj and a TCRMPersonBObj.)

/app/resources/personModel.json:

{
“fields”: {
“FirstName”: {
“label”: “First Name”,
“required”: false,
“type”: “string”,
“description”: “”,
“default_value”: “”,
“max_length”: 50
},
“LastName”: {
“label”: “Last Name”,
“required”: false,
“type”: “string”,
“description”: “”,
“default_value”: “”,
“max_length”: 50
},
“Phone”: {
“label”: “Phone.Listing”,
“required”: true,
“type”: “string”,
“description”: “”,
“default_value”: “”,
“max_length”: 50
},
“SocSecNo”: {
“label”: “Soc Sec No”,
“required”: false,
“type”: “string”,
“description”: “”,
“default_value”: “”,
“max_length”: 50
},
/* …Attributes Omitted…*/
“id”: {
“label”: “Id”,
“required”: false,
“type”: “string”,
“description”: “”,
“default_value”: “”,
“max_length”: 50
}
},
“collections”: {
}
}

Here’s a video that really captures the kind of development methodology and application architecture that, I think, works extremely well for MDM Server.  Rob Nicholson of IBM uses WebSphere sMash as a web 2.0 development platform that can call back end like transactions, like CICS.

The video, starting at about the 2:55 mark, starts illustrating the kind of blueprint I’m following. Resource handlers for RESTful web services, tooling for simple, quick UI generation, next-gen application mash ups, etc.

Just keeping track of the things I need to come back to:

• MDM Server doesn’t like the TCRMPartyContactMethodBObj’s in my addPerson XML.  I could add the phone number, but not the e-mail address.  It complained that I was using either an incorrect contact method category or contact method type.  My entries matched the code tables as far as I could tell.  Just dropped the e-mail for now.
• The data grid in the UI seems to break now when I retrieve an unfiltered list of persons (roughly about 110 of them).  But if I reduce the result set with search parameters, no problem.  Odd.  That was working yesterday even with a hundred names.
• This config is really taxing my poor laptop. Running the VMWare image, RMI Gateway, and sMash runtime causes my memory to start swapping out pretty badly. Definitely gets worse the longer I work with it.
• The UI doesn’t give very good visual indicators when requests are still being processed.
• The data grid looks a little barren because I’m only populating a handful of attributes.  I might need to think a little more about which attributes display in what context.  For example, TCRMPersonSearchBObj’s returned by the inList() service don’t have all the wrapper attributes, unlike the TCRMPersonBObj’s.  Need to think about the right way to configure all of that.

I’ve now wired the second of the RESTful web services, onAdd().  My demo application can now retrieve a list of Persons and add new Persons (by executing MDM server transactions that have been mapped.) No changes need to the UI since the Dojo widgets implicitly support the CRUDL actions.  I still need to add support for onRetrieve(), onUpdate(), and onDelete().  There’s a sample collection handler on the Project Zero website that has served as my pattern.  (http://www.projectzero.org/sMash/1.0.x/docs/zero.devguide.doc/zero.core/REST.html)

Every resource needs a handler to perform the RESTful web services.  By convention, WebSphere sMash will look for a file with the same name (as the resource model ) in the app/resources directory, except with a .groovy file extention rather than .json.

But I don’t want to script custom handlers for each resource.  So, I can write a single handler script and assign it to my resources by configuring the zero.config file.  Or, I can continue having separate handler files for each resource but use the same generic (write-one-time) code in each.  sMash uses the second approach in their published examples.  If you want to let sMash implicitly manage the persistence and retrieval, for example, you can put the same single line of code in each handler, i.e. ZRM.delegate().

On principal, I prefer the first approach.   However, doing that forfeits some of intelligence you get for free with the separate handler files.  Doing it the second way, you only need to implement methods that correspond directly to the RESTful web services, e.g. onList(), onCreate(), onDelete(), onUpdate(), and onRetrieve().  The sMash runtime will automatically invoke the correct method based on the context.  With the single, uber-handler, you respond to lower level events, like onPOST(), onGET(), etc., and must know how to interpret and process them.  So, separate handler files containing the same script it is.

In my handler code, I grab the name of the resource from the request path.  That lets me load the correct handler configuration file for the resource, i.e. the file that maps the RESTful services to the MDM services.

The ZRM Data store called by UI Dojo widget appears to be making unnecessary GETs to the resource handler.  I noticed this initially by watching the traffic flowing over the RMI Gateway.  I saw three searchPerson service invocations every time I loaded or refreshed the index.html page.

One of the developers on the Project Zero forums pointed me to Firebug, a Firefox plug in, that lets you view and debug the client side processing.  Sure enough, the console monitor showed the three calls:

Not a big deal for the demo, but unacceptable for this use case.  Extra external calls are too expensive.  Certainly, you don’t want extra calls on the onAdd() method.  I’ve posted the suspected defect to the Project Zero bug tracking forum.

Success!  Well, at least a successful first step.  My first application retrieves person data from MDM Server (calling the searchPerson service) and presents the result list in a Dojo data grid and form:

More importantly, this required zero programming.  I did some configuration and built several application artifacts with tools.  That’s all. The key, high level tasks included:

1. Define a “resource” called person and it’s attributes:  WebSphere sMash provides a simple GUI tool for modeling resources, i.e. assigning a name, defining attribute names, data types, constraints, etc.  The tool generates a JSON (Java Script Object Notation) representation of the model.
2. Configure a “wrapper” that maps this resource model to the MDM Server BObj, TCRMPersonBObj: A Groovy wrapper, among other things, associates resource model attributes with their corresponding BObj xPaths.  I used the sMash file editor to manually enter a JSON representation of the mappings.
3. Configure a “handler” that maps the sMash RESTful web services to MDM Server BObj’s and services: A Groovy handler associates sMash RESTful web services, i.e. onList(), onRetrieve(), onDelete(), onUpdate(), and OnCreate(), to the corresponding MDM Server services.  Again, I used the sMash file editor to manually enter a JSON representation of the mappings.  In this case, I mapped only onList() to searchPerson.
4. Overwrite the resource handler script for person generated by sMash with the generic MDM handler services: This delegates the responsibility for retrieving and updating the data to the generic MDM handler class.
5. Test the handler services: Testing the services with “Poster”, a Firefox plug-in, verified the connectivity to MDM Server and allowed me to inspect the retrieved BObj’s.
6. Build the client UI: The UI tools in sMash made this the easiest part of the whole exercise.  I created an html page and defined a ZRM data store (an abstraction for the person resource) with a few clicks.  After dragging and dropping a couple Dojo widgets onto the page and setting some properties, I was done.
7. Test the application: I launched the application from the sMash development environment and saw the MDM data on my screen.

Anyone familiar with the MDM Server transactions and BObjs can do this kind of configuration very easily.  And those who don’t know MDM Server at all, can learn to build the GUI’s from pre-configured handlers in sMash.

I’ve been writing, implementing, and now selling enterprise software for quite awhile, and I’m still perplexed by the complexity of it all.  Does it all really need to be this difficult?

I don’ think so.  Powerful scripting languages and new web 2.0 tools and technologies can radically simplify software development.  So I began to wonder, can I create robust, enterprise applications without coding at all?  I’m taking up that challenge.

I will build applications with zero programming.  Trivial little demo applications at first, of course.  But I intend to tackle increasingly sophisticated requirements and use cases. My tools are the Groovy scripting language and two familiar enterprise software packages from IBM, namely WebSphere sMash and InfoSphere Master Data Management Server.

But first, what exactly do I mean by “zero programming”? I can still write some code, of course, just not application code.  Building reusable, general purpose components and services is still fair game. Zero programming means the next guy who comes along can create his own app by configuring what’s already been built.

I’m ready to begin.