Custom functions to extend ICS transformations – ICS Definitive Guide #4

In the second release of 2017 (17.2.5) ICS will introduce the capability of importing and using custom functions. These functions are created using JavaScript and can be used in transformations, expressions and as a action in Orchestrations. In this definitive guide we will go through all ins and outs of this new capability. 

Custom functions – a new pillar of ICS

In our book we talked about the pillars of ICS; Connections, Integrations, Lookups, Agents, Adapters and Packages. This release introduced Libraries. A library is a set of Javascript functions. Keep in mind that the JavaScript functions are running server-side so some browser/client-side capabilities/APIs are not available. Functions can’t be created within ICS, but can be uploaded as part of a library in both JS and JAR file (collection of JS files) formats.

For this guide we have created two library files; string-utils and date-utils. The string-utils library contains seven functions and the date-utils library contains only two functions. Below is the contents of the latter library, where the first function calculates the duration between to two dates by returning a dayTimeDuration string as result. The second function is used for parsing ISO dates since the javascript engine ICS uses does not support ISO date when creating new Dates in javascript.

function getDurationBetweenDates(dateStr1,dateStr2) {
  var secStr1 = parseDate(dateStr1) / 1000;
  var secStr2 = parseDate(dateStr2) / 1000;

  var seconds = (secStr2 - secStr1) % 60;
  var minutes = Math.floor(((secStr2/60) - (secStr1/60)) % 60);
  var hours = Math.floor(((secStr2/3600) - (secStr1/3600)) % 24);

  if (seconds < 0 || minutes < 0 || hours < 0) {
    duration = "-PT" + Math.abs(hours) + "H" + Math.abs(minutes) + 
               "M" + Math.abs(seconds) + "S";
  } else {
    duration = "PT" + Math.abs(hours) + "H" + Math.abs(minutes) + 
               "M" + Math.abs(seconds) + "S";
  }
 
  return duration;
}

function parseDate(myDate){
  var parts, date, time, dt, ms;

  parts = myDate.split(/[T ]/); // Split on `T` to get date and time
  date = parts[0];
  time = parts[1];

  dt = new Date();

  parts = date.split(/[-\/]/); // Split date on - or /
  dt.setFullYear(parseInt(parts[0], 10));
  dt.setMonth(parseInt(parts[1], 10) - 1); // Months start at 0 in JS
  dt.setDate(parseInt(parts[2], 10));

  parts = time.split(/:/); // Split time on :
  dt.setHours(parseInt(parts[0], 10));
  dt.setMinutes(parseInt(parts[1], 10));
  dt.setSeconds(parseInt(parts[2], 10));

  ms = dt.getTime();
  return ms;
}

Because we want to have separate libraries for both types of utility functions, instead of making a JAR file with both libraries we will upload them as single JS files. To upload a library navigate to the Designer page and click on Libraries.

Register Libraries

This will navigate us to the Libaries page. On this page we can register our libraries by clicking on the Register button.

This will open a popup to register the library with. First we select the library file we want to upload and give the Library a name, a version, and a small description. By giving it a version you can add updated versions of the library. Once a library function is used by an integration the library can’t be deleted. So we can upload a newer version when a function is changed instead of removing all usages first.

Before we can use functions within integrations and transformations we need to configure all functions we want to use. For instance ICS needs to know the input and output types of the JavaScript arguments and in which components you want to make the functions available. In the current version (17.1.3 Early uptake) functions can only me made available in orchestrations, but in the future we will see the ability to also use functions in transformations using XPath and by adapters.

In this case the library only has one function name getDurationBetweenDates which has two string input arguments (dateStr1, dateStr2) and one string output argument (duration). When configuring the arguments we can select between three basic data types; boolean, number and string.

 

Also configure the parseDate function (both input and output are String). After configuring all the functions we can save the library and exit by clicking on done. After the page is refreshed we return to the libraries section of the designer where we can upload our second library String Utils – 1.0. This library, as we mentioned before, has seven functions to configure.

The following table shows the values you need to use in the configuration:

Function Name	Input(s)	Output
escapeForRegex	arg : string	arg : string
substringAfterIfContains	arg : string delim : string	subStr : string
substringAfterLast	arg : string delim : string	arg : string
substringAfterLastMatch	arg : string regex : string	arg : string
substringBeforeIfContains	arg : string delim : string	subStr : string
substringBeforeLast	arg : string delim : string	arg : string
substringBeforeLastMatch	arg : string regex : string	arg : string

Functions can call other functions available in same library. For example as shown in the next small snippet of the string-utils library. Notice that the function substringAfterLast calls the function escapeForRegex.

function substringAfterLast(arg, delim) {
  var pattern = "^.*"+escapeForRegex(delim);
  var re = new RegExp(pattern);
  arg = arg.replace(re, "");
  return arg;
}

function escapeForRegex(arg) {
  arg = arg.replace(/(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\))/g, "\\$1");
  return arg;
}

After configuring the functions we want to use i.e. substringAfterLast and getDurationBetweenDates we can use them in an integration based on the orchestration pattern.

Creating the orchestration

You can use any pre-created connections or like us create an inbound REST adapter and an outbound SOAP adapter named FlightSchedulesREST and FlightSchedulesASOAP (the resources to create these connections are linked below). The SOAP endpoint we invoke returns the flight schedule of the given Flight identification. Because we already discussed this many times we skip this for now. Navigate to the Integrations section in the Designer and create a new Orchestration.

Complete the create Orchestration with the following details:

Property	Values
Integration Name	FlightSchedules_Dg4
Identifier	This will be proposed based on the connection name and there is no need to change unless you’d like an alternate name.
Version	01.00.0000
Package Name	ics.blog.dg4
Description	This orchestration demonstrates the use of functions to enrich the result before returning a response.

Drag and drop your connection you want to expose to the trigger drop point. In our case we re-use the existing FlightSchedulesREST connection.

Let’s create the REST connection using the following details:

Step	Field	Value
Basic Info	What do you want to call your endpoint?	RetrieveSchedule
	What does the endpoint do?	Retrieve flight schedule for given flight identification
	Relative resource URI?	/schedules/{id}
	Action to perform?	POST
	Select options to configure	Tick the boxes  – Add and review parameters […] – Configure […] the response
Request Parameters	Template parameter: id	string
Response	Select response payload	XML schema
	Schema location	Choose File: FlightSchedules.xsd
	Element	GetScheduleResult
	Select type of payload	JSON

After going through above steps of the wizard the summary is shown displaying the REST Service URI, HTTP Method and Response Media Type. Click on Done to complete the wizard. This will create the trigger and for the response a Map and Return activity. Now we can drag and drop our SOAP connection we are going to invoke. For this we will be re-using our FlightSchedulesSOAP connection.

Create the SOAP connection using the following details:

Step	Field	Value
Basic Info	What do you want to call your endpoint?	GetSchedule
	What does the endpoint do?	Get schedule of given flight
	Use updated SOAP adapter runtime	Yes
Operations	Configure Headers	No
	Selected Port & Operation	FlightSchedulesSoap and GetSchedule

After going through above steps of the wizard the summary is shown displaying the WSDL URL, Selected Port, Operation and Message Exchange Pattern. Click on Done to complete the wizard. This will create the invoke activity and a Map activity to initialize the request. Click on the Map activity GetSchedule and map the element id from the TemplateParameters source to the ident element of the GetSchedule target. Click Save and Exit Mapper to return to the integration canvas.

We can now do the same for the response mapping. Click on the Map activity RetrieveSchedule and edit the mapping. With this mapping we can initialize the response. The response will use one of the custom functions we registered.

Because the XSD we used to create out REST adapter trigger has the same structure as the SOAP response we can map the object as is. Drag and drop the GetScheduleResult element from the source panel to the GetScheduleResult of the target panel. Because all the field are the same the child elements can be mapped automatically. In the Confimation dialog click Yes to accept auto mapping.

We want to calculate the duration of the flight between the departure and arrival time and return that in the duration element. We can’t do this directly in the mapping yet, but we can initialize a separate variable that holds the result using the Callout action and use that in the mapping. For now we can click on Save and Exit Mapper. Back in the Integration canvas we can drag and drop the Callout action from the Actions palette after the GetSchedule invoke and before the mapping.

This will show the Callout dialog. We will call the Callout FlightDuration so use that as the name of the action. To begin we need to select the function we want to call. Click on the button Function with the plus sign in front of it to select one.

The functions that are listed are the functions that are configured within the library. If a function is not configured correctly is will not show up in the list. In our case we select the function getDurationBetweenDates which is included in the Date Utils library.

After selecting the function we return to the Callout configuration page where we can map departureTime and arrivalTime elements to the dateStr1 and dateStr2 parameters.  For both parameters click on the pencil icon to the right to edit the value.

For the dateStr1 parameter use the expression:

$GetSchedule/nssrcmpr:GetScheduleResult/nssrcmpr:departureTime

For the dateStr2 parameter use the expression:

$GetSchedule/nssrcmpr:GetScheduleResult/nssrcmpr:arrivalTime

As you can in the fragments see we have provided a XPath to the values needed. Now that we have configured the callout we can save and exit the callout configuration page. Click on Done to return to the Integration canvas.

For here we can edit the RetrieveSchedule mapping again. Click on the Map activity and on the pencil icon to edit the mapping. Notice on the Source panel a new variable is visible which we can use in the mapping. First remove the mapping on the target element duration. This can be easily be done by right-clicking on the duration element and choose for the option Delete Mapping. In the confirmation dialog shown click on Yes. Finally we can drag and drop the source element output_FlightDuration_duration to the target element duration.

The final mapping is done. Click Save and Exit Mapper to return to the Integration Canvas. Before we can activate the orchestration we need to setup a business identifier. Click on the Tracking button on the top right corner of the canvas and create a tracking field based on the Template Parameter id. Give the tracking field the name Flight Id to make it more understandable.

Close the Business Identifier dialog and Save and Exit the Integration. The final result of the Orchestration looks like the following process.

Testing the function

Finally Save and Exit the integration canvas and activate the integration by clicking on the activation switch on the right side of the entry.

Once the confirmation dialog is shown click on the Activate button once again. After activation we are presented with the URL of the REST endpoint of the integration.

With the shown url we can’t retrieve the flight schedule, we do know the base URL and by visiting this URL we know which resource is available i.e. the resource we configured.

For example the metadata URL runs on:

https://ics4emeapartner-partnercloud17.integration.us2.oraclecloud.com/integration/flowapi/rest/FLIGHTSCHEDULES_DG4/v01/metadata

Based on that our resource for retrieving the flight schedule for flight KL1606 is:

https://ics4emeapartner-partnercloud17.integration.us2.oraclecloud.com/integration/flowapi/rest/FLIGHTSCHEDULES_DG4/v01/schedules/KL1606

Lets try to retrieve the flight KL1606 and check if the function did its job.

And the answer is YES. If we look at both departureTime and arrivalTime. Between the two dates are 11 hours, 35 minutes and 0 seconds which we see returned as a dayTime duration string.

One thing to keep in mind…

As we mentioned before, keep in mind functions only work with booleans, numbers and strings. The data type you use as a source of the function parameters should be in the same data type. Some functions may not worked correctly if the data type is different. You can of course address this with XSL type conversion operastions.

Resources

Here are the resources that we used that can be downloaded:

• DateUtils JavaScript library
• StringUtils JavaScript library
• Connection creation resources