IBM App Connect Enterprise (ACE)-Group Nodes
Introduction
Just like Software Design Patterns there are Enterprise Integration Patterns and they come in handy when designing middle-ware solutions. In this write-up, we’ll look at a example for Scatter-Gather pattern, one commonly used and very useful Enterprise Integration Pattern. As the name indicates this pattern scatters request messages, sends it to single or multiple end-points and gathers response messages. Assume there is a service, which when we pass a user id, responds with user information. Our requirement is to get a list of user information as response, calling the service multiple times with different user ids. This can be achieved by App Connects’ Group Nodes. Prior to version 10, this scenario required an IBM MQ and aggregation nodes. But with the introduction of Group Nodes in version 10, this mediation has become simple and straight forward. Let’s see how this can be achieved by the following example.
Example REST Project
For this exercise we are going to use the free online API found at https://jsonplaceholder.typicode.com/. You can check the functionality of this API using Postman and check out their documentation. For example when I call this end point https://jsonplaceholder.typicode.com/users/1, user information for id 1 will be returned. Below is the response I got.
{
"id": 1,
"name": "Leanne Graham",
"username: "Bret",
"email": "Sincere@april.biz",
"address": {
"street": "Kulas Light",
"suite": "Apt. 556",
"city": "Gwenborough",
"zipcode": "92998–3874",
"geo": {
"lat": "-37.3159",
"lng": "81.1496"
}
},
"phone": "1–770–736–8031 x56442",
"website": "hildegard.org",
"company": {
"name": "Romaguera-Crona",
"catchPhrase": "Multi-layered client-server neural-net",
"bs": "harness real-time e-markets"
}
}This API has a set of users ranging from user id 1 to 10. What we are going to do is develop a REST API which takes in two parameters start and end, fetch all those users between a given range and consolidate as a single JSON response. For example if I pass start = 1 and end = 3 user information 1,2,3 will be given as a single response.
Let’s see how we can achieve this with the help of IBM App Connect Enterprise group nodes.
First I created a REST API project with query parameters start and end. start and end parameters would act as the range indicators of user ids.
Below would be my message flow implementation.
One by one I’ll list the code listing and node properties.
Compute node getParams will be setting up the stage. It copies all the InputRoot and InputLocalEnvironment values, so that they can be used in the preceding node.
SET OutputRoot = InputRoot;
SET OutputLocalEnvironment = InputLocalEnvironment;
SET Environment.Destination.GroupScatter.Context = InputLocalEnvironment.Destination;
Take note of the line where GroupScatter.Context is being set with the LocalEnvironment destination. This line indicates, once all the processing is complete, where the response should be sent, the actual initiating source.
Next would be the group scatter node where the fan out process begins.
Take note the Group name property we have given. This would be unique for Group Gather and Group Complete nodes.
Next is the prepareInputs compute node. As the name indicates this is the node where the preparation of request occurs for the subsequent HTTP Asynchronous Request node. Between the BEGIN and END block following ESQL code should be written.
DECLARE I,J INT 1;
SET I = InputLocalEnvironment.HTTP.Input.QueryString.start;
SET J = InputLocalEnvironment.HTTP.Input.QueryString.end; WHILE I <= J DO
SET OutputLocalEnvironment.Destination.HTTP.RequestURL = 'http://jsonplaceholder.typicode.com/users/' || CAST(I AS CHARACTER);
SET I = I + 1;
PROPAGATE TO TERMINAL ‘out’;
END WHILE; RETURN FALSE;
First two lines are variable declarations. Next two lines are getting the query parameter values and setting them for the declared variables. Next is a loop which will iterate from the start to end values. Take note that the RETURN is FALSE. This is to stop the compute node from moving to the next node before completing all the iterations . With each iteration the endpoint will change based on the value of I which is found at the end of the URL as path parameter.
Next would be the HTTP Asynchronous Request. Properties would be Unique identifier and Web service URL. I have given a dummy value for Web service URL since this value would be overridden in the ESQL code in the previous compute node.
In the Group tab of properties tick the Add request to group and provide a Request folder name. This name will be used to gather the responses in the consolidateMsgs compute node.
Following will be the HTTP Asynchronous Response node properties.
Next is the Group Gather node and Group Complete nodes. Always keep in mind that the Group name property should be the same. Finally a compute node to consolidate received response messages. This is will be the ESQL between the BEGIN and END block.
DECLARE card INTEGER;
DECLARE idx INTEGER 1;
DECLARE Ref_grpRef reference TO InputRoot.ComIbmGroupCompleteNode.Group.Replies;
CREATE FIELD OutputRoot.JSON.Data.result IDENTITY(JSON.Array)result;
SET card = CARDINALITY(Ref_grpRef.FLDREMP[]);
WHILE idx <= card DO
SET OutputRoot.JSON.Data.result.Item[idx] = Ref_grpRef.FLDREMP[idx].Reply.Root.JSON.Data;
SET idx = idx + 1;
END WHILE;
SET OutputLocalEnvironment.Destination = InputRoot.ComIbmGroupCompleteNode.Group.Context;
RETURN TRUE;First two lines are a declarations. Next line would be a reference to InputRoot.ComIbmGroupCompleteNode.Group.Replies We create a JSON array to store the JSON responses. Get the cardinality of the FLDREMP where the responses got stored. We iterate and build the consolidated response in the next few lines and tell the App Connect where the response need to be sent by the following line of code.
SET OutputLocalEnvironment.Destination = InputRoot.ComIbmGroupCompleteNode.Group.Context;That’s it, we have done a basic sample for group nodes using IBM App Connect Enterprise. I’m not going to put the combined response since it will take up space but if done correctly, this code and configuration will work without any issues.
Conclusion
Service orchestration or implementing a Scatter-Gather Pattern is simple and straight forward in IBM App Connect just as any other middle-ware . In the IIB previous versions Aggregate nodes gave the same functionality but required an IBM MQ. With Group Nodes, need of an IBM MQ is eliminated. Few points to consider when developing the message flow,
- GroupScatter, GroupGather, GroupComplete should contain the same Group name property (In this example I have given
GRPTST). - Value of the Request folder property in the HTTP Asynchronous Request node would be referred while consolidating the messages using
InputRoot.ComIbmGroupCompleteNode.Group.Replies. - Timeouts should be carefully set for large groupings based on the API response times for GroupScatter, HTTP Asynchronous Request, and Group Gather nodes.
- HTTP Asynchronous Request and HTTP Asynchronous Response should have the same Unique identifier property
- In the given message flow error/exception handling is not done. You can observe most of the terminals are not connected. Only the happy path scenario is given here. For a complete solution all the terminals should be connected and errors/exceptions should be handled correctly.
