Joseph Vogel

iWay will now distribute a docker image which when instantiated will act as an ISM server very similar to a standalone server deployment. We are going to demonstrate iWay Service Manager running in a docker container with support for persistence. In this tutorial we will be using Docker Desktop running on a Microsoft Windows 2011 operating system First we need to load iWay Container Edition into Docker desktop. To do this on Windows open a command prompt and navigate to a directory which contains the docker file tar file. To load the docker image enter the following: docker load -i iwayContainerEdition92.tar Bringing up Docker Desktop we should see the newly added docker image, iwaycontainer. To create a docker container using this docker image click on the "run" icon (right arrow) on the right. A dialog will appear as shown below: The most minimal configuration for this container would be to map the iSM Console port as well as the iSM IBSE SOAP port. For support of persistence which means that when the container is brought down it remembers its previous state you will need to map to volumes. One for ISM's configurations and the other for its logging services HTTP/HTTPS Port Assignments Port Mappings Exposed Port Required Description Port Mappings Exposed Port Required Description ISM Runtime Console 9999 Yes Used to expose the ISM Runtime Console IBSE SOAP Port 9000 Yes Used for both iWay Business Service (SOAP) and iWay Integration Tools, IIT Envoy 8000 No Used to be monitored via SPOG/Sentinel technology Container Persistence Layer Volume Mapping Internal Location Description Configurations /opt/ibi/iway9/conf Used to persist ISM configurations and data Logging /opt/ibi/iway9/logs Used to persist collected logging information Clicking the Run button as shown above should start ISM in a container as shown below: Bringing up a Web browser and logging into the iSM runtime console using the standard iway/iway userid/password combination should result in the following: A running version of iWay Service Manager running a docker container. Customizing the Deployment The limitations of docker desktop does not allow us to customize the docker run command to the extent that we need. In a production environment you most likely would want to control the hostname of the container as well as the domain of this console. To bring up iWay Service Manager running in a container like this you will need to open a command prompt and enter the following: docker run -v "/c/dev7/test/mylogs:/opt/ibi/iway9/logs" -v "/c/dev7/test/myconfig:/opt/ibi/iway9/conf" -p 9999:9999 -p 9000:9000 -e IWAYWHITELIST=TIBCO-R91391RY --hostname=TIBCO-R91391RY --name myiway iwaycontainer where: Mounts Source Target Description /opt/ibi/iway9/logs /c/dev7/test/mylogs Custom log directory to support persistence /opt/ibi/iway9/conf /c/dev7/test/myconfig Custom configuration directory to support persistence Ports Source Target Description 9999 9999 Mapped Console port 9000 9999 ISM Soap Port Environmental Variables Variable Value Description IWAYWHITELIST <YOUR HOSTNAME> Comma delimited domain names of the server host (used to prevent CSRF attacks) Once running this ISM container will be accessible via both: http://localhost:9999 and http://TIBCO-R91391RY:9999 This should get you a sneak peak into what's coming in iWay 9.2. Look for a more detailed discussions on this topic and other containerization techniques involving iWay in the near future. Comments and feedback are always appreciated. Thanks..

iWay Service Manager ships with a unique connector which empowers developers to build flows which can implement ETL based operations. Challenges developers face in building ETL based integration solutions often come down to both performance and memory usage. The Dataflow connector is designed to assist in these challenges. Lets take a simple example where we are going to move database rows from one table to another using the Data Flow Connector. The Database Below is screenshot of the MySQL Workbench. We are going to use the Community Edition of MySQL In this MySQL instance we have two schemas. One named "demo" and the other "demo-target". Each of these schemas have a table named "models" which share the same database schema. In this example we will move the data found in demo.model to demo-target.model. Building our Flow To get started we load iWay Integration Tools (IIT) and create an application named "dataflowdemo" as shown below. Next we create a flow named "copydata" and drag and drop a Data Flow object from the Palette to the Flow Canvas. As shown below IIT indicates that we have a bit more work to do. We will have to configure the Data Flow connector to get rid of these errors and also draw the proper connections in the flow. We will start by arranging our objects as shown below: The Data Flow Connector is actually an iterator which queries the source database in each pass for a specific number of rows (batch) and creates an XML document which its values can be transformed by using an XSLT, iWay transform or external program. Configuring the Data Flow Connector Now we need to complete the flow by configuring the connector. Click on the "plus sign" to the right of the Data Flow Object's configuration drop down. This will bring up the following dialog: As we can see this connector needs its JNDI/JDBC Configuration requirements met. We will need to do this for both source and target dbs. Depending on your database these values will be different. Since we will be using MySQL yours might look something like this: Once we have our providers defined were back to completing our dataflow generic configuration. When finished your dataflow configuration should look like: Notice how we see no errors.. better yet nothing in "red". Clicking Finish brings back to the main configuration of the objects itself. Notice how we now have met the connectors configuration requirements. All which is left to do is to provide some additional configuration properties which is local to the object. This connector requires both a source and target query. For this example we have decided to keep it fairly simple. Type in both the Source Extraction Query Source Extraction Query SELECT model,categorycode,name,skucount FROM demo.models; and the Target Load Query. Target Load Query INSERT INTO demotarget.models (model, name, categorycode, skucount) values (?model,?name,?categorycode,?skucount) The results of this is that we should not see any errors as shown below. Once completed we need to include the MySQL JDBC driver the which the Data Flow connector will call to perform the data transfer. Adding the MySQL JDBC Driver Double click on the applications bundle will launch its editor. We are interested in the libraries tab as shown below. Eclipse gives you the option to add a driver found within your workspace or to browse your machine for the driver. We downloaded our driver from Maven Central and made it available on our local disk which we now bring in to our application. All our application setup and dependencies are now met. Testing our Application To test our application we have a couple options. We could IIT's step debugger... or invoke IIT's Test Run capability. Since the debugger is a bit more visual lets invoke the debugger by selecting the debug icon and clicking on the runtime launcher. We will use the Debug icon found at the top of IIT as shown below. Once the debugger is started you should see a view similar to the one below. By stepping through the flow you should see the variables change within the variable tag as well as the document be transformed via the execution of the flow. On its first pass of the iteration you will see the first extraction document After the first execution of the Data Flow connector the Source Extraction query is executed and a document is created as shown above. We now can transform the values accordingly which then feeds back into the connector to be loaded into the target data base using the target load query. We call this a batch. The size of the batch is configurable within the connector which can greatly impact both performance and memory usage. If we let this flow complete the end result is that our target database is populated as shown below: Notice that only the columns which were specified in our target query gets populated. The user has full control of what gets extracted, transformed and loaded. Follow Up If you like articles like this... please let us know and comment. Our goal is to provide value to you and your organization in helping you reach your goals. Feedback is always appreciated.

iWay Integration Tools (IIT) enables users to design and implement simple to complex integration patterns. Components of these implementations often share similar properties whose values may change over time or when deployed within different environments such as a development, test or production deployment. The challenge then becomes how to manage these properties in organized predictable manner. This usability challenge is clearly evident when designing flows within IIT's flow editor. IIT's flow editor comes equipped with a tool named Property Manager. Its main objective is to allow users to define variables which then will appear throughout the editor when assigning a value to a component property. When a user wishes to change the value of the property they do that in property manager and all components referencing that variable inherit the new value. Lets take a look at this in action. Here we have a flow named, simpleflow opened up in IIT's flow editor. This flow will use the File Connectors, "copy a file" action to copy a file from a source to target directory. Now we could just type in the paths into the connectors source and target properties, but what if these paths were used often in a more complex flow. What if in the future these paths changed ? To make our lives easier lets just create a couple of variables utilizing Property Manager. As users who know about "Property Manager" as of 9.2 this functionality is now embedded into the flow editors' canvas as shown above. We have created a sourceFile and targetFile variable. IIT now makes this variable available for assignment as shown below. By selecting the $sourceFile variable IIT will use its value and assign it to the "File Name" property of the File Connector. We do the same for the Target File Name as shown below When the flow is compiled... the compiler substitutes the values of these variables into the flow. Where ever a design time variable is used noted by the "$" sign the compiler will do the appropriate substitution. All variables are created and maintained in a single view make management a snap. If the values of these variable change you only have one place to perform the edit. For advanced users you might want to create a variable in property manager which evaluates to a runtime variable. In that case.. the evaluation will occur based on the running environment. In our use case above the file paths most likely won't be known until the application is deployed in a runtime environment. The many uses of runtime variables and how they are set will be covered in another topic.

We plan on covering a whole lot of topics in this forum. Up and coming topics include: Property Manager in iWay Integration Tools API First Development using iWay The Power of the Data Flow object in ETL Generating JavaDoc in a custom iWay Component So suggestions are welcome.

We came across the following requirement while integrating iWay Service Manager with Tibco Foresight. We needed the ability to remember a count after ISM would either be shut down or restarted. Of course we could of written the data to a file or introduce a database to the solution... but that would of introduced a bit of complexity which we just did not want in our solution. Introducing the iWay Object Store Connector. The iWay Object Store provides a means to store variable key/value pairs and have them persisted across server stop/start sessions, unlike special registers which are not retained if the server goes down for some reason. Object Store Component Flow Editor Palette now has an additional component that lets the user interact with the Object Store. Setting Variables in Object Store To set some variable(s) in the Object Store, drag an Object Store component onto the flow canvas, and select action "create/set object store variable(s)". On the Configuration tab specify an Object Name where variables are to be stored. The named object will be automatically added to the Object Store in case it doesn't yet exist in the Object Store. On the Variables tab add some key/value variable pairs. Removing Variables from Object Store To remove some variable(s) from an object in the Object Store, drag an Object Store component onto the flow canvas, and select action "remove object variable(s)". On the Configuration tab specify an Object Name where variables are to be removed. If all variables belonging to the named Object in the Object Store are to be removed, leave the Variables tab empty. If specific variables should be removed, add the specific variable names on the Variables tab. Test Variable Existence To test for existence of some variable(s) in an object in the Object Store, drag an Object Store component onto the flow canvas, and select action "test if object contains variable(s)". On the Configuration tab specify an Object Name where existence of variables is to be checked. When testing for existence of multiple variables, Boolean Operator of AND or OR will be used to join the comparison operations. Specify the variable names to test for existence on the Variables tab. Note that if no Object with the specified name is found in the Object Store, a status of "fail_object_exists" will be returned. Create an edge to handle this case when needed. Retrieving Variables from Object Store Several ways are provided for retrieving variable(s) stored in the Object Store. One way is by using the "retrieve object variable(s)" action of the Object Store component. Drag an Object Store component onto the flow canvas, and select action "retrieve object variable(s)". On the Configuration tab specify an Object Name where variables are to be retrieved from. If all variables belonging to the named Object are to be retrieved, leave the Variables tab empty. If specific variables are to be retrieved, add the specific variable names on the Variables tab. Variables will be returned as a JSON object. If no Object with the specified name is found in the Object Store, a status of "fail_object_exists" will be returned. Create an edge to handle this case when needed. Another way to retrieve variables is by using the iFL function _psreg(<object name>, <variable name>). For example, to retrieve variable 'var1' stored in the object 'object.1', the following iFL expression can be used _psreg('object.1', var1) Object Store Configuration Deployment Templates have several areas that let you customize your Object Store and optionally pre-configure the stored objects. Object Store uses Nitrite DB to persist the variables. Each deployed application has its own object store file located under <iwayhome>/config/<deployment name>/varsStore.db. The file is secured using a username password pair. The username and password can be set only once while creating the database. Nitrite does not store raw password, so retrieval or change of password is not possible. Adding a new username password pair is also not possible for existing database. Default values for username and password are iway/iway. It is recommended to change these settings under Settings/General Settings/Object Store. Object configuration presents a screen where you can specify a TTL (time to live in hours; can be decimal value). When TTL is provided, any variables that haven't been updated within the time slot equal to TTL value will be automatically removed from the object. Variables and their respective initial values may be added in the table below. So that's pretty much it. For those who wondering whether we support managing the object store via ISM's runtime console, the answer is yes. We are scheduled to release this feature in iWay 9.2 and hope it will assist you in your integration projects. Any feedback and/or enhancements which you may need of this feature would be greatly appreciated.