New Support for Request Retries with the REST Connector

[Artem]

When using the REST connector in IIT's flows, one issue that comes up is how to handle cases of network connectivity loss. For the 9.2 release we are introducing a new feature that allows the user to configure retry functionality in a flexible and highly reusable manner.

The new retry functionality is exposed under a dedicated tab that may be reached under the Properties view.

 

Under the Retry dropdown select an existing retry generic or click the + button to create a new one. One can use either of the two available types of retry generics: Retry with count, or Retry with comma delimited wait times...

When using Retry with count, you provide the number of times to attempt retry as well as the time in milliseconds to wait between retries. Additionally, you can control what happens if retry action fails. You can let the flow deal with the failure normally, throw a Retry or Fail exception which can be caught with a Catch object and then dealt with accordingly. Finally one can provide a custom exception message to include with the thrown exception.

   

When using Retry with comma delimited wait times option, you provide a list of comma separated wait times in milliseconds. The rest of the configuration options are identical...

   

Once you've configured and applied the Retry generic, the next step is to select the events that should trigger a retry. Select the checkboxes that apply.

Next you can test your flow to see the retry in action. Make sure whatever server your REST object is connecting to is down. Below is a sample log trace showing the retry working, where retry generic was configured to retry 2 times, pausing for 200ms before each retry, and to rely on default handling in the case when retry failed.

URL in is http://localhost:8080
no preemitters, emitting contents of doc, usestream=false encoding=UTF-8
couldn't send HTTP request: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8080 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused: connect
Error emitting nHTTP: XD[FAIL] cause: 3 subcause: 0 message: XDNHTTPEmit: Exception caught: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8080 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused: connect
HTTP Received error response -1
REST retrying 1 of 2 with a wait of 200ms
URL in is http://localhost:8080
no preemitters, emitting contents of doc, usestream=false encoding=UTF-8
couldn't send HTTP request: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8080 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused: connect
Error emitting nHTTP: XD[FAIL] cause: 3 subcause: 0 message: XDNHTTPEmit: Exception caught: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8080 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused: connect
HTTP Received error response -1
REST retrying 2 of 2 with a wait of 200ms
URL in is http://localhost:8080
no preemitters, emitting contents of doc, usestream=false encoding=UTF-8
couldn't send HTTP request: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8080 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused: connect
Error emitting nHTTP: XD[FAIL] cause: 3 subcause: 0 message: XDNHTTPEmit: Exception caught: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8080 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused: connect
HTTP Received error response -1
node REST returned edges: fail_connect,
REST has no wired edges to follow; node REST returned edges: fail_connect,
Entering error processing from node  agent_REST with edge $error
wakeUp called for pflow: W.ExecProcess.1~rest-flow
Error processing failed, flow line ending at <agent_REST:REST/type(Agent)>
Pflow thread stopping at REST
Ending pflow, number of subthreads still running=0
back from pflow, [REST] FAIL: Process flow failed, exception message: REST has no wired edges to follow; node REST returned edges: fail_connect,

You can see connection failing on the first execution of REST object. The flow then makes 2 retry attempts, waiting 200ms between each retry. Finally, the flow exits with an error.