Using the REST Client to Download a Text File

The Centerprise REST Client can be used to download other data besides XML and JSON. A common usage would be to download the contents of a text file for further processing. In these cases, there would be no output layout since the file is not being parsed. Instead, data would come from the “Content” field of the “ResponseInfo” object. This field contains the contents of the file as a text string.

This text string can then be processed as any other text value in Centerprise. For example, it can be directly written to a text file using the “WriteToTextFile” function. This would result in a copy of the file from the server and would be mostly suited as a step in a workflow. Another possibility would be to parse this text string using one of the text parsers available in the product. The benefit of this approach is that it can process files directly from the server and is closer in practice to how a local text file is processed.

Example 1 (dump contents to file)

In the example above, RestClient1 has been set up to Get a file at http://myfileserver:8080/Files/textfile1.txt@dummy. The headers collection has a value of Accept. Note the @dummy part of the url. It is only here to force the parent “Input” node to appear. The UrlParameters also appears, but it is being ignored in this case since we are not using parameters. We are, however, using the “Headers” section and we have mapped the constant / into the value for Accept. This is crucial for some servers to not get an error back.

Since there is no layout, the only output is the “ResponseInfo” section of the tree. Here we are mapping the “Content” field from the tree. This field contains the entirety of file. It is mapped into the “text” field of the “WriteTextToFile” function. The other parameter is mapped to a constant file path.

When this is run, a single request will be made and the contents of the REST response will be written to the file specified in the function.

Example 2 (parsing file contents)

In this example, RestClient1 is set up exactly as before. The only difference between this and Example 1 is that here we are processing the file directly in the dataflow rather than just downloading for processing in another flow. We do this by using the Delimited Parser. We are using the Delimited Parser here because the contents of the file are comma delimited. If the file were of another type, we would use the appropriate parser (fixed length, xml, json).

In this example, our file consists of three columns: Name, Rank, and SSN. Therefore, we add these fields to our Delimited Parser to match. The Passthru transformation is just being used as a stand-in for some kind of target.

Caveats

Headers

By default, Centerprise will add an “Accept” header. This value will be “application/json, application/xml, text/json, text/x-json, text/javascript, text/xml”. Because of this, some servers will reject the request because the response does not match any of these values. In order to get around this, the “Accept” header must be overwritten. To do this, add a row in the header grid and type the value “Accept” in it. Map a constant value of / into this field in the tree.

Input Object

As of 6.4.2.47, there is a bug in this component that will cause the “Headers” section of the input object in the tree to be hidden if there are no other inputs. Since there is no body for the Get request, there must be url parameters present. If your URI does not contain any parameters, just place a dummy @value at the end of the URL resource path. This will force the Headers section to show and unmapped @value parameter will harmlessly be ignored.

File Size

The entire contents of the file are stored in memory when using this method. For this reason, it is possible to overflow the system’s memory and crash Centerprise. This limit is dependent on the system it is running on.