Implementing Oracle Integration Cloud Service

Understanding Oracle's Integration Cloud Service and how to get the most out of your implementation

Interfacing with HCM (or other file source) via FTP – ICS Definitive Tip #8

HCMThe techniques for integrating Oracle Human Capital Management (HCM) and several other Oracle SaaS solutions can require the use of Oracle Transactional Business Analysis (OTBI). The need for OTBI comes down to the fact that to  access to the relevant data an API is not available. OTBI can be configured to generate a report and then transfer the report using FTP. When the report needs to be consumed by ICS then an obvious FTP location is needed. One such option to hold a file for ICS to retrieve is the FTP server provided with the SaaS services (details here).

However,  it has come to light that the original intent for this FTP service was for holding bulk data intended to be used for ‘priming’ your HCM instance. As a place for staging data for ICS is in the mid to long term it isn’t recommended. The roadmap for the SaaS product team may result in removing the FTP server.

FTP Data Staging

So the question begs, where should we put the data coming out of OTBI to be consumed elsewhere? Well the next option would be to use Oracle’s Managed File Transfer (MFT). Although historically listed with SOA Cloud Service (SOA CS), it is independently priced and has  become a 1st class citizen of the PaaS family more recently. For MFT to be an option it needs to include an FTP server which it does. But MFT also has the abilities for doing a number of orchestration processes, such as calling web services when files are ready. Meaning no need for downstream processes to have to pole for available files.

From Staging to ICS

With the file on the MFT server, we can then initiate consumption of the file more proactively. But one thing to keep in mind is that ICS does have payload limits and if the file is too big ICS can’t process it.  This leads us to two options – the first is the ICS File Staging capability that allows ICS to process large files in chunks – we will cover this feature in another post. It should be noted that this feature is not yet generally available at the time of writing (we believe GA will be Summer 2017).

The second option is to exploit MFT’s ability to have small pieces of Java incorporated into its processing flow as a pre-processor or a post-processor.  By using the post processor we can create a simple mechanism that allows you to parse the file and convert it into a series of calls to ICS. In doing this we can effectively convert a batch process into a microbatch or event stream. This model can be appealing as if the OTBI is a workaround to a future API then the MFT setup can be dropped once the API becomes available. All being well it will mean that the ICS integration remains potentially unchanged. The added benefit is that ICS integrations can be simpler. In some respects you’ve implemented a GoldenGate style connection on the cheap.

Although we have looked at this from the perspective of HCM and the fact that OTBI generates files. The strategy could be applied to any file sourced data integration.

Really a bulk Integration?

The last consideration is if you’d prefer or need the integrations to work on blocks of data, then it suggests that perhaps your end to end process is batch centric. For batch based processes then maybe a product such as Oracle Data Integrator Cloud Service (ODI CS) is a better tool for your integration process.

Previous

Custom functions to extend ICS transformations – ICS Definitive Guide #4

Next

ICS Pricing – ICS Definitive Tip #9

2 Comments

  1. Sander Rensen

    The on-premise Oracle File Adapter has a great feature to move files without the need of loading the payload or chunk files. This feature can also handle large files (used up to 5GB). Is this feature going to be made available in the Cloud?

    • Cloud version of the file adaptor is the FTP adaptor. The problem isnt the adaptor, its the access to file storage location. If you need to use MFT then using the postprocessor gets very easy

Leave a Reply

Powered by WordPress & Theme by Anders Norén