The techniques for integrating Oracle Human Capital Management (HCM) and several other Oracle SaaS solutions can require the use of Oracle Transactional Business Analysis (OTBI). The need for OTBI comes down to the fact that to access to the relevant data an API is not available. OTBI can be configured to generate a report and then transfer the report using FTP. When the report needs to be consumed by ICS then an obvious FTP location is needed. One such option to hold a file for ICS to retrieve is the FTP server provided with the SaaS services (details here).
However, it has come to light that the original intent for this FTP service was for holding bulk data intended to be used for ‘priming’ your HCM instance. As a place for staging data for ICS is in the mid to long term it isn’t recommended. The roadmap for the SaaS product team may result in removing the FTP server.
FTP Data Staging
So the question begs, where should we put the data coming out of OTBI to be consumed elsewhere? Well the next option would be to use Oracle’s Managed File Transfer (MFT). Although historically listed with SOA Cloud Service (SOA CS), it is independently priced and has become a 1st class citizen of the PaaS family more recently. For MFT to be an option it needs to include an FTP server which it does. But MFT also has the abilities for doing a number of orchestration processes, such as calling web services when files are ready. Meaning no need for downstream processes to have to pole for available files.
From Staging to ICS
With the file on the MFT server, we can then initiate consumption of the file more proactively. But one thing to keep in mind is that ICS does have payload limits and if the file is too big ICS can’t process it. This leads us to two options – the first is the ICS File Staging capability that allows ICS to process large files in chunks – we will cover this feature in another post. It should be noted that this feature is not yet generally available at the time of writing (we believe GA will be Summer 2017).
The second option is to exploit MFT’s ability to have small pieces of Java incorporated into its processing flow as a pre-processor or a post-processor. By using the post processor we can create a simple mechanism that allows you to parse the file and convert it into a series of calls to ICS. In doing this we can effectively convert a batch process into a microbatch or event stream. This model can be appealing as if the OTBI is a workaround to a future API then the MFT setup can be dropped once the API becomes available. All being well it will mean that the ICS integration remains potentially unchanged. The added benefit is that ICS integrations can be simpler. In some respects you’ve implemented a GoldenGate style connection on the cheap.
Although we have looked at this from the perspective of HCM and the fact that OTBI generates files. The strategy could be applied to any file sourced data integration.
Really a bulk Integration?
The last consideration is if you’d prefer or need the integrations to work on blocks of data, then it suggests that perhaps your end to end process is batch centric. For batch based processes then maybe a product such as Oracle Data Integrator Cloud Service (ODI CS) is a better tool for your integration process.