Time flies, with a 1/4 of a year behind us already including the first quarterly release. Last month we’ve seen a lot of attention around Oracle Hospitality Integration Platform (OHIP) which brings together different Hospitality cloud services as well as using technologies such as Integration Cloud to make it easy to interface 3rd party solutions.
Tag: Oracle Page 4 of 7
OIC has for some time now provided an FTP adaptor and more recently included a full FTP server capability. But both have limits on the file size and capacity. The file constraints (1GB for a file and 500GB for the FTP server) shouldn’t be an issue for day to day activities. But OIC is often used to support SaaS Financials and other cloud solutions which do have monthly process cycles which can generate significant data volumes, for example, payroll data. The question is how to handle such data with such constraints?
Data Integration?
There is a school of thought that points to the possibility when handling such large data volumes we should consider using Data Integration rather than a more event-centric integration tool. Personally, I think there is a lot of validity in the argument, and anyone dealing with such bulky data activities should review and question if it is a better answer.
That said, there are cases where it does stand-up. For example:
- If an organization is transitioning to a more event-driven or at least micro-batch model, you have to start the transition somewhere, but trying to line up changes everywhere can be problematic, so we have to start somewhere. Building an integration process so you have an event model developed, but in the interim, you need to take that bulk mechanism and convert it to a small stream of events.
- You may be working with a bulk data extract and only need a small subset of the data provided, it won’t help if the data is also represented using a verbose notation such as XML.
Other Approaches
How to overcome the constraint? Oracle databases aren’t so constrained, and SQLLoader can provide an easy means to ingest the data into a staging table. The benefit of this is:
- if you’re only needing a subset of the data you can pull just those columns from the table.
- the bulk of the use of XML to be self-describing can be shed through using the DB schema as being more prescriptive.
- SQL scripts can handle the checksum records removing that data and overhead from the integration process, leaving you to concentrate more on the business process.
If the data is still substantial once in the database there are a number of strategies to consume the data in more manageable chunks, such as
- Running SQL script on the database that takes each row and calls the OIC as a restful API point. This approach is potentially very interesting as it may then mean if you’re moving towards an event process in the future the API endpoint represents the future state and the database stored procedure is mimicking the future client behaviour.
- Use polling strategies and result set limits to control how much data is processed in a single execution of the integration. This approach does mean the integration needs to tag which records have been processed to avoid re-reading them.
Useful Links
With this being the last post of a very strange year, we wish all our readers a very Happy New Year and hope that 2021 brings a transition to normality.
| Article / Link | Author | Subject Matter | Connecting |
|---|---|---|---|
| Microprocess Consequences | Jan Kettenis | OIC | |
| Oracle Developer Meetups – Gone Virtual | Phil Wilkins | OIC & PCS | Various |
| Invoking Oracle Functions from OIC | Niall Commiskey | OIC | Functions |
| OIC Backups | John Graves | OIC | |
| Data Stitch Append for Complex Content Elements | Jorge Herreria | OIC | |
| OIC –> Insight –> embeddable Dashboards | Niall Commiskey | OIC, Insights | |
| Using Oracle Integration Cloud to integrate SAP & Oracle SaaS | Martijn de Grunt | OIC | SAP |
| Oracle JET or Oracle VBCS For Your Next Web App | Andrej Baranovskij | VBCS | |
| OIC –> EPM adapter setup | Niall Commiskey | OIC | EPM |
| Identity Propagation – VBCS > IC > Fusion Apps | Greg Mally | OIC | VBCS, Fusion Apps |
| Fault Actions Behavior in OIC | Greg Mally | OIC | |
| Archiving and Purging Process Automation Data in Oracle Integration | Arvind Venugopal | PCS | |
| OIC Twilio Adapter – send SMS / WhatsApp from Oracle Integration | Niall Commiskey | OIC | Twilio |
This month’s articles …
| Article / Link | Author | Subject Matter | Connecting |
|---|---|---|---|
| OIC REST API for Lookups | Niall Commiskey | OIC | |
| OIC 4 Netsuite – Polling for New / Updated Customers | Niall Commiskey | OIC | Betsuite |
| Oracle Integration – Adapter Enhancements to Non Oracle Applications | Arvind Venugopal | OIC | Non Oracle |
| Security Improvements for Database & FTP Adapters | Michael Meiner | OIC | DB & FTP |
| November 2020 Update | Antony Reynolds | OIC | |
| Oracle Integration November 2020 update for Oracle Applications Adapters | Prakash Masand | OIC | Oracle Apps |
| How to use the OCI Object Storage from the Oracle Integration Cloud | Daniel Martins Teixeira | OIC | OCI Object Store |
| Choosing Your Update Window | Antony Reynolds | OIC | |
| Announcing The Visual Builder Cookbook | Shay Shmeltzer | VBCS | |
| Connecting to API Gateway from Visual Builder with Authentication | Aparna Gaonkar | VBCS | |
| Global Variable and Data Stitch in OIC | Amit Kumar Suman | OIC | |
| Configure OTM connection in OIC | Amit Kumar Suman | OIC | OTM |
| Send attachment in OIC notification | Amit Kumar Suman | OIC |
Despite it being traditionally holiday season, there have been lots of articles written about OIC, including the prolific Niall Commiskey covering a lot of new OIC features …
Its been a busy month when it comes to blogging for the folks at Oracle, as July saw a new quarterly release with new usability improvements and connectors.