Understanding Integration Cloud and how to get the most out of your implementation

Tag: DB

July 2023 – New Articles

Article / LinkAuthorSubject MatterConnecting
How to become a certified Oracle Cloud expert? Advance your Career – free training and free certification!Jurgen KressOICEverything
Change Oracle Integration request parameters and Authentication using API GatewayOmid IzadkhastiOICAPI
OIC – How to use DB sequence (insert, access)SanddyOICDB
Exposing rest service supporting a file as attachment and JSON request SanddyOIC 
UCM using Oracle ERP Cloud adapter (download, upload)SanddyOICUCM
Transfer file to UCM using tech adaptors (REST upload, ErpIntegrationService SOAP Upload, ErpIntegrationService SOAP Download)SanddyOICUCM
ERP Integration Patterns – Bulk ImportDaniel TeixeiraOICERP
OIC3 OAuth with OCI DomainsNiall CommiskeyOIC3OCI
counting OIC messages for subsidiaries within an organizationNiall CommiskeyOIC 
OIC Projects RBAC – a simple exampleNiall CommiskeyOIC 
OIC & SFDC Streaming API and Platform EventsNiall CommiskeyOICSalesForce
Visual Builder invoking OCI Process AutomationNiall CommiskeyOICVB, OPA
What’s New in Oracle Integration 23.08Niall CommiskeyOICVarious
Oracle Cloud and the power of community in driving digital evolutionJurgen KressOIC 
Oracle Cloud Applications Integration (Part 1, Part 2, Part 3)Su Patel, Mike IsermanOICSaaS
Importing of Data into Oracle ERP Cloud using REST Services through OICRevanth RampellyOICERP

Training provision for OIC appears to be growing:

November 21 – New OIC Articles

Another month and another raft of great content. Lot of activity springing up around the new Oracle Hospitality Integration Platform (OHIP) with its new OIC Adapter.

Article / LinkAuthorSubject MatterConnecting
Clone an Existing Integration Flow in OIC Basics of OICOIC 
OIC: Synchronous versus Fire-and-Forget Process to Process InvocationJan KettenisOIC 
Oracle Integration Cloud – Gmail AdapterDaniel TeixeiraOICGMail
Between Oracle Integration Cloud (OIC) And Oracle Cloud Infrastructure (OCI)Atul KumarOIC 
How to call the OCI AI Language Service from the Oracle Integration CloudDaniel TeixeiraOICOCI AI Language Service
Creating Custom Messages/Notifications in Visual BuilderShay ShmeltzerVBS 
FlexDeploy for Oracle Integration CloudJürgen KressOIC 
Connecting Opera Cloud to the World – PaaS Partner Community Webcast November 30th 2021Jürgen KressOICOPERA (Oracle Hospitality)
OIC November Release New Features – Oracle Hospitality AdapterNiall CommiskeyOICOPERA (Oracle Hospitality)
OIC November 21 Release New FeaturesNiall CommiskeyOIC 
DB adapter – overcoming 10 MB limitNiall CommiskeyOICDB
Leveraging OCI AI Language Service from Oracle Integration (Part 1) (Part 2)Niall CommiskeyOICAI Language Service
Fetch & update Integration lookup using Visual BuilderAnkur JainVBS & OIC 
OHIP and Oracle Integration Cloud: Simplifying hospitality integrations even moreLuis WeirOICOHIP (Hospitality Integration)
November 2021 New Third Party Application AdaptersMadhav PoosarlaOICSugar CRM, eBay
Delivering Real Customer Value with Agile Innovation: Heathrow rapidly moves to Oracle Cloud and connects with hybrid Integration – Webcast November 16th 2021Jürgen KressOIC 
November 2021 Oracle Integration UpdateAntony ReynoldsOIC 
Answers are never far away with Oracle AssistantKevin HopkeOIC 
Exporting Data from Oracle HCM Cloud using OICHemanth LavesattiOIC 
CI/CD for Oracle Visual Builder ApplicationsVijayaraj GovindharajVBSFlexDeploy
Decouple complex flows into simplified multiple resource REST flowMadhav PoosarlaOIC 

Beating OIC’s FTP File Limits

OIC has for some time now provided an FTP adaptor and more recently included a full FTP server capability. But both have limits on the file size and capacity. The file constraints (1GB for a file and 500GB for the FTP server) shouldn’t be an issue for day to day activities. But OIC is often used to support SaaS Financials and other cloud solutions which do have monthly process cycles which can generate significant data volumes, for example, payroll data. The question is how to handle such data with such constraints?

Data Integration?

There is a school of thought that points to the possibility when handling such large data volumes we should consider using Data Integration rather than a more event-centric integration tool. Personally, I think there is a lot of validity in the argument, and anyone dealing with such bulky data activities should review and question if it is a better answer.

That said, there are cases where it does stand-up. For example:

  • If an organization is transitioning to a more event-driven or at least micro-batch model, you have to start the transition somewhere, but trying to line up changes everywhere can be problematic, so we have to start somewhere. Building an integration process so you have an event model developed, but in the interim, you need to take that bulk mechanism and convert it to a small stream of events.
  • You may be working with a bulk data extract and only need a small subset of the data provided, it won’t help if the data is also represented using a verbose notation such as XML.

Other Approaches

How to overcome the constraint? Oracle databases aren’t so constrained, and SQLLoader can provide an easy means to ingest the data into a staging table. The benefit of this is:

  • if you’re only needing a subset of the data you can pull just those columns from the table.
  • the bulk of the use of XML to be self-describing can be shed through using the DB schema as being more prescriptive.
  • SQL scripts can handle the checksum records removing that data and overhead from the integration process, leaving you to concentrate more on the business process.

If the data is still substantial once in the database there are a number of strategies to consume the data in more manageable chunks, such as

  • Running SQL script on the database that takes each row and calls the OIC as a restful API point. This approach is potentially very interesting as it may then mean if you’re moving towards an event process in the future the API endpoint represents the future state and the database stored procedure is mimicking the future client behaviour.
  • Use polling strategies and result set limits to control how much data is processed in a single execution of the integration. This approach does mean the integration needs to tag which records have been processed to avoid re-reading them.

Useful Links

Powered by WordPress & Theme by Anders Norén