Let us know how we can help
Sales & Pricing
Speak to a rep about your business needs
Help & Support
See our product support options
General inquiries and locations
Contact us
Control-M + OCI Data Science
Your OCI Data Science workflow starts before a job run ever executes — and it can fail when something upstream breaks. An OCI Object Storage training dataset arrives late. An OCI Data Integration task misses its processing window. Control-M orchestrates the full chain: Object Storage data arrival, OCI Data Integration transformation, OCI Data Science execution, and downstream delivery.
common workflow issues
Does this sound like your week?
These aren’t edge cases. They’re the normal operating conditions for teams running OCI Data Science pipelines across multiple tools. Here’s how Control-M handles each one.
DATA READINESS
Your Object Storage dataset is late. The training job is scheduled anyway.
Control-M makes the OCI Data Science job dependent on successful upstream processing instead of an isolated schedule. The job starts only after its required data workflow completes, preventing an incomplete dataset from propagating into model training.
PIPELINE FAILURE
Your pipeline run failed overnight. Downstream processing is still waiting.
Control-M monitors the OCI Data Science run, detects its completion status, and holds dependent work when execution fails. Configurable status polling and failure tolerance control how execution is monitored, preventing a failed ML step from cascading downstream.
MODEL DEPLOYMENT
Training finished at 2:00 AM. Model deployment still needs a handoff.
Control-M connects successful upstream completion directly to the next OCI Data Science action, including creating a model deployment endpoint. Training and deployment become dependent stages in one scheduled workflow, eliminating the manual trigger between them.
SLA RISK
The job is still running. Your morning scoring deadline is approaching
Control-M tracks the OCI Data Science job within its complete production workflow and applies SLA management across dependent steps. Teams can see when delays threaten the required delivery time and intervene before downstream consumers miss their window.
CROSS-TOOL DEPENDENCIES
OCI Data Integration finished. Your Data Science job has not started.
Control-M coordinates OCI Data Integration and OCI Data Science in the same workflow, using completion dependencies rather than disconnected schedules. The Data Science job starts after the required ETL task completes, removing timing gaps between services.
INTEGRATION FACTS
Control‑M + OCI Data Science
|
workload.types |
job runs · pipeline runs · model deployment endpoints · notebook sessions |
|
trigger.type |
time schedule · upstream job completion · file arrival · Control-M dependency · API-driven execution |
|
cross_tool.deps |
OCI Data Integration task · OCI Data Flow run · Object Storage delivery · Airflow DAG · file transfer · REST API call |
|
cloud.platforms |
Oracle Cloud Infrastructure · OCI Data Science · OCI Data Integration · OCI Data Flow · OCI Object Storage |
|
error_handling |
status polling · failure tolerance · downstream cascade prevention · automated job hold · SLA alerting |
|
throughput |
batch ML workloads · parallel job runs · repeatable pipeline runs · distributed data processing |
|
observability |
job execution status · pipeline run status · results and output · SLA tracking · dependency visibility · execution history |
end-to-end orchestration
One production workflow. Every tool in the stack.
Control-M orchestrates workflows across OCI Data Science, OCI Data Integration, OCI Data Flow, Object Storage, Airflow, file transfers, and cloud services in a single job flow — with dependency tracking, SLA visibility, and automated recovery across all of them.
- Cross-tool dependency: OCI Data Integration → OCI Data Science job → model deployment → analytics handoff
- Data-aware triggers: file arrival, API event, upstream task completion, pipeline completion
|
OCI Data Science |
start job runs · start pipeline runs · create model deployment endpoints · create and delete notebook sessions |
|
OCI Data Integration |
run ETL tasks · coordinate workspace execution · pass task parameters |
|
OCI Data Flow |
orchestrate Spark processing · coordinate processing dependencies · connect downstream execution |
|
OCI Object Storage |
coordinate data arrival · gate downstream execution · connect data-driven workflows |
|
Apache Airflow |
coordinate DAG execution · manage upstream dependencies · connect downstream handoffs |
|
Managed File Transfer |
secure file movement · delivery dependencies · workflow coordination |
|
REST APIs |
API-driven execution · cross-service handoffs · workflow integration |
airflow coexistance
Control-M doesn’t replace your Airflow DAGs. It runs the layer above them.
The objection is common: “We’re already on Airflow.” The issue isn’t what Airflow does – it’s what happens before and after Airflow runs. That’s where pipelines actually fail.
Airflow manages its DAG. Control-M manages everything surrounding it.
AIRFLOW HANDLES
DAG-level orchestration inside the data pipeline
- DAG-level task orchestration within data pipelines
- Python operators, sensors, and task dependencies
- Execution graph for jobs that run inside your pipeline
- Manages retries within a single DAG context
control-m adds
The coordination layer around your DAGs
- Coordination layer around DAGs — triggers Airflow based on upstream conditions: file arrivals, API events, other tool completions
- Tracks each DAG’s SLA contribution across the full end-to-end workflow, not just its own routine
- Manages failure recovery when upstream dependencies fail before Airflow even starts
- Existing DAGs don’t need to be rewritten or migrated
MONITOR ML WORKFLOWS
Monitor OCI Data Science execution across your pipeline
OCI Data Science provides visibility into its own jobs and pipeline runs, but production ML depends on work outside that boundary. Control-M brings those execution stages into one operational view so teams can monitor the complete pipeline:
-
Job and pipeline status
-
Upstream and downstream dependencies
-
Job results and output
-
End-to-end execution visibility
-
Cross-platform workflow status
SLA ASSURANCE
Keep OCI Data Science pipelines on schedule
A successful OCI Data Science run can still arrive too late for the process depending on it. Control-M manages the workload against end-to-end delivery requirements, exposing dependency delays and SLA risk before they disrupt downstream operations:
-
End-to-end SLA tracking
-
Dependency-aware execution control
-
Predictive delay visibility
-
Automated exception handling
-
Centralized workflow monitoring
Documentation
Bring order to complex workflows
Learn how Control-M helps teams orchestrate complex processes with greater visibility, coordination, and control.
