Scheduled ingestion

🛠️ Scheduled Ingestion in Veritone aiWARE

1. Create a Media Source

• In the Veritone Data Center UI, click New → Schedule Job to open the Select Import Source panel.
  
  

• Choose an adapter (e.g., FTP, S3, HTTP pull) and configure connection details, file patterns, and destination options.
  
  

• Optionally, register a structured data schema for downstream processing.
  
  

📘 Ingest Data – Create an Ingestion Job  

2. Create a Scheduled Job

• Configure a job to run at set intervals (e.g., daily, hourly, or continuous ingestion).
  
  

• Use the DAG builder to define the processing flow: ingestion → AI engines → output.
  
  

• Select your Media Source, processing engines (e.g., transcription, face detection), and frequency.
  
  

• You can update DAG logic later as needed.
  
  

Create and manage jobs

3. (Optional) Launch the Job Immediately

• Use the launchScheduledJob() API to trigger the job right away for testing or urgent runs.
  
  

• This does not affect the recurring schedule.
  
  

API reference available in linked docs above.

4. Retrieve Results

a. After Processing (AI-Enriched Outputs)

• Query TDOs and engine results using the REST API.
  
  

• Use filters like scheduledJobId to isolate results from a specific job.
  
  

📘 Retrieve Job Results
📘 AI Engine Output Formats

b. During Processing (Edge WebSockets)

• Subscribe to Veritone Edge WebSockets to receive real-time status updates (e.g., "Ingest Started", "Engine Complete", etc.).
  
  

✅ Summary