The DAG (Directed Acyclic Graph) builder in aiWARE is a user-friendly interface that simplifies the process of creating intricate task sequences, increasing efficiency by streamlining and optimizing your organization's operations, which reduces errors. Drag and drop tasks, set dependencies, and define execution schedules. This makes it easier to conceptualize, plan, and manage scheduled jobs.
The DAG builder is accessible in the Data Center utility through the schedule job wizard, in the processing step.
DAGs are created when you create a scheduled job in Data Center, which is done by stepping through the Scheduled Job wizard and creating a DAG as part of the process.
Create a Scheduled Job
- Access the Data Center utility by logging into aiWARE. Click V and then the Data Center icon.
- Click New > Schedule Job. The Select Import Source panel opens.
- Choose an import source from under the Pull list. Sources are dependent on your organization. You can choose different media chunk lengths by clicking Advanced Settings.
Once an import source is chosen, the Create Schedule panel opens.
The scheduled job wizard guides you through 6 steps.
1 - Select a Source
- Choose a source for the import source. To create a new source, click the drop-down list and then + Create New Source at the bottom of the list.
2 - Basic Info
- Give the job a name. The rest is optional.
- Select the Program Format and check the Is National box to help tag your job for later search.
3 - Schedule
- Choose the time and date for the job to run.
[Warning] If you skip this step, the job will run at a default time and you will be unable to find it until it runs.
4 - Processing
In this step, you can choose an available template, which is useful for jobs that are similar to previous ones.
Otherwise, click the Create a New Template button. The Dag Builder panel opens.
Build a DAG
While working in the Dag Builder panel, you are still in the 4 - Processing section of the Scheduled Job wizard.
- Give the DAG a name. Click Continue.
- In the DAG Builder window, you can zoom in and out. Click the plus sign on the source box you chose. The Choose a Node Type panel opens.
- Choose a node type.
There are three types of nodes you can add to your DAG. In most cases, you'll choose a cognition engine.
- Starter nodes: These nodes represent the first step of your DAG and are typically represented by a push or pull adapter responsible for collecting the data that will be processed from within your DAG.
- Cognition nodes: Cognition nodes represent AI models. These nodes process data and output AI model results to the TDO. Once you choose a cognition category, a list of engines in that category appears.
- Utility nodes: Utility nodes perform a discrete task against the data that is not cognitive or ingestion based. These are typically automatically inserted into the DAG as you go (see Dependencies, below). Examples of utility nodes include a thumbnail generation node or a chunk creator node, responsible for chunking up the data into processable bits.
- Make sure to add an output writer node, which is a Utility node, after any cognition node where you'd like the data saved. Results from cognition nodes will only be written back to the TDO if there's an output writer.
For example, for a video with audio in English where you want to save its Spanish translation, nodes would be:
ingest > chunk > transcription > translation > output writer
If you want to save both the English transcription and the Spanish translation, nodes would be:
ingest > chunk > transcription > two nodes (output writer and translation > output writer)
- Click the + Use This Engine button.
- Click the Add Node button.
- Review your information and click Save.
DAG mini map
A mini map is located at the bottom left corner of the DAG builder window. This color-coded map helps quickly navigate to the desired location on your DAG. This is especially helpful if your DAG is larger in size. Each node in your DAG is color coded based on its unique type:
- Green/Red: Starter node (push or pull)
- Yellow: Cognition nodes (AI models)
- Grey: Utility nodes (output writer, thumbnail generator, or SI2 Chunk creator)
Use your cursor to hover over the mini map and drag to the desired location in your DAG to quickly jump to that area.
Dependencies
aiWARE automatically inserts required nodes into your DAG as your create it. This will usually include utility nodes, which perform tasks such as chunking up your data into consumable processing chunks, or converting your video file into image frames so it can be processed by vision based cognition models such as facial recognition or logo detection.
These nodes are in a collapsed state when they are added and are represented in grey. To modify utility nodes, hover over the node and select Click to Show on the popup, which exposes the entire node and its applicable settings options.
5 - Content Templates
Create a new template to save for later job use.
6 - Location
- The location of the DAG is where you store it. Set different folders for jobs by clicking Change to the right of the Home folder.
- Once set, click the Create Scheduled Job button.
Modify a DAG
Locate the scheduled job in the Sources section.
Click the three dots icon next to a job and click Edit Source. The Schedule Job Edit Wizard will slide open.
