Create an IO adapter

IO adapters are engines that solve the issue of getting data into the aiWARE platform by ingesting content from external sources. Creating an IO adapter follows the same steps as creating an engine. This article repeats the steps with specifics to selecting adapters.

Create a new adapter

1. Access the Resource Center utility by logging into your aiWARE account. Click the utility icon . Select the Resource Center folder icon at the bottom . The Resource Center panel slides out.
2. In Resource Center, click the + Create New button. Choose Engine.

You will be guided through a wizard. 

Step 1: General

In the general step, set:

• Engine name and description
• GPU size
• Distribution flags

  To learn more about distribution flags, see distribution types for packages.

The distribution options are:

Step 2: Functionality

In the functionality step, set the adapter type. Under Engine class and category, click the header under Engine Class (Audio is selected by default) to bring up a list of all engines and adapters.
Adapters are:

• Generator
• Pull
• Push

• Engine mode. Use segment or stream engine modes instead of batch, which is deprecated.
• Tags. Tags can be custom set for ease of use when gathering engines and other resources into a package.
• Engine deployment model. Set deployment models depending on your use case and the way your engine will process the input data.

  Deployment model	Use case
  External access	Engines that require internet access for tasks such as license checks to an external server or downloading reference data, but don't send user data outside the container.
  Network isolated	Engines that perform their processing entirely within thier container, running only within Veritone's infrastructure. There is no internet access and no external calls are made. This is the most secure deployment model and recommended for government, legal and compliance, and media and entertainment uses, which often run on prem.
  External processing	The engine does some or all of its processing outside its container and therefore requires internet access. For example, user data may need to be sent to external services (such as cloud API) for processing.
  Human review	The engine requires human intervention to complete its processing, and therefore requires network access.

• Supported input types. Click the + Input Types button to check all media types your engine will support.
• Custom fields. See Custom fields for more information.

Step 3: Use cases

Add a summary of how and where the adapter engine will be used. This is an optional step.

Step 4: Integration

In the integration step, set:

• Input schemas
• Output schemas
• Engine processing flow
• Engine template support input types. See Creating your own single engine job template for more information.

Step 5: Confirmation

In the confirmation step, you can upload your adapter engine file. You can skip this step and upload a file later by editing an engine.  

Upload an engine file

1. Click the + Docker Image button.

Adapter engines are deployed in Docker containers, which are like lightweight virtual machines. The platform automatically creates instances of your engine to meet demand. All engines in aiWARE are packaged as executable Docker containers, which are exposed via webhook through Veritone's Engine Developer Toolkit. An engine webhook can pass data as a JSON object, which makes engines widely compatible with all the programming languages. The most common languages used for implementing engines are Go, Node.js, and Python.

You can deploy an adapter by adding the Docker image or a file of the Docker image to aiWARE. The steps below assume you have already written an adapter or obtained a third-party adapter engine.

Docker upload steps

To onboard a Docker image engine, you need the following:

• A Docker image engine in your computer or VM.
• Access to a Docker image repository. This documentation uses Docker Hub.

1. In the terminal, log in to the Docker image repository. Type this command:

   docker login

2. To tag and push the Docker image, type the following:

   docker tag engineName DockerHubUsername/engineName
   docker push DockerHubUsername/engineName

   where

   • engineName is the engine build in your local computer or VM.
   • DockerHubUsername is a repository in the default registry in Docker Hub.

File upload steps

To onboard an adapter engine with a file, you need the following:

• A Docker image engine in your computer or VM.
• Access to a storage repository that can be accessed by aiWARE. This documentation uses Amazon S3.

1. In the terminal, in a temporary directory, type the following:

   docker save engineName | gzip > engineName.tar.gz

2. In your browser, go to your account in Amazon S3.
3. Upload the engine file.
4. When the upload is complete, click Close. The Bucket page appears.
5. In the Objects tab, in the Files and Folders section, click the engine file checkbox, and then click Actions > Make public.
6. Select the engine file. The properties page appears.
7. In the Objects overview section, copy the Object URL.
8. See Editing an Engine below to access the engine build and enter the link.

Edit an engine

Once an adapter engine is created, you can edit some of its settings and upload your file.

• Click Resources > IO Adapters 

• Click your adapter in the list. Click the options icon to edit, upload a build, or delete the engine.

• Click the Builds tab of the engine detail to see the build history of the engine. You can pause builds of the engine here.
• If uploading a build, the Create New Build panel opens.

• More about adapters
• Adapter build concepts
• Veritone out-of-the-box adapters adapters

• Create a new adapter
• Upload an engine file
• Edit an engine