Adapters are engines that solve the issue of getting data into the aiWARE platform by ingesting content from external sources.
Overview of adapters
Adapters establish a connection with a source and then acquire, parse, and output data in a format that can be processed by AI Processing. The first engine in any job's task list is usually an adapter. Adapters facilitate the easy import of data from cloud storage, local storage, web video, FTP sources, and RSS feeds, including podcasts. Data can be audio, video files (streams), or text and images (chunk). This data can also be structured metadata from a variety of sources. Once ingested, the source data is stored securely in a repository that other engines will then access and store in AI Data.
Adapter sequence
The adapter workflow is structured as a sequence of actions that are executed one after the other to process a task from start to finish. The sequence consists of two basic concepts:
- Receiving a task
The adapter's workflow is launched when a new job is created that includes an ingestion task for the adapter to perform. Based on the information described in the task payload, the adapter will acquire specific data from an external source and transcode it into a format that can be used for processing. Some adapters can further break data into a set of time-segmented chunks, and output the chunks in a format that can be used for processing.
- Outputting the results
The adapter acquires the raw data from the input source and parses the received content into a set of time-based segments, except for the SI2 Audio/Video Chunk Creator engine, which outputs a stream of data transcoded to NUT format, typically so that subsequent processing can be done. If you are building your own adapter, it will need to follow the above steps.
Adapter categories
Adapters are defined by the way data is served up from the source. The protocol follows a client-server approach for communication between the data source and the adapter. Data is pulled from a server by a client-initiated request, and a separate request is made each time data is to be retrieved.
You can choose one of the following three adapters as part of creating an IO adapter.
Generator adapters
A generator adapter creates content from nowhere. For example, an engine that generates lorem ipsum text or generates a silent or white-noise audio file.
Pull adapters
A pull adapter asks the aiWARE system to retrieve content and pull it into the system. Pull adapters are the most common type of adapter implementation. Pull adapters are services that work "by reference" (being instructed to fetch media based on some pointer, such as an S3 bucket or Dropbox folder).
Push adapters
A push adapter lets other systems send (push) content to aiWARE. The data source serves as the client and drives the transfer operation by notifying the adapter (server) whenever new data is available. After the initial request, data continues to be pushed from the source to the adapter without any subsequent requests.
Adapter engine references
We offer four out-of-the-box adapters for popular sources such YouTube and podcasts. For information about these adapters, see Adapter references.
Use an adapter
To use an adapter in aiWARE, you first need to create it as a resource. See Create an IO adapter.
