Veritone's Library Application enables collections of digital assets to be created for custom engine training. After successful training, each engine can process media and identify specific entities that it was trained to identify. Libraries can be private to an organization and shared between organizations if needed. Before a library is built, engines must be configured to run them.
Set the engine to run libraries
An engine must configured to be library enabled. This can be done in the Veritone Developer app, or via GraphQL.
Set the library mode in Veritone Developer
In Veritone Developer, when creating a new engine, select Yes under Does this engine require a Library? You can also set this option after engine creation by:
- Select Engines in the left hand navigation. The list of engines appears.
- Find your engine in the list. Select it. The engine detail page opens. Select the
. The Engine Configuration page opens where you can make changes to your engine. Note that some settings cannot be changed, noted in red text. - Select the Customize tab and set the Does this engine require a Library? checkbox.
Set the library mode using GraphQL
To configure, set the libraryRequired field to true on the engine definition. Depending on the engine category, the engine is configured to consume one or more specific identifier types, specified by the libraryIdentifierTypes field. This value is defined on the engine category and cannot be changed.
The relationship between engine categories and identifier types is managed by Veritone and cannot be changed. To see the mappings between engine categories and identifier types, use the following query:
query {
engineCategories {
records {
id
name
libraryEntityIdentifierTypes {
records {
id
label
labelPlural
}
}
}
}
}
Engine categories that support libraries will have lists of compatible identifier types. For example:
{
"id": "6faad6b7-0837-45f9-b161-2f6bf31b7a07",
"name": "Facial Detection",
"libraryEntityIdentifierTypes": {
"records": [
{
"id": "face",
"label": "Face",
"labelPlural": "faces"
}
]
}
},
{
"id": "088a31be-9bd6-4628-a6f0-e4004e362ea0",
"name": "Object Detection",
"libraryEntityIdentifierTypes": {
"records": [
{
"id": "image",
"label": "image",
"labelPlural": "images"
}
]
}
}, ...
When selecting an engine category, make sure to choose one that has the correct set of identifier types for your engine. The linkage is important for training. When a library is created or modified, the system automatically attempts to train a new model for all compatible engines.
Engine modes
When enabled for libraries, the engine must support two specific modes, specified as "mode" in the task payload provided to the engine:
library-train: run the engine in training mode.library-run: run the engine against a provided TDO, or recording, ID and a previously trained library.
The engine will run in training mode whenever a compatible library is published after creation or update, such as when new entities are added. The engine will run in run mode whenever anyone uses the engine for its primary purpose, such as facial recognition. Thus, for a typical engine, library-run mode is a common choice.
Build a library
The simplest way to build a library to be consumed by your engine is via the Veritone Library app. Alternatively, you can use the Libraries GraphQL API to generate a library.
A library must be published before it is available as a training set for an engine. For deployed engines, this process is performed automatically and a training task is created for all applicable engines. The payload for each of these tasks is expected to contain two key values:
libraryId: the unique ID (UUID) of the library to train the engine withlibraryEngineModelId: a UUID referencing the model generated by this task.
The libraryEngineModelId is unique not only to this engine, but each training task, i.e., each time a library is modified and published, and new library engine model is generated. While the engine is under development and unable to be deployed and in an active state, the library engine model will need to be generated manually.
To generate manually, execute the following GraphQL query:
mutation {
createLibraryEngineModel(input: {
engineId: 'your-engine-id'
libraryId: 'your-library-id'
}) {
id
}
}
This query will return the libraryEngineModelId that can be used in your sample payload. It can be re-used as necessary.
