If you're following the steps to build a simple text-processing engine, this is step 6. To see all the steps, see the user guide process page.
In this step, test the engine in a live job in aiWARE. One of the easiest ways to do this is to use the GraphQL Playground web IDE.
Steps
1. Check the engine's availability.
Before running a job, check to see that the engine exists and is visible via the API.
a. Log in to Veritone.
b. Navigate to the GraphQL Sandbox IDE.
c. In the GraphQL Sandbox, create and execute a query like the following, using the engine's ID:
Example
query myEngine{
engine(id:"386e022f-edd1-46ec-b1cc-84d203a37250"){
builds {
records {
id
status
}
}
name
state
ownerOrganizationId
isPublic
manifest
mode
outputFormats
supportedInputFormats
supportedSourceTypes
}
}
This query shows the status of each build of the engine. If a builds shows "status": "deployed", that build is invoked if a job is run using the engine.
The query produces a familiar-looking set of fields and values. If not, or if an error is shown, do not run the engine until the error is addressed.
[Note] If the error message is "The requested object was not found", the engine is likely not in a Deployed state. Check the list of engines in Veritone Developer and deploy the engine, if required, or unpause it if it was paused.
2. Run a job and inspect the results.
To test the engine, run a GraphQL mutation that looks like this. Be sure to substitute the engine's ID in the appropriate place:
Example
mutation createVocabExtractionJob{
createJob(input: {
target: { startDateTime:1574311000, stopDateTime: 1574315000 }
# targetId: 1121185051 # comment this line if using without a TDO
clusterId :"rt-1cdc1d6d-a500-467a-bc46-d3c5bf3d6901"
tasks: [
{
# webstream adapter (WSA)
engineId: "9e611ad7-2d3b-48f6-a51b-0a1ba40fe255"
payload: { url: "http://se.cs.depaul.edu/Java/Chapter04/Lincoln.txt" }
ioFolders: [
{ referenceId: "wsaOutputFolder", mode: stream, type: output }
],
executionPreferences: { priority: -20000 }
}
{
# data-chunking engine
engineId: "8bdb0e3b-ff28-4f6e-a3ba-887bd06e6440"
payload:{ ffmpegTemplate: "rawchunk" }
ioFolders: [
{ referenceId: "chunkInputFolder", mode: stream, type: input },
{ referenceId: "chunkOutputFolder", mode: chunk, type: output }
],
executionPreferences: { parentCompleteBeforeStarting: true, priority: -20000 }
}
{
# The hello-world engine -- PUT THE ENGINE'S ID HERE:
engineId: "05a138cd-cedc-435c-8593-9c47f885391f"
ioFolders: [
{ referenceId: "engineInputFolder", mode: chunk, type: input },
{ referenceId: "engineOutputFolder", mode: chunk, type: output }
],
executionPreferences: { parentCompleteBeforeStarting: true, priority: -20000 }
}
{
# output writer
engineId: "8eccf9cc-6b6d-4d7d-8cb3-7ebf4950c5f3"
ioFolders: [
{ referenceId: "owInputFolder", mode: chunk, type: input }
],
executionPreferences: { parentCompleteBeforeStarting: true, priority: -20000 }
}
]
routes: [
{ ## WSA --> chunk
parentIoFolderReferenceId: "wsaOutputFolder"
childIoFolderReferenceId: "chunkInputFolder"
options: {}
}
{ ## chunk --> Engine
parentIoFolderReferenceId: "chunkOutputFolder"
childIoFolderReferenceId: "engineInputFolder"
options: {}
}
{ ## Engine --> output writer
parentIoFolderReferenceId: "engineOutputFolder"
childIoFolderReferenceId: "owInputFolder"
options: {}
}
]
}) {
targetId
id
}
}
The job has four tasks: an ingestion task involving the Veritone Webstream Adapter, a chunking task (in case the data is large enough to need chunking), the cognition engine, and Output Writer (a Veritone engine that aggregates JSON results).
The mutation produces a response like this:
Example
{
"data": {
"createJob": {
"id": "20114505_9aG5dkXpfh",
"targetId": "710414094"
}
}
}
The id of "20114505_9aG5dkXpfh" is the Job ID. The targetId is the ID of the TDO (Temporal Data Object) associated with this job.
3. Poll the job.
To determine the job's status, poll the job with this query. Substitute the appropriate jobId:
Example
query jobStatus {
job(id: "20114505_9aG5dkXpfh") {
status
targetId
tasks {
records {
status
engine {
id
name
}
}
}
}
}
In a few seconds or minutes, the job status changes from pending to running, and each task goes from queued to running to complete (or failed, if something went wrong).
4. Get the results.
When the job finishes with a status of complete, run this query to obtain the engine's results. Substitute the correct jobId:
Example
query getEngineOutput {
engineResults(jobId:"19104215_i6Vz2XQtCR") {
records {
tdoId
engineId
startOffsetMs
stopOffsetMs
jsondata
assetId
userEdited
}
}
}
For this Hello World engine, results look similar to this:
Example
{
"taskId": "19104215_i6Vz2XQtCR9tzRz",
"generatedDateUTC": "0001-01-01T00:00:00Z",
"validationContracts": [
"keyword"
],
"object": [
{
"label": "the",
"type": "keyword"
},
{
"label": "Gettysburg",
"type": "keyword"
},
{
"label": "Address",
"type": "keyword"
},
{
"label": "Fourscore",
"type": "keyword"
},
{
"label": "and",
"type": "keyword"
},
{
"label": "seven",
"type": "keyword"
},
{
"label": "years",
"type": "keyword"
},
{
"label": "ago",
"type": "keyword"
}
]
}
[Note] The keyword-object array in this JSON fragment was edited for length.
