[API][yes]
[Search][no]
[UI][partial]
Audio fingerprinting engines identify pre-recorded audio snippets in audio files based on a particular signature or "fingerprint."
In aiWARE, audio fingerprinting engines are trained based on one or more libraries containing entities with audio fingerprint identifiers. The engines report time spans where these audio snippets have been matched.
Deployment model
Audio fingerprinting engines are typically network isolated due to requiring libraries and engine models. Set this in Step 2 - Functionality of the engine registration process in the Developer utility. See Register an engine for more information.
Training and libraries
Since all audio fingerprinting engines identify entities, they are all required to be trainable via libraries.
Engine input
Audio-fingerprinting engines are usually stream processing engines. Set this in Step 2 - Functionality of the engine registration process in the Developer utility. See Register an engine for more information.
[Note] All engines that process audio will receive audio data with MIME type "audio/wav" (.mp3 and .mp4 are not natively supported). If your engine needs a format other than audio/wav, you will need to transcode incoming wav data to the appropriate target format using something like ffmpeg.
Engine output
Audio fingerprinting engine output should be stored as objects in the time-based series array in .aion. Each detected section of audio is represented as an object of type fingerprint. Because audio fingerprints always map back to an entity in a library, they should always include the entityId of the fingerprint in their output.
Example
{
"schemaId": "https://docs.veritone.com/schemas/vtn-standard/master.json",
"validationContracts": [
"audio-fingerprint"
],
"series": [
{
"startTimeMs": 817000,
"stopTimeMs": 845000,
"object": {
"type": "fingerprint",
"entityId": "e4f95344-622b-42bb-8e82-514d96d2df88",
"libraryId": "0fb79432-dcb1-40b0-bb6a-3f7e481aae3e",
"confidence": 0.002928053541550474
}
}
]
}
