Speaker detection engines

A speaker detection engine (also called a diarization or speaker separation engine) partitions an audio stream into segments, based on who is speaking and when. Unlike a speaker recognition engine, a speaker detection engine merely determines when speakers change and possibly which speakers are the same person. It does not try to identify who the speaker is.

Engine input

Audio-processing engines can be stream processing engines, or if processing will be stateless, they can be segment processing. See Engine processing modes to learn more.

All engines that process audio will receive audio data with MIME type "audio/wav" (.mp3 and .mp4 are not natively supported). If your engine needs a format other than audio/wav, you will need to transcode incoming wav data to the appropriate target format using something like ffmpeg.

Engine output

Within the time-based series array in the engine's output, each speaker detection record (that is, each series entry) should contain an object of type speaker.

Example

Here is an example of the simplest type of speaker detection output:

{
  "schemaId": "https://docs.veritone.com/schemas/vtn-standard/master.json",
  "validationContracts": [
    "speaker"
  ],
  "series": [
    {
      "startTimeMs": 0,
      "stopTimeMs": 10000,
      "object": {
        "type": "speaker",
        "label": "Tim"
      }
    }
  ]
}

Legacy format

There is a legacy format that is still supported but has been deprecated.

{
  "series": [
    {
      "startTimeMs": 0,
      "stopTimeMs": 10000,
      "speakerId": "<label>"
    }
  ]
}

• Learn about AION.
• Learn more about Engine processing modes.
• See the cognitive engine classes.
• See an overview of cognitive technology.
• View engines for your media type.

• Using aiWARE
• Use cognitive engines
• Choose an engine
• Capability
• Speech
• Speaker detection