If you're following the steps to build a simple text-processing engine, this is step 3. To see all the steps, see the user guide process page.
In this step, create a Dockerfile and use Docker to create a deployable engine build.
Veritone engines are Docker containers that run in the aiWARE platform. Conceptually, think of a Docker image as a self-contained, runnable black box that has a single well-defined entry point.
[Note] A Docker container is a running instance of a Docker image.
Before you begin
Install Docker on a MacOS or Linux development machine.
Steps
1. Ensure correct structure.
Be sure the project files are in the correct places so that Docker can find them. The project files should look like:
/hello-world
|— Dockerfile
|— index.js
|— keyword-extraction.js
|— manifest.json
|— package.json
2. Create a manifest (optional)
Note that the manifest file is not required, but is recommended as a best practice to document your engine. Veritone recommends the following useful references below.
"com.veritone.build.author": "john doe",
"com.veritone.build.date": "2022-04-07T13:03:46Z",
"com.veritone.engine.id": "1fc4d3d4-54ab-42d1-882c-cfc9df42f386",
"com.veritone.engine.name": "Translate J",
"com.veritone.git.author": "john doe",
"com.veritone.git.branch": "master",
"com.veritone.git.commit": "9477f60540233aec8ed46442c1345255afaf5291",
"com.veritone.git.path": "engines/aiware/translate/translate-j",
"com.veritone.git.repo": "veritone/engines"
}
3. Create the Dockerfile
A Dockerfile is a text file that explains to Docker how to build an image. Think of a it as a recipe made up of one-line commands (although if needed, commands can span more than one line).
The syntax for Dockerfiles is self-explanatory.
Here's the Dockerfile for the Hello World project:
FROM veritone/aiware-engine-toolkit as vt-engine-toolkit
FROM mhart/alpine-node:14.8
COPY . /app
COPY manifest.json /var/
WORKDIR /app
EXPOSE 8080
RUN apk update \
&& apk upgrade \
&& apk --no-cache add ca-certificates \
&& apk add --no-cache libc6-compat
RUN npm install
ENV VERITONE_WEBHOOK_READY="http://0.0.0.0:8080/ready"
ENV VERITONE_WEBHOOK_PROCESS="http://0.0.0.0:8080/process"
COPY --from=vt-engine-toolkit /opt/aiware/engine /opt/aiware/engine
RUN ["chmod", "+x", "/app/index.js"]
ENTRYPOINT [ "/opt/aiware/engine", "node", "index.js" ]
Line by line:
The Dockerfile should contain two FROM commands, at the very top. The purpose of this command is to specify the base image(s) from which the custom Docker image will be built.
[Tip] In the Docker world, every Docker image must be built off at least one parent (base) image.
The Dockerfile says to base the image off veritone/aiware-engine-toolkit (which is the Engine Toolkit SDK image) plus mhart/alpine-node:8, which is a minimal Node.js image built on Alpine Linux, available via https://hub.docker.com/r/mhart/alpine-node.
| Command | What it means |
|---|
COPY . /app | Copies (recursively) everything in the current context — that is, the current directory — to a folder called /app. |
COPY manifest.json /var/ | Copies the manifest file to the /var/ folder. |
WORKDIR /app | Signifies that /app is the working directory. |
EXPOSE 8080 | Makes port 8080 active for interaction with the engine. |
RUN apk update \ && apk upgrade \ && apk --no-cache add ca-certificates \ && apk add --no-cache libc6-compat | Updates Alpine Linux (apk) to include the latest CA-certs so that TLS works properly. It also adds the libc libraries that Engine Toolkit relies on (libc6-compat). |
RUN npm install | Runs the Node package manager, npm, using the install command. By default, npm install will install all modules listed as dependencies in package.json, if such a file exists. |
ENV VERITONE_WEBHOOK_READY="http://0.0.0.0:8080/ready" \ ENV VERITONE_WEBHOOK_PROCESS="http://0.0.0.0:8080/process" | Creates the environment variables that allow Engine Toolkit to know the webhooks it can use to contact the engine. |
COPY --from=vt-engine-toolkit /opt/aiware/engine /opt/aiware/engine | Copies the Engine Toolkit files to /opt/aiware/engine. |
RUN ["chmod", "+x", "/app/index.js"] | Grants the engine "execution" rights, so it can run. |
WORKDIR /app | Makes /app the working directory. |
ENTRYPOINT [ "/opt/aiware/engine", "node", "index.js" ] | Sets the entry point of the container as the Engine Toolkit, with node and index.js as arguments. (The Engine Toolkit will, in turn, start up Node.js using index.js as an argument.) |
4. Build the engine
To build the engine, run the following command in a terminal (e.g., a bash shell):
docker build -t hello-world .
[Warning] Don't omit the trailing period in this command.
The Docker daemon builds the image. The console shows something similar to this:
Example
Sending build context to Docker daemon 18.16MB
Step 1/10 : FROM mhart/alpine-node:8
---> b9df20fda4ef
Step 2/10 : COPY . /app
---> 8e9498d2fede
Step 3/10 : COPY manifest.json /var/
---> 3a2d603d9fb8
Step 4/10 : COPY ./dist/engine /app/engine
---> 53c0b02010e2
Step 5/10 : WORKDIR /app
---> Running in 8ae399220a7f
Removing intermediate container 8ae399220a7f
---> 51a7a86835c8
Step 6/10 : RUN npm install -i
---> Running in 41cd3c6522d5
npm notice created a lockfile as package-lock.json. Commit this file.
npm WARN hello-world@1.0.0 No description
npm WARN hello-world@1.0.0 No repository field.
added 71 packages from 47 contributors and audited 164 packages in 2.724s
found 0 vulnerabilities
Removing intermediate container 41cd3c6522d5
---> 003d48c3ce60
Step 7/10 : RUN apk update && apk upgrade && apk --no-cache add ca-certificates
---> Running in 96ff2c8839a5
fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/community/x86_64/APKINDEX.tar.gz
v3.6.5-44-gda55e27396 [http://dl-cdn.alpinelinux.org/alpine/v3.6/main]
v3.6.5-34-gf0ba0b43d5 [http://dl-cdn.alpinelinux.org/alpine/v3.6/community]
OK: 8442 distinct packages available
OK: 5 MiB in 15 packages
fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/community/x86_64/APKINDEX.tar.gz
(1/1) Installing ca-certificates (20161130-r3)
Executing busybox-1.26.2-r11.trigger
Executing ca-certificates-20161130-r3.trigger
OK: 6 MiB in 16 packages
Removing intermediate container 96ff2c8839a5
---> a51b8d2070b6
Step 8/10 : ENV VERITONE_WEBHOOK_READY="http://0.0.0.0:8080/ready"
---> Running in 9d3979283eb1
Removing intermediate container 9d3979283eb1
---> b3ade3dffab5
Step 9/10 : ENV VERITONE_WEBHOOK_PROCESS="http://0.0.0.0:8080/process"
---> Running in 3c01dfd014f3
Removing intermediate container 3c01dfd014f3
---> 389b21b9b7d3
Step 10/10 : ENTRYPOINT [ "/app/engine", "node", "index.js" ]
---> Running in d60071122938
Removing intermediate container d60071122938
---> a59635e1673d
Successfully built a59635e1673d
Successfully tagged hello-world:latest
The console messages are reasonably verbose (and, for the most part, self-explanatory). If the build encounters errors, the console notes which Dockerfile command failed and why.
In this example, the build went smoothly, which means the engine is ready to test.
Recap
This step to building an engine covered:
- Information on creating Docker build instructions in a simple text file called a Dockerfile.
- Information on basing a build off one or more preexisting parent builds or base image using the
FROM keyword. - Using
COPY to make Docker copy files. - Using
RUN to make Docker execute shell instructions during the build process. - Using
RUN commands to update an image's CA certificates so that container-based SSL works at runtime. - Using
ENV to specify container-specific environment variables. - Using
ENTRYPOINT to specify the runtime entry point of our Docker container. - Using the
docker build command to create an engine's build (the deployable image).
What to do next
Use Veritone's Engine Toolkit to test the engine locally in step 4: Test the engine build.
