Beyond text: images, audio, video, OCR and embeddings

The same key and wallet do things you may not have thought of yet.

5 minLesson 5 of 90 of 3 steps

The same key and wallet aren't only for text. This lesson is a map of everything else; for the details of each, we go to the API reference.

Images as input

Models with image input take it inside the same messages. The input and output section of each model's page says whether it's accepted.

Python
resp = client.chat.completions.create(
    model="deepseek/deepseek-v4-flash",
    messages=[{"role": "user", "content": [
        {"type": "text", "text": "What do you see in this image?"},
        {"type": "image_url", "image_url": {"url": "https://example.com/x.jpg"}},
    ]}],
)

Making images

POST /v1/images/generations makes images. It's billed per image, capped at 10 per request, and the answer comes back as base64.

Audio

  • Text to speech: POST /v1/audio/speech. The response is an audio file; its cost comes in the x-cost-rial header when it is known at once, otherwise read it from GET /v1/generation?id= with the request id.
  • Speech to text: POST /v1/audio/transcriptions and translations, by file upload.
  • Live conversation: wss://.../v1/realtime, on the Realtime protocol with one hold per session.

Video

POST /v1/videos submits the job and waits for the end, usually one to a few minutes. Set your client timeout to at least 10 minutes, or use the async route.

Reading documents

POST /v1/ocr turns a document or image into markdown text, priced per page. If you only want a model to read a PDF, send the file-parser plugin in your request:

JSON
"plugins": [{"id": "file-parser"}]

Embeddings and rerank

For semantic search and building question answering over your own documents, POST /v1/embeddings makes a vector for each text, and POST /v1/rerank orders results by how well they fit the question. In brief:

  1. Split documents into pieces and get an embedding for each.
  2. Turn the question into an embedding too and find the nearest pieces.
  3. Use rerank to bring the best to the top.
  4. Give those pieces to the model with the question.

The model list has it all; the endpoint field in GET /v1/models says which path each model answers on.

Each SDK has its own prefix: /v1 or /openai for OpenAI's SDK, /anthropic for Anthropic's, and /genai for Google's. All four are one engine and take the same key.

Read the model list0 of 3 steps

  1. Call GET /v1/models with your key.
  2. Look at the endpoint and architecture fields of a few models.
  3. Find an embedding or image model.

What you learned

  • Images, audio, video, OCR and embeddings all run on the same key.
  • The complete reference for each route is in the docs.
  • Next: working as a team.
Organizations, workspaces and membersOne wallet, several projects, and everyone sees only what they should.Done, next lesson