Beyond text: images, audio, video, OCR and embeddings
The same key and wallet do things you may not have thought of yet.
The same key and wallet aren't only for text. This lesson is a map of everything else; for the details of each, we go to the API reference.
Images as input
Models with image input take it inside the same messages. The input and output section of each model's page says whether it's accepted.
resp = client.chat.completions.create(
model="deepseek/deepseek-v4-flash",
messages=[{"role": "user", "content": [
{"type": "text", "text": "What do you see in this image?"},
{"type": "image_url", "image_url": {"url": "https://example.com/x.jpg"}},
]}],
)Making images
POST /v1/images/generations makes images. It's billed per image, capped at 10 per request, and the answer comes back as base64.
Audio
- Text to speech:
POST /v1/audio/speech. The response is an audio file; its cost comes in thex-cost-rialheader when it is known at once, otherwise read it fromGET /v1/generation?id=with the request id. - Speech to text:
POST /v1/audio/transcriptionsandtranslations, by file upload. - Live conversation:
wss://.../v1/realtime, on the Realtime protocol with one hold per session.
Video
POST /v1/videos submits the job and waits for the end, usually one to a few minutes. Set your client timeout to at least 10 minutes, or use the async route.
Reading documents
POST /v1/ocr turns a document or image into markdown text, priced per page. If you only want a model to read a PDF, send the file-parser plugin in your request:
"plugins": [{"id": "file-parser"}]
Embeddings and rerank
For semantic search and building question answering over your own documents, POST /v1/embeddings makes a vector for each text, and POST /v1/rerank orders results by how well they fit the question. In brief:
- Split documents into pieces and get an embedding for each.
- Turn the question into an embedding too and find the nearest pieces.
- Use rerank to bring the best to the top.
- Give those pieces to the model with the question.
The model list has it all; the endpoint field in GET /v1/models says which path each model answers on.
/v1 or /openai for OpenAI's SDK, /anthropic for Anthropic's, and /genai for Google's. All four are one engine and take the same key.Read the model list0 of 3 steps
- Call
GET /v1/modelswith your key. - Look at the
endpointandarchitecturefields of a few models. - Find an embedding or image model.
What you learned
- Images, audio, video, OCR and embeddings all run on the same key.
- The complete reference for each route is in the docs.
- Next: working as a team.
Keep your place
Sign in to the panel and each lesson and your path are saved, so you continue from the same spot on any device.