Streaming and tools

A model talks; a tool does things. This lesson shows the loop that joins the two.

6 minLesson 3 of 90 of 3 steps

Two features turn a single call into a product: streaming, so the user doesn't wait, and tools, so the model can do things.

Streaming

With stream=true the answer arrives in pieces and the first words appear sooner. For any live chat interface, it's the right choice.

Python
resp = client.chat.completions.create(
    model="deepseek/deepseek-v4-flash",
    messages=[{"role": "user", "content": "Write a short story"}],
    stream=True,
)
for chunk in resp:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)

The last piece before [DONE] has usage and its choices is empty. That's why the loop checks choices first.

If an answer arrives all at once instead of flowing, the buffering is in your own proxy, such as nginx. We send the right header.

Tools

A model can't connect to your database by itself, but it can say which function to call with what input. That's function calling.

  1. You send the request with the tool definitions.
  2. If the model wants a tool, finish_reason is tool_calls.
  3. You run the function yourself.
  4. You add the result to the messages with role tool and the same tool_call_id, and send again.

Tool schemas are part of the prompt and count as input tokens on every call of that conversation.

Tools we run for you

Three tools with the iranrouter: prefix you don't run yourself. We search or read, give the result to the model, and one final answer comes to you.

NameWhat it does
iranrouter:web_searchSearches the public web
iranrouter:web_fetchReads the text of one public page
iranrouter:datetimeDate and time in Tehran and UTC, with the Jalali date
Python
resp = client.chat.completions.create(
    model="deepseek/deepseek-v4-flash",
    messages=[{"role": "user", "content": "What is today's date?"}],
    tools=[{"type": "function", "function": {"name": "iranrouter:datetime"}}],
)
print(resp.choices[0].message.content)

The model must support function calling. Each round of this loop is a whole request, so one answer may show several rows in your usage. The loop is capped so it can't run away.

web_fetch opens only an address that appears exactly in your messages or an earlier search result. That stops a hostile page from walking your conversation out through a URL.

Call the datetime tool0 of 3 steps

  1. Run the sample above with your key.
  2. See that the answer has the right date.
  3. Count this request's rows in the logs.

What you learned

  • Streaming improves the experience, and usage comes in the last piece.
  • You run ordinary tools; we run the iranrouter: ones.
  • Next: making your app resilient when models go down.
Reliability: fallback models and safe retriesModels sometimes don't answer. Your app shouldn't go down with them.Done, next lesson