Talking to a Model

You know the model runs on a server, you talk to it through an API, and you prove who pays with an API key.

Now the full trip in detail: what your program sends, what comes back, and how to read it.

What a turn is

Imagine your program wants the model to answer: What is 2 + 2?

One full trip looks like this:

1. Your program builds a request — the text for the model, plus a few settings
2. It sends that request to the server, with your API key as proof
3. The server runs the model
4. Your program gets a response — the reply text, plus a little extra info

That full trip is one turn.

Think of a turn like one back-and-forth: you ask once, you get one answer package back. Later you will stack many turns. First lock this single trip in your head.

  Your program                      Server
  ┌──────────────┐   request       ┌──────────────┐
  │  prompt + key  │ ──────────────▶ │  model runs  │
  │              │ ◀────────────── │              │
  └──────────────┘   response      └──────────────┘

Request and response as JSON

Your program does not “talk” to the server in casual English over the wire.

It sends structured text — labeled fields the server can read. That format is usually JSON: text shaped like a form, with names and values.

Request — what you send

Here is a small, complete request for our 2 + 2 question:

{
  "model": "some-model",
  "messages": [
    {
      "role": "user",
      "content": "What is 2 + 2?"
    }
  ]
}

Walk each field:

Your API key travels with the request too, but usually outside this JSON — in a separate header the server checks first. The JSON is the prompt. The API key is the proof.

Companies rename fields (messages vs contents, and so on). The jobs stay the same: pick a model, send text, say who said it.

Response — what comes back

The server does not return only a bare string. It returns another JSON object. A simplified shape:

{
  "message": {
    "role": "assistant",
    "content": "2 + 2 is 4."
  },
  "finish_reason": "stop",
  "usage": {
    "input_tokens": 10,
    "output_tokens": 6
  }
}

Walk each part:

Real company responses wrap these ideas in slightly different nesting. Your job as a reader of the response is always the same: find the reply text, why it stopped, and usage.

Put the whole turn in one glance:

  REQUEST (JSON)              RESPONSE (JSON)
  model + messages  ──────▶   assistant content
  (+ API key beside it)   ◀── finish_reason + usage

For this chapter, role: "user" means your question and role: "assistant" means the model’s reply. That is enough. (More roles come later when conversations get longer.)

Limits and dials

The tiny request above is enough for a first mental model. Real requests often add limits and dials, for example:

{
  "model": "some-model",
  "messages": [
    { "role": "user", "content": "Explain gravity in two sentences." }
  ],
  "max_tokens": 60
}

max_tokens (names vary) means: do not let the reply grow past this much work.

Why it matters: without a limit, a long reply can cost more and take longer. With a limit that is too small, the reply can stop mid-thought — and finish_reason will often tell you that happened.

You do not need every dial today. Remember the pattern: prompt + settings, not the question alone.

Reading the response

Most beginners only print message.content. That works until it doesn’t.

Take two different finish reasons with the same kind of question:

Finished normally

{
  "message": { "role": "assistant", "content": "Gravity pulls masses together." },
  "finish_reason": "stop"
}

Hit a length limit

{
  "message": { "role": "assistant", "content": "Gravity pulls masses together. On Earth, it make" },
  "finish_reason": "length"
}

Same request idea. Different endings. If you only read the text, the second one looks like a weird half-sentence. If you also read finish_reason, you know the model was cut off.

Usage is the other field to notice. It is how billing and limits connect to this single turn.

Tokens and cost

You already know the model works in tokens — small pieces of text.

On a turn, usage usually splits into:

• Input tokens — pieces in what you sent (your prompt, and any other text in the request)
• Output tokens — pieces in what the model wrote

You usually pay for both.

Use the first example’s numbers:

"usage": {
  "input_tokens": 10,
  "output_tokens": 6
}

Rough story:

• Sending "What is 2 + 2?" (plus a little overhead) cost about 10 input tokens
• Writing "2 + 2 is 4." cost about 6 output tokens
• This turn’s work is 16 tokens total

Now change the question to a long paragraph, or ask for a long essay back. Both numbers climb. Longer prompts and longer replies cost more.

You do not need to count tokens by hand. You do need to know why usage exists in the response: it is the meter for this turn.

Each call forgets

Most model APIs start fresh every time.

The server does not keep a private memory of your last request. If you want the model to use earlier text, you must send that text again in the next request.

See it with two separate turns.

Turn 1 — you introduce a name

{
  "model": "some-model",
  "messages": [
    { "role": "user", "content": "My cat is named Mango." }
  ]
}

Turn 1 — reply (shortened)

{
  "message": {
    "role": "assistant",
    "content": "Got it — your cat is named Mango."
  }
}

Turn 2 — a new call, only the new question

{
  "model": "some-model",
  "messages": [
    { "role": "user", "content": "What is my cat's name?" }
  ]
}

Turn 2 did not include the Mango sentence. The server has no leftover memory from Turn 1. The model may guess, invent, or say it does not know — because that fact was never in this request.

So for one turn: there is no memory. Conversation memory is something you build later by sending history on purpose.

Ready for a real call

You now have the full picture for one turn: build a request, send it with your API key, read the response (text, finish reason, usage).

Next you will do that from a real program — same shape, live server.