OpenAI Async Tool Calling Without Lost Results

OpenAI async tool calling lets GPT-6 Astra continue useful work while your application runs a slow function or custom tool. The feature removes a model-side wait, not the engineering around the wait: your service still owns job execution, persistence, retries, authorization, and delivery of the eventual result.

This guide implements that missing production layer. It shows how to preserve the original call_id, continue from the latest response, recover pending work after a restart, and avoid confusing async tools with Background mode or parallel tool calls.

Table of contents#

• What OpenAI async tool calling changes
• How is async tool calling different from Background mode
• Build a complete async tool loop
• Why a durable job registry is required
• How should late tool results rejoin the conversation
• When should the model use a wait tool
• Handle retries failures and duplicate delivery
• Compatibility limits to design around
• Production rollout checklist
• Frequently asked questions
• Conclusion

What OpenAI async tool calling changes#

OpenAI added async tool calling to the Responses API on September 3, 2026, according to the official API changelog. A normal function call pauses the model until your application returns a result. When a function or custom tool definition includes async: true, the model may issue that call and continue with work that does not depend on its result.

The distinction is deliberately narrow:

1. The model emits a function_call or custom_tool_call item marked async.
2. Your application validates the arguments and starts the external job.
3. The model can produce independent work instead of waiting.
4. Your application later sends an output item carrying the original call_id.
5. The conversation continues from its latest response, even if other turns happened first.

OpenAI does not host the job or provide a queue for it. A ten-minute database export, CI build, vendor lookup, or human approval remains your operation. Async calling only changes when the model must stop.

This makes it useful for work that has a real independent branch. If the model cannot do anything valid until a lookup returns, ordinary synchronous calling is simpler. Marking every function async adds state without adding concurrency.

How is async tool calling different from Background mode#

The word “async” appears in several agent patterns, but they solve different waits.

Capability	What continues	Who runs the tool	How the client resumes
Async tool calling	The model can do independent work after issuing one tool call	Your application	Send the late output with its original call_id
Background mode	The whole response runs asynchronously	OpenAI runs response generation; client tools are unchanged	Poll or stream the response
Parallel tool calls	Several calls are emitted together	Usually your application	Return outputs for those calls in the normal loop
Programmatic Tool Calling	Model-generated code coordinates eligible tools	Hosted program plus your client executor where applicable	Preserve the program continuation contract

Background mode is appropriate when the response itself takes long enough that an HTTP request should not stay open. It does not make a client-owned function non-blocking inside the model’s turn. Conversely, async tool calling does not give you a response-status polling API for your database job.

Programmatic Tool Calling is also a separate orchestration route. It is designed for model-generated code that fetches, filters, joins, or aggregates tool results. OpenAI currently says not to configure async tools for that route. See the site’s Programmatic Tool Calling guide for its caller-preserving continuation rules.

Build a complete async tool loop#

The smallest useful example starts one deterministic slow job and returns its result later. Install the official JavaScript SDK and set OPENAI_API_KEY in your environment:

npm install openai

Save this as async-tool.mjs:

import OpenAI from "openai";

const openai = new OpenAI();

const tools = [
  {
    type: "function",
    name: "build_release_report",
    description: "Build a read-only release report for one repository.",
    async: true,
    strict: true,
    parameters: {
      type: "object",
      properties: {
        repository: { type: "string", minLength: 1 },
        task_handle: { type: "string", minLength: 1 },
      },
      required: ["repository", "task_handle"],
      additionalProperties: false,
    },
  },
];

async function buildReleaseReport({ repository, task_handle }) {
  // Replace this deterministic fixture with your authorized job service.
  await new Promise((resolve) => setTimeout(resolve, 250));
  return {
    task_handle,
    repository,
    status: "completed",
    findings: ["No unresolved release blockers in the demo fixture"],
  };
}

const first = await openai.responses.create({
  model: "gpt-6-astra",
  tools,
  instructions: [
    "Start the release report early.",
    "While it runs, provide a generic pre-release checklist.",
    "Do not invent repository findings.",
    "Use a task_handle unique within this conversation.",
  ].join(" "),
  input: "Review acme/widgets for release readiness.",
});

const call = first.output.find(
  (item) =>
    item.type === "function_call" &&
    item.name === "build_release_report",
);

if (!call) throw new Error("Expected build_release_report call");

const args = JSON.parse(call.arguments);
const job = buildReleaseReport(args).catch((error) => ({
  task_handle: args.task_handle,
  status: "failed",
  error: { code: "REPORT_FAILED", message: error.message },
}));

// Independent user turns could occur here. If they do, replace this with
// the most recent response ID before delivering the late result.
let latestResponseId = first.id;
const result = await job;

const resumed = await openai.responses.create({
  model: "gpt-6-astra",
  tools,
  previous_response_id: latestResponseId,
  input: [
    {
      type: "function_call_output",
      call_id: call.call_id,
      output: JSON.stringify(result),
    },
  ],
});

console.log(resumed.output_text);

Run it with your environment configured:

node async-tool.mjs

The example is intentionally read-only. For a real deployment, replace the in-process promise with a durable queue and store the association before acknowledging that the job has started.

Why a durable job registry is required#

An in-memory Map works only until the process restarts, autoscaling sends the callback to another instance, or a job finishes after the original worker is gone. A durable registry turns the late result into recoverable application state.

Store at least:

Field	Purpose
conversation_key	Your stable tenant-safe conversation identifier
task_handle	Model-visible handle, unique for the conversation
call_id	OpenAI identifier required when delivering the output
launch_response_id	Response that produced the call
latest_response_id	Current continuation head for that conversation
tool_name and argument hash	Audit and duplicate detection without logging unnecessary secrets
status	queued, running, completed, failed, delivered, or cancelled
attempt_count and timestamps	Retry policy, timeout handling, and operations
result reference	Pointer to bounded encrypted output, not necessarily the full payload

Use a uniqueness constraint such as (conversation_key, task_handle). OpenAI’s wait-tool pattern requires task handles to stay unique across the conversation, including completed and repeated lookups. Also treat result delivery as a state transition protected by a transaction or compare-and-set operation.

Do not use call_id as authorization. Before launching any tool, resolve the authenticated tenant and permitted resource server-side. The model’s arguments are untrusted input even when strict JSON Schema makes their shape predictable.

This is the same separation of context from durable state described in the context engineering guide: keep compact references in the conversation and operational truth in a store designed for recovery.

How should late tool results rejoin the conversation#

Two identifiers have different jobs:

• call_id identifies the original tool call and never changes for that job.
• previous_response_id points to the latest response in the conversation when you deliver the result.

Suppose response A launches an async call. The user then sends a correction and the model produces response B before the job finishes. The eventual function_call_output still carries A’s tool call_id, but the continuation request should use B as previous_response_id.

Serialize writes to each conversation head. If two workers read the same latest response and both create continuations, they can produce competing branches. A practical delivery worker should:

1. lock or version-check the conversation record;
2. read the latest response ID;
3. verify that the job has not already been delivered;
4. submit the bounded result with the original call_id;
5. atomically store the new response ID and mark the job delivered.

If your privacy configuration requires stateless continuation, preserve and replay the required response items instead of relying on stored-response chaining. Do not assume store: false alone is a complete data-retention policy; the site’s regional-processing guide explains why routing and retention controls must be verified separately.

When should the model use a wait tool#

Sometimes the model launches two slow tasks, completes independent work, and then needs both results to compare them. OpenAI documents an application-defined synchronous wait tool for this boundary.

Each async tool receives a unique task_handle. Your registry maps that handle to the original call_id and running job. A normal non-async wait_for_tasks tool accepts the handles the model now depends on. Your executor waits for only those jobs, then returns the newly completed outputs on their original call IDs before returning the wait call’s own status.

The ordering matters. Results should be present when the model resumes from the wait; a status saying “completed” without the corresponding tool outputs gives it nothing authoritative to use.

wait_for_tasks is not a hosted OpenAI tool. Your application defines its schema, maximum wait, cancellation behavior, and error contract. Only expose it when the workflow has meaningful work to perform before the dependency barrier. Otherwise, use an ordinary synchronous tool.

Handle retries failures and duplicate delivery#

Async orchestration creates two retry domains: running the business job and delivering its output to the Responses API. Keep them separate.

For job execution:

• Retry only errors your tool classifies as transient.
• Use an idempotency key derived from tenant, tool, and stable task identity for any side effect.
• Set a per-attempt timeout and an overall deadline.
• Persist terminal failure as structured data so the model can explain the failure without inventing a result.
• Bound output size and store large artifacts separately behind authorized references.

For result delivery:

• Retry transport failures with backoff and jitter.
• Never rerun the business job merely because delivery failed.
• Mark delivered only after the continuation succeeds.
• Reconcile records stuck in completed but not delivered.
• Record response IDs, call IDs, task handles, duration, attempts, and byte counts without logging secrets or full sensitive outputs.

Do not silently turn a failure into an empty success. Prefer a predictable result such as:

{
  "task_handle": "release_acme_widgets_01",
  "status": "failed",
  "error": {
    "code": "UPSTREAM_TIMEOUT",
    "retryable": false
  }
}

If a tool can write, deploy, purchase, or send, add explicit authorization and approval outside the model. Async execution should not weaken the same permission boundary you would enforce for a synchronous call. The AI coding-agent permission contract offers a useful model for separating proposed actions from authorized execution.

Compatibility limits to design around#

According to OpenAI’s current async tool calling documentation, the feature has four important fences:

• It is supported by GPT-6 Astra and later models.
• It applies to function and custom tools that your application runs.
• It does not apply to hosted built-in tools.
• It should not be configured for Programmatic Tool Calling.

OpenAI also says not to combine async tools with parallel tool calls in multi-agent mode. Treat that as an architecture constraint, not a prompt suggestion.

The launch also changes older Chat Completions assumptions. GPT-6 Astra tool calling requires the Responses API. A migration should therefore inventory endpoint usage, output-item parsing, conversation continuation, and tool schemas before switching the model name.

Avoid coding against an assumed maximum pending duration: the current async-tool guide does not specify one. Set your own job deadline, retention window, cancellation policy, and user-facing status behavior based on the business operation.

Production rollout checklist#

Start with one slow, read-only tool and test the state machine before adding side effects.

Test	Expected behavior
Process stops after launch	Worker recovers the persisted pending job
User sends another turn	Late result attaches using the latest response head
Same completion event arrives twice	Only one continuation is created
Tool times out	Structured terminal failure reaches the model
API delivery returns a transient error	Result is retried without rerunning the job
Two jobs finish together	Conversation updates are serialized
Result exceeds the contract	Payload is reduced or replaced by an authorized reference
Tenant identifier is manipulated	Server-side authorization rejects the call

Measure end-to-end task success, time until the first useful response, total completion time, outstanding-job age, duplicate suppression, delivery retries, result bytes, model tokens, and downstream cost. Async calling is valuable when it improves useful latency or workflow continuity without reducing correctness.

Frequently asked questions#

What is OpenAI async tool calling?#

It is a Responses API feature that lets a supported model continue independent work after it issues an application-run function or custom tool call marked async: true. Your application still executes the tool and later sends its output using the original call_id.

Is async tool calling the same as Background mode?#

No. Background mode makes the entire response asynchronous and gives the client a response to poll or stream. Async tool calling changes whether the model must pause for one client-owned tool result.

Which models support async tool calling?#

OpenAI currently documents support for GPT-6 Astra and later models. Check the official compatibility section before relying on the feature with another model.

Can hosted tools run asynchronously this way?#

No. The documented async: true behavior applies to function and custom tools executed by your application, not OpenAI-hosted built-in tools.

Do I need a task handle as well as a call ID?#

For simple delivery, the API correlation uses call_id. A model-visible task_handle becomes useful when several jobs are pending and the model may call an application-defined wait tool. Keep each handle unique for the entire conversation and map it to the original call ID in durable storage.

Conclusion#

OpenAI async tool calling can remove wasted model-side waiting from long agent workflows, but production reliability comes from the application around it. Persist every launch, preserve the original call ID, continue from the latest conversation head, serialize delivery, and separate business retries from API retries. Begin with one read-only slow tool and expand only after restart, duplicate, timeout, and late-result tests pass.