Root cause: sequential execution of "chat.completions.create"

3 "openai.chat.completions.create" spans ran one after another — each started only after the previous one finished. Total: 5.6s, slowest single call: 2.2s. Running them concurrently saves 3.4s — a 2.6× speedup.

Now (sequential)After fix (parallel)

5.6s

→

~2.2s

Fix: run calls in parallel

fix.py

from concurrent.futures import ThreadPoolExecutor

# Before — sequential, each call blocks the next:
for item in items:
    result = process(item)

# After — all calls run concurrently:
with ThreadPoolExecutor(max_workers=8) as pool:
    results = list(pool.map(process, items))

# Cost unchanged. Output unchanged. Time = slowest single call.

Waterfall

agent › run

agent.run

user: u_heavy_11

6.90s

openai › chat › completions › create

openai.chat.completions.create

gpt-4o1.7k tok$0.015faithful 0.93user: u_heavy_11

1.52s

Input

Summarize the Q3 earnings report attached. Focus on revenue, margin, and guidance.

Output

(answer to: Summarize the Q3 earnings report attache…)

openai › chat › completions › create

openai.chat.completions.create

gpt-4o1.9k tok$0.017faithful 0.91user: u_heavy_11

1.90s

Input

Given the customer ticket below, draft a refund response that follows policy P-204.

Output

(answer to: Given the customer ticket below, draft a…)

openai › chat › completions › create

openai.chat.completions.create

gpt-4o768 tok0.69¢faithful 0.91user: u_heavy_11

2.17s

Input

Translate the user manual section to Japanese, preserving the table structure.

Output

(answer to: Translate the user manual section to Jap…)

Tool: code_exec

tool.code_exec

user: u_heavy_11

470ms

Tool: calculator

tool.calculator

user: u_heavy_11

566ms

Tool: web_fetch

tool.web_fetch

user: u_heavy_11

170ms

Faithfulness · openai.chat.completions.create0.50

1 supported · 0 contradicted · 1 unsupported of 2 claims

unsupported
All chapters were translated.
supported
Tables were preserved in the output.