Cascade: "rag.answer" fanned out into 45 "openai.chat.completions.create" calls
"rag.answer" produced 45 "openai.chat.completions.create" calls in a single trace — the same operation, over and over. This is the fan-out / traversal-explosion pattern: cost and latency scale linearly with the input, and it will rate-limit your provider at volume.
Fix: batch the fan-out or cap the loop
# Before — one API call per item, unbounded (45 calls in one trace):
for item in items:
client.chat.completions.create(...) # or embed(item)
# After — batch what you can, cap what you can't:
inputs = [i.text for i in items[:MAX_FANOUT]] # e.g. MAX_FANOUT = 16
client.embeddings.create(model="text-embedding-3-small", input=inputs)
# If each item truly needs its own completion, ask whether 45 of them
# belong in one request at all — that is usually a retrieval/ranking bug.Waterfall
rag.answer
openai.chat.completions.create
Score chunk 1 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 2 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 3 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 4 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 5 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 6 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 7 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 8 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 9 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 10 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 11 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 12 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 13 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 14 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 15 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 16 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 17 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 18 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 19 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 20 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 21 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 22 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 23 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 24 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 25 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 26 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 27 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 28 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 29 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 30 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 31 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 32 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 33 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 34 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 35 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 36 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 37 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 38 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 39 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 40 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 41 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 42 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 43 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 44 of 45 for relevance to the question
(relevance score)
openai.chat.completions.create
Score chunk 45 of 45 for relevance to the question
(relevance score)