Simulation size directly impacts both the diversity of your testing and your simulation costs. Getting the right balance ensures comprehensive coverage without unnecessary expense.

​
Three Simulation Size Parameters

​
Number of Personas
integer
Each persona represents a distinct user with unique style, goals, and behavior patterns. Conversations with the same persona will feel like the same type of person is talking.
​
Max Conversations
integer
The maximum number of total conversations to generate. The actual number might be slightly under this, but typically very close.
​
Max Conversation Length
integer
The maximum number of turns (back-and-forth exchanges) in each conversation. Actual conversations will vary from 3 turns up to your maximum, determined by how the persona and your chatbot interact.
These are maximums - actual conversation lengths and counts may be lower based on natural conversation flow.

​
Sizing Guidelines

​
Small Volume (Under 100 conversations)

Maximize diversity with a 5:1 ratio of conversations to personas.
  • 25 conversations → 5 personas
  • 50 conversations → 10 personas
  • 75 conversations → 15 personas
This gives you the most variety in user types and interaction patterns.

​
Medium Volume (100-500 conversations)

Maintain ratio between 5:1 and 20:1 based on your goals.
  • Closer to 5:1 for maximum diversity
  • Closer to 20:1 for deeper testing of each persona type
  • 200 conversations → 10-40 personas
  • 300 conversations → 15-60 personas

​
Large Volume (Over 500 conversations)

Don’t exceed 20:1 ratio (20 conversations per persona).
  • 500 conversations → at least 25 personas
  • 1000 conversations → at least 50 personas
  • 2000 conversations → at least 100 personas
Beyond 20:1, you start seeing repetitive interactions that don’t add much testing value.

​
Common Testing Scenarios

Goal: Test a specific feature with diverse user typesRecommended Settings:
  • 25-50 conversations
  • 5-10 personas
  • 8-12 max turns (medium length for multi-step processes)
Best for: Quick support tickets, feature demos, basic transactions
Goal: Thorough testing across all chatbot capabilitiesRecommended Settings:
  • 200-500 conversations
  • 20-25 personas
  • 12-20 max turns (medium to long for consultative interactions)
Best for: Full chatbot audits, pre-launch testing, quarterly reviews
Goal: Find unusual failure modes and stress-test robustnessRecommended Settings:
  • 100-200 conversations
  • 10-15 personas
  • 15-25 max turns (long conversations reveal complex issues)
Best for: Complex problem-solving, educational sessions, detailed consultations
Goal: Ensure recent changes don’t break existing functionalityRecommended Settings:
  • 50-100 conversations
  • 10-15 personas
  • Match your typical conversation length (3-8 turns for simple Q&A, 15+ for complex workflows)
Best for: Post-deployment validation, continuous integration testing

​
T-shirt Sizing

Not sure what size to pick? Start with these defaults:
SizeConversationsPersonasMax Turns
Small100105
Medium250257
Large15007512
Run your first simulation, review the results, then adjust based on what you learn:
  • Need more diversity? Increase personas.
  • Missing edge cases? Increase total conversations.
  • Conversations ending too early? Increase max turns.

​
Resources

Join the Community

Connect with other developers, ask questions, and share feedback.

See Examples

See real simulation examples with different sizing strategies.

Write Chatbot Description

Learn how to write a chatbot description that supports your simulation intent effectively.

Define Simulation Intent

Learn how to define what you want to test with your simulations.
Questions? Join our developer community or contact support for help choosing the right simulation size.