What are Evaluations?
Evaluations are automated tests that assess your agent’s performance after each call. They help you measure success rates, identify issues, and maintain quality standards across all conversations.Evaluations provide quantitative metrics to track your agent’s performance over time.
How Evaluations Work
Each evaluation is a yes/no question that the AI analyzes based on the conversation transcript. After every call, Meetzy runs all configured evaluations and stores the results.Example Evaluations
AI Copilot Suggestions
The AI Copilot can automatically suggest performance metrics and evaluation criteria based on your agent’s purpose and conversation patterns.
Automatic Evaluation Recommendations
The AI Copilot analyzes your agent configuration and suggests relevant evaluations:- Context-Aware Suggestions: Based on your agent’s role (sales, support, appointments)
- Conversation Analysis: Recommendations from analyzing test conversations
- Industry Best Practices: Standard metrics for your use case
- Performance Optimization: Evaluations that identify improvement opportunities
Using Copilot Suggestions
1
Open AI Copilot
Click the sparkles icon in the right panel to access AI Copilot.
2
Request Evaluation Suggestions
Type something like “Suggest evaluations for my agent” or use the quick actions.
3
Review Recommendations
The Copilot will analyze your agent and suggest specific evaluation criteria.
4
Accept or Customize
Accept suggestions directly or modify them to fit your specific needs.
Copilot Suggestion Types
- Goal-Based
- Quality Metrics
- Compliance Checks
- Data Collection
Evaluations focused on primary objectives:
- “Did the agent achieve the call objective?”
- “Was the primary goal accomplished?”
- “Did the conversation meet success criteria?”
Applying Copilot Suggestions
When the AI Copilot suggests evaluations, they appear in a purple suggestion banner:1
Review Suggestions
Examine the suggested evaluation name and question for relevance.
2
Customize as Needed
Modify the evaluation question to match your specific requirements.
3
Accept or Reject
Use the “Accept All” button to add all suggestions, or reject to dismiss them.
4
Fine-tune
Edit accepted evaluations in the main interface to perfect them.
Creating Evaluations
1
Open Evaluations Section
Navigate to the Evaluations section in the left panel (✅ icon).
2
Add Evaluation
Click “Add Evaluation” to create a new test.
3
Enter Name
Give your evaluation a descriptive name.
4
Write Question
Create a clear yes/no question that can be answered from the transcript.
5
Save
Save the evaluation to add it to your agent.
Evaluation Properties
Writing Effective Questions
The clearer your question, the more accurate the evaluation results will be.
Good Questions
- ✅ “Did the customer agree to schedule a follow-up call?”
- ✅ “Did the agent verify the customer’s contact information?”
- ✅ “Was the customer’s primary concern addressed during the call?”
- ✅ “Did the agent mention the current promotion?”
Poor Questions
- ❌ “Was the call good?” (too vague)
- ❌ “How satisfied was the customer?” (not yes/no)
- ❌ “Rate the agent’s performance” (not answerable from transcript)
Copilot-Enhanced Question Writing
The AI Copilot can help improve your evaluation questions:- Clarity Enhancement: Makes vague questions more specific
- Binary Conversion: Converts complex questions into yes/no format
- Context Addition: Adds relevant context for better evaluation accuracy
- Best Practice Alignment: Ensures questions follow evaluation best practices
Using Templates
Meetzy provides pre-built evaluation templates for common use cases:- Sales
- Support
- Appointments
- “Did the customer express interest in the product?”
- “Was pricing discussed during the call?”
- “Did the agent attempt to close the sale?”
- “Were all customer objections addressed?”
Applying Templates
1
Click Templates
Click the “Templates” button in the Evaluations section.
2
Browse Categories
Select a category relevant to your use case.
3
Select Evaluations
Choose which evaluations to add.
4
Customize
Modify the questions to match your specific needs.
Managing Evaluations
Editing Evaluations
- Click the edit icon (✏️) next to any evaluation
- Modify the name or question
- Save changes
Deleting Evaluations
- Click the delete icon (🗑️) next to the evaluation
- Confirm deletion
Reordering Evaluations
Drag and drop evaluations to change their display order.Viewing Results
Evaluation results are available in multiple places:Best Practices
Start with Copilot Suggestions
Begin by asking the AI Copilot to suggest evaluations. This provides a solid foundation based on your agent’s configuration and industry best practices.
Evaluation Quality Guidelines
- Be Specific: The more specific your question, the more reliable the evaluation
- Avoid Overlap: Don’t create multiple evaluations that measure the same thing
- Start Simple: Begin with 3-5 core evaluations that measure your most important KPIs
- Review Regularly: Periodically review evaluation results to calibrate questions
- Use for Training: Low-scoring evaluations highlight areas for agent improvement
Copilot Integration Tips
- Iterate with AI: Use the Copilot to refine and improve existing evaluations
- Context Matters: Provide context about your business when asking for suggestions
- Test Suggestions: Use the Playground to validate Copilot-suggested evaluations
- Combine Sources: Mix Copilot suggestions with templates and custom evaluations
Common Evaluation Patterns
Goal Achievement
Compliance
Quality
Data Collection
Next Steps
AI Copilot
Get AI-powered evaluation suggestions
Playground
Test evaluations in real-time
Output Settings
Include evaluations in webhooks
Analytics
View evaluation results and trends