Evaluations

This guide provides step-by-step instructions for configuring evaluation parameters in your assistant settings. It is designed for users managing evaluations in environments leveraging LLMs. Prerequisites include access to assistant settings and familiarity with evaluation processes.

Main Content

1. Enabling Daily Evaluations

1. Navigate to the Evaluation Settings section.
2. Toggle Enable Daily Evaluations to the ON position.
3. Specify the time for the daily evaluations:
   • Hour: Choose the hour (0–23) for the evaluation to run.
   • Minute: Choose the minute (0–59) for the evaluation to run.

2. Setting a Maximum Questions Threshold

1. Use the Threshold setting to cap the number of questions included in evaluations.
   • This feature helps manage token usage and reduce costs associated with LLM operations.

3. Excluding Specific Email Domains

1. Specify email domains to exclude from evaluations:
   • For example, enter domains for your company or consultants whose activity should not be part of regular evaluations.

4. Persistence of Settings

• If Daily Evaluations are toggled off, all configured settings (e.g., time, thresholds, excluded domains) are retained.
• This ensures that when you re-enable the feature, your previous configuration remains intact.

Examples

• Scenario 1: Setting Daily Evaluations
  A user enables daily evaluations to run at 2:30 AM with a threshold of 50 questions. They exclude internal company domains (e.g., @company.com) to focus solely on customer interactions.

• Scenario 2: Temporarily Disabling Evaluations
  A user disables evaluations for a maintenance period but retains their configured settings for easy reactivation later.

Additional Notes or Best Practices

• Review and adjust thresholds periodically to optimize token usage.
• Regularly update excluded domains to reflect organizational changes or new consultant partnerships.

Conclusion

By configuring evaluation parameters, you can customize daily evaluations to align with your goals, conserve resources, and maintain precise control over the evaluation scope. Revisit these settings periodically to ensure they remain aligned with your requirements.