Datasets

The Hub provides centralized access to shared resources that support evaluation and testing workflows. One of the key components within the Hub is Datasets, which allows users to create and manage custom datasets for model evaluation.

In addition to built-in test categories and metrics, Pacific AI allows users to define their own evaluation datasets. These datasets can contain custom questions tailored to specific use cases, domains, or internal requirements.

To access datasets, navigate to Hub → Datasets.

---

Create a Dataset

Users can create a new dataset by clicking the Create button on the top-right of the Datasets page.

When creating a dataset, provide:

• Name – A unique dataset identifier
• Description – A brief summary of the dataset purpose, which is optional
• Metrics – Evaluation Metrics to be applied
• Category and Subcategory – Category and Subcategory used to organize datasets

Users can either:

• Select from existing categories
• Define new categories based on their requirements

---

Dataset Visibility (Private Datasets)

Datasets can be restricted to specific systems using the Private option.

• Enable Private to limit access
• Assign one or more systems
• Only selected systems will be able to use the dataset for evaluation

This is useful for system-specific testing or sensitive evaluation scenarios.

---

Manage Datasets

From the Datasets page, users can:

• View all available datasets
• Filter and sort datasets based on different criteria
• Edit dataset details
• Clone existing datasets to reuse configurations
• Delete datasets when no longer needed

All management actions are available through the kebab menu (three-dot menu) for each dataset.

---

Manage Questions

Each dataset consists of a set of evaluation questions.

To manage questions:

1. Click on the dataset
2. Navigate to the Questions section

Users can:

• Add questions manually
• Import questions using a CSV file
• Edit existing questions
• Delete questions
• Append additional questions to an existing dataset

This flexibility allows datasets to evolve over time as evaluation needs change.

Custom datasets enable organizations to design evaluations that align with their specific domain, policies, and performance expectations, beyond standard testing benchmarks.