Open the comparison view
- To access the experiment comaprison view, navigate to the Datasets & Experiments page.
- Select a dataset, which will open the Experiments tab.
- Select two or more experiments abd then click Compare.
Adjust the table display
You can toggle between different views by clicking Full or Compact at the top of the Comparing Experiments page. Toggling Full will show the full text of the input, output, and reference output for each run. If the reference output is too long to display in the table, you can click on Expand detailed view to view the full content. You can also select and hide individual feedback keys or individual metrics in the Display settings dropdown to isolate the information you need in the comparison view.View regressions and improvements
In the comparison view, runs that regressed on your specified feedback key against your baseline experiment will be highlighted in red, while runs that improved will be highlighted in green. At the top of each column, you can find how many runs in that experiment did better and how many did worse than your baseline experiment. Click on the regressions or improvements buttons on the top of each column to filter to the runs that regressed or improved in that specific experiment.Update baseline experiment and metric
In order to track regressions, you need to:- In the Baseline dropdown at the top of the comparison view, select a Baseline experiment against which to compare. By default, the newest experiment is selected as the baseline.
- Select a Feedback key (evaluation metric) you want to focus compare against. One will be assigned by default, but you can adjust as needed.
- Configure whether a higher score is better for the selected feedback key. This preference will be stored.