Skip to content

New evaluation listing call that includes hyperparameters #737

@joaquinvanschoren

Description

@joaquinvanschoren

Currently, if one wants to download all hyperparameters and evaluations of a specific flow, they need to first download the list of evaluations (fast), and then download runs one by one to get the hyperparameters (slow).

This is a problem for the current Frontend, plus it also complicates downloading meta-data from the server (last time I tried this it worked but took a very long time).

Since this is such a common use case, should we add an API call for that? Basically something
returning something like:

{
  "evaluations": {
    "evaluation": [
      {
        "run_id": "1",
        "task_id": "68",
        "flow_id": "61",
        "function": "area_under_roc_curve",
        "upload_time": "2014-04-06 23:30:40",
        "value": "0.839359",
        "array_data": "[0,0.99113,0.898048,0.874862,0.791282,0.807343,0.820674]",
        "setup_parameters": "parameter": [
            {
              "full_name": "weka.JRip(1)_F",
              "parameter_name": "F",
              "value": "3"
            },
           {
             "full_name": "weka.JRip(1)_N",
             "parameter_name": "N",
             "value": "2.0"
           },
           ...]
      }

Different options:

  • Add it as an option return_setups=true to the current evaluation list call. It would only work if there is a specific flow and function, otherwise return an error.
  • Add it as a new call `list_evaluations_and_setup(flow_id,function)
  • Use the current setup/id call, but that has the same issue that you need to ask for the setups one by one, plus the evaluation list does not return the setup id.

Thoughts?

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions