INNER CODE UNIT · Python

_performance_on_evaluation_data

dragondrop-cloud/cloud-concierge · nlpengine/main.py:226

def _performance_on_evaluation_data(
    textcat_multilabel_model,
    evaluation_data: List[Example],
) -> dict:
    """
    Assessing performance on a held-out evaluation set of data. Works by making predictions
    with the current model on the evaluation data set and then calculating
    the f1, recall, and precision score.
    """
    actual_list_of_dicts = []

    prediction_list_of_dicts = []

    for eval_example in evaluation_data:
        eval_example_dict = deepcopy(eval_example.to_dict())

        actual_list_of_dicts.append(eval_example_dict["doc_annotation"]["cats"])

View source record →