Hello,
Three raters give grades to each video. How is the ground truth defined ? Average, or something else ?
Created by SKJP Hi,
the evaluation metric you provided assumes classes (as discreate integers)
in a case of using an average for ground truth , the average may be a float
how does your code deal with this case (example, for 3 gradings of 4,5,5 : np.round(4.67) = 5 **but** int(4.67) = 4)?
this can change the ground truth and hurt the model quality Hi, essentially it's up to you. The evaluation scripts will use an average for the ground truth.