Skip to content

Incorrect loss normalization in semi-supervised training: Cross-entropy should use labeled_minibatch_size #64

@shamss778

Description

@shamss778

I think that the current implementation normalizes both classification and consistency losses by minibatch_size instead of using the appropriate denominators for each loss component. (unless I didnt iiunderstand the paper correctly)

Current code:
class_loss = class_criterion(class_logit, target_var) / minibatch_size

But isnt Cross-entropy losses supposed to be normalized by labeled_minibatch_size (samples that contribute to supervised learning).

class_loss = class_criterion(class_logit, target_var) / labeled_minibatch_size

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions