0

我正在使用 allennlp 2.1，我想将类权重传递给我使用的 pytorch-cross-entropy 损失函数。

@Head.register('model_head_two_layers')
class ModelHeadTwoLayers(Head):

    default_predictor = 'head_predictor'

    def __init__(self, vocab: Vocabulary, input_dim: int, output_dim: int, dropout: float = 0.0,
                 class_weights: Union[List[float], None] = None):
        super().__init__(vocab=vocab)
        self.input_dim = input_dim
        self.output_dim = output_dim
        self.layers = torch.nn.Sequential(
            torch.nn.Dropout(dropout),
            torch.nn.Linear(self.input_dim, self.input_dim),
            torch.nn.ReLU(inplace=True),
            torch.nn.Linear(self.input_dim, output_dim)
        )
        self.metrics = {
            'accuracy': CategoricalAccuracy(),
            'f1_macro': FBetaMeasure(average='macro')
        }
        if class_weights:
            self.class_weights = torch.FloatTensor(class_weights)
            self.cross_ent = torch.nn.CrossEntropyLoss(weight=self.class_weights)
        else:
            self.cross_ent = torch.nn.CrossEntropyLoss()

在配置文件中，我按如下方式传递类权重：

"heads": {
            "task_name": {
                "type": "model_head_two_layers",
                "input_dim": embedding_dim,
                "output_dim": 4,
                "dropout": dropout,
                "class_weights": [0.25, 0.90, 0.91, 0.94]
            }
        }

为了使类权重的顺序正确，我需要知道输出张量的哪个索引对应于哪个类。到目前为止，我知道的唯一方法是首先训练一个没有类权重的模型，然后进入模型的词汇目录并检查类名写入标签文件的顺序。

虽然这似乎可行......有没有更简单的方法来获得该映射而无需先训练模型？

4

1 回答 1

1

allennlp build-vocab您可以使用该命令在不训练模型的情况下生成词汇表。但我认为这里更好的解决方案是将 传递class_weights给您的模型作为来自 的映射label -> weight，然后使用该__init__函数构建权重数组。像这样的东西：

class ModelHeadTwoLayers(Head):
    def __init__(
        self,
        vocab: Vocabulary,
        input_dim: int,
        output_dim: int,
        dropout: float = 0.0,
        class_weights: Optional[Dict[str, float]] = None,
        label_namespace: str = "labels",
    ):
        super().__init__(vocab=vocab)

        # ...

        if class_weights:
            weights: List[float] = [0.0] * len(class_weights)
            for label, weight in class_weights.items():
                label_idx = self.vocab.get_token_index(label, namespace=label_namespace)
                weights[label_idx] = weight
            self.class_weights = torch.FloatTensor(weights)
            self.cross_ent = torch.nn.CrossEntropyLoss(weight=self.class_weights)
        else:
            self.cross_ent = torch.nn.CrossEntropyLoss()
于 2021-03-19T16:51:25.160 回答