INNER CODE UNIT · Python

num_tokens

lucidrains/x-transformers · train_discover_reversal.py:73

    num_tokens = num_fillers + 2

    device = default_device()
    torch.manual_seed(seed)

    decoder = TransformerWrapper(
        num_tokens = num_tokens,
        max_seq_len = 16,
        emb_dropout = 0.05,
        attn_layers = Decoder(
            dim = dim,
            depth = depth,
            heads = heads,
            attn_dropout = 0.05,
            ff_dropout = 0.05,
            ff_mult = 4
        )
    ).to(device)

View source record →