INNER CODE UNIT · Python

scheduler_post

QizhiPei/FABind · FABind/fabind/main_fabind.py:279

    scheduler_post = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=(args.total_epochs - args.warmup_epochs)*len(train_loader), eta_min=1e-5, last_epoch=last_epoch)
elif args.lr_scheduler == "cosine_decay_restart":
    scheduler_post = torch.optim.lr_scheduler.CosineAnnealingWarmRestarts(optimizer, T_0=10, eta_min=0.0001, last_epoch=last_epoch)

scheduler = torch.optim.lr_scheduler.SequentialLR(
    optimizer,
    schedulers=[scheduler_warm_up, scheduler_post],
    milestones=[args.warmup_epochs*len(train_loader)],
)

(
    model,
    optimizer,
    scheduler,
    train_loader,
) = accelerator.prepare(
    model, optimizer, scheduler, train_loader,
)

View source record →