INNER CODE UNIT · Python

prefill_seconds

invergent-ai/surogate · surogate/serve/tools/reference/qwen3_5/cli.py:371

            prefill_seconds = time.perf_counter() - prefill_started

            decode_started = time.perf_counter()
            generated = model.continue_generation(
                token,
                args.decode,
                stop_token_ids=stops,
                sampler=sampler,
                tap=tap,
            )
            _synchronize(model.device)
            decode_seconds = time.perf_counter() - decode_started

        generated_text = frontend.decode(generated, skip_special_tokens=True)
        stop_reason = (
            "disabled"
            if args.decode == 0
            else "stop_token"

View source record →