INNER CODE UNIT · Python

LLMServer

ai-infra-curriculum/ai-infra-engineer-learning · projects/project-103-llm-deployment/src/llm/server.py:42

class LLMServer:
    """
    High-level wrapper for LLM inference using vLLM.

    This class provides a unified interface for:
    - Model loading and initialization
    - Synchronous and asynchronous inference
    - Streaming text generation
    - Batch processing
    - GPU resource management

    Attributes:
        config: LLM configuration object
        engine: vLLM async engine instance
        tokenizer: Hugging Face tokenizer
        optimizer: Model optimization helper
    """

View source record →