INNER CODE UNIT · Python

model_id

cubist38/mlx-openai-server · app/server.py:401

                model_id = model_cfg.served_model_name  # guaranteed non-None after __post_init__

                # Serialize the dataclass config to a plain dict for
                # pickling across the spawn boundary.
                from dataclasses import asdict

                model_cfg_dict = asdict(model_cfg)

                queue_config = {
                    "timeout": model_cfg.queue_timeout,
                    "queue_size": model_cfg.queue_size,
                }

                if model_cfg.on_demand:
                    # Register on-demand model without spawning a subprocess.
                    # It will be loaded dynamically when a request arrives.
                    await registry.register_on_demand_model(
                        model_id=model_id,

View source record →