INNER CODE UNIT · Python
get_model_info
ai-infra-curriculum/ai-infra-engineer-learning · projects/project-103-llm-deployment/src/llm/server.py:275
def get_model_info(self) -> Dict[str, Any]:
"""
Get information about the loaded model.
Returns:
Dictionary with model metadata
TODO: Return information about:
1. Model name and architecture
2. Number of parameters
3. Quantization method
4. GPU memory usage
5. Maximum sequence length
6. Tensor parallel size
"""
# TODO: Gather model information
# Can use torch.cuda to get GPU memory stats
return {