INNER CODE UNIT · Python
generate_text_stream
ai-infra-curriculum/ai-infra-engineer-learning · projects/project-103-llm-deployment/src/api/main.py:328
async def generate_text_stream(
request: GenerateRequest,
api_key: str = Depends(verify_api_key),
llm: LLMServer = Depends(get_llm_server)
):
"""
Generate text with streaming response.
Args:
request: Generation request
api_key: API key
llm: LLM server
Returns:
Server-Sent Events stream
TODO: Implement streaming generation:
1. Validate request