o )i@sddlmZddlZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z er6ddlmZdd Zd$ddZd%ddZd&ddZGddde ZGdd d e Zd'd"d#ZdS)() annotationsN) TYPE_CHECKING)OpenAI)ChatCompletionMessageParam) CLISubcommand)FlexibleArgumentParsercCs(dd}ttj|ttj|dS)NcSstddS)Nr)sysexit)sigframer g/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/vllm/entrypoints/cli/openai.pysignal_handlersz1_register_signal_handlers..signal_handler)signalSIGINTSIGTSTP)rr r r _register_signal_handlerssrargsargparse.Namespacereturntuple[str, OpenAI]cCsft|j}|jptjdd}t||d}|jr|j}n |j }|j dj }t d|||fS)NZOPENAI_API_KEYZEMPTY)api_keybase_urlrz Using model: ) rurlrosenvirongetr model_namemodelslistdataidprint)rrrZ openai_clientrZavailable_modelsr r r _interactive_clis   r# system_prompt str | NonerstrclientrNonecCsg}|dur|d|dtd ztd}Wn ty#YdSw|d|d|jjj||d}|jdj}|j }||t|q) NsystemZrolecontent*Please enter a message for the chat model:T> usermodelmessagesr) appendr"inputEOFErrorchat completionscreatechoicesmessager+)r$rr' conversation input_messagechat_completionresponse_messageoutputr r r r50s&     r5parserrcCs:|jdtddd|jdtddd|jdtddd|S) Nz--urlzhttp://localhost:8000/v1z7url of the running OpenAI-Compatible RESTful API servertypedefaulthelpz --model-namez]The model name used in prompt completion, default to the first model in list models API call.z --api-keyzyAPI key for OpenAI services. If provided, this api key will overwrite the api key obtained through environment variables.) add_argumentr&r?r r r _add_query_optionsGs&rFc@:eZdZdZdZedddZedd d ZdddZdS) ChatCommandz(The `chat` subcommand for the vLLM CLI. r5rrrr(c Cst|\}}|j}g}|dur|d|d|jr7|d|jd|jjj||d}t|jdj j dStd zt d}Wn t yLYdSw|d|d|jjj||d}|jdj }|j }||t|q<) Nr)r*r.r/rr,Tr-) r#r$r2quickr5r6r7r"r8r9r+r3r4) rrr'r$r:r<r;r=r>r r r cmdcs8     zChatCommand.cmdr?rcCs2t||jdtddd|jddtddd |S) z'Add CLI arguments for the chat command.z--system-promptNz`The system prompt to be added to the chat template, used for models that support system prompts.r@-q--quickZMESSAGEzBSend a single prompt as MESSAGE and print the response, then exit.rAmetavarrCrFrDr&rEr r r add_cli_argsszChatCommand.add_cli_args subparsersargparse._SubParsersActioncC|jddddd}t|S)Nr5z5Generate chat completions via the running API server.zvllm chat [options]rC descriptionusage) add_parserrHrPselfrQr?r r r subparser_inits zChatCommand.subparser_initNrrrr(r?rrrrQrRrr __name__ __module__ __qualname____doc__name staticmethodrJrPrZr r r r rH_s ! rHc@rG)CompleteCommandz,The `complete` subcommand for the vLLM CLI. completerrrr(cCst|\}}|jr|jj||jd}t|jdjdStd ztd}Wn ty1YdSw|jj||d}|jdj}t|q!)N)r0promptrz Please enter prompt to complete:Tr-) r#rIr6r7r"r8textr3r4)rrr' completionZ input_promptr>r r r rJs(    zCompleteCommand.cmdr?rcCs t||jddtddd|S)z+Add CLI arguments for the complete command.rKrLZPROMPTz@Send a single prompt and print the completion output, then exit.rMrOrEr r r rPszCompleteCommand.add_cli_argsrQrRcCrS)NrfzOGenerate text completions based on the given prompt via the running API server.zvllm complete [options]rT)rWrerPrXr r r rZs zCompleteCommand.subparser_initNr[r\r]r^r r r r res   relist[CLISubcommand]cCs ttgS)N)rHrer r r r cmd_inits rk)rrrr)r$r%rr&r'rrr(r\)rrj) __future__rargparserrrtypingrZopenairZopenai.types.chatrZvllm.entrypoints.cli.typesrZ vllm.utilsrrr#r5rFrHrerkr r r r s$        C3