DeepSeek-V4-Flash-w8a8-mtp功能文本生成部署方式在8卡asend 910b使用vllm部署模型下载pipinstallmodelscopemkdirDeepSeek-V4-Flash-w8a8-mtp modelscope download--modelEco-Tech/DeepSeek-V4-Flash-w8a8-mtp--local_dirDeepSeek-V4-Flash-w8a8-mtp镜像quay.io/ascend/vllm-ascend:deepseekv4模型部署model_pathDeepSeek-V4-Flash-w8a8-mtpport${2:-8000}echomodel_path:$model_pathechoport:$portexportLD_PRELOAD/usr/lib/aarch64-linux-gnu/libjemalloc.so.2:$LD_PRELOADexportOMP_PROC_BINDfalseexportOMP_NUM_THREADS8exportPYTORCH_NPU_ALLOC_CONFexpandable_segments:TrueexportACL_OP_INIT_MODE1exportVLLM_ASCEND_ENABLE_FLASHCOMM11exportUSE_MULTI_GROUPS_KV_CACHE1exportTASK_QUEUE_ENABLE1exportHCCL_OP_EXPANSION_MODEAIVexportHCCL_BUFFSIZE512exportUSE_MULTI_BLOCK_POOL1sysctl-wvm.swappiness0sysctl-wkernel.numa_balancing0sysctlkernel.sched_migration_cost_ns50000vllm serve${model_path}\--host0.0.0.0\--max_model_len131072\--max-num-batched-tokens4096\--served-model-name DeepSeek-V4-Flash\--gpu-memory-utilization0.92\--data-parallel-size1\--tensor-parallel-size8\--max-num-seqs16\--enable-expert-parallel\--quantizationascend\--port${port}\--block-size128\--async-scheduling\--additional-config{enable_cpu_binding: true, multistream_overlap_shared_expert: false}\--model-loader-extra-config{enable_multithread_load:true,num_threads:16}\--speculative-config{num_speculative_tokens: 1,method: mtp}\--compilation-config{cudagraph_mode:FULL_DECODE_ONLY,cudagraph_capture_sizes:[2,4,6,8,10,12,14,16,18,20,22,24,32,36,40]}\--tokenizer-mode deepseek_v4\--tool-call-parser deepseek_v4\--enable-auto-tool-choice\--reasoning-parser deepseek_v4模型请求curl命令请求curlhttp://localhost:8000/v1/chat/completions\-HContent-Type: application/json\-d{ model: DeepSeek-V4-Flash, messages: [ {role: user, content: Please generating one chinese poetry!} ], max_tokens: 8192, temperature: 1.0, top_p: 0.95, presence_penalty: 1.5, extra_body: { top_k: 20 } }python请求方式fromopenaiimportOpenAI clientOpenAI(api_keyEMPTY,base_urlhttp://localhost:8000/v1,timeout3600)messages[{role:user,content:Please generating one chinese poetry!},]chat_responseclient.chat.completions.create(modelDeepSeek-V4-Flash,messagesmessages,max_tokens8192,temperature1.0,top_p0.95,presence_penalty1.5,extra_body{top_k:20,},)print(Chat response:,chat_response)性能测试vllm bench serve --base-url http://localhost:8000/v1/chat/completions\--backendvllm\--modelDeepSeek-V4-Flash\--tokenizerDeepSeek-V4-Flash-w8a8-mtp\--dataset-name random\--input-len1024\--output-len1024\--max-concurrency1\--num-prompts10\--save-result\--result-dir /path/to/your_save_dir\--result-filename ds-v4-flash_input1024_output1024_request1_num10.json 【LLM】DeepSeek-V4-Flash-w8a8-mtp服务化部署、请求和性能测试
DeepSeek-V4-Flash-w8a8-mtp功能文本生成部署方式在8卡asend 910b使用vllm部署模型下载pipinstallmodelscopemkdirDeepSeek-V4-Flash-w8a8-mtp modelscope download--modelEco-Tech/DeepSeek-V4-Flash-w8a8-mtp--local_dirDeepSeek-V4-Flash-w8a8-mtp镜像quay.io/ascend/vllm-ascend:deepseekv4模型部署model_pathDeepSeek-V4-Flash-w8a8-mtpport${2:-8000}echomodel_path:$model_pathechoport:$portexportLD_PRELOAD/usr/lib/aarch64-linux-gnu/libjemalloc.so.2:$LD_PRELOADexportOMP_PROC_BINDfalseexportOMP_NUM_THREADS8exportPYTORCH_NPU_ALLOC_CONFexpandable_segments:TrueexportACL_OP_INIT_MODE1exportVLLM_ASCEND_ENABLE_FLASHCOMM11exportUSE_MULTI_GROUPS_KV_CACHE1exportTASK_QUEUE_ENABLE1exportHCCL_OP_EXPANSION_MODEAIVexportHCCL_BUFFSIZE512exportUSE_MULTI_BLOCK_POOL1sysctl-wvm.swappiness0sysctl-wkernel.numa_balancing0sysctlkernel.sched_migration_cost_ns50000vllm serve${model_path}\--host0.0.0.0\--max_model_len131072\--max-num-batched-tokens4096\--served-model-name DeepSeek-V4-Flash\--gpu-memory-utilization0.92\--data-parallel-size1\--tensor-parallel-size8\--max-num-seqs16\--enable-expert-parallel\--quantizationascend\--port${port}\--block-size128\--async-scheduling\--additional-config{enable_cpu_binding: true, multistream_overlap_shared_expert: false}\--model-loader-extra-config{enable_multithread_load:true,num_threads:16}\--speculative-config{num_speculative_tokens: 1,method: mtp}\--compilation-config{cudagraph_mode:FULL_DECODE_ONLY,cudagraph_capture_sizes:[2,4,6,8,10,12,14,16,18,20,22,24,32,36,40]}\--tokenizer-mode deepseek_v4\--tool-call-parser deepseek_v4\--enable-auto-tool-choice\--reasoning-parser deepseek_v4模型请求curl命令请求curlhttp://localhost:8000/v1/chat/completions\-HContent-Type: application/json\-d{ model: DeepSeek-V4-Flash, messages: [ {role: user, content: Please generating one chinese poetry!} ], max_tokens: 8192, temperature: 1.0, top_p: 0.95, presence_penalty: 1.5, extra_body: { top_k: 20 } }python请求方式fromopenaiimportOpenAI clientOpenAI(api_keyEMPTY,base_urlhttp://localhost:8000/v1,timeout3600)messages[{role:user,content:Please generating one chinese poetry!},]chat_responseclient.chat.completions.create(modelDeepSeek-V4-Flash,messagesmessages,max_tokens8192,temperature1.0,top_p0.95,presence_penalty1.5,extra_body{top_k:20,},)print(Chat response:,chat_response)性能测试vllm bench serve --base-url http://localhost:8000/v1/chat/completions\--backendvllm\--modelDeepSeek-V4-Flash\--tokenizerDeepSeek-V4-Flash-w8a8-mtp\--dataset-name random\--input-len1024\--output-len1024\--max-concurrency1\--num-prompts10\--save-result\--result-dir /path/to/your_save_dir\--result-filename ds-v4-flash_input1024_output1024_request1_num10.json