Skip to content

请问是否可以提供转成onnx的相关指导文档，谢谢 #259

New issue
New issue
Open
Open
请问是否可以提供转成onnx的相关指导文档，谢谢#259
@IeohMingChan

Description

@IeohMingChan
IeohMingChan
opened on Nov 7, 2024
          您好，由于Reranker采用的是双向注意力，无kv cache机制，因此使用vllm部署并不会有较大的提升。您可以尝试转成onnx

Originally posted by @Kaguya-19 in #258 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions