AI RESEARCH
Power-Softmax: Towards Secure LLM Inference over Encrypted Data
arXiv CS.LG
•
ArXi:2410.09457v2 Announce Type: replace Modern cryptographic methods for implementing privacy-preserving LLMs such as \gls{HE} require the LLMs to have a polynomial form. Forming such a representation is challenging because transformers include non-polynomial components, such as \Softmax and layer normalization. Previous approaches have either directly approximated pre-trained models with large-degree polynomials, which are less efficient over HE, or replaced non-polynomial components with easier-to-approximate primitives before.