AI RESEARCH
SoftJAX & SoftTorch: Empowering Automatic Differentiation Libraries with Informative Gradients
arXiv CS.LG
•
ArXi:2603.08824v1 Announce Type: new Automatic differentiation (AD) frameworks such as JAX and PyTorch have enabled gradient-based optimization for a wide range of scientific fields. Yet, many "hard" primitives in these libraries such as thresholding, Boolean logic, discrete indexing, and sorting operations yield zero or undefined gradients that are not useful for optimization. While numerous "soft" relaxations have been proposed that provide informative gradients, the respective implementations are fragmented across projects, making them difficult to combine and compare. This work.