AI RESEARCH
INT8 quantization gives me better accuracy than FP16 ! [D]
r/MachineLearning
•
Hi everyone, I’m working on a deep learning model and I noticed something strange. When I compare different precisions: FP32 (baseline) FP16, INT8 (post-