Wow, that's illuminating. I naively thought the overhead of converting between datatypes would not make this worth that much (in favor of saving cache misses). Though does this also have anything to do with the AVX512 instructions?
The result is that Winograd convolutions can achieve an effective FMA rate of twice the peak rate of the CPU.
The Winograd transform reduces the required number of FMAs by a factor of 5x, but you can only do FMAs at half peak rate (because you are bandwidth limited), so you come out ahead by a factor of 2.5x in theory (2x in practice).
37ef_ced3|4 years ago
https://software.intel.com/sites/landingpage/IntrinsicsGuide...
The result is that Winograd convolutions can achieve an effective FMA rate of twice the peak rate of the CPU.
The Winograd transform reduces the required number of FMAs by a factor of 5x, but you can only do FMAs at half peak rate (because you are bandwidth limited), so you come out ahead by a factor of 2.5x in theory (2x in practice).
Without fp16, that 2x advantage would be lost.