Knowledge Fusion Distillation: Improving Distillation with Multi-scale Attention Mechanisms
{{output}}
The success of deep learning has brought breakthroughs in many fields. However, the increased performance of deep learning models is often accompanied by an increase in their depth and width, which conflicts with the storage, energy consumption, and computatio... ...