HiAP simplifies Vision Transformer deployment by automatically discovering efficient architectures in one training phase without manual sparsity targets, matching complex multi-stage methods while being easier to use.
HiAP is a pruning method that automatically removes unnecessary parts of Vision Transformers during training to make them faster and smaller for edge devices. Unlike existing approaches that require manual tuning, it uses a single training process to find optimal sub-networks by removing entire attention heads, FFN blocks, and individual neurons simultaneously.