Which tokens to use? investigating token reduction in vision transformers

JB Haurum, S Escalera, GW Taylor… - Proceedings of the …, 2023 - openaccess.thecvf.com
Since the introduction of the Vision Transformer (ViT), researchers have sought to make ViTs
more efficient by removing redundant information in the processed tokens. While different …

Crack segmentation on steel structures using boundary guidance model

Z He, W Chen, J Zhang, YH Wang - Automation in Construction, 2024 - Elsevier
Cracks are an essential indicator of infrastructure degradation, and achieving high-
precision, pixel-level crack segmentation is a common goal for artificial intelligence (AI) …

PipeTransUNet: CNN and Transformer fusion network for semantic segmentation and severity quantification of multiple sewer pipe defects

M Li, M Li, Q Ren, H Li, L **ao, X Fang - Applied Soft Computing, 2024 - Elsevier
With the continuous development of urbanization, the service life of sewer pipes is gradually
approaching a critical threshold. Defects within pipe networks can significantly affect the …

Global-local attention-based butterfly vision transformer for visualization-based malware classification

MM Belal, DM Sundaram - IEEE Access, 2023 - ieeexplore.ieee.org
In recent studies, convolutional neural networks (CNNs) are mostly used as dynamic
techniques for visualization-based malware classification and detection. Though vision …

[HTML][HTML] Deep learning for automated encrustation detection in sewer inspection

W Yusuf, H Alaka, M Ahmad, W Godoyon… - Intelligent Systems with …, 2024 - Elsevier
Rapid urbanization and population growth in recent decades have placed significant
pressure on urban cities to rely heavily on underground infrastructure, such as sewers and …

A comparative study of vision transformers and convolutional neural networks: sugarcane leaf diseases identification

S Öğrekçi, Y Ünal, MN Dudak - European Food Research and Technology, 2023 - Springer
Diseases in agricultural products cause significant decrease on harvest efficiency and
economic values of the products, early detection of diseases can prevent this loss. The …

Agglomerative Token Clustering

JB Haurum, S Escalera, GW Taylor… - European Conference on …, 2024 - Springer
Abstract We present Agglomerative Token Clustering (ATC), a novel token merging method
that consistently outperforms previous token merging and pruning methods across image …

Real-time defect detection in underground sewage pipelines using an improved YOLOv5 model

J Lu, W Song, Y Zhang, X Yin, S Zhao - Automation in Construction, 2025 - Elsevier
Sewer systems are critical to smart city infrastructure, but conventional pipeline inspection
methods cause high costs and inefficiency. This paper presents a real-time detection …

PRANCE: Joint Token-Optimization and Structural Channel-Pruning for Adaptive ViT Inference

Y Li, C Tang, Y Meng, J Fan, Z Chai, X Ma… - arxiv preprint arxiv …, 2024 - arxiv.org
We introduce PRANCE, a Vision Transformer compression framework that jointly optimizes
the activated channels and reduces tokens, based on the characteristics of inputs …

Infrastructure crack segmentation: Boundary guidance method and benchmark dataset

Z He, W Chen, J Zhang, YH Wang - arxiv preprint arxiv:2306.09196, 2023 - arxiv.org
Cracks provide an essential indicator of infrastructure performance degradation, and
achieving high-precision pixel-level crack segmentation is an issue of concern. Unlike the …