cesure

[ SEARCH ]

find a paper

Press Enter to search 200M+ papers via Semantic Scholar. Want something deeper? Run a Deep Discovery.

2,535,826 RESULTS

2025Real-time driver drowsiness detection using transformer architectures: a novel deep learning approachOsama F. Hassan, Ahmed F. Ibrahim, A. Gomaa et al. · Scientific ReportsA novel and robust deep learning-based framework for real-time driver drowsiness detection, leveraging state-of-the-art transformer architectures and transfer learning models to achieve unprecedented accuracy and reliability, and incorporating a real-time drowsiness scoring mechanism.2025Analysis of mean-field models arising from self-attention dynamics in transformer architectures with layer normalizationMartin Burger, Samira Kabri, Yury Korolev et al. · Philosophical transactions. Series A, Mathematical, physical, and engineering sciencesA rigorous framework for studying the gradient flow is provided and a possible metric geometry is suggested to study the general case (i.e. one that is not described by a gradient flow) and the stationary points of the induced self-attention dynamics are analysed.2025Automated non-PPE detection on construction sites using YOLOv10 and transformer architectures for surveillance and body worn cameras with benchmark datasetsSeunghyeon Wang · Scientific Reports2024Systematic Review of Hybrid Vision Transformer Architectures for Radiological Image AnalysisJi Woong Kim, Aisha Urooj Khan, Imon Banerjee · Journal of Imaging Informatics in Medicine2024Hybrid Transformer Architectures With Diverse Audio Features for Deepfake Speech ClassificationKhalid Zaman, Islam J A M Samiul, Melike Šah et al. · IEEE AccessThe results of the evaluation with instances of real and fake speech using the ASVspoof LA dataset with hybrid transformer models across various audio features indicate that the STFT feature performs best with the ResNet34-Transformer model, achieving a state-of-the-art performance with a development set equal error rate (EER) of 0.8%.2024A Historical Survey of Advances in Transformer ArchitecturesAli Reza Sajun, Imran A. Zualkernan, D. Sankalpa · Applied SciencesA survey of key works related to the early development and implementation of transformer models in various domains such as generative deep learning and as backbones of large language models is presented.2024IQA Vision Transformed: A Survey of Transformer Architectures in Perceptual Image Quality AssessmentM. Rehman, I. Nizami, Farman Ullah et al. · IEEE AccessThis paper reviews the integration of Vision Transformers (ViTs) into both no-reference (NR) and full-reference (FR) IQA methods, highlighting their promise as alternatives to traditional techniques, with a specific focus on ViTs.2024Training transformer architectures on few annotated data: an application to historical handwritten text recognitionKillian Barrere, Yann Soullard, Aurélie Lemaitre et al. · International Journal on Document Analysis and RecognitionThis paper proposes the use of a lightweight Transformer model to tackle the task of historical handwritten text recognition and presents a specific strategy, both for training and prediction, to deal with historical documents, where only a limited amount of training data are available.2024AI support for colonoscopy quality control using CNN and transformer architecturesJian Chen, Ganhong Wang, Jingjie Zhou et al. · BMC GastroenterologyThe AI-assisted quality system, based on the EfficientNetB2 model, integrates four key quality control indicators for colonoscopy to comprehensively manage and enhance these indicators using a single model, showcasing promising potential for clinical applications.2024Design tactics for tailoring transformer architectures to cybersecurity challengesÇigdem Avci Salma, B. Tekinerdogan, C. Catal · Cluster ComputingThis study unveils the design decisions and strategies crucial for successful implementation in diverse cybersecurity domains and emphasizes the significance of aligning design tactics with the unique business requirements and quality factors of each specific application domain.2021Masked-attention Mask Transformer for Universal Image SegmentationBowen Cheng, Ishan Misra, A. Schwing et al. · Computer Vision and Pattern RecognitionMask2Former is presented, a new archi-tecture capable of addressing any image segmentation task (panoptic, instance or semantic), and sets a new state-of-the-art for panoptic segmentation, instance segmentation and semantic segmentation.2021Restormer: Efficient Transformer for High-Resolution Image RestorationSyed Waqas Zamir, Aditya Arora, Salman Hameed Khan et al. · Computer Vision and Pattern RecognitionThis work proposes an efficient Transformer model by making several key designs in the building blocks (multi-head attention and feed-forward network) such that it can capture long-range pixel interactions, while still remaining applicable to large images.2023I3D: Transformer Architectures with Input-Dependent Dynamic Depth for Speech RecognitionYifan Peng, Jaesong Lee, Shinji Watanabe · IEEE International Conference on Acoustics, Speech, and Signal ProcessingA novel Transformer encoder with Input-Dependent Dynamic Depth (I3D) to achieve strong performance-efficiency trade-offs and interesting analysis on the gate probabilities and the input-dependency, which helps to better understand deep encoders.2022Spatio-Temporal Wind Speed Forecasting using Graph Networks and Novel Transformer ArchitecturesL. Ø. Bentsen, N. Warakagoda, Roy Stenbro et al. · Applied EnergyThe Fast Fourier Transformer (FFTransformer), which is a novel Transformer architecture based on signal decomposition and consists of two separate streams that analyse the trend and periodic components separately, is proposed.2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text TransformerColin Raffel, Noam Shazeer, Adam Roberts et al. · Journal of machine learning researchThis systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks and achieves state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more.2023Learned Image Compression with Mixed Transformer-CNN ArchitecturesJinming Liu, Heming Sun, J. Katto · Computer Vision and Pattern RecognitionThis paper proposes an efficient parallel Transformer-CNN Mixture (TCM) block with a controllable complexity to incorporate the local modeling ability of CNN and the non-local modelingAbility of transformers to improve the overall architecture of image compression models.2020Entity Matching with Transformer Architectures - A Step Forward in Data IntegrationUrsin Brunner, Kurt Stockinger · International Conference on Extending Database TechnologyThis paper empirically compares the capability of transformer architectures and transfer-learning on the task of EM and shows that transformer architectures outperform classical deep learning methods in EM by an average margin of 27.5%.2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence ScoringSamuel Humeau, Kurt Shuster, M. Lachaux et al.This work develops a new transformer architecture, the Poly-encoder, that learns global rather than token level self-attention features and achieves state-of-the-art results on three existing tasks.2022Deep Learning Approaches Based on Transformer Architectures for Image Captioning TasksRoberto Castro, Israel Pineda, W. Lim et al. · IEEE AccessThis paper focuses on visual attention, a state-of-the-art approach for image captioning tasks within the computer vision research area and shows that the correct selection of both the cost function and the gradient-based optimizer can significantly impact the captioning results.2021Audio Transformers: Transformer Architectures For Large Scale Audio Understanding. Adieu ConvolutionsPrateek Verma, J. Berger · arXiv.orgThis work proposes applying Transformer based architectures without convolutional layers to raw audio signals, and shows how the models learns a non-linear non constant band-width filter-bank, which shows an adaptable time frequency front end representation for the task of audio understanding.2021Globalizing BERT-based Transformer Architectures for Long Document SummarizationQuentin Grail · Conference of the European Chapter of the Association for Computational LinguisticsThis work introduces a novel hierarchical propagation layer that spreads information between multiple transformer windows and adopts a hierarchical approach where the input is divided in multiple blocks independently processed by the scaled dot-attentions and combined between the successive layers.2021A Generative Model for Raw Audio Using Transformer ArchitecturesPrateek Verma, C. Chafe · International Conference on Digital Audio EffectsThis paper proposes a novel way of doing audio synthesis at the waveform level using Transformer architectures, and shows how causal transformer generative models can be used for raw waveform synthesis.2021On the use of summarization and transformer architectures for profiling résumésA. Bondielli, F. Marcelloni · Expert systems with applicationsThis paper proposes a methodology to profile resumes based on summarization and transformer architectures for generating resume embeddings and on hierarchical clustering algorithms for grouping theseembeddings.2020KERMIT: Complementing Transformer Architectures with Encoders of Explicit Syntactic InterpretationsF. Zanzotto, Andrea Santilli, Leonardo Ranaldi et al. · Conference on Empirical Methods in Natural Language ProcessingKERMIT (Kernel-inspired Encoder with Recursive Mechanism for Interpretable Trees) is proposed to embed symbolic syntactic parse trees into artificial neural networks and to visualize how syntax is used in inference.2024Mammography classification with multi-view deep learning techniques: Investigating graph and transformer-based architecturesFrancesco Manigrasso, Rosario Milazzo, A. Russo et al. · Medical Image Anal.Evaluating novel transformer-based and graph-based architectures against state-of-the-art multi-view convolutional neural networks on a middle-scale dataset highlights the potential of a wide range of multi-view architectures for breast cancer classification, even in datasets of relatively modest size, although the detection of small lesions remains challenging without pixel-wise supervision or ad-hoc networks.