---
title: "CDCTNet: Transformer–CNN hybrid for colorectal lesion diagnosis"
id: "plos-one-7-colorectal-lesion-diagnosis-using-transformer-and-deep-learning-with-multiscale"
canonical_url: "https://medichelpline.com/clinical-feed/plos-one-7-colorectal-lesion-diagnosis-using-transformer-and-deep-learning-with-multiscale"
content_type: "clinical_feed_article"
specialty: "Oncology"
source_name: "PLOS ONE (Medicine)"
source_url: "https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664"
published_at: "2026-09-03T14:00:00.000Z"
evidence_level: "Journal Feed"
license: "CC-BY-NC-4.0 / Informational Use"
---
# CDCTNet: Transformer–CNN hybrid for colorectal lesion diagnosis
## Provenance & Clinical Metadata
- **Canonical URL:** https://medichelpline.com/clinical-feed/plos-one-7-colorectal-lesion-diagnosis-using-transformer-and-deep-learning-with-multiscale
- **Specialty:** [Oncology](https://medichelpline.com/clinical-feed/oncology.md)
- **Primary Source:** PLOS ONE (Medicine)
- **Source URL:** [Original Journal Publication](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664)
- **Published At:** 2026-09-03T14:00:00.000Z
- **Evidence Rating:** Journal Feed
## Executive GIST (TL;DR)
- Colorectal cancer is the third most common malignancy globally and early detection improves curability; conventional screening includes colonoscopy, stool tests, imaging, and molecular assays. - Machine learning (ML) and deep learning (DL) are increasingly applied to colorectal cancer (CRC) tasks: histopathology classification, polyp detection on colonoscopy, tumor segmentation on imaging, mutation status prediction, and outcome modeling. - Classical CNNs capture strong local spatial details but can miss long-range global feature relationships; traditional ML relies on handcrafted features and expert design. - The study presents **CDCTNet** (colorectal diagnosis convolution transformer network), a hierarchical hybrid model combining convolutional blocks and a Vision Transformer (**ViT**) encoder to capture both local and global contextual features from images. - CDCTNet uses two convolution blocks (Conv1 with 32 filters and Conv2 with 64 filters, each followed by 2×2 max pooling) to extract high-dimensional local spatial features. - In parallel to the CNN path, a **ViT** encoder provides global attention and long-range correlations across the feature map. - An **IEM** (information exchange module) mediates interaction between the convolutional features and the ViT embeddings: CNN spatial features are projected into embedding space, combined via learnable fusion parameters, then fed to the ViT multi-head attention to optimize contribution from both branches. - Extracted features from both branches are fused, pass through a global average pooling (GAP) layer, and are classified with a softmax output. - CDCTNet was evaluated on the publicly available **Kather** and **Kvasir** datasets (data source reported as Zenodo repository in the article) and achieved reported performance metrics: precision 96.60%, Kappa score 95.02%, recall 98.08%, and F1 score 97.94%. - The paper positions CDCTNet as leveraging complementary strengths of **CNN** and **ViT** with an explicit fusion module (IEM) to strengthen feature interaction for colorectal lesion detection. - Data availability, funding, and competing interest statements: dataset is publicly available via Zenodo link; authors reported no specific funding and declared no competing interests. - The manuscript includes a literature review summarizing multiple prior DL efforts in CRC for MSI/dMMR detection, KRAS prediction, polyp/optical diagnosis, lymph node metastasis prediction, response assessment, molecular pathway prediction, and biomarker retrieval, situating CDCTNet in this context.
## Clinical Analysis & Structured Key Points
[ Skip to main content ](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#main-content) Advertisement * [plos.org](https://plos.org/) * [Create account](https://community.plos.org/registration/new) * [Sign in](https://journals.plos.org/user/secure/login?page=%2Fplosone%2Farticle%3Fid%3D10.1371%2Fjournal.pone.0357664) * * About * Browse * Publish * [](https://journals.plos.org/plosone/ "PLOS One") * Search [advanced search](https://journals.plos.org/plosone/search) * [Browse Topics](https://journals.plos.org/plosone/subjectAreaBrowse) Browse Subject Areas ? Click through the PLOS taxonomy to find articles in your field. For more information about PLOS Subject Areas, click [here](https://github.com/PLOS/plos-thesaurus/blob/master/README.md "Link opens in new window"). [](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664) [](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664) * 0 [Save](https://journals.plos.org/plosone/article/metrics?id=10.1371/journal.pone.0357664#savedHeader) [Total Mendeley and Citeulike bookmarks.](https://journals.plos.org/plosone/article/metrics?id=10.1371/journal.pone.0357664#savedHeader) * 0 [Citation](https://journals.plos.org/plosone/article/metrics?id=10.1371/journal.pone.0357664#citedHeader) [Paper's citation count computed by Dimensions.](https://journals.plos.org/plosone/article/metrics?id=10.1371/journal.pone.0357664#citedHeader) * 26 [View](https://journals.plos.org/plosone/article/metrics?id=10.1371/journal.pone.0357664#viewedHeader) [PLOS views and downloads.](https://journals.plos.org/plosone/article/metrics?id=10.1371/journal.pone.0357664#viewedHeader) * 0 [Share](https://journals.plos.org/plosone/article/metrics?id=10.1371/journal.pone.0357664#discussedHeader) [Sum of Facebook, Twitter, Reddit and Wikipedia activity.](https://journals.plos.org/plosone/article/metrics?id=10.1371/journal.pone.0357664#discussedHeader) Open Access Peer-reviewed Research Article # Colorectal Lesion diagnosis using transformer and deep learning with multiscale feature interface * Dhirendra Prasad Yadav, Roles Conceptualization, Data curation, Methodology, Writing – original draft Affiliation Department of Computer Engineering & Applications, G.L.A. University, Mathura, Uttar Pradesh, India ⨯ * Bhisham Sharma , Roles Conceptualization, Data curation, Formal analysis, Investigation, Project administration, Software, Writing – review & editing * E-mail: bhisham.pec@gmail.com Affiliation Centre for Research Impact & Outcome, Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura, Punjab, India [ ![ORCID logo](https://journals.plos.org/resource/img/orcid_16x16.png) https://orcid.org/0000-0002-3400-3504 ](https://orcid.org/0000-0002-3400-3504 "ORCID Registry") ⨯ * Julian L. Webber, Roles Resources, Software, Supervision, Validation, Visualization, Writing – review & editing Affiliation Department of Electronics and Communication Engineering, Kuwait College of Science and Technology (KCST), Kuwait City, Kuwait ⨯ * Abolfazl Mehbodniya Roles Formal analysis, Methodology, Project administration, Software, Visualization Affiliation Department of Electronics and Communication Engineering, Kuwait College of Science and Technology (KCST), Kuwait City, Kuwait ⨯ # Colorectal Lesion diagnosis using transformer and deep learning with multiscale feature interface * Dhirendra Prasad Yadav, * Bhisham Sharma, * Julian L. Webber, * Abolfazl Mehbodniya ![PLOS](https://journals.plos.org/resource/img/logo-plos-full-color.svg) x * Published: September 3, 2026 * * [Article](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664) * [Authors](https://journals.plos.org/plosone/article/authors?id=10.1371/journal.pone.0357664) * [Metrics](https://journals.plos.org/plosone/article/metrics?id=10.1371/journal.pone.0357664) * [Comments](https://journals.plos.org/plosone/article/comments?id=10.1371/journal.pone.0357664) * [Media Coverage](http://plos.altmetric.com/details/doi/10.1371/journal.pone.0357664) * [Abstract](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#abstract0) * [1. Introduction](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#sec001) * [2. Literature Review](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#sec002) * [3. Proposed method](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#sec003) * [4. Results](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#sec007) * [5. Discussion](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#sec011) * [6. Conclusion](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#sec020) * [References](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#references) * [Reader Comments](https://journals.plos.org/plosone/article/comments?id=10.1371/journal.pone.0357664) * [Figures](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664) ## Abstract Colorectal cancer is the third most common malignancy worldwide. Manual screening requires expertise and resources. However, advancements in AI (artificial intelligence) have reduced the computation burden and time. Machine and deep learning have recently been used to diagnose colorectal lesions. The requirement of handcrafted features makes machine learning models expertise-dependent. At the same time, classical CNN (convolutional neural network) miss the global attention of the features. This work presents CDCTNet (colorectal diagnosis convolution transformer network), a hierarchical model for colorectal disease detection. Our model utilized two convolution blocks for the local high-dimensional spatial features from the lesion. In addition, the ViT encoder is used in parallel with the CNN block to provide a global correlation of the feature map. Furthermore, we designed an IEM block for the interaction of the features between the convolution block and ViT encoder to improve the attention on the features. The CDCTNet is evaluated on Kather and Kvasir datasets and obtained a precision and Kappa score of 96.60% and 95.02%, respectively. At the same time, CDCTNet has recall and F1 scores of 98.08% and 97.94%. ## Figures ![Fig 10](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.g010) ![Table 7](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.t007) ![Table 1](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.t001) ![Fig 1](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.g001) ![Fig 2](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.g002) ![Fig 3](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.g003) ![Fig 4](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.g004) ![Table 2](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.t002) ![Table 3](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.t003) ![Table 4](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.t004) ![Fig 5](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.g005) ![Fig 6](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.g006) ![Table 5](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.t005) ![Fig 7](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.g007) ![Table 6](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.t006) ![Fig 8](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.g008) ![Fig 9](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.g009) ![Fig 10](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.g010) ![Table 7](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.t007) ![Table 1](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.t001) ![Fig 1](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.g001) ![Fig 2](https://journals.plos.org/plosone/article/figure/image?size=inline&id=10.1371/journal.pone.0357664.g002) **Citation:** Yadav DP, Sharma B, Webber JL, Mehbodniya A (2026) Colorectal Lesion diagnosis using transformer and deep learning with multiscale feature interface. PLoS One 21(9): e0357664. https://doi.org/10.1371/journal.pone.0357664 **Editor:** Abel C. H. Chen, Chunghwa Telecom Co. Ltd., TAIWAN **Received:** March 21, 2025; **Accepted:** August 20, 2026; **Published:** September 3, 2026 **Copyright:** © 2026 Yadav et al. This is an open access article distributed under the terms of the [Creative Commons Attribution License](http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. **Data Availability:** The dataset used in this study is publicly available in the Zenodo repository and can be accessed at: . **Funding:** The author(s) received no specific funding for this work. **Competing interests:** The authors have declared that no competing interests exist. ## 1. Introduction The latest guidelines for colorectal cancer screening emphasize the need for an early start in the management of death statistics since it is highly curable if diagnosed early. Colorectal cancer, the third most common malignancy, affects more than 1.9 million patients and causes nearly 900,000 deaths worldwide annually [[1](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#pone.0357664.ref001)]. Established methods of screening, including colonoscopy, stool-based tests, and advanced imaging, have shown to have significantly reduced mortality. The recent WHO global cancer data brings before the public eye that there is disparity in regions over the utilization of cancer services, as screening services are still quite limited to low-income countries. These disparities will inevitably lead to diagnoses diagnosed at later stages and more deaths mainly due to the underserved populations [[2](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#pone.0357664.ref002)]. Colorectal cancers are abnormal growths appearing on parts of the large intestine: the colon and rectum. These tumors can either be benign or malignant. Polyps represent examples of benign types, whereas malignant cases present as colorectal cancer [[3](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#pone.0357664.ref003)]. The majority of cases originate as a polyp which, left untreated for long periods, then turns malignant. For a successful treatment, early detection is essential. Molecular testing, advanced imaging studies, and non-invasive screening tests are just some of the techniques used to detect colorectal cancer. The screening techniques include two types of FOBT: FIT is a more sensitive version that does not require any dietary restrictions, whereas gFOBT does require dietary restrictions. The Stool DNA Test (Cologuard) tests stool for mutations in DNA that would indicate advanced polyps or cancer. Since sigmoidoscopy only visualizes the lower portion of the colon, endoscopic studies such as colonoscopy allow direct visualization of the entire colon. EUS is useful in the evaluation of rectal cancer and essential for understanding the scope of tumor invasion [[4](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#pone.0357664.ref004)]. Imaging greatly helps the staging and especially diagnosis of colorectal cancer. Although MRI is utilized for precise staging of rectal cancers, the virtual colonoscopy or CT colonography permits non-invasive visualization of the colon and rectum. Older techniques include X-ray visualization with the Lower GI series or Barium Enema, while PET scans are employed through metabolic activity to detect metastasis and recurrence [[5](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#pone.0357664.ref005)]. By molecular and genetic testing, the tumors genetic makeup can be established; further tests are Mismatch Repair Deficiency (dMMR) and Microsatellite Instability (MSI). Recently, a new technology known as liquid biopsy has the capability to literally trail cancer without intervention directly into the system. It tracks through DNA or tumor cells present in the bloodstream. These methods make possible for accurate diagnosis, early detection, and proper treatment of colorectal cancers [[6](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#pone.0357664.ref006)]. Machine learning (ML) and deep learning (DL) are finding key roles in the detection and classification of CRC. Such advanced techniques can analyze large datasets like medical images, histopathological slides, genetic data, and clinical records to identify those patterns that predict cancerous growths [[7](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#pone.0357664.ref007),[8](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#pone.0357664.ref008)]. In histopathology, ML models can analyze digitized biopsy slides classified into tissue forms like benign, pre-cancerous, or malignant, and pathologists can diagnose more accurately. Colonoscopy employs DL models, especially CNNs, to classify video in real-time for the identification of polyps or early-stage tumors, hence improving dramatically detection rates by identifying lesions otherwise likely to be missed in the process of visual review [[9](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#pone.0357664.ref009)]. Finally, deep learning techniques like CNNs and U-Nets are applied in the segmentation of tumors in medical imaging, for instance, CT or MRI, hence enabling the accurate localization and staging of the tumor. Similarly, these have been applied in genomic data for the prediction of patient outcomes and treatment responses in association with genetic mutations like KRAS or MSI status. Since deep models are often data hungry, transfer learning could be well-applied to enable even the smallest domain-specific datasets at their disposal and hence enhancing the accuracy of detection. New applications of ML and DL models include predicting patient survival and recurrence risk, guiding a course of treatment, and even a personalized treatment strategy [[10](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#pone.0357664.ref010)]. The hurdles of data availability and model interpretability will allow AI technologies to be successfully incorporated into clinical workflows, so opportunities for more precise and efficient CRC detection, diagnosis, and treatment should contribute better to patient outcomes and facilitate the development of personalized medicine [[11](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#pone.0357664.ref011)]. The requirement of handcrafted features makes machine learning model expertise dependent. At the same time, classical CNN (convolutional neural network) miss the global attention of the features. This work presents CDCTNet (colorectal diagnosis convolution transformer network), a hierarchical model for colorectal disease detection. Our model utilized two convolution blocks for the local high-dimensional spatial features from the lesion. In addition, the ViT encoder is used in parallel with the CNN block to provide a global correlation of the feature map. Furthermore, we designed an IEM block for the interaction of the features between the convolution block and ViT encoder to improve the attention on the features. The CDCTNet is evaluated on Kather and Kvasir datasets. The contribution of the propose method is as follows. 1. (1) We combine CNN and ViT to leverage the colorectal lesion’s local spatial features and global contextual information. 2. (2) We strengthen the feature interaction between CNN and ViT using IEM block. To achieve this, spatial feature extracted from the CNN block is projected for the embedding space using dense projection. After that, contribution of the CNN and ViT feature is optimized using learnable fusion parameter before performing the multi-head attention in the ViT encoder. 3. (3) To evaluate model performance, we utilized the Kather and Kvasir dataset and achieved precision and Kappa score of 96.60% and 95.02%, respectively. Remainder of the manuscript is organized as follows. The colorectal diagnosis is summarized in section [2](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#sec002). At the same time, section [3](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#sec003) explains the architecture of CDCTNet. Moreover, section [4](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#sec007) provides quantitative results, and section [5](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#sec011) provides a detailed discussion. Finally, in section [6](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#sec020), we present the conclusion of the proposed method with future scope and limitations. ## 2. Literature Review Using MSI and dMMR in colorectal tumor cells on routine histology slides, this study developed a faster and cheaper deep-learning system compared to molecular assays. From the slides, the model trained a deep-learning detector that could classify samples as MSI; cross-validation was used to evaluate the model’s performance for N = 6406 specimens, and an external cohort of n = 771 specimens validated the results. It identified dMMR-positive tissues in a large global validation cohort at an AUROC of 0.96. This technology has potential to be translated into low-cost, high-throughput assessment of colorectal tissue samples [[12](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#pone.0357664.ref012)]. This identification of mutations within the KRAS gene is considered a key factor in devising specific treatments for patients diagnosed with CRC. Therefore, this paper applied pre-treatment CE-CT imaging to test the model’s predictive performance by estimating the status of KRAS mutation in CRC patients using a DL process with a ResNet. model recruited a total of 157 CRC patients whose pathology was confirmed. These patients were divided into two parts, namely: training with a total of n = 117 and a testing with n = 40 [[13](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357664#pone.0357664.ref013)]. CRC screening commonly used tool is colonoscopy. They train a deep learning model called CRCNet on 464,105 images from 12,179 patients to use in the optical diagnosis of colorectal cancer. Then we test the model on three distinct datasets of 2263 patients. AUPRC of CRCNet patients is 0. While better than average endoscopists, during achieving comparable recall rate on one test set and precision on the other two, CRCNet has performed with 93.7% versu
## Related Clinical Research

- [VEGFR2-targeted NIR-II fluorescent probe ICG-VTP for precise ovarian cancer imaging and surgical n](https://medichelpline.com/clinical-feed/pubmed-42184579.md) (DOI: 10.1016/j.bioorg.2026.110017)
- [Vorinostat-loaded PLGA-PEG-AEAA Nanocarrier Enhances Targeted Therapy for Colorectal Cancer](https://medichelpline.com/clinical-feed/pubmed-42546991.md) (DOI: 10.1016/j.ijpharm.2026.127274)
- [Traditional Chinese Medicine and the Immune Microenvironment in Colorectal Cancer: Mechanisms and](https://medichelpline.com/clinical-feed/frontiers-in-immunology-15-traditional-chinese-medicine-remodels-the-immune-microenvironment-of-colorectal.md)
- [Early pathology input in biologically driven cancer trials: why it matters](https://medichelpline.com/clinical-feed/british-journal-of-cancer-0-why-early-pathology-input-matters-in-biologically-rich-cancer-trials.md)
- [Late-stage cancer incidence as a screening-trial endpoint: an LLM-assisted evidence audit](https://medichelpline.com/clinical-feed/medrxiv-6-llm-assisted-evidence-audit-of-late-stage-cancer-incidence-as-a-screening-trial.md)

## Navigation
- [← Back to Oncology Feed](https://medichelpline.com/clinical-feed/oncology.md)
- [← All Clinical Specialties](https://medichelpline.com/clinical-feed.md)
## Medical & Regulatory Disclaimer

> [!CAUTION]
> MedicHelpline content is structured for research, educational, and professional discovery purposes. It does not constitute individual medical advice, clinical diagnosis, or treatment recommendations.
> Always verify dosing, contraindications, and regulatory alerts against official product labeling and primary regulatory sources before clinical decision-making.