DCAMIL: Eye-tracking guided dual-cross-attention multi-instance learning for refining fundus disease detection

Hongyang Jiang, Mengdi Gao, Jingqi Huang, Chen Tang, Xiaoqing Zhang, Jiang Liu

Research output: Journal PublicationArticlepeer-review

3 Citations (Scopus)

Abstract

Deep neural networks (DNNs) have facilitated the development of computer-aided diagnosis (CAD) systems for fundus diseases, helping ophthalmologists to reduce missed diagnoses and misdiagnosis rates. However, the majority of CAD systems are data-driven, but lack the prior medical knowledge that can be performance-friendly. In this regard, we innovatively proposed a human-in-the-loop (HITL) CAD system by leveraging ophthalmologists’ eye-tracking information. Concretely, the HITL CAD system was implemented on the multi-instance learning (MIL), where clinicians’ gaze maps were beneficial to cherry-pick diagnosis-related instances. Furthermore, the dual-cross-attention MIL (DCAMIL) network was utilized to curb the adverse effects of noisy instances. Meanwhile, both the sequence augmentation (SA) module and the domain adversarial network (DAN) were introduced to enrich and standardize the instances in the training bag, respectively, thereby enhancing the robustness of our method. We conduct comparative experiments on our newly-constructed datasets (namely, AMD-Gaze and DR-Gaze) for the AMD and early DR detection, respectively. Rigorous experiments demonstrate the feasibility of our HITL CAD system and the superiority of the proposed DCAMIL, which fully exploits ophthalmologists’ eye-tracking information. These investigations indicate that clinicians’ gaze maps, as prior medical knowledge, is potential to contribute to the CAD systems of clinical diseases.

Original languageEnglish
Article number122889
JournalExpert Systems with Applications
Volume243
DOIs
Publication statusPublished - 1 Jun 2024
Externally publishedYes

Keywords

  • Computer-aided diagnosis
  • Eye-tracking
  • Fundus disease
  • Human-in-the-loop
  • Multi-instance learning

ASJC Scopus subject areas

  • General Engineering
  • Computer Science Applications
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'DCAMIL: Eye-tracking guided dual-cross-attention multi-instance learning for refining fundus disease detection'. Together they form a unique fingerprint.

Cite this