Skip to Content

GLEAN – AI Chatbot for Eyecare

Category

Automation technology

Project size

36 MMs

Industry

Healthcare

Scope of work

Full-stack Development

Technologies
  • Image Processing: OpenCV, YOLO, Iris Extraction, Feature Segmentation
  • Text Processing: LLAMA 2, Transformer Attention Models
  • Environment: Python, Jupyter, Colab, HuggingFace, Visual Studio Code
  • Machine Learning: Supervised, Unsupervised, and Reinforcement Learning
  • Hardware: Intel Xeon W-1370P, 32GB RAM, RTX3090 GPU



About the project

This time, BHSOFT developed GLEAN – an AI-powered chatbot designed to revolutionize eye disease screening and patient interaction in the healthcare sector. The project was initiated to tackle the limitations of traditional diagnostic methods—namely cost, time, and accessibility—by offering a digital assistant that can support both doctors and patients in early detection and guidance.


Combining image processing, natural language processing, and vision-language models, GLEAN serves as a scalable, intelligent solution that delivers timely support, detailed medical analysis, and seamless user experiences for eye health management.

The primary goals were to develop a user-centric chatbot that could:


  • Detect and classify eye diseases through AI-powered image analysis
  • Process natural language inputs for both general and specialized medical queries
  • Support multimodal medical data: text, time series, images, and voice
  • Provide role-based access for both doctors and patients
  • Include messaging, appointment scheduling, and profile management
  • Be cost-effective, accessible, and suitable for remote diagnosis 

The team encountered several challenges during development:


  • Medical Accuracy: Ensuring the chatbot could analyze and interpret eye-related data reliably, including recognizing patterns in medical images.
  • Multimodal Data Integration: The system had to process various data types—text, time series, and images—and provide coherent insights.
  • User Personalization: Designing a flexible user experience that adapts to both professional and patient use cases without overwhelming either group.
  • Hardware Optimization: Deploying compute-heavy models while ensuring response time and system stability.

BHSoft employed an advanced AI architecture, integrating text and image processing with user-friendly design principles.


Vision and Language Fusion


  • Implemented image models (YOLO, segmentation, iris extraction) for real-time disease detection
  • Leveraged vision-language models (VL) to map medical images with diagnostic responses
  • Enabled image-based interaction, allowing users to submit photos and receive automated evaluations


NLP with LLAMA 2 & Transformers


  • Used LLAMA 2 with attention mechanisms to handle both generic and specialized medical queries
  • Included text encoding, semantic analysis, and self/multi-head attention to enhance understanding and dialogue quality


Chatbot Features and UI Design

  • Role-based access for patients and doctors
  • In-app messaging, chat history, group creation, and notifications
  • Profile management: avatar, language, contact info
  • Bot settings for reading voice, reading speed, and image suggestions
  • AI-enabled image interaction and suggestion delivery


Infrastructure Optimization

  • Deployed on a custom ML server with Intel Xeon CPUs and NVIDIA RTX 3090 GPU
  • Ensured real-time processing and scalable performance for AI tasks


Diagnostic Accuracy: Delivered reliable image recognition and interpretation through advanced ML models

Enhanced Patient Engagement: Made medical communication more approachable via chatbot interaction

Cross-Platform Utility: GLEAN handles diverse data types and user roles with a streamlined interface

Scalable AI Foundation: Designed to support future expansion with additional medical conditions and features