Introduction to mLLMCelltype

Overview

mLLMCelltype is an iterative multi-LLM consensus framework for cell type annotation in single-cell RNA sequencing data. By leveraging the complementary strengths of multiple large language models, this framework significantly improves annotation accuracy while providing transparent uncertainty quantification.

The package implements a novel approach where multiple large language models (LLMs) collaborate through structured deliberation to achieve more accurate and reliable cell type annotations than any single model could provide alone.

Background

Cell type annotation is a critical step in single-cell RNA sequencing (scRNA-seq) analysis. Traditional methods often rely on reference datasets or marker gene databases, which can be limited by the availability of high-quality references and the complexity of cell types across different tissues and conditions.

Large language models have shown promising results in cell type annotation by leveraging their extensive knowledge of biological literature and ability to reason about gene expression patterns. However, individual LLMs can produce hallucinations or make errors due to limitations in their training data or reasoning capabilities.

mLLMCelltype addresses these challenges by implementing a consensus-based approach where multiple LLMs collaborate to provide more reliable annotations.

Key Features

Multi-LLM Consensus Architecture

mLLMCelltype harnesses collective intelligence from diverse LLMs to overcome single-model limitations and biases. The package currently supports a wide range of models:

  • OpenAI GPT-4o/4.1
  • Anthropic Claude-3.7/3.5
  • Google Gemini-2.0/2.5 (including Gemini-2.0-Flash-Lite)
  • X.AI Grok-3
  • DeepSeek-V3
  • Alibaba Qwen2.5/Qwen3
  • Zhipu GLM-4
  • MiniMax
  • Stepfun
  • OpenRouter

By integrating multiple models with different architectures and training data, mLLMCelltype can achieve more robust and accurate annotations than any single model.

Structured Deliberation Process

The package enables LLMs to share reasoning, evaluate evidence, and refine annotations through multiple rounds of collaborative discussion. This structured deliberation process includes:

  1. Initial independent annotation by each LLM
  2. Identification of controversial clusters where models disagree
  3. Structured discussion where models share their reasoning and evaluate others’ arguments
  4. Consensus formation through iterative refinement

This process mimics how a panel of human experts might collaborate to reach a consensus on difficult cases.

Transparent Uncertainty Quantification

mLLMCelltype provides quantitative metrics to identify ambiguous cell populations that may require expert review:

  • Consensus Proportion: Measures the level of agreement among LLMs
  • Shannon Entropy: Quantifies the uncertainty in the annotations

These metrics help researchers identify which cell clusters have high confidence annotations and which may require further investigation.

Other Advanced Features

  • Hallucination Reduction: Cross-model deliberation actively suppresses inaccurate or unsupported predictions
  • Robust to Input Noise: Maintains high accuracy even with imperfect marker gene lists
  • Hierarchical Annotation Support: Optional extension for multi-resolution analysis
  • No Reference Dataset Required: Performs accurate annotation without pre-training or reference data
  • Complete Reasoning Chains: Documents the full deliberation process
  • Seamless Integration: Works directly with standard Scanpy/Seurat workflows
  • Modular Design: Easily incorporate new LLMs as they become available
  • Enhanced Marker Gene Visualization: Publication-ready bubble plots and heatmaps showing marker gene expression patterns across annotated cell types

Applicable Scenarios

mLLMCelltype is designed for a wide range of single-cell RNA sequencing analysis scenarios:

  • Novel tissue types where reference datasets may be limited
  • Rare or poorly characterized cell populations
  • Complex tissues with many similar cell types
  • Integrative analysis across multiple datasets
  • Quality control to validate annotations from other methods

Latest Updates

v1.1.4 (2025-04-24)

Bug Fixes

  • Fixed a critical issue with cluster index handling, now the package strictly accepts only 0-based indices (compatible with Seurat)
  • Fixed negative index (-1) issues that could occur when processing CSV input files
  • Added strict validation for input cluster indices to ensure they start from 0

Improvements

  • Removed automatic conversion logic from 1-based to 0-based indices
  • Added input validation to ensure cluster indices start from 0
  • Updated code comments for clarity

For a complete list of updates, please refer to the NEWS.md file.

Getting Started

To get started with mLLMCelltype, please refer to the Getting Started Guide and Usage Tutorial.

Citation

If you use mLLMCelltype in your research, please cite:

Yang, C., Zhang, X., & Chen, J. (2025). Large Language Model Consensus Substantially
Improves the Cell Type Annotation Accuracy for scRNA-seq Data. bioRxiv.
https://doi.org/10.1101/2025.04.10.647852

Next Steps