The repository for Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Survey, including related models, datasets/benchmarks, and other resource links.
🔥 2025.12.01: We have updated our paper and repository with new models, datasets, and additional features developed since February 2024. Please check the latest version of our paper and the updated repository for more details.
🌟 If you have a paper or resource you'd like to add, feel free to submit a pull request, open an issue, or email the author at [email protected].
BioTextBioinformatics 2019BioBERT: a pre-trained biomedical language representation model for biomedical text mining |BioTextEMNLP IJCNLP 2019SciBERT: A Pretrained Language Model for Scientific Text |BioTextBioNLP@ACL 2019(BlueBERT) Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets |BioTextEMNLP 2020Bio-Megatron: Larger Biomedical Domain Language Model |BioTextBioNLP@CHIL 2020ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission |BioTextBioNLP@ACL 2021BioM-Transformers: Building Large Biomedical Language Models with BERT, ALBERT and ELECTRA |BioTextHEALTH 2021(PubMedBERT) Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing |BioTextArxiv 2021SciFive: a text-to-text transformer model for biomedical literature |BioTextNeurIPS 2022(DRAGON) Deep Bidirectional Language-Knowledge Graph Pretraining |BioTextACL 2022LinkBERT: Pretraining Language Models with Document Links |BioTextBioNLP@ACL 2022BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model |BioTextBioinformatics 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining |BioTextArxiv 2022GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records |BioTextNature 2023Large language models encode clinical knowledge |BioTextACL 2023(ScholarBERT) The Diminishing Returns of Masked Language Models to Science |BioTextArxiv 2023PMC-LLaMA: Further Finetuning LLaMA on Medical Papers |BioTextArxiv 2023BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicine |BioTextArxiv 2023(GatortronGPT) A study of generative large language model for medical research and healthcare |BioTextArxiv 2023Clinical Camel: An Open-Source Expert-Level Medical Language Model with Dialogue-Based Knowledge Encoding |BioTextArxiv 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Models |BioTextArxiv 2023BioinspiredLLM: Conversational Large Language Model for the Mechanics of Biological and Bio-inspired Materials |BioTextArxiv 2023ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluation |BioTextArxiv 2023MedAlpaca - An Open-Source Collection of Medical Conversational AI Models and Training Data |BioTextArxiv 2024SciGLM: Training Scientific Language Models with Self-Reflective Instruction Annotation and Tuning |BioTextArxiv 2024BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Text |BioTextACL 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains |Text + MoleculeEMNLP 2021Text2Mol: Cross-Modal Molecule Retrieval with Natural Language Queries |Text + MoleculeEMNLP 2022(MolT5) Translation between Molecules and Natural Language |Text + MoleculeNature Communications 2022(KV-PLM) A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals |Text + MoleculeArxiv 2022(MoMu) A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language |Text + MoleculeICML 2023(Text+Chem T5) Unifying Molecular and Textual Representations via Multi-task Language Modelling |Text + MoleculeICML 2023(CLAMP) Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language |Text + MoleculeNeurIPS 2023GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning |Text + MoleculeAI4D3@NeurIPS 2023MoleculeGPT: Instruction Following Large Language Models for Molecular Property Prediction |Text + MoleculeDatasets&Benchmarks@NeurIPS 2023(ChemLLMBench) What indeed can GPT models do in chemistry? A comprehensive benchmark on eight tasks |Text + MoleculeACL 2023MolXPT: Wrapping Molecules with Text for Generative Pre-training |Text + MoleculeEMNLP 2023(TextReact) Predictive Chemistry Augmented with Text Retrieval |Text + MoleculeEMNLP 2023MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter |Text + MoleculeEMNLP 2023ReLM: Leveraging Language Models for Enhanced Chemical Reaction Prediction |Text + MoleculeNature Machine Intelligence 2023(MoleculeSTM) Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing |Text + MoleculeIEEE TAI 2023(AMAN) Adversarial Modality Alignment Network for Cross-Modal Molecule Retrieval |Text + MoleculeBioinformatics 2024MolLM: A Unified Language Model for Integrating Biomedical Text with 2D and 3D Molecular Representations |Text + MoleculeArxiv 2023(MolReGPT) Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective |Text + MoleculeArxiv 2023(CaR) Can Large Language Models Empower Molecular Property Prediction? |Text + MoleculeArxiv 2023MolFM: A Multimodal Molecular Foundation Model |Text + MoleculeArxiv 2023(ChatMol) Interactive Molecular Discovery with Natural Language |Text + MoleculeArxiv 2023InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery |Text + MoleculeArxiv 2023ChemCrow: Augmenting large-language models with chemistry tools |Text + MoleculeArxiv 2023GPT-MolBERTa: GPT Molecular Features Language Model for molecular property prediction |Text + MoleculeArxiv 2023nach0: Multimodal Natural and Chemical Languages Foundation Model |Text + MoleculeArxiv 2023DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule Graphs |Text + MoleculeAAAI 2024(Ada/Aug-T5) From Artificially Real to Real: Leveraging Pseudo Data from Large Language Models for Low-Resource Molecule Discovery |Text + MoleculeAAAI 2024MolTailor: Tailoring Chemical Molecular Representation to Specific Tasks via Text Prompts |Text + MoleculeAAAI 2024(TGM-DLM) Text-Guided Molecule Generation with Diffusion Language Model |Text + MoleculeComputers in Biology and Medicine 2024GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text |Text + MoleculeChemical Science 2024PolyNC: a natural and chemical language model for the prediction of unified polymer properties |Text + MoleculeArxiv 2024MolTC: Towards Molecular Relational Modeling In Language Models |Text + MoleculeArxiv 2024T-Rex: Text-assisted Retrosynthesis Prediction |Text + MoleculeCOLM 2024LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset |Text + MoleculeArxiv 2024(Drug-to-indication) Emerging Opportunities of Using Large Language Models for Translation Between Drug Molecules and Indications |Text + MoleculeArxiv 2024ChemDFM: Dialogue Foundation Model for Chemistry |Text + MoleculeBriefings in Bioinformatics 2024DrugAssist: A Large Language Model for Molecule Optimization |Text + MoleculeArxiv 2024ChemLLM: A Chemical Large Language Model |Text + MoleculeOpenReview(TEDMol) Text-guided Diffusion Model for 3D Molecule Generation |Text + MoleculeBioinformatics 2025(3DToMolo) Sculpting Molecules in 3D: A Flexible Substructure Aware Framework for Text-Oriented Molecular Optimization |Text + MoleculeIEEE TKDE 2024(ICMA) Large Language Models are In-Context Molecule Learners |Text + MoleculeArxiv 2024(LLMaMol) Benchmarking Large Language Models for Molecule Prediction Tasks |Text + MoleculeArxiv 20243M-Diffusion: Latent Multi-Modal Diffusion for Text-Guided Generation of Molecular Graphs |Text + MoleculeBiology 2024(TSMMG) Instruction Multi-Constraint Molecular Generation Using a Teacher-Student Large Language Model |Text + MoleculeEng Appl Artif Intell(SLM4CRP) A Self-feedback Knowledge Elicitation Approach for Chemical Reaction Predictions |Text + MoleculeICLR 2025Atomas: Hierarchical Alignment on Molecule-Text for Unified Molecule Understanding and Generation |Text + MoleculeACL 2024ReactXT: Understanding Molecular"Reaction-ship"via Reaction-Contextualized Molecule-Text Pretraining |Text + MoleculeL+M 2024ALMol: Aligned Language-Molecule Translation LLMs through Offline Preference Contrastive Optimisation |Text + MoleculeArxiv 2024LDMol: Text-Conditioned Molecule Diffusion Model Leveraging Chemically Informative Latent Space |Text + MoleculeArxiv 2024DrugLLM: Open Large Language Model for Few-shot Molecule Generation |Text + MoleculeArxiv 2024(HI-Mol) Data-Efficient Molecular Generation with Hierarchical Textual Inversion |Text + MoleculeKDD 2024(MV-Mol) Learning Multi-view Molecular Representations with Structured and Unstructured Knowledge |Text + MoleculeNLPCC 2024DRAK: Unlocking Molecular Insights with Domain-Specific Retrieval-Augmented Knowledge in LLMs |Text + MoleculeArxiv 2024HIGHT: Hierarchical Graph Tokenization for Graph-Language Alignment |Text + MoleculeEMNLP 2024PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomes |Text + MoleculeArxiv 20243D-MolT5: Towards Unified 3D Molecule-Text Modeling with 3D Molecular Tokenization |Text + MoleculeArxiv 2024MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction |Text + MoleculeArxiv 2024MolX: Enhancing Large Language Models for Molecular Learning with A Multi-Modal Extension |Text + MoleculeArxiv 2024(AMOLE) Vision Language Model is NOT All You Need: Augmentation Strategies for Molecule Language Models |Text + MoleculeArxiv 2024(Chemma-RC) Text-Augmented Multimodal LLMs for Chemical Reaction Condition Recommendation |Text + MoleculeICLR 2024(SMILES-probing) Chemical Language Models Have Problems with Chemistry: A Case Study on Molecule Captioning Task |Text + MoleculeArxiv 2024UniMoT: Unified Molecule-Text Language Model with Discrete Token Representation |Text + MoleculeECAI 2025(UTGDiff) Instruction-Based Molecular Graph Generation with Unified Text-Graph Diffusion Model |Text + MoleculeACL 2024Mol2Lang-VLM: Vision- and Text-Guided Generative Pre-trained Language Models for Advancing Molecule Captioning through Multimodal Fusion |Text + MoleculeACL 2024Lang2Mol-Diff: A Diffusion-Based Generative Model for Language-to-Molecule Translation Leveraging SELFIES Representation |Text + MoleculeACL 2024(Self-Augmentation) Enhancing Cross Text-Molecule Learning by Self-Augmentation |Text + MoleculeACL 2024MTSwitch: A Web-based System for Translation between Molecules and Texts |Text + MoleculeNeurIPS 2024SmileyLlama: Modifying Large Language Models for Directed Chemical Space Exploration |Text + MoleculeNeurIPS 2024(MoleculeSTM) Geometry-text Multi-modal Foundation Model for Reactivity-oriented Molecule Editing |Text + MoleculeACL 2025(MSR) Structural Reasoning Improves Molecular Understanding of LLM |Text + MoleculeArxiv 2024(TransDLM) Text-Guided Multi-Property Molecular Optimization with a Diffusion Language Model |Text + MoleculeArxiv 2024Can LLMs Generate Diverse Molecules? Towards Alignment with Structural Diversity |Text + MoleculeJCIM 2025(ChemLML) Chemical Language Model Linker: blending text and molecules with modular adapters |Text + MoleculeNeurIPS 2024(Chemlactica) Small Molecule Optimization with Large Language Models |Text + MoleculeNeurIPS 2024Question Rephrasing for Quantifying Uncertainty in Large Language Models: Applications in Molecular Chemistry Tasks |Text + MoleculeNeurIPS 2024(LLaMo) Large Language Model-based Molecular Graph Assistant |Text + MoleculeNeurIPS 2024(MolPuzzle) Can LLMs Solve Molecule Puzzles? A Multimodal Benchmark for Molecular Structure Elucidation |Text + MoleculeArxiv 2024(M³LLM) Exploring Hierarchical Molecular Graph Representation in Multimodal LLMs |Text + MoleculeArxiv 2024MolReFlect: Towards In-Context Fine-grained Alignments between Molecules and Texts |Text + MoleculeENMLP 2024(AMORE) Lost in Translation: Chemical Language Models and the Misunderstanding of Molecule Structures |Text + MoleculeBIBM 2024GeomCLIP: Contrastive Geometry-Text Pre-training for Molecules |Text + MoleculeBIBM 2024(CMTMR) Towards Cross-Modal Text-Molecule Retrieval with Better Modality Alignment |Text + MoleculeBIBM 2024(ORMA) Exploring Optimal Transport-Based Multi-Grained Alignments for Text-Molecule Retrieval |Text + MoleculeArxiv 2024PEIT: Property Enhanced Instruction Tuning for Multi-task Molecular Generation with LLMs |Text + MoleculeArxiv 2024(HME) Navigating Chemical-Linguistic Sharing Space with Heterogeneous Molecular Encoding |Text + MoleculeICLR 2025(Llamole) Multimodal Large Language3D-MOLT5 Models for Inverse Molecular Design with Retrosynthetic Planning |Text + MoleculeICLR 2025RetroInText: A Multimodal Large Language Model Enhanced Framework for Retrosynthetic Planning via In-Context Representation Learning |Text + MoleculeArxiv 2025OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discovery |Text + MoleculeNeurIPS 2025Omni-Mol: Multitask Molecular Model for Any-to-any Modalities |Text + MoleculeArxiv 2025Mol-LLM: Multimodal Generalist Molecular LLM with Improved Graph Utilization |Text + MoleculeNMI 2025(SLM4Mol) A Quantitative Analysis of Knowledge-Learning Preferences in Large Language Models in Molecular Science |Text + MoleculeArxiv 2025CLASS: Enhancing Cross-Modal Text-Molecule Retrieval Performance and Training Efficiency |Text + MoleculeNeurIPS 2025Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Model |Text + MoleculeBioinformatics 2025ChatMol: A Versatile Molecule Designer Based on the Numerically Enhanced Large Language Model |Text + MoleculeEMNLP 2025MV-CLAM: Multi-View Molecular Interpretation with Cross-Modal Projection via Language Model |Text + MoleculeArxiv 2025GraphT5: Unified Molecular Graph-Language Modeling via Multi-Modal Cross-Token Attention |Text + MoleculeJBHI 2025XMolCap: Advancing Molecular Captioning through Multimodal Fusion and Explainable Graph Neural Networks |Text + MoleculeJCIM 2025(ChatChemTS) Large language models open new way of AI-assisted molecule design for chemists |Text + MoleculeJCIM 2025(LLM-MPP) Effective and Explainable Molecular Property Prediction by Chain-of-Thought Enabled Large Language Models and Multi-Modal Molecular Information Fusion |Text + MoleculeAAAI 2025Graph2Token: Make LLMs Understand Molecule Graphs |Text + MoleculeAAAI 2025ExDDI: Explaining Drug-Drug Interaction Predictions with Natural Language |Text + MoleculeAAAI 2025ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area |Text + MoleculeICJAI 2025ChemDual: Enhancing Chemical Reaction and Retrosynthesis Prediction with Large Language Model and Dual-task Learning |Text + MoleculeACL 2025GeLLM³O: Generalizing Large Language Models for Multi-property Molecule Optimization |Text + MoleculeACL 2025Less for More: Enhanced Feedback-aligned Mixed LLMs for Molecule Caption Generation and Fine-Grained NLI Evaluation |Text + MoleculeEMNLP 2025(GeLLMO-C) Large Language Models for Controllable Multi-property Multi-objective Molecule Optimization |Text + MoleculeAAAI 2025ReactGPT: Understanding of Chemical Reactions via In-Context Tuning |Text + MoleculeAAAI 2025nach0-pc: Multi-task Language Model with Molecular Point Cloud Encoder |Text + MoleculeArxiv 2025mCLM: A Function-Infused and Synthesis-Friendly Modular Chemical Language Model |Text + MoleculeArxiv 2025ChemMLLM: Chemical Multimodal Large Language Model |Text + MoleculeArxiv 2025RTMol: Rethinking Molecule-text Alignment in a Round-trip View |Text + MoleculeArxiv 2025(CLEANMOL) Improving Chemical Understanding of LLMs via SMILES Parsing |Text + MoleculeNeurIPS 2025ModuLM: Enabling Modular and Multimodal Molecular Relational Learning with Large Language Models |Text + MoleculearXiv 2025(ToDi) TextOmics-Guided Diffusion for Hit-like Molecular Generation |Text + MoleculeArxiv 2025ChemDFM-R: An Chemical Reasoner LLM Enhanced with Atomized Chemical Knowledge |Text + MoleculePAKDD 2025Dual Learning Between Molecules and Natural Language |Text + MoleculeMM 2025CROP: Integrating Topological and Spatial Structures via Cross-View Prefixes for Molecular LLMs |Text + MoleculeArxiv 2025Mol-R1: Towards Explicit Long-CoT Reasoning in Molecule Discovery |Text + MoleculeArxiv 2025AttriLens-Mol: Attribute Guided Reinforcement Learning for Molecular Property Prediction with Large Language Models |Text + MoleculeBioinformatics 2025MolPrompt: improving multi-modal molecular pre-training with knowledge prompts |Text + MoleculeENMLP 2025(CAMT5) Training Text-to-Molecule Models with Context-Aware Tokenization |Text + MoleculeArxiv 2025Enhancing Molecular Property Prediction with Knowledge from Large Language Models |Text + MoleculeKnosys 2025(MolFinePrompt) Fine-grained multimodal molecular pretraining via prompt learning |Text + MoleculeArxiv 2025(MPPReasoner) Reasoning-Enhanced Large Language Models for Molecular Property Prediction |Text + MoleculeArxiv 2025(MECo) Coder as Editor: Code-Driven Interpretable Molecule Optimization |Text + MoleculeArxiv 2025Chem-R: Learning to Reason as a Chemist |Text + MoleculeArxiv 2025KnowMol: Advancing Molecular Large Language Models with Multi-Level Chemical Knowledge |Text + MoleculeIEEE Trans Comput Soc Syst 2025(Mol-LLM) Incorporating Molecular Knowledge in Large Language Models via Multimodal Modeling |Text + MoleculeMM 2025DeepMolTex: Deep Alignment of Molecular Graphs with Large Language Models via Mixture of Modality Experts |Text + MoleculeNeurocomputing 2025Mol-L2: Transferring text knowledge with frozen language models for molecular representation learning |Text + MoleculeACL 2025(MolRAG) Unlocking the Power of LLMs for Molecular Property Prediction |Text + MoleculeACL 2025Boosting LLM's Molecular Structure Elucidation with Knowledge Enhanced Tree Search Reasoning |Text + MoleculeEMNLP 2025(MolBridge) Bridging the Gap Between Molecule and Textual Descriptions via Substructure-aware Alignment |Text + MoleculeEMNLP 2025GAMIC: Graph-Aligned Molecular In-context Learning for Molecule Analysis via LLMs |Text + MoleculeEMNLP 2025How to Make Large Language Models Generate 100% Valid Molecules? |Text + MoleculeEMNLP 2025Molecular String Representation Preferences in Pretrained LLMs: A Comparative Study in Zero- & Few-Shot Molecular Property Prediction |Text + ProteinICLR 2022OntoProtein: Protein Pretraining With Gene Ontology Embedding |Text + ProteinRECOMB 2022ProTranslator: Zero-Shot Protein Function Prediction Using Textual Description |Text + ProteinICML 2023ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts |Text + ProteinNature Biotechnology 2023(ProGen) Large language models generate functional protein sequences across diverse families |Text + ProteinArxiv 2023(ProteinDT) A Text-guided Protein Design Framework |Text + ProteinTechRxiv 2023ProteinChat: Towards Achieving ChatGPT-Like Functionalities on Protein 3D Structures |Text + ProteinICML 2025 2023(ProteinChat v2) Multi-Modal Large Language Model Enables Protein Function Prediction |Text + ProteinAAAI 2024Prot2Text: Multimodal Protein's Function Generation with GNNs and Transformers |Text + ProteinArxiv 2024ProtAgents: Protein discovery via large language model multi-agent collaborations combining physics and machine learning |Text + ProteinTAI 2024ProLLaMA: A Protein Large Language Model for Multi-Task Protein Language Processing |Text + ProteinACL 2024ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training |Text + ProteinACL 2024ProtT3: Protein-to-Text Generation for Text-based Protein Understanding |Text + ProteinbioRxiv 2024ProteinCLIP: enhancing protein language models with natural language |Text + ProteinArxiv 2024ProLLM: Protein Chain-of-Thoughts Enhanced LLM for Protein-Protein Interaction Prediction |Text + ProteinKDD 2025(PAAG) Annotation-guided Protein Design with Multi-Level Domain Alignment |Text + ProteinbioRrxiv 2024(Pinal) Toward De Novo Protein Design from Natural Language |Text + ProteinArxiv 2024TourSynbio: A Multi-Modal Large Model and Agent Framework to Bridge Text and Protein Sequences for Protein Engineering |Text + ProteinKDD 2025(SEPIT) Structure-Enhanced Protein Instruction Tuning: Towards General-Purpose Protein Understanding with LLMs |Text + ProteinNeurIPS 2024(BioM3) Natural Language Prompts Guide the Design of Novel Functional Protein Sequences |Text + ProteinNeurIPS 2024(LLM4ProteinEvolution) Language Models for Text-guided Protein Evolution |Text + ProteinNeurIPS 2024MMSite: A Multi-modal Framework for the Identification of Active Sites in Proteins |Text + ProteinNeurIPS 2024MutaPLM: Protein Language Modeling for Mutation Explanation and Engineering |Text + ProteinArxiv 2024EvoLlama: Enhancing LLMs' Understanding of Proteins via Multimodal Structure and Sequence Representations |Text + ProteinBioinformatics 2024FAPM: functional annotation of proteins using multimodal models beyond structural modeling |Text + ProteinbioRxiv 2024ProCyon: A multimodel foundation model for protein phenotypes |Text + ProteinbioRxiv 2025(Evolla) Decoding the Molecular Language of Proteins with Evolla |Text + ProteinICLR 2025ProteinGPT: Multimodal LLM for Protein Property Prediction and Structure Understanding |Text + ProteinICLR 2025(MP4) A generalized protein design ML model enables generation of functional de novo proteins |Text + ProteinSIGIR 2025ProtChatGPT: Towards Understanding Proteins with Hybrid Representation and Large Language Models |Text + ProteinAAAI 2025Protclip: Function-informed protein multi-modal learning |Text + ProteinAAAI 2025(CtrlProt) Controllable Protein Sequence Generation with LLM Preference Optimization Authors |Text + ProteinNAACL 2025Protein2Text: Resampling Mechanism to Translate Protein Sequences into Human-Interpretable Text |Text + ProteinEMNLP 2025(RAPM): Rethinking Text-based Protein Understanding: Retrieval or LLM? |Text + ProteinNeurIPS 2025Prot2Text-V2: Protein Function Prediction with Multimodal Contrastive Alignment |Text + ProteinNeurIPS 2025(ProDVa) Protein Design with Dynamic Protein Vocabulary |Text + ProteinICML 2025ProteinAligner: A Tri-Modal Contrastive Learning Framework for Protein Representation Learning |Text + ProteinJCIM 2025Prottex: Structure-in-context reasoning and editing of proteins with large language models |Text + ProteinBioinformatics 2025Prot2Chat: protein large language model with early fusion of text, sequence, and structure |Text + ProteinNature Biotechnology 2025ProTrek: Navigating the Protein Universe through Tri-Modal Contrastive Learning |Text + ProteinNature Communications 2025Ab-initio amino acid sequence design from protein text description with ProtDAT |Text + ProteinACL 2025Enhancing Safe and Controllable Protein Generation via Knowledge Preference Optimization |Text + ProteinACL 2025(LLaPA) Large Language and Protein Assistant for Protein-Protein Interactions Prediction |Text + ProteinArxiv 2025Protein as a Section Language for LLMs |Text + ProteinArxiv 2025CMADiff: Cross-Modal Aligned Diffusion for Controllable Protein Generation |Text + ProteinArxiv 2025Guiding Generative Models for Protein Design: Prompting, Steering and Aligning |Text + ProteinICLR Submission 2026Caduceus: MoE-enhanced Foundation Models Unifying Biological and Natural Language |Text + BioMultiArxiv 2022Galactica: A Large Language Model for Science |Text + BioMultiEMNLP 2023BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations |Text + BioMultiArxiv 2023DARWIN Series: Domain Specific Large Language Models for Natural Science |Text + BioMultiArxiv 2023BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicine |Text + BioMultiArxiv 2023(StructChem) Structured Chemistry Reasoning with Large Language Models |Text + BioMultiNature Communications 2023(BioTranslator) Multilingual translation for zero-shot biomedical classification using BioTranslator |Text + BioMultiICLR 2024Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models |Text + BioMultiICLR 2024(ChatDrug) ChatGPT-powered Conversational Drug Editing Using Retrieval and Domain Feedback |Text + BioMultiICLR 2024BioBridge: Bridging Biomedical Foundation Models via Knowledge Graphs |Text + BioMultiAAAI 2024(KEDD) Towards Unified AI Drug Discovery with Multiple Knowledge Modalities |Text + BioMultiAAAI 2024（Otter Knowledge) Knowledge Enhanced Representation Learning for Drug Discovery |Text + BioMultiArxiv 2024ChatCell: Facilitating Single-Cell Analysis with Natural Language |Text + BioMultiICML 2024LangCell: Language-Cell Pre-training for Cell Identity Understanding |Text + BioMultiACL 2024BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuning |Text + BioMultiArxiv 2024MolBind: Multimodal Alignment of Language, Molecules, and Proteins |Text + BioMultiArxiv 2024Uni-SMART: Universal Science Multimodal Analysis and Research Transformer |Text + BioMultiICML 2024Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domains |Text + BioMultiArxiv 2024An Evaluation of Large Language Models in Bioinformatics Research |Text + BioMultibioRxiv 2024SciMind: A Multimodal Mixture-of-Experts Model for Advancing Pharmaceutical Sciences |Text + BioMultiNeurIPS 2024SciDFM: A Large Language Model with Mixture-of-Experts for Science |Text + BioMultiSci China Inf Sci 2024ChemDFM-X: Towards Large Multimodal Model for Chemistry |Text + BioMultiArxiv 2025(NatureLM) Nature Language Model: Deciphering the Language of Nature for Scientific Discovery |Text + BioMultiCell Rep Phys Sci 2024[(ChemDFM) Developing ChemDFM as a large language foundation model for chemistry](https://www.cell.com/cell-reports-physical-science/fulltext/S2666-3864(25) |Text + BioMultiInformation Fusion 2025(KFPPIMI) Improving protein–protein interaction modulator predictions via knowledge-fused language models |Text + BioMultiArxiv 2025STELLA: Towards Protein Function Prediction with Multimodal LLMs Integrating Sequence-Structure Representations |Text + BioMultiArxiv 2025(CAFT) Improving Large Language Models with Concept-Aware Fine-Tuning |Text + BioMultiArxiv 2025InstructPro: Natural Language Guided Ligand-Binding Protein Design |Text + BioMultiNMI 2025(InstructBioMol) Advancing biomolecular understanding and design following human instructions |Text + BioMultibioRxiv 2025DrugLM: A Unified Framework to Enhance Drug-Target Interaction Predictions by Incorporating Textual Embeddings via Language Models |Text + BioMultiArxiv 2025Intern-S1: A Scientific Multimodal Foundation Model |Text + BioMultiArxiv 2025Chem3DLLM: 3D Multimodal Large Language Models for Chemistry |Text + BioMultiArxiv 2025SciReasoner: Laying the Scientific Reasoning Ground Across Disciplines |Text + BioMultiArxiv 2025MolChord: Structure-Sequence Alignment for Protein-Guided Drug Design |
Pre-training-TextPubMedPre-training-TextbioRxivPre-training-TextMedRxivPre-training-TextS2ORCPre-training-TextMIMICPre-training-TextUF HealthPre-training-TextElsevier CorpusPre-training-TextEurpoe PMCPre-training-TextLibreTextPre-training-TextNLM literature archivePre-training-TextGAP-ReplayPre-training-MoleculeZINCPre-training-ProteinUniProtPre-training-Molecule, BioassayChEMBLPre-training-Molecule, BioassayGIMLETPre-training-Text, MoleculemCLMPre-training-Text, MoleculePEIT-GENPre-training-Text, Molecule, IUPAC, etcPubChemPre-training-Text, ProteinInterPTPre-training-Text, Protein, etcSTRINGPre-training-Text, MoleculeMolTextNetPre-training-Text, ProteinProtAnnotationPre-training-Text, ProteinProteinKG25Pre-training-Text, ProteinProtAnnoPre-training-Text, ProteinProtDescribePre-training-Text, ProteinProteinaligner pretrain dataPre-training-Text, ProteinProTADPre-training-Text, ProteinMolinst-SwissProtCLAPFine-tuning-TextBLURBFine-tuning-TextPubMedQAFine-tuning-TextSciQFine-tuning-TextBioASQFine-tuning-TextMedC-IFine-tuning-MoleculeMoleculeNetFine-tuning-MoleculeMoleculeACEFine-tuning-MoleculeTDCFine-tuning-MoleculeUSPTOFine-tuning-MoleculeGraph2graphFine-tuning-MoleculePubChem Molecule OptimizationFine-tuning-ProteinPEERFine-tuning-ProteinFLIPFine-tuning-ProteinTAPEFine-tuning-Text, MoleculePubChemSTMFine-tuning-Text, MoleculePseudoMD-1MFine-tuning-Text, MoleculeChEBI-20Fine-tuning-Text, MoleculeChEBI-20-MMFine-tuning-Text, MoleculeChEBL-diaFine-tuning-Text, MoleculePCdesFine-tuning-Text, MoleculeMoMuFine-tuning-Text, MoleculePubChemQAFine-tuning-Text, Molecule3D-MolTFine-tuning-Text, MoleculeMoleculeQAFine-tuning-Text, MoleculeMolTextQAFine-tuning-Text, MoleculeMolOpt-InstructionsFine-tuning-Text, MoleculeSMolInstructFine-tuning-Text, MoleculePubChem324kFine-tuning-Text, MoleculeKnowMol-100KFine-tuning-Text, MoleculeMolQAFine-tuning-Text, MoleculeMolTCFine-tuning-Text, MoleculeMol-LLaMA-InstructFine-tuning-Text, MoleculePEIT-LLMFine-tuning-Text, MoleculeSmileyLlamaFine-tuning-Text, MoleculeMuMOInstructFine-tuning-Text, MoleculeHiPubChemFine-tuning-Text, MoleculeExDDIFine-tuning-Text, MoleculeMMPFine-tuning-Text, MoleculeSLM4CRP_with_RTsFine-tuning-Text, Molecule, etcSciAssessFine-tuning-Text, Molecule, etcDrugBankFine-tuning-Text, Molecule, etcDARWINFine-tuning-Text, ProteinSwissProtFine-tuning-Text, ProteinUniProtQAFine-tuning-Text, ProteinInstructProteinFine-tuning-Text, ProteinOpen Protein InstructionsFine-tuning-Text, ProteinPDB-QAFine-tuning-Text, ProteinProteinQAFine-tuning-Text, ProteinProtein2Text-QAFine-tuning-Text, ProteinCAMEOFine-tuning-Text, ProteinSwiss-Prot Curated TripletsFine-tuning-Text, ProteinProCyon-InstructFine-tuning-Text, Molecule, ProteinMol-InstructionsFine-tuning-Text, Molecule, ProteinBiology-InstructionsBenchmark-TextSciEvalBenchmark-TextBioInfo-BenchBenchmark-TextBioMedEvalBenchmark-Text, MoleculeChemLLMBenchBenchmark-Text, MoleculeAI4ChemBenchmark-Text, MoleculeGPTChemBenchmark-Text, Molecule, etcStructChemBenchmark-Text, MoleculeS²-BenchBenchmark-Text, MoleculeMotifHalluBenchmark-Text, MoleculeMolCap-ArenaBenchmark-Text, MoleculeChemCoTBenchBenchmark-Text, MoleculeMolLangBenchBenchmark-Text, MoleculeMolErr2FixBenchmark-Text, MoleculeMolPuzzleBenchmark-Text, ProteinProteinLMBenchBenchmark-Text, ProteinProt-Inst-OOD
- Multimodal Pre-training Models of Molecular Representation for Drug Discovery NSF_2511
- A Comprehensive Survey of Multimodal LLMs for Scientific Discovery VLM4RWD@NeurIPS 2511
- A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers Arxiv_2508
- A Survey of Large Language Models for Text-Guided Molecular Discovery: from Molecule Generation to Optimization Arxiv_2505
- From Generalist to Specialist: A Survey of Large Language Models for Chemistry Arxiv_2412
- A Review of Large Language Models and Autonomous Agents in Chemistry Arxiv_2407
- A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery Arxiv 2406
- Bridging Text and Molecule: A Survey on Multimodal Frameworks for Molecule Arxiv 2403
- Bioinformatics and Biomedical Informatics with ChatGPT: Year One Review Arxiv 2403
- From Words to Molecules: A Survey of Large Language Models in Chemistry Arxiv 2402
- Scientific Language Modeling: A Quantitative Review of Large Language Models in Molecular Science Arxiv 2402
- Progress and Opportunities of Foundation Models in Bioinformatics Arxiv 2402
- Scientific Large Language Models: A Survey on Biological & Chemical Domains ACM Computing Surveys
- The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4 Arxiv 2311
- Transformers and Large Language Models for Chemistry and Drug Discovery Arxiv 2310
- Language models in molecular discovery Arxiv 2309
- What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks NeurIPS 2309
- Do Large Language Models Understand Chemistry? A Conversation with ChatGPT JCIM 2303
- A Systematic Survey of Chemical Pre-trained Models IJCAI 2023
- MultiPM4Drug
- Awesome-Scientific-Datasets-and-LLMs
- LLM4Chemistry
- LLMs-in-science
- Awesome-LLM-Centric-Molecular-Discovery
- LLM4ScientificDiscovery
- SLM4Mol
- Scientific-LLM-Survey
- Awesome-Bio-Foundation-Models
- Awesome-Molecule-Text
- LLM4Mol
- Awesome-Chemical-Pre-trained-Models
- Awesome-Chemistry-Datasets
- Awesome-Docking
This repository is contributed and updated by QizhiPei and Lijun Wu. If you have questions, don't hesitate to open an issue or ask me via [email protected] or Lijun Wu via [email protected]. We are happy to hear from you!
@article{pei2024leveraging,
title={Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Survey},
author={Pei, Qizhi and Wu, Lijun and Gao, Kaiyuan and Zhu, Jinhua and Wang, Yue and Wang, Zun and Qin, Tao and Yan, Rui},
journal={arXiv preprint arXiv:2403.01528},
year={2024}
}