FMRAG: retrieval-augmented multimodal large language models for fisheries intelligence

IntroductionMultimodal large language models (MLLMs) have exhibited significant potential for fisheries analysis. However, their inherent hallucination issues in species identification, ecological activity interpretation, environmental assessment, and biomass estimation severely restrict their reliability and practical application in real‑world fisheries management.MethodsThis study proposes FMRAG, a fisheries‑oriented multimodal retrieval‑augmented generation framework to enhance factual grounding and domain adaptability. The framework retrieves visually similar fishery images and corresponding textual records within a unified vision‑language embedding space, and integrates retrieved multimodal evidence with query inputs via a cross‑modal fusion mechanism to exploit fine‑grained visual features and domain‑specific textual knowledge. Three fisheries‑tailored fine‑tuning tasks are further introduced: image‑text association learning, visual concentration learning, and retrieval‑augmented reasoning learning, to strengthen multi‑image reasoning and multimodal alignment.ResultsExperimental results on species identification and biomass estimation demonstrate that FMRAG consistently outperforms baseline MLLMs and text‑only RAG methods, effectively reducing hallucinations and improving predictive accuracy. The proposed framework also shows superior performance in rare‑species recognition, temporal stability, and confidence calibration, and can be successfully transferred to models originally trained on single‑image inputs.DiscussionFMRAG provides an effective and practical solution for constructing trustworthy multimodal intelligence systems, supporting reliable and robust applications in fisheries monitoring and management.