A reproducible nine-step protocol for fine-tuning large language models on heterogeneous bioinformatics knowledge sources

Domain-specific large language models (LLMs) can improve access to complex bioinformatics resources, but their development is often limited by fragmented source material, inconsistent data preparation, evaluation leakage, and incomplete reporting of training decisions. We present a reproducible nine-step protocol for converting heterogeneous bioinformatics knowledge sources into quality-controlled question–answer (QA) datasets and parameter-efficiently fine-tuned language models. The workflow covers task definition, source registration, content acquisition, text canonicalisation, source-grounded question–answer generation, filtering, semantic deduplication, group-preserving data partitioning, low-rank adaptation (LoRA)-based fine-tuning, and comparative evaluation against zero-shot, retrieval-augmented, and four-bit deployment baselines. We demonstrate the protocol using PRSGPT, a domain-specific assistant derived from resources covering 32 polygenic risk score (PRS) tools, and BioStarsGPT, a corresponding assistant derived from BioStars community discussions, yielding 28,378 and 154,282 retained QA pairs, respectively. In zero-shot evaluation, Qwen2.5-7B achieved the highest Recall-Oriented Understudy for Gisting Evaluation (ROUGE) scores for PRSGPT, while Mistral-7B achieved the highest or joint-highest performance on several other measures. Qwen2.5-7B was selected for the principal fine-tuned and deployment comparisons in both applications. For PRSGPT, the fine-tuned model exceeded QA-pair retrieval on eight of 13 non-exact-match metrics. Descriptive BioStarsGPT comparisons indicated that retrieval remained competitive in the more heterogeneous forum-derived setting. On an independent 176-question benchmark, PRSGPT answered 109 questions correctly (61.9%), compared with 108 for Google Gemini (61.4%) and 126 for Claude (71.6%). Four-bit deployment caused substantial, dataset-dependent performance loss. The protocol emphasises provenance, leakage-aware evaluation, independent factual assessment, and transparent reporting, with code, datasets, models, prompts, and evaluation outputs openly available.