In the evolving landscape of microbiology and microbiome analysis, the integration
of machine learning is crucial for understanding complex microbial interactions, and
predicting and recognizing novel functionalities within extensive datasets. However,
the effectiveness of these methods in microbiology faces challenges due to the complex
and heterogeneous nature of microbial data, further complicated by low signal-to-noise
ratios, context-dependency, and a significant shortage of appropriately labeled datasets.
This study introduces the ProkBERT model family, a collection of large language models,
designed for genomic tasks. It provides a generalizable sequence representation for
nucleotide sequences, learned from unlabeled genome data. This approach helps overcome
the above-mentioned limitations in the field, thereby improving our understanding
of microbial ecosystems and their impact on health and disease.MethodsProkBERT
models are based on transfer learning and self-supervised methodologies, enabling
them to use the abundant yet complex microbial data effectively. The introduction
of the novel Local Context-Aware (LCA) tokenization technique marks a significant
advancement, allowing ProkBERT to overcome the contextual limitations of traditional
transformer models. This methodology not only retains rich local context but also
demonstrates remarkable adaptability across various bioinformatics tasks.ResultsIn
practical applications such as promoter prediction and phage identification, the ProkBERT
models show superior performance. For promoter prediction tasks, the top-performing
model achieved a Matthews Correlation Coefficient (MCC) of 0.74 for E.
coli and 0.62 in mixed-species contexts. In phage identification, ProkBERT
models consistently outperformed established tools like VirSorter2 and DeepVirFinder,
achieving an MCC of 0.85. These results underscore the models' exceptional accuracy
and generalizability in both supervised and unsupervised tasks.ConclusionsThe
ProkBERT model family is a compact yet powerful tool in the field of microbiology
and bioinformatics. Its capacity for rapid, accurate analyses and its adaptability
across a spectrum of tasks marks a significant advancement in machine learning applications
in microbiology. The models are available on GitHub (https://github.com/nbrg-ppcu/prokbert)
and HuggingFace (https://huggingface.co/nerualbioinfo)
providing an accessible tool for the community.
