Koina: Democratizing machine learning for proteomics research

Koina: a catalyst for accessible machine learning

Koina is a model repository designed to democratize access to ML models specifically for bottom-up proteomics analysis. By enabling the remote execution of ML models, Koina generates predictions in response to HTTPS requests, which are the standard protocols used for nearly all web traffic. This design choice enhances accessibility, allowing users to use ML models in any programming language without requiring specialized software and hardware. Such an approach not only facilitates the efficient sharing of centralized hardware resources but also permits easy horizontal scaling to accommodate varying demands from the user base (Fig. 1a). Furthermore, we maintain a public network of Koina instances at (https://koina.wilhelmlab.org) to minimize barriers to entry, distributing computational workloads across processing nodes hosted by various research institutions and spin-offs throughout Europe. The intention is to ensure just-in-time results delivery, thereby aligning with the principles of open and collaborative science. We envision scaling this public Koina-Network to meet the evolving needs of the research community, with research groups or institutions contributing additional hardware resources as needed, democratizing access to state-of-the-art ML models in the process. This decentralized architecture raises data security questions, particularly for sensitive clinical datasets. To address this, we provide a Docker image (Fig. 1a) that facilitates the deployment of a self-hosted instance within a secured network, ensuring no external communication. This option allows Koina to be utilized while supporting stringent data security measures.

The initial release of Koina specifically targets the analytical challenges of bottom-up proteomics. It offers access to models that predict a) fragmentation behavior in the gas phase under various conditions (MS/MS), b) chromatographic separation behavior in the liquid phase (RT), c) ion mobility of peptides, and d) peptide detectability. Specifically, Koina supports more than 30 models from the Prosit^9,14,15,16, MS2PIP^3,17,18,19, and AlphaPeptDeep⁶ (PeptDeep) ecosystems, along with standalone models like DeepLC²⁰, IM2Deep¹⁸, UniSpec²¹, and Chronologer²² (Fig. 1a). We selected these models not only for their widespread use within the community but also for their development by independent research groups and because they represent a diverse array of modeling approaches and training data choices, thus enabling better analysis across different types of experimental data.

A critical function of any model repository is to provide users with an overview of the state-of-the-art models available. Koina excels in this regard by offering concise, user-friendly documentation accessible online at (https://koina.wilhelmlab.org/docs), significantly simplifying the discovery of new proteomics ML models as well as enabling users to quickly judge which model is most suitable for their use case. Notably, this documentation is semi-automatically generated from an annotation file using OpenAPI and DOME standards^23,24, ensuring that ML developers can contribute new models without extensive web development knowledge. We aim to foster a collaborative community where machine learning developers can contribute their newly developed models to Koina. By doing so, we hope to increase accessibility and maximize the impact of these models, ultimately benefiting everyone working in proteomics.

One of the primary challenges users face when leveraging ML models is a lack of interoperability²⁵. Different developers often utilize unique input and output formats, complicating the process for users who must adapt their inputs accordingly. In the field of proteomics, this is particularly cumbersome for peptide sequences, given the multitude of available notation systems. Additionally, variations in supported post-translational modifications (PTMs) and peptide lengths across different models exacerbate the problem. To address these challenges, we developed a simple query interface (Fig. 1b) that standardizes interactions between all available models, based on the ProForma 2.0 format developed by the Proteomics Standards Initiative (PSI) in collaboration with members of the Consortium for Top-Down Proteomics (CTDP)^26,27. Ideally, users should not need to change their client-side code to use a new model. To achieve this, we first standardize the data types and formats for both inputs and outputs. Next, we focus on standardizing input formats; while this is generally easy for most data types, peptide sequences require special attention. We have created new pre-processing steps that convert the standard ProForma 2.0 sequence format into the specific formats needed by different models. For outputs, we aim to keep the original format to allow users to check their predictions against their original sources. We expect this approach will have a minimal effect on interoperability, as most metrics used in later data analysis are not significantly impacted by differences in spectrum normalization or retention times. This shared interface encapsulates the technically heterogeneous collections of models and their associated pre- and post-processing steps in independent computational units, abstracting unnecessary detail for end users in the form of a “workflow” or “execution graph” (Fig. 2). Pre- and post-processing steps can be written in Python, enabling developers to re-use most, if not all, of their code when publishing a model on Koina. Notably, the pre- and post-processing steps are designed as independent computational units, allowing for the re-use between models (Fig. 2) as well as the parallel execution of models, improving performance and inference latency. Consequently, Koina’s interface substantially improves interoperability between models, allowing rapid integration of all models available on Koina in third-party applications.

The encapsulated, directly executable models guarantee that dependencies are explicitly encoded, thereby establishing a solid foundation for the long-term reusability of ML models. We support this with a continuous integration (CI) pipeline using GitHub actions, allowing changes to pre- and post-processing scripts—such as performance optimizations or dependency updates—without introducing unintended effects on prediction reproducibility. Reproducibility is further supported via GitHub’s transparent change tracking and the release of separate Docker images for every version of the model. Version control is also facilitated through Zenodo, which stores the model files that are not tracked via git, ensuring that files cannot be altered without creating a new version. Additionally, Koina accommodates the simultaneous hosting of different versions of models, allowing users to access both the latest versions and previously released iterations. These design choices foster better reproducibility of published models while maintaining flexibility for publishing bug fixes or performance improvements. The importance of reproducibility in research and model development cannot be overstated. The importance of reproducibility in research and model development cannot be overstated^28,29. Reproducibility ensures that findings can be verified and trusted²⁹, which is crucial in scientific work where decisions may rely on model predictions²⁵. It fosters transparency and accountability, enabling other researchers to build upon existing work with confidence. A clear commitment to reproducibility provides a safeguard against errors and discrepancies that may arise during implementation or deployment²⁹. Hence, our emphasis on reproducibility not only strengthens our own work but also contributes to the larger scientific community by fostering shared knowledge and collaborative advancement.

Leveraging Koina to integrate ML in proteomics software

The language-agnostic nature of Koina allows it to be accessed using multiple different methods depending on the user’s specific needs, enabling many more people to utilize ML to assist with their data analysis than what would be possible without it. Koina simplifies the client’s role by managing all of the model-related logic on the server, leaving the client responsible only for minimal tasks related to sending HTTP/S requests, such as input formatting, batching, request preparation, and error handling. To further improve usability, we developed client packages for two of the most widely used data science languages, Python (KoinaPy available via PyPI) and R (KoinaR available via Bioconductor). These client packages allow users to generate predictions with as few as four lines of code (Supplementary Fig. 1a) in a readily usable format (Supplementary Fig. 1b). Additionally, we provide example code snippets at (https://koina.wilhelmlab.org), demonstrating how to obtain predictions for these client packages and other popular languages like Java, C# and JavaScript.

Koina has already been integrated with several popular proteomics data analysis software, such as Skyline³⁰, EncyclopeDIA³¹, Oktoberfest² and FragPipe^4,32,33 (Fig. 3a). These integrations enable users to apply deep learning in application-oriented downstream tasks like spectral library generation or PSM rescoring. Notably, due to the consistent interface between models, integration with Koina allows seamless interoperability between all models currently available on Koina and models released in the future as well.

Oktoberfest is an open-source Python package developed for generating spectral libraries, conducting collision energy (CE) calibration, and executing PSM rescoring. Initially, these features were part of a web service associated with Prosit⁹. In contrast to the original web service, Oktoberfest presents notable advantages: it is open-source, accommodates multiple search engines, and allows for the parallel rescoring of multiple searches. Koina has largely enabled the transition of this formerly closed-source workflow into an open-source Python package. This transition also broadens Oktoberfest’s capabilities by supporting predictions from various machine learning models, rather than being restricted to only Prosit models, thus enhancing its versatility and performance. For users aiming to integrate machine learning into their proteomics workflows, Oktoberfest offers both an application programming interface (API) and a command line interface (CLI), promoting seamless integration. However, some familiarity with Python is required. Available on GitHub and installable via PyPI, Oktoberfest ensures straightforward access for researchers and developers.

Koina is also integrated into EncyclopeDIA as a generic predicted library generation tool to support a wide variety of DDA and DIA library search engines. Through the application, users supply a FASTA database as well as parameters such as digestion enzyme constraints and desired charge states. Each peptide/charge state pair that passes the specified constraints is sent to Koina to predict retention times, ion mobility, and fragmentation patterns in separate network packets. The prediction interface is batched, highly threaded, and robustly engineered to account for network errors common in large proteome-sized prediction tasks. EncyclopeDIA also provides a wide variety of library conversion options that support a plethora of external tools, including Spectronaut, DIA-NN⁷, Scribe³⁴, SpectraST³⁵, MSPepSearch³⁶, and Skyline/BiblioSpec³⁷. Unique to DIA workflows, collision energies can be automatically adjusted based on peptide charge state to account for all peptides being co-fragmented, assuming a single normalized collision energy. Lastly, Koina is supported at both the command line and inside the EncyclopeDIA graphical user interface (GUI) to simplify both automated and manual workflows.

In contrast, the integration of Koina into Skyline enables users to predict MS/MS fragment ion intensities and peptide retention time (RT) for targeted peptides. This alleviates the need for time-consuming and potentially expensive DDA experiments to acquire spectral libraries. Predictions are made on the fly as needed to enable a transparent user experience. As a result, users can develop targeted assays without needing DDA runs and possibly even synthetic peptides for verification. While at the moment, only a subset of models available on Koina are supported and there is no possibility to set an alternative Koina server as a source for predictions, this integration nevertheless saves users time and money and makes Skyline a more powerful tool for proteomics research. Skyline also provides a seamless wrapper for EncyclopeDIA for direct DIA analysis, which uses Koina to predict FASTA-sized libraries as needed. Both Skyline and EncyclopeDIA are open-source tools under the Apache 2 license, making them convenient templates for efficiently implementing Koina using C# and Java.

FragPipe is a comprehensive computational platform for analyzing proteomics data. It is powered by a fast MSFragger³² database search engine capable of both conventional peptide identification searches and “open” post-translational modification analyses. FragPipe includes the MSBooster module for DL-based rescoring of PSMs coupled with Percolator³⁸ and Philosopher³³ for protein inference and false discovery rate (FDR) filtering. FragPipe provides complete data analysis workflows for processing quantitative proteomics data generated using all major DDA and DIA-based experimental strategies. We selected FragPipe to provide a detailed demonstration of how an existing computational pipeline, and more specifically, the open-source MSBooster module of FragPipe, can be integrated with Koina.

Even prior to this work, FragPipe included an ML-based rescoring tool called MSBooster⁴. However, the integration with Koina now enables MSBooster to access many more models for prediction besides the previous default, the prediction module of DIA-NN⁷, allowing users to compare and, more importantly, mix and match models all within the same software environment. The FragPipe GUI Validation Tab has been extended to include a Koina section (Supplementary Fig. 2a). MSBooster takes as input the list of PSMs reported by MSFragger (in.pin format). It filters out peptides too long for prediction, and if there are peptides with PTMs unsupported by a model, the unmodified version is queried, and its fragment m/z values are adjusted as previously described⁴. MSBooster can query and parse predictions from Koina for thousands of peptides per second using the default public server at (https://koina.wilhelmlab.org) (Fig. 3c, and Supplementary Data 1). When timing on a high-performance Linux server, the DIA-NN prediction module runs faster than the Koina models. When timing the models on a consumer laptop, DIA-NN predicts RT slower than the Koina models and PeptDeep even predicts MS/MS spectra faster than DIA-NN, suggesting that DIA-NN’s performance is more dependent on the local hardware compared to Koina’s performance. These results are consistent with an in-depth throughput and latency benchmark we performed. Depending on the used model, available Koina instances, and client-side concurrency, a throughput of more than 200,000 predictions per second (median > 30,000) was achieved (Supplementary Fig. 3−6), with only a subset of the available computational resources available to the full Koina-Network. The scaling behavior we observed also shows that with the addition of additional hardware, higher performance can be achieved for most models (Supplementary Fig. 7−14). To account for differences between the learned normalized collision energy (NCE) patterns of the predictors and the NCE settings on individual instruments, MSBooster performs an NCE calibration step when using Koina models (Fig. 3b, Methods). To assess the quality of the rescoring, MSBooster generates multiple plots, including a plot showing the rate of Koina prediction (Supplementary Fig. 2b) and a plot showing the distributions of the similarity scores for the PSMs at each NCE value (Supplementary Fig. 2c).

A predicted library is saved as a Mascot Generic Format (MGF) file if a Koina model was used or as a binary file if DIA-NN’s prediction module was used. The libraries can be loaded into FragPipe-PDV³⁹ viewer of FragPipe to compare experimental and predicted spectra (Fig. 3d). Here, it becomes easier to visualize the differences between models’ predictions. For example, DIA-NN does not predict intensities for fragments shorter than three amino acids long, as they are less informative for DIA-NN’s peptide-centric approach. Figure 3d shows one example in which the pyroglutamated peptide [U:Gln->pyro-Glu]-QHSQGGFVV exhibits a strong y1 fragment intensity, which is matched by PeptDeep’s prediction but not by DIA-NN’s, resulting in a nearly 0.2 drop in unweighted spectral entropy, which ranges from 0 to 1.

PSM Rescoring using Koina and MSBooster

Koina in MSBooster enables users to compare model performance systematically by keeping all other upstream (MSFragger) and downstream (Percolator rescoring using MSBooster generated.pin files, protein inference and FDR filtering with Philosopher) data processing steps and parameters the same. Because of differences in model architectures and training data, we hypothesize that the optimal model for rescoring will depend on the specifics of the dataset. To investigate this, we considered various experiments to pinpoint patterns between specific proteomics data types and optimal models. Specifically, we considered phosphoproteomics data from Arabidopsis thaliana⁴⁰ and mouse pancreatic ductal adenocarcinoma cell lines⁴¹, DDA and DIA human leukocyte antigen (HLA) data across different instruments^42,43,44,45, DIA data generated on the Orbitrap Astral mass spectrometer (Thermo Scientific)^46,47, and TMT 11 plex-labeled data^16,48 (Supplementary Data 2). We considered the spectral similarity and RT difference features calculated by MSBooster individually and together for PSM rescoring and found that generally, PeptDeep performs best for phosphoproteomics data (Supplementary Fig. 15 & 16), Prosit for HLA data, and DIA-NN for Astral data. Prosit and DIA-NN’s prediction module performed comparably on the TMT datasets (Supplementary Fig. 17 & 18). We summarize our results on the HLA data here as an example use case where PSM rescoring particularly benefited from Koina’s models.

It is well established that HLA immunopeptidomics data benefits greatly from PSM rescoring with predicted libraries^{3,6,14,15,49,50,51}, in part because the larger nonspecific peptide search space leads to many more ambiguous identifications⁵². HLA peptides fall into one of two classes based on which class of major histocompatibility complex (MHC) molecule they bind, similarly designated class I or II. They have been widely studied because of increased interest in HLA peptides for use as immunotherapeutics^53,54. Recent developments in DIA and instrumentation (such as the development of the Bruker timsTOF and Thermo Scientific Orbitrap Astral instruments) have provided deeper and more reproducible views of the HLA peptidome^17,55. We benchmarked all models on a variety of HLA datasets (Fig. 4a-g, Supplementary Data 2). Overall, a combination of the DIA-NN’s prediction module and Prosit RT models performed the best over baseline, while the Prosit models excelled in the MS/MS category. The largest gain enabled by MSBooster occurred in a DIA dataset⁴², where Prosit rescoring added 53.2% more peptide identifications. In addition, we found that combining the best individual RT and MS/MS models outperformed the current FragPipe default of using DIA-NN’s predictions to compute both the MS/MS and RT features for all datasets except for one of the DIA datasets (Fig. 4e). Interestingly, we also note that HLA class I datasets benefited more from predictions than HLA class II datasets, which aligns with previous literature¹⁴ (Fig. 4a–c, f, g, and Supplementary Data 2). This is evident across all RT and MS/MS models. A potential reason for this stems from HLA I peptides being more often singly charged compared to HLA II peptides: singly charged peptides have decreased fragmentation efficiency, causing lower performance of traditional peak counting-based scoring functions.

Our findings highlight how specialized models outperform generic ones. First, we note the differences in performance when rescoring spectra from different fragmentation methods. Distinct collision-induced dissociation (CID, resonance-type CID) and higher-energy collision dissociation (HCD, beam-type CID) models exist for Prosit and MS2PIP. However, only the HCD MS2PIP model was currently available on Koina at the time of writing, while the CID model is on the MS2PIP server at (https://iomics.ugent.be/ms2pip/19). Therefore, our comparison is limited to using the Prosit models. Marcu et al⁴³. used CID to fragment precursors in their class I sample (Fig. 4a). We found that the Prosit CID model performed best here, achieving 15% more peptide identifications over baseline and outperforming both its Prosit HCD counterpart (11%) and DIA-NN (6%). Likewise, the Prosit HCD model performed better on HCD fragmentation datasets^42,43 (Fig. 4b, c). Whichever model is superior is also evident in the distribution of target PSMs’ spectral similarity score distribution, which is shifted to lower similarity when the wrong fragmentation model is used (Supplementary Fig. 19).

Specialized models also improve the rescoring of MS/MS spectra from different mass spectrometers. Strangely, although Prosit timsTOF improves over DIA-NN and the Prosit HCD model trained on Orbitrap data for both class I (Prosit timsTOF: 19.1% improvement over baseline, Prosit HCD: 18%, DIA-NN: 12.8%) and II data from Phulphagar et al⁴⁵., this does not extend to the PeptDeep models, where “Lumos” PeptDeep predictions helped identify more peptides than “timsTOF” PeptDeep predictions did (Fig. 4f, g). We believe this discrepancy may arise because PeptDeep and Prosit were trained on spectra of different NCEs. While Prosit timsTOF was trained on energies ranging from 20–70 eV, PeptDeep timsTOF was only trained on 32–52 eV⁶. More importantly, all timsTOF spectra were annotated as 30 eV during the PeptDeep training phase. Though spectra do differ across Orbitrap and timsTOF instruments, there are certain hotspots across energy levels where they are highly correlated^15,56. Specifically, precursors fragmented with lower energies (around 20% and 20 eV on Orbitrap and timsTOF instruments, respectively) produce nearly identical spectra across instruments. Phulphagar et al. employed a scheme where precursor ion mobility and fragmentation energy were inversely correlated, with energy ranging from 55–10 eV. Though MSBooster’s NCE calibration step seeks the best NCE value for spectral prediction, the lower bound of this experimental NCE range is far below that of what PeptDeep timsTOF was trained on, potentially resulting in suboptimal predictions. However, because low-energy fragmented spectra are similar between Orbitrap and timsTOF, the PeptDeep model predicting for a Lumos instrument performs better since it was trained on spectra with HCD energy as low as 20%. Indeed, when we rescored another dataset acquired on a timsTOF with similar ion mobility settings as the data PeptDeep was trained on⁵⁷, both PeptDeep and Prosit timsTOF models outperformed their Orbitrap counterparts (Supplementary Fig. 20). Koina outperformed DIA-NN’s predictions the most on the CID and timsTOF datasets. This improved ability to identify peptides provides a compelling reason for incorporating the many models on Koina into FragPipe, as models trained on diverse datasets often improve upon DIA-NN’s predictions.

A heuristic approach to optimize model selection in MSBooster

We have observed empirically which models work best for phosphorylation, HLA, Astral DIA, and TMT data. However, it is outside the scope of this manuscript to exhaustively consider all types of proteomics data, and even within the same type of data, there is variability in the best-performing models. As Koina continues to grow and support more models, it may become overwhelming for users to determine which combinations of models work best for PSM rescoring in their data. To make it easier for the users to maximize peptide identifications, we have incorporated an optional module in MSBooster that attempts to determine the best MS/MS and RT model for a dataset. Importantly, the selected models may be from different frameworks (e.g., DeepLC for RT and Prosit for MS/MS), a benefit conferred by Koina’s one-stop shop for all these models.

Rather than having users rescore their entire dataset using each available model, we implemented a heuristic approach that selects models based on the agreement between their predicted and experimental values of a subset of the data (Methods). When developing this algorithm, we initially chose the MS/MS model with the highest median similarity and the RT model with the lowest median RT difference but found that the algorithm would sometimes report the wrong models, according to our empirical results. Figure 5 depicts one such case for an HLA DDA class I dataset⁴³. Though our algorithm correctly chose Prosit CID as the best MS/MS model (Fig. 5a), it also chose Prosit rather than DIA-NN as the best RT model (Fig. 5b). Though Prosit had the lowest median delta RT (1.38 minutes) and DIA-NN’s prediction model had the largest (2.03 minutes), out of the models tested, it was actually DIA-NN’s RT predictions that resulted in the most peptides identified on average, with a 6.1% increase over baseline. Compare this to Prosit RT, which performed the worst, with a 3.8% improvement over baseline and 312 fewer peptides than when using DIA-NN’s predictions. Supplementary Fig. 21 shows this same analysis for the other datasets.

It was clear that the median metric often produced incorrect predictions for the RT model search, so we tested several other metrics and summarized their performance across all datasets used here (Fig. 5c-d, Methods). We found that the “top consensus” metric performed well in finding the optimal RT model. By taking the ten largest delta RT values, the algorithm could better determine the best-performing RT model than the median method. Specifically, the top 10 consensus methods achieved a “heuristic summary score” of 0.899, while the median method scored 0.793 (Methods). While the best MS/MS method was achieved by taking the bottom 100 lowest spectral similarity scores (0.981 heuristic summary score), the median similarity method performed very similarly (0.978), and the methods chose different models in only two out of the thirteen datasets. Therefore, there was insufficient justification to move away from the median method, so we continue to use it to determine the best MS/MS model. The final metrics chosen were “median” for MS2 models and “top consensus” with 10 PSMs for RT models.

We also considered PSM rescoring using all models together. We calculated a separate score for each applicable model and used them together in Percolator rescoring. Using multiple MS/MS and RT models resulted in even more peptides than when using a single model each for MS/MS and RT (Supplementary Fig. 22). Single models recovered between 42 and 86% of the total peptides added on top of the default DIA-NN’s prediction by multiple model mode. This suggests that for most cases here, a single well-chosen model identified the plurality of peptides from multiple model mode without spending the time to predict all peptide candidates multiple times. A mode leveraging the predictive power of multiple Koina models simultaneously can be added to MSBooster in the future.