Protein Function Prediction

FLIP includes several benchmark datasets that contain a variety of protein sequences, each with a real-valued label indicating its "fitness" (how well the protein performs some particular function). The goal is to predict the fitness of a given protein sequence using the sequence. Different representations of protein sequences (e.g. learned embeddings from large language models) may prove helpful here. Some of the benchmark datasets (thermostability) contain a highly diverse set of sequences from many different protein families. Others (AAV, GB1) contain all sequences that are mutants of a single parent sequence. Each benchmark dataset features multiple "splits" -- different ways of train-test splitting the data to assess how well a model might generalize given limited information. The AAV benchmark, for example, features the "mutant vs designed" split in which a model is trained on randomly generated mutants and asked to predict the fitness of designed sequences, and the "seven vs many" split in which a model is trained on sequences with seven mutations and asked to make predictions for sequences with a different number of mutations.

Datasets. From the publicly accessible Structural Antibody Database (SAbDab), we collected a total of 7571 antibodyantigen complexes, with the sequence data in FASTA format and structural data in PDB format. Following previous studies [Pittala and Bailey-Kellogg, 2020], we used CD-HIT [Li and Godzik, 2006] to remove high-homology antibody and antigen sequences with the thresholds of 95% and 90% sequence identity, respectively. Subsequently, we excluded antibodies and antigens with any residue type rather than 20 naturally occurring types. Finally, we compiled a dataset consisting of 626 binding antibody-antigen pairs, including their sequences, structures, and corresponding interaction maps. Noteworthy, antibodies primarily bind to antigens through their CDR regions. Most researchers use Euclidean distance to define paratopes and epitopes, and we follow the usual way in our dataset: within the CDR regions/antigen, a residue is labeled as a paratope/epitope if the Euclidean distance between its backbone atom and any backbone atom on the other antigen/CDR regions is less than 4.5 ˚ A.

The PECAN dataset provides structural data for antibody-antigen interactions, specifically curated for paratope and epitope binding site prediction. It includes a diverse set of antibody-antigen complexes, ensuring a well-balanced and representative dataset for training and evaluating deep learning models in protein-protein interaction (PPI) tasks. Characteristics: - Focuses on antibody-antigen binding site prediction. - Derived from high-resolution X-ray crystallography structures. - Includes training, validation, and test splits. - Used as a benchmark dataset in multiple antibody binding site prediction studies. Motivation and Use Cases: - Facilitates the development of deep learning models for paratope and epitope identification. - Enables researchers to evaluate and compare methods for antibody-antigen interaction prediction. - Used in ParaSurf, a deep learning framework that achieves state-of-the-art results on this dataset.

To take advantage of the ever-increasing amount of structural data now available, we also trained Paragraph on a larger dataset. This new dataset was extracted from the Structural Antibody Database (SAbDab, Schneider et al., 2022) on March 31, 2022 and includes 1086 complexes which we divide into train, validation and test sets using a 60-20-20 split. Full details of both datasets are given in the Supplementary Information.

FLIP includes several benchmark datasets that contain a variety of protein sequences, each with a real-valued label indicating its "fitness" (how well the protein performs some particular function). The goal is to predict the fitness of a given protein sequence using the sequence. Different representations of protein sequences (e.g. learned embeddings from large language models) may prove helpful here. This sub-dataset (AAV) is a set of 201,426 training sequences and 82,583 test sequences in which the goal is to predict the fitness of mutants of the capsid protein from the adeno-associated virus (AAV). The training set proteins were designed, while the test set proteins are random mutants. The absolute value of the fitness is not important, but its ranking / relative value is -- protein designers would like to be able to pick a sequence with high fitness relative to those in the training set. Performance is therefore usually assessed using Spearman's r correlation coefficient.