Pre-Training Protein Bi-level Representation Through Span Mask Strategy On 3D Protein Chains
원문 초록
In recent years, there has been a surge in the development of 3D structure-based pre-trained protein models, representing a significant advancement over pre-trained protein language models in various downstream tasks. However, most existing structure-based pre-trained models primarily focus on the residue level, i.e., alpha carbon atoms, while ignoring other atoms like side chain atoms. We argue that modeling proteins at both residue and atom levels is important since the side chain atoms can also be crucial for numerous downstream tasks, for example, molecular docking. Nevertheless, we find that naively combining residue and atom information during pre-training typically fails. We identify a key reason is the information leakage caused by the inclusion of atom structure in the input, which renders residue-level pre-training tasks trivial and results in insufficiently expressive residue representations. To address this issue, we introduce a span mask pre-training strategy on 3D protein chains to learn meaningful representations of both residues and atoms. This leads to a simple yet effective approach to learning protein representation suitable for diverse downstream tasks. Extensive experimental results on binding site prediction and function prediction tasks demonstrate our proposed pre-training approach significantly outperforms other methods. Our code will be made public.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
CovDocker: Benchmarking Covalent Drug Design with Tasks, Datasets, and Solutions
PocketVina Enables Scalable and Highly Accurate Physically Valid Docking through Multi-Pocket Conditioning
PoseX: AI Defeats Physics Approaches on Protein-Ligand Cross Docking
Fast and Accurate Blind Flexible Docking
Scaling Structure Aware Virtual Screening to Billions of Molecules with SPRINT
QuickBind: A Light-Weight And Interpretable Molecular Docking Model
토론
이 논문과 연결된 포럼 글이 아직 없습니다.