TY  - CHAP
U1  - Konferenzveröffentlichung
A1  - Balabin, Helena
A1  - Hoyt, Charles Tapley
A1  - Gyori, Benjamin M.
A1  - Bachman, John
A1  - Tom Kodamullil, Alpha
A1  - Hofmann-Apitius, Martin
A1  - Domingo-Fernández, Daniel
T1  - ProtSTonKGs: A Sophisticated Transformer Trained on Protein Sequences, Text, and Knowledge Graphs
T2  - SWAT4HCLS 2022: Semantic Web Applications and Tools for Health Care and Life Sciences. 13th International Conference on Semantic Web Applications and Tools for Health Care and Life Sciences, Virtual Event, Leiden, Netherlands, January 10th to 14th, 2022
N2  - While most approaches individually exploit unstructured data from the biomedical literature or structured data from biomedical knowledge graphs, their union can better exploit the advantages of such approaches, ultimately improving representations of biology. Using multimodal transformers for such purposes can improve performance on context dependent classication tasks, as demonstrated by our previous model, the Sophisticated Transformer Trained on Biomedical Text and Knowledge Graphs (STonKGs). In this work, we introduce ProtSTonKGs, a transformer aimed at learning all-encompassing representations of protein-protein interactions. ProtSTonKGs presents an extension to our previous work by adding textual protein descriptions and amino acid sequences (i.e., structural information) to the text- and knowledge graph-based input sequence used in STonKGs. We benchmark ProtSTonKGs against STonKGs, resulting in improved F1 scores by up to 0.066 (i.e., from 0.204 to 0.270) in several tasks such as predicting protein interactions in several contexts. Our work demonstrates how multimodal transformers can be used to integrate heterogeneous sources of information, paving the foundation for future approaches that use multiple modalities for biomedical applications.
KW  - Knowledge Graphs
KW  - Natural Language Processing
KW  - Transformers
KW  - Bioinformatics
KW  - Machine Learning
U6  - https://nbn-resolving.org/urn:nbn:de:hbz:1044-opus-62113
UN  - https://nbn-resolving.org/urn:nbn:de:hbz:1044-opus-62113
UR  - https://ceur-ws.org/Vol-3127/#paper-13
UR  - https://nbn-resolving.org/urn:nbn:de:0074-3127-1
SN  - 1613-0073
SS  - 1613-0073
SP  - 103
EP  - 107
S1  - 5
PB  - RWTH Aachen
CY  - Aachen, Germany
ER  -