Poster
1

From Zero to Hero: Harnessing Transformers for Biomedical Named Entity Recognition in Zero- and Few-shot Contexts

Avatar
Voices Powered byElevenlabs logo

Avatar
user

What is the main result of the paper?

AI: The main result of the paper is the presentation of a method for zero- and few-shot named entity recognition in the biomedical domain, which showed state-of-the-art results for zero-shot NER. The method is based on encoder-based transformer models and input data modification. The authors also evaluated nine common biomedical classes and suggested that the method can be applied to other domains with annotated data for a larger number of classes.
Avatar
user

Do you see any red flags in the paper that would be against the policy of a typical journal?

AI: Based on the given answers, it is not possible to determine if there are any red flags in the paper that would go against the policy of a typical journal.
Avatar
user

what is the novel approach to harnessing transformers in this paper

AI: The novel approach to harnessing transformers in this paper is the use of encoder-based transformer models and input data modification to factorize regular multi-class token classification into binary classification, allowing for zero- and few-shot named entity recognition in the biomedical domain.

Authors

Miloš Košprdić, Nikola Prodanović, Adela Ljajić, Bojana Bašaragin, Nikola Milošević

Abstract

Supervised named entity recognition (NER) in the biomedical domain depends on large sets of annotated texts with the given named entities. The creation of such datasets can be time-consuming and expensive, while extraction of new entities requires additional annotation tasks and retraining the model. To address these challenges, this paper proposes a method for zero- and few-shot NER in the biomedical domain. The method is based on transforming the task of multi-class token classification into binary token classification and pre-training on a large amount of datasets and biomedical entities, which allow the model to learn semantic relations between the given and potentially novel named entity labels. We have achieved average F1 scores of 35.44% for zero-shot NER, 50.10% for one-shot NER, 69.94% for 10-shot NER, and 79.51% for 100-shot NER on 9 diverse evaluated biomedical entities with fine-tuned PubMedBERT-based model. The results demonstrate the effectiveness of the proposed method for recognizing new biomedical entities with no or limited number of examples, outperforming previous transformer-based methods, and being comparable to GPT3-based models using models with over 1000 times fewer parameters. We make models and developed code publicly available.

Follow Us on

0 comments

Add comment
Recommended SciCasts