Welcome to the FARM!

Framework for Adapting Representation Models

What is it?

FARM makes Transfer Learning with BERT & Co simple, fast and enterprise-ready. It’s build upon transformers and provides additional features to simplify the life of developers: Parallelized preprocessing, highly modular design, multi-task learning, experiment tracking, easy debugging and close integration with AWS SageMaker.

With FARM you can build fast proof-of-concepts for tasks like text classification, NER or question answering and transfer them easily into production.

Core features

  • Easy fine-tuning of language models to your task and domain language

  • Speed: AMP optimizers (~35% faster) and parallel preprocessing (16 CPU cores => ~16x faster)

  • Modular design of language model and prediction heads

  • Switch between heads or just combine them for multitask learning

  • Full Compatibility with transformers’ models and model hub

  • Smooth upgrading to newer language models

  • Integration of custom datasets via Processor class

  • Powerful experiment tracking & execution

  • Checkpointing & Caching to resume training and reduce costs with spot instances

  • Simple deployment and visualization to showcase your model

Task

BERT

RoBERTa

XLNet

ALBERT

DistilBERT

XLMRoBERTa

Text classification

x

x

x

x

x

x

NER

x

x

x

x

x

x

Question Answering

x

x

x

x

x

x

Language Model Fine-tuning

x

Text Regression

x

x

x

x

x

x

Multilabel Text classif.

x

x

x

x

x

x

Extracting embeddings

x

x

x

x

x

x

LM from scratch (beta)

x

Indices and tables