---
license: apache-2.0
inference: false
datasets:
- imdb
language:
- en
pipeline_tag: text-classification
---

# Perceiver IO sentiment analysis model (IMDb)

This model is a Perceiver IO sentiment analysis model fine-tuned on the [IMDb](https://huggingface.co/datasets/imdb) 
dataset. It is a [training example](https://github.com/krasserm/perceiver-io/blob/main/docs/training-examples.md#sentiment-analysis)
of the [perceiver-io](https://github.com/krasserm/perceiver-io) library.

## Model description

The configuration of the encoder is equal to the encoder configuration of [krasserm/perceiver-io-mlm-imdb](https://huggingface.co/krasserm/perceiver-io-mlm-imdb),
the generic decoder is configured with a classification output adapter (see [building blocks](https://github.com/krasserm/perceiver-io/blob/main/docs/building-blocks.md)).

## Intended use and limitations

The model can be used for sentiment analysis of movie reviews.

## Model training

Training details are described [here](https://github.com/krasserm/perceiver-io/blob/main/docs/training-examples.md#sentiment-analysis).
Encoder weights are initialized with the encoder weights of [krasserm/perceiver-io-mlm-imdb](https://huggingface.co/krasserm/perceiver-io-mlm-imdb)
decoder weights are randomly initialized. In a first step, only decoder weights are trained, with encoder weights frozen.
In a second step, all weights are fine-tuned. Training was done with [PyTorch Lightning](https://www.pytorchlightning.ai/index.html) 
and the resulting checkpoint was converted to this 🤗 model with a library-specific [conversion utility](#checkpoint-conversion).

## Usage examples

To use this model you first need to [install](https://github.com/krasserm/perceiver-io/blob/main/README.md#installation) 
the `perceiver-io` library with extension `text`.

```shell
pip install perceiver-io[text]
```

Then the model can be used with PyTorch. Either use the model and tokenizer directly

```python
from transformers import AutoModelForSequenceClassification, AutoTokenizer
from perceiver.model.text import classifier  # auto-class registration

repo_id = "krasserm/perceiver-io-txt-clf-imdb"

model = AutoModelForSequenceClassification.from_pretrained(repo_id)
tokenizer = AutoTokenizer.from_pretrained(repo_id)

mini_reviews = [
    "I've seen this movie yesterday and it was really boring.",
    "I can recommend this movie to all fantasy movie lovers.",
]

encoding = tokenizer(mini_reviews, padding=True, return_tensors="pt")
logits = model(**encoding).logits

predictions = logits.argmax(dim=-1)

for r, p in zip(mini_reviews, predictions.numpy()):
    print(f"{r} ({model.config.id2label[p]})")
```
```
I've seen this movie yesterday and it was really boring. (NEGATIVE)
I can recommend this movie to all fantasy movie lovers. (POSITIVE)
```

or use a `sentiment-analysis` pipeline:

```python
from transformers import pipeline
from perceiver.model.text import classifier  # auto-class registration

repo_id = "krasserm/perceiver-io-txt-clf-imdb"

mini_reviews = [
    "I've seen this movie yesterday and it was really boring.",
    "I can recommend this movie to all fantasy movie lovers.",
]

sentiment_pipeline = pipeline("sentiment-analysis", model=repo_id)
predictions = sentiment_pipeline(mini_reviews)

for r, p in zip(mini_reviews, predictions):
    print(f"{r} ({p['label']})")
```
```
I've seen this movie yesterday and it was really boring. (NEGATIVE)
I can recommend this movie to all fantasy movie lovers. (POSITIVE)
```

## Checkpoint conversion

The `krasserm/perceiver-io-mlm-imdb` model has been created from a training checkpoint with: 

```python
from perceiver.model.text.classifier import convert_imdb_classifier_checkpoint

convert_imdb_classifier_checkpoint(
    save_dir="krasserm/perceiver-io-txt-clf-imdb",
    ckpt_url="https://martin-krasser.com/perceiver/logs-0.8.0/txt_clf/version_1/checkpoints/epoch=006-val_loss=0.156.ckpt",
    tokenizer_name="krasserm/perceiver-io-mlm",
    push_to_hub=True,
)
```