ESM-2 Fine-Tuned with PEFT LoRA
This is an attempt to use Parameter Efficient Fine Tuning (PEFT) with Low Rank Adaptation (LoRA) for the ESM-2 model esm2_t6_8M_UR50D. The Github can be found here. The model is intended to serve as an RNA binding site predictor for protein sequences. However, there seems to be something going wrong either during training, or at inference, as the model seems to only predict non-binding sites. The problem is likely in the training setup. Any feedback, comments, or discussion would be greatly appreciated.
Training procedure
Framework versions
- PEFT 0.4.0
- Downloads last month
- 6
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
HF Inference API was unable to determine this model’s pipeline type.