|
--- |
|
license: mit |
|
datasets: |
|
- HuggingFaceH4/ultrafeedback_binarized |
|
language: |
|
- en |
|
base_model: |
|
- princeton-nlp/Llama-3-Base-8B-SFT |
|
--- |
|
|
|
This is an aligned model based on princeton-nlp/Llama-3-Base-8B-SFT. The alignment loss has two parts, preference loss and a regularization term. |
|
The alignment loss does not require the assumption of an existing underlying reward model. Moreover, the distribution's ``softness" is adjustable via the softmax exponent, an algorithm parameter. |
|
|
|
More information will be added later. |
