---
base_model:
- MaziyarPanahi/calme-3.2-instruct-78b
- dfurman/CalmeRys-78B-Orpo-v0.1
library_name: transformers
tags:
- mergekit
- merge
---
# **Calme-Ties-78B**

Calme-Ties-78B is a 78-billion-parameter model merged using the TIES methodology, based on the Qwen2 architecture. It integrates two sub-base models: *calme-3.2-instruct-78B* by MaziyarPanahi and *CalmeRys-78B-Orpo-v0.1* by dfurman, which serves as the base model. The merging process assigns equal weight and density to both models, with additional parameters enabling normalization and int8 masking. The model operates using the *bfloat16* data type.

| Model    | Model Name                     | Model Link |
|----------|--------------------------------|------------|
| Base Model | CalmeRys-78B-Orpo-v0.1       | [CalmeRys-78B-Orpo-v0.1](https://huggingface.co/dfurman/CalmeRys-78B-Orpo-v0.1) |
| Model 1  | calme-3.2-instruct-78B        | [calme-3.2-instruct-78B](https://huggingface.co/MaziyarPanahi/calme-3.2-instruct-78b) |
| Model 2  | CalmeRys-78B-Orpo-v0.1        | [CalmeRys-78B-Orpo-v0.1](https://huggingface.co/dfurman/CalmeRys-78B-Orpo-v0.1) |

# **Merged Models**

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

# **Merge Method**

This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [dfurman/CalmeRys-78B-Orpo-v0.1](https://huggingface.co/dfurman/CalmeRys-78B-Orpo-v0.1) as a base.

# **Models Merged**

The following models were included in the merge:
* [MaziyarPanahi/calme-3.2-instruct-78b](https://huggingface.co/MaziyarPanahi/calme-3.2-instruct-78b)

# **Configuration**

The following YAML configuration was used to produce this model:

```yaml
models:
  - model: MaziyarPanahi/calme-3.2-instruct-78b
    parameters:
      weight: 1
      density: 1
merge_method: ties
base_model: dfurman/CalmeRys-78B-Orpo-v0.1
parameters:
  weight: 1
  density: 1
  normalize: true
  int8_mask: true
dtype: bfloat16
```