Marco127/Argu_T1

SentenceTransformer based on sentence-transformers/multi-qa-mpnet-base-dot-v1

This is a sentence-transformers model finetuned from sentence-transformers/multi-qa-mpnet-base-dot-v1. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/multi-qa-mpnet-base-dot-v1
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Dot Product

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Marco127/Argu_T1")
# Run inference
sentences = [
    ' In the event of a disturbance, one polite request (warning) will\nbe given to reduce the noise. If our request is not followed, the guest will be asked to leave\nthe hotel without refund and may be charged Guest Compensation Disturbance Fee.',
    ' In the event of a disturbance, one polite request (warning) will\nbe given to reduce the noise. If our request is not followed, the guest will be asked to leave\nthe hotel without refund and may be charged Guest Compensation Disturbance Fee.',
    '\nWithout limiting the generality of the aforementioned, it applies to pay-to-view TV programmes or videos, as\nwell as telephone calls or any other expenses of a similar nature that is made from your room, you will be\ndeemed to be the contracting party.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Binary Classification

• Evaluated with BinaryClassificationEvaluator

Metric	Value
dot_accuracy	0.6746
dot_accuracy_threshold	49.0201
dot_f1	0.4933
dot_f1_threshold	35.0242
dot_precision	0.3293
dot_recall	0.9821
dot_ap	0.3294
dot_mcc	-0.0392

Training Details

Training Dataset

Unnamed Dataset

• Size: 672 training samples
• Columns: sentence1, sentence2, and label
• Approximate statistics based on the first 672 samples:

  	sentence1	sentence2	label
  type	string	string	int
  details	min: 11 tokens mean: 48.63 tokens max: 156 tokens	min: 11 tokens mean: 48.63 tokens max: 156 tokens	0: ~66.67% 1: ~33.33%

• Samples:

  sentence1	sentence2	label
  The pets can not be left without supervision if there is a risk of causing any damage or might disturb other guests.	The pets can not be left without supervision if there is a risk of causing any damage or might disturb other guests.	0
  Any guest in violation of these rules may be asked to leave the hotel with no refund. Extra copies of these rules are available at the Front Desk upon request.	Any guest in violation of these rules may be asked to leave the hotel with no refund. Extra copies of these rules are available at the Front Desk upon request.	0
  Consuming the products from the minibar involves additional costs. You can find the prices in the kitchen area.	Consuming the products from the minibar involves additional costs. You can find the prices in the kitchen area.	0

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 169 evaluation samples
• Columns: sentence1, sentence2, and label
• Approximate statistics based on the first 169 samples:

  	sentence1	sentence2	label
  type	string	string	int
  details	min: 13 tokens mean: 46.01 tokens max: 156 tokens	min: 13 tokens mean: 46.01 tokens max: 156 tokens	0: ~66.86% 1: ~33.14%

• Samples:

  sentence1	sentence2	label
  I understand and accept that the BON Hotels Group collects the personal information ("personal information") of all persons in my party for purposes of loyalty programmes and special offers. I, on behalf of all in my party, expressly consent and grant permission to the BON Hotels Group to: - collect, collate, process, study and use the personal information; and communicate directly with me/us from time to time, unless I have stated to the contrary below.	I understand and accept that the BON Hotels Group collects the personal information ("personal information") of all persons in my party for purposes of loyalty programmes and special offers. I, on behalf of all in my party, expressly consent and grant permission to the BON Hotels Group to: - collect, collate, process, study and use the personal information; and communicate directly with me/us from time to time, unless I have stated to the contrary below.	0
  However, in lieu of the above, any such goods will only be kept by us for 6 (six) months. At the end of which period, we reserve the right in our sole discretion to dispose thereof and you will have no right of recourse against us.	However, in lieu of the above, any such goods will only be kept by us for 6 (six) months. At the end of which period, we reserve the right in our sole discretion to dispose thereof and you will have no right of recourse against us.	0
  In cases where the hotel suffers damage (either physical, or moral) due to the guests’ violation of the above rules, it may charge a compensation fee in proportion to the damage. Moral damage may be for example disturbing other guests, thus ruining the reputation of the hotel.	In cases where the hotel suffers damage (either physical, or moral) due to the guests’ violation of the above rules, it may charge a compensation fee in proportion to the damage. Moral damage may be for example disturbing other guests, thus ruining the reputation of the hotel.	0

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 5
• warmup_ratio: 0.1
• fp16: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss	dot_ap
-1	-1	-	-	0.3294
2.3333	100	0.0004	0.0000	-
4.6905	200	0.0003	0.0000	-

Framework Versions

• Python: 3.11.11
• Sentence Transformers: 3.4.1
• Transformers: 4.48.3
• PyTorch: 2.5.1+cu124
• Accelerate: 1.3.0
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}