FT Embeddings 3e
Collection
7 items
•
Updated
This is a sentence-transformers model finetuned from jxm/cde-small-v2 on the embeddings-train-semantic dataset. It maps sentences & paragraphs to a None-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
SentenceTransformer(
(0): Transformer({}) with Transformer model: ContextualDocumentEmbeddingTransformer
)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("Lauther/emb-cde-small-v2-3e")
# Run inference
sentences = [
'What columns store the uncertainty values?',
'How are flow computers and measurement systems related?\nFlow computers can have multiple systems assigned to them. However, a measurement system can only be assigned to one flow computer.\n\nDatabase terminology:\nIn the database, this relationship is referred to as:\n- Meter streams\n- Meter runs\n- Sections\n\nStorage of the relationship:\nThe relationship between a flow computer and its assigned measurement system is stored in a special table.\n\nUser context:\nWhen a user refers to a "meter stream," they are indicating that they are searching for a measurement system assigned to a specific flow computer.',
'What is uncertainty?\nUncertainty is a measure of confidence in the precision and reliability of results obtained from equipment or measurement systems. It quantifies the potential error or margin of error in measurements.\n\nTypes of uncertainty:\nThere are two main types of uncertainty:\n1. Uncertainty of magnitudes (variables):\n - Refers to the uncertainty of specific variables, such as temperature or pressure.\n - It is calculated after calibrating a device or obtained from the equipment manufacturer\'s manual.\n - This uncertainty serves as a starting point for further calculations related to the equipment.\n\n2. Uncertainty of the measurement system:\n - Refers to the uncertainty calculated for the overall flow measurement.\n - It depends on the uncertainties of the individual variables (magnitudes) and represents the combined margin of error for the entire system.\n\nKey points:\n- The uncertainties of magnitudes (variables) are the foundation for calculating the uncertainty of the measurement system. Think of them as the "building blocks."\n- Do not confuse the two types of uncertainty:\n - **Uncertainty of magnitudes/variables**: Specific to individual variables (e.g., temperature, pressure).\n - **Uncertainty of the measurement system**: Specific to the overall flow measurement.\n\nDatabase storage for uncertainties:\nIn the database, uncertainty calculations are stored in two separate tables:\n1. Uncertainty of magnitudes (variables):\n - Stores the uncertainty values for specific variables (e.g., temperature, pressure).\n\n2. Uncertainty of the measurement system:\n - Stores the uncertainty values for the overall flow measurement system.\n\nHow to retrieve uncertainty data:\n- To find the uncertainty of the measurement system, join the measurement systems table with the uncertainty of the measurement system table.\n- To find the uncertainty of a specific variable (magnitude), join the measurement systems table with the uncertainty of magnitudes (variables) table.\n\nImportant note:\nDo not confuse the two types of uncertainty:\n- If the user requests the uncertainty of the measurement system, use the first join (measurement systems table + uncertainty of the measurement system table).\n- If the user requests the uncertainty of a specific variable (magnitude) in a report, use the second join (measurement systems table + uncertainty of magnitudes table).',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
sentence1
, sentence2
, and score
sentence1 | sentence2 | score | |
---|---|---|---|
type | string | string | float |
details |
|
|
|
sentence1 | sentence2 | score |
---|---|---|
What is the data type of differential pressure in the measurement system? |
What is uncertainty? |
0.15000000000000002 |
What is the structure of the &&&equipment_data&&& table? |
How are flow computers and measurement systems related? |
0.35000000000000003 |
Find the columns in the flow computer table that identify the flow computer. |
What kind of data store an equipment? |
0.1 |
CosineSimilarityLoss
with these parameters:{
"loss_fct": "torch.nn.modules.loss.MSELoss"
}
sentence1
, sentence2
, and score
sentence1 | sentence2 | score | |
---|---|---|---|
type | string | string | float |
details |
|
|
|
sentence1 | sentence2 | score |
---|---|---|
How can I filter uncertainty reports by equipment tag? |
How does a flow computer generate and store reports? |
0.09999999999999999 |
What is the purpose of the flow_data table? |
What is uncertainty? |
0.15000000000000002 |
What is the column name for the report date in the Reports table? |
What is equipment calibration? |
0.1 |
CosineSimilarityLoss
with these parameters:{
"loss_fct": "torch.nn.modules.loss.MSELoss"
}
eval_strategy
: stepsper_device_train_batch_size
: 4per_device_eval_batch_size
: 4gradient_accumulation_steps
: 4learning_rate
: 2e-05warmup_ratio
: 0.1overwrite_output_dir
: Falsedo_predict
: Falseeval_strategy
: stepsprediction_loss_only
: Trueper_device_train_batch_size
: 4per_device_eval_batch_size
: 4per_gpu_train_batch_size
: Noneper_gpu_eval_batch_size
: Nonegradient_accumulation_steps
: 4eval_accumulation_steps
: Nonetorch_empty_cache_steps
: Nonelearning_rate
: 2e-05weight_decay
: 0.0adam_beta1
: 0.9adam_beta2
: 0.999adam_epsilon
: 1e-08max_grad_norm
: 1.0num_train_epochs
: 3max_steps
: -1lr_scheduler_type
: linearlr_scheduler_kwargs
: {}warmup_ratio
: 0.1warmup_steps
: 0log_level
: passivelog_level_replica
: warninglog_on_each_node
: Truelogging_nan_inf_filter
: Truesave_safetensors
: Truesave_on_each_node
: Falsesave_only_model
: Falserestore_callback_states_from_checkpoint
: Falseno_cuda
: Falseuse_cpu
: Falseuse_mps_device
: Falseseed
: 42data_seed
: Nonejit_mode_eval
: Falseuse_ipex
: Falsebf16
: Falsefp16
: Falsefp16_opt_level
: O1half_precision_backend
: autobf16_full_eval
: Falsefp16_full_eval
: Falsetf32
: Nonelocal_rank
: 0ddp_backend
: Nonetpu_num_cores
: Nonetpu_metrics_debug
: Falsedebug
: []dataloader_drop_last
: Falsedataloader_num_workers
: 0dataloader_prefetch_factor
: Nonepast_index
: -1disable_tqdm
: Falseremove_unused_columns
: Truelabel_names
: Noneload_best_model_at_end
: Falseignore_data_skip
: Falsefsdp
: []fsdp_min_num_params
: 0fsdp_config
: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap
: Noneaccelerator_config
: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed
: Nonelabel_smoothing_factor
: 0.0optim
: adamw_torchoptim_args
: Noneadafactor
: Falsegroup_by_length
: Falselength_column_name
: lengthddp_find_unused_parameters
: Noneddp_bucket_cap_mb
: Noneddp_broadcast_buffers
: Falsedataloader_pin_memory
: Truedataloader_persistent_workers
: Falseskip_memory_metrics
: Trueuse_legacy_prediction_loop
: Falsepush_to_hub
: Falseresume_from_checkpoint
: Nonehub_model_id
: Nonehub_strategy
: every_savehub_private_repo
: Nonehub_always_push
: Falsegradient_checkpointing
: Falsegradient_checkpointing_kwargs
: Noneinclude_inputs_for_metrics
: Falseinclude_for_metrics
: []eval_do_concat_batches
: Truefp16_backend
: autopush_to_hub_model_id
: Nonepush_to_hub_organization
: Nonemp_parameters
: auto_find_batch_size
: Falsefull_determinism
: Falsetorchdynamo
: Noneray_scope
: lastddp_timeout
: 1800torch_compile
: Falsetorch_compile_backend
: Nonetorch_compile_mode
: Nonedispatch_batches
: Nonesplit_batches
: Noneinclude_tokens_per_second
: Falseinclude_num_input_tokens_seen
: Falseneftune_noise_alpha
: Noneoptim_target_modules
: Nonebatch_eval_metrics
: Falseeval_on_start
: Falseuse_liger_kernel
: Falseeval_use_gather_object
: Falseaverage_tokens_across_devices
: Falseprompts
: Nonebatch_sampler
: batch_samplermulti_dataset_batch_sampler
: proportionalEpoch | Step | Training Loss | Validation Loss |
---|---|---|---|
0.0307 | 10 | 0.3228 | - |
0.0613 | 20 | 0.1919 | - |
0.0920 | 30 | 0.2414 | - |
0.1226 | 40 | 0.1649 | - |
0.1533 | 50 | 0.1554 | - |
0.1839 | 60 | 0.1432 | - |
0.2146 | 70 | 0.138 | - |
0.2452 | 80 | 0.1656 | - |
0.2759 | 90 | 0.1306 | - |
0.3065 | 100 | 0.1396 | - |
0.3372 | 110 | 0.0934 | - |
0.3678 | 120 | 0.1263 | - |
0.3985 | 130 | 0.1209 | - |
0.4291 | 140 | 0.0839 | - |
0.4598 | 150 | 0.1128 | 0.0260 |
0.4904 | 160 | 0.0895 | - |
0.5211 | 170 | 0.1226 | - |
0.5517 | 180 | 0.086 | - |
0.5824 | 190 | 0.0733 | - |
0.6130 | 200 | 0.0827 | - |
0.6437 | 210 | 0.0861 | - |
0.6743 | 220 | 0.0774 | - |
0.7050 | 230 | 0.0784 | - |
0.7356 | 240 | 0.095 | - |
0.7663 | 250 | 0.0793 | - |
0.7969 | 260 | 0.0583 | - |
0.8276 | 270 | 0.0571 | - |
0.8582 | 280 | 0.0597 | - |
0.8889 | 290 | 0.0742 | - |
0.9195 | 300 | 0.0569 | 0.0177 |
0.9502 | 310 | 0.0702 | - |
0.9808 | 320 | 0.0642 | - |
1.0092 | 330 | 0.0526 | - |
1.0398 | 340 | 0.0604 | - |
1.0705 | 350 | 0.0491 | - |
1.1011 | 360 | 0.0598 | - |
1.1318 | 370 | 0.0616 | - |
1.1625 | 380 | 0.0557 | - |
1.1931 | 390 | 0.0552 | - |
1.2238 | 400 | 0.0474 | - |
1.2544 | 410 | 0.0316 | - |
1.2851 | 420 | 0.0555 | - |
1.3157 | 430 | 0.0554 | - |
1.3464 | 440 | 0.051 | - |
1.3770 | 450 | 0.0578 | 0.0162 |
1.4077 | 460 | 0.0461 | - |
1.4383 | 470 | 0.0624 | - |
1.4690 | 480 | 0.0505 | - |
1.4996 | 490 | 0.0506 | - |
1.5303 | 500 | 0.0608 | - |
1.5609 | 510 | 0.0465 | - |
1.5916 | 520 | 0.0326 | - |
1.6222 | 530 | 0.0501 | - |
1.6529 | 540 | 0.0419 | - |
1.6835 | 550 | 0.0403 | - |
1.7142 | 560 | 0.0315 | - |
1.7448 | 570 | 0.0336 | - |
1.7755 | 580 | 0.0427 | - |
1.8061 | 590 | 0.053 | - |
1.8368 | 600 | 0.0451 | 0.0144 |
1.8674 | 610 | 0.0419 | - |
1.8981 | 620 | 0.0382 | - |
1.9287 | 630 | 0.0428 | - |
1.9594 | 640 | 0.0335 | - |
1.9900 | 650 | 0.0606 | - |
2.0184 | 660 | 0.0317 | - |
2.0490 | 670 | 0.0338 | - |
2.0797 | 680 | 0.0256 | - |
2.1103 | 690 | 0.0262 | - |
2.1410 | 700 | 0.028 | - |
2.1716 | 710 | 0.0229 | - |
2.2023 | 720 | 0.0157 | - |
2.2330 | 730 | 0.0367 | - |
2.2636 | 740 | 0.0226 | - |
2.2943 | 750 | 0.034 | 0.0128 |
2.3249 | 760 | 0.0247 | - |
2.3556 | 770 | 0.0251 | - |
2.3862 | 780 | 0.0245 | - |
2.4169 | 790 | 0.0249 | - |
2.4475 | 800 | 0.0247 | - |
2.4782 | 810 | 0.0266 | - |
2.5088 | 820 | 0.0338 | - |
2.5395 | 830 | 0.026 | - |
2.5701 | 840 | 0.0304 | - |
2.6008 | 850 | 0.0248 | - |
2.6314 | 860 | 0.0347 | - |
2.6621 | 870 | 0.0241 | - |
2.6927 | 880 | 0.0204 | - |
2.7234 | 890 | 0.0204 | - |
2.7540 | 900 | 0.0147 | 0.0126 |
2.7847 | 910 | 0.0266 | - |
2.8153 | 920 | 0.0279 | - |
2.8460 | 930 | 0.0159 | - |
2.8766 | 940 | 0.0162 | - |
2.9073 | 950 | 0.0315 | - |
2.9379 | 960 | 0.0174 | - |
2.9686 | 970 | 0.0244 | - |
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}