README.md

---
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:408
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
base_model: BAAI/bge-base-en-v1.5
widget:
- source_sentence: What sets TechChefz apart?
  sentences:
  - 'Sharing Stories from Our Team


    Discover firsthand experiences, growth journeys, and the vibrant culture that
    fuels our success.


    I have been a part of Techchefz for 3 years, and I can confidently say it''s been
    a remarkable journey. From day one, I was welcomed into a vibrant community that
    values collaboration, creativity, and personal growth. The company culture here
    isn''t just a buzzword, it''s tangible in every interaction and initiative.

    profileImg


    Aashish Massand


    Sr. Manager Delivery


    TechChefz has been a transformative journey, equipping me with invaluable skills
    and fostering a supportive community. From coding fundamentals to advanced techniques,
    I''ve gained confidence and expertise. Grateful for this experience and opportunity.

    profileImg


    Pankaj Datt


    Associate Technology'
  - 'After a transformative scuba dive in the Maldives, Mayank Maggon made a pivotal
    decision to depart from the corporate ladder in December 2016. Fueled by a clear
    vision to revolutionize the digital landscape, Mayank set out to leverage the
    best technology ingredients, crafting custom applications and digital ecosystems
    tailored to clients'' specific needs, limitations, and budgets.


    However, this solo journey was not without its challenges. Mayank had to initiate
    the revenue engine by offering corporate trainings and conducting online batches
    for tech training across the USA. He also undertook small projects and subcontracted
    modules of larger projects for clients in the US, UK, and India. It was only after
    this initial groundwork that Mayank was able to hire a group of interns, whom
    he meticulously trained and groomed to prepare them for handling Enterprise Level
    Applications. This journey reflects Mayank''s resilience, determination, and entrepreneurial
    spirit in building TechChefz Digital from the ground up.


    With a passion for innovation and a relentless drive for excellence, Mayank has
    steered TechChefz Digital through strategic partnerships, groundbreaking projects,
    and exponential growth. His leadership has been instrumental in shaping TechChefz
    Digital into a leading force in the digital transformation arena, inspiring a
    culture of innovation and excellence that continues to propel the company forward.'
  - TechChefz Digital has established its presence in two countries, showcasing its
    global reach and influence. The company’s headquarters is strategically located
    in Noida, India, serving as the central hub for its operations and leadership.
    In addition to the headquarters, TechChefz Digital has expanded its footprint
    with offices in Delaware, United States, allowing the company to cater to the
    North American market with ease and efficiency.
- source_sentence: How does this solution comply with data regulations?
  sentences:
  - 'Introducing the world of General Insurance Firm

    In this project, we implemented Digital Solution and Implementation with Headless
    Drupal as the CMS, and lightweight React JS (Next JS SSR on Node JS) with the
    following features:

    PWA & AMP based Web Pages

    Page Speed Optimization

    Reusable and scalable React JS / Next JS Templates and Components

    Headless Drupal CMS with Content & Experience management, approval workflows,
    etc for seamless collaboration between the business and marketing teams

    Minimalistic Buy and Renewal Journeys for various products, with API integrations
    and adherence to data compliances


    We achieved 250% Reduction in Operational Time and Effort in managing the Content
    & Experience for Buy & renew Journeys,220% Reduction in Customer Drops during
    buy and renewal journeys, 300% Reduction in bounce rate on policy landing and
    campaign pages'
  - 'We assist businesses by transforming their goals, teams, and cultures with digital
    technology to make them colinear with the digital age. Through digitalization,
    organizations can facilitate advanced decision-making and management.

    '
  - 'Microservices Transformation Process

    Requirements Analysis

    We begin by understanding the client's needs and objectives for the website.
    Identify key features, functionality, and any specific design preferences.


    Planning

    Then create a detailed project plan outlining the scope, timeline, and milestones.
    Define the technology stack and development tools suitable for the project.


    User Experience Design

    Then comes the stage of Developing wireframes or prototypes to visualize the website''s
    structure and layout. We create a custom design that aligns with the brand identity
    and user experience goals.


    Development

    After getting Sign-off on Design from Client, we break the requirements into Sprints
    on Agile Methodology, and start developing them.


    Testing

    After each sprint we conduct thorough testing of the website to identify and fix
    any bugs or issues. Perform usability testing to ensure a positive user experience.

    Deployment

    After testing we deploy the website sprint by sprint, to a hosting environment,
    ensuring proper configuration for security and performance. Our expert DevOps
    team sets up any necessary domain and server configurations and ensure smooth
    running of website.'
- source_sentence: What tasks can we automate using machine learning?
  sentences:
  - 'Check out our latest news, announcements, and featured insights.


    Explore our latest insights and stay informed with our thought-provoking content.
    Dive in now for valuable perspectives.

    Our Featured Insights


    How UX and UI Work Together in Web Design


    Navigating the Post-Cookie Era: Strategies for Effective Targeting and Personalization


    Data-Driven Decision Making in Digital Advertising: Leveraging Analytics for Success


    SEO Unleashed: Navigating the Digital Landscape with Advanced Search Engine Optimization
    Tools


    Is manual testing replaced by automation Testing?'
  - 'In what ways can machine learning optimize our operations?

    Machine learning algorithms can analyze operational data to identify inefficiencies,
    predict maintenance needs, optimize supply chains, and automate repetitive tasks,
    significantly improving operational efficiency and reducing costs.'
  - Mayank Maggon is CEO of Techchefz Digital
- source_sentence: How can you help us grow our partnerships?
  sentences:
  - "Partner Experience (PX)\n From optimized collaboration tools to data-driven insights,\
    \ our solutions are designed to drive efficiency, transparency, and growth in\
    \ partner relationships. With a keen understanding of complexities of partner\
    \ ecosystems, we help enterprise brands unlock new opportunities, strengthen alliances,\
    \ and achieve shared success in today’s dynamic business environment."
  - At Techchefz Digital, we specialize in guiding companies through the complexities
    of adopting and integrating Artificial Intelligence and Machine Learning technologies.
    Our consultancy services are designed to enhance your operational efficiency and
    decision-making capabilities across all sectors. With a global network of AI/ML
    experts and a commitment to excellence, we are your partners in transforming innovative
    possibilities into real-world achievements.
  - 'COMMERCE PLATFORMS

    Discover the strength of our partnership.

    Adobe Commerce Cloud

    A comprehensive e-commerce platform that allows businesses to create, manage,
    and optimize their online stores. Formerly known as Magento Commerce, Adobe Commerce
    Cloud provides a range of features and capabilities to help businesses create
    engaging online shopping experiences, manage their products and catalogs, process
    orders, and drive online sales.

    Magento

    An open-source e-commerce platform that allows businesses to create online stores
    and manage their digital operations. It was first released in 2008 and has since
    become one of the most popular e-commerce platforms in the world.

    Shopify

    Salesforce Commerce Cloud (SFCC)'
- source_sentence: How is an Enterprise CMS different from a headless CMS?
  sentences:
  - 'How do I figure out how much your services will cost?

    Determining the cost of our services is best achieved through a 15-30 minute discovery
    call, where we can understand your unique requirements. Following that, we will
    provide a transparent and detailed price within 24-48 hours tailored specifically
    to you'
  - 'Discover the right CMS for your Business Requirements

    Headless CMS

    They separate the backend content repository from the frontend presentation layer,
    allowing content to be delivered to any device or platform via APIs offering flexibility
    and scalability.



    Enterprise CMS

    ECMSs are more comprehensive systems designed to manage all types of content within
    an organization, including documents, images, videos, and other digital assets.'
  - We offer custom software development, digital marketing strategies, and tailored
    solutions to drive tangible results for your business. Our expert team combines
    technical prowess with industry insights to propel your business forward in the
    digital landscape.
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
model-index:
- name: BGE base Financial Matryoshka
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 768
      type: dim_768
    metrics:
    - type: cosine_accuracy@1
      value: 0.0
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.0784313725490196
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.4019607843137255
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.5196078431372549
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.0
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.0261437908496732
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.08039215686274509
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.05196078431372548
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.0
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.0784313725490196
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.4019607843137255
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.5196078431372549
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.20681828171013134
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.11193977591036408
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.12704742492729623
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 512
      type: dim_512
    metrics:
    - type: cosine_accuracy@1
      value: 0.0
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.0784313725490196
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.4019607843137255
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.5196078431372549
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.0
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.0261437908496732
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.08039215686274509
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.05196078431372548
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.0
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.0784313725490196
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.4019607843137255
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.5196078431372549
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.20587690425273067
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.11086601307189538
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.12502250584870636
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 256
      type: dim_256
    metrics:
    - type: cosine_accuracy@1
      value: 0.0
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.06862745098039216
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.39215686274509803
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.5098039215686274
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.0
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.02287581699346405
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.0784313725490196
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.05098039215686274
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.0
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.06862745098039216
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.39215686274509803
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.5098039215686274
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.20200410483390918
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.10891690009337061
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.12124652633795324
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 128
      type: dim_128
    metrics:
    - type: cosine_accuracy@1
      value: 0.0
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.058823529411764705
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.3137254901960784
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.49019607843137253
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.0
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.0196078431372549
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.06274509803921569
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.04901960784313725
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.0
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.058823529411764705
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.3137254901960784
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.49019607843137253
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.18661585783989612
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.09673202614379077
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.11007694082793783
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 64
      type: dim_64
    metrics:
    - type: cosine_accuracy@1
      value: 0.0
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.029411764705882353
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.28431372549019607
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.4117647058823529
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.0
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.00980392156862745
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.05686274509803922
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.04117647058823529
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.0
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.029411764705882353
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.28431372549019607
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.4117647058823529
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.15696823886592676
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.08097572362278241
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.09297982754610348
      name: Cosine Map@100
---

# BGE base Financial Matryoshka

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
- **Language:** en
- **License:** apache-2.0

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("akashmaggon/bge-base-financial-matryoshka-finetuning-tcz-1")
# Run inference
sentences = [
    'How is an Enterprise CMS different from a headless CMS?',
    'Discover the right CMS for your Business Requirements\nHeadless CMS\nThey separate the backend content repository from the frontend presentation layer, allowing content to be delivered to any device or platform via APIs offering flexibility and scalability.\n\n\nEnterprise CMS\nECMSs are more comprehensive systems designed to manage all types of content within an organization, including documents, images, videos, and other digital assets.',
    'How do I figure out how much your services will cost?\nDetermining the cost of our services is best achieved through a 15-30 minute discovery call, where we can understand your unique requirements. Following that, we will provide a transparent and detailed price within 24-48 hours tailored specifically to you',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval

* Datasets: `dim_768`, `dim_512`, `dim_256`, `dim_128` and `dim_64`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | dim_768    | dim_512    | dim_256   | dim_128    | dim_64    |
|:--------------------|:-----------|:-----------|:----------|:-----------|:----------|
| cosine_accuracy@1   | 0.0        | 0.0        | 0.0       | 0.0        | 0.0       |
| cosine_accuracy@3   | 0.0784     | 0.0784     | 0.0686    | 0.0588     | 0.0294    |
| cosine_accuracy@5   | 0.402      | 0.402      | 0.3922    | 0.3137     | 0.2843    |
| cosine_accuracy@10  | 0.5196     | 0.5196     | 0.5098    | 0.4902     | 0.4118    |
| cosine_precision@1  | 0.0        | 0.0        | 0.0       | 0.0        | 0.0       |
| cosine_precision@3  | 0.0261     | 0.0261     | 0.0229    | 0.0196     | 0.0098    |
| cosine_precision@5  | 0.0804     | 0.0804     | 0.0784    | 0.0627     | 0.0569    |
| cosine_precision@10 | 0.052      | 0.052      | 0.051     | 0.049      | 0.0412    |
| cosine_recall@1     | 0.0        | 0.0        | 0.0       | 0.0        | 0.0       |
| cosine_recall@3     | 0.0784     | 0.0784     | 0.0686    | 0.0588     | 0.0294    |
| cosine_recall@5     | 0.402      | 0.402      | 0.3922    | 0.3137     | 0.2843    |
| cosine_recall@10    | 0.5196     | 0.5196     | 0.5098    | 0.4902     | 0.4118    |
| **cosine_ndcg@10**  | **0.2068** | **0.2059** | **0.202** | **0.1866** | **0.157** |
| cosine_mrr@10       | 0.1119     | 0.1109     | 0.1089    | 0.0967     | 0.081     |
| cosine_map@100      | 0.127      | 0.125      | 0.1212    | 0.1101     | 0.093     |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset


* Size: 408 training samples
* Columns: <code>anchor</code> and <code>positive</code>
* Approximate statistics based on the first 408 samples:
  |         | anchor                                                                            | positive                                                                            |
  |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
  | type    | string                                                                            | string                                                                              |
  | details | <ul><li>min: 8 tokens</li><li>mean: 12.63 tokens</li><li>max: 21 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 94.18 tokens</li><li>max: 270 tokens</li></ul> |
* Samples:
  | anchor                                                     | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
  |:-----------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>What's it like working at Techchefz?</code>          | <code>Join one of the most resourceful tech teams<br><br>Discover your future with us. Explore opportunities, values, and culture. Join a dynamic and innovative team at Techchefz.<br><br>LIFE AT TECHCHEFZ<br>Make an Impact from Day One.<br><br>We believe in the power of collaboration to create, innovate, and develop groundbreaking solutions. Our teams work closely with clients and partners to co-create solutions that drive innovation and business growth.<br>Your new journey awaits!</code> |
  | <code>How can I contact TechChefz if I'm in the US?</code> | <code>TechChefz Digital has established its presence in two countries, showcasing its global reach and influence. The company’s headquarters is strategically located in Noida, India, serving as the central hub for its operations and leadership. In addition to the headquarters, TechChefz Digital has expanded its footprint with offices in Delaware, United States, allowing the company to cater to the North American market with ease and efficiency.</code>                                       |
  | <code>What results can I expect from your services?</code> | <code>We offer custom software development, digital marketing strategies, and tailored solutions to drive tangible results for your business. Our expert team combines technical prowess with industry insights to propel your business forward in the digital landscape.</code>                                                                                                                                                                                                                              |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: epoch
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 4
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `fp16`: True
- `load_best_model_at_end`: True
- `optim`: adamw_torch_fused
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 4
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: True
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch      | Step  | dim_768_cosine_ndcg@10 | dim_512_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 |
|:----------:|:-----:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|
| 0.6154     | 1     | 0.2038                 | 0.1993                 | 0.1953                 | 0.1764                 | 0.1595                |
| 1.6154     | 2     | 0.2038                 | 0.1993                 | 0.1953                 | 0.1764                 | 0.1595                |
| **2.6154** | **3** | **0.2068**             | **0.2059**             | **0.202**              | **0.1866**             | **0.157**             |
| 3.6154     | 4     | 0.2068                 | 0.2059                 | 0.2020                 | 0.1866                 | 0.1570                |

* The bold row denotes the saved checkpoint.

### Framework Versions
- Python: 3.11.11
- Sentence Transformers: 3.3.1
- Transformers: 4.47.1
- PyTorch: 2.5.1+cu121
- Accelerate: 1.2.1
- Datasets: 3.2.0
- Tokenizers: 0.21.0

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->