Update README.md
Browse files
README.md
CHANGED
@@ -123,7 +123,14 @@ CLaMP 3 is a unified framework for cross-modal and cross-lingual music informati
|
|
123 |
- Trained on **M4-RAG**, a large-scale dataset of 2.31M high-quality music-text pairs across 27 languages and 194 countries.
|
124 |
- Introduces **WikiMT-X**, a benchmark containing 1,000 triplets of sheet music, audio, and text.
|
125 |
|
126 |
-
CLaMP 3
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
127 |
|
128 |
### Links
|
129 |
- CLaMP 3 Demo Page (Coming Soon...)
|
@@ -140,7 +147,7 @@ CLaMP 3 achieves state-of-the-art performance across multiple MIR tasks, advanci
|
|
140 |
|
141 |
- [classification/](https://github.com/sanderwood/clamp3/tree/main/classification): Includes scripts for classification tasks using extracted features, such as training linear classification models and making predictions.
|
142 |
|
143 |
-
- [preprocessing/](https://github.com/sanderwood/clamp3/tree/main/preprocessing): Scripts for converting your data into compatible formats (interleaved ABC notation, MTF, or -extracted audio features). These are required for CLaMP 3 to work with the data.
|
144 |
|
145 |
- [retrieval/](https://github.com/sanderwood/clamp3/tree/main/retrieval): Provides scripts for evaluating model performance, conducting semantic searches, and calculating similarity metrics based on extracted feature vectors.
|
146 |
|
|
|
123 |
- Trained on **M4-RAG**, a large-scale dataset of 2.31M high-quality music-text pairs across 27 languages and 194 countries.
|
124 |
- Introduces **WikiMT-X**, a benchmark containing 1,000 triplets of sheet music, audio, and text.
|
125 |
|
126 |
+
CLaMP 3 supports a wide range of applications in MIR and music research, including but not limited to:
|
127 |
+
- **Semantic retrieval:** Searching for music based on descriptive text or retrieving textual metadata based on audio or symbolic representations.
|
128 |
+
- **Zero-shot classification:** Categorizing music by genre, region, or other attributes without labeled training data.
|
129 |
+
- **Music quality assessment:** Measuring the **semantic distance** between reference ground truth and generated music using metrics analogous to **Fréchet Inception Distance (FID)**, providing an objective alternative to human evaluation.
|
130 |
+
- **Evaluation of generative models:** Assessing **text-to-music generation**, **music captioning**, and **symbolic-to-audio synthesis** models by quantifying their alignment across different modalities.
|
131 |
+
- **Computational musicology:** Enabling studies in **geographical musicology**, analyzing regional distributions and cross-cultural influences through large-scale multimodal datasets.
|
132 |
+
|
133 |
+
Importantly, these applications are **not restricted to any specific musical modality or language**. CLaMP 3's multimodal and multilingual design allows it to generalize across diverse datasets, making it a powerful tool for a wide range of music-related AI research.
|
134 |
|
135 |
### Links
|
136 |
- CLaMP 3 Demo Page (Coming Soon...)
|
|
|
147 |
|
148 |
- [classification/](https://github.com/sanderwood/clamp3/tree/main/classification): Includes scripts for classification tasks using extracted features, such as training linear classification models and making predictions.
|
149 |
|
150 |
+
- [preprocessing/](https://github.com/sanderwood/clamp3/tree/main/preprocessing): Scripts for converting your data into compatible formats (interleaved ABC notation, MTF, or MERT-extracted audio features). These are required for CLaMP 3 to work with the data.
|
151 |
|
152 |
- [retrieval/](https://github.com/sanderwood/clamp3/tree/main/retrieval): Provides scripts for evaluating model performance, conducting semantic searches, and calculating similarity metrics based on extracted feature vectors.
|
153 |
|