Nicolay Rusnachenko's picture

Nicolay Rusnachenko

nicolay-r

·

https://nicolay-r.github.io/

AI & ML interests

Information Retrieval・Medical Multimodal NLP (🖼+📝) Research Fellow @BU_Research・software developer http://arekit.io・PhD in NLP

Recent Activity

reacted to mmhamdy's post with 👀 about 6 hours ago

⛓ Evaluating Long Context #2: SCROLLS and ZeroSCROLLS In this series of posts about tracing the history of long context evaluation, we started with Long Range Arena (LRA). Introduced in 2020, Long Range Arens (LRA) is one of the earliest benchmarks designed to tackle the challenge of long context evaluation. But it wasn't introduced to evaluate LLMs, but rather the transformer architecture in general. 📜 The SCROLLS benchmark, introduced in 2022, addresses this gap in NLP/LLM research. SCROLLS challenges models with tasks that require reasoning over extended sequences (according to 2022 standards). So, what does it offer? 1️⃣ Long Text Focus: SCROLLS (unlike LRA) focus mainly on text and contain inputs with thousands of words, testing models' ability to synthesize information across lengthy documents. 2️⃣ Diverse Tasks: Includes summarization, question answering, and natural language inference across domains like literature, science, and business. 3️⃣ Unified Format: All datasets are available in a text-to-text format, facilitating easy evaluation and comparison of models. Building on SCROLLS, ZeroSCROLLS takes long text evaluation to the next level by focusing on zero-shot learning. Other features include: 1️⃣ New Tasks: Introduces tasks like sentiment aggregation and sorting book chapter summaries. 2️⃣ Leaderboard: A live leaderboard encourages continuous improvement and competition among researchers. 💡 What are some other landmark benchmarks in the history of long context evaluation? Feel free to share your thoughts and suggestions in the comments. - SCROLLS Paper: https://huggingface.co/papers/2201.03533 - ZeroSCROLLS Paper: https://huggingface.co/papers/2305.14196

reacted to sequelbox's post with 🧠 about 6 hours ago

Raiden is here! 63k creative-reasoning and analytic-reasoning prompts answered by DeepSeek's 685b R1 model! - All prompts from https://huggingface.co/datasets/microsoft/orca-agentinstruct-1M-v1 and all responses from https://huggingface.co/deepseek-ai/DeepSeek-R1 - A deep look at R1's reasoning skills! Use as you will. Get it now: https://huggingface.co/datasets/sequelbox/Raiden-DeepSeek-R1 for everyone :)

reacted to sequelbox's post with 👀 about 6 hours ago

Raiden is here! 63k creative-reasoning and analytic-reasoning prompts answered by DeepSeek's 685b R1 model! - All prompts from https://huggingface.co/datasets/microsoft/orca-agentinstruct-1M-v1 and all responses from https://huggingface.co/deepseek-ai/DeepSeek-R1 - A deep look at R1's reasoning skills! Use as you will. Get it now: https://huggingface.co/datasets/sequelbox/Raiden-DeepSeek-R1 for everyone :)

View all activity

Organizations

None yet

Posts 54

Post

2041

📢 If you wish to empower LLM with NER for texts in English, then I can recommend to use Spacy. Sharing the wrapper of Spacy NER models the bulk-ner dedicated for hadling CSV / JSONL content:
Script: https://github.com/nicolay-r/nlp-thirdgate/blob/master/tutorials/ner_spacy_383.sh
Code: https://raw.githubusercontent.com/nicolay-r/nlp-thirdgate/refs/heads/master/ner/spacy_383.py

What do you need to know about Spacy NER models:
☑️ Models represent a python packages; packages could be installed directly into environemnt or via python CLI.
☑️ Library has a pipeline for optimized request handling in batches.
☑️ Architecture: DNN embedding-based models (not transformers)

🤖 List of models (or see screenshot below):
https://huggingface.co/spacy
📋 Supported NER types:
https://github.com/explosion/spaCy/discussions/9147

⚠️ NOTE: chunking seems to be non-applicable due to specifics of models and usage of the internal pipeline mechanism

🚀 Performance for sentences (en):
Model: spacy/en_core_web_sm 🔥 530 sentences per second 🔥 (similar to larger solutions)

🌌 other wrappers for bulk-ner nlp-thirdgate: https://github.com/nicolay-r/nlp-thirdgate#ner

Post

2161

📢 If you wish to empower LLM with IR and named entity recognition module, then I got relevant findings.
Just tested Flair below is how you can start for adapting for processing your CSV / JSONL data via bulk-ner
👩‍💻 code: https://github.com/nicolay-r/nlp-thirdgate/blob/master/tutorials/ner_flair_0151.sh
🤖 models: https://huggingface.co/flair

Provider: https://raw.githubusercontent.com/nicolay-r/nlp-thirdgate/refs/heads/master/ner/flair_0151.py
Framework: https://github.com/nicolay-r/bulk-ner

🚀 Performance: the default ner model (Thinkpad X1 Nano)
Batch-size 1 6it/sec
Batch-size 10+ 12it/sec

🌌 other wrappers for bulk-ner nlp-thirdgate: https://github.com/nicolay-r/nlp-thirdgate

Collections 2

Papers 7

arxiv:2404.12342

arxiv:2404.03361

arxiv:2305.17679

arxiv:2006.13730

models 5

nicolay-r/flan-t5-emotion-cause-thor-base

Text Generation • Updated 13 days ago • 62 • 1

nicolay-r/flan-t5-tsa-prompt-xl

Text2Text Generation • Updated 13 days ago • 117

nicolay-r/flan-t5-tsa-thor-xl

Text2Text Generation • Updated 13 days ago • 113 • 2

nicolay-r/flan-t5-tsa-thor-large

Text2Text Generation • Updated 13 days ago • 118

nicolay-r/flan-t5-tsa-thor-base

Text2Text Generation • Updated 13 days ago • 132 • 1

datasets

None public yet