"Can't load tokenizer" error when deploying using AWS SageMaker snippet
Hello 👋
When attempting to deploy this model using the SageMaker snippet on the model card page, I get the following error:
OSError: Can't load tokenizer for 'soumyasanyal/nli-entailment-verifier-xxl'. If
you were trying to load it from 'https://huggingface.co/models', make sure you
don't have a local directory with the same name. Otherwise, make sure
'soumyasanyal/nli-entailment-verifier-xxl' is the correct path to a directory
containing all relevant files for a T5TokenizerFast tokenizer.
The endpoint fails to deploy on SageMaker AI.
Is the tokenizer exported with the model, or do I need to do something more than what's in the deploy snippet?
Hi jon-proem,
I believe the SageMaker snippet is autogenerated by some HuggingFace code. And that's leading to some confusion. Our model uses the same tokenizer as the google/flan-t5-xxl model; thus, we did not ship it separately. For more details on the correct usage, you can refer to the usage snippet on the model card page.
Thanks for getting back to me so quickly!
I see. That did indeed lead to a bit of confusion. I'm trying to deploy the model on AWS SageMaker, so I can evaluate it as a potential part of our solution. Have you tried deploying on AWS or another cloud provider?
// Jon
Hi Jon,
No, I have only tried this on our local servers. I believe the tokenizer has to be loaded separately in the SageMaker snippet, perhaps as an additional argument to the env of HuggingFaceModel. Also, the input format should be similar to the one we showcase in the usage template.