Tidzo
/

falcon-7b-esg-instruct

Text Generation

Inference Endpoints

Model card Files Files and versions Community

Tidzo commited on Jul 4, 2023

Commit

76a7207

·

1 Parent(s): 5548617

Update handler.py

Files changed (1) hide show

handler.py +0 -1

handler.py CHANGED Viewed

@@ -15,7 +15,6 @@ class EndpointHandler():
         config = PeftConfig.from_pretrained(path)
         bnb_config = BitsAndBytesConfig(
-            load_in_4bit=True,
             bnb_4bit_quant_type="nf4",
             bnb_4bit_compute_dtype=torch.float16,
         )

         config = PeftConfig.from_pretrained(path)
         bnb_config = BitsAndBytesConfig(
             bnb_4bit_quant_type="nf4",
             bnb_4bit_compute_dtype=torch.float16,
         )