how to make use of function calling in system message
#2
by
cmgzy
- opened
in the mistral official guide, it's called by:
completion_request = ChatCompletionRequest(
tools=[
Tool(
function=Function(
name="get_current_weather",
description="Get the current weather",
parameters={
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"format": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use. Infer this from the users location.",
},
},
"required": ["location", "format"],
},
)
)
],
messages=[
UserMessage(content="What's the weather like today in Paris?"),
],
)
If I wanna use it locally by mlx-lm, how should I write system message?
I managed to do that, here's the command:
mlx_lm.generate --model mlx-community/Mistral-7B-Instruct-v0.3-8bit --prompt "<s><s>[AVAILABLE_TOOLS] [{\"type\": \"function\", \"function\": {\"name\": \"get_current_weather\", \"description\": \"Get the current weather\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\", \"description\": \"The city and state, e.g. San Francisco, CA\"}, \"format\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"], \"description\": \"The temperature unit to use. Infer this from the users location.\"}}, \"required\": [\"location\", \"format\"]}}}][/AVAILABLE_TOOLS] [INST]What's the weather like in Paris?[/INST]" --max-tokens 2048 --temp 0.8 --top-p 0.95 --ignore-chat-template
and the response is:
[TOOL_CALLS] [{"name": "get_current_weather", "arguments": {"location": "Paris, FR", "format": "celsius"}}]
note that leading [TOOL_CALLS] is the proof of function calling
cmgzy
changed discussion status to
closed