Tool Calling and Function Execution

What is Tool Calling?

By default, LLMs can only generate text based on their training data. Tool calling (also known as function calling) is a mechanism that allows the model to decide when to call external APIs or execute code to fetch live data or perform actions.

How Function Calling Works

When using models like GPT-4, Claude 3, or Gemini, you provide a list of available tools along with your prompt. Each tool includes a description of what it does and the schema of its expected arguments.

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"}
            },
            "required": ["location"]
        }
    }
}]

The Execution Loop

1. The user asks: “What’s the weather in Tokyo?”
2. The LLM detects that it needs live data and returns a structured JSON requesting to call get_weather({"location": "Tokyo"}).
3. Your application intercepts this, executes the actual Python function to call a Weather API, and gets the result (e.g., “22°C, Sunny”).
4. Your application sends this result back to the LLM.
5. The LLM reads the result and generates a human-readable response: “It’s currently 22°C and sunny in Tokyo.”

Safety & Sandboxing

When giving LLMs the ability to execute actions (especially writing or executing code), sandboxing is critical. Never let an LLM run os.system() commands directly on your host machine. Always use isolated Docker containers or technologies like nsjail to prevent disastrous consequences if the model hallucinates a dangerous command.