Building Your First AI Agent: A Practical Guide to Local LLM Integration for Real-World Problem Solving

The rapidly evolving landscape of software development, artificial intelligence (AI), and Large Language Models (LLMs) has brought AI agents to the forefront of technological discourse. Professionals immersed in these fields frequently discuss and utilize AI agents, yet the journey from mere usage to active creation remains a significant step for many. This article serves as a comprehensive guide for individuals eager to transition from being consumers of AI agents to becoming their architects. It demystifies the core concept of an AI agent and provides a hands-on approach to building a functional agent using local LLM technology, demonstrating its potential for practical applications.

Defining the AI Agent: A Functional Framework

At its core, an AI agent can be broadly defined as a sophisticated software system that leverages a Large Language Model. This LLM, in turn, orchestrates a loop of interactions, calling upon one or more specialized tools to systematically achieve a predefined goal. The complexity of these goals can span a vast spectrum, from answering straightforward user queries to undertaking intricate tasks such as autonomously navigating and booking airline tickets via a web browser, or even contributing to the development of highly complex software solutions.

It is crucial to understand that the "loop" characteristic of agentic systems refers to the cyclical exchange between the LLM, the overarching application, and its associated tools. This does not necessitate the literal implementation of while or for programming loops. Instead, it describes a functional cycle: the model identifies a need, requests a specific tool, the application executes that tool, and the model then synthesizes the returned result to progress towards its objective. In simpler implementations, a single pass through this cycle may suffice to reach a conclusion, with each step explicitly outlined for clarity.

This article will guide readers through the construction of a foundational AI agent. While its initial functionality will be deliberately simple, this is a strategic design choice to facilitate an accessible learning curve, not a limitation of the agent’s inherent capabilities. The objective is to establish a solid understanding of the underlying mechanics before scaling to more complex implementations.

The Problem: Bridging the Gap Between Language Models and Structured Data

Consider a scenario where a small online retailer requires an intelligent support assistant capable of responding to customer inquiries about existing orders. The critical challenge here is that the order data resides within a structured database, a domain inaccessible to a language model operating solely on its textual understanding. To effectively address such queries, the LLM needs a secure and controlled mechanism to access and retrieve factual information from the business’s operational data.

This article will demonstrate the creation of such a controlled interface using a hypothetical online shop and a compact SQLite database. This database is designed with two key tables: orders, which stores essential order details such as order number, customer name, and date; and order_items, which enumerates the products, quantities, and prices associated with each order.

For illustrative purposes, our sample database contains a specific order, number 1001, placed by a customer named Sarah Jones. This order includes one mechanical keyboard priced at £79.95 and two USB-C cables, each costing £8.50. The calculated total for this order is £96.95.

The goal is to empower customers to query this order information using natural language, eliminating the need for them to manually access the database or construct SQL queries. A user might ask: "What is the total value of order 1001?"

Upon receiving this query, the LLM should recognize the requirement for order total information. It will then initiate a request to execute a predefined Python function, hypothetically named get_order_total. This function will interact with the SQLite database, retrieve the relevant data, and return it. Subsequently, the LLM will process this returned value and formulate a coherent, natural-language response for the user.

It is vital to emphasize that while the LLM can request an operation, the ultimate decision to authorize and execute that operation, along with the actual data retrieval, remains firmly under the control of the Python application. This hierarchical control ensures security and data integrity.

Our initial AI agent implementation will comprise four primary functional components:

1. Language Model Input: The language model receives the user’s question and a descriptive schema of an available Python function.
2. Tool Request: Based on its understanding, the model formulates a request to execute a specific function with provided arguments.
3. Python Execution and Validation: The Python environment intercepts this request. It verifies the requested function against an approved list, executes the permitted function, and securely returns its output.
4. Model Response Generation: The language model receives the function’s result and synthesizes it into a user-friendly answer.

Crucially, the LLM itself does not execute the Python function or directly interact with the SQLite database. These operations are managed by the Python program, maintaining a clear separation of concerns. To facilitate local development without requiring cloud accounts or API keys, this example will utilize Ollama, a powerful tool for running LLMs locally. The script is designed to explicitly print the requested function, its arguments, and the database result, rendering the internal workings of the agent visible and aiding in comprehension, rather than abstracting them within a proprietary framework.

While this script employs Ollama’s Python library, the underlying pattern of client-side tool invocation is transferable to other LLM APIs, including those from OpenAI and Anthropic. The specific SDK classes, field names, and message formats will vary across providers, meaning this exact script is not directly portable. However, the fundamental concepts—allowlisting, argument validation, function dispatch, handling tool results, and the bounded execution loop—are universally applicable.

Essential Prerequisites: Setting Up Your Development Environment

To embark on building your AI agent, ensure you have the following components in place:

• Operating System: Windows 10 or 11 is recommended for this guide, though the principles and core code are compatible with macOS and Linux.
• Disk Space: Sufficient free disk space is required to download and store a local LLM.
• PowerShell: This command-line shell will be used for executing installation and project commands.

Installing uv for Python Management:

We will leverage uv, a fast and efficient Python package installer and virtual environment manager. If uv is not already installed on your system, open PowerShell and execute the following command:

irm https://get.uv.sh | iex

After the installation completes, close your current PowerShell window and open a new one to ensure the uv command is recognized. Verify the installation by running:

uv --version

This command should display the installed version of uv.

Installing Ollama for Local LLM Execution:

Ollama is an indispensable tool for running large language models directly on your local machine, provided your system has adequate RAM to support your chosen model.

1. Download the Installer: Navigate to the official Ollama download page for Windows: https://ollama.com/download/windows. Download the installer package.
2. Run the Installer: Execute the downloaded installer and follow the on-screen prompts to complete the installation.

3. Download a Model: Once Ollama is installed, open a new PowerShell window. To download a suitable LLM, such as llama3, run the following command:

   ollama pull llama3

   As of this writing, the download size for llama3 is approximately 4.7 GB. Please note that model availability and sizes are subject to change. During the download and subsequent execution, monitor for any errors, and ensure that the selected model fits within your system’s available memory.

Project Setup: Establishing the Development Workspace

Begin by creating a dedicated directory for your project. Within this directory, you will install the Ollama Python library. Use uv to manage your Python environment:

# Create a new project directory (replace 'ai_agent_project' with your desired name)
mkdir ai_agent_project
cd ai_agent_project

# Install the Ollama Python library using uv
uv pip install ollama

The uv command will automatically download and install Python 3.14 if a suitable version is not found locally. There is no need to explicitly activate the virtual environment; uv run will manage its usage automatically.

Constructing the Foundation: Creating the Test Database

Before integrating the language model, it is prudent to establish and populate the database. This ensures a deterministic foundation for testing and debugging. If the database contains incorrect data, debugging the AI agent becomes significantly more challenging.

The database file will reside alongside your Python scripts and can be recreated at any time to reset the example’s data.

Create a file named build_shop_database.py and populate it with the following Python code:

import sqlite3
from pathlib import Path

DATABASE_NAME = "shop_agent.db"
DATABASE_PATH = Path(__file__).parent / DATABASE_NAME

def create_database():
    with sqlite3.connect(DATABASE_PATH) as conn:
        cursor = conn.cursor()

        # Create orders table
        cursor.execute("""
            CREATE TABLE IF NOT EXISTS orders (
                order_id INTEGER PRIMARY KEY,
                customer_name TEXT NOT NULL,
                order_date TEXT NOT NULL
            )
        """)

        # Create order_items table
        cursor.execute("""
            CREATE TABLE IF NOT EXISTS order_items (
                item_id INTEGER PRIMARY KEY AUTOINCREMENT,
                order_id INTEGER NOT NULL,
                product_name TEXT NOT NULL,
                quantity INTEGER NOT NULL,
                price REAL NOT NULL,
                FOREIGN KEY (order_id) REFERENCES orders (order_id)
            )
        """)

        # Insert sample data for order 1001
        cursor.execute("INSERT OR IGNORE INTO orders (order_id, customer_name, order_date) VALUES (?, ?, ?)",
                       (1001, 'Sarah Jones', '2023-10-27'))

        cursor.execute("INSERT OR IGNORE INTO order_items (order_id, product_name, quantity, price) VALUES (?, ?, ?, ?)",
                       (1001, 'Mechanical Keyboard', 1, 79.95))
        cursor.execute("INSERT OR IGNORE INTO order_items (order_id, product_name, quantity, price) VALUES (?, ?, ?, ?)",
                       (1001, 'USB-C Cable', 2, 8.50))

        conn.commit()
    print(f"Database 'DATABASE_NAME' created and populated successfully.")

if __name__ == "__main__":
    create_database()

Execute this script from your PowerShell terminal within the project directory:

uv run python build_shop_database.py

This script utilizes Python’s built-in sqlite3 module, eliminating the need for a separate database server installation. The conn.cursor() method creates a cursor object used to execute SQL commands. The CREATE TABLE IF NOT EXISTS statements ensure that tables are only created if they do not already exist, preventing errors on subsequent runs. Sample data for order 1001 is then inserted. The conn.commit() command saves the changes to the database file. The with sqlite3.connect(...) construct ensures that transactions are handled correctly, and the explicit conn.close() (implicitly handled by with) is particularly important on Windows to prevent the database file from remaining locked after the script finishes.

Developing the AI Agent: The single_tool.py Script

Now, let’s create the core AI agent script, named single_tool.py. This file will contain the Python code that orchestrates the interaction between the LLM and the database tool.

import sqlite3
import json
import os
from pathlib import Path
from ollama import Client

# --- Configuration ---
# Read the Ollama model name from the environment variable or use a default
MODEL_NAME = os.environ.get("OLLAMA_MODEL", "llama3")
DATABASE_NAME = "shop_agent.db"
DATABASE_PATH = Path(__file__).parent / DATABASE_NAME

# --- Tool Definition ---
def get_order_total(order_id: int) -> dict:
    """
    Retrieves the total value of a specific order from the database.

    Args:
        order_id: The ID of the order to retrieve.

    Returns:
        A dictionary containing the order total and currency, or an error message.
    """
    try:
        with sqlite3.connect(f"file:DATABASE_PATH?mode=ro", uri=True) as conn:
            cursor = conn.cursor()
            # Check if the order exists
            cursor.execute("SELECT COUNT(*) FROM orders WHERE order_id = ?", (order_id,))
            if cursor.fetchone()[0] == 0:
                return "error": f"Order ID order_id not found."

            # Calculate the total price for the order
            cursor.execute("""
                SELECT SUM(quantity * price)
                FROM order_items
                WHERE order_id = ?
            """, (order_id,))
            total_price = cursor.fetchone()[0]

            if total_price is None:
                return "error": f"Could not calculate total for order order_id. No items found."

            return "order_id": order_id, "total_value": round(total_price, 2), "currency": "GBP"
    except Exception as e:
        return "error": f"An error occurred while fetching order total: str(e)"

# --- AI Agent Logic ---
def run_agent(user_question: str):
    """
    Runs the AI agent to answer the user's question using available tools.
    """
    ollama_client = Client()
    messages = [
        
            "role": "system",
            "content": f"""You are a helpful assistant for an online shop.
Your goal is to answer customer questions about orders.
You have access to a tool that can retrieve order totals.
Only use the provided tools when necessary to answer the question.
If the requested order does not exist or an error occurs, clearly state that to the user.
Respond in a clear and concise manner.
Your available tools are:
- get_order_total: Retrieves the total value of a specific order.
  - Arguments:
    - order_id (int): The ID of the order to retrieve.
"""
        ,
        "role": "user", "content": user_question,
    ]

    # First interaction: Model decides if a tool is needed
    print("--- First Model Interaction ---")
    response = ollama_client.chat(
        model=MODEL_NAME,
        messages=messages,
        tools=[get_order_total] # Pass the tool function directly
    )

    messages.append(response["message"]) # Add model's response to history

    tool_calls = response["message"].get("tool_calls")

    if tool_calls:
        print(f"Model requested tool call: json.dumps(tool_calls, indent=2)")
        # Process tool calls (for simplicity, we handle one call at a time)
        for tool_call in tool_calls:
            function_name = tool_call["function"]["name"]
            function_args = json.loads(tool_call["function"]["arguments"])

            if function_name == "get_order_total":
                # Execute the tool and get the result
                tool_result = get_order_total(**function_args)
                print(f"Tool 'function_name' returned: json.dumps(tool_result, indent=2)")

                # Add tool result to messages
                messages.append(
                    
                        "tool_call_id": tool_call["id"],
                        "role": "tool",
                        "content": json.dumps(tool_result),
                    
                )
            else:
                print(f"Unknown function name: function_name")
                # Add an error message for unknown function
                messages.append(
                    
                        "tool_call_id": tool_call["id"],
                        "role": "tool",
                        "content": json.dumps("error": f"Unknown function: function_name"),
                    
                )

        # Second interaction: Model generates a final answer based on tool results
        print("n--- Second Model Interaction (Final Answer) ---")
        final_response = ollama_client.chat(
            model=MODEL_NAME,
            messages=messages
        )
        print(f"Final Answer: final_response['message']['content']")
        return final_response['message']['content']

    else:
        # Model answered directly without using a tool
        print(f"Model answered directly: response['message']['content']")
        return response['message']['content']

# --- Main Execution Block ---
if __name__ == "__main__":
    # Example usage
    questions = [
        "What is the total value of order 1001?",
        "Can you tell me the total for order 1001?",
        "What is the total value of order 9999?", # Non-existent order
        "How many items are in order 1001?", # Question the tool can't answer directly
    ]

    for q in questions:
        print(f"n--- Processing Question: 'q' ---")
        run_agent(q)
        print("-" * (len(f"--- Processing Question: 'q' ---") + 5)) # Separator

Deconstructing the Agent’s Mechanism

Let’s break down the functionality of the single_tool.py script:

• Configuration:

  • MODEL_NAME: This variable determines which LLM from Ollama will be used. It’s configured to read from the OLLAMA_MODEL environment variable, allowing flexibility to switch models without altering the script. If the variable isn’t set, it defaults to llama3.
  • DATABASE_NAME and DATABASE_PATH: These define the name and location of our SQLite database file, ensuring it’s consistently found relative to the script’s location.

• Tool Definition (get_order_total function):

  • This is a standard Python function designed to interact with the SQLite database.
  • It takes an order_id as input.
  • The function connects to the database in read-only mode (mode=ro) using a URI for robustness.
  • It first checks if the order_id exists in the orders table. If not, it returns a structured error message.
  • If the order exists, it queries the order_items table to calculate the sum of quantity * price.
  • It returns a JSON-serializable dictionary containing the order_id, the calculated total_value (rounded to two decimal places), and the currency.
  • Error handling is included to catch potential database exceptions.
  • Crucially, the function’s name, type annotations (order_id: int), and docstring serve as a descriptive contract for the LLM, explaining its purpose and expected parameters.

• AI Agent Logic (run_agent function):

  • Initialization: An instance of ollama.Client is created to interact with the local Ollama server.
  • messages List: This list maintains the conversation history.
    • System Message: The initial "system" message provides the LLM with context about its role (a shop assistant), its capabilities (access to get_order_total), and constraints (only use tools when necessary, report errors clearly). This system prompt is vital for guiding the LLM’s behavior.
    • User Message: This contains the actual query from the user.
  • First ollama_client.chat() Call: This is where the magic begins.
    • The model parameter specifies the LLM to use.
    • The messages list is passed to provide the conversation context.
    • tools=[get_order_total] is the key component. The Ollama SDK inspects the get_order_total function (its signature, type hints, and docstring) and converts this information into a JSON schema. This schema is sent to the LLM, not the raw Python code.
  • Processing the Model’s Response:
    • The response from the LLM is appended to the messages history. This is essential for the LLM to "remember" its own previous outputs.
    • The script checks if the LLM’s response contains tool_calls.
    • If tool_calls exist:
      • The script iterates through the requested tool calls. For simplicity, this example handles one tool call at a time.
      • It extracts the function_name and function_args (which are in JSON format and need to be parsed).
      • It then executes the actual Python function (get_order_total(**function_args)). The **function_args syntax unpacks the dictionary into keyword arguments for the function.
      • The tool_result (the dictionary returned by get_order_total) is then added back to the messages history with the role "tool". This informs the LLM of the outcome of the tool execution.
    • If no tool_calls exist: The LLM has provided a direct answer without needing a tool.
  • Second ollama_client.chat() Call:
    • This call is made with the updated messages list, which now includes the LLM’s initial response, its tool request, and the result of the tool execution (if any).
    • The LLM, armed with all this information, can now formulate a final, comprehensive answer.
    • The final_response['message']['content'] contains the LLM’s synthesized answer.

• Main Execution Block (if __name__ == "__main__":)

  • This block defines a list of example questions to test the agent’s capabilities, including valid queries, a query for a non-existent order, and a query that the current tool cannot directly answer.
  • It iterates through these questions, calls run_agent() for each, and prints the results, providing a clear demonstration of the agent’s behavior.

Running Your AI Agent

With the build_shop_database.py script executed and single_tool.py in place, you can now run your AI agent. Open your PowerShell terminal in the project directory and execute:

uv run python single_tool.py

The first time you run this, Ollama might take a moment to load the llama3 model into memory. Observe the output in your terminal. You should see the script processing each question, detailing the model’s tool requests, the tool’s execution, and the final generated answer.

Example Output Snippet (for "What is the total value of order 1001?"):

--- Processing Question: 'What is the total value of order 1001?' ---
--- First Model Interaction ---
Model requested tool call: 
  "id": "call_...",
  "type": "function",
  "function": 
    "name": "get_order_total",
    "arguments": "n  "order_id": 1001n"
  

Tool 'get_order_total' returned: "order_id": 1001, "total_value": 96.95, "currency": "GBP"

--- Second Model Interaction (Final Answer) ---
Final Answer: The total value of order 1001 is GBP 96.95.
-----------------------------------------------------------------

The exact phrasing of the final answer may vary slightly due to the probabilistic nature of LLMs. However, the numerical value should consistently reflect the correct calculation from the database.

Exploring Failure Paths and Robustness

To understand the agent’s robustness, let’s test scenarios where the tool might encounter issues or be insufficient:

1. Querying a Non-Existent Order:

If you ask for the total of an order that doesn’t exist (e.g., "What is the total value of order 9999?"), the get_order_total function is designed to return an error. The LLM should then use this information to inform the user.

Expected Output for "What is the total value of order 9999?":

--- Processing Question: 'What is the total value of order 9999?' ---
--- First Model Interaction ---
Model requested tool call: 
  "id": "call_...",
  "type": "function",
  "function": 
    "name": "get_order_total",
    "arguments": "n  "order_id": 9999n"
  

Tool 'get_order_total' returned: "error": "Order ID 9999 not found."

--- Second Model Interaction (Final Answer) ---
Final Answer: I'm sorry, but order ID 9999 was not found in our system. Please check the order number and try again.
-----------------------------------------------------------------

This demonstrates that the agent correctly processes the error returned by the tool and communicates it to the user.

2. Querying Beyond the Tool’s Capability:

Consider asking a question that the get_order_total tool is not designed to answer, such as "How many items are in order 1001?".

Expected Output for "How many items are in order 1001?":

--- Processing Question: 'How many items are in order 1001?' ---
--- First Model Interaction ---
Model answered directly: I can only provide the total value of an order. I cannot tell you the number of items in order 1001. Would you like to know the total value?
-----------------------------------------------------------------

In this case, the LLM recognizes that its available tool (get_order_total) cannot fulfill the request and opts to answer directly, explaining its limitation. While the system prompt guides this behavior, it’s important to note that LLMs can sometimes "hallucinate" or invent answers. For critical applications, implementing stricter validation within the Python code to ensure that an appropriate tool was actually called and that its output is sufficient is paramount.

Understanding the Model’s View: The JSON Schema

A crucial aspect of tool usage is understanding what information the LLM actually receives. Ollama, through its Python SDK, does not send your Python source code directly to the model. Instead, it inspects the provided function (like get_order_total) and generates a JSON schema that describes the function’s signature, parameters, types, and docstring.

For get_order_total, the LLM might see a schema similar to this:

  "name": "get_order_total",
  "description": "Retrieves the total value of a specific order from the database.",
  "parameters": 
    "type": "object",
    "properties": 
      "order_id": 
        "type": "integer",
        "description": "The ID of the order to retrieve."
      
    ,
    "required": ["order_id"]
  

The LLM uses this schema, along with the conversation history, to determine if the function is relevant to the user’s query and, if so, to generate arguments that conform to the schema. This highlights the importance of well-written docstrings, as they provide the LLM with contextual information for making informed decisions. It’s also important to note that schema generation is not a form of strict validation. The model could still potentially generate arguments that do not precisely match the schema (e.g., a string where an integer is expected). Handling such discrepancies is a key aspect of building more robust agents, which will be explored in future installments.

Debugging Agent Output: Tracing the Execution Flow

When an AI agent produces an unexpected or incorrect output, the first step in debugging is to meticulously examine the execution trace. Three common types of failures can manifest similarly to the end-user:

• Incorrect Tool Call: The LLM might call the wrong tool, provide incorrect arguments to a tool, or fail to call a tool when it should have.
• Tool Execution Error: The tool itself might fail due to unexpected data, database issues, or bugs in the tool’s code.
• LLM Misinterpretation: The LLM might correctly receive tool output but misinterpret it or fail to synthesize it into a coherent final answer.

By printing intermediate steps, as done in single_tool.py, you can pinpoint where the breakdown occurs. For instance, if the tool output is correct, but the final answer is wrong, the issue likely lies in the LLM’s final generation stage. If the tool call itself is malformed, the problem originates in the LLM’s decision-making process.

Experimenting with Different Models

The flexibility of the OLLAMA_MODEL environment variable allows for easy experimentation with different LLMs. To try another model:

1. Pull a New Model: Use Ollama to download another tool-capable model (e.g., mistral):

   ollama pull mistral

2. Set the Environment Variable: In your PowerShell session, before running the script, set the variable:

   $env:OLLAMA_MODEL="mistral"
   uv run python single_tool.py

When switching models, consider the following:

• Resource Requirements: Larger models typically demand more RAM and disk space and may process requests more slowly on the same hardware.
• Performance Comparison: Do not assume that a more fluent final sentence implies superior tool usage. Rigorously test various scenarios: missing order IDs, irrelevant questions, and ambiguous wording. Compare the requested function and arguments generated by each model to assess their effectiveness in tool integration.

Conclusion: A Foundation for Agentic Development

This article has provided a foundational understanding of AI agents, defining them as systems that use LLMs to interact with tools in a loop to achieve goals. We successfully built a simple, functional AI agent capable of querying a local SQLite database using Python and Ollama, all without requiring cloud infrastructure or API keys.

The process covered:

• Defining the core components of an AI agent.
• Setting up a local development environment with uv and Ollama.
• Creating a sample SQLite database for data storage.
• Developing a Python tool function (get_order_total) to interact with the database.
• Orchestrating the agent’s logic, including the LLM’s tool requests and the Python execution of those tools.
• Demonstrating the agent’s operation through practical examples and exploring failure scenarios.
• Explaining the underlying mechanism of JSON schema generation for LLM tool descriptions.

This deliberately simplified example serves as a crucial starting point. By making the mechanics visible, it empowers developers to grasp the fundamental principles of agent development. This knowledge forms a solid basis for constructing more sophisticated agents capable of utilizing multiple tools in sequential or parallel operations, paving the way for increasingly complex and autonomous AI systems. The journey into multi-tool agents and more advanced agentic architectures awaits in subsequent discussions.