Build an AI Agent from Scratch: Python Tutorial with OpenAI Function Calling
TL;DR — Build an AI agent from scratch: no framework, just Python + OpenAI. Leonie Monigatti: "Build from scratch using LLM API calls, function calling, and conversation memory. Frameworks exist (CrewAI, LangGraph) but building from scratch teaches fundamentals." ofox.ai: "Agents go beyond chatbots — calling APIs, executing code, making decisions in a loop. Function calling fundamentals → complete working agent with memory, multi-tool orchestration." OpenAI Guide: "While loop is central to agent functioning. Run multiple steps until exit condition met. Clear system prompt defines role, tools, constraints." AgileSoft Labs: "Most valuable skill in 2026. Autonomous systems that reason, plan, use tools, execute multi-step tasks." DEV.to: "Reason about next steps. Act on the world. Loop until goal achieved." Learn more with what is AI agent, function calling, tool loop, and agent memory.
Leonie Monigatti asks the key question: "What's the best way to get started building AI agent systems? There are countless frameworks for building AI agents available, such as CrewAI, LangGraph, and the OpenAI Agents SDK — but building from scratch teaches fundamentals."
ofox.ai frames the progression: "AI agents go beyond chatbots by taking autonomous actions — calling APIs, executing code, and making decisions in a loop. This guide builds agents from scratch in Python, starting with function calling fundamentals and progressing to a complete working agent with memory, multi-tool orchestration."
Agent Architecture
role, tools, constraints
output format"] Tools["Tool Definitions
function schemas
name, description, parameters"] Memory["Conversation Memory
messages list
system + user + assistant + tool"] end subgraph Loop["Agent Loop (while not done)"] Call["1. Call LLM
messages + tools
model decides next action"] Check{"2. Tool Calls?"} Execute["3. Execute Tools
run Python functions
get results"] Append["4. Append Results
role: tool messages
back to LLM"] Done["5. Final Answer
no tool calls
return content"] MaxStep["6. Max Steps
safety limit
return fallback"] end subgraph Tools["Available Tools"] Search["search(query)
web search"] Calculate["calculate(expr)
math computation"] FileRead["file_read(path)
read file contents"] FileWrite["file_write(path, content)
write to file"] HTTP["http_get(url)
fetch URL"] end System --> Call Tools --> Call Memory --> Call Call --> Check Check -->|Yes| Execute Execute --> Append Append --> Call Check -->|No| Done Call --> MaxStep Execute --> Search Execute --> Calculate Execute --> FileRead Execute --> FileWrite Execute --> HTTP
Agent Components
| Component | Purpose | Implementation |
|---|---|---|
| System prompt | Define role, tools, constraints | String in messages[0] |
| Tool definitions | Tell LLM what tools exist | OpenAI function schemas |
| Tool functions | Execute when called | Python functions |
| Agent loop | Run until goal reached | while loop with max_steps |
| Conversation memory | Persist across steps | messages list |
| Error handling | Handle tool failures | try/except per tool |
Implementation
from dataclasses import dataclass
from typing import Optional
from enum import Enum
import json
class AgentState(Enum):
RUNNING = "running"
DONE = "done"
MAX_STEPS = "max_steps"
ERROR = "error"
@dataclass
class BuildAgentFromScratchGuide:
"""Build AI agent from scratch implementation guide."""
def get_complete_agent(self) -> str:
"""Complete AI agent from scratch."""
return (
"import json\n"
"from openai import OpenAI\n"
"\n"
"client = OpenAI()\n"
"\n"
"# 1. Define tool functions\n"
"def search(query: str) -> str:\n"
" '''Search the web.'''\n"
" # Implement web search\n"
" return f'Results for: {query}'\n"
"\n"
"def calculate(expression: str) -> str:\n"
" '''Evaluate math.'''\n"
" try:\n"
" return str(eval(\n"
" expression))\n"
" except Exception as e:\n"
" return f'Error: {e}'\n"
"\n"
"def file_read(path: str) -> str:\n"
" '''Read file contents.'''\n"
" try:\n"
" with open(path) as f:\n"
" return f.read()\n"
" except Exception as e:\n"
" return f'Error: {e}'\n"
"\n"
"def file_write(\n"
" path: str, content: str\n"
") -> str:\n"
" '''Write to file.'''\n"
" try:\n"
" with open(path, 'w') as f:\n"
" f.write(content)\n"
" return 'File written'\n"
" except Exception as e:\n"
" return f'Error: {e}'\n"
"\n"
"# 2. Map tool names to functions\n"
"TOOL_FUNCTIONS = {\n"
" 'search': search,\n"
" 'calculate': calculate,\n"
" 'file_read': file_read,\n"
" 'file_write': file_write,\n"
"}\n"
"\n"
"# 3. Define tool schemas\n"
"TOOLS = [\n"
" {\n"
" 'type': 'function',\n"
" 'function': {\n"
" 'name': 'search',\n"
" 'description':\n"
" 'Search the web'\n"
" ' for current'\n"
" ' information.',\n"
" 'parameters': {\n"
" 'type': 'object',\n"
" 'properties': {\n"
" 'query': {\n"
" 'type': 'string',\n"
" 'description':\n"
" 'Search query'\n"
" }\n"
" },\n"
" 'required': ['query']\n"
" }\n"
" }\n"
" },\n"
" {\n"
" 'type': 'function',\n"
" 'function': {\n"
" 'name': 'calculate',\n"
" 'description':\n"
" 'Evaluate a math'\n"
" ' expression.',\n"
" 'parameters': {\n"
" 'type': 'object',\n"
" 'properties': {\n"
" 'expression': {\n"
" 'type': 'string',\n"
" 'description':\n"
" 'Math expression'\n"
" }\n"
" },\n"
" 'required':\n"
" ['expression']\n"
" }\n"
" }\n"
" },\n"
" {\n"
" 'type': 'function',\n"
" 'function': {\n"
" 'name': 'file_read',\n"
" 'description':\n"
" 'Read file'\n"
" ' contents.',\n"
" 'parameters': {\n"
" 'type': 'object',\n"
" 'properties': {\n"
" 'path': {\n"
" 'type': 'string',\n"
" 'description':\n"
" 'File path'\n"
" }\n"
" },\n"
" 'required': ['path']\n"
" }\n"
" }\n"
" },\n"
" {\n"
" 'type': 'function',\n"
" 'function': {\n"
" 'name': 'file_write',\n"
" 'description':\n"
" 'Write content to'\n"
" ' a file.',\n"
" 'parameters': {\n"
" 'type': 'object',\n"
" 'properties': {\n"
" 'path': {\n"
" 'type': 'string'\n"
" },\n"
" 'content': {\n"
" 'type': 'string'\n"
" }\n"
" },\n"
" 'required':\n"
" ['path',\n"
" 'content']\n"
" }\n"
" }\n"
" }\n"
"]\n"
"\n"
"# 4. System prompt\n"
"SYSTEM_PROMPT = '''You are an\n"
"autonomous AI agent. You help\n"
"users by using tools.\n"
"\n"
"Guidelines:\n"
"- Break complex tasks into steps\n"
"- Use tools when needed\n"
"- Reason before acting\n"
"- Verify tool results\n"
"- Never hallucinate outputs\n"
"- Stop when task is complete\n"
"\n"
"Available tools:\n"
"- search: web search\n"
"- calculate: math\n"
"- file_read: read files\n"
"- file_write: write files\n"
"'''\n"
"\n"
"# 5. Agent loop\n"
"def run_agent(\n"
" user_message: str,\n"
" max_steps: int = 10\n"
") -> str:\n"
" '''Run agent until done.'''\n"
" messages = [\n"
" {'role': 'system',\n"
" 'content': SYSTEM_PROMPT},\n"
" {'role': 'user',\n"
" 'content': user_message}\n"
" ]\n"
" \n"
" for step in range(max_steps):\n"
" print(f'--- Step {step+1} ---')\n"
" \n"
" # Call LLM\n"
" response = client.chat\\\n"
" .completions.create(\n"
" model='gpt-4o',\n"
" messages=messages,\n"
" tools=TOOLS)\n"
" \n"
" msg = response\\\n"
" .choices[0].message\n"
" messages.append(msg)\n"
" \n"
" # Check if done\n"
" if not msg.tool_calls:\n"
" return msg.content\n"
" \n"
" # Execute tools\n"
" for call in msg\\\n"
" .tool_calls:\n"
" name = call.function\\\n"
" .name\n"
" args = json.loads(\n"
" call.function\n"
" .arguments)\n"
" print(f' Tool: {name}'\n"
" f' {args}')\n"
" \n"
" func = TOOL_FUNCTIONS\\\n"
" .get(name)\n"
" if func:\n"
" try:\n"
" result = func(**args)\n"
" except Exception as e:\n"
" result = f'Error: {e}'\n"
" else:\n"
" result = (\n"
" f'Unknown tool: {name}')\n"
" \n"
" messages.append({\n"
" 'role': 'tool',\n"
" 'tool_call_id':\n"
" call.id,\n"
" 'content': str(result)\n"
" })\n"
" \n"
" return 'Max steps reached'\n"
"\n"
"# 6. Run the agent\n"
"result = run_agent(\n"
" 'Search for the latest Python'\n"
" ' version, then calculate'\n"
" ' 2^10')\n"
"print(result)"
)
def get_async_agent(self) -> str:
"""Async agent for concurrent tool execution."""
return (
"import asyncio\n"
"import json\n"
"from openai import AsyncOpenAI\n"
"\n"
"client = AsyncOpenAI()\n"
"\n"
"async def run_agent_async(\n"
" user_message: str,\n"
" max_steps: int = 10\n"
") -> str:\n"
" messages = [\n"
" {'role': 'system',\n"
" 'content': SYSTEM_PROMPT},\n"
" {'role': 'user',\n"
" 'content': user_message}\n"
" ]\n"
" \n"
" for step in range(max_steps):\n"
" response = await client\\\n"
" .chat.completions\\\n"
" .create(\n"
" model='gpt-4o',\n"
" messages=messages,\n"
" tools=TOOLS)\n"
" \n"
" msg = response\\\n"
" .choices[0].message\n"
" messages.append(msg)\n"
" \n"
" if not msg.tool_calls:\n"
" return msg.content\n"
" \n"
" # Execute tools concurrently\n"
" async def exec_tool(call):\n"
" name = call.function\\\n"
" .name\n"
" args = json.loads(\n"
" call.function\n"
" .arguments)\n"
" func = TOOL_FUNCTIONS\\\n"
" .get(name)\n"
" if func:\n"
" try:\n"
" result = func(**args)\n"
" except Exception as e:\n"
" result = f'Error: {e}'\n"
" else:\n"
" result = (\n"
" f'Unknown: {name}')\n"
" return {\n"
" 'role': 'tool',\n"
" 'tool_call_id':\n"
" call.id,\n"
" 'content': str(result)\n"
" }\n"
" \n"
" results = await asyncio\\\n"
" .gather(*[\n"
" exec_tool(c)\n"
" for c in msg.tool_calls\n"
" ])\n"
" messages.extend(results)\n"
" \n"
" return 'Max steps reached'\n"
"\n"
"# Run async agent\n"
"result = asyncio.run(\n"
" run_agent_async('Hello!'))"
)
def get_memory_management(self) -> dict:
"""Memory management strategies."""
return {
"short_term": (
"Messages list persists across "
"loop iterations. LLM sees "
"full conversation history "
"on each call."
),
"context_window": (
"For long conversations, "
"implement sliding window: "
"keep system prompt + last N "
"messages. Summarize older "
"messages."
),
"long_term": (
"Persist interactions to "
"database (PostgreSQL, "
"vector DB). Retrieve "
"relevant context using "
"semantic search before "
"each agent run."
),
"summarization": (
"When conversation exceeds "
"context limit, summarize "
"previous messages into "
"compact form. Replace "
"old messages with summary."
),
}
def get_error_handling(self) -> dict:
"""Error handling strategies."""
return {
"tool_errors": (
"Wrap each tool execution in "
"try/except. Return error "
"message as tool result so "
"LLM can adapt."
),
"max_steps": (
"Safety limit prevents "
"infinite loops. Default 10. "
"Return fallback message "
"when exceeded."
),
"unknown_tool": (
"If LLM calls unknown tool, "
"return error message. LLM "
"learns from feedback."
),
"invalid_args": (
"If tool arguments are "
"invalid, return error. "
"LLM may retry with "
"correct arguments."
),
"api_errors": (
"Handle OpenAI API errors "
"(rate limits, timeouts) "
"with retry and fallback."
),
}
def get_production_tips(self) -> dict:
"""Production tips."""
return {
"logging": (
"Log each step: tool name, "
"arguments, result, step "
"number. Essential for "
"debugging agent behavior."
),
"streaming": (
"Use stream=True for "
"real-time output. User "
"sees reasoning as it "
"happens."
),
"human_approval": (
"For sensitive actions "
"(file_write, HTTP calls), "
"require human approval "
"before execution."
),
"cost_control": (
"Track tokens per step. "
"Set budget limit. Stop "
"agent if budget exceeded."
),
"observability": (
"Log reasoning traces, "
"tool calls, outcomes. "
"Use Langfuse or similar "
"for agent observability."
),
}
Build AI Agent from Scratch Checklist
- [ ] No framework needed — just OpenAI SDK + Python functions
- [ ] Install: pip install openai
- [ ] Set OPENAI_API_KEY environment variable
- [ ] Define tool functions: search, calculate, file_read, file_write, http_get
- [ ] Map tool names to Python functions in TOOL_FUNCTIONS dict
- [ ] Define tool schemas: OpenAI function format with name, description, parameters
- [ ] Tool schema: type='function', function with name, description, JSON schema parameters
- [ ] Create system prompt: role, guidelines, available tools, constraints
- [ ] System prompt: "You are an autonomous AI agent. Break tasks into steps. Use tools when needed."
- [ ] System prompt: "Reason before acting. Verify tool results. Never hallucinate. Stop when complete."
- [ ] Initialize messages: system prompt + user message
- [ ] Agent loop: for step in range(max_steps)
- [ ] Call LLM: client.chat.completions.create with model, messages, tools
- [ ] Check if done: if not msg.tool_calls → return msg.content
- [ ] Execute tools: for call in msg.tool_calls → run function, append result
- [ ] Append tool results as role='tool' messages with tool_call_id
- [ ] Max steps safety limit: default 10, return fallback when exceeded
- [ ] Error handling: try/except per tool, return error as tool result
- [ ] Unknown tool: return error message, LLM adapts from feedback
- [ ] Invalid args: return error, LLM may retry with correct arguments
- [ ] API errors: handle rate limits and timeouts with retry
- [ ] Memory: messages list persists across loop iterations
- [ ] Memory: LLM sees full conversation history on each call
- [ ] Context window: sliding window for long conversations (keep system + last N)
- [ ] Context window: summarize older messages into compact form
- [ ] Long-term memory: persist to PostgreSQL or vector DB, retrieve with semantic search
- [ ] Async agent: use AsyncOpenAI and asyncio.gather for concurrent tool execution
- [ ] Async: execute multiple tool calls in parallel when LLM requests them
- [ ] Logging: log each step — tool name, arguments, result, step number
- [ ] Streaming: use stream=True for real-time reasoning output
- [ ] Human approval: require approval for sensitive actions (file_write, HTTP calls)
- [ ] Cost control: track tokens per step, set budget limit, stop if exceeded
- [ ] Observability: log reasoning traces, tool calls, outcomes with Langfuse
- [ ] While loop is central to agent functioning — run until exit condition met
- [ ] LLM decides when to call tools and when to stop — loop just executes decisions
- [ ] Frameworks exist (CrewAI, LangGraph, OpenAI Agents SDK) but scratch teaches fundamentals
- [ ] AI agents go beyond chatbots: calling APIs, executing code, making decisions in a loop
- [ ] Building from scratch is one of the most valuable skills in 2026
- [ ] Function calling enables LLM to decide which tool to use based on user request
- [ ] LLM receives tool definitions and decides which to call with what arguments
- [ ] Read what is AI agent for architecture
- [ ] Read function calling for tool schemas
- [ ] Read tool loop for ReAct pattern
- [ ] Read agent memory for persistence
- [ ] Test: agent completes multi-step task using tools
- [ ] Test: agent stops at max_steps without infinite loop
- [ ] Test: tool errors handled gracefully with error messages
- [ ] Test: memory persists across loop iterations
- [ ] Test: async agent executes tools concurrently
- [ ] Test: logging captures all steps for debugging
- [ ] Document tools, system prompt, max_steps, memory strategy, and error handling
FAQ
How do you build an AI agent from scratch in Python?
Use OpenAI function calling with a tool use loop, system prompt, and conversation memory — no framework needed. Leonie Monigatti: "Build an AI agent from scratch in Python using LLM API calls, function calling, and conversation memory. What is the best way to get started? Countless frameworks exist (CrewAI, LangGraph, OpenAI Agents SDK) but building from scratch teaches fundamentals." ofox.ai: "AI agents go beyond chatbots by taking autonomous actions — calling APIs, executing code, making decisions in a loop. This guide builds agents from scratch in Python, starting with function calling fundamentals and progressing to a complete working agent with memory, multi-tool orchestration." AgileSoft Labs: "Building an AI agent from scratch is one of the most valuable skills in 2026. Unlike traditional chatbots, AI agents are autonomous systems that reason, plan, use tools, and execute multi-step tasks without constant human intervention." Steps: (1) Define tools as function schemas. (2) Create system prompt. (3) Initialize conversation memory. (4) Run agent loop: call LLM → check for tool calls → execute tools → append results → repeat. (5) Stop when LLM returns final answer without tool calls.
What is the agent loop pattern in Python?
The agent loop is a while loop that calls the LLM, checks for tool calls, executes them, and repeats until done. OpenAI Guide: "This concept of a while loop is central to the functioning of an agent. In multi-agent systems, you can have a sequence of tool calls and handoffs between agents but allow the model to run multiple steps until an exit condition is met." DEV.to: "Reason about what to do next (planning, self-reflection). Act on the world (run tools, call APIs). Loop until goal is achieved." Loop: while not done: (1) Call LLM with messages and tools. (2) If LLM returns tool_calls: execute each tool, append results to messages. (3) If LLM returns content only (no tool_calls): done = True, return content. (4) If max_steps reached: return fallback. The LLM decides when to call tools and when to stop — the loop just executes its decisions.
How do you define tools for an AI agent?
Define tools as OpenAI function schemas with name, description, and JSON schema parameters. ofox.ai: "Starting with function calling fundamentals and progressing to complete working agent with multi-tool orchestration." Leonie Monigatti: "Function calling enables the LLM to decide which tool to use based on the user request. The LLM receives tool definitions and decides which to call." Tool schema: {type: 'function', function: {name: 'search', description: 'Search the web', parameters: {type: 'object', properties: {query: {type: 'string'}}, required: ['query']}}}. Each tool maps to a Python function that executes when called. The LLM sees the schema, decides which tool to call, and generates arguments. Your code executes the function and returns the result.
How do you add memory to an AI agent?
Maintain a conversation history list that persists across loop iterations. Leonie Monigatti: "Conversation memory — the agent remembers previous interactions and tool results." DEV.to: "Memory: short-term (current conversation) and long-term (past interactions). The agent maintains state across interactions." Memory implementation: (1) messages list initialized with system prompt and user message. (2) Each LLM response appended to messages. (3) Each tool result appended as role: 'tool' message. (4) LLM sees full history on each call. (5) For long conversations, implement summarization or sliding window to manage context length. (6) For long-term memory, persist to database (PostgreSQL, vector DB) and retrieve relevant context.
What should the system prompt for an AI agent include?
The system prompt defines the agent role, available tools, constraints, and behavior guidelines. OpenAI Guide: "An effective strategy for managing complexity is to give the model a clear system prompt that defines its role, available tools, and constraints." DEV.to: "System prompt: define agent role, reasoning approach, tool usage guidelines, and output format." System prompt should include: (1) Role: 'You are an autonomous agent that helps users by using tools.' (2) Instructions: 'Break complex tasks into steps. Use tools when needed. Reason before acting.' (3) Tool guidance: 'Use search for current information. Use calculate for math. Use file_read to read files.' (4) Constraints: 'Always verify tool results. Never hallucinate tool outputs. Stop when task is complete.' (5) Output format: 'Provide clear, structured responses.'
Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →