Have you ever complained about the uncertainty and limited capability of current agentic workflows? You are not alone. Red Hat is actively engaged in exploring these advancements, collaborating with open source projects like Llama Stack and exploring how we can build smarter agents capable of tackling increasingly complex real world problems.
In this article, we will walk you through our findings on simple agent architectures, culminating in an experiment that demonstrates the significant advantages of more sophisticated techniques, such as prompt chaining and the ReAct framework within the powerful Llama Stack ecosystem.
Single tool calls introduce bottlenecks
While exploring various agentic workflows, we've noticed that many agent implementations, which we’ll refer to as "simple agents," operate with a significant constraint: they can typically invoke only one tool at a time. Even though this might suffice for straightforward user queries, it quickly becomes a bottleneck when dealing with tasks that require multiple steps.
Imagine an agent tasked with finding nearby coffee shops that are open now and have Wi-Fi. A simple agent might provide instructions for searching relevant information or assume a location based on training data and use a web search tool to give recommendations. However, it cannot offer accurate and consistent suggestions without being explicitly given location information. Its lack of reasoning and inability to perform sequential tool calls limits its capacity to handle the complex dependencies often present in real-world scenarios.
Furthermore, these simple agents often rely on manually crafted reasoning steps embedded within their prompts. In contrast, large language models (LLMs), especially fine-tuned ones, demonstrate impressive reasoning capabilities. However, this static approach still lacks the adaptability required to handle unexpected outcomes or to adjust actions based on intermediate results. Additionally, the cost of training and deploying these large fine-tuned models poses a significant barrier for many.
These considerations led us to explore techniques that enhance the small language model-powered agent's ability to orchestrate multiple tool calls and reason dynamically.
Enhanced advanced reasoning
Our investigation led us to two promising techniques: prompt chaining and the ReAct (Reasoning + Acting) framework. These approaches offer elegant solutions to the limitations we observed with simpler agents.
Prompt chaining, depicted in Figure 1, introduces a structured way to connect multiple prompts into a coherent sequence. This allows the agent to maintain context across different tool calls. The output of one tool call feeds directly into the subsequent prompt, enabling the agent to perform multi-step reasoning and build upon the information gathered in previous steps. Think of it as a carefully designed dialogue where each turn leverages the insights from the last, guiding the agent towards the final goal.
ReAct, on the other hand, takes a more dynamic approach by combining reasoning and acting steps in an iterative loop. Instead of relying on pre-defined reasoning, the agent explicitly reasons about the current situation, decides which tool (if any) to use, executes the tool, and then observes the result. This observation then informs the next reasoning step, allowing the agent to adapt its plan based on the information it gathers. This "think-act-observe" cycle, as shown in Figure 2, can outperform existing baselines on decision-making tasks and lead to improved interpretability.
The experiment lowdown
To demonstrate the effectiveness of these techniques, we conducted an experiment comparing a simple agent, an agent utilizing prompt chaining, and an agent leveraging the ReAct framework on a task requiring multiple steps and dynamic decision-making within the Llama Stack.
Simple agent
Lets start by implementing a simple agent via Llama Stack. We’ll enable its built-in web search tool so it can address the simple task, "Find coffee shops near me that are open now, have Wi-Fi."
Here, we use ibm-granite/granite-3.2-8b-instruct as inference model. For detailed instructions on how to set up the Llama stack, please refer to our full demo notebook:
from llama_stack_client.lib.agents.agent import Agent
from llama_stack_client.lib.agents.event_logger import EventLogger
agent = Agent(
client,
model=model_id,
instructions="""You are a helpful websearch assistant. When you are asked to search the web you must use a tool.
Whenever a tool is called, be sure return the response in a friendly and helpful tone.
""" ,
tools=["builtin::websearch"],
sampling_params=sampling_params
)
user_prompts = [
"Find coffee shops near me that are open now, have Wi-Fi."
]
for prompt in user_prompts:
session_id = agent.create_session("simple-agent-session")
response = agent.create_turn(
messages=[{"role": "user","content": prompt,}],
session_id=session_id,
stream=stream
)
for log in EventLogger().log(response):
log.print()==================================================
Processing user query: Find coffee shops near me that are open now, have Wi-Fi.
==================================================
---------- 📍 Step 1: InferenceStep ----------
🛠️ Tool call Generated:
Tool call: brave_search, Arguments: {'query': 'coffee shops open now with Wi-Fi near me'}
---------- 📍 Step 2: ToolExecutionStep ----------
🔧 Executing tool...
{
│ 'query': 'coffee shops open now with Wi-Fi near me',
│ 'top_k': [
│ │ {
│ │ │ 'title': 'TOP 10 BEST Coffee Wifi in Columbus, OH - Updated 2024 - Yelp',
│ │ │ 'url': 'https://www.yelp.com/search?find_desc=Coffee+Wifi&find_loc=Columbus,+OH',
│ │ │ 'content': "Open Now Offers Delivery Offers Takeout Free Wi-Fi Outdoor Seating. Stauf's Coffee - Victorian Village. 4.4 (86 reviews) Coffee Roasteries Coffee & Tea ... Coffee Shops Open Late in Columbus, OH. Coffee Study Spot in Columbus, OH. Coffee and Tea in Columbus, OH. Coffeeshops in Columbus, OH.",
│ │ │ 'score': 0.55275756,
│ │ │ 'raw_content': None
│ │ },
│ │ {
│ │ │ 'title': 'The 15 Best Coffeeshops with WiFi in Columbus - Foursquare',
│ │ │ 'url': 'https://foursquare.com/top-places/columbus/work-with-wifi',
│ │ │ 'content': "Eliska H: I always enjoyed reading Twain, now I like drinking him too. Alex Scott: This is absolutely my favorite coffee shop in town. Amazing atmosphere and great drinks. ... Awesome local restaurant/coffee shop! Don't freak out if it takes a little longer To get your food, and try e West of Philly Sandwich! 13. Grandview Grind. 8.2.",
│ │ │ 'score': 0.43813655,
│ │ │ 'raw_content': None
│ │ },
│ │ {
│ │ │ 'title': '25+ local coffee shops and roasters in Columbus - CBUStoday',
│ │ │ 'url': 'https://cbustoday.6amcity.com/city-guide/live/local-coffee-shops-roasters-in-columbus-oh',
│ │ │ 'content': "This isn't your typical coffee shop, as it also offers a full menu of cocktails and beer. If you're just here for the coffee, the selections include maple-flavored lattes as well as classic drip-style coffee. If you want a little more kick in your drink, take a big sip of the espresso martini. Bottoms Up Coffee",
│ │ │ 'score': 0.14636794,
│ │ │ 'raw_content': None
│ │ },
│ │ {
│ │ │ 'title': '12 Best Coffee Shops in Columbus, Ohio to Try Today (2025 Update)',
│ │ │ 'url': 'https://cornercoffeestore.com/best-coffee-shops-in-columbus-ohio/',
│ │ │ 'content': "Pistacia Vera is a popular coffee shop, and it's easy to see why. It has a friendly staff, great-tasting coffee, and a huge menu of sandwiches, pastries, cookies, and desserts that are sure to make you a regular customer. The only downside is that this shop is usually only open until about 3 p.m., so you have to get there early.",
│ │ │ 'score': 0.13435538,
│ │ │ 'raw_content': None
│ │ },
│ │ {
│ │ │ 'title': '8 Incredible Coffee Shops In Columbus',
│ │ │ 'url': 'https://www.columbusnavigator.com/columbus-coffee-shops/',
│ │ │ 'content': 'Started in the Short North in 1991 as a sort of marketplace for underserved artists, the Gallery has grown over the years. It now calls Clintonville home and supports more than 1,500 artists and producers from around Ohio and the world. And of course, their coffee is sourced from fair trade farms. Location: 3535 N High St, Columbus, OH 43214',
│ │ │ 'score': 0.099917434,
│ │ │ 'raw_content': None
│ │ }
│ ]
}
---------- 📍 Step 3: InferenceStep ----------
🤖 Model Response:
Here are some coffee shops near you that are currently open and offer Wi-Fi:
1. **Stauf's Coffee - Victorian Village**
- [Yelp Listing](https://www.yelp.com/search?find_desc=Coffee+Wifi&find_loc=Columbus,+OH)
- This coffee shop is highly rated on Yelp with 4.4 stars from over 86 reviews. It's known for its cozy atmosphere and a variety of coffee options.
2. **The 15 Best Coffeeshops with WiFi in Columbus**
- [Foursquare Listing](https://foursquare.com/top-places/columbus/work-with-wifi)
- According to Foursquare, Alex Scott highly recommends this coffee shop for its amazing atmosphere and great drinks.
3. **Bottoms Up Coffee**
- [CBUStoday Listing](https://cbustoday.6amcity.com/city-guide/live/local-coffee-shops-roasters-in-columbus-oh)
- This local coffee shop offers a full menu of cocktails and beer along with their coffee. They're known for their maple-flavored lattes and espresso martinis.
4. **Pistacia Vera**
- [Corner Coffee Store Listing](https://cornercoffeestore.com/best-coffee-shops-in-columbus-ohio/)
- Pistacia Vera is a popular choice with a friendly staff and great-tasting coffee. They also offer a wide range of sandwiches, pastries, cookies, and desserts.
5. **The Gallery**
- [Columbus Navigator Listing](https://www.columbusnavigator.com/columbus-coffee-shops/)
- Started in the Short North in 1991, The Gallery now in Clintonville supports over 1,500 artists and producers. They source their coffee from fair trade farms.
Remember to check their current operating hours as they might vary. Enjoy your coffee break!
========== Query processing completed ========== Although the simple agent appears to work fine at first, it hallucinates and incorrectly reports the user's location, even though the pod is actually in Massachusetts.
Prompt chaining agent
Prompt chaining extends simple agents' capabilities by using a consistent session ID across prompts, enabling the output of one tool call to directly inform the next. Here, the original task is manually broken down into two separate tasks:
Step 1: Detect user location.
Step 2: Search for nearby coffee shops.
from src.client_tools import get_location
agent = Agent(
client,
model=model_id,
instructions="""You are a helpful assistant.
When a user asks about their location, you MUST use the get_location tool. When searching for nearby places, you MUST use the websearch tool.
""" ,
tools=[get_location, "builtin::websearch"],
sampling_params=sampling_params
)
user_prompts = [
"Where am I?",
"Find coffee shops near me that are open now, have Wi-Fi."
]
session_id = agent.create_session("prompt-chaining-session")
for prompt in user_prompts:
response = agent.create_turn(
messages=[{"role": "user","content": prompt,}],
session_id=session_id,
stream=stream
)
for log in EventLogger().log(response):
log.print()==================================================
Processing user query: Where am I?
==================================================
---------- 📍 Step 1: InferenceStep ----------
🛠️ Tool call Generated:
Tool call: get_location, Arguments: {'query': 'current location'}
---------- 📍 Step 2: ToolExecutionStep ----------
🔧 Executing tool...
'Your current location is: Cambridge, Massachusetts, US'
---------- 📍 Step 3: InferenceStep ----------
🤖 Model Response:
You are currently in Cambridge, Massachusetts, US.
========== Query processing completed ==========
==================================================
Processing user query: Find coffee shops near me that are open now, have Wi-Fi.
==================================================
---------- 📍 Step 1: InferenceStep ----------
🛠️ Tool call Generated:
Tool call: brave_search, Arguments: {'query': 'coffee shops open now with Wi-Fi in Cambridge, Massachusetts'}
---------- 📍 Step 2: ToolExecutionStep ----------
🔧 Executing tool...
{
│ 'query': 'coffee shops open now with Wi-Fi in Cambridge, Massachusetts',
│ 'top_k': [
│ │ {
│ │ │ 'title': 'TOP 10 BEST Coffee Wifi in Cambridge, MA - Updated 2024 - Yelp',
│ │ │ 'url': 'https://www.yelp.com/search?find_desc=coffee+wifi&find_loc=Cambridge,+MA',
│ │ │ 'content': "Top 10 Best Coffee Wifi in Cambridge, MA - December 2024 - Yelp - Broadsheet Coffee Roasters, Curio Coffee, bōm dough, Simon's Coffee Shop, Café Luna, Pepita Coffee, Revival Cafe & Kitchen, Cafe Zing, Diesel Cafe, The Sink ... Open Now Reservations Offers Delivery Offers Takeout Free Wi-Fi. 1. ... Coffee Shops Free Wifi Study in Cambridge, MA",
│ │ │ 'score': 0.90018827,
│ │ │ 'raw_content': None
│ │ },
│ │ {
│ │ │ 'title': 'Coffee Shops Open Late Cambridge, MA - Last Updated May 2025 - Yelp',
│ │ │ 'url': 'https://www.yelp.com/search?find_desc=Coffee+Shops+Open+Late&find_loc=Cambridge,+MA',
│ │ │ 'content': "Top 10 Best Coffee Shops Open Late in Cambridge, MA - May 2025 - Yelp - Cabot Café, Caffe Paradiso, Caffé Vittoria, Cafe Anthony, The Coffee Trike, Coffee Central, Mighty Squirrel Brewing, Dunkin', McDonald's. ... Open Now Offers Delivery Offers Takeout Free Wi-Fi Outdoor Seating.",
│ │ │ 'score': 0.8976328,
│ │ │ 'raw_content': None
│ │ },
│ │ {
│ │ │ 'title': "Coffee Connoisseur's Guide: 20 Top Coffee Shops in Cambridge, MA",
│ │ │ 'url': 'https://thebostondaybook.com/coffee-shops-in-cambridge/',
│ │ │ 'content': 'Join us as we explore some of the best coffee shops in Cambridge, MA, and discover the rich coffee culture that the city has to offer. 1. 1369 Coffee House. ... One of the perks of visiting Starbucks is the availability of WiFi. This makes it a convenient spot for students and remote workers, Rating: 8.2. Phone: +1 617-491-0442',
│ │ │ 'score': 0.8479807,
│ │ │ 'raw_content': None
│ │ },
│ │ {
│ │ │ 'title': '20 Best Coffee Shops in Cambridge, MA - CoffeeSpots.com',
│ │ │ 'url': 'https://coffeespots.com/best-coffee-shops-in-cambridge-ma/',
│ │ │ 'content': 'Jaho Coffee Roaster & Wine Bar. 9.2/10 (182 reviews). Website ]
}
---------- 📍 Step 3: InferenceStep ----------
🤖 Model Response:
Here are some coffee shops near you in Cambridge, Massachusetts that are open now and offer Wi-Fi:
1. [TOP 10 BEST Coffee Wifi in Cambridge, MA - Updated 2024 - Yelp](https://www.yelp.com/search?find_desc=coffee+wifi&find_loc=Cambridge,+MA) - Broadsheet Coffee Roasters, Curio Coffee, b\u014dm dough, Simon's Coffee Shop, Caf\u00e9 Luna, Pepita Coffee, Revival Cafe & Kitchen, Cafe Zing, Diesel Cafe, The Sink.
2. [Coffee Shops Open Late Cambridge, MA - Last Updated May 2025 - Yelp](https://www.yelp.com/search?find_desc=Coffee+Shops+Open+Late&find_loc=Cambridge,+MA) - Cabot Café, Caffe Vittoria, Caffè Anthony, The Coffee Trike, Coffee Central, Mighty Squirrel Brewing, Dunkin', McDonald's.
3. [Coffee Connoisseur's Guide: 20 Top Coffee Shops in Cambridge, MA](https://thebostondaybook.com/coffee-shops-in-cambridge/) - 1369 Coffee House, Simulacrum Coffee, Diesel Cafe, and more.
4. [20 Best Coffee Shops in Cambridge, MA - CoffeeSpots.com](https://coffeespots.com/best-coffee-shops-in-cambridge-ma/) - Jaho Coffee Roaster & Wine Bar, known for exceptional service, unique coffee creations, and a cozy ambiance.
5. [THE 10 BEST Cafés in Cambridge (Updated 2025) - Tripadvisor](https://www.tripadvisor.com/Restaurants-g60890-c8-Cambridge_Massachusetts.html) - A list of top-rated cafes in Cambridge, many of which offer Wi-Fi.
Please check their current status and opening hours before visiting. Enjoy your coffee!
========== Query processing completed ========== In this setup, prompt chaining enables the agent to use the client tool and the web search tool, allowing it to generate accurate recommendations without needing explicit location input.
ReAct agent
While our prompt chaining agent showed improvements over the baseline simple agent by successfully executing sequential tool calls, the ReAct approach with ReAct-style prompting performed even better, delivering a smoother multi-step experience and achieving higher tool-calling accuracy.
The following is an example of ReAct-style prompting, inspired by few-shot learning in the literature. It enables the core capabilities of a ReAct agent by injecting a few examples into the system prompt:
DEFAULT_REACT_AGENT_SYSTEM_PROMPT_TEMPLATE = """
You are an expert assistant [...] To do so, you have been given access to the following tools: <<tool_names>>
You must always respond in the following JSON format:
{
"thought": $THOUGHT_PROCESS,
"action": {
"tool_name": $TOOL_NAME,
"tool_params": $TOOL_PARAMS
},
"answer": $ANSWER
}
Specifically, this json should have a `thought` key, a `action` key and an `answer` key.
[... specificying the expected output of thought, action and answer ...]
[... few shots examples defined in the output format, e.g. JSON to show case reasoning traces ...]The following is how you define a ReAct agent in Llama Stack:
from llama_stack_client.lib.agents.react.agent import ReActAgent
from llama_stack_client.lib.agents.react.tool_parser import ReActOutput
agent = ReActAgent(
client=client,
model=model_id,
tools=[get_location, "builtin::websearch"],
response_format={
"type": "json_schema",
"json_schema": ReActOutput.model_json_schema(),
},
sampling_params=sampling_params,
)
user_prompts = [
"Find coffee shops near me that are open now, have Wi-Fi."
]
session_id = agent.create_session("ReAct-agent-session")
for prompt in user_prompts:
response = agent.create_turn(
messages=[{"role": "user","content": prompt,}],
session_id=session_id,
stream=stream
)
for log in EventLogger().log(response):
log.print()==================================================
Processing user query: Are there any weather-related risks in my area that could disrupt network connectivity or system availability?
==================================================
---------- 📍 Step 1: InferenceStep ----------
🤖 Model Response:
{
"thought": "I need to find out the user's current location and then check for any weather-related risks that could disrupt network connectivity or system availability. I will use the `get_location` tool to get the user's location and then the `web_search` tool to find any weather-related risks.",
"action": {
"tool_name": "get_location",
"tool_params": [{"name": "query", "value": "user's current location"}]
},
"answer": null
}
---------- 📍 Step 2: ToolExecutionStep ----------
🔧 Executing tool...
'Your current location is: Cambridge, Massachusetts, US'
---------- 📍 Step 3: InferenceStep ----------
🤖 Model Response:
{
"thought": "Now that I have the user's location, I will use the `web_search` tool to find any weather-related risks that could disrupt network connectivity or system availability in Cambridge, Massachusetts, US.",
"action": {
"tool_name": "web_search",
"tool_params": [{"name": "query", "value": "weather-related risks disrupting network connectivity or system availability in Cambridge, Massachusetts, US"}]
},
"answer": null
}
---------- 📍 Step 4: ToolExecutionStep ----------
🔧 Executing tool...
{
│ 'query': 'weather-related risks disrupting network connectivity or system availability in Cambridge, Massachusetts, US',
│ 'top_k': [
│ │ {
│ │ │ 'title': 'Weather in Cambridge, Massachusetts, US',
│ │ │ 'url': 'https://www.weatherapi.com/',
│ │ │ 'content': "{'location': {'name': 'Cambridge', 'region': 'Massachusetts', 'country': 'United States of America', 'lat': 42.375, 'lon': -71.1061, 'tz_id': 'America/New_York', 'localtime_epoch': 1746639190, 'localtime': '2025-05-07 13:33'}, 'current': {'last_updated_epoch': 1746639000, 'last_updated': '2025-05-07 13:30', 'temp_c': 21.1, 'temp_f': 70.0, 'is_day': 1, 'condition': {'text': 'Partly cloudy', 'icon': '//cdn.weatherapi.com/weather/64x64/day/116.png', 'code': 1003}, 'wind_mph': 10.3, 'wind_kph': 16.6, 'wind_degree': 206, 'wind_dir': 'SSW', 'pressure_mb': 1014.0, 'pressure_in': 29.94, 'precip_mm': 0.0, 'precip_in': 0.0, 'humidity': 49, 'cloud': 75, 'feelslike_c': 21.1, 'feelslike_f': 70.0, 'windchill_c': 21.2, 'windchill_f': 70.2, 'heatindex_c': 22.5, 'heatindex_f': 72.5, 'dewpoint_c': 13.1, 'dewpoint_f': 55.5, 'vis_km': 16.0, 'vis_miles': 9.0, 'uv': 3.6, 'gust_mph': 12.9, 'gust_kph': 20.7}}",
│ │ │ 'score': 0.5199194,
│ │ │ 'raw_content': None
│ │ },
│ │ {
│ │ │ 'title': 'How Does Bad Weather Disrupt Your Internet Connectivity',
│ │ │ 'url': 'https://yostream.io/blog/bad-weather-disrupts-internet-connectivity/',
│ │ │ 'content': 'Directly, lightning can strike network infrastructure such as cell towers, data centers, or internet exchanges, causing physical damage and disrupting connectivity. Severe weather conditions like hurricanes, floods, or landslides can cause physical damage to the fiber optic cables, resulting in outages or disruptions to internet connectivity. In rare cases, power outages caused by bad weather can also impact fiber optic internet connectivity, particularly if the internet service provider’s network infrastructure relies on power-dependent equipment. Internet outages and downtime can be caused by various weather-related factors, including power outages, physical damage to infrastructure, and flooding that affects data centers and network equipment. Heavy rainfall, thunderstorms, and other severe weather events can exacerbate network congestion by disrupting the infrastructure that supports internet connectivity.',
│ │ │ 'score': 0.4395795,
│ │ │ 'raw_content': None
│ │ },
│ │ {
│ │ │ 'title': 'How Does Weather Impact Your WiFi? ]
}
---------- 📍 Step 5: InferenceStep ----------
🤖 Model Response:
{
"thought": "Based on the search results, there are several ways weather can disrupt network connectivity or system availability in Cambridge, Massachusetts, US. These include direct strikes from lightning, physical damage to infrastructure from severe weather conditions, power outages, and network congestion. However, the current weather forecast for Cambridge, Massachusetts, US does not indicate any imminent severe weather events that could cause such disruptions.",
"action": null,
"answer": "While there are potential weather-related risks that could disrupt network connectivity or system availability in Cambridge, Massachusetts, US, the current weather forecast does not indicate any imminent severe weather events that could cause such disruptions."
}
========== Query processing completed ==========
As shown in the code output, the agent is able to automatically reason the correct tool sequence and execute it accordingly through a thought-action-observation loop, overall providing a smoother user experience.
Final thoughts
Based on our findings, we suggest trying frameworks like ReAct when building agents for complex, real-world applications. While prompt chaining offers a valuable step forward from basic single-tool agents, the dynamic reasoning and adaptability provided by ReAct unlock a new level of intelligence and robustness. By enabling agents to think, act, and observe, we can build systems that are far more capable of handling the messy and unpredictable nature of real-world tasks.
For those interested in diving deeper into these concepts and seeing them in action, we encourage you to explore the following resources:
- Delve into our Jupyter notebook and discover our advanced agentic series.
- Read the original ReAct prompting paper.
