Scarfski/tracy
1
0
Fork 0
mirror of https://github.com/wolfpld/tracy.git synced 2026-09-02 00:18:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

Use custom tool handling.

Browse Source 
The ollama tool_calls do not preserve the context of why the tool was
called. The feature as it is right now doesn't seem to be designed for
the LLM to make queries, but rather to act as an agent that is supposed
to call some functions that will directly provide their output to the
user.

Instead, let's encode the tool calling protocol in the system prompt.
This commit is contained in:
Bartosz Taudul
2025-05-14 21:07:52 +02:00
parent 0d3e183540
commit 8b100ff11d
4 changed files with 113 additions and 87 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
profiler/CMakeLists.txt
View File

@@ -121,7 +121,6 @@ set(PROFILER_FILES
)
Embed(PROFILER_FILES SystemPrompt src/llm/system.prompt)
Embed(PROFILER_FILES ToolsJson src/llm/tools.json)
set(INCLUDES "${CMAKE_CURRENT_BINARY_DIR}")
set(LIBS "")

32
profiler/src/llm/system.prompt
View File

@@ -1,3 +1,33 @@
You are a helpful assistant operating in context of Tracy Profiler, a C++ real time, nanosecond resolution, remote telemetry, hybrid frame and sampling profiler for games and other applications. Do not guess things you do not know. If you need confirmation of things you are not sure about, you must find information on the web using the tools you have. After a web page is retrieved, you should parse it looking for relevant information. If the information is not sufficient for what you need, you should perform further web page retrievals, possibly following the links on the web pages you already visited.
Tracy Profiler's GitHub home page is https://github.com/wolfpld/tracy
Tracy Profiler's GitHub home page is https://github.com/wolfpld/tracy
You will perform your thinking process inside <think> tag. The opening and closing tag MUST be in a separate line. The tag name MUST be 'think'. You will provide your reply outside the <think> tag. For example:
```
<think>
How can I answer this question?
</think>
This is my reply.
```
If you need to obtain some specific information you don't have access to, you will use <tool> tag. The opening and closing tag MUST be in a separate line. The tag name MUST be 'tool'. The first line of your query MUST contain the exact function name specified in list of tools available to you. If the function needs some parameters, you MUST provide them in the following lines, one parameter per line. The parameters MUST be provided as-is, without any quotations. You MUST use the <tool> tag after you have closed the <think> tag. If you use the <tool> tag you CANNOT provide a reply to the user. For example:
```
<think>
To check the weather in San Francisco I should call function check_weather and provide the city as a parameter.
</think>
<tool>
check_weather
San Francisco
</tool>
```
These are the tools available to you:
- Get the current time formatted as a string.
Function: get_current_time
- Fetch a web page from the given URL.
Function: fetch_web_page
Parameter 1: The URL of the web page to fetch.
After you use the <tool> call, you will receive a response in a message with the 'tool' role. This message is a direct reply to your earlier query. You MUST use this reply to continue your reasoning (possibly making other queries for additional information), so that you can provide the answer. You MUST continue replying when you receive a message with the 'tool' role.

160
profiler/src/profiler/TracyLlm.cpp
View File

@@ -11,7 +11,6 @@
#include "TracyLlm.hpp"
#include "TracyPrint.hpp"
#include "data/ToolsJson.hpp"
#include "data/SystemPrompt.hpp"
extern tracy::Config s_config;
@@ -49,9 +48,6 @@ TracyLlm::TracyLlm()
m_systemPrompt = Unembed( SystemPrompt );
auto tools = Unembed( ToolsJson );
m_tools = nlohmann::json::parse( tools->data(), tools->data() + tools->size() );
ResetChat();
m_jobs.emplace_back( WorkItem {
@@ -162,8 +158,6 @@ void TracyLlm::Draw()
ImGui::EndCombo();
}
ImGui::Checkbox( "Enable tools", &m_enableTools );
ImGui::AlignTextToFramePadding();
ImGui::TextUnformatted( "Context size:" );
ImGui::SameLine();
@@ -207,14 +201,14 @@ void TracyLlm::Draw()
int idx = 0;
int num = 0;
bool first = true;
bool wasToolResponse = false;
for( auto& line : *m_chat )
{
const auto uw = ImGui::CalcTextSize( ICON_FA_USER ).x;
const auto rw = ImGui::CalcTextSize( ICON_FA_ROBOT ).x;
const auto ew = ImGui::CalcTextSize( ICON_FA_CIRCLE_EXCLAMATION ).x;
const auto ww = ImGui::CalcTextSize( ICON_FA_WRENCH ).x;
const auto yw = ImGui::CalcTextSize( ICON_FA_REPLY ).x;
const auto mw = std::max( { uw, rw, ew, ww, yw } );
const auto mw = std::max( { uw, rw, ew, yw } );
const auto posStart = ImGui::GetCursorPos().x;
const auto& role = line["role"].get_ref<const std::string&>();
@@ -224,18 +218,19 @@ void TracyLlm::Draw()
const auto isUser = role == "user";
const auto isError = role == "error";
const auto isAssistant = role == "assistant";
const auto isTools = isAssistant && line.contains( "tool_calls" );
const auto isToolResponse = role == "tool";
if( first )
{
first = false;
}
else
else if( !isToolResponse && !wasToolResponse )
{
ImGui::Spacing();
}
wasToolResponse = isToolResponse;
float diff, offset;
if( isUser )
{
@@ -253,14 +248,6 @@ void TracyLlm::Draw()
ImGui::SameLine( 0, 0 );
ImGui::TextColored( ImVec4( 1.f, 0.25f, 0.25f, 1.f ), ICON_FA_CIRCLE_EXCLAMATION );
}
else if( isTools )
{
diff = mw - ww;
offset = diff / 2;
ImGui::Dummy( ImVec2( offset, 0 ) );
ImGui::SameLine( 0, 0 );
ImGui::TextColored( style.Colors[ImGuiCol_TextDisabled], ICON_FA_WRENCH );
}
else if( isAssistant )
{
diff = mw - rw;
@@ -295,7 +282,7 @@ void TracyLlm::Draw()
{
ImGui::PushStyleColor( ImGuiCol_Text, ImVec4( 1.f, 0.25f, 0.25f, 1.f ) );
}
else if( isTools || isToolResponse )
else if( isToolResponse )
{
ImGui::PushStyleColor( ImGuiCol_Text, style.Colors[ImGuiCol_TextDisabled] );
}
@@ -308,27 +295,7 @@ void TracyLlm::Draw()
assert( false );
}
if( isTools )
{
ImGui::PushFont( m_font );
for( auto& tool : line["tool_calls"] )
{
auto& func = tool["function"];
ImGui::TextWrapped( "%s", func["name"].get_ref<const std::string&>().c_str() );
if( func.contains( "arguments" ) )
{
ImGui::PushFont( m_smallFont );
auto& args = func["arguments"];
for( auto& arg : args.items() )
{
ImGui::TextWrapped( "%s: %s", arg.key().c_str(), arg.value().dump( 2 ).c_str() );
}
ImGui::PopFont();
}
}
ImGui::PopFont();
}
else if( isToolResponse )
if( isToolResponse )
{
ImGui::PushID( idx );
if( ImGui::TreeNode( "Tool response..." ) )
@@ -388,6 +355,30 @@ void TracyLlm::Draw()
ImGui::PopID();
ImGui::PopStyleColor();
}
else if( line == "<tool>" )
{
ImGui::PushStyleColor( ImGuiCol_Text, style.Colors[ImGuiCol_TextDisabled] );
ImGui::PushID( idx );
if( ImGui::TreeNode( "Tool query..." ) )
{
ImGui::PushFont( m_font );
while( it != cache.lines.end() && *it != "</tool>" )
{
ImGui::TextWrapped( "%s", (*it).c_str() );
++it;
}
if( it != cache.lines.end() ) ++it;
ImGui::PopFont();
ImGui::TreePop();
}
else
{
while( it != cache.lines.end() && *it != "</tool>" ) ++it;
if( it != cache.lines.end() ) ++it;
}
ImGui::PopID();
ImGui::PopStyleColor();
}
else
{
PrintLine( ctx, line, num++ );
@@ -557,7 +548,6 @@ void TracyLlm::SendMessage( const ollama::messages& messages )
req["stream"] = true;
req["options"] = options["options"];
req["keep_alive"] = "5m";
if( m_enableTools ) req["tools"] = m_tools; // enabling tools prevents streaming in ollama 0.6.8
res = m_ollama->chat( req, [this]( const ollama::response& response ) -> bool { return OnResponse( response ); });
}
@@ -581,6 +571,20 @@ void TracyLlm::SendMessage( const ollama::messages& messages )
}
}
static std::vector<std::string> SplitLines( const std::string& str )
{
std::vector<std::string> lines;
auto pos = 0;
while( pos < str.size() )
{
auto next = str.find( '\n', pos );
if( next == std::string::npos ) next = str.size();
if( pos != next ) lines.emplace_back( str.substr( pos, next - pos ) );
pos = next + 1;
}
return lines;
}
bool TracyLlm::OnResponse( const ollama::response& response )
{
std::lock_guard lock( m_lock );
@@ -601,21 +605,36 @@ bool TracyLlm::OnResponse( const ollama::response& response )
auto& json = response.as_json();
auto& message = json["message"];
if( message.contains( "tool_calls" ) ) back["tool_calls"] = message["tool_calls"];
if( json["done"] )
{
if( back.contains( "tool_calls" ) )
bool isTool = false;
auto& str = back["content"].get_ref<const std::string&>();
auto pos = str.find( "<tool>\n" );
if( pos != std::string::npos )
{
HandleToolCalls( back["tool_calls"] );
auto end = str.find( "\n</tool>", pos );
if( end != std::string::npos )
{
auto data = str.substr( pos + 7, end - pos - 7 );
auto lines = SplitLines( data );
if( !lines.empty() )
{
isTool = true;
auto tool = lines[0];
lines.erase( lines.begin() );
const auto reply = HandleToolCalls( tool, lines );
m_chat->emplace_back( ollama::message( "tool", reply ) );
m_jobs.emplace_back( WorkItem {
.task = Task::SendMessage,
.callback = nullptr,
.chat = std::make_unique<ollama::messages>( *m_chat )
} );
m_cv.notify_all();
m_jobs.emplace_back( WorkItem {
.task = Task::SendMessage,
.callback = nullptr,
.chat = std::make_unique<ollama::messages>( *m_chat )
} );
m_cv.notify_all();
}
}
}
else
if( !isTool )
{
m_responding = false;
m_focusInput = true;
@@ -725,29 +744,15 @@ void TracyLlm::CleanContext( LineContext& ctx)
}
}
void TracyLlm::HandleToolCalls( const nlohmann::json& calls )
std::string TracyLlm::HandleToolCalls( const std::string& name, const std::vector<std::string>& args )
{
std::string response;
for( auto& call : calls )
if( name == "get_current_time" ) return GetCurrentTime();
if( name == "fetch_web_page" )
{
auto& func = call["function"];
auto& name = func["name"].get_ref<const std::string&>();
auto& args = func["arguments"];
std::string result = "### Result of calling function " + name + ":\n";
if( name == "get_current_time" ) result += GetCurrentTime();
else if( name == "fetch_web_page" ) result += FetchWebPage( args );
else result = "### Unknown function: " + name + "\n";
result += "\n\n";
response += result;
if( args.empty() ) return "Missing URL argument";
return FetchWebPage( args[0] );
}
m_chat->emplace_back( ollama::message( "tool", response ) );
return "Unknown tool call: " + name;
}
std::string TracyLlm::GetCurrentTime()
@@ -770,11 +775,8 @@ static size_t WriteFn( void* _data, size_t size, size_t num, void* ptr )
return sz;
}
std::string TracyLlm::FetchWebPage( const nlohmann::json& args )
std::string TracyLlm::FetchWebPage( const std::string& url )
{
if( !args.contains( "url" ) ) return "## Error: Missing URL argument";
std::string url = args["url"].get_ref<const std::string&>();
static bool initialized = false;
if( !initialized )
{
@@ -784,7 +786,7 @@ std::string TracyLlm::FetchWebPage( const nlohmann::json& args )
}
auto curl = curl_easy_init();
if( !curl ) return "## Error: Failed to initialize cURL";
if( !curl ) return "Error: Failed to initialize cURL";
std::string buf;
@@ -800,13 +802,11 @@ std::string TracyLlm::FetchWebPage( const nlohmann::json& args )
std::string response;
if( res != CURLE_OK )
{
response = "## Error: " + std::string( curl_easy_strerror( res ) );
response = "Error: " + std::string( curl_easy_strerror( res ) );
}
else
{
response = "## Web page '" + url + "' content:\n";
response += html2md::Convert( buf );
response += "\n";
response = html2md::Convert( buf );
}
curl_easy_cleanup( curl );

7
profiler/src/profiler/TracyLlm.hpp
View File

@@ -11,7 +11,6 @@
#include <vector>
#include <imgui.h>
#include <json.hpp>
#include "TracyEmbed.hpp"
#include "tracy_robin_hood.h"
@@ -92,10 +91,10 @@ private:
void PrintLine( LineContext& ctx, const std::string& str, int num );
void CleanContext( LineContext& ctx);
void HandleToolCalls( const nlohmann::json& calls );
std::string HandleToolCalls( const std::string& name, const std::vector<std::string>& args );
std::string GetCurrentTime();
std::string FetchWebPage( const nlohmann::json& args );
std::string FetchWebPage( const std::string& url );
std::unique_ptr<Ollama> m_ollama;
@@ -115,7 +114,6 @@ private:
bool m_stop = false;
bool m_wasUpdated = false;
bool m_focusInput = false;
bool m_enableTools = true;
char* m_input;
std::unique_ptr<ollama::messages> m_chat;
@@ -126,7 +124,6 @@ private:
ImFont* m_bigFont;
std::shared_ptr<EmbedData> m_systemPrompt;
nlohmann::json m_tools;
};
}