How to handle a large amount of tools ?

[Adesin-fr]

I'm understanding that giving an Agent a lot of tools will clutter the Context Window of the used model.
However, I'm implementing an agent in a large ERP, where many tasks can be done, and all of them are tied to each other...
Something like, Customer has items, items can be repaired. Invoices and quotes belongs to a customer and an item. work orders targets a customer/item...
My Agent can interact with all of these domains, and a user should be able to ask "create a work order to repair Bob's item".
The agent will then need to call multiple tools to get all the pieces of his puzzle before being able to finally place a work order with the correct tasks, on bob's item.
That's a short overview, but many other domains are concerned by this ERP.

In the end, I might end up with more that 100 tools, and perhaps some tools will have many optionnal parameters to allow to search from many criterias...

I'm concerned about filling too much of the context window with all those tools definitions...

How would you handle this ?
It would be ideal if the agent could have only an abstract of all the tools, or even toolkits it can use, then the agent could ask for details about tools contained in the toolkit or how to use them ?

Another option, which I don't like and want to avoid, would be to create specialized agents, so the end user would have to choose which agent he wants to speak to first (The repair Team, the Sales team, etc...)... That's not ideal, since the repair team would have to talk with other agents to acomplish his tasks...

I was wondering if putting text versions of documents (Invoices, work orders, etc...) into a RAG would allow to avoid some searching tools... Instead of using a tool to search for previous invoices and work orders, the RAG would find them by similarity ??

Some more thoughts :
It might be interresting to create some sort of "lazy tools discovery" : We could add a short description to toolkits, then only submit those short descriptions to the LLM "the invoicing toolkit allows to..., the work order toolkits allows to...", and a "toolkit discover" tool, that allows to get the definition of tools contained in a toolkit. Once the LLM wants to explore one of those, it uses the "toolkit discover" and get the schema of every tools defined in a toolkit.
That seems a good idea to test to me, but the toolkit discover must then return the correct syntax of tools... Not sure how to do this part !

[ilvalerione]

It's not an easy topic. People are experimenting storing tools descriptions into a vector store and perform a semantic search with the user query to retrieve the subset of tools the agent should need to provide an answer. I think the FileVectorStore is perfectly suited for this use case, but I don't really know the effectiveness of this strategy.

Another things that comes in my mind is adding a preliminary step before the user input comes into the agent (maybe creating a workflow) to classify the user input compared to the list of tools using an LLM intelligence instead of a semantic search. I usually introduce this classification steps using the structured output to gather the exact values I need and a less expensive, and fast model. In your case you could imagine a list of topics, and create arrays of tools for each of these topics. The structured output can be built with an ENUM to let the agent gives you the exact topic for the user input, and then attach the right array of tools to the next agent to generate the final answer.

I don't know if they are real solutions, just trying to help you in the thinking process.

[Adesin-fr]

Thanks for you answer.
However, I don't understand how to declare tools from the "code side" (ie the Agent class) without adding them in the Tools array ?
If I only declare them in a vector store, when the agent will need to call a tool, since I won't declare them in the Agent available tools, will Neuron be able to invoke them ?

Can I make a unique "Query tool for role" tool that I expose to the agent, and then this tool will give the agent a "json representation" of all the tools it can use for a specific role. That's what I imagined, but then how Neuron will know how to invoke a tool an agent will be given afterword, but not declared in the code ? Should the JSON contain the full classname of the tool ?

I haven't seen the output of the content that is sent to the LLM and don't know yet how provided tools are serialized, and then when the llm wants to call a tool, how this happens.
Perhaps digging in the code to understand this part would clarify my problem ;)

Can you recommend any entry points I should look at to understand more clearly how the tools interactions between LLM and app works ?

[ilvalerione]

No, you can't provide the tools to the LLM as a tool response. Tools need to be attached to the HTTP request payload.

Tools are attached to the provider before the request is sent to the model: https://github.com/neuron-core/neuron-ai/blob/2.x/src/HandleChat.php#L41

So it could be easy to find a way to segment the tools needed for the current request outside of the agent. Before calling the chat method.