In a past "Thursdays with Koog" issue, we saw that Koog's AIAgent knows how to handle tool calls for us. We can provide a prompt and get the final response from the LLM, and any tool calls the LLM requests in between those are handled by the agent.

In truth, the AIAgent is involved in that logic, but the real work is handled by a strategy implementation.

Koog has an AIAgentStrategy interface with a bunch of sub-interfaces and implementations. In addition to a name, AIAgentStrategy defines execute(), which AIAgent itself wraps via its run() function, with a bit of additional indirection. execute() does the work. So, in the end, the strategy is what orchestrates the work with the LLM, and since the strategy is defined by an interface, we can plug in other strategies as needed.

The default strategy is built by the singleRunStrategy() factory function. Its KDoc shows the flow that Knosh uses:

1. Start the agent.
2. Call the LLM with the input.
3. Execute a tool based on the LLM's response.
4. Send the tool result back to the LLM.
5. Repeat until LLM indicates no further tool calls are needed or the agent finishes.

Basically, singleRunStrategy() is your "one-shot" pattern.

Under the covers, singleRunStrategy() is a one-off implementation of a graph strategy:

@JvmOverloads
public fun singleRunStrategy(parallelTools: Boolean = false): AIAgentGraphStrategy<String, String> = strategy<String, String>("single_run") {
    val nodeCallLLM by nodeLLMRequest()
    val nodeExecuteTool by nodeExecuteTools()
    val nodeSendToolResult by nodeLLMSendToolResults()

    edge(nodeStart forwardTo nodeCallLLM)
    edge(nodeCallLLM forwardTo nodeExecuteTool onToolCalls { true })
    edge(nodeCallLLM forwardTo nodeFinish onTextMessage { true })
    edge(nodeExecuteTool forwardTo nodeSendToolResult)
    edge(nodeSendToolResult forwardTo nodeFinish onTextMessage { true })
    edge(nodeSendToolResult forwardTo nodeExecuteTool onToolCalls { true })
}

(the above is from Koog 1.1.1)

This defines an object graph, with edge() connecting two processing nodes. The interaction begins at nodeStart and flows through the graph, eventually terminating at nodeFinish. The strategy() builder offers a DSL for defining the graph, designed to make the code read almost like plain English:

• When we start, forward the context to the LLM
• When we get a response from the LLM, if it is a tool call, execute the tool
• When we get a response from the LLM, if it is a text message, finish the strategy and return that message
• When a tool returns a result, forward it to the LLM
• If the response from sending the tool result is another tool call, execute the tool
• If the response from sending the tool result is a text message, finish the strategy and return that message

Strategies can get a lot more elaborate:

• Graphs can have subgraphs

• Graphs can configure history compression, so long prompts (especially with "thinking" messages) get compacted

• Graphs can use conditional branching based on custom business rules... such as triggering history compression when the prompt gets too big

• Nodes are strongly typed, which means nodes can transform data as needed

• Since nodes are simply interfaces, nodes can call out to application logic as needed, returning results

• And so on

The bad news is that nodes are synchronous, insofar as they implement a simple execute() function (often defined by a DSL-supplied lambda expression). execute() is a suspend fun, so you can do I/O and stuff, but in the end, execute() needs to return whatever the graph needs to continue.

A side effect of this approach is that creating a multi-turn agent, with several rounds of user input, gets weird. Effectively, you build a graph where you have a node that takes the latest message from the LLM and synchronously returns the next input from the user. That node can leverage coroutines, such as posting to a Flow or Channel and observing a Flow or Channel to get the response, but it needs to be able to return the user's message, which your graph can then process. Your UI needs to be conducive to such an arrangement, and that might take a while to "get your head wrapped around".

You don't have to use custom graphs. Knosh doesn't. You can create a multi-turn agent just by iteratively calling run() on your AIAgent set up to use singleRunStrategy(). Managing the overall prompt becomes your job: for your second turn, you need to determine how to combine your original prompt with the result from the LLM and (presumably) fresh input from the user, but without all of the tool responses (which were "consumed" in the process of giving you the LLM result). Personally, and off the cuff, I would not mind this and might prefer it to trying to twist my overall app to be visible just via a graph node. But, you have options.

Next week's "Thursdays with Koog" will explore some other Koog "knobs we can turn and switches we can flip", such as the maximum number of iterations to use, the temperature, and more.