Helping Your Agent Be Konsistent
In Which the Author Looks For More Guardrails
As I've mentioned before, I try to think in terms of guardrails when it comes to coding agents. How can the project have built-in ways of helping the agent do the right thing, rather than just putting instructions in AGENTS.md and hoping for the best?
That's why this Kotlinlang Slack post by Stefan Oltmann resonated with me.
Konsist is so helpful in a world where AI just won't respect rules in the AGENTS.md 👍
You may not be familiar with Konsist. Quoting its documentation:
Konsist is a structural linter (static code analyzer) designed for Kotlin language. Verifying [a]codebase with Konsist enables development teams to enforce architectural rules and class structures through automated testing.
I have tended to think of Konsist as being useful for enforcing things like clean architecture. Stefan's post points out that you can use it for any sort of "this should not reference that" rule. The post's example is preventing any reference to java.io, presumably for a Kotlin Multiplatform project:
/**
* java.io.File is forbidden. We must use kotlinx-io.
*/
@Test
fun testNoJavaIoFileUsage() {
val violatingFiles = Konsist
.scopeFromProject()
.classes()
.filter { klass ->
klass.containingFile
.imports
.any { it.name.startsWith("java.io.") }
}
assertTrue(
violatingFiles.isEmpty(),
"Forbidden import detected: ${violatingFiles.map { it.name }}"
)
}
Because this is written in the form of a test, so long as your agent is running your tests, violations of this rule can be found at the point of the agent making the mistake. I have my coding skills set up for red-green TDD, so agents are always running tests. Personally, I would have the assertion error mention the invalid import and the desired fix (e.g., "Forbidden java.io import detected: ${violatingFiles.map { it.name }} [use kotlin-io for I/O]"), to help steer agents towards what it is that you are expecting. Regardless, the test failure alone should help agents out.
I have not yet applied this to a project, but it is on my short-term TODO list...
koverGate is up to 0.2.0. This release adds an aggregated report option: if you add the plugin to the root project, the root project's gap report will be the combined results of all modules in the project. That saves the agent from having to read a bunch of individual reports when running a full-project coverage run.
-
We hold off from adopting Konsist because it is not maintained. Aren't you afraid of adding such a project to yours?
Add a comment: