Bad ideas turned tool-belt favorites: ssh-ballpit
What can start as a "poke and a prompt" tour of a bad idea with an assistant can sometimes turn up smile-inducing results.
ssh-ballpit was one of those projects for me. ssh-ballpit is SSH proxy that asks other LLMs whether a command should run. I expected it to be a bad idea; instead, it became something I reach for regularly.
For work and nerdy curiosity, I am regularly spinning up, tearing down and otherwise managing computers, physical, virtual, all sorts. Fiddling with computers and talking about them is how I pay my bills.
I sort of relatedly have come to appreciate the utility of having what I call "3rd Party Assistants" (Claude, Codex) as part of my easily accessible tools, but while I'm amazed at what they can do, I don't trust them.
I have a ~2016 Intel MacBook running t2linux/fedora. It sits at the bottom of my office bread-rack, in a separate VLAN with VPN-Provider Internet Access and highly filtered L3 access to my "LAN", but historically never SSH access.
I just have not and do not feel comfortable letting the big providers rummage around my network.
At the same time, I want the power of their assistants relatively accessible, and most of the current paradigms of limiting assistant access are dense and fiddly or missing the point, though I really do like the look of projects like DCG and Agent SSH.
I also have a desktop with an ~8 year old AMD GPU that can just barely run Qwen 2.5 coder via Ollama w/ Vulkan. I wondered if I could get that self-hosted "Open" LLM to review and validate the "safety" of commands requested by my "3rd Party Agents" on my own hosts, as a way of asserting some kind of control in the contextual loop.
So I started poking with Codex at an SSH proxy "agent" that asked other LLMs to review and approve the commands sent to it (presumably by another agent), and it worked better than expected.
Qwen 2.5 could catch a surprisingly useful number of scenarios I tested. When I tested it, Qwen on my old and dusty GPU was able to provide good value limiting my agents away from commands they shouldn't be using. Even its limited size became a benefit, if Claude asked to do something too complex for it to understand: Qwen's barfing would restrict the command from being run, and Claude would retry the same intent with safer commands.
While Qwen 2.5 on my own hardware caught a lot, it was also clear it wasn't enough, so I added a couple of other providers, and started asking a fanout of them for approvals. This returned great results. Gemini instructed to be watchful is a great catch for an ambitious Claude, where I can trust Gemini because my self-hosted Qwen said the same thing.
A couple of extra feature adds like approval caching, whitelisting, and suddenly this is a tool I'm using frequently to help me debug situations I'd normally be on my own for.
Using lightweight but capable models from the major providers, I can generally get by with the "Free Tiers" access alone.
Fun.
After this much talking about Agents and nonesense, here is a palette cleansing picture of a flower from the New York Botanical Garden:
