When you launch something small, the first users are the hardest to get. The people who'd want it are already on Hacker News and Reddit, asking things like "is there a tool that does X?" But I couldn't sit and read every thread every day to find them, and I didn't want to be the account that drops the same link everywhere.
So I started building an agent to do the searching for me.
What it does
You tell it about your product once: what it does, what problems it solves, and the words your customers use. It then searches Hacker News for threads where someone is stuck on one of those problems, picks the ones worth answering, and writes a reply for each.
The replies are meant to help first. Most of them don't mention the product at all. It only comes up when the person is asking for something the product actually does.
It never posts anything. The drafts go into a sheet, I read them, fix what needs fixing, and post the good ones myself. Anything with my name on it goes through me.
A few practical details:
- It runs on my laptop. The AI is Llama 3.2 (the 3B version), a small model, because my graphics card has 4 GB of memory and anything bigger crashed it. No paid APIs.
- Reddit blocks almost every request from scripts like this, so right now it only works on Hacker News. The Reddit part is built and waiting.
- Nothing is tied to one product. I've been testing it on RepoClimb, but you can point it at anything.
How a run works
Blue boxes are the agent, orange boxes are me.
- Search. It looks on Hacker News for phrases built from the product's problems, plus general ones like "what do you use for".
- Clear out the junk. Spam, job posts and launch posts go. Launches are the big one: around 62% of what comes back is "Show HN: I built X", and I don't want to reply to someone's launch with my own product. Threads nobody has touched in days go too.
- Read the thread. The model reads what's left and tells me what the person is actually asking for, and whether they're promoting something of their own.
- Decide. The model doesn't decide anything. Plain code takes what it read and decides whether to reply and whether the product can come up. Small models sound sure of themselves and are often wrong, so I don't let them make the call.
- Write. A draft of 60 to 110 words, with the answer first.
- Check. A second pass checks the draft. Is it salesy? Too short? Off topic? It also checks that every tool the draft names actually exists on GitHub, because small models make tools up. If a draft fails, the writer is told what was wrong and tries again, up to three times.
- Save. Drafts that pass go into the sheet, with a note on why they passed.
- I post it, or I don't. After I post, I log it so the agent never drafts a second reply to the same thread.
- Count replies. A small command checks my comment on Hacker News and counts the replies. That's the number I actually care about.
Rules I added after things went wrong
Most of my time went into teaching it when not to reply. These rules came out of that:
- It never posts on its own.
- At most 4 in 10 drafts can include the link, and only when the question really fits.
- No replies under launch posts.
- Any tool it names has to exist on GitHub, or that sentence gets cut.
- If the AI isn't running, the whole run stops. I added this after a silent fallback produced twelve useless drafts that all scored 0.98 and looked fine.
A real run on RepoClimb
RepoClimb tracks open-source projects and shows which ones are picking up momentum. The people who'd want it ask things like "is this library still maintained?" or "how do I find good new repos?" Here's a run from 11 November.
On the dashboard, you set how strict the checker is and how many threads to look at, then hit Run Live Scrape. The log on the right shows what it's doing. You can see it skip Reddit and go to Hacker News.
It loads search phrases based on RepoClimb's problems: "still maintained", "abandoned", "star history", "hidden gem", "bus factor". That list is the part you'd swap out for another product. It dropped 30 launch posts straight away and checked 12 threads to see if people were still talking in them.
Each thread gets a score for how likely the person is to want help. One got skipped because nobody had asked anything, so there was nothing to answer.
Two drafts passed. Each has an Edit and a Copy button, so I can tidy it up and paste it into the thread myself.
Neither one mentions RepoClimb. Both are tagged "Pure Goodwill", which is what I want here: nobody in those threads was asking about open-source projects, so the product didn't belong. These replies just help, and they build up the account a little.
They're fine but generic, though. The first one lists tools without saying much about them. I'd rewrite parts of both before posting. The whole run took about two minutes.
I ran it again a few minutes later and it found a better thread: someone asking which OCR tools people use with LLMs. The draft named real tools (Tesseract, Pytesseract, EasyOCR) and scored 0.93 for tone on the first try, the best of the day.
It still had a mistake. It said Tesseract comes "from Google's TensorFlow project", which isn't true. The checker confirmed the tools exist, but it can't tell when a real name comes with a wrong fact. I caught it after about ten seconds of reading. I'd fix that line and post the rest.
What I've learned so far
There isn't much to find. On earlier RepoClimb runs, a typical run found about 494 threads. 394 got past the junk filter, 18 were worth showing to the model, about 6 drafts passed every check, and about 1 was something I'd actually post. Some days it's zero. In RepoClimb's niche, Hacker News gets roughly one real question a week. A product in a busier space would see more.
A high score doesn't mean a good reply. I had it write four replies to the same thread. One was good. The other three suggested exactly what the person had said wasn't working for them. All four scored between 0.88 and 0.96. The checker can tell me a draft is safe, but only reading it tells me if it's good. Every real problem I've fixed so far, I found by reading drafts myself.
It misses good threads. Someone asked "What's the smallest agent harness that is still architecturally complete?", which is right in RepoClimb's area. The agent was searching for "agent framework", so it never saw it. It also skips posts that describe a problem without asking a question, and some of those people do want help. Looser search terms pull in too much unrelated stuff, so I'm still working on this one.
Drafts expire. One good draft sat in the sheet and I never posted it. After ten days the thread was too old. Three weeks later, that question still had no answers. That one's on me.
What I'm working on next
The next step is posting, not more code. I've spent weeks tuning drafts and I still don't know how a real person reacts to one. So I'm going to post the best ones by hand and see if anyone replies.
If nobody responds to good answers, this isn't the right channel, and more engineering won't change that. If people do respond, I'll put the work into finding more threads, starting with the ones it misses today.
I'll write an update once I have real replies to look at.