By March, about 42 percent of social clicks on job listings from QualityAssuranceJobs.com were landing on positions that had already closed. The old workflow had too many moving parts and too much lag between a job going live and it appearing on social feeds. For job seekers competing for roles in a tough market, this meant dead ends. For the team, it meant their distribution pipeline was working against them.
The Problem With the Old Setup
The original pipeline used a popular automation platform connecting an job board's RSS feed to a social media scheduling tool. Three points of delay stacked together.
The job board's RSS feed was never designed as a real-time interface. It published updates on its own schedule. The automation platform picked up those updates, adding lag. The final hop to the social scheduling tool added more.
The result was over an hour between a job posting going live and it reaching followers. In a competitive job market, an hour can mean the difference between an open position and a filled one.
Beyond speed, reliability was a problem. The team spent roughly 10 hours each week testing, retesting, and chasing missed posts. Listings kept slipping through, which explained the 42 percent error rate.
Why They Chose to Build Rather Than Patch
When you keep hitting the same wall with an off-the-shelf tool, that is a signal. You are not looking for a new integration. You need something you own.
The requirements were specific: poll for new content on a tight schedule, deduplicate against what was already posted, create a clean social payload, and send it to the API. The language choice mattered less than the principle: fast polling, simple deduplication, minimal payload.
They chose Clojure for the implementation because the language's approach to composable functions matched how they thought about the problem. Any tool that handles polling, deduplication, and API posting would work.
How the New Pipeline Works
The system, built in-house, runs on a straightforward cycle.
New job listings arrive as drafts in the scheduling platform. A person reviews each one and publishes it. That human gate is intentional: a person decides what deserves promotion.
From there, the custom system polls for newly posted jobs every 10 minutes, active from 8 a.m. To 10 p.m. Eastern. This proved to be the practical balance: active enough to catch same-day postings without over-engineering the infrastructure.
Before doing anything else, each new job is checked against a simple SQLite table containing job IDs and timestamps. If the system has already posted it, it skips. This deduplication step keeps the pipeline clean even when the same listing appears twice in the feed.
The social payload is deliberately minimal: job title, company name, location, and a link. Something like "QA Engineer (Remote) — Company Name — City, State." No fluff, just the information job seekers need to decide quickly.
Our Analysis: When to Build Your Own Pipeline
For teams distributing time-sensitive content, this case illustrates a useful principle. Off-the-shelf automation platforms handle a wide range of use cases effectively. But when latency is the actual product requirement, stacking third-party tools often creates compounding delays you cannot easily diagnose or fix.
The trade-off is real. Building your own solution means taking on monitoring, error handling, and updates. That is ownership, and it costs time upfront and continuously. The benefit is complete control over timing and the ability to debug every step of the pipeline.
Who wins by building? Teams where latency directly affects business outcomes and where patch attempts have proven futile. Who should stick with off-the-shelf tools? Anyone still in the experimentation phase, or where errors are tolerable.
The key signal is measurable: if your distribution pipeline is producing persistent errors at scale, patch attempts have diminishing returns. At some point, owning the workflow outright is the faster and more reliable path.
What to Watch Next
The team reports the 42 percent expired listing rate dropped significantly after the new pipeline launched. That is the metric that matters: not the sophistication of the tool, but whether job seekers land on open positions when they click.
Bluesky attachment handling remains an open item, a reminder that owning the pipeline means an ongoing backlog of improvements rather than waiting for a vendor update.
Key takeaways
- Stacking multiple third-party integrations compounds latency
- Human review of what gets promoted remains worth keeping
- Simple deduplication logic prevents most duplicate posts
- Building your own pipeline trades flexibility for complete control
- Measure the error rate, not just the infrastructure complexity
- Job board operators:** If your social distribution is producing measurable dead ends, patches have diminishing returns.
Frequently asked questions
What caused the high percentage of expired listings in the original workflow?
Multiple stacked integrations between the job board RSS feed, an automation platform, and a social scheduling tool created compounding delays, sometimes exceeding an hour between posting and social distribution.
Why build a custom pipeline instead of fixing the existing automation setup?
When an off-the-shelf solution produces measurable errors at scale and patch attempts continue failing, building a targeted solution you own becomes the more reliable path.
How does the new system handle duplicate job postings?
A SQLite table tracks job IDs and timestamps. Before posting, the system checks whether a job has already been published and skips duplicates.
Is Clojure necessary for building this kind of pipeline?
No. The language is secondary to the core logic: poll for new content, deduplicate, and post to an API. Any language that handles those functions works.
How often does the custom system poll for new job listings?
The system polls every 10 minutes between 8 a.m. And 10 p.m. Eastern, a window tested as the practical balance between speed and infrastructure overhead.