System: Headless Outreach & Messaging Daemon
Category: Automation · Browser Engineering
Reading Time: 6 min read
Running persistent headless browser sessions on Linux without getting blocked
Overcoming QR re-authentication, anti-bot mechanisms, and dynamic DOM shifts.
Worked through: Selenium · Headless Chrome · Linux systemd · Session Persistence · Rate Limiting
01. WHY I BUILT IT
The Trigger & Underlying Problem
The Trigger: Delivering urgent hearing notices required sending messages across web portals with active human confirmation.
The Core Problem: Copy-pasting notice links and case details manually took 3+ hours daily and resulted in human typos in case numbers.
Why Standard Solutions Were Insufficient: Official enterprise APIs were prohibitively expensive for low-volume operational notices.
02. PRACTICAL IMPACT
What It Solves in Practice
PRACTICAL PROBLEM
Dispatches templated, verified hearing notices automatically on a headless Linux host while tracking delivery status.
BENEFICIARIES
Case managers communicating with parties who do not check postal mail.
BEFORE THIS SYSTEM
Staff manually copied numbers from spreadsheets and typed messages into browser tabs one by one.
03. IMPLEMENTATION
Architecture & Key Engineering Decisions
A Python automation daemon running on a Linux host under `systemd`. Operates headless Chromium attached to a persistent user-data directory on disk.
Key Decisions:
- Mounted a persistent Chrome user-data profile on disk so session cookies and local storage tokens survive server reboots, bypassing recurring QR code scans.
- Simulated natural typing rhythms using Selenium ActionChains with randomized Gaussian pauses between keystrokes.
- Logged message hashes into SQLite to ensure idempotency across automated runs.
04. CHALLENGES & SOLUTIONS
What Failed in Practice & How It Was Addressed
WHAT WENT WRONG
- Pasting text instantaneously into input fields triggered anti-spam bot detection and temporary account blocks.
- React DOM class re-hashes broke static XPath queries after portal frontend updates.
- Chromium zombie processes accumulated in memory when unhandled exceptions halted the script.
HOW I SOLVED IT
- Implemented humanized keystroke timing (`time.sleep(random.uniform(0.05, 0.15))`) and simulated mouse movements.
- Switched from brittle absolute XPaths to semantic text locators and explicit condition waits (`WebDriverWait` with `presence_of_element_located`).
- Wrapped execution in a strict `try...finally: driver.quit()` lifecycle block and registered clean SIGTERM traps in the `systemd` service unit.
05. REFLECTION & LESSONS
Takeaways & Future Iterations
What I Learned:
- Browser automation is never set-and-forget; you must monitor DOM evolution and implement resilient error recovery.
- Headless processes on Linux require careful memory and process supervision.
WHAT I WOULD CHANGE NOW:
Today, I would migrate the daemon from Selenium to Playwright for native async support, lighter resource consumption, and superior auto-waiting selectors.