The Problem

Recruiters write queries the way they think. "Full stack engineer, 2-4 years, Bangalore, SaaS background, has shipped to production." Boolean search engines force them to translate that into AND/OR keyword soup, then guess which permutations a resume might use. Saral skips that translation. The recruiter writes the brief. The platform parses intent, embeds it, searches semantically, and ranks against the structured filters that actually matter.

My Role

I owned platform architecture, infrastructure, the backend server, and the entire search data flow - from query parsing to result storage. The frontend was built by a separate engineer; my contract ended at the API. That meant designing the service topology, picking the database stack, building the Fastify gateway, integrating the FastAPI search engine, wiring the LLM fallback chain, and shipping it on AWS without setting fire to the budget.

Architecture

Server
Fastify on Node.js 22 with TypeScript and Drizzle ORM. The API gateway owns auth, request validation, result storage, pagination, the credit ledger, and orchestration of background work like backfill and contact unlocks. Engine
FastAPI on Python 3.13 with SQLAlchemy. Owns the search pipeline end to end - query parsing into structured labels, hybrid vector and metadata retrieval, scoring, and post-scoring semantic evaluation by an LLM. Embedding Sidecar
A local BGE-M3 service generating 1024-dimensional vectors. Running the model in-process gives deterministic latency and removes the per-call cost of a hosted embedding API. The trade-off is a five-minute cold start, which is acceptable for a long-lived ECS task. Data Layer
PostgreSQL 16 with pgvector. Auth, searches, results, and credits live in one schema; the candidate corpus and embeddings in another. Logical separation today, physical separation tomorrow without a refactor. Infrastructure
AWS in ap-south-1. ECS Fargate Spot for compute. Application Load Balancer with host-based routing across app.*, api.*, and engine.*. Total monthly compute cost lives in the low double digits.

The Search Pipeline

Every search runs the same sequence. The recruiter's brief goes to Groq for structured label extraction, with Azure OpenAI as a fallback. The brief and its expansions are embedded by the sidecar. PostgreSQL plus pgvector runs cosine similarity gated by metadata filters from the parsed labels. Hard scoring on role match, experience fit, skills, and location runs deterministically. The top thirty are sent back to the LLM in batches for semantic evaluation of fuzzy requirements - "real SaaS experience" - returning yes, no, or maybe. A composite score collapses the signals. Rank is finalized server-side and is the contract that pagination depends on.

Notable Engineering Decisions

Semantic evaluation is post-scoring, not pre-filter.
A missing fuzzy signal cannot disqualify a candidate who scored well on hard facts. If the LLM is down, the result still ships with a semanticWarning flag. Degraded, not broken. Rank is canonical, set by Engine.
The frontend cannot re-sort. Pagination, tie-breaking, and "next page" all assume the original order. This sounds obvious; in practice it is the contract you regret not enforcing later. Two LLMs in a fallback chain.
Groq is the primary because it is fast and cheap. Azure OpenAI is the fallback because it has the SLA. The chain is configured per call, not per environment. Progressive backfill, not batch.
When the corpus returns too few results, the server enriches an external talent database in mini-batches of three to five candidates. The recruiter sees results immediately and watches the page fill in every five seconds. Earlier versions tried to do everything at once and froze the UI for two minutes. Per-page credits with idempotent charges.
Each page reveal is one credit charge, idempotent against the candidates already seen by the same account. Charging on reveal aligns the bill with the value the recruiter actually consumed. The credit ledger is append-only; failures issue a positive rolled_back entry rather than mutating the original deduction.

Challenges Solved

Coverage
Three discovery tiers run in parallel for backfill. A precise first pass, a broadened second pass that drops experience and location, and a SERP fallback. All deduped against a knownUsernames set built before any external call, with (searchId, candidateId) uniqueness as the second line of defense. Reliability
The vendor's discovery endpoint times out non-deterministically under load. A retry wrapper bounded to a single retry on 5xx with a five-second delay raised the hit rate above ninety-five percent. Rate-limit responses are treated as a signal, not a hiccup; concurrency drops dynamically and exponential backoff respects Retry-After. Cost
ECS Fargate Spot, a local embedding sidecar, and per-page billing keep the platform's expensive pieces in line with usage. The LLM is the cost line; the platform itself is not.

Impact

119k+ candidates indexed with embeddings.
Three services plus an embedding sidecar in production.
Sub-double-digit monthly platform compute cost.
Zero silent failures in the search pipeline - degraded results always carry a flag.

Technologies

Node.js 22 · Fastify · TypeScript · Drizzle ORM · PostgreSQL 16 · pgvector · Better Auth · pg-boss · Python 3.13 · FastAPI · SQLAlchemy · BGE-M3 · Groq · Azure OpenAI · Resend · AWS ECS Fargate · Application Load Balancer · Docker · Next.js 16