Full-stack SaaS · Extractly
Turning a folder of CVs into a structured spreadsheet, automatically
Every hiring drive ends the same way. A recruiter has 80 CVs. Half are PDFs. A dozen are scanned images. A few are Word documents. They open each one manually, copy a name here, an email there, paste it into a spreadsheet. Two hours later they have a half-complete table and a headache. Extractly automates that entirely.
You create a job, pick which fields to extract, upload the files, and get back a clean spreadsheet. PDFs, DOCX files, even photographed CVs — it handles all of them. This is how I built it.
The upload pipeline — files never touch my server
The first design decision that forced me to think carefully: where do the files go? The obvious path is to send them to Django, save them to disk, then push them to storage. But that puts the API server in the critical path of every upload. With 100 files per job, that's a lot of unnecessary load.
Instead, every file goes directly from the browser to Cloudinary. Django's only role in the upload is generating a cryptographically signed set of upload parameters: a timestamp, a signature, and a specific folder path with the job and tenant ID baked in. The browser uses those params to POST the file straight to Cloudinary. Django never sees the bytes.
The webhook endpoint is public (no JWT required), but every request is signature-verified. Anyone can hit the URL; only Cloudinary can produce a valid signature. It also has to be idempotent: Cloudinary retries on non-200 responses, so receiving the same webhook twice for the same file has no effect.
Duplicate CV detection — three independent layers
When a user re-downloads a file that already exists on their machine, the OS renames it automatically:
arsh.pdf becomes arsh (1).pdf, then arsh(2).pdf. Because the strings
differ, a naive exact-match check treats all three as separate files — wasting LLM credits and polluting the
sheet with identical rows.
Blocking duplicates reliably required three independent layers. Each one protects against a different failure mode: a user mistake, a bypassed frontend, or a concurrent race condition.
uploadStore.ts normalizes every filename before adding it to the queue, stripping OS copy
suffixes like (1) or (2) so arsh (1).pdf and arsh.pdf
resolve to the same key. The duplicate is dropped silently with a toast warning. Zero API calls
wasted.
POST /jobs/{id}/upload/sign/ normalizes the incoming filename and compares it against every
file already registered for that job. A 409 CONFLICT is returned before a Cloudinary signature
is issued. The file is never uploaded and no storage credit is charged.
POST /jobs/{id}/files/ runs two deduplication passes: first within the incoming payload
itself (intra-batch), then against every file already in the database for that job. A
SELECT FOR UPDATE lock prevents two concurrent registration calls from both inserting the same
file. Only genuinely new files reach bulk_create.
# Backend normalization — strips OS copy suffixes before any comparison
def normalize_filename(filename: str) -> str:
stem, ext = os.path.splitext(filename)
return re.sub(r'\s*\(\d+\)$', '', stem) + ext
# In UploadSignView.post():
normalized_name = normalize_filename(filename)
existing_names = {
normalize_filename(n)
for n in File.objects.filter(job=job).values_list('original_filename', flat=True)
}
if normalized_name in existing_names:
raise ConflictError(f"A file equivalent to '{filename}' already exists in this job.")
If an entire batch registration call contains only duplicates, it succeeds silently, and the job moves to
QUEUED using the existing file count. Idempotency at every layer means retries are safe and the
user never sees a confusing error for something that isn't actually wrong.
The extraction engine — Celery Chord
When the user triggers extraction, I don't dispatch one task per file. I dispatch a Celery Chord. The chord splits all verified files into batches of ten, runs those batches concurrently, and fires a single callback once every batch has resolved, regardless of success or failure.
_dispatch_chord()
│
├── process_batch(files[0:10]) ─┐
├── process_batch(files[10:20]) ├── run in parallel
├── process_batch(files[20:30]) │ across Celery workers
│ ... ─┘
│
└── on_chord_complete() ← fires exactly once, after all batches finish
counts done_files / failed_files
COMPLETE | PARTIAL | FAILED
The key property: on_chord_complete() fires exactly once, after all batches are done. It
counts the results and transitions the job to its final status. If 85 files succeeded and 2 failed, the job is
PARTIAL. The sheet unlocks, the successful rows are available, and the failed rows appear as null
cells. The whole job doesn't fail because of two bad files.
The frontend polls GET /jobs/{id}/status/ every few seconds during processing. The response is
intentionally tiny: just status, done_files, and failed_files. No rows,
no snapshot data. Cheap to call, cheap to serve.
Inside each batch — the OCR + LLM pipeline
Every file goes through the same pipeline inside process_batch():
File type detection PDF → pdfplumber → raw_text DOCX → python-docx → raw_text Image → OCR.space Engine 2 (remote URL as input) → raw_text raw_text saved to database ← V2 re-extraction hook build_prompt(fields_snapshot) → injects field names, types, and custom hints → instructs LLM to return null for missing fields groq_extract() → llama-3.3-70b-versatile → JSON mode, temp=0, 60s timeout → tenacity: 4 retries, 1–20s backoff Pydantic v2 validates the response pass → ExtractedRow.save() File.status = DONE fail → ExtractedRow(null) File.status = FAILED
The fields_snapshot, which is the list of fields HR defined at job creation, is frozen at that
moment and never changes. It drives the extraction prompt, the column order in the sheet view, and every export
for that job's entire lifetime. This was a deliberate product decision: if the field config could change after
some files had already been extracted, the resulting sheet would be inconsistent. Immutability solves that
entirely.
Pydantic v2 acts as an unbreakable contract between the LLM and the database. The LLM is asked to return a JSON object matching the fields_snapshot schema. If it hallucinates a field name, adds extra keys, or returns the wrong type, Pydantic catches it and tenacity retries the call. The data that reaches the database is always structurally valid.
Bugs that taught me things
The DRF query parameter that 404s before your view runs
After the entire extraction pipeline was working end-to-end, the export endpoint kept returning
404 Not Found. The rows endpoint on the same job returned 200. The URL was registered —
resolve() confirmed it. The view code looked correct.
The culprit was DRF's content negotiation. DRF reserves ?format= as a system parameter. When a
request arrives, DRF calls perform_content_negotiation() inside APIView.dispatch() —
before authentication, before permissions, before your view code runs. It looks for a renderer matching the
requested format. I only had JSONRenderer configured, so ?format=xlsx produced a 404
before I ever got control.
The fix: rename the parameter from format to export_format everywhere, including
the view, frontend, and integration tests.
format as a query parameter name in a DRF
view. The debugging approach that found it: test the endpoint without the query parameter first. If you get a
400 (view reached, validation failed) but a 404 with the parameter, something before your view is intercepting
the request.The webhook that arrived before the database transaction committed
Cloudinary fires webhooks almost immediately after an upload completes. In testing, some webhook requests
were arriving while Django's database transaction was still committing, meaning the webhook handler tried to
look up a File record by its Cloudinary public ID, found nothing, and silently dropped the
verification.
The fix was wrapping the post-registration work in Django's transaction.on_commit():
def register_files(job, file_data_list):
File.objects.bulk_create([...])
job.status = JobStatus.QUEUED
job.save()
# Only notify AFTER the transaction is fully committed
transaction.on_commit(lambda: notify_upload_complete(job.id))
This guarantees that any subsequent webhook lookup always finds a committed record. The idempotency guard on the webhook handler covers the rare case of duplicate delivery.
transaction.on_commit(). The transaction is
not visible to other processes until that callback fires.The duplicate that wasn't — OS copy suffixes and a silent typo
Duplicate CVs were silently passing through the original upload pipeline. When a user re-downloads a file on
Windows or macOS, the OS renames it: arsh.pdf becomes arsh (1).pdf. Because the
strings differ, the original exact-match check treated them as different files. Consequently, the same CV was
processed twice, two LLM calls were charged, and two identical rows appeared in the sheet.
Building the three-layer duplicate detection system (frontend store normalization, sign endpoint rejection, batch registration deduplication) fixed the core problem. But during that work, a second bug surfaced. In the all-duplicates path (where the entire batch registration payload turns out to be duplicates and no new files are inserted), the job counter update had a typo:
# Before — silent 500 waiting to happen in production job.save(update_fields=["total_fields", "status"]) # After — correct field name job.save(update_fields=["total_files", "status"])
One character difference between total_fields and total_files. In the happy path
(at least one new file), this code is never reached. Only the all-duplicate branch hits it. It would have
raised a django.core.exceptions.FieldError 500 in production the first time a user tried to
re-register an already-uploaded batch.
What's built vs. what's next
Phase 1 is shipped and live. The full pipeline, including upload, webhook verification, async extraction, OCR, LLM, export, merge, and Google Sheets integration, all works in production.
The hooks for Phase 2 are already in the codebase. Every file's raw_text is stored in the
database. That means delta re-extraction (re-running the LLM on cached text without re-uploading) is an
activation away, not a rebuild. WebSocket job progress is the obvious next UX improvement over the current
polling approach.