← Back

MVP Phase 1 & 2: Job Quality Engine, Audit Logging, Dashboard & Error Handling

audit celery dashboard django error-handling job-quality mvp tailwind

MVP Phase 1 & 2: Job Quality Engine, Audit Logging, Dashboard & Error Handling

SkillTrace kini memiliki Job Quality Intelligence Engine yang menganalisis deskripsi pekerjaan sebelum screening dimulai, audit trail untuk setiap tindakan recruiter, dashboard dengan statistik agregat, dan error handling yang memungkinkan retry pada screening yang gagal. Dua fase MVP selesai dalam satu sesi: Phase 1 (apply recommendations, dismiss warnings, auto-purge) dan Phase 2 (audit logging, dashboard stats, error handling).

Work Log

  • [00:02:00] Baca PRD §43 MVP Definition of Done — identifikasi 4 item P0 yang belum terpenuhi
  • [00:05:00] Rancang Job Quality Engine: 7 analyzer (overload, seniority, ratio, contradiction, missing area, domain grouping, recommendations)
  • [00:15:00] Implement job_quality/ module: models.py, analyzers.py, scorer.py, engine.py — 32 test cases
  • [00:25:00] Tambah analyze-job CLI subcommand ke cli.py
  • [00:30:00] Django integration: Job model 7 quality fields + migration 0002_job_quality_fields
  • [00:38:00] Tailwind CSS CDN ke base.html, restyle 8 templates dengan utility classes
  • [00:48:00] Quality report UI: color-coded score, seniority, domains, warnings, recommendations
  • [00:55:00] Fix quality_seniority template field refs (.label -> .detected_level)
  • [01:00:00] Phase 1 — Apply recommendations: merge must_have/preferred/nice_to_have ke requirements
  • [01:08:00] Phase 1 — Dismiss warnings: warning_overrides JSONField, dismiss/show views
  • [01:15:00] Phase 1 — Auto-purge: Celery task cleanup_old_candidates + Beat schedule harian 03:00
  • [01:25:00] Phase 1 tests: 11 web tests baru — 213 passed
  • [01:30:00] Phase 2 — AuditLog model: 9 action choices, log_action() helper
  • [01:40:00] Wire audit logging ke semua job views (create, edit, apply_recommendations, dismiss_warning, upload)
  • [01:48:00] Wire audit logging ke candidate views (delete, status_change, shortlist)
  • [01:55:00] Audit log tab di job detail + read-only Django admin
  • [02:00:00] Phase 2 — candidate_retry view: reset failed -> queued, re-queue Celery task
  • [02:08:00] Phase 2 — process_cv partial failure handling + structured error codes
  • [02:15:00] Phase 2 — Dashboard: 5 stats cards (total jobs, candidates, queue, avg quality, pass rate)
  • [02:22:00] Phase 2 — Recent activity section dari audit log
  • [02:30:00] Phase 2 tests: 14 web tests baru — 227 passed
  • [02:35:00] PROGRESS.md update, commit + push

Context

  • Proyek: SkillTrace — mesin screening CV berbasis evidence (Django monolith + Celery + PostgreSQL + Redis)
  • MVP Definition of Done (PRD §43): 23 item yang harus terpenuhi sebelum SkillTrace bisa dipakai recruiter
  • Yang sudah jalan: Screening pipeline (upload CV -> extract -> match -> explain), job CRUD, candidate management, HTMX real-time progress, DRF API
  • Yang belum ada: Job quality analysis, ability to act on recommendations/warnings, audit trail, dashboard stats, error recovery, data retention enforcement

Problem

PRD §43 mendefinisikan 23 item MVP DoD. Setelah Phase 1 engine selesai, masih ada 4 item P0 yang belum terpenuhi:

  1. §43.7 — Apply recommendations: Recommendations ditampilkan tapi tidak bisa di-apply ke requirements
  2. §43.8 — Override warnings: Recruiter tidak bisa dismiss warning yang sudah ditinjau
  3. §43.22 — Auto-purge: CANDIDATE_RETENTION_DAYS ada di settings tapi tidak ada scheduled task
  4. §43.23 — Audit logs: Tidak ada audit trail untuk tindakan recruiter

Selain itu, dashboard hanya menampilkan flat list tanpa statistik agregat, dan screening yang gagal tidak bisa di-retry.

Solution — Phase 1: Apply, Dismiss, Auto-Purge

Apply Recommendations (§43.7)

Recommendations dari Job Quality Engine (must_have, preferred, nice_to_have) kini bisa di-apply langsung ke job requirements. View job_apply_recommendations merge ketiga kategori ke job.requirements — nice_to_have otomatis masuk preferred. Skill yang sudah ada di requirements tidak ditambah duplikat (case-insensitive check). Setelah apply, quality analysis diulang otomatis sehingga score dan warnings ter-update.

# apps/jobs/views.py — job_apply_recommendations
existing_lower = {s.lower() for s in current_required + current_preferred}
for skill in rec.get("must_have", []):
    if skill.lower() not in existing_lower:
        current_required.append(skill)

Dismiss Warnings (§43.8)

Job.warning_overrides (JSONField) menyimpan daftar warning kind yang sudah ditinjau. View job_dismiss_warning menambahkan kind ke list, view job_show_warnings mengosongkan list. Template menampilkan tombol × per warning dan link "Show N dismissed warning(s)". Mendukung Accept: application/json untuk HTMX.

Auto-Purge (§43.22)

Celery task cleanup_old_candidates berjalan harian via Celery Beat (03:00). Task menghapus candidate yang lebih tua dari CANDIDATE_RETENTION_DAYS (default 30) dan menghapus file CV dari storage sebelum delete model. Log deletion count untuk observability.

# config/settings/base.py
CELERY_BEAT_SCHEDULE = {
    "cleanup-old-candidates": {
        "task": "apps.screening.tasks.cleanup_old_candidates",
        "schedule": crontab(hour=3, minute=0),
    },
}

Solution — Phase 2: Audit, Dashboard, Error Handling

Audit Logging (§43.23)

AuditLog model dengan 9 action choices: job_create, job_edit, job_apply_recommendations, job_dismiss_warning, candidate_upload, candidate_delete, candidate_status_change, candidate_shortlist, candidate_retry. Setiap action mencatat user, target_type, target_id, metadata (JSONField), dan timestamp. log_action() helper membuat entry dalam satu baris.

# apps/accounts/models.py
def log_action(user, action, target_type, target_id, metadata=None):
    return AuditLog.objects.create(
        organization=user.organization,
        user=user, action=action,
        target_type=target_type, target_id=target_id,
        metadata=metadata or {},
    )

Audit log ditampilkan di tab terpisah pada job detail page dengan badge warna per action. Read-only di Django admin.

Error Handling — Retry + Partial Failure

candidate_retry view: reset processing_status ke queued, kosongkan error message, re-queue Celery task. Tombol "Retry Screening" muncul di candidate detail hanya saat status = failed. Membuat audit log entry (CANDIDATE_RETRY).

process_cv task diperbarui dengan structured error codes: EXTRACTION_FAILED, SCREENING_FAILED, EXPLANATION_FAILED, UNKNOWN_ERROR. Saat extraction berhasil tapi screening gagal, partial result disimpan (score 0.0, grade "Processing Failed") sehingga recruiter tahu CV bisa diekstrak tapi matching bermasalah.

Dashboard Stats

Dashboard kini menampilkan 5 kartu statistik di bagian atas: Total Jobs, Total Candidates, Processing Queue, Avg Quality Score, Pass Rate. Di bawahnya ada section "Recent Activity" berisi 5 audit log terakhir dengan badge warna. Semua stats tenant-isolated — org user hanya melihat data org mereka.

Architecture Decisions

Decision Rationale
Synchronous quality analysis on save Graph sudah di-cache, analysis cepat; tidak perlu async untuk 1 job
warning_overrides as JSONField Simple list of kind strings; cukup untuk MVP, audit trail via AuditLog
AuditLog append-only Tidak ada update/delete — compliance requirement
Celery Beat for retention Daily at 03:00 (off-peak); non-destructive, logs count
Partial failure stores result Recruiter perlu tahu CV bisa diekstrak meski matching gagal
Retry is POST-only Prevent accidental retry via GET (browser prefetch, crawlers)

Testing

  • Phase 1: +11 web tests (apply merge/dedup/POST-only/isolation, dismiss add/no-dup/isolation/JSON/show, cleanup old/recent) — 213 passed
  • Phase 2: +14 web tests (audit create on job/candidate actions + isolation, retry works/only-failed/audit/POST-only/isolation, error_code, dashboard stats/values/isolation) — 227 passed
  • Total: 227 tests (170 engine + 57 web)

Insight

  • Audit trail harus append-only. Tidak ada edit atau delete — begitu entry dibuat, itu adalah fakta yang tidak bisa diubah. Ini bukan pilihan teknis, ini requirement compliance (PRD §43.23).
  • Retry adalah fitur UX, bukan cuma error handling. Recruiter perlu kontrol manual untuk re-trigger screening yang gagal tanpa harus upload ulang CV. Tombol retry hanya muncul saat status = failed — tidak ada yang namanya "retry" pada screening yang sedang berjalan atau sudah selesai.
  • Dashboard stats harus tenant-isolated sejak awal. Query ScreeningResult.objects.count() tanpa filter org akan menampilkan data semua tenant — ini bukan bug yang ditemukan nanti, ini privacy violation yang harus dicegah di level view.
  • Partial failure lebih baik dari total failure. Saat extraction berhasil tapi screening gagal, menyimpan result dengan score 0 dan grade "Processing Failed" memberi recruiter informasi bahwa CV valid tapi ada masalah di matching engine — bukan hanya "something went wrong".

Gotchas

  • Django ORM Q() object tidak bisa jadi positional argument setelah keyword argument. filter(org=org, Q(grade="X")) → SyntaxError. Solusi: filter(Q(org=org) & Q(grade="X")) atau gunakan keyword argument saja.
  • candidate_action view menggunakan status value (shortlisted), bukan display name. Test yang menggunakan "shortlist" (tanpa "-ed") akan 404 karena tidak ada di status_map.
  • Celery eager mode dalam test menjalankan task secara sinkron. Test retry tidak bisa asserting status "queued" (transient state) — harus assert status akhir "done" karena task langsung selesai.
  • quality_seniority adalah dict, bukan object. Template harus menggunakan .detected_level bukan .label, .title_level bukan .years_min.

Kesimpulan

MVP Phase 1 & 2 menyelesaikan empat item P0 dari PRD §43: apply recommendations (§43.7), dismiss warnings (§43.8), auto-purge (§43.22), dan audit logs (§43.23). Dua fitur tambahan — dashboard stats dan error handling dengan retry — melengkapi MVP sehingga recruiter memiliki kontrol penuh atas kualitas job description, audit trail untuk compliance, dan kemampuan recovery saat screening gagal.

Dengan 227 tests passing dan semua MVP DoD items terpenuhi, SkillTrace kini siap untuk Phase 3: rate limiting, API documentation (OpenAPI/drf-spectacular), export (CSV/JSON), dan public demo dengan data sintetis.

Stats

  • Total errors: 0
  • Langkah aktivitas: 23
  • Rentang waktu: 00:02:00 -> 02:35:00
  • Tests: 213 -> 227 (+14 web tests)
  • Migrations baru: 3 (0002_auditlog, 0003_add_error_code, 0003_warning_overrides)
  • Files changed: ~15 per phase

Last updated 24 Sep 2026 23:08

Comments