MY WORK
Contact
Case Study: Human-in-the-loop ML
Human-Centered Workflows Improves AI/ML Fraud Mitigation
A community college system with 100+ colleges needed to improve adoption of its fraud detection tool—so analysts could work faster, trust risk signals, and strengthen the AI/ML feedback loop that improves early detection over time.
THE CHALLENGE
When Risk Signals Outpace Analyst Workflows
A community college system relies on an internal fraud detection and triage tool to quarantine suspicious applications using ML-based risk signals.
The legacy experience created friction at every step—manual cross-checking across systems, limited search and control over queues, and low confidence in opaque scoring—resulting in inconsistent adoption and a weaker feedback loop for improving detection over time.
Project Objective
Redesign and modernize the fraud triage experience to streamline review, improve trust in signals, and enable iterative enhancement.
Business Impact
Increase adoption and operational throughput so suspicious activity is caught earlier and handled more consistently across a distributed system.
User Value
Give analysts faster, more flexible workflows and clearer context so they can make confident decisions with less cognitive load.
how i led
Role, Team, & Constraints
I led product strategy and experience design in close partnership with platform leadership and engineering, shaping an MVP that delivered near-term workflow wins while laying the groundwork for faster iteration and stronger model operations over time.
1
Product Strategy & Experience Lead
Owned the UX strategy and end-to-end experience direction—from framing the product thesis and “Parity + 1” MVP approach to synthesizing research, validating prototypes, and shaping the phased roadmap.
2
Team Composition
Worked with a cross-functional delivery team spanning product, UX, engineering, and platform stakeholders, plus a cross-college working group of fraud analysts and supervisors to validate workflows and prioritize the highest-value enhancements.
3
Key Constraints
Deliver measurable usability and adoption improvements quickly, minimize backend disruption, and modernize enabling platform capabilities (auth + frontend standards) so the tool could evolve rapidly after MVP. x
WHAT WE DID
Research to Roadmap
Aligned on the primary user and their jobs-to-be-done
We focused the roadmap on the Analyst persona (primary user) and included Supervisors as a secondary persona, clarifying tasks, pain points, and where the current tool broke down in real work.
Designed for a human-in-the-loop system
Improving the UX not only increases adoption and efficiency, it also strengthens the UI→ML feedback loop with cleaner, higher-quality labels—improving model precision over time, boosting systemwide fraud detection, and reinforcing trust in the tool.
Synthesized research into themes that guided the roadmap
We mapped the end-to-end triage workflow, conducted a heuristics review of the current state tool, and explored best-in-class fraud monitoring solutions for inspiration and best practices.
This research and discovery informed six key themes.
Designed a modern triage workflow. Built on a scalable foundation.
We built and iterated high-fidelity prototypes with the working group, focusing on high-leverage workflow improvements:
  • Sortable/filterable application queues to manage review load more efficiently
  • Improved information hierarchy + scannability in application details to reduce cognitive load
  • A grid-driven interaction model (AG Grid) that supports power-user workflows and future enhancement paths
To reduce friction and increase consistency across admin tools, we aligned on enabling technical upgrades alongside UX improvements, including front-end modernization (NextJS) to standardize patterns and speed iteration beyond V1.
Tiered product plan enable near-term wins with a long-term vision
A pragmatic V1 (MVP) anchored in “Parity + 1”—preserve core functionality while introducing targeted enhancements that unlock iterative improvement..
We defined a roadmap that kept momentum after MVP—sequencing V1.x fast follows (including features needing backend changes) and a longer-range path to V2+ capabilities like richer metadata, explainability, and deeper pattern analysis.
V1 (MVP)
Parity + 1 - Preserve existing functionality but design in a more modern, thoughtful way
Prioritize Time-to-Release - Start with out-of-the-box grid functionality and minimal backend changes
V1.x Fast Follows
User requested features that required backend changes
Outstanding discovery themes, including score explanability, fraud history, & accountability
Agile releases to maintain V1 momentum and get enhancements into users’ hands sooner
V2
More complex, high-value user requested features
Richer context data and account levelt metadata
Enable pattern analysis across suspicious applications
V3, V4
  • High value, high complexity features: eg. Rules Engine, ML anomaly detection, etc.
  • User Control, Quality of Life Improvements: eg. Custom Views, Full Screen Mode, Application Notes, User Highlighting
  • Adjust priorities as we learn from previous releases
Success
Outcomes & Impact
01
Faster Triage
Sortable, filterable queues helped analysts focus quickly and reduce time spent hunting for the right data.
02
Stronger Decisions
Richer context and explainability patterns increased confidence and supported more consistent, auditable actions.
03
Smoother Access
Clear MVP specifications through flows, wireframes, and hi-fi prototypes enabled faster development cycles with fewer ambiguities and reduced technical debt.
04
Faster Delivery
NextJS patterns and a product-oriented sprint cadence created a foundation for rapid, iterative improvements across admin tools.
Key Takeaways
Human-centered workflows + explainability are adoption multipliers—and adoption improves model quality when the UI captures better human feedback.
“Parity + 1” balanced quick wins with a scalable path, while a tighter UI→ML feedback loop operationalized CCC’s broader fraud mitigation strategy.
View more case studies:
0→1 MVP | Human-centered AI | Experience Modernization | Human-in-the-loop ML
Let's Create Something Exceptional Together!
Doris Mao Lin | Los Angeles, CA | 310-497-2949 | dorismaolin@icloud.com | linkedin.com/in/dorisl/ | dorismaolin.me