How Our Course Recommendation Engine Actually Works

The standard playbook for a recommendation engine is to watch what people click, what they finish, and what they come back to, and use that behavior to predict what to show them next. It's a good playbook for a streaming service. It's a genuinely risky one for a mental health platform, because the course someone clicks most and finishes fastest isn't necessarily the one that's actually helping them — sometimes it's just the shortest, or the least emotionally demanding, which are not the same thing as the most useful.

We built YouMindo's course recommendation engine to resist that failure mode from the start, which meant deliberately not building the thing a standard recommendation-system tutorial would tell you to build first. It took longer to ship, because seeding recommendations from clinical assessment data required close, ongoing collaboration with our clinical team in a way that pure behavioral modeling wouldn't have — but it meant the engine had a defensible answer to 'why was this course recommended' from the very first version, instead of an answer that boiled down to 'because other people liked it.'

Starting from assessment results, not clicks

The engine's primary signal is a client's assessment scores — what they've told the platform they're working on, through the intake process and periodic check-ins — not their click or completion history. A course gets recommended because it's clinically relevant to what an assessment indicates someone is dealing with, before the algorithm has any behavioral data about that person at all. Click behavior only enters the picture later, as a secondary adjustment layer on top of that clinical seed.

Why we didn't start with collaborative filtering

The obvious technical approach — 'people with assessment profiles similar to yours also completed these courses' — is exactly the kind of collaborative filtering that works well for retail and streaming. We prototyped it and didn't ship it. It tends to converge toward whatever's already popular, because popular courses accumulate more completion data, which makes them more likely to be recommended, which makes them more popular still. That loop actively worked against surfacing the right course for someone with a less common profile.

The mistake: recommending what's popular

An early internal build did lean on completion-rate-weighted recommendations as a tiebreaker whenever assessment relevance scored two courses similarly. In testing, this quietly pushed a handful of shorter, broadly appealing courses to the top of almost everyone's recommendations, regardless of what their assessment actually indicated. We removed completion-rate weighting from the tiebreaker entirely and replaced it with a simpler rule: when relevance scores tie, show the course reviewed most recently by our clinical content team, not the one most people finish.

How re-ranking actually happens

Once a client starts interacting with recommended courses, behavioral data does start to matter — but only within the set of courses that already passed the clinical relevance bar, never to introduce an unrelated course just because it's popular elsewhere. If someone consistently skips modules involving written reflection and engages more with audio content, the engine will favor audio-format courses among the clinically relevant options, but it won't recommend an audio course just because it's easy if the underlying topic doesn't match what the assessment flagged.

Avoiding the trap of only recommending the easy stuff

We noticed early that any signal weighted toward completion rate will systematically favor shorter, lighter courses over longer ones that address harder material, simply because more people finish the short ones. We built an explicit counterweight: courses addressing a client's primary assessed concern get a relevance floor that behavioral signals can't push below, even if that course has a lower average completion rate platform-wide. The goal is relevance to the person, not ease of completion in aggregate.

Why we cap recommendations at three

Our course catalog has grown well past what any client would want listed at once, and our first version showed a longer ranked list — six to eight courses — on the theory that more choice meant more chance of a match. Usage data showed the opposite: clients presented with a shorter list of three were meaningfully more likely to start any course at all, compared to clients facing a longer list who more often started nothing. We cut the default recommendation surface down to three, with the full catalog still browsable separately.

The therapist override

A therapist working with a client on YouMindo can override or manually reorder that client's recommended courses directly, and that override persists — the algorithm doesn't quietly re-rank around it on the next update. This was a deliberate design choice: a therapist's clinical judgment about what their specific client needs right now should outrank a general-purpose scoring model, and the system should make that override durable, not something that has to be re-asserted every time the underlying data changes.

What we can't fully account for

The engine's picture of someone is only as good as their assessment history and how recently it was updated, and clients don't always retake assessments as often as their actual situation changes. Someone going through a hard month might be shown recommendations still calibrated to how they were doing two months ago. We've added prompts encouraging more frequent check-ins, but we haven't solved the underlying lag between someone's real situation and what the data reflects.

What we measure to know if it's working

We track whether recommended courses that clients actually start correlate with meaningful movement on the assessment dimension that course was meant to address, not just whether the course gets started or finished. Completion alone is a weak proxy for whether a course helped, and we've tried to build our internal success metric around the outcome the course was supposed to produce rather than around engagement with the course itself.

A recommendation engine tuned purely for engagement would probably show better completion numbers than the one we built. We think it would also be recommending the wrong thing to a meaningful share of people, quietly, in a way that would be hard to notice from the outside. Starting from what someone's actually working on, even at some cost to completion rates, was the trade we were willing to make.