Server Health Monitor watches seven learning platforms for HCL Software. This case study covers one narrow problem inside it: knowing whether a course page and its videos actually load — not just whether the server replied.
▲Two red blocks. Ninety-four green. The green blocks include the window where the course catalogue rendered empty — because the homepage still returned in 393 ms, and that is all the check was ever asked to measure.
Uptime monitoring answers one question: is the front door open? Support gets asked a different one: can a learner finish a course?
Server Health Monitor checks seven endpoints every fifteen minutes. A check passes when the host responds and responds quickly. That is a genuine signal — it catches hard outages, expired certificates, and DNS failures. It is also blind to the entire class of failure that produces the most tickets in a learning platform, because in every one of those cases the server answers perfectly.
Only the last row is caught today. The four above it are the ones that reach a human first — and the human is usually a learner, not an operator.
The Overview headline reads “All 7 sites are working fine.” The uptime card reads 100.00%. Four rows further down the same page, HCLSoftware U sits at 92.0% with two red blocks in its 24-hour strip, and iSuccess responds in 1,939 ms under a green “Working” badge.
The summary and the evidence beneath it disagree, and both are computed from the same shallow signal. That is not a copy bug to patch. It is the symptom that defined the brief.
Five inputs, chosen to separate what people say goes wrong from what the record shows goes wrong.
| Method | Sample | What it was for |
|---|---|---|
| Semi-structured interviews | 9 participants · 30–45 min | 5 LMS platform operations, 2 support desk leads, 2 on-call engineers. How an incident is currently noticed, confirmed, and communicated. |
| Support ticket review | 90 days, LMS-tagged | Coded by failure class and by who reported it first, to test whether monitoring or people detect problems. |
| Diary study | 3 operators · 2 weeks | Every manual "is it actually up?" check logged with trigger, duration, and outcome. |
| Competitive teardown | 6 synthetic-monitoring tools | Scored on authoring effort, LMS-specific assertions, scheduling, and failure evidence. |
| Instrumentation review | Existing check definition | Documented exactly what the current check asserts — and everything it does not. |
Six findings survived triangulation across interviews, tickets, and diaries. Quotes are composites drawn from the interview set, not verbatim attributions.
Learners are the monitoring system. The first signal of a degraded platform is almost always a message from someone trying to use it — not an alert.
“I find out we have a problem when three people ask me the same question in ten minutes.”
Confirmation is the expensive part. Before anyone can escalate, an operator opens the platform themselves, signs in, finds a course, and tries to play something. That reproduction loop is manual, repeated per platform, and repeated again for every person who asks.
Nobody trusts the green. Operators re-checked manually even when the dashboard was fully green, because they had been burned by a green dashboard during a real outage.
“The dashboard tells me the site is up. It has never once told me a course won't open.”
Partial failure hurts most. Whole-platform outages are rare, loud, and quickly owned. The damaging failures are narrow: one course, one media host, one region, one browser.
Test authoring was the blocker, not test running. The team knew what to check. Nobody on it wrote Playwright or Selenium, and there was no route to a test that did not go through an engineering request.
“I can tell you exactly what to click. I can't tell a script how to find it.”
Noisy alerts get muted within a week. Every participant who had used alerting before had silenced a channel. Any new alert had to be provably quiet before it would be trusted.
Three roles came out of the interviews with meaningfully different needs. The module is designed for the first; the other two consume its output.
Each principle answers a specific finding and constrains a specific design decision.
| # | Principle | From | How it shows up in the design |
|---|---|---|---|
| P1 | Assert the outcome, not the response code | F1, F3 | A journey passes only when the thing a learner needs is present — the course grid rendered, the video element playable. HTTP status is not a pass condition. |
| P2 | Anyone who can describe a test can build one | F5 | AI Auto-Pilot accepts plain English and generates the steps. Manual Steps stays for people who want exact control. Neither requires code. |
| P3 | Alert on confirmed failure, never on a blip | F6 | One failure triggers a retry, not an email. Two consecutive failures make it an incident. |
| P4 | A summary may never contradict its evidence | F3 | The Overview headline is derived from the worst journey result, not from an average of response times. |
| P5 | Every failure ships with its reproduction | F2, F4 | A failed run stores the step that broke, a screenshot at that moment, the URL, and the timestamp — so confirmation is reading, not re-doing. |
| P6 | Authoring is a two-minute task | F5 | Name, prompt, Save & Run. Scheduling is a default, not a required step. |
Synthetic Journeys sits beside Link Monitoring rather than inside it. The two answer different questions and should never be mistaken for each other — that mistake is the original problem.
It was the cheaper build, and it was rejected. A link check and a journey produce results that look identical in a table and mean completely different things. Collapsing them would have reproduced the exact confusion the project exists to remove — a row that says “Working” when the learner sees nothing.
Link checks already run every fifteen minutes. Journeys are heavier — a real browser, a real sign-in — so they run less often and are staggered so that seven platforms never start at the same minute.
# proposed default schedule — one line per journey */15 * * * * link checks, all 7 endpoints (existing) 7 * * * * HCLSoftware U · catalogue loads 22 * * * * HCLSoftware U · course video plays 37 * * * * iSuccess · sign-in and learning home 52 */2 * * * Percipio · course launch 15 6,13 * * 1-5 full regression across all platforms
The current-state journey for a partial failure — a course catalogue that renders empty while the homepage responds normally.
| Feature | Behaviour | Serves |
|---|---|---|
| AI Auto-Pilot authoring | A journey name and a plain-English prompt produce a runnable browser test. No selectors, no code. | P2, P6 |
| Learning-specific assertions | Course grid is populated · a named course opens · the video player loads its media · search returns results. | P1 |
| Cron scheduling | Per-journey interval, staggered across platforms, editable as a plain expression. | P1 |
| Two-strike alerting | Retry once after 60 seconds. Email only on a second consecutive failure. | P3 |
| Failure evidence pack | Failing step, screenshot at failure, URL, duration, timestamp — attached to the run. | P5 |
| Honest Overview verdict | The headline is derived from the worst current journey result, never from average response time. | P4 |
Slack and Teams routing, per-journey escalation policies, synthetic checks for non-learning systems, video quality-of-experience scoring, and mobile-app journeys. Each is defensible; none of them close the gap between “responded” and “worked”, which is the only thing release 1 is measured on.
Three screens carry the whole module: an empty state that teaches, a builder that accepts a sentence, and a result that proves what happened. These were drawn to settle the shape before any of it was worth refining.
Structure and hierarchy resolved at greyscale, so that layout decisions were argued on their merits rather than on colour.
Colour applied last, and only where it carries meaning: teal for the module in focus, and the status semantics — teal, amber, red — reserved exclusively for run results.
Seven screens from the running product, in the order a new operator meets them.
This screen is the argument for the whole module. The headline reads “All 7 sites are working fine” and the uptime card reads 100.00%, while HCLSoftware U shows 92.0% with two red blocks and iSuccess responds in 1,939 ms under a green badge. Nothing here is a lie — every number is correctly computed from what the check measures. The check simply measures the wrong thing.
Principle P4 targets exactly this line: once journeys are running, the verdict is derived from the worst current journey result rather than from an average response time, so the sentence and the table can no longer disagree.
Deliberately left alone. This module does its job well — it just answers a narrower question than its results imply. Journeys were added beside it rather than folded into it.
The subtitle carries the definition — automated browser tests that verify user experience flows such as logins and course loads — because this is a concept most operators are meeting for the first time. The sketch round proposed naming a concrete first journey here; that refinement is still open.
Save is disabled until there is something to save, and Save & Run is the emphasised action. The placeholder text teaches the prompt format by example rather than through help copy.
These two prompts are the entire thesis in plain English. Neither mentions a status code, a selector, or a timeout. Both assert something a learner would notice — and both are written by the person who already knew what to check but previously had no way to express it.
Authoring and scheduling are shipped. Result surfacing is the open work.
The builder proves the authoring hypothesis: a platform operator with no browser-automation background can create a working test by describing it. That was the blocker identified in F5, and it is gone.
| Gap | Why it matters |
|---|---|
| Run detail with evidence | Without the failing step and a screenshot, P5 is unmet and an alert still triggers the manual reproduction loop it was meant to replace. |
| Journey results on Overview | The verdict line is still computed from response time. Until it reads from journeys, the contradiction in section 13 remains on the screen. |
| Visible schedule and next run | Cron is configured outside the interface today. Operators cannot see when a journey last ran or when it runs next, which undermines trust in it. |
| Two-strike rule in the UI | The retry behaviour needs to be stated on the journey, or a first failure will be read as an outage. |
Four measures, all baselined before release so the comparison means something. None of them is uptime.