Where an LLM actually belongs in a monitoring system

The task

Planned maintenance runs on the network constantly, and outages happen at the same time. The danger is in the overlap: a crew takes down a ring for maintenance exactly where traffic already sits on a backup path after an outage. The system in question looks for such overlaps by city and time window and warns before the work starts.

Events arrive from four sources: the monitoring system, BPMS, corporate mail and manual input. Three of them are structured. The fourth is not.

The rule: an LLM where there is no schema

"Run everything through the model" is expensive both in money and in debugging. Here the model works on exactly one stretch - parsing letters from external operators.

The reason is that letters have no schema. One operator writes "scheduled maintenance on 12.07 from 01:00 to 05:00, city N", another attaches a spreadsheet, a third phrases it so that the date has to be assembled from two paragraphs. Regular expressions break on this every week; a model does not.

Everything else is plain deterministic code:

• polling the monitoring API and reading the BPMS queue;
• deduplicating events;
• comparing time intervals and finding overlaps;
• auto-closing conflicts that are no longer relevant;
• metrics, UI, CSV/Excel import.

Comparing two intervals is not a job for a neural network. It is start_a < end_b and start_b < end_a, and it must give the same answer on Monday and on Friday.

What this buys you

Predictability. When a conflict is flagged wrongly, the question "where is the bug" has an answer. If the model made the decision, the answer would be a shrug.

Cost. The model is invoked on the volume of incoming mail, not the volume of monitoring events — orders of magnitude apart.

Testability. Conflict detection is covered by unit and integration tests against a database spun up in a container. You cannot test LLM parsing the same way — there you only check the response format and robustness to junk input.

The architecture in three services

mail-agent  → IMAP → noise filter → LLM → structured event ─┐
                                                            ↓
monitoring / BPMS / manual input ─────────────────→ detector (dedup, overlap
                                                            search, auto-close)
                                                            ↓
                                                  PostgreSQL → UI

Every service exposes Prometheus metrics, logs land in Loki, the database schema is versioned with migrations. Boring infrastructure is the precondition for anyone getting to use the interesting part.

What I would do differently

The noise filter in front of the model should have been there from day one, not after the first token bill: the overwhelming majority of corporate mail has nothing to do with planned work, and simple rules cut it away.