Where an LLM actually belongs in a monitoring system

The task

Planned maintenance runs on the network all the time, and outages happen alongside it. The danger lies in overlaps, for example when a crew takes down a ring for maintenance where redundancy has already been lost to an outage. The system described here finds such overlaps by city and time window and warns about them before the work starts.

Events come from four sources: the monitoring system, BPMS, corporate mail and manual input. All of them are structured except mail.

The rule: use an LLM where there is no schema

Running everything through a model is expensive, both in money and in debugging effort. In this system the model is used only to parse letters from external operators.

These letters have no schema. One operator writes "scheduled maintenance 12.07 from 01:00 to 05:00 Moscow time, city N", another attaches a spreadsheet, and a third phrases things so that the date has to be pieced together from two paragraphs. Regular expressions break on these letters every week, while the model handles them.

Everything else is ordinary deterministic code:

• polling the monitoring API and reading the BPMS queue;
• deduplicating events;
• comparing time intervals and finding overlaps;
• automatically closing conflicts that are no longer relevant;
• metrics, UI, CSV/Excel import.

Comparing two intervals does not need a neural network. The condition start_a < end_b and start_b < end_a is enough, and it gives the same answer on Monday and on Friday.

What this gives in practice

Predictability. When a conflict is detected incorrectly, the location of the error can be found. When a model makes the decision, tracing the cause is much harder.

Cost. The model is called at the volume of incoming mail, which is orders of magnitude smaller than the volume of monitoring events.

Testability. Conflict detection is covered by unit and integration tests against a database running in a container. LLM parsing cannot be tested the same way, so only the response format and robustness to junk input are checked.

Three-service architecture

mail-agent  → IMAP → noise filter → LLM → structured event ─┐
                                                            ↓
monitoring / BPMS / manual input ─────────────────→ detector (dedup, overlap
                                                            search, auto-close)
                                                            ↓
                                                  PostgreSQL → UI

Each service exports Prometheus metrics, logs are collected in Loki, and the database schema is versioned with migrations. Without solid infrastructure like this, even the most interesting part of a system is of little use to anyone.

What I would do differently

The noise filter in front of the model should have been added from the start, before the first token bill. The vast majority of corporate mail has nothing to do with planned maintenance and can be filtered out with simple rules.