Which automated health checks belong in every production Xperience by Kentico implementation?

I'm using Xperience by Kentico SaaS uptime monitoring checkers to ensure the highest priority experiences on the Community Portal are working.

For example, if the Q&A discussions page doesn't return a 200, I want to know about it immediately.

I'd love to see everyone else's thoughts on this.

We havent started to use automated tests with XbyK but like you it is high on my list and I was looking through the documentation just last week for what is available within the platform. Our human QC test plans for dev/staging just hit 400 tests but our smoke plans for production I have kept to under 20 tests and we run a smoke test at any PROD code change.

Operational problems - we are building honeypot fields both into forms and shopping carts and IP usage restrictions in. i.e. 3 uses in 5 mintues and you are blocked for the month, you use this hidden field and you are blocked for 3 hours with an onscreen message to say call us if you are a human to resolve this issue. This is saving us the most time. Stopping nuisance that gets through the WAF and captcha and fills databases with rubbish.

I would love to know how you do API reachability and latency - because my devops always tell me it isnt possible to alert on an API outage - usually because of firewall constraints. I do get a response email when it is up, or when the nightly/weekly batch succeeded and all is well. But not when it the API is down or the process failed and I really struggle to notice the absence of the success messages when it does occur. I would love to learn more.

What you have said is a really good list but as the production/ project manager I would add these:
5 min cadence on your uptime tool of choice for these three
Footer sentence load - footers are below the page and scheduled to load late, they change infrequently (so dont falsely alert often) it is also important to monitor them on home but functional pages (with forms or ecommerce) for example. The sentence load time gives you better confidence than a 200 return in my opinion because it proved the whole page loaded reasonable time.

Form Submit button text - complex business critical forms 5 min monitoring of the save button. It also helps load time for the forms keeping them in cache.

Search with a complex query - i.e. on a site where search has had faults we not only probe search loads but undertake a complex search and seek a sentence return on a page we know is permanant. This helped us troubleshoot problems with the search indexing in K13 and we have stuck with this as a good monitoring practice in XbyK, especially as we are delivering filterable search now with XbyK and taxonomies.

I am thinking that monthly is the right cadence for these
Widget use - pick your 2 most complicated widgets - we are finding the widgets the most fragile part of our layer. They are breaking most often on refresh from Kentico. Dont just do the code test, the test needs to be create a page and add the widget in the page builder with configuration end to end (a computer use agent can be trained to do that for you as a user would do it but we only use computer use agents in dev environments without user data in them - see my other contribution to a chat today - https://community.kentico.com/q-and-a/q/anyone-running-coding-agents-headlessly-claude-code-routines-etc-on-xperience-projects-736a260d )
Ecommerce successful payments and failed payment with email verification. We have hidden low cost products to achieve this and the client knows if it sees a submission from Leroy Llama that is us doing our testing. I work the test costs into our retainer. Dont forget a failed card - you want to see the onscreen messages.
Form fills - especially as we have custom field types and functions like draft save, I have playwright scripts for business forms. Again Leroy Llama completes the form - they know it is a test and to ignore but to check the CRM for successful transfer.

AI screenshot comparisons on sentinel pages - I have built and am productising right now an agent that sets the baseline for the site after first release and checks periodically for issues. I have been working on the agent for about 6 months to replace the human QC but I am still working out how much token use is worth it and where to deploy it- https://community.kentico.com/q-and-a/q/anyone-running-coding-agents-headlessly-claude-code-routines-etc-on-xperience-projects-736a260d )

I've been on this topic for a while and currently have been basing our approach on a service called Checkly.

Here is roughly the kind of things we monitor, the first 3 are non-negotiable on every project, the rest are case-by-case.

Uptime

• plain status check plus TLS expiry, every minute

• we keep it dumb on purpose, it's billed per monitor not per run, and it only answers "can a visitor reach this"

API health check

• hits the standard ASP.NET Core health endpoint and reads the body, not just the status code.

• typically runs every 5 minutes (this can get expensive, clients can opt in to higher frequency under their SLA)

• reports on Kentico-specific health:

  • stale scheduled tasks

  • email queue

  • search index health (if it's not algolia, that's a dependency check)

  • web farm

  • read-only mode on

  • unhandled exceptions

• reports on Infra-specific health

  • db

  • keyvault

  • storage

• reports on external dependencies

  • API connections and credentials

Security headers

• its own check, runs once daily, reports a missing header as degraded not down

• separate from the launch audit because headers get quietly dropped the moment someone touches web.config or a cdn rule

Restricted admin

• checks to see if admin route is accessible or not

• has helped with keeping on top of clients tinkering with their firewall rules!

GDPR / analytics tracking consent compliance (Playwright)

• location aware, looks at what actually fires before consent rather than what the consent tool says it's doing

• restricted regions enforce, everywhere else logs and passes

• has caught more than we expected, mostly tag manager firing before consent

Key journeys (Playwright)

• mostly app specific checks here, but this is the most powerful and most insightful of the lot

• Playwright is extremely powerful for regression and having the option to run these in the smoke test suite or regularly has given us and our clients alot of peace of mind

• primarily run post deploy in the pipeline as a gate, the question is "did this deploy break it"

• sometimes we might decide with a client that certain journeys should be checked periodically and that's trivial to set up

Each of the checks are configured to return Healthy, Degraded, Critical. Degraded is always a signal to act on.

I think my favourite part about all this is really how Checkly allows us to define checks as code, so they can live within the repo and aligned to each solution. From there they can be either run within pipelines via Checkly CLI or deployed to Checkly as a monitored check. The developers can write their playwright scripts for standard test automation, and have the option of easily having them reused as smoke or monitored checks. You can also "pause", or "mute" monitored checks at the beginning of deploy pipelines and re-enable at the end so you don't get obvious known outages being logged.

We also have a dashboard which pulls all Checkly checks and shows their status via their API, which is a nice overview to have. Would like to make some of this visible inside the CMS but for us the primary focus has been on our visibility so we can support our clients better.