Server outage | museum-digital: blog

md:down (Temporarily)

Joshua Ramon Enslin — Tue, 24 Feb 2026 08:32:35 +0000

Between 6 a.m. and 9:15 a.m., museum-digital’s main server unfortunately experienced another outage. The immediate outage has been resolved now.

The larger issue remained the same as so often during the rocky months last year:

With IIIF processes stopping, they keep holding onto RAM, until eventually the server is overloaded
With no available RAM left, the database (MySQL) falls over

To bring back the server, we needed to:

Stop the web server
Restart PHP (freeing up RAM)
Restart MySQL (bringing it back)
Restart the web server

This was, to some extend, to be expected: PHP ran without issue for some weeks, and the outage shows in the real world how close we can get. As such, the solution will be to further reduce available resources for the IIIF setup and regularly restart PHP.

For some background, see also:

Maintenance / Outage: Search Database not Available for Today

Joshua Ramon Enslin — Wed, 11 Jun 2025 15:54:52 +0000

Some minutes ago, our search server crashed. To bring everything back to normal, we will have to re-index, which will take a few hours. As that is being done, most of the frontend will remain operational without issue. musdb will, depending on the instance, be unavailable into the night.

Server Outage / Switch Fault (March 6th, 2024)

Joshua Ramon Enslin — Wed, 06 Mar 2024 09:56:31 +0000

Between March 6th, 2024, 9:54 a.m. and 10:34 a.m. museum-digital’s main server was unavailable due to a switch fault at the data center at our hosting provider, Hetzner. Details can be seen here on Hetzner‘s website.

By 10:34 the error was fixed by the technicians in the data center and all systems at museum-digital are operational again.

We are sorry for the downtime and looking into ways to better mitigate similar issues in the future.