museum-digital:blog - Frontend

Braving the Waves. Surviving in Times of AI.

2026-10-01T00:00:00+00:00

Now that the preview release of musdb’s new UI has finally been released, there’s time to write about the other sides of museum-digital. More usual struggles. Which is to say: New, larger waves of AI scrapers and bots that once again started to impair our services starting July this year.

We maintain that hypocrisy is not for us. I personally have been involved in discussions about the FAIR principles from 2015 onwards. And in 2015 they were already running under that label for a while. As a formalized document, they’ve been around since 2016, but disregarding the label, the principles and surrounding discussions go back, I guess, at least to the 1990s.

In the cultural sector, with many institutions being primarily tax-payer funded, we want to provide our data and services in a Findable, Accessible, Interoperable, and Reusable manner. Relevant here are primarily accessible, interoperable, and reusable.

Realistically, accessibility refers to two things: Immediate, universal barriers like a requirement for authentication should be avoided where possible. On the other hand, user-specific barriers should be reduced as much as possible as well: Human-readable pages should e.g. be designed with sufficient contrast.

If one sees other services and machines as users, which may be reductionist but is logically much more coherent than doing otherwise, then interoperable refers simply to a special form of accessibility: As data should be provided in a way that as many human users can use as possible, it should also be served in formats that machines can read. Ideally in ways that machines can read without further adjustments - following open standards.

Reusable then builds upon these. As the data is now freely accessible by others, setting appropriate licenses allows for the creative reuse of the data.

Ten years after the FAIR principles were formalized, we finally have someone who reuses our data on a massive scale. We should rejoice. Unfortunately all our previous conceptions of how that reuse might take place turned out to be wrong.

Celebrating AI Scrapers?

AI scrapers flooding web services, and especially larger, well-established infrastructures with requests has been an ongoing story for some two years now. Last year museum-digital was met with a first large wave of AI scrapers. See the previous blog posts (1, 2, 3).

On the one hand, using our data for training AI models is undoubtedly reuse. It is even productive reuse: It’s a small contribution to the training of technology that, at its current quality, sounded like science fiction 10 years ago.

On the other hand, this type of reuse takes place without attribution. AI scrapers work based on scale; Machine-readable APIs are uninteresting to scrapers. They suck up as much data as possible, as quickly as possible. Any page-specific adjustments, such as finding and using the API, would reduce the speed of scraping. The requests are so numerous, that they routinely crash servers. Last but not least, those prominently benefitting from the AI hype are far from sympathetic people.

But principles are principles.

It, again, cannot be denied that what AI scraping is eventually aimed at is reuse. Even though they use HTML rather than the APIs we lovingly carved for them, they interoperate with our services in the way most accessible to them.

The problem is, that they are not playing fair. And if the previously unlikely number of requests leads to servers crashing and shutting down, then all principles were for naught. A service that is providing no data at all cannot provide them FAIRly either.

At museum-digital we have seen the new requests as an opportunity to improve our software. If it can withstand the onslaught of thousands of bots a second, it is confirmedly battle-tested and stable. Improving our publishing software on the other hand also meant cutting out overly resource-hungry functionalities. Last year we dropped most publicly available, server-side PDF generation capabilities, we moved the IIIF APIs to an separate configuration that is specced to be able to fall over without impairing the rest of our services. We improved the parsing of search terms and introduced rate-limiting (how many requests a single IP can perform for a given time).

The new wave of AI scrapers since July is even bigger than last year’s. Those improvements were not sufficient to keep our primary server, which hosts the production databases, stable.

From Each According to Their Ability: Load Shedding

We continued on last year’s course however, analyzing the scrapers’ requests, their influence on server stability (which is to say, which requests were especially resource-intensive) and adjusted our setup accordingly. Note, that we did not scale up: museum-digital runs on exactly the same bare-metal servers today as it did two years ago.

In analysis, we identified two types of especially resource-intensive browsing behaviors that are almost entirely limited to scrapers (as well as some very engaged researches, to whom we express our regret for now regularly banning them):

Scrapers follow links on search pages, especially the facet search, and end up combining more and more search parameters. It is unlikely that a human would perform a search for objects that “Are related to Berlin, and related to Germany, and related to Institution X, but not related to Institution Y, and also related to a time between 1990 and 2000.” For an automated scraper it is normal behavior.
If a published object page has been updated, a snapshot of its current state is automatically saved to provide an archived version of the page at a given time. This is thought mainly for researches to be able to cite a given state of the page. In everyday operation, archive pages should be barely used. They are also de-listed from search machines. But saving snapshots again and again leads to the existence of a large number of links and separate pages scrapers can scrape to no benefit to anybody.

Both functionalities are legitimately useful but scale badly.

We hence introduced load shedding: Whenever a search is performed or an archive page is accessed, the current server load is evaluated. If it is above a certain threshold, the server does not perform complicated search queries (depending on how high load is, it is restricted to a minimum of three search parameters) or does not load the archive page. Instead a warning is presented, that the given action is currently unavailable and a custom HTTP error code is sent. If the same IP causes that error code to be sent twice, the IP is blocked for a while.

As of the time of writing writing, we have thus blocked a total of 131,014,530 IPs in two month, with 5,802,771 being currently blocked.

This approach follows our principles: We try to be fair with everybody. But if users are not fair and do not follow advise, we don’t feel obliged to continue playing fair either.
Unfortunately this approach works on the level of blocking whole IPs. As “residential proxies” - back in the days we called them botnets infecting home routers - have become an ever larger problem, it is not unlikely that actually interested, normal users also unknowingly host an AI scraper at home. I fear I’ve already encountered one such case, were a colleague could not access museum-digital for seemingly no good reason from her home network. Otherwise, it is surprisingly effective - the number of requests has barely been reduced, but our services are essentially stable.

The only crash since happened the day before yesterday and was caused by the combination of bots and an error in our controlled vocabularies - the latter of which is entirely our fault.

Numbers Games: Logging Strategy

After the above-mentioned actions had effectively returned stability to our publicly accessible portals, one issue remained for internal services: File uploads were incredibly slow. It turned out that the large number of requests being permanently logged overburdened the SSD. We have hence disabled the general access logging, restricting ourselves to logging requests that either cause errors or trigger slow responses. This reduces the ability to react well-informed to new waves of requests and actual attacks. Thankfully those also commonly trigger actual errors we can still identify.

Fairness, FAIRness

With these actions we have again averted the need to use more drastic strategies that would limit accessibility to users - human and machine alike - like blanket bans on whole regions of the world or the installation of software such as Anubis. And, again, we have not yet needed to improve our hardware (and pay the price for that), even though that might be in order eventually. It might be wise for other reasons as well.

A holistic picture of one’s setup - software, hardware, systems administration - and the capability to think and act upon these together helps a lot.
But, if anything, hosting in a FAIR, principled way still works in these trying times.

info

AI Use

Post image generated using Anima Base v1.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

State of Development, December 2025

2026-01-12T00:00:00+00:00

December 2025 was an interesting month for museum-digital. An update to the PHP version used as well as a flood of requests by what is most likely AI scrapers forced us to make changes for improved stability, reducing and reformulating features rather than adding new ones and working on matters of systems administration over purely matters of code quite often. Add to that the long-promised update of the terms of use for German museums to more structured and lawyer-approved ones, and you get yet more small changes that do not directly concern the work of museums with museum-digital but rather improve the necessary context.

musdb

Object overview

In the default tile view of the object overview page, hovering over an object image thus far revealed the object’s name. As object names are often too long to display fully and inventory numbers are the primary means of identifying an object in most museums, this preview text has now been extended to include the inventory numer.

Hovering over an object image in the tile view now also displays the inventory number.

User management

New Options for Managing User Accounts: Disabling Accounts & Setting Account Expiry Dates

Two new options on user editing pages allow disabling logins on an account and setting an expiry date for the account. Both can be useful for administration: If a new worker joins the museum for a project with a clear-cut limitation on funding and time, one can now set the account expiry at the beginning of the project to the end of it. The accounts will then automatically be deleted when the project ends. Similarly, colleagues that leave service temporarily but for a prolonged time (e.g. for a sabbatical) and will not need to use their accounts for that time can have their accounts disabled.

Two new options allow disabling user accounts and setting expiry dates for user accounts.

List of Terms of Use

A new tab on a user’s (own) account settings page provides the option to list all usage agreements / terms of use a user has agreed to in the context of their use of museum-digital / musdb and when they agreed to them.

A new tab on the user page lists all user agreements for musdb that the user has agreed to and when they did so.

Imports

Limiting Report Mail Size

When a user runs imports themselves using the WebDAV upload, the end of the import process - no matter if it is successful or fails - is marked by the sending of a report via mail. This report usually contains a list of noteworthy operations that happened during the import, e.g. which objects of which inventory number were imported to which object in musdb, identified by its ID. As imports grow, this list of operation grows. To not encounter issues sending the report, it is henceforth limited to a maximum of 2 MB or 10000 lines.

Dry-run Mode

Sometimes it is useful to try running an import to see if it will actually work but not actually process any data. This option has been available in the importer command line interface for a while, among others powering museum-digital:qa. It is now available in the import configuration for self-run imports as well using the setting dry-run. Enabling the setting accordingly stops the importer from actually writing the data into the database and changes the behavior if values that need to be mapped to values in controlled lists at museum-digital are encountered. Usually an import stops the moment such data is to be imported and not yet mapped. During a dry run, the error is collected and the import proceeds. All unmapped entries are listed together at the end of the import, allowing for a simpler mapping (possibly aided by concordance.museum-digital.org).

Dashboard

The first page of the dashboard, which for almost all users also means the start page of musdb right after the login process, was significantly reworked during the last month. The almost entirely unused notetaking features and discourse integration were removed in favor of a feed of recent blog posts. See also the section “Communications” in the respective blog post.

The dashboard in musdb now features a feed of recent news relevant to the development of museum-digital and whatever is going on regionally. The posts are sorted chronologically.

Annotations for the Vocabulary Editing Team

Each event, displayed as a tile on object editing pages, featured speech bubble icons behind each time / actor / place to provide additional comments and hints for the central vocabulary editing team. This positioning of the annotation feature led to confusion over the years, with some users using the feature to comment on the relationship between the entity and the object (for which the event notes should be used). We hence repositioned the links and moved them to the respective entity’s page (e.g. a place page for giving hints and comments on a place entry). The hinting / commenting feature for times has been altogether removed, as providing comments to clarify the meaning of e.g. a year never made much sense.

Smaller Updates and Bugfixes

Fixed a bug in the HTML generated for listing other objects linked to an object. Links to the other object were broken and are not anymore.
Image editing pages now embed the image directly instead of using the IIIF API. This reduces resource usage and increases stability at no cost.
Removed option to manually trigger the rewriting of EXIF and IPTC metadata of object images. Rewriting takes place in the background whenever an image or a linked object is updated, making user-triggered updates obsolete.
Re-introduce option to repeat linking to the last used linked object
Updated Swagger UI to version 5.30.3

Frontend

As stated above and lengthily described in the previous blog posts (here, here, and here) we struggled with stability over the last month. This means that most changes in the frontend are aimed at improving stability.

Reworked Default Image Page

Thoroughly described in Updates, AI scrapers, and Resilience, we replaced the default view for single object image pages. While the default view was previously built around the IIIF viewer Mirador, the new default view uses OpenLayers and the unmediated image file for capabilities such as zooming. The new view also brings with it some new features, such as an option to reference specific sections of an image.

The reworked default image page.

Serving Resource-Intensive Pages / Functionalities Only When Resources Are Available

PDF generation, the IIIF Image API, and the suggestions for alternative search queries on failed search pages are now limited to reduce their impact on the overall system stability. This follows two strategies:

Suggestions on failed search pages and PDF generation will only appear if the overall load on the system is low. The threshold for when or when they are not provided is influenced by the user’s browser language: If a user uses a browser set to the primary language of a given instance of museum-digital (e.g. German in Hesse, Hungarian in Budapest), the threshold is much higher, meaning users will be able to access the pages at a medium server load. In the case of PDFs, high server load will forward users to the print dialogue for object pages instead of receiving a PDF generated on the server side.
PDF generation and the IIIF Image API are served with a different PHP configuration and set of processes than the rest of the frontend. This configuration significantly reduces available resources for these two functionalities.
The option to generate PDFs featuring all images of an object with between 10 and 40 images has been entirely removed. Given its constraints, the feature was hard to explain and rarely accessible anyway. The primary “users” were noticeably AI scrapers.

Image Search

The image search feature was refactored and reduced to further separate it from the primary object search. The number of available search options has been reduced to be more easily explainable and reduce possibilities for very resource-intensive queries.

The reworked image search settings overlay.

Batch Export of Object Metadata / OAI

Updated the LIDO API to almost entirely match the LIDO as generated during exports from musdb

Smaller Updates and Bugfixes

Improved performance of object search by tags and places by filtering searched entities to those who are actually linked in the given instance of museum-digital.
Object groups with only one object are henceforth not displayed and linked on object pages anymore
Fixed link in footer: Clicking on “museum-digital” should lead to the home / start page of the given instance of musdb.
Updated Swagger UI to version 5.30.3

nodac

User-provided comments / hints have been removed for times (see above)
Tooltips for linked objects now display which institution an object belongs to
- This is particularly important for vocabulary editors who do not have access to the museums’ data. This way they get a limited preview with the required information for unpublished objects despite their otherwise lacking permissions.

info

AI Use

(Post image generated using AI)

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

Trimming.

2025-12-29T00:00:00+00:00

In the last weeks we struggled with server stability. As written before, the critical, resource-heavy and publicly available tasks have for a long time been the generation of timelines (and thus complicated search queries) and on the other hand those involving the processing or generation of large files; namely the IIIF API and PDF generation.

In the last post, I detailed how we severely restricted the availability of the public PDF generation functionalities in museum-digital according to available system resources. That, as it turned out, was not enough to bring reliable stability to our systems. After the server fell over on December 26th once more, we hence moved the IIIF Image API into the same PHP setup used for PDF generation - meaning that any user/IP can only request the API 10 times a minute and that for any instance of museum-digital, only one PHP worker serves it. This allowed us to severely reduce the maximum available resources per worker for the frontend outside of those two use cases (where the IIIF Image API may use up to 80 MB of RAM, no other part of the frontend will go beyond 5). Since then, the system runs as smoothly as if AI scraping had never become an issue.

A Limited Goodbye to IIIF & Server-Side Image Manipulation

Now, what does that mean in practice? On the one hand, we have not fully removed the IIIF image API. All links generated using it remain valid and will be served, even if comparatively slowly.

On the other hand the user experience with viewing the images in a IIIF viewer will be significantly worse, even though this strongly depends on the IIIF viewer. The most popular “full” IIIF viewers being Mirador and Universal Viewer, significant problems (or a complete inability to use an object’s images) are to be expected with Mirador. Mirador in its default configuration loads multiple segments of an image separately to then assemble the displayed image from those - with the creation of the segments happening on the server, thus consuming resources centrally. It also seems to set extremely low limits on accepted response times, which museum-digital’s IIIF Image API now regularly exceeds due to the aggressive rate limiting. Simply looking at the demo installation of Universal Viewer, the software seems to be much more targeted in its API calls and might still work well despite the restrictions.

As far as I know, there are no published numbers on the market share of the different IIIF image viewers. And about whether IIIF viewers external to whoever provides the API are actually regularly used or not. The most jaded - and likely true - assumption would be that the share of users who use IIIF without a viewer hosted next to the API is miniscule and that most users will use one of the abovementioned. Our experience, once again, seems to support that hypothesis: We released our implementation of IIIF 2 in 2020, but essentially nobody noticed before we also started hosting a IIIF viewer.

As we do use Mirador as a viewer, assume the “visible” IIIF image API at museum-digital to be more or less broken. Developers and those making direct use of the API without our installation of Mirador can still benefit from the API. But those are comparatively few.

The radical restriction of resources provided to the IIIF Image API is thus likely indeed a goodbye to IIIF, if a limited one. The basic idea is great - to create a unified way to reference parts of an image (or later a wider media file) and annotate it. In times of significantly increased bot activity, reduced funds, and foreseeably rising hosting costs, our example may be an early sign that the decision to realize that aim by specifying an API to be implemented by the data providers restricts the ability to fully support IIIF to very well resourced institutions. And as funds are shrinking, that is less and less institutions. Let’s hope that the most basic need IIIF wished to fulfill can be achieved in a different way in the future; one that is accessible to anybody. Realistically this means that computing would need to happen on the client PCs, not on a server.

To end the saga on a more positive note: Since we limited the IIIF Image API, our systems run wonderfully smoothly again and we were able to reduce the overall rate limiting on the rest of museum-digital’s portals. We will monitor the situation and increase the limit slowly to allow more simultaneous API requests without risking stability.

Communications

Second, the whole ordeal posed a challenge to our communication channels. If any significant error occurs anywhere on museum-digital, I personally am sent an encrypted error message via mail. Usually. In this case, the primary component falling over was the PHP server, which is also responsible for managing the sending of mails. If a service fell over, the primary way to learn of it was receiving mails about that instead. Reaction times were thus worse than they needed to be. This means that we need to improve our monitoring.

On the other hand there was the issue of explaining what was going on. We had a thread about it in the forum, which few people read. We had the blog posts. Which few people read. We lack (or lacked) a unified source of information about current events that we can assume people to read. The blog could and should be exactly that.

At the top right of the login screen of musdb, the two most recent blog posts from the respective region as well as from the “development” category of the blog have been shown for years. Then we turned on the “remember me” feature by default, which means that people only very rarely see the login page at all anymore.

The first page most users see upon logging in or opening musdb while logged in is the dashboard, the default subsection of which previously offered a summary of the database contents a user has access to, a tile for writing personal notes to oneself, a tile with messages from the respective regional administrators, a tile for the integration of a discourse forum, and links to the museum elsewhere on the web.

The summary of database contents and the links to the museum elsewhere are certainly useful. The other features not so much. Checking their actual use revealed that barely anybody used any of the note-taking features (likely also because musdb itself offers better alternatives elsewhere), while the discourse integration has not been in use for years. The very first features one sees when opening musdb were thus largely unused, wasting space that could be filled with a feed of relevant blog entries.

And so we removed the unused features and replaced them with a more prettily designed feed. This feed now contains the two newest blog posts from the development feed in the user’s language, the regional or national feed (again in the user’s language) as well as - importantly - the English-language development feed. None of the most recent development-related posts were translated to any language other than the original English, mainly because the time was better spent trying to alleviate or fix the issues than describing them in yet another language. Besides, most people know enough English to grasp the posts. And for those who do not: Community contributions to the blog - also translations for those who do not - are always welcome.

The dashboard in musdb now features a feed of recent news relevant to the development of museum-digital and whatever is going on regionally. The posts are sorted chronologically.

info

AI Use

(Post image generated using Z-Image)

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

Cleaning Out Our Closet

2025-12-22T00:00:00+00:00

Since the last post (i.e. the update to PHP 8.5 amid an onslaught of AI scrapers) and the later introduction of much stricter per-IP rate limiting, the stability issues around md are better - but they are not yet completely resolved.

As such, we have expanded our efforts in rewriting and reformulating key resource-intensive functionalities for increased stability. Different from before, we have also started to fully remove or disable functionalities that are simply not tenable anymore under the current conditions.

PDF Generation

Thus far, there were two basic types of PDFs that were generated (on the server side) in museum-digital’s portals: PDF representations of object pages (“data sheet”) on the one hand and PDFs encapsulating all images of an object in one document for easy printing.

The latter was - simply by nature of its envisioned task - extremely resource-intensive. All image files had to be loaded from disk, embedded into the PDF, compressed and served. The option had thus been available for fewer and fewer objects. Where it was originally available in case of any object with more than three images, it was later limited to objects of less than 40 images. As such, its availability was increasingly hard to communicate clearly, while its usefulness was relatively reduced with the introduction of a new download option for all images of an object. Its natural resource-intensiveness remained a problem however, and as scrapers will click any link they can find, this type of PDF generation continued to be used quite regularly (every few seconds before the recent surge in bot activity). As of last week, the functionality has been entirely removed.

The “data sheet” PDF generation has been further limited as well. As stated in the previous blog post, its usefulness is significantly reduced with the introduction of a print stylesheet (you will get better results simply pressing CTRL + P on an object page and printing the page to PDF). Nevertheless, it remained rather popular and has not been removed entirely. To reduce its impact on server stability, we however further limited its availability: If the server load is any higher than comfortable, the PDF will not be generated and an error message will appear. If the load is high (up from around 70% of comfortable) and the user’s browser language is not the default language of an instance of museum-digital, the same error message will appear.

Failed Search Pages

If a search query for objects fails, users are forwarded to a failed search page, on suggestions for alternative search queries are made. This is essentially the same as Google automatically suggesting corrections when search terms contain typos. Identifying the alternatives and offering previews for each is not free. As it is simply suggestions, the benefit or general accuracy of the suggestions fluctuates from case to case.

Now, looking at the logs, we had a large number of queries for non-existing entities - obviously scrapers who were trying out different IDs after analyzing the URL scheme. Each of those queries was executed and then forwarded to the failed search page, triggering the loading of suggestions and previews and thus further using resources on the server for little benefit (besides getting more links to scrape). We have now introduced a similar logic to the limitations on the data sheet PDF generation. Suggestions and previews are only generated when server load is comparatively low, with non-primary language users being slightly disadvantaged vis-a-vis primary-language users in an instance.

Timelines

Timelines remain popular - and a problem. A very common type of query we would see in our logs would combine timelines with searches by start and end date. This was likely due to another possible loop of endless URL generation for scrapers - specify a timeline until it forwards to search pages for a given timespan, then open the timeline for that timespan. Exactly that behavior has now been made impossible. If a search by a timeline (“start after”, “end before”) has been set, timelines will not be offered in the sidebar anymore. Trying to generate them for such a search using URL manipulation or the API will return an error page.

Search: Cleanup, Image Search & Checking Entity Existence Early

A more messy way of optimizations hit the core of the object search. In around 2021, we introduced a new search logic. Almost all pages relying on the core search logic - search overview pages, maps for objects, timelines, were adjusted to work with the new logic. The only exception from this was the image search. Still, as the new search logic re-used some of the old search logic’s functions, we kept both as separate classes, which grew over time. Simply loading the new search logic took about one ms (without OPCache enabled, measured through PHPBench). This sounds like little, but hints at a lack of modularization of the code and gains relevance with many unpredictable requests with servers automatically spinning up and down.

And indeed, in writing the new search logic, we did not modularize thoroughly HTML generation, query building and database querying. With last weeks updates, there are now separate classes for each of these and functionalities relevant only to the old search functions have been moved to class managing the image search logic. This reduces startup time for only the new / main search logic by about half (ca. 0.6 ms).

Second, we reduced the available search options for image searches. The remaining search parameters are either those actually relevant to the images or those linked to the controlled vocabularies. As a positive side effect, this also solves some issues in communication: Making it legible what the difference between searching images by their own license and by the license of (unrelated) metadata of objects the images are linked to is, is complicated.

Finally, as stated above, the logs revealed a lot of queries for objects linked to e.g. either entirely non-existent places or places that are not linked to any object in the instance of museum-digital altogether. When a place or tag is queried, we hence check whether there exists any public mention of the entity in the current instance of museum-digital during query building. If there is no link at all, it is clear early on that a more detailed (i.e. costly) query combining the search by that entity with other parameters will not return any results.

The Current Situation

All these improvements help, but a look at the current real-world numbers is warranted. On the one hand, the database server now often falls down to half or even less of the expected server load. This is a positive sign for system stability outside of peak times.

On the other hand, there are noticably spikes in the morning (around 10:20 in Germany) and in the afternoon (starting around 5 p.m.). The spike in the morning is likely related to the start of workdays and has led to the server falling over multiple times last week. This can likely be fixed only with a further tuning of the PHP-FPM settings. The spikes in the afternoon and early evening on the other hand remain hard to explain, but are altogether much less critical.

We’re on it.

info

AI Use

(Post image generated using Z-Image)

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

Updates, AI scrapers, and Resilience

2025-12-09T00:00:00+00:00

Between Thursday last week (November 27th) and yesterday (December 6th), museum-digital has seen its most unstable week in about four years. Now that the dust has settled a bit, there’s finally some time to discuss what happened and how we managed to tackle the multiple issues leading to the (very noticeable) instability.

Background

Scrapers

There were (or are) two factors simultaneously pushing our servers to their limits and requiring changes. On the one hand, scraping of museum-digital has gotten even more aggressive. Where we usually has something around 10-30 requests per second across all of museum-digital a year ago, we had around 300 two weeks ago. Right now it’s often between 500 and 700. This number excludes any access to static files.

As I’ve written elsewhere, the scrapers are mostly noticable by coming from IP ranges in Asia or (to a lesser extent) the US. On the other hand, the IPs change constantly and user-agents etc. resemble regular users. Likely they simply use an actual chrome browser for scraping. Which is to say, attempting to block them is futile. Worse yet, attempts to block scrapers would likely also impact some real users.

Fortunately museum-digital is run on dedicates servers paid by time rather than by compute. The onslaught of scrapers thus has no financial impact on us. But the scrapers still use resources, and as they try to scrape as many different pages as possible, it is much harder to optimize for them than it is to optimize for actual human users (see this article on a similar issue at Wikimedia).

Either way, AI scrapers can result in improvements. Viewed positively, they essentially act as a free stress test on a service and enforce efficiency in all aspects. If most pages are optimized for performance already, scrapers will find the unoptimized ones and bring down a service by overusing those. Which is to say, they help to identify yet unoptimized scripts/pages/classes and enforce that necessary changes are made. At museum-digital, there are three main weak spots that are hard to optimize: timelines, image manipulation (including the IIIF API), and PDF generation.

PHP

On November 20th PHP 8.5 was released. Thus far, museum-digital had been running on PHP 8.3 for web hosting and PHP 8.4 on the command line. When we attempted to update to 8.4 last year, the server fell over. This was mainly caused by the IIIF API (and thus, image manipulation via libvips).

Dependencies at museum-digital are (like pretty much universal with PHP) handled using the package manager composer. Setting up a new instance of museum-digital, composer (managed on version 8.4) required PHP 8.4 or later to run - the new instance was thus unable, being stuck on version 8.3 for hosting.

That leaves two options: Either to set up composer using PHP 8.3 again, or to simply update everything to the current version. While PHP 8.3 will be supported until 2027, it is generally advisable to update when possible. So updating it was.

Importantly, PHP at museum-digital is run via PHP-FPM. Before the update, we had one socket running per subdomain. This means, that if a PHP process serving the frontend stopped working for any reason, users in musdb were impacted as well.

Upgrading PHP to version 8.5

Once we upgraded the PHP version to 8.5 on Thursday, the same problems we faced with PHP 8.4 appeared again. The server would run rather smoothly for some hours, then more and more PHP processes would die and PHP-FPM would fall over for a given subdomain, and users would get a 504 gateway timeout error. Again, the IIIF API and image manipulation were the main causes of PHP-FPM getting stuck. Of course, the number of AI scrappers continuing to use the site did not help.

PHP-FPM settings

A natural first point to consider was the configuration of PHP-FPM. PHP-FPM knows three basic modes for running an application:

ondemand You define a maximum number of processes the application may use. When a new request is made, idle processes get used. If there is no idle process, PHP-FPM starts a new one. After a specified number of requests or a given number of seconds, an old process is closed. This is primarily aimed at being able to scale way down - if there is no requests, there will be no processes (which is to say, less resources used). On the other hand, starting new processes takes time.
static You define a number of processes that should always be running for the application. This means that there should always be processes already started and ready for usage, but it also means that those processes take up resources even when they are little used. Which is to say, this is useful if one has a high and constant stream of users.
dynamic You define a maximum number of processes, as well as how many processes should be always running for immediate use, and a (minimum and maximum) number of spare processes to keep running. PHP-FPM then manages if more processes should be started or if one of the already running ones shall be used. This, in theory, is useful if one wants to reliably and quickly serve users, expects some use all the time, but wants the server to dynamically scale up and down as needed.

With museum-digital spread out over around 80 subdomains, we had thus far used the ondemand mode for most subdomains. Only the largest and most used instances / subdomains of museum-digital were run using dynamic mode. With the update to PHP 8.4 and then 8.5, the behavior of the ondemand mode seems to have changed. If one process dies, the whole subdomain goes seems to go down with it (I have not found a documentation on this, but it’s evident from the last two weeks).

We hence moved critical subdomains impacted by the errors (which is to say, any “regular” instance of museum-digital) to dynamic mode. As dynamic mode enforces stricter limits on how many processes can be run respective to the available hardware (which is to say, dynamic mode requires a better-written configuration), this also meant that we needed to adjust the specified numbers of processes per subdomain according to their use.

To actually grasp real use of a subdomain including bots, we turned to the logs we keep for about a week (and then rotate out). In server logs, usually one line corresponds to a single request. With a small script, we loop all the different subdomains and check how many requests were made. To be really sure that only requests to relevant PHP scripts are processed, we filter them by the presence of the substring “php” before counting. The result for today between 1 a.m. and 4 p.m. looks as follows:

| Requests count in instance                         |      Total |      musdb |        PDF | 
| -----                                              |      ----- |      ----- |      ----- | 
| agrargeschichte.museum-digital.de                  |     341508 |       1245 |        719 | 
| bawue.museum-digital.de                            |     454228 |      12559 |       6819 | 
| bayern.museum-digital.de                           |     176291 |          0 |        158 | 
| berlin.museum-digital.de                           |     223280 |      14917 |       6814 | 
| brandenburg.museum-digital.de                      |      63286 |       6927 |       3873 | 
| bremen.museum-digital.de                           |     221208 |          0 |       2026 | 
| bund.museum-digital.de                             |        261 |        167 |          5 | 
| collectors.museum-digital.de                       |     108398 |        449 |        648 | 
| hamburg.museum-digital.de                          |      35489 |          0 |         11 | 
| hessen.museum-digital.de                           |      50932 |       7962 |       2486 | 
| meckpomm.museum-digital.de                         |      94177 |         11 |        139 | 
| nds.museum-digital.de                              |     137703 |       4105 |       4134 | 
| owl.museum-digital.de                              |     427667 |       1258 |       2412 | 
| rheinland.museum-digital.de                        |      64838 |       1753 |       1276 | 
| rlp.museum-digital.de                              |     207944 |       7405 |       7532 | 
| sachsen.museum-digital.de                          |     120931 |      16117 |       6034 | 
| saarland.museum-digital.de                         |        210 |          0 |          1 | 
| smb.museum-digital.de                              |     228542 |          0 |      11517 | 
| sh.museum-digital.de                               |      21098 |          0 |         48 | 
| st.museum-digital.de                               |     317913 |       6243 |       6217 | 
| thue.museum-digital.de                             |     117893 |          0 |        495 | 
| westfalen.museum-digital.de                        |     101584 |       2033 |       3310 | 
| br.museum-digital.org                              |      43413 |          0 |         16 | 
| jateng.id.museum-digital.org                       |        211 |          0 |          0 | 
| jatim.id.museum-digital.org                        |      23410 |          0 |        159 | 
| lazio.it.museum-digital.org                        |        295 |          0 |          0 | 
| ma.pl.museum-digital.org                           |        385 |          0 |          0 | 
| noe.at.museum-digital.org                          |     906386 |          0 |        369 | 
| tirol.at.museum-digital.org                        |        537 |          0 |          7 | 
| vbg.at.museum-digital.org                          |         96 |          0 |          0 | 
| wien.at.museum-digital.org                         |     472305 |        586 |       3243 | 
| ulster.ie.museum-digital.org                       |      28869 |          0 |          2 | 
| connacht.ie.museum-digital.org                     |        392 |          0 |          0 | 
| va.srb.museum-digital.org                          |       5599 |          0 |         22 | 
| ko.rou.museum-digital.org                          |       9036 |        635 |        567 | 
| mm.rou.museum-digital.org                          |        235 |          0 |          0 | 
| ca.usa.museum-digital.org                          |       3946 |          0 |          0 | 
| ma.usa.museum-digital.org                          |        357 |          0 |          0 | 
| ny.usa.museum-digital.org                          |      19576 |          0 |        294 | 
| syddanmark.dk.museum-digital.org                   |        675 |          0 |          9 | 
| de.pt.museum-digital.org                           |       1241 |          0 |         29 | 
| zh.ch.museum-digital.org                           |     233280 |        512 |        650 | 
| ba.hu.museum-digital.org                           |      99927 |       1901 |         72 | 
| be.hu.museum-digital.org                           |     100830 |        244 |       3005 | 
| bk.hu.museum-digital.org                           |     489446 |         55 |       3985 | 
| bu.hu.museum-digital.org                           |     213616 |       6206 |       5753 | 
| bz.hu.museum-digital.org                           |     598550 |        680 |       1788 | 
| cs.hu.museum-digital.org                           |      88585 |          0 |       1054 | 
| fe.hu.museum-digital.org                           |     199812 |          7 |        215 | 
| gs.hu.museum-digital.org                           |     216680 |       4215 |        912 | 
| hb.hu.museum-digital.org                           |      61250 |          0 |         65 | 
| he.hu.museum-digital.org                           |      26312 |          7 |         26 | 
| jn.hu.museum-digital.org                           |      11970 |          0 |        131 | 
| ke.hu.museum-digital.org                           |     370219 |       2959 |       1680 | 
| no.hu.museum-digital.org                           |     119487 |          0 |       1545 | 
| pe.hu.museum-digital.org                           |     603846 |       2957 |       1446 | 
| so.hu.museum-digital.org                           |     308116 |       6151 |       6698 | 
| sz.hu.museum-digital.org                           |        116 |          0 |          0 | 
| to.hu.museum-digital.org                           |      52406 |          0 |       1229 | 
| va.hu.museum-digital.org                           |     184231 |       2839 |       1666 | 
| ve.hu.museum-digital.org                           |    1015509 |       3672 |        296 | 
| za.hu.museum-digital.org                           |        199 |          0 |          6 | 
| ce.cz.museum-digital.org                           |          3 |          0 |          0 | 
| ccc.cz.museum-digital.org                          |         17 |          0 |          0 | 
| academia.hu.museum-digital.org                     |       9158 |          0 |         13 | 
| cherkasy.ua.museum-digital.org                     |      25567 |          0 |         26 | 
| chernihiv.ua.museum-digital.org                    |       3258 |         99 |        156 | 
| dnipro.ua.museum-digital.org                       |      26725 |          0 |        109 | 
| donetsk.ua.museum-digital.org                      |         17 |          0 |          0 | 
| ivfr.ua.museum-digital.org                         |        722 |          0 |          9 | 
| kharkiv.ua.museum-digital.org                      |      12932 |          0 |         39 | 
| kyiv.ua.museum-digital.org                         |     436482 |       5967 |       1351 | 
| kyivska.ua.museum-digital.org                      |       2159 |          0 |         79 | 
| lviv.ua.museum-digital.org                         |     163358 |        188 |        274 | 
| poltava.ua.museum-digital.org                      |       7657 |        284 |          3 | 
| odesa.ua.museum-digital.org                        |         93 |          0 |          1 | 
| rivne.ua.museum-digital.org                        |      59510 |         65 |        156 | 
| sumy.ua.museum-digital.org                         |      35890 |        303 |          3 | 
| ternopil.ua.museum-digital.org                     |     150700 |         37 |        184 | 
| zhytomyr.ua.museum-digital.org                     |          3 |          0 |          0 | 
| vinnytsia.ua.museum-digital.org                    |      14229 |          0 |          0 | 
| volyn.ua.museum-digital.org                        |      16705 |          0 |        485 | 
| zakarpattia.ua.museum-digital.org                  |       2865 |          0 |         30 | 
| zaporizhzhia.ua.museum-digital.org                 |      24348 |        338 |         56 | 
| scotland.museum-digital.org                        |          0 |          0 |          0 | 
| md.museum-digital.org                              |          0 |          0 |          0 | 
| demo.museum-digital.org                            |         12 |          2 |          0 | 
| goethehaus.museum-digital.de                       |     260072 |          0 |         85 | 
| lmw.museum-digital.de                              |     326724 |          0 |         65 | 
| gedenkstaetten.museum-digital.de                   |       3474 |          0 |          0 | 
| turcica.museum-digital.de                          |      75533 |          0 |          1 | 
| nat.museum-digital.de                              |    1238860 |          0 |       4657 | 
| at.museum-digital.org                              |     631578 |          0 |         89 | 
| cz.museum-digital.org                              |          2 |          0 |          0 | 
| dk.museum-digital.org                              |       5415 |          0 |          4 | 
| hu.museum-digital.org                              |     359619 |          0 |       2827 | 
| id.museum-digital.org                              |       8030 |          0 |          0 | 
| ie.museum-digital.org                              |       2073 |          0 |          0 | 
| it.museum-digital.org                              |         78 |          0 |          0 | 
| rou.museum-digital.org                             |       8277 |          0 |        466 | 
| pl.museum-digital.org                              |        142 |          0 |          0 | 
| pt.museum-digital.org                              |          0 |          0 |          0 | 
| srb.museum-digital.org                             |        565 |          0 |          0 | 
| ua.museum-digital.org                              |     232115 |          0 |        805 | 
| usa.museum-digital.org                             |       3752 |          0 |         34 | 
| ch.museum-digital.org                              |      53417 |          0 |          1 | 
| global.museum-digital.org                          |     727690 |          0 |       2199 |

Note that the number of requests obviously is also impacted by bots changing attention - once a scraper is done with one subdomain, they turn to the next. The elevated number of requests in ve.hu.museum-digital.org is normal, but still starkly exaggerated when compared to other days. The Germany-wide instance is persistently the most frequented one, usually the global one is second at around 80% of requests.

Now equipped with actual numbers, we could scale the PHP-FPM to a much more suitable configuration than before (we had thus far never bothered counting actual requests, instead relying on the number of objects).

A second step in the PHP-FPM configuration was to reduce the impact the problems had. Previously there was one shared configuration and socket per subdomain. On the one hand, this meant that stuck processes in the frontend impacted users in musdb (and vice-versa). On the other hand, some constraints on resource usage cannot be set on a per-directory level but must be set per PHP-FPM socket / server (see the PHP documentation on user.ini and the list of php.ini directives). As the frontend and musdb have different requirements (frontend: low maximum memory use, short timeouts, no file uploads, generally strict settings; musdb: long timeouts for uploads, generally more lenient), being able to configure them independent of each other is useful in general.

We thus separated the configuration for the frontend, musdb, and PDF generation in the frontend; providing dedicated sockets for each. The frontend has a reduced priority on the system overall, strict constraints on how it may be used, etc. The settings are stricter than they were before. musdb has an elevated priority and more lenient settings (file uploads, longer timeouts), in fact more lenient than before. Finally, PDF generation is a special case as it offers no real benefit over the browser’s print tool (see MDN on print CSS), while being resource-intensive. As such, it has a far reduced priority and very strict settings.

With the separated configuration and sockets, we can now better tailor the configuration to each application’s needs and have the added benefit of problems in one application not impacting the other.

Code

As we had already prepared the codebase for PHP 8.4 awaiting an eventual upgrade, the upgrade to PHP 8.5 only required minimal changes. Aside from the deprecation of the functions finfo_close() and curl_close(), references to which were accordingly removed from the code, the update necessitated no further work.

Scaling in Software

Improving the PHP configuration was not enough to fix the issues, especially with the now increased number of requests from bots. To get some breathing room, we adjusted the most resource-intensive pages.

Frontend

In the frontend these are, again, the IIIF API, PDF generation, and timelines. Finally, we made changes to the pages for failed searches to better handle high load situations.

Image pages

The IIIF API was used for the main image pages in the frontend. We used (and use) Mirador as a IIIF viewer. Simply opening an image page thus meant three requests to fetch different regions of an image. Zooming into the image triggered further requests to fetch the relevant parts of the image. Cropping the image to the requested region with IIIF happens on the server (which is no problem if there are few users, but is turning into a problem when you have hundreds of requests per second).

We thus changed the default of image pages: The new default image page is the old, non-IIIF one. As features like zooming into images, that Mirador comes with, are popular and useful and the old image page did not support those, we worked to improve the page. To do so, we rely on OpenLayers, a library we already use for maps. Besides including maps from tile servers, OpenLayers also supports loading simple image files - which we do here. The image is hence loaded once in full size and zooming etc. happen entirely in the browser.

Taking the opportunity, we improved the page overall. An often noticed problem of image pages thus far was, that users who opened image pages coming from external services (think Google Images) had problems identifying that the image was an object image and that there is further object data to be found on object pages. The updated image pages now come with a header stating reflecting the name of the image, the name of the object and the name of the institution. Note that many images do not feature a dedicated title, musdb uses the object name as a default image title in that case, which is why the object title will often appear twice in the header. Maybe this can be used as an encouragement for the colleagues working in musdb to more consistently set expressive image titles in the future.

Also new is a mini map at the bottom left, displaying where in the wider context of the image one has currently zoomed in, as well as the ability to link exactly the region one has currently zoomed into. To enable the latter, the URL updates as one zooms or navigates around the image. Somebody else opening the same URL will then open exactly the same image region the linking person was viewing when copying the URL. Finally, we finally set specific Content Security Policies relevant to the currently opened media. If the displayed media entry is an internally stored image, no external images need to be allowed to load. If the displayed media entry is an audio file stored on archive.org, archive.org needs to be whitelisted as a source for audio files - but only archive.org and no other page. Previously, embedding images from anywhere on the net was allowed, increasing the potential damage a potential attacker may cause.

Making the use of Mirador a secondary, non-default option reduced the need for server-side image manipulation and the corresponding resource use significantly. The IIIF remains largely unchanged, but its use must now be requested explicitly.

PDF generation

As stated above, PDF generation brings little advantages to the browser’s print functionality in combination with object pages. On the contrary, the PDFs generated using the frontend’s templates feature less information. But they come with the file ending “.pdf” and seem to be extremely popular with bots. On the other hand, PDF generation means, among others, loading whatever images are to be embedded into the PDF and manipulating them fit into the PDF. The resulting files are significantly larger than the corresponding HTML files and thus also use more of the available bandwidth.

The update to handle PDF generation respective to resource usage was already introduced in the last months: publicly linked PDFs are now only generated if overall load on the server is low, if a user has set their browser language to any language different from a museum-digital instance’s default language. As most scrapers do not bother to change their browser language (which means they come with either none, English or Chinese), this means they will mostly be unable to trigger the generation of PDFs. They see an error page instead.

Failed Search Pages

If a user tries to execute a search query without any results, they will get suggestions for similar search terms - similar to how Google will ask one searching for “Berrlin”, if they meant “Berlin”. Trying to identify suitable suggestions obviously costs resources and whether the suggestions are actually what a user wanted is by nature hit or miss - it’s suggestions after all. In the case of scrapers, suggesting alternative search queries offers them a never-ending stream of possible search queries to run and keep scraping the subdomain with - to nobody’s benefit (not even the scrapers’, as they likely got the same content with other search queries already).

We thus now use the same function used to identify whether PDFs should be generated for a user to check if search suggestions should be provided. It a user comes with a non-default browser language and resource use is high, no suggestions will be provided.

Timelines

Timeline pages as implemented in museum-digital’s frontend offer another source of endless links and search queries, as they link to further and further specifications of the time searched by. Again, an improvement already introduced months ago, was to better parse queries by time: If a user searches for objects that are linked to times “after 1920” and “after 1930”, the latter already includes the former. “After 1920 and after 1930” means exactly the same as “after 1930”. Which is one join instead of two - half the resource usage.

A minor improvement we noticed on the side was impact of automatic redirects in the timelines. Say, a user searches objects by their link to a given tag and then generates a timeline for said objects. If all objects were created in the 20th century, the timeline will automatically redirect so as to “zoom” into a more appropriate time scale than from the big bang to now. Until the last weekend, script execution was not stopped when that redirect happened - which means that all database queries for time time from the big bang to now were still executed even though the user never got to see them. That is now fixed.

The Anti-Climactical Solution

All of those changes got the frontend more or less stable. Problems with uploading images remained however. Finally, the only thing that helped was uninstalling libvips (which we use for image manipulation) and reinstalling it. That seems to have fixed the issues.

Especially as the number of requests from scrapers continues to increase, the current strategy outlined above seems to be fruitful. By reducing the use (and sometimes the availability altogether) of especially resource-intensive and - depending on the context - little useful functionalities, much stability and can be gained.

The update seems to finally be largely completed (aside from maybe some further fine-tuning of the PHP-FPM configuration) and museum-digital is stable despite the bot problem, while we haven’t had to take more drastic or costly actions yet - such as blocking or adding additional servers.

info

AI Use

(Post image AI-generated using Z-Image)

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

State of Development, November 2025

2025-12-03T00:00:00+00:00

Frontend

On source / reference pages, linked objects are now sorted by the position within the source work on which they are referenced or which they do themselves reference
The target URL of the regular / unspecified search bar for objects now follows the new, prettier URL schema
Support for an OAI-PMH API for object metadata
- Standardized endpoint for aggregators seeking to retrieve data in batch
- Metadata formats thus far supported:
  - LIDO
  - OAI-DC (mandatory)
- See also: Blog
PDFs are only generated for users with a browser set to a non-default language if load on the server is low
- The resource use caused by AI bots scraping museum-digital has been growing and growing. Generally, we see bots included in our mission to enable access to cultural heritage. On the other hand, nobody is served if the service is bogged down by bots. One functionality that is commonly used among bots and resource intensive is the generation of PDFs for object pages. The same information can be loaded from the object page itself and printed to a PDF using the browser’s print option. There are thus rather few downsides to limiting access to PDF generation to timmes, when server load is low. So that’s what we did.
Collection-specific ISIL identifiers are now also used in the LIDO API
Alternative numbers of an object can now be displayed on object pages
- This includes tooltips for types of alternative numbers, that can be set by the museum on the institution-wide settings pages of musdb

musdb

Search for objects
- Type-ahead search for languages (of the object’s content)
- Search by object’s revision status (open, read-ony, archived, etc.)
Batch editing of objects’ revision status
Parameters of the full text search index were updated to improve the search of word compounds in German

Importer

Core

The dry-run mode now does not abort an import anymore, if an unmapped value is encountered. Unmapped entries are collected and displayed together afterwards.
- This means, that unmapped entries can now much more easily be copied to and mapped in concordance.museum-digital.org
Support for the import of alternative numbers (of objects)
Support for the import of space hierarchies

Parser

AdlibXml
- Support for importing objects’ alternative numbers
CsvXml
- Support for importing objects’ alternative numbers
CsvLocations
- New parser for csv-based imports of space hierarchies
ImageByInvno
- New setting: append_chars (Adds suffixes, that exist in the inventory number, but not in file names)

info

AI Use

Post image generated using AI.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

State of Dev, September 2025

2025-11-25T00:00:00+00:00

Development

Frontend

Objects that are linked to a source as referencing the source or being referenced in it are now listed on the source’s page
- Example: “Novalis Schriften. Die Werke Friedrich von Hardenbergs. Historisch-kritische Ausgabe. Erster Band: Das dichterische Werk. 3. Auflage”
The object page now displays the new transcription fields for notes, status, and type of the transcription
New types of classifications for the link between an object and a tag
- Taxon
- Topic
- Mentioned subject (similar to the existing “display subject”)
Dependencies
- Updated OpenLayers to Version 10.6

Source pages in the public frontend / portals of museum-digital now display all objects linked to the source as either referencing the source or being referenced by their source.

musdb

Renaming vocabulary entries to blacklisted terms is now impossible
- Someone managed to create an actor “unknown” by adding a different actor and renaming the new actor entry to “unknown” afterwards.
Added a sidebar to object group overview pages for additional filtering options
New types of classifications for the link between an object and a tag
- Taxon
- Topic
- Mentioned subject (similar to the existing “display subject”)
New APIs for listing all vocabulary entries linked to the objects of a museum
Dependencies
- Updated OpenLayers to Version 10.6

Importer

Core
- Deaccessions are now covered by the import tool
- Recipients of deaccessions are linked to the address book / contact list
Parser
- CSVXML: The CSVXML parser now covers deaccessions as well
- ImageByInvno: Added an option to ignore any characters before a given starting combination of characters (if there is a consistent start of the inventory number).

CSVXML

New fields
- tag_related_identifier_type
- tag_related_identifier

Dissemination

Presentation “Von museum-digital zum eigenen Online-Katalog” for the Working Group Digitization of the State Museum Association of Rhineland-Palatine
- Slides: PDF
- Slides: ODP

info

AI Use

Post image generated using AI.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

State of Development, October 2025

2025-11-25T00:00:00+00:00

Development

Frontend

Significantly reworked the display of transcriptions on object pages
- Titles of transcriptions are now displayed
  - If none is set, the type of the transcription (original or translation) is used as a replacement
- Transcriptions are sorted by their titles
- Improved the display of transcriptions in tiles
  - Problems with vertical scrolling are now solved
  - If only one transcription has been recorded, it will be displayed on the full width of the page
  - If there are more than two transcriptions for an object, they are folded in by default and can be unfolded later on
Batch export of object metadata via the API
- Thus far available in JSON & LIDO
- API documentation
- See also
Dots as a separator in floating point numbers for object measurements are replaced with a comma in languages that require that
Collection-specific ISIL IDs are used in the LIDO API

musdb

Added a field for recording titles / names of transcriptions
Added the option to set collection-specific ISIL IDs
Setting object type tags via the improvement suggestions now correctly classifies the thus created link between object and tag
Additional shapes are now available
- E.g.: round, square
Object groups can now be filtered by whether they have a superordinate one or not

Dissemination

2025-10-08: Presentation at the Autumn Conference of the Working Group Documentation of the German Museum Association: “Interoperabilität schaffen - Geschichten aus 1001 Importen”
- PDF
- ODP
2025-10-14: Talk on a workshop of the project CiVers (Citation of Versioned Web Pages by PID)
- PDF
- ODP
2025-10-17: museum-digital Usertagung 2025

info

AI Use

Post image generated using AI.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

Making Interoperability Easy

2025-11-24T00:00:00+00:00

Interoperability has been one of the focal issues around museum-digital practically since its inception. Offering different, simple ways to bring data into the system was a necessary requirement to even think of what we do. And offering simple ways to get the data out of the system again is just good practice - though all too often neglected.

To that end, there have traditionally been two primary ways for data retrieval. In musdb, one could run batch exports and receive a ZIP with some form of XML files. One per object, with the objects matching the results of any given object search.

On the other hand, there is the public API. Using URL manipulation, one can access the (primary) contents of each page in a machine-readable way. To access the JSON representation of an object’s published metadata, where the object’s ID is 7141 in the Hesse instance of museum-digital (URL: https://hessen.museum-digital.de/object/7141), one simply has to insert json to the path: <a href="https://hessen.museum-digital.de/json/object/7141">https://hessen.museum-digital.de/json/object/7141</a>.

Next to the default JSON output, additional APIs are offered wherever suitable for a given data type. For objects, the primary additional output method is a LIDO API.

Thus far, the main limitation of the public API was that it only allowed one object (or institution, collection, etc.) to be queried at a time.

Querying Object Metadata in Batches

After a significant refactoring of the code to load object data for object pages - primarily to improve caching and allow for parallelized requests to the database - we are now finally able to offer APIs for querying object metadata in batch. Thanks to grouped database queries, performance and resource usage scale nicely. Taking simply the currently most recent objects in the Germany-wide instance of museum-digital: Loading all object data of one object and presenting it in JSON takes 0.0087 seconds and loading and generating the JSON for the 100 most recent objects takes 0.197 seconds (or 0.00197 per object). Note that not all queries for all aspects of an object’s metadata are grouped yet, performance may thus get even better over time. This does also not yet account for the overhead of the many HTTPs requests one would previously need to execute to get each object’s metadata one by one - real performance improvements are thus even greater.

Now, how to access object metadata in batches?

The batch access is linked with the search API and reuses its main query parameter (“s”). Say, if one is searching for objects related to Berlin (a.k.a. the place of the ID 61), the URL of the respective search page would be https://global.museum-digital.org/objects?s=place%3A61. The corresponding API for retrieving all of the objects’ published metadata would then be https://global.museum-digital.org/export/json/place:61?limit=24&offset=0. Like the search page itself and its primary API (/json/objects), the full batch export API is paginated with a maximum of currently 100 objects per page being returned.

Additional to URL manipulation, the batch export API is linked in the menu of object search results pages.

Currently, the batch export of full object metadata is available for JSON and LIDO (XML) representations of the object data. More can rather easily be added later, should a demand arise.

OAI

Implementing a performant way to export full object metadata in bulk was one of the two main missing components for the long-missing implementation of an OAI-PMH API.

OAI is a standard tailored towards data harvesting. Say, an external service like the German Digital Library or Worldcat wants to do something with external data from diverse sources, e.g. to also display objects or implement a search across the different collections / libraries to find which one has which object / book. To be able to do so, they need to be able to access the respective data in some way. Ideally, using a common standard that describes how to query data, helps to identify any data sets that need to be updated or added (or deleted), and finally presents a uniform way to access the data periodically. That, exactly, is OAI-PMH.

In a nutshell: OAI-PMH allows other services to copy all (published) data from another service in a maschine-readable way and can thus significantly improve reuse in aggregation. Of course this only applies to technical questions; legally, potential re-users need to comply with the metadata license applied by the initial data provider regardless of the (technical) means of access.

Since last week, museum-digital now provides a OAI-PMH API at /oai respective to a given subdomain. E.g.: https://hessen.museum-digital.de/oai. As of now, the OAI-PMH API provides access to the objects’ metadata using LIDO (XML) and the mandatory OAI-DC format.

Note that there are some caveats remaining for now: First, the LIDO representation of object metadata is not (and can by definition not be) as complete and fine-grained as the JSON API. It is also not exactly similar to the LIDO as returned by exports from musdb (one is formed natively in PHP, the other using XSLT, leading to divergent development paths). Also, the LIDO output lists different identifiers from the ones used by the OAI-PMH API and the OAI-DC representations otherwise.

Finally, the OAI-PMH API at museum-digital does not implement OAI-PMH data sets to group collections. Instead, it follows the existing search logic (essentially providing a new endpoint per query). Example searches via OAI might thus look as follows:

All objects from the Agrargeschichte instance of museum-digital, represented in OAI-DC: https://agrargeschichte.museum-digital.de/oai?verb=ListRecords&metadataPrefix=oai_dc
All objects linked to Berlin (place #61), in the Berlin instance of museum-digital, represented in LIDO: https://berlin.museum-digital.de/oai/place:61?verb=ListRecords&metadataPrefix=lido
All objects of the Freies Deutsches Hochstift, Frankfurt am Main, represented in LIDO: https://hessen.museum-digital.de/oai/institution:1?verb=ListRecords&metadataPrefix=lido
All objects from Baranya with only their identifiers: https://ba.hu.museum-digital.org/oai?verb=ListIdentifiers&metadataPrefix=lido

Credits

Credits where credit is due: the Städel Museum deserves praise for their implementation of OAI-PMH. For me personally, seeing the Städel’s API was the first time I saw a visibly stateless OAI-PMH implementation, enabled by the ingenious idea of using machine-readable, JSON-encoded resumption tokens over UIDs that have to be resolved on the server side. I spent weeks (or months?) making museum-digital’s frontend stateless. By following the Städel’s example, it can remain so even while offering an OAI-PMH API.

info

AI Use

Post image generated using AI.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

State of Dev, June & July 2025

2025-08-23T00:00:00+00:00

June and especially July were at first glance once again rather slow months in terms of development at museum-digital.

Generally, the pace and type of development seems to have changed this year. Rather than doing many small improvements all over the place, there is less but larger and more labor intensive changes and new features. See for example the versioning in musdb (January), the tool for automating imports based on what others call a “hot folder” and the sort option to sort objects by their images’ aesthetics score (both presented here in March), and the new tool for suggesting formulations for object descriptions using large language models (June).

July

Frontend

Translation of the software to Hindi and Telugu
Grouping of tags by their relation to a given object
If an object is linked to more than 10 tags, the tag list of object pages quickly starts looking unorganized and messy. In such cases, the tags will now be displayed grouped by their relationship to the object (thus far: general tag, material, technique, object type, display subject)

musdb

New Features

Export option for the specific LIDO as expected by the German Digital Library’s “colonial contexts” portal

Improvements & Changes

The minimum length of fulltext search terms for object search parameters is now visibly enforced in the extended search user interface
To not overly burden the fulltext search server, any term in a full text search in musdb needs to be a minimum of two characters long. Thus far, shorter full text search parameters were simply ignored. This certainly was confusing at times. Since June, attempting to perform search queries with shorter search parameters is made impossible by a check in the extended search overlay.
Reception history of objects: Statements of the relevant position within a source can now be 40 characters long
Transcriptions
- May now be up to 4000000 long
- New fields: Notes on the transcription, status, aims

Bugfixes

Fixed a bug in the batch editing of specific visibility flags for data fields on the addendum tab

Juni

Frontend

Performance improvements
- Object search now runs without a connection to the full text search server if no full text search parameter is set
- If multiple search parameters for an earliest / latest time have been set, they are parsed and combined before being forwarded to the database
Improvements in the deletion of temporarily created PDF files (PDF export)
Navigation has been translated to Tamil

musdb

New Features

Recipient of deaccessed objects can now be linked from within the address book
New tool for AI-aided formulation o object descriptions (based on existing other metadata)

Improvements

Object search now runs without a connection to the full text search server if no full text search parameter is set

Importer

The CSVXML parser has been extended to cover new event types and markings
Object groups automatically generated to group all objects of an import can now receive a description as set within the import configuration

nodac

The list of selectable languages for the navigation of nodac has now been restricted to those in which there is actually a complete translation

info

AI Use

Post image generated using AI.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

Hindi

2025-07-13T00:00:00+00:00

After Kannada, Tamil and Telugu, museum-digital can now also be browsed in Hindi. As always, thanks and enjoy!

info

Image credits

“Devanagari_letters.svg” is licensed under Creative Commons Attribution-Share Alike 4.0 International, by बडा काजी, and retrieved via Wikipedia Commons.

info

License

This content is licensed under a Creative Commons Attribution-ShareAlike 4.0 International license..

State of Dev, May 2025

2025-07-07T00:00:00+00:00

Frontend

New Features

Tabs are available in the configuration when the site is installed as a Progressive Web App (PWA) (See also)

Improvements

Duplicate search query parameters for objects associated with times before / after are removed.
Example: If one searches for “Objects after 1900, whose first recorded associated time is also after 2000”, this is a duplicate query. 2000 is always after 1900, so one of the two parameters can be removed.
Searches by time begin / end are also the foundation for the timeline. As the timeline is both beloved by web crawlers and resource-intensive to generate, this change significantly reduced server load.
Any links on timeline pages that do not reference single object pages are marked as rel=nofollow
Which is to say that bots are told to ignore them.

musdb

New Features

Tabs are available in the configuration when musdb is installed as a Progressive Web App (PWA) (See also)
New search option for objects: “Can be published”
Aggregate search for objects that have not yet been published and which comply with the minimum requirements for publication.
Separated measurements can now be batch-edited
Available via “assign results”
Institution-wide setting to enforce the use of the user-defined object editing interface for all users of the institution
Exhibitions can now be searched via the API (/exhibition/list)
A user’s user-agent is checked whenever a page is requested. If it changed, the user will be logged out automatically.
Protection against session hijacking.

Improvements

Panorama images for tours through exhibitions / institutions are calculated down to 2400 px height rather than 1400 px height
The APIs for searching for entries in controlled vocabularies are now available via the main API
See e.g. <a href="https://de.handbook.museum-digital.info/musdb/API/index.html#/actor/actorSearchLinkedToObjects">/actor/search_linked_to_objects/{term}</a> , <a href="https://de.handbook.museum-digital.info/musdb/API/index.html#/actor/actorSearch">/actor/search/{term}</a> etc.

Bugfixes

Fix: “Visiting scientists” couldn’t open the “location” tab (this erroneously required museum-wide editing permissions)
Fix: User-defined defaults for descriptions of new objects were ignored when new objects were to be added
Fix: Thumbnails were displayed as duplicate images linked to exhibitions (tag “images”)
Fix: Prefixing via the batch editing was broken

Importer

New Features

New parser for CSV exports / imports from Robotron Daphne

Improvements

CSVXML Parser
- New literature-related fields (ISSN, editor, etc.) are now covered
- References to wikidata are now respected as tags are imported
The maximum length of any single given tag/keyword is centrally set to 95 characters

Bugfixes

Some more recently added fields from the “admininstration” tag of musdb’s object pages had been read by the importer but their transfer into the database had not been implemented thus far.

nodac

New Features

Tabs are available in the configuration when nodac is installed as a Progressive Web App (PWA) (See also)

CSVXML

Improvements

Added new literature fields: type, editor, edition / issue, ISSN

info

AI Use

Post image generated using AI.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

Browse museum-digital in: Telugu

2025-07-02T00:00:00+00:00

After we were gladly able to release the Tamil translation of museum-digital’s public portals last week, we are equally glad to do so with the all-new Telugu translation.

As always, many thanks to everyone who’s made that possible!

info

Image credits

“Telugu consonants.svg” von Psiĥedelisto, lizensiert unter CC BY-SA 4.0, via Wikimedia Commons

info

License

This content is licensed under a Creative Commons Attribution-ShareAlike 4.0 International license..

Browse museum-digital in: Tamil

2025-06-23T00:00:00+00:00

Today, we’re happy to release the Tamil translation of the public frontend of museum-digital. Thanks to everybody who’s made that possible!

As always, users with browsers set to using Tamil as their main language will now automatically see the Tamil translation of the site navigation etc. Others can select to navigate the site in Tamil using the language selector at the top right.

info

Image Credits

“Tamil Community - Worldwide.PNG” von Coppercholride, via Wikimedia Commons, licensed under Creative Commons Attribution-ShareAlike 3.0 Unported.

info

License

This content is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported.

State of Dev, March & April 2025

2025-06-08T00:00:00+00:00

Frontend

A Kannada translation of the software is now available
Museums can now select to display their own citation notes in the menu for such on object pages
This is especially relevant in case the object itself is to be cited (rather than its record online).
“Or” search queries can be combined within one search parameter for select attribute search types, e.g. places and tags. The Syntax is as follows: place:61~1
- For now, this can only be used using the internal query language. As such, there is no corresponding option in the UI for search settings. On the other hand, the search option is thus available using the API.
- This option is not available when searching for times or full events

musdb

Institution-wide settings
- Free text fields that double with similar controlled fields may now be hidden
  - The acquisition of an object may e.g. be directly recorded on the context of a given object or as a separate acquisition process. Recording it as an acquisition process is slightly more labor-intensive, but allows a more fine-grained and accurate documentation. With the new setting, the data fields for recording acquisitions directly in the object context can be hidden from all users of a museum to ensure a uniform use of the preferred functionality.
  - Institution-specific notes on how to cite objects may now be recorded for display on published object pages.
Search queries for inventory numbers are now character-level searches again (rather than following a fulltext search logic)
Some event types are incomplete, i.e. they cannot contain a place, or an actor, or a time. This incompleteness was handled differently between musdb, the import tool, and the CSVXML import preparation tool. Now, it is determined by a centralized list and thus similar across all of museum-digital.
Refactoring of the administrative command line interface
This mainly concerns auto-correction tools, but also results in that exports for the quick export option are now automatically generated daily.
Separated measurements are now positioned at the very top of the “addendum” tab of object editing pages.
It is now possible to record web links for object groups

Importer

The import tool can now be used as a harvester
- First use case is a harvester for LIDO records delivered via an OAI-PMH interface.
Externally stored images in formats other than JPG can now be imported
Significantly extended the LIDO parser to (among others) support the import of multilingual object data

info

AI Use

Post image generated using AI.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

Kannada

2025-03-25T00:00:00+00:00

There’s lots of talk about the FAIR principles - publishing findable and accessible data. Logically, to be findable and accessible, the data should at the very least be described in the language of users, even if it is not itself translated. And that means, that before object information becomes multilingual, platforms should become (more) multilingual and especially cover languages that may not be in the first line of languages covered by your next-door Large Language Model for translation.

Today we can gladly announce another large step for making museum-digital more findable and accessible: The frontend is now available in Kannada.

Users with browsers set to using Kannada as their main language will now automatically see the Kannada translation of the site navigation etc. Others can select to navigate the site in Kannada using the language selector at the top right.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

State of Dev, February 2025

2025-03-25T00:00:00+00:00

In terms of development happening around museum-digital, February 2025 was a rather calm month. While more happened in the “machine room”, immediately visible changes are mostly restricted to bugfixes. And a whole new tool.

Here, as so often, in a list format:

Frontend

Bugfix: The detailed description was not reflected in the object API even if set to be so via musdb
Translation: The frontend is now available in Kannada

musdb

Bugfix: Fixed error in setting up new two factor authentication via TOTP (app-based 2fa)
Bugfix: Symbols for image rotation switched (counter-clockwise symbol rotated clockwise and vice-versa)
Feature: If the generation of a PDF catalogue is triggered via the sidebar of series editing pages, the objects will follow their order within the object group

Importer

If no explicit name/title for a resource (audio, video, PDF, 3D, externally hosted image) was supplied, there was no fallback. Now, the importer follows the same logic as musdb and will fall back to using the linked object’s object name as a resource title if none is supplied.
The CSVXML parser can now handle multiple objects per XML file.

Auto uploader

The auto uploader is an entirely new tool. When first started, users will be asked to enter the necessary data for running imports. And they are asked for the path to a folder which will subsequently be monitored for automatic uploads.

Whenever the tool then detects suitable contents for an import in the folder, the data will be uploaded and automatically imported. Intermediate steps like the generation of a settings file for the import are covered by the tool.

The tool can thus be useful for museums that regularly import data formed in a consistent format - say, museums using museum-digital for publishing while mainting a separate collection management system. It should be uninteresting to those who spend much effort on preparing imports (e.g. via CSVXML) or who want to migrate to musdb altogether, as it only becomes useful in the case of regular use.

The code is available here, licensed under GPL v3.
See also the more detailed German-language blog post about the auto uploader.

info

AI Use

Post image generated using AI.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

Sort by Beauty

2025-03-06T00:00:00+00:00

Last month a new sort option appeared on museum-digital: “Aesthetics prediction”. Based on the LAION aesthetics predictor, each published object’s main thumbnail aesthetics are scored. The objects can then be sorted according to this score.

AI, Aesthetics and Discrimination

When working with AI it is common to criticize the inherent biases of the models used. Already underrepresented entities (people, viewpoints, etc.) are excluded, because they are not sufficiently represented in the data the model was trained on. In turn, AI repeats what it learned - pre-existing biases in society are replayed and reinforced by AI.

A second, well-founded criticism against AI applications is their impreciseness, their tendency to “hallucinate” entirely wrong results, and the lack of reproducibility of results.

Both criticisms are entirely valid. And both hint at why AI might be a useful tool exactly for generating a sort order by aesthetics. Like AI, aesthetics are imprecise: Ask 10 people to rank some pictures by their aesthetic appeal and you will get 10 different answers. But the general thrust of the answers will likely be more or less the same. There are societal rules - biases - for what pictures are more or less beautiful. But they are fuzzy and interpreted differently by each and every observer. A sense of aesthetics is about learning and reproducing those biases - learned by the inputs we learn in everyday life -, just as AI will be biased based on the data it is trained on.

Sorting then is, in essence, an exercise of discrimination. Sorting entries by ID / age in a database system will favor the newest entries and discriminate against older ones. Sorting entries alphabetically in an ascending order will favor entries whose title starts with “A” while it will discriminate against those whose title starts with “Z”. Sorting based on an AI-generated rank is discriminating. Sorting by beauty or aesthetics is as well. AI allows sorting by beauty.

What is discriminated against?

Now, if everything in sorting is discriminating, it is all the more important to consider what is actually evaluated. In other words, on what basis the discrimination takes place when ranking digitized museum objects by the beauty of their digital reproductions. As stated above, the objects are scored based on the aesthetics score established for their main thumbnails.

Images then have a range of aspects that influence their aesthetics. Image composition, lighting, contrast, the motive, and many more (an art historian could likely list hundreds). For the common critique, it is essentially only the motive that matters: Favoring images of white women over images of Asian men reproduces a range of societal problems that should not be reproduced. Transferred to object photography, the motive may be further differentiated between object type and the actual motive (e.g. of a painting). And such a discrimination is noticeable in the results: Paintings seem to be generally slightly better ranked than pictures of tools. A bias based on the displayed subjects of e.g. different paintings is not something anybody from the team would have noticed, but is to be assumed that one will exists.

On the other hand, the influence of motive-focused biases is many times weaker than the actually useful discrimination based on the technical aspects of the images. Objects images taken by a professional photographer with up-to-date equipment are ranked much better than images taken using a digital camera from the 1990s. Images without a timestamp or a watermark are ranked better than ones that do feature one. Images taken with proper lighting and contrast settings are ranked much better than images taken in a dark room, presenting the objects in gray on black. Similarly, one of the most important facets contributing to the aesthetics score the predictor returns seems to be the composition: If an object is centered in the photo taken, it will score way above images featuring multiple objects at different corners of the image (a classic example would be images that show both the actual object and a photo bar).

To reiterate, these technical aspects are visibly the main contributors to the score. And discriminating based on those images is actually useful: It allows museum-digital to present newer users who are just browsing the published collections with objects recorded with a more consistently high visual quality.

What are the alternatives, or who is the audience?

museum-digital suffers the old problem of a lack of a specified target audience. A common user may be a hobbyist looking for other versions of the model train they just bought. They might be a person interested in what art of the 16th century looked like. Or which museum to visit next. Or they might be a specialist, with a much better idea of what they are actually looking for. It is common for users to leave the page after less than a minute. But there is also an astonishing number of users who stay for hours. Accordingly, it is all the more important to offer (sort) options catering to different needs.

A new user who chanced upon the platform and randomly browses it with no further background on museums will benefit from the new sorting method. Sort orders like sorting by ID or title are linear and follow a consistent logic - but that logic may in essence just reflect its own type of randomness. Sorting newer objects over older ones is essentially a random sort order, if the museum does - as is usual - record objects as they are needed in an exhibition, newly enter the museum or are simply the next object in the shelf - all of which follow a very particular, not publicly comprehensible logic. Similarly, sorting objects by their names or titles is linear. In contrast to the object entry’s age in the database, it is also immediately comprehensible to users. But museums are free to determine the object name freely - and often this is necessary. As per the best practices around publication on museum-digital, an object name should be descriptive and usable to distinguish between different objects. As most objects simply are nameless by themselves, this often means that a colleague at the responsible museum invented a name. Seeing a green vase, there may be a rather short list of names that come to mind for most people - “green vase”, “vase, green”, “vase”, “green-ish vase”, “vase in pale green” -, making the selection of the actual name comprehensible. But “green vase” is in a very different position from “vase, green” when sorting objects alphabetically. One might actually be involuntarily be sorting the objects by curator.

Other than an object’s title/name and institution, the one other facet users see when listing objects on an overview or search page is the object’s thumbnail. And as a sense of aesthetics - fuzzy as it certainly is - is roughly shared among most people (with globalization, even actually most people), sorting objects of diverse origins by the aesthetics of their thumbnails suddenly starts to look like a comparatively relatable and reasonable sort order. Nobody will agree 100%, but the rough order instinctively makes sense. That’s more than can be said about the alternatives, unless one actually looks at the sort settings.

Presenting new users with a more consistently appealing and streamlined set of search results (at the first glance at least) on the other hand might encourage them to stay for longer. And if they stay longer, they might end up actually chancing upon more objects - including ones with less well-taken pictures - eventually.

For those users who are researchers and/or specialists, who need a linear, logical sorting, the aesthetics prediction sort option is obviously of much less merit. But it is safe to assume that is this group of users are overrepresented in the above-mentioned group of people who browse the site for hours. And it is also rather safe to assume, that they are generally more used to online databases and the existence of different sort options. Consequently, it can be assumed that they are able to change the sort settings to their needs - or at least have a higher likelihood of being able to do so.

Being very useful for the general public while less so for specialists who can be assumed to be more skilled anyway, it is sensible to make aesthetics-based sorting the default. There are thus three different default sort orders, depending on what objects one lists:

If a user lists all objects of a given instance, the default sort order remains sorting by the age of the entries
If a fulltext search was performed, the results are sorted by how well the query string matched the entry by default
Otherwise - in case of ID-based search queries like a query for all objects created in Berlin - the search results are sorted using the new aesthetics prediction by default

Operationalizing the Prediction

On a side note, operationalizing the prediction proved a challenge in itself. All of museum-digital’s servers are built for traditional web hosting. Hence, they feature quite powerful CPUs, lots of RAM and no GPU. In other words: They are least suitable for AI applications. And they are all the more unsuitable for scoring over a million thumbnails in bulk while remaining otherwise performant.

If the existing servers cannot be used, there are three reasonable alternatives. First, we could have simply rented another server. As a mostly volunteer-run project, this was not an option. Second, we could have used browser-based AI to calculate the score on users’ machines - we might have e.g. let an uploading user’s machine calculate the score of a thumbnail whenever the user uploads an image. But the users’ PCs vary widely, laptops and tablets (usually again without GPUs) have become more and more popular when compared to workstation, and thus such an approach would have made uploading images unbearably slow. It would have also not helped with calculating scores for the already pre-existing million of objects and their main thumbnails. Again, this was not really an option.

Finally, we could use our private machines (specifically: mine). And that is the approach we chose. Based on a new search API parameter aesthetics_score for querying objects whose thumbnails have not been scored yet (aesthetics_score:10001), my PC downloads the yet unevaluated object thumbnails and evaluates a score for each. These scores are then locally stored in one SQLite database per instance of museum-digital and exported into a CSV file. The CSV file is then uploaded to the server and ingested back into the database to set the aesthetics score for the relevant images. The search index is automatically updated with the new scores following the upload.

This structure unfortunately also means that the scoring does not occur in real-time. Depending on when I turn on or shut down my PC, the most recently published objects may remain uncategorized for a while. To be able to fairly represent such objects, they are assigned an impossibly high score that serves to both sort them above the already-scored objects while also marking them as yet unscored. Specifically, the aesthetics predictor returns a score between 0 and 10. As calculating with full integers is generally much more simple than working with floating point numbers, the score is multiplied by a thousand and then rounded, leaving one with a score between 0 and 10000. An object yet unscored will hence be assigned a default score of 10001.

info

AI Use

Post image generated using AI.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

State of Dev, December 2024 & January 2025

2025-02-14T00:00:00+00:00

December 2024

Frontend

Dates in TEI transcriptions are parsed, irrespective of whether when="" oder when='' was used
Notes for markings are now publicly displayed
This was missing thus far and is now implemented similar to how event notes are displayed. If a note exists, a small “[?]” appears behind the marking title line. Upon hovering over it, a tooltip appears with the relevant information.

musdb

Names and descriptions of exhibitions and object groups can now be translated
Version control
Log of “current locations“ of an object can be exported as a CSV file
Uploaded object images can now be hidden or published in one batch operation
API extended
- (New functions)
- Transfer object dimensions
- List images and resources for an object
- Image metadata
- Publish / hide object images

January 2025

Frontend

Objects can now be sorted by the aesthetics of the thumbnail (A dedicated blog post on this will follow soon)

musdb

Batch transfer between between free text fields of object data
Alignment of the maximum field length for notes on opening hours is now consistent between UI and database
Bug fixed with switching between institutions during consistency checks fixed (relevant only to users with administrative access to multiple museums)
Literature can now be searched by editors

Importer

Core
- Automatic transformation of life dates for actors
  - Year of death “01.01.2012” now becomes “2012”, instead of 01.01 as before
- “?” and “(?)” are removed from the beginning and end of imported keywords
- Various types of brackets in keyword names are converted to regular brackets
Parser
- Stricter internal implementation of settings, all imports can now implement the start_at setting
  - This is particularly useful for the repeated execution of imports that abort due to new, previously uncovered elements and other debugging.
- New parsers:
  - MODS (mainly used in library contexts)
  - Parser for Exports from Faust for the Händel-Haus
  - Parser for XML dumps from MuseumPlus Classic (MsSQL > XML export per table > Import)
  - Bugfixes
    - Field “Verwender” in Primus parser was mapped to production events
    - Material / technology are now imported correctly in the parser for BeeCollect exports for the Industrial Museums of Saxony
„Frontend“
- CLI now also has options for switching off the import of individual areas
- Help text for command line tool

nodac

Splitting of keywords now also recognizes keywords that should be split into places, times, etc.
- Bsp.: „Helm; Berlin“ > Schlagwort „Helm“ + Ort „Berlin“
- Example: “helmet; Berlin” > keyword “helmet” + place “Berlin”
When searching for keywords with ambiguous names, both keywords and generally ambiguous terms are now taken into account
Times can now be merged with others directly from the time edit page

info

AI Use

Post image generated using AI.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

State of Development, November 2024: "Real" separated Measurements and a Better Recognition of Tags, Places, etc.

2025-01-13T00:00:00+00:00

musdb

Basic re-implementation of separated measurements for objects (tab: “addendum”)
The type of measurement (width, length, etc.) is now managed using a controlled list, which can easily be extended. This also allows for measurement types of different levels of specificity (“width” vs. “width of socle”)
The new implementation allows users to identify whether a measurement is exact and add notes for each measurement
Measured values now need to be entered as a floating point number - consistent search for objects smaller or larger than a given size is thus made possible (previously only available for entries where the system could deduce a numeric value from whatever had been entered)
Unique naming components of entered tags are automatically parsed into a relation type. German: “Apfel (Motiv)” is automatically split into the tag “Apple” and the relation type “display subject”.
Users entering a tag with such a naming component that is known to belong to another vocabulary will have their input auto-corrected to reflect both. Entering the tag “Berlin (Motiv)” will be auto-corrected to a link to a “displayed place” “Berlin”.
The list of spaces that can be linked to an object as its current location is now sorted alphabetically

Frontend

Re-implemented separated Measurements

Import

The import tool now also uses the new implementation of separated measurements
Unique naming components of entered tags are automatically parsed into a relation type. German: “Apfel (Motiv)” is automatically split into the tag “Apple” and the relation type “display subject”.
Users entering a tag with such a naming component that is known to belong to another vocabulary will have their input auto-corrected to reflect both. Entering the tag “Berlin (Motiv)” will be auto-corrected to a link to a “displayed place” “Berlin”.
Links between two objects are now imported using the dedicated data type for this purpose. They had previously been imported as regular web links.

nodac

When merging two entries, links between the entries and collections, exhibitions, etc. are now reflected and rewritten
previously, links to such data types prevented the completion of the merge
If an entry’s name is marked to always belong to e.g. a tag, the buttons for transferring the entry to the actor, place, or time vocabularies are now hidden.
If an entry is moved between vocabularies, links between the entry and objects as “display subject” / “displayed place” / “displayed actor” are reflected and translated into new links using the appropriate link type.
A new context menu on overview pages allows for a quick access to functionalities like the splitting of entries

info

AI Use

Post image generated using AI.

tip

tip

This content is licensed under a Creative Commons Attribution 4.0 International license.

State of Development, October 2024: Searching Objects Currently On Exhibition, Linking Location and Acquisition of Literature

2024-11-06T00:00:00+00:00

After the blog has been very quiet this year with regard to the technical development of museum-digital, we are now trying to publish the summaries of new developments - enriched with screenshots - that are prepared for the monthly “regional administrators” rounds in Germany anyway.

These are in the form of listings, and this is how it should be here too.

Frontend

Features & Improvements

Some improvements in background scripts, especially better handling of timeouts when calculating “Similar objects” in very large instances
Contributors, linked locations and times for an object group are now listed alphabetically by name
Table headers for event components (who, when, where) are now only displayed in the A4 PDF if there is also content for the row
New search option for object searches: “Is currently on display”
Links to the Themator now use the new URL scheme of the Themator
(https://themator.museum-digital.de/t/690 instead of https://themator.museum-digital.de/ausgabe/showthema.php?m_tid=690&tid=690)

The new filter option “Currently on display” in the overlay for the advanced search for objects in the frontend of museum -digital.

The contributors to an object group are now sorted alphabetically by sorted by name .

Bugfixes

Error when searching for controlled list terms that contained multiple spaces via the “Refine search” overlay (search for license “Public Domain Mark”)
Exactness setting in the “refine search” overlay was not transferred to the actual search query
Simple embedding of an object (analogous to YouTube videos, for example; accessible via the “Cite” menu of an object page) had various errors / now works again

musdb

Features & Improvements

In the API documentation of musdb there is now a note that the frontend also has an API
- Frontend API
  - You do not need to authenticate yourself to use the frontend API
  - The frontend API tends to be faster and easier to use
  - Is read-only
- musdb API
  - Can do more: Can also see non-public stocks and fields / data types
  - Is much more granular (more queries for the same data, but you likely get exactly the data you are looking for instead of e.g. all data known about a given object)
  - Can be used for writing data
Suggestion lists when searching for vocabulary terms in the side column of the object search page have been revised
- Tooltips appear when hovering over
- Implementation in Vanilla JS, removing jQuery
- (this means significantly better performance of the search results list in list format, because jQuery no longer needs to be loaded)

The suggestion lists for places, times, persons and keywords in the quick search function of the object search mask have been re-implemented. The main visible benefit is that explanations now appear directly when hovering over the terms in the list.

User page / Login
- Log of logins now also with IP and user agents
- Login via login persisted in the browser (“Remember me”) is logged and displayed
- All browsers permanently logged in via cookie are forced to log in again after a password change
- New option to invalidate all remembered logins on other devices (browser must be logged in again)

The “Login log” in the account settings can be used to track when and in what context one’s own user account was accessed in musdb. This allows for the identification of account takeovers by third parties. Newly logged and/or displayed are: IP address used to log in, the user agent (identification of the browser) and whether the browser was automatically logged in via a permanent login cookie (“Remember me”).

A new button in the toolbar of the account settings in musdb allows you to log out all permanently logged in browsers / devices from your own account.

Object
- More restrictions for the publication of object data records.
- An object can no longer be published if:
  - … the object name is the same as the object description
  - … the description contains the character string “lorem ipsum”
- When object entries are unpublished / hidden, the images linked to the image are renamed (thus invalidating links to the images). When publishing the object again, this is reversed so that existing links work again.
- Spaces in selection lists are now listed alphabetically as the actual location when linking
Literature
- Acquisitions can now be linked to literature
  - Previous owners etc. can thus be linked to a literature entry
- Spaces (actual location) can be linked to literature

Via the new tab “Administration” tab on tab on the literature editing page, the location and access context of the literature entry can be linked. This can be useful if the literature module is also used to manage the museum library. is also used to manage the museum library .

Bugfixes

Overlay for setting searches for objects: Multi-word search terms were converted into multiple searches instead of being searched as a string of words (“red helmet” > “red” AND “helmet” instead of “red helmet”)
Error when searching for controlled list terms that contained multiple spaces via the “Refine search” overlay (search for license “Public Domain Mark”)

Importer

Link between literature and spaces (actual location) as well as acquisitions is implemented in the “core” of the import tool
ImageByInvno parser (assignment of images to objects via inventory numbers contained in the file name) can now be used to import PDF files

files.museum-digital.org

Added a small script to enhance PDF metadata based on an XML sidecar file. See e.g.: https://files.museum-digital.org/de/Praesentationen/2024-10-18_md-deutschland-eV-stellt-sich-vor_Usertreffen_MA.xml

info

AI Use

Post image generated using illustriousXL_smoothftSPO.

tip

tip

This content is licensed under a Creative Commons Attribution 4.0 International license.

Reconciliation APIs arrive to museum-digital

2024-07-03T00:00:00+00:00

Imagine you have a spreadsheet with potentially unclean data or data that is not confirmed to be interoperable. A museum may want to migrate their data to a different system or share it with an aggregator or a researcher may want to analyze data from different museums where each has their own thesaurus. To make the data legible to an external service or to understand that where one museum says “shoe” and another says “shoes” the exact same thing is meant, it makes sense to check the terms in question against larger (norm data) databases and add their ID for the entity in question. Thus, “shoe” and “shoes” become understandable as the same.

This process is called reconciliation. One of the most used tools to actually connect to external services and reconcile data this way is OpenRefine. But to do so, OpenRefine needs to be able to communicate with such services. The open standard by which they communicate is the Reconciliation API specification. Depending on how completely the API has been implemented, the consuming software may offer additional features such as offering alternatives, suggesting entries or displaying previews of entries made available through the API.

Since this weekend, museum-digital offers such APIs for different types of data.

In md:term, reconciliation APIs were added for each of the available controlled vocabularies. Besides the controlled vocabularies of museum-digital (tags, actors, places, times) this means that users of OpenRefine can now reconcile their data with select other vocabularies such as the Oberbegriffsdatei or the hessische Systematik. Besides implementing exact and partial matching of requested terms, this implementation features previews.

Two additional points are noteworthy in terms of md:term’s implementation of the reconciliation API specification.

1) Similar to museum-digital’s import tool, md:term’s reconciliation service attempts to clean up entered terms before reconciling. As such times are automatically rewritten (“1912 bis 1914” [1912 through 1914] becomes “1912-1914”), question marks and other markers of uncertainty are removed, and - whereever known - autocorrections stored in the city (Frankfurt a.M. > Frankfurt am Main) are applied to the terms.
2) Multilinguality: Whereas the current version of the regular reconciliation API does not consider multilinguality, the above-mentioned auto-corrections and rewriting are language-dependent. And so are previews and names of entities in md:term for museum-digital’s vocabularies. Hence, an additional parameter for the requested language can be supplied.

Reconciliation APIs for md:term may be found on the download page linked in the sidebar. Going this way, the language parameter is automatically set to the user’s current language.

Reconciliation by Inventory Number

To identify objects available published through museum-digital, a reconciliation API now exists in the frontend as well. It can be found at /reconcile/invno/ on each instance of museum-digital (e.g. here for the Hessian instance.
It reconciles entries by their inventory number. An additional number may be added to the URL to search only within a single institution. Where https://hessen.museum-digital.de/reconcile/invno/ allows matching objects by inventory numbers across all institutions publishing their objects on the Hessian instance of museum-digital, the https://hessen.museum-digital.de/reconcile/invno/1 will thus only search within the collections of the Freies Deutsches Hochstift / Frankfurter Goethe-Museum (the institution of the ID 1).

Reconciling GLAM data using the MD:term reconciliation API: an example

As they have much more experience with OpenRefine, I asked the colleagues from digiS whether they could provide a description of how to actually use the reconciliation API from a user perspective. This section is theirs:

Using the reconciliation API in OpenRefine is straightforward (cf. the OpenRefine documentation for a general introduction). Let’s walk through a typical workflow using GLAM data and the MD:term API!

Our sample dataset (adapted from actual data provided by the Sportmuseum Berlin) features, among others, columns on material/technique, object type and places.

These elements are just strings and consequently not linked to any authority file. Strings are not guaranteed to be unequivocal, nor are they particularly suitable for retrieval, programmatic reuse or multilingual contexts. In order for the data to be as useful as possible, let’s reconcile them with an authority file, in our case the MD:term vocabularies now readily accessible also via a reconciliation API.

If you have never used the MD:term reconciliation service you first have to add it to OpenRefine. Click on the small triangle in the header of the column you want to reconcile. Select Reconcile → Start reconciling.

In the following window, add the API endpoint you want to use (cf. https://term.museum-digital.de/downloads for a list of MD:term endpoints).

Now you can select the service for reconciliation:

In the following window, the only really important decision you have to take is whether or not to automatically match “candidates with high confidence”. Auto-matching speeds up the whole process, but you have less control over what happens with your data.

Hit “Start reconciling” to trigger the reconciliation process. You can now see that the strings have become links. In fact, they have been linked to possible entries in MD:term. If you choose the auto-match option, the most probable candidates have been selected automatically. If not, you have to select the candidate you want to match manually by clicking the arrow to the left. One arrow matches the value in the selected row only, two arrow match all identical values in the entire columns automatically.

Once you’re done with the reconciliation you can, for example, write the IDs of the reconciled values into a new column. Click once again on the small triangle on the left side of the top row of the column you’re working on. Go to Reconcile and Add entity identifiers column.

Name the column that is to be added and you will have a new column that contains the MD:term IDs.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

A Re-Designed Frontend - Version 2024

2024-01-08T00:00:00+00:00

Over the holidays we worked on a re-design of the frontend of museum-digital. The reasons were manyfold. While the old design of the frontend still looked well, discussing it with different people - especially those who were not regular users - revealed some shortcomings. Others had naturally developed after five years of use or been inherent in the basic concept of the old page design.

This blog post will first deal with the issues of the old design. A summary of the aims of the new design follows as they were often formulated as a direct response to the shortcomings of the then present. Finally, a tour of screenshots through the new design concludes the post.

The new page design will be published on the night from Tuesday to Wednesday.

For those seeking only a quick overview there also is a small playlist on Youtube.

Problems of the Old Design

Inconsistency

A natural problem of the old design was its growing inconsitency. Quite a number of features and even some sections of the frontend were developed after the last re-design (early to mid 2017). As new features were added, they often did not reuse existing styles and instead used slightly different re-implementations of the same.

Hidden Features or “To View or to Act?”

The likely most consequential issue of the old design lies at its very foundation. In 2017 we had done the redesign aiming to create a design that was very much focused on presenting museum objects in a clear and attractive way.

To underscore the issue of clarity, as well as for their then increasing popularity and the possibility to implement one simply even for users of the Internet Explorer (see below), we chose to go with a stream design. Some sidebars existed for presenting additional information, but the aim was to always clearly focus only one thing. While the design gave quite some space to gaps between different sections to improve focus, it ironically reduced our freedom to label buttons (as can be most evidently seen in the settings bar for searching objects).

In summary: Many features were - at times intentionally - hidden. And thus users did not find them or even learn that they existed.

Internet Explorer

This is not necessarily an issue by itself, but a self-limitation that has now become unnecessary. Was our aim to keep the page at least somewhat usable using Internet Explorer 10. Microsoft’s Internet Explorer did, until its demise, not support many of the newer web standards and trying to keep the page simple to maintain while usable in the browser meant that we could only use older styling options (e.g. CSS grids were not supported).

Lines

Especially during the re-design we noticed that the old design had overused lines. Lines to differentiate different sections, lines to form tiles, lines around buttons. While it had not necessarily occured to anybody I personally have talked to (or they did not say so), a comparison with the new design reveals that the number dividers and separating lines effectively lead to cluttered user interface.

Basic Principles of the New Design

Users Want to See, Read - and Then Do Something About It!

As stated above, the frontend has had quite a number of hard to find features - especially interactive ones like a the option to compare different objects or a watch list (the latter even being hidden behind an option in the now obsolete settings page) and export options. Often, they were hidden by being styled as normal entries in a list. This list in turn could either be found at the very bottom of the page (objects) or in a sidebar. This stems from the idea that users of the site primarily want to see and read things and should not be burdened with anything else unless they are explicitly looking for it.

The most radical change of the new design is our leaving that assumption behind. The new design is instead based on the assumption that people want to see and read things - and then do something about them. Be it to save them, to compare and further investigate them, or to share them.

Buttons (styled as such) offering options like the export of a data sheet, subscribing to a feed, comparison, and the watch list are hence now available immediately below the headline of any page offering such options. Whenever the screen size permits, these buttons also feature text explaining their functionality instead of being made up entirely of icons.

Do What Feels Natural

The old design featured some idiosyncrasies. To take the most blatant example: Right next to the page title there was a search bar. The position of the search bar at the top of every page suggests it being a general search bar for all sections of the page. In reality, it was a search bar only for objects. The new design now features a general search bar in its place.

In a similar vein we tried to identify unintuitively positioned or superfluous links. Some features (e.g. the maps for displaying a museum’s location or the locations of all museums represented in a given instance of museum-digital) were integrated into the pages linking to them (the institution’s page or the institution page respectively. They are now accessible as tabs, allowing users to quickly navigate back after viewing the map.

Don’t Repeat Yourself

The old design featured quite a number repeated content. Where possible we tried to identify such duplicates and offer only one, clearer way to access a certain information or visualization.

Hovering over a tag e.g. made links to the timeline and map visualizing objects linked to that tag accessible to users immediately from a given object’s page. These links were removed in favor of letting users click on the tag, see the objects linked to the tag, and then navigate to the timeline or map from there. On the one hand this more clearly communicates what is actually visualized (the objects linked to the tag rather than the tag itself), and on the other it allowed us to remove a flickering and likely unintuitive interaction.

Shapes and Positions Over Lines

This speaks for itself: While the old design used straight lines as dividers a lot, we tried to communicate the separation of different elements primarily by their positioning, size and shape in the new design.

A Tour Through the Re-designed Frontend

Start page

The new start page opens with a rundown of the current number of entries (and their development) and a slogan whose words double as links leading to essential functionalities.

A new feature on the start page is a section that is offered only to return visitors of the page, Here one can see one’s last search queries (for objects) and return to them.

The selected objects on the start page.

Object Overview: Display options

The grid view of object search results. This view is the default on larger screens.

The list view of object search results. This view is the default on mobile devices.

Extended search

Previously there were two extended search options for objects. One, linked from the navigation, listed the search options in a grid. The other - a full screen overlay - featured additional options such as negative searches, but was hidden behind an unlabelled button. The new design only features a single overlay for configuring extended searches.

Object overview: Facetted search

The facetted search has now moved into the sidebar of the object overview page and is thus much more obviously available.

Object Overview: List Results by Museum

The list of objects grouped by museum has now been integrated into the object overview. Previously it was a separate page.

General search

The general search function searches across the different types of entries in a given instance of md. Up to five entries are directly displayed as one types.

Collection overview

The collection overview has been almost entirely reimplemented. It now loads much (!) faster and supports displaying all of an institution’s subcollections, regardless of their “indentation” in the hierarchy of subcollections and sub-subcollections.

Museum overview

The museum overview has similarly been reimplemented to a significant extend. When searching, an institution’s place is now considered besides its name.

The museum overview now features a map of the museums. This map corresponds to possible search queries. Previously, a similar map had been a separate page.

Institution pages

The collection list on institution pages has - like the other sections of the page - now been moved into a tab. It can now be filtered and collapsed.

The object section on institution pages now features some summary information on the objects rather than consisting only of links and numbers.

Object Overview: Interactions Visualization

A new visualization is built upon LIDO events: What happened to the selected objects (say, search results) and when did it happen?

Object Overview: Tag Clouds

Tag clouds now also include actors, places, and times. They can be generated for any object search query.

Object Page

An object page. Significantly, options and interactions such as the generation of a PDF data sheet are now made available immediately below the page title.

Second half of the object page.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

API documentation for frontend and md:term

2023-08-16T00:00:00+00:00

md:term was designed to provide an API first and foremost. The frontend supported a full API providing to all access that the HTML version does since about 2016. We never got around to fully and systematically document the APIs however. Starting today, an OpenAPI documentation is available for both md:term and the frontend of museum-digital.

Along with this, we can gladly annouce a new, updated and much more performant search API for md:term (see the API documentation for now).

You can access the documentation here:

md:term
frontend (via the global portal)

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

Summary of the monthly user meetup (April 2023) / New features and improvements

2023-05-07T00:00:00+00:00

We continued the series of monthly user meetups and again discussed the new features and improvements. A summary can be found below.

New Developments

The last month has been an exceptionally slow month in terms of technical development around museum-digital. There are however some newsworthy tidbits.

musdb

Recording external IDs for museums

Museums, like all of us, are present in more and more databases. For linking different databases, it is useful - sometimes necessary - to know the ID of the same entity in both databases. Hungarian law thus mandates collection management systems, which are to be accredited for fully paperless use in museums, to allow storing a museum’s ID with the Hungarian Statistical Office. musdb can now do exactly that: Store external IDs of museums in a given list of external source repositories / databases.

The list of available external databases and the regular expressions to validate IDs in them are available here. For now, the list only contains the Hungarian Statistical Office, but the more options there will be in the future, the better.

More fields covered by “list results” and Excel export

The “list results” page (a.k.a. table view) for object search results offers the option to view the selected objects’ data in a customizable tabular format. Mainly because of that exact tabular format, it is not possible to display everything a full database view of the object offers using the “list results” page. Over the last month, we have nevertheless extended the list of displayable fields in the “list results” table.

Thus, it is now possible to display all translations for the object type, object name, descriptions, etc. of an object. Similarly, it is now possible to list all Weblinks linked to the object as a compiled (comma-separated) field. As the automated report generation and the Excel export tools build upon the same basic code as the “list results” page, these new fields are now also available in those cases.

New field for objects: “Last change of permanent location”

It is now possible to manually record the last date, the permanent location of a given object has been updated. This obviously makes sense for tracking relocations and re-organizations within the museum. Importantly, the field needs to be manually filled out, as the date of the last change of the permanent location may be far in the past or frankly unrelated to the database, even if there is one (say, the museum moves the depot to a different place, but the date is only later updated in the database via a batch editing process).

User interface

After there were some reports of people not seeing the difference between a valid form in musdb and an invalid / incomplete one, we updated the design of submit buttons. Submit buttons in incomplete forms are now blurred out, to add another hint at the incompleteness of the form.

Frontend

Improved performance and decreased bandwidth usage: serving images in webp

Over the last years, there have been a number of new image formats seeking to combine all the features of older and well-established formats like JPG and PNG with an improved compression. The most established format of that newer generation of image formats is webp, which is by now well-supported by all modern browsers and most local image viewers.

To save bandwidth and improve loading speed, we now store and serve full-sized webp versions of newly uploaded object images along with the regular jpg versions. If possible, the webp version is served on object pages. An additional benefit is the aforementioned support for features jpg files do not support, such as transparent image backgrounds.

iFrame embedding now only possible from whitelisted sources

If a museum wants to embed their data from museum-digital into their own website, there are traditionally two ways to do so. The better, but much more complicated option is to use the API to fetch the relevant data and present them in any way one wants. Much easier (and cheaper obviously) is embedding the museum’s data using iframes.

Unfortunately, iframes can also be used for attacks on users’ login data. We have thus now restricted this option to only allow embedding from whitelisted sources. If a museum wants to embed their data this way, this means that they now need to notify their regional administrators beforehand.

Importer

The importer now supports imports for exports from Startext’s HiDa in the configuration of the Saxon State Agency for Museums.

Other News

In other news, the Youtube channel has picked up steam, with some new tutorials both in German and Ukrainian.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

Summary of the monthly user meetup (March 2023)

2023-04-05T00:00:00+00:00

Yesterday, we held our regular user meetup as scheduled. As promised, below you can find an overview of the new features and updates below some more general points.

General

YouTube channel

There now is a museum-digital YouTube channel. For now, one can find some German-language screencasts on different features in musdb and nodac there.

New sections on project page (www.museum-digital.org)

As already described in the previous post, the project page www.museum-digital.org now features a calendar. Since then, we have also added a small page listing people working on museum-digital - e.g. some of the coordinators and developers - as well as power users. The list will obviously always be more than incomplete, but if you want to be listed, just send a mail.

Development

musdb

Allow batch editing of photographers on image list

Running batch updates on the ownership, license status and photographer of a set of objects can be done either via the institution-wide settings page or via a selection on the image list page. The batch updating functionality of the institution-wide settings page works by replacing the given entries in one of the fields for all objects of the museum - it is thus not suitable if one is to only update the photographer field for a list of objects from a single collection (rather than the whole museum).

Batch updating images’ legal information from the image list (click on an image in the list and drag it to activate selection mode, then update by selection) was only possible in the case of image licenses and rights holders. With a small update, it is now possible to also batch-update the images’ photographers field that way.

The API can now update object event

Thus far, the API could only be used for adding wholly new object events (who did what, when, and where, with the object) or deleting them. With the updates of this month, now possible to directly update events.

This is already actively used in customizing musdb using a Tampermonkey script by some of the norm data editors. They can thus add a custom button to transfer incomplete actor or place information to the object’s description. This button, on the other hand, would not be useful for regular users at all.

UI improvements for the object search interface and new sort option

After a very fruitful discussion and feedback from Hungarian colleagues, we got around to some user interface improvements in the object overview and search interface. A lengthy discussion can be found in a dedicated blog post in German. To sum up: there are now headlines for each subsection of the sidebar and tooltips explain the different buttons and search and sort options upon hovering one’s mouse over them. Additionally, a new sort option was added to sort objects by the number characters within the inventory numbers of the searched objects.

Two bugfixes: Navigating to the previous or next object in a selection

It is a very simple but similarly useful feature: One runs an object search and thus creates a result list of objects. After viewing or editing one of the objects, one can then proceed to the next one in the results list by clicking at the arrows at the top of the sidebar.

Unfortunately, musdb had two bugs preventing this navigation for some of the available sort options. On the one hand, newer sort options required manually adding the search fields to a list specific to the previous/next navigation thus far - and had not been covered yet. Navigating to the next object after sorting by e.g. the length of the searched objects was thus impossible so far. The specific list of sort fields has now been removed in the previous/next navigation. Instead, the main list of sort options from the main object search class is directly used - in effect preventing such issues of “forgotten” values from ever appearing again.

A second issue concerned string-based sort options. For an improved performance, the previous / next navigation works by querying the underlying search index for the next object where e.g. the object ID is lower than the current one (`WHERE id > 20000`). The same search logic is straight-out not possible with string-based sort options, as search queries for values greater than or smaller than a given one cannot be executed with string-based fields.

Fortunately, the issue came up just after we had implemented an option to search and sort objects by the numeric components of their inventory numbers. Thus, the previous / next navigation can fall back to that sort option if a user wants to navigate to the next object after sorting by inventory number. As stated back when we introduced the new search and sort option: It should work for most museums. Unfortunately, there is no such option to mitigate the issues of string-based sorting when sorting objects by their names. The best we could do in this case is displaying a clear error message, stating that the previous / next navigation is not available when sorting by the object’s names.

“Home location” can now be selected as a required field when adding new objects

On the institution-wide settings page, museum directors (or those holding the same user role within musdb) can determine which fields are required when adding new objects. The list of selectable fields automatically covers all free text fields linked to the object. Repeat fields and references to other sections of musdb need to be manually implemented however. As the home or permanent location of an object is a most logical and common field to be required for all object, it was only consequential to prioritize making it available as a required field. We have done so now.

Major performance improvements in “list results” page

The “list results” page provides for a table view of object search results, allows the export of search results to Excel, and is - under the hood - also used for generating custom reports. We managed to achieve a considerable improvement in the page’s performance by bulk loading data and simplifying the code of the HTML page. Both adjustments should not really be noticable when one tries to list or export only some objects. Exporting some thousand objects should work considerably faster now however.

frontend

PDF metadata for main object PDF export

XMP metadata are now written into exported PDF files. Thus, users can more easily identify authorship, titles etc., especially when using specialized software like e-book readers.

Linking from collection descriptions

Collection descriptions are now being parsed for the existence of URLs. If one, starting with https:// (http:// is ignored) exists, the URL is automatically transformed into a clickable link.

nodac

We added some translation variables on the start page.

themator

The “themator” has received a small user interface improvement for mobile devices. The topic-specific navigation or table of contents is now moved below rather than above the main content of a page in the regular “topic” viewing mode on mobile devices.

Importer

Importing GIFs

To simplify the handling of images, museum-digital internally only uses JPG image files (webp versions are generated for a faster loading, but are optional). While they lack some support for features like transparency, JPGs provide for a much better compression with images of three-dimensional things and subjects not featuring solid colors. Newer formats like AVIF and Webp unfortunately still lack the support among locally installed image viewers for us to fully switch over to them.

The importer handles input PNGs by converting them to JPG files. Starting this month, the same can be done with input GIF files.

Improved handling of blacklisted actors, places and times

One of the key issues with imports is what happens after imports. Specifically, museum-digital uses controlled vocabularies for actors, places, times and tags that can be linked to an object directly or via events. When people enter new such entries using musdb, there are a number of safeguards and nudges to ensure productive inputs. Adding a new actor, for example, requires one to enter at least 10 characters of a description for the actor. Input fields for the date of birth and death are also provided. If possible, one can enter a Wikipedia or Wikidata URL (or ID) and directly fetch the relevant information from there.

During imports, there is no space for asking questions to the importing institution. Incomplete or false actor, place, time and tag names (for example, if two actors are linked as one; “John Doe and Jane Doe” is not one single actor!) are given, these are necessarily imported into our controlled vocabularies by default. Depending on the import, this puts a significant burden on our volunteer group of vocabulary editors.

To eventually get to a solution, we have added options for automatically rewriting terms (e.g. “Berlin, Germany” will be autocorrected to “Berlin”) and blacklisting them completely (“Good morning …” is no actor“). Thus far, blacklisted terms were simply ignored by the importer.

“Unknown painter” is not an actor, but the information that it is a painter may still be useful. To allow us to blacklist more and keep the controlled vocabularies “clean” without losing such data, the importer now imports blacklisted terms in event components (“who painted it?”) into the event annotation. If the only known information on the whole event is blacklisted, the event cannot be created - an event always requires at least one linked actor, place or time. The event annotation of “empty events” is hence moved to the object description automatically.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

New Features at museum-digital (November 2022)

2023-01-02T00:00:00+00:00

After trying a monthly change log once some month ago, we have unfortunately been rather lenient with notifying everyone of new features and updates in the last months. To approach betterment, here there is a list of the updates of November 2022 the form of screenshots. As a very large update is upcoming in the next days, a separate post on the updates of December 2022 will follow tonight.

musdb

New fields

New section for time limits on administration tab of object pages. Special mention should go to the fields “Freeze period” and “Publish object at”. Filling out these fields enables some automation: A “frozen” object cannot be published before the entered date has been reached. This may e.g. be useful with archival material that cannot be published before a given date. The “Publish object at” field offers a counterpart to this. If a date has been entered into this field and the date is reached, the object will be published automatically by the system.

A non-public closer location for an object (which may e.g. be necessary with archeological findings, whose finding spots have no name and are not to be published to not give information to grave robbers) can now be set using a map on the addendum tab.

A number of new fields for noting conditions on how the object should best be displayed in exhibitions, among others, have been added on the “remarks” tab of object pages in musdb.

On the “restoration” tab of object pages, generic fields can be entered with the name of the described feature and the value. Because of the flexible subject of these fields however, they make searching in the fields much harder. Hence, new fields that are applicable to almost all museum objects have been added as easily searchable, “static” properties of an object: Minimum and maximum viable temperature, minimum and maximum viable humidity, and the maximum lux an object may be exposed to.

Other page

It is quite common for users of musdb to only use the same some event types all the time, while not needing many of the other available event types. People working at archeological museums will likely need the “found” event type all the time, while barely ever using the event type “copied by hand”. To directly access those often used types, one can now click the “star” symbols at the end of a line for an event type when accessing the page for selecting the event type of a new event. Favorited event types will then be listed in a bottom sheet on the page.

Batch updating object information

The batch update menu for objects’ visibility can now also be used to set the visibility of publishable fields that are publishable on a field level.

The “batch assignment” menu can now be used to assign spaces, owners, linked loans, and full events (e.g. the creation of objects by a given artist at a given time) to all objects of a search results list.

A “smaller” way of batch updating objects can be used in the object overview by selecting an object by clicking and dragging an object. Now, objects can be selected and updated in bulk. The menu for doing these updates (visible here at the top of the screenshot) now comes with an additional option: “Open in new tab”. By clicking on this menu option, all selected objects are opened in new tabs. As browsers often prevent the opening of multiple tabs in bulk, one may have to allow the opening of pop-ups for musdb in the browser to use this functionality.

Instituion-wide settings and adding new objects

The download button for images in the frontend has been repurposed to enable bulk downloading of all images of an object. While the images are downloaded, the users see an overlay where the museum may display a message (e.g. on how to use the images, or for asking the users to notify the museum about the images being reused in print). The message can be set in the institution-wide settings (available for users of the role “museum director” by hovering over the academy symbol in the navigation in musdb and then selecting the menu option “settings”).

The instituion-wide settings page now also comes with two other new features. On the one hand, users can now be required to select a tag for the object type when adding new objects. On the other, the inventory number suggestion when adding new objects has been improved. It is now possible to generate inventory numbers with variable length numerical components (e.g. ABC-9; followed by ABC-10).

Notifications

The notification framework in musdb has been fully rewritten. Along with that comes the option to specifically subscribe to email notifications only for some types of notifications. To do so, one can navigate to one’s personal settings. A new tab “notifications” on this page allows setting the primary route of notification and a fallback. If the primary route is set to “email” for upcoming ends to loans, the user will immediately receive a mail once the system recognizes an upcoming end to a loan. If the primary route here is set to “Internal” and “Email” is set to be the fallback route, the user will only see a notification on the upcoming loan in the notification overlay within musdb for a week. If the notification has not been marked as read after a week, a mail will be sent.

In other news

The calendar feature (accessible under the puzzle symbol in the main navigation) can now display tasks or make them subscribable via WebCal (thus implementing a “reminder” as had often been requested)
PDF of all linked information is now in A4 and uses a two-column layout
Ukrainian translation
“Simple A5 PDF” now covers inventorization fields on rear side

md:term

If two actors have been joined and one has an old links to the page of the actor now deleted, one is now referred onwards to the new, single actor entry. The same works with transferrals between vocabularies (an actor that was transformed into a tag, etc.).
Ukrainian Translation

md:term is now available in Ukrainian!

Frontend

Ukrainian Translation
JSON-based settings for specific institution pages have been removed
Bulk download of object images
- An overlay with a message from the museum can be displayed during batch downloads (see above)

CSVXML

Almost completely rewritten
The served page now is completely static and all checks and conversions run directly in the browser. This way, no uploads actually happen and the application is completely uncritical to the server’s security. On the other hand, this allows for installing CSVXML as a progressive web app and using it offline.
We also added some explanatory texts did small updates to the design of the page. A footer now links to the source code and offers to refresh all cached contents of the page (this may be useful when visiting the page after a long time, as the whole application is cached in the browser for offline use).
A new check also checks for the file encoding. A warning is provided if the data does not appear to be UTF-8-encoded.
CSVXML is now released open source under the AGPL.

CSVXML has been (almost) completely rewritten).

Importer

While the individual parsers for the different export formats are updated very often, the core scripts of the importer are very stable. November 2022 however came with a large update to these core sections, as more categories of data that had not before been covered by the importer (many of them new) can now be imported:

Contact information (e.g. for object owners; loan partner institution)
Object’s movement log
Minimum and maximum temperature, humidity, and lux of an object
Loans
Events / Appointments

In terms of the parsers, we extended the LIDO parser to cover the new fields suggested by the upcoming EODEM standard for exchanging loan object information.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

musdb and frontend available in French and Russian

2022-11-08T00:00:00+00:00

In the last month we have been able to finally release the long-anticipated French and Russian translations of the frontend of museum-digital as well as musdb. A Ukrainian translation is currently being worked on as well. Thanks a lot to the colleagues from Speyer and Berlin-Karlshorst for organizing and translating!

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

Monthly museum-digital user meetup (September 2022) / New Features

2022-09-23T00:00:00+00:00

On September 6th 2022, we continued our monthly user meetups. As should best become the norm, we discussed recent new features of the preceeding month and plan a next meetup on the first Tuesday of October (October 4th, 2022, 5 p.m. at https://meet.jit.si/museum-digital-meetup-202210).

A summary of the new features and updates can be found in the following.

Recent Updates and New Features

musdb

Support for Imperial Measurement Units for Separated Measurements

Thus far, musdb only supported metric units for length, width, diameter, etc. It is now also possible to enter measurements in inches and feet. For searching objects smaller or larger than a given number, the number is still expected to be provided in millimeters.

Reworked Institution-Wide Settings Page

Certainly the main focus of our work on improving musdb has been a rather extensive overhaul of the institution-wide settings page. This page is available for users of the user role “museum director” and can be reached through the academy-icon in the navigation.

With the batch of updates, the page gets a new design with an always visible table of contents and rather extensive descriptions of each of the available settings. We removed some outdated and confusing options while adding new ones for the newly added strict modes and inventory number suggestions.

Suggestions for Inventory Numbers for New Objects

On the institution-wide settings page, it is now possible to set a scheme for how the museum forms its inventory numbers. Variable parts are expressed using placeholders:

{no} for a number counting up. For a counter padded to a given number of digits, a number may be added after the place holder (e.g. {no}6 for a 6 digit counter in the inventory number.
{c_sig} for a signature of the collection
{year} for the ongoing calendar year).

If a regular inventory number in the house is, say, “2022-000025”, the scheme would thus be {year}-{no}6.

If an inventory number scheme has been set, an additional suggestion button will appear in front of the entry field for inventory numbers on the page for adding new objects.

Searching for Objects by Their Entry Dates

Similar to how secondary search indexes for measurements greater or smaller than a given value have existed for some time, entry dates, estimation dates, etc. are now made properly searchable through the object search in musdb. For this, dates need to be set in a coherent, maschine-readable form that is greatly aided by the new strict modes.

Strict Modes

See the dedicated blog post on this topic.

If one wanted to delete an object group, one thus far needed to find the object group in the object group overview and delete it from there. There was no option to delete it right from the page of the object group itself.

For a more straight-forward control, we now added deletion buttons on the editing pages of object groups, exhibitions, etc. These deletion buttons can be found in the toolbar on the right of the screen.

PNG Supported When Uploading Object Images

Since object images are almost always photos or scans that feature a multitude of colors and few solid blocks of color, JPG is a much better format than PNG for storing (compressed) object images. musdb hence only accepted JPG images for uploads thus far. Since some museums however have internal rules for keeping their object images’ working copies in PNG, we now allow PNG uploads and then convert the PNG images to JPG during the upload process.

Frontend

License Descriptions Consistently Displayed on Object Pages

Until last month, a description of the metadata license of an object page (to be found at the bottom of the page) was only available for the license CC BY-NC-SA. We now removed the exception and made descriptions and links to the license text available for all available licenses.

Importer

Improved documentation
Further modularize parsers
Added new parsers for EasyDB (work in progress)
Allow importing images by their URLs to locally hosted ones

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

Referencable Maps in the Frontend and in musdb

2022-07-22T00:00:00+00:00

Development on museum-digital will be concentrated on more internal functionalities in the next months, but some small improvements continue to be made in the public frontend. Like today: We have now implemented the option to directly link to a position on a map view for object search results both in the frontend and in musdb.

To allow this, navigating or zooming to a given position within the map now automatically updates the URL to include the new selected positioning on the map. If one thus copies the URL from the browser and shares it to a colleague, the colleague will be able to open the map at the same position.

Additionally, we have been able to fulfill an often-voiced request for the map view: a legend of the many available event type icons on “objects on map” pages in the frontend is now accessible by clicking on the new “Question Mark” button at the top right of the map.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

IIIF and museum-digital

2022-04-03T00:00:00+00:00

Many have seen it and the feedback has been consistently positive thus far: For many objects, museum-digital can now offer a revamped image viewer with improved features, such as a much better zoom function. This is made possible by the image viewer Mirador and our IIIF API.

The collection of paintings of the Stiftung Preußische Schlösser und Gärten as a IIIF Collection in Mirador’s demo instance

The fact that museum-digital offers an IIIF API at all, and that a IIIF viewer ist now used at museum-digital, is noteworthy, but quick to report. Especially, since both IIIF and Mirador are very well-documented on their own websites (IIIF und Miradorr).

But if one looks into it deeply, it becomes obvious, that the IIIF-based image view is not available for all objects.

Limitations of museum-digital’s implementation of IIIF

IIIF offers a standardized API format for retrieving image metadata through an appropriately formulated URL, but it also enables the retrieval of image files that are cropped or rotated accordingly on the server side. This server-side image editing is useful for making IIIF-based applications run on low-powered devices and it provides easily referencable URLs for specific sections of an image. But it also means, that all images made available through the IIIF API should be available on the server itself in a high resolution.

At museum-digital, museums can upload their images - but since some museums have their own image servers, museums can also link images from external sources. In this latter case, museum-digital only stores previews up to a width of 500 pixels. These previews are however not usable for an adequate server-side image editing as required for implementing IIIF. Thus we offer IIIF manifests only for objects that exclusively have locally stored images.

We currently also implement IIIF only in version 2.1, which does not yet support other media types such as PDFs, videos, audio files or 3D representations of objects. Hence, objects that have any linked representations of these types are also excluded from the IIIF API.

IIIF Collections

One of the nicer features of IIIF are IIIF collections, with which many objects / manifests can be grouped into a collection. For implementing the minimal requirements for an IIIF API two years ago, we had thus far already set up the relevant files for IIIF collections - but we didn’t fill them with contents.

Now, IIIF collections at museum-digital represent collections and object groups. The limitations described above persist however: IIIF collections at museum-digital only list objects with exclusively internally stored images.

IIIF collections can be accessed - as always - by their respective URL. E.g.:

https://brandenburg.museum-digital.de/apis/iiif-presentation/collection/c217

This URL represents the collections of paintings of the Stiftung Preußische Schlösser und Gärten. The noteworthy section is the last: c217. Since the IIIF collection API at museum-digital represents three different types of groupings at once - institutions, collections, and series / object groups - the first character of this URL section represents the respective type of grouping. The ID of the entry follows.

An “i” in front of the ID thus represents institutions, an “s” represents series. One example of a IIIF collection for an object group may thus be:

https://hessen.museum-digital.de/apis/iiif-presentation/collection/s20

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

More Stability and Some New Features

2020-09-15T00:00:00+00:00

Over the last months, we have not been inactive in further developing museum-digital. We mainly worked to improve the legibility of our code under the hood - say, most changes will hopefully not be noticeable, but an improved speed of especially searches for objects may be a positive and visible effect.

Either way, we have also worked on some new features: A dedicated area for markings (stamps, watermarks, etc.), one for references and reception (e.g. the publication history of an object), and entering uniform information for the field “object type” has gotten easier. Most importantly, we have added some new event types to better meet the requirements of provenance research.

Markings

Especially for art historians, a precise recording of information on different types of markings, that have been added to an object for indicating authorship and ownership are essential. Be it stamps, watermarks, signatures. Thus far, information on these could only be added using the unspecific field “Inscription” in museum-digital. The new “markings” area (to be found at the bottom of the tab “Addendum” in musdb) aims to tackle this problem. The term “markings” is deliberately chosen to be wide in meaning to also emcompass logos and other types of markings that are more relevant for e.g. technical collections.

Markings in musdb can be found at the bottom of the “addendum” tab.

If a user of musdb wants to add a new marking, they can scroll down to the bottom of the “Addendum” tab and find an accordion button. A click on this button toggles open the area for adding new markings. First, the tye of the marking and its position on the object need to be selected. A transcription must also be provided to safe. Additional information to add are the creator, a possible mentioned time on the marking, its size and a note.

Creators and times are drawn from the centralized controlled vocabularies for actors and times respectively. Hence, they can be entered by tiping the given term and then selecting the correct value from a drop down menu appearing as one types. If the term does not exist yet, it can be added by clicking on the “+” symbol right before the text field.

Once a marking has been entered, it is public for now, even though we consider adding an option to hide markings even when their object is public.

Markings in the frontend of museum-digital (at the top left of the screenshot).

Reception and References

With “reception”, there is a whole new tab in musdb for linking objects with sources. These sources come from a new controlled vocabulary for sources, which is used together by all institutions and runs on a similar level as actors, times, places, and tags. Reception now describes all instances, in which an object is referenced in a source - e.g. if a painting was printed in a book, or a photo published in a newspaper. Similarly, the discussion of a custom-made car in a car magazine may be recorded here.

The counterpoint to this is the “is reference” area just below reception: This can be used, if the object itself references a publication. E.g., a painting may be drawn after a scene from a book.

Reception and references in musdb

Like markings, reception and references are by default public.

References and reception are shown at the bottom of object pages. A click on a source name opens the source page, where one can find all objects referencing the given source.

Object Types From a List

museum-digital traditonally treats object types as a free text field, while many - if not most - museum databases provide controlled lists of available object types. We use keywords / tags to provide the same functionality, but it surely makes sense to also have some level of uniformity in the field “object type”.

Hence, there is a new aid in adding uniform object types when adding a new object in musdb. As one types in the field, a drop-down list of similarly-phrased keywords appears. Clicking on an entry from that list enters its value into the “object type” field and (if one is adding a new object) automatically links the object with the selected tag.

JSON-LD Metadata for Image Licenses

Another minor improvement is that single image pages now feature JSON-LD markup describing the respective image license. This enables especially Google, to provide users with a more detailed view on how to attain the license to use a given image file.

Google Images shows a small “Licennsable” marker at the left of an image. Clicking on it shows more information on the image license at the right.

Importantly, images from museum-digital now also appear in Google Images if one opts to only view Creative Commons-licensed works.

Events: New Event Types and References for Events

The probably most important change are the changes made for entering events in museum-digital. One the one hand, important missing event types have been added. These are:

Assembled (to be used for objects, that have been assembled from parts. E.g. a car)
Auctioned
Bought
Owned
Sold
Restorated
Damaged
Destroyed
Lost
Edited (for books and publications, that have an editor)

As can be seen from the list, most of the new event types are related to provenances (for all museums that want to use these event types: Please (!!) ensure that you have cleared the necessary rights to publish this information, as events are public). To make events at museum-digital more closely meet the requirements of provenance researchers, a missing piece were also sources for events, which can now also be added.

To do so, the addition of sources for events must first be activated on a museum-level. To do so, a museum director or user with similar permissions must first navigate to the “institution-wide settings” page in musdb, and activate the event sources there.

Once they are activated, an accordion button appears on event editing pages (accessible by clicking on the event type of an already added event). Here, sources can be linked using a drop-down menu similar to the ones used in the features described above.

Finally, with options to add event annotations and sources, there was an increased danger of cluttering the object page in the frontend of museum-digital. Hence, we redesigned the display of event annotations to now appear in a tooltip - if an event has an annotation or a source, an “i”-symbol appears next to the respective element for the event type. Hovering over that element makes sources and annotations visible.

Beitragsbild: Caravaggio: „Der ungläubige Thomas“ (CC BY-NC-SA @ Stiftung Preußische Schlösser und Gärten Berlin-Brandenburg / Murza, Gerhard)

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

OpenSearch: Search museum-digital directly from the browser's search bar

2020-06-05T00:00:00+00:00

OpenSearch is an open web standard for describing search functions of web services. If a website supports it, the browser will take note and offer the user to install the website as a search machine.

One especially nice aspect of it is, that OpenSearch can be implemented in just a few lines of code. It was thus easy to bring to both the frontend of museum-digital and musdb, to make searching objects right from the browser’s search bar possible, just as you can quickly search Wikipedia or Amazon on many browsers by default.

At the top right, next to the search (loupe) symbol, there appears a green “+” …

… clicking on the search symbol offers installing the instance of museum-digital as a search provider.

Since musdb does not feature a simple, unqualified search function (one always needs to specify, whether one searches for a tag, place, etc.), OpenSearch offers using the full text search on musdb.

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.

Türkçe

2020-02-05T00:00:00+00:00

The frontend of museum-digital is now available in Turkish. As a very nice side effect, this also unlocks the Turkish translations for keywords, and place and actor names we have been gathering for the last half a year.

A screenshot of the public frontend of museum-digital in Turkish. Currently, the “advanced search” page is opened.

Image credits: Eva Gonzalès: „Girl Awakening“, Kunsthalle Bremen – Der Kunstverein in Bremen (CC BY-NC-SA)

info

License

This content is licensed under a Creative Commons Attribution 4.0 International license.