musdb

Object overview

In the default tile view of the object overview page, hovering over an object image thus far revealed the object’s name. As object names are often too long to display fully and inventory numbers are the primary means of identifying an object in most museums, this preview text has now been extended to include the inventory numer.

User management

New Options for Managing User Accounts: Disabling Accounts & Setting Account Expiry Dates

Two new options on user editing pages allow disabling logins on an account and setting an expiry date for the account. Both can be useful for administration: If a new worker joins the museum for a project with a clear-cut limitation on funding and time, one can now set the account expiry at the beginning of the project to the end of it. The accounts will then automatically be deleted when the project ends. Similarly, colleagues that leave service temporarily but for a prolonged time (e.g. for a sabbatical) and will not need to use their accounts for that time can have their accounts disabled.

List of Terms of Use

A new tab on a user’s (own) account settings page provides the option to list all usage agreements / terms of use a user has agreed to in the context of their use of museum-digital / musdb and when they agreed to them.

Imports

Limiting Report Mail Size

When a user runs imports themselves using the WebDAV upload, the end of the import process – no matter if it is successful or fails – is marked by the sending of a report via mail. This report usually contains a list of noteworthy operations that happened during the import, e.g. which objects of which inventory number were imported to which object in musdb, identified by its ID. As imports grow, this list of operation grows. To not encounter issues sending the report, it is henceforth limited to a maximum of 2 MB or 10000 lines.

Dry-run Mode

Sometimes it is useful to try running an import to see if it will actually work but not actually process any data. This option has been available in the importer command line interface for a while, among others powering museum-digital:qa. It is now available in the import configuration for self-run imports as well using the setting dry-run. Enabling the setting accordingly stops the importer from actually writing the data into the database and changes the behavior if values that need to be mapped to values in controlled lists at museum-digital are encountered. Usually an import stops the moment such data is to be imported and not yet mapped. During a dry run, the error is collected and the import proceeds. All unmapped entries are listed together at the end of the import, allowing for a simpler mapping (possibly aided by concordance.museum-digital.org).

Dashboard

The first page of the dashboard, which for almost all users also means the start page of musdb right after the login process, was significantly reworked during the last month. The almost entirely unused notetaking features and discourse integration were removed in favor of a feed of recent blog posts. See also the section “Communications” in the respective blog post.

Annotations for the Vocabulary Editing Team

Each event, displayed as a tile on object editing pages, featured speech bubble icons behind each time / actor / place to provide additional comments and hints for the central vocabulary editing team. This positioning of the annotation feature led to confusion over the years, with some users using the feature to comment on the relationship between the entity and the object (for which the event notes should be used). We hence repositioned the links and moved them to the respective entity’s page (e.g. a place page for giving hints and comments on a place entry). The hinting / commenting feature for times has been altogether removed, as providing comments to clarify the meaning of e.g. a year never made much sense.

Smaller Updates and Bugfixes

• Fixed a bug in the HTML generated for listing other objects linked to an object. Links to the other object were broken and are not anymore.
• Image editing pages now embed the image directly instead of using the IIIF API. This reduces resource usage and increases stability at no cost.
• Removed option to manually trigger the rewriting of EXIF and IPTC metadata of object images. Rewriting takes place in the background whenever an image or a linked object is updated, making user-triggered updates obsolete.
• Re-introduce option to repeat linking to the last used linked object
• Updated Swagger UI to version 5.30.3

Frontend

As stated above and lengthily described in the previous blog posts (here, here, and here) we struggled with stability over the last month. This means that most changes in the frontend are aimed at improving stability.

Reworked Default Image Page

Thoroughly described in Updates, AI scrapers, and Resilience, we replaced the default view for single object image pages. While the default view was previously built around the IIIF viewer Mirador, the new default view uses OpenLayers and the unmediated image file for capabilities such as zooming. The new view also brings with it some new features, such as an option to reference specific sections of an image.

Serving Resource-Intensive Pages / Functionalities Only When Resources Are Available

PDF generation, the IIIF Image API, and the suggestions for alternative search queries on failed search pages are now limited to reduce their impact on the overall system stability. This follows two strategies:

• Suggestions on failed search pages and PDF generation will only appear if the overall load on the system is low. The threshold for when or when they are not provided is influenced by the user’s browser language: If a user uses a browser set to the primary language of a given instance of museum-digital (e.g. German in Hesse, Hungarian in Budapest), the threshold is much higher, meaning users will be able to access the pages at a medium server load. In the case of PDFs, high server load will forward users to the print dialogue for object pages instead of receiving a PDF generated on the server side.
• PDF generation and the IIIF Image API are served with a different PHP configuration and set of processes than the rest of the frontend. This configuration significantly reduces available resources for these two functionalities.
• The option to generate PDFs featuring all images of an object with between 10 and 40 images has been entirely removed. Given its constraints, the feature was hard to explain and rarely accessible anyway. The primary “users” were noticeably AI scrapers.

Image Search

The image search feature was refactored and reduced to further separate it from the primary object search. The number of available search options has been reduced to be more easily explainable and reduce possibilities for very resource-intensive queries.

Batch Export of Object Metadata / OAI

Updated the LIDO API to almost entirely match the LIDO as generated during exports from musdb

Smaller Updates and Bugfixes

• Improved performance of object search by tags and places by filtering searched entities to those who are actually linked in the given instance of museum-digital.
• Object groups with only one object are henceforth not displayed and linked on object pages anymore
• Fixed link in footer: Clicking on “museum-digital” should lead to the home / start page of the given instance of musdb.
• Updated Swagger UI to version 5.30.3

nodac

• User-provided comments / hints have been removed for times (see above)
• Tooltips for linked objects now display which institution an object belongs to
  • This is particularly important for vocabulary editors who do not have access to the museums’ data. This way they get a limited preview with the required information for unpublished objects despite their otherwise lacking permissions.

This content is licensed under a Creative Commons Attribution 4.0 International license.

Development

musdb

• Added a tool for the AI-aided detection of displayed subjects in images for tagging
  • Has to be explicitly turned on on a per-collection basis, as it makes sense only for certain types of objects (like paintings, drawings, photographs)
  • Usable within the tagging overlay on object editing pages
  • See also: blog post about the feature (German)
• The maximum length of contents in the data field “edition” of literature entries has been extended to 50 characters
• Uploaded PDFs may now be up to 40 MB large
• New command line option to reset all permissions of a user account to the default provided by their user role
• New event type: “Changed”

nodac

• Added AI-generated suggestions for tag definitions and translations of tag names in a sidebar
  • This is also re-used to identify duplicate tags

Dissemination

• Poster presented at CoRDI 2025 (Archived) on August 27, 2025, in Aachen: “To Educate or to Enforce – The Case for Underhanded Methods to Improve Research Data”
  • PDF
  • Abstract / Zenodo

This content is licensed under a Creative Commons Attribution 4.0 International license.

New Features

• Tabs are available in the configuration when the site is installed as a Progressive Web App (PWA) (See also)

Improvements

• Duplicate search query parameters for objects associated with times before / after are removed.
  Example: If one searches for “Objects after 1900, whose first recorded associated time is also after 2000”, this is a duplicate query. 2000 is always after 1900, so one of the two parameters can be removed.
  Searches by time begin / end are also the foundation for the timeline. As the timeline is both beloved by web crawlers and resource-intensive to generate, this change significantly reduced server load.
• Any links on timeline pages that do not reference single object pages are marked as rel=nofollow
  Which is to say that bots are told to ignore them.

musdb

New Features

• Tabs are available in the configuration when musdb is installed as a Progressive Web App (PWA) (See also)
• New search option for objects: “Can be published”
  Aggregate search for objects that have not yet been published and which comply with the minimum requirements for publication.
• Separated measurements can now be batch-edited
  Available via “assign results”
• Institution-wide setting to enforce the use of the user-defined object editing interface for all users of the institution
• Exhibitions can now be searched via the API (/exhibition/list)
• A user’s user-agent is checked whenever a page is requested. If it changed, the user will be logged out automatically.
  Protection against session hijacking.

Improvements

• Panorama images for tours through exhibitions / institutions are calculated down to 2400 px height rather than 1400 px height
• The APIs for searching for entries in controlled vocabularies are now available via the main API
  See e.g. /actor/search_linked_to_objects/{term} , /actor/search/{term} etc.

Bugfixes

• Fix: “Visiting scientists” couldn’t open the “location” tab (this erroneously required museum-wide editing permissions)
• Fix: User-defined defaults for descriptions of new objects were ignored when new objects were to be added
• Fix: Thumbnails were displayed as duplicate images linked to exhibitions (tag “images”)
• Fix: Prefixing via the batch editing was broken

Importer

New Features

• New parser for CSV exports / imports from Robotron Daphne

Improvements

• CSVXML Parser
  • New literature-related fields (ISSN, editor, etc.) are now covered
  • References to wikidata are now respected as tags are imported
• The maximum length of any single given tag/keyword is centrally set to 95 characters

Bugfixes

• Some more recently added fields from the “admininstration” tag of musdb’s object pages had been read by the importer but their transfer into the database had not been implemented thus far.

nodac

New Features

• Tabs are available in the configuration when nodac is installed as a Progressive Web App (PWA) (See also)

CSVXML

Improvements

• Added new literature fields: type, editor, edition / issue, ISSN

Frontend

• Dates in TEI transcriptions are parsed, irrespective of whether when="" oder when='' was used
• Notes for markings are now publicly displayed
  This was missing thus far and is now implemented similar to how event notes are displayed. If a note exists, a small “[?]” appears behind the marking title line. Upon hovering over it, a tooltip appears with the relevant information.

musdb

• Names and descriptions of exhibitions and object groups can now be translated
• Version control
• Log of “current locations” of an object can be exported as a CSV file
• Uploaded object images can now be hidden or published in one batch operation
• API extended
  • (New functions)
  • Transfer object dimensions
  • List images and resources for an object
  • Image metadata
  • Publish / hide object images

January 2025

Frontend

• Objects can now be sorted by the aesthetics of the thumbnail (A dedicated blog post on this will follow soon)

musdb

• Batch transfer between between free text fields of object data
• Alignment of the maximum field length for notes on opening hours is now consistent between UI and database
• Bug fixed with switching between institutions during consistency checks fixed (relevant only to users with administrative access to multiple museums)
• Literature can now be searched by editors

Importer

• Core
  • Automatic transformation of life dates for actors
    • Year of death “01.01.2012” now becomes “2012”, instead of 01.01 as before
  • “?” and “(?)” are removed from the beginning and end of imported keywords
  • Various types of brackets in keyword names are converted to regular brackets
• Parser
  • Stricter internal implementation of settings, all imports can now implement the start_at setting
    • This is particularly useful for the repeated execution of imports that abort due to new, previously uncovered elements and other debugging.
  • New parsers:
    • MODS (mainly used in library contexts)
    • Parser for Exports from Faust for the Händel-Haus
    • Parser for XML dumps from MuseumPlus Classic (MsSQL > XML export per table > Import)
    • Bugfixes
      • Field “Verwender” in Primus parser was mapped to production events
      • Material / technology are now imported correctly in the parser for BeeCollect exports for the Industrial Museums of Saxony
• „Frontend“
  • CLI now also has options for switching off the import of individual areas
  • Help text for command line tool

nodac

• Splitting of keywords now also recognizes keywords that should be split into places, times, etc.
  • Bsp.: „Helm; Berlin“ > Schlagwort „Helm“ + Ort „Berlin“
  • Example: “helmet; Berlin” > keyword “helmet” + place “Berlin”
• When searching for keywords with ambiguous names, both keywords and generally ambiguous terms are now taken into account
• Times can now be merged with others directly from the time edit page

This content is licensed under a Creative Commons Attribution 4.0 International license.

On the one hand, using shared controlled vocabularies allows for a centralized editing team. Work that’s been done for one museum is available for all. This allows for a generally higher data quality, improved search options, etc. The only downside to centralized vocabularies is rules: Without everybody following a basic set of rules for the different vocabularies the vocabularies would surely be overcome by chaos quickly.

The two most basic rules are that 1) all entries need to be clearly identifiable and 2) actor names – from actual people to companies and governments to peoples – belong to the controlled vocabularies for actors; place names – cities, countries, streets – belong to the vocabulary for places, and so on. In effect, this means that an actor entry “Gaius Iulius Caesar” (better yet, “Gaius Iulius Caesar (-100–44)”) is a good one. It is clearly identifiable. An actor name “Bosch” should not end up in the controlled vocabularies – there are many people and companies sharing that name. If one means to actually record the German tool maker, one can specify as much using brackets and / or using the full name, e.g. “Robert Bosch GmbH” or “Bosch (tool making company)”.

musdb comes with selection lists enabling a simple reuse of existing vocabulary entries. Where a term or a name has not yet been recorded in museum-digital’s controlled vocabularies, the option to import vocabulary data from Wikipedia / Wikidata and the Gemeinsame Normdatei, and alternatively the need to add a description of at least ten characters enforce some level of identifiable and minimally researched entries.

And then there are imports. During imports, there is no way to force importing users to add additional data. To the contrary, especially during data migrations museums import data from different departments and different decades, leaving one with different spellings for the same person and a very varied quality of the recording overall. Importing thus often means bringing unclean, non-uniform data into the vocabularies. Often enough manual cleanup and enrichment are required to make the data usable – and to stop the “unclean” data from disturbing other users, e.g. by appearing in selection lists and then being falsely linked to another museums entries (a classic example are place entries like “Frankfurt”. Both Frankfurt an der Oder and Frankfurt am Main are significant places – how significant they are depends on where a museum may be located. Leaving the unidentifiable place (name) in the database will easily lead to other museums linking the same entry while actually referring to the other Frankfurt).

Automation can however significantly reduce the work of the colleagues manually editing the vocabularies while also allowing their efforts to have longer-lasting internal benefits to the system. The import tool thus features a number of checks to identify the entries actually referred to. Those checks relevant to the import of references to actors will be introduced over the rest of this blog post.

Rough outline

Actors from (or for) the controlled vocabularies usually enter museum-digital by being linked to objects via events. The same usually happens in the case of imports: The import data states that an object was, e.g., created by a given person. This translates to a new event (what happened to the object? And who did it when and where?) being created. The actor of this event is then set based on the provided name, optionally taking into accounts life dates and links to norm data repositories, in so far as they are made available in the import data.

Preparation

Cleaning the name

To identify the actor name, it is first cleaned and made slightly more uniform on a simple level of spelling.

• Leading and trailing spaces, semi colons, tab stops and newlines are removed
• Duplicate white spaces are removed
• Brackets of the different types (“()”, “{}”, “[]”) are replaced with simple brackets (“()”)
• Unwanted components / specifiers of a name are removed (e.g. “mythological creature” or empty brackets)
• Language-specific name components are replaced with their preferred variants in the vocabularies (German: “d.Ä.” is extended to “(der Ältere)” [the Elder])
• Indicators for uncertainty are stripped from the name, e.g. trailing question marks. They are separately used to identify the certainty of an actor link in a dedicated fields that describes exactly that.

Thus, a name stated in the import data as “; Hans Holbein d.Ä.?” will be imported as “Hans Holbein (der Ältere)”.

Parsing life dates from the input name

If no life dates have been provided explicitly in the import data, they may still be included as part of the actor name. Some museums whose database does not feature dedicated fields for an actor’s date of birth and death will append the years in brackets. Hence, a check is done to see if there are any brackets at the end of the name – and if so, whether the bracket contains a parsable time span. In this case, the time span identified will be used for the actor’s life dates and the brackets’ content will removed from the name (it is not actually a part of the actor’s name after all).

“Hans Holbein (der Ältere) (1465-1524)” with no explicitly provided years of birth and death will thus be transformed to “Hans Holbein (der Ältere)”, born 1465 and dead since 1524.

Identifying the actor

After preparing the inputted actor data, it is checked against the database in various ways. The listed checks below take place in a chronological order. If one returns a positive result and works to identify the actor, no following checks are run.

Checking based on norm data links

If links to external norm data repositories (the Library of Congress authority files, the National Diet Library, the French National Library, etc.) are provided in the import data, the actor vocabulary at museum-digital is checked whether it contains an entry featuring the same external ID. If that is the case, this actor is doubtlessly identified.

Checking global rewrites

For many names, there are different spellings. Similarly, many people use different names over the course of their lives. But the person referred to stays the same.

As not all names are significant enough to be listed as synonyms in their own right, museum-digital features a global (though language-specific) list of “permanent rewrites”. If a museum exported a reference to a “Julius Caesar”, this will be automatically rewritten to “Gaius Iulius Caesar”, thus identifying the person of the same name.

This central list is built and extended as the vocabulary editors do their work. Whenever two actor entries are merged in nodac, editors have the option to mark the name of the actor being merged into the other as to be permanently rewritten.

Checking institution-level rewrites

Whenever a new actor entry is added through musdb or by the way of imports, this is marked down in the database. Which museum used what term, resulting in which entry. Which entry may in the future be updated to refer to another. This is most simply explained using an example from the realm of places – the above-used example of “Frankfurt”:

If a museum newly brings “Frankfurt” to the controlled vocabulary for places, an additional entry is made in the database to note that the museum was the first to add an entry named “Frankfurt”, and that that finally resulted in the creation of a new place with the ID #99999. The object data itself makes it clear that the museum actually referred to “Frankfurt am Main” (ID: #217). If the vocabulary editors thus merge “Frankfurt” (#99999) with “Frankfurt am Main” (#217), this obviously cannot be a permanent rewrite. But the museum-specific log can be updated: The museum entered “Frankfurt”, and it referred to the entry #217 (Frankfurt am Main). If the museum thus enters “Frankfurt” without any further specification in the future, the importer and musdb can automatically identify Frankfurt am Main as the place actually referred to. The same logic applies to the import of actor data.

This check is obviously based on the assumption of some level of uniformity between different workers at the same museum. If a museum in Frankfurt an der Oder has almost all of their employees referring to Frankfurt an der Oder as the canonical “Frankfurt”, except for one, this will obviously backfire. But it is to be hoped that this case is rare enough – and that such a very different colleague would recognize their own non-conformity and choose to use the full names for both places to be able to clearly document (and communicate overall) with their colleagues.

Checking by name and life dates

If all the above checks did not work, the actor’s name and life dates are checked against the controlled vocabulary for actors itself. Does anybody with the same name, who was born and died at the provided years, exist in the actor vocabulary (either going by the language-specific translation of the name or by the “base entry” of an any language).

If that is not the case, the actor likely does not exist in the database yet.

Identification failed. Should a new entry be created?

If the actor does not exist in the database yet, they need to be added. Maybe.

Some names are known to be in clear conflict with the rules of the vocabulary for actors.

Checking the blacklist

museum-digital keeps a centralized blacklist for actor names that are known to be in clear conflict to the rules of the actor vocabulary. This list mainly contains either deliberately unspecific actor names (“unknown”; “unidentified”) or very unspecific ones (“John” [a random given name without a family name], “painter” [the name of a profession, not even of the actor themselves]).

If the entered actor name is contained in this blacklist, the attempt to link any actor to the event is aborted. The free-text note about the event is instead extended to mark that an actor of the blacklisted name was noted to be related to the event.

Checking if the actor is actually a place, time, or tag

A classic problem is vocabulary entries being entered in an unsuitable vocabulary – often for simple reasons likely colleagues slipping to a wrong column when working with a table calculation program. And thus one is asked to import an actor “14th century”.

The “14th century” is obviously a time, not an actor. For this, too, there is a centralized list that can be extended using nodac. Whenever the vocabulary editors move an entry from one vocabulary to another, they have the option to mark the entry’s name as always being of the target type.

Say the 14th century was previously entered as a tag. A vocabulary editor moved the entry to the vocabulary for times and marked the string “14th century” as one always referring to a time. If one then attempts to link a new actor “14th century”, the import tool can automatically recognize that this name actually refers to a time and link a time instead of an actor to the event. The attempt to add an actor is thus redirected to a different category.

All else failed: The actor is being added

If all the above checks did neither result in identifying a pre-existing record, nor in identifying the input name as invalid in one way or another, the actor is added to the controlled vocabularies and linked to the event. The entry now needs to be cleaned and enriched manually by the vocabulary editors.