musdb

Object overview

In the default tile view of the object overview page, hovering over an object image thus far revealed the object’s name. As object names are often too long to display fully and inventory numbers are the primary means of identifying an object in most museums, this preview text has now been extended to include the inventory numer.

User management

New Options for Managing User Accounts: Disabling Accounts & Setting Account Expiry Dates

Two new options on user editing pages allow disabling logins on an account and setting an expiry date for the account. Both can be useful for administration: If a new worker joins the museum for a project with a clear-cut limitation on funding and time, one can now set the account expiry at the beginning of the project to the end of it. The accounts will then automatically be deleted when the project ends. Similarly, colleagues that leave service temporarily but for a prolonged time (e.g. for a sabbatical) and will not need to use their accounts for that time can have their accounts disabled.

List of Terms of Use

A new tab on a user’s (own) account settings page provides the option to list all usage agreements / terms of use a user has agreed to in the context of their use of museum-digital / musdb and when they agreed to them.

Imports

Limiting Report Mail Size

When a user runs imports themselves using the WebDAV upload, the end of the import process – no matter if it is successful or fails – is marked by the sending of a report via mail. This report usually contains a list of noteworthy operations that happened during the import, e.g. which objects of which inventory number were imported to which object in musdb, identified by its ID. As imports grow, this list of operation grows. To not encounter issues sending the report, it is henceforth limited to a maximum of 2 MB or 10000 lines.

Dry-run Mode

Sometimes it is useful to try running an import to see if it will actually work but not actually process any data. This option has been available in the importer command line interface for a while, among others powering museum-digital:qa. It is now available in the import configuration for self-run imports as well using the setting dry-run. Enabling the setting accordingly stops the importer from actually writing the data into the database and changes the behavior if values that need to be mapped to values in controlled lists at museum-digital are encountered. Usually an import stops the moment such data is to be imported and not yet mapped. During a dry run, the error is collected and the import proceeds. All unmapped entries are listed together at the end of the import, allowing for a simpler mapping (possibly aided by concordance.museum-digital.org).

Dashboard

The first page of the dashboard, which for almost all users also means the start page of musdb right after the login process, was significantly reworked during the last month. The almost entirely unused notetaking features and discourse integration were removed in favor of a feed of recent blog posts. See also the section “Communications” in the respective blog post.

Annotations for the Vocabulary Editing Team

Each event, displayed as a tile on object editing pages, featured speech bubble icons behind each time / actor / place to provide additional comments and hints for the central vocabulary editing team. This positioning of the annotation feature led to confusion over the years, with some users using the feature to comment on the relationship between the entity and the object (for which the event notes should be used). We hence repositioned the links and moved them to the respective entity’s page (e.g. a place page for giving hints and comments on a place entry). The hinting / commenting feature for times has been altogether removed, as providing comments to clarify the meaning of e.g. a year never made much sense.

Smaller Updates and Bugfixes

• Fixed a bug in the HTML generated for listing other objects linked to an object. Links to the other object were broken and are not anymore.
• Image editing pages now embed the image directly instead of using the IIIF API. This reduces resource usage and increases stability at no cost.
• Removed option to manually trigger the rewriting of EXIF and IPTC metadata of object images. Rewriting takes place in the background whenever an image or a linked object is updated, making user-triggered updates obsolete.
• Re-introduce option to repeat linking to the last used linked object
• Updated Swagger UI to version 5.30.3

Frontend

As stated above and lengthily described in the previous blog posts (here, here, and here) we struggled with stability over the last month. This means that most changes in the frontend are aimed at improving stability.

Reworked Default Image Page

Thoroughly described in Updates, AI scrapers, and Resilience, we replaced the default view for single object image pages. While the default view was previously built around the IIIF viewer Mirador, the new default view uses OpenLayers and the unmediated image file for capabilities such as zooming. The new view also brings with it some new features, such as an option to reference specific sections of an image.

Serving Resource-Intensive Pages / Functionalities Only When Resources Are Available

PDF generation, the IIIF Image API, and the suggestions for alternative search queries on failed search pages are now limited to reduce their impact on the overall system stability. This follows two strategies:

• Suggestions on failed search pages and PDF generation will only appear if the overall load on the system is low. The threshold for when or when they are not provided is influenced by the user’s browser language: If a user uses a browser set to the primary language of a given instance of museum-digital (e.g. German in Hesse, Hungarian in Budapest), the threshold is much higher, meaning users will be able to access the pages at a medium server load. In the case of PDFs, high server load will forward users to the print dialogue for object pages instead of receiving a PDF generated on the server side.
• PDF generation and the IIIF Image API are served with a different PHP configuration and set of processes than the rest of the frontend. This configuration significantly reduces available resources for these two functionalities.
• The option to generate PDFs featuring all images of an object with between 10 and 40 images has been entirely removed. Given its constraints, the feature was hard to explain and rarely accessible anyway. The primary “users” were noticeably AI scrapers.

Image Search

The image search feature was refactored and reduced to further separate it from the primary object search. The number of available search options has been reduced to be more easily explainable and reduce possibilities for very resource-intensive queries.

Batch Export of Object Metadata / OAI

Updated the LIDO API to almost entirely match the LIDO as generated during exports from musdb

Smaller Updates and Bugfixes

• Improved performance of object search by tags and places by filtering searched entities to those who are actually linked in the given instance of museum-digital.
• Object groups with only one object are henceforth not displayed and linked on object pages anymore
• Fixed link in footer: Clicking on “museum-digital” should lead to the home / start page of the given instance of musdb.
• Updated Swagger UI to version 5.30.3

nodac

• User-provided comments / hints have been removed for times (see above)
• Tooltips for linked objects now display which institution an object belongs to
  • This is particularly important for vocabulary editors who do not have access to the museums’ data. This way they get a limited preview with the required information for unpublished objects despite their otherwise lacking permissions.

This content is licensed under a Creative Commons Attribution 4.0 International license.

• On source / reference pages, linked objects are now sorted by the position within the source work on which they are referenced or which they do themselves reference
• The target URL of the regular / unspecified search bar for objects now follows the new, prettier URL schema
• Support for an OAI-PMH API for object metadata
  • Standardized endpoint for aggregators seeking to retrieve data in batch
  • Metadata formats thus far supported:
    • LIDO
    • OAI-DC (mandatory)
  • See also: Blog
• PDFs are only generated for users with a browser set to a non-default language if load on the server is low
  • The resource use caused by AI bots scraping museum-digital has been growing and growing. Generally, we see bots included in our mission to enable access to cultural heritage. On the other hand, nobody is served if the service is bogged down by bots. One functionality that is commonly used among bots and resource intensive is the generation of PDFs for object pages. The same information can be loaded from the object page itself and printed to a PDF using the browser’s print option. There are thus rather few downsides to limiting access to PDF generation to timmes, when server load is low. So that’s what we did.
• Collection-specific ISIL identifiers are now also used in the LIDO API
• Alternative numbers of an object can now be displayed on object pages
  • This includes tooltips for types of alternative numbers, that can be set by the museum on the institution-wide settings pages of musdb

musdb

• Search for objects
  • Type-ahead search for languages (of the object’s content)
  • Search by object’s revision status (open, read-ony, archived, etc.)
• Batch editing of objects’ revision status
• Parameters of the full text search index were updated to improve the search of word compounds in German

Importer

Core

• The dry-run mode now does not abort an import anymore, if an unmapped value is encountered. Unmapped entries are collected and displayed together afterwards.
  • This means, that unmapped entries can now much more easily be copied to and mapped in concordance.museum-digital.org
• Support for the import of alternative numbers (of objects)
• Support for the import of space hierarchies

Parser

• AdlibXml
  • Support for importing objects’ alternative numbers
• CsvXml
  • Support for importing objects’ alternative numbers
• CsvLocations
  • New parser for csv-based imports of space hierarchies
• ImageByInvno
  • New setting: append_chars (Adds suffixes, that exist in the inventory number, but not in file names)

This content is licensed under a Creative Commons Attribution 4.0 International license.

Frontend

• Objects that are linked to a source as referencing the source or being referenced in it are now listed on the source’s page
  • Example: “Novalis Schriften. Die Werke Friedrich von Hardenbergs. Historisch-kritische Ausgabe. Erster Band: Das dichterische Werk. 3. Auflage”
• The object page now displays the new transcription fields for notes, status, and type of the transcription
• New types of classifications for the link between an object and a tag
  • Taxon
  • Topic
  • Mentioned subject (similar to the existing “display subject”)
• Dependencies
  • Updated OpenLayers to Version 10.6

musdb

• Renaming vocabulary entries to blacklisted terms is now impossible
  • Someone managed to create an actor “unknown” by adding a different actor and renaming the new actor entry to “unknown” afterwards.
• Added a sidebar to object group overview pages for additional filtering options
• New types of classifications for the link between an object and a tag
  • Taxon
  • Topic
  • Mentioned subject (similar to the existing “display subject”)
• New APIs for listing all vocabulary entries linked to the objects of a museum
• Dependencies
  • Updated OpenLayers to Version 10.6

Importer

• Core
  • Deaccessions are now covered by the import tool
  • Recipients of deaccessions are linked to the address book / contact list
• Parser
  • CSVXML: The CSVXML parser now covers deaccessions as well
  • ImageByInvno: Added an option to ignore any characters before a given starting combination of characters (if there is a consistent start of the inventory number).

CSVXML

• New fields
  • tag_related_identifier_type
  • tag_related_identifier

Dissemination

• Presentation “Von museum-digital zum eigenen Online-Katalog” for the Working Group Digitization of the State Museum Association of Rhineland-Palatine
  • Slides: PDF
  • Slides: ODP

This content is licensed under a Creative Commons Attribution 4.0 International license.

Generally, the pace and type of development seems to have changed this year. Rather than doing many small improvements all over the place, there is less but larger and more labor intensive changes and new features. See for example the versioning in musdb (January), the tool for automating imports based on what others call a “hot folder” and the sort option to sort objects by their images’ aesthetics score (both presented here in March), and the new tool for suggesting formulations for object descriptions using large language models (June).

July

Frontend

• Translation of the software to Hindi and Telugu
• Grouping of tags by their relation to a given object
  If an object is linked to more than 10 tags, the tag list of object pages quickly starts looking unorganized and messy. In such cases, the tags will now be displayed grouped by their relationship to the object (thus far: general tag, material, technique, object type, display subject)

musdb

New Features

• Export option for the specific LIDO as expected by the German Digital Library’s “colonial contexts” portal

Improvements & Changes

• The minimum length of fulltext search terms for object search parameters is now visibly enforced in the extended search user interface
  To not overly burden the fulltext search server, any term in a full text search in musdb needs to be a minimum of two characters long. Thus far, shorter full text search parameters were simply ignored. This certainly was confusing at times. Since June, attempting to perform search queries with shorter search parameters is made impossible by a check in the extended search overlay.
• Reception history of objects: Statements of the relevant position within a source can now be 40 characters long
• Transcriptions
  • May now be up to 4000000 long
  • New fields: Notes on the transcription, status, aims

Bugfixes

• Fixed a bug in the batch editing of specific visibility flags for data fields on the addendum tab

Juni

Frontend

• Performance improvements
  • Object search now runs without a connection to the full text search server if no full text search parameter is set
  • If multiple search parameters for an earliest / latest time have been set, they are parsed and combined before being forwarded to the database
• Improvements in the deletion of temporarily created PDF files (PDF export)
• Navigation has been translated to Tamil

musdb

New Features

• Recipient of deaccessed objects can now be linked from within the address book
• New tool for AI-aided formulation o object descriptions (based on existing other metadata)

Improvements

• Object search now runs without a connection to the full text search server if no full text search parameter is set

Importer

• The CSVXML parser has been extended to cover new event types and markings
• Object groups automatically generated to group all objects of an import can now receive a description as set within the import configuration

nodac

The list of selectable languages for the navigation of nodac has now been restricted to those in which there is actually a complete translation

New Features

• Tabs are available in the configuration when the site is installed as a Progressive Web App (PWA) (See also)

Improvements

• Duplicate search query parameters for objects associated with times before / after are removed.
  Example: If one searches for “Objects after 1900, whose first recorded associated time is also after 2000”, this is a duplicate query. 2000 is always after 1900, so one of the two parameters can be removed.
  Searches by time begin / end are also the foundation for the timeline. As the timeline is both beloved by web crawlers and resource-intensive to generate, this change significantly reduced server load.
• Any links on timeline pages that do not reference single object pages are marked as rel=nofollow
  Which is to say that bots are told to ignore them.

musdb

New Features

• Tabs are available in the configuration when musdb is installed as a Progressive Web App (PWA) (See also)
• New search option for objects: “Can be published”
  Aggregate search for objects that have not yet been published and which comply with the minimum requirements for publication.
• Separated measurements can now be batch-edited
  Available via “assign results”
• Institution-wide setting to enforce the use of the user-defined object editing interface for all users of the institution
• Exhibitions can now be searched via the API (/exhibition/list)
• A user’s user-agent is checked whenever a page is requested. If it changed, the user will be logged out automatically.
  Protection against session hijacking.

Improvements

• Panorama images for tours through exhibitions / institutions are calculated down to 2400 px height rather than 1400 px height
• The APIs for searching for entries in controlled vocabularies are now available via the main API
  See e.g. /actor/search_linked_to_objects/{term} , /actor/search/{term} etc.

Bugfixes

• Fix: “Visiting scientists” couldn’t open the “location” tab (this erroneously required museum-wide editing permissions)
• Fix: User-defined defaults for descriptions of new objects were ignored when new objects were to be added
• Fix: Thumbnails were displayed as duplicate images linked to exhibitions (tag “images”)
• Fix: Prefixing via the batch editing was broken

Importer

New Features

• New parser for CSV exports / imports from Robotron Daphne

Improvements

• CSVXML Parser
  • New literature-related fields (ISSN, editor, etc.) are now covered
  • References to wikidata are now respected as tags are imported
• The maximum length of any single given tag/keyword is centrally set to 95 characters

Bugfixes

• Some more recently added fields from the “admininstration” tag of musdb’s object pages had been read by the importer but their transfer into the database had not been implemented thus far.

nodac

New Features

• Tabs are available in the configuration when nodac is installed as a Progressive Web App (PWA) (See also)

CSVXML

Improvements

• Added new literature fields: type, editor, edition / issue, ISSN

• A Kannada translation of the software is now available
• Museums can now select to display their own citation notes in the menu for such on object pages
  This is especially relevant in case the object itself is to be cited (rather than its record online).
• “Or” search queries can be combined within one search parameter for select attribute search types, e.g. places and tags. The Syntax is as follows: place:61~1
  • For now, this can only be used using the internal query language. As such, there is no corresponding option in the UI for search settings. On the other hand, the search option is thus available using the API.
  • This option is not available when searching for times or full events

musdb

• Institution-wide settings
  • Free text fields that double with similar controlled fields may now be hidden
    • The acquisition of an object may e.g. be directly recorded on the context of a given object or as a separate acquisition process. Recording it as an acquisition process is slightly more labor-intensive, but allows a more fine-grained and accurate documentation. With the new setting, the data fields for recording acquisitions directly in the object context can be hidden from all users of a museum to ensure a uniform use of the preferred functionality.
    • Institution-specific notes on how to cite objects may now be recorded for display on published object pages.
• Search queries for inventory numbers are now character-level searches again (rather than following a fulltext search logic)
• Some event types are incomplete, i.e. they cannot contain a place, or an actor, or a time. This incompleteness was handled differently between musdb, the import tool, and the CSVXML import preparation tool. Now, it is determined by a centralized list and thus similar across all of museum-digital.
• Refactoring of the administrative command line interface
  This mainly concerns auto-correction tools, but also results in that exports for the quick export option are now automatically generated daily.
  
• Separated measurements are now positioned at the very top of the “addendum” tab of object editing pages.
• It is now possible to record web links for object groups

Importer

• The import tool can now be used as a harvester
  • First use case is a harvester for LIDO records delivered via an OAI-PMH interface.
• Externally stored images in formats other than JPG can now be imported
• Significantly extended the LIDO parser to (among others) support the import of multilingual object data

This content is licensed under a Creative Commons Attribution 4.0 International license.

Here, as so often, in a list format:

Frontend

• Bugfix: The detailed description was not reflected in the object API even if set to be so via musdb
• Translation: The frontend is now available in Kannada

musdb

• Bugfix: Fixed error in setting up new two factor authentication via TOTP (app-based 2fa)
• Bugfix: Symbols for image rotation switched (counter-clockwise symbol rotated clockwise and vice-versa)
• Feature: If the generation of a PDF catalogue is triggered via the sidebar of series editing pages, the objects will follow their order within the object group

Importer

• If no explicit name/title for a resource (audio, video, PDF, 3D, externally hosted image) was supplied, there was no fallback. Now, the importer follows the same logic as musdb and will fall back to using the linked object’s object name as a resource title if none is supplied.
• The CSVXML parser can now handle multiple objects per XML file.

Auto uploader

The auto uploader is an entirely new tool. When first started, users will be asked to enter the necessary data for running imports. And they are asked for the path to a folder which will subsequently be monitored for automatic uploads.

Whenever the tool then detects suitable contents for an import in the folder, the data will be uploaded and automatically imported. Intermediate steps like the generation of a settings file for the import are covered by the tool.

The tool can thus be useful for museums that regularly import data formed in a consistent format – say, museums using museum-digital for publishing while mainting a separate collection management system. It should be uninteresting to those who spend much effort on preparing imports (e.g. via CSVXML) or who want to migrate to musdb altogether, as it only becomes useful in the case of regular use.

The code is available here, licensed under GPL v3.
See also the more detailed German-language blog post about the auto uploader.

This content is licensed under a Creative Commons Attribution 4.0 International license.

Frontend

• Dates in TEI transcriptions are parsed, irrespective of whether when="" oder when='' was used
• Notes for markings are now publicly displayed
  This was missing thus far and is now implemented similar to how event notes are displayed. If a note exists, a small “[?]” appears behind the marking title line. Upon hovering over it, a tooltip appears with the relevant information.

musdb

• Names and descriptions of exhibitions and object groups can now be translated
• Version control
• Log of “current locations” of an object can be exported as a CSV file
• Uploaded object images can now be hidden or published in one batch operation
• API extended
  • (New functions)
  • Transfer object dimensions
  • List images and resources for an object
  • Image metadata
  • Publish / hide object images

January 2025

Frontend

• Objects can now be sorted by the aesthetics of the thumbnail (A dedicated blog post on this will follow soon)

musdb

• Batch transfer between between free text fields of object data
• Alignment of the maximum field length for notes on opening hours is now consistent between UI and database
• Bug fixed with switching between institutions during consistency checks fixed (relevant only to users with administrative access to multiple museums)
• Literature can now be searched by editors

Importer

• Core
  • Automatic transformation of life dates for actors
    • Year of death “01.01.2012” now becomes “2012”, instead of 01.01 as before
  • “?” and “(?)” are removed from the beginning and end of imported keywords
  • Various types of brackets in keyword names are converted to regular brackets
• Parser
  • Stricter internal implementation of settings, all imports can now implement the start_at setting
    • This is particularly useful for the repeated execution of imports that abort due to new, previously uncovered elements and other debugging.
  • New parsers:
    • MODS (mainly used in library contexts)
    • Parser for Exports from Faust for the Händel-Haus
    • Parser for XML dumps from MuseumPlus Classic (MsSQL > XML export per table > Import)
    • Bugfixes
      • Field “Verwender” in Primus parser was mapped to production events
      • Material / technology are now imported correctly in the parser for BeeCollect exports for the Industrial Museums of Saxony
• „Frontend“
  • CLI now also has options for switching off the import of individual areas
  • Help text for command line tool

nodac

• Splitting of keywords now also recognizes keywords that should be split into places, times, etc.
  • Bsp.: „Helm; Berlin“ > Schlagwort „Helm“ + Ort „Berlin“
  • Example: “helmet; Berlin” > keyword “helmet” + place “Berlin”
• When searching for keywords with ambiguous names, both keywords and generally ambiguous terms are now taken into account
• Times can now be merged with others directly from the time edit page

This content is licensed under a Creative Commons Attribution 4.0 International license.

Say, a museum’s previous database used actor roles over an event structure to express who created an object. As such, the museum entered that the object has a linked actor X that is linked to the object as a “main creator” and a linked time Y marked as the “creation time”. During the import, these roles (“main creator” and “creation time”) are then translated to museuem-digital’s event types to form an event: The object was created by actor X at the time Y. This works, because the terms “main creator” and “creation time” have been matched to the creation event type.

If a term is not yet matched to a corresponding value of a controlled list in museum-digital, the importer will simply abort the import. On the one hand this is a way to uselessly require resources for an import that cannot be completed anyway. On the other, it is tedious. One recognizes yet unmatched entries only one by one.

A Small New Tool

A small new tool, available at concordance.museum-digital.org, makes the process a bit less tedious. Users can upload all the import data from a given field (e.g. the actor roles) – one a line – and check whether they are already matched using the concordance lists or not.

For entries that are not yet matched, the tool will offer selection boxes to perform the matching using the graphical user interface. Once all entries have been matched, one can then generate the relevant lines of code to enter the missing entries to the concordance list upon the click of a button.

While simply checking and extending the relevant open source lists should be trivial even to most non-technical users, this way is certainly more convenient. Importantly, it also removes the need to run the import multiple times until one does not encounter errors caused by unmatched entries anymore. And, well, it certainly is also more convenient to match to regular human language values than to the internal IDs of the target values.

The concordance checker’s MIT-licensed code can be found here.

This content is licensed under a Creative Commons Attribution 4.0 International license.

• Basic re-implementation of separated measurements for objects (tab: “addendum”)
• The type of measurement (width, length, etc.) is now managed using a controlled list, which can easily be extended. This also allows for measurement types of different levels of specificity (“width” vs. “width of socle”)
• The new implementation allows users to identify whether a measurement is exact and add notes for each measurement
• Measured values now need to be entered as a floating point number – consistent search for objects smaller or larger than a given size is thus made possible (previously only available for entries where the system could deduce a numeric value from whatever had been entered)
• Unique naming components of entered tags are automatically parsed into a relation type. German: “Apfel (Motiv)” is automatically split into the tag “Apple” and the relation type “display subject”.
• Users entering a tag with such a naming component that is known to belong to another vocabulary will have their input auto-corrected to reflect both. Entering the tag “Berlin (Motiv)” will be auto-corrected to a link to a “displayed place” “Berlin”.
• The list of spaces that can be linked to an object as its current location is now sorted alphabetically

Frontend

• Re-implemented separated Measurements

Import

• The import tool now also uses the new implementation of separated measurements
• Unique naming components of entered tags are automatically parsed into a relation type. German: “Apfel (Motiv)” is automatically split into the tag “Apple” and the relation type “display subject”.
• Users entering a tag with such a naming component that is known to belong to another vocabulary will have their input auto-corrected to reflect both. Entering the tag “Berlin (Motiv)” will be auto-corrected to a link to a “displayed place” “Berlin”.
• Links between two objects are now imported using the dedicated data type for this purpose. They had previously been imported as regular web links.

nodac

• When merging two entries, links between the entries and collections, exhibitions, etc. are now reflected and rewritten
• previously, links to such data types prevented the completion of the merge
• If an entry’s name is marked to always belong to e.g. a tag, the buttons for transferring the entry to the actor, place, or time vocabularies are now hidden.
• If an entry is moved between vocabularies, links between the entry and objects as “display subject” / “displayed place” / “displayed actor” are reflected and translated into new links using the appropriate link type.
• A new context menu on overview pages allows for a quick access to functionalities like the splitting of entries

These are in the form of listings, and this is how it should be here too.

Frontend

Features & Improvements

• Some improvements in background scripts, especially better handling of timeouts when calculating “Similar objects” in very large instances
• Contributors, linked locations and times for an object group are now listed alphabetically by name
• Table headers for event components (who, when, where) are now only displayed in the A4 PDF if there is also content for the row
• New search option for object searches: “Is currently on display”
• Links to the Themator now use the new URL scheme of the Themator
  (https://themator.museum-digital.de/t/690 instead of https://themator.museum-digital.de/ausgabe/showthema.php?m_tid=690&tid=690)

Bugfixes

• Error when searching for controlled list terms that contained multiple spaces via the “Refine search” overlay (search for license “Public Domain Mark”)
• Exactness setting in the “refine search” overlay was not transferred to the actual search query
• Simple embedding of an object (analogous to YouTube videos, for example; accessible via the “Cite” menu of an object page) had various errors / now works again

musdb

Features & Improvements

• In the API documentation of musdb there is now a note that the frontend also has an API
  • Frontend API
    • You do not need to authenticate yourself to use the frontend API
    • The frontend API tends to be faster and easier to use
    • Is read-only
  • musdb API
    • Can do more: Can also see non-public stocks and fields / data types
    • Is much more granular (more queries for the same data, but you likely get exactly the data you are looking for instead of e.g. all data known about a given object)
    • Can be used for writing data
• Suggestion lists when searching for vocabulary terms in the side column of the object search page have been revised
  • Tooltips appear when hovering over
  • Implementation in Vanilla JS, removing jQuery
  • (this means significantly better performance of the search results list in list format, because jQuery no longer needs to be loaded)

• User page / Login
  • Log of logins now also with IP and user agents
  • Login via login persisted in the browser (“Remember me”) is logged and displayed
  • All browsers permanently logged in via cookie are forced to log in again after a password change
  • New option to invalidate all remembered logins on other devices (browser must be logged in again)

• Object
  • More restrictions for the publication of object data records.
  • An object can no longer be published if:
    • … the object name is the same as the object description
    • … the description contains the character string “lorem ipsum”
  • When object entries are unpublished / hidden, the images linked to the image are renamed (thus invalidating links to the images). When publishing the object again, this is reversed so that existing links work again.
  • Spaces in selection lists are now listed alphabetically as the actual location when linking
• Literature
  • Acquisitions can now be linked to literature
    • Previous owners etc. can thus be linked to a literature entry
  • Spaces (actual location) can be linked to literature

Bugfixes

• Overlay for setting searches for objects: Multi-word search terms were converted into multiple searches instead of being searched as a string of words (“red helmet” > “red” AND “helmet” instead of “red helmet”)
• Error when searching for controlled list terms that contained multiple spaces via the “Refine search” overlay (search for license “Public Domain Mark”)

Importer

• Link between literature and spaces (actual location) as well as acquisitions is implemented in the “core” of the import tool
• ImageByInvno parser (assignment of images to objects via inventory numbers contained in the file name) can now be used to import PDF files

files.museum-digital.org

• Added a small script to enhance PDF metadata based on an XML sidecar file. See e.g.: https://files.museum-digital.org/de/Praesentationen/2024-10-18_md-deutschland-eV-stellt-sich-vor_Usertreffen_MA.xml

Main post image generated using illustriousXL_smoothftSPO

This content is licensed under a Creative Commons Attribution 4.0 International license.

Coming from that basic idea, there is now a second use case for museum-digital:qa: It is now capable of being used as a conversion tool. As usual, most formats supported for imports to museum-digital can be used to upload data – e.g. a simple CSV structure we regularly use for importing data previously held in table calculation programs – that is then converted to those XML formats to which users can export their object data in musdb (again recycling code from there).

At the moment conversions to LIDO (both versions 1.0 and 1.1) and EODEM are supported. Especially the latter may prove interesting: EODEM is a very recent extension of LIDO aimed at enabling a much simplified data exchange between different institutions in the context of loans. The lending museum exports the relevant objects’ data from their collection management system in EODEM; the borrowing institution then imports it and immediately has all the relevant information in their respective collection management system without having to manually re-type the data as has been usual for the last years. EODEMs utility in practice however depends on how widely it will be supported by the different collection management systems.

By offering a conversion option from different formats to EODEM, museum-digital:qa may thus help to make the benefits of EODEM available also to those museums whose collection management software does not yet support exports to the standard.

Image: Bell Telelphone crew with a dog, ca. 1892

This content is licensed under a Creative Commons Attribution 4.0 International license.

For time names a much more extensive cleaning is done, both in the case of imports and when working directly in musdb. Time names in the sense of the controlled vocabulary of museum-digital describe clearly defined times. This may be timespans, years, or full dates. But the number of possible input values is thus already significantly limited. In fact, it is small enough, and timespans, years, etc. are uniform enough to allow the automatic parsing of time names. Where that is possible, they are automatically rewritten to their default form in the controlled vocabulary and automatically translated to some 30 languages.

In the case of place names a similar cleaning and consolidating of place names was limited to the simple stripping of leading and trailing spaces and commas and the removal of indicators for uncertainty (e.g. trailing question marks). Since the weekend, place names are more extensively rewritten to ease the work of the vocabulary editors and ensure a more immediately consistent data entry both in musdb and when importing.

General rewriting

1. Simple spelling issues

The very first step in rewriting entered place names remains the removal of superfluous characters. As such, duplicate white spaces, trailing commas, etc. are removed.

Example: , Berlin , > Berlin

2. Removal of indicators for uncertainty

Next, known indicators for uncertainty are used to update the dedicated flag describing the certainty of the link to the place and then stripped. As they indicate the relation between object and place, they are not actually part of the place name and can thus be removed safely.

Example 1: Berlin ? > Berlin
Example 2: Maybe Budapest > Budapest

3. Removal of duplicates in enumerations

Commas are used in input place names in two ways. On the one hand, they may be used to further specify a place name (Beijing, PRC), on the other, they are often used in import data to designate multiple space names at a time (Beijing, Tokyo, Nairobi; this contradicts the logic of a database altogether and needs to be cleaned up manually be the vocabulary editors).

In both cases, duplicate names in the enumeration are superfluous and can be removed.

Example 1: Berlin, Germany, Germany > Berlin, Germany

4. Language-dependent rewriting

The next steps in rewriting entries depend on the language of the entry. In musdb, the language the user set to use musdb is used to guess the language they are entering data in. In the case of imports, the default language of the given instance of museum-digital that a museum imports to is used.

4.1. Extension of common abbreviations

Where there are abbreviations, there are also unabbreviated names. And surely, both will be used. Will inevitably leads to duplicate entries. In the case of some common abbreviations, they are thus automatically rewritten to a canonical form.

Example (German): Adalberthstr. 12 (Berlin) > Adalbertstraße 12 (Berlin)
Example (Hungarian): Vaci u. 12 (Budapest) > Vaci utca 12 (Budapest)

4.2. Reordering names in commas based on indicators for more specific place names

As stated above (3.), commas may either indicate a specification of a single place, or they may indicate that the entry actually refers to more than one place. Some components of a name can be used to almost certainly determine that the former is the case – and which place of the given list is the specific one and which one is a superordinate named mainly for clarification. Common such names are “street”, “plaza”, “pier”.

If there is exactly one comma and such a name component is encountered, the entered place name can be rewritten to contain the less specific name only in brackets.

Example (German): Berlin, Adalberthstraße 12 > Adalbertstraße 12 (Berlin)
Example (Hungarian): Vaci utca 12, Budapest > Vaci utca 12 (Budapest)

If both names contain such an indicator, no rewriting is applied. Vaci utca 12, Vaci utca 13‘ is thus not rewritten.

4.3. Budapest special: Extending the names of districts

Street names in Budapest are usually referred to including the naming of the district. These districts are referred to in a number of ways. If they are referred to using roman numerals, this is automatically extended to the canonical form.
This rewrite is only applied if the language is set to Hungarian.

Example: Petőfi Sándor utca 3. Budapest, IV. > Petőfi Sándor utca 3. (Budapest, 4. kerület)

4.4. Reordering names in commas based on country names

Similar to the rewrites described in 4.2., country names can be used to indicate a hierarchical relationship between two places in a comma-separated list. If one given name is a country name and the other is not, it is likely that the non-country name is part of the given country. The comma can be replaced with brackets while the name can be reordered into the preferred specific (unspecific) form.
This check is also applied to names separated by hyphens.

Example (German): Budapest, Ungarn > Budapest (Ungarn)
Example (Hungarian): Berlin-Németország > Berlin (Németország)

There are however some common cases in which this logic does not apply – significantly cardinal directions. If one such term is found to be the non-country part, the rewrite is not applied. As such West-Deutschland remains West-Deutschland without being rewritten.

The list of names of countries and historical countries used stems from Wikidata (thanks!).

Where do these rewrites apply?

The rewrites listed above are now implemented in musdb, the import tool, and nodac.

They have also been used to consolidate existing place names, allowing us to identify some 500 duplicate place entries over the weekend (that amounts to almost 0.7 percent of the whole vocabulary). Clearly, identifying similar cases of regularly appearing, varied ways to express the same thing, and determining a canonical way of naming places in such cases holds a lot of potential for reducing the editors workload and improving data quality for everybody.

On the one hand, using shared controlled vocabularies allows for a centralized editing team. Work that’s been done for one museum is available for all. This allows for a generally higher data quality, improved search options, etc. The only downside to centralized vocabularies is rules: Without everybody following a basic set of rules for the different vocabularies the vocabularies would surely be overcome by chaos quickly.

The two most basic rules are that 1) all entries need to be clearly identifiable and 2) actor names – from actual people to companies and governments to peoples – belong to the controlled vocabularies for actors; place names – cities, countries, streets – belong to the vocabulary for places, and so on. In effect, this means that an actor entry “Gaius Iulius Caesar” (better yet, “Gaius Iulius Caesar (-100–44)”) is a good one. It is clearly identifiable. An actor name “Bosch” should not end up in the controlled vocabularies – there are many people and companies sharing that name. If one means to actually record the German tool maker, one can specify as much using brackets and / or using the full name, e.g. “Robert Bosch GmbH” or “Bosch (tool making company)”.

musdb comes with selection lists enabling a simple reuse of existing vocabulary entries. Where a term or a name has not yet been recorded in museum-digital’s controlled vocabularies, the option to import vocabulary data from Wikipedia / Wikidata and the Gemeinsame Normdatei, and alternatively the need to add a description of at least ten characters enforce some level of identifiable and minimally researched entries.

And then there are imports. During imports, there is no way to force importing users to add additional data. To the contrary, especially during data migrations museums import data from different departments and different decades, leaving one with different spellings for the same person and a very varied quality of the recording overall. Importing thus often means bringing unclean, non-uniform data into the vocabularies. Often enough manual cleanup and enrichment are required to make the data usable – and to stop the “unclean” data from disturbing other users, e.g. by appearing in selection lists and then being falsely linked to another museums entries (a classic example are place entries like “Frankfurt”. Both Frankfurt an der Oder and Frankfurt am Main are significant places – how significant they are depends on where a museum may be located. Leaving the unidentifiable place (name) in the database will easily lead to other museums linking the same entry while actually referring to the other Frankfurt).

Automation can however significantly reduce the work of the colleagues manually editing the vocabularies while also allowing their efforts to have longer-lasting internal benefits to the system. The import tool thus features a number of checks to identify the entries actually referred to. Those checks relevant to the import of references to actors will be introduced over the rest of this blog post.

Rough outline

Actors from (or for) the controlled vocabularies usually enter museum-digital by being linked to objects via events. The same usually happens in the case of imports: The import data states that an object was, e.g., created by a given person. This translates to a new event (what happened to the object? And who did it when and where?) being created. The actor of this event is then set based on the provided name, optionally taking into accounts life dates and links to norm data repositories, in so far as they are made available in the import data.

Preparation

Cleaning the name

To identify the actor name, it is first cleaned and made slightly more uniform on a simple level of spelling.

• Leading and trailing spaces, semi colons, tab stops and newlines are removed
• Duplicate white spaces are removed
• Brackets of the different types (“()”, “{}”, “[]”) are replaced with simple brackets (“()”)
• Unwanted components / specifiers of a name are removed (e.g. “mythological creature” or empty brackets)
• Language-specific name components are replaced with their preferred variants in the vocabularies (German: “d.Ä.” is extended to “(der Ältere)” [the Elder])
• Indicators for uncertainty are stripped from the name, e.g. trailing question marks. They are separately used to identify the certainty of an actor link in a dedicated fields that describes exactly that.

Thus, a name stated in the import data as “; Hans Holbein d.Ä.?” will be imported as “Hans Holbein (der Ältere)”.

Parsing life dates from the input name

If no life dates have been provided explicitly in the import data, they may still be included as part of the actor name. Some museums whose database does not feature dedicated fields for an actor’s date of birth and death will append the years in brackets. Hence, a check is done to see if there are any brackets at the end of the name – and if so, whether the bracket contains a parsable time span. In this case, the time span identified will be used for the actor’s life dates and the brackets’ content will removed from the name (it is not actually a part of the actor’s name after all).

“Hans Holbein (der Ältere) (1465-1524)” with no explicitly provided years of birth and death will thus be transformed to “Hans Holbein (der Ältere)”, born 1465 and dead since 1524.

Identifying the actor

After preparing the inputted actor data, it is checked against the database in various ways. The listed checks below take place in a chronological order. If one returns a positive result and works to identify the actor, no following checks are run.

Checking based on norm data links

If links to external norm data repositories (the Library of Congress authority files, the National Diet Library, the French National Library, etc.) are provided in the import data, the actor vocabulary at museum-digital is checked whether it contains an entry featuring the same external ID. If that is the case, this actor is doubtlessly identified.

Checking global rewrites

For many names, there are different spellings. Similarly, many people use different names over the course of their lives. But the person referred to stays the same.

As not all names are significant enough to be listed as synonyms in their own right, museum-digital features a global (though language-specific) list of “permanent rewrites”. If a museum exported a reference to a “Julius Caesar”, this will be automatically rewritten to “Gaius Iulius Caesar”, thus identifying the person of the same name.

This central list is built and extended as the vocabulary editors do their work. Whenever two actor entries are merged in nodac, editors have the option to mark the name of the actor being merged into the other as to be permanently rewritten.

Checking institution-level rewrites

Whenever a new actor entry is added through musdb or by the way of imports, this is marked down in the database. Which museum used what term, resulting in which entry. Which entry may in the future be updated to refer to another. This is most simply explained using an example from the realm of places – the above-used example of “Frankfurt”:

If a museum newly brings “Frankfurt” to the controlled vocabulary for places, an additional entry is made in the database to note that the museum was the first to add an entry named “Frankfurt”, and that that finally resulted in the creation of a new place with the ID #99999. The object data itself makes it clear that the museum actually referred to “Frankfurt am Main” (ID: #217). If the vocabulary editors thus merge “Frankfurt” (#99999) with “Frankfurt am Main” (#217), this obviously cannot be a permanent rewrite. But the museum-specific log can be updated: The museum entered “Frankfurt”, and it referred to the entry #217 (Frankfurt am Main). If the museum thus enters “Frankfurt” without any further specification in the future, the importer and musdb can automatically identify Frankfurt am Main as the place actually referred to. The same logic applies to the import of actor data.

This check is obviously based on the assumption of some level of uniformity between different workers at the same museum. If a museum in Frankfurt an der Oder has almost all of their employees referring to Frankfurt an der Oder as the canonical “Frankfurt”, except for one, this will obviously backfire. But it is to be hoped that this case is rare enough – and that such a very different colleague would recognize their own non-conformity and choose to use the full names for both places to be able to clearly document (and communicate overall) with their colleagues.

Checking by name and life dates

If all the above checks did not work, the actor’s name and life dates are checked against the controlled vocabulary for actors itself. Does anybody with the same name, who was born and died at the provided years, exist in the actor vocabulary (either going by the language-specific translation of the name or by the “base entry” of an any language).

If that is not the case, the actor likely does not exist in the database yet.

Identification failed. Should a new entry be created?

If the actor does not exist in the database yet, they need to be added. Maybe.

Some names are known to be in clear conflict with the rules of the vocabulary for actors.

Checking the blacklist

museum-digital keeps a centralized blacklist for actor names that are known to be in clear conflict to the rules of the actor vocabulary. This list mainly contains either deliberately unspecific actor names (“unknown”; “unidentified”) or very unspecific ones (“John” [a random given name without a family name], “painter” [the name of a profession, not even of the actor themselves]).

If the entered actor name is contained in this blacklist, the attempt to link any actor to the event is aborted. The free-text note about the event is instead extended to mark that an actor of the blacklisted name was noted to be related to the event.

Checking if the actor is actually a place, time, or tag

A classic problem is vocabulary entries being entered in an unsuitable vocabulary – often for simple reasons likely colleagues slipping to a wrong column when working with a table calculation program. And thus one is asked to import an actor “14th century”.

The “14th century” is obviously a time, not an actor. For this, too, there is a centralized list that can be extended using nodac. Whenever the vocabulary editors move an entry from one vocabulary to another, they have the option to mark the entry’s name as always being of the target type.

Say the 14th century was previously entered as a tag. A vocabulary editor moved the entry to the vocabulary for times and marked the string “14th century” as one always referring to a time. If one then attempts to link a new actor “14th century”, the import tool can automatically recognize that this name actually refers to a time and link a time instead of an actor to the event. The attempt to add an actor is thus redirected to a different category.

All else failed: The actor is being added

If all the above checks did neither result in identifying a pre-existing record, nor in identifying the input name as invalid in one way or another, the actor is added to the controlled vocabularies and linked to the event. The entry now needs to be cleaned and enriched manually by the vocabulary editors.

The success and wide-spread adoption of EODEM however obviously still depends on a number of factors. On the one hand, developers and software vendors need to implement EODEM import and export functionality in their collection management systems. On the other hand, this implementation needs to be accessible and simple to use.

musdb, again, has had the first condition covered for some months now. Accessibility – especially for smaller museums with less tech-savvy staff – was lacking however. Over the last months, it has improved a lot. It is thus now possible to both export and import EODEM data without having to use specialized software beyond one’s file manager and browser.

Exporting EODEM data

The exporting of EODEM data for loans has been described before and remains as simple as ever. Simply link the objects with the outgoing loan and use the respective export option on the loan editing page of the outgoing loan.

The Problem of Flexible Imports

museum-digital’s import tool offers a rather wide range of selectable “parsers” (essentially the format of data to import) and settings for many of these. It may be used for importing five objects at a time without any linked image files – or 50000 with remote image files to fetch. While the former import will be finished in seconds, the latter may take hours depending on the speed of the remote image server.

To prevent timeouts and allow users to control the importer to its full flexibility, and to allow for an easy automation, the upload workflow works via a WebDAV mount. Here, the import is configured using a plain text configuration file and the import data can be uploaded into respective folders for metadata and media files. Finally, a script checks every four hours, if there are import data in the respective directory for a museum and imports them in the background. This process works wonderfully for large imports and migrations. It is however anything but accessible for small scale imports.

Since EODEM imports follow a pre-defined standard and have a rather narrow use case, we chose to add a direct upload form for EODEM import data on the loan editing page in musdb. The sidebar of that page now features a new box “Import EODEM data”. By clicking on it, an overlay opens, prompting the user to upload their EODEM XML files. The XML files are then uploaded into the usual import directory for the museum and the configuration is automatically written with the common settings for EODEM imports.

The upload box in the sidebar is now updated and presents a message detailing the remaining workflow: Wait for some time until the import data has been processed. After the import has been finished, the user receives a mail to the mail address entered for their musdb account.

Looking at the imported object data, one may see a small new feature. The importer automatically determines whether it should update or add new objects by way of the inventory number. If two museums then use the same inventory number schema, this may lead to the loan object information overwriting the data of another object in the museum. To prevent such cases – rare as they may be – the ID of the loan is automatically prepended to the loan object’s inventory number.

EODEM imports can thus now be done directly from the web interface and with minimal worry about data loss. The only caveat may be, that EODEM imports done this way don’t support the import of object images, as this would have required an additional setting to determine if the images were sent together with the import metadata, are located on a web-accessible server, or if they are simply missing.

A Glimpse into the Future

musdb now covers both conditions – EODEM imports and exports are not only implemented and usable anymore, but they are hopefully very easy to use now as well. The EODEM working group remains close to a release of the final standard, and other implementations are announced for the upcoming versions of some other collection management systems.

It is likely that testing the EODEM import with real-world data from other systems will require some further adjustments despite all standardization. We’re looking forward to such cases and more users for the EODEM standard both in terms of vendors supporting it and actual museums profiting from it.

At museum-digital, there are four main centrally controlled vocabularies – actors, places, times, and tags. In more traditional collection management software however, the main field to control is usually the object type (is the object a helmet or a painting?). Simple tagging of the objects can serve as a rather good replacement for a controlled set of object types if one simply repeats the object type as a tag.

As long as one only uses simple tagging, it can however do so only to some extend. If the object is a painting that displays a hammer, the object would feature the tags “painting” and “hammer”. If the object is a hammer with an engraving of a painting, it would feature the very same tags. Take the first example: If one wanted to be able to express that “painting” is the object type and “hammer” is something displayed on the object, one would either need to skip using tags again and use the dedicated fields again – or one needs to be able to categorize tags.

We doubled down on tagging by implementing a categorization mechanism for object tags while streamlining the tagging process in musdb and making it faster and easier to use overall.

The new tagging overlay

When clicking the usual link for linking (new) tags with an object in musdb, one will now be presented with an overlay that replaces the previous page of the same functionality. The layout is rather similar to the previous page – new tags and simple links to actors, places, and times are usually be entered using input fields on the left of the overlay. By typing in the relevant input field, a drop-down list appears, from which entries to be linked can be selected. A new tag, actor, place or time entry can be entered by pressing the “enter” key after typing the name. A sidebar on the right of the overlay allows the simple repetition of the last 10 tags, actors, places, or times the user linked to objects.

The differences between the old page and the new overlay play out on two levels. First, there is the simple fact of the new implementation moving a lot of the logic to the client side, i.e. your browser. If one only selects entries from the drop-down menus, this means that all communication with the server runs in the background. Page reloads are thus only necessary when one enters a new, thus far unknown, tags etc. As the overlay does not close after linking a tag, multiple pre-existing tags can be linked quickly one after the other. In effect, this amounts to a much smoother experience in tagging objects than before.

On the other hand, the overlay unites the existing tagging-related functionalities in one. To do so, the currently linked tags, actors, places, and times are displayed in the overlay as small tiles and can be deleted from there using the trash can icon. Tiles for tags feature an additional select field for categorizing the relationship between object and tag. By default, the relationship is set to the generic “tag”. Alternative categorization options are currently:

• Object type
• Material
• Technique
• Display subject

By categorizing the tags in the appropriate way, one keeps all the advantages the controlled vocabulary for tags brings – multilinguality, links to norm data repositories, and findability using hierarchichal searches, while now being specific about the relationship.

Additionally, another icon (the arrow downwards in a hexagon) offers narrowing down the tag to a more specific version. If an object has for example been tagged as a “portrait”, pressing on the icon will open a sidebar suggesting to replace that tag with “self-portrait”.

Effects on search

Of course, categorizing the links helps little without the relevant search options. Hence, we added five new search fields in the extended search menu: Tags (object type), tags (material), etc.

This content is licensed under a Creative Commons Attribution 4.0 International license.

General

YouTube channel

There now is a museum-digital YouTube channel. For now, one can find some German-language screencasts on different features in musdb and nodac there.

New sections on project page (www.museum-digital.org)

As already described in the previous post, the project page www.museum-digital.org now features a calendar. Since then, we have also added a small page listing people working on museum-digital – e.g. some of the coordinators and developers – as well as power users. The list will obviously always be more than incomplete, but if you want to be listed, just send a mail.

Development

musdb

Allow batch editing of photographers on image list

Running batch updates on the ownership, license status and photographer of a set of objects can be done either via the institution-wide settings page or via a selection on the image list page. The batch updating functionality of the institution-wide settings page works by replacing the given entries in one of the fields for all objects of the museum – it is thus not suitable if one is to only update the photographer field for a list of objects from a single collection (rather than the whole museum).

Batch updating images’ legal information from the image list (click on an image in the list and drag it to activate selection mode, then update by selection) was only possible in the case of image licenses and rights holders. With a small update, it is now possible to also batch-update the images’ photographers field that way.

The API can now update object event

Thus far, the API could only be used for adding wholly new object events (who did what, when, and where, with the object) or deleting them. With the updates of this month, now possible to directly update events.

This is already actively used in customizing musdb using a Tampermonkey script by some of the norm data editors. They can thus add a custom button to transfer incomplete actor or place information to the object’s description. This button, on the other hand, would not be useful for regular users at all.

UI improvements for the object search interface and new sort option

After a very fruitful discussion and feedback from Hungarian colleagues, we got around to some user interface improvements in the object overview and search interface. A lengthy discussion can be found in a dedicated blog post in German. To sum up: there are now headlines for each subsection of the sidebar and tooltips explain the different buttons and search and sort options upon hovering one’s mouse over them. Additionally, a new sort option was added to sort objects by the number characters within the inventory numbers of the searched objects.

Two bugfixes: Navigating to the previous or next object in a selection

It is a very simple but similarly useful feature: One runs an object search and thus creates a result list of objects. After viewing or editing one of the objects, one can then proceed to the next one in the results list by clicking at the arrows at the top of the sidebar.

Unfortunately, musdb had two bugs preventing this navigation for some of the available sort options. On the one hand, newer sort options required manually adding the search fields to a list specific to the previous/next navigation thus far – and had not been covered yet. Navigating to the next object after sorting by e.g. the length of the searched objects was thus impossible so far. The specific list of sort fields has now been removed in the previous/next navigation. Instead, the main list of sort options from the main object search class is directly used – in effect preventing such issues of “forgotten” values from ever appearing again.

A second issue concerned string-based sort options. For an improved performance, the previous / next navigation works by querying the underlying search index for the next object where e.g. the object ID is lower than the current one (`WHERE id > 20000`). The same search logic is straight-out not possible with string-based sort options, as search queries for values greater than or smaller than a given one cannot be executed with string-based fields.

Fortunately, the issue came up just after we had implemented an option to search and sort objects by the numeric components of their inventory numbers. Thus, the previous / next navigation can fall back to that sort option if a user wants to navigate to the next object after sorting by inventory number. As stated back when we introduced the new search and sort option: It should work for most museums. Unfortunately, there is no such option to mitigate the issues of string-based sorting when sorting objects by their names. The best we could do in this case is displaying a clear error message, stating that the previous / next navigation is not available when sorting by the object’s names.

“Home location” can now be selected as a required field when adding new objects

On the institution-wide settings page, museum directors (or those holding the same user role within musdb) can determine which fields are required when adding new objects. The list of selectable fields automatically covers all free text fields linked to the object. Repeat fields and references to other sections of musdb need to be manually implemented however. As the home or permanent location of an object is a most logical and common field to be required for all object, it was only consequential to prioritize making it available as a required field. We have done so now.

Major performance improvements in “list results” page

The “list results” page provides for a table view of object search results, allows the export of search results to Excel, and is – under the hood – also used for generating custom reports. We managed to achieve a considerable improvement in the page’s performance by bulk loading data and simplifying the code of the HTML page. Both adjustments should not really be noticable when one tries to list or export only some objects. Exporting some thousand objects should work considerably faster now however.

frontend

PDF metadata for main object PDF export

XMP metadata are now written into exported PDF files. Thus, users can more easily identify authorship, titles etc., especially when using specialized software like e-book readers.

Linking from collection descriptions

Collection descriptions are now being parsed for the existence of URLs. If one, starting with https:// (http:// is ignored) exists, the URL is automatically transformed into a clickable link.

nodac

We added some translation variables on the start page.

themator

The “themator” has received a small user interface improvement for mobile devices. The topic-specific navigation or table of contents is now moved below rather than above the main content of a page in the regular “topic” viewing mode on mobile devices.

Importer

Importing GIFs

To simplify the handling of images, museum-digital internally only uses JPG image files (webp versions are generated for a faster loading, but are optional). While they lack some support for features like transparency, JPGs provide for a much better compression with images of three-dimensional things and subjects not featuring solid colors. Newer formats like AVIF and Webp unfortunately still lack the support among locally installed image viewers for us to fully switch over to them.

The importer handles input PNGs by converting them to JPG files. Starting this month, the same can be done with input GIF files.

Improved handling of blacklisted actors, places and times

One of the key issues with imports is what happens after imports. Specifically, museum-digital uses controlled vocabularies for actors, places, times and tags that can be linked to an object directly or via events. When people enter new such entries using musdb, there are a number of safeguards and nudges to ensure productive inputs. Adding a new actor, for example, requires one to enter at least 10 characters of a description for the actor. Input fields for the date of birth and death are also provided. If possible, one can enter a Wikipedia or Wikidata URL (or ID) and directly fetch the relevant information from there.

During imports, there is no space for asking questions to the importing institution. Incomplete or false actor, place, time and tag names (for example, if two actors are linked as one; “John Doe and Jane Doe” is not one single actor!) are given, these are necessarily imported into our controlled vocabularies by default. Depending on the import, this puts a significant burden on our volunteer group of vocabulary editors.

To eventually get to a solution, we have added options for automatically rewriting terms (e.g. “Berlin, Germany” will be autocorrected to “Berlin”) and blacklisting them completely (“Good morning …” is no actor”). Thus far, blacklisted terms were simply ignored by the importer.

“Unknown painter” is not an actor, but the information that it is a painter may still be useful. To allow us to blacklist more and keep the controlled vocabularies “clean” without losing such data, the importer now imports blacklisted terms in event components (“who painted it?”) into the event annotation. If the only known information on the whole event is blacklisted, the event cannot be created – an event always requires at least one linked actor, place or time. The event annotation of “empty events” is hence moved to the object description automatically.

This content is licensed under a Creative Commons Attribution 4.0 International license.

musdb

New fields

Other page

Batch updating object information

Instituion-wide settings and adding new objects

Notifications

In other news

• The calendar feature (accessible under the puzzle symbol in the main navigation) can now display tasks or make them subscribable via WebCal (thus implementing a “reminder” as had often been requested)
• PDF of all linked information is now in A4 and uses a two-column layout
• Ukrainian translation
• “Simple A5 PDF” now covers inventorization fields on rear side

md:term

• If two actors have been joined and one has an old links to the page of the actor now deleted, one is now referred onwards to the new, single actor entry. The same works with transferrals between vocabularies (an actor that was transformed into a tag, etc.).
• Ukrainian Translation

Frontend

• Ukrainian Translation
• JSON-based settings for specific institution pages have been removed
• Bulk download of object images
  • An overlay with a message from the museum can be displayed during batch downloads (see above)

CSVXML

• Almost completely rewritten
• The served page now is completely static and all checks and conversions run directly in the browser. This way, no uploads actually happen and the application is completely uncritical to the server’s security. On the other hand, this allows for installing CSVXML as a progressive web app and using it offline.
• We also added some explanatory texts did small updates to the design of the page. A footer now links to the source code and offers to refresh all cached contents of the page (this may be useful when visiting the page after a long time, as the whole application is cached in the browser for offline use).
• A new check also checks for the file encoding. A warning is provided if the data does not appear to be UTF-8-encoded.
• CSVXML is now released open source under the AGPL.

Importer

While the individual parsers for the different export formats are updated very often, the core scripts of the importer are very stable. November 2022 however came with a large update to these core sections, as more categories of data that had not before been covered by the importer (many of them new) can now be imported:

• Contact information (e.g. for object owners; loan partner institution)
• Object’s movement log
• Minimum and maximum temperature, humidity, and lux of an object
• Loans
• Events / Appointments

In terms of the parsers, we extended the LIDO parser to cover the new fields suggested by the upcoming EODEM standard for exchanging loan object information.

A summary of the new features and updates can be found in the following.

Recent Updates and New Features

musdb

Support for Imperial Measurement Units for Separated Measurements

Thus far, musdb only supported metric units for length, width, diameter, etc. It is now also possible to enter measurements in inches and feet. For searching objects smaller or larger than a given number, the number is still expected to be provided in millimeters.

Reworked Institution-Wide Settings Page

Certainly the main focus of our work on improving musdb has been a rather extensive overhaul of the institution-wide settings page. This page is available for users of the user role “museum director” and can be reached through the academy-icon in the navigation.

With the batch of updates, the page gets a new design with an always visible table of contents and rather extensive descriptions of each of the available settings. We removed some outdated and confusing options while adding new ones for the newly added strict modes and inventory number suggestions.

Suggestions for Inventory Numbers for New Objects

On the institution-wide settings page, it is now possible to set a scheme for how the museum forms its inventory numbers. Variable parts are expressed using placeholders:

• {no} for a number counting up. For a counter padded to a given number of digits, a number may be added after the place holder (e.g. {no}6 for a 6 digit counter in the inventory number.
• {c_sig} for a signature of the collection
• {year} for the ongoing calendar year).

If a regular inventory number in the house is, say, “2022-000025”, the scheme would thus be {year}-{no}6.

If an inventory number scheme has been set, an additional suggestion button will appear in front of the entry field for inventory numbers on the page for adding new objects.

Searching for Objects by Their Entry Dates

Similar to how secondary search indexes for measurements greater or smaller than a given value have existed for some time, entry dates, estimation dates, etc. are now made properly searchable through the object search in musdb. For this, dates need to be set in a coherent, maschine-readable form that is greatly aided by the new strict modes.

Strict Modes

See the dedicated blog post on this topic.

Deletion Buttons in the Toolbar of Entry Pages

If one wanted to delete an object group, one thus far needed to find the object group in the object group overview and delete it from there. There was no option to delete it right from the page of the object group itself.

For a more straight-forward control, we now added deletion buttons on the editing pages of object groups, exhibitions, etc. These deletion buttons can be found in the toolbar on the right of the screen.

PNG Supported When Uploading Object Images

Since object images are almost always photos or scans that feature a multitude of colors and few solid blocks of color, JPG is a much better format than PNG for storing (compressed) object images. musdb hence only accepted JPG images for uploads thus far. Since some museums however have internal rules for keeping their object images’ working copies in PNG, we now allow PNG uploads and then convert the PNG images to JPG during the upload process.

Frontend

License Descriptions Consistently Displayed on Object Pages

Until last month, a description of the metadata license of an object page (to be found at the bottom of the page) was only available for the license CC BY-NC-SA. We now removed the exception and made descriptions and links to the license text available for all available licenses.

Importer

• Improved documentation
• Further modularize parsers
• Added new parsers for EasyDB (work in progress)
• Allow importing images by their URLs to locally hosted ones

This content is licensed under a Creative Commons Attribution 4.0 International license.

The next monthly meetup is scheduled for Tuesday, September 6th 2022, 5 p.m. (this post will be updated with a link to the meeting about a week before the meeting, as we are currently trying to organize a meeting room at Zoom or a similar service).

Update: The meeting will take place at https://meet.jit.si/museum-digital-meetup-202209

Recent Updates and New Features

musdb

XML Export

Creating new export formats for XML exports (beyond the main export formats md:xml and LIDO 1.0) is now possible using XSL files stored in a new git sub-repository. This git sub-repository is open source. As registration on our main Gitea instance is however restricted, please send a mail before contributing.

In line with this, we have also improved the representation of uncertainty in our default LIDO 1.0 exports. As version 1.1 of the LIDO standard has been recently released, we have also begun working on a LIDO 1.1 export.

Note: The pre-generated exports available through the “quick export” option are and will still only generated for LIDO 1.0 and md:xml.

Required coordinates when adding new museums

This update is only relevant to regional administrators. Coordinates of the institution are now required for adding new institutions.

QR Codes

QR codes can now also be exported in SVG and EPS, the two primary vector graphics format for use on the web and professional print layouting.

Another important improvement on QR codes is a new page that simply lists all QR codes leading to the object editing pages of a given search result in musdb. This page offers the option to manually set the size of the QR codes, so that suitable QR codes for internal collection and location management can be printed in bulk in a less paper-consuming way.

Calendar

A calendar view for all the events in the museum (events, start and end of an exhibition, start and end of loans) is now available. It can be reached by hovering the mouse over the puzzle symbol in the navigation. On the calendar page, single calendars can be selected or deselected and one subscribed to using one’s external calendar programm (e.g. Outlook, Thunderbird, or a mobile phone’s calendar).

The calendar, and especially the possibility to subscribe to the calendars, is also a first step towards enabling users to set reminders on object pages as this makes much more sense if the reminders can be integrated with one’s regular calendar application.

The calendar view is built using TUI calendar.

Institution-wide Settings and Adding Objects

The page for institution-wide settings has been re-designed to make it easier navigatable. The table of contents is now always visible, and the section “links to the institution elsewhere on the web” has been moved to the institution page.

There however are also some new options on the page. On the one hand, users can now enable or disable the image categorization when uploading images directly when adding new objects (the image recognition is render blocking and often hurts performance more than it actually benefits the inventorization). On the other, it is now possible to add additional fields to be directly accessible upon adding new objects and to make these required. It thus, for example, becomes possible to require a location to be provided before it can be recorded in the system altogether.

Watch list

The watch list feature has been completely re-implemented. The watch list is now generated completely in the browser using easily cache-able data retrieved from APIs. This, first of all, makes the watch list much faster to load and much less of a burden on the server side.

But the new watch list also comes with some new features:

• Users can now have multiple watch lists on the server
• Users can share their currently active watch list with other users of the same museum
• It is now possible to search objects by a watch list (or transfer the watch list into a search result). Thus, all batch editing and export functions available for object search results are now also available for watch lists. Obviously, this also allows for the watch list’s objects to be filtered by their place, collection, etc.
• Objects can now be added to the watch list in bulk.

Object search

The map view of object search results now updates the URL with positions and zoom factor when one browses the map. One can thus share a link to the exact same position one is currently viewing to one’s colleagues.

API

The API can now be used to delete objects.

Design

Error messages are now visible for four seconds as opposed to the regular 2.4 seconds other feedback messages on inputs are visible.

Frontend

Similar to the referencable maps of object search results in musdb, “objects on a map” maps in the frontend are now also referencable. We have also added a legend of event types on the same map.

museum-digital.org

museum-digital.org, the general website introducing museum-digital, has been reworked on a new technical basis. It is now statically generated once a day, reducing server costs and increasing performance. There are also many new texts, some new features (such as FAQs) and banner pictures that make the page more attractive.

The contents of the page are licensed under CC BY 4.0 International and can be edited on GitHub (See the repositories for the English version of the site and the German version).

Importer

The parser for importing images by their file names now offers a setting option to mark a certain part of a file name as an indicator to import the given image file as a hidden / non-public image.

Image: Courtesy NASA/JPL-Caltech.

This content is licensed under a Creative Commons Attribution 4.0 International license.