2021 March-September Development Cycle

Owned by Maggie Dickson (old account) (Unlicensed)
Last updated: Oct 12, 2021
6 min read

Project Proposal/Charter: ​​Confluence - Duke Wiki

The DDR development cycle that ran from March to September 2021 focused on the first two sections of the above project charter: 

  • Improve object and metadata download functionality from Public and Staff apps

  • Fix and update the DDR-->ArchivesSpace feature

 

WORK AREAS

Export/download

Summary

The goal of the export/download work area was to enhance data export functionality for both DUL staff and end users.  

  • Staff can now export any type of content from DDR via the DDR staff interface (including derivative formats); the data will be exported to a CIFS server where it can be downloaded and moved to Box or FTP or however the data needs to be delivered.  Instructions are available on confluence (please send farrell, Maggie and/or Molly feedback if you have it).

  • End users can now export pdfs of items that include more than 100 components (images) from the DDR public interface. For an example see this Early Manuscript item.  

  • Digital Collections end users can now download file types (such as PDFs) that don’t display in the in-browser viewer. For an example see this item.

Completed stories related to export/download

  • DDR-2253 - As a staff user, I want to export checksums for a collection in a way that checksum exports don't fail when a collection is too large. (completed during March 1-15 sprint)

  • DDR-2272 - As a developer I want to know why large batch exports sometimes fail. (completed during March 29-April 11 sprint)

  • DDR-2273 - As a library staff member I want to choose which file types to batch export (completed during the April 26-May 9 sprint)

  • DDR-2259 - As a public user, I would like to be able to access/download file types that don't trigger the image viewer in Digital Collections (completed during the May 10-23 sprint)

  • DDR-2274 - As a library staff member I want to batch export files to mounted storage. (completed during the June 7-20 sprint)

  • DDR-2275 - As a library staff member I want to export images from the image server. (completed during the June 7-20 sprint)

  • DDR-2309 - Exported file extensions are not assigned as expected (completed during the June 21-July 3 sprint)

  • DDR-2316 - As a developer, I want to remove deprecated code & configs for max_download restrictions (completed during the June 21-July 3 sprint)

  • DDR-2310 - As a staff user exporting DDR content, I want the list of exportable objects to be more clear so that I can export the correct objects. (completed during the July 6-19 sprint)

  • DDR-2280 - As a repository analyst, I want the system to assign the role of Downloader to the Rubenstein Library Staff group at ingest for all collections in the Digital Collections and Rubenstein Library admin sets, so that I do not have to do it manually. (completed during the July 20-August 2 sprint)

  • DDR-2252 – As a public user, I want to download still image items that have more than 100 components so that I can look at them offline.  (completed during the July 20-August 2 sprint)

  • DDR-2322 – Raise the storage limit on DDR test to 1 terabyte (completed during the August 3-16 sprint)

  • DDR-2336 - As a public user, I want the list of components in an item to be ordered alphanumerically. (completed during the September 13-26 sprint)

 

Ingest

Summary

This work facilitates successful ingesting of content into DDR, as well as the ability to troubleshoot errors surrounding ingest more effectively and without developer involvement. 

  • DDR-2250 - As a repository services analyst, I want to be able to manually generate a batch processor report for any ingest. (completed during March 15-28 sprint)

    • increases the repository services analysts flexibility in performing and troubleshooting DDR ingest.

  • DDR-2247 - As a repository services analyst, I want to be able to use batch ingest tools from all 4 staging locations (completed during March 15-28 sprint)

    • increases the repository services analysts flexibility in performing and troubleshooting DDR ingest.

  • DDR-2295 - As a repository analyst, I would like the batch processor report "log" to be more verbose (completed during the May 10-23 sprint)

    • responds to a need from DCCS repository services analysts and POs to see verbose reports for troubleshooting purposes

  • DDR-2249 - As a repository services analyst, I want to be able to terminate or restart (preferred, if possible) a batch whose processing has stalled. (completed during the August 3-16 sprint)

    • The developers have resolved the issue where batch jobs stall indefinitely, which is good (i.e. a batch will either fail or succeed). It will require significant effort to develop functionality to restart batches that have failed. The product owners will revisit the follow-on work in a future cycle.

  • DDR-2332 - As a repository services analyst, I want to be able to re-run a batch which failed because of EZID connection issues. (completed during the September 13-27 sprint)

 

Fix and update the DDR-->ArchivesSpace feature

Summary

The goal of revisiting the DDR-->ArchivesSpace feature was to fix a feature deployed by DUL developers ca. 2018-2019, but which broke when we moved ArchivesSpace from a locally-hosted project to one hosted by Lyrasis. Updating this feature allows RLTS staff to automate linking between items in Digital Collections and description in ArchivesSpace. Additional work may be undertaken in the future to link digital objects outside of Digital Collections to description in ArchivesSpace using this connector.

  • DDR-2287 - As a DDR administrator, I want to rewrite the DDR-ASpace connector so that it is compatible with Duke’s Lyrasis-hosted ArchivesSpace instance (completed during the April 26-May 9 sprint)

  • DDR-2284 - As ArchivesSpace administrator, I want staff to authenticate to ArchivesSpace hosted by Lyrasis using Duke Shibboleth (completed during the May 10-23 sprint)

 

Performance

Summary

These stories were all focused on improving the performance of the DDR application, so pages load faster for end users. 

  • DDR-2255 – As a user of ddr-xenon, I want the search results to load more quickly (completed during the April 12-25 sprint)

    • improves load times on DDR pages

  • DDR-2290 - As a developer I would like to truncate long user queries before sending the request to Solr (completed during the April 26-May 9 sprint)

    • improve DDR performance

  • DDR-1976 - Prevent deep paging and deep facet paging (completed during the April 26-May 9 sprint)

    • improve DDR performance

  • DDR-2256: As the digital heritage center I want to harvest metadata via ddr-xenon's OAI feed more quickly. (completed during the May 24-June 6 sprint)   

    • related to sharing metadata with the Digital Public Library of America.

 

Miscellaneous

  • DDR-2254 – As a Repository Services Analyst, I would like the ability to deaccession items in DDR (completed during March 15-28 sprint)

  • DDR-2232 - As a researcher who views an embedded restricted item, I want to see the same permissions alerts I'd see on the regular DDR item page. (completed during the March 29-11 sprint)

    • viewers of a restricted item embedded in a collection guide will see a message about the restriction rather than a mysterious black box

  • DDR-2224 – As a curator, I want to be able to preview showcase images on collection portals before the items are public (completed during the April 12-25 sprint)

    • resolves an issue with Digital Collections collection portals that make use of showcase images. Previously, if the showcase image referred to an item that was not yet published, a grey box would appear in its place on the landing page making it annoying to preview by staff.

  • DDR-2299/2300 – a couple of stories related to a continuous release cycle (completed during the May 10-23 sprint)

    • related to releasing new features for testing on an ongoing basis, instead of waiting until a bunch of features are ‘ready’ and releasing them all at once

  • DDR-2317 - As a Repository Services Analyst, I want to run a rake task to manually assign missing ARKs to items in DDR. (completed during the August 3-16 sprint)

  • DDR-2318 (related to DDR-2317) - As a developer I would like a rake task to run periodically as a cron job that assigns ARKs to items that are missing them. (completed during the August 3-16 sprint)

  • DDR-2038 – As a user, I want to be able to select a search of the entire DDR from the search box on the home page, so that I can search it all at once. (completed during the August 17-29 sprint)

  • DDR-2325 - As a developer, I would like to create a restore process for DDR Test that can be used ahead of test fests, so there is always a reliable set of collections and metadata that can be used for testing. (completed during the August 30-September 12 sprint)