EXCLUSIVE - How the National Library of Australia is preparing for the digital age
In 2012 the National Library of Australia (the ‘Library’) embarked on an ambitious transformation program to replace its digital library infrastructure, a shared and collaborative platform used to provide access to a national collection of documentary resources, knowledge, stories and culture.
In late 2016 OpenGov interviewed David Wong, the Library’s Assistant Director-General of Information Technology and CIO. At that point the Library had achieved the first program goal of replacing legacy systems used for digitising and managing digitised content. Work had commenced on building new capabilities for collecting and managing born-digital content, or content created digitally, such as electronic publications, websites, and social media.
The program was successfully completed in June 2017. Subsequently, the Library’s effort were recognised by the Australian Institute of Project Management at its national awards, winning the 2017 ICT/Telecommunications category and overall Project of the Year. The project also won the 2017 Asia Pacific Federation of Project Management PM Achievement Awards “Telecommunication and Information Technology” category.
To learn more about the final stages of the program, its outcomes, and how the Library is building on the foundations laid by the program, OpenGov caught up with Mr Wong recently.
He discussed program benefits and areas where expectations were exceeded.
Collecting digital content
“The original scope of the program was to replace end-of-life systems, those systems which were mainly for digitising physical content and then managing that content. There were aspirations to build systems that collect born digital content, that is content produced or natively digital, important as technology advances and spreads”, Mr Wong explained.
With increasing proportions of content produced electronically and delivered through the web, mobile devices, and social media, the Library felt that digital collecting should be a bigger priority in order to fulfil their goal of collecting today what will be important tomorrow and for generations to come. Additional funding was secured to expand the scope of the remaining program stages. To accommodate these requirements, systems supporting born digital content were built in the latter half of the program.
He added that efforts were also driven by a new Commonwealth government legislation which extended legal deposit provisions to electronic publications, mandating that publishers deposit digital publications with the Library. Mr Wong posited “More publishers today are producing content electronically, with some only producing electronically and dispensing with print versions altogether. Now the Library has a digital deposit system for publishers that is easy to use, which means legal deposit obligations are not burdensome. The system also provides batch deposit functionality, allowing publishers to deposit multiple items at once”.
Mr Wong said, “Publishers generally applauded our efforts. It saves publishers time, it saves the Library time. And because we hold the publications in digital form we are also able to protect, digitally preserve, and manage the collection much more easily. Access to the content is also much easier, where permissions are granted.”
Digital collecting systems also benefit creators of manuscripts and unpublished items, including digital archives and audio visual collections. Previously the donors and creators would send files and content by email or physically post digital carriers, such as USB sticks or CDs. The items would then sit on a share drive at the Library and staff members would have to manually process the content.
In addition to the time and cost involved, the metadata and description were not standardised. The same fields would be described in different ways. The amount of metadata provided varied, making it difficult to appraise and categorise that content. Library staff would have to go through each item individually and in some cases have to contact creators for more information. “Our online submission system for pictures and manuscripts performs automated checks on incoming digital material and provides a secure storage environment”, Mr Wong explained.
Workflow efficiency improvement
The other area where the project surpassed expectations was internal workflow efficiency improvement.
“It was always an aim of the program to look at workflows and make them more efficient. And the magnitude of the increase in throughput and volume has been large, even though workflows can be complex. We can acquire, digitise, curate and provide access to items across our collections much more quickly than before,” Mr Wong explained.
In some cases the Library has achieved in excess of a 100 times increase in productivity. The reduction in time between creator deposits to user access has been reduced from weeks, or even months in some cases, to hours.
This improvement has been achieved by automating workflow steps as much as possible, ensuring that systems were interoperable and processes within and across systems were streamlined. “Now that systems are interoperable the transfer of data from one system to another is immediate”, Mr Wong explained.
Building on the foundation
We asked Mr Wong how the Library is building on the foundation laid by the digital library infrastructure replacement project.
“There is continuous and incremental improvement across the ecosystem, to all the parts including workflow systems, public delivery systems and the content repositories”.
Over the course of the program the Library also matured project management and software development methodologies, working very closely with business areas and unlocking organisation-wide capability. However Mr Wong cautioned that there was work to be done to improve the Library’s digital platforms and technology capabilities in order to meet rapidly evolving business needs and user expectations, and in some cases ambiguous and uncertain requirements. “Flexible infrastructure and agile processes will be needed just to stay in the game”, Mr Wong said. He indicated the Library is actively exploring cloud services and operating models for its digital platforms and corporate systems.
The CIO also believes that the Library needs to maintain an internal development capability, particularly for core library services and activities, as this is central to innovation. “Traditionally, the IT market does not serve libraries very well because it is not a very big sector, therefore there haven’t been adequate commercial incentives or competition to develop products to meet the unique needs of libraries and memory institutions. So, ICT developers at the Library had to build a lot of software internally, and we expect this to continue”, according to Mr Wong.
Another key area of focus is projects to improve the Library’s online presence. The Library has a large base of returning users, particularly researchers and family historians who use the Trove service. But there is a large proportion of one-off visitors, known as “culture snackers”. So the Library is seeking a way to interest those people and have them return. That involves making services easier to use and more attractive. It will also involve integration with social media in the future.
Mr Wong said, “We want to improve our Library’s online presence, and improve engagement with our digital services. It is one thing to provide access to lots of content, but it needs to be easily findable and, once found, what’s delivered needs to be appealing and engaging. The snippet they were after should lead to other snippets, views and perspectives that provide new context.”
In addition, Mr Wong revealed that the library is also exploring emerging technologies such as machine learning. He said, “We feel that machine learning has potentially huge applications for curation, content classification, search and digital assistance. With such a vast physical and digital collection, the Library needs a sustainable and scalable way to provide access to its corpus. We expect technology to dramatically speed up content description and classification, and, combined with knowledge and insight from the crowd, provide a richer and more accessible national collection for all“.
The team plans on releasing its new web archive service in 2018 which will provide access to over 10 billion pages of web content from the .au domain, archived over the last 15 years. Innovations include the use of Bayesian filtering to improve search results and the use of machine learning to classify images.