In 2012 the National Library of Australia (the ‘Library’)
embarked on an ambitious transformation program to replace its digital library
infrastructure, a shared and collaborative platform used to provide access to a
national collection of documentary resources, knowledge, stories and culture.
In late 2016 OpenGov interviewed
David Wong, the Library’s Assistant Director-General of Information Technology
and CIO. At that point the Library had achieved the first program goal of
replacing legacy systems used for digitising and managing digitised content. Work
had commenced on building new capabilities for collecting and managing
born-digital content, or content created digitally, such as electronic
publications, websites, and social media.
The program was successfully completed in June
2017. Subsequently, the Library’s effort were recognised
by the Australian Institute of Project Management at its national awards, winning
the 2017 ICT/Telecommunications category and overall Project of the Year. The
project also won the 2017 Asia Pacific Federation of Project Management PM
Achievement Awards “Telecommunication and Information Technology” category.
To learn more about the final stages of the
program, its outcomes, and how the Library is building on the foundations laid
by the program, OpenGov caught up with Mr Wong recently.
He discussed program benefits and areas
where expectations were exceeded.
“The original scope of the program was to
replace end-of-life systems, those systems which were mainly for digitising
physical content and then managing that content. There were aspirations to
build systems that collect born digital content, that is content produced or
natively digital, important as technology advances and spreads”, Mr Wong
With increasing proportions of content
produced electronically and delivered through the web, mobile devices, and
social media, the Library felt that digital collecting should be a bigger
priority in order to fulfil their goal of collecting today what will be
important tomorrow and for generations to come. Additional funding was secured
to expand the scope of the remaining program stages. To accommodate these
requirements, systems supporting born digital content were built in the latter
half of the program.
He added that efforts were also driven by a
new Commonwealth government legislation which extended legal deposit provisions
to electronic publications, mandating that publishers deposit digital publications
with the Library. Mr Wong posited “More publishers today are producing content
electronically, with some only producing electronically and dispensing with
print versions altogether. Now the Library has a digital deposit system for publishers
that is easy to use, which means legal deposit obligations are not burdensome.
The system also provides batch deposit functionality, allowing publishers to
deposit multiple items at once”.
Mr Wong said, “Publishers generally applauded
our efforts. It saves publishers time, it saves the Library time. And because
we hold the publications in digital form we are also able to protect, digitally
preserve, and manage the collection much more easily. Access to the content is
also much easier, where permissions are granted.”
Digital collecting systems also benefit
creators of manuscripts and unpublished items, including digital archives and
audio visual collections. Previously the donors and creators would send files
and content by email or physically post digital carriers, such as USB sticks or
CDs. The items would then sit on a share drive at the Library and staff members
would have to manually process the content.
In addition to the time and cost involved,
the metadata and description were not standardised. The same fields would be
described in different ways. The amount of metadata provided varied, making it difficult
to appraise and categorise that content. Library staff would have to go through
each item individually and in some cases have to contact creators for more
information. “Our online submission system for pictures and manuscripts
performs automated checks on incoming digital material and provides a secure
storage environment”, Mr Wong explained.
The other area where the project
surpassed expectations was internal workflow efficiency improvement.
“It was always an aim of the program to
look at workflows and make them more efficient. And the magnitude of the
increase in throughput and volume has been large, even though workflows can be
complex. We can acquire, digitise, curate and provide access to items across
our collections much more quickly than before,” Mr Wong explained.
In some cases the Library has achieved in
excess of a 100 times increase in productivity. The reduction in time between
creator deposits to user access has been reduced from weeks, or even months in
some cases, to hours.
This improvement has been achieved by
automating workflow steps as much as possible, ensuring that systems were
interoperable and processes within and across systems were streamlined. “Now that
systems are interoperable the transfer of data from one system to another is
immediate”, Mr Wong explained.
on the foundation
We asked Mr Wong how the Library is
building on the foundation laid by the digital library infrastructure replacement project.
“There is continuous and incremental
improvement across the ecosystem, to all the parts including workflow systems,
public delivery systems and the content repositories”.
Over the course of the program the Library
also matured project management and software development methodologies, working
very closely with business areas and unlocking organisation-wide capability.
However Mr Wong cautioned that there was work to be done to improve the
Library’s digital platforms and technology capabilities in order to meet
rapidly evolving business needs and user expectations, and in some cases
ambiguous and uncertain requirements. “Flexible infrastructure and agile processes
will be needed just to stay in the game”, Mr Wong said. He indicated the
Library is actively exploring cloud services and operating models for its
digital platforms and corporate systems.
CIO also believes that the Library needs to maintain an internal development
capability, particularly for core library services and activities, as this is central
to innovation. “Traditionally, the IT market does not serve libraries very well
because it is not a very big sector, therefore there haven’t been adequate commercial
incentives or competition to develop products to meet the unique needs of
libraries and memory institutions. So, ICT developers at the Library had to
build a lot of software internally, and we expect this to continue”, according
to Mr Wong.
Another key area of focus is projects to
improve the Library’s online presence. The Library has a large base of
returning users, particularly researchers and family historians who use the
Trove service. But there is a large proportion of one-off visitors, known as
“culture snackers”. So the Library is seeking a way to interest those people
and have them return. That involves making services easier to use and more
attractive. It will also involve integration with social media in the future.
Mr Wong said, “We want to improve our Library’s
online presence, and improve engagement with our digital services. It is one
thing to provide access to lots of content, but it needs to be easily findable
and, once found, what’s delivered needs to be appealing and engaging. The
snippet they were after should lead to other snippets, views and perspectives
that provide new context.”
In addition, Mr Wong revealed that the
library is also exploring emerging technologies such as machine learning. He
said, “We feel that machine learning has potentially huge applications for curation,
content classification, search and digital assistance. With such a vast
physical and digital collection, the Library needs a sustainable and scalable
way to provide access to its corpus. We expect technology to dramatically speed
up content description and classification, and, combined with knowledge and
insight from the crowd, provide a richer and more accessible national collection
The team plans on releasing its new web
archive service in 2018 which will provide access to over 10 billion pages of
web content from the .au domain, archived over the last 15 years. Innovations
include the use of Bayesian filtering to improve search results and the use of
machine learning to classify images.