\section{Managing RDF input files} \label{sec:rdflib} Complex projects require RDF resources from many locations and typically wish to load these in different combinations. For example loading a small subset of the data for debugging purposes or load a different set of files for experimentation. The library \pllib{semweb/rdf_library.pl} manages sets of RDF files spread over different locations, including file and network locations. The original version of this library supported metadata about collections of RDF sources in an RDF file called \jargon{Manifest}. The current version supports both the \href{http://www.w3.org/TR/void/}{VoID} format and the original format. VoID files (typically named \file{void.ttl}) can use elements from the RDF Manifest vocabulary to support features that are not supported by VoID. \subsection{The Manifest file} \label{sec:semweb-rdf-manifest} A manifest file is an RDF file, often in \href{http://www.w3.org/TeamSubmission/turtle/}{Turtle} format, that provides meta-data about RDF resources. Often, a manifest will describe RDF files in the current directory, but it can also describe RDF resources at arbitrary URL locations. The RDF schema for RDF library meta-data can be found in \file{rdf_library.ttl}. The namespace for the RDF library format is defined as \url{http://www.swi-prolog.org/rdf/library/} and abbreviated as \const{lib}. The schema defines three root classes: lib:Namespace, lib:Ontology and lib:Virtual, which we describe below. \begin{description} \resitem{lib:Ontology} This is a subclass of owl:Ontology. It has two subclasses, lib:Schema and lib:Instances. These three classes are currently processed equally. The following properties are recognised on lib:Ontology: \begin{description} \resitem {dc:title} Title of the ontology. Displayed by \index{rdf_list_library/0}\predref{rdf_list_library}{0}. \resitem {owl:versionInfo} Version of the ontology. Displayed by \index{rdf_list_library/0}\predref{rdf_list_library}{0}. \resitem {owl:imports} Ontologies imported. If \index{rdf_load_library/2}\predref{rdf_load_library}{2} is used to load this ontology, the ontologies referenced here are loaded as well. There are two subProperties: lib:schema and lib:instances with the obvious meaning. \resitem {lib:source} Defines the named graph into which the resource is loaded. If this ends in a \const{\Sdiv}, the basename of each loaded file is appended to the given source. Defaults to the URL the RDF is loaded from. \resitem {lib:baseURI} Defines the base for processing the RDF data. If not provided this defaults to the named graph, which in turn defaults to the URL the RDF is loaded from. \end{description} \resitem{lib:Virtual} Virtual ontologies do not refer to an RDF resource themselves. They only import other resources. For example the W3C WordNet manifest defines \const{wn-basic} and \const{wn-full} as virtual resources. The lib:Virtual resource is used as a second rdf:type: \begin{code} a lib:Ontology ; a lib:Virtual ; ... \end{code} \noindent \resitem{lib:CloudNode} Used by ClioPatria to combine this ontology and all data it imports into a node in the automatically generated datacloud. \resitem{lib:Namespace} Defines a URL to be a namespace. The definition provides the preferred mnemonic and can be referenced in the lib:providesNamespace and lib:usesNamespace properties. The \index{rdf_load_library/2}\predref{rdf_load_library}{2} predicates registers encountered namespace mnemonics with rdf-db using \index{rdf_register_ns/2}\predref{rdf_register_ns}{2}. Typically namespace declarations use @{prefix} declarations. E.g.\ \begin{code} @prefix lib: . @prefix rdfs: . [ a lib:Namespace ; lib:mnemonic "rdfs" ; lib:namespace rdfs: ] . \end{code} \noindent \end{description} \subsubsection{Support for the VoID and VANN vocabularies} \label{sec:semweb-void} The \href{http://www.w3.org/TR/void/}{VoID} aims at resolving the same problem as the Manifest files described here. In addition, the \href{http://vocab.org/vann/}{VANN} vocabulary provides the information about preferred namepaces prefixes. The RDF library manager can deal with VoID files. The following relations apply: \begin{itemize} \item VoID \const{Dataset} and \const{Linkset} are similar to \const{lib:Ontology}, but a VoID resource is always \jargon{Virtual}. I.e., the VoID URI itself never refers to an RDF document. \item The \const{owl:imports} and its lib specializations are replaced by \const{void:subset} (referring to another VoID dataset) and \const{void:dataDump} (referring to a concrete document). \item A description of the dataset is given using \const{dcterms:description} rather than \const{rdfs:comment} \item The RDF library recognises \const{lib:source}, \const{lib:baseURI} and \const{lib:Cloudnode}, which have no equivalent in VoID. \item The RDF library recognises \const{vann:preferredNamespacePrefix} and \const{vann:preferredNamespaceUri} as alternatives to its proprietary way for defining prefixes. The domain of these predicates is unclear. The library recognises them regardless of the domain. Note that the range of \const{vann:preferredNamespaceUri} is a \emph{literal}. A disadvantage of that is that the Turtle prefix declaration cannot be reused. \end{itemize} Currently, the RDF metadata is \emph{not} stored in the RDF database. It is processed by low-level primitives that do \emph{not} perform RDFS reasoning. In particular, this means that rdfs:supPropertyOf and rdfs:subClassOf cannot be used to specialise the RDF meta vocabulary. \subsubsection{Finding manifest files} \label{sec:semweb-find-manifest} The initial metadata file(s) are loaded into the system using \index{rdf_attach_library/1}\predref{rdf_attach_library}{1}. \begin{description} \predicate{rdf_attach_library}{1}{+FileOrDirectory} Load meta-data on RDF repositories from \arg{FileOrDirectory}. If the argument is a directory, this directory is processed recursively and each for each directory, a file named \file{void.ttl}, \file{Manifest.ttl} or \file{Manifest.rdf} is loaded (in this order of preference). Declared namespaces are added to the rdf-db namespace list. Encountered ontologies are added to a private database of \file{rdf_list_library.pl}. Each ontology is given an \jargon{identifier}, derived from the basename of the URL without the extension. This, using the declaration below, the identifier of the declared ontology is \const{wn-basic}. \begin{code} a void:Dataset ; dcterms:title "Basic WordNet" ; ... \end{code} \noindent \predicate{rdf_list_library}{0}{} List the available resources in the library. Currently only lists resources that have a dcterms:title property. See \secref{usage} for an example. \end{description} It is possible for the initial set of manifests to refer to RDF files that are not covered by a manifest. If such a reference is encountered while loading or listing a library, the library manager will look for a manifest file in the directory holding the referenced RDF file and load this manifest. If a manifest is found that covers the referenced file, the directives found in the manifest will be followed. Otherwise the RDF resource is simply loaded using the current defaults. Further exploration of the library is achieved using \index{rdf_list_library/1}\predref{rdf_list_library}{1} or \index{rdf_list_library/2}\predref{rdf_list_library}{2}: \begin{description} \predicate{rdf_list_library}{1}{+Id} Same as \term{rdf_list_library}{Id, []}. \predicate{rdf_list_library}{2}{+Id, +Options} Lists the resources that will be loaded if \arg{Id} is handed to \index{rdf_load_library/2}\predref{rdf_load_library}{2}. See \index{rdf_attach_library/1}\predref{rdf_attach_library}{1} for how ontology identifiers are generated. In addition it checks the existence of each resource to help debugging library dependencies. Before doing its work, \index{rdf_list_library/2}\predref{rdf_list_library}{2} reloads manifests that have changed since they were loaded the last time. For HTTP resources it uses the HEAD method to verify existence and last modification time of resources. \predicate{rdf_load_library}{2}{+Id, +Options} Load the given library. First \index{rdf_load_library/2}\predref{rdf_load_library}{2} will establish what resources need to be loaded and whether all resources exist. Than it will load the resources. \end{description} \subsection{Usage scenarios} \label{sec:usage} Typically, a project will use a single file using the same format as a manifest file that defines alternative configurations that can be loaded. This file is loaded at program startup using \index{rdf_attach_library/1}\predref{rdf_attach_library}{1}. Users can now list the available libraries using \index{rdf_list_library/0}\predref{rdf_list_library}{0} and \index{rdf_list_library/1}\predref{rdf_list_library}{1}: \begin{code} 1 ?- rdf_list_library. ec-core-vocabularies E-Culture core vocabularies ec-all-vocabularies All E-Culture vocabularies ec-hacks Specific hacks ec-mappings E-Culture ontology mappings ec-core-collections E-Culture core collections ec-all-collections E-Culture all collections ec-medium E-Culture medium sized data (artchive+aria) ec-all E-Culture all data \end{code} \noindent Now we can list a specific category using \index{rdf_list_library/1}\predref{rdf_list_library}{1}. Note this loads two additional manifests referenced by resources encountered in \const{ec-mappings}. If a resource does not exist is is flagged using \const{[NOT FOUND]}. \begin{code} 2 ?- rdf_list_library('ec-mappings'). % Loaded RDF manifest /home/jan/src/eculture/vocabularies/mappings/Manifest.ttl % Loaded RDF manifest /home/jan/src/eculture/collections/aul/Manifest.ttl . . . . . . file:///home/jan/src/eculture/vocabularies/mappings/interface_class_mapping.ttl . . . file:///home/jan/src/eculture/vocabularies/mappings/interface_property_mapping.ttl . . . . . file:///home/jan/src/eculture/vocabularies/mappings/ethnographic_property_mapping.ttl . . . file:///home/jan/src/eculture/vocabularies/mappings/eculture_properties.ttl . . . file:///home/jan/src/eculture/vocabularies/mappings/eculture_property_semantics.ttl . . . . . file:///home/jan/src/eculture/vocabularies/mappings/eculture_situations.ttl . . . file:///home/jan/src/eculture/collections/aul/aul.rdfs . . file:///home/jan/src/eculture/collections/aul/aul.rdf . . file:///home/jan/src/eculture/collections/aul/aul9styles.rdf . . file:///home/jan/src/eculture/collections/aul/extractedperiods.rdf . . file:///home/jan/src/eculture/collections/aul/manual-periods.rdf \end{code} \noindent \subsubsection{Referencing resources} \label{sec:semweb-manifest-resources} Resources and manifests are located either on the local filesystem or on a network resource. The initial manifest can also be loaded from a file or a URL. This defines the initial \jargon{base URL} of the document. The base URL can be overruled using the Turtle @{base} directive. Other documents can be referenced relative to this base URL by exploiting Turtle's URI expansion rules. Turtle resources can be specified in three ways, as absolute URLs (e.g.\ \verb$), as relative URL to the base (e.g.\ \verb$<../rdf/ontology.rdf$>) or following a \jargon{prefix} (e.g.\ prefix:ontology). The prefix notation is powerful as we can define multiple of them and define resources relative to them. Unfortunately, prefixes can only be defined as absolute URLs or URLs relative to the base URL. Notably, they cannot be defined relative to other prefixes. In addition, a prefix can only be followed by a Qname, which excludes \verb$.$ and \verb$/$. Easily relocatable manifests must define all resources relative to the base URL. Relocation is automatic if the manifest remains in the same hierarchy as the resources it references. If the manifest is copied elsewhere (i.e.\ for creating a local version) it can use @{base} to refer to the resource hierarchy. We can point to directories holding manifest files using @{prefix} declarations. There, we can reference \jargon{Virtual} resources using prefix:name. Here is an example, were we first give some line from the initial manifest followed by the definition of the virtual RDFS resource. \begin{code} @base . @prefix base: . a lib:Ontology ; a lib:Virtual ; dc:title "E-Culture core vocabularies" ; owl:imports base:rdfs , base:owl , base:dc , base:vra , ... \end{code} \noindent \begin{code} a lib:Schema ; a lib:Virtual ; rdfs:comment "RDF Schema" ; lib:source rdfs: ; lib:schema . \end{code} \noindent \subsection{Putting it all together} \label{sec:semweb-rdflib-example} In this section we provide skeleton code for filling the RDF database from a password protected HTTP repository. The first line loads the application. Next we include modules that enable us to manage the RDF library, RDF database caching and HTTP connections. Then we setup the HTTP authentication, enable caching of processed RDF files and load the initial manifest. Finally \index{load_data/0}\predref{load_data}{0} loads all our RDF data. \begin{code} :- use_module(server). :- use_module(library(http/http_open)). :- use_module(library(semweb/rdf_library)). :- use_module(library(semweb/rdf_cache)). :- http_set_authorization('http://www.example.org/rdf', basic(john, secret)). :- rdf_set_cache_options([ global_directory('RDF-Cache'), create_global_directory(true) ]). :- rdf_attach_library('http://www.example.org/rdf/Manifest.ttl'). %% load_data % % Load our RDF data load_data :- rdf_load_library('all'). \end{code} \noindent \subsection{Example: A metadata file for W3C WordNet} \label{sec:w3cmanifest} The VoID metadata below allows for loading WordNet in the two predefined versions using one of \begin{code} ?- rdf_load_library('wn-basic', []). ?- rdf_load_library('wn-full', []). \end{code} \noindent \begin{code} @prefix void: . @prefix vann: . @prefix lib: . @prefix owl: . @prefix rdf: . @prefix rdfs: . @prefix xsd: . @prefix dc: . @prefix wn20s: . @prefix wn20i: . [ vann:preferredNamespacePrefix "wn20i" ; vann:preferredNamespaceUri "http://www.w3.org/2006/03/wn/wn20/instances/" ] . [ vann:preferredNamespacePrefix "wn20s" ; vann:preferredNamespaceUri "http://www.w3.org/2006/03/wn/wn20/schema/" ] . a void:Dataset ; dc:description "Common files between full and basic version" ; lib:source wn20i: ; void:dataDump , , , , , , , , , , , , . a void:Dataset ; void:subset ; void:dataDump . a lib:Schema ; lib:source wn20s: ; void:dataDump . a void:Dataset ; lib:source wn20s: ; void:dataDump . a void:Dataset ; a lib:CloudNode ; dc:title "Basic WordNet" ; dc:description "Light version of W3C WordNet" ; owl:versionInfo "2.0" ; lib:source wn20i: ; void:subset , , . a void:Dataset ; lib:source wn20s: ; void:dataDump . a void:Dataset ; a lib:CloudNode ; dc:title "Full WordNet" ; dc:description "Full version of W3C WordNet" ; owl:versionInfo "2.0" ; lib:source wn20i: ; void:subset , , ; void:dataDump , , , , , , . \end{code} \noindent %%