Evaluating FAIR Digital Object and Linked Data as distributed object systems

FDO approaches

FDO is an evolving concept. A set of FDO Demonstrators (Wittenburg et al., 2022) highlights how current adapters are approaching implementations of FDO from different angles:

• Building on the Digital Object concept, using the simplified DOIPV2.0 (2018) specification, which detail how to exchange JSON objects through a text-based protocol¹ (usually TCP/IP over TLS). The main DOIP operations are retrieving, creating and updating digital objects. These are mostly realised using the reference implementation Cordra (Tupelo-Schneck & Lannom, 2022). FDO types are registered in the local Cordra instance, where they are specified using JSON Schema (Wright et al., 2022) and PIDs are assigned using the Handle system. Several type registries have been established.

• Following a Linked Data approach, but using the DOIP protocol, e.g., using JSON-LD and schema.org within DOIP in Materal Sciences archives (Riccardi et al., 2022).

• Approaching the FDO principles from existing Linked Data practices on the Web, e.g., WorkflowHub use of RO-Crate and schema.org (Soiland-Reyes et al., 2022b).

From this it becomes apparent that there is a potentially large overlap between the goals and approaches of FAIR Digital Objects and Linked Data, which we will cover in section From the Semantic Web to Linked Data.

Next steps for FDO

The FAIR Digital Object Forum (FAIR Digital Objects, 2022a) working groups have prepared detailed requirement documents (FAIR Digital Objects, 2022b) setting out the path for realising FDOs, named FDO Recommendations. As of 2023-06-17, most of these documents are open for public review, while some are still in draft stages for internal review. We provide an overview of these documents in ‘An Overview of Upcoming FDO Specifications’. These documents clarify the future aims and focus of FAIR Digital Objects (Lannom et al., 2022b). Except for the DOIP endorsement, all of these documents are conceptual, in the sense that they permit any technical implementation of FDO, if used according to the recommendations. Going forward, a key strategy of the Forum is the use of profiles to help define specific attributes in metadata that are necessary for domains or application contexts. However, these are not yet fully implemented in the implementations considered here.

Existing FDO implementations (Wittenburg et al., 2022) are thus not fully consolidated in choices such as protocols, type systems and serialisations—this divergence and corresponding additional technical requirements mean that FDOs are not yet in a single ecosystem.

A brief history of the Semantic Web

The Semantic Web was developed as a vision by Berners-Lee & Fischetti (1999), at a time that the Web had already become widely established for information exchange, being a global set of hypermedia documents which are cross-related using universal links in the form of URLs. The foundations of the Web (e.g., URLs, HTTP, SSL/TLS, HTML, CSS, ECMAScript/JavaScript, media types) were standardised by W3C, Ecma, IETF and later WHATWG. The goal of Semantic Web was to further develop the machine-readable aspects of the Web, in particular adding meaning (or semantics) to not just the link relations, but also to the resources that the URLs identified, and for machines thus being able to meaningfully navigate across such resources, e.g., to answer a particular query.

Through W3C, the Semantic Web was realised with the Resource Description Framework (RDF) (Schreiber & Raimond, 2014) that used triples of subject-predicate-object statements, with its initial serialisation format (Lassila & Swick, 1999) being RDF/XML (XML was at the time seen as a natural data-focused evolution from the document-centric SGML and HTML).

While triple-based knowledge representations were not new (Stanczyk, 1987), the main innovation of RDF was the use of global identifiers in the form of URIs² as the primary identifier of the subject (what the statement is about), predicate (relation/attribute of the subject) and object (what is pointed to). By using URIs not just for documents³ , the Semantic Web builds a self-described system of types and properties, where the meaning of a relation can be resolved by following its hyperlink to the definition within a vocabulary. By applying these principles as well to any kind of resource that could be described at a URL, this then forms a global distributed Semantic Web.

The early days of the Semantic Web saw fairly lightweight approaches with the establishment of vocabularies such as FOAF (to describe people and their affiliations) and Dublin Core (for bibliographic data). Vocabularies themselves were formalised using RDFS or simply as human-readable HTML web pages defining each term. The main approach of this Web of Data was that a URI identified a resource (e.g., an author) with a HTML representation for human readers, along with a RDF representation for machine-readable data of the same resource. By using content negotiation in HTTP, the same identifier could be used in both views, avoiding index.html vs index.rdf exposure in the URLs. The concept of namespaces gave a way to give a group of RDF resources with the same URI base from a Semantic Web-aware service a common prefix, avoiding repeated long URLs.

The mid-2000s saw large academic interest and growth of the Semantic Web, with the development of more formal representation system for ontologies, such as OWL (W3C OWL Working Group, 2012), allowing complex class hierarchies and logic inference rules following open world paradigm. More human-readable syntaxes for RDF such as Turtle evolved at this time, and conferences such as ISWC (Horrocks & Hendler, 2002) gained traction, with a large interest in knowledge representation and logic systems based on Semantic Web technologies evolving at the same time.

Established Semantic Web services and standards include: SPARQL (SPARQL Working Group, 2013) (pattern-based triple queries), named graphs (Wood, Cyganiak & Lanthaler, 2014) (triples expanded to quads to indicate statement source or represent conflicting views), triple/quad stores (graph databases such as OpenLink Virtuoso, GraphDB, 4Store), mature RDF libraries (including Redland RDF, Apache Jena, Eclipse RDF4J, RDFLib, RDF.rb, rdflib.js), and graph visualisation.

RDF is one way to implement knowledge graphs, a system of named edges and nodes⁴ (Nurdiati & Hoede, 2008), which when used to represent a sufficiently detailed model of the world, can then be queried and processed to answer detailed research questions. The creation of RDF-based knowledge graphs grew particularly in fields like bioinformatics, e.g., for describing genomes and proteins (Goble & Stevens, 2008; Williams et al., 2012). In theory, the use of RDF by the life sciences would enable interoperability between the many data repositories and support combined views of the many aspects of bio-entities—however in practice most institutions ended up making their own ontologies and identifiers, for what to the untrained eye would mean roughly the same. One can argue that the toll of adding the semantic logic system of rich ontologies meant that small, but fundamental, differences in opinion (e.g., should a gene identifier signify just the particular DNA sequence letters, or those letters as they appear in a particular position on a human chromosome?) led to large differences in representational granularity, and thus needed different identifiers.

Facing these challenges, thanks to the use of universal identifiers in the form of URIs, mappings could retrospectively be developed not just between resources, but also across vocabularies. Such mappings can be expressed themselves using lightweight and flexible RDF vocabularies such as SKOS (Isaac & Summers, 2009) (e.g., dct:title skos:closeMatch schema:name to indicate near equivalence of two properties). Exemplifying the need for such cross-references, automated ontology mappings have identified large potential overlaps like 372 definitions of Person (Hu et al., 2011).

The move towards Open Science data sharing practices did from the late 2000s encourage knowledge providers to distribute collections of RDF descriptions as downloadable datasets⁵ , so that their clients can avoid thousands of HTTP requests for individual resources. This enabled local processing, mapping and data integration across datasets (e.g., Open PHACTS (Groth et al., 2014)), rather than relying on the providers’ RDF and SPARQL endpoints (which could become overloaded when handling many concurrent, complex queries).

With these trends, an emerging problem was that adopters of the Semantic Web primarily utillised it as a set of graph technologies, with little consideration to existing Web resources. This meant that links stayed mainly within a single information system, with little URI reuse even with large term overlaps (Kamdar, Tudorache & Musen, 2017). Just like link rot affect regular Web pages and their citations from scholarly communication (Klein et al., 2014), a majority of described RDF resources in the Linked Open Data (LOD) Cloud’s gathering of more than thousand datasets do not actually link to (still) downloadable (dereferenceable) Linked Data (Polleres et al., 2020). Another challenge facing potential adopters is the plethora of choices, not just to navigate, understand and select to reuse the many possible vocabularies and ontologies (Carriero et al., 2020), but also technological choices on RDF serialisation (at least 7 formats), type system (RDFS (Guha & Brickley, 2014), OWL (W3C OWL Working Group, 2012), OBO (Tirmizi et al., 2011), SKOS (Isaac & Summers, 2009)), and deployment challenges (Sauermann et al., 2008) (e.g., hash vs slash in namespaces, HTTP status codes and PID redirection strategies).

Linked Data: rebuilding the Web of Data

The Linked Data (LD) concept (Bizer, Heath & Berners-Lee, 2009) was kickstarted as a set of best practices (Berners-Lee, 2006) to bring the Web aspect of the Semantic Web back into focus. Crucial to Linked Data is the reuse of existing URIs, rather than making new identifiers. This means a loosening of the semantic restrictions previously applied, and an emphasis on building navigable data resources, rather than elaborate graph representations.

Vocabularies like schema.org evolved not long after, intended for lightweight semantic markup of existing Web pages, primarily to improve search engines’ understanding of types and embedded data. In addition to several such embedded microformats (OGP, 2022; Sporny et al., 2015; Microdata, 2023), we find JSON-LD (Sporny et al., 2020) as a Web-focused RDF serialisation that aims for improved programmatic generation and consumption, including from Web applications. JSON-LD is as of 2023-05-18 used⁶ by 45% of the top 10 million websites (W3Techs, 2023).

Recently there has been a renewed emphasis to improve the Developer Experience (Verborgh, 2018) for consumption of Linked Data, for instance RDF Shapes—expressed in SHACL (Kontokostas & Knublauch, 2017) or ShEx (Baker & Hommeaux, 2019)—can be used to validate RDF Data (Gayo et al., 2017; Thornton et al., 2019) before consuming it programmatically, or reshaping data to fit other models. While a varied set of tools for Linked Data consumptions have been identified, most of them still require developers to gain significant knowledge of the underlying Semantic Web technologies, which hampers adaption by non-LD experts (Klímek et al., 2019), which then tend to prefer non-semantic two-dimensional formats such as CSV files.

A valid concern is that the Semantic Web research community has still not fully embraced the Web, and that the “final 20%” engineering effort is frequently overlooked in favour of chasing new trends such as Big Data and AI, rather than making powerful Linked Data technologies available to the wider groups of Web developers (Verborgh & Vander Sande, 2020). One bridging gap here by the Linked Data movement has been “Linked Data by stealth” approaches such as structured data entry spreadsheets powered by ontologies (Wolstencroft et al., 2011), the use of Linked Data as part of REST Web APIs (Page, De Roure & Martinez, 2011), and as shown by the big uptake by publishers to annotate the Web using schema.org (Bernstein, Hendler & Noy, 2016), with vocabulary use patterns documented by copy-pastable JSON-LD examples, rather than by formalised ontologies or developer requirements to understand the full Semantic Web stack.

Linked Data provides technologies that have evolved over time to satisfy its primary purpose of data interoperability. The needs to embrace the Web and developer experience have been central lessons learned. In contrast, FDO is a new approach with many different potential paths forward, and having a partial overlap with the aims of Linked Data.

Observations

Based on the analysis shown in Table 1, we draw the following conclusions:

The Web has already showed us how one can compose workflows of hetereogeneous Web Services (Wolstencroft et al., 2013). However, this is mostly done via developer or human interaction (Lamprecht et al., 2021). Similiarly, FDO does not enable automatic composition because operation semantics are not well defined. There is a question as to whether the extensive documentation and broad developer usage that is available for Web APIs could potentially be utilised for FDO.

A difference between Web technologies and FDO is the stringency of the requirements for both syntax and semantics. Whereas the Web allows many different syntactic formats (e.g., from HTML to XML, PDFs), FDO realised with DOIP requires JSON. On the semantic front, FDO mandates that every object have a well-defined type and structured form. This is clearly not the case on the Web.

In terms of connectivity and the deployment of applications, the Web has a plethora of software, services, and protocols that are widely deployed. These have shown interoperability. The Web standards bodies (e.g., IETF and W3C) follow the OpenStand principles (OpenStand, 2017) to embrace openness, transparency, and broad consensus. In contrast, FDO has a small number of implementations and corresponding protocols, although with a growing community, as evidenced at the first international FDO conference (Loo, 2022). This is not to say that it is not worth developing further Handle+DOIP implementations in the future, but we note that the current FDO functionality can easily be implemented using Web technologies, even as DOIP-over-HTTP (CNRI, 2023b).

It is also a question as to whether a highly constrained protocol revolving around persistent identifiers is in fact necessary. For example, DOIs are mostly resolved on the web using HTTP redirects with the common https://doi.org/ prefix, hiding their Handle nature as an implementation detail (DOI, 2019).

Mapping of metamodel concepts

The Interoperability Framework for Fast Data also provides a brief metamodel which we use in Table 2 to map and examplify corresponding concepts in FDO’s DOIP realization and the Web using HTTP semantics (Fielding, Nottingham & Reschke, 2022).

From this mapping⁸ we can identify the conceptual similarities between DOIP and HTTP, often with common terminology. Notable are that neither DOIP or HTTP have strong support for transactions (explored further in section Comparing FDO and Web as middleware infrastructures), as well that HTTP has poor direct support for processes, as the Web is primarily stateless by design.

Observations

• G1 and G2 call for stability and trustworthiness. While the foundations of both DOIP and Linked Data approaches are now well established—the FDO requirements and in particular how they can be implemented are still taking shape and subject to change.

• Machine actionability (G4, G6) is a core feature of both FDOs and Linked Data. Conceptually they differ in the way types and operations are discovered, with FDO seemingly more rigorous. In practice, however, we see that DOIP also relies on dynamic discovery of operations and that operation expectations for types (FDOF7) have not yet been defined.

• FDO proposes that types can have additional operations beyond CRUD (FDOF5, FDOF6), while Linked Data mainly achieves this with RESTful patterns using CRUD on additional resources, e.g., order/152/items. These are mainly stylistics but affect the architectural view—FDOs have more of an object-oriented approach.

• FDO puts strong emphasis on the use of PIDs (FDOF1, FDOF2, FDOF3, FDOF5), but in current practice DOIP use local types, local extended operations (FDOF5) and attributes (FDOF4) that are not bound to any global namespace.

• Linked Data have a strong emphasis on semantics (FDOF8), and metadata schemas developed by community agreements (FDOF10). FDO types need to be made reusable across servers.

• While FDO recommends nested metadata FDOs (FDOF8, FDOF9), in practice this is not found (or linked with custom keys), particularly due to lack of namespaces and the favouring of local types rather than type/property re-use. Linked Data frequently have multiple representations, but often not sufficiently linked (link relation alternate (Nottingham, 2017)) or related (prov:specializationOf from Lebo, McGuinness & Sahoo (2013)).

• FDO collections are not yet defined for DOIP, while Linked Data seemingly have too many alternatives. LDP has specific native support for containers.

• Tombstones for deleted resources are not well supported, nor specified, for either approach, although the continued availability of metadata when data is removed is a requirement for FAIR principles (RDA-A2-01M).

• DOIP supports multiple chunks of data for an object (FDOF3), while Linked Data can support content-negotiation. In either case it can be unclear to clients what is the meaning or equivalence of any additional chunks.

Observations

Based on the analysis in Table 4, we make the following observations:

• With respect to the aspect of Performance, it is interesting to note that the first version of DOIP (Reilly, 2009) supported multiplexed channels similar to HTTP/2 (allowing concurrent transfer of several digital objects). Multiplexing was removed for the much simplified DOIP 2.0 (DOIPV2.0, 2018). Unlike DOIP 1.0, DOIP 2.0 will require a DO response to be sent back completely, as a series of segments (which again can be split the bytes of each binary element into sized chunks), before transmission of another DO response can start on the transport channel. It is unclear what is the purpose of splitting a binary into chunks on a channel which no longer can be multiplexed and the only property of a chunk is its size¹⁰ .

• HTTP has strong support for scalability and caching, but this mostly assumes read-operations from static resources. FDO has no view on immutability or validity of retrieved objects, but this should be taken into consideration to support large-scale usage.

• HTTP optimisations for performance (e.g., HTTP/2, multiplexing) is largely used for commercial media distribution (e.g., Netflix), and not commonly used by providers of FAIR data

• Cloud deployment of Web applications give many middleware benefits (Scalability, Distribution, Access transparancy, Location transparancy)—it is unclear how DOIP as a custom protocol would perform in a cloud setting as most of this infrastructure assumes HTTP as the protocol.

• Programmatically the Web is rather unstructured as middleware, as there are many implementation choices. Usually it is undeclared what to expect for a given URI/service, and programmers follow documented examples for a particular service rather than automated programmatic exploration across providers. This mean one can consider the Web as an ecosystem of smaller middlewares with commonalities.

• Many providers of FAIR Linked Data also provide programmatic REST API endpoints, e.g., UNIPROT, ChEMBL, but keeping the FAIR aspects such as retrieving metadata in such a scenario may require combining different services using multiple formats and identifier conventions.

Observations

• Linked Data in general is strong on metadata indicators, but LDP approach is weak as it has little concrete metadata guidance.

• FDO/DOIP are stronger on identifier indicators, while Linked Data approach for identifiers relies on best practices.

• Indicators on standard protocols (RDA-A1-04M, RDA-A1-04D, RDA-A1.1-01M, RDA-A1.1-01D) favour LDP’s mature standards (HTTP, URI)—the DOIPv2 specification (DOIPV2.0, 2018) has currently only a couple of implementations and is expressed informally. The underlying Handle system for PIDs is arguably mature and commonly used by researchers (this article alone references about 80 DOIs), however DOIs are more commonly accessed as HTTP redirects through resolvers like https://doi.org/ and http://hdl.handle.net/ rather than the Handle protocol.

• RDA-A1-02M and RDA-A1-02D highlights access by manual intervention, which is common for http/https URIs, but also using above PID resolvers for DOIP implementation CORDRA (e.g., https://hdl.handle.net/21.14100/90ec1c7b-6f5e-4e12-9137-0cedd16d1bce), yet neither LDP, FDO nor DOIP specifications recommends human-readable representations to be provided

• Neither DOIP nor LDP require license to be expressed (RDA-R1.1-01M, RDA-R1.1-02M, RDA-R1.1-03M), yet this is crucial for re-use and machine actionability of FAIR data and metadata to be legal

• Machine-understandable types, provenance and data/metadata standards (RDA-R1.1-03M RDA-R1.3-02M, RDA-R1.3-02M, RDA-R1.3-02D) are important for machine actionability, but are currently unspecified for FDOs. Blanchi et al. (2023) explores possible machine-readable FDO types, however the type systems themselves have not yet been formalised. Linked Data on the other side have too many semantic and syntactic type systems, making it difficult to write consistent clients.

• Indicators for FAIR data are weak for either approach, as too much reliance is put on metadata. For instance in Linked Data, given a URL of a CSV file, what is its persistant identifier or license information? Signposting (Van de Sompel & Nelson, 2015) can improve findability of metadata using HTTP Link relations, which enable an FDO-like overlay for any HTTP resource. In DOIP, responses for bytestreams can include the data identifier: if that is a PID (not enforced by DOIP), its metadata is accessible.

• Resolving FDOs via Handle PIDs to the corresponding DOIP server is currently undefined by FDO and DOIP specifications. 0.TYPE/DOIPServiceInfo lookup is only possible once DOIP server is known.

Observations

Firstly, we observe that the EOSC IF recommendations are at a high level, mainly affecting governance and practices by communities. This Organizational level is also highlighted by the FDO recommendations, for instance the FDO Typing (Lannom et al., 2022c) propose a governance structure to recognize community-endorsed services. While these community aspects are not mandated by Linked Data practices, best practices have become established for aspects like ontology development (Norris et al., 2021). EOSC IF’s Technical layer is likewise at a architecturally high level, such as service-level agreements, but also highlight PID policies which is strongly required by FDO, while Linked Data communities choose PID practices separately. The recommendations for the Semantic layer is largely already implemented by Linked Data practices, yet for FDO mostly consist of encouragements. For instance clear definitions of semantic concepts is required by FDO guidelines, but how to technically define them has not been formalised by FDO specifications.

The Legal layer of interoperability is perhaps the one most emphasised by EOSC, by enabling collaboration across organizational barriers to joinly build a research infrastructure, but this is an area that both FDO and Linked Data are relatively weak in directly supporting. The EOSC IF recommendations in this layer are largely related to governance practices and metadata, for instance licensing, privacy and usage policies; these are also essential for cross-institutional and cross-repository access of FAIR objects.

Likewise, search and indexing is important FAIR aspect for Findability, but is poorly supported globally by FDO and Linked Data. Efforts such as Open Research Knowledge Graph (ORKG) (Jaradeh et al., 2019), DataCite’s PID Graph (Fenner & Aryani, 2019) and Google Knowledge Graph (Singhal, 2012) have improved programmatic findability to some degree, however not significantly for domain-specific semantic artefacts, currently scattered across multiple semantic catalogues (Corcho et al., 2023). There is a strong role for organizations like EOSC to provide such broader registries, moving beyond scholarly output metadata federations. The EOSC Marketplace (https://marketplace.eosc-portal.eu/) has for instance recently been expanded to include training material, software and data sources.