diff --git a/README.md b/README.md index a517c2ab..720b3210 100644 --- a/README.md +++ b/README.md @@ -2,23 +2,24 @@ # pyjelly -**pyjelly** is a Python implementation of [Jelly](http://w3id.org/jelly), a high-performance binary serialization format and streaming protocol for RDF knowledge graphs. +**pyjelly** is a Python implementation of [Jelly](https://w3id.org/jelly), a high-performance binary serialization format and streaming protocol for RDF knowledge graphs. **Documentation, usage guide and more: https://w3id.org/jelly/pyjelly** ## Features -- **Fast reading and writing** of RDF knowledge graphs in the [Jelly format](http://w3id.org/jelly). -- Reading and writing **[SPARQL query results](https://w3id.org/jelly/pyjelly/dev/sparql)** in [Jelly-SPARQL](https://w3id.org/jelly/dev/specification/sparql), with rdflib or the generic API. -- **Standalone [generic API](https://w3id.org/jelly/pyjelly/dev/generic-sink)** with no third-party dependencies, allowing for: +- **[Fast](https://w3id.org/jelly/dev/performance/rdf/#pyjelly) reading and writing** of RDF knowledge graphs in the [Jelly format](https://w3id.org/jelly). pyjelly writes [Jelly-RDF 1.2](https://w3id.org/jelly/dev/specification/serialization/) by default, and reads Jelly-RDF 1.0–1.2. +- Reading and writing **[SPARQL query results](https://w3id.org/jelly/pyjelly/dev/sparql)** in [Jelly-SPARQL](https://w3id.org/jelly/dev/specification/sparql), with RDFLib or the generic API. +- **Standalone [generic API](https://w3id.org/jelly/pyjelly/dev/generic-sink)** that doesn't need RDFLib or any other RDF library, allowing for: - Serialization and parsing of statements to and from Jelly files. - Parsing and serializing streams of graphs and statements. -- Precise control over **serialization options, framing and compression**. + - [RDF 1.2](https://www.w3.org/TR/rdf12-concepts/) triple terms and literals with a base direction. +- Precise control over **serialization options and framing**. - **Seamless** integration with: - - **[rdflib](https://w3id.org/jelly/pyjelly/dev/getting-started)** + - **[RDFLib](https://w3id.org/jelly/pyjelly/dev/getting-started)** - **[RDFLib-Neo4j](https://w3id.org/jelly/pyjelly/dev/rdflib-neo4j-integration)** - **[NetworkX](https://w3id.org/jelly/pyjelly/dev/networkx-integration)** -- **Stream processing support** for large datasets or streams of all [physical stream types](https://w3id.org/jelly/dev/specification/reference/#physicalstreamtype). +- **Stream processing support** for large datasets and streams of many RDF graphs or datasets. **pyjelly** is useful when dealing with (see [full description](https://w3id.org/jelly/pyjelly/dev/overview/#use-cases)): @@ -44,7 +45,7 @@ g.parse("http://xmlns.com/foaf/spec/index.rdf") g.serialize(destination="foaf.jelly", format="jelly") ``` -To read a Jelly file and convert it to an rdflib `Graph`: +To read a Jelly file and convert it to an RDFLib `Graph`: ```python from rdflib import Graph diff --git a/docs/generic-sink.md b/docs/generic-sink.md index db70153d..a00d87a5 100644 --- a/docs/generic-sink.md +++ b/docs/generic-sink.md @@ -10,10 +10,7 @@ Install pyjelly from PyPI: pip install pyjelly ``` -### Requirements - -- Python 3.10 or newer -- Linux, macOS, or Windows +The requirements are the same as [with RDFLib](getting-started.md#requirements). ## Usage without external libraries @@ -27,13 +24,19 @@ To make a set of triples/quads and write them to a Jelly file, use: This example uses pyjelly’s simple custom triple/quad type, which is easy to create and work with. +The terms in the statements are `IRI`, `BlankNode` and `Literal`. You can read their parts with `IRI.iri`, `BlankNode.identifier`, and `Literal.lex`, `Literal.langtag`, `Literal.datatype` (the datatype IRI as a string) and `Literal.direction`. `str()` gives a term in N-Triples syntax, e.g., `` or `"Hello"@en`. + ## Parsing statements from a Jelly file -To load triples/quads into your python object from a `.jelly` file, see: +To load triples/quads from a `.jelly` file into a `GenericStatementSink`, see: {{ code_example('generic/02_parse.py') }} -Which retrieves data from your `.jelly` file. +`GenericStatementSink` keeps all statements in memory. To read a large file statement by statement instead, use `parse_jelly_flat` in a generator: + +{{ code_example('generic/03_streaming.py') }} + +Only the statement that is being processed is kept in memory. ### Parsing a stream of graphs @@ -41,25 +44,17 @@ Similarly, to process a Jelly stream as a stream of graphs through generic API, {{ code_example('generic/06_parse_grouped.py') }} -Where we use a [dataset of weather measurements](https://w3id.org/riverbench/datasets/lod-katrina/dev) and count the number of triples in each graph. +We use a [dataset of weather measurements](https://w3id.org/riverbench/datasets/lod-katrina/dev) and count the number of triples in each graph. ### Parsing a stream of statements -You can also process a Jelly stream as a flat stream with only generic API: - -We look through a fragment of Denmark's OpenStreetMap to find all city names: +You can also process a Jelly stream as a flat stream of statements. We look through a fragment of Denmark's OpenStreetMap to find all city names: {{ code_example('generic/07_parse_flat.py') }} -We get a generator of stream events, which allows us to process the file statement-by-statement, however with no external libraries used. +`parse_jelly_flat` returns a generator of statements, so the file is processed statement by statement. -## Streaming data - -If you need to process a certain quantity of statements both efficiently and iteratively, you can provide a simple generator: - -{{ code_example('generic/03_streaming.py') }} - -With this method you avoid storing all statements in memory, which greatly improves performance. +## Serializing streams ### Serializing a stream of graphs @@ -67,35 +62,34 @@ If you have a generator object containing graphs, you can use a generic approach {{ code_example('generic/04_serialize_grouped.py')}} -Grouped data is streamed in its original form, no need for additional RDF libraries like RDFLib. +Each sink is written as one [RDF Message]({{ proto_link("specification/serialization/#stream-types") }}), so the grouping is kept when the stream is read with `parse_jelly_grouped`. ### Serializing a stream of statements -Serializing a generator object of statements to `.jelly` file through generic API: +If you have a generator object containing statements, you can serialize it like this: {{ code_example('generic/05_serialize_flat.py')}} -Data is transmitted and kept ordered and simple. +The statements are written in order, as one flat stream. -### Working with byte buffers and Kafka +## Working with byte buffers and Kafka When working with Kafka or other message brokers, you may want to write Jelly data to a byte buffer instead of a file. You can do this by using the `BytesIO` class from the `io` module: {{ code_example('generic/08_byte_buffer.py')}} -The `data` variable is of type `bytes`, and can be passed to Kafka with [`KafkaProducer.send(value=data)`](https://kafka-python.readthedocs.io/en/master/), or any other API that accepts byte buffers. Same trick may be used when working with the [RDFLib integration](getting-started.md). +The `data` variable is of type `bytes`, and can be passed to Kafka with [`KafkaProducer.send(value=data)`](https://kafka-python.readthedocs.io/en/master/), or any other API that accepts byte buffers. With the [RDFLib integration](getting-started.md), write to the buffer with `graph.serialize(destination=buffer, format="jelly")`, and read the bytes back with `graph.parse(data=data, format="jelly")`. When working with Kafka, you should be aware of the broker's offset management and partitioning strategies. Data within one Jelly stream must be strictly ordered and no frames may be dropped. If you have less strict ordering guarantees, you should split up the stream into multiple Jelly streams, each with guaranteed consistency. -### Accessing stream frame metadata +## Accessing stream frame metadata -It is possible to access [`RdfStreamFrame` metadata](https://w3id.org/jelly/dev/specification/serialization/#stream-frame-metadata) in pyjelly via **context variables**. +It is possible to access [`RdfStreamFrame` metadata]({{ proto_link("specification/serialization/#stream-frame-metadata") }}) in pyjelly via **context variables**. To do that, define a **context variable** before parsing and pass it as argument to parser functions, e.g.: {{ code_example('generic/09_metadata.py')}} - -### See also +## See also If you are familiar with RDFLib, you can use pyjelly together with RDFLib in a similar way. [See the dedicated guide](getting-started.md). diff --git a/docs/getting-started.md b/docs/getting-started.md index 7b835f07..489af65a 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -42,6 +42,8 @@ Prefixes bound in RDFLib's namespace manager will then be written into the Jelly g.namespace_manager.bind("ex", EX, replace=True) ``` + RDFLib binds about 30 common prefixes to every new graph, and they are all written as well. To write only the prefixes you bind yourself, create the graph with `Graph(bind_namespaces="none")`. + ### Parsing a graph @@ -53,7 +55,7 @@ RDFLib will reconstruct the graph from the Jelly file. ### Parsing a stream of graphs -You can process a Jelly stream as a stream of graphs. A Jelly file consists of "frames" (batches of statements) – we can load each frame as a separate RDFLib graph. +You can process a Jelly stream as a stream of graphs. A Jelly stream can hold a sequence of graphs, each in its own [RDF Message]({{ proto_link("specification/serialization/#stream-types") }}) – we can load each one as a separate RDFLib graph. In this example, we use a [dataset of weather measurements](https://w3id.org/riverbench/datasets/lod-katrina/dev). We count the number of triples in each graph: @@ -82,7 +84,6 @@ If you have a generator object containing graphs, you can easily serialize it in {{ code_example('rdflib/06_serialize_grouped.py')}} This method allows for transmitting logically grouped data, preserving their original division: each graph is one [RDF Message]({{ proto_link("specification/serialization/#stream-types") }}). -For more precise control over frame serialization you can use [lower-level API](api.md) ### Serializing a stream of statements @@ -91,17 +92,20 @@ If you have a generator object containing statements, you can easily serialize i {{ code_example('rdflib/07_serialize_flat.py')}} The flat method transmits the data as a continuous sequence of statements, keeping it simple and ordered. -For more precise control over frame serialization you can use [lower-level API](api.md) + +For more control over framing and other options, see [`pyjelly.integrations.rdflib.serialize`](api.md#pyjelly.integrations.rdflib.serialize) in the API reference. ### File extension support -You can generally omit the `format="jelly"` parameter if the file ends in `.jelly` – RDFLib will auto-detect the format: +When parsing, you can generally omit the `format="jelly"` parameter if the file ends in `.jelly` – RDFLib will auto-detect the format: {{ code_example('rdflib/03_parse_autodetect.py') }} !!! warning - Unfortunately, the way this is implemented in RDFLib is a bit wonky, so it will only work if you explicitly import `pyjelly.integrations.rdflib`, or you used `format="jelly"` in the `serialize()` or `parse()` call before. + Unfortunately, the way this is implemented in RDFLib is a bit wonky, so it will only work if you explicitly import `pyjelly.integrations.rdflib`, or you used `format="jelly"` in a `serialize()` or `parse()` call before. + + This only works for parsing. `serialize()` does not look at the file extension, so `g.serialize("out.jelly")` writes Turtle. Always pass `format="jelly"` when serializing. ## See also diff --git a/docs/index.md b/docs/index.md index c9feb8b1..b0c703c8 100644 --- a/docs/index.md +++ b/docs/index.md @@ -6,7 +6,7 @@ ## Features -- **Fast reading and writing** of RDF knowledge graphs in the [Jelly format](http://w3id.org/jelly) +- **Fast reading and writing** of RDF knowledge graphs in the [Jelly format]({{ proto_link("") }}) - **[SPARQL query results](sparql.md)** in [Jelly-SPARQL]({{ proto_link("specification/sparql/") }}) - **Seamless integration with [RDFLib](https://rdflib.readthedocs.io/)** - **Stream processing support** for large datasets or streams of many RDF graphs/datasets @@ -15,21 +15,23 @@ - **[Getting started](getting-started.md)** - **[Overview and supported features](overview.md)** -- **[Generic interface (usage without RDFLib)](generic-sink.md)** +- **[Generic API (usage without RDFLib)](generic-sink.md)** - **[SPARQL query results](sparql.md)** +- **[Neo4j](rdflib-neo4j-integration.md)** and **[NetworkX](networkx-integration.md)** integrations - **[API reference](api.md)** - **[Contributing](contributing/index.md)** – how to report issues, contribute code, and request features ## Compatibility -pyjelly is compatible with Python 3.10 and newer. Python 3.10–3.14 are tested in CI and are guaranteed to work. +pyjelly is compatible with Python 3.10 and newer. Python 3.10–3.15 are tested in CI and are guaranteed to work. The following table shows the compatibility of the pyjelly implementation with other libraries: -| pyjelly | Python | RDFLib | -|-----------------------------------------------------------------------------------------------------------------------------------------------------|------------|----------------| -| [0.7.0](https://w3id.org/jelly/pyjelly/0.7.0) – **[{{ python_package_version() }}](https://w3id.org/jelly/pyjelly/{{ python_package_version() }})** | 3.10 – 3.14| 7.x (>= 7.1.4) | -| [0.1.0](https://w3id.org/jelly/pyjelly/0.1.0) – [0.6.2](https://w3id.org/jelly/pyjelly/0.6.2) | 3.9 – 3.13 | 7.x (>= 7.1.4) | +| pyjelly | Python | RDFLib | Jelly-RDF | +|-----------------------------------------------------------------------------------------------------------------------------------------------------|------------|----------------|-----------| +| [0.9.0](https://w3id.org/jelly/pyjelly/0.9.0) – **[{{ python_package_version() }}](https://w3id.org/jelly/pyjelly/{{ python_package_version() }})** | 3.10 – 3.15| 7.x (>= 7.1.4) | 1.2 | +| [0.7.0](https://w3id.org/jelly/pyjelly/0.7.0) – [0.8.2](https://w3id.org/jelly/pyjelly/0.8.2) | 3.10 – 3.14| 7.x (>= 7.1.4) | 1.1 | +| [0.1.0](https://w3id.org/jelly/pyjelly/0.1.0) – [0.6.2](https://w3id.org/jelly/pyjelly/0.6.2) | 3.9 – 3.13 | 7.x (>= 7.1.4) | 1.1 | See the **[release notes on GitHub](https://github.com/Jelly-RDF/pyjelly/releases)** for more details. diff --git a/docs/networkx-integration.md b/docs/networkx-integration.md index c21e547d..00d3ccf5 100644 --- a/docs/networkx-integration.md +++ b/docs/networkx-integration.md @@ -5,11 +5,9 @@ NetworkX is a Python package that represents complex networks as graphs and allo Install the following libraries: ```bash -pip install pyjelly[rdflib] networkx==3.2.1 matplotlib==3.9.4 +pip install pyjelly[rdflib] networkx matplotlib ``` -Below there are few useful examples to follow. - ## Parse graph, show it Let's investigate relationships between worldwide political figures (support and opposition relations)! @@ -23,7 +21,7 @@ We can easily load it: Output from `print()`: ```text -Loaded graph with 90000 instances. +Loaded graph with 90000 triples. ``` Convert it into a convenient NetworkX graph: @@ -48,26 +46,26 @@ Output from `print()`: ```text Top 5 nodes sorted by degree: -Socrates: 241 +Sócrates: 241 Cavaco: 189 Passos Coelho: 187 Costa: 179 -Antonio Costa: 168 +António Costa: 168 ``` What is the shortest path between two nodes? We can check: -{{ code_example('networkx_integration/01_parse_calculate_visualize.py', 75, 79) }} +{{ code_example('networkx_integration/01_parse_calculate_visualize.py', 68, 81) }} Output from `print()`: ```text -Shortest path from Socrates to Obama: Socrates -> Marcelo Rebelo de Sousa -> Durao Barroso -> Obama +Shortest path from Socrates to Obama: Socrates -> Marcelo Rebelo de Sousa -> António Costa -> Obama ``` However, it's best to see the full picture (for our example we truncate to 10 nodes for clarity): -{{ code_example('networkx_integration/01_parse_calculate_visualize.py', 81, 92) }} +{{ code_example('networkx_integration/01_parse_calculate_visualize.py', 83, 94) }} The graph presents as follows @@ -78,7 +76,7 @@ The graph presents as follows In summary: -{{ snippet_admonition('examples/networkx_integration/01_parse_calculate_visualize.py', 0, 95, title="Entire example", expanded=False) }} +{{ snippet_admonition('examples/networkx_integration/01_parse_calculate_visualize.py', 0, 96, title="Entire example", expanded=False) }} We converted an RDFLib graph to NetworkX, calculated insightful metrics and visualized the graph. @@ -86,7 +84,7 @@ For more info about the data source please see the [Politiquices dataset in Rive ## Serialize NetworkX graph -This example shows how to write a NetworkX graph to a Jelly file.: +This example shows how to write a NetworkX graph to a Jelly file: {{ code_example('networkx_integration/02_serialize.py') }} diff --git a/docs/overview.md b/docs/overview.md index f1aa7bca..0c1d96d7 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -9,12 +9,12 @@ With Jelly, you can transmit both flat and structured streams of triples, quads, * Full support for reading and writing Jelly-encoded RDF data * Reading and writing SPARQL query results in [Jelly-SPARQL](sparql.md) * Seamless integration with [RDFLib](https://rdflib.readthedocs.io/) (*"works just like Turtle"*) -* Standalone generic API with no third-party dependencies [Generic API](generic-sink.md) +* Standalone [generic API](generic-sink.md) that doesn't need RDFLib or any other RDF library * Support for all Jelly stream types * Tools for working with delimited and non-delimited Jelly streams -* Fine-grained control over serialization options, compression, and framing +* Fine-grained control over serialization options and framing -## Overview +## Supported features ### Jelly-RDF versions @@ -38,7 +38,11 @@ A Jelly-RDF 1.2 stream is either one graph or dataset, or a sequence of [RDF Mes ### RDF 1.2 -The generic API supports [RDF 1.2](https://www.w3.org/TR/rdf12-concepts/): triple terms, as a `Triple` in the object of a statement, and directional language-tagged strings, as `Literal("text", "en", direction="ltr")`. To declare the RDF version for a stream, set `rdf_version` in `StreamParameters`. RDFLib 7 doesn't support RDF 1.2 yet. +The [generic API](generic-sink.md) supports [RDF 1.2](https://www.w3.org/TR/rdf12-concepts/): triple terms, as a `Triple` in the object of a statement, and directional language-tagged strings, as `Literal("text", "en", direction="ltr")`. RDFLib 7 doesn't support RDF 1.2 yet. + +To declare that a stream uses RDF 1.2, set `rdf_version=jelly.RDF_VERSION_1_2` in `StreamParameters` (`jelly` is `pyjelly.jelly`). `GenericStatementSink.serialize()` doesn't take options, so write the statements with `flat_stream_to_file` or `grouped_stream_to_file`: + +{{ code_example('generic/11_rdf_1_2.py') }} ### SPARQL query results @@ -61,18 +65,13 @@ Use cases for pyjelly include: - We are working on support for pandas and other data science libraries – stay tuned for updates! - **Database dumps and bulk loads** – quickly read and write large RDF datasets with Jelly, reducing storage space and improving database maintenance tasks. -## pyjelly integration with external libraries +## Integrations with other libraries -To learn how to use popular third-party libraries that connect with RDFLib, please see: +pyjelly works with libraries that connect to RDFLib, see: - **[RDFLib-Neo4j integration](rdflib-neo4j-integration.md)** - **[NetworkX integration](networkx-integration.md)** -## Generic API - -pyjelly includes its own [generic API](generic-sink.md) for working with RDF data (no RDFLib or other external RDF libraries required). -It provides simple, built-in types for triples and quads, allowing you to create, read, and write data directly in the Jelly format. - ## Pre-compiled binaries pyjelly is written in pure Python, so you can run it on any platform that supports Python. We also provide pre-compiled binaries using [mypyc](https://mypyc.readthedocs.io/en/latest/) for better performance. Supported platforms include Windows, macOS, and Linux (x86_64 and ARM64). When installing `pyjelly`, your package manager will automatically download the right binary, or, if your platform is not supported, it will automatically fall back to the pure Python implementation. You can find the full list of supported platforms on [pypi](https://pypi.org/project/pyjelly/#files). diff --git a/docs/rdflib-neo4j-integration.md b/docs/rdflib-neo4j-integration.md index e283c54a..508ea57a 100644 --- a/docs/rdflib-neo4j-integration.md +++ b/docs/rdflib-neo4j-integration.md @@ -14,11 +14,11 @@ pip install pyjelly[rdflib] rdflib-neo4j ## Parsing data from a Jelly file into Neo4j -To parse data from a `.jelly` file into the Neo4j database, use the following example (insert your own credentials to AuraDB): +To parse data from a `.jelly` file into the Neo4j database, use the following example (insert the URI and credentials of your Neo4j instance, e.g., AuraDB): -{{ snippet_admonition('examples/neo4j_integration/01_rdflib_neo4j_parse_grouped.py',0, 50, title="Parsing", expanded=True) }} +{{ snippet_admonition('examples/neo4j_integration/01_rdflib_neo4j_parse.py', 0, 100, title="Parsing", expanded=True) }} -which inserts your data into your AuraDB database. +which inserts your data into your Neo4j database. ## Related sources diff --git a/docs/sparql.md b/docs/sparql.md index d55ead53..89aabf0a 100644 --- a/docs/sparql.md +++ b/docs/sparql.md @@ -40,7 +40,7 @@ for row in Graph(store=store).query("SELECT * WHERE { ?s ?p ?o } LIMIT 10"): ### Large results -By default, `Result.parse` reads all rows at once. With `lazy=True`, the rows are read as the result is iterated, like RDFLib does with the results of its own queries. The stream must then stay open until you are done with the result. +By default, `Result.parse` reads all rows at once. With `Result.parse(file, format="jelly", lazy=True)`, the rows are read as the result is iterated, like RDFLib does with the results of its own queries. The stream must then stay open until you are done with the result. When writing, the rows of an RDFLib result are written as the query makes them, so the first frames are written before the query is done. Each frame is flushed when it's written. @@ -79,12 +79,16 @@ options = SparqlOptions( # Sizes of the lookups, see below lookup_preset=LookupPreset(max_names=4096, max_prefixes=256, max_datatypes=32), # How many values (rows times variables) a frame may have, at most - frame_values=4096, + frame_values=1024, ) results.serialize(destination="results.jellys", format="jelly", options=options) ``` -The default lookup sizes are 8192 names, 1024 prefixes and 64 datatypes. Readers accept lookups of up to 16384 names, 4096 prefixes and 256 datatypes. +The default lookup sizes are 8192 names, 1024 prefixes and 64 datatypes, and a frame has at most 4096 values by default. Readers accept lookups of up to 16384 names, 4096 prefixes and 256 datatypes. + +!!! note + + `LookupPreset()` without arguments has the smaller defaults of Jelly-RDF (4000 names, 150 prefixes and 32 datatypes), not the defaults of `SparqlOptions`. Give all three sizes when you make a `LookupPreset` for SPARQL results. With `delimited=False`, the results are written as one frame without its size, e.g., to be used in Kafka or MQTT. Then the lookups must be large enough to store all terms of the result set. Files and HTTP responses must be delimited, which is the default. To send the frames of the results as separate messages, see [writing frame by frame](#writing-frame-by-frame). diff --git a/examples/generic/03_streaming.py b/examples/generic/03_streaming.py index 4383c27f..72c5e02a 100644 --- a/examples/generic/03_streaming.py +++ b/examples/generic/03_streaming.py @@ -1,16 +1,15 @@ -from pyjelly.integrations.generic.generic_sink import * +from pyjelly.integrations.generic.parse import parse_jelly_flat # Helper generator that streams statements from a Jelly file path -def stream_triples(jelly_path): - generic_sink = GenericStatementSink() +def stream_statements(jelly_path): with open(jelly_path, "rb") as f: - generic_sink.parse(f) - yield from (stmt for stmt in generic_sink) + # Statements are read one by one, as they are needed + yield from parse_jelly_flat(f) # Example usage, just printing: -for triple in stream_triples("output.jelly"): - print(triple) +for statement in stream_statements("output.jelly"): + print(statement) print("All done.") diff --git a/examples/generic/04_serialize_grouped.py b/examples/generic/04_serialize_grouped.py index 5e4c5042..9d25eb99 100644 --- a/examples/generic/04_serialize_grouped.py +++ b/examples/generic/04_serialize_grouped.py @@ -3,16 +3,17 @@ import random -# Helper function to generate a generator of graphs +# Example generator that yields graphs def generate_sample_sinks(): - content = ( - IRI("http://example.com/sensor"), - IRI("http://example.com/humidity"), - IRI(f"http://example.com/{random.random()}"), - ) for _ in range(10): sink = GenericStatementSink() - sink.add(Triple(*content)) + sink.add( + Triple( + IRI("http://example.com/sensor"), + IRI("http://example.com/humidity"), + Literal(str(random.random())), + ) + ) yield sink diff --git a/examples/generic/05_serialize_flat.py b/examples/generic/05_serialize_flat.py index d09f06bb..9f253d3d 100644 --- a/examples/generic/05_serialize_flat.py +++ b/examples/generic/05_serialize_flat.py @@ -5,13 +5,12 @@ # Example generator that yields raw triples def generate_sample_triples(): - content = ( - IRI("http://example.com/sensor"), - IRI("http://example.com/humidity"), - IRI(f"http://example.com/{random.random()}"), - ) for _ in range(10): - yield Triple(*content) + yield Triple( + IRI("http://example.com/sensor"), + IRI("http://example.com/humidity"), + Literal(str(random.random())), + ) output_file = "flat_output.jelly" diff --git a/examples/generic/06_parse_grouped.py b/examples/generic/06_parse_grouped.py index 90eeab6c..55fb75b8 100644 --- a/examples/generic/06_parse_grouped.py +++ b/examples/generic/06_parse_grouped.py @@ -1,5 +1,6 @@ import gzip import urllib.request +from itertools import islice from pyjelly.integrations.generic.parse import parse_jelly_grouped @@ -15,8 +16,6 @@ # Parse into sinks (one per graph) graphs = parse_jelly_grouped(jelly_stream) - # First 50 - for i, graph in enumerate(graphs): + # Only the first 50 graphs, the rest will not be parsed + for i, graph in enumerate(islice(graphs, 50)): print(f"Graph {i} in the stream has {len(graph)} triples") - if i >= 50: - break diff --git a/examples/generic/07_parse_flat.py b/examples/generic/07_parse_flat.py index 94263925..fc8d1fd9 100644 --- a/examples/generic/07_parse_flat.py +++ b/examples/generic/07_parse_flat.py @@ -19,9 +19,9 @@ gzip.open(response) as jelly_stream, ): for event in parse_jelly_flat(jelly_stream): - if isinstance(event, Triple): - if str(event.p) == str(predicate_to_look_for): - city_names.add(str(event.o)) + if isinstance(event, Triple): # we are only interested in triples + if event.p == predicate_to_look_for: + city_names.add(event.o.lex) print(f"Found {len(city_names)} unique city names in the dataset.") print("Sample city names:") diff --git a/examples/generic/09_metadata.py b/examples/generic/09_metadata.py index 57d50523..3b6115e4 100644 --- a/examples/generic/09_metadata.py +++ b/examples/generic/09_metadata.py @@ -1,16 +1,15 @@ +from collections.abc import MutableMapping from contextvars import ContextVar -from google.protobuf.internal.containers import ScalarMap import urllib.request -from pyjelly.integrations.rdflib.parse import parse_jelly_grouped -frame_metadata: ContextVar[ScalarMap[str, bytes]] = ContextVar("frame_metadata") +from pyjelly.integrations.generic.parse import parse_jelly_grouped + +frame_metadata: ContextVar[MutableMapping[str, bytes]] = ContextVar("frame_metadata") url = "https://registry.petapico.org/nanopubs.jelly" with urllib.request.urlopen(url) as response: graphs = parse_jelly_grouped(response, frame_metadata=frame_metadata) - for i, graph in enumerate(graphs): - if i == 1: # Just show first instance - break - print(f"Graph {i} in the stream has {len(graph)} triples.") - metadata = frame_metadata.get() - print(f"Graph #{i}: {metadata}") + # Just show the first graph + graph = next(graphs) + print(f"The first graph in the stream has {len(graph)} triples.") + print(f"Its frame metadata: {frame_metadata.get()}") diff --git a/examples/generic/11_rdf_1_2.py b/examples/generic/11_rdf_1_2.py new file mode 100644 index 00000000..97766053 --- /dev/null +++ b/examples/generic/11_rdf_1_2.py @@ -0,0 +1,31 @@ +from pyjelly import jelly +from pyjelly.integrations.generic.generic_sink import IRI, Literal, Triple +from pyjelly.integrations.generic.parse import parse_jelly_flat +from pyjelly.integrations.generic.serialize import flat_stream_to_file +from pyjelly.options import StreamParameters +from pyjelly.serialize.streams import SerializerOptions + +alice = IRI("http://example.org/alice") + + +def generate_statements(): + # A triple term in the object of a statement + yield Triple( + alice, + IRI("http://example.org/says"), + Triple(alice, IRI("http://example.org/knows"), IRI("http://example.org/bob")), + ) + # A language-tagged string with a base direction + yield Triple( + alice, IRI("http://example.org/name"), Literal("Alice", "en", direction="ltr") + ) + + +# Declare that the stream uses RDF 1.2 +options = SerializerOptions(params=StreamParameters(rdf_version=jelly.RDF_VERSION_1_2)) +with open("rdf12.jelly", "wb") as out_file: + flat_stream_to_file(generate_statements(), out_file, options=options) + +with open("rdf12.jelly", "rb") as in_file: + for statement in parse_jelly_flat(in_file): + print(statement) diff --git a/examples/neo4j_integration/01_rdflib_neo4j_parse_grouped.py b/examples/neo4j_integration/01_rdflib_neo4j_parse.py similarity index 74% rename from examples/neo4j_integration/01_rdflib_neo4j_parse_grouped.py rename to examples/neo4j_integration/01_rdflib_neo4j_parse.py index 017453fc..d3514272 100644 --- a/examples/neo4j_integration/01_rdflib_neo4j_parse_grouped.py +++ b/examples/neo4j_integration/01_rdflib_neo4j_parse.py @@ -6,12 +6,12 @@ # Example file from the Riverbench example_file, _ = urllib.request.urlretrieve("https://w3id.org/riverbench/v/dev.jelly") -# Please introduce your credentials +# Please introduce your credentials, e.g., of your AuraDB instance aura_db_uri = "aura_db_uri" aura_db_username = "neo4j" aura_db_pwd = "aura_db_pwd" -# Prepare the authentication data to the AuraDB +# Prepare the authentication data for Neo4j auth_data = { "uri": aura_db_uri, "database": "neo4j", @@ -29,20 +29,21 @@ # Make a graph with Neo4jStore object neo4j_sink = Graph(store=Neo4jStore(config=config)) -# Parse the file into the AuraDB +# Parse the file into Neo4j neo4j_sink.parse(example_file, format="jelly") -# Close the stream +# Close the store, which writes the remaining batch neo4j_sink.close(True) # Cypher query to check loaded data with GraphDatabase.driver( aura_db_uri, auth=(aura_db_username, aura_db_pwd) ).session() as session: + # Triples with literal objects are stored as node properties, not relationships count = session.run("MATCH ()-[r]->() RETURN count(r) AS c").single()["c"] - print(f"Loaded {count} triples") + print(f"Loaded {count} relationships") for r in session.run( - "MATCH (s)-[r]->(o) RETURN s.uri AS s, type(r) AS p, coalesce(o.uri,o.name) AS o LIMIT 5" + "MATCH (s)-[r]->(o) RETURN s.uri AS s, type(r) AS p, o.uri AS o LIMIT 5" ): print(r["s"], f"-[{r['p']}]->", r["o"]) diff --git a/examples/networkx_integration/01_parse_calculate_visualize.py b/examples/networkx_integration/01_parse_calculate_visualize.py index 3136e36f..fdac18a0 100644 --- a/examples/networkx_integration/01_parse_calculate_visualize.py +++ b/examples/networkx_integration/01_parse_calculate_visualize.py @@ -24,7 +24,7 @@ # Parse RDF from the Jelly format rdf_g = Graph() rdf_g.parse(example_file, format="jelly") -print(f"Loaded graph with {len(rdf_g)} instances.") +print(f"Loaded graph with {len(rdf_g)} triples.") # First step for filtering the graph rdf_g_filtered = Graph() @@ -73,7 +73,11 @@ # Example calculation, shortest path between two nodes (provided at least two nodes) source = next(n for n in nx_g if norm(n) == "socrates") target = next(n for n in nx_g if norm(n) == "obama") -path = nx.shortest_path(nx_g, source=source, target=target) +# There may be several shortest paths, take the first one in alphabetical order +path = min( + nx.all_shortest_paths(nx_g, source=source, target=target), + key=lambda p: [str(n) for n in p], +) print(f"Shortest path from {source} to {target}: {' -> '.join(path)}") # Take first 10 nodes diff --git a/examples/rdflib/01_serialize.py b/examples/rdflib/01_serialize.py index b2c607a4..6f1c8f82 100644 --- a/examples/rdflib/01_serialize.py +++ b/examples/rdflib/01_serialize.py @@ -2,4 +2,5 @@ g = Graph() g.parse("https://www.w3.org/TR/vocab-ssn/integrated/examples/sunspots.ttl") -g.serialize(destination="foaf.jelly", format="jelly") +g.serialize(destination="sunspots.jelly", format="jelly") +print(f"Wrote {len(g)} triples to sunspots.jelly") diff --git a/examples/rdflib/02_parse.py b/examples/rdflib/02_parse.py index adbd2cd3..6d3491d1 100644 --- a/examples/rdflib/02_parse.py +++ b/examples/rdflib/02_parse.py @@ -1,7 +1,7 @@ from rdflib import Graph g = Graph() -g.parse("foaf.jelly", format="jelly") +g.parse("sunspots.jelly", format="jelly") print("Parsed triples:") for s, p, o in g: diff --git a/examples/rdflib/03_parse_autodetect.py b/examples/rdflib/03_parse_autodetect.py index c2298c1d..b7ea4d68 100644 --- a/examples/rdflib/03_parse_autodetect.py +++ b/examples/rdflib/03_parse_autodetect.py @@ -2,4 +2,5 @@ import pyjelly.integrations.rdflib g = Graph() -g.parse("foaf.jelly") +g.parse("sunspots.jelly") +print(f"Parsed {len(g)} triples") diff --git a/examples/rdflib/04_parse_grouped.py b/examples/rdflib/04_parse_grouped.py index fabc8db2..daa08fb3 100644 --- a/examples/rdflib/04_parse_grouped.py +++ b/examples/rdflib/04_parse_grouped.py @@ -1,5 +1,6 @@ import gzip import urllib.request +from itertools import islice from pyjelly.integrations.rdflib.parse import parse_jelly_grouped @@ -13,8 +14,6 @@ gzip.open(response) as jelly_stream, ): graphs = parse_jelly_grouped(jelly_stream) - for i, graph in enumerate(graphs): + # Only the first 50 graphs, the rest will not be parsed + for i, graph in enumerate(islice(graphs, 50)): print(f"Graph {i} in the stream has {len(graph)} triples") - # Limit to 50 graphs for demonstration -- the rest will not be parsed - if i >= 50: - break diff --git a/examples/rdflib/05_parse_flat.py b/examples/rdflib/05_parse_flat.py index 4b16fe7a..782fd507 100644 --- a/examples/rdflib/05_parse_flat.py +++ b/examples/rdflib/05_parse_flat.py @@ -25,6 +25,6 @@ city_names.add(event.o) print(f"Found {len(city_names)} unique city names in the dataset.") -print("10 random city names:") +print("Sample city names:") for city in list(city_names)[:10]: print(f"- {city}") diff --git a/examples/rdflib/08_namespaces.py b/examples/rdflib/08_namespaces.py index e47d0dcd..2ba62774 100644 --- a/examples/rdflib/08_namespaces.py +++ b/examples/rdflib/08_namespaces.py @@ -1,21 +1,20 @@ -from rdflib import Graph, Namespace, URIRef, Literal -from pyjelly.integrations.rdflib.serialize import SerializerOptions, StreamParameters +from rdflib import Graph, Literal, Namespace +from rdflib.namespace import FOAF + +from pyjelly.options import StreamParameters +from pyjelly.serialize.streams import SerializerOptions # Build a tiny graph and bind a prefix -g = Graph() EX = Namespace("http://example.org/") -g.namespace_manager.bind("ex", EX) -g.add((EX.alice, URIRef("http://xmlns.com/foaf/0.1/name"), Literal("Alice"))) - -print("IN namespaces:", dict(g.namespaces())) +g = Graph() +g.bind("ex", EX) +g.add((EX.alice, FOAF.name, Literal("Alice"))) # Enable namespace declarations in Jelly output options = SerializerOptions(params=StreamParameters(namespace_declarations=True)) +g.serialize("namespaces.jelly", format="jelly", options=options) -# Serialize with options -g.serialize("sample_test.jelly", format="jelly", options=options) - -# Parse back and check namespaces +# Parse back and check that the prefix is there g_new = Graph() -g_new.parse("sample_test.jelly", format="jelly") -print("OUT namespaces:", dict(g_new.namespaces())) +g_new.parse("namespaces.jelly", format="jelly") +print("Prefix ex:", dict(g_new.namespaces()).get("ex")) diff --git a/examples/rdflib/10_sparql_results_lazy.py b/examples/rdflib/10_sparql_results_lazy.py index 91d41eee..e06f9a83 100644 --- a/examples/rdflib/10_sparql_results_lazy.py +++ b/examples/rdflib/10_sparql_results_lazy.py @@ -2,10 +2,6 @@ from rdflib.query import Result from pyjelly.errors import JellyIncompleteResultsError -from pyjelly.integrations.rdflib.sparql import ( - parse_jelly_sparql, - serialize_jelly_sparql, -) # Many results, made one by one results = Result("SELECT") @@ -17,12 +13,11 @@ } for i in range(100_000) ) -with open("many_results.jellys", "wb") as file: - serialize_jelly_sparql(results, file) +results.serialize(destination="many_results.jellys", format="jelly") # Read the rows as they are needed, not all at once with open("many_results.jellys", "rb") as file: - parsed = parse_jelly_sparql(file, lazy=True) + parsed = Result.parse(file, format="jelly", lazy=True) try: total = sum(row.number.toPython() for row in parsed) except JellyIncompleteResultsError as error: diff --git a/mkdocs.yml b/mkdocs.yml index 4356d7b8..d6506bfc 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -34,6 +34,10 @@ plugins: show_symbol_type_heading: false show_symbol_type_toc: true summary: true + # Hide private members and the rdf_pb2 alias imported by sparql_pb2 + filters: ["!^_[^_]", "!^rdf__pb2$"] + # Read the source, not the modules that mypyc compiles next to it + allow_inspection: false paths: [pyjelly/] inventories: - http://docs.python.org/3/objects.inv @@ -56,6 +60,11 @@ markdown_extensions: emoji_index: !!python/name:material.extensions.emoji.twemoji emoji_generator: !!python/name:material.extensions.emoji.to_svg +exclude_docs: | + /env.py + /overrides/ + /examples/ + repo_url: https://github.com/Jelly-RDF/pyjelly repo_name: Jelly-RDF/pyjelly edit_uri: edit/main/docs/ @@ -123,6 +132,10 @@ nav: - Neo4j: "rdflib-neo4j-integration.md" - NetworkX: "networkx-integration.md" - API reference: "api.md" + - Performance: + - Jelly-RDF: "https://w3id.org/jelly/dev/performance/rdf/#pyjelly" + - Jelly-SPARQL: "https://w3id.org/jelly/dev/performance/sparql/#pyjelly" + - Jelly-RDF 1.1 vs 1.2: "https://w3id.org/jelly/dev/performance/rdf-1-1-vs-1-2/#pyjelly" - Contributing: - "contributing/index.md" - Code of conduct: "contributing/code_of_conduct.md" diff --git a/pyjelly/integrations/generic/generic_sink.py b/pyjelly/integrations/generic/generic_sink.py index 0165fdea..21ff4f6f 100644 --- a/pyjelly/integrations/generic/generic_sink.py +++ b/pyjelly/integrations/generic/generic_sink.py @@ -13,6 +13,7 @@ def __repr__(self) -> str: DefaultGraph = _DefaultGraph() +"""The name of the default graph, used as the graph name of quads in it.""" class BlankNode: @@ -23,11 +24,16 @@ def __init__(self, identifier: str) -> None: # is a reserved word for the MSVC compiler self._id: str = identifier + @property + def identifier(self) -> str: + """The identifier of the blank node, without the `_:` prefix.""" + return self._id + def __str__(self) -> str: return f"_:{self._id}" def __repr__(self) -> str: - return f"BlankNode(identifier={self._id})" + return f"BlankNode({self._id!r})" def __eq__(self, other: object) -> bool: if isinstance(other, BlankNode): @@ -44,11 +50,16 @@ class IRI: def __init__(self, iri: str) -> None: self._iri: str = iri + @property + def iri(self) -> str: + """The IRI as a string, without angle brackets.""" + return self._iri + def __str__(self) -> str: return f"<{self._iri}>" def __repr__(self) -> str: - return f"IRI({self._iri})" + return f"IRI({self._iri!r})" def __eq__(self, other: object) -> bool: if isinstance(other, IRI): @@ -82,6 +93,26 @@ def __init__( self._datatype: str | None = datatype self._direction: str | None = direction + @property + def lex(self) -> str: + """The lexical form, without quotes or escapes.""" + return self._lex + + @property + def langtag(self) -> str | None: + """The language tag, or None if there is none.""" + return self._langtag + + @property + def datatype(self) -> str | None: + """The datatype IRI as a string, or None if it is not given.""" + return self._datatype + + @property + def direction(self) -> str | None: + """The base direction, "ltr" or "rtl", or None if there is none.""" + return self._direction + def __str__(self) -> str: lex = ( self._lex.replace("\\", "\\\\") @@ -130,28 +161,43 @@ class Triple(NamedTuple): """Class for RDF triples.""" s: Node + """The subject.""" p: Node + """The predicate.""" o: Node + """The object.""" class Quad(NamedTuple): """Class for RDF quads.""" s: Node + """The subject.""" p: Node + """The predicate.""" o: Node + """The object.""" g: GraphName + """The graph name, `DefaultGraph` for the default graph.""" class Prefix(NamedTuple): """Class for generic namespace declaration.""" prefix: str + """The prefix.""" iri: IRI + """The namespace IRI.""" @mypyc_attr(allow_interpreted_subclasses=True) class GenericStatementSink: + """ + A graph or dataset of generic statements, kept in memory in order. + + It holds triples or quads, and the namespaces bound to it. + """ + _store: deque[Triple | Quad] def __init__(self, identifier: GraphName = DefaultGraph) -> None: @@ -172,9 +218,11 @@ def __init__(self, identifier: GraphName = DefaultGraph) -> None: self._id = identifier def add(self, statement: Triple | Quad) -> None: + """Add a statement at the end of the sink.""" self._store.append(statement) def bind(self, prefix: str, namespace: IRI) -> None: + """Bind a prefix to a namespace, replacing an earlier binding of it.""" self._namespaces.update({prefix: namespace}) def __iter__(self) -> Generator[Triple | Quad]: @@ -185,14 +233,17 @@ def __len__(self) -> int: @property def namespaces(self) -> Generator[tuple[str, IRI]]: + """The bound namespaces, as (prefix, namespace IRI) pairs.""" yield from self._namespaces.items() @property def identifier(self) -> GraphName: + """The name of the graph, `DefaultGraph` if it is not given.""" return self._id @property def store(self) -> Generator[Triple | Quad]: + """The statements, in the order they were added.""" yield from self._store @property @@ -207,6 +258,16 @@ def is_triples_sink(self) -> bool: return bool(self._store) and len(self._store[0]) == TRIPLE_ARITY def parse(self, input_file: IO[bytes]) -> None: + """ + Read a Jelly stream into the sink. + + The statements, namespaces and identifier of the sink are replaced with + those read from the stream. + + Args: + input_file (IO[bytes]): binary file to read. + + """ from pyjelly.integrations.generic.parse import ( # noqa: PLC0415 parse_jelly_to_graph, ) @@ -217,6 +278,16 @@ def parse(self, input_file: IO[bytes]) -> None: self._id = parsed_result._id def serialize(self, output_file: IO[bytes]) -> None: + """ + Write the sink as a Jelly stream, with the default options. + + To choose the options, use `flat_stream_to_file` or + `grouped_stream_to_file` from `pyjelly.integrations.generic.serialize`. + + Args: + output_file (IO[bytes]): binary file to write. + + """ from pyjelly.integrations.generic.serialize import ( # noqa: PLC0415 grouped_stream_to_file, ) diff --git a/pyjelly/integrations/rdflib/statements.py b/pyjelly/integrations/rdflib/statements.py index c75d3938..acabd0da 100644 --- a/pyjelly/integrations/rdflib/statements.py +++ b/pyjelly/integrations/rdflib/statements.py @@ -36,14 +36,17 @@ def __new__(cls, s: Node, p: Node, o: Node) -> Self: @property def s(self) -> Node: + """The subject.""" return self[0] @property def p(self) -> Node: + """The predicate.""" return self[1] @property def o(self) -> Node: + """The object.""" return self[2] @@ -67,18 +70,22 @@ def __new__(cls, s: Node, p: Node, o: Node, g: GraphName) -> Self: @property def s(self) -> Node: + """The subject.""" return self[0] @property def p(self) -> Node: + """The predicate.""" return self[1] @property def o(self) -> Node: + """The object.""" return self[2] @property def g(self) -> GraphName: + """The graph name.""" return self[3] @@ -105,8 +112,10 @@ def __new__(cls, prefix: str, iri: rdflib.URIRef) -> Self: @property def prefix(self) -> str: + """The prefix.""" return self[0] @property def iri(self) -> rdflib.URIRef: + """The namespace IRI.""" return self[1] diff --git a/pyjelly/options.py b/pyjelly/options.py index 5bd7cac0..3e4fef6b 100644 --- a/pyjelly/options.py +++ b/pyjelly/options.py @@ -96,6 +96,19 @@ def register_mimetypes(extension: str = ".jelly") -> None: @mypyc_attr(allow_interpreted_subclasses=True) @dataclass(frozen=True) class LookupPreset: + """ + Sizes of the lookups of a stream. + + The defaults are those of Jelly-RDF. Jelly-SPARQL streams use larger lookups + by default, see `SparqlOptions`. + + Attributes: + max_names: size of the name lookup, at least 8. + max_prefixes: size of the prefix lookup, 0 to not use prefixes. + max_datatypes: size of the datatype lookup. + + """ + max_names: int = DEFAULT_NAME_LOOKUP_SIZE max_prefixes: int = DEFAULT_PREFIX_LOOKUP_SIZE max_datatypes: int = DEFAULT_DATATYPE_LOOKUP_SIZE @@ -107,6 +120,7 @@ def __post_init__(self) -> None: @classmethod def small(cls) -> Self: + """Return small lookups: 128 names, 32 prefixes and 32 datatypes.""" return cls(max_names=128, max_prefixes=32, max_datatypes=32) diff --git a/pyjelly/serialize/streams.py b/pyjelly/serialize/streams.py index 00ab1b21..06769bb0 100644 --- a/pyjelly/serialize/streams.py +++ b/pyjelly/serialize/streams.py @@ -44,6 +44,20 @@ @dataclass class SerializerOptions: + """ + Options for writing a Jelly-RDF stream. + + Attributes: + flow: how statements are split into frames. If None, it is chosen from + the logical stream type. + frame_size: target number of rows per frame. + logical_type: logical stream type. If unspecified, it is guessed from + the data. + params: other stream parameters, e.g., the protocol version. + lookup_preset: lookup sizes. + + """ + flow: FrameFlow | None = None frame_size: int = DEFAULT_FRAME_SIZE logical_type: LogicalStreamType = jelly.LOGICAL_STREAM_TYPE_UNSPECIFIED diff --git a/tests/integration_tests/test_sink_generic/test_sink.py b/tests/integration_tests/test_sink_generic/test_sink.py index 87dcfbae..5942b67c 100644 --- a/tests/integration_tests/test_sink_generic/test_sink.py +++ b/tests/integration_tests/test_sink_generic/test_sink.py @@ -49,9 +49,9 @@ def test_len(self) -> None: "expected_repr", ), [ - ("bn1", "BlankNode(identifier=bn1)"), - ("929292", "BlankNode(identifier=929292)"), - ("utf-8~", "BlankNode(identifier=utf-8~)"), + ("bn1", "BlankNode('bn1')"), + ("929292", "BlankNode('929292')"), + ("utf-8~", "BlankNode('utf-8~')"), ], ) def test_bn_repr(input_id: str, expected_repr: str) -> None: @@ -282,56 +282,59 @@ def test_iri_str_representation(iri: str, expected_str: str) -> None: "expected_repr", ), [ - ("http://example.com", "IRI(http://example.com)"), - ("https://example.com", "IRI(https://example.com)"), - ("http://example.com/resource", "IRI(http://example.com/resource)"), + ("http://example.com", "IRI('http://example.com')"), + ("https://example.com", "IRI('https://example.com')"), + ("http://example.com/resource", "IRI('http://example.com/resource')"), ( "https://example.com/path/to/resource", - "IRI(https://example.com/path/to/resource)", + "IRI('https://example.com/path/to/resource')", ), - ("http://example.com#fragment", "IRI(http://example.com#fragment)"), - ("http://example.com?query=value", "IRI(http://example.com?query=value)"), + ("http://example.com#fragment", "IRI('http://example.com#fragment')"), + ("http://example.com?query=value", "IRI('http://example.com?query=value')"), ( "http://example.com/resource?query=value#fragment", - "IRI(http://example.com/resource?query=value#fragment)", + "IRI('http://example.com/resource?query=value#fragment')", ), - ("ftp://example.com", "IRI(ftp://example.com)"), - ("mailto:user@example.com", "IRI(mailto:user@example.com)"), - ("file:///path/to/file", "IRI(file:///path/to/file)"), - ("urn:isbn:0451450523", "IRI(urn:isbn:0451450523)"), - ("http://example.com/résource", "IRI(http://example.com/résource)"), + ("ftp://example.com", "IRI('ftp://example.com')"), + ("mailto:user@example.com", "IRI('mailto:user@example.com')"), + ("file:///path/to/file", "IRI('file:///path/to/file')"), + ("urn:isbn:0451450523", "IRI('urn:isbn:0451450523')"), + ("http://example.com/résource", "IRI('http://example.com/résource')"), ( "http://example.com/resource with spaces", - "IRI(http://example.com/resource with spaces)", + "IRI('http://example.com/resource with spaces')", ), - ("http://example.com/München", "IRI(http://example.com/München)"), - ("http://example.com/日本語", "IRI(http://example.com/日本語)"), - ("", "IRI()"), - ("relative/path", "IRI(relative/path)"), - ("//example.com", "IRI(//example.com)"), - (":", "IRI(:)"), + ("http://example.com/München", "IRI('http://example.com/München')"), + ("http://example.com/日本語", "IRI('http://example.com/日本語')"), + ("", "IRI('')"), + ("relative/path", "IRI('relative/path')"), + ("//example.com", "IRI('//example.com')"), + (":", "IRI(':')"), ( "http://example.com/very/long/path/to/resource/with/many/segments", - "IRI(http://example.com/very/long/path/to/resource/with/many/segments)", + "IRI('http://example.com/very/long/path/to/resource/with/many/segments')", ), ( "http://example.com/path-with-dashes", - "IRI(http://example.com/path-with-dashes)", + "IRI('http://example.com/path-with-dashes')", ), ( "http://example.com/path_with_underscores", - "IRI(http://example.com/path_with_underscores)", + "IRI('http://example.com/path_with_underscores')", + ), + ( + "http://example.com/path.with.dots", + "IRI('http://example.com/path.with.dots')", ), - ("http://example.com/path.with.dots", "IRI(http://example.com/path.with.dots)"), ( "http://example.com/path~with~tildes", - "IRI(http://example.com/path~with~tildes)", + "IRI('http://example.com/path~with~tildes')", ), ( "http://example.com/path%20with%20encoded%20spaces", - "IRI(http://example.com/path%20with%20encoded%20spaces)", + "IRI('http://example.com/path%20with%20encoded%20spaces')", ), - ("http://example.com/path%C3%A9", "IRI(http://example.com/path%C3%A9)"), + ("http://example.com/path%C3%A9", "IRI('http://example.com/path%C3%A9')"), ], ) def test_iri_repr(iri: str, expected_repr: str) -> None: @@ -369,3 +372,26 @@ def test_literal_with_base_direction() -> None: ) assert literal != Literal("مرحبا", "ar") assert hash(literal) != hash(Literal("مرحبا", "ar")) + + +def test_term_accessors() -> None: + assert BlankNode("b1").identifier == "b1" + assert IRI("http://example.com/s").iri == "http://example.com/s" + literal = Literal('say "hi"', "en", direction="ltr") + assert literal.lex == 'say "hi"' + assert literal.langtag == "en" + assert literal.datatype is None + assert literal.direction == "ltr" + typed = Literal("1", datatype="http://www.w3.org/2001/XMLSchema#integer") + assert typed.langtag is None + assert typed.datatype == "http://www.w3.org/2001/XMLSchema#integer" + assert typed.direction is None + + +def test_term_accessors_are_read_only() -> None: + iri = IRI("http://example.com/s") + with pytest.raises(AttributeError): + iri.iri = "http://example.com/other" # type: ignore[misc] + literal = Literal("a") + with pytest.raises(AttributeError): + literal.lex = "b" # type: ignore[misc]