Tidy up docs, examples, code before release

README.md

@@ -2,23 +2,24 @@
 
 # pyjelly
 
-**pyjelly** is a Python implementation of [Jelly](http://w3id.org/jelly), a high-performance binary serialization format and streaming protocol for RDF knowledge graphs.
+**pyjelly** is a Python implementation of [Jelly](https://w3id.org/jelly), a high-performance binary serialization format and streaming protocol for RDF knowledge graphs.
 
 **Documentation, usage guide and more: https://w3id.org/jelly/pyjelly**
 
 ## Features
 
-- **Fast reading and writing** of RDF knowledge graphs in the [Jelly format](http://w3id.org/jelly).
-- Reading and writing **[SPARQL query results](https://w3id.org/jelly/pyjelly/dev/sparql)** in [Jelly-SPARQL](https://w3id.org/jelly/dev/specification/sparql), with rdflib or the generic API.
-- **Standalone [generic API](https://w3id.org/jelly/pyjelly/dev/generic-sink)** with no third-party dependencies, allowing for:
+- **[Fast](https://w3id.org/jelly/dev/performance/rdf/#pyjelly) reading and writing** of RDF knowledge graphs in the [Jelly format](https://w3id.org/jelly). pyjelly writes [Jelly-RDF 1.2](https://w3id.org/jelly/dev/specification/serialization/) by default, and reads Jelly-RDF 1.0–1.2.
+- Reading and writing **[SPARQL query results](https://w3id.org/jelly/pyjelly/dev/sparql)** in [Jelly-SPARQL](https://w3id.org/jelly/dev/specification/sparql), with RDFLib or the generic API.
+- **Standalone [generic API](https://w3id.org/jelly/pyjelly/dev/generic-sink)** that doesn't need RDFLib or any other RDF library, allowing for:
     - Serialization and parsing of statements to and from Jelly files.
     - Parsing and serializing streams of graphs and statements.
-- Precise control over **serialization options, framing and compression**.
+    - [RDF 1.2](https://www.w3.org/TR/rdf12-concepts/) triple terms and literals with a base direction.
+- Precise control over **serialization options and framing**.
 - **Seamless** integration with: 
-    - **[rdflib](https://w3id.org/jelly/pyjelly/dev/getting-started)**
+    - **[RDFLib](https://w3id.org/jelly/pyjelly/dev/getting-started)**
     - **[RDFLib-Neo4j](https://w3id.org/jelly/pyjelly/dev/rdflib-neo4j-integration)**
     - **[NetworkX](https://w3id.org/jelly/pyjelly/dev/networkx-integration)**
-- **Stream processing support** for large datasets or streams of all [physical stream types](https://w3id.org/jelly/dev/specification/reference/#physicalstreamtype).
+- **Stream processing support** for large datasets and streams of many RDF graphs or datasets.
 
 **pyjelly** is useful when dealing with (see [full description](https://w3id.org/jelly/pyjelly/dev/overview/#use-cases)):
 
@@ -44,7 +45,7 @@ g.parse("http://xmlns.com/foaf/spec/index.rdf")
 g.serialize(destination="foaf.jelly", format="jelly")
 ```
 
-To read a Jelly file and convert it to an rdflib `Graph`:
+To read a Jelly file and convert it to an RDFLib `Graph`:
 
 ```python
 from rdflib import Graph

docs/generic-sink.md

@@ -10,10 +10,7 @@ Install pyjelly from PyPI:
 pip install pyjelly
 ```
 
-### Requirements
-
-- Python 3.10 or newer  
-- Linux, macOS, or Windows
+The requirements are the same as [with RDFLib](getting-started.md#requirements).
 
 ## Usage without external libraries
 
@@ -27,75 +24,72 @@ To make a set of triples/quads and write them to a Jelly file, use:
 
 This example uses pyjelly’s simple custom triple/quad type, which is easy to create and work with.
 
+The terms in the statements are `IRI`, `BlankNode` and `Literal`. You can read their parts with `IRI.iri`, `BlankNode.identifier`, and `Literal.lex`, `Literal.langtag`, `Literal.datatype` (the datatype IRI as a string) and `Literal.direction`. `str()` gives a term in N-Triples syntax, e.g., `<http://example.com/subject>` or `"Hello"@en`.
+
 ## Parsing statements from a Jelly file
 
-To load triples/quads into your python object from a `.jelly` file, see:
+To load triples/quads from a `.jelly` file into a `GenericStatementSink`, see:
 
 {{ code_example('generic/02_parse.py') }}
 
-Which retrieves data from your `.jelly` file.
+`GenericStatementSink` keeps all statements in memory. To read a large file statement by statement instead, use `parse_jelly_flat` in a generator:
+
+{{ code_example('generic/03_streaming.py') }}
+
+Only the statement that is being processed is kept in memory.
 
 ### Parsing a stream of graphs
 
 Similarly, to process a Jelly stream as a stream of graphs through generic API, see:
 
 {{ code_example('generic/06_parse_grouped.py') }}
 
-Where we use a [dataset of weather measurements](https://w3id.org/riverbench/datasets/lod-katrina/dev) and count the number of triples in each graph.
+We use a [dataset of weather measurements](https://w3id.org/riverbench/datasets/lod-katrina/dev) and count the number of triples in each graph.
 
 ### Parsing a stream of statements
 
-You can also process a Jelly stream as a flat stream with only generic API:
-
-We look through a fragment of Denmark's OpenStreetMap to find all city names:
+You can also process a Jelly stream as a flat stream of statements. We look through a fragment of Denmark's OpenStreetMap to find all city names:
 
 {{ code_example('generic/07_parse_flat.py') }}
 
-We get a generator of stream events, which allows us to process the file statement-by-statement, however with no external libraries used.
+`parse_jelly_flat` returns a generator of statements, so the file is processed statement by statement.
 
-## Streaming data
-
-If you need to process a certain quantity of statements both efficiently and iteratively, you can provide a simple generator:
-
-{{ code_example('generic/03_streaming.py') }}
-
-With this method you avoid storing all statements in memory, which greatly improves performance.
+## Serializing streams
 
 ### Serializing a stream of graphs
 
 If you have a generator object containing graphs, you can use a generic approach for serialization:
 
 {{ code_example('generic/04_serialize_grouped.py')}}
 
-Grouped data is streamed in its original form, no need for additional RDF libraries like RDFLib. 
+Each sink is written as one [RDF Message]({{ proto_link("specification/serialization/#stream-types") }}), so the grouping is kept when the stream is read with `parse_jelly_grouped`.
 
 ### Serializing a stream of statements
 
-Serializing a generator object of statements to `.jelly` file through generic API:
+If you have a generator object containing statements, you can serialize it like this:
 
 {{ code_example('generic/05_serialize_flat.py')}}
 
-Data is transmitted and kept ordered and simple.
+The statements are written in order, as one flat stream.
 
-### Working with byte buffers and Kafka
+## Working with byte buffers and Kafka
 
 When working with Kafka or other message brokers, you may want to write Jelly data to a byte buffer instead of a file. You can do this by using the `BytesIO` class from the `io` module:
 
 {{ code_example('generic/08_byte_buffer.py')}}
 
-The `data` variable is of type `bytes`, and can be passed to Kafka with [`KafkaProducer.send(value=data)`](https://kafka-python.readthedocs.io/en/master/), or any other API that accepts byte buffers. Same trick may be used when working with the [RDFLib integration](getting-started.md).
+The `data` variable is of type `bytes`, and can be passed to Kafka with [`KafkaProducer.send(value=data)`](https://kafka-python.readthedocs.io/en/master/), or any other API that accepts byte buffers. With the [RDFLib integration](getting-started.md), write to the buffer with `graph.serialize(destination=buffer, format="jelly")`, and read the bytes back with `graph.parse(data=data, format="jelly")`.
 
 When working with Kafka, you should be aware of the broker's offset management and partitioning strategies. Data within one Jelly stream must be strictly ordered and no frames may be dropped. If you have less strict ordering guarantees, you should split up the stream into multiple Jelly streams, each with guaranteed consistency.
 
-### Accessing stream frame metadata 
+## Accessing stream frame metadata
 
-It is possible to access [`RdfStreamFrame` metadata](https://w3id.org/jelly/dev/specification/serialization/#stream-frame-metadata) in pyjelly via **context variables**.
+It is possible to access [`RdfStreamFrame` metadata]({{ proto_link("specification/serialization/#stream-frame-metadata") }}) in pyjelly via **context variables**.
 To do that, define a **context variable** before parsing and pass it as argument to parser functions, e.g.:
 
 {{ code_example('generic/09_metadata.py')}}
 
-
-### See also
+## See also
 
 If you are familiar with RDFLib, you can use pyjelly together with RDFLib in a similar way. [See the dedicated guide](getting-started.md).
 

docs/getting-started.md

@@ -42,6 +42,8 @@ Prefixes bound in RDFLib's namespace manager will then be written into the Jelly
     g.namespace_manager.bind("ex", EX, replace=True)
     ```
 
+    RDFLib binds about 30 common prefixes to every new graph, and they are all written as well. To write only the prefixes you bind yourself, create the graph with `Graph(bind_namespaces="none")`.
+
 
 ### Parsing a graph
 
@@ -53,7 +55,7 @@ RDFLib will reconstruct the graph from the Jelly file.
 
 ### Parsing a stream of graphs
 
-You can process a Jelly stream as a stream of graphs. A Jelly file consists of "frames" (batches of statements) – we can load each frame as a separate RDFLib graph.
+You can process a Jelly stream as a stream of graphs. A Jelly stream can hold a sequence of graphs, each in its own [RDF Message]({{ proto_link("specification/serialization/#stream-types") }}) – we can load each one as a separate RDFLib graph.
 
 In this example, we use a [dataset of weather measurements](https://w3id.org/riverbench/datasets/lod-katrina/dev). We count the number of triples in each graph:
 
@@ -82,7 +84,6 @@ If you have a generator object containing graphs, you can easily serialize it in
 {{ code_example('rdflib/06_serialize_grouped.py')}}
 
 This method allows for transmitting logically grouped data, preserving their original division: each graph is one [RDF Message]({{ proto_link("specification/serialization/#stream-types") }}).
-For more precise control over frame serialization you can use [lower-level API](api.md)
 
 ### Serializing a stream of statements
 
@@ -91,17 +92,20 @@ If you have a generator object containing statements, you can easily serialize i
 {{ code_example('rdflib/07_serialize_flat.py')}}
 
 The flat method transmits the data as a continuous sequence of statements, keeping it simple and ordered.
-For more precise control over frame serialization you can use [lower-level API](api.md)
+
+For more control over framing and other options, see [`pyjelly.integrations.rdflib.serialize`](api.md#pyjelly.integrations.rdflib.serialize) in the API reference.
 
 ### File extension support
 
-You can generally omit the `format="jelly"` parameter if the file ends in `.jelly` – RDFLib will auto-detect the format:
+When parsing, you can generally omit the `format="jelly"` parameter if the file ends in `.jelly` – RDFLib will auto-detect the format:
 
 {{ code_example('rdflib/03_parse_autodetect.py') }}
 
 !!! warning 
 
-    Unfortunately, the way this is implemented in RDFLib is a bit wonky, so it will only work if you explicitly import `pyjelly.integrations.rdflib`, or you used `format="jelly"` in the `serialize()` or `parse()` call before.
+    Unfortunately, the way this is implemented in RDFLib is a bit wonky, so it will only work if you explicitly import `pyjelly.integrations.rdflib`, or you used `format="jelly"` in a `serialize()` or `parse()` call before.
+
+    This only works for parsing. `serialize()` does not look at the file extension, so `g.serialize("out.jelly")` writes Turtle. Always pass `format="jelly"` when serializing.
 
 ## See also
 

docs/index.md

@@ -6,7 +6,7 @@
 
 ## Features
 
-- **Fast reading and writing** of RDF knowledge graphs in the [Jelly format](http://w3id.org/jelly)
+- **Fast reading and writing** of RDF knowledge graphs in the [Jelly format]({{ proto_link("") }})
 - **[SPARQL query results](sparql.md)** in [Jelly-SPARQL]({{ proto_link("specification/sparql/") }})
 - **Seamless integration with [RDFLib](https://rdflib.readthedocs.io/)**
 - **Stream processing support** for large datasets or streams of many RDF graphs/datasets
@@ -15,21 +15,23 @@
 
 - **[Getting started](getting-started.md)**
 - **[Overview and supported features](overview.md)**
-- **[Generic interface (usage without RDFLib)](generic-sink.md)**
+- **[Generic API (usage without RDFLib)](generic-sink.md)**
 - **[SPARQL query results](sparql.md)**
+- **[Neo4j](rdflib-neo4j-integration.md)** and **[NetworkX](networkx-integration.md)** integrations
 - **[API reference](api.md)**
 - **[Contributing](contributing/index.md)** – how to report issues, contribute code, and request features
 
 ## Compatibility
 
-pyjelly is compatible with Python 3.10 and newer. Python 3.10–3.14 are tested in CI and are guaranteed to work.
+pyjelly is compatible with Python 3.10 and newer. Python 3.10–3.15 are tested in CI and are guaranteed to work.
 
 The following table shows the compatibility of the pyjelly implementation with other libraries:
 
-| pyjelly                                                                                                                                             | Python     | RDFLib         |
-|-----------------------------------------------------------------------------------------------------------------------------------------------------|------------|----------------|
-| [0.7.0](https://w3id.org/jelly/pyjelly/0.7.0) – **[{{ python_package_version() }}](https://w3id.org/jelly/pyjelly/{{ python_package_version() }})** | 3.10 – 3.14| 7.x (>= 7.1.4) |
-| [0.1.0](https://w3id.org/jelly/pyjelly/0.1.0) – [0.6.2](https://w3id.org/jelly/pyjelly/0.6.2) | 3.9 – 3.13 | 7.x (>= 7.1.4) |
+| pyjelly                                                                                                                                             | Python     | RDFLib         | Jelly-RDF |
+|-----------------------------------------------------------------------------------------------------------------------------------------------------|------------|----------------|-----------|
+| [0.9.0](https://w3id.org/jelly/pyjelly/0.9.0) – **[{{ python_package_version() }}](https://w3id.org/jelly/pyjelly/{{ python_package_version() }})** | 3.10 – 3.15| 7.x (>= 7.1.4) | 1.2 |
+| [0.7.0](https://w3id.org/jelly/pyjelly/0.7.0) – [0.8.2](https://w3id.org/jelly/pyjelly/0.8.2) | 3.10 – 3.14| 7.x (>= 7.1.4) | 1.1 |
+| [0.1.0](https://w3id.org/jelly/pyjelly/0.1.0) – [0.6.2](https://w3id.org/jelly/pyjelly/0.6.2) | 3.9 – 3.13 | 7.x (>= 7.1.4) | 1.1 |
 
 
 See the **[release notes on GitHub](https://github.com/Jelly-RDF/pyjelly/releases)** for more details.

docs/networkx-integration.md

@@ -5,11 +5,9 @@ NetworkX is a Python package that represents complex networks as graphs and allo
 Install the following libraries:
 
 ```bash
-pip install pyjelly[rdflib] networkx==3.2.1 matplotlib==3.9.4
+pip install pyjelly[rdflib] networkx matplotlib
 ```
 
-Below there are few useful examples to follow.
-
 ## Parse graph, show it
 
 Let's investigate relationships between worldwide political figures (support and opposition relations)!
@@ -23,7 +21,7 @@ We can easily load it:
 Output from `print()`:
 
 ```text
-Loaded graph with 90000 instances.
+Loaded graph with 90000 triples.
 ```
 
 Convert it into a convenient NetworkX graph:
@@ -48,26 +46,26 @@ Output from `print()`:
 
 ```text
 Top 5 nodes sorted by degree:
-Socrates: 241
+Sócrates: 241
 Cavaco: 189
 Passos Coelho: 187
 Costa: 179
-Antonio Costa: 168
+António Costa: 168
 ```
 
 What is the shortest path between two nodes? We can check:
 
-{{ code_example('networkx_integration/01_parse_calculate_visualize.py', 75, 79) }}
+{{ code_example('networkx_integration/01_parse_calculate_visualize.py', 68, 81) }}
 
 Output from `print()`:
 
 ```text
-Shortest path from Socrates to Obama: Socrates -> Marcelo Rebelo de Sousa -> Durao Barroso -> Obama
+Shortest path from Socrates to Obama: Socrates -> Marcelo Rebelo de Sousa -> António Costa -> Obama
 ```
 
 However, it's best to see the full picture (for our example we truncate to 10 nodes for clarity):
 
-{{ code_example('networkx_integration/01_parse_calculate_visualize.py', 81, 92) }}
+{{ code_example('networkx_integration/01_parse_calculate_visualize.py', 83, 94) }}
 
 The graph presents as follows
 
@@ -78,15 +76,15 @@ The graph presents as follows
 
 In summary:
 
-{{ snippet_admonition('examples/networkx_integration/01_parse_calculate_visualize.py', 0, 95, title="Entire example", expanded=False) }}
+{{ snippet_admonition('examples/networkx_integration/01_parse_calculate_visualize.py', 0, 96, title="Entire example", expanded=False) }}
 
 We converted an RDFLib graph to NetworkX, calculated insightful metrics and visualized the graph.  
 
 For more info about the data source please see the [Politiquices dataset in RiverBench](https://riverbench.github.io/v/2.1.0/datasets/politiquices/) and its [original source (Portuguese)](http://www.politiquices.pt/).
 
 ## Serialize NetworkX graph
 
-This example shows how to write a NetworkX graph to a Jelly file.:
+This example shows how to write a NetworkX graph to a Jelly file:
 
 {{ code_example('networkx_integration/02_serialize.py') }}