Repository navigation
Add final benchmarks - #398
Closed
Ostrzyciel wants to merge 24 commits into
Closed
Ostrzyciel wants to merge 24 commits into
Ostrzyciel wants to merge 24 commits into
Conversation
sparql.proto imports rdf.proto, so the script points the generated import at pyjelly.jelly. The messages are exported from pyjelly.jelly, like the ones of Jelly-RDF.
Read and write SPARQL query results (SELECT and ASK) in Jelly-SPARQL 1.0, following Jelly-JVM. The columns are those of Jelly-RDF 1.2, so the encoder and decoder reuse them. - pyjelly.serialize.sparql: SparqlStream writes the frames of result sets. Frames hold 4096 values by default and end early when the lookups are full. If getting the rows fails midway, the result set ends with a trailer that has the error. - pyjelly.parse.sparql: SparqlDecoder checks and decodes the frames, and read_result_set(s) gives result sets whose rows are read as they are iterated. A trailer with an error raises JellyIncompleteResultsError after the rows; a stream that ends without a trailer is marked as truncated. - rdflib: Result.parse() and Result.serialize() read and write the "jelly" format (and application/x-jelly-sparql, so rdflib's SPARQLStore can ask endpoints for it). - The generic API reads and writes results with its own terms, including triple terms and literals with a base direction. - PUNCTUATED streams (a sequence of result sets) are supported, with blank nodes scoped to each result set. Shared code changes: TermEncoder.new_frame() budgets the datatypes of several literals, the frame writer no longer needs write() to return a count (rdflib's result streams don't), delimiting detection takes the tags of the frame type, and out-of-range lookup references are reported as JellyConformanceError instead of IndexError.
Run the tests of jelly-protobuf's test/sparql with the rdflib integration and the generic API, like the Jelly-RDF ones. rdflib skips the RDF 1.2 categories, as it has no triple terms or base directions. The parse tests compare with the expected SPARQL JSON results, the serialize tests check the options, the trailer, and that the output reads back as the input. The conformance report script writes separate reports for Jelly-SPARQL.
parse_jelly_sparql(lazy=True), also through Result.parse(), reads the rows as the result is iterated, like rdflib does with the results of its own queries.
A page on SPARQL query results: reading and writing them with rdflib (including from a SPARQL endpoint), lazy reading, incomplete results, options, sequences of results, and the generic API. The examples run in the tests, like the others.
A "sparql" suite of the benchmarks (--suite sparql), as in Jelly-JVM: RiverBench datasets read as the results of SELECT * queries, and the synthetic result sets of Jelly-JVM's generator (ported). Jelly-SPARQL, through rdflib and the generic API, is compared with rdflib's JSON, XML and CSV results. Throughput counts cells (rows times variables). The measurement, processes and sweeps are those of the RDF benchmarks.
Bugs: - Results are read and written as they come over a connection: a buffered input (e.g. from socket.makefile()) is no longer wrapped in a second buffer, which waited for 8 KiB, and each frame is flushed. - Writing an rdflib result that was partly iterated wrote only the rest. - Bindings keyed by plain strings were written as unbound. - An incomplete boolean result read with the generic API looked complete. - Lazy rdflib results had no links until iterated. - Results read from an incomplete or truncated stream were written again as complete. API changes: - parse/serialize_jelly_sparql_results are now ..._sequence, and the generic API can write sequences too. In a sequence, an incomplete rdflib result sets its `error` instead of ending the sequence. - A truncated stream issues JellyTruncatedResultsWarning, and a frame cut short or input that isn't Jelly-SPARQL raises a clear error. - SparqlOptions.frame_size is now frame_values, as it counts values. - Unknown keyword arguments of Result.parse/serialize raise TypeError. - Rows can be iterated once, an iteration stopped early can go on, and reading from a closed input says so. Bytes are accepted as input. - Trailers name the type of the error that stopped the results.
Reading: the decoder makes rdflib's bindings (dicts of the bound variables) straight from a frame's columns, with a row factory compiled with mypyc, instead of making tuples that a Python loop turns into dicts. An eager read takes the rows a frame at a time (SparqlResultSet.collect()). Writing: SparqlStream.binding() takes a binding and looks the variables up in compiled code, instead of a Python loop making a list for each row. Instructions per cell (geometric mean of 5 datasets, CPython 3.15): rdflib writes -24%, rdflib reads -39%. The output is the same.
Two literals with the same lexical form, language tag and datatype are the same term, so rdflib's __eq__ (Python code) is only called when these differ. The variables of a dict binding are looked up with dict's own get. Instructions per cell of rdflib writes, geometric mean of 5 datasets: -4.8%. Jelly-RDF writes use the same comparison. The output is the same.
A few cells are cheaper to append than to slice or repeat as a new list, which the layouts of columns with runs and unbound cells did for every token. A non-delimited stream no longer looks up its options for each row. Instructions per cell of reads, geometric mean of 5 datasets: generic -2.5%, rdflib -1.7% (up to -6% with many runs). The output is the same.
mypyc indexes a value typed as Sequence through the general item lookup, with the index as an int object, which took 12% of a generic write. Rows that are tuples or lists, and lists of variables, are now indexed as such. Instructions per cell of writes, geometric mean of 5 datasets: generic -5.5%, rdflib -5.2%. The output is the same.
In code compiled with mypyc, iterating a list is cheaper than indexing it, which checks the bounds each time. Done for the name ids of IRIs (also in Jelly-RDF) and for rdflib's bindings. Instructions per cell of reads, geometric mean of 5 datasets: generic -1.6%, rdflib -2.8%. The output is the same.
Consecutive values of a column are compared to find runs. IRIs, blank nodes and literals of the generic API are now compared by their strings, without a call to their __eq__, and terms of different classes are never equal. Jelly-RDF writes use the same comparison. Instructions per cell of generic writes, geometric mean of 5 datasets: -3.1%. The output is the same.
The decoder now also takes a row factory, which **kwargs: int doesn't fit.
A frame whose lookups had no room for another row was ended before the next row was encoded. If that row failed, the full frame was never returned, so its rows were lost, and if it was the first frame, the stream had no options. Now the frame ends right after the row that fills it. The frames are the same.
Plain, direct wording instead of roundabout phrases like "Return how many cells write writes".
rdflib reads SPARQL TSV results but can't write them, so the TSV it reads is written by a helper, and writing is skipped. Also rewords the comments of the benchmark code.
Result.parse() and Result.serialize() take format="jellys" too, after the file extension. "jelly" stays the documented name.
…ailer When Fuseki cancels a query (e.g., on a timeout), Jelly-JVM ends the stream with an error trailer, and then Fuseki appends a text message to the response. The reader went on after the trailer, as concatenated streams may follow, and raised a "stream is not valid" error instead. If the trailer already said that the results are incomplete, the reader now stops there and raises JellyIncompleteResultsError, with the rows read before.
Also update the default frame size for Jelly-RDF to 1024, which was found in Jelly-JVM to work best.
Ostrzyciel
force-pushed
the
piotr/hardening
branch
from
October 27, 2026 08:31
88399d2 to
c74cc26
Compare
Member
Author
|
eh, bad target branch |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Also update the default frame size for Jelly-RDF to 1024, which was found in Jelly-JVM to work best.