diff --git a/README.md b/README.md index a38dc24..b3ec073 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ Parse SQL into a plain Perl data structure and back again # SYNOPSIS ```perl -use SQL::AST::Simple qw( parse unparse ); +use SQL::AST::Simple qw( parse unparse parse_expr unparse_expr ); my $ast = parse('SELECT a, b FROM t WHERE a > 1', dialect => 'postgresql'); @@ -14,15 +14,20 @@ my $ast = parse('SELECT a, b FROM t WHERE a > 1', dialect => 'postgresql'); $ast->[0]{Query}{body}{Select}{from}[0]{relation}{Table}{name}[0]{Identifier}{value} = 'u'; say unparse($ast); # SELECT a, b FROM u WHERE a > 1 + +# Expressions can be handled on their own, without a statement around them: +$ast->[0]{Query}{body}{Select}{selection} = parse_expr('a > 1 AND b = 2'); +say unparse_expr($ast->[0]{Query}{body}{Select}{selection}); # a > 1 AND b = 2 ``` # DESCRIPTION -This module bundles the Rust -[sqlparser](https://github.com/apache/datafusion-sqlparser-rs) crate and -exposes exactly two operations: turning SQL text into the parser's abstract -syntax tree as an ordinary Perl data structure, and turning such a data -structure back into SQL text. There is no object layer; the tree is what +This module provides Perl bindings for the Rust +[sqlparser](https://crates.io/crates/sqlparser) crate. It +exposes two operations: turning SQL text into the parser's abstract syntax +tree as an ordinary Perl data structure, and turning such a data structure +back into SQL text. Each comes in a form for whole statements and a form +for a lone expression. There is no object layer; the tree is what the crate's serde serialization produces, decoded from JSON. That keeps the module small and makes every node the crate knows about available without any wrapping, at the cost of a somewhat verbose structure. @@ -54,6 +59,23 @@ Options: `duckdb`, `databricks`, `hive`, `spark` (or `sparksql`) and `teradata`. +## parse\_expr + +```perl +my $expr = parse_expr($sql); +my $expr = parse_expr($sql, dialect => $name); +``` + +Parses `$sql` as a single expression, such as the condition of a `WHERE` +clause, and returns it as a hash reference. The whole of `$sql` must be +consumed by the expression; a leading `WHERE` keyword or anything left +over after the expression is an error. Takes the same `dialect` option +as ["parse"](#parse). + +The result is exactly what appears inside a statement wherever the crate +expects an expression, so it can be spliced into a tree from ["parse"](#parse), +for instance as the `selection` of a `SELECT`. + ## unparse ```perl @@ -73,6 +95,17 @@ Options: If true, statements are formatted with indentation and newlines rather than on a single line, and are joined with `";\n"`. +## unparse\_expr + +```perl +my $sql = unparse_expr($expr); +``` + +Takes an expression hash reference, as returned by ["parse\_expr"](#parse_expr) or +lifted out of a statement, and returns the SQL text. Throws an exception +if the structure does not deserialize into a valid expression. There is +no `pretty` option; expressions are always rendered on one line. + # THE DATA STRUCTURE The tree mirrors the Rust types of the `sqlparser` crate one to one, as @@ -96,14 +129,14 @@ first. - Most nodes carry a `span` hash recording where they appeared in the source. ["unparse"](#unparse) ignores the contents but requires the field to be present, so the easiest way to build a new node is to parse a small -snippet and lift the piece you need out of the result, rather than -constructing hashes by hand. +snippet (with ["parse\_expr"](#parse_expr) for an expression) and lift the piece you +need out of the result, rather than constructing hashes by hand. The easiest way to learn the shape for a given construct is to parse an -example and dump it. The exact shape depends on the bundled crate -version, which is pinned in the distribution's `ffi/Cargo.toml`; a -release that bumps it may change the structure and will say so in the -change log. +example and dump it. The exact shape depends on the version of the +crate the bindings are built against, which is pinned in the +distribution's `ffi/Cargo.toml`; a release that bumps it may change the +structure and will say so in the change log. # CAVEATS @@ -118,14 +151,10 @@ install time. # SEE ALSO -- [https://github.com/apache/datafusion-sqlparser-rs](https://github.com/apache/datafusion-sqlparser-rs) +- [https://crates.io/crates/sqlparser](https://crates.io/crates/sqlparser) The parser this module wraps. -- [FFI::Platypus::Lang::Rust](https://metacpan.org/pod/FFI::Platypus::Lang::Rust) - - How the Rust code is bundled and called. - # AUTHOR Graham Ollis diff --git a/ffi/src/lib.rs b/ffi/src/lib.rs index 44d323a..27cb22a 100644 --- a/ffi/src/lib.rs +++ b/ffi/src/lib.rs @@ -5,9 +5,10 @@ //! returned to the caller is a NUL terminated, heap allocated C string that //! must be released with `sql_ast_simple_free`. -use sqlparser::ast::Statement; -use sqlparser::dialect::dialect_from_str; +use sqlparser::ast::{Expr, Statement}; +use sqlparser::dialect::{dialect_from_str, Dialect}; use sqlparser::parser::Parser; +use sqlparser::tokenizer::Token; use std::ffi::{c_char, CStr, CString}; use std::ptr; @@ -45,6 +46,12 @@ fn finish(err_out: *mut *mut c_char, result: Result) -> *mut c_c } } +/// Look up a dialect by name. +fn dialect(name: *const c_char) -> Result, String> { + let name = unsafe { borrow(name) }?; + dialect_from_str(name).ok_or_else(|| format!("unknown dialect: {name}")) +} + /// Parse `sql` using the named dialect and return the AST as a JSON array /// of statements. #[no_mangle] @@ -54,16 +61,38 @@ pub extern "C" fn sql_ast_simple_parse( err_out: *mut *mut c_char, ) -> *mut c_char { let result = (|| { - let dialect_name = unsafe { borrow(dialect) }?; + let dialect = self::dialect(dialect)?; let sql = unsafe { borrow(sql) }?; - let dialect = - dialect_from_str(dialect_name).ok_or_else(|| format!("unknown dialect: {dialect_name}"))?; let ast = Parser::parse_sql(&*dialect, sql).map_err(|e| e.to_string())?; serde_json::to_string(&ast).map_err(|e| e.to_string()) })(); finish(err_out, result) } +/// Parse `sql` as a single expression, such as the body of a `WHERE` +/// clause, and return it as JSON. Anything left over after the expression +/// is an error. +#[no_mangle] +pub extern "C" fn sql_ast_simple_parse_expr( + dialect: *const c_char, + sql: *const c_char, + err_out: *mut *mut c_char, +) -> *mut c_char { + let result = (|| { + let dialect = self::dialect(dialect)?; + let sql = unsafe { borrow(sql) }?; + let mut parser = Parser::new(&*dialect) + .try_with_sql(sql) + .map_err(|e| e.to_string())?; + let expr = parser.parse_expr().map_err(|e| e.to_string())?; + parser + .expect_token(&Token::EOF) + .map_err(|e| e.to_string())?; + serde_json::to_string(&expr).map_err(|e| e.to_string()) + })(); + finish(err_out, result) +} + /// Turn a JSON array of statements (as produced by `sql_ast_simple_parse`, /// possibly modified) back into SQL text. Statements are joined with /// `"; "`, or `";\n"` when `pretty` is set. @@ -85,6 +114,21 @@ pub extern "C" fn sql_ast_simple_unparse( finish(err_out, result) } +/// Turn a JSON expression (as produced by `sql_ast_simple_parse_expr`, +/// possibly modified) back into SQL text. +#[no_mangle] +pub extern "C" fn sql_ast_simple_unparse_expr( + json: *const c_char, + err_out: *mut *mut c_char, +) -> *mut c_char { + let result = (|| { + let json = unsafe { borrow(json) }?; + let expr: Expr = serde_json::from_str(json).map_err(|e| e.to_string())?; + Ok(expr.to_string()) + })(); + finish(err_out, result) +} + /// Release a string returned by any function in this library. #[no_mangle] pub extern "C" fn sql_ast_simple_free(p: *mut c_char) { diff --git a/lib/SQL/AST/Simple.pm b/lib/SQL/AST/Simple.pm index ccd1253..3c1b600 100644 --- a/lib/SQL/AST/Simple.pm +++ b/lib/SQL/AST/Simple.pm @@ -4,7 +4,8 @@ use 5.042; use warnings; use FFI::Platypus 2.00; use JSON::MaybeXS (); -use Carp (); +use Carp qw( croak ); +use Ref::Util qw( is_plain_arrayref is_plain_hashref ); use Exporter qw( import ); # ABSTRACT: Parse SQL into a plain Perl data structure and back again @@ -12,7 +13,7 @@ use Exporter qw( import ); =head1 SYNOPSIS - use SQL::AST::Simple qw( parse unparse ); + use SQL::AST::Simple qw( parse unparse parse_expr unparse_expr ); my $ast = parse('SELECT a, b FROM t WHERE a > 1', dialect => 'postgresql'); @@ -22,13 +23,18 @@ use Exporter qw( import ); say unparse($ast); # SELECT a, b FROM u WHERE a > 1 + # Expressions can be handled on their own, without a statement around them: + $ast->[0]{Query}{body}{Select}{selection} = parse_expr('a > 1 AND b = 2'); + say unparse_expr($ast->[0]{Query}{body}{Select}{selection}); # a > 1 AND b = 2 + =head1 DESCRIPTION -This module bundles the Rust -L crate and -exposes exactly two operations: turning SQL text into the parser's abstract -syntax tree as an ordinary Perl data structure, and turning such a data -structure back into SQL text. There is no object layer; the tree is what +This module provides Perl bindings for the Rust +L crate. It +exposes two operations: turning SQL text into the parser's abstract syntax +tree as an ordinary Perl data structure, and turning such a data structure +back into SQL text. Each comes in a form for whole statements and a form +for a lone expression. There is no object layer; the tree is what the crate's serde serialization produces, decoded from JSON. That keeps the module small and makes every node the crate knows about available without any wrapping, at the cost of a somewhat verbose structure. @@ -62,6 +68,21 @@ C. =back +=head2 parse_expr + + my $expr = parse_expr($sql); + my $expr = parse_expr($sql, dialect => $name); + +Parses C<$sql> as a single expression, such as the condition of a C +clause, and returns it as a hash reference. The whole of C<$sql> must be +consumed by the expression; a leading C keyword or anything left +over after the expression is an error. Takes the same C option +as L. + +The result is exactly what appears inside a statement wherever the crate +expects an expression, so it can be spliced into a tree from L, +for instance as the C of a C