Skip to content

[FLINK-40489][table] Add UUID_V4() and UUID_V7() functions - #29154

Open
manner wants to merge 2 commits into
apache:masterfrom
manner:FLINK-40489
Open

[FLINK-40489][table] Add UUID_V4() and UUID_V7() functions#29154
manner wants to merge 2 commits into
apache:masterfrom
manner:FLINK-40489

Conversation

@manner

@manner manner commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

This pull request adds two new built-in SQL scalar functions, UUID_V4() and UUID_V7(), generating RFC 9562 UUID values — a random version 4 UUID and a time-ordered version 7 UUID.

Brief change log

  • Added BuiltInFunctionDefinitions.UUID_V4 / UUID_V7
  • Added UuidGenerationUtils with generateV4()/generateV7(), producing the 16-byte internal representation of a UUID value directly (no round-trip through java.util.UUID)
  • Added UuidV4Function / UuidV7Function runtime classes
  • Added uuidV4() / uuidV7() to the Java and Scala Table API expression DSL
  • Documented both functions in sql_functions.yml (English and Chinese)

Verifying this change

This change added tests and can be verified as follows:

  • Added UuidGenerationUtilsTest, directly asserting on the generated bytes: correct length, version nibble, variant bits, and (for v7) that the embedded timestamp matches System.currentTimeMillis() at call time
  • Added UuidFunctionsITCase, covering the SQL and Table API return type (UUID) and the version nibble in the canonical string form, executed end-to-end against a MiniCluster

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): yesExpressions.java (@PublicEvolving) gains two new additive methods, uuidV4() and uuidV7()
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): yes — only exercised when a query explicitly calls UUID_V4()/UUID_V7()
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? docs (sql_functions.yml) and JavaDocs

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Sonnet 5)

@flinkbot

flinkbot commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Comment on lines +838 to +850
/** Returns a random RFC 9562 version 4 (pseudo randomly generated) {@code UUID} value. */
public static ApiExpression uuidV4() {
return apiCall(BuiltInFunctionDefinitions.UUID_V4);
}

/**
* Returns a time-ordered RFC 9562 version 7 {@code UUID} value, generated from the current
* timestamp and a random component.
*/
public static ApiExpression uuidV7() {
return apiCall(BuiltInFunctionDefinitions.UUID_V7);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider adding the functions into the expression.py too

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that work even though there's no UUID type specified in pyflink/table/types.py?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are tests in flink-python failing ci without this btw ;)

Comment on lines +41 to +55
public static byte[] generateV4() {
final byte[] bytes = new byte[UuidType.BYTE_LENGTH];
Holder.SECURE_RANDOM.nextBytes(bytes);

// set the version to 4
bytes[6] &= 0x0F;
bytes[6] |= 0x40;

// set the variant to IETF
bytes[8] &= 0x3F;
bytes[8] |= (byte) 0x80;

return bytes;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The randomUUID method of java.util.UUID does the same:

    /**
     * Static factory to retrieve a type 4 (pseudo randomly generated) UUID.
     *
     * The {@code UUID} is generated using a cryptographically strong pseudo
     * random number generator.
     *
     * @return  A randomly generated {@code UUID}
     */
    public static UUID randomUUID() {
        SecureRandom ng = Holder.numberGenerator;

        byte[] randomBytes = new byte[16];
        ng.nextBytes(randomBytes);
        randomBytes[6]  &= 0x0f;  /* clear version        */
        randomBytes[6]  |= 0x40;  /* set to version 4     */
        randomBytes[8]  &= 0x3f;  /* clear variant        */
        randomBytes[8]  |= (byte) 0x80;  /* set to IETF variant  */
        return new UUID(randomBytes);
    }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's right. But randomUUID() builds a UUID from the random bytes, and we'd then have to extract the bytes right back out of it. Generating them directly avoids that copying and the object instantiation.
However, I'm not sure if it's worth it in this case, happy to switch to randomUUID() if you'd rather keep it simple.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it's better to reuse as jdk might apply optimization with version bump which we will get for free in this case

Comment on lines +89 to +95
/**
* Holder for the {@link SecureRandom} instance, so that seeding it is deferred until a {@code
* UUID} is actually generated for the first time.
*/
private static final class Holder {
static final SecureRandom SECURE_RANDOM = new SecureRandom();
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why a class and not just a field?

Suggested change
/**
* Holder for the {@link SecureRandom} instance, so that seeding it is deferred until a {@code
* UUID} is actually generated for the first time.
*/
private static final class Holder {
static final SecureRandom SECURE_RANDOM = new SecureRandom();
}
/**
* Holder for the {@link SecureRandom} instance, so that seeding it is deferred until a {@code
* UUID} is actually generated for the first time.
*/
private static final SecureRandom SECURE_RANDOM = new SecureRandom();

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was to keep the SecureRandom lazily instantiated, so it's only created when actually used (similar to java.util.UUID). But since both/all methods in the class use the RNG, the class is never loaded without needing it, so the holder doesn't help at all.

@github-actions github-actions Bot added the community-reviewed PR has been reviewed by the community. label Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-reviewed PR has been reviewed by the community.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants