Skip to content

GH-50936: [C++][Integration] Replace RapidJSON with simdjson - #50937

Merged
pitrou merged 4 commits into
apache:mainfrom
Reranko05:gh-35460-integration2
Aug 27, 2026
Merged

GH-50936: [C++][Integration] Replace RapidJSON with simdjson#50937
pitrou merged 4 commits into
apache:mainfrom
Reranko05:gh-35460-integration2

Conversation

@Reranko05

@Reranko05 Reranko05 commented Aug 21, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Collaborator

Rationale for this change

This PR continues the simdjson migration by replacing the RapidJSON usage in the C++ JSON integration implementation with simdjson's DOM API and Arrow's existing JsonWriter.

Changes

  • Replace RapidJSON DOM parsing with simdjson's DOM API.
  • Update the integration JSON reader and internal parsing helpers to use simdjson::dom types.
  • Replace RapidJSON JSON serialization with Arrow's existing JsonWriter.
  • Handle optional JSON array members without creating an invalid simdjson array.
  • Remove the unused RapidJSON dependencies from the CMake and Meson integration test targets.
  • Update remaining RapidJSON-specific implementation references.

Fixes: #50936

@github-actions github-actions Bot added the awaiting review Awaiting review label Aug 21, 2026
@Reranko05
Reranko05 marked this pull request as ready for review August 21, 2026 08:32
@Reranko05
Reranko05 requested a review from pitrou as a code owner August 21, 2026 08:32
Copilot AI lite review requested due to automatic review settings August 21, 2026 08:32
Copilot AI reviewed Aug 21, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Reranko05
Reranko05 requested review from kou and rok August 21, 2026 08:32
@Reranko05
Reranko05 force-pushed the gh-35460-integration2 branch from e50b949 to 72a256a Compare August 24, 2026 07:50
@Reranko05

Reranko05 commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator Author

@kou, Could you review this when you have time?

kou
kou reviewed Aug 24, 2026
View reviewed changes
Comment thread cpp/src/arrow/integration/json_internal.cc Outdated
@github-actions github-actions Bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Aug 24, 2026
pitrou
pitrou reviewed Aug 24, 2026
View reviewed changes
Comment thread cpp/src/arrow/integration/json_internal.cc
Comment thread cpp/src/arrow/integration/json_internal.cc
ARROW_ASSIGN_OR_RAISE(
auto timezone_string,
internal::ResolveSimdjsonResult(timezone.get_string(), "field was not a string"));
return timestamp(unit, std::string(timezone_string));

@pitrou pitrou Aug 24, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The explicit std::string cast isn't necessary, is it?

@Reranko05 Reranko05 Aug 24, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked this, but timestamp() currently takes a const std::string&, so the conversion is required here.

Comment thread cpp/src/arrow/integration/json_internal.cc Outdated
Comment thread cpp/src/arrow/integration/json_integration.cc Outdated
Comment thread cpp/src/arrow/integration/json_internal.cc
Copilot AI review requested due to automatic review settings August 24, 2026 09:42
Copilot AI reviewed Aug 24, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 24, 2026
Copilot AI review requested due to automatic review settings August 24, 2026 09:57
@Reranko05
Reranko05 force-pushed the gh-35460-integration2 branch from 70f43b3 to e81a967 Compare August 24, 2026 09:57
Copilot AI reviewed Aug 24, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI review requested due to automatic review settings August 24, 2026 10:00
@Reranko05
Reranko05 force-pushed the gh-35460-integration2 branch from e81a967 to 6c8592c Compare August 24, 2026 10:00
Copilot AI reviewed Aug 24, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI review requested due to automatic review settings August 24, 2026 10:58
@Reranko05
Reranko05 force-pushed the gh-35460-integration2 branch from 6c8592c to f4b4750 Compare August 24, 2026 10:58
Copilot AI reviewed Aug 24, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Reranko05
Reranko05 requested review from kou and pitrou August 24, 2026 13:06
pitrou
pitrou reviewed Aug 25, 2026
View reviewed changes
Comment thread cpp/src/arrow/integration/json_internal.cc Outdated
Copilot AI review requested due to automatic review settings August 26, 2026 19:32
Copilot AI reviewed Aug 26, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI review requested due to automatic review settings August 26, 2026 20:24
@Reranko05
Reranko05 force-pushed the gh-35460-integration2 branch from 2009ca3 to 9d0209f Compare August 26, 2026 20:24
Copilot AI reviewed Aug 26, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Reranko05

Reranko05 commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator Author

@pitrou Could you review this when you have time?

@Reranko05
Reranko05 requested a review from pitrou August 27, 2026 08:22
pitrou
pitrou approved these changes Aug 27, 2026
View reviewed changes

@pitrou pitrou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Reranko05 . CI failures look unrelated.

@pitrou
pitrou merged commit a1d23e5 into apache:main Aug 27, 2026
66 of 71 checks passed
@pitrou pitrou removed the awaiting change review Awaiting change review label Aug 27, 2026
@pitrou pitrou mentioned this pull request Aug 27, 2026
Closed
@github-actions github-actions Bot added the awaiting committer review Awaiting committer review label Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@pitrou pitrou pitrou approved these changes
@rok rok Awaiting requested review from rok
@kou kou Awaiting requested review from kou

Assignees

No one assigned

Labels

awaiting committer review Awaiting committer review CI: Extra: C++ Run extra C++ CI Component: C++

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[C++][Integration] Replace RapidJSON with simdjson

4 participants

@Reranko05 @kou @pitrou