Skip to content

DAOS-17322 ddb: DTX aggregation with ddb (Part 1) - #16745

Merged
gnailzenh merged 24 commits into
masterfrom
ckochhof/dev/master/daos-17322-part1
Sep 26, 2025
Merged

gnailzenh merged 24 commits into
masterfrom
ckochhof/dev/master/daos-17322-part1

Conversation

@knard38

@knard38 knard38 commented Aug 18, 2025 •

Copy link
Copy Markdown
Contributor

Description

First PR to add DTX entries aggregation sub commands to DDB.

This preliminary PR introduce the following modifications:

  • Cleanup of the SCons file src/utils/ddb/tests/SConscript
  • Fix miscellaneous issues detected by clangd linter.
  • Add a new DDB sub command dtx_stat allowing to print statistics on DTX entries.

This PR will be followed by the PR #16746 introducing a new DDB sub command dtx_aggr allowing to aggregate DTX entries with DDB.

Steps for the author:

  • Commit message follows the guidelines.
  • Appropriate Features or Test-tag pragmas were used.
  • Appropriate Functional Test Stages were run.
  • At least two positive code reviews including at least one code owner from each category referenced in the PR.
  • Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

  • Gatekeeper requested (daos-gatekeeper added as a reviewer).

Cleanup of SCons file.

Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
Miscellaneous fixes detected by clangd.

Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
Add the following new command to ddb:
- dtx_stat: Print statistics on DTX entries

Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
@github-actions

github-actions Bot commented Aug 18, 2025 •

Copy link
Copy Markdown

Ticket title is 'DTX aggregation with ddb'
Status is 'In Review'
Labels: '2.8pp'
https://daosio.atlassian.net/browse/DAOS-17322

@knard38 knard38 self-assigned this Aug 19, 2025
@knard38 knard38 added the CR Catastrophic Recovery Feature label Aug 19, 2025
@knard38
knard38 requested review from Nasf-Fan and janekmi August 19, 2025 05:49
@knard38
knard38 marked this pull request as ready for review August 19, 2025 05:50
@knard38
knard38 requested review from a team as code owners August 19, 2025 05:50
Name: "dtx_stat",
Aliases: nil,
Help: "Stat on DTX entries",
LongHelp: "Print statistic on the DTX entries",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we print the DTX statistics information like that:
if container UUID is specified in the path, then for the container; otherwise, if only pool UUID is available, then for the DTX summary of the pool or every container's DTX statistics information in the pool.

@knard38 knard38 Sep 5, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Print stat for all containers.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Print stat for all containers.

Fixed with commit 2b72e15

Comment thread src/utils/ddb/ddb_commands.c Outdated
if (strftime(buf, buf_size, "%F %T", &date) == 0)
return -DER_INVAL;

buf_len = strnlen(buf, buf_size);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buflen = strftime();
if (buflen == 0)
    return -DER_INVAL;

@knard38 knard38 Sep 5, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Remove useless strlen

@knard38 knard38 Sep 10, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Remove useless strlen

Fixed with commit 983d9d4

Comment thread src/utils/ddb/ddb_commands.c Outdated
if (!SUCCESS(rc))
goto done;

if (!itp_has_cont(&itp)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is better to support that: if only specify pool UUID, then iterate the containers in the pool and print each container's DTX statistics information. One benefit for such ability is that we can verify whether the count of the total DTX entries for pool can match the summary of all containers' or not.

@knard38 knard38 Sep 5, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Print stat for all containers.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with commit 2b72e15

Comment thread src/utils/ddb/ddb_commands.c Outdated
if (!SUCCESS(rc))
goto done;

rc = vos_dtx_cmt_reindex(coh);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we just want to obtain the committed DTX entries statistics information, it is unnecessary to reindex them. vos_dtx_cmt_reindex() is time consuming operation. Instead, we can iterate each vos_dtx_blob_df and summary the vos_dtx_blob_df::dbd_count. That is more efficient.

@knard38 knard38 Sep 5, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Replace vos_dtx_cmt_reindex() with iterate on vos_dtx_blob_df

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Replace vos_dtx_cmt_reindex() with iterate on vos_dtx_blob_df

Fixed with commit 3ff2b3a

Comment thread src/utils/ddb/ddb_parse.c Outdated
int
ddb_date2epoch(const char *date, uint64_t *epoch)
{
struct tm date_tm;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

struct tm date_tm = {0}; then no need memset();

@knard38 knard38 Sep 5, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Use static initialisation instead of explicit initialisation.

@knard38 knard38 Sep 10, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Use static initialisation instead of explicit initialisation.

Fixed with commit 983d9d4

@Nasf-Fan
Nasf-Fan self-requested a review August 23, 2025 02:01
D_ALLOC_ARRAY(_buf, _buf_size); \
assert_non_null(_buf); \
regerror(_rc, &_preg, _buf, _buf_size); \
if (_buf) \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have assert assert_non_null(_buf);

@knard38 knard38 Sep 5, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Remove useless test

@knard38 knard38 Sep 10, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with commit 983d9d4

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Nasf-Fan , at the end this test is needed to manage gcc false positive warning.

[2025-09-11T18:30:29.037Z] In file included from src/utils/ddb/tests/ddb_commands_tests.c:21:
[2025-09-11T18:30:29.037Z] src/utils/ddb/tests/ddb_commands_tests.c: In function ‘dtx_stat_tests’:
[2025-09-11T18:30:29.037Z] src/utils/ddb/tests/ddb_cmocka.h:89:25: error: ‘%s’ directive argument is null [-Werror=format-overflow=]
[2025-09-11T18:30:29.038Z]    89 |                         print_error("ERROR: invalid regex '%s': %s\n", regex, _buf);               \
[2025-09-11T18:30:29.038Z]       |                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2025-09-11T18:30:29.038Z] src/utils/ddb/tests/ddb_commands_tests.c:422:9: note: in expansion of macro ‘assert_regex_match’
[2025-09-11T18:30:29.038Z]   422 |         assert_regex_match(
[2025-09-11T18:30:29.038Z]       |         ^~~~~~~~~~~~~~~~~~
[2025-09-11T18:30:29.038Z] src/utils/ddb/tests/ddb_cmocka.h:89:65: note: format string is defined here
[2025-09-11T18:30:29.038Z]    89 |                         print_error("ERROR: invalid regex '%s': %s\n", regex, _buf);               \

More details could be found in the log of the PR build #5
This test was reintroduced with the commit 7c96b6a

Comment thread src/utils/ddb/tests/ddb_cmocka.h Outdated
if (_buf) \
print_error("ERROR: invalid regex '%s': %s\n", regex, _buf); \
D_FREE(_buf); \
regfree(&_preg); \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If regcomp() return failure, then _preg maybe uninitialized?

@knard38 knard38 Sep 5, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • fix invalid regfree()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • fix invalid regfree()

Fixed with commit 77e3c21

@Nasf-Fan
Nasf-Fan self-requested a review August 23, 2025 02:21
kjacque
kjacque previously approved these changes Sep 9, 2025

@kjacque kjacque left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only looked at the Go side in detail. LGTM.

Integrate reviewers comments:
- Remove useless test
- Remove useless strlen
- Use static initialisation instead of explicit initialisation.

Features: aggregation
Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
@daosbuild3

Copy link
Copy Markdown
Collaborator

@daosbuild3

Copy link
Copy Markdown
Collaborator

@daosbuild3

Copy link
Copy Markdown
Collaborator

@daosbuild3

Copy link
Copy Markdown
Collaborator

Integrate reviewers comments:
- fix invalid regfree()

Features: aggregation
Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
@daosbuild3

Copy link
Copy Markdown
Collaborator

@daosbuild3

Copy link
Copy Markdown
Collaborator

@daosbuild3

Copy link
Copy Markdown
Collaborator

@daosbuild3

Copy link
Copy Markdown
Collaborator

@daosbuild3

Copy link
Copy Markdown
Collaborator

Test stage Unit Test with memcheck on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-16745/9/testReport/

@daosbuild3

Copy link
Copy Markdown
Collaborator

Test stage Unit Test bdev with memcheck on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-16745/9/testReport/

@daosbuild3

Copy link
Copy Markdown
Collaborator

@daosbuild3

Copy link
Copy Markdown
Collaborator

@daosbuild3

Copy link
Copy Markdown
Collaborator

Test stage Unit Test with memcheck on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-16745/10/testReport/

@daosbuild3

Copy link
Copy Markdown
Collaborator

Test stage Unit Test bdev with memcheck on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-16745/10/testReport/

@knard38

knard38 commented Sep 18, 2025

Copy link
Copy Markdown
Contributor Author

@Nasf-Fan , please could you have a look at the C code and if the latest changes and comments are OK for you.
@kjacque , please could you have a look at the GO code. Since your last review, I just made the container parameter to be optional.

Comment thread src/vos/vos_dtx.c Outdated
dbd = umem_off2ptr(umm, cont->vc_cont_df->cd_dtx_committed_head);
while (dbd != NULL) {
D_ASSERTF(dbd->dbd_magic == DTX_CMT_BLOB_MAGIC,
"Corrupted committed DTX blob (2) %x\n", dbd->dbd_magic);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Different from regular vos logic, ddb is usually to handle system corruption, then it is better not assertion the on-disk data, instead, we can report some error.

@knard38 knard38 Sep 18, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Change assert into error code.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Change assert into error code.

Fixed with commit 49be495

Comment thread src/utils/ddb/ddb_commands.c Outdated
ctx = args->ctx;

itp_set_cont(&itp, entry->ie_couuid, args->cont_seen);
++args->cont_seen;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better ++(args->cont_seen); to avoid confusion.

@knard38 knard38 Sep 18, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • fix C coding standard

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • fix C coding standard

Fixed with commit 49be495

Comment thread src/utils/ddb/ddb_commands.c Outdated
param.ip_hdl = ctx->dc_poh;
param.ip_epr.epr_hi = DAOS_EPOCH_MAX;
do {
struct vos_iter_anchors anchors = {0};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move anchors outside the while() block; otherwise, you will reset the iteration repeatedly and fall into dead loop.

@knard38 knard38 Sep 18, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arghhhh, not yet used to manage this kind of iterator.

  • Fix iterator management.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arghhhh, not yet used to manage this kind of iterator.

  • Fix iterator management.

Fixed with commit 49be495

Comment thread src/dtx/tests/dts_count.c Outdated
Dbds[i].dbd_cap = DBD_BLOB_DF_CAP;
DbdsOff[i] = umem_ptr2off(&Pool.vp_umm, &Dbds[i]);
Dbds[i].dbd_next =
(i == DBD_COUNT - 1) ? UMOFF_NULL : umem_ptr2off(&Pool.vp_umm, &Dbds[i + 1]);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The real committed DTX table is something like that: for each vos_dtx_blob_df, the vos_dtx_cmt_ent_df array will be contiguous with its vos_dtx_blob_df header. But in your test_setup, all the headers of vos_dtx_blob_df are together, that does not match the real case.
Since the test logic does not use any committed DTX entry, then maybe you can only allocate array for vos_dtx_cmt_ent_df headers without vos_dtx_cmt_ent_df array. Otherwise, the position for &Dbds[i + 1]) may belong to some vos_dtx_cmt_ent_df.

BTW, in DAOS engine, the C logic variable name is usually lowercase. I would suggest the same for the test logic. Thanks!

On the other hand, there is not strict requirement for adding unit test for the new VOS API vos_dtx_get_cmt_cnt that is quite simple and only used for ddb.

@knard38 knard38 Sep 18, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, in DAOS engine, the C logic variable name is usually lowercase. I would suggest the same for the test logic. Thanks!

I have to confess that I have used some existing test file as example and do not take attention of the case variable name issue.

  • Fix coding standard issue

@knard38 knard38 Sep 18, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The real committed DTX table is something like that: for each vos_dtx_blob_df, the vos_dtx_cmt_ent_df array will be contiguous with its vos_dtx_blob_df header. But in your test_setup, all the headers of vos_dtx_blob_df are together, that does not match the real case.
Since the test logic does not use any committed DTX entry, then maybe you can only allocate array for vos_dtx_cmt_ent_df headers without vos_dtx_cmt_ent_df array. Otherwise, the position for &Dbds[i + 1]) may belong to some vos_dtx_cmt_ent_df.

  • Fix DTX blob offset

@knard38 knard38 Sep 18, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the other hand, there is not strict requirement for adding unit test for the new VOS API vos_dtx_get_cmt_cnt that is quite simple and only used for ddb.

I agree that this test is over kill considering the complexity of the tested function. From my side, it allowed me to better understand this part of the DAOS code.
I have no issue to remove it if you think that it could make some confusion. If yes, before removing it, could it be possible to push the fix of the invalid DTX blob offset for being sure that I properly understood the issue.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, in DAOS engine, the C logic variable name is usually lowercase. I would suggest the same for the test logic. Thanks!

I have to confess that I have used some existing test file as example and do not take attention of the case variable name issue.

  • Fix coding standard issue

Fixed with commit 49be495

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The real committed DTX table is something like that: for each vos_dtx_blob_df, the vos_dtx_cmt_ent_df array will be contiguous with its vos_dtx_blob_df header. But in your test_setup, all the headers of vos_dtx_blob_df are together, that does not match the real case.
Since the test logic does not use any committed DTX entry, then maybe you can only allocate array for vos_dtx_cmt_ent_df headers without vos_dtx_cmt_ent_df array. Otherwise, the position for &Dbds[i + 1]) may belong to some vos_dtx_cmt_ent_df.

  • Fix DTX blob offset

Fixed with commit 49be495

Comment thread src/utils/ddb/ddb_commands.c Outdated
if (!SUCCESS(rc))
goto done;

cnt_tmp = vos_dtx_get_cmt_cnt(coh);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest vos_dtx_get_cmt_cnt() can return failure if hit corruption instead of triggering assertion.

@knard38 knard38 Sep 18, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Change assert into error code.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Change assert into error code.

Fixed with commit 49be495

@Nasf-Fan

Copy link
Copy Markdown
Contributor

@Nasf-Fan , please could you have a look at the C code and if the latest changes and comments are OK for you.

Thanks @knard38 for the changes, most of them looks fine for me, just some small issues to be resolved.

Integrate reviewers comments:
- Change assert into error code.
- Fix C coding standard.
- Fix iterator management.
- Fix coding standard issue
- Fix DTX blob offset

Features: aggregation
Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
Comment thread src/dtx/tests/dts_count.c Outdated
assert_true(Dbds[i].dbd_count <= DBD_BLOB_DF_CAP);
struct vos_dtx_blob_df *dbd;

dbd = (struct vos_dtx_blob_df *)((uint64_t)(&mock_dbds[0]) + i * CELL_SIZE);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it correct? (uint64_t)(&mock_dbds[0]) seems to be (void *)(&mock_dbds[0]). Honestly, we can simplify the logic to initialize each mock_dbds[i] independently, something like that:

for (i = 0; i < DBD_COUNT; i++) {
    D_ALLOC(mock_dbds[i], CELL_SIZE);
    if (mock_dbds[i] == NULL)
        return -DER_NOMEM;
    ...
    mock_dbds_off[i] = xxx;

    if (i > 0) {
        mock_dbds[i].dbd_prev = mock_dbds_off[i - 1];
        mock_dbds[i - 1].dbd_next = mock_dbds_off[i];
    } else {
        mock_dbds[i].dbd_prev = UMOFF_NULL;
    }
}

mock_dbds[0].dbd_next = mock_dbds_off[1];

Correspondingly, destroy them one by one in test_teardown:

for (i = 0; i < DBD_COUNT; i--) {
    D_FREE(mock_dbds[i]);
}

Then the logic is more easy to be understood and avoid bugs during calculate next and prev.

@knard38 knard38 Sep 19, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfectly agree, using one allocation is over complicated for no good reason (i.e. just avoid to have two loop: one for memory allocation and one for initialisation).
Going to fix this as you suggested.

  • Use multiple allocation.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfectly agree, using one allocation is over complicated for no good reason (i.e. just avoid to have two loop: one for memory allocation and one for initialisation). Going to fix this as you suggested.

  • Use multiple allocation.

Fixed with commit 9d81546

Comment thread src/vos/vos_dtx.c Outdated

return cnt;
*cnt = tmp;
rc = -DER_SUCCESS;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] I am wondering why return -DER_SUCCESS instead of 0? That seems confused although we usually define DER_SUCCESS as 0. You have already described \return 0 on success, error otherwise in the header. So please just return 0 and adjust related test cases. Thanks!

@knard38 knard38 Sep 19, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Replace -DER_SUCCESS with 0

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Replace -DER_SUCCESS with 0

Fixed with commit 9d81546

Integrate reviewers comments:
- Replace -DER_SUCCESS with 0
- Use multiple allocation

Features: aggregation
Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
Fix clang format issue

Features: aggregation
Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
Comment thread src/dtx/tests/dts_count.c
for (i = 0; i < DBD_COUNT; i++) {
struct vos_dtx_blob_df *dbd;
D_ALLOC(mock_dbds[i], CELL_SIZE);
assert_non_null(mock_dbds[i]);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mock_dbds_off[i] = umem_ptr2off(&mock_pool.vp_umm, mock_dbds[i]);

Add above line in this for section. Then in subsequent for section, we can directly use them, for example:

		if (i == DBD_COUNT - 1)
			mock_dbds[i]->dbd_next = UMOFF_NULL;
		else
			mock_dbds[i]->dbd_next = mock_dbds_off[i + 1];
		if (i == 0)
			mock_dbds[i]->dbd_prev = UMOFF_NULL;
		else
			mock_dbds[i]->dbd_prev = mock_dbds_off[i - 1];

Anyway, just some code optimization, not fatal. If you have other chance to refresh the patch, you can consider.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I have no mandatory change requested, I will integrate your remarks in the following up PR introducing the ddb aggregation command.

@knard38 knard38 Sep 22, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Simplify linked list management

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Simplify linked list management

Fixed with commit 182b4ac of PR #16746

@daosbuild3

Copy link
Copy Markdown
Collaborator

@kjacque kjacque left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go changes LGTM. Didn't look closely at the rest.

@daosbuild3

Copy link
Copy Markdown
Collaborator

@daosbuild3

Copy link
Copy Markdown
Collaborator

Test stage Functional Hardware Large MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-16745/15/execution/node/1300/log

@knard38
knard38 requested a review from a team September 25, 2025 04:17
@knard38

knard38 commented Sep 25, 2025

Copy link
Copy Markdown
Contributor Author

@daos-stack/daos-gatekeeper please could you lend this PR.

@gnailzenh
gnailzenh merged commit 9860712 into master Sep 26, 2025
49 checks passed
@gnailzenh
gnailzenh deleted the ckochhof/dev/master/daos-17322-part1 branch September 26, 2025 02:29
@knard38 knard38 changed the title DAOS-17322 ddb: DTX aggregation with ddb DAOS-17322 ddb: DTX aggregation with ddb (Part 1) Oct 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CR Catastrophic Recovery Feature

Development

Successfully merging this pull request may close these issues.

6 participants