feat: add HTTP catalog for Python packages and repository metrics

.gitignore

@@ -8,6 +8,7 @@ __pycache__/
 # Distribution / packaging
 .Python
 env/
+.venv/
 build/
 develop-eggs/
 dist/
@@ -61,3 +62,6 @@ target/
 
 # PyCharm
 .idea
+
+# VS Code / Cursor
+.vscode/

CHANGES/1358.feature

@@ -0,0 +1 @@
+Added repository package catalog and metrics endpoints, plus ``collapse_builds`` and ``base_version`` on the Python package content API. Existing installs pick up access policy for the new actions on migrate unless the policy was customized.

docs/index.md

@@ -12,6 +12,7 @@ The REST API documentation for `pulp_python` is available [here](site:pulp_pytho
 
 - [Create local mirrors of PyPI](site:pulp_python/docs/user/guides/sync/) that you have full control over
 - [Upload your own Python packages](site:pulp_python/docs/user/guides/upload/)
+- [Browse the package catalog](site:pulp_python/docs/user/guides/catalog/) over the REST API
 - [Perform pip install](site:pulp_python/docs/user/guides/host/) from your Pulp Python repositories
 - Download packages on-demand to reduce disk usage
 - Every operation creates a restorable snapshot with Versioned Repositories

docs/user/guides/_SUMMARY.md

@@ -1,6 +1,7 @@
 * [Set up your own PyPI](pypi.md)
 * [Sync from Remote Repositories](sync.md)
 * [Upload and Manage Content](upload.md)
+* [Browse the package catalog](catalog.md)
 * [Host Python Content](host.md)
 * [Vulnerability Report](vulnerability_report.md)
 * [Attestation Hosting](attestation.md)

docs/user/guides/catalog.md

@@ -0,0 +1,96 @@
+# Browse the package catalog
+
+Pulp CLI commands for these endpoints are generated from the OpenAPI spec in a separate package; until that is updated, use HTTP.
+
+The content list (`/pulp/api/v3/content/python/packages/`) returns **one row per distribution file** (wheel, sdist, …). For catalog UIs and automation that need **one row per package name**, plus repository metrics, use the repository package index.
+
+These endpoints default to the **latest complete repository version**. `{pulp_id}` is the repository UUID. Pass `repository_version` (HREF or PRN) to read a specific version of that repository.
+
+## List packages
+
+```bash
+http GET "${BASE_ADDR}/pulp/api/v3/repositories/python/python/${REPO_PK}/packages/?limit=10"
+```
+
+Pagination `count` is the number of **distinct packages** (`name_normalized`), not files.
+
+Each row includes both a simple version list and per-version metadata:
+
+```json
+{
+  "name": "shelf-reader",
+  "name_normalized": "shelf-reader",
+  "versions": ["0.1"],
+  "latest_releases": [
+    {
+      "version": "0.1",
+      "release": "",
+      "created_at": "2026-08-10T10:45:08.099362Z"
+    }
+  ]
+}
+```
+
+`set(versions)` is always the same as `set(latest_releases[].version)`. There is one `latest_releases` entry per **logical version** (after stripping a trailing rebuild suffix `\.[a-zA-Z]+-\d+$`), not per wheel or sdist.
+
+`created_at` is when that logical version entered the repository: the earliest `RepositoryContent.pulp_created` among its files, falling back to the content unit's `pulp_created`. `release` is empty until Python rebuilds are stored.
+
+### Prefix search
+
+```bash
+http GET "${BASE_ADDR}/pulp/api/v3/repositories/python/python/${REPO_PK}/packages/" \
+  name_normalized__istartswith==shelf
+```
+
+`name_normalized__istartswith` and `name__istartswith` are case-insensitive (`ILIKE`). Prefix search belongs on this index, not on the flat content list.
+
+## Repository metrics
+
+```bash
+http GET "${BASE_ADDR}/pulp/api/v3/repositories/python/python/${REPO_PK}/metrics/"
+```
+
+```json
+{
+  "package_count": 3,
+  "version_count": 9,
+  "build_count": 9
+}
+```
+
+Counts use Python package content units in that repository version (not filtered by `packagetype`):
+
+| Field | Identity |
+|-------|----------|
+| `package_count` | distinct `name_normalized` |
+| `version_count` | distinct `(name_normalized, base_version)` after rebuild-suffix strip |
+| `build_count` | distinct `(name_normalized, full version)` |
+
+Until rebuild suffixes exist, `version_count` equals `build_count`.
+
+## List versions of a package
+
+Use the existing content API. Pass `packagetype=sdist` for one representative file per PEP version (retry with `packagetype=bdist_wheel` if a release is wheel-only).
+
+`collapse_builds=true` keeps one unit per logical version (`name_normalized` + `base_version`), the one with the latest `pulp_created`. Do not nest rebuilds on this list. Clients can drain Pulp `next` if the page is full.
+
+```bash
+http GET "${BASE_ADDR}/pulp/api/v3/content/python/packages/" \
+  name==shelf-reader \
+  packagetype==sdist \
+  collapse_builds==true \
+  repository_version=="${LATEST_VERSION_HREF}"
+```
+
+Every content row includes `base_version` (stripped version; equal to `version` when there is no suffix).
+
+## Get one version
+
+Omit `collapse_builds`. Filter with `name`, `version`, and `packagetype=sdist`:
+
+```bash
+http GET "${BASE_ADDR}/pulp/api/v3/content/python/packages/" \
+  name==shelf-reader \
+  version==0.1 \
+  packagetype==sdist
+```

pulp_python/app/catalog.py

@@ -0,0 +1,166 @@
+"""Helpers for repository package catalog, metrics, and rebuild collapse."""
+
+from collections import defaultdict
+
+from django.db.models import CharField, Func, Max, Min, Q, Value
+from packaging.version import InvalidVersion, Version
+
+from pulp_python.app.models import PythonPackageContent
+
+# POSIX regex for REGEXP_REPLACE. PostgreSQL does not treat ``\d`` as digits.
+BUILD_SUFFIX_PG_REGEX = r"\.[a-zA-Z]+-[0-9]+$"
+
+
+def base_version_annotation(field_name="version"):
+    """SQL expression that strips a trailing rebuild suffix from ``version``.
+
+    PostgreSQL POSIX regex does not treat ``\\d`` as digits, so the SQL pattern
+    uses ``[0-9]`` while the Python pattern in ``strip_build_suffix`` uses ``\\d``.
+    Implemented with ``REGEXP_REPLACE`` so it does not depend on Django's
+    ``RegexpReplace`` (not present in every Django 4.2/5.2 packaging Pulp uses).
+    """
+    return Func(
+        field_name,
+        Value(BUILD_SUFFIX_PG_REGEX),
+        Value(""),
+        function="REGEXP_REPLACE",
+        output_field=CharField(),
+    )
+
+
+def collapse_python_builds(queryset):
+    """Keep one content unit per ``(name_normalized, base_version)``.
+
+    ``base_version`` is ``version`` with a trailing rebuild suffix stripped.
+    The unit with the latest ``pulp_created`` is kept. Callers that want one
+    row per logical version (not per wheel/sdist) should also filter
+    ``packagetype``.
+    """
+    return (
+        queryset.prefetch_related(None)
+        .annotate(_collapse_base_version=base_version_annotation())
+        .order_by("name_normalized", "_collapse_base_version", "-pulp_created")
+        .distinct("name_normalized", "_collapse_base_version")
+    )
+
+
+def python_packages_in_version(repository_version):
+    """Python package content contained in ``repository_version``."""
+    if repository_version is None:
+        return PythonPackageContent.objects.none()
+    return PythonPackageContent.objects.filter(pk__in=repository_version.content)
+
+
+def apply_package_prefix_filters(queryset, name_normalized_prefix=None, name_prefix=None):
+    """Apply case-insensitive prefix filters used by the package index."""
+    if name_normalized_prefix:
+        queryset = queryset.filter(name_normalized__istartswith=name_normalized_prefix)
+    if name_prefix:
+        queryset = queryset.filter(name__istartswith=name_prefix)
+    return queryset
+
+
+def distinct_package_names_qs(content_qs):
+    """One row per distinct ``name_normalized``, ordered for stable pagination."""
+    return (
+        content_qs.order_by()
+        .values("name_normalized")
+        .annotate(name=Max("name"))
+        .order_by("name_normalized")
+    )
+
+
+def _version_sort_key(version):
+    try:
+        return (0, Version(version))
+    except InvalidVersion:
+        return (1, version)
+
+
+def assemble_package_index(content_qs, name_rows, repository, repository_version):
+    """Build package-index dicts for ``name_rows``.
+
+    ``created_at`` is the earliest repository-membership time
+    (``RepositoryContent.pulp_created``) of any file of that logical version
+    in ``repository_version``, falling back to the content unit's ``pulp_created``.
+    """
+    if not name_rows or repository_version is None:
+        return []
+
+    names = [row["name_normalized"] for row in name_rows]
+    name_by_normalized = {row["name_normalized"]: row["name"] for row in name_rows}
+
+    in_this_version = Q(
+        version_memberships__repository=repository,
+        version_memberships__version_added__number__lte=repository_version.number,
+    ) & (
+        Q(version_memberships__version_removed__isnull=True)
+        | Q(version_memberships__version_removed__number__gt=repository_version.number)
+    )
+
+    release_rows = (
+        content_qs.filter(name_normalized__in=names)
+        .annotate(_base_version=base_version_annotation())
+        .values("name_normalized", "_base_version")
+        .annotate(
+            membership_created=Min(
+                "version_memberships__pulp_created",
+                filter=in_this_version,
+            ),
+            unit_created=Min("pulp_created"),
+        )
+    )
+
+    releases_by_name = defaultdict(list)
+    for rel in release_rows:
+        releases_by_name[rel["name_normalized"]].append(rel)
+
+    result = []
+    for row in name_rows:
+        normalized = row["name_normalized"]
+        rels = sorted(
+            releases_by_name.get(normalized, []),
+            key=lambda item: _version_sort_key(item["_base_version"]),
+        )
+        versions = [item["_base_version"] for item in rels]
+        latest_releases = [
+            {
+                "version": item["_base_version"],
+                "release": "",
+                "created_at": item["membership_created"] or item["unit_created"],
+            }
+            for item in rels
+        ]
+        result.append(
+            {
+                "name": name_by_normalized[normalized],
+                "name_normalized": normalized,
+                "versions": versions,
+                "latest_releases": latest_releases,
+            }
+        )
+    return result
+
+
+def repository_metrics(content_qs):
+    """Distinct package / logical-version / build counts for package content.
+
+    Identity is always ``PythonPackageContent`` (not filtered by packagetype):
+
+    * ``package_count``: distinct ``name_normalized``
+    * ``version_count``: distinct ``(name_normalized, base_version)``
+    * ``build_count``: distinct ``(name_normalized, version)``
+
+    Until rebuild suffixes exist, ``version_count`` equals ``build_count``.
+    """
+    content_qs = content_qs.order_by()
+    return {
+        "package_count": content_qs.values("name_normalized").distinct().count(),
+        "version_count": (
+            content_qs.annotate(_base_version=base_version_annotation())
+            .values("name_normalized", "_base_version")
+            .distinct()
+            .count()
+        ),
+        "build_count": content_qs.values("name_normalized", "version").distinct().count(),
+    }