Skip to content

feat: add ZVG-Portal provider for forced-sale auctions (Zwangsversteigerungen) - #499

Merged
orangecoding merged 6 commits into
orangecoding:masterfrom
nmehlei:feat/zvg-portal-provider
Nov 3, 2026
Merged

orangecoding merged 6 commits into
orangecoding:masterfrom
nmehlei:feat/zvg-portal-provider

Conversation

@nmehlei

@nmehlei nmehlei commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds a provider for the ZVG-Portal (zvg-portal.de), the German justice portal that lists forced-sale auctions. Auctions rarely appear on the commercial portals, so this finds houses and plots nobody else lists.

The portal has no GET search: a search is a form POST answered with a server-rendered table. The job URL only describes the search and the provider replays it:

https://www.zvg-portal.de/index.php?button=Suchen&land_abk=sh&obj_arr=3,15,16&plz_prefix=22,25

  • land_abk: state code (required by the portal, lowercase)
  • obj_arr: comma separated object type ids, numbered as in the portal's form (3 Einfamilienhaus, 15 Baugrundstück, 16 unbebautes Grundstück, ...). Sent as obj_arr[], the field the portal filters on; leave it out and every type of the state comes back
  • any other field of the search form (plz, ort, ...) is passed through
  • plz_prefix (not a portal field): keeps only results with a postcode starting with one of the prefixes, for a state-wide search where only a region is wanted

Notes:

  • The price is the court's Verkehrswert (appraised value), not an asking price; the description says so.
  • The portal's detail page (button=showZvg) answers "error" without a portal Referer, so a link from a notification cannot open it. Listings link to the portal's search page instead (unique via an extra zvg_id parameter), and fetchDetails reads the detail page server-side for the court's description, the living area and the auction location.
  • The listing id is the court's record alone, and the court's placeholder price of 1 EUR is left empty.
  • Plain fetch with a timeout, needsBrowser: false.

Related issue

none

AI disclosure (required)

Fredy accepts AI assisted contributions, but they have to be declared. An
undeclared AI PR will be closed. Tick exactly one box:

  • ai:none - I wrote this myself. No AI generated code, text or commit messages.
  • ai:assisted - AI helped with parts of it (autocomplete, refactoring, tests, docs). I reviewed and understand every line.
  • ai:generated - AI produced most or all of this PR. I reviewed it, but it is largely machine written.

Which AI: Claude Code, Sonnet 5.5

How much: the provider, its test, the fixture plumbing and the downloader were drafted with Claude Code; I tested it against the live portal, reviewed every line and ran the checks.

Why: a forced-sale source for houses and plots in my area, and faster scaffolding of a provider with an unusual POST search.

Checklist

  • yarn test:offline passes (or yarn test if the change touches a live provider)
  • yarn lint and yarn format:check pass
  • The change is useful for everybody, not a custom tweak for my own setup
  • I have read and answered the AI disclosure above honestly

@orangecoding orangecoding left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice idea, forced sales are a source nobody else covers. Can't merge it yet though. The object type filter doesn't do anything on the portal side (details inline), so every job pulls every auction of the whole state. The rest is smaller stuff.

Comment thread lib/provider/zvgPortal.js Outdated
Comment on lines +51 to +52
if (name === 'obj_liste') {
for (const id of value.split(',')) if (id.trim()) body.append('obj_liste', id.trim());

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

obj_liste is only the left select in the portal form, the list of types you can pick from. What actually gets submited as the filter is obj_arr[]: insertObj() copies the options over from obj_liste and checkFormular() selects everything in obj_arr right before submit. The server just ignores obj_liste. And without [] PHP would only keep the last value anyway, you can see that on the result page, it echoes <input type=hidden name=obj_liste value='16'>.

Tried it against the live portal with NRW:

  • no type filter: 910 Termine
  • obj_liste=3&obj_liste=15&obj_liste=16: 910
  • obj_arr[]=3&obj_arr[]=15&obj_arr[]=16: 238

So right now every job gets every object type of the whole state. Flats, garages, parking spots, all of it, and on the first run all of that is new, gets geocoded and notifed. The SH test only passes because SH has 3 auctions at the moment and they're all Einfamilienhäuser. Can't tell a working filter from a broken one there.

Please send the ids as obj_arr[]. Whatever the URL parameter ends up beeing called, the doc block (l. 14/17) and the PR description should match it. A small test on the POST body would have caught this btw.

Comment thread tools/testFixtures/downloadFixtures.js Outdated
Comment on lines +354 to +357
const body = new URLSearchParams({ ger_id: '0', order_by: '2', art: '' });
for (const [name, value] of params) {
if (name === 'button') continue;
if (name === 'obj_liste') value.split(',').forEach((id) => body.append('obj_liste', id));

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a second copy of parseSearchUrl, incl. the same obj_liste bug. The betterhomes downloader imports buildSearchPayload from its provider for exactly that reason. Can you export parseSearchUrl and use it here? Then the fixture gets recorded with the same request the provider actualy sends.

Comment thread lib/provider/zvgPortal.js Outdated
return {
...listing,
size: listing.size ?? extractNumber(livingArea ?? plotArea ?? ''),
description: [listing.description, description, place ? `Ort der Versteigerung: ${place}` : null]

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fetchDetails appends to listing.description, so running it twice gives a different result than running it once. That bites with the manual "fetch details" button (refetchListingDetails): it passes in the stored row, and after the first fetch (or a run with details enabled) that row already has the court text in it. Every click appends Beschreibung and Ort again and isImprovement happyly calls it updated.

Simulated it with a live detail page: 560 → 1046 → 1532 chars, court text in there 1x, 2x, 3x. A second call should give the same description, e.g. just dont append text that's already there.

Comment thread lib/provider/zvgPortal.js Outdated
const rows = parseResults(await response.text());
return plzPrefixes.length === 0
? rows
: rows.filter((row) => plzPrefixes.some((p) => new RegExp(`\\b${p}\\d*\\s`).test(row.address ?? '')));

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few things with the prefix match. \b${p}\d*\s also matches house numbers followed by a space. With prefix 4 on the live NRW list I get 6 adresses that aren't in a 4xxxx postcode, for example Seydlitzstr. 48 a, 58840 Plettenberg or Wintersheide 4 - 14, 33689 Bielefeld. The prefix also goes into new RegExp unescaped, so a typo like plz_prefix=2( throws and the whole run fails. And the regex is rebuilt for every row.

Matching against the actual postcodes gets rid of all three:

(row.address?.match(/\b\d{5}\b/g) ?? []).some((zip) => plzPrefixes.some((p) => zip.startsWith(p)))

Comment thread lib/provider/zvgPortal.js Outdated
.text()
.replace(/\s+/g, ' ');
const description = /Beschreibung:\s*(.*?)\s*Verkehrswert/.exec(text)?.[1];
const place = /Ort der Versteigerung:\s*(.*?)\s*(?:Informationen zum|Gericht:)/.exec(text)?.[1];

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small one: on pages without a court link Ort der Versteigerung is followed by GeoServer: instead (zvg_id=168706, nw, AG Essen), so place stays empty there. Adding GeoServer: to the alternatives shuold do it.

… fetchDetails idempotent

- the portal filters on obj_arr[]; obj_liste is only the picker list and was ignored
- postcode prefixes are compared with the 5-digit postcodes of the address (no regex from user input)
- fetchDetails no longer appends text the description already has
- place of the auction also ends at GeoServer:
- the fixture downloader reuses the provider's parseSearchUrl
- tests for the POST body, the postcode filter, repeated fetchDetails and GeoServer
…living-area match, request timeout, docs

- the id is the court's record alone; a corrected Verkehrswert no longer stores and announces the auction twice
- the 1 EUR the court enters before it sets a Verkehrswert is no price
- living area also from 'Wohnfl.', 'Wohnfläche: ca.', 'rd.', 'EG'
- 30s timeout on every request; warn when the portal answers a bare error
- empty address parts ', ,' are dropped
- tests assert the POST actually sent, the id, the placeholder, the living-area forms and, live, that obj_arr narrows a large state
- doc/providers.md: ZVG-Portal section
@nmehlei

nmehlei commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review, you're right on all points. I reproduced the type filter issue against the live portal: NRW returns 772 auctions with no filter and with obj_liste, and 199 with obj_arr[].

Pushed two commits. The first addresses your comments:

  • The ids are sent as obj_arr[], the URL parameter is obj_arr, and the doc block and PR description match.
  • parseSearchUrl is exported and downloadFixtures.js uses it.
  • fetchDetails no longer appends text that's already there.
  • The plz_prefix filter matches the 5-digit postcodes (your suggestion).
  • GeoServer: is added as an end marker for the place.

The second is from a review of my own against the live portal: the listing id no longer includes the price (a corrected Verkehrswert would otherwise have been stored and announced twice), the 1 EUR placeholder isn't taken for a price, more living-area spellings are read, requests have a timeout, and I added a doc section. One limitation I'd rather state than hide: the portal's detail page only answers with a portal Referer, so the notification link can only open the search page.

Tests now assert the POST that is sent, and a live-only test checks that the type filter really narrows a large state.

@orangecoding orangecoding left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good now, thanks for fixing everything so quickly. The NRW live test is a nice touch.

@orangecoding
orangecoding merged commit 9fcebf8 into orangecoding:master Nov 3, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants