[Migration Engine Part 1] Implement the foundation for migration engine

.github/workflows/ci.yml

@@ -27,7 +27,7 @@ jobs:
         # Categories are declared by the <TestCategory> property in each test project. A category can
         # span several projects that share infrastructure, so that the container is provisioned once
         # and the union of their build closures is compiled once.
-        test-category: [ DefaultCore, DefaultMonitoring, Raven, SqlServer, PostgreSql, RabbitMQ, AzureServiceBus, AzureStorageQueues, MSMQ, SQS, IBMMQ ]
+        test-category: [ DefaultCore, DefaultMonitoring, Migration, Raven, SqlServer, PostgreSql, RabbitMQ, AzureServiceBus, AzureStorageQueues, MSMQ, SQS, IBMMQ ]
         include:
           - os: windows-latest
             os-name: Windows
@@ -87,7 +87,7 @@ jobs:
         id: select
         run: ./tools/select-test-projects.ps1 -Category ${{ matrix.test-category }}
       - name: Download RavenDB Server
-        if: matrix.test-category == 'DefaultCore'
+        if: contains(fromJSON('["DefaultCore", "Migration"]'), matrix.test-category)
         run: ./tools/download-ravendb-server.ps1
       - name: Build
         id: build

README.md

@@ -33,6 +33,10 @@ It's also possible to [locally test containers built from PRs in GitHub Containe
 If the instance is executed for the first time, it must set up the required infrastructure. To do so, once the instance is configured to use the selected transport and persister, run it in setup mode. This can be done by using the `Setup {instance name}` launch profile that is defined in 
 the `launchSettings.json` file of each instance. When started in setup mode, the instance will start as usual, execute the setup process, and exit. At this point the instance can be run normally by using the non-setup launch profile.
 
+## Migrating from RavenDB to SQL Server or PostgreSQL
+
+See [Migrating from RavenDB to SQL Server or PostgreSQL](docs/migration/ravendb-to-sql-migration-instructions.md).
+
 ## Secrets
 
 Testing using the [CI workflow](/.github/workflows/ci.yml) depends on the following secrets. The Particular values for these secrets are stored in the secure note named **ServiceControl Repo Secrets**.

docs/migration/ravendb-to-sql-migration-instructions.md

@@ -0,0 +1,54 @@
+# Migrating from RavenDB to SQL Server or PostgreSQL
+
+This page covers what you can run today. How the migration works, and what is planned, is in the [migration overview](ravendb-to-sql-migration-overview.md) and the [system design diagram](migration-system-design-diagram.png).
+
+> [!NOTE]
+> Copying data is not built yet. The one migration command available is the source report. It sends RavenDB only reads, but loading a database lets RavenDB's own expiration, its automatic deletion of documents past their retention date, run against it. If you are keeping the RavenDB database as a fallback, back it up before you run the report, as [Goals](ravendb-to-sql-migration-overview.md#goals) explains.
+
+## Before you start
+
+The source is a ServiceControl error instance on RavenDB. Keep its RavenDB settings in its configuration: the migration reads RavenDB through them, including after `PersistenceType` is switched to SQL Server or PostgreSQL.
+
+| Setting | Environment variable | What it is |
+| --- | --- | --- |
+| `ServiceControl/RavenDB/ConnectionString` | `SERVICECONTROL_RAVENDB_CONNECTIONSTRING` | An external RavenDB server. Leave unset for an embedded database |
+| `ServiceControl/DbPath` | `SERVICECONTROL_DBPATH` | The embedded database's data directory |
+| `ServiceControl/RavenDB/DatabaseName` | `SERVICECONTROL_RAVENDB_DATABASENAME` | The primary database, `primary` by default |
+| `LicensingComponent/RavenDB/ThroughputDatabaseName` | `LICENSINGCOMPONENT_RAVENDB_THROUGHPUTDATABASENAME` | The throughput database, `throughput` by default |
+| `ServiceControl/RavenDB/ClientCertificatePath` or `ServiceControl/RavenDB/ClientCertificateBase64`, with `ServiceControl/RavenDB/ClientCertificatePassword` | `SERVICECONTROL_RAVENDB_CLIENTCERTIFICATEPATH` and so on | A secured external server's client certificate |
+| `ServiceControl/ErrorRetentionPeriod` | `SERVICECONTROL_ERRORRETENTIONPERIOD` | Required. Don't change it during the move |
+
+`ServiceControl/Migration/SourcePersistenceType` defaults to `RavenDB` and needs no setting.
+
+## Report on the source
+
+Run the instance's executable with `--migration-source-report`:
+
+```powershell
+# Installed on Windows, from the instance's installation folder
+.\ServiceControl.exe --migration-source-report
+```
+
+```shell
+# Container, against an external RavenDB server
+docker run --rm --env-file servicecontrol.env ghcr.io/particular/servicecontrol:<version> --migration-source-report
+```
+
+From source, build `src/ServiceControl` and run the same command from its output folder, as in [How to run/debug locally](../../README.md#how-to-rundebug-locally).
+
+The report prints the RavenDB server version, whether the source is embedded or external and where it is, both database names with the setting each came from, and a row count for every collection in both databases.
+
+- **External server:** run it while ServiceControl is running. It sends only reads, and the note above about expiration applies to a server you are keeping as a fallback.
+- **Embedded database:** stop the ServiceControl service, run the report, then start the service again. The report starts its own RavenDB process against the data directory, which cannot happen while the instance holds it.
+- **Container with an embedded database:** not supported, because the container image does not ship the RavenDB server. Point the instance at an external RavenDB server instead.
+
+## If the report fails
+
+The error names the setting to fix:
+
+- **"has no database named ..."**: the database name setting it quotes is wrong.
+- **"refused its client certificate access ..."**: grant that certificate Read access to the database, or supply a certificate that has it.
+
+## Not available yet
+
+Copying the data (`MigrationMode`), the dry run, and the status and verify commands are planned but not built. The planned steps are in [Migration workflow](ravendb-to-sql-migration-overview.md#migration-workflow).