Cloudstack copy snapshots error across zones

[arpanbht]

We are experiencing erros while testing to create volume snapshots across multiple zones.

We have two separate management servers and created two separate zones and when we are creating snapshots of the volumes we are selecting both the zones but it is not creating the snapshot in the 2nd zone, it is creating it only on the primary zones.

We have two separate nfs servers for both the zones, and when we are manually copying the snapshot to the 2nd zones it is giving this error

 failure to copy snapshot ID: 5609 from snapshot chain
2026-08-27 06:35:40,925 WARN  [c.c.a.AlertManagerImpl] (RemoteHostEndPoint-4:[ctx-3c5b75c0]) (logid:3a343310) alertType=[28] dataCenterId=[4] podId=[null] clusterId=[null] message=[Failed to copy snapshot: 887bacf5-042e-417a-a2f0-7fa0aaeaac21 with error: Connection refused].
2026-08-27 06:35:40,927 WARN  [c.c.a.AlertManagerImpl] (RemoteHostEndPoint-4:[ctx-3c5b75c0]) (logid:3a343310) No recipients set in global setting 'alert.email.addresses', skipping sending alert with subject [Failed to copy snapshot: 887bacf5-042e-417a-a2f0-7fa0aaeaac21 with error: Connection refused] and content [Failed to copy snapshot: 887bacf5-042e-417a-a2f0-7fa0aaeaac21 with error: Connection refused].
2026-08-27 06:35:40,927 ERROR [o.a.c.s.i.BaseImageStoreDriverImpl] (RemoteHostEndPoint-4:[ctx-3c5b75c0]) (logid:3a343310) Failed to copy snapshot: 887bacf5-042e-417a-a2f0-7fa0aaeaac21 with error: Connection refused
2026-08-27 06:35:40,934 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-84:[ctx-35c4b8f9, job-54912]) (logid:97d5a8af) Unexpected exception while executing org.apache.cloudstack.api.command.user.snapshot.CopySnapshotCmd com.cloud.utils.exception.CloudRuntimeException: Failed to copy snapshot
        at com.cloud.storage.snapshot.SnapshotManagerImpl.copySnapshot(SnapshotManagerImpl.java:2343)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:109)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
        at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
        at jdk.proxy3/jdk.proxy3.$Proxy245.copySnapshot(Unknown Source)
        at org.apache.cloudstack.api.command.user.snapshot.CopySnapshotCmd.execute(CopySnapshotCmd.java:188)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:173)
        at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:110)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:686)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:634)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)

2026-08-27 06:35:40,935 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-84:[ctx-35c4b8f9, job-54912]) (logid:97d5a8af) Complete async job-54912, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed to copy snapshot"}
2026-08-27 06:35:40,935 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-84:[ctx-35c4b8f9, job-54912]) (logid:97d5a8af) Publish async job-54912 complete on message bus
2026-08-27 06:35:40,935 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-84:[ctx-35c4b8f9, job-54912]) (logid:97d5a8af) Wake up jobs related to job-54912
2026-08-27 06:35:40,935 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-84:[ctx-35c4b8f9, job-54912]) (logid:97d5a8af) Update db status for job-54912
2026-08-27 06:35:40,936 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-84:[ctx-35c4b8f9, job-54912]) (logid:97d5a8af) Wake up jobs joined with job-54912 and disjoin all subjobs created from job- 54912

And from the image given below we are copying the snapshot across zones

Do we need to sync both the nfs in order to do this?

[DaanHoogland]

I’ve seen this mention before. Did you search ‘issues’ and PRs, @arpanbht ? a fix may already be out or awaiting merge/release.

[arpanbht]

@DaanHoogland Haven't look at the previous issues or prs. I will get back to you if I can't find any help.

[prashanthr2]

Tough to confirm without full logs.. but this looks similar to #13900

@arpanbht please check above, I think this is the issue @DaanHoogland was mentioning