Secondary Storage Vm creation failure in zone

[fespinozasanchez]

ISSUE TYPE

• Bug Report

VERSION

• CloudStack 4.20.1.0
• MariaDB Galera 15.1 Distrib 10.5.27-MariaDB, for Linux (x86_64) using EditLine wrapper
• ceph version 19.2.3 (c92aebb279828e9c3c1f5d24613efca272649e62) squid (stable)

OS / ENVIRONMENT

• Rocky Linux 9.6

CONFIGURATION

• 1 Mgmt
• 2 Hypervisor KVM

Normal CloudStack configuration according to documentation
Advanced networking with 4 Network Interfaces:

Cloudbr0 → Management (LAN)
Cloudbr1 → Public (WAN)
Cloudbr2 → Guest
Cloudbr3 → Storage

Primary Storage: RBD (Ceph) pool cloudstack
Secondary Storage: RGW S3 (Ceph) bucket cloudstack

Issue:
The following virtual machines fail to deploy:

v-3-VM
s-4-VM

They cannot be created because the system reports insufficient capacity to deploy the VM:

2025-09-05 14:10:29,334 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-9:[ctx-5b3cf25c, job-3/job-159, ctx-75f60b19]) (logid:2f8eebb8) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {"id":3,"instanceName":"v-3-VM","state":"Starting","type":"ConsoleProxy","uuid":"855516ca-9e46-420e-9c6a-c92c9ad32eb5"}Scope=interface com.cloud.dc.DataCenter; id=3
2025-09-05 14:10:29,334 INFO  [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-9:[ctx-5b3cf25c, job-3/job-159, ctx-75f60b19]) (logid:2f8eebb8) Rethrow exception com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {"id":3,"instanceName":"v-3-VM","state":"Starting","type":"ConsoleProxy","uuid":"855516ca-9e46-420e-9c6a-c92c9ad32eb5"}Scope=interface com.cloud.dc.DataCenter; id=3

There is capacity in the primary storage and in the hypervisors to be able to deploy the virtual machines, so the problem does not lie in that.

I previously made a similar installation, but instead of ceph for primary and secondary storage, I used a TrueNAS NFS, there I have no problem, so perhaps the problem may lie in the Primary Storage, since Cloudstack is able to download the template and store it in the Secondary Storage (Bucket), but cloudstack recognizes the capabilities of the primary storage, so I am in doubt and I cannot understand what is wrong.

I attach a more extended log file

2025-09-05.log

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[DaanHoogland]

@fespinozasanchez your error begins here:

2025-09-05 14:09:55,731 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] (Work-Job-Executor-7:[ctx-79a7b15f, job-3/job-153, ctx-dd4bc9b5]) (logid:2f8eebb8) needCacheStorage true, dest at null dest role Primarytemplate/tmpl/1/3/routing-3/systemvmtemplate-4.20.1-x86_64-kvm.qcow2.bz2 src role Image

It seems your s3 secondary storage needs an NFS staging store. see https://docs.cloudstack.apache.org/en/4.21.0.0/installguide/configuration.html#adding-secondary-storage