Qumulo failover with DFS namespaces: why clients stay on the secondary

DFS fails over to a Qumulo secondary on its own, but often won’t fail back. Why that happens, which DFS settings matter, and how to bring SMB clients back reliably.

Short answer: DFS sends clients to another folder target when the current one is unreachable. It does not send them back just because the preferred target is available again, as long as the secondary still answers. To fail back SMB clients reliably, the secondary share has to become unreachable for a short time, longer than the clients’ referral cache, and only then be restored.

Why DFS is the better failover path for SMB

SMB clients that connect to a Qumulo cluster directly by name or IP use a Kerberos ticket issued for that host. If you move the floating IPs to another cluster, the ticket no longer matches and clients have to reconnect. Users see broken drive mappings and open files.

With a DFS namespace, clients connect to \\domain\namespace\folder. Behind each folder are two folder targets: the share on the primary Qumulo cluster and the replicated share on the secondary. When the primary target stops answering, the DFS client tries the next target in its referral list. Clients keep their connection to the namespace and authenticate against the secondary as a normal new target.

A typical setup

Element Example
Namespace folder \\corp.example\files\Media
Folder target 1 (preferred) \\qumulo-prod-dfs.corp.example\Media
Folder target 2 \\qumulo-dr-dfs.corp.example\Media
Replication /Media on prod → /qumulo-prod/Media on DR

Use a dedicated FQDN and dedicated floating IPs for DFS targets, separate from NFS and from SMB shares that clients reach directly. Then you can take the DFS address off the primary without touching other clients.

The share on the secondary must exist before you need it, with the same name and the same share permissions. Its directory stays read-only as long as replication runs.

Failover: DFS does most of the work

A planned failover with DFS looks like this:

  1. Stop writes on the primary and let the final replication job finish.
  2. Make the target directories writable on the secondary (qq replication_make_target_writable).
  3. Remove the DFS floating IPs from the primary. Do not move them to the secondary: the secondary is reached under its own name.
  4. Rename the source share on the primary, so a client that still holds a session cannot write to it.

Clients that try the primary target get no answer and move to the next target. How quickly depends on the client’s open sessions and its referral cache.

In an unplanned failover the primary is already gone, so DFS moves clients on its own once you have made the targets writable. Watch out when the primary comes back: its DFS IPs and shares return with it, and clients may land on a stale, writable copy. Remove the IPs and set the source read-only before that happens.

Failback: where it goes wrong

After failover, the secondary holds the current data. Failback means replicating that data back to the primary, switching the replication direction back and returning clients to the primary.

The last step is the problem. During failback you set the secondary read-only again, but it stays reachable: the share exists and answers. DFS checks whether a target can be reached, not whether it is writable. Clients therefore stay on the secondary and receive “access denied” errors when they save, while the primary is ready.

Two DFS settings decide how long this lasts:

  • Referral cache duration. Clients cache the list of folder targets. By default, namespace root referrals are cached for 300 seconds and folder referrals for 1,800 seconds (30 minutes).
  • Target failback. With target failback enabled on a folder, clients return to the preferred target when it is available again. It is disabled by default, and it only takes effect when the client next evaluates its referrals.

In practice, target failback alone does not bring every client back to the primary. That is why the temporary rename described below is the safety net.

The reliable way: make the secondary unreachable for a moment

What works in practice is to take the secondary target away for a short time:

  1. Finish replicating the changes back and make the primary writable again.
  2. Restore the DFS floating IPs and the original share names on the primary.
  3. Rename the share on the secondary temporarily. Clients that access it now get “not found”, which DFS treats as a failed target, and move to the primary.
  4. Keep the secondary share renamed for longer than the folder referral cache duration, so that clients with a cached referral also move.
  5. Rename the share on the secondary back, so it is ready for the next failover.

Renaming is enough. You don’t have to delete shares or change DNS, and the share configuration stays intact.

Checklist for DFS on Qumulo

  • Separate FQDN and floating IPs for DFS targets
  • Same share name and share permissions on both clusters
  • Folder targets ordered: primary first, secondary second
  • Folder referral cache duration known and documented
  • Failback runbook includes the temporary rename on the secondary
  • Failover and failback tested with a real Windows client

How yuzuy automates this

The yuzuy software runs as a virtual machine (VM) on your own hypervisor, controls the Qumulo clusters through their API and has a dedicated DFS Failover mode. In each failover relationship you select which floating IPs belong to DFS. On failover, yuzuy removes these IPs from the primary without moving them, sets the source directories read-only, renames the source shares and makes the replicated directories on the secondary writable. Every step is checked before the next one starts.

On failback, yuzuy restores the IPs and share names on the primary and renames the shares on the secondary for a configurable time. You set that time to match your DFS cache configuration, and yuzuy renames the shares back afterwards. You don’t need to remember the rename at 2 a.m., and you don’t need to know the cache duration by heart.

The same relationship can include IP Failover for NFS clients, so one click covers both protocols. After an unplanned failover, yuzuy checks every minute whether the primary is back and removes its IPs.

Failover / Failback · How to fail over Qumulo Continuous Replication for SMB and NFS clients

FAQ

Do SMB clients keep their connection during a DFS failover? They keep their connection to the DFS namespace and continue with the secondary folder target. Open files on the failed primary are lost, as with any server outage.

Why don’t clients return to the primary after failback? Because the secondary still answers. DFS moves clients only when a target is unreachable, not when it is read-only.

How long should the secondary share stay renamed during failback? Longer than the folder referral cache duration. If folders in your namespace use different cache durations, set the rename time in yuzuy to the longest one.

Sources

Keep reading

Talk to the people who build it.

Questions, a demo or a quote: we reply within one business day.