
Far Sync, Redo routes and Priorities with alternates
Far Sync is a great technology. When implementing a Disaster Recovery (DR) scenario, the first remark is usually “We want zero Data Loss”. This is exactly what Data Guard is meant for in Synchronous mode. Together with that usually comes “But our second site is really far away”. Those two requirements, don’t mix very well, that is where Far Sync comes into play.
Recently I received a question: “but what happens if the network on the Far Sync location is unstable”?
Well, we can solve that by adding an alternate Far Sync instance and adding Redo Routes and, as of 12.2, priorities (and eventually groups).
So let me illustrate this with an example. I won’t dig into the how-to-set-this-up as the Oracle Documentation contains great information on how to do this correctly.
I would definitely recommend you to use the Data Guard broker, as this massively eases the administration and the setup of this installation.
My configuration looks like this

And in the broker it looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
DGMGRL> show configuration lag Configuration - dgdemo Protection Mode: MaxAvailability Members: dgdemovm1 - Primary database dgdemovm1fs1 - Far sync instance Transport Lag: 0 seconds (computed 1 second ago) dgdemovm2 - (*) Physical standby database Transport Lag: 0 seconds (computed 1 second ago) Apply Lag: 0 seconds (computed 1 second ago) Members Not Receiving Redo: dgdemovm2fs2 - Far sync instance dgdemovm1fs1a - Far sync instance (alternate of dgdemovm1fs1) dgdemovm2fs2a - Far sync instance Fast-Start Failover: Enabled in Zero Data Loss Mode Configuration Status: SUCCESS (status updated 56 seconds ago) DGMGRL> |
So primary sends redo to FS1 and that one sends it to the standby.
In this case following redo routes are active
1 2 3 4 5 6 7 |
DGMGRL> show database dgdemovm1 redoroutes RedoRoutes = '(LOCAL: (dgdemovm1fs1 SYNC PRIORITY=1, dgdemovm1fs1a SYNC PRIORITY=2, dgdemovm2 ASYNC PRIORITY=3) ) ' DGMGRL> show far_sync dgdemovm1fs1 redoroutes RedoRoutes = '(dgdemovm1:dgdemovm2 async)' DGMGRL> show database dgdemovm2 redoroutes RedoRoutes = '(LOCAL: (dgdemovm2fs2 SYNC PRIORITY=1, dgdemovm2fs2a SYNC PRIORITY=2, dgdemovm1 ASYNC PRIORITY=3) ) ' DGMGRL> |
Then I kill the dgdemovm1fs1 far_sync and we expect that dgdemovm1fs1a takes over, and end up with following situation:

So let’s verify.
1 2 3 4 5 6 7 |
[oracle@oelvm1fs1 ~]$ ps -ef |grep -i pmon oracle 5368 1 0 07:41 ? 00:00:00 ora_pmon_dgdemovm1fs1 oracle 12928 10819 0 10:05 pts/5 00:00:00 grep --color=auto -i pmon [oracle@oelvm1fs1 ~]$ kill 5368 [oracle@oelvm1fs1 ~]$ ps -ef |grep -i pmon oracle 12936 10819 0 10:05 pts/5 00:00:00 grep --color=auto -i pmon [oracle@oelvm1fs1 ~]$ |
So it’s dead now.
In the alertlog of the FS1
1 2 3 4 |
... 2020-02-24T10:06:00.667721-05:00 Instance terminated by DMON, pid = 5423 ... |
So this one is definitely dead.
When we now check the broker lag, because that refreshes on execution of the command:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
DGMGRL> show configuration lag Configuration - dgdemo Protection Mode: MaxAvailability Members: dgdemovm1 - Primary database dgdemovm1fs1a - Far sync instance (alternate of dgdemovm1fs1) Transport Lag: 0 seconds (computed 1 second ago) dgdemovm2 - (*) Physical standby database Transport Lag: 0 seconds (computed 1 second ago) Apply Lag: 0 seconds (computed 1 second ago) Members Not Receiving Redo: dgdemovm1fs1 - Far sync instance Warning: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor dgdemovm2fs2 - Far sync instance dgdemovm2fs2a - Far sync instance Fast-Start Failover: Enabled in Zero Data Loss Mode Configuration Status: WARNING (status updated 61 second ago) DGMGRL> |
We see that fs1a took over as expected.
So now we will bring back the fs1 instance:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
[oracle@oelvm1fs1 ~]$ sqlplus / as sysdba SQL*Plus: Release 19.0.0.0.0 - Production on Mon Feb 24 10:10:23 2020 Version 19.3.0.0.0 Copyright (c) 1982, 2019, Oracle. All rights reserved. Connected to an idle instance. SQL> startup mount ORACLE instance started. Total System Global Area 1073738488 bytes Fixed Size 9143032 bytes Variable Size 276824064 bytes Database Buffers 784334848 bytes Redo Buffers 3436544 bytes Database mounted. SQL> exit Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.3.0.0.0 [oracle@oelvm1fs1 ~]$ |
And we now expect to be in this situation again

Does it do that?
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
DGMGRL> show configuration lag Configuration - dgdemo Protection Mode: MaxAvailability Members: dgdemovm1 - Primary database dgdemovm1fs1 - Far sync instance Transport Lag: 0 seconds (computed 0 seconds ago) dgdemovm2 - (*) Physical standby database Transport Lag: 0 seconds (computed 0 seconds ago) Apply Lag: 0 seconds (computed 0 seconds ago) Members Not Receiving Redo: dgdemovm2fs2 - Far sync instance dgdemovm1fs1a - Far sync instance (alternate of dgdemovm1fs1) dgdemovm2fs2a - Far sync instance Fast-Start Failover: Enabled in Zero Data Loss Mode Configuration Status: SUCCESS (status updated 25 seconds ago) DGMGRL> |
It does!
This is also one of the reasons I also use Virtual box, so I can simulate things easily. I loaded my system heavily so it gives time to check logs and check broker status, and in the meantime DURING the FS1 Far sync came back, you can see that temporarily they are there both:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
DGMGRL> show configuration lag Configuration - dgdemo Protection Mode: MaxAvailability Members: dgdemovm1 - Primary database dgdemovm1fs1 - Far sync instance Warning: ORA-16856: transport lag could not be determined Transport Lag: (unknown) dgdemovm2 - (*) Physical standby database Transport Lag: 0 seconds (computed 1 second ago) Apply Lag: 0 seconds (computed 1 second ago) dgdemovm1fs1a - Far sync instance (alternate of dgdemovm1fs1) Transport Lag: 0 seconds (computed 1 second ago) Members Not Receiving Redo: dgdemovm2fs2 - Far sync instance dgdemovm2fs2a - Far sync instance Fast-Start Failover: Enabled in Zero Data Loss Mode Configuration Status: WARNING (status updated 56 seconds ago) DGMGRL> |
You see? FS1 has a higher priority than FS1A so the moment that one comes back it gets our preference. However, before it is fully functional, we still trust FS1A to do the job before we fully switch to FS1 again.
Ok let’s play with priorities here.
These are the Redo routes what we got in place remember:
1 2 3 4 5 6 7 8 9 |
DGMGRL> show database dgdemovm1 redoroutes RedoRoutes = '(LOCAL: (dgdemovm1fs1 SYNC PRIORITY=1, dgdemovm1fs1a SYNC PRIORITY=2, dgdemovm2 ASYNC PRIORITY=3) ) ' DGMGRL> show database dgdemovm2 redoroutes RedoRoutes = '(LOCAL: (dgdemovm2fs2 SYNC PRIORITY=1, dgdemovm2fs2a SYNC PRIORITY=2, dgdemovm1 ASYNC PRIORITY=3) ) ' DGMGRL> show far_sync dgdemovm1fs1 redoroutes RedoRoutes = '(dgdemovm1:dgdemovm2 async)' DGMGRL> show far_sync dgdemovm1fs1a redoroutes RedoRoutes = '(dgdemovm1:dgdemovm2 async)' DGMGRL> |
Before we can play with RedoRoutes and priorities, it is necessary to disable fast start failover (FSFO). It is not relevant for this blog post to have it enabled anyway.
1 2 3 |
DGMGRL> disable fast_start failover; Disabled. DGMGRL> |
There we go.
We can now update the priorities.
1 2 3 4 5 6 |
DGMGRL> edit database dgdemovm1 set property redoroutes = '(LOCAL: (dgdemovm1fs1 SYNC PRIORITY=1, dgdemovm1fs1a SYNC PRIORITY=1, dgdemovm2 ASYNC PRIORITY=2) ) '; Property "redoroutes" updated DGMGRL> DGMGRL> edit database dgdemovm2 set property redoroutes = '(LOCAL: (dgdemovm2fs2 SYNC PRIORITY=1, dgdemovm2fs2a SYNC PRIORITY=1,dgdemovm1 ASYNC PRIORITY=2) ) '; Property "redoroutes" updated DGMGRL> |
Trust but verify
1 2 3 4 5 |
DGMGRL> show database dgdemovm1 redoroutes RedoRoutes = '(LOCAL: (dgdemovm1fs1 SYNC PRIORITY=1, dgdemovm1fs1a SYNC PRIORITY=1, dgdemovm2 ASYNC PRIORITY=2) ) ' DGMGRL> show database dgdemovm2 redoroutes RedoRoutes = '(LOCAL: (dgdemovm2fs2 SYNC PRIORITY=1, dgdemovm2fs2a SYNC PRIORITY=1,dgdemovm1 ASYNC PRIORITY=2) ) ' DGMGRL> |
So we are good to go. What we did here is that we put the same priority for FS1 and FS1A. This means, they should be treated equally and not go back to the other FS-instance the moment the previous one comes back.
So let’s show this.
We will kill pmon from FS1 again to get from this

to this again

There goes poor pmon again
1 2 3 4 5 6 7 |
[oracle@oelvm1fs1 ~]$ ps -ef |grep -i pmon oracle 13170 1 0 10:10 ? 00:00:00 ora_pmon_dgdemovm1fs1 oracle 14151 10819 0 10:28 pts/5 00:00:00 grep --color=auto -i pmon [oracle@oelvm1fs1 ~]$ kill 13170 [oracle@oelvm1fs1 ~]$ ps -ef |grep -i pmon oracle 14164 10819 0 10:28 pts/5 00:00:00 grep --color=auto -i pmon [oracle@oelvm1fs1 ~]$ |
As we saw before, the alternate takes over as expected
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
DGMGRL> show configuration lag Configuration - dgdemo Protection Mode: MaxAvailability Members: dgdemovm1 - Primary database dgdemovm1fs1a - Far sync instance (alternate of dgdemovm1fs1) Transport Lag: 0 seconds (computed 1 second ago) dgdemovm2 - Physical standby database Transport Lag: 0 seconds (computed 1 second ago) Apply Lag: 0 seconds (computed 1 second ago) Members Not Receiving Redo: dgdemovm1fs1 - Far sync instance Warning: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor dgdemovm2fs2 - Far sync instance dgdemovm2fs2a - Far sync instance Fast-Start Failover: Disabled Configuration Status: WARNING (status updated 98 seconds ago) DGMGRL> |
Notice the ORA-12514 Warning. It is true! So even we know something is wrong, just by looking at the broker.
So what happens now when we bring it back?What you think? It has the same priority right?
1 2 3 4 5 6 7 8 9 10 |
SQL> startup mount ORACLE instance started. Total System Global Area 1073738488 bytes Fixed Size 9143032 bytes Variable Size 276824064 bytes Database Buffers 784334848 bytes Redo Buffers 3436544 bytes Database mounted. SQL> |
Then we verify in the broker
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
DGMGRL> show configuration lag Configuration - dgdemo Protection Mode: MaxAvailability Members: dgdemovm1 - Primary database dgdemovm1fs1a - Far sync instance (alternate of dgdemovm1fs1) Transport Lag: 0 seconds (computed 0 seconds ago) dgdemovm2 - Physical standby database Transport Lag: 0 seconds (computed 0 seconds ago) Apply Lag: 0 seconds (computed 0 seconds ago) Members Not Receiving Redo: dgdemovm1fs1 - Far sync instance dgdemovm2fs2 - Far sync instance dgdemovm2fs2a - Far sync instance Fast-Start Failover: Disabled Configuration Status: SUCCESS (status updated 21 seconds ago) DGMGRL> |
It clears the warning, but it does NOT fail back to FS1. They have the same priority, so for the redo stream all is ok to use this route.
The moment we kill fs1a

We expect to fall back to the original route.
So, lets do this
1 2 3 4 5 6 7 |
[oracle@oelvm1fs1a ~]$ ps -ef |grep -i pmon oracle 5485 1 0 07:46 ? 00:00:00 ora_pmon_dgdemovm1fs1a oracle 14176 10637 0 10:35 pts/5 00:00:00 grep --color=auto -i pmon [oracle@oelvm1fs1a ~]$ kill 5485 [oracle@oelvm1fs1a ~]$ ps -ef |grep -i pmon oracle 14183 10637 0 10:35 pts/5 00:00:00 grep --color=auto -i pmon [oracle@oelvm1fs1a ~]$ |
Then we see that fs1 takes over again
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
DGMGRL> show configuration lag Configuration - dgdemo Protection Mode: MaxAvailability Members: dgdemovm1 - Primary database dgdemovm1fs1 - Far sync instance Transport Lag: 0 seconds (computed 0 seconds ago) dgdemovm2 - Physical standby database Transport Lag: 0 seconds (computed 0 seconds ago) Apply Lag: 0 seconds (computed 0 seconds ago) Members Not Receiving Redo: dgdemovm2fs2 - Far sync instance dgdemovm1fs1a - Far sync instance (alternate of dgdemovm1fs1) Warning: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor dgdemovm2fs2a - Far sync instance Fast-Start Failover: Disabled Configuration Status: WARNING (status updated 78 seconds ago) DGMGRL> |
With the same Warning, but this time for FS1A.
When we then bring back FS1A
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
[oracle@oelvm1fs1a ~]$ sqlplus / as sysdba SQL*Plus: Release 19.0.0.0.0 - Production on Mon Feb 24 10:37:54 2020 Version 19.3.0.0.0 Copyright (c) 1982, 2019, Oracle. All rights reserved. Connected to an idle instance. SQL> startup mount ORACLE instance started. Total System Global Area 1073738488 bytes Fixed Size 9143032 bytes Variable Size 276824064 bytes Database Buffers 784334848 bytes Redo Buffers 3436544 bytes Database mounted. SQL> Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.3.0.0.0 [oracle@oelvm1fs1a ~]$ |
And we check in the broker and we expect no changes apart of getting rid of ORA-12514, right?
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
DGMGRL> show configuration lag Configuration - dgdemo Protection Mode: MaxAvailability Members: dgdemovm1 - Primary database dgdemovm1fs1 - Far sync instance Transport Lag: 0 seconds (computed 1 second ago) dgdemovm2 - Physical standby database Transport Lag: 0 seconds (computed 0 seconds ago) Apply Lag: 0 seconds (computed 0 seconds ago) Members Not Receiving Redo: dgdemovm2fs2 - Far sync instance dgdemovm1fs1a - Far sync instance (alternate of dgdemovm1fs1) dgdemovm2fs2a - Far sync instance Fast-Start Failover: Disabled Configuration Status: SUCCESS (status updated 46 seconds ago) DGMGRL> |
And that is basically what it does!
As you see, I did nothing manually and basically it comes down to the question “against what do you want to be protected”?
But an unstable Far Sync can be handled by setting equal priorities. Real High Availability, can be achieved by prioritising the one over the other.
This is how we can provide Zero Data Loss, at ANY distance.
As always, questions, remarks?
find me on twitter @vanpupi