Fortigate FGCP vs FGSP for High Availability

Fortigate Clustering Protocol (FGCP)

I have been working on Fortinet Fortigates for over two years now. Most customer that do High Availablilty (HA) use the standard Active/Passive which used Fortigate Clustering Protocol (FGCP). This configuration requires Layer 2 adjacvency to the other Fortigate unit. This configuration contains a single master at any given time and at least one subordinate or ’slave’ unit. The master controls the HA in this cluster based on the mode it is in. Active/Passive (AP) or Active/Active (AA). These firewalls in the cluster communicate with each other via a heartbeat interface and exchange information such as ’state’. In the event that the master unit becomes ‘unhealthy’, meaning that a monitored interface is down or it otherwise becomes off-line, the ’slave’ unit promotes itself to the new master. Depending on specific configuration, when the original master returns to its healthy state, the role of Master will either remain on the original slave device or the old master, becomes the new master again.

NOTE: If you have two ‘Masters’ in the cluster, you have done something wrong. This is known as split-brain.

With FGCP, the ‘set overide disable’ is the default. The cluster reacts based on network conditions. With ’set override enable’ if the ‘original master’ will continue to attempt to become ‘master’. This COULD cause unnecessary flapping which is not desrable. If you want the same Fortigate unit to become master everytime, then set it to ‘enable’ however normally, I see that ‘disable’ works fine.

In FGCP, the Fortigate will use a virtual MAC address generated by the Fortigate when HA is configured. Only one IP address per interface is required. The second unit (slave) does not respond to packets except for the heat beat interface(s). Once you turn on HA, you will temporarily lose connectivity to the device while the MAC address is enabled.

NOTE: If you have two sets of HA’d devices that are L2 adjacent, make sure you change the ‘group id’ if not, you will experience some strange behavior because the MAC address is generated based on the group id.

The FGCP Configuration on the Master device

config system ha
set group-name “HAGroup”
set mode a-p
set password Fortinet
set hbdev “ha” 50 “port14” 0
set session-pickup enable
set override disable
set priority 180
end

The FGCP Configuration on the Slave device

config system ha
set group-name “HAGroup”
set mode a-p
set password Fortinet
set hbdev “ha” 50 “port14” 0
set session-pickup enable
set override disable
set priority 125
end

Notice the ‘set priority’ value. The higher the number, the more chances it has in becoming the master. This is not guaranteed as the election process can be priority, uptime, even serial number.

FortiGate Session Life Support Protocol (FGSP)

FGSP has been around since FortiOS v5.0. Before then, it was known as ’standalone session sync’. The previous version had limitations such as: Only TCP support, and no ECMP. The redesign now supports TCP, UDP, and ICMP and NAT. In addition to the aforementioned, it also supports config sync. The use of traffic load-balancers to share load across two FortiGates is possible with FGSP. FGSP is perfect for environments using equal cost load balancing of links between routers/L3 switches. Also called equal-cost multi-path (ECMP) routing or multipath routing.

In FGSP both FortiGates are in an active state and will process packets, which is the desired function when using ECMP routing in a network. Unlike the FGCP flavor, the Fortigates in FGSP have unique IP addresses. There is no need for virtual MAC address since each device communicates via its own IP address.

* FortiGates running FGSP act as independent devices when compared to FGCP.
* Interface IP addresses must be unique to each interface, there are no shared IP addresses, and there is no clustering.
* FortiGate configurations are not automatically synchronized. It must first be enabled.
* The same ForitGate models should be used for both peers to avoid uneven performance.
* The names of the interfaces, including VLAN interfaces, aggregate interfaces, etc… must be the same onboth peers.
* It is possible to synchronize sessions from only specific VDOMs.
* Session synchronizations filters can be enabled to only synchronize specific traffic.
* You cannot run FGSP and FGCP at the same time on the same FortiGates.
* VRPP is supported in FGSP.
* Performance limitations should be considered in case of a failover scenario. One device must be able to handle the load

With FGSP, you can see a SYN packet processed via FG1, then the SYN/ACK on FG2, followed by ACK on FG1. This is perfectly acceptable in FGSP.

config system session-sync
edit 1
set peerip 10.10.10.1
set syncvd “root”

config system session-sync
edit 1
set peerip 10.10.10.2
set syncvd “root

 

Troubleshooting steps

 

get system ha status – This commands give you information about your HA cluster.

HA Health Status: OK
Model: FortiGate-60E
Mode: HA A-P
Group: 200
Debug: 0
Cluster Uptime: 239 days 09:08:45
Cluster state change time: 2019-02-01 22:53:09

As you can see with this output, you can see group id, uptime, state change time, as well as other output like why the master is the actual master, etc.

diagnose sys ha checksum cluster – This command will show the checksum of the cluster members.

================== FGT6<REMOVED> ==================

is_manage_master()=1, is_root_master()=1
debugzone
global: 68 67 61 21 de d9 a4 9b 0e a0 6b 31 1e ce 83 41
root: 28 ee c9 27 d2 50 94 f3 06 f6 20 3d 1d 6a 7b 9f
all: 2b 34 8d 9b 23 36 b8 b4 7b fa 32 b4 03 d9 42 67

checksum
global: 68 67 61 21 de d9 a4 9b 0e a0 6b 31 1e ce 83 41
root: 28 ee c9 27 d2 50 94 f3 06 f6 20 3d 1d 6a 7b 9f
all: 2b 34 8d 9b 23 36 b8 b4 7b fa 32 b4 03 d9 42 67

================== FGT6<REMOVED> ==================

is_manage_master()=0, is_root_master()=0
debugzone
global: 68 67 61 21 de d9 a4 9b 0e a0 6b 31 1e ce 83 41
root: 28 ee c9 27 d2 50 94 f3 06 f6 20 3d 1d 6a 7b 9f
all: 2b 34 8d 9b 23 36 b8 b4 7b fa 32 b4 03 d9 42 67

checksum
global: 68 67 61 21 de d9 a4 9b 0e a0 6b 31 1e ce 83 41
root: 28 ee c9 27 d2 50 94 f3 06 f6 20 3d 1d 6a 7b 9f
all: 2b 34 8d 9b 23 36 b8 b4 7b fa 32 b4 03 d9 42 67

Notice how the last numbers line up. This is a good thing. If you do not see these numbers match, you can run the following command:

diagnose sys ha checksum show %section%

Where %section% is ‘global’ or ‘root’