Quick Overview of the Supervisor Engine Upgrade Process
Before we dive into the upgrade process, let’s take a quick look at the steps to be followed. This will help understand the process and caveats of each step.
When upgrading a system with redundant Supervisor Engines the upgrade process has to be performed in a specific way as each Supervisor Engine is upgraded in turn.
Following is a brief overview of the upgrade steps:
- Load the new IOS image on to the Active Supervisor Engine (SE1)
- Copy IOS image to Standby Supervisor Engine (SE2)
- Configure Supervisor Engines to load the new image upon reboot
- Set Configuration-Register variable to ensure newest image is loaded upon bootup
- Force reload of Standby Supervisor Engine (SE2) & Switchover to Standby Supervisor Engine (SE2). This now becomes the new Active Supervisor Engine
- Force reload of previously Active Supervisor Engine (SE1).
Step 1: Loading the New IOS Image on to the Active Supervisor Engine (SE1)
The first step is to copy the new IOS image on to the active Supervisor Engine (SE1). For this, a TFTP server is required for the file transfer. Once the TFTP server is ready, we issue the necessary command to initiate the file transfer:
4507R# copy tftp bootflash:
Address or name of remote host []? 10.0.0.76
Source filename []? cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin
Destination filename [cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin]?
Accessing tftp://10.0.0.76/cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin...
Loading cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin from 10.0.0.36 (via Vlan2): !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[OK - 119576292 bytes]
119576292 bytes copied in 356.708 secs (335222 bytes/sec)
Note that the new IOS image is saved to the bootflash: file system. Cisco Supervisor Engines use the bootflash: file system rather than the flash: file system which most of us are used to.
If in doubt, simply make use of the show file system command that will reveal the file systems on your Catalyst switch:
4507R# show file system
File Systems:
Size(b) Free(b) Type Flags Prefixes
* 831541248 718979072 disk rw bootflash:
141433856 139310080 rom ro crashinfo:
16915456 16660992 disk rw kinfo:
- - disk rw slot0:
- - disk rw usb0:
- - opaque rw system:
- - opaque rw tmpsys:
524284 523248 flash rw cat4000_flash:
524284 510196 nvram rw slavenvram:
820875264 708313088 flash rw slavebootflash:
524284 523248 flash rw slavecat4000_flash:
- - flash rw slaveslot0:
- - flash rw slaveusb0:
139940864 137817088 opaque ro slavecrashinfo:
- - opaque rw slavercsf:
16915456 16660992 flash rw slavekinfo:
- - opaque rw null:
- - opaque ro tar:
- - network rw tftp:
- - opaque wo syslog:
524284 510196 nvram rw nvram:
- - network rw rcp:
- - network rw http:
- - network rw ftp:
- - opaque ro cns:
- - opaque rw revrcsf:
Step 2: Copy IOS Image to Standby Supervisor Engine (SE2)
Once the IOS image is loaded on to the active Supervisor Engine (SE1), it must be copied to the standby Supervisor Engine (SE2). For this, we use the copy bootflash: slavebootflash: command. Note that the slavebootflash: file system refers to the bootflash: of the standby Supervisor Engine, regardless of which physical engine is on standby mode.
4507R# copy bootflash: slavebootflash:
Source filename []? cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin
Destination filename [cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin]?
Copy in progress...CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
119576292 bytes copied in 99.404 secs (1202932 bytes/sec)
4507R#
To verify the image is correctly loaded on both Supervisor Engine bootflash systems, use the show bootflash & show slavebootflash commands:
4507R# show bootflash:
-#- --length-- ---------date/time--------- path
1 112337548 Feb 26 2013 08:44:27 +00:00 cat4500e-universal.SPA.03.03.00.SG.151-1.SG.bin
2 119576292 Mar 21 2013 03:07:21 +00:00 cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin
557035520 bytes available (232263680 bytes used)
Filesystem: bootflash
Mounted: Unknown
4507R# show slavebootflash:
-#- --length-- ---------date/time--------- path
1 112337548 Feb 26 2013 06:46:26 +00:00 cat4500e-universal.SPA.03.03.00.SG.151-1.SG.bin
2 119576292 Mar 21 2013 01:09:33 +00:00 cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin
546914304 bytes available (232263680 bytes used)
Note: Notice that the first IOS image listed is the previous version (03.03.00.SG.151-1.SG). We’ll need to keep this information in mind for our next step.
Step 3: Configure Supervisor Engines to Load the New Image Upon Reboot
In our next step, we configure the active Supervisor Engine to load the new IOS image when it reboots. This is easily done using the boot system flash command as shown below:
4507R(config)# boot system flash bootflash:cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin
It is highly likely the boot system flash bootflash command already exists in the system’s configuration for the previous IOS image, so we’ll need to remove the command from the configuration to ensure the newest IOS image we just uploaded (03.04.00.SG.151-2.SG) is the only one referenced:
4507R(config)# no boot system flash bootflash:cat4500e-universal.SPA.03.03.00.SG.151-1.SG.bin
Next, we must ensure the configuration is saved to the startup-configuration:
4507R# wr mem
Building configuration...
% VRF table-id 0 not activeCompressed configuration from 12839 bytes to 3791 bytes[OK]
*Mar 21 01:15:28.356: %C4K_REDUNDANCY-5-CONFIGSYNC: The private-config has been successfully synchronized to the standby supervisor
*Mar 21 01:15:29.098: %C4K_REDUNDANCY-5-CONFIGSYNC: The startup-config has been successfully synchronized to the standby supervisor
Readers wondering about the % VRF table-id 0 not activeCompressed message when saving the running-configuration should not be alarmed as this is a cosmetic bug and can be safely ignored. If no such message was reported when saving the configuration, it means that the IOS currently running does not have this cosmetic bug.
On another note, every time we save our configuration to startup-config the system will immediately synchronize the configuration changes to the standby supervisor. When this happens, we’ll receive a message, similar to the one show above, confirming the synchronization has been successful.
Step 4: Set Configuration-Register Variable to Ensure Newest Image is Loaded Upon Bootup
Cisco Supervisor Engines, by factory default, have their configuration register set to 0x2101. While this value is a combination of settings, we will focus on two specific values: 0x2101 & 0x2102. The value 0x2101 instructs the system to boot the first system image in the onboard flash memory (bootflash). This is usually the oldest image in the flash. The value of 0x2102 instructs the system to use the image specified in the BOOT environment variable, which is essentially whatever was specified in the previous step (No.3) using the boot system flash bootflash: command.
To view the environment variables on both Supervisor Engines, use the show bootvar command:
4507R# show bootvar
BOOT variable = bootflash:cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin,1;
CONFIG_FILE variable does not exist
BOOTLDR variable does not exist
Configuration register is 0x2101
Standby BOOT variable = bootflash:cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin,1;
Standby CONFIG_FILE variable does not exist
Standby BOOTLDR variable does not exist
Standby Configuration register is 0x2101
Notice how the newly uploaded IOS image cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin is set for the BOOT variable, however the Configuration register value is 0x2101, which means it will force the Supervisor Engine to boot the first image it is going to find on the bootflash. Which image is that? Let’s refresh our memory:
4507R# show bootflash: all
-#- --length-- ---------date/time--------- path
1 112337548 Feb 26 2013 08:44:27 +00:00 cat4500e-universal.SPA.03.03.00.SG.151-1.SG.bin
2 119576292 Mar 21 2013 03:07:21 +00:00 cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin
4507R# show slavebootflash:
-#- --length-- ---------date/time--------- path
1 112337548 Feb 26 2013 06:46:26 +00:00 cat4500e-universal.SPA.03.03.00.SG.151-1.SG.bin
2 119576292 Mar 21 2013 01:09:33 +00:00 cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin
As we can see, despite the boot system flash command being correctly set to load the second IOS image, the configuration register setting of 0x2101 will force the Supervisor Engine to boot the first image found, that is: cat4500e-universal.SPA.03.03.00.SG.151-1.SG.bin.
To overcome this problem, we have two options:
1) Delete all older IOS images from the Bootflash
2) Set the configuration register to 0x2102.
We decided to set the configuration register just to be on the safe side:
4507R(config)# config-register 0x2102
*Mar 21 01:19:55.542: %C4K_REDUNDANCY-5-CONFIGSYNC: The config-reg has been successfully synchronized to the standby supervisor
4507R# wr mem
Building configuration...
% VRF table-id 0 not activeCompressed configuration from 12849 bytes to 3791 bytes[OK]
*Mar 21 01:20:08.352: %C4K_REDUNDANCY-5-CONFIGSYNC: The private-config has been successfully synchronized to the standby supervisor
*Mar 21 01:20:09.091: %C4K_REDUNDANCY-5-CONFIGSYNC: The startup-config has been successfully synchronized to the standby supervisor
Take note that the system is confirming that both configuration register and startup-config have been successfully synchronized with the standby supervisor.
…and there’s that cosmetic bug again :)
Step 5: Force Reload of Standby Supervisor (SE2) and Switchover to Standby Supervisor Engine (SE2)
At this point we are ready to force the standby Supervisor Engine(SE2) to reload. Once this happens, the Supervisor Engine (SE2) will load the new IOS. Once we confirm the new IOS is loaded, we can then make the standby Supervisor Engine (SE2) the active Supervisor Engine.
To force the reload of the standby Supervisor Engine (SE2), use the redundancy reload peer command:
4507R# redundancy reload peer
Reload peer [confirm]
4507R#
*Mar 21 01:21:02.318: %RF-5-RF_RELOAD: Peer reload. Reason: Unknown Reason
*Mar 21 01:21:05.314: %C4K_REDUNDANCY-3-COMMUNICATION: Communication with the peer Supervisor has been lost
*Mar 21 01:21:05.327: %C4K_REDUNDANCY-3-SIMPLEX_MODE: The peer Supervisor has been lost
The second (standby) Supervisor Engine is now restarting. This process will take a couple of minutes and will have no negative impact on the 4507R switch.
Once the IOS has loaded and the restart process is complete we will receive a message similar to the following:
*Mar 21 01:24:54.312: %C4K_REDUNDANCY-6-DUPLEX_MODE: The peer Supervisor has been detected
*Mar 21 01:25:36.066: %C4K_IOSMODPORTMAN-6-MODULEONLINE: Module 4 (WS-X45-SUP7L-E S/N: CAT1714L4T4 Hw: 1.1) is online
*Mar 21 01:25:36.094: %C4K_REDUNDANCY-6-MODE: ACTIVE supervisor initializing for sso mode
*Mar 21 01:25:36.344: %C4K_REDUNDANCY-3-COMMUNICATION: Communication with the peer Supervisor has been established
*Mar 21 01:25:37.098: %C4K_REDUNDANCY-6-MODE: ACTIVE supervisor initializing for sso mode
The above messages indicate that the standby Supervisor Engine has successfully completed its reboot and is fully synchronized with the active Supervisor Engine. The system shows the product ID (WS-X45-SUP7L-E), serial number and hardware revision of the Supervisor Engine that has just established communication with the active Supervisor Engine.
To verify that the standby Supervisor Engine is running the latest and greatest IOS image we just loaded, use the show module command:
4507R# show module
Chassis Type : WS-C4507R+E
Power consumed by backplane : 40 Watts
Mod Ports Card Type Model Serial No.
---+-----+--------------------------------------+------------------+-----------
1 48 10/100/1000BaseT Premium POE E Series WS-X4748-RJ45V+E CAT1754L4C7
2 48 10/100/1000BaseT Premium POE E Series WS-X4648-RJ45V+E JAE171515SY
3 6 Sup 7L-E 10GE (SFP+), 1000BaseX (SFP) WS-X45-SUP7L-E CAT1712L2X5
4 6 Sup 7L-E 10GE (SFP+), 1000BaseX (SFP) WS-X45-SUP7L-E CAT1714L4T4
5 48 10/100/1000BaseT Premium POE E Series WS-X4648-RJ45V+E JAE170525D3
M MAC addresses Hw Fw Sw Status
--+--------------------------------+---+------------+----------------+---------
1 4c4e.352e.a2cc to 4c4e.352e.a2fb 1.3 Ok
2 0006.f6e9.7520 to 0006.f6e9.754f 3.1 Ok
3 0006.f620.2e80 to 0006.f620.2e85 1.1 15.0(1r)SG3 03.03.00.SG Ok
4 0006.f620.2e86 to 0006.f620.2e8b 1.1 15.0(1r)SG3 03.04.00.SG Ok
5 0006.f673.51d4 to 0006.f673.5203 3.1 Ok
Mod Redundancy role Operating mode Redundancy status
----+-------------------+-------------------+----------------------------------
3 Active Supervisor SSO Active
4 Standby Supervisor SSO Standby hot
The show module command provides a generous amount of information including all line cards installed, Supervisor Engine models, IOS software versions, serial numbers, operational status and redundancy mode (SSO by default).
We’ve highlighted our standby Supervisor Engine which happens to be in slot No.4. Notice the software image the system is reporting for both Supervisor Engines. Our current active Supervisor Engine is running version 03.03.00.SG, while our standby Supervisor Engine is running version 03.04.00.SG – the new IOS loaded!
We’ve now confirmed that the standby Supervisor Engine has loaded the new IOS and is fully operational, waiting eagerly to manage our 4507R switch and all its data!!
Step 6: Force Reload of Previously Active Supervisor Engine (SE1)
On our final step, we will need to force the reload of the active Supervisor Engine (SE1) so it too can boot using the new IOS image. Forcing a Supervisor Engine switch over is an experience when you consider what’s really happening inside the switch at that moment. During the process of the switchover we don’t get LEDs lighting up, flashing like crazy. The SUP ACTIVE LED will simply switch off from the current active Supervisor Engine and switch on on our previously standby Supervisor Engine, indicating it is now the new active engine.
To initiate the Supervisor Engine switchover, use the redundancy force-switchover command as shown below:
4507R# redundancy force-switchover
This will reload the active unit and force switchover to standby[confirm]
Preparing for switchover..
*Mar 21 01:27:57.007: %SYS-5-SWITCHOVER: Switchover requested by Virtual Exec. Reason: Stateful Switchover.
Once the command is entered and we’ve confirmed by hitting ENTER, we will lose our telnet session to the 4507R. This is normal expected behaviour – do not be alarmed! Apart from the switch cutting our telnet session, users will not notice any service disruption – the switchover will be completely transparent to them no matter the network load during the switchover.
To reconnect to the switch, simply telnet back into the same IP address. If connected via console cable it will be necessary to connect it to the new active supervisor engine in order to continue controlling the switch.
Engineers who would like to monitor, via telnet or direct console cable connection (to the new active engine), the reload progress of the Supervisor Engine can use the show module command. By typing the command we will see the 4507R identifying a Supervisor Engine in slot 3 (that’s SE1 that is reloading), however, further down we will see that no information about the engine’s MAC address or IOS software is provided due to the fact that it has not fully booted into its new IOS. In addition the redundancy status of the first engine is Disabled – an expected result since the Supervisor Engine (SE1) has not booted yet.
4507R# show module
Chassis Type : WS-C4507R+E
Power consumed by backplane : 40 Watts
Mod Ports Card Type Model Serial No.
---+-----+--------------------------------------+------------------+-----------
1 48 10/100/1000BaseT Premium POE E Series WS-X4748-RJ45V+E CAT1754L4C7
2 48 10/100/1000BaseT Premium POE E Series WS-X4648-RJ45V+E JAE171515SY
3 Supervisor
4 6 Sup 7L-E 10GE (SFP+), 1000BaseX (SFP) WS-X45-SUP7L-E CAT1714L4T4
5 48 10/100/1000BaseT Premium POE E Series WS-X4648-RJ45V+E JAE170525D3
M MAC addresses Hw Fw Sw Status
--+--------------------------------+---+------------+----------------+---------
1 4c4e.352e.a2cc to 4c4e.352e.a2fb 1.3 Ok
2 0006.f6e9.7520 to 0006.f6e9.754f 3.1 Ok
3 Unknown Unknown Unknown Other
4 0006.f620.2e86 to 0006.f620.2e8b 1.1 15.0(1r)SG3 03.04.00.SG Ok
5 0006.f673.51d4 to 0006.f673.5203 3.1 Ok
Mod Redundancy role Operating mode Redundancy status
----+-------------------+-------------------+----------------------------------
3 Standby Supervisor SSO Disabled
4 Active Supervisor SSO Active
At this point, we can continue to issue the show module command and monitor the output changes. At some point the system will show the Supervisor Engine (SE1) loaded with the new IOS, however, the redundancy status will go through the following phases until it is ready (Standby hot):
Standby Supervisor Redundancy Status Cycle:
- Disabled
- In progress to Standby cold
- Standby cold
- In progress to Issu negotiation la
- In progress to Standby config
- In progress to Standby bulk
- Standby hot
When the Supervisor Engine reaches Standby hot status, it is ready to take over in the event the active Supervisor Engine fails.
Following is the expected output when the Supervisor Engine IOS is loaded and fully synced with the active Supervisor Engine:
4507R# show module
Chassis Type : WS-C4507R+E
Power consumed by backplane : 40 Watts
Mod Ports Card Type Model Serial No.
---+-----+--------------------------------------+------------------+-----------
1 48 10/100/1000BaseT Premium POE E Series WS-X4748-RJ45V+E CAT1754L4C7
2 48 10/100/1000BaseT Premium POE E Series WS-X4648-RJ45V+E JAE171515SY
3 6 Sup 7L-E 10GE (SFP+), 1000BaseX (SFP) WS-X45-SUP7L-E CAT1712L2X5
4 6 Sup 7L-E 10GE (SFP+), 1000BaseX (SFP) WS-X45-SUP7L-E CAT1714L4T4
5 48 10/100/1000BaseT Premium POE E Series WS-X4648-RJ45V+E JAE170525D3
M MAC addresses Hw Fw Sw Status
--+--------------------------------+---+------------+----------------+---------
1 4c4e.352e.a2cc to 4c4e.352e.a2fb 1.3 Ok
2 0006.f6e9.7520 to 0006.f6e9.754f 3.1 Ok
3 0006.f620.2e80 to 0006.f620.2e85 1.1 15.0(1r)SG3 03.04.00.SG Ok
4 0006.f620.2e86 to 0006.f620.2e8b 1.1 15.0(1r)SG3 03.04.00.SG Ok
5 0006.f673.51d4 to 0006.f673.5203 3.1 Ok
Mod Redundancy role Operating mode Redundancy status
----+-------------------+-------------------+----------------------------------
3 Standby Supervisor SSO Standby hot
4 Active Supervisor SSO Active
