Our Operations Team where faced with an unfamiliar situation a few weeks back where we needed to re-ip an EMC VNX5300 that had it’s warranty extended and was relocated to a new Availability Zone. Generally once warranty terms and finance expire old SANs like this get returned to sender to be replaced by a newer model or new storage of some description.
As a change of topic on this blog I thought it would be good to document the process as there isn’t a lot of info out there on what needs to be done apart from the official EMC VNX Documentation locked away behind the EMC Client Portal.
In the example below we are initially concerned with the Control Station and Storage Processors of the VNX and we will change their IPs via SSH and not via Serial as suggested in the EMC guides. I would still suggest doing this via Serial if you have easy physical access to the SAN however in my case it was at the other side of the country. Networking wise I had a dual homed VM which had Network Adapters in each VLAN/PortGroup to allow the change from old to new.
Component | IP Address Values | Existing IP | New IP |
Control Station 0 | IPv4 IP Address: IPv4 Subnet: IP4v Gateway: |
10.1.4.5 255.255.255.0 10.1.4.1 |
10.2.5.50 255.255.255.0 10.2.5.1 |
Control Station 1 | IPv4 IP Address: IPv4 Subnet: IP4v Gateway: |
10.1.4.6 255.255.255.0 10.1.4.1 |
10.2.5.51 255.255.255.0 10.2.5.1 |
Storage Processor A | IPv4 IP Address: IPv4 Subnet: |
10.1.4.6 255.255.255.0 |
10.2.5.6 255.255.255.0 |
Storage Processor B | IPv4 IP Address: IPv4 Subnet: |
10.1.4.7 255.255.255.0 |
10.2.5.7 255.255.255.0 |
Change IP Address of Primary Control Station:
- SSH to CS0
- Change CS0 IP to the new value
1
2
3
4
5
|
EMC VNX Control Station Linux release 3.0 (NAS 7.1.79)
root@10.1.4.6‘s password:
Last login: Tue Jul 28 10:47:30 2015 from 10.42.5.254
EMC VNX Control Station Linux Thu Aug 25 10:11:14 EDT 2011
[root@STRG–001–CS0 ~]# /nas/bin/nas_cs -set -ip4address 10.2.5.50 -ip4netmask 255.255.255.0 -ip4gateway 10.2.5.1
|
Change IP Address of Secondary Control Station:
- Failover CS0 to force CS1 as Primary
1
2
3
4
5
6
|
[root@STRG–001–CS0 ~]# /nas/sbin/cs_standby -failover
The system will reboot, do you wish to continue? [yes or no]: yes
Failing over from Primary Control Station...
Broadcast message from root (pts/0) (Tue Jul 28 13:19:13 2015):
The system is going down for reboot NOW!
|
- This Operation can take about 15 minutes to complete and can be monitored on CS1 by running the command below
1
2
3
4
5
6
7
8
|
[root@STRG–001–CS1 ~]# watch /nasmcd/sbin/getreason
Every 2.0s: /nasmcd/sbin/getreason Tue Jul 28 14:41:11 2015
10 – slot_0 primary control station
11 – slot_1 secondary control station
5 – slot_2 contacted
5 – slot_3 contacted
|
- During the failover you will see the status of slot_0 and slot_1 swap over while slot_0 goes offline
- Once slot_0 comes back online SSH into the new IP to confirm access of CS0
- SSH to CS1
- Change CS1 IP to the new value
1
|
[root@STRG–001–CS1 ~]# /nas/bin/nas_cs -set -ip4address 10.2.5.51 -ip4netmask 255.255.255.0 -ip4gateway 10.2.5.1
|
- Failover CS1 to force CS0 as Primary
1
2
3
4
5
6
|
[root@STRG–001–CS1 ~]# /nas/sbin/cs_standby –failover
The system will reboot, do you wish to continue? [yes or no]: yes
Failing over from Primary Control Station…
Broadcast message from root (pts/0) (Tue Jul 28 13:19:13 2015):
The system is going down for reboot NOW!
|
- After another 15 minutes both CS IPs have changed and CS0 is again Primary
Change IP Address of Storage Processor A and B from CS0:
- The script below will do all the work and the output given is extremely useful as it goes along.
- Once that’s finished, check the status of the Storage Processors
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
|
[root@STRG–001–CS0]# /nas/sbin/clariion_mgmt -modify -spa_ip 10.2.5.8 -spb_ip 10.2.5.9
Checking if running as root...yes
Checking if model is supported...yes
Checking if integrated system...yes
Checking if interface eth3 is configured...yes
Checking if interface eth3:1 is configured...no
Checking if interface eth3:2 is configured...no
Checking if IP (10.2.5.8) is available...yes
Checking if IP (10.2.5.9) is available...yes
Checking if SP (10.2.5.6) is up...yes
Checking if SP (10.2.5.7) is up...yes
Checking if a gateway is setup for eth3...yes
Checking if CS IP address matches domain record...yes
Step 1/14 [28185788431]: Setting up Proxy ARP for SPA on Control Station
Adding host specific route for SPA
Adding rules to allow outbound traffic from SPA
Adding ARP entry for SPA
Updating /etc/hosts entry for SPA
Step 2/14 [28185788417]: Changing SPA IP address.
Changing SPA IP from 10.1.5.6 to 10.2.5.8 (subnetmask 255.255.255.0, gateway 10.2.5.1)
Step 3/14 [28185788421]: Waiting for SPA to restart.
Waiting for SPA to go down.....done (21 secs)
Step 4/14 [28185788425]: Waiting for ping response from SPA.
Waiting for 10.2.5.8 to respond...done (2 secs)
Step 5/14 [28185788427]: Waiting for VNX OE for Block software to start on SPA.
Waiting for VNX OE for Block software to start on SPA...done (62 secs)
Waiting until SPB sees 10.2.5.8 in the domain...done (5 secs)
Step 6/14 [28185788429]: Updating VNX for File database with SPA IP address.
Adding rules to allow outbound traffic from SPB
Updating SYMAPI database with new Block IP addresses...done (67 secs)
Step 7/14 [28185788435]: Removing old Proxy ARP setup for SPA on Control Station
Removing host specific route for SPA
Removing rules that allow outbound traffic from SPA
Removing ARP entry for SPA
Step 8/14 [28185788432]: Setting up Proxy ARP for SPB on Control Station
Adding host specific route for SPB
Adding rules to allow outbound traffic from SPB
Adding ARP entry for SPB
Updating /etc/hosts entry for SPB
Step 9/14 [28185788418]: Changing SPB IP address.
Changing SPB IP from 10.1.5.7 to 10.2.5.9 (subnetmask 255.255.255.0, gateway 10.2.5.1)
Step 10/14 [28185788422]: Waiting for SPB to restart.
Waiting for SPB to go down.....done (21 secs)
Step 11/14 [28185788426]: Waiting for ping response from SPB.
Waiting for 10.2.5.9 to respond...done (2 secs)
Step 12/14 [28185788428]: Waiting for VNX OE for Block software to start on SPB.
Waiting for VNX OE for Block software to start on SPB...done (61 secs)
Waiting until SPA sees 10.2.5.9 in the domain...done (1 secs)
Step 13/14 [28185788430]: Updating VNX for File database with SPB IP address.
Updating SYMAPI database with new Block IP addresses...done (59 secs)
Step 14/14 [28185788436]: Removing old Proxy ARP setup for SPB on Control Station
Removing host specific route for SPB
Removing rules that allow outbound traffic from SPB
Removing ARP entry for SPB
FINISH: Operation took a total time of 5 minutes 35 seconds to complete.
[root@STRG–001–CS0]#
[root@STRG–001–CS0]# /nas/sbin/clariion_mgmt -info
Public IP address for SPA: 10.2.5.8
Public IP address for SPB: 10.2.5.9
Start on boot : yes
Current implementation : Proxy–ARP
Status : Started
|
From here you can access Unisphere from the new SP IP Addresses and go on to reconfigure the rest of the VNXs settings.