[Read This Fine Material] from Joshua Hoblitt

How to demote a GPFS 3.5 server to a regular client


Here is an example of demoting two GPFS servers named foonsd1 and foonsd2 into being regular GPFS clients. This means removing them from being part of cluster “quorum”, declaring that it is to be a client node (ie, can not be a token manager), designating new configuration servers, and changing the licensing terms for those nodes. Three new GPFS servers have already been added to the cluster so that at no point is quorum lost.

In order to change the quorum status of a node, GPFS must not be running.

# mmshutdown -N foonsd1,foonsd2
Sun Mar 23 14:20:38 MST 2014: mmshutdown: Starting force unmount of GPFS file systems
Sun Mar 23 14:20:43 MST 2014: mmshutdown: Shutting down GPFS daemons
foonsd2.example.org:  Shutting down!
foonsd1.example.org:  Shutting down!
foonsd2.example.org:  'shutdown' command about to kill process 3256
foonsd2.example.org:  Unloading modules from /lib/modules/2.6.32-431.5.1.el6.x86_64/extra
foonsd2.example.org:  Unloading module mmfs26
foonsd1.example.org:  'shutdown' command about to kill process 3242
foonsd1.example.org:  Unloading modules from /lib/modules/2.6.32-431.5.1.el6.x86_64/extra
foonsd1.example.org:  Unloading module mmfs26
foonsd2.example.org:  Unloading module mmfslinux
foonsd2.example.org:  Unloading module tracedev
foonsd1.example.org:  Unloading module mmfslinux
foonsd1.example.org:  Unloading module tracedev
Sun Mar 23 14:20:53 MST 2014: mmshutdown: Finished
# mmchnode --client --noquorum -N foonsd1,foonsd2
Sun Mar 23 14:21:04 MST 2014: mmchnode: Processing node foonsd1.example.org
Sun Mar 23 14:21:04 MST 2014: mmchnode: Processing node foonsd2.example.org
Verifying GPFS is stopped on all affected nodes ...
mmchnode: Propagating the cluster configuration data to all
  affected nodes.  This is an asynchronous process.
# mmlscluster 

GPFS cluster information
  GPFS cluster name:         fooamcp.example.org
  GPFS cluster id:           10159024612008204251
  GPFS UID domain:           fooamcp.example.org
  Remote shell command:      /usr/bin/ssh
  Remote file copy command:  /usr/bin/scp

GPFS cluster configuration servers:
  Primary server:    foonsd1.example.org
  Secondary server:  foonsd2.example.org

 Node  Daemon node name      IP address     Admin node name       Designation
   1   foo01.example.org  foo01.example.org    
   2   foo02.example.org  foo02.example.org    
   3   foo03.example.org  foo03.example.org    
   4   foo04.example.org  foo04.example.org    
   5   foo05.example.org  foo05.example.org    
   7   foo07.example.org  foo07.example.org    
   8   foo08.example.org  foo08.example.org    
   9   foo09.example.org  foo09.example.org    
  10   foo10.example.org  foo10.example.org    
  11   foo11.example.org  foo11.example.org    
  12   foo12.example.org  foo12.example.org    
  13   foo13.example.org  foo13.example.org    
  14   foo14.example.org  foo14.example.org    
  15   foo15.example.org  foo15.example.org    
  16   foonsd1.example.org  foonsd1.example.org  
  17   foonsd2.example.org  foonsd2.example.org  
  18   foo06.example.org  foo06.example.org    
  19   foodb1.example.org  foodb1.example.org   
  20   foonsd5.example.org  foonsd5.example.org  quorum-manager
  21   foonsd3.example.org  foonsd3.example.org  quorum-manager
  22   foonsd4.example.org  foonsd4.example.org  quorum-manager

Changing the configuration servers can be done as long as the new servers are online. I don’t believe it matters if GPFS is running on the old config serers or not.

# mmchcluster -p foonsd3 -s foonsd4
mmchcluster: GPFS cluster configuration servers:
mmchcluster:   Primary server:    foonsd3.example.org
mmchcluster:   Secondary server:  foonsd4.example.org
mmchcluster: Propagating the new server information to the rest of the nodes.
mmchcluster: Command successfully completed

Lastly, we need to change the nodes to having client rather than server licenses.

# mmchlicense client --accept -N foonsd1,foonsd2

The following nodes will be designated as possessing GPFS client licenses:
mmchlicense: Command successfully completed
mmchlicense: Propagating the cluster configuration data to all
  affected nodes.  This is an asynchronous process.
# mmlslicense -L
 Node name                      Required license   Designated license
foo01.example.org                 client              client
foo02.example.org                 client              client
foo03.example.org                 client              client
foo04.example.org                 client              client
foo05.example.org                 client              client
foo07.example.org                 client              client
foo08.example.org                 client              client
foo09.example.org                 client              client
foo10.example.org                 client              client
foo11.example.org                 client              client
foo12.example.org                 client              client
foo13.example.org                 client              client
foo14.example.org                 client              client
foo15.example.org                 client              client
foonsd1.example.org               client              client
foonsd2.example.org               client              client
foo06.example.org                 client              client
foodb1.example.org                client              client
foonsd5.example.org               server              server
foonsd3.example.org               server              server
foonsd4.example.org               server              server

 Summary information 
Number of nodes defined in the cluster:                         21
Number of nodes with server license designation:                 3
Number of nodes with client license designation:                18
Number of nodes still requiring server license designation:      0
Number of nodes still requiring client license designation:      0

Leave a Reply