Aug 04 11:02:16 fenced fenced 3.0.12 started Aug 04 11:02:16 fenced cpg_join fenced:daemon ... Aug 04 11:02:16 fenced setup_cpg_daemon 12 Aug 04 11:02:16 fenced group_mode 3 compat 0 Aug 04 11:02:16 fenced fenced:daemon conf 1 1 0 memb 2 join 2 left Aug 04 11:02:16 fenced fenced:daemon ring 1:12 8 memb 1 2 3 4 5 6 7 8 Aug 04 11:02:16 fenced receive_protocol from 2 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:16 fenced daemon node 2 max 0.0.0.0 run 0.0.0.0 Aug 04 11:02:16 fenced daemon node 2 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:16 fenced set_protocol member_count 1 propose daemon 1.1.1 Aug 04 11:02:16 fenced receive_protocol from 2 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:16 fenced daemon node 2 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:16 fenced daemon node 2 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:16 fenced run protocol from nodeid 2 Aug 04 11:02:16 fenced daemon run 1.1.1 max 1.1.1 Aug 04 11:02:16 fenced receive_protocol from 2 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:16 fenced daemon node 2 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:16 fenced daemon node 2 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:16 fenced fenced:daemon conf 2 1 0 memb 1 2 join 1 left Aug 04 11:02:16 fenced receive_protocol from 1 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:16 fenced daemon node 1 max 0.0.0.0 run 0.0.0.0 Aug 04 11:02:16 fenced daemon node 1 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:16 fenced receive_protocol from 2 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:16 fenced daemon node 2 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:16 fenced daemon node 2 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:16 fenced receive_protocol from 1 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:16 fenced daemon node 1 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:16 fenced daemon node 1 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:16 fenced client connection 3 fd 13 Aug 04 11:02:17 fenced added 8 nodes from ccs Aug 04 11:02:17 fenced cpg_join fenced:default ... Aug 04 11:02:17 fenced fenced:default ring 1:12 8 memb 1 2 3 4 5 6 7 8 Aug 04 11:02:17 fenced fenced:default conf 2 1 0 memb 1 2 join 2 left Aug 04 11:02:17 fenced add_change cg 1 joined nodeid 2 Aug 04 11:02:17 fenced add_change cg 1 m 2 j 1 r 0 f 0 Aug 04 11:02:17 fenced check_ringid done cluster 12 cpg 1:12 Aug 04 11:02:17 fenced check_quorum done Aug 04 11:02:17 fenced send_start 2:1 flags 1 started 0 m 2 j 1 r 0 f 0 Aug 04 11:02:17 fenced receive_start 1:1 len 392 Aug 04 11:02:17 fenced match_change 1:1 skip cg 1 we are not in members Aug 04 11:02:17 fenced find_change 1:1 no match Aug 04 11:02:17 fenced wait_messages cg 1 need 2 of 2 Aug 04 11:02:17 fenced receive_start 1:2 len 392 Aug 04 11:02:17 fenced match_change 1:2 matches cg 1 Aug 04 11:02:17 fenced wait_messages cg 1 need 1 of 2 Aug 04 11:02:17 fenced receive_start 2:1 len 392 Aug 04 11:02:17 fenced match_change 2:1 matches cg 1 Aug 04 11:02:17 fenced wait_messages cg 1 got all 2 Aug 04 11:02:17 fenced set_master from 0 to low node 1 Aug 04 11:02:17 fenced receive_complete 1:2 len 392 Aug 04 11:02:21 fenced fenced:daemon conf 3 1 0 memb 1 2 3 join 3 left Aug 04 11:02:21 fenced receive_protocol from 1 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 1 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 1 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 2 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 2 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 2 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 3 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 3 max 0.0.0.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 3 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 3 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 3 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 3 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced fenced:daemon conf 4 1 0 memb 1 2 3 4 join 4 left Aug 04 11:02:21 fenced receive_protocol from 1 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 1 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 1 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 2 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 2 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 2 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 4 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 4 max 0.0.0.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 4 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 4 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 4 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 4 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced fenced:daemon conf 5 1 0 memb 1 2 3 4 5 join 5 left Aug 04 11:02:21 fenced receive_protocol from 1 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 1 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 1 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 2 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 2 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 2 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 5 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 5 max 0.0.0.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 5 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 5 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 5 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 5 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced fenced:daemon conf 6 1 0 memb 1 2 3 4 5 6 join 6 left Aug 04 11:02:21 fenced receive_protocol from 1 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 1 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 1 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 2 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 2 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 2 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 5 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 5 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 5 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 6 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 6 max 0.0.0.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 6 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 6 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 6 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 6 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced fenced:daemon conf 7 1 0 memb 1 2 3 4 5 6 7 join 7 left Aug 04 11:02:21 fenced receive_protocol from 1 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 1 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 1 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 2 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 2 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 2 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 5 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 5 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 5 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 6 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 6 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 6 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 7 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 7 max 0.0.0.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 7 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 7 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 7 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 7 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced fenced:daemon conf 8 1 0 memb 1 2 3 4 5 6 7 8 join 8 left Aug 04 11:02:21 fenced receive_protocol from 5 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 5 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 5 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 6 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 6 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 6 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 7 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 7 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 7 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 8 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 8 max 0.0.0.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 8 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 1 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 1 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 1 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 2 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 2 max 1.1.1.0 run 1.1.1.1 Aug 04 11:02:21 fenced daemon node 2 join 1280912536 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 8 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 8 max 1.1.1.0 run 0.0.0.0 Aug 04 11:02:21 fenced daemon node 8 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 3 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 3 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 3 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 3 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 3 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 3 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 3 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 3 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 3 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 3 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 3 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 3 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 3 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 3 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 3 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 4 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 4 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 4 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 4 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 4 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 4 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 4 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 4 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 4 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:21 fenced receive_protocol from 4 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 4 max 1.1.1.0 run 1.1.1.0 Aug 04 11:02:21 fenced daemon node 4 join 1280912541 left 0 local quorum 1280912536 Aug 04 11:02:22 fenced fenced:default conf 3 1 0 memb 1 2 3 join 3 left Aug 04 11:02:22 fenced add_change cg 2 joined nodeid 3 Aug 04 11:02:22 fenced add_change cg 2 m 3 j 1 r 0 f 0 Aug 04 11:02:22 fenced check_ringid done cluster 12 cpg 1:12 Aug 04 11:02:22 fenced check_quorum done Aug 04 11:02:22 fenced send_start 2:2 flags 2 started 1 m 3 j 1 r 0 f 0 Aug 04 11:02:22 fenced receive_start 1:3 len 392 Aug 04 11:02:22 fenced match_change 1:3 matches cg 2 Aug 04 11:02:22 fenced wait_messages cg 2 need 2 of 3 Aug 04 11:02:22 fenced receive_start 2:2 len 392 Aug 04 11:02:22 fenced match_change 2:2 matches cg 2 Aug 04 11:02:22 fenced wait_messages cg 2 need 1 of 3 Aug 04 11:02:22 fenced receive_start 3:1 len 392 Aug 04 11:02:22 fenced match_change 3:1 matches cg 2 Aug 04 11:02:22 fenced wait_messages cg 2 got all 3 Aug 04 11:02:22 fenced set_master from 1 to complete node 1 Aug 04 11:02:22 fenced receive_complete 1:3 len 392 Aug 04 11:02:22 fenced fenced:default conf 4 1 0 memb 1 2 3 4 join 4 left Aug 04 11:02:22 fenced add_change cg 3 joined nodeid 4 Aug 04 11:02:22 fenced add_change cg 3 m 4 j 1 r 0 f 0 Aug 04 11:02:22 fenced check_ringid done cluster 12 cpg 1:12 Aug 04 11:02:22 fenced check_quorum done Aug 04 11:02:22 fenced send_start 2:3 flags 2 started 2 m 4 j 1 r 0 f 0 Aug 04 11:02:22 fenced receive_start 1:4 len 392 Aug 04 11:02:22 fenced match_change 1:4 matches cg 3 Aug 04 11:02:22 fenced wait_messages cg 3 need 3 of 4 Aug 04 11:02:22 fenced receive_start 2:3 len 392 Aug 04 11:02:22 fenced match_change 2:3 matches cg 3 Aug 04 11:02:22 fenced wait_messages cg 3 need 2 of 4 Aug 04 11:02:22 fenced receive_start 3:2 len 392 Aug 04 11:02:22 fenced match_change 3:2 matches cg 3 Aug 04 11:02:22 fenced wait_messages cg 3 need 1 of 4 Aug 04 11:02:22 fenced receive_start 4:1 len 392 Aug 04 11:02:22 fenced match_change 4:1 matches cg 3 Aug 04 11:02:22 fenced wait_messages cg 3 got all 4 Aug 04 11:02:22 fenced set_master from 1 to complete node 1 Aug 04 11:02:22 fenced receive_complete 1:4 len 392 Aug 04 11:02:22 fenced fenced:default conf 5 1 0 memb 1 2 3 4 8 join 8 left Aug 04 11:02:22 fenced add_change cg 4 joined nodeid 8 Aug 04 11:02:22 fenced add_change cg 4 m 5 j 1 r 0 f 0 Aug 04 11:02:22 fenced check_ringid done cluster 12 cpg 1:12 Aug 04 11:02:22 fenced check_quorum done Aug 04 11:02:22 fenced send_start 2:4 flags 2 started 3 m 5 j 1 r 0 f 0 Aug 04 11:02:22 fenced receive_start 2:4 len 392 Aug 04 11:02:22 fenced match_change 2:4 matches cg 4 Aug 04 11:02:22 fenced wait_messages cg 4 need 4 of 5 Aug 04 11:02:22 fenced receive_start 3:3 len 392 Aug 04 11:02:22 fenced match_change 3:3 matches cg 4 Aug 04 11:02:22 fenced wait_messages cg 4 need 3 of 5 Aug 04 11:02:22 fenced receive_start 4:2 len 392 Aug 04 11:02:22 fenced match_change 4:2 matches cg 4 Aug 04 11:02:22 fenced wait_messages cg 4 need 2 of 5 Aug 04 11:02:22 fenced receive_start 1:5 len 392 Aug 04 11:02:22 fenced match_change 1:5 matches cg 4 Aug 04 11:02:22 fenced wait_messages cg 4 need 1 of 5 Aug 04 11:02:22 fenced fenced:default conf 6 1 0 memb 1 2 3 4 5 8 join 5 left Aug 04 11:02:22 fenced add_change cg 5 joined nodeid 5 Aug 04 11:02:22 fenced add_change cg 5 m 6 j 1 r 0 f 0 Aug 04 11:02:22 fenced check_ringid done cluster 12 cpg 1:12 Aug 04 11:02:22 fenced check_quorum done Aug 04 11:02:22 fenced send_start 2:5 flags 2 started 3 m 6 j 1 r 0 f 0 Aug 04 11:02:22 fenced fenced:default conf 7 1 0 memb 1 2 3 4 5 7 8 join 7 left Aug 04 11:02:22 fenced add_change cg 6 joined nodeid 7 Aug 04 11:02:22 fenced add_change cg 6 m 7 j 1 r 0 f 0 Aug 04 11:02:22 fenced check_ringid done cluster 12 cpg 1:12 Aug 04 11:02:22 fenced check_quorum done Aug 04 11:02:22 fenced send_start 2:6 flags 2 started 3 m 7 j 1 r 0 f 0 Aug 04 11:02:22 fenced receive_start 8:1 len 392 Aug 04 11:02:22 fenced match_change 8:1 matches cg 4 Aug 04 11:02:22 fenced wait_messages cg 6 need 7 of 7 Aug 04 11:02:22 fenced receive_start 8:2 len 392 Aug 04 11:02:22 fenced match_change 8:2 skip cg 4 already start Aug 04 11:02:22 fenced match_change 8:2 matches cg 5 Aug 04 11:02:22 fenced wait_messages cg 6 need 7 of 7 Aug 04 11:02:22 fenced receive_start 1:6 len 392 Aug 04 11:02:22 fenced match_change 1:6 skip cg 4 already start Aug 04 11:02:22 fenced match_change 1:6 matches cg 5 Aug 04 11:02:22 fenced wait_messages cg 6 need 7 of 7 Aug 04 11:02:22 fenced receive_start 1:7 len 392 Aug 04 11:02:22 fenced match_change 1:7 skip cg 4 already start Aug 04 11:02:22 fenced match_change 1:7 skip cg 5 already start Aug 04 11:02:22 fenced match_change 1:7 matches cg 6 Aug 04 11:02:22 fenced wait_messages cg 6 need 6 of 7 Aug 04 11:02:22 fenced receive_start 2:5 len 392 Aug 04 11:02:22 fenced match_change 2:5 skip cg 4 already start Aug 04 11:02:22 fenced match_change 2:5 matches cg 5 Aug 04 11:02:22 fenced wait_messages cg 6 need 6 of 7 Aug 04 11:02:22 fenced receive_start 2:6 len 392 Aug 04 11:02:22 fenced match_change 2:6 skip cg 4 already start Aug 04 11:02:22 fenced match_change 2:6 skip cg 5 already start Aug 04 11:02:22 fenced match_change 2:6 matches cg 6 Aug 04 11:02:22 fenced wait_messages cg 6 need 5 of 7 Aug 04 11:02:22 fenced receive_start 3:4 len 392 Aug 04 11:02:22 fenced match_change 3:4 skip cg 4 already start Aug 04 11:02:22 fenced match_change 3:4 matches cg 5 Aug 04 11:02:22 fenced wait_messages cg 6 need 5 of 7 Aug 04 11:02:22 fenced receive_start 3:5 len 392 Aug 04 11:02:22 fenced match_change 3:5 skip cg 4 already start Aug 04 11:02:22 fenced match_change 3:5 skip cg 5 already start Aug 04 11:02:22 fenced match_change 3:5 matches cg 6 Aug 04 11:02:22 fenced wait_messages cg 6 need 4 of 7 Aug 04 11:02:22 fenced receive_start 4:3 len 392 Aug 04 11:02:22 fenced match_change 4:3 skip cg 4 already start Aug 04 11:02:22 fenced match_change 4:3 matches cg 5 Aug 04 11:02:22 fenced wait_messages cg 6 need 4 of 7 Aug 04 11:02:22 fenced receive_start 4:4 len 392 Aug 04 11:02:22 fenced match_change 4:4 skip cg 4 already start Aug 04 11:02:22 fenced match_change 4:4 skip cg 5 already start Aug 04 11:02:22 fenced match_change 4:4 matches cg 6 Aug 04 11:02:22 fenced wait_messages cg 6 need 3 of 7 Aug 04 11:02:22 fenced receive_start 8:3 len 392 Aug 04 11:02:22 fenced match_change 8:3 skip cg 4 already start Aug 04 11:02:22 fenced match_change 8:3 skip cg 5 already start Aug 04 11:02:22 fenced match_change 8:3 matches cg 6 Aug 04 11:02:22 fenced wait_messages cg 6 need 2 of 7 Aug 04 11:02:22 fenced receive_start 5:1 len 392 Aug 04 11:02:22 fenced match_change 5:1 skip cg 4 sender not member Aug 04 11:02:22 fenced match_change 5:1 matches cg 5 Aug 04 11:02:22 fenced wait_messages cg 6 need 2 of 7 Aug 04 11:02:22 fenced receive_start 7:1 len 392 Aug 04 11:02:22 fenced match_change 7:1 skip cg 4 sender not member Aug 04 11:02:22 fenced match_change 7:1 skip cg 5 sender not member Aug 04 11:02:22 fenced match_change 7:1 matches cg 6 Aug 04 11:02:22 fenced wait_messages cg 6 need 1 of 7 Aug 04 11:02:22 fenced receive_start 5:2 len 392 Aug 04 11:02:22 fenced match_change 5:2 skip cg 4 sender not member Aug 04 11:02:22 fenced match_change 5:2 skip cg 5 already start Aug 04 11:02:22 fenced match_change 5:2 matches cg 6 Aug 04 11:02:22 fenced wait_messages cg 6 got all 7 Aug 04 11:02:22 fenced set_master from 1 to complete node 1 Aug 04 11:02:22 fenced fenced:default conf 8 1 0 memb 1 2 3 4 5 6 7 8 join 6 left Aug 04 11:02:22 fenced add_change cg 7 joined nodeid 6 Aug 04 11:02:22 fenced add_change cg 7 m 8 j 1 r 0 f 0 Aug 04 11:02:22 fenced check_ringid done cluster 12 cpg 1:12 Aug 04 11:02:22 fenced check_quorum done Aug 04 11:02:22 fenced send_start 2:7 flags 2 started 4 m 8 j 1 r 0 f 0 Aug 04 11:02:22 fenced receive_start 7:2 len 392 Aug 04 11:02:22 fenced match_change 7:2 matches cg 7 Aug 04 11:02:22 fenced wait_messages cg 7 need 7 of 8 Aug 04 11:02:22 fenced receive_start 8:4 len 392 Aug 04 11:02:22 fenced match_change 8:4 matches cg 7 Aug 04 11:02:22 fenced wait_messages cg 7 need 6 of 8 Aug 04 11:02:22 fenced receive_complete 1:7 len 392 Aug 04 11:02:22 fenced wait_messages cg 7 need 6 of 8 Aug 04 11:02:22 fenced receive_start 1:8 len 392 Aug 04 11:02:22 fenced match_change 1:8 matches cg 7 Aug 04 11:02:22 fenced wait_messages cg 7 need 5 of 8 Aug 04 11:02:22 fenced receive_start 2:7 len 392 Aug 04 11:02:22 fenced match_change 2:7 matches cg 7 Aug 04 11:02:22 fenced wait_messages cg 7 need 4 of 8 Aug 04 11:02:22 fenced receive_start 3:6 len 392 Aug 04 11:02:22 fenced match_change 3:6 matches cg 7 Aug 04 11:02:22 fenced wait_messages cg 7 need 3 of 8 Aug 04 11:02:22 fenced receive_start 4:5 len 392 Aug 04 11:02:22 fenced match_change 4:5 matches cg 7 Aug 04 11:02:22 fenced wait_messages cg 7 need 2 of 8 Aug 04 11:02:22 fenced receive_start 5:3 len 392 Aug 04 11:02:22 fenced match_change 5:3 matches cg 7 Aug 04 11:02:22 fenced wait_messages cg 7 need 1 of 8 Aug 04 11:02:22 fenced receive_start 6:1 len 392 Aug 04 11:02:22 fenced match_change 6:1 matches cg 7 Aug 04 11:02:22 fenced wait_messages cg 7 got all 8 Aug 04 11:02:22 fenced set_master from 1 to complete node 1 Aug 04 11:02:22 fenced receive_complete 1:8 len 392 Aug 04 11:09:39 fenced fenced:daemon ring 1:16 8 memb 1 2 3 4 5 6 7 8 Aug 04 11:09:39 fenced fenced:default ring 1:16 8 memb 1 2 3 4 5 6 7 8 Aug 04 11:10:23 fenced fenced:daemon ring 1:20 8 memb 1 2 3 4 5 6 7 8 Aug 04 11:10:23 fenced fenced:default ring 1:20 8 memb 1 2 3 4 5 6 7 8 Aug 04 11:12:54 fenced fenced:daemon ring 1:24 8 memb 1 2 3 4 5 6 7 8 Aug 04 11:12:54 fenced fenced:default ring 1:24 8 memb 1 2 3 4 5 6 7 8 Aug 04 11:13:20 fenced cluster node 0 removed seq 24 Aug 04 11:13:20 fenced node_history_cluster_remove no nodeid 0 Aug 04 11:13:22 fenced cluster node 0 added seq 24 Aug 04 11:13:43 fenced cluster node 0 removed seq 24 Aug 04 11:13:47 fenced cluster node 0 added seq 24 Aug 04 11:13:47 fenced cluster node 5 removed seq 28 Aug 04 11:13:47 fenced cluster node 6 removed seq 28 Aug 04 11:13:47 fenced cluster node 7 removed seq 28 Aug 04 11:13:47 fenced cluster node 5 added seq 32 Aug 04 11:13:47 fenced cluster node 6 added seq 32 Aug 04 11:13:47 fenced cluster node 7 added seq 32 Aug 04 11:14:01 fenced fenced:daemon conf 7 0 1 memb 1 2 3 4 6 7 8 join left 5 Aug 04 11:14:01 fenced fenced:daemon conf 6 0 1 memb 1 2 3 4 7 8 join left 6 Aug 04 11:14:01 fenced fenced:daemon conf 5 0 1 memb 1 2 3 4 8 join left 7 Aug 04 11:14:01 fenced fenced:default conf 7 0 1 memb 1 2 3 4 6 7 8 join left 5 Aug 04 11:14:01 fenced add_change cg 8 remove nodeid 5 reason 3 Aug 04 11:14:01 fenced add_change cg 8 m 7 j 0 r 1 f 1 Aug 04 11:14:01 fenced add_victims node 5 Aug 04 11:14:01 fenced check_ringid cluster 32 cpg 1:24 Aug 04 11:14:01 fenced fenced:default conf 6 0 1 memb 1 2 3 4 7 8 join left 6 Aug 04 11:14:01 fenced add_change cg 9 remove nodeid 6 reason 3 Aug 04 11:14:01 fenced add_change cg 9 m 6 j 0 r 1 f 1 Aug 04 11:14:01 fenced add_victims node 6 Aug 04 11:14:01 fenced check_ringid cluster 32 cpg 1:24 Aug 04 11:14:01 fenced fenced:default conf 5 0 1 memb 1 2 3 4 8 join left 7 Aug 04 11:14:01 fenced add_change cg 10 remove nodeid 7 reason 3 Aug 04 11:14:01 fenced add_change cg 10 m 5 j 0 r 1 f 1 Aug 04 11:14:01 fenced add_victims node 7 Aug 04 11:14:01 fenced check_ringid cluster 32 cpg 1:24 Aug 04 11:14:01 fenced fenced:daemon ring 1:32 8 memb 1 2 3 4 5 6 7 8 Aug 04 11:14:01 fenced fenced:default ring 1:32 8 memb 1 2 3 4 5 6 7 8 Aug 04 11:14:01 fenced check_ringid done cluster 32 cpg 1:32 Aug 04 11:14:01 fenced check_quorum done Aug 04 11:14:01 fenced send_start 2:10 flags 2 started 5 m 5 j 0 r 1 f 1 Aug 04 11:14:01 fenced cpg_mcast_joined retried 4 start Aug 04 11:14:01 fenced receive_start 8:7 len 432 Aug 04 11:14:01 fenced match_change 8:7 skip cg 8 expect counts 7 0 1 1 Aug 04 11:14:01 fenced match_change 8:7 skip cg 9 expect counts 6 0 1 1 Aug 04 11:14:01 fenced match_change 8:7 matches cg 10 Aug 04 11:14:01 fenced wait_messages cg 10 need 4 of 5 Aug 04 11:14:01 fenced receive_start 1:11 len 432 Aug 04 11:14:01 fenced match_change 1:11 skip cg 8 expect counts 7 0 1 1 Aug 04 11:14:01 fenced match_change 1:11 skip cg 9 expect counts 6 0 1 1 Aug 04 11:14:01 fenced match_change 1:11 matches cg 10 Aug 04 11:14:01 fenced wait_messages cg 10 need 3 of 5 Aug 04 11:14:01 fenced receive_start 2:10 len 432 Aug 04 11:14:01 fenced match_change 2:10 skip cg 8 expect counts 7 0 1 1 Aug 04 11:14:01 fenced match_change 2:10 skip cg 9 expect counts 6 0 1 1 Aug 04 11:14:01 fenced match_change 2:10 matches cg 10 Aug 04 11:14:01 fenced wait_messages cg 10 need 2 of 5 Aug 04 11:14:01 fenced receive_start 3:9 len 432 Aug 04 11:14:01 fenced match_change 3:9 skip cg 8 expect counts 7 0 1 1 Aug 04 11:14:01 fenced match_change 3:9 skip cg 9 expect counts 6 0 1 1 Aug 04 11:14:01 fenced match_change 3:9 matches cg 10 Aug 04 11:14:01 fenced wait_messages cg 10 need 1 of 5 Aug 04 11:14:01 fenced receive_start 4:8 len 432 Aug 04 11:14:01 fenced match_change 4:8 skip cg 8 expect counts 7 0 1 1 Aug 04 11:14:01 fenced match_change 4:8 skip cg 9 expect counts 6 0 1 1 Aug 04 11:14:01 fenced match_change 4:8 matches cg 10 Aug 04 11:14:01 fenced wait_messages cg 10 got all 5 Aug 04 11:14:01 fenced set_master from 1 to complete node 1 Aug 04 11:14:01 fenced fencing deferred to rhel6-node1 Aug 04 11:14:32 fenced cluster node 5 removed seq 36 Aug 04 11:14:32 fenced cluster node 6 removed seq 36 Aug 04 11:14:32 fenced cluster node 7 removed seq 36 Aug 04 11:14:32 fenced receive_victim_done 1:11 flags 2 len 80 Aug 04 11:14:32 fenced receive_victim_done 1:11 remove victim 7 time 1280913255 how 1 Aug 04 11:14:32 fenced receive_victim_done 1:11 flags 2 len 80 Aug 04 11:14:32 fenced receive_victim_done 1:11 remove victim 6 time 1280913260 how 1 Aug 04 11:14:32 fenced receive_victim_done 1:11 flags 2 len 80 Aug 04 11:14:32 fenced receive_victim_done 1:11 remove victim 5 time 1280913264 how 1 Aug 04 11:14:32 fenced receive_complete 1:11 len 432 Aug 04 11:14:32 fenced fenced:daemon ring 1:36 5 memb 1 2 3 4 8 Aug 04 11:14:32 fenced fenced:default ring 1:36 5 memb 1 2 3 4 8 Aug 04 11:17:45 fenced fenced:daemon ring 1:40 5 memb 1 2 3 4 8 Aug 04 11:17:45 fenced fenced:default ring 1:40 5 memb 1 2 3 4 8 Aug 04 11:18:49 fenced cluster node 1 removed seq 48 Aug 04 11:18:49 fenced cluster node 3 removed seq 48 Aug 04 11:18:49 fenced cluster node 4 removed seq 48 Aug 04 11:18:49 fenced cluster node 8 removed seq 48 Aug 04 11:18:50 fenced cluster node 3 added seq 60 Aug 04 11:18:50 fenced cluster node 4 added seq 60 Aug 04 11:18:50 fenced cluster node 1 added seq 60 Aug 04 11:18:50 fenced cluster node 8 added seq 60 Aug 04 11:18:50 fenced fenced:daemon conf 4 0 1 memb 2 3 4 8 join left 1 Aug 04 11:18:50 fenced fenced:daemon conf 3 0 1 memb 2 3 4 join left 8 Aug 04 11:18:50 fenced fenced:default conf 4 0 1 memb 2 3 4 8 join left 1 Aug 04 11:18:50 fenced add_change cg 11 remove nodeid 1 reason 3 Aug 04 11:18:50 fenced add_change cg 11 m 4 j 0 r 1 f 1 Aug 04 11:18:50 fenced add_victims node 1 Aug 04 11:18:50 fenced check_ringid cluster 60 cpg 1:40 Aug 04 11:18:50 fenced fenced:default conf 3 0 1 memb 2 3 4 join left 8 Aug 04 11:18:50 fenced add_change cg 12 remove nodeid 8 reason 3 Aug 04 11:18:50 fenced add_change cg 12 m 3 j 0 r 1 f 1 Aug 04 11:18:50 fenced add_victims node 8 Aug 04 11:18:50 fenced check_ringid cluster 60 cpg 1:40 Aug 04 11:18:50 fenced fenced:daemon ring 1:60 5 memb 1 2 3 4 8 Aug 04 11:18:50 fenced fenced:default ring 1:60 5 memb 1 2 3 4 8 Aug 04 11:18:50 fenced check_ringid done cluster 60 cpg 1:60 Aug 04 11:18:50 fenced check_quorum done Aug 04 11:18:50 fenced send_start 2:12 flags 2 started 6 m 3 j 0 r 1 f 1 Aug 04 11:18:50 fenced cpg_mcast_joined retried 1 start Aug 04 11:18:50 fenced receive_start 2:12 len 432 Aug 04 11:18:50 fenced match_change 2:12 skip cg 11 expect counts 4 0 1 1 Aug 04 11:18:50 fenced match_change 2:12 matches cg 12 Aug 04 11:18:50 fenced wait_messages cg 12 need 2 of 3 Aug 04 11:18:50 fenced receive_start 3:11 len 432 Aug 04 11:18:50 fenced match_change 3:11 skip cg 11 expect counts 4 0 1 1 Aug 04 11:18:50 fenced match_change 3:11 matches cg 12 Aug 04 11:18:50 fenced wait_messages cg 12 need 1 of 3 Aug 04 11:18:50 fenced receive_start 4:10 len 432 Aug 04 11:18:50 fenced match_change 4:10 skip cg 11 expect counts 4 0 1 1 Aug 04 11:18:50 fenced match_change 4:10 matches cg 12 Aug 04 11:18:50 fenced wait_messages cg 12 got all 3 Aug 04 11:18:50 fenced set_master from 1 to complete node 2 Aug 04 11:18:50 fenced rhel6-node8 not a cluster member after 0 sec post_fail_delay Aug 04 11:18:50 fenced rhel6-node1 not a cluster member after 0 sec post_fail_delay Aug 04 11:18:50 fenced fencing node rhel6-node8 Aug 04 11:18:51 fenced fence rhel6-node8 dev 0.0 agent fence_virsh result: error from agent Aug 04 11:18:51 fenced fence rhel6-node8 failed Aug 04 11:18:54 fenced fencing node rhel6-node8 Aug 04 11:18:58 fenced fence rhel6-node8 dev 0.0 agent fence_virsh result: success Aug 04 11:18:58 fenced fence rhel6-node8 success Aug 04 11:18:58 fenced send_victim_done cg 12 flags 2 victim nodeid 8 Aug 04 11:18:58 fenced fencing node rhel6-node1 Aug 04 11:19:03 fenced fence rhel6-node1 dev 0.0 agent fence_virsh result: success Aug 04 11:19:03 fenced fence rhel6-node1 success Aug 04 11:19:03 fenced send_victim_done cg 12 flags 2 victim nodeid 1 Aug 04 11:19:03 fenced send_complete 2:12 flags 2 started 6 m 3 j 0 r 1 f 1 Aug 04 11:19:19 fenced cluster node 1 removed seq 64 Aug 04 11:19:19 fenced cluster node 8 removed seq 64 Aug 04 11:19:19 fenced receive_victim_done 2:12 flags 2 len 80 Aug 04 11:19:19 fenced receive_victim_done 2:12 remove victim 8 time 1280913538 how 1 Aug 04 11:19:19 fenced receive_victim_done 2:12 flags 2 len 80 Aug 04 11:19:19 fenced receive_victim_done 2:12 remove victim 1 time 1280913543 how 1 Aug 04 11:19:19 fenced receive_complete 2:12 len 432 Aug 04 11:19:22 fenced fenced:daemon ring 2:64 3 memb 2 3 4 Aug 04 11:19:22 fenced fenced:default ring 2:64 3 memb 2 3 4