<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <div class="moz-cite-prefix">Quorum doesn't prevent split-brains,
      stonith (fencing) does. <br>
      <br>
      <a class="moz-txt-link-freetext" href="https://www.alteeve.com/w/The_2-Node_Myth">https://www.alteeve.com/w/The_2-Node_Myth</a><br>
      <br>
      There is no way to use quorum-only to avoid a potential
      split-brain. You might be able to make it less likely with enough
      effort, but never prevent it.<br>
      <br>
      digimer<br>
      <br>
      On 2017-11-14 10:45 PM, Garima wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:86CE92308A86944992E6D1B25BD638AEAE79734A@EXCH-MB01-DEL.nectechnologies.in">
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
      <meta name="Generator" content="Microsoft Word 15 (filtered
        medium)">
      <style><!--
/* Font Definitions */
@font-face
        {font-family:Helvetica;
        panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
        {font-family:"Cambria Math";
        panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0cm;
        margin-bottom:.0001pt;
        font-size:12.0pt;
        font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
        {mso-style-priority:99;
        color:blue;
        text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
        {mso-style-priority:99;
        color:purple;
        text-decoration:underline;}
span.apple-converted-space
        {mso-style-name:apple-converted-space;}
span.EmailStyle18
        {mso-style-type:personal-reply;
        font-family:"Calibri",sans-serif;
        color:#1F497D;}
.MsoChpDefault
        {mso-style-type:export-only;
        font-size:10.0pt;}
@page WordSection1
        {size:612.0pt 792.0pt;
        margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
        {page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
      <div class="WordSection1">
        <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif;color:#1F497D;mso-fareast-language:EN-US">Hello
            All,<o:p></o:p></span></p>
        <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
        <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif;color:#1F497D;mso-fareast-language:EN-US">Split-brain
            situation occurs due to there is a drop in quorum which
            leads to Spilt-brain situation and status information is not
            exchanged between both two nodes of the cluster. <o:p></o:p></span></p>
        <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif;color:#1F497D;mso-fareast-language:EN-US">This
            can be avoided if quorum communicates between both the
            nodes.
            <o:p></o:p></span></p>
        <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif;color:#1F497D;mso-fareast-language:EN-US">I
            have checked the code. In My opinion these files need to be
            updated (quorum.py/stonith.py) to avoid the spilt-brain
            situation to maintain Active-Passive configuration.<o:p></o:p></span></p>
        <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
        <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif;color:#1F497D;mso-fareast-language:EN-US">Regards,<o:p></o:p></span></p>
        <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif;color:#1F497D;mso-fareast-language:EN-US">Garima<o:p></o:p></span></p>
        <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
        <div>
          <div style="border:none;border-top:solid #E1E1E1
            1.0pt;padding:3.0pt 0cm 0cm 0cm">
            <p class="MsoNormal"><b><span
                  style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif"
                  lang="EN-US">From:</span></b><span
                style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif"
                lang="EN-US"> Derek Wuelfrath
                [<a class="moz-txt-link-freetext" href="mailto:dwuelfrath@inverse.ca">mailto:dwuelfrath@inverse.ca</a>]
                <br>
                <b>Sent:</b> 13 November 2017 20:55<br>
                <b>To:</b> Cluster Labs - All topics related to
                open-source clustering welcomed
                <a class="moz-txt-link-rfc2396E" href="mailto:users@clusterlabs.org">&lt;users@clusterlabs.org&gt;</a><br>
                <b>Subject:</b> Re: [ClusterLabs] Pacemaker responsible
                of DRBD and a systemd resource<o:p></o:p></span></p>
          </div>
        </div>
        <p class="MsoNormal"><o:p> </o:p></p>
        <p class="MsoNormal">Hello Ken !<o:p></o:p></p>
        <div>
          <p class="MsoNormal"><o:p> </o:p></p>
        </div>
        <div>
          <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
            <p class="MsoNormal">Make sure that the systemd service is
              not enabled. If pacemaker is<br>
              managing a service, systemd can't also be trying to start
              and stop it.<o:p></o:p></p>
          </blockquote>
          <p class="MsoNormal"><o:p> </o:p></p>
        </div>
        <div>
          <p class="MsoNormal">It is not. I made sure of this in the
            first place :)<o:p></o:p></p>
        </div>
        <div>
          <p class="MsoNormal"><o:p> </o:p></p>
        </div>
        <div>
          <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
            <p class="MsoNormal">Beyond that, the question is what log
              messages are there from around<br>
              the time of the issue (on both nodes).<o:p></o:p></p>
          </blockquote>
          <p class="MsoNormal"><o:p> </o:p></p>
        </div>
        <div>
          <p class="MsoNormal">Well, that’s the thing. There is not much
            log messages telling what is actually happening. The
            ’systemd’ resource is not even trying to start (nothing in
            either log for that resource). Here are the logs from my
            last attempt:<o:p></o:p></p>
        </div>
        <div>
          <p class="MsoNormal">Scenario:<o:p></o:p></p>
        </div>
        <div>
          <p class="MsoNormal">- Services were running on
            ‘pancakeFence2’. DRBD was synced and connected<o:p></o:p></p>
        </div>
        <div>
          <p class="MsoNormal">- I rebooted ‘pancakeFence2’. Services
            failed to ‘pancakeFence1’<o:p></o:p></p>
        </div>
        <div>
          <p class="MsoNormal">- After ‘pancakeFence2’ comes back,
            services are running just fine on ‘pancakeFence1’ but DRBD
            is in Standalone due to split-brain<o:p></o:p></p>
        </div>
        <div>
          <p class="MsoNormal"><o:p> </o:p></p>
        </div>
        <div>
          <p class="MsoNormal">Logs for pancakeFence1: <a
              href="https://pastebin.com/dVSGPP78"
              moz-do-not-send="true">https://pastebin.com/dVSGPP78</a><o:p></o:p></p>
        </div>
        <div>
          <p class="MsoNormal">Logs for pancakeFence2: <a
              href="https://pastebin.com/at8qPkHE"
              moz-do-not-send="true">https://pastebin.com/at8qPkHE</a><o:p></o:p></p>
        </div>
        <div>
          <p class="MsoNormal"><o:p> </o:p></p>
        </div>
        <div>
          <p class="MsoNormal">It really looks like the status checkup
            mechanism of corosync/pacemaker for a systemd resource force
            the resource to “start” and therefore, start the ones above
            that resource in the group (DRBD in instance).<o:p></o:p></p>
        </div>
        <div>
          <p class="MsoNormal">This does not happen for a regular OCF
            resource (IPaddr2 per example)<o:p></o:p></p>
        </div>
        <div>
          <div>
            <div>
              <div>
                <div>
                  <div>
                    <div>
                      <div>
                        <div>
                          <div>
                            <div>
                              <div>
                                <div>
                                  <p class="MsoNormal"><span
style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif;color:black"><br>
                                      Cheers!<o:p></o:p></span></p>
                                </div>
                                <div>
                                  <p class="MsoNormal"><span
style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif;color:black">-dw<o:p></o:p></span></p>
                                </div>
                                <div>
                                  <p class="MsoNormal"><span
style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif;color:black"><o:p> </o:p></span></p>
                                </div>
                                <div>
                                  <p class="MsoNormal"><span
style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif;color:black">--<o:p></o:p></span></p>
                                </div>
                                <div>
                                  <p class="MsoNormal"><span
style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif;color:black">Derek
                                      Wuelfrath<o:p></o:p></span></p>
                                </div>
                                <div>
                                  <p class="MsoNormal"><span
style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif;color:black"><a
href="mailto:dwuelfrath@inverse.ca" moz-do-not-send="true">dwuelfrath@inverse.ca</a> ::
                                      +1.514.447.4918 (x110) ::
                                      +1.866.353.6153 (x110)<o:p></o:p></span></p>
                                </div>
                                <div>
                                  <p class="MsoNormal"><span
style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif;color:black">Inverse
                                      inc. :: Leaders behind SOGo (<a
                                        href="https://www.sogo.nu/"
                                        moz-do-not-send="true">www.sogo.nu</a>),
                                      PacketFence (<a
                                        href="https://www.packetfence.org/"
                                        moz-do-not-send="true">www.packetfence.org</a>)
                                      and Fingerbank (<a
                                        href="https://www.fingerbank.org"
                                        moz-do-not-send="true">www.fingerbank.org</a>)<o:p></o:p></span></p>
                                </div>
                              </div>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
        <div>
          <p class="MsoNormal"><br>
            <br>
            <o:p></o:p></p>
          <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
            <div>
              <p class="MsoNormal">On Nov 10, 2017, at 11:39, Ken
                Gaillot &lt;<a href="mailto:kgaillot@redhat.com"
                  moz-do-not-send="true">kgaillot@redhat.com</a>&gt;
                wrote:<o:p></o:p></p>
            </div>
            <p class="MsoNormal"><o:p> </o:p></p>
            <div>
              <p class="MsoNormal"><span
                  style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif">On
                  Thu, 2017-11-09 at 20:27 -0500, Derek Wuelfrath wrote:<br
                    style="font-variant-caps:
                    normal;text-align:start;-webkit-text-stroke-width:
                    0px;word-spacing:0px">
                  <br>
                </span><o:p></o:p></p>
              <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
                <p class="MsoNormal"><span
                    style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif">Hello
                    there,<br>
                    <br>
                    First post here but following since a while!<o:p></o:p></span></p>
              </blockquote>
              <p class="MsoNormal"><span
                  style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif"><br>
                  Welcome!<br>
                  <br style="font-variant-caps:
                    normal;text-align:start;-webkit-text-stroke-width:
                    0px;word-spacing:0px">
                  <br>
                </span><o:p></o:p></p>
              <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
                <p class="MsoNormal"><span
                    style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif"><br>
                    Here’s my issue,<br>
                    we are putting in place and running this type of
                    cluster since a<br>
                    while and never really encountered this kind of
                    problem.<br>
                    <br>
                    I recently set up a Corosync / Pacemaker / PCS
                    cluster to manage DRBD<br>
                    along with different other resources. Part of theses
                    resources are<br>
                    some systemd resources… this is the part where
                    things are “breaking”.<br>
                    <br>
                    Having a two servers cluster running only DRBD or
                    DRBD with an OCF<br>
                    ipaddr2 resource (Cluser IP in instance) works just
                    fine. I can<br>
                    easily move from one node to the other without any
                    issue.<br>
                    As soon as I add a systemd resource to the resource
                    group, things are<br>
                    breaking. Moving from one node to the other using
                    standby mode works<br>
                    just fine but as soon as Corosync / Pacemaker
                    restart involves<br>
                    polling of a systemd resource, it seems like it is
                    trying to start<br>
                    the whole resource group and therefore, create a
                    split-brain of the<br>
                    DRBD resource.<o:p></o:p></span></p>
              </blockquote>
              <p class="MsoNormal"><span
                  style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif"><br>
                  My first two suggestions would be:<br>
                  <br>
                  Make sure that the systemd service is not enabled. If
                  pacemaker is<br>
                  managing a service, systemd can't also be trying to
                  start and stop it.<br>
                  <br>
                  Fencing is the only way pacemaker can resolve
                  split-brains and certain<br>
                  other situations, so that will help in the recovery.<br>
                  <br>
                  Beyond that, the question is what log messages are
                  there from around<br>
                  the time of the issue (on both nodes).<br>
                  <br>
                  <br style="font-variant-caps:
                    normal;text-align:start;-webkit-text-stroke-width:
                    0px;word-spacing:0px">
                  <br>
                </span><o:p></o:p></p>
              <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
                <p class="MsoNormal"><span
                    style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif"><br>
                    It is the best explanation / description of the
                    situation that I can<br>
                    give. If it need any clarification, examples, … I am
                    more than open<br>
                    to share them.<br>
                    <br>
                    Any guidance would be appreciated :)<br>
                    <br>
                    Here’s the output of a ‘pcs config’<br>
                    <br>
                    <a href="https://pastebin.com/1TUvZ4X9"
                      moz-do-not-send="true">https://pastebin.com/1TUvZ4X9</a><br>
                    <br>
                    Cheers!<br>
                    -dw<br>
                    <br>
                    --<br>
                    Derek Wuelfrath<br>
                    <a href="mailto:dwuelfrath@inverse.ca"
                      moz-do-not-send="true">dwuelfrath@inverse.ca</a> ::
                    +1.514.447.4918 (x110) :: +1.866.353.6153<br>
                    (x110)<br>
                    Inverse inc. :: Leaders behind SOGo (<a
                      href="http://www.sogo.nu" moz-do-not-send="true">www.sogo.nu</a>),
                    PacketFence<br>
                    (<a href="http://www.packetfence.org"
                      moz-do-not-send="true">www.packetfence.org</a>)
                    and Fingerbank (<a href="http://www.fingerbank.org"
                      moz-do-not-send="true">www.fingerbank.org</a>)<o:p></o:p></span></p>
              </blockquote>
              <p class="MsoNormal"><span
                  style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif">--<span
                    class="apple-converted-space"> </span><br>
                  Ken Gaillot &lt;</span><a
                  href="mailto:kgaillot@redhat.com"
                  moz-do-not-send="true"><span
                    style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif">kgaillot@redhat.com</span></a><span
style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif">&gt;<br>
                  <br>
                  _______________________________________________<br>
                  Users mailing list:<span class="apple-converted-space"> </span></span><a
                  href="mailto:Users@clusterlabs.org"
                  moz-do-not-send="true"><span
                    style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif">Users@clusterlabs.org</span></a><span
style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif"><br>
                </span><a
                  href="http://lists.clusterlabs.org/mailman/listinfo/users"
                  moz-do-not-send="true"><span
                    style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif">http://lists.clusterlabs.org/mailman/listinfo/users</span></a><span
style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif"><br>
                  <br>
                  Project Home:<span class="apple-converted-space"> </span></span><a
                  href="http://www.clusterlabs.org/"
                  moz-do-not-send="true"><span
                    style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif">http://www.clusterlabs.org</span></a><span
style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif"><br>
                  Getting started:<span class="apple-converted-space"> </span></span><a
href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf"
                  moz-do-not-send="true"><span
                    style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</span></a><span
style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif"><br>
                  Bugs:<span class="apple-converted-space"> </span></span><a
                  href="http://bugs.clusterlabs.org/"
                  moz-do-not-send="true"><span
                    style="font-size:9.0pt;font-family:&quot;Helvetica&quot;,sans-serif">http://bugs.clusterlabs.org</span></a><o:p></o:p></p>
            </div>
          </blockquote>
        </div>
        <p class="MsoNormal"><o:p> </o:p></p>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
Users mailing list: <a class="moz-txt-link-abbreviated" href="mailto:Users@clusterlabs.org">Users@clusterlabs.org</a>
<a class="moz-txt-link-freetext" href="http://lists.clusterlabs.org/mailman/listinfo/users">http://lists.clusterlabs.org/mailman/listinfo/users</a>

Project Home: <a class="moz-txt-link-freetext" href="http://www.clusterlabs.org">http://www.clusterlabs.org</a>
Getting started: <a class="moz-txt-link-freetext" href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a>
Bugs: <a class="moz-txt-link-freetext" href="http://bugs.clusterlabs.org">http://bugs.clusterlabs.org</a>
</pre>
    </blockquote>
    <p><br>
    </p>
    <pre class="moz-signature" cols="72">-- 
Digimer
Papers and Projects: <a class="moz-txt-link-freetext" href="https://alteeve.com/w/">https://alteeve.com/w/</a>
"I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops." - Stephen Jay Gould</pre>
  </body>
</html>