defaria.com/blogs/Status/archives/000574.html

   1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
   2    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
   3 <html xmlns="http://www.w3.org/1999/xhtml" id="sixapart-standard">
   4 <head>
   5    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
   6    <meta name="generator" content="Movable Type 5.2.3" />
   7
   8    <link rel="stylesheet" href="http://defaria.com/blogs/Status/styles-site.css" type="text/css" />
   9    <link rel="alternate" type="application/atom+xml" title="Atom" href="http://defaria.com/blogs/Status/atom.xml" />
  10    <link rel="alternate" type="application/rss+xml" title="RSS 2.0" href="http://defaria.com/blogs/Status/index.xml" />
  11
  12    <title>Status for Andrew DeFaria: JVM Stack/Heap Sizes</title>
  13
  14    <link rel="start" href="http://defaria.com/blogs/Status/" title="Home" />
  15    <link rel="prev" href="http://defaria.com/blogs/Status/archives/000570.html" title="OMAPS.pm bug" />
  16    <link rel="next" href="http://defaria.com/blogs/Status/archives/000575.html" title="Load Balancing Redirection" />
  17
  18
  19
  20
  21
  22    <script type="text/javascript" src="http://defaria.com/blogs/Status/mt-site.js"></script>
  23 </head>
  24 <body class="layout-one-column" onload="individualArchivesOnLoad(commenter_name)">
  25    <div id="container">
  26       <div id="container-inner" class="pkg">
  27
  28          <div id="banner">
  29             <div id="banner-inner" class="pkg">
  30                <h1 id="banner-header"><a href="http://defaria.com/blogs/Status/" accesskey="1">Status for Andrew DeFaria</a></h1>
  31                <h2 id="banner-description">Searchable status reports and work log</h2>
  32             </div>
  33          </div>
  34
  35          <div id="pagebody">
  36             <div id="pagebody-inner" class="pkg">
  37                <div id="alpha">
  38                   <div id="alpha-inner" class="pkg">
  39
  40                      <p class="content-nav">
  41                         <a href="http://defaria.com/blogs/Status/archives/000570.html">&laquo; OMAPS.pm bug</a> |
  42                         <a href="http://defaria.com/blogs/Status/">Main</a>
  43                         | <a href="http://defaria.com/blogs/Status/archives/000575.html">Load Balancing Redirection &raquo;</a>
  44                      </p>
  45
  46                      <a id="a000574"></a>
  47                      <div class="entry" id="entry-574">
  48                         <h3 class="entry-header">JVM Stack/Heap Sizes</h3>
  49                         <div class="entry-content">
  50                            <div class="entry-body">
  51                               <ul>
  52   <li>Looked into JVM stack and heap sizes on dfls83-85</li>
  53 </ul>
  54                            </div>
  55                            <div id="more" class="entry-more">
  56                               <p>As you know there have been service interruptions in CQWeb. I keep looking at the logs for clues. About the only consistent thing is an error similar to this:</p>
  57
  58 <div class=code><pre>
  59 2006-09-28 00:10:20 Ajp13Processor[8009][15] process: invoke
  60 java.net.SocketException: Connection reset by peer: socket write error
  61         at java.net.SocketOutputStream.socketWrite0(Native Method)
  62         at java.net.SocketOutputStream.socketWrite(Unknown Source)
  63         at java.net.SocketOutputStream.write(Unknown Source)
  64         at org.apache.ajp.Ajp13.send(Ajp13.java:525)
  65         at org.apache.ajp.RequestHandler.finish(RequestHandler.java:495)
  66         at org.apache.ajp.Ajp13.finish(Ajp13.java:395)
  67         at org.apache.ajp.tomcat4.Ajp13Response.finishResponse(Ajp13Response.java:196)
  68         at org.apache.ajp.tomcat4.Ajp13Processor.process(Ajp13Processor.java:464)
  69         at org.apache.ajp.tomcat4.Ajp13Processor.run(Ajp13Processor.java:551)
  70         at java.lang.Thread.run(Unknown Source)
  71 </pre></div>
  72
  73 <p>Now "Connection reset by peer" could be an error that the process gets because the service has stopped so this could be more of a symptom than a cure. However searching for "Ajp13Processor socket write error" points me to <a href="http://mail-archives.apache.org/mod_mbox/tomcat-users/200203.mbox/%3C751828609.20020327090907@e-box.dk%3E">this
  74 post</a> which suggests increasing the stack and heap sizes for the JVM. Problems that are intermittent can be consistent with running out of stack or heap size.<./p>
  75
  76 <p>According to the Clearquest Web Administration Guide:</p>
  77
  78 <blockquote>
  79   <hr size="2" width="100%">
  80   <h3>Controlling Java VM Memory Consumption </h3>
  81
  82   <p>You can configure the memory consumption of Java processes used by New ClearQuest Web by adjusting the parameters in property files under the various components.</p>
  83
  84   <h4>Windows </h4>
  85
  86   <p>This section describes the configuration changes for New ClearQuest Web Java VM memory consumption for processes running on Microsoft Windows. To specify the VM memory consumption:</p>
  87
  88   <ol>
  89     <li>Open the appropriate configuration file for the New ClearQuest Web component whose memory consumption you want to reconfigure. For the ClearQuest Web application:
  90
  91       <table border="1" cellpadding="2" cellspacing="0" width="100%">
  92         <tbody>
  93           <tr>
  94             <th align="left" valign="top">Component</th>
  95             <th align="left" valign="top">Configuration file</th>
  96           </tr>
  97           <tr>
  98             <td valign="top">Apache Tomcat Server</td>
  99             <td valign="top">C:\Program
 100 Files\Rational\Common\rwp\bin\jk_service2.in.properties</td>
 101           </tr>
 102           <tr>
 103             <td valign="top">Rational Web Platform</td>
 104             <td valign="top">C:\Program
 105 Files\Rational\Common\rwp\bin\jk_service2.properties</td>
 106           </tr>
 107         </tbody>
 108       </table>
 109       <br>
 110     <p>For the ClearQuest server:</p>
 111       <table border="1" cellpadding="2" cellspacing="0" width="100%">
 112         <tbody>
 113           <tr>
 114             <th align="left" valign="top">Component</th>
 115             <th align="left" valign="top">Configuration file</th>
 116           </tr>
 117           <tr>
 118             <td valign="top">ClearQuest Request Manager</td>
 119             <td valign="top">C:\Program
 120 Files\Rational\ClearQuest\cqweb\cqserver\requestmgr_service.properties</td>
 121           </tr>
 122           <tr>
 123             <td valign="top">ClearQuest Registry Server</td>
 124             <td valign="top">C:\Program
 125 Files\Rational\ClearQuest\cqweb\cqregsvr\cqregsvr_service.properties</td>
 126           </tr>
 127         </tbody>
 128       </table>
 129     </li>
 130     <li>Modify the section shown below:<br>
 131       <br>
 132     <div class=code><pre>
 133 # # JVM Options #
 134 # Useful Options:
 135 # -Xms2m = Initial heap size, modify for desired size
 136 # -Xmx256m = Maximum heap size, modify for desired size
 137 # -Xrs = Available in Jdk1.3.1 to avoid JVM termination during logoff
 138 #
 139 wrapper.jvm.options=-Xrs -Xms2m -Xmx256m
 140     </pre></div></li>
 141   </ol>
 142   <hr size="2" width="100%">
 143 </blockquote>
 144
 145 <p>I looked at these config files on the three machines (dfls83-85) and they were pretty much set to the default:</p>
 146
 147 <div class=code><pre>
 148 <font color=blue><b>Ltx0062320:</b></font><u>for server in 83 84 85; do grep
 149 wrapper.jvm.options= //dfls$server/Rational/Common/rwp/bin/jk_service2*properties
 150 //dfls$server/Rational/ClearQuest/cqweb/cqserver/requestmgr_service.properties
 151 //dfls
 152 $server/Rational/ClearQuest/cqweb/cqregsvr/cqregsvr_service.properties;
 153 done</u>
 154 //dfls83/Rational/Common/rwp/bin/jk_service2.default.properties:wrapper.jvm.options=-Xrs
 155 -Xms2m -Xmx256m<br>
 156 //dfls83/Rational/Common/rwp/bin/jk_service2.in.properties:wrapper.jvm.options=-Xrs
 157 -Xms2m -Xmx256m<br>
 158 //dfls83/Rational/Common/rwp/bin/jk_service2.properties:wrapper.jvm.options=-Xrs
 159 -Xms2m -Xmx256m<br>
 160 //dfls83/Rational/ClearQuest/cqweb/cqserver/requestmgr_service.properties:wrapper.jvm.options=-Xrs<br>
 161 //dfls83/Rational/ClearQuest/cqweb/cqregsvr/cqregsvr_service.properties:wrapper.jvm.options=-Xrs<br>
 162 //dfls84/Rational/Common/rwp/bin/jk_service2.default.properties:wrapper.jvm.options=-Xrs
 163 -Xms2m -Xmx256m<br>
 164 //dfls84/Rational/Common/rwp/bin/jk_service2.in.properties:wrapper.jvm.options=-Xrs
 165 -Xms2m -Xmx256m<br>
 166 //dfls84/Rational/Common/rwp/bin/jk_service2.properties:wrapper.jvm.options=-Xrs
 167 -Xms2m -Xmx256m<br>
 168 //dfls84/Rational/ClearQuest/cqweb/cqserver/requestmgr_service.properties:wrapper.jvm.options=-Xrs<br>
 169 //dfls84/Rational/ClearQuest/cqweb/cqregsvr/cqregsvr_service.properties:wrapper.jvm.options=-Xrs<br>
 170 //dfls85/Rational/Common/rwp/bin/jk_service2.default.properties:wrapper.jvm.options=-Xrs
 171 -Xms2m -Xmx256m<br>
 172 //dfls85/Rational/Common/rwp/bin/jk_service2.in.properties:wrapper.jvm.options=-Xrs
 173 -Xms2m -Xmx256m<br>
 174 //dfls85/Rational/Common/rwp/bin/jk_service2.properties:wrapper.jvm.options=-Xrs
 175 -Xms2m -Xmx256m<br>
 176 //dfls85/Rational/ClearQuest/cqweb/cqserver/requestmgr_service.properties:wrapper.jvm.options=-Xrs<br>
 177 //dfls85/Rational/ClearQuest/cqweb/cqregsvr/cqregsvr_service.properties:wrapper.jvm.options=-Xrs<br>
 178  </pre></div>
 179
 180 </blockquote>
 181   <p>All these machines have 2 gig of main memory and largely just serve CQWeb. Indeed the CQWeb service processes are consuming most of the memory:</p>
 182
 183 <div align="center"><img title="dfls83"  src="/blogs/Status/images/dfls83.jpg" alt="" height="541"  width="524">
 184 <b>dfls83</b><br>
 185 <img title="dfls84" src="/blogs/Status/images/dfls84.jpg" alt=""  height="545" width="525">
 186 <b>dfls84</b>
 187 <br>
 188 <img title="dfls85" src="/blogs/Status/images/dfls85.jpg" alt="" height="577" width="528">
 189 <b>dfls85</b>
 190 </div>
 191
 192 <p>I think we should try setting at least the following:</p>
 193
 194 <div class=code><pre>
 195 wrapper.jvm.options=-Xrs -Xms128m -Xmx512m
 196 </pre></div>
 197
 198 <p>A restart of all CQ Web services would probably be needed for the changes to become effective. The above settings start off the jvm @ 128m for all 4 processes thus a total memory footprint of 512 Meg and limit each process to 512 Meg max for a total footprint of 2 Gig (when full). We might want to bounce this idea off IBM/Rational support to see if all 4 process should have the same settings or if we show vary
 199 them.</p>
 200                            </div>
 201                         </div>
 202                         <p class="entry-footer">
 203                            <span class="post-footers">Posted by  on September 28, 2006 11:53 AM</span> <span class="separator">|</span> <a class="permalink" href="http://defaria.com/blogs/Status/archives/000574.html">Permalink</a>
 204                         </p>
 205                      </div>
 206
 207
 208
 209
 210                   </div>
 211                </div>
 212             </div>
 213          </div>
 214       </div>
 215    </div>
 216 </body>
 217 </html>