1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
2 "http://www.w3.org/TR/html4/strict.dtd">
5 <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
6 <meta name="GENERATOR" content="Mozilla/4.61 [en] (Win98; U) [Netscape]">
8 <title>ClearSCM: Clearquest: Daemon</title>
10 <link rel="stylesheet" type="text/css" media="screen" href="/css/Article.css">
11 <link rel="stylesheet" type="text/css" media="screen" href="/css/Code.css">
12 <link rel="stylesheet" type="text/css" media="print" href="/css/Print.css">
13 <link rel="SHORTCUT ICON" href="http://clearscm.com/favicon.ico" type="image/png">
16 <script src="http://www.google-analytics.com/urchin.js" type="text/javascript">
18 <script type="text/javascript">
19 _uacct = "UA-89317-1";
25 include "clearscm.php";
35 <?php start_box ("cs3");?>
36 <h2>Clearquest Daemon</h2>
41 <p>At a previous company I was asked to provide a mechanism for
42 <i>controlled checkins</i> of code into release branches. It was
43 decided not to use <font class="standout">Rational's UCM
44 Model</font> since the company was small and it's needs were
45 simple. Additionally the company wanted to be able to produce
46 <i>Release Notes</i> depicting which bugs were fixed in the release
47 in an automated fashion. They did not want to incur significant
48 overhead when checking in code and wanted to tightly control which
49 bugs went into which release branch.</p>
51 <?php start_box ("cs2");?>
52 <b>Problem Statement:</b> Provide a mechanism for <u>controlled
53 checkins</u> and a way to automate Release Notes for releases.
58 <p>The environment of this company was as follows:</p>
61 <li>Small company - ~30 Engineers in Santa Clara, USA and ~20 in
64 <li>All Windows shop</li>
66 <li>Rational Clearcase LT</li>
68 <li>Rational Clearquest</li>
70 <li>Rational Multisite</li>
72 <li>One main server serving both Clearcase and Clearquest</li>
74 <li>Slow VPN WAN to Shanghai</li>
77 <p>Multisite and the Shanhai office were not initially rolled out
78 but the design considered them nonetheless. Unfortunately
79 Multisiting of the Clearquest database was ruled out as too
80 expensive for our little startup company.</p>
84 <p>The requirements for this Clearcase/Clearquest integration were as follows:
87 <li>Verify that all elements checked into a release branch were associated with
88 a Clearquest defect intended for that release.</li>
90 <li>Verify the defect was:
95 <li>Only in certain states (Must be in <font class="standout">Assigned</font>
96 or <font class="standout">Resolved</font>).</li>
98 <li>On <i>the list</i> of defects for this release.</li>
100 <li>Different release branches will have different <i>lists</i>.</li>
103 <li>Allow for some branches to not require a defect number while
104 those releases were in a state of "development".</li>
106 <li>Defect numbers will be entered by the engineers as part of the
107 comment. This process should allow multiple defect numbers per
110 <li>Provide a way to lock out checkins of defects for building.</li>
112 <li>Provide a way to generate Release Notes for a release based on
113 the defects fixed.</li>
119 <p>There were certain assumptions and other processes already put into place
120 that assisted in the solution.</p>
123 <li>All checkins that required a bug ID would have a label applied to them
124 that consisted of the bug ID.</li>
126 <li>When engineers were done checking in these labels would be locked so
127 that further checkins for this bug were stopped.</li>
129 <li>Engineers would be allowed to continue to work on the release branch
130 while the release was building</li>
133 <h3>Check In Trigger</h3>
136 <li><i>Controlled checkins</i> would be done through a check in
137 trigger that would make sure that the conditions were right to
138 allow checkin to proceed.</li>
140 <li>In order to retrieve data from Clearquest CQPerl was used.</li>
142 <li>Initial testing of this trigger showed that it took a very
143 long time to connect to the Clearquest database only to retrieve a
144 bit of information. If many elements were to be checked in the
145 opening and closing of the database made the checkins take
148 <li>Our sister lab in Shanghai, China would also participate in
149 this process therefore the trigger must also must minimize wait
150 time over the WAN.</li>
154 <p>A better method was needed<blink>...</blink></p>
156 <img src="BeforeCQD.jpg" border=0>
161 <li>In order to minimize database open/close times a daemon was
162 developed that would hold the Clearquest database open and respond
163 to requests for information through a socket.</li>
165 <li>The daemon would return information about a bug ID to the
166 caller. This drastically sped up the process for the Checkin
169 <li>Additionally this general purpose daemon could be used in
170 other ways (e.g. Web Page Based Release Notes).</li>
173 <img src="CQD.jpg" border=0>
175 <h3>CQPerl Problems</h3>
177 <p>A good daemon process:</p>
180 <li>Puts itself into <i>Daemon mode</i></li>
182 <li>Is <font class="standout">Multithreaded</font>. This means
183 that it responds to a request and forks a child process off to
184 handle the request so that the parent process can accept the next
188 <p>Since, at the time, CQPerl was the only supported way to
189 interface with Clearquest it had to be used. Because CQPerl is based
190 off of ActiveState Perl a number of problems arose:</p>
193 <li>ActiveState Perl does <b>not</b> support calling <font
194 class="standout">setsid</font> which is required to enter
195 <i>Daemon mode</i>.</li>
197 <li>ActiveState Perl does not reliably handle signals. This mean
198 that the parent process could not reliably catch <font
199 class="standout">SIGCLD</font> deaths</li>
202 <p>As a result the Clearquest Daemon Process is <font
203 class="standout">not</font> multithreaded. Since the company
204 is small and requests relatively infrequent this was an acceptable
205 limitation. Still when processing large lists of Release Notes and
206 over the WAN the service would, at times, be unavailable.</p>
210 <p>The question remained then, <b>How does one go into daemon mode?</b></p>
212 <p>Here I resorted to using something that the company was already
213 using - <a href="http://cygwin.com">Cygwin</a>.</p>
215 <p>Cygwin is a Linux emulation running under Windows. It is one of
216 the most complete emulations I have found. We used it to build
217 (gnumake) as well as many other things.</p>
219 <p>Cygwin has a program called cygrunsrv which allows you to
220 daemonize any other process.</p>
222 <h3>Multithreading</h3>
224 <p>The problem with making the server multithreaded was harder to
225 resolve. Code was written to perform multithreading but the
226 unreliability of signal handling proved to be a problem that could
227 not be easily overcome.</p>
229 <p>Options for a multithreading included:</p>
232 <li>Figure out how to handle signals properly under ActiveState
233 Perl. Research was done on ActiveState's forums and eventually the
234 engineer for ActiveState Perl said that signals just can't be
235 reliably done under Windows.</li>
237 <li>Rewrite code into another language. The client/server could
238 have been rewritten into another language that supported
239 multithreading however much work had already been done on the
240 daemon and a few clients, also written in Perl, would need to also
241 be rewritten or interfaced to this other language</li>
245 <p>In the end it was decided since the demand on the server would not
246 be that great, that a single threaded server would suffice.</p>
248 <h3>Client/Server</h3>
250 <?php start_box ("cs3");?>
251 <b>In depth:</b> Code listings for <a href="cqd.php">CQD
252 Daemon</a>, <a href="cqc.php">CQC Client</a> and <a
253 href="cqc.pm.php">cqc.pm</a>
256 <p>Since this is a client server application the <a href="cqd.php">CQD Daemon</a>
257 was written as well as a <a href="cqc.php">CQC Client</a>. A Perl module named
258 <a href="cqc.pm.php">cqc.pm</a> was made to define the API for CDQ.</p>
260 <p>The test client, CQC, ended up being a useful command line tool
261 to get information about a bug from Clearquest. A user could, for
262 example, obtain the owner of a bug by simply doing:</p>
264 <div class="code"><pre>
267 $ cqc 1322 owner headline
269 headline: Unable to modify ACLS that are created (observed during ACL tests)
275 <?php start_box ("cs3");?>
276 <b>In depth:</b> Code listing for <a href="CheckinPreop.php">Check
280 <p>A preop <i>Checkin Trigger</i> was created to:</p>
283 <li>Make sure that a comment was specified</li>
285 <li>Extract all bug IDs from the comment</li>
287 <li>If the check in was on a release branch requiring bug ID checkin then
288 the trigger would make sure:</li>
291 <li>The bug ID existed in Clearquest, was owned and in the
294 <li>The bug ID label was not locked.</li>
296 <li>The bug ID was listed in a file for that release branch
297 (i.e. <release branch>.lst)</li>
301 <p>A postop <i>Checkin Trigger</i> would then create labels for the
302 bug IDs and apply those labels to the checked in elements.</p>
304 <h3>Release Notes</h3>
306 <?php start_box ("cs3");?>
307 <b>In depth:</b> Code listing for <a href="rn.php">Releasenote CGI
311 <p>With the Clearquest Daemon satisifying requests and with the
312 Checkin Trigger already relying on a flat file of bug IDs for a
313 release, generating a web page of release notes merely involved some
314 ordinary formatting of a web page and a calling of the daemon to
315 supply Clearquest information in a tabular format.</p>
317 <p>Additionally web pages were created to allow addition of bug IDs
318 to the release list</p>
320 <p>Since CQD returns all fields in the defect record a web page
321 showing all details of a defect was also developed</p>
323 <p>And example of Release notes is shown <a
324 href="Releasenotes.html">here</a>.</p>
328 <?php copyright ();?>
331 <script language="JavaScript" src="/JavaScript/Menus.js" type="text/javascript"></script>