Clearquest Daemon

Overview

At a previous company I was asked to provide a mechanism for controlled checkins of code into release branches. It was decided not to use Rational's UCM Model since the company was small and it's needs were simple. Additionally the company wanted to be able to produce Release Notes depicting which bugs were fixed in the release in an automated fashion. They did not want to incur significant overhead when checking in code and wanted to tightly control which bugs went into which release branch.

Environment

The environment of this company was as follows:

• Small company - ~30 Engineers in Santa Clara, USA and ~20 in Shanghai, China
• All Windows shop
• Rational Clearcase LT
• Rational Clearquest
• Rational Multisite
• One main server serving both Clearcase and Clearquest
• Slow VPN WAN to Shanghai

Multisite and the Shanhai office were not initially rolled out but the design considered them nonetheless. Unfortunately Multisiting of the Clearquest database was ruled out as too expensive for our little startup company.

Requirements

The requirements for this Clearcase/Clearquest integration were as follows:

• Verify that all elements checked into a release branch were associated with a Clearquest defect intended for that release.
• Verify the defect was:
  • Owned
  • Only in certain states (Must be in Assigned or Resolved).
  • On the list of defects for this release.
  • Different release branches will have different lists.
• Allow for some branches to not require a defect number while those releases were in a state of "development".
• Defect numbers will be entered by the engineers as part of the comment. This process should allow multiple defect numbers per checkin.
• Provide a way to lock out checkins of defects for building.
• Provide a way to generate Release Notes for a release based on the defects fixed.

Assumptions

There were certain assumptions and other processes already put into place that assisted in the solution.

• All checkins that required a bug ID would have a label applied to them that consisted of the bug ID.
• When engineers were done checking in these labels would be locked so that further checkins for this bug were stopped.
• Engineers would be allowed to continue to work on the release branch while the release was building

Check In Trigger

• Controlled checkins would be done through a check in trigger that would make sure that the conditions were right to allow checkin to proceed.
• In order to retrieve data from Clearquest CQPerl was used.
• Initial testing of this trigger showed that it took a very long time to connect to the Clearquest database only to retrieve a bit of information. If many elements were to be checked in the opening and closing of the database made the checkins take a long time!
• Our sister lab in Shanghai, China would also participate in this process therefore the trigger must also must minimize wait time over the WAN.

A better method was needed...

Daemon

• In order to minimize database open/close times a daemon was developed that would hold the Clearquest database open and respond to requests for information through a socket.
• The daemon would return information about a bug ID to the caller. This drastically sped up the process for the Checkin Trigger.
• Additionally this general purpose daemon could be used in other ways (e.g. Web Page Based Release Notes).

CQPerl Problems

A good daemon process:

• Puts itself into Daemon mode
• Is Multithreaded. This means that it responds to a request and forks a child process off to handle the request so that the parent process can accept the next client.

Since, at the time, CQPerl was the only supported way to interface with Clearquest it had to be used. Because CQPerl is based off of ActiveState Perl a number of problems arose:

1. ActiveState Perl does not support calling setsid which is required to enter Daemon mode.
2. ActiveState Perl does not reliably handle signals. This mean that the parent process could not reliably catch SIGCLD deaths

As a result the Clearquest Daemon Process is not multithreaded. Since the company is small and requests relatively infrequent this was an acceptable limitation. Still when processing large lists of Release Notes and over the WAN the service would, at times, be unavailable.

SetSID

The question remained then, How does one go into daemon mode?

Here I resorted to using something that the company was already using - Cygwin.

Cygwin is a Linux emulation running under Windows. It is one of the most complete emulations I have found. We used it to build (gnumake) as well as many other things.

Cygwin has a program called cygrunsrv which allows you to daemonize any other process.

Multithreading

The problem with making the server multithreaded was harder to resolve. Code was written to perform multithreading but the unreliability of signal handling proved to be a problem that could not be easily overcome.

Options for a multithreading included:

• Figure out how to handle signals properly under ActiveState Perl. Research was done on ActiveState's forums and eventually the engineer for ActiveState Perl said that signals just can't be reliably done under Windows.
• Rewrite code into another language. The client/server could have been rewritten into another language that supported multithreading however much work had already been done on the daemon and a few clients, also written in Perl, would need to also be rewritten or interfaced to this other language

In the end it was decided since the demand on the server would not be that great, that a single threaded server would suffice.

Client/Server

Since this is a client server application the CQD Daemon was written as well as a CQC Client. A Perl module named cqc.pm was made to define the API for CDQ.

The test client, CQC, ended up being a useful command line tool to get information about a bug from Clearquest. A user could, for example, obtain the owner of a bug by simply doing:

    $ cqc 1234 owner
    swang
    $ cqc 1322 owner headline
    owner: jliu
    headline: Unable to modify ACLS that are created (observed during ACL tests)
    $
  

Trigger

A preop Checkin Trigger was created to:

• Make sure that a comment was specified
• Extract all bug IDs from the comment
• If the check in was on a release branch requiring bug ID checkin then the trigger would make sure:
• The bug ID existed in Clearquest, was owned and in the proper state.
• The bug ID label was not locked.
• The bug ID was listed in a file for that release branch (i.e. <release branch>.lst)

A postop Checkin Trigger would then create labels for the bug IDs and apply those labels to the checked in elements.

Release Notes

With the Clearquest Daemon satisifying requests and with the Checkin Trigger already relying on a flat file of bug IDs for a release, generating a web page of release notes merely involved some ordinary formatting of a web page and a calling of the daemon to supply Clearquest information in a tabular format.

Additionally web pages were created to allow addition of bug IDs to the release list

Since CQD returns all fields in the defect record a web page showing all details of a defect was also developed

And example of Release notes is shown here.