Status for Andrew DeFaria

Clearquest 2003.06.15 now uses Code Pages to insure that data entered into Clearquest databases are correct. When incorrect data is encountered there is a problem. The question is, which code page should we use?

Chris had went through this decision when he migrated Clearquest data to San Diego. We are faced with this decision now by moving to 2003.06.15.

If you have ClearQuest databases that were created with previous versions of ClearQuest, they may contain data from a variety of code pages. When you set the ClearQuest data code page, the data in your databases is not converted to characters in the selected code page. If your database contains characters that do not map to the newcode page characters, data corruption will occur.

There seems to be 4 choices for code pages:

1. ASCII
2. Latin-1
3. Chinese - Simplified
4. Japanese

I'm going to assume 3 and 4 are non issues. This leaves 1 and 2. Chris went with #1 and I think that's a good choice because ASCII is a base code set. However going with #1 means we need to clean up data now as we already have non ASCII characters in the databases.

I believe if we went with #2 our non ASCII characters would not be a problem. However:

All Windows clients must run the same operating system code page, and that code page must match the ClearQuest data code page. If you have mixed-platform environments (both Windows and UNIX clients), or clients using different operating system code pages, you must set the ClearQuest data code page to 20217 (ASCII), which is the common character set of all code pages. An alternate usage model is to set the ClearQuest data code page to a non-ASCII value and require that all users with UNIX systems interact with ClearQuest only with the Webclient.

And

• If you set the ClearQuest data code page to a non-ASCII value, users can only modify data in that database from a Windows client running the same operating system code page. If the code pages do not match, the database is opened in read-only mode.
• If you set the ClearQuest data code page to a non-ASCII value, UNIX clients will have read-only access to the databases. (UNIX users can choose to use the only the Web client, which prevents data corruption if a non-ASCII data code page value is selected.)
• If you set the ClearQuest data code page to a non-ASCII value, invalid characters can still potentially enter the database without being detected by ClearQuest. For example, ClearQuest cannot validate text that you cut from an e-mail or Web page and paste into a database record. If the e-mail or Web page text contains characters outside of the ClearQuest data code page, the characters are corrupted during display and may show up as invalid characters (for example, a question mark (?) character).

Bottom line appears to be use ASCII and bite the bullet now by cleaning up data or use Latin-1 and avoid data cleanup now with potentially a larger clean up later.

Posted by at 6:32 PM | Permalink

• Discussed SJ/Irvine migration
• Drew up Clearquest Franchise plan
• Modifed CheckCodePage.pl to show the invalid characters in a word context
• Investigating Clearquest Data Code Pages

Clearquest Data Code Pages

It seems, in an effort to better support international charactersets, Clearquest is tighening its enforcement of data code pages. In practice here this means that the default character set of US-ASCII will not longer due. I've been scanning the data for invalid characters and we've got 'em. Oddly it's stuff like the apostrophe in worlds like "shouldn't" and the hyphen in phrases such as "this - or that". Now when I go in and replace them with typed versions it works OK. I can only think that this might be a result of copy and pasting from Microsoft Word, which tends to use such odd versions of such simple characters.

Audit_Log

Another problem raises its head in an odd way. After changing the odd apostrophe to a simple apostrophe and saving the record from the Clearquest client, I again get errors, this time in the Audit_Log. Seems that we have a hook script that basically captures the old and new strings of what had changed and logs them. Problem is, the old string has the old bad characters! Plus Audit_Log is not editable! So now we're stuck! Perhaps when we programmatically convert the database we will not hit a problem however I suspect that in order to add a record we need to validate it and the creation of Audit_Log is a function of Validate. If so we might be able to temporarily turn off the Create_Audit_Log function.

Posted by at 2:48 PM | Permalink

• Worked on mktriggers script to mktriggers for all regions, all vobs

Trigger Standardization

I have been trying to centralize and standardize triggers. The idea here is to be able to add all the triggers to all of the necessary vobs quickly and easily and to insure that triggers - implementors of policy - are consistently applied. As triggers are not replicated by Multisite it is essential that you can add triggers to a replicate if needed. For example, we need it to add triggers to the new replicas of the San Jose and Irvine vobs. Also new triggers and new vobs come into being and you usually want to make sure that all vobs have the appropriate triggers in force.

Note: For example, the vob /vobs/CommEngine is replicated from Irvine <-> San Jose. Irvine has a set of triggers applied to their replica and we have a different set.

Within Broadcom there are various vobs in various regions. This makes automation a little more difficult but not impossible. Some standardization of what triggers go where and how Clearcase gains access to the trigger code is necessary. One way I've seen companies accomplish this in the past was to create an administration vob and place trigger code in that vob. This vob would be replicated to all sites and trigger code made available through the use of a snapshot view in a well known location. Other administrative scripts can be shared in a similar fashion. Additionally we need to address differences between Windows and Unix.

Inconsistent paths

As it stands now triggers seem to be in a few places and I don't think that this places are kept in sync. As such I see inconsistencies that are confusing me. For example there are the following paths to triggers mentioned in /home/vobadm/scripts/mktrtype.sh:

       ccase-atla-1)
          NTPATH="\\\cc-atla-storage\ccase\script\trigger"
          UNIXPATH="/projects/ccase/script/trigger"
          ;;

       ccase-blr-1)
          NTPATH="\\\cc-blr-storage\ccase\script\trigger"
          UNIXPATH="/projects/ccase/script/trigger"
          ;;

       ccase-brsa-1)
          NTPATH="\\\cc-brsa-storage\ccase\script\trigger"
          UNIXPATH="/projects/ccase/script/trigger"
          ;;

       ccase-irva-2)
          NTPATH="\\\Fs-irva-37\ccase\script\trigger"
          UNIXPATH="/projects/ccase/script/trigger"
          ;;

       ccase-irva-4)
          NTPATH="\\\Fs-irva-37\ccase\script\trigger"
          UNIXPATH="/projects/ccase/script/trigger"
          ;;

       ccase-irva-tst)
          NTPATH="\\\Fs-irva-37\ccase\script\trigger"
          UNIXPATH="/projects/ccase/script/trigger"
          ;;

       ccase-mhtb-1)
          NTPATH="\\\cc-mhtb-storage\ccase\script\trigger"
          UNIXPATH="/projects/ccase/script/trigger"
          ;;

       ccase-peka-1)
          NTPATH="\\\cc-peka-storage\ccase\script\trigger"
          UNIXPATH="/projects/ccase/script/trigger"
          ;;

       ccase-rmna-3)
          NTPATH="\\\cc-rmna-storage\ccase\script\trigger"
          UNIXPATH="/projects/ccase/script/trigger"
          ;;

       ccase-sdoa-1|ccase-sdoa-2)
          NTPATH="\\\Fs-irva-37\ccase\script\trigger"
          UNIXPATH="/projects/ccase/script/trigger"
          ;;

       ccase-sj1-1)
          NTPATH="\\\cc-sj-storage\ccase\bse\script\trigger"
          UNIXPATH="/projects/ccase/bse/script/trigger"
          ;;

       ccase-tlva-1|ccase-tlva-1.il.broadcom.com)
          NTPATH="\\\cc-tlva-storage\ccase\script\trigger"
          UNIXPATH="/projects/ccase/script/trigger"
          ;;

As you can see the NTPATH varies where the UNIXPATH is largely the same (ccase-sj1-1 puts scripts under bse). Also, with ccase-rmna-3 it says /projects/ccase/script/trigger yet is currently using /projects/cc4/triggers and the differences between those two directories are many. Which one is correct? Why the bse directory for San Jose?

I assume that the reasons for the different NTPATH names is to avoid going over the WAN to get trigger script code. I also assume that this is mitigated on Unix by use of the automounter and yet we cannot say that UNIXPATH is always the same due to the difference in San Jose. I wonder if DFS could be used to provide a consistent, globally well known path to trigger code under Windows...

In any event the question that rises is how are these various repositories of code kept in sync if at all?

Inconsistent Application of Triggers

Normally there are a certain set of triggers that are implementing policy that are consistently applied to all vobs (or at least all vobs in a region). For example, usually empty branches are considered bad and to be avoided. Thus there is normally a trigger script applied to all vobs to avoid this. Or if an organization determines that rmelem should not be performed then it is enforced with a trigger on all vobs. Yet there doesn't seem to be any triggers consistently applied to all vobs. In fact many vobs have no triggers at all!

I see, for example, 3 different triggers for preop checkins: GIpreci, preci and test_preci. Triggers could be written that can be applied to any vob and then they act only if the conditions are such that they should do something. IOW for situations where they don't apply they simple exit.

There are also triggers that do nothing but allow people to proceed or not based on whether the user is on an "approved" list. They carry with them a long list of -nusers. I would think that that would be more difficult to maintain than for a trigger to be written that opens up a file of approved users and validates the user based on that. Then to add/change/delete users one merely needs to update the data file, not re-create a trigger in a vob and then have to worry about what other vobs need their triggers updated. Of course the trigger would need to deal with the issue of what is the global file pathname to that data file.

Posted by at 11:04 PM | Permalink