" /> Status for Andrew DeFaria: October 2, 2005 - October 8, 2005 Archives

« September 25, 2005 - October 1, 2005 | Main | October 9, 2005 - October 15, 2005 »

October 8, 2005

SJ VOB Move

The VOB move for San Jose went fairly well. Without the normal user load on the servers the dumping and loading process was much quicker. I've attached Jennifer's spread sheet and updated it to reflect which vobs we've moved and how long it took as well as what sort of reduction we got in the DB sizes. All moved vobs are tagged on ccase-sj1-7 and Clearcase is still off on ccase-sj1-1 - for now. We will turn that on before Jennifer and Chin start with their testing. Yet left to do is the Multisite chreplica and re instituting of the cronjobs, etc. - nothing that would imped users Monday morning as well as clean up of backup areas assuming we reach a go on the go/no go tomorrow afternoon.

I've also attached a tar image of the log files that we managed to capture for the dump and load process.

October 7, 2005

ctmerge

  • Started incorporating ctmerge and other old Clearcase/Clearquest oriented scripts in to adm vob
  • Investigated some Multisite issues WRT this SJ VOB move
  • Working with Shivdutt on copying VOB storage over to /projects/cc-test

October 6, 2005

PQA.pm/adm VOB

  • Converted Code Page routines into a PQA.pm Perl Module
  • Changed routines to check all fields. Clearquest returns character data for fields such as DATE_TIME fields. These record definitions are now complete and will be useful when it comes time to perform the conversion
  • Waiting for new schema to start coding/testing conversion process. Contacted David Shaw who had helped us last time to get the databases onto our test server
  • Finally get adm VOB working and am starting to add my stuff into this VOB in a controlled fashion. This is not just a place to dump all of our scripts rather it's a place to start centralizing our code in a manner consistent with proper software engineering principals (Structured coding, code reuse, generalization, object oriented design principals, etc.)

October 5, 2005

HTML Characters/VOB Distribution

  • Changed CheckCodePage to use HTML character equivalents
  • Derived a plan of redistributing VOBs between VOB servers based on VOB database size

HTML Characters

Decided to translate non US ASCII characters into their HTML equivalents. Therefore © changes to "©" For characters that have no handy name equivalent used the form of &#n; where n is the decimal number for the character.

VOB Distribution

Based on Jennifer's email and adjusting for size of DB here is how I'd split the vobs between ccase-sj1-1 and ccase-sj1-7. This distribution balances the vob database size between the two machines.

Proposed split of vobs
Vob ccase-sj1-1 ccase-sj1-7
\UCM-Projects 436.8  
\SetTop   4545.9
\magnum 442.7  
\BSEAV   452.1
\rockford 97.4  
\TestTools   15.4
\DVTSJ 17.5  
\TiVo   49.4
\LinuxSupport   18.8
\SetTopGI 377.9  
\SetTopMot 288.8  
\Ref_Linux_Kernel   1647.2
\BSE_MS 482.5  
\tmmsky   14.7
\nds 14.5  
\Mot_P3_TCMTC   0.2
\CommEngine 1506.7  
\BSSS   28.9
\softmodem 11.7  
\applications   8.4
\bsp 2.4  
\CP64_TC   21.2
\Mot_P3_TvMon 68.6  
\DSR207   5
\dsr207_tvmon 5.2  
\BcmLib_Dsr530   5
\BcmLib_Dsr550 11.1  
\DSR550P3_BSP   3.5
\Documentation   30.6
\SQA 317.9  
\Web 112.7  
\Test 30.3  
\Firmware   95
\echostarUK 483.9  
\kylin 53.9  
\CFE   25.2
\brcm_wince 1.1  
\Tools   7
\Motorola_lib 6.3  
\MOT_97320   10.5
\BcmLib_Dsr580 3.3  
\BcmLib_Dsr580_Venom2_P2   4.7
\BcmLib_Dsr500   11.1
\BCM_HAL 24.4  
\DVI3K 15.1  
\DViTV   3.1
\DSR550P3   21.6
\ArchiveSetTop 189.8  
\Dvtsw 296  
\TestPriv   3
\bxUCM_support_7315sc 4  
\HAL_test   1.8
\BCM_test 10.2  
\UCM-CQTest   31.1
\delivertest 0.5  
\UCMrmcomp   0.4
\UCM-Test 3.6  
\BSE-SYS   8.8
\A1 0.3  
\echostar 600.6  
\bknittel_web_pvob 0  
\bknittel_web   0
\ss-test 1145.7  
\last   0
\bxUCM_proj 3.4  
\bxUCM_support   11.3
\bxUCM_system 0.8  
Total 7067.6 7080.9

October 4, 2005

rgy_swtichover/Triggers

  • Responded to IBM Rational Support regarding rgy_switchover
  • Added prohibit_operation to trigger list
  • Instituted the evil twin trigger
  • Obtained Chris' CQ merge scripts and started looking in to that
  • Went back to analyzing the PQA CQ data for invalid characters

rgy_switchover

IBM Rational responded

Steven Chaves wrote:

Andrew,

Even though there is no documentation saying that about DHCP, UNIX, and Clearcase environment, I would agree with you that rgy_switchover is useless in your situation. It seems to have no problem if you have no Interop environment. I believe that documents show say that works for one platform not Interop environments.

My response to this was:

Where in the documents does it state that rgy_switchover is only supported in non interop environments?

I would think that it would be fairly common to have a Clearcase shop in which there are some Unix servers and many Windows clients - even Unix/Linux clients. You are saying that in such environments rgy_switchover is essentially broken in that it doesn't accomplish what it was intended to do.

I feel, but have not managed to proof yet, that if the Windows machine name resolved through DNS then rgy_switchover would work fine. Can you test this scenario? Create an environment where you have two Unix servers, one being the primary registry server and the other the backup registry server. Have 4 clients, 2 Unix and 2 Windows with DHCP assigned IP addresses. Configure 1 Windows machine with a machine name that resolves in DNS via nslookup to it's IP address. The other Windows client's machine name should not resolve in DNS. Same thing with the Unix machine, one resolves, one doesn't.

Then do rgy_switchover. I think you will find that all machines whose names resolve to IP addresses through DNS will switchover and all machines whose names don't resolve in DNS will fail to switchover.

If that's the case then the documentation should clearly indicate that rgy_switchover will fail on any machine whose name does not resolve to it's IP address in DNS.

Ray, why don't Windows machine names (e.g. my machine - ltsjca-adefaria) resolve in DNS using nslookup? I think it is possible to set it up so that Windows machine names resolve in DNS and are still DHCP assigned.

Prohibit Operation

Many companies add a trigger such that any new element created is immediately changed to be owned by vobadm. This way individuals do not own the element - vobadm does - which is closer to say "these aren't your elements - they are the company's". It also has the nice side effect of automatically disallowing certain potentially dangerous operations like rmelem from being done by non-owners. So then only vobadm can rmelems.

Here at Broadcom they take a different approach: Rather than changing ownership to vobadm they put a trigger on rmelem and rmver with an -nuser vobadm. I'm not sure I agree with not allowing users to rmver.

Luckily I was able to add -nusers vobadm to the "Type" line in triggers.dat and it was just passed along. We really should implement an "Options" line for additional options.

October 3, 2005

SJ Vob move/Triggers

  • Discussed how to best handle the upcomming SJ vob move
  • Added handling for UCMOBJECT triggers and the few additional triggers on the docs vob

SJ Vob Move

I got to thinking over the weekend more about this vob move and trying to tie it to why we are moving the vobs. If we are trying to gain performance then how do we know that this move will accomplish that? It occurred to me that we have not adequately identified the performance problem we are trying to solve. If performance is the issue then just changing architectures is not likely to solve that problem.

Lacking any real description of the performance problem the users are experiencing the best we can do is optimize performance for vob service. In general, Rational recommends that you do not over load a server with too many vobs. More specifically you need to be concerned about the total size of your vob databases. What you are trying to do is insure that you have enough memory to fit the databases of the most commonly used vobs.

The old Solaris machine has 1 CPU and 4 gig of memory. The new Linux box also has 4 gig of memory but 2 CPUs. Observation reveals that the Solaris machine is not CPU bound - increasing CPU horsepower or number of CPUs will not increase performance.

I feel the best course of action at this point would be to identify the commonly used vobs and separate them between the Solaris and Linux machines thus decreasing the load on both servers, increasing the overall amount of memory (8 gig - 4 on one and 4 on the other) and allow for parallelization. Additionally, only 1/2 of the data need move. Need to sell this to Jennifer and Chin.