- Worked out a plan for how to perform the PQA Merge this weekend
- Concerned about the size discrepancy in the database I wrote a little utility, check_attachments, to total up the size of all attachments in all the databases. This turned up a bug in pqamerge which I fixed. Reperforming pqamerge
- There is a problem with using clearprompt to prompt for list input for bin_merge - You can only have one line of prompting text and only 50 characters in that line. It's gonna be hard to describe a binary merge situation in 50 characters or less. Investigated making a PerlTk list dialog. In theory it can be done as ccperl does support PerlTk.
- Discussed with Naga the email link issue with PQA
TransferAttachments
I knew there was a good reason why this was eating at me. There is indeed a bug. My attachments transfer routine was not getting called at all for Prod! I wrote a small Perl script to check this and it yielded:
$ cqperl check_attachments Totaling attachments in TO... Totaling attachments in Prod... Totaling attachments in Cont... Total attachment size for TO = 272253137 Total attachment size for Prod = 2683569547 Total attachment size for Cont = 272253137
Notice that the size of the attachments for TO matches the size of attachments for Cont! IOW no Prod attachments got transferred at all!
Turns out I was only calling TransferAttachments when the field name was AttachmentsBRCM (and TransferAttachments then did all the attachments - both Attachments and AttachmentsBRCM). However, in Prod the field is named AttachmentBRCM - note the singularity here! I tell ya my eyes are going.
I'm changing my code not be dependent on the field name and to just call TransferAttachments for each record. TransferAttachments doesn't need to know the field name - it just does all of them.
Unfortunately this means that I have to delete all of the records currently in the Cont database on p4test. And it also means that the approximate 4 hour running time will probably increase. However this does explain the discrepancy in the database sizes.
Out of Disk Space
Shivdutt Jha wrote:Good job Andrew, another mystery is solved.
What is more telling is that we have 2.6 Gig of attachments!
In any event my merge failed with out of disk space. The pqamerge script uses the current directory to temporarily hold the attachments as the are transfered from one DB to the other. Turns out that Prod00010818 has two very large attachments, one 328485466 bytes and another 219095209 bytes. Unfortunately I was running in ~vobadm/My Documents and that disk filled. So another lesson learned - run pqamerge in a directory on a disk with lots of disk space! Alas this means I have to start this process over again (I'm trying to get a clean run and a timing of how long to expect the merge to run. pqamerge outputs how long it takes to do it work).