Date: Wed, 27 Feb 2013 11:22:26 -0500 (EST)
From: Ken Young 
To: cqi@cfa.harvard.edu, jzhao@cfa.harvard.edu, mgurwell@cfa.harvard.edu, aargon@cfa.harvard.edu
Subject: Chunk order when the SWARM correlator is active

Dear SMA Data Stakeholder,

   In current visibility data files, the chunks are written in
monotonically increasing order.   There can be missing chunks, because we
occasionally assign 0 chips/chunk to certain chunks in order to get higher
resolution elsewhere.   Unless someone objects, when the new correlator
comes on line, I plan to break this monotonicity.   For example, in
our most frequently used mode (4 GHz BW, one receiver) the chunks will
appear in the order

c1, s01, s02, ... s24, s49, s50, s25, s26, ... s48

The reason I want to do that is that the online system really considers
4 GHz mode (double bandwidth) to be a two receiver mode, because both
receiver pathways are active.   I have a kludge in dataCatcher which
causes the data to be written out as if 4 GHz mode was a one receiver, 48
chunk mode, but that kludge will prevent the chunks from being written in
a monotonic sequence when the new correlator comes on line (the exact
reason why the kludge breaks is because the upper 24 chunks must be
re-ordered since they undergo an additional lower sideband conversion
in the Bandwidth Doubler Assembly, which makes the frequency scale run
backwards).

Please let me know if the chunks being written in a non-monotonic order
will present hardships for you.   I can change the order, but it will be
quite a mess to do so.

Thanks,

Taco
Date: Wed, 27 Feb 2013 11:41:00 -0500 (EST)
From: Ken Young 
To: Mark A. Gurwell 
Cc: cqi@cfa.harvard.edu, jzhao@cfa.harvard.edu, aargon@cfa.harvard.edu, David Wilner 
Subject: Re: Chunk order when the SWARM correlator is active

On Wed, 27 Feb 2013, Mark A. Gurwell wrote:

> Charlie Qi will need to weigh in here, as the ordering my
> strongly impact data readin times.  Or maybe it won't I don't
> know...

I already checked with Charlie, because I had the same worry about readin
time.   But he's ok with this as long as all of the chunks for one rx,
baseline and sideband are written together, and as long as they all follow
the pseudo continuum chunk.

Taco

Date: Wed, 27 Feb 2013 15:06:16 -0500 (EST)
From: Ken Young 
To: Mark A. Gurwell 
Cc: cqi@cfa.harvard.edu, jzhao@cfa.harvard.edu, aargon@cfa.harvard.edu, David Wilner 
Subject: Re: Chunk order when the SWARM correlator is active

   I've now got a working version of dataCatcher which produces fake data
(an interpolated copy of s01) mimicking the output of the new correlator.
For our "continuum mode" double bandwidth setup, with 32 channels per
chunk on all old chunks, the file size is increased by a factor of 20.7.

Taco