| Subject: | A report on handling large size SMA data in Miriad | from the interim correlator | Date: | September 26, 2012 | From: | Jun-Hui Zhao |
1. Test data files from recent testing SMA runs
120505_02:52:05 integration 1.3 sec # of int 13983 # of bsln 28 # of chan 6145 # of rx 1 size 20 Gbytes 120813_03:37:15 integration 1.3 sec # of int 48536 # of bsln 10 # of chan 6145 # of rx 2 size 50 Gbytes
2. Computer used
processor : vendor_id : GenuineIntel cpu family : 6 model : 44 model name : Intel(R) Xeon(R) CPU X5687 @ 3.60GHz stepping : 2 cpu MHz : 3591.170 cache size : 12288 KB physical id : 0 siblings : 4 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt aes lahf_lm ida arat epb dts tpr_shadow vnm i flexpriority ept vpid bogomips : 7182.34 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management:
free -m
total used free shared buffers cached
Mem: 23956 15682 8273 0 363 14282
-/+ buffers/cache: 1036 22919
Swap: 26207 176 26031
Red Hat Enterprise Linux Workstation release 6.3 (Santiago)
A hybrid test version: Miriad_CVS 4.3.4 and SMA beta 1.4.5
3. Summary of test
It takes 14.6 min to convert the data into Miriad with smalod with no errors from the Miriad uvio routines.
It takes 46.9 min to convert the uv data into Miriad with smalod by unable calling the relevant uv-routines for a few bad variable strings in the data which cause segfaults in Miriad uvio subroutines.
Error in parsing bsl header: 10 bls reported are inconsistent with 21 bls in the visibility plot. ...
Jun-Hui Zhao (2012-09-15) Astrophysicist at SAO 60 Garden St., MS 78 Cambridge, MA 02138 Tel: 617 496 7895 Email: jzhao@cfa.harvard.edu