Running MMCR Ingest Code at BNL
4/16/03
1. cd to ~/dev/ingest/src/mmcr
2. Populate DATA/data/collection/sgp/sgpmmcrC1.00/ with raw data to
ingest
For sgp, this would be
cd DATA/data/collection/sgp/sgpmmcrC1.00/
tar -xvf <path
to>/sgpmmcrC1.00.20000302.000000.raw.pop4_mom.tar
tar -xvf <path
to>/sgpmmcrC1.00.20000302.000000.raw.pop4_spc.tar
tar -xvf <path
to>/sgpmmcrcalC1.00.20000302.000000.raw.netcdf.tar
tar -xvf <path
to>/sgpmmcrmonC1.00.20000302.000000.raw.netcdf.tar
tar -xvf <path
to>/sgpmmcrpowC1.00.20000302.000000.raw.netcdf.tar
New names exist for raw files following 4/1/01 (I believe that is
the
date):
sgpmmcrC1.00.20011005.000000.raw.log.tar
(health, and d log files)
sgpmmcrC1.00.20011027.000000.raw.cal.tar (CalibrationTable
file)
sgpmmcrmomentsC1.00.20011002.000112.raw.mom.tar
(d and h files)
sgpmmcrcalC1.00.20011001.000108.raw.nc.tar (cal
files)
sgpmmcrmonC1.00.20011001.000150.raw.nc.tar
(mon files)
3. To account for the hour offset, if applicable for this day
(1999.12.17 - 2000.03.22:1800Z):
Move the last hour of data (2300, 2330) into
/export/pcArscl1/marchIOP/tmpHold
for the use of the next day to be ingested
Copy the last hour of the previous day into, for sgp, DATA/data/collection/sgp/sgpmmcrC1.00/
4. Be sure you are running the proper version of the
code.
The source in the file mmcr_ingest.c currently contains the generic
ingest.
If the hour offset is needed, copy the mmcr_ingest.c code
into
a temporary file name (e.g. mmcr_ingest.GENERIC.c), copy the mmcr_ingest.HOUROFFSET.c
code
into mmcr_ingest.c and do the make.
5. Run script to strip ARM prefixes off these raw data file names:
~/scripts/mmcrStripARMprefix.pl <dir with filenames to
strip>
The resulting file names should look like this:
YYYYMMDDHHMMSS######Cal.nc (don't really need this?)
dYYJJJa####.mom (or dYYJJJa####.spc)
hYYJJJa####.mom (or hYYJJJa####.spc)
monYYYYJJJHH####.nc
powYYYYJJJHH####.nc
6. Set TDBSERVER environment variable for tdb:
setenv TDBSERVER stratus
7. Run ingest (note: must run it while cd'd to test directory):
cd test
for SGP: debugRuntest
for NSA: nsaRuntest