PRP PROCESSING CHECKLIST   SOAR HOME    ARM OCEAN HOME    SOAR OPERATIONS    CRUISE CHECKLIST  BACK TO MAIN SOAR PROCESSING PAGE  

Edit Date: June 28,2002
Processing Started DATE:___________
 
Processing Completed DATE:___________

LVL0 PREPARATION AND ENTER RAW DATA 

CRUISE:___________________________ (e.g. mr00k3, called 'expname' below)
DATES:____________________________
JULIAN DAYS:______________________
PRP SERIAL NUMBER: ___________________(e.g. 07, called 'sn' below)
PRP CAL ID: _______________________ (e.g. 0107)
PRP HEAD SERIAL NUMBER: _______________(e.g. 465, called 'hdsn' below)
CRUISE START PORT: _________________________________
CRUISE END PORT: ___________________________________


PREPARATION
DOCUMENTATION:
1. BLUE FOLDER WITH CRUISE NAME
2. CALIBRATION DOCUMENT PRINT OUT FOR FOLDER
3. CRUISE REPORTS, PHOTOS, ETC.
 
___Open Matlab


___cd drive:\matlab\soarmatlab 'D' drive on Avalon.(MaryJane's computer)
     Some of us have written commands to execute this, but this will always work.

___SetupSoar to define "drive" and "expname"


___CreateExperiment to set up all files.

   (Creates the Param file by calling MakeParam.m at the end.)
   (Also moves the 6 cal files listed below to the proper directory and checks to make    sure that they are all there.)

___Edit the Param_expname.dat text file. Add any comments that might help.
   (% 'comments')
 
___CALIBRATION FILES ARE COMPLETE _______________ (x)
Be sure all these files exist in the Level0/Pxx/ folder
   ___INFO_sn_calid.txt          (calibration info file)
   ___PrpCal_sn_calid.mat        (calibration data)
   ___HdCal_hdsn_hdcalid.mat     (head data)
   ___PrpCal_sn_calid.pdf        (from the PrpCal processing)
   ___TOA_hdsn_hdcalid.dat       (Top of atmosphere irradiance for each channel)
    ___ZeAngErr_hdsn_hdcalid.mat  (zenith angle correction file.)

___Place DA files in the DA folder in Level0/P??/DA of the experiment you're working on.





LEVEL 0 NAVIGATION FILE FORMATS
This is a list of the Nav filename formats from each ship. Check to be sure they are correct.
 
___ NAVIGATION RAW FILES IN LEVEL 0 FOLDERS
Be sure the files and folders are correct and in level0 folder
 
RON BROWN
SCS: RADIOMETER_nnn.ELG
___ Create a folder called "scs" in the Lv0 folder of the cruise you're working on.
___ Move the *.elg files into this folder.

KAIMIMOANA
SCS: RADIOMETER_nnn.ELG

EXPLORER OF THE SEAS

___ Move the ExSeas data folders from the "expname_data" folder to the level0 folder of the experiment that you just created. They are BW, GPS1, GPS2, GYRO, PIR, PSP, SPEEDLOG.
___ Delete expname_data if all folders have been moved.
___ Open "Avalon D:\matlab\soarmatlab\perl\soar_FixInput.pl"
___ Edit this file as described on lines 20 thru 44 and then SAVE.
___ Open a DOS window and change directories to the folder where the PERL.exe file resides.
___ In the DOS window type Perl drive:\matlab\soarmatlab\perl\soar_fixinput.pl This will run the PERL routine to clean up the data.
___ ExMakeHoldover. This makes a holdover folder for the 8th day and moves over the extra data.
___ ExMergeHoldover. This combines this weeks data from Saturday with the first part of the same day that was part of the data from lastweek.
___ Below is an example of the file formats. Make sure that they are correct.
GPS: yyyyMMjjjhhGPS1GPRMC.dat
GPS2: yyyyMMjjjhhGPS2GPGGA.dat and yyyyMMjjjhhGPS2GPVTG.dat
GYRO: yyyyMMjjjhhGyroHxHDT.dat and yyyyMMjjjhhGyroPPPRP.dat

MIRAI

___Create folders called "Furuno" and "SOJ" in the Level0 folder of the cruise.
___Furuno: FurunoGPSGGA_YYYYMMDD.RAW
___Furuno: FurunoGpsVTG_YYYYMMDD.RAW
___Soj: MiraiSOJ_YYYYMMDD.RAW

PALAU (No Navigation files here.)

___MakePalauNav


BNL ROOF(No Navigation files here.)

___MakeFixSiteNav


___Checkdafolder; to see that all da files are in place.

Print out the file drive\expname\"level0\ascii"\expname"_DaSummary.txt" and place in the cruise folder.









LEVEL 1 PROCESS LEVEL 0 RAW DATA TO LEVEL 1
Use the master list of field names
 
PROCESS NAVIGATION DATA


EXPLORER OF THE SEAS
  ___cd exseasnav
  ___explorersnav(Runs the next 6 functions).
     ___Gps1GPRMC_read(yyyy,[jd1:jd2])
     ___Gps1GPRMC_avg(yyyy,[jd1:jd2])
     ___GyroHxHDT_read(yyyy,[jd1:jd2])
     ___GyroHxHDT_avg(yyyy,[jd1:jd2])
     ___GyroPPPRP_read(yyyy,[jd1:jd2])
     ___GyroPPPRP_avg(yyyy,[jd1:jd2])


MIRAI
   ___CD miraimatlab : go to the Mirai matlab folder
   ___RARead_GGA: Convert raw Furuno GPS to GpsGga.mat(about 10 MB)
   ___GgaLevel0ToLevel1
   ___SojRead
   ___SojLevel0ToLevel1
   ___RSRead_vtg
   ___VtgLevel0ToLevel1



KAIMIMOANA
    ___CD drive\matlab\soarmatlab\kaimatlab\
    ___SetupLevel0Scs
    ___ReadScs
    ___ScsLevel0toLevel1


RONALD BROWN
    ___CD drive\matlab\soarmatlab\rbmatlab\
    ___ReadScs_RB
    ___AvgRbScs


PROCESS DA0 FILES


   ___SMV, then select 'D'

   ___SetupDa0

   ___Da0Read;Makes 3-sec mat files
      Note: if a time shift occurs and you need to make a shift in the time base,
      this is the place to do it. In this case call the function as
      da0read(yyyy,jd,delta_dt) where delta_dt is the time shift in days.
      For example if the time base must be shifted back one hour,
      then delta_dt = -1/24;
   ___Enter time shift here_____________ if needed.
   ___Confirm time base is accurate. Plots are automatically generated.
   ___DA0Average








LEVEL1 TO LEVEL2 EDITING

___Edit level 1 data using:
   i=JJJ; t1=jdf2dt(2001,i); t2=t1+10; level1to2edit('VAR','ID',t1,t2);

   Where JJJ = cruise start Julian day, VAR = variable to edit(PSP, PIR,etc.
   and ID = variable identifier(P05, gps1,soj,scs,etc.).

   PRP(ID = P??):(PRP is done for ALL cruises)
      psp, piru, tcase, tdome, pitch, pitch_std, roll, roll_std, az, thead, shadow


   EXPLORER OF THE SEAS:
      gps1: lat, lon, sog, cog, magvar
      Gyro: hdg, pitchgyro, rollgyro


   Kaimimoana:

      scs: cog,hdg,lat,lon,sog

      imet(ID='scs'): lw,rh,sst,sw,tair,wdr,wdt,wsr,wst


   RON BROWN:

      imet: wsr, wdr, wst, wdt, ta, rh, psp, sst5
      scs: lat, lon, hdg, cog, sog


   Mirai:
      GPS: cog, lat, lon, sog
      SOJ: lat, lon, logspd, hdg,
      SOJ(IMET): bpsl, depth, ppt, rh, sal, sst, sw, tair, wdr, wdt, wsr, wst.


___Missing COG or SOG? MakeSogCog will create level2 COG and SOG files from llevel2 lat and lon files.
 
___ Fill Dock Checklist can be used if the ship was sitting at the dock for several days with Nav data.

___Infocreate Recreates the info file to include Nav data.
    CHECK THE INFO FILE
___Print a copy of info file for the folder.


PSP AND PIR EXTRA PROCESSING


___SMV Then select (s) To clear the info global buffer.


___Smoothing This executes the next 7 scripts.


   ___1. MakePir-- makes pir from (piru, tcase, tdome)


   ___2. PspZero -- removes extraneous negative values and nightime values
       <-10 and >+10 W/m^2.Saves the result as level2/pspz.mat.


   ___3. MakeClearSky makes a clear sky theoretical time series, swclear.mat.
 
PITCH AND ROLL SMOOTHING
   ___4. SmoothPitchandRoll-- to make lp versions of pitch and roll mean tilt


   ___5. SmoothPitchandRollStd-- indication of motion


   ___6. PlotPitchAndRoll(Print this plot)
       Check Pitch and roll for accuracy.
       Use PlotPitchandroll.m to plot the entire cruise.
       Edit pitch and roll if necessary.
 
   ___7. MakeNavCruise to make the nav files in level 2.
 
For Explorer of the Seas only.
 These process the Explorer's radiation data and swaps with our broadband measurements.
  ___cd exseasrad

  ___ExplorersRad(Executes the next 7 functions).

     ___ReadExSeaPSP

     ___ReadExSeaPIR

     ___PspAverage

     ___PIRAverage

     ___ExSeaLevel1to2MoveRad

     ___SwapPsp

     ___SwapPir

AVG AND SUMMARIES HOUR AND DAILY AVERAGES AND MORE PROCESSING

 
___AllCruiseAvgs This computes hourly averages for: pspz, pir, piru, tcase, tdome.
 
___DayAverageRad produces local day averages of radiation data.
Print out the file ascii/expname_DailyRad.txt and put it in the processing folder.
 
___PirThermCompare
 
___PSPZSWCompare (Kaimimoana Only)
 
___AzGpsCompare(Not done for fixed sites)
 
___AzGyroCompare(Not done for fixed sites)
 
___GyroGpsCompare(Not done for fixed sites)
 
___TrackPlot(Not done for fixed sites)
 
___PlotDaysAll_Rad   for a specific time period, typically 10 days. Only for long cruises.
 
___PlotBarPspPirDaily makes ps file with daily averages.
 
___PlotSummary_rad   A complete cruise summary plot.
 

RSR PROCESS FRSR SWEEP DATA TO AOT


___ALL_PROCESS_FRSR_NEW (Runs the next 11 scripts in order.)
   ___ RdSweep
   ___ TwoMinSweep
   ___ TwoMinGlobal
   ___ HorizNew
   ___ DiffuseNew
   ___ NormalIrrNew
   ___ GlobalIrrNew
   ___ Level1to2MoveFrsr
   ___ AotComputeNew
   ___ AotFilterNew
   ___ AotCloudFilter
 
Note: if a time shift occurs and you need to make a shift in the time base, this is the place to do it. In this case call the function as rdsweep(yyyy, jd, det, delta_dt) where delta_dt is the time shift in days.
For example if the time base must be shifted back one hour, then delta_dt = -1/24;
______ enter time shift here if needed. Set det = 0 to process all detectors, and
call all jds in a loop.
 
____  PlotAotSummary;   or   PlotAotsummary(lim); where 'lim' is the AOT plot limit (e.g. 1.0)
 
NOTE:
If the results look suspect then run the EDIT script on the variables below and run the scripts after the edit section to recompute AOT and create the new plots. If the data looks OK then skip down to the section labeled"Archive Broadband Data".
___ Use Edit2('var'); to check the FRSR files at level 2.
Spot check these files as required.
 
g1__  g2__  g3__  g4__  g5__  g6__  g7__  
gerr1__  gerr1__  gerr1__  gerr2__  gerr3__  gerr4__  gerr5__  gerr6__  gerr7__  
n1__  n2__  n3__  n4__  n5__  n6__  n7__  
nerr1__  nerr1__  nerr1__  nerr2__  nerr3__  nerr4__  nerr5__  nerr6__  nerr7__  
d1__  d2__  d3__  d4__  d5__  d6__  d7__  
derr1__  derr1__  derr1__  derr2__  derr3__  derr4__  derr5__  derr6__  derr7__  
 
   ___ AotComputeNew
   ___ AotFilterNew
   ___ AotCloudFilter

AR-BB ARCHIVE BROADBAND DATA  

 
___ MakeAscii_rad.m
makes ASCII file for radiation data. Req's 'lat', 'lon', 'hdg','pitch','roll','pspz','pir','piru'
 
___ MakeSeabass_rad.m
Makes the correct SeaBass ASCII file for submission to SIMBIOS.
 
___ SUBMIT SEABASS FILES TO SIMBIOS
 
___ SUBMIT ASCII AND SEABASS FILES TO ARM DATA ARCHIVE
 
 

AR-RSR ARCHIVE SHADOWBAND DATA  

___ MakeAscii_frsr.m
Makes ASCII data file for ARM.
 
___ MakeSeabass_frsr.m
Makes seabass ASCII file for SIMBIOS.
 
___ SUBMIT SEABASS FILES TO SIMBIOS
 
___ SUBMIT ASCII AND SEABASS FILES TO ARM DATA ARCHIVE
 
 
 

WEB UPDATE THE WEB PAGES  

 
___ Run Level2Web.m to create the web pages. The procedure must run correctly.
 
___ run Level2Ftp.m to add the text files to the local ftp site.
 
Using CoffeeCup Editor or other web editor:
 
Edit web pages with cruise link at "www/cruises/platformname/expname/index.html"
____ cruises/platformname/platformname.html
____ cruises/platformname/expname/index.html and add the cruise description and links to the cruise.
 
Copy files to public servers:
 
___ Update these files on www.gim.bnl.gov web site.
___ cruises/expname/ folder.
___ cruises/index.htrml
___ soarops/index.html
___ soarops/SoarProcessing/cruisechecklist.html
 
Using a web browser:
 
___ Go to web/cruises/index.html.
 
___ Jump to this cruise name. The cruise page appears with the track plot.
___ Track plot okay.
___ Summary plot okay.
___ Info file okay.
___ Cal file okay.
 
___ Jump to the FTP site and check for these files:
___ expname_DailyRad.txt
___ expname_datareport.pdf
___ expname_frsr.txt
___ expname_info.txt
___ expname_rad.txt
___ expname_rad_dy.txt
___ expname_seabass_frsr.txt
___ expname_seabass_rad.txt
 
 

DR-d DRAFT DATA REPORT

 
_____ LaTex generator scripts.
Create the file of LaTeX commands for writing the data report.
_____TeXInfo.m    Makes the newcommands needed to run the data report
 
_____TexAotPlots.m    Makes the caption commands for all eps plots in the folder eps/aot.
 
_____ TexSummary.m    Makes captions for all summary eps plots.
 
___ Move the following folders and files to the Mac for publication.
 
_____ Convert all PS files to EPS. Follow the fields_files.html checklist.
 
_____ Move all EPS files to the datareport/eps folder.
 
 
 
_____ LaTex report generator.
 
_____ Print out the draft version.
 
_____ PDF version. Store in FTP site.
 
 

DR-Final FINAL DATA REPORT

 
 
 
 
 

CD FINAL CD ARCHIVAL      

Fri, Mar 30, 2001
 
R. Michael Reynolds      Brookhaven National Laboratory      reynolds@bnl.gov