Converting .rtd to .ncdf

Steps:

1. Install LabVIEW on local machine at http://www.ni.com/download-labview/ (Installation may take a while, and students can get student version for free by contacting OIT, please see more information at http://www.colorado.edu/oit/software-hardware/site-licenses/labview).

2. Copy ConvertRTD.exe from breeze at /data/fielddata/WC_Info/Software to local machine using scp, FileZilla, etc..

3. Copy .rtd files of interest to local machine from breeze.

4. Run ConvertRTD.exe on local machine.

5. Choose converting one file only ("Rtd file path", option on top) or converting .rtd files in a folder ("Directory" and "Destination Folder", option at bottom), by clicking the switch in the middle. "Current folder" is an option on the bottom right once you are in the directory.

6. Click "START" to convert .rtd files into "Converted_xxx.rtd" files.

7. Copy "Converted_xxx.rtd" files back to breeze, say, /home/student/your_id. Make sure different WC rtd files are in different folders, say WC61's in one folder "/home/student/your_id/XPIA/WC61", to avoid making multiple .nc files of same time but from different WCs.

8. Copy "rtd_to_cdf" from /data/fielddata/WC_Info/Software to same directory, say, /home/student/your_id

9. Run "rtd_to_cdf" by typing "./rtd_to_cdf".

10.1 Type in directory containing "Converted_xxx.rtd" files, say, "XPIA/WC61"

10.2 Type in prefix, say "XPIA_"

10.3 Type in observed height levels of windcube, which are typically 40 to 220 m of 10 different heights, so type "10"

10.4 Type in time adjustments. If WC was taking measurements at UTC times, then type "0"

11. The program should say "Finished!" at the end and you can find .nc files in directory.

Written by Joseph Lee
10.50 14 April 2015

XPIA BAO WC61 & 68 Basic Maintenance and Troubleshooting

On 2015/04/03 between about 1500-1600 MDT, Paul Quelet serviced the WCv1 LiDARs running at the BAO during XPIA. Next day (2015/04/04), have checked both systems remotely. Seem to be taking more vertical data and have stronger cell signals.

WC61 (NCAR):

• 2149 45-55 seconds (approximately), wiped lens manually. Should be NaNs in data files.
• Lens wipe seems to have helped LiDAR remotely sense more vertical levels? (So far this seems to be the case...
• Still running very strong.
• Usually transferring data (although not always full file size).
• Strong Verizon dongle signal (usually ~3 bars).
• System requested Verizon software update upon login 2015/04/04. After saying "Yes", was not able to update, and said update will come back in two days. Should happen ~0830 MDT 2015/04/06?
• Sun cover on.

WC68 (CU):

• 2213-2215 installed WC49 cover on top of WC68. Put Verizon dongle upward to extend length and wiped lens off manually.
• Lens wipe seems to have helped LiDAR remotely sense more vertical levels? (So far this seems to be the case...
• Running well.
• Manually downloaded data from 2015/03/20 to 2015/04/03 through remote desktop connection.
• Has had trouble transferring data remotely or allowing TeamViewer login.
• 2015/04/04 0000 UTC FTP data transfer seemed much improved with dongle position change.

Yes, I am aware that this is the non-matching sun cover number on the WC. However, it was quite a feat of strength to get the cover out of the Leosphere box and transport it to the site alone.

Summary of XPIA AXYS Vindicator LiDAR Deployment & WC Operations, 30 March 2015

AXYS Vindicator LiDAR Deployment

• Yanis Gryshan led the way to install two AXYS Vindicator LiDARs just to the east of the LiDAR supersite at the BAO, across the dirt driving path from the two WC v1s from CU, the Halo scanning LiDAR, and the NRG/Leosphere v2 profiling LiDAR.
• The Vindicator lidars stand on Lundquist group WindCube flat platforms.
• Each system has a serial number, noted as "13" and "15", these being AXYS generation 3 systems.
• Single power cord run across the road with orange plastic fencing noting the placement. Power cable connections secured with electrical tape.
• 10-minute averaged data can be accessed in real time through AXYS user login website, which Yanis and colleagues will disburse.
• Data is distributed through cell network in real time. Also system has satellite connections available if cell network goes down.
• (Much can be done by AXYS remotely once systems are running.)
• Systems have installed GPS and compass for position can orientation that can be viewed on website also.
• (Technically, both Vindicators are facing directly south, but these adjustments can be made in post processing, and may show systems aligned property due to correction in the software according to Yanis Gryshan.)
• 1-second data is available on compact flash card inside electrical box. Access to the CF card should be via coordination with AXYS and Julie Lundquist.
• Data heights are 49, 60, 80, 100, 120, and 149.7m (where the start and ending heights are near the BAO tower sonic heights, but are dependent on the Vindicator system setup).
• On-site system login is very similar to a WindCube LiDAR where the system IP Address must be specified and a Laptop can be used for a remote connection with an ethernet (CAT 5) cable. Would need to obtain IP addresses for each Vindicator LiDAR from Yanis or AXYS.
• LiDAR pair is to be deployed ~2 months.
• Washer fluid and compressed air tank should last ~6 weeks without changing needs. These components wash off the optical apertures if they are obstructed or dirty.

AXYS Vindicator Next Steps

• Arrival of UMBC motion table on truck will be coordinated with BAO staff Dan Wolfe, safety officer Otto, Julie Lundquist, and Andy Clifton. UMBC should be bringing compressed air tanks so that Julie can return compressed air rental tanks to Broomfield ASAP. Note that two compressed air tanks should last the two lidars three months.
• AXYS LiDAR closer to the road ("15") will be deployed to the UMBC motion table for testing.
• To deploy, the LiDAR does not need to be powered down, but could be via the red turn switch on the west side.
• There is ~4.5m cable that can be extended out to keep the RS power cables connected between the systems.
• Care must be taken in taking the Vindicator onto the motion table as cords / tubes must be threaded out from the central stand to not be obstructed keeping the systems connected.

AXYS Vindicator Storage Boxes

• The AXYS Vindicator storage boxes are put back together and stored under the BAO middle shipping container next to the tower.
• The boxes are covered with tarps to keep dry from weather.
• The top and front panels are held in place only with two screws.
• Robertson drives (square heads) are required to seal / unseal these boxes.
• All remaining Robertson head screws are in sealed ziploc bags inside the AXYS boxes.
• Note: The Robertson (square head) bit still needs to be returned to the BAO since it goes with the AXYS tool set. (Might want to hold onto it until packing up the Vindicator LiDARs for opening the boxes.)

General Field Work Notes:

• Halo scanning LiDAR at the BAO LiDAR supersite was observed to be moving and scanning upon further examination. Julie Lundquist observed very little change in aperture orientation. Later, Paul Quelet confirmed the change of scanning head position. Hopefully, this system is working well.
• Created on breeze in /data/fielddata/XPIA/ the following folders: MGAUS_photos and BAO_LiDAR_supersite_photos, and AXYS_Vindicator_deploy_photos. These are the start of some folders where photos can be transferred, shared, and used.

CU / NCAR WindCube LiDAR Notes:

• Morning of 30 March 2015, Julie Lundquist manually wiped off the apertures of the WC LiDAR systems. Should see momentary loss of data on the date 30 March in MDT morning.
• As of 30 March, WC68 is about 58% full on the local instrument D: drive data storage. This is cautioned with yellow in the WindSoft software. (WC61 is only ~40ish % full, so going strong).
• WC61 continues its great performance. Did not make any changes to the system. Cover is on and Verizon cell dongle hangs vertically from entrance door, but still under sun shield cover. Usually has about 3 bars of Verizon 3G cell service.
• WC68 has been inconsistent in cell data connection for remote login or data transfer (TeamViewer or FTP). Usually only has 1 bar of Verizon 3G cell service.
• Testing of WC68 Verizon dongle on 30 March revealed stretching out the dongle to top of WC68 produced greater cell signal strength. When put away without sun shield, dongle was stretched more horizontally, so (update) 1-2 bars of Verizon 3G cell service is now occurring.
• WC68 continues to not transfer the full files in FTP. (Update: 31 March 2015.) WC61 did not transfer full files via FTP either.
• Could consider installing WC68 sun shield and hanging Verizon dongle vertically for better remote connectivity. 
• On site, Paul Quelet was able to remote desktop with ethernet cable to WC68. Transferred all current Alarm, .rtd, and .sta files. Copied them over to breeze.
• Changed WC_FTP.txt (script) on WC61 and WC68 to dates of 2015_03_3* so that only March 30 & 31 files are transferred.
• Must change WC61 and WC68 WC_FTP.txt scripts to 2015_04* at the beginning of April to transfer all April files.
• WC16 (new v2 system) needs more careful steps to remote desktop into to obtain 1-second data files. Also, requires FileZilla according to Welly Cobden from NRG to complete this data transfer. The manual must be read carefully here and executed on site.
• WC49 is unplugged completely. Still would need an external keyboard, monitor, and mouse ("old school" style) to troubleshoot problems with on board computer. (Not quite a priority at this point.)

XPIA WindCubes Internet Connections, Data Transfer, Availability, Data Viewer Update

1) Connecting via TeamViewer

WC61:

--Has been nearly immediate when trying to connect.

--Good speed rendering and data transfer if needed most of the time.

WC68:

--Goes online and offline in terms of being able to connect.

--Slow speed rendering once a connection is established. Fast connection maybe only one time per day.

2) Data Transfer:

WC61:

* Has transferred up to the most recent files

* Did not transfer the full size of files timestamped 19 March and 21 March via FTP.

* Sometimes the scheduled data transfer is interrupted multiple days in a row. Usually by 4 days past, the full files have been transferred. Current FTP script moves entire current month (e.g. 2015_03*) files, but could be more specific (e.g. 2015_03_2*) to reduce number of files transferred, especially with intermittent internet connection.

* Must modify at beginning of April to 2015_04* files.

WC68:

* Is three days behind FTP file transfer when examined on 26 March 2015.

* Two most recent days' relevant data (.sta files) are in the D:\DATA EOLID\ARCHIVE\ folder instead of the D:\DATA EOLID\DATA\ folder where the FTP transfer usually occurs from. This does not have a good work around yet, and should be noted for obtaining data files.

* Must modify at beginning of April to 2015_04* files.

3) Data Availability

• Both Wind Cubes have had trouble reaching remotely sensing altitudes between 140-220m for 22-24 March 2015.
• Difficult to say how this is affected by scatterers. PTQ does not see direct correlation with the HRDL data in terms of Aerosol Backscatter Signal Strength (http://www.esrl.noaa.gov/csd/groups/csd3/measurements/xpia/hrdl/).

4) DataViewer Update:

WC61:

--Since the 19 and 21 March data files were only 22 minutes and 30 minutes of data in the files, went to WC61 via TeamViewer, and was able to transfer the full sized data files directly.

--Included all data in DataViewer figures.

WC68:

--Since 22 & 23 March were not transferred yet to breeze, transferred the files from the \ARCHIVE\ folder

--Included all data in DataViewer figures.

http://breeze.colorado.edu/DataViewer/

5) Few Notes:

A) Will not change FTP scripts in the middle of an experiment. This is best done during a development phase.

B) There are other Windows FTP free softwares existing, but not worth exploring until after XPIA (WC operating systems are supposed to go to UNIX-based I believe.)

Corrections to Windcube updates

Thanks, Paul, for the summary in your post of 3/14/15. Some corrections are necessary.... the lidar supersite at the middle south guy wire consists of :
- 2 CU v1 Windcubes (WC49, WC 68)
- 1 NCAR v1 Windcube (WC61)
- 1 HALO photonics scanning lidar (not an AXYS system)
- 1 Leosphere v2 Windcube offshore 8.66 system (with motion compensation). Logging into that system is beautifully simple. Use the "laptop" ethernet port and then use a browser to go to "http://windcube/windweb". Please do not change any settings when accessing this system (currently it is set to collect data at 40, 50, 60, 80, 100, 120, 140, 150, 160, 180, 200, and 220m - nice to have 12 levels!). Here's an image of the windweb from the 8.66 manual:

The two AXYS systems (one for the ground, one for the motion table) are currently in boxes just east of the trailer in which the windcube storage boxes are stored near the base of the main tower. Current plan is for Yanis from AXYS to arrive on 30 March to install.

Julie's version of the status:
WC49:

• Operating but not likely collecting usable data. Most recent access (25 Feb) showed all NaN. Not ftp-ing data anymore since modem disconnected.
• Cannot log into this system with ethernet cable in person, perhaps because of a bad port.
• Previous alarm files noted laser errors ("Alarm: EDFA").
• To Do: Welly Cobden suggests take keyboard, mouse, and external monitor into field with side panel open to troubleshoot computer ports at some point. Julie attempted this on 13 March but brought USB keyboard, mouse, monitor cables. Need to bring old-school keyboard, mouse, and monitor cables. Not a priority since WC61 and WC68 functioning reasonably well.

WC61:

• Has been continuously taking data.
• Have intermittent difficulties logging in via TeamViewer (JKL: these challenges continue depending on modem connection, but seldom persist longer than an hour.)
• Welly suggested that noise was related to fan and overheating. Will not address until system goes to NRG in April

WC68: 

• More difficult to log into with Team Viewer probably due to modem issues. Data are being ftped to breeze (/data/ftp/incoming/instrument_data) every few days (scheduled daily but ftps fail if modem not connecting at just the right time). When logging in with Team Viewer, JKL sometimes gets a blank white screen but file transfer can still occur (hence my diagnosis of modem bandwidth issue)
• WC68 was "stopping" itself due to Drive issues (see /data/ftp/incoming/instrument_data/WC68_ALM files) that seem to have stopped since Welly's adjustment of drive accuracy tolerance (from 1 to 3) on 12 Mar. These drive errors are due to the motor aging (which is interesting as this motor has been replaced a few times).

The HALO scanning schedule involves periodic stares at the 50m tower sonic. Please try not to obstruct its view when visiting the WCs. Its schedule is:

1. 8-beam VAD once every 12 minutes (each scan takes ~40 sec), 
2. 50 m sonic stare for ~10 minutes once per hour,
3. Target sector scan (Erie HS Stadium light posts) once per day, 
4. Vertical staring the rest of the time. 
   

XPIA A2e BAO Windcubes Status Update

LiDAR supersite below inner south guy wire at the BAO is quite crowded with 3 CU Leosphere/NRG v1 Windcube LiDARs, AXYS scanning LiDAR, and a new Leosphere/NRG v2 profiling LiDAR installed on Thursday 12 March by Welly Cobden and Julie Lundquist.

Summary of past few days of WC v1 LiDARs:

WC49:

• Appears to be taking data right now...? (Confirm Julie?)
• Cannot log into this system with ethernet cable in person, perhaps because of a bad port.
• Previous alarm files noted laser errors.
• To Do: Welly Cobden suggests take keyboard, mouse, and external monitor into field with side panel open to troubleshoot computer ports at some point

WC61:

• Has been continuously taking data this week.
• Have had trouble logging into it sometimes with TeamViewer, but problem seems to have worked itself out.
• Intermittent sound coming from side had to do with temperature controlling the system. Would probably stop of the heat dessicant was removed, but will not as continues to run fine taking data.

WC68: 

• Logged on today (14 March 2015) and discovered the need for Verizon software update.
• Restarted TeamViewer, should be online just fine.
• Previous issue with the stage rotating had to do with a setting of how far off the stage can get over time as the optical head moves. Was set to a very low value. Welly Cobden changed it to a slightly higher value. Should hopefully remove optical head stoppage and scanner errors.
• As a result, some missing data from earlier this week (9 & 10 March 2015).

(Quelet)

Update: WC68 is working now. Just needed to login to TeamViewer again with the all Teamviewer Accounts login. This information is contained at the end of the updated WindCube procedures document which is on breeze in the /data/fielddata/XPIA/. (Quelet)