Virga Server Overview
Data Downloading and Storage
Virga is an Windows 2008 server responsible for the collection and display of climate data through various means. Recall that data is collected from various sources through the Internet, as well as through the telephone network. The internet collection scheme is shown in Figure 1.
Figure 1.
Data is collected from these sources using a collection of scripts written in either Perl or Python and a software package from Campbell Scientific, called Loggernet. All data download scripts are stored on the D: drive of Virga under a directory called nmcc_lib. See Figure 2.
Figure 2.
Depending on the network data being downloaded, the data will be stored in a folder for the network under D:\webroot\raw_data\. For Example, if MRGCD data is being downloaded, then the raw data from MRGCD will be stored in D:\webroot\raw_data\mrgcd\. The data scripts will be discussed in later documentation. After this data has been downloaded, it is available for viewing by going to the following url: http://virga.nmsu.edu/raw_data/.
The telephone download scheme works in a similar manner. See Figure 3 for a visualization of the telephone download scheme. However, instead of using scripts to download the data, Loggernet must be used to contact and download station data. Depending on which network the station belongs to, Loggernet will download the data to the network folder under D:\webroot\raw_data\, as discussed earlier.
Figure 3.
Data is downloaded automatically, according to the schedule in the following table.
Time (MST) |
Operation |
12:15 AM |
Using Loggernet, Download the NM Climate Center station data through the phone network to the D:\webroot\raw_data\nmcc\ directory. |
01:05 AM |
Using Loggernet, Download the NM Vineyard Data (WINE) through the phone network to the D:\webroot\raw_data\wine\ directory. |
02:05 AM |
Using Loggernet, Download Dr. Salim Bawazir's data (EBID) through the phone network to the D:\webroot\raw_data\ebid\ directory. |
03:15 AM |
Download METAR data via the Internet, using scripts in the D:\nmcc_lib\ directory. Data is downloaded to D:\raw_data\metar\. |
03:15 AM |
Download UPR data via the Internet, using scripts in the D:\nmcc_lib\ directory. Data is downloaded to D:\raw_data\upr\. |
03:15 AM |
Download CWOP data via the Internet, using scripts in the D:\nmcc_lib\ directory. Data is downloaded to D:\raw_data\cwop\. |
03:15 AM |
Download SNOTEL data via the Internet, using scripts in the D:\nmcc_lib\ directory. Data is downloaded to D:\raw_data\cwop\. |
03:15 AM |
Download RAWS data via the Internet, using scripts in the D:\nmcc_lib\ directory. Data is downloaded to D:\raw_data\cwop\. |
08:35 AM |
Download LANL data via the Internet and repeat downloading every 15 minutes indefinitely. Stores data in D:\raw_data\lanl\ directory. |
Table 1.
Data Processing
The data is process in the schedule task program
From start go to the schedule task. Click on + sign to open task list. Go to NMCC tasks. In the do process look at tasks to find the program that is call. It is a phython script that copies the data from d:webroot to the infile for the current year and then call the program from the cns to process the in files to the cns data file.
Adding the LoggerNet to the start up menu
To add the LoggerNet to the folder C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup then change the label for the new start up menu.
Todo
How to mount all share drives automatically when server is restarted?