Photizo
The GWS/EEI Weather Data Analysis and QA/QC Project

264 - 265, if __name__ == “__main__”: calls the main function

114, def main(): declares the main function

115, import optparse: used to parse command-line options

116, import os: for better portability between platforms

117, import sys: provides access to variables and functions that interact with the interpreter, for example sys.argv

119, import configobj: reads the photizo ini file

121, import kid: template language for XML based vocabularies, used to generate the diagnostics and station pages

123, import photizo.DSI: reads the Data Storage Interface, used to locate and store station readings

124, from photizo.common.util import format: utility to convert values to floats and control decimal points

126, version number

128, all_diag_data = {} instantiates a dictionary that will hold the diag data

129, current_messages = {} instantiates an array that will hold station messages for stale data

131 - 136, set up the command-line parser: the usage variable specifies the input format (program name followed by path to ini file); the parser is instantiated to call a function with usage and version number arguments; verbosity options are added by declaring a function with the necessary arguments; parser defaults are set.

139, (options, args) = parser.parse_args(): parses the arguments

143 - 146: if no arguments are given, print a warning to stdout, call the parser.print_usage() function and exit

147, cfg = configobj.ConfigObj(args[0], file_error=True): get the config file

152 - 153: read the network name and the data storage location from the config file

156 - 157: get a list of stations from the config file and sort them

159 - 165: order for rendering stations in the diag template - read the station order from the config file; if it exists, look for groups of stations; if they exist, load them into the kid template

171, os.path.join(cfg[‘Main’][‘templateDir’], ‘stationPage.kid’)): join the path from the template to the station page (since the diag template links to station pages)

173 - 178: read the generate-include-page option from the config file; instantiate the station-include template by calling kid.load_template() with arguments - get the template directory path from the config file and join it to the station-include template path

179 - 180: if the above fails, no station include template is generated

182 - 188: for each station in the list, if the option is verbose, print the station name to stdout while processing

190 - 195: look for the current conditions data file suffixes from the Stations section of the config file (these are the tables/files that update the current conditions pages); then look in the Main section of the config file and if that fails, go back to the station section and look for all the data tables / files that are imported for that station (Import Data File Suffixes)

197 - 201: if the required tables are found, instantiate the variable; if the verbose option is used, print the table names

203 - 212: if the data filename base is found (the first part of the file name, which corresponds to the station name) and the station files are found, instantiate the ‘data’ variable; ‘data’ uses the data storage interface (DSI) to open the sql object (sqob) table from the path specified in the config file - string arguments passed to this function are the ‘?’ query operator followed by the network and station names which are read from the config file, and the table name(s).

214 - 219, d = all_diag_data[station]: instantiate d, a list of stations assigned to the all_diag_data dictionary (line 128); call a function where the station name, station info (instantiated on line 61 in the process_station function), data (line 203 above), the config file, the station page template and the station include template for each station are passed as arguments to the function.

219, data.close(): once the arguments/data have been passed to the function and stored in the variable, close it.

221 - 225: if there’s an I/O error, call an exception - for any station in the d list (line 214), print the timestamp / data file missing error for the station name / descriptive name; then close the data variable.

227 - 232: if the station does not have datafiles, show it’s offline by replacing the timeStamp with a message: ‘d’ is then set to a hash containing the strings to be printed with the associated variable names (station, station descriptive name)

234 - 242: check for stale data - looks for ‘DataRecency’ in the config file (threshold default 0.25), timeDiff in d, defined in the def process_station() function based on OS datetime features and whether it exists as a float; if the number evaluated is larger than the threshold in the config file and if the station doesn’t already exist in the ‘current_messages’ list, assign the station to the list under the ‘stale’ (nested) list and assign values that will be posted to the diag page as old data for that station: current_messages[station][‘stale’] = [‘Old Data (age in days)’, “%.2f” % allDiagData[station][‘timeDiff’]]

244 - 252, Genshi: not implemented

254 - 259: write to the kid templates - open the path to the output directory specified in the config file, join it to the diag file name specified in the config file; write to the diag kid template; join the path to the template directory specified in the config file to ‘stationDiag.kid,’ the file name for the diag template; instantiate baseStationInfo to use the station order in the config file for the diag page; instantiate a diag variable which will contain the all_diag_data dictionary and format / serialize the data on the diag page.

Functions required to support main():

line 13, def process_station(): uses arguments for station, station_info, station_data, cfg, and the station page and include templates, drawn from the config file or defined in the function.

line 16 - 21: import python modules - see top of page for os and sys modules; import datetime, time and pytz (the python timezone library) for date-time calculations; import traceback for error-catching

line 23 - 26: import the photizo DSI (Data Storage Interface, see above); import conversion functions from unit_conversions.py for temperature, speed and barometric pressure (metric to Imperial and vice versa); import format function from util.py to format floats; import porList (short for ‘Period of Record List’) class from util.py to format the readings for station pages; determine if station data and the DSI exist.

line 28 - 30: instantiate diag_fields; check the Main section of the config file for diagFields; if none, get the station name, descriptive name, timeStamp and the fields commonly known as diag fields: battery voltage, panel temp, solar panel voltage - avg, all common name keys in the config file column maps.

line 32: instantiate the station_diag_data dictionary

line 34 - 35: instantiate the latest reading for battery voltage and panel temp via a function that is assigned to a variable

line 37 - 43: function to determine the time delay since the latest reading; uses the time and datetime modules and assigns the returned values to time_diff; time_diff_days then calculates the time difference in days; time_diff values are appended as a list to the station_diag_data and station_data variables

line 45 - 47: if the config file supports it, instantiate the output directory; join the path to the output directory specified in the config file to the output directory for station_info (variable instantiated on line 78) - why?

line 49 - 52: instantiate last_day_readings; use porList as a function to format station_data[ts]?; arguments are station_data and its attribute, get_last_period(hours=24), and the length of station_data; then reverse the readings to get the last one first

54 - 55: if the path to the output directory doesn’t exist, create it

(:notoc:)