Recent Changes - Search:

Project Status

Project Documents


Bug Tracking

edit SideBar


Photizo GP (Generate Page, New)

Imported python modules:

+ cPickle: converts Python object hierarchy into a byte stream and vice versa (written in C)
+ the optparse module: used to command-line options
+ datetime: for date math
+ stmplib: to send mail (with notifications of communications outages or error messages)
+ sys: used to parse command-line arguments, and access to variables and functions that interact with the interpreter, for example sys.argv
+ traceback: functions used to trace errors for error messages
+ time: to parse the timestamps
+ warnings: for error messages
+ kid: template language for XML based vocabularies, used to generate the diagnostics and station pages
+ pytz: world timezone definitions for python

imported photizo modules:

+ common.config: reads the ini file
+ DSI: Data Storage Interface - backend agnostic interface to photizo data storage
+ common.unit_conversions: converts units from Celcius/metric to Fahrenheit/standard measurements
+ common.util:
* format: Converts a value to float (if not already) and formats it according to the decimal point value passed in
* listify: if a value is a string, returns a list with a single element. This prevents data-type run-time errors when reading in config/ini file values
* porList: name derived from "Period of Record List." Adds properties from the template module but behaves as a list object

def process_station():

Arguments passed to this function are the station name, station description, raw data and the template for the Conditions page or include page (if nested in a client web page/site).

Instantiate the config file (cfg) and retrieve the diag_fields from the config file; check the Main section of the config file for diagFields; if none, set to station name, descriptive name, time stamp and the battery voltage, panel temp, solar panel voltage - avg, which are also common name keys in the config file column maps.

Instantiate a station_diag_data dictionary. Instantiate a dictionary of readings for each station (station_data[latest_reading_ts]) to get the latest readings where battery voltage and panel temp are not blank.

Determine the time delay since the latest reading; subtract the reading time from the current time and calculate the fractional part of a day. This is then assigned to the timeDiff key of station_diag_data and station_info dicts.

if the config[Main] dictionary is not set to ‘diag only’, a condition (web) page must be generated: get the output directory - join the path to the output directory specified in the config file to the station output directory.

Instantiate porList by passing the last 24 hours of readings and the total number of readings for the station and reverse the list to get the latest readings first.

If the path to the output directory doesn’t exist, create it.

Get the html extension for the generated page; instantiate a station page template with keyword arguments for latest reading, last day readings, station info, conversion functions and format; write the result to the output directory, joined to the station and the file extension, in html-strict format. If the station include template is true (a config file setting), the procedure is to link the client web page to the appropriate iframe or virtual include file on our server.

If the try clause fails, print an error, which contains the station name and the error ‘e’ and which calls a traceback function that formats the exception output.

The diag fields will be read into a station diag data dictionary if they exist, and ignored if they don't.

Return the station_diag_data (dictionary object)

def send_email():

Function supports email notifications to personnel specified in the config file: arguments are source and destination, subject, message and host. Defines host as localhost, a msg string for the header, and a connect function to smtp and sendmail.

def send_messages():

Function supports the above email notifications: arguments are network, messages, emails, diagURL, and whether the notification is new or not. Assigns subject line and builds a dictionary of message (msg) information that will be read from program output and posted to the message, based on network (nslakes, cchrc etc).

def write_late_log():

Function reads the config file, calculates current time normalized to Standard Time, initializes the path to the log file based on the config file (one per network) and its header (timestamp, newest age in days, oldest age in days). Reads the timestamps from the latest readings dictionary, calculates the difference between the current time and the latest timestamp and appends that to the log.

def main():

Initializes dictionaries for diag data, current and new messages. Set up the command line parser to read in command line arguments such as the path to the ini file and verbose options, output the usual errors if the command line arguments provided are incorrect.

Initialize a dictionary for each network with the network name and data store location; get the list of stations and sort them in the order specified in the config file, by network project.

(Placeholder for Genshi, a Python library that provides an integrated set of components for parsing, generating, and processing HTML, XML or other textual content.)

Load the stations kid template for the network if the config file setting for diag-only is false. If generate-include is set to true load the include template.

Issue verbose output to view execution of the program by station (command line output).

Build data table dictionaries from Current Conditions files if they exist, otherwise default to the Imported files (both are listed in the ini file). Create list objects from those tables, show the tables used during program execution (command line output).

If a station specified in the config file is missing from the station order list, print a warning.

If the station_info dictionary of imported files exists, instantiate a data storage interface object.

Call the function process_station with the following arguments: station name, description, data, and the station page and include templates.

Write the latest timestamps to the log file if lateLog == True

If a station is missing its data files, issue a warning. If station has no data files, replace the timestamp on the Condition page with an offline message.

Check for stale data: if the time difference is greater than the data recency threshold specified in the config file, a message is created to be emailed.

Open the diag page for the network and write the diag data in station order (as listed in the config file), using the format module to format the numbers (days old, panel temp etc).

Send notification messages: initialize the path to station notices, load saved messages with the cPickle module or initialize a dictionary of saved messages (for future ones). If the saved messages dictionary has no key for the station, then all the messages are new. If the saved messages do not have a notice, then it is a new message.

Initialize saved messages as current messages and append them to email addresses. Send all messages every six hours and new messages hourly.

Issue an exception error if notifications cannot be sent. (:notoc:)

Edit - History - Print - Recent Changes - Search
Page last modified on April 01, 2010, at 12:23 AM