One of the ways I have been occupying myself is with flight training for a private pilot license or more correctly called a private pilot certificate. It's been off and on lately because I have had to take some time off to focus on other projects, but next month I will be resuming at as fast of a pace as my pocketbook will allow. Since I needed some pilot/flying related activities to keep my interest during this down time I decided to write a Perl parsing script for aviation weather reports called METAR's. But, first how did I find myself in the left seat of an airplane for the first time? Well, read on.
About a year ago as a birthday present I was given a "discovery flight" by my parents. A discovery flight is where a certified flight instructor sticks you in the pilot seat, guides you to take off and fly a plane around for about an hour before landing at the airport where you took off from. This is akin to the "visitation" rooms at the humane society where they let you play with the puppies and cats, all the while knowing that the puppy ( or aircraft, rather ) that you have been playing with is soon to become your highest priority.
That's how it happened with me anyway. On my discovery flight I pushed the throttle forward. We soon accelerated to 50 knots and when I pulled the yoke toward my chest, the Cessna 152 parted the runway and I became speechless; my breath had been taken thru amazement of the reality of flight. It is one thing to understand the physics of flight and totally different to experience it. General aviation or small aircraft flight isn't anything like the experience of large commercial airplanes.
In fact, I was so taken by this flying that I decided to continue flight lessons, working towards a private pilot certificate. As I mentioned above I have experienced quite a few training delays along the way including taking time off to start Lakeside Electronics, LLC, so in the interim I wanted to come up with some activities related to flying.
Early on in my flight training I recognized the impact and importance of weather on general aviation or I should say, aviation in general. There are even aviation specific weather reports such as METARs that pilots access for preflight weather planning. You can look up a report for an airport near you using their ICAO station identifier code here http://aviationweather.gov/adds/metars/.
Here is an example METAR string from the airport I fly out of in Ann Arbor, Michigan: KARB 021653Z 34015G23KT 10SM OVC033 04/M03 A2996 RMK AO2 SLP150 T00441033.
As you read the above report you can probably make out what some of the sub-strings mean. For example, before researching I postulated that " OVC033 " meant overcast, but I did not fathom that the second half was referring to cloud height in 100's of feet. Could " SLP150 " be something to do with slip? Nope, Sea Level Pressure - in tens, units and tenths to be added to 1000 hPa no less. In other words, 15.0 hPa + 1000 hPa = 1015.0 hPa. However, if the string were a high number like SLP965 you would add that value ( 96.5hPa ) to 900 hPa instead for a reading of 996.5 hPa. Yes, its a strange little protocol...
Almost certainly I needed to study up on how to interpret this METAR report. Fortuitously, it would seem, I had concurrently begun learning the programming language Perl. A perfect storm, it would seem, to write a Perl based parsing script for METAR weather reports and display these data in a more human readable format was afoot.
The goal of course is two fold. A) To learn Perl and B) to memorize a method of METAR interpretation for when I need to read the raw string. Quite obviously there are numerable ways to accomplish these tasks. There also exists METAR decoders and even a Perl Module for this task, but this would violate my goals as set forth and thus decided to write my own Perl METAR parsing script.
When I turned to Google for information about the METAR protocol I found this page http://www.met.tamu.edu/class/metar/quick-metar.html helpful indeed. If you recall, we discussed the sub-string for sea level pressure above. There are too, other little gotchas along the way when trying to programmatically decode METAR reports. For instance, some data are always reported, some data are optionally reported and within the raw string as a whole, the sometimes reported data is intermingled with the always reported data. I also have reason to believe that the information contained in the remarks ( RMK ) section can partially exist as plain English, and so that is where my parsing stops - before we get to the optional remarks. I may look into further coding of the remarks, but for now I am satisfied with the script.
This Perl script uses Curl to fetch a METAR string from weather.noaa.gov. Then it parses the string, does some formatting and a couple of calculations for Celsius to Fahrenheit conversion then writes the data to a text file. Using a program on my macbook called GeekTools I am able to display the contents of the Perl generated text file on my desktop. It looks like this on my desktop.
Before continuing, I must say that you shouldn't use this script for flight planning and also, that it is largely untested. Please report any bugs that you may find. Part of my ongoing memorization of METAR syntax is to read the string and my scripts output and look for errors.
If you want to play along, install Geektools from the link in the previous paragraph. You will also want to copy the Perl script from below. Save the Perl script in a folder somewhere on your computer. I named my folder "metar" and located in ~/Documents/Weather/. Start up GeekTools and configure like so...
The command in the above screen cap is "cat ~/Documents/Weather/metar/metar_datafile.txt" without quotes. Change this as required for your particular path and file name.
Below is the code. It is definitely a work in progress. Every time I spot an error I correct it but I haven't seen any errors in a while now. Let me know if you do.
#!/usr/bin/perl -w
# this program gathers METAR data from NOAA, decodes it and does some formatting before writing the data to a text file
# this file is monitored and displayed on my macbook desktop using GeekTools
#
# Under no circumstances should you use this for flight planning. Also, no guarantee is made that this software even works at all.
#
# Below is an example of the metar_datafile.txt after the curl command. There are 2 lines as you can see.
#
# 2012/10/27 11:53
# KARB 300153Z AUTO 35018G34KT 10SM OVC095 06/M05 A2978 RMK AO2 PK WND 35034/0144 SLP090 T00561050
#
#
#while( 1 ){
my $ICAO_STATION = "KARB"; # airport nearby
my $raw_metar_data = `curl --silent http://weather.noaa.gov/pub/data/observations/metar/stations/$ICAO_STATION.TXT`;
chomp $raw_metar_data;
my ( $date_time, $current_metar_line ) = split /\n/,$raw_metar_data;
chomp $date_time;
chomp $current_metar_line;
# put the METAR string into an array
my @metar_array = split / /, $current_metar_line;
my $array_index = 0;
open (MYFILE, '>metar_datafile.txt'); #open for write >> would be for apend
#
# print the full array we are about to decode separated by spaces
print MYFILE "@metar_array\n";
#
# print the station ID as read
print MYFILE "ICAO Station ID: $metar_array[ $array_index ]\n";
++$array_index;
#
# print the day of the metar transmission
my $day = substr( $metar_array[ $array_index ], 0, 2 );
print MYFILE "Day: $day";
# print ^st, ^nd, ^rd or ^th at the end of the date just for a bit of fun
#my $right_digit = substr( $day, 1, 1 );
#$day = "3";
if( ( scalar $day == 1 ) || ( scalar $day == 21 ) || ( scalar $day == 31 ) ){
print MYFILE "st\n";
}
elsif( ( scalar $day == 2 ) || ( scalar $day == 22 ) ){
print MYFILE "nd\n";
}
elsif( ( scalar $day == 3 ) || ( scalar $day == 23 ) ){
print MYFILE "rd\n";
}else{
print MYFILE "th\n";
}
#
# print the time of the metar transmission in zulu time aka 0 GMT
my $zulu_hours = substr( $metar_array[ $array_index ], 2, 2 );
my $zulu_minutes = substr( $metar_array[ $array_index ], 4, 2 );
print MYFILE "Report Time ( Zulu ): $zulu_hours:$zulu_minutes\n";
++$array_index;
#
# decode and print report type ( auto or corrected or none )
#
# test String
# $metar_array[ $array_index ] = "COR";
if( $metar_array[ $array_index ] eq "AUTO" ){
print MYFILE "Report Autonomy: Automatic - No human intervention.\n";
++$array_index;
}
elsif( $metar_array[ $array_index ] eq "COR" ){
print MYFILE "Report Autonomy: Corected observations.\n";
++$array_index;
}
else{
print MYFILE "Report Autonomy: Human observer or Automatic with human oversight.\n";
}
#
# decode and print wind data
print MYFILE "Wind Condition: ";
# the first three characters are either VBR or they contain wind direction
my $wind_direction = substr( $metar_array[ $array_index ], 0, 3 );
# the next two characters always contain wind speed
my $wind_speed = substr( $metar_array[ $array_index ], 3, 2 );
if( $wind_direction eq "VBR" ){
print MYFILE "Direction variable at $wind_speed knots.\n";
++$array_index;
}
# else if the string contains a 'V' for variable wind over 6 knots...
elsif( $metar_array[ $array_index ] =~ /V/ ){
print MYFILE "Wind from $wind_direction degrees at $wind_speed knots - ";
my $dir1 = substr( $metar_array[ $array_index ], 9, 3 );
my $dir2 = substr( $metar_array[ $array_index ], 13, 3 );
print MYFILE "Variable between $dir1 and $dir2 degrees.\n";
++$array_index;
}
# else if the string contains a 'G' for gust
elsif( $metar_array[ $array_index ] =~ /G/ ){
print MYFILE "Wind from $wind_direction degrees at $wind_speed knots - ";
my $gust_speed = substr( $metar_array[ $array_index ], 6, 2 );
print MYFILE "Gusts to $gust_speed knots.\n";
++$array_index;
}
else{
print MYFILE "Wind from $wind_direction degrees at $wind_speed knots.\n";
++$array_index;
}
#
# decode and print visibility
print MYFILE "Visibility: ";
# remove "SM" from the end of the string leaving the visibility in statute miles
my $visibility = $metar_array[ $array_index ];
chop $visibility;
chop $visibility;
# if the string still contains an 'M' after the chops there is a special case
if( $visibility =~ /M/ ){
print MYFILE "Less than 1/4 statute miles.\n";
}
elsif( $visibility eq 9999 ){
print MYFILE "Greater than maximum recorded value.\n";
}
else{
print MYFILE "$visibility statute miles.\n";
}
++$array_index;
#
# decode and print runway visual range if reported
# this string should contain a forward slash if it contains runway visual data
#
# test string
# $metar_array[ $array_index ] = "R16/P20000VM211FT";
if( $metar_array[ $array_index ] =~ /\// ){
print MYFILE "Runway Visual Range: ";
my( $runway, $range ) = split /\//, $metar_array[ $array_index ];
# remove the 'R' from the runway number
$runway =~ s/.//;
# remove "FT" from the range(s)
chop $range;
chop $range;
# there is a 'V' in $range_a if the visual range is variable
if( $range =~ /V/ ){
my( $range_a, $range_b ) = split /V/, $range;
print MYFILE "Runway $runway has a visual range between ";
# check for the M or P modifier
if( $range_a =~ /M/ ){
#remove the 'M'
$range_a =~ s/.//;
print MYFILE "less than $range_a and ";
}
elsif( $range_a =~ /P/ ){
# remove the 'P'
$range_a =~ s/.//;
print MYFILE "greater than $range_a and ";
}
else{
print MYFILE "$range_a and ";
}
if( $range_b =~ /M/ ){
#remove the 'M'
$range_b =~ s/.//;
print MYFILE "less than $range_b feet.\n";
}
elsif( $range_b =~ /P/ ){
# remove the 'P'
$range_b =~ s/.//;
print MYFILE "greater than $range_b feet.\n";
}
else{
print MYFILE "$range_b feet.\n";
}
}
else{
print MYFILE "Runway $runway has a visual range of ";
# check for the M or P modifier
if( $range =~ /M/ ){
#remove the 'M'
$range =~ s/.//;
print MYFILE "less than $range feet.\n";
}
elsif( $range =~ /P/ ){
# remove the 'P'
$range =~ s/.//;
print MYFILE "greater than $range feet.\n";
}
else{
print MYFILE "$range feet.\n";
}
}
++$array_index;
}
#
# decode and print weather phenomena if it exists
#
# test string
#$metar_array[ $array_index ] = "+RAPRTS-DRPL";
my $weather_phenom = $metar_array[ $array_index ];
# if the current string does not contain cloud cover data then it is weather phenomena
if( $weather_phenom !~ "SCK" & $weather_phenom !~ "CLR" & $weather_phenom !~ "FEW" & $weather_phenom !~ "SCT" &
$weather_phenom !~ "BKN" & $weather_phenom !~ "OVC" & $weather_phenom !~ "VV" )
{
print MYFILE "Weather Phenomena: ";
my $string_index = 0;
my $string_length = length $weather_phenom;
while( $string_index < $string_length ){
my $sub_string = substr( $weather_phenom, $string_index, 2 );
if( $sub_string =~ /\+/ ){
print MYFILE "Heavy ";
++$string_index;
}
elsif( $sub_string =~ /\-/ ){
print MYFILE "Light ";
++$string_index;
}
#else{
# print MYFILE "Moderate ";
#}
$sub_string = substr( $weather_phenom, $string_index, 2 );
# hash lookup
%weather_type = ( "VC" => "Vicinity ",
"MI" => "Shallow ",
"PR" => "Partial ",
"BC" => "Patches ",
"DR" => "Low Drifting ",
"BL" => "Blowing ",
"SH" => "Showers ",
"TS" => "Thunderstorm ",
"FZ" => "Freezing ",
"DZ" => "Drizzle ",
"RA" => "Rain ",
"SN" => "Snow ",
"SG" => "Snow grains ",
"IC" => "Ice crystals ",
"PL" => "Ice Pellets ",
"GR" => "Hail ",
"GS" => "Small hail ",
"UP" => "Unknown ",
"BR" => "Mist ",
"FG" => "Fog ",
"FU" => "Smoke ",
"VA" => "Volcanic ash",
"DU" => "Widespread dust ",
"SA" => "Sand ",
"HZ" => "Haze ",
"PY" => "Spray ",
"PO" => "Well developed dust/sand swirls ",
"SQ" => "Squalls ",
"FC" => "Funnel clouds including tornadoes or waterspouts ",
"SS" => "Sandstorm ",
"DS" => "Duststorm ",
);
print MYFILE "$weather_type{ $sub_string }";
# see if there is another weather code
$string_index += 2;
}
print MYFILE "\n";
++$array_index;
}
#
# decode and print the cloud cover data ( always present )
#
# test string
#$metar_array[ $array_index ] = "BKN120";
print MYFILE "Cloud cover: ";
# if there are multiple cloud conditions which can occur...
while( $metar_array[ $array_index ] =~ /(SCK|CLR|FEW|SCT|BKN|OVC|VV)/ ){
my $sky = 0;
my $cloud_height = 0;
if( $metar_array[ $array_index ] =~ /VV/ ){
$sky = substr( $metar_array[ $array_index ], 0, 2 );
$cloud_height = substr( $metar_array[ $array_index ], 2, 3 );
}
else{
$sky = substr( $metar_array[ $array_index ], 0, 3 );
$cloud_height = substr( $metar_array[ $array_index ], 3, 3 );
}
%sky_condition = ( "SCK" => "Sky Clear ",
"CLR" => "Clear sky",
"FEW" => "Few clouds ",
"SCT" => "Scattered clouds ",
"BKN" => "Broken clouds ",
"OVC" => "Overcast clouds ",
"VV" => "Vertical visibility ",
);
print MYFILE "$sky_condition{ $sky }";
# if $sky has clouds in it then report the cloud height.
# if skies are clear, do not report cloud height
if( $sky =~ /(FEW|SCT|BKN|OVC|VV)/ ){
# cloud height is given in hundreds of feet
$cloud_height *= 100;
print MYFILE " at $cloud_height feet. ";
}
++$array_index;
}
#
# decode and print the temperature and dewpoint ( always present )
my( $temperature_c, $dewpoint_c ) = split /\//, $metar_array[ $array_index ];
# if it is a negative number
if( $temperature_c =~ /M/ ){
# remove the 'M' from the temperature
$temperature_c =~ s/.//;
$temperature_c *= -1; # and make it a negative number
}
if( $dewpoint_c =~ /M/ ){
# remove the 'M' from the dewpoint
$dewpoint_c =~ s/.//;
$dewpoint_c *= -1; # and make it a negative number
}
my $temperature_f = $temperature_c * 1.8 + 32;
my $dewpoint_f = $dewpoint_c * 1.8 + 32;
print MYFILE "\nTemperature: $temperature_c degrees C / Dewpoint: $dewpoint_c degrees C.\n";
print MYFILE "Temperature: $temperature_f degrees F / Dewpoint: $dewpoint_f degrees F.\n";
++$array_index;
#
# decode and print atmospheric pressure ( always present )
#
# test_string
#$metar_array[ $array_index ] = "Q1234";
#$metar_array[ $array_index ] = "A1234";
my $atm = $metar_array[ $array_index ];
# if our pressure is reported in mb aka hPa
if( $atm =~ /Q/ ){
# remove the 'Q' from the pressure
$atm =~ s/.//;
print MYFILE "Atmospheric Pressure: $atm hPa\n";
}
# if our pressure is reported in mb aka hPa
elsif( $atm =~ /A/ ){
# remove the 'A' from the pressure
$atm =~ s/.//;
my $whole_inhg = substr( $atm, 0, 2 );
my $frac_inhg = substr( $atm, 2, 2 );
print MYFILE "Atmospheric Pressure: $whole_inhg.$frac_inhg inHg\n";
}
++$array_index;
#
# decode and print
close (MYFILE);
#sleep( 60 );
#}
Ok, onward and upward. Clear for takeoff.
Comments