Breakdown of the KML Plugin

From Geoqo
(Difference between revisions)
Jump to: navigation, search
 
 
Line 728: Line 728:
 
This is actually from the GeoDB::Export::Density code file:
 
This is actually from the GeoDB::Export::Density code file:
  
.
+
Initial preamble and setup:
 +
 
 +
  # Copyright (C) 2007 Wes Hardaker
 +
  # License: GNU GPLv2. See the COPYING file for details.
 +
  package GeoDB::Export::Density;
 +
 
 +
  use strict;
 +
  use GeoDB::Utils;
 +
 
 +
Here's the brunt of the code in this function (which is called from
 +
the KML code described above).  It's passed a reference to the object
 +
itself, the set of caches, the filehandle to print to.  The $sub
 +
argument isn't used (yet):
 +
 
 +
(the @_ array is the perl way of passing arguments)
 +
 
 +
  sub get_density_map {
 +
      my ($self, $set, $fh, $sub) = @_;
 +
 
 +
The density plot is created using an grid of squares.  Normally this
 +
grid is an equal number of sub-squares on each side, but they can be
 +
different if the user set the ''width'' or ''height'' options instead
 +
of the ''size'' option which sets both.
 +
 
 +
      # width/height of the image
 +
      my $binnumx =
 +
        $self->{'options'}{'width'} || $self->{'options'}{'size'};
 +
      my $binnumy =
 +
        $self->{'options'}{'height'} || $self->{'options'}{'size'};
 +
 
 +
This remembers a special ''spread'' option that we'll describe more
 +
below.
 +
 
 +
      # radius not diameter
 +
      my $spread = $self->{'options'}{'spread'};
 +
 
 +
We need to remember the maximum lat/lon boundaries of the data.  This
 +
code simply loops through each waypoint and sees if the coordinates
 +
are more/less than our previously memorized values.
 +
 
 +
      my ($maxlat, $minlat, $maxlon, $minlon) = (-10000, 10000, -10000, 10000);
 +
      $set->foreach(sub {
 +
        $maxlat = max($_[0]{'data'}->{'lat'}, $maxlat);
 +
        $minlat = min($_[0]{'data'}->{'lat'}, $minlat);
 +
        $maxlon = max($_[0]{'data'}->{'lon'}, $maxlon);
 +
        $minlon = min($_[0]{'data'}->{'lon'}, $minlon);
 +
    });
 +
 
 +
We let the user actually set the grid area to be even smaller if they
 +
had specified a restricted area:
 +
 
 +
      # let user override certain ones
 +
      $maxlat = $self->{'options'}{'nmax'} || $maxlat;
 +
      $minlat = $self->{'options'}{'nmin'} || $minlat;
 +
      $maxlon = $self->{'options'}{'wmax'} || $maxlon;
 +
      $minlon = $self->{'options'}{'wmin'} || $minlon;
 +
 
 +
The parse coordinates function can parse strings like ''W121 23.123''
 +
into floating point numbers like ''-121.3853833'' (because the user is
 +
allowed to specify the options in many formats):
 +
 
 +
      ($maxlat, $maxlon) = parse_coords($maxlat, $maxlon);
 +
      ($minlat, $minlon) = parse_coords($minlat, $minlon);
 +
 
 +
Ok, we now have the boundaries of the grid.  It likely won't be
 +
perfectly square from one extreme to the other, which is fine.
 +
 
 +
We also calculate the width of the grid so we subtract the min lat/lon
 +
numbers from their maxes:
 +
 
 +
      GEODEBUG(4,"boundaries: $maxlat, $minlat, $maxlon, $minlon\n");
 +
      my $latdiff = $maxlat - $minlat;
 +
      my $londiff = $maxlon - $minlon;
 +
 
 +
Now we calculate a filled circle.  This is a speed up technique that
 +
will help us later.  The ''spread'' option mentioned above is used
 +
here.  The way the density plot works, each cache affects not only the
 +
density square directly above it, but a "spread" of density squares
 +
around it in a circle.  This gives a much smoother gradient in color
 +
spread as you're looking at the results.  A spread of 1 means no
 +
spread and the results are quite boring.  A spread of 5-10 is
 +
recommended, and the default is 5.
 +
 
 +
This code creates a double array called '@spread', where the middle of
 +
the array is filled with 1s, and the outer edges filled with 0s:
 +
 
 +
  NOT PART OF THE CODE:
 +
 
 +
  00100
 +
  01110
 +
  11111
 +
  01110
 +
  00100
 +
 
 +
As you can see, it should look like a small circle of 1s.  A bigger
 +
spread will be a more recognizable circle ;-)
 +
 
 +
      GEODEBUG(5,"filling circle\n");
 +
      my $textcir;
 +
      my @spread;
 +
      for (my $i = 0; $i <= $spread*2; $i++) {
 +
  for (my $j = 0; $j <= $spread*2; $j++) {
 +
      my $xd = $spread-$i;
 +
      my $yd = $spread-$j;
 +
      $spread[$i][$j] = (sqrt($xd*$xd + $yd*$yd) <= $spread) ? 1 : 0;
 +
      if ($spread[$i][$j]) {
 +
  $textcir .= "X";
 +
      } else {
 +
  $textcir .= ".";
 +
      }
 +
  }
 +
  $textcir .= "\n";
 +
      }
 +
 
 +
If you set the debug level with --debug 7 you'll actually get the
 +
circle printed to the output screen:
 +
 
 +
      GEODEBUG(7,"circle:\n$textcir");
 +
 
 +
Now that we have our circle, we have to take all the caches in the
 +
entire set and put each cache into a double array the size of the grid
 +
itself (called @results).  Each cache will end up inside one spot in
 +
this array.  So, lets say we have a latitude spread from N38 to N37
 +
and a grid size of 200.  That means that our array will have 200 spots
 +
in it, with all the caches from N37 to N37 + (38-31)/200 affecting the
 +
first spot in the array.  It's actually a double array, of course, so
 +
it's a square not a line.
 +
 
 +
      GEODEBUG(5, "putting caches into bin buckets\n");
 +
 
 +
      my ($x, $y, @results);
 +
 
 +
First the progress bar (too fast to see):
 +
 
 +
      my $count = 0;
 +
      my $next = 0;
 +
      $self->setup_progress($set->num(), "Creating Density: analyzing waypoints");
 +
 
 +
And for each cache, we calculate the spot in the array.  When we find
 +
it, we increase the count in that array spot by one.  If multiple
 +
caches fall within an array spot, the array spot count will be higher.
 +
 
 +
      $set->foreach(
 +
    sub {
 +
        $next = $self->set_progress($count++)
 +
  if ($count >= $next);
 +
        $x = int(($_[0]{'data'}{'lat'} - $minlat) * $binnumx / $latdiff);
 +
        $y = int(($_[0]{'data'}{'lon'} - $minlon) * $binnumy / $londiff);
 +
        $results[$x][$y]++;
 +
    });
 +
      $self->stop_progress();
 +
 
 +
So, if that worked you would end up with an array of numbers, where
 +
the bigger numbers would mean more caches.  Something like:
 +
 
 +
  NOT PART OF THE CODE:
 +
 
 +
  11231
 +
  12141
 +
  12363
 +
  12221
 +
  32200
 +
 
 +
Now, what really happens is that there is a lot more zeros and the
 +
caches don't actually affect a huge grid very well.  So, now we merge
 +
the spread array calculated above with the actual cache count we just
 +
performed.
 +
 
 +
So...  To do this, we loop over the entire double array of just
 +
calculated cache counts.  At each spot, we take the cache density
 +
number just calculated and multiple it by the 1s/0s in the circle
 +
array calculated even further back.  Each sub-square in the just
 +
calculated density plot ends up affecting a spread by spread set of
 +
density sub-squares in a new array called @spreadresults.
 +
 
 +
      my @spreadresults;
 +
      my $maxcount = 0;
 +
 
 +
This time the progress bar is very visible, as this code is the
 +
slowest part of generating the KML file....  It takes about 10-15
 +
seconds to generate a 200x200 grid on an average machine.
 +
 
 +
      $next = 0;
 +
      $self->setup_progress($binnumx, "Creating Density: Spreading Results");
 +
 
 +
Here we actually do the calculations.  Loop over the x and y parts of
 +
the density plot first:
 +
 
 +
      for (my $i = 0; $i <= $binnumx; $i++) {
 +
  $next = $self->set_progress($i);
 +
  for (my $j = 0; $j <= $binnumy; $j++) {
 +
 
 +
Then dive into the circle array for each output grid:
 +
 
 +
      for (my $x = $i-$spread; $x <= $i+$spread; $x++) {
 +
  for (my $y = $j-$spread; $y <= $j+$spread; $y++) {
 +
 
 +
If the circle spot is a 1:
 +
 
 +
      if ($spread[$x - $i + $spread][$y - $j + $spread]) {
 +
 
 +
And if the indexes to the new array aren't outside the array
 +
boundaries:
 +
 
 +
  if ($x >= 0 && $x <= $#results
 +
      && $y >= 0 && $y <= $#results) {
 +
 
 +
Then add the density to the square in question.
 +
 
 +
      $spreadresults[$i][$j] += $results[$x][$y];
 +
  }
 +
      }
 +
  }
 +
      }
 +
 
 +
Whew.
 +
 
 +
But we also need to know the maximum value of any density square
 +
calculated in this whole process.  We'll need this later, which I'll
 +
explain then.
 +
 
 +
      $maxcount = max($maxcount,$spreadresults[$i][$j])
 +
  }
 +
      }
 +
      $self->stop_progress();
 +
 
 +
Now, we can actually write out the results.  We remember a bunch of
 +
options first though:
 +
 
 +
      GEODEBUG(5, "Finally writting out results\n");
 +
      my $dump = $self->{'options'}{'dump'};
 +
      my $alt = $self->{'options'}{'altitude'};
 +
      my $extrude = $self->{'options'}{'extrude'};
 +
      my $opaque = $self->{'options'}{'opaque'};
 +
      $opaque = int(255*$opaque/100);
 +
      my $linewidth = $self->{'options'}{'linewidth'};
 +
      my $dozeros = $self->{'options'}{'doempty'};
 +
      my $name = $self->{'options'}{'dataname'};
 +
 
 +
      my $count;
 +
      my $mode = $self->{'options'}{'densitymode'};
 +
 
 +
And then we create the density folder in the output file:
 +
 
 +
      print $fh "<Folder><name>$name: Density Plot</name>\n";
 +
 
 +
And loop through the grid we calculated:
 +
 
 +
      for (my $i = 0; $i <= $binnumx; $i++) {
 +
  for (my $j = 0; $j <= $binnumy; $j++) {
 +
      $count++;
 +
      if ($spreadresults[$i][$j] > 0 || $dozeros) {
 +
 
 +
For each spot, we create a Polygone KML structure.  The color of this
 +
structure is colored by the rainbow from red to purple.  This is done
 +
by a percentage of the current square compared to the maximum we ever
 +
saw (IE, really sparse grids will still have a purple so purple is
 +
relative to the area being examined).  HSV is the color mode where the
 +
HUE is the color, 0 being red, .8 or so being purple (1 would be all
 +
the way back to red again).
 +
 
 +
  my $color = sprintf("%02x%02x%02x%02x",
 +
      reverse(hsv_to_rgb((($maxcount == 0) ? 0 : .8*$spreadresults[$i][$j] / $maxcount)), $opaque));
 +
 
 +
Finally print the entire KML subsquare structure:
 +
 
 +
  print $fh "
 +
    <Placemark id=\"GeoCD$count\">
 +
      <name>GeoCD$count</name>
 +
      <Style>
 +
        <PolyStyle>
 +
          <color>$color</color>
 +
        </PolyStyle>
 +
        <LineStyle>
 +
          <width>$linewidth</width>
 +
          <color>$color</color>
 +
        </LineStyle>
 +
      </Style>
 +
      <Polygon id=\"poly$count\">
 +
        <extrude>$extrude</extrude>
 +
        <tessellate>1</tessellate>
 +
        <altitudeMode>$mode</altitudeMode>
 +
        <outerBoundaryIs>
 +
          <LinearRing>
 +
            <coordinates>
 +
  ";
 +
 
 +
Adding in the coordinates of the subsquare:
 +
 
 +
  print $fh
 +
    ($minlon+$j/$binnumy*$londiff , "," ,
 +
    $minlat+$i/$binnumx*$latdiff , ",$alt " ,
 +
    $minlon+($j+1)/$binnumy*$londiff , "," ,
 +
    $minlat+$i/$binnumx*$latdiff , ",$alt " ,
 +
    $minlon+($j+1)/$binnumy*$londiff , "," ,
 +
    $minlat+($i+1)/$binnumx*$latdiff , ",$alt " ,
 +
    $minlon+$j/$binnumy*$londiff , "," ,
 +
    $minlat+($i+1)/$binnumx*$latdiff , ",$alt " ,
 +
    $minlon+$j/$binnumy*$londiff , "," ,
 +
    $minlat+$i/$binnumx*$latdiff , ",$alt ",
 +
    "\n");
 +
  print $fh "          </coordinates>
 +
          </LinearRing>
 +
        </outerBoundaryIs>
 +
      </Polygon>
 +
    </Placemark>\n";
 +
      }
 +
  }
 +
      }
 +
 
 +
And close out the folder.
 +
 
 +
      print $fh "</Folder>\n";
 +
  }
 +
 
 +
This is the function that converts HSV color structures to RGB:
 +
 
 +
  sub hsv_to_rgb {
 +
      my $h = shift;
 +
      $h = $h * 6;
 +
      my $s = 1;
 +
      my $v = 1;
 +
      my $i = int($h);
 +
      my $f = $h - $i;
 +
      if ($i % 2 == 0) {
 +
  $f = 1 - $f;
 +
      }
 +
      my $m = $v * (1 - $s);
 +
      my $n = $v * (1 - $s * $f);
 +
      return ($v * 255, $n * 255, $m * 255) if ($i == 6 || $i == 0);
 +
      return ($n * 255, $v * 255, $m * 255) if ($i == 1);
 +
      return ($m * 255, $v * 255, $n * 255) if ($i == 2);
 +
      return ($m * 255, $n * 255, $v * 255) if ($i == 3);
 +
      return ($n * 255, $m * 255, $v * 255) if ($i == 4);
 +
      return ($v * 255, $m * 255, $n * 255) if ($i == 5);
 +
  }
 +
 
 +
  1;

Latest revision as of 06:50, 9 December 2007

Contents

KML Code overview

The KML export code is shown in full below (at least a snapshot of it at one point in time, namely near December 2007).

This is written up below in sections in hope that people may learn from it (note that the KML plugin does more complex things than most plugins).

Header Section

This section of code simply imports a bunch of other needed modules, declares the version, etc.

 # Copyright (C) 2007 Wes Hardaker
 # License: GNU GPLv2.  See the COPYING file for details.
 package GeoDB::Export::Kml;
 
 use strict;
 use GeoDB::Export;
 use GeoDB::Utils;
 use GeoDB::Export::Density;
 use IO::File;
 use QWizard;
 our $VERSION = "0.95";

Here we specify that we're a child class of two other modules, the GeoDB::Export and GeoDB::Export::Density modules. Since we're an export plugin, we must inherit from the first. The second is a second code file that contains the actual KML density calculating bits (and eventually it should be a generic density system that could be used to output things other than KML, but right now it isn't).

In perl, the @ISA variable is what defines your parent classes:

 our @ISA = qw(GeoDB::Export GeoDB::Export::Density);

KML Option Definitions

The following is an array of keyword/human-string pairs we use just a bit later.

 our @altitudeValues =
   (relativeToGround => 'relative to the ground',
    absolute => 'absolute height above sea level');

The GEOOPTIONS array is a geoqo special variable that is looked for by code much higher up. This is where we declare all the options to the given plugin. The KML plugin has more than it's fair share of options (more than any other) so this section is pretty long. The array contains a list of perl hashes, where each hash contains keyword/value pairs describing information about the option.

These options are used by many other parts of the geoqo code. They automatically turn into the GUI help screen, they automatically get used by the -d help:export/kml help output, the defaults are all set by things above, the command line parsing (eg: -e kml:dataname=blah:file.kml) is all handled elsewhere based on the data in this array. All good geoqo plugins should use this special array name to declare their options to take advantage of these features. (even the [http://www.geoqo.org/usage.html geoqo manual page] takes information from this!)

 our @GEOOPTIONS =
   (

The first option is a label that is printed to user:

    { name => 'bogus',
      type => 'label',
      text => 'Basic Settings:',
    },

This second one is a normal text box in the GUI that just needs a name (and it's indented slightly. It's default value, if the user doesn't pick one, is Export from GeoQO:

    { name => 'dataname',
      text => 'Name to call the results:',
      default => 'Export from GeoQO',
      indent => 1,
    },

These three are checkbox options and the resulting value stored in the dowaypoints, etc... option variables and will be a 1 or a 0 based on ifit's checked or not. The refresh_on_change token is needed to make the gui screen redraw after the user changes the value (we'll see why in a second).

    { name => 'dowaypoints',
      text => 'Include waypoint/geocache markers:',
      type => 'checkbox',
      refresh_on_change => 1,
      default => 1,
    },
    { name => 'dodensity',
      text => 'Create a density map:',
      type => 'checkbox',
      refresh_on_change => 1,
      default => 1,
    },
    { name => 'dojail',
      text => 'Show .1 mile radius circles',
      type => 'checkbox',
      refresh_on_change => 1,
      default => 0,
    },

This just puts a blank line for a separator:

    "",

And we start a new section with a new label. Here note that the doif section tests the results of the dowaypoints variable above. If it's set, then this widget is shown. If not, then it's hidden (and the refresh_on_change mentioned above forces this to hide and unhide when the user clicks on the checkbox or not). All of the following options make use of this feature.

We'll show a bunch of options in a row now, and you can see the general point by simply studying them for a bit. Note that the second option is also a menu when shown in the GUI:

These all relate to saving each waypoint in the output file.

    { name => 'bogus',
      type => 'label',
      text => 'Configuration of the Waypoints:',
      doif => sub {qwparam('dowaypoints') eq '1'},
    },
    { name => 'waypointstyle',
      type => 'menu',
      text => 'Waypoint Style',
      labels => ['geoqo' => 'Unique icons per difficulty and waypoint type',
 		'pushpin' => 'A single pushpin at each waypoint',
 		],
      default => 'geoqo',
      indent => 1,
      doif => sub {qwparam('dowaypoints') eq '1'},
    },
    { name => 'waypointheight',
      text => 'Waypoint Altitude',
      default => 150,
      indent => 1,
      helpdesc => '(0 means on the ground)',
      doif => sub {qwparam('dowaypoints') eq '1'},
    },
    { name => 'waypointmode',
      text => 'Altitude mode',
      type => 'menu',
      labels => \@altitudeValues,
      default => 'relativeToGround',
      doif => sub {qwparam('dowaypoints') eq '1'},
      indent => 1,
    },
    { name => 'waypointextrude',
      type => 'checkbox',
      text => 'Draw a line to the ground:',
      indent => 1,
      default => 1,
      doif => sub {qwparam('dowaypoints') eq '1'},
    },
    { name => 'includedescriptions',
      type => 'checkbox',
      text => 'Include cache descriptions in output:',
      indent => 1,
      default => 1,
      doif => sub {qwparam('dowaypoints') eq '1'},
    },

Then we move on to the options relating to density plots.

    "",
    { name => 'bogus',
      type => 'label',
      text => 'Configuration of the density grid:',
      doif => sub {qwparam('dodensity') eq '1'},
    },
    { name => 'squareratio',
      default => '1',
      type => 'checkbox',
      text => 'Use a square grid',
      indent => 1,
      refresh_on_change => 1,
      doif => sub {qwparam('dodensity') eq '1'},
    },
    { name => 'size',
      default => 200,
      indent => 1,
      title => 'Number of squares on a side of the density plot',
      help => 'Creates a plot where there are SIZE by SIZE squares of colored rectangles',
      doif => sub {qwparam('squareratio') eq '1' && qwparam('dodensity') eq '1'},
    },
    { name => 'width',
      indent => 1,
      title => 'Width in squares of the rectangular density plot',
      help => 'Normally equal to the size value but can be set independently.  It specifies the width (in squares) of the density plot.',
      doif => sub {qwparam('squareratio') ne '1' && qwparam('dodensity') eq '1'},
    },
    { name => 'height',
      indent => 1,
      title => 'Height in squares of the rectangular density plot',
      help => 'Normally equal to the size value but can be set independently.  It specifies the height (in squares) of the density plot.',
      doif => sub {qwparam('squareratio') ne '1' && qwparam('dodensity') eq '1'},
    },
    { name => 'spread',
      title => 'Spreading Distance',
      indent => 1,
      help => 'Sets the spread that determines the number of neighboring squares that a given cache will effect.  EG, a spread of 5 will affect a circle of density squares in a radius of 5.  This generally should be a low value of probably not more than 5 or so.',
      default => 5,
      doif => sub {qwparam('dodensity') eq '1'},
    },
    { name => 'doempty',
      title => 'Include empty squares in the density plot:',
      help => 'If set to 1, even empty squares will be colored (red).  Otherwise the empty squares are removed from the plot leaving the plane ground underneath.',
      indent => 1,
      type => 'checkbox',
      default => 0,
      doif => sub {qwparam('dodensity') eq '1'},
    },
 
 
    "",
    { name => 'bogus',
      type => 'label',
      text => 'Configuration Parameters for shape style:',
      doif => sub {qwparam('dodensity') eq '1'},
    },
    { name => 'altitude',
      title => 'Altitude of density plot squares',
      indent => 1,
      help => 'The altitude to set the density plot squares at.',
      default => 1000,
      doif => sub {qwparam('dodensity') eq '1'},
    },
    { name => 'densitymode',
      text => 'Altitude mode:',
      type => 'menu',
      labels => \@altitudeValues,
      default => 'relativeToGround',
      doif => sub {qwparam('dodensity') eq '1'},
      indent => 1,
    },
    { name => 'extrude',
      title => 'Extrude the density plot to the ground:',
      help => 'Whether or not to extrude the density plot squares down to the ground.',
      type => 'checkbox',
      indent => 1,
      default => 0,
      doif => sub {qwparam('dodensity') eq '1'},
    },
    { name => 'opaque',
      title => 'Opaqueness of the density squares:',
      helpdesc => '(0-100; 0 being completely see-through)',
      help => 'The opacity of the density squares.  Must be between 0 and 100, with 0 being completely see-through (and thus completely useless).',
      indent => 1,
      default => 20,
      doif => sub {qwparam('dodensity') eq '1'},
    },
    { name => 'linewidth',
      title => 'Line width:',
      help => 'The width of the border lines to draw (0-4).  0 means don\'t draw borders on the squares.',
      indent => 1,
      default => 0,
      doif => sub {qwparam('dodensity') eq '1'},
    },

These options relate to doing a 528 foot circle around each cache (which I refer to as a "jail" since it prevents other caches from being placed near it):

    "",
    { name => 'bagus',
      type => 'label',
      text => 'Configure .1 mile radius circles:',
      doif => sub {qwparam('dojail') eq '1'},
    },
    { name => 'jailaltitude',
      text => 'Altitude of the circles:',
      default => '100',
      doif => sub {qwparam('dojail') eq '1'},
      indent => 1,
    },
    { name => 'jailmode',
      text => 'Altitude mode:',
      type => 'menu',
      labels => \@altitudeValues,
      default => 'relativeToGround',
      doif => sub {qwparam('dojail') eq '1'},
      indent => 1,
    },
 

And finally, some extra options that let you limit the input data by a square boundary:

    "",
    { name => 'bogus',
      type => 'label',
      text => 'Limit the input data:',
    },
    { name => 'limitdata',
      default => '1',
      type => 'checkbox',
      indent => 1,
      text => 'Use all the data from the search set',
      refresh_on_change => 1,
    },
    { name => 'nmax',
      title => 'Maximum North coordinate:',
      indent => 1,
      help => 'The maximum north coordinate value to created the grid over.  Normally this is automatically set by the maximum value found in the data, but can be overridden',
      doif => sub {qwparam('limitdata') ne '1'},
    },
    { name => 'nmin',
      title => 'Minimum North coordinate:',
      indent => 1,
      help => 'The minimum north coordinate value to created the grid over.  Normally this is automatically set by the minimum value found in the data, but can be overridden',
      doif => sub {qwparam('limitdata') ne '1'},
    },
    { name => 'wmax',
      title => 'Maximum West coordinate:',
      indent => 1,
      help => 'The maximum west coordinate value to created the grid over.  Normally this is automatically set by the maximum value found in the data, but can be overridden',
      doif => sub {qwparam('limitdata') ne '1'},
    },
    { name => 'wmin',
      title => 'Minimum West coordinate:',
      indent => 1,
      help => 'The minimum west coordinate value to created the grid over.  Normally this is automatically set by the minimum value found in the data, but can be overridden',
      doif => sub {qwparam('limitdata') ne '1'},
    }
   );

Whew. Done.

The Actual KML Export Magic

First a simple subroutine called _x() which just replaces >, < and & with legal values in XML (> becomes &gt;):

 sub _x {
     return GeoDB::Export::_add_basic_entity(@_);
 }

Here's the actual function that gets called by geoqo when someone wants to export to KML. The GUI screen showing all of the options above may have already been called at this point, or command line option processing may have been. Either way, the function only needs to assume that it's all been dealt with.

Every export module needs to implement the export_it function. The function passed 3 arguments: A self reference to the KML export object, a file name to put the results in, and a GeoQO set that is the waypoint data it should be exporting.

 sub export_it {
     my ($self, $file, $set) = @_;

Open the file and set the $fh variable as the filehandle to write to:

     GEODEBUG(1,"writing file: $file\n");
     my $fh = new IO::File ">$file";

Print the starting XML header to the file:

     print $fh "<?xml version=\"1.0\" encoding=\"utf-8\"?>
 <kml xmlns=\"http://earth.google.com/kml/2.2\">
 <Document>

The next line shows the use of the dataname option variable, which was one of the first in the GEOOPTIONS variable above describing the options that can be passed.

   <name>$self->{'options'}{'dataname'}</name>
   <open>1</open>
   <description>This file was created using GeoQO, which is available from http://www.geoqo.org/</description>
 ";
 

We set up some basic variables that remember the possible icons (shapes), difficulty levels (diffs):

     # these match dist/makeicons
     my @shapes = ('micro', 'small', 'regular', 'large', 'nosize', 'multi',
 		  'virtual', 'unknown', 'solved', 'webcam', 'event', 'earth');
     my @diffs = (qw(1 1.5 2 2.5 3 3.5 4 4.5 5));
 
     my %shapes;
 

If the style they selected in the waypointstyle option was geoqo (the default) then they want a different icon per cache type and difficulty level combination. We need to output all the style definitions into the KML file, so we do that next and it goes near the top of the KML file.

It does this by writing a section of KML for each difficulty for each shape. These link to icons stored on the geoqo website:

     # define the styles needed
     if ($self->{'options'}{'waypointstyle'} eq 'geoqo') {
 	foreach my $shape (@shapes) {
 	    $shapes{$shape} = 1; # cache for later check if we can do it
 	    foreach my $diff (@diffs) {
 		$diff =~ s/\.//;
 		print $fh "  <Style id=\"$shape$diff\">
     <IconStyle>
       <Icon>
         <href>http://www.geoqo.org/images/icons/geocaches-$shape-$diff.png</href>
       </Icon>
     </IconStyle>
   </Style>
 ";
 	    }
 	}
     }
 

Now that we have the styles created, we have to create the top level folder structure in the KML language:

     print $fh "<Folder>
   <name>$self->{'options'}{'dataname'}</name>
   <description>$self->{'options'}{'dataname'}</description>\n";

Now we just process a bunch of options above into local variables; this is not strictly necessary, but it's a bit faster to refer to local variables a bunch of times than repeatedly look into the $self->{'options'}{'....'} structure for each look up if we're going to do it a lot:

     # remember options
     my $geoqostyle = 0;
     $geoqostyle = 1 if ($self->{'options'}{'waypointstyle'} eq 'geoqo');
     my $doextrude = "";
     my $doextrude = "\n      <extrude>1</extrude>"
       if ($self->{'options'}{'waypointextrude'});
     my $height = $self->{'options'}{'waypointheight'};
     my $mode = $self->{'options'}{'waypointmode'};
 
     my $dodescriptions = $self->{'options'}{'includedescriptions'};

Waypoint Creation

We only export the waypoints themselves if the dowaypoints option (checkbox) was set:

     # nopoints is historic...
     if ($self->{'options'}{'dowaypoints'} && !$self->{'options'}{'nopoints'}) {

If so, then we need to create a sub-folder within the KML file and name it appropriately:

 	print $fh "  <Folder>
     <name>$self->{'options'}{'dataname'}: Waypoints</name>\n";

This starts setting up the progress bar and sets the maximum number that we're going to call it. In this case, we're going to set it to the number of waypoint points in the set. Note that not all the GUI backends support progress bars (EG, Tk doesn't which means windows users won't see it):

 	my $count = 0;
 	my $next = 0;
 	$self->setup_progress($set->num(), "Creating Waypoints");

The GeoDB::Set object that we were passed which is full of waypoints has a nice foreach function that can be called. You pass it a reference to another function (in this case, a function declared right inside the argument list). Basically, all the code starting with the sub { and beyond gets called once per waypoint. The subroutine is passed a GeoDB::Waypoint object as the option.

This bit of code prints each waypoint one at a time as the subroutine defined gets called multiple times. The $_[0] statement refers to the GeoDB::Waypoint object. All the waypoint data in one of these objects is stored in the $object->{'data'} hash. EG, the latitude is stored in: $object->{'data'}{'lat'}

 	$set->foreach(sub {
 			  $next = $self->set_progress($count++)
 			    if ($count >= $next);
 			  my $style = ($geoqostyle) ? get_tomtom_style($_[0])
 			    : "#khStyle652";
 			  my $c=$_[0]{'data'};
 			  my $type = ($c->{'subtype'} ? "$c->{'type'}|$c->{'subtype'}" : $c->{'type'});
 
 			  my $description = "";
 			  if ($dodescriptions) {
 			      $description = "<description><![CDATA[";
 			      if ($c->{'groundspeak_short_description_html'} eq 'True') {
 				  $description .=
 				    $c->{'groundspeak_short_description'};
 			      } else {
 				  $description .=
"
" .
  				      _x($c->{'groundspeak_short_description'}) .
  					"

";
 			      }
 
 			      if ($c->{'groundspeak_long_description_html'} eq 'True') {
 				  $description .=
 				    $c->{'groundspeak_long_description'}
 			      } else {
 				  $description .=
"
" .
  				      _x($c->{'groundspeak_long_description'}) .
  					"
";
 			      }
 			      $description .= "]]></description>";
 			  }
 
 			  print $fh "
   <Placemark>
     <name>" . _x($c->{'ident'} . ": " . $c->{'urlname'}) . "</name>$description
     <styleUrl>$style</styleUrl>
     <Point>$doextrude
       <tessellate>0</tessellate>
       <altitudeMode>$mode</altitudeMode>
       <coordinates>" . _x($c->{'lon'}) . "," . _x($c->{'lat'}) . ",$height</coordinates>
     </Point>
   </Placemark>
 ";
 		      });

That was a lot of code just to print out each waypoint description, but you'll note a lot of it was deciding whether to print description information and formatting the output.

Then we stop the progress update so the progress bar window goes away:

 	$self->stop_progress();

And close off the waypoint folder:

 	print $fh "</Folder>\n";
     }

528 Foot Circle Creation ("jails")

Next we do the 528 foot circles, but only if requested via the dojail option:

     # create jails around the cache
     if ($self->{'options'}{'dojail'}) {

And we place them in their own folder too, just like the waypoints above:

 	print $fh "  <Folder>
     <name>$self->{'options'}{'dataname'}: Waypoint .1 Mile Boundaries</name>\n";

First, we need to calculate a ring of points. We basically just store a ring of latitude, longitude points inside two arrays (distringx and distringy). The cos and sin functions are used to plot the circumference of a circle in 16 points. Then we multiply it by the radius distance, which is .001428 between each lat/lon. This actually works for latitude but for the longitude it'll be right at the equator but very very wrong at the poles. This needs to be accounted for in the future...

 	# calculate a ring
 	my (@distringx, @distringy);
 	my $ringcount = 16;
 	my $radiusdist = .001428;  # right for lat, wrong for long
 	for (my $i = 0; $i < $ringcount; $i++) {
 	    push @distringx, cos(2 * 3.1415 * $i / $ringcount) * $radiusdist;
 	    push @distringy, sin(2 * 3.1415 * $i / $ringcount) * $radiusdist;
 	}
 	push @distringx, cos(0) * $radiusdist;
 	push @distringy, sin(0) * $radiusdist;

More options to just remember in variables for later:

 	my $altitude = $self->{'options'}{'jailaltitude'};
 	my $mode = $self->{'options'}{'jailmode'};

This color is transparent (80), no blue (00), no green (00) and maximum red (ff):

 	my $color = "800000ff";
 	my $linewidth = 1;
 	my $extrude = 1;

Again we start a progress bar (but it's so fast usually you won't see it):

 	my $count = 0;
 	my $next = 0;
 	$self->setup_progress($set->num(), "Creating Jail Circles");

Now, we use the same foreach method mentioned above.

 	$set->foreach(sub {

These update the progress bar:

 			  $count++;
 			  $next = $self->set_progress($count++)
 			    if ($count >= $next);

Now we create a variable reference to the waypoint data in the 'data' hash: (not really necessary, but...)

 			  my $c=$_[0]{'data'};

And print a Polygon KML object for each cache:

 			  print $fh "
    <Placemark id=\"jail$count\">
     <name>GeoJail$count</name>
     <Style>
       <PolyStyle>
         <color>$color</color>
       </PolyStyle>
       <LineStyle>
         <width>$linewidth</width>
         <color>$color</color>
       </LineStyle>
     </Style>
     <Polygon id=\"poly$count\">
       <extrude>$extrude</extrude>
       <tessellate>0</tessellate>
       <altitudeMode>$mode</altitudeMode>
       <outerBoundaryIs>
         <LinearRing>
           <coordinates>
 ";

Here we print coordinates for the ring in the real wold by taking the lat/lon of the waypoint and then add in the circle difference at each point that we calculated above. Sometimes the circle point will be negative, sometimes positive. IE, the points this prints out will rotate around the waypoint point at a fixed distance (528 feet) at each point from the starting lat/lon location of the waypoint.

 			  for (my $i = 0; $i <= $ringcount; $i++) {
 			      print $fh ($c->{'lon'}+$distringy[$i]) . "," .
 				($c->{'lat'}+$distringx[$i]) . "," .
 				  $altitude, " ";
 			  }

Then print the closing part of the waypoint circle KML code:

 			  print $fh "
           </coordinates>
         </LinearRing>
       </outerBoundaryIs>
     </Polygon>
   </Placemark>
 ";
 		      });

And stop the progress bar again, and close the circle:

 	$self->stop_progress();
 	
 	print $fh "</Folder>\n";
     }

Density Plot Function Invocation

Now, we actually print the density plot. The get_density_map function is actually defined inside the GeoDB::Export::Density module instead. We'll include it below and explain it as well. In the meantime, just know that it prints out a bunch more KML polygons to represent the density clouds:

     # nodensity is historic
     $self->get_density_map($set, $fh)
       if ($self->{'options'}{'dodensity'} && !$self->{'options'}{'nodensity'});

And then we close the whole thing off.

The remaining code in this section are some functions that calculate a icon name from the geocache information. Traditionals are put into icons according to size, the rest are put into icons according to cache type. The icon name also has the difficulty level in it, so a small icon at difficulty 2 becomes small2. The decimal is dropped, so a small at 2.5 becomes small25.

 print $fh "</Folder>
 </Document>
 </kml>
 ";
 }
 
 my %containers =
   (
    'Micro' 	=> 'micro',
    'Small' 	=> 'small',
    'Regular'    => 'regular',
    'Not chosen' => 'nosize',
    'Other'      => 'nosize',
    'Large'      => 'large',
   );
 
 my %types =
   (
    'Multi-cache' => 'multi',
    'Unknown Cache' => 'unknown',
    'Virtual Cache' => 'virtual',
    'Webcam Cache' => 'webcam',
    'Event Cache' => 'event',
    'Earthcache' => 'earth',
    # XXX!!!  need letterbox!
    # 'Letterbox Hybrid' => 'letter',
   );
 
 sub get_tomtom_style {
     # this matches the code in scripts/devices/tomtom too
     my $cache = $_[0];
 
     # XXX: infield vs solved...
 
     # traditionals break down by size
     if ($_[0]{'data'}{'subtype'} eq 'Traditional Cache') {
 	if (exists($containers{$_[0]{data}{groundspeak_container}})) {
 	    my $diff = $_[0]{data}{groundspeak_difficulty};
 	    $diff =~ s/\.//;
 	    return "#$containers{$_[0]{data}{groundspeak_container}}$diff";
 	}
     } else {
 	if (exists($types{$_[0]{data}{subtype}})) {
 	    return "#$types{$_[0]{data}{subtype}}$_[0]{data}{groundspeak_difficulty}";
 	}
 	# the rest get icons for the type
     }
     return "#khStyle652";
 }

Density Plot Code

This is actually from the GeoDB::Export::Density code file:

Initial preamble and setup:

 # Copyright (C) 2007 Wes Hardaker
 # License: GNU GPLv2.  See the COPYING file for details.
 package GeoDB::Export::Density;
 use strict;
 use GeoDB::Utils;

Here's the brunt of the code in this function (which is called from the KML code described above). It's passed a reference to the object itself, the set of caches, the filehandle to print to. The $sub argument isn't used (yet):

(the @_ array is the perl way of passing arguments)

 sub get_density_map {
     my ($self, $set, $fh, $sub) = @_;

The density plot is created using an grid of squares. Normally this grid is an equal number of sub-squares on each side, but they can be different if the user set the width or height options instead of the size option which sets both.

     # width/height of the image
     my $binnumx =
       $self->{'options'}{'width'} || $self->{'options'}{'size'};
     my $binnumy =
       $self->{'options'}{'height'} || $self->{'options'}{'size'};

This remembers a special spread option that we'll describe more below.

     # radius not diameter
     my $spread = $self->{'options'}{'spread'};

We need to remember the maximum lat/lon boundaries of the data. This code simply loops through each waypoint and sees if the coordinates are more/less than our previously memorized values.

     my ($maxlat, $minlat, $maxlon, $minlon) = (-10000, 10000, -10000, 10000);
     $set->foreach(sub {
 		      $maxlat = max($_[0]{'data'}->{'lat'}, $maxlat);
 		      $minlat = min($_[0]{'data'}->{'lat'}, $minlat);
 		      $maxlon = max($_[0]{'data'}->{'lon'}, $maxlon);
 		      $minlon = min($_[0]{'data'}->{'lon'}, $minlon);
 		  });

We let the user actually set the grid area to be even smaller if they had specified a restricted area:

     # let user override certain ones
     $maxlat = $self->{'options'}{'nmax'} || $maxlat;
     $minlat = $self->{'options'}{'nmin'} || $minlat;
     $maxlon = $self->{'options'}{'wmax'} || $maxlon;
     $minlon = $self->{'options'}{'wmin'} || $minlon;

The parse coordinates function can parse strings like W121 23.123 into floating point numbers like -121.3853833 (because the user is allowed to specify the options in many formats):

     ($maxlat, $maxlon) = parse_coords($maxlat, $maxlon);
     ($minlat, $minlon) = parse_coords($minlat, $minlon);

Ok, we now have the boundaries of the grid. It likely won't be perfectly square from one extreme to the other, which is fine.

We also calculate the width of the grid so we subtract the min lat/lon numbers from their maxes:

     GEODEBUG(4,"boundaries: $maxlat, $minlat, $maxlon, $minlon\n");
     my $latdiff = $maxlat - $minlat;
     my $londiff = $maxlon - $minlon;

Now we calculate a filled circle. This is a speed up technique that will help us later. The spread option mentioned above is used here. The way the density plot works, each cache affects not only the density square directly above it, but a "spread" of density squares around it in a circle. This gives a much smoother gradient in color spread as you're looking at the results. A spread of 1 means no spread and the results are quite boring. A spread of 5-10 is recommended, and the default is 5.

This code creates a double array called '@spread', where the middle of the array is filled with 1s, and the outer edges filled with 0s:

 NOT PART OF THE CODE:
 
 00100
 01110
 11111
 01110
 00100

As you can see, it should look like a small circle of 1s. A bigger spread will be a more recognizable circle ;-)

     GEODEBUG(5,"filling circle\n");
     my $textcir;
     my @spread;
     for (my $i = 0; $i <= $spread*2; $i++) {
 	for (my $j = 0; $j <= $spread*2; $j++) {
 	    my $xd = $spread-$i;
 	    my $yd = $spread-$j;
 	    $spread[$i][$j] = (sqrt($xd*$xd + $yd*$yd) <= $spread) ? 1 : 0;
 	    if ($spread[$i][$j]) {
 		$textcir .= "X";
 	    } else {
 		$textcir .= ".";
 	    }
 	}
 	$textcir .= "\n";
     }

If you set the debug level with --debug 7 you'll actually get the circle printed to the output screen:

     GEODEBUG(7,"circle:\n$textcir");

Now that we have our circle, we have to take all the caches in the entire set and put each cache into a double array the size of the grid itself (called @results). Each cache will end up inside one spot in this array. So, lets say we have a latitude spread from N38 to N37 and a grid size of 200. That means that our array will have 200 spots in it, with all the caches from N37 to N37 + (38-31)/200 affecting the first spot in the array. It's actually a double array, of course, so it's a square not a line.

     GEODEBUG(5, "putting caches into bin buckets\n");
     my ($x, $y, @results);

First the progress bar (too fast to see):

     my $count = 0;
     my $next = 0;
     $self->setup_progress($set->num(), "Creating Density: analyzing waypoints");

And for each cache, we calculate the spot in the array. When we find it, we increase the count in that array spot by one. If multiple caches fall within an array spot, the array spot count will be higher.

     $set->foreach(
 		  sub {
 		      $next = $self->set_progress($count++)
 			if ($count >= $next);
 		      $x = int(($_[0]{'data'}{'lat'} - $minlat) * $binnumx / $latdiff);
 		      $y = int(($_[0]{'data'}{'lon'} - $minlon) * $binnumy / $londiff);
 		      $results[$x][$y]++;
 		  });
     $self->stop_progress();

So, if that worked you would end up with an array of numbers, where the bigger numbers would mean more caches. Something like:

 NOT PART OF THE CODE:
 
 11231
 12141
 12363
 12221
 32200

Now, what really happens is that there is a lot more zeros and the caches don't actually affect a huge grid very well. So, now we merge the spread array calculated above with the actual cache count we just performed.

So... To do this, we loop over the entire double array of just calculated cache counts. At each spot, we take the cache density number just calculated and multiple it by the 1s/0s in the circle array calculated even further back. Each sub-square in the just calculated density plot ends up affecting a spread by spread set of density sub-squares in a new array called @spreadresults.

     my @spreadresults;
     my $maxcount = 0;

This time the progress bar is very visible, as this code is the slowest part of generating the KML file.... It takes about 10-15 seconds to generate a 200x200 grid on an average machine.

     $next = 0;
     $self->setup_progress($binnumx, "Creating Density: Spreading Results");

Here we actually do the calculations. Loop over the x and y parts of the density plot first:

     for (my $i = 0; $i <= $binnumx; $i++) {
 	$next = $self->set_progress($i);
 	for (my $j = 0; $j <= $binnumy; $j++) {

Then dive into the circle array for each output grid:

 	    for (my $x = $i-$spread; $x <= $i+$spread; $x++) {
 		for (my $y = $j-$spread; $y <= $j+$spread; $y++) {

If the circle spot is a 1:

 		    if ($spread[$x - $i + $spread][$y - $j + $spread]) {

And if the indexes to the new array aren't outside the array boundaries:

 			if ($x >= 0 && $x <= $#results
 			    && $y >= 0 && $y <= $#results) {

Then add the density to the square in question.

 			    $spreadresults[$i][$j] += $results[$x][$y];
 			}
 		    }
 		}
 	    }

Whew.

But we also need to know the maximum value of any density square calculated in this whole process. We'll need this later, which I'll explain then.

 	    $maxcount = max($maxcount,$spreadresults[$i][$j])
 	}
     }
     $self->stop_progress();

Now, we can actually write out the results. We remember a bunch of options first though:

     GEODEBUG(5, "Finally writting out results\n");
     my $dump = $self->{'options'}{'dump'};
     my $alt = $self->{'options'}{'altitude'};
     my $extrude = $self->{'options'}{'extrude'};
     my $opaque = $self->{'options'}{'opaque'};
     $opaque = int(255*$opaque/100);
     my $linewidth = $self->{'options'}{'linewidth'};
     my $dozeros = $self->{'options'}{'doempty'};
     my $name = $self->{'options'}{'dataname'};
     my $count;
     my $mode = $self->{'options'}{'densitymode'};

And then we create the density folder in the output file:

     print $fh "<Folder><name>$name: Density Plot</name>\n";

And loop through the grid we calculated:

     for (my $i = 0; $i <= $binnumx; $i++) {
 	for (my $j = 0; $j <= $binnumy; $j++) {
 	    $count++;
 	    if ($spreadresults[$i][$j] > 0 || $dozeros) {

For each spot, we create a Polygone KML structure. The color of this structure is colored by the rainbow from red to purple. This is done by a percentage of the current square compared to the maximum we ever saw (IE, really sparse grids will still have a purple so purple is relative to the area being examined). HSV is the color mode where the HUE is the color, 0 being red, .8 or so being purple (1 would be all the way back to red again).

 		my $color = sprintf("%02x%02x%02x%02x", 
 				    reverse(hsv_to_rgb((($maxcount == 0) ? 0 : .8*$spreadresults[$i][$j] / $maxcount)), $opaque));

Finally print the entire KML subsquare structure:

 		print $fh "
    <Placemark id=\"GeoCD$count\">
     <name>GeoCD$count</name>
     <Style>
       <PolyStyle>
         <color>$color</color>
       </PolyStyle>
       <LineStyle>
         <width>$linewidth</width>
         <color>$color</color>
       </LineStyle>
     </Style>
     <Polygon id=\"poly$count\">
       <extrude>$extrude</extrude>
       <tessellate>1</tessellate>
       <altitudeMode>$mode</altitudeMode>
       <outerBoundaryIs>
         <LinearRing>
           <coordinates>
 ";

Adding in the coordinates of the subsquare:

 		print $fh
 		  ($minlon+$j/$binnumy*$londiff , "," ,
 		   $minlat+$i/$binnumx*$latdiff , ",$alt " , 
 		   $minlon+($j+1)/$binnumy*$londiff , "," ,
 		   $minlat+$i/$binnumx*$latdiff , ",$alt " , 
 		   $minlon+($j+1)/$binnumy*$londiff , "," ,
 		   $minlat+($i+1)/$binnumx*$latdiff , ",$alt " , 
 		   $minlon+$j/$binnumy*$londiff , "," ,
 		   $minlat+($i+1)/$binnumx*$latdiff , ",$alt " , 
 		   $minlon+$j/$binnumy*$londiff , "," ,
 		   $minlat+$i/$binnumx*$latdiff , ",$alt ", 
 		   "\n");
 		print $fh "          </coordinates>
         </LinearRing>
       </outerBoundaryIs>
     </Polygon>
   </Placemark>\n";
 	    }
 	}
     }

And close out the folder.

     print $fh "</Folder>\n";
 }

This is the function that converts HSV color structures to RGB:

 sub hsv_to_rgb {
     my $h = shift;
     $h = $h * 6;
     my $s = 1;
     my $v = 1;
     my $i = int($h);
     my $f = $h - $i;
     if ($i % 2 == 0) {
 	$f = 1 - $f;
     }
     my $m = $v * (1 - $s);
     my $n = $v * (1 - $s * $f);
     return ($v * 255, $n * 255, $m * 255) if ($i == 6 || $i == 0);
     return ($n * 255, $v * 255, $m * 255) if ($i == 1);
     return ($m * 255, $v * 255, $n * 255) if ($i == 2);
     return ($m * 255, $n * 255, $v * 255) if ($i == 3);
     return ($n * 255, $m * 255, $v * 255) if ($i == 4);
     return ($v * 255, $m * 255, $n * 255) if ($i == 5);
 }
 1;
Personal tools