Earth Mapper
Introduction

Lot's of sites have stats gadgets and visitor counters. Some even use map plotters.

On discovering that Google Earth could be embedded in a web page with a few lines of Javascript and a <div> element, I decided to try and use it to plot the users and visitors to a site. I eventually came up with this:





Basic Install
WARNING! Ensure you have made appropriate backups before installing this program.

1.

Copy the following code into the <head> tags of the page you want to display the Google Earth map on.


<script src="http://www.google.com/jsapi?key=API_KEY"> </script>
<script type="text/javascript">
	var ge;
	google.load("earth", "1");
	function init() 
	{
		google.earth.createInstance('map3d', initCB, failureCB);
	}
		
	function initCB(instance) 
	{
		ge = instance;
		ge.getWindow().setVisibility(true);
		ge.getNavigationControl().setVisibility(ge.VISIBILITY_AUTO);
			         
		var href = 'KML_FILE';
			
		var link = ge.createLink('');
		link.setHref(href);
			
		var networkLink = ge.createNetworkLink('');
		networkLink.set(link, true, true); // Sets the link, refreshVisibility, and flyToView
			
		ge.getFeatures().appendChild(networkLink);
	}
		
	function failureCB(errorCode) 
	{
	}
				
	google.setOnLoadCallback(init);
</script>

2.

Replace the phrase 'API_KEY' with the Google Maps API Key for your site. You can obtain one here.

3.

Replace the phrase 'KML_FILE' with the FULL path to where you will place your 'generate_kml.php' file and 'earthmapper' folder. (Relative paths do not work.)

4.

Add the following code to the file between the <body> tags.


<div id="map3d" style="height: 400px; width: 600px;"></div>

5.

Open the 'generate_kml.php' file and replace the following variable values with your appropriate database data.


$db_server = '';
$db_name = '';
$db_username = '';
$db_password = '';

$ip_field = '';
$ip_table = '';

6.

Upload the files to the places you specified and open the page you wish to display in your web browser.



phpBB Install

The phpBB module is designed to be installed with the AutoMOD add-on. So if you don't know how to use that I recommend looking that up and trying it out first.

Once the ConSof Earth Mapper phpBB Module is installed you need to go to the Administration Control Panel of your phpBB board and add a new tab to the Admin Board.

The Admin Board requires a sub category before a page is added. So I recommend using:

Earth Mapper -> Settings -> ACP_EM_SETTINGS

Once complete you will need to provide the full path to the 'generate_kml.php' file and the Google Maps API key for your site.

To add the plugin to a post create a new BBCode in the ACP. Set the BBCode to:


[earthmapper][/earthmapper]

And the html to:


<div id="map3d" style="height: 400px; width: 600px;"></div>


Process Overview
A flowchart outlining the basic process of loading Google Earth and plotting visitor locations.
img i: Process flowchart.

The first issue was getting a list of ip addresses and the associated cities. Enter MaxMind GeoLite City. This provided me with a complete list.

I made the mistake of trying to import a 150MB CSV data file into a MySQL database. A week wasted. I then looked at the GeoCity API and the binary database. I had the whole thing finished in an hour.

Having achieved a working ability to query the GeoCity Lite database and create an array of cities from which site visitors where coming from, I had to begin plotting the cities on the globe. Google Earth makes this incredibly simple with their use of KML files.

A KML file in it's simplest is an XML file. It was a simple matter to loop through the array and output it as an appropriate KML file.

Plug the KML file into the Google Earth plugin and your done!



Code Overview: Javascript in the <head> tags.

Easiest place to start is with making sure you can get the Google Earth plug-in to work. First things first, you will need a Google Maps API Key. Good news, it's free. Better news, you only need one per domain.

Now let's get started with a simple html page:


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">

<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type" />
<title>Untitled 1</title>
</head>

<body>

</body>

</html>

In order to get a Google Earth object on to an HTML page we need some Javascript in the <head> tags. And here it is:


<script src="http://www.google.com/jsapi?key=[YOUR_GOOGLE_MAPS_API_KEY_HERE]"> </script>
<script type="text/javascript">
	var ge;
	google.load("earth", "1");
	function init() 
	{
		google.earth.createInstance('map3d', initCB, failureCB);
	}
		
	function initCB(instance) 
	{
		ge = instance;
		ge.getWindow().setVisibility(true);
		ge.getNavigationControl().setVisibility(ge.VISIBILITY_AUTO);
	}
		
	function failureCB(errorCode) 
	{
	}
				
	google.setOnLoadCallback(init);
</script>

Ok, so that is the Javascript code. That is probably the more complicated section for getting Google Earth on the page. I'm not going to dissect it as Google explains it clearly enough on their site.

I will point out that the first <script> tag that contains the link to Google and the url variable 'key' must be changed for it to work on your site. If you view the source code for this site you will clearly see the API Key for this site. I suppose there are ways to hide it. But if you try and use that on your site Google Earth will NOT display.

There are tons of things you can do to change and play with Google Earth. Check out the Google Earth API examples to see how far it can go.

Onto the final most complicated piece of code in the whole thing:


<div id="map3d" style="height: 400px; width: 600px;"></div>

Ok, so not the most compicated piece of code. But that is all the html code you need to display Google Earth on a web page. As long as you have the Javascript referenced on the page you can put it on the page any where you like.

So, to put the whole thing together, save it in a file and upload it to your site. (Making sure you change the API Key)


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
   <title>Sample</title>
   <script src="http://www.google.com/jsapi?key=ABQIAAAAfaTHoWITLg-0UwDhtxfH0hSFlAt8q2Cl06rchHeh1MQ2fkPAihTnDYJqgieqce79MgElZUjX6MsY_g"> </script>
   <script type="text/javascript">
      var ge;
      google.load("earth", "1");
      function init() {
         google.earth.createInstance('map3d', initCB, failureCB);
         
               }
               
      function initCB(instance) {
         ge = instance;
         ge.getWindow().setVisibility(true);
         ge.getNavigationControl().setVisibility(ge.VISIBILITY_AUTO);
         
		
      }

      function failureCB(errorCode) {
      }
		
      google.setOnLoadCallback(init);

   </script>

</head>
<body>
   <div id="map3d" style="height: 400px; width: 600px;"></div>
</body>
</html>
FYI: You can remove the navigation controls by removing the line: ge.getNavigationControl().setVisibility(ge.VISIBILITY_AUTO);

Setting up the GeoCityLite Database

This database is a godsend! AND FREE! You can of course pay for other services but the free database is perfect my needs.

This is where I wasted a week. I downloaded the CSV files and tried to import them into MySQL. BIG MISTAKE! I love MySQL, but this was not feasible with my resources. One of the files is over 100MB in size, I tried splitting and everything, but no go. If you find a way then feel free to share it.

I then decided to actually read the GeoCity site and see how they did it. Enter the binary database and API. Like I said earlier, an hour after I found this I finished the entire Earth Mapper program and had it running on two sites.

So you need to go and download the the binary database and the Pure PHP Module.

Got them?

Good. Now make sure you just downloaded the Pure PHP Module, as that's what we are going to use here.

We need to set up our file/directory structure. This is the format:

A folder structure diagram.
img ii: Folder Structure.

It should be obvious once you have downloaded the GeoCity packages that the binary database goes in the data folder and the Pure PHP Module goes in the GeoCity folder.

Now that we have things sorted out you can mess around with the sample files in the Pure PHP Module, it will help you have a bit more understanding as to what we are doing.

Next we move onto generating the KML file.



Code Overview: Generating the KML

Ok, so for me this was relatively easy. Mainly because I work with php and SQL all the time. You could easily create a static KML file and it would look a little like this:


<kml>
	<Document>
		<Placemark>
			<name>Melbourne: 4 users</name>
			<Point>
				<coordinates>144.9667,-37.8167</coordinates>
			</Point>
		</Placemark>

		<Placemark>
			<name>Stockholm: 1 users</name>
			<Point>
				<coordinates>18.05,59.3333</coordinates>
			</Point>
		</Placemark>

		<Placemark>
			<name>Cambridge: 1 users</name>
			<Point>
				<coordinates>0.11670000000001,52.2</coordinates>
			</Point>
		</Placemark>
		
		<Placemark>
			<name>San Francisco: 1 users</name>
			<Point>
				<coordinates>-122.4294,37.7645</coordinates>
			</Point>
		</Placemark>
	</Document>
</kml>

Problem is that we don't want to have to keep updating this file every time the visitor log gets updated.

So we'll assume you have an SQL table that stores the ip addresses of your sites visitors.

Our first task is to let the browser/calling method know that we are about to create an XML file:


header("Content-Type:text/xml");

This will also make it easier for debugging later.

Next we include the GeoCity Pure PHP Module files. If you followed the folder structure above properly you should have no problems.


include("GeoCity/geoipcity.inc");
include("GeoCity/geoipregionvars.php");

Next up we need to initialise the GeoCity Database. I'd like to point out that to create this file I took sample file and removed what I didn't need and altered what I did need. I always like to start by modifying something I know works.


$gi = geoip_open("data/GeoLiteCity.dat",GEOIP_STANDARD);

What? You thought that was gonna be a really complicated process? Moving on!

In the interests of maing things easier to edit and so forth later we will set up some variables that can be easily altered at a later date.


$db_server = 'localhost';
$db_name = 'mydb';
$db_username = 'username';
$db_password = 'password';
						
$ip_field = 'ip_address';
$ip_table = 'ip_visitor_log';

The initial four variables are standard variables used for accessing a MySQL Database. You can always alter the code to either include a custom config file or to use your own wrapper class to access the ip address records. It's not really important to the end product.

The last two variables are important. I didn't want people to have to fiddle with the SQL query that will follow, so I put those in so that the code can remain somewhat, dynamic.

Now we connect, set up the query and call it.


$db = mysql_connect($db_server, $db_username, $db_password);

mysql_select_db($db_name, $db);

$sql = "SELECT $ip_field AS ip_address" .
		" FROM $ip_table" . 
		" WHERE NOT $ip_field = ''"
		. " ORDER BY $ip_field";

$result = mysql_query($sql);

Now you can see where all those variables came into play.

A plain language explanation of the SQL query is as follows:

Select ip field values from the ip table and rename column as 'ip_address'. Do not select blank fields. Order by the ip addresses.

Pretty simple and straight forward.

Now that we have a result from the SQL call, we need to loop through it and create an array. We need to create the array before we print out the XML code because we will have multiple addresses in the same city, and we only want one placemark per city.


$cities = array();

while($row = mysql_fetch_array($result))
{
	$record = geoip_record_by_addr($gi,$row['ip_address']);
	
	if($record->city != '')
	{
		if(!array_key_exists($record->city, $cities))
		{
			$cities[$record->city] = array();
		}
		
		$cities[$record->city]['name'] = $record->city;
		
		$cities[$record->city]['count'] += 1;
		
		$cities[$record->city]['lat'] = $record->latitude;
		
		$cities[$record->city]['long'] = $record->longitude;
	}
	
}

Again, pretty simple. We create an array to hold the cities and the user count. We then loop through the SQL results and add each city to the array and increment the count when needed.

All that's left is to loop through our new array and prep the XML nodes for the final output.


foreach($cities as $city)
{
	$output .= '<Placemark>';
	$output .= "<name>{$city['name']}: {$city['count']} users</name>";
	$output .= '<Point>';
	$output .= "<coordinates>{$city['long']},{$city['lat']}</coordinates>";
	$output .= '</Point>';
	$output .= '</Placemark>';
	
}

The XML code is relatively simple. You can make it more complicated when you decide to use custom icons and such. But I'll leave that to you for now. As you can see we only needed four pieces of information to create each Placemark node.

Now that we have our XML data prepped we need to out put the whole thing.


echo '<?xml version="1.0" encoding="UTF-8"?>';
echo '<kml xmlns="http://www.opengis.net/kml/2.2">';

echo '<Document>';

echo $output;

echo '</Document>';

echo '</kml>';

As you can see the output code is as simple as the rest. You just need to enclose the Placemark nodes in Document and kml nodes.

Finally we close the MySQL connection and GeoCity Database.


geoip_close($gi);

mysql_close($db);

That is the sum total of what it takes to generate the kml file dynamically. Don't believe me? Try it! I'll even give you the link to the kml output for this site:

generate_kml.php



Code Overview: Connecting Google Earth to the KML file.

Now we need to go back to our Javascript in the <head> tags of our html file or display page. We need to enable kml files and the like.


<script src="http://www.google.com/jsapi?key=API_KEY"> </script>
<script type="text/javascript">
	var ge;
	google.load("earth", "1");
	function init() 
	{
		google.earth.createInstance('map3d', initCB, failureCB);
	}
		
	function initCB(instance) 
	{
		ge = instance;
		ge.getWindow().setVisibility(true);
		ge.getNavigationControl().setVisibility(ge.VISIBILITY_AUTO);
			         
		var href = 'KML_FILE';
			
		var link = ge.createLink('');
		link.setHref(href);
			
		var networkLink = ge.createNetworkLink('');
		networkLink.set(link, true, true); // Sets the link, refreshVisibility, and flyToView
			
		ge.getFeatures().appendChild(networkLink);
	}
		
	function failureCB(errorCode) 
	{
	}
				
	google.setOnLoadCallback(init);
</script>

To be honest you might as well copy and paste that in the place of the previous Javascript. It is easier. But you can always play about with it.

Remember to replace 'KML_FILE' with the FULL path to were your 'generate_kml.php' file will be. Relative paths don't work.

That is pretty much all the code you need to plot your visitors on a Google Earth Globe.



Acknowledgements

This product includes GeoLite data created by MaxMind, available from http://maxmind.com/