The Spatialite collection of software includes the core library, libspatialite, a simple GIS viewer spatialite-gis and a GUI program for managing sqlite based spatial databases: spatialite_gui. The version 2.3.1 is called “stable”, but 2.4.0 has been out for almost a year now, and 3.0 is already on the horizon. Here we’re looking at the 2.4.0 version There are binaries in the downloads website, including for x86_64 systems, but I couldn’t get them to work since they are, it seems, built against newer versions of the geos and geotiff libraries than come packaged for Enterprise Linux. Keeping with the policy of installing as much as possible thru yum, and doing compiling only when necessary, we’ll leave the system geos and geotiff alone, and go for a homemade spatialite_gui. I downloaded the sources for three packages from the spatialite 2.4.0 download page: spatialite-tools, spatialite_gui and libgaiagraphics.

First comes the requisite libgaiagraphics. Unpack the tarball and drop into its directory. This library needs to find the geotiff header files, so you must have libgeotiff-devel installed through yum. In addition the geotiff  headers on Fedora and EL systems get put into a subdirectory  /usr/include/libgeotiff, so for the configure step to find these needed headers first create a CPPFLAGS environment variable then run the usual compile steps as:
export CPPFLAGS="-I/usr/include/libgeotiff"
./configure
make
and as root do
make install

Next untar the spatialite_gui package and go into its directory. We’ll need sqlite headers for this step so be sure you have done yum install sqlite-devel. But here I found one glitch. It seems that the sqlite3 header file, /usr/include/sqlite3.h as installed on EL6 is lacking one #define statement that causes the compilation of spatialte_gui to fail. So as root, edit /usr/include/sqlite3.h and add
#define SQLITE_STMTSTATUS_AUTOINDEX 3
after line 5272.
Now you’re ready to do ./configure && make
And running make install as root will drop the new spatialite_gui binary into /usr/local/bin.

Finally open spatialite-tools and go thru the same three configure and compile steps. This worked without issues for me, leaving the suite of tools in /usr/local/bin. If you want an icon for a desktop or menu shortcut, it’ll be in your download directory under spatialite_gui-1.4.0/icons…

I’ll present three methods and point out the advantages and drawbacks of each. The first and second for those with “CLI phobia”, and the third – easier in my opinion – a command line option.

The QGIS method

We begin with a straightforward “copy – paste” process right in Quantum GIS. You DO have QGIS installed already, I hope… First activate the GPS plugin in QGIS (this plugin should get installed automatically with QGIS), and click on the GPS icon to open the GPS Tools window. Go over to the “Load GPX File” tab, and browse to find your GPX file. Click “Open” and you’ll see the waypoints or tracks from your GPS on the QGIS canvas.

Now in the QGIS menu, choose “Layer”->”New”->”New Spatialite Layer”.  You’ll need to:

• Select the spatialite *.db file
• Enter a name for the new layer
• Be sure the SRID is set to 4326 (The Lon/Lat WGS84 based spatial reference system used by all GPS instruments)
• Click to add an autoincrementing primary key
• Add any other attribute columns you’ll need

Then click OK to add the new layer into into the Spatialite DB, and into QGIS. Now comes the copy/paste dance: choose the GPX layer in the Table of Contents, and using the Select tool, select all of the features (so that they all appear yellow) and click the “Copy Features” button on the edit toolbar. Now Choose from the TOC the new empty spatialite layer, and click the “Toggle Editing” button to begin editing this new layer. And finally click the “Paste Features” button, again from the editing toolbar, to paste all the selected GPX features into the new spatialite table. Simple and elegant, but…

…if you open up the new layer’s attribute table you’ll see it’s emtpy.  ”So what”, you might ask. Well every GPS captures, along with each point location, at least a waypoint name, and elevation. In the original GPX file these appear as XML elements. If this additional information is not important then the above method should be fine. But if you want to keep the waypoint numbers and elevation, you’ll need a better method.

The intermediate CSV method

For our second try, we need two GUI tools; the spatialite-gui binary and GPSbabel GUI. GPSBabel is traditionally a command line program, but there’s a “Front End” for Windows users. This program reads and writes practically every GPS file format in use on the planet. The steps to importing a GPX file with attributes are as follows. Using GPSBabel we convert the original GPX to the “Universal CSV” format. Then in Spatialite we create a VirtualTable based on this file. And then, using an SQL insert within spatialite we transfer all the GPS locations to a permanent table, and finally create the requisite Geometry column for this new table. Here are the details:

The GPSBabel command to convert to Universal CSV format reads (assuming the original GPX file is waypoints.gpx):

gpsbabel -w -i gpx -f waypoints.gpx  -o unicsv -F waypoints.csv

Alternatively, Using the GPSBabel GUI should be intuitive, choose the input format as GPX (or whatever format you downloaded from your  GPS), and output format Universal CSV. Select the files and check waypoints or tracks, whichever is appropriate. Then run the conversion.

Now fire up spatialite-gui. On the button bar, locate and click the button “Virtual CSV/TXT”. Browse to your waypoints.csv from the previous step and click Open to select the file. In the next “Creating VIrtual CSV/TXT” window change the table name to something like  ”waypoints_tmp”, be sure to select “Comma” as the Column Separator. and click OK. You now have access to the CSV file from within spatialite. You can examine the table structure by right-clicking on it’s name and choosing “Show Columns”. The data is viewable by right-click and “Query table”.

It’s now time to create a new, permanent table for the waypoints. In the query window type the following:

CREATE TABLE waypoints (

long DOUBLE,
lat DOUBLE,
elev DOUBLE,
wpt TEXT );

At this point we want to insert all the rows from the GPX table – waypoints_tmp – into the new waypoints table. Here’s the required SQL.

INSERT INTO waypoints (long, lat,elev, wpt)
SELECT Longitude, Latitude, Altitude, Name FROM waypoints_tmp;

That should leave us with a fully populated, permanent waypoints table. At this point a word of explanation is in order. Why was it necessary to leap-frog over that Virtual CSV/TXT? Why not just import the CSV table directly into a permanent Spatialite table? The reason lies with how numeric columns are recognized by the CSV/TXT importer. If numbers appear as negative values, i.e. preceded by a minus sign (latitudes south of the equator, longitudes west of Greenwich, or below sea level elevations like at the Dead Sea) then these numbers get imported as text, and they are then useless for creating the Geometry column. When we hand crafted our permanent waypoints table as above, we set the long,lat,elev types to DOUBLE. Then, when inserting the values from the virtual table, the “text” with its minus sign got coerced into true numeric values.

We have yet one more step to complete. So far this new table is NOT a spatial table. The Longitude and Latitude values appear as attribute columns but we haven’t yet added the necessary Geometry column. Two more SQL statements complete the job:

SELECT AddGeometryColumn('waypoints','Geometry',4326,'POINT',2);
UPDATE waypoints SET Geometry=MakePoints(long.lat,4326);

The first Spatialite function adds the additional Geometry column, the metadata and constraints, and the second command populates that column based on the longitude/latitude values. Now we have a spatial table with attribute values, imported from a GPS file.

Yet one additional method is available. For those tired of reading such a long post, don’t worry, this final one is short.

The OGR command method

The ubiquitous GDAL library includes a sometimes overlooked, but powerful tool for handling vector format conversions. It’s called ogr2ogr. Since OGR can read and write both GPX formatted files and Spatialite database layers, we can concoct a one liner ogr2ogr command to push GPS locations from a GPX file directly into a Spatialite database layer, and including their attributes. SO, without further ado here it is:

ogr2ogr -append -f "SQLite" \

-dsco SPATIALITE=yes -dsco INIT_WITH_EPSG=yes -t_srs epsg:4326 \
spatial_data.db waypoints.gpx \
-nln waypoints

Just a few words of explanation: -append allows you to add a table to an existing Spatialite database. Based on this maillist thread, it seems that adding the -dsco (dataset creation options) and the target spatial reference system is a good idea. The -nln parameter is the new layer name – that is the new Spatialite table. Pay attention to the unusual order of the files on the command line: First is the destination database, and after the source (in this case) GPX file.

That’s it, the whole procedure distilled into one command. Now who still thinks the CLI is old fashioned or clunky?

 One of the popular sources for freely available digital elevation data is NASA’s SRTM program. These DEM tiles are distributed in GeoTiff format (among others) and each tile covers a 5 degree by 5 degree square. Often your area of interest will fall across several tiles. When you complete Tim’s process, including using a polygon “cutline”  your final color relief raster will show the country (or region) of interest with white space around the edges. In many cases this white space can be quite large, making the final tiff file much larger than is necessary. For example, I created a color relief of Burkina Faso. This African country extends from about 2 degrees W to about 6 degrees W. So I needed two SRTM tiles  to cover the E-W extent: one from 0 to 5 degrees W and another from 5-10 degrees W. Also in the N-S direction I needed two 5 degree tiles to cover the country. So I started with four SRTM DEM tiles covering an area about four times the size of the Burkina Faso. After running

gdalwarp -cutline  ....

I was left me with a raster hundreds of kilometers larger in each direction than the actual borders. To clip out only the extent I needed required one more application of a GDAL tool: gdal_translate with the -projwin option. This allowed me to define the upper left and lower right X-Y coordinates of a “window” to be clipped out of the original raster thus creating a new GeoTiff file, less than 1/2 the size of the original.

In my case, the invocation of gdal_translate was:

gdal_translate -projwin -6 15.1 -2.5 8.5 bkf_relief.tif bkf_relief_clipped.tif

We begin, as usual, getting the basic tools for building, the python GUI software and the GIS stack of libraries. The best approach is to configure your system to use the EPEL repository. Here’s the list of required packages (run as root):

yum install geos proj proj-epsg gdal proj-devel geos-devel gdal-devel proj-nad
yum install gcc gcc-c++ gcc-gfortran flex byacc ncurses-devel bison pkgconfig readline-devel blas-devel lapack-devel
yum install wxPython wxPython-devel wxGTK wxGTK-devel swig python-psycopg2 numpy PyQt4 PyQt4-devel sip sip-devel
yum install mesa-libGLU-devel libXmu-devel lesstif lesstif-devel libjpeg-devel
yum install tcl tcl-devel tk tk-devel fftw2 fftw2-devel hdf5
yum install unixODBC postgresql-devel mysql-devel sqlite-devel freetype-devel

One of the dependencies for building GRASS: mesa-libGLw and mesa-libGLw-devel is still missing from the Scientific Linux and EPEL repositories. But it’s available for Fedora, and the Fedora 13 binary seems to install fine. So download both of the above from the Fedora Project website (be sure to choose the correct package for your architecture). and run
rpm -Uhv mesa-libGLw-*fc15*
to get them installed.
Now download the Fedora source rpm for GRASS from here for GRASS 6.4.0 . Yes it’s currently one minor version behind, but we’ll fix that presently…
You can not just install the src rpm, since it’s configured for a different distro. But all we want it to extract the spec file and patches. Here’s a simple way to do that (no root privileges here):
rpm2cpio grass-6.4.0-3.fc15.src.rpm | cpio -idmv
This breaks open the rpm archive and extracts all its files.

OK, time to setup your rpm build environment (See the Fedora Project wiki for details)
rpmdev-setuptree

From the Fedora source rpm that we unpacked a moment ago copy all the *.patch and the grasslogo_vector_small.png into the ~/rpmbuild/SOURCES directory. Also copy the grass.spec file into the ~/rpmbuild/SPECS directory. Next we download the grass source tarball for 6.4.1: Change into the rpmbuild SOURCES directory and get the source:
wget http://grass.osgeo.org/grass64/source/grass-6.4.1.tar.gz
Now we must enter one last simple change to the spec file in order to build this latest version. Open ~/rpmbuild/SPECS/grass.spec with your favorite editor and alter the second line to read:
Version: 6.4.1
instead of the original 6.4.0

Now we should be ready to roll. Change into the spec directory and issue:
rpmbuild -bb grass.spec
Time for a beverage while the computer churns along. When it’s done, you should have, under the rpmbuild/RPMS directory three grass*.rpm files.
At this point you need to get root privileges again for installing and:
rpm -Uhv rpmbuild/RPMS/grass*.rpm

Since ESRI released the “shapefile” geographic data format in the early 1990′s, using a *.dbf file to hold the geographic feature attributes, the DBF format cemented a grip on a whole new field of computer use. But this was only one late example of successfully applying a simple data storage tool to a much wider scope. Earlier Microsoft’s Foxpro offered a database application development environment around dbf. And they were preceded by others.

Now, dbf is not without its own quirks and shortcomings:

• lack of some data types
• limited characters in column headers
• many SQL functions are not supported
• not scalable to large datasets

Today the more feature-rich sqlite is fast overtaking dbf in many computer applications, and its spatially enabled cousin spatiallite from here may well push the ubiqutous shapefile off of its position as industry standard. So how has this irksome DBF stuck around for so long? Those of us who use open source software, and recognize the advantages of open standards are not surprised at all.

Here’s the story: dBase was first designed by C. Wayne Ratliff when he worked at the Jet Propulsion Laboratory as a kind of office wager. A few years later the software “startup” Aston Tate chose that format and ported their applications from DOS to UNIX and VMS platforms while advancing from dBase II to dBase III. (As a result Aston Tate grew to be one of the top three software manufacturers in the world at the time) Later still Microsoft entered the arena with FoxPro, and litigation over the rights to dBase ensued. Interestingly, SCO was also involved in the legal battle as well. While the suit waged on, Aston Tate was purchased by Borland, who inherited the whole messy affair. Bottom line: the suit was lost since the whole dBase line actually grew from Ratliff’s original work that was in the public domain! Here’s how FoxPro explains it:

On December 11, 1990, Judge Hatter issued an order invalidating Ashton-Tate’s copyrights…
Ashton-Tate had knowingly deceived the United States Copyright Office…
Ashton-Tate knew that its programs were derived from JPLDIS, a public domain program developed by the Jet Propulsion Laboratory, it had “repeatedly failed” to disclose this fact …

Ah ha! So enforcing the “open-ness” of a file format, and refusing to allow copyright law to apply to public domain technologies can insure that those technologies will live a full and fruitful life span!

So here’s this grand daddy of a database format that we GIS user’s are “stuck” with for now. How do we work with it? Well, traditionally, spreadsheet programs such as LibreOffice (Open Office) Calc have handled dbf quite well. You can open, alter and save dBase III with almost no issues using LibreOffice. Microsoft, in their wisdom, decided to drop support for *.dbf beginning with MS Office 2007. So MS user’s can either stay with MS Office 2003 or grab a copy of LibreOffice and use Calc for their DBF based shapefiles.

Before closing, here’s a tip. dBase has support for several data types, one of which is “Boolean”. In dBase terms it’s called “Logical”. A column of type Logical appears as either ‘T’ and ‘F’ or ‘Y’ and ‘N’. Opening a DBF in Calc, you’ll see that each column header has the attribute name (maximum 10 characters) a comma and the attribute type. So text column headers from a DBF will appear in Calc like: LABEL,C,24. That’s a 24 byte wide text attribute named ‘LABEL’. Or: X_COORD,N,12,6. Which means a 12 byte wide numeric attribute named ‘X_COORD’ with 6 decimal places. But whenever you open a dbf in LibreOffice with a boolean attribute, the boolean type becomes a one character wide string. If you then save back to dbf, you’ve lost the boolean type and it turns into single character instead. If you need to maintain the column as a boolean (Logical) then do this in Calc:

• change the column header to look like this: “UPDATED,L” which indicates a boolean (Logical) data type named ‘UPDATED”
• and replace all the single T’s with the full word ‘TRUE’, and all the F’s with the full word ‘FALSE”. Use Calc’s Search and Replace for this.

Now when you save to DBF the column will remain as a ‘Logical’ data type. In other words, LibreOffice needs the full strings TRUE and FALSE in order to recognize the cells as boolean values. Then they get saved to the DBF as boolean ‘T’ or ‘F’.

At work we recently purchased a Panasonic TZ-10 with an embedded GPS.  I want the photos to pop open when I click on their location on a map. Picasa already does this on Googlemaps with photos that contain location data in their EXIF information header. But we want to implement this feature on our maps, with our data layers. And using our favorite open source GIS software. The toolkit I suggest includes QGIS together with the highly regarded ExifTool. Using this method we create a point shapefile of photograph locations. Then, with the “Actions” functionality built into QGIS we’ll call up an image display program to show the pictures at a mouse click.

(A new, experimental plugin called photo2shape is available from GIS-LAB to extract the photo locations from the exif data  directly into a shapefile. However I couldn’t get it working, and others have reported bugs, as is expected in early versions. I’ll report back as the plugin matures)

Installing ExifTool

To install ExifTool on linux, first download the installation tarball from the ExifTool web site. Unzip and untar the package. Then follow these straight-forward steps. For Windows users there’s a binary package ready to use. Unzip it into a directory such as C:\Program Files\Exiftool, and as per this explanation rename the executable exiftool(k).exe to exiftool.exe.

Using ExifTool

This perl based program has a rich list of command line options including selection of EXIF tags, and output formatting. We’ll demonstrate a few.  On Windows, the exiftool executable won’t necessarily be in the PATH, so open a command window and change directory to “C:\Programs Files\ExifTool” (if that is where the exiftool.exe lives).  Also you’ll use the backslash ‘\’ character as the Windows directory separator.  Other than that the commands work the same on both platforms.  First try running with the -l option to list all available EXIF tags in your photos:

micha@hayun-820:~$ exiftool -l /media/data/pix/ExifTool/P1000034.JPG | grep "GPS"
....
GPS Satellites
....
GPS Latitude
GPS Longitude

So this image contains EXIF tags with GPS data. Now we’ll have a quick look at the data in these tags:

micha@hayun-820:~$ exiftool -GPSLongitude -GPSLatitude /media/data/pix/ExifTool/P1000034.JPG
GPS Longitude                   : 35 deg 11' 9.27" E
GPS Latitude                    : 30 deg 36' 51.73" N

We enter the tag name as -TagName using a dash, and no spaces in the name, for each tag, and exiftool outputs the information. Now check the -c option to format the output of the Lon/Lat coordinates.

micha@hayun-820:~$ exiftool -c "%.6f" -GPSLongitude -GPSLatitude /media/data/pix/ExifTool/P1000034.JPG
GPS Longitude                   : 35.185908 E
GPS Latitude                    : 30.614369 N

Formatting follows the conventions of C printf formatting syntax. We also want to extract the date/time of the photo and some exposure information.  The date is formatted with the -d option, and follows the UNIX date formatting, so:

micha@hayun-820:~$ exiftool -c "%.6f" -d "%d/%m/%Y-%H:%M" -CreateDate -Aperture -ShutterSpeed -GPSLongitude -GPSLatitude /media/data/pix/ExifTool/P1000034.JPG
Create Date                     : 31/10/2010-08:32
Aperture                        : 4.5
Shutter Speed                   : 1/500
GPS Longitude                   : 35.185908 E
GPS Latitude                    : 30.614369 N

Finally we want to output the details in table form and extract from several photos at once. We will also need the photo filename and path for later use in defining the QGIS action. ExifTool has the -p option for this. This option can take either a list of the tag names, each preceeded by a ‘$’, or a filename which contains the same list of tag names. The way the tag names are listed in the file is the way they are output.  For example, in order to create a CSV (Comma Separated Values) file, we create and use a file like:

micha@hayun-820:~$ cat /media/data/pix/ExifTool/exif_format.txt
$Filename,$Directory,$CreateDate,$ShutterSpeed,$Aperture,$GPSLongitude ,$GPSLatitude ,$GPSSatellites

The tag names in the format file are commas-separated, so we get equivalent commas in the output. Now by handing off the above format filename to the -p option, here’s the result:

micha@hayun-820:~$ exiftool -c "%.6f" -d "%d/%m/%Y" -p "/media/data/pix/ExifTool/exif_format.txt"  /media/data/pix/ExifTool/*.JPG
P1000032.JPG,/media/data/pix/ExifTool,29/10/2010,1/250,3.9,35.301453 E ,30.806672 N ,4
P1000033.JPG,/media/data/pix/ExifTool,29/10/2010,1/200,4.0,35.301453 E ,30.806672 N ,4
P1000034.JPG,/media/data/pix/ExifTool,31/10/2010,1/500,4.5,35.185908 E ,30.614369 N ,4
3 image files read

We’re getting close…  We need to dump this output into a “point_locations.csv” file, but only after entering the tag names as column headers (separated by commas) as the first line of this file. Then re-run the above command but redirect the output to the same point_locations.csv file:

micha@hayun-820:~$ echo "Filename,Path,Date,Shutter,Aperture,Longitude,Latitude,Satellites" >> point_locations.csv
micha@hayun-820:~$ exiftool -c "%.6f" -d "%d/%m/%Y" -p "/media/data/pix/ExifTool/exif_format.txt"  /media/data/pix/ExifTool/*.JPG  >> point_locations.csv
 3 image files read

Here’s the final CSV file:

micha@hayun-820:~$ cat point_locations.csv
Filename,Path,Date,Shutter,Aperture,Longitude,Latitude,Satellites
P1000032.JPG,/media/data/pix/ExifTool,29/10/2010,1/250,3.9,35.301453 E ,30.806672 N ,4
P1000033.JPG,/media/data/pix/ExifTool,29/10/2010,1/200,4.0,35.301453 E ,30.806672 N ,4
P1000034.JPG,/media/data/pix/ExifTool,31/10/2010,1/500,4.5,35.185908 E ,30.614369 N ,4

One more important chore. Exiftool gives the Lon/Lat coordinates with the N-E-S or W qualifier. That won’t work when importing into QGIS. The coordinates have to be pure numbers. So open the CSV file in your favorite editor, and do a search/replace to remove all instances of ‘ N ‘ or ‘ E ‘.  Note that in the above formatting step I left a space after the $GPSLongitude and $GPSLatitude entries. This then leaves a space after the Lon and Lat values in the resulting CSV.  Use this trick to make the search/replace easier.  And, of course, if your photos are in the Southern hemisphere, or west of Greenwich, the value followed by an ‘S’ or ‘W’ must be made negative.

Displaying photos in QGIS

We next fire-up QGIS and start the Delimited Text plugin. Point it to your CSV file, check that it is parsed properly, and click OK to import the photo locations.

Once the photo locations are imported, open the layer’s Properties window and go to the Actions tab. Enter a name for the action such as “Display photo” and in the Action text box enter the program you want to run to display the photos. On linux systems running Gnome, just enter gthumb or eog (Eye of Gnome).  Windows users can start the “Windows Picture and Fax Viewer” with the following command:

rundll32.exe C:\Windows\system32\shimgvw.dll,ImageView_Fullscreen

You might also have an alternate image viewer installed such as Irfanview. In that case the display command will be:

"C:\Program Files\Irfanview\i_view32.exe"

After the program name you’ll choose the attribute fields “Path” and “Filename” which hold the full path to each photo. Be sure to enter a slash ‘/’ as directory separator between the path and filename. Windows users will of course use the backslash. And another important note for Windows: You must add another ‘%’ character after the Path and Filename column headers. So the full “Action” on a Windows system will look like:

rundll32.exe C:\Windows\system32\shimgvw.dll,ImageView_Fullscreen %Path%\%Filename%

Save the action.  Now using the QGIS information button, click on one of the photo points, and open the Action item at the start of the attribute list. Click on the “Display photo” action and the photo should open.

]]> http://www.surfaces.co.il/smile-youre-on-candid-gps/feed/ 8 http://creativecommons.org/licenses/by-nc-sa/3.0/ http://www.surfaces.co.il/qgis-on-windows-with-ecw/ http://www.surfaces.co.il/qgis-on-windows-with-ecw/#comments Fri, 22 Oct 2010 13:29:36 +0000 Micha Silver http://www.surfaces.co.il/?p=793 Windows users who need a full featured GIS platform can benefit from the fruits of open source software and choose Quantum GIS.  The latest version, 1.6.0, will be released in a few weeks.  And the preview is already available to download and install.

QGIS now offers better labeling, a richer set of map composer tools, and easy to use editing of vector layers.  The plugin architecture gives more demanding users access to over 100 python based advanced analysis and support tools.  And new plugins continue popping up like mushrooms after the rain.  Furthermore QGIS supports, by way of the ubiquitous GDAL library, almost the whole gamut of spatial data formats, from the well known shapefile, google’s *.kml as well as *.gml,  OpenStreetMap‘s *.osm files, and the new kid on the block: spatialite.  The spatially enabled database, PostGIS is fully accessible and managed right from within QGIS.  Among the raster formats are of course GeoTiff, ASCII grids, and  ERDAS img files.  The more complex digital elevation model and weather data such as Hierarchical Database Format – *.hdf – sometimes used by NASA, the USGS *.dem format, and raw SRTM *.hgt files have support built-in.

What’s missing? The very popular “wavelet compression”  imagery formats, ERDAS ECW and LizardTech’s MrSid are both proprietary, and their licensing does not allow free redistribution. Thus QGIS cannot have *.ecw support right out of the box.  However ERDAS does offer free download of their Software Development Kit (SDK), which gives readonly access to *.ecw compressed imagery, provided you agree to their licensing.

So to get QGIS up and running, including ECW support, here’s what you need to do:

Start by browsing to the ERDAS download site. After accepting their licensing terms (requires responding to an email), you’ll be able to get the SDK. Follow the straight-forward instructions here to run the SDK installer, and then copy the required DLL’s into your \windows\system32 directory.

Now go to the QGIS download page and get the OSGeo4W installer. This installer is not a “next->next->next” standalone installation, but rather a framework for getting (and later updating) lots of GIS software.  Note that it requires an internet connection to bring down all the software. (Although it is possible to download the collection of installations on one internet connected computer then copy the whole installation directory and run it locally on another disconnected machine…)

Once you start osgeo4w-setup.exe at the first screen be sure to choose “Advanced Install”.  There are a few more windows where you set download location, and final installation path. You can leave the defaults. Then, after it gets the list of available packages, you’ll be presented with a window containing five categories of software. Expand (click the ‘+’) the Desktop category and click on qgis-dev (click on the circular arrow symbol to toggle to install) as well as  gpsbabelfe.

Next expand the Libs section. Scroll slowly down the list, and again click on the double arrow symbol to install each of the following (I’m including several python packages that are required for some plugins you might want later on):

gdal17-ecw
matplotlib
python-rpy
python-rpy2
python-shapely
qgis-grass-plugin
wxpython
psycopg2
pyspatialite

Assuming you don’t need any of the web applications, click “Next” to start the download and installation. The process takes some time, and on completion you’ll have icons on the desktop to run QGIS (as well as gpsbabel to connect to a GPS, and GRASS for heavy-duty raster and vector analyses).
Startup QGIS, and under the “Raster” menu select the “Information” tool. Click “Select” to browse to a *.ecw file, and then click OK, and you should see in the “Raster Info” window the format and CRS details about the selected file.
It’s worth mentioning that QGIS is under rapid development. You can rerun the osgeo4w-setup.exe any time in the future to identify and install updated packages.

In almost all distros you will find the basic GIS programs, albeit older versions, available in the standard software repositories. Then in almost all cases adding a special GIS repo will give you access to more software, and newer versions.  Still there are some subtle differences. My ratings are not meant to imply preference for any one distro: it’s just an evaluation of how easy it is to get up to speed with GIS.

I put Ubuntu at the top of my list. First you add the Ubuntu GIS repo to your sources.list file as per the instructions here then a few apt-get commands will bring down the whole collection of software.  But Ubuntu offers two additional features not available in other distros (to the best of my knowledge) that win it the trophy. The OSGeo LiveDVD project has created a full blown GIS workstation installation, based on Xubuntu, and containing over 40 of the best FOSS GIS software. It’s available either as a Live DVD image (which you can use to make a bootable DVD or USB flash disk) and as a ready to run virtual machine. Secondly, with only a small effort you can add support for spatial file formats that do not necessarily come built-in such as HDF4, HDF5 and the license encumbered MrSID and ECW formats. On most other distros this will require recompiling of GDAL.

OpenSUSE comes in a very close second.  The Geo repo file to be added to /etc/zypp/repo.d is available here then after doing “sudo zypper update” you’ll get access to the latest and greatest GIS programs with “sudo zypper install ….”. I have found the (private) repo of Francesco Teodori with packages for GRASS and GDAL compiled with ECW support. Installing R statistics requires addition of yet another repository.  See details on the CRAN website.

Next must come Debian. The best, most updated repo for GIS software is http://debian.gfoss.it with software for both Lenny (stable) and Squeeze (testing). Adding it to the sources.list will get you late versions of GRASS, QGIS, the proj and geos libraries, as well as a fairly recent GDAL, with support for HDF4 and HDF5 (but, alas no ECW…).  Lenny users will find that they also need to add the backports repo in order to get all dependencies.  A long list of precompiled r-* packages are also apt-get-able, so Debian is front and center with spatial analysis almost out of the box.

Releases of Fedora do come packed with GIS software, but all are older versions.  QGIS is at 1.0, GRASS still back at 6.2, and GDAL at version 1.5.  I haven’t found a reliable Fedora repo with really recent GIS appications.  So, as far as I know, getting a Fedora machine up to date with the later model GIS programs will require a lot of do-it-yourself compiling from source.  But that’s Fedora…

The popular server platform, CentOS, can be nudged into service as a GIS workstation, but since it’s a 4 year old OS, many of it’s base packages (notably python is still at 2.4 on CentOS 5.5) are just too old to be compatible with the newer desktop applications. By adding the Extra Packages for Enterprise Linux EPEL repo you can  install GRASS with a recent GDAL. PostGIS and Mapserver are also within an easy “yum install” command. But don’t even bother trying QGIS. Bottom line: CentOS 5 is the correct choice for a GIS database server, but for desktop use, let’s wait for CentOS 6.

[Ammendment: By a gross oversight, I forgot to mention the actual GIS repository for CentOS. Thanks to Peter Hopfgartner for the reminder. EPEL gets you only the dependencies: you need the Enterprise Linux GIS repo at elgis for the GIS packages. ELGIS differs from EPEL in that it allows replacing of base CentOS packages. Explanations on the ELGIS website. The repo offers both a stable branch and a testing branch, with the newest versions.]

I have no experience with Mandriva, but I’m fairly sure that their software collection includes the GIS stuff. I leave to others  to fill this gap…

[Ammendment: see Markus' comment below for details about Mandriva]

Take a second and add your vote to the poll at the right. And I’m interested in any feedback or amendments you care to offer.

First off, if you haven’t yet, get the basic packages for compiling software, etc. (I enter my username into the sudoers file, similar to Ubuntu, so that sudo works, without being ‘root’ all the time)

~$ sudo yum install gcc gcc-c++ gcc-gfortran flex byacc cmake ccmake \

           ncurses-devel bison wget gsl gsl-devel expat-devel

Next add the available geospatial packages, as well as several python and Qt libraries that we’ll need for GRASS and QGIS packages that we can install without compiling:

~$sudo yum install proj proj-epsg proj-devel geos libgeotiff libgeotiff-devel \
     wxGTK wxGTK-devel swig python-psycopg2 numpy \
     PyQt4 PyQt4-devel sip sip-devel wxPython wxPython-devel

My goal is to have available in GRASS and QGIS both Spatialite based GIS layers and ECW compressed rasters. These libraries are not available on Fedora, so I’ll need to add them by hand and then compile GDAL to include support for these formats. Next I’ll need to work my way up the toolchain, compiling both GRASS and QGIS to use this new GDAL. We begin with spatialite. Get the source:

~$ wget http://www.gaia-gis.it/spatialite-2.3.1/libspatialite-amalgamation-2.4.0.tar.gz
~$ tar xvzf libspatialite-amalgamation-2.4.0.tar.gz
~$ cd libspatialite-amalgamation-2.4.0

At this point we already bump into a problem with the lack of static binaries in Fedora packaging. Spatialite requires the  geos and proj packages. But it looks for *.la files which are missing from all Fedora libraries now. It seems that libtool reads these *.la text files to find what libraries should be linked. In the absence of a suitable *.la file, when we try to compile spatialite, libtool causes a failure. So here’s the workaround I used. I just create three text files with the definitions needed, and then spatialite happily compiles. In /usr/lib you’ll need libgeos_c.la, libgeos.la, and libproj.la. Thanks to Sandro Furieri on the SpatiaLite Users Forum for this tip.  They should look like this:

~$ sudo vi /usr/lib/libgeos_c.la
          dlname='libgeos_c.so'
          library_names='libgeos_c.so'
          inherited_linker_flags=''
          dependency_libs=' -L/usr/lib /usr/lib/libgeos.la'
          weak_library_names=''
          current=7
          age=6
          revision=2
          installed=yes
          shouldnotlink=no
          dlopen=''
          dlpreopen=''
          libdir='/usr/lib'

For each of the additional *.la files change the ‘dlname=’ and ‘library_names=’  entries as required.
Now go ahead and

~$./configure
~$make && sudo make install

.
Next step is the ECW libraries.  This has been faily well documented already. First you’ll have to register on the ERDAS site to download their SDK, then compile and install it. What you will need, after compiling the ECW library, is to add a *.conf file in /etc/ld.so.conf.d so that ldconfig will “find” the new library.

~$sudo vi /etc/ld.so.conf.d/gdal.conf

       /usr/local/lib
~$sudo ldconfig

Now we’re ready for GDAL, etc.  First download the sources:

~$wget http://download.osgeo.org/gdal/gdal-1.7.2.tar.gz
~$wget http://grass.osgeo.org/grass64/source/grass-6.4.0RC6.tar.gz

(or a more recent source from svn)

~$wget http://download.osgeo.org/qgis/src/qgis_1.5.0.tar.gz

Unzip each tarball and, beginning with GDAL we trip over the next change in Fedora. Along with the “no static binaries” policy, the linker ‘ld’ no longer links libraries that are not explicitly listed in the gcc compile statement. Previously, if one binary linked to a second that linked to a third, then there was “implicit” linking that brought in the third binary. This is no longer the case. The error that pops up with gdal concerns the pthread library. GDAL doesn’t use this inherently, but it is used somewhere along the compile chain. Again as a “quick and dirty” workaround I stuck in statements to have GDAL explicitly link to pthread as follows:

~$ cd gdal-1.7.2
~$LIBS="-lpthread" LDFLAGS="-lpthread" ./configure --with-geos=yes --with-sqlite3 --with-ecw=/usr/local --with-python --with-spatialite=/usr/local
~$make
~$sudo make install

With this done, we’re ready for a straightforward compile of GRASS and QGIS:

~$ cd ../grass-6.4.svn_src_snapshot_2010_08_07 

 (or whatever svn snapshot you downloaded…) 

./configure --prefix=/usr/local/grass-6.4 \
     --with-proj-share=/usr/share/proj \
     --with-postgres --enable-largefile  --with-sqlite \
     --with-freetype  --with-freetype-includes=/usr/include/freetype2 \
     --with-python --with-wxwidgets --with-cxx
make && sudo make install
sudo ln -s /usr/local/grass-6.4/bin/grass64 /usr/bin/

You most likely will also want the gdal-grass package:

~$wget http://download.osgeo.org/gdal/gdal-grass-1.4.3.tar.gz
~$/configure --with-grass=/usr/local/grass-6.4/grass-6.4.0svn --with-gdal=/usr/local/bin/gdal-config 
~#make && sudo make install

Before moving on to QGIS, we’ll get the R statistics package setup.

~$sudo yum install R-core rpy R-devel xml2 libxml2 libxml2-devel

I’ll now add the spatial packages to R. I like to keep all R packages under the system directory /usr/lib/R. However this directory is read only. Since this is my machine, I allow myself to change permissions, and then I can install additional packages without creating a separate R package directory in my home dir.

~$sudo chgrp -R <my username> /usr/lib/R
~$sudo chmod -R g+w /usr/lib/R

Now open an R session and run:

~$R
> install.packages('sp', dependencies='true')
> install.packages('spgrass6', dependencies='true')
>q()

Next, for the rgdal package. R needs to locate the grass libraries. The R rpm packages which I installed above assume that the grass libs are in /usr/local/lib. But I installed GRASS myself elsewhere, so a quick soft link solves that issue:

~$sudo ln -s /usr/local/grass-6.4/grass-6.4.0svn/lib/libgrass_* /usr/local/lib
~$wget http://cran.r-project.org/src/contrib/rgdal_0.6-28.tar.gz
~$R CMD INSTALL rgdal_0.6-28.tar.gz -l /usr/lib/R/library/

and that’s done.

And finally QGIS:

~$ cd ../qgis-1.5.0
~$mkdir build; cd build
~$cmake -DCMAKE_INSTALL_PREFIX:PATH="/usr/local/qgis-1.5.0"  -DGRASS_PREFIX:PATH="/usr/local/grass-6.4/grass-6.4.0svn" -L ..

(Take note of the ‘..’ at the end of the cmake line). The -L option to cmake outputs a list of the config options. Scan thru it to make sure everything is located on your system. Then:

make && sudo make install
sudo ln -s /usr/local/qgis-1.5.0/bin/qgis /usr/bin

At this point you should be back in Kansas

Under the Docs directory are, in addition to the exercises, some manuals and other training material in pdf format. The Geodata directory contains spatial data from naturalearthdata.org, Open Street Map, the European Environment Agency, as well as raster tiles from NASA’s ASTER DEM and the Landsat programs.  Each of the exercises will require loading different data layers.  All data have been clipped to the area of Croatia, and Bosnia-Hercegovina, and all are licensed so as to be freely usable.  Finally, under the Presentations directory are the slides used to explain various aspects of basic cartography and GIS theory.

If you find this learning material worthwhile, or have any comments for improvement, please let me know. I expect that I’ll continue polishing up the material, adding new presentations, and perhaps additional data sources. So check back.

*Update*

I’ve posted the tutorial in separate pieces so interested readers can have a preview before downloading the whole 100 MB zipe file. Here are the links:

• The  Exercises (160 KB)
• The  Presentations (12 MB)
• The  Whole package (100 MB)

Licensing details:

GIS Training with QGIS by Micha Silver is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.