Load partial shapefile into Postgis / GeoDjango project

Question

I have a shapefile with Canadian postal codes, but am only looking to load a small subset of the data. I can load the entire data file and use SQL or Django queries to prune the data, but the load process takes about 2 hours on the slower machines I'm using.

As the data I'm actually after is about 10% of the dataset, this isn't a very efficient process.

I'm following the instructions in the Geodjango tutorial, specifically the following code:

from django.contrib.gis.utils import LayerMapping
from geoapp.models import TestGeo
mapping = {'name' : 'str', # The 'name' model field maps to the 'str' layer field.
               'poly' : 'POLYGON', # For geometry fields use OGC name.
               } # The mapping is a dictionary
lm = LayerMapping(TestGeo, 'test_poly.shp', mapping)
lm.save(verbose=True) # Save the layermap, imports the data.

Is there a way to only only import data with a particular name, as in the example above?

I'm limited to the Linux / OS X command line, so wouldn't be able to utilize any GUI tools.

Answer 1

Thanks for everyone here and on Postgis for their help, particularly ThomasG77 for this answer.

The following line did the trick:

ogr2ogr PostalCodes.shp CANmep.shp -sql "select * from CANmep where substr(postalcode,1,3) in ('M1C', 'M1R')"

ogr2ogr comes with GDAL. brew install gdal will install GDAL on OS X. If you're on another *nix system, the following installs it from source:

$ wget http://download.osgeo.org/gdal/gdal-1.9.2.tar.gz
$ tar xzf gdal-1.9.2.tar.gz
$ cd gdal-1.9.2
$ ./configure
$ make
$ sudo make install

Answer 2

If the required postal codes you need won't change for some time, try creating a shapefile of selected postal codes with QGIS. If you're not familiar with QGIS it's worth looking into. I use it to prepare the file for the webapp before uploading such as crs conversion, editing, attribute table and maybe simplify the geometry.

Theres plenty of tutorials and great help at gis.stackexchange

If you haven't done so already,take this question to gis.stackexchange.

hope this helps get you started, and feel free to ask for more info. I was new to django/geodjango not long ago and appreciated all of the help I received. Django is not for the faint of heart.

Michael