GIS Programming with Python and QGIS - Part 1

Introduction

I will start by introducing the following key words: Programming, Python and QGIS

Programming:

There is high tendency that you as a geospatial professional/graduate/student, you already know what "programming" is all about?

Well, if you don't, here is a brief explanation according to WikiPedia: Computer programming (often shortened to programming) is a process that leads from an original formulation of a computing problem to executable computer programs. The purpose of programming is to find a sequence of instructions that will automate performing a specific task or solving a given problem.

Computer Programming is a much need skill for advanced GIS experts. These days, knowledge and competence in programming is a essential skill set and a critical requirement for most Geospatial job opportunities.  A vast majority of job openings in the Geospatial industry require some expertise in programming, application development and software management.

Some awesome articles I have come across on the web that talked about GIS Programming are listed below (I strongly reccomend you read them);-

1) Learning GIS programming: An overview - by Muthukumar Kumar

2) Should All GIS Users Learn to Code? - by Adena Schutzberg

3) Learning GIS Programming - by Caitlin Dempsey
4) Programming in ArcGIS with Python – A Beginners Guide by Eric van Rees

Python:

Python is a widely used high-level, general-purpose, interpreted, dynamic programming language. Its design philosophy emphasizes code readability, and its syntax allows programmers to express concepts in fewer lines of code than possible in languages such as C++ or Java (Source WikiPedia).

Of all programming languages, many consider python programming language to be the front-runner in the Geospatial industry. Python has been a standard language in GIS because ESRI (ArcGIS) and open source (QGIS) tend to gravitate toward it. Python is very popular with Geospatial professionals working in the ESRI environment and is considered as the primary scripting language for ArcGIS.

Python provides many opportunities for integration within GIS computing systems. Cross-platform capabilities and ease of integration with other languages (C, C++, FORTRAN, and Java) - (CPython, IronPython, Jython, PyPy) mean that Python is most successful in gluing systems together. Because of the fluid language design, the development of large-scale applications is also easily supported. Many libraries and tools have already been developed for working with GIS data in Python. The basics are covered, including the manipulation of shapefiles, grids, and images, as well as more sophisticated stuff such as scripting pyQGIS API and interaction with Web services and databases.

Python can provide you with a complete set of tools for your GIS toolbox. In combination with QGIS, the possibilities are endless. New technologies, such as Web Map Service (WMS), Web Feature Service (WFS), and Web Coverage Service (WCS) are widely supported in Python. There are many online and paper resources to help you when developing Python scripts.

Other programming languages that are being used in the Geospatial industry includes C, C++, C#, Visual Basic.NET, Flex, Java, ActionScript, JavaScript,PHP, VBA for ArcObjects, etc.

QGIS:

QGIS (previously known as Quantum GIS) is a cross-platform free and open-source desktop geographic information system (GIS) application that provides data viewing, editing, and analysis (Source WikiPedia).

Similar to ArcGIS software, QGIS allows users to create maps with many layers using different map projections. Maps can be assembled in different formats and for different uses. QGIS allows maps to be composed of raster or vector layers.

QGIS integrates with other open-source GIS packages, including PostGIS, GRASS, and MapServer to give users extensive functionality. Plugins written in Python or C++ extend QGIS's capabilities. 

Overview of the QGIS Python Console

The QGIS Python Console is the environment where you execute python codes. This works without installing any additional python interpreter since QGIS installation comes with its own python interpreter.

So, by mere installation of QGIS, you are ready to execute python script as am going to demonstrate below.

To open the Python Console in QGIS, do any of the following:-

1) Go to the menu "Plugins >> Python Console"

2) Click on the "Python Console" icon on the tools bar

3) Press "Ctrl + Alt + P" on your keyboard

The Python Console should now appear at the bottom of the map view screen as seen above. The Python Console window can be moved and docked at any position on the screen.

On the QGIS Python Console you will find the following;-

~ the title bar: Minimize and close buttons are here. You can use the title bar to move the console to any position on the screen.

~ the toolbar: icons to the following commands are displayed (Clear console, Import Class, Run Command, Show Editor, Settings, and Help).

I will talk more about the toolbar soon.

~ the display window: the results of expressions are displayed here

~ the interactive prompt (>>>): where to type python expressions and press enter button to execute

From to tools on the tool bar, you can Clear console, Import Class, Run Command, Show Editor, Settings, and Help. Every tool on the tools bar is self explanatory. However, am going to discuss more on Show Editor and Settings.

Settings: This allows you to adjust the console and editor properties such as font style, font size, font color, autocompletion etc.

Lets increase the console and editor font's size. Click on the "Settings" icon on the tools bar, it will open a dialog box where you can specify the settings accordingly. Click on "OK" to apply the changes.

Show Editor: This will open up the editor panel. Here you can create, open, edit and save python modules (files) just as you can do in any text editor.

Lets open the "Editor" panel by clicking on "Show Editor" icon. It will show on the left side of the console as seen below, here you can create, open, edit, save and execute a .py file by making use of

the icons on the editor's tool bar. To close the editor window, click back on the "Show Editor" button to toggle the display back to hide.

At this point you need to be familiar with python scripting to use both the Console and Editor to execute python code. There are many resources on learn python on the internet. Check this resource for over 70+ materials for learning python programming.

Some other excellent resources for learning python using videos include: YouTube.com, Udemy.com, Coursera.com, Udacity.com etc.

If you are conversant with basics of python programming then you can move on with using the QGIS python console.

Executing Python script on QGIS python Console and Editor

To see the version of python installed with your QGIS software: type "import sys" and press enter/run button then type "sys.version" and press enter/run button again.

Anytime you typed an expression on the console, you most press enter/run button to execute it.

My QGIS python version is "2.7.5" as seen above. If we are to  use the editor, we will type and save the expressions before executing.

Now since you know the version of python installed with your QGIS, lets demonstrate the traditional "hello world program" on both the Console and Editor.

On the Console: type in " print 'Hello World, from QGIS Console' " and press enter/run button.

On the Editor: type in " print 'Hello World, from QGIS Editor' " then save it as (hello.py) in any folder of your choice and press "run script" button.

The advantage of using the editor is that you have your script file saved locally that you can refer to anytime. Unlike on the console, whenever you close the program, everything you typed disappears.

Getting Help with QGIS Python Console

You can use the help button to open the help window. Also by using the "help()" function you can get python specific help. For example, lets see what modules that are installed on our QGIS python installation.

Type " help('modules') ", this will display all the modules that are currently installed on the python that came with QGIS installation. My installed modules  are as listed below;-

Column 1	Column 2	Column 3	Column 4
ArgImagePlugin	_symtable	hotshot	smtpd
BaseHTTPServer	_testcapi	htmlentitydefs	smtplib
Bastion	_threading_local	htmllib	sndhdr
BdfFontFile	_warnings	httplib	socket
BmpImagePlugin	_weakref	httplib2	sqlite3
BufrStubImagePlugin	_weakrefset	ihooks	sre
CGIHTTPServer	_win32sysloader	imageop	sre_compile
ConfigParser	_winreg	imaplib	sre_constants
ContainerIO	_winxptheme	imghdr	sre_parse
Cookie	abc	imp	ssl
CurImagePlugin	adodbapi	importlib	sspi
DcxImagePlugin	afxres	imputil	sspicon
DocXMLRPCServer	aifc	inspect	stat
EpsImagePlugin	algs	io	statvfs
ExifTags	antigravity	isapi	string
FitsStubImagePlugin	anydbm	itertools	stringold
FliImagePlugin	argparse	jinja2	stringprep
FontFile	array	json	strop
FpxImagePlugin	ast	keyword	struct
GbrImagePlugin	asynchat	lib2to3	subprocess
GdImageFile	asyncore	linecache	sunau
GdalTools	atexit	lizmap	sunaudio
GifImagePlugin	audiodev	locale	symbol
GimpGradientFile	audioop	logging	symtable
GimpPaletteFile	backports	macpath	sys
GribStubImagePlugin	base64	macurl2path	sysconfig
HTMLParser	bdb	mailbox	tablemanager
Hdf5StubImagePlugin	binascii	mailcap	tabnanny
IcnsImagePlugin	binhex	markupbase	tarfile
IcoImagePlugin	bisect	markupsafe	telnetlib
ImImagePlugin	bsddb	marshal	tempfile
Image	bz2	math	tests
ImageChops	cPickle	matplotlib	textwrap
ImageCms	cProfile	md5	this
ImageColor	cStringIO	mhlib	thread
ImageDraw	calendar	mimetools	threading
ImageDraw2	certifi	mimetypes	time
ImageEnhance	cgi	mimify	timeit
ImageFile	cgitb	mkgraticule	timer
ImageFileIO	chunk	mmap	toaiff
ImageFilter	cmath	mmapfile	token
ImageFont	cmd	mmsystem	tokenize
ImageGL	code	modeler	tools
ImageGrab	codecs	modulefinder	tornado
ImageMath	codeop	mpl_toolkits	trace
ImageMode	collections	msilib	traceback
ImageOps	colorsys	msvcrt	tty
ImagePalette	commands	multifile	types
ImagePath	commctrl	multiprocessing	ui_frmEliminate
ImageQt	compileall	mutex	ui_frmGeometry
ImageSequence	compiler	netbios	ui_frmGeoprocessing
ImageShow	console	netrc	ui_frmIntersectLines
ImageStat	contextlib	networkx	ui_frmMeanCoords
ImageTk	contour	new	ui_frmMergeShapes
ImageTransform	cookielib	nntplib	ui_frmPointDistance
ImageWin	copy	nose	ui_frmPointsInPolygon
ImtImagePlugin	copy_reg	nt	ui_frmRandPoints
IptcImagePlugin	core	ntpath	ui_frmRandom
JpegImagePlugin	csv	ntsecuritycon	ui_frmReProject
McIdasImagePlugin	ctypes	nturl2path	ui_frmRegPoints
MetaSearch	curses	numbers	ui_frmSelectByLocation
MicImagePlugin	datetime	numericalDigitize	ui_frmSimplify
MimeWriter	dateutil	numericalVertexEdit	ui_frmSpatialIndex
MpegImagePlugin	db_manager	numpy	ui_frmSpatialJoin
MspImagePlugin	dbhash	odbc	ui_frmSubsetSelect
OleFileIO	dbi	ogr	ui_frmSumLines
OpenGL	dde	opcode	ui_frmVectorGrid
PIL	decimal	openlayers_plugin	ui_frmVectorSplit
PSDraw	difflib	operator	ui_frmVisual
PaletteFile	dircache	optparse	unicodedata
PalmImagePlugin	dis	os	unittest
PcdImagePlugin	distutils	os2emxpath	urllib
PcfFontFile	doDefineProj	osgeo	urllib2
PcxImagePlugin	doEliminate	osr	urlparse
PdfImagePlugin	doGeometry	owslib	user
PixarImagePlugin	doGeoprocessing	parser	uu
PngImagePlugin	doIntersectLines	pct2rgb	uuid
PpmImagePlugin	doMeanCoords	pdb	voronoi
ProcessingPlugin	doMergeShapes	perfmon	warnings
PsdImagePlugin	doPointDistance	pickle	wave
PyQt4	doPointsInPolygon	pickletools	weakref
Qgis2threejs	doRandPoints	pipes	webbrowser
Queue	doRandom	pkgutil	whichdb
SgiImagePlugin	doRegPoints	platform	win2kras
SimpleHTTPServer	doSelectByLocation	plistlib	win32api
SimpleXMLRPCServer	doSimplify	pluginbuilder	win32clipboard
SocketServer	doSpatialIndex	popen2	win32com
SpiderImagePlugin	doSpatialJoin	poplib	win32con
StringIO	doSubsetSelect	posixfile	win32console
SunImagePlugin	doSumLines	posixpath	win32cred
TarIO	doValidate	pprint	win32crypt
TgaImagePlugin	doVectorGrid	processing	win32cryptcon
TiffImagePlugin	doVectorSplit	profile	win32event
TiffTags	doVisual	pstats	win32evtlog
UserDict	doctest	psycopg2	win32evtlogutil
UserList	dumbdbm	pty	win32file
UserString	dummy_thread	py_compile	win32gui
WalImageFile	dummy_threading	pyclbr	win32gui_struct
WmfImagePlugin	email	pydoc	win32help
XVThumbImagePlugin	encodings	pydoc_data	win32inet
XbmImagePlugin	epsg_tr	pyexpat	win32inetcon
XpmImagePlugin	errno	pygments	win32job
_LWPCookieJar	esri2wkt	pylab	win32lz
_MozillaCookieJar	exceptions	pyodbc	win32net
__builtin__	expressions	pyparsing	win32netcon
__future__	fTools	pyplugin_installer	win32pdh
_abcoll	filecmp	pyspatialite	win32pdhquery
_ast	fileinput	pythoncom	win32pdhutil
_bisect	fnmatch	pytz	win32pipe
_bsddb	formatter	pywin	win32print
_codecs	fpformat	pywin32_testutil	win32process
_codecs_cn	fractions	pywintypes	win32profile
_codecs_hk	ftools_utils	qgis	win32ras
_codecs_iso2022	ftplib	qgis2leaf	win32rcparser
_codecs_jp	functools	qgis2web	win32security
_codecs_kr	future_builtins	qgiscloud	win32service
_codecs_tw	gc	quopri	win32serviceutil
_collections	gcps2vec	random	win32timezone
_csv	gcps2wld	rasutil	win32trace
_ctypes	gdal	re	win32traceutil
_elementtree	gdal2tiles	regcheck	win32transaction
_functools	gdal2xyz	regutil	win32ts
_hashlib	gdal_auth	repr	win32ui
_heapq	gdal_calc	rexec	win32uiole
_hotshot	gdal_edit	rfc822	win32verstamp
_imaging	gdal_fillnodata	rgb2pct	win32wnet
_imagingmath	gdal_merge	rlcompleter	winerror
_io	gdal_polygonize	robotparser	winioctlcon
_json	gdal_proximity	runpy	winnt
_locale	gdal_retile	sched	winperf
_lsprof	gdal_sieve	scipy	winsound
_md5	gdalchksum	script	winxpgui
_msi	gdalcompare	select	winxptheme
_multibytecodec	gdalconst	servicemanager	wsgiref
_multiprocessing	gdalident	sets	wx
_osx_support	gdalimport	sgmllib	wxPython
_pyio	gdalmove	sha	wxversion
_random	gdalnumeric	shapely	xdrlib
_sha	genericpath	shelve	xlrd
_sha256	getopt	shlex	xlwt
_sha512	getpass	shutil	xml
_socket	gettext	signal	xmllib
_sqlite3	glob	simplejson	xmlrpclib
_sre	gui	sip	xxsubtype
_ssl	gzip	sipconfig	zipfile
_strptime	hashlib	sipdistutils	zipimport
_struct	heapq	site	zlib
_subprocess	hmac	six

As you can see from above, there are so much modules installed by default that we don't necessarily have to install another module to start GIS programming in QGIS. Some of the geospatial specific installed modules are shapely, matplotlib, gdalnumeric, gdal2xyz, gdal, numpy, gdal_polygonize, gdal_calc, pyqgis (QGIS API) etc.

If you want to know how any of the above module objects works, all you have to do on the console is call "help('object_name')". Also some other useful functions for getting help are "dir() - which displays all the object methods" and " 'object'.__doc__ - which displays documentation string of an object".

Lets clear the console screen and demonstrate the use of dir() and 'object'.__doc__

Assuming we need to learn more about the shapely module, we can type any of these on our QGIS console:-

1) help('shapely')

2) dir('shapely')

3) 'shapely'.__doc__

Lets continue... GIS Programming with Python and QGIS - Part 2 here.

Thanks for reading.