Tuesday, June 18, 2019

How to work with Geographical APIs

API - is the acronym for "Application Programming Interface" which basically means some sets of actions that allow interaction with some bunch of data hosted somewhere on a remote server. These interactions are commonly achieved by sending HTTP request to the server where the data resides. When such a data is related to some geographic location, then it is said you are working with a geographical API.

Imagine there is database of all elevation/heights of points around the world which you can access by providing the location's latitude and longitude. If you want to obtain an elevation of a point, you don't need to travel to that location just to get the elevation value. You will simply send a request to the API of the organization that has the database to retrieve the value.

Or on another hand, there is a database of latitude and longitude values of places around the world. If you need the latitude and longitude of a place you just send a request to with the name of that place to retrieve the latitude and longitude value. This is what the Google Geocode API does, we will see more of other APIs in the coming sections.

Who maintains Geographical API?
An API can be developed and maintained by any individual or organization. The bottom line is that you have access to some data legally and you want users to have access to them by sending HyperText Transfer Protocol (HTTP) requests. Then you will host the data on a server and define some instructions for accessing the data.

To make further clarifications, many organizations such as NASA, Google, ESRI, Mapbox, Government Organisations, etc who have access to large geographic data often have API for accessing and interacting with the data. At the time of writing, the above list are major providers of geo-data API.

There are several method available for the HTTP request, however the most commonly used once are the GET and POST methods. Others are described in the table below:-

SNMethod and Description
The GET method is used to retrieve information from the given server using a given URI. Requests using GET should only retrieve data and should have no other effect on the data.
Same as GET, but transfers the status line and header section only.
A POST request is used to send data to the server, for example, customer information, file upload, etc. using HTML forms.
Replaces all current representations of the target resource with the uploaded content.
Removes all current representations of the target resource given by a URI.
Establishes a tunnel to the server identified by a given URI.
Describes the communication options for the target resource.
Performs a message loop-back test along the path to the target resource.

Where to fine Geographical API?
Just about any individual or organisation that maintains a large volume of geographical data would probably has an API where such data is exposed to developer. However, there are few places to look when searching for geographical API services as listed below:-

1) Rapid API

2) ProgrammableWeb Mashup & API Directory

3) Any API

Above are few places where various APIs are listed and you can use the keywords "Geography", "Location" or "Mapping" to narrow down the search list. In few minutes, I will get some APIs from the directories for demonstration in the section below.

As you can see from each API description, there are literally hundreds of geographical API from different organizations that you can use for different kind of data exploration. You just need to know what the API can do and use it according to there terms of usage as I will explain in the next section.

How to work with Geographical API
1) The first thing is to know what kind of data you want, then head over to the API directories above and search for it availability.

2) If it is available, read and study the documentation provided for the API and use it accordingly.

3) Use any programming language to send/make the HTTP requests.

Example: Lets say we want to work with "Elevation" data across the globe in our project. Obviously we can travel all the globe to collect elevations of points locations, then what do we do? The solution lies in a API, we need an API the provide accurate elevations of places on the globe.

Let's search for such a API...

There are many API providers the can provide "Elevations", some are: Google Elevation API, Open-Elevation API, ElevationAPI.ioBing Maps Elevations etc.

Now, we have several options to choose from. Here I am going to work with ElevationAPI.io. Lets read and study its documentation.

From the doc, you will see that to retrieve the elevation of a point you need to provied the point's latitude and longitude pair in this HTTP GET request format: 'https://elevation-api.io/api/elevation?points=(39.90974,-106.17188),(62.52417,10.02487)&key=YOUR-API-KEY-HERE'.

Further study of the API doc reveals that the API_Key is an optional parameter in the GET url. So, it means we can obtain an elevation of a point at 5km resolution like this: https://elevation-api.io/api/elevation?points=(39.90974,-106.17188)'

The output result is in JSON format and you can parse this easily in python or similar programming language or use an online tool like 'JSON Editor Online' to view the result in a friendly form.

The above concept is the same for any other type of API data you want to work with begin it: Geocode API, Distance API, Satelite Image API, IPGeolocation API, Drone UAV API, etc just to name a few.

That is it!

Saturday, June 8, 2019

Documenting a python script and REST API with 'PyDoc' and 'Swagger OpenAPI Specification' respectively

Documenting a python app appropriately goes along way in communicating what the app does and what is expected of it.

In this post, I will introduce you to two tools you can easily use to create and keep appropriate documentation of your python app.

The first is the PyDoc module which is a standard documentation module in python programming language similar to PerlDoc and JavaDoc for Perl and Java programming languages respectively. With PyDoc, we can generate text and HTML pages with documentation specifics.

The second tool is the Swagger (aka OpenAPI ) which is an open-source software framework backed by a large ecosystem of tools that helps developers design, build, document, and consume RESTful Web services.

PyDoc: Documenting a python script

Pydoc is a Python documentation tool. On cmd enter this: python -m pydoc
You will see that all that the pydoc module is capable of doing as explained below....

1) pydoc <name> ...
    Show text documentation on something.  <name> may be the name of a
    Python keyword, topic, function, module, or package, or a dotted
    reference to a class or function within a module or module in a
    package.  If <name> contains a '\', it is used as the path to a
    Python source file to document. If name is 'keywords', 'topics',
    or 'modules', a listing of these things is displayed.

As an example enter: python -m pydoc pandas to see the documentation on the pandas module (note you should have already install "pandas" for you to see its documentation).

2) pydoc -k <keyword>
    Search for a keyword in the synopsis lines of all available modules.

An example to search for the keyword 'sql' is: python -m pydoc -k sql
All the keywords related to 'sql' on your python environment will be returned as seen below.

3) pydoc -p <port>
    Start an HTTP server on the given port on the local machine.  Port
    number 0 can be used to get an arbitrary unused port.

For example, python -m pydoc -p 0 started an arbitrary unused port '61281' http://localhost:61281 in my case. Yours will definitely be on a different port.

On the cmd use letter 'b' to browse and 'q' to quit/stop the HTTP server.

4) pydoc -b
    Start an HTTP server on an arbitrary unused port and open a Web browser
    to interactively browse documentation.  The -p option can be used with
    the -b option to explicitly specify the server port.

Example is: python -m pydoc -b will start the server doc automatically, it is just a handy shortcut for the above.

5) pydoc -w <name> ...
    Write out the HTML documentation for a module to a file in the current
    directory.  If <name> contains a '\', it is treated as a filename; if
    it names a directory, documentation is written for all the contents.

This enable you generate a HTML doc for a module or a script you have written. Example is: python -m pydoc -w XXX where XXX can be a module or a script file name.

This is useful if you want to share or host the doc, you simply share the resulting html file with others. Below is a written sample script I did like to document. Save it as 'testScript.py' and run the command from the folder that contains the script: python -m pydoc -w testScript

This will generate a html doc for the script that you can share with other developers as seen below. The script contains classes, methods and function with multi line comments (doc strings) in them, the html doc is generated based on those multi line comments (doc strings).

Author: Umar Yusuf
Date: 2019/04/01
This python script, demonstrates how a module is documented for easy future reference.
Hope you like it!

class ClassA(object):
    """Here is the docstring for ClassA"""
    def __init__(self, arg):
        super(ClassA, self).__init__()
        self.arg = arg

    def myMethod(self):
        '''A function in a class is a know as a METHOD'''

class ClassB(ClassA):
    """Here is the docstring for second ClassB. It inherits from ClassA"""
    def __init__(self, arg):
        super(ClassB, self).__init__()
        self.arg = arg

# --------------------
# These are functions since they are outside the class definition
# Python program to multiply two numbers

def multiply(a, b):
    This function takes two numbers in the form of input and multiplies them.
    It displays the multiplication as the result.
    print("Result= ", (a * b))

# functions calling
multiply(5, 2)

a) Show module's doc: python -m pydoc XXX 
b) Search for keyword: python -m pydoc -k XXX
c) Start python doc on http port: python -m pydoc -p 0
d) Shortcut for http server doc: python -m pydoc -b
e) Generates HTML file for modules or written scripts: python -m pydoc -w XXX