Geocoding and Reverse Geocoding with Python

Disclaimer: I originally submitted this article to DataCamp on "Jan 27, 2018". Since they didn't publish it on the platform, I have decided to do it here so that someone out there will find it useful.

Download the original files in HTML and Jupyter Notebook formats

DataCamp Tutorial - Geocoding and Reverse Geocoding with Python

The increasing use of location-aware data and technologies that are able to give directions relative to location and access geographically aware data has given rise to category of data scientists with strong knowledge of geospatial data - Geo-data Scientists.

In this tutorial, you will discover how to use PYTHON to carry out geocoding task. Specifically, you will learn to use GeoPy, Pandas and Folium PYTHON libraries to complete geocoding tasks. Because this is a geocoding tutorial, the article will cover more of GeoPy than Pandas. If you are not familiar with Pandas, you should definitely consider studying the Pandas Tutorial by Karlijn Willems so also this Pandas cheat sheet will be handy to your learning.

Tutorial Overview

• What is Geocoding?
• Geocoding with Python
• Putting it all together – Bulk Geocoding
• Accuracy of the Result
• Mapping Geocoding Result
• Conclusion

What is Geocoding?

A very common task faced by Geo-data Scientist is the conversion of physical human-readable addresses of places into latitude and longitude geographical coordinates. This process is known as “Geocoding” while the reverse case (that is converting latitude and longitude coordinates into physical addresses) is known as “Reverse Geocoding”. To clarify this explanation, here is an example using the datacamp USA office address:-

Geocoding: is converting an address like “Empire State Building 350 5th Ave, Floor 77 New York, NY 10118” to “latitude 40.7484284, longitude -73.9856546”.

Reverse Geocoding: is converting “latitude 40.7484284, longitude -73.9856546” to address “Empire State Building 350 5th Ave, Floor 77 New York, NY 10118”.

Now that you have seen how to do forward and reverse geocoding manually, let’s see how it can be done programmatically in PYTHON on larger dataset by calling some APIs.

Amazon EC2 - Using SFTP to download or upload files

In this post you will learn how to upload/download files to/from Amazon EC2 Instance using FileZilla and SFTP.

SFTP stands for: Secure Shell (SSH) File Transfer Protocol or Secure File Transfer Protocol.

Step 1:

The very first step is to download your AWS console key file in .pem and convert it to .ppk file.

The key file you download from AWS console will be a .pem file and filezilla doesn't read such a file. So you need to convert it to .ppk file. You can use PuttyGen (puttygen.exe) for this conversion from .pem to .ppk.

Step 2:
Download and install FileZilla client from here Download FileZilla.

Step 3:
Now, launch FileZilla and add your .ppk key file of your AWS instance for public key authentication as follow;

~ Go to Edit >> Settings

~ On the Settings dialog box, under 'Connection', select 'SFTP' then click on 'Add Key File...'. Navigate to where you saved your converted .ppk key file and upload it. Then click on 'Ok' button.

Step 4:
Lets login to the server. Go to File >> Site Manager

Step 5:
Click on 'New Site' button, under 'General' tab select 'SFTP' as the protocol and insert your instance username and password. Finally click on "Connect" button.

Host name is something like: ec2-21-xx-xx-xxx.compute-1.amazonaws.com
Username is: ec2-user
Password is optional.

You should now be connected as seen on the status...

Step 6:
You can now navigate your local site folders on the right side pane and upload to the remote site folder on the left pane. Similarly for the downloading of files.

Note: the files which are owned by root cannot be uploaded and downloaded. You will get permission denied error when you try to download a file owned by root.

So, if you want to download those files, you will use PuTTY to SSH into the machine and change the owner of that file to the normal user using the following command.

sudo chown user:user /folder/file

Replace "user:user" with the appropriate root user name.



That is it!
I hope you find it useful.