Saturday, May 15, 2021

Python Django Setup Template

Here listed below are code snippets for creating a "PROJECT_NAME" Django Project with "APP_NAME" App.

1) Initial Set Up

  • mkdir My_Django_Folder
  • cd My_Django_Folder
  • pipenv install django
  • pipenv shell
  • django-admin startproject PROJECT_FOLDER_NAME .
  • python manage.py migrate
  • python manage.py createsuperuser
  • python manage.py startapp APP_FOLDER_NAME
  • python manage.py runserver

mkdir My_Django_Folder
cd My_Django_Folder
pipenv install django
pipenv shell
django-admin startproject PROJECT_FOLDER_NAME .

python manage.py migrate
python manage.py createsuperuser

python manage.py startapp APP_FOLDER_NAME
python manage.py runserver

Your project folder structure should look like this when you type 'tree' on the command prompt:-


It should also contain two files ("Pipfile" and "Pipfile.lock") that makes the virtual environment work properly.

If everything went successfully, you should get the welcome screen that congratulates you when you access the localhost with the given port number.





2) Add HTML/CSS/JS templates

At this point, you want to display a custom front-end the user instead of the default seen above. A front-end framework such as Bootstrap can be use, however, I will keep it simple by using a simple html/css/js files.

Recall from the step above, we create a django PROJECT and APP. Now, we need to edit some files within both folders like this:-
  • PROJECT folder: here we will edit these files - settings.py and urls.py
  • APP folder: here we will edit/create these files - urls.pyviews.py, and models.py. Note we could also add our templates folder here for the index.html but I prefer to create a root templates folder and point to it by updating the TEMPLATES list in PROJECT settings.py.

The following code snippets will add HTML/CSS/JS templates
  • mkdir templates
  • echo > templates/index.html (touch templates/index.html)
  • add your html code to the templates\index.html file
  • Update INSTALLED_APPS and TEMPLATES from PROJECT's settings.py
  • Update urls.py file from PROJECT's folder
  • Create a new urls.py file in app folder 'APP_FOLDER_NAME' and add view path
  • Update views.py file from the app folder 'APP_FOLDER_NAME' by defining the view added above. Since the name was called 'home', it most be the same in the views.py file.


If you do python manage.py runserver, this should load our html page from the templates folder as seen below:-


Notice that this is a very basic web page that didn't make use of a database, hence we didn't talk about the models.py file.

To add more pages for example, lets add contact page. We will create the contact.html file in the templates folder and update urls.py and views.py file in the app folder.



Note: If you don't want the .html extension to appear in the url like this "http://localhost:8000/contact.html", the edit urls.py file to remove the extension so that the path is like so... "http://localhost:8000/contact"



3) Adding Static file

In django, to use external CSS, JS, or Image files within HTML document you need to tell it how to get them by creating a root folder named "static" to house all the static files. The steps involved are as follow:-
  1. First thing is to update PROJECT's settings.py file by adding STATICFILES_DIRS


  2. Create a root folder named "static" and create another sub-folder with name of the app 'APP_FOLDER_NAME' 
  3. Add your static files there, for example I added an image 'buhari.jpg'
  4. Update the html file by first typing "{% load static %}" on the very first line. Then point to the file using this "{% static 'APP_FOLDER_NAME/buhari.jpg' %}"

The new home page should look like this...



This is the django way of doing things! In normal html, the inserted image tag will just look like this: <img src="APP_FOLDER_NAME/buhari.jpg " />

Likewise inserting a css or js file like this: <link rel="stylesheet" type="text/css" href="style.css"> in normal html will look like this: <link rel="stylesheet" type="text/css" href="{% static 'APP_FOLDER_NAME/style.css' %}"> in django way.

Monday, May 10, 2021

Extracting Information from PDF with Python

There are several python libraries available that allows the extraction of pdf data. Some notable once are listed below:-

1) PyPDF2
2) Tabula-py (requires JAVA installed)
3) Zamzar.com API
4) Camelot
5) TIKA
6) PDFMiner

All the above are great have there pros and cons which depends on the pdf file in question. You will need to try each one out to find out the one that works best for your specific case.

In my own case, TIKA has been the best so far for all the pdf I have worked with. So in this blog post, I will demonstrate how the 'tika' module is used to extract information from a pdf file.

Lets get started....

To install this module, you most has java 7 or above installed on your machine. Then using it is just as easy as this:-

from tika import parser # pip install tika (need Java 7+ installed)

raw = parser.from_file('C:/Desktop/pdf_file_name.pdf')
print(raw['content'])

The content can then be stored in a variable for further manipulation.



Example

Lets extract the content of this PDF file:-



The result will look like this below. Note that I added .strip() function that will remove extra white spaces at the beginning and end of the content result.







That is it!

Thursday, May 6, 2021

Python script to separate shapefile based on type (Point, Line and Polygon)

 Here I got a folder that contains several shapefiles of mixed types (i.e Point, Line and Polygon shapefiles all in the same folder), now I want separate each type of shapefile into one separate folder. That is all Point shapefiles into same folder, all Line shapefiles into same folder, and all Polygon shapefiles into same folder.


Here we got twelve shapefiles that look like this:-


To complete this manually, we have to open all the twelve files and save each to its respective folder. This is not we wanted and it is time consuming, so I will use python scripting to complete this.

Let's get our hands dirty...

Monday, May 3, 2021

Geo-calibrating an SVG map for use in mapsvg.com wordpress plugin

 Scalable Vector Graphic (SVG) file isn't the best for creating geospatial referenced maps. However, it is possible to maintain map data that have geo-like coordinates that are close to the geolocation they represent. SVGs are not commonly used for geo-enable maps on the web, instead they are commonly used for drawing maps just for visual purpose on web pages.

If you really want to work with geo-enabled map on a website page where spatial reference system is of paramount importance, then you should use file format such as GeoJSON. Or store your map in a spatial database.

There are many reasons why you would want to make your SVG file have spatial capability. A common reason is when you want to overlay your SVG drawing on a geographical base map as used in MapSVG wordpress plugin. The mapsvg plugin allows you to add custom maps in svg format to wordpress web pages. If you want the map overlay the svg map on Google maps background, then you must geo-calibrate the map as we will see in a moment.

The plugin isn't free, at the time of writing it is about $49. However, there is a demo admin panel (with the following login details: demo / demo123) where you can try it out for free. We will use this to upload a custom map that is geo-enable.