Thursday, September 17, 2020

PyQGIS - Get attribute column names and dataType

 Ocationally, I did like to check the attribute fields/columns of a layer I am working on. Sometimes to get their names, types or just to get the count of how many fields/columns are on that layer.

You could access some of these information when you open the layer's property window, under the information tab as seen below.

Thursday, September 10, 2020

Pandas Dataframe to Python Dictionary

Assuming we have this table of "Confirmed COVID-19 Cases in Nigeria" as seen below and we want to convert either the whole table or some of its columns to a python dictionary. If you don't know already, a python dictionary is a data structure that consists of a collection of key-value pairs.

The column are as follow:-

  1. Column A: States Affected
  2. Column B: No. of Cases (Lab Confirmed)
  3. Column C: No. of Cases (on admission)
  4. Column D: No. Discharged
  5. Column E: No. of Deaths

Monday, August 31, 2020

Generating fake or dummy data using Python Faker library

There are many legit reasons why you will want to have access fake/dummy data sets. Some of these reasons are listed below:-
1) They are very useful when you are just starting out building an app and you don't have any data yet.
2) It is useful for testing or filling databases with some dummy data.
3) To protect data Privacy due to security and many other constraints.

There times when you need to have access to large amount of real world data to test an app you are developing. Then you suddenly realized that you don't have such data set available so you won't be able to put your app to real world test before it's final launch.

If you find yourself in the cenario above, then you are not alone. On this page, I will introduce you to a python module that will help you generate good amount of dummy or fake data that looks just like the once in real life for you to test run your application.

The name of the module is called: faker

Faker library
With this python package, you can generate test data without infringing on peoples' privacy. For example you can generate real names, addresses, latitude/longitude coordinates, phone numbers, fax numbers, occupations, profile titles, email addresses, website addresses, job titles, text data, random numbers, currencies, words, birthdates, hashes and uuids, date/time etc.

Let assume we are need to test a banking database, so due to sensitive nature of this kind of data we can use a real production data. So we need dummy/fake people's bank details to test run the app database.

This is where the python Faker library comes in handy. Let see how it is used.

First step is to install it using:
pip install Faker

Next is to import the module and initialize a faker generator like this:
from faker import Faker
fake = Faker()
Now we can use the fake object to generate all sorts of data type attributes as follow:-

# Numerical data type
fake.pydecimal(left_digits=5, right_digits=3, positive=True, min_value=None, max_value=None)
fake.pyfloat(left_digits=3, right_digits=3, positive=False, min_value=None, max_value=None)
fake.pyint(min_value=0, max_value=9999, step=1)

# String data type

# Internet related strings...

# Date/Time ....

# Other data types/structures
fake.random_int(0, 100) # fake.random_int(min=0, max=9999, step=1)
fake.pystr(min_chars=None, max_chars=10)
fake.pylist(5, False, 'str') # (nb_elements=5, variable_nb_elements=True, *value_types='str')
fake.pytuple(10, True, 'str') # (nb_elements=10, variable_nb_elements=True, *value_types='tuple')
fake.pydict(10, True, 'url') # (nb_elements=10, variable_nb_elements=True, *value_types='url')
fake.pyiterable(10, True, 'date') # (nb_elements=10, variable_nb_elements=True, *value_types='date')
fake.pyset(10, True, 'list') # (nb_elements=10, variable_nb_elements=True, *value_types='list')
fake.pystruct(10, 'float') # (count=10, value_types='float') - NOTE: *value_types can be any of the datatypes: int, float, str, url, date, list, tuple, dict, set

# If you noticed the issue with *, then see this link:

Wednesday, August 5, 2020

PyQT5 and wxPython Implimentation of ROT13

As stated on this wikipedia page, ROT13 ("rotate by 13 places", sometimes hyphenated ROT-13) is a simple letter substitution cipher (a secret or disguised way of writing) that replaces a letter with the 13th letter after it, in the alphabet.

There is an implimentation in javascript at as seen below. Here on this blog we will walk through implimenting it in python desktop GUI (both PyQT5 and wxPython).

Reviewing where rot13 was used in python
The same secret or disguised way of writing is behind the python built-in module "The Zen of Python, by Tim Peters" (import this).

When you run/import the 'this' module for the first time, it display some strings as seen below.

If you access the location of the 'this' module and open it in text editor, you will find a abstract "s" string variable. So where is the beautiful and explicit text above coming from?

s = """Gur Mra bs Clguba, ol Gvz Crgref

Ornhgvshy vf orggre guna htyl.
Rkcyvpvg vf orggre guna vzcyvpvg.
Fvzcyr vf orggre guna pbzcyrk.
Pbzcyrk vf orggre guna pbzcyvpngrq.
Syng vf orggre guna arfgrq.
Fcnefr vf orggre guna qrafr.
Ernqnovyvgl pbhagf.
Fcrpvny pnfrf nera'g fcrpvny rabhtu gb oernx gur ehyrf.
Nygubhtu cenpgvpnyvgl orngf chevgl.
Reebef fubhyq arire cnff fvyragyl.
Hayrff rkcyvpvgyl fvyraprq.
Va gur snpr bs nzovthvgl, ershfr gur grzcgngvba gb thrff.
Gurer fubhyq or bar-- naq cersrenoyl bayl bar --boivbhf jnl gb qb vg.
Nygubhtu gung jnl znl abg or boivbhf ng svefg hayrff lbh'er Qhgpu.
Abj vf orggre guna arire.
Nygubhtu arire vf bsgra orggre guna *evtug* abj.
Vs gur vzcyrzragngvba vf uneq gb rkcynva, vg'f n onq vqrn.
Vs gur vzcyrzragngvba vf rnfl gb rkcynva, vg znl or n tbbq vqrn.
Anzrfcnprf ner bar ubaxvat terng vqrn -- yrg'f qb zber bs gubfr!"""

d = {}
for c in (65, 97):
    for i in range(26):
        d[chr(i+c)] = chr((i+13) % 26 + c)

print("".join([d.get(c, c) for c in s]))

It turns out that the module uses the rot13 algorithm to display beautiful text on the front end. This is easily verified by copy the string into as shown below.

Friday, July 31, 2020

Mapping African countries open for international tourism

On the 14th of February 2020, the COVID-19 pandemic was confirmed to have spread to Africa, which leads to shutting down of many activities including the 'International Tourism' sector. The first confirmed case was in Egypt, and the first confirmed case in sub-Saharan Africa was in Nigeria. Learn more on this Wikipedia page.

Five months later, a lot has changed, and some of the Africa countries are reopening for tourism activities. Check "Africa Reopening For International Tourism – A-Z List of Countries" for more details.

Given this, I decided to create a map to show the current reopening status (as at the 31st of July, 2020) of each country as either: 'Not Open for Tourism', 'Reopening Soon', or 'Now Open'.

As at the 31st of July, 2020 the Africa international tourism opening status is as follow:-
a) Reopening Soon = 13 countries
b) Not Open For Tourism = 28 countries
c) Now Open = 10 countries

The status of the four countries is unknown, and these countries are Western Sahara, Ivory Coast, Comoros and Swaziland.

See detailed breakdown on the table below...

To prepare the map, we will just link the status above to the map of Africa and the categorize the countries base on the status as seen below.

Stay safe and happy tourism.

Thursday, July 16, 2020

Fix 'wish.exe' cannot be found while running TkInter's GUI builder (PAGE)

This guide will show you how to fix 'wish.exe' cannot be found while running page gui builder for python on windows OS.

When you download PAGE and installed it on windows, you will get this error "Windows cannot find 'wish.exe'. Make sure you typed the name correctly, and then try again.".

This is because PAGE (the Python Automatic GUI Generator) requires Tcl/Tk ActiveTcl package. Tcl/Tk is required for executing PAGE. Tcl/Tk is required because PAGE is written in Tcl/Tk.

The solution
Head over to ActiveTcl page, download and install a copy for your windows OS.

You may be promted to register on the website before you download it, just follow the prompts and download a copy of ActiveTcl. Then run it as usual to install it.

With this out of the way, your PAGE TkInter GUI Generator software should lunch correctly as seen below.


Tuesday, July 14, 2020

Extracting the Duration of Video file

Few days ago, I was working on a project where I have to lookup the duration of video files and compare the length duration on local disk with the corresponding duration of video online.

So, instead of going between video on local disk and online repeated, I thought what if I could use python to extract all the duration lengths into a table (CSV/Excel) and compare the columns.

Here is how I went about doing it. I made use of 'moviepy' which is a Python module for video editing, which can be used for basic operations (like cuts, concatenations, title insertions).

Installing moviepy
Use: pip install moviepy
This will some other dependent packages such as: decorator, tqdm, urllib3, idna, chardet, requests, proglog, numpy, pillow, imageio, imageio-ffmpeg

To be sure you installation was successful, try to import the module as seen above. If you got not error then your installation was successful and you are good to move on.

The poblem
Here I have some set of videos I downloaded from an online course and I listed their durations in a spreadsheet file as seen below. Now I want to compare each video duration/length to be sure it was downloaded correctly.

The code to extract the video duration to be compared withwhat was obtained online is as below:-

import glob
import datetime
from moviepy.editor import VideoFileClip

folder_path = r'C:\Users\Yusuf_08039508010\Desktop\videos_tut'
videoFiles = glob.glob(folder_path + str('\\*ts'))

# Converts Seconds to: hours, mins, seconds using 'datetime' module
def convert_sec(vid_seconds):
 return str(datetime.timedelta(seconds=vid_seconds))

for v in videoFiles:
    clip = VideoFileClip(v)
    # duration in seconds (create custom function to convert to: H:M:S)
    video_duration = clip.duration

    print (v.split('\\')[-1], convert_sec(int(video_duration)))

Basically, the code uses the 'glob' module to access all the videos in the folder and used the 'datetime' module to convert the video duration in seconds provided by the 'moviepy' module.

The result is a seen below;-

Thank you for reading.

Wednesday, July 8, 2020

PyQT5 and wxPython - Hex Color Spinner

Lets make a 'Hex Color Spinner' desktop app in both PyQT5 and wxPython.

This is going to be a simple desktop window frame that will spin out random hex color and value when a button is pressed.

By working through this app, we will get familiarized with:-
1) creating the GUI structure in both Qt designer and wxformbuilder.
2) connecting the widgets to functions.
3) read and update values on widgets.
4) change window's background color

Hexadecimal color code
A hex color code is made-up of '#' sign followed by six characters which include the combination of numbers between 0-9 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9) and letters between A-F ("A", "B", "C", "D", "E", "F"). Notye that the letters can either be in upper or lower case.

So, for example: #F35EEA, #5BDB6B, #9FE091, #8DA7ED, #ADD075 etc will all yeild unique HEX colors.

To come up with such random string in python, there are many possible ways. Below is one way to do it.

# import 'choice' function from the random module...
from random import choice

# define a list of legal HEX characters...
hex = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, "A", "B", "C", "D", "E", "F"]

# loop through the list six time while selecting one character per loop...
hex_list = [str(choice(hex)) for x in range(6)]

# join the choosen characters and prefix with the '#' symbol...
hex_string = '#' + ''.join(hex_list)

Now that we have a python code that generates the hex color code, we will wrap it in a function to be called within our GUI.

For the GUI in pyqt, I made use of Qt designer to quickly created it as seen below.

The qt designer GUI consist of a QLable and a QPushButton arranged in a verticalLayot. I made use of verticalSpacer to give space above and belwo the widgets which were arranged vertically.

wxPython GUI
For the GUI in wxPython, I made use of wxFormBuilder to quickly created it as seen below.

The GUI is made off wxStaticText and wxButton arranged in a vertical BoxSizer. A spacer is used to give space above and belwo the widgets.

Running the App

In both cases above, we will generate the GUI file and call it in a file which loads the GUI and then connect the button to the function that runs the hex color code generator above.

PyQt5 code

import sys
from PyQt5.QtWidgets import *
from PyQt5.QtCore import *
from PyQt5.QtGui import *
from PyQt5 import uic

from random import choice

class MyWindow(QMainWindow):
 """docstring for MyWindow"""
 def __init__(self):
  super(MyWindow, self).__init__()
  uic.loadUi('gui_Project.ui', self)


 def change_color_func(self):
  # print('#####################')
  hex = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, "A", "B", "C", "D", "E", "F"]

  hex_list = [str(choice(hex)) for x in range(6)]

  hex_string = '#' + ''.join(hex_list)

  # ------------------------
  # Change the text of '#XXXXXX' to current hex text

  # ------------------------
  # Change background color to current hex color
  self.setStyleSheet("background-color: {};".format(hex_string))

if __name__ == '__main__':
 app = QApplication(sys.argv)
 window = MyWindow()

wxpython code

import wx
from gui import MyWindow_1

from random import choice

class MyWindow_2(MyWindow_1):
 def __init__(self, parent):
  MyWindow_1.__init__ (self, parent)
 def change_color_func(self, event):
  # print('#####################')
  hex = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, "A", "B", "C", "D", "E", "F"]

  hex_list = [str(choice(hex)) for x in range(6)]

  hex_string = '#' + ''.join(hex_list)

  # ---------------
  # Change the text of '#XXXXXX' to current hex text

  # ------------------------
  # Change background color to current hex color

app = wx.App(0)

The result should look like below....

Download the files from this github repository.

Tuesday, June 23, 2020

Python pandas get length (row count) of a dataframe

There are several ways of knowing the length of your dataframe rows. Here are three ways using index, shape and count methods.

Manually inspecting this dataframe below, we see that it has 10 rows (note the index count starts from 0). Now, let check for this information dynamically.

1) index
Using the index method, it returns the 'RangeIndex' start, stop and step for the dataframe. So, wrapping it in a len() function will give the stop value that is the length of the datafarme as seen below.

2) shape
Using the shape method will return a tuple like so (row, col), where the first element is row and the second element is column.

3) count
Using the count method returns a 'pandas.core.series.Series' that contains number of items for each column.


Thursday, June 18, 2020

SVG2GeoJSON - SVG file with added Geo-Referencing

Recently, I had a task to convert an SVG image map into GeoJSON format for use in the GIS. This is a complicated thing to do because SVG vector images don't have spatial reference.

Fortunately, I stumbled on this NodeJS app package named svg2geojson written by Prognoz. This library converts an SVG image to a (slightly-incorrect) GeoJSON

To make use of this library, you need to update your SVG map by placing two 'GeoItems' inside a Prognoz MetaInfo element as a direct child of the '<svg>' element at the root of your document.

Now how do you find the X/Y and Longitude/Latitude attributes within the 'GeoItems' for your specific SVG image? This question is what I will attempt to answer in this blog post.

The svg2geojson documentation says: "These map opposing X/Y corners in your SVG coordinate space to Longitude/Latitude coordinates on the world. Note that the SVG coordinate space has Y increasing down (toward the south), while Latitude increases upwards (towards the north)."

That is something like the daigram below:-