Running a custom python function in QGIS

 In this post, I will explain how custom python functions are made in QGIS. 

Functions in QGIS are listed with the following categories:-

Aggregates Functions
Array Functions
Color Functions
Conditional Functions
Conversions Functions
Custom Functions
Date and Time Functions
Fields and Values
Files and Paths Functions
Form Functions
Fuzzy Matching Functions
General Functions
Geometry Functions
Layout Functions
Map Layers
Maps Functions
Mathematical Functions
Operators
Processing Functions
Rasters Functions
Record and Attributes Functions
Relations
String Functions
User Expressions
Variables
Recent Functions

So, if QGIS has all these functions why would one ever need a custom function?

The answer is simple, these are not all the functions in the world. So, most likely there is a function that hasn't been implemented then you can write your custom function.

Lets look at a simple scenario.

Assuming, we have this little python script that generates Hex color codes like these: #40B994, #13E7BC, #3F50EB, #E28326 etc and we want to use it to generate new attribute column with random hex color codes.

from random import choice

def color_func():
    hex = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, "A", "B", "C", "D", "E", "F"]

    hex_list = [str(choice(hex)) for x in range(6)]

    hex_string = '#' + ''.join(hex_list)
    return hex_string

The function will be called like this in the field calculator. Note the function's name is: color_func()

Convert cURL command to Python Request

cURL stands for Client URL (Uniform Resource Locator). It is a tool to transfer data from or to a server, using any of the supported protocols (HTTP, FTP, IMAP, POP3, SCP, SFTP, SMTP, TFTP, TELNET, LDAP or FILE).

A simple usage call is: curl http://umaryusuf.com

This will return the HTML content of the website page provided as seen below.

cURL is commonly use by service provide to grant access to their API request without depending on any programming language.

As an example, lets convert this sample API Code from ScraperAPI in cURL format to python requests format:-

curl "http://api.scraperapi.com?api_key=we709a6dkbask80kjbaskjoie2nsaqa7&url=http://httpbin.org/ip"

There are several ways to convert cURL to python, the one I use more often is a tool by Nick Carneiro (https://curl.trillworks.com). So, copy and paste the cURL code to generate a python version.

import requests

params = (
    ('api_key', 'we709a6dkbask80kjbaskjoie2nsaqa7'),
    ('url', 'http://httpbin.org/ip'),
)

response = requests.get('http://api.scraperapi.com/', params=params)

#NB. Original query string below. It seems impossible to parse and
#reproduce query strings 100% accurately so the one below is given
#in case the reproduced version is not "correct".
# response = requests.get('http://api.scraperapi.com?api_key=we709a6dkbask80kjbaskjoie2nsaqa7&url=http://httpbin.org/ip')

That is it!

Assign color to vector layer based on HTML Notation (HEX color) codes in attribute table

 Here, I have a polygon layer with 'color' attribute column which contains HEX color codes as seen below.

We want to assign each polygon to its attribute color value, for example: Delta = #6808C9.

This is close to 'Categorized Symbology', but the only difference is that the colors are from the attribute column. To do this, we have to edit the 'Fill Color' expression to read the color column.

This way, each polygon is given a HEX color code that corresponds to the value on the attribute column.

Happy Mapping!

Map two data frames base on common column

 The aim is to enrich a dataframe from a second dataframe. In other words, copy corresponding cell in a dataframe onto a second dataframe based on their common columns.

This is very common task that is difficult to explain! This works like a "vLookUp" function found in MS excel. Where we have two tables and we want to update the second table using correspond values based on column that relates them.

If the two tables have same name for the relating/common column, then the merge() function can be used like so: -

df2.merge(df1, on="common_column_name", how='inner')

On the other hand, if the relating/common column have varying column names and you don't want rename them. The below example would be more flexible:-

We create dictionary from columns in df2 and map it to df1

import pandas as pd

df1 = pd.DataFrame({'lkey': ['foo', 'bar', 'baz'],
                    'value0': [1, 2, 3]})

df2 = pd.DataFrame({'rkey': ['foo', 'bar', 'baz'],
                    'value1': [5, 6, 7],
                   'value2': [8, 9, 10]})

# This is a list of columns in: df2
s = ['rkey', 'value1', 'value2'] # Don't want to create it manually? then use: s = df2.columns.tolist()

# loop through all columns of df1 and map to df2
for column in s:
    # make a mapping dictionary from df...
    mydict = dict(zip(df2['rkey'], df2[column]))
    
    # create new column based on the map column and dictionary
    df1['new_'+column] = df1['lkey'].map(mydict)

df1

Here below is a real world example:-

Below are two tables with different attribute columns from countries in the world. The 'Country Name' and 'Country Code' columns are the same for both tables, so we need to merge them together.

Since, the two table have the same column names we can use this approach...

df1.merge(df2, on="Country Name", how='outer')

In case, the result isn't what you expected. Then play around with the parameters to get what you wanted.

Enjoy!

QGIS - Solve Launch alert on "python modules numpy and matplotlib both not installed"

When you start QGIS and got a screen start error that reads:-

The contour plugin is disabled as it requires python modules numpy and matplotlib which are not both installed.

This is custom error caused by a deprecated matplolib import command "from matplotlib.mlab import griddata". You can click on 'Ok' button to continue loading the QGIS software.

However, clicking on that button is somewhat annoying and you will definitely like to fix the error permanently.

The error comes from the 'contour' plugin that was installed in the plugins directory at: 
C:\Users\YourPcName\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\contour

Open the python file "ContourDialog.py" in your text editor. You would find and comment out the import statement "from matplotlib.mlab import griddata" and save the file.

This should fix the warning error massage from launching the software when you restart it.

The warning message box that pops up above is actually located in this same file somewhere below.

That is it!

Creating Zip file for items within multiple folders

 The script below will access all children folders within a parent directory then zip the contents and return back to the parent directory before moving to the next child folder.

The process continue until all children folders are processed.

# list of folders to zip their contents...
folders = glob.glob('*')

# Change dir to the first folder, zip its contents and return back to parent dir...
for folder in folders:
    # Change directory to folder in list of folders...
    os.chdir(folder)
    
    list_files = glob.glob('*')
    with zipfile.ZipFile(folder+'.zip', 'w') as zipF:
        for file in list_files:
            zipF.write(file, compress_type=zipfile.ZIP_DEFLATED)
    
    # Change directory back to main folder...
    os.chdir(r"C:\Users\Yusuf_08039508010\Documents\DandR")

print('Done...')

Note that the script uses the folder's name to name the resulting zip file as seen below.

That is it!