Convert HEXEWKB to Latitude/Longitude in python

from shapely import wkb

hexlocation_list = ["0101000020E6100000AECB9307F9D812400F2ADCE003704940", 
                    "0101000020E6100000E40AAE6CD6DA1240941F95531C704940", 
                    "0101000020E6100000C0D7C68E7CD81240F550364044704940", 
                    "0101000020E6100000CB752BC86AC8ED3FF232E58BDA7E4440", 
                    "0101000020E6100000DB81DF2B5F7822C0DFBB7262B4744A40"]


for hexlocation in hexlocation_list:
    point = wkb.loads(hexlocation, hex=True)
    
    longitude, latitude = point.x, point.y
    print(longitude, latitude)

Related articles:
1- How to convert HEXEWKB to Latitude, Longitude (in python)?

2- How to convert HEXEWKB to Latitude, Longitude (in java)?

Making the Map of 10 States Ready To Pay N30,000 Minimum Wage

President Buhari had in April 2019 signed the new wage bill aimed at boosting the morale of the Nigerian workers into law.

According to Nigeria Labour Congress (NLC), only ten (10) states listed below met the December 31, 2019 deadline set by organised labour. NLC had on December 11, 2019 at a meeting with its state chairmen in Abuja, set December 31 of the same year for all state governors to conclude negotiations with workers in their states following an agreement with the Federal Government on October 18, 2019.

As GIS practitioners, lets visualize these 10 states on a map to see if we can deduce any spatial pattern!

The 10 states are:-

• Abuja (Federal Level)
• Adamawa
• Bauchi
• Borno
• Jigawa
• Kaduna
• Kano
• Katsina
• Kebbi
• Lagos and
• Ebonyi

Mapping Steps

Step 1:
Get the vector map of Nigeria in shapefile format. You can download one from HERE (State, LGA and Wards) or from GADM Data web portal.

Step 2:
Load the state shapefile into your favorite GIS software, here I will use QGIS.


Step 3:
Select the 10 states from the list above that met the NLC December 31, 2019 minimum wage deadline.

You can do this selection from the attribute table or by simply selecting from the map directly. Then give them a different color from the remaining states.

Here I will use green color for states that met with the deadline and white for those that didn't.

To make the selection the query will look like this: "state_name" in  ('fct', 'Adamawa', 'Bauchi', 'Borno', 'Jigawa', 'Kaduna', 'Kano', 'Katsina', 'Kebbi', 'Lagos', 'Ebonyi')

The final map as seen below:-

From the resulting map, there are some spatial patter readily seen such:-

• Most of the northern states responded to the deadline.
• Most of the wealthy states with oil wells failed to met up with the deadline.
• States with larger land mass responded well
• ect

We can now begin to ask some spatial related questions to uncover some possible reasons why it was easy for these states to met the 30,000 wage on time.

• Does this has to do with the states low/high population?
• Does it has to do with the states' generated revenue?
• Does it means the Governors of those states have more sympathy for it's workers?
• Or cold it be because the states are oil producing states and receives high allocation from you he federal government?
• etc

Off curse, this is simple case map that a graphic oriented software such as coreldraw or photoshop can easily do. But think Spatially to uncover hidden information!

City-Data.com Zip Codes Data Wrangling

Here I found myself working data collection from City-Data.com where we needed to three variables ('Median year house/condo built', 'Median household income', 'Median house or condo value' and 'Median resident age') for a few hundred zip codes as seen below.

There are several polygons that makeup a zip code, so an average value for all the variable is what we needed.

The raw data collected from the web page is like this:-

55388
2004
Median household income: $74,048
Median house or condo value: $158,600
Median resident age: 30.8
1976
Median household income: $74,583
Median house or condo value: $305,400
Median resident age: 48
1962
Median household income: $68,958
Median house or condo value: $218,900
Median resident age: 41.7
1994
Median household income: $58,587
Median house or condo value: $172,900
Median resident age: 39.5
1997
Median household income: $94,042
Median house or condo value: $233,300
Median resident age: 36.7
1986
Median household income: $138,917
Median house or condo value: $446,200
Median resident age: 39.9
2000
Median household income: $104,125
Median house or condo value: $263,800
Median resident age: 34.6

The expected output from above that will be used in excel average function is like this:-

55388
=AVERAGE(2004, 1976, 1962, 1994, 1997, 1986, 2000)
=AVERAGE(74048, 74583, 68958, 58587, 94042, 138917, 104125)
=AVERAGE(158600, 305400, 218900, 172900, 233300, 446200, 263800)
=AVERAGE(30.8, 48, 41.7, 39.5, 36.7, 39.9, 34.6)

Convert list comprehension to for loop in python

Lets see the concept of going from 'list comprehension' to 'for loop'.




Above is a nested python list of list that contains names of countries. The list comprehension returns the names of countries with less than ten characters.

Lets try to 'incomprehension' the list :)... more like trying to decompress it into for loop!

The "ten_character" list will start as an empty list then we append the filtered result to it from the for loops.

So, the arrangement is to take the first for loop, follow by the second for loop, then the if conditional statement and then append the result at the end. See the script below...







The complete source code:-

# List of countries...
countries = [['San Marino', 'Sao Tome and Principe', 'Saudi Arabia', 'Senegal', 'Serbia', 'Seychelles', 'Sierra Leone', 'Singapore', 'Slovakia', 'Slovenia', 'Solomon Islands', 'Somalia', 'South Africa', 'South Georgia and the South Sandwich Islands'], 
             ['Nepal', 'Netherlands', 'Netherlands Antilles', 'New Caledonia', 'New Zealand', 'Nicaragua', 'Niger', 'Nigeria', 'Niue', 'Norfolk Island', 'North Korea', 'Northern Mariana Islands', 'Norway'], 
             ['Togo', 'Tokelau', 'Tonga', 'Trinidad and Tobago', 'Tunisia', 'Turkey', 'Turkmenistan', 'Turks and Caicos Islands', 'Tuvalu', 'U.S. Minor Outlying Islands', 'U.S. Virgin Islands', 'Uganda', 'Ukraine', 'United Arab Emirates', 'United Kingdom', 'United States', 'Uruguay', 'Uzbekistan', 'Vanuatu', 'Vatican City', 'Venezuela', 'Vietnam', 'Wallis and Futuna', 'Western Sahara'], 
             ['Afghanistan', 'Albania', 'Algeria', 'American Samoa', 'Andorra', 'Angola', 'Anguilla', 'Antarctica', 'Antigua and Barbuda', 'Argentina', 'Armenia', 'Aruba', 'Australia', 'Austria', 'Azerbaijan', 'Bahamas', 'Bahrain', 'Bangladesh', 'Barbados', 'Belarus', 'Belgium', 'Belize', 'Benin', 'Bermuda', 'Bhutan', 'Bolivia', 'Bosnia and Herzegovina', 'Botswana', 'Bouvet Island', 'Brazil', 'British Indian Ocean Territory']]

# Country name less than 10 character lenght...
ten_character = [ten for sublist in countries for ten in sublist if len(ten) < 10]

print(ten_character)

# List of countries...
countries = [['San Marino', 'Sao Tome and Principe', 'Saudi Arabia', 'Senegal', 'Serbia', 'Seychelles', 'Sierra Leone', 'Singapore', 'Slovakia', 'Slovenia', 'Solomon Islands', 'Somalia', 'South Africa', 'South Georgia and the South Sandwich Islands'], 
             ['Nepal', 'Netherlands', 'Netherlands Antilles', 'New Caledonia', 'New Zealand', 'Nicaragua', 'Niger', 'Nigeria', 'Niue', 'Norfolk Island', 'North Korea', 'Northern Mariana Islands', 'Norway'], 
             ['Togo', 'Tokelau', 'Tonga', 'Trinidad and Tobago', 'Tunisia', 'Turkey', 'Turkmenistan', 'Turks and Caicos Islands', 'Tuvalu', 'U.S. Minor Outlying Islands', 'U.S. Virgin Islands', 'Uganda', 'Ukraine', 'United Arab Emirates', 'United Kingdom', 'United States', 'Uruguay', 'Uzbekistan', 'Vanuatu', 'Vatican City', 'Venezuela', 'Vietnam', 'Wallis and Futuna', 'Western Sahara'], 
             ['Afghanistan', 'Albania', 'Algeria', 'American Samoa', 'Andorra', 'Angola', 'Anguilla', 'Antarctica', 'Antigua and Barbuda', 'Argentina', 'Armenia', 'Aruba', 'Australia', 'Austria', 'Azerbaijan', 'Bahamas', 'Bahrain', 'Bangladesh', 'Barbados', 'Belarus', 'Belgium', 'Belize', 'Benin', 'Bermuda', 'Bhutan', 'Bolivia', 'Bosnia and Herzegovina', 'Botswana', 'Bouvet Island', 'Brazil', 'British Indian Ocean Territory']]



ten_character = []

for sublist in countries:
    for ten in sublist:
        if len(ten) < 10:
            ten_character.append(ten)

            
print(ten_character)

That is it!

JavaScript Recursive Factorial Algorithm

This blog post is about "Recursive Function in JavaScript". The ability for a function to call it self is know as 'Recursion'.

A recursive function has two parts namely: the base case (which gets called when the recursion is satisfied needs to terminate) and the recursive case (which continue to call the function they are in until the base case condition is satisfied)

Recursion is particularly useful for divide and conquer problems. A simple problem that naturally best solved by recursive solution is 'calculating factorials'.

The recursive factorial algorithm defines two cases:
~ the base case when x is zero, and
~ the recursive case when x is greater than zero.

Factorial of a number 'x' is the product of all positive integers less than or equal to the number 'x'. For example the factorial of 6 is: 1 x 2 x 3 x 4 x 5 x 6 = 720

The implementation is as follow:-

function myRecur_factorial(x){
    
    if (x == 1){
        return x; // This is the: Base Case
    } else{
        return(x * myRecur_factorial(x-1)) // This is the: Recursive Case
    }
}

x = 6

console.log(`The factorial of ${x} is: ` + myRecur_factorial(x))

Here is a non-recursive approach to writing factorial of a number in JavaScript.

This make use of 'range function' and the 'for loop'. Because JavaScript has built-in range function, we will define one as follow..

function range(start, end) {
    return (new Array(end - start + 1)).fill(undefined).map((_, i) => i + start);
}

number = 6
factorial = 1


for (i of range(1, number)) {
    
    factorial = factorial * i;
}

console.log(`The factorial value for the ${number} is: ` + factorial);

Recursion and Iteration are somewhat similar and often confused by programmers because they both repeat a series of operations. It is mostly not clear if recursion or iteration is a better solution to a particular problem. The following table highlights some differences between recursion and iteration:

Recursion	Iteration
When the base case is reached, it terminates	It terminates when a defined condition is met
Space in memory is required to store each  recursive call	Iteration is not stored in memory
Stack Overflow Error occurs when a recursion is  infinite	Infinite iteration will not return error

Another example of recursion is when we used a recursive approach to generating all the possible permutations of a given string, X, of a given length Y:

QGIS - Remove Neighboring features in Atlas map

Lets take a polygon layer as an example. As seen below, I need to make an atlas map showing only a single polygon feature from the states layer. So, I want only the polygon marked green to be displayed while others in red cross should be hidden.

The end result would be this...

The solution

Step 1: Add an 'id' field to the attribute table. If the 'id' field already exist, then skip this step.

Step 2: Set the layer symbology to 'Rule-based' and set the filter rule to '$id = @atlas_featureid'

Step 3: Click on 'Ok' to apply the settings. Now the all other polygon will remain hidden except the one in atlas preview.


That is it!

Generating Zip code map download links

Overview
For each of the US states zip code map listed on this page, we want to construct the map download link.

There several ways to get this task completed including using selenium or request/beatifulsoup modules. However, in this exercise, we are going to keep it simple and assume that we know the pattern at which the maps download links are made-up (indeed the pattern is same for all states and readily known) and we just need to generate them based on the file names.

The download link pattern is: URL + state_name + -zip-code-map.png

Examples:-

https://www.unitedstateszipcodes.org/maps/alabama-zip-code-map.png, 

https://www.unitedstateszipcodes.org/maps/alaska-zip-code-map.png, 

https://www.unitedstateszipcodes.org/maps/arizona-zip-code-map.png, 

e.t.c

So, we can easily construct each state's download link from their respective names...

Objectives
At the end of this tutorial, you should become familiar with:-
a) To become familiar with using string concatenation
b) To become familiar with using string split(), replace() and lower() methods
c) To become familiar with convert pandas series to list using tolist() method
d) To be able to use for loop to append string to empty list



Pseudocode
a) Read the spreadsheet containing the file names into a list
b) Clean the names to remove unwanted characters such as ' ('
c) Concatenate the strings to form the URLs



Code Snippet

import pandas as pd

# Read the spreadsheet file...
zip_df = pd.read_csv(r"C:\Users\Yusuf_08039508010\Desktop\GIS Data Processing Scripts\US_ZipMap_Size.csv")

# Convert the column to a list...
zip_list = zip_df['Maps'].tolist()


# ----------------------
# For each item in the list, split at ' (' and keep the first part...
name_list = []
for item in zip_list:
    name_list.append(item.split(' (')[0])


# ----------------------
# Download link URL is: 'https://www.unitedstateszipcodes.org/maps/' + stateName + '-zip-code-map.png'
download_link = []
for name in name_list:
    download_link.append('https://www.unitedstateszipcodes.org/maps/' + name.replace(' ', '-').lower() + '.png')

download_link

Explanation

Step 1: First we import pandas module and read the spreadsheet file into a dataframe.

import pandas as pd

# Read the spreadsheet file...
zip_df = pd.read_csv(r"C:\Users\Yusuf_08039508010\Desktop\GIS Data Processing Scripts\US_ZipMap_Size.csv")

# Convert the column to a list...
zip_list = zip_df['Maps'].tolist()

Step 2: Next, we need to split the sting and keep the useful part. The part needed is that before the ' (' character. Note that the character has a space followed by the open brace/parentheses.

# ----------------------
# For each item in the list, split at ' (' and keep the first part...
name_list = []
for item in zip_list:
    name_list.append(item.split(' (')[0])

Step 3: The last step is to replace spaces within the string by '-' and concatenate the url string to the variable string. The first part of the string is: 'https://www.unitedstateszipcodes.org/maps/' while the end part of the string is: '.png'

download_link = []
for name in name_list:
    download_link.append('https://www.unitedstateszipcodes.org/maps/' + name.replace(' ', '-').lower() + '.png')

Assignment Takeaway
An exercise to help you learn further is: Write a script that will extend the above script by adding the resulting list to a new column that corresponds to the file names as seen below, then save the result to spreadsheet file. The result will look like this:-

Reference Material

1] https://www.programiz.com/python-programming/methods/string/split
2] https://www.w3schools.com/python/ref_string_split.asp
3] https://www.programiz.com/python-programming/methods/string/replace

Building SQL Expressions in ArcGIS and QGIS

In the modern GIS industry, programming skill is an essential asset and one of the programming languages the is very popular within the industry is Structure Query Language (SQL) as you will later find out how it is been used in few moments.

Most of the query expressions used in ArcGIS or QGIS desktop software are derived from SQL. SQL is a standard language for storing, manipulating and retrieving data in databases.

Both ArcGIS and QGIS support the following common database engines: SQLite, MySQL, SQL Server, MS Access, Oracle, Sybase, Informix, PostgreSQL, and other database systems. When connected to any of them, you can take full advantage of SQL in GIS.

For small GIS projects where database isn't required, we make use of traditional GIS files type such as "Shapefile" which allow us to use query expressions that adhere to standard SQL expressions.


What is an expressions?
An expressions is a combination of "Constants, Variables, Operators and Functions" organized in an ordered statement that returns an output value. Expressions are unique to the computer language they are created in. An example of SQL expressions is: "SELECT * FROM <Layer_name> WHERE <Field_name> <Operator> <Value or String>".

If you data resides in a shapefiles or any of these (coverages, INFO tables, and dBASE tables), this part of the SQL expression (SELECT * FROM <Layer_name> WHERE) is automatically supplied for you, so you only provide this part (<Field_name> <Operator> <Value or String>) to query your data.

Since GIS data is made up of Spatial and Attribute, it is worth noting that 'Attribute Query' is similar to the standard SQL queries found in traditional database systems and this will be our focus in this article. On the other hand, 'Spatial Query' which allow operations such as "Contains, Equals, Intersects, Is Disjoint, Overlaps, Touches, Within and Crosses" requires some extension installed on traditional database systems to make them work.


Building SQL Expressions for Attribute Query

There are many places you can build expressions within both ArcGIS and QGIS software, some of the common places are listed below:-

ArcGIS Tools:
~ Select By Attribute
~ Definition Query
~ Field Calculator
~ Label Expression Dialog Box
~ Add Query Layer

QGIS Tools:
~ Select By Expression
~ Filter Query Builder
~ Field Calculator
~ Label Expression Dialog Box
~ Layer Property Display
~ Database Manager

Calculating the total size of zip code maps

On this US printable zip codes maps page, there is a list of all the US states zip code maps with their respective sizes in braces like this "Alabama ZIP Code Map (3.59MB)" as seen below...

Lets calculate the total size of all the maps using python scripting!

Off course, there are several or even better ways to get this done. But here we want to test our python skills on this, let us stick to using python 😏.

Some other reasons it is good idea we use python is that we can easily use our python skill to:-
1) make HTTP request to scrape/download the map data
2) generate the download links on the fly
3) create a bot to monitor change in map size (which could indicate the map has been updated).
4) visualization of the string including map/geographic visualization.

The list can go on and on, but I will keep it simple here to just calculate the total sum the map sizes.

Step 1:
First thing is to get the string/text off the web page into our python environment. There are several ways to do this as I have mentioned above, but I will just select, copy and paste it in a CSV file as seen below.

Step 2:
Read the CSV file in python. Here I will use the pandas module to read the CSV file, could have also used the CSV module to do this.

Filtering Missing Zip codes out of master Zip codes list

Here I have a list of zip codes, I want to know the missing zip code from the given list (these are the postal code in Texas, USA).

List 'available_zipcodes' contains the master zip codes and list 'given_zipcodes' contain the provided or working zip codes. Now I want check and filter out those zipcode that are NOT in the master zip codes.

These three lines of python code below will do it. It uses the 'for' loop with and 'if' statement. Basically, we loop through the list of 'given_zipcodes' and if it is not in the 'available_zipcodes', then we print it out.

If you care to run the script and don't want to type all that out, here below is the Code is...

available_zipcodes = [77389, 77086, 77346, 77018, 77040, 77388, 77065, 77080, 77041, 77396, 77385, 77354, 77382, 77067, 77066, 77090, 77345, 77355, 77373, 77339, 77043, 77302, 77304, 77070, 77375, 77095, 77433, 77069, 77038, 77091, 77380, 77092, 77316, 77429, 77377, 77379, 77064, 77088, 77338, 77449, 77386, 77381, 77493, 77356, 77068, 77014, 77084, 77055, 77301, 77303, 77384]

given_zipcodes = [77325, 77339, 77345, 77346, 77380, 77381, 77382, 77383, 77384, 77385, 77386, 77301, 77302, 77303, 77304, 77316, 77354, 77356, 77389, 77014, 77018, 77038, 77040, 77041, 77043, 77055, 77064, 77065, 77066, 77067, 77068, 77069, 77070, 77080, 77084, 77086, 77088, 77090, 77091, 77092, 77095, 77375, 77377, 77379, 77388, 77429, 77433, 77449, 77493, 77373, 77338, 77347, 77391, 77396, 77355]


for zipcode in given_zipcodes:
    if zipcode not in available_zipcodes:
        print(zipcode)

In the case above, the missing zip codes are: 77325, 77383, 77347, 77391

Note: In a production job. these zip codes will probably come in a text file, just read the file into python lists and loop through as seen above.

That is it!