Sunday, July 31, 2022

Getting Started with ArcGIS and QGIS Python Scripting API - ArcPy and PyQGIS


On this blog, I intend to document some common GIS operation via the Python Scripting API for both ArcGIS (ArcPy) and QGIS (PyQGIS).

Lets begin....


1) The documentation

When you decided to work with a new API, then its documentation should always be your first place to learn more about the API's capabilities. Here below is where you will find the respective docs as at the time of writing.

The ArcGIS (ArcPy) is available at: http://desktop.arcgis.com/en/documentation/
Note that there are two versions of the ArcGIS desktop software: ArcMap (Including ArcCatalog, ArcScene, & ArcGlobe) and ArcGIS Pro. The focus of this blog is on: ArcMap.




 The QGIS (PyQGIS) is available at: QGIS Python API documentation project



You can compliment this docs with good books written by different authors:-

ArcPy related Books:
~ ArcPy and ArcGIS – Geospatial Analysis with Python by Silas Toms
~ Python Scripting for ArcGIS by Paul A. Zandbergen
~ Python For ArcGIS by Laura Tateosian
~ Python Scripting for ArcGIS by Paul A. Zandbergen


PyQGIS related Books:
~ PyQGIS developer cookbook by QGIS Project Team
~ The PyQGIS Programmer's Guide: Extending QGIS 3 with Python 3 by Gary Sherman
~ The Pyqgis Programmer's Guide by Gary Sherman
~ Mastering Geospatial Development with QGIS 3.x: An in-depth guide to becoming proficient in spatial data analysis using QGIS 3.4 and 3.6 with Python by Shammunul Islam, Simon Miles, et al.
~ QGIS Python Programming Cookbook by Joel Lawhead
~ Building Mapping Applications with QGIS by by Erik Westra


2) Launch and customize python window/console

In ArcGIS it is located under 'Geoprocessing' menu while in QGIS, it is under the 'Plugin' menu. Access the respective menus and launch the python window/console. you could also launch the window/console from the related icon on the tools bar.





To customize the ArcGIS python window, right click on the environment and select what you want to customize.



To customize the QGIS python console, click on the 'options' button as seen below. Then select the settings you want to customize accordingly.



3) The built-in documentation

Lets try to see the list of valid methods and attributes for the APIs. The primary object in the ArcGIS API is 'arcpy' so lets call the dir() method on it like this: dir(arcpy)
>>> dir(arcpy)
['ASCII3DToFeatureClass_3d', 'ASCIIToRaster_conversion', 'AcceptConnections', 'AddAttachments_management', 'AddCADFields_conversion', 'AddCodedValueToDomain_management', 'AddColormap_management', 'AddDataStoreItem', 'AddEdgeEdgeConnectivityRuleToGeometricNetwork_management', 'AddEdgeJunctionConnectivityRuleToGeometricNetwork_management', 'AddError', 'AddFeatureClassToTerrain_3d', 'AddFeatureClassToTopology_management', 'AddFieldConflictFilter_management', 'AddFieldDelimiters', 'AddFieldToAnalysisLayer_na', 'AddField_management', 'AddFilesToLasDataset_management', 'AddGeometryAttributes_management', 'AddGlobalIDs_management', 'AddIDMessage', 'AddIncrementingIDField_management', 'AddIndex_management', 'AddItem_arc', 'AddJoin_management', 'AddLocations_na', 'AddMessage', 'AddRasterToGeoPackage_conversion', 'AddRastersToMosaicDataset_management', 'AddRepresentation_cartography', 'AddReturnMessage', 'AddRuleToTopology_management', 'AddSpatialIndex_management', 'AddSubtype_management', 'AddSurfaceInformation_3d', 'AddTerrainPyramidLevel_3d', 'AddToolbox', 'AddWarning', 'AddXY_arc', 'AddXY_management', 'AddZInformation_3d', 'Adjust3DZ_management', 'AggregatePoints_cartography', 'AggregatePolygons_arc', 'AggregatePolygons_cartography', 'AlignFeatures_edit', 'AlignMarkerToStrokeOrFill_cartography', 'AlterAliasName', 'AlterField_management', 'AlterMosaicDatasetSchema_management', 'AlterVersion_management', 'AnalyzeControlPoints_management', 'AnalyzeDatasets_management', 'AnalyzeMosaicDataset_management', 'AnalyzeToolsForPro_management', 'Analyze_management', 'Annotation', 'AppendAnnotation_management', 'AppendControlPoints_management', 'AppendParcelFabric_fabric', 'AppendTerrainPoints_3d', 'Append_arc', 'Append_management', 'ApplyBlockAdjustment_management', 'ApplySymbologyFromLayer_management', 'ArcDLG_arc', 'ArcRoute_arc', 'ArcS57_arc', 'ArcSDESQLExecute', 'ArealInterpolationLayerToPolygons_ga', 'Array', 'AsShape', 'Aspect_3d', 'AssignDefaultToField_management', 'AssignDomainToField_management', 'AverageNearestNeighbor_stats', 'BatchBuildPyramids_management', 'BatchCalculateStatistics_management', 'BatchProject_management', 'BearingDistanceToLine_management', 'Buffer3D_3d', 'Buffer_analysis', 'Buffer_arc', 'BuildBoundary_management', 'BuildFootprints_management', 'BuildMosaicDatasetItemCache_management', 'BuildNetwork_na', 'BuildOverviews_management', 'BuildPyramids_management', 'BuildPyramidsandStatistics_management', 'BuildRasterAttributeTable_management', 'BuildSeamlines_management', 'BuildStereoModel_management', 'BuildTerrain_3d', 'Build_arc', 'CADToGeodatabase_conversion', 'CalculateAdjacentFields_cartography', 'CalculateAreas_stats', 'CalculateCellSizeRanges_management', 'CalculateCentralMeridianAndParallels_cartography', 'CalculateDefaultClusterTolerance_management', 'CalculateDefaultGridIndex_management', 'CalculateDistanceBand_stats', 'CalculateEndTime_management', 'CalculateField_management', 'CalculateGridConvergenceAngle_cartography', 'CalculateLineCaps_cartography', 'CalculateLocations_na', 'CalculatePolygonMainAngle_cartography', 'CalculateRepresentationRule_cartography', 'CalculateStatistics_management', 'CalculateTransformationErrors_edit', 'CalculateUTMZone_cartography', 'CalculateValue_management', 'CalibrateRoutes_lr', 'CentralFeature_stats', 'ChangeLasClassCodes_3d', 'ChangePrivileges_management', 'ChangeTerrainReferenceScale_3d', 'ChangeTerrainResolutionBounds_3d', 'ChangeVersion_management', 'CheckExtension', 'CheckGeometry_management', 'CheckInExtension', 'CheckOutExtension', 'CheckProduct', 'ClassifyLasBuilding_3d', 'ClassifyLasByHeight_3d', 'ClassifyLasGround_3d', 'Clean_arc', 'ClearEnvironment', 'ClearWorkspaceCache_management', 'Clip_analysis', 'Clip_arc', 'Clip_management', 'ClustersOutliersRendered_stats', 'ClustersOutliers_stats', 'CollapseDualLinesToCenterline_arc', 'CollapseDualLinesToCenterline_cartography', 'CollapseRoadDetail_cartography', 'CollectEventsRendered_stats', 'CollectEvents_stats', 'ColorBalanceMosaicDataset_management', 'Command', 'Compact_management', 'CompareReplicaSchema_management', 'CompositeBands_management', 'CompressFileGeodatabaseData_management', 'Compress_management', 'ComputeBlockAdjustment_management', 'ComputeCameraModel_management', 'ComputeControlPoints_management', 'ComputeDirtyArea_management', 'ComputeMosaicCandidates_management', 'ComputePansharpenWeights_management', 'ComputeTiePoints_management', 'ConcatenateDateAndTimeFields_ta', 'ConfigureGeodatabaseLogFileTables_management', 'ConsolidateLayer_management', 'ConsolidateLocator_geocoding', 'ConsolidateLocator_management', 'ConsolidateMap_management', 'ConsolidateResult_management', 'ConstructSightLines_3d', 'ContourAnnotation_cartography', 'ContourList_3d', 'ContourWithBarriers_3d', 'Contour_3d', 'ConvertCoordinateNotation_management', 'ConvertDiagram_schematics', 'ConvertMapServerCacheStorageFormat_server', 'ConvertSpatialWeightsMatrixtoTable_stats', 'ConvertTimeField_management', 'ConvertTimeZone_management', 'CopyFeatures_management', 'CopyParameter', 'CopyParcelFabric_fabric', 'CopyRasterCatalogItems_management', 'CopyRaster_management', 'CopyRows_management', 'CopyRuntimeGdbToFileGdb_conversion', 'CopyTin_3d', 'CopyTraversedSourceFeatures_na', 'Copy_management', 'CountRenderer_stats', 'CreateAddressLocator_geocoding', 'CreateArcInfoWorkspace_management', 'CreateArcSDEConnectionFile_management', 'CreateCartographicPartitions_cartography', 'CreateCompositeAddressLocator_geocoding', 'CreateCustomGeoTransformation_management', 'CreateDatabaseConnection_management', 'CreateDatabaseUser_management', 'CreateDatabaseView_management', 'CreateDiagram_schematics', 'CreateDomain_management', 'CreateEnterpriseGeodatabase_management', 'CreateFeatureDataset_management', 'CreateFeatureclass_management', 'CreateFileGDB_management', 'CreateFishnet_management', 'CreateFolder_management', 'CreateGPSDDraft', 'CreateGeocodeSDDraft', 'CreateGeometricNetwork_management', 'CreateImageSDDraft', 'CreateLabels_arc', 'CreateLasDataset_management', 'CreateMapServerCache_server', 'CreateMapTilePackage_management', 'CreateMosaicDataset_management', 'CreateObject', 'CreateOrthoCorrectedRasterDataset_management', 'CreateOverpass_cartography', 'CreatePansharpenedRasterDataset_management', 'CreatePersonalGDB_management', 'CreateRandomPoints_management', 'CreateRandomRaster_management', 'CreateRandomValueGenerator', 'CreateRasterCatalog_management', 'CreateRasterDataset_management', 'CreateRasterType_management', 'CreateReferencedMosaicDataset_management', 'CreateRelationshipClass_management', 'CreateReplicaFootPrints_management', 'CreateReplicaFromServer_management', 'CreateReplica_management', 'CreateRole_management', 'CreateRoutes_lr', 'CreateRuntimeContent_management', 'CreateSQLiteDatabase_management', 'CreateSchematicFolder_schematics', 'CreateScratchName', 'CreateSpaceTimeCube_stpm', 'CreateSpatialReference_management', 'CreateSpatialType_management', 'CreateSpatiallyBalancedPoints_ga', 'CreateTable_management', 'CreateTerrain_3d', 'CreateThiessenPolygons_analysis', 'CreateTin_3d', 'CreateTopology_management', 'CreateTurnFeatureClass_na', 'CreateUnRegisteredFeatureclass_management', 'CreateUnRegisteredTable_management', 'CreateUnderpass_cartography', 'CreateUniqueName', 'CreateVersion_management', 'CreateVersionedView_management', 'Create_arc', 'CrossValidationResult', 'CrossValidation_ga', 'CulDeSacMasks_cartography', 'Cursor', 'Curvature_3d', 'CutFill_3d', 'DEMToRaster_conversion', 'DLGArc_arc', 'DecimateTinNodes_3d', 'DecryptPYT', 'DefineMosaicDatasetNoData_management', 'DefineOverviews_management', 'DefineProjection_arc', 'DefineProjection_management', 'DeleteCodedValueFromDomain_management', 'DeleteColormap_management', 'DeleteDomain_management', 'DeleteFeatures_management', 'DeleteField_management', 'DeleteGlobeServerCache_server', 'DeleteGridsAndGraticules_cartography', 'DeleteIdentical_management', 'DeleteMapServerCache_server', 'DeleteMosaicDataset_management', 'DeleteRasterAttributeTable_management', 'DeleteRasterCatalogItems_management', 'DeleteRows_management', 'DeleteSchemaGeodatabase_management', 'DeleteTerrainPoints_3d', 'DeleteVersion_management', 'Delete_management', 'DelineateBuiltUpAreas_cartography', 'DelineateTinDataArea_3d', 'DensifySamplingNetwork_ga', 'Densify_edit', 'Describe', 'DetectFeatureChanges_management', 'DetectGraphicConflict_cartography', 'DiagnoseVersionMetadata_management', 'DiagnoseVersionTables_management', 'Dice_management', 'Difference3D_3d', 'DiffusionInterpolationWithBarriers_ga', 'Dimension', 'DirectionalDistribution_stats', 'DirectionalMean_stats', 'Directions_na', 'DisableArchiving_management', 'DisableAttachments_management', 'DisableEditorTracking_management', 'DisconnectUser', 'DisperseMarkers_cartography', 'DissolveNetwork_na', 'DissolveRouteEvents_lr', 'Dissolve_arc', 'Dissolve_management', 'Divide_3d', 'DomainToTable_management', 'DownloadRasters_management', 'DropIndex_arc', 'DropItem_arc', 'DropRepresentation_cartography', 'ESRITranslator_conversion', 'EdgematchFeatures_edit', 'EditRasterFunction_management', 'EditTin_3d', 'EliminatePolygonPart_management', 'Eliminate_arc', 'Eliminate_management', 'EmergingHotSpotAnalysis_stpm', 'EmpiricalBayesianKriging_ga', 'EnableArchiving_management', 'EnableAttachments_management', 'EnableEditorTracking_management', 'EnableEnterpriseGeodatabase_management', 'EncloseMultiPatch_3d', 'EncryptPYT', 'ErasePoint_edit', 'Erase_analysis', 'Erase_arc', 'ExcelToTable_conversion', 'ExecuteError', 'ExecuteWarning', 'Exists', 'ExploratoryRegression_stats', 'ExportAcknowledgementMessage_management', 'ExportCAD_conversion', 'ExportDataChangeMessage_management', 'ExportGeodatabaseConfigurationKeywords_management', 'ExportMapServerCache_server', 'ExportMetadataMultiple_conversion', 'ExportMetadata_conversion', 'ExportMosaicDatasetGeometry_management', 'ExportMosaicDatasetItems_management', 'ExportMosaicDatasetPaths_management', 'ExportRasterCatalogPaths_management', 'ExportRasterWorldFile_management', 'ExportReplicaSchema_management', 'ExportTileCache_management', 'ExportTo3DWebScene_3d', 'ExportTopologyErrors_management', 'ExportWebMap_server', 'ExportXMLWorkspaceDocument_management', 'ExportXYv_stats', 'Export_arc', 'ExtendLine_edit', 'Extent', 'ExtractDataAndEmailTask_server', 'ExtractDataTask_server', 'ExtractData_server', 'ExtractLas_3d', 'ExtractPackage_management', 'ExtractSubDataset_management', 'ExtractValuesToTable_ga', 'ExtrudeBetween_3d', 'FeatureClassToFeatureClass_conversion', 'FeatureClassToGeodatabase_conversion', 'FeatureClassToShapefile_conversion', 'FeatureClassZToASCII_3d', 'FeatureCompare_management', 'FeatureEnvelopeToPolygon_management', 'FeatureOutlineMasks_cartography', 'FeatureSet', 'FeatureTo3DByAttribute_3d', 'FeatureToLine_management', 'FeatureToNetCDF_md', 'FeatureToPoint_management', 'FeatureToPolygon_management', 'FeatureToRaster_conversion', 'FeatureVerticesToPoints_management', 'FeatureclassToCoverage_conversion', 'FeaturesFromCityEngineRules_3d', 'FeaturesToJSON_conversion', 'Field', 'FieldInfo', 'FieldMap', 'FieldMappings', 'FileCompare_management', 'Filter', 'FindClosestFacilities_na', 'FindConflicts_arc', 'FindDisconnectedFeaturesInGeometricNetwork_management', 'FindIdentical_management', 'FindRoutes_na', 'FlipLine_edit', 'Flip_management', 'FloatToRaster_conversion', 'Float_3d', 'Frequency_analysis', 'FromScriptingArcObject', 'FromWKB', 'FromWKT', 'GACalculateZValue_ga', 'GACreateGeostatisticalLayer_ga', 'GAGetModelParameter_ga', 'GALayerToContour_ga', 'GALayerToGrid_ga', 'GALayerToPoints_ga', 'GAMovingWindowKriging_ga', 'GANeighborhoodSelection_ga', 'GASemivariogramSensitivity_ga', 'GASetModelParameter_ga', 'GPXtoFeatures_conversion', 'GaussianGeostatisticalSimulations_ga', 'Generalize_edit', 'GenerateAttachmentMatchTable_management', 'GenerateEdgematchLinks_edit', 'GenerateExcludeArea_management', 'GenerateFgdbLicense_management', 'GenerateLicensedFgdb_management', 'GenerateMapServerCacheTilingScheme_server', 'GenerateNearTable_analysis', 'GenerateNetworkSpatialWeights_stats', 'GenerateOriginDestinationCostMatrix_na', 'GeneratePointCloud_management', 'GeneratePointsAlongLines_management', 'GenerateRasterFromRasterFunction_management', 'GenerateRubbersheetLinks_edit', 'GenerateServiceAreas_na', 'GenerateSpatialWeightsMatrix_stats', 'GenerateTessellation_management', 'GenerateTileCacheTilingScheme_management', 'Generate_arc', 'GeoProcessor', 'GeoTaggedPhotosToPoints_management', 'GeocodeAddresses_geocoding', 'GeodeticDensify_management', 'GeographicallyWeightedRegression_stats', 'Geometry', 'GeostatisticalDatasets', 'GetActivePortalURL', 'GetArgumentCount', 'GetCellValue_management', 'GetCount_management', 'GetIDMessage', 'GetImageEXIFProperties', 'GetInstallInfo', 'GetLayoutTemplatesInfo_server', 'GetLogHistory', 'GetMaxSeverity', 'GetMessage', 'GetMessageCount', 'GetMessages', 'GetPackageInfo', 'GetParameter', 'GetParameterAsText', 'GetParameterCount', 'GetParameterInfo', 'GetParameterValue', 'GetRasterProperties_management', 'GetReturnCode', 'GetSeverity', 'GetSeverityLevel', 'GetSigninToken', 'GetSystemEnvironment', 'GetUTMFromLocation', 'GlobalPolynomialInterpolation_ga', 'Graph', 'GraphTemplate', 'GraphicBuffer_analysis', 'GridIndexFeatures_cartography', 'GroupingAnalysis_stats', 'HighLowClustering_stats', 'HillShade_3d', 'HotSpotsRendered_stats', 'HotSpots_stats', 'IDEdit_arc', 'IDW_ga', 'Identity_analysis', 'Identity_arc', 'Idw_3d', 'Import3DFiles_3d', 'ImportCADAnnotation_conversion', 'ImportCoverageAnnotation_conversion', 'ImportFromE00_conversion', 'ImportGeodatabaseConfigurationKeywords_management', 'ImportMapServerCache_server', 'ImportMessage_management', 'ImportMetadata_conversion', 'ImportMosaicDatasetGeometry_management', 'ImportReplicaSchema_management', 'ImportTileCache_management', 'ImportToolbox', 'ImportXMLWorkspaceDocument_management', 'Import_arc', 'IncreaseMaximumEdges_na', 'IncrementalSpatialAutocorrelation_stats', 'Index', 'IndexItem_arc', 'InsertCursor', 'Inside3D_3d', 'Int_3d', 'Integrate_management', 'InterpolateFromPointCloud_management', 'InterpolatePolyToPatch_3d', 'InterpolateShape_3d', 'Intersect3DLineWithMultiPatch_3d', 'Intersect3DLineWithSurface_3d', 'Intersect3D_3d', 'Intersect_analysis', 'Intersect_arc', 'IntersectingLayersMasks_cartography', 'Intervisibility_3d', 'IsClosed3D_3d', 'IsSynchronous', 'JSONToFeatures_conversion', 'JoinField_management', 'JoinItem_arc', 'KMLToLayer_conversion', 'KernelInterpolationWithBarriers_ga', 'Kriging_3d', 'LASToMultipoint_3d', 'LandXMLToTin_3d', 'LasDatasetStatistics_management', 'LasDatasetToRaster_conversion', 'LasDatasetToTin_3d', 'LasPointStatsAsRaster_management', 'LasPointStatsByArea_3d', 'Layer3DToFeatureClass_3d', 'LayerToKML_conversion', 'LineOfSight_3d', 'ListDataStoreItems', 'ListDatasets', 'ListEnvironments', 'ListFeatureClasses', 'ListFields', 'ListFiles', 'ListIndexes', 'ListInstallations', 'ListPortalURLs', 'ListPrinterNames', 'ListRasters', 'ListSpatialReferences', 'ListTables', 'ListToolboxes', 'ListTools', 'ListTransformations', 'ListUsers', 'ListVersions', 'ListWorkspaces', 'LoadSettings', 'LoadTopologyToParcelFabric_fabric', 'LocalOutlierAnalysis_stpm', 'LocalPolynomialInterpolation_ga', 'LocateFeaturesAlongRoutes_lr', 'LocateLasPointsByProximity_3d', 'LocateOutliers_3d', 'LogUsageMetering', 'Lookup_3d', 'MDPublisher_conversion', 'MXDToWebMap_server', 'MakeClosestFacilityLayer_na', 'MakeFeatureLayer_management', 'MakeGraph_management', 'MakeGridsAndGraticulesLayer_cartography', 'MakeImageServerLayer_management', 'MakeLasDatasetLayer_management', 'MakeLocationAllocationLayer_na', 'MakeMosaicLayer_management', 'MakeNetCDFFeatureLayer_md', 'MakeNetCDFRasterLayer_md', 'MakeNetCDFTableView_md', 'MakeODCostMatrixLayer_na', 'MakeOPeNDAPRasterLayer_md', 'MakeParcelFabricLayer_fabric', 'MakeParcelFabricTableView_fabric', 'MakeQueryLayer_management', 'MakeQueryTable_management', 'MakeRasterCatalogLayer_management', 'MakeRasterLayer_management', 'MakeRouteEventLayer_lr', 'MakeRouteLayer_na', 'MakeServiceAreaLayer_na', 'MakeTableView_management', 'MakeTrackingLayer_ta', 'MakeVehicleRoutingProblemLayer_na', 'MakeWCSLayer_management', 'MakeXYEventLayer_management', 'ManageGlobeServerCacheTiles_server', 'ManageMapServerCacheScales_server', 'ManageMapServerCacheStatus_server', 'ManageMapServerCacheTiles_server', 'ManageTileCache_management', 'MapServerCacheTilingSchemeToPolygons_cartography', 'MapToKML_conversion', 'MatchPhotosToRowsByTime_management', 'MeanCenter_stats', 'MedianCenter_stats', 'MergeDividedRoads_cartography', 'MergeMosaicDatasetItems_management', 'Merge_management', 'MetadataImporter_conversion', 'MigrateRelationshipClass_management', 'MigrateStorage_management', 'MinimumBoundingGeometry_management', 'MinimumBoundingVolume_3d', 'Minus_3d', 'Mirror_management', 'MosaicToNewRaster_management', 'Mosaic_management', 'MultiDistanceSpatialClustering_stats', 'MultiPatchFootprint_3d', 'MultipartToSinglepart_management', 'Multipatch', 'MultipatchToCollada_conversion', 'MultipatchToRaster_conversion', 'MultipleRingBuffer_analysis', 'Multipoint', 'NaturalNeighbor_3d', 'Near3D_3d', 'Near_analysis', 'Near_arc', 'NetCDFFileProperties', 'NumPyArrayToRaster', 'ObserverPoints_3d', 'OptimizedHotSpotAnalysis_stats', 'OptimizedOutlierAnalysis_stats', 'OrdinaryLeastSquares_stats', 'OverlayRouteEvents_lr', 'PDFToTIFF_conversion', 'PackageLayer_management', 'PackageLocator_geocoding', 'PackageLocator_management', 'PackageMap_management', 'PackageResult_management', 'Parameter', 'ParseFieldName', 'ParseTableName', 'PivotTable_management', 'Plus_3d', 'Point', 'PointDistance_analysis', 'PointDistance_arc', 'PointFileInformation_3d', 'PointGeometry', 'PointNode_arc', 'PointToRaster_conversion', 'PointsToLine_management', 'PolyRegion_arc', 'Polygon', 'PolygonNeighbors_analysis', 'PolygonToLine_management', 'PolygonToRaster_conversion', 'PolygonVolume_3d', 'Polyline', 'PolylineToRaster_conversion', 'PopulateAlternateIDFields_na', 'ProductInfo', 'ProjectRaster_management', 'Project_arc', 'Project_management', 'PropagateDisplacement_cartography', 'QuickExport_interop', 'QuickImport_interop', 'RadialBasisFunctions_ga', 'RandomNumberGenerator', 'Raster', 'RasterCatalogToRasterDataset_management', 'RasterCompare_management', 'RasterDomain_3d', 'RasterTin_3d', 'RasterToASCII_conversion', 'RasterToDTED_management', 'RasterToFloat_conversion', 'RasterToGeodatabase_conversion', 'RasterToMultipoint_3d', 'RasterToNetCDF_md', 'RasterToNumPyArray', 'RasterToOtherFormat_conversion', 'RasterToPoint_conversion', 'RasterToPolygon_conversion', 'RasterToPolyline_conversion', 'RasterToVideo_conversion', 'ReExportUnacknowledgedMessages_management', 'RebuildAddressLocator_geocoding', 'RebuildGeometricNetwork_management', 'RebuildIndexes_management', 'RecalculateFeatureClassExtent_management', 'ReclassByASCIIFile_3d', 'ReclassByTable_3d', 'Reclassify_3d', 'ReconcileVersion_management', 'ReconcileVersions_management', 'RecordSet', 'RecoverFileGDB_management', 'RefreshActiveView', 'RefreshCatalog', 'RefreshTOC', 'RegionClass_arc', 'RegionPoly_arc', 'RegisterAsVersioned_management', 'RegisterRaster_management', 'RegisterWithGeodatabase_management', 'RegularizeBuildingFootprint_3d', 'RematchAddresses_geocoding', 'RemoveAttachments_management', 'RemoveConnectivityRuleFromGeometricNetwork_management', 'RemoveDataStoreItem', 'RemoveDomainFromField_management', 'RemoveEmptyFeatureClassFromGeometricNetwork_management', 'RemoveFeatureClassFromTerrain_3d', 'RemoveFeatureClassFromTopology_management', 'RemoveFieldConflictFilter_management', 'RemoveFilesFromLasDataset_management', 'RemoveIndex_management', 'RemoveJoin_management', 'RemoveOverride_cartography', 'RemoveRastersFromMosaicDataset_management', 'RemoveRuleFromTopology_management', 'RemoveSpatialIndex_management', 'RemoveSubtype_management', 'RemoveTerrainPyramidLevel_3d', 'RemoveToolbox', 'Rename_management', 'Renode_arc', 'RepairGeometry_management', 'RepairMosaicDatasetPaths_management', 'RepairRasterCatalogPaths_management', 'RepairVersionMetadata_management', 'RepairVersionTables_management', 'ReplaceTerrainPoints_3d', 'Resample_management', 'Rescale_management', 'Reselect_arc', 'ResetEnvironments', 'ResetProgressor', 'ResolveBuildingConflicts_cartography', 'ResolveRoadConflicts_cartography', 'Result', 'ReverseGeocode_geocoding', 'Rotate_management', 'Row', 'RubbersheetFeatures_edit', 'S57Arc_arc', 'SDTSExport_arc', 'SDTSImport_arc', 'SaveGraph_management', 'SaveSettings', 'SaveToLayerFile_management', 'Schema', 'SearchCursor', 'SearchNeighborhoodSmooth', 'SearchNeighborhoodSmoothCircular', 'SearchNeighborhoodStandard', 'SearchNeighborhoodStandardCircular', 'SelectByDimension_md', 'SelectData_management', 'SelectFeatureByOverride_cartography', 'SelectLayerByAttribute_management', 'SelectLayerByLocation_management', 'Select_analysis', 'SendEmailWithZipFileAttachment_server', 'SetClusterTolerance_management', 'SetDefaultSubtype_management', 'SetFlowDirection_management', 'SetLasClassCodesUsingFeatures_3d', 'SetLayerRepresentation_cartography', 'SetLogHistory', 'SetMosaicDatasetProperties_management', 'SetParameter', 'SetParameterAsText', 'SetProduct', 'SetProgressor', 'SetProgressorLabel', 'SetProgressorPosition', 'SetRasterProperties_management', 'SetRepresentationControlPointAtIntersect_cartography', 'SetRepresentationControlPointByAngle_cartography', 'SetSeverityLevel', 'SetSubtypeField_management', 'SetValueForRangeDomain_management', 'SharePackage_management', 'Shift_management', 'SignInToPortal_server', 'SignOutFromPortal_server', 'SimilaritySearch_stats', 'SimplifyBuilding_arc', 'SimplifyBuilding_cartography', 'SimplifyLineOrPolygon_arc', 'SimplifyLine_cartography', 'SimplifyPolygon_cartography', 'SkylineBarrier_3d', 'SkylineGraph_3d', 'Skyline_3d', 'Slice_3d', 'Slope_3d', 'SmoothLine_cartography', 'SmoothPolygon_cartography', 'Snap_edit', 'SolveLocationAllocation_na', 'SolveVehicleRoutingProblem_na', 'Solve_na', 'SortCodedValueDomain_management', 'Sort_management', 'SpatialAutocorrelation_stats', 'SpatialJoin_analysis', 'SpatialReference', 'SplineWithBarriers_3d', 'Spline_3d', 'SplitByAttributes_analysis', 'SplitLineAtPoint_management', 'SplitLine_management', 'SplitMosaicDatasetItems_management', 'SplitRaster_management', 'Split_analysis', 'Split_arc', 'StackProfile_3d', 'StageService_server', 'StandardDistance_stats', 'StandardizeAddresses_geocoding', 'Statistics_analysis', 'StripMapIndexFeatures_cartography', 'SubsetFeatures_ga', 'SunShadowVolume_3d', 'SurfaceAspect_3d', 'SurfaceContour_3d', 'SurfaceDifference_3d', 'SurfaceSlope_3d', 'SurfaceVolume_3d', 'SymDiff_analysis', 'SynchronizeChanges_management', 'SynchronizeMetadata_conversion', 'SynchronizeMosaicDataset_management', 'TINCompare_management', 'TableCompare_management', 'TableSelect_analysis', 'TableToDBASE_conversion', 'TableToDomain_management', 'TableToEllipse_management', 'TableToExcel_conversion', 'TableToGeodatabase_conversion', 'TableToNetCDF_md', 'TableToRelationshipClass_management', 'TableToTable_conversion', 'TabulateIntersection_analysis', 'TerrainToPoints_3d', 'TerrainToRaster_3d', 'TerrainToTin_3d', 'TestSchemaLock', 'Thiessen_arc', 'ThinRoadNetwork_cartography', 'TigerArc_arc', 'TigerTool_arc', 'TileLas_3d', 'TiledLabelsToAnnotation_cartography', 'Times_3d', 'TinDomain_3d', 'TinEdge_3d', 'TinLine_3d', 'TinNode_3d', 'TinPolygonTag_3d', 'TinRaster_3d', 'TinTriangle_3d', 'Tolerance_arc', 'TopoToRasterByFile_3d', 'TopoToRaster_3d', 'TraceGeometricNetwork_management', 'TrackIntervalsToFeature_ta', 'TrackIntervalsToLine_ta', 'TransferAttributes_edit', 'TransformFeatures_edit', 'TransformRouteEvents_lr', 'Transform_arc', 'TransposeFields_management', 'Trend_3d', 'TrimLine_edit', 'TruncateTable_management', 'TurnTableToTurnFeatureClass_na', 'USGSMPTranslator_conversion', 'UncompressFileGeodatabaseData_management', 'Ungenerate_arc', 'Union3D_3d', 'Union_analysis', 'Union_arc', 'UnregisterAsVersioned_management', 'UnsplitLine_management', 'UpdateAnalysisLayerAttributeParameter_na', 'UpdateAnnotation_management', 'UpdateByAlternateIDFields_na', 'UpdateByGeometry_na', 'UpdateCursor', 'UpdateDiagram_schematics', 'UpdateDiagrams_schematics', 'UpdateEnterpriseGeodatabaseLicense_management', 'UpdateOverride_cartography', 'UpdateTrafficData_na', 'UpdateTrafficIncidents_na', 'Update_analysis', 'Update_arc', 'UpgradeDataset_management', 'UpgradeGDB_management', 'UpgradeMapServerCacheStorageFormat_server', 'UpgradeMetadata_conversion', 'UpgradeNetwork_na', 'UpgradeParcelFabric_fabric', 'UpgradeSpatialReference_management', 'UploadServiceDefinition_server', 'Usage', 'VPFExport_arc', 'VPFImport_arc', 'VPFTile_arc', 'ValidateDataStoreItem', 'ValidateFieldName', 'ValidateMetadataMultiple_conversion', 'ValidateMetadata_conversion', 'ValidateTableName', 'ValidateTopology_management', 'Value', 'ValueTable', 'VerifyAndRepairGeometricNetworkConnectivity_management', 'Viewshed2_3d', 'Viewshed_3d', 'Visibility_3d', 'VisualizeSpaceTimeCube2D_stpm', 'VisualizeSpaceTimeCube3D_stpm', 'WFSToFeatureClass_conversion', 'WarpFromFile_management', 'Warp_management', 'WorkspaceToRasterCatalog_management', 'WorkspaceToRasterDataset_management', 'XMLSchemaValidator_conversion', 'XSLTransform_conversion', 'XYToLine_management', 'ZRenderer_stats', '_NumPyArrayToRaster', '_RasterToNumPyArray', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '_base', '_ga', '_gptooldoc', '_graph', '_management', '_mapping', '_na', 'analysis', 'arc', 'arcobjectconversion', 'arcobjects', 'arcpy', 'cartography', 'conversion', 'convertArcObjectToPythonObject', 'da', 'ddd', 'edit', 'env', 'f', 'fabric', 'ga', 'geocoding', 'geometries', 'geoprocessing', 'glob', 'gp', 'imp', 'import_local', 'interop', 'lr', 'management', 'mapping', 'md', 'mixins', 'na', 'numpy', 'os', 'passthrough_attr', 'sa', 'schematics', 'server', 'stats', 'stpm', 'sys', 'ta', 'time', 'toolbox', 'utils', 'warnings']


For QGIS it is called 'iface', so we do dir(iface).
>>> dir(iface)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'actionAbout', 'actionAddAfsLayer', 'actionAddAllToOverview', 'actionAddAmsLayer', 'actionAddFeature', 'actionAddOgrLayer', 'actionAddPart', 'actionAddPgLayer', 'actionAddRasterLayer', 'actionAddRing', 'actionAddToOverview', 'actionAddWmsLayer', 'actionAllEdits', 'actionCancelAllEdits', 'actionCancelEdits', 'actionCheckQgisVersion', 'actionCopyFeatures', 'actionCopyLayerStyle', 'actionCreatePrintLayout', 'actionCustomProjection', 'actionCutFeatures', 'actionDeletePart', 'actionDeleteRing', 'actionDeleteSelected', 'actionDraw', 'actionDuplicateLayer', 'actionExit', 'actionFeatureAction', 'actionHelpContents', 'actionHideAllLayers', 'actionHideDeselectedLayers', 'actionHideSelectedLayers', 'actionIdentify', 'actionLayerProperties', 'actionLayerSaveAs', 'actionManagePlugins', 'actionMapTips', 'actionMeasure', 'actionMeasureArea', 'actionMoveFeature', 'actionNewBookmark', 'actionNewProject', 'actionNewVectorLayer', 'actionOpenFieldCalculator', 'actionOpenProject', 'actionOpenStatisticalSummary', 'actionOpenTable', 'actionOptions', 'actionPan', 'actionPanToSelected', 'actionPasteFeatures', 'actionPasteLayerStyle', 'actionPluginListSeparator', 'actionProjectProperties', 'actionQgisHomePage', 'actionRemoveAllFromOverview', 'actionRollbackAllEdits', 'actionRollbackEdits', 'actionSaveActiveLayerEdits', 'actionSaveAllEdits', 'actionSaveEdits', 'actionSaveMapAsImage', 'actionSaveProject', 'actionSaveProjectAs', 'actionSelect', 'actionSelectFreehand', 'actionSelectPolygon', 'actionSelectRadius', 'actionSelectRectangle', 'actionShowAllLayers', 'actionShowBookmarks', 'actionShowLayoutManager', 'actionShowPythonDialog', 'actionShowSelectedLayers', 'actionSimplifyFeature', 'actionSplitFeatures', 'actionSplitParts', 'actionToggleEditing', 'actionToggleFullScreen', 'actionVertexTool', 'actionVertexToolActiveLayer', 'actionZoomActualSize', 'actionZoomFullExtent', 'actionZoomIn', 'actionZoomLast', 'actionZoomNext', 'actionZoomOut', 'actionZoomToLayer', 'actionZoomToSelected', 'activeLayer', 'addCustomActionForLayer', 'addCustomActionForLayerType', 'addDatabaseToolBarIcon', 'addDatabaseToolBarWidget', 'addDockWidget', 'addLayerMenu', 'addMeshLayer', 'addPluginToDatabaseMenu', 'addPluginToMenu', 'addPluginToRasterMenu', 'addPluginToVectorMenu', 'addPluginToWebMenu', 'addProject', 'addRasterLayer', 'addRasterToolBarIcon', 'addRasterToolBarWidget', 'addToolBar', 'addToolBarIcon', 'addToolBarWidget', 'addUserInputWidget', 'addVectorLayer', 'addVectorToolBarIcon', 'addVectorToolBarWidget', 'addWebToolBarIcon', 'addWebToolBarWidget', 'addWindow', 'advancedDigitizeToolBar', 'askForDatumTransform', 'attributesToolBar', 'blockSignals', 'browserModel', 'buildStyleSheet', 'cadDockWidget', 'childEvent', 'children', 'closeMapCanvas', 'connectNotify', 'copySelectionToClipboard', 'createNewMapCanvas', 'currentLayerChanged', 'currentThemeChanged', 'customEvent', 'dataSourceManagerToolBar', 'databaseMenu', 'databaseToolBar', 'defaultStyleSheetFont', 'defaultStyleSheetOptions', 'deleteLater', 'deregisterLocatorFilter', 'destroyed', 'digitizeToolBar', 'disconnect', 'disconnectNotify', 'dumpObjectInfo', 'dumpObjectTree', 'dynamicPropertyNames', 'editMenu', 'editableLayers', 'event', 'eventFilter', 'fileToolBar', 'findChild', 'findChildren', 'firstRightStandardMenu', 'getFeatureForm', 'helpMenu', 'helpToolBar', 'iconSize', 'inherits', 'initializationCompleted', 'insertAddLayerAction', 'installEventFilter', 'invalidateLocatorResults', 'isSignalConnected', 'isWidgetType', 'isWindowType', 'killTimer', 'layerMenu', 'layerSavedAs', 'layerToolBar', 'layerTreeCanvasBridge', 'layerTreeView', 'layoutDesignerClosed', 'layoutDesignerOpened', 'layoutDesignerWillBeClosed', 'mainWindow', 'mapCanvas', 'mapCanvases', 'mapNavToolToolBar', 'messageBar', 'messageTimeout', 'metaObject', 'moveToThread', 'newLayerMenu', 'newProject', 'newProjectCreated', 'objectName', 'objectNameChanged', 'openFeatureForm', 'openLayoutDesigner', 'openLayoutDesigners', 'openMessageLog', 'openURL', 'parent', 'pasteFromClipboard', 'pluginManagerInterface', 'pluginMenu', 'pluginToolBar', 'preloadForm', 'projectMenu', 'projectRead', 'property', 'pyqtConfigure', 'rasterMenu', 'rasterToolBar', 'receivers', 'registerCustomDropHandler', 'registerCustomLayoutDropHandler', 'registerLocatorFilter', 'registerMainWindowAction', 'registerMapLayerConfigWidgetFactory', 'registerOptionsWidgetFactory', 'reloadConnections', 'removeAddLayerAction', 'removeCustomActionForLayerType', 'removeDatabaseToolBarIcon', 'removeDockWidget', 'removeEventFilter', 'removePluginDatabaseMenu', 'removePluginMenu', 'removePluginRasterMenu', 'removePluginVectorMenu', 'removePluginWebMenu', 'removeRasterToolBarIcon', 'removeToolBarIcon', 'removeVectorToolBarIcon', 'removeWebToolBarIcon', 'removeWindow', 'saveStyleSheetOptions', 'sender', 'senderSignalIndex', 'setActiveLayer', 'setObjectName', 'setParent', 'setProperty', 'settingsMenu', 'shapeDigitizeToolBar', 'showAttributeTable', 'showLayerProperties', 'showLayoutManager', 'showOptionsDialog', 'signalsBlocked', 'startTimer', 'staticMetaObject', 'statusBarIface', 'takeAppScreenShots', 'thread', 'timerEvent', 'tr', 'unregisterCustomDropHandler', 'unregisterCustomLayoutDropHandler', 'unregisterMainWindowAction', 'unregisterMapLayerConfigWidgetFactory', 'unregisterOptionsWidgetFactory', 'vectorLayerTools', 'vectorMenu', 'vectorToolBar', 'viewMenu', 'webMenu', 'webToolBar', 'windowMenu', 'zoomFull', 'zoomToActiveLayer', 'zoomToNext', 'zoomToPrevious']


4) Get details from document/project file

ArcGIS document/project file is save with the extension .mxd, while QGIS extension is .qgz or .qgs. And the can be manipulated via their respective APIs as seen below:-

ArcGIS: 
# Create project instance...
prjt = arcpy.mapping.MapDocument('current')

# project file path and name..
prjt.filePath

# project title
prjt.title

# project date saved
prjt.dateSaved

# project author
prjt.author

# project layer count
arcpy.mapping.ListLayers(prjt)

# project coordinate reference system
prjt.activeDataFrame.mapUnits

# set project working space/folder
arcpy.env.workspace

# list shp in working space/folder
arcpy.ListFeatureClasses('*.shp')

QGIS:
# Create project instance...
prjt = QgsProject.instance()

# project name..
prjt.fileName() 

# project folder
prjt.homePath()

# project layer count
prjt.count()

# project coordinate reference system
prjt.crs()
prjt.crs().geographicCrsAuthId()
prjt.crs().authid()

# project bounds
prjt.crs().bounds().asWktCoordinates()

Friday, July 15, 2022

How to make map of study area or area of interest

 Most often it is a good idea to show your readers where your project/research site is located. This is often called map of "study area" or "area of study" or "area of interest".

Whatever it is called, it is that map that usually comes in the introductory part of your report that introduces where your research area is geographically located.

Let's see how we can make one.

Assuming we are conducting a research work in a particular LGA within Niger state of Nigeria, then we need to show our readers where the study area is using a map. The general process will involve the following steps:-

  1.  Source for GIS data layer
  2.  Filter data layer using "Select by location"
  3.  Cartographic styling
  4.  Export map for use in written report
Lets take a look at each step in details.

Tuesday, July 5, 2022

How to execute a string of code as a code in Python

 Lets say you got a string representation of a python syntax that you actually want to execute/run not just display it on the console, how do you execute/run such a string code?

print( "print('Hello World')" )

Above code will print print('Hello World') instead of 'Hello World'. Also the code below will display len('Hello World') instead length of the characters as 11.

print( "len('Hello World')" )

There are two function that can help you get the task done. They are the exec and the eval functions. See difference between exec and eval here


# Using exec function
exec( "print('Hello World')" )
exec( '''print( "print('Hello World')" )''' )

# Using eval function
eval( "print('Hello World')" )
eval( '''print( "print('Hello World')" )''' )


print( "len('Hello World')" ) # Using exec function exec( "len('Hello World')" ) exec( '''len( "print('Hello World')" )''' ) # Using eval function eval( "len('Hello World')" ) eval( '''len( "print('Hello World')" )''' )

I found this very useful when I want to evaluate a dynamically generated Python expression.
Happy coding.

Friday, July 1, 2022

Python GIS Data Wrangling - U.S. Drought Monitor

 This post was inspired by John Nelson YouTube video on "How to Make This Drought Map Pt 1: DATA WRANGLING", where he manually wrangled the dataset for the year 2018.

What he did was great if you are just doing it for a single year. If you intend to repeat the workflow for several years, then the process can be time consuming and prone to mistakes. For this reason, I will recreate the workflow using python scripting and the whole process can be automated with few button clicks.


More specifically, I will cover the following processes:-

  1. Download and extract the zip folder
  2. Combine the shapefiles into a single folder
  3. Merge the shapefiles into shapefile


Lets get started.

Sunday, June 26, 2022

QGIS Select by attribute - specific character or text string

 Lets say you want to perfect attribute selection where the entry contains a specific word or letter, this can get complicated if the word is not alone in the table cell.

There are couple of ways to solve this kind of query. The most advance way is using regular expression, I have written about regular expression on the page titled: Working with Regular Expression in QGIS

An easier approach would be to use the LIKE operator together with the % wildcard symbol. For example, lets say you want to search for ward names that contain the letter "o", the expression will look like this:-

"ward_name" LIKE '%o%'

You just need to add the text string you want to search for in between %...% as seen below:-

"ward_name" LIKE '%Add Something To Search%'

That is it!

Saturday, June 25, 2022

Useful code snippets to Clip polygon with another polygon and Convert Vector file in Geopackage to Shapefile in QGIS

 Clip polygon with another polygon using processing algorithm in QGIS

# Clip polygon with another polygon using processing algorithm in QGIS

ovly = 'C:\\Users\\Yusuf_08039508010\\Desktop\\Working_Files\\...\\Name_Bouganville Lower - N.gpkg'

outfile = ovly.split('\\')[-1].replace('Name_', '').replace('.gpkg', '')
print('Processing...', outfile)

parameters = {'INPUT':'C:\\Users\\Yusuf_08039508010\\Desktop\\Working_Files\\...\\Results\\SHP\\Woody_02.shp',
'OVERLAY':ovly,
'OUTPUT':f'C:/Users/Yusuf_08039508010/Desktop/Working_Files/.../Results/SHP/Study Location Individual Files/SHP with Holes/{outfile}.shp'}

processing.run("native:clip", parameters)



# -----------------------------------------------
# Bulk Clip polygon with another polygon using processing algorithm in QGIS

import glob

overlay_file = glob.glob('C:\\Users\\Yusuf_08039508010\\Desktop\\Working_Files\\Fiverr\\2022\\06-June\\Masking None Vegetation Ground Covers\\Results\\SHP\\Study Location Individual Files\\*.gpkg')

failed = []
for ovly in overlay_file:
    try:
        outfile = ovly.split('\\')[-1].replace('Name_', '').replace('.gpkg', '')
        print('Processing...', outfile)

        parameters = {'INPUT':'C:\\Users\\Yusuf_08039508010\\Desktop\\Working_Files\\Fiverr\\2022\\06-June\\Masking None Vegetation Ground Covers\\Results\\SHP\\Woody_02.shp',
        'OVERLAY':ovly,
        'OUTPUT':f'C:/Users/Yusuf_08039508010/Desktop/Working_Files/Fiverr/2022/06-June/Masking None Vegetation Ground Covers/Results/SHP/Study Location Individual Files/SHP with Holes/{outfile}.shp'}

        processing.run("native:clip", parameters)
    except Exception:
        failed.append(ovly)

print('Done...')



Convert Vector file in Geopackage to Shapefile

# Convert Vector file in Geopackage to Shapefile


gpkg = 'C:\\Users\\Yusuf_08039508010\\Desktop\\Working_Files\\...\\Name_Bouganville Lower - N.gpkg'

# Read vector layer and test it is valid....
input_file = QgsVectorLayer(gpkg, "polygon", "ogr")
input_file.isValid()

# Construct output file name...
out_filename = gpkg.split('\\')[-1].replace('.gpkg', '')

# Write the file to disc...
out_folder = r"C:\Users\Yusuf_08039508010\Desktop\Working_Files\...\Results\SHP\Study Location Individual Files\Failed files"
QgsVectorFileWriter.writeAsVectorFormat(input_file, f"{out_folder}\\{out_filename}.shp", "UTF-8", input_file.crs(), "ESRI Shapefile")
print('Done...') # ----------------------------------------------------------- # Bulk Convert Vector file in Geopackage to Shapefile import glob gpkg_folder = r"C:\Users\Yusuf_08039508010\Desktop\Working_Files\Fiverr\2022\06-June\Masking None Vegetation Ground Covers\Results\Deliverables\Shp\gpk" gpkg_files = glob.glob(f'{gpkg_folder}\\*.gpkg') for gpkg in gpkg_files: # Read vector layer and test it is valid.... input_file = QgsVectorLayer(gpkg, "polygon", "ogr") input_file.isValid() # Construct output file name... out_filename = gpkg.split('\\')[-1].split('_')[-1].replace('.gpkg', '') # Write the file to disc... out_folder = r"C:\Users\Yusuf_08039508010\Desktop\Working_Files\Fiverr\2022\06-June\Masking None Vegetation Ground Covers\Results\Deliverables\Shp" QgsVectorFileWriter.writeAsVectorFormat(input_file, f"{out_folder}\\{out_filename}.shp", "UTF-8", input_file.crs(), "ESRI Shapefile") print('Done...')



Different ways to read map layers into pyqgis

#1 - Read active layer from the layer panel
layer = iface.activeLayer() # or
layer = qgis.utils.iface.activeLayer()

  
#2 - Read all layers listed on the layers panel...
layers_on_panel = QgsProject.instance().mapLayers()


#3 - Read layer by name from the layer panel...
layer = QgsProject.instance().mapLayersByName('Churches')


#4 - Read from file...
vector_file = r"C:\Users\Yusuf_08039508010\Desktop\...\NGA_adm2.shp"
layer = QgsVectorLayer(vector_file, 'DISPLAYNAME', 'ogr')



Save all shapefile layer listed on the layer panel into a new directory/folder

# Save layers on layer panel to new folder...
import shutil

newFolder = r'C:\Users\Yusuf_08039508010\Desktop\Map of StudyArea\BMC SHP'

# Read all layers listed on the layers panel...
layers_on_panel = QgsProject.instance().mapLayers()

# Loop over each file and copy it to new folder location...
for k, v in layers_on_panel.items():
    # Construct file name...
    fn = v.name() + '.shp'
    
    # Get file path...
    fp1 = v.source()
    fp2 = v.source().replace('.shp', '') + '.dbf'
    fp3 = v.source().replace('.shp', '') + '.shx'
    fp4 = v.source().replace('.shp', '') + '.prj'
    fp5 = v.source().replace('.shp', '') + '.prj'
    
    print('Copying....', fn)
    print('')
    
    # Copy the shapefiles to new folder location...
    shutil.copy( fp1, newFolder ) # shutil.copy( src, dst )
    shutil.copy( fp2, newFolder )
    shutil.copy( fp3, newFolder )
    shutil.copy( fp4, newFolder )
    shutil.copy( fp5, newFolder )
    
    # break

print('Done...')


Friday, June 24, 2022

Extracting 'Standardized Precipitation-Evapotranspiration Index (SPEI)' data from netCDF file into CSV spreadsheet

 In this post, we shall learn how to use python to extract netCDF data into CSV.


What is NetCDF?

NetCDF stands for "network Common Data Form" and it is a file format for storing multidimensional scientific data (variables) such as temperature, humidity, pressure, wind speed, precipitation and direction. There are several versions of netCDF and the version we are using here is version 4 (netCDF4).


The Dataset

The dataset am going to use is the Global 01-month 1901-2020 'Standardized Precipitation-Evapotranspiration Index (SPEI )' provided by Digital.csis



Read more details on this dataset on: SPEI Global Drought Monitor Database

How to open netCDF dataset

The netCDF data can be visualized using QGIS software. This will give us a quick overview as to what variables and attributes are contained in the file.


Some GIS packages allow reading netCDF data, such as QGIS, ArcGIS and IDRISI Taiga. There are other netCDF viewers, such as Panoply (developed in Java), ncBrowse, ncview, and nCDF_Browser.

Tuesday, May 17, 2022

Arcpy and python scripts for Arcmap desktop GIS software

 Listed below are some scripts in python/acrpy for performing various useful tasks in Arcmap/ArcGIS desktop software.


Selected and save shapefile based on attribute

# Code to save selected shp based on attribute...

import arcpy

# Path to input shp...
nig_shp = r"C:\Users\Yusuf_08039508010\Desktop\...\Data\New Data from Scratch\data\NIG_ADM.shp"

# Path to output shp...
out_put_path = r"C:\Users\Yusuf_08039508010\Desktop\...\Data\New Data from Scratch\data\out\nez_3.shp"

# Select and save polygons where "Weight"=5...
arcpy.Select_analysis(nig_shp, out_put_path, '"Weight"=5')

# Select and save polygons where "geographic"='NEZ'...
arcpy.Select_analysis(nig_shp, out_put_path, '"geographic"=\'NEZ\'')


Create new attribute field in bulk

# Create new attribute field

import arcpy


shp = r"C:\Users\Yusuf_08039508010\Desktop\...\Data\New Data from Scratch\data\NIG_ADM.shp"

field_name = 'Yr_2025'
field_type = 'Integer'

arcpy.AddField_management(shp, field_name, field_type)

print('New field has been added...')



#Exercise: Modify the arcpy code snippet above to create attribute fields from years from 1980 to 2020.

# Solution....
for year in range(1980, 2021):
        print('Processing....', year)

        field_name = 'Yr_{}'.format(year)
        field_type = 'Integer'

        arcpy.AddField_management(shp, field_name, field_type)
# Another version of bulk creation of attribute fields...
import arcpy

for x in range(1980, 1990):
    print('Creating field...', x)

    field_name = 'AA_{}'.format(x)
    field_type = 'Integer'

    arcpy.AddField_management('NIG_ADM', field_name, field_type)


Buffer at multiple distances

# Buffer at multiple distance

import arcpy

my_buffer_dist = [5000, 8000, 11000, 14000, 17000, 20000, 23000, 26000, 29000, 32000]

for distance in my_buffer_dist:
    print('Processing...', distance)

    out_file_name = "Buffer_{}m".format(distance)

    arcpy.Buffer_analysis("Lagos_to_Kano", out_file_name, distance)


Select features from provided list

import arcpy
from arcpy import env


# Get number of features in shp
feature_count = int( arcpy.GetCount_management(nig_shp).getOutput(0) )

# Define custome workspace directory...
env.workspace

states = ['Abia', 'Adamawa', 'Akwa Ibom', 'Anambra', 'Bauchi', 'Bayelsa', 'Benue', 'Borno', 'Cross River', 'Delta', 'Ebonyi', 'Edo', 'Ekiti', 'Enugu', 'Abuja', 'Gombe', 'Imo', 'Jigawa', 'Kaduna', 'Kano', 'Katsina', 'Kebbi', 'Kogi', 'Kwara', 'Lagos', 'Nasarawa', 'Niger', 'Ogun', 'Ondo', 'Osun', 'Oyo', 'Plateau', 'Rivers', 'Sokoto', 'Taraba', 'Yobe', 'Zamfara']

out_folder = r"C:\Users\Yusuf_08039508010\Desktop\...\Data\New Data from Scratch\data\out"

for f in states:
    print('Processing object...', f)
    out_put_path = out_folder + '\{}.shp'.format(f)
    arcpy.Select_analysis(nig_shp, out_put_path, '"state_name"=\'{}.format(f)\'')


Label based on column with varying font size and color

# Label based on column with varying font size and color (to be used in label EXPRESSION)
def FindLabel ( [state_name] ): len_of_name = len( [state_name] ) if len_of_name < 6: return " <CLR red='255' green='0' blue='0'> <FNT size = '{}'> {} </FNT> </CLR> ".format( len_of_name, [state_name] ) elif len_of_name >= 6: return " <CLR red='0' green='0' blue='255'> <FNT size = '{}'> {} </FNT> </CLR> ".format( len_of_name, [state_name] )


Multiline label with varying font size

# Multiline label with varying font size (to be used in label EXPRESSION)

def FindLabel ( [state_name], [Weight] ):
  return [state_name] +  "\n"  +  "<FNT size = '14'> {} </FNT>".format([Weight])



Return random value from a list into cell of an attribute field
# Return random value from a list (to be used in field Calculator)

import random
meters_per_hr = [30000, 35000, 40000]

def random_km():
    return random.choice(meters_per_hr)




Hope you find it useful for you next project.

Monday, January 31, 2022

LeafletJS Vs Python Folium Web map

 In this post, I will show how various components are made in both LeafletJS and Python Folium.


First map: Initialize a map with center, zoom and openstreetmap background



LeafletJS
<!DOCTYPE html>
<html>
<head>
	<script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.0.0-beta.2.rc.2/leaflet.js"></script>
	<link href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.0.0-beta.2.rc.2/leaflet.css" rel="stylesheet" />
	<script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet.draw/0.2.3/leaflet.draw.js"></script>

	<link href="https://cdnjs.cloudflare.com/ajax/libs/leaflet.draw/0.2.3/leaflet.draw.css" rel="stylesheet" />

	<meta charset="utf-8">
	<title>Web map....</title>
</head>

<style type="text/css">
	html, body, #map { margin: 0; height: 100%; width: 100%; }
</style>


<body>



  <div id='map'></div>



  <script>
  	// center of the map
	var center = [8.242, 7.671];

	// Create the map
	var map = L.map('map').setView(center, 7);

	// Set up the OSM layer
	L.tileLayer(
	  'https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
	    attribution: 'Data © <a href="http://osm.org/copyright">OpenStreetMap</a>',
	    maxZoom: 18
	  }).addTo(map);





  </script>

</body>
</html>


Folium

import folium

# initialize a map with center, zoom and openstreetmap background...
mapObj = folium.Map(location=[8.242, 7.671],
                     zoom_start=7, tiles='openstreetmap')


mapObj






Draw point



LeafletJS
l


Folium

f






point

Draw line



LeafletJS
l


Folium

f






point

Draw polygon



LeafletJS
l


Folium

f






point

Plot geojson data



LeafletJS
l


Folium

f






point

Add layer control



LeafletJS
l


Folium

f






point

Add HTML



LeafletJS
l


Folium

f









Tuesday, January 18, 2022

Geopandas Vs Folium - Generate Web map from data

 The code snippet below will demonstrate how to create an interactive choropleth web map using Geopandas and Folium libraries.


The latest version of Geopandas has the explore() method which can create a leafletjs map as seen above. 


import geopandas as gpd

# Read shp...
gdf = gpd.read_file(r"NGA_adm1.shp")

# Create web map obj...
mymap = gdf.explore(column='geographic')

# Save to file...
mymap.save('map.html')







import folium
import geopandas as gpd

zones = {'NEZ':1, 'SEZ':2, 'SSZ':3, 'SWZ':4, 'NCZ':5, 'NWZ':6}

# Read shp...
gdf = gpd.read_file(r"NGA_adm1.shp")

gdf.reset_index(level=0, inplace=True)
gdf['Weight'] = gdf['geographic'].map(zones)
gdf['index'] = gdf['index'].apply( lambda x: str(x) )

# Create folium map obj...
mymap = folium.Map(location=[8.67, 7.22], zoom_start=6)

folium.Choropleth(
    geo_data=geo_json_str, 
    data=gdf,
    name = 'Choropleth Map',
    columns = ['index','Weight', 'state_name'],
    key_on = 'feature.id',
    fill_color = 'YlGnBu', # RdYlGn
    legend_name = 'Name of Legend...',
    smooth_factor=  0
    
    ).add_to(mymap)

mymap