Python

Historically, the open-source spatial analysis workflows lived in different silos Python: You use Pandas, GeoPandas, and Jupyter Notebooks. SQL: You use SQL with PostGIS. If you lived in the Python ecosystem, you rarely have to switch to SQL for analysis and vice versa. Many people, including me, had little motivation to sharpen their SQL skills when you could get away with doing things in Python. Many of our students who wanted to learn SQL, found themselves choosing between these two stacks and found SQL had a much higher friction to get started. Recently, I have been using DuckDB and find that it is the perfect bridge between these two ecosystems. DuckDB + Python + LLMs provide the easiest and most rewarding pathway for Python users to learn and incorporate SQL in their workflows. In this post, we will cover the following topics What is DuckDB? Using DuckDB to learn SQL with the help of LLMs Example Workflow with LLMs Querying and Loading Administrative Boundaries from GeoBoundaries Advanced Workflows Extracting Overture Maps Data Extracting Farm Boundaries from Global Fields of The World (FTW) Open our companion notebook learning_sql_duckdb.ipynb in Google Colab to follow along and run the queries yourself. ...

Welcome to #PythonDatavizChallenge - Learn Mapping and Data Visualization with Python in 30 Days! We have designed this challenge to help you learn how to create charts, maps, animations, dashboards and interactive mapping applications using Python ! Spend 30 minutes each day for the next 30 days to level-up your Python dataviz skills. We have spent over 2 years building and refining this course and are excited to share it with you all - completely free. The challenge is a series of short videos, one set for each day, that cover the full course material step by step. The material covers both static and dynamic plotting libraries along with the app framework - Streamlit. At the end of the course, you will have the necessary skills to build data-powered web mapping apps and dashboards. Ready for #PythonDatavizChallenge? Read on to know the details. This is an intermediate course that assumes good working knowledge of Python. If you are new to programming, complete our Python Foundation for Spatial Analysis course first. ...

ISRO recently released the full archive of medium and low-resolution Earth Observation dataset to the public. This includes the imagery from LISS-IV camera aboard ResourceSat-2 and ResourceSat-2A satellites. This is currently the highest spatial resolution imagery available in the public domain for India. In this post, I want to cover the steps required to download the imagery and apply the pre-processing steps required to make this data ready for analysis - specifically how to programmatically convert the DN values to TOA Reflectance. We will use modern Python libraries such as XArray, rioxarray, and dask - which allow use to seamlessly work with large datasets and use all the available compute power on your machine. ...

An important concept in spatial statistics is pixel weights. When calculating pixel statistics with a polygon, partial pixel overlaps are treated differently by different packages and you need to understand this to evaluate the accuracy of your results. Consider the following image. What is the correct answer? ...

Matplotlib has functionality to created animations and can be used to create dynamic visualizations. In this post, I will explain the concepts and techniques for creating animated charts using Python and Matplotlib. I find this technique very helpful in creating animations showing how certain algorithms work. This post also contains Python implementations of two common geometry simplification algorithms and they will used to create animations showing each step of the algorithm. Since both of these implementations use a recursive function, the technique shown in the post can be extended to visualize other recursive functions using matplotlib. You will learn how to create animated plots like below. ...

Spatial indexing methods help speed up spatial queries. Most GIS software and databases provide a mechanism to compute and use spatial index for your data layers. QGIS as well as PostGIS use a spatial indexing scheme based on R-Tree data structure - which creates a hierarchical tree using bounding boxes of geometries. This is quite efficient and results in big speedup in certain types of spatial queries. Check out Spatial Indexing section of my course Advanced QGIS where I show how to use R-Tree based Spatial index in QGIS. If you use Python for geoprocesisng, the GeoPandas library also provides an easy to use implementation of R-Tree based spatial index using the .sidex attribute. University of Helsinki’s AutoGIS course has an excellent example of using spatial index with geopandas. In this post, I want to talk about another spatial indexing system called H3. ...

When working with raster data, you may sometimes need to deal with data gaps. These could be the result of sensor malfunction, processing errors or data corruption. Below is an example of data gap (i.e. no data values) in aerial imagery. Source Image: © Commission for Lands (COLA) ; Revolutionary Government of Zanzibar (RGoZ), Downloaded from OpenAerialMap. (Note: The data gap is simulated using a python script and is not part of the original dataset) ...