Saline System Visualizer

This project is the final assignment for the Python Foundation for Spatial Analysis course offered by Spatial Thoughts (https://spatialthoughts.com/) and guided by Ujaval Gandhi and Vigna Purohit.

The National Geology and Mining Service (SERNAGEOMIN) of Chile maintains a repository of publicly accessible databases on their website: https://repositorio.sernageomin.cl/home

In 2023, SERNAGEOMIN released a database on saline systems titled "Composición química y mineralogía de los salares de Pajonales, La Isla, Las Parinas y Aguilar, regiones de Antofagasta y Atacama, Chile" (Chemical Composition and Mineralogy of the Salt Flats of Pajonales, La Isla, Las Parinas, and Aguilar, Regions of Antofagasta and Atacama, Chile). This database contains detailed information on four saline systems.

The data is provided in XLSX format and is organized into five sheets:

Crust Mineralogy
Crust Chemistry
Water Chemistry
Isotopic Data
Sample Locations

This project utilizes the published data to create an interactive HTML map. The map displays the locations of the samples, and when a sample is clicked, a pop-up window appears with the following information:

Classification of the most abundant minerals
Sample IDs
The three most abundant minerals and their classifications
A pie chart showing the proportion of the main anions of the saline crust: carbonate, sulfate, and chloride.

The dataset includes values below the detection limit and missing carbonate data for certain samples, particularly from the Las Parinas and Pajonales salt flats, which were replaced with 0 to facilitate calculations. This approach, while necessary, introduces artifacts that must be accounted for in subsequent analysis.

The database file can be downloaded from: https://repositorio.sernageomin.cl/items/fe59c11b-c077-4fd7-821a-e03cc19972ab

Import necesary libraries

In [1]:

import os
import base64
import folium
import pandas as pd
from io import BytesIO
import geopandas as gpd
import matplotlib.pyplot as plt

Load the file and create dataframes with each sheet to be used

In [2]:

# Get path of the file 
file_dir = 'data'
file_name = 'BD_GQ_13_Composicion_quimica_y_mineralogia.xlsx'
file_path = os.path.join(file_dir, file_name)

# Load the mineralogy sheet into DataFrame
sheet_name = 'MINERALOGÍA COSTRA'
database = gpd.read_file(file_path, layer=sheet_name).reset_index()
database.columns = database.iloc[2]
data_min = database.drop(database.index[range(0,3)]).reset_index()
data_min = data_min[['Código de muestra', 'Salar', 'Mineral principal 1', 'Mineral principal 2', 'Mineral principal 3']]

#Load the information sheet
info = gpd.read_file(file_path, layer='CMS')
info.columns = info.iloc[2]
data_info = info.drop(info.index[range(0, 3)])
data_info.rename(columns={
    'UTM\nnorte': 'UTM norte',
    'UTM\neste': 'UTM este'
}, inplace=True)
data_info.columns

# Load the chemistry sheet into DataFrame
sheet_name = 'QUÍMICA COSTRA'
database = gpd.read_file(file_path, layer=sheet_name)
database.iloc[3,0] = database.iloc[2,0]
database.iloc[3,1] = database.iloc[2,1]
database.columns = database.iloc[3]
data_chem = database.drop(database.index[range(0, 4)]).reset_index()

# Clean elements names
data_chem.columns = data_chem.columns.str.replace('(%)', '')
data_chem.columns = data_chem.columns.str.replace('(ppm)', '')
data_chem.columns = data_chem.columns.str.replace('-', '')
data_chem.columns = data_chem.columns.str.replace('2', '')
data_chem.columns = data_chem.columns.str.strip()

Merge the three dataframes on the sample ID ('Código de muestra')

In [3]:

# Merge mineralogy sheet with the location sheet
merged_data = data_min.merge(data_info[['Código de muestra', 'UTM norte', 'UTM este', 'Proyección']], 
                         on='Código de muestra', 
                         how='left',
                         suffixes=('', '_info'))

# Get resulting dataframe with the chemical sheet
sulfate = 'SO4'
chloride = 'Cl'
carbonate = 'CO3'

df= merged_data.merge(data_chem[['Código de muestra', sulfate, chloride, carbonate]],
                     on='Código de muestra',
                     how='left',
                     suffixes=('', '_chem'))

Values below the detection limit were identified and replaced with 0 to ensure consistency in the calculations. Additionally, samples from Pajonales and Las Parinas lack carbonate data. In such cases, missing values were also replaced with 0. This step is necessary to calculate the percentages of each mineral accurately. However, it introduces an artifact in the data, as the absence of carbonate values does not necessarily imply a concentration of 0. This artifact must be carefully considered in further analysis to avoid misinterpretation of results.

In [4]:

# Function to verify if the value es under detection limit
def verify_under_limit(value):
    if isinstance(value, str) and (value.startswith('<') or value.startswith('-')):
        return 0  
    try:
        return float(value)
    except (ValueError, TypeError):
        return value  

# Apply function
df[sulfate] = df[sulfate].apply(verify_under_limit)
df[chloride] = df[chloride].apply(verify_under_limit)
df[carbonate] = df[carbonate].apply(verify_under_limit)

Change Coordinate Reference System into World Geodetic System 1984

In [5]:

# Get geometry points from column values
data = gpd.GeoDataFrame(df, geometry=gpd.points_from_xy(df['UTM este'], df['UTM norte']), crs='EPSG:32719')

# Change coordinate system
data = data.to_crs('EPSG:4326')

Clean spaces from mineral names

In [6]:

# Clean strings that end with a space character
def clean_string(value):
    if isinstance(value, str) and value.endswith(' '):
        return value[:-1]
    else:
        return value 

# Apply clean_string function to minerals column
data['Mineral principal 1'] = data['Mineral principal 1'].apply(clean_string)
data['Mineral principal 2'] = data['Mineral principal 2'].apply(clean_string)
data['Mineral principal 3'] = data['Mineral principal 3'].apply(clean_string)

List of mineral classifications.

This list was generated with the assistance of DeepSeek, an AI tool. Caution is adviced.

In [7]:

mineral_classification = {
    # Sulfates
    'anhidrita': {
        'classification': 'calcium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'anhydrite'
    },
    'yeso': {
        'classification': 'hydrated calcium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'gypsum'
    },
    'yeso (trazas)': {
        'classification': 'hydrated calcium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'gypsum (trace)'
    },
    'yeso deutérico': {
        'classification': 'hydrated calcium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'deuteric gypsum'
    },
    'bloedita': {
        'classification': 'sodium magnesium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'bloedite'
    },
    'blöedita': {
        'classification': 'sodium magnesium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'blödite'
    },
    'epsonita': {
        'classification': 'hydrated magnesium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'epsomite'
    },
    'glauberita': {
        'classification': 'sodium calcium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'glauberite'
    },
    'mirabilita': {
        'classification': 'hydrated sodium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'mirabilite'
    },
    'thenardita': {
        'classification': 'sodium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'thenardite'
    },
    'polihalita': {
        'classification': 'potassium magnesium calcium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'polyhalite'
    },
    'alumbre de potasio': {
        'classification': 'potassium aluminum sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'potassium alum'
    },
    'amonioalunita': {
        'classification': 'ammonium aluminum sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'ammonium alunite'
    },
    'picromerita': {
        'classification': 'potassium magnesium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'picromerite'
    },
    'singenita': {
        'classification': 'potassium magnesium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'syngenite'
    },
    'langbeinita': {
        'classification': 'potassium magnesium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'langbeinite'
    },

    # Carbonates
    'calcita': {
        'classification': 'calcium carbonate',
        'simple_classification': 'carbonate',
        'mineral_eng': 'calcite'
    },
    'calcita magnésica': {
        'classification': 'calcium magnesium carbonate',
        'simple_classification': 'carbonate',
        'mineral_eng': 'magnesian calcite'
    },
    'aragonito': {
        'classification': 'calcium carbonate',
        'simple_classification': 'carbonate',
        'mineral_eng': 'aragonite'
    },
    'dolomita': {
        'classification': 'calcium magnesium carbonate',
        'simple_classification': 'carbonate',
        'mineral_eng': 'dolomite'
    },
    'magnesita': {
        'classification': 'magnesium carbonate',
        'simple_classification': 'carbonate',
        'mineral_eng': 'magnesite'
    },
    'siderita': {
        'classification': 'iron carbonate',
        'simple_classification': 'carbonate',
        'mineral_eng': 'siderite'
    },
    'nahcolita': {
        'classification': 'sodium carbonate',
        'simple_classification': 'carbonate',
        'mineral_eng': 'nahcolite'
    },
    'trona': {
        'classification': 'sodium carbonate and bicarbonate',
        'simple_classification': 'carbonate',
        'mineral_eng': 'trona'
    },
    'nasitina': {
        'classification': 'sodium carbonate',
        'simple_classification': 'carbonate',
        'mineral_eng': 'natron'
    },

    # Silicates
    'albita': {
        'classification': 'sodium aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'albite'
    },
    'albita cálcica': {
        'classification': 'sodium calcium aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'calcic albite'
    },
    'albita/anortita': {
        'classification': 'sodium calcium aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'albite/anorthite'
    },
    'anortita': {
        'classification': 'calcium aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'anorthite'
    },
    'andesina': {
        'classification': 'sodium calcium aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'andesine'
    },
    'bitownita': {
        'classification': 'calcium aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'bytownite'
    },
    'labradorita': {
        'classification': 'calcium sodium aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'labradorite'
    },
    'plagioclasa': {
        'classification': 'calcium sodium aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'plagioclase'
    },
    'microclina': {
        'classification': 'potassium aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'microcline'
    },
    'caolinita': {
        'classification': 'hydrated aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'kaolinite'
    },
    'caolinita (trazas)': {
        'classification': 'hydrated aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'kaolinite (trace)'
    },
    'montmorillonita': {
        'classification': 'sodium magnesium aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'montmorillonite'
    },
    'nontronita': {
        'classification': 'iron aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'nontronite'
    },
    'moscovita': {
        'classification': 'potassium aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'muscovite'
    },
    'muscovita (sericita)': {
        'classification': 'potassium aluminum silicate',
        'simple_classification': 'silicate',
        'mineral_eng': 'muscovite (sericite)'
    },

    # Chlorides
    'halita': {
        'classification': 'sodium chloride',
        'simple_classification': 'chloride',
        'mineral_eng': 'halite'
    },
    'silvita': {
        'classification': 'potassium chloride',
        'simple_classification': 'chloride',
        'mineral_eng': 'sylvite'
    },
    'silvita sódica': {
        'classification': 'potassium sodium chloride',
        'simple_classification': 'chloride',
        'mineral_eng': 'sodium sylvite'
    },
    'carobiita': {
        'classification': 'potassium chloride',
        'simple_classification': 'chloride',
        'mineral_eng': 'carobbiite'
    },

    # Oxides
    'hematita': {
        'classification': 'iron oxide',
        'simple_classification': 'oxide',
        'mineral_eng': 'hematite'
    },
    'trazas de hematita': {
        'classification': 'iron oxide',
        'simple_classification': 'oxide',
        'mineral_eng': 'hematite (trace)'
    },
    'rutilo': {
        'classification': 'titanium oxide',
        'simple_classification': 'oxide',
        'mineral_eng': 'rutile'
    },
    'cristobalita': {
        'classification': 'silicon oxide',
        'simple_classification': 'oxide',
        'mineral_eng': 'cristobalite'
    },
    'cristobalita (trazas)': {
        'classification': 'silicon oxide',
        'simple_classification': 'oxide',
        'mineral_eng': 'cristobalite (trace)'
    },
    'cuarzo': {
        'classification': 'silicon oxide',
        'simple_classification': 'oxide',
        'mineral_eng': 'quartz'
    },
    'trazas de cuarzo': {
        'classification': 'silicon oxide',
        'simple_classification': 'oxide',
        'mineral_eng': 'quartz (trace)'
    },
    'ópalo': {
        'classification': 'hydrated silicon oxide',
        'simple_classification': 'oxide',
        'mineral_eng': 'opal'
    },
    'claudetita': {
        'classification': 'arsenic oxide',
        'simple_classification': 'oxide',
        'mineral_eng': 'claudetite'
    },

    # Nitrates
    'niter': {
        'classification': 'potassium nitrate',
        'simple_classification': 'nitrate',
        'mineral_eng': 'niter'
    },
    'nitratina': {
        'classification': 'sodium nitrate',
        'simple_classification': 'nitrate',
        'mineral_eng': 'nitratine'
    },

    # Borates
    'inderita': {
        'classification': 'magnesium borate',
        'simple_classification': 'borate',
        'mineral_eng': 'inderite'
    },
    'inyoíta': {
        'classification': 'calcium borate',
        'simple_classification': 'borate',
        'mineral_eng': 'inyoite'
    },
    'uralborita': {
        'classification': 'calcium borate',
        'simple_classification': 'borate',
        'mineral_eng': 'uralborite'
    },
    'hexahidroborita': {
        'classification': 'calcium borate',
        'simple_classification': 'borate',
        'mineral_eng': 'hexahydroborite'
    },
    'korzhinskita': {
        'classification': 'calcium borate',
        'simple_classification': 'borate',
        'mineral_eng': 'korzhinskite'
    },
    'probertita': {
        'classification': 'sodium calcium borate',
        'simple_classification': 'borate',
        'mineral_eng': 'probertite'
    },

    # Others
    'cinabrio (?)': {
        'classification': 'mercury sulfide',
        'simple_classification': 'sulfide',
        'mineral_eng': 'cinnabar'
    },
    'trazas de pirita': {
        'classification': 'iron sulfide',
        'simple_classification': 'sulfide',
        'mineral_eng': 'pyrite (trace)'
    },
    'yodo (trazas)': {
        'classification': 'elemental iodine',
        'simple_classification': 'native element',
        'mineral_eng': 'iodine (trace)'
    },
    'd´ansita (?)': {
        'classification': 'sodium magnesium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'dansite'
    },
    'javorieita': {
        'classification': 'potassium magnesium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'javorieite'
    },
    'omonwaita': {
        'classification': 'sodium magnesium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'omonwaite'
    },
    'aphthitalita': {
        'classification': 'potassium sodium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'aphthitalite'
    },
    'despujolsita': {
        'classification': 'calcium magnesium sulfate',
        'simple_classification': 'sulfate',
        'mineral_eng': 'despujolsite'
    },
    'hilgardita': {
        'classification': 'calcium borate',
        'simple_classification': 'borate',
        'mineral_eng': 'hilgardite'
    },
    'hidroclorborita': {
        'classification': 'calcium borate',
        'simple_classification': 'borate',
        'mineral_eng': 'hydrochlorborite'
    },
}

Get classification of the mineralogy of each sample

In [8]:

def classify(sample):
    mineral1 = sample['Mineral principal 1']
    mineral2 = sample['Mineral principal 2']
    mineral3 = sample['Mineral principal 3']
    
    # Try to get the classification of the #1 most abundant mineral and its name in English
    try:
        min1 = mineral_classification[mineral1]['simple_classification']
        mineral1_eng = mineral_classification[mineral1]['mineral_eng']
        sample['mineral1'] = mineral1_eng
        sample['mineral1_class'] =  mineral_classification[mineral1]['classification']
    except:
        min1 = ''
        sample['mineral1'] = min1
        sample['mineral1_class'] = min1

    # Try to get the classification of the #2 most abundant mineral and its name in English
    try:
        min2 = mineral_classification[mineral2]['simple_classification']
        mineral2_eng = mineral_classification[mineral2]['mineral_eng']
        sample['mineral2'] = mineral2_eng
        sample['mineral2_class'] = mineral_classification[mineral2]['classification']
    except:
        min2 = ''
        sample['mineral2'] = min2
        sample['mineral2_class'] = min2

    # Try to get the classification of the #3 most abundant mineral and its name in English
    try:
        min3 = mineral_classification[mineral3]['simple_classification']
        mineral3_eng = mineral_classification[mineral3]['mineral_eng']
        sample['mineral3'] = mineral3_eng  
        sample['mineral3_class'] = mineral_classification[mineral3]['classification']
    except:
        min3 = ''
        sample['mineral3'] = min3
        sample['mineral3_class'] = min3

    # Classification
    if min1 != '':

        # Main classification is the classification of the #1 most abundant mineral
        sample['main_classification'] = f'{min1}s'.capitalize()

        # Determine overall classification and main minerals
        classifications = []
        minerals = []

        classifications.append(f'{min1}s')
        minerals.append(mineral1_eng)

        if min2 != '' and min2 != min1:
            classifications.append(f'{min2}s')
            minerals.append(mineral2_eng)

        if min3 != '' and min3 != min1 and min3 != min2:
            classifications.append(f'{min3}s')
            minerals.append(mineral3_eng)

        sample['overall_classification'] = ', '.join(classifications).capitalize()
        sample['main_minerals'] = ', '.join(minerals).capitalize()
    else:
        sample['overall_classification'] = ''
        sample['main_minerals'] = ''

    return sample

# Apply classification function
classified_data = data.apply(classify, axis=1)

# Get lat lon values from GeoDataFrame
classified_data['lon']  = classified_data.geometry.x
classified_data['lat']  = classified_data.geometry.y

# Convert selected column in numeric data
classified_data[sulfate] = pd.to_numeric(classified_data[sulfate], errors='coerce')
classified_data[chloride] = pd.to_numeric(classified_data[chloride], errors='coerce')
classified_data[carbonate] = pd.to_numeric(classified_data[carbonate], errors='coerce')

Create Folium Map

In [9]:

# Create Folium Map
m = folium.Map(
    location=[-25.74, -68.63], 
    tiles= 'OpenTopoMap')

# Create pie chart with Matplotlib
def create_radial_chart(values, labels, colors):
    fig, ax = plt.subplots()
    ax.pie(values, labels=labels, colors=colors, labeldistance=None)
    plt.legend(fontsize='x-large', borderpad=0.8, bbox_to_anchor=(0.85, 1.0), loc='upper left')
    
    # Convert the figure into a base64 image
    buffer = BytesIO()
    plt.savefig(buffer, format='png')
    plt.close()
    buffer.seek(0)
    return base64.b64encode(buffer.read()).decode()

# Create a marker for each sample
for sample in range(len(classified_data)):
    # Get lat and lon for each sample
    lat = classified_data.iloc[sample]['lat']
    lon = classified_data.iloc[sample]['lon']

    # Get information: classification and mineralogy
    sample_code = classified_data.iloc[sample]['Código de muestra']
    classification = classified_data.iloc[sample]['main_classification']
    description = classified_data.iloc[sample]['overall_classification']
    min1 = classified_data.iloc[sample]['mineral1']
    min2 = classified_data.iloc[sample]['mineral2']
    min3 = classified_data.iloc[sample]['mineral3']
    min1_class = classified_data.iloc[sample]['mineral1_class']
    min2_class = classified_data.iloc[sample]['mineral2_class']
    min3_class = classified_data.iloc[sample]['mineral3_class']
    min1_info = min1 + ': ' + min1_class if min1!= '' else ''
    min2_info = min2 + ': ' + min2_class if min2!= '' else ''
    min3_info = min3 + ': ' + min3_class if min3!= '' else ''
    
    color_mapping = {
    'Sulfates': 'beige',
    'Chlorides': 'lightgreen',
    'Carbonates': 'lightblue'}
    
    # Get color for the classified mineral using the color mapping variable
    color_map = color_mapping.get(classification, 'gray') 
    component = [sulfate, chloride, carbonate]
    
    value1 = float(classified_data.iloc[sample][sulfate])
    value2 = float(classified_data.iloc[sample][chloride])
    value3 = float(classified_data.iloc[sample][carbonate])
    total = value1 + value2 + value3
    values = [value1 * 100 / total, value2 * 100 / total, value3 * 100 /total]
    
    # Create pie chart
    colors = ['#F4D03F', '#58D68D', '#5DADE2' ]
    chart_image = create_radial_chart(values, component, colors)

    # Create html to be displayed
    html = f'''<h3>{description}</h3>
            <b>Sample: </b>{sample_code}<br><br>
            <b>Main minerals: </b><br><br>
             {min1_info}<br>
             {min2_info}<br>
             {min3_info}<br><br>
             <b>Main anions:</b>
            <img src="data:image/png;base64,{chart_image}" width="300">
            '''

    iframe = folium.IFrame(html,
                       width=320,
                       height=500)

    popup = folium.Popup(iframe,
                     max_width=320)
    # Add markers
    folium.Marker(
        location=[lat, lon],
        popup=popup,
        icon=folium.Icon(color=color_map, icon='glyphicon-certificate')  # Cambia 'info-sign' por el ícono que prefieras
    ).add_to(m)
    
# Save and show the map
m.save('mapa.html')

Visualize the Folium Map

In [10]:

Out[10]:

Make this Notebook Trusted to load map: File -> Trust Notebook