The Linux wget command is a command-line utility that downloads files from the internet using HTTP, HTTPS, and FTP protocols. It’s designed to work non-interactively, meaning it can run in the background while you’re logged out\u2014making it perfect for retrieving large files, automating downloads in scripts, and handling unreliable network connections. Most Linux distributions include wget by default, but if you need to install it, here’s how to do it across different distros.<\/p>\n\n\n\n
Ubuntu\/Debian-based distros: (difference between apt and apt-get<\/a>)<\/strong><\/p>\n\n\n Fedora-based distros:<\/strong><\/p>\n\n\n Red Hat-based distros:<\/strong><\/p>\n\n\n Arch Linux-based distros<\/strong>:<\/p>\n\n\n Alpine Linux:<\/strong><\/p>\n\n\n The wget command stands apart from other download utilities because of its robust feature set designed specifically for automation and unreliable networks. Unlike browser-based downloads or simpler tools, wget was built with scripting and unattended operations in mind.<\/p>\n\n\n\n Here are the key capabilities that make wget indispensable for system administrators and developers:<\/p>\n\n\n\n The basic syntax of the wget command is:<\/strong><\/p>\n\n\n The simplest way to use wget is to provide it with a URL. Let’s download the Ubuntu Server ISO as a practical example. This demonstrates how wget handles large file downloads and displays progress information.<\/p>\n\n\n When you run this command, wget performs several operations in sequence. First, it resolves the domain name to an IP address. Then it establishes a connection to the server and begins downloading. The output shows you detailed information about the transfer, including the server connection details, file size, download speed, and an estimated time to completion with a progress bar.<\/p>\n\n\n\n By default, wget saves the file with its original filename from the server. But what if you want to save it with a different name? That’s where the The This downloads the Ubuntu ISO but saves it as “ubuntu-server.iso” in your current directory. The Rather than downloading files to your current directory, you can direct wget to save files in a specific location using the This command downloads the ISO directly into your Downloads folder. The directory must exist before running the command, or wget will return an error. You can create the directory first with Network interruptions happen. Your WiFi drops, your SSH connection times out, or you accidentally hit Ctrl+C. The Here’s how this works in practice: when you run wget with the This is especially valuable when downloading multi-gigabyte files like Linux ISOs, database backups, or video files. Instead of starting over from scratch each time your connection drops, you simply re-run the same wget command with While wget’s detailed output is helpful for interactive use, it becomes noise when running downloads in scripts or cron jobs. The good news is that wget provides several options to control exactly what information it displays.<\/p>\n\n\n\n The The Sometimes you want to see download progress without all the verbose server connection details. The If you’re using an older version of wget (pre-1.16), you might need to add Rather than running wget multiple times, you can download several files in a single command by listing all URLs in a text file. This approach is cleaner, more maintainable, and perfect for automation.<\/p>\n\n\n\n First, create a text file with one URL per line. Let’s call it Now use the wget processes each URL sequentially, downloading them one after another. You can combine this with other options we’ve discussed. For example, This method is particularly powerful for bulk downloads of software packages, documentation sets, or media files. You can even generate the URL list programmatically with scripts if you need to download files following a pattern.<\/p>\n\n\n\n When you’re sharing a network connection or running background downloads, limiting wget’s bandwidth usage prevents it from saturating your connection and affecting other applications. The You can express the rate limit in bytes, kilobytes (k suffix), or megabytes (m suffix). Here’s a practical example limiting the download to 500 kilobytes per second:<\/p>\n\n\n For slower connections or when you want to be very conservative with bandwidth, you might limit to 100k or even 50k. On faster connections where you just want to leave some bandwidth for other applications, you might set it to 2m or 5m.<\/p>\n\n\n\n The One limitation: you cannot change the rate limit while a download is in progress. However, you can stop the download (Ctrl+C), then resume it with a different rate limit using This resumes the download from where you left off but now limits the speed to 200 KB\/s. This technique is useful when network conditions change\u2014for example, if other people start using your connection during the day, you can pause and resume with a lower rate limit.<\/p>\n\n\n\n Many web servers and FTP sites require authentication before allowing downloads. wget handles HTTP Basic Authentication and FTP authentication through several methods, each with different security trade-offs.<\/p>\n\n\n\n For HTTP or HTTPS sites, use the However, this approach has a security problem: the password appears in your terminal command, in your shell history, and is visible to anyone who runs This prompts you to enter the password interactively, keeping it out of logs and process listings. The password won’t be echoed to the screen as you type it.<\/p>\n\n\n\n For FTP downloads, use You can also embed credentials directly in the URL, though this has the same security concerns:<\/p>\n\n\n For automated scripts that need authentication, store credentials in a The One of wget’s most powerful features is its ability to recursively download entire websites, creating local copies suitable for offline browsing. This is invaluable for archiving content, creating offline documentation mirrors, or analyzing website structure.<\/p>\n\n\n\n The Let me break down what each option does:<\/p>\n\n\n\n Without constraints, wget will follow every link it finds, potentially downloading far more than you intended. Use these options to control the scope:<\/p>\n\n\n The Important warning:<\/strong> Be cautious when mirroring large sites. A site with thousands of pages can take hours to download and consume significant disk space. Always check robots.txt and respect the website’s terms of service. Many sites explicitly prohibit automated scraping. Use the This adds a 2-second wait between each request, making your mirror operation much more respectful to the server.<\/p>\n\n\n\n Website owners and developers need to identify broken links (404 errors) across their sites. wget’s The Here’s what’s happening:<\/p>\n\n\n\n After the spider completes, search the log file for 404 errors:<\/p>\n\n\n This pipeline extracts just the URLs that returned 404 errors, sorted and deduplicated. You’ll get a clean list of broken links to fix. The For larger sites, consider adding For long-running downloads, keeping your terminal open is impractical. The When you run this command, wget immediately returns control to your terminal and displays a message like “Continuing in background, pid 12345.” The process ID (12345 in this example) is important\u2014you can use it to check the download status or stop it if needed.<\/p>\n\n\n\n wget automatically logs background download progress to a file named You can also specify a custom log file location:<\/p>\n\n\n To stop a background download, use the Background downloads are particularly useful in combination with tools like Sometimes wget refuses to download from HTTPS sites due to certificate validation issues. While you should generally fix the underlying certificate problem, there are legitimate cases where you might need to bypass these checks\u2014like when dealing with self-signed certificates in development environments.<\/p>\n\n\n The Let me show you a couple of advanced techniques that may come handy when you start using the wget command regularly. <\/p>\n\n\n\n The If the remote file hasn’t changed since your last download, wget skips it entirely. This saves bandwidth and time when running regular sync jobs.<\/p>\n\n\n\n For unreliable connections, increase the retry count and add wait time between attempts:<\/p>\n\n\n This attempts the download up to 10 times, waits 5 seconds between retries, and even retries when the server explicitly refuses connections.<\/p>\n\n\n\n Some servers block wget’s default user-agent. Impersonate a browser if necessary:<\/p>\n\n\n This makes wget identify itself as a Chrome browser on Windows, bypassing basic user-agent filters.<\/p>\n\n\n\n\nsudo apt install wget\n<\/pre><\/div>\n\n\n
\nsudo dnf install wget\n<\/pre><\/div>\n\n\n
\nsudo yum install wget\n<\/pre><\/div>\n\n\n
\nsudo pacman -S wget\n<\/pre><\/div>\n\n\n
\napk add wget\n<\/pre><\/div>\n\n\n
What Makes wget Different from Other Download Tools?<\/h2>\n\n\n\n
\n
\nwget [options] <URL>\n<\/pre><\/div>\n\n\n
How Do I Download Files with wget?<\/h2>\n\n\n\n
\nwget https:\/\/releases.ubuntu.com\/22.04\/ubuntu-22.04.3-live-server-amd64.iso\n<\/pre><\/div>\n\n\n
-O<\/code> (capital letter O) option comes in handy.<\/p>\n\n\n\nSaving Downloads with a Custom Filename<\/h3>\n\n\n\n
-O<\/code> option lets you specify exactly what you want to call the downloaded file. This is particularly useful when downloading files with cryptic names or when you want to maintain a specific naming convention in your scripts.<\/p>\n\n\n\nwget https:\/\/releases.ubuntu.com\/22.04\/ubuntu-22.04.3-live-server-amd64.iso -O ubuntu-server.iso\n<\/pre><\/div>\n\n\n
-O<\/code> option is especially valuable when working with dynamically generated URLs that might produce files like “download.php?id=12345” \u2013 you can give them meaningful names instead.<\/p>\n\n\n\nSpecifying a Different Download Directory<\/h3>\n\n\n\n
-P<\/code> option. This is cleaner for organizing downloads and essential when automating downloads in scripts.<\/p>\n\n\n\nwget -P ~\/Downloads https:\/\/releases.ubuntu.com\/22.04\/ubuntu-22.04.3-live-server-amd64.iso\n<\/pre><\/div>\n\n\n
mkdir -p ~\/Downloads<\/code> if needed.<\/p>\n\n\n\nHow Do I Resume an Interrupted Download?<\/h2>\n\n\n\n
-c<\/code> option (short for “continue”) is what makes wget particularly valuable for large file downloads over unstable connections.<\/p>\n\n\n\nwget -c https:\/\/releases.ubuntu.com\/22.04\/ubuntu-22.04.3-live-server-amd64.iso\n<\/pre><\/div>\n\n\n
-c<\/code> option, it checks if a partially downloaded file already exists in the current directory. If it finds one, wget contacts the server and requests only the remaining portion of the file, starting from where the previous download stopped. The server sends an HTTP Range header indicating it will resume from byte position X, and the download continues seamlessly.<\/p>\n\n\n\n-c<\/code> and pick up where you left off. The savings in time and bandwidth can be significant, particularly on slower connections or when downloading from geographically distant servers.<\/p>\n\n\n\nHow Can I Control wget’s Output Display?<\/h2>\n\n\n\n
Completely Quiet Mode<\/h3>\n\n\n\n
-q<\/code> flag stands for “quiet” and suppresses all output completely. This is useful for scripts where you only care about the exit status (0 for success, non-zero for failure).<\/p>\n\n\n\nwget -q <URL>\n<\/pre><\/div>\n\n\n
Non-Verbose Mode<\/h3>\n\n\n\n
-nv<\/code> option provides a middle ground\u2014it turns off verbose output but still displays error messages and completion notices. This is better for logging purposes since you’ll know if something went wrong.<\/p>\n\n\n\nwget -nv <URL>\n<\/pre><\/div>\n\n\n
Showing Only the Progress Bar<\/h3>\n\n\n\n
--show-progress<\/code> option combined with -q<\/code> gives you exactly that\u2014just a clean progress bar.<\/p>\n\n\n\nwget -q --show-progress https:\/\/releases.ubuntu.com\/22.04\/ubuntu-22.04.3-live-server-amd64.iso\n<\/pre><\/div>\n\n\n
--progress=bar:force<\/code> to ensure the progress bar displays correctly. The combination of these flags is particularly useful in terminal multiplexers like tmux or screen, where you want to monitor progress without cluttering your terminal history.<\/p>\n\n\n\nHow Do I Download Multiple Files at Once?<\/h2>\n\n\n\n
downloads.txt<\/code>:<\/p>\n\n\n\ncat > downloads.txt << EOF\nhttps:\/\/wordpress.org\/latest.tar.gz\nhttps:\/\/github.com\/docker\/compose\/releases\/download\/v2.23.0\/docker-compose-linux-x86_64\nhttps:\/\/go.dev\/dl\/go1.21.5.linux-amd64.tar.gz\nEOF\n<\/pre><\/div>\n\n\n
-i<\/code> option (short for “input file”) to pass this list to wget:<\/p>\n\n\n\nwget -i downloads.txt\n<\/pre><\/div>\n\n\n
wget -i downloads.txt -P ~\/Downloads -q --show-progress<\/code> downloads all files to your Downloads folder while showing only progress bars.<\/p>\n\n\n\nHow Can I Limit Download Speed with wget?<\/h2>\n\n\n\n
--limit-rate<\/code> option gives you precise control over download speed.<\/p>\n\n\n\nSetting Bandwidth Limits<\/h3>\n\n\n\n
\nwget --limit-rate=500k https:\/\/releases.ubuntu.com\/22.04\/ubuntu-22.04.3-live-server-amd64.iso\n<\/pre><\/div>\n\n\n
Understanding How Rate Limiting Works<\/h3>\n\n\n\n
--limit-rate<\/code> option sets an average bandwidth target rather than a hard ceiling. wget might briefly exceed the specified rate as it fills network buffers, but it will throttle back to maintain the average over time. This approach works better with how TCP\/IP networking actually functions.<\/p>\n\n\n\n-c<\/code>:<\/p>\n\n\n\nwget -c --limit-rate=200k https:\/\/releases.ubuntu.com\/22.04\/ubuntu-22.04.3-live-server-amd64.iso\n<\/pre><\/div>\n\n\n
How Do I Download from Password-Protected Sites?<\/h2>\n\n\n\n
HTTP\/HTTPS Authentication<\/h3>\n\n\n\n
--http-user<\/code> and --http-password<\/code> options:<\/p>\n\n\n\nwget --http-user=myusername --http-password=mypassword https:\/\/example.com\/protected\/file.zip\n<\/pre><\/div>\n\n\n
ps<\/code> while wget is running. For better security, use the --ask-password<\/code> option:<\/p>\n\n\n\nwget --http-user=myusername --ask-password https:\/\/example.com\/protected\/file.zip\n<\/pre><\/div>\n\n\n
FTP Authentication<\/h3>\n\n\n\n
--ftp-user<\/code> and --ftp-password<\/code>:<\/p>\n\n\n\nwget --ftp-user=ftpuser --ftp-password=ftppass ftp:\/\/ftp.example.com\/files\/archive.tar.gz\n<\/pre><\/div>\n\n\n
\nwget ftp:\/\/username:password@ftp.example.com\/files\/archive.tar.gz\n<\/pre><\/div>\n\n\n
Storing Credentials Securely<\/h3>\n\n\n\n
.wgetrc<\/code> file in your home directory. This keeps passwords out of your scripts and command history.<\/p>\n\n\n\ncat > ~\/.wgetrc << EOF\nhttp_user=myusername\nhttp_password=mypassword\nftp_user=ftpuser\nftp_password=ftppass\nEOF\n\nchmod 600 ~\/.wgetrc\n<\/pre><\/div>\n\n\n
chmod 600<\/code> command ensures only you can read this file. Now wget will automatically use these credentials without needing them in your commands. For sensitive production environments, consider using more sophisticated secrets management tools rather than storing passwords in plain text files.<\/p>\n\n\n\nCan I Mirror an Entire Website with wget?<\/h2>\n\n\n\n
Basic Website Mirroring<\/h3>\n\n\n\n
-m<\/code> option (short for “mirror”) enables recursive downloading. Combined with other options, you can create a complete, browsable local copy:<\/p>\n\n\n\nwget -m -p -k -E https:\/\/example.com\n<\/pre><\/div>\n\n\n
\n
-m<\/code> (mirror): Enables recursive downloading with infinite recursion depth, timestamps preservation, and other settings optimal for mirroring<\/li>\n\n\n\n-p<\/code> (page-requisites): Downloads all files necessary to properly display HTML pages, including images, CSS files, and JavaScript<\/li>\n\n\n\n-k<\/code> (convert-links): Converts links in downloaded HTML files to point to local files instead of the original URLs, making the site browsable offline<\/li>\n\n\n\n-E<\/code> (adjust-extension): Adds proper file extensions like .html to files that don’t have them, which helps local web browsers render them correctly<\/li>\n<\/ul>\n\n\n\nControlling Mirror Depth and Scope<\/h3>\n\n\n\n
\nwget -m -p -k -E -l 3 --no-parent https:\/\/example.com\/documentation\/\n<\/pre><\/div>\n\n\n
-l 3<\/code> option limits recursion to three levels deep from the starting URL. The --no-parent<\/code> option prevents wget from following links to parent directories, keeping your mirror focused on the specified path. This is essential when you only want to mirror a section of a site, like documentation, without grabbing the entire domain.<\/p>\n\n\n\n--wait<\/code> option to add delays between requests to avoid overwhelming servers:<\/p>\n\n\n\nwget -m -p -k -E --wait=2 https:\/\/example.com\n<\/pre><\/div>\n\n\n
How Can I Find Broken Links on a Website?<\/h2>\n\n\n\n
--spider<\/code> mode was designed specifically for this purpose\u2014it crawls through links without actually downloading content, checking only if resources exist.<\/p>\n\n\n\nRunning a Link Check<\/h3>\n\n\n\n
--spider<\/code> option combined with recursive crawling checks every link on a site:<\/p>\n\n\n\nwget --spider -r -l 5 https:\/\/yourwebsite.com -o link-check.log\n<\/pre><\/div>\n\n\n
\n
--spider<\/code>: Enables spider mode\u2014wget only checks if files exist without downloading them<\/li>\n\n\n\n-r<\/code>: Enables recursive link following<\/li>\n\n\n\n-l 5<\/code>: Limits recursion depth to 5 levels to keep the check manageable<\/li>\n\n\n\n-o link-check.log<\/code>: Saves all output to a log file for later analysis<\/li>\n<\/ul>\n\n\n\nAnalyzing the Results<\/h3>\n\n\n\n
\ngrep -B 2 '404' link-check.log | grep 'http' | cut -d ' ' -f 4 | sort -u\n<\/pre><\/div>\n\n\n
-B 2<\/code> flag shows two lines before each 404 match, giving you context about which page contains the broken link.<\/p>\n\n\n\n--no-parent<\/code> to restrict the spider to specific sections, and use --wait=1<\/code> to avoid hammering your server with rapid-fire requests.<\/p>\n\n\n\nHow Do I Run Downloads in the Background?<\/h2>\n\n\n\n
-b<\/code> option sends wget to the background, letting you close your terminal or continue working while the download proceeds.<\/p>\n\n\n\nwget -b https:\/\/releases.ubuntu.com\/22.04\/ubuntu-22.04.3-live-server-amd64.iso\n<\/pre><\/div>\n\n\n
Monitoring Background Downloads<\/h3>\n\n\n\n
wget-log<\/code> in the current directory. Monitor it in real-time with:<\/p>\n\n\n\ntail -f wget-log\n<\/pre><\/div>\n\n\n
\nwget -b -o ~\/downloads\/ubuntu.log https:\/\/releases.ubuntu.com\/22.04\/ubuntu-22.04.3-live-server-amd64.iso\n<\/pre><\/div>\n\n\n
kill<\/code> command with the process ID:<\/p>\n\n\n\nkill 12345\n<\/pre><\/div>\n\n\n
screen<\/code> or tmux<\/code>, which let you start downloads over SSH and safely disconnect without interrupting them.<\/p>\n\n\n\nHow Do I Handle HTTPS Certificate Errors?<\/h2>\n\n\n\n
\nwget --no-check-certificate https:\/\/self-signed.example.com\/file.zip\n<\/pre><\/div>\n\n\n
--no-check-certificate<\/code> option disables SSL certificate validation. Use this cautiously and only when you trust the source, as it makes you vulnerable to man-in-the-middle attacks. For production environments, always fix certificate issues properly rather than bypassing validation.<\/p>\n\n\n\nAdvanced wget Command Techniques Worth Knowing<\/h2>\n\n\n\n
Downloading Files with Timestamping<\/h3>\n\n\n\n
-N<\/code> option enables timestamping, which only downloads files if they’re newer than your local copy. This is perfect for keeping local mirrors synchronized:<\/p>\n\n\n\nwget -N https:\/\/example.com\/daily-data.csv\n<\/pre><\/div>\n\n\n
Using wget with Retries<\/h3>\n\n\n\n
\nwget --tries=10 --retry-connrefused --waitretry=5 https:\/\/unstable-server.com\/file.tar.gz\n<\/pre><\/div>\n\n\n
Downloading with Custom User-Agent Strings<\/h3>\n\n\n\n
\nwget --user-agent="Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36" https:\/\/example.com\/file.pdf\n<\/pre><\/div>\n\n\n
Conclusion<\/h2>\n\n\n\n