How to Download Files Using PowerShell?

PowerShell is a powerful command-line shell and scripting language that allows you to automate various administrative tasks on Windows. As someone who regularly downloads files, I know how tedious it can be to manually download each file individually. That’s why I’ve turned to PowerShell to automate my file downloads. In this guide, I’ll walk you through the process of using PowerShell to download files from URLs, including multiple files and files with authentication.

Benefits of using PowerShell for file downloads

Using PowerShell to download files has several benefits. First, it saves time by automating the process. You can download multiple files at once, and you don’t have to click through each link manually. Second, PowerShell allows you to authenticate with websites that require login credentials to access files. Finally, PowerShell is scriptable, which means you can write a PowerShell script to download files automatically on a schedule. Alternatively, you can download multiple files from a CSV file using PowerShell.

Using Invoke-WebRequest cmdlet for downloading files in PowerShell

PowerShell offers the Invoke-WebRequest cmdlet to download files. This cmdlet can be used to download files from URLs. It can handle more complex web interactions, such as authentication and custom headers. You can use this cmdlet to download files from the internet or internal servers using HTTP, HTTPS, FTP, etc.

Some of the common parameters you can use with Invoke-WebRequest command:

  • -Uri – The URL of the file to download
  • -OutFile – The local path to save the downloaded file
  • -Headers – Add custom headers to the request
  • -ContentType – Specify content types like JSON, XML, etc.
  • -Method – HTTP methods like GET, POST, DELETE, etc.
  • -Credential – Provide login credentials if the site requires authentication
  • -UserAgent – Specify a custom user agent string

This provides you with a lot of control over the request and handling the response.

Please note that there are other methods to download files using PowerShell, such as the Start-BitsTransfer, Invoke-RestMethod cmdlets or creating a System.Net.WebClient object in .Net framework. However, using the Invoke-WebRequest cmdlet is a simple and effective solution. You can use Invoke-WebRequest as a PowerShell wget Alternative.

How to download a file from a URL using PowerShell?

To download a file from a URL using PowerShell, you can use the Invoke-WebRequest cmdlet. Here’s an example:

Invoke-WebRequest -Uri "https://7-zip.org/a/7z2301-x64.exe" -OutFile "C:\Downloads\7Zip.exe"

This command downloads a file from the given URL and saves it to the location “C:\Downloads\7Zip.exe”. It overwrites the existing file without any warning if it exists already. You can bulk download multiple files from a CSV file using:

Import-Csv C:\Temp\DownloadFiles.csv | Start-BitsTransfer

Downloading files with a progress bar

To download a file with a progress bar, you can use the Write-Progress cmdlet to display a progress bar. Here’s an example to download zip files using PowerShell:

Invoke-WebRequest -Uri "https://SharePointDiary.com/file.zip" -OutFile "C:\Downloads\file.zip" -UseBasicParsing -PassThru | Write-Progress -Activity "Downloading file" -Status "Progress"

This command downloads a file and displays a progress bar with the “Downloading file” activity.

The CURL and WGet commands are the aliases of Invoke-WebRequest in PowerShell:

PowerShell curl wget

Downloading a Single file using Web.Client Method

The .NET System.Net.WebClient class provides another option for downloading files and data. Here is an example:

$Client = New-Object System.Net.WebClient
$Client.DownloadFile("https://www.sharepointdiary.com/videso/Google-Console.mp4", "C:\Downloads\Google-Console.mp4")

This will download the file from the URL to the local path.

Using the Start-BitsTransfer cmdlet to Download Files

The Start-BitsTransfer cmdlet is part of the BITS (Background Intelligent Transfer Service) PowerShell module. BITS is a service in Microsoft Windows that facilitates asynchronous, prioritized, resuming and throttled transfer of files between machines using idle network bandwidth. To download files using the Start-BitsTransfer cmdlet in PowerShell, you can use the script below:

#Parameters
$URL = "https://www.sharepointdiary.com/videso/Google-Console.mp4"
$DownloadPath = "C:\Downloads\Google-Console.mp4"

Start-BitsTransfer -Source $URL -Destination $DownloadPath

This method is especially useful for downloading large files or when you want to manage background file transfers efficiently.

start-bitsTransfer powershell

Download Multiple Files using PowerShell

To download multiple files using PowerShell, you can use a loop to iterate through a list of URLs and download each file. Here’s an example:

$urls = "https://www.sharepointdiary.com/file1.zip", "https://www.sharepointdiary.com/file2.zip", "https://www.sharepointdiary.com/file3.zip"

ForEach ($url in $urls) {
    Invoke-WebRequest -Uri $url -OutFile "C:\Downloads\$(Split-Path -Leaf $url)"
}

This command downloads three files from the URLs in the $urls array and saves each file to the destination location “C:\Downloads” with the original filename.

Let’s download multiple files from URLs with a Progress Bar using PowerShell:

# List of file URLs to download
$Urls = @(
    "https://www.sharepointdiary.com/videos/Introduction_to_Google_Analytics.mp4",
    "https://www.sharepointdiary.com/videos/Installation_and_Setup.mp4",
    "https://www.sharepointdiary.com/videos/HANDS-ON_TRAINING_Setup.mp4"
)

# Destination folder
$DestinationFolder = "C:\Downloads"

# Ensure the destination folder exists
If (-not (Test-Path $DestinationFolder)) {
    New-Item -Path $DestinationFolder -ItemType Directory
}

# Download each file
$TotalUrls = $urls.Count
$Counter = 0
ForEach($URL in $Urls)
{
    $FileName = [System.IO.Path]::GetFileName($url)  # Extract file name from URL
    $DestinationPath = Join-Path -Path $DestinationFolder -ChildPath $FileName

    # Display main progress
    Write-Progress -Activity "Downloading files ($($Counter+1) of $TotalUrls)" -Status ("Downloading " + $FileName) -PercentComplete (($Counter / $TotalUrls) * 100)

    # Download file with sub-progress bar for individual file download
    Invoke-WebRequest -Uri $url -OutFile $destinationPath
    $Counter++
}

Write-Progress -Activity "Downloading files" -Completed -Status "All files downloaded!"
Write-Host "All files downloaded successfully!" -ForegroundColor Green

How to download files with authentication using PowerShell?

You can pass login credentials in the request headers to download files with authentication using PowerShell. Here’s an example:

# Parameters
$UserName = "SharePointDiary"
$Password = "password goes here"

# Setup Credentials
$SecurePassword = ConvertTo-SecureString $Password -AsPlainText -Force
$Credential = New-Object System.Management.Automation.PSCredential ($username, $securePassword)

Invoke-WebRequest -Uri "https://SharePointDiary.com/securefile.zip" -OutFile "C:\Downloads\securefile.zip" -Credential $credential

This command downloads a file from a secure URL that requires authentication. The login credentials are passed in the $credential object.

Download Files from the FTP Server using PowerShell

The same method also works for FTP Servers. Here is the script to download files from an FTP Server:

# FTP server Login details
$UserName = "dlpuser"
$Password = "rNrKYTX9g7z3RgJRmxWuGHbeu"
$URL = "ftp://dlpuser@ftp.dlptest.com/test-cron-2023-10-29.xlsx"
$Destination = "C:\Downloads\CronJob.xlsx"

# Setup Credentials
$SecurePassword = ConvertTo-SecureString $Password -AsPlainText -Force
$Credential = New-Object System.Management.Automation.PSCredential ($username, $securePassword)

#Download File from FTP
Invoke-WebRequest -Uri $URL -OutFile $Destination -Credential $credential

Here, the $destination variable stores the location to download a file on a local computer.

Troubleshooting common issues with PowerShell file downloads

If you encounter issues with PowerShell file downloads, here are a few troubleshooting tips:

  • Empty/Zero byte file – This usually indicates the download was interrupted or failed. Check the request status code and response for errors. Retry the download.
  • Check your internet connection
  • Access/Authorization errors – The site may be blocking the request due to invalid credentials or user agents. Verify your headers/credentials or use a browser user agent string.
  • Timeout errors – This could mean a slow network. Retry with a larger TimeoutSec value in Invoke-WebRequest parameters.
  • Check the URL for typos or errors
  • Check your download location for sufficient disk space
  • Concurrent download limits – Servers may block if hitting request limits. Use throttling, queues, and pause between requests.
  • Connection/Network errors – Validate network connectivity. Check for proxies, firewalls, VPNs, or other network issues blocking access.
  • Destination access denied – Script may lack permissions to target folder. Adjust permissions or run as administrator if needed.

Carefully inspecting error messages and response codes will help diagnose most issues. Adding error handling and logging will make troubleshooting easier.

Best practices for downloading Files with PowerShell

To ensure successful downloads with PowerShell, here are a few best practices:

  • Validate the URLs and check for redirects before downloading to avoid getting unexpected files. Handle errors gracefully in your scripts.
  • Use the -UseBasicParsing parameter for non-HTML files
  • Handle authentication securely – avoid putting credentials directly in the URL. Use the Credential parameter or authentication headers instead.
  • Use the -OutFile parameter to specify the download location
  • Use user-agent strings that identify your script so that servers can identify and handle your requests properly.
  • Add logging to your scripts to record activities for troubleshooting and auditing. Monitor progress for large downloads so you can track status and retry if needed.
  • Use hash validation like Get-FileHash after download to verify integrity if the source provides file hashes.
  • Use the -Credential parameter for authenticated downloads

Conclusion

Downloading files from the internet and servers is a common task that can be easily automated using PowerShell. With Invoke-WebRequest, you can download files from URLs, authenticate with websites, and even download multiple files simultaneously. Whether you need to pull reports, user uploads, or system files, PowerShell makes it simple to write scripts that can download virtually any file or data from any source. By following the techniques covered in this guide, you can streamline your file downloads and save time. Give it a try and see how much time you can save!

What is the PowerShell equivalent of wget?

The equivalent of wget in Windows PowerShell is the Invoke-WebRequest cmdlet. It allows you to send HTTP and HTTPS requests to a web page and retrieve the content.

How do I download a SharePoint file using PowerShell?

To download a SharePoint file using PowerShell, you can use the “Get-PnPFile” cmdlet from the PnP PowerShell module. Here is an example of how to do it:
Get-PnPFile -Url "/sites/yoursite/Shared Documents/filename.ext" -Path "C:\Download" -FileName filename.ext -AsFile

What is the alternative to curl in PowerShell?

The alternative to curl in PowerShell is the Invoke-WebRequest and Invoke-RestMethod cmdlets. It allows you to send HTTP and HTTPS requests to a RESTful web service and retrieve the response.

How do I download a zip file from PowerShell?

To download a zip file using PowerShell, you can use the Invoke-WebRequest cmdlet. Here is an example of the command you can use:
Invoke-WebRequest -Uri "URL of the zip file" -OutFile "Path to save the downloaded file"

How do I add credentials to a PowerShell script?

To add credentials to a PowerShell script, you can use the Get-Credential cmdlet to prompt the user for their username and password. Then, you can store the credentials in a variable and use it in your script. Here’s an example:
$credentials = Get-Credential
You can also hard code credentials in the PowerShell script using the PSCredential object. Here is an example:
$username = "domain\username"
$password = ConvertTo-SecureString "password" -AsPlainText -Force
$Credential = New-Object System.Management.Automation.PSCredential ($username, $password)

How do I download a file from the Internet using PowerShell?

To download a file from the Internet using PowerShell, you can use the Invoke-WebRequest cmdlet. Here is an example of how to do it:
Invoke-WebRequest -Uri $url -OutFile $localFilePath

Salaudeen Rajack

Salaudeen Rajack - Information Technology Expert with Two-decades of hands-on experience, specializing in SharePoint, PowerShell, Microsoft 365, and related products. He has held various positions including SharePoint Architect, Administrator, Developer and consultant, has helped many organizations to implement and optimize SharePoint solutions. Known for his deep technical expertise, He's passionate about sharing the knowledge and insights to help others, through the real-world articles!

One thought on “How to Download Files Using PowerShell?

  • The invoke-webrequest is a simple way to do it, but it stops after 39Mb and leave the rest behind

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *