SharePoint Online: Download All Files from a Document Library using PowerShell
Requirement: PowerShell to download all files from a SharePoint Online document library.
How to Download a Document Library in SharePoint Online?
Document libraries are a core component of SharePoint Online, providing users with a central location to store, organize, and access files. However, there may be times when users need to download the entire contents of a document library for backup, migration, or other purposes. If you need to download a document library in SharePoint Online, there are a few different ways to go about it. In this blog post, I’ll show you how to download all files from a SharePoint Online library using the web browser, File explorer, and PowerShell. This can be helpful if you need to gather all the files in a library for archival or another purpose. Let’s get started!
Downloading multiple files at once from SharePoint Online is as easy as copying files with File Explorer (Open the library in File Explorer, Copy-Paste the document library folder to your local disk). To download all files from a SharePoint Online document library with the web browser, follow these steps:
- Navigate to your SharePoint document library >> Select all Files and folders.
- Click on the Download button from the toolbar. This provides you with a Zip file with all files and folders from the document library. Similarly, you can download folders as well.
Please note that this feature is available only with modern SharePoint Online UI, and these methods don’t download any document metadata. You need at least the View Only permission level to download documents from SharePoint. Also, it’s worth mentioning if certain file types, such as PDF files are protected by Information Rights Management (IRM), you may not be able to download documents.
Alright. Let’s download documents from the SharePoint Online document library using PowerShell.
You can also use OneDrive sync client to download a document library to your local machine! Just Go to the document library or folder you want to download >> Click on the sync button in the toolbar and follow the wizard!
SharePoint Online: Download All Files from a Library using PowerShell CSOM
We can use PowerShell to download a document library in SharePoint Online. But why PowerShell? Maybe because Explorer view didn’t work, or you want to automate the process! Here is the SharePoint Online PowerShell to download the document library:
#Load SharePoint CSOM Assemblies
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"
Function Download-FilesFromLibrary()
{
param
(
[Parameter(Mandatory=$true)] [string] $SiteURL,
[Parameter(Mandatory=$true)] [string] $LibraryName,
[Parameter(Mandatory=$true)] [string] $TargetFolder
)
Try {
#Setup Credentials to connect
$Cred= Get-Credential
$Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($Cred.Username, $Cred.Password)
#Setup the context
$Ctx = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
$Ctx.Credentials = $Credentials
#Get all files from the Library
$List = $Ctx.Web.Lists.GetByTitle($LibraryName)
$Ctx.Load($List)
$Ctx.ExecuteQuery()
#Get Files from the Folder
$Folder = $List.RootFolder
$FilesColl = $Folder.Files
$Ctx.Load($FilesColl)
$Ctx.ExecuteQuery()
Foreach($File in $FilesColl)
{
$TargetFile = $TargetFolder+$File.Name
#Download the file
$FileInfo = [Microsoft.SharePoint.Client.File]::OpenBinaryDirect($Ctx,$File.ServerRelativeURL)
$WriteStream = [System.IO.File]::Open($TargetFile,[System.IO.FileMode]::Create)
$FileInfo.Stream.CopyTo($WriteStream)
$WriteStream.Close()
}
Write-host -f Green "All Files from Library '$LibraryName' Downloaded to Folder '$TargetFolder' Successfully!" $_.Exception.Message
}
Catch {
write-host -f Red "Error Downloading Files from Library!" $_.Exception.Message
}
}
#Set parameter values
$SiteURL="https://crescent.sharepoint.com/sites/sales/"
$LibraryName="Documents"
$TargetFolder="C:\Documents\"
#Call the function to download file
Download-FilesFromLibrary -SiteURL $SiteURL -LibraryName $LibraryName -TargetFolder $TargetFolder
This PowerShell script downloads all files from the library. What if your library has Sub-Folders? Well, let’s change the PowerShell to download a library from SharePoint Online.
Download All Files including Sub-Folders from Library using PowerShell:
Let’s download files from the SharePoint Online document library using PowerShell, including all files from all sub-folders.
#Load SharePoint CSOM Assemblies
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"
Function Download-AllFilesFromLibrary()
{
param
(
[Parameter(Mandatory=$true)] [string] $SiteURL,
[Parameter(Mandatory=$true)] [Microsoft.SharePoint.Client.Folder] $SourceFolder,
[Parameter(Mandatory=$true)] [string] $TargetFolder
)
Try {
#Create Local Folder, if it doesn't exist
$FolderName = ($SourceFolder.ServerRelativeURL) -replace "/","\"
$LocalFolder = $TargetFolder + $FolderName
If (!(Test-Path -Path $LocalFolder)) {
New-Item -ItemType Directory -Path $LocalFolder | Out-Null
}
#Get all Files from the folder
$FilesColl = $SourceFolder.Files
$Ctx.Load($FilesColl)
$Ctx.ExecuteQuery()
#Iterate through each file and download
Foreach($File in $FilesColl)
{
$TargetFile = $LocalFolder+"\"+$File.Name
#Download the file
$FileInfo = [Microsoft.SharePoint.Client.File]::OpenBinaryDirect($Ctx,$File.ServerRelativeURL)
$WriteStream = [System.IO.File]::Open($TargetFile,[System.IO.FileMode]::Create)
$FileInfo.Stream.CopyTo($WriteStream)
$WriteStream.Close()
write-host -f Green "Downloaded File:"$TargetFile
}
#Process Sub Folders
$SubFolders = $SourceFolder.Folders
$Ctx.Load($SubFolders)
$Ctx.ExecuteQuery()
Foreach($Folder in $SubFolders)
{
If($Folder.Name -ne "Forms")
{
#Call the function recursively
Download-AllFilesFromLibrary -SiteURL $SiteURL -SourceFolder $Folder -TargetFolder $TargetFolder
}
}
}
Catch {
write-host -f Red "Error Downloading Files from Library!" $_.Exception.Message
}
}
#Set parameter values
$SiteURL="https://crescent.sharepoint.com/sites/sales/"
$LibraryName="Documents"
$TargetFolder="C:\Docs"
#Setup Credentials to connect
$Cred= Get-Credential
$Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($Cred.Username, $Cred.Password)
#Setup the context
$Ctx = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
$Ctx.Credentials = $Credentials
#Get the Library
$List = $Ctx.Web.Lists.GetByTitle($LibraryName)
$Ctx.Load($List)
$Ctx.Load($List.RootFolder)
$Ctx.ExecuteQuery()
#Call the function: sharepoint online download multiple files powershell
Download-AllFilesFromLibrary -SiteURL $SiteURL -SourceFolder $List.RootFolder -TargetFolder $TargetFolder
PnP PowerShell to Download a Document Library from SharePoint Online
This PowerShell script downloads all files and folders from the document library to the local directory.
#Function to Download All Files from a SharePoint Online Folder - Recursively
Function Download-SPOFolder([Microsoft.SharePoint.Client.Folder]$Folder, $DestinationFolder)
{
#Get the Folder's Site Relative URL
$FolderURL = $Folder.ServerRelativeUrl.Substring($Folder.Context.Web.ServerRelativeUrl.Length)
$LocalFolder = $DestinationFolder + ($FolderURL -replace "/","\")
#Create Local Folder, if it doesn't exist
If (!(Test-Path -Path $LocalFolder)) {
New-Item -ItemType Directory -Path $LocalFolder | Out-Null
Write-host -f Yellow "Created a New Folder '$LocalFolder'"
}
#Get all Files from the folder
$FilesColl = Get-PnPFolderItem -FolderSiteRelativeUrl $FolderURL -ItemType File
#Iterate through each file and download
Foreach($File in $FilesColl)
{
Get-PnPFile -ServerRelativeUrl $File.ServerRelativeUrl -Path $LocalFolder -FileName $File.Name -AsFile -force
Write-host -f Green "`tDownloaded File from '$($File.ServerRelativeUrl)'"
}
#Get Subfolders of the Folder and call the function recursively
$SubFolders = Get-PnPFolderItem -FolderSiteRelativeUrl $FolderURL -ItemType Folder
Foreach ($Folder in $SubFolders | Where {$_.Name -ne "Forms"})
{
Download-SPOFolder $Folder $DestinationFolder
}
}
#Set Parameters
$SiteURL = "https://crescent.sharepoint.com"
$LibraryURL = "/Shared Documents" #Site Relative URL
$DownloadPath ="C:\Temp\Docs\"
#Connect to PnP Online
Connect-PnPOnline -Url $SiteURL -Credentials (Get-Credential)
#Get The Root folder of the Library
$Folder = Get-PnPFolder -Url $LibraryURL
#Call the function to download the document library
Download-SPOFolder $Folder $DownloadPath
PnP PowerShell to Download a Larger Document Library
As the Get-PnPFolderItem cmdlet can’t handle libraries with more than 5000 items as of today and ends up with the “The attempted operation is prohibited because it exceeds the list view threshold.” error – here is the alternative approach to download all files and folders from a larger document library.
#Parameters
$SiteURL = "https://crescent.sharepoint.com/sites/marketing"
$ListName = "Branding"
$DownloadPath ="C:\Temp\NewDocs"
#Connect to SharePoint Online
Connect-PnPOnline $SiteURL -Interactive
$Web = Get-PnPWeb
#Get the list
$List = Get-PnPList -Identity $ListName
#Get all Items from the Library - with progress bar
$global:counter = 0
$ListItems = Get-PnPListItem -List $ListName -PageSize 500 -Fields ID -ScriptBlock { Param($items) $global:counter += $items.Count; Write-Progress -PercentComplete `
($global:Counter / ($List.ItemCount) * 100) -Activity "Getting Items from List:" -Status "Processing Items $global:Counter to $($List.ItemCount)";}
Write-Progress -Activity "Completed Retrieving Folders from List $ListName" -Completed
#Get all Subfolders of the library
$SubFolders = $ListItems | Where {$_.FileSystemObjectType -eq "Folder" -and $_.FieldValues.FileLeafRef -ne "Forms"}
$SubFolders | ForEach-Object {
#Ensure All Folders in the Local Path
$LocalFolder = $DownloadPath + ($_.FieldValues.FileRef.Substring($Web.ServerRelativeUrl.Length)) -replace "/","\"
#Create Local Folder, if it doesn't exist
If (!(Test-Path -Path $LocalFolder)) {
New-Item -ItemType Directory -Path $LocalFolder | Out-Null
}
Write-host -f Yellow "Ensured Folder '$LocalFolder'"
}
#Get all Files from the folder
$FilesColl = $ListItems | Where {$_.FileSystemObjectType -eq "File"}
#Iterate through each file and download
$FilesColl | ForEach-Object {
$FileDownloadPath = ($DownloadPath + ($_.FieldValues.FileRef.Substring($Web.ServerRelativeUrl.Length)) -replace "/","\").Replace($_.FieldValues.FileLeafRef,'')
Get-PnPFile -ServerRelativeUrl $_.FieldValues.FileRef -Path $FileDownloadPath -FileName $_.FieldValues.FileLeafRef -AsFile -force
Write-host -f Green "Downloaded File from '$($_.FieldValues.FileRef)'"
}
Alternate Method to download a document library from SharePoint Online
To copy files from SharePoint online to a local folder, use this PowerShell script:
#Set Parameters
$TenantSiteURL = "https://crescent.sharepoint.com"
$SiteRelativeURL = "/sites/marketing"
$LibraryName = "Project Documents"
$DownloadPath ="C:\Temp\Docs"
#Connect to PnP Online as Virtual Drive "SPO:\"
Connect-PnPOnline -Url $TenantSiteURL -CreateDrive -Credentials (Get-Credential)
#Change the Path and navigate to the source site
Set-Location -Path SPO:\$SiteRelativeURL
#Download Document Library to Local Drive
Copy-PnpItemProxy -Recurse -Force $LibraryName $DownloadPath
Conclusion
In conclusion, downloading a document library from SharePoint Online is a straightforward process that can be accomplished using a variety of methods. Whether you are a seasoned SharePoint professional or a novice user, you can choose the native SharePoint web browser interface, Explorer view, or a PowerShell script. By following the steps outlined in this guide, you can easily download a document library from SharePoint Online. This will ensure that your important files are backed up and accessible when you need them.
Can I download all files from all libraries on a site? Sure, You can either iterate through all document libraries and call the function or use this method: How to Download All Files from SharePoint Online Site using PowerShell?
Use the “Map Network Drive” or “File Explorer” features to download attachments from SharePoint List. You can also use PowerShell to download all attachments from SharePoint Online lists.
More info: Download attachments from SharePoint Online list
Grant users “Restricted View” permissions to block the download option from the SharePoint Online document library.
More info: Block Download Option for Documents in SharePoint Online
You can use the “Copy To” feature in SharePoint Online to copy all files from one document library to another. PowerShell can also be utilized to copy a library between SharePoint Online sites.
More info: SharePoint Online copy document library to another site
You can save the source document library as a template and Create a new document library from the saved template to make a copy of a document library. And then copy all files and folders from the source to the destination library if needed. PowerShell scripts can help you to automate these tasks.
More info: SharePoint Online Copy a Document library
To download a folder from SharePoint Online, navigate to the folder you want to download and click on the “Download” button in the toolbar. This will download the entire folder and its contents as a compressed ZIP file. You can then extract the files from the ZIP file on your local computer. Downloading a Folder from SharePoint Online with PowerShell is also possible! How to Download a Folder from SharePoint Online?
Works great. For those who may have sharepoint libraries that have really long paths (255+ chars), you should enable long paths in Windows first. See: https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry
I want to download a site collection, but without the folder named as site collection. How can I do this?
How to create a User form Module to get the User Input Info to run the script.
Basically, I am Technical Background User, Just Based on your Blog, I did this Code and working fine. But the additional requirement to create User Form for Update Info. Kindly Help me……..
#Function to Download All Files from a SharePoint Online Folder – Recursively
Function Download-SPOFolder([Microsoft.SharePoint.Client.Folder]$Folder, $DestinationFolder)
{
#Get the Folder’s Site Relative URL
$FolderURL = $Folder.ServerRelativeUrl.Substring($Folder.Context.Web.ServerRelativeUrl.Length)
$LocalFolder = $DestinationFolder + ($FolderURL -replace “/”,”\”)
#Get all Files from the folder
$FilesColl = Get-PnPFolderItem -FolderSiteRelativeUrl $FolderURL -ItemType File
#If current web folder contains files then create local folder
if($FilesColl)
{
#Create Local Folder, if it doesn’t exist
If (!(Test-Path -Path $LocalFolder)) {
New-Item -ItemType Directory -Path $LocalFolder | Out-Null
Write-host -f Yellow “Created a New Folder ‘$LocalFolder'”
}
}
#Iterate through each file and download
Foreach($File in $FilesColl)
{
Get-PnPFile -ServerRelativeUrl $File.ServerRelativeUrl -Path $LocalFolder -FileName $File.Name -AsFile -force
Write-host -f Green “`tDownloaded File from ‘$($File.ServerRelativeUrl)'”
}
#Get Subfolders of the Folder and call the function recursively
$SubFolders = Get-PnPFolderItem -FolderSiteRelativeUrl $FolderURL -ItemType Folder
Foreach ($Folder in $SubFolders | Where {$_.Name -ne “Forms”})
{
Download-SPOFolder $Folder $DestinationFolder
}
}
#Set Parameters
$SiteURL = “https://XXXXXXXX365.sharepoint.com/sites/arp_AED”
$LibraryURL = “/sites/arp_gl/Shared Documents/ARP_GL_Attachments/FY2023/Mukachevo_Parent” #Folder Relative URL
$DownloadPath =”C:\BCP Activity\ARP_Download_Files\GL”
#Set user name and password to connect
#$userName = “gunasekaran.Peter@XXCD.com”
#$password = “Xafddsafsadf@334345”
$securePassword = ConvertTo-SecureString -String $password -AsPlainText -Force
$Credential = New-Object -TypeName System.Management.Automation.PSCredential -argumentlist $userName, $securePassword
#Connect to PnP Online
Connect-PnPOnline -Url $SiteURL -Credential $Credential
#Get The Root folder of the Library
$Folder = Get-PnPFolder -Url $LibraryURL
#Call the function to download the document library
Download-SPOFolder $Folder $DownloadPath
Write-Host -f Yellow “All the documents from the folder has been downloaded in your Target Folder 1..😊😊😊!”
I am using the PnP PowerShell script to Download a Larger Document Library and i need to capture errors (i.e. the files that were skipped for whatever reason). I put a try catch in the script, however, it only prints line 38 (Write-host -f Green “Downloaded File from ‘$($_.FieldValues.FileRef)'”).
First of all I must thank you about these great scripts you have shared!
I do get error when running the script and all the files do not get downloaded.
Get-PnPFile : One or more errors occurred.
At F:\SPOLibrary-Library2-Copy.ps1:41 char:9
+ Get-PnPFile -ServerRelativeUrl $_.FieldValues.FileRef -Path $ …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : WriteError: (:) [Get-PnPFile], AggregateException
+ FullyQualifiedErrorId : EXCEPTION,PnP.PowerShell.Commands.Files.GetFile
Line 41:
Get-PnPFile -ServerRelativeUrl $_.FieldValues.FileRef -Path $FileDownloadPath -FileName $_.FieldValues.FileLeafRef -AsFile force
Using the PnP script, is there a way to only copy files that don’t exist or files that have been modified since the last time the script has been run?
how to modify script “PnP PowerShell to Download a Document Library from SharePoint Online” so that if the file has already been copied in the local forder from a previous run, to skip it , rather than re-download it ?
Replace with the below (from Line#34)
#Iterate through each file and download $FilesColl | ForEach-Object { $FileDownloadPath = ($DownloadPath + ($_.FieldValues.FileRef.Substring($Web.ServerRelativeUrl.Length)) -replace "/","\").Replace($_.FieldValues.FileLeafRef,'') #Check File Exists $FilePath = Join-Path -Path $FileDownloadPath -ChildPath $_.FieldValues.FileLeafRef If (-not(Test-Path -Path $FilePath -PathType Leaf)) { Get-PnPFile -ServerRelativeUrl $_.FieldValues.FileRef -Path $FileDownloadPath -FileName $_.FieldValues.FileLeafRef -AsFile -force Write-host -f Green "Downloaded File from '$($_.FieldValues.FileRef)'" } Else { Write-host "File '$FilePath' Exists Already..." -f Yellow } }I got this error message when running the script.
Cannot convert argument “context”, with value: “Microsoft.SharePoint.Client.ClientContext”, for “OpenBinaryDirect” to type “Microsoft.SharePoint.Client.ClientContext”: “Cannot convert the
“Microsoft.SharePoint.Client.ClientContext” value of type “Microsoft.SharePoint.Client.ClientContext” to type “Microsoft.SharePoint.Client.ClientContext”.”
The authenticate method in the script doesn’t work for me, so I changed it to:
$authManager = New-Object OfficeDevPnP.Core.AuthenticationManager;
$context = $authManager.GetWebLoginClientContext($SiteURL);
Guess your accounts are MFA enabled! Use PnP PowerShell script that can natively handle two-factor enabled accounts.
Hi, I am trying to use the PNP method for large libraries, the item scan is running and completing (908847 items) but the next stages do not appear to do anything? Am I missing something?
Amazing! Thank you for updating the code. It took a week to download, but I got all I needed.
I then ran your PowerShell code that lists items in a document library, modified to exclude the “| Where {$_.FileSystemObjectType -eq “File”}” to confirm why certain items weren’t included. (i.e. had Invalid, Folder, Web Member names)
Please let me know how I can buy you a coffee!
https://www.sharepointdiary.com/2018/08/sharepoint-online-powershell-to-get-all-files-in-document-library.html
Great content! I had been searching for ages to find something like this.
Unfortunately for me, when downloading a document library with around 600,000 files it gave an error saying:
“Get-PnPFolderItem : The attempted operation is prohibited because it exceeds the list view threshold.”
I’ve been searching for weeks but can’t find a work around. Is this a hard limitation or is there some way to circumvent the limit?
Yes! This is because the Get-PnPFolderItem cmdlet doesn’t have the capacity to handle libraries with more than 5000 items as of today. Post has been updated with a PnP PowerShell script to handle these situations.
Thanks for sharing all this stuff. I make use of your script collection.
Just used the script:
PnP PowerShell to Download a Document Library from SharePoint Online
I needed the LastModified and TimeCreated File properties to be able to do some stuff with the files afterwards. Maybe this is useful for someone else also. Just add this to the shared script in Line 19:
$(get-item ($LocalFolder+”\”+$File.Name)).lastwritetime=$file.TimeLastModified
$(get-item ($LocalFolder+”\”+$File.Name)).CreationTime=$file.TimeCreated
Hey – thank you for this work.
I had one question on, how do we get the COM assemblies:
Add-Type -Path “C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll”
Add-Type -Path “C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll”
I get an error out of these.
Thank you
You have to Install the SharePoint Online PowerShell Module, and then you don’t have to reference these assemblies (Because, CSOM is a subset of SharePoint Online PowerShell Module).
Install-Module Microsoft.Online.SharePoint.PowerShell
Thanks for sharing this script. We have sub folders under each Library. Is there any way I can download all Libraries and sub folders under it ? My main SPO site name is https://xyz.sharepoint.com/ and I have multiple Libraries on this site. Each Libraries have sub folder and files under neath. Please advice.
You can either iterate through all document libraries and call the function or use this method: How to Download All Files from SharePoint Online Site using PowerShell?
Thank you! A Powershell newbie, I needed a script which would download all current files and their versions from SharePoint. Accessing the versions proved to be a challenge as various scripts I tried either came up with (404) Not Found or (403) Forbidden. I was able to piece together a function which I call from your script. Within your Foreach($File in $FilesColl) loop, I added GetFileVersions as the last statement. That calls the below function.
Function GetFileVersions
{
#Get versions of specific document from library
$fileVersions = $file.Versions;
$ctx.Load($fileVersions);
$ctx.ExecuteQuery();
$vcount = $fileversions.count;
If ($vcount -ne “0” ) {
# Download all versions of specific file as individual docs
$fname,$fext = $file.name.split(‘.’); # Separate filename and extension to different variables
foreach ($fversion in $fileVersions) {
$str = $fversion.OpenBinaryStream();
$ctx.ExecuteQuery();
$dtime = $fversion.created.ToString(“u”) -replace “:”,””; # Convert date/time created to formatted string and replace semicolons in the time with blank
$filename =$localfolder + “” + $fname + “~” + $dtime + “.” + $fext ; # Compose the filename as the target folder, file name + ~+ datetime created.fileextension (i.e. book~2019-02-12 153659Z.xlsx)
$fs = New-Object IO.FileStream $filename ,’Create’,’Write’,’Read’;
$str.Value.CopyTo($fs);
#To use the Write-Log function, see the script at https://gallery.technet.microsoft.com/scriptcenter/Write-Log-PowerShell-999c32d0
#$vmessage = $filename + “*” + $fversion.versionlabel; # Message to write to the log file
#Write-Log $vmessage;
$fs.Close();
}
if ($fs.isdisposable) {$fs.Dispose();}
}
}
Use this script to download all versions of a file: PowerShell to Download All Versions of a Document in SharePoint Online