SharePoint Online: Migrate a Folder with Sub-Folders and Files using PowerShell
Requirement: Migrate a folder with its sub-folders and files from the local disk to SharePoint Online using PowerShell.
PowerShell to copy a Folder with Its Sub-Folders and Files to SharePoint Online:
Migrating a folder (and all its sub-folders and files) from your local computer to SharePoint Online can be a daunting task. Fortunately, PowerShell can help make this process quick and easy. This article will show you how to copy a folder with sub-folders and files to SharePoint Online using PowerShell.
Here is the PnP PowerShell to copy all files and sub-folders from a given folder from a local drive (or network drive) to SharePoint Online:
#Function to Copy Multiple Files with Folder structure to SharePoint Online Document Library
Function Migrate-PnPFolderToSPO()
{
param
(
[Parameter(Mandatory=$true)] [string] $SiteURL,
[Parameter(Mandatory=$true)] [string] $SourceFolderPath,
[Parameter(Mandatory=$true)] [string] $LibraryName,
[Parameter(Mandatory=$true)] [string] $LogFile
)
Try {
Add-content $Logfile -value "`n---------------------- File Upload Script Started: $(Get-date -format 'dd/MM/yyy hh:mm:ss tt')-------------------"
#Connect to PnP Online
Connect-PnPOnline -Url $SiteURL -Interactive
#Get the Target Folder to Upload
$Web = Get-PnPWeb
$List = Get-PnPList $LibraryName -Includes RootFolder
$TargetFolder = $List.RootFolder
$TargetFolderSiteRelativeURL = $TargetFolder.ServerRelativeURL.Replace($Web.ServerRelativeUrl,[string]::Empty)
#Get All Items from the Source
$Source = Get-ChildItem -Path $SourceFolderPath -Recurse
$SourceItems = $Source | Select FullName, PSIsContainer, @{Label='TargetItemURL';Expression={$_.FullName.Replace($SourceFolderPath,$TargetFolderSiteRelativeURL).Replace("\","/")}}
Add-content $Logfile -value "Number of Items Found in the Source: $($SourceItems.Count)"
#Upload Source Items from Fileshare to Target SharePoint Online document library
$Counter = 1
$SourceItems | ForEach-Object {
#Calculate Target Folder URL
$TargetFolderURL = (Split-Path $_.TargetItemURL -Parent).Replace("\","/")
$ItemName = Split-Path $_.FullName -leaf
#Replace Invalid Characters
$ItemName = [RegEx]::Replace($ItemName, "[{0}]" -f ([RegEx]::Escape([String]'\*:<>?/\|')), '_')
#Display Progress bar
$Status = "uploading '" + $ItemName + "' to " + $TargetFolderURL +" ($($Counter) of $($SourceItems.Count))"
Write-Progress -Activity "Uploading ..." -Status $Status -PercentComplete (($Counter / $SourceItems.Count) * 100)
If($_.PSIsContainer)
{
#Ensure Folder
$Folder = Resolve-PnPFolder -SiteRelativePath ($TargetFolderURL+"/"+$ItemName)
Write-host "Ensured Folder '$($ItemName)' to Folder $TargetFolderURL"
Add-content $Logfile -value "Ensured Folder '$($ItemName)' to Folder $TargetFolderURL"
}
Else
{
#Upload File
If($TargetFolderURL.StartsWith("/")) {$TargetFolderURL = $TargetFolderURL.Remove(0,1) }
$File = Add-PnPFile -Path $_.FullName -Folder $TargetFolderURL
Write-host "Uploaded File '$($_.FullName)' to Folder $TargetFolderURL"
Add-content $Logfile -value "Uploaded File '$($_.FullName)' to Folder $TargetFolderURL"
}
$Counter++
}
}
Catch {
Write-host -f Red "Error:" $_.Exception.Message
Add-content $Logfile -value "Error:$($_.Exception.Message)"
}
Finally {
Add-content $Logfile -value "---------------------- File upload Script Completed: $(Get-date -format 'dd/MM/yyy hh:mm:ss tt')-----------------"
}
}
#Call the Function to Upload a Folder to SharePoint Online
Migrate-PnPFolderToSPO -SiteURL "https://crescent.sharepoint.com/sites/marketing" -SourceFolderPath "C:\Documents" -LibraryName "Migration" -LogFile "C:\Temp\Migration-LOG.log"
This script copies the contents of a folder to the SharePoint Online document library. It overwrites any existing file in the target SharePoint Online library and creates a folder if it doesn’t exist on the target site. We can also use this script to migrate files and folders from a network file share to the SharePoint Online document library!
In summary, using PowerShell to migrate a local folder with sub-folders and files to SharePoint Online provides a powerful and efficient solution for transferring data from a local system to the cloud. This method not only preserves the structure of the folder and its content, but it also allows for the automation of the migration process, making it quick and easy for organizations to move their data to the cloud.
Hello, thank you for your efforts and sharing this script!
Unfortunately during file migration when errors pop up e.g.: file is too big- these errors pop up in the console but are not saved in the log as if the catch does not capture the errors
Best regards
Add-PnPFile : Server relative urls must start with SPWeb.ServerRelativeUrl
At line:54 char:34
Well, This is because “Add-PnPFile” cmdlet in the new PnP.PowerShell only accepts the folder without the leading “/” character. E.g. Add-PnPFile -Path “C:\temp\sample.txt” -Folder “Branding/2020/New” will work. However,
Add-PnPFile -Path “C:\temp\sample.txt” -Folder “/Branding/2020/New” will fail!
So, I’ve updated the script now and verified it! Line#53
Hey,
I tried to use this script but I couldn’t upload .pbix files with it. All I have is subfolder structure and .pbix files in it and it is creating the folder structure in SharePoint Online document library but not copying the .pbix files over it.
Can you please help
no bigger than 250MB..
i read that “It overwrites any existing file in the target SharePoint Online library”
how can we check if a file/files already exist then it can move forward to next upload! like an incremental procedure?
Use:
$FileExists = Get-PnPFile -Url $FileServerRelativeURL -ErrorAction SilentlyContinue
If(!$FileExists)
{
#Upload File
}
This was very helpful for me. Many thanks!
hey great script ive been working on modifying this slightly i am trying to use this to move copy everything in a D: to a sharepoint site but when i change the -SourceFolderPath to D:* its throwing an error Add-PnPFile : Access denied. You do not have permission to perform this action or access this resource. but when i say do somthing like -sourcefolderpath D:Test it copys the contents fine any ideas what im doing wrong
That’s the default behavior! If you try to create any file in root of the drive, you’ll get access denied even from GUI. However, If you run the script as Administrator, you won’t face this issue.
great script! thanks for sharing…
but, what if i want to copy inside a subfolder(or more levels)..for example inside of “shared documents”
regards
nevermind….solved by adding:
$TargetFolderSiteRelativeURL = $TargetFolderSiteRelativeURL + “/TEST”
hello
great script! really…but, what if i want to copy to subfolders in “shared documents”……eg “shared fodersTEST”
regards