Copy Document Library Between Tenants in SharePoint Online using PowerShell
Requirement: Copy a document library with files and folders from one SharePoint Online tenant to another.
How to copy a SharePoint Online document library across tenants?
There are many paid 3rd party tools, like ShareGate, Metalogix, etc., to do this efficiently, and We can use PowerShell (with limitations) to do this migration. This Export Document Libraries.
Please note, there are limitations in this PowerShell method, like this script doesn’t copy:
- Version History
- Permissions
- Workflows
- Document Library Settings
- Limited metadata, etc.
From the given parameters, the Export script downloads either a single document library with their files and folders or all libraries from the source site along with their metadata field values “Created”, “Created By”, “Modified”, and “Modified By” to the local disk. And then, you can use the Import script to upload the document libraries to the target site.
Export Document Library in SharePoint Online using PowerShell
Set the parameters in the #region Parameters section as per your environment and run the script. This will copy all files and folders and extract metadata to your local disk.
#Function to Extract Metadata of a File to CSV File
Function Extract-PnPFileMetadata([Microsoft.SharePoint.Client.File]$SPFile)
{
Try {
#Calculate URLs
$FileLibraryRelativeURL = $SPFile.ServerRelativeURL.Replace($global:Library.RootFolder.ServerRelativeURL,[string]::Empty)
#Calculate Absolute URL
If($global:Web.ServerRelativeUrl -eq "/")
{
$FileAbsoluteURL= $("{0}{1}" -f $global:Web.Url, $SPFile.ServerRelativeUrl)
}
Else
{
$FileAbsoluteURL= $("{0}{1}" -f $global:Web.Url.Replace($global:Web.ServerRelativeUrl,[string]::Empty), $SPFile.ServerRelativeUrl)
}
#Get Autor, Editor of the file
Get-PnPProperty -ClientObject $SPFile -Property Author, ModifiedBy
#Extract the Metadata of the file
$Metadata = New-Object PSObject
$Metadata | Add-Member -MemberType NoteProperty -name "FileName" -value $SPFile.Name
$Metadata | Add-Member -MemberType NoteProperty -name "ParentLibrary" -value $global:Library.Title
$Metadata | Add-Member -MemberType NoteProperty -name "FileAbsoluteURL" -value $FileAbsoluteURL
$Metadata | Add-Member -MemberType NoteProperty -name "FileLibraryRelativeURL" -value $FileLibraryRelativeURL
$Metadata | Add-Member -MemberType NoteProperty -Name "CreatedBy" -value $SPFile.Author.LoginName
$Metadata | Add-Member -MemberType NoteProperty -name "ModifiedBy" -value $SPFile.ModifiedBy.LoginName
$Metadata | Add-Member -MemberType NoteProperty -name "CreatedOn" -value $SPFile.TimeCreated
$Metadata | Add-Member -MemberType NoteProperty -name "ModifiedOn" -value $SPFile.TimeLastModified
#Send the Metadata to CSV File
$Metadata | Export-Csv $global:MetadataFile -NoTypeInformation -Append
Write-host -f Green "`t`tMetadata Extracted from the File:"$SPFile.ServerRelativeURL
}
Catch {
write-host -f Red "Error Getting Metadata:" $_.Exception.Message
}
}
#Function to Download a Document Library from SharePoint Online site
Function Export-PnPLibrary()
{
param
(
[Parameter(Mandatory=$true)] [string]$LibraryName
)
Try {
Write-host "Exporting Library:"$LibraryName -f Yellow
#Get the Library
$global:Library = Get-PnPList -Identity $LibraryName -Includes RootFolder
#Create a Local Folder for Document Library, if it doesn't exist
$LocalFolder = $global:DownloadPath +"\" +$global:Library.RootFolder.Name
If (!(Test-Path -Path $LocalFolder)) {
New-Item -ItemType Directory -Path $LocalFolder | Out-Null
}
Write-host -f Yellow "`tEnsured Folder for Document Library '$LocalFolder'"
#Get all Items from the Library - with progress bar
$global:counter = 0
$LibraryItems = Get-PnPListItem -List $LibraryName -PageSize 500 -Fields ID,FieldValues -ScriptBlock { Param($items) $global:counter += $items.Count; Write-Progress -PercentComplete `
($global:Counter / ($global:Library.ItemCount) * 100) -Activity "Getting Items from Library:" -Status "Processing Items $global:Counter to $($global:Library.ItemCount)";}
Write-Progress -Activity "Completed Retrieving Folders from Library $LibraryName" -Completed
#Get all Subfolders of the library
$SubFolders = $LibraryItems | Where {$_.FileSystemObjectType -eq "Folder" -and $_.FieldValues.FileLeafRef -ne "Forms"}
$SubFolders | ForEach-Object {
#Ensure All Folders in the Local Path
$LocalFolder = $global:DownloadPath + ($_.FieldValues.FileRef.Substring($global:Web.ServerRelativeUrl.Length)) -replace "/","\"
#Create Local Folder, if it doesn't exist
If (!(Test-Path -Path $LocalFolder)) {
New-Item -ItemType Directory -Path $LocalFolder | Out-Null
}
Write-host -f Yellow "`tEnsured Folder '$LocalFolder'"
}
#Get all Files from the folder
$FilesColl = $LibraryItems | Where {$_.FileSystemObjectType -eq "File"}
$global:Filess = $FilesColl
#Iterate through each file and download
$FilesColl | ForEach-Object {
Try {
$FileDownloadPath = ($global:DownloadPath + ($_.FieldValues.FileRef.Substring($global:Web.ServerRelativeUrl.Length)) -replace "/","\").Replace($_.FieldValues.FileLeafRef,[string]::Empty)
Get-PnPFile -Url $_.FieldValues.FileRef -Path $FileDownloadPath -FileName $_.FieldValues.FileLeafRef -AsFile -Force -ErrorAction Stop
Write-host -f Green "`tDownloaded File from '$($_.FieldValues.FileRef)'"
#Get the Metadata of the File
$File = Get-PnPProperty -ClientObject $_ -Property File
Extract-PnPFileMetadata -SPFile $File
}
Catch {
write-host -f Red "`tError Downloading File from '$($_.FieldValues.FileRef)' : "$_.Exception.Message
}
}
}
Catch {
write-host -f Red "`tError:" $_.Exception.Message
}
}
#Function to export all libraries in a SharePoint Site
Function Export-PnPLibraries()
{
Try {
#Arry to Skip System Lists and Libraries
$SystemLists =@("Converted Forms", "Master Page Gallery", "Customized Reports", "Form Templates", "List Template Gallery", "Theme Gallery",
"Reporting Templates", "Solution Gallery", "Style Library", "Web Part Gallery","Site Assets", "wfpub", "Site Pages", "Images")
#Filter Document Libraries to Scan
$LibraryCollection = Get-PnPList | Where {$_.BaseType -eq "DocumentLibrary" -and $_.Hidden -eq $false -and $SystemLists -notcontains $_.Title -and $_.ItemCount -gt 0}
#Loop through each document library
ForEach($Library in $LibraryCollection)
{
#Call the function to download the document library
Export-PnPLibrary -LibraryName $Library.Title
}
}
Catch {
Write-host -f Red "Error Downloading Libraries:" $_.Exception.Message
}
}
#region Parameters
$SourceSiteURL = "https://crescent.sharepoint.com/sites/ops"
$LibraryName = "Branding"
$global:DownloadPath= "C:\Temp\Migration"
#endregion Parameters
#Connect to SharePoint Online
Connect-PnPOnline $SourceSiteURL -Interactive
$global:Web = Get-PnPWeb
$global:MetadataFile = "$global:DownloadPath\Metadata.csv"
#Delete any existing files and folders in the download location
If (Test-Path $global:DownloadPath) {Get-ChildItem -Path $global:DownloadPath -Recurse| ForEach-object {Remove-item -Recurse -path $_.FullName }}
#Call the function to download a library
Export-PnPLibrary -LibraryName $LibraryName
#To Export all libraries, use:
#Export-PnPLibraries
Import Document Libraries to SharePoint Online
Once exported, you can import the document Libraries to a different tenant with the below PowerShell script. Just make sure the user IDs are valid in the “Metadata.csv” file from the above script. If needed, you can replace the IDs to match the destination tenant.
#Function to Ensure SharePoint Online User
Function Ensure-PnPUser([string]$UserID)
{
Try {
#Try to Get the User
$User = Get-PnPUser -Identity $UserID
If($User -eq $null) {
$User = New-PnPUser -LoginName $UserID
}
#Return the User Object
$User
}
Catch {
write-host -f Red "`t`t`tError Resolving User $UserID :" $_.Exception.Message
Return $Null
}
}
#Function to Set the Metadata of a Document
Function SetPnP-DocumentMetadata()
{
param
(
[Parameter(Mandatory=$true)] [Microsoft.SharePoint.Client.File] $File,
[Parameter(Mandatory=$true)] [Microsoft.SharePoint.Client.List] $TargetLibrary
)
Try {
#Calculate the Library Relative URL of the File
$TargetFolder = Get-PnPProperty -ClientObject $TargetLibrary -Property RootFolder
$FileLibraryRelativeURL = $File.ServerRelativeUrl.Replace($TargetLibrary.RootFolder.ServerRelativeUrl,[string]::Empty)
$FileItem = Get-PnPProperty -ClientObject $File -Property ListItemAllFields
#Import Metadata CSV File
$MetadataFile = Import-Csv -LiteralPath $global:MetadataFile
#Get the Metadata of the File
$Metadata = $MetadataFile | Where-Object {($_.ParentLibrary -eq ($TargetLibrary.Title)) -and $_.FileLibraryRelativeURL -eq $FileLibraryRelativeURL}
If($Metadata)
{
Write-host -f Yellow "`t`tUpdating Metadata for File '$($File.ServerRelativeURL)'"
#Get 'Created By' and 'Modified By' Users
$FileMetadata = @{}
$Author = Ensure-PnPUser -UserID $Metadata.CreatedBy
$Editor = Ensure-PnPUser -UserID $Metadata.ModifiedBy
$FileMetadata.add("Created",[DateTime]$Metadata.CreatedOn)
$FileMetadata.add("Modified",[DateTime]$Metadata.ModifiedOn)
If($Author -ne $Null)
{
$FileMetadata.add("Author", $Author.LoginName)
}
If($Editor -ne $Null)
{
$FileMetadata.add("Editor", $Editor.LoginName)
}
#Update document properties
Set-PnPListItem -List $TargetLibrary -Identity $FileItem.Id -Values $FileMetadata | Out-Null
Write-host -f Green "`t`t`tMetadata has been Updated Successfully!"
}
}
Catch {
write-host -f Red "`t`t`tError updating Metadata of the Document:"$_.Exception.Message
}
}
#Function to Import all Files and Folders from Local Folder to SharePoint Online
Function ImportPnP-Library()
{
param
(
[Parameter(Mandatory=$true)] [string] $SourceLibraryPath,
[Parameter(Mandatory=$true)] [Microsoft.SharePoint.Client.List] $TargetLibrary
)
Try {
#Get the Target Folder to Upload
$TargetFolder = Get-PnPProperty -ClientObject $TargetLibrary -Property RootFolder
$TargetFolderSiteRelativeURL = $TargetFolder.ServerRelativeURL.Replace($global:Web.ServerRelativeUrl+"/",[string]::Empty)
Get-ChildItem $SourceLibraryPath -Recurse | ForEach-Object {
$TargetFolderRelativeURL = $TargetFolderSiteRelativeURL+$_.FullName.Replace($SourceLibraryPath,[string]::Empty).Replace("\","/")
#write-host $TargetFolderRelativeURL
If ($_.PSIsContainer -eq $True) #If its a Folder, ensure it!
{
Write-host -f Yellow "`t`tEnsuring Folder '$TargetFolderRelativeURL'"
#Ensure Folder
Resolve-PnPFolder -SiteRelativePath $TargetFolderRelativeURL | Out-Null
}
Else #Its a File, Upload it!
{
#Calculate the Parent Folder for File
$TargetFolderURL = (Split-Path $TargetFolderRelativeURL -Parent).Replace("\","/")
$SourceFilePath = $_.FullName
Write-host -f Yellow "`t`tUploading File '$_' to Folder:"$TargetFolderURL
$File = Add-PnPFile -Path $SourceFilePath -Folder $TargetFolderURL
Write-host "`t`t`tFile Uploaded Successfully!" -ForegroundColor Green
#Update Metadata of the File
SetPnP-DocumentMetadata -File $File -TargetLibrary $TargetLibrary
}
}
}
Catch {
write-host -f Red "`t`t`tError Importing Library:" $_.Exception.Message
}
}
#Function to Ensure a SharePoint Online document library
Function EnsurePnP-DocumentLibrary()
{
param
(
[Parameter(Mandatory=$true)] [string] $LibraryName
)
Try {
Write-host -f Yellow "`nEnsuring Library '$LibraryName'"
#Check if the Library exist already
$List = Get-PnPList | Where {$_.Title -eq $LibraryName}
If($List -eq $Null)
{
#Create Document Library
$List = New-PnPList -Title $LibraryName -Template DocumentLibrary -OnQuickLaunch
write-host -f Green "`tNew Document Library '$LibraryName' has been created!"
}
Else
{
#Get the Library
$List = Get-PnPList -Identity $LibraryName
Write-Host -f Magenta "`tA Document Library '$LibraryName' Already exist!"
}
Return $List
}
Catch {
write-host -f Red "`tError Creating Document Library!" $_.Exception.Message
}
}
#Main Function
Function Import-PnPLibraries()
{
Try {
#Get Top Level Folders from the Source as "Document Libraries"
$SourceLibraries = Get-ChildItem -Directory -Path $Global:SourcePath
#Create Document Libraries
ForEach($SourceLibrary in $SourceLibraries)
{
#call the function to Ensure document library
$TargetLibrary = EnsurePnP-DocumentLibrary -LibraryName $SourceLibrary.Name
#Import Files and Folders from the Source to the Destination
ImportPnP-Library -SourceLibraryPath $SourceLibrary.FullName -TargetLibrary $TargetLibrary
}
}
Catch {
write-host -f Red "Error:" $_.Exception.Message
}
}
#region Parameters
$Global:SourcePath = "C:\Temp\Migration"
$Global:TargetSiteURL = "https://national.sharepoint.com/Sites/Operations"
#endregion Parameters
#Connect to SharePoint Online
Connect-PnPOnline $TargetSiteURL -Interactive
$global:Web = Get-PnPWeb
$global:MetadataFile = "$global:SourcePath\Metadata.csv"
#Import a Single Document Library
$Library = EnsurePnP-DocumentLibrary "Mason"
ImportPnP-Library -SourceLibraryPath "$Global:SourcePath\Mason" -TargetLibrary $Library
#Call the function to import all document libraries from the source path
#Import-PnPLibraries
You can use this PowerShell script to back up a document library in SharePoint Online as well.
Related posts:
- To copy a document library in SharePoint Online: How to Copy a document library in SharePoint Online?
- To copy a document library from one site to another in SharePoint Online using PowerShell, use: SharePoint Online: How to copy a document library to another site?
This Script is not copying the custom column and column formatting from source tenant to destination tenant.
Please provide a solution for this.
how can I copy the custom value .could you please provide me some solution.
This Script is not copying the custom column and column formatting from source tenant to destination tenant.
Please provide a solution for this.
I would like to know how can i export the metadata of custom site column, since i have configured all list from a site with several columns and i’d like to save it on the excel file.
Hi Salaudeen
Would like to thank you for the great method. I tried both scripts and the results as expected for copying the files+metadata but when it comes to Folders it will create the folder without the metadata “CreatedBy” and “Created”. can you help us to update the script to solve this issue.
Thanks in advanced.
Best Regards Osama