Document Library Migration/Upgrade PowerShell SharePoint SharePoint 2010 SharePoint 2013 SharePoint 2016 SharePoint Online 

Migrate SharePoint 2010 Document Libraries to SharePoint Online using PowerShell

Salaudeen Rajack 6 Comments

Requirement: Migrate Document Libraries from SharePoint 2010 to SharePoint Online.

How to Migrate SharePoint 2010 Document Libraries to SharePoint Online?

There are several methods that can be used to migrate a document library from SharePoint 2010 to SharePoint Online, including the SharePoint Migration Tool (SPMT), 3rd party Migration tools like ShareGate, etc. Here is one that uses PowerShell! The idea is to export all document libraries from SharePoint On-premises to a local drive, export the metadata of each document to a CSV file, and then import them to SharePoint Online.

Before we begin, please note, this is not a full-fledged migration script that can migrate metadata, security, version history, etc. This script only copies document libraries with its file folders and metadata “Created by”, “Created On”, “Modified By”, and “Modified On” to SharePoint Online. You need 3rd party migration tools if you want to migrate all the missing pieces!

Step 1: Export SharePoint On-Premises Document Library

As a first step, Login to any of your SharePoint On-premises web front-end servers and run the below PowerShell script to export either a single document library or all document libraries from SharePoint 2010 to the local disk of the server. Change the Run-time variables at the bottom of the script according to your environment. Please note, this script exports libraries of a given site (not site collection), you can extend it if needed!

#region Runtime-Variables
$Global:SourceSiteURL = “https://SP2010.crescent.com/sites/marketing”
$Global:LocalFolderPath =“C:\Migration\Downloads”
#endregion Runtime-Variables

PowerShell to Export All Document Libraries from SharePoint 2010

This script Downloads On-premises Document Libraries from SharePoint 2010 to local disk.

Add-PSSnapin Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue

#Function to Extract Metadata of a File to CSV File
Function Extract-SPFileMetadata([Microsoft.SharePoint.SPFile]$SPFile)
{
    Try {
        #Calculate URLs
        $FileLibraryRelativeURL = $SPFile.ServerRelativeURL.Replace($SPFile.DocumentLibrary.RootFolder.ServerRelativeUrl,[string]::Empty)
        #Calculate Absolute URL
        If($global:Web.ServerRelativeUrl -eq "/")
        {
            $FileAbsoluteURL=  $("{0}{1}" -f $global:Web.Url, $SPFile.ServerRelativeUrl)
        }
        else
        {
            $FileAbsoluteURL=  $("{0}{1}" -f $global:Web.Url.Replace($global:Web.ServerRelativeUrl,[string]::Empty), $SPFile.ServerRelativeUrl)
        } 

        #Extract the Metadata of the file
        $Metadata = New-Object PSObject 
        $Metadata | Add-Member -MemberType NoteProperty -name "FileName" -value $SPFile.Name
        $Metadata | Add-Member -MemberType NoteProperty -name "ParentLibrary" -value $SPFile.DocumentLibrary.Title
        $Metadata | Add-Member -MemberType NoteProperty -name "FileAbsoluteURL" -value $FileAbsoluteURL
        $Metadata | Add-Member -MemberType NoteProperty -name "FileLibraryRelativeURL" -value $FileLibraryRelativeURL
        $Metadata | Add-Member -MemberType NoteProperty -Name "CreatedBy" -value $SPFile.Author.Email
        $Metadata | Add-Member -MemberType NoteProperty -name "ModifiedBy" -value $SPFile.ModifiedBy.Email
        $Metadata | Add-Member -MemberType NoteProperty -name "CreatedOn" -value $SPFile.TimeCreated 
        $Metadata | Add-Member -MemberType NoteProperty -name "ModifiedOn" -value $SPFile.TimeLastModified

        #Send the Metadata to CSV File
        If (-not(Test-Path $global:MetadataFile))
        {
            $Metadata | Export-Csv $global:MetadataFile -NoTypeInformation
        }
        else
        {
            #Append data to Existing CSV File - Remove Duplicates
            $CSVData = [Object[]] (Import-Csv $global:MetadataFile)
            $CSVData + $Metadata | Sort FileAbsoluteURL -unique | Export-Csv $global:MetadataFile -NoTypeInformation
        }
        Write-host -f Green "`tMetadata Extracted from the File:"$SPFile.ServerRelativeURL
     }
    Catch {
        write-host -f Red "Error Getting Metadatas:" $_.Exception.Message
    }
}

#Function to Download All Files from a SharePoint Folder
Function Download-SPFolder($SPFolderURL, $LocalFolderPath)
{
    Try {
        #Get the Source SharePoint Folder
        $SPFolder = $global:web.GetFolder($SPFolderURL)
 
        $LocalFolderPath = Join-Path $LocalFolderPath $SPFolder.Name 
        #Ensure the destination local folder exists! 
        If (!(Test-Path -path $LocalFolderPath))
        {    
            #If it doesn't exist, Create
            $LocalFolder = New-Item $LocalFolderPath -type directory 
        }
 
        #Loop through each file in the folder and download it to Destination
        ForEach ($File in $SPFolder.Files) 
        {
            #Download the file
            $Data = $File.OpenBinary()
            $FilePath= Join-Path $LocalFolderPath $File.Name
            [System.IO.File]::WriteAllBytes($FilePath, $data)
            Write-host -f Green "`tDownloaded the File:"$File.ServerRelativeURL
         
            #Get the Metadata of the File
            Extract-SPFileMetadata -SPFile $File
        }
 
        #Process the Sub-Folders & Recursively call the function
        ForEach ($SubFolder in $SPFolder.SubFolders)
        {
            If($SubFolder.Name -ne "Forms") #Leave "Forms" Folder
            {
                #Call the function Recursively
                Download-SPFolder $SubFolder $LocalFolderPath
            }
        }
    }
    Catch {
        Write-host -f Red "Error Downloading Document Library:" $_.Exception.Message
    }  
}

#Function to export a Single Library
Function Export-SPLibrary()
{
    param
    (
        [Parameter(Mandatory=$true)] [string] $LibraryName
    )
    Try {
        #Get the Source Web
        $global:Web = Get-SPWeb $global:SourceSiteURL
        $global:MetadataFile = "$global:LocalFolderPath\Metadata.csv"

        #Get the library
        $Library = $Global:Web.Lists.TryGetList($LibraryName)

        #Get All Files from the Library and export
        If($Library -ne $Null)
        {
            Write-host -f magenta "Downloading Document Library:" $Library.Title

            #Call the function to download the document library
            Download-SPFolder -SPFolderURL $Library.RootFolder.Url -LocalFolderPath $global:LocalFolderPath
        }
        Else
        {
            Write-host -f Yellow "Could not Find Document Library:" $LibraryName
        }

        Write-host "*** Completed downloading Library '$LibraryName'  ***"
    }
    Catch {
        Write-host -f Red "Error Downloading Document Library:" $_.Exception.Message
    }  
}

#Function to export all libraries in a SharePoint Site
Function Export-SPAllLibraries()
{
    Try {
        #Get the Source Web
        $global:Web = Get-SPWeb $global:SourceSiteURL

        #Delete any existing files and folders in the download location
        If (Test-Path $global:LocalFolderPath) {Get-ChildItem -Path $global:LocalFolderPath -Recurse| ForEach-object {Remove-item -Recurse -path $_.FullName }}

        #Arry to Skip System Libraries
        $SystemLibraries =@("Pages", "Converted Forms", "Master Page Gallery", "Customized Reports", 
                 "Form Templates", "Images", "List Template Gallery", "Theme Gallery", "Reporting Templates", 
                          "Site Collection Documents", "Site Collection Images", "Site Pages", "Solution Gallery", 
                               "Style Library", "Web Part Gallery","Site Assets", "wfpub")

        #Get all document libraries - Exclude Hidden and System Libraries
        $LibraryCollection = $Web.lists | Where-Object  { ($_.BaseType -eq "DocumentLibrary") -and ($_.hidden -eq $false) -and ($SystemLibraries -notcontains $_.Title)}

        #Iterate through each list and export
        ForEach($Library in $LibraryCollection)
        {
            Write-host -f magenta "Downloading Document Library:" $Library.Title

            #Call the function to download the document library
            Download-SPFolder -SPFolderURL $Library.RootFolder.Url -LocalFolderPath $global:LocalFolderPath
        } 

        Write-host "*** Downloaded All Libraries From the Given Site! ***"
    }
    Catch {
        Write-host -f Red "Error Downloading All Document Libraries:" $_.Exception.Message
    }  
}

#region Runtime-Variables
$Global:SourceSiteURL = "https://SP2010.crescent.com/sites/marketing"
$Global:LocalFolderPath ="C:\Migration\Downloads"
#endregion Runtime-Variables

#Call the Function to export all document libraries from a site
Export-SPLibraries

#To Export a specific library, use: 
#Export-SPLibrary -LibraryName "Shared Documents"

Run the Script. Once executed, this script downloads all document libraries to the given LocalFolderPath. With this script, you can either download all document libraries or a specific document library from a SharePoint On-premises site.

Also, the script extracts the document’s metadata to a CSV file named “Metadata.csv” in the given $LocalFolderPath.

Update the Metadata If needed

The metadata.csv file has File’s metadata for fields such as “Created By”, “Modified By”, etc. Make sure the Email IDs listed in the CSV file are valid in the SharePoint Online environment. If needed, You can update this CSV file and save it.

Step 2. Import Document Libraries to SharePoint Online

Now, copy the downloaded folder (Here, it’s “C:\Migration\Downloads”) to your laptop or any machine where SharePoint Online Client SDK is installed, and the target SharePoint Online site is accessible. Copy the below script to PowerShell ISE and change the highlighted run time variables according to your environment. Ensure the site given at TargetSiteURL is already created, and then execute the script!

#region Runtime-Variables
$Global:SourcePath = “C:\Migration\Downloads”
$Global:TargetSiteURL = “https://crescent.sharepoint.com/Sites/Marketing”
#endregion Runtime-Variables

Upload SharePoint 2010 Document Libraries to SharePoint Online and Set Metadata

Let’s upload on-premises content to SharePoint Online using PowerShell cmdlets:

#Load SharePoint CSOM Assemblies
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"

#Function to Ensure SharePoint Online User
Function Ensure-SPOUser([string]$UserID)
{
    Try {
        $User = $Global:Web.EnsureUser($UserID)
        $Global:Ctx.Load($User)
        $Global:Ctx.ExecuteQuery()
        Return $User
    }
    Catch {
        write-host -f Red "`t`t`tError Resolving User $UserID :" $_.Exception.Message
        Return $Null
    }
}

#Function to Set the Metadata of a Document
Function SetSPO-DocumentMetadata()
{
    param
    (
        [Parameter(Mandatory=$true)] [Microsoft.SharePoint.Client.File] $File,
        [Parameter(Mandatory=$true)] [Microsoft.SharePoint.Client.List] $TargetLibrary
    )    
    Try {
        #Calculate the Library Relative URL of the File
        $Global:Ctx.Load($TargetLibrary.RootFolder)
        $Global:Ctx.ExecuteQuery()
        $FileLibraryRelativeURL = $File.ServerRelativeUrl.Replace($TargetLibrary.RootFolder.ServerRelativeUrl,[string]::Empty)

        #Import Metadata CSV File
        $MetadataCSVPath = "$Global:SourcePath\Metadata.csv"
        $MetadataFile = Import-Csv -LiteralPath $MetadataCSVPath 
        #Get the Metadata of the File
        $Metadata = $MetadataFile | Where-Object {($_.ParentLibrary -eq ($TargetLibrary.Title)) -and $_.FileLibraryRelativeURL -eq $FileLibraryRelativeURL}
        If($Metadata)
        {
            Write-host -f Yellow "`t`tUpdating Metadata for File '$($File.ServerRelativeURL)'"

            #Get 'Created By' and 'Modified By' Users
            $Author = Ensure-SPOUser -UserID $Metadata.CreatedBy
            $Editor= Ensure-SPOUser -UserID $Metadata.ModifiedBy

            #Set Metadata of the File
            $ListItem = $File.ListItemAllFields
            If($Author -ne $Null)
            {
                $Listitem["Author"] = $Author
            }
            If($Editor -ne $Null)
            {
                $ListItem["Editor"] = $Editor
            }
            $ListItem["Created"] =  $Metadata.CreatedOn
            $ListItem["Modified"] = $Metadata.ModifiedOn
            $ListItem.Update()
            $Global:Ctx.ExecuteQuery()

            Write-host -f Green "`t`t`tMetadata for '$($File.ServerRelativeURL)' has been Updated Successfully!"
        }
    }
    Catch {
        write-host -f Red "`t`t`tError updating Metadata of the Document:" $_.Exception.Message
    }
}

#Function to migrate all Files and Folders from Local Path to SharePoint Online
Function MigrateSPO-LocalFolder()
{ 
    param
    (
        [Parameter(Mandatory=$true)] [string] $SourceFolderPath,
        [Parameter(Mandatory=$true)] [Microsoft.SharePoint.Client.List] $TargetLibrary
    )
    Try {
        #Get the Target Folder to Upload
        $TargetFolder = $TargetLibrary.RootFolder
        $Global:Ctx.Load($TargetFolder)
        $Global:Ctx.ExecuteQuery()
 
        #Get All Files and Folders from the Source
        Get-ChildItem $SourceFolderPath -Recurse | ForEach-Object {
            If ($_.PSIsContainer -eq $True) #If its a Folder!
            {
                $TargetFolderRelativeURL = $TargetFolder.ServerRelativeURL+$_.FullName.Replace($SourceFolderPath,[string]::Empty).Replace("\","/")
                Write-host -f Yellow "`t`tEnsuring Folder '$TargetFolderRelativeURL'"
                     
                #Check If Folder Exists
                Try {
                    $Folder = $Global:Web.GetFolderByServerRelativeUrl($TargetFolderRelativeURL)
                    $Global:Ctx.Load($Folder)
                    $Global:Ctx.ExecuteQuery()
  
                    Write-host -f Magenta "`t`t`tFolder Already Exists: $TargetFolderRelativeURL"
                }
                Catch {
                    #Create New Sub-Folder
                    $Folder= $Global:Web.Folders.Add($TargetFolderRelativeURL)
                    $Global:Ctx.ExecuteQuery()
                    Write-host -f Green "`t`t`tCreated Folder at "$TargetFolderRelativeURL
                    $Global:Ctx.ExecuteQuery()
                }
            }
            Else #Its a File, Upload it!
            {
                #Calculate Source and Destination File Paths
                $TargetFileURL = $TargetFolder.ServerRelativeURL + $_.DirectoryName.Replace($SourceFolderPath,[string]::Empty).Replace("\","/") +"/"+$_.Name          
                $SourceFilePath = $_.FullName
 
                Write-host -f Yellow "`t`tUploading File '$_' to URL "$TargetFileURL
                #Get the file from disk
                $FileStream = ([System.IO.FileInfo] (Get-Item $SourceFilePath)).OpenRead()
    
                #Upload the File to SharePoint Library's Folder
                $FileCreationInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation
                $FileCreationInfo.Overwrite = $True
                $FileCreationInfo.ContentStream = $FileStream
                $FileCreationInfo.URL = $TargetFileURL
                $FileUploaded = $TargetFolder.Files.Add($FileCreationInfo)
                $Global:Ctx.ExecuteQuery()
                $FileStream.Close()                

                Write-host "`t`t`tFile '$TargetFileURL' Uploaded Successfully!" -ForegroundColor Green

                #get the uploaded file
                $File = $Global:Web.GetFileByServerRelativeUrl($TargetFileURL)
                $Global:Ctx.Load($File)
                $Global:Ctx.ExecuteQuery()

                #Update Metadata of the File
                SetSPO-DocumentMetadata -File $File -TargetLibrary $TargetLibrary
            }
        }
    }
    Catch {
        write-host -f Red "`t`t`tError Uploading File:" $_.Exception.Message
    }
}

#Function to Ensure a SharePoint Online document library
Function EnsureSPO-DocumentLibrary()
{
    param
    (
        [Parameter(Mandatory=$true)] [string] $DocumentLibraryName
    )    
    Try {
        Write-host -f Yellow "`nEnsuring Document Library '$DocumentLibraryName'"
        
        #Get All Existing Lists from the web
        $Lists = $Global:Web.Lists
        $Global:Ctx.Load($Lists)
        $Global:Ctx.ExecuteQuery()

        #Check if the Library name doesn't exist already
        If(!($Lists.Title -contains $DocumentLibraryName))
        {
            #Create Document Library
            $ListInfo = New-Object Microsoft.SharePoint.Client.ListCreationInformation
            $ListInfo.Title = $DocumentLibraryName
            $ListInfo.TemplateType = 101 #Document Library
            $List = $Global:Web.Lists.Add($ListInfo)
            $List.Update()
            $Global:Ctx.ExecuteQuery()
 
            write-host  -f Green "`tNew Document Library '$DocumentLibraryName' has been created!"
            Return $List
        }
        Else
        {
            #Get the Library
            $List = $Global:Web.Lists.GetByTitle($DocumentLibraryName)
            $Global:Ctx.Load($List)
            $Global:Ctx.ExecuteQuery()
            Return $List

            Write-Host -f Magenta "`tA Document Library '$DocumentLibraryName' Already exist!"
        }
    }
    Catch {
        write-host -f Red "`tError Creating Document Library!" $_.Exception.Message
    }
}

#Main Function
Function Import-SPOLibraries()
{
    Try {
        #Setup the context
        $Global:Ctx = New-Object Microsoft.SharePoint.Client.ClientContext($Global:TargetSiteURL)
        $Global:Ctx.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($Global:Cred.UserName,$Global:Cred.Password)
 
        #Get the Web
        $Global:Web = $Global:Ctx.Web
        $Global:Ctx.Load($Global:Web)
        $Global:Ctx.ExecuteQuery()

        #Get Top Level Folders from the Source as "Document Libraries"
        $SourceLibraries = Get-ChildItem -Directory -Path $Global:SourcePath

        #Create Document Libraries
        ForEach($SourceLibrary in $SourceLibraries)
        {
            #call the function to Ensure document library
            $TargetLibrary = EnsureSPO-DocumentLibrary -DocumentLibraryName $SourceLibrary.Name

            #Migrate Files and Folders from the Source to the Destination
            MigrateSPO-LocalFolder -SourceFolderPath $SourceLibrary.FullName -TargetLibrary $TargetLibrary
        }
        Write-host -f Cyan "`n*** Import Completed! ***`n"
    }
    Catch {
        write-host -f Red "Error:" $_.Exception.Message
    }
}

#region Runtime-Variables
$Global:SourcePath = "C:\Migration\Downloads"
$Global:TargetSiteURL = "https://crescent.sharepoint.com/Sites/Marketing"
#endregion Runtime-Variables

#Get Credentials to connect 
$Global:Cred = Get-Credential -Message "Enter the Admin Credentials:"

#Call the function to import document libraries and files to SharePoint Online
Import-SPOLibraries

This script is written to copy document libraries site-by-site and can be extended further!

Salaudeen Rajack

Salaudeen Rajack - SharePoint Expert with Two decades of SharePoint Experience. Love to Share my knowledge and experience with the SharePoint community, through real-time articles!

6 thoughts on “Migrate SharePoint 2010 Document Libraries to SharePoint Online using PowerShell”

  • How to get all versions of the file while exporting

    Reply

  • When I run this script it only capture default fields such as created , created. how to migrated other metadata of the file.
    Thanks,
    Mohandar

    Reply

  • Hi,
    Does this also takes care for (download and upload) different document versions from On-premise and upload to SharePoint Online?

    Thanks

    Reply

  • wow! this will be helpful and will help to lessen the costs for migration. hope there will be a script for SharePoint Online to SharePoint Online

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *