Migrate File Share to SharePoint Online using PowerShell

Requirement: Migrate All Files and Folders from a Network File Share to SharePoint Online.

PowerShell to Bulk Upload All Files from Network File Share to SharePoint Online
Lets migrate a file share to SharePoint Online using PowerShell. Here is my source:
copy file share with metadata to sharepoint online using powershell
Lets copy file share with metadata to SharePoint Online using PowerShell
#Load SharePoint CSOM Assemblies
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"

#Function to migrate all Files and Folders from FileShare to SharePoint Online
Function Migrate-FileShareToSPO()
{ 
    param
    (
        [Parameter(Mandatory=$true)] [string] $SiteURL,
        [Parameter(Mandatory=$true)] [string] $TargetLibraryName,
        [Parameter(Mandatory=$true)] [string] $SourceFolderPath
    )

    #Setup Credentials to connect
    $Cred= Get-Credential
    $Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($Cred.Username, $Cred.Password)
 
    #Setup the context
    $Ctx = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
    $Ctx.Credentials = $Credentials
      
    #Get the Target Folder to Upload
    $Web = $Ctx.Web
    $Ctx.Load($Web)
    $List = $Web.Lists.GetByTitle($TargetLibraryName)
    $Ctx.Load($List)
    $TargetFolder = $List.RootFolder
    $Ctx.Load($TargetFolder)
    $Ctx.ExecuteQuery() 

    #Get All Files and Folders from the Source
    Get-ChildItem $SourceFolderPath -Recurse | ForEach-Object {
        If ($_.PSIsContainer -eq $True) #If its a Folder!
        {
            $TargetFolderRelativeURL = $TargetFolder.ServerRelativeURL+$_.FullName.Replace($SourceFolderPath,"").Replace("\","/")
            Write-host -f Yellow "Ensuring Folder '$TargetFolderRelativeURL' Exists..."
                    
            #Check Folder Exists
            Try {
                #Get Source Folder's metadata
                $CreatedDate= [DateTime]$_.CreationTime
                $ModifiedDate =  [DateTime]$_.LastWriteTime

                $Folder = $Web.GetFolderByServerRelativeUrl($TargetFolderRelativeURL)
                $Ctx.Load($Folder)
                $Ctx.ExecuteQuery()
 
                #Write-host -f Green "Folder Already Exists!"
            }
            Catch {
                #Create New Sub-Folder
                $Folder=$Web.Folders.Add($TargetFolderRelativeURL)
                $Ctx.ExecuteQuery()
                Write-host -f Green "Created Folder at "$TargetFolderRelativeURL

                #Set Metadata of the Folder
                $FolderProperties = $Folder.ListItemAllFields
                $FolderProperties["Created"] = $CreatedDate
                $FolderProperties["Modified"] = $ModifiedDate
                $FolderProperties.Update()
                $Ctx.ExecuteQuery()
            }
        }
        Else #If its a File
        {
            $TargetFileURL = $TargetFolder.ServerRelativeURL + $_.DirectoryName.Replace($SourceFolderPath,"").Replace("\","/") +"/"+$_.Name          
            $SourceFilePath = $_.FullName

            Write-host -f Yellow "Uploading File '$_' to URL "$TargetFileURL
            #Get the file from disk
            $FileStream = ([System.IO.FileInfo] (Get-Item $SourceFilePath)).OpenRead()
   
            #Upload the File to SharePoint Library's Folder
            $FileCreationInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation
            $FileCreationInfo.Overwrite = $true
            $FileCreationInfo.ContentStream = $FileStream
            $FileCreationInfo.URL = $TargetFileURL
            $FileUploaded = $TargetFolder.Files.Add($FileCreationInfo)
  
            $Ctx.ExecuteQuery()  
            #Close file stream
            $FileStream.Close()

            #Update Metadata of the File
            $FileProperties = $FileUploaded.ListItemAllFields
            $FileProperties["Created"] = [datetime]$_.CreationTime 
            $FileProperties["Modified"] =[datetime]$_.LastWriteTime
            $FileProperties.Update()
            $Ctx.ExecuteQuery()
            Write-host "File '$TargetFileURL' Uploaded Successfully!" -ForegroundColor Green
        }
    }
}

#Set parameter values
$SiteURL="https://crescenttech.sharepoint.com/sites/marketing"
$TargetLibraryName="Documents"
$SourceFolderPath="\\Cre-LT575\Salaudeen\Project Documents"
 
#Call the function to Upload All files & folders from network Fileshare to SharePoint Online
Migrate-FileShareToSPO -SiteURL $SiteURL -SourceFolderPath $SourceFolderPath -TargetLibraryName $TargetLibraryName
This PowerShell script uploads all files and folders from given network file share to an existing SharePoint Online document library.
migrate file share to sharepoint online powershell
You can use this PowerShell script to Bulk upload files & folders from a local drive to SharePoint Online or OneDrive! Simply call the function "Migrate-FileShareToSPO" with relevant parameters

Import File Share to SharePoint Online Document Library using PnP PowerShell
This time, lets upload files from a network file share to SharePoint Online library with PnP PowerShell
#Parameters
$SiteURL = "https://crescent.sharepoint.com/sites/london/"
$SourceFolderPath = "\\FileServer\Upload"
$ListName = "Documents"

#Connect to PnP Online
Connect-PnPOnline -Url $SiteURL -UseWebLogin

#Get the Target Folder to Upload
$Web = Get-PnPWeb
$List = Get-PnPList $ListName -Includes RootFolder
$TargetFolder = $List.RootFolder

#Get All Files from the Source Folder
$Files = Get-ChildItem -Path $SourceFolderPath -Recurse | Where {!$_.PSIsContainer }

#Iterate through Each file and Upload to SharePoint Online Library
$Counter = 0
$Files | ForEach-Object {
        #Calculate Target Folder URL for the file
        $TargetFolderURL = $TargetFolder.ServerRelativeURL.Replace($Web.ServerRelativeUrl,"") + $_.DirectoryName.Replace($SourceFolderPath,"").Replace("\","/") 
        
        #Display Progress bar
        $Status  = "Uploading File '" + $_.Name + "' to " + $TargetFolderURL +" ($($Counter) of $($Files.Count))"
        Write-Progress -Activity "Uploading Files..." -Status $Status -PercentComplete (($Counter / $Files.Count)  * 100)

        #Check if File exists Already
        $FileURL = $TargetFolderURL+"/"+$_.Name
        $FileExists = Get-PnPFile -Url $FileURL -ErrorAction SilentlyContinue
        If(!$FileExists)
        {
            #Upload File to Folder
            $File  = Add-PnPFile -Path $_.FullName -Folder $TargetFolderURL
            Write-host "Uploaded File $($_.FullName) to Folder $TargetFolderURL"
        }
        $Counter++
}

File share to SharePoint Online Migration using PowerShell
What if you want to incrementally migrate files from fileshare to SharePoint Online? Well, this script imports all files from network file share to SharePoint Online document library.
#Function to Import FileShare to SharePoint Online Document Library
Function Import-FileSharetoSPO()
{
    param
    (
        [Parameter(Mandatory=$true)] [string] $SiteURL,
        [Parameter(Mandatory=$true)] [string] $SourceFolderPath,
        [Parameter(Mandatory=$true)] [string] $ListName,            
        [Parameter(Mandatory=$true)] [string] $LogFile,
        [Parameter(Mandatory=$true)] [string] $ConfigFilePath
    )

    Try {
        Add-content $Logfile -value "`n---------------------- File Import Script Started: $(Get-date -format 'dd/MM/yyy hh:mm:ss tt')-------------------" 
    
        #Connect to PnP Online
        Connect-PnPOnline -Url $SiteURL -UseWebLogin

        #Initialize Config File
        Try {
        $ConfigFile = [XML] (Get-Content $ConfigFilePath -ErrorAction Stop)
        }
        Catch [System.Management.Automation.ItemNotFoundException] {
            $Config = "<?xml version='1.0' encoding='UTF-8'?><Config><Timestamp LastRun='01/01/2000 12:00:00 AM'></Timestamp></Config>"
            New-Item -ItemType "File" -Path $ConfigFilePath | Out-Null
            Set-Content -Path $ConfigFilePath -Value $Config    
        }
    
        #Get the Last Run time of the script from Config file
        $ConfigFile = [XML] (Get-Content $ConfigFilePath)
        $ScriptLastRunTime = [DateTime]::ParseExact($ConfigFile.Config.Timestamp.GetAttribute("LastRun"), "dd/MM/yyyy hh:mm:ss tt", $null)

        #Update LastRun Timestamp in config file
        $ConfigFile.Config.Timestamp.SetAttribute("LastRun", (Get-date -format "dd/MM/yyy hh:mm:ss tt"))
        $ConfigFile.Save($ConfigFilePath)

        #Get the Most Recent Item Created Timestamp
        $LastItemTimestamp =  Get-ChildItem -Path $SourceFolderPath -Recurse | Measure-Object -Maximum -Property CreationTime | Select -ExpandProperty Maximum

        #Check if there are new items since last script executed
        If($LastItemTimestamp -ge $ScriptLastRunTime)
        {
            Write-host -f green "Found Changes to the Source since the last execution of script..." 
            Add-content $Logfile -value "Found Changes to the Source since the last execution of script!" 

            #Get the Target Folder to Upload
            $Web = Get-PnPWeb
            $List = Get-PnPList $ListName -Includes RootFolder
            $TargetFolder = $List.RootFolder
            $TargetFolderSiteRelativeURL = $TargetFolder.ServerRelativeURL.Replace($Web.ServerRelativeUrl,"")

            #Get All New Items from the Source
            $SourceItems = Get-ChildItem -Path $SourceFolderPath -Recurse | Where-Object {$_.CreationTime -gt $ScriptLastRunTime}
            $Source = $SourceItems | Select FullName, PSIsContainer, @{Label='TargetItemURL';Expression={$_.FullName.Replace($SourceFolderPath,$TargetFolderSiteRelativeURL).Replace("\","/")}}
            Add-content $Logfile -value "Number of New Items Found in the Source: $($SourceItems.Count)"

            #Upload Source Items from Fileshare to Target SharePoint Online document library
            $Counter = 1
            $Source | ForEach-Object {
                    #Calculate Target Folder URL
                    $TargetFolderURL = (Split-Path $_.TargetItemURL -Parent).Replace("\","/")
                    $ItemName = Split-Path $_.FullName -leaf
        
                    #Display Progress bar
                    $Status  = "Importing '" + $ItemName + "' to " + $TargetFolderURL +" ($($Counter) of $($SourceItems.Count))"
                    Write-Progress -Activity "Uploading ..." -Status $Status -PercentComplete (($Counter / $SourceItems.Count) * 100)

                    If($_.PSIsContainer)
                    {
                        #Ensure Folder
                        $Folder  = Resolve-PnPFolder -SiteRelativePath ($TargetFolderURL+"/"+$ItemName)
                        Write-host "Ensured Folder '$($ItemName)' to Folder $TargetFolderURL"
                        Add-content $Logfile -value "Ensured Folder '$($ItemName)' to Folder $TargetFolderURL"
                    }
                    Else
                    {
                        #Upload File
                        $File  = Add-PnPFile -Path $_.FullName -Folder $TargetFolderURL
                        Write-host "Uploaded File '$($_.FullName)' to Folder $TargetFolderURL"
                        Add-content $Logfile -value "Uploaded File '$($_.FullName)' to Folder $TargetFolderURL"
                    }
                    $Counter++
            }
        }
        Else
        {
            Write-host -f Yellow "No Changes detected in the Source!"
            Add-content $Logfile -value "No Changes detected in the Source!" 
        } 
    }
    Catch {
        Write-host -f Red "Error:" $_.Exception.Message 
        Add-content $Logfile -value "Error:$($_.Exception.Message)"
    }
    Finally {
       Add-content $Logfile -value "---------------------- File Import Script Completed: $(Get-date -format 'dd/MM/yyy hh:mm:ss tt')-----------------"
    }
}

#Call the Function with different parameters
Import-FileSharetoSPO -SiteURL "https://crescent.sharepoint.com/sites/Funds" -SourceFolderPath "\\fileserver\Notices" `
                            -ListName "Notices" -LogFile "E:\FileShareImportScript\Notices-LOG.log" -ConfigFilePath "E:\FileShareImportScript\Notices-CONFIG.config"

Import-FileSharetoSPO -SiteURL "https://crescent.sharepoint.com/sites/Funds" -SourceFolderPath "\\fileserver\Reports" `
                            -ListName "Reports" -LogFile "E:\FileShareImportScript\Reports-LOG.log" -ConfigFilePath "E:\FileShareImportScript\Reports-CONFIG.config"

For the first time, when you execute the script, it creates a config file and updates "LastRun" timestamp in it. From the next time, it fetches files and folders created after the last run time of the script and uploads them.
Migrate File Share to SharePoint Online using PowerShell Migrate File Share to SharePoint Online using PowerShell Reviewed by Salaudeen Rajack on January 21, 2018 Rating: 5

No comments:

Please Login and comment to get your questions answered!

Powered by Blogger.