How to Sync File Share to SharePoint Online using PowerShell?

Requirement: Sync Fileshare to SharePoint Online using PowerShell

sync file share to sharepoint online document library using powershell

PowerShell Script to Sync File share to SharePoint Online document Library

If you are using File Shares, you may find the need to keep your file shares in sync with SharePoint Online or OneDrive for Business. Perhaps you want to keep data in both places up to date or wish to take advantage of the additional storage and collaboration features that SharePoint Online offers. In either case, PowerShell provides a way to automate this process and make it less error-prone easily. This article will show you how to use PowerShell to sync your file shares to SharePoint Online.

So, How to sync a file server to SharePoint Online? While my other script, Migrate File Share to SharePoint Online using PowerShell, imports files and folders from network file share to SharePoint Online, this script syncs file share to SharePoint Online document library.

Import-Module SharePointPnPPowerShellOnline

#Function to Import Files from Fileshare to SharePoint Online
Function Import-FileShareToSPO
{
 param
    (
        [Parameter(Mandatory=$true)] [string] $SiteURL,
        [Parameter(Mandatory=$true)] [string] $SourceFolderPath,
        [Parameter(Mandatory=$true)] [string] $TargetLibraryName,            
        [Parameter(Mandatory=$true)] [string] $LogFile
    )

    Try {
        Add-content $Logfile -value "`n---------------------- Import FileShare Script Started: $(Get-date -format 'dd/MM/yyy hh:mm:ss tt')-------------------"   
    
        #Get Number of Source Items from the Source Folder
        $SourceItemsCount =  (Get-ChildItem -Path $SourceFolderPath -Recurse).count

        #Get the Target Library to Upload
        $Web = Get-PnPWeb
        $Library = Get-PnPList $TargetLibraryName -Includes RootFolder
        $TargetFolder = $Library.RootFolder

        #Get the site relative path of the target folder
        If($web.ServerRelativeURL -eq "/")
        {
            $TargetFolderSiteRelativeURL = $TargetFolder.ServerRelativeUrl
        }
        Else
        {        
            $TargetFolderSiteRelativeURL = $TargetFolder.ServerRelativeURL.Replace($Web.ServerRelativeUrl,[string]::Empty) 
        }  
 
        #Get All Items from the Source
        $SourceItems = Get-ChildItem -Path $SourceFolderPath -Recurse
        $Source = @($SourceItems | Select FullName,  PSIsContainer,
                                     @{Label='TargetItemURL';Expression={$_.FullName.Replace($SourceFolderPath,$TargetFolderSiteRelativeURL).Replace("\","/")}}, 
                                            @{Label='LastUpdated';Expression={$_.LastWriteTimeUtc.ToString()}})

        #Get All Files from the target document library - In batches of 2000
        $TargetFiles = Get-PnPListItem -List $TargetLibraryName -PageSize 2000
        $Target = @($TargetFiles | Select @{Label='FullName';Expression={$_.FieldValues.FileRef.Replace($TargetFolder.ServerRelativeURL,$SourceFolderPath).Replace("/","\")}},
                                                @{Label='PSIsContainer';Expression={$_.FileSystemObjectType -eq "Folder"}},
                                                    @{Label='TargetItemURL';Expression={$_.FieldValues.FileRef.Replace($Web.ServerRelativeUrl,[string]::Empty)}},
                                                        @{Label='LastUpdated';Expression={$_.FieldValues.Modified.ToUniversalTime().ToString()}})

        #Compare Source and Target and upload/update files that are not in the target
        $Counter = 1
        $FilesDiff = Compare-Object -ReferenceObject $Source -DifferenceObject $Target -Property FullName, PSIsContainer, TargetItemURL, LastUpdated
        #$FilesDiff | Export-csv -path "C:\Temp\diff.csv" -NoTypeInformation
        $SourceDelta = @($FilesDiff | Where {$_.SideIndicator -eq "<="}) 
        $SourceDeltaCount = $SourceDelta.Count

        #Check if Source Files are changed
        If($SourceDeltaCount -gt 0)
        {
            Write-host "Found $SourceDeltaCount new differences in the Source!" 
            Add-content $Logfile -value "Found $SourceDeltaCount new differences in the Source!"   
    
            $SourceDelta | Sort-Object TargetItemURL | ForEach-Object {
                #Calculate Target Folder URL for the file
                $TargetFolderURL = (Split-Path $_.TargetItemURL -Parent).Replace("\","/")
                If($TargetFolderURL.StartsWith("/")) {$TargetFolderURL = $TargetFolderURL.Remove(0,1) }
                $ItemName = Split-Path $_.FullName -leaf
                #Replace Invalid Characters
                $ItemName = [RegEx]::Replace($ItemName, "[{0}]" -f ([RegEx]::Escape([String]'\*:<>?/\|')), '_')

                #Display Progress bar
                $Status  = "Importing $ItemName to $TargetFolderURL $($Counter) of $($SourceDeltaCount)"
                Write-Progress -Activity "Importing Files from the Source..." -Status $Status -PercentComplete (($Counter / $SourceDeltaCount) * 100)

                If($_.PSIsContainer)
                {
                    #Ensure Folder
                    $Folder  = Resolve-PnPFolder -SiteRelativePath ($TargetFolderURL+"/"+$ItemName) -Includes ListItemAllFields
                    
                    Set-PnPListItem -List $TargetLibraryName -Identity $Folder.ListItemAllFields.Id -Values @{"Modified"=  ([DateTime]$_.LastUpdated).ToLocalTime()} | Out-null
                    Write-host "Ensured Folder '$($ItemName)' to Folder $TargetFolderURL"
                    Add-content $Logfile -value "Ensured Folder '$($ItemName)' to Folder $TargetFolderURL"
                }
                Else
                {
                    #Upload File
                    $File  = Add-PnPFile -Path $_.FullName -Folder $TargetFolderURL -Values @{"Modified"=  ([DateTime]$_.LastUpdated).ToLocalTime()}
                    Write-host "Ensured File '$($_.FullName)' to Folder $TargetFolderURL"
                    Add-content $Logfile -value "Ensured File '$($_.FullName)' to Folder $TargetFolderURL"
                }
                $Counter++
            }
        }
        Else
        {
            Write-host "Found no new Items in the Source! Total Items in Source: $SourceItemsCount , Number Items in Target: $($Library.Itemcount)" 
            Add-content $Logfile -value "Found no new Items in the Source! Items in Source: $SourceItemsCount ,  Number Items in Target: $($Library.Itemcount)" 
        }      
    }
    Catch {
        Write-host -f Red "Error:" $_.Exception.Message 
        Add-content $Logfile -value "Error:$($_.Exception.Message)"
    }
    Finally {
       Add-content $Logfile -value "---------------------- Import File Share Script Completed: $(Get-date -format 'dd/MM/yyy hh:mm:ss tt')-----------------"
    }
}

#Function to Remove Files Delta in SharePoint Online (Files that are no longer exists in FileShare )
Function Remove-FileShareDeltaInSPO
{
 param
    (
        [Parameter(Mandatory=$true)] [string] $SiteURL,
        [Parameter(Mandatory=$true)] [string] $SourceFolderPath,
        [Parameter(Mandatory=$true)] [string] $TargetLibraryName,            
        [Parameter(Mandatory=$true)] [string] $LogFile
    )

    Try {
        Add-content $Logfile -value "`n---------------------- Remove FileShare Delta Script Started: $(Get-date -format 'dd/MM/yyy hh:mm:ss tt')-------------------"   

        #Get Number of Source Items from the Source Folder
        $SourceItemsCount =  (Get-ChildItem -Path $SourceFolderPath -Recurse).count

        #Get the Target Library
        $Web = Get-PnPWeb
        $Library = Get-PnPList $TargetLibraryName -Includes RootFolder
        $TargetFolder = $Library.RootFolder

        #Get the site relative path of the target folder
        If($web.ServerRelativeURL -eq "/")
        {
            $TargetFolderSiteRelativeURL = $TargetFolder.ServerRelativeUrl
        }
        Else
        {        
            $TargetFolderSiteRelativeURL = $TargetFolder.ServerRelativeURL.Replace($Web.ServerRelativeUrl,[string]::Empty) 
        }        
 
        #Get All Items from the Source
        $SourceItems = Get-ChildItem -Path $SourceFolderPath -Recurse
        $Source = @($SourceItems | Select FullName,  PSIsContainer,
                                     @{Label='TargetItemURL';Expression={$_.FullName.Replace($SourceFolderPath,$TargetFolderSiteRelativeURL).Replace("\","/")}}, 
                                            @{Label='LastUpdated';Expression={$_.LastWriteTimeUtc.ToString()}})

        #Get All Files from the target document library - In batches of 2000
        $TargetFiles = Get-PnPListItem -List $TargetLibraryName -PageSize 2000
        $Target = @($TargetFiles | Select @{Label='FullName';Expression={$_.FieldValues.FileRef.Replace($TargetFolder.ServerRelativeURL,$SourceFolderPath).Replace("/","\")}},
                                                @{Label='PSIsContainer';Expression={$_.FileSystemObjectType -eq "Folder"}},
                                                    @{Label='TargetItemURL';Expression={$_.FieldValues.FileRef.Replace($Web.ServerRelativeUrl,[string]::Empty)}},
                                                        @{Label='LastUpdated';Expression={$_.FieldValues.Modified.ToUniversalTime().ToString()}})

        #Compare Source and Target and remove files that are not in the Source
        $Counter = 1
        $FilesDiff = Compare-Object -ReferenceObject $Source -DifferenceObject $Target -Property FullName, PSIsContainer, TargetItemURL, LastUpdated     
        $TargetDelta = @($FilesDiff | Where {$_.SideIndicator -eq "=>"})
        $TargetDeltaCount = $TargetDelta.Count

        #Check if Target Files Needs to be deleted
        If($TargetDeltaCount -gt 0)
        {
            Write-host "Found $TargetDeltaCount differences in the Target!" 
            Add-content $Logfile -value "Found $TargetDeltaCount differences in the Target!"   
    
            $TargetDelta | Sort-Object TargetItemURL -Descending | ForEach-Object {
                #Display Progress bar
                $Status  = "Removing Item " + $_.TargetItemURL +" ($($Counter) of $($TargetDeltaCount))"
                Write-Progress -Activity "Removing Items in the Target..." -Status $Status -PercentComplete (($Counter / $TargetDeltaCount) * 100)

                If($_.PSIsContainer)
                {
                    #Empty and Remove the Folder
                    $Folder  = Get-PnPFolder -Url $_.TargetItemURL -ErrorAction SilentlyContinue
                    If($Folder -ne $Null)
                    {
                        $Folder.Recycle() | Out-Null
                        Invoke-PnPQuery

                        Write-host "Removed Folder '$($_.TargetItemURL)'"
                        Add-content $Logfile -value "Removed Folder '$($_.TargetItemURL)'"
                    }
                }
                Else
                {
                    $File = Get-PnPFile -Url $_.TargetItemURL -ErrorAction SilentlyContinue
                    If($File -ne $Null)
                    {
                        #Remove the File
                        Remove-PnPFile -SiteRelativeUrl $_.TargetItemURL -Force
                        Write-host "Removed File '$($_.TargetItemURL)'"
                        Add-content $Logfile -value "Removed File '$($_.TargetItemURL)'"
                    }
                }
                $Counter++
            }
        }
        Else
        {
            Write-host "Found no differences in the Target! Total Items in Source: $SourceItemsCount , Number Items in Target: $($Library.Itemcount)" 
            Add-content $Logfile -value "Found no differences in the Target! Items in Source: $SourceItemsCount ,  Number Items in Target: $($Library.Itemcount)" 
        }
    }
    Catch {
        Write-host -f Red "Error:" $_.Exception.Message 
        Add-content $Logfile -value "Error:$($_.Exception.Message)"
    }
    Finally {
       Add-content $Logfile -value "---------------------- Remove FileShare Delta Script Completed: $(Get-date -format 'dd/MM/yyy hh:mm:ss tt')-----------------"
    }
}

Function Sync-FileShareToSPO()
{
 param
    (
        [Parameter(Mandatory=$true)] [string] $SiteURL,
        [Parameter(Mandatory=$true)] [string] $SourceFolderPath,
        [Parameter(Mandatory=$true)] [string] $TargetLibraryName,            
        [Parameter(Mandatory=$true)] [string] $LogFile
    )

    Try {
        #Connect to PnP Online
        Connect-PnPOnline -Url $SiteURL -Interactive

        #Call the function to Import New Files from Fileshare to SPO
        Import-FileShareToSPO -SiteURL $SiteURL -SourceFolderPath $SourceFolderPath -TargetLibraryName $TargetLibraryName -LogFile $LogFile

        #Call the function to Remove Files in SPO that are moved/deleted in Fileshare
        Remove-FileShareDeltaInSPO -SiteURL $SiteURL -SourceFolderPath $SourceFolderPath -TargetLibraryName $TargetLibraryName -LogFile $LogFile
    }
    Catch {
        Write-host -f Red "Error:" $_.Exception.Message
    }
    Finally {
       DisConnect-PnPOnline
    }
}

#Call the Function to Sync Files from Fileshare to SharePoint Online
Sync-FileShareToSPO -SiteURL "https://crescent.sharepoint.com/sites/Projects" -SourceFolderPath "\\FileServer\Reports" `
                            -TargetLibraryName "Documents" -LogFile "C:\Temp\SyncFileShare-LOG.log"

This script checks the timestamp of each file from the given file share and uploads it to SharePoint Online if they are updated in the File share. Similarly, any file deleted in Fileshare will be deleted in SharePoint Online as well. It generates a log file also every time it runs. Please note, this script does only one-way sync from Fileshare to SharePoint Online. You can Schedule PowerShell script in Windows Task scheduler to run on schedule. E.g., for every 10 min.

You can also use this script to sync files from the local drive to SharePoint by changing the SourceFolderPath parameter to any folder on your computer. E.g. “C:\Temp\Upload”

Salaudeen Rajack

Salaudeen Rajack is a SharePoint Architect with Two decades of SharePoint Experience. He loves sharing his knowledge and experiences with the SharePoint community, through his real-world articles!

16 thoughts on “How to Sync File Share to SharePoint Online using PowerShell?

  • Hello
    Good work this is a nice script

    But could you help me to change it on other direction?

    So this means to sync from Sharepoint Online to local file share?

    I am not that powershell expert 🙂

    Thanks

    Reply
    • Why don’t you use the OneDrive Client to sync your SharePoint Document Library with your local folder?

      Reply
      • Problem is that the files placed at the library are used by many different people and also by a little program which access the files from a local share. Currently we have an old SharePoint on-prem which can be accessed by unc path \\@SSL\….. to do the sync via script to a file share. But this no more possible with SPO
        So i need a fews hints how to get files from SPO compare with local share and only download modifications
        Like you do but in the other direction

        I hope u understand 🙂

        Reply
  • Hello,
    this is a nice article.

    Could someone help me to edit this script to do the sync in other direction?

    Source should be SPO and Target a local directory
    — I am not the best powershell expert 😉

    Thanks for any hint how to quickly do that 🙂

    Reply
  • Hello your script helped me a lot, I wonder how is it possible to synchronize to a specific folder in the destination library, and not at the root of the one so precisely.

    Reply
  • This site has saved me – and so has this script. But I would really like it to replace the owner of the file as well. All of our files are synced with OneDrive and have that service account as the owner of ALL on the 365 site. My local files all have the correct owner. I think somewhere in here I should add a label but I do not know which one, much less what would come after.
    $Source =
    @($SourceItems | Select FullName, PSIsContainer, @{Label=’TargetItemURL’;Expression=$_.FullName.Replace($SourceFolderPath,$TargetFolderSiteRelativeURL).Replace(“\”,”/”)}},
    @{Label=’LastUpdated’;Expression={$_.LastWriteTimeUtc.ToString()}})

    Any ideas?
    Thanks

    Reply
  • effectively, the “-Values @{“Modified”= ([DateTime]$_.LastUpdated).ToLocalTime()}” parameters seems to broke the script :
    Add-PnPFile : Attempted to perform an unauthorized operation.
    Au caractère Ligne:92 : 29
    + … $File = Add-PnPFile -Path $_.FullName -Folder $TargetFolderURL – …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : WriteError: (:) [Add-PnPFile], ServerUnauthorizedAccessException
    + FullyQualifiedErrorId : EXCEPTION,PnP.PowerShell.Commands.Files.AddFile

    Then just after, the remove section remove all the files just uploaded.
    A little update of the command could be very cool !!!
    thanks in advance

    Reply
  • Same Error as above. Server relative urls must start with SPWeb.ServerRelativeUrl

    Reply
  • Ensured File ‘millnlstoragecadlogfileswijziging.txt’ to Folder /Shared Documents
    Add-PnPFile : Server relative urls must start with SPWeb.ServerRelativeUrl
    At D:_MillscriptsSync-FileShareToSPO.ps1:84 char:30
    … $File = Add-PnPFile -Path $_.FullName -Folder $TargetFolderURL -V …
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    CategoryInfo : WriteError: (:) [Add-PnPFile], ServerException
    FullyQualifiedErrorId : EXCEPTION,PnP.PowerShell.Commands.Files.AddFile

    Reply
    • Well, This is because the “Add-PnPFile” cmdlet in new PnP.PowerShell only accepts the folder without leading “/” character. E.g. Add-PnPFile -Path “C:\temp\sample.txt” -Folder “Branding/2020/New” will work. However,
      Add-PnPFile -Path “C:\temp\sample.txt” -Folder “/Branding/2020/New” will fail!

      So, I’ve updated the script now and verified! Line#64

      Reply
  • Hello
    first – thanks for the good work Salaudeen
    I also have a problem with the date-format as i’m based in germany:

    Error: Cannot convert value “17.02.2021 08:44:11” to type “System.DateTime”. Error: “String was not recognized as a valid DateTime.”

    did you already found an answer for this?

    thanks to all!

    Reply
    • For Date Time Field values, You can use this format: [System.DateTime]”12/17/2021 08:44:11″ (Assuming your site’s regional settings are MM/DD/YYYY)

      Reply
  • Hi there, I appreciate your work and have used some of your scripts for my batch jobs.

    I am based at India and this script has two issues which I would like to bring to your attention.

    In getting list section: Comparing dates with UTC format does not work. I got a workaround though as @{Label=’LastUpdated’;Expression={$_.LastWriteTimeUTC.ToString(‘MM-dd-yyyy HH:mm’, [System.Globalization.CultureInfo]::InvariantCulture)}}) and @{Label=’LastUpdated’;Expression={$_.FieldValues.Modified.ToUniversalTime().ToString(‘MM-dd-yyyy HH:mm’, [System.Globalization.CultureInfo]::InvariantCulture)}}).

    The script uploads files after this modification, still there is second issue which I am trying to figure out.
    After uploading files and folder, the “Compare Source and Target and remove files which are not in the Source” section starts and removes all files and folders it just uploaded.

    Maybe it’s related to same date comparisons. I will let you know.

    Please keep up the good work.

    and

    Reply
  • Hi There, Thanks for putting effort into making these scripts available!
    I’m running into issues with larger files…. seems like there is a 250 MB limit?
    _Add-PnPFile : The underlying connection was closed: An unexpected error occurred on a send.
    https://github.com/pnp/PnP-PowerShell/issues/936

    Do you know of a way around this?

    Reply

Leave a Reply