SharePoint Online: Upload Large Files using PowerShell

Requirement: Upload large files to SharePoint Online using PowerShell

PowerShell to Upload large file to SharePoint Online
If you try to upload files with > 250 MB file size to SharePoint Online using CSOM Files.Add method, or PnP PowerShell Add-PnPFile methods you'll end-up in errors:

• "The remote server returned an error: (503) Server Unavailable." 
• "Add-PnPFile : The request message is too big. The server does not allow messages larger than 262144000 bytes"
• Add-PnPFile: The underlying connection was closed: An unexpected error occurred on a send
• Add-PnPFile : The remote server returned an error: (404) Not Found
  

To mitigate the errors, Here is how to upload a larger file to SharePoint Online using PowerShell:

#Load SharePoint CSOM Assemblies
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"

#Parameters
$SiteURL = "https://crescent.sharepoint.com/sites/marketing" 
$TargetFolderURL = "/sites/marketing/Shared Documents"
$FilePath =  "C:\Users\Thomas\Downloads\WFx64.zip"
 
#Get Credentials to connect
$Cred = Get-Credential

Try { 
    #Setup the context
    $Ctx = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
    $Ctx.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($Cred.UserName,$Cred.Password)

    #Get the file from disk
    $FileStream = ([System.IO.FileInfo] (Get-Item $FilePath)).OpenRead()

    #Calculate Target URL
    $SourceFileName = Split-path $FilePath -leaf
    $TargetFileURL = $TargetFolderURL+"/"+$SourceFileName

    #Upload Large File to SharePoint Online
    $Ctx.RequestTimeout = [System.Threading.Timeout]::Infinite
    [Microsoft.SharePoint.Client.File]::SaveBinaryDirect($Ctx, $TargetFileURL, $FileStream,$True)
             
    Write-host "File uploaded Successfully!" -f Green
}
Catch {
    write-host "Error Uploading File: $($_.Exception.Message)" -foregroundcolor Red
}

However, when you upload a larger file from slow network, it timeouts if the upload operation takes more than 30 minutes.

PowerShell to Upload Large Files in Chunks to SharePoint Online:
Instead of uploading a larger file in one single stretch, we can split and upload in smaller chunks.

#Function to Upload Large File to SharePoint Online Library
Function Upload-LargeFile($FilePath, $LibraryName, $FileChunkSize=10)
{
    Try {
        #Get File Name
        $FileName = [System.IO.Path]::GetFileName($FilePath)
        $UploadId = [GUID]::NewGuid()

        #Get the folder to upload
        $Library = $Ctx.Web.Lists.GetByTitle($LibraryName)
        $Ctx.Load($Library)
        $Ctx.Load($Library.RootFolder)
        $Ctx.ExecuteQuery()

        $BlockSize = $FileChunkSize * 1024 * 1024  
        $FileSize = (Get-Item $FilePath).length
        If($FileSize -le $BlockSize)
        {
            #Regular upload
            $FileStream = New-Object IO.FileStream($FilePath,[System.IO.FileMode]::Open)
            $FileCreationInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation
            $FileCreationInfo.Overwrite = $true
            $FileCreationInfo.ContentStream = $FileStream
            $FileCreationInfo.URL = $FileName
            $Upload = $Docs.RootFolder.Files.Add($FileCreationInfo)
            $ctx.Load($Upload)
            $ctx.ExecuteQuery()
        }
        Else
        {
            #Large File Upload in Chunks
            $ServerRelativeUrlOfRootFolder = $Library.RootFolder.ServerRelativeUrl
            [Microsoft.SharePoint.Client.File]$Upload
            $BytesUploaded = $null  
            $Filestream = $null
            $Filestream = [System.IO.File]::Open($FilePath, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read, [System.IO.FileShare]::ReadWrite)
            $BinaryReader = New-Object System.IO.BinaryReader($Filestream)
            $Buffer = New-Object System.Byte[]($BlockSize)
            $LastBuffer = $null
            $Fileoffset = 0
            $TotalBytesRead = 0
            $BytesRead
            $First = $True
            $Last = $False

            #Read data from the file in blocks
            While(($BytesRead = $BinaryReader.Read($Buffer, 0, $Buffer.Length)) -gt 0)
            {  
                $TotalBytesRead = $TotalBytesRead + $BytesRead  
                If ($TotalBytesRead -eq $FileSize) 
                {  
                    $Last = $True
                    $LastBuffer = New-Object System.Byte[]($BytesRead)
                    [Array]::Copy($Buffer, 0, $LastBuffer, 0, $BytesRead)  
                }
                If($First) 
                {  
                    #Create the File in Target
                    $ContentStream = New-Object System.IO.MemoryStream
                    $FileCreationInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation
                    $FileCreationInfo.ContentStream = $ContentStream
                    $FileCreationInfo.Url = $FileName
                    $FileCreationInfo.Overwrite = $true
                    $Upload = $Library.RootFolder.Files.Add($FileCreationInfo)
                    $Ctx.Load($Upload)

                    #Start FIle upload by uploading the first slice
                    $s = new-object System.IO.MemoryStream(, $Buffer)  
                    $BytesUploaded = $Upload.StartUpload($UploadId, $s)
                    $Ctx.ExecuteQuery()  
                    $fileoffset = $BytesUploaded.Value  
                    $First = $False  
                }  
                Else
                {  
                    #Get the File Reference
                    $Upload = $ctx.Web.GetFileByServerRelativeUrl($Library.RootFolder.ServerRelativeUrl + [System.IO.Path]::AltDirectorySeparatorChar + $FileName);
                    If($Last) 
                    {
                        $s = [System.IO.MemoryStream]::new($LastBuffer)
                        $Upload = $Upload.FinishUpload($UploadId, $fileoffset, $s)
                        $Ctx.ExecuteQuery()
                        Write-Host "File Upload completed!" -f Green                        
                    }
                    Else
                    {
                        #Update fileoffset for the next slice
                        $s = [System.IO.MemoryStream]::new($buffer)
                        $BytesUploaded = $Upload.ContinueUpload($UploadId, $fileoffset, $s)
                        $Ctx.ExecuteQuery()
                        $fileoffset = $BytesUploaded.Value
                    }
                }
            }
        }
    }
    Catch {
        Write-Host $_.Exception.Message -ForegroundColor Red
    }
    Finally {
        If($Filestream -ne $null)
        {
            $Filestream.Dispose()
        }
    }
}

#Connect to SharePoint Online site
Connect-PnPOnline "https://crescent.sharepoint.com/sites/marketing" -UseWebLogin
$Ctx = Get-PnPContext

#Call the function to Upload File
Upload-LargeFile -FilePath "C:\Users\Thomas\Downloads\235021WFx64.rar" -LibraryName "Documents"

BTW, I've converted the method into PowerShell, described in https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/upload-large-files-sample-app-for-sharepoint