SharePoint Online: Copy List Items between Site Collections using PowerShell

Requirement: Copy List Items between Site Collections in SharePoint Online using PowerShell.

powershell to copy list Items between site collections in sharepoint online

PowerShell to Copy List Items between SharePoint Online Site Collections

When you need to copy list items between site collections in SharePoint Online, PowerShell can be a quick and easy way. This blog post will show you how to use PowerShell to copy list items from one site collection to another. This can be helpful if you have to copy data between two different sites or want to make a backup of your data, or you need to consolidate information from multiple lists into one location, or if you’re looking to migrate data from one site collection to another.

Just set parameters accordingly and run the script. Make sure your target list is already created with the same structure (either by creating a new list from an existing list or saving the list as a template and creating a new list instance so that the internal names are intact!). More here: How to Copy a List in SharePoint Online?

#Function to copy attachments between list items
Function Copy-SPOAttachments()
{
    param
    (
        [Parameter(Mandatory=$true)] [Microsoft.SharePoint.Client.ListItem] $SourceItem,
        [Parameter(Mandatory=$true)] [Microsoft.SharePoint.Client.ListItem] $DestinationItem
    )
    Try {
        #Get All Attachments from Source list items
        $Attachments = Get-PnPProperty -ClientObject $SourceItem -Property "AttachmentFiles" -Connection $SourceConn
        $Attachments | ForEach-Object {
            #Download the Attachment to Temp
            $File  = Get-PnPFile -Connection $SourceConn -Url $_.ServerRelativeUrl -FileName $_.FileName -Path $Env:TEMP -AsFile -Force
            #Add Attachment to Destination List Item
            $FileStream = New-Object IO.FileStream(($Env:TEMP+"\"+$_.FileName),[System.IO.FileMode]::Open)  
            $AttachmentInfo = New-Object -TypeName Microsoft.SharePoint.Client.AttachmentCreationInformation
            $AttachmentInfo.FileName = $_.FileName
            $AttachmentInfo.ContentStream = $FileStream
            $AttachFile = $DestinationItem.AttachmentFiles.Add($AttachmentInfo)
            Invoke-PnPQuery -Connection $DestinationConn
      
            #Delete the Temporary File
            Remove-Item -Path $Env:TEMP\$($_.FileName) -Force
        }
    }
    Catch {
        write-host -f Red "Error Copying Attachments:" $_.Exception.Message
    }
}

#Function to copy list items from one list to another
Function Copy-SPOListItems()
{
    param
    (
        [Parameter(Mandatory=$true)] [Microsoft.SharePoint.Client.List] $SourceList,
        [Parameter(Mandatory=$true)] [Microsoft.SharePoint.Client.List] $DestinationList
    )
    Try {
        #Get All Items from the Source List in batches 
        Write-Progress -Activity "Reading Source..." -Status "Getting Items from Source List. Please wait..."
        $SourceListItems = Get-PnPListItem -List $SourceList -PageSize 500 -Connection $SourceConn
        $SourceListItemsCount= $SourceListItems.count
        Write-host "Total Number of Items Found:"$SourceListItemsCount      
  
        #Get fields to Update from the Source List - Skip Read only, hidden fields, content type and attachments
        $SourceListFields = Get-PnPField -List $SourceList -Connection $SourceConn | Where { (-Not ($_.ReadOnlyField)) -and (-Not ($_.Hidden)) -and ($_.InternalName -ne  "ContentType") -and ($_.InternalName -ne  "Attachments") }

        #Loop through each item in the source and Get column values, add them to Destination
        [int]$Counter = 1
        ForEach($SourceItem in $SourceListItems)
        {  
            $ItemValue = @{}
            #Map each field from source list to Destination list
            Foreach($SourceField in $SourceListFields)
            {
                #Check if the Field value is not Null
                If($SourceItem[$SourceField.InternalName] -ne $Null)
                {
                    #Handle Special Fields
                    $FieldType  = $SourceField.TypeAsString                    
  
                    If($FieldType -eq "User" -or $FieldType -eq "UserMulti") #People Picker Field
                    {
                        $PeoplePickerValues = $SourceItem[$SourceField.InternalName] | ForEach-Object { $_.Email}
                        $ItemValue.add($SourceField.InternalName,$PeoplePickerValues)
                    }
                    ElseIf($FieldType -eq "Lookup" -or $FieldType -eq "LookupMulti") # Lookup Field
                    {
                        $LookupIDs = $SourceItem[$SourceField.InternalName] | ForEach-Object { $_.LookupID.ToString()}
                        $ItemValue.add($SourceField.InternalName,$LookupIDs)
                    }
                    ElseIf($FieldType -eq "URL") #Hyperlink
                    {
                        $URL = $SourceItem[$SourceField.InternalName].URL
                        $Description  = $SourceItem[$SourceField.InternalName].Description
                        $ItemValue.add($SourceField.InternalName,"$URL, $Description")
                    }
                    ElseIf($FieldType -eq "TaxonomyFieldType" -or $FieldType -eq "TaxonomyFieldTypeMulti") #MMS
                    {
                        $TermGUIDs = $SourceItem[$SourceField.InternalName] | ForEach-Object { $_.TermGuid.ToString()}                    
                        $ItemValue.add($SourceField.InternalName,$TermGUIDs)
                    }
                    Else
                    {
                        #Get Source Field Value and add to Hashtable                        
                        $ItemValue.add($SourceField.InternalName,$SourceItem[$SourceField.InternalName])
                    }
                }
            }
            #Copy Created by, Modified by, Created, Modified Metadata values
            $ItemValue.add("Created", $SourceItem["Created"]);
            $ItemValue.add("Modified", $SourceItem["Modified"]);
            $ItemValue.add("Author", $SourceItem["Author"].Email);
            $ItemValue.add("Editor", $SourceItem["Editor"].Email);

            Write-Progress -Activity "Copying List Items:" -Status "Copying Item ID '$($SourceItem.Id)' from Source List ($($Counter) of $($SourceListItemsCount))" -PercentComplete (($Counter / $SourceListItemsCount) * 100)
            
            #Copy column value from Source to Destination
            $NewItem = Add-PnPListItem -List $DestinationList -Values $ItemValue -Connection $DestinationConn
  
            #Copy Attachments
            Copy-SPOAttachments -SourceItem $SourceItem -DestinationItem $NewItem
  
            Write-Host "Copied Item ID from Source to Destination List:$($SourceItem.Id) ($($Counter) of $($SourceListItemsCount))"
            $Counter++
        }
   }
    Catch {
        Write-host -f Red "Error:" $_.Exception.Message 
    }
}
  
#Set Parameters
$SourceSiteURL = "https://crescent.sharepoint.com/sites/Retail"
$SourceListName = "Projects"
 
$DestinationSiteURL = "https://crescent.sharepoint.com/sites/Sales"
$DestinationListName = "Projects"
 
#Connect to Source and destination sites
$SourceConn = Connect-PnPOnline -Url $SourceSiteURL -Interactive -ReturnConnection
$SourceList = Get-PnPList -Identity $SourceListName -Connection $SourceConn
 
$DestinationConn = Connect-PnPOnline -Url $DestinationSiteURL -Interactive -ReturnConnection
$DestinationList = Get-PnPList -Identity $DestinationListName -Connection $DestinationConn
  
#Call the Function to Copy List Items between Lists
Copy-SPOListItems -SourceList $SourceList -DestinationList $DestinationList

If you are copying list items between sites, make sure all parent lookup lists are created in the target site first!

In conclusion, it’s often necessary to copy items from one list to another in SharePoint Online, which can be done manually but can be time-consuming and error-prone if done for a large number of items. PowerShell allows you to automate repetitive tasks and perform actions on multiple items at once. Using PowerShell to copy items from one list to another in SharePoint Online is a fast and efficient way to transfer data. By connecting to SharePoint Online, retrieving the lists, and using the Add() method to copy the items, you can easily copy items from one list to another.

Salaudeen Rajack

Salaudeen Rajack - Information Technology Expert with Two-decades of hands-on experience, specializing in SharePoint, PowerShell, Microsoft 365, and related products. He has held various positions including SharePoint Architect, Administrator, Developer and consultant, has helped many organizations to implement and optimize SharePoint solutions. Known for his deep technical expertise, He's passionate about sharing the knowledge and insights to help others, through the real-world articles!

23 thoughts on “SharePoint Online: Copy List Items between Site Collections using PowerShell

  • Hi Salaudeen, thank you very much for these scripts. Works quite well and help me enormously. However, disregard I have lookup referenced list created first in another site, and populated with values, of the second list (with lookup values to the first), fields remain blank. Something is not working here, or I am doing something wrong thousand of times of testing:

    $LookupIDs = $SourceItem[$SourceField.InternalName] | ForEach-Object { $_.LookupID.ToString()}
    $ItemValue.add($SourceField.InternalName,$LookupIDs)

    Your help highly appreciated.

    Reply
  • Thank you very much! Works perferct!

    Reply
  • Hi Saluadeen,

    Thank you for the script it works a dream.

    I have however run into an issue, i’m trying to copy a list from a deleted user onedrive that has been restored and the user no longer exists in people picker/ is an orphaned user.

    I believe the issue is either caused by line 95 & 96 or by lines 64 -67.

    Is there a way to work around this issue?

    Reply
  • Getting error:
    Copy-SPOListItems -SourceList $SourceList -DestinationList $DestinationList
    Copy-SPOListItems : Cannot bind argument to parameter ‘SourceList’ because it is null.
    At line:3 char:31
    + Copy-SPOListItems -SourceList $SourceList -DestinationList $Destinati …

    Can you help with this error?

    Reply
  • I’m getting this error:
    Copy-SPOListItems : The term ‘Copy-SPOListItems’ is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path
    was included, verify that the path is correct and try again.

    Reply
    • addressed it already

      Reply
  • The latest pnp.powershell module 1.11.0 breaks this script. “The object is used in the context different from the one associated with the object.”

    Reply
      • Thanks Salaudeen. Confirmed working with the 1.11.0 pnp module in our environment. Very much appreciate all the information you have provided across your website. Really helpful.

        Reply
  • Hi Salaudeen,
    I am having the error below running this script.
    Would you be kind to assist me to resolve it, please?

    “InvalidOperation: C:\CopyListItemsBetweenSPOSiteCollectionsPnP.ps1:37
    Line |
    37 | … ameter(Mandatory = $true)] [Microsoft.SharePoint.Client.List] $Source …
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    | Unable to find type [Microsoft.SharePoint.Client.List].”

    Reply
  • Hi Saluadeen, this is great and has been really useful for many of my lists. What would be really good is if it could keep the ID’s. I need the ID’s to be static when migrating Lists with Lookup values.

    I have seen another approach, although not using Powershell, which creates the item then checks it matches the ID and if it doesn’t it will delete it and then recreate until it does. Is this possible?

    Thanks for your help and very useful website!

    Reply
  • hi, How I should modify script if list item include user field which is inactive

    Reply
    • I get the same error for inactive users “The specified user could not be found”, & the script stops copying list items there. @Salaudeen: Can you suggest a fix for this, please?

      Reply
  • I am also getting same error, any fix you find for this?

    Reply
    • @Salaudeen: Any update on this? How to copy list items irrespective of the status of the users? Currently, the script stops running when it encounters an inactive user.

      Reply
  • I get this error: Copy-SPOListItems : Cannot process argument transformation on parameter ‘SourceList’. Cannot convert the “Microsoft.SharePoint.Client.List” value of type “Microsoft.SharePoint.Client.List” to type
    “Microsoft.SharePoint.Client.List”.

    Not the most helpful of errors…

    Reply
    • Restarting the PowerShell ISE fixes this issue.

      Reply
  • Hello Sir, thank you for the script, much appreciated, it works well for copying within the same tenant but can this be tweaked to copy to another tenant, as the users cannot be found in new tenant, can we just get the text outputs for it to resolve in people picker, or is there any other way

    Reply
  • Thanks a lot for this post, it’s very helpful.
    What if I wanted to keep the Author, Created, Created By, Modified and Modified By fields?
    How can this be accomplished?
    From your script I see you have a section dedicated to handle special fields but these are not being kept.
    Any suggestion is very appreciated because I am having a really hard time trying to figure it out.
    Thanks in advance.

    Reply
      • Hello, thanks a lot for the post. It is very useful.

        When the items are copied, the system adds “by PnP Management Shell on behalf of” to the metadata. Is it possible to remove this and keep only the source information?

        Thank you!

        Reply
    • Thank a lot, it Worked.
      I saw my mistake.The created and the Modified was working but on the Author and Editor I was missing “.Email” and just adding code like below.
      $ItemValue.add(“Author”, $SourceItem[“Author”]);
      $ItemValue.add(“Editor”, $SourceItem[“Editor”]);

      Again, many thanks.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *