For loop Caml Query | How to use Caml Query for loop to increment the date range by month and also year once the loop reaches January?

Question

I am using Powershell scripting to get data from a sharepoint list which exceeds the threshold of 5000 items.

I want to create a for loop which increments by month and year.

The Code in the for loop would move the source SharePoint list data to target SharePoint list.

I cannot completely move all the data at a time due to the limitations of SharePoint online (threshold).

I also want to ensure that all the data of that particular month gets transferred to the target list. Basically, if a month has more than 5000 items, I want the for loop to run again and move the remaining data.

Again, how should I create the for loop for the same?

Current caml query code without the for loop:

$Query = New-Object Microsoft.SharePoint.Client.CamlQuery
$Query.ViewXml = "@
<View>
    <Query>
     <And>
      <Gt>
       <FieldRef Name='Date' />
        <Value IncludeTimeValue='TRUE' Type='DateTime'>2020-12-01T18:37:21Z</Value>
      </Gt>
      <Lt>
       <FieldRef Name='Date' />
        <Value IncludeTimeValue='TRUE' Type='DateTime'>2020-12-31T18:37:24Z</Value>
      </Lt>
     </And>
    </Query>
    <RowLimit Paged='True'>100</RowLimit>
</View>"

Ask me more questions if needed.

Answer 1

As a workaround, you could do the loop like this:

    $List = $Ctx.Web.Lists.GetByTitle($ListName)
    $Ctx.Load($List)
    $Ctx.ExecuteQuery()
 
    #Define Query to get List Items in batch
    $Query = New-Object Microsoft.SharePoint.Client.CamlQuery
    $Query.ViewXml = @"
    <View Scope='RecursiveAll'>
        <Query>
            <OrderBy><FieldRef Name='Date' Ascending='TRUE'/></OrderBy>
        </Query>
        <RowLimit Paged="TRUE">100</RowLimit>
    </View>
"@
 
    #Get List Items in Batch
    Do
    {
        $ListItems = $List.GetItems($Query)
        $Ctx.Load($ListItems)
        $Ctx.ExecuteQuery()
        $ListItems.count
        $Query.ListItemCollectionPosition = $ListItems.ListItemCollectionPosition
        foreach($item in $listItems)
        {
            Try
            {
              #Move item
             }
            Catch [System.Exception]
            {
            # This shouldn't happen, but just in case
            Write-Host $_.Exception.Message
            }
        }
    }
    
    While($Query.ListItemCollectionPosition -ne $null)

Reference: https://www.sharepointdiary.com/2016/12/sharepoint-online-get-all-items-from-large-lists-powershell-csom.html#ixzz6if7p4ZJu