Emptying SharePoint Recycle Bin

Question

Our SharePoint 2007 installation has a particularily immense First-Stage Recycle Bin. I wrote a PowerShell script a while back that deletes old items. Just for fun, that's posted below.

UPDATE2: We have 42,500,000+ records in the Recycle Bin table in the Content Database!!
I figured out that our BDLC job that was scheduled before I came on the team has been pumping so much data in the RecycleBin table that even SharePoint can't manage it properly without timing out. Even our scheduled script can only remove 1000 records every 30 minutes. Do the math on that, then feel sorry for me. There are so many records that now you can't even enforce quotas without SharePoint timing out. It's looking like we will have to freeze the deleting of records in BDLC

UPDATE1: I executed my PowerShell script to delete each item and ran SQL Profiler to find out proc_DeleteRecycleBinItem is being executed which touches multiple tables so we're going to stick with a scheduled PowerShell script to run/report every N minutes.

We have 38,500,000+ records in the Recycle Bin table in the Content Database!!

I believe that the reason this is so immense is because we have BDLC by Layer2 syncing data from other systems and it is Recycling deleted records. In addition, it is so large that even the native SharePoint Timer Job cannot control it and times out...

I know I would get shot by the SharePoint Mafia by asking this but... has anyone ever deleted rows from the actual RecycleBin table in a SharePoint content database? Ours is 15430792 KB (14.7 GB).

I'm aware that Microsoft will drop support if you modify a content database. Please don't post that as the answer. Of course that is not best practice. This question is clearly: has anyone ever tried it?...

I'm simply looking for a faster way to clean-up the Recycle Bin. You can see my scripts below so obviously I've been trying to establish some maintainence with it. The Bin gets so large that it takes forever for scripts to run because there is so much data.

Our user data is actually 203790168 KB (194.3 GB) and I wrote a PS script to get large files so I can manage that size as well. That, also below to contibute back to the ether.

Also, we have a BDLC (w/ 3rd party tool from Layer2) that sync's data to and fro SQL. I belive this job deletes lots of data regularly that make the RecycleBin table huge.

I guess the answer to my own question might be... something like... schedule a task to run the maintaince script I've already written... hmmm...

function RemoveOldItems-SPFirstStageRecycleBin([string]$url, [int]$rowlimit, [int]$days) 
{
    $siteCollection = New-Object Microsoft.SharePoint.SPSite($url);  
    $recycleQuery = New-Object Microsoft.SharePoint.SPRecycleBinQuery;
    $recycleQuery.ItemState = [Microsoft.SharePoint.SPRecycleBinItemState]::FirstStageRecycleBin
    $recycleQuery.OrderBy = [Microsoft.SharePoint.SPRecycleBinOrderBy]::DeletedDate
    $recycleQuery.RowLimit = $rowlimit

    $recycledItems = $siteCollection.GetRecycleBinItems($recycleQuery);

    $count = $recycledItems.Count;

    for($i = 0; $i -lt $count; $i++){
        $age = ((Get-Date) - $recycledItems[$i].DeletedDate).Days;
        if($age -gt $days){
            $g = New-Object System.Guid($recycledItems[$i].ID);
            $recycledItems.Delete($g);
        }
    }

    $siteCollection.Dispose()
}



function Get-DocInventory() {
    [void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")
    $farm = [Microsoft.SharePoint.Administration.SPFarm]::Local
    foreach ($spService in $farm.Services) {
        if (!($spService -is [Microsoft.SharePoint.Administration.SPWebService])) {
            continue;
        }

        foreach ($webApp in $spService.WebApplications) {
            if ($webApp -is [Microsoft.SharePoint.Administration.SPAdministrationWebApplication]) { continue }

            foreach ($site in $webApp.Sites) {
                foreach ($web in $site.AllWebs) {
                    foreach ($list in $web.Lists) {
                        if ($list.BaseType -ne "DocumentLibrary") {
                            continue
                        }
                        foreach ($item in $list.Items) {
                            $data = @{
                                "Web Application" = $webApp.ToString()
                                "Site" = $site.Url
                                "Web" = $web.Url
                                "list" = $list.Title
                                "Item ID" = $item.ID
                                "Item URL" = $item.Url
                                "Item Title" = $item.Title
                                "Item Created" = $item["Created"]
                                "Item Modified" = $item["Modified"]
                                "Size (kb)" = $item.File.Length/1KB
                                "Size (gb)" = $item.File.Length/1GB

                            }

                            Write-Host $item.Url -ForegroundColor DarkGray

                            # Only add files larger than 100 MB
                            if($item.File.Length -gt 100MB){
                                Write-Host $site.Url + $item.Url -ForegroundColor Red
                                New-Object PSObject -Property $data
                            }
                        }
                    }
                    $web.Dispose();
                }
                $site.Dispose()
            }
        }
    }
}
#Get-DocInventory | Out-GridView
Get-DocInventory | Export-Csv -NoTypeInformation -Path D:\Logs\inventory.csv


$jobName = "Get Large Lists"

function Get-LargeLists() {
    [void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")

    #create stop watch
    [System.Diagnostics.Stopwatch] $sw;
    $sw = New-Object System.Diagnostics.Stopwatch
    $sw.Start()

    $lists = @()
    $reportSiteUrl = "http://my-site.gov/sites/applications/reporting/"
    $reportListUrl = $reportSiteUrl + "Lists/Large%20Lists/"
    $farm = [Microsoft.SharePoint.Administration.SPFarm]::Local
    foreach ($spService in $farm.Services) {
        if (!($spService -is [Microsoft.SharePoint.Administration.SPWebService])) {continue}

        foreach ($webApp in $spService.WebApplications) {
            if ($webApp -is [Microsoft.SharePoint.Administration.SPAdministrationWebApplication]) {continue}

            foreach ($site in $webApp.Sites) {
                foreach ($web in $site.AllWebs) {
                    foreach ($list in $web.Lists) {
                        # only add items that have 1000 items or more
                        if($list.ItemCount -le 99){continue}

                        # create new object
                        $o = New-Object Object
                        Add-Member -InputObject $o -MemberType NoteProperty -Name SiteCollectionUrl -Value $list.ParentWeb.Site.RootWeb.Url
                        Add-Member -InputObject $o -MemberType NoteProperty -Name ListURL -Value ($list.ParentWeb.Url + "/" + $list.RootFolder.Url)
                        Add-Member -InputObject $o -MemberType NoteProperty -Name Title -Value $list.Title
                        Add-Member -InputObject $o -MemberType NoteProperty -Name Description -Value $list.Description
                        Add-Member -InputObject $o -MemberType NoteProperty -Name ItemCount -Value $list.ItemCount

                        # add object to $list global array
                        $lists += $o
                    }
                    $web.Dispose()
                }
                $site.Dispose()
            }
        }
    }

    #export array to csv
    $lists | Export-Csv D:\Logs\large_lists.csv -NoTypeInformation -Force

    #connect to SharePoint Site and List
    $s = New-Object Microsoft.SharePoint.SPSite($reportSiteUrl)
    $w = $s.openweb()
    $l = $w.GetList($reportListUrl)

    #clear SharePoint List
    $query = New-Object Microsoft.SharePoint.SPQuery
    $query.ViewAttributes = "Scope='Recursive'"
    $query.Query = ""

    $items = $l.GetItems($query)
    $items | % { $l.GetItemById($_.Id).Delete() }

    #export to SharePoint List
    $lists | ForEach{
        $item = $l.Items.Add()
        $item["Title"] = $_.Title
        $item["SiteCollectionUrl"] = $_.SiteCollectionUrl

        $u = New-Object Microsoft.SharePoint.SPFieldUrlValue
        $u.Description = "Link"
        $u.Url = $_.ListURL
        $item["URL"] = $u

        $item["Description"] = $_.Description
        $item["Count"] = $_.ItemCount
        $item.Update()
    }

    $w.Dispose()
    $s.Dispose()

    #stop timer and log event
    $sw.Stop()

    C:\_scripts\sp_log.ps1 -jobName $jobName -message "Reported large lists on SharePoint Farm." -type "Success" -duration $sw.Elapsed.Seconds -linkTitle "Link" -linkUrl "http://my-site.gov/sites/applications/reporting/Lists/Large%20Lists/"
}

#catch exceptions
trap [Exception]{
    C:\_scripts\sp_log.ps1 -jobName $jobName -message  $_.Exception.Message -type "Error" -duration $sw.Elapsed.Seconds -linkTitle "Link" -linkUrl "http://my-site.gov/sites/applications/reporting/Lists/Large%20Lists/"
}

Get-LargeLists

Answer 1

I executed my PowerShell script to delete each item and ran SQL Profiler to find out proc_DeleteRecycleBinItem is being executed which touches multiple tables so we're going to stick with a scheduled PowerShell script to run/report every N minutes.